Skip to content

feat(netgw): Divert bound destinations to a socket - #131

Open
ananos wants to merge 2 commits into
chore/bump-hvi-vmm-7766307from
feat/netgw-divert
Open

ananos wants to merge 2 commits into
chore/bump-hvi-vmm-7766307from
feat/netgw-divert

Conversation

@ananos

@ananos ananos commented Oct 23, 2026

Copy link
Copy Markdown
Member

Summary

brig's credential broker serves each sandbox at brig.internal. On hull, the sandbox reaches it through its network gateway. The gateway dials every allowed destination itself, so there was no way to hand a guest's connection to a local process.

This PR adds --divert host:port=socket to hull network-gateway. A guest TCP connection to a diverted destination goes to a Unix socket: the gateway writes one preamble line and then relays the bytes both ways unchanged. It fails closed, so an unreachable socket or an owner that fails resets the guest, and a diverted connection is never dialed.

The second commit stops the gateway from dialing its own address. brig answers brig.internal with the gateway's address, so nothing other than the divert should answer there.

This is stacked on #130 (the hvi-vmm bump), so that the experimental channel carries both. The brig side is the broker PR stack in brig-sh/brig, whose last PR routes brig.internal to microVMs. brig checks for --divert and refuses a routed run on a hull without it.

Related issues

Refs brig-sh/brig#265, brig-sh/brig#302

Changes

  • --divert host:port=socket, repeatable. A named host matches every address the gateway resolved it to, and the set only grows. Diversion is checked before the egress policy.
  • Preamble BRIG-DIVERT/1 <dst-ip> <dst-port> <src-ip> <src-port>, then a plain relay with half-close. The source address is whatever the guest sent, so the docs say not to use it as identity: the socket path identifies the sandbox.
  • UDP to a diverted destination is dropped, so QUIC falls back to TCP.
  • GET /diverts on --api lists each divert with its counters.
  • TCP to the gateway's own address is reset after the diverts are matched, and UDP to it is dropped.
  • docs/networking.md documents the flag, the preamble and the fail-closed rules.

Checklist

  • make test passes
  • make macos builds hull, vz-runner and hvi, if Go, Swift or Rust code changed
  • I have added or updated tests covering the change
  • I have run the e2e harnesses (test/*.py) for changes touching boot, console, or the run path
  • I have updated the affected docs (README, docs/)

Negative tests cover each fail-closed claim: missing socket, refusal, an owner that fails partway through a reply, deny policy, an address not yet resolved, gateway ports 80/8443/22 and UDP. golangci-lint v2.12.2 is clean for darwin and linux, and go test -race ./internal/netgw/ passes. The e2e harnesses do not exercise the gateway. A run through brig with a routed policy is pending the brig stack.

brig's credential broker serves a sandbox at a name the sandbox reaches
through its gateway, so the gateway needs a way to hand a guest's TCP
connections to a destination to a local process. It dialed every
allowed destination itself, so there was no way to put that process in
the path.

--divert host:port=socket hands such connections to a Unix socket. The
gateway writes one preamble line,
"BRIG-DIVERT/1 <dst-ip> <dst-port> <src-ip> <src-port>", and then relays
the guest's bytes both ways unchanged, with half-close.

- A named host matches every address this gateway resolved it to: at
  start, every --egress-refresh, and on each guest query. The set only
  grows, so a rotated address is never dialed directly.
- Diversion is checked before the egress policy. The socket's owner
  applies its own rules. A diverted name is answered under deny-default,
  but not pinned, so other ports on its addresses stay under the policy.
- It fails closed. An unreachable socket, or an owner that closes
  without writing, resets the guest. A diverted connection is never
  dialed.
- UDP to a diverted destination is dropped, so QUIC falls back to TCP.
- GET /diverts on --api lists each divert with its counters.

Refs: brig-sh/brig#265
Signed-off-by: Anastassios Nanos <ananos@nofire.ai>
A guest connection to the gateway's address on a port no divert names
was dialed from the host. It then reached whatever answered that
address on the host's network.

brig now answers brig.internal with the gateway's address, and diverts
its port 443 to the broker. Nothing else should answer there.

- TCP to the gateway's address is reset, after the diverts are matched.
- UDP to it is dropped. Its DNS port is an endpoint on the stack, and
  the forwarders never see it.

The test starts a gateway the way brig does: a --host record of
brig.internal and a divert of port 443. It checks the record under
deny-default, the divert, and that ports 80, 8443 and 22 are reset and
never dialed, with and without a policy.

Refs: brig-sh/brig#302
Refs: brig-sh/brig#265
Signed-off-by: Anastassios Nanos <ananos@nofire.ai>
@ananos
ananos marked this pull request as ready for review October 23, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant