Manage the overlay interface instead of asking for it

The agent printed a list of `ip` commands and asked a human to run them.
That is fragile in the way hand-held setup always is: a persistent TUN
does not survive a reboot, a changed address allocation needs another
manual round, and a run that died leaves a half-configured interface the
next run trips over.

On Linux the agent now creates the interface, sets the MTU, brings it up
and assigns both overlay addresses itself, over netlink in process. No
`ip` is invoked, so nothing this path does can be influenced by PATH, a
shell, or anything a remote peer said.

Cleanup stops being an action. The interface is tied to an open file
descriptor and is deliberately not persistent, so the kernel removes it
when the agent goes — cleanly, by panic, by SIGKILL or by power loss
alike. That also retires `keep_addr_on_down` and `nodad`, which existed
only because an interface nobody held open lost carrier.

Anything still left behind is repaired rather than tripped over: an
abandoned TUN is replaced along with its stale addresses. Two cases
refuse instead of guessing — a link that is not a TUN, because a name
collision is no reason to destroy somebody's bridge, and a TUN another
process holds open, because that is a working overlay belonging to
someone else.

CAP_NET_ADMIN is kept out of the effective set except around the calls
that use it. Two facts shape how: capabilities are per thread, and
netlink checks the credentials of whichever thread calls sendmsg, which
with an async client is the connection task rather than the caller. So
netlink runs on one dedicated thread with a current-thread runtime where
nothing is polled outside a block_on, and opening the TUN descriptor is
synchronous with no await between the guard and its release.

The decision of what to change is a pure function, tested on every
platform; only the execution is behind the provisioner trait. macOS and
Windows get an implementation that refuses with an explanation and falls
back to attaching to a prepared interface, plus a mock host the tests
drive the whole plugin lifecycle against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
2026-09-21 14:33:19 +01:00
co-authored by Claude Opus 5
parent 944d98389f
commit b23e832a73
15 changed files with 2598 additions and 150 deletions
+80 -17
View File
@@ -267,22 +267,85 @@ async fn main() -> Result<()> {
* **Userspace costs CPU.** Kernel WireGuard is faster. A kernel backend could
return behind the same boundary, but it would give up transport-provided NAT
traversal unless paired with a local proxy.
* **A persistent TUN interface needs `keep_addr_on_down`.** Without a process
attached it has no carrier, and Linux then flushes its IPv6 addresses. The
setup printed by `tsunagi tun-setup` sets it; the agent checks the address is
present *and usable* — not tentative, not DAD-failed — before attaching, and
reports what it actually found. This applies to IPv6 only: IPv4 addresses
survive carrier loss, so the overlay IPv4 address is added once and stays.
* **The overlay IPv4 address is not derivable, so `tun-setup` cannot print it
on a fresh state directory.** It is allocated and signed at run time, so the
first `tsunagi up` is what names it; afterwards `tun-setup` reads it back
from `state.sqlite` — a read that takes no directory lock and so does not
disturb a running agent — and includes the `ip address add` line.
* **The agent cannot assign the overlay address itself.** The `tun` crate sets
addresses through an IPv4-only ioctl, so the IPv6 overlay address must come
from `ip -6 address add` or an equivalent. The agent verifies the address is
present, via `/proc/net/if_inet6`, and refuses with the exact command rather
than running an interface that could never receive anything. Doing it
in-process would mean speaking netlink, which is not implemented.
* **The interface is managed, not prepared.** On Linux the agent creates the
TUN interface and configures it over netlink, in process. See
*Provisioning* below.
* **No provisioner exists for macOS or Windows yet.** There the agent attaches
to an interface prepared by hand and says so.
* **The system interface path is not exercised by the default suite**, because
it needs privileges. Everything else about the data plane is.
## Provisioning the interface
Everything the printed `ip` recipe used to do happens in process now, over
netlink. The shape of it is a reconciliation rather than a sequence of
commands: the agent is handed a plan — name, MTU, the addresses the interface
should carry and no others — observes what is actually on the host, and
applies the difference. Running it twice changes nothing the second time.
The decision of *what* to change is
`dataplane::wireguard::provision::plan_changes`: pure, platform-independent
and unit-tested on every platform. Only the execution is behind
`InterfaceProvisioner`, which has three implementations — netlink on Linux, a
`MockProvisioner` over a pretend host for the tests, and one that refuses with
an explanation everywhere else.
### Cleanup is the default, not an action
The interface is created by opening `/dev/net/tun` and is **not** made
persistent, so the kernel destroys it when the last descriptor closes. A clean
shutdown, a panic, `SIGKILL` and a power cut all leave the same amount behind:
nothing. There is no path by which a dead agent leaves an interface, because
keeping one alive is what requires a live process.
This also removes the two settings the manual recipe needed. `keep_addr_on_down`
existed only because an interface nobody held open lost carrier and had its
IPv6 addresses flushed; `nodad` only because duplicate address detection
cannot finish without carrier. An interface held open for its whole life has
carrier for its whole life.
### Repairing what an older run left
Two things can still be sitting on the name: an interface created persistent
by the old recipe, and — narrowly — one from a run killed between `TUNSETIFF`
and the agent recording it. Both are replaced, which discards their stale
addresses with them.
The two refusals are the interesting part:
* **A link that is not a TUN is never touched.** The name is derived from the
network id, so colliding with a real device is unlikely rather than
impossible, and deleting somebody's bridge is not a recoverable mistake.
* **A TUN another process holds open is never deleted.** Carrier is the
signal: a TUN has it exactly while something is attached. An attached one is
a working overlay, almost certainly a second agent on this host, and it is
told to use a different `--wg-prefix` instead.
### Privilege
`CAP_NET_ADMIN` is required and is kept out of the *effective* set except
around the netlink calls that need it. `setcap cap_net_admin+p` leaves it
permitted but not effective at exec, which is the resting state; the agent
raises it for a few milliseconds at startup and again when its address
allocation changes.
Two facts shape how that is done. Capabilities on Linux are **per thread**,
and netlink checks the credentials of whichever thread calls `sendmsg` —
which, with an async client, is the connection task and not the caller. So
raising a capability around an `await` would be wrong in the way that works
until the scheduler moves the task.
Therefore: all netlink work runs on one dedicated thread with a current-thread
runtime, where nothing is polled outside a `block_on`, and the capability is
raised immediately before that call and lowered immediately after. Opening the
TUN descriptor is the other privileged act; it is a synchronous call with no
`await` between the guard and the release, so it stays on its own thread by
construction.
### What it cannot be told to do
Nothing here takes a name, an address or a command from the network. The
interface name is derived from the network id, the addresses come from the
local plugin and the signed allocation records, and no external program is
executed at any point — there is no `ip`, no shell and no `PATH` involved.