Separate control and data logically, move WireGuard into userspace, add a CLI

Corrects the architecture on two points raised in review, while the project
is still small enough to change cheaply.

1. Control and data are separated *logically*, not physically.

The old reading — "nothing but control may ride on iroh" — threw away iroh's
whole value and would have forced the data plane to reimplement STUN, ICE and
a relay. Now both planes ride on iroh with different ALPNs and different
connections, so the data plane inherits hole punching and relay fallback,
while proto/ still knows nothing about packets and dataplane/ knows nothing
about the control protocol.

New boundary: PacketTransport / PacketLink, an authenticated unreliable
datagram channel per (network, peer, protocol). tsunagi/data/1 runs the same
membership handshake, then DataOpen/DataOpenAck, then QUIC datagrams. Only
the smaller endpoint id dials, so exactly one link exists per pair.

A plugin is handed links and never learns reachability, so the WireGuard
announcement shrank to a public key: there is no address left to lie about.

2. WireGuard now runs in userspace, on boringtun's protocol state machine.

No kernel module, no wg tool, no ip shell-out, no loopback proxy: the wgtool,
backend and bridge modules are gone. Only creating a TUN device needs
privileges, and that sits behind TunFactory, so the entire data plane —
handshake, encryption, routing, address ownership — is tested with none.

Address ownership is enforced rather than believed: outbound packets go to
the owner of the destination address, inbound packets are dropped unless
their source is the address derived for the peer that sent them.

3. A `tsunagi` binary: secret, doctor, id, up. It owns the runtime, the
logging subscriber and Ctrl-C, which the library still refuses to.

Also fixes a reference cycle where IrohTransport held Arc<Inner>, which kept
the databases open and the directory lock held after shutdown; two storage
tests caught it once the cycle existed.

81 tests pass offline with no privileges, including real IPv6 packets
crossing a real WireGuard tunnel over real iroh connections. Verified by
hand: two CLI processes forming a mesh both on loopback and via n0 discovery
using only an endpoint id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
2026-09-21 11:55:20 +01:00
co-authored by Claude Opus 5
parent ea7aaa2b69
commit 21be7e9b44
35 changed files with 3987 additions and 2664 deletions
+30 -12
View File
@@ -12,12 +12,21 @@ authentication, participant announcements, capability exchange and — later —
state synchronisation and delivery of IP-plugin data.
**Data plane.** Separate plugins create IP connectivity. WireGuard is the first
one and is implemented — see [wireguard.md](wireguard.md). Plugin keys,
configuration and lifecycle are separate from iroh identity and from the
one and is implemented in userspace — see [wireguard.md](wireguard.md). Plugin
keys, configuration and lifecycle are separate from iroh identity and from the
network secret. The core moves an opaque, bounded payload and never parses it.
Only control messages travel over iroh. User IP traffic is not tunnelled
through it; WireGuard packets travel over WireGuard's own UDP sockets.
**The transport in between.** Plugins do not open connections. They are handed
a `PacketLink` — an authenticated, unreliable datagram channel to one peer for
one protocol — and never learn how it is carried.
The separation between the two planes is **logical, not physical**. Both ride
on iroh, on different ALPNs and different connections. That is deliberate:
iroh's whole value is hole punching a direct path between peers behind NAT,
with a relay as fallback, and a data plane that refused to use it would have to
reimplement all of it. What the separation buys is that `proto` knows nothing
about packets and `dataplane` knows nothing about the control protocol, so
either can be replaced on its own.
A plugin talks to the core through three narrow hooks — `on_network_activated`,
a `PluginContext` for re-announcements and error reports, and a bounded
@@ -36,27 +45,36 @@ and the agent stays manageable.
| `proto` | message format, handshake, membership proof, protocol limits |
| `agent` | agent and per-network lifecycle, reconnect, in-process message routing |
| `storage` | mandatory state and the separately recoverable cache |
| `dataplane::transport` | authenticated datagram links to peers; where reachability lives |
| `dataplane` | the contract IP plugins implement, plus the WireGuard plugin |
Abstractions exist only where something is really substituted or really needs
isolating for tests: `NetworkDiscovery`, `IpPlugin`, and `WireguardBackend`
(which is what lets the plugin be tested in full without root). Everything else
is a concrete type.
isolating for tests: `NetworkDiscovery`, `IpPlugin`, `PacketTransport` /
`PacketLink` (so the data plane's carrier can change), and `TunFactory` (which
is what lets the whole data plane be tested without privileges). Everything
else is a concrete type.
## Runtime shape
```text
Agent one persistent identity, one iroh endpoint,
├── EndpointAdapter one state directory, N networks
├── EndpointAdapter (two ALPNs) one state directory, N networks
├── Storage (state.sqlite + cache.sqlite + ownership lock)
├── accept loop task ── weak ref, exits when the agent is dropped
├── IrohTransport ── data plane links, weak ref back to the agent
├── accept loop task ── routes by ALPN; weak ref, exits when the agent drops
├── plugin request loop ── re-announcements and plugin error reports
└── NetworkRuntime per NetworkId
├── discovery + dial loop (bounded concurrency, backoff with jitter)
── Session per peer
├── reader task ── frames in -> SessionEvent
└── writer task ── encoded frames out
── Session per peer (control)
├── reader task ── frames in -> SessionEvent
└── writer task ── encoded frames out
└── PacketLink per (peer, plugin protocol), handed to the plugin
```
Every strong reference from a background task back to the agent is a `Weak`.
A cycle there would keep the databases open and the directory lock held
forever after shutdown.
The library starts no runtime, installs no logging subscriber, handles no
signals, never forks and never calls `process::exit`. Startup
(`Agent::spawn`) and shutdown (`Agent::shutdown`) are explicit, background