Separate control and data logically, move WireGuard into userspace, add a CLI

Corrects the architecture on two points raised in review, while the project
is still small enough to change cheaply.

1. Control and data are separated *logically*, not physically.

The old reading — "nothing but control may ride on iroh" — threw away iroh's
whole value and would have forced the data plane to reimplement STUN, ICE and
a relay. Now both planes ride on iroh with different ALPNs and different
connections, so the data plane inherits hole punching and relay fallback,
while proto/ still knows nothing about packets and dataplane/ knows nothing
about the control protocol.

New boundary: PacketTransport / PacketLink, an authenticated unreliable
datagram channel per (network, peer, protocol). tsunagi/data/1 runs the same
membership handshake, then DataOpen/DataOpenAck, then QUIC datagrams. Only
the smaller endpoint id dials, so exactly one link exists per pair.

A plugin is handed links and never learns reachability, so the WireGuard
announcement shrank to a public key: there is no address left to lie about.

2. WireGuard now runs in userspace, on boringtun's protocol state machine.

No kernel module, no wg tool, no ip shell-out, no loopback proxy: the wgtool,
backend and bridge modules are gone. Only creating a TUN device needs
privileges, and that sits behind TunFactory, so the entire data plane —
handshake, encryption, routing, address ownership — is tested with none.

Address ownership is enforced rather than believed: outbound packets go to
the owner of the destination address, inbound packets are dropped unless
their source is the address derived for the peer that sent them.

3. A `tsunagi` binary: secret, doctor, id, up. It owns the runtime, the
logging subscriber and Ctrl-C, which the library still refuses to.

Also fixes a reference cycle where IrohTransport held Arc<Inner>, which kept
the databases open and the directory lock held after shutdown; two storage
tests caught it once the cycle existed.

81 tests pass offline with no privileges, including real IPv6 packets
crossing a real WireGuard tunnel over real iroh connections. Verified by
hand: two CLI processes forming a mesh both on loopback and via n0 discovery
using only an endpoint id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
2026-09-21 11:55:20 +01:00
co-authored by Claude Opus 5
parent ea7aaa2b69
commit 21be7e9b44
35 changed files with 3987 additions and 2664 deletions
+24 -14
View File
@@ -17,18 +17,27 @@ Design accordingly: a majority is not a root of trust.
Keep these separate. Crossing them is the main thing to review for.
- **Control plane vs data plane.** iroh carries control messages only. User IP
traffic is never tunnelled through it. The core must never parse a plugin's
payload — see `src/dataplane/mod.rs`. Only
- **Control plane vs data plane.** The separation is **logical, not physical**.
The control protocol in `src/proto/` knows nothing about packets, and
`src/dataplane/` knows nothing about the control protocol; either can be
replaced on its own. Both may ride on iroh — refusing to would throw away
iroh's NAT traversal and force the data plane to reimplement it. They use
different ALPNs and different connections, so a busy or broken data plane
cannot disturb control traffic.
- **Plugins never learn reachability.** An `IpPlugin` is handed a `PacketLink`
per peer and moves datagrams over it. Addresses, hole punching and relays
belong to `src/dataplane/transport/`. A plugin announcement says *who*, never
*where*.
- **The core never parses a plugin payload.** See `src/dataplane/mod.rs`. Only
`src/dataplane/wireguard/announcement.rs` interprets WireGuard payloads, and
only after bounding every field. A data plane failure must never stop the
control plane.
- **Derived, not claimed.** A WireGuard peer's `AllowedIPs` are always derived
locally from its public key. Never take them from what the peer announces, or
a member can route another member's traffic to itself.
- **Derived, not claimed.** A peer's overlay address is derived from its public
key. Outbound packets are routed to the owner of the destination address;
inbound packets are dropped unless their source is the address derived for
the peer that sent them. Never trust an address a peer announces.
- **Plugins own their system objects.** A plugin creates and removes its own
interface and nothing else. An interface that already exists and is not ours
is refused, never adopted. Never touch routing, DNS or firewall settings.
interface and nothing else. Never touch routing, DNS or firewall settings.
- **Device identity vs network identity.** The iroh endpoint id is the device's
public key. `NetworkId` is derived from name + secret only. Never conflate
them, and never let one change the other.
@@ -88,7 +97,8 @@ Keep these separate. Crossing them is the main thing to review for.
| `src/proto/` | framing, message formats, membership handshake |
| `src/net.rs` | iroh endpoint adapter and observability snapshots |
| `src/agent/` | agent lifecycle, per-network runtimes, sessions, events, status |
| `src/dataplane/` | the contract IP plugins implement, and the WireGuard plugin |
| `src/dataplane/` | the plugin contract, the packet transport, and the WireGuard plugin |
| `src/bin/tsunagi.rs`| the command line agent; the only place that owns a runtime, a logger and signals |
| `tests/` | integration tests; `tests/common/` is the shared harness |
Add abstractions only at real substitution or testing boundaries. Do not add a
@@ -106,11 +116,11 @@ trait per struct. Prefer one crate with clear modules over many small crates.
happen.
- The default suite must pass with no internet, no DHT, no public relay, no
administrator rights and no changes to OS network settings. Anything needing
the internet, or root, stays out of the default set — the real WireGuard
backend's tests live in `tests/wireguard_system.rs` behind `--ignored`.
- The WireGuard backend may be substituted (`RecordingBackend`). Its key
handling, announcements, derived addressing, configuration builder and
reconciliation may not.
the internet or root stays out of the default set.
- The WireGuard packet interface may be substituted (`MemoryTunFactory`). Its
key handling, announcements, derived addressing, the WireGuard protocol
itself and address-ownership enforcement may not — the default suite runs
real handshakes and real encryption.
- Running several library instances in one process is not a test of several
system processes; do not describe it as one.