The setup recipe failed with a missing sysctl directory and "RTNETLINK
answers: Invalid argument". The cause was the default MTU of 1100.
IPv6 requires a minimum MTU of 1280 (RFC 8200) and Linux enforces it by
tearing IPv6 down on any interface below it: the per-device
/proc/sys/net/ipv6/conf entries disappear and an address can no longer be
assigned. Evidence on the test host: every interface at 1280 or above has
an IPv6 conf directory, every interface below it (1230, 1100) has none.
So the overlay MTU is now 1280, which is also the floor. A smaller value is
refused when the plugin opens, naming the reason, rather than surfacing as
an obscure netlink error after the user has already run four commands.
That leaves no slack against the other constraint: a packet needs mtu + 32
bytes of transport datagram, so 1312. A direct QUIC path offers roughly
1380 and fits; a relayed path may not, so the plugin now reports the exact
numbers when a link cannot carry a full-size packet, instead of only
counting silent drops.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Running `tsunagi up` twice with the same arguments failed with "network ...
is already active", and then dropped the iroh endpoint without closing it.
A configured network is activated automatically at startup, so the second
run found it already up. `join_network` is declarative — "be a member of
this network" — so joining one that is already active now succeeds and
changes nothing. `activate_network` stays strict for callers that
specifically want to know whether an inactive network was started.
The CLI now closes the agent on the error path too, and handles SIGTERM as
well as Ctrl-C, so a service manager stopping the agent gets the same clean
shutdown an interactive user does.
Also documents the two lookups people conflate: resolving one endpoint's
address is iroh's public pkarr/DNS service and works today, which is why
`--peer <endpoint-id>` needs no address; finding who is in a network is this
project's `NetworkDiscovery` and is still static bootstrap only. Notes in
the README and the threat model that `n0` and `direct` publish this
endpoint's addresses to a public third-party service.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Corrects the architecture on two points raised in review, while the project
is still small enough to change cheaply.
1. Control and data are separated *logically*, not physically.
The old reading — "nothing but control may ride on iroh" — threw away iroh's
whole value and would have forced the data plane to reimplement STUN, ICE and
a relay. Now both planes ride on iroh with different ALPNs and different
connections, so the data plane inherits hole punching and relay fallback,
while proto/ still knows nothing about packets and dataplane/ knows nothing
about the control protocol.
New boundary: PacketTransport / PacketLink, an authenticated unreliable
datagram channel per (network, peer, protocol). tsunagi/data/1 runs the same
membership handshake, then DataOpen/DataOpenAck, then QUIC datagrams. Only
the smaller endpoint id dials, so exactly one link exists per pair.
A plugin is handed links and never learns reachability, so the WireGuard
announcement shrank to a public key: there is no address left to lie about.
2. WireGuard now runs in userspace, on boringtun's protocol state machine.
No kernel module, no wg tool, no ip shell-out, no loopback proxy: the wgtool,
backend and bridge modules are gone. Only creating a TUN device needs
privileges, and that sits behind TunFactory, so the entire data plane —
handshake, encryption, routing, address ownership — is tested with none.
Address ownership is enforced rather than believed: outbound packets go to
the owner of the destination address, inbound packets are dropped unless
their source is the address derived for the peer that sent them.
3. A `tsunagi` binary: secret, doctor, id, up. It owns the runtime, the
logging subscriber and Ctrl-C, which the library still refuses to.
Also fixes a reference cycle where IrohTransport held Arc<Inner>, which kept
the databases open and the directory lock held after shutdown; two storage
tests caught it once the cycle existed.
81 tests pass offline with no privileges, including real IPv6 packets
crossing a real WireGuard tunnel over real iroh connections. Verified by
hand: two CLI processes forming a mesh both on loopback and via n0 discovery
using only an endpoint id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first IP plugin, built on the data plane boundary the core already had.
Plugin:
- one X25519 key per network in the plugin's own wireguard.sqlite, separate
from the iroh identity and from the network secret; a damaged store is an
error, never a silently regenerated identity
- deterministic IPv6 ULA overlay: every member derives the same /64 from the
network id and its own /128 from its WireGuard public key, so no
coordinator allocates addresses
- AllowedIPs are derived locally, never taken from a peer's announcement, so
a member cannot claim another member's overlay address; a mismatched claim
is rejected
- bounded, versioned, validated announcement carried as the existing opaque
capability payload, which the core still never parses
- each agent builds its own full-mesh configuration (N-1 peers) and
reconciles on every change and on a timer, repairing drift
- WireguardBackend abstraction: RecordingBackend in memory, and WgToolBackend
driving real wg/ip on Linux, split into a pure planner plus parsers and a
thin executor so everything interesting is testable without root
Core, three generic additions the plugin needed:
- IpPlugin::on_network_activated, so per-network state is ready before peers
- PluginContext for re-announcements and error reports from plugin tasks,
with errors counted by the owning network runtime
- IpPlugin::shutdown, awaited with a grace period, so system objects go away
94 tests pass offline with no privileges: 35 new WireGuard unit tests and 12
integration tests over real iroh connections. The real wg/ip backend needs
root and is behind --ignored in tests/wireguard_system.rs; it was not run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Working library with real iroh connections, not an interface sketch:
- persistent device identity in state.sqlite, stable across restarts
- deterministic network space derived from name + secret via HKDF-SHA256,
with frozen labels and unambiguous length-prefixed encoding
- replaceable discovery returning unverified candidates only; static
bootstrap, in-memory test backend and a composite
- real iroh connections plus an explicit mutual membership proof:
HMAC-SHA256 over a role-separated transcript bound to the TLS exporter,
the network id and both endpoint identities
- small versioned control protocol: handshake, announcement, ping/pong
- multiple networks per agent with enforced isolation
- automatic reconnect with bounded backoff and jitter
- mandatory state vs disposable cache, with a real directory ownership lock
- status snapshots, event stream and honest diagnostics
47 integration and unit tests cover the required scenarios offline on
loopback. Snapshots, revocations and WireGuard are designed for and
documented, not implemented.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tsunagiandClaude Opus 5