tsunagiandClaude Opus 5
21be7e9b44
Corrects the architecture on two points raised in review, while the project is still small enough to change cheaply. 1. Control and data are separated *logically*, not physically. The old reading — "nothing but control may ride on iroh" — threw away iroh's whole value and would have forced the data plane to reimplement STUN, ICE and a relay. Now both planes ride on iroh with different ALPNs and different connections, so the data plane inherits hole punching and relay fallback, while proto/ still knows nothing about packets and dataplane/ knows nothing about the control protocol. New boundary: PacketTransport / PacketLink, an authenticated unreliable datagram channel per (network, peer, protocol). tsunagi/data/1 runs the same membership handshake, then DataOpen/DataOpenAck, then QUIC datagrams. Only the smaller endpoint id dials, so exactly one link exists per pair. A plugin is handed links and never learns reachability, so the WireGuard announcement shrank to a public key: there is no address left to lie about. 2. WireGuard now runs in userspace, on boringtun's protocol state machine. No kernel module, no wg tool, no ip shell-out, no loopback proxy: the wgtool, backend and bridge modules are gone. Only creating a TUN device needs privileges, and that sits behind TunFactory, so the entire data plane — handshake, encryption, routing, address ownership — is tested with none. Address ownership is enforced rather than believed: outbound packets go to the owner of the destination address, inbound packets are dropped unless their source is the address derived for the peer that sent them. 3. A `tsunagi` binary: secret, doctor, id, up. It owns the runtime, the logging subscriber and Ctrl-C, which the library still refuses to. Also fixes a reference cycle where IrohTransport held Arc<Inner>, which kept the databases open and the directory lock held after shutdown; two storage tests caught it once the cycle existed. 81 tests pass offline with no privileges, including real IPv6 packets crossing a real WireGuard tunnel over real iroh connections. Verified by hand: two CLI processes forming a mesh both on loopback and via n0 discovery using only an endpoint id. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3.9 KiB
Planned state synchronisation
Nothing in this document is implemented. The proof of concept exchanges hostname and capability announcements over live sessions and keeps no replicated history. That is also why WireGuard peer membership is session-scoped today: a peer leaves the overlay when its control session ends, because there is no agreed durable state to keep it. This file records the intended direction so the module boundaries in architecture.md stay compatible with it, and so nobody mistakes the current announcements for synchronisation.
There is no fake "ready CRDT" here and no snapshots that are not actually verified.
The problem
The network may be unstable. A participant can come back after months. So:
- no dependence on the author of a change being online;
- no dependence on acknowledgements from every participant ever seen;
- any available replica holding the signed data must be able to hand it to a returning participant without the original author present.
The model
Signed self-contained state per author, merged between replicas.
A record contains: the network, the author, the author's own retained version, the full current content, and a signature. A change log may speed delivery up, but recovery must never require the entire chain from the first event.
A network snapshot is a set of verifiable authored records plus the revocations needed to interpret them. It is not a SQLite dump, and not a single document trusted merely because the neighbour who forwarded it signed it.
Merge rules
- A snapshot is merged into local state, never substituted for it wholesale.
- An older version never rolls back a newer known one.
- Absence from a snapshot does not mean deletion.
- Duplicates do not change the result.
- Two conflicting signed records at the same version from the same author need explicit handling; they are not resolved by luck.
- Neither arrival order nor system clocks decide a winner.
- Compaction must not drop what is needed to stop revoked records being resurrected.
Hostnames
A hostname is a mutable binding to a persistent author, not an identity. A rename must be a signed record that revokes the specific old binding and announces the new one, ideally atomically in one record.
Turning a computer off is not a revocation of its hostname and does not remove the participant. Revocations are not dropped merely because they are old, and no acknowledgement from offline peers is required to keep working.
Storage requirement this creates
When signed records land, writing the event and bumping the author's own
counter must happen in one SQLite transaction, committed before the change is
published to the network. SQLite gives atomic commit; use it instead of
separate, inconsistent writes. state.sqlite already has a schema version and
migrations for this.
Limits to state honestly
- Data that every copy has lost is not recoverable from the secret.
- A signature proves authorship, not global freshness: a replica can be behind, and you cannot tell from the signature alone.
- An isolated new client can end up with incomplete state and has no way to know what it is missing.
- Anyone who knows the secret can author records, so a majority of records is not evidence of anything.
Future tests
These are not implemented and must not be reported as passing:
- snapshot merge against the rules above, including conflicting same-version records;
- revocation propagation and resistance to resurrection after compaction;
- long network partitions and rejoin after an extended absence;
- hostname rename with atomic revoke-and-announce;
- recovery of a returning participant from a replica that is not the author;
- NAT traversal and hole punching between real hosts;
- relay fallback behaviour against a self-hosted relay;
- multi-process and multi-host deployment, as opposed to several library instances inside one test process.