tsunagi/docs/protocol.md

Protocol

Module boundaries are in architecture.md; the threat model is in threat-model.md.

Versioning

Two versions exist and are independent:

• Identity scheme, tsunagi-network-id-v1. Frozen. Changing it creates a different network space for the same name and secret.
• Control protocol, ALPN tsunagi/ctrl/8, PROTOCOL_VERSION = 8.

Upgrading the crate or bumping the control protocol must never change an existing NetworkId.

Control version 8 adds signed, multi-writer VIP field snapshots to version 7's node retirements. Data ALPN is tsunagi/data/6 for the single internal WireGuard channel. All members must upgrade together. Network ids, derivation labels, handshake encoding and existing record signatures are unchanged. Local CLI/tray IPC is version 23; restart the agent with matching clients. SQLite schema 10 adds vip_fields, preserving identity, network configuration, signed node records and retirement proofs. See virtual services, state sync and AWG.

Canonical encoding

Network identity derivation and the membership transcript use length-prefixed fields, written here as LP(x) = u32_be(len(x)) || x. String concatenation is never used, so different field splits cannot produce the same bytes. VIP field signatures use their separate domain and canonical postcard encoding described in virtual services.

Network space identity

salt = SHA-256( LP("tsunagi-network-id-v1") || LP(name_utf8) )
prk  = HKDF-SHA256-Extract(salt, ikm = secret_bytes)

info(label)   = LP("tsunagi-network-id-v1") || LP(label)
network_id    = HKDF-Expand(prk, info("network-id"),     32)
discovery_key = HKDF-Expand(prk, info("discovery-key"),  32)
auth_key      = HKDF-Expand(prk, info("handshake-auth"), 32)

HKDF's info parameter is what separates the three outputs (RFC 5869 §3.2). The labels and this encoding are frozen.

Name rules. 1..=64 bytes of UTF-8, no control characters, no leading or trailing ASCII whitespace — such a name is rejected, not trimmed. The name is used verbatim: no case folding, no Unicode normalisation. Home and home are different network spaces.

Secret rules. Used verbatim: never trimmed, case-folded, normalised or truncated. At least 16 bytes; NetworkSecret::generate() produces 32 random bytes. The canonical text form is tsn1 followed by lowercase unpadded base32.

Key separation. network_id is public. discovery_key is what a discovery backend is told; it is secret-derived but is not a credential — learning it does not help pass the handshake, and it must never be used as a password or bearer token. auth_key never leaves the process.

No competing genesis

A network's description is NetworkDescriptor { scheme, name, network_id }. It contains no creator identity, no creation time and no owner signature, so two agents started independently with the same parameters produce byte-identical descriptors. No minimum-hash rule, no vote, no leader. The secret is not part of it.

A network's name does not change inside an existing space. A different name or secret is a different space.

Framing

One QUIC bidirectional stream per session. A frame is u32_be(len) || payload. The announced length is checked against Limits::max_frame_len (64 KiB by default) before any buffer of that size is allocated. Payloads are postcard, a compact deterministic serde format — not a general RPC framework.

No encryption is layered on top of iroh: QUIC/TLS already provides confidentiality, integrity and endpoint authentication.

Handshake

A successful iroh connection proves only which endpoint is on the other side, because the endpoint id is the public key in the TLS certificate. Anyone can dial us. Membership of a specific network is proved separately.

iroh exposes the TLS exporter (RFC 5705) through Connection::export_keying_material. That yields the same secret bytes on both ends of this connection, which is what stops a proof being replayed elsewhere. It proves nothing about the shared secret on its own, because both ends of any connection can compute it. The membership proof is the HMAC keyed by auth_key; the exporter output is just one of its inputs.

cb = TLS-Exporter(label = "tsunagi/handshake/v1", context = network_id, 32)

transcript(role) = LP("tsunagi-handshake-v1")
                || LP(role)                    // "initiator-proof" | "responder-proof"
                || LP(u16_be(protocol_version))
                || LP(network_id)              // 32 bytes
                || LP(initiator_endpoint_id)   // 32 bytes
                || LP(responder_endpoint_id)   // 32 bytes
                || LP(cb)                      // 32 bytes
                || LP(nonce_initiator)         // 16 bytes
                || LP(nonce_responder)         // 16 bytes

proof(role) = HMAC-SHA256(auth_key, transcript(role))

What each input buys:

input	property
auth_key	membership in this network space
cb	binding to this connection; a captured proof is useless on another
network_id	binding to this network space
both endpoint ids	binding to these two identities
distinct role labels	no reflection: a proof cannot be bounced back
both nonces	freshness contributed by each side

Message order:

initiator -> responder : Hello     { version, network_id, nonce_i }
initiator <- responder : HelloAck  { version, nonce_r }
initiator -> responder : AuthProof { proof(initiator) }
initiator <- responder : AuthProof { proof(responder) }   // only after the first verified

The responder emits nothing derived from auth_key until the initiator's proof verifies, so a caller who does not know the secret learns nothing. Comparison is constant time. Endpoint ids always come from the TLS certificate, never from a peer's claim. Until both steps complete, no regular control message is accepted in either direction. The whole exchange is bounded by Limits::handshake_timeout.

A Hello naming a network this agent does not have active is rejected with "unknown network". Because the claim is unverified at that point, the rejection event does not report a network id.

The data plane protocol

IP plugin packets never travel on a control connection. They use their own ALPN, tsunagi/data/6, on their own iroh connection:

initiator -> responder : (the same membership handshake as above)
initiator -> responder : DataOpen    { protocol, max_datagram }
initiator <- responder : DataOpenAck { accepted, max_datagram }
thereafter             : QUIC datagrams carrying fragments of opaque payloads

The membership handshake is identical and bound to the same network, so a data channel cannot be opened by somebody who does not know the secret. protocol is bounded and must name a plugin the responder actually runs; otherwise the channel is declined, which is an ordinary outcome rather than an error.

Only one side dials — the one with the smaller endpoint id — so two agents never open two channels for the same thing.

Packets ride as QUIC datagrams: unreliable and unordered, which is what a tunnelled protocol wants, and free of the head-of-line blocking a stream would add. max_datagram negotiates the reassembled payload limit (at most 65536 bytes), independently of the current QUIC path MTU. The transport reads QUIC's current datagram capacity for every fragment, including after migration.

Every datagram starts with a 16-byte header: packet ID (u64), total payload length (u32), and byte offset (u32), all big endian. The payload is the complete peer-relay envelope containing opaque plugin bytes. Relays reassemble before forwarding and fragment again for their outgoing path; they never decrypt the plugin's payload. Small packets use the same framing in one part.

Reassembly tolerates reordering and exact duplicates. Conflicting lengths, overlapping data and excessive fragment counts discard that packet. Each link allows at most 64 incomplete packets, 256 KiB of their payload buffers, and 128 fragments per packet. Incomplete packets expire after five seconds, without holding up subsequent packets. Packet IDs are scoped to a QUIC connection; reassembly is created only after the membership handshake. All limits live in config.rs. There is no transport-layer retransmission added by this framing.

Version 4 added a routing envelope inside the transport fragmentation framing, and version 5 added the end-to-end protocol to it:

field	bytes	encoding
envelope version	1	2
remaining hops	1	initially 16; transit decrements, drops at 1
source peer	32	original endpoint public key
destination peer	32	final endpoint public key
flow id	8	big endian opaque stable hash supplied by the plugin
protocol	8	big endian FNV-1a of the internal protocol id (wireguard-e2e)
payload	remaining	end-to-end encrypted plugin bytes

The fixed header is 82 bytes. Zero/oversized hop limits, unknown versions, short frames, oversized datagrams and unknown sources are rejected. A frame addressed here is delivered to the inbox of its source and protocol; transit goes straight to its next transport, whatever kind of transport that is. The network is bound to the authenticated transport connection, not supplied by the frame. The protocol names which end-to-end tunnel the payload belongs to and is independent of the links it crosses: a datagram of one protocol can travel over a link opened for another, and a middle member needs no protocol in common with either end. Intermediate nodes cannot decrypt or authenticate the inner payload; the destination's tunnel does that, so a frame naming the wrong protocol or a forged source is simply discarded there. Flow ids are routing hints, not authorization proofs.

Reachable { links: [{ peer, protocol }] } lists the sender's direct links; protocol is the kind of transport the link is, for diagnostics, and routing ignores it. Each row belongs to the authenticated sender and is replaced atomically, expires after 90 seconds, and is withdrawn on session closure. Only authenticated members enter the graph; local edges always come from actual links. Announcements refresh on the existing control maintenance interval and are sent immediately on link changes. Topology stays volatile, outside signed state.

Version 5 also adds hostname_since to a signed hostname claim (signing domain tsunagi-signed-record-v3): a name belongs to whoever claimed it first. Stored records signed before are discarded on upgrade and re-published by their authors.

Upgrade both endpoints and intermediate peers together. Older control and data ALPNs cannot interoperate. The network secret, device identity, identity derivation, transcript encoding and stored network configuration are unchanged.

Separate connections mean separate congestion control, so a saturated data plane cannot delay control messages, and a data plane failure cannot take the control plane down with it.

Control messages

After authentication, every frame is an Envelope { network_id, message } and the network_id is re-checked against the session's network on every message. A mismatch ends that session and is counted as a protocol violation; it does not affect other networks.

message	meaning
Announce { hostname, capabilities, broadcast }	this agent's hostname, IP-plugin capabilities and local broadcast participation
Transport { protocol, payload }	private, bounded per-peer transport negotiation; never state or a public message event
Ping { seq, payload }	small request used to verify the exchange
Pong { seq, payload }	the echoed reply
State { records }	a snapshot of signed records, merged into what the receiver holds
Retirements { markers }	signed cleanup markers with the target's original signed claim as proof
Peers { peers }	unverified member address hints, requiring their own handshake
Reachable { links }	sender's current direct data links, by kind of transport
Bye { reason }	graceful goodbye; not a revocation of anything

broadcast is a per-network local opt-in, enabled by default and treated as false until an authenticated announcement arrives. It governs IP broadcast fanout and local admission; encrypted transit still uses the existing envelope. See broadcast.md for scope and persistence.

A State snapshot is merged, never substituted: an author missing from it is left untouched. Each record carries its own signature, so a peer forwarding somebody else's record cannot alter it, and a record that fails verification is dropped without affecting the rest of the batch. See sync-model.md.

PluginCapability { protocol, version, enabled, data } is opaque to the core: data is bounded and handed to the matching plugin unparsed. Nothing in it is ever treated as a shell command, filesystem path or OS setting.

Every decoded message is validated against Limits before it reaches anything else. A rejected message never stops a network or the agent.

Limits

Defaults from Limits, all configurable:

limit	default
frame payload	64 KiB
hostname	255 bytes
capabilities per announcement	16
capability payload	4 KiB
echo payload	4 KiB
reason string	256 bytes
handshake timeout	10 s
dial timeout	10 s
write timeout	30 s
concurrent dials per network	8
sessions per network	64
inbound handshakes in flight	32
outbound queue per session	64

Liveness of an established session is delegated to QUIC: iroh configures keep-alives and an idle timeout, so a dead peer surfaces as a read error rather than needing a heartbeat in this protocol.