Aleksandr Bogomiakov
d941cdac7a
Protocol selection previously belonged to tsng up and therefore to the service command line. A blocked or degraded carrier could only be removed by editing the service and restarting the whole agent. Make the ordered selection a local per-network setting instead. Add Agent::set_protocols and a local control request shared by CLI and tray. Keep tsng protocols without arguments as the protocol catalog; naming a network shows its selection and supplying a comma-separated list replaces it. Accept exact names, full ids and unique prefixes using shared selector rules; ambiguous selectors and invalid lists change nothing. An explicit none selects control-only operation. Offline edits take the directory lock and apply on the next start; live edits never write behind the daemon. Register every built-in plugin and transport in the CLI. Treat up --protocol as ordered initial defaults for networks without an override, allowing a protocol absent from the unit to be enabled later. Retain the targeting of unqualified startup options and allow explicitly scoped options to configure initially disabled protocols. Persist NULL for inherited preferences and an empty selection distinctly in SQLite schema 8; migration preserves existing identities, secrets, signed records and counters. Serialize changes with network lifecycle operations and commit preferences before applying or announcing them. Retire only the affected network's data links and E2E channels, cancel pending opens, and reject obsolete completion results by attempt id. Clear peer data channels when authenticated protocol capabilities change while retaining control sessions and topology. Replay allocations and peer identities after asynchronously prepared plugins become ready. A control-only start defers creation of the agent's single interface until data is enabled; plugins receive a sink that can use it later. Automatic carrier fallback continues to preserve E2E state; explicit policy changes renegotiate it. Identity derivation and peer wire formats are unchanged. Add a collapsible protocols row to each tray network tile, with a catalog provided by the daemon, descriptions, immediately applied checkboxes and preference-order buttons. Keep editing disabled until the worker refreshes status so a second click cannot overwrite an edit using a stale snapshot. Local IPC wire version is 20 for the new request and report fields; upgrade and restart agent, CLI and tray together. Crate release versions remain unchanged; the requested RC tag is v0.1.0-rc.30. Validation on Linux through nix develop: - cargo fmt --all -- --check - cargo clippy --locked --workspace --all-targets -- -D warnings - cargo test --locked --workspace --all-targets: 509 passed, 2 existing ignored tests, with inherited TSUNAGI_CONTROL_SOCKET and TSUNAGI_CONTROL_GROUP removed Tests use real iroh membership, SQLite, UDP, AWG and WireGuard encryption, QUIC carriers and memory TUNs. Cover either endpoint-id role, full-size packet delivery after switching, an unaffected second active network, saved selection after restart, empty/invalid selections, first enablement without restart, rapid changes racing old opens, live and offline CLI changes, startup-absent protocols and preservation of signed state through migration. Earlier compile and focused-test iterations caught initializer mistakes, a short test secret, asynchronous plugin allocation readiness, and an inherited system socket path; corrected before the successful final checks. Plain cargo was unavailable in the host shell, so validation used the Nix toolchain. Manual tray interaction, macOS/Windows, privileged host TUNs and public-network traversal were not run.
132 lines
7.3 KiB
Markdown
132 lines
7.3 KiB
Markdown
# Pairwise Amnezia transport
|
||
|
||
The CLI's default carrier preference is `wg,awg,wg-quic,tcp-tls`. To select
|
||
AWG explicitly:
|
||
|
||
```sh
|
||
tsng up --protocol awg
|
||
tsng up --protocol wg,awg,wg-quic,tcp-tls
|
||
```
|
||
|
||
The startup list is the initial per-network preference. A saved network choice
|
||
overrides it and can enable any built-in protocol at runtime:
|
||
|
||
```sh
|
||
tsng protocols home awg,wg-quic
|
||
tsng protocols home awg
|
||
```
|
||
|
||
Changes are local to this device, persist across restarts, and retire existing
|
||
data links and end-to-end channels for that network before renegotiation.
|
||
Authenticated control sessions, membership, addresses and other networks stay
|
||
live. Removing every protocol (`none` in the CLI, all checkboxes off in the tray)
|
||
leaves the network control-only. This is different from automatic carrier
|
||
fallback, which preserves the selected end-to-end channel.
|
||
|
||
`awg` is Tsunagi AWG wire version 1. Its packet masking engine is adapted from
|
||
[RunaRoo/AmneziaWG-RS](https://github.com/RunaRoo/AmneziaWG-RS), with the pinned
|
||
source and license recorded in
|
||
[UPSTREAM.md](../crates/tsunagi-awg/UPSTREAM.md). It uses the existing boringtun
|
||
Noise/WireGuard implementation instead of upstream's Node, global executors
|
||
and OS interface machinery. Compatibility with ordinary Amnezia servers is
|
||
not implemented: this carrier connects authenticated Tsunagi members.
|
||
|
||
## Negotiation and lifetime
|
||
|
||
There is one independent UDP socket and cancellable packet-engine task per
|
||
`(network, peer)` pair. The smaller endpoint id initiates, following the same
|
||
rule as other data carriers. Both sides must announce enabled AWG wire version
|
||
1. An authenticated iroh control session is required throughout; AWG supplies
|
||
no alternate discovery or membership mechanism.
|
||
|
||
Every `open` generates new masking parameters, a header-protection key, an
|
||
outer WireGuard key pair and a PSK. The initiator sends a private `Offer` with
|
||
a random attempt id, its public key, candidate socket addresses and the shared
|
||
profile. The responder validates it, creates its own fresh key pair and socket,
|
||
and returns an `Answer` naming that attempt. Both send masked handshakes and
|
||
encrypted delivery probes to the candidates. Only authenticated WireGuard
|
||
**data**, not a STUN response or an incoming UDP address alone, verifies a path.
|
||
|
||
Negotiation payloads are bounded at 4096 bytes (a caller can lower the limit),
|
||
redacted in Debug, excluded from public message events and erased on drop.
|
||
They are never capabilities, signed records, cache entries or exported status.
|
||
At most 128 pairs are admitted per transport, with bounded command/data queues,
|
||
at most 32 candidates per attempt and finite establishment deadlines.
|
||
Stale answers and duplicate offers are rejected; a new attempt replaces and
|
||
closes an older pair engine. Cancellation, peer departure, network deactivation
|
||
and shutdown retire sockets without waiting for a remote reader.
|
||
|
||
A process restart, a failed setup, or a complete fallback cycle returning to AWG
|
||
starts fresh negotiation. Normal WireGuard rekeying within a live attempt keeps
|
||
that attempt's masking profile. A failed pair does not reconfigure other pairs
|
||
or other networks. A full mesh of 15 members uses 14 engines per member and 105
|
||
independent pairs in total; it does not create additional system interfaces.
|
||
|
||
## Masking
|
||
|
||
Each profile randomizes disjoint H1–H4 header ranges, a 12–48-byte S1–S4 prefix,
|
||
a ChaCha20 header-protection key, handshake trailers, junk packet count/lengths,
|
||
and extra content padding. Prefix lengths are equal within one profile so
|
||
header detection has one offset. A fresh random nonce is in every packet's
|
||
prefix; handshake headers and transport headers/counters are protected.
|
||
The standard WireGuard header is restored before boringtun authenticates a
|
||
message. This preserves boringtun's standard MAC calculations, which is another
|
||
reason this is a Tsunagi-specific AWG wire version.
|
||
|
||
Before a handshake, an I1 signature packet is generated from either a complete
|
||
DNS query or SIP OPTIONS template, with fresh identifiers and consistent length
|
||
fields, followed by random junk packets. I2–I5 are unused in this version.
|
||
The templates are structural generators, not replayed captures. QUIC imitation
|
||
is not selected. Masking can change DPI classification; it is not a promise of
|
||
indistinguishability or that any particular firewall will admit it.
|
||
|
||
## Packets, NAT and fallback
|
||
|
||
The pair engine carries opaque relay envelopes inside authenticated outer WG
|
||
packets. It fragments those envelopes into bounded chunks and expires incomplete
|
||
assemblies; a logical datagram is bounded at 64 KiB and an outer UDP packet is
|
||
at most 1200 bytes. The encapsulated IPv4 headers are internal framing only: no interface or OS address is created for a
|
||
pair. Overlay addresses, routing, ownership checks and end-to-end encryption
|
||
remain independent. A relay decrypts its own adjacent AWG carrier to forward
|
||
opaque end-to-end ciphertext, never the destination's IP packet.
|
||
|
||
Local IPv4 candidates and simultaneous encrypted probes provide direct UDP
|
||
connectivity. In `direct`/`relay` reach mode the CLI also uses
|
||
`stun.l.google.com:19302` to discover a server-reflexive address on **this pair's
|
||
socket**, with a 1.2-second bound. A STUN result is only a candidate. Local reach
|
||
mode makes no STUN request. Configure or disable it with
|
||
`-o awg:stun=HOST:PORT` or `-o awg:stun=off`. Failed discovery still permits local
|
||
candidates. IPv6-only paths, symmetric NAT and networks blocking arbitrary UDP
|
||
may fail AWG and fall back; there is no AWG-over-QUIC substitution or AWG TURN
|
||
relay hidden behind its name.
|
||
|
||
Data delivery checks go through the actual end-to-end encrypted tunnel, once
|
||
per second, independently of control-plane RTT. Requests and replies bind a
|
||
fresh private marker/key to the sender's endpoint id. Unanswered requests expire
|
||
after three seconds. After a 20-second warmup, at least 15 completed observations
|
||
and at least 60% loss in the last 20 observations retire a carrier. Probes are
|
||
consumed locally after source/destination admission and never enter the TUN.
|
||
AWG additionally checks encrypted outer delivery throughout the pair lifetime.
|
||
|
||
Setup failure or persistent runtime loss advances to the next mutually supported
|
||
carrier in the initiator's order. Earlier expired cooldowns do not jump ahead of
|
||
untried carriers. Retry cooldowns remain bounded; reaching AWG again negotiates
|
||
new parameters. Control sessions and the logical end-to-end tunnel survive a
|
||
carrier change. Status names the end-to-end protocol and the actual carrier
|
||
separately, so `wg` over `awg` is a valid result.
|
||
|
||
Control wire version 6 introduces the private transport message. All members
|
||
must upgrade together; device identities, network ids, handshake transcript
|
||
encoding, signed state and stored network configuration are unchanged. Data
|
||
ALPN remains `tsunagi/data/5`; the relay envelope is unchanged.
|
||
|
||
## Verification
|
||
|
||
The default rootless, offline tests use real iroh endpoints, membership proofs,
|
||
SQLite, UDP sockets, masked WireGuard handshakes, end-to-end encryption and
|
||
memory TUNs. They cover independent three-member pairs, full-size bidirectional
|
||
packets, restart freshness/socket release, 80% runtime datagram loss, fallback
|
||
to real iroh and fresh negotiation on retry, plus malformed profiles/messages,
|
||
role checks, probe forgery and STUN response parsing. These tests do not establish
|
||
DPI effectiveness, WAN throughput, public NAT traversal or per-pair memory cost.
|