Add ephemeral pairwise Amnezia transport and runtime fallback
Plain WG and QUIC can be blocked or severely degraded while the iroh control connection still works. Add an awg carrier with one independent UDP socket, WireGuard engine and randomized masking profile per network/peer attempt. The smaller endpoint id negotiates fresh outer keys, PSK, header protection, header ranges, padding and signature templates through private authenticated control messages. Restart or fallback reentry generates a new profile; ordinary rekeying preserves the current profile. No pair parameters enter signed state, capabilities, persistent storage or public message events. Private payloads and serialized control buffers redact and zeroize themselves. Adapt the bounded packet masking engine from RunaRoo/AmneziaWG-RS at commit 51579a7d6e8882d1d8d6d00822f09e18a9423b73, retaining its Apache license and provenance. Declare WTFPL AND Apache-2.0 in package metadata to cover both new glue and the adapted source. Reuse boringtun instead of upstream's Node/global executors and interface machinery. This is Tsunagi AWG wire version 1, with standard WG headers restored before authentication; external Amnezia server compatibility is not implemented. DNS/SIP signature generators vary fields instead of replaying captures. Each attempt closes independently, bounds allocation, queues, candidates and fragment assemblies, and supports the full 64 KiB carrier contract using UDP packets at most 1200 bytes. Yield between fragment batches so large messages do not starve local socket readers. Keep carrier encryption separate from end-to-end tunnels so multihop relays still forward opaque ciphertext. The system owns the sole interface, signed addresses and admission. Plugins receive neither network secrets nor paths. Expose a bounded opaque control sender at the transport boundary; core control messages remain outside protocol crates. Optional STUN on the pair's actual socket adds IPv4 public candidates in CLI direct/relay mode, with a finite query deadline and an off/alternate-server option. Local mode remains offline. Symmetric NAT, IPv6-only paths and arbitrary UDP blocking can still require fallback; no AWG-over-QUIC or hidden AWG relay is introduced. Default carrier order is wg, awg, wg-quic, tcp-tls. Advance the fallback cycle after setup failures and sustained runtime loss, even after earlier cooldowns expire. Authenticated delivery probes traverse the actual encrypted tunnel, fit the interface MTU and never enter the TUN. Retire after a 20-second warmup and at least 60% loss over at least 15 completed recent observations. Preserve logical tunnels and control sessions while explicitly closing physical links, including TCP reader/writer tasks. Retain the newest private probe settings across independently queued control negotiation and inbound installation. CLI/tray reports keep end-to-end protocol and actual carrier separate. Control ALPN/version becomes tsunagi/ctrl/6 for the new private message; upgrade all members together. Data ALPN/envelopes remain version 5. Network identity derivation, transcript encoding, device identities, signed records, saved network configuration and release versions are unchanged. Validation on Linux through nix develop: - cargo fmt --all -- --check - cargo clippy --locked --workspace --all-targets -- -D warnings - cargo test --locked --workspace --all-targets, with the inherited TSUNAGI_CONTROL_SOCKET override removed: 504 passed, 2 pre-existing ignored Tests use real iroh membership, SQLite, UDP sockets and WireGuard encryption with memory TUNs. Cover three independent pairs, full-size IP and carrier frames, restart freshness/socket release, 80% carrier datagram loss, fallback to real iroh, fresh retries with working delivery checks, unequal small MTUs, malformed negotiations/profiles, role and resource bounds, probe forgery and STUN parsing. Earlier compile/lint iterations and focused tests caught RNG API/initializer mistakes, a retry assertion racing path installation, and a large-fragment burst overflowing loopback buffers; corrected before these successful checks. Normalized upstream license line endings for diff checks. Public NAT/STUN connectivity, actual DPI effectiveness, WAN performance, per-pair memory measurement, macOS/Windows, privileged TUN operations and manual tray interaction were not exercised.
This commit is contained in:
1 parent
ed28e86748
commit
e701986a39
35 files changed
+3977
-41
No files matched your search
+116
@@ -0,0 +1,116 @@
|
||||
# Pairwise Amnezia transport
|
||||
|
||||
The CLI's default carrier preference is `wg,awg,wg-quic,tcp-tls`. To select
|
||||
AWG explicitly:
|
||||
|
||||
```sh
|
||||
tsng up --protocol awg
|
||||
tsng up --protocol wg,awg,wg-quic,tcp-tls
|
||||
```
|
||||
|
||||
`awg` is Tsunagi AWG wire version 1. Its packet masking engine is adapted from
|
||||
[RunaRoo/AmneziaWG-RS](https://github.com/RunaRoo/AmneziaWG-RS), with the pinned
|
||||
source and license recorded in
|
||||
[UPSTREAM.md](../crates/tsunagi-awg/UPSTREAM.md). It uses the existing boringtun
|
||||
Noise/WireGuard implementation instead of upstream's Node, global executors
|
||||
and OS interface machinery. Compatibility with ordinary Amnezia servers is
|
||||
not implemented: this carrier connects authenticated Tsunagi members.
|
||||
|
||||
## Negotiation and lifetime
|
||||
|
||||
There is one independent UDP socket and cancellable packet-engine task per
|
||||
`(network, peer)` pair. The smaller endpoint id initiates, following the same
|
||||
rule as other data carriers. Both sides must announce enabled AWG wire version
|
||||
1. An authenticated iroh control session is required throughout; AWG supplies
|
||||
no alternate discovery or membership mechanism.
|
||||
|
||||
Every `open` generates new masking parameters, a header-protection key, an
|
||||
outer WireGuard key pair and a PSK. The initiator sends a private `Offer` with
|
||||
a random attempt id, its public key, candidate socket addresses and the shared
|
||||
profile. The responder validates it, creates its own fresh key pair and socket,
|
||||
and returns an `Answer` naming that attempt. Both send masked handshakes and
|
||||
encrypted delivery probes to the candidates. Only authenticated WireGuard
|
||||
**data**, not a STUN response or an incoming UDP address alone, verifies a path.
|
||||
|
||||
Negotiation payloads are bounded at 4096 bytes (a caller can lower the limit),
|
||||
redacted in Debug, excluded from public message events and erased on drop.
|
||||
They are never capabilities, signed records, cache entries or exported status.
|
||||
At most 128 pairs are admitted per transport, with bounded command/data queues,
|
||||
at most 32 candidates per attempt and finite establishment deadlines.
|
||||
Stale answers and duplicate offers are rejected; a new attempt replaces and
|
||||
closes an older pair engine. Cancellation, peer departure, network deactivation
|
||||
and shutdown retire sockets without waiting for a remote reader.
|
||||
|
||||
A process restart, a failed setup, or a complete fallback cycle returning to AWG
|
||||
starts fresh negotiation. Normal WireGuard rekeying within a live attempt keeps
|
||||
that attempt's masking profile. A failed pair does not reconfigure other pairs
|
||||
or other networks. A full mesh of 15 members uses 14 engines per member and 105
|
||||
independent pairs in total; it does not create additional system interfaces.
|
||||
|
||||
## Masking
|
||||
|
||||
Each profile randomizes disjoint H1–H4 header ranges, a 12–48-byte S1–S4 prefix,
|
||||
a ChaCha20 header-protection key, handshake trailers, junk packet count/lengths,
|
||||
and extra content padding. Prefix lengths are equal within one profile so
|
||||
header detection has one offset. A fresh random nonce is in every packet's
|
||||
prefix; handshake headers and transport headers/counters are protected.
|
||||
The standard WireGuard header is restored before boringtun authenticates a
|
||||
message. This preserves boringtun's standard MAC calculations, which is another
|
||||
reason this is a Tsunagi-specific AWG wire version.
|
||||
|
||||
Before a handshake, an I1 signature packet is generated from either a complete
|
||||
DNS query or SIP OPTIONS template, with fresh identifiers and consistent length
|
||||
fields, followed by random junk packets. I2–I5 are unused in this version.
|
||||
The templates are structural generators, not replayed captures. QUIC imitation
|
||||
is not selected. Masking can change DPI classification; it is not a promise of
|
||||
indistinguishability or that any particular firewall will admit it.
|
||||
|
||||
## Packets, NAT and fallback
|
||||
|
||||
The pair engine carries opaque relay envelopes inside authenticated outer WG
|
||||
packets. It fragments those envelopes into bounded chunks and expires incomplete
|
||||
assemblies; a logical datagram is bounded at 64 KiB and an outer UDP packet is
|
||||
at most 1200 bytes. The encapsulated IPv4 headers are internal framing only: no interface or OS address is created for a
|
||||
pair. Overlay addresses, routing, ownership checks and end-to-end encryption
|
||||
remain independent. A relay decrypts its own adjacent AWG carrier to forward
|
||||
opaque end-to-end ciphertext, never the destination's IP packet.
|
||||
|
||||
Local IPv4 candidates and simultaneous encrypted probes provide direct UDP
|
||||
connectivity. In `direct`/`relay` reach mode the CLI also uses
|
||||
`stun.l.google.com:19302` to discover a server-reflexive address on **this pair's
|
||||
socket**, with a 1.2-second bound. A STUN result is only a candidate. Local reach
|
||||
mode makes no STUN request. Configure or disable it with
|
||||
`-o awg:stun=HOST:PORT` or `-o awg:stun=off`. Failed discovery still permits local
|
||||
candidates. IPv6-only paths, symmetric NAT and networks blocking arbitrary UDP
|
||||
may fail AWG and fall back; there is no AWG-over-QUIC substitution or AWG TURN
|
||||
relay hidden behind its name.
|
||||
|
||||
Data delivery checks go through the actual end-to-end encrypted tunnel, once
|
||||
per second, independently of control-plane RTT. Requests and replies bind a
|
||||
fresh private marker/key to the sender's endpoint id. Unanswered requests expire
|
||||
after three seconds. After a 20-second warmup, at least 15 completed observations
|
||||
and at least 60% loss in the last 20 observations retire a carrier. Probes are
|
||||
consumed locally after source/destination admission and never enter the TUN.
|
||||
AWG additionally checks encrypted outer delivery throughout the pair lifetime.
|
||||
|
||||
Setup failure or persistent runtime loss advances to the next mutually supported
|
||||
carrier in the initiator's order. Earlier expired cooldowns do not jump ahead of
|
||||
untried carriers. Retry cooldowns remain bounded; reaching AWG again negotiates
|
||||
new parameters. Control sessions and the logical end-to-end tunnel survive a
|
||||
carrier change. Status names the end-to-end protocol and the actual carrier
|
||||
separately, so `wg` over `awg` is a valid result.
|
||||
|
||||
Control wire version 6 introduces the private transport message. All members
|
||||
must upgrade together; device identities, network ids, handshake transcript
|
||||
encoding, signed state and stored network configuration are unchanged. Data
|
||||
ALPN remains `tsunagi/data/5`; the relay envelope is unchanged.
|
||||
|
||||
## Verification
|
||||
|
||||
The default rootless, offline tests use real iroh endpoints, membership proofs,
|
||||
SQLite, UDP sockets, masked WireGuard handshakes, end-to-end encryption and
|
||||
memory TUNs. They cover independent three-member pairs, full-size bidirectional
|
||||
packets, restart freshness/socket release, 80% runtime datagram loss, fallback
|
||||
to real iroh and fresh negotiation on retry, plus malformed profiles/messages,
|
||||
role checks, probe forgery and STUN response parsing. These tests do not establish
|
||||
DPI effectiveness, WAN throughput, public NAT traversal or per-pair memory cost.
|
||||
@@ -20,8 +20,9 @@ network secret. The core moves an opaque, bounded payload and never parses it.
|
||||
a `PacketLink` — an authenticated, unreliable datagram channel to one peer for
|
||||
one protocol — and never learn how it is carried.
|
||||
|
||||
The separation between the two planes is **logical, not physical**. Both ride
|
||||
on iroh, on different ALPNs and different connections. That is deliberate:
|
||||
The separation between the two planes is **logical, not physical**. The control plane uses
|
||||
iroh; the data plane can use iroh on its own ALPN and connection, or independent
|
||||
UDP/TLS carriers. That is deliberate:
|
||||
iroh's whole value is hole punching a direct path between peers behind NAT,
|
||||
with a relay as fallback, and a data plane that refused to use it would have to
|
||||
reimplement all of it. What the separation buys is that `proto` knows nothing
|
||||
@@ -32,6 +33,10 @@ A plugin talks to the core through three narrow hooks — `on_network_activated`
|
||||
a `PluginContext` for re-announcements and error reports, and a bounded
|
||||
`shutdown` — so the core never learns anything protocol-specific.
|
||||
|
||||
Private pair negotiation uses a bounded transport sender supplied by the core.
|
||||
Protocol crates do not encode or parse control messages and never receive the
|
||||
network secret. No pair profiles enter signed state.
|
||||
|
||||
A data plane failure never stops the daemon: the control plane keeps running
|
||||
and the agent stays manageable.
|
||||
|
||||
@@ -68,6 +73,7 @@ And in crates of their own:
|
||||
|
||||
| crate | responsibility |
|
||||
|---|---|
|
||||
| `tsunagi-awg` | ephemeral pairwise Amnezia masking, private negotiation and independently encrypted UDP carriers; reuses the end-to-end WireGuard engine |
|
||||
| `tsunagi-wg-quic` | the `wg-quic` protocol: its keys, its announcement, its tunnels |
|
||||
| `tsunagi` | the command line agent, built as the `tsng` executable: the only place that owns a runtime, a logger and signals |
|
||||
|
||||
|
||||
+6
-1
@@ -9,11 +9,15 @@ Two versions exist and are independent:
|
||||
|
||||
- **Identity scheme**, `tsunagi-network-id-v1`. Frozen. Changing it creates a
|
||||
different network space for the same name and secret.
|
||||
- **Control protocol**, ALPN `tsunagi/ctrl/5`, `PROTOCOL_VERSION = 5`.
|
||||
- **Control protocol**, ALPN `tsunagi/ctrl/6`, `PROTOCOL_VERSION = 6`.
|
||||
|
||||
Upgrading the crate or bumping the control protocol must never change an
|
||||
existing `NetworkId`.
|
||||
|
||||
Control version 6 adds private transport negotiation for pairwise AWG and data
|
||||
delivery checks. Data ALPN remains `tsunagi/data/5`. All members must upgrade
|
||||
together; network ids and stored state are unchanged. See [amnezia.md](amnezia.md).
|
||||
|
||||
## Canonical encoding
|
||||
|
||||
Everything that is hashed or MAC'd uses length-prefixed fields, written here as
|
||||
@@ -230,6 +234,7 @@ not affect other networks.
|
||||
| message | meaning |
|
||||
|---|---|
|
||||
| `Announce { hostname, capabilities, broadcast }` | this agent's hostname, IP-plugin capabilities and local broadcast participation |
|
||||
| `Transport { protocol, payload }` | private, bounded per-peer transport negotiation; never state or a public message event |
|
||||
| `Ping { seq, payload }` | small request used to verify the exchange |
|
||||
| `Pong { seq, payload }` | the echoed reply |
|
||||
| `State { records }` | a snapshot of signed records, merged into what the receiver holds |
|
||||
|
||||
+9
-1
@@ -96,6 +96,14 @@ topology change can move a flow; datagrams remain unreliable and unordered,
|
||||
and there is no promise to preserve order across a physical path failure.
|
||||
Logical peer links and end-to-end encryption state survive these changes.
|
||||
|
||||
## Carrier health and switching
|
||||
|
||||
Carrier setup failures and persistent encrypted delivery loss advance the
|
||||
initiator's ordered fallback cycle. Logical end-to-end links survive while
|
||||
physical links are retired explicitly. The AWG carrier negotiates a fresh
|
||||
private pair profile on every reentry, without changing topology's durable
|
||||
state. See [amnezia.md](amnezia.md) for thresholds and bounds.
|
||||
|
||||
## Verification and measurement
|
||||
|
||||
The default offline suite exercises four real agents, iroh transports and
|
||||
@@ -116,5 +124,5 @@ counts sends without storing frames. Results are CPU forwarding cost, **not**
|
||||
end-to-end network latency or VPN throughput; encryption, fragmentation,
|
||||
sockets, congestion and scheduling contribute separately.
|
||||
|
||||
Wire compatibility: control ALPN `tsunagi/ctrl/5`, data ALPN `tsunagi/data/5`.
|
||||
Wire compatibility: control ALPN `tsunagi/ctrl/6`, data ALPN `tsunagi/data/5`.
|
||||
Upgrade every participant together; saved identities and network state persist.
|
||||
@@ -177,3 +177,10 @@ TSUNAGI_TEST_LOG=tsunagi=debug cargo test --test multi_peer -- --nocapture
|
||||
|
||||
The library never installs a global subscriber; the harness opts in only when
|
||||
that variable is set.
|
||||
|
||||
The `tsunagi-awg` suite runs real pair negotiation and masked UDP WireGuard
|
||||
engines with memory TUNs. A loss wrapper drops datagrams at the carrier boundary
|
||||
without substituting iroh, membership, encryption or SQLite. It checks runtime
|
||||
loss detection, fallback and fresh profiles, in addition to restart and input
|
||||
validation. Tests are offline; public STUN/DPI/NAT effectiveness needs separate
|
||||
real-network validation.
|
||||
+13
-2
@@ -26,8 +26,8 @@ Read this before relying on anything here. The protocol is in
|
||||
neither receive nor forge another member's traffic. See
|
||||
[wireguard.md](wireguard.md#address-ownership-is-enforced-not-announced).
|
||||
- **Tunnelled traffic is end-to-end encrypted by WireGuard**, independently of
|
||||
this crate. The transport underneath is also encrypted by iroh, but the
|
||||
tunnel's confidentiality does not depend on that.
|
||||
this crate. Carriers may be iroh, TLS, direct WG or
|
||||
pairwise AWG; confidentiality does not depend on a particular carrier.
|
||||
- **Resource bounds.** Frame lengths are validated before allocation; strings,
|
||||
lists, queues, concurrent dials and in-flight handshakes are all bounded;
|
||||
handshakes, dials and writes have timeouts.
|
||||
@@ -113,3 +113,14 @@ for key separation, HMAC-SHA256 for the membership proof, constant-time
|
||||
verification, iroh's Ed25519 endpoint keys and QUIC/TLS for the transport, and
|
||||
RFC 5705 TLS exporter output for channel binding. There is no custom encryption
|
||||
layer and no custom PAKE.
|
||||
|
||||
## Pairwise AWG masking
|
||||
|
||||
AWG profiles, PSKs and outer keys are private to one authenticated pair attempt,
|
||||
not shared network state. Address candidates are unverified until authenticated
|
||||
outer WireGuard data arrives. Header protection and structured signature packets
|
||||
reduce fixed wire markers; they do not guarantee admission by DPI or hide traffic
|
||||
analysis. iroh control availability remains a prerequisite. Encrypted end-to-end
|
||||
delivery probes detect sustained carrier loss, never confer membership and pass
|
||||
the same signed source/destination admission as ordinary IP traffic. See
|
||||
[amnezia.md](amnezia.md) for limits and tested scope.
|
||||
Reference in new issue
Block a user