Aleksandr Bogomiakov
2166ba4800
AWG-only policy also selected a competing end-to-end plugin, so compatible external links could still have no compatible encrypted data channel. Carrier changes replaced tunnel state and status selected counters from different engines. Use one internal WireGuard plugin and a separate DataCarrier contract for external capabilities/lifecycle. Keep the logical link, keys, handshake, interface and counter epoch across policy, fallback and direct/relay changes. Control/discovery transport selection and membership remain independent. Share the bounded per-neighbour UDP WireGuard engine between ordinary WG and Amnezia masking. Encrypt the relay envelope inside the outer hop channel with fresh pair keys/PSK. Remove duplicate WG key/device code and the TCP/TLS inner cipher. TLS carries opaque internal WireGuard frames. Refuse missing native transports instead of letting default iroh impersonate their protocol names. Probe the actual authenticated carrier, independently of the shared IP tunnel; failed local probe submissions count as loss too. Apply the vendored iroh interface exclusion to actual direct candidates, not just public discovery records. Exclude overlay subnets from introductions and propagate the actual owned interface name, including kernel-assigned utun names, to every carrier. Production logs showed paths through the overlay itself. This removes a recursive transport path; it does not establish the historical cause of the reported 100 GB absolute counter. Exit-node traffic is also real peer IP traffic even when local applications are idle. Hide inner cipher choices/handshake clutter in CLI and tray. Report external routes, readiness and actionable policy/version failures. Count original IP packets only when their ciphertext is accepted by the logical transport, including queued packets' exact unpadded lengths. Exclude physical probes and keepalives. Export counter epochs; sum per-peer rate deltas and discard stale histories so joins/restarts cannot turn old totals into fresh traffic. Status also prints exact IP byte totals for comparison with the tray. Coordinated upgrade: data ALPN 6, internal wireguard-e2e capability v4, wg/awg v2, QUIC carrier v1, TLS v2, local IPC v23. Control ALPN 8, device/network identity derivation, signed records and saved network choices stay unchanged. Keep the historical wg-quic key store; transactionally import missing keys from wg then awg without replacing canonical keys. Forget removes legacy copies too, on the blocking pool, to prevent resurrection after restart. Preserve historic cipher options as shared settings, rejecting contradictory values. Validation on Linux in the Nix development shell: - cargo fmt --all -- --check - cargo clippy --locked --workspace --all-targets -- -D warnings - cargo test --locked --workspace --all-targets (562 passed, 3 ignored) The inherited service socket/secret/group environment overrides were removed for isolated CLI process tests. A new carrier test initially raced the first peer announcement; its deadline polling now waits for a peer row explicitly. Real loopback endpoints, membership, SQLite, WG/AWG/TLS encryption and memory TUNs cover mixed-carrier relaying and one smoltcp TCP connection surviving relay/direct/policy changes and a dropped transmission. Counter tests check exact plaintext lengths, refused sends, epochs, joins and stale histories. Public DHT, privileged kernel and manual benchmark tests remain ignored; macOS/Windows native TUNs, desktop GUI and WAN/DPI behavior were not run.
3.6 KiB
Bounded path retries
This is the crates.io iroh 1.2.0 source, with registry bookkeeping and its
independent Cargo.lock omitted. The workspace selects it with [patch.crates-io].
- Published source commit:
17c0612f80f78f5288e97b818b1360ae6ea0a51a. - Crates.io archive SHA-256:
b2f8d1cfffc83efe39a1031aab423ce09cb8048071baba550508931e9a81ce46. - Upstream source, manifests, examples, tests and license are otherwise retained.
State::open_path_on_conn appended a failed path once per connection. The actor
takes the queue every 333 ms and retries every entry on every connection to that
remote. Two failing connections therefore double duplicate work each round.
Tsunagi deliberately keeps control and data on separate QUIC connections, so
merging those connections would violate its architecture and conceal the bug.
On 2026-10-14 a Linux rc.33 daemon reached OOM after roughly five hours. Before
the kill, /proc showed anonymous mappings of 40 GiB and 10 GiB, later 20 GiB.
Pointers and container capacities were consistent with the retry queue and its
drained batch. The active worker's stack was in iroh's retry loop,
open_path_on_all_conns and open_path_on_conn, identified through executable
addresses and tracing metadata in the stripped static binary. This matches
upstream issue 4509.
The local change extracts the enqueue operation into pending_paths.rs, rejects
duplicates, and limits pending candidates to 64, evicting the oldest before
admitting a new distinct path. The current retry batch and next batch are each
bounded; no wire formats, identity derivation, timers or authentication change.
The upstream deduplication fix was merged in
PR 4551 on October 6. Both the
published 1.2.0 source used here and the 1.3.0 crates.io archive (released
September 28) still have the unconditional enqueue. Upgrading to 1.3.0 alone
does not fix this OOM. Remove this override when adopting a release that
includes the fix, retaining a retry regression check.
crates/tsunagi-core/tests/iroh_path_retries.rs includes the exact admission
module so its tests run in the default workspace suite, even though vendored
dependencies are excluded from workspace membership. The tests reproduce
multi-connection retry multiplication, preserve FIFO order under duplicate
failures, exercise success draining, and bound length and capacity under 10,000
distinct candidates. They fail with the original unconditional enqueue.
These are queue algorithm tests; the normal integration suite continues to use
real iroh endpoints, membership handshakes, SQLite and encryption.
Excluding overlay candidates
Builder::addr_filter filters discovery publication only. That is insufficient
for an overlay: iroh still put the overlay interface address in the local
candidate watch used for path creation, allowing its own transport packets to
re-enter the overlay. Production logs contained verified data paths via
10.13.37.* overlay addresses while their underlay links cycled.
The separate optional Builder::direct_addr_filter applies an AddrFilter
before storing local direct addresses. Default behavior is unchanged. Tsunagi
uses its existing local-interface exclusion for both this filter and publication,
including the actual kernel-assigned interface name after creation. Control authentication and
wire formats are unchanged. The rootless real-endpoint regression is
crates/tsunagi-core/tests/iroh_candidates.rs; overlay subnet filtering for
introductions is also covered in the core. This prevents the recursive path;
it does not prove the historical cause of any particular high byte counter.