On Windows (Wintun), iface.name in netdev is an adapter GUID, while
iface.friendly_name is 'tsun0'. Checking only iface.name failed to
identify and exclude Wintun overlay interfaces. Consequently, Windows
nodes announced their own overlay IP addresses (e.g. 10.13.37.26) as
candidate endpoints. Remote peers latched onto this overlay IP as the
WireGuard target, routing encrypted packets into tsun0 (recursive loop),
resulting in failed handshakes and subsequent HANDSHAKE(REKEY_TIMEOUT)
and CONNECTION_EXPIRED.
Similarly, Docker bridges (br-*, docker0) and virtual bridges were
announced and alphabetically sorted before physical LAN or Tailscale
interfaces, causing peers to attempt WireGuard handshakes with internal
Docker bridge addresses (e.g. 172.21.0.1).
Key changes:
- In crates/tsunagi-wireguard and crates/tsunagi-tcp-tls:
- Robustly identify overlay interfaces by inspecting iface.name,
iface.friendly_name, and iface.description for 'tsun', 'tsunagi',
and 'wintun'.
- Filter out container and virtual bridge interfaces (docker, br-,
veth, virbr, cni, flannel, cali, kube, dummy, lxc, podman, vethernet,
vboxnet, vmnet).
- Prioritize candidates by physical interface score: interfaces with
a default gateway (physical LAN/WAN) and Tailscale (100.64.0.0/10)
score highest and sort to candidate #0.
- In on_peer_capability, filter out any announced address falling inside
any active overlay IPv4 range (defense in depth on receiver).
- In crates/tsunagi-wireguard/src/transport.rs:
- In WireguardLink::send, broadcast Handshake Initiation packets (type 1)
to all candidate addresses so initial handshakes and rekeys probe all
candidate paths and recover if an endpoint roamed or NAT expired.
- In accept_loop, prioritize exact socket address matches before falling
back to candidate IP matches for NAT port roaming.
- In crates/tsunagi/src/net.rs:
- Update iroh AddrFilter to inspect friendly_name and description,
preventing Wintun overlay IPs from leaking into iroh discovery endpoints.
- Tests:
- Add unit tests for candidate interface filtering, IP validation, and
priority scoring in tsunagi-wireguard and tsunagi-tcp-tls.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
Peers previously advertised overlay interface (tsun0) addresses as physical
candidate endpoints for both iroh discovery and tcp-tls announcements. When an
intermediate tunnel was active between two peers (e.g., between a host and a VM),
iroh probes to the remote peer's overlay address were routed through the existing
overlay, causing iroh to select the overlay IP itself as a 'direct' endpoint.
This caused recursive tunneling (QUIC over WireGuard over TLS) and confusing
status output showing peers connecting to their own overlay IP addresses.
Additionally, in tsunagi status, peer rows were displayed with unaligned
columns, mixing control plane status and data plane metrics in a way that was
difficult to scan.
Key improvements:
- In crates/tsunagi/src/net.rs, add an AddrFilter to the iroh EndpointBuilder
to dynamically filter out any local addresses assigned to tsun* overlay
interfaces before publishing to address discovery services (DHT/pkarr/STUN).
- In crates/tsunagi-tcp-tls/src/plugin.rs, exclude tsun* interfaces from
local_ip_candidates() so overlay IPs are never advertised as physical
TCP endpoints.
- In crates/tsunagi-cli/src/main.rs, exclude tsun* interfaces from
netwatch_addresses() so the local addresses section reflects only actual
physical/host network interfaces.
- In crates/tsunagi-cli/src/main.rs, update member_row() to format member
details with aligned columnar layout: the overlay IP is placed first and
padded, followed by control plane transport and latency, and line 2 pads
tunnel metrics so that the data link endpoint (via ...) aligns vertically.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets --jobs 2 -- --test-threads=2 passed (349 tests passed).
By default, the agent now enables all available data plane protocols
starting with `tcp-tls,wg-quic`, removing the need to manually pass `--protocol`
to enable HTTPS-camouflage transport. The `--protocol` option remains available
to explicitly restrict or select specific protocols (or reorder their priority).
Furthermore, in `crates/tsunagi-tcp-tls`, if default port 443 is unavailable
(e.g., due to permission constraints or address in use), the transport now
attempts fallback port 8443, and if 8443 is also unavailable, gracefully falls
back to an ephemeral port rather than failing agent startup.
Key changes:
- In `crates/tsunagi-cli/src/main.rs`, update `UpArgs::protocols` default_value
to `"tcp-tls,wg-quic"`.
- In `crates/tsunagi-cli/src/main.rs`, reorder `PROTOCOLS` registry with `tcp-tls`
first, followed by `wg-quic`.
- In `crates/tsunagi-tcp-tls/src/transport.rs`, add multi-tier fallback
(443 -> 8443 -> ephemeral) for unconstrained port bindings.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (349 tests passed).
- cargo build --release passed.
When multiple data plane protocols (such as tcp-tls and wg-quic) are configured,
the agent previously attempted only the first mutually supported protocol in
served order. If that protocol failed to connect or dropped (due to NAT, firewall,
or handshake failure), ensure_links retried the exact same failed protocol
on every tick without ever falling back to alternative supported protocols.
Additionally, tcp-tls mistakenly included 127.0.0.1 in peer candidate lists,
causing peers to connect to their own local listener and fail certificate
verification (NotValidForName). Finally, BitTorrent DHT bootstrap failures
in the third-party mainline crate were logged as noisy ERRORs despite overlay
connectivity operating normally.
Key improvements:
- In crates/tsunagi/src/agent/network.rs, implement ProtocolDialState with
exponential backoff (3s, 6s, 12s, 24s, 48s, capped at 60s).
- In ensure_links, select the highest-priority mutually supported protocol
that is not currently on backoff cooldown. If a protocol fails, immediately
record the failure and fall back to alternative mutually supported protocols
without waiting for the periodic ticker.
- In crates/tsunagi-tcp-tls, remove localhost loopback from local_ip_candidates
and capability announcements, and prevent open() from connecting to our own
listening port.
- In crates/tsunagi-cli/src/main.rs, set mainline=off in the default tracing
filter so benign BitTorrent DHT routing table bootstrap errors do not alarm users.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (349 tests passed).
- cargo build --release passed.
When running on Windows with Wintun, assigning an overlay IPv4 address
via netsh triggers Duplicate Address Detection (DAD). During this tentative
window, binding a UDP socket to the address returns WSAEADDRNOTAVAIL
(os error 10049), causing the agent to prematurely log a false-positive
PluginError ('allocated ... but the address is not on any interface').
Furthermore, Russian TSPU / DPI filters block unrecognized SNIs and
non-standard ALPNs on TCP port 443 with TCP RST injection (os error 10054).
The default 'tsunagi.local' SNI and 'tsunagi-tls/1' ALPN were trivially
fingerprinted.
Key improvements:
- In crates/tsunagi/src/overlay/tun.rs, enhance address_is_local on Windows
to inspect active OS interfaces via netdev::get_interfaces() if UDP
bind fails.
- In crates/tsunagi/src/agent/network.rs, introduce an asynchronous retry
loop in publish_allocations on Windows (up to 1.5s in 150ms intervals)
to gracefully allow DAD and IP helper updates to complete before
reporting missing addresses.
- In crates/tsunagi-tcp-tls, add support for custom SNI domain via
protocol options (-o sni=<domain>), defaulting to 'cloudflare.com'
instead of 'tsunagi.local'.
- Switch TLS ALPN in tsunagi-tcp-tls from 'tsunagi-tls/1' to 'http/1.1'
to camouflage handshake traffic as standard HTTPS web traffic.
- Update tests to verify custom and default SNI behavior.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (348 tests passed).
- cargo build --release passed.
When file capabilities are granted with `sudo setcap ...+p`, capabilities
are placed into the thread's permitted set rather than its effective set.
While TUN interface provisioning in `tsunagi` properly acquired and dropped
CAP_NET_ADMIN using `caps::raise`, `tsunagi-tcp-tls` attempted to bind port 443
directly without activating CAP_NET_BIND_SERVICE, resulting in EACCES
(os error 13 Permission denied).
Fixes and improvements:
- Add `caps` dependency to `tsunagi-tcp-tls` on Linux.
- Implement `NetBindServiceGuard` to temporarily raise CAP_NET_BIND_SERVICE
into the effective set for the synchronous `TcpListener::bind` invocation
and immediately drop it afterwards.
- Clarify diagnostic warnings to distinguish between default port 443 fallback
and explicitly requested port failure.
- Update all privilege diagnostics and error hints across the codebase
to uniformly recommend granting both capabilities:
`sudo setcap cap_net_admin,cap_net_bind_service+p <path>`.
Validation on Linux:
- `cargo fmt --all -- --check` passed.
- `cargo clippy --locked --workspace --all-targets -- -D warnings` passed.
- `cargo test --locked --workspace --all-targets` passed (all workspace unit,
integration, and protocol tests passed).
Implement the `tcp-tls` protocol crate and generic tunnel codec abstractions
alongside `wg-quic` to circumvent deep packet inspection (DPI/TSPU), UDP
blocking, and QUIC throttling. Traffic uses standard TCP on port 443 with
TLS 1.3 framing, looking identical to regular HTTPS traffic to network
observers.
Architectural highlights:
- TunnelCodec & GenericTunnelPlugin: Provide an extensible plugin abstraction
in `crates/tsunagi/src/dataplane/tunnel.rs` that encapsulates background task
lifecycles, per-peer channels, and sink dispatch, so new transport plugins
only define cryptographic transformations and handshake payloads.
- MultiPacketTransport & Protocol Priority Selection: Support multiple
simultaneous transports in `AgentConfig` and dynamically dispatch `open()`
calls by protocol identifier. In `ensure_links`, choose the highest-priority
mutually supported protocol per peer to avoid duplicate links.
- TLS 1.3 & Certificate Pinning: Generate self-signed Ed25519 X.509 certificates
derived from the persistent `DeviceIdentity` secret. Clients pin peer
certificates using `PinnedEndpointVerifier` against `iroh::EndpointId`.
- End-to-end Encryption: `PeerCipher` uses X25519 static-static ECDH, HKDF-SHA256,
and ChaCha20-Poly1305 with incrementing 64-bit nonces. Relayed traffic across
intermediate nodes remains confidential and tamper-proof.
- Privilege handling: Default to TCP port 443 with fallback to 8443 on
`PermissionDenied`, providing an actionable message recommending:
`sudo setcap cap_net_admin,cap_net_bind_service+p <path>`.
Validation on Linux:
- `cargo fmt --all -- --check` passed.
- `cargo clippy --locked --workspace --all-targets -- -D warnings` passed.
- `cargo test --locked --workspace --all-targets` passed (all workspace unit,
integration, CLI, and new `tsunagi-tcp-tls` tests passed).
Aleksandr Bogomiakov