bb38562b0a28305a25efa2a1e643ca8f3e57c8a5
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksandr Bogomiakov
|
d2778f0b5a |
Prevent WireGuard timer loop failure spam and fix handshake broadcast
When a WireGuard tunnel handshake timed out or a rekey failed,
Inner::on_tunnel_failed closed the transport link and reported the failure
to the agent core. However:
1. The peer remained in WireguardDevice::inner.peers. Once boringtun
declared ConnectionExpired (after 90s), self.tunn.is_expired() remained
permanently true on all subsequent 250ms timer ticks. drive_timers
repeatedly detected failed = true every 250ms, flooding logs with
'WARN WireGuard tunnel failed due to rekey timeout, triggering fallback'
and sending repeated protocol failure events to the core 4 times/sec.
2. In WireguardLink::send, is_handshake_init incorrectly expected an
envelope header offset (msg[74] == 1) when checking for initiation
packets, which failed on raw WireGuard UDP packets. As a result,
handshake initiations were not broadcast to all candidate endpoints.
3. In WireguardTransport::accept_loop, packet source matching contained
dead envelope destination matching inspecting ephemeral key bytes
against local endpoint IDs.
Fixes:
- In crates/tsunagi-wireguard/src/device.rs:
- Add failed: bool to FlowTunnel to latch failure state. Once a failure
is reported, update_timers returns (TunnResult::Done, false) on all
subsequent ticks until a new handshake is started or completed.
- In Inner::on_tunnel_failed, remove the failed peer from inner.peers so
drive_timers ceases ticking on the dead tunnel.
- In crates/tsunagi-wireguard/src/plugin.rs:
- In Worker::sync, retain only active unclosed links in state.links.
- In crates/tsunagi-wireguard/src/transport.rs:
- Check is_handshake_init directly as '!msg.is_empty() && msg[0] == 1',
ensuring handshake initiations are broadcast to all candidate addresses.
- Clean up matched_peer to look up links and peer candidate addresses directly.
- Tests:
- Update test_flow_tunnel_rekey_timeout_triggers_failure to assert failure
latching and verify no repeated failure reports on subsequent ticks.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed.
- cargo build --release passed.
|
||
|
Aleksandr Bogomiakov
|
3930dff2dc |
Detect WireGuard tunnel rekey failure and trigger automatic fallback
Previously, because UDP is connectionless, when a pure WireGuard tunnel
encountered handshake or rekey failures (e.g. boringtun logging
`WARN HANDSHAKE(REKEY_TIMEOUT)` and `CONNECTION_EXPIRED`), the
`WireguardLink` never closed. It remained active in `agent.links` and
as a direct hop in `RelayHub.raw`. Consequently, outgoing packets
continued to be blackholed over the dead direct path, preventing
`RelayHub` from using working multihop relay routes and preventing
the agent from falling back to alternative protocols (`tcp-tls` or
`wg-quic`).
Key changes:
- In `crates/tsunagi-wireguard/src/device.rs`:
- In `FlowTunnel`, track active handshake and rekey attempts via
`handshake_started_at`. If an active handshake attempt remains
unanswered beyond `HANDSHAKE_TIMEOUT` (15s) or boringtun reports
`ConnectionExpired`, declare the tunnel dead.
- Reset timeout timers upon successful handshake reception in
`decapsulate`.
- In `drive_timers`, when tunnel failure is detected, trigger
`inner.on_tunnel_failed(peer)`, which closes the direct transport
link and notifies the agent via `PluginContext::report_peer_protocol_failure`.
- In `crates/tsunagi-wireguard/src/transport.rs`:
- Add `WireguardLink::close` to transition `is_closed` to true and
notify `closed_notify`.
- In `WireguardLink::recv`, select on `closed_notify` so readers
unblock and return `None` immediately on close.
- Add `WireguardTransport::close_link` to close active links by
network and peer.
- In `crates/tsunagi/src/dataplane/mod.rs`:
- Add `PluginRequest::ProtocolFailure` and public helper
`PluginContext::report_peer_protocol_failure`.
- In `crates/tsunagi/src/agent/mod.rs` and `crates/tsunagi/src/agent/network.rs`:
- Add `NetCommand::ProtocolFailure` to runtime command handling.
- On protocol failure: remove dead direct link from `self.links`,
clear it from `RelayHub.raw` (so `RelayHub` immediately routes
over available multihop relays), record failure on `wg` for that
peer with exponential backoff cooldown, emit `DataLinkDown`, and
immediately call `ensure_links()` to dial the next mutually supported
protocol (`tcp-tls` or `wg-quic`).
- Tests:
- Add `test_flow_tunnel_rekey_timeout_triggers_failure` and
`test_flow_tunnel_reset_on_handshake` unit tests in `device.rs`.
- Add `test_wireguard_link_close_and_transport_close_link` in
`transport_tests.rs`.
- Add `test_protocol_dial_state_cooldown_and_fallback_selection` in
`network.rs`.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
- Built release binary: target/release/tsunagi.
|
||
|
Aleksandr Bogomiakov
|
35acc4d231 |
Fix WireGuard relay preemption with UDP probing, allow VM bridges, and categorize status addresses
When remote peers communicate across networks with mixed connectivity (e.g.
a remote Linux node, a local Linux host, and a Windows VM guest under KVM):
- Previously, `WireguardTransport` immediately created and installed
unverified direct links into `RelayHub` in `notify_peer_capability` and
`open()`. When a remote peer had only unroutable private addresses
(e.g. 192.168.122.151 on a KVM NAT bridge), this dead direct link
preempted the working 2-hop relay, causing packet drops, failed handshakes,
and REKEY_TIMEOUT errors.
- Host-guest VM bridges (`virbr*`) were previously filtered out, preventing
the VM guest from discovering the host's bridge IP (192.168.122.1) and
forcing it to attempt cross-interface connections to host Wi-Fi/LAN.
- Concurrent activation of multiple networks sharing one `tsun0` interface
raced in `sync_addresses`, resulting in netlink `File exists (os error 17)`
and `The object already exists` on Windows.
Key fixes:
- In `crates/tsunagi-wireguard/src/transport.rs`:
- Implement bidirectional UDP probing with `TSUNAGI_WG_PROBE_V1` PING/PONG.
- In `open()`, probe candidate endpoints with retransmission up to 1200ms.
If unreachable, return `TransportError::Unreachable`, preventing dead
links from preempting valid relay routes in `RelayHub`.
- In `notify_peer_capability`, send hole-punching probes without prematurely
installing unverified links.
- In `accept_loop`, reply to PING probes with PONG and install confirmed
inbound links targeting the verified source address.
- In `WireguardLink::send`, inspect byte 74 of enveloped packets to detect
handshake initiations, ensuring confirmed links route data packets only
to the active target.
- In `crates/tsunagi-wireguard/src/plugin.rs`:
- Permit VM bridges (`virbr*`) as candidate interfaces (priority 10).
- Add `prioritize_local_subnet` to prioritize candidates sharing a local
IPv4 subnet with the receiver (e.g. 192.168.122.0/24).
- In `crates/tsunagi/src/overlay/interface.rs` & provisioners:
- Add async mutex `sync_lock` to serialize concurrent address syncs across networks.
- Handle netlink EEXIST (os error 17) and Windows already-exists errors
idempotently as success.
- In `crates/tsunagi-cli/src/main.rs`:
- Split `local addresses` in `tsunagi status` into `announced` (candidate
addresses in priority order with interface and rank), `not announced`
(container bridges/internal addresses with interface name), and `loopback`.
- Tests:
- Add `test_wireguard_transport_unreachable_probe_times_out` and
`test_matches_subnet` unit tests.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
|
||
|
Aleksandr Bogomiakov
|
c791102ad9 |
Fix overlay and virtual interface candidate leakage and WireGuard rekeying
On Windows (Wintun), iface.name in netdev is an adapter GUID, while
iface.friendly_name is 'tsun0'. Checking only iface.name failed to
identify and exclude Wintun overlay interfaces. Consequently, Windows
nodes announced their own overlay IP addresses (e.g. 10.13.37.26) as
candidate endpoints. Remote peers latched onto this overlay IP as the
WireGuard target, routing encrypted packets into tsun0 (recursive loop),
resulting in failed handshakes and subsequent HANDSHAKE(REKEY_TIMEOUT)
and CONNECTION_EXPIRED.
Similarly, Docker bridges (br-*, docker0) and virtual bridges were
announced and alphabetically sorted before physical LAN or Tailscale
interfaces, causing peers to attempt WireGuard handshakes with internal
Docker bridge addresses (e.g. 172.21.0.1).
Key changes:
- In crates/tsunagi-wireguard and crates/tsunagi-tcp-tls:
- Robustly identify overlay interfaces by inspecting iface.name,
iface.friendly_name, and iface.description for 'tsun', 'tsunagi',
and 'wintun'.
- Filter out container and virtual bridge interfaces (docker, br-,
veth, virbr, cni, flannel, cali, kube, dummy, lxc, podman, vethernet,
vboxnet, vmnet).
- Prioritize candidates by physical interface score: interfaces with
a default gateway (physical LAN/WAN) and Tailscale (100.64.0.0/10)
score highest and sort to candidate #0.
- In on_peer_capability, filter out any announced address falling inside
any active overlay IPv4 range (defense in depth on receiver).
- In crates/tsunagi-wireguard/src/transport.rs:
- In WireguardLink::send, broadcast Handshake Initiation packets (type 1)
to all candidate addresses so initial handshakes and rekeys probe all
candidate paths and recover if an endpoint roamed or NAT expired.
- In accept_loop, prioritize exact socket address matches before falling
back to candidate IP matches for NAT port roaming.
- In crates/tsunagi/src/net.rs:
- Update iroh AddrFilter to inspect friendly_name and description,
preventing Wintun overlay IPs from leaking into iroh discovery endpoints.
- Tests:
- Add unit tests for candidate interface filtering, IP validation, and
priority scoring in tsunagi-wireguard and tsunagi-tcp-tls.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
|
||
|
Aleksandr Bogomiakov
|
f80841da63 |
Fix WireGuard multi-network demux, candidate discovery, and status reporting
When a peer belongs to multiple overlay networks served by one agent (e.g.
a VM participating in both 'hexor' and 'ultradesu'), packets received on
the shared WireGuard UDP socket could not be differentiated by source IP/port
alone. Previously, `WireguardTransport::accept_loop` picked the first
matching link found in iteration order, starving other networks for that peer
and causing rekey timeouts (`WARN HANDSHAKE(REKEY_TIMEOUT)`,
`ERROR CONNECTION_EXPIRED(REKEY_ATTEMPT_TIME)`) and packet loss on the
starved network.
Additionally:
- When a peer had multiple candidate endpoints (such as LAN, VM bridge, and
Tailscale CGNAT addresses), `WireguardLink` sent handshake initiations
only to `addrs[0]`. If the first address was unreachable from the remote
side (e.g. host's Tailscale IP unreachable from inside a local VM), the
handshake never completed.
- In `build_report`, peers known to the `wg-quic` plugin were added with
empty tunnels even if an active `wg` (pure WireGuard) tunnel existed,
causing `tsunagi status` to falsely report 'no WireGuard handshake yet'
for connected and pingable peers.
- `local_ip_candidates` previously included IPv6 addresses which caused
OS error 22 (EINVAL) when passed to the IPv4-bound UDP socket.
Key fixes:
- In `crates/tsunagi-wireguard/src/transport.rs`:
- Store `peer_networks` as `HashSet<NetworkId>` per peer.
- In `WireguardLink`, maintain a list of candidate addresses and a
`confirmed` flag. Before a handshake response confirms the active path,
transmit datagrams (handshake initiations) to all candidate addresses.
Upon receiving an inbound datagram, lock onto the confirmed address.
- In `accept_loop`, deliver inbound datagrams to all active links
associated with the sender's endpoint ID across all networks. Boringtun's
cryptographic index and AEAD verification naturally demultiplexes and
accepts the packet in the correct network while discarding it in others.
- Filter candidate addresses to IPv4 to match the bound socket family.
- In `crates/tsunagi-wireguard/src/plugin.rs`:
- Only collect IPv4 interfaces in `local_ip_candidates`.
- In `crates/tsunagi-cli/src/main.rs`:
- Refactor `build_report` to evaluate all protocols per peer and prioritize
whichever protocol has an active tunnel (`wg` > `tcp-tls` > `wg-quic`).
- In `crates/tsunagi-wireguard/tests/plugin_tests.rs`:
- Add `test_wireguard_two_networks_same_peers` verifying simultaneous
bidirectional packet transmission between two peers sharing two networks
over a single `WireguardTransport`.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
|
||
|
Aleksandr Bogomiakov
|
7070a16104 |
Add pure WireGuard direct UDP data plane plugin and transport
Peers communicating on uncensored networks can benefit from raw WireGuard
performance over direct UDP without QUIC framing or double-encryption
overhead. This provides Tailscale-like direct tunnel performance while
preserving Tsunagi's decentralized mesh architecture.
Key changes:
- Create `tsunagi-wireguard` crate implementing the `IpPlugin` and
`PacketTransport` contracts:
- `WG_PROTOCOL = "wg"` with wire version 1.
- Bounded `WgAnnouncement` carrying WireGuard public keys, listening UDP
ports, and candidate IP addresses (filtering out overlay interfaces).
- `WgKeyStore` managing persistent per-network WireGuard X25519 keys
in SQLite with zeroization on drop.
- `WireguardTransport` providing direct UDP packet transmission, candidate
address tracking, inbound link delivery, and roaming support.
- `WireguardDevice` running userspace WireGuard encryption via `boringtun`,
timer maintenance, and AllowedIPs source enforcement.
- Integrate `wg` into `tsunagi-cli`:
- Set default protocols to `"wg,tcp-tls,wg-quic"` so all protocols are
enabled by default in order of throughput and performance.
- Support `"wireguard"` as a convenience alias for `"wg"` in `--protocol`.
- Bind `PureWgTransport` and launch the inbound datagram accept loop on `up`.
- Pass `pure_wg` plugin to `AgentControl` and update `build_report` so
active WireGuard tunnels and overlay metrics are reported in `tsunagi status`.
- Clean up WireGuard keys on network removal in `forget_protocol_state`.
- Comprehensive test coverage in `tsunagi-wireguard`:
- `announcement_tests`: round-trip serialization and verification of malformed,
mismatched network, zero port, or self-claiming announcements.
- `transport_tests`: loopback UDP link delivery between transports.
- `plugin_tests`: end-to-end two-peer handshake and bidirectional IP packet
transit over direct UDP.
Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed.
|