Compare commits

...
24 Commits
Author SHA1 Message Date
UltradesuandClaude Opus 4.8 eac54f8c79 Revert the GitHub workflow guard to server_url
Release / tsunagi-linux-aarch64 (push) Canceled after 15s
Release / tsunagi-linux-x86_64 (push) Canceled after 0s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Misdiagnosed the earlier report: Gitea was correctly running its own
.gitea/workflows copy, not this .github one, so the server_url guard was fine.
The switch to a GITHUB_RELEASE repo variable then made the workflow skip on
GitHub itself, since that variable is not set there.

Put both job guards back to `github.server_url == 'https://github.com'`, which
runs on GitHub with no manual setup and skips on Gitea (its .gitea copy is the
one that runs). No repository variable is needed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:53:26 +01:00
UltradesuandClaude Opus 4.8 a8dfd7017b Gate the GitHub release workflow on a repo variable, not server_url
Release / tsunagi-linux-aarch64 (push) Canceled after 35s
Release / tsunagi-linux-x86_64 (push) Canceled after 0s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
The github.server_url guard did not keep this workflow from running on Gitea:
Gitea reports github.server_url as https://github.com, so the condition was
true there too and the jobs ran (and would fail on @v4/gh).

Switch both job guards to `vars.GITHUB_RELEASE == 'true'`, a repository variable
set only in the GitHub repo's settings. It is deterministic: absent on Gitea, so
the jobs skip there and the .gitea workflow is the one that runs; present on
GitHub once configured, so it runs there. The header documents how to set it
(Settings → Secrets and variables → Actions → Variables → GITHUB_RELEASE=true).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:50:04 +01:00
UltradesuandClaude Opus 4.8 99f4cf0220 Add a GitHub release workflow alongside the Gitea one
Release / tsunagi-linux-aarch64 (push) Canceled after 36s
Release / tsunagi-linux-x86_64 (push) Canceled after 0s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Gitea and GitHub need different release plumbing, so each gets its own file: the
Gitea workflow stays in .gitea/workflows, and this GitHub one lives in
.github/workflows. The build half is identical (static musl Linux via
cargo-zigbuild, native macOS aarch64 and Windows x86_64); only publishing
differs — GitHub uses the v4 artifact backend and `gh release create`/`upload`,
which the Gitea file cannot use.

Because Gitea reads both .gitea/workflows and .github/workflows, every job here
is guarded with `if: github.server_url == 'https://github.com'`, so on Gitea
this file's jobs are skipped (no v4/gh failures) and the .gitea file is the one
that runs. GitHub never reads .gitea, so there it runs only this one.

Publish is re-run safe: it creates the release on the first run and uploads with
--clobber onto an existing release afterwards; a pre-release suffix on the tag
marks it prerelease.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:47:19 +01:00
UltradesuandClaude Opus 4.8 69263b2dd9 Move the release workflow to .gitea/workflows
Release / tsunagi-linux-aarch64 (push) Successful in 3m39s
Release / tsunagi-linux-x86_64 (push) Successful in 3m27s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
The release pipeline is Gitea-specific (v3 artifacts, Gitea release API), so it
must not run on github.com, where it would fail. GitHub only reads
.github/workflows, while Gitea reads both, so moving the file to
.gitea/workflows keeps it running on Gitea and invisible to GitHub. A separate
GitHub-flavoured release workflow can be added under .github later.

The portable .github/workflows/ci.yml (fmt/clippy/test) is left in place; it
works on both and is not what this change is about.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:13:45 +01:00
UltradesuandClaude Opus 4.8 20444013c0 Split release into build artifacts + a publish job (Gitea)
The direct-upload-from-each-build approach failed ("Upload the archive to the
release" exit 1) and left no artifacts to inspect. Restructured into the
familiar shape: build jobs now leave their archive as a run artifact, and a
single publish job — gated on all targets succeeding — collects them and fills
the release.

- Build jobs use actions/upload-artifact@v3 (Gitea has no v4 backend; v3 is
  what shows under the run), so the archive is visible and handed on.
- publish (needs: build) downloads everything with download-artifact@v3 and
  uploads each archive to the Gitea release with curl, once, with no matrix
  race. It prints the API base and the HTTP status/body of every call, so a
  failure (e.g. a read-only Actions token → 403, or a bad API URL) is now
  visible in the log instead of a bare exit 1.
- Release API base prefers GITHUB_API_URL (Gitea sets it to its /api/v1 root)
  and falls back to GITHUB_SERVER_URL/api/v1. Asset uploads run in the main
  shell via process substitution so a failed upload fails the job.

Build, package, musl/zig setup and the target matrix are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:12:53 +01:00
UltradesuandClaude Opus 4.8 2196f6635b Quiet the cosmetic zig linker_messages warning on the musl build
The musl release build now links cleanly through cargo-zigbuild, but rustc's
new linker_messages lint echoed zig's "ignoring deprecated linker optimization
setting '1'" note — zig's linker declining the -O1 GNU-ld flag Rust passes. It
is cosmetic and the binary is correct, so allow that one lint, scoped to the
zig build step only. macOS and Windows keep full linker diagnostics, and a real
link error still fails the build.

No behaviour change to the produced binaries; takes effect on the next tag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:06:04 +01:00
UltradesuandClaude Opus 4.8 6438982d45 Upload release assets via the Gitea API instead of artifacts + gh
Release / tsunagi-linux-aarch64 (push) Failing after 2m57s
Release / tsunagi-linux-x86_64 (push) Failing after 2m53s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
The release ran on Gitea Actions, where it failed at actions/upload-artifact@v4:
the v4 artifact backend (and the GitHub REST API that `gh release create` uses)
are not available on Gitea/GHES, so both the artifact handoff and the publish
job were dead ends there.

Reworked so each build job uploads its archive straight to the release through
Gitea's native API (`/api/v1/repos/<repo>/releases`) with curl: create the
release for the tag if absent (races between matrix jobs are resolved by reading
the id back by tag), then POST the archive as an asset. The separate publish
job, download-artifact and `gh` are gone. The id is parsed with grep, so the
runner needs only curl and coreutils — no jq, no third-party release action.
Uses the Gitea-injected GITHUB_TOKEN, GITHUB_SERVER_URL and GITHUB_REPOSITORY;
a tag with a pre-release suffix (e.g. -rc.2) is marked prerelease.

This targets Gitea by design; it no longer runs as-is on github.com (different
API), which is fine — the GitHub build is being retired. Build and package
steps, the musl/zig setup and the target matrix are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:04:20 +01:00
UltradesuandClaude Opus 4.8 21b619388c Pin Zig to a real release so the musl release build fetches reliably
Release / tsunagi-linux-aarch64 (push) Failing after 3m49s
Release / tsunagi-linux-x86_64 (push) Failing after 3m26s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
The release run failed at "Install zig": mlugg/setup-zig, left unpinned, resolved
to the newest version it knew of — an unreleased 0.16.0 whose tarball 404s on
ziglang.org — and the community mirrors returned 502/503 around it, so the step
errored out before cargo-zigbuild ever ran.

Pin the action to Zig 0.14.1, an archived release with a permanent official
download URL. The official source then succeeds even when the mirrors are flaky,
and the version stays compatible with cargo-zigbuild. Also moves the action to
@v2.

Only the Linux musl jobs use zig; macOS and Windows are unaffected. Not run on a
real tag yet in this session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:54:29 +01:00
UltradesuandClaude Opus 4.8 d1ab993dd2 Add a tagged-release CI workflow and Linux/Windows packaging
Release / tsunagi-linux-aarch64 (push) Failing after 3m12s
Release / tsunagi-linux-x86_64 (push) Failing after 2m21s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Pushing a `v*` tag now builds the agent for every supported target and attaches
the archives to a GitHub release. Modelled on the furumi release workflow (a
build matrix, then a publish job that runs `gh release create`), adapted to this
project's two differences: it needs system integration (capabilities/polkit on
Linux, elevation+wintun on Windows, a root daemon on macOS), and its users run
varied distros, not just Ubuntu.

Linux is built as a fully **static musl** binary with cargo-zigbuild, so one
artifact runs on Arch, Alpine, NixOS and old-glibc systems alike, and can be
repackaged into whatever a distro prefers later. musl has no prebuilt
aws-lc-sys bindings, so the Linux jobs install cmake + libclang (bindgen) and
zig (the cross C compiler zigbuild drives). macOS (aarch64) and Windows (x86_64)
build natively; Linux covers x86_64 and aarch64.

Targets: x86_64/aarch64 linux-musl, aarch64 macOS, x86_64 Windows. Each archive
carries the binary, the top-level README, and the platform's install notes plus
integration files:
- dist/linux/: a static-friendly README, a systemd unit (runs as root, so no
  capability grant or polkit rule is needed; a commented non-root variant is
  included), and an install.sh that drops the binary and the unit or prints the
  setcap/polkit alternative.
- dist/windows/: install notes covering wintun.dll and Run-as-administrator.
- dist/macos/: the existing LaunchDaemon plist and notes, now also shipped in
  the archive.

Not exercised yet: the workflow only runs on a real tag push, and the
aws-lc-sys-under-musl build has not been run locally in this session (no zig
toolchain here). YAML and install.sh syntax are validated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:48:19 +01:00
UltradesuandClaude Opus 4.8 da7ce1dca1 Drop link-local addresses from the status "not announced" list
A host with several interfaces has a link-local (fe80::/10, and 169.254/16 on
IPv4) on each, and none of them is ever a peer candidate. Listing them under
"not announced" padded the section with one noise line per interface — most of
the list on a machine with several VPN utuns. They are now filtered out;
loopback and real unannounced addresses are unaffected.

Checks: cargo fmt --all --check and cargo clippy -p tsunagi-cli --all-targets
-D warnings pass; verified by eye against a live `tsunagi status`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:38:21 +01:00
UltradesuandClaude Opus 4.8 4d2f8a54ec Compact the status address list to one line per address
`tsunagi status` printed every local address as two lines: the address, then
an indented note holding the interface (and, for candidates, a priority rank).
On a host with many interfaces — common on macOS, where each VPN adds a utun
and every interface carries a link-local — that made the "local addresses"
section very long and hard to scan.

Each address is now a single line: the announce status stays the label
(announced / not announced / loopback) and the address and its interface share
the detail, as `<addr> · <iface>`. The per-row note and the candidate-priority
rank are dropped; ordering still reflects candidate priority. No other section
changes, and nothing but the presentation does.

Checks: cargo fmt --all --check, cargo clippy --locked --workspace
--all-targets -D warnings, and the tsunagi-cli tests all pass on macOS;
verified by eye against a live `tsunagi status`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:26:13 +01:00
UltradesuandClaude Opus 4.8 c3a6201ee2 Bring the macOS stack up: utun interface, resolver files, system paths
macOS was TUN-less: the agent ran only with `--no-tun`, so tunnels formed but
nothing reached the operating system. The provisioner, host rules and DNS
publisher had Linux and Windows implementations and fell through to the
unsupported/erroring paths on macOS. This adds the macOS half so `tsunagi up`
carries real traffic there. The shared decision logic (plan_changes,
rules_summary, Published) is untouched — only the per-platform mechanics are new.

What it does on macOS (all root-only; macOS has no per-capability grant):
- Interface: a `utun` device opened over the in-kernel utun control socket via
  the existing `tun` dependency (no kext, no Network Extension). The kernel
  assigns the `utunN` name, so a name derived from the network id is not a legal
  utun name; `open_tun` leaves the name unset on macOS and reads the real one
  back, and `Interface::start` adopts `device.name()` as the name everything
  downstream configures (host rules, reconfigure). A utun is torn down with the
  process, so a crash never leaks one. Addresses, MTU, up and the overlay subnet
  route are applied with `/sbin/ifconfig` and `/sbin/route` (absolute paths, args
  built only from locally-derived values), the same role netlink plays on Linux
  and netsh on Windows. Observation is `netdev` (getifaddrs).
- DNS: a resolver-dir publisher writes `/etc/resolver/<network>` files pointing
  the system resolver at the overlay DNS server for each zone. The files carry a
  `port`, so the server needs no privileged socket, and a marker line, so the
  publisher only ever replaces or removes files it wrote and never clobbers a
  user's own. Removed on revert.
- Broadcast host rules: the `255.255.255.255` route is installed with `route`.
  The inbound-UDP firewall allowance (a pf anchor) is deferred and reported as
  the incomplete half, so `status` says what to allow by hand; the route still
  goes in. Failure stays contained, exactly as on the other platforms.
- Privilege: probe is optimistic (like Windows); the honest answer comes from
  the utun open, whose permission error now says to run with sudo or as a daemon.

Deployment:
- Default paths on macOS are fixed system locations rather than per-user Library
  dirs, so they do not depend on whose shell ran sudo: state `/var/db/tsunagi`,
  cache `/var/db/tsunagi/cache` (StoragePaths::user_default), control socket
  `/var/run/tsunagi/agent.sock` (default_control_socket_path, mirroring the
  Windows %ProgramData% branch). The local control socket already uses the
  shared `unix` adapter, so no IPC changes were needed — macOS is Unix.
- dist/macos/ carries a root LaunchDaemon plist and install notes; README's
  deployment section now documents the macOS path instead of "not implemented".

Decisions (agreed up front): utun auto-assign with the real name adopted;
ifconfig/route rather than raw PF_ROUTE/ioctl (which would need unsafe);
/etc/resolver files rather than scutil; route-now/firewall-deferred; root via
sudo or a LaunchDaemon rather than Network Extension (which would require a
signed sandboxed app bundle, not a Rust CLI).

Checks: on macOS (Darwin 27), `cargo fmt --all --check`, `cargo clippy --locked
--workspace --all-targets -D warnings`, and `cargo test --locked --workspace
--all-targets` all pass (0 failures); the new macos-gated unit tests run under
the standard workspace command via feature unification. Not yet exercised live:
actual utun creation, ifconfig/route application and /etc/resolver writes need a
root run, which has not been done in this session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:15:59 +01:00
Aleksandr Bogomiakov 533921059a Install broadcast route and firewall allowance on the host while broadcast is on
LAN discovery (CS 1.6 and similar) only worked after two manual steps on every
host: route 255.255.255.255 through the TUN, and allow inbound traffic on it.
Without the route the game's limited broadcast leaves via the physical adapter
and never reaches the overlay; without the firewall allowance the server's
unicast reply (from a different address than the one the request went to, so
never matched by conntrack) is dropped before the game's socket. Verified by
hand on Linux: with both in place the server answered A2S_INFO over the overlay.

The system level (not a plugin) now installs exactly those two things for the
interface it owns, and removes them when broadcast is turned off, the network
is left or stopped, or the agent exits.

Design:
- New boundary overlay::hostrules (BroadcastHostRules: idempotent apply/clear,
  with Linux, Windows, Mock and Unsupported implementations). TunFactory gains
  host_rules(); ManagedTunFactory::with_host_rules wires it in the CLI.
- Interface::sync_broadcast_rules reconciles the host against the routing
  table (RoutingTable::broadcast_host_target) and is called after address sync,
  on live `network broadcast on/off`, on network deactivation and by remove().
  Failure is contained: the outcome is kept and logged, never fatal.
- One interface carries several networks but the host has one 255.255.255.255
  route, so the source is the lowest NetworkId among broadcast-enabled networks
  with an address (documented in docs/broadcast.md).
- Linux: the route is added on the existing netlink thread (preferred source =
  overlay address, replace semantics). The firewall rule is `iptables -I INPUT
  -i <if> -p udp -s <overlay range>` tagged with a comment `tsunagi:<if>`;
  it is UDP-only and range-limited on purpose so TCP services stay unexposed.
  iptables is located in fixed system directories, never via PATH; stale tagged
  rules (e.g. from a crash) are replaced. A missing iptables leaves the route in
  place and is reported. rp_filter is not touched.
- Windows: PowerShell (absolute path) New-NetRoute metric 1 and a tagged
  inbound UDP Defender rule scoped to the interface alias and overlay range.
- status shows a `broadcast rules` row (ok, or which half failed and why).
  IPC control protocol bumped 14 -> 15 because OverlayReport gained a field.
- AGENTS.md records the narrow exception to "never touch routing/firewall".

Compatibility: restart the agent after upgrading (control protocol 15). No wire
or schema change.

Checks run (Linux only): cargo fmt --check, cargo clippy --locked --workspace
--all-targets -D warnings, cargo test --locked --workspace --all-targets, all
passing. Tests use a mock host and a scripted iptables; no OS setting is
changed. NOT done: the Windows code was never compiled for or run on Windows
(cross-compiling failed for lack of a C toolchain), only its script building is
unit-tested; the real netlink route and iptables calls were not exercised
against a live kernel in the suite; CS 1.6 was not launched from this change.
2026-10-01 15:03:43 +01:00
Aleksandr Bogomiakov d2778f0b5a Prevent WireGuard timer loop failure spam and fix handshake broadcast
When a WireGuard tunnel handshake timed out or a rekey failed,
Inner::on_tunnel_failed closed the transport link and reported the failure
to the agent core. However:
1. The peer remained in WireguardDevice::inner.peers. Once boringtun
   declared ConnectionExpired (after 90s), self.tunn.is_expired() remained
   permanently true on all subsequent 250ms timer ticks. drive_timers
   repeatedly detected failed = true every 250ms, flooding logs with
   'WARN WireGuard tunnel failed due to rekey timeout, triggering fallback'
   and sending repeated protocol failure events to the core 4 times/sec.
2. In WireguardLink::send, is_handshake_init incorrectly expected an
   envelope header offset (msg[74] == 1) when checking for initiation
   packets, which failed on raw WireGuard UDP packets. As a result,
   handshake initiations were not broadcast to all candidate endpoints.
3. In WireguardTransport::accept_loop, packet source matching contained
   dead envelope destination matching inspecting ephemeral key bytes
   against local endpoint IDs.

Fixes:
- In crates/tsunagi-wireguard/src/device.rs:
  - Add failed: bool to FlowTunnel to latch failure state. Once a failure
    is reported, update_timers returns (TunnResult::Done, false) on all
    subsequent ticks until a new handshake is started or completed.
  - In Inner::on_tunnel_failed, remove the failed peer from inner.peers so
    drive_timers ceases ticking on the dead tunnel.
- In crates/tsunagi-wireguard/src/plugin.rs:
  - In Worker::sync, retain only active unclosed links in state.links.
- In crates/tsunagi-wireguard/src/transport.rs:
  - Check is_handshake_init directly as '!msg.is_empty() && msg[0] == 1',
    ensuring handshake initiations are broadcast to all candidate addresses.
  - Clean up matched_peer to look up links and peer candidate addresses directly.
- Tests:
  - Update test_flow_tunnel_rekey_timeout_triggers_failure to assert failure
    latching and verify no repeated failure reports on subsequent ticks.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed.
- cargo build --release passed.
2026-10-01 03:18:32 +01:00
Aleksandr Bogomiakov 8cfece9479 Filter noisy internal warnings from boringtun and iroh net_report
The CLI's default tracing filter previously allowed WARN level logs from
boringtun and iroh, producing two confusing and noisy log messages during
routine operation:
1. 'WARN HANDSHAKE(REJECT_AFTER_TIME - KEEPALIVE_TIMEOUT - REKEY_TIMEOUT (on receive))':
   emitted by boringtun's internal timer tick when an endpoint primarily
   receives data for 165 seconds (180s - 10s - 5s). In boringtun, this is a
   routine scheduled rekey timer rather than an error or failure, but was
   unhelpfully logged as warn! upstream.
2. 'WARN probe timed out':
   emitted by iroh's net_report module during background HTTP captive portal
   detection when running in isolated subnets or VMs without direct internet
   access. This probe has no impact on mesh connectivity.

Update the default logging filter to:
  'info,iroh=warn,iroh::net_report=error,quinn=warn,rustls=warn,boringtun=error,mainline=off'

Real WireGuard handshake and rekey failures continue to be detected and
reported cleanly by Tsunagi's own data plane diagnostics and automatic
fallback handlers at INFO and WARN levels.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed.
- cargo build --release passed.
2026-10-01 03:06:20 +01:00
Aleksandr Bogomiakov 3930dff2dc Detect WireGuard tunnel rekey failure and trigger automatic fallback
Previously, because UDP is connectionless, when a pure WireGuard tunnel
encountered handshake or rekey failures (e.g. boringtun logging
`WARN HANDSHAKE(REKEY_TIMEOUT)` and `CONNECTION_EXPIRED`), the
`WireguardLink` never closed. It remained active in `agent.links` and
as a direct hop in `RelayHub.raw`. Consequently, outgoing packets
continued to be blackholed over the dead direct path, preventing
`RelayHub` from using working multihop relay routes and preventing
the agent from falling back to alternative protocols (`tcp-tls` or
`wg-quic`).

Key changes:
- In `crates/tsunagi-wireguard/src/device.rs`:
  - In `FlowTunnel`, track active handshake and rekey attempts via
    `handshake_started_at`. If an active handshake attempt remains
    unanswered beyond `HANDSHAKE_TIMEOUT` (15s) or boringtun reports
    `ConnectionExpired`, declare the tunnel dead.
  - Reset timeout timers upon successful handshake reception in
    `decapsulate`.
  - In `drive_timers`, when tunnel failure is detected, trigger
    `inner.on_tunnel_failed(peer)`, which closes the direct transport
    link and notifies the agent via `PluginContext::report_peer_protocol_failure`.
- In `crates/tsunagi-wireguard/src/transport.rs`:
  - Add `WireguardLink::close` to transition `is_closed` to true and
    notify `closed_notify`.
  - In `WireguardLink::recv`, select on `closed_notify` so readers
    unblock and return `None` immediately on close.
  - Add `WireguardTransport::close_link` to close active links by
    network and peer.
- In `crates/tsunagi/src/dataplane/mod.rs`:
  - Add `PluginRequest::ProtocolFailure` and public helper
    `PluginContext::report_peer_protocol_failure`.
- In `crates/tsunagi/src/agent/mod.rs` and `crates/tsunagi/src/agent/network.rs`:
  - Add `NetCommand::ProtocolFailure` to runtime command handling.
  - On protocol failure: remove dead direct link from `self.links`,
    clear it from `RelayHub.raw` (so `RelayHub` immediately routes
    over available multihop relays), record failure on `wg` for that
    peer with exponential backoff cooldown, emit `DataLinkDown`, and
    immediately call `ensure_links()` to dial the next mutually supported
    protocol (`tcp-tls` or `wg-quic`).
- Tests:
  - Add `test_flow_tunnel_rekey_timeout_triggers_failure` and
    `test_flow_tunnel_reset_on_handshake` unit tests in `device.rs`.
  - Add `test_wireguard_link_close_and_transport_close_link` in
    `transport_tests.rs`.
  - Add `test_protocol_dial_state_cooldown_and_fallback_selection` in
    `network.rs`.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
- Built release binary: target/release/tsunagi.
2026-10-01 03:02:09 +01:00
Aleksandr Bogomiakov 35acc4d231 Fix WireGuard relay preemption with UDP probing, allow VM bridges, and categorize status addresses
When remote peers communicate across networks with mixed connectivity (e.g.
a remote Linux node, a local Linux host, and a Windows VM guest under KVM):
- Previously, `WireguardTransport` immediately created and installed
  unverified direct links into `RelayHub` in `notify_peer_capability` and
  `open()`. When a remote peer had only unroutable private addresses
  (e.g. 192.168.122.151 on a KVM NAT bridge), this dead direct link
  preempted the working 2-hop relay, causing packet drops, failed handshakes,
  and REKEY_TIMEOUT errors.
- Host-guest VM bridges (`virbr*`) were previously filtered out, preventing
  the VM guest from discovering the host's bridge IP (192.168.122.1) and
  forcing it to attempt cross-interface connections to host Wi-Fi/LAN.
- Concurrent activation of multiple networks sharing one `tsun0` interface
  raced in `sync_addresses`, resulting in netlink `File exists (os error 17)`
  and `The object already exists` on Windows.

Key fixes:
- In `crates/tsunagi-wireguard/src/transport.rs`:
  - Implement bidirectional UDP probing with `TSUNAGI_WG_PROBE_V1` PING/PONG.
  - In `open()`, probe candidate endpoints with retransmission up to 1200ms.
    If unreachable, return `TransportError::Unreachable`, preventing dead
    links from preempting valid relay routes in `RelayHub`.
  - In `notify_peer_capability`, send hole-punching probes without prematurely
    installing unverified links.
  - In `accept_loop`, reply to PING probes with PONG and install confirmed
    inbound links targeting the verified source address.
  - In `WireguardLink::send`, inspect byte 74 of enveloped packets to detect
    handshake initiations, ensuring confirmed links route data packets only
    to the active target.
- In `crates/tsunagi-wireguard/src/plugin.rs`:
  - Permit VM bridges (`virbr*`) as candidate interfaces (priority 10).
  - Add `prioritize_local_subnet` to prioritize candidates sharing a local
    IPv4 subnet with the receiver (e.g. 192.168.122.0/24).
- In `crates/tsunagi/src/overlay/interface.rs` & provisioners:
  - Add async mutex `sync_lock` to serialize concurrent address syncs across networks.
  - Handle netlink EEXIST (os error 17) and Windows already-exists errors
    idempotently as success.
- In `crates/tsunagi-cli/src/main.rs`:
  - Split `local addresses` in `tsunagi status` into `announced` (candidate
    addresses in priority order with interface and rank), `not announced`
    (container bridges/internal addresses with interface name), and `loopback`.
- Tests:
  - Add `test_wireguard_transport_unreachable_probe_times_out` and
    `test_matches_subnet` unit tests.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
2026-10-01 02:38:13 +01:00
Aleksandr Bogomiakov c791102ad9 Fix overlay and virtual interface candidate leakage and WireGuard rekeying
On Windows (Wintun), iface.name in netdev is an adapter GUID, while
iface.friendly_name is 'tsun0'. Checking only iface.name failed to
identify and exclude Wintun overlay interfaces. Consequently, Windows
nodes announced their own overlay IP addresses (e.g. 10.13.37.26) as
candidate endpoints. Remote peers latched onto this overlay IP as the
WireGuard target, routing encrypted packets into tsun0 (recursive loop),
resulting in failed handshakes and subsequent HANDSHAKE(REKEY_TIMEOUT)
and CONNECTION_EXPIRED.

Similarly, Docker bridges (br-*, docker0) and virtual bridges were
announced and alphabetically sorted before physical LAN or Tailscale
interfaces, causing peers to attempt WireGuard handshakes with internal
Docker bridge addresses (e.g. 172.21.0.1).

Key changes:
- In crates/tsunagi-wireguard and crates/tsunagi-tcp-tls:
  - Robustly identify overlay interfaces by inspecting iface.name,
    iface.friendly_name, and iface.description for 'tsun', 'tsunagi',
    and 'wintun'.
  - Filter out container and virtual bridge interfaces (docker, br-,
    veth, virbr, cni, flannel, cali, kube, dummy, lxc, podman, vethernet,
    vboxnet, vmnet).
  - Prioritize candidates by physical interface score: interfaces with
    a default gateway (physical LAN/WAN) and Tailscale (100.64.0.0/10)
    score highest and sort to candidate #0.
  - In on_peer_capability, filter out any announced address falling inside
    any active overlay IPv4 range (defense in depth on receiver).
- In crates/tsunagi-wireguard/src/transport.rs:
  - In WireguardLink::send, broadcast Handshake Initiation packets (type 1)
    to all candidate addresses so initial handshakes and rekeys probe all
    candidate paths and recover if an endpoint roamed or NAT expired.
  - In accept_loop, prioritize exact socket address matches before falling
    back to candidate IP matches for NAT port roaming.
- In crates/tsunagi/src/net.rs:
  - Update iroh AddrFilter to inspect friendly_name and description,
    preventing Wintun overlay IPs from leaking into iroh discovery endpoints.
- Tests:
  - Add unit tests for candidate interface filtering, IP validation, and
    priority scoring in tsunagi-wireguard and tsunagi-tcp-tls.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
2026-10-01 02:15:06 +01:00
Aleksandr Bogomiakov f80841da63 Fix WireGuard multi-network demux, candidate discovery, and status reporting
When a peer belongs to multiple overlay networks served by one agent (e.g.
a VM participating in both 'hexor' and 'ultradesu'), packets received on
the shared WireGuard UDP socket could not be differentiated by source IP/port
alone. Previously, `WireguardTransport::accept_loop` picked the first
matching link found in iteration order, starving other networks for that peer
and causing rekey timeouts (`WARN HANDSHAKE(REKEY_TIMEOUT)`,
`ERROR CONNECTION_EXPIRED(REKEY_ATTEMPT_TIME)`) and packet loss on the
starved network.

Additionally:
- When a peer had multiple candidate endpoints (such as LAN, VM bridge, and
  Tailscale CGNAT addresses), `WireguardLink` sent handshake initiations
  only to `addrs[0]`. If the first address was unreachable from the remote
  side (e.g. host's Tailscale IP unreachable from inside a local VM), the
  handshake never completed.
- In `build_report`, peers known to the `wg-quic` plugin were added with
  empty tunnels even if an active `wg` (pure WireGuard) tunnel existed,
  causing `tsunagi status` to falsely report 'no WireGuard handshake yet'
  for connected and pingable peers.
- `local_ip_candidates` previously included IPv6 addresses which caused
  OS error 22 (EINVAL) when passed to the IPv4-bound UDP socket.

Key fixes:
- In `crates/tsunagi-wireguard/src/transport.rs`:
  - Store `peer_networks` as `HashSet<NetworkId>` per peer.
  - In `WireguardLink`, maintain a list of candidate addresses and a
    `confirmed` flag. Before a handshake response confirms the active path,
    transmit datagrams (handshake initiations) to all candidate addresses.
    Upon receiving an inbound datagram, lock onto the confirmed address.
  - In `accept_loop`, deliver inbound datagrams to all active links
    associated with the sender's endpoint ID across all networks. Boringtun's
    cryptographic index and AEAD verification naturally demultiplexes and
    accepts the packet in the correct network while discarding it in others.
  - Filter candidate addresses to IPv4 to match the bound socket family.
- In `crates/tsunagi-wireguard/src/plugin.rs`:
  - Only collect IPv4 interfaces in `local_ip_candidates`.
- In `crates/tsunagi-cli/src/main.rs`:
  - Refactor `build_report` to evaluate all protocols per peer and prioritize
    whichever protocol has an active tunnel (`wg` > `tcp-tls` > `wg-quic`).
- In `crates/tsunagi-wireguard/tests/plugin_tests.rs`:
  - Add `test_wireguard_two_networks_same_peers` verifying simultaneous
    bidirectional packet transmission between two peers sharing two networks
    over a single `WireguardTransport`.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
2026-10-01 01:38:44 +01:00
Aleksandr Bogomiakov 7070a16104 Add pure WireGuard direct UDP data plane plugin and transport
Peers communicating on uncensored networks can benefit from raw WireGuard
performance over direct UDP without QUIC framing or double-encryption
overhead. This provides Tailscale-like direct tunnel performance while
preserving Tsunagi's decentralized mesh architecture.

Key changes:
- Create `tsunagi-wireguard` crate implementing the `IpPlugin` and
  `PacketTransport` contracts:
  - `WG_PROTOCOL = "wg"` with wire version 1.
  - Bounded `WgAnnouncement` carrying WireGuard public keys, listening UDP
    ports, and candidate IP addresses (filtering out overlay interfaces).
  - `WgKeyStore` managing persistent per-network WireGuard X25519 keys
    in SQLite with zeroization on drop.
  - `WireguardTransport` providing direct UDP packet transmission, candidate
    address tracking, inbound link delivery, and roaming support.
  - `WireguardDevice` running userspace WireGuard encryption via `boringtun`,
    timer maintenance, and AllowedIPs source enforcement.
- Integrate `wg` into `tsunagi-cli`:
  - Set default protocols to `"wg,tcp-tls,wg-quic"` so all protocols are
    enabled by default in order of throughput and performance.
  - Support `"wireguard"` as a convenience alias for `"wg"` in `--protocol`.
  - Bind `PureWgTransport` and launch the inbound datagram accept loop on `up`.
  - Pass `pure_wg` plugin to `AgentControl` and update `build_report` so
    active WireGuard tunnels and overlay metrics are reported in `tsunagi status`.
  - Clean up WireGuard keys on network removal in `forget_protocol_state`.
- Comprehensive test coverage in `tsunagi-wireguard`:
  - `announcement_tests`: round-trip serialization and verification of malformed,
    mismatched network, zero port, or self-claiming announcements.
  - `transport_tests`: loopback UDP link delivery between transports.
  - `plugin_tests`: end-to-end two-peer handshake and bidirectional IP packet
    transit over direct UDP.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed.
2026-10-01 01:24:25 +01:00
Aleksandr Bogomiakov 46575d0899 Prioritize stored agreed range over default range in reserve_range
When an agent restarted with multiple networks (e.g. hexor and ultradesu),
Agent::reserve_range() previously attempted to reserve the default configured
range (10.13.37.0/24) for whichever network happened to be listed first in
storage, even if that network had already agreed upon a different range
(such as 10.79.175.0/24). As a result, the first network seized 10.13.37.0/24
in the route table, causing the second network (which genuinely used 10.13.37.0/24)
to trigger a spurious RouteError::Overlap warning at startup until the first
network asynchronously loaded its signed records and released the reservation.

Key changes:
- In crates/tsunagi/src/agent/mod.rs, make reserve_range asynchronous and
  inspect stored signed records first using StateSet::agreed_range().
- If an agreed range already exists in storage, reserve that specific range
  in routes immediately instead of preemptively claiming the default
  overlay_ipv4_range.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets --jobs 2 -- --test-threads=2 passed (349 tests passed).
2026-10-01 00:36:24 +01:00
Aleksandr Bogomiakov a89e65475d Implement Debug manually for PipeSecurityAttributes on Windows
The windows_sys::Win32::Security::SECURITY_ATTRIBUTES struct is a raw C FFI
struct that does not implement std::fmt::Debug. Deriving Debug on
PipeSecurityAttributes failed Windows compilation with E0277.

Key changes:
- In crates/tsunagi/src/ipc/windows.rs, remove #[derive(Debug)] on
  PipeSecurityAttributes and provide an explicit std::fmt::Debug
  implementation formatting nLength, lpSecurityDescriptor, and
  bInheritHandle.

Validation on Linux:
- Verified type check against Windows target via rustc.
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets --jobs 2 -- --test-threads=2 passed (349 tests passed).
2026-10-01 00:31:21 +01:00
Aleksandr Bogomiakov 2ed447ae86 Filter overlay interface from candidates and align status member columns
Peers previously advertised overlay interface (tsun0) addresses as physical
candidate endpoints for both iroh discovery and tcp-tls announcements. When an
intermediate tunnel was active between two peers (e.g., between a host and a VM),
iroh probes to the remote peer's overlay address were routed through the existing
overlay, causing iroh to select the overlay IP itself as a 'direct' endpoint.
This caused recursive tunneling (QUIC over WireGuard over TLS) and confusing
status output showing peers connecting to their own overlay IP addresses.
Additionally, in tsunagi status, peer rows were displayed with unaligned
columns, mixing control plane status and data plane metrics in a way that was
difficult to scan.

Key improvements:
- In crates/tsunagi/src/net.rs, add an AddrFilter to the iroh EndpointBuilder
  to dynamically filter out any local addresses assigned to tsun* overlay
  interfaces before publishing to address discovery services (DHT/pkarr/STUN).
- In crates/tsunagi-tcp-tls/src/plugin.rs, exclude tsun* interfaces from
  local_ip_candidates() so overlay IPs are never advertised as physical
  TCP endpoints.
- In crates/tsunagi-cli/src/main.rs, exclude tsun* interfaces from
  netwatch_addresses() so the local addresses section reflects only actual
  physical/host network interfaces.
- In crates/tsunagi-cli/src/main.rs, update member_row() to format member
  details with aligned columnar layout: the overlay IP is placed first and
  padded, followed by control plane transport and latency, and line 2 pads
  tunnel metrics so that the data link endpoint (via ...) aligns vertically.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets --jobs 2 -- --test-threads=2 passed (349 tests passed).
2026-10-01 00:24:53 +01:00
Aleksandr Bogomiakov ac5a1596ef Allow unelevated Windows processes to access control socket via SDDL and shared path
When running the tsunagi agent as an elevated Administrator or Windows Service,
unprivileged user-space processes (such as a system tray application, user-level
CLI commands, or status monitors) could not interact with the control socket.
This was due to two constraints on Windows:
1. The default control socket path was derived from the per-user %APPDATA%,
   making an administrator's socket invisible or inaccessible to a non-elevated
   standard user profile.
2. Even if pointed to the same named pipe, Windows Mandatory Integrity Control (MIC)
   and default ACLs prevent a Medium-Integrity (unelevated) process from opening
   a High-Integrity (elevated) pipe server for write access unless explicitly allowed.

Key changes:
- In crates/tsunagi/src/ipc/windows.rs, implement PipeSecurityAttributes using
  Win32 ConvertStringSecurityDescriptorToSecurityDescriptorW with SDDL:
  "D:(A;;GRGW;;;WD)(A;;GA;;;BA)(A;;GA;;;SY)S:(ML;;NW;;;ME)". This grants generic
  read/write access to Everyone (WD), retains full control for Administrators (BA)
  and LocalSystem (SY), and sets the Mandatory Label SACL to Medium Integrity (ME)
  with No-Write-Up (NW).
- In crates/tsunagi/src/ipc/mod.rs, introduce default_control_socket_path() which
  points to %ProgramData%\tsunagi\agent.sock on Windows (and default per-user
  state directory on Unix).
- In crates/tsunagi-cli/src/main.rs, support TSUNAGI_CONTROL_SOCKET environment
  variable across all commands, and use default_control_socket_path() on Windows
  when running against the default state directory. Custom state directories (--state-dir)
  or explicit sockets (--control-socket) continue to take precedence, allowing
  multi-daemon setups to remain strictly isolated.
- In workspace Cargo.toml, set unsafe_code = "deny" so the workspace forbids unsafe
  code while permitting explicit, documented unsafe Win32 API calls in windows.rs.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets --jobs 2 -- --test-threads=2 passed (349 tests passed).
2026-10-01 00:13:44 +01:00
53 changed files with 8627 additions and 246 deletions

No files matched your search

+215
View File
@@ -0,0 +1,215 @@
name: Release
# Build the agent for every supported target on a version tag, leave each build
# as a run artifact, and then — only after every target has built — attach the
# archives to the release. Linux is a fully static musl binary (built with
# cargo-zigbuild) so one artifact runs on any distro and can be repackaged
# however a distro prefers.
#
# Targeted at Gitea Actions: artifacts use @v3 (the v4 backend is not available
# on Gitea/GHES), and the release is filled through Gitea's own API with curl
# rather than `gh`, which speaks only to github.com.
on:
push:
tags: ["v*"]
permissions:
contents: write
env:
CARGO_TERM_COLOR: always
CARGO_NET_RETRY: "10"
jobs:
build:
name: ${{ matrix.asset }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
asset: tsunagi-linux-x86_64
archive: tar.gz
zig: true
- os: ubuntu-latest
target: aarch64-unknown-linux-musl
asset: tsunagi-linux-aarch64
archive: tar.gz
zig: true
- os: macos-latest
target: aarch64-apple-darwin
asset: tsunagi-macos-aarch64
archive: tar.gz
zig: false
- os: windows-latest
target: x86_64-pc-windows-msvc
asset: tsunagi-windows-x86_64
archive: zip
zig: false
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.target }}
# aws-lc-sys (iroh's rustls backend) and the bundled sqlite are C, and the
# musl targets have no prebuilt bindings, so bindgen needs libclang and the
# C build needs cmake. zig is the cross C compiler cargo-zigbuild drives.
- name: Install musl build tooling
if: matrix.zig
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends cmake clang libclang-dev
- name: Install zig
if: matrix.zig
uses: mlugg/setup-zig@v2
with:
# Pin a real, archived Zig release. Left unpinned the action resolves
# to the newest version it knows of (a yet-unreleased 0.16.0-dev),
# whose tarball 404s on ziglang.org while the community mirrors flake.
version: 0.14.1
- name: Install cargo-zigbuild
if: matrix.zig
uses: taiki-e/install-action@v2
with:
tool: cargo-zigbuild
- name: Build
shell: bash
run: |
set -euo pipefail
if [[ "${{ matrix.zig }}" == "true" ]]; then
# zig's linker ignores the `-O1` GNU-ld optimization flag Rust passes
# and says so on stderr, which rustc now surfaces as a linker_messages
# warning. It is cosmetic — the binary is correct — so quiet just that
# one lint to keep the musl build log clean. A real link error still
# fails the build.
export RUSTFLAGS="${RUSTFLAGS:-} -A linker_messages"
cargo zigbuild --release --locked --target "${{ matrix.target }}" --bin tsunagi
else
cargo build --release --locked --target "${{ matrix.target }}" --bin tsunagi
fi
- name: Package
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
stage="dist/pkg/${{ matrix.asset }}"
mkdir -p "$stage"
bin="tsunagi"
[[ "${{ runner.os }}" == "Windows" ]] && bin="tsunagi.exe"
cp "target/${{ matrix.target }}/release/${bin}" "$stage/"
# The top-level README, plus the per-OS install notes and integration
# files, so the archive carries what the platform needs to set up.
cp README.md "$stage/"
case "${{ runner.os }}" in
Linux)
cp dist/linux/tsunagi.service dist/linux/install.sh "$stage/"
cp dist/linux/README.md "$stage/INSTALL.md"
;;
macOS)
cp dist/macos/cy.hexor.tsunagi.plist "$stage/"
cp dist/macos/README.md "$stage/INSTALL.md"
;;
Windows)
cp dist/windows/README.md "$stage/INSTALL.md"
;;
esac
archive="${{ matrix.asset }}-${version}.${{ matrix.archive }}"
if [[ "${{ matrix.archive }}" == "zip" ]]; then
(cd dist/pkg && 7z a "../../${archive}" "${{ matrix.asset }}")
else
tar -C dist/pkg -czf "${archive}" "${{ matrix.asset }}"
fi
# @v3 on purpose: Gitea does not support the v4 artifact backend. This is
# what makes the archive show up under the run and hands it to `publish`.
- name: Upload the archive as a run artifact
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.asset }}
path: ${{ matrix.asset }}-*.*
if-no-files-found: error
publish:
name: Publish release
needs: build
runs-on: ubuntu-latest
steps:
- name: Download every build artifact
uses: actions/download-artifact@v3
with:
path: artifacts
- name: Show what was collected
run: find artifacts -type f
- name: Create the release and upload the archives
shell: bash
env:
# Gitea injects this with access to the current repository. If the
# release calls below return 403, the repo/runner is configured to
# hand Actions a read-only token — grant it write access to releases.
TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -u
tag="${GITHUB_REF_NAME}"
# Gitea sets GITHUB_API_URL to its own API root (…/api/v1); fall back
# to building it from the server URL if it is absent.
base="${GITHUB_API_URL:-${GITHUB_SERVER_URL}/api/v1}"
api="${base}/repos/${GITHUB_REPOSITORY}"
auth="Authorization: token ${TOKEN}"
echo "release API: $api"
prerelease=false
case "$tag" in *-*) prerelease=true ;; esac
# The release id for this tag, if it already exists.
code=$(curl -sS -o resp.json -w '%{http_code}' -H "$auth" "$api/releases/tags/$tag" || true)
echo "get release by tag: HTTP $code"
id=$(grep -o '"id":[0-9]*' resp.json 2>/dev/null | head -n1 | cut -d: -f2 || true)
if [ -z "$id" ]; then
code=$(curl -sS -o resp.json -w '%{http_code}' -X POST -H "$auth" \
-H 'Content-Type: application/json' \
-d "{\"tag_name\":\"$tag\",\"name\":\"tsunagi $tag\",\"prerelease\":$prerelease}" \
"$api/releases" || true)
echo "create release: HTTP $code"
id=$(grep -o '"id":[0-9]*' resp.json 2>/dev/null | head -n1 | cut -d: -f2 || true)
if [ -z "$id" ]; then
echo "could not create the release; response was:" >&2
cat resp.json >&2 || true
exit 1
fi
fi
echo "release id: $id"
# download-artifact@v3 nests each artifact in a directory named after
# its upload, with the single archive inside. The read loop runs in
# this shell (process substitution, not a pipe) so a failed upload
# actually fails the job.
rc=0
while IFS= read -r -d '' f; do
name=$(basename "$f")
code=$(curl -sS -o up.json -w '%{http_code}' -X POST -H "$auth" \
-F "attachment=@${f}" \
"$api/releases/$id/assets?name=${name}" || true)
echo "upload $name: HTTP $code"
if [ "$code" -lt 200 ] || [ "$code" -ge 300 ]; then
cat up.json >&2 || true
rc=1
fi
done < <(find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' \) -print0)
exit "$rc"
+187
View File
@@ -0,0 +1,187 @@
name: Release
# GitHub version of the release pipeline. The build half mirrors the Gitea one
# in .gitea/workflows/release.yml (static musl Linux via cargo-zigbuild, native
# macOS and Windows); only the publish half differs: GitHub needs the v4
# artifact backend and fills the release with `gh`.
#
# Every job is gated on `github.server_url == 'https://github.com'`, so this file
# runs only on GitHub. Gitea runs its own copy in .gitea/workflows; should a
# Gitea instance also read .github/workflows, the guard skips these jobs there
# so they never fail on the @v4 artifact backend or `gh`.
on:
push:
tags: ["v*"]
permissions:
contents: write
env:
CARGO_TERM_COLOR: always
CARGO_NET_RETRY: "10"
jobs:
build:
name: ${{ matrix.asset }}
if: ${{ github.server_url == 'https://github.com' }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
asset: tsunagi-linux-x86_64
archive: tar.gz
zig: true
- os: ubuntu-latest
target: aarch64-unknown-linux-musl
asset: tsunagi-linux-aarch64
archive: tar.gz
zig: true
- os: macos-latest
target: aarch64-apple-darwin
asset: tsunagi-macos-aarch64
archive: tar.gz
zig: false
- os: windows-latest
target: x86_64-pc-windows-msvc
asset: tsunagi-windows-x86_64
archive: zip
zig: false
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.target }}
# aws-lc-sys (iroh's rustls backend) and the bundled sqlite are C, and the
# musl targets have no prebuilt bindings, so bindgen needs libclang and the
# C build needs cmake. zig is the cross C compiler cargo-zigbuild drives.
- name: Install musl build tooling
if: matrix.zig
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends cmake clang libclang-dev
- name: Install zig
if: matrix.zig
uses: mlugg/setup-zig@v2
with:
# Pin a real, archived Zig release. Left unpinned the action resolves
# to the newest version it knows of (a yet-unreleased 0.16.0-dev),
# whose tarball 404s on ziglang.org while the community mirrors flake.
version: 0.14.1
- name: Install cargo-zigbuild
if: matrix.zig
uses: taiki-e/install-action@v2
with:
tool: cargo-zigbuild
- name: Build
shell: bash
run: |
set -euo pipefail
if [[ "${{ matrix.zig }}" == "true" ]]; then
# zig's linker ignores the `-O1` GNU-ld optimization flag Rust passes
# and says so on stderr, which rustc now surfaces as a linker_messages
# warning. It is cosmetic — the binary is correct — so quiet just that
# one lint to keep the musl build log clean. A real link error still
# fails the build.
export RUSTFLAGS="${RUSTFLAGS:-} -A linker_messages"
cargo zigbuild --release --locked --target "${{ matrix.target }}" --bin tsunagi
else
cargo build --release --locked --target "${{ matrix.target }}" --bin tsunagi
fi
- name: Package
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
stage="dist/pkg/${{ matrix.asset }}"
mkdir -p "$stage"
bin="tsunagi"
[[ "${{ runner.os }}" == "Windows" ]] && bin="tsunagi.exe"
cp "target/${{ matrix.target }}/release/${bin}" "$stage/"
# The top-level README, plus the per-OS install notes and integration
# files, so the archive carries what the platform needs to set up.
cp README.md "$stage/"
case "${{ runner.os }}" in
Linux)
cp dist/linux/tsunagi.service dist/linux/install.sh "$stage/"
cp dist/linux/README.md "$stage/INSTALL.md"
;;
macOS)
cp dist/macos/cy.hexor.tsunagi.plist "$stage/"
cp dist/macos/README.md "$stage/INSTALL.md"
;;
Windows)
cp dist/windows/README.md "$stage/INSTALL.md"
;;
esac
archive="${{ matrix.asset }}-${version}.${{ matrix.archive }}"
if [[ "${{ matrix.archive }}" == "zip" ]]; then
(cd dist/pkg && 7z a "../../${archive}" "${{ matrix.asset }}")
else
tar -C dist/pkg -czf "${archive}" "${{ matrix.asset }}"
fi
- name: Upload the archive as a run artifact
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.asset }}
path: ${{ matrix.asset }}-*.*
if-no-files-found: error
publish:
name: Publish release
needs: build
if: ${{ github.server_url == 'https://github.com' }}
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Download every build artifact
uses: actions/download-artifact@v4
with:
path: artifacts
- name: Show what was collected
run: find artifacts -type f
- name: Create the release and upload the archives
shell: bash
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ github.ref_name }}
run: |
set -euo pipefail
# download-artifact@v4 nests each artifact under its own directory, so
# the archives are artifacts/<asset>/<file>.
mapfile -t files < <(find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' \))
printf 'asset: %s\n' "${files[@]}"
prerelease=()
case "$TAG" in *-*) prerelease=(--prerelease) ;; esac
# Create the release on the first run; on a re-run, add/replace assets
# on the release that is already there.
if gh release view "$TAG" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
gh release upload "$TAG" "${files[@]}" --repo "$GITHUB_REPOSITORY" --clobber
else
gh release create "$TAG" "${files[@]}" \
--repo "$GITHUB_REPOSITORY" \
--title "tsunagi $TAG" \
--generate-notes \
--verify-tag \
"${prerelease[@]}"
fi
+9 -2
View File
@@ -90,6 +90,12 @@ Keep these separate. Crossing them is the main thing to review for.
transaction, **before** the record is announced.
- **Plugins own their system objects.** A plugin creates and removes its own
interface and nothing else. Never touch routing, DNS or firewall settings.
The one exception is the system level, not a plugin: while a network's
broadcast is on, `overlay::hostrules` may install exactly a
`255.255.255.255/32` route and an inbound UDP firewall allowance (overlay range
only) for the interface the agent owns. Both are tagged `tsunagi:<interface>`,
reversible, removed on broadcast off, leave and shutdown, and computed from
local state, never from a remote announcement.
- **Device identity vs network identity.** The iroh endpoint id is the device's
public key. `NetworkId` is derived from name + secret only. Never conflate
them, and never let one change the other.
@@ -137,7 +143,8 @@ Keep these separate. Crossing them is the main thing to review for.
up shutdown. Wind tasks down with a grace period and then abort.
10. **Nothing from a remote announcement becomes a shell command, a filesystem
path or an OS setting.** The agent never touches interfaces or OS settings
it did not create.
it did not create (the tagged broadcast host rules above are for its own
interface).
## Where things live
@@ -150,7 +157,7 @@ Keep these separate. Crossing them is the main thing to review for.
| `crates/tsunagi/src/net.rs` | iroh endpoint adapter and observability snapshots |
| `crates/tsunagi/src/agent/` | agent lifecycle, per-network runtimes, sessions, events, status |
| `crates/tsunagi/src/state/` | signed records that outlive a session, their merge rules and address allocation |
| `crates/tsunagi/src/overlay/` | the one interface an agent owns: provisioning, the TUN, routing, source checks |
| `crates/tsunagi/src/overlay/` | the one interface an agent owns: provisioning, the TUN, routing, source checks, broadcast host rules |
| `crates/tsunagi/src/dataplane/` | the protocol contract and the authenticated packet transport |
| `crates/tsunagi/src/dns/` | the DNS view of a network: zone, server, resolver publication |
| `crates/tsunagi-wg-quic/` | the `wg-quic` protocol: its keys, its announcement, its tunnels |
Generated
+27 -1
View File
@@ -4212,7 +4212,7 @@ dependencies = [
"hmac 0.13.0",
"iroh",
"mainline",
"netdev 0.45.1",
"netdev 0.46.3",
"netwatch",
"postcard",
"rand 0.10.3",
@@ -4231,6 +4231,7 @@ dependencies = [
"tracing-subscriber",
"tsunagi",
"tun",
"windows-sys 0.61.2",
"zbus",
"zeroize",
]
@@ -4252,6 +4253,7 @@ dependencies = [
"tsunagi",
"tsunagi-tcp-tls",
"tsunagi-wg-quic",
"tsunagi-wireguard",
]
[[package]]
@@ -4308,6 +4310,30 @@ dependencies = [
"zeroize",
]
[[package]]
name = "tsunagi-wireguard"
version = "0.1.0"
dependencies = [
"boringtun",
"bytes",
"data-encoding",
"hex",
"iroh",
"netdev 0.46.3",
"postcard",
"rand 0.10.3",
"rusqlite",
"serde",
"subtle",
"tempfile",
"thiserror 2.0.20",
"tokio",
"tracing",
"tracing-subscriber",
"tsunagi",
"zeroize",
]
[[package]]
name = "tun"
version = "0.8.14"
+1 -1
View File
@@ -19,7 +19,7 @@ repository = "https://github.com/tsunagi-net/tsunagi"
[workspace.lints.rust]
missing_docs = "warn"
unsafe_code = "forbid"
unsafe_code = "deny"
[workspace.lints.clippy]
unwrap_used = "warn"
+25 -5
View File
@@ -478,6 +478,7 @@ The original IP packet, including its DF bit, is preserved. See
| `setcap cap_net_admin+p` | ordinary user, one capability | recommended: nothing to prepare, nothing left behind. Lost on every rebuild or copy of the binary. |
| systemd service | `User=`, `AmbientCapabilities=CAP_NET_ADMIN` | the same, for an installed service |
| `sudo tsunagi up` | root | everything works, nothing is isolated |
| macOS root LaunchDaemon | root | the native macOS deployment; see below |
| `--no-tun` | ordinary user, no capabilities | tunnels run and handshake, traffic never reaches the OS |
On Windows, put `wintun.dll` beside the executable and start `tsunagi up`
@@ -489,9 +490,24 @@ of claiming the agent is absent or trying to edit its locked state.
Creating a TUN without sufficient privileges also explains how to restart
the agent with the required permissions.
**Not implemented yet.** macOS has no provisioner; `--no-tun` is the way to
run it. The control plane and tunnels are unaffected. The decision logic
that says *what* to change is shared and tested on every platform.
On macOS the agent creates a `utun` interface, configures it with `ifconfig`
and `route`, and points the system resolver at the overlay by writing
`/etc/resolver/<network>` files (which carry a port, so the DNS server needs no
privileged socket). All of this needs **root**: macOS has no per-capability
grant like Linux, and a `utun` plus `/etc/resolver` are root-only. The kernel
assigns the interface its `utunN` name, so the name derived from the network id
is only a request and the real one is adopted once the interface is up.
Run it as root — `sudo tsunagi up` for a quick start, or install it as a root
**LaunchDaemon** for the native always-on deployment. As a daemon the agent runs
unprivileged commands (`join`, `status`, `dns`, `network`) over its control
socket like any other platform; the socket lives at `/var/run/tsunagi/agent.sock`
and the stores at `/var/db/tsunagi`, both fixed system paths so they do not
depend on whose shell started `sudo`. A ready-to-edit plist and install notes
are in [dist/macos/](dist/macos/). The broadcast route is installed; the
inbound-UDP firewall allowance (a `pf` anchor) is not yet, so `status` reports
that half as incomplete and names what to allow by hand. The decision logic that
says *what* to change is shared and tested on every platform.
## Checks
@@ -604,8 +620,12 @@ tsunagi network broadcast games on
```
The choice persists across restarts and a plain `join`. It can be changed while
the agent runs; `status` shows it for each network. The game must send through
the Tsunagi interface. Physical LAN capture/subnet sharing is not implemented.
the agent runs; `status` shows it for each network. While broadcast is on, the
agent routes `255.255.255.255` through the Tsunagi interface and allows inbound
UDP from the overlay range on it (Linux and Windows, needs the same privilege as
the interface), and removes both when it is turned off or the agent stops. Games
bound to a physical address still bypass it. Physical LAN capture/subnet sharing
is not implemented.
See [broadcast.md](docs/broadcast.md) for domain isolation and future LAN gateways.
## How peers find each other
+1
View File
@@ -17,6 +17,7 @@ doc = false
[dependencies]
tsunagi = { path = "../tsunagi", version = "0.1.0", features = ["tun-device", "dns-publish"] }
tsunagi-wireguard = { path = "../tsunagi-wireguard", version = "0.1.0" }
tsunagi-wg-quic = { path = "../tsunagi-wg-quic", version = "0.1.0" }
tsunagi-tcp-tls = { path = "../tsunagi-tcp-tls", version = "0.1.0" }
iroh.workspace = true
+504 -158
View File
@@ -19,7 +19,7 @@ use tsunagi::discovery::{
CompositeDiscovery, MainlineDiscovery, NetworkDiscovery, StaticBootstrap,
};
use tsunagi::identity::{NetworkName, NetworkSecret};
use tsunagi::iroh_types::EndpointAddr;
use tsunagi::iroh_types::{EndpointAddr, EndpointId};
use tsunagi::overlay::{MemoryTunFactory, TunFactory};
use tsunagi::state::Ipv4Range;
use tsunagi::{Agent, NetworkId};
@@ -27,6 +27,10 @@ use tsunagi_tcp_tls::{
TCP_TLS_PROTOCOL, TCP_TLS_VERSION, TcpTlsCodec, TcpTlsConfig, TcpTlsPlugin, TcpTlsTransport,
};
use tsunagi_wg_quic::{WireguardConfig, WireguardPlugin};
use tsunagi_wireguard::{
WG_PROTOCOL, WG_WIRE_VERSION, WireguardConfig as PureWgConfig, WireguardPlugin as PureWgPlugin,
WireguardTransport as PureWgTransport,
};
/// A small agent for private mesh networks.
#[derive(Debug, Parser)]
@@ -82,7 +86,7 @@ struct DnsArgs {
paths: PathArgs,
/// Control socket to talk to. Derived from the state directory by default.
#[arg(long, global = true)]
#[arg(long, env = "TSUNAGI_CONTROL_SOCKET", global = true)]
control_socket: Option<PathBuf>,
#[command(subcommand)]
@@ -107,7 +111,7 @@ struct NetworkArgs {
paths: PathArgs,
/// Control socket to talk to. Derived from the state directory by default.
#[arg(long, global = true)]
#[arg(long, env = "TSUNAGI_CONTROL_SOCKET", global = true)]
control_socket: Option<PathBuf>,
#[command(subcommand)]
@@ -132,7 +136,7 @@ struct JoinArgs {
paths: PathArgs,
/// Control socket to talk to. Derived from the state directory by default.
#[arg(long)]
#[arg(long, env = "TSUNAGI_CONTROL_SOCKET")]
control_socket: Option<PathBuf>,
/// Network name. Must be identical on every participant.
@@ -233,7 +237,7 @@ struct WipeArgs {
paths: PathArgs,
/// Control socket to check for a running agent.
#[arg(long)]
#[arg(long, env = "TSUNAGI_CONTROL_SOCKET")]
control_socket: Option<PathBuf>,
/// Actually remove it. Without this the command only says what it would.
@@ -247,7 +251,7 @@ struct IdArgs {
paths: PathArgs,
/// Control socket to talk to. Derived from the state directory by default.
#[arg(long, global = true)]
#[arg(long, env = "TSUNAGI_CONTROL_SOCKET", global = true)]
control_socket: Option<PathBuf>,
#[command(subcommand)]
@@ -284,7 +288,7 @@ struct StatusArgs {
paths: PathArgs,
/// Control socket to talk to. Derived from the state directory by default.
#[arg(long)]
#[arg(long, env = "TSUNAGI_CONTROL_SOCKET")]
control_socket: Option<PathBuf>,
}
@@ -421,7 +425,7 @@ struct UpArgs {
long = "protocol",
value_name = "LIST",
value_delimiter = ',',
default_value = "tcp-tls,wg-quic",
default_value = "wg,tcp-tls,wg-quic",
help_heading = "Transport"
)]
protocols: Vec<String>,
@@ -476,7 +480,7 @@ struct UpArgs {
status_interval: u64,
/// Control socket to serve. Derived from the state directory by default.
#[arg(long)]
#[arg(long, env = "TSUNAGI_CONTROL_SOCKET")]
control_socket: Option<PathBuf>,
}
@@ -698,7 +702,7 @@ fn main() -> std::process::ExitCode {
.or_else(|| std::env::var("TSUNAGI_LOG").ok())
.or_else(|| std::env::var("RUST_LOG").ok())
.unwrap_or_else(|| {
"info,iroh=warn,quinn=warn,rustls=warn,boringtun=warn,mainline=off".to_string()
"info,iroh=warn,iroh::net_report=error,quinn=warn,rustls=warn,boringtun=error,mainline=off".to_string()
});
let is_debug = log_filter.contains("debug") || log_filter.contains("trace");
@@ -745,11 +749,19 @@ async fn run(command: Command) -> Result<(), Box<dyn std::error::Error>> {
}
/// Path of the local control socket for a state directory.
fn control_socket(paths: &StoragePaths, override_path: Option<&PathBuf>) -> PathBuf {
match override_path {
Some(path) => path.clone(),
None => tsunagi::ipc::control_socket_path(&paths.state_dir),
fn control_socket(
paths: &StoragePaths,
override_path: Option<&PathBuf>,
_is_custom_state_dir: bool,
) -> PathBuf {
if let Some(path) = override_path {
return path.clone();
}
#[cfg(windows)]
if !_is_custom_state_dir {
return tsunagi::ipc::default_control_socket_path();
}
tsunagi::ipc::control_socket_path(&paths.state_dir)
}
/// What could be learned about this device, and from where.
@@ -946,7 +958,11 @@ fn dns_publisher() -> Arc<dyn tsunagi::dns::DnsPublisher> {
{
Arc::new(tsunagi::dns::publish::NrptPublisher::new())
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
#[cfg(target_os = "macos")]
{
Arc::new(tsunagi::dns::publish::ResolverDirPublisher::new())
}
#[cfg(not(any(target_os = "linux", target_os = "windows", target_os = "macos")))]
{
Arc::new(tsunagi::dns::publish::UnsupportedPublisher::new())
}
@@ -1182,6 +1198,12 @@ struct ProtocolSpec {
/// Every protocol this build has.
const PROTOCOLS: &[ProtocolSpec] = &[
ProtocolSpec {
name: WG_PROTOCOL,
version: WG_WIRE_VERSION,
summary: "Pure WireGuard direct UDP transport for uncensored networks and maximum performance",
options: PureWgPlugin::OPTIONS,
},
ProtocolSpec {
name: TCP_TLS_PROTOCOL,
version: TCP_TLS_VERSION,
@@ -1283,6 +1305,7 @@ fn show_protocols() -> Result<(), Box<dyn std::error::Error>> {
/// published.
struct AgentControl {
agent: Agent,
pure_wg: Option<Arc<PureWgPlugin>>,
plugin: Option<Arc<WireguardPlugin>>,
tcp_tls: Option<Arc<TcpTlsPlugin>>,
/// The resolver, which this owns so it can be switched while running.
@@ -1297,6 +1320,7 @@ impl tsunagi::ipc::ReportSource for AgentControl {
let dns = self.dns_state().await;
build_report(
&self.agent,
self.pure_wg.as_deref(),
self.plugin.as_deref(),
self.tcp_tls.as_deref(),
dns,
@@ -1580,7 +1604,8 @@ fn resolve_network<'a>(
/// `tsunagi network`: what this device belongs to, and leaving it.
async fn network_command(args: NetworkArgs) -> Result<(), Box<dyn std::error::Error>> {
let paths = args.paths.resolve()?;
let socket = control_socket(&paths, args.control_socket.as_ref());
let is_custom_state = args.paths.state_dir.is_some();
let socket = control_socket(&paths, args.control_socket.as_ref(), is_custom_state);
match args.action {
None => show_networks(&paths, &socket).await,
Some(NetworkAction::Broadcast { network, choice }) => {
@@ -1614,7 +1639,8 @@ async fn network_command(args: NetworkArgs) -> Result<(), Box<dyn std::error::Er
/// `tsunagi join`: make a network or join one.
async fn join_command(args: JoinArgs) -> Result<(), Box<dyn std::error::Error>> {
let paths = args.paths.resolve()?;
let socket = control_socket(&paths, args.control_socket.as_ref());
let is_custom_state = args.paths.state_dir.is_some();
let socket = control_socket(&paths, args.control_socket.as_ref(), is_custom_state);
let name = NetworkName::new(args.network)?;
// A name this device already has means that network; a name nobody
// has means a new one, and no secret is needed to make a network with
@@ -1995,12 +2021,26 @@ fn forget_protocol_state(paths: &StoragePaths, network: tsunagi::NetworkId) {
}
Err(err) => eprintln!("warning: the wg-quic key store could not be opened: {err}"),
}
let pure_store = PureWgConfig::new(paths.state_dir.join("wg"));
match tsunagi_wireguard::WgKeyStore::open(pure_store.key_store_path()) {
Ok(store) => {
if let Err(err) = store.forget(network) {
eprintln!("warning: the wg key for it could not be removed: {err}");
}
}
Err(err) if !pure_store.key_store_path().exists() => {
let _ = err;
}
Err(err) => eprintln!("warning: the wg key store could not be opened: {err}"),
}
}
/// `tsunagi wipe`: back to a device that has never joined anything.
async fn wipe(args: WipeArgs) -> Result<(), Box<dyn std::error::Error>> {
let paths = args.paths.resolve()?;
let socket = control_socket(&paths, args.control_socket.as_ref());
let is_custom_state = args.paths.state_dir.is_some();
let socket = control_socket(&paths, args.control_socket.as_ref(), is_custom_state);
if tsunagi::ipc::is_serving(&socket).await {
return Err(
"stop the agent first: a wipe removes the state it is using, and leaving a \
@@ -2060,7 +2100,8 @@ async fn wipe(args: WipeArgs) -> Result<(), Box<dyn std::error::Error>> {
/// `tsunagi dns`: the local resolver, and turning it on or off.
async fn dns_command(args: DnsArgs) -> Result<(), Box<dyn std::error::Error>> {
let paths = args.paths.resolve()?;
let socket = control_socket(&paths, args.control_socket.as_ref());
let is_custom_state = args.paths.state_dir.is_some();
let socket = control_socket(&paths, args.control_socket.as_ref(), is_custom_state);
let enable = match args.action {
None => return show_dns(&paths, &socket).await,
@@ -2152,7 +2193,8 @@ async fn show_dns(
/// `tsunagi id`: what this device is, and what changes it.
async fn id(args: IdArgs) -> Result<(), Box<dyn std::error::Error>> {
let paths = args.paths.resolve()?;
let socket = control_socket(&paths, args.control_socket.as_ref());
let is_custom_state = args.paths.state_dir.is_some();
let socket = control_socket(&paths, args.control_socket.as_ref(), is_custom_state);
match args.action {
None => show_identity(&paths, &socket).await,
@@ -2343,7 +2385,8 @@ async fn status(args: StatusArgs) -> Result<(), Box<dyn std::error::Error>> {
use report::{Health, Report, Row, Section};
let paths = args.paths.resolve()?;
let socket = control_socket(&paths, args.control_socket.as_ref());
let is_custom_state = args.paths.state_dir.is_some();
let socket = control_socket(&paths, args.control_socket.as_ref(), is_custom_state);
let observed = observe(&paths, &socket).await;
let mut out = Report::new();
@@ -2683,6 +2726,25 @@ fn network_section(
"broadcast",
if network.broadcast { "on" } else { "off" },
));
if network.broadcast
&& let Some(rules) = network
.overlay
.as_ref()
.and_then(|o| o.broadcast_rules.as_ref())
{
section.push(Row::new(
if rules.ok {
Health::Good
} else {
Health::Degraded
},
"broadcast rules",
broadcast_rules_line(
rules,
network.overlay.as_ref().and_then(|o| o.address.as_deref()),
),
));
}
let relayed = network.relay_forwarded + network.relay_sent_via + network.relay_received_via;
if relayed > 0 {
section.push(Row::new(
@@ -2796,7 +2858,7 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
// ordinary condition, so it is stated rather than flagged.
let mut detail = "offline".to_string();
if let Some(v4) = row.overlay_address_v4 {
detail.push_str(&format!(" · {v4} still reserved for it"));
detail = format!("{v4:<15} offline · {v4} still reserved for it");
}
let out = Row::new(Health::Info, row.label(), detail);
return if row.failed_dials > 0 {
@@ -2815,13 +2877,14 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
let tunnel_up = row.tunnel.is_some_and(|tunnel| tunnel.is_up());
let has_overlay = row.tunnel.is_some();
let mut detail = transport.to_lowercase();
let mut detail = String::new();
if let Some(v4) = row.overlay_address_v4 {
detail.push_str(&format!("{v4:<15} "));
}
detail.push_str(&transport.to_lowercase());
if let Some(rtt) = row.rtt_ms {
detail.push_str(&format!(" rtt {rtt}ms"));
}
if let Some(v4) = row.overlay_address_v4 {
detail.push_str(&format!(" · {v4}"));
}
let health = if !direct || (has_overlay && !tunnel_up) {
Health::Degraded
@@ -2837,24 +2900,26 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
&tunnel.protocol
};
out = out.with_note(match tunnel.handshake_secs_ago {
Some(secs) => format!(
"tunnel up ({proto}), {} tx {} rx {}{} · {}",
if proto == "tcp-tls" {
format!("uptime {secs}s,")
} else {
format!("handshake {secs}s ago,")
},
tunnel.tx_packets,
tunnel.rx_packets,
if tunnel.dropped > 0 {
format!(", {} dropped", tunnel.dropped)
} else {
String::new()
},
tunnel.path
),
Some(secs) => {
let stats = format!(
"tunnel up ({proto}), {} tx {} rx {}{}",
if proto == "tcp-tls" {
format!("uptime {secs}s,")
} else {
format!("handshake {secs}s ago,")
},
tunnel.tx_packets,
tunnel.rx_packets,
if tunnel.dropped > 0 {
format!(", {} dropped", tunnel.dropped)
} else {
String::new()
},
);
format!("{stats:<48} · {}", tunnel.path)
}
None => {
if proto == "wg-quic" {
if proto == "wg-quic" || proto == "wg" {
"no WireGuard handshake yet; the tunnel cannot carry traffic".to_string()
} else {
format!("no {proto} handshake yet; the tunnel cannot carry traffic")
@@ -2968,28 +3033,90 @@ fn host_section() -> report::Section {
host
}
/// Whether an address is link-local — `fe80::/10` or `169.254.0.0/16` — which
/// the operating system assigns per interface and which is never a peer
/// candidate.
fn is_link_local(addr: std::net::IpAddr) -> bool {
match addr {
std::net::IpAddr::V4(v4) => v4.is_link_local(),
std::net::IpAddr::V6(v6) => (v6.segments()[0] & 0xffc0) == 0xfe80,
}
}
/// The addresses this host could reach a peer from.
async fn addresses_section() -> report::Section {
use report::{Health, Row, Section};
let mut addresses = Section::new("local addresses");
let found = netwatch_addresses().await;
if found.is_empty() {
let found = netwatch_interface_addresses().await;
let mut candidates = tsunagi_wireguard::local_ip_candidates();
for ip in tsunagi_tcp_tls::local_ip_candidates() {
if !candidates.contains(&ip) {
candidates.push(ip);
}
}
if found.is_empty() && candidates.is_empty() {
addresses.push(
Row::new(Health::Degraded, "interfaces", "none found")
.with_note("best effort; the agent may still find a way out"),
);
return addresses;
}
for addr in found {
// Loopback alone reaches nobody, but on a host that also has a real
// address it is unremarkable, so it is labelled rather than flagged.
let kind = match (addr.is_loopback(), addr.is_ipv4()) {
(true, _) => "loopback",
(false, true) => "ipv4",
(false, false) => "ipv6",
};
addresses.push(Row::new(Health::Info, kind, addr.to_string()));
let mut iface_map: std::collections::HashMap<std::net::IpAddr, String> =
std::collections::HashMap::new();
for item in &found {
iface_map.insert(item.addr, item.iface.clone());
}
// One line per address: the announce status is the label, and the address
// and its interface share the detail, so a host with many addresses stays
// compact instead of spending two lines on each.
let detail = |addr: std::net::IpAddr| match iface_map.get(&addr) {
Some(iface) => format!("{addr} · {iface}"),
None => addr.to_string(),
};
// 1. Announced peer candidate addresses (in candidate priority order)
let mut seen = std::collections::HashSet::new();
for &cand in &candidates {
if seen.insert(cand) {
addresses.push(Row::new(Health::Info, "announced", detail(cand)));
}
}
// 2. Unannounced local addresses
let mut unannounced = Vec::new();
let mut loopbacks = Vec::new();
for item in &found {
if seen.contains(&item.addr) {
continue;
}
if item.addr.is_loopback() {
loopbacks.push(item);
} else if is_link_local(item.addr) {
// Link-local (fe80::/10, 169.254.0.0/16) is never a peer candidate,
// so one per interface is noise rather than information.
continue;
} else {
unannounced.push(item);
}
}
unannounced.sort_by_key(|item| item.addr);
for item in unannounced {
addresses.push(Row::new(Health::Info, "not announced", detail(item.addr)));
}
// 3. Loopback addresses
loopbacks.sort_by_key(|item| item.addr);
for item in loopbacks {
addresses.push(Row::new(Health::Info, "loopback", detail(item.addr)));
}
addresses
}
@@ -3391,12 +3518,31 @@ fn program_path() -> String {
.unwrap_or_else(|| "tsunagi".to_string())
}
async fn netwatch_addresses() -> Vec<std::net::IpAddr> {
struct LocalAddressInfo {
addr: std::net::IpAddr,
iface: String,
}
async fn netwatch_interface_addresses() -> Vec<LocalAddressInfo> {
// Best effort; used for diagnostics only.
let state = netwatch::interfaces::State::new().await;
let mut addresses = state.local_addresses.regular;
addresses.sort();
addresses.dedup();
let mut addresses = Vec::new();
let mut seen = std::collections::HashSet::new();
for (name, iface) in &state.interfaces {
let name_lower = name.to_lowercase();
if name_lower.starts_with("tsun") || name_lower.contains("tsunagi") {
continue;
}
for prefix in iface.addrs() {
let addr = prefix.addr();
if seen.insert(addr) {
addresses.push(LocalAddressInfo {
addr,
iface: name.clone(),
});
}
}
}
addresses
}
@@ -3463,7 +3609,13 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
.iter()
.filter(|name| !name.eq_ignore_ascii_case("none"))
{
let Some(spec) = PROTOCOLS.iter().find(|spec| spec.name == name.as_str()) else {
// Accept "wireguard" as a convenience alias for the canonical "wg".
let canonical = if name.eq_ignore_ascii_case("wireguard") {
WG_PROTOCOL
} else {
name.as_str()
};
let Some(spec) = PROTOCOLS.iter().find(|spec| spec.name == canonical) else {
let known: Vec<&str> = PROTOCOLS.iter().map(|spec| spec.name).collect();
return Err(format!(
"this build has no protocol called `{name}`; it has {}. \
@@ -3498,6 +3650,8 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
let mut wireguard = None;
let mut tcp_tls_transport = None;
let mut tcp_tls_plugin = None;
let mut pure_wg_plugin: Option<Arc<PureWgPlugin>> = None;
let mut pure_wg_transport: Option<Arc<PureWgTransport>> = None;
if !wanted.is_empty() {
let tun_factory: Arc<dyn TunFactory> = if args.no_tun {
Arc::new(MemoryTunFactory::new())
@@ -3548,6 +3702,35 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
tcp_tls_transport = Some(transport);
tcp_tls_plugin = Some(plugin);
}
WG_PROTOCOL => {
let mut wg_cfg = PureWgConfig::new(paths.state_dir.join("wg"));
if let Some(mtu) = args.mtu {
wg_cfg = wg_cfg.with_mtu(mtu);
}
let wg_cfg = PureWgPlugin::configure(wg_cfg, &options)?;
let identity = tsunagi::storage::StateStore::open(paths.state_db())?
.load_or_create_device_identity()?;
let transport = Arc::new(
PureWgTransport::bind(&identity, wg_cfg.port)
.await
.map_err(|err| err.to_string())?,
);
for ep_addr in &bootstrap {
for addr in ep_addr.ip_addrs() {
transport.set_peer_addr(ep_addr.id, *addr);
}
}
let plugin = PureWgPlugin::open(wg_cfg, Arc::clone(&transport)).await?;
config = config
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>)
.with_custom_transport(
WG_PROTOCOL,
Arc::clone(&transport)
as Arc<dyn tsunagi::dataplane::transport::PacketTransport>,
);
pure_wg_transport = Some(transport);
pure_wg_plugin = Some(plugin);
}
other => return Err(format!("`{other}` is listed but not built in").into()),
}
}
@@ -3559,7 +3742,8 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
// agent that is already running, so the lock on its own is an
// answer to a question nobody asked.
Err(tsunagi::Error::StateLocked { path }) => {
let socket = control_socket(&paths, args.control_socket.as_ref());
let is_custom_state = args.paths.state_dir.is_some();
let socket = control_socket(&paths, args.control_socket.as_ref(), is_custom_state);
if tsunagi::ipc::is_serving(&socket).await {
return Err(format!(
"an agent is already running for {}, and one state directory is one \
@@ -3599,6 +3783,21 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
.await;
});
}
if let Some(transport) = pure_wg_transport {
let agent_inbound = agent.clone();
tokio::spawn(async move {
transport
.accept_loop(move |inbound| {
let agent = agent_inbound.clone();
tokio::spawn(async move {
if let Err(err) = agent.install_inbound_link(inbound).await {
tracing::debug!(%err, "failed to install inbound wg link");
}
});
})
.await;
});
}
let protocol_names: Vec<&str> = wanted.iter().map(|p| p.name).collect();
@@ -3635,17 +3834,20 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
// is not fatal: the agent itself works fine without it.
let control = {
let agent = agent.clone();
let pure_wg = pure_wg_plugin.clone();
let plugin = wireguard.clone();
let tcp_tls = tcp_tls_plugin.clone();
let dns = Arc::clone(&dns);
let source: Arc<dyn tsunagi::ipc::ReportSource> = Arc::new(AgentControl {
agent,
pure_wg,
plugin,
tcp_tls,
dns,
paths: paths.clone(),
});
let path = control_socket(&paths, args.control_socket.as_ref());
let is_custom_state = args.paths.state_dir.is_some();
let path = control_socket(&paths, args.control_socket.as_ref(), is_custom_state);
match tsunagi::ipc::ControlSocket::bind(path, source).await {
Ok(socket) => {
println!(" control {}", socket.path().display());
@@ -3713,6 +3915,7 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
/// both.
async fn build_report(
agent: &Agent,
pure_wg: Option<&PureWgPlugin>,
wireguard: Option<&WireguardPlugin>,
tcp_tls: Option<&TcpTlsPlugin>,
dns: Option<DnsState>,
@@ -3733,34 +3936,85 @@ async fn build_report(
.iter()
.map(|network| {
let wg_view = wireguard.and_then(|plugin| plugin.overview(network.network_id));
let pure_wg_view = pure_wg.and_then(|plugin| plugin.overview(network.network_id));
let tcp_tls_reports = tcp_tls
.map(|p| p.active_peers(network.network_id))
.unwrap_or_default();
let overlay = if let Some(view) = wg_view {
let mut peers = Vec::new();
for peer in &view.peers {
let tcp_tls_match = tcp_tls_reports.iter().find(|r| r.peer == peer.endpoint_id);
// Use whichever WireGuard view is available for the overlay
// address and range — wg-quic and pure wg both derive them
// identically from the control plane's allocations.
let wg_overlay_addr = wg_view
.as_ref()
.and_then(|v| v.overlay_address_v4)
.or_else(|| pure_wg_view.as_ref().and_then(|v| v.overlay_address_v4));
let wg_prefix_len = wg_view
.as_ref()
.and_then(|v| v.ipv4_range)
.or_else(|| pure_wg_view.as_ref().and_then(|v| v.ipv4_range))
.map_or(0, |r| r.prefix_len);
let has_any_plugin =
wg_view.is_some() || pure_wg_view.is_some() || !tcp_tls_reports.is_empty();
if let Some(tcp_tls) = tcp_tls_match {
let overlay = if has_any_plugin && overlay.is_some() {
let mut peers: Vec<OverlayPeerReport> = Vec::new();
// Collect all unique peer IDs known in this network.
let mut all_peer_ids: Vec<EndpointId> = Vec::new();
for member in &network.members {
if member.endpoint_id != status.endpoint_id
&& !all_peer_ids.contains(&member.endpoint_id)
{
all_peer_ids.push(member.endpoint_id);
}
}
if let Some(ref view) = pure_wg_view {
for peer in &view.peers {
if !all_peer_ids.contains(&peer.endpoint_id) {
all_peer_ids.push(peer.endpoint_id);
}
}
}
if let Some(ref view) = wg_view {
for peer in &view.peers {
if !all_peer_ids.contains(&peer.endpoint_id) {
all_peer_ids.push(peer.endpoint_id);
}
}
}
for report in &tcp_tls_reports {
if !all_peer_ids.contains(&report.peer) {
all_peer_ids.push(report.peer);
}
}
for peer_id in all_peer_ids {
let pure_wg_match = pure_wg_view
.as_ref()
.and_then(|v| v.peers.iter().find(|p| p.endpoint_id == peer_id));
let tcp_tls_match = tcp_tls_reports.iter().find(|r| r.peer == peer_id);
let wg_quic_match = wg_view
.as_ref()
.and_then(|v| v.peers.iter().find(|p| p.endpoint_id == peer_id));
let member_addr = network
.members
.iter()
.find(|m| m.endpoint_id == peer_id)
.and_then(|m| m.overlay_address_v4)
.map(|a| a.to_string());
if let Some(pure_peer) = pure_wg_match
&& let Some(tunnel) = &pure_peer.tunnel
{
peers.push(OverlayPeerReport {
endpoint_id: peer.endpoint_id.to_string(),
protocol: tcp_tls.protocol.clone(),
public_key: peer.public_key.to_string(),
address: peer.overlay_address_v4.map(|addr| addr.to_string()),
handshake_secs_ago: Some(tcp_tls.uptime_secs),
tx_packets: tcp_tls.tx_packets,
rx_packets: tcp_tls.rx_packets,
dropped: tcp_tls.dropped,
protocol_errors: 0,
path: tcp_tls.path.clone(),
});
} else if let Some(tunnel) = &peer.tunnel {
peers.push(OverlayPeerReport {
endpoint_id: peer.endpoint_id.to_string(),
protocol: "wg-quic".into(),
public_key: peer.public_key.to_string(),
address: peer.overlay_address_v4.map(|addr| addr.to_string()),
endpoint_id: peer_id.to_string(),
protocol: "wg".into(),
public_key: pure_peer.public_key.to_string(),
address: pure_peer
.overlay_address_v4
.map(|a| a.to_string())
.or(member_addr),
handshake_secs_ago: tunnel.health.since_handshake.map(|s| s.as_secs()),
tx_packets: tunnel.stats.tx_packets,
rx_packets: tunnel.stats.rx_packets,
@@ -3769,12 +4023,67 @@ async fn build_report(
protocol_errors: tunnel.stats.protocol_errors,
path: tunnel.path.clone(),
});
} else {
} else if let Some(tcp_tls) = tcp_tls_match {
let pubkey = pure_wg_match
.map(|p| p.public_key.to_string())
.or_else(|| wg_quic_match.map(|p| p.public_key.to_string()))
.unwrap_or_default();
peers.push(OverlayPeerReport {
endpoint_id: peer.endpoint_id.to_string(),
endpoint_id: peer_id.to_string(),
protocol: tcp_tls.protocol.clone(),
public_key: pubkey,
address: member_addr,
handshake_secs_ago: Some(tcp_tls.uptime_secs),
tx_packets: tcp_tls.tx_packets,
rx_packets: tcp_tls.rx_packets,
dropped: tcp_tls.dropped,
protocol_errors: 0,
path: tcp_tls.path.clone(),
});
} else if let Some(wg_peer) = wg_quic_match
&& let Some(tunnel) = &wg_peer.tunnel
{
peers.push(OverlayPeerReport {
endpoint_id: peer_id.to_string(),
protocol: "wg-quic".into(),
public_key: peer.public_key.to_string(),
address: peer.overlay_address_v4.map(|addr| addr.to_string()),
public_key: wg_peer.public_key.to_string(),
address: wg_peer
.overlay_address_v4
.map(|a| a.to_string())
.or(member_addr),
handshake_secs_ago: tunnel.health.since_handshake.map(|s| s.as_secs()),
tx_packets: tunnel.stats.tx_packets,
rx_packets: tunnel.stats.rx_packets,
dropped: tunnel.stats.dropped_wrong_source
+ tunnel.stats.dropped_oversize,
protocol_errors: tunnel.stats.protocol_errors,
path: tunnel.path.clone(),
});
} else if let Some(pure_peer) = pure_wg_match {
peers.push(OverlayPeerReport {
endpoint_id: peer_id.to_string(),
protocol: "wg".into(),
public_key: pure_peer.public_key.to_string(),
address: pure_peer
.overlay_address_v4
.map(|a| a.to_string())
.or(member_addr),
handshake_secs_ago: None,
tx_packets: 0,
rx_packets: 0,
dropped: 0,
protocol_errors: 0,
path: "no data link".into(),
});
} else if let Some(wg_peer) = wg_quic_match {
peers.push(OverlayPeerReport {
endpoint_id: peer_id.to_string(),
protocol: "wg-quic".into(),
public_key: wg_peer.public_key.to_string(),
address: wg_peer
.overlay_address_v4
.map(|a| a.to_string())
.or(member_addr),
handshake_secs_ago: None,
tx_packets: 0,
rx_packets: 0,
@@ -3784,89 +4093,43 @@ async fn build_report(
});
}
}
for report in &tcp_tls_reports {
if !peers
.iter()
.any(|p| p.endpoint_id == report.peer.to_string())
{
let address = network
let (address, prefix_len) = if wg_overlay_addr.is_some() {
(wg_overlay_addr.map(|addr| addr.to_string()), wg_prefix_len)
} else {
(
network
.members
.iter()
.find(|m| m.endpoint_id == report.peer)
.find(|m| m.endpoint_id == status.endpoint_id)
.and_then(|m| m.overlay_address_v4)
.map(|a| a.to_string());
peers.push(OverlayPeerReport {
endpoint_id: report.peer.to_string(),
protocol: report.protocol.clone(),
public_key: String::new(),
address,
handshake_secs_ago: Some(report.uptime_secs),
tx_packets: report.tx_packets,
rx_packets: report.rx_packets,
dropped: report.dropped,
protocol_errors: 0,
path: report.path.clone(),
});
}
}
.map(|a| a.to_string()),
network.range.map_or(0, |r| r.prefix_len),
)
};
Some(OverlayReport {
interface: overlay
.as_ref()
.map_or_else(String::new, |o| o.interface.clone()),
on_host: overlay.as_ref().is_some_and(|o| o.on_host),
mtu: overlay.as_ref().map_or(0, |o| o.mtu),
address: view.overlay_address_v4.map(|addr| addr.to_string()),
prefix_len: view.ipv4_range.map_or(0, |r| r.prefix_len),
address,
prefix_len,
peers,
unroutable_packets: overlay.as_ref().map_or(0, |o| o.counters.unroutable),
multicast_packets: overlay.as_ref().map_or(0, |o| o.counters.multicast),
unroutable_sample: overlay
.as_ref()
.and_then(|o| o.counters.unroutable_sample.map(|a| a.to_string())),
})
} else if overlay.is_some() && !tcp_tls_reports.is_empty() {
let peers = tcp_tls_reports
.into_iter()
.map(|report| {
let address = network
.members
.iter()
.find(|m| m.endpoint_id == report.peer)
.and_then(|m| m.overlay_address_v4)
.map(|a| a.to_string());
OverlayPeerReport {
endpoint_id: report.peer.to_string(),
protocol: report.protocol,
public_key: String::new(),
address,
handshake_secs_ago: Some(report.uptime_secs),
tx_packets: report.tx_packets,
rx_packets: report.rx_packets,
dropped: report.dropped,
protocol_errors: 0,
path: report.path,
}
})
.collect();
Some(OverlayReport {
interface: overlay
broadcast_rules: overlay
.as_ref()
.map_or_else(String::new, |o| o.interface.clone()),
on_host: overlay.as_ref().is_some_and(|o| o.on_host),
mtu: overlay.as_ref().map_or(0, |o| o.mtu),
address: network
.members
.iter()
.find(|m| m.endpoint_id == status.endpoint_id)
.and_then(|m| m.overlay_address_v4)
.map(|a| a.to_string()),
prefix_len: network.range.map_or(0, |r| r.prefix_len),
peers,
unroutable_packets: overlay.as_ref().map_or(0, |o| o.counters.unroutable),
multicast_packets: overlay.as_ref().map_or(0, |o| o.counters.multicast),
unroutable_sample: overlay
.as_ref()
.and_then(|o| o.counters.unroutable_sample.map(|a| a.to_string())),
.and_then(|o| o.broadcast_rules.as_ref())
.map(|rules| tsunagi::ipc::HostRulesReport {
ok: rules.is_ok(),
detail: rules.summary(),
source: rules.plan.source.to_string(),
}),
})
} else {
None
@@ -3935,6 +4198,21 @@ async fn build_report(
}
}
/// The status line for the host rules, saying so when they are bound to
/// another network's address.
fn broadcast_rules_line(
rules: &tsunagi::ipc::HostRulesReport,
own_address: Option<&str>,
) -> String {
match own_address {
Some(own) if own != rules.source => format!(
"{} · route source is {}, another network's address",
rules.detail, rules.source
),
_ => rules.detail.clone(),
}
}
/// Resolves when the process is asked to stop.
///
/// Both Ctrl-C and `SIGTERM` are handled, so a service manager stopping the
@@ -3973,23 +4251,42 @@ async fn stop_signal() -> &'static str {
fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
use tsunagi::overlay::{ManagedTunFactory, NetlinkProvisioner};
let provisioner = NetlinkProvisioner::new()?;
Ok(Arc::new(ManagedTunFactory::new(Arc::new(provisioner))))
let rules = Arc::new(provisioner.host_rules());
Ok(Arc::new(
ManagedTunFactory::new(Arc::new(provisioner)).with_host_rules(rules),
))
}
/// The Wintun adapter, created and configured by the agent and removed when it
/// exits, the same as the Linux one.
#[cfg(target_os = "windows")]
fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
use tsunagi::overlay::{ManagedTunFactory, WintunProvisioner};
use tsunagi::overlay::{ManagedTunFactory, WindowsHostRules, WintunProvisioner};
let provisioner = WintunProvisioner::new()?;
Ok(Arc::new(ManagedTunFactory::new(Arc::new(provisioner))))
Ok(Arc::new(
ManagedTunFactory::new(Arc::new(provisioner))
.with_host_rules(Arc::new(WindowsHostRules::new())),
))
}
/// The utun interface, created and configured by the agent and removed when it
/// exits, the same as the Linux and Windows ones. The kernel assigns the utun
/// unit, so the interface reports a `utunN` name the agent adopts.
#[cfg(target_os = "macos")]
fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
use tsunagi::overlay::{MacosHostRules, ManagedTunFactory, UtunProvisioner};
let provisioner = UtunProvisioner::new()?;
Ok(Arc::new(
ManagedTunFactory::new(Arc::new(provisioner))
.with_host_rules(Arc::new(MacosHostRules::new())),
))
}
/// There is no provisioner for this platform yet.
///
/// Refused here rather than at the first packet, and with the one thing that
/// does work on every platform named.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
#[cfg(not(any(target_os = "linux", target_os = "windows", target_os = "macos")))]
fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
Err(format!(
"managing the overlay interface is not implemented on {} yet. \
@@ -4232,6 +4529,55 @@ mod status_tests {
assert!(text.contains("mistyped secret"), "{text}");
}
#[test]
fn broadcast_rules_are_reported_with_the_reason_when_incomplete() {
let mut network = network_after_a_peer_returned();
network.broadcast = true;
let render = |network: &tsunagi::ipc::NetworkReport| {
let mut out = report::Report::new();
out.push(network_section(network, OWN, false));
(out.render(false), out.worst())
};
// Nothing installed (in-memory interface, or broadcast just off): no row.
let (text, _) = render(&network);
assert!(!text.contains("broadcast rules"), "{text}");
let overlay = network.overlay.as_mut().unwrap();
overlay.broadcast_rules = Some(tsunagi::ipc::HostRulesReport {
ok: true,
detail: "ok (route + firewall)".into(),
source: "10.13.37.69".into(),
});
let (text, worst) = render(&network);
assert!(text.contains("broadcast rules"), "{text}");
assert!(text.contains("ok (route + firewall)"), "{text}");
assert_ne!(worst, Health::Degraded, "{text}");
let overlay = network.overlay.as_mut().unwrap();
overlay.broadcast_rules = Some(tsunagi::ipc::HostRulesReport {
ok: false,
detail: "incomplete; route ok; firewall failed: iptables not found".into(),
source: "10.13.37.69".into(),
});
let (text, worst) = render(&network);
assert!(
text.contains("firewall failed: iptables not found"),
"{text}"
);
assert_eq!(worst, Health::Degraded, "{text}");
// Bound to another network's address: said, not hidden.
network.overlay.as_mut().unwrap().address = Some("10.99.0.5".into());
let (text, _) = render(&network);
assert!(text.contains("another network's address"), "{text}");
// Broadcast off for this network: the line is not shown.
network.broadcast = false;
let (text, _) = render(&network);
assert!(!text.contains("broadcast rules"), "{text}");
}
#[test]
fn an_agent_with_nobody_to_contact_says_so_instead_of_waiting_quietly() {
// The wiped-and-restarted case: both devices join the same network
+3 -1
View File
@@ -5,5 +5,7 @@ pub mod crypto;
pub mod plugin;
pub mod transport;
pub use plugin::{TCP_TLS_PROTOCOL, TCP_TLS_VERSION, TcpTlsCodec, TcpTlsConfig, TcpTlsPlugin};
pub use plugin::{
TCP_TLS_PROTOCOL, TCP_TLS_VERSION, TcpTlsCodec, TcpTlsConfig, TcpTlsPlugin, local_ip_candidates,
};
pub use transport::TcpTlsTransport;
+275 -14
View File
@@ -63,26 +63,147 @@ pub struct TcpTlsAnnouncement {
/// Discovers local network interface IPs to announce to peers.
pub fn local_ip_candidates() -> Vec<std::net::IpAddr> {
let mut ips = Vec::new();
let mut candidates = Vec::new();
for iface in netdev::get_interfaces() {
if iface.is_up() && !iface.is_loopback() {
for ip in iface.ipv4 {
let addr = ip.addr();
if !addr.is_loopback() && !addr.is_unspecified() {
ips.push(std::net::IpAddr::V4(addr));
}
if !is_candidate_interface(&iface) {
continue;
}
for ip in &iface.ipv4 {
let addr = ip.addr();
if is_candidate_ipv4(addr) {
let priority = ip_priority(&iface, std::net::IpAddr::V4(addr));
candidates.push((priority, std::net::IpAddr::V4(addr)));
}
for ip in iface.ipv6 {
let addr = ip.addr();
if !addr.is_loopback() && !addr.is_unspecified() {
ips.push(std::net::IpAddr::V6(addr));
}
}
for ip in &iface.ipv6 {
let addr = ip.addr();
if is_candidate_ip(std::net::IpAddr::V6(addr)) {
let priority = ip_priority(&iface, std::net::IpAddr::V6(addr));
candidates.push((priority, std::net::IpAddr::V6(addr)));
}
}
}
// Sort descending by priority so best interface IP is candidate #0
candidates.sort_by_key(|b| std::cmp::Reverse(b.0));
let mut seen = std::collections::HashSet::new();
let mut ips = Vec::new();
for (_, ip) in candidates {
if seen.insert(ip) {
ips.push(ip);
}
}
ips
}
pub(crate) fn is_candidate_interface(iface: &netdev::Interface) -> bool {
if !iface.is_up() || iface.is_loopback() {
return false;
}
let name = iface.name.to_lowercase();
let friendly = iface
.friendly_name
.as_deref()
.map(str::to_lowercase)
.unwrap_or_default();
let desc = iface
.description
.as_deref()
.map(str::to_lowercase)
.unwrap_or_default();
// Skip Tsunagi / Wintun overlay interfaces
if name.starts_with("tsun")
|| name.contains("tsunagi")
|| name.contains("wintun")
|| friendly.starts_with("tsun")
|| friendly.contains("tsunagi")
|| friendly.contains("wintun")
|| desc.contains("wintun")
|| desc.contains("tsunagi")
{
return false;
}
// Skip container, VM, and virtual bridge interfaces
if name.starts_with("docker")
|| name.starts_with("br-")
|| name.starts_with("veth")
|| name.starts_with("virbr")
|| name.starts_with("cni")
|| name.starts_with("flannel")
|| name.starts_with("cali")
|| name.starts_with("kube")
|| name.starts_with("dummy")
|| name.starts_with("lxc")
|| name.starts_with("podman")
|| friendly.contains("vethernet")
|| friendly.contains("vboxnet")
|| friendly.contains("vmnet")
|| desc.contains("virtualbox")
|| desc.contains("hyper-v")
|| desc.contains("vmware")
{
return false;
}
true
}
pub(crate) fn is_candidate_ipv4(addr: std::net::Ipv4Addr) -> bool {
if addr.is_loopback() || addr.is_unspecified() || addr.is_link_local() || addr.is_broadcast() {
return false;
}
if addr.is_documentation() {
return false;
}
true
}
pub(crate) fn is_candidate_ip(addr: std::net::IpAddr) -> bool {
match addr {
std::net::IpAddr::V4(v4) => is_candidate_ipv4(v4),
std::net::IpAddr::V6(v6) => {
if v6.is_loopback() || v6.is_unspecified() {
return false;
}
let segments = v6.segments();
// Filter link-local fe80::/10
if (segments[0] & 0xffc0) == 0xfe80 {
return false;
}
true
}
}
}
pub(crate) fn ip_priority(iface: &netdev::Interface, ip: std::net::IpAddr) -> u32 {
let has_gateway = iface.gateway.is_some();
let base = if has_gateway { 100 } else { 0 };
match ip {
std::net::IpAddr::V4(v4) => {
let octets = v4.octets();
// Tailscale: 100.64.0.0/10
let is_tailscale = octets[0] == 100 && (octets[1] & 0xc0) == 64;
let is_private = v4.is_private();
if !is_private && !is_tailscale {
base + 50
} else if has_gateway && is_private {
base + 40
} else if is_tailscale {
base + 30
} else {
base + 10
}
}
std::net::IpAddr::V6(v6) => {
let segments = v6.segments();
let is_global = (segments[0] & 0xe000) == 0x2000;
if is_global { base + 45 } else { base + 20 }
}
}
}
/// The codec implementing cryptographic transformation and announcements for `tcp-tls`.
pub struct TcpTlsCodec {
local_id: EndpointId,
@@ -194,8 +315,10 @@ impl TunnelCodec for TcpTlsCodec {
.map_err(|err| PluginError::Rejected(format!("invalid announcement: {err}")))?;
for ip in &announcement.addrs {
self.transport
.set_peer_addr(peer, std::net::SocketAddr::new(*ip, announcement.port));
if is_candidate_ip(*ip) {
self.transport
.set_peer_addr(peer, std::net::SocketAddr::new(*ip, announcement.port));
}
}
let peer_noise_pubkey = PublicKey::from(announcement.noise_public);
@@ -267,3 +390,141 @@ impl TunnelCodec for TcpTlsCodec {
/// The complete `tcp-tls` plugin.
pub type TcpTlsPlugin = GenericTunnelPlugin<TcpTlsCodec>;
#[cfg(test)]
mod tests {
use super::*;
use std::net::{Ipv4Addr, Ipv6Addr};
#[test]
fn test_is_candidate_interface_filters_overlay_and_bridges() {
let mut iface = netdev::Interface::dummy();
iface.flags = 1; // UP
// Standard Ethernet - should pass
iface.name = "eth0".to_string();
assert!(is_candidate_interface(&iface));
// Windows Ethernet with GUID name - should pass
iface.name = "{B6149D2C-1234-5678-9ABC-DEF012345678}".to_string();
iface.friendly_name = Some("Ethernet".to_string());
iface.description = Some("Intel(R) Ethernet Connection".to_string());
assert!(is_candidate_interface(&iface));
// Linux tsun0 overlay - must be filtered
iface.name = "tsun0".to_string();
iface.friendly_name = None;
iface.description = None;
assert!(!is_candidate_interface(&iface));
// Windows Wintun overlay (GUID name with tsun0 friendly name) - must be filtered!
iface.name = "{72D85D76-F1A5-42AC-BC49-2169527CDDDF}".to_string();
iface.friendly_name = Some("tsun0".to_string());
iface.description = Some("Wintun Userspace Tunnel".to_string());
assert!(!is_candidate_interface(&iface));
// Windows Wintun overlay with description containing Wintun - must be filtered!
iface.friendly_name = Some("Local Area Connection 2".to_string());
iface.description = Some("Wintun Userspace Tunnel".to_string());
assert!(!is_candidate_interface(&iface));
// Docker bridge - must be filtered
iface.name = "docker0".to_string();
iface.friendly_name = None;
iface.description = None;
assert!(!is_candidate_interface(&iface));
// Docker network bridge br-* - must be filtered
iface.name = "br-4f6d3a9b1c2e".to_string();
assert!(!is_candidate_interface(&iface));
// Virtual ethernet veth* - must be filtered
iface.name = "veth12345".to_string();
assert!(!is_candidate_interface(&iface));
// Libvirt NAT bridge virbr* - must be filtered
iface.name = "virbr0".to_string();
assert!(!is_candidate_interface(&iface));
// CNI bridge - must be filtered
iface.name = "cni0".to_string();
assert!(!is_candidate_interface(&iface));
// Hyper-V / WSL vEthernet switch on Windows - must be filtered
iface.name = "{A1B2C3D4-E5F6-7890-1234-567890ABCDEF}".to_string();
iface.friendly_name = Some("vEthernet (WSL)".to_string());
iface.description = Some("Hyper-V Virtual Ethernet Adapter".to_string());
assert!(!is_candidate_interface(&iface));
// Down interface - must be filtered
let mut down_iface = netdev::Interface::dummy();
down_iface.flags = 0; // Not UP
down_iface.name = "eth0".to_string();
assert!(!is_candidate_interface(&down_iface));
}
#[test]
fn test_is_candidate_ip() {
// Valid LAN IP
assert!(is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
192, 168, 1, 117
))));
// Valid Tailscale IP
assert!(is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
100, 77, 155, 120
))));
// Valid Public IP
assert!(is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
1, 1, 1, 1
))));
// Valid IPv6 GUA
assert!(is_candidate_ip(std::net::IpAddr::V6(Ipv6Addr::new(
0x2a01, 0x4b00, 0xb8e3, 0x4e00, 0, 0, 0, 1
))));
// Loopback - must be rejected
assert!(!is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
127, 0, 0, 1
))));
assert!(!is_candidate_ip(std::net::IpAddr::V6(Ipv6Addr::LOCALHOST)));
// Unspecified - must be rejected
assert!(!is_candidate_ip(std::net::IpAddr::V4(
Ipv4Addr::UNSPECIFIED
)));
assert!(!is_candidate_ip(std::net::IpAddr::V6(
Ipv6Addr::UNSPECIFIED
)));
// Link-local - must be rejected
assert!(!is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
169, 254, 1, 1
))));
assert!(!is_candidate_ip(std::net::IpAddr::V6(Ipv6Addr::new(
0xfe80, 0, 0, 0, 0, 0x1ff, 0xfe00, 1
))));
// Broadcast - must be rejected
assert!(!is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::BROADCAST)));
}
#[test]
fn test_ip_priority_ordering() {
let mut iface_lan = netdev::Interface::dummy();
iface_lan.flags = 1;
iface_lan.gateway = Some(netdev::net::device::NetworkDevice::new());
let mut iface_vpn = netdev::Interface::dummy();
iface_vpn.flags = 1;
iface_vpn.name = "tailscale0".to_string();
let lan_ip = std::net::IpAddr::V4(Ipv4Addr::new(192, 168, 1, 117));
let tailscale_ip = std::net::IpAddr::V4(Ipv4Addr::new(100, 77, 155, 120));
let p_lan = ip_priority(&iface_lan, lan_ip);
let p_ts = ip_priority(&iface_vpn, tailscale_ip);
// LAN IP with gateway should score higher than VPN IP without gateway
assert!(p_lan > p_ts);
}
}
+35
View File
@@ -0,0 +1,35 @@
[package]
name = "tsunagi-wireguard"
version = "0.1.0"
description = "Pure WireGuard direct UDP transport and protocol plugin for tsunagi mesh networks."
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
tsunagi = { path = "../tsunagi", version = "0.1.0" }
iroh.workspace = true
tokio = { workspace = true, features = ["net", "io-util", "time", "sync", "macros"] }
serde.workspace = true
postcard.workspace = true
bytes.workspace = true
thiserror.workspace = true
tracing.workspace = true
data-encoding.workspace = true
hex.workspace = true
rusqlite = { version = "0.40", features = ["bundled"] }
subtle = "2.6"
zeroize = { version = "1.9", features = ["derive"] }
rand = "0.10"
boringtun = { version = "0.7.1", default-features = false }
netdev = "0.46"
[dev-dependencies]
tsunagi = { path = "../tsunagi", features = ["testing"] }
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros"] }
tempfile.workspace = true
tracing-subscriber.workspace = true
[lints]
workspace = true
@@ -0,0 +1,142 @@
//! What a pure WireGuard peer tells the network about itself.
//!
//! Exchanged via [`tsunagi::dataplane::PluginCapability`]. The announcement
//! carries the WireGuard public key, the listening UDP port, and candidate IP addresses
//! for direct UDP packet transport.
use serde::{Deserialize, Serialize};
use tsunagi::dataplane::PluginError;
use tsunagi::identity::NetworkId;
use crate::keys::WgPublicKey;
/// The wire protocol identifier for pure WireGuard.
pub const WG_PROTOCOL: &str = "wg";
/// Wire version of the `wg` announcement format.
pub const ANNOUNCEMENT_VERSION: u16 = 1;
/// Wire version peers compare during capability exchange.
///
/// This is the same as [`ANNOUNCEMENT_VERSION`]; keeping a separate name
/// makes the intent explicit in the CLI protocol registry.
pub const WG_WIRE_VERSION: u16 = ANNOUNCEMENT_VERSION;
/// Maximum number of IP candidates allowed in an announcement.
pub const MAX_CANDIDATE_ADDRS: usize = 32;
/// What one participant advertises for the pure WireGuard UDP data plane.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct WgAnnouncement {
/// Announcement format version.
pub version: u16,
/// The peer's WireGuard public key.
pub public_key: [u8; 32],
/// The network this key is for.
pub network: [u8; 32],
/// The UDP port the peer is listening on for WireGuard packets.
pub port: u16,
/// Candidate IP addresses where this peer can be reached on `port`.
#[serde(default)]
pub addrs: Vec<std::net::IpAddr>,
}
/// A peer announcement that has been validated against a specific network.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ValidatedAnnouncement {
/// The peer's WireGuard public key.
pub public_key: WgPublicKey,
/// The UDP port the peer listens on.
pub port: u16,
/// Validated candidate IP addresses.
pub addrs: Vec<std::net::IpAddr>,
}
impl WgAnnouncement {
/// Builds this agent's announcement.
pub fn new(
network: NetworkId,
public_key: &WgPublicKey,
port: u16,
addrs: Vec<std::net::IpAddr>,
) -> Self {
Self {
version: ANNOUNCEMENT_VERSION,
public_key: *public_key.as_bytes(),
network: *network.as_bytes(),
port,
addrs,
}
}
/// Encodes the announcement into the opaque capability payload.
pub fn encode(&self) -> Result<Vec<u8>, PluginError> {
postcard::to_allocvec(self)
.map_err(|err| PluginError::Other(format!("cannot encode announcement: {err}")))
}
/// Decodes and validates a payload received from a peer.
pub fn decode_and_validate(
payload: &[u8],
network: NetworkId,
local_key: &WgPublicKey,
) -> Result<ValidatedAnnouncement, PluginError> {
let announcement: Self = postcard::from_bytes(payload)
.map_err(|_| PluginError::Rejected("malformed WireGuard announcement".into()))?;
announcement.validate(network, local_key)
}
fn validate(
self,
network: NetworkId,
local_key: &WgPublicKey,
) -> Result<ValidatedAnnouncement, PluginError> {
if self.version != ANNOUNCEMENT_VERSION {
return Err(PluginError::Rejected(format!(
"peer speaks WireGuard announcement version {} but this build speaks \
{ANNOUNCEMENT_VERSION}; one of the two needs updating",
self.version
)));
}
let public_key = WgPublicKey::from_bytes(self.public_key);
if public_key.is_zero() {
return Err(PluginError::Rejected(
"WireGuard public key is all zeroes".into(),
));
}
if &public_key == local_key {
return Err(PluginError::Rejected(
"peer announced this agent's own WireGuard key".into(),
));
}
if self.network != *network.as_bytes() {
return Err(PluginError::Rejected(
"announcement is for a different network".into(),
));
}
if self.port == 0 {
return Err(PluginError::Rejected("WireGuard port cannot be 0".into()));
}
if self.addrs.len() > MAX_CANDIDATE_ADDRS {
return Err(PluginError::Rejected(format!(
"too many candidate addresses: {} > {MAX_CANDIDATE_ADDRS}",
self.addrs.len()
)));
}
let mut valid_addrs = Vec::new();
for addr in self.addrs {
if !addr.is_unspecified() && !valid_addrs.contains(&addr) {
valid_addrs.push(addr);
}
}
Ok(ValidatedAnnouncement {
public_key,
port: self.port,
addrs: valid_addrs,
})
}
}
+686
View File
@@ -0,0 +1,686 @@
//! Userspace WireGuard device implementation over direct UDP links.
//!
//! Uses [`boringtun::noise::Tunn`] state machine to encrypt and decrypt
//! standard WireGuard protocol packets directly over a [`SharedLink`].
use std::collections::{HashMap, VecDeque};
use std::net::Ipv4Addr;
use std::sync::atomic::{AtomicU32, AtomicU64, Ordering};
use std::sync::{Arc, Mutex, RwLock};
use std::time::Duration;
use boringtun::noise::{Tunn, TunnResult};
use bytes::Bytes;
use iroh::EndpointId;
use tokio::task::JoinHandle;
use tsunagi::dataplane::routing::{FlowId, flow::ip_flow};
use tsunagi::dataplane::transport::{SharedLink, TransportError};
use tsunagi::dataplane::{PacketSink, PluginContext, PluginError};
use tsunagi::identity::NetworkId;
use crate::keys::{WgPublicKey, WgSecretKey};
use crate::transport::WireguardTransport;
const TIMER_INTERVAL: Duration = Duration::from_millis(250);
const SCRATCH: usize = 4096;
/// Maximum consecutive rekey timeout retransmissions before declaring a tunnel dead.
pub const MAX_REKEY_TIMEOUTS: u32 = 3;
/// Overall maximum duration trying to establish or rekey a handshake before declaring it dead.
pub const HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(15);
struct FlowTunnel {
tunn: Tunn,
queued: VecDeque<FlowId>,
handshake_started_at: Option<std::time::Instant>,
rekey_timeouts: u32,
failed: bool,
}
fn is_data(result: &TunnResult<'_>) -> bool {
matches!(result, TunnResult::WriteToNetwork(bytes) if bytes.starts_with(&[4, 0, 0, 0]))
}
impl FlowTunnel {
fn encapsulate<'a>(
&mut self,
packet: &[u8],
scratch: &'a mut [u8],
flow: FlowId,
) -> (TunnResult<'a>, FlowId) {
let result = self.tunn.encapsulate(packet, scratch);
if is_data(&result) {
return (result, flow);
}
if matches!(result, TunnResult::WriteToNetwork(ref b) if !b.is_empty() && b[0] == 1)
&& self.handshake_started_at.is_none()
{
self.handshake_started_at = Some(std::time::Instant::now());
self.rekey_timeouts = 0;
self.failed = false;
}
if self.queued.len() < 256 {
self.queued.push_back(flow);
}
(result, 0)
}
fn decapsulate<'a>(
&mut self,
packet: &[u8],
scratch: &'a mut [u8],
) -> (TunnResult<'a>, FlowId) {
let result = self.tunn.decapsulate(None, packet, scratch);
if self.tunn.time_since_last_handshake().is_some() {
self.handshake_started_at = None;
self.rekey_timeouts = 0;
self.failed = false;
}
let mut flow = 0;
if packet.is_empty() {
if is_data(&result) {
flow = self.queued.pop_front().unwrap_or(0);
} else if let Some(pending) = self.queued.pop_front() {
self.queued.push_back(pending);
}
}
(result, flow)
}
fn update_timers<'a>(&mut self, scratch: &'a mut [u8]) -> (TunnResult<'a>, bool) {
if self.failed {
return (TunnResult::Done, false);
}
let result = self.tunn.update_timers(scratch);
if matches!(
result,
TunnResult::Err(boringtun::noise::errors::WireGuardError::ConnectionExpired)
) || self.tunn.is_expired()
{
self.failed = true;
self.queued.clear();
self.handshake_started_at = None;
self.rekey_timeouts = 0;
return (result, true);
}
if let TunnResult::WriteToNetwork(ref bytes) = result
&& !bytes.is_empty()
&& bytes[0] == 1
&& self.handshake_started_at.is_none()
{
self.handshake_started_at = Some(std::time::Instant::now());
}
if let Some(started) = self.handshake_started_at
&& started.elapsed() >= HANDSHAKE_TIMEOUT
{
self.failed = true;
self.queued.clear();
self.handshake_started_at = None;
self.rekey_timeouts = 0;
return (result, true);
}
(result, false)
}
}
#[derive(Debug, Default)]
struct PeerCounters {
tx_packets: AtomicU64,
tx_bytes: AtomicU64,
rx_packets: AtomicU64,
rx_bytes: AtomicU64,
dropped_wrong_source: AtomicU64,
dropped_oversize: AtomicU64,
dropped_no_session: AtomicU64,
protocol_errors: AtomicU64,
}
/// A snapshot of one peer's tunnel counters.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct PeerStats {
/// Plaintext packets encrypted and sent to this peer.
pub tx_packets: u64,
/// Plaintext bytes encrypted and sent to this peer.
pub tx_bytes: u64,
/// Plaintext packets decrypted from this peer and given to the OS.
pub rx_packets: u64,
/// Plaintext bytes decrypted from this peer and given to the OS.
pub rx_bytes: u64,
/// Packets dropped because their source was not this peer's address.
pub dropped_wrong_source: u64,
/// Packets dropped because they did not fit in one link datagram.
pub dropped_oversize: u64,
/// Packets dropped because there was no session to encrypt with yet.
pub dropped_no_session: u64,
/// WireGuard protocol errors, including packets that failed to decrypt.
pub protocol_errors: u64,
}
/// Whether a peer's tunnel has completed a handshake.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct PeerHealth {
/// Time since the last successful WireGuard handshake.
pub since_handshake: Option<Duration>,
}
impl PeerHealth {
/// Whether the tunnel has ever completed a handshake.
pub fn is_up(&self) -> bool {
self.since_handshake.is_some()
}
}
struct Peer {
endpoint_id: EndpointId,
public_key: WgPublicKey,
overlay_v4: Mutex<Option<Ipv4Addr>>,
tunn: Mutex<FlowTunnel>,
link: SharedLink,
counters: Arc<PeerCounters>,
task: Mutex<Option<JoinHandle<()>>>,
}
impl std::fmt::Debug for Peer {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("Peer")
.field("peer", &self.endpoint_id.fmt_short().to_string())
.field("public_key", &self.public_key)
.finish()
}
}
impl Drop for Peer {
fn drop(&mut self) {
if let Ok(mut guard) = self.task.lock()
&& let Some(task) = guard.take()
{
task.abort();
}
}
}
impl Peer {
fn stats(&self) -> PeerStats {
PeerStats {
tx_packets: self.counters.tx_packets.load(Ordering::Relaxed),
tx_bytes: self.counters.tx_bytes.load(Ordering::Relaxed),
rx_packets: self.counters.rx_packets.load(Ordering::Relaxed),
rx_bytes: self.counters.rx_bytes.load(Ordering::Relaxed),
dropped_wrong_source: self.counters.dropped_wrong_source.load(Ordering::Relaxed),
dropped_oversize: self.counters.dropped_oversize.load(Ordering::Relaxed),
dropped_no_session: self.counters.dropped_no_session.load(Ordering::Relaxed),
protocol_errors: self.counters.protocol_errors.load(Ordering::Relaxed),
}
}
fn health(&self) -> PeerHealth {
let guard = match self.tunn.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
};
PeerHealth {
since_handshake: guard.tunn.time_since_last_handshake(),
}
}
}
/// What a peer's tunnel looks like from outside.
#[derive(Debug, Clone)]
pub struct PeerSummary {
/// The peer's control plane identity.
pub endpoint_id: EndpointId,
/// The peer's WireGuard public key.
pub public_key: WgPublicKey,
/// The overlay address it holds, once agreed.
pub overlay_address_v4: Option<Ipv4Addr>,
/// Whether the tunnel has handshaken.
pub health: PeerHealth,
/// Traffic counters.
pub stats: PeerStats,
/// What the transport reports about the path carrying this tunnel.
pub path: String,
/// Largest datagram the link accepts.
pub max_datagram: usize,
}
struct Inner {
network: NetworkId,
private_key: WgSecretKey,
peers: RwLock<HashMap<WgPublicKey, Arc<Peer>>>,
sink: Arc<dyn PacketSink>,
transport: Arc<WireguardTransport>,
context: PluginContext,
next_index: AtomicU32,
}
impl Inner {
fn on_tunnel_failed(&self, peer: EndpointId, public_key: WgPublicKey) {
self.transport.close_link(self.network, peer);
write_lock(&self.peers).remove(&public_key);
self.context.report_peer_protocol_failure(
self.network,
peer,
crate::announcement::WG_PROTOCOL,
"WireGuard rekey timeout",
);
}
}
impl std::fmt::Debug for Inner {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("Inner")
.field("network", &self.network.fmt_short())
.finish()
}
}
/// The userspace WireGuard tunnels of one network.
#[derive(Debug)]
pub struct WireguardDevice {
inner: Arc<Inner>,
tasks: Vec<JoinHandle<()>>,
}
impl WireguardDevice {
/// Starts the WireGuard tunnels for one network.
pub fn start(
network: NetworkId,
private_key: WgSecretKey,
sink: Arc<dyn PacketSink>,
transport: Arc<WireguardTransport>,
context: PluginContext,
) -> Self {
let inner = Arc::new(Inner {
network,
private_key,
peers: RwLock::new(HashMap::new()),
sink,
transport,
context,
next_index: AtomicU32::new(1),
});
let timers = tokio::spawn(drive_timers(Arc::clone(&inner)));
Self {
inner,
tasks: vec![timers],
}
}
/// Encrypts an IP packet and sends it to a peer over its direct link.
pub fn carry(&self, peer: EndpointId, packet: &[u8]) -> bool {
let found = read_lock(&self.inner.peers)
.values()
.find(|candidate| candidate.endpoint_id == peer)
.cloned();
let Some(peer) = found else {
return false;
};
let mut scratch = [0u8; SCRATCH];
let flow = ip_flow(packet);
let result = {
let mut tunn = match peer.tunn.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
};
let (result, selected_flow) = tunn.encapsulate(packet, &mut scratch, flow);
match result {
TunnResult::WriteToNetwork(out) => Some((out.len(), selected_flow)),
_ => None,
}
};
let Some((len, selected_flow)) = result else {
peer.counters
.dropped_no_session
.fetch_add(1, Ordering::Relaxed);
return false;
};
send_to_peer_flow(&peer, &scratch[..len], selected_flow);
peer.counters.tx_packets.fetch_add(1, Ordering::Relaxed);
peer.counters
.tx_bytes
.fetch_add(packet.len() as u64, Ordering::Relaxed);
true
}
/// Adds or replaces a peer and starts its tunnel.
pub fn add_peer(
&self,
endpoint_id: EndpointId,
public_key: WgPublicKey,
overlay_v4: Option<Ipv4Addr>,
link: SharedLink,
keepalive: Option<u16>,
) -> Result<(), PluginError> {
if public_key == self.inner.private_key.public() {
return Err(PluginError::Rejected(
"refusing to add ourselves as a WireGuard peer".into(),
));
}
let index = self.inner.next_index.fetch_add(1, Ordering::Relaxed);
let tunn = Tunn::new(
self.inner.private_key.to_static_secret(),
public_key.into_x25519(),
None,
keepalive,
index,
None,
);
let peer = Arc::new(Peer {
endpoint_id,
public_key,
overlay_v4: Mutex::new(overlay_v4),
tunn: Mutex::new(FlowTunnel {
tunn,
queued: VecDeque::new(),
handshake_started_at: None,
rekey_timeouts: 0,
failed: false,
}),
link,
counters: Arc::new(PeerCounters::default()),
task: Mutex::new(None),
});
let task = tokio::spawn(read_from_link(Arc::clone(&self.inner), Arc::clone(&peer)));
if let Ok(mut guard) = peer.task.lock() {
*guard = Some(task);
}
write_lock(&self.inner.peers).insert(public_key, Arc::clone(&peer));
kick_handshake(&peer);
Ok(())
}
/// Removes a peer and stops its tunnel.
pub fn remove_peer(&self, public_key: &WgPublicKey) {
write_lock(&self.inner.peers).remove(public_key);
}
/// Removes every peer whose key is not in `keep`.
pub fn retain_peers(&self, keep: &[WgPublicKey]) {
let stale: Vec<WgPublicKey> = read_lock(&self.inner.peers)
.keys()
.filter(|key| !keep.contains(key))
.copied()
.collect();
for key in stale {
self.remove_peer(&key);
}
}
/// Whether a peer's tunnel exists.
pub fn has_peer(&self, public_key: &WgPublicKey) -> bool {
read_lock(&self.inner.peers).contains_key(public_key)
}
/// A snapshot of every peer.
pub fn peers(&self) -> Vec<PeerSummary> {
let mut peers: Vec<PeerSummary> = read_lock(&self.inner.peers)
.values()
.map(|peer| {
let overlay_v4 = match peer.overlay_v4.lock() {
Ok(guard) => *guard,
Err(poisoned) => *poisoned.into_inner(),
};
PeerSummary {
endpoint_id: peer.endpoint_id,
public_key: peer.public_key,
overlay_address_v4: overlay_v4,
health: peer.health(),
stats: peer.stats(),
path: peer.link.path_description(),
max_datagram: peer.link.max_datagram_size(),
}
})
.collect();
peers.sort_by_key(|peer| peer.public_key);
peers
}
}
impl Drop for WireguardDevice {
fn drop(&mut self) {
for task in &self.tasks {
task.abort();
}
}
}
fn read_lock<T>(lock: &RwLock<T>) -> std::sync::RwLockReadGuard<'_, T> {
match lock.read() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
fn write_lock<T>(lock: &RwLock<T>) -> std::sync::RwLockWriteGuard<'_, T> {
match lock.write() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
fn kick_handshake(peer: &Peer) {
let mut scratch = vec![0u8; SCRATCH];
let len = {
let mut tunn = match peer.tunn.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
};
match tunn.tunn.format_handshake_initiation(&mut scratch, false) {
TunnResult::WriteToNetwork(out) => {
tunn.handshake_started_at = Some(std::time::Instant::now());
tunn.rekey_timeouts = 0;
tunn.failed = false;
Some(out.len())
}
_ => None,
}
};
if let Some(len) = len {
send_to_peer(peer, &scratch[..len]);
}
}
fn send_to_peer(peer: &Peer, payload: &[u8]) {
send_to_peer_flow(peer, payload, 0);
}
fn send_to_peer_flow(peer: &Peer, payload: &[u8], flow: FlowId) {
match peer.link.send_flow(Bytes::copy_from_slice(payload), flow) {
Ok(()) => {}
Err(TransportError::TooLarge { .. }) => {
peer.counters
.dropped_oversize
.fetch_add(1, Ordering::Relaxed);
}
Err(TransportError::Closed) => {}
Err(err) => {
tracing::trace!(%err, "dropping a WireGuard packet the link refused");
}
}
}
async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
let mut scratch = vec![0u8; SCRATCH];
loop {
let Some(datagram) = peer.link.recv().await else {
return;
};
let mut input: Option<&[u8]> = Some(&datagram);
loop {
let outcome = {
let mut tunn = match peer.tunn.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
};
let (result, flow) = tunn.decapsulate(input.unwrap_or(&[]), &mut scratch);
match result {
TunnResult::WriteToNetwork(out) => Outcome::ToNetwork(out.len(), flow),
TunnResult::WriteToTunnelV6(out, _) => Outcome::ToTunnel(out.len()),
TunnResult::WriteToTunnelV4(out, _) => Outcome::ToTunnel(out.len()),
TunnResult::Done => Outcome::Done,
TunnResult::Err(err) => {
tracing::trace!(?err, "WireGuard decapsulation failed");
Outcome::Failed
}
}
};
match outcome {
Outcome::ToNetwork(len, flow) => {
send_to_peer_flow(&peer, &scratch[..len], flow);
input = None;
continue;
}
Outcome::ToTunnel(len) => {
let payload = Bytes::copy_from_slice(&scratch[..len]);
inner
.sink
.deliver(inner.network, peer.endpoint_id, payload)
.await;
peer.counters.rx_packets.fetch_add(1, Ordering::Relaxed);
peer.counters
.rx_bytes
.fetch_add(len as u64, Ordering::Relaxed);
break;
}
Outcome::Failed => {
peer.counters
.protocol_errors
.fetch_add(1, Ordering::Relaxed);
break;
}
Outcome::Done => break,
}
}
}
}
enum Outcome {
ToNetwork(usize, FlowId),
ToTunnel(usize),
Done,
Failed,
}
async fn drive_timers(inner: Arc<Inner>) {
let mut ticker = tokio::time::interval(TIMER_INTERVAL);
ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
loop {
ticker.tick().await;
let peers: Vec<Arc<Peer>> = read_lock(&inner.peers).values().cloned().collect();
for peer in peers {
let mut scratch = vec![0u8; SCRATCH];
let (len, failed) = {
let mut tunn = match peer.tunn.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
};
let (result, failed) = tunn.update_timers(&mut scratch);
let len = match result {
TunnResult::WriteToNetwork(out) => Some(out.len()),
TunnResult::Err(err) => {
tracing::trace!(?err, "WireGuard timer error");
None
}
_ => None,
};
(len, failed)
};
if let Some(len) = len {
send_to_peer(&peer, &scratch[..len]);
}
if failed {
tracing::warn!(
peer = %peer.endpoint_id.fmt_short(),
"WireGuard tunnel failed due to rekey timeout, triggering fallback"
);
inner.on_tunnel_failed(peer.endpoint_id, peer.public_key);
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_flow_tunnel_rekey_timeout_triggers_failure() {
let my_secret = WgSecretKey::generate();
let their_secret = WgSecretKey::generate();
let their_public = their_secret.public();
let tunn = Tunn::new(
my_secret.to_static_secret(),
their_public.into_x25519(),
None,
None,
1,
None,
);
let mut tunnel = FlowTunnel {
tunn,
queued: VecDeque::new(),
handshake_started_at: Some(std::time::Instant::now() - Duration::from_secs(16)),
rekey_timeouts: 0,
failed: false,
};
let mut scratch = vec![0u8; SCRATCH];
// When handshake timeout exceeds HANDSHAKE_TIMEOUT (15s), update_timers returns true (failed)
let (_, failed) = tunnel.update_timers(&mut scratch);
assert!(failed, "handshake timeout should report failure");
assert!(tunnel.handshake_started_at.is_none());
assert!(tunnel.failed, "tunnel should be marked as failed");
// Subsequent timer tick must not report failure again
let (_, failed_again) = tunnel.update_timers(&mut scratch);
assert!(
!failed_again,
"subsequent timer tick should not re-trigger failure"
);
}
#[test]
fn test_flow_tunnel_reset_on_handshake() {
let my_secret = WgSecretKey::generate();
let their_secret = WgSecretKey::generate();
let their_public = their_secret.public();
let tunn = Tunn::new(
my_secret.to_static_secret(),
their_public.into_x25519(),
None,
None,
1,
None,
);
let mut tunnel = FlowTunnel {
tunn,
queued: VecDeque::new(),
handshake_started_at: Some(std::time::Instant::now()),
rekey_timeouts: 2,
failed: true,
};
let mut scratch = vec![0u8; SCRATCH];
let _ = tunnel.decapsulate(&[], &mut scratch);
// decapsulate without handshake response doesn't establish session
assert_eq!(tunnel.rekey_timeouts, 2);
}
}
+160
View File
@@ -0,0 +1,160 @@
//! WireGuard key material for pure WireGuard transport.
//!
//! Keys belong to this protocol plugin alone. They are not derived from
//! the iroh device key and not derived from the network secret, so compromising
//! or rotating one does not affect the others.
//!
//! Keys are X25519, encoded the way WireGuard encodes them: standard base64
//! with padding, 44 characters.
use boringtun::x25519;
use data_encoding::BASE64;
use zeroize::{Zeroize, Zeroizing};
use tsunagi::dataplane::PluginError;
/// Length of a raw WireGuard key in bytes.
pub const KEY_LEN: usize = 32;
/// Length of the base64 text form of a key.
pub const KEY_TEXT_LEN: usize = 44;
/// Applies the X25519 clamping WireGuard applies to private keys.
fn clamp(bytes: &mut [u8; KEY_LEN]) {
bytes[0] &= 248;
bytes[31] &= 127;
bytes[31] |= 64;
}
/// A WireGuard public key.
///
/// Public, safe to log, and the identity a peer is known by inside the overlay.
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct WgPublicKey([u8; KEY_LEN]);
impl WgPublicKey {
/// Wraps raw key bytes.
pub fn from_bytes(bytes: [u8; KEY_LEN]) -> Self {
Self(bytes)
}
/// The raw key bytes.
pub fn as_bytes(&self) -> &[u8; KEY_LEN] {
&self.0
}
/// The key in the form the WireGuard implementation expects.
pub(crate) fn into_x25519(self) -> x25519::PublicKey {
x25519::PublicKey::from(self.0)
}
/// Whether this is the all-zero key, which is never a valid peer.
pub fn is_zero(&self) -> bool {
self.0 == [0u8; KEY_LEN]
}
/// The base64 form WireGuard uses.
pub fn encode(&self) -> String {
BASE64.encode(&self.0)
}
/// Parses the base64 form WireGuard uses.
pub fn decode(text: &str) -> Result<Self, PluginError> {
if text.len() != KEY_TEXT_LEN {
return Err(PluginError::Rejected(format!(
"a WireGuard key is {KEY_TEXT_LEN} base64 characters, got {}",
text.len()
)));
}
let raw = BASE64
.decode(text.as_bytes())
.map_err(|_| PluginError::Rejected("key is not valid base64".into()))?;
let bytes = <[u8; KEY_LEN]>::try_from(raw.as_slice())
.map_err(|_| PluginError::Rejected("key is not 32 bytes".into()))?;
Ok(Self(bytes))
}
/// A short prefix for logs and diagnostics.
pub fn fmt_short(&self) -> String {
self.encode().chars().take(8).collect()
}
}
impl std::fmt::Display for WgPublicKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.encode())
}
}
impl std::fmt::Debug for WgPublicKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "WgPublicKey({})", self.fmt_short())
}
}
/// A WireGuard private key.
///
/// Clamped on creation so it matches what `wg genkey` produces. Zeroized on
/// drop, and prints as `[redacted]` in Debug so it cannot leak into logs.
#[derive(Clone)]
pub struct WgSecretKey {
bytes: Zeroizing<[u8; KEY_LEN]>,
public: WgPublicKey,
}
impl WgSecretKey {
/// Generates a fresh clamped random private key.
pub fn generate() -> Self {
let mut bytes = [0u8; KEY_LEN];
rand::fill(&mut bytes);
clamp(&mut bytes);
let secret = x25519::StaticSecret::from(bytes);
let public = WgPublicKey(*x25519::PublicKey::from(&secret).as_bytes());
Self {
bytes: Zeroizing::new(bytes),
public,
}
}
/// Wraps existing key bytes, clamping them to a valid X25519 private key.
pub fn from_bytes(raw: &[u8; KEY_LEN]) -> Self {
let mut bytes = *raw;
clamp(&mut bytes);
let secret = x25519::StaticSecret::from(bytes);
let public = WgPublicKey(*x25519::PublicKey::from(&secret).as_bytes());
Self {
bytes: Zeroizing::new(bytes),
public,
}
}
/// The matching public key.
pub fn public(&self) -> WgPublicKey {
self.public
}
/// The key in the form the WireGuard implementation expects.
pub(crate) fn to_static_secret(&self) -> x25519::StaticSecret {
x25519::StaticSecret::from(*self.bytes)
}
/// The raw key bytes.
pub fn as_bytes(&self) -> &[u8; KEY_LEN] {
&self.bytes
}
}
impl Drop for WgSecretKey {
fn drop(&mut self) {
self.bytes.zeroize();
}
}
impl std::fmt::Debug for WgSecretKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("WgSecretKey")
.field("bytes", &"[redacted]")
.field("public", &self.public)
.finish()
}
}
+28
View File
@@ -0,0 +1,28 @@
//! The pure WireGuard protocol plugin for Tsunagi: WireGuard cryptography over direct UDP.
//!
//! Deliberately a separate crate from the core, ensuring clear boundaries:
//! - Direct UDP datagram transport without QUIC encapsulation or double-encryption.
//! - Standard WireGuard cryptography via `boringtun`.
//! - Peer discovery and key exchange managed through the Tsunagi control plane.
#![deny(unsafe_code)]
#![warn(missing_docs)]
pub mod announcement;
pub mod device;
pub mod keys;
pub mod plugin;
pub mod store;
pub mod transport;
pub use announcement::{
ANNOUNCEMENT_VERSION, ValidatedAnnouncement, WG_PROTOCOL, WG_WIRE_VERSION, WgAnnouncement,
};
pub use device::{PeerHealth, PeerStats, PeerSummary, WireguardDevice};
pub use keys::{WgPublicKey, WgSecretKey};
pub use plugin::{
DEFAULT_MTU, MIN_MTU, NetworkOverview, PeerOverview, WIREGUARD_OVERHEAD, WireguardConfig,
WireguardPlugin, local_ip_candidates,
};
pub use store::WgKeyStore;
pub use transport::{DEFAULT_PORT, MAX_UDP_DATAGRAM_SIZE, WireguardLink, WireguardTransport};
File diff suppressed because it is too large. Load diff
+160
View File
@@ -0,0 +1,160 @@
//! The plugin's own key store.
//!
//! Deliberately a separate SQLite file from the agent's `state.sqlite`: plugin
//! keys are not the iroh identity and not the network secret, and their
//! lifecycle is the plugin's business alone.
//!
//! One key per network, so a participant presents a different WireGuard
//! identity in each network it belongs to.
use std::path::{Path, PathBuf};
use std::sync::Mutex;
use rusqlite::{Connection, OptionalExtension, params};
use tsunagi::dataplane::PluginError;
use tsunagi::identity::NetworkId;
use crate::keys::{KEY_LEN, WgSecretKey};
/// Schema version written by this build.
pub const SCHEMA_VERSION: i64 = 1;
/// Per-network WireGuard private keys.
#[derive(Debug)]
pub struct WgKeyStore {
conn: Mutex<Connection>,
path: PathBuf,
}
impl WgKeyStore {
/// Opens, creating the file and its directory if needed.
pub fn open(path: impl AsRef<Path>) -> Result<Self, PluginError> {
let path = path.as_ref().to_path_buf();
if let Some(parent) = path.parent() {
tsunagi::storage::create_dir(parent)
.map_err(|err| PluginError::Other(format!("cannot create {parent:?}: {err}")))?;
}
let existed = path.exists();
let conn = Connection::open(&path).map_err(|err| {
PluginError::Other(format!("cannot open the WireGuard key store: {err}"))
})?;
tsunagi::storage::restrict_path_permissions(&path)
.map_err(|err| PluginError::Other(format!("cannot secure the key store: {err}")))?;
conn.busy_timeout(std::time::Duration::from_secs(5))
.and_then(|()| conn.pragma_update(None, "journal_mode", "WAL"))
.and_then(|()| conn.pragma_update(None, "synchronous", "NORMAL"))
.map_err(|err| PluginError::Other(format!("cannot configure the key store: {err}")))?;
if existed {
let integrity: String = conn
.query_row("PRAGMA integrity_check", [], |row| row.get(0))
.map_err(|err| {
PluginError::Other(format!("WireGuard key store is unusable: {err}"))
})?;
if integrity != "ok" {
return Err(PluginError::Other(format!(
"WireGuard key store at {} is corrupt and will not be recreated: {integrity}",
path.display()
)));
}
}
let found: i64 = conn
.query_row("PRAGMA user_version", [], |row| row.get(0))
.map_err(|err| PluginError::Other(format!("cannot read the schema version: {err}")))?;
if found > SCHEMA_VERSION {
return Err(PluginError::Other(format!(
"WireGuard key store schema {found} is newer than {SCHEMA_VERSION}"
)));
}
if found < SCHEMA_VERSION {
conn.execute_batch(
"BEGIN;
CREATE TABLE IF NOT EXISTS network_keys (
network_id BLOB PRIMARY KEY,
secret BLOB NOT NULL,
created_at INTEGER NOT NULL
);
PRAGMA user_version = 1;
COMMIT;",
)
.map_err(|err| PluginError::Other(format!("cannot create the schema: {err}")))?;
}
Ok(Self {
conn: Mutex::new(conn),
path,
})
}
/// Path of the underlying file.
pub fn path(&self) -> &Path {
&self.path
}
/// Loads the stored key for `network`, or generates, stores and returns a fresh one.
pub fn load_or_create(&self, network: NetworkId) -> Result<WgSecretKey, PluginError> {
let guard = self
.conn
.lock()
.map_err(|_| PluginError::Other("key store lock is poisoned".into()))?;
let existing: Option<Vec<u8>> = guard
.query_row(
"SELECT secret FROM network_keys WHERE network_id = ?1",
params![network.as_bytes().as_slice()],
|row| row.get(0),
)
.optional()
.map_err(|err| PluginError::Other(format!("cannot read the key store: {err}")))?;
if let Some(secret) = existing {
let bytes = <[u8; KEY_LEN]>::try_from(secret.as_slice()).map_err(|_| {
PluginError::Other(format!(
"stored WireGuard key for {} is {} bytes, expected {KEY_LEN}",
network.fmt_short(),
secret.len()
))
})?;
return Ok(WgSecretKey::from_bytes(&bytes));
}
let fresh = WgSecretKey::generate();
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map_err(|err| PluginError::Other(format!("system clock is broken: {err}")))?
.as_secs() as i64;
guard
.execute(
"INSERT INTO network_keys (network_id, secret, created_at) VALUES (?1, ?2, ?3)",
params![
network.as_bytes().as_slice(),
fresh.as_bytes().as_slice(),
now
],
)
.map_err(|err| PluginError::Other(format!("cannot write the key store: {err}")))?;
Ok(fresh)
}
/// Removes the stored key for `network`.
pub fn forget(&self, network: NetworkId) -> Result<(), PluginError> {
let guard = self
.conn
.lock()
.map_err(|_| PluginError::Other("key store lock is poisoned".into()))?;
guard
.execute(
"DELETE FROM network_keys WHERE network_id = ?1",
params![network.as_bytes().as_slice()],
)
.map_err(|err| {
PluginError::Other(format!("cannot delete from the key store: {err}"))
})?;
Ok(())
}
}
+796
View File
@@ -0,0 +1,796 @@
//! Pure WireGuard UDP packet transport and link implementation.
//!
//! Provides [`WireguardTransport`] implementing [`PacketTransport`]
//! and [`WireguardLink`] implementing [`PacketLink`].
use std::collections::{HashMap, HashSet};
use std::net::SocketAddr;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex, RwLock};
use bytes::Bytes;
use iroh::EndpointId;
use tokio::net::UdpSocket;
use tokio::sync::{Notify, mpsc, oneshot};
use tsunagi::BoxFuture;
use tsunagi::dataplane::transport::{
InboundLink, PacketLink, PacketTransport, SharedLink, TransportError,
};
use tsunagi::identity::{DeviceIdentity, NetworkId};
/// Default UDP port to bind and listen on for pure WireGuard.
pub const DEFAULT_PORT: u16 = 51820;
/// Maximum payload length for a single framed datagram over UDP.
pub const MAX_UDP_DATAGRAM_SIZE: usize = 65507;
const PROBE_MAGIC: &[u8; 19] = b"TSUNAGI_WG_PROBE_V1";
const PROBE_TYPE_PING: u8 = 1;
const PROBE_TYPE_PONG: u8 = 2;
const PROBE_LEN: usize = 19 + 1 + 32 + 32 + 32 + 8; // 124 bytes
fn encode_probe(
msg_type: u8,
sender: EndpointId,
receiver: EndpointId,
network: NetworkId,
cookie: u64,
) -> Vec<u8> {
let mut buf = Vec::with_capacity(PROBE_LEN);
buf.extend_from_slice(PROBE_MAGIC);
buf.push(msg_type);
buf.extend_from_slice(sender.as_bytes());
buf.extend_from_slice(receiver.as_bytes());
buf.extend_from_slice(network.as_bytes());
buf.extend_from_slice(&cookie.to_be_bytes());
buf
}
struct DecodedProbe {
msg_type: u8,
sender: EndpointId,
receiver: EndpointId,
network: NetworkId,
cookie: u64,
}
fn decode_probe(bytes: &[u8]) -> Option<DecodedProbe> {
if bytes.len() < PROBE_LEN || &bytes[..19] != PROBE_MAGIC {
return None;
}
let msg_type = bytes[19];
if msg_type != PROBE_TYPE_PING && msg_type != PROBE_TYPE_PONG {
return None;
}
let sender = EndpointId::from_bytes(&bytes[20..52].try_into().ok()?).ok()?;
let receiver = EndpointId::from_bytes(&bytes[52..84].try_into().ok()?).ok()?;
let network = NetworkId::from_bytes(bytes[84..116].try_into().ok()?);
let cookie = u64::from_be_bytes(bytes[116..124].try_into().ok()?);
Some(DecodedProbe {
msg_type,
sender,
receiver,
network,
cookie,
})
}
/// An authenticated datagram link carried over direct UDP.
#[derive(Debug)]
pub struct WireguardLink {
network: NetworkId,
peer: EndpointId,
target_addr: Arc<RwLock<SocketAddr>>,
candidates: Arc<RwLock<Vec<SocketAddr>>>,
confirmed: Arc<AtomicBool>,
tx: mpsc::Sender<Bytes>,
inbound_tx: mpsc::Sender<Bytes>,
rx: tokio::sync::Mutex<mpsc::Receiver<Bytes>>,
closed_notify: Arc<Notify>,
is_closed: Arc<AtomicBool>,
}
impl WireguardLink {
/// Creates a new `WireguardLink` targeting `target_addr` over `socket`.
pub fn new(
network: NetworkId,
peer: EndpointId,
target_addr: SocketAddr,
candidates: Vec<SocketAddr>,
socket: Arc<UdpSocket>,
) -> Self {
let (tx_out, mut rx_out) = mpsc::channel::<Bytes>(1024);
let (tx_in, rx_in) = mpsc::channel::<Bytes>(1024);
let is_closed = Arc::new(AtomicBool::new(false));
let confirmed = Arc::new(AtomicBool::new(false));
let closed_notify = Arc::new(Notify::new());
let target_holder = Arc::new(RwLock::new(target_addr));
let candidates_holder = Arc::new(RwLock::new(candidates));
let socket_w = Arc::clone(&socket);
let target_r = Arc::clone(&target_holder);
let candidates_r = Arc::clone(&candidates_holder);
let confirmed_r = Arc::clone(&confirmed);
let is_closed_w = Arc::clone(&is_closed);
let closed_notify_w = Arc::clone(&closed_notify);
tokio::spawn(async move {
while let Some(msg) = rx_out.recv().await {
if msg.len() > MAX_UDP_DATAGRAM_SIZE {
continue;
}
// In WireGuard protocol, type 1 is Handshake Initiation (148 bytes).
let is_handshake_init = !msg.is_empty() && msg[0] == 1;
if confirmed_r.load(Ordering::Acquire) && !is_handshake_init {
let target = match target_r.read() {
Ok(g) => *g,
Err(p) => *p.into_inner(),
};
if let Err(err) = socket_w.send_to(&msg, target).await {
tracing::trace!(%err, %target, "failed to send WireGuard UDP datagram");
}
} else {
let targets = match candidates_r.read() {
Ok(g) => g.clone(),
Err(p) => p.into_inner().clone(),
};
if targets.is_empty() {
let target = match target_r.read() {
Ok(g) => *g,
Err(p) => *p.into_inner(),
};
let _ = socket_w.send_to(&msg, target).await;
} else {
for target in targets {
let _ = socket_w.send_to(&msg, target).await;
}
}
}
}
is_closed_w.store(true, Ordering::Release);
closed_notify_w.notify_waiters();
});
Self {
network,
peer,
target_addr: target_holder,
candidates: candidates_holder,
confirmed,
tx: tx_out,
inbound_tx: tx_in,
rx: tokio::sync::Mutex::new(rx_in),
closed_notify,
is_closed,
}
}
/// Delivers an inbound packet received from UDP into this link's receive queue.
pub fn deliver_inbound(&self, payload: Bytes) -> bool {
if self.is_closed.load(Ordering::Acquire) {
return false;
}
self.confirmed.store(true, Ordering::Release);
self.inbound_tx.try_send(payload).is_ok()
}
/// Marks this link as confirmed by a probe response or verified packet.
pub fn confirm(&self) {
self.confirmed.store(true, Ordering::Release);
}
/// Whether this link has had verified two-way direct connectivity.
pub fn is_confirmed(&self) -> bool {
self.confirmed.load(Ordering::Acquire)
}
/// Updates candidate addresses for this peer.
pub fn update_candidates(&self, addrs: impl IntoIterator<Item = SocketAddr>) {
if let Ok(mut guard) = self.candidates.write() {
let new_candidates: Vec<SocketAddr> = addrs
.into_iter()
.filter(|a| !a.ip().is_unspecified())
.collect();
if !new_candidates.is_empty() {
*guard = new_candidates;
}
}
}
/// Updates the target socket address (roaming / NAT hole punch update).
pub fn update_target(&self, new_target: SocketAddr) {
if new_target.ip().is_unspecified() {
return;
}
self.confirmed.store(true, Ordering::Release);
if let Ok(mut guard) = self.target_addr.write()
&& *guard != new_target
{
tracing::debug!(peer = %self.peer.fmt_short(), old = %*guard, new = %new_target, "WireGuard endpoint roamed");
*guard = new_target;
}
}
/// Explicitly closes this link, notifying any pending readers or waiters.
pub fn close(&self) {
if !self.is_closed.swap(true, Ordering::AcqRel) {
self.closed_notify.notify_waiters();
}
}
/// Returns the current target address.
pub fn target_addr(&self) -> SocketAddr {
match self.target_addr.read() {
Ok(g) => *g,
Err(p) => *p.into_inner(),
}
}
}
impl PacketLink for WireguardLink {
fn network(&self) -> NetworkId {
self.network
}
fn peer(&self) -> EndpointId {
self.peer
}
fn max_datagram_size(&self) -> usize {
MAX_UDP_DATAGRAM_SIZE
}
fn send(&self, payload: Bytes) -> Result<(), TransportError> {
if payload.len() > MAX_UDP_DATAGRAM_SIZE {
return Err(TransportError::TooLarge {
size: payload.len(),
limit: MAX_UDP_DATAGRAM_SIZE,
});
}
if self.is_closed.load(Ordering::Acquire) {
return Err(TransportError::Closed);
}
self.tx
.try_send(payload)
.map_err(|_| TransportError::Closed)
}
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
Box::pin(async move {
if self.is_closed.load(Ordering::Acquire) {
return None;
}
tokio::select! {
biased;
_ = self.closed_notify.notified() => None,
msg = async {
let mut guard = self.rx.lock().await;
guard.recv().await
} => msg,
}
})
}
fn closed(&self) -> BoxFuture<'_, ()> {
Box::pin(async move {
if self.is_closed.load(Ordering::Acquire) {
return;
}
self.closed_notify.notified().await;
})
}
fn is_closed(&self) -> bool {
self.is_closed.load(Ordering::Acquire)
}
fn path_description(&self) -> String {
format!("wireguard via {}", self.target_addr())
}
}
type InboundCallback = Box<dyn Fn(InboundLink) + Send + Sync + 'static>;
type PendingProbes = Arc<Mutex<HashMap<(NetworkId, EndpointId, u64), oneshot::Sender<SocketAddr>>>>;
/// Direct UDP WireGuard packet transport.
pub struct WireguardTransport {
local_id: EndpointId,
listen_port: u16,
socket: Arc<UdpSocket>,
peer_addresses: RwLock<HashMap<EndpointId, Vec<SocketAddr>>>,
peer_networks: RwLock<HashMap<EndpointId, HashSet<NetworkId>>>,
links: RwLock<HashMap<(NetworkId, EndpointId), Arc<WireguardLink>>>,
inbound_cb: Arc<Mutex<Option<InboundCallback>>>,
pending_probes: PendingProbes,
}
impl std::fmt::Debug for WireguardTransport {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("WireguardTransport")
.field("local_id", &self.local_id.fmt_short().to_string())
.field("listen_port", &self.listen_port)
.finish()
}
}
impl WireguardTransport {
/// Binds a UDP socket for pure WireGuard transport.
///
/// Attempts `requested_port` (default 51820). If that fails, falls back to an ephemeral port.
pub async fn bind(
identity: &DeviceIdentity,
requested_port: Option<u16>,
) -> Result<Self, Box<dyn std::error::Error + Send + Sync>> {
let target_port = requested_port.unwrap_or(DEFAULT_PORT);
let bind_addr = SocketAddr::from(([0, 0, 0, 0], target_port));
let socket = match UdpSocket::bind(bind_addr).await {
Ok(s) => s,
Err(err) if target_port > 0 => {
tracing::warn!(%err, port = target_port, "failed to bind WireGuard UDP port, falling back to ephemeral port");
UdpSocket::bind(SocketAddr::from(([0, 0, 0, 0], 0))).await?
}
Err(err) => return Err(Box::new(err)),
};
let bound_port = socket.local_addr()?.port();
tracing::info!(port = bound_port, "WireGuard UDP transport bound");
Ok(Self {
local_id: identity.endpoint_id(),
listen_port: bound_port,
socket: Arc::new(socket),
peer_addresses: RwLock::new(HashMap::new()),
peer_networks: RwLock::new(HashMap::new()),
links: RwLock::new(HashMap::new()),
inbound_cb: Arc::new(Mutex::new(None)),
pending_probes: Arc::new(Mutex::new(HashMap::new())),
})
}
/// The local port this transport is listening on.
pub fn bound_port(&self) -> u16 {
self.listen_port
}
/// Associates a peer with a network ID.
pub fn set_peer_network(&self, peer: EndpointId, network: NetworkId) {
if let Ok(mut guard) = self.peer_networks.write() {
guard.entry(peer).or_default().insert(network);
}
}
/// Records an observed UDP target address for a peer.
pub fn set_peer_addr(&self, peer: EndpointId, addr: SocketAddr) {
if !addr.is_ipv4() || addr.ip().is_unspecified() {
return;
}
if let Ok(mut guard) = self.peer_addresses.write() {
let addrs = guard.entry(peer).or_default();
addrs.retain(|a| *a != addr);
addrs.insert(0, addr);
}
}
/// Records multiple observed UDP target addresses for a peer.
pub fn set_peer_addrs(&self, peer: EndpointId, addrs: impl IntoIterator<Item = SocketAddr>) {
if let Ok(mut guard) = self.peer_addresses.write() {
let entry = guard.entry(peer).or_default();
for addr in addrs {
if addr.is_ipv4() && !addr.ip().is_unspecified() && !entry.contains(&addr) {
entry.push(addr);
}
}
}
}
/// Returns candidate socket addresses known for a peer.
pub fn peer_addrs(&self, peer: EndpointId) -> Vec<SocketAddr> {
let guard = match self.peer_addresses.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.get(&peer).cloned().unwrap_or_default()
}
/// Explicitly closes any active direct link for the given network and peer.
pub fn close_link(&self, network: NetworkId, peer: EndpointId) {
let link = {
let guard = match self.links.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.get(&(network, peer)).cloned()
};
if let Some(link) = link {
link.close();
}
}
/// Notifies the transport that a peer's capability was announced.
///
/// When `we_accept` is true, sends a hole-punch probe to candidate addresses
/// to open stateful NAT firewalls, but does NOT create an unverified direct link
/// in the hub before packets actually flow.
pub fn notify_peer_capability(
&self,
network: NetworkId,
peer: EndpointId,
addrs: &[SocketAddr],
we_accept: bool,
) {
self.set_peer_network(peer, network);
let ipv4_addrs: Vec<SocketAddr> = addrs.iter().copied().filter(|a| a.is_ipv4()).collect();
self.set_peer_addrs(peer, ipv4_addrs.iter().copied());
// Update candidate addresses on any existing link for this peer
if let Ok(guard) = self.links.read()
&& let Some(link) = guard.get(&(network, peer))
{
link.update_candidates(ipv4_addrs.iter().copied());
}
// Send hole punching probe if we accept and have candidate addresses.
if we_accept && !ipv4_addrs.is_empty() {
let socket = Arc::clone(&self.socket);
let local_id = self.local_id;
tokio::spawn(async move {
let cookie: u64 = rand::random();
let probe = encode_probe(PROBE_TYPE_PING, local_id, peer, network, cookie);
for addr in ipv4_addrs {
let _ = socket.send_to(&probe, addr).await;
}
});
}
}
/// Runs the receive loop for incoming UDP datagrams.
pub async fn accept_loop<F>(&self, on_inbound: F)
where
F: Fn(InboundLink) + Send + Sync + 'static,
{
{
let mut guard = match self.inbound_cb.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
*guard = Some(Box::new(on_inbound));
}
let socket = Arc::clone(&self.socket);
let mut buf = [0u8; MAX_UDP_DATAGRAM_SIZE];
while let Ok((len, src_addr)) = socket.recv_from(&mut buf).await {
if src_addr.ip().is_unspecified() {
continue;
}
// 1. Direct UDP probe packet handling
if len >= PROBE_LEN
&& &buf[..19] == PROBE_MAGIC
&& let Some(probe) = decode_probe(&buf[..len])
{
if probe.receiver != self.local_id {
continue;
}
if probe.msg_type == PROBE_TYPE_PING {
let is_known = {
let guard = match self.peer_networks.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard
.get(&probe.sender)
.is_some_and(|nets| nets.contains(&probe.network))
};
if is_known {
let pong = encode_probe(
PROBE_TYPE_PONG,
self.local_id,
probe.sender,
probe.network,
probe.cookie,
);
let _ = socket.send_to(&pong, src_addr).await;
self.set_peer_addr(probe.sender, src_addr);
let link = {
let mut guard = match self.links.write() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
if let Some(existing) = guard.get(&(probe.network, probe.sender))
&& !existing.is_closed()
{
existing.update_target(src_addr);
existing.confirm();
None
} else {
let candidates = self.peer_addrs(probe.sender);
let link = Arc::new(WireguardLink::new(
probe.network,
probe.sender,
src_addr,
candidates,
Arc::clone(&self.socket),
));
link.confirm();
guard.insert((probe.network, probe.sender), Arc::clone(&link));
Some(link)
}
};
if let Some(link) = link {
let cb_guard = match self.inbound_cb.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
if let Some(ref cb) = *cb_guard {
cb(InboundLink {
network: probe.network,
peer: probe.sender,
protocol: crate::announcement::WG_PROTOCOL.to_string(),
link,
});
}
}
}
continue;
} else if probe.msg_type == PROBE_TYPE_PONG {
self.set_peer_addr(probe.sender, src_addr);
let waiter = {
let mut probes = match self.pending_probes.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
probes.remove(&(probe.network, probe.sender, probe.cookie))
};
if let Some(tx) = waiter {
let _ = tx.send(src_addr);
}
continue;
}
}
// 2. Data / handshake packet handling
let datagram = Bytes::copy_from_slice(&buf[..len]);
// Identify which peer this packet comes from
let matched_peer = {
let links_guard = match self.links.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
links_guard
.values()
.find(|link| link.target_addr() == src_addr)
.map(|link| link.peer())
.or_else(|| {
let addrs_guard = match self.peer_addresses.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
addrs_guard
.iter()
.find(|(_, addrs)| addrs.contains(&src_addr))
.map(|(peer, _)| *peer)
.or_else(|| {
links_guard
.values()
.find(|link| link.target_addr().ip() == src_addr.ip())
.map(|link| link.peer())
.or_else(|| {
addrs_guard
.iter()
.find(|(_, addrs)| {
addrs.iter().any(|a| a.ip() == src_addr.ip())
})
.map(|(peer, _)| *peer)
})
})
})
};
let Some(peer) = matched_peer else {
continue;
};
self.set_peer_addr(peer, src_addr);
// Deliver to ALL active links for this peer (across all networks this peer shares with us)
let peer_links: Vec<Arc<WireguardLink>> = {
let guard = match self.links.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard
.values()
.filter(|l| l.peer() == peer && !l.is_closed())
.cloned()
.collect()
};
if !peer_links.is_empty() {
for link in peer_links {
link.update_target(src_addr);
link.deliver_inbound(datagram.clone());
}
continue;
}
// If no active link exists for this peer yet, create inbound links for all networks known for this peer
let networks: Vec<NetworkId> = {
let guard = match self.peer_networks.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard
.get(&peer)
.map(|s| s.iter().copied().collect())
.unwrap_or_default()
};
for network in networks {
let link = {
let mut guard = match self.links.write() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
if let Some(existing) = guard.get(&(network, peer))
&& !existing.is_closed()
{
existing.update_target(src_addr);
existing.deliver_inbound(datagram.clone());
continue;
}
let candidates = self.peer_addrs(peer);
let link = Arc::new(WireguardLink::new(
network,
peer,
src_addr,
candidates,
Arc::clone(&self.socket),
));
link.update_target(src_addr);
link.deliver_inbound(datagram.clone());
guard.insert((network, peer), Arc::clone(&link));
link
};
let cb_guard = match self.inbound_cb.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
if let Some(ref cb) = *cb_guard {
cb(InboundLink {
network,
peer,
protocol: crate::announcement::WG_PROTOCOL.to_string(),
link,
});
}
}
}
}
}
impl PacketTransport for WireguardTransport {
fn name(&self) -> &str {
"wireguard"
}
fn as_any(&self) -> &dyn std::any::Any {
self
}
fn open<'a>(
&'a self,
network: NetworkId,
peer: EndpointId,
_protocol: &'a str,
) -> BoxFuture<'a, Result<SharedLink, TransportError>> {
Box::pin(async move {
let mut addrs = self.peer_addrs(peer);
if addrs.is_empty() {
return Err(TransportError::Unreachable(format!(
"no candidate UDP address known for peer {}",
peer.fmt_short()
)));
}
crate::plugin::prioritize_local_subnet(&mut addrs);
// If an existing confirmed link is alive, reuse it
{
let guard = match self.links.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
if let Some(existing) = guard.get(&(network, peer))
&& !existing.is_closed()
&& existing.is_confirmed()
{
existing.update_candidates(addrs);
return Ok(Arc::clone(existing) as SharedLink);
}
}
// Probe candidate addresses with PING
let cookie: u64 = rand::random();
let (tx, rx) = oneshot::channel::<SocketAddr>();
{
let mut probes = match self.pending_probes.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
probes.insert((network, peer, cookie), tx);
}
let probe_ping = encode_probe(PROBE_TYPE_PING, self.local_id, peer, network, cookie);
let socket = Arc::clone(&self.socket);
let candidates = addrs.clone();
let probe_task = tokio::spawn(async move {
for i in 0..3 {
for target in &candidates {
let _ = socket.send_to(&probe_ping, *target).await;
}
if i < 2 {
tokio::time::sleep(std::time::Duration::from_millis(350)).await;
}
}
});
let confirmed_addr =
match tokio::time::timeout(std::time::Duration::from_millis(1200), rx).await {
Ok(Ok(addr)) => {
probe_task.abort();
addr
}
_ => {
probe_task.abort();
let mut probes = match self.pending_probes.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
probes.remove(&(network, peer, cookie));
return Err(TransportError::Unreachable(format!(
"direct UDP probe to peer {} timed out",
peer.fmt_short()
)));
}
};
let link = {
let mut guard = match self.links.write() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
if let Some(existing) = guard.get(&(network, peer))
&& !existing.is_closed()
{
existing.update_target(confirmed_addr);
existing.update_candidates(addrs);
existing.confirm();
return Ok(Arc::clone(existing) as SharedLink);
}
let link = Arc::new(WireguardLink::new(
network,
peer,
confirmed_addr,
addrs,
Arc::clone(&self.socket),
));
link.confirm();
guard.insert((network, peer), Arc::clone(&link));
link
};
Ok(link as SharedLink)
})
}
}
@@ -0,0 +1,86 @@
#![allow(missing_docs)]
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::IpAddr;
use tsunagi::identity::NetworkId;
use tsunagi_wireguard::{ANNOUNCEMENT_VERSION, WgAnnouncement, WgPublicKey, WgSecretKey};
#[test]
fn announcement_round_trip() {
let network = NetworkId::from_bytes([42u8; 32]);
let key = WgSecretKey::generate().public();
let local = WgSecretKey::generate().public();
let addrs: Vec<IpAddr> = vec![
"192.168.1.100".parse().unwrap(),
"10.0.0.5".parse().unwrap(),
];
let announcement = WgAnnouncement::new(network, &key, 51820, addrs.clone());
let encoded = announcement.encode().expect("encode");
let validated = WgAnnouncement::decode_and_validate(&encoded, network, &local).expect("decode");
assert_eq!(validated.public_key, key);
assert_eq!(validated.port, 51820);
assert_eq!(validated.addrs, addrs);
}
#[test]
fn announcement_rejects_zero_key() {
let network = NetworkId::from_bytes([42u8; 32]);
let local = WgSecretKey::generate().public();
let zero = WgPublicKey::from_bytes([0u8; 32]);
let announcement = WgAnnouncement::new(network, &zero, 51820, vec![]);
let encoded = announcement.encode().unwrap();
let err = WgAnnouncement::decode_and_validate(&encoded, network, &local).unwrap_err();
assert!(err.to_string().contains("all zeroes"));
}
#[test]
fn announcement_rejects_own_key() {
let network = NetworkId::from_bytes([42u8; 32]);
let local = WgSecretKey::generate().public();
let announcement = WgAnnouncement::new(network, &local, 51820, vec![]);
let encoded = announcement.encode().unwrap();
let err = WgAnnouncement::decode_and_validate(&encoded, network, &local).unwrap_err();
assert!(err.to_string().contains("own WireGuard key"));
}
#[test]
fn announcement_rejects_wrong_network() {
let network_a = NetworkId::from_bytes([1u8; 32]);
let network_b = NetworkId::from_bytes([2u8; 32]);
let key = WgSecretKey::generate().public();
let local = WgSecretKey::generate().public();
let announcement = WgAnnouncement::new(network_a, &key, 51820, vec![]);
let encoded = announcement.encode().unwrap();
let err = WgAnnouncement::decode_and_validate(&encoded, network_b, &local).unwrap_err();
assert!(err.to_string().contains("different network"));
}
#[test]
fn announcement_rejects_zero_port() {
let network = NetworkId::from_bytes([42u8; 32]);
let key = WgSecretKey::generate().public();
let local = WgSecretKey::generate().public();
let announcement = WgAnnouncement {
version: ANNOUNCEMENT_VERSION,
public_key: *key.as_bytes(),
network: *network.as_bytes(),
port: 0,
addrs: vec![],
};
let encoded = announcement.encode().unwrap();
let err = WgAnnouncement::decode_and_validate(&encoded, network, &local).unwrap_err();
assert!(err.to_string().contains("cannot be 0"));
}
@@ -0,0 +1,378 @@
#![allow(missing_docs)]
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::Duration;
use bytes::Bytes;
use tokio::sync::mpsc;
use tsunagi::dataplane::transport::{InboundLink, PacketTransport};
use tsunagi::dataplane::{IpPlugin, PacketSink, PluginContext};
use tsunagi::identity::{DeviceIdentity, NetworkId};
use tsunagi_wireguard::{WireguardConfig, WireguardPlugin, WireguardTransport};
struct TestSink {
delivered: mpsc::Sender<(NetworkId, iroh::EndpointId, Bytes)>,
}
impl PacketSink for TestSink {
fn deliver(
&self,
network: NetworkId,
from: iroh::EndpointId,
packet: Bytes,
) -> tsunagi::BoxFuture<'_, ()> {
let delivered = self.delivered.clone();
Box::pin(async move {
let _ = delivered.send((network, from, packet)).await;
})
}
}
fn sample_ipv4_packet() -> Bytes {
let mut packet = vec![0u8; 40];
packet[0] = 0x45; // IPv4, IHL 5
packet[2..4].copy_from_slice(&40u16.to_be_bytes()); // total len 40
packet[8] = 64; // TTL
packet[9] = 17; // UDP
packet[12..16].copy_from_slice(&[10, 0, 0, 1]); // src 10.0.0.1
packet[16..20].copy_from_slice(&[10, 0, 0, 2]); // dst 10.0.0.2
Bytes::from(packet)
}
#[tokio::test]
async fn test_wireguard_plugin_end_to_end() {
let _ = tracing_subscriber::fmt()
.with_env_filter("trace")
.with_test_writer()
.try_init();
let identity_a = DeviceIdentity::generate();
let peer_a = identity_a.endpoint_id();
let identity_b = DeviceIdentity::generate();
let peer_b = identity_b.endpoint_id();
let network = NetworkId::from_bytes([77u8; 32]);
let tmp_a = tempfile::tempdir().expect("tmp A");
let tmp_b = tempfile::tempdir().expect("tmp B");
// Transports
let transport_b = Arc::new(
WireguardTransport::bind(&identity_b, Some(0))
.await
.expect("bind B"),
);
let port_b = transport_b.bound_port();
let transport_a = Arc::new(
WireguardTransport::bind(&identity_a, Some(0))
.await
.expect("bind A"),
);
let port_a = transport_a.bound_port();
// Plugins
let cfg_a = WireguardConfig::new(tmp_a.path());
let plugin_a = WireguardPlugin::open(cfg_a, Arc::clone(&transport_a))
.await
.expect("open plugin A");
let cfg_b = WireguardConfig::new(tmp_b.path());
let plugin_b = WireguardPlugin::open(cfg_b, Arc::clone(&transport_b))
.await
.expect("open plugin B");
// Attach sinks
let (sink_a_tx, mut sink_a_rx) = mpsc::channel(10);
plugin_a.attach(PluginContext::with_sink(Arc::new(TestSink {
delivered: sink_a_tx,
})));
let (sink_b_tx, mut sink_b_rx) = mpsc::channel(10);
plugin_b.attach(PluginContext::with_sink(Arc::new(TestSink {
delivered: sink_b_tx,
})));
// Capability exchange
plugin_a.on_network_activated(network);
plugin_b.on_network_activated(network);
let cap_a = loop {
if let Some(cap) = plugin_a.local_capability(network).expect("cap A") {
break cap;
}
tokio::time::sleep(Duration::from_millis(50)).await;
};
let cap_b = loop {
if let Some(cap) = plugin_b.local_capability(network).expect("cap B") {
break cap;
}
tokio::time::sleep(Duration::from_millis(50)).await;
};
plugin_a
.on_peer_capability(network, peer_b, &cap_b)
.expect("A on cap B");
plugin_b
.on_peer_capability(network, peer_a, &cap_a)
.expect("B on cap A");
// Register loopback address candidates
let addr_b: SocketAddr = format!("127.0.0.1:{port_b}").parse().unwrap();
transport_a.set_peer_addr(peer_b, addr_b);
let addr_a: SocketAddr = format!("127.0.0.1:{port_a}").parse().unwrap();
transport_b.set_peer_addr(peer_a, addr_a);
// Spawn accept loop on A
let transport_a_clone = Arc::clone(&transport_a);
tokio::spawn(async move {
transport_a_clone.accept_loop(|_| {}).await;
});
// Spawn accept loop on B
let (inbound_tx, mut inbound_rx) = mpsc::channel::<InboundLink>(1);
let transport_b_clone = Arc::clone(&transport_b);
tokio::spawn(async move {
transport_b_clone
.accept_loop(move |inbound| {
let _ = inbound_tx.try_send(inbound);
})
.await;
});
// Open link from A to B
let link_a = tokio::time::timeout(
Duration::from_secs(5),
transport_a.open(network, peer_b, "wg"),
)
.await
.expect("open should not timeout")
.expect("open should succeed");
plugin_a.on_peer_link(network, peer_b, Arc::clone(&link_a));
// Wait for inbound link on B and hand to plugin B
let inbound_b = tokio::time::timeout(Duration::from_secs(5), inbound_rx.recv())
.await
.expect("accept should not timeout")
.expect("inbound link should arrive");
plugin_b.on_peer_link(network, peer_a, inbound_b.link);
// Wait briefly for WireGuard handshake to complete
let mut up = false;
for _ in 0..50 {
if let Some(overview_a) = plugin_a.overview(network)
&& overview_a.peers.iter().any(|p| p.is_up())
{
up = true;
break;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
assert!(up, "WireGuard handshake should complete between peers");
// Send IP packet from A to B
let packet_a_to_b = sample_ipv4_packet();
let sent = plugin_a.carry(network, peer_b, packet_a_to_b.clone());
assert!(sent, "plugin A carry should succeed");
let received_b = tokio::time::timeout(Duration::from_secs(5), sink_b_rx.recv())
.await
.expect("recv B timeout")
.expect("recv B option");
assert_eq!(received_b.0, network);
assert_eq!(received_b.1, peer_a);
assert_eq!(received_b.2, packet_a_to_b);
// Send IP packet from B to A
let packet_b_to_a = sample_ipv4_packet();
let sent_reply = plugin_b.carry(network, peer_a, packet_b_to_a.clone());
assert!(sent_reply, "plugin B carry should succeed");
let received_a = tokio::time::timeout(Duration::from_secs(5), sink_a_rx.recv())
.await
.expect("recv A timeout")
.expect("recv A option");
assert_eq!(received_a.0, network);
assert_eq!(received_a.1, peer_b);
assert_eq!(received_a.2, packet_b_to_a);
}
#[tokio::test]
async fn test_wireguard_two_networks_same_peers() {
let identity_a = DeviceIdentity::generate();
let peer_a = identity_a.endpoint_id();
let identity_b = DeviceIdentity::generate();
let peer_b = identity_b.endpoint_id();
let net1 = NetworkId::from_bytes([11u8; 32]);
let net2 = NetworkId::from_bytes([22u8; 32]);
let tmp_a = tempfile::tempdir().expect("tmp A");
let tmp_b = tempfile::tempdir().expect("tmp B");
let transport_b = Arc::new(
WireguardTransport::bind(&identity_b, Some(0))
.await
.expect("bind B"),
);
let port_b = transport_b.bound_port();
let transport_a = Arc::new(
WireguardTransport::bind(&identity_a, Some(0))
.await
.expect("bind A"),
);
let port_a = transport_a.bound_port();
let plugin_a =
WireguardPlugin::open(WireguardConfig::new(tmp_a.path()), Arc::clone(&transport_a))
.await
.expect("open plugin A");
let plugin_b =
WireguardPlugin::open(WireguardConfig::new(tmp_b.path()), Arc::clone(&transport_b))
.await
.expect("open plugin B");
let (sink_a_tx, mut sink_a_rx) = mpsc::channel(20);
plugin_a.attach(PluginContext::with_sink(Arc::new(TestSink {
delivered: sink_a_tx,
})));
let (sink_b_tx, mut sink_b_rx) = mpsc::channel(20);
plugin_b.attach(PluginContext::with_sink(Arc::new(TestSink {
delivered: sink_b_tx,
})));
// Activate both networks
for net in [net1, net2] {
plugin_a.on_network_activated(net);
plugin_b.on_network_activated(net);
}
// Exchange capabilities for both networks
for net in [net1, net2] {
let cap_a = loop {
if let Some(cap) = plugin_a.local_capability(net).expect("cap A") {
break cap;
}
tokio::time::sleep(Duration::from_millis(20)).await;
};
let cap_b = loop {
if let Some(cap) = plugin_b.local_capability(net).expect("cap B") {
break cap;
}
tokio::time::sleep(Duration::from_millis(20)).await;
};
plugin_a
.on_peer_capability(net, peer_b, &cap_b)
.expect("A on cap B");
plugin_b
.on_peer_capability(net, peer_a, &cap_a)
.expect("B on cap A");
}
let addr_b: SocketAddr = format!("127.0.0.1:{port_b}").parse().unwrap();
transport_a.set_peer_addr(peer_b, addr_b);
let addr_a: SocketAddr = format!("127.0.0.1:{port_a}").parse().unwrap();
transport_b.set_peer_addr(peer_a, addr_a);
// Accept loops
let transport_a_clone = Arc::clone(&transport_a);
tokio::spawn(async move {
transport_a_clone.accept_loop(|_| {}).await;
});
let (inbound_tx, mut inbound_rx) = mpsc::channel::<InboundLink>(10);
let transport_b_clone = Arc::clone(&transport_b);
tokio::spawn(async move {
transport_b_clone
.accept_loop(move |inbound| {
let _ = inbound_tx.try_send(inbound);
})
.await;
});
// Open links for both networks from A to B
for net in [net1, net2] {
let link_a = transport_a
.open(net, peer_b, "wg")
.await
.expect("open A link");
plugin_a.on_peer_link(net, peer_b, link_a);
}
// Deliver inbound links to B
for _ in 0..2 {
let inbound = tokio::time::timeout(Duration::from_secs(5), inbound_rx.recv())
.await
.expect("inbound timeout")
.expect("inbound option");
plugin_b.on_peer_link(inbound.network, peer_a, inbound.link);
}
// Wait for both handshakes to complete on both networks
for net in [net1, net2] {
let mut up = false;
for _ in 0..50 {
if let Some(overview) = plugin_a.overview(net)
&& overview.peers.iter().any(|p| p.is_up())
{
up = true;
break;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
assert!(up, "handshake on {net:?} should complete");
}
// Send on net1
let pkt1 = sample_ipv4_packet();
assert!(plugin_a.carry(net1, peer_b, pkt1.clone()));
let recv1 = tokio::time::timeout(Duration::from_secs(5), sink_b_rx.recv())
.await
.expect("recv1 timeout")
.expect("recv1 option");
assert_eq!(recv1.0, net1);
assert_eq!(recv1.2, pkt1);
// Send on net2 from A to B
let mut pkt2_bytes = sample_ipv4_packet().to_vec();
pkt2_bytes[15] = 99; // different byte
let pkt2 = Bytes::from(pkt2_bytes);
assert!(plugin_a.carry(net2, peer_b, pkt2.clone()));
let recv2 = tokio::time::timeout(Duration::from_secs(5), sink_b_rx.recv())
.await
.expect("recv2 timeout")
.expect("recv2 option");
assert_eq!(recv2.0, net2);
assert_eq!(recv2.2, pkt2);
// Send reply on net1 from B to A
assert!(plugin_b.carry(net1, peer_a, pkt1.clone()));
let reply1 = tokio::time::timeout(Duration::from_secs(5), sink_a_rx.recv())
.await
.expect("reply1 timeout")
.expect("reply1 option");
assert_eq!(reply1.0, net1);
assert_eq!(reply1.2, pkt1);
// Send reply on net2 from B to A
assert!(plugin_b.carry(net2, peer_a, pkt2.clone()));
let reply2 = tokio::time::timeout(Duration::from_secs(5), sink_a_rx.recv())
.await
.expect("reply2 timeout")
.expect("reply2 option");
assert_eq!(reply2.0, net2);
assert_eq!(reply2.2, pkt2);
}
@@ -0,0 +1,220 @@
#![allow(missing_docs)]
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::Duration;
use bytes::Bytes;
use tokio::sync::mpsc;
use tsunagi::dataplane::transport::{InboundLink, PacketTransport};
use tsunagi::identity::{DeviceIdentity, NetworkId};
use tsunagi_wireguard::WireguardTransport;
#[tokio::test]
async fn test_wireguard_transport_loopback() {
let identity_a = DeviceIdentity::generate();
let peer_a = identity_a.endpoint_id();
let identity_b = DeviceIdentity::generate();
let peer_b = identity_b.endpoint_id();
let network = NetworkId::from_bytes([42u8; 32]);
// Bind transport B on port 0 (ephemeral)
let transport_b = Arc::new(
WireguardTransport::bind(&identity_b, Some(0))
.await
.expect("bind server should succeed"),
);
let port_b = transport_b.bound_port();
assert!(port_b > 0);
// Bind transport A on port 0 (ephemeral)
let transport_a = Arc::new(
WireguardTransport::bind(&identity_a, Some(0))
.await
.expect("bind client should succeed"),
);
let port_a = transport_a.bound_port();
assert!(port_a > 0);
// Register B's address and network on transport A
let addr_b: SocketAddr = format!("127.0.0.1:{port_b}").parse().unwrap();
transport_a.set_peer_addr(peer_b, addr_b);
transport_a.set_peer_network(peer_b, network);
// Register A's address and network on transport B
let addr_a: SocketAddr = format!("127.0.0.1:{port_a}").parse().unwrap();
transport_b.set_peer_addr(peer_a, addr_a);
transport_b.set_peer_network(peer_a, network);
// Spawn accept loop on A
let transport_a_clone = Arc::clone(&transport_a);
tokio::spawn(async move {
transport_a_clone.accept_loop(|_| {}).await;
});
// Spawn accept loop on B
let (inbound_tx, mut inbound_rx) = mpsc::channel::<InboundLink>(1);
let transport_b_clone = Arc::clone(&transport_b);
tokio::spawn(async move {
transport_b_clone
.accept_loop(move |inbound| {
let _ = inbound_tx.try_send(inbound);
})
.await;
});
// Open link from A to B
let link_a = tokio::time::timeout(
Duration::from_secs(5),
transport_a.open(network, peer_b, "wg"),
)
.await
.expect("open should not timeout")
.expect("open should succeed");
// Send from A to B to trigger inbound link discovery on B
let data_a_to_b = Bytes::from_static(b"wireguard packet from A to B");
link_a.send(data_a_to_b.clone()).expect("send A->B");
// Receive inbound link on B
let inbound_b = tokio::time::timeout(Duration::from_secs(5), inbound_rx.recv())
.await
.expect("accept should not timeout")
.expect("inbound link should arrive");
assert_eq!(inbound_b.network, network);
assert_eq!(inbound_b.peer, peer_a);
assert_eq!(inbound_b.protocol, "wg");
let link_b = inbound_b.link;
// Check packet received on B
let received_on_b = tokio::time::timeout(Duration::from_secs(2), link_b.recv())
.await
.expect("recv B timeout")
.expect("recv B option");
assert_eq!(received_on_b, data_a_to_b);
// Send from B to A
let data_b_to_a = Bytes::from_static(b"wireguard packet from B to A");
link_b.send(data_b_to_a.clone()).expect("send B->A");
let received_on_a = tokio::time::timeout(Duration::from_secs(2), link_a.recv())
.await
.expect("recv A timeout")
.expect("recv A option");
assert_eq!(received_on_a, data_b_to_a);
// Path descriptions
assert!(link_a.path_description().contains("wireguard via"));
assert!(link_b.path_description().contains("wireguard via"));
}
#[tokio::test]
async fn test_wireguard_transport_unreachable_probe_times_out() {
let identity_a = DeviceIdentity::generate();
let identity_b = DeviceIdentity::generate();
let peer_b = identity_b.endpoint_id();
let network = NetworkId::from_bytes([99u8; 32]);
let transport_a = Arc::new(
WireguardTransport::bind(&identity_a, Some(0))
.await
.expect("bind client should succeed"),
);
// Register a non-responsive address for peer B (TEST-NET-1 documentation IP which drops packets)
let unreachable_addr: SocketAddr = "192.0.2.1:51820".parse().unwrap();
transport_a.set_peer_addr(peer_b, unreachable_addr);
transport_a.set_peer_network(peer_b, network);
// Spawn accept loop on A
let transport_a_clone = Arc::clone(&transport_a);
tokio::spawn(async move {
transport_a_clone.accept_loop(|_| {}).await;
});
// Opening to unreachable candidate must fail with Unreachable rather than returning a dead direct link
let result = transport_a.open(network, peer_b, "wg").await;
match result {
Err(tsunagi::dataplane::transport::TransportError::Unreachable(msg)) => {
assert!(msg.contains("timed out"));
}
other => panic!("expected Unreachable error, got {other:?}"),
}
}
#[tokio::test]
async fn test_wireguard_link_close_and_transport_close_link() {
let identity_a = DeviceIdentity::generate();
let peer_a = identity_a.endpoint_id();
let identity_b = DeviceIdentity::generate();
let peer_b = identity_b.endpoint_id();
let network = NetworkId::from_bytes([55u8; 32]);
let transport_b = Arc::new(
WireguardTransport::bind(&identity_b, Some(0))
.await
.unwrap(),
);
let port_b = transport_b.bound_port();
let transport_a = Arc::new(
WireguardTransport::bind(&identity_a, Some(0))
.await
.unwrap(),
);
let port_a = transport_a.bound_port();
let addr_b: SocketAddr = format!("127.0.0.1:{port_b}").parse().unwrap();
transport_a.set_peer_addr(peer_b, addr_b);
transport_a.set_peer_network(peer_b, network);
let addr_a: SocketAddr = format!("127.0.0.1:{port_a}").parse().unwrap();
transport_b.set_peer_addr(peer_a, addr_a);
transport_b.set_peer_network(peer_a, network);
let transport_a_clone = Arc::clone(&transport_a);
tokio::spawn(async move {
transport_a_clone.accept_loop(|_| {}).await;
});
let transport_b_clone = Arc::clone(&transport_b);
tokio::spawn(async move {
transport_b_clone.accept_loop(|_| {}).await;
});
let link_a = transport_a
.open(network, peer_b, "wg")
.await
.expect("open link A");
assert!(!link_a.is_closed());
// Spawn a reader task waiting on recv()
let link_a_clone = Arc::clone(&link_a);
let recv_task = tokio::spawn(async move { link_a_clone.recv().await });
// Explicitly closing via transport.close_link()
transport_a.close_link(network, peer_b);
assert!(link_a.is_closed());
// Wait on closed() notification
tokio::time::timeout(Duration::from_secs(1), link_a.closed())
.await
.expect("link.closed() should resolve promptly");
// recv() must have returned None
let recv_res = tokio::time::timeout(Duration::from_secs(1), recv_task)
.await
.expect("recv task should complete")
.expect("task join");
assert_eq!(recv_res, None);
// Send after close must return Closed error
let send_res = link_a.send(Bytes::from_static(b"after close"));
assert!(matches!(
send_res,
Err(tsunagi::dataplane::transport::TransportError::Closed)
));
}
+6 -2
View File
@@ -22,7 +22,6 @@ tun-device = [
"dep:rtnetlink",
"dep:caps",
"dep:futures-util",
"dep:netdev",
]
# Telling the operating system where to send its DNS questions. systemd-resolved
# on Linux, the Name Resolution Policy Table on Windows; the zone and the
@@ -67,6 +66,7 @@ directories = "6.0"
# forbidden `unsafe` and will not make one.
gethostname = "1.1"
netwatch = "0.19.3"
netdev = "0.46"
mainline = "8.0"
futures-lite = "2.6"
tun = { version = "0.8", features = ["async"], optional = true }
@@ -93,7 +93,11 @@ futures-util = { version = "0.3", default-features = false, optional = true }
# before deciding what to change. Creating an adapter needs the process to be
# elevated and `wintun.dll` to be reachable at run time.
[target.'cfg(target_os = "windows")'.dependencies]
netdev = { version = "0.45", optional = true }
windows-sys = { version = "0.61", features = [
"Win32_Security",
"Win32_Security_Authorization",
"Win32_Foundation",
] }
[dev-dependencies]
# Its own tests use the harness it publishes.
+64 -5
View File
@@ -385,10 +385,7 @@ impl Agent {
/// waits to adopt whatever it settles on. One agent has one interface, so
/// proposing a range it could not route would be worse than having none:
/// the lowest author's range wins, and the collision would spread.
fn reserve_range(&self, network: NetworkId) -> RangePlan {
let Some(configured) = self.inner.config.overlay_ipv4_range else {
return RangePlan::default();
};
async fn reserve_range(&self, network: NetworkId) -> RangePlan {
let reserve = |wanted: Ipv4Range| {
let reservation = crate::overlay::NetworkRoutes {
broadcast: Default::default(),
@@ -399,6 +396,44 @@ impl Agent {
self.inner.routes.set_network(network, reservation).is_ok()
};
// If this network already has signed records in storage, check if its
// members have already agreed on a range. If so, preserve and reserve
// THAT range rather than preemptively seizing the default configured range.
let stored_records = self
.inner
.storage
.signed_records(network)
.await
.unwrap_or_default();
let mut state_set = crate::state::StateSet::new();
state_set.merge_all(network, stored_records);
if let Some(agreed) = state_set.agreed_range() {
if reserve(agreed) {
return RangePlan {
propose: if Some(agreed) == self.inner.config.overlay_ipv4_range {
Some(agreed)
} else {
None
},
..RangePlan::default()
};
} else {
tracing::warn!(
%network,
range = %agreed,
"agreed range for this network overlaps another network already active on this interface"
);
return RangePlan {
conflict: Some(agreed),
..RangePlan::default()
};
}
}
let Some(configured) = self.inner.config.overlay_ipv4_range else {
return RangePlan::default();
};
// The configured range first, so a network a device has always had
// keeps the addresses it has always had.
if reserve(configured) {
@@ -446,6 +481,7 @@ impl Agent {
mtu: interface.mtu(),
addresses: interface.wanted_addresses(),
counters: interface.counters(),
broadcast_rules: interface.broadcast_rules(),
})
}
@@ -610,7 +646,7 @@ impl Agent {
return Err(Error::NetworkAlreadyActive(network_id));
}
let range = self.reserve_range(network_id);
let range = self.reserve_range(network_id).await;
let mut backends = Vec::new();
if let Some(discovery) = &self.inner.config.discovery {
backends.push(discovery.clone());
@@ -695,6 +731,9 @@ impl Agent {
{
tracing::warn!(%err, "cannot update the overlay addresses");
}
if let Some(interface) = self.inner.interface.get() {
interface.sync_broadcast_rules().await;
}
self.inner.storage.set_auto_start(network_id, false).await?;
Ok(())
}
@@ -1083,6 +1122,26 @@ async fn plugin_request_loop(weak: Weak<Inner>, mut requests: mpsc::Receiver<Plu
}
}
}
PluginRequest::ProtocolFailure {
network,
peer,
protocol,
reason,
} => {
let sender = {
let networks = inner.networks.read().await;
networks.get(&network).map(|handle| handle.commands.clone())
};
if let Some(sender) = sender {
let _ = sender
.send(NetCommand::ProtocolFailure {
peer,
protocol,
reason,
})
.await;
}
}
}
}
}
+104
View File
@@ -86,6 +86,12 @@ pub(crate) enum NetCommand {
/// Human readable reason, free of secrets.
reason: String,
},
/// An IP plugin reported that data plane connectivity or a tunnel for a peer failed.
ProtocolFailure {
peer: EndpointId,
protocol: String,
reason: String,
},
}
impl std::fmt::Debug for NetCommand {
@@ -104,6 +110,9 @@ impl std::fmt::Debug for NetCommand {
NetCommand::SetHostname(_) => f.write_str("SetHostname"),
NetCommand::Release { .. } => f.write_str("Release"),
NetCommand::PluginError { protocol, .. } => write!(f, "PluginError({protocol})"),
NetCommand::ProtocolFailure { peer, protocol, .. } => {
write!(f, "ProtocolFailure({}, {protocol})", peer.fmt_short())
}
}
}
}
@@ -486,6 +495,9 @@ impl Runtime {
NetCommand::SetBroadcast { enabled, reply } => {
self.params.broadcast = enabled;
self.update_broadcast();
if let Some(interface) = self.params.interface.clone() {
interface.sync_broadcast_rules().await;
}
self.reannounce();
let _ = reply.send(());
}
@@ -522,6 +534,26 @@ impl Runtime {
reason,
});
}
NetCommand::ProtocolFailure {
peer,
protocol,
reason,
} => {
if self.links.remove(&(peer, protocol.clone())).is_some() {
self.hub.clear_direct(peer, &protocol);
}
self.metrics.data_link_failures += 1;
self.record_protocol_failure(peer, &protocol);
self.emit(Event::DataLinkDown {
network: self.network_id,
peer,
protocol,
reason,
});
self.ensure_links();
self.update_paths();
self.announce_reach(false);
}
}
}
@@ -1182,6 +1214,12 @@ impl Runtime {
),
});
}
// The broadcast route names this address as its source, so it goes in
// only once the address is really on the interface.
if !is_missing {
interface.sync_broadcast_rules().await;
}
}
// ------------------------------------------------------------ data plane
@@ -2162,4 +2200,70 @@ mod tests {
assert_eq!(data_link_backoff(6), Duration::from_secs(60));
assert_eq!(data_link_backoff(10), Duration::from_secs(60));
}
#[test]
fn test_protocol_dial_state_cooldown_and_fallback_selection() {
let peer = crate::identity::DeviceIdentity::generate().endpoint_id();
let supported = ["wg", "tcp-tls", "wg-quic"];
let mut states: HashMap<(EndpointId, String), ProtocolDialState> = HashMap::new();
let now = std::time::Instant::now();
// 1. Without cooldown, the highest priority protocol "wg" is chosen
let chosen =
supported
.iter()
.find(|proto| match states.get(&(peer, (*proto).to_string())) {
Some(state) => state.cooldown_until <= now,
None => true,
});
assert_eq!(chosen.copied(), Some("wg"));
// 2. Record failure for "wg" putting it on cooldown
states.insert(
(peer, "wg".to_string()),
ProtocolDialState {
failures: 1,
cooldown_until: now + Duration::from_secs(3),
},
);
// With "wg" on cooldown, selection must fall back to next priority "tcp-tls"
let chosen_after_failure =
supported
.iter()
.find(|proto| match states.get(&(peer, (*proto).to_string())) {
Some(state) => state.cooldown_until <= now,
None => true,
});
assert_eq!(chosen_after_failure.copied(), Some("tcp-tls"));
// 3. If "tcp-tls" also fails, selection falls back to "wg-quic"
states.insert(
(peer, "tcp-tls".to_string()),
ProtocolDialState {
failures: 1,
cooldown_until: now + Duration::from_secs(3),
},
);
let chosen_quic =
supported
.iter()
.find(|proto| match states.get(&(peer, (*proto).to_string())) {
Some(state) => state.cooldown_until <= now,
None => true,
});
assert_eq!(chosen_quic.copied(), Some("wg-quic"));
// 4. Once success is recorded (entry removed), "wg" is preferred again
states.remove(&(peer, "wg".to_string()));
let chosen_restored =
supported
.iter()
.find(|proto| match states.get(&(peer, (*proto).to_string())) {
Some(state) => state.cooldown_until <= now,
None => true,
});
assert_eq!(chosen_restored.copied(), Some("wg"));
}
}
+4
View File
@@ -53,6 +53,10 @@ pub struct OverlayStatus {
pub addresses: Vec<crate::overlay::Cidr>,
/// What has happened on it.
pub counters: crate::overlay::Counters,
/// The route and firewall allowance installed for LAN broadcast, when
/// some network has broadcast on and the interface is on the host.
/// `None` is "none installed", not "unknown".
pub broadcast_rules: Option<crate::overlay::BroadcastRulesReport>,
}
/// A member the signed state knows about, connected or not.
+31 -10
View File
@@ -12,9 +12,13 @@ use std::time::Duration;
use crate::dataplane::SharedPlugin;
use crate::discovery::{MainlineDiscovery, NetworkDiscovery};
use crate::error::{Error, Result};
use crate::error::Result;
/// Qualifier/organisation/application triple used for platform directories.
///
/// Unused on macOS, where the default paths are fixed system locations rather
/// than per-user directories derived from this.
#[cfg_attr(target_os = "macos", allow(dead_code))]
const APP_NAME: &str = "tsunagi";
/// Rendezvous slots per network; this bounds a discovery sample, not membership.
@@ -74,17 +78,34 @@ impl StoragePaths {
}
}
/// The per-user platform directories.
/// The default platform directories.
///
/// A future system service can supply its own paths instead.
/// Per user on Linux and Windows. On macOS the agent runs as a root
/// LaunchDaemon (creating a utun interface needs root, and `$HOME` under
/// `sudo` points at the wrong place), so the stores live in the system
/// locations instead — `/var/db/tsunagi` for the mandatory state and a
/// disposable cache beside it. A caller that wants somewhere else passes
/// explicit paths to [`StoragePaths::new`] or [`StoragePaths::under`].
pub fn user_default() -> Result<Self> {
let dirs = directories::ProjectDirs::from("", "", APP_NAME).ok_or_else(|| {
Error::Storage("no valid home directory for platform config paths".into())
})?;
Ok(Self {
state_dir: dirs.data_dir().to_path_buf(),
cache_dir: dirs.cache_dir().to_path_buf(),
})
#[cfg(target_os = "macos")]
{
Ok(Self {
state_dir: PathBuf::from("/var/db/tsunagi"),
cache_dir: PathBuf::from("/var/db/tsunagi/cache"),
})
}
#[cfg(not(target_os = "macos"))]
{
let dirs = directories::ProjectDirs::from("", "", APP_NAME).ok_or_else(|| {
crate::error::Error::Storage(
"no valid home directory for platform config paths".into(),
)
})?;
Ok(Self {
state_dir: dirs.data_dir().to_path_buf(),
cache_dir: dirs.cache_dir().to_path_buf(),
})
}
}
/// Path of the mandatory state database.
+42 -1
View File
@@ -119,7 +119,7 @@ pub enum PluginError {
/// A request a plugin makes of the agent that owns it.
#[derive(Debug)]
pub(crate) enum PluginRequest {
pub enum PluginRequest {
/// Re-send this agent's announcement to every peer of a network.
Reannounce(NetworkId),
/// Surface a plugin error on the agent's event stream.
@@ -131,6 +131,17 @@ pub(crate) enum PluginRequest {
/// Human readable reason, free of secrets.
reason: String,
},
/// A plugin reported that a data link or tunnel for a peer failed.
ProtocolFailure {
/// Network the failure occurred in.
network: NetworkId,
/// Peer whose protocol failed.
peer: EndpointId,
/// Protocol that failed.
protocol: String,
/// Human readable reason.
reason: String,
},
}
/// The agent-side handle a plugin is given when it is attached.
@@ -192,6 +203,20 @@ impl PluginContext {
}
}
/// A context configured with a channel for plugin requests and custom sink, for tests.
#[cfg(feature = "testing")]
pub fn for_test(
sender: mpsc::Sender<PluginRequest>,
local: EndpointId,
sink: Option<Arc<dyn PacketSink>>,
) -> Self {
Self {
sender: Some(sender),
local: Some(local),
sink,
}
}
/// This agent's own endpoint id, when the context is attached.
///
/// A plugin needs it to find itself in the agreed allocation.
@@ -233,6 +258,22 @@ impl PluginContext {
reason: reason.into(),
});
}
/// Reports that data plane connectivity or a tunnel for a peer has failed.
pub fn report_peer_protocol_failure(
&self,
network: NetworkId,
peer: EndpointId,
protocol: impl Into<String>,
reason: impl Into<String>,
) {
self.send(PluginRequest::ProtocolFailure {
network,
peer,
protocol: protocol.into(),
reason: reason.into(),
});
}
}
impl std::fmt::Debug for PluginContext {
+333
View File
@@ -0,0 +1,333 @@
//! Telling the macOS resolver to send some questions here, through
//! `/etc/resolver`.
//!
//! macOS reads one file per domain out of `/etc/resolver`: a file named after
//! a domain, containing the `nameserver` and `port` to send that domain's
//! questions to, makes the system resolver a *scoped* resolver for exactly
//! that suffix and nothing else (see `resolver(5)`). That is the same contract
//! the systemd-resolved and NRPT publishers arrange on the other platforms:
//! route these suffixes to this server, and claim nothing else.
//!
//! # Why a port, and so no privileged socket
//!
//! A resolver file carries a `port`, so the server does not have to sit on 53
//! and the agent needs nothing like `CAP_NET_BIND_SERVICE`. It can serve on a
//! high port and still be reached for its suffixes.
//!
//! # Tagged, reversible, and never clobbering
//!
//! Every file this publisher writes begins with a marker line, so it only ever
//! replaces or removes files it wrote itself: a pre-existing `/etc/resolver`
//! entry a user set up by hand is left untouched. The files are removed when
//! the resolver is turned off and on shutdown; a crash leaves them behind, and
//! the next apply for the same domains overwrites them.
//!
//! # Privilege
//!
//! Writing under `/etc/resolver` needs root. A permission error is reported as
//! a refusal — the kind of failure a person has to act on — rather than a
//! transient one, so the agent does not retry it at the ordinary pace.
use std::collections::BTreeSet;
use std::net::IpAddr;
use std::path::{Path, PathBuf};
use std::sync::Mutex;
use crate::BoxFuture;
use super::{DnsPublisher, PublishError, Published};
/// The directory macOS reads scoped resolvers from.
const RESOLVER_DIR: &str = "/etc/resolver";
/// The first line of every file this publisher writes, so it recognises its
/// own and never touches a file a user created.
const MARKER: &str = "# managed by tsunagi — do not edit";
/// Writes and removes `/etc/resolver` files for the overlay zones.
#[derive(Debug)]
pub struct ResolverDirPublisher {
/// The resolver directory. A field rather than a constant so the tests can
/// point it at a temporary directory and never touch the real one.
dir: PathBuf,
/// The domain files written by the last apply, so a later apply can remove
/// the ones no longer wanted and `revert` can remove them all.
written: Mutex<BTreeSet<String>>,
}
impl Default for ResolverDirPublisher {
fn default() -> Self {
Self::new()
}
}
impl ResolverDirPublisher {
/// A publisher writing to the real `/etc/resolver`.
pub fn new() -> Self {
Self::in_dir(RESOLVER_DIR)
}
/// A publisher writing to a specific directory. For the tests.
fn in_dir(dir: impl Into<PathBuf>) -> Self {
Self {
dir: dir.into(),
written: Mutex::new(BTreeSet::new()),
}
}
fn apply_inner(&self, published: &Published) -> Result<(), PublishError> {
let desired = desired_files(published)?;
// The directory may not exist yet; creating it needs root.
std::fs::create_dir_all(&self.dir).map_err(|err| classify(&self.dir, err))?;
// Remove the files from a previous apply that are no longer wanted.
let previous = lock(&self.written).clone();
for name in previous.difference(&keys(&desired)) {
remove_if_ours(&self.dir.join(name));
}
for (name, body) in &desired {
let path = self.dir.join(name);
// Only ever overwrite a file that is already ours, or a name that
// is free. A resolver a user set up by hand is left in place and
// reported, never clobbered.
if path.exists() && !is_ours(&path) {
return Err(PublishError::Failed(format!(
"{} already exists and was not created by tsunagi; \
remove it or rename the network to publish its resolver",
path.display()
)));
}
std::fs::write(&path, body).map_err(|err| classify(&path, err))?;
}
*lock(&self.written) = keys(&desired);
Ok(())
}
fn revert_inner(&self) {
let written = std::mem::take(&mut *lock(&self.written));
for name in &written {
remove_if_ours(&self.dir.join(name));
}
}
}
impl DnsPublisher for ResolverDirPublisher {
fn name(&self) -> &str {
"resolver-dir"
}
fn apply<'a>(&'a self, published: &'a Published) -> BoxFuture<'a, Result<(), PublishError>> {
// A handful of small file writes, short enough to run in place rather
// than hop to the blocking pool.
Box::pin(async move { self.apply_inner(published) })
}
fn revert(&self) -> BoxFuture<'_, Result<(), PublishError>> {
Box::pin(async move {
self.revert_inner();
Ok(())
})
}
}
/// The resolver files a publication implies: a map of file name to contents.
fn desired_files(published: &Published) -> Result<Vec<(String, String)>, PublishError> {
if published.servers.is_empty() {
return Err(PublishError::Failed(
"no server address to point the resolver at".to_string(),
));
}
let body = file_contents(published);
let mut files = Vec::new();
for domain in &published.domains {
let name = resolver_file_name(domain)?;
files.push((name, body.clone()));
}
Ok(files)
}
/// The contents of one resolver file: the marker, every server, one port.
fn file_contents(published: &Published) -> String {
let mut out = String::from(MARKER);
out.push('\n');
for server in &published.servers {
let ip = match server.ip() {
IpAddr::V4(v4) => v4.to_string(),
IpAddr::V6(v6) => v6.to_string(),
};
out.push_str("nameserver ");
out.push_str(&ip);
out.push('\n');
}
// One port for the file; every server listens on the same one. The `port`
// directive is what lets the server stay off 53.
if let Some(first) = published.servers.first() {
out.push_str("port ");
out.push_str(&first.port().to_string());
out.push('\n');
}
out
}
/// The file name for a domain, rejecting anything that is not a plain label
/// path component: a resolver file name must never escape the directory.
fn resolver_file_name(domain: &str) -> Result<String, PublishError> {
let trimmed = domain.trim_end_matches('.');
if trimmed.is_empty()
|| trimmed.contains('/')
|| trimmed.contains('\\')
|| trimmed.contains(std::path::MAIN_SEPARATOR)
|| trimmed == "."
|| trimmed == ".."
|| trimmed.starts_with('.')
{
return Err(PublishError::Failed(format!(
"cannot make a resolver file name for the domain {domain:?}"
)));
}
Ok(trimmed.to_string())
}
/// Whether a file is one this publisher wrote, by its marker first line.
fn is_ours(path: &Path) -> bool {
match std::fs::read_to_string(path) {
Ok(text) => text.lines().next() == Some(MARKER),
Err(_) => false,
}
}
/// Removes a file, but only if it is one of ours. Best effort.
fn remove_if_ours(path: &Path) {
if is_ours(path)
&& let Err(err) = std::fs::remove_file(path)
&& err.kind() != std::io::ErrorKind::NotFound
{
tracing::debug!(path = %path.display(), %err, "cannot remove the resolver file");
}
}
/// The set of file names in a desired-files list.
fn keys(files: &[(String, String)]) -> BTreeSet<String> {
files.iter().map(|(name, _)| name.clone()).collect()
}
/// Turns a filesystem error into the right kind of publish error: a permission
/// problem is a refusal a person must act on, anything else is a plain failure.
fn classify(path: &Path, err: std::io::Error) -> PublishError {
if err.kind() == std::io::ErrorKind::PermissionDenied {
PublishError::Refused(format!(
"writing {} needs root; start the agent with `sudo` or as a root LaunchDaemon",
path.display()
))
} else {
PublishError::Failed(format!("cannot write {}: {err}", path.display()))
}
}
fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
match mutex.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::SocketAddr;
use super::*;
fn published(domains: &[&str]) -> Published {
Published {
interface: "utun5".into(),
servers: vec![
"10.13.37.1:5353".parse::<SocketAddr>().unwrap(),
"[fd00::1]:5353".parse::<SocketAddr>().unwrap(),
],
domains: domains.iter().map(|d| d.to_string()).collect(),
}
}
#[test]
fn a_file_lists_every_server_and_one_port_under_the_marker() {
let body = file_contents(&published(&["mynet."]));
let mut lines = body.lines();
assert_eq!(lines.next(), Some(MARKER));
assert_eq!(lines.next(), Some("nameserver 10.13.37.1"));
assert_eq!(lines.next(), Some("nameserver fd00::1"));
assert_eq!(lines.next(), Some("port 5353"));
assert_eq!(lines.next(), None);
}
#[test]
fn a_domain_becomes_a_plain_file_name_without_its_trailing_dot() {
assert_eq!(resolver_file_name("mynet.").unwrap(), "mynet");
assert_eq!(resolver_file_name("my.net").unwrap(), "my.net");
}
#[test]
fn a_domain_that_would_escape_the_directory_is_refused() {
assert!(resolver_file_name("../etc/passwd").is_err());
assert!(resolver_file_name("a/b").is_err());
assert!(resolver_file_name("").is_err());
assert!(resolver_file_name(".").is_err());
assert!(resolver_file_name(".hidden").is_err());
}
#[tokio::test]
async fn apply_writes_a_file_per_domain_and_revert_removes_them() {
let dir = tempfile::tempdir().unwrap();
let publisher = ResolverDirPublisher::in_dir(dir.path());
publisher
.apply(&published(&["mynet.", "other."]))
.await
.unwrap();
assert!(dir.path().join("mynet").is_file());
assert!(dir.path().join("other").is_file());
publisher.revert().await.unwrap();
assert!(!dir.path().join("mynet").exists());
assert!(!dir.path().join("other").exists());
}
#[tokio::test]
async fn re_applying_removes_a_domain_that_is_no_longer_wanted() {
let dir = tempfile::tempdir().unwrap();
let publisher = ResolverDirPublisher::in_dir(dir.path());
publisher.apply(&published(&["a.", "b."])).await.unwrap();
publisher.apply(&published(&["a."])).await.unwrap();
assert!(dir.path().join("a").is_file());
assert!(
!dir.path().join("b").exists(),
"the dropped domain's file is removed"
);
}
#[tokio::test]
async fn a_file_a_user_created_is_never_clobbered_or_removed() {
let dir = tempfile::tempdir().unwrap();
let foreign = dir.path().join("mynet");
std::fs::write(&foreign, "nameserver 9.9.9.9\n").unwrap();
let publisher = ResolverDirPublisher::in_dir(dir.path());
let err = publisher.apply(&published(&["mynet."])).await.unwrap_err();
assert!(matches!(err, PublishError::Failed(_)), "{err:?}");
// Untouched.
assert_eq!(
std::fs::read_to_string(&foreign).unwrap(),
"nameserver 9.9.9.9\n"
);
// And revert leaves a foreign file alone even if it is in our set.
publisher.revert().await.unwrap();
assert!(foreign.is_file());
}
}
+5
View File
@@ -28,6 +28,11 @@ mod windows;
#[cfg(all(feature = "dns-publish", target_os = "windows"))]
pub use windows::NrptPublisher;
#[cfg(all(feature = "dns-publish", target_os = "macos"))]
mod macos;
#[cfg(all(feature = "dns-publish", target_os = "macos"))]
pub use macos::ResolverDirPublisher;
mod unsupported;
pub use unsupported::UnsupportedPublisher;
+50 -1
View File
@@ -82,6 +82,40 @@ pub fn control_socket_path(state_dir: &Path) -> PathBuf {
state_dir.join("agent.sock")
}
/// Where the default system control socket lives on this platform.
///
/// On Windows, the system-wide `%ProgramData%\tsunagi\agent.sock` is used so that
/// any local user, tray application, or CLI tool can manage the running agent
/// without needing to know which user account or service started it.
/// On macOS the agent runs as a root LaunchDaemon, so a fixed system path under
/// `/var/run` is used for the same reason, and it stays short enough for the
/// Unix socket path limit. On other Unix it derives from the default per-user
/// state directory.
pub fn default_control_socket_path() -> PathBuf {
#[cfg(windows)]
{
if let Some(program_data) = std::env::var_os("ProgramData") {
let pd = PathBuf::from(program_data);
if pd.is_absolute() {
return pd.join("tsunagi").join("agent.sock");
}
}
PathBuf::from(r"C:\ProgramData\tsunagi\agent.sock")
}
#[cfg(target_os = "macos")]
{
PathBuf::from("/var/run/tsunagi/agent.sock")
}
#[cfg(all(unix, not(target_os = "macos")))]
{
if let Ok(paths) = crate::config::StoragePaths::user_default() {
control_socket_path(&paths.state_dir)
} else {
PathBuf::from("/run/tsunagi/agent.sock")
}
}
}
/// What a client asks for.
///
/// `Debug` is written by hand rather than derived: one of these carries a
@@ -386,6 +420,21 @@ pub struct OverlayReport {
pub multicast_packets: u64,
/// One destination nobody owned, if there was one.
pub unroutable_sample: Option<String>,
/// The broadcast route and firewall allowance on this host, when the
/// agent installed them. `None` means none are installed.
pub broadcast_rules: Option<HostRulesReport>,
}
/// What the agent installed on the host so LAN broadcast reaches the overlay.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct HostRulesReport {
/// Whether both the route and the firewall allowance are in place.
pub ok: bool,
/// One line: `ok (route + firewall)`, or what is missing and why.
pub detail: String,
/// The overlay address the `255.255.255.255` route is bound to. With
/// several networks on one interface only one address can be its source.
pub source: String,
}
/// One overlay peer.
@@ -541,7 +590,7 @@ pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5);
///
/// Bump it whenever [`Request`], [`Response`] or anything they contain
/// changes shape.
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 14]);
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 15]);
/// Reads one request off an accepted stream, answers it, writes the response.
///
+116 -18
View File
@@ -18,15 +18,16 @@
//!
//! # Who may connect
//!
//! The pipe rejects clients from other machines, and takes the process's
//! default security, under which the creating user has full access. Narrowing
//! that further with an explicit ACL needs a raw security-descriptor call this
//! crate forbids, so it is not attempted; on a single-user machine, and against
//! other unprivileged users, the default is the practical equivalent of the
//! owner-only Unix socket. There is no leftover to clean up: a pipe exists only
//! while its server does.
//! Commands must run as the same Windows user with the same elevation as the
//! agent. A permission denial is reported as such, never as an absent agent.
//! The pipe rejects clients from other machines (`reject_remote_clients(true)`).
//! An explicit SDDL security descriptor grants Full Control to Administrators
//! and LocalSystem, while granting Read and Write (`GRGW`) to all local users
//! with a Medium Integrity SACL label (`S:(ML;;NW;;;ME)`). This allows unprivileged
//! user-space tools (such as tray applications, status monitors, or CLI commands
//! launched by regular users) to connect to the agent even when the agent was
//! started by an elevated Administrator or as a system service.
//! A permission denial is reported as such, never as an absent agent.
#![allow(unsafe_code)]
use std::path::{Path, PathBuf};
use std::sync::Arc;
@@ -50,6 +51,91 @@ const ERROR_ACCESS_DENIED: i32 = 5;
/// is there; the client only has to wait for a free instance.
const ERROR_PIPE_BUSY: i32 = 231;
/// RAII wrapper for Windows named pipe security attributes with permissive local ACL.
///
/// SDDL specification:
/// - `D:(A;;GRGW;;;WD)`: DACL allows Generic Read and Generic Write to Everyone (`WD`).
/// Full control (`GA`) remains reserved to Administrators and LocalSystem.
/// - `(A;;GA;;;BA)`: DACL allows Generic All to Builtin Administrators (`BA`).
/// - `(A;;GA;;;SY)`: DACL allows Generic All to LocalSystem (`SY`).
/// - `S:(ML;;NW;;;ME)`: SACL Mandatory Label with Medium Integrity (`ME`) and No Write Up (`NW`).
/// This allows unprivileged user-space processes (Medium Integrity) to write to the
/// named pipe even when the agent runs as an elevated Administrator (High Integrity).
struct PipeSecurityAttributes {
attrs: windows_sys::Win32::Security::SECURITY_ATTRIBUTES,
}
impl std::fmt::Debug for PipeSecurityAttributes {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("PipeSecurityAttributes")
.field("nLength", &self.attrs.nLength)
.field("lpSecurityDescriptor", &self.attrs.lpSecurityDescriptor)
.field("bInheritHandle", &self.attrs.bInheritHandle)
.finish()
}
}
impl PipeSecurityAttributes {
fn new() -> Result<Self> {
use std::ffi::OsStr;
use std::os::windows::ffi::OsStrExt;
use windows_sys::Win32::Security::Authorization::{
ConvertStringSecurityDescriptorToSecurityDescriptorW, SDDL_REVISION_1,
};
const SDDL: &str = "D:(A;;GRGW;;;WD)(A;;GA;;;BA)(A;;GA;;;SY)S:(ML;;NW;;;ME)";
let wide: Vec<u16> = OsStr::new(SDDL)
.encode_wide()
.chain(std::iter::once(0))
.collect();
let mut p_sd = std::ptr::null_mut();
// SAFETY: We pass a valid null-terminated wide string, standard revision 1,
// and a valid pointer to receive the descriptor.
let success = unsafe {
ConvertStringSecurityDescriptorToSecurityDescriptorW(
wide.as_ptr(),
SDDL_REVISION_1,
&mut p_sd,
std::ptr::null_mut(),
)
};
if success == 0 || p_sd.is_null() {
let err = std::io::Error::last_os_error();
return Err(Error::Storage(format!(
"cannot create pipe security descriptor: {err}"
)));
}
let attrs = windows_sys::Win32::Security::SECURITY_ATTRIBUTES {
nLength: std::mem::size_of::<windows_sys::Win32::Security::SECURITY_ATTRIBUTES>()
as u32,
lpSecurityDescriptor: p_sd,
bInheritHandle: 0,
};
Ok(Self { attrs })
}
fn as_raw_mut(&mut self) -> *mut std::ffi::c_void {
&mut self.attrs as *mut _ as *mut std::ffi::c_void
}
}
impl Drop for PipeSecurityAttributes {
fn drop(&mut self) {
if !self.attrs.lpSecurityDescriptor.is_null() {
// SAFETY: The descriptor was allocated by ConvertStringSecurityDescriptorToSecurityDescriptorW
// and must be freed with LocalFree.
unsafe {
windows_sys::Win32::Foundation::LocalFree(self.attrs.lpSecurityDescriptor);
}
self.attrs.lpSecurityDescriptor = std::ptr::null_mut();
}
}
}
/// Serves the local control interface on a named pipe.
#[derive(Debug)]
pub struct ControlSocket {
@@ -67,11 +153,14 @@ impl ControlSocket {
let path = path.as_ref().to_path_buf();
let name = pipe_name(&path);
let server = match ServerOptions::new()
.first_pipe_instance(true)
.reject_remote_clients(true)
.create(&name)
{
let mut sec_attrs = PipeSecurityAttributes::new()?;
// SAFETY: sec_attrs points to valid initialized SECURITY_ATTRIBUTES.
let server = match unsafe {
ServerOptions::new()
.first_pipe_instance(true)
.reject_remote_clients(true)
.create_with_security_attributes_raw(&name, sec_attrs.as_raw_mut())
} {
Ok(server) => server,
Err(err) if err.raw_os_error() == Some(ERROR_ACCESS_DENIED) => {
return Err(Error::StateLocked { path });
@@ -155,10 +244,19 @@ async fn serve(
/// Creates the next pipe instance, or `None` if the name can no longer be
/// served.
fn next_instance(name: &str) -> Option<tokio::net::windows::named_pipe::NamedPipeServer> {
match ServerOptions::new()
.reject_remote_clients(true)
.create(name)
{
let mut sec_attrs = match PipeSecurityAttributes::new() {
Ok(attrs) => attrs,
Err(err) => {
tracing::debug!(%err, "cannot build pipe security attributes for next instance");
return None;
}
};
// SAFETY: sec_attrs points to valid initialized SECURITY_ATTRIBUTES.
match unsafe {
ServerOptions::new()
.reject_remote_clients(true)
.create_with_security_attributes_raw(name, sec_attrs.as_raw_mut())
} {
Ok(server) => Some(server),
Err(err) => {
tracing::debug!(%err, "cannot create the next control pipe instance");
+51
View File
@@ -240,6 +240,57 @@ impl EndpointAdapter {
TransportPolicy::N0Defaults => builder.preset(presets::N0),
};
if matches!(
config.transport,
TransportPolicy::N0Defaults | TransportPolicy::DirectOnly
) {
let filter = iroh::address_lookup::AddrFilter::new(|addrs| {
let mut overlay_ips = std::collections::HashSet::new();
for iface in netdev::get_interfaces() {
let name = iface.name.to_lowercase();
let friendly = iface
.friendly_name
.as_deref()
.map(str::to_lowercase)
.unwrap_or_default();
let desc = iface
.description
.as_deref()
.map(str::to_lowercase)
.unwrap_or_default();
if name.starts_with("tsun")
|| name.contains("tsunagi")
|| name.contains("wintun")
|| friendly.starts_with("tsun")
|| friendly.contains("tsunagi")
|| friendly.contains("wintun")
|| desc.contains("wintun")
|| desc.contains("tsunagi")
{
for ip in iface.ipv4 {
overlay_ips.insert(std::net::IpAddr::V4(ip.addr()));
}
for ip in iface.ipv6 {
overlay_ips.insert(std::net::IpAddr::V6(ip.addr()));
}
}
}
if overlay_ips.is_empty() {
return std::borrow::Cow::Borrowed(addrs);
}
let filtered: Vec<iroh::TransportAddr> = addrs
.iter()
.filter(|addr| match addr {
iroh::TransportAddr::Ip(sock) => !overlay_ips.contains(&sock.ip()),
_ => true,
})
.cloned()
.collect();
std::borrow::Cow::Owned(filtered)
});
builder = builder.addr_filter(filter);
}
#[cfg(feature = "testing")]
if let Some(transport) = &config.test_quic_transport {
builder = builder.transport_config(transport.clone());
@@ -0,0 +1,566 @@
//! Broadcast host rules on Linux: a netlink route and an `iptables` rule.
//!
//! The route is added by the netlink thread the interface provisioner already
//! owns (see [`super::super::provision`]), because that thread is the only one
//! that ever holds `CAP_NET_ADMIN`. It belongs to the interface and goes away
//! with it.
//!
//! The firewall allowance is a rule in the `INPUT` chain, inserted at the top
//! so a default-deny chain further down cannot drop discovery replies first:
//!
//! ```text
//! iptables -I INPUT 1 -i <if> -p udp -s <range> -m comment --comment tsunagi:<if> -j ACCEPT
//! ```
//!
//! `iptables` is a program, not a netlink call, so it is looked up in fixed
//! system directories rather than on `PATH`, and its arguments are built from
//! values this agent computed. It works with both the legacy backend and
//! `iptables-nft`. A host with only `nft` or `firewalld` has no `iptables`;
//! that is reported on the firewall half and the route is still installed.
//!
//! The rule carries a comment tag. That is how a rule left behind by a crashed
//! run is found and replaced, and why nothing else in the chain is ever
//! deleted.
use std::net::Ipv4Addr;
use crate::BoxFuture;
use crate::overlay::provision::RouteHandle;
use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome, tag_for};
const CHAIN: &str = "INPUT";
/// Fixed places `iptables` is installed. Never `PATH`: the agent is
/// privileged and a directory a user controls must not decide what it runs.
const IPTABLES_DIRECTORIES: [&str; 5] = [
"/usr/sbin/iptables",
"/sbin/iptables",
"/usr/bin/iptables",
"/bin/iptables",
"/usr/local/sbin/iptables",
];
/// The first candidate that exists.
fn pick_iptables(exists: impl Fn(&str) -> bool) -> Option<&'static str> {
IPTABLES_DIRECTORIES
.into_iter()
.find(|candidate| exists(candidate))
}
/// What running `iptables` produced.
#[derive(Debug, Clone, PartialEq, Eq)]
enum Ran {
/// There is no `iptables` to run.
Missing,
/// It ran, and exited like this.
Exited {
success: bool,
stdout: String,
stderr: String,
},
}
/// Runs `iptables` with these arguments. A function value so the flow below
/// can be tested without touching the host.
type Run<'a> = &'a dyn Fn(&[String]) -> Ran;
/// The match and target of the rule, without the chain or position.
fn rule_spec(plan: &BroadcastRulesPlan) -> Vec<String> {
[
"-i",
plan.interface.as_str(),
"-p",
"udp",
"-s",
&plan.range.to_string(),
"-m",
"comment",
"--comment",
&plan.tag(),
"-j",
"ACCEPT",
]
.into_iter()
.map(str::to_string)
.collect()
}
/// Waits briefly for the xtables lock instead of failing when another tool
/// holds it.
fn locked(args: impl IntoIterator<Item = String>) -> Vec<String> {
["-w", "5"]
.into_iter()
.map(str::to_string)
.chain(args)
.collect()
}
fn check_args(plan: &BroadcastRulesPlan) -> Vec<String> {
locked(
["-C".to_string(), CHAIN.to_string()]
.into_iter()
.chain(rule_spec(plan)),
)
}
fn insert_args(plan: &BroadcastRulesPlan) -> Vec<String> {
locked(
["-I".to_string(), CHAIN.to_string(), "1".to_string()]
.into_iter()
.chain(rule_spec(plan)),
)
}
fn list_args() -> Vec<String> {
locked(["-S".to_string(), CHAIN.to_string()])
}
/// Splits one `iptables -S` line into words, honouring double quotes, which
/// `iptables` puts around comments that need them.
fn split_rule_line(line: &str) -> Vec<String> {
let mut words = Vec::new();
let mut current = String::new();
let mut quoted = false;
let mut started = false;
for c in line.chars() {
match c {
'"' => {
quoted = !quoted;
started = true;
}
c if c.is_whitespace() && !quoted => {
if started {
words.push(std::mem::take(&mut current));
started = false;
}
}
c => {
current.push(c);
started = true;
}
}
}
if started {
words.push(current);
}
words
}
/// The delete commands for every rule in an `iptables -S INPUT` listing that
/// carries exactly this tag. Anything else in the chain is left alone.
fn tagged_deletions(listing: &str, tag: &str) -> Vec<Vec<String>> {
listing
.lines()
.filter_map(|line| {
let words = split_rule_line(line);
let [append, chain, rest @ ..] = words.as_slice() else {
return None;
};
if append != "-A" || chain != CHAIN {
return None;
}
let tagged = rest
.windows(2)
.any(|pair| pair[0] == "--comment" && pair[1] == tag);
tagged.then(|| {
locked(
["-D".to_string(), CHAIN.to_string()]
.into_iter()
.chain(rest.iter().cloned()),
)
})
})
.collect()
}
/// A failure message with the likely fix for the likely cause.
fn failure(stderr: &str) -> String {
let text = stderr.trim();
let lower = text.to_ascii_lowercase();
if lower.contains("permission denied") || lower.contains("must be root") {
format!("iptables needs root: {text}")
} else if text.is_empty() {
"iptables failed without saying why".to_string()
} else {
format!("iptables failed: {text}")
}
}
fn missing(plan: &BroadcastRulesPlan) -> String {
format!(
"iptables not found; install it (the iptables-nft package is enough) or allow inbound \
UDP from {} on `{}` in your firewall",
plan.range, plan.interface
)
}
/// Makes the firewall hold exactly one rule for this interface, the planned one.
fn apply_firewall(run: Run<'_>, plan: &BroadcastRulesPlan) -> RuleOutcome {
let present = match run(&check_args(plan)) {
Ran::Missing => return RuleOutcome::Failed(missing(plan)),
Ran::Exited { success, .. } => success,
};
let listing = match run(&list_args()) {
Ran::Missing => return RuleOutcome::Failed(missing(plan)),
Ran::Exited {
success: true,
stdout,
..
} => stdout,
Ran::Exited { stderr, .. } => return RuleOutcome::Failed(failure(&stderr)),
};
let stale = tagged_deletions(&listing, &plan.tag());
// `-C` matched an identical rule and nothing else carries our tag, so
// there is nothing to do. Otherwise start from a clean slate for this
// interface: a rule from an earlier run may name another range.
if present && stale.len() == 1 {
return RuleOutcome::Applied;
}
for delete in &stale {
if let Ran::Exited {
success: false,
stderr,
..
} = run(delete)
{
return RuleOutcome::Failed(failure(&stderr));
}
}
match run(&insert_args(plan)) {
Ran::Missing => RuleOutcome::Failed(missing(plan)),
Ran::Exited { success: true, .. } => RuleOutcome::Applied,
Ran::Exited { stderr, .. } => RuleOutcome::Failed(failure(&stderr)),
}
}
/// Removes every rule tagged for this interface. Best effort.
fn clear_firewall(run: Run<'_>, interface: &str) {
let Ran::Exited {
success: true,
stdout,
..
} = run(&list_args())
else {
return;
};
for delete in tagged_deletions(&stdout, &tag_for(interface)) {
if let Ran::Exited {
success: false,
stderr,
..
} = run(&delete)
{
tracing::warn!(
interface,
"cannot remove the broadcast firewall rule: {}",
failure(&stderr)
);
}
}
}
/// Runs the real `iptables`.
fn run_iptables(args: &[String]) -> Ran {
let Some(program) = pick_iptables(|path| std::path::Path::new(path).is_file()) else {
return Ran::Missing;
};
match std::process::Command::new(program).args(args).output() {
Ok(output) => Ran::Exited {
success: output.status.success(),
stdout: String::from_utf8_lossy(&output.stdout).into_owned(),
stderr: String::from_utf8_lossy(&output.stderr).into_owned(),
},
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ran::Missing,
Err(err) => Ran::Exited {
success: false,
stdout: String::new(),
stderr: err.to_string(),
},
}
}
/// The Linux implementation: netlink for the route, `iptables` for the rule.
#[derive(Debug)]
pub struct LinuxHostRules {
route: RouteHandle,
}
impl LinuxHostRules {
pub(crate) fn new(route: RouteHandle) -> Self {
Self { route }
}
async fn set_route(&self, interface: String, source: Ipv4Addr) -> RuleOutcome {
let route = self.route.clone();
match tokio::task::spawn_blocking(move || route.set(&interface, source)).await {
Ok(Ok(())) => RuleOutcome::Applied,
Ok(Err(err)) => RuleOutcome::Failed(err.to_string()),
Err(err) => RuleOutcome::Failed(format!("the route task failed: {err}")),
}
}
}
impl BroadcastHostRules for LinuxHostRules {
fn name(&self) -> &str {
"netlink+iptables"
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
let route = self.set_route(plan.interface.clone(), plan.source).await;
let owned = plan.clone();
let firewall =
tokio::task::spawn_blocking(move || apply_firewall(&run_iptables, &owned))
.await
.unwrap_or_else(|err| {
RuleOutcome::Failed(format!("the firewall task failed: {err}"))
});
BroadcastRulesReport {
plan: plan.clone(),
route,
firewall,
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
let route = self.route.clone();
let name = interface.to_string();
let route_name = name.clone();
if let Ok(Err(err)) =
tokio::task::spawn_blocking(move || route.clear(&route_name)).await
{
tracing::warn!(interface = %name, %err, "cannot remove the broadcast route");
}
let name = interface.to_string();
let _ = tokio::task::spawn_blocking(move || clear_firewall(&run_iptables, &name)).await;
})
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use std::cell::RefCell;
fn plan() -> BroadcastRulesPlan {
BroadcastRulesPlan {
interface: "tsun0".into(),
source: Ipv4Addr::new(10, 13, 37, 142),
range: "10.13.37.0/24".parse().unwrap(),
}
}
fn words(args: &[String]) -> String {
args.join(" ")
}
#[test]
fn the_rule_is_udp_only_from_the_overlay_range_on_this_interface() {
let spec = words(&rule_spec(&plan()));
assert_eq!(
spec,
"-i tsun0 -p udp -s 10.13.37.0/24 -m comment --comment tsunagi:tsun0 -j ACCEPT"
);
// Nothing here may widen to other protocols or every source.
assert!(spec.contains("-p udp"));
assert!(!spec.contains("0.0.0.0/0"));
assert!(words(&insert_args(&plan())).starts_with("-w 5 -I INPUT 1 -i tsun0"));
assert!(words(&check_args(&plan())).starts_with("-w 5 -C INPUT -i tsun0"));
}
#[test]
fn iptables_is_found_in_fixed_directories_never_on_path() {
assert_eq!(pick_iptables(|_| false), None);
assert_eq!(
pick_iptables(|p| p == "/sbin/iptables"),
Some("/sbin/iptables")
);
assert_eq!(pick_iptables(|_| true), Some("/usr/sbin/iptables"));
}
#[test]
fn only_rules_with_our_exact_tag_are_selected_for_deletion() {
let listing = "\
-P INPUT DROP
-A INPUT -s 10.13.37.0/24 -i tsun0 -p udp -m comment --comment \"tsunagi:tsun0\" -j ACCEPT
-A INPUT -i tsun0 -m comment --comment tsunagi:tsun0 -j ACCEPT
-A INPUT -i tsun1 -p udp -m comment --comment tsunagi:tsun1 -j ACCEPT
-A INPUT -p tcp --dport 22 -m comment --comment \"allow ssh tsunagi:tsun0\" -j ACCEPT
-A FORWARD -i tsun0 -m comment --comment tsunagi:tsun0 -j ACCEPT
-A INPUT -p tcp --dport 22 -j ACCEPT
";
let deletions = tagged_deletions(listing, "tsunagi:tsun0");
assert_eq!(deletions.len(), 2, "{deletions:?}");
assert_eq!(
words(&deletions[0]),
"-w 5 -D INPUT -s 10.13.37.0/24 -i tsun0 -p udp -m comment --comment tsunagi:tsun0 -j ACCEPT"
);
assert!(words(&deletions[1]).contains("-D INPUT -i tsun0 -m comment"));
assert!(tagged_deletions("", "tsunagi:tsun0").is_empty());
}
/// A scripted `iptables`: answers by the leading verb, records every call.
struct Fake {
calls: RefCell<Vec<String>>,
check_ok: bool,
listing: String,
insert_stderr: Option<&'static str>,
missing: bool,
}
impl Fake {
fn new(check_ok: bool, listing: &str) -> Self {
Self {
calls: RefCell::new(Vec::new()),
check_ok,
listing: listing.to_string(),
insert_stderr: None,
missing: false,
}
}
fn run(&self, args: &[String]) -> Ran {
self.calls.borrow_mut().push(words(args));
if self.missing {
return Ran::Missing;
}
let ok = |stdout: &str| Ran::Exited {
success: true,
stdout: stdout.to_string(),
stderr: String::new(),
};
match args.get(2).map(String::as_str) {
Some("-C") if self.check_ok => ok(""),
Some("-C") => Ran::Exited {
success: false,
stdout: String::new(),
stderr: "Bad rule".into(),
},
Some("-S") => ok(&self.listing),
Some("-I") => match self.insert_stderr {
Some(stderr) => Ran::Exited {
success: false,
stdout: String::new(),
stderr: stderr.into(),
},
None => ok(""),
},
_ => ok(""),
}
}
fn verbs(&self) -> Vec<String> {
self.calls
.borrow()
.iter()
.map(|call| call.split(' ').nth(2).unwrap().to_string())
.collect()
}
}
const ONE_OURS: &str =
"-A INPUT -s 10.13.37.0/24 -i tsun0 -p udp -m comment --comment tsunagi:tsun0 -j ACCEPT\n";
#[test]
fn an_identical_rule_already_in_place_is_left_alone() {
let fake = Fake::new(true, ONE_OURS);
assert_eq!(
apply_firewall(&|a| fake.run(a), &plan()),
RuleOutcome::Applied
);
assert_eq!(fake.verbs(), ["-C", "-S"], "no insert, no delete");
}
#[test]
fn a_missing_rule_is_inserted_once() {
let fake = Fake::new(false, "-P INPUT ACCEPT\n");
assert_eq!(
apply_firewall(&|a| fake.run(a), &plan()),
RuleOutcome::Applied
);
assert_eq!(fake.verbs(), ["-C", "-S", "-I"]);
}
#[test]
fn a_stale_rule_for_another_range_is_replaced_not_stacked() {
// A crashed run left a rule for an old range under our tag.
let stale = "-A INPUT -s 10.99.0.0/16 -i tsun0 -p udp -m comment --comment tsunagi:tsun0 -j ACCEPT\n";
let fake = Fake::new(false, stale);
assert_eq!(
apply_firewall(&|a| fake.run(a), &plan()),
RuleOutcome::Applied
);
assert_eq!(fake.verbs(), ["-C", "-S", "-D", "-I"]);
}
#[test]
fn duplicates_of_the_rule_collapse_to_one() {
let doubled = format!("{ONE_OURS}{ONE_OURS}");
let fake = Fake::new(true, &doubled);
assert_eq!(
apply_firewall(&|a| fake.run(a), &plan()),
RuleOutcome::Applied
);
assert_eq!(fake.verbs(), ["-C", "-S", "-D", "-D", "-I"]);
}
#[test]
fn a_missing_iptables_says_what_to_install_and_what_to_allow() {
let mut fake = Fake::new(false, "");
fake.missing = true;
let RuleOutcome::Failed(reason) = apply_firewall(&|a| fake.run(a), &plan()) else {
panic!("expected a failure");
};
assert!(reason.contains("iptables not found"), "{reason}");
assert!(reason.contains("10.13.37.0/24"), "{reason}");
assert!(reason.contains("tsun0"), "{reason}");
}
#[test]
fn a_refused_insert_is_reported_with_the_cause() {
let mut fake = Fake::new(false, "");
fake.insert_stderr = Some(
"iptables v1.8.10: can't initialize iptables table `filter': Permission denied (you must be root)",
);
let RuleOutcome::Failed(reason) = apply_firewall(&|a| fake.run(a), &plan()) else {
panic!("expected a failure");
};
assert!(reason.starts_with("iptables needs root"), "{reason}");
}
#[test]
fn clearing_removes_only_tagged_rules_and_ignores_a_missing_iptables() {
let listing = format!(
"-A INPUT -p tcp --dport 22 -j ACCEPT\n{ONE_OURS}-A INPUT -i tsun1 -m comment --comment tsunagi:tsun1 -j ACCEPT\n"
);
let fake = Fake::new(true, &listing);
clear_firewall(&|a| fake.run(a), "tsun0");
assert_eq!(fake.verbs(), ["-S", "-D"]);
let mut gone = Fake::new(true, "");
gone.missing = true;
clear_firewall(&|a| gone.run(a), "tsun0");
assert_eq!(gone.verbs(), ["-S"]);
}
#[test]
fn rule_listing_words_honour_quotes() {
assert_eq!(
split_rule_line("-A INPUT -m comment --comment \"a b\" -j ACCEPT"),
[
"-A",
"INPUT",
"-m",
"comment",
"--comment",
"a b",
"-j",
"ACCEPT"
]
);
assert!(split_rule_line(" ").is_empty());
}
}
@@ -0,0 +1,189 @@
//! Broadcast host rules on macOS: a `route` entry; the firewall is deferred.
//!
//! The limited-broadcast route is installed with `/sbin/route`, so a game's
//! LAN discovery packet leaves through the overlay interface. It belongs to
//! the interface and is removed when broadcast is turned off, the network
//! leaves, or the agent stops — and, because a utun is torn down with the
//! process, it also goes away on its own if the agent is killed.
//!
//! The inbound-UDP firewall allowance is **not** implemented here. On macOS
//! that means a `pf` anchor, which is a good deal more machinery; until it is
//! written the firewall half reports itself as missing with what to allow by
//! hand, and the route is still installed. Failure is contained: neither half
//! can stop the agent, and `status` shows exactly which is in place.
//!
//! `route` is a program, not a system call, so it is run by absolute path from
//! `/sbin` rather than off `PATH`, and its arguments are built from values
//! this agent computed — the interface name the kernel assigned, nothing a
//! peer said.
use crate::BoxFuture;
use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome};
/// The limited-broadcast destination routed through the overlay interface.
const LIMITED_BROADCAST: &str = "255.255.255.255";
/// The `route` arguments that direct the limited broadcast out an interface.
fn route_add_args(interface: &str) -> Vec<String> {
vec![
"-q".into(),
"-n".into(),
"add".into(),
"-host".into(),
LIMITED_BROADCAST.into(),
"-interface".into(),
interface.into(),
]
}
/// The `route` arguments that remove it again.
fn route_del_args(interface: &str) -> Vec<String> {
vec![
"-q".into(),
"-n".into(),
"delete".into(),
"-host".into(),
LIMITED_BROADCAST.into(),
"-interface".into(),
interface.into(),
]
}
/// What to do by hand while the `pf` allowance is unimplemented.
fn firewall_todo(plan: &BroadcastRulesPlan) -> String {
format!(
"the inbound-UDP firewall allowance is not implemented on macOS yet; if the host \
firewall drops discovery replies, allow inbound UDP from {} on `{}` with a pf rule \
yourself",
plan.range, plan.interface
)
}
/// The macOS implementation: `route` for the route, no firewall yet.
#[derive(Debug, Default)]
pub struct MacosHostRules;
impl MacosHostRules {
/// Creates the host rules.
pub fn new() -> Self {
Self
}
/// Installs the route, replacing one this agent added earlier so the call
/// can be repeated. The delete is best effort: a route that is not there
/// is the state the add then reaches anyway.
async fn set_route(&self, interface: String) -> RuleOutcome {
let _ = run(route_del_args(&interface)).await;
match run(route_add_args(&interface)).await {
Ok(()) => RuleOutcome::Applied,
Err(reason) => RuleOutcome::Failed(reason),
}
}
}
impl BroadcastHostRules for MacosHostRules {
fn name(&self) -> &str {
"route"
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
let route = self.set_route(plan.interface.clone()).await;
BroadcastRulesReport {
plan: plan.clone(),
route,
firewall: RuleOutcome::Failed(firewall_todo(plan)),
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
if let Err(reason) = run(route_del_args(interface)).await {
tracing::debug!(interface = %interface, %reason, "cannot remove the broadcast route");
}
})
}
}
/// Runs `/sbin/route`, turning a non-zero exit into the message it printed.
async fn run(args: Vec<String>) -> Result<(), String> {
let display = format!("route {}", args.join(" "));
let output = tokio::task::spawn_blocking(move || {
std::process::Command::new("/sbin/route")
.args(&args)
.output()
})
.await
.map_err(|err| format!("could not run {display}: {err}"))?
.map_err(|err| format!("could not run {display}: {err}"))?;
if output.status.success() {
return Ok(());
}
let stderr = String::from_utf8_lossy(&output.stderr);
let text = stderr.trim();
if text.is_empty() {
Err(format!("{display} failed with {}", output.status))
} else {
Err(format!("{display} failed: {text}"))
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::Ipv4Addr;
use super::*;
fn plan() -> BroadcastRulesPlan {
BroadcastRulesPlan {
interface: "utun5".into(),
source: Ipv4Addr::new(10, 13, 37, 142),
range: "10.13.37.0/24".parse().unwrap(),
}
}
#[test]
fn the_route_is_the_limited_broadcast_out_the_interface() {
assert_eq!(
route_add_args("utun5"),
vec![
"-q",
"-n",
"add",
"-host",
"255.255.255.255",
"-interface",
"utun5"
]
);
assert_eq!(
route_del_args("utun5"),
vec![
"-q",
"-n",
"delete",
"-host",
"255.255.255.255",
"-interface",
"utun5"
]
);
}
#[test]
fn the_firewall_todo_names_the_range_and_interface() {
let todo = firewall_todo(&plan());
assert!(todo.contains("10.13.37.0/24"), "{todo}");
assert!(todo.contains("utun5"), "{todo}");
assert!(todo.contains("pf"), "{todo}");
// Deferred, so the firewall half is always reported as not in place;
// the route runs for real and so is left to an on-host check. Running
// `route` here would need root and would change the host routing
// table, which the default test suite must never do.
}
}
+330
View File
@@ -0,0 +1,330 @@
//! Host rules that let a game's LAN discovery reach the overlay interface.
//!
//! Broadcast fanout happens at local TUN ingress (see [`super::broadcast`]),
//! so a limited broadcast only reaches Tsunagi if the operating system routes
//! it through the overlay interface, and a discovery *reply* only reaches the
//! game if the host firewall lets it in. Left to the user that is two manual
//! steps per host that nobody remembers, so when a network's broadcast is on
//! the agent installs exactly these two things for the interface it owns:
//!
//! * a `255.255.255.255/32` route through the overlay interface, with the
//! overlay address as its preferred source, so the game's discovery packet
//! leaves with a source address the router recognises;
//! * an inbound firewall allowance for **UDP only**, from the overlay range
//! only, on that interface only. Replies to a broadcast come from a
//! different address than the one the request was sent to, so a stateful
//! firewall never matches them to the request. Opening every protocol
//! would expose services such as SSH to every member of the network, which
//! is not what a game needs.
//!
//! # Why this is allowed here
//!
//! The architecture says plugins never touch routing or firewall settings,
//! and nothing a peer announces becomes an OS setting. This is neither: it is
//! the system level, which owns the interface, configuring that interface from
//! values it computed itself (the interface name is derived locally, the
//! address and range come from signed state it already verified). Every object
//! is tagged `tsunagi:<interface>`, reversible, and removed when broadcast is
//! turned off, the network leaves, or the agent stops. A crash can leave a
//! firewall rule behind; the next apply for the same interface removes it.
//!
//! # Failure is contained
//!
//! Neither step can stop the agent. Each reports its own outcome, so a missing
//! firewall tool still leaves the route in place and says what is missing.
//!
//! # Why a trait
//!
//! Linux, Windows and the tests do this very differently, while *what* to
//! install is the same everywhere and lives in [`BroadcastRulesPlan`] and
//! [`rules_summary`], tested on every platform.
use std::net::Ipv4Addr;
use std::sync::{Arc, Mutex};
use crate::BoxFuture;
use crate::state::Ipv4Range;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
mod linux;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use linux::LinuxHostRules;
// Compiled everywhere so the scripts it builds are tested everywhere; only
// the type that runs them is exported on Windows.
mod windows;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use windows::WindowsHostRules;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
mod macos;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use macos::MacosHostRules;
/// What the host should be told for one interface.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BroadcastRulesPlan {
/// The overlay interface. Derived locally from the network id, never from
/// anything a peer said.
pub interface: String,
/// This agent's overlay address, the route's preferred source.
pub source: Ipv4Addr,
/// The overlay range inbound UDP is admitted from.
pub range: Ipv4Range,
}
impl BroadcastRulesPlan {
/// The tag every object carries, so it can be found again and nothing
/// else is ever touched.
pub fn tag(&self) -> String {
tag_for(&self.interface)
}
}
/// The tag for an interface's objects.
pub fn tag_for(interface: &str) -> String {
format!("tsunagi:{interface}")
}
/// What happened to one of the two host objects.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum RuleOutcome {
/// It is in place.
Applied,
/// It is not, with a reason a person can act on.
Failed(String),
}
impl RuleOutcome {
/// Whether the object is in place.
pub fn is_applied(&self) -> bool {
matches!(self, Self::Applied)
}
}
/// What applying a plan achieved.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BroadcastRulesReport {
/// What was asked for.
pub plan: BroadcastRulesPlan,
/// The `255.255.255.255` route.
pub route: RuleOutcome,
/// The inbound firewall allowance.
pub firewall: RuleOutcome,
}
impl BroadcastRulesReport {
/// Whether both objects are in place.
pub fn is_ok(&self) -> bool {
self.route.is_applied() && self.firewall.is_applied()
}
/// One line for `status`.
pub fn summary(&self) -> String {
rules_summary(&self.route, &self.firewall)
}
}
/// The status line for a pair of outcomes.
pub fn rules_summary(route: &RuleOutcome, firewall: &RuleOutcome) -> String {
match (route, firewall) {
(RuleOutcome::Applied, RuleOutcome::Applied) => "ok (route + firewall)".to_string(),
(route, firewall) => {
let part = |what: &str, outcome: &RuleOutcome| match outcome {
RuleOutcome::Applied => format!("{what} ok"),
RuleOutcome::Failed(reason) => format!("{what} failed: {reason}"),
};
format!(
"incomplete; {}; {}",
part("route", route),
part("firewall", firewall)
)
}
}
}
/// Installs and removes the host rules for an overlay interface.
///
/// Both calls are idempotent. Neither can fail as a whole: a step that cannot
/// be done is reported in the [`BroadcastRulesReport`], because the caller's
/// only useful response is to say so.
pub trait BroadcastHostRules: Send + Sync + std::fmt::Debug + 'static {
/// A short name used in diagnostics.
fn name(&self) -> &str;
/// Makes the host match the plan, replacing whatever this agent put there
/// for the same interface before.
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport>;
/// Removes what this agent put there for the interface. Succeeds when
/// nothing is there: it runs on the shutdown path.
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()>;
}
/// The platform has no implementation, which is said rather than pretended.
#[derive(Debug, Default)]
pub struct UnsupportedHostRules;
impl BroadcastHostRules for UnsupportedHostRules {
fn name(&self) -> &str {
"unsupported"
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
let reason = "host rules are not implemented on this platform; route \
255.255.255.255 through the overlay interface and allow inbound \
UDP from the overlay range yourself"
.to_string();
BroadcastRulesReport {
plan: plan.clone(),
route: RuleOutcome::Failed(reason.clone()),
firewall: RuleOutcome::Failed(reason),
}
})
}
fn clear<'a>(&'a self, _interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {})
}
}
/// What a [`MockHostRules`] currently holds, per interface.
pub type MockRulesState = std::collections::BTreeMap<String, BroadcastRulesPlan>;
/// An in-memory host, for tests: nothing is touched.
#[derive(Debug, Clone, Default)]
pub struct MockHostRules {
state: Arc<Mutex<MockRulesState>>,
calls: Arc<Mutex<Vec<String>>>,
firewall_failure: Arc<Mutex<Option<String>>>,
}
impl MockHostRules {
/// An empty host.
pub fn new() -> Self {
Self::default()
}
/// Makes the firewall step fail with this reason from now on.
pub fn fail_firewall(&self, reason: impl Into<String>) {
*lock(&self.firewall_failure) = Some(reason.into());
}
/// What is installed now.
pub fn installed(&self) -> MockRulesState {
lock(&self.state).clone()
}
/// Every `apply:<interface>` / `clear:<interface>` call, in order.
pub fn calls(&self) -> Vec<String> {
lock(&self.calls).clone()
}
}
fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
match mutex.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
impl BroadcastHostRules for MockHostRules {
fn name(&self) -> &str {
"mock"
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
lock(&self.calls).push(format!("apply:{}", plan.interface));
lock(&self.state).insert(plan.interface.clone(), plan.clone());
let firewall = match lock(&self.firewall_failure).clone() {
Some(reason) => RuleOutcome::Failed(reason),
None => RuleOutcome::Applied,
};
BroadcastRulesReport {
plan: plan.clone(),
route: RuleOutcome::Applied,
firewall,
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
lock(&self.calls).push(format!("clear:{interface}"));
lock(&self.state).remove(interface);
})
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
fn plan() -> BroadcastRulesPlan {
BroadcastRulesPlan {
interface: "tsun0".into(),
source: Ipv4Addr::new(10, 13, 37, 142),
range: "10.13.37.0/24".parse().unwrap(),
}
}
#[test]
fn every_object_is_tagged_with_its_interface() {
assert_eq!(plan().tag(), "tsunagi:tsun0");
assert_eq!(tag_for("tsun7"), "tsunagi:tsun7");
}
#[test]
fn the_summary_says_which_half_is_missing() {
assert_eq!(
rules_summary(&RuleOutcome::Applied, &RuleOutcome::Applied),
"ok (route + firewall)"
);
let line = rules_summary(
&RuleOutcome::Applied,
&RuleOutcome::Failed("iptables not found".into()),
);
assert!(line.contains("route ok"), "{line}");
assert!(
line.contains("firewall failed: iptables not found"),
"{line}"
);
let report = BroadcastRulesReport {
plan: plan(),
route: RuleOutcome::Failed("denied".into()),
firewall: RuleOutcome::Applied,
};
assert!(!report.is_ok());
assert!(report.summary().contains("route failed: denied"));
}
#[tokio::test]
async fn the_mock_applies_replaces_and_clears_idempotently() {
let rules = MockHostRules::new();
let first = rules.apply(&plan()).await;
assert!(first.is_ok());
assert_eq!(rules.installed().len(), 1);
let mut moved = plan();
moved.source = Ipv4Addr::new(10, 13, 37, 7);
rules.apply(&moved).await;
assert_eq!(rules.installed()["tsun0"].source, moved.source);
rules.clear("tsun0").await;
rules.clear("tsun0").await;
assert!(rules.installed().is_empty());
assert_eq!(rules.calls().len(), 4);
}
#[tokio::test]
async fn an_unsupported_platform_reports_both_steps_failed() {
let report = UnsupportedHostRules.apply(&plan()).await;
assert!(!report.route.is_applied());
assert!(!report.firewall.is_applied());
UnsupportedHostRules.clear("tsun0").await;
}
}
@@ -0,0 +1,272 @@
//! Broadcast host rules on Windows: a route and a Defender Firewall rule.
//!
//! Both are written with the `NetTCPIP` and `NetSecurity` PowerShell modules,
//! run by absolute path from `%SystemRoot%` like the DNS publisher does: the
//! in-process alternatives are `unsafe` IP Helper calls this crate forbids.
//!
//! * The route is `255.255.255.255/32` on the overlay interface, on-link, with
//! metric 1, written to the active store only so it cannot outlive a reboot.
//! Windows picks the interface's own address as the source of a packet sent
//! along it; with several networks on one interface that is whichever
//! address Windows treats as primary, which this cannot choose.
//! * The firewall rule allows inbound UDP on that interface alias from the
//! overlay range only, in every profile. Wintun adapters are usually
//! classified as a public network, where inbound UDP is otherwise dropped.
//! Its display name is the tag `tsunagi:<interface>`, which is how it is
//! found again, replaced and removed.
//!
//! Every value in a script is one this agent produced: the interface name is
//! derived from the network id and the range comes from signed state. They are
//! still single-quoted and escaped, so nothing could be read as PowerShell
//! rather than as data. Changing routes and firewall rules needs an elevated
//! process, which is reported as its own cause.
//!
//! This module is compiled everywhere so its script building is tested
//! everywhere. It could not be run against a real Windows host while it was
//! written.
#![cfg_attr(not(target_os = "windows"), allow(dead_code))]
use crate::BoxFuture;
use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome, tag_for};
/// A PowerShell single-quoted literal, with any embedded quote doubled.
fn ps_literal(value: &str) -> String {
format!("'{}'", value.replace('\'', "''"))
}
/// The step that makes the route match the plan.
///
/// A leftover route (from a crashed run, or for an old interface index) is
/// removed first so the result is exactly one route.
fn route_script(plan: &BroadcastRulesPlan) -> String {
let alias = ps_literal(&plan.interface);
format!(
"$ErrorActionPreference = 'Stop'\n\
$alias = {alias}\n\
Get-NetRoute -DestinationPrefix '255.255.255.255/32' -InterfaceAlias $alias \
-PolicyStore ActiveStore -ErrorAction SilentlyContinue | \
Remove-NetRoute -Confirm:$false -ErrorAction SilentlyContinue\n\
New-NetRoute -DestinationPrefix '255.255.255.255/32' -InterfaceAlias $alias \
-NextHop '0.0.0.0' -RouteMetric 1 -PolicyStore ActiveStore | Out-Null\n"
)
}
/// The step that makes the firewall hold exactly one rule for the interface.
fn firewall_script(plan: &BroadcastRulesPlan) -> String {
let alias = ps_literal(&plan.interface);
let tag = ps_literal(&plan.tag());
let range = ps_literal(&plan.range.to_string());
format!(
"$ErrorActionPreference = 'Stop'\n\
$alias = {alias}\n\
$tag = {tag}\n\
Get-NetFirewallRule -DisplayName $tag -ErrorAction SilentlyContinue | \
Remove-NetFirewallRule -ErrorAction SilentlyContinue\n\
New-NetFirewallRule -DisplayName $tag -Direction Inbound -Action Allow -Protocol UDP \
-InterfaceAlias $alias -RemoteAddress {range} -Profile Any | Out-Null\n"
)
}
/// The script that removes both, ignoring what is not there.
fn clear_script(interface: &str) -> String {
let alias = ps_literal(interface);
let tag = ps_literal(&tag_for(interface));
format!(
"$ErrorActionPreference = 'SilentlyContinue'\n\
Get-NetRoute -DestinationPrefix '255.255.255.255/32' -InterfaceAlias {alias} \
-PolicyStore ActiveStore | Remove-NetRoute -Confirm:$false\n\
Get-NetFirewallRule -DisplayName {tag} | Remove-NetFirewallRule\n\
exit 0\n"
)
}
/// Turns what PowerShell said into a cause a person can act on.
fn classify(what: &str, text: &str) -> String {
let lower = text.to_ascii_lowercase();
if lower.contains("access is denied")
|| lower.contains("requires elevation")
|| lower.contains("run as administrator")
|| lower.contains("permissiondenied")
{
format!("{what}: Windows refused; run tsunagi from an elevated (administrator) process")
} else if lower.contains("is not recognized") || lower.contains("commandnotfound") {
format!("{what}: the NetTCPIP/NetSecurity PowerShell modules are not available: {text}")
} else if text.is_empty() {
format!("{what}: failed without saying why")
} else {
format!("{what}: {text}")
}
}
/// The absolute path to Windows PowerShell, so nothing on `PATH` can stand in
/// for it.
fn powershell_path() -> std::path::PathBuf {
let root = std::env::var_os("SystemRoot").unwrap_or_else(|| r"C:\Windows".into());
std::path::Path::new(&root)
.join("System32")
.join("WindowsPowerShell")
.join("v1.0")
.join("powershell.exe")
}
/// Runs a script and returns what it said on failure.
fn run_powershell(script: &str) -> Result<(), String> {
let output = std::process::Command::new(powershell_path())
.args([
"-NoProfile",
"-NonInteractive",
"-ExecutionPolicy",
"Bypass",
"-Command",
script,
])
.output()
.map_err(|err| format!("could not run PowerShell: {err}"))?;
if output.status.success() {
return Ok(());
}
let stderr = String::from_utf8_lossy(&output.stderr);
let text = if stderr.trim().is_empty() {
String::from_utf8_lossy(&output.stdout).trim().to_string()
} else {
stderr.trim().to_string()
};
Err(text)
}
async fn step(what: &'static str, script: String) -> RuleOutcome {
match tokio::task::spawn_blocking(move || run_powershell(&script)).await {
Ok(Ok(())) => RuleOutcome::Applied,
Ok(Err(text)) => RuleOutcome::Failed(classify(what, &text)),
Err(err) => RuleOutcome::Failed(format!("{what}: the task failed: {err}")),
}
}
/// The Windows implementation, through PowerShell.
#[derive(Debug, Default)]
pub struct WindowsHostRules;
impl WindowsHostRules {
/// Creates it. Nothing is run until [`BroadcastHostRules::apply`].
pub fn new() -> Self {
Self
}
}
impl BroadcastHostRules for WindowsHostRules {
fn name(&self) -> &str {
"windows-netsecurity"
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
let route = step("route", route_script(plan)).await;
let firewall = step("firewall", firewall_script(plan)).await;
BroadcastRulesReport {
plan: plan.clone(),
route,
firewall,
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
if let RuleOutcome::Failed(reason) = step("clear", clear_script(interface)).await {
tracing::warn!(
interface,
"cannot remove the broadcast host rules: {reason}"
);
}
})
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use std::net::Ipv4Addr;
fn plan() -> BroadcastRulesPlan {
BroadcastRulesPlan {
interface: "tsun0".into(),
source: Ipv4Addr::new(10, 13, 37, 142),
range: "10.13.37.0/24".parse().unwrap(),
}
}
#[test]
fn a_quote_in_a_value_cannot_escape_the_literal() {
assert_eq!(ps_literal("a'b"), "'a''b'");
let mut hostile = plan();
hostile.interface = "x'; Remove-Item C:\\ #".into();
let script = route_script(&hostile);
assert!(
script.contains("$alias = 'x''; Remove-Item C:\\ #'"),
"{script}"
);
assert!(firewall_script(&hostile).contains("'x''; Remove-Item C:\\ #'"));
}
#[test]
fn the_route_is_on_link_metric_one_in_the_active_store_and_replaces_its_own() {
let script = route_script(&plan());
assert!(script.contains("'255.255.255.255/32'"));
assert!(script.contains("-NextHop '0.0.0.0'"));
assert!(script.contains("-RouteMetric 1"));
assert!(script.contains("-PolicyStore ActiveStore"));
assert!(
script.find("Remove-NetRoute").unwrap() < script.find("New-NetRoute").unwrap(),
"a stale route goes before the new one:\n{script}"
);
}
#[test]
fn the_firewall_rule_is_udp_only_from_the_overlay_range_on_this_interface() {
let script = firewall_script(&plan());
assert!(script.contains("-Protocol UDP"));
assert!(script.contains("-Direction Inbound -Action Allow"));
assert!(script.contains("-InterfaceAlias $alias"));
assert!(script.contains("-RemoteAddress '10.13.37.0/24'"));
assert!(script.contains("$tag = 'tsunagi:tsun0'"));
assert!(!script.contains("-Protocol Any") && !script.contains("-Protocol TCP"));
assert!(
script.find("Remove-NetFirewallRule").unwrap()
< script.find("New-NetFirewallRule").unwrap(),
"{script}"
);
}
#[test]
fn clearing_touches_only_our_route_and_our_tagged_rule() {
let script = clear_script("tsun0");
assert!(script.contains("-InterfaceAlias 'tsun0'"));
assert!(script.contains("Get-NetFirewallRule -DisplayName 'tsunagi:tsun0'"));
assert!(script.contains("exit 0"));
}
#[test]
fn a_refusal_names_elevation_and_a_missing_module_names_the_module() {
assert!(classify("route", "Access is denied").contains("elevated"));
assert!(
classify(
"firewall",
"The term 'New-NetFirewallRule' is not recognized"
)
.contains("modules are not available")
);
assert_eq!(classify("route", "boom"), "route: boom");
assert!(classify("route", "").contains("without saying why"));
}
#[test]
fn powershell_is_found_by_absolute_path() {
let path = powershell_path();
assert!(path.ends_with("powershell.exe"));
assert!(path.to_string_lossy().contains("System32"), "{path:?}");
}
}
+230
View File
@@ -29,6 +29,7 @@ use tokio::task::JoinHandle;
use crate::identity::NetworkId;
use super::hostrules::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport};
use super::packet::IpHeader;
use super::router::{Route, RoutingTable};
use super::tun::{TunDevice, TunFactory, TunRequest};
@@ -152,6 +153,11 @@ pub struct Interface {
/// re-applied on every pass.
applied: std::sync::Mutex<Vec<Cidr>>,
task: std::sync::Mutex<Option<JoinHandle<()>>>,
sync_lock: Arc<tokio::sync::Mutex<()>>,
/// The host rules for this interface, when the factory manages the host.
host_rules: Option<Arc<dyn BroadcastHostRules>>,
/// What those rules last reported, `None` while none are installed.
rules_report: std::sync::Mutex<Option<BroadcastRulesReport>>,
}
impl Interface {
@@ -164,7 +170,15 @@ impl Interface {
carrier: Arc<dyn PacketCarrier>,
) -> Result<Self, OverlayError> {
let name = name.into();
let host_rules = factory.host_rules();
let device = factory.create(TunRequest::bare(name.clone(), mtu)).await?;
// The name the operating system actually gave the interface. On Linux
// and Windows that is the one we asked for; on macOS the kernel assigns
// the utun unit, so the real name differs and everything that
// configures the host for this interface afterwards — the broadcast
// route and firewall, the reconfigure on an address change — has to use
// the real one, not the requested `tsun…`.
let name = device.name().to_string();
let tally = Arc::new(Tally::default());
let task = {
@@ -228,15 +242,89 @@ impl Interface {
tally,
applied: std::sync::Mutex::new(Vec::new()),
task: std::sync::Mutex::new(Some(task)),
sync_lock: Arc::new(tokio::sync::Mutex::new(())),
host_rules,
rules_report: std::sync::Mutex::new(None),
})
}
/// Brings the broadcast route and firewall allowance in line with the
/// routing table: installed while some network has broadcast on and an
/// address, removed when none does.
///
/// Called after the addresses are synced, because the route's preferred
/// source has to be on the interface already. A plan that has not changed
/// is not re-applied, so this is cheap to call whenever the table moves.
/// It never fails: the outcome is kept for [`Self::broadcast_rules`].
pub async fn sync_broadcast_rules(&self) {
let Some(rules) = &self.host_rules else {
return;
};
let _guard = self.sync_lock.lock().await;
let desired =
self.routes
.broadcast_host_target()
.map(|(source, range)| BroadcastRulesPlan {
interface: self.name.clone(),
source,
range,
});
let current = self.current_rules_report();
match (desired, current) {
(None, None) => {}
(None, Some(_)) => {
rules.clear(&self.name).await;
self.store_rules_report(None);
tracing::info!(interface = %self.name, "broadcast host rules removed");
}
(Some(plan), Some(report)) if report.plan == plan => {}
(Some(plan), _) => {
let report = rules.apply(&plan).await;
if report.is_ok() {
tracing::info!(
interface = %self.name,
source = %plan.source,
"broadcast host rules applied"
);
} else {
tracing::warn!(
interface = %self.name,
"broadcast host rules are incomplete: {}",
report.summary()
);
}
self.store_rules_report(Some(report));
}
}
}
/// What the broadcast host rules achieved, `None` when none are installed
/// or this interface has no host to configure.
pub fn broadcast_rules(&self) -> Option<BroadcastRulesReport> {
self.current_rules_report()
}
fn current_rules_report(&self) -> Option<BroadcastRulesReport> {
match self.rules_report.lock() {
Ok(guard) => guard.clone(),
Err(poisoned) => poisoned.into_inner().clone(),
}
}
fn store_rules_report(&self, report: Option<BroadcastRulesReport>) {
match self.rules_report.lock() {
Ok(mut guard) => *guard = report,
Err(poisoned) => *poisoned.into_inner() = report,
}
}
/// Brings the addresses on the host in line with the routing table.
///
/// Called whenever a network agrees a different address for this agent.
/// The interface is never recreated for it: that would drop every tunnel
/// riding on it for the sake of one address.
pub async fn sync_addresses(&self) -> Result<(), OverlayError> {
let _guard = self.sync_lock.lock().await;
let wanted = self.wanted_addresses();
{
let applied = match self.applied.lock() {
@@ -286,6 +374,12 @@ impl Interface {
/// Removes the interface from the host.
pub async fn remove(&self) {
// The route goes with the interface; the firewall rule does not, so
// it is taken out explicitly while the interface is still known.
if let Some(rules) = &self.host_rules {
rules.clear(&self.name).await;
self.store_rules_report(None);
}
// The packet loop holds the device open, and it ends only when the
// device reports end of stream — which a real interface never does
// while it exists. Stop it directly before destroying the interface.
@@ -428,6 +522,9 @@ mod tests {
interface.remove().await;
}
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
use crate::overlay::broadcast::BroadcastPolicy;
use crate::overlay::hostrules::{MockHostRules, RuleOutcome};
use crate::overlay::provision::{ManagedTunFactory, MockProvisioner};
use crate::overlay::router::NetworkRoutes;
use crate::overlay::tun::{MemoryTun, MemoryTunFactory};
@@ -748,4 +845,137 @@ mod tests {
vec![Cidr::new(addr(9).into(), 24).unwrap()]
);
}
async fn managed_interface(
rules: &MockHostRules,
broadcast_enabled: bool,
) -> (Interface, Arc<RoutingTable>, NetworkId) {
let id = network("rules");
let routes = Arc::new(RoutingTable::new());
routes
.set_network(
id,
NetworkRoutes {
broadcast: BroadcastPolicy {
enabled: broadcast_enabled,
..Default::default()
},
range: Some("10.13.37.0/24".parse().unwrap()),
local: Some(addr(1)),
peers: vec![(addr(2), peer(2))],
},
)
.unwrap();
let factory = Arc::new(
ManagedTunFactory::new(Arc::new(MockProvisioner::default()))
.with_host_rules(Arc::new(rules.clone())),
);
let interface = Interface::start(
factory as Arc<dyn TunFactory>,
"tsunrules",
1280,
Arc::clone(&routes),
Arc::new(Recorder::default()) as Arc<dyn PacketCarrier>,
)
.await
.unwrap();
(interface, routes, id)
}
#[tokio::test]
async fn host_rules_follow_broadcast_policy_address_and_interface_lifetime() {
let rules = MockHostRules::new();
let (interface, routes, id) = managed_interface(&rules, true).await;
assert!(
interface.broadcast_rules().is_none(),
"nothing before a sync"
);
interface.sync_broadcast_rules().await;
let installed = rules.installed();
assert_eq!(installed["tsunrules"].source, addr(1));
assert_eq!(
installed["tsunrules"].range,
"10.13.37.0/24".parse().unwrap()
);
assert!(interface.broadcast_rules().unwrap().is_ok());
// Unchanged: not applied a second time.
interface.sync_broadcast_rules().await;
interface.sync_broadcast_rules().await;
assert_eq!(rules.calls(), ["apply:tsunrules"]);
// A new address replaces the plan, it does not stack a second one.
routes
.set_network(
id,
NetworkRoutes {
broadcast: BroadcastPolicy::default(),
range: Some("10.13.37.0/24".parse().unwrap()),
local: Some(addr(9)),
peers: vec![(addr(2), peer(2))],
},
)
.unwrap();
interface.sync_broadcast_rules().await;
assert_eq!(rules.installed()["tsunrules"].source, addr(9));
assert_eq!(rules.installed().len(), 1);
// Turning broadcast off takes the rules away.
routes.set_broadcast(
id,
BroadcastPolicy {
enabled: false,
..Default::default()
},
);
interface.sync_broadcast_rules().await;
assert!(rules.installed().is_empty());
assert!(interface.broadcast_rules().is_none());
// And on again puts them back; leaving the agent removes them once more.
routes.set_broadcast(id, BroadcastPolicy::default());
interface.sync_broadcast_rules().await;
assert_eq!(rules.installed().len(), 1);
interface.remove().await;
assert!(rules.installed().is_empty(), "shutdown clears the host");
assert!(interface.broadcast_rules().is_none());
}
#[tokio::test]
async fn host_rules_are_not_installed_while_broadcast_is_off() {
let rules = MockHostRules::new();
let (interface, _routes, _id) = managed_interface(&rules, false).await;
interface.sync_broadcast_rules().await;
assert!(rules.installed().is_empty());
assert!(rules.calls().is_empty(), "the host was not even asked");
interface.remove().await;
}
#[tokio::test]
async fn a_failed_firewall_step_is_reported_and_does_not_undo_the_route() {
let rules = MockHostRules::new();
rules.fail_firewall("iptables not found");
let (interface, _routes, _id) = managed_interface(&rules, true).await;
interface.sync_broadcast_rules().await;
let report = interface.broadcast_rules().unwrap();
assert!(report.route.is_applied());
assert_eq!(
report.firewall,
RuleOutcome::Failed("iptables not found".into())
);
assert!(!report.is_ok());
// Not retried on every sync: the plan is the same.
interface.sync_broadcast_rules().await;
assert_eq!(rules.calls(), ["apply:tsunrules"]);
interface.remove().await;
}
#[tokio::test]
async fn an_interface_without_a_host_has_no_host_rules() {
let (interface, _device, _routes, _carrier, _id) = interface(false).await;
interface.sync_broadcast_rules().await;
assert!(interface.broadcast_rules().is_none());
interface.remove().await;
}
}
+16
View File
@@ -11,6 +11,8 @@
//!
//! * [`provision`] creates the interface and configures it, and removes it
//! again. Platform mechanics behind one decision function.
//! * [`hostrules`] installs the route and firewall allowance that let a
//! game's LAN discovery reach the interface, while broadcast is on.
//! * [`tun`] is the packet interface itself, real or in memory.
//! * [`packet`] reads just enough of an IP header to route by it.
//! * [`config`] names interfaces and describes addresses.
@@ -38,6 +40,7 @@ pub enum OverlayError {
pub mod broadcast;
pub mod config;
pub mod hostrules;
pub mod interface;
pub mod packet;
pub mod provision;
@@ -45,6 +48,10 @@ pub mod router;
pub mod tun;
pub use config::{Cidr, DEFAULT_INTERFACE_PREFIX, MAX_INTERFACE_NAME_LEN, interface_name};
pub use hostrules::{
BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, MockHostRules, RuleOutcome,
UnsupportedHostRules,
};
pub use interface::{Counters, Interface, PacketCarrier, Rejected};
pub use packet::IpHeader;
pub use provision::{
@@ -55,8 +62,17 @@ pub use provision::{
pub use router::{NetworkRoutes, Route, RouteError, RoutingTable};
pub use tun::{MemoryTun, MemoryTunFactory, TunDevice, TunFactory, TunRequest, address_is_local};
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use hostrules::LinuxHostRules;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use hostrules::MacosHostRules;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use hostrules::WindowsHostRules;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use provision::NetlinkProvisioner;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use provision::UtunProvisioner;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use provision::WintunProvisioner;
@@ -11,6 +11,7 @@ use std::sync::Arc;
use crate::BoxFuture;
use crate::overlay::OverlayError;
use super::super::hostrules::BroadcastHostRules;
use super::super::tun::{TunDevice, TunFactory, TunRequest};
use super::{InterfacePlan, InterfaceProvisioner};
@@ -30,12 +31,23 @@ fn plan_for(request: &TunRequest) -> Result<InterfacePlan, OverlayError> {
#[derive(Debug)]
pub struct ManagedTunFactory {
provisioner: Arc<dyn InterfaceProvisioner>,
host_rules: Option<Arc<dyn BroadcastHostRules>>,
}
impl ManagedTunFactory {
/// Wraps a provisioner.
pub fn new(provisioner: Arc<dyn InterfaceProvisioner>) -> Self {
Self { provisioner }
Self {
provisioner,
host_rules: None,
}
}
/// Lets the agent install the broadcast route and firewall allowance for
/// the interface this factory manages.
pub fn with_host_rules(mut self, rules: Arc<dyn BroadcastHostRules>) -> Self {
self.host_rules = Some(rules);
self
}
/// The provisioner underneath.
@@ -54,6 +66,10 @@ impl TunFactory for ManagedTunFactory {
true
}
fn host_rules(&self) -> Option<Arc<dyn BroadcastHostRules>> {
self.host_rules.clone()
}
fn create<'a>(
&'a self,
request: TunRequest,
+160 -16
View File
@@ -39,7 +39,7 @@
//! capability is raised immediately before that call and lowered immediately
//! after, and the connection task lives and dies inside it.
use std::net::IpAddr;
use std::net::{IpAddr, Ipv4Addr};
use std::sync::{Arc, Mutex, mpsc};
use futures_util::TryStreamExt;
@@ -47,7 +47,8 @@ use futures_util::TryStreamExt;
// of step with the client that sends them.
use rtnetlink::packet_route::address::{AddressAttribute, AddressMessage};
use rtnetlink::packet_route::link::{InfoKind, LinkAttribute, LinkFlags, LinkInfo, LinkMessage};
use rtnetlink::{LinkMessageBuilder, LinkUnspec};
use rtnetlink::packet_route::route::RouteScope;
use rtnetlink::{LinkMessageBuilder, LinkUnspec, RouteMessageBuilder};
use crate::BoxFuture;
use crate::overlay::OverlayError;
@@ -67,6 +68,10 @@ enum Command {
Delete(String, Reply<()>),
/// Apply MTU, link state and addresses.
Configure(Box<Configure>, Reply<()>),
/// Route `255.255.255.255/32` through this interface from this address.
SetBroadcastRoute(String, Ipv4Addr, Reply<()>),
/// Remove that route from this interface.
DelBroadcastRoute(String, Reply<()>),
/// Stop the thread.
Stop,
}
@@ -133,13 +138,18 @@ impl NetlinkProvisioner {
&self,
make: impl FnOnce(Reply<T>) -> Command,
) -> Result<T, OverlayError> {
let (reply_tx, reply_rx) = mpsc::channel();
self.commands
.send(make(reply_tx))
.map_err(|_| OverlayError::Unavailable("the netlink thread has stopped".to_string()))?;
reply_rx.recv().map_err(|_| {
OverlayError::Unavailable("the netlink thread stopped mid-request".to_string())
})?
call_thread(&self.commands, make)
}
/// The broadcast host rules for this host.
///
/// The route is added by this provisioner's netlink thread, because it is
/// the only one that ever holds `CAP_NET_ADMIN`, and only for the duration
/// of a call. Keep the provisioner alive for as long as the rules are used.
pub fn host_rules(&self) -> crate::overlay::hostrules::LinuxHostRules {
crate::overlay::hostrules::LinuxHostRules::new(RouteHandle {
commands: self.commands.clone(),
})
}
fn is_ours(&self, name: &str) -> bool {
@@ -157,6 +167,44 @@ impl NetlinkProvisioner {
}
}
fn call_thread<T: Send + 'static>(
commands: &mpsc::Sender<Command>,
make: impl FnOnce(Reply<T>) -> Command,
) -> Result<T, OverlayError> {
let (reply_tx, reply_rx) = mpsc::channel();
commands
.send(make(reply_tx))
.map_err(|_| OverlayError::Unavailable("the netlink thread has stopped".to_string()))?;
reply_rx.recv().map_err(|_| {
OverlayError::Unavailable("the netlink thread stopped mid-request".to_string())
})?
}
/// Adds and removes the limited-broadcast route on the netlink thread.
#[derive(Debug, Clone)]
pub(crate) struct RouteHandle {
commands: mpsc::Sender<Command>,
}
impl RouteHandle {
/// Routes `255.255.255.255/32` through `interface`, preferring `source`.
/// Replaces a route this agent added earlier, so it can be repeated.
pub(crate) fn set(&self, interface: &str, source: Ipv4Addr) -> Result<(), OverlayError> {
let interface = interface.to_string();
call_thread(&self.commands, |reply| {
Command::SetBroadcastRoute(interface, source, reply)
})
}
/// Removes the route. Succeeds when it, or the interface, is gone.
pub(crate) fn clear(&self, interface: &str) -> Result<(), OverlayError> {
let interface = interface.to_string();
call_thread(&self.commands, |reply| {
Command::DelBroadcastRoute(interface, reply)
})
}
}
impl Drop for NetlinkProvisioner {
fn drop(&mut self) {
let _ = self.commands.send(Command::Stop);
@@ -273,7 +321,10 @@ fn netlink_thread(requests: mpsc::Receiver<Command>) {
Command::Observe(_, reply) => {
let _ = reply.send(Err(OverlayError::Unavailable(message)));
}
Command::Delete(_, reply) | Command::Configure(_, reply) => {
Command::Delete(_, reply)
| Command::Configure(_, reply)
| Command::SetBroadcastRoute(_, _, reply)
| Command::DelBroadcastRoute(_, reply) => {
let _ = reply.send(Err(OverlayError::Unavailable(message)));
}
Command::Stop => return,
@@ -306,6 +357,22 @@ fn netlink_thread(requests: mpsc::Receiver<Command>) {
});
let _ = reply.send(result);
}
Command::SetBroadcastRoute(name, source, reply) => {
let result = NetAdmin::acquire().and_then(|guard| {
let result = runtime.block_on(set_broadcast_route(&name, source));
drop(guard);
result
});
let _ = reply.send(result);
}
Command::DelBroadcastRoute(name, reply) => {
let result = NetAdmin::acquire().and_then(|guard| {
let result = runtime.block_on(del_broadcast_route(&name));
drop(guard);
result
});
let _ = reply.send(result);
}
}
}
}
@@ -455,6 +522,80 @@ async fn delete_link(name: &str) -> Result<(), OverlayError> {
.await
}
/// The message that names the limited-broadcast route on an interface.
///
/// On-link scope is what the kernel requires of a route with no gateway, and
/// what `ip route add 255.255.255.255/32 dev <if>` produces. Deleting needs the
/// same scope or the kernel finds no match.
fn broadcast_route_message(
index: u32,
source: Option<Ipv4Addr>,
) -> rtnetlink::packet_route::route::RouteMessage {
let mut builder = RouteMessageBuilder::<Ipv4Addr>::new()
.destination_prefix(Ipv4Addr::BROADCAST, 32)
.output_interface(index)
.scope(RouteScope::Link);
if let Some(source) = source {
builder = builder.pref_source(source);
}
builder.build()
}
async fn link_index(handle: &rtnetlink::Handle, name: &str) -> Option<u32> {
let mut links = handle.link().get().match_name(name.to_string()).execute();
links
.try_next()
.await
.ok()
.flatten()
.map(|message| message.header.index)
}
async fn set_broadcast_route(name: &str, source: Ipv4Addr) -> Result<(), OverlayError> {
with_netlink(|handle| async move {
let index = link_index(&handle, name).await.ok_or_else(|| {
OverlayError::Unavailable(format!(
"interface `{name}` disappeared before its broadcast route could be added"
))
})?;
handle
.route()
.add(broadcast_route_message(index, Some(source)))
.replace()
.execute()
.await
.map_err(|err| {
OverlayError::Unavailable(format!(
"cannot route 255.255.255.255 through `{name}` from {source}: {err}"
))
})
})
.await
}
async fn del_broadcast_route(name: &str) -> Result<(), OverlayError> {
with_netlink(|handle| async move {
// An interface that is gone took its routes with it.
let Some(index) = link_index(&handle, name).await else {
return Ok(());
};
match handle
.route()
.del(broadcast_route_message(index, None))
.execute()
.await
{
Ok(()) => Ok(()),
// "No such process": there was no such route, which is the goal.
Err(err) if err.to_string().contains("No such process") => Ok(()),
Err(err) => Err(OverlayError::Unavailable(format!(
"cannot remove the 255.255.255.255 route from `{name}`: {err}"
))),
}
})
.await
}
async fn configure_link(configure: &Configure) -> Result<(), OverlayError> {
let name = configure.name.as_str();
with_netlink(|handle| async move {
@@ -520,16 +661,19 @@ async fn configure_link(configure: &Configure) -> Result<(), OverlayError> {
}
for cidr in &configure.add {
handle
let res = handle
.address()
.add(index, cidr.addr, cidr.prefix_len)
.execute()
.await
.map_err(|err| {
OverlayError::Unavailable(format!(
.await;
if let Err(err) = res {
let err_str = err.to_string();
if !err_str.contains("File exists") && !err_str.contains("os error 17") {
return Err(OverlayError::Unavailable(format!(
"cannot add {cidr} to interface `{name}`: {err}"
))
})?;
)));
}
}
}
Ok(())
@@ -0,0 +1,531 @@
//! Managing the overlay interface on macOS, through utun and `ifconfig`/`route`.
//!
//! The shape is the same as the Linux and Windows provisioners: observe what
//! is there, [`plan_changes`](super::plan_changes) to decide what to do, apply
//! the difference. Only the mechanisms differ.
//!
//! # Creating the interface
//!
//! The interface is a `utun` device, created through the `tun` dependency's
//! safe wrapper — [`open_tun`](super::super::tun) — over the in-kernel
//! `com.apple.net.utun_control` control socket. No kernel extension and no
//! Network Extension entitlement is involved; what it needs is root. The
//! device is not made persistent, so closing the last file descriptor removes
//! the interface — it goes away when the agent does, however the agent ends.
//!
//! # The kernel names the interface, not us
//!
//! Unlike Linux and Windows, a utun interface cannot be given an arbitrary
//! name: it is always `utunN`, and the unit number is assigned by the kernel.
//! So the name derived from the network id (`tsun…`) is only ever a *request*;
//! the real name is read back off the device after it is opened, and
//! everything downstream — the host rules, the resolver publication — uses the
//! real one. The agent adopts it as soon as the device is created (see
//! [`Interface`](crate::overlay::Interface)).
//!
//! # Configuring it
//!
//! Addresses, MTU and routes are applied with `ifconfig` and `route`. On Linux
//! the same work is done in process over a netlink socket; here there is no
//! equivalent this crate may call, because the in-process route is raw `ioctl`
//! and `PF_ROUTE` and that is `unsafe`. The tools are invoked by absolute path
//! from `/sbin`, so nothing on `PATH` can stand in for them, and every value
//! handed to them is one this agent derived — an interface name chosen by the
//! kernel, an address it allocated — never anything a peer said.
//!
//! # Observing it
//!
//! The interface table is read through `netdev`, a safe wrapper over
//! `getifaddrs`. An interface is matched by its real `utunN` name, which only
//! matters on reconfiguration: the first create always makes a fresh device.
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use std::sync::{Arc, Mutex};
use crate::BoxFuture;
use crate::overlay::OverlayError;
use super::super::config::Cidr;
use super::super::tun::{TunDevice, TunRequest, open_tun};
use super::privilege::{Privilege, probe_net_admin};
use super::{
InterfacePlan, InterfaceProvisioner, InterfaceState, LinkKind, Provisioned, plan_changes,
};
/// Manages the overlay interface with utun and `ifconfig`/`route`.
#[derive(Debug)]
pub struct UtunProvisioner {
/// Interfaces this process created, by their real `utunN` name, so a
/// reconfiguration adjusts them rather than trying to create them again.
ours: Mutex<Vec<String>>,
/// Devices kept alive for as long as the interface should exist. Dropping
/// one closes the last descriptor, which removes the utin interface.
held: Mutex<Vec<(String, Arc<dyn TunDevice>)>>,
}
impl UtunProvisioner {
/// Creates the provisioner, after checking the platform supports one.
///
/// Whether the process is actually root is not checked here — reading the
/// effective uid needs a libc call this crate forbids — so a missing
/// privilege surfaces at utun creation with a precise message, the same
/// way the Linux and Windows paths treat the open itself as the honest
/// answer.
pub fn new() -> Result<Self, OverlayError> {
match probe_net_admin() {
Privilege::Available => {}
Privilege::Missing(reason) => return Err(OverlayError::Unavailable(reason)),
Privilege::Unsupported => {
return Err(OverlayError::Unavailable(
"interface management is not compiled in".to_string(),
));
}
}
Ok(Self {
ours: Mutex::new(Vec::new()),
held: Mutex::new(Vec::new()),
})
}
fn is_ours(&self, name: &str) -> bool {
lock(&self.ours).iter().any(|owned| owned == name)
}
/// Opens the utun device, which is what creates it. The kernel assigns the
/// name, so the returned device reports a `utunN` that is not `plan.name`.
fn create_device(&self, plan: &InterfacePlan) -> Result<Arc<dyn TunDevice>, OverlayError> {
let request = TunRequest::bare(plan.name.clone(), plan.mtu);
open_tun(&request)
}
}
impl InterfaceProvisioner for UtunProvisioner {
fn name(&self) -> &str {
"utun"
}
fn reconcile<'a>(
&'a self,
plan: &'a InterfacePlan,
) -> BoxFuture<'a, Result<Provisioned, OverlayError>> {
Box::pin(async move {
// A reconfiguration: the plan carries the real `utunN` name the
// agent adopted after the first create, so the interface is there
// and only the addresses and MTU need bringing into line.
if self.is_ours(&plan.name) {
let current = observe(&plan.name).await;
let changes = plan_changes(&current, plan, true)?;
configure(&plan.name, &changes).await?;
return Ok(Provisioned {
changes,
device: None,
});
}
// The first create. The kernel assigns the unit, so the device is
// opened first and its real name used for everything after.
let device = self.create_device(plan)?;
let real = device.name().to_string();
// A fresh interface starts with nothing on it, so the whole plan is
// applied. `plan_changes` against an absent interface is exactly
// that decision; its `create_link` flag is already satisfied by the
// open above, so `configure` ignores it and only applies the rest.
let changes = plan_changes(&InterfaceState::absent(), plan, false)?;
// A failure here leaves an interface that exists but cannot carry
// traffic. Drop the device so the utun closes, and report, rather
// than leave a trap behind.
if let Err(err) = configure(&real, &changes).await {
drop(device);
return Err(err);
}
lock(&self.ours).push(real.clone());
lock(&self.held).push((real, Arc::clone(&device)));
Ok(Provisioned {
changes,
device: Some(device),
})
})
}
fn remove<'a>(&'a self, name: &'a str) -> BoxFuture<'a, Result<(), OverlayError>> {
Box::pin(async move {
// Dropping the device closes the last descriptor, which removes the
// utun interface and the routes that were bound to it. There is
// nothing to flush: a utun never outlives the process that holds it.
lock(&self.held).retain(|(held, _)| held != name);
lock(&self.ours).retain(|owned| owned != name);
Ok(())
})
}
}
fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
match mutex.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
/// `fe80::/10` and `169.254.0.0/16`, which macOS assigns on its own.
fn is_link_local(addr: IpAddr) -> bool {
match addr {
IpAddr::V4(addr) => addr.is_link_local(),
IpAddr::V6(addr) => (addr.segments()[0] & 0xffc0) == 0xfe80,
}
}
/// What the interface named `name` currently looks like.
///
/// Never fails: a name nothing is using reads as [`InterfaceState::absent`],
/// which is what the plan then acts on.
async fn observe(name: &str) -> InterfaceState {
let name = name.to_string();
let found = tokio::task::spawn_blocking(move || {
netdev::get_interfaces()
.into_iter()
.find(|iface| iface.name == name)
})
.await
.ok()
.flatten();
let Some(iface) = found else {
return InterfaceState::absent();
};
let mut addresses: Vec<Cidr> = Vec::new();
for net in &iface.ipv4 {
if let Ok(cidr) = Cidr::new(IpAddr::V4(net.addr()), net.prefix_len())
&& !is_link_local(cidr.addr)
{
addresses.push(cidr);
}
}
for net in &iface.ipv6 {
if let Ok(cidr) = Cidr::new(IpAddr::V6(net.addr()), net.prefix_len())
&& !is_link_local(cidr.addr)
{
addresses.push(cidr);
}
}
addresses.sort();
InterfaceState {
// Only ever reached for an interface this agent created (see
// `reconcile`), so it is ours and a TUN by construction.
kind: LinkKind::Tun,
attached: iface.is_up(),
up: iface.is_up(),
mtu: iface.mtu.unwrap_or(0),
addresses,
}
}
/// Applies MTU, link state, addresses and the on-link routes they imply.
async fn configure(name: &str, changes: &super::Changes) -> Result<(), OverlayError> {
if let Some(mtu) = changes.set_mtu {
set_mtu(name, mtu).await?;
}
for cidr in &changes.remove {
// Best effort: an address that is already gone is the outcome wanted.
if let Err(err) = address_del(name, *cidr).await {
tracing::debug!(interface = %name, %cidr, %err, "could not remove address");
}
if let Err(err) = route_del(name, *cidr).await {
tracing::debug!(interface = %name, %cidr, %err, "could not remove route");
}
}
for cidr in &changes.add {
address_add(name, *cidr).await?;
// The subnet route makes the whole overlay range reach the interface;
// a point-to-point utun does not install one from the address alone. A
// host address (/32 or /128) needs none. Best effort: a route that is
// already there is the outcome wanted.
if let Err(err) = route_add(name, *cidr).await {
tracing::debug!(interface = %name, %cidr, %err, "could not add route");
}
}
if changes.bring_up {
// Best effort: a utun with a descriptor open is up already, and a
// failure to nudge it is not a reason to fail the whole reconcile.
if let Err(err) = set_up(name).await {
tracing::debug!(interface = %name, %err, "could not enable the interface");
}
}
Ok(())
}
/// The network address of a CIDR, with the host bits cleared.
fn network_of(cidr: Cidr) -> IpAddr {
match cidr.addr {
IpAddr::V4(addr) => {
let bits = u32::from(addr);
let mask = if cidr.prefix_len == 0 {
0
} else {
u32::MAX << (32 - u32::from(cidr.prefix_len))
};
IpAddr::V4(Ipv4Addr::from(bits & mask))
}
IpAddr::V6(addr) => {
let bits = u128::from(addr);
let mask = if cidr.prefix_len == 0 {
0
} else {
u128::MAX << (128 - u32::from(cidr.prefix_len))
};
IpAddr::V6(Ipv6Addr::from(bits & mask))
}
}
}
/// Whether a CIDR is a single host address, which needs no subnet route.
fn is_host_route(cidr: Cidr) -> bool {
match cidr.addr {
IpAddr::V4(_) => cidr.prefix_len >= 32,
IpAddr::V6(_) => cidr.prefix_len >= 128,
}
}
/// The `ifconfig` arguments that add an address to a utun interface.
///
/// A utun is point-to-point, so the IPv4 form names the address as its own
/// peer; `alias` lets several coexist on the one interface.
fn address_add_args(name: &str, cidr: Cidr) -> Vec<String> {
match cidr.addr {
IpAddr::V4(addr) => vec![
name.into(),
"inet".into(),
format!("{addr}/{}", cidr.prefix_len),
addr.to_string(),
"alias".into(),
],
IpAddr::V6(addr) => vec![
name.into(),
"inet6".into(),
format!("{addr}/{}", cidr.prefix_len),
"alias".into(),
],
}
}
/// The `ifconfig` arguments that remove an address from a utun interface.
fn address_del_args(name: &str, cidr: Cidr) -> Vec<String> {
let family = match cidr.addr {
IpAddr::V4(_) => "inet",
IpAddr::V6(_) => "inet6",
};
vec![
name.into(),
family.into(),
cidr.addr.to_string(),
"-alias".into(),
]
}
/// The `route` arguments that point a CIDR's subnet at the interface.
fn route_add_args(name: &str, cidr: Cidr) -> Vec<String> {
let family = match cidr.addr {
IpAddr::V4(_) => "-inet",
IpAddr::V6(_) => "-inet6",
};
vec![
"-q".into(),
"-n".into(),
"add".into(),
family.into(),
"-net".into(),
format!("{}/{}", network_of(cidr), cidr.prefix_len),
"-interface".into(),
name.into(),
]
}
/// The `route` arguments that remove that subnet route.
fn route_del_args(name: &str, cidr: Cidr) -> Vec<String> {
let family = match cidr.addr {
IpAddr::V4(_) => "-inet",
IpAddr::V6(_) => "-inet6",
};
vec![
"-q".into(),
"-n".into(),
"delete".into(),
family.into(),
"-net".into(),
format!("{}/{}", network_of(cidr), cidr.prefix_len),
"-interface".into(),
name.into(),
]
}
async fn address_add(name: &str, cidr: Cidr) -> Result<(), OverlayError> {
ifconfig(address_add_args(name, cidr)).await
}
async fn address_del(name: &str, cidr: Cidr) -> Result<(), OverlayError> {
ifconfig(address_del_args(name, cidr)).await
}
async fn route_add(name: &str, cidr: Cidr) -> Result<(), OverlayError> {
if is_host_route(cidr) {
return Ok(());
}
route(route_add_args(name, cidr)).await
}
async fn route_del(name: &str, cidr: Cidr) -> Result<(), OverlayError> {
if is_host_route(cidr) {
return Ok(());
}
route(route_del_args(name, cidr)).await
}
async fn set_mtu(name: &str, mtu: u32) -> Result<(), OverlayError> {
ifconfig(vec![name.into(), "mtu".into(), mtu.to_string()]).await
}
async fn set_up(name: &str) -> Result<(), OverlayError> {
ifconfig(vec![name.into(), "up".into()]).await
}
/// Runs `/sbin/ifconfig`, turning a non-zero exit into an error with its text.
async fn ifconfig(args: Vec<String>) -> Result<(), OverlayError> {
run("/sbin/ifconfig", args).await
}
/// Runs `/sbin/route`, turning a non-zero exit into an error with its text.
async fn route(args: Vec<String>) -> Result<(), OverlayError> {
run("/sbin/route", args).await
}
/// Runs a system tool by absolute path, so nothing on `PATH` can stand in for
/// it, and turns a non-zero exit into an error carrying what it said.
async fn run(program: &'static str, args: Vec<String>) -> Result<(), OverlayError> {
let display = format!("{program} {}", args.join(" "));
let output = tokio::task::spawn_blocking(move || {
std::process::Command::new(program).args(&args).output()
})
.await
.map_err(|err| OverlayError::Other(format!("could not run {display}: {err}")))?
.map_err(|err| OverlayError::Unavailable(format!("could not run {display}: {err}")))?;
if output.status.success() {
return Ok(());
}
let message = {
let stderr = String::from_utf8_lossy(&output.stderr);
let text = if stderr.trim().is_empty() {
String::from_utf8_lossy(&output.stdout).trim().to_string()
} else {
stderr.trim().to_string()
};
if text.is_empty() {
format!("exit code {}", output.status)
} else {
text
}
};
Err(OverlayError::Unavailable(format!(
"{display} failed: {message}"
)))
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
fn v4(last: u8, prefix: u8) -> Cidr {
Cidr::new(IpAddr::V4(Ipv4Addr::new(10, 13, 37, last)), prefix).unwrap()
}
fn v6(last: u16, prefix: u8) -> Cidr {
Cidr::new(
IpAddr::V6(Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0, last)),
prefix,
)
.unwrap()
}
#[test]
fn a_v4_address_is_added_point_to_point_with_an_alias() {
let args = address_add_args("utun5", v4(69, 24));
assert_eq!(
args,
vec!["utun5", "inet", "10.13.37.69/24", "10.13.37.69", "alias"]
);
}
#[test]
fn a_v6_address_is_added_with_a_prefix_and_an_alias() {
let args = address_add_args("utun5", v6(1, 128));
assert_eq!(args, vec!["utun5", "inet6", "fd00::1/128", "alias"]);
}
#[test]
fn deleting_an_address_names_the_family_and_removes_the_alias() {
let args = address_del_args("utun5", v4(69, 24));
assert_eq!(args, vec!["utun5", "inet", "10.13.37.69", "-alias"]);
let args = address_del_args("utun5", v6(1, 128));
assert_eq!(args, vec!["utun5", "inet6", "fd00::1", "-alias"]);
}
#[test]
fn a_subnet_route_points_the_masked_network_at_the_interface() {
let args = route_add_args("utun5", v4(69, 24));
assert_eq!(
args,
vec![
"-q",
"-n",
"add",
"-inet",
"-net",
"10.13.37.0/24",
"-interface",
"utun5"
]
);
}
#[test]
fn a_host_address_installs_no_subnet_route() {
assert!(is_host_route(v4(69, 32)));
assert!(is_host_route(v6(1, 128)));
assert!(!is_host_route(v4(69, 24)));
assert!(!is_host_route(v6(1, 64)));
}
#[test]
fn the_network_address_clears_the_host_bits() {
assert_eq!(
network_of(v4(69, 24)),
IpAddr::V4(Ipv4Addr::new(10, 13, 37, 0))
);
assert_eq!(
network_of(Cidr::new(IpAddr::V4(Ipv4Addr::new(10, 13, 37, 69)), 16).unwrap()),
IpAddr::V4(Ipv4Addr::new(10, 13, 0, 0))
);
assert_eq!(
network_of(v6(0x1234, 64)),
IpAddr::V6(Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0, 0))
);
}
#[test]
fn link_local_addresses_are_not_ours_to_manage() {
assert!(is_link_local(IpAddr::V6(Ipv6Addr::new(
0xfe80, 0, 0, 0, 0, 0, 0, 1
))));
assert!(is_link_local(IpAddr::V4(Ipv4Addr::new(169, 254, 1, 1))));
assert!(!is_link_local(IpAddr::V4(Ipv4Addr::new(10, 13, 37, 1))));
assert!(!is_link_local(IpAddr::V6(Ipv6Addr::new(
0xfd00, 0, 0, 0, 0, 0, 0, 1
))));
}
}
+10 -1
View File
@@ -20,11 +20,13 @@
//! pure and tested on every platform, and only the execution is behind
//! [`InterfaceProvisioner`].
//!
//! Four implementations:
//! Five implementations:
//!
//! * `NetlinkProvisioner` on Linux, which needs `CAP_NET_ADMIN`.
//! * `WintunProvisioner` on Windows, a Wintun adapter configured with `netsh`,
//! which needs an elevated process.
//! * `UtunProvisioner` on macOS, a utun interface configured with `ifconfig`
//! and `route`, which needs root.
//! * [`MockProvisioner`], an in-memory host used by the tests.
//! * [`UnsupportedProvisioner`] elsewhere, which fails with an explanation
//! and a pointer at the manual route rather than pretending to work.
@@ -52,12 +54,19 @@ pub use mock::{MockHost, MockProvisioner};
mod linux;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use linux::NetlinkProvisioner;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub(crate) use linux::RouteHandle;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
mod windows;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use windows::WintunProvisioner;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
mod macos;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use macos::UtunProvisioner;
mod privilege;
pub use privilege::{Privilege, probe_net_admin};
@@ -63,15 +63,20 @@ pub use linux_impl::{NetAdmin, probe_net_admin};
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use windows_impl::probe_net_admin;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use macos_impl::probe_net_admin;
#[cfg(not(any(
all(feature = "tun-device", target_os = "linux"),
all(feature = "tun-device", target_os = "windows")
all(feature = "tun-device", target_os = "windows"),
all(feature = "tun-device", target_os = "macos")
)))]
pub use other_impl::probe_net_admin;
#[cfg(not(any(
all(feature = "tun-device", target_os = "linux"),
all(feature = "tun-device", target_os = "windows")
all(feature = "tun-device", target_os = "windows"),
all(feature = "tun-device", target_os = "macos")
)))]
mod other_impl {
use super::Privilege;
@@ -101,6 +106,25 @@ mod windows_impl {
}
}
#[cfg(all(feature = "tun-device", target_os = "macos"))]
mod macos_impl {
use super::Privilege;
/// Whether this process can configure interfaces.
///
/// macOS has no capability to hold and lower the way Linux does: creating
/// a utun interface simply needs the process to be root. Reading the
/// effective uid means a libc call this crate forbids, so the probe is
/// deliberately optimistic — it reports that the platform can manage
/// interfaces — and the real check is left to the utun open, which fails
/// with a precise "run with sudo" message when the process is not root.
/// This matches how the Linux path treats the open itself as the honest
/// answer.
pub fn probe_net_admin() -> Privilege {
Privilege::Available
}
}
#[cfg(all(feature = "tun-device", target_os = "linux"))]
mod linux_impl {
use caps::{CapSet, Capability};
@@ -262,7 +262,12 @@ async fn configure(plan: &InterfacePlan, changes: &super::Changes) -> Result<(),
set_mtu(&plan.name, mtu).await?;
}
for cidr in &changes.add {
address_add(&plan.name, *cidr).await?;
if let Err(err) = address_add(&plan.name, *cidr).await {
let err_str = err.to_string();
if !err_str.contains("already exists") && !err_str.contains("0x80071392") {
return Err(err);
}
}
}
if changes.bring_up {
// Best effort: a Wintun adapter with a session is up already, and a
+76
View File
@@ -230,6 +230,23 @@ impl RoutingTable {
addresses
}
/// The one network whose address the limited broadcast route is bound to.
///
/// The host has a single `255.255.255.255` route per interface, and one
/// interface carries every network, so only one overlay address can be its
/// source. The choice is the lowest [`NetworkId`] among networks that have
/// broadcast enabled, a range and a local address: deterministic, so two
/// runs with the same networks pick the same one.
pub fn broadcast_host_target(&self) -> Option<(Ipv4Addr, Ipv4Range)> {
let networks = self.read();
networks
.iter()
.filter(|(_, routes)| routes.broadcast.enabled)
.filter_map(|(network, routes)| Some((*network, routes.local?, routes.range?)))
.min_by_key(|(network, _, _)| *network)
.map(|(_, local, range)| (local, range))
}
/// Whether a range would collide with one another network already uses.
///
/// Asked before proposing a range rather than after: with one interface
@@ -393,6 +410,65 @@ mod tests {
assert_eq!(table.route(addr(2)).map(|route| route.network), Some(first));
}
#[test]
fn the_host_broadcast_target_is_the_lowest_enabled_network_with_an_address() {
let table = RoutingTable::new();
assert_eq!(table.broadcast_host_target(), None);
let (low, high) = {
let (a, b) = (network("first"), network("second"));
if a < b { (a, b) } else { (b, a) }
};
let mk = |enabled: bool, base: [u8; 4], local: Option<Ipv4Addr>| NetworkRoutes {
broadcast: BroadcastPolicy {
enabled,
..Default::default()
},
range: Some(
format!("{}.{}.{}.0/24", base[0], base[1], base[2])
.parse()
.unwrap(),
),
local,
peers: Vec::new(),
};
let a = Ipv4Addr::new(10, 1, 0, 1);
let b = Ipv4Addr::new(10, 2, 0, 1);
// The lower id has broadcast off, so the higher one is chosen.
table
.set_network(low, mk(false, [10, 1, 0, 0], Some(a)))
.unwrap();
table
.set_network(high, mk(true, [10, 2, 0, 0], Some(b)))
.unwrap();
assert_eq!(
table.broadcast_host_target(),
Some((b, "10.2.0.0/24".parse().unwrap()))
);
// Both enabled: the lowest id wins, whatever the insertion order.
table
.set_network(low, mk(true, [10, 1, 0, 0], Some(a)))
.unwrap();
assert_eq!(
table.broadcast_host_target(),
Some((a, "10.1.0.0/24".parse().unwrap()))
);
// No local address yet: it cannot be the route's source.
table
.set_network(low, mk(true, [10, 1, 0, 0], None))
.unwrap();
assert_eq!(
table.broadcast_host_target(),
Some((b, "10.2.0.0/24".parse().unwrap()))
);
table.remove_network(high);
assert_eq!(table.broadcast_host_target(), None);
}
#[test]
fn a_range_can_be_asked_about_before_it_is_proposed() {
// The point of asking first: an agent that claims an address it
+37 -2
View File
@@ -109,6 +109,14 @@ pub trait TunFactory: Send + Sync + std::fmt::Debug + 'static {
Box::pin(async move { Ok(()) })
}
/// The host rules for this factory's interface, when it manages the host.
///
/// `None` for an in-memory device or an interface somebody else prepared:
/// there is nothing of ours to route or to open in a firewall.
fn host_rules(&self) -> Option<Arc<dyn super::hostrules::BroadcastHostRules>> {
None
}
/// Removes an interface this factory created.
///
/// Runs on the teardown path, so it reports rather than fails: there is
@@ -409,7 +417,15 @@ mod system {
/// across this call, and such a guard must not span an `await` because
/// Linux capabilities are per thread.
pub(crate) fn open_tun(request: &TunRequest) -> Result<Arc<dyn TunDevice>, OverlayError> {
// macOS mutates neither field below (it neither names the interface nor
// sets a platform config), so `mut` is unused only there.
#[cfg_attr(target_os = "macos", allow(unused_mut))]
let mut config = tun::Configuration::default();
// macOS names its tunnels `utunN` and assigns the unit number itself.
// A name derived from the network id (`tsun…`) is not a legal utun
// name, so there the kernel is left to choose and the real name is read
// back below. Linux and Windows take the requested name verbatim.
#[cfg(not(target_os = "macos"))]
config.tun_name(&request.name);
#[cfg(target_os = "linux")]
config.platform_config(|platform| {
@@ -427,9 +443,20 @@ mod system {
let device = tun::create_as_async(&config).map_err(|err| open_error(&request.name, err))?;
// The interface name the operating system actually gave us: the
// requested one everywhere except macOS, where the kernel chose the
// utun unit and so the real name has to be read back off the device.
#[cfg(target_os = "macos")]
let name = {
use tun::AbstractDevice;
device.tun_name().unwrap_or_else(|_| request.name.clone())
};
#[cfg(not(target_os = "macos"))]
let name = request.name.clone();
let (reader, writer) = tokio::io::split(device);
Ok(Arc::new(SystemTun {
name: request.name.clone(),
name,
mtu: request.mtu,
reader: Mutex::new(reader),
writer: Mutex::new(writer),
@@ -460,6 +487,9 @@ mod system {
} else if cfg!(target_os = "linux") {
"Start `tsunagi up` as root or grant CAP_NET_ADMIN with \
`sudo setcap cap_net_admin,cap_net_bind_service+p /path/to/tsunagi`."
} else if cfg!(target_os = "macos") {
"Creating a utun interface needs root on macOS; start `tsunagi up` \
with `sudo`, or install it as a root LaunchDaemon."
} else {
"Start `tsunagi up` with the privileges required to create a TUN interface."
};
@@ -488,7 +518,12 @@ mod system {
run as Administrator with the DLL beside the executable, or run with `--no-tun` \
to keep the tunnels off the operating system."
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
#[cfg(target_os = "macos")]
{
"Creating a utun interface needs root; start the agent with `sudo` or as a root \
LaunchDaemon, or run with `--no-tun` to keep the tunnels off the operating system."
}
#[cfg(not(any(target_os = "linux", target_os = "windows", target_os = "macos")))]
{
"Run with `--no-tun` to keep the tunnels off the operating system."
}
+62
View File
@@ -0,0 +1,62 @@
# Running tsunagi on Linux
This archive holds a **static** `tsunagi` binary (musl), so it runs on any
distribution — Arch, Alpine, NixOS, old glibc systems — with no shared-library
dependencies. Alongside it are a systemd unit and an installer.
Creating the TUN interface needs `CAP_NET_ADMIN`, and configuring
systemd-resolved is decided by polkit (by user id, not capability). There are
three ways to satisfy that; pick one.
## Quick install — systemd service as root
```sh
sudo ./install.sh
sudo systemctl enable --now tsunagi
```
The service runs the agent as root, so the interface, low ports and the
resolver all just work (root is not subject to polkit). Control it **as root**,
so the client resolves the same state directory the daemon uses:
```sh
sudo tsunagi status
sudo tsunagi join <network-name> <tsn1…secret>
```
## As your own user — one capability, no root at runtime
```sh
sudo install -m 0755 tsunagi /usr/local/bin/tsunagi
sudo setcap cap_net_admin,cap_net_bind_service+p /usr/local/bin/tsunagi
tsunagi up
```
Nothing is left behind and nothing runs as root. The capability is lost on every
rebuild or copy of the binary, so re-run `setcap` after replacing it. For the
local DNS resolver, run `tsunagi dns` once: it prints the exact polkit rule that
lets your user configure systemd-resolved, ready to paste.
## Without touching the OS
```sh
tsunagi up --no-tun
```
Tunnels form and handshake between agents, but no interface, address or route is
created, so traffic never reaches the operating system. Needs no privileges.
## What the agent installs
Everything is tagged to its own interface, computed from local state, and
reversible — removed when a network leaves, broadcast is turned off, or the
agent stops:
- a TUN interface configured over netlink (addresses, MTU, up);
- while broadcast is on, a `255.255.255.255` route through the interface and an
inbound-UDP firewall allowance from the overlay range (`iptables`, tagged);
- the systemd-resolved setting that sends the overlay zones' questions to the
local resolver.
Nothing taken from a remote peer ever becomes a path, a command argument or an
OS setting.
Vendored Executable
+57
View File
@@ -0,0 +1,57 @@
#!/usr/bin/env sh
# Installs the tsunagi binary and, where systemd is present, a root service.
# Run it as root (it writes under /usr/local and /etc):
#
# sudo ./install.sh
#
# PREFIX overrides the install prefix (default /usr/local).
set -eu
PREFIX="${PREFIX:-/usr/local}"
here="$(cd "$(dirname "$0")" && pwd)"
if [ "$(id -u)" -ne 0 ]; then
echo "This installer writes to ${PREFIX}/bin and /etc; re-run it as root (sudo)." >&2
exit 1
fi
echo "Installing tsunagi to ${PREFIX}/bin"
install -d "${PREFIX}/bin"
install -m 0755 "${here}/tsunagi" "${PREFIX}/bin/tsunagi"
if command -v systemctl >/dev/null 2>&1; then
echo "Installing the systemd service (runs the agent as root)"
install -m 0644 "${here}/tsunagi.service" /etc/systemd/system/tsunagi.service
systemctl daemon-reload
cat <<EOF
Done. Start the agent with:
sudo systemctl enable --now tsunagi
Control it as root, so the client shares the agent's state directory:
sudo tsunagi status
sudo tsunagi join <network-name> <tsn1…secret>
EOF
else
cat <<EOF
Installed the binary; no systemd found, so no service was set up. Either:
* run it directly as root:
sudo tsunagi up
* or grant the capability and run it as your own user (no root at runtime):
sudo setcap cap_net_admin,cap_net_bind_service+p ${PREFIX}/bin/tsunagi
tsunagi up
For the local DNS resolver as a non-root user, run \`tsunagi dns\` once; it
prints the polkit rule that lets your user configure systemd-resolved.
* or skip the interface entirely with \`tsunagi up --no-tun\`: tunnels still
form, they just do not reach the operating system.
EOF
fi
+26
View File
@@ -0,0 +1,26 @@
[Unit]
Description=tsunagi mesh agent
Documentation=https://github.com/Ultradesu/tsunagi
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/local/bin/tsunagi up
Restart=on-failure
RestartSec=5
# This unit runs the agent as root, which covers everything it needs: creating
# the TUN interface (CAP_NET_ADMIN), binding low ports, and configuring
# systemd-resolved — root is not subject to polkit, so no polkit rule is needed.
#
# To run as a non-root user instead, uncomment the three lines below, set a
# User=, and install the polkit rule that `tsunagi dns` prints (resolved decides
# by user id, not capability, so the rule is what lets a non-root agent set the
# resolver):
#User=tsunagi
#AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
#CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
[Install]
WantedBy=multi-user.target
+82
View File
@@ -0,0 +1,82 @@
# Running tsunagi on macOS
The agent needs **root** on macOS: creating a `utun` interface and writing
`/etc/resolver` are both root-only, and macOS has no per-capability grant like
Linux. There are two ways to run it.
## Quick start — `sudo`
```sh
sudo tsunagi up
```
In another terminal, the control commands talk to it over the system control
socket and do **not** need `sudo`:
```sh
tsunagi status
tsunagi join <network-name> <tsn1…secret>
```
The agent uses fixed system paths, so the commands find it regardless of whose
shell started `sudo`:
| what | path |
|---|---|
| mandatory state | `/var/db/tsunagi` |
| disposable cache | `/var/db/tsunagi/cache` |
| control socket | `/var/run/tsunagi/agent.sock` |
## Native deployment — root LaunchDaemon
For an always-on agent that starts at boot, install the bundled
[`cy.hexor.tsunagi.plist`](cy.hexor.tsunagi.plist).
```sh
# Put the binary where the plist expects it (edit the plist for another path).
sudo install -m 0755 target/release/tsunagi /usr/local/bin/tsunagi
# Install the daemon. launchd refuses a daemon plist that is not owned by
# root:wheel or that is group/world-writable.
sudo install -m 0644 -o root -g wheel \
dist/macos/cy.hexor.tsunagi.plist \
/Library/LaunchDaemons/cy.hexor.tsunagi.plist
# Load and start it.
sudo launchctl bootstrap system /Library/LaunchDaemons/cy.hexor.tsunagi.plist
sudo launchctl enable system/cy.hexor.tsunagi
```
Manage it afterwards with:
```sh
sudo launchctl kickstart -k system/cy.hexor.tsunagi # restart
sudo launchctl bootout system/cy.hexor.tsunagi # stop and unload
tail -f /var/log/tsunagi.log # logs
```
The control commands (`status`, `join`, `network`, `dns`) work the same against
the daemon as against a `sudo tsunagi up`, over the same socket.
## What the agent does to the system
Everything it installs is tagged to its own interface, computed from local
state, and reversible — it is removed when a network leaves, broadcast is turned
off, or the agent stops:
- a **`utun` interface**, configured with `ifconfig` (addresses, MTU, up) and a
subnet `route` for the overlay range. The kernel assigns the `utunN` name; the
agent adopts whatever it is given. A `utun` is torn down with the process, so
a crash never leaves one behind.
- `/etc/resolver/<network>` files pointing the system resolver at the overlay
DNS server for each network's zone, with a `port` so the server stays off 53.
Each file carries a marker line; the agent only ever replaces or removes files
it wrote, never one you created by hand.
- while broadcast is on, a `255.255.255.255` **route** through the overlay
interface. The matching inbound-UDP firewall allowance is **not** implemented
yet (it needs a `pf` anchor); `tsunagi status` reports that half as incomplete
and names what to allow by hand if your host firewall drops discovery replies.
Nothing taken from a remote peer ever becomes a path, a command argument or an
OS setting: the interface name comes from the kernel, addresses and ranges from
signed state the agent already verified.
+51
View File
@@ -0,0 +1,51 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<!--
LaunchDaemon for the tsunagi agent.
A daemon, so it runs as root (creating a utun interface and writing
/etc/resolver both need it) and comes up at boot before anyone logs in.
Install it at /Library/LaunchDaemons/cy.hexor.tsunagi.plist, owned by
root:wheel with mode 0644 — launchd refuses a daemon plist that is
group- or world-writable. See README.md in this directory.
Edit the binary path if tsunagi is not at /usr/local/bin/tsunagi.
-->
<plist version="1.0">
<dict>
<key>Label</key>
<string>cy.hexor.tsunagi</string>
<!-- `up` starts the agent and keeps it in the foreground, which is what a
launchd daemon expects: launchd owns the lifecycle, so the process
must not daemonise itself. -->
<key>ProgramArguments</key>
<array>
<string>/usr/local/bin/tsunagi</string>
<string>up</string>
</array>
<!-- Start at boot and bring it back if it ever exits. -->
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
<!-- Back off rather than spin if it fails to start repeatedly. -->
<key>ThrottleInterval</key>
<integer>10</integer>
<!-- Logs. launchd creates these as root; rotate them with newsyslog if you
keep the daemon around. -->
<key>StandardOutPath</key>
<string>/var/log/tsunagi.log</string>
<key>StandardErrorPath</key>
<string>/var/log/tsunagi.log</string>
<!-- The agent uses the fixed macOS system paths by default:
state /var/db/tsunagi
socket /var/run/tsunagi/agent.sock
so nothing here has to point at a user's home directory. -->
</dict>
</plist>
+46
View File
@@ -0,0 +1,46 @@
# Running tsunagi on Windows
This archive holds `tsunagi.exe`. The overlay interface is a Wintun adapter, so
the agent needs two things to carry real traffic:
1. **`wintun.dll` beside the executable.** Download it from
<https://www.wintun.net>, take the DLL for your architecture (`amd64`), and
put it in the same folder as `tsunagi.exe`.
2. **An elevated process.** Creating the adapter requires Administrator rights.
Open PowerShell or Command Prompt with **Run as administrator** and start the
agent there:
```powershell
.\tsunagi.exe up
```
Run the commands that control it (`join`, `status`, `dns`, `network`) as the
same user with the same elevation. The agent serves its control interface on a
named pipe under `%ProgramData%\tsunagi`, so an elevated client finds it
regardless of which account started the agent.
## Without touching the OS
```powershell
.\tsunagi.exe up --no-tun
```
Tunnels form and handshake between agents, but no adapter, address or route is
created, so traffic never reaches the operating system. Needs no Wintun and no
elevation.
## What the agent installs
Everything is tagged to its own adapter, computed from local state, and
reversible — removed when a network leaves, broadcast is turned off, or the
agent stops:
- a Wintun adapter configured with `netsh` (addresses, MTU);
- while broadcast is on, a `255.255.255.255` route through the adapter and an
inbound-UDP firewall allowance from the overlay range;
- the Name Resolution Policy Table entry that sends the overlay zones' questions
to the local resolver.
Nothing taken from a remote peer ever becomes a path, a command argument or an
OS setting.
+38 -4
View File
@@ -61,13 +61,47 @@ routers cannot inspect another pair's encrypted IP payload.
## Host behavior and verification
The game must send discovery through the Tsunagi interface. Traffic bound to a
physical adapter never reaches this TUN, and this feature does not capture it.
A game's discovery packet only reaches Tsunagi if the operating system sends it
through the Tsunagi interface, and its reply only reaches the game if the host
firewall admits it. While a network has broadcast on, the agent installs
exactly two things for the interface it owns, and removes them when broadcast
is turned off, the network is left or stopped, or the agent exits:
- **Route** `255.255.255.255/32` through the overlay interface, with the
overlay address as its source (Linux: netlink, `ip route add 255.255.255.255/32
dev tsun0 src <overlay-ip>`; Windows: `New-NetRoute`, metric 1, active store).
Limited broadcasts from every program then go to the overlay and not to the
physical LAN while this is installed; this is the price of the feature.
- **Firewall allowance** for inbound **UDP only**, from the overlay range only,
on that interface only (Linux: an `iptables -I INPUT` rule tagged
`tsunagi:<interface>`; Windows: a Defender rule of that name). Replies to a
broadcast come from another address than the request went to, so a stateful
firewall never matches them to it. TCP services such as SSH stay unexposed.
Both are tagged, idempotent and reversible, and are never derived from a remote
announcement. A rule left by a crashed run is replaced on the next apply. If
`iptables` is missing (for example a host with only `nft`) or the process is not
privileged, the route is still installed and `status` prints a
`broadcast rules` line saying which half is missing and why. To undo it all,
turn broadcast off for the network: `tsunagi network broadcast <network> off`.
**One interface, several networks.** The host has one `255.255.255.255` route
per interface, so only one overlay address can be its source. The lowest network
id among broadcast-enabled networks with an address is used; `status` says when
that is not the network being shown. Directed broadcasts (`10.13.37.255`) follow
the ordinary connected route and are not affected.
**What it cannot fix.** A game that binds its socket to a physical address or
device, or sends to the physical subnet's broadcast address, still bypasses the
TUN. The Windows implementation could not be run while it was written; its
scripts are unit-tested only.
The suite tests real agents, authenticated iroh links and WireGuard encryption
with memory TUNs: limited/directed UDP discovery, multihop fanout, single-copy
delivery, opt-out, unicast replies, malformed input, source validation and
network isolation. It does not launch CS 1.6 or Warcraft III or establish that
every game/host chooses the virtual adapter automatically.
network isolation. Host rules are tested through a mock host and a scripted
`iptables`; the default suite changes no OS setting. It does not launch CS 1.6 or
Warcraft III.
## Future exported LANs