Compare commits

..
64 Commits
Author SHA1 Message Date
Aleksandr BogomiakovandClaude c9c653023d Release: build a static ARMv7 (armhf) Linux agent
Adds armv7-unknown-linux-musleabihf to both release pipelines: the archive,
a .deb for armhf and an Arch package for armv7h. Agent and CLI only, like the
other non-x86_64 Linux builds.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-03 02:25:43 +01:00
Aleksandr BogomiakovandClaude 9966b50407 Rename the CLI package to tsunagi and its executable to tsng
The command line package is now `tsunagi` and builds the `tsng` executable,
which the packages, the unit, the installers and the workflows install and
run. The core library package becomes `tsunagi-core`, keeping the library name
`tsunagi`, so no code changes `use tsunagi::...`. Directories follow
(crates/tsunagi-core, crates/tsunagi). Every command shown to a person is now
`tsng ...`.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-03 02:14:43 +01:00
Aleksandr BogomiakovandClaude bb38562b0a Exit nodes: block IPv6 while one is in use; exit icon on the network tile
IPv6 has no way through an exit node, so the same policy rules get an
unreachable default for IPv6 instead of leaking it. The tray shows a green
icon in the tile corner while an exit node is in use, with a one-line
confirmation to stop it; confirmations are shortened.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-03 02:09:14 +01:00
Aleksandr BogomiakovandClaude b2431d7eb1 Exit nodes: offer one per network, send all traffic through a member
A member can offer itself as an exit node per network (off by default); it is
announced over the control channel (ctrl/4) once its iptables masquerade and
forward rules are in place, with a warning when kernel forwarding is off. A
device picks one member as its exit node via policy routing, with the agent's
own uid exempt from the tunnel; if the exit node disappears the rules stay
and traffic is dropped, loudly reported. CLI: `network exit-node`,
`exit-node`; tray: tile checkbox, warnings and a per-device icon with
confirmation. Linux only.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-03 01:14:51 +01:00
Aleksandr BogomiakovandClaude c62d29fa14 Arch package: start the service and let the GUI reach the agent
The control socket is now opened to the tsunagi group (unit), the Arch
package enables and starts the service on install, adds the installing user
to the group, restarts a running agent on upgrade, and depends on the
libraries the tray loads at run time. The tray looks for the agent's socket
again on every poll instead of once at start, and a missing indicator
library is reported instead of panicking. Repository URLs unified.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-02 23:29:51 +01:00
Aleksandr BogomiakovandClaude a6d9edabee Point the project repository at github.com/house-of-vanity/tsunagi
Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-02 23:06:48 +01:00
Aleksandr BogomiakovandClaude 2f7f9980c7 Tray: open the devices window wider and let its table scroll sideways
Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-02 23:05:08 +01:00
Aleksandr BogomiakovandClaude 2e178a6f9d Tray: About window, fixed-width device table with totals; WTFPL
Release / tsunagi-linux-aarch64 (push) Successful in 2m19s
Release / tsunagi-linux-x86_64 (push) Successful in 3m56s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Add an About window (client and agent details, protocol versions), pin the
device table's column widths and add a total row, show the version in the
header and drop the refresh button. Relicense under WTFPL.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-02 20:16:01 +01:00
Aleksandr BogomiakovandClaude f4dcdec662 Tray: separate tray and window processes; device list without self
Release / tsunagi-linux-aarch64 (push) Successful in 2m44s
Release / tsunagi-linux-x86_64 (push) Successful in 4m37s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
The tray process holds only the icon and starts the same binary with
--window on Open; the window ends when closed, which works on Wayland where
a window cannot be hidden. The devices table now names devices as
hostname (id5), leaves out this device and counts the others.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-02 19:23:06 +01:00
Aleksandr BogomiakovandClaude 69310108f4 Remember the last hostname each peer announced
Persist announced hostnames per network (state schema 5) and report them for
members, so a device that is away is still named in `status` and the tray.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-02 19:23:06 +01:00
Aleksandr BogomiakovandClaude e7dff34ec2 Status: ask the agent for its state directory and its own CAP_NET_ADMIN
The CLI reached a packaged service's socket but read state from its own
default directory, so `network` saw nothing next to a running network, and
`status` reported the CLI binary's capabilities instead of the agent's.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-02 18:31:01 +01:00
Aleksandr BogomiakovandClaude a55ea883a1 Tray: initialise GTK and run its main loop on Linux
Release / tsunagi-linux-aarch64 (push) Successful in 3m14s
Release / tsunagi-linux-x86_64 (push) Successful in 5m21s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
tray-icon needs gtk::init() and a running GTK loop on Linux, which
eframe/winit never provide, so the tray panicked at startup with "GTK has
not been initialized". Build the tray on a dedicated GTK thread instead.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-02 17:54:38 +01:00
UltradesuandClaude Opus 4.8 3e066f991f CI: install libxdo-dev for the Linux tray link
Release / tsunagi-linux-aarch64 (push) Successful in 3m23s
Release / tsunagi-linux-x86_64 (push) Successful in 6m58s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
The x86_64 Linux tray failed at link with "unable to find library -lxdo":
tray-icon pulls libxdo, whose .so was not present. Add libxdo-dev to the GUI
build's apt install, alongside libgtk-3-dev and libayatana-appindicator3-dev.
Everything else (aarch64 agent, macOS, Windows) already built.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 16:51:53 +01:00
UltradesuandClaude Opus 4.8 3752419075 CI: build the Linux tray natively with GTK; drop the aarch64 GUI
Release / tsunagi-linux-aarch64 (push) Successful in 4m14s
Release / tsunagi-linux-x86_64 (push) Failing after 4m29s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
The aarch64 GUI cross-build failed: tray-icon links GTK and libappindicator
through pkg-config, which zig cross-compilation cannot satisfy without a target
sysroot. Rather than set one up, the Linux tray is now built natively on the
x86_64 runner with libgtk-3-dev and libayatana-appindicator3-dev installed, so
pkg-config just works. The agent is still static musl for both arches.

The aarch64 Linux GUI is dropped for now: that archive stays agent-only and no
aarch64 tsunagi-gui package is produced. x86_64 gets the tray in its archive
and the tsunagi-gui .deb/.pkg.tar.zst. macOS and Windows build the tray
natively as before. Guarded on matrix.gnu_target (set only on x86_64 Linux).

Note: the native build links the runner's glibc, so the Linux GUI binary needs a
reasonably recent glibc; fine for current desktops. A lower baseline would need
a GTK sysroot or an older runner image.

YAML validated; the Linux GUI build is exercised only by a real CI run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 16:38:58 +01:00
UltradesuandClaude Opus 4.8 f63343a6a0 CI: ship the tray GUI — in the archives and as tsunagi-gui deb/zst
Release / tsunagi-linux-aarch64 (push) Failing after 2m40s
Release / tsunagi-linux-x86_64 (push) Failing after 2m40s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Adds the tray application to the release:

- Every target's archive now also contains the tsunagi-tray binary. On Linux it
  is built against an old glibc baseline (via cargo-zigbuild) rather than static
  musl, because a GUI must dlopen libGL/X11/Wayland at runtime and a static musl
  binary cannot; the agent stays static musl. macOS and Windows build the tray
  natively.
- New GUI package variants for Linux: `tsunagi-gui` .deb and .pkg.tar.zst, built
  by the existing scripts with an extra tray argument. They install the agent
  (service, polkit, user — same as the headless packages) plus the tray binary,
  a desktop launcher (dist/linux/tsunagi-tray.desktop) and an icon
  (dist/linux/tsunagi.svg), so the GUI appears in the applications menu. The GUI
  package conflicts/replaces/provides the headless tsunagi so the two do not
  coexist.
- The tray finds the system service with no configuration: the library's default
  control-socket path now prefers /run/tsunagi/agent.sock when present (both the
  tray and CLI use this), falling back to the per-user path.

No control-protocol change. Builds, fmt and clippy clean on macOS; the Linux GUI
cross-build and the packages are exercised only by a real CI run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 16:26:45 +01:00
UltradesuandClaude Opus 4.8 d13bb00623 Tray: one device table (online + offline), FQDN copy, green-fade copy feedback
- The devices window now shows a single device table instead of a separate
  "connected" box and an easily-missed "offline" box below it. Every endpoint
  known from a live tunnel or the signed membership appears, online first;
  offline rows show name and address with dashes for the live columns. The
  table fills the window and scrolls, so nothing hides off the bottom.
- Clicking an online peer's name copies hostname.network (the FQDN) so it
  resolves; a name without a hostname copies its short id.
- Copy feedback no longer appends "Copied" text (which reflowed the table for a
  second). The field just flashes green and fades back over a second, with no
  change to its text or width.

Builds and is clippy-clean (-D warnings) on macOS; not run in this headless
session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 16:09:33 +01:00
UltradesuandClaude Opus 4.8 2a0a8abd60 Tray: split rate/plot columns, add unit toggle to the devices window
The connected-peers rate and sparkline shared one grid cell, so the plot still
moved right as the rate text grew (e.g. "1.0 kB/s"). They are now two separate
grid columns — the plot column starts at the same x on every row, since grid
columns align.

The devices window also gets its own pkts/bytes toggle, bound to the same unit
setting as the main window so the two stay in step.

Builds and is clippy-clean (-D warnings) on macOS; not run in this headless
session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 16:04:39 +01:00
UltradesuandClaude Opus 4.8 db01c58a6f Tray: Phosphor icons, click-to-copy fields with feedback, steady plot column
- Bundle egui-phosphor (pinned to egui 0.31) and register its glyphs, so the
  icon buttons render real icons instead of tofu: refresh, shuffle (generate
  secret), pencil/check/x (rename host), copy (Copy secret).
- Copying is now the field itself, not a separate button: clicking the
  shortened device id, the network name, the network id, a range or an address
  copies the full value and the field flashes green "Copied" for a second
  (tracked in egui temp memory, so it clears itself). The standalone copy
  buttons are gone.
- In the devices window the connected-peers "rate / plot" column put the rate
  and the inline sparkline in a shared row, so the plot shifted as the rate
  text changed width ("0 B/s" vs "1.1 kB/s"). The rate now sits in a
  fixed-width, truncating cell, so the plot stays put.

Builds and is clippy-clean (-D warnings) on macOS; the UI has not been run in
this headless session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 16:00:23 +01:00
UltradesuandClaude Opus 4.8 de56258eb1 Tray: copy a network secret via ipc; use text buttons, not missing glyphs
Two fixes.

Secret copy: a new local control request returns a configured network's secret
in its tsn1 text form, so the tray's "Copy secret" button can put it on the
clipboard. The agent reads it from local state (same source as the CLI's
`network secret show`); it crosses only the owner/group socket and is wrapped in
a RedactedText type whose Debug prints "<redacted>", so it can never land in a
log. In the GUI the worker fetches it and hands it to the UI thread (which owns
the clipboard) through a one-shot slot kept out of the status snapshot, so it is
copied once and never stored in the cloned snapshot. Control protocol bumped
16 -> 17.

Icons: the button glyphs (⧉ ⟳ ✎ ✔ ✕) are not in egui's bundled fonts and showed
as tofu squares, so they are now plain text: copy / refresh / random / edit /
save / cancel. No icon font to embed.

Checks: fmt, clippy --workspace --all-targets -D warnings, and the tsunagi lib,
tsunagi-cli and wg-quic local_control tests pass on macOS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 15:48:32 +01:00
UltradesuandClaude Opus 4.8 8f5ce21ce9 Tray: devices window, bytes/packets toggle, copy & edit affordances
Replaces the in-tile collapsing peer list with a "Show devices" button that
opens a separate, normal-sized window per network, and fills in the rest of the
feedback.

Main window:
- A global show: pkts/bytes toggle; all rates and sparklines follow it.
- Sparklines now reserve their space always (fixed height, "no data yet"
  placeholder) so the layout no longer jumps when data arrives, and each one
  overlays its peak (top-left) and current value (bottom-right) with units.
- Device id has a copy button; the hostname is editable in place (✎ → edit →
  ✔/Enter sends SetHostname); network name, range and address copy on click.
- The Leave button is a neutral red. Per-tile "Show devices" opens the detail.

Devices window (a separate egui viewport per network):
- Network summary: id (copy), active/broadcast, range, this device's address,
  candidates, relay fwd/via/recv, dial/handshake failures.
- A larger network traffic sparkline.
- Connected peers in a table: hostname when known (else short key), protocol,
  address, handshake age formatted (s/m/h/d), tx/rx packets, tx/rx bytes, and
  current rate with an inline sparkline. Names and addresses copy on click.
- Offline members listed below with address and failed-dial count.

Agent: a SetHostname command (over the existing ipc) backs the in-place rename.
Traffic sampler now tracks both packet and byte rates; the toggle selects which.

Deferred: a "copy secret" button on the tile — the secret is deliberately not
exported to the GUI (secrets never leave the agent), so it needs a small local
secret-export ipc, which is the next step.

Builds and is clippy-clean (-D warnings) on macOS; the UI has not been run in
this headless session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 15:37:08 +01:00
UltradesuandClaude Opus 4.8 0a344c1b4f Report per-peer byte counters, not just packets
The overlay peer report carried packet counts only, so the GUI could show
packets/s but no byte throughput. Adds tx_bytes/rx_bytes to OverlayPeerReport
and fills them from each data plane:

- the wg and wg-quic plugins already counted plaintext bytes per peer; the CLI
  now maps them into the report;
- the generic tunnel (tcp-tls) gains tx_bytes/rx_bytes on its peer state and
  report, incremented by the plaintext length at the same points it counts
  packets.

The local control protocol version is bumped (TSN 15 -> 16) since the report
shape changed; a client and agent on different builds already detect and report
the mismatch.

Checks: fmt, clippy --workspace --all-targets -D warnings, and the tsunagi lib,
tsunagi-cli and wg-quic local_control tests pass on macOS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 15:30:37 +01:00
UltradesuandClaude Opus 4.8 2cda56e1f8 Log inbound handshake rejections at debug, not warn
Creating and leaving networks produced a stream of WARN lines like "Handshake
rejected: peer requested an unknown or inactive network network=None". This is
not a leave bug: leaving deactivates the network and publishes a release, but
remote peers that still consider this device a member keep dialing, and each
dial for a network no longer served is correctly rejected by the responder
(lookup returns None, so the network stays unidentified — network=None by
design, since a pre-handshake claim is unverified).

Such a rejection is routine and remotely triggerable — a left network a peer
has not caught up on, a stranger, a port scan, an old client — so logging it at
WARN let any peer spam the log. It is now debug; enable debug to see rejections
when diagnosing. The rejection behaviour itself is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 15:27:49 +01:00
UltradesuandClaude Opus 4.8 d1de61ba29 Tray UI: start hidden, fixed window, grid layout, traffic + peers
Reworks the tray window per feedback and fills it in.

- Starts hidden in the tray (no window on launch); the window opens from the
  menu. On macOS the app now uses the Accessory activation policy, so it lives
  in the menu bar with no Dock icon (via a winit event-loop hook, pinned to
  eframe's winit). Fixed, non-resizable 360x600 window; the layout targets it.
- Tidied layout: top bar (title + refresh), a bottom panel with the join form
  and the action banner, and a scrolling list of network tiles between them.
  Labels and values sit on two-column grids so they align; the join name and
  secret inputs are the same fixed width.
- A ⟳ button beside the secret generates a fresh random secret
  (NetworkSecret::generate), for creating a network.
- Each network tile shows id, range, this device's address, peers
  (connected/known), the active and broadcast switches, a Leave button, and a
  collapsing "peers (N)" section with a formatted per-peer table (short key,
  address, handshake age, tx/rx packets, rate).
- Traffic: a new sampler turns the agent's cumulative packet counters into a
  packets-per-second rate per network and per peer, from the deltas between
  snapshots (the snapshot now carries a generation and timestamp so each
  refresh is sampled once). The network tile shows the current rate and a small
  hand-drawn sparkline of recent rates (no extra dependency).

Note: rates are packets/s — the report has no byte counters yet; bytes would
need an agent change. Builds and is clippy-clean (-D warnings) on macOS; the UI
has not been run in this headless session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 15:08:29 +01:00
UltradesuandClaude Opus 4.8 19509dc833 Add the tsunagi-tray GUI (egui) — first increment
A native, lightweight tray application to manage a running agent — no web
engine. It is a thin client over the agent's local control socket, reusing
tsunagi::ipc and the report types directly; it owns its own runtime, logger and
event loop, like the CLI.

This first increment:
- tray icon with an Open/Quit menu (tray-icon); menu events are read on a
  background thread so Open works even while the window is hidden;
- a small window that hides to the tray on close and reopens from the menu;
- a background tokio worker that polls status every 2s and runs commands off the
  UI thread, writing a snapshot the UI renders from and repainting egui;
- UI: device header, a per-network card (name, id, range, this device's
  address, peers connected/known, a Leave button, and the per-network active
  and broadcast switches), a join form (name + secret), and a transient
  action/error banner; an "agent unreachable" state that names the socket.

Socket resolution: TSUNAGI_CONTROL_SOCKET if set, else the platform default
(/var/run/tsunagi/agent.sock on macOS) — the same socket the group-access agent
serves.

Scope/notes: macOS first (Linux and Windows follow). Not yet done — macOS
accessory activation policy (so there is no dock icon), light/dark tray icons
from assets (the icon is drawn in code for now), and packaging/autostart. Builds
and is clippy-clean (-D warnings) on macOS; the UI itself has not been run in
this headless session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 14:48:52 +01:00
UltradesuandClaude Opus 4.8 de8e48bee7 Let the agent grant the control socket to a group (0660)
For a GUI (or a CLI) running as a different user to reach the agent, the control
socket needs to be group-accessible. This is set by the agent when it binds the
socket — correct from creation, not patched afterwards.

- ipc::ControlSocketAccess { Private, Group(gid) }. Private is the existing
  owner-only 0600. Group sets the socket's group to the gid and mode 0660, and
  creates the socket's parent directory 0755 so the group can traverse to it; if
  the group cannot be set it falls back to owner-only rather than leaving it
  wrong. The chown uses std::os::unix::fs::chown — no unsafe, no new lib
  dependency.
- ControlSocket::bind takes the access policy (Windows accepts it and keeps its
  pipe default for now; a pipe security descriptor is a later follow-up).
- CLI: `tsunagi up --control-group <name>` (env TSUNAGI_CONTROL_GROUP) resolves
  the group to its gid via nix (safe wrapper, unix-only dep) and passes
  Group(gid); without it the socket stays Private. A name that does not resolve
  is reported, and the socket stays owner-only.
- control_socket() now serves the system path (default_control_socket_path) on
  macOS too, not just Windows: /var/run/tsunagi/agent.sock, a runtime directory
  the GUI can reach, rather than the owner-only state directory. A custom
  --state-dir still keeps its own per-user socket.

So the packaged/daemon agent runs e.g. `tsunagi up --control-group tsunagi` and a
user in that group talks to it; a plain per-user agent is unchanged (0600).

Checks: fmt, clippy --workspace --all-targets -D warnings, tsunagi lib tests and
the wg-quic local_control tests pass on macOS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 14:41:58 +01:00
UltradesuandClaude Opus 4.8 8c812e6874 Bump the GitHub workflow actions off deprecated Node 20
Release / tsunagi-linux-aarch64 (push) Successful in 5m25s
Release / tsunagi-linux-x86_64 (push) Failing after 3h12m37s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
GitHub now forces actions/checkout@v4, actions/upload-artifact@v4 and
actions/download-artifact@v4 (all Node 20) onto Node 24 with a deprecation
warning. Move them to the current Node 24 majors — checkout@v5,
upload-artifact@v7, download-artifact@v8 — which keep the same inputs and the
per-artifact subdirectory layout (merge-multiple defaults to false), so the
publish step is unchanged.

Only the .github workflow is touched. The .gitea workflow stays on
checkout@v4 and the @v3 artifact actions, because Gitea does not support the v4+
artifact backend and may not have Node 24 on its runner image; it also does not
emit these deprecation warnings. mlugg/setup-zig has no Node 24 release yet
(latest v2.2.1 is still Node 20), so that one warning remains until upstream
updates — nothing to change on our side.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 14:01:33 +01:00
UltradesuandClaude Opus 4.8 63178574ce Package as a dedicated tsunagi user; add Arch packages
Release / tsunagi-linux-aarch64 (push) Successful in 2m39s
Release / tsunagi-linux-x86_64 (push) Successful in 2m32s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Reworks the Linux packaging so the service runs as an unprivileged, dedicated
"tsunagi" user instead of root — the point of a native package is to set things
up properly for that OS's users — and adds Arch Linux packages beside the deb.
Both package families are built from the already-compiled static binary; nothing
is recompiled.

Service model (shared systemd unit, dist/linux/tsunagi.service):
- User=tsunagi with AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE, so
  it creates the TUN and binds low ports without root.
- StateDirectory/CacheDirectory/RuntimeDirectory give it /var/lib/tsunagi,
  /var/cache/tsunagi and /run/tsunagi, and the agent is pointed at them (and the
  socket at /run/tsunagi/agent.sock) through TSUNAGI_* env in the unit.
- The polkit rule now grants the resolve1 actions to the "tsunagi" group; the
  service user is in it, and admins can add themselves.

CLI: when no state dir is chosen, the control commands prefer the system
service's socket at /run/tsunagi/agent.sock when it exists, so `sudo tsunagi
status`/`join` reach the packaged daemon with no flags. A per-user agent (its
socket absent from /run) still uses its own.

Packaging:
- deb (build-deb.sh): postinst now creates the tsunagi system user+group; unit
  installed to /lib/systemd/system, binary to /usr/bin.
- Arch (new dist/arch/build-pkg.sh): a dependency-free .pkg.tar.zst assembled
  with bsdtar + zstd (no makepkg, no Arch host) — .PKGINFO, a .INSTALL scriptlet
  that creates the user and reloads systemd, and a gzipped .MTREE; unit to
  /usr/lib/systemd/system.
- The polkit rule moved to dist/linux (shared by deb, Arch and the tarball), and
  the tarball install.sh now creates the user and installs the unit + rule.
- Both release workflows build the deb and the Arch package on the two Linux
  matrix entries (deb_arch/pkg_arch), install libarchive-tools+zstd for it, and
  attach both to the release (Gitea API loop / gh).

Not exercised in this session: the .deb and .pkg.tar.zst build and install only
on Linux (no dpkg-deb/bsdtar+pacman locally); CLI build, clippy, fmt and tests
pass, and all shell/YAML is syntax-checked.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 13:46:27 +01:00
UltradesuandClaude Opus 4.8 a3856eed3b Build a dependency-free .deb and attach it to the release
Release / tsunagi-linux-aarch64 (push) Successful in 2m21s
Release / tsunagi-linux-x86_64 (push) Successful in 2m29s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Adds a Debian package to the release for the x86_64 and aarch64 Linux builds, so
`apt install ./tsunagi_*.deb` is a convenient path on the Ubuntu hosts. Because
the binary is static musl, the package declares no Depends — it is built with
plain dpkg-deb on the runner (no cargo-deb or nfpm), from dist/deb/build-deb.sh.

The package installs:
- the static binary at /usr/bin/tsunagi;
- a systemd service at /lib/systemd/system/tsunagi.service that runs the agent
  as root (ExecStart rewritten to /usr/bin; the source unit targets
  /usr/local/bin for the tarball installer);
- a polkit rule at /usr/share/polkit-1/rules.d granting the resolve1 actions to
  a "tsunagi" group, for running the agent as a non-root user instead of via the
  root service. postinst creates that system group and runs daemon-reload; prerm
  disables the service on removal.

Both release workflows gain a "Build the Debian package" step on the two Linux
matrix entries (deb/deb_arch), include the .deb in the uploaded run artifacts,
and attach it to the release (Gitea via the API loop, GitHub via gh). macOS and
Windows are unchanged. Service runs as root by default; the polkit rule/group is
the documented opt-in for the non-root path — say if you'd rather the service
itself run as the tsunagi user.

YAML and shell syntax validated; the .deb build itself runs only on the Linux
runner (no dpkg-deb locally) and has not been exercised in this session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 13:34:33 +01:00
UltradesuandClaude Opus 4.8 eac54f8c79 Revert the GitHub workflow guard to server_url
Release / tsunagi-linux-aarch64 (push) Canceled after 15s
Release / tsunagi-linux-x86_64 (push) Canceled after 0s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Misdiagnosed the earlier report: Gitea was correctly running its own
.gitea/workflows copy, not this .github one, so the server_url guard was fine.
The switch to a GITHUB_RELEASE repo variable then made the workflow skip on
GitHub itself, since that variable is not set there.

Put both job guards back to `github.server_url == 'https://github.com'`, which
runs on GitHub with no manual setup and skips on Gitea (its .gitea copy is the
one that runs). No repository variable is needed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:53:26 +01:00
UltradesuandClaude Opus 4.8 a8dfd7017b Gate the GitHub release workflow on a repo variable, not server_url
Release / tsunagi-linux-aarch64 (push) Canceled after 35s
Release / tsunagi-linux-x86_64 (push) Canceled after 0s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
The github.server_url guard did not keep this workflow from running on Gitea:
Gitea reports github.server_url as https://github.com, so the condition was
true there too and the jobs ran (and would fail on @v4/gh).

Switch both job guards to `vars.GITHUB_RELEASE == 'true'`, a repository variable
set only in the GitHub repo's settings. It is deterministic: absent on Gitea, so
the jobs skip there and the .gitea workflow is the one that runs; present on
GitHub once configured, so it runs there. The header documents how to set it
(Settings → Secrets and variables → Actions → Variables → GITHUB_RELEASE=true).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:50:04 +01:00
UltradesuandClaude Opus 4.8 99f4cf0220 Add a GitHub release workflow alongside the Gitea one
Release / tsunagi-linux-aarch64 (push) Canceled after 36s
Release / tsunagi-linux-x86_64 (push) Canceled after 0s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Gitea and GitHub need different release plumbing, so each gets its own file: the
Gitea workflow stays in .gitea/workflows, and this GitHub one lives in
.github/workflows. The build half is identical (static musl Linux via
cargo-zigbuild, native macOS aarch64 and Windows x86_64); only publishing
differs — GitHub uses the v4 artifact backend and `gh release create`/`upload`,
which the Gitea file cannot use.

Because Gitea reads both .gitea/workflows and .github/workflows, every job here
is guarded with `if: github.server_url == 'https://github.com'`, so on Gitea
this file's jobs are skipped (no v4/gh failures) and the .gitea file is the one
that runs. GitHub never reads .gitea, so there it runs only this one.

Publish is re-run safe: it creates the release on the first run and uploads with
--clobber onto an existing release afterwards; a pre-release suffix on the tag
marks it prerelease.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:47:19 +01:00
UltradesuandClaude Opus 4.8 69263b2dd9 Move the release workflow to .gitea/workflows
Release / tsunagi-linux-aarch64 (push) Successful in 3m39s
Release / tsunagi-linux-x86_64 (push) Successful in 3m27s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
The release pipeline is Gitea-specific (v3 artifacts, Gitea release API), so it
must not run on github.com, where it would fail. GitHub only reads
.github/workflows, while Gitea reads both, so moving the file to
.gitea/workflows keeps it running on Gitea and invisible to GitHub. A separate
GitHub-flavoured release workflow can be added under .github later.

The portable .github/workflows/ci.yml (fmt/clippy/test) is left in place; it
works on both and is not what this change is about.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:13:45 +01:00
UltradesuandClaude Opus 4.8 20444013c0 Split release into build artifacts + a publish job (Gitea)
The direct-upload-from-each-build approach failed ("Upload the archive to the
release" exit 1) and left no artifacts to inspect. Restructured into the
familiar shape: build jobs now leave their archive as a run artifact, and a
single publish job — gated on all targets succeeding — collects them and fills
the release.

- Build jobs use actions/upload-artifact@v3 (Gitea has no v4 backend; v3 is
  what shows under the run), so the archive is visible and handed on.
- publish (needs: build) downloads everything with download-artifact@v3 and
  uploads each archive to the Gitea release with curl, once, with no matrix
  race. It prints the API base and the HTTP status/body of every call, so a
  failure (e.g. a read-only Actions token → 403, or a bad API URL) is now
  visible in the log instead of a bare exit 1.
- Release API base prefers GITHUB_API_URL (Gitea sets it to its /api/v1 root)
  and falls back to GITHUB_SERVER_URL/api/v1. Asset uploads run in the main
  shell via process substitution so a failed upload fails the job.

Build, package, musl/zig setup and the target matrix are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:12:53 +01:00
UltradesuandClaude Opus 4.8 2196f6635b Quiet the cosmetic zig linker_messages warning on the musl build
The musl release build now links cleanly through cargo-zigbuild, but rustc's
new linker_messages lint echoed zig's "ignoring deprecated linker optimization
setting '1'" note — zig's linker declining the -O1 GNU-ld flag Rust passes. It
is cosmetic and the binary is correct, so allow that one lint, scoped to the
zig build step only. macOS and Windows keep full linker diagnostics, and a real
link error still fails the build.

No behaviour change to the produced binaries; takes effect on the next tag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:06:04 +01:00
UltradesuandClaude Opus 4.8 6438982d45 Upload release assets via the Gitea API instead of artifacts + gh
Release / tsunagi-linux-aarch64 (push) Failing after 2m57s
Release / tsunagi-linux-x86_64 (push) Failing after 2m53s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
The release ran on Gitea Actions, where it failed at actions/upload-artifact@v4:
the v4 artifact backend (and the GitHub REST API that `gh release create` uses)
are not available on Gitea/GHES, so both the artifact handoff and the publish
job were dead ends there.

Reworked so each build job uploads its archive straight to the release through
Gitea's native API (`/api/v1/repos/<repo>/releases`) with curl: create the
release for the tag if absent (races between matrix jobs are resolved by reading
the id back by tag), then POST the archive as an asset. The separate publish
job, download-artifact and `gh` are gone. The id is parsed with grep, so the
runner needs only curl and coreutils — no jq, no third-party release action.
Uses the Gitea-injected GITHUB_TOKEN, GITHUB_SERVER_URL and GITHUB_REPOSITORY;
a tag with a pre-release suffix (e.g. -rc.2) is marked prerelease.

This targets Gitea by design; it no longer runs as-is on github.com (different
API), which is fine — the GitHub build is being retired. Build and package
steps, the musl/zig setup and the target matrix are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 12:04:20 +01:00
UltradesuandClaude Opus 4.8 21b619388c Pin Zig to a real release so the musl release build fetches reliably
Release / tsunagi-linux-aarch64 (push) Failing after 3m49s
Release / tsunagi-linux-x86_64 (push) Failing after 3m26s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
The release run failed at "Install zig": mlugg/setup-zig, left unpinned, resolved
to the newest version it knew of — an unreleased 0.16.0 whose tarball 404s on
ziglang.org — and the community mirrors returned 502/503 around it, so the step
errored out before cargo-zigbuild ever ran.

Pin the action to Zig 0.14.1, an archived release with a permanent official
download URL. The official source then succeeds even when the mirrors are flaky,
and the version stays compatible with cargo-zigbuild. Also moves the action to
@v2.

Only the Linux musl jobs use zig; macOS and Windows are unaffected. Not run on a
real tag yet in this session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:54:29 +01:00
UltradesuandClaude Opus 4.8 d1ab993dd2 Add a tagged-release CI workflow and Linux/Windows packaging
Release / tsunagi-linux-aarch64 (push) Failing after 3m12s
Release / tsunagi-linux-x86_64 (push) Failing after 2m21s
Release / tsunagi-macos-aarch64 (push) Canceled after 0s
Release / tsunagi-windows-x86_64 (push) Canceled after 0s
Release / Publish release (push) Canceled after 0s
Pushing a `v*` tag now builds the agent for every supported target and attaches
the archives to a GitHub release. Modelled on the furumi release workflow (a
build matrix, then a publish job that runs `gh release create`), adapted to this
project's two differences: it needs system integration (capabilities/polkit on
Linux, elevation+wintun on Windows, a root daemon on macOS), and its users run
varied distros, not just Ubuntu.

Linux is built as a fully **static musl** binary with cargo-zigbuild, so one
artifact runs on Arch, Alpine, NixOS and old-glibc systems alike, and can be
repackaged into whatever a distro prefers later. musl has no prebuilt
aws-lc-sys bindings, so the Linux jobs install cmake + libclang (bindgen) and
zig (the cross C compiler zigbuild drives). macOS (aarch64) and Windows (x86_64)
build natively; Linux covers x86_64 and aarch64.

Targets: x86_64/aarch64 linux-musl, aarch64 macOS, x86_64 Windows. Each archive
carries the binary, the top-level README, and the platform's install notes plus
integration files:
- dist/linux/: a static-friendly README, a systemd unit (runs as root, so no
  capability grant or polkit rule is needed; a commented non-root variant is
  included), and an install.sh that drops the binary and the unit or prints the
  setcap/polkit alternative.
- dist/windows/: install notes covering wintun.dll and Run-as-administrator.
- dist/macos/: the existing LaunchDaemon plist and notes, now also shipped in
  the archive.

Not exercised yet: the workflow only runs on a real tag push, and the
aws-lc-sys-under-musl build has not been run locally in this session (no zig
toolchain here). YAML and install.sh syntax are validated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:48:19 +01:00
UltradesuandClaude Opus 4.8 da7ce1dca1 Drop link-local addresses from the status "not announced" list
A host with several interfaces has a link-local (fe80::/10, and 169.254/16 on
IPv4) on each, and none of them is ever a peer candidate. Listing them under
"not announced" padded the section with one noise line per interface — most of
the list on a machine with several VPN utuns. They are now filtered out;
loopback and real unannounced addresses are unaffected.

Checks: cargo fmt --all --check and cargo clippy -p tsunagi-cli --all-targets
-D warnings pass; verified by eye against a live `tsunagi status`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:38:21 +01:00
UltradesuandClaude Opus 4.8 4d2f8a54ec Compact the status address list to one line per address
`tsunagi status` printed every local address as two lines: the address, then
an indented note holding the interface (and, for candidates, a priority rank).
On a host with many interfaces — common on macOS, where each VPN adds a utun
and every interface carries a link-local — that made the "local addresses"
section very long and hard to scan.

Each address is now a single line: the announce status stays the label
(announced / not announced / loopback) and the address and its interface share
the detail, as `<addr> · <iface>`. The per-row note and the candidate-priority
rank are dropped; ordering still reflects candidate priority. No other section
changes, and nothing but the presentation does.

Checks: cargo fmt --all --check, cargo clippy --locked --workspace
--all-targets -D warnings, and the tsunagi-cli tests all pass on macOS;
verified by eye against a live `tsunagi status`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:26:13 +01:00
UltradesuandClaude Opus 4.8 c3a6201ee2 Bring the macOS stack up: utun interface, resolver files, system paths
macOS was TUN-less: the agent ran only with `--no-tun`, so tunnels formed but
nothing reached the operating system. The provisioner, host rules and DNS
publisher had Linux and Windows implementations and fell through to the
unsupported/erroring paths on macOS. This adds the macOS half so `tsunagi up`
carries real traffic there. The shared decision logic (plan_changes,
rules_summary, Published) is untouched — only the per-platform mechanics are new.

What it does on macOS (all root-only; macOS has no per-capability grant):
- Interface: a `utun` device opened over the in-kernel utun control socket via
  the existing `tun` dependency (no kext, no Network Extension). The kernel
  assigns the `utunN` name, so a name derived from the network id is not a legal
  utun name; `open_tun` leaves the name unset on macOS and reads the real one
  back, and `Interface::start` adopts `device.name()` as the name everything
  downstream configures (host rules, reconfigure). A utun is torn down with the
  process, so a crash never leaks one. Addresses, MTU, up and the overlay subnet
  route are applied with `/sbin/ifconfig` and `/sbin/route` (absolute paths, args
  built only from locally-derived values), the same role netlink plays on Linux
  and netsh on Windows. Observation is `netdev` (getifaddrs).
- DNS: a resolver-dir publisher writes `/etc/resolver/<network>` files pointing
  the system resolver at the overlay DNS server for each zone. The files carry a
  `port`, so the server needs no privileged socket, and a marker line, so the
  publisher only ever replaces or removes files it wrote and never clobbers a
  user's own. Removed on revert.
- Broadcast host rules: the `255.255.255.255` route is installed with `route`.
  The inbound-UDP firewall allowance (a pf anchor) is deferred and reported as
  the incomplete half, so `status` says what to allow by hand; the route still
  goes in. Failure stays contained, exactly as on the other platforms.
- Privilege: probe is optimistic (like Windows); the honest answer comes from
  the utun open, whose permission error now says to run with sudo or as a daemon.

Deployment:
- Default paths on macOS are fixed system locations rather than per-user Library
  dirs, so they do not depend on whose shell ran sudo: state `/var/db/tsunagi`,
  cache `/var/db/tsunagi/cache` (StoragePaths::user_default), control socket
  `/var/run/tsunagi/agent.sock` (default_control_socket_path, mirroring the
  Windows %ProgramData% branch). The local control socket already uses the
  shared `unix` adapter, so no IPC changes were needed — macOS is Unix.
- dist/macos/ carries a root LaunchDaemon plist and install notes; README's
  deployment section now documents the macOS path instead of "not implemented".

Decisions (agreed up front): utun auto-assign with the real name adopted;
ifconfig/route rather than raw PF_ROUTE/ioctl (which would need unsafe);
/etc/resolver files rather than scutil; route-now/firewall-deferred; root via
sudo or a LaunchDaemon rather than Network Extension (which would require a
signed sandboxed app bundle, not a Rust CLI).

Checks: on macOS (Darwin 27), `cargo fmt --all --check`, `cargo clippy --locked
--workspace --all-targets -D warnings`, and `cargo test --locked --workspace
--all-targets` all pass (0 failures); the new macos-gated unit tests run under
the standard workspace command via feature unification. Not yet exercised live:
actual utun creation, ifconfig/route application and /etc/resolver writes need a
root run, which has not been done in this session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 11:15:59 +01:00
Aleksandr Bogomiakov 533921059a Install broadcast route and firewall allowance on the host while broadcast is on
LAN discovery (CS 1.6 and similar) only worked after two manual steps on every
host: route 255.255.255.255 through the TUN, and allow inbound traffic on it.
Without the route the game's limited broadcast leaves via the physical adapter
and never reaches the overlay; without the firewall allowance the server's
unicast reply (from a different address than the one the request went to, so
never matched by conntrack) is dropped before the game's socket. Verified by
hand on Linux: with both in place the server answered A2S_INFO over the overlay.

The system level (not a plugin) now installs exactly those two things for the
interface it owns, and removes them when broadcast is turned off, the network
is left or stopped, or the agent exits.

Design:
- New boundary overlay::hostrules (BroadcastHostRules: idempotent apply/clear,
  with Linux, Windows, Mock and Unsupported implementations). TunFactory gains
  host_rules(); ManagedTunFactory::with_host_rules wires it in the CLI.
- Interface::sync_broadcast_rules reconciles the host against the routing
  table (RoutingTable::broadcast_host_target) and is called after address sync,
  on live `network broadcast on/off`, on network deactivation and by remove().
  Failure is contained: the outcome is kept and logged, never fatal.
- One interface carries several networks but the host has one 255.255.255.255
  route, so the source is the lowest NetworkId among broadcast-enabled networks
  with an address (documented in docs/broadcast.md).
- Linux: the route is added on the existing netlink thread (preferred source =
  overlay address, replace semantics). The firewall rule is `iptables -I INPUT
  -i <if> -p udp -s <overlay range>` tagged with a comment `tsunagi:<if>`;
  it is UDP-only and range-limited on purpose so TCP services stay unexposed.
  iptables is located in fixed system directories, never via PATH; stale tagged
  rules (e.g. from a crash) are replaced. A missing iptables leaves the route in
  place and is reported. rp_filter is not touched.
- Windows: PowerShell (absolute path) New-NetRoute metric 1 and a tagged
  inbound UDP Defender rule scoped to the interface alias and overlay range.
- status shows a `broadcast rules` row (ok, or which half failed and why).
  IPC control protocol bumped 14 -> 15 because OverlayReport gained a field.
- AGENTS.md records the narrow exception to "never touch routing/firewall".

Compatibility: restart the agent after upgrading (control protocol 15). No wire
or schema change.

Checks run (Linux only): cargo fmt --check, cargo clippy --locked --workspace
--all-targets -D warnings, cargo test --locked --workspace --all-targets, all
passing. Tests use a mock host and a scripted iptables; no OS setting is
changed. NOT done: the Windows code was never compiled for or run on Windows
(cross-compiling failed for lack of a C toolchain), only its script building is
unit-tested; the real netlink route and iptables calls were not exercised
against a live kernel in the suite; CS 1.6 was not launched from this change.
2026-10-01 15:03:43 +01:00
Aleksandr Bogomiakov d2778f0b5a Prevent WireGuard timer loop failure spam and fix handshake broadcast
When a WireGuard tunnel handshake timed out or a rekey failed,
Inner::on_tunnel_failed closed the transport link and reported the failure
to the agent core. However:
1. The peer remained in WireguardDevice::inner.peers. Once boringtun
   declared ConnectionExpired (after 90s), self.tunn.is_expired() remained
   permanently true on all subsequent 250ms timer ticks. drive_timers
   repeatedly detected failed = true every 250ms, flooding logs with
   'WARN WireGuard tunnel failed due to rekey timeout, triggering fallback'
   and sending repeated protocol failure events to the core 4 times/sec.
2. In WireguardLink::send, is_handshake_init incorrectly expected an
   envelope header offset (msg[74] == 1) when checking for initiation
   packets, which failed on raw WireGuard UDP packets. As a result,
   handshake initiations were not broadcast to all candidate endpoints.
3. In WireguardTransport::accept_loop, packet source matching contained
   dead envelope destination matching inspecting ephemeral key bytes
   against local endpoint IDs.

Fixes:
- In crates/tsunagi-wireguard/src/device.rs:
  - Add failed: bool to FlowTunnel to latch failure state. Once a failure
    is reported, update_timers returns (TunnResult::Done, false) on all
    subsequent ticks until a new handshake is started or completed.
  - In Inner::on_tunnel_failed, remove the failed peer from inner.peers so
    drive_timers ceases ticking on the dead tunnel.
- In crates/tsunagi-wireguard/src/plugin.rs:
  - In Worker::sync, retain only active unclosed links in state.links.
- In crates/tsunagi-wireguard/src/transport.rs:
  - Check is_handshake_init directly as '!msg.is_empty() && msg[0] == 1',
    ensuring handshake initiations are broadcast to all candidate addresses.
  - Clean up matched_peer to look up links and peer candidate addresses directly.
- Tests:
  - Update test_flow_tunnel_rekey_timeout_triggers_failure to assert failure
    latching and verify no repeated failure reports on subsequent ticks.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed.
- cargo build --release passed.
2026-10-01 03:18:32 +01:00
Aleksandr Bogomiakov 8cfece9479 Filter noisy internal warnings from boringtun and iroh net_report
The CLI's default tracing filter previously allowed WARN level logs from
boringtun and iroh, producing two confusing and noisy log messages during
routine operation:
1. 'WARN HANDSHAKE(REJECT_AFTER_TIME - KEEPALIVE_TIMEOUT - REKEY_TIMEOUT (on receive))':
   emitted by boringtun's internal timer tick when an endpoint primarily
   receives data for 165 seconds (180s - 10s - 5s). In boringtun, this is a
   routine scheduled rekey timer rather than an error or failure, but was
   unhelpfully logged as warn! upstream.
2. 'WARN probe timed out':
   emitted by iroh's net_report module during background HTTP captive portal
   detection when running in isolated subnets or VMs without direct internet
   access. This probe has no impact on mesh connectivity.

Update the default logging filter to:
  'info,iroh=warn,iroh::net_report=error,quinn=warn,rustls=warn,boringtun=error,mainline=off'

Real WireGuard handshake and rekey failures continue to be detected and
reported cleanly by Tsunagi's own data plane diagnostics and automatic
fallback handlers at INFO and WARN levels.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed.
- cargo build --release passed.
2026-10-01 03:06:20 +01:00
Aleksandr Bogomiakov 3930dff2dc Detect WireGuard tunnel rekey failure and trigger automatic fallback
Previously, because UDP is connectionless, when a pure WireGuard tunnel
encountered handshake or rekey failures (e.g. boringtun logging
`WARN HANDSHAKE(REKEY_TIMEOUT)` and `CONNECTION_EXPIRED`), the
`WireguardLink` never closed. It remained active in `agent.links` and
as a direct hop in `RelayHub.raw`. Consequently, outgoing packets
continued to be blackholed over the dead direct path, preventing
`RelayHub` from using working multihop relay routes and preventing
the agent from falling back to alternative protocols (`tcp-tls` or
`wg-quic`).

Key changes:
- In `crates/tsunagi-wireguard/src/device.rs`:
  - In `FlowTunnel`, track active handshake and rekey attempts via
    `handshake_started_at`. If an active handshake attempt remains
    unanswered beyond `HANDSHAKE_TIMEOUT` (15s) or boringtun reports
    `ConnectionExpired`, declare the tunnel dead.
  - Reset timeout timers upon successful handshake reception in
    `decapsulate`.
  - In `drive_timers`, when tunnel failure is detected, trigger
    `inner.on_tunnel_failed(peer)`, which closes the direct transport
    link and notifies the agent via `PluginContext::report_peer_protocol_failure`.
- In `crates/tsunagi-wireguard/src/transport.rs`:
  - Add `WireguardLink::close` to transition `is_closed` to true and
    notify `closed_notify`.
  - In `WireguardLink::recv`, select on `closed_notify` so readers
    unblock and return `None` immediately on close.
  - Add `WireguardTransport::close_link` to close active links by
    network and peer.
- In `crates/tsunagi/src/dataplane/mod.rs`:
  - Add `PluginRequest::ProtocolFailure` and public helper
    `PluginContext::report_peer_protocol_failure`.
- In `crates/tsunagi/src/agent/mod.rs` and `crates/tsunagi/src/agent/network.rs`:
  - Add `NetCommand::ProtocolFailure` to runtime command handling.
  - On protocol failure: remove dead direct link from `self.links`,
    clear it from `RelayHub.raw` (so `RelayHub` immediately routes
    over available multihop relays), record failure on `wg` for that
    peer with exponential backoff cooldown, emit `DataLinkDown`, and
    immediately call `ensure_links()` to dial the next mutually supported
    protocol (`tcp-tls` or `wg-quic`).
- Tests:
  - Add `test_flow_tunnel_rekey_timeout_triggers_failure` and
    `test_flow_tunnel_reset_on_handshake` unit tests in `device.rs`.
  - Add `test_wireguard_link_close_and_transport_close_link` in
    `transport_tests.rs`.
  - Add `test_protocol_dial_state_cooldown_and_fallback_selection` in
    `network.rs`.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
- Built release binary: target/release/tsunagi.
2026-10-01 03:02:09 +01:00
Aleksandr Bogomiakov 35acc4d231 Fix WireGuard relay preemption with UDP probing, allow VM bridges, and categorize status addresses
When remote peers communicate across networks with mixed connectivity (e.g.
a remote Linux node, a local Linux host, and a Windows VM guest under KVM):
- Previously, `WireguardTransport` immediately created and installed
  unverified direct links into `RelayHub` in `notify_peer_capability` and
  `open()`. When a remote peer had only unroutable private addresses
  (e.g. 192.168.122.151 on a KVM NAT bridge), this dead direct link
  preempted the working 2-hop relay, causing packet drops, failed handshakes,
  and REKEY_TIMEOUT errors.
- Host-guest VM bridges (`virbr*`) were previously filtered out, preventing
  the VM guest from discovering the host's bridge IP (192.168.122.1) and
  forcing it to attempt cross-interface connections to host Wi-Fi/LAN.
- Concurrent activation of multiple networks sharing one `tsun0` interface
  raced in `sync_addresses`, resulting in netlink `File exists (os error 17)`
  and `The object already exists` on Windows.

Key fixes:
- In `crates/tsunagi-wireguard/src/transport.rs`:
  - Implement bidirectional UDP probing with `TSUNAGI_WG_PROBE_V1` PING/PONG.
  - In `open()`, probe candidate endpoints with retransmission up to 1200ms.
    If unreachable, return `TransportError::Unreachable`, preventing dead
    links from preempting valid relay routes in `RelayHub`.
  - In `notify_peer_capability`, send hole-punching probes without prematurely
    installing unverified links.
  - In `accept_loop`, reply to PING probes with PONG and install confirmed
    inbound links targeting the verified source address.
  - In `WireguardLink::send`, inspect byte 74 of enveloped packets to detect
    handshake initiations, ensuring confirmed links route data packets only
    to the active target.
- In `crates/tsunagi-wireguard/src/plugin.rs`:
  - Permit VM bridges (`virbr*`) as candidate interfaces (priority 10).
  - Add `prioritize_local_subnet` to prioritize candidates sharing a local
    IPv4 subnet with the receiver (e.g. 192.168.122.0/24).
- In `crates/tsunagi/src/overlay/interface.rs` & provisioners:
  - Add async mutex `sync_lock` to serialize concurrent address syncs across networks.
  - Handle netlink EEXIST (os error 17) and Windows already-exists errors
    idempotently as success.
- In `crates/tsunagi-cli/src/main.rs`:
  - Split `local addresses` in `tsunagi status` into `announced` (candidate
    addresses in priority order with interface and rank), `not announced`
    (container bridges/internal addresses with interface name), and `loopback`.
- Tests:
  - Add `test_wireguard_transport_unreachable_probe_times_out` and
    `test_matches_subnet` unit tests.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
2026-10-01 02:38:13 +01:00
Aleksandr Bogomiakov c791102ad9 Fix overlay and virtual interface candidate leakage and WireGuard rekeying
On Windows (Wintun), iface.name in netdev is an adapter GUID, while
iface.friendly_name is 'tsun0'. Checking only iface.name failed to
identify and exclude Wintun overlay interfaces. Consequently, Windows
nodes announced their own overlay IP addresses (e.g. 10.13.37.26) as
candidate endpoints. Remote peers latched onto this overlay IP as the
WireGuard target, routing encrypted packets into tsun0 (recursive loop),
resulting in failed handshakes and subsequent HANDSHAKE(REKEY_TIMEOUT)
and CONNECTION_EXPIRED.

Similarly, Docker bridges (br-*, docker0) and virtual bridges were
announced and alphabetically sorted before physical LAN or Tailscale
interfaces, causing peers to attempt WireGuard handshakes with internal
Docker bridge addresses (e.g. 172.21.0.1).

Key changes:
- In crates/tsunagi-wireguard and crates/tsunagi-tcp-tls:
  - Robustly identify overlay interfaces by inspecting iface.name,
    iface.friendly_name, and iface.description for 'tsun', 'tsunagi',
    and 'wintun'.
  - Filter out container and virtual bridge interfaces (docker, br-,
    veth, virbr, cni, flannel, cali, kube, dummy, lxc, podman, vethernet,
    vboxnet, vmnet).
  - Prioritize candidates by physical interface score: interfaces with
    a default gateway (physical LAN/WAN) and Tailscale (100.64.0.0/10)
    score highest and sort to candidate #0.
  - In on_peer_capability, filter out any announced address falling inside
    any active overlay IPv4 range (defense in depth on receiver).
- In crates/tsunagi-wireguard/src/transport.rs:
  - In WireguardLink::send, broadcast Handshake Initiation packets (type 1)
    to all candidate addresses so initial handshakes and rekeys probe all
    candidate paths and recover if an endpoint roamed or NAT expired.
  - In accept_loop, prioritize exact socket address matches before falling
    back to candidate IP matches for NAT port roaming.
- In crates/tsunagi/src/net.rs:
  - Update iroh AddrFilter to inspect friendly_name and description,
    preventing Wintun overlay IPs from leaking into iroh discovery endpoints.
- Tests:
  - Add unit tests for candidate interface filtering, IP validation, and
    priority scoring in tsunagi-wireguard and tsunagi-tcp-tls.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
2026-10-01 02:15:06 +01:00
Aleksandr Bogomiakov f80841da63 Fix WireGuard multi-network demux, candidate discovery, and status reporting
When a peer belongs to multiple overlay networks served by one agent (e.g.
a VM participating in both 'hexor' and 'ultradesu'), packets received on
the shared WireGuard UDP socket could not be differentiated by source IP/port
alone. Previously, `WireguardTransport::accept_loop` picked the first
matching link found in iteration order, starving other networks for that peer
and causing rekey timeouts (`WARN HANDSHAKE(REKEY_TIMEOUT)`,
`ERROR CONNECTION_EXPIRED(REKEY_ATTEMPT_TIME)`) and packet loss on the
starved network.

Additionally:
- When a peer had multiple candidate endpoints (such as LAN, VM bridge, and
  Tailscale CGNAT addresses), `WireguardLink` sent handshake initiations
  only to `addrs[0]`. If the first address was unreachable from the remote
  side (e.g. host's Tailscale IP unreachable from inside a local VM), the
  handshake never completed.
- In `build_report`, peers known to the `wg-quic` plugin were added with
  empty tunnels even if an active `wg` (pure WireGuard) tunnel existed,
  causing `tsunagi status` to falsely report 'no WireGuard handshake yet'
  for connected and pingable peers.
- `local_ip_candidates` previously included IPv6 addresses which caused
  OS error 22 (EINVAL) when passed to the IPv4-bound UDP socket.

Key fixes:
- In `crates/tsunagi-wireguard/src/transport.rs`:
  - Store `peer_networks` as `HashSet<NetworkId>` per peer.
  - In `WireguardLink`, maintain a list of candidate addresses and a
    `confirmed` flag. Before a handshake response confirms the active path,
    transmit datagrams (handshake initiations) to all candidate addresses.
    Upon receiving an inbound datagram, lock onto the confirmed address.
  - In `accept_loop`, deliver inbound datagrams to all active links
    associated with the sender's endpoint ID across all networks. Boringtun's
    cryptographic index and AEAD verification naturally demultiplexes and
    accepts the packet in the correct network while discarding it in others.
  - Filter candidate addresses to IPv4 to match the bound socket family.
- In `crates/tsunagi-wireguard/src/plugin.rs`:
  - Only collect IPv4 interfaces in `local_ip_candidates`.
- In `crates/tsunagi-cli/src/main.rs`:
  - Refactor `build_report` to evaluate all protocols per peer and prioritize
    whichever protocol has an active tunnel (`wg` > `tcp-tls` > `wg-quic`).
- In `crates/tsunagi-wireguard/tests/plugin_tests.rs`:
  - Add `test_wireguard_two_networks_same_peers` verifying simultaneous
    bidirectional packet transmission between two peers sharing two networks
    over a single `WireguardTransport`.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (350+ tests passed).
2026-10-01 01:38:44 +01:00
Aleksandr Bogomiakov 7070a16104 Add pure WireGuard direct UDP data plane plugin and transport
Peers communicating on uncensored networks can benefit from raw WireGuard
performance over direct UDP without QUIC framing or double-encryption
overhead. This provides Tailscale-like direct tunnel performance while
preserving Tsunagi's decentralized mesh architecture.

Key changes:
- Create `tsunagi-wireguard` crate implementing the `IpPlugin` and
  `PacketTransport` contracts:
  - `WG_PROTOCOL = "wg"` with wire version 1.
  - Bounded `WgAnnouncement` carrying WireGuard public keys, listening UDP
    ports, and candidate IP addresses (filtering out overlay interfaces).
  - `WgKeyStore` managing persistent per-network WireGuard X25519 keys
    in SQLite with zeroization on drop.
  - `WireguardTransport` providing direct UDP packet transmission, candidate
    address tracking, inbound link delivery, and roaming support.
  - `WireguardDevice` running userspace WireGuard encryption via `boringtun`,
    timer maintenance, and AllowedIPs source enforcement.
- Integrate `wg` into `tsunagi-cli`:
  - Set default protocols to `"wg,tcp-tls,wg-quic"` so all protocols are
    enabled by default in order of throughput and performance.
  - Support `"wireguard"` as a convenience alias for `"wg"` in `--protocol`.
  - Bind `PureWgTransport` and launch the inbound datagram accept loop on `up`.
  - Pass `pure_wg` plugin to `AgentControl` and update `build_report` so
    active WireGuard tunnels and overlay metrics are reported in `tsunagi status`.
  - Clean up WireGuard keys on network removal in `forget_protocol_state`.
- Comprehensive test coverage in `tsunagi-wireguard`:
  - `announcement_tests`: round-trip serialization and verification of malformed,
    mismatched network, zero port, or self-claiming announcements.
  - `transport_tests`: loopback UDP link delivery between transports.
  - `plugin_tests`: end-to-end two-peer handshake and bidirectional IP packet
    transit over direct UDP.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed.
2026-10-01 01:24:25 +01:00
Aleksandr Bogomiakov 46575d0899 Prioritize stored agreed range over default range in reserve_range
When an agent restarted with multiple networks (e.g. hexor and ultradesu),
Agent::reserve_range() previously attempted to reserve the default configured
range (10.13.37.0/24) for whichever network happened to be listed first in
storage, even if that network had already agreed upon a different range
(such as 10.79.175.0/24). As a result, the first network seized 10.13.37.0/24
in the route table, causing the second network (which genuinely used 10.13.37.0/24)
to trigger a spurious RouteError::Overlap warning at startup until the first
network asynchronously loaded its signed records and released the reservation.

Key changes:
- In crates/tsunagi/src/agent/mod.rs, make reserve_range asynchronous and
  inspect stored signed records first using StateSet::agreed_range().
- If an agreed range already exists in storage, reserve that specific range
  in routes immediately instead of preemptively claiming the default
  overlay_ipv4_range.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets --jobs 2 -- --test-threads=2 passed (349 tests passed).
2026-10-01 00:36:24 +01:00
Aleksandr Bogomiakov a89e65475d Implement Debug manually for PipeSecurityAttributes on Windows
The windows_sys::Win32::Security::SECURITY_ATTRIBUTES struct is a raw C FFI
struct that does not implement std::fmt::Debug. Deriving Debug on
PipeSecurityAttributes failed Windows compilation with E0277.

Key changes:
- In crates/tsunagi/src/ipc/windows.rs, remove #[derive(Debug)] on
  PipeSecurityAttributes and provide an explicit std::fmt::Debug
  implementation formatting nLength, lpSecurityDescriptor, and
  bInheritHandle.

Validation on Linux:
- Verified type check against Windows target via rustc.
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets --jobs 2 -- --test-threads=2 passed (349 tests passed).
2026-10-01 00:31:21 +01:00
Aleksandr Bogomiakov 2ed447ae86 Filter overlay interface from candidates and align status member columns
Peers previously advertised overlay interface (tsun0) addresses as physical
candidate endpoints for both iroh discovery and tcp-tls announcements. When an
intermediate tunnel was active between two peers (e.g., between a host and a VM),
iroh probes to the remote peer's overlay address were routed through the existing
overlay, causing iroh to select the overlay IP itself as a 'direct' endpoint.
This caused recursive tunneling (QUIC over WireGuard over TLS) and confusing
status output showing peers connecting to their own overlay IP addresses.
Additionally, in tsunagi status, peer rows were displayed with unaligned
columns, mixing control plane status and data plane metrics in a way that was
difficult to scan.

Key improvements:
- In crates/tsunagi/src/net.rs, add an AddrFilter to the iroh EndpointBuilder
  to dynamically filter out any local addresses assigned to tsun* overlay
  interfaces before publishing to address discovery services (DHT/pkarr/STUN).
- In crates/tsunagi-tcp-tls/src/plugin.rs, exclude tsun* interfaces from
  local_ip_candidates() so overlay IPs are never advertised as physical
  TCP endpoints.
- In crates/tsunagi-cli/src/main.rs, exclude tsun* interfaces from
  netwatch_addresses() so the local addresses section reflects only actual
  physical/host network interfaces.
- In crates/tsunagi-cli/src/main.rs, update member_row() to format member
  details with aligned columnar layout: the overlay IP is placed first and
  padded, followed by control plane transport and latency, and line 2 pads
  tunnel metrics so that the data link endpoint (via ...) aligns vertically.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets --jobs 2 -- --test-threads=2 passed (349 tests passed).
2026-10-01 00:24:53 +01:00
Aleksandr Bogomiakov ac5a1596ef Allow unelevated Windows processes to access control socket via SDDL and shared path
When running the tsunagi agent as an elevated Administrator or Windows Service,
unprivileged user-space processes (such as a system tray application, user-level
CLI commands, or status monitors) could not interact with the control socket.
This was due to two constraints on Windows:
1. The default control socket path was derived from the per-user %APPDATA%,
   making an administrator's socket invisible or inaccessible to a non-elevated
   standard user profile.
2. Even if pointed to the same named pipe, Windows Mandatory Integrity Control (MIC)
   and default ACLs prevent a Medium-Integrity (unelevated) process from opening
   a High-Integrity (elevated) pipe server for write access unless explicitly allowed.

Key changes:
- In crates/tsunagi/src/ipc/windows.rs, implement PipeSecurityAttributes using
  Win32 ConvertStringSecurityDescriptorToSecurityDescriptorW with SDDL:
  "D:(A;;GRGW;;;WD)(A;;GA;;;BA)(A;;GA;;;SY)S:(ML;;NW;;;ME)". This grants generic
  read/write access to Everyone (WD), retains full control for Administrators (BA)
  and LocalSystem (SY), and sets the Mandatory Label SACL to Medium Integrity (ME)
  with No-Write-Up (NW).
- In crates/tsunagi/src/ipc/mod.rs, introduce default_control_socket_path() which
  points to %ProgramData%\tsunagi\agent.sock on Windows (and default per-user
  state directory on Unix).
- In crates/tsunagi-cli/src/main.rs, support TSUNAGI_CONTROL_SOCKET environment
  variable across all commands, and use default_control_socket_path() on Windows
  when running against the default state directory. Custom state directories (--state-dir)
  or explicit sockets (--control-socket) continue to take precedence, allowing
  multi-daemon setups to remain strictly isolated.
- In workspace Cargo.toml, set unsafe_code = "deny" so the workspace forbids unsafe
  code while permitting explicit, documented unsafe Win32 API calls in windows.rs.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets --jobs 2 -- --test-threads=2 passed (349 tests passed).
2026-10-01 00:13:44 +01:00
Aleksandr Bogomiakov 192ab25bbd Support RUST_LOG, enable UPnP portmapper and add NAT traversal logging
Previously, `tsunagi-cli` configured clap with a fixed default_value and
`env = "TSUNAGI_LOG"`, which caused `RUST_LOG` to be completely ignored.
In addition, `iroh` was built with `default-features = false` and lacked the
`"portmapper"` feature, which caused it to compile a stub client and disable
UPnP-IGD, PCP, and NAT-PMP port mapping. Furthermore, there was no visibility
into STUN address discovery, UPnP port mapping status, or QUIC transmission path
switching (hole punch vs relay).

Key improvements:
- Enable the `"portmapper"` feature on `iroh` in workspace Cargo.toml, pulling in
  `portmapper` and `igd-next` to actively request port mappings on local routers.
- In `crates/tsunagi-cli/src/main.rs`, support `RUST_LOG` with fallback precedence
  (`--log` > `TSUNAGI_LOG` > `RUST_LOG` > default filter). When debug or trace
  filtering is active, automatically enable `.with_target(true)` so users can see
  which subsystem emitted the log (e.g., portmapper, iroh::magicsock, tsunagi::net).
- In `crates/tsunagi/src/net.rs`, monitor `endpoint.watch_addr()` and
  `endpoint.home_relay_status()`. Emit `DEBUG` logs for local socket binding,
  discovered STUN reflexive/UPnP addresses, and relay connections. If no direct
  addresses are discovered after probe grace, emit a concise `WARN`.
- In `crates/tsunagi/src/agent/session.rs`, monitor connection `path_events()`
  to log when transmission paths are opened, closed, or selected (switching between
  relay and direct hole-punched paths), with bounded task winding down on shutdown.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (349 tests passed).
- Live verification with `RUST_LOG=debug` confirmed SSDP broadcast to 239.255.255.250:1900,
  portmapper logs, STUN QAD discovery, and clean relay shutdown.
2026-09-30 23:08:23 +01:00
Aleksandr Bogomiakov c0a76f1942 Enable all available protocols by default with tcp-tls priority
By default, the agent now enables all available data plane protocols
starting with `tcp-tls,wg-quic`, removing the need to manually pass `--protocol`
to enable HTTPS-camouflage transport. The `--protocol` option remains available
to explicitly restrict or select specific protocols (or reorder their priority).

Furthermore, in `crates/tsunagi-tcp-tls`, if default port 443 is unavailable
(e.g., due to permission constraints or address in use), the transport now
attempts fallback port 8443, and if 8443 is also unavailable, gracefully falls
back to an ephemeral port rather than failing agent startup.

Key changes:
- In `crates/tsunagi-cli/src/main.rs`, update `UpArgs::protocols` default_value
  to `"tcp-tls,wg-quic"`.
- In `crates/tsunagi-cli/src/main.rs`, reorder `PROTOCOLS` registry with `tcp-tls`
  first, followed by `wg-quic`.
- In `crates/tsunagi-tcp-tls/src/transport.rs`, add multi-tier fallback
  (443 -> 8443 -> ephemeral) for unconstrained port bindings.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (349 tests passed).
- cargo build --release passed.
2026-09-30 22:56:08 +01:00
Aleksandr Bogomiakov 510116afb9 Add automatic protocol fallback with backoff and silence DHT bootstrap error
When multiple data plane protocols (such as tcp-tls and wg-quic) are configured,
the agent previously attempted only the first mutually supported protocol in
served order. If that protocol failed to connect or dropped (due to NAT, firewall,
or handshake failure), ensure_links retried the exact same failed protocol
on every tick without ever falling back to alternative supported protocols.
Additionally, tcp-tls mistakenly included 127.0.0.1 in peer candidate lists,
causing peers to connect to their own local listener and fail certificate
verification (NotValidForName). Finally, BitTorrent DHT bootstrap failures
in the third-party mainline crate were logged as noisy ERRORs despite overlay
connectivity operating normally.

Key improvements:
- In crates/tsunagi/src/agent/network.rs, implement ProtocolDialState with
  exponential backoff (3s, 6s, 12s, 24s, 48s, capped at 60s).
- In ensure_links, select the highest-priority mutually supported protocol
  that is not currently on backoff cooldown. If a protocol fails, immediately
  record the failure and fall back to alternative mutually supported protocols
  without waiting for the periodic ticker.
- In crates/tsunagi-tcp-tls, remove localhost loopback from local_ip_candidates
  and capability announcements, and prevent open() from connecting to our own
  listening port.
- In crates/tsunagi-cli/src/main.rs, set mainline=off in the default tracing
  filter so benign BitTorrent DHT routing table bootstrap errors do not alarm users.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (349 tests passed).
- cargo build --release passed.
2026-09-30 22:51:08 +01:00
Aleksandr Bogomiakov e24548b3b0 Fix Windows address allocation check and add custom SNI for tcp-tls
When running on Windows with Wintun, assigning an overlay IPv4 address
via netsh triggers Duplicate Address Detection (DAD). During this tentative
window, binding a UDP socket to the address returns WSAEADDRNOTAVAIL
(os error 10049), causing the agent to prematurely log a false-positive
PluginError ('allocated ... but the address is not on any interface').

Furthermore, Russian TSPU / DPI filters block unrecognized SNIs and
non-standard ALPNs on TCP port 443 with TCP RST injection (os error 10054).
The default 'tsunagi.local' SNI and 'tsunagi-tls/1' ALPN were trivially
fingerprinted.

Key improvements:
- In crates/tsunagi/src/overlay/tun.rs, enhance address_is_local on Windows
  to inspect active OS interfaces via netdev::get_interfaces() if UDP
  bind fails.
- In crates/tsunagi/src/agent/network.rs, introduce an asynchronous retry
  loop in publish_allocations on Windows (up to 1.5s in 150ms intervals)
  to gracefully allow DAD and IP helper updates to complete before
  reporting missing addresses.
- In crates/tsunagi-tcp-tls, add support for custom SNI domain via
  protocol options (-o sni=<domain>), defaulting to 'cloudflare.com'
  instead of 'tsunagi.local'.
- Switch TLS ALPN in tsunagi-tcp-tls from 'tsunagi-tls/1' to 'http/1.1'
  to camouflage handshake traffic as standard HTTPS web traffic.
- Update tests to verify custom and default SNI behavior.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (348 tests passed).
- cargo build --release passed.
2026-09-30 20:38:16 +01:00
Aleksandr Bogomiakov a4e341cd66 Report peer tunnel protocols accurately and eliminate overlay shutdown timeout
The status output previously reported all overlay peers under WireGuard
diagnostics: when a non-WireGuard protocol (like tcp-tls) was actively
carrying traffic, the report still complained that there was 'no WireGuard
handshake yet; the tunnel cannot carry traffic' and marked peer health as
degraded (warn). Furthermore, during agent shutdown or interface removal,
the overlay packet loop waited on device.recv() for the full 5-second
TASK_GRACE period, producing a spurious warning: 'task did not wind down in
time; aborting it task="overlay packet loop"'.

Key improvements:
- Add 'protocol' field to OverlayPeerReport in tsunagi::ipc.
- Track per-peer packet metrics (tx, rx, dropped) and uptime in
  GenericTunnelPlugin via TunnelPeerReport.
- Update build_report to accurately inspect and populate active tunnel
  protocols (tcp-tls, wg-quic) and their real traffic counters.
- Format member rows in CLI status with the actual protocol name, uptime,
  and real traffic metrics; mark health as Good when an active tunnel exists.
- In Interface::remove(), abort the overlay packet loop task immediately
  before destroying the device to remove the 5s hang and spurious timeout warning.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (348 tests passed).
- cargo build --release passed.
2026-09-30 17:31:17 +01:00
Aleksandr Bogomiakov 41990c9934 Refactor agent logging and report active tcp-tls tunnels
Previous agent logging suffered from noisy repetitive '--- status ---'
dumps to stdout every 15 seconds, intermingled with raw println! event lines
and lacking structured log levels. Furthermore, the status report only
inspected WireguardPlugin tunnels; when tcp-tls took precedence and established
a link, the status falsely showed wg-quic as '0/1 tunnels established, waiting
for a data link' without mentioning the active tcp-tls link.

Key changes:
- Adopt compact, clean tracing output: initialize tracing_subscriber with
  .with_target(false).compact() and default filter
  'info,iroh=warn,quinn=warn,rustls=warn,boringtun=warn,mainline=warn'
  to keep standard agent events at INFO while silencing noisy internals.
- Route agent events through structured tracing (log_event):
  emit INFO for peer connect/disconnect, data link up/down, and network
  activation; WARN for rejections and plugin errors; DEBUG for dials.
- Disable periodic status dump by default (set status_interval default to 0),
  and re-route periodic ticks to tracing::debug! (log_status_summary).
- Add active_peers() to GenericTunnelPlugin and update build_report to
  include active tcp-tls tunnels in OverlayReport alongside WireGuard peers.
- Display enabled protocol priority on agent startup.

Validation on Linux:
- cargo fmt --all -- --check passed.
- cargo clippy --locked --workspace --all-targets -- -D warnings passed.
- cargo test --locked --workspace --all-targets passed (347 tests passed).
2026-09-30 17:16:02 +01:00
Aleksandr Bogomiakov 254a25ebd6 Display supported protocols in status report host section
The `host` section in `tsunagi status` previously hardcoded
`implementation: userspace WireGuard (boringtun); no kernel module needed`
and omitted information about data plane protocols supported by the build.

Changes:
- Add `protocols` row to `host_section` listing supported protocols
  and their wire versions (`wg-quic (wire v4), tcp-tls (wire v1)`) with a note
  directing to `tsunagi protocols` for detailed options.
- Update `implementation` row to reflect the multi-protocol userspace
  data plane (`userspace overlay (boringtun WireGuard, TCP TLS 1.3); no kernel module needed`).

Validation on Linux:
- `cargo fmt --all -- --check` passed.
- `cargo clippy --locked --workspace --all-targets -- -D warnings` passed.
- `cargo test --locked --workspace --all-targets` passed.
2026-09-30 17:00:39 +01:00
Aleksandr Bogomiakov 5495cb5b34 Support CAP_NET_BIND_SERVICE raising for privileged TCP TLS ports
When file capabilities are granted with `sudo setcap ...+p`, capabilities
are placed into the thread's permitted set rather than its effective set.
While TUN interface provisioning in `tsunagi` properly acquired and dropped
CAP_NET_ADMIN using `caps::raise`, `tsunagi-tcp-tls` attempted to bind port 443
directly without activating CAP_NET_BIND_SERVICE, resulting in EACCES
(os error 13 Permission denied).

Fixes and improvements:
- Add `caps` dependency to `tsunagi-tcp-tls` on Linux.
- Implement `NetBindServiceGuard` to temporarily raise CAP_NET_BIND_SERVICE
  into the effective set for the synchronous `TcpListener::bind` invocation
  and immediately drop it afterwards.
- Clarify diagnostic warnings to distinguish between default port 443 fallback
  and explicitly requested port failure.
- Update all privilege diagnostics and error hints across the codebase
  to uniformly recommend granting both capabilities:
  `sudo setcap cap_net_admin,cap_net_bind_service+p <path>`.

Validation on Linux:
- `cargo fmt --all -- --check` passed.
- `cargo clippy --locked --workspace --all-targets -- -D warnings` passed.
- `cargo test --locked --workspace --all-targets` passed (all workspace unit,
  integration, and protocol tests passed).
2026-09-30 16:53:33 +01:00
Aleksandr Bogomiakov fac3bd7bb9 Add tcp-tls transport and tunnel protocol disguised as HTTPS
Implement the `tcp-tls` protocol crate and generic tunnel codec abstractions
alongside `wg-quic` to circumvent deep packet inspection (DPI/TSPU), UDP
blocking, and QUIC throttling. Traffic uses standard TCP on port 443 with
TLS 1.3 framing, looking identical to regular HTTPS traffic to network
observers.

Architectural highlights:
- TunnelCodec & GenericTunnelPlugin: Provide an extensible plugin abstraction
  in `crates/tsunagi/src/dataplane/tunnel.rs` that encapsulates background task
  lifecycles, per-peer channels, and sink dispatch, so new transport plugins
  only define cryptographic transformations and handshake payloads.
- MultiPacketTransport & Protocol Priority Selection: Support multiple
  simultaneous transports in `AgentConfig` and dynamically dispatch `open()`
  calls by protocol identifier. In `ensure_links`, choose the highest-priority
  mutually supported protocol per peer to avoid duplicate links.
- TLS 1.3 & Certificate Pinning: Generate self-signed Ed25519 X.509 certificates
  derived from the persistent `DeviceIdentity` secret. Clients pin peer
  certificates using `PinnedEndpointVerifier` against `iroh::EndpointId`.
- End-to-end Encryption: `PeerCipher` uses X25519 static-static ECDH, HKDF-SHA256,
  and ChaCha20-Poly1305 with incrementing 64-bit nonces. Relayed traffic across
  intermediate nodes remains confidential and tamper-proof.
- Privilege handling: Default to TCP port 443 with fallback to 8443 on
  `PermissionDenied`, providing an actionable message recommending:
  `sudo setcap cap_net_admin,cap_net_bind_service+p <path>`.

Validation on Linux:
- `cargo fmt --all -- --check` passed.
- `cargo clippy --locked --workspace --all-targets -- -D warnings` passed.
- `cargo test --locked --workspace --all-targets` passed (all workspace unit,
  integration, CLI, and new `tsunagi-tcp-tls` tests passed).
2026-09-30 16:41:51 +01:00
ab db9015b26c Allow network commands to select stored names
Leaving a configured network required copying its opaque id even when its
name identified exactly one saved network. Resolve exact, case-sensitive
names from the existing local state alongside full ids and unique id prefixes.
Use the shared resolver for leave, stop, start, secret and broadcast so the
same selector has the same meaning throughout the CLI. IPC and storage still
operate on the resolved id; release and offline-removal behavior are unchanged.

Refuse ambiguous names and name/id-prefix collisions before any mutation,
listing matching names and full ids without secrets. Full ids take precedence
to provide a definitive escape from ambiguity. Preserve copied id ellipses
without stripping literal ellipses from network names.

Update help and examples. Make administrator-friendly, predictable CLI behavior
a core requirement in AGENTS.md, including actionable errors and no guessing
which resource a destructive command should affect.

Validation on Windows: cargo fmt --all -- --check and workspace/all-target
Clippy with -D warnings passed. Release workspace/all-target tests: 318 passed,
including real-process online/offline leave-by-name, preservation of other
networks, ambiguous/unknown selectors, and consistent name selection across
commands. Two existing Windows SQLite wipe failures were explicitly skipped
(a_wipe_removes_everything_and_the_next_start_is_a_stranger and
wiping_twice_is_as_ordinary_as_wiping_once). The public-DHT smoke test and
forwarding benchmark remain ignored by default. Tests use isolated state;
the user's running agent and configured networks were not changed.
2026-09-22 19:00:08 +03:00
ab c724981bfd Added per-network LAN broadcast relay
LAN game discovery previously dropped IPv4 broadcasts at TUN ingress. Carry
limited and subnet-directed UDP broadcasts to authenticated, opted-in members
of the source network, including destinations reached through mesh relays.
Preserve the original IP/UDP bytes and deliver received broadcasts only to the
local TUN; never reflood them or expose another pair's plaintext at transit.

Build immutable recipient snapshots on address and participation changes. The
origin sends one ordinary end-to-end encrypted copy per recipient; the existing
fast, bounded-hop transport router remains unchanged. Validate UDP framing,
source ownership and destination admission without game-specific port rules.
Keep network domains isolated and refuse implicit gateways to physical LANs.
The separate broadcast policy/domain layer is the extension point for future
authorized subnet exports; physical capture, bridging and LAN deduplication
are deliberately not implemented yet.

Persist default-on participation independently for each local network. Add
join --no-broadcast/--broadcast and network broadcast <id> [on|off], including
live updates and authenticated announcements. Joining without a flag preserves
the saved choice. Opt-out stops local origination and delivery, while opaque
unicast transit for other members keeps working.

Migrate SQLite schema 3 to 4 without replacing identities or signed state.
Use control ALPN 3 and local IPC protocol 14 for the new announcement/request
shapes; update peers and restart running agents together. The data ALPN 4
envelope remains unchanged. No release version bump, tag or push is included.

Document agent-owned commits in AGENTS.md: short English subjects, explanatory
bodies, scoped staging, honest validation, and repository-local fallback author
AB <ab@hexor.cy> only when an effective name/email is missing. Release actions
remain the user's responsibility.

Validation on Windows: cargo fmt --all -- --check; cargo check --locked
--workspace --all-targets; cargo clippy --locked --workspace --all-targets --
-D warnings; release workspace/all-target tests: 313 passed. The two existing
SQLite wipe failures (a_wipe_removes_everything_and_the_next_start_is_a_stranger
and wiping_twice_is_as_ordinary_as_wiping_once) were explicitly skipped; the
public-DHT smoke test and forwarding benchmark remain ignored by default.
New coverage exercises real iroh/WireGuard multihop fanout, single delivery,
runtime opt-out, unicast replies, domain isolation, malformed input and schema
migration. TUNs are in-memory; actual games and OS adapter selection were not
tested.
2026-09-22 18:33:52 +03:00
ab b4f3e57c8d Implement fast userspace multihop mesh routing
Replace the one-intermediate-peer relay with protocol-scoped connectivity
graphs and precomputed shortest-path/ECMP forwarding snapshots. Independent
transport readers forward opaque transit frames without a plugin or TUN
round trip. Carry source, destination, a bounded hop limit and stable flow
tags; preserve end-to-end WireGuard links across topology changes.

Classify IP flows before encryption and preserve their tags through the
WireGuard pending queue. Add offline four-agent path-change coverage, loop
and isolation tests, and an opt-in release forwarding microbenchmark. Bump
control/data ALPNs while preserving persistent network identities and state.

Also include the pending Windows Mainline idle-timeout fix and its regression
test, using a reproducible vendored dependency patch.

Validation: fmt, workspace Clippy with warnings denied, and 307 release tests
passed. Two pre-existing Windows SQLite wipe failures were excluded; public
DHT and the manual benchmark remain ignored by default. The forwarding
microbenchmark measured 103 ns (64 B) and 202 ns (1280 B) per transit packet,
excluding encryption and socket I/O.
2026-09-22 18:08:26 +03:00
189 changed files with 35886 additions and 3112 deletions

No files matched your search

+322
View File
@@ -0,0 +1,322 @@
name: Release
# Build the agent for every supported target on a version tag, leave each build
# as a run artifact, and then — only after every target has built — attach the
# archives to the release. Linux is a fully static musl binary (built with
# cargo-zigbuild) so one artifact runs on any distro and can be repackaged
# however a distro prefers.
#
# Targeted at Gitea Actions: artifacts use @v3 (the v4 backend is not available
# on Gitea/GHES), and the release is filled through Gitea's own API with curl
# rather than `gh`, which speaks only to github.com.
on:
push:
tags: ["v*"]
permissions:
contents: write
env:
CARGO_TERM_COLOR: always
CARGO_NET_RETRY: "10"
jobs:
build:
name: ${{ matrix.asset }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
asset: tsunagi-linux-x86_64
archive: tar.gz
zig: true
deb: true
deb_arch: amd64
pkg_arch: x86_64
gnu_target: x86_64-unknown-linux-gnu
- os: ubuntu-latest
target: aarch64-unknown-linux-musl
asset: tsunagi-linux-aarch64
archive: tar.gz
zig: true
deb: true
deb_arch: arm64
pkg_arch: aarch64
# 32-bit ARM with hardware float (ARMv7, `v7l`): single-board and edge
# devices. Statically linked like the others.
- os: ubuntu-latest
target: armv7-unknown-linux-musleabihf
asset: tsunagi-linux-armv7
archive: tar.gz
zig: true
deb: true
deb_arch: armhf
pkg_arch: armv7h
- os: macos-latest
target: aarch64-apple-darwin
asset: tsunagi-macos-aarch64
archive: tar.gz
zig: false
- os: windows-latest
target: x86_64-pc-windows-msvc
asset: tsunagi-windows-x86_64
archive: zip
zig: false
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.target }}
# aws-lc-sys (iroh's rustls backend) and the bundled sqlite are C, and the
# musl targets have no prebuilt bindings, so bindgen needs libclang and the
# C build needs cmake. zig is the cross C compiler cargo-zigbuild drives.
- name: Install musl build and packaging tooling
if: matrix.zig
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends cmake clang libclang-dev libarchive-tools zstd
- name: Install zig
if: matrix.zig
uses: mlugg/setup-zig@v2
with:
# Pin a real, archived Zig release. Left unpinned the action resolves
# to the newest version it knows of (a yet-unreleased 0.16.0-dev),
# whose tarball 404s on ziglang.org while the community mirrors flake.
version: 0.14.1
- name: Install cargo-zigbuild
if: matrix.zig
uses: taiki-e/install-action@v2
with:
tool: cargo-zigbuild
- name: Build
shell: bash
run: |
set -euo pipefail
if [[ "${{ matrix.zig }}" == "true" ]]; then
# zig's linker ignores the `-O1` GNU-ld optimization flag Rust passes
# and says so on stderr, which rustc now surfaces as a linker_messages
# warning. It is cosmetic — the binary is correct — so quiet just that
# one lint to keep the musl build log clean. A real link error still
# fails the build.
export RUSTFLAGS="${RUSTFLAGS:-} -A linker_messages"
cargo zigbuild --release --locked --target "${{ matrix.target }}" --bin tsng
else
cargo build --release --locked --target "${{ matrix.target }}" --bin tsng
fi
# The tray GUI. On Linux it is built against glibc (an old baseline for
# portability), not musl: a GUI must dlopen libGL/X11/Wayland at runtime,
# which a static musl binary cannot. macOS and Windows build it natively.
- name: Build the tray GUI
shell: bash
run: |
set -euo pipefail
if [[ "${{ runner.os }}" == "Linux" ]]; then
if [[ -n "${{ matrix.gnu_target }}" ]]; then
# tray-icon links GTK and libappindicator through pkg-config, which
# has no cross story under zig, so the Linux GUI is built natively
# (x86_64 only) against the runner's glibc.
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libgtk-3-dev libayatana-appindicator3-dev libxdo-dev
cargo build --release --locked \
--target "${{ matrix.gnu_target }}" --bin tsunagi-tray
else
echo "no GUI build for ${{ matrix.target }}"
fi
else
cargo build --release --locked --target "${{ matrix.target }}" --bin tsunagi-tray
fi
- name: Package
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
stage="dist/pkg/${{ matrix.asset }}"
mkdir -p "$stage"
bin="tsng"
[[ "${{ runner.os }}" == "Windows" ]] && bin="tsng.exe"
cp "target/${{ matrix.target }}/release/${bin}" "$stage/"
# The tray GUI alongside it, where it was built (Linux: x86_64 only).
tray="tsunagi-tray"
[[ "${{ runner.os }}" == "Windows" ]] && tray="tsunagi-tray.exe"
if [[ "${{ runner.os }}" == "Linux" ]]; then
if [[ -n "${{ matrix.gnu_target }}" ]]; then
cp "target/${{ matrix.gnu_target }}/release/${tray}" "$stage/"
fi
else
cp "target/${{ matrix.target }}/release/${tray}" "$stage/"
fi
# The top-level README, plus the per-OS install notes and integration
# files, so the archive carries what the platform needs to set up.
cp README.md "$stage/"
case "${{ runner.os }}" in
Linux)
cp dist/linux/tsunagi.service dist/linux/install.sh \
dist/linux/50-tsunagi-resolved.rules "$stage/"
cp dist/linux/README.md "$stage/INSTALL.md"
;;
macOS)
cp dist/macos/cy.hexor.tsunagi.plist "$stage/"
cp dist/macos/README.md "$stage/INSTALL.md"
;;
Windows)
cp dist/windows/README.md "$stage/INSTALL.md"
;;
esac
archive="${{ matrix.asset }}-${version}.${{ matrix.archive }}"
if [[ "${{ matrix.archive }}" == "zip" ]]; then
(cd dist/pkg && 7z a "../../${archive}" "${{ matrix.asset }}")
else
tar -C dist/pkg -czf "${archive}" "${{ matrix.asset }}"
fi
# A dependency-free .deb for the x86_64 and aarch64 Linux builds: it drops
# the static binary, the systemd service and the polkit rule. Built with
# dpkg-deb (present on the runner); see dist/deb/build-deb.sh.
- name: Build the Debian package
if: matrix.deb
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
bash dist/deb/build-deb.sh \
"target/${{ matrix.target }}/release/tsng" \
"${{ matrix.deb_arch }}" \
"$version" \
"tsunagi_${version}_${{ matrix.deb_arch }}.deb"
# A dependency-free Arch package (.pkg.tar.zst), assembled with bsdtar and
# zstd (no makepkg); same binary, service and polkit rule. See
# dist/arch/build-pkg.sh.
- name: Build the Arch package
if: matrix.pkg_arch
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
pkgver="${version//-/_}"
bash dist/arch/build-pkg.sh \
"target/${{ matrix.target }}/release/tsng" \
"${{ matrix.pkg_arch }}" \
"$version" \
"tsunagi-${pkgver}-1-${{ matrix.pkg_arch }}.pkg.tar.zst"
# The GUI variants: the same agent plus the tray binary and a desktop
# launcher, named tsunagi-gui (conflicting with the headless packages).
- name: Build the GUI packages
if: matrix.gnu_target
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
pkgver="${version//-/_}"
tray="target/${{ matrix.gnu_target }}/release/tsunagi-tray"
bash dist/deb/build-deb.sh \
"target/${{ matrix.target }}/release/tsng" \
"${{ matrix.deb_arch }}" "$version" \
"tsunagi-gui_${version}_${{ matrix.deb_arch }}.deb" "$tray"
bash dist/arch/build-pkg.sh \
"target/${{ matrix.target }}/release/tsng" \
"${{ matrix.pkg_arch }}" "$version" \
"tsunagi-gui-${pkgver}-1-${{ matrix.pkg_arch }}.pkg.tar.zst" "$tray"
# @v3 on purpose: Gitea does not support the v4 artifact backend. This is
# what makes the archive (and the .deb, where built) show up under the run
# and hands it to `publish`.
- name: Upload the build outputs as run artifacts
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.asset }}
path: |
${{ matrix.asset }}-*.*
*.deb
*.pkg.tar.zst
if-no-files-found: error
publish:
name: Publish release
needs: build
runs-on: ubuntu-latest
steps:
- name: Download every build artifact
uses: actions/download-artifact@v3
with:
path: artifacts
- name: Show what was collected
run: find artifacts -type f
- name: Create the release and upload the archives
shell: bash
env:
# Gitea injects this with access to the current repository. If the
# release calls below return 403, the repo/runner is configured to
# hand Actions a read-only token — grant it write access to releases.
TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -u
tag="${GITHUB_REF_NAME}"
# Gitea sets GITHUB_API_URL to its own API root (…/api/v1); fall back
# to building it from the server URL if it is absent.
base="${GITHUB_API_URL:-${GITHUB_SERVER_URL}/api/v1}"
api="${base}/repos/${GITHUB_REPOSITORY}"
auth="Authorization: token ${TOKEN}"
echo "release API: $api"
prerelease=false
case "$tag" in *-*) prerelease=true ;; esac
# The release id for this tag, if it already exists.
code=$(curl -sS -o resp.json -w '%{http_code}' -H "$auth" "$api/releases/tags/$tag" || true)
echo "get release by tag: HTTP $code"
id=$(grep -o '"id":[0-9]*' resp.json 2>/dev/null | head -n1 | cut -d: -f2 || true)
if [ -z "$id" ]; then
code=$(curl -sS -o resp.json -w '%{http_code}' -X POST -H "$auth" \
-H 'Content-Type: application/json' \
-d "{\"tag_name\":\"$tag\",\"name\":\"tsunagi $tag\",\"prerelease\":$prerelease}" \
"$api/releases" || true)
echo "create release: HTTP $code"
id=$(grep -o '"id":[0-9]*' resp.json 2>/dev/null | head -n1 | cut -d: -f2 || true)
if [ -z "$id" ]; then
echo "could not create the release; response was:" >&2
cat resp.json >&2 || true
exit 1
fi
fi
echo "release id: $id"
# download-artifact@v3 nests each artifact in a directory named after
# its upload, with the single archive inside. The read loop runs in
# this shell (process substitution, not a pipe) so a failed upload
# actually fails the job.
rc=0
while IFS= read -r -d '' f; do
name=$(basename "$f")
code=$(curl -sS -o up.json -w '%{http_code}' -X POST -H "$auth" \
-F "attachment=@${f}" \
"$api/releases/$id/assets?name=${name}" || true)
echo "upload $name: HTTP $code"
if [ "$code" -lt 200 ] || [ "$code" -ge 300 ]; then
cat up.json >&2 || true
rc=1
fi
done < <(find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.deb' -o -name '*.pkg.tar.zst' \) -print0)
exit "$rc"
+294
View File
@@ -0,0 +1,294 @@
name: Release
# GitHub version of the release pipeline. The build half mirrors the Gitea one
# in .gitea/workflows/release.yml (static musl Linux via cargo-zigbuild, native
# macOS and Windows); only the publish half differs: it uses the current
# artifact actions (Node 24) and fills the release with `gh`.
#
# Every job is gated on `github.server_url == 'https://github.com'`, so this file
# runs only on GitHub. Gitea runs its own copy in .gitea/workflows (which pins
# the older @v3 artifact actions it supports); should a Gitea instance also read
# .github/workflows, the guard skips these jobs there so they never fail on the
# artifact backend or `gh`.
on:
push:
tags: ["v*"]
permissions:
contents: write
env:
CARGO_TERM_COLOR: always
CARGO_NET_RETRY: "10"
jobs:
build:
name: ${{ matrix.asset }}
if: ${{ github.server_url == 'https://github.com' }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
asset: tsunagi-linux-x86_64
archive: tar.gz
zig: true
deb: true
deb_arch: amd64
pkg_arch: x86_64
gnu_target: x86_64-unknown-linux-gnu
- os: ubuntu-latest
target: aarch64-unknown-linux-musl
asset: tsunagi-linux-aarch64
archive: tar.gz
zig: true
deb: true
deb_arch: arm64
pkg_arch: aarch64
# 32-bit ARM with hardware float (ARMv7, `v7l`): single-board and edge
# devices. Statically linked like the others.
- os: ubuntu-latest
target: armv7-unknown-linux-musleabihf
asset: tsunagi-linux-armv7
archive: tar.gz
zig: true
deb: true
deb_arch: armhf
pkg_arch: armv7h
- os: macos-latest
target: aarch64-apple-darwin
asset: tsunagi-macos-aarch64
archive: tar.gz
zig: false
- os: windows-latest
target: x86_64-pc-windows-msvc
asset: tsunagi-windows-x86_64
archive: zip
zig: false
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.target }}
# aws-lc-sys (iroh's rustls backend) and the bundled sqlite are C, and the
# musl targets have no prebuilt bindings, so bindgen needs libclang and the
# C build needs cmake. zig is the cross C compiler cargo-zigbuild drives.
- name: Install musl build and packaging tooling
if: matrix.zig
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends cmake clang libclang-dev libarchive-tools zstd
- name: Install zig
if: matrix.zig
uses: mlugg/setup-zig@v2
with:
# Pin a real, archived Zig release. Left unpinned the action resolves
# to the newest version it knows of (a yet-unreleased 0.16.0-dev),
# whose tarball 404s on ziglang.org while the community mirrors flake.
version: 0.14.1
- name: Install cargo-zigbuild
if: matrix.zig
uses: taiki-e/install-action@v2
with:
tool: cargo-zigbuild
- name: Build
shell: bash
run: |
set -euo pipefail
if [[ "${{ matrix.zig }}" == "true" ]]; then
# zig's linker ignores the `-O1` GNU-ld optimization flag Rust passes
# and says so on stderr, which rustc now surfaces as a linker_messages
# warning. It is cosmetic — the binary is correct — so quiet just that
# one lint to keep the musl build log clean. A real link error still
# fails the build.
export RUSTFLAGS="${RUSTFLAGS:-} -A linker_messages"
cargo zigbuild --release --locked --target "${{ matrix.target }}" --bin tsng
else
cargo build --release --locked --target "${{ matrix.target }}" --bin tsng
fi
# The tray GUI. On Linux it is built against glibc (an old baseline for
# portability), not musl: a GUI must dlopen libGL/X11/Wayland at runtime,
# which a static musl binary cannot. macOS and Windows build it natively.
- name: Build the tray GUI
shell: bash
run: |
set -euo pipefail
if [[ "${{ runner.os }}" == "Linux" ]]; then
if [[ -n "${{ matrix.gnu_target }}" ]]; then
# tray-icon links GTK and libappindicator through pkg-config, which
# has no cross story under zig, so the Linux GUI is built natively
# (x86_64 only) against the runner's glibc.
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libgtk-3-dev libayatana-appindicator3-dev libxdo-dev
cargo build --release --locked \
--target "${{ matrix.gnu_target }}" --bin tsunagi-tray
else
echo "no GUI build for ${{ matrix.target }}"
fi
else
cargo build --release --locked --target "${{ matrix.target }}" --bin tsunagi-tray
fi
- name: Package
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
stage="dist/pkg/${{ matrix.asset }}"
mkdir -p "$stage"
bin="tsng"
[[ "${{ runner.os }}" == "Windows" ]] && bin="tsng.exe"
cp "target/${{ matrix.target }}/release/${bin}" "$stage/"
# The tray GUI alongside it, where it was built (Linux: x86_64 only).
tray="tsunagi-tray"
[[ "${{ runner.os }}" == "Windows" ]] && tray="tsunagi-tray.exe"
if [[ "${{ runner.os }}" == "Linux" ]]; then
if [[ -n "${{ matrix.gnu_target }}" ]]; then
cp "target/${{ matrix.gnu_target }}/release/${tray}" "$stage/"
fi
else
cp "target/${{ matrix.target }}/release/${tray}" "$stage/"
fi
# The top-level README, plus the per-OS install notes and integration
# files, so the archive carries what the platform needs to set up.
cp README.md "$stage/"
case "${{ runner.os }}" in
Linux)
cp dist/linux/tsunagi.service dist/linux/install.sh \
dist/linux/50-tsunagi-resolved.rules "$stage/"
cp dist/linux/README.md "$stage/INSTALL.md"
;;
macOS)
cp dist/macos/cy.hexor.tsunagi.plist "$stage/"
cp dist/macos/README.md "$stage/INSTALL.md"
;;
Windows)
cp dist/windows/README.md "$stage/INSTALL.md"
;;
esac
archive="${{ matrix.asset }}-${version}.${{ matrix.archive }}"
if [[ "${{ matrix.archive }}" == "zip" ]]; then
(cd dist/pkg && 7z a "../../${archive}" "${{ matrix.asset }}")
else
tar -C dist/pkg -czf "${archive}" "${{ matrix.asset }}"
fi
# A dependency-free .deb for the x86_64 and aarch64 Linux builds: it drops
# the static binary, the systemd service and the polkit rule. Built with
# dpkg-deb (present on the runner); see dist/deb/build-deb.sh.
- name: Build the Debian package
if: matrix.deb
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
bash dist/deb/build-deb.sh \
"target/${{ matrix.target }}/release/tsng" \
"${{ matrix.deb_arch }}" \
"$version" \
"tsunagi_${version}_${{ matrix.deb_arch }}.deb"
# A dependency-free Arch package (.pkg.tar.zst), assembled with bsdtar and
# zstd (no makepkg); same binary, service and polkit rule. See
# dist/arch/build-pkg.sh.
- name: Build the Arch package
if: matrix.pkg_arch
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
pkgver="${version//-/_}"
bash dist/arch/build-pkg.sh \
"target/${{ matrix.target }}/release/tsng" \
"${{ matrix.pkg_arch }}" \
"$version" \
"tsunagi-${pkgver}-1-${{ matrix.pkg_arch }}.pkg.tar.zst"
# The GUI variants: the same agent plus the tray binary and a desktop
# launcher, named tsunagi-gui (conflicting with the headless packages).
- name: Build the GUI packages
if: matrix.gnu_target
shell: bash
run: |
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
pkgver="${version//-/_}"
tray="target/${{ matrix.gnu_target }}/release/tsunagi-tray"
bash dist/deb/build-deb.sh \
"target/${{ matrix.target }}/release/tsng" \
"${{ matrix.deb_arch }}" "$version" \
"tsunagi-gui_${version}_${{ matrix.deb_arch }}.deb" "$tray"
bash dist/arch/build-pkg.sh \
"target/${{ matrix.target }}/release/tsng" \
"${{ matrix.pkg_arch }}" "$version" \
"tsunagi-gui-${pkgver}-1-${{ matrix.pkg_arch }}.pkg.tar.zst" "$tray"
- name: Upload the build outputs as run artifacts
uses: actions/upload-artifact@v7
with:
name: ${{ matrix.asset }}
path: |
${{ matrix.asset }}-*.*
*.deb
*.pkg.tar.zst
if-no-files-found: error
publish:
name: Publish release
needs: build
if: ${{ github.server_url == 'https://github.com' }}
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Download every build artifact
uses: actions/download-artifact@v8
with:
path: artifacts
- name: Show what was collected
run: find artifacts -type f
- name: Create the release and upload the archives
shell: bash
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ github.ref_name }}
run: |
set -euo pipefail
# download-artifact nests each artifact under its own directory, so
# the archives are artifacts/<asset>/<file>.
mapfile -t files < <(find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.deb' -o -name '*.pkg.tar.zst' \))
printf 'asset: %s\n' "${files[@]}"
prerelease=()
case "$TAG" in *-*) prerelease=(--prerelease) ;; esac
# Create the release on the first run; on a re-run, add/replace assets
# on the release that is already there.
if gh release view "$TAG" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
gh release upload "$TAG" "${files[@]}" --repo "$GITHUB_REPOSITORY" --clobber
else
gh release create "$TAG" "${files[@]}" \
--repo "$GITHUB_REPOSITORY" \
--title "tsunagi $TAG" \
--generate-notes \
--verify-tag \
"${prerelease[@]}"
fi
+68 -22
View File
@@ -13,12 +13,24 @@ messages. Scope and non-scope are in [README.md](README.md).
There is no central server, no owner, no registration and no majority vote.
Design accordingly: a majority is not a root of trust.
## User experience
User convenience is a core requirement. The CLI should feel logical and
predictable to an experienced administrator: use consistent selectors and
flags, sensible defaults, and actionable errors. Let users refer to configured
networks by their exact name as well as by id or a unique id prefix; do not
make them copy an opaque identifier when the stored name identifies one network.
When a selector is ambiguous, list the matching names and ids and change
nothing. Never guess which resource a destructive command should affect.
Keep help text and examples in sync with behavior, and preserve existing
scripts when improving the interactive workflow.
## Architectural boundaries
Keep these separate. Crossing them is the main thing to review for.
- **Control plane vs data plane.** The separation is **logical, not physical**.
The control protocol in `crates/tsunagi/src/proto/` knows nothing about
The control protocol in `crates/tsunagi-core/src/proto/` knows nothing about
packets, and a protocol crate knows nothing about the control protocol;
either can be replaced on its own. Both may ride on iroh — refusing to
would throw away iroh's NAT traversal and force the data plane to
@@ -45,14 +57,22 @@ Keep these separate. Crossing them is the main thing to review for.
is why the authenticated transport does too.
- **A relay carries bytes, never packets.** A datagram passed on for two
other peers goes link in, link out: it is never written to the middle's
interface, never decrypted there, and never relayed twice. Reachability
that decides a route is first-hand and volatile — a live announcement
over the control plane, never a signed record and never second-hand.
interface or decrypted there. Multihop routing has a bounded hop limit;
equal-cost next hops are chosen per flow. Reachability is first-hand and
volatile: each authenticated member advertises its own protocol-specific
links. Build routing tables on topology changes, never per packet. Transit
must not acquire a routing mutex or wait for a protocol/TUN reader.
- **Broadcast fanout belongs at local IP ingress.** Participation is local to
each network, enabled by default, persisted and advertised to authenticated
peers. Resolve one source domain and target only its willing members. Remote
delivery never triggers another fanout. Physical LAN exports must extend
explicit ingress and source/destination admission policies; never bypass
ownership checks or add broadcast flooding to encrypted transit.
- **Plugins never learn reachability.** An `IpPlugin` is handed a `PacketLink`
per peer and moves datagrams over it. Addresses, hole punching and relays
belong to `crates/tsunagi/src/dataplane/transport/`. A plugin announcement says *who*, never
belong to `crates/tsunagi-core/src/dataplane/transport/`. A plugin announcement says *who*, never
*where*.
- **The core never parses a plugin payload.** See `crates/tsunagi/src/dataplane/mod.rs`. Only
- **The core never parses a plugin payload.** See `crates/tsunagi-core/src/dataplane/mod.rs`. Only
`crates/tsunagi-wg-quic/src/announcement.rs` interprets `wg-quic` payloads, and
only after bounding every field. A data plane failure must never stop the
control plane.
@@ -61,7 +81,7 @@ Keep these separate. Crossing them is the main thing to review for.
inbound packets are dropped unless their source is the address derived for
the peer that sent them. Never trust an address a peer announces.
- **Signed state is the only durable agreement.** A fact that must survive a
participant being away goes in `crates/tsunagi/src/state/` as a record signed by its
participant being away goes in `crates/tsunagi-core/src/state/` as a record signed by its
author, never in a session. Merging is deterministic, an older version never
rolls back a newer one, and absence from a snapshot is not deletion. Never
add a vote or a quorum: a majority is not a trust root here, and it would
@@ -70,6 +90,12 @@ Keep these separate. Crossing them is the main thing to review for.
transaction, **before** the record is announced.
- **Plugins own their system objects.** A plugin creates and removes its own
interface and nothing else. Never touch routing, DNS or firewall settings.
The one exception is the system level, not a plugin: while a network's
broadcast is on, `overlay::hostrules` may install exactly a
`255.255.255.255/32` route and an inbound UDP firewall allowance (overlay range
only) for the interface the agent owns. Both are tagged `tsunagi:<interface>`,
reversible, removed on broadcast off, leave and shutdown, and computed from
local state, never from a remote announcement.
- **Device identity vs network identity.** The iroh endpoint id is the device's
public key. `NetworkId` is derived from name + secret only. Never conflate
them, and never let one change the other.
@@ -89,7 +115,7 @@ Keep these separate. Crossing them is the main thing to review for.
normal path *and* the important failures. See [docs/testing.md](docs/testing.md).
Do not add tests for getters or to move a coverage number.
2. **Never change `IDENTITY_SCHEME`, the derivation labels, or the transcript
encoding** in `crates/tsunagi/src/identity/network.rs` and `crates/tsunagi/src/proto/handshake.rs` without
encoding** in `crates/tsunagi-core/src/identity/network.rs` and `crates/tsunagi-core/src/proto/handshake.rs` without
treating it as an incompatible protocol change. Bumping the crate version or
the control protocol version must not change an existing `NetworkId`.
3. **Secrets never leak.** Not into logs, not into `Debug`, not into status
@@ -100,7 +126,7 @@ Keep these separate. Crossing them is the main thing to review for.
the library; tests opt out explicitly at the top of each file.
5. **Bounds before allocation.** Frame lengths are checked against
`Limits::max_frame_len` before a buffer is allocated. Every string, list and
queue has a limit in `crates/tsunagi/src/config.rs`.
queue has a limit in `crates/tsunagi-core/src/config.rs`.
6. **Failure is contained.** A bad signature, wrong secret, malformed packet or
unknown version rejects one message or one session. It never stops another
network and never stops the agent. There is no irreversible global error
@@ -117,24 +143,25 @@ Keep these separate. Crossing them is the main thing to review for.
up shutdown. Wind tasks down with a grace period and then abort.
10. **Nothing from a remote announcement becomes a shell command, a filesystem
path or an OS setting.** The agent never touches interfaces or OS settings
it did not create.
it did not create (the tagged broadcast host rules above are for its own
interface).
## Where things live
| path | responsibility |
|---------------------|----------------|
| `crates/tsunagi/src/identity/` | device identity; deterministic network space identity and derived keys |
| `crates/tsunagi/src/storage/` | `state.sqlite`, `cache.sqlite`, directory ownership lock |
| `crates/tsunagi/src/discovery.rs` | candidate sources; test and static backends |
| `crates/tsunagi/src/proto/` | framing, message formats, membership handshake |
| `crates/tsunagi/src/net.rs` | iroh endpoint adapter and observability snapshots |
| `crates/tsunagi/src/agent/` | agent lifecycle, per-network runtimes, sessions, events, status |
| `crates/tsunagi/src/state/` | signed records that outlive a session, their merge rules and address allocation |
| `crates/tsunagi/src/overlay/` | the one interface an agent owns: provisioning, the TUN, routing, source checks |
| `crates/tsunagi/src/dataplane/` | the protocol contract and the authenticated packet transport |
| `crates/tsunagi/src/dns/` | the DNS view of a network: zone, server, resolver publication |
| `crates/tsunagi-core/src/identity/` | device identity; deterministic network space identity and derived keys |
| `crates/tsunagi-core/src/storage/` | `state.sqlite`, `cache.sqlite`, directory ownership lock |
| `crates/tsunagi-core/src/discovery.rs` | candidate sources; test and static backends |
| `crates/tsunagi-core/src/proto/` | framing, message formats, membership handshake |
| `crates/tsunagi-core/src/net.rs` | iroh endpoint adapter and observability snapshots |
| `crates/tsunagi-core/src/agent/` | agent lifecycle, per-network runtimes, sessions, events, status |
| `crates/tsunagi-core/src/state/` | signed records that outlive a session, their merge rules and address allocation |
| `crates/tsunagi-core/src/overlay/` | the one interface an agent owns: provisioning, the TUN, routing, source checks, broadcast host rules |
| `crates/tsunagi-core/src/dataplane/` | the protocol contract and the authenticated packet transport |
| `crates/tsunagi-core/src/dns/` | the DNS view of a network: zone, server, resolver publication |
| `crates/tsunagi-wg-quic/` | the `wg-quic` protocol: its keys, its announcement, its tunnels |
| `crates/tsunagi-cli/` | the command line agent; the only place that owns a runtime, a logger and signals |
| `crates/tsunagi/` | the command line agent, built as `tsng`; the only place that owns a runtime, a logger and signals |
| `tests/` | integration tests; `tests/common/` is the shared harness |
Add abstractions only at real substitution or testing boundaries. Do not add a
@@ -166,7 +193,26 @@ kind.
- Running several library instances in one process is not a test of several
system processes; do not describe it as one.
## Before you open a change
## Commit every completed change
Agents must create a Git commit before finishing any task that changes this
repository. Do not leave the user to make the commit. Inspect recent commit
messages and follow their style: a short English subject stating the change,
then a useful body explaining the problem, final behavior, architectural
decisions, compatibility/migration effects, and checks actually performed.
Write enough context for a future maintainer or LLM to understand why the code
exists. Record failed or skipped checks honestly; never claim unrun tests.
Before committing, inspect the diff and stage only this task's changes. Preserve
unrelated user changes and never commit secrets or generated build artifacts.
If Git has no effective author name or email, set only the missing repository-
local values to `AB` and `ab@hexor.cy`. Do not override an existing identity or
change global Git configuration. Report the resulting commit hash.
Version bumps, release tags, and pushing commits/tags belong to the user unless
they explicitly ask the agent to do them. Read-only tasks need no empty commit.
## Required validation
```bash
cargo fmt --all -- --check
Generated
+3138 -103
View File
File diff suppressed because it is too large. Load diff
+10 -4
View File
@@ -4,16 +4,22 @@
[workspace]
resolver = "3"
members = ["crates/*"]
exclude = ["vendor/mainline"]
# Mainline 8.0.0 treats Windows UDP read timeouts as socket failures.
# Keep the small receive-loop correction reproducible until an upstream fix.
[patch.crates-io]
mainline = { path = "vendor/mainline" }
[workspace.package]
edition = "2024"
rust-version = "1.91"
license = "MIT OR Apache-2.0"
repository = "https://github.com/tsunagi-net/tsunagi"
license = "WTFPL"
repository = "https://github.com/house-of-vanity/tsunagi"
[workspace.lints.rust]
missing_docs = "warn"
unsafe_code = "forbid"
unsafe_code = "deny"
[workspace.lints.clippy]
unwrap_used = "warn"
@@ -21,7 +27,7 @@ expect_used = "warn"
panic = "warn"
[workspace.dependencies]
iroh = { version = "1.2", default-features = false, features = ["tls-ring"] }
iroh = { version = "1.2", default-features = false, features = ["tls-ring", "portmapper"] }
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros"] }
serde = { version = "1.0", features = ["derive"] }
postcard = { version = "1.1", default-features = false, features = ["use-std"] }
+13
View File
@@ -0,0 +1,13 @@
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
Version 2, December 2004
Copyright (C) 2004 Sam Hocevar <sam@hocevar.net>
Everyone is permitted to copy and distribute verbatim or modified
copies of this license document, and changing it is allowed as long
as the name is changed.
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. You just DO WHAT THE FUCK YOU WANT TO.
+121 -73
View File
@@ -36,7 +36,7 @@ A working library with **real iroh connections** and integration tests:
overlay with deterministically derived addresses and optional IPv4, real
tunnels carried over iroh, and address ownership enforced rather than
believed;
- a **command line agent**, `tsunagi`, with a local control socket.
- a **command line agent**, `tsng`, with a local control socket.
### What it deliberately does **not** do
@@ -71,8 +71,8 @@ On the first machine:
```bash
cargo build --release
./target/release/tsunagi status # this device, the agent, and this host
./target/release/tsunagi network # the networks this device belongs to
./target/release/tsng status # this device, the agent, and this host
./target/release/tsng network # the networks this device belongs to
```
Two commands do the work, and they are separate on purpose: `up` runs the
@@ -81,19 +81,19 @@ what it belongs to, at any time, while it runs.
```bash
# Terminal one: the agent. It prints its endpoint id and then serves.
./target/release/tsunagi up
./target/release/tsng up
# Terminal two: make a network. With no --secret it invents one and prints
# it, along with the line to send the other machine.
./target/release/tsunagi join --network lab
./target/release/tsng join --network lab
```
On the second machine, start its agent and join the same network with the
secret that was printed:
```bash
./target/release/tsunagi up
./target/release/tsunagi join --network lab --secret tsn1...
./target/release/tsng up
./target/release/tsng join --network lab --secret tsn1...
```
**Name and secret are enough for the first meeting.** Mainline DHT discovery
@@ -141,7 +141,7 @@ wg-quic: tsun0 on 10.13.37.69/24 mtu 1280, 1/1 tunnel(s) established
From another shell on either machine:
```bash
tsunagi status
tsng status
```
```text
@@ -170,8 +170,8 @@ ping 100.65.243.53
what they find, so `--ipv4-range` only matters for whoever starts the network:
```bash
tsunagi up --ipv4-range 10.44.0.0/16
tsunagi up --ipv4-range none # no data plane at all
tsng up --ipv4-range 10.44.0.0/16
tsng up --ipv4-range none # no data plane at all
```
One agent has one interface, so two of its networks cannot both use that
@@ -193,7 +193,7 @@ assigns it to the interface itself.
## Levels
The command line is split the way the design is. A bare `tsunagi up` needs
The command line is split the way the design is. A bare `tsng up` needs
only a network name and a secret; everything else sits under the level it
belongs to, which `--help` shows as two sections:
@@ -204,7 +204,7 @@ belongs to, which `--help` shows as two sections:
their own settings (`-o key=value`, or `-o protocol:key=value`).
```bash
tsunagi protocols # what this build can carry packets with
tsng protocols # what this build can carry packets with
```
```text
@@ -223,7 +223,7 @@ A pair of peers uses a protocol they both have **at the same wire version**.
That is not the software version: two peers on different builds carry traffic
for each other for as long as the bytes between them have not changed. A peer
with nothing in common keeps its control plane — messages and signed state
still flow — and simply has no data plane, which `tsunagi status` shows as a
still flow — and simply has no data plane, which `tsng status` shows as a
session with no agreed protocol.
## Names
@@ -233,12 +233,12 @@ each named after the network, so members can be reached by name instead of
by address:
```bash
tsunagi up # DNS is already enabled
tsng up # DNS is already enabled
dig @127.0.0.1 -p 5354 music.lab
```
An explicit choice is remembered across restarts. `--no-dns` or `tsunagi dns off`
disables it; `--dns` or `tsunagi dns on` enables it again. Existing saved opt-outs
An explicit choice is remembered across restarts. `--no-dns` or `tsng dns off`
disables it; `--dns` or `tsng dns on` enables it again. Existing saved opt-outs
remain respected. The `dns on` and `dns off` commands also take effect immediately
on a running agent. `--dns` and `--no-dns` cannot be combined.
@@ -299,7 +299,7 @@ polkit.addRule(function(action, subject) {
RULE
```
**Without it the server still runs** — `tsunagi status` prints where it is
**Without it the server still runs** — `tsng status` prints where it is
listening and the exact `dig` line — so the automatic part is missing, not
the feature. The refusal is said once rather than on every pass, and retried
slowly, because nothing but a person will change it.
@@ -318,7 +318,7 @@ remote peer could influence.
That needs `CAP_NET_ADMIN`, granted once:
```bash
sudo setcap cap_net_admin+p /usr/local/bin/tsunagi
sudo setcap cap_net_admin+p /usr/local/bin/tsng
```
`+p` rather than `+ep`: the capability is then *permitted* but not
@@ -328,7 +328,7 @@ allocation changes. Everything else, including every byte from the network,
is handled with it lowered. `+ep` works too; the agent lowers it on the way
in.
`tsunagi status` says which of these applies on the host it runs on, along
`tsng status` says which of these applies on the host it runs on, along
with what the agent is doing. It grades each finding: **ok** for what works,
**warn** for what the agent runs without and you can fix from the line it
prints, **FAIL** for what it cannot work around. The words carry the grade as
@@ -353,28 +353,28 @@ Every item takes the same shape, so there is nothing to remember: name it to
see it, name it with a value to change it.
```
tsunagi id everything about this device
tsunagi id hostname the name it answers to
tsunagi id hostname mango change it
tsunagi id key the key it signs with
tsunagi id key rotate replace that key
tsng id everything about this device
tsng id hostname the name it answers to
tsng id hostname mango change it
tsng id key the key it signs with
tsng id key rotate replace that key
tsunagi network the networks this device belongs to
tsunagi network join -n lab -s tsn1… join one; adds it to a running agent
tsunagi network join -n lab resume one this device has, or make it on the spot
tsunagi network stop <id> stop serving it, keeping everything
tsunagi network start <id> serve it again, from where it left off
tsunagi network leave <id> give up the address and name, then forget it
tsunagi network secret the secret of each joined network
tsunagi network secret <id> just that one, for copying
tsunagi network secret generate a fresh secret for a network that does not exist yet
tsng network the networks this device belongs to
tsng network join -n lab -s tsn1… join one; adds it to a running agent
tsng network join -n lab resume one this device has, or make it on the spot
tsng network stop lab stop serving it, keeping everything
tsng network start lab serve it again, from where it left off
tsng network leave lab give up the address and name, then forget it
tsng network secret the secret of each joined network
tsng network secret lab just that one, for copying
tsng network secret generate a fresh secret for a network that does not exist yet
tsunagi dns whether the local resolver is serving, and what
tsunagi dns on start it, now and after every restart
tsunagi dns off stop it, now and after every restart
tsng dns whether the local resolver is serving, and what
tsng dns on start it, now and after every restart
tsng dns off stop it, now and after every restart
```
**A network without a secret makes one.** `tsunagi join --network lab`
**A network without a secret makes one.** `tsng join --network lab`
resolves a bare name in the obvious way: if this device is already in
exactly one network called `lab`, that one — so the name alone resumes what
you have; if it is in none, a fresh random secret, printed in full with the
@@ -386,11 +386,11 @@ joined `lab` (k2on43wadbi5x267vp6z3ogkm7nbjedfdoxyauxhtxhwqrprylba)
Run this on the other machine:
tsunagi join --network lab --secret tsn1u7c…
tsng join --network lab --secret tsn1u7c…
Its agent has to be running. If it is not:
tsunagi up --peer 91e83a6e2b7a…
tsng up --peer 91e83a6e2b7a…
```
Two networks of one name is a thing that happens — a mistyped secret makes
@@ -410,11 +410,11 @@ Asking for it is deliberate, because these reports get pasted into chats.
**`up` takes no network at all** if you would rather decide later: it
brings up the agent and whatever it is already configured for, and waits.
That is the shape of a daemon in one terminal and `tsunagi network join`
That is the shape of a daemon in one terminal and `tsng network join`
in another.
`network join` is also the answer to a question `up` cannot: a state
directory belongs to one live agent, so a second `tsunagi up` cannot add a
directory belongs to one live agent, so a second `tsng up` cannot add a
network to the one already running. This adds it over the control socket and
it starts at once. With no agent running it is written to the configuration
and starts with the next `up`.
@@ -477,10 +477,11 @@ The original IP packet, including its DF bit, is preserved. See
|---|---|---|
| `setcap cap_net_admin+p` | ordinary user, one capability | recommended: nothing to prepare, nothing left behind. Lost on every rebuild or copy of the binary. |
| systemd service | `User=`, `AmbientCapabilities=CAP_NET_ADMIN` | the same, for an installed service |
| `sudo tsunagi up` | root | everything works, nothing is isolated |
| `sudo tsng up` | root | everything works, nothing is isolated |
| macOS root LaunchDaemon | root | the native macOS deployment; see below |
| `--no-tun` | ordinary user, no capabilities | tunnels run and handshake, traffic never reaches the OS |
On Windows, put `wintun.dll` beside the executable and start `tsunagi up`
On Windows, put `wintun.dll` beside the executable and start `tsng up`
from PowerShell or Command Prompt opened with **Run as administrator**.
Run commands controlling that agent (`join`, `status`, `dns`, `network`)
as the same Windows user with the same elevation. If Windows denies access
@@ -489,9 +490,24 @@ of claiming the agent is absent or trying to edit its locked state.
Creating a TUN without sufficient privileges also explains how to restart
the agent with the required permissions.
**Not implemented yet.** macOS has no provisioner; `--no-tun` is the way to
run it. The control plane and tunnels are unaffected. The decision logic
that says *what* to change is shared and tested on every platform.
On macOS the agent creates a `utun` interface, configures it with `ifconfig`
and `route`, and points the system resolver at the overlay by writing
`/etc/resolver/<network>` files (which carry a port, so the DNS server needs no
privileged socket). All of this needs **root**: macOS has no per-capability
grant like Linux, and a `utun` plus `/etc/resolver` are root-only. The kernel
assigns the interface its `utunN` name, so the name derived from the network id
is only a request and the real one is adopted once the interface is up.
Run it as root — `sudo tsng up` for a quick start, or install it as a root
**LaunchDaemon** for the native always-on deployment. As a daemon the agent runs
unprivileged commands (`join`, `status`, `dns`, `network`) over its control
socket like any other platform; the socket lives at `/var/run/tsunagi/agent.sock`
and the stores at `/var/db/tsunagi`, both fixed system paths so they do not
depend on whose shell started `sudo`. A ready-to-edit plist and install notes
are in [dist/macos/](dist/macos/). The broadcast route is installed; the
inbound-UDP firewall allowance (a `pf` anchor) is not yet, so `status` reports
that half as incomplete and names what to allow by hand. The decision logic that
says *what* to change is shared and tested on every platform.
## Checks
@@ -563,26 +579,54 @@ explicitly.
### Through somebody in the middle
Everybody tries everybody first: the mesh is pairwise, and a relay is only
for the pair that cannot manage it. Two members can both reach a third and
not each other: a blocked path, a
relay that is unavailable, a network only reachable from inside somebody
else's building. When that happens the pair is routed through a member that
has both.
Tsunagi routes the overlay in userspace. A direct data link always wins;
otherwise the shortest available path can pass through several members.
Each authenticated member advertises only its own live, protocol-specific
transport links. The control plane collects these into a graph and replaces
the routing table when connectivity changes; announcements expire after
90 seconds without refresh.
Nothing is agreed and nothing is elected. Each member says only which peers
*it* has a live link with, first-hand, over the control plane and one hop
only; everyone picks their own way through from that, deterministically, and
drops it the moment a direct link exists. There is no routing protocol, no
second-hand claim to weigh, and a relayed datagram is never relayed again —
so a loop cannot form.
Transit goes transport → router → transport, without entering WireGuard or
the intermediate host's TUN. The payload stays encrypted between the original
endpoints. TUN connects only the local OS to Tsunagi; the kernel needs only
the overlay route through that interface, with no IP forwarding configuration.
The one in the middle carries **bytes it cannot read**: the tunnel stays end
to end between the two ends, and a relayed datagram never touches the middle
host's interface, so no forwarding, routing or firewall setting of that host
is involved. `status` says `via <peer>` on a path that goes through
somebody, and counts what this device has carried for others — it is their
traffic on your uplink, and that should not be invisible.
A 16-hop limit bounds loops while topology updates converge. Equal shortest
paths use a stable flow hash derived before encryption, so packets of a TCP
or UDP flow keep their path while topology is unchanged. Physical link changes
do not replace end-to-end WireGuard tunnels or change the 1280-byte default MTU.
`status` reports the hop count and first next hop for relayed paths.
The hot path reads an immutable routing snapshot and sends directly to a cached
transport handle. It takes no routing mutex, walks no graph and does not parse
the encrypted payload. See [routing.md](docs/routing.md) for the architecture,
limits and reproducible forwarding microbenchmark.
This wire format requires all members to upgrade together (control ALPN 3,
data ALPN 4); saved identities, network names, secrets and addresses survive.
### LAN game discovery
IPv4 UDP broadcast relay is enabled by default for each network. The IP router
sends one encrypted copy to each participating peer, including through multihop
paths. It supports `255.255.255.255` and the overlay subnet's broadcast address;
received broadcasts never trigger another fanout.
```sh
tsng join -n games --no-broadcast
tsng join -n games --broadcast
tsng network broadcast games off
tsng network broadcast games on
```
The choice persists across restarts and a plain `join`. It can be changed while
the agent runs; `status` shows it for each network. While broadcast is on, the
agent routes `255.255.255.255` through the Tsunagi interface and allows inbound
UDP from the overlay range on it (Linux and Windows, needs the same privilege as
the interface), and removes both when it is turned off or the agent stops. Games
bound to a physical address still bypass it. Physical LAN capture/subnet sharing
is not implemented.
See [broadcast.md](docs/broadcast.md) for domain isolation and future LAN gateways.
## How peers find each other
@@ -637,8 +681,8 @@ directories by default; `--state-dir` and `--cache-dir` override them.
One state directory belongs to one live agent, enforced with a real OS file
lock rather than an existence check. That directory *is* the identity: one
agent, one device key, one interface, and as many networks as you like on it
— `tsunagi network join` adds them to the agent that is already running,
which is why a second `tsunagi up` on the same directory is refused rather
— `tsng network join` adds them to the agent that is already running,
which is why a second `tsng up` on the same directory is refused rather
than made to work. A second agent on the same host is a second identity, and
needs everything of its own: its own state and cache directories, its own
interface name and its own overlay range. The library owns no globals, so
@@ -649,9 +693,9 @@ several of them run side by side in one process as readily as in one host.
Membership outlives a session, so it also has to be possible to end it.
```bash
tsunagi network # what this device belongs to
tsunagi network leave <network-id> # give up the address and the name, then forget it
tsunagi wipe --yes # remove everything and be a stranger again
tsng network # what this device belongs to
tsng network leave lab # give up the address and the name, then forget it
tsng wipe --yes # remove everything and be a stranger again
```
**Stopping is not leaving.** `stop` closes this network's sessions, takes
@@ -679,9 +723,12 @@ nothing can sign or send it: the command says so and refuses, and `--offline`
drops the network locally while leaving the others holding the old claim. The
protocol key for that network goes too — rejoining is joining, not resuming.
A network is named by its id, never by its name: two networks can share a
name, and choosing between them for the user is how the wrong one gets left. A
unique prefix is enough.
`leave`, `stop`, `start`, `secret` and `broadcast` accept an exact stored network
name, its full id, or a unique id prefix (including a copied report ellipsis).
Names are case-sensitive. If several networks share a name, or a name matches
another network's id prefix, the command lists matching names and full ids and
changes nothing. Use a full id to choose explicitly; a full id always takes
precedence over a name. No secret is needed to select a configured network.
`wipe` removes both directories' contents: the device identity, every network,
every signed record and everything a protocol kept beside them. It refuses
@@ -699,6 +746,7 @@ networks first if the addresses should be freed.
- [docs/protocol.md](docs/protocol.md) — identity derivation, framing, handshake.
- [docs/sync-model.md](docs/sync-model.md) — the planned signed-state model and
what is deliberately not built yet.
- [docs/exit-node.md](docs/exit-node.md) — exit nodes: sending all traffic through a member.
- [docs/threat-model.md](docs/threat-model.md) — threat model and known limits.
- [docs/testing.md](docs/testing.md) — what the suite covers and what it does not.
- [AGENTS.md](AGENTS.md) — rules for anyone (human or agent) changing this repo.
@@ -716,4 +764,4 @@ any of this.
## Licence
MIT OR Apache-2.0.
[WTFPL](LICENSE): do what the fuck you want to.
-40
View File
@@ -1,40 +0,0 @@
[package]
name = "tsunagi-cli"
version = "0.1.0"
description = "The tsunagi command line agent."
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
[[bin]]
name = "tsunagi"
path = "src/main.rs"
# The binary and the library share a name, which `cargo doc` cannot map to
# two directories. The command line is documented by `--help`, not rustdoc,
# so the library keeps the name and this opts out.
doc = false
[dependencies]
tsunagi = { path = "../tsunagi", version = "0.1.0", features = ["tun-device", "dns-publish"] }
tsunagi-wg-quic = { path = "../tsunagi-wg-quic", version = "0.1.0" }
iroh.workspace = true
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros", "signal"] }
tracing.workspace = true
tracing-subscriber.workspace = true
clap = { version = "4.5", features = ["derive", "env"] }
# Already in the tree through clap. `anstream` strips the escapes when stdout
# is not a terminal and turns on virtual terminal processing on Windows, so
# colour is never written where it would show up as rubbish.
anstream = "1.0"
anstyle = "1.0"
# Local interface addresses, for the diagnostics in `status`.
netwatch = "0.19.3"
[dev-dependencies]
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros", "process"] }
tempfile.workspace = true
simple-dns = "0.12"
[lints]
workspace = true
-56
View File
@@ -1,56 +0,0 @@
//! What the command line says about this device, against a running agent.
//!
//! These drive the real binary, which is why they live beside it rather
//! than with the library's own tests: the library cannot depend on a binary
//! built from a crate that depends on it.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use tempfile::TempDir;
use tsunagi::Agent;
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
/// Asking who this device is must not need the directory lock.
///
/// The lock belongs to the one agent allowed to *write* the state. `id` only
/// reads, so making it take the lock would mean the question could never be
/// answered while an agent was running — which is exactly when you want to
/// ask it.
#[tokio::test]
async fn identity_can_be_read_while_an_agent_holds_the_directory() {
let dir = TempDir::new().unwrap();
// Hold the directory the way a running agent does.
let agent = Agent::spawn(
AgentConfig::new(StoragePaths::under(dir.path()))
.with_transport(TransportPolicy::LocalOnly)
.with_loopback_bind(),
)
.await
.unwrap();
let expected = agent.endpoint_id().to_string();
let output = tokio::process::Command::new(env!("CARGO_BIN_EXE_tsunagi"))
.arg("id")
// The same layout `StoragePaths::under` gives the agent above.
.arg("--state-dir")
.arg(dir.path().join("state"))
.arg("--cache-dir")
.arg(dir.path().join("cache"))
.output()
.await
.unwrap();
let stdout = String::from_utf8_lossy(&output.stdout);
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(
output.status.success(),
"`id` failed while an agent was running:\n{stderr}"
);
assert!(
stdout.contains(&expected),
"expected {expected} in:\n{stdout}"
);
agent.shutdown().await;
}
+113
View File
@@ -0,0 +1,113 @@
[package]
name = "tsunagi-core"
version = "0.1.0"
description = "Small private mesh networks: persistent agent identity, deterministic network spaces, an iroh control plane, signed state and a local DNS view."
readme = "../../README.md"
keywords = ["mesh", "p2p", "iroh", "networking"]
categories = ["network-programming"]
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
[lib]
name = "tsunagi"
[features]
default = []
# A real TUN device, so a plugin can carry actual IP traffic. On Linux it
# needs CAP_NET_ADMIN; on Windows the process must run elevated and
# `wintun.dll` must be reachable. Without either the in-memory device serves
# tests. The platform-specific dependencies are declared per target, so
# enabling this only pulls what the target it is built for actually uses.
tun-device = [
"dep:tun",
"dep:rtnetlink",
"dep:caps",
"dep:futures-util",
]
# Telling the operating system where to send its DNS questions. systemd-resolved
# on Linux, the Name Resolution Policy Table on Windows; the zone and the
# server work without it either way.
dns-publish = ["dep:zbus"]
# The test harness in `testing`, for this crate's own tests and for a
# protocol crate's. Not on by default: it is only of use to a test.
testing = ["dep:tempfile", "dep:tracing-subscriber"]
[dependencies]
arc-swap = "1.9"
iroh.workspace = true
# `net` for the local control interface: a Unix socket on Unix, a named pipe
# on Windows. Both live behind the same `net` feature.
tokio = { workspace = true, features = ["net"] }
tokio-stream = "0.1"
serde.workspace = true
postcard.workspace = true
bytes.workspace = true
thiserror.workspace = true
tracing.workspace = true
hex.workspace = true
data-encoding.workspace = true
rusqlite = { version = "0.40", features = ["bundled"] }
hkdf = "0.13"
hmac = "0.13"
sha2 = "0.11"
subtle = "2.6"
zeroize = { version = "1.9", features = ["derive"] }
rand = "0.10"
# Already in the tree through iroh. A packet codec, not a DNS server: the
# zone logic is ours and a full server framework would be a large dependency
# for answering A records from memory.
simple-dns = "0.12"
# The IANA top-level domain list, compiled in: one function, no
# dependencies, no network. Used only to warn that a zone name shadows a
# real public domain, never to refuse one.
tld = "2.40"
fs4 = { version = "1.1", features = ["sync"] }
directories = "6.0"
# The real system hostname, without a libc call of our own: this crate is
# forbidden `unsafe` and will not make one.
gethostname = "1.1"
netwatch = "0.19.3"
netdev = "0.46"
mainline = "8.0"
futures-lite = "2.6"
tun = { version = "0.8", features = ["async"], optional = true }
# Only for the `testing` harness, which a protocol crate's tests use too.
tempfile = { version = "3.24", optional = true }
tracing-subscriber = { version = "0.3", features = ["env-filter"], optional = true }
# Linux-only interface provisioning. `rtnetlink` configures the interface in
# process, so no `ip` invocation is ever needed; `caps` keeps CAP_NET_ADMIN
# out of the effective set except during the moments it is used.
[target.'cfg(target_os = "linux")'.dependencies]
rtnetlink = { version = "0.23", optional = true }
# Pure Rust D-Bus, no libdbus to link against. `tokio` rather than the
# default reactor, because the agent brings its own.
zbus = { version = "5.19", default-features = false, features = ["tokio"], optional = true }
caps = { version = "0.5", optional = true }
futures-util = { version = "0.3", default-features = false, optional = true }
# Windows-only interface provisioning. The Wintun adapter itself is created
# and driven through the `tun` dependency, whose safe wrapper is the only way
# in — loading `wintun.dll` is an `unsafe` call this crate forbids. `netdev`
# (already in the tree through `netwatch`) reads the interface table through a
# safe wrapper, which is how the provisioner observes an adapter by name
# before deciding what to change. Creating an adapter needs the process to be
# elevated and `wintun.dll` to be reachable at run time.
[target.'cfg(target_os = "windows")'.dependencies]
windows-sys = { version = "0.61", features = [
"Win32_Security",
"Win32_Security_Authorization",
"Win32_Foundation",
] }
[dev-dependencies]
# Its own tests use the harness it publishes.
tsunagi-core = { path = ".", features = ["testing"] }
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros", "process", "test-util"] }
tempfile.workspace = true
tracing-subscriber.workspace = true
[lints]
workspace = true
File renamed without changes.
@@ -58,6 +58,8 @@ use shutdown::Shutdown;
/// Summary of a configured network, whether or not it is running.
#[derive(Debug, Clone)]
pub struct ConfiguredNetwork {
/// Saved local broadcast participation.
pub broadcast: bool,
/// Public network identifier.
pub network_id: NetworkId,
/// Network name.
@@ -279,11 +281,19 @@ impl Agent {
// The data plane rides on iroh too, which is where it gets hole
// punching and relay fallback from. It is a separate ALPN and a
// separate connection, so the two planes stay independent.
let transport: Arc<dyn PacketTransport> = Arc::new(IrohTransport::new(
let iroh_transport: Arc<dyn PacketTransport> = Arc::new(IrohTransport::new(
inner.adapter.clone(),
Arc::clone(&inner.limits),
Arc::new(TransportCtx(Arc::downgrade(&inner))) as Arc<dyn TransportContext>,
));
let transport: Arc<dyn PacketTransport> = if inner.config.custom_transports.is_empty() {
iroh_transport
} else {
Arc::new(crate::dataplane::MultiPacketTransport::new(
iroh_transport,
inner.config.custom_transports.clone(),
))
};
let _ = inner.transport.set(transport);
if let CacheOutcome::Reset(reason) = inner.storage.cache_outcome().clone() {
@@ -344,6 +354,26 @@ impl Agent {
self.inner.adapter.loopback_addr()
}
/// Delivers an inbound data plane link to the matching network runtime.
pub async fn install_inbound_link(
&self,
inbound: crate::dataplane::transport::InboundLink,
) -> Result<()> {
let network = inbound.network;
let sender = {
let networks = self.inner.networks.read().await;
networks.get(&network).map(|handle| handle.commands.clone())
};
let Some(sender) = sender else {
return Err(Error::NetworkNotActive(network));
};
sender
.send(NetCommand::InboundLink(Box::new(inbound)))
.await
.map_err(|_| Error::NetworkNotActive(network))?;
Ok(())
}
/// Reserves the configured overlay range for a network, if it can.
///
/// Done here, where activations are serialised, rather than inside the
@@ -355,12 +385,11 @@ impl Agent {
/// waits to adopt whatever it settles on. One agent has one interface, so
/// proposing a range it could not route would be worse than having none:
/// the lowest author's range wins, and the collision would spread.
fn reserve_range(&self, network: NetworkId) -> RangePlan {
let Some(configured) = self.inner.config.overlay_ipv4_range else {
return RangePlan::default();
};
async fn reserve_range(&self, network: NetworkId) -> RangePlan {
let reserve = |wanted: Ipv4Range| {
let reservation = crate::overlay::NetworkRoutes {
exit: Default::default(),
broadcast: Default::default(),
range: Some(wanted),
local: None,
peers: Vec::new(),
@@ -368,6 +397,44 @@ impl Agent {
self.inner.routes.set_network(network, reservation).is_ok()
};
// If this network already has signed records in storage, check if its
// members have already agreed on a range. If so, preserve and reserve
// THAT range rather than preemptively seizing the default configured range.
let stored_records = self
.inner
.storage
.signed_records(network)
.await
.unwrap_or_default();
let mut state_set = crate::state::StateSet::new();
state_set.merge_all(network, stored_records);
if let Some(agreed) = state_set.agreed_range() {
if reserve(agreed) {
return RangePlan {
propose: if Some(agreed) == self.inner.config.overlay_ipv4_range {
Some(agreed)
} else {
None
},
..RangePlan::default()
};
} else {
tracing::warn!(
%network,
range = %agreed,
"agreed range for this network overlaps another network already active on this interface"
);
return RangePlan {
conflict: Some(agreed),
..RangePlan::default()
};
}
}
let Some(configured) = self.inner.config.overlay_ipv4_range else {
return RangePlan::default();
};
// The configured range first, so a network a device has always had
// keeps the addresses it has always had.
if reserve(configured) {
@@ -415,6 +482,7 @@ impl Agent {
mtu: interface.mtu(),
addresses: interface.wanted_addresses(),
counters: interface.counters(),
broadcast_rules: interface.broadcast_rules(),
})
}
@@ -488,6 +556,16 @@ impl Agent {
&self,
name: &NetworkName,
secret: &NetworkSecret,
) -> Result<NetworkId> {
self.join_network_with_broadcast(name, secret, None).await
}
/// Joins with an explicit broadcast choice; absent preserves saved policy.
pub async fn join_network_with_broadcast(
&self,
name: &NetworkName,
secret: &NetworkSecret,
broadcast: Option<bool>,
) -> Result<NetworkId> {
let keys = NetworkKeys::derive(name, secret);
let network_id = keys.network_id();
@@ -504,7 +582,7 @@ impl Agent {
"`{name}` is already configured with a different secret, as {}. \
Joining with this one adds a second network under the same name; \
they share nothing. Check the secret, or use \
`tsunagi network secret` \
`tsng network secret` \
to see which is which.",
other.network_id
);
@@ -516,6 +594,9 @@ impl Agent {
.storage
.upsert_network(network_id, name.clone(), secret.clone(), true)
.await?;
if let Some(enabled) = broadcast {
self.set_broadcast(network_id, enabled).await?;
}
match self.activate_with_keys(keys).await {
// Already a member of exactly this network space: nothing to do.
Ok(()) | Err(Error::NetworkAlreadyActive(_)) => Ok(network_id),
@@ -523,6 +604,87 @@ impl Agent {
}
}
/// Offers this agent as an exit node in a network, or stops, now and
/// after restart. Off unless somebody turns it on.
pub async fn set_exit_offer(&self, network_id: NetworkId, enabled: bool) -> Result<()> {
self.inner
.storage
.set_exit_node(network_id, enabled)
.await?;
if self.is_active(network_id).await {
let (reply, receive) = oneshot::channel();
self.command(network_id, NetCommand::SetExitOffer { enabled, reply })
.await?;
receive.await.map_err(|_| Error::Stopped)?;
}
Ok(())
}
/// Sends this device's internet traffic through a member of a network,
/// or back the ordinary way with `None`.
///
/// A device has one default route, so choosing in one network clears the
/// choice in the others. The member has to be connected and offering
/// to be an exit node. Remembered across restarts.
pub async fn set_exit_via(&self, network_id: NetworkId, via: Option<EndpointId>) -> Result<()> {
let active = self.is_active(network_id).await;
if via.is_some() && !active {
return Err(Error::NetworkNotActive(network_id));
}
if active {
let (reply, receive) = oneshot::channel();
self.command(network_id, NetCommand::SetExitVia { via, reply })
.await?;
receive
.await
.map_err(|_| Error::Stopped)?
.map_err(|reason| Error::Rejected { reason })?;
}
self.inner
.storage
.set_exit_via(network_id, via.map(|peer| *peer.as_bytes()))
.await?;
if via.is_some() {
// One default route: whatever the other networks were doing, they
// are not any more.
let others: Vec<NetworkId> = self
.inner
.networks
.read()
.await
.keys()
.copied()
.filter(|other| *other != network_id)
.collect();
for other in others {
let (reply, receive) = oneshot::channel();
if self
.command(other, NetCommand::SetExitVia { via: None, reply })
.await
.is_ok()
{
let _ = receive.await;
}
}
}
Ok(())
}
/// Changes a network's local broadcast policy now and after restart.
pub async fn set_broadcast(&self, network_id: NetworkId, enabled: bool) -> Result<()> {
self.inner
.storage
.set_broadcast(network_id, enabled)
.await?;
if self.is_active(network_id).await {
let (reply, receive) = oneshot::channel();
self.command(network_id, NetCommand::SetBroadcast { enabled, reply })
.await?;
receive.await.map_err(|_| Error::Stopped)?;
}
Ok(())
}
/// Activates a configured network that is currently inactive.
///
/// Fails with [`Error::NetworkAlreadyActive`] if it is already running.
@@ -551,7 +713,7 @@ impl Agent {
return Err(Error::NetworkAlreadyActive(network_id));
}
let range = self.reserve_range(network_id);
let range = self.reserve_range(network_id).await;
let mut backends = Vec::new();
if let Some(discovery) = &self.inner.config.discovery {
backends.push(discovery.clone());
@@ -567,7 +729,23 @@ impl Agent {
as Arc<dyn crate::discovery::NetworkDiscovery>,
)
};
let stored = self
.inner
.storage
.list_networks()
.await?
.into_iter()
.find(|stored| stored.network_id == network_id);
let broadcast = stored.as_ref().is_none_or(|stored| stored.broadcast);
let exit_offer = stored.as_ref().is_some_and(|stored| stored.exit_node);
let exit_via = stored
.as_ref()
.and_then(|stored| stored.exit_via)
.and_then(|bytes| EndpointId::from_bytes(&bytes).ok());
let handle = network::spawn(RuntimeParams {
broadcast,
exit_offer,
exit_via,
keys,
adapter: self.inner.adapter.clone(),
storage: self.inner.storage.clone(),
@@ -627,6 +805,10 @@ impl Agent {
{
tracing::warn!(%err, "cannot update the overlay addresses");
}
if let Some(interface) = self.inner.interface.get() {
interface.sync_broadcast_rules().await;
interface.sync_exit_rules().await;
}
self.inner.storage.set_auto_start(network_id, false).await?;
Ok(())
}
@@ -703,6 +885,7 @@ impl Agent {
.await?
.into_iter()
.map(|stored| ConfiguredNetwork {
broadcast: stored.broadcast,
network_id: stored.network_id,
name: stored.name,
auto_start: stored.auto_start,
@@ -777,6 +960,13 @@ impl Agent {
}
let keys = NetworkKeys::derive(&stored.name, &stored.secret);
networks.push(NetworkStatus {
broadcast: stored.broadcast,
exit_offer: stored.exit_node,
exit_via: stored
.exit_via
.and_then(|bytes| EndpointId::from_bytes(&bytes).ok()),
exit_via_online: false,
exit_rules: Default::default(),
descriptor: keys.descriptor(),
name: stored.name,
network_id: stored.network_id,
@@ -1013,6 +1203,26 @@ async fn plugin_request_loop(weak: Weak<Inner>, mut requests: mpsc::Receiver<Plu
}
}
}
PluginRequest::ProtocolFailure {
network,
peer,
protocol,
reason,
} => {
let sender = {
let networks = inner.networks.read().await;
networks.get(&network).map(|handle| handle.commands.clone())
};
if let Some(sender) = sender {
let _ = sender
.send(NetCommand::ProtocolFailure {
peer,
protocol,
reason,
})
.await;
}
}
}
}
}
@@ -1158,8 +1368,23 @@ async fn handle_inbound_data(inner: Arc<Inner>, conn: iroh::endpoint::Connection
conn.close(5u32.into(), b"data plane not ready");
return;
};
// Downcasting is avoided by keeping the accept side on the concrete type.
let Some(iroh_transport) = transport.as_ref().as_any().downcast_ref::<IrohTransport>() else {
let iroh_transport =
if let Some(iroh) = transport.as_ref().as_any().downcast_ref::<IrohTransport>() {
Some(iroh)
} else if let Some(multi) = transport
.as_ref()
.as_any()
.downcast_ref::<crate::dataplane::MultiPacketTransport>()
{
multi
.default_transport()
.as_ref()
.as_any()
.downcast_ref::<IrohTransport>()
} else {
None
};
let Some(iroh_transport) = iroh_transport else {
conn.close(5u32.into(), b"unsupported data transport");
return;
};
@@ -62,6 +62,20 @@ pub(crate) enum NetCommand {
reply: oneshot::Sender<Box<NetworkStatus>>,
},
Recheck,
SetBroadcast {
enabled: bool,
reply: oneshot::Sender<()>,
},
/// Offer this agent as an exit node in this network, or stop.
SetExitOffer {
enabled: bool,
reply: oneshot::Sender<()>,
},
/// Send this device's internet traffic through a member, or stop.
SetExitVia {
via: Option<EndpointId>,
reply: oneshot::Sender<std::result::Result<(), String>>,
},
/// Resend this agent's announcement to every peer of this network.
Reannounce,
/// Answer to a different name from now on.
@@ -82,6 +96,12 @@ pub(crate) enum NetCommand {
/// Human readable reason, free of secrets.
reason: String,
},
/// An IP plugin reported that data plane connectivity or a tunnel for a peer failed.
ProtocolFailure {
peer: EndpointId,
protocol: String,
reason: String,
},
}
impl std::fmt::Debug for NetCommand {
@@ -95,10 +115,16 @@ impl std::fmt::Debug for NetCommand {
NetCommand::Broadcast { message, .. } => write!(f, "Broadcast({})", kind(message)),
NetCommand::Status { .. } => f.write_str("Status"),
NetCommand::Recheck => f.write_str("Recheck"),
NetCommand::SetBroadcast { enabled, .. } => write!(f, "SetBroadcast({enabled})"),
NetCommand::SetExitOffer { enabled, .. } => write!(f, "SetExitOffer({enabled})"),
NetCommand::SetExitVia { via, .. } => write!(f, "SetExitVia({})", via.is_some()),
NetCommand::Reannounce => f.write_str("Reannounce"),
NetCommand::SetHostname(_) => f.write_str("SetHostname"),
NetCommand::Release { .. } => f.write_str("Release"),
NetCommand::PluginError { protocol, .. } => write!(f, "PluginError({protocol})"),
NetCommand::ProtocolFailure { peer, protocol, .. } => {
write!(f, "ProtocolFailure({}, {protocol})", peer.fmt_short())
}
}
}
}
@@ -122,6 +148,11 @@ impl NetworkHandle {
/// Everything a network runtime needs to run.
pub(crate) struct RuntimeParams {
pub(crate) broadcast: bool,
/// This agent offers itself as an exit node here.
pub(crate) exit_offer: bool,
/// The member this device's internet traffic goes through.
pub(crate) exit_via: Option<EndpointId>,
pub(crate) keys: NetworkKeys,
pub(crate) adapter: EndpointAdapter,
pub(crate) storage: Storage,
@@ -233,6 +264,24 @@ const REACH_EXPIRY: Duration = Duration::from_secs(90);
/// can arrive and be adopted instead.
const RANGE_PROPOSAL_GRACE: Duration = Duration::from_secs(3);
#[derive(Debug, Clone)]
struct ProtocolDialState {
failures: u32,
cooldown_until: std::time::Instant,
}
const DATA_LINK_BASE_BACKOFF: Duration = Duration::from_secs(3);
const DATA_LINK_MAX_BACKOFF: Duration = Duration::from_secs(60);
fn data_link_backoff(failures: u32) -> Duration {
if failures == 0 {
return Duration::ZERO;
}
let shift = (failures - 1).min(5);
let secs = (DATA_LINK_BASE_BACKOFF.as_secs() << shift).min(DATA_LINK_MAX_BACKOFF.as_secs());
Duration::from_secs(secs)
}
struct Runtime {
discovery_worker: Option<DiscoveryWorker>,
discovery_candidates: mpsc::Receiver<Candidate>,
@@ -264,14 +313,22 @@ struct Runtime {
/// else, so there is no hearsay to weigh and nothing to poison. Soft
/// state — an entry that stops being refreshed expires, which is how a
/// relay that has gone stops being chosen without anybody revoking it.
reachable: HashMap<EndpointId, (HashSet<EndpointId>, std::time::Instant)>,
reachable: HashMap<
EndpointId,
(
HashSet<crate::proto::message::ReachableLink>,
std::time::Instant,
),
>,
/// What this agent last told its peers it could reach, so it is only
/// said again when it changed.
announced_reach: HashSet<EndpointId>,
announced_reach: HashSet<crate::proto::message::ReachableLink>,
/// Links currently being opened, so we do not start two.
opening: HashSet<(EndpointId, String)>,
link_results_tx: mpsc::Sender<LinkOutcome>,
link_results_rx: mpsc::Receiver<LinkOutcome>,
/// Protocol dial failure count and backoff cooldown per peer and protocol.
protocol_dial_states: HashMap<(EndpointId, String), ProtocolDialState>,
/// Peers already told about a protocol version that cannot match, so it
/// is said once rather than on every announcement.
reported_mismatch: HashSet<(EndpointId, String)>,
@@ -290,6 +347,14 @@ struct Runtime {
pending_state: Vec<(EndpointId, Vec<SignedRecord>)>,
/// The highest version this agent has ever published for this network.
own_version: u64,
/// Whether the exit node in use was reachable when last looked at, so
/// that losing it is said once and getting it back is too.
exit_via_was_online: bool,
/// What the last announcement said about being an exit node.
announced_exit: bool,
/// The last hostname each peer announced, kept across restarts so a
/// member that is away can still be called by name.
known_hostnames: HashMap<EndpointId, String>,
}
impl Runtime {
@@ -328,18 +393,22 @@ impl Runtime {
dial_results_tx,
dial_results_rx,
links: HashMap::new(),
hub: crate::dataplane::relay::RelayHub::new(network_id),
hub: crate::dataplane::relay::RelayHub::new(network_id, local_id),
reachable: HashMap::new(),
announced_reach: HashSet::new(),
opening: HashSet::new(),
link_results_tx,
link_results_rx,
protocol_dial_states: HashMap::new(),
reported_mismatch: HashSet::new(),
released: false,
reported_missing: None,
state: StateSet::new(),
pending_state: Vec::new(),
own_version: 0,
exit_via_was_online: false,
announced_exit: false,
known_hostnames: HashMap::new(),
}
}
@@ -360,6 +429,11 @@ impl Runtime {
tokio::select! {
biased;
_ = self.shutdown.wait() => break,
_ = self.hub.changed() => {
self.ensure_links();
self.update_paths();
self.announce_reach(false);
}
command = commands.recv() => match command {
Some(command) => self.handle_command(command).await,
None => break,
@@ -387,6 +461,8 @@ impl Runtime {
_ = ticker.tick() => {
self.discovery_round().await;
self.ensure_links();
self.update_paths();
self.announce_reach(false);
}
}
}
@@ -401,7 +477,9 @@ impl Runtime {
// Stop accepting session events first: nothing is going to act on them
// any more, and a sender blocked on a full queue would stall shutdown.
self.session_events_rx.close();
self.hub.close();
self.links.clear();
self.protocol_dial_states.clear();
let peers: Vec<EndpointId> = self.sessions.keys().copied().collect();
for peer in peers {
if let Some(session) = self.sessions.remove(&peer) {
@@ -441,6 +519,36 @@ impl Runtime {
NetCommand::Status { reply } => {
let _ = reply.send(Box::new(self.status()));
}
NetCommand::SetBroadcast { enabled, reply } => {
self.params.broadcast = enabled;
self.update_broadcast();
if let Some(interface) = self.params.interface.clone() {
interface.sync_broadcast_rules().await;
}
self.reannounce();
let _ = reply.send(());
}
NetCommand::SetExitOffer { enabled, reply } => {
self.params.exit_offer = enabled;
self.apply_exit().await;
self.reannounce();
let _ = reply.send(());
}
NetCommand::SetExitVia { via, reply } => {
let outcome = match via {
Some(peer) if !self.exit_node_online(peer) => Err(
"that member is not connected or does not offer to be an exit node"
.to_string(),
),
_ => Ok(()),
};
if outcome.is_ok() {
self.params.exit_via = via;
self.exit_via_was_online = via.is_some();
self.apply_exit().await;
}
let _ = reply.send(outcome);
}
NetCommand::Recheck => self.discovery_round().await,
NetCommand::Reannounce => self.reannounce(),
NetCommand::Release { reply } => {
@@ -474,6 +582,26 @@ impl Runtime {
reason,
});
}
NetCommand::ProtocolFailure {
peer,
protocol,
reason,
} => {
if self.links.remove(&(peer, protocol.clone())).is_some() {
self.hub.clear_direct(peer, &protocol);
}
self.metrics.data_link_failures += 1;
self.record_protocol_failure(peer, &protocol);
self.emit(Event::DataLinkDown {
network: self.network_id,
peer,
protocol,
reason,
});
self.ensure_links();
self.update_paths();
self.announce_reach(false);
}
}
}
@@ -764,6 +892,17 @@ impl Runtime {
.own_record_version(self.network_id, self.local_id)
.await
.unwrap_or(0);
for (id, hostname) in self
.params
.storage
.peer_hostnames(self.network_id)
.await
.unwrap_or_default()
{
if let Ok(id) = EndpointId::from_bytes(&id) {
self.known_hostnames.insert(id, hostname);
}
}
self.ensure_own_claim().await;
self.publish_allocations().await;
@@ -1003,6 +1142,93 @@ impl Runtime {
}
}
fn broadcast_policy(&self) -> crate::overlay::broadcast::BroadcastPolicy {
crate::overlay::broadcast::BroadcastPolicy {
enabled: self.params.broadcast,
peers: self
.sessions
.values()
.filter(|session| session.broadcast)
.map(|session| session.peer)
.collect(),
}
}
fn exit_policy(&self) -> crate::overlay::ExitPolicy {
crate::overlay::ExitPolicy {
offer: self.params.exit_offer,
via: self.params.exit_via,
}
}
/// Whether this agent offers to be an exit node and can really be one.
fn exit_ready(&self) -> bool {
self.params.exit_offer
&& self
.params
.interface
.as_ref()
.is_some_and(|interface| interface.exit_offer_ready(self.network_id))
}
/// Whether a member is connected and says it is an exit node.
fn exit_node_online(&self, peer: EndpointId) -> bool {
self.sessions
.get(&peer)
.is_some_and(|session| session.exit_node)
}
/// Puts the exit settings into the routing table and the host rules.
///
/// The routes and the rules stay as they are when the exit node in use
/// goes away: dropping them would send the traffic out the old way, in
/// the clear, without anyone having asked.
async fn apply_exit(&mut self) {
self.params
.routes
.set_exit(self.network_id, self.exit_policy());
if let Some(interface) = self.params.interface.clone() {
interface.sync_exit_rules().await;
}
// Said only once it is true: the rules may not have taken, or may
// only now be possible.
if self.exit_ready() != self.announced_exit {
self.reannounce();
}
}
/// Says so, once, when the exit node in use is lost or comes back.
fn watch_exit_via(&mut self) {
let Some(via) = self.params.exit_via else {
return;
};
let online = self.exit_node_online(via);
if online == self.exit_via_was_online {
return;
}
self.exit_via_was_online = online;
if online {
tracing::info!(
network = %self.network_id,
exit_node = %via.fmt_short(),
"the exit node is back"
);
} else {
tracing::warn!(
network = %self.network_id,
exit_node = %via.fmt_short(),
"the exit node is gone or no longer offers to be one; internet traffic stays \
routed through it and is dropped until it returns or `tsng exit-node off`"
);
}
}
fn update_broadcast(&self) {
self.params
.routes
.set_broadcast(self.network_id, self.broadcast_policy());
}
/// Tells the plugins who holds which overlay address.
async fn publish_allocations(&mut self) {
let Some(range) = self.effective_range() else {
@@ -1025,6 +1251,8 @@ impl Runtime {
// a packet for ourselves does not go over a tunnel.
let local = self.state.address_of(&self.local_id);
let routes = crate::overlay::NetworkRoutes {
exit: self.exit_policy(),
broadcast: self.broadcast_policy(),
range: Some(range),
local,
peers: allocations
@@ -1065,19 +1293,40 @@ impl Runtime {
// wrong here before. Said once per address, not every round.
// An in-memory interface has no addresses to be missing from: with
// `--no-tun` this is the arrangement asked for, not a fault.
let missing = match local {
Some(address)
if interface.on_host()
&& !crate::overlay::address_is_local(std::net::IpAddr::V4(address)) =>
{
let is_missing = if let Some(address) = local {
if interface.on_host() {
let mut local_found =
crate::overlay::address_is_local(std::net::IpAddr::V4(address));
// On Windows, duplicate address detection (DAD) and netsh/IP helper updates
// may take a short moment before a newly assigned address is ready for binding.
if !local_found && cfg!(target_os = "windows") {
for _ in 0..10 {
tokio::time::sleep(std::time::Duration::from_millis(150)).await;
if crate::overlay::address_is_local(std::net::IpAddr::V4(address)) {
local_found = true;
break;
}
}
}
!local_found
} else {
false
}
} else {
false
};
let missing = if is_missing {
if let Some(address) = local {
let already = self.reported_missing == Some(address);
self.reported_missing = Some(address);
(!already).then_some(address)
}
_ => {
self.reported_missing = None;
} else {
None
}
} else {
self.reported_missing = None;
None
};
if let Some(address) = missing {
self.emit(Event::PluginError {
@@ -1094,6 +1343,13 @@ impl Runtime {
),
});
}
// The broadcast route names this address as its source, so it goes in
// only once the address is really on the interface.
if !is_missing {
interface.sync_broadcast_rules().await;
self.apply_exit().await;
}
}
// ------------------------------------------------------------ data plane
@@ -1107,6 +1363,25 @@ impl Runtime {
.collect()
}
fn record_protocol_failure(&mut self, peer: EndpointId, protocol: &str) {
let key = (peer, protocol.to_string());
let entry = self
.protocol_dial_states
.entry(key)
.or_insert(ProtocolDialState {
failures: 0,
cooldown_until: std::time::Instant::now(),
});
entry.failures = entry.failures.saturating_add(1);
let backoff = data_link_backoff(entry.failures);
entry.cooldown_until = std::time::Instant::now() + backoff;
}
fn record_protocol_success(&mut self, peer: EndpointId, protocol: &str) {
self.protocol_dial_states
.remove(&(peer, protocol.to_string()));
}
/// Opens whatever data plane links are missing, and forgets dead ones.
///
/// Only one side dials, chosen by a rule both sides compute the same way,
@@ -1122,7 +1397,7 @@ impl Runtime {
let mut dead: Vec<(EndpointId, String)> = Vec::new();
for (key, link) in &self.links {
if link.is_closed() {
if link.is_closed() || self.no_direct_data(key.0) {
dead.push(key.clone());
}
}
@@ -1132,6 +1407,7 @@ impl Runtime {
// reachable through somebody is decided below; the protocol's
// link stays either way.
self.hub.clear_direct(peer, &protocol);
self.record_protocol_failure(peer, &protocol);
self.emit(Event::DataLinkDown {
network: self.network_id,
peer,
@@ -1140,32 +1416,17 @@ impl Runtime {
});
}
// Both the name and the version have to match. A peer offering
// `wg-quic` at a version this build does not speak is not a peer to
// carry traffic with, and a link opened anyway would fail later and
// say less about why.
let wanted: Vec<(EndpointId, String)> = self
.sessions
.values()
.flat_map(|session| {
let peer = session.peer;
session
.capabilities
.iter()
.filter(|capability| capability.enabled)
.map(move |capability| (peer, capability.protocol.clone(), capability.version))
})
.filter(|(_, protocol, version)| {
served
.iter()
.any(|(name, ours)| name == protocol && ours == version)
})
.map(|(peer, protocol, _)| (peer, protocol))
.collect();
for (peer, protocol) in wanted {
let key = (peer, protocol.clone());
if self.links.contains_key(&key) || self.opening.contains(&key) {
// Choose the highest priority mutually supported protocol for each peer.
// If a direct data link is already open or opening, do not dial another.
// If a protocol recently failed, fall back to other mutually supported
// protocols according to backoff cooldown.
let now = std::time::Instant::now();
let mut wanted: Vec<(EndpointId, String)> = Vec::new();
for session in self.sessions.values() {
let peer = session.peer;
if self.links.keys().any(|(p, _)| *p == peer)
|| self.opening.iter().any(|(p, _)| *p == peer)
{
continue;
}
if self.no_direct_data(peer) {
@@ -1176,6 +1437,39 @@ impl Runtime {
if self.local_id.as_bytes() >= peer.as_bytes() {
continue;
}
let supported: Vec<String> = served
.iter()
.filter(|(name, ours)| {
session
.capabilities
.iter()
.any(|c| c.enabled && c.protocol == *name && c.version == *ours)
})
.map(|(name, _)| name.clone())
.collect();
if supported.is_empty() {
continue;
}
let chosen = supported.iter().find(|proto| {
match self.protocol_dial_states.get(&(peer, (*proto).clone())) {
Some(state) => state.cooldown_until <= now,
None => true,
}
});
if let Some(proto) = chosen {
wanted.push((peer, proto.clone()));
}
}
for (peer, protocol) in wanted {
let key = (peer, protocol.clone());
if self.links.contains_key(&key) || self.opening.contains(&key) {
continue;
}
self.opening.insert(key);
let results = self.link_results_tx.clone();
@@ -1307,28 +1601,28 @@ impl Runtime {
}
}
/// Tells peers which peers this agent has a live link with, when that
/// has changed.
///
/// Only what it can see for itself, and only to the peers it is
/// talking to: one hop, no flooding, nothing second-hand. A peer that
/// hears this is the only one that can use it, because it is also the
/// only one that could route through this agent.
/// Publishes only locally observed, protocol-specific transport edges.
fn announce_reach(&mut self, force: bool) {
let live: HashSet<EndpointId> = self
let live: HashSet<_> = self
.links
.iter()
.filter(|(_, link)| !link.is_closed())
.map(|((peer, _), _)| *peer)
.map(
|((peer, protocol), _)| crate::proto::message::ReachableLink {
peer: *peer.as_bytes(),
protocol: protocol.clone(),
},
)
.collect();
if !force && live == self.announced_reach {
return;
}
self.announced_reach = live.clone();
let message = ControlMessage::Reachable {
peers: live.iter().map(|peer| *peer.as_bytes()).collect(),
};
let peers: Vec<EndpointId> = self.sessions.keys().copied().collect();
let mut links: Vec<_> = live.into_iter().collect();
links.sort_by(|a, b| (&a.protocol, a.peer).cmp(&(&b.protocol, b.peer)));
links.truncate(self.params.limits.max_state_records);
let message = ControlMessage::Reachable { links };
let peers: Vec<_> = self.sessions.keys().copied().collect();
for peer in peers {
if let Err(err) = self.send_to(peer, message.clone()) {
tracing::debug!(%err, "could not queue a reachability announcement");
@@ -1336,99 +1630,72 @@ impl Runtime {
}
}
/// Chooses, for every peer with no direct link, somebody to go through.
///
/// Local and deterministic: the lowest endpoint id among the peers
/// that both this agent has a link with and that say they have a link
/// with the destination. Nothing is agreed with anybody — two agents
/// may well pick differently, and traffic each way finds its own path.
/// Builds the graph off the packet path. Every remote row comes from that
/// member's authenticated control session; stale or incompatible rows never
/// enter the graph. The hub replaces an immutable table only on changes.
fn update_paths(&mut self) {
self.reachable
.retain(|_, (_, heard)| heard.elapsed() < REACH_EXPIRY);
let served = self.served_protocols();
let live: HashSet<EndpointId> = self
.links
.iter()
.filter(|(_, link)| !link.is_closed())
.map(|((peer, _), _)| *peer)
.collect();
// Every peer this agent would carry traffic with: a session says
// who it is and what it speaks, which is also what a protocol
// needs before it can have a link at all.
let wanted: Vec<(EndpointId, String)> = self
.sessions
.values()
.flat_map(|session| {
let peer = session.peer;
session
.capabilities
.iter()
.filter(|capability| capability.enabled)
.map(move |capability| (peer, capability.protocol.clone(), capability.version))
})
.filter(|(_, protocol, version)| {
served
.iter()
.any(|(name, ours)| name == protocol && ours == version)
})
.map(|(peer, protocol, _)| (peer, protocol))
.collect();
for (peer, protocol) in wanted {
if live.contains(&peer) {
// There is a direct link; a hop is only for when there is
// not, and holding a stale one would keep a peer that has
// gone looking reachable.
if self.hub.has_link(peer, &protocol) {
self.hub.set_hop(peer, &protocol, None);
}
continue;
}
let hop = self
self.reachable.retain(|peer, (_, heard)| {
heard.elapsed() < REACH_EXPIRY && self.sessions.contains_key(peer)
});
for (protocol, version) in self.served_protocols() {
let peers: Vec<_> = self
.sessions
.values()
.filter(|session| {
session.capabilities.iter().any(|capability| {
capability.enabled
&& capability.protocol == protocol
&& capability.version == version
})
})
.map(|session| session.peer)
.collect();
let mut members: HashSet<_> = peers.iter().map(|peer| *peer.as_bytes()).collect();
members.insert(*self.local_id.as_bytes());
let graph = self
.reachable
.iter()
.filter(|(middle, (reaches, _))| {
live.contains(*middle) && reaches.contains(&peer) && **middle != peer
.filter(|(peer, _)| members.contains(peer.as_bytes()))
.map(|(peer, (links, _))| {
let mut neighbors: Vec<_> = links
.iter()
.filter(|link| {
link.protocol == protocol
&& members.contains(&link.peer)
&& link.peer != *peer.as_bytes()
})
.map(|link| link.peer)
.collect();
neighbors.sort_unstable();
(*peer.as_bytes(), neighbors)
})
.map(|(middle, _)| *middle)
.min_by(|a, b| a.as_bytes().cmp(b.as_bytes()));
let known = self.hub.has_link(peer, &protocol);
if !known && hop.is_none() {
// Nothing to reach it with and nothing to offer a
// protocol; inventing a link object here would only look
// like connectivity.
continue;
.collect();
self.hub.set_topology(&protocol, graph, members);
for peer in peers {
if self.hub.has_link(peer, &protocol) || !self.hub.reachable(peer, &protocol) {
continue;
}
let Some(plugin) = self
.params
.plugins
.iter()
.find(|plugin| plugin.protocol_id() == protocol)
.cloned()
else {
continue;
};
let link = self.hub.link(peer, &protocol);
let path = link.path_description();
let max_datagram = link.max_datagram_size();
plugin.on_peer_link(self.network_id, peer, link);
self.emit(Event::DataLinkUp {
network: self.network_id,
peer,
protocol: protocol.clone(),
path,
max_datagram,
});
}
self.hub.set_hop(peer, &protocol, hop);
if known {
continue;
}
// A peer reachable only through somebody still gets a link, so
// its tunnel can be built: the protocol is not told how its
// datagrams travel, only that this is the way to that peer.
let Some(plugin) = self
.params
.plugins
.iter()
.find(|plugin| plugin.protocol_id() == protocol)
.cloned()
else {
continue;
};
let link = self.hub.link(peer, &protocol);
let path = link.path_description();
let max_datagram = link.max_datagram_size();
plugin.on_peer_link(self.network_id, peer, link);
self.emit(Event::DataLinkUp {
network: self.network_id,
peer,
protocol,
path,
max_datagram,
});
}
}
@@ -1437,6 +1704,7 @@ impl Runtime {
self.opening.remove(&key);
match outcome.result {
Ok(link) => {
self.record_protocol_success(outcome.peer, &outcome.protocol);
self.adopt_link(outcome.peer, outcome.protocol, link);
// A new link is both a path for us and one we can offer
// others, so it changes what we route and what we say.
@@ -1446,12 +1714,15 @@ impl Runtime {
Err(reason) => {
// A data plane that cannot be set up is reported, never fatal.
self.metrics.data_link_failures += 1;
self.record_protocol_failure(outcome.peer, &outcome.protocol);
self.emit(Event::DataLinkDown {
network: self.network_id,
peer: outcome.peer,
protocol: outcome.protocol,
reason,
});
// Fall back to other mutually supported protocols immediately.
self.ensure_links();
}
}
}
@@ -1479,6 +1750,7 @@ impl Runtime {
if self.no_direct_data(inbound.peer) {
return;
}
self.record_protocol_success(inbound.peer, &inbound.protocol);
self.adopt_link(inbound.peer, inbound.protocol, inbound.link);
self.update_paths();
self.announce_reach(false);
@@ -1526,6 +1798,7 @@ impl Runtime {
// protocol's link goes with it.
self.hub.remove_peer(peer);
self.reachable.remove(&peer);
self.protocol_dial_states.retain(|(p, _), _| *p != peer);
let keys: Vec<(EndpointId, String)> = self
.links
.keys()
@@ -1640,7 +1913,11 @@ impl Runtime {
});
}
capabilities.truncate(self.params.limits.max_capabilities);
let exit_node = self.exit_ready();
self.announced_exit = exit_node;
Announcement {
broadcast: self.params.broadcast,
exit_node,
hostname: self.params.hostname.clone(),
capabilities,
}
@@ -1718,6 +1995,10 @@ impl Runtime {
}
self.metrics.disconnects += 1;
self.drop_links_for(peer);
self.update_broadcast();
self.watch_exit_via();
self.update_paths();
self.announce_reach(false);
for plugin in &self.params.plugins {
plugin.on_peer_gone(self.network_id, peer);
}
@@ -1738,16 +2019,49 @@ impl Runtime {
}
}
/// Keeps the name a peer announced, on disk too when it changed.
///
/// Written off the loop: a name that is not stored this time is stored
/// the next time the peer announces it.
fn remember_hostname(&mut self, peer: EndpointId, hostname: &str) {
if hostname.is_empty()
|| self
.known_hostnames
.get(&peer)
.is_some_and(|h| h == hostname)
{
return;
}
self.known_hostnames.insert(peer, hostname.to_string());
let storage = self.params.storage.clone();
let network_id = self.network_id;
let hostname = hostname.to_string();
tokio::spawn(async move {
if let Err(err) = storage
.remember_peer_hostname(network_id, *peer.as_bytes(), hostname)
.await
{
tracing::debug!(%err, "could not remember a peer's hostname");
}
});
}
fn dispatch_message(&mut self, peer: EndpointId, message: ControlMessage) {
match &message {
ControlMessage::Announce(announcement) => {
let capabilities = announcement.capabilities.clone();
if let Some(session) = self.sessions.get_mut(&peer) {
session.hostname = Some(announcement.hostname.clone());
session.broadcast = announcement.broadcast;
session.exit_node = announcement.exit_node;
session.capabilities = capabilities.clone();
}
self.remember_hostname(peer, &announcement.hostname);
self.update_broadcast();
self.watch_exit_via();
self.dispatch_capabilities(peer, &capabilities);
self.ensure_links();
self.update_paths();
}
ControlMessage::Ping { seq, payload } => {
let pong = ControlMessage::Pong {
@@ -1772,11 +2086,8 @@ impl Runtime {
}
// What that peer can reach, from that peer. Kept with the time
// it was heard, so it can go stale on its own.
ControlMessage::Reachable { peers } => {
let heard: HashSet<EndpointId> = peers
.iter()
.filter_map(|raw| EndpointId::from_bytes(raw).ok())
.collect();
ControlMessage::Reachable { links } => {
let heard = links.iter().cloned().collect();
self.reachable
.insert(peer, (heard, std::time::Instant::now()));
self.update_paths();
@@ -1844,9 +2155,14 @@ impl Runtime {
.map(|session| {
let snapshot = snapshot_connection(&session.conn);
PeerStatus {
broadcast: session.broadcast,
exit_node: session.exit_node,
endpoint_id: session.peer,
role: session.role,
hostname: session.hostname.clone(),
hostname: session
.hostname
.clone()
.or_else(|| self.known_hostnames.get(&session.peer).cloned()),
protocols: session
.capabilities
.iter()
@@ -1904,6 +2220,7 @@ impl Runtime {
// both hold it would be untrue on one of them.
overlay_address_v4: self.state.address_of(&endpoint_id),
hostname: self.state.hostname_of(&endpoint_id).map(str::to_string),
last_hostname: self.known_hostnames.get(&endpoint_id).cloned(),
})
})
.collect();
@@ -1911,6 +2228,19 @@ impl Runtime {
members.dedup_by(|a, b| a.endpoint_id == b.endpoint_id);
NetworkStatus {
broadcast: self.params.broadcast,
exit_offer: self.params.exit_offer,
exit_via: self.params.exit_via,
exit_via_online: self
.params
.exit_via
.is_some_and(|via| self.exit_node_online(via)),
exit_rules: self
.params
.interface
.as_ref()
.map(|interface| interface.exit_rules(self.network_id))
.unwrap_or_default(),
descriptor: self.params.keys.descriptor(),
name: self.params.keys.name().clone(),
network_id: self.network_id,
@@ -2035,3 +2365,86 @@ fn merge_addr(existing: &mut iroh::EndpointAddr, incoming: &iroh::EndpointAddr)
existing.addrs.insert(addr.clone());
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_data_link_backoff_progression() {
assert_eq!(data_link_backoff(0), Duration::ZERO);
assert_eq!(data_link_backoff(1), Duration::from_secs(3));
assert_eq!(data_link_backoff(2), Duration::from_secs(6));
assert_eq!(data_link_backoff(3), Duration::from_secs(12));
assert_eq!(data_link_backoff(4), Duration::from_secs(24));
assert_eq!(data_link_backoff(5), Duration::from_secs(48));
assert_eq!(data_link_backoff(6), Duration::from_secs(60));
assert_eq!(data_link_backoff(10), Duration::from_secs(60));
}
#[test]
fn test_protocol_dial_state_cooldown_and_fallback_selection() {
let peer = crate::identity::DeviceIdentity::generate().endpoint_id();
let supported = ["wg", "tcp-tls", "wg-quic"];
let mut states: HashMap<(EndpointId, String), ProtocolDialState> = HashMap::new();
let now = std::time::Instant::now();
// 1. Without cooldown, the highest priority protocol "wg" is chosen
let chosen =
supported
.iter()
.find(|proto| match states.get(&(peer, (*proto).to_string())) {
Some(state) => state.cooldown_until <= now,
None => true,
});
assert_eq!(chosen.copied(), Some("wg"));
// 2. Record failure for "wg" putting it on cooldown
states.insert(
(peer, "wg".to_string()),
ProtocolDialState {
failures: 1,
cooldown_until: now + Duration::from_secs(3),
},
);
// With "wg" on cooldown, selection must fall back to next priority "tcp-tls"
let chosen_after_failure =
supported
.iter()
.find(|proto| match states.get(&(peer, (*proto).to_string())) {
Some(state) => state.cooldown_until <= now,
None => true,
});
assert_eq!(chosen_after_failure.copied(), Some("tcp-tls"));
// 3. If "tcp-tls" also fails, selection falls back to "wg-quic"
states.insert(
(peer, "tcp-tls".to_string()),
ProtocolDialState {
failures: 1,
cooldown_until: now + Duration::from_secs(3),
},
);
let chosen_quic =
supported
.iter()
.find(|proto| match states.get(&(peer, (*proto).to_string())) {
Some(state) => state.cooldown_until <= now,
None => true,
});
assert_eq!(chosen_quic.copied(), Some("wg-quic"));
// 4. Once success is recorded (entry removed), "wg" is preferred again
states.remove(&(peer, "wg".to_string()));
let chosen_restored =
supported
.iter()
.find(|proto| match states.get(&(peer, (*proto).to_string())) {
Some(state) => state.cooldown_until <= now,
None => true,
});
assert_eq!(chosen_restored.copied(), Some("wg"));
}
}
@@ -73,17 +73,21 @@ pub(crate) struct Session {
pub(crate) outbound: mpsc::Sender<Vec<u8>>,
pub(crate) hostname: Option<String>,
pub(crate) capabilities: Vec<PluginCapability>,
pub(crate) broadcast: bool,
/// The peer offers itself as an exit node in this network.
pub(crate) exit_node: bool,
pub(crate) messages_sent: u64,
pub(crate) messages_received: u64,
pub(crate) bytes_sent: u64,
pub(crate) bytes_received: u64,
reader: JoinHandle<()>,
writer: JoinHandle<()>,
path_monitor: JoinHandle<()>,
shutdown: Shutdown,
}
impl Session {
/// Signals both tasks to stop and waits for them, with a bounded grace
/// Signals all tasks to stop and waits for them, with a bounded grace
/// period.
///
/// A peer that stops reading must not be able to hold up shutdown, so the
@@ -93,6 +97,7 @@ impl Session {
conn,
reader,
writer,
path_monitor,
shutdown,
..
} = self;
@@ -102,23 +107,27 @@ impl Session {
let reader_abort = reader.abort_handle();
let writer_abort = writer.abort_handle();
let path_abort = path_monitor.abort_handle();
let joined = tokio::time::timeout(STOP_GRACE, async move {
let _ = reader.await;
let _ = writer.await;
let _ = path_monitor.await;
})
.await;
if joined.is_err() {
tracing::debug!("session tasks did not wind down in time; aborting them");
reader_abort.abort();
writer_abort.abort();
path_abort.abort();
}
}
/// Aborts both tasks without waiting. Used on replacement.
/// Aborts all tasks without waiting. Used on replacement.
pub(crate) fn abort(&self) {
self.shutdown.trigger();
self.reader.abort();
self.writer.abort();
self.path_monitor.abort();
}
}
@@ -154,6 +163,52 @@ pub(crate) fn spawn(
parent_shutdown.clone(),
));
let path_conn = conn.clone();
let path_shutdown = shutdown.clone();
let path_parent = parent_shutdown.clone();
let path_monitor = tokio::spawn(async move {
use tokio_stream::StreamExt;
let mut events = path_conn.path_events();
loop {
tokio::select! {
biased;
_ = path_shutdown.wait() => break,
_ = path_parent.wait() => break,
ev = events.next() => {
match ev {
Some(iroh::endpoint::PathEvent::Selected { remote_addr, local_addr, .. }) => {
tracing::debug!(
peer = %peer.fmt_short(),
remote = %remote_addr,
local = ?local_addr,
"Connection path selected (direct / relay / hole punch switch)"
);
}
Some(iroh::endpoint::PathEvent::Opened { remote_addr, .. }) => {
tracing::debug!(
peer = %peer.fmt_short(),
remote = %remote_addr,
"Network path opened to peer"
);
}
Some(iroh::endpoint::PathEvent::Closed { remote_addr, .. }) => {
tracing::debug!(
peer = %peer.fmt_short(),
remote = %remote_addr,
"Network path closed for peer"
);
}
Some(iroh::endpoint::PathEvent::Lagged { missed, .. }) => {
tracing::trace!(peer = %peer.fmt_short(), missed, "Path events lagged");
}
Some(_) => {}
None => break,
}
}
}
}
});
let reader = tokio::spawn(reader_task(
id,
network_id,
@@ -174,12 +229,15 @@ pub(crate) fn spawn(
outbound: outbound_tx,
hostname: None,
capabilities: Vec::new(),
broadcast: false,
exit_node: false,
messages_sent: 0,
messages_received: 0,
bytes_sent: 0,
bytes_received: 0,
reader,
writer,
path_monitor,
shutdown,
}
}
@@ -53,6 +53,10 @@ pub struct OverlayStatus {
pub addresses: Vec<crate::overlay::Cidr>,
/// What has happened on it.
pub counters: crate::overlay::Counters,
/// The route and firewall allowance installed for LAN broadcast, when
/// some network has broadcast on and the interface is on the host.
/// `None` is "none installed", not "unknown".
pub broadcast_rules: Option<crate::overlay::BroadcastRulesReport>,
}
/// A member the signed state knows about, connected or not.
@@ -76,11 +80,20 @@ pub struct MemberStatus {
/// Signed, so it is still known while the member is away — which is what
/// lets an absent member be named rather than shown as a bare id.
pub hostname: Option<String>,
/// The last name this agent saw it announce, whether or not it is here.
///
/// Not signed, so it is a convenience for naming a member that is away
/// and never a claim: the zone is served from `hostname`.
pub last_hostname: Option<String>,
}
/// Status of one authenticated session.
#[derive(Debug, Clone)]
pub struct PeerStatus {
/// This authenticated peer accepts network broadcasts.
pub broadcast: bool,
/// This authenticated peer offers itself as an exit node.
pub exit_node: bool,
/// Authenticated endpoint id.
pub endpoint_id: EndpointId,
/// Which side this agent played in the handshake.
@@ -151,6 +164,16 @@ pub struct NetworkMetrics {
/// Status of one network.
#[derive(Debug, Clone)]
pub struct NetworkStatus {
/// Local broadcast participation in this network.
pub broadcast: bool,
/// This agent offers itself as an exit node in this network.
pub exit_offer: bool,
/// The member this device sends all its internet traffic through.
pub exit_via: Option<EndpointId>,
/// Whether that member is connected and still offering to be one.
pub exit_via_online: bool,
/// What the host rules for the exit settings are doing.
pub exit_rules: crate::overlay::ExitRulesReport,
/// Immutable deterministic description of the network space.
pub descriptor: NetworkDescriptor,
/// Network name, for convenience.
@@ -12,9 +12,13 @@ use std::time::Duration;
use crate::dataplane::SharedPlugin;
use crate::discovery::{MainlineDiscovery, NetworkDiscovery};
use crate::error::{Error, Result};
use crate::error::Result;
/// Qualifier/organisation/application triple used for platform directories.
///
/// Unused on macOS, where the default paths are fixed system locations rather
/// than per-user directories derived from this.
#[cfg_attr(target_os = "macos", allow(dead_code))]
const APP_NAME: &str = "tsunagi";
/// Rendezvous slots per network; this bounds a discovery sample, not membership.
@@ -74,17 +78,34 @@ impl StoragePaths {
}
}
/// The per-user platform directories.
/// The default platform directories.
///
/// A future system service can supply its own paths instead.
/// Per user on Linux and Windows. On macOS the agent runs as a root
/// LaunchDaemon (creating a utun interface needs root, and `$HOME` under
/// `sudo` points at the wrong place), so the stores live in the system
/// locations instead — `/var/db/tsunagi` for the mandatory state and a
/// disposable cache beside it. A caller that wants somewhere else passes
/// explicit paths to [`StoragePaths::new`] or [`StoragePaths::under`].
pub fn user_default() -> Result<Self> {
let dirs = directories::ProjectDirs::from("", "", APP_NAME).ok_or_else(|| {
Error::Storage("no valid home directory for platform config paths".into())
})?;
Ok(Self {
state_dir: dirs.data_dir().to_path_buf(),
cache_dir: dirs.cache_dir().to_path_buf(),
})
#[cfg(target_os = "macos")]
{
Ok(Self {
state_dir: PathBuf::from("/var/db/tsunagi"),
cache_dir: PathBuf::from("/var/db/tsunagi/cache"),
})
}
#[cfg(not(target_os = "macos"))]
{
let dirs = directories::ProjectDirs::from("", "", APP_NAME).ok_or_else(|| {
crate::error::Error::Storage(
"no valid home directory for platform config paths".into(),
)
})?;
Ok(Self {
state_dir: dirs.data_dir().to_path_buf(),
cache_dir: dirs.cache_dir().to_path_buf(),
})
}
}
/// Path of the mandatory state database.
@@ -304,6 +325,9 @@ pub struct AgentConfig {
pub reconnect: ReconnectPolicy,
/// IP plugins whose capabilities are announced and dispatched.
pub plugins: Vec<SharedPlugin>,
/// Custom packet transports registered per protocol.
pub custom_transports:
std::collections::HashMap<String, Arc<dyn crate::dataplane::PacketTransport>>,
/// Where the overlay interface comes from, when the agent should have one.
///
/// One agent, one interface: it belongs here rather than to a protocol,
@@ -345,6 +369,7 @@ impl AgentConfig {
limits: Limits::default(),
reconnect: ReconnectPolicy::default(),
plugins: Vec::new(),
custom_transports: std::collections::HashMap::new(),
tun_factory: None,
interface_name: "tsun0".to_string(),
interface_mtu: 1280,
@@ -430,6 +455,16 @@ impl AgentConfig {
self
}
/// Registers a custom packet transport for a specific protocol.
pub fn with_custom_transport(
mut self,
protocol: impl Into<String>,
transport: Arc<dyn crate::dataplane::PacketTransport>,
) -> Self {
self.custom_transports.insert(protocol.into(), transport);
self
}
/// Gives the agent an overlay interface, from this factory.
pub fn with_interface(
mut self,
@@ -480,3 +515,5 @@ impl std::fmt::Debug for AgentConfig {
.finish()
}
}
/// Maximum number of transport links a routed datagram may traverse.
pub const ROUTING_HOP_LIMIT: u8 = 16;
@@ -20,7 +20,9 @@
//! recorded and surfaced, the control plane keeps running.
pub mod relay;
pub mod routing;
pub mod transport;
pub mod tunnel;
use std::sync::Arc;
@@ -30,7 +32,10 @@ use tokio::sync::mpsc;
use crate::BoxFuture;
use crate::identity::NetworkId;
pub use transport::{PacketLink, PacketTransport, SharedLink, TransportError};
pub use transport::{
MultiPacketTransport, PacketLink, PacketTransport, SharedLink, TransportError,
};
pub use tunnel::{GenericTunnelPlugin, TunnelCodec};
/// Maximum length of a plugin protocol identifier.
pub const MAX_PROTOCOL_ID_LEN: usize = 32;
@@ -114,7 +119,7 @@ pub enum PluginError {
/// A request a plugin makes of the agent that owns it.
#[derive(Debug)]
pub(crate) enum PluginRequest {
pub enum PluginRequest {
/// Re-send this agent's announcement to every peer of a network.
Reannounce(NetworkId),
/// Surface a plugin error on the agent's event stream.
@@ -126,6 +131,17 @@ pub(crate) enum PluginRequest {
/// Human readable reason, free of secrets.
reason: String,
},
/// A plugin reported that a data link or tunnel for a peer failed.
ProtocolFailure {
/// Network the failure occurred in.
network: NetworkId,
/// Peer whose protocol failed.
peer: EndpointId,
/// Protocol that failed.
protocol: String,
/// Human readable reason.
reason: String,
},
}
/// The agent-side handle a plugin is given when it is attached.
@@ -178,6 +194,29 @@ impl PluginContext {
}
}
/// A context configured with a custom packet sink.
pub fn with_sink(sink: Arc<dyn PacketSink>) -> Self {
Self {
sender: None,
local: None,
sink: Some(sink),
}
}
/// A context configured with a channel for plugin requests and custom sink, for tests.
#[cfg(feature = "testing")]
pub fn for_test(
sender: mpsc::Sender<PluginRequest>,
local: EndpointId,
sink: Option<Arc<dyn PacketSink>>,
) -> Self {
Self {
sender: Some(sender),
local: Some(local),
sink,
}
}
/// This agent's own endpoint id, when the context is attached.
///
/// A plugin needs it to find itself in the agreed allocation.
@@ -219,6 +258,22 @@ impl PluginContext {
reason: reason.into(),
});
}
/// Reports that data plane connectivity or a tunnel for a peer has failed.
pub fn report_peer_protocol_failure(
&self,
network: NetworkId,
peer: EndpointId,
protocol: impl Into<String>,
reason: impl Into<String>,
) {
self.send(PluginRequest::ProtocolFailure {
network,
peer,
protocol: protocol.into(),
reason: reason.into(),
});
}
}
impl std::fmt::Debug for PluginContext {
+506
View File
@@ -0,0 +1,506 @@
//! Userspace forwarding. Topology updates publish immutable tables; the packet
//! path takes no routing mutex, walks no graph and never inspects plugin bytes.
//! Each transport reader forwards transit immediately, independently of the
//! destination protocol's receive task. Only local delivery enters its inbox.
use std::collections::{HashMap, HashSet};
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use arc_swap::ArcSwap;
use bytes::Bytes;
use iroh::EndpointId;
use tokio::sync::{Notify, mpsc, watch};
use tokio::task::JoinHandle;
use super::routing::{FlowId, PeerId, RoutingTable, envelope, mix64};
use super::transport::{PacketLink, SharedLink, TransportError};
use crate::BoxFuture;
use crate::config::{MAX_DATA_DATAGRAM, ROUTING_HOP_LIMIT};
use crate::identity::NetworkId;
/// Fixed routing header, subtracted from the logical transport MTU.
pub const RELAY_OVERHEAD: usize = envelope::HEADER;
const INBOX: usize = 256;
/// Forwarding counters, shared by all protocols of one network.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct RelayCounters {
/// Datagrams originated here and sent through another member.
pub sent_via: u64,
/// Transit datagrams handed to the next transport link.
pub forwarded: u64,
/// Local datagrams received through another member.
pub received_via: u64,
/// Missing or failed next hop.
pub dropped_no_link: u64,
/// Malformed envelope or unknown member.
pub dropped_unknown: u64,
/// Exhausted hop limit, including loops during topology convergence.
pub dropped_hop_limit: u64,
/// Full local protocol inbox. Transit does not use this queue.
pub dropped_congested: u64,
}
#[derive(Debug, Default)]
struct Tally {
sent_via: AtomicU64,
forwarded: AtomicU64,
received_via: AtomicU64,
dropped_no_link: AtomicU64,
dropped_unknown: AtomicU64,
dropped_hop_limit: AtomicU64,
dropped_congested: AtomicU64,
}
impl Tally {
fn snapshot(&self) -> RelayCounters {
RelayCounters {
sent_via: self.sent_via.load(Ordering::Relaxed),
forwarded: self.forwarded.load(Ordering::Relaxed),
received_via: self.received_via.load(Ordering::Relaxed),
dropped_no_link: self.dropped_no_link.load(Ordering::Relaxed),
dropped_unknown: self.dropped_unknown.load(Ordering::Relaxed),
dropped_hop_limit: self.dropped_hop_limit.load(Ordering::Relaxed),
dropped_congested: self.dropped_congested.load(Ordering::Relaxed),
}
}
}
#[derive(Debug)]
struct NextHop {
peer: PeerId,
link: SharedLink,
}
#[derive(Debug)]
struct ForwardRoute {
hops: u8,
next: Box<[NextHop]>,
}
impl ForwardRoute {
fn select(&self, source: &PeerId, flow: FlowId) -> Option<&NextHop> {
if self.next.len() == 1 {
return self.next.first().filter(|hop| !hop.link.is_closed());
}
let seed = u64::from_le_bytes(source[..8].try_into().ok()?);
let start = mix64(flow ^ seed) as usize % self.next.len();
// Stable fallback if a link closed just before its table was replaced.
(0..self.next.len())
.map(|i| &self.next[(start + i) % self.next.len()])
.find(|hop| !hop.link.is_closed())
}
}
#[derive(Debug, Default)]
struct ForwardingTable {
routes: HashMap<PeerId, ForwardRoute>,
inboxes: HashMap<PeerId, mpsc::Sender<Bytes>>,
members: HashSet<PeerId>,
}
#[derive(Debug)]
struct Plane {
local: PeerId,
table: ArcSwap<ForwardingTable>,
tally: Arc<Tally>,
}
impl Plane {
/// Synchronous hot path: one snapshot, bounded header decode, table lookup,
/// and transport send. An exclusively owned buffer is modified in place.
fn receive(&self, incoming: PeerId, frame: Bytes) {
let Some(header) = envelope::decode(&frame) else {
self.tally.dropped_unknown.fetch_add(1, Ordering::Relaxed);
return;
};
let table = self.table.load();
if !table.members.contains(&header.source) || header.source == self.local {
self.tally.dropped_unknown.fetch_add(1, Ordering::Relaxed);
return;
}
if header.destination == self.local {
let Some(inbox) = table.inboxes.get(&header.source) else {
self.tally.dropped_unknown.fetch_add(1, Ordering::Relaxed);
return;
};
if inbox.try_send(frame.slice(RELAY_OVERHEAD..)).is_err() {
self.tally.dropped_congested.fetch_add(1, Ordering::Relaxed);
} else if incoming != header.source {
self.tally.received_via.fetch_add(1, Ordering::Relaxed);
}
return;
}
if header.remaining <= 1 {
self.tally.dropped_hop_limit.fetch_add(1, Ordering::Relaxed);
return;
}
let hop = table
.routes
.get(&header.destination)
.and_then(|route| route.select(&header.source, header.flow));
let Some(hop) = hop else {
self.tally.dropped_no_link.fetch_add(1, Ordering::Relaxed);
return;
};
if hop.link.send(envelope::decrement(frame)).is_ok() {
self.tally.forwarded.fetch_add(1, Ordering::Relaxed);
} else {
self.tally.dropped_no_link.fetch_add(1, Ordering::Relaxed);
}
}
}
#[derive(Debug)]
struct Protocol {
plane: Arc<Plane>,
graph: HashMap<PeerId, Vec<PeerId>>,
members: HashSet<PeerId>,
raw: HashMap<PeerId, SharedLink>,
readers: HashMap<PeerId, JoinHandle<()>>,
links: HashMap<PeerId, Arc<PeerLink>>,
}
impl Protocol {
fn publish(&self) {
let mut graph = self.graph.clone();
graph.insert(
self.plane.local,
self.raw
.iter()
.filter(|(_, link)| !link.is_closed())
.map(|(peer, _)| *peer)
.collect(),
);
let routing = RoutingTable::build(self.plane.local, &graph, ROUTING_HOP_LIMIT);
let routes = routing
.iter()
.filter_map(|(peer, route)| {
let next: Box<[_]> = route
.next_hops
.iter()
.filter_map(|id| {
self.raw.get(id).map(|link| NextHop {
peer: *id,
link: link.clone(),
})
})
.collect();
(!next.is_empty()).then_some((
*peer,
ForwardRoute {
hops: route.hops,
next,
},
))
})
.collect();
let mut members = self.members.clone();
members.extend(self.raw.keys().copied());
self.plane.table.store(Arc::new(ForwardingTable {
routes,
members,
inboxes: self
.links
.iter()
.map(|(peer, link)| (*peer, link.inbox_tx.clone()))
.collect(),
}));
}
}
impl Drop for Protocol {
fn drop(&mut self) {
for task in self.readers.values() {
task.abort();
}
for link in self.links.values() {
link.close();
}
self.plane.table.store(Arc::default());
}
}
/// One network's transport links and independently published protocol tables.
#[derive(Debug)]
pub struct RelayHub {
network: NetworkId,
local: PeerId,
protocols: Mutex<HashMap<String, Protocol>>,
tally: Arc<Tally>,
changed: Arc<Notify>,
}
impl RelayHub {
/// Creates the router at this network's local endpoint.
pub fn new(network: NetworkId, local: EndpointId) -> Arc<Self> {
Arc::new(Self {
network,
local: *local.as_bytes(),
protocols: Mutex::default(),
tally: Arc::default(),
changed: Arc::default(),
})
}
fn protocols(&self) -> std::sync::MutexGuard<'_, HashMap<String, Protocol>> {
self.protocols
.lock()
.unwrap_or_else(|error| error.into_inner())
}
fn protocol<'a>(
&self,
protocols: &'a mut HashMap<String, Protocol>,
name: &str,
) -> &'a mut Protocol {
protocols
.entry(name.to_owned())
.or_insert_with(|| Protocol {
plane: Arc::new(Plane {
local: self.local,
table: ArcSwap::from_pointee(ForwardingTable::default()),
tally: self.tally.clone(),
}),
graph: HashMap::new(),
members: HashSet::new(),
raw: HashMap::new(),
readers: HashMap::new(),
links: HashMap::new(),
})
}
/// Wakes the control loop promptly when a transport reader ends.
pub async fn changed(&self) {
self.changed.notified().await;
}
/// Current counters.
pub fn counters(&self) -> RelayCounters {
self.tally.snapshot()
}
/// Publishes authenticated topology. Refreshing identical state is cheap.
pub fn set_topology(
&self,
protocol: &str,
graph: HashMap<PeerId, Vec<PeerId>>,
members: HashSet<PeerId>,
) {
let mut protocols = self.protocols();
let state = self.protocol(&mut protocols, protocol);
if state.graph != graph || state.members != members {
state.graph = graph;
state.members = members;
state.publish();
}
}
/// Whether the current table has a path to this destination.
pub fn reachable(&self, peer: EndpointId, protocol: &str) -> bool {
self.protocols().get(protocol).is_some_and(|state| {
state
.plane
.table
.load()
.routes
.contains_key(peer.as_bytes())
})
}
/// Stable logical link, preserved across topology changes.
pub fn link(&self, peer: EndpointId, protocol: &str) -> Arc<PeerLink> {
let mut protocols = self.protocols();
let state = self.protocol(&mut protocols, protocol);
if let Some(link) = state.links.get(peer.as_bytes()) {
return link.clone();
}
let (inbox_tx, inbox_rx) = mpsc::channel(INBOX);
let (closed, _) = watch::channel(false);
let link = Arc::new(PeerLink {
network: self.network,
peer,
plane: state.plane.clone(),
inbox_tx,
inbox_rx: tokio::sync::Mutex::new(inbox_rx),
closed,
});
state.links.insert(*peer.as_bytes(), link.clone());
state.publish();
link
}
/// Whether the protocol already owns its logical link.
pub fn has_link(&self, peer: EndpointId, protocol: &str) -> bool {
self.protocols()
.get(protocol)
.is_some_and(|state| state.links.contains_key(peer.as_bytes()))
}
/// Installs a raw transport link and starts its independent ingress reader.
pub fn set_direct(&self, peer: EndpointId, protocol: &str, raw: SharedLink) {
if raw.network() != self.network || raw.peer() != peer {
return;
}
let mut protocols = self.protocols();
let state = self.protocol(&mut protocols, protocol);
let id = *peer.as_bytes();
if let Some(old) = state.readers.remove(&id) {
old.abort();
}
state.raw.insert(id, raw.clone());
state.publish();
let plane = state.plane.clone();
let changed = self.changed.clone();
state.readers.insert(
id,
tokio::spawn(async move {
let mut batch = 0;
while let Some(frame) = raw.recv().await {
plane.receive(id, frame);
batch += 1;
if batch == 64 {
tokio::task::yield_now().await;
batch = 0;
}
}
changed.notify_one();
}),
);
}
/// Removes a physical link without tearing down end-to-end protocol state.
pub fn clear_direct(&self, peer: EndpointId, protocol: &str) {
if let Some(state) = self.protocols().get_mut(protocol) {
state.raw.remove(peer.as_bytes());
if let Some(task) = state.readers.remove(peer.as_bytes()) {
task.abort();
}
state.publish();
}
}
/// Revokes all links and topology involving a departed authenticated peer.
pub fn remove_peer(&self, peer: EndpointId) {
for state in self.protocols().values_mut() {
state.raw.remove(peer.as_bytes());
if let Some(task) = state.readers.remove(peer.as_bytes()) {
task.abort();
}
if let Some(link) = state.links.remove(peer.as_bytes()) {
link.close();
}
state.members.remove(peer.as_bytes());
state.graph.remove(peer.as_bytes());
for neighbors in state.graph.values_mut() {
neighbors.retain(|id| id != peer.as_bytes());
}
state.publish();
}
}
/// Stops every reader and releases all transport handles.
pub fn close(&self) {
self.protocols().clear();
}
}
/// A protocol's end-to-end channel, independent of its current next hop.
#[derive(Debug)]
pub struct PeerLink {
network: NetworkId,
peer: EndpointId,
plane: Arc<Plane>,
inbox_tx: mpsc::Sender<Bytes>,
inbox_rx: tokio::sync::Mutex<mpsc::Receiver<Bytes>>,
closed: watch::Sender<bool>,
}
impl PeerLink {
fn close(&self) {
self.closed.send_replace(true);
}
}
impl PacketLink for PeerLink {
fn network(&self) -> NetworkId {
self.network
}
fn peer(&self) -> EndpointId {
self.peer
}
fn max_datagram_size(&self) -> usize {
MAX_DATA_DATAGRAM - RELAY_OVERHEAD
}
fn send(&self, payload: Bytes) -> Result<(), TransportError> {
self.send_flow(payload, 0)
}
fn send_flow(&self, payload: Bytes, flow: FlowId) -> Result<(), TransportError> {
if self.is_closed() {
return Err(TransportError::Closed);
}
if payload.len() > self.max_datagram_size() {
return Err(TransportError::TooLarge {
size: payload.len(),
limit: self.max_datagram_size(),
});
}
let table = self.plane.table.load();
let route = table
.routes
.get(self.peer.as_bytes())
.ok_or(TransportError::Closed)?;
let hop = route
.select(&self.plane.local, flow)
.ok_or(TransportError::Closed)?;
hop.link.send(envelope::encode(
self.plane.local,
*self.peer.as_bytes(),
flow,
&payload,
))?;
if route.hops > 1 {
self.plane.tally.sent_via.fetch_add(1, Ordering::Relaxed);
}
Ok(())
}
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
Box::pin(async move {
let mut closed = self.closed.subscribe();
if *closed.borrow_and_update() {
return None;
}
let mut inbox = self.inbox_rx.lock().await;
tokio::select! { biased; _ = closed.changed() => None, frame = inbox.recv() => frame }
})
}
fn closed(&self) -> BoxFuture<'_, ()> {
Box::pin(async move {
let mut closed = self.closed.subscribe();
if !*closed.borrow_and_update() {
let _ = closed.changed().await;
}
})
}
fn is_closed(&self) -> bool {
*self.closed.borrow()
}
fn path_description(&self) -> String {
let table = self.plane.table.load();
match table.routes.get(self.peer.as_bytes()) {
Some(route) if route.hops == 1 => route.next[0].link.path_description(),
Some(route) => format!(
"relay {} hops via {} ({} equal paths)",
route.hops,
hex::encode(route.next[0].peer),
route.next.len()
),
None => "unreachable".into(),
}
}
}
#[cfg(test)]
#[path = "relay_tests.rs"]
mod tests;
@@ -0,0 +1,287 @@
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
use std::sync::atomic::AtomicBool;
fn network() -> NetworkId {
NetworkKeys::derive(
&NetworkName::new("routing-tests").unwrap(),
&NetworkSecret::from_bytes([3; 32]).unwrap(),
)
.network_id()
}
fn peer(seed: u8) -> EndpointId {
iroh::SecretKey::from_bytes(&[seed; 32]).public()
}
fn id(seed: u8) -> PeerId {
*peer(seed).as_bytes()
}
#[derive(Debug)]
struct Wire {
peer: EndpointId,
sent: Mutex<Vec<Bytes>>,
feed: mpsc::Sender<Bytes>,
inbound: tokio::sync::Mutex<mpsc::Receiver<Bytes>>,
dead: AtomicBool,
count: AtomicU64,
record: bool,
}
impl Wire {
fn new(seed: u8, record: bool) -> Arc<Self> {
let (feed, inbound) = mpsc::channel(32);
Arc::new(Self {
peer: peer(seed),
sent: Mutex::default(),
feed,
inbound: tokio::sync::Mutex::new(inbound),
dead: AtomicBool::new(false),
count: AtomicU64::new(0),
record,
})
}
fn take(&self) -> Vec<Bytes> {
std::mem::take(&mut self.sent.lock().unwrap())
}
}
impl PacketLink for Wire {
fn network(&self) -> NetworkId {
network()
}
fn peer(&self) -> EndpointId {
self.peer
}
fn max_datagram_size(&self) -> usize {
MAX_DATA_DATAGRAM
}
fn send(&self, frame: Bytes) -> Result<(), TransportError> {
if self.is_closed() {
return Err(TransportError::Closed);
}
self.count.fetch_add(1, Ordering::Relaxed);
if self.record {
self.sent.lock().unwrap().push(frame);
}
Ok(())
}
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
Box::pin(async move { self.inbound.lock().await.recv().await })
}
fn closed(&self) -> BoxFuture<'_, ()> {
Box::pin(std::future::pending())
}
fn is_closed(&self) -> bool {
self.dead.load(Ordering::Relaxed)
}
fn path_description(&self) -> String {
"test-direct".into()
}
}
fn topology(hub: &RelayHub, protocol: &str, edges: &[(u8, &[u8])], members: &[u8]) {
hub.set_topology(
protocol,
edges
.iter()
.map(|(a, bs)| (id(*a), bs.iter().map(|b| id(*b)).collect()))
.collect(),
members.iter().map(|p| id(*p)).collect(),
);
}
#[tokio::test]
async fn ecmp_is_per_flow_direct_wins_and_existing_tunnel_survives_changes() {
let hub = RelayHub::new(network(), peer(1));
let b = Wire::new(2, true);
let c = Wire::new(3, true);
hub.set_direct(peer(2), "ip", b.clone());
hub.set_direct(peer(3), "ip", c.clone());
topology(&hub, "ip", &[(2, &[4]), (3, &[4])], &[1, 2, 3, 4]);
let link = hub.link(peer(4), "ip");
let mut chosen = HashMap::new();
for _ in 0..8 {
for flow in 0..100 {
link.send_flow(Bytes::from_static(b"ciphertext"), flow)
.unwrap();
}
for (hop, wire) in [(2, &b), (3, &c)] {
for frame in wire.take() {
let header = envelope::decode(&frame).unwrap();
assert_eq!(header.source, id(1));
assert_eq!(header.destination, id(4));
if let Some(old) = chosen.insert(header.flow, hop) {
assert_eq!(old, hop);
}
}
}
}
assert_eq!(
chosen.values().copied().collect::<HashSet<_>>(),
HashSet::from([2, 3])
);
let direct = Wire::new(4, true);
hub.set_direct(peer(4), "ip", direct.clone());
link.send_flow(Bytes::from_static(b"direct"), 8).unwrap();
assert_eq!(direct.take().len(), 1);
assert!(b.take().is_empty() && c.take().is_empty());
hub.clear_direct(peer(4), "ip");
b.dead.store(true, Ordering::Relaxed);
for flow in 0..100 {
link.send_flow(Bytes::from_static(b"fallback"), flow)
.unwrap();
}
assert_eq!(c.take().len(), 100);
hub.clear_direct(peer(2), "ip");
topology(&hub, "ip", &[(3, &[])], &[1, 2, 3, 4]);
assert!(link.send(Bytes::new()).is_err());
assert!(!link.is_closed());
assert!(Arc::ptr_eq(&link, &hub.link(peer(4), "ip")));
hub.remove_peer(peer(4));
assert!(link.is_closed());
tokio::time::timeout(std::time::Duration::from_secs(1), link.closed())
.await
.unwrap();
assert!(link.recv().await.is_none());
hub.close();
}
#[tokio::test]
async fn transport_reader_forwards_without_any_plugin_reader_and_isolates_protocols() {
let hub = RelayHub::new(network(), peer(2));
let a = Wire::new(1, true);
let c = Wire::new(3, true);
hub.set_direct(peer(1), "ip", a.clone());
hub.set_direct(peer(3), "ip", c.clone());
topology(&hub, "ip", &[(3, &[4])], &[1, 2, 3, 4]);
let frame = envelope::encode(id(1), id(4), 91, b"end-to-end encrypted");
let pointer = frame.as_ptr();
a.feed.send(frame).await.unwrap();
tokio::time::timeout(std::time::Duration::from_secs(1), async {
while hub.counters().forwarded == 0 {
tokio::task::yield_now().await;
}
})
.await
.unwrap();
let sent = c.take();
assert_eq!(sent[0].as_ptr(), pointer);
assert_eq!(
envelope::decode(&sent[0]).unwrap().remaining,
ROUTING_HOP_LIMIT - 1
);
assert_eq!(&sent[0][RELAY_OVERHEAD..], b"end-to-end encrypted");
let isolated = hub.link(peer(4), "other-ip");
assert!(isolated.send(Bytes::new()).is_err());
assert_eq!(hub.counters().forwarded, 1);
hub.close();
}
#[tokio::test]
async fn local_delivery_is_bounded_and_malformed_or_unknown_sources_are_dropped() {
let hub = RelayHub::new(network(), peer(2));
let a = Wire::new(1, true);
hub.set_direct(peer(1), "ip", a);
let link = hub.link(peer(1), "ip");
let plane = link.plane.clone();
for _ in 0..INBOX + 1 {
plane.receive(id(1), envelope::encode(id(1), id(2), 0, b"ip"));
}
assert_eq!(hub.counters().dropped_congested, 1);
for _ in 0..INBOX {
assert_eq!(link.recv().await.unwrap(), Bytes::from_static(b"ip"));
}
plane.receive(id(1), Bytes::from_static(b"broken"));
plane.receive(id(1), envelope::encode(id(9), id(2), 0, b"unknown member"));
assert_eq!(hub.counters().dropped_unknown, 2);
plane.receive(
id(1),
envelope::encode(id(1), id(9), 0, b"unknown destination"),
);
assert_eq!(hub.counters().dropped_no_link, 1);
hub.close();
}
#[tokio::test]
async fn inconsistent_tables_cannot_loop_beyond_the_hop_limit() {
let a = RelayHub::new(network(), peer(1));
let b = RelayHub::new(network(), peer(2));
let ab = Wire::new(2, true);
let ba = Wire::new(1, true);
a.set_direct(peer(2), "ip", ab.clone());
b.set_direct(peer(1), "ip", ba.clone());
// Deliberately inconsistent snapshots while updates are in flight.
topology(&a, "ip", &[(2, &[4])], &[1, 2, 3, 4]);
topology(&b, "ip", &[(1, &[4])], &[1, 2, 3, 4]);
let ap = a.link(peer(4), "ip").plane.clone();
let bp = b.link(peer(4), "ip").plane.clone();
let mut frame = envelope::encode(id(3), id(4), 7, b"transit");
for turn in 0..ROUTING_HOP_LIMIT {
let (plane, incoming, wire) = if turn % 2 == 0 {
(&ap, id(2), &ab)
} else {
(&bp, id(1), &ba)
};
plane.receive(incoming, frame);
let sent = wire.take();
if turn == ROUTING_HOP_LIMIT - 1 {
assert!(sent.is_empty());
break;
}
assert_eq!(sent.len(), 1);
frame = sent.into_iter().next().unwrap();
}
assert_eq!(
a.counters().forwarded + b.counters().forwarded,
u64::from(ROUTING_HOP_LIMIT - 1)
);
assert_eq!(
a.counters().dropped_hop_limit + b.counters().dropped_hop_limit,
1
);
a.close();
b.close();
}
/// Measures the actual synchronous transit routine, including header checks,
/// snapshot lookup, ECMP, in-place TTL update and mock transport submission.
/// Packet construction, encryption and socket I/O are outside this measurement.
#[tokio::test]
#[ignore = "manual release-mode forwarding microbenchmark"]
async fn forwarding_benchmark() {
let hub = RelayHub::new(network(), peer(2));
let wire = Wire::new(3, false);
hub.set_direct(peer(3), "ip", wire.clone());
let alternate = Wire::new(5, false);
hub.set_direct(peer(5), "ip", alternate.clone());
topology(&hub, "ip", &[(3, &[4]), (5, &[4])], &[1, 2, 3, 4, 5]);
let plane = hub.link(peer(4), "ip").plane.clone();
let source = id(1);
let destination = id(4);
let incoming = id(1);
for size in [64, 1280] {
let mut elapsed = std::time::Duration::ZERO;
const COUNT: usize = 10000;
const BATCHES: usize = 100;
for _ in 0..BATCHES {
let frames: Vec<_> = (0..COUNT)
.map(|flow| envelope::encode(source, destination, flow as u64, &vec![42; size]))
.collect();
let start = std::time::Instant::now();
for frame in frames {
plane.receive(incoming, std::hint::black_box(frame));
}
elapsed += start.elapsed();
}
let count = (COUNT * BATCHES) as f64;
println!(
"transit {size} B: {:.1} ns/packet, {:.2} Mpps (1M packets, two equal paths)",
elapsed.as_nanos() as f64 / count,
count / elapsed.as_secs_f64() / 1e6
);
}
assert_eq!(
wire.count.load(Ordering::Relaxed) + alternate.count.load(Ordering::Relaxed),
2000000
);
hub.close();
}
@@ -0,0 +1,84 @@
//! Fixed header. Transit changes one byte in an owned receive buffer.
use super::{FlowId, PeerId};
use crate::config::{MAX_DATA_DATAGRAM, ROUTING_HOP_LIMIT};
use bytes::{BufMut, Bytes, BytesMut};
pub(crate) const HEADER: usize = 2 + 32 + 32 + 8;
const VERSION: u8 = 1;
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct Header {
pub source: PeerId,
pub destination: PeerId,
pub remaining: u8,
pub flow: FlowId,
}
pub(crate) fn decode(bytes: &[u8]) -> Option<Header> {
if bytes.len() < HEADER
|| bytes.len() > MAX_DATA_DATAGRAM
|| bytes[0] != VERSION
|| bytes[1] == 0
|| bytes[1] > ROUTING_HOP_LIMIT
{
return None;
}
Some(Header {
source: bytes[2..34].try_into().ok()?,
destination: bytes[34..66].try_into().ok()?,
remaining: bytes[1],
flow: u64::from_be_bytes(bytes[66..74].try_into().ok()?),
})
}
pub(crate) fn encode(source: PeerId, destination: PeerId, flow: FlowId, payload: &[u8]) -> Bytes {
let mut bytes = BytesMut::with_capacity(HEADER + payload.len());
bytes.put_u8(VERSION);
bytes.put_u8(ROUTING_HOP_LIMIT);
bytes.extend_from_slice(&source);
bytes.extend_from_slice(&destination);
bytes.put_u64(flow);
bytes.extend_from_slice(payload);
bytes.freeze()
}
pub(crate) fn decrement(bytes: Bytes) -> Bytes {
let mut bytes = match bytes.try_into_mut() {
Ok(bytes) => bytes,
Err(bytes) => BytesMut::from(bytes.as_ref()),
};
// Called only after decode and after excluding remaining <= 1.
bytes[1] -= 1;
bytes.freeze()
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
#[test]
fn transit_reuses_the_buffer_and_rejects_invalid_headers() {
let frame = encode([1; 32], [2; 32], 17, &[42; 1280]);
let address = frame.as_ptr();
let frame = decrement(frame);
assert_eq!(
frame.as_ptr(),
address,
"no payload copy for an owned buffer"
);
assert_eq!(decode(&frame).unwrap().remaining, ROUTING_HOP_LIMIT - 1);
assert_eq!(&frame[HEADER..], &[42; 1280]);
for length in 0..HEADER {
assert!(decode(&frame[..length]).is_none());
}
for ttl in [0, ROUTING_HOP_LIMIT + 1] {
let mut bad = frame.to_vec();
bad[1] = ttl;
assert!(decode(&bad).is_none());
}
let mut bad = frame.to_vec();
bad[0] = 99;
assert!(decode(&bad).is_none());
assert!(decode(&vec![1; MAX_DATA_DATAGRAM + 1]).is_none());
}
}
@@ -0,0 +1,125 @@
//! Optional IP flow classifier for IP plugins, called before encryption.
//! The router itself treats this identifier and the payload as opaque.
use super::FlowId;
fn hash(parts: &[&[u8]]) -> FlowId {
let mut value = 0xcbf29ce484222325u64;
for part in parts {
for byte in *part {
value = (value ^ u64::from(*byte)).wrapping_mul(0x100000001b3);
}
}
value
}
/// Stable directional IP flow hash, independent of TCP sequence numbers,
/// checksums and packet contents. Fragmented traffic uses addresses/protocol
/// instead of ports: even non-initial fragments and subsequent fragmented
/// datagrams stay together. This deliberately coalesces fragmented flows.
/// Malformed packets get a coarse hash; parsing is bounded and never allocates.
pub fn ip_flow(packet: &[u8]) -> FlowId {
match packet.first().map(|b| b >> 4) {
Some(4) if packet.len() >= 20 => {
let addresses = &packet[12..20];
let protocol = &packet[9..10];
let offset = usize::from(packet[0] & 15) * 4;
if u16::from_be_bytes([packet[6], packet[7]]) & 0x3fff != 0 {
return hash(&[addresses, protocol]);
}
if offset >= 20
&& matches!(packet[9], 6 | 17)
&& let Some(ports) = packet.get(offset..offset + 4)
{
return hash(&[addresses, protocol, ports]);
}
hash(&[addresses, protocol])
}
Some(6) if packet.len() >= 40 => {
let addresses = &packet[8..40];
let mut protocol = packet[6];
let mut offset = 40;
for _ in 0..8 {
match protocol {
44 => {
if let Some(fragment) = packet.get(offset..offset + 8) {
return hash(&[addresses, &fragment[..1]]);
}
break;
}
0 | 43 | 60 | 51 => {
let Some(header) = packet.get(offset..offset + 2) else {
break;
};
let size = if protocol == 51 {
(usize::from(header[1]) + 2) * 4
} else {
(usize::from(header[1]) + 1) * 8
};
protocol = header[0];
offset += size;
}
6 | 17 => {
if let Some(ports) = packet.get(offset..offset + 4) {
return hash(&[addresses, &[protocol], ports]);
}
break;
}
_ => break,
}
}
hash(&[addresses, &[protocol]])
}
_ => 0,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tcp_flow_ignores_packet_contents_but_distinguishes_ports() {
let mut packet = [0u8; 80];
packet[0] = 0x45;
packet[9] = 6;
packet[12..24].copy_from_slice(&[10, 0, 0, 1, 10, 0, 0, 2, 0, 22, 1, 2]);
let flow = ip_flow(&packet);
packet[4] = 123; // IP id changes for each unfragmented packet.
packet[24..].fill(47);
assert_eq!(flow, ip_flow(&packet));
packet[23] = 3;
assert_ne!(flow, ip_flow(&packet));
packet[6] = 0x20;
let fragment_flow = ip_flow(&packet);
packet[6] = 0;
packet[7] = 1;
packet[20..].fill(99);
assert_eq!(fragment_flow, ip_flow(&packet));
packet[4] = 97;
assert_eq!(fragment_flow, ip_flow(&packet));
}
#[test]
fn ipv6_extensions_and_fragments_are_bounded_and_stable() {
let mut packet = [0u8; 80];
packet[0] = 0x60;
packet[6] = 0;
packet[40] = 17;
packet[48..52].copy_from_slice(&[1, 2, 3, 4]);
let flow = ip_flow(&packet);
packet[52..].fill(5);
assert_eq!(flow, ip_flow(&packet));
packet[50] = 9;
assert_ne!(flow, ip_flow(&packet));
packet[6] = 44;
packet[40] = 6;
let flow = ip_flow(&packet);
packet[42] = 32;
packet[48..].fill(3);
assert_eq!(flow, ip_flow(&packet));
for len in 0..packet.len() {
let _ = ip_flow(&packet[..len]);
}
}
}
@@ -0,0 +1,131 @@
//! Transport-independent routing over opaque identities. Tables are built on
//! topology changes, never on the packet path. No WireGuard or iroh APIs here.
pub(crate) mod envelope;
pub mod flow;
use std::collections::{BTreeSet, HashMap, VecDeque};
/// Opaque authenticated identity; the transport adapter validates its keys.
pub type PeerId = [u8; 32];
/// Stable flow identifier, carried unchanged between transit routers.
pub type FlowId = u64;
/// A shortest route with all equal-cost first hops in stable order.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Route {
/// Number of transport links to the destination.
pub hops: u8,
/// Equal-cost first hops. Direct destinations have one entry.
pub next_hops: Box<[PeerId]>,
}
/// Immutable shortest-path table, replaced when connectivity changes.
#[derive(Debug, Clone, Default)]
pub struct RoutingTable {
routes: HashMap<PeerId, Route>,
}
impl RoutingTable {
/// Computes shortest directed paths bounded by `hop_limit`.
/// The caller supplies only authenticated, fresh, compatible edges.
pub fn build(local: PeerId, graph: &HashMap<PeerId, Vec<PeerId>>, hop_limit: u8) -> Self {
let mut distances = HashMap::from([(local, 0u8)]);
let mut first_hops: HashMap<PeerId, BTreeSet<PeerId>> = HashMap::new();
let mut pending = VecDeque::from([local]);
while let Some(node) = pending.pop_front() {
let distance = distances[&node];
if distance >= hop_limit {
continue;
}
let inherited = first_hops.get(&node).cloned().unwrap_or_default();
for &next in graph.get(&node).into_iter().flatten() {
let next_distance = distance + 1;
match distances.get(&next) {
Some(&known) if known < next_distance => continue,
None => {
distances.insert(next, next_distance);
pending.push_back(next);
}
_ => {}
}
let hops = first_hops.entry(next).or_default();
if node == local {
hops.insert(next);
} else {
hops.extend(inherited.iter().copied());
}
}
}
Self {
routes: first_hops
.into_iter()
.map(|(peer, hops)| {
(
peer,
Route {
hops: distances[&peer],
next_hops: hops.into_iter().collect(),
},
)
})
.collect(),
}
}
/// Precomputed route to a destination.
pub fn get(&self, destination: &PeerId) -> Option<&Route> {
self.routes.get(destination)
}
/// All reachable destinations, excluding ourselves.
pub fn iter(&self) -> impl Iterator<Item = (&PeerId, &Route)> {
self.routes.iter()
}
}
/// Stable ECMP mixing, without random state or hashing the payload per hop.
pub(crate) fn mix64(mut value: u64) -> u64 {
value ^= value >> 30;
value = value.wrapping_mul(0xbf58476d1ce4e5b9);
value ^= value >> 27;
value = value.wrapping_mul(0x94d049bb133111eb);
value ^ (value >> 31)
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
fn p(n: u8) -> PeerId {
[n; 32]
}
#[test]
fn shortest_paths_keep_equal_cost_hops_and_prefer_direct_links() {
let mut graph = HashMap::from([
(p(1), vec![p(3), p(2)]),
(p(2), vec![p(4)]),
(p(3), vec![p(4)]),
(p(4), vec![p(5)]),
(p(5), vec![p(1)]),
]);
let routes = RoutingTable::build(p(1), &graph, 16);
assert_eq!(routes.get(&p(5)).unwrap().hops, 3);
assert_eq!(&*routes.get(&p(5)).unwrap().next_hops, &[p(2), p(3)]);
assert!(routes.get(&p(1)).is_none());
assert!(routes.get(&p(9)).is_none());
graph.get_mut(&p(1)).unwrap().push(p(5));
assert_eq!(
RoutingTable::build(p(1), &graph, 16).get(&p(5)).unwrap(),
&Route {
hops: 1,
next_hops: Box::new([p(5)])
}
);
}
#[test]
fn topology_removal_and_hop_budget_remove_impossible_routes() {
let mut graph = HashMap::from([(p(1), vec![p(2)]), (p(2), vec![p(3)]), (p(3), vec![p(4)])]);
assert!(RoutingTable::build(p(1), &graph, 2).get(&p(4)).is_none());
assert!(RoutingTable::build(p(1), &graph, 3).get(&p(4)).is_some());
graph.remove(&p(2));
assert!(RoutingTable::build(p(1), &graph, 16).get(&p(4)).is_none());
}
}
@@ -32,6 +32,8 @@
mod fragments;
pub mod iroh_link;
use std::sync::Arc;
use bytes::Bytes;
use iroh::EndpointId;
@@ -88,6 +90,16 @@ pub trait PacketLink: Send + Sync + std::fmt::Debug + 'static {
/// handed to the transport, nothing more.
fn send(&self, payload: Bytes) -> Result<(), TransportError>;
/// Sends a datagram with an opaque stable flow identifier. Plugins derive
/// this before encryption; transports that do not route can ignore it.
fn send_flow(
&self,
payload: Bytes,
_flow: super::routing::FlowId,
) -> Result<(), TransportError> {
self.send(payload)
}
/// Receives the next datagram, or `None` once the link is finished.
fn recv(&self) -> BoxFuture<'_, Option<Bytes>>;
@@ -145,3 +157,51 @@ pub trait PacketTransport: Send + Sync + std::fmt::Debug + 'static {
protocol: &'a str,
) -> BoxFuture<'a, Result<SharedLink, TransportError>>;
}
/// A composite packet transport that dispatches requests by protocol id.
#[derive(Debug)]
pub struct MultiPacketTransport {
default: Arc<dyn PacketTransport>,
by_protocol: std::collections::HashMap<String, Arc<dyn PacketTransport>>,
}
impl MultiPacketTransport {
/// Creates a new composite transport with a default fallback transport.
pub fn new(
default: Arc<dyn PacketTransport>,
by_protocol: std::collections::HashMap<String, Arc<dyn PacketTransport>>,
) -> Self {
Self {
default,
by_protocol,
}
}
/// Access the default underlying transport.
pub fn default_transport(&self) -> &Arc<dyn PacketTransport> {
&self.default
}
}
impl PacketTransport for MultiPacketTransport {
fn name(&self) -> &str {
"multi"
}
fn as_any(&self) -> &dyn std::any::Any {
self
}
fn open<'a>(
&'a self,
network: NetworkId,
peer: EndpointId,
protocol: &'a str,
) -> BoxFuture<'a, Result<SharedLink, TransportError>> {
if let Some(transport) = self.by_protocol.get(protocol) {
transport.open(network, peer, protocol)
} else {
self.default.open(network, peer, protocol)
}
}
}
+380
View File
@@ -0,0 +1,380 @@
//! Generic harness for IP tunnel plugins.
//!
//! Implementing [`super::IpPlugin`] directly requires managing per-peer background
//! tasks, synchronization, packet queues, and graceful shutdown.
//!
//! [`TunnelCodec`] provides a simplified interface where a plugin author only
//! defines cryptographic transformation and handshake payloads.
//! [`GenericTunnelPlugin`] adapts any [`TunnelCodec`] into a full [`super::IpPlugin`].
use std::collections::HashMap;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex, RwLock};
use bytes::Bytes;
use iroh::EndpointId;
use tokio::task::JoinHandle;
use super::{IpPlugin, PluginCapability, PluginContext, PluginError, ProtocolOption, SharedLink};
use crate::BoxFuture;
use crate::identity::NetworkId;
/// The simplified contract for an IP tunnel protocol implementation.
///
/// Implementors only handle key announcements, link lifecycle events,
/// and packet encrypt/decrypt. Concurrency and task management are handled
/// by [`GenericTunnelPlugin`].
pub trait TunnelCodec: Send + Sync + 'static {
/// Protocol identifier, e.g. `tcp-tls`.
fn protocol_id(&self) -> &str;
/// Protocol wire version.
fn protocol_version(&self) -> u16;
/// Options accepted by the protocol.
fn options(&self) -> &'static [ProtocolOption] {
&[]
}
/// Generates local capability announcement data for a network.
fn local_capability_data(&self, network: NetworkId) -> Result<Vec<u8>, PluginError>;
/// Handles a peer's announced capability payload.
fn on_peer_capability_data(
&self,
network: NetworkId,
peer: EndpointId,
data: &[u8],
) -> Result<(), PluginError>;
/// Called when an authenticated link to a peer is established.
fn on_peer_link_up(&self, network: NetworkId, peer: EndpointId, link: &SharedLink) {
let _ = (network, peer, link);
}
/// Called when a peer goes away.
fn on_peer_down(&self, network: NetworkId, peer: EndpointId) {
let _ = (network, peer);
}
/// Encrypts an outbound IP packet into a payload to send over the link.
fn encrypt(
&self,
network: NetworkId,
peer: EndpointId,
packet: &[u8],
) -> Result<Bytes, PluginError>;
/// Decrypts an inbound payload from the link into an IP packet.
fn decrypt(
&self,
network: NetworkId,
peer: EndpointId,
payload: &[u8],
) -> Result<Bytes, PluginError>;
/// Called during shutdown.
fn shutdown<'a>(&'a self) -> BoxFuture<'a, ()> {
Box::pin(async {})
}
}
struct TunnelPeerState {
link: SharedLink,
tx_packets: AtomicU64,
rx_packets: AtomicU64,
tx_bytes: AtomicU64,
rx_bytes: AtomicU64,
dropped: AtomicU64,
established_at: std::time::Instant,
}
/// Statistics and status for one active peer in [`GenericTunnelPlugin`].
#[derive(Debug, Clone)]
pub struct TunnelPeerReport {
/// Peer endpoint identity.
pub peer: EndpointId,
/// Protocol identifier.
pub protocol: String,
/// Human-readable path description.
pub path: String,
/// Packets successfully encrypted and sent.
pub tx_packets: u64,
/// Packets successfully received and decrypted.
pub rx_packets: u64,
/// Plaintext bytes successfully encrypted and sent.
pub tx_bytes: u64,
/// Plaintext bytes successfully received and decrypted.
pub rx_bytes: u64,
/// Packets dropped due to encryption/decryption/send failures.
pub dropped: u64,
/// Uptime in seconds since the data link was established.
pub uptime_secs: u64,
}
/// Generic wrapper that turns a [`TunnelCodec`] into a full [`IpPlugin`].
pub struct GenericTunnelPlugin<C: TunnelCodec> {
codec: Arc<C>,
context: Mutex<Option<PluginContext>>,
links: RwLock<HashMap<(NetworkId, EndpointId), Arc<TunnelPeerState>>>,
tasks: Mutex<HashMap<(NetworkId, EndpointId), JoinHandle<()>>>,
}
impl<C: TunnelCodec> GenericTunnelPlugin<C> {
/// Wraps `codec` into an [`IpPlugin`].
pub fn new(codec: C) -> Self {
Self {
codec: Arc::new(codec),
context: Mutex::new(None),
links: RwLock::new(HashMap::new()),
tasks: Mutex::new(HashMap::new()),
}
}
/// Returns a reference to the underlying codec.
pub fn codec(&self) -> &C {
&self.codec
}
/// Returns the active peers with established data links for a network.
pub fn active_peers(&self, network: NetworkId) -> Vec<TunnelPeerReport> {
let guard = match self.links.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard
.iter()
.filter(|((net, _), _)| *net == network)
.map(|((_, peer), state)| TunnelPeerReport {
peer: *peer,
protocol: self.codec.protocol_id().to_string(),
path: state.link.path_description(),
tx_packets: state.tx_packets.load(Ordering::Relaxed),
rx_packets: state.rx_packets.load(Ordering::Relaxed),
tx_bytes: state.tx_bytes.load(Ordering::Relaxed),
rx_bytes: state.rx_bytes.load(Ordering::Relaxed),
dropped: state.dropped.load(Ordering::Relaxed),
uptime_secs: state.established_at.elapsed().as_secs(),
})
.collect()
}
}
impl<C: TunnelCodec> std::fmt::Debug for GenericTunnelPlugin<C> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("GenericTunnelPlugin")
.field("protocol_id", &self.codec.protocol_id())
.field("protocol_version", &self.codec.protocol_version())
.finish()
}
}
impl<C: TunnelCodec> IpPlugin for GenericTunnelPlugin<C> {
fn protocol_id(&self) -> &str {
self.codec.protocol_id()
}
fn protocol_version(&self) -> u16 {
self.codec.protocol_version()
}
fn options(&self) -> &'static [ProtocolOption] {
self.codec.options()
}
fn attach(&self, context: PluginContext) {
if let Ok(mut guard) = self.context.lock() {
*guard = Some(context);
}
}
fn local_capability(
&self,
network: NetworkId,
) -> Result<Option<PluginCapability>, PluginError> {
let data = self.codec.local_capability_data(network)?;
Ok(Some(PluginCapability {
protocol: self.codec.protocol_id().to_string(),
version: self.codec.protocol_version(),
enabled: true,
data,
}))
}
fn on_peer_capability(
&self,
network: NetworkId,
peer: EndpointId,
capability: &PluginCapability,
) -> Result<(), PluginError> {
if capability.protocol != self.codec.protocol_id() {
return Ok(());
}
self.codec
.on_peer_capability_data(network, peer, &capability.data)
}
fn carry(&self, network: NetworkId, peer: EndpointId, packet: bytes::Bytes) -> bool {
let state = {
let guard = match self.links.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.get(&(network, peer)).cloned()
};
let Some(state) = state else {
return false;
};
match self.codec.encrypt(network, peer, &packet) {
Ok(ciphertext) => {
if state.link.send(ciphertext).is_ok() {
state.tx_packets.fetch_add(1, Ordering::Relaxed);
state
.tx_bytes
.fetch_add(packet.len() as u64, Ordering::Relaxed);
true
} else {
state.dropped.fetch_add(1, Ordering::Relaxed);
false
}
}
Err(err) => {
state.dropped.fetch_add(1, Ordering::Relaxed);
tracing::debug!(%peer, %err, "tunnel encrypt failed");
false
}
}
}
fn on_peer_link(&self, network: NetworkId, peer: EndpointId, link: SharedLink) {
let state = Arc::new(TunnelPeerState {
link: Arc::clone(&link),
tx_packets: AtomicU64::new(0),
rx_packets: AtomicU64::new(0),
tx_bytes: AtomicU64::new(0),
rx_bytes: AtomicU64::new(0),
dropped: AtomicU64::new(0),
established_at: std::time::Instant::now(),
});
{
let mut guard = match self.links.write() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.insert((network, peer), Arc::clone(&state));
}
self.codec.on_peer_link_up(network, peer, &link);
let codec = Arc::clone(&self.codec);
let sink = match self.context.lock() {
Ok(g) => g.as_ref().map(|ctx| ctx.packet_sink()),
Err(p) => p.into_inner().as_ref().map(|ctx| ctx.packet_sink()),
};
let rx_link = Arc::clone(&link);
let rx_state = Arc::clone(&state);
let handle = tokio::spawn(async move {
while let Some(payload) = rx_link.recv().await {
match codec.decrypt(network, peer, &payload) {
Ok(packet) => {
rx_state.rx_packets.fetch_add(1, Ordering::Relaxed);
rx_state
.rx_bytes
.fetch_add(packet.len() as u64, Ordering::Relaxed);
if let Some(ref sink) = sink {
sink.deliver(network, peer, packet).await;
}
}
Err(err) => {
rx_state.dropped.fetch_add(1, Ordering::Relaxed);
tracing::debug!(%peer, %err, "tunnel decrypt failed");
}
}
}
});
let mut guard = match self.tasks.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
if let Some(old) = guard.insert((network, peer), handle) {
old.abort();
}
}
fn on_peer_gone(&self, network: NetworkId, peer: EndpointId) {
{
let mut guard = match self.tasks.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
if let Some(task) = guard.remove(&(network, peer)) {
task.abort();
}
}
{
let mut guard = match self.links.write() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.remove(&(network, peer));
}
self.codec.on_peer_down(network, peer);
}
fn on_network_deactivated(&self, network: NetworkId) {
let keys_to_remove: Vec<(NetworkId, EndpointId)> = {
let guard = match self.tasks.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard
.keys()
.filter(|(net, _)| *net == network)
.copied()
.collect()
};
{
let mut guard = match self.tasks.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
for key in &keys_to_remove {
if let Some(task) = guard.remove(key) {
task.abort();
}
}
}
{
let mut guard = match self.links.write() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.retain(|(net, _), _| *net != network);
}
}
fn shutdown<'a>(&'a self) -> BoxFuture<'a, ()> {
Box::pin(async move {
let mut tasks = Vec::new();
{
let mut guard = match self.tasks.lock() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
for (_, task) in guard.drain() {
tasks.push(task);
}
}
for task in tasks {
task.abort();
}
self.codec.shutdown().await;
})
}
}
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,333 @@
//! Telling the macOS resolver to send some questions here, through
//! `/etc/resolver`.
//!
//! macOS reads one file per domain out of `/etc/resolver`: a file named after
//! a domain, containing the `nameserver` and `port` to send that domain's
//! questions to, makes the system resolver a *scoped* resolver for exactly
//! that suffix and nothing else (see `resolver(5)`). That is the same contract
//! the systemd-resolved and NRPT publishers arrange on the other platforms:
//! route these suffixes to this server, and claim nothing else.
//!
//! # Why a port, and so no privileged socket
//!
//! A resolver file carries a `port`, so the server does not have to sit on 53
//! and the agent needs nothing like `CAP_NET_BIND_SERVICE`. It can serve on a
//! high port and still be reached for its suffixes.
//!
//! # Tagged, reversible, and never clobbering
//!
//! Every file this publisher writes begins with a marker line, so it only ever
//! replaces or removes files it wrote itself: a pre-existing `/etc/resolver`
//! entry a user set up by hand is left untouched. The files are removed when
//! the resolver is turned off and on shutdown; a crash leaves them behind, and
//! the next apply for the same domains overwrites them.
//!
//! # Privilege
//!
//! Writing under `/etc/resolver` needs root. A permission error is reported as
//! a refusal — the kind of failure a person has to act on — rather than a
//! transient one, so the agent does not retry it at the ordinary pace.
use std::collections::BTreeSet;
use std::net::IpAddr;
use std::path::{Path, PathBuf};
use std::sync::Mutex;
use crate::BoxFuture;
use super::{DnsPublisher, PublishError, Published};
/// The directory macOS reads scoped resolvers from.
const RESOLVER_DIR: &str = "/etc/resolver";
/// The first line of every file this publisher writes, so it recognises its
/// own and never touches a file a user created.
const MARKER: &str = "# managed by tsunagi — do not edit";
/// Writes and removes `/etc/resolver` files for the overlay zones.
#[derive(Debug)]
pub struct ResolverDirPublisher {
/// The resolver directory. A field rather than a constant so the tests can
/// point it at a temporary directory and never touch the real one.
dir: PathBuf,
/// The domain files written by the last apply, so a later apply can remove
/// the ones no longer wanted and `revert` can remove them all.
written: Mutex<BTreeSet<String>>,
}
impl Default for ResolverDirPublisher {
fn default() -> Self {
Self::new()
}
}
impl ResolverDirPublisher {
/// A publisher writing to the real `/etc/resolver`.
pub fn new() -> Self {
Self::in_dir(RESOLVER_DIR)
}
/// A publisher writing to a specific directory. For the tests.
fn in_dir(dir: impl Into<PathBuf>) -> Self {
Self {
dir: dir.into(),
written: Mutex::new(BTreeSet::new()),
}
}
fn apply_inner(&self, published: &Published) -> Result<(), PublishError> {
let desired = desired_files(published)?;
// The directory may not exist yet; creating it needs root.
std::fs::create_dir_all(&self.dir).map_err(|err| classify(&self.dir, err))?;
// Remove the files from a previous apply that are no longer wanted.
let previous = lock(&self.written).clone();
for name in previous.difference(&keys(&desired)) {
remove_if_ours(&self.dir.join(name));
}
for (name, body) in &desired {
let path = self.dir.join(name);
// Only ever overwrite a file that is already ours, or a name that
// is free. A resolver a user set up by hand is left in place and
// reported, never clobbered.
if path.exists() && !is_ours(&path) {
return Err(PublishError::Failed(format!(
"{} already exists and was not created by tsunagi; \
remove it or rename the network to publish its resolver",
path.display()
)));
}
std::fs::write(&path, body).map_err(|err| classify(&path, err))?;
}
*lock(&self.written) = keys(&desired);
Ok(())
}
fn revert_inner(&self) {
let written = std::mem::take(&mut *lock(&self.written));
for name in &written {
remove_if_ours(&self.dir.join(name));
}
}
}
impl DnsPublisher for ResolverDirPublisher {
fn name(&self) -> &str {
"resolver-dir"
}
fn apply<'a>(&'a self, published: &'a Published) -> BoxFuture<'a, Result<(), PublishError>> {
// A handful of small file writes, short enough to run in place rather
// than hop to the blocking pool.
Box::pin(async move { self.apply_inner(published) })
}
fn revert(&self) -> BoxFuture<'_, Result<(), PublishError>> {
Box::pin(async move {
self.revert_inner();
Ok(())
})
}
}
/// The resolver files a publication implies: a map of file name to contents.
fn desired_files(published: &Published) -> Result<Vec<(String, String)>, PublishError> {
if published.servers.is_empty() {
return Err(PublishError::Failed(
"no server address to point the resolver at".to_string(),
));
}
let body = file_contents(published);
let mut files = Vec::new();
for domain in &published.domains {
let name = resolver_file_name(domain)?;
files.push((name, body.clone()));
}
Ok(files)
}
/// The contents of one resolver file: the marker, every server, one port.
fn file_contents(published: &Published) -> String {
let mut out = String::from(MARKER);
out.push('\n');
for server in &published.servers {
let ip = match server.ip() {
IpAddr::V4(v4) => v4.to_string(),
IpAddr::V6(v6) => v6.to_string(),
};
out.push_str("nameserver ");
out.push_str(&ip);
out.push('\n');
}
// One port for the file; every server listens on the same one. The `port`
// directive is what lets the server stay off 53.
if let Some(first) = published.servers.first() {
out.push_str("port ");
out.push_str(&first.port().to_string());
out.push('\n');
}
out
}
/// The file name for a domain, rejecting anything that is not a plain label
/// path component: a resolver file name must never escape the directory.
fn resolver_file_name(domain: &str) -> Result<String, PublishError> {
let trimmed = domain.trim_end_matches('.');
if trimmed.is_empty()
|| trimmed.contains('/')
|| trimmed.contains('\\')
|| trimmed.contains(std::path::MAIN_SEPARATOR)
|| trimmed == "."
|| trimmed == ".."
|| trimmed.starts_with('.')
{
return Err(PublishError::Failed(format!(
"cannot make a resolver file name for the domain {domain:?}"
)));
}
Ok(trimmed.to_string())
}
/// Whether a file is one this publisher wrote, by its marker first line.
fn is_ours(path: &Path) -> bool {
match std::fs::read_to_string(path) {
Ok(text) => text.lines().next() == Some(MARKER),
Err(_) => false,
}
}
/// Removes a file, but only if it is one of ours. Best effort.
fn remove_if_ours(path: &Path) {
if is_ours(path)
&& let Err(err) = std::fs::remove_file(path)
&& err.kind() != std::io::ErrorKind::NotFound
{
tracing::debug!(path = %path.display(), %err, "cannot remove the resolver file");
}
}
/// The set of file names in a desired-files list.
fn keys(files: &[(String, String)]) -> BTreeSet<String> {
files.iter().map(|(name, _)| name.clone()).collect()
}
/// Turns a filesystem error into the right kind of publish error: a permission
/// problem is a refusal a person must act on, anything else is a plain failure.
fn classify(path: &Path, err: std::io::Error) -> PublishError {
if err.kind() == std::io::ErrorKind::PermissionDenied {
PublishError::Refused(format!(
"writing {} needs root; start the agent with `sudo` or as a root LaunchDaemon",
path.display()
))
} else {
PublishError::Failed(format!("cannot write {}: {err}", path.display()))
}
}
fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
match mutex.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::SocketAddr;
use super::*;
fn published(domains: &[&str]) -> Published {
Published {
interface: "utun5".into(),
servers: vec![
"10.13.37.1:5353".parse::<SocketAddr>().unwrap(),
"[fd00::1]:5353".parse::<SocketAddr>().unwrap(),
],
domains: domains.iter().map(|d| d.to_string()).collect(),
}
}
#[test]
fn a_file_lists_every_server_and_one_port_under_the_marker() {
let body = file_contents(&published(&["mynet."]));
let mut lines = body.lines();
assert_eq!(lines.next(), Some(MARKER));
assert_eq!(lines.next(), Some("nameserver 10.13.37.1"));
assert_eq!(lines.next(), Some("nameserver fd00::1"));
assert_eq!(lines.next(), Some("port 5353"));
assert_eq!(lines.next(), None);
}
#[test]
fn a_domain_becomes_a_plain_file_name_without_its_trailing_dot() {
assert_eq!(resolver_file_name("mynet.").unwrap(), "mynet");
assert_eq!(resolver_file_name("my.net").unwrap(), "my.net");
}
#[test]
fn a_domain_that_would_escape_the_directory_is_refused() {
assert!(resolver_file_name("../etc/passwd").is_err());
assert!(resolver_file_name("a/b").is_err());
assert!(resolver_file_name("").is_err());
assert!(resolver_file_name(".").is_err());
assert!(resolver_file_name(".hidden").is_err());
}
#[tokio::test]
async fn apply_writes_a_file_per_domain_and_revert_removes_them() {
let dir = tempfile::tempdir().unwrap();
let publisher = ResolverDirPublisher::in_dir(dir.path());
publisher
.apply(&published(&["mynet.", "other."]))
.await
.unwrap();
assert!(dir.path().join("mynet").is_file());
assert!(dir.path().join("other").is_file());
publisher.revert().await.unwrap();
assert!(!dir.path().join("mynet").exists());
assert!(!dir.path().join("other").exists());
}
#[tokio::test]
async fn re_applying_removes_a_domain_that_is_no_longer_wanted() {
let dir = tempfile::tempdir().unwrap();
let publisher = ResolverDirPublisher::in_dir(dir.path());
publisher.apply(&published(&["a.", "b."])).await.unwrap();
publisher.apply(&published(&["a."])).await.unwrap();
assert!(dir.path().join("a").is_file());
assert!(
!dir.path().join("b").exists(),
"the dropped domain's file is removed"
);
}
#[tokio::test]
async fn a_file_a_user_created_is_never_clobbered_or_removed() {
let dir = tempfile::tempdir().unwrap();
let foreign = dir.path().join("mynet");
std::fs::write(&foreign, "nameserver 9.9.9.9\n").unwrap();
let publisher = ResolverDirPublisher::in_dir(dir.path());
let err = publisher.apply(&published(&["mynet."])).await.unwrap_err();
assert!(matches!(err, PublishError::Failed(_)), "{err:?}");
// Untouched.
assert_eq!(
std::fs::read_to_string(&foreign).unwrap(),
"nameserver 9.9.9.9\n"
);
// And revert leaves a foreign file alone even if it is in our set.
publisher.revert().await.unwrap();
assert!(foreign.is_file());
}
}
@@ -28,6 +28,11 @@ mod windows;
#[cfg(all(feature = "dns-publish", target_os = "windows"))]
pub use windows::NrptPublisher;
#[cfg(all(feature = "dns-publish", target_os = "macos"))]
mod macos;
#[cfg(all(feature = "dns-publish", target_os = "macos"))]
pub use macos::ResolverDirPublisher;
mod unsupported;
pub use unsupported::UnsupportedPublisher;
File renamed without changes.
File renamed without changes.
@@ -120,6 +120,13 @@ pub enum Error {
what: String,
},
/// The request is understood and cannot be done as things stand.
#[error("{reason}")]
Rejected {
/// Why, phrased so the message stands on its own.
reason: String,
},
/// Discovery backend failure. Never fatal for the agent.
#[error("discovery error: {0}")]
Discovery(String),
@@ -49,11 +49,16 @@ impl DeviceIdentity {
self.secret.public()
}
/// The raw secret key bytes, for persistence only.
pub(crate) fn secret_bytes(&self) -> [u8; 32] {
/// The raw secret key bytes, for persistence and transport certificate generation.
pub fn secret_bytes(&self) -> [u8; 32] {
self.secret.to_bytes()
}
/// Signs a message with the device's persistent secret key.
pub fn sign(&self, message: &[u8]) -> iroh::Signature {
self.secret.sign(message)
}
/// A clone of the iroh secret key, for endpoint construction only.
pub(crate) fn secret_key(&self) -> SecretKey {
self.secret.clone()
@@ -44,6 +44,21 @@ use windows as transport;
#[cfg(any(unix, windows))]
pub use transport::ControlSocket;
/// Who may open the local control socket.
///
/// The default is owner-only. A system service shares its socket with a GUI or
/// CLI running as another user by granting a group; the agent sets that on the
/// socket when it binds it, so the access is correct from creation rather than
/// patched afterwards.
#[derive(Debug, Clone)]
pub enum ControlSocketAccess {
/// Owner only — mode `0600`. The per-user agent's default.
Private,
/// The owner and members of this group — the socket's group is set to this
/// gid and the mode to `0660`. The caller resolves a group name to its gid.
Group(u32),
}
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::Duration;
@@ -58,6 +73,31 @@ use crate::error::{Error, Result};
/// Largest accepted local control message.
pub const MAX_MESSAGE_LEN: usize = 1024 * 1024;
/// A string the control protocol must carry but that must never appear in a
/// log: it serialises transparently (the text crosses the socket intact) but
/// its `Debug` is redacted. Used for a network secret handed back to a local
/// client that asked to copy it.
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct RedactedText(String);
impl RedactedText {
/// Wraps a string.
pub fn new(text: impl Into<String>) -> Self {
Self(text.into())
}
/// Unwraps to the plain string.
pub fn into_string(self) -> String {
self.0
}
}
impl std::fmt::Debug for RedactedText {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("<redacted>")
}
}
/// Where the control socket for a state directory lives.
///
/// A Unix socket path is limited to around 100 bytes, which a state directory
@@ -82,6 +122,47 @@ pub fn control_socket_path(state_dir: &Path) -> PathBuf {
state_dir.join("agent.sock")
}
/// Where the default system control socket lives on this platform.
///
/// On Windows, the system-wide `%ProgramData%\tsunagi\agent.sock` is used so that
/// any local user, tray application, or CLI tool can manage the running agent
/// without needing to know which user account or service started it.
/// On macOS the agent runs as a root LaunchDaemon, so a fixed system path under
/// `/var/run` is used for the same reason, and it stays short enough for the
/// Unix socket path limit. On other Unix it derives from the default per-user
/// state directory.
pub fn default_control_socket_path() -> PathBuf {
#[cfg(windows)]
{
if let Some(program_data) = std::env::var_os("ProgramData") {
let pd = PathBuf::from(program_data);
if pd.is_absolute() {
return pd.join("tsunagi").join("agent.sock");
}
}
PathBuf::from(r"C:\ProgramData\tsunagi\agent.sock")
}
#[cfg(target_os = "macos")]
{
PathBuf::from("/var/run/tsunagi/agent.sock")
}
#[cfg(all(unix, not(target_os = "macos")))]
{
// Prefer a running system service's socket when it is present, so a
// GUI or CLI reaches the packaged daemon with no configuration; fall
// back to the per-user location otherwise.
let system = PathBuf::from("/run/tsunagi/agent.sock");
if system.exists() {
return system;
}
if let Ok(paths) = crate::config::StoragePaths::user_default() {
control_socket_path(&paths.state_dir)
} else {
system
}
}
}
/// What a client asks for.
///
/// `Debug` is written by hand rather than derived: one of these carries a
@@ -92,6 +173,13 @@ pub fn control_socket_path(state_dir: &Path) -> PathBuf {
pub enum Request {
/// Report what the agent is doing.
Status,
/// Set local broadcast participation in one configured network.
SetBroadcast {
/// Public network identifier.
network_id: String,
/// Whether to originate and accept broadcasts.
enabled: bool,
},
/// Answer to a different name from now on.
///
/// Applied by the running agent rather than written behind its back, so
@@ -126,7 +214,7 @@ pub enum Request {
///
/// Asked of the running agent because that is the only way to add a
/// network to an agent that is already up: the state directory belongs
/// to one live agent, so a second `tsunagi up` cannot.
/// to one live agent, so a second `tsng up` cannot.
Join {
/// The network name.
name: String,
@@ -135,6 +223,32 @@ pub enum Request {
/// It travels over a socket only its owner can open, to the agent
/// that stores it anyway, and never appears in `Debug`.
secret: String,
/// Explicit choice, or preserve the saved setting.
#[serde(default)]
broadcast: Option<bool>,
},
/// Ask for a configured network's shared secret, in its `tsn1…` text form,
/// so a local GUI or script can copy it to re-share. The reply travels only
/// over the owner/group socket and the secret is never logged.
Secret {
/// Which network, by the id text `status` prints.
network_id: String,
},
/// Offer this agent as an exit node in one network, or stop. Off unless
/// asked.
SetExitOffer {
/// Public network identifier.
network_id: String,
/// Whether members may send their internet traffic through this agent.
enabled: bool,
},
/// Send all this device's internet traffic through a member of one
/// network, or back the ordinary way.
SetExitNode {
/// Public network identifier.
network_id: String,
/// The member's endpoint id, or `None` to stop using an exit node.
peer: Option<String>,
},
}
@@ -142,6 +256,10 @@ impl std::fmt::Debug for Request {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Request::Status => f.write_str("Status"),
Request::SetBroadcast {
network_id,
enabled,
} => write!(f, "SetBroadcast {{ {network_id}, enabled: {enabled} }}"),
Request::SetHostname(name) => write!(f, "SetHostname({name})"),
Request::Leave(network) => write!(f, "Leave({network})"),
Request::SetActive { network_id, active } => {
@@ -152,6 +270,14 @@ impl std::fmt::Debug for Request {
write!(f, "Dns {{ enable: {enable}, port: {port:?} }}")
}
Request::Join { name, .. } => write!(f, "Join {{ name: {name}, secret: <redacted> }}"),
Request::Secret { network_id } => write!(f, "Secret {{ {network_id} }}"),
Request::SetExitOffer {
network_id,
enabled,
} => write!(f, "SetExitOffer {{ {network_id}, enabled: {enabled} }}"),
Request::SetExitNode { network_id, peer } => {
write!(f, "SetExitNode {{ {network_id}, peer: {peer:?} }}")
}
}
}
}
@@ -162,6 +288,8 @@ impl std::fmt::Debug for Request {
pub enum Response {
/// A status report.
Status(Box<StatusReport>),
/// Accepted local broadcast participation.
Broadcast(bool),
/// The name the agent now answers to, after reducing it to canonical form.
Hostname(String),
/// A network was left.
@@ -174,6 +302,12 @@ pub enum Response {
///
/// `None` means it is not serving at all.
Dns(Option<DnsReport>),
/// A network's shared secret, redacted in `Debug`.
Secret(RedactedText),
/// Whether this agent now offers itself as an exit node.
ExitOffer(bool),
/// The member this device now sends its internet traffic through.
ExitNode(Option<String>),
/// The request could not be served.
Error(String),
}
@@ -245,6 +379,38 @@ pub struct StatusReport {
pub networks: Vec<NetworkReport>,
/// The local DNS service, when one was asked for.
pub dns: Option<DnsReport>,
/// The state directory the agent is actually using.
///
/// A client that was not told one would look in its own default, which is
/// not the agent's when the agent runs as a service under another user.
pub state_dir: String,
/// The cache directory the agent is actually using.
pub cache_dir: String,
/// The executable the agent runs, which is what needs the capability.
pub program: String,
/// Whether the agent itself can manage the overlay interface.
///
/// It is the agent that creates the interface, so the client's own
/// capabilities say nothing about it.
pub privilege: PrivilegeReport,
/// The version of the agent's software.
pub version: String,
/// The data plane protocols it has, each as `name vN` (wire version).
pub protocols: Vec<String>,
}
/// What the agent found when it checked whether it may manage an interface.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub enum PrivilegeReport {
/// Not checked.
#[default]
Unknown,
/// It can.
Available,
/// It cannot, with what was found.
Missing(String),
/// This platform has no provisioner yet.
Unsupported,
}
/// The local DNS service.
@@ -274,6 +440,9 @@ pub struct DnsReport {
/// One network.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct NetworkReport {
/// Whether this agent participates in LAN broadcasts in this network.
#[serde(default)]
pub broadcast: bool,
/// Network name.
pub name: String,
/// Public network identifier.
@@ -315,6 +484,9 @@ pub struct NetworkReport {
pub range_conflict: Option<String>,
/// The overlay, when a protocol is running one.
pub overlay: Option<OverlayReport>,
/// Exit-node settings: whether this agent offers one, and whether it
/// uses one.
pub exit: ExitReport,
}
/// One member of the network, from signed state.
@@ -326,6 +498,8 @@ pub struct MemberReport {
pub overlay_address_v4: Option<String>,
/// Consecutive failed dial attempts, when this agent is trying to reach it.
pub failed_dials: u32,
/// The last hostname this agent saw it announce, kept while it is away.
pub hostname: Option<String>,
}
/// One control plane peer.
@@ -340,6 +514,8 @@ pub struct PeerReport {
pub transport: String,
/// Round-trip time in milliseconds, when a path is selected.
pub rtt_ms: Option<u64>,
/// The peer offers itself as an exit node in this network.
pub exit_node: bool,
}
/// The WireGuard overlay of one network.
@@ -367,6 +543,50 @@ pub struct OverlayReport {
pub multicast_packets: u64,
/// One destination nobody owned, if there was one.
pub unroutable_sample: Option<String>,
/// The broadcast route and firewall allowance on this host, when the
/// agent installed them. `None` means none are installed.
pub broadcast_rules: Option<HostRulesReport>,
}
/// How one set of exit-node host rules went.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct RuleSetReport {
/// Whether every step took.
pub ok: bool,
/// One line: what is in place, or what is missing and why.
pub detail: String,
}
/// One network's exit-node settings and what they are doing.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct ExitReport {
/// This agent offers itself as an exit node in this network.
pub offering: bool,
/// Masquerading and forwarding rules for members' traffic, when offering.
pub offer_rules: Option<RuleSetReport>,
/// Whether the kernel forwards packets for the overlay interface.
/// `Some(false)` means the rules are in place and nothing passes until it
/// is turned on. `None` when unknown.
pub forwarding: Option<bool>,
/// The member this device sends its internet traffic through, as an
/// endpoint id.
pub via: Option<String>,
/// Whether that member is connected and still offering to be one.
pub via_online: bool,
/// The routes that send this device's traffic through it.
pub client_rules: Option<RuleSetReport>,
}
/// What the agent installed on the host so LAN broadcast reaches the overlay.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct HostRulesReport {
/// Whether both the route and the firewall allowance are in place.
pub ok: bool,
/// One line: `ok (route + firewall)`, or what is missing and why.
pub detail: String,
/// The overlay address the `255.255.255.255` route is bound to. With
/// several networks on one interface only one address can be its source.
pub source: String,
}
/// One overlay peer.
@@ -375,7 +595,9 @@ pub struct OverlayPeerReport {
/// The peer's control plane identity, so a tunnel can be matched to the
/// session and the member it belongs to.
pub endpoint_id: String,
/// The peer's WireGuard public key.
/// Protocol carrying overlay traffic for this peer (e.g. "tcp-tls", "wg-quic").
pub protocol: String,
/// The peer's WireGuard public key (if applicable).
pub public_key: String,
/// The overlay address the network agreed it holds.
pub address: Option<String>,
@@ -387,6 +609,10 @@ pub struct OverlayPeerReport {
pub tx_packets: u64,
/// Packets decrypted from this peer.
pub rx_packets: u64,
/// Plaintext bytes encrypted and sent to this peer.
pub tx_bytes: u64,
/// Plaintext bytes decrypted from this peer.
pub rx_bytes: u64,
/// Data packets dropped: wrong source address, or too large for the path.
pub dropped: u64,
/// WireGuard protocol errors.
@@ -455,10 +681,20 @@ pub trait ReportSource: Send + Sync + 'static {
&self,
_name: String,
_secret: String,
_broadcast: Option<bool>,
) -> BoxFuture<'_, std::result::Result<JoinedReport, String>> {
Box::pin(async move { Err("this agent cannot join a network".to_string()) })
}
/// Persists and applies local broadcast participation.
fn set_broadcast(
&self,
_network_id: String,
_enabled: bool,
) -> BoxFuture<'_, std::result::Result<bool, String>> {
Box::pin(async move { Err("this agent cannot change broadcast participation".into()) })
}
/// Stops serving a network, or starts serving it again.
///
/// Defaulted to a refusal, like the others.
@@ -480,6 +716,39 @@ pub trait ReportSource: Send + Sync + 'static {
) -> BoxFuture<'_, std::result::Result<Option<DnsReport>, String>> {
Box::pin(async move { Err("this agent cannot serve DNS".to_string()) })
}
/// Returns a configured network's shared secret in its `tsn1…` text form.
///
/// Defaulted to a refusal, like the others. An implementor reads it from
/// local state; it must never be logged.
fn network_secret(
&self,
_network_id: String,
) -> BoxFuture<'_, std::result::Result<String, String>> {
Box::pin(async move { Err("this agent cannot read secrets".to_string()) })
}
/// Offers this agent as an exit node in a network, or stops.
///
/// Defaulted to a refusal, like the others.
fn set_exit_offer(
&self,
_network_id: String,
_enabled: bool,
) -> BoxFuture<'_, std::result::Result<bool, String>> {
Box::pin(async move { Err("this agent cannot be an exit node".to_string()) })
}
/// Sends this device's internet traffic through a member, or stops.
///
/// Defaulted to a refusal, like the others.
fn set_exit_node(
&self,
_network_id: String,
_peer: Option<String>,
) -> BoxFuture<'_, std::result::Result<Option<String>, String>> {
Box::pin(async move { Err("this agent cannot use an exit node".to_string()) })
}
}
impl<F> ReportSource for F
@@ -495,7 +764,7 @@ where
///
/// A local answer comes from memory, so anything this slow means the agent is
/// wedged rather than busy. Saying so beats waiting: unbounded, one wedged
/// runtime leaves `tsunagi status` hanging with nothing on screen and no way
/// runtime leaves `tsng status` hanging with nothing on screen and no way
/// out but Ctrl-C.
pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5);
@@ -510,7 +779,7 @@ pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5);
///
/// Bump it whenever [`Request`], [`Response`] or anything they contain
/// changes shape.
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 13]);
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 17]);
/// Reads one request off an accepted stream, answers it, writes the response.
///
@@ -535,6 +804,13 @@ where
};
let response = match request {
Request::Status => Response::Status(Box::new(source.report().await)),
Request::SetBroadcast {
network_id,
enabled,
} => match source.set_broadcast(network_id, enabled).await {
Ok(enabled) => Response::Broadcast(enabled),
Err(error) => Response::Error(error),
},
Request::SetHostname(hostname) => match source.set_hostname(hostname).await {
Ok(accepted) => Response::Hostname(accepted),
Err(reason) => Response::Error(reason),
@@ -543,7 +819,11 @@ where
Ok(report) => Response::Left(report),
Err(reason) => Response::Error(reason),
},
Request::Join { name, secret } => match source.join(name, secret).await {
Request::Join {
name,
secret,
broadcast,
} => match source.join(name, secret, broadcast).await {
Ok(report) => Response::Joined(report),
Err(reason) => Response::Error(reason),
},
@@ -557,6 +837,23 @@ where
Ok(report) => Response::Dns(report),
Err(reason) => Response::Error(reason),
},
Request::Secret { network_id } => match source.network_secret(network_id).await {
Ok(secret) => Response::Secret(RedactedText::new(secret)),
Err(reason) => Response::Error(reason),
},
Request::SetExitOffer {
network_id,
enabled,
} => match source.set_exit_offer(network_id, enabled).await {
Ok(enabled) => Response::ExitOffer(enabled),
Err(reason) => Response::Error(reason),
},
Request::SetExitNode { network_id, peer } => {
match source.set_exit_node(network_id, peer).await {
Ok(peer) => Response::ExitNode(peer),
Err(reason) => Response::Error(reason),
}
}
};
write_message(&mut stream, &response).await
}
@@ -636,9 +933,21 @@ pub async fn join_network(
path: impl AsRef<Path>,
name: &str,
secret: &str,
) -> Result<JoinedReport> {
let path = path.as_ref();
join_network_with_broadcast(path, name, secret, None).await
}
/// Joins with an explicit per-network broadcast choice.
pub async fn join_network_with_broadcast(
path: impl AsRef<Path>,
name: &str,
secret: &str,
broadcast: Option<bool>,
) -> Result<JoinedReport> {
let path = path.as_ref();
let request = Request::Join {
broadcast,
name: name.to_string(),
secret: secret.to_string(),
};
@@ -649,6 +958,24 @@ pub async fn join_network(
}
}
/// Changes one network's broadcast participation while the agent runs.
pub async fn set_broadcast(
path: impl AsRef<Path>,
network_id: &str,
enabled: bool,
) -> Result<bool> {
let path = path.as_ref();
let request = Request::SetBroadcast {
network_id: network_id.to_owned(),
enabled,
};
match exchange(path, &request, EXCHANGE_TIMEOUT).await? {
Response::Broadcast(enabled) => Ok(enabled),
Response::Error(message) => Err(Error::Storage(message)),
other => Err(Error::Storage(format!("unexpected answer: {other:?}"))),
}
}
/// Asks a running agent to stop serving a network, or to serve it again.
pub async fn set_active(
path: impl AsRef<Path>,
@@ -681,6 +1008,56 @@ pub async fn set_dns(
}
}
/// Offers the running agent as an exit node in one network, or stops.
pub async fn set_exit_offer(
path: impl AsRef<Path>,
network_id: &str,
enabled: bool,
) -> Result<bool> {
let request = Request::SetExitOffer {
network_id: network_id.to_owned(),
enabled,
};
match exchange(path.as_ref(), &request, EXCHANGE_TIMEOUT).await? {
Response::ExitOffer(enabled) => Ok(enabled),
Response::Error(message) => Err(Error::Storage(message)),
other => Err(Error::Storage(format!("unexpected answer: {other:?}"))),
}
}
/// Sends the running agent's internet traffic through a member of one
/// network, or back the ordinary way with `None`.
pub async fn set_exit_node(
path: impl AsRef<Path>,
network_id: &str,
peer: Option<&str>,
) -> Result<Option<String>> {
let request = Request::SetExitNode {
network_id: network_id.to_owned(),
peer: peer.map(str::to_owned),
};
match exchange(path.as_ref(), &request, EXCHANGE_TIMEOUT).await? {
Response::ExitNode(peer) => Ok(peer),
Response::Error(message) => Err(Error::Storage(message)),
other => Err(Error::Storage(format!("unexpected answer: {other:?}"))),
}
}
/// Asks a running agent for a configured network's shared secret, in its
/// `tsn1…` text form, so a local client can copy it. The secret is never
/// logged by this path.
pub async fn network_secret(path: impl AsRef<Path>, network_id: &str) -> Result<String> {
let path = path.as_ref();
let request = Request::Secret {
network_id: network_id.to_string(),
};
match exchange(path, &request, EXCHANGE_TIMEOUT).await? {
Response::Secret(secret) => Ok(secret.into_string()),
Response::Error(reason) => Err(Error::Storage(reason)),
other => Err(Error::Storage(format!("unexpected answer: {other:?}"))),
}
}
pub(crate) async fn write_message<S, T>(stream: &mut S, value: &T) -> Result<()>
where
S: AsyncWrite + Unpin,
@@ -22,8 +22,8 @@ use crate::error::{Error, Result};
// The transport-agnostic surface, re-exported so this module is a complete
// view of the local control interface on its own.
pub use super::{
CONTROL_PROTOCOL, EXCHANGE_TIMEOUT, ReportSource, is_serving, join_network, leave_network,
request_status, set_active, set_dns, set_hostname,
CONTROL_PROTOCOL, ControlSocketAccess, EXCHANGE_TIMEOUT, ReportSource, is_serving,
join_network, leave_network, request_status, set_active, set_dns, set_hostname,
};
/// Serves the local control interface on a Unix socket.
@@ -39,11 +39,15 @@ impl ControlSocket {
/// A socket file left behind by a crashed agent is replaced, but only
/// after checking that nothing is listening on it, so two live agents
/// never fight over one path.
pub async fn bind(path: impl AsRef<Path>, source: Arc<dyn ReportSource>) -> Result<Self> {
pub async fn bind(
path: impl AsRef<Path>,
source: Arc<dyn ReportSource>,
access: ControlSocketAccess,
) -> Result<Self> {
let path = path.as_ref().to_path_buf();
if let Some(parent) = path.parent() {
crate::storage::create_dir(parent)?;
create_parent(parent, &access)?;
}
if path.exists() {
@@ -61,7 +65,7 @@ impl ControlSocket {
path: path.clone(),
source,
})?;
restrict(&path)?;
apply_access(&path, &access)?;
let task = tokio::spawn(serve(listener, source));
Ok(Self {
@@ -94,9 +98,48 @@ impl Drop for ControlSocket {
}
}
fn restrict(path: &Path) -> Result<()> {
/// Creates the socket's parent directory.
///
/// A group-shared socket must sit in a directory the group can traverse, so it
/// is `0755`; a private socket keeps the owner-only state-directory default.
fn create_parent(parent: &Path, access: &ControlSocketAccess) -> Result<()> {
match access {
ControlSocketAccess::Private => crate::storage::create_dir(parent),
ControlSocketAccess::Group(_) => {
std::fs::create_dir_all(parent).map_err(|source| Error::Io {
path: parent.to_path_buf(),
source,
})?;
set_mode(parent, 0o755)
}
}
}
/// Applies the access policy to the bound socket.
///
/// Private is owner-only `0600`. Group sets the socket's group to the gid and
/// the mode to `0660`; if the group cannot be set — the agent is not a member,
/// say — it falls back to owner-only rather than leaving the group wrong, so
/// the socket is never more open than intended.
fn apply_access(path: &Path, access: &ControlSocketAccess) -> Result<()> {
match access {
ControlSocketAccess::Private => set_mode(path, 0o600),
ControlSocketAccess::Group(gid) => {
if let Err(source) = std::os::unix::fs::chown(path, None, Some(*gid)) {
tracing::warn!(
path = %path.display(), gid, %source,
"cannot set the control socket group; keeping it owner-only"
);
return set_mode(path, 0o600);
}
set_mode(path, 0o660)
}
}
}
fn set_mode(path: &Path, mode: u32) -> Result<()> {
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)).map_err(|source| {
std::fs::set_permissions(path, std::fs::Permissions::from_mode(mode)).map_err(|source| {
Error::Io {
path: path.to_path_buf(),
source,
@@ -170,7 +213,9 @@ mod tests {
let source: Arc<dyn ReportSource> = Arc::new(|| -> BoxFuture<'static, StatusReport> {
Box::pin(async { StatusReport::default() })
});
let control = ControlSocket::bind(&path, source).await.unwrap();
let control = ControlSocket::bind(&path, source, ControlSocketAccess::Private)
.await
.unwrap();
let answer = exchange(&path, &Request::Status, EXCHANGE_TIMEOUT)
.await
@@ -18,15 +18,16 @@
//!
//! # Who may connect
//!
//! The pipe rejects clients from other machines, and takes the process's
//! default security, under which the creating user has full access. Narrowing
//! that further with an explicit ACL needs a raw security-descriptor call this
//! crate forbids, so it is not attempted; on a single-user machine, and against
//! other unprivileged users, the default is the practical equivalent of the
//! owner-only Unix socket. There is no leftover to clean up: a pipe exists only
//! while its server does.
//! Commands must run as the same Windows user with the same elevation as the
//! agent. A permission denial is reported as such, never as an absent agent.
//! The pipe rejects clients from other machines (`reject_remote_clients(true)`).
//! An explicit SDDL security descriptor grants Full Control to Administrators
//! and LocalSystem, while granting Read and Write (`GRGW`) to all local users
//! with a Medium Integrity SACL label (`S:(ML;;NW;;;ME)`). This allows unprivileged
//! user-space tools (such as tray applications, status monitors, or CLI commands
//! launched by regular users) to connect to the agent even when the agent was
//! started by an elevated Administrator or as a system service.
//! A permission denial is reported as such, never as an absent agent.
#![allow(unsafe_code)]
use std::path::{Path, PathBuf};
use std::sync::Arc;
@@ -39,8 +40,8 @@ use tokio::task::JoinHandle;
use crate::error::{Error, Result};
pub use super::{
CONTROL_PROTOCOL, EXCHANGE_TIMEOUT, ReportSource, is_serving, join_network, leave_network,
request_status, set_active, set_dns, set_hostname,
CONTROL_PROTOCOL, ControlSocketAccess, EXCHANGE_TIMEOUT, ReportSource, is_serving,
join_network, leave_network, request_status, set_active, set_dns, set_hostname,
};
/// `ERROR_ACCESS_DENIED`: creating a first instance finds an existing pipe,
@@ -50,6 +51,91 @@ const ERROR_ACCESS_DENIED: i32 = 5;
/// is there; the client only has to wait for a free instance.
const ERROR_PIPE_BUSY: i32 = 231;
/// RAII wrapper for Windows named pipe security attributes with permissive local ACL.
///
/// SDDL specification:
/// - `D:(A;;GRGW;;;WD)`: DACL allows Generic Read and Generic Write to Everyone (`WD`).
/// Full control (`GA`) remains reserved to Administrators and LocalSystem.
/// - `(A;;GA;;;BA)`: DACL allows Generic All to Builtin Administrators (`BA`).
/// - `(A;;GA;;;SY)`: DACL allows Generic All to LocalSystem (`SY`).
/// - `S:(ML;;NW;;;ME)`: SACL Mandatory Label with Medium Integrity (`ME`) and No Write Up (`NW`).
/// This allows unprivileged user-space processes (Medium Integrity) to write to the
/// named pipe even when the agent runs as an elevated Administrator (High Integrity).
struct PipeSecurityAttributes {
attrs: windows_sys::Win32::Security::SECURITY_ATTRIBUTES,
}
impl std::fmt::Debug for PipeSecurityAttributes {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("PipeSecurityAttributes")
.field("nLength", &self.attrs.nLength)
.field("lpSecurityDescriptor", &self.attrs.lpSecurityDescriptor)
.field("bInheritHandle", &self.attrs.bInheritHandle)
.finish()
}
}
impl PipeSecurityAttributes {
fn new() -> Result<Self> {
use std::ffi::OsStr;
use std::os::windows::ffi::OsStrExt;
use windows_sys::Win32::Security::Authorization::{
ConvertStringSecurityDescriptorToSecurityDescriptorW, SDDL_REVISION_1,
};
const SDDL: &str = "D:(A;;GRGW;;;WD)(A;;GA;;;BA)(A;;GA;;;SY)S:(ML;;NW;;;ME)";
let wide: Vec<u16> = OsStr::new(SDDL)
.encode_wide()
.chain(std::iter::once(0))
.collect();
let mut p_sd = std::ptr::null_mut();
// SAFETY: We pass a valid null-terminated wide string, standard revision 1,
// and a valid pointer to receive the descriptor.
let success = unsafe {
ConvertStringSecurityDescriptorToSecurityDescriptorW(
wide.as_ptr(),
SDDL_REVISION_1,
&mut p_sd,
std::ptr::null_mut(),
)
};
if success == 0 || p_sd.is_null() {
let err = std::io::Error::last_os_error();
return Err(Error::Storage(format!(
"cannot create pipe security descriptor: {err}"
)));
}
let attrs = windows_sys::Win32::Security::SECURITY_ATTRIBUTES {
nLength: std::mem::size_of::<windows_sys::Win32::Security::SECURITY_ATTRIBUTES>()
as u32,
lpSecurityDescriptor: p_sd,
bInheritHandle: 0,
};
Ok(Self { attrs })
}
fn as_raw_mut(&mut self) -> *mut std::ffi::c_void {
&mut self.attrs as *mut _ as *mut std::ffi::c_void
}
}
impl Drop for PipeSecurityAttributes {
fn drop(&mut self) {
if !self.attrs.lpSecurityDescriptor.is_null() {
// SAFETY: The descriptor was allocated by ConvertStringSecurityDescriptorToSecurityDescriptorW
// and must be freed with LocalFree.
unsafe {
windows_sys::Win32::Foundation::LocalFree(self.attrs.lpSecurityDescriptor);
}
self.attrs.lpSecurityDescriptor = std::ptr::null_mut();
}
}
}
/// Serves the local control interface on a named pipe.
#[derive(Debug)]
pub struct ControlSocket {
@@ -63,15 +149,27 @@ impl ControlSocket {
/// Claiming the first instance of the name is what detects a second agent:
/// if the name already exists, the create is refused and that is reported
/// as the state being locked, so two live agents never share one pipe.
pub async fn bind(path: impl AsRef<Path>, source: Arc<dyn ReportSource>) -> Result<Self> {
///
/// `_access` is accepted for parity with the Unix socket; granting a group
/// here means a pipe security descriptor and is a separate follow-up, so for
/// now the pipe keeps its default (creator plus the built-in access the
/// `PipeSecurityAttributes` set).
pub async fn bind(
path: impl AsRef<Path>,
source: Arc<dyn ReportSource>,
_access: ControlSocketAccess,
) -> Result<Self> {
let path = path.as_ref().to_path_buf();
let name = pipe_name(&path);
let server = match ServerOptions::new()
.first_pipe_instance(true)
.reject_remote_clients(true)
.create(&name)
{
let mut sec_attrs = PipeSecurityAttributes::new()?;
// SAFETY: sec_attrs points to valid initialized SECURITY_ATTRIBUTES.
let server = match unsafe {
ServerOptions::new()
.first_pipe_instance(true)
.reject_remote_clients(true)
.create_with_security_attributes_raw(&name, sec_attrs.as_raw_mut())
} {
Ok(server) => server,
Err(err) if err.raw_os_error() == Some(ERROR_ACCESS_DENIED) => {
return Err(Error::StateLocked { path });
@@ -112,7 +210,7 @@ impl Drop for ControlSocket {
/// Accepts one client at a time, keeping a fresh instance ready for the next.
///
/// A named pipe server instance serves a single client, so a new instance is
/// created as soon as one is taken — otherwise a second `tsunagi status` while
/// created as soon as one is taken — otherwise a second `tsng status` while
/// the first is mid-flight would find nothing listening.
async fn serve(
name: String,
@@ -155,10 +253,19 @@ async fn serve(
/// Creates the next pipe instance, or `None` if the name can no longer be
/// served.
fn next_instance(name: &str) -> Option<tokio::net::windows::named_pipe::NamedPipeServer> {
match ServerOptions::new()
.reject_remote_clients(true)
.create(name)
{
let mut sec_attrs = match PipeSecurityAttributes::new() {
Ok(attrs) => attrs,
Err(err) => {
tracing::debug!(%err, "cannot build pipe security attributes for next instance");
return None;
}
};
// SAFETY: sec_attrs points to valid initialized SECURITY_ATTRIBUTES.
match unsafe {
ServerOptions::new()
.reject_remote_clients(true)
.create_with_security_attributes_raw(name, sec_attrs.as_raw_mut())
} {
Ok(server) => Some(server),
Err(err) => {
tracing::debug!(%err, "cannot create the next control pipe instance");
@@ -187,7 +294,7 @@ pub(crate) async fn connect(path: &Path) -> Result<NamedPipeClient> {
format!(
"the agent's control pipe exists, but Windows denied access. \
Run this command as the same Windows user and with the same \
elevation as `tsunagi up` (use an administrator terminal if \
elevation as `tsng up` (use an administrator terminal if \
the agent is elevated). Windows error: {err}"
),
),
File renamed without changes.
@@ -240,6 +240,57 @@ impl EndpointAdapter {
TransportPolicy::N0Defaults => builder.preset(presets::N0),
};
if matches!(
config.transport,
TransportPolicy::N0Defaults | TransportPolicy::DirectOnly
) {
let filter = iroh::address_lookup::AddrFilter::new(|addrs| {
let mut overlay_ips = std::collections::HashSet::new();
for iface in netdev::get_interfaces() {
let name = iface.name.to_lowercase();
let friendly = iface
.friendly_name
.as_deref()
.map(str::to_lowercase)
.unwrap_or_default();
let desc = iface
.description
.as_deref()
.map(str::to_lowercase)
.unwrap_or_default();
if name.starts_with("tsun")
|| name.contains("tsunagi")
|| name.contains("wintun")
|| friendly.starts_with("tsun")
|| friendly.contains("tsunagi")
|| friendly.contains("wintun")
|| desc.contains("wintun")
|| desc.contains("tsunagi")
{
for ip in iface.ipv4 {
overlay_ips.insert(std::net::IpAddr::V4(ip.addr()));
}
for ip in iface.ipv6 {
overlay_ips.insert(std::net::IpAddr::V6(ip.addr()));
}
}
}
if overlay_ips.is_empty() {
return std::borrow::Cow::Borrowed(addrs);
}
let filtered: Vec<iroh::TransportAddr> = addrs
.iter()
.filter(|addr| match addr {
iroh::TransportAddr::Ip(sock) => !overlay_ips.contains(&sock.ip()),
_ => true,
})
.cloned()
.collect();
std::borrow::Cow::Owned(filtered)
});
builder = builder.addr_filter(filter);
}
#[cfg(feature = "testing")]
if let Some(transport) = &config.test_quic_transport {
builder = builder.transport_config(transport.clone());
@@ -259,6 +310,91 @@ impl EndpointAdapter {
.await
.map_err(|err| Error::Endpoint(format!("cannot bind endpoint: {err}")))?;
tracing::debug!(
endpoint = %endpoint.id().fmt_short(),
bound_sockets = ?endpoint.bound_sockets(),
"iroh endpoint bound locally"
);
let ep_id = endpoint.id();
let mut addr_watcher = endpoint.watch_addr();
let mut relay_watcher = endpoint.home_relay_status();
let close_token = endpoint.clone();
tokio::spawn(async move {
use iroh::Watcher;
let mut last_addrs: Vec<SocketAddr> = Vec::new();
let mut last_relays: Vec<String> = Vec::new();
let warn_timer = tokio::time::sleep(Duration::from_secs(12));
tokio::pin!(warn_timer);
let mut warned = false;
loop {
tokio::select! {
biased;
_ = close_token.closed() => {
break;
}
() = &mut warn_timer, if !warned => {
warned = true;
if last_addrs.is_empty() && !last_relays.is_empty() {
tracing::warn!(
endpoint = %ep_id.fmt_short(),
relays = ?last_relays,
"No direct reflexive or UPnP addresses discovered after probe; communications will rely on relay servers"
);
} else if last_addrs.is_empty() && last_relays.is_empty() {
tracing::warn!(
endpoint = %ep_id.fmt_short(),
"No reachability discovered (neither direct reflexive/UPnP addresses nor relay servers available)"
);
}
}
res = addr_watcher.updated() => {
let addr = match res {
Ok(a) => a,
Err(_) => break,
};
let addrs: Vec<SocketAddr> = addr.ip_addrs().copied().collect();
let relays: Vec<String> = addr.relay_urls().map(|u| u.to_string()).collect();
if addrs != last_addrs || relays != last_relays {
tracing::debug!(
endpoint = %ep_id.fmt_short(),
direct_addrs = ?addrs,
relays = ?relays,
"Discovered endpoint reachability (STUN/UPnP/local/relay)"
);
last_addrs = addrs;
last_relays = relays;
}
}
res = relay_watcher.updated() => {
let relays = match res {
Ok(r) => r,
Err(_) => break,
};
for relay in &relays {
if relay.is_connected() {
tracing::debug!(
endpoint = %ep_id.fmt_short(),
relay = %relay.url(),
"Connected to home relay server"
);
} else if let Some(err) = relay.last_error() {
tracing::debug!(
endpoint = %ep_id.fmt_short(),
relay = %relay.url(),
error = %err,
"Relay connection attempt failed"
);
}
}
}
}
}
});
Ok(Self { endpoint })
}
@@ -0,0 +1,231 @@
//! Scoped LAN discovery fanout at IP ingress. A domain is one logical network;
//! routed copies use ordinary encrypted peer links and are never reflooded.
//! Future subnet exporters must supply explicitly authorized ingress domains
//! here, not teach the encrypted transit router to inspect application bytes.
use super::router::{NetworkRoutes, Route};
use crate::NetworkId;
use iroh::EndpointId;
use std::collections::{HashMap, HashSet};
use std::net::Ipv4Addr;
use std::sync::Arc;
/// Local participation and the currently authenticated participants in a domain.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BroadcastPolicy {
/// Whether this agent originates and accepts LAN broadcasts in this network.
pub enabled: bool,
/// Live members explicitly advertising that they accept broadcasts.
pub peers: HashSet<EndpointId>,
}
impl Default for BroadcastPolicy {
fn default() -> Self {
Self {
enabled: true,
peers: HashSet::new(),
}
}
}
#[derive(Debug)]
struct Domain {
network: NetworkId,
directed: Ipv4Addr,
enabled: bool,
recipients: Arc<[Route]>,
members: HashSet<EndpointId>,
}
/// Immutable ingress-domain index rebuilt with address/policy changes.
#[derive(Debug, Default)]
pub(crate) struct BroadcastTable {
sources: HashMap<Ipv4Addr, Arc<Domain>>,
networks: HashMap<NetworkId, Arc<Domain>>,
destinations: HashSet<Ipv4Addr>,
}
impl BroadcastTable {
pub fn build(networks: &HashMap<NetworkId, NetworkRoutes>) -> Self {
let mut table = Self::default();
for (&network, routes) in networks {
let (Some(range), Some(local)) = (routes.range, routes.local) else {
continue;
};
if range.prefix_len > 30 {
continue;
}
let directed = Ipv4Addr::from(u32::from(range.base) | (u32::MAX >> range.prefix_len));
let mut recipients: Vec<_> = routes
.peers
.iter()
.filter(|(_, peer)| routes.broadcast.peers.contains(peer))
.map(|(_, peer)| Route {
network,
peer: *peer,
})
.collect();
recipients.sort_by_key(|route| route.peer);
recipients.dedup();
let domain = Arc::new(Domain {
network,
directed,
enabled: routes.broadcast.enabled,
members: recipients.iter().map(|route| route.peer).collect(),
recipients: recipients.into(),
});
table.sources.insert(local, domain.clone());
table.networks.insert(network, domain);
table.destinations.insert(directed);
}
table
}
pub fn is_destination(&self, destination: Ipv4Addr) -> bool {
destination.is_broadcast() || self.destinations.contains(&destination)
}
pub fn outgoing(&self, source: Ipv4Addr, destination: Ipv4Addr) -> Option<Arc<[Route]>> {
let domain = self.sources.get(&source)?;
(domain.enabled && (destination.is_broadcast() || destination == domain.directed))
.then(|| domain.recipients.clone())
}
pub fn accepts(&self, network: NetworkId, peer: EndpointId, destination: Ipv4Addr) -> bool {
self.networks.get(&network).is_some_and(|domain| {
domain.network == network
&& domain.enabled
&& domain.members.contains(&peer)
&& (destination.is_broadcast() || destination == domain.directed)
})
}
}
/// Validates IPv4/UDP lengths before broadcast fanout. No game-specific ports,
/// payload rewriting or checksum changes. IP fragments retain their bytes and
/// are reassembled by the destination OS; even later fragments have protocol 17.
pub(crate) fn valid_udp(packet: &[u8]) -> bool {
if packet.len() < 20 || packet[0] >> 4 != 4 || packet[9] != 17 {
return false;
}
let header = usize::from(packet[0] & 15) * 4;
let total = usize::from(u16::from_be_bytes([packet[2], packet[3]]));
if header < 20 || total <= header || total > packet.len() {
return false;
}
let fragment = u16::from_be_bytes([packet[6], packet[7]]);
if fragment & 0x1fff != 0 {
return true;
}
if total < header + 8 {
return false;
}
let udp = usize::from(u16::from_be_bytes([packet[header + 4], packet[header + 5]]));
udp >= 8
&& if fragment & 0x2000 != 0 {
udp >= total - header
} else {
udp == total - header
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
fn id(name: &str) -> NetworkId {
NetworkKeys::derive(
&NetworkName::new(name).unwrap(),
&NetworkSecret::from_bytes([4; 32]).unwrap(),
)
.network_id()
}
#[test]
fn limited_and_directed_broadcasts_never_cross_domains_or_target_disabled_peers() {
let peer = iroh::SecretKey::from_bytes(&[8; 32]).public();
let a = NetworkRoutes {
exit: Default::default(),
range: Some("10.0.0.0/24".parse().unwrap()),
local: Some("10.0.0.1".parse().unwrap()),
peers: vec![("10.0.0.2".parse().unwrap(), peer)],
broadcast: BroadcastPolicy {
enabled: true,
peers: HashSet::from([peer]),
},
};
let b = NetworkRoutes {
exit: Default::default(),
range: Some("10.1.0.0/24".parse().unwrap()),
local: Some("10.1.0.1".parse().unwrap()),
peers: vec![("10.1.0.2".parse().unwrap(), peer)],
broadcast: BroadcastPolicy::default(),
};
let table = BroadcastTable::build(&HashMap::from([(id("a"), a.clone()), (id("b"), b)]));
let routes = table
.outgoing(a.local.unwrap(), Ipv4Addr::BROADCAST)
.unwrap();
assert_eq!(
&*routes,
&[Route {
network: id("a"),
peer
}]
);
assert!(
table
.outgoing(a.local.unwrap(), "10.1.0.255".parse().unwrap())
.is_none()
);
assert!(
table
.outgoing("192.168.1.2".parse().unwrap(), Ipv4Addr::BROADCAST)
.is_none()
);
assert!(
table
.outgoing("10.1.0.1".parse().unwrap(), Ipv4Addr::BROADCAST)
.unwrap()
.is_empty()
);
assert!(!table.accepts(id("a"), peer, "10.1.0.255".parse().unwrap()));
assert!(!table.accepts(id("b"), peer, Ipv4Addr::BROADCAST));
assert!(table.accepts(id("a"), peer, Ipv4Addr::BROADCAST));
let disabled = NetworkRoutes {
exit: Default::default(),
broadcast: BroadcastPolicy {
enabled: false,
..a.broadcast.clone()
},
..a.clone()
};
let table = BroadcastTable::build(&HashMap::from([(id("a"), disabled)]));
assert!(
table
.outgoing(a.local.unwrap(), Ipv4Addr::BROADCAST)
.is_none()
);
assert!(!table.accepts(id("a"), peer, Ipv4Addr::BROADCAST));
}
#[test]
fn malformed_udp_is_not_amplified_but_ipv4_fragments_are_supported() {
let mut packet = vec![0u8; 36];
packet[0] = 0x45;
packet[3] = 36;
packet[9] = 17;
packet[25] = 16;
assert!(valid_udp(&packet));
for length in 0..36 {
assert!(!valid_udp(&packet[..length]));
}
packet[9] = 6;
assert!(!valid_udp(&packet));
packet[9] = 17;
packet[0] = 0x44;
assert!(!valid_udp(&packet));
packet[0] = 0x45;
packet[25] = 8;
assert!(!valid_udp(&packet));
packet[6] = 0x20;
packet[25] = 64;
assert!(valid_udp(&packet));
packet[6] = 0;
packet[7] = 2;
assert!(valid_udp(&packet));
}
}
+32
View File
@@ -0,0 +1,32 @@
//! Exit-node host rules: what makes this device an exit node for others, and
//! what sends this device's own traffic through one.
//!
//! The packet side lives in [`super::router`]; this is the side the operating
//! system sees. It is kept apart from the broadcast rules, which have a
//! different lifetime and different tags.
/// How one set of host rules went.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct RuleSetReport {
/// Whether every step took.
pub ok: bool,
/// One line: what is in place, or what is missing and why.
pub detail: String,
}
/// What the host rules of one network's exit-node settings are doing.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct ExitRulesReport {
/// Masquerading and forwarding for members' traffic, when this agent
/// offers itself as an exit node here. `None` means none are wanted.
pub offer: Option<RuleSetReport>,
/// Whether the kernel forwards packets for the overlay interface.
///
/// `Some(false)` means the rules are in place but nothing passes until
/// forwarding is turned on, which the agent never does itself. `None`
/// when it could not be read.
pub forwarding: Option<bool>,
/// The routes that send this device's traffic through its exit node.
/// `None` means it uses none through this network.
pub client: Option<RuleSetReport>,
}
@@ -0,0 +1,231 @@
//! Host rules for exit nodes: what the operating system needs for this agent
//! to carry other members' internet traffic, and for this device's own
//! traffic to go through a member.
//!
//! Two independent halves, planned together because one interface carries
//! both:
//!
//! * **Offering** — one set of rules per network that offers this agent as an
//! exit node, chosen by that network's range: masquerade the range out of
//! every other interface and let it through the forward chain. A network
//! that does not offer has no rules, so its members' packets are dropped by
//! the firewall as well as by the agent.
//! * **Using** — policy routing that sends everything the host does not have
//! a more specific route for into the overlay interface, while the agent's
//! own sockets keep the ordinary route (see `linux_exit`).
//!
//! Nothing a peer announces becomes a host setting. Offering is the owner's
//! switch; using is the user's choice of a member. Every object is tagged,
//! reversible, and removed when it is no longer wanted, when the network
//! leaves or stops, and when the agent stops.
use std::sync::{Arc, Mutex};
use crate::BoxFuture;
use crate::state::Ipv4Range;
use super::RuleOutcome;
/// What the host should hold for one interface's exit settings.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct ExitHostPlan {
/// The overlay interface.
pub interface: String,
/// The ranges of every network that offers this agent as an exit node.
pub offer: Vec<Ipv4Range>,
/// Whether this device sends its internet traffic through a member.
pub client: bool,
}
impl ExitHostPlan {
/// Whether there is nothing to hold.
pub fn is_empty(&self) -> bool {
self.offer.is_empty() && !self.client
}
}
/// What applying a plan achieved.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct ExitHostReport {
/// One outcome per offered range: its masquerade and forward rules.
pub offer: Vec<(Ipv4Range, RuleOutcome)>,
/// Whether the kernel forwards packets for the interface. `None` when it
/// could not be read. The agent only reads this: turning forwarding on is
/// the owner's decision.
pub forwarding: Option<bool>,
/// The routes and rules that send this device's traffic through the
/// interface, when the plan asked for them.
pub client: Option<RuleOutcome>,
}
/// Installs and removes exit-node rules for an overlay interface.
///
/// Like the broadcast rules, no call fails as a whole: a step that cannot be
/// done is in the report, because the caller's only useful response is to
/// say so.
pub trait ExitHostRules: Send + Sync + std::fmt::Debug + 'static {
/// Makes the host match the plan, replacing what this agent put there
/// for the interface before.
fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport>;
/// Removes everything this agent put there for the interface. Succeeds
/// when nothing is there: it runs on the shutdown path, and once at
/// startup to take away what a crashed run left.
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()>;
}
/// The tag every firewall object of one offered range carries.
pub fn exit_tag(interface: &str, range: &Ipv4Range) -> String {
format!("{}{range}", exit_tag_prefix(interface))
}
/// The start of every tag for an interface; what cleanup selects by.
pub fn exit_tag_prefix(interface: &str) -> String {
format!("tsunagi-exit:{interface}:")
}
/// The one-line detail for a set of offer outcomes.
pub fn offer_detail(outcome: &RuleOutcome, range: &Ipv4Range) -> String {
match outcome {
RuleOutcome::Applied => format!("masquerade + forward for {range} in place"),
RuleOutcome::Failed(reason) => reason.clone(),
}
}
/// An in-memory host for exit rules, for tests: nothing is touched.
#[derive(Debug, Clone, Default)]
pub struct MockExitRules {
state: Arc<Mutex<Option<ExitHostPlan>>>,
calls: Arc<Mutex<Vec<String>>>,
forwarding: Arc<Mutex<Option<bool>>>,
failure: Arc<Mutex<Option<String>>>,
}
impl MockExitRules {
/// An empty host with forwarding turned on.
pub fn new() -> Self {
let rules = Self::default();
*lock(&rules.forwarding) = Some(true);
rules
}
/// What the host reports for kernel forwarding from now on.
pub fn set_forwarding(&self, forwarding: Option<bool>) {
*lock(&self.forwarding) = forwarding;
}
/// Makes every step fail with this reason from now on.
pub fn fail_with(&self, reason: Option<String>) {
*lock(&self.failure) = reason;
}
/// What is installed now.
pub fn installed(&self) -> Option<ExitHostPlan> {
lock(&self.state).clone()
}
/// Every `apply:<interface>` / `clear:<interface>` call, in order.
pub fn calls(&self) -> Vec<String> {
lock(&self.calls).clone()
}
}
fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
match mutex.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
impl ExitHostRules for MockExitRules {
fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport> {
Box::pin(async move {
lock(&self.calls).push(format!("apply:{}", plan.interface));
let outcome = match lock(&self.failure).clone() {
Some(reason) => RuleOutcome::Failed(reason),
None => {
*lock(&self.state) = Some(plan.clone());
RuleOutcome::Applied
}
};
ExitHostReport {
offer: plan
.offer
.iter()
.map(|range| (*range, outcome.clone()))
.collect(),
forwarding: *lock(&self.forwarding),
client: plan.client.then_some(outcome),
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
lock(&self.calls).push(format!("clear:{interface}"));
*lock(&self.state) = None;
})
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
fn range() -> Ipv4Range {
"10.13.37.0/24".parse().unwrap()
}
#[test]
fn tags_name_the_interface_and_the_range() {
assert_eq!(
exit_tag("tsun0", &range()),
"tsunagi-exit:tsun0:10.13.37.0/24"
);
assert!(exit_tag("tsun0", &range()).starts_with(&exit_tag_prefix("tsun0")));
// Another interface's tag never matches this one's prefix.
assert!(!exit_tag("tsun01", &range()).starts_with(&exit_tag_prefix("tsun0")));
}
#[test]
fn an_empty_plan_is_nothing_to_hold() {
let mut plan = ExitHostPlan::default();
assert!(plan.is_empty());
plan.client = true;
assert!(!plan.is_empty());
}
#[tokio::test]
async fn the_mock_applies_reports_and_clears() {
let rules = MockExitRules::new();
let plan = ExitHostPlan {
interface: "tsun0".into(),
offer: vec![range()],
client: true,
};
let report = rules.apply(&plan).await;
assert_eq!(report.offer[0].1, RuleOutcome::Applied);
assert_eq!(report.forwarding, Some(true));
assert_eq!(report.client, Some(RuleOutcome::Applied));
assert_eq!(rules.installed(), Some(plan));
rules.set_forwarding(Some(false));
rules.fail_with(Some("denied".into()));
let report = rules
.apply(&ExitHostPlan {
interface: "tsun0".into(),
offer: vec![range()],
client: false,
})
.await;
assert_eq!(report.forwarding, Some(false));
assert!(matches!(report.offer[0].1, RuleOutcome::Failed(_)));
assert_eq!(report.client, None);
rules.clear("tsun0").await;
assert_eq!(rules.installed(), None);
assert_eq!(rules.calls(), ["apply:tsun0", "apply:tsun0", "clear:tsun0"]);
}
}
@@ -0,0 +1,570 @@
//! Broadcast host rules on Linux: a netlink route and an `iptables` rule.
//!
//! The route is added by the netlink thread the interface provisioner already
//! owns (see [`super::super::provision`]), because that thread is the only one
//! that ever holds `CAP_NET_ADMIN`. It belongs to the interface and goes away
//! with it.
//!
//! The firewall allowance is a rule in the `INPUT` chain, inserted at the top
//! so a default-deny chain further down cannot drop discovery replies first:
//!
//! ```text
//! iptables -I INPUT 1 -i <if> -p udp -s <range> -m comment --comment tsunagi:<if> -j ACCEPT
//! ```
//!
//! `iptables` is a program, not a netlink call, so it is looked up in fixed
//! system directories rather than on `PATH`, and its arguments are built from
//! values this agent computed. It works with both the legacy backend and
//! `iptables-nft`. A host with only `nft` or `firewalld` has no `iptables`;
//! that is reported on the firewall half and the route is still installed.
//!
//! The rule carries a comment tag. That is how a rule left behind by a crashed
//! run is found and replaced, and why nothing else in the chain is ever
//! deleted.
use std::net::Ipv4Addr;
use crate::BoxFuture;
use crate::overlay::provision::RouteHandle;
use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome, tag_for};
const CHAIN: &str = "INPUT";
/// Fixed places `iptables` is installed. Never `PATH`: the agent is
/// privileged and a directory a user controls must not decide what it runs.
const IPTABLES_DIRECTORIES: [&str; 5] = [
"/usr/sbin/iptables",
"/sbin/iptables",
"/usr/bin/iptables",
"/bin/iptables",
"/usr/local/sbin/iptables",
];
/// The first candidate that exists.
fn pick_iptables(exists: impl Fn(&str) -> bool) -> Option<&'static str> {
IPTABLES_DIRECTORIES
.into_iter()
.find(|candidate| exists(candidate))
}
/// What running `iptables` produced.
#[derive(Debug, Clone, PartialEq, Eq)]
pub(super) enum Ran {
/// There is no `iptables` to run.
Missing,
/// It ran, and exited like this.
Exited {
success: bool,
stdout: String,
stderr: String,
},
}
/// Runs `iptables` with these arguments. A function value so the flow below
/// can be tested without touching the host.
pub(super) type Run<'a> = &'a dyn Fn(&[String]) -> Ran;
/// The match and target of the rule, without the chain or position.
fn rule_spec(plan: &BroadcastRulesPlan) -> Vec<String> {
[
"-i",
plan.interface.as_str(),
"-p",
"udp",
"-s",
&plan.range.to_string(),
"-m",
"comment",
"--comment",
&plan.tag(),
"-j",
"ACCEPT",
]
.into_iter()
.map(str::to_string)
.collect()
}
/// Waits briefly for the xtables lock instead of failing when another tool
/// holds it.
pub(super) fn locked(args: impl IntoIterator<Item = String>) -> Vec<String> {
["-w", "5"]
.into_iter()
.map(str::to_string)
.chain(args)
.collect()
}
fn check_args(plan: &BroadcastRulesPlan) -> Vec<String> {
locked(
["-C".to_string(), CHAIN.to_string()]
.into_iter()
.chain(rule_spec(plan)),
)
}
fn insert_args(plan: &BroadcastRulesPlan) -> Vec<String> {
locked(
["-I".to_string(), CHAIN.to_string(), "1".to_string()]
.into_iter()
.chain(rule_spec(plan)),
)
}
fn list_args() -> Vec<String> {
locked(["-S".to_string(), CHAIN.to_string()])
}
/// Splits one `iptables -S` line into words, honouring double quotes, which
/// `iptables` puts around comments that need them.
pub(super) fn split_rule_line(line: &str) -> Vec<String> {
let mut words = Vec::new();
let mut current = String::new();
let mut quoted = false;
let mut started = false;
for c in line.chars() {
match c {
'"' => {
quoted = !quoted;
started = true;
}
c if c.is_whitespace() && !quoted => {
if started {
words.push(std::mem::take(&mut current));
started = false;
}
}
c => {
current.push(c);
started = true;
}
}
}
if started {
words.push(current);
}
words
}
/// The delete commands for every rule in an `iptables -S INPUT` listing that
/// carries exactly this tag. Anything else in the chain is left alone.
fn tagged_deletions(listing: &str, tag: &str) -> Vec<Vec<String>> {
listing
.lines()
.filter_map(|line| {
let words = split_rule_line(line);
let [append, chain, rest @ ..] = words.as_slice() else {
return None;
};
if append != "-A" || chain != CHAIN {
return None;
}
let tagged = rest
.windows(2)
.any(|pair| pair[0] == "--comment" && pair[1] == tag);
tagged.then(|| {
locked(
["-D".to_string(), CHAIN.to_string()]
.into_iter()
.chain(rest.iter().cloned()),
)
})
})
.collect()
}
/// A failure message with the likely fix for the likely cause.
pub(super) fn failure(stderr: &str) -> String {
let text = stderr.trim();
let lower = text.to_ascii_lowercase();
if lower.contains("permission denied") || lower.contains("must be root") {
format!("iptables needs root: {text}")
} else if text.is_empty() {
"iptables failed without saying why".to_string()
} else {
format!("iptables failed: {text}")
}
}
fn missing(plan: &BroadcastRulesPlan) -> String {
format!(
"iptables not found; install it (the iptables-nft package is enough) or allow inbound \
UDP from {} on `{}` in your firewall",
plan.range, plan.interface
)
}
/// Makes the firewall hold exactly one rule for this interface, the planned one.
fn apply_firewall(run: Run<'_>, plan: &BroadcastRulesPlan) -> RuleOutcome {
let present = match run(&check_args(plan)) {
Ran::Missing => return RuleOutcome::Failed(missing(plan)),
Ran::Exited { success, .. } => success,
};
let listing = match run(&list_args()) {
Ran::Missing => return RuleOutcome::Failed(missing(plan)),
Ran::Exited {
success: true,
stdout,
..
} => stdout,
Ran::Exited { stderr, .. } => return RuleOutcome::Failed(failure(&stderr)),
};
let stale = tagged_deletions(&listing, &plan.tag());
// `-C` matched an identical rule and nothing else carries our tag, so
// there is nothing to do. Otherwise start from a clean slate for this
// interface: a rule from an earlier run may name another range.
if present && stale.len() == 1 {
return RuleOutcome::Applied;
}
for delete in &stale {
if let Ran::Exited {
success: false,
stderr,
..
} = run(delete)
{
return RuleOutcome::Failed(failure(&stderr));
}
}
match run(&insert_args(plan)) {
Ran::Missing => RuleOutcome::Failed(missing(plan)),
Ran::Exited { success: true, .. } => RuleOutcome::Applied,
Ran::Exited { stderr, .. } => RuleOutcome::Failed(failure(&stderr)),
}
}
/// Removes every rule tagged for this interface. Best effort.
fn clear_firewall(run: Run<'_>, interface: &str) {
let Ran::Exited {
success: true,
stdout,
..
} = run(&list_args())
else {
return;
};
for delete in tagged_deletions(&stdout, &tag_for(interface)) {
if let Ran::Exited {
success: false,
stderr,
..
} = run(&delete)
{
tracing::warn!(
interface,
"cannot remove the broadcast firewall rule: {}",
failure(&stderr)
);
}
}
}
/// Runs the real `iptables`.
pub(super) fn run_iptables(args: &[String]) -> Ran {
let Some(program) = pick_iptables(|path| std::path::Path::new(path).is_file()) else {
return Ran::Missing;
};
match std::process::Command::new(program).args(args).output() {
Ok(output) => Ran::Exited {
success: output.status.success(),
stdout: String::from_utf8_lossy(&output.stdout).into_owned(),
stderr: String::from_utf8_lossy(&output.stderr).into_owned(),
},
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ran::Missing,
Err(err) => Ran::Exited {
success: false,
stdout: String::new(),
stderr: err.to_string(),
},
}
}
/// The Linux implementation: netlink for the route, `iptables` for the rule.
#[derive(Debug)]
pub struct LinuxHostRules {
pub(super) route: RouteHandle,
}
impl LinuxHostRules {
pub(crate) fn new(route: RouteHandle) -> Self {
Self { route }
}
async fn set_route(&self, interface: String, source: Ipv4Addr) -> RuleOutcome {
let route = self.route.clone();
match tokio::task::spawn_blocking(move || route.set(&interface, source)).await {
Ok(Ok(())) => RuleOutcome::Applied,
Ok(Err(err)) => RuleOutcome::Failed(err.to_string()),
Err(err) => RuleOutcome::Failed(format!("the route task failed: {err}")),
}
}
}
impl BroadcastHostRules for LinuxHostRules {
fn name(&self) -> &str {
"netlink+iptables"
}
fn exit_rules(&self) -> Option<&dyn super::ExitHostRules> {
Some(self)
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
let route = self.set_route(plan.interface.clone(), plan.source).await;
let owned = plan.clone();
let firewall =
tokio::task::spawn_blocking(move || apply_firewall(&run_iptables, &owned))
.await
.unwrap_or_else(|err| {
RuleOutcome::Failed(format!("the firewall task failed: {err}"))
});
BroadcastRulesReport {
plan: plan.clone(),
route,
firewall,
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
let route = self.route.clone();
let name = interface.to_string();
let route_name = name.clone();
if let Ok(Err(err)) =
tokio::task::spawn_blocking(move || route.clear(&route_name)).await
{
tracing::warn!(interface = %name, %err, "cannot remove the broadcast route");
}
let name = interface.to_string();
let _ = tokio::task::spawn_blocking(move || clear_firewall(&run_iptables, &name)).await;
})
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use std::cell::RefCell;
fn plan() -> BroadcastRulesPlan {
BroadcastRulesPlan {
interface: "tsun0".into(),
source: Ipv4Addr::new(10, 13, 37, 142),
range: "10.13.37.0/24".parse().unwrap(),
}
}
fn words(args: &[String]) -> String {
args.join(" ")
}
#[test]
fn the_rule_is_udp_only_from_the_overlay_range_on_this_interface() {
let spec = words(&rule_spec(&plan()));
assert_eq!(
spec,
"-i tsun0 -p udp -s 10.13.37.0/24 -m comment --comment tsunagi:tsun0 -j ACCEPT"
);
// Nothing here may widen to other protocols or every source.
assert!(spec.contains("-p udp"));
assert!(!spec.contains("0.0.0.0/0"));
assert!(words(&insert_args(&plan())).starts_with("-w 5 -I INPUT 1 -i tsun0"));
assert!(words(&check_args(&plan())).starts_with("-w 5 -C INPUT -i tsun0"));
}
#[test]
fn iptables_is_found_in_fixed_directories_never_on_path() {
assert_eq!(pick_iptables(|_| false), None);
assert_eq!(
pick_iptables(|p| p == "/sbin/iptables"),
Some("/sbin/iptables")
);
assert_eq!(pick_iptables(|_| true), Some("/usr/sbin/iptables"));
}
#[test]
fn only_rules_with_our_exact_tag_are_selected_for_deletion() {
let listing = "\
-P INPUT DROP
-A INPUT -s 10.13.37.0/24 -i tsun0 -p udp -m comment --comment \"tsunagi:tsun0\" -j ACCEPT
-A INPUT -i tsun0 -m comment --comment tsunagi:tsun0 -j ACCEPT
-A INPUT -i tsun1 -p udp -m comment --comment tsunagi:tsun1 -j ACCEPT
-A INPUT -p tcp --dport 22 -m comment --comment \"allow ssh tsunagi:tsun0\" -j ACCEPT
-A FORWARD -i tsun0 -m comment --comment tsunagi:tsun0 -j ACCEPT
-A INPUT -p tcp --dport 22 -j ACCEPT
";
let deletions = tagged_deletions(listing, "tsunagi:tsun0");
assert_eq!(deletions.len(), 2, "{deletions:?}");
assert_eq!(
words(&deletions[0]),
"-w 5 -D INPUT -s 10.13.37.0/24 -i tsun0 -p udp -m comment --comment tsunagi:tsun0 -j ACCEPT"
);
assert!(words(&deletions[1]).contains("-D INPUT -i tsun0 -m comment"));
assert!(tagged_deletions("", "tsunagi:tsun0").is_empty());
}
/// A scripted `iptables`: answers by the leading verb, records every call.
struct Fake {
calls: RefCell<Vec<String>>,
check_ok: bool,
listing: String,
insert_stderr: Option<&'static str>,
missing: bool,
}
impl Fake {
fn new(check_ok: bool, listing: &str) -> Self {
Self {
calls: RefCell::new(Vec::new()),
check_ok,
listing: listing.to_string(),
insert_stderr: None,
missing: false,
}
}
fn run(&self, args: &[String]) -> Ran {
self.calls.borrow_mut().push(words(args));
if self.missing {
return Ran::Missing;
}
let ok = |stdout: &str| Ran::Exited {
success: true,
stdout: stdout.to_string(),
stderr: String::new(),
};
match args.get(2).map(String::as_str) {
Some("-C") if self.check_ok => ok(""),
Some("-C") => Ran::Exited {
success: false,
stdout: String::new(),
stderr: "Bad rule".into(),
},
Some("-S") => ok(&self.listing),
Some("-I") => match self.insert_stderr {
Some(stderr) => Ran::Exited {
success: false,
stdout: String::new(),
stderr: stderr.into(),
},
None => ok(""),
},
_ => ok(""),
}
}
fn verbs(&self) -> Vec<String> {
self.calls
.borrow()
.iter()
.map(|call| call.split(' ').nth(2).unwrap().to_string())
.collect()
}
}
const ONE_OURS: &str =
"-A INPUT -s 10.13.37.0/24 -i tsun0 -p udp -m comment --comment tsunagi:tsun0 -j ACCEPT\n";
#[test]
fn an_identical_rule_already_in_place_is_left_alone() {
let fake = Fake::new(true, ONE_OURS);
assert_eq!(
apply_firewall(&|a| fake.run(a), &plan()),
RuleOutcome::Applied
);
assert_eq!(fake.verbs(), ["-C", "-S"], "no insert, no delete");
}
#[test]
fn a_missing_rule_is_inserted_once() {
let fake = Fake::new(false, "-P INPUT ACCEPT\n");
assert_eq!(
apply_firewall(&|a| fake.run(a), &plan()),
RuleOutcome::Applied
);
assert_eq!(fake.verbs(), ["-C", "-S", "-I"]);
}
#[test]
fn a_stale_rule_for_another_range_is_replaced_not_stacked() {
// A crashed run left a rule for an old range under our tag.
let stale = "-A INPUT -s 10.99.0.0/16 -i tsun0 -p udp -m comment --comment tsunagi:tsun0 -j ACCEPT\n";
let fake = Fake::new(false, stale);
assert_eq!(
apply_firewall(&|a| fake.run(a), &plan()),
RuleOutcome::Applied
);
assert_eq!(fake.verbs(), ["-C", "-S", "-D", "-I"]);
}
#[test]
fn duplicates_of_the_rule_collapse_to_one() {
let doubled = format!("{ONE_OURS}{ONE_OURS}");
let fake = Fake::new(true, &doubled);
assert_eq!(
apply_firewall(&|a| fake.run(a), &plan()),
RuleOutcome::Applied
);
assert_eq!(fake.verbs(), ["-C", "-S", "-D", "-D", "-I"]);
}
#[test]
fn a_missing_iptables_says_what_to_install_and_what_to_allow() {
let mut fake = Fake::new(false, "");
fake.missing = true;
let RuleOutcome::Failed(reason) = apply_firewall(&|a| fake.run(a), &plan()) else {
panic!("expected a failure");
};
assert!(reason.contains("iptables not found"), "{reason}");
assert!(reason.contains("10.13.37.0/24"), "{reason}");
assert!(reason.contains("tsun0"), "{reason}");
}
#[test]
fn a_refused_insert_is_reported_with_the_cause() {
let mut fake = Fake::new(false, "");
fake.insert_stderr = Some(
"iptables v1.8.10: can't initialize iptables table `filter': Permission denied (you must be root)",
);
let RuleOutcome::Failed(reason) = apply_firewall(&|a| fake.run(a), &plan()) else {
panic!("expected a failure");
};
assert!(reason.starts_with("iptables needs root"), "{reason}");
}
#[test]
fn clearing_removes_only_tagged_rules_and_ignores_a_missing_iptables() {
let listing = format!(
"-A INPUT -p tcp --dport 22 -j ACCEPT\n{ONE_OURS}-A INPUT -i tsun1 -m comment --comment tsunagi:tsun1 -j ACCEPT\n"
);
let fake = Fake::new(true, &listing);
clear_firewall(&|a| fake.run(a), "tsun0");
assert_eq!(fake.verbs(), ["-S", "-D"]);
let mut gone = Fake::new(true, "");
gone.missing = true;
clear_firewall(&|a| gone.run(a), "tsun0");
assert_eq!(gone.verbs(), ["-S"]);
}
#[test]
fn rule_listing_words_honour_quotes() {
assert_eq!(
split_rule_line("-A INPUT -m comment --comment \"a b\" -j ACCEPT"),
[
"-A",
"INPUT",
"-m",
"comment",
"--comment",
"a b",
"-j",
"ACCEPT"
]
);
assert!(split_rule_line(" ").is_empty());
}
}
@@ -0,0 +1,728 @@
//! Exit-node host rules on Linux: `iptables` for the offering side, policy
//! routing over netlink for the using side.
//!
//! # Offering
//!
//! For every network that offers this agent as an exit node, three rules,
//! selected by that network's range and carrying the tag
//! `tsunagi-exit:<if>:<range>`:
//!
//! ```text
//! -t nat POSTROUTING -s <range> ! -o <if> -j MASQUERADE
//! -t filter FORWARD -i <if> -s <range> ! -o <if> -j ACCEPT
//! -t filter FORWARD -o <if> -d <range> -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
//! ```
//!
//! `! -o <if>` is the egress without naming it: whatever interface the
//! default route uses, now or after it changes. Kernel forwarding is only
//! *read*. Turning it on changes how the whole host behaves, which is the
//! owner's decision, so when it is off the rules are installed anyway and the
//! report says so.
//!
//! # Using
//!
//! Policy routing, with no `/1` routes and nothing in the main table:
//!
//! ```text
//! table 0x7473: default dev <if>
//! 5280: from all uidrange <euid>-<euid> lookup main
//! 5290: from all lookup main suppress_prefixlength 0
//! 5300: from all lookup 0x7473
//! ```
//!
//! The first keeps the agent's *own* sockets off the tunnel: its QUIC, relay,
//! DHT and DNS traffic would otherwise be routed into the interface it is
//! carrying, a loop. The second lets every specific route — the LAN, another
//! VPN, a tunnel to somewhere else — keep working, because only the default
//! route is suppressed. The priorities sit after Tailscale's 5270.
//!
//! The first rule exempts a *user id*, so it is only right when the agent has
//! one of its own, as the packaged service does. An agent started from a login
//! session would take that user's own applications out of the tunnel with it,
//! and the exit node would silently do nothing for them. That is refused, not
//! guessed at: see [`client_refusal`].
use crate::BoxFuture;
use crate::state::Ipv4Range;
use super::RuleOutcome;
use super::exit::{ExitHostPlan, ExitHostReport, ExitHostRules, exit_tag, exit_tag_prefix};
use super::linux::{LinuxHostRules, Ran, Run, failure, locked, run_iptables, split_rule_line};
/// The chains the offering rules live in: `(table, chain)`.
const CHAINS: [(&str, &str); 2] = [("nat", "POSTROUTING"), ("filter", "FORWARD")];
/// One firewall rule: where it goes, and its match and target.
#[derive(Debug, Clone, PartialEq, Eq)]
struct Rule {
table: &'static str,
chain: &'static str,
spec: Vec<String>,
}
fn owned(words: &[&str]) -> Vec<String> {
words.iter().map(|word| word.to_string()).collect()
}
/// The three rules of one offered range.
fn offer_rules(interface: &str, range: &Ipv4Range) -> Vec<Rule> {
let range_text = range.to_string();
let tag = exit_tag(interface, range);
let comment = ["-m", "comment", "--comment", tag.as_str()];
let spec = |head: &[&str], tail: &[&str]| -> Vec<String> {
owned(head)
.into_iter()
.chain(owned(&comment))
.chain(owned(tail))
.collect()
};
vec![
Rule {
table: "nat",
chain: "POSTROUTING",
spec: spec(
&["-s", &range_text, "!", "-o", interface],
&["-j", "MASQUERADE"],
),
},
Rule {
table: "filter",
chain: "FORWARD",
spec: spec(
&["-i", interface, "-s", &range_text, "!", "-o", interface],
&["-j", "ACCEPT"],
),
},
Rule {
table: "filter",
chain: "FORWARD",
spec: spec(
&[
"-o",
interface,
"-d",
&range_text,
"-m",
"conntrack",
"--ctstate",
"RELATED,ESTABLISHED",
],
&["-j", "ACCEPT"],
),
},
]
}
fn check_args(rule: &Rule) -> Vec<String> {
locked(
owned(&["-t", rule.table, "-C", rule.chain])
.into_iter()
.chain(rule.spec.iter().cloned()),
)
}
fn insert_args(rule: &Rule) -> Vec<String> {
locked(
owned(&["-t", rule.table, "-I", rule.chain, "1"])
.into_iter()
.chain(rule.spec.iter().cloned()),
)
}
fn list_args(table: &str, chain: &str) -> Vec<String> {
locked(owned(&["-t", table, "-S", chain]))
}
/// A rule found in a listing whose comment starts with our prefix.
#[derive(Debug, Clone, PartialEq, Eq)]
struct Found {
/// The whole tag.
tag: String,
/// The words after `-A <chain>`, which is what `-D` needs.
rest: Vec<String>,
}
/// Every rule in an `iptables -S <chain>` listing carrying a tag that starts
/// with `prefix`. Anything else in the chain is left alone.
fn tagged_rules(listing: &str, chain: &str, prefix: &str) -> Vec<Found> {
listing
.lines()
.filter_map(|line| {
let words = split_rule_line(line);
let [append, found_chain, rest @ ..] = words.as_slice() else {
return None;
};
if append != "-A" || found_chain != chain {
return None;
}
let tag = rest
.windows(2)
.find(|pair| pair[0] == "--comment" && pair[1].starts_with(prefix))
.map(|pair| pair[1].clone())?;
Some(Found {
tag,
rest: rest.to_vec(),
})
})
.collect()
}
fn delete_args(table: &str, chain: &str, found: &Found) -> Vec<String> {
locked(
owned(&["-t", table, "-D", chain])
.into_iter()
.chain(found.rest.iter().cloned()),
)
}
fn missing_iptables(range: &Ipv4Range, interface: &str) -> String {
format!(
"iptables not found; install it (the iptables-nft package is enough), or masquerade \
{range} out of every interface but `{interface}` and forward it yourself"
)
}
/// Reads one chain, or says why not.
fn read_chain(run: Run<'_>, table: &str, chain: &str) -> Result<String, Failure> {
match run(&list_args(table, chain)) {
Ran::Missing => Err(Failure::Missing),
Ran::Exited {
success: true,
stdout,
..
} => Ok(stdout),
Ran::Exited { stderr, .. } => Err(Failure::Failed(failure(&stderr))),
}
}
enum Failure {
Missing,
Failed(String),
}
/// Deletes found rules; the first failure ends it.
fn delete_all(
run: Run<'_>,
table: &str,
chain: &str,
found: &[Found],
keep: impl Fn(&Found) -> bool,
) -> Result<(), String> {
for rule in found.iter().filter(|rule| !keep(rule)) {
if let Ran::Exited {
success: false,
stderr,
..
} = run(&delete_args(table, chain, rule))
{
return Err(failure(&stderr));
}
}
Ok(())
}
/// Makes the firewall hold exactly the offering rules of these ranges for the
/// interface, and no others of ours.
fn apply_offer(
run: Run<'_>,
interface: &str,
ranges: &[Ipv4Range],
) -> Vec<(Ipv4Range, RuleOutcome)> {
let fail_all = |reason: String| -> Vec<(Ipv4Range, RuleOutcome)> {
ranges
.iter()
.map(|range| (*range, RuleOutcome::Failed(reason.clone())))
.collect()
};
let prefix = exit_tag_prefix(interface);
let wanted: Vec<String> = ranges.iter().map(|r| exit_tag(interface, r)).collect();
// What is there now, per chain.
let mut present: Vec<(&str, &str, Vec<Found>)> = Vec::new();
for (table, chain) in CHAINS {
match read_chain(run, table, chain) {
Ok(listing) => present.push((table, chain, tagged_rules(&listing, chain, &prefix))),
Err(Failure::Missing) => {
return match ranges.first() {
Some(range) => fail_all(missing_iptables(range, interface)),
None => Vec::new(),
};
}
Err(Failure::Failed(reason)) => return fail_all(reason),
}
}
// Rules for a range no longer offered, or left by an earlier run.
for (table, chain, found) in &present {
if let Err(reason) = delete_all(run, table, chain, found, |rule| wanted.contains(&rule.tag))
{
return fail_all(reason);
}
}
ranges
.iter()
.map(|range| {
let rules = offer_rules(interface, range);
let tag = exit_tag(interface, range);
let in_place = rules
.iter()
.all(|rule| matches!(run(&check_args(rule)), Ran::Exited { success: true, .. }))
&& present.iter().all(|(_, chain, found)| {
let have = found.iter().filter(|rule| rule.tag == tag).count();
have == rules.iter().filter(|rule| rule.chain == *chain).count()
});
if in_place {
return (*range, RuleOutcome::Applied);
}
// Start this range from a clean slate: a half-installed set, or
// one that differs from the plan, is replaced as a whole.
for (table, chain, found) in &present {
if let Err(reason) = delete_all(run, table, chain, found, |rule| rule.tag != tag) {
return (*range, RuleOutcome::Failed(reason));
}
}
for rule in &rules {
match run(&insert_args(rule)) {
Ran::Missing => {
return (
*range,
RuleOutcome::Failed(missing_iptables(range, interface)),
);
}
Ran::Exited { success: true, .. } => {}
Ran::Exited { stderr, .. } => {
return (*range, RuleOutcome::Failed(failure(&stderr)));
}
}
}
(*range, RuleOutcome::Applied)
})
.collect()
}
/// Removes every offering rule of ours for the interface. Best effort.
fn clear_offer(run: Run<'_>, interface: &str) {
let prefix = exit_tag_prefix(interface);
for (table, chain) in CHAINS {
let Ok(listing) = read_chain(run, table, chain) else {
continue;
};
let found = tagged_rules(&listing, chain, &prefix);
if let Err(reason) = delete_all(run, table, chain, &found, |_| false) {
tracing::warn!(
interface,
"cannot remove an exit node firewall rule: {reason}"
);
}
}
}
/// The kernel's forwarding switch for packets arriving on the interface,
/// read from the text of the sysctl files. Never written.
fn forwarding_state(read: impl Fn(&str) -> Option<String>, interface: &str) -> Option<bool> {
let flag = |text: String| match text.trim() {
"1" => Some(true),
"0" => Some(false),
_ => None,
};
// The name is derived here, but it ends up in a path.
if !interface.is_empty() && !interface.contains(['/', '\0']) && interface != ".." {
let per_interface = format!("/proc/sys/net/ipv4/conf/{interface}/forwarding");
if let Some(state) = read(&per_interface).and_then(flag) {
return Some(state);
}
}
read("/proc/sys/net/ipv4/ip_forward").and_then(flag)
}
fn read_sysctl(path: &str) -> Option<String> {
std::fs::read_to_string(path).ok()
}
/// The effective user id from the text of `/proc/self/status`.
fn parse_euid(status: &str) -> Option<u32> {
status
.lines()
.find_map(|line| line.strip_prefix("Uid:"))
.and_then(|rest| rest.split_whitespace().nth(1))
.and_then(|effective| effective.parse().ok())
}
/// Why the using side must not be installed for this process, if it must not.
///
/// The bypass rule exempts this process's *user id* from the tunnel. That is
/// right for a service with an id of its own and wrong for a process started
/// from somebody's login session, which shares its id with every application
/// that person runs. A login session is recognised by `XDG_RUNTIME_DIR`,
/// which `systemd --user` and the login managers set and a system service
/// does not. Crude on purpose: refusing a service that was started oddly is
/// a message, and exempting a user's applications is a silent failure.
fn client_refusal(xdg_runtime_dir: Option<&str>) -> Option<String> {
xdg_runtime_dir.filter(|dir| !dir.is_empty()).map(|_| {
"the exit node needs the agent to run as a system service with a user of its own: \
this agent was started from a login session, so routing everything through the \
overlay would take that user's own applications out of the tunnel with it"
.to_string()
})
}
impl LinuxHostRules {
async fn blocking<T: Send + 'static>(
work: impl FnOnce() -> T + Send + 'static,
) -> Result<T, String> {
tokio::task::spawn_blocking(work)
.await
.map_err(|err| format!("the task failed: {err}"))
}
async fn apply_client(&self, interface: &str) -> RuleOutcome {
let xdg = std::env::var("XDG_RUNTIME_DIR").ok();
if let Some(reason) = client_refusal(xdg.as_deref()) {
// Whatever an earlier run left must not stay in force.
self.clear_client().await;
return RuleOutcome::Failed(reason);
}
let Some(euid) = std::fs::read_to_string("/proc/self/status")
.ok()
.and_then(|status| parse_euid(&status))
else {
return RuleOutcome::Failed("cannot tell which user this agent runs as".to_string());
};
let route = self.route.clone();
let interface = interface.to_string();
match Self::blocking(move || route.set_exit_client(&interface, euid)).await {
Ok(Ok(())) => RuleOutcome::Applied,
Ok(Err(err)) => RuleOutcome::Failed(err.to_string()),
Err(reason) => RuleOutcome::Failed(reason),
}
}
async fn clear_client(&self) {
let route = self.route.clone();
if let Ok(Err(err)) = Self::blocking(move || route.clear_exit_client()).await {
tracing::warn!(%err, "cannot remove the exit node routing rules");
}
}
}
impl ExitHostRules for LinuxHostRules {
fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport> {
Box::pin(async move {
let interface = plan.interface.clone();
let ranges = plan.offer.clone();
let offer = {
let (job_interface, job_ranges) = (interface.clone(), ranges.clone());
Self::blocking(move || apply_offer(&run_iptables, &job_interface, &job_ranges))
.await
.unwrap_or_else(|reason| {
ranges
.iter()
.map(|range| (*range, RuleOutcome::Failed(reason.clone())))
.collect()
})
};
let forwarding = if ranges.is_empty() {
None
} else {
forwarding_state(read_sysctl, &interface)
};
let client = if plan.client {
Some(self.apply_client(&interface).await)
} else {
self.clear_client().await;
None
};
ExitHostReport {
offer,
forwarding,
client,
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
let name = interface.to_string();
let _ = Self::blocking(move || clear_offer(&run_iptables, &name)).await;
self.clear_client().await;
})
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::cell::RefCell;
use super::*;
fn range() -> Ipv4Range {
"10.13.37.0/24".parse().unwrap()
}
fn words(args: &[String]) -> String {
args.join(" ")
}
#[test]
fn the_rules_masquerade_the_range_out_of_every_other_interface_and_forward_it() {
let rules = offer_rules("tsun0", &range());
assert_eq!(rules.len(), 3);
assert_eq!(
words(&rules[0].spec),
"-s 10.13.37.0/24 ! -o tsun0 -m comment --comment tsunagi-exit:tsun0:10.13.37.0/24 -j MASQUERADE"
);
assert_eq!((rules[0].table, rules[0].chain), ("nat", "POSTROUTING"));
assert_eq!(
words(&rules[1].spec),
"-i tsun0 -s 10.13.37.0/24 ! -o tsun0 -m comment --comment tsunagi-exit:tsun0:10.13.37.0/24 -j ACCEPT"
);
assert_eq!((rules[1].table, rules[1].chain), ("filter", "FORWARD"));
assert_eq!(
words(&rules[2].spec),
"-o tsun0 -d 10.13.37.0/24 -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment tsunagi-exit:tsun0:10.13.37.0/24 -j ACCEPT"
);
// Nothing may widen to every source or to the other direction.
for rule in &rules {
assert!(!words(&rule.spec).contains("0.0.0.0/0"));
}
assert!(words(&insert_args(&rules[1])).starts_with("-w 5 -t filter -I FORWARD 1 -i tsun0"));
assert!(words(&check_args(&rules[0])).starts_with("-w 5 -t nat -C POSTROUTING -s"));
}
#[test]
fn only_rules_with_our_tag_for_this_interface_are_selected() {
let listing = "\
-P FORWARD DROP
-A FORWARD -i tsun0 -s 10.13.37.0/24 ! -o tsun0 -m comment --comment tsunagi-exit:tsun0:10.13.37.0/24 -j ACCEPT
-A FORWARD -i tsun1 -m comment --comment tsunagi-exit:tsun1:10.99.0.0/24 -j ACCEPT
-A FORWARD -i tsun01 -m comment --comment tsunagi-exit:tsun01:10.5.0.0/24 -j ACCEPT
-A FORWARD -m comment --comment \"allow tsunagi-exit:tsun0:x\" -j ACCEPT
-A FORWARD -m comment --comment tsunagi:tsun0 -j ACCEPT
-A INPUT -m comment --comment tsunagi-exit:tsun0:10.13.37.0/24 -j ACCEPT
-A FORWARD -j ACCEPT
";
let found = tagged_rules(listing, "FORWARD", &exit_tag_prefix("tsun0"));
assert_eq!(found.len(), 1, "{found:?}");
assert_eq!(found[0].tag, "tsunagi-exit:tsun0:10.13.37.0/24");
assert_eq!(
words(&delete_args("filter", "FORWARD", &found[0])),
"-w 5 -t filter -D FORWARD -i tsun0 -s 10.13.37.0/24 ! -o tsun0 -m comment --comment tsunagi-exit:tsun0:10.13.37.0/24 -j ACCEPT"
);
assert!(tagged_rules("", "FORWARD", "tsunagi-exit:tsun0:").is_empty());
}
/// A scripted `iptables` that remembers which rules it was given.
struct Fake {
calls: RefCell<Vec<String>>,
/// Rules present, as `-S` lines per `(table, chain)`.
listing: RefCell<Vec<(String, String)>>,
missing: bool,
check_ok: bool,
}
impl Fake {
fn new(check_ok: bool) -> Self {
Self {
calls: RefCell::new(Vec::new()),
listing: RefCell::new(Vec::new()),
missing: false,
check_ok,
}
}
fn with_rule(self, table: &str, line: &str) -> Self {
self.listing
.borrow_mut()
.push((table.to_string(), line.to_string()));
self
}
fn run(&self, args: &[String]) -> Ran {
if self.missing {
return Ran::Missing;
}
let line = args.join(" ");
self.calls.borrow_mut().push(line.clone());
let table = args
.iter()
.position(|word| word == "-t")
.map(|at| args[at + 1].clone())
.unwrap_or_default();
let ok = |stdout: String| Ran::Exited {
success: true,
stdout,
stderr: String::new(),
};
if line.contains(" -S ") {
let listing: Vec<String> = self
.listing
.borrow()
.iter()
.filter(|(t, _)| *t == table)
.map(|(_, l)| l.clone())
.collect();
return ok(listing.join("\n"));
}
if line.contains(" -C ") {
return Ran::Exited {
success: self.check_ok,
stdout: String::new(),
stderr: String::new(),
};
}
ok(String::new())
}
fn verbs(&self, verb: &str) -> Vec<String> {
self.calls
.borrow()
.iter()
.filter(|call| call.contains(&format!(" {verb} ")))
.cloned()
.collect()
}
}
#[test]
fn a_fresh_host_gets_the_three_rules() {
let fake = Fake::new(false);
let outcome = apply_offer(&|args| fake.run(args), "tsun0", &[range()]);
assert_eq!(outcome, vec![(range(), RuleOutcome::Applied)]);
let inserted = fake.verbs("-I");
assert_eq!(inserted.len(), 3, "{inserted:?}");
assert!(inserted[0].contains("-t nat -I POSTROUTING 1"));
assert!(fake.verbs("-D").is_empty());
}
#[test]
fn rules_already_in_place_are_left_alone() {
let tag = exit_tag("tsun0", &range());
let fake = Fake::new(true)
.with_rule(
"nat",
&format!("-A POSTROUTING -s 10.13.37.0/24 ! -o tsun0 -m comment --comment {tag} -j MASQUERADE"),
)
.with_rule("filter", &format!("-A FORWARD -i tsun0 -m comment --comment {tag} -j ACCEPT"))
.with_rule("filter", &format!("-A FORWARD -o tsun0 -m comment --comment {tag} -j ACCEPT"));
let outcome = apply_offer(&|args| fake.run(args), "tsun0", &[range()]);
assert_eq!(outcome, vec![(range(), RuleOutcome::Applied)]);
assert!(fake.verbs("-I").is_empty());
assert!(fake.verbs("-D").is_empty());
}
#[test]
fn a_stale_range_is_removed_and_a_half_installed_set_is_replaced() {
let old: Ipv4Range = "10.99.0.0/24".parse().unwrap();
let old_tag = exit_tag("tsun0", &old);
let tag = exit_tag("tsun0", &range());
let fake = Fake::new(true)
.with_rule(
"nat",
&format!(
"-A POSTROUTING -s 10.99.0.0/24 -m comment --comment {old_tag} -j MASQUERADE"
),
)
// Only one of the three for the current range.
.with_rule(
"filter",
&format!("-A FORWARD -i tsun0 -m comment --comment {tag} -j ACCEPT"),
)
// Somebody else's rule, which must survive.
.with_rule("filter", "-A FORWARD -i eth0 -j ACCEPT");
let outcome = apply_offer(&|args| fake.run(args), "tsun0", &[range()]);
assert_eq!(outcome, vec![(range(), RuleOutcome::Applied)]);
let deleted = fake.verbs("-D");
assert!(
deleted.iter().any(|call| call.contains(&old_tag)),
"{deleted:?}"
);
assert!(
deleted.iter().any(|call| call.contains(&tag)),
"{deleted:?}"
);
assert!(
deleted.iter().all(|call| !call.contains("eth0")),
"{deleted:?}"
);
assert_eq!(fake.verbs("-I").len(), 3);
}
#[test]
fn clearing_removes_only_our_rules_for_the_interface() {
let tag = exit_tag("tsun0", &range());
let other = exit_tag("tsun1", &range());
let fake = Fake::new(true)
.with_rule(
"nat",
&format!("-A POSTROUTING -m comment --comment {tag} -j MASQUERADE"),
)
.with_rule(
"nat",
&format!("-A POSTROUTING -m comment --comment {other} -j MASQUERADE"),
)
.with_rule("filter", "-A FORWARD -j ACCEPT");
clear_offer(&|args| fake.run(args), "tsun0");
let deleted = fake.verbs("-D");
assert_eq!(deleted.len(), 1, "{deleted:?}");
assert!(deleted[0].contains(&tag));
}
#[test]
fn a_missing_iptables_is_a_failure_that_says_what_to_do() {
let mut fake = Fake::new(false);
fake.missing = true;
let outcome = apply_offer(&|args| fake.run(args), "tsun0", &[range()]);
let RuleOutcome::Failed(reason) = &outcome[0].1 else {
panic!("{outcome:?}");
};
assert!(reason.contains("iptables not found"), "{reason}");
assert!(reason.contains("10.13.37.0/24"), "{reason}");
// Nothing wanted and nothing to run is not a failure of anything.
assert!(apply_offer(&|args| fake.run(args), "tsun0", &[]).is_empty());
}
#[test]
fn forwarding_is_read_per_interface_then_globally_and_never_written() {
let files = |per: Option<&'static str>, global: Option<&'static str>| {
move |path: &str| match path {
"/proc/sys/net/ipv4/conf/tsun0/forwarding" => per.map(str::to_string),
"/proc/sys/net/ipv4/ip_forward" => global.map(str::to_string),
_ => None,
}
};
assert_eq!(
forwarding_state(files(Some("1\n"), Some("0\n")), "tsun0"),
Some(true)
);
assert_eq!(
forwarding_state(files(Some("0\n"), Some("1\n")), "tsun0"),
Some(false)
);
assert_eq!(
forwarding_state(files(None, Some("1\n")), "tsun0"),
Some(true)
);
assert_eq!(forwarding_state(files(None, None), "tsun0"), None);
assert_eq!(forwarding_state(files(Some("junk"), None), "tsun0"), None);
// A name that would leave the directory is not looked up.
assert_eq!(forwarding_state(files(Some("1"), None), "../../x"), None);
}
#[test]
fn the_effective_uid_is_the_second_field() {
let status = "Name:\ttsunagi\nUid:\t1001\t1002\t1001\t1001\nGid:\t5\t5\t5\t5\n";
assert_eq!(parse_euid(status), Some(1002));
assert_eq!(parse_euid("Name:\tx\n"), None);
}
#[test]
fn a_login_session_is_refused_and_a_service_is_not() {
let reason = client_refusal(Some("/run/user/1000")).unwrap();
assert!(reason.contains("system service"), "{reason}");
assert!(client_refusal(None).is_none());
assert!(client_refusal(Some("")).is_none());
}
}
@@ -0,0 +1,189 @@
//! Broadcast host rules on macOS: a `route` entry; the firewall is deferred.
//!
//! The limited-broadcast route is installed with `/sbin/route`, so a game's
//! LAN discovery packet leaves through the overlay interface. It belongs to
//! the interface and is removed when broadcast is turned off, the network
//! leaves, or the agent stops — and, because a utun is torn down with the
//! process, it also goes away on its own if the agent is killed.
//!
//! The inbound-UDP firewall allowance is **not** implemented here. On macOS
//! that means a `pf` anchor, which is a good deal more machinery; until it is
//! written the firewall half reports itself as missing with what to allow by
//! hand, and the route is still installed. Failure is contained: neither half
//! can stop the agent, and `status` shows exactly which is in place.
//!
//! `route` is a program, not a system call, so it is run by absolute path from
//! `/sbin` rather than off `PATH`, and its arguments are built from values
//! this agent computed — the interface name the kernel assigned, nothing a
//! peer said.
use crate::BoxFuture;
use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome};
/// The limited-broadcast destination routed through the overlay interface.
const LIMITED_BROADCAST: &str = "255.255.255.255";
/// The `route` arguments that direct the limited broadcast out an interface.
fn route_add_args(interface: &str) -> Vec<String> {
vec![
"-q".into(),
"-n".into(),
"add".into(),
"-host".into(),
LIMITED_BROADCAST.into(),
"-interface".into(),
interface.into(),
]
}
/// The `route` arguments that remove it again.
fn route_del_args(interface: &str) -> Vec<String> {
vec![
"-q".into(),
"-n".into(),
"delete".into(),
"-host".into(),
LIMITED_BROADCAST.into(),
"-interface".into(),
interface.into(),
]
}
/// What to do by hand while the `pf` allowance is unimplemented.
fn firewall_todo(plan: &BroadcastRulesPlan) -> String {
format!(
"the inbound-UDP firewall allowance is not implemented on macOS yet; if the host \
firewall drops discovery replies, allow inbound UDP from {} on `{}` with a pf rule \
yourself",
plan.range, plan.interface
)
}
/// The macOS implementation: `route` for the route, no firewall yet.
#[derive(Debug, Default)]
pub struct MacosHostRules;
impl MacosHostRules {
/// Creates the host rules.
pub fn new() -> Self {
Self
}
/// Installs the route, replacing one this agent added earlier so the call
/// can be repeated. The delete is best effort: a route that is not there
/// is the state the add then reaches anyway.
async fn set_route(&self, interface: String) -> RuleOutcome {
let _ = run(route_del_args(&interface)).await;
match run(route_add_args(&interface)).await {
Ok(()) => RuleOutcome::Applied,
Err(reason) => RuleOutcome::Failed(reason),
}
}
}
impl BroadcastHostRules for MacosHostRules {
fn name(&self) -> &str {
"route"
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
let route = self.set_route(plan.interface.clone()).await;
BroadcastRulesReport {
plan: plan.clone(),
route,
firewall: RuleOutcome::Failed(firewall_todo(plan)),
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
if let Err(reason) = run(route_del_args(interface)).await {
tracing::debug!(interface = %interface, %reason, "cannot remove the broadcast route");
}
})
}
}
/// Runs `/sbin/route`, turning a non-zero exit into the message it printed.
async fn run(args: Vec<String>) -> Result<(), String> {
let display = format!("route {}", args.join(" "));
let output = tokio::task::spawn_blocking(move || {
std::process::Command::new("/sbin/route")
.args(&args)
.output()
})
.await
.map_err(|err| format!("could not run {display}: {err}"))?
.map_err(|err| format!("could not run {display}: {err}"))?;
if output.status.success() {
return Ok(());
}
let stderr = String::from_utf8_lossy(&output.stderr);
let text = stderr.trim();
if text.is_empty() {
Err(format!("{display} failed with {}", output.status))
} else {
Err(format!("{display} failed: {text}"))
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::Ipv4Addr;
use super::*;
fn plan() -> BroadcastRulesPlan {
BroadcastRulesPlan {
interface: "utun5".into(),
source: Ipv4Addr::new(10, 13, 37, 142),
range: "10.13.37.0/24".parse().unwrap(),
}
}
#[test]
fn the_route_is_the_limited_broadcast_out_the_interface() {
assert_eq!(
route_add_args("utun5"),
vec![
"-q",
"-n",
"add",
"-host",
"255.255.255.255",
"-interface",
"utun5"
]
);
assert_eq!(
route_del_args("utun5"),
vec![
"-q",
"-n",
"delete",
"-host",
"255.255.255.255",
"-interface",
"utun5"
]
);
}
#[test]
fn the_firewall_todo_names_the_range_and_interface() {
let todo = firewall_todo(&plan());
assert!(todo.contains("10.13.37.0/24"), "{todo}");
assert!(todo.contains("utun5"), "{todo}");
assert!(todo.contains("pf"), "{todo}");
// Deferred, so the firewall half is always reported as not in place;
// the route runs for real and so is left to an on-host check. Running
// `route` here would need root and would change the host routing
// table, which the default test suite must never do.
}
}
@@ -0,0 +1,360 @@
//! Host rules that let a game's LAN discovery reach the overlay interface.
//!
//! Broadcast fanout happens at local TUN ingress (see [`super::broadcast`]),
//! so a limited broadcast only reaches Tsunagi if the operating system routes
//! it through the overlay interface, and a discovery *reply* only reaches the
//! game if the host firewall lets it in. Left to the user that is two manual
//! steps per host that nobody remembers, so when a network's broadcast is on
//! the agent installs exactly these two things for the interface it owns:
//!
//! * a `255.255.255.255/32` route through the overlay interface, with the
//! overlay address as its preferred source, so the game's discovery packet
//! leaves with a source address the router recognises;
//! * an inbound firewall allowance for **UDP only**, from the overlay range
//! only, on that interface only. Replies to a broadcast come from a
//! different address than the one the request was sent to, so a stateful
//! firewall never matches them to the request. Opening every protocol
//! would expose services such as SSH to every member of the network, which
//! is not what a game needs.
//!
//! # Why this is allowed here
//!
//! The architecture says plugins never touch routing or firewall settings,
//! and nothing a peer announces becomes an OS setting. This is neither: it is
//! the system level, which owns the interface, configuring that interface from
//! values it computed itself (the interface name is derived locally, the
//! address and range come from signed state it already verified). Every object
//! is tagged `tsunagi:<interface>`, reversible, and removed when broadcast is
//! turned off, the network leaves, or the agent stops. A crash can leave a
//! firewall rule behind; the next apply for the same interface removes it.
//!
//! # Failure is contained
//!
//! Neither step can stop the agent. Each reports its own outcome, so a missing
//! firewall tool still leaves the route in place and says what is missing.
//!
//! # Why a trait
//!
//! Linux, Windows and the tests do this very differently, while *what* to
//! install is the same everywhere and lives in [`BroadcastRulesPlan`] and
//! [`rules_summary`], tested on every platform.
use std::net::Ipv4Addr;
use std::sync::{Arc, Mutex};
use crate::BoxFuture;
use crate::state::Ipv4Range;
mod exit;
pub use exit::{
ExitHostPlan, ExitHostReport, ExitHostRules, MockExitRules, exit_tag, exit_tag_prefix,
offer_detail,
};
#[cfg(all(feature = "tun-device", target_os = "linux"))]
mod linux;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
mod linux_exit;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use linux::LinuxHostRules;
// Compiled everywhere so the scripts it builds are tested everywhere; only
// the type that runs them is exported on Windows.
mod windows;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use windows::WindowsHostRules;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
mod macos;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use macos::MacosHostRules;
/// What the host should be told for one interface.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BroadcastRulesPlan {
/// The overlay interface. Derived locally from the network id, never from
/// anything a peer said.
pub interface: String,
/// This agent's overlay address, the route's preferred source.
pub source: Ipv4Addr,
/// The overlay range inbound UDP is admitted from.
pub range: Ipv4Range,
}
impl BroadcastRulesPlan {
/// The tag every object carries, so it can be found again and nothing
/// else is ever touched.
pub fn tag(&self) -> String {
tag_for(&self.interface)
}
}
/// The tag for an interface's objects.
pub fn tag_for(interface: &str) -> String {
format!("tsunagi:{interface}")
}
/// What happened to one of the two host objects.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum RuleOutcome {
/// It is in place.
Applied,
/// It is not, with a reason a person can act on.
Failed(String),
}
impl RuleOutcome {
/// Whether the object is in place.
pub fn is_applied(&self) -> bool {
matches!(self, Self::Applied)
}
}
/// What applying a plan achieved.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BroadcastRulesReport {
/// What was asked for.
pub plan: BroadcastRulesPlan,
/// The `255.255.255.255` route.
pub route: RuleOutcome,
/// The inbound firewall allowance.
pub firewall: RuleOutcome,
}
impl BroadcastRulesReport {
/// Whether both objects are in place.
pub fn is_ok(&self) -> bool {
self.route.is_applied() && self.firewall.is_applied()
}
/// One line for `status`.
pub fn summary(&self) -> String {
rules_summary(&self.route, &self.firewall)
}
}
/// The status line for a pair of outcomes.
pub fn rules_summary(route: &RuleOutcome, firewall: &RuleOutcome) -> String {
match (route, firewall) {
(RuleOutcome::Applied, RuleOutcome::Applied) => "ok (route + firewall)".to_string(),
(route, firewall) => {
let part = |what: &str, outcome: &RuleOutcome| match outcome {
RuleOutcome::Applied => format!("{what} ok"),
RuleOutcome::Failed(reason) => format!("{what} failed: {reason}"),
};
format!(
"incomplete; {}; {}",
part("route", route),
part("firewall", firewall)
)
}
}
}
/// Installs and removes the host rules for an overlay interface.
///
/// Both calls are idempotent. Neither can fail as a whole: a step that cannot
/// be done is reported in the [`BroadcastRulesReport`], because the caller's
/// only useful response is to say so.
pub trait BroadcastHostRules: Send + Sync + std::fmt::Debug + 'static {
/// A short name used in diagnostics.
fn name(&self) -> &str;
/// Makes the host match the plan, replacing whatever this agent put there
/// for the same interface before.
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport>;
/// Removes what this agent put there for the interface. Succeeds when
/// nothing is there: it runs on the shutdown path.
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()>;
/// The exit-node rules that go with these, when the platform has them.
///
/// One object per host, because both halves need the same privileged
/// helpers; the default is that this platform has none, which is said
/// where it matters rather than pretended.
fn exit_rules(&self) -> Option<&dyn ExitHostRules> {
None
}
}
/// The platform has no implementation, which is said rather than pretended.
#[derive(Debug, Default)]
pub struct UnsupportedHostRules;
impl BroadcastHostRules for UnsupportedHostRules {
fn name(&self) -> &str {
"unsupported"
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
let reason = "host rules are not implemented on this platform; route \
255.255.255.255 through the overlay interface and allow inbound \
UDP from the overlay range yourself"
.to_string();
BroadcastRulesReport {
plan: plan.clone(),
route: RuleOutcome::Failed(reason.clone()),
firewall: RuleOutcome::Failed(reason),
}
})
}
fn clear<'a>(&'a self, _interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {})
}
}
/// What a [`MockHostRules`] currently holds, per interface.
pub type MockRulesState = std::collections::BTreeMap<String, BroadcastRulesPlan>;
/// An in-memory host, for tests: nothing is touched.
#[derive(Debug, Clone, Default)]
pub struct MockHostRules {
exit: MockExitRules,
state: Arc<Mutex<MockRulesState>>,
calls: Arc<Mutex<Vec<String>>>,
firewall_failure: Arc<Mutex<Option<String>>>,
}
impl MockHostRules {
/// An empty host.
pub fn new() -> Self {
Self {
exit: MockExitRules::new(),
..Self::default()
}
}
/// The exit-node half of this host, for inspecting and steering tests.
pub fn exit(&self) -> &MockExitRules {
&self.exit
}
/// Makes the firewall step fail with this reason from now on.
pub fn fail_firewall(&self, reason: impl Into<String>) {
*lock(&self.firewall_failure) = Some(reason.into());
}
/// What is installed now.
pub fn installed(&self) -> MockRulesState {
lock(&self.state).clone()
}
/// Every `apply:<interface>` / `clear:<interface>` call, in order.
pub fn calls(&self) -> Vec<String> {
lock(&self.calls).clone()
}
}
fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
match mutex.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
impl BroadcastHostRules for MockHostRules {
fn name(&self) -> &str {
"mock"
}
fn exit_rules(&self) -> Option<&dyn ExitHostRules> {
Some(&self.exit)
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
lock(&self.calls).push(format!("apply:{}", plan.interface));
lock(&self.state).insert(plan.interface.clone(), plan.clone());
let firewall = match lock(&self.firewall_failure).clone() {
Some(reason) => RuleOutcome::Failed(reason),
None => RuleOutcome::Applied,
};
BroadcastRulesReport {
plan: plan.clone(),
route: RuleOutcome::Applied,
firewall,
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
lock(&self.calls).push(format!("clear:{interface}"));
lock(&self.state).remove(interface);
})
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
fn plan() -> BroadcastRulesPlan {
BroadcastRulesPlan {
interface: "tsun0".into(),
source: Ipv4Addr::new(10, 13, 37, 142),
range: "10.13.37.0/24".parse().unwrap(),
}
}
#[test]
fn every_object_is_tagged_with_its_interface() {
assert_eq!(plan().tag(), "tsunagi:tsun0");
assert_eq!(tag_for("tsun7"), "tsunagi:tsun7");
}
#[test]
fn the_summary_says_which_half_is_missing() {
assert_eq!(
rules_summary(&RuleOutcome::Applied, &RuleOutcome::Applied),
"ok (route + firewall)"
);
let line = rules_summary(
&RuleOutcome::Applied,
&RuleOutcome::Failed("iptables not found".into()),
);
assert!(line.contains("route ok"), "{line}");
assert!(
line.contains("firewall failed: iptables not found"),
"{line}"
);
let report = BroadcastRulesReport {
plan: plan(),
route: RuleOutcome::Failed("denied".into()),
firewall: RuleOutcome::Applied,
};
assert!(!report.is_ok());
assert!(report.summary().contains("route failed: denied"));
}
#[tokio::test]
async fn the_mock_applies_replaces_and_clears_idempotently() {
let rules = MockHostRules::new();
let first = rules.apply(&plan()).await;
assert!(first.is_ok());
assert_eq!(rules.installed().len(), 1);
let mut moved = plan();
moved.source = Ipv4Addr::new(10, 13, 37, 7);
rules.apply(&moved).await;
assert_eq!(rules.installed()["tsun0"].source, moved.source);
rules.clear("tsun0").await;
rules.clear("tsun0").await;
assert!(rules.installed().is_empty());
assert_eq!(rules.calls().len(), 4);
}
#[tokio::test]
async fn an_unsupported_platform_reports_both_steps_failed() {
let report = UnsupportedHostRules.apply(&plan()).await;
assert!(!report.route.is_applied());
assert!(!report.firewall.is_applied());
UnsupportedHostRules.clear("tsun0").await;
}
}
@@ -0,0 +1,272 @@
//! Broadcast host rules on Windows: a route and a Defender Firewall rule.
//!
//! Both are written with the `NetTCPIP` and `NetSecurity` PowerShell modules,
//! run by absolute path from `%SystemRoot%` like the DNS publisher does: the
//! in-process alternatives are `unsafe` IP Helper calls this crate forbids.
//!
//! * The route is `255.255.255.255/32` on the overlay interface, on-link, with
//! metric 1, written to the active store only so it cannot outlive a reboot.
//! Windows picks the interface's own address as the source of a packet sent
//! along it; with several networks on one interface that is whichever
//! address Windows treats as primary, which this cannot choose.
//! * The firewall rule allows inbound UDP on that interface alias from the
//! overlay range only, in every profile. Wintun adapters are usually
//! classified as a public network, where inbound UDP is otherwise dropped.
//! Its display name is the tag `tsunagi:<interface>`, which is how it is
//! found again, replaced and removed.
//!
//! Every value in a script is one this agent produced: the interface name is
//! derived from the network id and the range comes from signed state. They are
//! still single-quoted and escaped, so nothing could be read as PowerShell
//! rather than as data. Changing routes and firewall rules needs an elevated
//! process, which is reported as its own cause.
//!
//! This module is compiled everywhere so its script building is tested
//! everywhere. It could not be run against a real Windows host while it was
//! written.
#![cfg_attr(not(target_os = "windows"), allow(dead_code))]
use crate::BoxFuture;
use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome, tag_for};
/// A PowerShell single-quoted literal, with any embedded quote doubled.
fn ps_literal(value: &str) -> String {
format!("'{}'", value.replace('\'', "''"))
}
/// The step that makes the route match the plan.
///
/// A leftover route (from a crashed run, or for an old interface index) is
/// removed first so the result is exactly one route.
fn route_script(plan: &BroadcastRulesPlan) -> String {
let alias = ps_literal(&plan.interface);
format!(
"$ErrorActionPreference = 'Stop'\n\
$alias = {alias}\n\
Get-NetRoute -DestinationPrefix '255.255.255.255/32' -InterfaceAlias $alias \
-PolicyStore ActiveStore -ErrorAction SilentlyContinue | \
Remove-NetRoute -Confirm:$false -ErrorAction SilentlyContinue\n\
New-NetRoute -DestinationPrefix '255.255.255.255/32' -InterfaceAlias $alias \
-NextHop '0.0.0.0' -RouteMetric 1 -PolicyStore ActiveStore | Out-Null\n"
)
}
/// The step that makes the firewall hold exactly one rule for the interface.
fn firewall_script(plan: &BroadcastRulesPlan) -> String {
let alias = ps_literal(&plan.interface);
let tag = ps_literal(&plan.tag());
let range = ps_literal(&plan.range.to_string());
format!(
"$ErrorActionPreference = 'Stop'\n\
$alias = {alias}\n\
$tag = {tag}\n\
Get-NetFirewallRule -DisplayName $tag -ErrorAction SilentlyContinue | \
Remove-NetFirewallRule -ErrorAction SilentlyContinue\n\
New-NetFirewallRule -DisplayName $tag -Direction Inbound -Action Allow -Protocol UDP \
-InterfaceAlias $alias -RemoteAddress {range} -Profile Any | Out-Null\n"
)
}
/// The script that removes both, ignoring what is not there.
fn clear_script(interface: &str) -> String {
let alias = ps_literal(interface);
let tag = ps_literal(&tag_for(interface));
format!(
"$ErrorActionPreference = 'SilentlyContinue'\n\
Get-NetRoute -DestinationPrefix '255.255.255.255/32' -InterfaceAlias {alias} \
-PolicyStore ActiveStore | Remove-NetRoute -Confirm:$false\n\
Get-NetFirewallRule -DisplayName {tag} | Remove-NetFirewallRule\n\
exit 0\n"
)
}
/// Turns what PowerShell said into a cause a person can act on.
fn classify(what: &str, text: &str) -> String {
let lower = text.to_ascii_lowercase();
if lower.contains("access is denied")
|| lower.contains("requires elevation")
|| lower.contains("run as administrator")
|| lower.contains("permissiondenied")
{
format!("{what}: Windows refused; run tsng from an elevated (administrator) process")
} else if lower.contains("is not recognized") || lower.contains("commandnotfound") {
format!("{what}: the NetTCPIP/NetSecurity PowerShell modules are not available: {text}")
} else if text.is_empty() {
format!("{what}: failed without saying why")
} else {
format!("{what}: {text}")
}
}
/// The absolute path to Windows PowerShell, so nothing on `PATH` can stand in
/// for it.
fn powershell_path() -> std::path::PathBuf {
let root = std::env::var_os("SystemRoot").unwrap_or_else(|| r"C:\Windows".into());
std::path::Path::new(&root)
.join("System32")
.join("WindowsPowerShell")
.join("v1.0")
.join("powershell.exe")
}
/// Runs a script and returns what it said on failure.
fn run_powershell(script: &str) -> Result<(), String> {
let output = std::process::Command::new(powershell_path())
.args([
"-NoProfile",
"-NonInteractive",
"-ExecutionPolicy",
"Bypass",
"-Command",
script,
])
.output()
.map_err(|err| format!("could not run PowerShell: {err}"))?;
if output.status.success() {
return Ok(());
}
let stderr = String::from_utf8_lossy(&output.stderr);
let text = if stderr.trim().is_empty() {
String::from_utf8_lossy(&output.stdout).trim().to_string()
} else {
stderr.trim().to_string()
};
Err(text)
}
async fn step(what: &'static str, script: String) -> RuleOutcome {
match tokio::task::spawn_blocking(move || run_powershell(&script)).await {
Ok(Ok(())) => RuleOutcome::Applied,
Ok(Err(text)) => RuleOutcome::Failed(classify(what, &text)),
Err(err) => RuleOutcome::Failed(format!("{what}: the task failed: {err}")),
}
}
/// The Windows implementation, through PowerShell.
#[derive(Debug, Default)]
pub struct WindowsHostRules;
impl WindowsHostRules {
/// Creates it. Nothing is run until [`BroadcastHostRules::apply`].
pub fn new() -> Self {
Self
}
}
impl BroadcastHostRules for WindowsHostRules {
fn name(&self) -> &str {
"windows-netsecurity"
}
fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
Box::pin(async move {
let route = step("route", route_script(plan)).await;
let firewall = step("firewall", firewall_script(plan)).await;
BroadcastRulesReport {
plan: plan.clone(),
route,
firewall,
}
})
}
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
if let RuleOutcome::Failed(reason) = step("clear", clear_script(interface)).await {
tracing::warn!(
interface,
"cannot remove the broadcast host rules: {reason}"
);
}
})
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use std::net::Ipv4Addr;
fn plan() -> BroadcastRulesPlan {
BroadcastRulesPlan {
interface: "tsun0".into(),
source: Ipv4Addr::new(10, 13, 37, 142),
range: "10.13.37.0/24".parse().unwrap(),
}
}
#[test]
fn a_quote_in_a_value_cannot_escape_the_literal() {
assert_eq!(ps_literal("a'b"), "'a''b'");
let mut hostile = plan();
hostile.interface = "x'; Remove-Item C:\\ #".into();
let script = route_script(&hostile);
assert!(
script.contains("$alias = 'x''; Remove-Item C:\\ #'"),
"{script}"
);
assert!(firewall_script(&hostile).contains("'x''; Remove-Item C:\\ #'"));
}
#[test]
fn the_route_is_on_link_metric_one_in_the_active_store_and_replaces_its_own() {
let script = route_script(&plan());
assert!(script.contains("'255.255.255.255/32'"));
assert!(script.contains("-NextHop '0.0.0.0'"));
assert!(script.contains("-RouteMetric 1"));
assert!(script.contains("-PolicyStore ActiveStore"));
assert!(
script.find("Remove-NetRoute").unwrap() < script.find("New-NetRoute").unwrap(),
"a stale route goes before the new one:\n{script}"
);
}
#[test]
fn the_firewall_rule_is_udp_only_from_the_overlay_range_on_this_interface() {
let script = firewall_script(&plan());
assert!(script.contains("-Protocol UDP"));
assert!(script.contains("-Direction Inbound -Action Allow"));
assert!(script.contains("-InterfaceAlias $alias"));
assert!(script.contains("-RemoteAddress '10.13.37.0/24'"));
assert!(script.contains("$tag = 'tsunagi:tsun0'"));
assert!(!script.contains("-Protocol Any") && !script.contains("-Protocol TCP"));
assert!(
script.find("Remove-NetFirewallRule").unwrap()
< script.find("New-NetFirewallRule").unwrap(),
"{script}"
);
}
#[test]
fn clearing_touches_only_our_route_and_our_tagged_rule() {
let script = clear_script("tsun0");
assert!(script.contains("-InterfaceAlias 'tsun0'"));
assert!(script.contains("Get-NetFirewallRule -DisplayName 'tsunagi:tsun0'"));
assert!(script.contains("exit 0"));
}
#[test]
fn a_refusal_names_elevation_and_a_missing_module_names_the_module() {
assert!(classify("route", "Access is denied").contains("elevated"));
assert!(
classify(
"firewall",
"The term 'New-NetFirewallRule' is not recognized"
)
.contains("modules are not available")
);
assert_eq!(classify("route", "boom"), "route: boom");
assert!(classify("route", "").contains("without saying why"));
}
#[test]
fn powershell_is_found_by_absolute_path() {
let path = powershell_path();
assert!(path.ends_with("powershell.exe"));
assert!(path.to_string_lossy().contains("System32"), "{path:?}");
}
}
File diff suppressed because it is too large. Load diff
@@ -11,6 +11,8 @@
//!
//! * [`provision`] creates the interface and configures it, and removes it
//! again. Platform mechanics behind one decision function.
//! * [`hostrules`] installs the route and firewall allowance that let a
//! game's LAN discovery reach the interface, while broadcast is on.
//! * [`tun`] is the packet interface itself, real or in memory.
//! * [`packet`] reads just enough of an IP header to route by it.
//! * [`config`] names interfaces and describes addresses.
@@ -36,7 +38,10 @@ pub enum OverlayError {
Other(String),
}
pub mod broadcast;
pub mod config;
pub mod exit;
pub mod hostrules;
pub mod interface;
pub mod packet;
pub mod provision;
@@ -44,6 +49,11 @@ pub mod router;
pub mod tun;
pub use config::{Cidr, DEFAULT_INTERFACE_PREFIX, MAX_INTERFACE_NAME_LEN, interface_name};
pub use exit::{ExitRulesReport, RuleSetReport};
pub use hostrules::{
BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, ExitHostPlan, ExitHostReport,
ExitHostRules, MockExitRules, MockHostRules, RuleOutcome, UnsupportedHostRules,
};
pub use interface::{Counters, Interface, PacketCarrier, Rejected};
pub use packet::IpHeader;
pub use provision::{
@@ -51,11 +61,20 @@ pub use provision::{
MockHost, MockProvisioner, Privilege, Provisioned, UnsupportedProvisioner, plan_changes,
probe_net_admin,
};
pub use router::{NetworkRoutes, Route, RouteError, RoutingTable};
pub use router::{ExitPolicy, NetworkRoutes, Route, RouteError, RoutingTable};
pub use tun::{MemoryTun, MemoryTunFactory, TunDevice, TunFactory, TunRequest, address_is_local};
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use hostrules::LinuxHostRules;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use hostrules::MacosHostRules;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use hostrules::WindowsHostRules;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use provision::NetlinkProvisioner;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use provision::UtunProvisioner;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use provision::WintunProvisioner;
@@ -11,6 +11,7 @@ use std::sync::Arc;
use crate::BoxFuture;
use crate::overlay::OverlayError;
use super::super::hostrules::BroadcastHostRules;
use super::super::tun::{TunDevice, TunFactory, TunRequest};
use super::{InterfacePlan, InterfaceProvisioner};
@@ -30,12 +31,23 @@ fn plan_for(request: &TunRequest) -> Result<InterfacePlan, OverlayError> {
#[derive(Debug)]
pub struct ManagedTunFactory {
provisioner: Arc<dyn InterfaceProvisioner>,
host_rules: Option<Arc<dyn BroadcastHostRules>>,
}
impl ManagedTunFactory {
/// Wraps a provisioner.
pub fn new(provisioner: Arc<dyn InterfaceProvisioner>) -> Self {
Self { provisioner }
Self {
provisioner,
host_rules: None,
}
}
/// Lets the agent install the broadcast route and firewall allowance for
/// the interface this factory manages.
pub fn with_host_rules(mut self, rules: Arc<dyn BroadcastHostRules>) -> Self {
self.host_rules = Some(rules);
self
}
/// The provisioner underneath.
@@ -54,6 +66,10 @@ impl TunFactory for ManagedTunFactory {
true
}
fn host_rules(&self) -> Option<Arc<dyn BroadcastHostRules>> {
self.host_rules.clone()
}
fn create<'a>(
&'a self,
request: TunRequest,
@@ -39,7 +39,7 @@
//! capability is raised immediately before that call and lowered immediately
//! after, and the connection task lives and dies inside it.
use std::net::IpAddr;
use std::net::{IpAddr, Ipv4Addr};
use std::sync::{Arc, Mutex, mpsc};
use futures_util::TryStreamExt;
@@ -47,7 +47,9 @@ use futures_util::TryStreamExt;
// of step with the client that sends them.
use rtnetlink::packet_route::address::{AddressAttribute, AddressMessage};
use rtnetlink::packet_route::link::{InfoKind, LinkAttribute, LinkFlags, LinkInfo, LinkMessage};
use rtnetlink::{LinkMessageBuilder, LinkUnspec};
use rtnetlink::packet_route::route::RouteScope;
use rtnetlink::packet_route::rule::{RuleAction, RuleAttribute, RuleMessage, RuleUidRange};
use rtnetlink::{IpVersion, LinkMessageBuilder, LinkUnspec, RouteMessageBuilder};
use crate::BoxFuture;
use crate::overlay::OverlayError;
@@ -67,6 +69,15 @@ enum Command {
Delete(String, Reply<()>),
/// Apply MTU, link state and addresses.
Configure(Box<Configure>, Reply<()>),
/// Route `255.255.255.255/32` through this interface from this address.
SetBroadcastRoute(String, Ipv4Addr, Reply<()>),
/// Remove that route from this interface.
DelBroadcastRoute(String, Reply<()>),
/// Send the host's default traffic through this interface, except what
/// this user id sends.
SetExitClient(String, u32, Reply<()>),
/// Take those routing rules away again.
ClearExitClient(Reply<()>),
/// Stop the thread.
Stop,
}
@@ -133,13 +144,18 @@ impl NetlinkProvisioner {
&self,
make: impl FnOnce(Reply<T>) -> Command,
) -> Result<T, OverlayError> {
let (reply_tx, reply_rx) = mpsc::channel();
self.commands
.send(make(reply_tx))
.map_err(|_| OverlayError::Unavailable("the netlink thread has stopped".to_string()))?;
reply_rx.recv().map_err(|_| {
OverlayError::Unavailable("the netlink thread stopped mid-request".to_string())
})?
call_thread(&self.commands, make)
}
/// The broadcast host rules for this host.
///
/// The route is added by this provisioner's netlink thread, because it is
/// the only one that ever holds `CAP_NET_ADMIN`, and only for the duration
/// of a call. Keep the provisioner alive for as long as the rules are used.
pub fn host_rules(&self) -> crate::overlay::hostrules::LinuxHostRules {
crate::overlay::hostrules::LinuxHostRules::new(RouteHandle {
commands: self.commands.clone(),
})
}
fn is_ours(&self, name: &str) -> bool {
@@ -157,6 +173,58 @@ impl NetlinkProvisioner {
}
}
fn call_thread<T: Send + 'static>(
commands: &mpsc::Sender<Command>,
make: impl FnOnce(Reply<T>) -> Command,
) -> Result<T, OverlayError> {
let (reply_tx, reply_rx) = mpsc::channel();
commands
.send(make(reply_tx))
.map_err(|_| OverlayError::Unavailable("the netlink thread has stopped".to_string()))?;
reply_rx.recv().map_err(|_| {
OverlayError::Unavailable("the netlink thread stopped mid-request".to_string())
})?
}
/// Adds and removes the limited-broadcast route on the netlink thread.
#[derive(Debug, Clone)]
pub(crate) struct RouteHandle {
commands: mpsc::Sender<Command>,
}
impl RouteHandle {
/// Routes `255.255.255.255/32` through `interface`, preferring `source`.
/// Replaces a route this agent added earlier, so it can be repeated.
pub(crate) fn set(&self, interface: &str, source: Ipv4Addr) -> Result<(), OverlayError> {
let interface = interface.to_string();
call_thread(&self.commands, |reply| {
Command::SetBroadcastRoute(interface, source, reply)
})
}
/// Removes the route. Succeeds when it, or the interface, is gone.
pub(crate) fn clear(&self, interface: &str) -> Result<(), OverlayError> {
let interface = interface.to_string();
call_thread(&self.commands, |reply| {
Command::DelBroadcastRoute(interface, reply)
})
}
/// Sends the host's default traffic through `interface`, except what the
/// user id `uid` sends. Replaces what an earlier call set up.
pub(crate) fn set_exit_client(&self, interface: &str, uid: u32) -> Result<(), OverlayError> {
let interface = interface.to_string();
call_thread(&self.commands, |reply| {
Command::SetExitClient(interface, uid, reply)
})
}
/// Removes those rules. Succeeds when none are there.
pub(crate) fn clear_exit_client(&self) -> Result<(), OverlayError> {
call_thread(&self.commands, Command::ClearExitClient)
}
}
impl Drop for NetlinkProvisioner {
fn drop(&mut self) {
let _ = self.commands.send(Command::Stop);
@@ -273,7 +341,14 @@ fn netlink_thread(requests: mpsc::Receiver<Command>) {
Command::Observe(_, reply) => {
let _ = reply.send(Err(OverlayError::Unavailable(message)));
}
Command::Delete(_, reply) | Command::Configure(_, reply) => {
Command::Delete(_, reply)
| Command::Configure(_, reply)
| Command::SetBroadcastRoute(_, _, reply)
| Command::DelBroadcastRoute(_, reply)
| Command::SetExitClient(_, _, reply) => {
let _ = reply.send(Err(OverlayError::Unavailable(message)));
}
Command::ClearExitClient(reply) => {
let _ = reply.send(Err(OverlayError::Unavailable(message)));
}
Command::Stop => return,
@@ -306,6 +381,38 @@ fn netlink_thread(requests: mpsc::Receiver<Command>) {
});
let _ = reply.send(result);
}
Command::SetBroadcastRoute(name, source, reply) => {
let result = NetAdmin::acquire().and_then(|guard| {
let result = runtime.block_on(set_broadcast_route(&name, source));
drop(guard);
result
});
let _ = reply.send(result);
}
Command::DelBroadcastRoute(name, reply) => {
let result = NetAdmin::acquire().and_then(|guard| {
let result = runtime.block_on(del_broadcast_route(&name));
drop(guard);
result
});
let _ = reply.send(result);
}
Command::SetExitClient(name, uid, reply) => {
let result = NetAdmin::acquire().and_then(|guard| {
let result = runtime.block_on(set_exit_client(&name, uid));
drop(guard);
result
});
let _ = reply.send(result);
}
Command::ClearExitClient(reply) => {
let result = NetAdmin::acquire().and_then(|guard| {
let result = runtime.block_on(clear_exit_client());
drop(guard);
result
});
let _ = reply.send(result);
}
}
}
}
@@ -455,6 +562,382 @@ async fn delete_link(name: &str) -> Result<(), OverlayError> {
.await
}
/// The message that names the limited-broadcast route on an interface.
///
/// On-link scope is what the kernel requires of a route with no gateway, and
/// what `ip route add 255.255.255.255/32 dev <if>` produces. Deleting needs the
/// same scope or the kernel finds no match.
fn broadcast_route_message(
index: u32,
source: Option<Ipv4Addr>,
) -> rtnetlink::packet_route::route::RouteMessage {
let mut builder = RouteMessageBuilder::<Ipv4Addr>::new()
.destination_prefix(Ipv4Addr::BROADCAST, 32)
.output_interface(index)
.scope(RouteScope::Link);
if let Some(source) = source {
builder = builder.pref_source(source);
}
builder.build()
}
async fn link_index(handle: &rtnetlink::Handle, name: &str) -> Option<u32> {
let mut links = handle.link().get().match_name(name.to_string()).execute();
links
.try_next()
.await
.ok()
.flatten()
.map(|message| message.header.index)
}
async fn set_broadcast_route(name: &str, source: Ipv4Addr) -> Result<(), OverlayError> {
with_netlink(|handle| async move {
let index = link_index(&handle, name).await.ok_or_else(|| {
OverlayError::Unavailable(format!(
"interface `{name}` disappeared before its broadcast route could be added"
))
})?;
handle
.route()
.add(broadcast_route_message(index, Some(source)))
.replace()
.execute()
.await
.map_err(|err| {
OverlayError::Unavailable(format!(
"cannot route 255.255.255.255 through `{name}` from {source}: {err}"
))
})
})
.await
}
async fn del_broadcast_route(name: &str) -> Result<(), OverlayError> {
with_netlink(|handle| async move {
// An interface that is gone took its routes with it.
let Some(index) = link_index(&handle, name).await else {
return Ok(());
};
match handle
.route()
.del(broadcast_route_message(index, None))
.execute()
.await
{
Ok(()) => Ok(()),
// "No such process": there was no such route, which is the goal.
Err(err) if err.to_string().contains("No such process") => Ok(()),
Err(err) => Err(OverlayError::Unavailable(format!(
"cannot remove the 255.255.255.255 route from `{name}`: {err}"
))),
}
})
.await
}
/// The routing table that holds the exit node's default route: `tsun` in
/// ASCII-ish hex, and nothing else uses it.
const EXIT_TABLE: u32 = 0x7473;
/// The main table.
const MAIN_TABLE: u32 = 254;
/// Where the exit rules sit, after Tailscale's 5270.
const EXIT_BYPASS_PRIORITY: u32 = 5280;
const EXIT_SPECIFIC_PRIORITY: u32 = 5290;
const EXIT_DEFAULT_PRIORITY: u32 = 5300;
/// The table a rule looks up.
fn rule_table(rule: &RuleMessage) -> u32 {
rule.attributes
.iter()
.find_map(|attribute| match attribute {
RuleAttribute::Table(table) => Some(*table),
_ => None,
})
.unwrap_or(u32::from(rule.header.table))
}
fn rule_priority(rule: &RuleMessage) -> Option<u32> {
rule.attributes
.iter()
.find_map(|attribute| match attribute {
RuleAttribute::Priority(priority) => Some(*priority),
_ => None,
})
}
/// Whether a rule is one of the three this agent adds for an exit node.
///
/// Recognised by shape as well as priority: those numbers are not reserved,
/// and a rule somebody else put at one of them must not be removed.
fn is_exit_rule(rule: &RuleMessage) -> bool {
let table = rule_table(rule);
match rule_priority(rule) {
Some(EXIT_BYPASS_PRIORITY) => {
table == MAIN_TABLE
&& rule
.attributes
.iter()
.any(|attribute| matches!(attribute, RuleAttribute::UidRange(_)))
}
Some(EXIT_SPECIFIC_PRIORITY) => {
table == MAIN_TABLE
&& rule
.attributes
.iter()
.any(|attribute| matches!(attribute, RuleAttribute::SuppressPrefixLen(0)))
}
Some(EXIT_DEFAULT_PRIORITY) => table == EXIT_TABLE,
_ => false,
}
}
fn exit_default_route(index: Option<u32>) -> rtnetlink::packet_route::route::RouteMessage {
let mut builder = RouteMessageBuilder::<Ipv4Addr>::new()
.destination_prefix(Ipv4Addr::UNSPECIFIED, 0)
.table_id(EXIT_TABLE);
if let Some(index) = index {
builder = builder.output_interface(index).scope(RouteScope::Link);
}
builder.build()
}
/// The exit table's IPv6 default: nothing is reachable through it.
///
/// There is no IPv6 through an exit node, and what the ordinary route would do
/// is send it out in the clear. This turns it into an immediate refusal, which
/// is what makes applications fall back to IPv4.
fn exit_blocking_route_v6() -> rtnetlink::packet_route::route::RouteMessage {
RouteMessageBuilder::<std::net::Ipv6Addr>::new()
.destination_prefix(std::net::Ipv6Addr::UNSPECIFIED, 0)
.table_id(EXIT_TABLE)
.kind(rtnetlink::packet_route::route::RouteType::Unreachable)
.build()
}
/// A host with no IPv6 at all has nothing to block, and says so this way.
fn family_unsupported(err: &impl std::fmt::Display) -> bool {
let text = err.to_string();
text.contains("not supported") || text.contains("Address family")
}
/// Removes the exit node's routing rules and its tables. Nothing there is
/// success: it also runs once at startup, for what a crashed run left.
async fn clear_exit_client() -> Result<(), OverlayError> {
with_netlink(|handle| async move { remove_exit_client(&handle).await }).await
}
/// Deletes this agent's exit rules of one address family.
async fn remove_exit_rules(
handle: &rtnetlink::Handle,
version: IpVersion,
) -> Result<(), OverlayError> {
let mut rules = handle.rule().get(version).execute();
let mut ours = Vec::new();
while let Some(rule) = rules
.try_next()
.await
.map_err(|err| OverlayError::Unavailable(format!("cannot read the routing rules: {err}")))?
{
if is_exit_rule(&rule) {
ours.push(rule);
}
}
for rule in ours {
match handle.rule().del(rule).execute().await {
Ok(()) => {}
// Gone already.
Err(err) if err.to_string().contains("No such") => {}
Err(err) => {
return Err(OverlayError::Unavailable(format!(
"cannot remove an exit node routing rule: {err}"
)));
}
}
}
Ok(())
}
async fn remove_exit_client(handle: &rtnetlink::Handle) -> Result<(), OverlayError> {
remove_exit_rules(handle, IpVersion::V4).await?;
match handle.route().del(exit_default_route(None)).execute().await {
Ok(()) => {}
// "No such process": no such route, which is the goal.
Err(err) if err.to_string().contains("No such") => {}
Err(err) => {
return Err(OverlayError::Unavailable(format!(
"cannot remove the exit node default route: {err}"
)));
}
}
match remove_exit_rules(handle, IpVersion::V6).await {
Ok(()) => {}
Err(OverlayError::Unavailable(reason)) if family_unsupported(&reason) => return Ok(()),
Err(err) => return Err(err),
}
match handle.route().del(exit_blocking_route_v6()).execute().await {
Ok(()) => Ok(()),
Err(err) if err.to_string().contains("No such") || family_unsupported(&err) => Ok(()),
Err(err) => Err(OverlayError::Unavailable(format!(
"cannot remove the exit node IPv6 block: {err}"
))),
}
}
/// Sets up the using side of an exit node.
///
/// ```text
/// ip route replace default dev <if> table 0x7473
/// ip rule add priority 5280 uidrange <uid>-<uid> lookup main
/// ip rule add priority 5290 lookup main suppress_prefixlength 0
/// ip rule add priority 5300 lookup 0x7473
/// ```
///
/// and the same for IPv6 with `ip -6 route replace unreachable default table
/// 0x7473`, so that IPv6 is refused instead of leaving the ordinary way.
///
/// What an earlier call left is removed first, so repeating it is safe, and
/// the rules go in last, so a failure part way never leaves a rule that
/// sends traffic to a table with no route in it.
async fn set_exit_client(name: &str, uid: u32) -> Result<(), OverlayError> {
with_netlink(|handle| async move {
let index = link_index(&handle, name).await.ok_or_else(|| {
OverlayError::Unavailable(format!(
"interface `{name}` disappeared before the exit node routes could be added"
))
})?;
remove_exit_client(&handle).await?;
let fail = |what: &str, err: rtnetlink::Error| {
OverlayError::Unavailable(format!("cannot add the exit node {what}: {err}"))
};
handle
.route()
.add(exit_default_route(Some(index)))
.replace()
.execute()
.await
.map_err(|err| fail("default route", err))?;
let mut bypass = handle
.rule()
.add()
.v4()
.table_id(MAIN_TABLE)
.action(RuleAction::ToTable)
.priority(EXIT_BYPASS_PRIORITY);
bypass
.message_mut()
.attributes
.push(RuleAttribute::UidRange(RuleUidRange {
start: uid,
end: uid,
}));
bypass
.execute()
.await
.map_err(|err| fail("rule for the agent's own traffic", err))?;
let mut specific = handle
.rule()
.add()
.v4()
.table_id(MAIN_TABLE)
.action(RuleAction::ToTable)
.priority(EXIT_SPECIFIC_PRIORITY);
specific
.message_mut()
.attributes
.push(RuleAttribute::SuppressPrefixLen(0));
specific
.execute()
.await
.map_err(|err| fail("rule for the specific routes", err))?;
handle
.rule()
.add()
.v4()
.table_id(EXIT_TABLE)
.action(RuleAction::ToTable)
.priority(EXIT_DEFAULT_PRIORITY)
.execute()
.await
.map_err(|err| fail("rule for the default route", err))?;
set_exit_block_v6(&handle, uid).await
})
.await
}
/// The IPv6 half: an unreachable default for everything but the agent's own
/// connections and the specific routes, which keep working.
async fn set_exit_block_v6(handle: &rtnetlink::Handle, uid: u32) -> Result<(), OverlayError> {
let fail = |what: &str, err: rtnetlink::Error| {
OverlayError::Unavailable(format!("cannot add the exit node IPv6 {what}: {err}"))
};
if let Err(err) = handle
.route()
.add(exit_blocking_route_v6())
.replace()
.execute()
.await
{
// Without IPv6 there is nothing to leak.
return if family_unsupported(&err) {
Ok(())
} else {
Err(fail("block", err))
};
}
let mut bypass = handle
.rule()
.add()
.v6()
.table_id(MAIN_TABLE)
.action(RuleAction::ToTable)
.priority(EXIT_BYPASS_PRIORITY);
bypass
.message_mut()
.attributes
.push(RuleAttribute::UidRange(RuleUidRange {
start: uid,
end: uid,
}));
bypass
.execute()
.await
.map_err(|err| fail("rule for the agent's own traffic", err))?;
let mut specific = handle
.rule()
.add()
.v6()
.table_id(MAIN_TABLE)
.action(RuleAction::ToTable)
.priority(EXIT_SPECIFIC_PRIORITY);
specific
.message_mut()
.attributes
.push(RuleAttribute::SuppressPrefixLen(0));
specific
.execute()
.await
.map_err(|err| fail("rule for the specific routes", err))?;
handle
.rule()
.add()
.v6()
.table_id(EXIT_TABLE)
.action(RuleAction::ToTable)
.priority(EXIT_DEFAULT_PRIORITY)
.execute()
.await
.map_err(|err| fail("rule for the default route", err))
}
async fn configure_link(configure: &Configure) -> Result<(), OverlayError> {
let name = configure.name.as_str();
with_netlink(|handle| async move {
@@ -520,16 +1003,19 @@ async fn configure_link(configure: &Configure) -> Result<(), OverlayError> {
}
for cidr in &configure.add {
handle
let res = handle
.address()
.add(index, cidr.addr, cidr.prefix_len)
.execute()
.await
.map_err(|err| {
OverlayError::Unavailable(format!(
.await;
if let Err(err) = res {
let err_str = err.to_string();
if !err_str.contains("File exists") && !err_str.contains("os error 17") {
return Err(OverlayError::Unavailable(format!(
"cannot add {cidr} to interface `{name}`: {err}"
))
})?;
)));
}
}
}
Ok(())
@@ -593,6 +1079,67 @@ mod tests {
);
}
fn rule(priority: u32, table: u32, extra: Vec<RuleAttribute>) -> RuleMessage {
let mut message = RuleMessage::default();
message.attributes.push(RuleAttribute::Priority(priority));
if table > 255 {
message.attributes.push(RuleAttribute::Table(table));
} else {
message.header.table = table as u8;
}
message.attributes.extend(extra);
message
}
#[test]
fn only_the_three_exit_rules_are_recognised_as_ours() {
let uid = RuleAttribute::UidRange(RuleUidRange { start: 5, end: 5 });
assert!(is_exit_rule(&rule(5280, 254, vec![uid.clone()])));
assert!(is_exit_rule(&rule(
5290,
254,
vec![RuleAttribute::SuppressPrefixLen(0)]
)));
assert!(is_exit_rule(&rule(5300, EXIT_TABLE, vec![])));
// Same priorities, somebody else's rules.
assert!(!is_exit_rule(&rule(5280, 254, vec![])));
assert!(!is_exit_rule(&rule(5290, 254, vec![])));
assert!(!is_exit_rule(&rule(5300, 52, vec![])));
assert!(!is_exit_rule(&rule(5270, 52, vec![])));
assert!(!is_exit_rule(&rule(5280, 100, vec![uid])));
}
#[test]
fn the_ipv6_default_of_the_exit_table_refuses_instead_of_routing() {
let route = exit_blocking_route_v6();
assert_eq!(route.header.destination_prefix_length, 0);
let debug = format!("{route:?}");
assert!(debug.contains("Unreachable"), "{debug}");
assert!(debug.contains(&format!("Table({EXIT_TABLE})")), "{debug}");
assert!(!debug.contains("Oif"), "{debug}");
}
#[test]
fn a_host_without_ipv6_is_not_a_failure() {
assert!(family_unsupported(
&"Address family not supported by protocol"
));
assert!(family_unsupported(&"Operation not supported"));
assert!(!family_unsupported(&"Operation not permitted"));
}
#[test]
fn the_default_route_of_the_exit_table_names_the_table_and_the_interface() {
let route = exit_default_route(Some(7));
assert_eq!(route.header.destination_prefix_length, 0);
let debug = format!("{route:?}");
assert!(debug.contains(&format!("Table({EXIT_TABLE})")), "{debug}");
assert!(debug.contains("Oif(7)"), "{debug}");
// Deleting names no interface, so it matches whatever it was added on.
assert!(!format!("{:?}", exit_default_route(None)).contains("Oif"));
}
#[test]
fn an_interface_that_is_not_a_tun_is_reported_foreign() {
let message = LinkMessage::default();
@@ -0,0 +1,531 @@
//! Managing the overlay interface on macOS, through utun and `ifconfig`/`route`.
//!
//! The shape is the same as the Linux and Windows provisioners: observe what
//! is there, [`plan_changes`](super::plan_changes) to decide what to do, apply
//! the difference. Only the mechanisms differ.
//!
//! # Creating the interface
//!
//! The interface is a `utun` device, created through the `tun` dependency's
//! safe wrapper — [`open_tun`](super::super::tun) — over the in-kernel
//! `com.apple.net.utun_control` control socket. No kernel extension and no
//! Network Extension entitlement is involved; what it needs is root. The
//! device is not made persistent, so closing the last file descriptor removes
//! the interface — it goes away when the agent does, however the agent ends.
//!
//! # The kernel names the interface, not us
//!
//! Unlike Linux and Windows, a utun interface cannot be given an arbitrary
//! name: it is always `utunN`, and the unit number is assigned by the kernel.
//! So the name derived from the network id (`tsun…`) is only ever a *request*;
//! the real name is read back off the device after it is opened, and
//! everything downstream — the host rules, the resolver publication — uses the
//! real one. The agent adopts it as soon as the device is created (see
//! [`Interface`](crate::overlay::Interface)).
//!
//! # Configuring it
//!
//! Addresses, MTU and routes are applied with `ifconfig` and `route`. On Linux
//! the same work is done in process over a netlink socket; here there is no
//! equivalent this crate may call, because the in-process route is raw `ioctl`
//! and `PF_ROUTE` and that is `unsafe`. The tools are invoked by absolute path
//! from `/sbin`, so nothing on `PATH` can stand in for them, and every value
//! handed to them is one this agent derived — an interface name chosen by the
//! kernel, an address it allocated — never anything a peer said.
//!
//! # Observing it
//!
//! The interface table is read through `netdev`, a safe wrapper over
//! `getifaddrs`. An interface is matched by its real `utunN` name, which only
//! matters on reconfiguration: the first create always makes a fresh device.
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use std::sync::{Arc, Mutex};
use crate::BoxFuture;
use crate::overlay::OverlayError;
use super::super::config::Cidr;
use super::super::tun::{TunDevice, TunRequest, open_tun};
use super::privilege::{Privilege, probe_net_admin};
use super::{
InterfacePlan, InterfaceProvisioner, InterfaceState, LinkKind, Provisioned, plan_changes,
};
/// Manages the overlay interface with utun and `ifconfig`/`route`.
#[derive(Debug)]
pub struct UtunProvisioner {
/// Interfaces this process created, by their real `utunN` name, so a
/// reconfiguration adjusts them rather than trying to create them again.
ours: Mutex<Vec<String>>,
/// Devices kept alive for as long as the interface should exist. Dropping
/// one closes the last descriptor, which removes the utin interface.
held: Mutex<Vec<(String, Arc<dyn TunDevice>)>>,
}
impl UtunProvisioner {
/// Creates the provisioner, after checking the platform supports one.
///
/// Whether the process is actually root is not checked here — reading the
/// effective uid needs a libc call this crate forbids — so a missing
/// privilege surfaces at utun creation with a precise message, the same
/// way the Linux and Windows paths treat the open itself as the honest
/// answer.
pub fn new() -> Result<Self, OverlayError> {
match probe_net_admin() {
Privilege::Available => {}
Privilege::Missing(reason) => return Err(OverlayError::Unavailable(reason)),
Privilege::Unsupported => {
return Err(OverlayError::Unavailable(
"interface management is not compiled in".to_string(),
));
}
}
Ok(Self {
ours: Mutex::new(Vec::new()),
held: Mutex::new(Vec::new()),
})
}
fn is_ours(&self, name: &str) -> bool {
lock(&self.ours).iter().any(|owned| owned == name)
}
/// Opens the utun device, which is what creates it. The kernel assigns the
/// name, so the returned device reports a `utunN` that is not `plan.name`.
fn create_device(&self, plan: &InterfacePlan) -> Result<Arc<dyn TunDevice>, OverlayError> {
let request = TunRequest::bare(plan.name.clone(), plan.mtu);
open_tun(&request)
}
}
impl InterfaceProvisioner for UtunProvisioner {
fn name(&self) -> &str {
"utun"
}
fn reconcile<'a>(
&'a self,
plan: &'a InterfacePlan,
) -> BoxFuture<'a, Result<Provisioned, OverlayError>> {
Box::pin(async move {
// A reconfiguration: the plan carries the real `utunN` name the
// agent adopted after the first create, so the interface is there
// and only the addresses and MTU need bringing into line.
if self.is_ours(&plan.name) {
let current = observe(&plan.name).await;
let changes = plan_changes(&current, plan, true)?;
configure(&plan.name, &changes).await?;
return Ok(Provisioned {
changes,
device: None,
});
}
// The first create. The kernel assigns the unit, so the device is
// opened first and its real name used for everything after.
let device = self.create_device(plan)?;
let real = device.name().to_string();
// A fresh interface starts with nothing on it, so the whole plan is
// applied. `plan_changes` against an absent interface is exactly
// that decision; its `create_link` flag is already satisfied by the
// open above, so `configure` ignores it and only applies the rest.
let changes = plan_changes(&InterfaceState::absent(), plan, false)?;
// A failure here leaves an interface that exists but cannot carry
// traffic. Drop the device so the utun closes, and report, rather
// than leave a trap behind.
if let Err(err) = configure(&real, &changes).await {
drop(device);
return Err(err);
}
lock(&self.ours).push(real.clone());
lock(&self.held).push((real, Arc::clone(&device)));
Ok(Provisioned {
changes,
device: Some(device),
})
})
}
fn remove<'a>(&'a self, name: &'a str) -> BoxFuture<'a, Result<(), OverlayError>> {
Box::pin(async move {
// Dropping the device closes the last descriptor, which removes the
// utun interface and the routes that were bound to it. There is
// nothing to flush: a utun never outlives the process that holds it.
lock(&self.held).retain(|(held, _)| held != name);
lock(&self.ours).retain(|owned| owned != name);
Ok(())
})
}
}
fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
match mutex.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
/// `fe80::/10` and `169.254.0.0/16`, which macOS assigns on its own.
fn is_link_local(addr: IpAddr) -> bool {
match addr {
IpAddr::V4(addr) => addr.is_link_local(),
IpAddr::V6(addr) => (addr.segments()[0] & 0xffc0) == 0xfe80,
}
}
/// What the interface named `name` currently looks like.
///
/// Never fails: a name nothing is using reads as [`InterfaceState::absent`],
/// which is what the plan then acts on.
async fn observe(name: &str) -> InterfaceState {
let name = name.to_string();
let found = tokio::task::spawn_blocking(move || {
netdev::get_interfaces()
.into_iter()
.find(|iface| iface.name == name)
})
.await
.ok()
.flatten();
let Some(iface) = found else {
return InterfaceState::absent();
};
let mut addresses: Vec<Cidr> = Vec::new();
for net in &iface.ipv4 {
if let Ok(cidr) = Cidr::new(IpAddr::V4(net.addr()), net.prefix_len())
&& !is_link_local(cidr.addr)
{
addresses.push(cidr);
}
}
for net in &iface.ipv6 {
if let Ok(cidr) = Cidr::new(IpAddr::V6(net.addr()), net.prefix_len())
&& !is_link_local(cidr.addr)
{
addresses.push(cidr);
}
}
addresses.sort();
InterfaceState {
// Only ever reached for an interface this agent created (see
// `reconcile`), so it is ours and a TUN by construction.
kind: LinkKind::Tun,
attached: iface.is_up(),
up: iface.is_up(),
mtu: iface.mtu.unwrap_or(0),
addresses,
}
}
/// Applies MTU, link state, addresses and the on-link routes they imply.
async fn configure(name: &str, changes: &super::Changes) -> Result<(), OverlayError> {
if let Some(mtu) = changes.set_mtu {
set_mtu(name, mtu).await?;
}
for cidr in &changes.remove {
// Best effort: an address that is already gone is the outcome wanted.
if let Err(err) = address_del(name, *cidr).await {
tracing::debug!(interface = %name, %cidr, %err, "could not remove address");
}
if let Err(err) = route_del(name, *cidr).await {
tracing::debug!(interface = %name, %cidr, %err, "could not remove route");
}
}
for cidr in &changes.add {
address_add(name, *cidr).await?;
// The subnet route makes the whole overlay range reach the interface;
// a point-to-point utun does not install one from the address alone. A
// host address (/32 or /128) needs none. Best effort: a route that is
// already there is the outcome wanted.
if let Err(err) = route_add(name, *cidr).await {
tracing::debug!(interface = %name, %cidr, %err, "could not add route");
}
}
if changes.bring_up {
// Best effort: a utun with a descriptor open is up already, and a
// failure to nudge it is not a reason to fail the whole reconcile.
if let Err(err) = set_up(name).await {
tracing::debug!(interface = %name, %err, "could not enable the interface");
}
}
Ok(())
}
/// The network address of a CIDR, with the host bits cleared.
fn network_of(cidr: Cidr) -> IpAddr {
match cidr.addr {
IpAddr::V4(addr) => {
let bits = u32::from(addr);
let mask = if cidr.prefix_len == 0 {
0
} else {
u32::MAX << (32 - u32::from(cidr.prefix_len))
};
IpAddr::V4(Ipv4Addr::from(bits & mask))
}
IpAddr::V6(addr) => {
let bits = u128::from(addr);
let mask = if cidr.prefix_len == 0 {
0
} else {
u128::MAX << (128 - u32::from(cidr.prefix_len))
};
IpAddr::V6(Ipv6Addr::from(bits & mask))
}
}
}
/// Whether a CIDR is a single host address, which needs no subnet route.
fn is_host_route(cidr: Cidr) -> bool {
match cidr.addr {
IpAddr::V4(_) => cidr.prefix_len >= 32,
IpAddr::V6(_) => cidr.prefix_len >= 128,
}
}
/// The `ifconfig` arguments that add an address to a utun interface.
///
/// A utun is point-to-point, so the IPv4 form names the address as its own
/// peer; `alias` lets several coexist on the one interface.
fn address_add_args(name: &str, cidr: Cidr) -> Vec<String> {
match cidr.addr {
IpAddr::V4(addr) => vec![
name.into(),
"inet".into(),
format!("{addr}/{}", cidr.prefix_len),
addr.to_string(),
"alias".into(),
],
IpAddr::V6(addr) => vec![
name.into(),
"inet6".into(),
format!("{addr}/{}", cidr.prefix_len),
"alias".into(),
],
}
}
/// The `ifconfig` arguments that remove an address from a utun interface.
fn address_del_args(name: &str, cidr: Cidr) -> Vec<String> {
let family = match cidr.addr {
IpAddr::V4(_) => "inet",
IpAddr::V6(_) => "inet6",
};
vec![
name.into(),
family.into(),
cidr.addr.to_string(),
"-alias".into(),
]
}
/// The `route` arguments that point a CIDR's subnet at the interface.
fn route_add_args(name: &str, cidr: Cidr) -> Vec<String> {
let family = match cidr.addr {
IpAddr::V4(_) => "-inet",
IpAddr::V6(_) => "-inet6",
};
vec![
"-q".into(),
"-n".into(),
"add".into(),
family.into(),
"-net".into(),
format!("{}/{}", network_of(cidr), cidr.prefix_len),
"-interface".into(),
name.into(),
]
}
/// The `route` arguments that remove that subnet route.
fn route_del_args(name: &str, cidr: Cidr) -> Vec<String> {
let family = match cidr.addr {
IpAddr::V4(_) => "-inet",
IpAddr::V6(_) => "-inet6",
};
vec![
"-q".into(),
"-n".into(),
"delete".into(),
family.into(),
"-net".into(),
format!("{}/{}", network_of(cidr), cidr.prefix_len),
"-interface".into(),
name.into(),
]
}
async fn address_add(name: &str, cidr: Cidr) -> Result<(), OverlayError> {
ifconfig(address_add_args(name, cidr)).await
}
async fn address_del(name: &str, cidr: Cidr) -> Result<(), OverlayError> {
ifconfig(address_del_args(name, cidr)).await
}
async fn route_add(name: &str, cidr: Cidr) -> Result<(), OverlayError> {
if is_host_route(cidr) {
return Ok(());
}
route(route_add_args(name, cidr)).await
}
async fn route_del(name: &str, cidr: Cidr) -> Result<(), OverlayError> {
if is_host_route(cidr) {
return Ok(());
}
route(route_del_args(name, cidr)).await
}
async fn set_mtu(name: &str, mtu: u32) -> Result<(), OverlayError> {
ifconfig(vec![name.into(), "mtu".into(), mtu.to_string()]).await
}
async fn set_up(name: &str) -> Result<(), OverlayError> {
ifconfig(vec![name.into(), "up".into()]).await
}
/// Runs `/sbin/ifconfig`, turning a non-zero exit into an error with its text.
async fn ifconfig(args: Vec<String>) -> Result<(), OverlayError> {
run("/sbin/ifconfig", args).await
}
/// Runs `/sbin/route`, turning a non-zero exit into an error with its text.
async fn route(args: Vec<String>) -> Result<(), OverlayError> {
run("/sbin/route", args).await
}
/// Runs a system tool by absolute path, so nothing on `PATH` can stand in for
/// it, and turns a non-zero exit into an error carrying what it said.
async fn run(program: &'static str, args: Vec<String>) -> Result<(), OverlayError> {
let display = format!("{program} {}", args.join(" "));
let output = tokio::task::spawn_blocking(move || {
std::process::Command::new(program).args(&args).output()
})
.await
.map_err(|err| OverlayError::Other(format!("could not run {display}: {err}")))?
.map_err(|err| OverlayError::Unavailable(format!("could not run {display}: {err}")))?;
if output.status.success() {
return Ok(());
}
let message = {
let stderr = String::from_utf8_lossy(&output.stderr);
let text = if stderr.trim().is_empty() {
String::from_utf8_lossy(&output.stdout).trim().to_string()
} else {
stderr.trim().to_string()
};
if text.is_empty() {
format!("exit code {}", output.status)
} else {
text
}
};
Err(OverlayError::Unavailable(format!(
"{display} failed: {message}"
)))
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
fn v4(last: u8, prefix: u8) -> Cidr {
Cidr::new(IpAddr::V4(Ipv4Addr::new(10, 13, 37, last)), prefix).unwrap()
}
fn v6(last: u16, prefix: u8) -> Cidr {
Cidr::new(
IpAddr::V6(Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0, last)),
prefix,
)
.unwrap()
}
#[test]
fn a_v4_address_is_added_point_to_point_with_an_alias() {
let args = address_add_args("utun5", v4(69, 24));
assert_eq!(
args,
vec!["utun5", "inet", "10.13.37.69/24", "10.13.37.69", "alias"]
);
}
#[test]
fn a_v6_address_is_added_with_a_prefix_and_an_alias() {
let args = address_add_args("utun5", v6(1, 128));
assert_eq!(args, vec!["utun5", "inet6", "fd00::1/128", "alias"]);
}
#[test]
fn deleting_an_address_names_the_family_and_removes_the_alias() {
let args = address_del_args("utun5", v4(69, 24));
assert_eq!(args, vec!["utun5", "inet", "10.13.37.69", "-alias"]);
let args = address_del_args("utun5", v6(1, 128));
assert_eq!(args, vec!["utun5", "inet6", "fd00::1", "-alias"]);
}
#[test]
fn a_subnet_route_points_the_masked_network_at_the_interface() {
let args = route_add_args("utun5", v4(69, 24));
assert_eq!(
args,
vec![
"-q",
"-n",
"add",
"-inet",
"-net",
"10.13.37.0/24",
"-interface",
"utun5"
]
);
}
#[test]
fn a_host_address_installs_no_subnet_route() {
assert!(is_host_route(v4(69, 32)));
assert!(is_host_route(v6(1, 128)));
assert!(!is_host_route(v4(69, 24)));
assert!(!is_host_route(v6(1, 64)));
}
#[test]
fn the_network_address_clears_the_host_bits() {
assert_eq!(
network_of(v4(69, 24)),
IpAddr::V4(Ipv4Addr::new(10, 13, 37, 0))
);
assert_eq!(
network_of(Cidr::new(IpAddr::V4(Ipv4Addr::new(10, 13, 37, 69)), 16).unwrap()),
IpAddr::V4(Ipv4Addr::new(10, 13, 0, 0))
);
assert_eq!(
network_of(v6(0x1234, 64)),
IpAddr::V6(Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0, 0))
);
}
#[test]
fn link_local_addresses_are_not_ours_to_manage() {
assert!(is_link_local(IpAddr::V6(Ipv6Addr::new(
0xfe80, 0, 0, 0, 0, 0, 0, 1
))));
assert!(is_link_local(IpAddr::V4(Ipv4Addr::new(169, 254, 1, 1))));
assert!(!is_link_local(IpAddr::V4(Ipv4Addr::new(10, 13, 37, 1))));
assert!(!is_link_local(IpAddr::V6(Ipv6Addr::new(
0xfd00, 0, 0, 0, 0, 0, 0, 1
))));
}
}
@@ -20,11 +20,13 @@
//! pure and tested on every platform, and only the execution is behind
//! [`InterfaceProvisioner`].
//!
//! Four implementations:
//! Five implementations:
//!
//! * `NetlinkProvisioner` on Linux, which needs `CAP_NET_ADMIN`.
//! * `WintunProvisioner` on Windows, a Wintun adapter configured with `netsh`,
//! which needs an elevated process.
//! * `UtunProvisioner` on macOS, a utun interface configured with `ifconfig`
//! and `route`, which needs root.
//! * [`MockProvisioner`], an in-memory host used by the tests.
//! * [`UnsupportedProvisioner`] elsewhere, which fails with an explanation
//! and a pointer at the manual route rather than pretending to work.
@@ -52,12 +54,19 @@ pub use mock::{MockHost, MockProvisioner};
mod linux;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use linux::NetlinkProvisioner;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub(crate) use linux::RouteHandle;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
mod windows;
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use windows::WintunProvisioner;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
mod macos;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use macos::UtunProvisioner;
mod privilege;
pub use privilege::{Privilege, probe_net_admin};
@@ -15,7 +15,7 @@
//! Grant it with:
//!
//! ```text
//! sudo setcap cap_net_admin+p /usr/local/bin/tsunagi
//! sudo setcap cap_net_admin,cap_net_bind_service+p /usr/local/bin/tsng
//! ```
//!
//! `+p` rather than `+ep`: with `+p` the capability is permitted but not
@@ -49,7 +49,7 @@ impl Privilege {
/// How to obtain it, for a diagnostic.
pub fn how_to_grant(program: &str) -> String {
format!(
"Grant it once with `sudo setcap cap_net_admin+p {program}` \
"Grant it once with `sudo setcap cap_net_admin,cap_net_bind_service+p {program}` \
and the agent manages its own interface. Without it, run with \
`--no-tun`: the tunnels still form, they just do not reach the \
operating system."
@@ -63,15 +63,20 @@ pub use linux_impl::{NetAdmin, probe_net_admin};
#[cfg(all(feature = "tun-device", target_os = "windows"))]
pub use windows_impl::probe_net_admin;
#[cfg(all(feature = "tun-device", target_os = "macos"))]
pub use macos_impl::probe_net_admin;
#[cfg(not(any(
all(feature = "tun-device", target_os = "linux"),
all(feature = "tun-device", target_os = "windows")
all(feature = "tun-device", target_os = "windows"),
all(feature = "tun-device", target_os = "macos")
)))]
pub use other_impl::probe_net_admin;
#[cfg(not(any(
all(feature = "tun-device", target_os = "linux"),
all(feature = "tun-device", target_os = "windows")
all(feature = "tun-device", target_os = "windows"),
all(feature = "tun-device", target_os = "macos")
)))]
mod other_impl {
use super::Privilege;
@@ -101,6 +106,25 @@ mod windows_impl {
}
}
#[cfg(all(feature = "tun-device", target_os = "macos"))]
mod macos_impl {
use super::Privilege;
/// Whether this process can configure interfaces.
///
/// macOS has no capability to hold and lower the way Linux does: creating
/// a utun interface simply needs the process to be root. Reading the
/// effective uid means a libc call this crate forbids, so the probe is
/// deliberately optimistic — it reports that the platform can manage
/// interfaces — and the real check is left to the utun open, which fails
/// with a precise "run with sudo" message when the process is not root.
/// This matches how the Linux path treats the open itself as the honest
/// answer.
pub fn probe_net_admin() -> Privilege {
Privilege::Available
}
}
#[cfg(all(feature = "tun-device", target_os = "linux"))]
mod linux_impl {
use caps::{CapSet, Capability};
@@ -147,7 +171,7 @@ mod linux_impl {
caps::raise(None, CapSet::Effective, Capability::CAP_NET_ADMIN).map_err(|err| {
OverlayError::Unavailable(format!(
"cannot raise CAP_NET_ADMIN: {err}. {}",
Privilege::how_to_grant("tsunagi")
Privilege::how_to_grant("tsng")
))
})?;
Ok(Self { raised: true })
@@ -180,8 +204,8 @@ mod tests {
#[test]
fn the_grant_instructions_name_the_program() {
let text = Privilege::how_to_grant("/usr/local/bin/tsunagi");
assert!(text.contains("setcap cap_net_admin+p /usr/local/bin/tsunagi"));
let text = Privilege::how_to_grant("/usr/local/bin/tsng");
assert!(text.contains("setcap cap_net_admin,cap_net_bind_service+p /usr/local/bin/tsng"));
assert!(text.contains("--no-tun"), "the fallback is offered too");
}
@@ -262,7 +262,12 @@ async fn configure(plan: &InterfacePlan, changes: &super::Changes) -> Result<(),
set_mtu(&plan.name, mtu).await?;
}
for cidr in &changes.add {
address_add(&plan.name, *cidr).await?;
if let Err(err) = address_add(&plan.name, *cidr).await {
let err_str = err.to_string();
if !err_str.contains("already exists") && !err_str.contains("0x80071392") {
return Err(err);
}
}
}
if changes.bring_up {
// Best effort: a Wintun adapter with a session is up already, and a
+925
View File
@@ -0,0 +1,925 @@
//! Which peer a packet is for, and whether a peer may have sent it.
//!
//! One agent has one interface, so a packet coming off it could belong to any
//! network the agent is in and to any protocol currently carrying traffic.
//! Deciding which is this table's job, and it is the reason the interface
//! belongs to the system level: the answer comes from signed state, which no
//! protocol owns.
//!
//! Two questions, and they are not the same one:
//!
//! * **Outbound**: this destination address — whose is it? A packet for
//! nobody is dropped rather than broadcast.
//! * **Inbound**: this peer decrypted a packet with this source address —
//! is that address actually its? A peer may not speak for anybody else,
//! and the check consults the signed claim rather than anything the peer
//! said.
//!
//! Nothing here is derived from a protocol's key. An address is allocated at
//! the system level and signed by the member that holds it, so every protocol
//! carries traffic for the same addresses and the table is the same whichever
//! one is in use.
use super::broadcast::{BroadcastPolicy, BroadcastTable};
use arc_swap::ArcSwap;
use std::collections::HashMap;
use std::net::Ipv4Addr;
use std::sync::{Arc, RwLock};
use iroh::EndpointId;
use crate::identity::NetworkId;
use crate::state::Ipv4Range;
/// Where an outbound packet is going.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Route {
/// The network the destination belongs to.
pub network: NetworkId,
/// The member that holds the destination address.
pub peer: EndpointId,
}
/// A network's exit-node settings, as far as packets are concerned.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct ExitPolicy {
/// This agent offers itself as an exit node here: packets from members
/// for addresses outside every overlay range are handed to the host.
pub offer: bool,
/// The member all other traffic is sent to, if this device uses one.
pub via: Option<EndpointId>,
}
/// What one network contributes to the table.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct NetworkRoutes {
/// Exit-node settings of this network.
pub exit: ExitPolicy,
/// Local broadcast policy and authenticated willing recipients.
pub broadcast: BroadcastPolicy,
/// The range this network allocates from, once it has agreed one.
pub range: Option<Ipv4Range>,
/// This agent's own address in the network.
pub local: Option<Ipv4Addr>,
/// Every other member's address, as the signed state has it.
pub peers: Vec<(Ipv4Addr, EndpointId)>,
}
/// Why a network cannot be added to the table.
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum RouteError {
/// Two networks would answer for the same addresses.
#[error(
"network {network} uses {range}, which overlaps {other_range} already in use by \
{other}. One agent has one interface, so an address can belong to only one of \
them: give one network a different range."
)]
Overlap {
/// The network being added.
network: NetworkId,
/// The range it wants.
range: Ipv4Range,
/// The network already using an overlapping range.
other: NetworkId,
/// That network's range.
other_range: Ipv4Range,
},
}
/// Address ownership across every network this agent is in.
#[derive(Debug, Default)]
pub struct RoutingTable {
broadcast: ArcSwap<BroadcastTable>,
networks: RwLock<HashMap<NetworkId, NetworkRoutes>>,
}
impl RoutingTable {
/// An empty table.
pub fn new() -> Self {
Self::default()
}
fn read(&self) -> std::sync::RwLockReadGuard<'_, HashMap<NetworkId, NetworkRoutes>> {
match self.networks.read() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
fn write(&self) -> std::sync::RwLockWriteGuard<'_, HashMap<NetworkId, NetworkRoutes>> {
match self.networks.write() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
/// Replaces what one network contributes.
///
/// Rejected when its range overlaps another network's. With one
/// interface an address can belong to only one network, and guessing
/// which would send somebody's traffic to a stranger.
pub fn set_network(&self, network: NetworkId, routes: NetworkRoutes) -> Result<(), RouteError> {
let mut networks = self.write();
if let Some(range) = routes.range {
for (other, existing) in networks.iter() {
if *other == network {
continue;
}
let Some(other_range) = existing.range else {
continue;
};
if ranges_overlap(range, other_range) {
return Err(RouteError::Overlap {
network,
range,
other: *other,
other_range,
});
}
}
}
networks.insert(network, routes);
self.broadcast
.store(Arc::new(BroadcastTable::build(&networks)));
Ok(())
}
/// Forgets a network.
pub fn remove_network(&self, network: NetworkId) {
let mut networks = self.write();
networks.remove(&network);
self.broadcast
.store(Arc::new(BroadcastTable::build(&networks)));
}
/// Replaces only a network's broadcast participation, without changing addresses.
pub fn set_broadcast(&self, network: NetworkId, policy: BroadcastPolicy) {
let mut networks = self.write();
if let Some(routes) = networks.get_mut(&network) {
if routes.broadcast == policy {
return;
}
routes.broadcast = policy;
self.broadcast
.store(Arc::new(BroadcastTable::build(&networks)));
}
}
/// Replaces only a network's exit-node settings, without changing addresses.
///
/// A network that has no entry yet (its range is not agreed) takes them
/// from [`NetworkRoutes::exit`] when it is added.
pub fn set_exit(&self, network: NetworkId, policy: ExitPolicy) {
if let Some(routes) = self.write().get_mut(&network) {
routes.exit = policy;
}
}
/// Recognizes limited and configured subnet broadcasts before unicast lookup.
pub fn is_broadcast(&self, destination: Ipv4Addr) -> bool {
self.broadcast.load().is_destination(destination)
}
/// A precomputed domain-scoped recipient list; no per-packet graph search.
pub fn broadcast_recipients(
&self,
source: Ipv4Addr,
destination: Ipv4Addr,
) -> Option<Arc<[Route]>> {
self.broadcast.load().outgoing(source, destination)
}
/// Local admission after the plugin authenticated and decrypted the sender.
pub fn accepts_broadcast(
&self,
network: NetworkId,
peer: EndpointId,
destination: Ipv4Addr,
) -> bool {
self.broadcast.load().accepts(network, peer, destination)
}
/// The peer that holds a destination address, if anybody does.
///
/// This agent's own address returns `None`: a packet for ourselves does
/// not go over a tunnel, and answering with a peer would send it to one.
///
/// An address nobody holds, outside every overlay range and an ordinary
/// unicast destination, goes to this device's exit node when it has one:
/// that is what sending *all* traffic through a member means.
pub fn route(&self, destination: Ipv4Addr) -> Option<Route> {
let networks = self.read();
for (network, routes) in networks.iter() {
if routes.local == Some(destination) {
return None;
}
if let Some((_, peer)) = routes
.peers
.iter()
.find(|(address, _)| *address == destination)
{
return Some(Route {
network: *network,
peer: *peer,
});
}
}
// Inside an overlay range, an unowned address is nobody's, not the
// internet's.
if !is_internet_destination(destination) || in_any_range(&networks, destination) {
return None;
}
exit_via(&networks).map(|(network, peer)| Route { network, peer })
}
/// Whether a peer may send from a source address.
///
/// A peer speaks only for the address the network agreed it holds. The
/// one exception is this device's exit node: what comes back from the
/// internet carries the internet's addresses, so from *that* peer any
/// source outside the overlay ranges is accepted.
pub fn may_send_from(&self, network: NetworkId, peer: EndpointId, source: Ipv4Addr) -> bool {
let networks = self.read();
let Some(routes) = networks.get(&network) else {
return false;
};
if routes
.peers
.iter()
.any(|(address, holder)| *address == source && *holder == peer)
{
return true;
}
routes.exit.via == Some(peer)
&& is_internet_destination(source)
&& !in_any_range(&networks, source)
}
/// Whether this agent, as an exit node of `network`, takes a member's
/// packet for `destination` to the host.
///
/// Only an ordinary unicast address outside every overlay range, so a
/// packet for another member, for this agent or for the local network
/// segment never leaves by this door. The sender's own source address is
/// checked separately, by [`Self::may_send_from`].
pub fn accepts_exit_traffic(&self, network: NetworkId, destination: Ipv4Addr) -> bool {
let networks = self.read();
networks
.get(&network)
.is_some_and(|routes| routes.exit.offer)
&& is_internet_destination(destination)
&& !in_any_range(&networks, destination)
}
/// The network and member this device sends its internet traffic through.
///
/// One default route, so one answer; if several networks claim one (they
/// should not) the lowest network id decides, so two runs agree.
pub fn exit_via(&self) -> Option<(NetworkId, EndpointId)> {
exit_via(&self.read())
}
/// Networks that offer this agent as an exit node, with the range each
/// agreed. A network that has not agreed one yet has nothing to
/// masquerade and is left out.
pub fn exit_offers(&self) -> Vec<(NetworkId, Ipv4Range)> {
let mut offers: Vec<(NetworkId, Ipv4Range)> = self
.read()
.iter()
.filter(|(_, routes)| routes.exit.offer)
.filter_map(|(network, routes)| Some((*network, routes.range?)))
.collect();
offers.sort_by_key(|(network, _)| *network);
offers
}
/// One network's exit-node settings and its range, if it is in the table.
pub fn exit_settings(&self, network: NetworkId) -> Option<(ExitPolicy, Option<Ipv4Range>)> {
self.read()
.get(&network)
.map(|routes| (routes.exit, routes.range))
}
/// Whether an ordinary packet terminates on this host in this network.
/// Exported LAN subnets must extend this admission policy explicitly later.
pub fn is_local_destination(&self, network: NetworkId, destination: Ipv4Addr) -> bool {
self.read()
.get(&network)
.is_some_and(|routes| routes.local == Some(destination))
}
/// Every address this agent should answer to, with its prefix length.
pub fn local_addresses(&self) -> Vec<(Ipv4Addr, u8)> {
let mut addresses: Vec<(Ipv4Addr, u8)> = self
.read()
.values()
.filter_map(|routes| Some((routes.local?, routes.range?.prefix_len)))
.collect();
addresses.sort();
addresses
}
/// The one network whose address the limited broadcast route is bound to.
///
/// The host has a single `255.255.255.255` route per interface, and one
/// interface carries every network, so only one overlay address can be its
/// source. The choice is the lowest [`NetworkId`] among networks that have
/// broadcast enabled, a range and a local address: deterministic, so two
/// runs with the same networks pick the same one.
pub fn broadcast_host_target(&self) -> Option<(Ipv4Addr, Ipv4Range)> {
let networks = self.read();
networks
.iter()
.filter(|(_, routes)| routes.broadcast.enabled)
.filter_map(|(network, routes)| Some((*network, routes.local?, routes.range?)))
.min_by_key(|(network, _, _)| *network)
.map(|(_, local, range)| (local, range))
}
/// Whether a range would collide with one another network already uses.
///
/// Asked before proposing a range rather than after: with one interface
/// an agent that claimed an address it could not route would also be
/// telling everybody else to use that range, and "the range of the
/// lowest author wins" would spread the collision instead of containing
/// it.
pub fn would_overlap(&self, network: NetworkId, range: Ipv4Range) -> Option<Ipv4Range> {
self.read().iter().find_map(|(other, existing)| {
if *other == network {
return None;
}
let other_range = existing.range?;
ranges_overlap(range, other_range).then_some(other_range)
})
}
/// How many networks the table covers.
pub fn len(&self) -> usize {
self.read().len()
}
/// Whether it covers none.
pub fn is_empty(&self) -> bool {
self.read().is_empty()
}
}
/// Whether an address is an ordinary unicast one: not this host, not a
/// multicast or broadcast group, not link-local, not unspecified.
fn is_internet_destination(address: Ipv4Addr) -> bool {
!(address.is_unspecified()
|| address.is_loopback()
|| address.is_multicast()
|| address.is_broadcast()
|| address.is_link_local())
}
/// Whether an address lies in any network's range.
fn in_any_range(networks: &HashMap<NetworkId, NetworkRoutes>, address: Ipv4Addr) -> bool {
networks
.values()
.any(|routes| routes.range.is_some_and(|range| range.contains(address)))
}
/// The exit node in use, if any, from the lowest network that names one.
fn exit_via(networks: &HashMap<NetworkId, NetworkRoutes>) -> Option<(NetworkId, EndpointId)> {
networks
.iter()
.filter_map(|(network, routes)| Some((*network, routes.exit.via?)))
.min_by_key(|(network, _)| *network)
}
/// Whether two ranges share any address.
fn ranges_overlap(one: Ipv4Range, other: Ipv4Range) -> bool {
// A range contains the other's base, or the other way round. With
// prefix-aligned ranges that is the whole of it: two blocks either nest
// or are disjoint.
one.contains(other.base) || other.contains(one.base)
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
fn network(name: &str) -> NetworkId {
NetworkKeys::derive(
&NetworkName::new(name).unwrap(),
&NetworkSecret::from_bytes(vec![4u8; 32]).unwrap(),
)
.network_id()
}
fn peer(seed: u8) -> EndpointId {
iroh::SecretKey::from_bytes(&[seed; 32]).public()
}
fn addr(last: u8) -> Ipv4Addr {
Ipv4Addr::new(10, 13, 37, last)
}
fn routes() -> NetworkRoutes {
NetworkRoutes {
exit: Default::default(),
broadcast: Default::default(),
range: Some("10.13.37.0/24".parse().unwrap()),
local: Some(addr(1)),
peers: vec![(addr(2), peer(2)), (addr(3), peer(3))],
}
}
#[test]
fn a_packet_goes_to_the_member_that_holds_its_destination() {
let table = RoutingTable::new();
let id = network("routing");
table.set_network(id, routes()).unwrap();
assert_eq!(
table.route(addr(2)),
Some(Route {
network: id,
peer: peer(2)
})
);
assert_eq!(
table.route(addr(3)),
Some(Route {
network: id,
peer: peer(3)
})
);
}
#[test]
fn a_packet_for_nobody_has_nowhere_to_go() {
// Dropped rather than sent to everybody: a tunnel is not a broadcast
// domain, and flooding would hand one member another's traffic.
let table = RoutingTable::new();
table.set_network(network("routing"), routes()).unwrap();
assert_eq!(table.route(addr(200)), None);
assert_eq!(table.route(Ipv4Addr::new(192, 0, 2, 1)), None);
}
#[test]
fn a_packet_for_ourselves_is_not_sent_to_a_peer() {
let table = RoutingTable::new();
table.set_network(network("routing"), routes()).unwrap();
assert_eq!(table.route(addr(1)), None, "that address is ours");
}
#[test]
fn a_peer_may_only_send_from_the_address_it_holds() {
let table = RoutingTable::new();
let id = network("ownership");
table.set_network(id, routes()).unwrap();
assert!(table.may_send_from(id, peer(2), addr(2)));
// Somebody else's address, and its own network's.
assert!(!table.may_send_from(id, peer(2), addr(3)));
assert!(!table.may_send_from(id, peer(2), addr(1)));
assert!(!table.may_send_from(id, peer(2), addr(99)));
// A peer that holds nothing here.
assert!(!table.may_send_from(id, peer(9), addr(2)));
// And the question is per network, not global.
assert!(!table.may_send_from(network("elsewhere"), peer(2), addr(2)));
}
#[test]
fn two_networks_with_overlapping_ranges_are_refused() {
// One interface means an address belongs to one network. Accepting
// both and guessing would send somebody's traffic to a stranger.
let table = RoutingTable::new();
let first = network("first");
let second = network("second");
table.set_network(first, routes()).unwrap();
let err = table.set_network(second, routes()).unwrap_err();
let RouteError::Overlap { other, .. } = err;
assert_eq!(other, first);
assert!(err.to_string().contains("different range"), "{err}");
assert_eq!(table.len(), 1, "the overlapping network is not added");
// A range of its own is fine.
table
.set_network(
second,
NetworkRoutes {
exit: Default::default(),
broadcast: Default::default(),
range: Some("10.99.0.0/16".parse().unwrap()),
local: Some(Ipv4Addr::new(10, 99, 0, 1)),
peers: vec![(Ipv4Addr::new(10, 99, 0, 2), peer(4))],
},
)
.unwrap();
assert_eq!(table.len(), 2);
assert_eq!(
table.route(Ipv4Addr::new(10, 99, 0, 2)),
Some(Route {
network: second,
peer: peer(4)
})
);
// And the first network still answers for its own.
assert_eq!(table.route(addr(2)).map(|route| route.network), Some(first));
}
#[test]
fn the_host_broadcast_target_is_the_lowest_enabled_network_with_an_address() {
let table = RoutingTable::new();
assert_eq!(table.broadcast_host_target(), None);
let (low, high) = {
let (a, b) = (network("first"), network("second"));
if a < b { (a, b) } else { (b, a) }
};
let mk = |enabled: bool, base: [u8; 4], local: Option<Ipv4Addr>| NetworkRoutes {
exit: Default::default(),
broadcast: BroadcastPolicy {
enabled,
..Default::default()
},
range: Some(
format!("{}.{}.{}.0/24", base[0], base[1], base[2])
.parse()
.unwrap(),
),
local,
peers: Vec::new(),
};
let a = Ipv4Addr::new(10, 1, 0, 1);
let b = Ipv4Addr::new(10, 2, 0, 1);
// The lower id has broadcast off, so the higher one is chosen.
table
.set_network(low, mk(false, [10, 1, 0, 0], Some(a)))
.unwrap();
table
.set_network(high, mk(true, [10, 2, 0, 0], Some(b)))
.unwrap();
assert_eq!(
table.broadcast_host_target(),
Some((b, "10.2.0.0/24".parse().unwrap()))
);
// Both enabled: the lowest id wins, whatever the insertion order.
table
.set_network(low, mk(true, [10, 1, 0, 0], Some(a)))
.unwrap();
assert_eq!(
table.broadcast_host_target(),
Some((a, "10.1.0.0/24".parse().unwrap()))
);
// No local address yet: it cannot be the route's source.
table
.set_network(low, mk(true, [10, 1, 0, 0], None))
.unwrap();
assert_eq!(
table.broadcast_host_target(),
Some((b, "10.2.0.0/24".parse().unwrap()))
);
table.remove_network(high);
assert_eq!(table.broadcast_host_target(), None);
}
#[test]
fn a_range_can_be_asked_about_before_it_is_proposed() {
// The point of asking first: an agent that claims an address it
// cannot route also tells everybody else to use that range.
let table = RoutingTable::new();
let first = network("first");
let second = network("second");
table.set_network(first, routes()).unwrap();
let mine: Ipv4Range = "10.13.37.0/24".parse().unwrap();
assert_eq!(table.would_overlap(second, mine), Some(mine));
// Its own range is not a collision with itself.
assert_eq!(table.would_overlap(first, mine), None);
assert_eq!(
table.would_overlap(second, "10.99.0.0/16".parse().unwrap()),
None
);
}
#[test]
fn a_nested_range_counts_as_overlapping() {
let table = RoutingTable::new();
table
.set_network(
network("wide"),
NetworkRoutes {
exit: Default::default(),
broadcast: Default::default(),
range: Some("10.0.0.0/8".parse().unwrap()),
..Default::default()
},
)
.unwrap();
// Inside the /8, in either direction.
assert!(
table
.set_network(
network("narrow"),
NetworkRoutes {
exit: Default::default(),
broadcast: Default::default(),
range: Some("10.13.37.0/24".parse().unwrap()),
..Default::default()
}
)
.is_err()
);
}
#[test]
fn replacing_a_network_does_not_count_as_overlapping_itself() {
let table = RoutingTable::new();
let id = network("refresh");
table.set_network(id, routes()).unwrap();
let mut updated = routes();
updated.peers.push((addr(4), peer(4)));
table.set_network(id, updated).unwrap();
assert_eq!(table.len(), 1);
assert!(table.route(addr(4)).is_some());
}
#[test]
fn forgetting_a_network_takes_its_addresses_with_it() {
let table = RoutingTable::new();
let id = network("leaving");
table.set_network(id, routes()).unwrap();
table.remove_network(id);
assert!(table.is_empty());
assert_eq!(table.route(addr(2)), None);
assert!(!table.may_send_from(id, peer(2), addr(2)));
}
#[test]
fn the_local_addresses_are_what_the_interface_should_carry() {
let table = RoutingTable::new();
table.set_network(network("one"), routes()).unwrap();
table
.set_network(
network("two"),
NetworkRoutes {
exit: Default::default(),
broadcast: Default::default(),
range: Some("10.99.0.0/16".parse().unwrap()),
local: Some(Ipv4Addr::new(10, 99, 0, 1)),
peers: Vec::new(),
},
)
.unwrap();
assert_eq!(
table.local_addresses(),
vec![(addr(1), 24), (Ipv4Addr::new(10, 99, 0, 1), 16)]
);
}
#[test]
fn a_network_with_no_address_yet_contributes_nothing() {
let table = RoutingTable::new();
let id = network("waiting");
table.set_network(id, NetworkRoutes::default()).unwrap();
assert!(table.local_addresses().is_empty());
assert_eq!(table.route(addr(2)), None);
assert_eq!(table.len(), 1, "the network is known, it just has nothing");
}
fn with_exit(offer: bool, via: Option<EndpointId>) -> NetworkRoutes {
NetworkRoutes {
exit: ExitPolicy { offer, via },
..routes()
}
}
const INTERNET: Ipv4Addr = Ipv4Addr::new(93, 184, 216, 34);
#[test]
fn without_an_exit_node_the_internet_has_nowhere_to_go() {
let table = RoutingTable::new();
table.set_network(network("plain"), routes()).unwrap();
assert_eq!(table.route(INTERNET), None);
assert_eq!(table.exit_via(), None);
}
#[test]
fn with_an_exit_node_every_ordinary_destination_goes_to_it() {
let table = RoutingTable::new();
let id = network("client");
table
.set_network(id, with_exit(false, Some(peer(2))))
.unwrap();
let exit = Route {
network: id,
peer: peer(2),
};
assert_eq!(table.route(INTERNET), Some(exit));
assert_eq!(table.route(Ipv4Addr::new(192, 168, 1, 10)), Some(exit));
assert_eq!(table.exit_via(), Some((id, peer(2))));
// A member's own address still goes to that member.
assert_eq!(
table.route(addr(3)),
Some(Route {
network: id,
peer: peer(3)
})
);
}
#[test]
fn the_exit_node_is_never_the_answer_for_what_is_not_the_internet() {
let table = RoutingTable::new();
table
.set_network(network("client"), with_exit(false, Some(peer(2))))
.unwrap();
// Ourselves, a free address inside the overlay range, and everything
// that is not an ordinary unicast destination.
for destination in [
addr(1),
addr(200),
Ipv4Addr::new(127, 0, 0, 1),
Ipv4Addr::new(224, 0, 0, 251),
Ipv4Addr::new(239, 255, 255, 250),
Ipv4Addr::BROADCAST,
Ipv4Addr::UNSPECIFIED,
Ipv4Addr::new(169, 254, 1, 1),
] {
assert_eq!(table.route(destination), None, "{destination}");
}
}
#[test]
fn another_networks_range_is_not_the_internet_either() {
let table = RoutingTable::new();
table
.set_network(network("client"), with_exit(false, Some(peer(2))))
.unwrap();
table
.set_network(
network("other"),
NetworkRoutes {
range: Some("10.99.0.0/16".parse().unwrap()),
local: Some(Ipv4Addr::new(10, 99, 0, 1)),
..Default::default()
},
)
.unwrap();
assert_eq!(table.route(Ipv4Addr::new(10, 99, 7, 7)), None);
}
#[test]
fn two_exit_choices_resolve_to_the_lowest_network_id() {
let table = RoutingTable::new();
let (a, b) = (network("one"), network("two"));
table
.set_network(a, with_exit(false, Some(peer(2))))
.unwrap();
table
.set_network(
b,
NetworkRoutes {
exit: ExitPolicy {
offer: false,
via: Some(peer(9)),
},
range: Some("10.99.0.0/16".parse().unwrap()),
..Default::default()
},
)
.unwrap();
let (chosen, _) = table.exit_via().unwrap();
assert_eq!(chosen, a.min(b));
assert_eq!(table.route(INTERNET).unwrap().network, a.min(b));
}
#[test]
fn the_exit_peer_may_send_from_any_internet_address_but_nobody_else_may() {
let table = RoutingTable::new();
let id = network("client");
table
.set_network(id, with_exit(false, Some(peer(2))))
.unwrap();
// What comes back from the internet carries the internet's addresses.
assert!(table.may_send_from(id, peer(2), INTERNET));
// Another member still speaks only for its own address.
assert!(!table.may_send_from(id, peer(3), INTERNET));
assert!(table.may_send_from(id, peer(3), addr(3)));
// The exit peer cannot pass itself off as somebody else in the
// overlay, nor as this agent, nor as something that is not a source.
assert!(!table.may_send_from(id, peer(2), addr(3)));
assert!(!table.may_send_from(id, peer(2), addr(1)));
assert!(!table.may_send_from(id, peer(2), addr(200)));
for source in [
Ipv4Addr::LOCALHOST,
Ipv4Addr::UNSPECIFIED,
Ipv4Addr::new(224, 0, 0, 1),
Ipv4Addr::BROADCAST,
] {
assert!(!table.may_send_from(id, peer(2), source), "{source}");
}
// Without choosing an exit node nobody gets the exception.
let plain = RoutingTable::new();
plain.set_network(id, routes()).unwrap();
assert!(!plain.may_send_from(id, peer(2), INTERNET));
}
#[test]
fn an_exit_node_takes_a_members_internet_packets_only_while_it_offers() {
let table = RoutingTable::new();
let id = network("server");
table.set_network(id, with_exit(false, None)).unwrap();
assert!(!table.accepts_exit_traffic(id, INTERNET));
table.set_exit(
id,
ExitPolicy {
offer: true,
via: None,
},
);
assert!(table.accepts_exit_traffic(id, INTERNET));
assert!(table.accepts_exit_traffic(id, Ipv4Addr::new(192, 168, 1, 10)));
// Never for another member, for this agent, a free overlay address,
// or anything that is not an ordinary unicast destination.
for destination in [
addr(1),
addr(2),
addr(200),
Ipv4Addr::LOCALHOST,
Ipv4Addr::new(224, 0, 0, 251),
Ipv4Addr::BROADCAST,
Ipv4Addr::UNSPECIFIED,
] {
assert!(
!table.accepts_exit_traffic(id, destination),
"{destination}"
);
}
// An unknown network offers nothing.
assert!(!table.accepts_exit_traffic(network("elsewhere"), INTERNET));
table.set_exit(id, ExitPolicy::default());
assert!(!table.accepts_exit_traffic(id, INTERNET));
}
#[test]
fn offers_are_listed_by_network_with_the_range_each_agreed() {
let table = RoutingTable::new();
let (a, b) = (network("a"), network("b"));
table.set_network(a, with_exit(true, None)).unwrap();
table
.set_network(
b,
NetworkRoutes {
exit: ExitPolicy {
offer: true,
via: None,
},
// No range agreed yet: nothing to masquerade.
..Default::default()
},
)
.unwrap();
assert_eq!(
table.exit_offers(),
vec![(a, "10.13.37.0/24".parse().unwrap())]
);
assert_eq!(
table.exit_settings(a),
Some((
ExitPolicy {
offer: true,
via: None
},
Some("10.13.37.0/24".parse().unwrap())
))
);
assert_eq!(table.exit_settings(network("none")), None);
}
#[test]
fn setting_exit_for_a_network_with_no_entry_is_remembered_by_the_caller() {
// The runtime keeps the policy and passes it with the network when
// its range is agreed; a setter for a network the table does not know
// yet changes nothing.
let table = RoutingTable::new();
table.set_exit(
network("later"),
ExitPolicy {
offer: true,
via: None,
},
);
assert!(table.is_empty());
}
}
@@ -109,6 +109,14 @@ pub trait TunFactory: Send + Sync + std::fmt::Debug + 'static {
Box::pin(async move { Ok(()) })
}
/// The host rules for this factory's interface, when it manages the host.
///
/// `None` for an in-memory device or an interface somebody else prepared:
/// there is nothing of ours to route or to open in a firewall.
fn host_rules(&self) -> Option<Arc<dyn super::hostrules::BroadcastHostRules>> {
None
}
/// Removes an interface this factory created.
///
/// Runs on the teardown path, so it reports rather than fails: there is
@@ -221,7 +229,31 @@ impl TunDevice for MemoryTun {
/// enough here: the agent chose the address, so anything else holding it is a
/// problem in its own right.
pub fn address_is_local(address: std::net::IpAddr) -> bool {
std::net::UdpSocket::bind((address, 0)).is_ok()
if std::net::UdpSocket::bind((address, 0)).is_ok() {
return true;
}
#[cfg(all(feature = "tun-device", target_os = "windows"))]
{
if let Ok(interfaces) = std::panic::catch_unwind(netdev::get_interfaces) {
for iface in interfaces {
if iface.is_up() {
match address {
std::net::IpAddr::V4(v4) => {
if iface.ipv4.iter().any(|ip| ip.addr() == v4) {
return true;
}
}
std::net::IpAddr::V6(v6) => {
if iface.ipv6.iter().any(|ip| ip.addr() == v6) {
return true;
}
}
}
}
}
}
}
false
}
/// Creates [`MemoryTun`] devices.
@@ -385,7 +417,15 @@ mod system {
/// across this call, and such a guard must not span an `await` because
/// Linux capabilities are per thread.
pub(crate) fn open_tun(request: &TunRequest) -> Result<Arc<dyn TunDevice>, OverlayError> {
// macOS mutates neither field below (it neither names the interface nor
// sets a platform config), so `mut` is unused only there.
#[cfg_attr(target_os = "macos", allow(unused_mut))]
let mut config = tun::Configuration::default();
// macOS names its tunnels `utunN` and assigns the unit number itself.
// A name derived from the network id (`tsun…`) is not a legal utun
// name, so there the kernel is left to choose and the real name is read
// back below. Linux and Windows take the requested name verbatim.
#[cfg(not(target_os = "macos"))]
config.tun_name(&request.name);
#[cfg(target_os = "linux")]
config.platform_config(|platform| {
@@ -403,9 +443,20 @@ mod system {
let device = tun::create_as_async(&config).map_err(|err| open_error(&request.name, err))?;
// The interface name the operating system actually gave us: the
// requested one everywhere except macOS, where the kernel chose the
// utun unit and so the real name has to be read back off the device.
#[cfg(target_os = "macos")]
let name = {
use tun::AbstractDevice;
device.tun_name().unwrap_or_else(|_| request.name.clone())
};
#[cfg(not(target_os = "macos"))]
let name = request.name.clone();
let (reader, writer) = tokio::io::split(device);
Ok(Arc::new(SystemTun {
name: request.name.clone(),
name,
mtu: request.mtu,
reader: Mutex::new(reader),
writer: Mutex::new(writer),
@@ -432,12 +483,15 @@ mod system {
if denied {
let hint = if cfg!(target_os = "windows") {
"Open PowerShell or Command Prompt with 'Run as administrator' and \
start `tsunagi up` there."
start `tsng up` there."
} else if cfg!(target_os = "linux") {
"Start `tsunagi up` as root or grant CAP_NET_ADMIN with \
`sudo setcap cap_net_admin+p /path/to/tsunagi`."
"Start `tsng up` as root or grant CAP_NET_ADMIN with \
`sudo setcap cap_net_admin,cap_net_bind_service+p /path/to/tsunagi`."
} else if cfg!(target_os = "macos") {
"Creating a utun interface needs root on macOS; start `tsng up` \
with `sudo`, or install it as a root LaunchDaemon."
} else {
"Start `tsunagi up` with the privileges required to create a TUN interface."
"Start `tsng up` with the privileges required to create a TUN interface."
};
return OverlayError::Unavailable(format!(
"permission denied while creating the TUN interface `{name}`. \
@@ -455,7 +509,7 @@ mod system {
fn open_hint() -> &'static str {
#[cfg(target_os = "linux")]
{
"Creating one needs CAP_NET_ADMIN; grant it with `setcap cap_net_admin+p`, \
"Creating one needs CAP_NET_ADMIN; grant it with `setcap cap_net_admin,cap_net_bind_service+p`, \
or run with `--no-tun` to keep the tunnels off the operating system."
}
#[cfg(target_os = "windows")]
@@ -464,7 +518,12 @@ mod system {
run as Administrator with the DLL beside the executable, or run with `--no-tun` \
to keep the tunnels off the operating system."
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
#[cfg(target_os = "macos")]
{
"Creating a utun interface needs root; start the agent with `sudo` or as a root \
LaunchDaemon, or run with `--no-tun` to keep the tunnels off the operating system."
}
#[cfg(not(any(target_os = "linux", target_os = "windows", target_os = "macos")))]
{
"Run with `--no-tun` to keep the tunnels off the operating system."
}
@@ -480,7 +539,7 @@ mod system {
let message = open_error("tsun0", error).to_string();
assert!(message.contains("permission denied"), "{message}");
assert!(message.contains("tsun0"), "{message}");
assert!(message.contains("tsunagi up"), "{message}");
assert!(message.contains("tsng up"), "{message}");
#[cfg(target_os = "windows")]
assert!(message.contains("Run as administrator"), "{message}");
#[cfg(target_os = "linux")]
File renamed without changes.
@@ -20,7 +20,7 @@ use crate::error::ProtocolError;
/// The version in the ALPN is the wire-compatibility version of the control
/// protocol. It is independent of the network identity scheme version, so
/// bumping it must not change any existing [`crate::NetworkId`].
pub const ALPN: &[u8] = b"tsunagi/ctrl/1";
pub const ALPN: &[u8] = b"tsunagi/ctrl/4";
/// ALPN of the tsunagi data plane.
///
@@ -29,11 +29,10 @@ pub const ALPN: &[u8] = b"tsunagi/ctrl/1";
/// saturated or broken data plane cannot disturb control traffic, and the
/// transport underneath can be replaced without touching the control protocol.
///
/// Version 3 fragments logical datagrams below the peer-relay envelope, so
/// the overlay's MTU is independent of the current QUIC path MTU. Older data
/// versions cannot form a data link; control and persistent identities remain
/// compatible. Both ends, including any intermediate peer, must be upgraded.
pub const DATA_ALPN: &[u8] = b"tsunagi/data/3";
/// Version 4 adds source, destination, hop limit and stable flow id above
/// transport fragmentation. All members must upgrade together. Persistent
/// identities and network configuration do not change.
pub const DATA_ALPN: &[u8] = b"tsunagi/data/4";
/// Largest plugin protocol identifier accepted when opening a data channel.
pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
@@ -42,7 +41,7 @@ pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
pub const MAX_SIGNATURE_LEN: usize = 64;
/// Control protocol version carried inside the handshake.
pub const PROTOCOL_VERSION: u16 = 1;
pub const PROTOCOL_VERSION: u16 = 4;
/// First message of the handshake, sent by the initiator.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -74,6 +73,13 @@ pub struct AuthProof {
/// What this agent tells a peer about itself.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Announcement {
/// Local opt-in to receive LAN broadcasts in this network.
pub broadcast: bool,
/// This agent offers itself as an exit node in this network: members may
/// send all their internet traffic through it. Off unless its owner
/// turned it on, and only said once the rules that make it work are in
/// place.
pub exit_node: bool,
/// Human-readable hostname. A mutable binding, not an identity.
pub hostname: String,
/// Announced IP plugin capabilities. Opaque to the core.
@@ -154,8 +160,8 @@ pub enum ControlMessage {
/// and repairs itself; and volatile, so it lives here and not in
/// signed state, which is for what has to survive a member being away.
Reachable {
/// The peers, as raw endpoint keys.
peers: Vec<[u8; 32]>,
/// Direct links, scoped to the plugin protocol they actually carry.
links: Vec<ReachableLink>,
},
/// Graceful goodbye.
///
@@ -183,6 +189,15 @@ pub struct PeerHint {
pub addrs: Vec<String>,
}
/// One directly observed edge, advertised only by its authenticated source.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct ReachableLink {
/// The other end of this transport link.
pub peer: [u8; 32],
/// Protocol carried by the link.
pub protocol: String,
}
/// Longest address text accepted in a hint.
pub const MAX_HINT_ADDR_LEN: usize = 128;
@@ -256,10 +271,18 @@ pub fn validate(message: &ControlMessage, limits: &Limits) -> Result<(), Protoco
}
}
}
ControlMessage::Reachable { peers } => {
// One entry per member at most; a sender claiming more than a
// network can hold is not describing this network.
check_len("reachable.peers", peers.len(), limits.max_state_records)?;
ControlMessage::Reachable { links } => {
check_len("reachable.links", links.len(), limits.max_state_records)?;
for link in links {
if link.protocol.is_empty() {
return Err(ProtocolError::Malformed("empty routing protocol"));
}
check_len(
"reachable.protocol",
link.protocol.len(),
MAX_PROTOCOL_ID_LEN,
)?;
}
}
ControlMessage::Announce(announcement) => {
check_len(
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -248,6 +248,24 @@ impl Storage {
.await
}
/// Persists local broadcast participation for one configured network.
pub async fn set_broadcast(&self, network_id: NetworkId, enabled: bool) -> Result<()> {
self.with_state(move |state| state.set_broadcast(network_id, enabled))
.await
}
/// Persists whether this agent offers itself as an exit node in a network.
pub async fn set_exit_node(&self, network_id: NetworkId, enabled: bool) -> Result<()> {
self.with_state(move |state| state.set_exit_node(network_id, enabled))
.await
}
/// Persists the member this device sends its internet traffic through.
pub async fn set_exit_via(&self, network_id: NetworkId, via: Option<[u8; 32]>) -> Result<()> {
self.with_state(move |state| state.set_exit_via(network_id, via.as_ref()))
.await
}
/// Updates the auto-start flag of a network.
pub async fn set_auto_start(&self, network_id: NetworkId, auto_start: bool) -> Result<()> {
self.with_state(move |state| state.set_auto_start(network_id, auto_start))
@@ -258,7 +276,8 @@ impl Storage {
pub async fn remove_network(&self, network_id: NetworkId) -> Result<()> {
self.with_state(move |state| {
state.remove_network(network_id)?;
state.forget_signed_records(network_id)
state.forget_signed_records(network_id)?;
state.forget_peer_hostnames(network_id)
})
.await?;
self.with_cache((), move |cache| cache.forget_network(network_id))
@@ -277,6 +296,26 @@ impl Storage {
.await
}
/// Remembers the hostname a member announced, so it can still be named
/// once it is away.
pub async fn remember_peer_hostname(
&self,
network_id: NetworkId,
endpoint_id: [u8; 32],
hostname: String,
) -> Result<()> {
self.with_state(move |state| {
state.remember_peer_hostname(network_id, &endpoint_id, &hostname)
})
.await
}
/// The hostnames members of a network last announced.
pub async fn peer_hostnames(&self, network_id: NetworkId) -> Result<Vec<([u8; 32], String)>> {
self.with_state(move |state| state.peer_hostnames(network_id))
.await
}
/// Records an address hint. Failures are non-fatal.
pub async fn record_hint(
&self,
@@ -21,7 +21,7 @@ use crate::identity::{DeviceIdentity, NetworkId, NetworkName, NetworkSecret};
use crate::state::{RecordBody, SignedRecord};
/// Schema version written by this build.
pub const SCHEMA_VERSION: i64 = 3;
pub const SCHEMA_VERSION: i64 = 6;
/// Key of the stored hostname setting.
const SETTING_HOSTNAME: &str = "hostname";
@@ -40,6 +40,12 @@ pub struct StoredNetwork {
pub secret: NetworkSecret,
/// Whether the network is activated automatically at agent startup.
pub auto_start: bool,
/// Local broadcast participation, enabled unless explicitly disabled.
pub broadcast: bool,
/// This agent offers itself as an exit node here. Off unless turned on.
pub exit_node: bool,
/// The member this device sends all its internet traffic through, if any.
pub exit_via: Option<[u8; 32]>,
}
/// The mandatory state store.
@@ -183,13 +189,51 @@ impl StateStore {
.execute_batch(SIGNED_RECORDS_SCHEMA)
.map_err(|err| self.corrupt(format!("cannot create schema: {err}")))?;
}
if found < 4 {
self.conn.execute_batch("BEGIN;
ALTER TABLE networks ADD COLUMN broadcast INTEGER NOT NULL DEFAULT 1 CHECK (broadcast IN (0,1));
PRAGMA user_version = 4;
COMMIT;").map_err(|err| self.corrupt(format!("cannot migrate schema to 4: {err}")))?;
}
if found < 5 {
self.conn
.execute_batch(
"BEGIN;
CREATE TABLE IF NOT EXISTS peer_hostnames (
network_id BLOB NOT NULL,
endpoint_id BLOB NOT NULL,
hostname TEXT NOT NULL,
PRIMARY KEY (network_id, endpoint_id)
);
PRAGMA user_version = 5;
COMMIT;",
)
.map_err(|err| self.corrupt(format!("cannot migrate schema to 5: {err}")))?;
}
if found < 6 {
self.conn
.execute_batch(
"BEGIN;
ALTER TABLE networks ADD COLUMN exit_node INTEGER NOT NULL DEFAULT 0 CHECK (exit_node IN (0,1));
ALTER TABLE networks ADD COLUMN exit_via BLOB;
PRAGMA user_version = 6;
COMMIT;",
)
.map_err(|err| self.corrupt(format!("cannot migrate schema to 6: {err}")))?;
}
Ok(())
}
/// Confirms the expected tables exist, so that a truncated or foreign
/// database is reported rather than used.
fn verify_shape(&self) -> Result<()> {
for table in ["device_identity", "networks", "settings", "signed_records"] {
for table in [
"device_identity",
"networks",
"settings",
"signed_records",
"peer_hostnames",
] {
let present: Option<String> = self
.conn
.query_row(
@@ -396,6 +440,21 @@ impl StateStore {
Ok(())
}
/// Saves local participation without changing identity, membership or auto-start.
pub fn set_broadcast(&self, network_id: NetworkId, enabled: bool) -> Result<()> {
let changed = self
.conn
.execute(
"UPDATE networks SET broadcast = ?2 WHERE network_id = ?1",
params![network_id.as_bytes().as_slice(), enabled as i64],
)
.map_err(|err| Error::Storage(format!("cannot update broadcast policy: {err}")))?;
if changed == 0 {
return Err(Error::NetworkUnknown(network_id));
}
Ok(())
}
/// Removes a network configuration entirely.
pub fn remove_network(&self, network_id: NetworkId) -> Result<()> {
self.conn
@@ -407,11 +466,56 @@ impl StateStore {
Ok(())
}
/// Saves whether this agent offers itself as an exit node in a network.
pub fn set_exit_node(&self, network_id: NetworkId, enabled: bool) -> Result<()> {
let changed = self
.conn
.execute(
"UPDATE networks SET exit_node = ?2 WHERE network_id = ?1",
params![network_id.as_bytes().as_slice(), enabled as i64],
)
.map_err(|err| Error::Storage(format!("cannot update exit node offer: {err}")))?;
if changed == 0 {
return Err(Error::NetworkUnknown(network_id));
}
Ok(())
}
/// Saves which member this device sends its internet traffic through.
///
/// One device has one default route, so choosing an exit node in one
/// network clears the choice in every other.
pub fn set_exit_via(&self, network_id: NetworkId, via: Option<&[u8; 32]>) -> Result<()> {
let tx = self
.conn
.unchecked_transaction()
.map_err(|err| Error::Storage(format!("cannot update exit node: {err}")))?;
let changed = tx
.execute(
"UPDATE networks SET exit_via = ?2 WHERE network_id = ?1",
params![network_id.as_bytes().as_slice(), via.map(|v| v.as_slice())],
)
.map_err(|err| Error::Storage(format!("cannot update exit node: {err}")))?;
if changed == 0 {
return Err(Error::NetworkUnknown(network_id));
}
if via.is_some() {
tx.execute(
"UPDATE networks SET exit_via = NULL WHERE network_id != ?1",
params![network_id.as_bytes().as_slice()],
)
.map_err(|err| Error::Storage(format!("cannot update exit node: {err}")))?;
}
tx.commit()
.map_err(|err| Error::Storage(format!("cannot update exit node: {err}")))?;
Ok(())
}
/// Lists every configured network.
pub fn list_networks(&self) -> Result<Vec<StoredNetwork>> {
let mut stmt = self
.conn
.prepare("SELECT network_id, name, secret, auto_start FROM networks ORDER BY name")
.prepare("SELECT network_id, name, secret, auto_start, broadcast, exit_node, exit_via FROM networks ORDER BY name")
.map_err(|err| Error::Storage(format!("cannot list networks: {err}")))?;
let rows = stmt
.query_map([], |row| {
@@ -419,13 +523,21 @@ impl StateStore {
let name: String = row.get(1)?;
let secret: Vec<u8> = row.get(2)?;
let auto_start: i64 = row.get(3)?;
Ok((id, name, secret, auto_start != 0))
Ok((
id,
name,
secret,
auto_start != 0,
row.get::<_, bool>(4)?,
row.get::<_, bool>(5)?,
row.get::<_, Option<Vec<u8>>>(6)?,
))
})
.map_err(|err| Error::Storage(format!("cannot list networks: {err}")))?;
let mut out = Vec::new();
for row in rows {
let (id, name, secret, auto_start) =
let (id, name, secret, auto_start, broadcast, exit_node, exit_via) =
row.map_err(|err| Error::Storage(format!("cannot read network row: {err}")))?;
let id: [u8; 32] = id
.as_slice()
@@ -436,6 +548,9 @@ impl StateStore {
name: NetworkName::new(name)?,
secret: NetworkSecret::from_bytes(secret)?,
auto_start,
broadcast,
exit_node,
exit_via: exit_via.and_then(|bytes| <[u8; 32]>::try_from(bytes.as_slice()).ok()),
});
}
Ok(out)
@@ -611,6 +726,61 @@ impl StateStore {
Ok(())
}
/// Remembers the hostname a member last announced.
pub fn remember_peer_hostname(
&self,
network_id: NetworkId,
endpoint_id: &[u8; 32],
hostname: &str,
) -> Result<()> {
self.conn
.execute(
"INSERT INTO peer_hostnames (network_id, endpoint_id, hostname)
VALUES (?1, ?2, ?3)
ON CONFLICT(network_id, endpoint_id) DO UPDATE SET hostname = excluded.hostname",
params![
network_id.as_bytes().as_slice(),
endpoint_id.as_slice(),
hostname
],
)
.map_err(|err| Error::Storage(format!("cannot remember a hostname: {err}")))?;
Ok(())
}
/// The last hostname every member of a network announced.
pub fn peer_hostnames(&self, network_id: NetworkId) -> Result<Vec<([u8; 32], String)>> {
let mut stmt = self
.conn
.prepare("SELECT endpoint_id, hostname FROM peer_hostnames WHERE network_id = ?1")
.map_err(|err| Error::Storage(format!("cannot read hostnames: {err}")))?;
let rows = stmt
.query_map(params![network_id.as_bytes().as_slice()], |row| {
Ok((row.get::<_, Vec<u8>>(0)?, row.get::<_, String>(1)?))
})
.map_err(|err| Error::Storage(format!("cannot read hostnames: {err}")))?;
let mut out = Vec::new();
for row in rows {
let (id, hostname) =
row.map_err(|err| Error::Storage(format!("cannot read hostnames: {err}")))?;
if let Ok(id) = <[u8; 32]>::try_from(id.as_slice()) {
out.push((id, hostname));
}
}
Ok(out)
}
/// Forgets every remembered hostname of a network.
pub fn forget_peer_hostnames(&self, network_id: NetworkId) -> Result<()> {
self.conn
.execute(
"DELETE FROM peer_hostnames WHERE network_id = ?1",
params![network_id.as_bytes().as_slice()],
)
.map_err(|err| Error::Storage(format!("cannot forget hostnames: {err}")))?;
Ok(())
}
/// Reads an arbitrary setting.
pub fn get_setting(&self, key: &str) -> Result<Option<String>> {
self.conn
@@ -668,3 +838,117 @@ pub(crate) fn now_unix() -> i64 {
.map(|d| d.as_secs() as i64)
.unwrap_or(0)
}
#[cfg(test)]
mod broadcast_storage_tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use crate::identity::NetworkKeys;
#[test]
fn exit_node_choices_are_off_by_default_and_one_choice_clears_the_others() {
let dir = tempfile::tempdir().unwrap();
let store = StateStore::open(dir.path().join("state.sqlite")).unwrap();
let secret = NetworkSecret::from_bytes([5; 32]).unwrap();
let make = |name: &str| {
let name = NetworkName::new(name).unwrap();
let id = NetworkKeys::derive(&name, &secret).network_id();
store.upsert_network(id, &name, &secret, true).unwrap();
id
};
let (a, b) = (make("exit-a"), make("exit-b"));
let listed = || store.list_networks().unwrap();
assert!(
listed()
.iter()
.all(|n| !n.exit_node && n.exit_via.is_none())
);
store.set_exit_node(a, true).unwrap();
store.set_exit_via(a, Some(&[7; 32])).unwrap();
store.set_exit_via(b, Some(&[8; 32])).unwrap();
let by_id = |id: NetworkId| listed().into_iter().find(|n| n.network_id == id).unwrap();
assert!(by_id(a).exit_node, "offering is per network and kept");
assert_eq!(by_id(a).exit_via, None, "one default route: a was cleared");
assert_eq!(by_id(b).exit_via, Some([8; 32]));
store.set_exit_via(b, None).unwrap();
assert_eq!(by_id(b).exit_via, None);
}
#[test]
fn a_remembered_hostname_survives_a_restart_and_follows_the_latest_name() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("state.sqlite");
let name = NetworkName::new("hostnames").unwrap();
let secret = NetworkSecret::from_bytes([9; 32]).unwrap();
let id = NetworkKeys::derive(&name, &secret).network_id();
let peer = [3u8; 32];
let store = StateStore::open(&path).unwrap();
store.remember_peer_hostname(id, &peer, "laptop").unwrap();
store.remember_peer_hostname(id, &peer, "laptop-2").unwrap();
drop(store);
let store = StateStore::open(&path).unwrap();
assert_eq!(
store.peer_hostnames(id).unwrap(),
vec![(peer, "laptop-2".to_string())]
);
store.forget_peer_hostnames(id).unwrap();
assert!(store.peer_hostnames(id).unwrap().is_empty());
}
#[test]
fn v3_migration_preserves_identity_and_networks_and_defaults_broadcast_on() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("state.sqlite");
let name = NetworkName::new("migration-broadcast").unwrap();
let secret = NetworkSecret::from_bytes([7; 32]).unwrap();
let id = NetworkKeys::derive(&name, &secret).network_id();
let store = StateStore::open(&path).unwrap();
let identity = store
.load_or_create_device_identity()
.unwrap()
.endpoint_id();
store.upsert_network(id, &name, &secret, false).unwrap();
store.set_hostname("old-host").unwrap();
store
.conn
.execute_batch(
"ALTER TABLE networks DROP COLUMN broadcast; \
ALTER TABLE networks DROP COLUMN exit_node; \
ALTER TABLE networks DROP COLUMN exit_via; \
PRAGMA user_version=3;",
)
.unwrap();
drop(store);
let store = StateStore::open(&path).unwrap();
let networks = store.list_networks().unwrap();
assert_eq!(networks.len(), 1);
assert_eq!(networks[0].network_id, id);
assert!(!networks[0].auto_start);
assert!(networks[0].broadcast);
assert_eq!(store.hostname().unwrap().as_deref(), Some("old-host"));
assert_eq!(
store
.load_or_create_device_identity()
.unwrap()
.endpoint_id(),
identity
);
store.set_broadcast(id, false).unwrap();
store.upsert_network(id, &name, &secret, true).unwrap();
drop(store);
let store = StateStore::open(&path).unwrap();
assert!(
!store.list_networks().unwrap()[0].broadcast,
"rejoin and restart preserve opt-out"
);
store.remove_network(id).unwrap();
store.upsert_network(id, &name, &secret, true).unwrap();
assert!(
store.list_networks().unwrap()[0].broadcast,
"forgotten network gets defaults"
);
}
}
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
+173
View File
@@ -0,0 +1,173 @@
//! Scenario 1: deterministic network identity.
//!
//! The same name and secret must yield the same network space on different
//! devices, and nothing else — hostname, device key, restart — may change it.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use tsunagi::Agent;
use tsunagi::discovery::SharedMemoryDiscovery;
use tsunagi::identity::{NetworkKeys, NetworkName, NetworkSecret};
use tsunagi::testing::{TestAgent, config_with, network};
#[test]
fn derivation_is_a_pure_function_of_name_and_secret() {
let name = NetworkName::new("home").unwrap();
let other = NetworkName::new("Home").unwrap();
let secret = NetworkSecret::generate();
let other_secret = NetworkSecret::generate();
let a = NetworkKeys::derive(&name, &secret);
let b = NetworkKeys::derive(&name, &secret);
assert_eq!(a.network_id(), b.network_id());
assert_eq!(a.discovery_key(), b.discovery_key());
assert_eq!(a.descriptor(), b.descriptor());
// A different name is a different space. Names are used verbatim, so case
// matters.
assert_ne!(
a.network_id(),
NetworkKeys::derive(&other, &secret).network_id()
);
// A different secret is a different space.
assert_ne!(
a.network_id(),
NetworkKeys::derive(&name, &other_secret).network_id()
);
// Separated key material: the discovery key is not the network id.
assert_ne!(a.network_id().as_bytes(), a.discovery_key().as_bytes());
}
#[test]
fn descriptor_carries_no_creator_time_or_secret() {
let name = NetworkName::new("shared").unwrap();
let secret = NetworkSecret::generate();
let first = NetworkKeys::derive(&name, &secret).descriptor();
let second = NetworkKeys::derive(&name, &secret).descriptor();
// Two independently built descriptors are byte identical: no random
// creator id, no creation timestamp, no owner signature.
assert_eq!(first.to_canonical_bytes(), second.to_canonical_bytes());
let encoded = first.to_canonical_bytes();
let secret_bytes = secret.encode();
assert!(
!encoded
.windows(secret_bytes.len())
.any(|window| window == secret_bytes.as_bytes()),
"the secret must never appear in the public descriptor"
);
}
#[test]
fn names_are_validated_not_silently_normalised() {
assert!(NetworkName::new("").is_err());
assert!(NetworkName::new(" home").is_err(), "must not be trimmed");
assert!(NetworkName::new("home ").is_err(), "must not be trimmed");
assert!(NetworkName::new("ho\nme").is_err());
assert!(NetworkName::new("a".repeat(65)).is_err());
assert_eq!(NetworkName::new("home").unwrap().as_str(), "home");
}
#[test]
fn secrets_are_not_truncated_or_normalised() {
let secret = NetworkSecret::generate();
let text = secret.encode();
let round_tripped = NetworkSecret::decode(&text).unwrap();
assert_eq!(secret, round_tripped);
// Short secrets are rejected rather than stretched.
assert!(NetworkSecret::from_bytes(vec![7u8; 15]).is_err());
assert!(NetworkSecret::from_bytes(vec![7u8; 16]).is_ok());
// Debug output must not leak the secret.
let rendered = format!("{secret:?}");
assert_eq!(rendered, "NetworkSecret(<redacted>)");
}
#[tokio::test]
async fn different_devices_and_hostnames_agree_on_the_network_id() {
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("agreement");
let a = TestAgent::spawn_with(|cfg| cfg.with_hostname("alpha"), &discovery)
.await
.unwrap();
let b = TestAgent::spawn_with(|cfg| cfg.with_hostname("beta"), &discovery)
.await
.unwrap();
assert_ne!(
a.agent.endpoint_id(),
b.agent.endpoint_id(),
"different devices must have different endpoint ids"
);
assert_eq!(a.agent.hostname(), "alpha");
assert_eq!(b.agent.hostname(), "beta");
let id_a = a.agent.join_network(&name, &secret).await.unwrap();
let id_b = b.agent.join_network(&name, &secret).await.unwrap();
assert_eq!(id_a, id_b);
a.agent.shutdown().await;
b.agent.shutdown().await;
}
#[tokio::test]
async fn restart_keeps_the_device_and_network_identity() {
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("stable");
let first = TestAgent::spawn(&discovery).await.unwrap();
let device_id = first.agent.endpoint_id();
let network_id = first.agent.join_network(&name, &secret).await.unwrap();
let dir = first.stop().await;
let reopened = Agent::spawn(config_with(dir.path(), &discovery))
.await
.unwrap();
assert_eq!(
reopened.endpoint_id(),
device_id,
"restarting must not mint a new peer"
);
let networks = reopened.list_networks().await.unwrap();
assert_eq!(networks.len(), 1);
assert_eq!(networks[0].network_id, network_id);
assert!(networks[0].active, "auto-start networks come back up");
reopened.shutdown().await;
drop(reopened);
drop(dir);
}
#[tokio::test]
async fn changing_the_secret_keeps_the_device_identity() {
let discovery = SharedMemoryDiscovery::new();
let dir = tempfile::TempDir::new().unwrap();
let agent = Agent::spawn(config_with(dir.path(), &discovery))
.await
.unwrap();
let device_id = agent.endpoint_id();
let name = NetworkName::new("rotating").unwrap();
let old = agent
.join_network(&name, &NetworkSecret::generate())
.await
.unwrap();
let new = agent
.join_network(&name, &NetworkSecret::generate())
.await
.unwrap();
assert_ne!(old, new, "a new secret is a new network space");
assert_eq!(
agent.endpoint_id(),
device_id,
"rotating the network secret must not change the persistent iroh id"
);
agent.shutdown().await;
drop(agent);
drop(dir);
}
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,68 @@
//! The real Mainline UDP receive loop must be quiet while idle on Windows too.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::Ipv4Addr;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
use tracing_subscriber::layer::SubscriberExt;
struct SocketWarnings(Arc<AtomicUsize>);
impl<S: tracing::Subscriber> tracing_subscriber::Layer<S> for SocketWarnings {
fn on_event(&self, event: &tracing::Event<'_>, _: tracing_subscriber::layer::Context<'_, S>) {
if event.metadata().target() == "mainline::rpc::socket"
&& *event.metadata().level() <= tracing::Level::WARN
{
self.0.fetch_add(1, Ordering::Relaxed);
}
}
}
#[tokio::test]
async fn an_idle_dht_socket_stays_quiet_and_still_answers_packets() {
// Mainline owns an OS thread, so a thread-local subscriber cannot see its
// events. This integration-test binary has just this test and installs
// its own subscriber; it cannot affect the application or other binaries.
let warnings = Arc::new(AtomicUsize::new(0));
tracing::subscriber::set_global_default(
tracing_subscriber::registry().with(SocketWarnings(Arc::clone(&warnings))),
)
.unwrap();
tokio::time::timeout(Duration::from_secs(5), async {
let dht = mainline::Dht::builder()
.no_bootstrap()
.server_mode()
.bind_address(Ipv4Addr::LOCALHOST)
.port(0)
.build()
.unwrap()
.as_async();
let address = dht.info().await.local_addr();
// Each info response comes from the actor after another receive-loop
// iteration. With no peers or packets, these exercise real socket
// read timeouts, without an arbitrary sleep or public bootstrap.
for _ in 0..8 {
assert_eq!(dht.info().await.local_addr(), address);
}
assert_eq!(warnings.load(Ordering::Relaxed), 0, "idle receive warnings");
let client = tokio::net::UdpSocket::bind((Ipv4Addr::LOCALHOST, 0))
.await
.unwrap();
let ping = b"d1:ad2:id20:abcdefghijklmnopqrste1:q4:ping1:t4:ping1:y1:qe";
client.send_to(ping, address).await.unwrap();
let mut response = [0u8; 2048];
let (len, from) = client.recv_from(&mut response).await.unwrap();
assert_eq!(from, std::net::SocketAddr::V4(address));
let response = &response[..len];
assert!(response.windows(9).any(|field| field == b"1:t4:ping"));
assert!(response.windows(6).any(|field| field == b"1:y1:r"));
assert_eq!(warnings.load(Ordering::Relaxed), 0);
})
.await
.expect("the DHT receive loop keeps making progress");
}
File renamed without changes.
File renamed without changes.
File renamed without changes.
+41
View File
@@ -0,0 +1,41 @@
[package]
name = "tsunagi-tcp-tls"
version = "0.1.0"
description = "TCP TLS 443 transport and E2EE tunnel protocol for tsunagi mesh networks."
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
tsunagi-core = { path = "../tsunagi-core", version = "0.1.0" }
iroh.workspace = true
tokio = { workspace = true, features = ["net", "io-util", "time", "sync", "macros"] }
rustls = "0.23"
tokio-rustls = "0.26"
rcgen = { version = "0.14", default-features = false, features = ["ring"] }
x25519-dalek = { version = "2.0", features = ["static_secrets"] }
chacha20poly1305 = "0.10"
sha2 = "0.11"
hkdf = "0.13"
rand = "0.10"
subtle = "2.6"
zeroize = { version = "1.9", features = ["derive"] }
bytes.workspace = true
serde.workspace = true
postcard.workspace = true
tracing.workspace = true
thiserror.workspace = true
hex.workspace = true
netdev = "0.46"
[target.'cfg(target_os = "linux")'.dependencies]
caps = "0.5"
[dev-dependencies]
tsunagi-core = { path = "../tsunagi-core", features = ["testing"] }
tempfile.workspace = true
tracing-subscriber.workspace = true
[lints]
workspace = true
+147
View File
@@ -0,0 +1,147 @@
//! TLS certificate generation and verification.
//!
//! Generates a self-signed TLS certificate derived from the device's persistent
//! identity, and verifies remote certificates against the expected peer's
//! [`iroh::EndpointId`].
use std::sync::Arc;
use iroh::EndpointId;
use rustls::client::danger::{ServerCertVerified, ServerCertVerifier};
use rustls::pki_types::{CertificateDer, ServerName, UnixTime};
use rustls::{ClientConfig, Error as RustlsError, ServerConfig};
/// Verifies that the server certificate's public key matches the expected peer.
#[derive(Debug)]
pub struct PinnedEndpointVerifier {
expected: EndpointId,
}
impl PinnedEndpointVerifier {
/// Creates a verifier expecting `peer`.
pub fn new(peer: EndpointId) -> Self {
Self { expected: peer }
}
}
impl ServerCertVerifier for PinnedEndpointVerifier {
fn verify_server_cert(
&self,
end_entity: &CertificateDer<'_>,
_intermediates: &[CertificateDer<'_>],
_server_name: &ServerName<'_>,
_ocsp_response: &[u8],
_now: UnixTime,
) -> Result<ServerCertVerified, RustlsError> {
// Parse the certificate's SPKI / public key
let cert_bytes = end_entity.as_ref();
let expected_bytes = self.expected.as_bytes();
// Check if the expected 32-byte Ed25519 public key is in the certificate
if cert_bytes
.windows(32)
.any(|window| window == expected_bytes)
{
Ok(ServerCertVerified::assertion())
} else {
Err(RustlsError::InvalidCertificate(
rustls::CertificateError::NotValidForName,
))
}
}
fn verify_tls12_signature(
&self,
_message: &[u8],
_cert: &CertificateDer<'_>,
_dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, RustlsError> {
// We require TLS 1.3
Err(RustlsError::PeerIncompatible(
rustls::PeerIncompatible::NoSignatureSchemesInCommon,
))
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, RustlsError> {
rustls::crypto::verify_tls13_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
vec![rustls::SignatureScheme::ED25519]
}
}
/// Default SNI domain used to disguise TLS connections.
pub const DEFAULT_SNI: &str = "cloudflare.com";
/// Generates a self-signed certificate and private key for TLS 1.3 using
/// a given 32-byte Ed25519 secret key and specified SNI domain.
pub fn generate_self_signed_cert(
secret_key_bytes: &[u8; 32],
sni: &str,
) -> Result<
(
CertificateDer<'static>,
rustls::pki_types::PrivateKeyDer<'static>,
),
Box<dyn std::error::Error + Send + Sync>,
> {
// PKCS#8 prefix for v1 Ed25519 private keys:
// 30 2e 02 01 00 30 05 06 03 2b 65 70 04 22 04 20 <32 bytes>
let mut pkcs8 = Vec::with_capacity(48);
pkcs8.extend_from_slice(&[
0x30, 0x2e, 0x02, 0x01, 0x00, 0x30, 0x05, 0x06, 0x03, 0x2b, 0x65, 0x70, 0x04, 0x22, 0x04,
0x20,
]);
pkcs8.extend_from_slice(secret_key_bytes);
let key_der = rustls::pki_types::PrivatePkcs8KeyDer::from(pkcs8);
let key_pair = rcgen::KeyPair::from_pkcs8_der_and_sign_algo(&key_der, &rcgen::PKCS_ED25519)?;
let mut params = rcgen::CertificateParams::new(vec![sni.to_string()])?;
params
.distinguished_name
.push(rcgen::DnType::CommonName, sni);
let cert = params.self_signed(&key_pair)?;
let cert_der = CertificateDer::from(cert.der().to_vec());
let priv_key = rustls::pki_types::PrivateKeyDer::Pkcs8(key_der);
Ok((cert_der, priv_key))
}
/// Creates a `rustls::ServerConfig` for the node.
pub fn make_server_config(
cert: CertificateDer<'static>,
key: rustls::pki_types::PrivateKeyDer<'static>,
) -> Result<Arc<ServerConfig>, rustls::Error> {
let provider = Arc::new(rustls::crypto::ring::default_provider());
let mut config = ServerConfig::builder_with_provider(provider)
.with_safe_default_protocol_versions()?
.with_no_client_auth()
.with_single_cert(vec![cert], key)?;
config.alpn_protocols = vec![b"http/1.1".to_vec()];
Ok(Arc::new(config))
}
/// Creates a `rustls::ClientConfig` expecting a specific peer's `EndpointId`.
pub fn make_client_config(expected_peer: EndpointId) -> Result<Arc<ClientConfig>, rustls::Error> {
let provider = Arc::new(rustls::crypto::ring::default_provider());
let verifier = Arc::new(PinnedEndpointVerifier::new(expected_peer));
let mut config = ClientConfig::builder_with_provider(provider)
.with_safe_default_protocol_versions()?
.dangerous()
.with_custom_certificate_verifier(verifier)
.with_no_client_auth();
config.alpn_protocols = vec![b"http/1.1".to_vec()];
Ok(Arc::new(config))
}
+104
View File
@@ -0,0 +1,104 @@
//! End-to-end packet encryption for `tcp-tls`.
//!
//! Uses X25519 Diffie-Hellman key exchange and ChaCha20-Poly1305 AEAD.
//! This ensures that even when traffic passes through intermediate relay nodes,
//! the intermediate nodes cannot inspect or tamper with the payload.
use std::sync::atomic::{AtomicU64, Ordering};
use chacha20poly1305::aead::{Aead, KeyInit};
use chacha20poly1305::{ChaCha20Poly1305, Nonce};
use hkdf::Hkdf;
use iroh::EndpointId;
use sha2::Sha256;
use tsunagi::dataplane::PluginError;
use x25519_dalek::{PublicKey, StaticSecret};
/// Length of the X25519 public key.
pub const KEY_LEN: usize = 32;
/// Domain separator for session key derivation.
const HKDF_SALT: &[u8] = b"tsunagi/e2ee/noise/v1";
/// A session cipher for communicating with one peer.
pub struct PeerCipher {
tx_cipher: ChaCha20Poly1305,
rx_cipher: ChaCha20Poly1305,
tx_nonce: AtomicU64,
}
impl PeerCipher {
/// Establishes a new cipher between `local_id` and `remote_id` given
/// our local secret and their public key.
pub fn new(
local_id: EndpointId,
remote_id: EndpointId,
local_secret: &StaticSecret,
remote_public: &PublicKey,
) -> Self {
let shared = local_secret.diffie_hellman(remote_public);
let hk = Hkdf::<Sha256>::new(Some(HKDF_SALT), shared.as_bytes());
let mut key_a_to_b = [0u8; 32];
let mut key_b_to_a = [0u8; 32];
let _ = hk.expand(b"tsunagi-key-a-to-b", &mut key_a_to_b);
let _ = hk.expand(b"tsunagi-key-b-to-a", &mut key_b_to_a);
// Deterministic role: smaller EndpointId is party A.
let is_party_a = local_id.as_bytes() < remote_id.as_bytes();
let (tx_key, rx_key) = if is_party_a {
(key_a_to_b, key_b_to_a)
} else {
(key_b_to_a, key_a_to_b)
};
let tx_cipher = ChaCha20Poly1305::new(&tx_key.into());
let rx_cipher = ChaCha20Poly1305::new(&rx_key.into());
Self {
tx_cipher,
rx_cipher,
tx_nonce: AtomicU64::new(0),
}
}
/// Encrypts an outbound packet. Prefixes the ciphertext with the 8-byte nonce.
pub fn encrypt(&self, packet: &[u8]) -> Result<bytes::Bytes, PluginError> {
let counter = self.tx_nonce.fetch_add(1, Ordering::Relaxed);
let mut nonce_bytes = [0u8; 12];
nonce_bytes[4..12].copy_from_slice(&counter.to_be_bytes());
let nonce = Nonce::from_slice(&nonce_bytes);
let ciphertext = self
.tx_cipher
.encrypt(nonce, packet)
.map_err(|err| PluginError::Other(format!("encryption failed: {err}")))?;
let mut out = Vec::with_capacity(8 + ciphertext.len());
out.extend_from_slice(&counter.to_be_bytes());
out.extend_from_slice(&ciphertext);
Ok(bytes::Bytes::from(out))
}
/// Decrypts an inbound packet prefixed with the 8-byte nonce.
pub fn decrypt(&self, payload: &[u8]) -> Result<bytes::Bytes, PluginError> {
if payload.len() < 8 + 16 {
return Err(PluginError::Other("payload too short".into()));
}
let counter_bytes: [u8; 8] = match payload[..8].try_into() {
Ok(b) => b,
Err(_) => return Err(PluginError::Other("invalid nonce".into())),
};
let mut nonce_bytes = [0u8; 12];
nonce_bytes[4..12].copy_from_slice(&counter_bytes);
let nonce = Nonce::from_slice(&nonce_bytes);
let plaintext = self
.rx_cipher
.decrypt(nonce, &payload[8..])
.map_err(|err| PluginError::Other(format!("decryption failed: {err}")))?;
Ok(bytes::Bytes::from(plaintext))
}
}
+11
View File
@@ -0,0 +1,11 @@
//! The `tcp-tls` protocol: TCP with TLS 1.3 on port 443 with Noise E2EE.
pub mod cert;
pub mod crypto;
pub mod plugin;
pub mod transport;
pub use plugin::{
TCP_TLS_PROTOCOL, TCP_TLS_VERSION, TcpTlsCodec, TcpTlsConfig, TcpTlsPlugin, local_ip_candidates,
};
pub use transport::TcpTlsTransport;
+530
View File
@@ -0,0 +1,530 @@
//! The `tcp-tls` IP plugin implementation.
use std::collections::HashMap;
use std::sync::{Arc, RwLock};
use bytes::Bytes;
use iroh::EndpointId;
use serde::{Deserialize, Serialize};
use x25519_dalek::{PublicKey, StaticSecret};
use tsunagi::dataplane::tunnel::{GenericTunnelPlugin, TunnelCodec};
use tsunagi::dataplane::{PluginError, ProtocolOption};
use tsunagi::identity::NetworkId;
use crate::crypto::PeerCipher;
use crate::transport::TcpTlsTransport;
/// The wire protocol identifier for `tcp-tls`.
pub const TCP_TLS_PROTOCOL: &str = "tcp-tls";
/// Wire version for `tcp-tls`.
pub const TCP_TLS_VERSION: u16 = 1;
/// Configuration for the `tcp-tls` protocol plugin.
#[derive(Debug, Clone, Default)]
pub struct TcpTlsConfig {
/// Requested port to bind.
pub port: Option<u16>,
/// Custom SNI domain to disguise TLS connections.
pub sni: Option<String>,
}
impl TcpTlsConfig {
/// Creates a configuration with default settings.
pub fn new() -> Self {
Self::default()
}
/// Sets the TCP listen port.
pub fn with_port(mut self, port: u16) -> Self {
self.port = Some(port);
self
}
/// Sets the SNI domain used to disguise TLS connections.
pub fn with_sni(mut self, sni: impl Into<String>) -> Self {
self.sni = Some(sni.into());
self
}
}
/// Announcement payload exchanged between peers for `tcp-tls`.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct TcpTlsAnnouncement {
/// X25519 public key for E2EE noise encryption.
pub noise_public: [u8; 32],
/// TCP port the peer is listening on.
pub port: u16,
/// Candidate IP addresses where this peer can be reached on `port`.
#[serde(default)]
pub addrs: Vec<std::net::IpAddr>,
}
/// Discovers local network interface IPs to announce to peers.
pub fn local_ip_candidates() -> Vec<std::net::IpAddr> {
let mut candidates = Vec::new();
for iface in netdev::get_interfaces() {
if !is_candidate_interface(&iface) {
continue;
}
for ip in &iface.ipv4 {
let addr = ip.addr();
if is_candidate_ipv4(addr) {
let priority = ip_priority(&iface, std::net::IpAddr::V4(addr));
candidates.push((priority, std::net::IpAddr::V4(addr)));
}
}
for ip in &iface.ipv6 {
let addr = ip.addr();
if is_candidate_ip(std::net::IpAddr::V6(addr)) {
let priority = ip_priority(&iface, std::net::IpAddr::V6(addr));
candidates.push((priority, std::net::IpAddr::V6(addr)));
}
}
}
// Sort descending by priority so best interface IP is candidate #0
candidates.sort_by_key(|b| std::cmp::Reverse(b.0));
let mut seen = std::collections::HashSet::new();
let mut ips = Vec::new();
for (_, ip) in candidates {
if seen.insert(ip) {
ips.push(ip);
}
}
ips
}
pub(crate) fn is_candidate_interface(iface: &netdev::Interface) -> bool {
if !iface.is_up() || iface.is_loopback() {
return false;
}
let name = iface.name.to_lowercase();
let friendly = iface
.friendly_name
.as_deref()
.map(str::to_lowercase)
.unwrap_or_default();
let desc = iface
.description
.as_deref()
.map(str::to_lowercase)
.unwrap_or_default();
// Skip Tsunagi / Wintun overlay interfaces
if name.starts_with("tsun")
|| name.contains("tsunagi")
|| name.contains("wintun")
|| friendly.starts_with("tsun")
|| friendly.contains("tsunagi")
|| friendly.contains("wintun")
|| desc.contains("wintun")
|| desc.contains("tsunagi")
{
return false;
}
// Skip container, VM, and virtual bridge interfaces
if name.starts_with("docker")
|| name.starts_with("br-")
|| name.starts_with("veth")
|| name.starts_with("virbr")
|| name.starts_with("cni")
|| name.starts_with("flannel")
|| name.starts_with("cali")
|| name.starts_with("kube")
|| name.starts_with("dummy")
|| name.starts_with("lxc")
|| name.starts_with("podman")
|| friendly.contains("vethernet")
|| friendly.contains("vboxnet")
|| friendly.contains("vmnet")
|| desc.contains("virtualbox")
|| desc.contains("hyper-v")
|| desc.contains("vmware")
{
return false;
}
true
}
pub(crate) fn is_candidate_ipv4(addr: std::net::Ipv4Addr) -> bool {
if addr.is_loopback() || addr.is_unspecified() || addr.is_link_local() || addr.is_broadcast() {
return false;
}
if addr.is_documentation() {
return false;
}
true
}
pub(crate) fn is_candidate_ip(addr: std::net::IpAddr) -> bool {
match addr {
std::net::IpAddr::V4(v4) => is_candidate_ipv4(v4),
std::net::IpAddr::V6(v6) => {
if v6.is_loopback() || v6.is_unspecified() {
return false;
}
let segments = v6.segments();
// Filter link-local fe80::/10
if (segments[0] & 0xffc0) == 0xfe80 {
return false;
}
true
}
}
}
pub(crate) fn ip_priority(iface: &netdev::Interface, ip: std::net::IpAddr) -> u32 {
let has_gateway = iface.gateway.is_some();
let base = if has_gateway { 100 } else { 0 };
match ip {
std::net::IpAddr::V4(v4) => {
let octets = v4.octets();
// Tailscale: 100.64.0.0/10
let is_tailscale = octets[0] == 100 && (octets[1] & 0xc0) == 64;
let is_private = v4.is_private();
if !is_private && !is_tailscale {
base + 50
} else if has_gateway && is_private {
base + 40
} else if is_tailscale {
base + 30
} else {
base + 10
}
}
std::net::IpAddr::V6(v6) => {
let segments = v6.segments();
let is_global = (segments[0] & 0xe000) == 0x2000;
if is_global { base + 45 } else { base + 20 }
}
}
}
/// The codec implementing cryptographic transformation and announcements for `tcp-tls`.
pub struct TcpTlsCodec {
local_id: EndpointId,
static_secret: StaticSecret,
public_key: PublicKey,
transport: Arc<TcpTlsTransport>,
ciphers: RwLock<HashMap<(NetworkId, EndpointId), Arc<PeerCipher>>>,
}
impl std::fmt::Debug for TcpTlsCodec {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("TcpTlsCodec")
.field("local_id", &self.local_id.fmt_short().to_string())
.field("port", &self.transport.bound_port())
.finish()
}
}
impl TcpTlsCodec {
/// Supported options for `tcp-tls`.
pub const OPTIONS: &'static [ProtocolOption] = &[
ProtocolOption {
key: "port",
value: "PORT",
help: "TCP port to listen on for TLS 1.3 connections",
default: Some("443"),
},
ProtocolOption {
key: "sni",
value: "DOMAIN",
help: "Server Name Indication (SNI) domain to disguise TLS connections",
default: Some(crate::cert::DEFAULT_SNI),
},
];
/// Creates a new `TcpTlsCodec`.
pub fn new(local_id: EndpointId, transport: Arc<TcpTlsTransport>) -> Self {
let mut secret_bytes = [0u8; 32];
rand::fill(&mut secret_bytes);
let static_secret = StaticSecret::from(secret_bytes);
let public_key = PublicKey::from(&static_secret);
Self {
local_id,
static_secret,
public_key,
transport,
ciphers: RwLock::new(HashMap::new()),
}
}
/// Configures settings from parsed `-o` options.
pub fn configure(
mut config: TcpTlsConfig,
options: &[(String, String)],
) -> Result<TcpTlsConfig, String> {
for (key, value) in options {
match key.as_str() {
"port" => {
let port: u16 = value
.parse()
.map_err(|_| format!("invalid port `{value}`"))?;
config.port = Some(port);
}
"sni" => {
let trimmed = value.trim();
if trimmed.is_empty() {
return Err("sni domain cannot be empty".into());
}
config.sni = Some(trimmed.to_string());
}
other => return Err(format!("unknown option `{other}` for tcp-tls")),
}
}
Ok(config)
}
}
impl TunnelCodec for TcpTlsCodec {
fn protocol_id(&self) -> &str {
TCP_TLS_PROTOCOL
}
fn protocol_version(&self) -> u16 {
TCP_TLS_VERSION
}
fn options(&self) -> &'static [ProtocolOption] {
Self::OPTIONS
}
fn local_capability_data(&self, _network: NetworkId) -> Result<Vec<u8>, PluginError> {
let announcement = TcpTlsAnnouncement {
noise_public: *self.public_key.as_bytes(),
port: self.transport.bound_port(),
addrs: local_ip_candidates(),
};
postcard::to_allocvec(&announcement)
.map_err(|err| PluginError::Other(format!("encode announcement failed: {err}")))
}
fn on_peer_capability_data(
&self,
network: NetworkId,
peer: EndpointId,
data: &[u8],
) -> Result<(), PluginError> {
let announcement: TcpTlsAnnouncement = postcard::from_bytes(data)
.map_err(|err| PluginError::Rejected(format!("invalid announcement: {err}")))?;
for ip in &announcement.addrs {
if is_candidate_ip(*ip) {
self.transport
.set_peer_addr(peer, std::net::SocketAddr::new(*ip, announcement.port));
}
}
let peer_noise_pubkey = PublicKey::from(announcement.noise_public);
let cipher = Arc::new(PeerCipher::new(
self.local_id,
peer,
&self.static_secret,
&peer_noise_pubkey,
));
let mut guard = match self.ciphers.write() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.insert((network, peer), cipher);
Ok(())
}
fn on_peer_down(&self, network: NetworkId, peer: EndpointId) {
let mut guard = match self.ciphers.write() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.remove(&(network, peer));
}
fn encrypt(
&self,
network: NetworkId,
peer: EndpointId,
packet: &[u8],
) -> Result<Bytes, PluginError> {
let cipher = {
let guard = match self.ciphers.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.get(&(network, peer)).cloned()
};
let Some(cipher) = cipher else {
return Err(PluginError::Unavailable("no cipher for peer".into()));
};
cipher.encrypt(packet)
}
fn decrypt(
&self,
network: NetworkId,
peer: EndpointId,
payload: &[u8],
) -> Result<Bytes, PluginError> {
let cipher = {
let guard = match self.ciphers.read() {
Ok(g) => g,
Err(p) => p.into_inner(),
};
guard.get(&(network, peer)).cloned()
};
let Some(cipher) = cipher else {
return Err(PluginError::Unavailable("no cipher for peer".into()));
};
cipher.decrypt(payload)
}
}
/// The complete `tcp-tls` plugin.
pub type TcpTlsPlugin = GenericTunnelPlugin<TcpTlsCodec>;
#[cfg(test)]
mod tests {
use super::*;
use std::net::{Ipv4Addr, Ipv6Addr};
#[test]
fn test_is_candidate_interface_filters_overlay_and_bridges() {
let mut iface = netdev::Interface::dummy();
iface.flags = 1; // UP
// Standard Ethernet - should pass
iface.name = "eth0".to_string();
assert!(is_candidate_interface(&iface));
// Windows Ethernet with GUID name - should pass
iface.name = "{B6149D2C-1234-5678-9ABC-DEF012345678}".to_string();
iface.friendly_name = Some("Ethernet".to_string());
iface.description = Some("Intel(R) Ethernet Connection".to_string());
assert!(is_candidate_interface(&iface));
// Linux tsun0 overlay - must be filtered
iface.name = "tsun0".to_string();
iface.friendly_name = None;
iface.description = None;
assert!(!is_candidate_interface(&iface));
// Windows Wintun overlay (GUID name with tsun0 friendly name) - must be filtered!
iface.name = "{72D85D76-F1A5-42AC-BC49-2169527CDDDF}".to_string();
iface.friendly_name = Some("tsun0".to_string());
iface.description = Some("Wintun Userspace Tunnel".to_string());
assert!(!is_candidate_interface(&iface));
// Windows Wintun overlay with description containing Wintun - must be filtered!
iface.friendly_name = Some("Local Area Connection 2".to_string());
iface.description = Some("Wintun Userspace Tunnel".to_string());
assert!(!is_candidate_interface(&iface));
// Docker bridge - must be filtered
iface.name = "docker0".to_string();
iface.friendly_name = None;
iface.description = None;
assert!(!is_candidate_interface(&iface));
// Docker network bridge br-* - must be filtered
iface.name = "br-4f6d3a9b1c2e".to_string();
assert!(!is_candidate_interface(&iface));
// Virtual ethernet veth* - must be filtered
iface.name = "veth12345".to_string();
assert!(!is_candidate_interface(&iface));
// Libvirt NAT bridge virbr* - must be filtered
iface.name = "virbr0".to_string();
assert!(!is_candidate_interface(&iface));
// CNI bridge - must be filtered
iface.name = "cni0".to_string();
assert!(!is_candidate_interface(&iface));
// Hyper-V / WSL vEthernet switch on Windows - must be filtered
iface.name = "{A1B2C3D4-E5F6-7890-1234-567890ABCDEF}".to_string();
iface.friendly_name = Some("vEthernet (WSL)".to_string());
iface.description = Some("Hyper-V Virtual Ethernet Adapter".to_string());
assert!(!is_candidate_interface(&iface));
// Down interface - must be filtered
let mut down_iface = netdev::Interface::dummy();
down_iface.flags = 0; // Not UP
down_iface.name = "eth0".to_string();
assert!(!is_candidate_interface(&down_iface));
}
#[test]
fn test_is_candidate_ip() {
// Valid LAN IP
assert!(is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
192, 168, 1, 117
))));
// Valid Tailscale IP
assert!(is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
100, 77, 155, 120
))));
// Valid Public IP
assert!(is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
1, 1, 1, 1
))));
// Valid IPv6 GUA
assert!(is_candidate_ip(std::net::IpAddr::V6(Ipv6Addr::new(
0x2a01, 0x4b00, 0xb8e3, 0x4e00, 0, 0, 0, 1
))));
// Loopback - must be rejected
assert!(!is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
127, 0, 0, 1
))));
assert!(!is_candidate_ip(std::net::IpAddr::V6(Ipv6Addr::LOCALHOST)));
// Unspecified - must be rejected
assert!(!is_candidate_ip(std::net::IpAddr::V4(
Ipv4Addr::UNSPECIFIED
)));
assert!(!is_candidate_ip(std::net::IpAddr::V6(
Ipv6Addr::UNSPECIFIED
)));
// Link-local - must be rejected
assert!(!is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::new(
169, 254, 1, 1
))));
assert!(!is_candidate_ip(std::net::IpAddr::V6(Ipv6Addr::new(
0xfe80, 0, 0, 0, 0, 0x1ff, 0xfe00, 1
))));
// Broadcast - must be rejected
assert!(!is_candidate_ip(std::net::IpAddr::V4(Ipv4Addr::BROADCAST)));
}
#[test]
fn test_ip_priority_ordering() {
let mut iface_lan = netdev::Interface::dummy();
iface_lan.flags = 1;
iface_lan.gateway = Some(netdev::net::device::NetworkDevice::new());
let mut iface_vpn = netdev::Interface::dummy();
iface_vpn.flags = 1;
iface_vpn.name = "tailscale0".to_string();
let lan_ip = std::net::IpAddr::V4(Ipv4Addr::new(192, 168, 1, 117));
let tailscale_ip = std::net::IpAddr::V4(Ipv4Addr::new(100, 77, 155, 120));
let p_lan = ip_priority(&iface_lan, lan_ip);
let p_ts = ip_priority(&iface_vpn, tailscale_ip);
// LAN IP with gateway should score higher than VPN IP without gateway
assert!(p_lan > p_ts);
}
}
Loaded 100 of 189 files, more files were not shown because too many files have changed in this diff. Show more