Relay across any mix of links, one online answer for every client, first claim owns a hostname

A tunnel no longer depends on the kind of link under it. The relay hub keeps one forwarding table per network instead of one per protocol, the envelope names the end-to-end protocol, and a neighbour reached over several kinds of transport is sent to over the best open one. A member in the middle needs no protocol in common with either end. Which protocol a pair runs is settled by the lower endpoint id's announced order, so both ends agree without asking. A direct link that comes up later is shorter than the path through others, so traffic moves to it with the same logical link. The agent now answers "is it online" once: a live control session or any data path. Members and peers report how they are reached (direct over a named transport, relayed through a named member, or no data path yet), and `tsng status` and the tray both read that instead of a session list and a WireGuard handshake respectively. A hostname belongs to whoever claimed it first. A claim carries the time its author first claimed the name, signed with the record, and replicas order by (time, id) so every member names the same holder. A later claimant keeps working by address, is not found by the name, and is told so in its status, `tsng status`, the tray and a HostnameTaken event. Wire change: control and data ALPN 5, control PROTOCOL_VERSION 5, envelope version 2, signing domain v3, state schema 7, local control protocol 18. Upgrade every member together. Co-Authored-By: Claude <noreply@anthropic.com>
Rewrite README as a short project front page; add a Nix flake
2026-10-05 11:39:33 +01:00 · 2026-10-03 17:47:44 +01:00 · 2026-10-03 14:38:16 +01:00 · 2026-10-03 14:16:05 +01:00 · 2026-10-03 13:43:01 +01:00 · 2026-10-03 12:33:22 +01:00
48 changed files with 4786 additions and 1037 deletions
@@ -0,0 +1,5 @@
 # Windows: link the C runtime statically, so the agent and the tray start on a
 # machine that has no Visual C++ redistributable installed. The installer ships
 # nothing but our own files and Wintun.
 [target.x86_64-pc-windows-msvc]
 rustflags = ["-C", "target-feature=+crt-static"]
@@ -190,6 +190,29 @@ jobs:
            tar -C dist/pkg -czf "${archive}" "${{ matrix.asset }}"
          fi
      # The Windows installer: the agent as a service, Wintun, the tray and the
      # PATH entry in one setup program. Wintun is the signed library from
      # wintun.net, pinned by hash so a changed download fails the build.
      - name: Build the Windows installer
        if: runner.os == 'Windows'
        shell: pwsh
        run: |
          $ErrorActionPreference = 'Stop'
          $version = $env:GITHUB_REF_NAME.TrimStart('v')
          Invoke-WebRequest -Uri https://www.wintun.net/builds/wintun-0.14.1.zip -OutFile wintun.zip
          $hash = (Get-FileHash wintun.zip -Algorithm SHA256).Hash
          if ($hash -ne '07C256185D6EE3652E09FA55C0B673E2624B565E02C4B9091C79CA7D2F24EF51') {
            throw "wintun.zip has an unexpected hash: $hash"
          }
          Expand-Archive wintun.zip -DestinationPath wintun-dl
          choco install innosetup -y --no-progress
          $iscc = Join-Path ${env:ProgramFiles(x86)} 'Inno Setup 6\ISCC.exe'
          $stage = Join-Path $PWD 'dist/pkg/${{ matrix.asset }}'
          & $iscc "/DAppVersion=$version" "/DSourceDir=$stage" `
            "/DWintunDir=$(Join-Path $PWD 'wintun-dl/wintun/bin/amd64')" `
            "/O$PWD" dist/windows/tsunagi.iss
          if ($LASTEXITCODE -ne 0) { throw "ISCC failed with $LASTEXITCODE" }
      # A dependency-free .deb for the x86_64 and aarch64 Linux builds: it drops
      # the static binary, the systemd service and the polkit rule. Built with
      # dpkg-deb (present on the runner); see dist/deb/build-deb.sh.
@@ -251,6 +274,7 @@ jobs:
            ${{ matrix.asset }}-*.*
            *.deb
            *.pkg.tar.zst
            tsunagi-setup-*.exe
          if-no-files-found: error
  publish:
@@ -322,5 +346,5 @@ jobs:
              cat up.json >&2 || true
              rc=1
            fi
-          done < <(find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.deb' -o -name '*.pkg.tar.zst' \) -print0)
+          done < <(find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.deb' -o -name '*.pkg.tar.zst' -o -name 'tsunagi-setup-*.exe' \) -print0)
          exit "$rc"
@@ -192,6 +192,29 @@ jobs:
            tar -C dist/pkg -czf "${archive}" "${{ matrix.asset }}"
          fi
      # The Windows installer: the agent as a service, Wintun, the tray and the
      # PATH entry in one setup program. Wintun is the signed library from
      # wintun.net, pinned by hash so a changed download fails the build.
      - name: Build the Windows installer
        if: runner.os == 'Windows'
        shell: pwsh
        run: |
          $ErrorActionPreference = 'Stop'
          $version = $env:GITHUB_REF_NAME.TrimStart('v')
          Invoke-WebRequest -Uri https://www.wintun.net/builds/wintun-0.14.1.zip -OutFile wintun.zip
          $hash = (Get-FileHash wintun.zip -Algorithm SHA256).Hash
          if ($hash -ne '07C256185D6EE3652E09FA55C0B673E2624B565E02C4B9091C79CA7D2F24EF51') {
            throw "wintun.zip has an unexpected hash: $hash"
          }
          Expand-Archive wintun.zip -DestinationPath wintun-dl
          choco install innosetup -y --no-progress
          $iscc = Join-Path ${env:ProgramFiles(x86)} 'Inno Setup 6\ISCC.exe'
          $stage = Join-Path $PWD 'dist/pkg/${{ matrix.asset }}'
          & $iscc "/DAppVersion=$version" "/DSourceDir=$stage" `
            "/DWintunDir=$(Join-Path $PWD 'wintun-dl/wintun/bin/amd64')" `
            "/O$PWD" dist/windows/tsunagi.iss
          if ($LASTEXITCODE -ne 0) { throw "ISCC failed with $LASTEXITCODE" }
      # A dependency-free .deb for the x86_64 and aarch64 Linux builds: it drops
      # the static binary, the systemd service and the polkit rule. Built with
      # dpkg-deb (present on the runner); see dist/deb/build-deb.sh.
@@ -250,6 +273,7 @@ jobs:
            ${{ matrix.asset }}-*.*
            *.deb
            *.pkg.tar.zst
            tsunagi-setup-*.exe
          if-no-files-found: error
  publish:
@@ -278,7 +302,7 @@ jobs:
          # download-artifact nests each artifact under its own directory, so
          # the archives are artifacts/<asset>/<file>.
-          mapfile -t files < <(find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.deb' -o -name '*.pkg.tar.zst' \))
+          mapfile -t files < <(find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.deb' -o -name '*.pkg.tar.zst' -o -name 'tsunagi-setup-*.exe' \))
          printf 'asset: %s\n' "${files[@]}"
          prerelease=()
@@ -8,7 +8,7 @@ before touching the code. Do not restate the full design here; follow the links.
 A proof-of-concept agent library for small private mesh networks. A user
 supplies a network name and one shared secret; agents derive the same network
 space independently, find each other, prove membership and exchange control
-messages. Scope and non-scope are in [README.md](README.md).
+messages. What it does and does not do is in [docs/architecture.md](docs/architecture.md) and [docs/threat-model.md](docs/threat-model.md).
 There is no central server, no owner, no registration and no majority vote.
 Design accordingly: a majority is not a root of trust.
@@ -59,8 +59,10 @@ Keep these separate. Crossing them is the main thing to review for.
  other peers goes link in, link out: it is never written to the middle's
  interface or decrypted there. Multihop routing has a bounded hop limit;
  equal-cost next hops are chosen per flow. Reachability is first-hand and
-  volatile: each authenticated member advertises its own protocol-specific
+  volatile: each authenticated member advertises its own direct links, of
-  links. Build routing tables on topology changes, never per packet. Transit
+  whatever kind. A link is a way to reach a neighbour and a protocol is an
  end-to-end tunnel to a peer; the two meet only in the envelope, so a
  tunnel never depends on which transports carry it. Build routing tables on topology changes, never per packet. Transit
  must not acquire a routing mutex or wait for a protocol/TUN reader.
 - **Broadcast fanout belongs at local IP ingress.** Participation is local to
  each network, enabled by default, persisted and advertised to authenticated
@@ -6250,6 +6250,7 @@ dependencies = [
 "tsunagi-tcp-tls",
 "tsunagi-wg-quic",
 "tsunagi-wireguard",
 "windows-service",
 ]
 [[package]]
@@ -7135,6 +7136,17 @@ dependencies = [
 "windows-link",
 ]
 [[package]]
 name = "windows-service"
 version = "0.8.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "857224b3b211c6f3616921f081ee54721ee3ad2ace2fac6a6337e032f7b4dcf2"
 dependencies = [
 "bitflags 2.13.2",
 "widestring",
 "windows-sys 0.61.2",
 ]
 [[package]]
 name = "windows-strings"
 version = "0.1.0"
@@ -1,766 +1,183 @@
-# tsunagi
+<p align="center">
-
+  <img src="dist/linux/tsunagi.svg" width="96" alt="">
-A proof-of-concept Rust library for **small private mesh networks** — a handful
+</p>
-of friends, home machines, a few servers. Units to dozens of participants, not
+
-thousands.
+<h1 align="center">tsunagi</h1>
-
+
-The end user configures exactly two things:
+<p align="center">
-
+  <b>A mesh VPN with no server.</b><br>
-```text
+  Pick a network name, share a secret, and your machines are on one private network.
-network_name
+</p>
-secret          # one shared secret; "password" and "secret" mean the same value
+
-```
+<p align="center">
-
+  <a href="https://github.com/house-of-vanity/tsunagi/releases/latest"><img alt="release" src="https://img.shields.io/github/v/release/house-of-vanity/tsunagi?include_prereleases"></a>
-From those, every agent independently derives the same network space. There is
+  <a href="https://github.com/house-of-vanity/tsunagi/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/house-of-vanity/tsunagi/actions/workflows/ci.yml/badge.svg"></a>
-no central server, no network owner with special powers, no registration and no
+  <a href="LICENSE"><img alt="license" src="https://img.shields.io/badge/license-WTFPL-blue"></a>
-majority vote. Anyone who knows the parameters can join; nobody has to trust
+</p>
-anybody else.
+
-
+```console
-## What this proof of concept actually does
+laptop$ tsng join -n home
-
+joined `home`
 A working library with **real iroh connections** and integration tests:
 - persistent device identity stored in SQLite, stable across restarts;
 - several independent networks at once in one agent;
 - deterministic network identity derived from name + secret;
 - candidates supplied by a replaceable discovery component;
 - automatic first contact through the public Mainline DHT (BEP44);
 - real iroh connections plus an explicit mutual proof of network membership;
 - a small versioned control protocol: handshake, hostname/capability
  announcement, ping/pong;
 - automatic reconnect with bounded exponential backoff and jitter;
 - status snapshots, an event stream and honest diagnostics;
 - configuration restored after a restart;
 - correct behaviour when the disposable cache is missing or corrupt;
 - a **WireGuard data plane**, in userspace: its own key per network, an IPv6
  overlay with deterministically derived addresses and optional IPv4, real
  tunnels carried over iroh, and address ownership enforced rather than
  believed;
 - a **command line agent**, `tsng`, with a local control socket.
 ### What it deliberately does **not** do
 Not implemented, and not pretended to be: DNS, routing through
 intermediate participants, a full CRDT, dynamically loaded plugins, a system
 service, a complete CLI, or a local control socket. Snapshot synchronisation
 and signed revocations are designed for but not implemented — see
 [docs/sync-model.md](docs/sync-model.md). The WireGuard plugin's own limits,
 including that its system backend is Linux-only, are in
 [docs/wireguard.md](docs/wireguard.md#limits-and-future-work).
 **Control and data are separated logically, not physically.** Both ride on
 iroh, on different ALPNs and different connections, so the data plane inherits
 iroh's hole punching and relay fallback instead of reimplementing them — while
 the control protocol still knows nothing about packets and can keep a different
 transport underneath it later. Filtering user traffic remains the operating
 system's and the user's responsibility, not this library's.
 ## Requirements
 - Rust 1.91 or newer (iroh 1.2 requires it), edition 2024. Pinned dependencies
  in `Cargo.lock`.
 - No internet, no public DHT, no public relay, no administrator rights and no changes
  to OS network settings are needed to build or test.
 - WireGuard runs in userspace (boringtun): **no kernel module and no `wg`
  tool**. Only creating a real network interface needs `CAP_NET_ADMIN`, and
  `--no-tun` skips even that.
 ## Trying it on two machines
 On the first machine:
 ```bash
 cargo build --release
 ./target/release/tsng status          # this device, the agent, and this host
 ./target/release/tsng network         # the networks this device belongs to
 ```
 Two commands do the work, and they are separate on purpose: `up` runs the
 agent — one per device, for as long as you want it — and `join` decides
 what it belongs to, at any time, while it runs.
 ```bash
 # Terminal one: the agent. It prints its endpoint id and then serves.
 ./target/release/tsng up
 # Terminal two: make a network. With no --secret it invents one and prints
 # it, along with the line to send the other machine.
 ./target/release/tsng join --network lab
 ```
 On the second machine, start its agent and join the same network with the
 secret that was printed:
 ```bash
 ./target/release/tsng up
 ./target/release/tsng join --network lab --secret tsn1...
 ```
 **Name and secret are enough for the first meeting.** Mainline DHT discovery
 is enabled by default. Each active network publishes this device's endpoint
 and searches until the first authenticated connection. While connected it
 only republishes; after 60 seconds without any authenticated connection it
 searches again. Publication is approximately every five minutes, and records
 older than 15 minutes are ignored. DHT storage nodes may retain them longer.
 Use `--no-dht` to disable DHT, or `--peer <endpoint-id>` for optional manual
 bootstrap. `--reach local` disables public DHT regardless of the DHT flag.
 Endpoint records are public, signed, and contain no network secret. Discovery
 is not authentication: every connection must still prove network membership.
 Details and failure behavior are in [docs/mainline-dht.md](docs/mainline-dht.md).
 **One introduction is enough for the whole network.** Members tell each
 other about the members they know, and every author of a signed record is
 somebody to try, so a device pointed at one member ends up talking to all
 of them rather than to the one that happened to be on its command line.
 What travels is a candidate — an address somebody has seen — and it is
 authenticated by the handshake like any other; being introduced grants
 nothing. It is deliberately the same shape a lookup in a distributed hash
 table would return, so that is a source to add beside this one rather than
 a redesign.
 An agent with nobody to contact says so in `status` rather than sitting
 there looking patient.
 Within a few seconds both print something like:
 ```text
  + peer b47c958462 connected over direct rtt=Some(4.5ms)
  + data link to b47c958462 for wg-quic: direct via 192.0.2.7:41234, datagram 1382
 --- status ---
 control: 1 peer(s), 0 dial failure(s), 0 handshake failure(s)
 wg-quic: tsun0 on 10.13.37.69/24 mtu 1280, 1/1 tunnel(s) established
  4jO4kx9Z 10.13.37.237 handshake 3s ago tx=0 rx=0 dropped=0 path=direct via 192.0.2.7:41234
 ```
 `1/1 tunnel(s) established` means a real WireGuard handshake completed.
 ## Checking that it works
 From another shell on either machine:
 ```bash
 tsng status
 ```
 ```text
 endpoint  7d76ccbbc21bf30767e14422c0494740a2cecc02aa9f82d5b8d57bdae350e7fc
 hostname  tsunagi-7d76ccbbc2
 bound     0.0.0.0:41641
 network lab (z2o4qwrvnj3zb6st2aoqg4abf342j662q2ujttqrsmz22argk2ba)  active
  peer b345d5271b  tsunagi-b345d5271b  Direct  rtt 24ms
  overlay tsunz2o4qwrvnj3 fd09:…:c1c6/64 and 100.110.49.177 mtu 1280  1/1 tunnel(s) up
    SDsEb/WF  fd09:…:c4b / 100.65.243.53  handshake 4s ago  tx 0 rx 0  Direct via Ip(…)
 ```
 `1/1 tunnel(s) up` and a recent handshake mean the tunnel is live. Then send
 real traffic to the peer's overlay address:
 ```bash
 ping6 fd09:…:c4b        # or
 ping  100.65.243.53
 ```
 `tx` and `rx` in the status should start moving.
 **Addresses are allocated and then remembered.** The default range is
 `10.13.37.0/24`; the first member to join settles it and later members adopt
 what they find, so `--ipv4-range` only matters for whoever starts the network:
 ```bash
 tsng up --ipv4-range 10.44.0.0/16
 tsng up --ipv4-range none   # no data plane at all
 ```
 One agent has one interface, so two of its networks cannot both use that
 range. The second takes the range **derived from its own network id**: not
 picked locally — every member derives the same one from something they all
 already have — so it is an agreement rather than a guess, and a device in
 several networks gets an address in each.
 An address is claimed with a record signed by that member's persistent device
 key, stored, and merged between every replica. A member that disappears for a
 month comes back to the same address, because the claim outlived the session.
 No vote is involved — see
 [docs/wireguard.md](docs/wireguard.md#ipv4-allocated-signed-and-kept) and
 [docs/sync-model.md](docs/sync-model.md).
 Because the address is allocated at run time rather than derived, it is not
 known until the agent has started and agreed with its peers. The agent then
 assigns it to the interface itself.
 ## Levels
 The command line is split the way the design is. A bare `tsng up` needs
 only a network name and a secret; everything else sits under the level it
 belongs to, which `--help` shows as two sections:
 * **System** — what the agent itself does: how it reaches peers (`--reach`),
  the one overlay interface it owns (`--interface`, `--mtu`, `--no-tun`), the
  address range (`--ipv4-range`), and the local resolver (`--no-dns` to disable).
 * **Transport** — which protocols carry packets (`--protocol`, a list) and
  their own settings (`-o key=value`, or `-o protocol:key=value`).
 ```bash
 tsng protocols          # what this build can carry packets with
 ```
 ```text
 wg-quic  (wire version 4)
        what                  WireGuard's cryptography carried in iroh's QUIC datagrams
        -o keepalive=SECONDS  keeps a tunnel and its link warm through a NAT
        -o mtu=BYTES          largest packet a tunnel will carry, at least 576
 ```
 Each protocol declares its own settings, so the agent can list them without
 knowing anything about the protocol, and a setting no selected protocol takes
 is refused rather than ignored. `--protocol none` runs the control plane by
 itself.
 A pair of peers uses a protocol they both have **at the same wire version**.
 That is not the software version: two peers on different builds carry traffic
 for each other for as long as the bytes between them have not changed. A peer
 with nothing in common keeps its control plane — messages and signed state
 still flow — and simply has no data plane, which `tsng status` shows as a
 session with no agreed protocol.
 ## Names
 DNS is enabled by default and serves a local zone for **every network this device is in**,
 each named after the network, so members can be reached by name instead of
 by address:
 ```bash
 tsng up               # DNS is already enabled
 dig @127.0.0.1 -p 5354 music.lab
 ```
 An explicit choice is remembered across restarts. `--no-dns` or `tsng dns off`
 disables it; `--dns` or `tsng dns on` enables it again. Existing saved opt-outs
 remain respected. The `dns on` and `dns off` commands also take effect immediately
 on a running agent. `--dns` and `--no-dns` cannot be combined.
 The zone of a network is its name — `--dns-zone` is gone, because with
 several networks there is no single zone to name. A network name may contain
 dots, so `--network lab.internal` is how you get `music.lab.internal`.
 Names come from signed state, which is the point: **a member that is
 switched off still resolves**, because its claim outlived the session.
 The answers are IPv4 addresses, because that is what the overlay is. The
 *questions* are taken on `127.0.0.1` and `[::1]`, over UDP and TCP, so a
 resolver reaches it over whichever family it uses; both are published to the
 system resolver together. Loopback and nothing else: the zones are a view
 for the host running the agent, and binding an overlay address would put
 them in front of the whole mesh — an agent in two networks would then answer
 one network's questions about the other's names.
 The zone is the network name, and it is yours to choose. A name that
 shadows a real public domain is reported and then used: a network called
 `ru` warns that every public `.ru` name becomes unreachable from this host,
 and then does it. Anything that collides with nothing — which is most names
 — is said nothing about, because a warning that fires on every private name
 anybody picks is how people learn to ignore warnings.
 On Linux the agent tells systemd-resolved to send questions for that suffix
 here, over D-Bus, scoped to the overlay interface and as a *routing* domain
 so it never becomes the resolver for anything else. resolved drops the whole
 setting when the interface goes, and the interface goes with the agent.
 Only an interface the agent created, though. Under `--no-tun` there is no
 host interface at all, and the agent says so and serves the zone on
 loopback rather than configuring whatever else on the host happens to share
 the name.
 That last step needs permission that `CAP_NET_ADMIN` does not give:
 systemd-resolved asks polkit, and polkit decides by **user**, not by
 capability, so there is no way for the agent to arrange it from inside. On
 a desktop the refusal reads `Interactive authentication required`.
 Running as a system service is enough. Otherwise the agent prints the rule
 that grants it — the four actions it calls and nothing else — ready to
 paste:
 ```bash
 sudo tee /etc/polkit-1/rules.d/50-tsunagi-resolved.rules > /dev/null <<'RULE'
 polkit.addRule(function(action, subject) {
    var allowed = [
        "org.freedesktop.resolve1.set-dns-servers",
        "org.freedesktop.resolve1.set-domains",
        "org.freedesktop.resolve1.set-default-route",
        "org.freedesktop.resolve1.revert"
    ];
    if (allowed.indexOf(action.id) >= 0 && subject.user == "YOUR-USER") {
        return polkit.Result.YES;
    }
 });
 RULE
 ```
 **Without it the server still runs** — `tsng status` prints where it is
 listening and the exact `dig` line — so the automatic part is missing, not
 the feature. The refusal is said once rather than on every pass, and retried
 slowly, because nothing but a person will change it.
 The server is authoritative for its zone and nothing else. No recursion, no
 forwarding, no cache: pointing a resolver at it can never make it a route to
 the outside.
 ## Privileges
 On Linux the agent **manages its own overlay interface**. It creates the TUN
 interface, sets the MTU, brings it up and assigns both overlay addresses, all
 over netlink in process — no `ip` invocation, no shell, nothing that a
 remote peer could influence.
 That needs `CAP_NET_ADMIN`, granted once:
 ```bash
 sudo setcap cap_net_admin+p /usr/local/bin/tsng
 ```
 `+p` rather than `+ep`: the capability is then *permitted* but not
 *effective*, and the agent raises it only around the handful of netlink calls
 that need it — a few milliseconds at startup, and again if its address
 allocation changes. Everything else, including every byte from the network,
 is handled with it lowered. `+ep` works too; the agent lowers it on the way
 in.
 `tsng status` says which of these applies on the host it runs on, along
 with what the agent is doing. It grades each finding: **ok** for what works,
 **warn** for what the agent runs without and you can fix from the line it
 prints, **FAIL** for what it cannot work around. The words carry the grade as
 well as the colour, so the report reads the same piped to a file or on a
 terminal without colour, and it honours `NO_COLOR`.
 Members are listed online first, then the ones that are away. A member that
 is away is reported plainly rather than flagged: in a mesh of laptops it is
 the ordinary condition, not a fault. The signed state is what makes that
 sayable — it remembers who belongs while they are gone, so the report can say
 "offline, 10.13.37.99 still reserved for it" instead of leaving a
 dial-failure counter to imply it. Counters are history and are never graded:
 a peer that left and came back should not leave the report looking broken.
 The other two commands split along the line the system itself draws. `id` is
 this **device**: the key it signs with and the name it answers to. `network`
 is what it **belongs to**: which networks, their secrets, joining and
 leaving. A device outlives every network it is in, and a network outlives
 any device in it, so a command that mixed them had to be read twice.
 Every item takes the same shape, so there is nothing to remember: name it to
 see it, name it with a value to change it.
 ```
 tsng id                      everything about this device
 tsng id hostname             the name it answers to
 tsng id hostname mango       change it
 tsng id key                  the key it signs with
 tsng id key rotate           replace that key
 tsng network                 the networks this device belongs to
 tsng network join -n lab -s tsn1…   join one; adds it to a running agent
 tsng network join -n lab     resume one this device has, or make it on the spot
 tsng network stop lab        stop serving it, keeping everything
 tsng network start lab       serve it again, from where it left off
 tsng network leave lab       give up the address and name, then forget it
 tsng network secret          the secret of each joined network
 tsng network secret lab      just that one, for copying
 tsng network secret generate a fresh secret for a network that does not exist yet
 tsng dns                     whether the local resolver is serving, and what
 tsng dns on                  start it, now and after every restart
 tsng dns off                 stop it, now and after every restart
 ```
 **A network without a secret makes one.** `tsng join --network lab`
 resolves a bare name in the obvious way: if this device is already in
 exactly one network called `lab`, that one — so the name alone resumes what
 you have; if it is in none, a fresh random secret, printed in full with the
 line to send the other machine:
 ```
 joined `lab` (k2on43wadbi5x267vp6z3ogkm7nbjedfdoxyauxhtxhwqrprylba)
  secret  tsn1u7c…
-Run this on the other machine:
+server$ tsng join -n home -s tsn1u7c…
 server$ tsng id hostname server
-  tsng join --network lab --secret tsn1u7c…
+laptop$ ssh server.home
 Its agent has to be running. If it is not:
  tsng up --peer 91e83a6e2b7a…
 ```
-Two networks of one name is a thing that happens — a mistyped secret makes
+No account, no coordination server, no admin console, no keys to approve.
-one — so joining says whether the network was already here, and warns when
+Anyone who knows the **name** and the **secret** is in. Nobody else can find
-another configured network answers to the same name. A name is a label and
+the network, let alone enter it.
 the id is the identity.
-That is the ad-hoc case: one person makes a network and sends the command
+## Why
 round. The secret is printed *only* when the agent invented it — there is
 nowhere else to read it from — and never when it was supplied, because then
 it is already yours. Two networks of one name and no secret is the one case
 with no answer, and it says so instead of choosing.
-A secret is printed by `network secret` and nowhere else — not by `id`, not
+Tailscale, Headscale and ZeroTier are great, but each needs a control server:
-by `status`, not in a log, a `Debug` rendering or anything sent to a peer.
+theirs, or one you have to run and keep alive. tsunagi doesn't. Machines find
-Asking for it is deliberate, because these reports get pasted into chats.
+each other through the public BitTorrent DHT, connect directly (hole punching,
 with relay fallback), and keep the network state among themselves.
-**`up` takes no network at all** if you would rather decide later: it
+|                        | Tailscale          | Headscale        | ZeroTier            | **tsunagi**         |
-brings up the agent and whatever it is already configured for, and waits.
+| ---------------------- | ------------------ | ---------------- | ------------------- | ------------------- |
-That is the shape of a daemon in one terminal and `tsng network join`
+| Control server         | hosted by vendor   | you run it       | hosted or self-run  | **none**            |
-in another.
+| Account / sign-in      | yes                | yes              | yes                 | **no**              |
 | To join a network      | sign in, get approved | pre-auth key  | network ID, approval | **name + secret**  |
 | Direct connections     | yes                | yes              | yes                 | yes                 |
 | Exit nodes             | yes                | yes              | via routes          | yes                 |
-`network join` is also the answer to a question `up` cannot: a state
+## Features
 directory belongs to one live agent, so a second `tsng up` cannot add a
 network to the one already running. This adds it over the control socket and
 it starts at once. With no agent running it is written to the configuration
 and starts with the next `up`.
-The name is part of the signed state, so changing it revokes the previous
+- **Two things to know:** a network name and a secret. A secret is generated for you when you make a network.
-one: there is one record per author, a new version replaces the whole claim,
+- **Direct, encrypted links** between members (WireGuard cryptography), with relay fallback behind strict NATs.
-and no replica can keep the old name standing. Changing it while the agent
+- **Names for machines:** `server.home` just works, even while `server` is switched off. A name belongs to whoever claimed it first; a later device with the same name keeps working by address and is told so.
-runs goes through the agent, which republishes and tells its peers straight
+- **Exit nodes:** send all your internet traffic through any member that offers one.
-away.
+- **Multi-hop:** members that can't reach each other directly are routed through a member that can, over whatever kind of link each hop has. A member reached that way is online, and shown as relayed; traffic moves to a direct link by itself the moment one comes up.
 - **LAN games:** UDP broadcasts are relayed, so "LAN" lobbies show up across the mesh.
 - **Several networks at once** on one device.
 - **Tray app** for Windows, macOS and Linux; a command line for everything.
-Replacing the signing key makes this device a different member, and it loses
+## Install
 the address and name the old key held — nothing can sign on a retired key's
 behalf, and by design there is no authority that could overrule an author. So
 the outgoing key signs a release for every network on its way out, which
 frees them for whoever wants them next, and the whole thing commits at once.
-`status` and `id` both prefer a running agent, which is live and
+Grab a build from the [latest release](https://github.com/house-of-vanity/tsunagi/releases/latest).
 authoritative, and fall back to reading the state store when there is none.
 Reading takes no directory lock, so neither has to wait for the agent it is
 asking about — nor does either need one to be running.
-### It cleans up after itself
+| Platform              | How                                                                              |
 | --------------------- | -------------------------------------------------------------------------------- |
 | **Windows**           | Run `tsunagi-setup-<version>-x86_64.exe`. Installs the service, the tray and `tsng`. |
 | **macOS**             | `brew tap house-of-vanity/tap && brew install --cask tsunagi-gui`<br>CLI only: `brew install tsunagi` |
 | **Debian / Ubuntu**   | `sudo apt install ./tsunagi-gui_<version>_amd64.deb` (`tsunagi_…` for no tray)   |
 | **Arch**              | `sudo pacman -U tsunagi-gui-<version>-1-x86_64.pkg.tar.zst`                      |
 | **NixOS**             | [Flake and module](#nixos) below                                                 |
 | **Any other Linux**   | Unpack the tarball, then `sudo ./install.sh && sudo systemctl enable --now tsunagi` |
-The interface is tied to an open file descriptor and is deliberately **not**
+Linux packages run the agent as an unprivileged service user. Add yourself to
-made persistent, so the kernel removes it when the agent exits — on a clean
+the `tsunagi` group (`sudo usermod -aG tsunagi $USER`, then log in again) to
-shutdown, on a panic, on `SIGKILL`, on power loss alike. Keeping it is what
+use `tsng` and the tray without `sudo`.
 would take an action; removing it is the default.
-If something is left behind anyway — an interface made by an older version's
+## Quick start
 manual recipe, or one from a run killed in the instant between creating it and
 recording it — the next start **replaces it**, along with any stale addresses
 it carried. Two things are never touched:
-* an interface that is not a TUN, because the name colliding with somebody's
+```sh
-  bridge is not a reason to destroy the bridge;
+tsng join -n home                 # first machine: makes the network and prints a secret
-* a TUN that another process is holding open, because that is a working
+tsng join -n home -s tsn1…        # every other machine
-  overlay belonging to somebody else — most likely a second agent on this
+tsng status                       # who is online, over which path
  host, which should be given a different `--wg-prefix`.
 Both of those refuse with an explanation rather than guessing.
 Two settings the manual recipe used to need are gone with it.
 `keep_addr_on_down` existed only because an interface nobody held open lost
 carrier and had its IPv6 addresses flushed, and `nodad` only because duplicate
 address detection can never finish without carrier. An interface held open for
 its whole life has carrier for its whole life.
 ### The MTU is 1280
 The default interface MTU stays at 1280 on direct paths, relays and paths
 inside another VPN. Large encrypted packets are split into smaller QUIC
 datagrams and reassembled before reaching WireGuard. SSH and full-size TCP
 segments therefore do not require a manual MTU override or MSS adjustment.
 The original IP packet, including its DF bit, is preserved. See
 [docs/wireguard.md](docs/wireguard.md#mtu) for limits and compatibility.
 ### Summary
 | approach | agent runs as | notes |
 |---|---|---|
 | `setcap cap_net_admin+p` | ordinary user, one capability | recommended: nothing to prepare, nothing left behind. Lost on every rebuild or copy of the binary. |
 | systemd service | `User=`, `AmbientCapabilities=CAP_NET_ADMIN` | the same, for an installed service |
 | `sudo tsng up` | root | everything works, nothing is isolated |
 | macOS root LaunchDaemon | root | the native macOS deployment; see below |
 | `--no-tun` | ordinary user, no capabilities | tunnels run and handshake, traffic never reaches the OS |
 On Windows, put `wintun.dll` beside the executable and start `tsng up`
 from PowerShell or Command Prompt opened with **Run as administrator**.
 Run commands controlling that agent (`join`, `status`, `dns`, `network`)
 as the same Windows user with the same elevation. If Windows denies access
 to the agent's control pipe, the command reports a permission error instead
 of claiming the agent is absent or trying to edit its locked state.
 Creating a TUN without sufficient privileges also explains how to restart
 the agent with the required permissions.
 On macOS the agent creates a `utun` interface, configures it with `ifconfig`
 and `route`, and points the system resolver at the overlay by writing
 `/etc/resolver/<network>` files (which carry a port, so the DNS server needs no
 privileged socket). All of this needs **root**: macOS has no per-capability
 grant like Linux, and a `utun` plus `/etc/resolver` are root-only. The kernel
 assigns the interface its `utunN` name, so the name derived from the network id
 is only a request and the real one is adopted once the interface is up.
 Run it as root — `sudo tsng up` for a quick start, or install it as a root
 **LaunchDaemon** for the native always-on deployment. As a daemon the agent runs
 unprivileged commands (`join`, `status`, `dns`, `network`) over its control
 socket like any other platform; the socket lives at `/var/run/tsunagi/agent.sock`
 and the stores at `/var/db/tsunagi`, both fixed system paths so they do not
 depend on whose shell started `sudo`. A ready-to-edit plist and install notes
 are in [dist/macos/](dist/macos/). The broadcast route is installed; the
 inbound-UDP firewall allowance (a `pf` anchor) is not yet, so `status` reports
 that half as incomplete and names what to allow by hand. The decision logic that
 says *what* to change is shared and tested on every platform.
 ## Checks
 ```bash
 cargo fmt --all -- --check
 cargo clippy --locked --workspace --all-targets -- -D warnings
 cargo test --locked --workspace --all-targets
 ```
-The whole suite runs offline on loopback. Set `TSUNAGI_TEST_LOG=tsunagi=debug`
+Every member gets an address in `10.13.37.0/24` and a name like `<host>.home`.
-to see agent logs while a test runs.
+Machines are called `tsunagi-<id>` until you rename them: `tsng id hostname laptop`.
-There are also two runnable demos, which are demos and not substitutes for the
+### Exit nodes
 tests:
-```bash
+```sh
-cargo run --example two_agents       # control plane only
+tsng network exit-node home on    # on the machine that should be the exit
 tsng exit-node                    # on another one: who offers an exit
 tsng exit-node laptop             # send everything through it
 tsng exit-node off
 ```
-Both run with no privileges and change nothing on the host.
+The tray does the same with a checkbox. An exit node sees everything you send
 through it, so use only members you trust.
 [Details and per-OS notes](docs/exit-node.md).
-## Usage
+## Platforms
-```rust,no_run
+| Platform                  | Build        | CLI `tsng` | Tray | Exit node: offer / use | Packages                            |
-use std::sync::Arc;
+| ------------------------- | ------------ | :--------: | :--: | :--------------------: | ----------------------------------- |
-use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
+| Linux x86_64              | release      |     ✔      |  ✔   |        ✔ / ✔           | `.deb`, `.pkg.tar.zst`, tarball, Nix |
-use tsunagi::discovery::SharedMemoryDiscovery;
+| Linux aarch64             | release      |     ✔      |  –¹  |        ✔ / ✔           | `.deb`, `.pkg.tar.zst`, tarball, Nix |
-use tsunagi::identity::{NetworkName, NetworkSecret};
+| Linux armv7               | release      |     ✔      |  –¹  |        ✔ / ✔           | `.deb`, `.pkg.tar.zst`, tarball      |
-use tsunagi::proto::ControlMessage;
+| macOS, Apple silicon      | release      |     ✔      |  ✔   |        ✔ / ✔²          | Homebrew, tarball                    |
-use tsunagi::{Agent, Result};
+| macOS, Intel              | from source  |     ✔³     |  ✔³  |        ✔ / ✔²³         | –                                    |
 | Windows x86_64            | release      |     ✔      |  ✔   |        ✔ / ✔²          | installer, zip                       |
-// The library never starts a runtime, installs a logger, handles signals,
+<sup>¹ Not in the release; builds from source (and through Nix).
-// forks, or calls process::exit. The binary owns all of that.
+² Newer and less battle-tested than on Linux.
-#[tokio::main]
+³ Builds from source, not covered by CI.</sup>
 async fn main() -> Result<()> {
    let config = AgentConfig::new(StoragePaths::user_default()?)
        .with_transport(TransportPolicy::N0Defaults)
        .with_discovery(Arc::new(SharedMemoryDiscovery::new()));
-    let agent = Agent::spawn(config).await?;
+The Linux CLI is a single static binary: it runs on any distribution.
-    let name = NetworkName::new("kitchen-table")?;
+## NixOS
    let secret = NetworkSecret::generate();       // 32 random bytes
    println!("share this: {}", secret.encode().as_str());
-    let network = agent.join_network(&name, &secret).await?;
+The repository is a flake with a package and a module.
-    let mut events = agent.subscribe();
+```nix
-    tokio::spawn(async move {
+# flake.nix
-        while let Ok(event) = events.recv().await {
+{
-            println!("{event:?}");
+  inputs.tsunagi.url = "github:house-of-vanity/tsunagi";
  outputs = { nixpkgs, tsunagi, ... }: {
    nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
      modules = [
        tsunagi.nixosModules.default
        {
          services.tsunagi = {
            enable = true;
            tray.enable = true;      # optional, for desktops
            users = [ "alice" ];     # may use tsng and the tray without sudo
          };
        }
-    });
+      ];
-
+    };
-    for peer in agent.network_status(network).await?.connected_peers() {
+  };
        agent
            .send(network, peer, ControlMessage::Ping { seq: 1, payload: vec![] })
            .await?;
    }
    agent.shutdown().await;
    Ok(())
 }
 ```
-`TransportPolicy::LocalOnly` is the default, so a plain `AgentConfig::new` never
+Just trying it: `nix run github:house-of-vanity/tsunagi#tsunagi-cli -- --help`.
-reaches the internet by accident. Opt into `DirectOnly` or `N0Defaults`
+`packages.<system>.tsunagi` (with the tray) and `tsunagi-cli` are also exported,
-explicitly.
+along with `overlays.default`.
-### Through somebody in the middle
+## Build from source
-Tsunagi routes the overlay in userspace. A direct data link always wins;
+Rust 1.91 or newer.
 otherwise the shortest available path can pass through several members.
 Each authenticated member advertises only its own live, protocol-specific
 transport links. The control plane collects these into a graph and replaces
 the routing table when connectivity changes; announcements expire after
 90 seconds without refresh.
 Transit goes transport → router → transport, without entering WireGuard or
 the intermediate host's TUN. The payload stays encrypted between the original
 endpoints. TUN connects only the local OS to Tsunagi; the kernel needs only
 the overlay route through that interface, with no IP forwarding configuration.
 A 16-hop limit bounds loops while topology updates converge. Equal shortest
 paths use a stable flow hash derived before encryption, so packets of a TCP
 or UDP flow keep their path while topology is unchanged. Physical link changes
 do not replace end-to-end WireGuard tunnels or change the 1280-byte default MTU.
 `status` reports the hop count and first next hop for relayed paths.
 The hot path reads an immutable routing snapshot and sends directly to a cached
 transport handle. It takes no routing mutex, walks no graph and does not parse
 the encrypted payload. See [routing.md](docs/routing.md) for the architecture,
 limits and reproducible forwarding microbenchmark.
 This wire format requires all members to upgrade together (control ALPN 3,
 data ALPN 4); saved identities, network names, secrets and addresses survive.
 ### LAN game discovery
 IPv4 UDP broadcast relay is enabled by default for each network. The IP router
 sends one encrypted copy to each participating peer, including through multihop
 paths. It supports `255.255.255.255` and the overlay subnet's broadcast address;
 received broadcasts never trigger another fanout.
 ```sh
-tsng join -n games --no-broadcast
+cargo build --release --bin tsng               # the agent and CLI
-tsng join -n games --broadcast
+cargo build --release --bin tsunagi-tray       # the tray (Linux needs GTK 3 and libayatana-appindicator)
 tsng network broadcast games off
 tsng network broadcast games on
 ```
-The choice persists across restarts and a plain `join`. It can be changed while
+Windows needs [Wintun](https://www.wintun.net) next to `tsng.exe`.
-the agent runs; `status` shows it for each network. While broadcast is on, the
+Per-platform notes: [Linux](dist/linux/README.md), [Arch](dist/arch/README.md),
-agent routes `255.255.255.255` through the Tsunagi interface and allows inbound
+[macOS](dist/macos/README.md), [Windows](dist/windows/README.md).
 UDP from the overlay range on it (Linux and Windows, needs the same privilege as
 the interface), and removes both when it is turned off or the agent stops. Games
 bound to a physical address still bypass it. Physical LAN capture/subnet sharing
 is not implemented.
 See [broadcast.md](docs/broadcast.md) for domain isolation and future LAN gateways.
-## How peers find each other
+## How it works
-Two different lookups are involved, and only one of them is this project's:
+- Your name and secret derive a network identity. Members publish signed
  contact records to the public Mainline DHT under it, and find each other there.
 - Every connection proves knowledge of the secret before it's accepted.
  Traffic between members is end-to-end encrypted, relays included.
 - Addresses and names are claimed with each device's own key and replicated
  between members. There is no one in charge, so nothing to be taken down.
-**1. Resolving one endpoint's address — iroh's, and it works today.**
+The secret is the only key, so make it a good one (the generated ones are).
-With `--reach relay` or `--reach direct`, iroh publishes a signed
+The [threat model](docs/threat-model.md) says what this does and doesn't protect against.
 record of this endpoint's addresses, keyed by its endpoint id, to the public
 service run by Number 0 — "n0", the company behind iroh — at `dns.iroh.link`,
 over pkarr and DNS, and resolves other endpoints the same way. That is why `--peer <endpoint-id>` works with no address attached:
 iroh looks it up. None of that code is ours.
 **2. Finding who is in a network — Mainline DHT by default.**
 `NetworkDiscovery` supplies unverified candidates. The Mainline backend uses
 a signing key derived from the network name and secret to publish signed
 BEP44 records containing each writer's endpoint ID and addresses. Its 16
 slots provide starting candidates, not a complete membership list or a limit
 on network size. After the first authenticated connection, introductions and
 signed state reveal the other members. `StaticBootstrap` (`--peer`) remains
 available, and tests use either a local Mainline Testnet or in-memory discovery.
 What this means in practice:
 - With `relay` or `direct`, **your endpoint id and IP addresses are published
  to a public third-party service** (Number 0's, unless you change it). They are not secret, and the network secret is
  never published, but an observer of that service learns that your endpoint
  exists and where it is. `--reach local` publishes nothing.
 - With DHT enabled, endpoint records are also published to the public Mainline
  DHT. They are signed, not encrypted; membership still requires the secret.
 - A relay, when one is needed, sees the volume and timing of your traffic — not
  its contents. The default relays are Number 0's, in the US, EU and
  Asia-Pacific.
 ## Storage
 Two physically separate SQLite files, placed wherever the library's
 configuration says (`StoragePaths`). A future system service supplies its own
 paths; tests always use temporary directories.
 | file           | holds                                            | when damaged            |
 |----------------|--------------------------------------------------|-------------------------|
 | `state.sqlite` | device identity, network configuration, hostname | clear error, never reset |
 | `cache.sqlite` | address hints and other recoverable data          | discarded and recreated  |
 The `wg-quic` protocol keeps its own keys in its own store under `wg-quic/`,
 because a protocol's keys are neither the iroh identity nor the network
 secret.
 The command line agent puts everything under the platform's per-user
 directories by default; `--state-dir` and `--cache-dir` override them.
 One state directory belongs to one live agent, enforced with a real OS file
 lock rather than an existence check. That directory *is* the identity: one
 agent, one device key, one interface, and as many networks as you like on it
 — `tsng network join` adds them to the agent that is already running,
 which is why a second `tsng up` on the same directory is refused rather
 than made to work. A second agent on the same host is a second identity, and
 needs everything of its own: its own state and cache directories, its own
 interface name and its own overlay range. The library owns no globals, so
 several of them run side by side in one process as readily as in one host.
 ### Leaving, and starting over
 Membership outlives a session, so it also has to be possible to end it.
 ```bash
 tsng network                     # what this device belongs to
 tsng network leave lab           # give up the address and the name, then forget it
 tsng wipe --yes                  # remove everything and be a stranger again
 ```
 **Stopping is not leaving.** `stop` closes this network's sessions, takes
 its address off the interface and keeps it from starting again, and that is
 all: the configuration, the secret, the signed state and the protocol key
 stay exactly as they are, nothing is announced, and to the others this
 device is simply away — an ordinary condition they already handle, with its
 address and name still reserved for it. `start` picks it up where it left
 off. A network named on the `up` command line is started by that command
 whatever its stored state, and the start-up banner says so rather than
 letting a `stop` quietly come back.
 `leave` is the other one, and it removes, locally: the configuration and
 the secret, this network's signed records, its cached address hints and the
 protocol key it used there. What it keeps is this author's version counter
 for that network — a rejoin has to continue above the release, or every
 replica would treat the new claim as stale. Everything about *other*
 networks, and the device identity itself, is untouched.
 `leave` publishes a signed `Release` **first**, while the agent is running and
 its sessions are up, so the address and the name are freed for the others
 instead of staying reserved to a member that has gone. They pass the tombstone
 on, so a member that was away hears it from them. With no agent running,
 nothing can sign or send it: the command says so and refuses, and `--offline`
 drops the network locally while leaving the others holding the old claim. The
 protocol key for that network goes too — rejoining is joining, not resuming.
 `leave`, `stop`, `start`, `secret` and `broadcast` accept an exact stored network
 name, its full id, or a unique id prefix (including a copied report ellipsis).
 Names are case-sensitive. If several networks share a name, or a name matches
 another network's id prefix, the command lists matching names and full ids and
 changes nothing. Use a full id to choose explicitly; a full id always takes
 precedence over a name. No secret is needed to select a configured network.
 `wipe` removes both directories' contents: the device identity, every network,
 every signed record and everything a protocol kept beside them. It refuses
 while an agent is running, and refuses a directory with no `state.sqlite` in
 it, so a mistyped `--state-dir` cannot take somebody's documents with it.
 Without `--yes` it only says what it would remove. It is not a goodbye: nobody
 is told, because after it there is no key left to sign anything with. Leave the
 networks first if the addresses should be freed.
 ## Documentation
- [docs/architecture.md](docs/architecture.md) — module boundaries and runtime.
+[Architecture](docs/architecture.md) ·
- [docs/wireguard.md](docs/wireguard.md) — the WireGuard plugin: overlay
+[Protocol](docs/protocol.md) ·
-  addressing, announcements, backends, reconciliation.
+[WireGuard data plane](docs/wireguard.md) ·
- [docs/protocol.md](docs/protocol.md) — identity derivation, framing, handshake.
+[Routing](docs/routing.md) ·
- [docs/sync-model.md](docs/sync-model.md) — the planned signed-state model and
+[Exit nodes](docs/exit-node.md) ·
-  what is deliberately not built yet.
+[Threat model](docs/threat-model.md) ·
- [docs/exit-node.md](docs/exit-node.md) — exit nodes: sending all traffic through a member.
+[Contributing](AGENTS.md)
 - [docs/threat-model.md](docs/threat-model.md) — threat model and known limits.
 - [docs/testing.md](docs/testing.md) — what the suite covers and what it does not.
 - [AGENTS.md](AGENTS.md) — rules for anyone (human or agent) changing this repo.
 ## Security in one paragraph
 Membership is proved by an HMAC over a transcript keyed by a value derived from
 the shared secret, bound to the specific iroh connection through the TLS
 exporter, to the network id, to both endpoint identities and to distinct role
 labels. This targets high-entropy secrets: there is no PAKE here, so a short
 human passphrase is guessable offline by anyone who can reach the handshake.
 Anyone who knows the secret is a full participant and can create many
 identities. Read [docs/threat-model.md](docs/threat-model.md) before relying on
 any of this.
 ## Licence
@@ -129,6 +129,19 @@ pub enum Event {
        /// Why it is not up.
        reason: String,
    },
    /// The name this agent claimed was claimed first by another member.
    ///
    /// Nothing stops working: this agent keeps its address and every
    /// connection. It just is not found by that name until it is renamed or
    /// the member that holds the name lets it go.
    HostnameTaken {
        /// The network.
        network: NetworkId,
        /// The name that is taken.
        name: String,
        /// The member that holds it.
        holder: EndpointId,
    },
    /// An IP plugin reported an error. Never fatal.
    PluginError {
        /// The network the call was scoped to.
@@ -26,8 +26,8 @@ mod status;
 pub use events::Event;
 pub use status::{
-    AgentStatus, CandidateStatus, MemberStatus, NetworkMetrics, NetworkState, NetworkStatus,
+    AgentStatus, CandidateStatus, DataPath, HostnameConflict, MemberStatus, NetworkMetrics,
-    OverlayStatus, PeerStatus,
+    NetworkState, NetworkStatus, OverlayStatus, PeerStatus,
 };
 use std::collections::HashMap;
@@ -800,6 +800,11 @@ impl Agent {
        // itself stays, because the agent owns it and other networks may
        // still be using it.
        self.inner.routes.remove_network(network_id);
        if let Some(interface) = self.inner.interface.get() {
            interface
                .set_underlay(network_id, Vec::new(), Vec::new())
                .await;
        }
        if let Some(interface) = self.inner.interface.get()
            && let Err(err) = interface.sync_addresses().await
        {
@@ -976,6 +981,7 @@ impl Agent {
                // An inactive network has no runtime to ask; the roster and
                // the range come from one. Empty, not invented.
                members: Vec::new(),
                hostname_conflict: None,
                range: None,
                range_conflict: None,
                metrics: NetworkMetrics::default(),
@@ -32,7 +32,8 @@ use super::events::Event;
 use super::session::{self, Session, SessionEvent};
 use super::shutdown::Shutdown;
 use super::status::{
-    CandidateStatus, MemberStatus, NetworkMetrics, NetworkState, NetworkStatus, PeerStatus,
+    CandidateStatus, DataPath, HostnameConflict, MemberStatus, NetworkMetrics, NetworkState,
    NetworkStatus, PeerStatus,
 };
 /// An inbound connection that already passed the handshake.
@@ -320,6 +321,10 @@ struct Runtime {
            std::time::Instant,
        ),
    >,
    /// The protocols this agent last announced, in the order it announced
    /// them. The order is its preference, and the lower of two ids settles
    /// which protocol a pair runs.
    local_capabilities: Vec<PluginCapability>,
    /// What this agent last told its peers it could reach, so it is only
    /// said again when it changed.
    announced_reach: HashSet<crate::proto::message::ReachableLink>,
@@ -338,6 +343,9 @@ struct Runtime {
    /// publish a claim afterwards — the periodic check, a peer's records
    /// arriving — would silently take it back.
    released: bool,
    /// The name this agent last said was taken, so it is said once per
    /// change rather than on every round.
    reported_name_taken: Option<HostnameConflict>,
    /// The address last reported as absent from the interface, so it is said
    /// once rather than for ever.
    reported_missing: Option<std::net::Ipv4Addr>,
@@ -366,6 +374,16 @@ impl Runtime {
        let (link_results_tx, link_results_rx) = mpsc::channel(64);
        let (candidate_tx, discovery_candidates) =
            mpsc::channel(params.limits.max_discovery_candidates.max(1));
        let hub = crate::dataplane::relay::RelayHub::new(network_id, local_id);
        // The order the protocols were configured in is also how links are
        // preferred when a neighbour can be reached over more than one.
        hub.set_priority(
            params
                .plugins
                .iter()
                .map(|plugin| plugin.protocol_id().to_string())
                .collect(),
        );
        let discovery_worker = params.discovery.as_ref().map(|backend| {
            DiscoveryWorker::spawn(
                backend.clone(),
@@ -393,8 +411,9 @@ impl Runtime {
            dial_results_tx,
            dial_results_rx,
            links: HashMap::new(),
-            hub: crate::dataplane::relay::RelayHub::new(network_id, local_id),
+            hub,
            reachable: HashMap::new(),
            local_capabilities: Vec::new(),
            announced_reach: HashSet::new(),
            opening: HashSet::new(),
            link_results_tx,
@@ -403,6 +422,7 @@ impl Runtime {
            reported_mismatch: HashSet::new(),
            released: false,
            reported_missing: None,
            reported_name_taken: None,
            state: StateSet::new(),
            pending_state: Vec::new(),
            own_version: 0,
@@ -617,6 +637,8 @@ impl Runtime {
                tracing::debug!(%err, "could not queue re-announcement");
            }
        }
        // What was announced decides which protocol each pair runs.
        self.update_paths();
    }
    /// Queues a message without blocking the runtime loop.
@@ -669,6 +691,7 @@ impl Runtime {
        // alone, so something has to look again; this runs on a timer and
        // the check is a comparison when nothing has changed.
        self.ensure_own_claim().await;
        self.report_hostname_taken();
        // The same timer repairs the relay picture: reachability is soft
        // state, repeated while it holds and forgotten when it stops.
        self.update_paths();
@@ -712,6 +735,34 @@ impl Runtime {
        self.start_dials();
        // Last, so it carries what this round learned.
        self.introduce_peers();
        self.refresh_underlay().await;
    }
    /// Hands the interface the underlay destinations this network depends
    /// on: the verified direct IPv4 paths of every control connection, and
    /// the relays in use. Only hosts that cannot exempt the agent from an
    /// exit node's default route read it.
    async fn refresh_underlay(&self) {
        let Some(interface) = self.params.interface.clone() else {
            return;
        };
        let mut addresses = Vec::new();
        let mut hosts = Vec::new();
        for session in self.sessions.values() {
            for path in snapshot_connection(&session.conn).paths {
                match path.remote {
                    PathAddr::Ip(std::net::SocketAddr::V4(socket)) => addresses.push(*socket.ip()),
                    PathAddr::Relay(url) => hosts.extend(relay_host(&url)),
                    _ => {}
                }
            }
        }
        for url in self.params.adapter.snapshot().relay_urls {
            hosts.extend(relay_host(&url));
        }
        interface
            .set_underlay(self.network_id, addresses, hosts)
            .await;
    }
    fn add_candidate(&mut self, candidate: Candidate) {
@@ -1003,12 +1054,25 @@ impl Runtime {
            }
        };
        // A name keeps the time it was first claimed for as long as this
        // member keeps the name, so a later claimant cannot overtake it by
        // the passage of time, and a new or changed name starts the clock
        // afresh.
        let hostname_since = wanted_hostname.as_ref().map(|name| {
            let kept = self
                .state
                .get(&self.local_id)
                .filter(|record| record.body.hostname() == Some(name.as_str()))
                .and_then(|record| record.body.hostname_since());
            kept.unwrap_or_else(unix_millis)
        });
        let body = RecordBody::Claim {
            address: wanted_address,
            // The range only travels with an address, so a member of an
            // IPv6-only network does not assert one.
            range: wanted_address.and(range),
            hostname: wanted_hostname,
            hostname_since,
        };
        // Nothing to say is not the same as saying nothing changed: a member
@@ -1072,6 +1136,7 @@ impl Runtime {
        for (peer, records) in std::mem::take(&mut self.pending_state) {
            self.receive_state(peer, records).await;
        }
        self.report_hostname_taken();
    }
    /// Sends everything we know to every peer.
@@ -1630,68 +1695,93 @@ impl Runtime {
        }
    }
    /// The protocol this agent and a member run their tunnel in, if they have
    /// one in common.
    ///
    /// Both ends must name the same one without asking each other, and what
    /// each side was configured to prefer can differ. So the member with the
    /// lower id decides, by the order it announces its protocols in, and the
    /// other side reads that order from the announcement it already has.
    fn tunnel_protocol(&self, session: &Session) -> Option<String> {
        let usable = |ours: &[PluginCapability], theirs: &[PluginCapability], name: &str| {
            ours.iter().any(|capability| {
                capability.enabled
                    && capability.protocol == name
                    && theirs.iter().any(|other| {
                        other.enabled
                            && other.protocol == name
                            && other.version == capability.version
                    })
            })
        };
        let (decides, other) = if self.local_id.as_bytes() < session.peer.as_bytes() {
            (&self.local_capabilities, &session.capabilities)
        } else {
            (&session.capabilities, &self.local_capabilities)
        };
        decides
            .iter()
            .find(|capability| usable(decides, other, &capability.protocol))
            .map(|capability| capability.protocol.clone())
    }
    /// Builds the graph off the packet path. Every remote row comes from that
-    /// member's authenticated control session; stale or incompatible rows never
+    /// member's authenticated control session, and a row says only who that
-    /// enter the graph. The hub replaces an immutable table only on changes.
+    /// member has a link to, never what kind of link: a member in the middle
    /// needs no protocol in common with either end. Stale rows never enter the
    /// graph. The hub replaces an immutable table only on changes.
    fn update_paths(&mut self) {
        self.reachable.retain(|peer, (_, heard)| {
            heard.elapsed() < REACH_EXPIRY && self.sessions.contains_key(peer)
        });
-        for (protocol, version) in self.served_protocols() {
+        let mut members: HashSet<_> = self.sessions.keys().map(|peer| *peer.as_bytes()).collect();
-            let peers: Vec<_> = self
+        members.insert(*self.local_id.as_bytes());
-                .sessions
+        let graph = self
-                .values()
+            .reachable
-                .filter(|session| {
+            .iter()
-                    session.capabilities.iter().any(|capability| {
+            .filter(|(peer, _)| members.contains(peer.as_bytes()))
-                        capability.enabled
+            .map(|(peer, (links, _))| {
-                            && capability.protocol == protocol
+                let mut neighbors: Vec<_> = links
                            && capability.version == version
                    })
                })
                .map(|session| session.peer)
                .collect();
            let mut members: HashSet<_> = peers.iter().map(|peer| *peer.as_bytes()).collect();
            members.insert(*self.local_id.as_bytes());
            let graph = self
                .reachable
                .iter()
                .filter(|(peer, _)| members.contains(peer.as_bytes()))
                .map(|(peer, (links, _))| {
                    let mut neighbors: Vec<_> = links
                        .iter()
                        .filter(|link| {
                            link.protocol == protocol
                                && members.contains(&link.peer)
                                && link.peer != *peer.as_bytes()
                        })
                        .map(|link| link.peer)
                        .collect();
                    neighbors.sort_unstable();
                    (*peer.as_bytes(), neighbors)
                })
                .collect();
            self.hub.set_topology(&protocol, graph, members);
            for peer in peers {
                if self.hub.has_link(peer, &protocol) || !self.hub.reachable(peer, &protocol) {
                    continue;
                }
                let Some(plugin) = self
                    .params
                    .plugins
                    .iter()
-                    .find(|plugin| plugin.protocol_id() == protocol)
+                    .filter(|link| members.contains(&link.peer) && link.peer != *peer.as_bytes())
-                    .cloned()
+                    .map(|link| link.peer)
-                else {
+                    .collect();
-                    continue;
+                neighbors.sort_unstable();
-                };
+                neighbors.dedup();
-                let link = self.hub.link(peer, &protocol);
+                (*peer.as_bytes(), neighbors)
-                let path = link.path_description();
+            })
-                let max_datagram = link.max_datagram_size();
+            .collect();
-                plugin.on_peer_link(self.network_id, peer, link);
+        self.hub.set_topology(graph, members);
        let wanted: Vec<(EndpointId, String)> = self
            .sessions
            .values()
            .filter_map(|session| Some((session.peer, self.tunnel_protocol(session)?)))
            .collect();
        for (peer, protocol) in wanted {
            if self.hub.has_link(peer, &protocol) || !self.hub.reachable(peer) {
                continue;
            }
            let Some(plugin) = self
                .params
                .plugins
                .iter()
                .find(|plugin| plugin.protocol_id() == protocol)
                .cloned()
            else {
                continue;
            };
            let link = self.hub.link(peer, &protocol);
            let relayed = self.hub.route(peer).is_some_and(|route| route.hops > 1);
            let path = link.path_description();
            let max_datagram = link.max_datagram_size();
            plugin.on_peer_link(self.network_id, peer, link);
            // A direct link has already been reported as one.
            if relayed {
                self.emit(Event::DataLinkUp {
                    network: self.network_id,
                    peer,
-                    protocol: protocol.clone(),
+                    protocol,
                    path,
                    max_datagram,
                });
@@ -1758,30 +1848,25 @@ impl Runtime {
    /// Hands a link to the plugin that owns its protocol.
    fn adopt_link(&mut self, peer: EndpointId, protocol: String, link: SharedLink) {
-        let Some(plugin) = self
+        if !self
            .params
            .plugins
            .iter()
-            .find(|plugin| plugin.protocol_id() == protocol)
+            .any(|plugin| plugin.protocol_id() == protocol)
-            .cloned()
+        {
        else {
            return;
-        };
+        }
        let path = link.path_description();
        let max_datagram = link.max_datagram_size();
        self.links
            .insert((peer, protocol.clone()), Arc::clone(&link));
-        // The protocol is handed the hub's link, not this one: the same
+        // What this is, is a way to reach the peer. Which protocol's
-        // object for as long as the peer is a peer, whatever happens to
+        // tunnel runs over it is decided in `update_paths`, and that tunnel
-        // the path under it. A tunnel is not torn down because a direct
+        // is handed the hub's link, not this one: the same object for as
-        // link came or went.
+        // long as the peer is a peer, whatever happens to the path under
-        let first_time = !self.hub.has_link(peer, &protocol);
+        // it. A tunnel is not torn down because a direct link came or went.
        self.hub.set_direct(peer, &protocol, link);
        let shared = self.hub.link(peer, &protocol);
        let max_datagram = shared.max_datagram_size();
        if first_time {
            plugin.on_peer_link(self.network_id, peer, shared);
        }
        self.metrics.data_links_established += 1;
        self.emit(Event::DataLinkUp {
            network: self.network_id,
@@ -1913,6 +1998,13 @@ impl Runtime {
            });
        }
        capabilities.truncate(self.params.limits.max_capabilities);
        self.local_capabilities = capabilities
            .iter()
            .map(|capability| PluginCapability {
                data: Vec::new(),
                ..capability.clone()
            })
            .collect();
        let exit_node = self.exit_ready();
        self.announced_exit = exit_node;
        Announcement {
@@ -2147,6 +2239,69 @@ impl Runtime {
    // ----------------------------------------------------------------- status
    /// Whether a member can be reached now, by any path.
    ///
    /// The single definition every client reports: a live control session,
    /// or a data path to it through other members. Being reached over a long
    /// path is being online; how well is `data_path`'s to say.
    fn is_online(&self, peer: EndpointId) -> bool {
        peer == self.local_id || self.sessions.contains_key(&peer) || self.hub.reachable(peer)
    }
    /// How packets reach a member, which says nothing about whether it is
    /// there.
    fn data_path(&self, peer: EndpointId) -> DataPath {
        match self.hub.route(peer) {
            None => DataPath::None,
            Some(route) if route.hops <= 1 => DataPath::Direct {
                transport: self
                    .hub
                    .direct_kinds(peer)
                    .into_iter()
                    .next()
                    .unwrap_or_default(),
            },
            Some(route) => DataPath::Relayed {
                hops: route.hops,
                via: route.via,
            },
        }
    }
    /// Says so when this agent's own name is held by someone else, and
    /// again if that changes. Never anything but a statement: the agent keeps
    /// working by address either way.
    fn report_hostname_taken(&mut self) {
        let taken = self.hostname_conflict_of(&self.local_id);
        if taken == self.reported_name_taken {
            return;
        }
        if let Some(conflict) = &taken {
            tracing::warn!(
                network = %self.network_id,
                name = %conflict.name,
                holder = %conflict.holder.fmt_short(),
                "this name was claimed first by another member; it stays reachable by \
                 address but is not found by this name until it is renamed"
            );
            self.emit(Event::HostnameTaken {
                network: self.network_id,
                name: conflict.name.clone(),
                holder: conflict.holder,
            });
        }
        self.reported_name_taken = taken;
    }
    fn hostname_conflict_of(&self, author: &EndpointId) -> Option<HostnameConflict> {
        self.state
            .hostname_taken(author)
            .map(|(name, holder)| HostnameConflict {
                name: name.to_string(),
                holder,
            })
    }
    fn status(&self) -> NetworkStatus {
        let served = self.served_protocols();
        let mut peers: Vec<PeerStatus> = self
@@ -2184,6 +2339,7 @@ impl Runtime {
                    control_messages_received: session.messages_received,
                    control_bytes_sent: session.bytes_sent,
                    control_bytes_received: session.bytes_received,
                    data_path: self.data_path(session.peer),
                }
            })
            .collect();
@@ -2221,6 +2377,9 @@ impl Runtime {
                    overlay_address_v4: self.state.address_of(&endpoint_id),
                    hostname: self.state.hostname_of(&endpoint_id).map(str::to_string),
                    last_hostname: self.known_hostnames.get(&endpoint_id).cloned(),
                    hostname_conflict: self.hostname_conflict_of(&endpoint_id),
                    online: self.is_online(endpoint_id),
                    data_path: self.data_path(endpoint_id),
                })
            })
            .collect();
@@ -2248,6 +2407,7 @@ impl Runtime {
            peers,
            candidates,
            members,
            hostname_conflict: self.hostname_conflict_of(&self.local_id),
            range: self.effective_range(),
            // Only worth reporting while it is actually stuck: with a
            // fallback in play the configured range being another
@@ -2262,6 +2422,15 @@ impl Runtime {
    }
 }
 /// Milliseconds since the Unix epoch, for the time a name was first claimed.
 fn unix_millis() -> u64 {
    std::time::SystemTime::now()
        .duration_since(std::time::UNIX_EPOCH)
        .map_or(u64::MAX, |elapsed| {
            u64::try_from(elapsed.as_millis()).unwrap_or(u64::MAX)
        })
 }
 /// Which side dialled, given a role and the two identities.
 fn initiator_of(role: Role, local: EndpointId, peer: EndpointId) -> EndpointId {
    match role {
@@ -2448,3 +2617,48 @@ mod tests {
        assert_eq!(chosen_restored.copied(), Some("wg"));
    }
 }
 /// The host name of a relay URL such as `https://relay.example:443/`.
 fn relay_host(url: &str) -> Option<String> {
    let rest = url.split_once("://").map_or(url, |(_, rest)| rest);
    let authority = rest.split(['/', '?', '#']).next()?;
    let host = authority.rsplit_once('@').map_or(authority, |(_, h)| h);
    let host = if host.starts_with('[') {
        host.split_once(']')?.0.trim_start_matches('[')
    } else {
        host.split(':').next()?
    };
    (!host.is_empty()
        && host.len() <= 253
        && host
            .chars()
            .all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == ':'))
    .then(|| host.to_ascii_lowercase())
 }
 #[cfg(test)]
 mod relay_host_tests {
    use super::relay_host;
    #[test]
    fn the_host_is_taken_out_of_a_relay_url() {
        assert_eq!(
            relay_host("https://relay.example.com./").as_deref(),
            Some("relay.example.com.")
        );
        assert_eq!(
            relay_host("https://Euw1-1.relay.iroh.network:443/x").as_deref(),
            Some("euw1-1.relay.iroh.network")
        );
        assert_eq!(
            relay_host("http://192.0.2.7:3340").as_deref(),
            Some("192.0.2.7")
        );
        assert_eq!(
            relay_host("https://[2001:db8::1]:443/").as_deref(),
            Some("2001:db8::1")
        );
        assert_eq!(relay_host("https:///").as_deref(), None);
        assert_eq!(relay_host("https://bad host/").as_deref(), None);
    }
 }
@@ -59,6 +59,43 @@ pub struct OverlayStatus {
    pub broadcast_rules: Option<crate::overlay::BroadcastRulesReport>,
 }
 /// How packets reach a member right now.
 ///
 /// Being reachable is one thing and being reachable well is another, and
 /// the second is never a reason to call a member away: a member that can only
 /// be reached through somebody else is online, over a longer path.
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub enum DataPath {
    /// Nothing carries packets to it yet. The member is connected for
    /// control only, which is the state of a data path being set up.
    None,
    /// A direct link, of the named kind of transport.
    Direct {
        /// The kind of transport carrying it, e.g. `wg`.
        transport: String,
    },
    /// Through other members.
    Relayed {
        /// Number of links on the way, at least two.
        hops: u8,
        /// The member the first of them goes to.
        via: EndpointId,
    },
 }
 /// A name that a member claimed and somebody else holds.
 ///
 /// A name belongs to whoever claimed it first. The member that claimed it
 /// later is still a member and still reachable by its address; it is just
 /// not found by that name, on any member of the network.
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub struct HostnameConflict {
    /// The name that is taken.
    pub name: String,
    /// The member that holds it.
    pub holder: EndpointId,
 }
 /// A member the signed state knows about, connected or not.
 ///
 /// This is the durable roster: it comes from signed records, so a member that
@@ -85,6 +122,15 @@ pub struct MemberStatus {
    /// Not signed, so it is a convenience for naming a member that is away
    /// and never a claim: the zone is served from `hostname`.
    pub last_hostname: Option<String>,
    /// The name it claimed, when somebody else claimed it first.
    pub hostname_conflict: Option<HostnameConflict>,
    /// Whether this agent can reach it now, by any path: a live control
    /// session, or a data path through other members. The one answer to "is
    /// it online" for every client, so that none of them has to work it out
    /// from a different part of the picture. This agent counts as online.
    pub online: bool,
    /// How packets reach it. [`DataPath::None`] while it is offline.
    pub data_path: DataPath,
 }
 /// Status of one authenticated session.
@@ -128,6 +174,8 @@ pub struct PeerStatus {
    pub control_bytes_sent: u64,
    /// Control payload bytes read from this session.
    pub control_bytes_received: u64,
    /// How packets reach this peer right now.
    pub data_path: DataPath,
 }
 /// Counters scoped to one logical network.
@@ -188,6 +236,10 @@ pub struct NetworkStatus {
    pub candidates: Vec<CandidateStatus>,
    /// Members the signed state knows about, whether connected or not.
    pub members: Vec<MemberStatus>,
    /// This agent's own name, when another member claimed it first and so
    /// holds it. This agent keeps working by address; nobody finds it by
    /// this name until it is renamed or the other member lets it go.
    pub hostname_conflict: Option<HostnameConflict>,
    /// The overlay range this network uses, once it has one.
    pub range: Option<crate::state::Ipv4Range>,
    /// The range it could not have, because another network on this agent
@@ -67,10 +67,41 @@ impl Tally {
    }
 }
 /// The links to one neighbour, best first. Which transport carries a frame
 /// is decided here and nowhere else: the end-to-end protocol is in the
 /// envelope and never depends on it.
 #[derive(Debug)]
 struct NextHop {
    peer: PeerId,
-    link: SharedLink,
+    links: Box<[SharedLink]>,
 }
 impl NextHop {
    fn usable(&self) -> bool {
        self.links.iter().any(|link| !link.is_closed())
    }
    fn first_open(&self) -> Option<&SharedLink> {
        self.links.iter().find(|link| !link.is_closed())
    }
    /// Hands a frame to the best open link that accepts it. A link that
    /// cannot carry the frame (too large for its path, closed a moment ago)
    /// leaves it to the next one; the frame is only copied when there is
    /// another to try.
    fn send(&self, frame: Bytes) -> Result<(), TransportError> {
        let mut open = self.links.iter().filter(|link| !link.is_closed());
        let Some(mut current) = open.next() else {
            return Err(TransportError::Closed);
        };
        for next in open {
            if current.send(frame.clone()).is_ok() {
                return Ok(());
            }
            current = next;
        }
        current.send(frame)
    }
 }
 #[derive(Debug)]
@@ -82,21 +113,22 @@ struct ForwardRoute {
 impl ForwardRoute {
    fn select(&self, source: &PeerId, flow: FlowId) -> Option<&NextHop> {
        if self.next.len() == 1 {
-            return self.next.first().filter(|hop| !hop.link.is_closed());
+            return self.next.first().filter(|hop| hop.usable());
        }
        let seed = u64::from_le_bytes(source[..8].try_into().ok()?);
        let start = mix64(flow ^ seed) as usize % self.next.len();
        // Stable fallback if a link closed just before its table was replaced.
        (0..self.next.len())
            .map(|i| &self.next[(start + i) % self.next.len()])
-            .find(|hop| !hop.link.is_closed())
+            .find(|hop| hop.usable())
    }
 }
 #[derive(Debug, Default)]
 struct ForwardingTable {
    routes: HashMap<PeerId, ForwardRoute>,
-    inboxes: HashMap<PeerId, mpsc::Sender<Bytes>>,
+    /// Local delivery, by who sent it and which end-to-end protocol it is.
    inboxes: HashMap<(PeerId, u64), mpsc::Sender<Bytes>>,
    members: HashSet<PeerId>,
 }
@@ -121,7 +153,7 @@ impl Plane {
            return;
        }
        if header.destination == self.local {
-            let Some(inbox) = table.inboxes.get(&header.source) else {
+            let Some(inbox) = table.inboxes.get(&(header.source, header.protocol)) else {
                self.tally.dropped_unknown.fetch_add(1, Ordering::Relaxed);
                return;
            };
@@ -144,7 +176,7 @@ impl Plane {
            self.tally.dropped_no_link.fetch_add(1, Ordering::Relaxed);
            return;
        };
-        if hop.link.send(envelope::decrement(frame)).is_ok() {
+        if hop.send(envelope::decrement(frame)).is_ok() {
            self.tally.forwarded.fetch_add(1, Ordering::Relaxed);
        } else {
            self.tally.dropped_no_link.fetch_add(1, Ordering::Relaxed);
@@ -152,27 +184,65 @@ impl Plane {
    }
 }
 /// One physical link to a neighbour, of whatever kind of transport.
 #[derive(Debug)]
-struct Protocol {
+struct RawLink {
-    plane: Arc<Plane>,
+    kind: String,
-    graph: HashMap<PeerId, Vec<PeerId>>,
+    link: SharedLink,
-    members: HashSet<PeerId>,
+    reader: JoinHandle<()>,
    raw: HashMap<PeerId, SharedLink>,
    readers: HashMap<PeerId, JoinHandle<()>>,
    links: HashMap<PeerId, Arc<PeerLink>>,
 }
-impl Protocol {
+/// How a destination is reached right now.
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub struct RouteSummary {
    /// Number of transport links to it. One is a direct link.
    pub hops: u8,
    /// The neighbour the first link goes to. For a direct link that is the
    /// destination itself.
    pub via: EndpointId,
 }
 #[derive(Debug)]
 struct State {
    plane: Arc<Plane>,
    /// Transport kinds, best first. A neighbour with several links is
    /// reached over the first of these it has.
    priority: Vec<String>,
    graph: HashMap<PeerId, Vec<PeerId>>,
    members: HashSet<PeerId>,
    raw: HashMap<PeerId, Vec<RawLink>>,
    links: HashMap<(PeerId, String), Arc<PeerLink>>,
 }
 impl State {
    fn rank(&self, kind: &str) -> usize {
        self.priority
            .iter()
            .position(|known| known == kind)
            .unwrap_or(usize::MAX)
    }
    /// A neighbour's links, best kind first.
    fn ordered(&self, peer: &PeerId) -> Vec<&RawLink> {
        let mut links: Vec<&RawLink> = self
            .raw
            .get(peer)
            .map(|links| links.iter().collect())
            .unwrap_or_default();
        links.sort_by_key(|raw| (self.rank(&raw.kind), raw.kind.clone()));
        links
    }
    fn publish(&self) {
        let mut graph = self.graph.clone();
-        graph.insert(
+        let mut neighbours: Vec<PeerId> = self
-            self.plane.local,
+            .raw
-            self.raw
+            .iter()
-                .iter()
+            .filter(|(_, links)| links.iter().any(|raw| !raw.link.is_closed()))
-                .filter(|(_, link)| !link.is_closed())
+            .map(|(peer, _)| *peer)
-                .map(|(peer, _)| *peer)
+            .collect();
-                .collect(),
+        neighbours.sort_unstable();
-        );
+        graph.insert(self.plane.local, neighbours);
        let routing = RoutingTable::build(self.plane.local, &graph, ROUTING_HOP_LIMIT);
        let routes = routing
            .iter()
@@ -181,10 +251,12 @@ impl Protocol {
                    .next_hops
                    .iter()
                    .filter_map(|id| {
-                        self.raw.get(id).map(|link| NextHop {
+                        let links: Box<[_]> = self
-                            peer: *id,
+                            .ordered(id)
-                            link: link.clone(),
+                            .into_iter()
-                        })
+                            .map(|raw| raw.link.clone())
                            .collect();
                        (!links.is_empty()).then_some(NextHop { peer: *id, links })
                    })
                    .collect();
                (!next.is_empty()).then_some((
@@ -204,17 +276,21 @@ impl Protocol {
            inboxes: self
                .links
                .iter()
-                .map(|(peer, link)| (*peer, link.inbox_tx.clone()))
+                .map(|((peer, _), link)| ((*peer, link.tag), link.inbox_tx.clone()))
                .collect(),
        }));
    }
    fn stop_readers(&mut self) {
        for raw in self.raw.values().flatten() {
            raw.reader.abort();
        }
    }
 }
-impl Drop for Protocol {
+impl Drop for State {
    fn drop(&mut self) {
-        for task in self.readers.values() {
+        self.stop_readers();
            task.abort();
        }
        for link in self.links.values() {
            link.close();
        }
@@ -222,12 +298,17 @@ impl Drop for Protocol {
    }
 }
-/// One network's transport links and independently published protocol tables.
+/// One network's transport links and its published forwarding table.
 ///
 /// There is one table for the whole network, whatever protocols run over it.
 /// A link is a way to reach a neighbour; a protocol is an end-to-end channel
 /// to a peer. The two meet only in the envelope, which names the protocol, so
 /// a tunnel keeps working over any mix of transports and a middle member
 /// needs no protocol in common with either end.
 #[derive(Debug)]
 pub struct RelayHub {
    network: NetworkId,
-    local: PeerId,
+    state: Mutex<State>,
    protocols: Mutex<HashMap<String, Protocol>>,
    tally: Arc<Tally>,
    changed: Arc<Notify>,
 }
@@ -235,40 +316,38 @@ pub struct RelayHub {
 impl RelayHub {
    /// Creates the router at this network's local endpoint.
    pub fn new(network: NetworkId, local: EndpointId) -> Arc<Self> {
        let tally = Arc::new(Tally::default());
        Arc::new(Self {
            network,
-            local: *local.as_bytes(),
+            state: Mutex::new(State {
-            protocols: Mutex::default(),
+                plane: Arc::new(Plane {
-            tally: Arc::default(),
+                    local: *local.as_bytes(),
                    table: ArcSwap::from_pointee(ForwardingTable::default()),
                    tally: tally.clone(),
                }),
                priority: Vec::new(),
                graph: HashMap::new(),
                members: HashSet::new(),
                raw: HashMap::new(),
                links: HashMap::new(),
            }),
            tally,
            changed: Arc::default(),
        })
    }
-    fn protocols(&self) -> std::sync::MutexGuard<'_, HashMap<String, Protocol>> {
+    fn state(&self) -> std::sync::MutexGuard<'_, State> {
-        self.protocols
+        self.state.lock().unwrap_or_else(|error| error.into_inner())
            .lock()
            .unwrap_or_else(|error| error.into_inner())
    }
-    fn protocol<'a>(
+    /// Sets which kinds of transport are preferred, best first, when a
-        &self,
+    /// neighbour can be reached over more than one.
-        protocols: &'a mut HashMap<String, Protocol>,
+    pub fn set_priority(&self, kinds: Vec<String>) {
-        name: &str,
+        let mut state = self.state();
-    ) -> &'a mut Protocol {
+        if state.priority != kinds {
-        protocols
+            state.priority = kinds;
-            .entry(name.to_owned())
+            state.publish();
-            .or_insert_with(|| Protocol {
+        }
                plane: Arc::new(Plane {
                    local: self.local,
                    table: ArcSwap::from_pointee(ForwardingTable::default()),
                    tally: self.tally.clone(),
                }),
                graph: HashMap::new(),
                members: HashSet::new(),
                raw: HashMap::new(),
                readers: HashMap::new(),
                links: HashMap::new(),
            })
    }
    /// Wakes the control loop promptly when a transport reader ends.
@@ -282,14 +361,11 @@ impl RelayHub {
    }
    /// Publishes authenticated topology. Refreshing identical state is cheap.
-    pub fn set_topology(
+    ///
-        &self,
+    /// Every row of `graph` is one member's own account of who it has a link
-        protocol: &str,
+    /// to; what kind of link it is does not matter here.
-        graph: HashMap<PeerId, Vec<PeerId>>,
+    pub fn set_topology(&self, graph: HashMap<PeerId, Vec<PeerId>>, members: HashSet<PeerId>) {
-        members: HashSet<PeerId>,
+        let mut state = self.state();
    ) {
        let mut protocols = self.protocols();
        let state = self.protocol(&mut protocols, protocol);
        if state.graph != graph || state.members != members {
            state.graph = graph;
            state.members = members;
@@ -298,22 +374,44 @@ impl RelayHub {
    }
    /// Whether the current table has a path to this destination.
-    pub fn reachable(&self, peer: EndpointId, protocol: &str) -> bool {
+    pub fn reachable(&self, peer: EndpointId) -> bool {
-        self.protocols().get(protocol).is_some_and(|state| {
+        self.state()
-            state
+            .plane
-                .plane
+            .table
-                .table
+            .load()
-                .load()
+            .routes
-                .routes
+            .contains_key(peer.as_bytes())
-                .contains_key(peer.as_bytes())
+    }
    /// How a destination is reached right now, if it is.
    pub fn route(&self, peer: EndpointId) -> Option<RouteSummary> {
        let table = self.state().plane.table.load_full();
        let route = table.routes.get(peer.as_bytes())?;
        let via = EndpointId::from_bytes(&route.next.first()?.peer).ok()?;
        Some(RouteSummary {
            hops: route.hops,
            via,
        })
    }
-    /// Stable logical link, preserved across topology changes.
+    /// The kinds of transport with an open direct link to a peer, best
    /// first.
    pub fn direct_kinds(&self, peer: EndpointId) -> Vec<String> {
        let state = self.state();
        state
            .ordered(peer.as_bytes())
            .into_iter()
            .filter(|raw| !raw.link.is_closed())
            .map(|raw| raw.kind.clone())
            .collect()
    }
    /// Stable end-to-end channel for a protocol, preserved across topology
    /// and transport changes.
    pub fn link(&self, peer: EndpointId, protocol: &str) -> Arc<PeerLink> {
-        let mut protocols = self.protocols();
+        let mut state = self.state();
-        let state = self.protocol(&mut protocols, protocol);
+        let key = (*peer.as_bytes(), protocol.to_owned());
-        if let Some(link) = state.links.get(peer.as_bytes()) {
+        if let Some(link) = state.links.get(&key) {
            return link.clone();
        }
        let (inbox_tx, inbox_rx) = mpsc::channel(INBOX);
@@ -321,88 +419,118 @@ impl RelayHub {
        let link = Arc::new(PeerLink {
            network: self.network,
            peer,
            tag: envelope::protocol_tag(protocol),
            plane: state.plane.clone(),
            inbox_tx,
            inbox_rx: tokio::sync::Mutex::new(inbox_rx),
            closed,
        });
-        state.links.insert(*peer.as_bytes(), link.clone());
+        state.links.insert(key, link.clone());
        state.publish();
        link
    }
-    /// Whether the protocol already owns its logical link.
+    /// Whether the protocol already owns its end-to-end channel to a peer.
    pub fn has_link(&self, peer: EndpointId, protocol: &str) -> bool {
-        self.protocols()
+        self.state()
-            .get(protocol)
+            .links
-            .is_some_and(|state| state.links.contains_key(peer.as_bytes()))
+            .contains_key(&(*peer.as_bytes(), protocol.to_owned()))
    }
    /// Installs a raw transport link and starts its independent ingress reader.
-    pub fn set_direct(&self, peer: EndpointId, protocol: &str, raw: SharedLink) {
+    ///
    /// A neighbour may have one link per kind of transport; installing a
    /// second of the same kind replaces the first.
    pub fn set_direct(&self, peer: EndpointId, kind: &str, raw: SharedLink) {
        if raw.network() != self.network || raw.peer() != peer {
            return;
        }
-        let mut protocols = self.protocols();
+        let mut state = self.state();
        let state = self.protocol(&mut protocols, protocol);
        let id = *peer.as_bytes();
        if let Some(old) = state.readers.remove(&id) {
            old.abort();
        }
        state.raw.insert(id, raw.clone());
        state.publish();
        let plane = state.plane.clone();
        let changed = self.changed.clone();
-        state.readers.insert(
+        let reader_link = raw.clone();
-            id,
+        let reader = tokio::spawn(async move {
-            tokio::spawn(async move {
+            let mut batch = 0;
-                let mut batch = 0;
+            while let Some(frame) = reader_link.recv().await {
-                while let Some(frame) = raw.recv().await {
+                plane.receive(id, frame);
-                    plane.receive(id, frame);
+                batch += 1;
-                    batch += 1;
+                if batch == 64 {
-                    if batch == 64 {
+                    tokio::task::yield_now().await;
-                        tokio::task::yield_now().await;
+                    batch = 0;
                        batch = 0;
                    }
                }
-                changed.notify_one();
+            }
-            }),
+            changed.notify_one();
-        );
+        });
        let links = state.raw.entry(id).or_default();
        if let Some(position) = links.iter().position(|known| known.kind == kind) {
            links.remove(position).reader.abort();
        }
        links.push(RawLink {
            kind: kind.to_owned(),
            link: raw,
            reader,
        });
        state.publish();
    }
-    /// Removes a physical link without tearing down end-to-end protocol state.
+    /// Removes one physical link without tearing down end-to-end protocol state.
-    pub fn clear_direct(&self, peer: EndpointId, protocol: &str) {
+    pub fn clear_direct(&self, peer: EndpointId, kind: &str) {
-        if let Some(state) = self.protocols().get_mut(protocol) {
+        let mut state = self.state();
-            state.raw.remove(peer.as_bytes());
+        let mut changed = false;
-            if let Some(task) = state.readers.remove(peer.as_bytes()) {
+        if let Some(links) = state.raw.get_mut(peer.as_bytes()) {
-                task.abort();
+            if let Some(position) = links.iter().position(|known| known.kind == kind) {
                links.remove(position).reader.abort();
                changed = true;
            }
            if links.is_empty() {
                state.raw.remove(peer.as_bytes());
            }
        }
        if changed {
            state.publish();
        }
    }
    /// Revokes all links and topology involving a departed authenticated peer.
    pub fn remove_peer(&self, peer: EndpointId) {
-        for state in self.protocols().values_mut() {
+        let mut state = self.state();
-            state.raw.remove(peer.as_bytes());
+        if let Some(links) = state.raw.remove(peer.as_bytes()) {
-            if let Some(task) = state.readers.remove(peer.as_bytes()) {
+            for raw in links {
-                task.abort();
+                raw.reader.abort();
            }
-            if let Some(link) = state.links.remove(peer.as_bytes()) {
+        }
        let gone: Vec<_> = state
            .links
            .keys()
            .filter(|(id, _)| id == peer.as_bytes())
            .cloned()
            .collect();
        for key in gone {
            if let Some(link) = state.links.remove(&key) {
                link.close();
            }
            state.members.remove(peer.as_bytes());
            state.graph.remove(peer.as_bytes());
            for neighbors in state.graph.values_mut() {
                neighbors.retain(|id| id != peer.as_bytes());
            }
            state.publish();
        }
        state.members.remove(peer.as_bytes());
        state.graph.remove(peer.as_bytes());
        for neighbors in state.graph.values_mut() {
            neighbors.retain(|id| id != peer.as_bytes());
        }
        state.publish();
    }
    /// Stops every reader and releases all transport handles.
    pub fn close(&self) {
-        self.protocols().clear();
+        let mut state = self.state();
        state.stop_readers();
        state.raw.clear();
        for link in state.links.values() {
            link.close();
        }
        state.links.clear();
        state.graph.clear();
        state.members.clear();
        state.plane.table.store(Arc::default());
    }
 }
@@ -411,6 +539,7 @@ impl RelayHub {
 pub struct PeerLink {
    network: NetworkId,
    peer: EndpointId,
    tag: u64,
    plane: Arc<Plane>,
    inbox_tx: mpsc::Sender<Bytes>,
    inbox_rx: tokio::sync::Mutex<mpsc::Receiver<Bytes>>,
@@ -454,10 +583,11 @@ impl PacketLink for PeerLink {
        let hop = route
            .select(&self.plane.local, flow)
            .ok_or(TransportError::Closed)?;
-        hop.link.send(envelope::encode(
+        hop.send(envelope::encode(
            self.plane.local,
            *self.peer.as_bytes(),
            flow,
            self.tag,
            &payload,
        ))?;
        if route.hops > 1 {
@@ -489,7 +619,9 @@ impl PacketLink for PeerLink {
    fn path_description(&self) -> String {
        let table = self.plane.table.load();
        match table.routes.get(self.peer.as_bytes()) {
-            Some(route) if route.hops == 1 => route.next[0].link.path_description(),
+            Some(route) if route.hops == 1 => route.next[0]
                .first_open()
                .map_or_else(|| "unreachable".into(), |link| link.path_description()),
            Some(route) => format!(
                "relay {} hops via {} ({} equal paths)",
                route.hops,
@@ -16,6 +16,9 @@ fn peer(seed: u8) -> EndpointId {
 fn id(seed: u8) -> PeerId {
    *peer(seed).as_bytes()
 }
 fn tag() -> u64 {
    envelope::protocol_tag("ip")
 }
 #[derive(Debug)]
 struct Wire {
@@ -78,9 +81,8 @@ impl PacketLink for Wire {
    }
 }
-fn topology(hub: &RelayHub, protocol: &str, edges: &[(u8, &[u8])], members: &[u8]) {
+fn topology(hub: &RelayHub, edges: &[(u8, &[u8])], members: &[u8]) {
    hub.set_topology(
        protocol,
        edges
            .iter()
            .map(|(a, bs)| (id(*a), bs.iter().map(|b| id(*b)).collect()))
@@ -96,7 +98,7 @@ async fn ecmp_is_per_flow_direct_wins_and_existing_tunnel_survives_changes() {
    let c = Wire::new(3, true);
    hub.set_direct(peer(2), "ip", b.clone());
    hub.set_direct(peer(3), "ip", c.clone());
-    topology(&hub, "ip", &[(2, &[4]), (3, &[4])], &[1, 2, 3, 4]);
+    topology(&hub, &[(2, &[4]), (3, &[4])], &[1, 2, 3, 4]);
    let link = hub.link(peer(4), "ip");
    let mut chosen = HashMap::new();
    for _ in 0..8 {
@@ -132,7 +134,7 @@ async fn ecmp_is_per_flow_direct_wins_and_existing_tunnel_survives_changes() {
    }
    assert_eq!(c.take().len(), 100);
    hub.clear_direct(peer(2), "ip");
-    topology(&hub, "ip", &[(3, &[])], &[1, 2, 3, 4]);
+    topology(&hub, &[(3, &[])], &[1, 2, 3, 4]);
    assert!(link.send(Bytes::new()).is_err());
    assert!(!link.is_closed());
    assert!(Arc::ptr_eq(&link, &hub.link(peer(4), "ip")));
@@ -152,8 +154,8 @@ async fn transport_reader_forwards_without_any_plugin_reader_and_isolates_protoc
    let c = Wire::new(3, true);
    hub.set_direct(peer(1), "ip", a.clone());
    hub.set_direct(peer(3), "ip", c.clone());
-    topology(&hub, "ip", &[(3, &[4])], &[1, 2, 3, 4]);
+    topology(&hub, &[(3, &[4])], &[1, 2, 3, 4]);
-    let frame = envelope::encode(id(1), id(4), 91, b"end-to-end encrypted");
+    let frame = envelope::encode(id(1), id(4), 91, tag(), b"end-to-end encrypted");
    let pointer = frame.as_ptr();
    a.feed.send(frame).await.unwrap();
    tokio::time::timeout(std::time::Duration::from_secs(1), async {
@@ -170,12 +172,136 @@ async fn transport_reader_forwards_without_any_plugin_reader_and_isolates_protoc
        ROUTING_HOP_LIMIT - 1
    );
    assert_eq!(&sent[0][RELAY_OVERHEAD..], b"end-to-end encrypted");
    let isolated = hub.link(peer(4), "other-ip");
    assert!(isolated.send(Bytes::new()).is_err());
    assert_eq!(hub.counters().forwarded, 1);
    hub.close();
 }
 #[tokio::test]
 async fn a_frame_is_delivered_only_to_the_protocol_it_names() {
    let hub = RelayHub::new(network(), peer(2));
    hub.set_direct(peer(1), "ip", Wire::new(1, true));
    let ip = hub.link(peer(1), "ip");
    let other = hub.link(peer(1), "other-ip");
    let plane = ip.plane.clone();
    plane.receive(id(1), envelope::encode(id(1), id(2), 0, tag(), b"for ip"));
    plane.receive(
        id(1),
        envelope::encode(
            id(1),
            id(2),
            0,
            envelope::protocol_tag("other-ip"),
            b"for other",
        ),
    );
    plane.receive(
        id(1),
        envelope::encode(
            id(1),
            id(2),
            0,
            envelope::protocol_tag("nobody-runs-this"),
            b"for nobody",
        ),
    );
    assert_eq!(ip.recv().await.unwrap(), Bytes::from_static(b"for ip"));
    assert_eq!(
        other.recv().await.unwrap(),
        Bytes::from_static(b"for other")
    );
    assert_eq!(hub.counters().dropped_unknown, 1);
    hub.close();
 }
 /// A middle member joins two neighbours over different transports, and the
 /// two ends share a protocol the middle has never heard of.
 #[tokio::test]
 async fn a_tunnel_crosses_a_middle_member_whose_links_are_of_different_kinds() {
    let a = RelayHub::new(network(), peer(1));
    let c = RelayHub::new(network(), peer(3));
    let b = RelayHub::new(network(), peer(2));
    let a_to_c = Wire::new(3, true);
    let c_to_a = Wire::new(1, true);
    let c_to_b = Wire::new(2, true);
    let b_to_c = Wire::new(3, true);
    a.set_direct(peer(3), "wg", a_to_c.clone());
    c.set_direct(peer(1), "wg", c_to_a);
    c.set_direct(peer(2), "wg-quic", c_to_b.clone());
    b.set_direct(peer(3), "wg-quic", b_to_c.clone());
    topology(&a, &[(3, &[1, 2])], &[1, 2, 3]);
    topology(&b, &[(3, &[1, 2])], &[1, 2, 3]);
    topology(&c, &[(1, &[3]), (2, &[3])], &[1, 2, 3]);
    let outgoing = a.link(peer(2), "tunnel");
    let incoming = b.link(peer(1), "tunnel");
    outgoing
        .send_flow(Bytes::from_static(b"sealed"), 5)
        .unwrap();
    let frame = a_to_c.take().remove(0);
    // The middle forwards it onto its other kind of link without looking
    // inside.
    c.state().plane.receive(id(1), frame);
    let frame = c_to_b.take().remove(0);
    assert_eq!(c.counters().forwarded, 1);
    b.state().plane.receive(id(3), frame);
    assert_eq!(
        incoming.recv().await.unwrap(),
        Bytes::from_static(b"sealed")
    );
    assert_eq!(b.counters().received_via, 1);
    assert_eq!(
        b.route(peer(1)),
        Some(RouteSummary {
            hops: 2,
            via: peer(3)
        })
    );
    a.close();
    b.close();
    c.close();
 }
 #[tokio::test]
 async fn a_neighbour_is_reached_over_the_preferred_kind_and_falls_back_when_it_is_lost() {
    let hub = RelayHub::new(network(), peer(1));
    hub.set_priority(vec!["wg".into(), "tcp-tls".into(), "wg-quic".into()]);
    let slow = Wire::new(2, true);
    let fast = Wire::new(2, true);
    hub.set_direct(peer(2), "wg-quic", slow.clone());
    let link = hub.link(peer(2), "tunnel");
    link.send(Bytes::from_static(b"one")).unwrap();
    assert_eq!(slow.take().len(), 1);
    assert_eq!(hub.direct_kinds(peer(2)), ["wg-quic"]);
    // A better kind appears: traffic moves to it with nothing to redo.
    hub.set_direct(peer(2), "wg", fast.clone());
    assert_eq!(hub.direct_kinds(peer(2)), ["wg", "wg-quic"]);
    link.send(Bytes::from_static(b"two")).unwrap();
    assert_eq!(fast.take().len(), 1);
    assert!(slow.take().is_empty());
    // It dies: the worse one carries on, and the logical link is untouched.
    fast.dead.store(true, Ordering::Relaxed);
    link.send(Bytes::from_static(b"three")).unwrap();
    assert_eq!(slow.take().len(), 1);
    assert!(!link.is_closed());
    assert_eq!(
        hub.route(peer(2)),
        Some(RouteSummary {
            hops: 1,
            via: peer(2)
        })
    );
    hub.clear_direct(peer(2), "wg-quic");
    hub.clear_direct(peer(2), "wg");
    assert!(!hub.reachable(peer(2)));
    assert!(Arc::ptr_eq(&link, &hub.link(peer(2), "tunnel")));
    hub.close();
 }
 #[tokio::test]
 async fn local_delivery_is_bounded_and_malformed_or_unknown_sources_are_dropped() {
    let hub = RelayHub::new(network(), peer(2));
@@ -184,18 +310,21 @@ async fn local_delivery_is_bounded_and_malformed_or_unknown_sources_are_dropped(
    let link = hub.link(peer(1), "ip");
    let plane = link.plane.clone();
    for _ in 0..INBOX + 1 {
-        plane.receive(id(1), envelope::encode(id(1), id(2), 0, b"ip"));
+        plane.receive(id(1), envelope::encode(id(1), id(2), 0, tag(), b"ip"));
    }
    assert_eq!(hub.counters().dropped_congested, 1);
    for _ in 0..INBOX {
        assert_eq!(link.recv().await.unwrap(), Bytes::from_static(b"ip"));
    }
    plane.receive(id(1), Bytes::from_static(b"broken"));
-    plane.receive(id(1), envelope::encode(id(9), id(2), 0, b"unknown member"));
+    plane.receive(
        id(1),
        envelope::encode(id(9), id(2), 0, tag(), b"unknown member"),
    );
    assert_eq!(hub.counters().dropped_unknown, 2);
    plane.receive(
        id(1),
-        envelope::encode(id(1), id(9), 0, b"unknown destination"),
+        envelope::encode(id(1), id(9), 0, tag(), b"unknown destination"),
    );
    assert_eq!(hub.counters().dropped_no_link, 1);
    hub.close();
@@ -210,11 +339,11 @@ async fn inconsistent_tables_cannot_loop_beyond_the_hop_limit() {
    a.set_direct(peer(2), "ip", ab.clone());
    b.set_direct(peer(1), "ip", ba.clone());
    // Deliberately inconsistent snapshots while updates are in flight.
-    topology(&a, "ip", &[(2, &[4])], &[1, 2, 3, 4]);
+    topology(&a, &[(2, &[4])], &[1, 2, 3, 4]);
-    topology(&b, "ip", &[(1, &[4])], &[1, 2, 3, 4]);
+    topology(&b, &[(1, &[4])], &[1, 2, 3, 4]);
    let ap = a.link(peer(4), "ip").plane.clone();
    let bp = b.link(peer(4), "ip").plane.clone();
-    let mut frame = envelope::encode(id(3), id(4), 7, b"transit");
+    let mut frame = envelope::encode(id(3), id(4), 7, tag(), b"transit");
    for turn in 0..ROUTING_HOP_LIMIT {
        let (plane, incoming, wire) = if turn % 2 == 0 {
            (&ap, id(2), &ab)
@@ -253,7 +382,7 @@ async fn forwarding_benchmark() {
    hub.set_direct(peer(3), "ip", wire.clone());
    let alternate = Wire::new(5, false);
    hub.set_direct(peer(5), "ip", alternate.clone());
-    topology(&hub, "ip", &[(3, &[4]), (5, &[4])], &[1, 2, 3, 4, 5]);
+    topology(&hub, &[(3, &[4]), (5, &[4])], &[1, 2, 3, 4, 5]);
    let plane = hub.link(peer(4), "ip").plane.clone();
    let source = id(1);
    let destination = id(4);
@@ -264,7 +393,9 @@ async fn forwarding_benchmark() {
        const BATCHES: usize = 100;
        for _ in 0..BATCHES {
            let frames: Vec<_> = (0..COUNT)
-                .map(|flow| envelope::encode(source, destination, flow as u64, &vec![42; size]))
+                .map(|flow| {
                    envelope::encode(source, destination, flow as u64, tag(), &vec![42; size])
                })
                .collect();
            let start = std::time::Instant::now();
            for frame in frames {
@@ -3,8 +3,8 @@ use super::{FlowId, PeerId};
 use crate::config::{MAX_DATA_DATAGRAM, ROUTING_HOP_LIMIT};
 use bytes::{BufMut, Bytes, BytesMut};
-pub(crate) const HEADER: usize = 2 + 32 + 32 + 8;
+pub(crate) const HEADER: usize = 2 + 32 + 32 + 8 + 8;
-const VERSION: u8 = 1;
+const VERSION: u8 = 2;
 #[derive(Debug, PartialEq, Eq)]
 pub(crate) struct Header {
@@ -12,6 +12,15 @@ pub(crate) struct Header {
    pub destination: PeerId,
    pub remaining: u8,
    pub flow: FlowId,
    /// Which end-to-end protocol the payload belongs to, whatever links carry it.
    pub protocol: u64,
 }
 /// A stable 64-bit tag for a protocol id (FNV-1a), the same on every member.
 pub(crate) fn protocol_tag(name: &str) -> u64 {
    name.bytes().fold(0xcbf2_9ce4_8422_2325, |hash, byte| {
        (hash ^ u64::from(byte)).wrapping_mul(0x0000_0100_0000_01b3)
    })
 }
 pub(crate) fn decode(bytes: &[u8]) -> Option<Header> {
@@ -28,16 +37,24 @@ pub(crate) fn decode(bytes: &[u8]) -> Option<Header> {
        destination: bytes[34..66].try_into().ok()?,
        remaining: bytes[1],
        flow: u64::from_be_bytes(bytes[66..74].try_into().ok()?),
        protocol: u64::from_be_bytes(bytes[74..82].try_into().ok()?),
    })
 }
-pub(crate) fn encode(source: PeerId, destination: PeerId, flow: FlowId, payload: &[u8]) -> Bytes {
+pub(crate) fn encode(
    source: PeerId,
    destination: PeerId,
    flow: FlowId,
    protocol: u64,
    payload: &[u8],
 ) -> Bytes {
    let mut bytes = BytesMut::with_capacity(HEADER + payload.len());
    bytes.put_u8(VERSION);
    bytes.put_u8(ROUTING_HOP_LIMIT);
    bytes.extend_from_slice(&source);
    bytes.extend_from_slice(&destination);
    bytes.put_u64(flow);
    bytes.put_u64(protocol);
    bytes.extend_from_slice(payload);
    bytes.freeze()
 }
@@ -58,7 +75,7 @@ mod tests {
    use super::*;
    #[test]
    fn transit_reuses_the_buffer_and_rejects_invalid_headers() {
-        let frame = encode([1; 32], [2; 32], 17, &[42; 1280]);
+        let frame = encode([1; 32], [2; 32], 17, protocol_tag("wg"), &[42; 1280]);
        let address = frame.as_ptr();
        let frame = decrement(frame);
        assert_eq!(
@@ -66,7 +83,10 @@ mod tests {
            address,
            "no payload copy for an owned buffer"
        );
-        assert_eq!(decode(&frame).unwrap().remaining, ROUTING_HOP_LIMIT - 1);
+        let header = decode(&frame).unwrap();
        assert_eq!(header.remaining, ROUTING_HOP_LIMIT - 1);
        assert_eq!(header.protocol, protocol_tag("wg"));
        assert_ne!(protocol_tag("wg"), protocol_tag("wg-quic"));
        assert_eq!(&frame[HEADER..], &[42; 1280]);
        for length in 0..HEADER {
            assert!(decode(&frame[..length]).is_none());
@@ -484,11 +484,43 @@ pub struct NetworkReport {
    pub range_conflict: Option<String>,
    /// The overlay, when a protocol is running one.
    pub overlay: Option<OverlayReport>,
    /// This agent's own name, when another member claimed it first.
    pub hostname_conflict: Option<HostnameConflictReport>,
    /// Exit-node settings: whether this agent offers one, and whether it
    /// uses one.
    pub exit: ExitReport,
 }
 /// How packets reach a member right now.
 #[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
 pub enum DataPathReport {
    /// Nothing carries packets to it: it is away, or only connected for
    /// control while a data path is being set up.
    #[default]
    None,
    /// A direct link.
    Direct {
        /// The kind of transport carrying it, e.g. `wg`.
        transport: String,
    },
    /// Through other members.
    Relayed {
        /// Number of links on the way, at least two.
        hops: u8,
        /// The member the first of them goes to, as an endpoint id.
        via: String,
    },
 }
 /// A name somebody else claimed first.
 #[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
 pub struct HostnameConflictReport {
    /// The name that is taken.
    pub name: String,
    /// The member that holds it, as an endpoint id.
    pub holder: String,
 }
 /// One member of the network, from signed state.
 #[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
 pub struct MemberReport {
@@ -500,6 +532,15 @@ pub struct MemberReport {
    pub failed_dials: u32,
    /// The last hostname this agent saw it announce, kept while it is away.
    pub hostname: Option<String>,
    /// Whether this agent can reach it now, by any path. The one answer
    /// every client gives to "is it online": a member reached through
    /// others is online, over a longer path.
    pub online: bool,
    /// How packets reach it.
    pub path: DataPathReport,
    /// The name it claimed, when somebody else claimed it first. It still
    /// works by address; it is not found by that name.
    pub hostname_conflict: Option<HostnameConflictReport>,
 }
 /// One control plane peer.
@@ -516,6 +557,8 @@ pub struct PeerReport {
    pub rtt_ms: Option<u64>,
    /// The peer offers itself as an exit node in this network.
    pub exit_node: bool,
    /// How packets reach it.
    pub path: DataPathReport,
 }
 /// The WireGuard overlay of one network.
@@ -779,7 +822,7 @@ pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5);
 ///
 /// Bump it whenever [`Request`], [`Response`] or anything they contain
 /// changes shape.
-pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 17]);
+pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 18]);
 /// Reads one request off an accepted stream, answers it, writes the response.
 ///
@@ -35,6 +35,17 @@ pub struct ExitHostPlan {
    pub offer: Vec<Ipv4Range>,
    /// Whether this device sends its internet traffic through a member.
    pub client: bool,
    /// The range of every network on the interface. Only the using side
    /// reads it, to keep this agent's own traffic *to* the overlay inside it
    /// where the host has no per-user routing (macOS).
    pub overlay: Vec<Ipv4Range>,
    /// Underlay addresses this agent talks to directly and must keep reaching
    /// outside the overlay's default route. Only filled while `client` is set
    /// and only for hosts that report [`ExitHostRules::needs_bypass`].
    pub bypass: Vec<std::net::Ipv4Addr>,
    /// Host names (relay servers) that must stay reachable the same way; the
    /// host rules resolve them.
    pub bypass_hosts: Vec<String>,
 }
 impl ExitHostPlan {
@@ -72,6 +83,13 @@ pub trait ExitHostRules: Send + Sync + std::fmt::Debug + 'static {
    /// when nothing is there: it runs on the shutdown path, and once at
    /// startup to take away what a crashed run left.
    fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()>;
    /// Whether using an exit node on this host needs the underlay addresses
    /// the agent talks to, because the host cannot exempt the agent's own
    /// traffic from the default route by user id (Windows). The default is no.
    fn needs_bypass(&self) -> bool {
        false
    }
 }
 /// The tag every firewall object of one offered range carries.
@@ -99,6 +117,7 @@ pub struct MockExitRules {
    calls: Arc<Mutex<Vec<String>>>,
    forwarding: Arc<Mutex<Option<bool>>>,
    failure: Arc<Mutex<Option<String>>>,
    needs_bypass: Arc<Mutex<bool>>,
 }
 impl MockExitRules {
@@ -114,6 +133,11 @@ impl MockExitRules {
        *lock(&self.forwarding) = forwarding;
    }
    /// Whether this host asks for the underlay addresses, like Windows.
    pub fn set_needs_bypass(&self, needs: bool) {
        *lock(&self.needs_bypass) = needs;
    }
    /// Makes every step fail with this reason from now on.
    pub fn fail_with(&self, reason: Option<String>) {
        *lock(&self.failure) = reason;
@@ -138,6 +162,10 @@ fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
 }
 impl ExitHostRules for MockExitRules {
    fn needs_bypass(&self) -> bool {
        *lock(&self.needs_bypass)
    }
    fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport> {
        Box::pin(async move {
            lock(&self.calls).push(format!("apply:{}", plan.interface));
@@ -204,6 +232,8 @@ mod tests {
            interface: "tsun0".into(),
            offer: vec![range()],
            client: true,
            overlay: vec![range()],
            ..Default::default()
        };
        let report = rules.apply(&plan).await;
        assert_eq!(report.offer[0].1, RuleOutcome::Applied);
@@ -218,6 +248,8 @@ mod tests {
                interface: "tsun0".into(),
                offer: vec![range()],
                client: false,
                overlay: Vec::new(),
                ..Default::default()
            })
            .await;
        assert_eq!(report.forwarding, Some(false));
@@ -19,6 +19,10 @@
 use crate::BoxFuture;
 use std::sync::{Arc, Mutex};
 use super::exit::{ExitHostPlan, ExitHostReport, ExitHostRules};
 use super::macos_exit::{apply_plan, clear_all, run_tool};
 use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome};
 /// The limited-broadcast destination routed through the overlay interface.
@@ -60,14 +64,19 @@ fn firewall_todo(plan: &BroadcastRulesPlan) -> String {
    )
 }
-/// The macOS implementation: `route` for the route, no firewall yet.
+/// The macOS implementation: `route` for the route, no firewall yet, and
 /// `pf` plus `route` for exit nodes (see [`super::macos_exit`]).
 #[derive(Debug, Default)]
-pub struct MacosHostRules;
+pub struct MacosHostRules {
    /// The reference `pfctl -E` handed out, held while exit-node rules need
    /// `pf` on and released when they are removed.
    pub(super) pf_token: Arc<Mutex<Option<String>>>,
 }
 impl MacosHostRules {
    /// Creates the host rules.
    pub fn new() -> Self {
-        Self
+        Self::default()
    }
    /// Installs the route, replacing one this agent added earlier so the call
@@ -82,11 +91,46 @@ impl MacosHostRules {
    }
 }
 impl ExitHostRules for MacosHostRules {
    fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport> {
        Box::pin(async move {
            let job_plan = plan.clone();
            let token = Arc::clone(&self.pf_token);
            tokio::task::spawn_blocking(move || apply_plan(&run_tool, &token, &job_plan))
                .await
                .unwrap_or_else(|err| {
                    let failed = RuleOutcome::Failed(format!("the task failed: {err}"));
                    ExitHostReport {
                        offer: plan
                            .offer
                            .iter()
                            .map(|range| (*range, failed.clone()))
                            .collect(),
                        forwarding: None,
                        client: plan.client.then_some(failed),
                    }
                })
        })
    }
    fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
        Box::pin(async move {
            let name = interface.to_string();
            let token = Arc::clone(&self.pf_token);
            let _ = tokio::task::spawn_blocking(move || clear_all(&run_tool, &token, &name)).await;
        })
    }
 }
 impl BroadcastHostRules for MacosHostRules {
    fn name(&self) -> &str {
        "route"
    }
    fn exit_rules(&self) -> Option<&dyn ExitHostRules> {
        Some(self)
    }
    fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
        Box::pin(async move {
            let route = self.set_route(plan.interface.clone()).await;
@@ -0,0 +1,915 @@
 //! Exit-node host rules on macOS: a `pf` anchor and `route` entries.
 //!
 //! Compiled on every platform so the rule text and the parsers are tested
 //! everywhere; only `MacosHostRules` runs them, and only on macOS.
 //!
 //! # Offering
 //!
 //! Per offered range, in the anchor `com.apple/tsunagi-exit-<if>`:
 //!
 //! ```text
 //! nat on <egress> inet from <range> to any -> (<egress>)
 //! pass in quick on <if> inet from <range> to ! <range>
 //! ```
 //!
 //! The anchor lives under `com.apple/` on purpose: the stock `/etc/pf.conf`
 //! evaluates `nat-anchor "com.apple/*"` and `anchor "com.apple/*"`, so the
 //! rules take effect without editing the user's ruleset. `pf` cannot say
 //! "every interface but one" in a translation address, so the egress is the
 //! interface of the default route **when the rules are applied**; after the
 //! default route moves (Wi-Fi to Ethernet) the rules are re-applied by
 //! switching the exit node off and on, or restarting the agent.
 //!
 //! The kernel's `net.inet.ip.forwarding` is only read, never written: it
 //! changes how the whole host behaves, which is the owner's decision.
 //!
 //! # Using
 //!
 //! macOS has no `uidrange` policy routing, and the agent is root, so the
 //! Linux scheme does not carry over. Instead:
 //!
 //! ```text
 //! route add -net 0.0.0.0/1   -interface <if>
 //! route add -net 128.0.0.0/1 -interface <if>
 //! pass out quick on <if> route-to (<egress> <gateway>) inet proto { tcp udp }
 //!     from any to ! <overlay ranges> user <agent uid>
 //! nat on <egress> inet from (<if>) to any -> (<egress>)
 //! ```
 //!
 //! The two halves outrank the default route without replacing it, and every
 //! more specific route (LAN, another VPN) keeps working. The `route-to` rule
 //! sends the *agent's own* traffic — QUIC, relay, DHT, which would otherwise
 //! loop into the tunnel it carries — back out the physical interface, and the
 //! translation gives it that interface's address, since an unbound socket
 //! picked the overlay address when the route pointed at the overlay. Like the
 //! Linux exemption this is by user id, so anything else running as root also
 //! bypasses the tunnel.
 //!
 //! IPv6 is blocked while an exit node is in use, as on Linux: `::/1` and
 //! `8000::/1` are rejecting routes, so applications fall back to IPv4 at once
 //! instead of leaking.
 //!
 //! Every object is derived from local state and the interface the agent owns;
 //! none of it comes from a peer.
 use std::net::Ipv4Addr;
 use std::sync::Mutex;
 use crate::state::Ipv4Range;
 use super::RuleOutcome;
 use super::exit::{ExitHostPlan, ExitHostReport};
 /// The programs this module runs, always by absolute path.
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub(super) enum Tool {
    Route,
    Pfctl,
    Sysctl,
    Id,
 }
 impl Tool {
    fn path(self) -> &'static str {
        match self {
            Self::Route => "/sbin/route",
            Self::Pfctl => "/sbin/pfctl",
            Self::Sysctl => "/usr/sbin/sysctl",
            Self::Id => "/usr/bin/id",
        }
    }
 }
 /// What running a tool came to.
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub(super) enum Ran {
    /// The program is not there.
    Missing,
    /// It ran.
    Exited {
        success: bool,
        stdout: String,
        stderr: String,
    },
 }
 /// Runs a tool with optional standard input; injectable so the logic is
 /// tested without touching the host.
 pub(super) type Run<'a> = &'a dyn Fn(Tool, &[String], Option<&str>) -> Ran;
 /// Runs a real tool. Blocking: callers use `spawn_blocking`.
 pub(super) fn run_tool(tool: Tool, args: &[String], stdin: Option<&str>) -> Ran {
    use std::io::Write;
    use std::process::{Command, Stdio};
    let mut command = Command::new(tool.path());
    command
        .args(args)
        .stdin(if stdin.is_some() {
            Stdio::piped()
        } else {
            Stdio::null()
        })
        .stdout(Stdio::piped())
        .stderr(Stdio::piped());
    let mut child = match command.spawn() {
        Ok(child) => child,
        Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ran::Missing,
        Err(err) => {
            return Ran::Exited {
                success: false,
                stdout: String::new(),
                stderr: format!("could not run {}: {err}", tool.path()),
            };
        }
    };
    if let (Some(text), Some(mut pipe)) = (stdin, child.stdin.take()) {
        // A write error shows up as the tool's own failure below.
        let _ = pipe.write_all(text.as_bytes());
    }
    match child.wait_with_output() {
        Ok(output) => Ran::Exited {
            success: output.status.success(),
            stdout: String::from_utf8_lossy(&output.stdout).into_owned(),
            stderr: String::from_utf8_lossy(&output.stderr).into_owned(),
        },
        Err(err) => Ran::Exited {
            success: false,
            stdout: String::new(),
            stderr: format!("could not run {}: {err}", tool.path()),
        },
    }
 }
 fn owned(words: &[&str]) -> Vec<String> {
    words.iter().map(|word| word.to_string()).collect()
 }
 /// A name that is safe in a `pf` rule, an anchor path and a table name.
 fn safe_name(name: &str) -> bool {
    !name.is_empty() && name.len() <= 32 && name.chars().all(|c| c.is_ascii_alphanumeric())
 }
 /// The anchor holding every rule of one interface.
 fn anchor_name(interface: &str) -> Result<String, String> {
    if safe_name(interface) {
        Ok(format!("com.apple/tsunagi-exit-{interface}"))
    } else {
        Err(format!(
            "`{interface}` is not an interface name this agent can write a pf rule for"
        ))
    }
 }
 /// The default route as `route -n get default` reports it.
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub(super) struct DefaultRoute {
    pub interface: String,
    pub gateway: Option<Ipv4Addr>,
 }
 /// Reads `interface:` and `gateway:` out of `route -n get default`.
 fn parse_default_route(text: &str) -> Option<DefaultRoute> {
    let field = |name: &str| {
        text.lines().find_map(|line| {
            let (key, value) = line.split_once(':')?;
            (key.trim() == name).then(|| value.trim().to_string())
        })
    };
    let interface = field("interface").filter(|name| safe_name(name))?;
    let gateway = field("gateway").and_then(|value| value.parse().ok());
    Some(DefaultRoute { interface, gateway })
 }
 fn failure(stderr: &str, stdout: &str) -> String {
    let text = if stderr.trim().is_empty() {
        stdout.trim()
    } else {
        stderr.trim()
    };
    if text.is_empty() {
        "the command failed without saying why".to_string()
    } else {
        text.to_string()
    }
 }
 /// Runs a tool and turns a failure into a sentence a person can act on.
 fn run_ok(
    run: Run<'_>,
    tool: Tool,
    args: &[String],
    stdin: Option<&str>,
 ) -> Result<String, String> {
    match run(tool, args, stdin) {
        Ran::Missing => Err(format!("{} not found", tool.path())),
        Ran::Exited {
            success: true,
            stdout,
            ..
        } => Ok(stdout),
        Ran::Exited { stderr, stdout, .. } => Err(format!(
            "{} {} failed: {}",
            tool.path(),
            args.join(" "),
            failure(&stderr, &stdout)
        )),
    }
 }
 /// The route traffic leaves by now, which is the egress for translation and
 /// for the agent's own sockets. Read *before* the overlay routes are added.
 fn default_route(run: Run<'_>, overlay: &str) -> Result<DefaultRoute, String> {
    let out = run_ok(run, Tool::Route, &owned(&["-n", "get", "default"]), None)
        .map_err(|why| format!("cannot find the default route: {why}"))?;
    let route = parse_default_route(&out)
        .ok_or_else(|| "the host has no default route to send traffic out of".to_string())?;
    if route.interface == overlay {
        return Err(format!(
            "the default route already goes through `{overlay}`"
        ));
    }
    Ok(route)
 }
 /// Whether the kernel forwards IPv4. Read only.
 fn forwarding_state(run: Run<'_>) -> Option<bool> {
    let out = run_ok(
        run,
        Tool::Sysctl,
        &owned(&["-n", "net.inet.ip.forwarding"]),
        None,
    )
    .ok()?;
    match out.trim() {
        "1" => Some(true),
        "0" => Some(false),
        _ => None,
    }
 }
 /// The effective user id of this process, from `id -u`.
 fn agent_uid(run: Run<'_>) -> Result<u32, String> {
    run_ok(run, Tool::Id, &owned(&["-u"]), None)
        .ok()
        .and_then(|out| out.trim().parse().ok())
        .ok_or_else(|| "cannot tell which user this agent runs as".to_string())
 }
 /// What the using side adds to the rules.
 #[derive(Debug, Clone, PartialEq, Eq)]
 struct ClientRules<'a> {
    uid: u32,
    /// Every overlay range: the agent's own traffic *to* those still goes
    /// through the interface.
    overlay: &'a [Ipv4Range],
 }
 /// The text of the anchor: translation first, filtering after, which is the
 /// order `pf` insists on.
 fn pf_rules(
    interface: &str,
    egress: &DefaultRoute,
    offer: &[Ipv4Range],
    client: Option<&ClientRules<'_>>,
 ) -> String {
    let out = &egress.interface;
    let mut nat = Vec::new();
    let mut filter = Vec::new();
    for range in offer {
        nat.push(format!("nat on {out} inet from {range} to any -> ({out})"));
        filter.push(format!(
            "pass in quick on {interface} inet from {range} to ! {range}"
        ));
    }
    let mut tables = Vec::new();
    if let Some(client) = client {
        nat.push(format!(
            "nat on {out} inet from ({interface}) to any -> ({out})"
        ));
        let target = if client.overlay.is_empty() {
            "any".to_string()
        } else {
            let ranges: Vec<String> = client.overlay.iter().map(|r| r.to_string()).collect();
            // A table, because `! { a, b }` expands to one rule per element
            // in older `pf`, which matches everything.
            tables.push(format!(
                "table <tsunagi_overlay_{interface}> const {{ {} }}",
                ranges.join(", ")
            ));
            format!("! <tsunagi_overlay_{interface}>")
        };
        let via = match egress.gateway {
            Some(gateway) => format!("({out} {gateway})"),
            None => out.clone(),
        };
        filter.push(format!(
            "pass out quick on {interface} route-to {via} inet proto {{ tcp udp }} \
             from any to {target} user {}",
            client.uid
        ));
    }
    tables
        .into_iter()
        .chain(nat)
        .chain(filter)
        .collect::<Vec<_>>()
        .join("\n")
        + "\n"
 }
 /// Turns `pf` on if nobody has, holding the reference `pfctl` hands out so it
 /// can be released when the agent is done. One reference at most.
 fn acquire_pf(run: Run<'_>, slot: &Mutex<Option<String>>) -> Result<(), String> {
    if lock(slot).is_some() {
        return Ok(());
    }
    match run(Tool::Pfctl, &owned(&["-E"]), None) {
        Ran::Missing => Err(format!("{} not found", Tool::Pfctl.path())),
        Ran::Exited {
            success,
            stdout,
            stderr,
        } => {
            // `pfctl -E` reports `Token : <n>` on stderr, and also says so
            // when pf is already enabled.
            let token = stdout.lines().chain(stderr.lines()).find_map(parse_token);
            if !success && token.is_none() {
                return Err(format!("cannot enable pf: {}", failure(&stderr, &stdout)));
            }
            *lock(slot) = token;
            Ok(())
        }
    }
 }
 fn parse_token(line: &str) -> Option<String> {
    let (key, value) = line.split_once(':')?;
    let value = value.trim();
    (key.trim().eq_ignore_ascii_case("token")
        && !value.is_empty()
        && value.chars().all(|c| c.is_ascii_digit()))
    .then(|| value.to_string())
 }
 fn release_pf(run: Run<'_>, slot: &Mutex<Option<String>>) {
    if let Some(token) = lock(slot).take()
        && let Err(why) = run_ok(run, Tool::Pfctl, &["-X".to_string(), token], None)
    {
        tracing::debug!(%why, "cannot release the pf reference");
    }
 }
 fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
    match mutex.lock() {
        Ok(guard) => guard,
        Err(poisoned) => poisoned.into_inner(),
    }
 }
 fn flush_args(anchor: &str) -> Vec<String> {
    owned(&["-q", "-a", anchor, "-F", "all"])
 }
 fn load_args(anchor: &str) -> Vec<String> {
    owned(&["-q", "-a", anchor, "-f", "-"])
 }
 /// The two halves that out-rank the default route.
 const HALVES: [&str; 2] = ["0.0.0.0/1", "128.0.0.0/1"];
 /// The IPv6 halves, made unreachable.
 const HALVES_V6: [&str; 2] = ["::/1", "8000::/1"];
 fn route_add_args(half: &str, interface: &str) -> Vec<String> {
    owned(&["-q", "-n", "add", "-net", half, "-interface", interface])
 }
 fn route_del_args(half: &str, interface: &str) -> Vec<String> {
    owned(&["-q", "-n", "delete", "-net", half, "-interface", interface])
 }
 fn route_add_v6_args(half: &str) -> Vec<String> {
    owned(&["-q", "-n", "add", "-inet6", "-net", half, "::1", "-reject"])
 }
 fn route_del_v6_args(half: &str) -> Vec<String> {
    owned(&["-q", "-n", "delete", "-inet6", "-net", half, "::1"])
 }
 /// Takes away the overlay default routes. Best effort: what is not there is
 /// the state wanted.
 fn clear_routes(run: Run<'_>, interface: &str) {
    for half in HALVES {
        let _ = run(Tool::Route, &route_del_args(half, interface), None);
    }
    for half in HALVES_V6 {
        let _ = run(Tool::Route, &route_del_v6_args(half), None);
    }
 }
 fn set_routes(run: Run<'_>, interface: &str) -> Result<(), String> {
    clear_routes(run, interface);
    for half in HALVES {
        if let Err(why) = run_ok(run, Tool::Route, &route_add_args(half, interface), None) {
            clear_routes(run, interface);
            return Err(why);
        }
    }
    for half in HALVES_V6 {
        // A host without IPv6 has nothing to block; not a failure.
        if let Err(why) = run_ok(run, Tool::Route, &route_add_v6_args(half), None) {
            tracing::debug!(%why, "cannot block IPv6 for the exit node");
        }
    }
    Ok(())
 }
 /// Makes the host match the plan.
 pub(super) fn apply_plan(
    run: Run<'_>,
    token: &Mutex<Option<String>>,
    plan: &ExitHostPlan,
 ) -> ExitHostReport {
    let forwarding = if plan.offer.is_empty() {
        None
    } else {
        forwarding_state(run)
    };
    let outcome = apply_rules(run, token, plan);
    let verdict = |ok: bool, why: &str| {
        if ok {
            RuleOutcome::Applied
        } else {
            RuleOutcome::Failed(why.to_string())
        }
    };
    match outcome {
        Ok(()) => ExitHostReport {
            offer: plan
                .offer
                .iter()
                .map(|range| (*range, RuleOutcome::Applied))
                .collect(),
            forwarding,
            client: plan.client.then_some(RuleOutcome::Applied),
        },
        Err(why) => ExitHostReport {
            offer: plan
                .offer
                .iter()
                .map(|range| (*range, verdict(false, &why)))
                .collect(),
            forwarding,
            client: plan.client.then(|| verdict(false, &why)),
        },
    }
 }
 fn apply_rules(
    run: Run<'_>,
    token: &Mutex<Option<String>>,
    plan: &ExitHostPlan,
 ) -> Result<(), String> {
    let interface = plan.interface.as_str();
    let anchor = anchor_name(interface)?;
    let egress = default_route(run, interface)?;
    let uid = if plan.client {
        Some(agent_uid(run)?)
    } else {
        None
    };
    let client = uid.map(|uid| ClientRules {
        uid,
        overlay: &plan.overlay,
    });
    let rules = pf_rules(interface, &egress, &plan.offer, client.as_ref());
    acquire_pf(run, token)?;
    run_ok(run, Tool::Pfctl, &load_args(&anchor), Some(&rules))
        .map_err(|why| format!("cannot load the pf rules: {why}"))?;
    // The routes come after the rules that keep the agent's own traffic off
    // them: the other order would loop for as long as it takes.
    if plan.client {
        set_routes(run, interface)?;
    } else {
        clear_routes(run, interface);
    }
    Ok(())
 }
 /// Removes everything this agent put there for the interface.
 pub(super) fn clear_all(run: Run<'_>, token: &Mutex<Option<String>>, interface: &str) {
    clear_routes(run, interface);
    if let Ok(anchor) = anchor_name(interface) {
        // Flushing an anchor nothing was ever loaded into is not an error
        // worth reporting.
        if let Err(why) = run_ok(run, Tool::Pfctl, &flush_args(&anchor), None) {
            tracing::debug!(%why, "cannot flush the exit node pf anchor");
        }
    }
    release_pf(run, token);
 }
 #[cfg(test)]
 mod tests {
    #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
    use std::cell::RefCell;
    use super::*;
    fn range() -> Ipv4Range {
        "10.13.37.0/24".parse().unwrap()
    }
    const DEFAULT_ROUTE: &str = "   route to: default\n\
 destination: default\n\
       mask: default\n\
    gateway: 192.168.1.1\n\
  interface: en0\n\
      flags: <UP,GATEWAY,DONE,STATIC,PRCLONING,GLOBAL>\n";
    fn egress() -> DefaultRoute {
        DefaultRoute {
            interface: "en0".into(),
            gateway: Some(Ipv4Addr::new(192, 168, 1, 1)),
        }
    }
    /// A scripted host that records every call.
    struct Fake {
        calls: RefCell<Vec<String>>,
        stdins: RefCell<Vec<String>>,
        default_route: Option<String>,
        forwarding: &'static str,
        pf_missing: bool,
        pf_load_fails: bool,
        route_add_fails: bool,
        pf_enable_output: &'static str,
    }
    impl Fake {
        fn new() -> Self {
            Self {
                calls: RefCell::new(Vec::new()),
                stdins: RefCell::new(Vec::new()),
                default_route: Some(DEFAULT_ROUTE.to_string()),
                forwarding: "1\n",
                pf_missing: false,
                pf_load_fails: false,
                route_add_fails: false,
                pf_enable_output: "pf enabled\nToken : 4242\n",
            }
        }
        fn run(&self, tool: Tool, args: &[String], stdin: Option<&str>) -> Ran {
            let line = format!("{:?} {}", tool, args.join(" "));
            self.calls.borrow_mut().push(line);
            if let Some(text) = stdin {
                self.stdins.borrow_mut().push(text.to_string());
            }
            let ok = |stdout: &str, stderr: &str| Ran::Exited {
                success: true,
                stdout: stdout.to_string(),
                stderr: stderr.to_string(),
            };
            let fail = |why: &str| Ran::Exited {
                success: false,
                stdout: String::new(),
                stderr: why.to_string(),
            };
            match tool {
                Tool::Id => ok("0\n", ""),
                Tool::Sysctl => ok(self.forwarding, ""),
                Tool::Route if args.first().map(String::as_str) == Some("-n") => {
                    match &self.default_route {
                        Some(text) => ok(text, ""),
                        None => fail("route: writing to routing socket: not in table"),
                    }
                }
                Tool::Route if args.contains(&"add".to_string()) && self.route_add_fails => {
                    fail("route: writing to routing socket: File exists")
                }
                Tool::Route => ok("", ""),
                Tool::Pfctl if self.pf_missing => Ran::Missing,
                Tool::Pfctl if args == ["-E"] => ok("", self.pf_enable_output),
                Tool::Pfctl if args.contains(&"-f".to_string()) && self.pf_load_fails => {
                    fail("pfctl: Syntax error")
                }
                Tool::Pfctl => ok("", ""),
            }
        }
        fn calls(&self) -> Vec<String> {
            self.calls.borrow().clone()
        }
        fn position(&self, needle: &str) -> Option<usize> {
            self.calls.borrow().iter().position(|c| c.contains(needle))
        }
        fn count(&self, needle: &str) -> usize {
            self.calls
                .borrow()
                .iter()
                .filter(|c| c.contains(needle))
                .count()
        }
    }
    fn plan(offer: bool, client: bool) -> ExitHostPlan {
        ExitHostPlan {
            interface: "utun5".into(),
            offer: if offer { vec![range()] } else { Vec::new() },
            client,
            overlay: vec![range()],
            ..Default::default()
        }
    }
    #[test]
    fn the_default_route_is_read_from_route_output() {
        assert_eq!(parse_default_route(DEFAULT_ROUTE), Some(egress()));
        // A point-to-point default has no gateway line to use.
        let ppp = "destination: default\n  interface: ppp0\n";
        assert_eq!(
            parse_default_route(ppp),
            Some(DefaultRoute {
                interface: "ppp0".into(),
                gateway: None
            })
        );
        assert_eq!(parse_default_route("route: not in table"), None);
        // An interface name that is not plain letters and digits never ends
        // up in a rule.
        assert_eq!(parse_default_route("interface: en0; pass all\n"), None);
    }
    #[test]
    fn the_anchor_is_under_com_apple_and_only_for_plain_names() {
        assert_eq!(
            anchor_name("utun5").unwrap(),
            "com.apple/tsunagi-exit-utun5"
        );
        assert!(anchor_name("").is_err());
        assert!(anchor_name("utun5 pass all").is_err());
        assert!(anchor_name("../x").is_err());
    }
    #[test]
    fn offering_translates_the_range_and_admits_it_in() {
        let text = pf_rules("utun5", &egress(), &[range()], None);
        assert_eq!(
            text,
            "nat on en0 inet from 10.13.37.0/24 to any -> (en0)\n\
             pass in quick on utun5 inet from 10.13.37.0/24 to ! 10.13.37.0/24\n"
        );
        // Translation precedes filtering, as pf requires.
        assert!(text.find("nat on").unwrap() < text.find("pass in").unwrap());
    }
    #[test]
    fn using_sends_the_agents_own_traffic_back_out_the_physical_interface() {
        let other: Ipv4Range = "10.99.0.0/16".parse().unwrap();
        let overlay = [range(), other];
        let client = ClientRules {
            uid: 0,
            overlay: &overlay,
        };
        let text = pf_rules("utun5", &egress(), &[], Some(&client));
        assert!(
            text.starts_with(
                "table <tsunagi_overlay_utun5> const { 10.13.37.0/24, 10.99.0.0/16 }\n"
            ),
            "{text}"
        );
        assert!(
            text.contains("nat on en0 inet from (utun5) to any -> (en0)\n"),
            "{text}"
        );
        assert!(
            text.contains(
                "pass out quick on utun5 route-to (en0 192.168.1.1) inet proto { tcp udp } \
                 from any to ! <tsunagi_overlay_utun5> user 0\n"
            ),
            "{text}"
        );
        // Only the agent's user, and only what is not the overlay.
        assert!(!text.contains("from any to any"));
        let direct = DefaultRoute {
            interface: "ppp0".into(),
            gateway: None,
        };
        let text = pf_rules(
            "utun5",
            &direct,
            &[],
            Some(&ClientRules {
                uid: 501,
                overlay: &[],
            }),
        );
        assert!(text.contains("route-to ppp0 inet"), "{text}");
        assert!(text.contains("to any user 501"), "{text}");
        assert!(!text.contains("table"), "{text}");
    }
    #[test]
    fn the_overlay_default_routes_are_halves_and_ipv6_is_rejected() {
        assert_eq!(
            route_add_args("0.0.0.0/1", "utun5").join(" "),
            "-q -n add -net 0.0.0.0/1 -interface utun5"
        );
        assert_eq!(
            route_del_args("128.0.0.0/1", "utun5").join(" "),
            "-q -n delete -net 128.0.0.0/1 -interface utun5"
        );
        assert_eq!(
            route_add_v6_args("8000::/1").join(" "),
            "-q -n add -inet6 -net 8000::/1 ::1 -reject"
        );
        assert_eq!(
            route_del_v6_args("::/1").join(" "),
            "-q -n delete -inet6 -net ::/1 ::1"
        );
    }
    #[test]
    fn the_pf_token_is_found_in_either_stream() {
        assert_eq!(parse_token("Token : 1234"), Some("1234".into()));
        assert_eq!(parse_token("token: 9"), Some("9".into()));
        assert_eq!(parse_token("pf enabled"), None);
        assert_eq!(parse_token("Token : x; rm"), None);
    }
    #[test]
    fn offering_loads_the_anchor_and_reads_but_never_writes_forwarding() {
        let fake = Fake::new();
        let token = Mutex::new(None);
        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
        assert_eq!(report.offer, vec![(range(), RuleOutcome::Applied)]);
        assert_eq!(report.forwarding, Some(true));
        assert_eq!(report.client, None);
        let calls = fake.calls();
        assert!(
            calls
                .iter()
                .any(|c| c == "Pfctl -q -a com.apple/tsunagi-exit-utun5 -f -"),
            "{calls:?}"
        );
        assert_eq!(
            fake.stdins.borrow()[0],
            "nat on en0 inet from 10.13.37.0/24 to any -> (en0)\n\
             pass in quick on utun5 inet from 10.13.37.0/24 to ! 10.13.37.0/24\n"
        );
        // sysctl is only ever asked, never told.
        assert!(
            calls
                .iter()
                .filter(|c| c.starts_with("Sysctl"))
                .all(|c| c == "Sysctl -n net.inet.ip.forwarding"),
            "{calls:?}"
        );
        // No client routes were added for an offer alone.
        assert_eq!(fake.count(" add "), 0);
        assert_eq!(token.lock().unwrap().as_deref(), Some("4242"));
    }
    #[test]
    fn forwarding_off_is_reported_and_unreadable_is_none() {
        let mut fake = Fake::new();
        fake.forwarding = "0\n";
        let token = Mutex::new(None);
        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
        assert_eq!(report.forwarding, Some(false));
        fake.forwarding = "junk";
        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
        assert_eq!(report.forwarding, None);
    }
    #[test]
    fn the_client_rules_are_loaded_before_the_routes_that_need_them() {
        let fake = Fake::new();
        let token = Mutex::new(None);
        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(false, true));
        assert_eq!(report.client, Some(RuleOutcome::Applied));
        assert!(report.offer.is_empty());
        let load = fake.position("-f -").unwrap();
        let add = fake.position(" add -net 0.0.0.0/1").unwrap();
        assert!(load < add, "{:?}", fake.calls());
        assert!(fake.position("add -net 128.0.0.0/1").is_some());
        assert!(fake.position("-inet6 -net ::/1 ::1 -reject").is_some());
        assert!(fake.stdins.borrow()[0].contains("user 0"));
        // The default route was read before any overlay route existed.
        assert!(fake.position("-n get default").unwrap() < add);
    }
    #[test]
    fn no_routes_are_added_when_the_rules_did_not_load() {
        let mut fake = Fake::new();
        fake.pf_load_fails = true;
        let token = Mutex::new(None);
        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, true));
        let Some(RuleOutcome::Failed(why)) = report.client else {
            panic!("{report:?}");
        };
        assert!(why.contains("cannot load the pf rules"), "{why}");
        assert!(why.contains("Syntax error"), "{why}");
        assert!(matches!(report.offer[0].1, RuleOutcome::Failed(_)));
        // Nothing sent the host's traffic into a tunnel it could not exempt
        // the agent from.
        assert_eq!(fake.count(" add "), 0);
    }
    #[test]
    fn a_failing_route_is_a_failure_and_leaves_nothing_behind() {
        let mut fake = Fake::new();
        fake.route_add_fails = true;
        let token = Mutex::new(None);
        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(false, true));
        let Some(RuleOutcome::Failed(why)) = report.client else {
            panic!("{report:?}");
        };
        assert!(why.contains("File exists"), "{why}");
        // The cleanup after the failed add removed whatever did land.
        let last_add = fake
            .calls()
            .iter()
            .rposition(|c| c.contains(" add "))
            .unwrap();
        assert!(
            fake.calls()[last_add..]
                .iter()
                .any(|c| c.contains("delete"))
        );
    }
    #[test]
    fn a_missing_pfctl_or_default_route_says_what_is_wrong() {
        let mut fake = Fake::new();
        fake.pf_missing = true;
        let token = Mutex::new(None);
        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
        let RuleOutcome::Failed(why) = &report.offer[0].1 else {
            panic!("{report:?}");
        };
        assert!(why.contains("/sbin/pfctl not found"), "{why}");
        let mut fake = Fake::new();
        fake.default_route = None;
        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
        let RuleOutcome::Failed(why) = &report.offer[0].1 else {
            panic!("{report:?}");
        };
        assert!(why.contains("default route"), "{why}");
        assert_eq!(fake.count("-f -"), 0);
    }
    #[test]
    fn a_default_route_through_the_overlay_is_refused() {
        let mut fake = Fake::new();
        fake.default_route = Some("gateway: 10.13.37.1\ninterface: utun5\n".into());
        let token = Mutex::new(None);
        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
        let RuleOutcome::Failed(why) = &report.offer[0].1 else {
            panic!("{report:?}");
        };
        assert!(why.contains("already goes through"), "{why}");
    }
    #[test]
    fn pf_is_enabled_once_and_released_once_on_clear() {
        let fake = Fake::new();
        let token = Mutex::new(None);
        let run = |t, a: &[String], s: Option<&str>| fake.run(t, a, s);
        apply_plan(&run, &token, &plan(true, false));
        apply_plan(&run, &token, &plan(true, true));
        assert_eq!(fake.count("Pfctl -E"), 1, "{:?}", fake.calls());
        clear_all(&run, &token, "utun5");
        assert_eq!(fake.count("Pfctl -X 4242"), 1);
        assert!(
            fake.position("-a com.apple/tsunagi-exit-utun5 -F all")
                .is_some()
        );
        assert!(
            fake.position("delete -net 0.0.0.0/1 -interface utun5")
                .is_some()
        );
        assert_eq!(*token.lock().unwrap(), None);
        // A second clear has nothing to release and still succeeds.
        clear_all(&run, &token, "utun5");
        assert_eq!(fake.count("Pfctl -X"), 1);
    }
    #[test]
    fn dropping_the_client_removes_its_routes() {
        let fake = Fake::new();
        let token = Mutex::new(None);
        let run = |t, a: &[String], s: Option<&str>| fake.run(t, a, s);
        apply_plan(&run, &token, &plan(true, false));
        assert!(fake.position("delete -net 0.0.0.0/1").is_some());
        assert_eq!(fake.count(" add "), 0);
    }
 }
@@ -61,11 +61,16 @@ pub use linux::LinuxHostRules;
 // Compiled everywhere so the scripts it builds are tested everywhere; only
 // the type that runs them is exported on Windows.
 mod windows;
 #[cfg_attr(not(target_os = "windows"), allow(dead_code))]
 mod windows_exit;
 #[cfg(all(feature = "tun-device", target_os = "windows"))]
 pub use windows::WindowsHostRules;
 // Compiled everywhere so its rule text and parsers are tested everywhere.
 #[cfg(all(feature = "tun-device", target_os = "macos"))]
 mod macos;
 #[cfg_attr(not(target_os = "macos"), allow(dead_code))]
 mod macos_exit;
 #[cfg(all(feature = "tun-device", target_os = "macos"))]
 pub use macos::MacosHostRules;
@@ -29,6 +29,8 @@
 use crate::BoxFuture;
 use super::exit::{ExitHostPlan, ExitHostReport, ExitHostRules};
 use super::windows_exit;
 use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome, tag_for};
 /// A PowerShell single-quoted literal, with any embedded quote doubled.
@@ -113,6 +115,11 @@ fn powershell_path() -> std::path::PathBuf {
 /// Runs a script and returns what it said on failure.
 fn run_powershell(script: &str) -> Result<(), String> {
    run_powershell_output(script).map(|_| ())
 }
 /// Runs a script and returns what it printed, or what it said on failure.
 fn run_powershell_output(script: &str) -> Result<String, String> {
    let output = std::process::Command::new(powershell_path())
        .args([
            "-NoProfile",
@@ -125,7 +132,7 @@ fn run_powershell(script: &str) -> Result<(), String> {
        .output()
        .map_err(|err| format!("could not run PowerShell: {err}"))?;
    if output.status.success() {
-        return Ok(());
+        return Ok(String::from_utf8_lossy(&output.stdout).into_owned());
    }
    let stderr = String::from_utf8_lossy(&output.stderr);
    let text = if stderr.trim().is_empty() {
@@ -155,11 +162,73 @@ impl WindowsHostRules {
    }
 }
 /// The IPv4 addresses a host name resolves to; empty when it does not.
 fn resolve_host(host: &str) -> Vec<std::net::Ipv4Addr> {
    use std::net::{IpAddr, ToSocketAddrs};
    (host, 443)
        .to_socket_addrs()
        .map(|addrs| {
            addrs
                .filter_map(|addr| match addr.ip() {
                    IpAddr::V4(ip) => Some(ip),
                    IpAddr::V6(_) => None,
                })
                .collect()
        })
        .unwrap_or_default()
 }
 impl ExitHostRules for WindowsHostRules {
    fn needs_bypass(&self) -> bool {
        true
    }
    fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport> {
        Box::pin(async move {
            let job = plan.clone();
            tokio::task::spawn_blocking(move || {
                windows_exit::apply_plan(
                    &|script| run_powershell_output(script),
                    &resolve_host,
                    &job,
                )
            })
            .await
            .unwrap_or_else(|err| {
                let failed = RuleOutcome::Failed(format!("the task failed: {err}"));
                ExitHostReport {
                    offer: plan
                        .offer
                        .iter()
                        .map(|range| (*range, failed.clone()))
                        .collect(),
                    forwarding: None,
                    client: plan.client.then_some(failed),
                }
            })
        })
    }
    fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
        Box::pin(async move {
            let name = interface.to_string();
            let _ = tokio::task::spawn_blocking(move || {
                windows_exit::clear_all(&|script| run_powershell_output(script), &name)
            })
            .await;
        })
    }
 }
 impl BroadcastHostRules for WindowsHostRules {
    fn name(&self) -> &str {
        "windows-netsecurity"
    }
    fn exit_rules(&self) -> Option<&dyn ExitHostRules> {
        Some(self)
    }
    fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
        Box::pin(async move {
            let route = step("route", route_script(plan)).await;
@@ -0,0 +1,603 @@
 //! Exit-node host rules on Windows: WinNAT through PowerShell.
 //!
 //! Compiled on every platform so the scripts and the plan logic are tested
 //! everywhere; only `WindowsHostRules` runs them, and only on Windows. It
 //! could not be run against a real Windows host while it was written.
 //!
 //! # Offering
 //!
 //! One NAT object per offered range, named with the usual tag
 //! `tsunagi-exit:<if>:<range>`:
 //!
 //! ```text
 //! New-NetNat -Name <tag> -InternalIPInterfaceAddressPrefix <range>
 //! ```
 //!
 //! WinNAT translates that range out of whichever interface the traffic leaves
 //! by, which is the Windows counterpart of the Linux masquerade. Windows
 //! forwards per interface, and the egress is not known here, so forwarding is
 //! only *read* for the overlay interface; turning it on for the interfaces
 //! involved is the owner's decision (`Set-NetIPInterface -Forwarding Enabled`).
 //! Some Windows versions allow a single NAT per host: a second offered range
 //! then reports its own failure.
 //!
 //! # Using
 //!
 //! Windows has no per-user or per-process routing, so the Linux scheme (an
 //! exemption for the agent's user) does not carry over. Instead the agent's
 //! own underlay traffic is exempted by *destination*:
 //!
 //! ```text
 //! New-NetRoute 0.0.0.0/1   on the overlay interface, metric 1
 //! New-NetRoute 128.0.0.0/1 on the overlay interface, metric 1
 //! New-NetRoute <ip>/32 via the physical default gateway, metric 7373
 //! ```
 //!
 //! The halves out-rank the default route without replacing it, so every more
 //! specific route (LAN, another VPN) keeps working. A `/32` per address the
 //! agent talks to directly (the verified direct paths of its connections, and
 //! the relays it uses, resolved when the rules are applied) keeps that traffic
 //! on the physical network instead of looping into the tunnel it carries. The
 //! metric 7373 marks those routes as ours, which is how stale ones are found
 //! and removed. Everything is written to the active store, so a reboot clears
 //! it. IPv6 is blackholed with `::/1` and `8000::/1` on the overlay interface,
 //! which has no IPv6: applications fall back to IPv4 after a delay, where
 //! Linux and macOS answer at once. The bypass follows the agent's paths as
 //! they change; a path that appears after the last apply is reached through
 //! the tunnel until the next one.
 //!
 //! The bypass routes are written before the halves, and removed after them.
 use super::RuleOutcome;
 use std::net::Ipv4Addr;
 use super::exit::{ExitHostPlan, ExitHostReport, exit_tag, exit_tag_prefix};
 use crate::state::Ipv4Range;
 /// The route metric that marks a bypass route as this agent's.
 const BYPASS_METRIC: u32 = 7373;
 /// Runs a PowerShell script and returns what it printed.
 pub(super) type RunScript<'a> = &'a dyn Fn(&str) -> Result<String, String>;
 /// Resolves a host name to its IPv4 addresses.
 pub(super) type Resolve<'a> = &'a dyn Fn(&str) -> Vec<Ipv4Addr>;
 /// A PowerShell single-quoted literal, with any embedded quote doubled.
 fn ps_literal(value: &str) -> String {
    format!("'{}'", value.replace('\'', "''"))
 }
 /// Creates the NAT of one range, replacing one this agent made before.
 fn nat_script(interface: &str, range: &Ipv4Range) -> String {
    let name = ps_literal(&exit_tag(interface, range));
    let prefix = ps_literal(&range.to_string());
    format!(
        "$ErrorActionPreference = 'Stop'\n\
         $name = {name}\n\
         Get-NetNat -Name $name -ErrorAction SilentlyContinue | \
         Remove-NetNat -Confirm:$false -ErrorAction SilentlyContinue\n\
         New-NetNat -Name $name -InternalIPInterfaceAddressPrefix {prefix} | Out-Null\n"
    )
 }
 /// Removes this interface's NAT objects except those named in `keep`.
 fn prune_script(interface: &str, keep: &[Ipv4Range]) -> String {
    let prefix = ps_literal(&exit_tag_prefix(interface));
    let keep: Vec<String> = keep
        .iter()
        .map(|range| ps_literal(&exit_tag(interface, range)))
        .collect();
    format!(
        "$ErrorActionPreference = 'SilentlyContinue'\n\
         $prefix = {prefix}\n\
         $keep = @({})\n\
         Get-NetNat | Where-Object {{ $_.Name.StartsWith($prefix) -and \
         ($keep -notcontains $_.Name) }} | Remove-NetNat -Confirm:$false\n\
         exit 0\n",
        keep.join(", ")
    )
 }
 /// Prints `Enabled` or `Disabled` for the interface's IPv4 forwarding.
 fn forwarding_script(interface: &str) -> String {
    let alias = ps_literal(interface);
    format!(
        "$ErrorActionPreference = 'Stop'\n\
         (Get-NetIPInterface -InterfaceAlias {alias} -AddressFamily IPv4).Forwarding\n"
    )
 }
 /// Prints `<interface index> <next hop>` of the best default route that is
 /// not through the overlay interface.
 fn gateway_script(interface: &str) -> String {
    let alias = ps_literal(interface);
    format!(
        "$ErrorActionPreference = 'Stop'\n\
         $alias = {alias}\n\
         $best = Get-NetRoute -DestinationPrefix '0.0.0.0/0' -AddressFamily IPv4 \
         -PolicyStore ActiveStore | Where-Object {{ $_.InterfaceAlias -ne $alias -and \
         $_.NextHop -ne '0.0.0.0' }} | Sort-Object {{ $_.RouteMetric + \
         (Get-NetIPInterface -InterfaceIndex $_.InterfaceIndex -AddressFamily IPv4).InterfaceMetric }} | \
         Select-Object -First 1\n\
         if (-not $best) {{ throw 'the host has no default gateway outside the overlay' }}\n\
         \"$($best.InterfaceIndex) $($best.NextHop)\"\n"
    )
 }
 /// The default gateway as the script above printed it.
 fn parse_gateway(text: &str) -> Option<(u32, Ipv4Addr)> {
    let mut words = text.split_whitespace();
    let index = words.next()?.parse().ok()?;
    let hop = words.next()?.parse().ok()?;
    words.next().is_none().then_some((index, hop))
 }
 /// Replaces the bypass routes with one per address, through the gateway.
 fn bypass_script(index: u32, gateway: Ipv4Addr, addresses: &[Ipv4Addr]) -> String {
    let list: Vec<String> = addresses
        .iter()
        .map(|address| ps_literal(&address.to_string()))
        .collect();
    format!(
        "$ErrorActionPreference = 'Stop'\n\
         Get-NetRoute -AddressFamily IPv4 -PolicyStore ActiveStore -ErrorAction SilentlyContinue | \
         Where-Object {{ $_.RouteMetric -eq {BYPASS_METRIC} -and $_.DestinationPrefix.EndsWith('/32') }} | \
         Remove-NetRoute -Confirm:$false -ErrorAction SilentlyContinue\n\
         foreach ($ip in @({})) {{\n\
         New-NetRoute -DestinationPrefix \"$ip/32\" -InterfaceIndex {index} -NextHop '{gateway}' \
         -RouteMetric {BYPASS_METRIC} -PolicyStore ActiveStore -ErrorAction SilentlyContinue | Out-Null\n\
         }}\n",
        list.join(", ")
    )
 }
 /// Sends everything else through the overlay interface.
 fn halves_script(interface: &str) -> String {
    let alias = ps_literal(interface);
    format!(
        "$ErrorActionPreference = 'Stop'\n\
         $alias = {alias}\n\
         foreach ($p in '0.0.0.0/1', '128.0.0.0/1') {{\n\
         Get-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -PolicyStore ActiveStore \
         -ErrorAction SilentlyContinue | Remove-NetRoute -Confirm:$false -ErrorAction SilentlyContinue\n\
         New-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -NextHop '0.0.0.0' \
         -RouteMetric 1 -PolicyStore ActiveStore | Out-Null\n\
         }}\n\
         foreach ($p in '::/1', '8000::/1') {{\n\
         Get-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -PolicyStore ActiveStore \
         -ErrorAction SilentlyContinue | Remove-NetRoute -Confirm:$false -ErrorAction SilentlyContinue\n\
         New-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -NextHop '::' \
         -RouteMetric 1 -PolicyStore ActiveStore -ErrorAction SilentlyContinue | Out-Null\n\
         }}\n"
    )
 }
 /// Removes the halves and every bypass route. What is not there is the state
 /// wanted.
 fn remove_client_script(interface: &str) -> String {
    let alias = ps_literal(interface);
    format!(
        "$ErrorActionPreference = 'SilentlyContinue'\n\
         $alias = {alias}\n\
         foreach ($p in '0.0.0.0/1', '128.0.0.0/1', '::/1', '8000::/1') {{\n\
         Get-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -PolicyStore ActiveStore | \
         Remove-NetRoute -Confirm:$false\n\
         }}\n\
         Get-NetRoute -AddressFamily IPv4 -PolicyStore ActiveStore | \
         Where-Object {{ $_.RouteMetric -eq {BYPASS_METRIC} -and $_.DestinationPrefix.EndsWith('/32') }} | \
         Remove-NetRoute -Confirm:$false\n\
         exit 0\n"
    )
 }
 /// Every address to keep direct: the ones given and what the names resolve
 /// to, sorted and without duplicates.
 fn bypass_addresses(
    resolve: Resolve<'_>,
    addresses: &[Ipv4Addr],
    hosts: &[String],
 ) -> Vec<Ipv4Addr> {
    let mut all: std::collections::BTreeSet<Ipv4Addr> = addresses.iter().copied().collect();
    for host in hosts {
        all.extend(resolve(host));
    }
    all.into_iter()
        .filter(|a| !a.is_loopback() && !a.is_unspecified() && !a.is_multicast())
        .collect()
 }
 fn apply_client(
    run: RunScript<'_>,
    resolve: Resolve<'_>,
    plan: &ExitHostPlan,
 ) -> Result<(), String> {
    let interface = plan.interface.as_str();
    let gateway = run(&gateway_script(interface))?;
    let (index, hop) = parse_gateway(&gateway)
        .ok_or_else(|| format!("cannot read the default gateway from `{}`", gateway.trim()))?;
    let addresses = bypass_addresses(resolve, &plan.bypass, &plan.bypass_hosts);
    run(&bypass_script(index, hop, &addresses))?;
    if let Err(why) = run(&halves_script(interface)) {
        // Do not leave a half-installed set: bypass routes alone are harmless
        // but the next apply starts clean.
        let _ = run(&remove_client_script(interface));
        return Err(why);
    }
    Ok(())
 }
 fn parse_forwarding(text: &str) -> Option<bool> {
    match text.trim().to_ascii_lowercase().as_str() {
        "enabled" | "1" => Some(true),
        "disabled" | "0" => Some(false),
        _ => None,
    }
 }
 /// Makes the host match the plan.
 pub(super) fn apply_plan(
    run: RunScript<'_>,
    resolve: Resolve<'_>,
    plan: &ExitHostPlan,
 ) -> ExitHostReport {
    let interface = plan.interface.as_str();
    // What an earlier plan left and this one no longer wants.
    let pruned = run(&prune_script(interface, &plan.offer));
    let offer = plan
        .offer
        .iter()
        .map(|range| {
            let outcome = match &pruned {
                Err(why) => RuleOutcome::Failed(why.clone()),
                Ok(_) => match run(&nat_script(interface, range)) {
                    Ok(_) => RuleOutcome::Applied,
                    Err(why) => RuleOutcome::Failed(why),
                },
            };
            (*range, outcome)
        })
        .collect();
    let client = if plan.client {
        Some(match apply_client(run, resolve, plan) {
            Ok(()) => RuleOutcome::Applied,
            Err(why) => RuleOutcome::Failed(why),
        })
    } else {
        // Whatever an earlier plan left must not stay in force.
        let _ = run(&remove_client_script(interface));
        None
    };
    let forwarding = if plan.offer.is_empty() {
        None
    } else {
        run(&forwarding_script(interface))
            .ok()
            .and_then(|text| parse_forwarding(&text))
    };
    ExitHostReport {
        offer,
        forwarding,
        client,
    }
 }
 /// Removes every NAT object of this agent's for the interface.
 pub(super) fn clear_all(run: RunScript<'_>, interface: &str) {
    if let Err(why) = run(&prune_script(interface, &[])) {
        tracing::warn!(interface, "cannot remove the exit node NAT: {why}");
    }
    if let Err(why) = run(&remove_client_script(interface)) {
        tracing::warn!(interface, "cannot remove the exit node routes: {why}");
    }
 }
 #[cfg(test)]
 mod tests {
    #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
    use std::cell::RefCell;
    use super::*;
    fn range() -> Ipv4Range {
        "10.13.37.0/24".parse().unwrap()
    }
    fn plan(client: bool) -> ExitHostPlan {
        ExitHostPlan {
            interface: "tsun0".into(),
            offer: vec![range()],
            client,
            overlay: vec![range()],
            ..Default::default()
        }
    }
    #[test]
    fn the_nat_is_named_with_the_tag_and_replaces_its_own() {
        let script = nat_script("tsun0", &range());
        assert!(
            script.contains("$name = 'tsunagi-exit:tsun0:10.13.37.0/24'"),
            "{script}"
        );
        assert!(script.contains("-InternalIPInterfaceAddressPrefix '10.13.37.0/24'"));
        assert!(
            script.find("Remove-NetNat").unwrap() < script.find("New-NetNat").unwrap(),
            "{script}"
        );
    }
    #[test]
    fn a_quote_in_a_value_cannot_escape_the_literal() {
        let script = nat_script("x'; Remove-Item C:\\ #", &range());
        assert!(script.contains("'tsunagi-exit:x''; Remove-Item C:\\ #:10.13.37.0/24'"));
        assert!(prune_script("x'y", &[]).contains("'tsunagi-exit:x''y:'"));
        assert!(forwarding_script("x'y").contains("'x''y'"));
    }
    #[test]
    fn pruning_keeps_wanted_ranges_and_only_selects_this_interfaces_nats() {
        let script = prune_script("tsun0", &[range()]);
        assert!(script.contains("$prefix = 'tsunagi-exit:tsun0:'"));
        assert!(script.contains("$keep = @('tsunagi-exit:tsun0:10.13.37.0/24')"));
        assert!(script.contains("StartsWith($prefix)"));
        assert!(script.contains("exit 0"));
        assert!(prune_script("tsun0", &[]).contains("$keep = @()"));
    }
    #[test]
    fn forwarding_text_is_read_never_written() {
        assert_eq!(parse_forwarding("Enabled\r\n"), Some(true));
        assert_eq!(parse_forwarding("Disabled"), Some(false));
        assert_eq!(parse_forwarding("what"), None);
        let script = forwarding_script("tsun0");
        assert!(script.contains("Get-NetIPInterface"));
        assert!(!script.contains("Set-NetIPInterface"));
    }
    #[test]
    fn offering_prunes_then_creates_and_reads_forwarding() {
        let calls = RefCell::new(Vec::new());
        let run = |script: &str| -> Result<String, String> {
            calls.borrow_mut().push(script.to_string());
            Ok(if script.contains("Get-NetIPInterface") {
                "Disabled\n".to_string()
            } else {
                String::new()
            })
        };
        let report = apply_plan(&run, &no_names, &plan(false));
        assert_eq!(report.offer, vec![(range(), RuleOutcome::Applied)]);
        assert_eq!(report.forwarding, Some(false));
        assert_eq!(report.client, None);
        let calls = calls.borrow();
        assert!(calls[0].contains("StartsWith"), "{calls:?}");
        assert!(calls[1].contains("New-NetNat"), "{calls:?}");
        // Without a client, the routes of an earlier plan are taken away.
        assert!(calls[2].contains("RouteMetric -eq 7373"), "{calls:?}");
        assert!(calls[3].contains("Get-NetIPInterface"), "{calls:?}");
    }
    #[test]
    fn a_failing_nat_is_that_ranges_failure() {
        let run = |script: &str| -> Result<String, String> {
            if script.contains("New-NetNat") {
                Err("A NAT with this prefix already exists".into())
            } else {
                Ok(String::new())
            }
        };
        let report = apply_plan(&run, &no_names, &plan(false));
        let RuleOutcome::Failed(why) = &report.offer[0].1 else {
            panic!("{report:?}");
        };
        assert!(why.contains("already exists"), "{why}");
    }
    fn no_names(_: &str) -> Vec<Ipv4Addr> {
        Vec::new()
    }
    fn client_plan() -> ExitHostPlan {
        ExitHostPlan {
            interface: "tsun0".into(),
            offer: Vec::new(),
            client: true,
            overlay: vec![range()],
            bypass: vec![Ipv4Addr::new(138, 201, 61, 182)],
            bypass_hosts: vec!["relay.example".into()],
        }
    }
    #[test]
    fn the_gateway_is_read_from_the_script_output() {
        assert_eq!(
            parse_gateway("12 192.168.1.1\r\n"),
            Some((12, Ipv4Addr::new(192, 168, 1, 1)))
        );
        assert_eq!(parse_gateway(""), None);
        assert_eq!(parse_gateway("12"), None);
        assert_eq!(parse_gateway("12 gw"), None);
        assert_eq!(parse_gateway("12 1.1.1.1 extra"), None);
        let script = gateway_script("tsun0");
        assert!(script.contains("-ne $alias"));
        assert!(script.contains("-ne '0.0.0.0'"));
    }
    #[test]
    fn bypass_routes_are_marked_host_routes_through_the_physical_gateway() {
        let script = bypass_script(
            12,
            Ipv4Addr::new(192, 168, 1, 1),
            &[Ipv4Addr::new(1, 2, 3, 4), Ipv4Addr::new(5, 6, 7, 8)],
        );
        assert!(script.contains("@('1.2.3.4', '5.6.7.8')"), "{script}");
        assert!(script.contains("-InterfaceIndex 12 -NextHop '192.168.1.1'"));
        assert!(script.contains("-RouteMetric 7373"));
        assert!(script.contains("-PolicyStore ActiveStore"));
        // Stale marked routes go first, and only /32 ones with our metric.
        assert!(script.find("Remove-NetRoute").unwrap() < script.find("New-NetRoute").unwrap());
        assert!(script.contains("RouteMetric -eq 7373"));
        assert!(script.contains("EndsWith('/32')"));
    }
    #[test]
    fn the_halves_go_through_the_overlay_and_ipv6_is_blackholed() {
        let script = halves_script("tsun0");
        assert!(script.contains("'0.0.0.0/1', '128.0.0.0/1'"));
        assert!(script.contains("-InterfaceAlias $alias -NextHop '0.0.0.0'"));
        assert!(script.contains("'::/1', '8000::/1'"));
        assert!(script.contains("-PolicyStore ActiveStore"));
        assert!(
            !script.contains("0.0.0.0/0"),
            "the default route is left alone"
        );
        assert!(halves_script("x'y").contains("'x''y'"));
    }
    #[test]
    fn addresses_are_resolved_merged_and_cleaned() {
        let resolve = |host: &str| -> Vec<Ipv4Addr> {
            match host {
                "relay.example" => vec![Ipv4Addr::new(9, 9, 9, 9), Ipv4Addr::new(1, 2, 3, 4)],
                _ => Vec::new(),
            }
        };
        let all = bypass_addresses(
            &resolve,
            &[
                Ipv4Addr::new(1, 2, 3, 4),
                Ipv4Addr::LOCALHOST,
                Ipv4Addr::UNSPECIFIED,
            ],
            &["relay.example".into(), "gone.example".into()],
        );
        assert_eq!(
            all,
            vec![Ipv4Addr::new(1, 2, 3, 4), Ipv4Addr::new(9, 9, 9, 9)]
        );
    }
    #[test]
    fn the_bypass_is_written_before_the_halves_that_need_it() {
        let calls = RefCell::new(Vec::new());
        let run = |script: &str| -> Result<String, String> {
            calls.borrow_mut().push(script.to_string());
            Ok(if script.contains("Select-Object -First 1") {
                "12 192.168.1.1\n".to_string()
            } else {
                String::new()
            })
        };
        let resolve = |_: &str| vec![Ipv4Addr::new(9, 9, 9, 9)];
        let report = apply_plan(&run, &resolve, &client_plan());
        assert_eq!(report.client, Some(RuleOutcome::Applied));
        let calls = calls.borrow();
        let at = |needle: &str| calls.iter().position(|c| c.contains(needle)).unwrap();
        assert!(at("Select-Object -First 1") < at("-RouteMetric 7373"));
        assert!(at("-RouteMetric 7373") < at("'0.0.0.0/1', '128.0.0.0/1'"));
        let bypass = &calls[at("-RouteMetric 7373")];
        assert!(
            bypass.contains("'9.9.9.9'") && bypass.contains("'138.201.61.182'"),
            "{bypass}"
        );
    }
    #[test]
    fn nothing_goes_through_the_overlay_when_the_bypass_cannot_be_written() {
        let run = |script: &str| -> Result<String, String> {
            if script.contains("Select-Object -First 1") {
                Ok("12 192.168.1.1".to_string())
            } else if script.contains("-RouteMetric 7373") && script.contains("foreach ($ip") {
                Err("Access is denied".to_string())
            } else {
                assert!(!script.contains("-NextHop '0.0.0.0'"), "{script}");
                Ok(String::new())
            }
        };
        let report = apply_plan(&run, &no_names, &client_plan());
        assert_eq!(
            report.client,
            Some(RuleOutcome::Failed("Access is denied".into()))
        );
    }
    #[test]
    fn a_missing_gateway_is_a_failure_that_installs_no_route() {
        let run = |script: &str| -> Result<String, String> {
            assert!(!script.contains("New-NetRoute"), "{script}");
            if script.contains("Select-Object -First 1") {
                Err("the host has no default gateway outside the overlay".into())
            } else {
                Ok(String::new())
            }
        };
        let report = apply_plan(&run, &no_names, &client_plan());
        let Some(RuleOutcome::Failed(why)) = report.client else {
            panic!("{report:?}");
        };
        assert!(why.contains("no default gateway"), "{why}");
    }
    #[test]
    fn a_failed_halves_step_removes_what_was_written() {
        let calls = RefCell::new(Vec::new());
        let run = |script: &str| -> Result<String, String> {
            calls.borrow_mut().push(script.to_string());
            if script.contains("Select-Object -First 1") {
                Ok("12 192.168.1.1".to_string())
            } else if script.contains("'128.0.0.0/1'") && script.contains("New-NetRoute") {
                Err("boom".to_string())
            } else {
                Ok(String::new())
            }
        };
        let report = apply_plan(&run, &no_names, &client_plan());
        assert!(matches!(report.client, Some(RuleOutcome::Failed(_))));
        assert!(calls.borrow().last().unwrap().contains("Remove-NetRoute"));
    }
    #[test]
    fn dropping_the_client_removes_its_routes_and_clearing_does_too() {
        let calls = RefCell::new(Vec::new());
        let run = |script: &str| -> Result<String, String> {
            calls.borrow_mut().push(script.to_string());
            Ok(String::new())
        };
        let report = apply_plan(&run, &no_names, &plan(false));
        assert_eq!(report.client, None);
        assert!(
            calls
                .borrow()
                .iter()
                .any(|c| c.contains("RouteMetric -eq 7373"))
        );
        let script = remove_client_script("tsun0");
        assert!(script.contains("'0.0.0.0/1', '128.0.0.0/1', '::/1', '8000::/1'"));
        assert!(script.contains("exit 0"));
        // Only routes on our interface, or marked ours, are touched.
        assert!(script.contains("-InterfaceAlias $alias"));
        calls.borrow_mut().clear();
        clear_all(&run, "tsun0");
        assert!(
            calls
                .borrow()
                .iter()
                .any(|c| c.contains("RouteMetric -eq 7373"))
        );
    }
    #[test]
    fn clearing_prunes_everything_for_the_interface() {
        let seen = RefCell::new(Vec::<String>::new());
        let run = |script: &str| -> Result<String, String> {
            seen.borrow_mut().push(script.to_string());
            Ok(String::new())
        };
        clear_all(&run, "tsun0");
        let seen = seen.borrow();
        assert!(seen[0].contains("$keep = @()"));
        assert!(seen[0].contains("'tsunagi-exit:tsun0:'"));
    }
 }
@@ -164,6 +164,17 @@ pub struct Interface {
    rules_report: std::sync::Mutex<Option<BroadcastRulesReport>>,
    /// What the exit-node rules were last asked for, and what came of it.
    exit_state: std::sync::Mutex<ExitState>,
    /// Where each network's runtime reaches its peers and relays outside the
    /// overlay; read only by hosts that cannot exempt the agent from the
    /// exit node's default route.
    underlay: std::sync::Mutex<std::collections::HashMap<NetworkId, Underlay>>,
 }
 /// Underlay destinations one network's runtime currently depends on.
 #[derive(Debug, Default, Clone, PartialEq, Eq)]
 struct Underlay {
    addresses: std::collections::BTreeSet<std::net::Ipv4Addr>,
    hosts: std::collections::BTreeSet<String>,
 }
 /// The exit-node host rules as this interface last left them.
@@ -275,6 +286,7 @@ impl Interface {
            host_rules,
            rules_report: std::sync::Mutex::new(None),
            exit_state: std::sync::Mutex::new(ExitState::default()),
            underlay: std::sync::Mutex::new(std::collections::HashMap::new()),
        })
    }
@@ -350,10 +362,20 @@ impl Interface {
            .collect();
        ranges.sort_by_key(|range| (range.base, range.prefix_len));
        ranges.dedup();
        let client = self.routes.exit_via().is_some();
        let overlay = self.routes.overlay_ranges();
        let (bypass, bypass_hosts) = if client && exit.needs_bypass() {
            self.underlay_targets(&overlay)
        } else {
            (Vec::new(), Vec::new())
        };
        let wanted = ExitHostPlan {
            interface: self.name.clone(),
            offer: ranges,
-            client: self.routes.exit_via().is_some(),
+            client,
            overlay,
            bypass,
            bypass_hosts,
        };
        let first = !std::mem::replace(&mut self.exit_state().cleaned, true);
@@ -398,9 +420,15 @@ impl Interface {
            tracing::warn!(
                interface = %self.name,
                "kernel forwarding is off for this interface, so nothing is forwarded for the \
-                 exit node yet: `sysctl -w net.ipv4.conf.{0}.forwarding=1` (or \
+                 exit node yet: {}",
-                 `net.ipv4.ip_forward=1`), and set it in sysctl.d to keep it",
+                if cfg!(target_os = "windows") {
-                self.name
+                    "`Set-NetIPInterface -Forwarding Enabled` on the interfaces involved"
                } else if cfg!(target_os = "macos") {
                    "`sysctl -w net.inet.ip.forwarding=1`, set again after every reboot"
                } else {
                    "`sysctl -w net.ipv4.ip_forward=1` (or `net.ipv4.conf.<interface>.forwarding=1`), \
                     and set it in sysctl.d to keep it"
                }
            );
        }
        match &report.client {
@@ -419,6 +447,68 @@ impl Interface {
        state.report = Some(report);
    }
    /// Tells the interface where a network's runtime reaches peers and
    /// relays outside the overlay. When the host needs those kept out of the
    /// exit node's default route and the set changed, the rules follow.
    pub async fn set_underlay(
        &self,
        network: NetworkId,
        addresses: impl IntoIterator<Item = std::net::Ipv4Addr>,
        hosts: impl IntoIterator<Item = String>,
    ) {
        let new = Underlay {
            addresses: addresses.into_iter().collect(),
            hosts: hosts.into_iter().collect(),
        };
        let changed = {
            let mut map = match self.underlay.lock() {
                Ok(guard) => guard,
                Err(poisoned) => poisoned.into_inner(),
            };
            if new == Underlay::default() {
                map.remove(&network).is_some()
            } else {
                map.insert(network, new.clone()) != Some(new)
            }
        };
        let needed = changed
            && self.routes.exit_via().is_some()
            && self
                .host_rules
                .as_ref()
                .and_then(|rules| rules.exit_rules())
                .is_some_and(|exit| exit.needs_bypass());
        if needed {
            self.sync_exit_rules().await;
        }
    }
    /// Every underlay destination worth keeping direct, minus anything that
    /// is the overlay itself, loopback or link-local.
    fn underlay_targets(
        &self,
        overlay: &[crate::state::Ipv4Range],
    ) -> (Vec<std::net::Ipv4Addr>, Vec<String>) {
        let map = match self.underlay.lock() {
            Ok(guard) => guard,
            Err(poisoned) => poisoned.into_inner(),
        };
        let mut addresses = std::collections::BTreeSet::new();
        let mut hosts = std::collections::BTreeSet::new();
        for underlay in map.values() {
            addresses.extend(underlay.addresses.iter().copied().filter(|address| {
                !address.is_loopback()
                    && !address.is_link_local()
                    && !address.is_multicast()
                    && !address.is_unspecified()
                    && !address.is_broadcast()
                    && !overlay.iter().any(|range| range.contains(*address))
            }));
            hosts.extend(underlay.hosts.iter().cloned());
        }
        (addresses.into_iter().collect(), hosts.into_iter().collect())
    }
    fn exit_state(&self) -> std::sync::MutexGuard<'_, ExitState> {
        match self.exit_state.lock() {
            Ok(guard) => guard,
@@ -443,7 +533,7 @@ impl Interface {
                "this overlay interface is not on the host (no TUN device), so there is \
                 nothing to forward to",
            ),
-            Some(false) => Some("exit nodes are only supported on Linux yet"),
+            Some(false) => Some("exit nodes are not supported on this platform yet"),
            Some(true) => None,
        };
        let failed = |detail: &str| RuleSetReport {
@@ -1336,6 +1426,65 @@ mod tests {
        assert_eq!(interface.counters().received, 1);
    }
    #[tokio::test]
    async fn the_underlay_reaches_the_rules_only_for_hosts_that_need_it_and_only_while_using_one() {
        let rules = MockHostRules::new();
        let (interface, routes, id) = managed_interface(&rules, false).await;
        let relay = || vec!["relay.example".to_string()];
        let direct = || {
            vec![
                Ipv4Addr::new(138, 201, 61, 182),
                Ipv4Addr::new(10, 13, 37, 9), // the overlay itself
                Ipv4Addr::LOCALHOST,
                Ipv4Addr::new(169, 254, 1, 1),
            ]
        };
        // A host that does not ask never gets them, so Linux and macOS do not
        // re-apply their rules whenever a path moves.
        routes.set_exit(id, using(peer(2)));
        interface.set_underlay(id, direct(), relay()).await;
        interface.sync_exit_rules().await;
        let plan = rules.exit().installed().unwrap();
        assert!(plan.client);
        assert!(plan.bypass.is_empty() && plan.bypass_hosts.is_empty());
        // A host that does gets the usable ones, and follows changes.
        rules.exit().set_needs_bypass(true);
        interface.set_underlay(id, direct(), relay()).await;
        interface.sync_exit_rules().await;
        let plan = rules.exit().installed().unwrap();
        assert_eq!(plan.bypass, vec![Ipv4Addr::new(138, 201, 61, 182)]);
        assert_eq!(plan.bypass_hosts, relay());
        let applies = || {
            rules
                .exit()
                .calls()
                .iter()
                .filter(|c| c.starts_with("apply"))
                .count()
        };
        let before = applies();
        interface.set_underlay(id, direct(), relay()).await;
        assert_eq!(applies(), before, "an unchanged set is not applied again");
        interface
            .set_underlay(id, vec![Ipv4Addr::new(5, 6, 7, 8)], relay())
            .await;
        assert_eq!(applies(), before + 1);
        assert_eq!(
            rules.exit().installed().unwrap().bypass,
            vec![Ipv4Addr::new(5, 6, 7, 8)]
        );
        // Not using an exit node: nothing to keep direct.
        routes.set_exit(id, Default::default());
        interface.sync_exit_rules().await;
        interface
            .set_underlay(id, vec![Ipv4Addr::new(1, 1, 1, 1)], relay())
            .await;
        assert!(rules.exit().installed().is_none());
    }
    #[tokio::test]
    async fn the_exit_rules_follow_the_routing_table_and_leave_with_the_interface() {
        let rules = MockHostRules::new();
@@ -293,6 +293,18 @@ impl RoutingTable {
        offers
    }
    /// The range of every network in the table that has agreed one, sorted.
    pub fn overlay_ranges(&self) -> Vec<Ipv4Range> {
        let mut ranges: Vec<Ipv4Range> = self
            .read()
            .values()
            .filter_map(|routes| routes.range)
            .collect();
        ranges.sort_by_key(|range| (range.base, range.prefix_len));
        ranges.dedup();
        ranges
    }
    /// One network's exit-node settings and its range, if it is in the table.
    pub fn exit_settings(&self, network: NetworkId) -> Option<(ExitPolicy, Option<Ipv4Range>)> {
        self.read()
@@ -20,7 +20,7 @@ use crate::error::ProtocolError;
 /// The version in the ALPN is the wire-compatibility version of the control
 /// protocol. It is independent of the network identity scheme version, so
 /// bumping it must not change any existing [`crate::NetworkId`].
-pub const ALPN: &[u8] = b"tsunagi/ctrl/4";
+pub const ALPN: &[u8] = b"tsunagi/ctrl/5";
 /// ALPN of the tsunagi data plane.
 ///
@@ -30,9 +30,11 @@ pub const ALPN: &[u8] = b"tsunagi/ctrl/4";
 /// transport underneath can be replaced without touching the control protocol.
 ///
 /// Version 4 adds source, destination, hop limit and stable flow id above
-/// transport fragmentation. All members must upgrade together. Persistent
+/// transport fragmentation. Version 5 adds the end-to-end protocol to that
-/// identities and network configuration do not change.
+/// envelope, so a tunnel no longer depends on the kind of link under it, and
-pub const DATA_ALPN: &[u8] = b"tsunagi/data/4";
+/// a first-claimed time to a signed hostname claim. All members must upgrade
 /// together. Persistent identities and network configuration do not change.
 pub const DATA_ALPN: &[u8] = b"tsunagi/data/5";
 /// Largest plugin protocol identifier accepted when opening a data channel.
 pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
@@ -41,7 +43,7 @@ pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
 pub const MAX_SIGNATURE_LEN: usize = 64;
 /// Control protocol version carried inside the handshake.
-pub const PROTOCOL_VERSION: u16 = 4;
+pub const PROTOCOL_VERSION: u16 = 5;
 /// First message of the handshake, sent by the initiator.
 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -150,7 +150,7 @@ pub fn derived_ipv4_range(network: NetworkId) -> Ipv4Range {
 }
 /// Frozen domain separator for the bytes a record signature covers.
-pub const RECORD_DOMAIN: &str = "tsunagi-signed-record-v2";
+pub const RECORD_DOMAIN: &str = "tsunagi-signed-record-v3";
 /// Longest hostname a record may carry.
 ///
@@ -207,6 +207,18 @@ pub enum RecordBody {
        range: Option<Ipv4Range>,
        /// The name this author answers to.
        hostname: Option<String>,
        /// When this author first claimed that name, in milliseconds since
        /// the Unix epoch, kept across every later version that keeps the
        /// name. It is what makes a name belong to whoever claimed it first
        /// rather than to whoever happens to sort lowest, so a member that
        /// arrives later with the same name cannot take it over.
        ///
        /// The author's own word: a member with a wrong clock can claim an
        /// earlier time. That costs it nothing it could not do by other
        /// means, since anybody who knows the network secret is a member,
        /// and it cannot take an address or reach anything it was not
        /// already entitled to.
        hostname_since: Option<u64>,
    },
    /// This author gives up everything it claimed.
    ///
@@ -243,6 +255,14 @@ impl RecordBody {
        }
    }
    /// When the author first claimed the hostname, if it has one.
    pub fn hostname_since(&self) -> Option<u64> {
        match self {
            RecordBody::Claim { hostname_since, .. } => *hostname_since,
            RecordBody::Release => None,
        }
    }
    fn canonical(&self) -> Vec<u8> {
        let mut out = Vec::with_capacity(96);
        match self {
@@ -250,6 +270,7 @@ impl RecordBody {
                address,
                range,
                hostname,
                hostname_since,
            } => {
                push_lp(&mut out, b"claim");
                push_opt(
@@ -270,6 +291,13 @@ impl RecordBody {
                        .as_deref(),
                );
                push_opt(&mut out, hostname.as_deref().map(str::as_bytes));
                push_opt(
                    &mut out,
                    hostname_since
                        .map(|since| since.to_be_bytes())
                        .as_ref()
                        .map(|bytes| &bytes[..]),
                );
            }
            RecordBody::Release => {
                push_lp(&mut out, b"release");
@@ -391,8 +419,12 @@ impl SignedRecord {
            address,
            range,
            hostname,
            hostname_since,
        } = &self.body
        {
            if hostname.is_none() && hostname_since.is_some() {
                return Err(StateError::Malformed("a time was claimed for no hostname"));
            }
            // Bounds before anything is believed, because all of this came
            // off the network.
            if let Some(range) = range {
@@ -585,25 +617,33 @@ impl StateSet {
    /// Who currently holds each claimed hostname.
    ///
-    /// The same rule as addresses, for the same reason: a name is owned, two
+    /// A name belongs to the member that claimed it first, and the others
-    /// members may claim one, and every replica has to reach the same answer
+    /// that claim it keep working by address but are not found by that name.
-    /// about who has it without asking anybody.
+    /// "First" is the time each author signed into its own claim, with the
    /// lower id deciding a tie and a claim with no time counting as last, so
    /// every replica reaches the same answer about who has it without
    /// asking anybody. It is a pure function of the records: nothing depends
    /// on the order they arrived in, so there is no race to lose.
    pub fn hostname_holders(&self) -> HashMap<String, EndpointId> {
-        let mut holders: HashMap<String, EndpointId> = HashMap::new();
+        let mut holders: HashMap<String, (u64, EndpointId)> = HashMap::new();
        for (author, record) in &self.records {
            let Some(hostname) = record.body.hostname() else {
                continue;
            };
            let claim = (record.body.hostname_since().unwrap_or(u64::MAX), *author);
            holders
                .entry(hostname.to_string())
                .and_modify(|held| {
-                    if author.as_bytes() < held.as_bytes() {
+                    if (claim.0, claim.1.as_bytes()) < (held.0, held.1.as_bytes()) {
-                        *held = *author;
+                        *held = claim;
                    }
                })
-                .or_insert(*author);
+                .or_insert(claim);
        }
        holders
            .into_iter()
            .map(|(hostname, (_, author))| (hostname, author))
            .collect()
    }
    /// The hostname an author holds, if it holds one uncontested.
@@ -612,6 +652,19 @@ impl StateSet {
        (self.hostname_holders().get(hostname) == Some(author)).then_some(hostname)
    }
    /// The name an author claimed and the member that holds it instead, when
    /// somebody who claimed it earlier does.
    pub fn hostname_taken(&self, author: &EndpointId) -> Option<(&str, EndpointId)> {
        let hostname = self.records.get(author)?.body.hostname()?;
        let holder = *self.hostname_holders().get(hostname)?;
        (holder != *author).then_some((hostname, holder))
    }
    /// When an author first claimed the name it holds now, if it has one.
    pub fn hostname_since_of(&self, author: &EndpointId) -> Option<u64> {
        self.records.get(author)?.body.hostname_since()
    }
    /// The address an author holds, if it holds one uncontested.
    pub fn address_of(&self, author: &EndpointId) -> Option<Ipv4Addr> {
        let address = self.records.get(author)?.body.claimed_address()?;
@@ -661,6 +714,7 @@ mod tests {
            address: Some(address.parse().unwrap()),
            range: Some(range()),
            hostname: None,
            hostname_since: None,
        }
    }
@@ -720,6 +774,7 @@ mod tests {
                address: Some("10.99.0.1".parse().unwrap()),
                range: Some(range()),
                hostname: None,
                hostname_since: None,
            },
        );
        assert!(matches!(outside.verify(id), Err(StateError::Malformed(_))));
@@ -735,6 +790,7 @@ mod tests {
                    prefix_len: 31,
                }),
                hostname: None,
                hostname_since: None,
            },
        );
        assert!(matches!(no_hosts.verify(id), Err(StateError::Malformed(_))));
@@ -776,6 +832,7 @@ mod tests {
                    address: None,
                    range: None,
                    hostname: Some(bad.to_string()),
                    hostname_since: None,
                },
            );
            assert!(
@@ -797,6 +854,7 @@ mod tests {
                address: Some("10.13.37.5".parse().unwrap()),
                range: None,
                hostname: None,
                hostname_since: None,
            },
        );
        assert!(matches!(record.verify(id), Err(StateError::Malformed(_))));
@@ -818,6 +876,7 @@ mod tests {
                    address: None,
                    range: None,
                    hostname: Some("music".into()),
                    hostname_since: None,
                },
            )
        };
@@ -860,6 +919,7 @@ mod tests {
                    address: None,
                    range: None,
                    hostname: Some(name.to_string()),
                    hostname_since: None,
                },
            )
        };
@@ -879,6 +939,109 @@ mod tests {
        assert_eq!(set.hostname_of(&secret.public()), Some("new"));
    }
    fn named_at(
        secret: &SecretKey,
        id: NetworkId,
        version: u64,
        name: &str,
        since: u64,
    ) -> SignedRecord {
        SignedRecord::sign(
            secret,
            id,
            version,
            RecordBody::Claim {
                address: None,
                range: None,
                hostname: Some(name.into()),
                hostname_since: Some(since),
            },
        )
    }
    #[test]
    fn a_name_belongs_to_whoever_claimed_it_first_whatever_their_ids_or_arrival_order() {
        let id = network("naming");
        let mut keys = [SecretKey::generate(), SecretKey::generate()];
        keys.sort_by_key(|key| *key.public().as_bytes());
        let [low, high] = keys;
        // The member with the higher id claimed first, so it holds the name,
        // and the lower id does not take it back by sorting first.
        let early = named_at(&high, id, 1, "music", 1_000);
        let late = named_at(&low, id, 1, "music", 2_000);
        for order in [[&early, &late], [&late, &early]] {
            let mut set = StateSet::new();
            for record in order {
                set.merge(id, (*record).clone()).unwrap();
            }
            assert_eq!(set.hostname_of(&high.public()), Some("music"));
            assert_eq!(set.hostname_of(&low.public()), None);
            assert_eq!(
                set.hostname_taken(&low.public()),
                Some(("music", high.public()))
            );
            assert_eq!(set.hostname_taken(&high.public()), None);
        }
        // At the same instant the lower id decides, the same way everywhere.
        let mut set = StateSet::new();
        set.merge(id, named_at(&high, id, 1, "music", 5)).unwrap();
        set.merge(id, named_at(&low, id, 1, "music", 5)).unwrap();
        assert_eq!(set.hostname_of(&low.public()), Some("music"));
        assert_eq!(set.hostname_of(&high.public()), None);
    }
    #[test]
    fn a_name_that_loses_is_found_again_once_the_holder_lets_it_go() {
        let id = network("naming");
        let first = SecretKey::generate();
        let second = SecretKey::generate();
        let mut set = StateSet::new();
        set.merge(id, named_at(&first, id, 1, "music", 1)).unwrap();
        set.merge(id, named_at(&second, id, 1, "music", 2)).unwrap();
        assert_eq!(set.hostname_of(&second.public()), None);
        // Renaming starts the clock afresh, so it cannot be used to jump the
        // queue for a name somebody else already holds.
        set.merge(id, named_at(&first, id, 2, "other", 3)).unwrap();
        assert_eq!(set.hostname_of(&second.public()), Some("music"));
        set.merge(id, named_at(&first, id, 3, "music", 4)).unwrap();
        assert_eq!(set.hostname_of(&second.public()), Some("music"));
        assert_eq!(set.hostname_of(&first.public()), None);
        // And a release hands it on.
        set.merge(id, SignedRecord::sign(&second, id, 2, RecordBody::Release))
            .unwrap();
        assert_eq!(set.hostname_of(&first.public()), Some("music"));
    }
    #[test]
    fn a_time_with_no_name_is_not_a_valid_claim() {
        let id = network("naming");
        let secret = SecretKey::generate();
        let record = SignedRecord::sign(
            &secret,
            id,
            1,
            RecordBody::Claim {
                address: None,
                range: None,
                hostname: None,
                hostname_since: Some(1),
            },
        );
        assert!(matches!(record.verify(id), Err(StateError::Malformed(_))));
        // The time is signed: moving it breaks the signature, so nobody can
        // improve another member's place in the queue.
        let mut forged = named_at(&secret, id, 1, "music", 9_000);
        if let RecordBody::Claim { hostname_since, .. } = &mut forged.body {
            *hostname_since = Some(1);
        }
        assert_eq!(forged.verify(id).unwrap_err(), StateError::BadSignature);
    }
    #[test]
    fn a_release_gives_up_the_name_as_well_as_the_address() {
        let secret = SecretKey::generate();
@@ -894,6 +1057,7 @@ mod tests {
                    address: Some("10.13.37.5".parse().unwrap()),
                    range: Some(range()),
                    hostname: Some("music".into()),
                    hostname_since: None,
                },
            ),
        )
@@ -1057,6 +1221,7 @@ mod tests {
                    address: Some("10.99.0.7".parse().unwrap()),
                    range: Some(custom),
                    hostname: None,
                    hostname_since: None,
                },
            ),
        )
@@ -21,7 +21,7 @@ use crate::identity::{DeviceIdentity, NetworkId, NetworkName, NetworkSecret};
 use crate::state::{RecordBody, SignedRecord};
 /// Schema version written by this build.
-pub const SCHEMA_VERSION: i64 = 6;
+pub const SCHEMA_VERSION: i64 = 7;
 /// Key of the stored hostname setting.
 const SETTING_HOSTNAME: &str = "hostname";
@@ -221,6 +221,23 @@ impl StateStore {
                )
                .map_err(|err| self.corrupt(format!("cannot migrate schema to 6: {err}")))?;
        }
        // Migration 6 -> 7: a claim gained the time its hostname was first
        // claimed, which changed the signing domain. Records signed before
        // no longer verify, so they are discarded rather than kept as rows
        // every read has to reject; each member re-publishes its own claim on
        // its next run and the others arrive again from their authors. The
        // version counter is kept, so nothing is ever published twice at one
        // version.
        if found < 7 {
            self.conn
                .execute_batch(
                    "BEGIN;
                     DELETE FROM signed_records;
                     PRAGMA user_version = 7;
                     COMMIT;",
                )
                .map_err(|err| self.corrupt(format!("cannot migrate schema to 7: {err}")))?;
        }
        Ok(())
    }
@@ -0,0 +1,308 @@
 //! Connectivity does not depend on what kind of link each hop happens to be,
 //! and a member is online however far away it is.
 #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
 use std::collections::{HashMap, HashSet};
 use std::sync::{Arc, Mutex};
 use bytes::Bytes;
 use iroh::EndpointId;
 use tsunagi::NetworkId;
 use tsunagi::agent::DataPath;
 use tsunagi::dataplane::transport::SharedLink;
 use tsunagi::dataplane::{IpPlugin, PluginCapability, PluginError};
 use tsunagi::discovery::SharedMemoryDiscovery;
 use tsunagi::testing::{TestAgent, network, wait_for_peers, wait_until};
 /// A protocol that moves bytes between two members unchanged and keeps the
 /// link it was handed, so a test can use the link directly.
 #[derive(Debug)]
 struct Carrier {
    protocol: String,
    links: Mutex<HashMap<EndpointId, SharedLink>>,
 }
 impl Carrier {
    fn new(protocol: &str) -> Arc<Self> {
        Arc::new(Self {
            protocol: protocol.into(),
            links: Mutex::default(),
        })
    }
    fn link(&self, peer: EndpointId) -> Option<SharedLink> {
        self.links.lock().unwrap().get(&peer).cloned()
    }
 }
 impl IpPlugin for Carrier {
    fn protocol_id(&self) -> &str {
        &self.protocol
    }
    fn protocol_version(&self) -> u16 {
        1
    }
    fn local_capability(
        &self,
        _network: NetworkId,
    ) -> Result<Option<PluginCapability>, PluginError> {
        Ok(Some(PluginCapability {
            protocol: self.protocol.clone(),
            version: 1,
            enabled: true,
            data: Vec::new(),
        }))
    }
    fn on_peer_capability(
        &self,
        _network: NetworkId,
        _peer: EndpointId,
        _capability: &PluginCapability,
    ) -> Result<(), PluginError> {
        Ok(())
    }
    fn on_peer_link(&self, _network: NetworkId, peer: EndpointId, link: SharedLink) {
        self.links.lock().unwrap().insert(peer, link);
    }
    fn on_peer_gone(&self, _network: NetworkId, peer: EndpointId) {
        self.links.lock().unwrap().remove(&peer);
    }
    fn on_network_deactivated(&self, _network: NetworkId) {}
 }
 type Blocked = Arc<Mutex<HashSet<EndpointId>>>;
 async fn member(
    discovery: &SharedMemoryDiscovery,
    plugins: &[Arc<Carrier>],
    blocked: &Blocked,
 ) -> TestAgent {
    let plugins: Vec<_> = plugins.to_vec();
    let blocked = Arc::clone(blocked);
    TestAgent::spawn_with(
        move |config| {
            let config = config.with_unreachable_data_peers(blocked);
            plugins
                .into_iter()
                .fold(config, |config, plugin| config.with_plugin(plugin))
        },
        discovery,
    )
    .await
    .unwrap()
 }
 async fn exchange(from: &SharedLink, to: &SharedLink, payload: &'static [u8]) {
    from.send(Bytes::from_static(payload)).unwrap();
    let got = tokio::time::timeout(tsunagi::testing::DEADLINE, to.recv())
        .await
        .expect("the datagram should arrive")
        .expect("the link stays open");
    assert_eq!(got, Bytes::from_static(payload));
 }
 #[tokio::test]
 async fn a_tunnel_crosses_a_member_that_shares_no_protocol_with_either_end_and_moves_to_a_direct_link_by_itself()
 {
    // A and B share one protocol and nothing else. C shares a different
    // one with each, so every link here is of a different kind and the
    // middle has never heard of what A and B speak. A and B cannot reach
    // each other directly, which is the case a blocked UDP path produces.
    let discovery = SharedMemoryDiscovery::new();
    let (name, secret) = network("mixed-transports");
    let blocked_a: Blocked = Arc::default();
    let blocked_b: Blocked = Arc::default();
    let a_shared = Carrier::new("shared");
    let b_shared = Carrier::new("shared");
    let a = member(
        &discovery,
        &[Carrier::new("a-and-c"), Arc::clone(&a_shared)],
        &blocked_a,
    )
    .await;
    let b = member(
        &discovery,
        &[Carrier::new("c-and-b"), Arc::clone(&b_shared)],
        &blocked_b,
    )
    .await;
    let c = member(
        &discovery,
        &[Carrier::new("a-and-c"), Carrier::new("c-and-b")],
        &Blocked::default(),
    )
    .await;
    let (a_id, b_id, c_id) = (
        a.agent.endpoint_id(),
        b.agent.endpoint_id(),
        c.agent.endpoint_id(),
    );
    blocked_a.lock().unwrap().insert(b_id);
    blocked_b.lock().unwrap().insert(a_id);
    let network_id = a.agent.join_network(&name, &secret).await.unwrap();
    b.agent.join_network(&name, &secret).await.unwrap();
    c.agent.join_network(&name, &secret).await.unwrap();
    for agent in [&a, &b, &c] {
        wait_for_peers(&agent.agent, network_id, 2).await;
    }
    // The tunnel comes up through C, and data goes both ways.
    let to_b = wait_until("a has a link to b", || async { a_shared.link(b_id) }).await;
    let to_a = wait_until("b has a link to a", || async { b_shared.link(a_id) }).await;
    exchange(&to_b, &to_a, b"a to b, sealed").await;
    exchange(&to_a, &to_b, b"b to a, sealed").await;
    // The status says what is true: online, and through whom.
    let status = a.agent.network_status(network_id).await.unwrap();
    let b_peer = status.peers.iter().find(|p| p.endpoint_id == b_id).unwrap();
    assert_eq!(b_peer.data_path, DataPath::Relayed { hops: 2, via: c_id });
    let c_peer = status.peers.iter().find(|p| p.endpoint_id == c_id).unwrap();
    assert!(matches!(&c_peer.data_path, DataPath::Direct { transport } if transport == "a-and-c"));
    let b_member = wait_until("a lists b as a member", || async {
        let status = a.agent.network_status(network_id).await.ok()?;
        status.members.into_iter().find(|m| m.endpoint_id == b_id)
    })
    .await;
    assert!(b_member.online);
    let carried = c.agent.network_status(network_id).await.unwrap().relay;
    assert!(carried.forwarded >= 2, "{carried:?}");
    // The direct path becomes possible: the same link carries on, now
    // without C, and nobody has to do anything about it.
    blocked_a.lock().unwrap().clear();
    blocked_b.lock().unwrap().clear();
    wait_until("a reaches b directly", || async {
        let status = a.agent.network_status(network_id).await.ok()?;
        let peer = status.peers.iter().find(|p| p.endpoint_id == b_id)?;
        matches!(&peer.data_path, DataPath::Direct { transport } if transport == "shared")
            .then_some(())
    })
    .await;
    let forwarded = c
        .agent
        .network_status(network_id)
        .await
        .unwrap()
        .relay
        .forwarded;
    exchange(&to_b, &to_a, b"direct now").await;
    assert!(
        Arc::ptr_eq(&to_b, &a_shared.link(b_id).unwrap()),
        "the tunnel's link is the one it always had"
    );
    assert_eq!(
        c.agent
            .network_status(network_id)
            .await
            .unwrap()
            .relay
            .forwarded,
        forwarded,
        "nothing goes through C any more"
    );
    a.stop().await;
    b.stop().await;
    c.stop().await;
 }
 #[tokio::test]
 async fn a_name_belongs_to_the_member_that_claimed_it_first_on_every_member() {
    let discovery = SharedMemoryDiscovery::new();
    let (name, secret) = network("first-claim");
    let spawn = |hostname: &'static str| {
        let discovery = discovery.clone();
        async move {
            TestAgent::spawn_with(move |config| config.with_hostname(hostname), &discovery)
                .await
                .unwrap()
        }
    };
    let first = spawn("music").await;
    let network_id = first.agent.join_network(&name, &secret).await.unwrap();
    let first_id = first.agent.endpoint_id();
    wait_until("the first claim is signed", || async {
        let status = first.agent.network_status(network_id).await.ok()?;
        status
            .members
            .iter()
            .any(|m| m.endpoint_id == first_id && m.hostname.as_deref() == Some("music"))
            .then_some(())
    })
    .await;
    // Whatever the ids turn out to be, the later one does not take it.
    let second = spawn("music").await;
    let third = spawn("studio").await;
    second.agent.join_network(&name, &secret).await.unwrap();
    third.agent.join_network(&name, &secret).await.unwrap();
    let second_id = second.agent.endpoint_id();
    for agent in [&first, &second, &third] {
        wait_for_peers(&agent.agent, network_id, 2).await;
    }
    for (viewer, label) in [(&first, "first"), (&second, "second"), (&third, "third")] {
        let status = wait_until(&format!("{label} sees both claims"), || async {
            let status = viewer.agent.network_status(network_id).await.ok()?;
            // The roster is the authors of signed records, so a member
            // being in it means its claim has arrived.
            let both = [first_id, second_id]
                .iter()
                .all(|id| status.members.iter().any(|m| m.endpoint_id == *id));
            both.then_some(status)
        })
        .await;
        let holder = status
            .members
            .iter()
            .find(|m| m.endpoint_id == first_id)
            .unwrap();
        let loser = status
            .members
            .iter()
            .find(|m| m.endpoint_id == second_id)
            .unwrap();
        assert_eq!(holder.hostname.as_deref(), Some("music"), "{label}");
        assert_eq!(loser.hostname, None, "{label}: no name for the later one");
        let conflict = loser.hostname_conflict.as_ref().expect(label);
        assert_eq!(conflict.name, "music");
        assert_eq!(conflict.holder, first_id);
        assert!(holder.hostname_conflict.is_none(), "{label}");
        // Nothing about being second makes it less of a member.
        assert!(
            loser.online || viewer.agent.endpoint_id() == second_id,
            "{label}"
        );
    }
    // The later one is told so itself, and keeps working.
    let own = second.agent.network_status(network_id).await.unwrap();
    let conflict = own.hostname_conflict.expect("told its name is taken");
    assert_eq!(
        (conflict.name.as_str(), conflict.holder),
        ("music", first_id)
    );
    assert!(
        first
            .agent
            .network_status(network_id)
            .await
            .unwrap()
            .hostname_conflict
            .is_none()
    );
    first.stop().await;
    second.stop().await;
    third.stop().await;
 }
@@ -9,13 +9,17 @@ use std::collections::HashMap;
 use eframe::egui;
-use tsunagi::ipc::{NetworkReport, OverlayPeerReport};
+use tsunagi::ipc::{DataPathReport, NetworkReport, OverlayPeerReport};
 use crate::agent::{AgentClient, Command};
 use crate::exit;
 use crate::format;
 use crate::stats::{Traffic, Unit};
 /// A device that works but not as well as it could: reached through others,
 /// or with a name somebody else holds.
 const AMBER: egui::Color32 = egui::Color32::from_rgb(0xd9, 0xa4, 0x2c);
 /// A readable title for a network's window.
 pub(crate) fn title(network: &NetworkReport) -> String {
    format!("tsunagi · {}", network.name)
@@ -23,7 +27,15 @@ pub(crate) fn title(network: &NetworkReport) -> String {
 /// One row of the device table, built from a tunnel and/or a signed member.
 struct Device<'a> {
    /// The agent's own answer to whether it can be reached by any path.
    online: bool,
    /// How packets reach it.
    path: Option<&'a DataPathReport>,
    /// What to call the member its traffic goes through, when it does.
    via_name: Option<String>,
    /// The name it claimed that another member holds, and what to call that
    /// member.
    name_taken: Option<(&'a str, String)>,
    /// The last hostname seen for it, kept while it is offline.
    hostname: Option<&'a str>,
    /// What clicking the hostname copies: `hostname.network`, so it resolves.
@@ -93,6 +105,13 @@ fn name_cell(ui: &mut egui::Ui, device: &Device<'_>) {
            }
            _ => format::copy_field(ui, &short_id(device.id), device.id),
        }
        if let Some((name, holder)) = &device.name_taken {
            ui.label(egui::RichText::new(egui_phosphor::regular::WARNING).color(AMBER))
                .on_hover_text(format!(
                    "`{name}` was claimed first by {holder}. This device works by address, \
                     but nobody finds it by that name."
                ));
        }
    });
 }
@@ -276,6 +295,11 @@ fn known_ids<'a>(network: &'a NetworkReport, own_id: &str) -> Vec<&'a str> {
    ids
 }
 /// How many other devices can be reached now, by any path.
 pub(crate) fn online_count(network: &NetworkReport, own_id: &str) -> usize {
    devices(network, own_id).iter().filter(|d| d.online).count()
 }
 /// How many other devices this network knows of, connected or not.
 pub(crate) fn known_count(network: &NetworkReport, own_id: &str) -> usize {
    known_ids(network, own_id).len()
@@ -310,10 +334,29 @@ fn devices<'a>(network: &'a NetworkReport, own_id: &str) -> Vec<Device<'a>> {
        .map(|id| {
            let overlay = overlay_peers.iter().find(|p| p.endpoint_id == id);
            let member = network.members.iter().find(|m| m.endpoint_id == id);
-            let online = overlay.is_some_and(|p| p.handshake_secs_ago.is_some());
+            let online = member.is_some_and(|m| m.online)
                || network.peers.iter().any(|p| p.endpoint_id == id);
            let path = network
                .peers
                .iter()
                .find(|p| p.endpoint_id == id)
                .map(|p| &p.path)
                .or_else(|| member.map(|m| &m.path));
            let via_name = match path {
                Some(DataPathReport::Relayed { via, .. }) => Some(exit::name_of(network, via)),
                _ => None,
            };
            let name_taken = member.and_then(|m| {
                m.hostname_conflict
                    .as_ref()
                    .map(|c| (c.name.as_str(), exit::name_of(network, &c.holder)))
            });
            let hostname = hostnames.get(id).copied();
            Device {
                online,
                path,
                via_name,
                name_taken,
                hostname,
                copy: hostname.map(|h| format!("{h}.{}", network.name)),
                id,
@@ -463,15 +506,7 @@ fn device_table(
        .spacing([10.0, 4.0])
        .show(ui, |ui| {
            for (column, header) in [
-                "device",
+                "device", "exit", "proto", "address", "link", "pkts", "bytes", "rate", "plot",
                "exit",
                "proto",
                "address",
                "handshake",
                "pkts",
                "bytes",
                "rate",
                "plot",
            ]
            .into_iter()
            .enumerate()
@@ -496,13 +531,7 @@ fn device_table(
                        ui.weak("—");
                    }
                });
-                cell(ui, 4, &mut widths, |ui| {
+                cell(ui, 4, &mut widths, |ui| link_cell(ui, device));
                    ui.label(
                        device
                            .handshake_secs
                            .map_or_else(|| "offline".to_string(), format::age),
                    );
                });
                match device.overlay.filter(|_| device.online) {
                    Some(peer) => {
@@ -580,6 +609,121 @@ fn device_table(
    ui.memory_mut(|m| m.data.insert_temp(widths_id, widths));
 }
 /// How a device is reached, and how long its tunnel has been up.
 ///
 /// A device reached through others is online, over a longer path, and says
 /// so: the colour is the only difference in how it is treated.
 fn link_cell(ui: &mut egui::Ui, device: &Device<'_>) {
    if !device.online {
        ui.label("offline");
        return;
    }
    let (path, good) = match device.path {
        Some(DataPathReport::Direct { transport }) => (format!("direct · {transport}"), true),
        Some(DataPathReport::Relayed { hops, .. }) => (
            format!(
                "via {} · {hops} hops",
                device.via_name.as_deref().unwrap_or("another device")
            ),
            false,
        ),
        Some(DataPathReport::None) | None => ("no data path".to_string(), false),
    };
    let age = device.handshake_secs.map(format::age);
    let text = match age {
        Some(age) => format!("{path} · {age}"),
        None => path,
    };
    let response = if good {
        ui.label(text)
    } else {
        ui.label(egui::RichText::new(text).color(AMBER))
    };
    if !good {
        response.on_hover_text(match device.path {
            Some(DataPathReport::Relayed { .. }) => {
                "no direct link yet: traffic goes through another device, and moves to a \
                 direct link by itself as soon as one comes up"
            }
            _ => "connected, but no path carries packets yet",
        });
    }
 }
 fn yes_no(value: bool) -> &'static str {
    if value { "yes" } else { "no" }
 }
 #[cfg(test)]
 mod tests {
    #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
    use super::*;
    use tsunagi::ipc::{HostnameConflictReport, MemberReport};
    fn member(id: &str, online: bool, path: DataPathReport) -> MemberReport {
        MemberReport {
            endpoint_id: id.into(),
            online,
            path,
            ..Default::default()
        }
    }
    #[test]
    fn a_device_reached_through_another_is_online_whatever_its_tunnel_says() {
        // The tray used to call a device online only while its tunnel had
        // handshaken, which disagreed with the command line about the same
        // device. Both now read the agent's answer.
        let network = NetworkReport {
            members: vec![
                member("own", true, DataPathReport::None),
                member(
                    "far",
                    true,
                    DataPathReport::Relayed {
                        hops: 2,
                        via: "near".into(),
                    },
                ),
                member("away", false, DataPathReport::None),
            ],
            ..Default::default()
        };
        let all = devices(&network, "own");
        let far = all.iter().find(|d| d.id == "far").unwrap();
        assert!(far.online);
        assert!(matches!(
            far.path,
            Some(DataPathReport::Relayed { hops: 2, .. })
        ));
        assert!(!all.iter().find(|d| d.id == "away").unwrap().online);
        assert_eq!(online_count(&network, "own"), 1);
        assert_eq!(known_count(&network, "own"), 2);
    }
    #[test]
    fn a_name_somebody_else_holds_is_carried_to_the_row() {
        let mut taken = member("late", true, DataPathReport::None);
        taken.hostname_conflict = Some(HostnameConflictReport {
            name: "music".into(),
            holder: "early".into(),
        });
        let network = NetworkReport {
            members: vec![member("early", true, DataPathReport::None), taken],
            ..Default::default()
        };
        let all = devices(&network, "own");
        let late = all.iter().find(|d| d.id == "late").unwrap();
        assert_eq!(
            late.name_taken.as_ref().map(|(name, _)| *name),
            Some("music")
        );
        assert!(
            all.iter()
                .find(|d| d.id == "early")
                .unwrap()
                .name_taken
                .is_none()
        );
    }
 }
@@ -13,6 +13,12 @@ pub(crate) const GOOD: egui::Color32 = egui::Color32::from_rgb(0x3c, 0xb0, 0x4a)
 pub(crate) const LOUD: egui::Color32 = egui::Color32::from_rgb(0xe0, 0x4a, 0x3c);
 /// What to run to turn kernel forwarding on, which the agent never does.
 #[cfg(target_os = "macos")]
 pub(crate) const FORWARDING_COMMAND: &str = "sudo sysctl -w net.inet.ip.forwarding=1";
 #[cfg(target_os = "windows")]
 pub(crate) const FORWARDING_COMMAND: &str =
    "Get-NetIPInterface | Set-NetIPInterface -Forwarding Enabled";
 #[cfg(not(any(target_os = "macos", target_os = "windows")))]
 pub(crate) const FORWARDING_COMMAND: &str = "sudo sysctl -w net.ipv4.ip_forward=1";
 /// One thing wrong, loud enough to read at a glance.
@@ -243,8 +243,8 @@ fn draw_network(
                }
                ui.weak("peers");
                ui.label(format!(
-                    "{} connected · {} known",
+                    "{} online · {} known",
-                    network.peers.len(),
+                    devices::online_count(network, own_id),
                    devices::known_count(network, own_id)
                ));
                ui.end_row();
@@ -42,6 +42,7 @@ fn source(agent: Agent, plugin: Arc<WireguardPlugin>) -> Arc<dyn tsunagi::ipc::R
                            transport: format!("{:?}", peer.transport),
                            rtt_ms: peer.rtt.map(|rtt| rtt.as_millis() as u64),
                            exit_node: peer.exit_node,
                            path: Default::default(),
                        })
                        .collect(),
                    overlay: plugin.overview(net.network_id).map(|view| {
@@ -34,6 +34,9 @@ netwatch = "0.19.3"
 [target.'cfg(unix)'.dependencies]
 nix = { version = "0.31", features = ["user"] }
 [target."cfg(windows)".dependencies]
 windows-service = "0.8"
 [dev-dependencies]
 tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros", "process"] }
 tempfile.workspace = true
@@ -11,6 +11,9 @@ use std::path::PathBuf;
 use std::sync::Arc;
 use std::time::Duration;
 #[cfg(windows)]
 mod winservice;
 use clap::{Args, Parser, Subcommand, ValueEnum};
 use tsunagi::agent::Event;
 use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
@@ -84,6 +87,10 @@ enum Command {
    ExitNode(ExitNodeArgs),
    /// Removes everything this device has stored and starts over.
    Wipe(WipeArgs),
    /// Runs the agent under the Windows service manager. Registered by the
    /// installer, not typed.
    #[command(hide = true)]
    Service(Box<UpArgs>),
 }
 #[derive(Debug, Args)]
@@ -757,12 +764,26 @@ fn main() -> std::process::ExitCode {
    let is_debug = log_filter.contains("debug") || log_filter.contains("trace");
-    tracing_subscriber::fmt()
+    let builder = tracing_subscriber::fmt()
        .with_env_filter(tracing_subscriber::EnvFilter::new(&log_filter))
        .with_target(is_debug)
-        .compact()
+        .compact();
-        .with_writer(std::io::stderr)
+    // A service has no console: it writes to a file.
-        .init();
+    match matches!(cli.command, Command::Service(_))
        .then(open_service_log)
        .flatten()
    {
        Some(file) => builder
            .with_ansi(false)
            .with_writer(std::sync::Mutex::new(file))
            .init(),
        None => builder.with_writer(std::io::stderr).init(),
    }
    #[cfg(windows)]
    if let Command::Service(args) = cli.command {
        return run_service(*args);
    }
    // The library never starts a runtime; this binary owns it.
    let runtime = match tokio::runtime::Builder::new_multi_thread()
@@ -785,6 +806,89 @@ fn main() -> std::process::ExitCode {
    }
 }
 /// The log file of the Windows service, `%ProgramData%\Tsunagi\logs\tsng.log`.
 ///
 /// The previous one is kept as `tsng.log.old` once it passes a few megabytes,
 /// so a service that logs for months does not fill the disk. `None` anywhere
 /// else, and when the file cannot be opened.
 fn open_service_log() -> Option<std::fs::File> {
    if !cfg!(windows) {
        return None;
    }
    let root = std::env::var_os("ProgramData")?;
    open_log_in(&PathBuf::from(root).join("Tsunagi").join("logs"))
 }
 /// Opens `tsng.log` in a directory for appending, moving a large one aside.
 fn open_log_in(dir: &std::path::Path) -> Option<std::fs::File> {
    std::fs::create_dir_all(dir).ok()?;
    let log = dir.join("tsng.log");
    if std::fs::metadata(&log).is_ok_and(|meta| meta.len() > SERVICE_LOG_LIMIT) {
        let _ = std::fs::rename(&log, dir.join("tsng.log.old"));
    }
    std::fs::OpenOptions::new()
        .create(true)
        .append(true)
        .open(log)
        .ok()
 }
 /// Past this size the service log is moved aside, keeping one older file.
 const SERVICE_LOG_LIMIT: u64 = 8 * 1024 * 1024;
 /// The directories and control socket of the Windows service, unless the
 /// service command line names its own.
 ///
 /// A service runs as the system account, whose profile is the wrong home for
 /// a device's identity, so the state is kept under `%ProgramData%\Tsunagi`,
 /// where the installer locks it to administrators and the system. The control
 /// socket stays at the system-wide default every client already looks at,
 /// which is what lets a tray and an unelevated `tsng status` find the service.
 #[cfg_attr(not(windows), allow(dead_code))]
 fn service_defaults(mut args: UpArgs, program_data: &std::path::Path) -> UpArgs {
    let root = program_data.join("Tsunagi");
    args.paths
        .state_dir
        .get_or_insert_with(|| root.join("data"));
    args.paths
        .cache_dir
        .get_or_insert_with(|| root.join("cache"));
    args.control_socket
        .get_or_insert_with(tsunagi::ipc::default_control_socket_path);
    args
 }
 /// Hands the process to the Windows service manager, with `up` as the body.
 #[cfg(windows)]
 fn run_service(args: UpArgs) -> std::process::ExitCode {
    let program_data = std::env::var_os("ProgramData")
        .map(PathBuf::from)
        .filter(|path| path.is_absolute())
        .unwrap_or_else(|| PathBuf::from(r"C:\ProgramData"));
    let args = service_defaults(args, &program_data);
    let args = std::sync::Mutex::new(Some(args));
    let body = move || -> Result<(), String> {
        let Some(args) = args.lock().ok().and_then(|mut slot| slot.take()) else {
            return Err("the service was started twice".to_string());
        };
        let runtime = tokio::runtime::Builder::new_multi_thread()
            .enable_all()
            .build()
            .map_err(|err| format!("cannot start the async runtime: {err}"))?;
        runtime.block_on(up(args)).map_err(|err| err.to_string())
    };
    match winservice::run(Box::new(body)) {
        Ok(()) => std::process::ExitCode::SUCCESS,
        Err(err) => {
            eprintln!(
                "error: cannot run as a service ({err}); `tsng service` is started by the \
                 Windows service manager, use `tsng up` to run the agent yourself"
            );
            std::process::ExitCode::FAILURE
        }
    }
 }
 async fn run(command: Command) -> Result<(), Box<dyn std::error::Error>> {
    match command {
        Command::Id(args) => id(args).await,
@@ -796,6 +900,11 @@ async fn run(command: Command) -> Result<(), Box<dyn std::error::Error>> {
        Command::Dns(args) => dns_command(args).await,
        Command::ExitNode(args) => exit_node_command(args).await,
        Command::Wipe(args) => wipe(args).await,
        // Taken in `main`, before there is a runtime: the service manager
        // wants the process's thread, not an async task.
        Command::Service(_) => Err("`tsng service` is started by the Windows service \
                                    manager; use `tsng up` to run the agent yourself"
            .into()),
    }
 }
@@ -2085,15 +2194,29 @@ async fn broadcast_command(
 }
 /// What to run to turn kernel forwarding on, which the agent never does.
 #[cfg(target_os = "macos")]
 const FORWARDING_COMMAND: &str = "sudo sysctl -w net.inet.ip.forwarding=1";
 #[cfg(target_os = "windows")]
 const FORWARDING_COMMAND: &str = "Get-NetIPInterface | Set-NetIPInterface -Forwarding Enabled   # in an administrator PowerShell";
 #[cfg(not(any(target_os = "macos", target_os = "windows")))]
 const FORWARDING_COMMAND: &str = "sudo sysctl -w net.ipv4.ip_forward=1";
 /// How to keep forwarding on across reboots.
 #[cfg(target_os = "macos")]
 const FORWARDING_PERSIST: &str = "macOS forgets it at reboot; set it again from a boot script \
                                  such as a LaunchDaemon";
 #[cfg(target_os = "windows")]
 const FORWARDING_PERSIST: &str = "the setting is kept across reboots";
 #[cfg(not(any(target_os = "macos", target_os = "windows")))]
 const FORWARDING_PERSIST: &str =
    "keep it across reboots with `net.ipv4.ip_forward = 1` in /etc/sysctl.d/99-tsunagi.conf";
 /// The warning for an exit node whose rules are in place while the kernel is
 /// not forwarding, so that nothing passes.
 fn forwarding_off_note() -> String {
    format!(
        "kernel forwarding is OFF, so nothing passes yet: run `{FORWARDING_COMMAND}` \
-         (and keep it across reboots with `net.ipv4.ip_forward = 1` in \
+         ({FORWARDING_PERSIST})"
         /etc/sysctl.d/99-tsunagi.conf)"
    )
 }
@@ -3033,7 +3156,18 @@ fn dns_absent_section() -> report::Section {
 struct MemberRow<'a> {
    endpoint_id: &'a str,
    hostname: Option<&'a str>,
-    /// `Some` exactly when there is an authenticated session right now.
+    /// Whether it can be reached now, by any path: the agent's own answer,
    /// the same one every other client gives.
    online: bool,
    /// How packets reach it.
    path: Option<&'a tsunagi::ipc::DataPathReport>,
    /// What to call the member the first of those links goes to.
    via_label: Option<String>,
    /// The name it claimed that another member holds, and what to call the
    /// holder.
    name_taken: Option<(&'a str, String)>,
    /// `Some` exactly when there is an authenticated session right now: how
    /// that control connection reaches the peer.
    transport: Option<&'a str>,
    rtt_ms: Option<u64>,
    overlay_address_v4: Option<&'a str>,
@@ -3047,7 +3181,7 @@ struct MemberRow<'a> {
 impl MemberRow<'_> {
    fn online(&self) -> bool {
-        self.transport.is_some()
+        self.online
    }
    /// What to call it: the name it announced — the last one, when it is
@@ -3071,6 +3205,10 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve
        rows.entry(id).or_insert_with(|| MemberRow {
            endpoint_id: id,
            hostname: None,
            online: false,
            path: None,
            via_label: None,
            name_taken: None,
            transport: None,
            rtt_ms: None,
            overlay_address_v4: None,
@@ -3087,11 +3225,19 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve
        row.overlay_address_v4 = member.overlay_address_v4.as_deref();
        row.failed_dials = member.failed_dials;
        row.hostname = member.hostname.as_deref();
        row.online = member.online;
        row.path = Some(&member.path);
        row.name_taken = member
            .hostname_conflict
            .as_ref()
            .map(|conflict| (conflict.name.as_str(), exit_name(network, &conflict.holder)));
    }
    for peer in &network.peers {
        let row = entry(&mut rows, &peer.endpoint_id);
        row.hostname = peer.hostname.as_deref().or(row.hostname);
        row.exit_node = peer.exit_node;
        row.online = true;
        row.path = Some(&peer.path);
        row.transport = Some(&peer.transport);
        row.rtt_ms = peer.rtt_ms;
    }
@@ -3107,6 +3253,9 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve
    for row in rows.values_mut() {
        row.exit_in_use = network.exit.via.as_deref() == Some(row.endpoint_id);
        if let Some(tsunagi::ipc::DataPathReport::Relayed { via, .. }) = row.path {
            row.via_label = Some(exit_name(network, via));
        }
    }
    // This agent is in the roster too — it signs claims like everyone else —
@@ -3162,6 +3311,23 @@ fn network_section(
            ),
        );
    }
    if let Some(taken) = &network.hostname_conflict {
        section.push(
            Row::new(
                Health::Degraded,
                "hostname",
                format!(
                    "`{}` is held by {}, which claimed it first",
                    taken.name,
                    exit_name(network, &taken.holder)
                ),
            )
            .with_note(
                "this device works by address, but nobody finds it by that name. \
                 Pick another with `tsng id hostname <name>`.",
            ),
        );
    }
    if let Some(conflict) = &network.range_conflict {
        section.push(
            Row::new(
@@ -3424,7 +3590,10 @@ fn exit_name(network: &tsunagi::ipc::NetworkReport, id: &str) -> String {
 fn member_row(row: &MemberRow<'_>) -> report::Row {
    use report::{Health, Row};
-    let Some(transport) = row.transport else {
+    let taken = row.name_taken.as_ref().map(|(name, holder)| {
        format!("  ·  name `{name}` is held by {holder}: reachable by address only")
    });
    if !row.online {
        // Away. Not a fault of this agent, and in a mesh of laptops it is the
        // ordinary condition, so it is stated rather than flagged.
        let mut detail = "offline".to_string();
@@ -3434,6 +3603,7 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
        if row.exit_in_use {
            detail.push_str("  ·  your exit node");
        }
        detail.extend(taken);
        let out = Row::new(Health::Info, row.label(), detail);
        return if row.failed_dials > 0 {
            // Attributed to the member it concerns, rather than left as a
@@ -3445,17 +3615,36 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
        } else {
            out
        };
-    };
+    }
-    let direct = transport.eq_ignore_ascii_case("direct");
+    use tsunagi::ipc::DataPathReport;
    let direct_control = row
        .transport
        .is_none_or(|transport| transport.eq_ignore_ascii_case("direct"));
    let tunnel_up = row.tunnel.is_some_and(|tunnel| tunnel.is_up());
    let has_overlay = row.tunnel.is_some();
    // Online is online: a member reached through others is as much here as
    // one reached directly, and is said to be reached that way.
    let (path, data_path_good) = match row.path {
        Some(DataPathReport::Direct { transport }) => (format!("direct {transport}"), true),
        Some(DataPathReport::Relayed { hops, .. }) => (
            format!(
                "relayed via {} ({hops} hops)",
                row.via_label.as_deref().unwrap_or("another member")
            ),
            false,
        ),
        Some(DataPathReport::None) | None => ("connected, no data path yet".to_string(), false),
    };
    let mut detail = String::new();
    if let Some(v4) = row.overlay_address_v4 {
        detail.push_str(&format!("{v4:<15}  "));
    }
-    detail.push_str(&transport.to_lowercase());
+    detail.push_str(&path);
    if let Some(transport) = row.transport {
        detail.push_str(&format!("  ·  control {}", transport.to_lowercase()));
    }
    if let Some(rtt) = row.rtt_ms {
        detail.push_str(&format!("  rtt {rtt}ms"));
    }
@@ -3464,8 +3653,9 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
    } else if row.exit_node {
        detail.push_str("  ·  exit node");
    }
    detail.extend(taken);
-    let health = if !direct || (has_overlay && !tunnel_up) {
+    let health = if !data_path_good || !direct_control || (has_overlay && !tunnel_up) {
        Health::Degraded
    } else {
        Health::Good
@@ -4767,6 +4957,7 @@ async fn build_report(
                        transport: peer.transport.to_string(),
                        rtt_ms: peer.rtt.map(|rtt| rtt.as_millis() as u64),
                        exit_node: peer.exit_node,
                        path: data_path_report(&peer.data_path),
                    })
                    .collect(),
                members: network
@@ -4788,6 +4979,9 @@ async fn build_report(
                            .last_hostname
                            .clone()
                            .or_else(|| member.hostname.clone()),
                        online: member.online,
                        path: data_path_report(&member.data_path),
                        hostname_conflict: member.hostname_conflict.as_ref().map(conflict_report),
                    })
                    .collect(),
                range: network.range.map(|range| range.to_string()),
@@ -4799,6 +4993,7 @@ async fn build_report(
                    network.metrics.control_messages_received,
                ),
                overlay,
                hostname_conflict: network.hostname_conflict.as_ref().map(conflict_report),
            }
        })
        .collect();
@@ -4832,6 +5027,28 @@ async fn build_report(
    }
 }
 fn data_path_report(path: &tsunagi::agent::DataPath) -> tsunagi::ipc::DataPathReport {
    match path {
        tsunagi::agent::DataPath::None => tsunagi::ipc::DataPathReport::None,
        tsunagi::agent::DataPath::Direct { transport } => tsunagi::ipc::DataPathReport::Direct {
            transport: transport.clone(),
        },
        tsunagi::agent::DataPath::Relayed { hops, via } => tsunagi::ipc::DataPathReport::Relayed {
            hops: *hops,
            via: via.to_string(),
        },
    }
 }
 fn conflict_report(
    conflict: &tsunagi::agent::HostnameConflict,
 ) -> tsunagi::ipc::HostnameConflictReport {
    tsunagi::ipc::HostnameConflictReport {
        name: conflict.name.clone(),
        holder: conflict.holder.to_string(),
    }
 }
 /// One network's exit-node settings, as the control interface reports them.
 fn exit_report(network: &tsunagi::agent::NetworkStatus) -> tsunagi::ipc::ExitReport {
    let rules = |report: &Option<tsunagi::overlay::RuleSetReport>| {
@@ -4886,7 +5103,14 @@ async fn stop_signal() -> &'static str {
            _ = terminate.recv() => "terminated",
        }
    }
-    #[cfg(not(unix))]
+    #[cfg(windows)]
    {
        tokio::select! {
            _ = tokio::signal::ctrl_c() => "interrupted",
            _ = winservice::stop_requested() => "stopped by the service manager",
        }
    }
    #[cfg(not(any(unix, windows)))]
    {
        let _ = tokio::signal::ctrl_c().await;
        "interrupted"
@@ -5130,6 +5354,7 @@ mod status_tests {
    fn network_after_a_peer_returned() -> NetworkReport {
        NetworkReport {
            exit: Default::default(),
            hostname_conflict: None,
            broadcast: true,
            name: "LAB".into(),
            network_id: "xa7gyz".into(),
@@ -5144,6 +5369,9 @@ mod status_tests {
                transport: "direct".into(),
                rtt_ms: Some(24),
                exit_node: false,
                path: tsunagi::ipc::DataPathReport::Direct {
                    transport: "wg".into(),
                },
            }],
            members: vec![
                MemberReport {
@@ -5151,12 +5379,18 @@ mod status_tests {
                    overlay_address_v4: Some("10.13.37.69".into()),
                    failed_dials: 0,
                    hostname: None,
                    ..Default::default()
                },
                MemberReport {
                    endpoint_id: ONLINE.into(),
                    overlay_address_v4: Some("10.13.37.237".into()),
                    failed_dials: 0,
                    hostname: None,
                    online: true,
                    path: tsunagi::ipc::DataPathReport::Direct {
                        transport: "wg".into(),
                    },
                    ..Default::default()
                },
            ],
            range: Some("10.13.37.0/24".into()),
@@ -5398,6 +5632,7 @@ mod status_tests {
            overlay_address_v4: Some("10.13.37.99".into()),
            failed_dials: 9,
            hostname: None,
            ..Default::default()
        });
        let rows = member_rows(&network, OWN);
@@ -5435,10 +5670,7 @@ mod status_tests {
        out.push(network_section(&network, OWN, false));
        let text = out.render(false);
        assert_eq!(out.worst(), Health::Degraded, "{text}");
-        assert!(
+        assert!(text.contains(FORWARDING_COMMAND), "{text}");
            text.contains("sudo sysctl -w net.ipv4.ip_forward=1"),
            "{text}"
        );
        assert!(text.contains("ok (nat + forward)"), "{text}");
        network.exit.forwarding = Some(true);
@@ -5521,6 +5753,7 @@ mod status_tests {
            overlay_address_v4: Some("10.13.37.99".into()),
            failed_dials: 0,
            hostname: Some("laptop".into()),
            ..Default::default()
        });
        let rows = member_rows(&network, OWN);
@@ -5541,6 +5774,7 @@ mod status_tests {
            overlay_address_v4: Some("10.13.37.99".into()),
            failed_dials: 9,
            hostname: None,
            ..Default::default()
        });
        let mut out = report::Report::new();
@@ -5554,6 +5788,69 @@ mod status_tests {
        assert!(text.contains("1 of 2 online"), "{text}");
    }
    #[test]
    fn a_member_reached_through_another_is_online_and_says_through_whom() {
        // No direct link, a path through somebody else: online all the same,
        // and said to be, rather than being shown as away or as a fault.
        let mut network = network_after_a_peer_returned();
        network.members.push(MemberReport {
            endpoint_id: AWAY.into(),
            overlay_address_v4: Some("10.13.37.99".into()),
            hostname: Some("laptop".into()),
            online: true,
            path: tsunagi::ipc::DataPathReport::Relayed {
                hops: 2,
                via: ONLINE.into(),
            },
            ..Default::default()
        });
        let rows = member_rows(&network, OWN);
        let away = rows.iter().find(|row| row.endpoint_id == AWAY).unwrap();
        assert!(away.online());
        let mut out = report::Report::new();
        out.push(network_section(&network, OWN, false));
        let text = out.render(false);
        assert!(text.contains("2 of 2 online"), "{text}");
        assert!(
            text.contains("relayed via music (2 hops)"),
            "names the member it goes through: {text}"
        );
        assert!(!text.contains("offline"), "{text}");
        assert_eq!(out.worst(), Health::Degraded, "{text}");
    }
    #[test]
    fn a_name_somebody_else_holds_is_said_on_both_the_member_and_this_device() {
        let mut network = network_after_a_peer_returned();
        network.members[1].hostname_conflict = Some(tsunagi::ipc::HostnameConflictReport {
            name: "music".into(),
            holder: OWN.into(),
        });
        let text = {
            let mut out = report::Report::new();
            out.push(network_section(&network, OWN, false));
            out.render(false)
        };
        assert!(
            text.contains("name `music` is held by") && text.contains("reachable by address only"),
            "{text}"
        );
        let mut network = network_after_a_peer_returned();
        network.hostname_conflict = Some(tsunagi::ipc::HostnameConflictReport {
            name: "studio".into(),
            holder: ONLINE.into(),
        });
        let mut out = report::Report::new();
        out.push(network_section(&network, OWN, false));
        let text = out.render(false);
        assert!(text.contains("`studio` is held by music"), "{text}");
        assert!(text.contains("tsng id hostname"), "{text}");
        assert_eq!(out.worst(), Health::Degraded, "{text}");
    }
    #[test]
    fn a_relayed_peer_is_graded_as_degraded_quality() {
        let mut network = network_after_a_peer_returned();
@@ -5583,6 +5880,9 @@ mod status_tests {
        // With a peer connected the same counter is just history.
        let mut network = network_after_a_peer_returned();
        network.peers.clear();
        for member in &mut network.members {
            member.online = false;
        }
        network.overlay = Some(overlay(Vec::new()));
        network.handshake_failures = 4;
@@ -5932,3 +6232,93 @@ mod network_context_tests {
        assert_eq!(standing, NetworkStanding::Stopped);
    }
 }
 #[cfg(test)]
 mod service_log_tests {
    #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
    use std::io::Write;
    use super::*;
    #[test]
    fn the_log_appends_and_a_large_one_is_moved_aside() {
        let dir = tempfile::tempdir().unwrap();
        let logs = dir.path().join("logs");
        let mut file = open_log_in(&logs).unwrap();
        writeln!(file, "one").unwrap();
        drop(file);
        let mut file = open_log_in(&logs).unwrap();
        writeln!(file, "two").unwrap();
        drop(file);
        assert_eq!(
            std::fs::read_to_string(logs.join("tsng.log")).unwrap(),
            "one\ntwo\n"
        );
        let big = std::fs::File::options()
            .append(true)
            .open(logs.join("tsng.log"))
            .unwrap();
        big.set_len(SERVICE_LOG_LIMIT + 1).unwrap();
        drop(big);
        let file = open_log_in(&logs).unwrap();
        assert_eq!(file.metadata().unwrap().len(), 0);
        assert!(logs.join("tsng.log.old").exists());
    }
    #[test]
    fn a_log_that_cannot_be_opened_is_none() {
        let dir = tempfile::tempdir().unwrap();
        let file = dir.path().join("file");
        std::fs::write(&file, b"x").unwrap();
        // A directory cannot be made under a regular file.
        assert!(open_log_in(&file.join("logs")).is_none());
    }
    #[test]
    fn the_service_keeps_its_state_under_program_data_unless_told_otherwise() {
        use clap::Parser;
        let parse = |extra: &[&str]| {
            let mut words = vec!["tsng", "service"];
            words.extend_from_slice(extra);
            match Cli::try_parse_from(words).unwrap().command {
                Command::Service(args) => *args,
                other => panic!("{other:?}"),
            }
        };
        let root = std::path::Path::new("/pd");
        let args = service_defaults(parse(&[]), root);
        assert_eq!(
            args.paths.state_dir,
            Some(root.join("Tsunagi").join("data"))
        );
        assert_eq!(
            args.paths.cache_dir,
            Some(root.join("Tsunagi").join("cache"))
        );
        assert_eq!(
            args.control_socket,
            Some(tsunagi::ipc::default_control_socket_path())
        );
        let args = service_defaults(
            parse(&["--state-dir", "/mine", "--control-socket", "/mine/sock"]),
            root,
        );
        assert_eq!(args.paths.state_dir, Some(PathBuf::from("/mine")));
        assert_eq!(args.control_socket, Some(PathBuf::from("/mine/sock")));
    }
    #[test]
    fn the_service_command_is_refused_off_the_service_manager() {
        use clap::Parser;
        let cli = Cli::try_parse_from(["tsng", "service"]).unwrap();
        assert!(matches!(cli.command, Command::Service(_)));
        let runtime = tokio::runtime::Builder::new_current_thread()
            .build()
            .unwrap();
        let err = runtime.block_on(run(cli.command)).unwrap_err();
        assert!(err.to_string().contains("service manager"), "{err}");
    }
 }
@@ -0,0 +1,121 @@
 //! The agent as a Windows service.
 //!
 //! `tsng service` is what the installer registers with the service control
 //! manager (`sc create`); nobody types it. The manager starts the process,
 //! which must call into the dispatcher quickly and then report its state, so
 //! this is the only place the Windows service API is touched. Everything the
 //! agent does is the same `up` that runs in a console; the only differences
 //! are that it stops when the manager says so rather than on Ctrl-C, and that
 //! it logs to a file because a service has no console.
 //!
 //! The one `unsafe` here is not ours: `define_windows_service!` expands to the
 //! `extern "system"` entry point the manager calls, which has to be unsafe.
 #![allow(unsafe_code)]
 use std::ffi::OsString;
 use std::sync::OnceLock;
 use std::time::Duration;
 use windows_service::service::{
    ServiceControl, ServiceControlAccept, ServiceExitCode, ServiceState, ServiceStatus, ServiceType,
 };
 use windows_service::service_control_handler::{
    self, ServiceControlHandlerResult, ServiceStatusHandle,
 };
 use windows_service::{define_windows_service, service_dispatcher};
 /// The name the installer registers the service under.
 pub const SERVICE_NAME: &str = "Tsunagi";
 /// How long the manager waits for a stop before it gives up on us. The agent's
 /// own shutdown is bounded and shorter.
 const STOP_WAIT: Duration = Duration::from_secs(25);
 /// Set when the manager asks the service to stop.
 static STOP: tokio::sync::Notify = tokio::sync::Notify::const_new();
 /// What `up` does, handed over by `main` because the manager's entry point
 /// cannot take arguments.
 type Body = Box<dyn Fn() -> Result<(), String> + Send + Sync>;
 static BODY: OnceLock<Body> = OnceLock::new();
 /// Where the manager's status handle is kept, so the stop handler can report.
 static STATUS: OnceLock<ServiceStatusHandle> = OnceLock::new();
 /// Resolves when the manager has asked the service to stop.
 pub async fn stop_requested() {
    STOP.notified().await;
 }
 define_windows_service!(ffi_service_main, service_main);
 /// Hands the process to the service control manager and returns when the
 /// service has stopped. Fails at once when not started by the manager, which
 /// is how running `tsng service` in a console is reported.
 pub fn run(body: Body) -> Result<(), windows_service::Error> {
    let _ = BODY.set(body);
    service_dispatcher::start(SERVICE_NAME, ffi_service_main)
 }
 fn service_main(_arguments: Vec<OsString>) {
    let result = serve();
    let code = match &result {
        Ok(()) => ServiceExitCode::Win32(0),
        Err(err) => {
            tracing::error!("the service failed: {err}");
            // A non-zero exit lets the recovery actions restart it.
            ServiceExitCode::ServiceSpecific(1)
        }
    };
    if let Some(handle) = STATUS.get() {
        let _ = handle.set_service_status(status(ServiceState::Stopped, code, Duration::ZERO));
    }
 }
 fn serve() -> Result<(), String> {
    let handle = service_control_handler::register(SERVICE_NAME, |control| match control {
        ServiceControl::Stop | ServiceControl::Shutdown => {
            if let Some(handle) = STATUS.get() {
                let _ = handle.set_service_status(status(
                    ServiceState::StopPending,
                    ServiceExitCode::Win32(0),
                    STOP_WAIT,
                ));
            }
            STOP.notify_one();
            ServiceControlHandlerResult::NoError
        }
        ServiceControl::Interrogate => ServiceControlHandlerResult::NoError,
        _ => ServiceControlHandlerResult::NotImplemented,
    })
    .map_err(|err| format!("cannot register with the service control manager: {err}"))?;
    let _ = STATUS.set(handle);
    handle
        .set_service_status(status(
            ServiceState::Running,
            ServiceExitCode::Win32(0),
            Duration::ZERO,
        ))
        .map_err(|err| format!("cannot report the service as running: {err}"))?;
    match BODY.get() {
        Some(body) => body(),
        None => Err("the service has nothing to run".to_string()),
    }
 }
 fn status(state: ServiceState, exit_code: ServiceExitCode, wait_hint: Duration) -> ServiceStatus {
    ServiceStatus {
        service_type: ServiceType::OWN_PROCESS,
        current_state: state,
        controls_accepted: if state == ServiceState::Running {
            ServiceControlAccept::STOP | ServiceControlAccept::SHUTDOWN
        } else {
            ServiceControlAccept::empty()
        },
        exit_code,
        checkpoint: 0,
        wait_hint,
        process_id: None,
    }
 }
@@ -22,7 +22,7 @@ service's socket at `/run/tsunagi/agent.sock`):
 ```sh
 sudo tsng status
-sudo tsng join <network-name> <tsn1…secret>
+sudo tsng join -n <network-name> -s <tsn1…secret>
 ```
 ## How it runs
@@ -28,7 +28,7 @@ group, so after `sudo usermod -aG tsunagi "$USER"` (and logging in again) no
 ```sh
 sudo tsng status
-sudo tsng join <network-name> <tsn1…secret>
+sudo tsng join -n <network-name> -s <tsn1…secret>
 ```
 ## As your own user — one capability, no service
@@ -48,7 +48,7 @@ Done. Start the agent with:
 Control it with sudo (the CLI finds the service's socket automatically):
    sudo tsng status
-    sudo tsng join <network-name> <tsn1…secret>
+    sudo tsng join -n <network-name> -s <tsn1…secret>
 EOF
 else
    cat <<EOF
@@ -15,7 +15,7 @@ socket and do **not** need `sudo`:
 ```sh
 tsng status
-tsng join <network-name> <tsn1…secret>
+tsng join -n <network-name> -s <tsn1…secret>
 ```
 The agent uses fixed system paths, so the commands find it regardless of whose
@@ -1,5 +1,34 @@
 # Running tsunagi on Windows
 ## The installer (recommended)
 `tsunagi-setup-<version>-x86_64.exe` does everything in one go. Run it as an
 administrator (it asks):
 - installs `tsng.exe`, the tray app and **Wintun** to `C:\Program Files\Tsunagi`;
 - registers and starts the **Tsunagi** Windows service (it starts at boot and
  restarts after a crash), running as the system account;
 - adds `tsng` to the PATH and an inbound Windows Firewall rule for the agent
  (both are tasks you can untick);
 - starts the tray icon at sign-in.
 Where things live: the device's identity and networks in
 `%ProgramData%\Tsunagi\data` (readable only by administrators and the system),
 the log in `%ProgramData%\Tsunagi\logs\tsng.log`. The service keeps its own
 state: networks joined by an agent you started by hand (`tsng up`, which keeps
 its data in your user profile) are not carried over; join them again with
 `tsng join`. The installer stops such an agent, because it cannot run beside the
 service.
 `tsng status`, `tsng join` and the tray work from an ordinary, non-elevated
 terminal. Manage the service with the Services app or `sc stop Tsunagi` /
 `sc start Tsunagi`. Silent install: `/VERYSILENT /SUPPRESSMSGBOXES`; choose
 tasks with `/TASKS="!autostart,addtopath,firewall"`. Uninstalling removes the
 service, the firewall rule and the PATH entry, and asks before deleting the
 data.
 ## By hand (the zip)
 This archive holds `tsng.exe`. The overlay interface is a Wintun adapter, so
 the agent needs two things to carry real traffic:
@@ -0,0 +1,242 @@
 ; Inno Setup script for the Tsunagi Windows installer.
 ;
 ; One installer that leaves a working machine:
 ;   * the agent (tsng.exe), as a Windows service that starts at boot
 ;   * Wintun, the driver library behind the overlay interface
 ;   * the tray application, started at sign-in
 ;   * `tsng` on the PATH, and an inbound Defender Firewall rule for the agent
 ;
 ; Built in CI (see .github/workflows/release.yml); locally:
 ;   ISCC.exe /DAppVersion=0.1.0 /DSourceDir=<dir with tsng.exe and tsunagi-tray.exe>
 ;            /DWintunDir=<wintun\bin\amd64> /O<output dir> dist\windows\tsunagi.iss
 ; Silent install: tsunagi-setup-<version>-x86_64.exe /VERYSILENT /SUPPRESSMSGBOXES
 ; Pick tasks with /TASKS="!autostart,addtopath,firewall".
 #ifndef AppVersion
  #define AppVersion "0.0.0"
 #endif
 #ifndef SourceDir
  #error SourceDir must point at the directory holding tsng.exe and tsunagi-tray.exe
 #endif
 #ifndef WintunDir
  #error WintunDir must point at the directory holding wintun.dll
 #endif
 #define ServiceName "Tsunagi"
 [Setup]
 AppId={{F635E5DD-250D-499F-B0B3-AFC7CC64A15D}
 AppName=Tsunagi
 AppVersion={#AppVersion}
 AppPublisher=Tsunagi
 DefaultDirName={autopf}\Tsunagi
 DefaultGroupName=Tsunagi
 DisableProgramGroupPage=yes
 PrivilegesRequired=admin
 ArchitecturesAllowed=x64compatible
 ArchitecturesInstallIn64BitMode=x64compatible
 OutputBaseFilename=tsunagi-setup-{#AppVersion}-x86_64
 Compression=lzma2
 SolidCompression=yes
 WizardStyle=modern
 ChangesEnvironment=yes
 UninstallDisplayName=Tsunagi
 ; The service and the tray are stopped by hand below, so the installer does
 ; not need to ask Windows to close applications.
 CloseApplications=no
 [Tasks]
 Name: "autostart"; Description: "Start the tray icon when I sign in"; GroupDescription: "Tsunagi:"
 Name: "addtopath"; Description: "Add tsng to the PATH"; GroupDescription: "Tsunagi:"
 Name: "firewall"; Description: "Allow the agent through Windows Firewall (direct connections between devices)"; GroupDescription: "Tsunagi:"
 [Files]
 Source: "{#SourceDir}\tsng.exe"; DestDir: "{app}"; Flags: ignoreversion
 Source: "{#SourceDir}\tsunagi-tray.exe"; DestDir: "{app}"; Flags: ignoreversion
 ; Wintun is the unmodified, signed library from wintun.net; its licence allows
 ; shipping it with an application and asks for the licence text beside it.
 ; In the archive the text sits at its root, two levels above bin\amd64.
 Source: "{#WintunDir}\wintun.dll"; DestDir: "{app}"; Flags: ignoreversion
 Source: "{#WintunDir}\..\..\LICENSE.txt"; DestDir: "{app}\licenses"; DestName: "wintun-LICENSE.txt"; Flags: ignoreversion
 [Dirs]
 ; Identity and network secrets live here; locked down below.
 Name: "{commonappdata}\Tsunagi\data"
 Name: "{commonappdata}\Tsunagi\cache"
 Name: "{commonappdata}\Tsunagi\logs"
 [Icons]
 Name: "{group}\Tsunagi"; Filename: "{app}\tsunagi-tray.exe"
 Name: "{commonstartup}\Tsunagi"; Filename: "{app}\tsunagi-tray.exe"; Tasks: autostart
 [Registry]
 Root: HKLM; Subkey: "SYSTEM\CurrentControlSet\Control\Session Manager\Environment"; \
    ValueType: expandsz; ValueName: "Path"; ValueData: "{olddata};{app}"; \
    Check: NeedsAddPath('{app}'); Tasks: addtopath
 [Run]
 ; Only the system account and administrators may read the state: it holds the
 ; device's key and every network's secret. SIDs, so it works in any language.
 Filename: "{sys}\icacls.exe"; \
    Parameters: """{commonappdata}\Tsunagi\data"" /inheritance:r /grant:r *S-1-5-18:(OI)(CI)F *S-1-5-32-544:(OI)(CI)F"; \
    Flags: runhidden; StatusMsg: "Protecting the agent's data..."
 Filename: "{sys}\icacls.exe"; \
    Parameters: """{commonappdata}\Tsunagi\cache"" /inheritance:r /grant:r *S-1-5-18:(OI)(CI)F *S-1-5-32-544:(OI)(CI)F"; \
    Flags: runhidden
 Filename: "{sys}\icacls.exe"; \
    Parameters: """{commonappdata}\Tsunagi\logs"" /inheritance:r /grant:r *S-1-5-18:(OI)(CI)F *S-1-5-32-544:(OI)(CI)F"; \
    Flags: runhidden
 ; The tray runs as the person who installed, not elevated, like it does at sign-in.
 Filename: "{app}\tsunagi-tray.exe"; Description: "Start the Tsunagi tray"; \
    Flags: nowait postinstall skipifsilent runasoriginaluser
 [UninstallRun]
 Filename: "{sys}\netsh.exe"; Parameters: "advfirewall firewall delete rule name=""Tsunagi"""; \
    Flags: runhidden; RunOnceId: "RemoveFirewallRule"
 [Code]
 const
  ServiceName = '{#ServiceName}';
  EnvKey = 'SYSTEM\CurrentControlSet\Control\Session Manager\Environment';
 function Sc(const Params: String): Integer;
 var
  Code: Integer;
 begin
  if Exec(ExpandConstant('{sys}\sc.exe'), Params, '', SW_HIDE, ewWaitUntilTerminated, Code) then
    Result := Code
  else
    Result := -1;
 end;
 function ServiceExists: Boolean;
 begin
  // 1060 is ERROR_SERVICE_DOES_NOT_EXIST.
  Result := Sc('query ' + ServiceName) <> 1060;
 end;
 procedure StopService;
 var
  Code: Integer;
 begin
  // `net stop` waits until the service has stopped, or says it was not running.
  Exec(ExpandConstant('{sys}\net.exe'), 'stop ' + ServiceName, '', SW_HIDE, ewWaitUntilTerminated, Code);
 end;
 procedure StopTray;
 var
  Code: Integer;
 begin
  Exec(ExpandConstant('{sys}\taskkill.exe'), '/F /IM tsunagi-tray.exe', '', SW_HIDE, ewWaitUntilTerminated, Code);
 end;
 // An agent started by hand (`tsng up`) holds the control pipe and the Wintun
 // adapter, and the file about to be replaced: it cannot run beside the service.
 procedure StopStrayAgents;
 var
  Code: Integer;
 begin
  Exec(ExpandConstant('{sys}\taskkill.exe'), '/F /IM tsng.exe', '', SW_HIDE, ewWaitUntilTerminated, Code);
 end;
 function NeedsAddPath(Param: String): Boolean;
 var
  Current: String;
 begin
  if not RegQueryStringValue(HKLM, EnvKey, 'Path', Current) then
  begin
    Result := True;
    exit;
  end;
  Result := Pos(';' + Uppercase(ExpandConstant(Param)) + ';', ';' + Uppercase(Current) + ';') = 0;
 end;
 procedure RemoveFromPath;
 var
  Current, Entry: String;
  At: Integer;
 begin
  if not RegQueryStringValue(HKLM, EnvKey, 'Path', Current) then
    exit;
  Entry := ';' + ExpandConstant('{app}');
  At := Pos(Uppercase(Entry), Uppercase(Current));
  if At > 0 then
  begin
    Delete(Current, At, Length(Entry));
    RegWriteExpandStringValue(HKLM, EnvKey, 'Path', Current);
  end;
 end;
 // Stops what is about to be replaced: the files are in use while it runs.
 function PrepareToInstall(var NeedsRestart: Boolean): String;
 begin
  StopTray;
  if ServiceExists then
    StopService;
  StopStrayAgents;
  Result := '';
 end;
 procedure InstallService;
 var
  App: String;
 begin
  App := ExpandConstant('{app}\tsng.exe');
  // The command line is `tsng service`: it keeps its state under
  // %ProgramData%\Tsunagi and serves the system-wide control socket.
  if ServiceExists then
    Sc('config ' + ServiceName + ' binPath= "\"' + App + '\" service" start= auto obj= LocalSystem')
  else
    Sc('create ' + ServiceName + ' binPath= "\"' + App + '\" service" start= auto obj= LocalSystem DisplayName= "Tsunagi agent"');
  Sc('description ' + ServiceName + ' "Tsunagi: private mesh network agent"');
  // Restart after a crash, backing off, and forget the failures after a day.
  Sc('failure ' + ServiceName + ' reset= 86400 actions= restart/5000/restart/5000/restart/30000');
  Sc('start ' + ServiceName);
 end;
 procedure InstallFirewallRule;
 var
  Code: Integer;
  Netsh: String;
 begin
  Netsh := ExpandConstant('{sys}\netsh.exe');
  Exec(Netsh, 'advfirewall firewall delete rule name="Tsunagi"', '', SW_HIDE, ewWaitUntilTerminated, Code);
  if WizardIsTaskSelected('firewall') then
    Exec(Netsh,
      'advfirewall firewall add rule name="Tsunagi" dir=in action=allow enable=yes profile=any program="' +
      ExpandConstant('{app}\tsng.exe') + '"',
      '', SW_HIDE, ewWaitUntilTerminated, Code);
 end;
 procedure CurStepChanged(CurStep: TSetupStep);
 begin
  if CurStep = ssPostInstall then
  begin
    InstallFirewallRule;
    InstallService;
  end;
 end;
 procedure CurUninstallStepChanged(CurUninstallStep: TUninstallStep);
 begin
  if CurUninstallStep = usUninstall then
  begin
    StopTray;
    if ServiceExists then
    begin
      StopService;
      Sc('delete ' + ServiceName);
    end;
    StopStrayAgents;
    RemoveFromPath;
  end;
  if CurUninstallStep = usPostUninstall then
  begin
    // The data is the device's identity and its networks: kept unless asked,
    // and never removed by a silent uninstall.
    if (not UninstallSilent) and
       (MsgBox('Also delete this device''s identity, networks and logs?' + #13#10 +
               'Choose No to keep them for a later reinstall.', mbConfirmation, MB_YESNO or MB_DEFBUTTON2) = IDYES) then
      DelTree(ExpandConstant('{commonappdata}\Tsunagi'), True, True, True);
  end;
 end;
@@ -59,6 +59,49 @@ be an exit node in one network and not in another.
  specific IPv6 routes keep working, and so does the agent's own IPv6. A host
  with no IPv6 has nothing to block and is left alone.
 ## macOS
 macOS has no `iptables` and no per-user policy routing, so the same behaviour
 is built from `pf` and `route`. The agent runs as root there.
 - **Offering:** a `pf` anchor `com.apple/tsunagi-exit-<interface>` (the stock
  `/etc/pf.conf` already evaluates `com.apple/*`) holds
  `nat on <egress> from <range> to any -> (<egress>)` and a `pass in` rule.
  `pf` is enabled if it was off, and that reference is released when the rules
  go. Forwarding is only read: `sudo sysctl -w net.inet.ip.forwarding=1`, which
  macOS forgets at reboot. The egress is the default route's interface **at the
  time the rules are applied**; after it changes (Wi-Fi to Ethernet) switch the
  offer off and on.
 - **Using:** `0.0.0.0/1` and `128.0.0.0/1` routes through the overlay interface,
  and a `pf` `route-to` rule that sends the agent's own traffic (matched by its
  user id, root) back out the physical interface, with translation to that
  interface's address. Anything else you run as root also bypasses the tunnel.
  IPv6 is blocked with rejecting `::/1` and `8000::/1` routes.
 - Not verified on a live host when this was written: run `tsng exit-node` and
  check `sudo pfctl -a com.apple/tsunagi-exit-utunN -s rules -s nat`.
 ## Windows
 - **Offering:** each offered range gets a WinNAT object
  `tsunagi-exit:<interface>:<range>` (`New-NetNat`), removed with the offer, the
  network and the agent. Forwarding is read for the overlay interface and never
  changed; Windows forwards per interface, so enable it on the ones involved
  (`Get-NetIPInterface | Set-NetIPInterface -Forwarding Enabled`, administrator
  PowerShell). Some Windows versions allow one NAT per host; a second range then
  reports its own failure.
 - **Using:** Windows has no per-user routing, so the agent's own traffic is kept
  off the tunnel *by destination*. `0.0.0.0/1` and `128.0.0.0/1` go through the
  overlay interface; a `/32` route with metric 7373 (the marker for "ours")
  through the physical default gateway is added for every address the agent
  talks to directly: verified direct paths of its connections, plus the relays
  it uses, resolved when the rules are applied. The set follows path changes. A
  path that appears after the last apply goes through the tunnel until the next
  one. IPv6 is blackholed with `::/1` and `8000::/1` on the overlay interface,
  so applications fall back to IPv4 after a delay rather than at once.
  Everything is in the active store: a reboot clears it.
 - Not verified on a live host when this was written. To inspect:
  `Get-NetRoute -PolicyStore ActiveStore | ? { $_.RouteMetric -eq 7373 -or $_.InterfaceAlias -like 'tsun*' }`.
 ## What it does not do
 - **The agent must run as a system service** (its own user). The uid exemption
@@ -9,7 +9,7 @@ Two versions exist and are independent:
 - **Identity scheme**, `tsunagi-network-id-v1`. Frozen. Changing it creates a
  different network space for the same name and secret.
- **Control protocol**, ALPN `tsunagi/ctrl/4`, `PROTOCOL_VERSION = 4`.
+- **Control protocol**, ALPN `tsunagi/ctrl/5`, `PROTOCOL_VERSION = 5`.
 Upgrading the crate or bumping the control protocol must never change an
 existing `NetworkId`.
@@ -135,7 +135,7 @@ event does not report a network id.
 ## The data plane protocol
 IP plugin packets never travel on a control connection. They use their own
-ALPN, `tsunagi/data/4`, on their own iroh connection:
+ALPN, `tsunagi/data/5`, on their own iroh connection:
 ```text
 initiator -> responder : (the same membership handshake as above)
@@ -172,32 +172,46 @@ holding up subsequent packets. Packet IDs are scoped to a QUIC connection;
 reassembly is created only after the membership handshake. All limits live in
 `config.rs`. There is no transport-layer retransmission added by this framing.
-Version 4 adds a routing envelope inside the transport fragmentation framing:
+Version 4 added a routing envelope inside the transport fragmentation framing,
 and version 5 added the end-to-end protocol to it:
 | field | bytes | encoding |
 |---|---:|---|
-| envelope version | 1 | `1` |
+| envelope version | 1 | `2` |
 | remaining hops | 1 | initially `16`; transit decrements, drops at `1` |
 | source peer | 32 | original endpoint public key |
 | destination peer | 32 | final endpoint public key |
 | flow id | 8 | big endian opaque stable hash supplied by the plugin |
 | protocol | 8 | big endian FNV-1a of the end-to-end protocol id (`wg`, `tcp-tls`, ...) |
 | payload | remaining | end-to-end encrypted plugin bytes |
-The fixed header is 74 bytes. Zero/oversized hop limits, unknown versions,
+The fixed header is 82 bytes. Zero/oversized hop limits, unknown versions,
 short frames, oversized datagrams and unknown sources are rejected. A frame
-addressed here is delivered to its source's protocol inbox; transit goes straight
+addressed here is delivered to the inbox of its source and protocol; transit
-to its next transport. The network and protocol are bound to the authenticated
+goes straight to its next transport, whatever kind of transport that is. The
-transport connection, not supplied by the frame. Intermediate nodes cannot
+network is bound to the authenticated transport connection, not supplied by
-decrypt or authenticate the inner WireGuard payload; the destination does that.
+the frame. The protocol names which end-to-end tunnel the payload belongs to
-Flow ids are routing hints, not authorization proofs.
+and is independent of the links it crosses: a datagram of one protocol can
 travel over a link opened for another, and a middle member needs no protocol
 in common with either end. Intermediate nodes cannot decrypt or authenticate
 the inner payload; the destination's tunnel does that, so a frame naming the
 wrong protocol or a forged source is simply discarded there. Flow ids are
 routing hints, not authorization proofs.
-Control ALPN 3 carries `Reachable { links: [{ peer, protocol }] }`. Each row
+`Reachable { links: [{ peer, protocol }] }` lists the sender's direct links;
-belongs to the authenticated sender and is replaced atomically, expires after
+`protocol` is the kind of transport the link is, for diagnostics, and routing
-90 seconds, and is withdrawn on session closure. Only compatible authenticated
+ignores it. Each row belongs to the authenticated sender and is replaced
-members enter a protocol's graph; local edges always come from actual links.
+atomically, expires after 90 seconds, and is withdrawn on session closure. Only
 authenticated members enter the graph; local edges always come from actual
 links.
 Announcements refresh on the existing control maintenance interval and are
 sent immediately on link changes. Topology stays volatile, outside signed state.
 Version 5 also adds `hostname_since` to a signed hostname claim (signing domain
 `tsunagi-signed-record-v3`): a name belongs to whoever claimed it first. Stored
 records signed before are discarded on upgrade and re-published by their
 authors.
 Upgrade both endpoints and intermediate peers together. Older control and data
 ALPNs cannot interoperate. The network secret, device identity, identity
 derivation, transcript encoding and stored network configuration are unchanged.
@@ -220,7 +234,7 @@ not affect other networks.
 | `Pong { seq, payload }` | the echoed reply |
 | `State { records }` | a snapshot of signed records, merged into what the receiver holds |
 | `Peers { peers }` | unverified member address hints, requiring their own handshake |
-| `Reachable { links }` | sender's current direct data links, scoped by protocol |
+| `Reachable { links }` | sender's current direct data links, by kind of transport |
 | `Bye { reason }` | graceful goodbye; not a revocation of anything |
 `broadcast` is a per-network local opt-in, enabled by default and treated as
@@ -15,12 +15,27 @@ the default 1280-byte interface MTU also works on small QUIC paths.
 ## Graph and lifecycle
 Each authenticated control session advertises that peer's own direct data
-links, including the plugin protocol. Tables are separate for each network
+links. There is one table for each network, whatever protocols run over it: a
-and protocol; only members offering a matching enabled protocol version enter
+link is a way to reach a neighbour, and a protocol is an end-to-end tunnel to a
-the graph. A snapshot refreshes on the maintenance interval, expires after
+peer. The envelope names the protocol, so the two are independent. A neighbour
-90 seconds, and is withdrawn when the session ends. Link arrival, closure and
+reached over several kinds of transport (UDP, TCP, QUIC datagrams) is sent to
-changed announcements rebuild routes immediately. Unchanged announcements
+over the best open one, in the order the protocols were configured, and over
-refresh their age without republishing the forwarding snapshot.
+the next if that one fails or cannot carry the frame. Every authenticated
 member enters the graph; a member in the middle needs no protocol in common
 with either end, only a link to each. A snapshot refreshes on the maintenance
 interval, expires after 90 seconds, and is withdrawn when the session ends.
 Link arrival, closure and changed announcements rebuild routes immediately.
 Unchanged announcements refresh their age without republishing the forwarding
 snapshot.
 Which protocol a pair of members runs their tunnel in is settled without
 asking: the member with the lower endpoint id decides, by the order it
 announces its protocols in, among those both have enabled at the same wire
 version. Both ends read the same answer from what they already hold. The
 tunnel is created as soon as the peer is reachable by any path, and it does not
 change when a direct link comes or goes underneath it: as soon as a direct
 link comes up it is shorter than the path through others, and traffic moves to
 it with nothing to redo. The user sees only a lower round-trip time.
 Breadth-first search computes the shortest directed paths and every equal-cost
 first hop in sorted order. An actual direct link always wins. The first row is
@@ -30,14 +45,29 @@ loops if different nodes have not yet received the same topology update.
 The existing control plane still forms authenticated pairwise sessions.
 Routing provides multihop **data** paths among these members; it does not add
-control-plane flooding or carry control sessions through the data plane.
+control-plane flooding or carry control sessions through the data plane, so a
 member that has no control session with another has no data path to it either.
 Announcements are volatile, not durable membership or availability guarantees.
 ## What "online" means
 One definition, computed by the agent and reported identically by `tsng status`
 and the tray: a member is online when this agent can reach it by any path — a
 live authenticated control session (QUIC keep-alive every 5 seconds, so a
 vanished peer is noticed within about 15), or a data path to it through other
 members. It is never inferred from ping, which a firewall may block, or from a
 tunnel's last handshake, which says only when traffic last flowed.
 How it is reached is reported separately: `direct` over a named transport,
 `relayed` through a named member over N hops, or no data path yet while one is
 being set up. Being relayed is not being degraded in any way but distance, and
 is shown so it is not a surprise.
 ## Packet path
 The control loop binds next hops to transport handles in an immutable table
 and publishes it through `ArcSwap`. Readers do not acquire the topology mutex.
-The transit routine validates the fixed 74-byte envelope, looks up the source
+The transit routine validates the fixed 82-byte envelope, looks up the source
 and destination, selects a cached next hop, decrements one byte, and calls the
 transport's synchronous datagram send. It never searches the graph, formats an
 endpoint string, validates an Ed25519 key, decrypts, or touches TUN.
@@ -86,5 +116,5 @@ counts sends without storing frames. Results are CPU forwarding cost, **not**
 end-to-end network latency or VPN throughput; encryption, fragmentation,
 sockets, congestion and scheduling contribute separately.
-Wire compatibility: control ALPN `tsunagi/ctrl/4`, data ALPN `tsunagi/data/4`.
+Wire compatibility: control ALPN `tsunagi/ctrl/5`, data ALPN `tsunagi/data/5`.
 Upgrade every participant together; saved identities and network state persist.
@@ -43,7 +43,9 @@ document trusted merely because the neighbour who forwarded it signed it.
 - Duplicates do not change the result.
 - Two conflicting signed records at the same version from the same author need
  explicit handling; they are not resolved by luck.
- Neither arrival order nor system clocks decide a winner.
+- Neither arrival order nor the receiving replica's clock decides a winner. The
  one thing an author's own clock is used for is the time it first claimed a
  hostname, signed into its record (see below).
 - Compaction must not drop what is needed to stop revoked records being
  resurrected.
@@ -53,6 +55,22 @@ A hostname is a mutable binding to a persistent author, not an identity. A
 rename must be a signed record that revokes the specific old binding and
 announces the new one, ideally atomically in one record.
 A hostname belongs to the member that claimed it **first**. Each claim carries
 `hostname_since`, the author's own time in milliseconds when it first claimed
 that exact name, signed with the rest of the record and kept across every later
 version that keeps the name; renaming starts it afresh. Replicas order claims
 by `(hostname_since, author id)`, a pure function of the records, so every
 replica names the same holder whatever order the records arrived in and there
 is no race to lose. A member that arrives later with a name already held keeps
 working by address and keeps its claim, but is not found by that name on any
 member; it is told so in its status and in `tsng status`, and finds the name
 again if the holder renames or releases it.
 It is the author's word, so a member with a clock set far back can claim an
 older time. That can reorder who holds a *name* and nothing else — addresses,
 keys and everything the data plane trusts are decided without it — and anybody
 who knows the secret could take a name by other means anyway.
 Turning a computer off is not a revocation of its hostname and does not remove
 the participant. Revocations are not dropped merely because they are old, and
 no acknowledgement from offline peers is required to keep working.
@@ -26,8 +26,8 @@ system can hand us IP packets, and even that is behind a trait
 | `ManagedTunFactory` | `CAP_NET_ADMIN` | traffic actually reaches the OS |
 `ManagedTunFactory` creates the interface and configures it; see
-*Provisioning the interface* below and *Privileges* in
+*Provisioning the interface* below and Privileges in
-[../README.md](../README.md#privileges). With no capability the agent runs
+[../dist/linux/README.md](../dist/linux/README.md). With no capability the agent runs
 with `--no-tun`: everything but the last hop into the kernel still works.
 [boringtun]: https://docs.rs/boringtun
@@ -184,7 +184,7 @@ QUIC DATAGRAM frames themselves cannot fragment.
 Reassembly is bounded and expires incomplete packets; one lost fragment loses
 one packet, without blocking unrelated traffic. The logical data payload limit
 is 64 KiB, with the relay envelope subtracted before it reaches the plugin.
-The new framing uses data ALPN `tsunagi/data/4`; both ends and intermediate
+The new framing uses data ALPN `tsunagi/data/5`; both ends and intermediate
 peers need the updated binary. Network identities and saved state do not change.
 See [protocol.md](protocol.md#the-data-plane-protocol) for the wire format.
@@ -0,0 +1,27 @@
 {
  "nodes": {
    "nixpkgs": {
      "locked": {
        "lastModified": 1790769526,
        "narHash": "sha256-5AIVFLRcx8YAwbFTVPTs7YhwJH7JyHfAQsvXmpovcXI=",
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "c9fe7d12cd78d1adcd12dd15e24432dde5b155a0",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "ref": "nixpkgs-unstable",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "root": {
      "inputs": {
        "nixpkgs": "nixpkgs"
      }
    }
  },
  "root": "root",
  "version": 7
 }
@@ -0,0 +1,39 @@
 {
  description = "tsunagi: serverless mesh VPN. A network name and a secret are all it takes.";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
  outputs =
    { self, nixpkgs }:
    let
      systems = [
        "x86_64-linux"
        "aarch64-linux"
      ];
      forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f nixpkgs.legacyPackages.${system});
    in
    {
      packages = forAllSystems (pkgs: rec {
        tsunagi = pkgs.callPackage ./nix/package.nix { };
        tsunagi-cli = pkgs.callPackage ./nix/package.nix { withTray = false; };
        default = tsunagi;
      });
      overlays.default = final: _prev: {
        tsunagi = final.callPackage ./nix/package.nix { };
        tsunagi-cli = final.callPackage ./nix/package.nix { withTray = false; };
      };
      nixosModules.default = import ./nix/module.nix self;
      devShells = forAllSystems (pkgs: {
        default = pkgs.mkShell {
          inputsFrom = [ self.packages.${pkgs.stdenv.hostPlatform.system}.tsunagi ];
          packages = [
            pkgs.rustfmt
            pkgs.clippy
          ];
        };
      });
    };
 }
@@ -0,0 +1,132 @@
 flake:
 {
  config,
  lib,
  pkgs,
  ...
 }:
 let
  cfg = config.services.tsunagi;
  socket = "/run/tsunagi/agent.sock";
 in
 {
  options.services.tsunagi = {
    enable = lib.mkEnableOption "the tsunagi mesh agent";
    package = lib.mkOption {
      type = lib.types.package;
      default =
        if cfg.tray.enable then
          flake.packages.${pkgs.stdenv.hostPlatform.system}.tsunagi
        else
          flake.packages.${pkgs.stdenv.hostPlatform.system}.tsunagi-cli;
      defaultText = lib.literalExpression "tsunagi.packages.\${system}.tsunagi (tsunagi-cli without the tray)";
      description = "The tsunagi package to run.";
    };
    tray.enable = lib.mkEnableOption "the tray client, started with every graphical session";
    users = lib.mkOption {
      type = lib.types.listOf lib.types.str;
      default = [ ];
      example = [ "alice" ];
      description = ''
        Users allowed to talk to the agent without sudo (`tsng status`, `tsng join`,
        the tray). They are added to the `tsunagi` group.
      '';
    };
    protocols = lib.mkOption {
      type = lib.types.nullOr (lib.types.listOf lib.types.str);
      default = null;
      example = [
        "wg-quic"
        "wg"
      ];
      description = "Packet protocols for `tsng up --protocol`, best first. Null keeps the agent's default.";
    };
  };
  config = lib.mkIf cfg.enable {
    users.groups.tsunagi = { };
    users.users = {
      tsunagi = {
        isSystemUser = true;
        group = "tsunagi";
        home = "/var/lib/tsunagi";
        description = "tsunagi agent";
      };
    }
    // lib.genAttrs cfg.users (_: {
      extraGroups = [ "tsunagi" ];
    });
    systemd.services.tsunagi = {
      description = "tsunagi mesh agent";
      documentation = [ "https://github.com/house-of-vanity/tsunagi" ];
      wantedBy = [ "multi-user.target" ];
      wants = [ "network-online.target" ];
      after = [ "network-online.target" ];
      environment = {
        TSUNAGI_STATE_DIR = "/var/lib/tsunagi";
        TSUNAGI_CACHE_DIR = "/var/cache/tsunagi";
        TSUNAGI_CONTROL_SOCKET = socket;
        TSUNAGI_CONTROL_GROUP = "tsunagi";
      };
      serviceConfig = {
        ExecStart = lib.concatStringsSep " " (
          [ "${cfg.package}/bin/tsng up" ]
          ++ lib.optional (cfg.protocols != null) "--protocol ${lib.concatStringsSep "," cfg.protocols}"
        );
        User = "tsunagi";
        Group = "tsunagi";
        AmbientCapabilities = [
          "CAP_NET_ADMIN"
          "CAP_NET_BIND_SERVICE"
        ];
        CapabilityBoundingSet = [
          "CAP_NET_ADMIN"
          "CAP_NET_BIND_SERVICE"
        ];
        StateDirectory = "tsunagi";
        StateDirectoryMode = "0700";
        CacheDirectory = "tsunagi";
        RuntimeDirectory = "tsunagi";
        RuntimeDirectoryMode = "0755";
        Restart = "on-failure";
        RestartSec = 5;
      };
    };
    # Exit nodes: the agent looks for iptables only in fixed FHS directories.
    systemd.tmpfiles.rules = [
      "L+ /usr/bin/iptables - - - - ${pkgs.iptables}/bin/iptables"
    ];
    # Names of mesh members are served locally and handed to the system resolver.
    services.resolved.enable = lib.mkDefault true;
    # polkit decides by group, not capability, who may configure systemd-resolved.
    security.polkit.extraConfig = ''
      polkit.addRule(function(action, subject) {
          var allowed = [
              "org.freedesktop.resolve1.set-dns-servers",
              "org.freedesktop.resolve1.set-domains",
              "org.freedesktop.resolve1.set-default-route",
              "org.freedesktop.resolve1.revert"
          ];
          if (allowed.indexOf(action.id) >= 0 && subject.isInGroup("tsunagi")) {
              return polkit.Result.YES;
          }
      });
    '';
    environment.systemPackages = [ cfg.package ];
    environment.sessionVariables.TSUNAGI_CONTROL_SOCKET = socket;
    environment.etc."xdg/autostart/tsunagi-tray.desktop" = lib.mkIf cfg.tray.enable {
      source = "${cfg.package}/share/applications/tsunagi-tray.desktop";
    };
  };
 }
@@ -0,0 +1,94 @@
 {
  withTray ? true,
  lib,
  rustPlatform,
  pkg-config,
  copyDesktopItems,
  wrapGAppsHook3,
  gtk3,
  libayatana-appindicator,
  libglvnd,
  libxkbcommon,
  vulkan-loader,
  wayland,
  xdotool,
  libx11,
  libxcursor,
  libxi,
  libxrandr,
 }:
 let
  # The tray is an egui app that loads graphics, windowing and appindicator
  # libraries at run time, so they have to be put on its library path by hand.
  trayRuntimeLibs = [
    libayatana-appindicator
    libglvnd
    libxkbcommon
    vulkan-loader
    wayland
    libx11
    libxcursor
    libxi
    libxrandr
  ];
 in
 rustPlatform.buildRustPackage {
  pname = if withTray then "tsunagi" else "tsunagi-cli";
  version = "0.1.0";
  src = lib.cleanSource ../.;
  cargoLock.lockFile = ../Cargo.lock;
  cargoBuildFlags = [
    "--bin"
    "tsng"
  ]
  ++ lib.optionals withTray [
    "--bin"
    "tsunagi-tray"
  ];
  # The suite opens real sockets and interfaces; it runs in CI instead.
  doCheck = false;
  nativeBuildInputs = [ pkg-config ] ++ lib.optionals withTray [
    copyDesktopItems
    wrapGAppsHook3
  ];
  buildInputs = lib.optionals withTray [
    gtk3
    libayatana-appindicator
    xdotool
  ];
  dontWrapGApps = true;
  postInstall = ''
    install -Dm644 dist/linux/50-tsunagi-resolved.rules \
      "$out/share/doc/tsunagi/50-tsunagi-resolved.rules"
  ''
  + lib.optionalString withTray ''
    install -Dm644 dist/linux/tsunagi.svg \
      "$out/share/icons/hicolor/scalable/apps/tsunagi.svg"
    install -Dm644 dist/linux/tsunagi-tray.desktop \
      "$out/share/applications/tsunagi-tray.desktop"
  '';
  postFixup = lib.optionalString withTray ''
    wrapProgram "$out/bin/tsunagi-tray" \
      "''${gappsWrapperArgs[@]}" \
      --prefix LD_LIBRARY_PATH : ${lib.makeLibraryPath trayRuntimeLibs}:/run/opengl-driver/lib
  '';
  meta = {
    description = "Serverless mesh VPN: join a network with a name and a secret";
    homepage = "https://github.com/house-of-vanity/tsunagi";
    license = lib.licenses.wtfpl;
    mainProgram = "tsng";
    platforms = [
      "x86_64-linux"
      "aarch64-linux"
    ];
  };
 }
Author	SHA1	Message	Date
Aleksandr BogomiakovandClaude	97377e3703	Relay across any mix of links, one online answer for every client, first claim owns a hostname A tunnel no longer depends on the kind of link under it. The relay hub keeps one forwarding table per network instead of one per protocol, the envelope names the end-to-end protocol, and a neighbour reached over several kinds of transport is sent to over the best open one. A member in the middle needs no protocol in common with either end. Which protocol a pair runs is settled by the lower endpoint id's announced order, so both ends agree without asking. A direct link that comes up later is shorter than the path through others, so traffic moves to it with the same logical link. The agent now answers "is it online" once: a live control session or any data path. Members and peers report how they are reached (direct over a named transport, relayed through a named member, or no data path yet), and `tsng status` and the tray both read that instead of a session list and a WireGuard handshake respectively. A hostname belongs to whoever claimed it first. A claim carries the time its author first claimed the name, signed with the record, and replicas order by (time, id) so every member names the same holder. A later claimant keeps working by address, is not found by the name, and is told so in its status, `tsng status`, the tray and a HostnameTaken event. Wire change: control and data ALPN 5, control PROTOCOL_VERSION 5, envelope version 2, signing domain v3, state schema 7, local control protocol 18. Upgrade every member together. Co-Authored-By: Claude <noreply@anthropic.com>	2026-10-05 11:39:33 +01:00
Aleksandr BogomiakovandClaude	c184ef0914	Rewrite README as a short project front page; add a Nix flake The README is now a landing page: the idea (a mesh VPN with no server, joined with a network name and a secret), install and platform tables, exit nodes, and a short build section. Implementation detail stays in docs/. The flake builds tsng and the tray from source and ships a NixOS module (services.tsunagi). Dist install notes now show the real `join -n -s` syntax, and two links that pointed into the old README are redirected. Co-Authored-By: Claude <noreply@anthropic.com>	2026-10-03 17:47:44 +01:00
Aleksandr BogomiakovandClaude	e6ec62bcba	Windows installer: find the Wintun licence at the archive root Co-Authored-By: Claude <noreply@anthropic.com>	2026-10-03 14:38:16 +01:00
UltradesuandClaude Sonnet 5.5	21328bf818	Windows installer: agent service, Wintun, tray and PATH in one setup program Running on Windows needed wintun.dll by hand, an elevated console and a manually started agent; without the DLL the agent silently ran with no interface (and so no exit node or routing at all). `tsng service` (hidden) runs the same `up` under the Windows service manager: src/winservice.rs registers with the SCM through the windows-service crate, reports Running/StopPending/Stopped, and maps a stop request onto the existing shutdown path (stop_signal). The only unsafe is the extern "system" entry point that define_windows_service! expands to, allowed in that module alone. A service has no console, so it logs to %ProgramData%\Tsunagi\logs\tsng.log (one older file kept past 8 MiB). Its state defaults to %ProgramData%\Tsunagi\{data,cache} and its control socket to the system-wide default every client already uses, so an unelevated `tsng status` and the tray find it. dist/windows/tsunagi.iss (Inno Setup) installs tsng.exe, the tray and Wintun 0.14.1 (pinned by SHA-256 in CI, licence text shipped), registers the "Tsunagi" service as LocalSystem with auto start and restart-on-failure, locks the data/cache/log directories to SYSTEM and Administrators by SID, and by tasks adds tsng to the PATH, an inbound firewall rule for the agent and a tray autostart. It stops the service, the tray and any hand-started agent before replacing files; uninstall removes the service, rule and PATH entry and asks before deleting the data (never on a silent uninstall). The release workflows (GitHub and Gitea) build it on the Windows job and publish tsunagi-setup-<version>-x86_64.exe. .cargo/config.toml links the MSVC C runtime statically so nothing needs a Visual C++ redistributable on a clean machine. Compatibility: an agent run by hand keeps its data in the user profile; the service has its own, so networks must be joined again once. Checks: cargo fmt --check, clippy -D warnings, cargo test --workspace passed on macOS (one unrelated wireguard restart test failed once under load and passed on three reruns). winservice.rs was cargo-checked for x86_64-pc-windows-msvc in a scratch crate; the whole tsunagi crate cannot be built for Windows from macOS (aws-lc needs the Windows SDK). The .iss script and the CI step were NOT compiled or run: no Inno Setup or Windows here. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>	2026-10-03 14:16:05 +01:00
UltradesuandClaude Sonnet 5.5	7c176e0b74	Exit nodes on Windows: use a member as exit node via destination bypass routes Choosing an exit node on Windows did nothing: the using side was refused, because Windows has no per-user routing and the agent's own traffic would loop into the tunnel it carries. Now the overlay interface gets 0.0.0.0/1 and 128.0.0.0/1 routes (and ::/1, 8000::/1 as an IPv6 blackhole), and every underlay address the agent talks to directly gets a /32 route through the physical default gateway, marked with route metric 7373 so stale ones are found and removed. Bypass routes are written before the halves and removed after them; a failed step removes what was written. All routes are in the active store. The addresses come from the runtime: each network runtime hands the interface the verified direct IPv4 paths of its control connections and the relay hosts in use (Interface::set_underlay, refreshed every dial round); relay names are resolved when the rules are applied. ExitHostRules::needs_bypass (true only on Windows) gates it, so Linux and macOS plans never carry or re-apply on path changes. ExitHostPlan gains `bypass` and `bypass_hosts`; networks leaving forget their entry. Limits: a path that appears after the last apply goes through the tunnel until the next apply; data-plane-only addresses are not collected separately (the data links ride the same iroh paths). Nothing was run on Windows. Checks: cargo fmt --check, cargo clippy --locked --workspace --all-targets -D warnings and cargo test --locked --workspace --all-targets passed on macOS; scripts and ordering are unit-tested with an injected runner and resolver, and Interface::set_underlay with the mock host. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>	2026-10-03 13:43:01 +01:00
UltradesuandClaude Sonnet 5.5	637915600e	Exit nodes on Windows: offer through WinNAT; using is refused Windows reported exit nodes as unsupported. The offering side now creates one WinNAT object per offered range (New-NetNat, named tsunagi-exit:<if>:<range>), prunes this interface's stale ones, reads (never writes) IPv4 forwarding of the overlay interface, and removes everything on clear. All values are single-quoted PowerShell literals. The using side is deliberately not implemented: Windows has no per-user or per-process routing, so the agent's own QUIC/relay traffic cannot be kept out of a default route through the overlay and would loop. It reports that reason and installs no route. The CLI/tray forwarding hint and the kernel-forwarding warning name the Windows command; docs/exit-node.md has a Windows section. Checks: cargo fmt --check, cargo clippy --locked --workspace --all-targets -D warnings and cargo test --locked --workspace --all-targets passed on macOS. The script builders and plan logic are unit-tested with an injected runner on every platform; nothing was run against a real Windows host or cross-compiled (no Windows target installed). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>	2026-10-03 12:33:22 +01:00
UltradesuandClaude Sonnet 5.5	47c5d38d0c	Exit nodes on macOS: pf anchor for offering, routes plus route-to for using The exit-node host rules existed only for Linux (iptables + policy routing); macOS reported "only supported on Linux yet". Offering: a pf anchor com.apple/tsunagi-exit-<if> (evaluated by the stock pf.conf) holds a nat rule per offered range and a pass-in rule. pf is enabled if needed and the pfctl -E reference is released when the rules are removed. net.inet.ip.forwarding is only read, never written, as on Linux. Using: macOS has no uidrange policy routing and the agent is root, so the overlay gets 0.0.0.0/1 and 128.0.0.0/1 routes and a pf route-to rule (matched by the agent's uid) sends the agent's own QUIC/relay traffic back out the physical interface, translated to its address. The rules are loaded before the routes, so a failed load never sends traffic into a tunnel it cannot exempt the agent from. IPv6 is blocked with rejecting ::/1 and 8000::/1 routes. ExitHostPlan gains `overlay` (every network range) for the route-to exemption; Linux ignores it. Known limits: the egress is the default route's interface at apply time; other root processes also bypass the tunnel. Forwarding hints in the CLI and tray name the macOS sysctl. The rule text and command order are unit-tested with a scripted host; nothing was run against a live pf/route (needs root), so the route-to + nat second-pass behaviour is unverified. Checks: cargo fmt --check, cargo clippy --locked --workspace --all-targets -D warnings and cargo test --locked --workspace --all-targets, run on macOS only. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>	2026-10-03 12:29:40 +01:00