Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Aleksandr Bogomiakov
|
781e359775 | ||
|
Aleksandr Bogomiakov
|
d5e02ec256 | ||
 Aleksandr BogomiakovandClaude
|
97377e3703 | ||
|
Aleksandr BogomiakovandClaude
|
c184ef0914 |
No files matched your search
@@ -1 +1,2 @@
|
||||
/target
|
||||
/.claude
|
||||
@@ -8,7 +8,7 @@ before touching the code. Do not restate the full design here; follow the links.
|
||||
A proof-of-concept agent library for small private mesh networks. A user
|
||||
supplies a network name and one shared secret; agents derive the same network
|
||||
space independently, find each other, prove membership and exchange control
|
||||
messages. Scope and non-scope are in [README.md](README.md).
|
||||
messages. What it does and does not do is in [docs/architecture.md](docs/architecture.md) and [docs/threat-model.md](docs/threat-model.md).
|
||||
|
||||
There is no central server, no owner, no registration and no majority vote.
|
||||
Design accordingly: a majority is not a root of trust.
|
||||
@@ -59,8 +59,10 @@ Keep these separate. Crossing them is the main thing to review for.
|
||||
other peers goes link in, link out: it is never written to the middle's
|
||||
interface or decrypted there. Multihop routing has a bounded hop limit;
|
||||
equal-cost next hops are chosen per flow. Reachability is first-hand and
|
||||
volatile: each authenticated member advertises its own protocol-specific
|
||||
links. Build routing tables on topology changes, never per packet. Transit
|
||||
volatile: each authenticated member advertises its own direct links, of
|
||||
whatever kind. A link is a way to reach a neighbour and a protocol is an
|
||||
end-to-end tunnel to a peer; the two meet only in the envelope, so a
|
||||
tunnel never depends on which transports carry it. Build routing tables on topology changes, never per packet. Transit
|
||||
must not acquire a routing mutex or wait for a protocol/TUN reader.
|
||||
- **Broadcast fanout belongs at local IP ingress.** Participation is local to
|
||||
each network, enabled by default, persisted and advertised to authenticated
|
||||
|
||||
@@ -1,766 +1,183 @@
|
||||
# tsunagi
|
||||
|
||||
A proof-of-concept Rust library for **small private mesh networks** — a handful
|
||||
of friends, home machines, a few servers. Units to dozens of participants, not
|
||||
thousands.
|
||||
|
||||
The end user configures exactly two things:
|
||||
|
||||
```text
|
||||
network_name
|
||||
secret # one shared secret; "password" and "secret" mean the same value
|
||||
```
|
||||
|
||||
From those, every agent independently derives the same network space. There is
|
||||
no central server, no network owner with special powers, no registration and no
|
||||
majority vote. Anyone who knows the parameters can join; nobody has to trust
|
||||
anybody else.
|
||||
|
||||
## What this proof of concept actually does
|
||||
|
||||
A working library with **real iroh connections** and integration tests:
|
||||
|
||||
- persistent device identity stored in SQLite, stable across restarts;
|
||||
- several independent networks at once in one agent;
|
||||
- deterministic network identity derived from name + secret;
|
||||
- candidates supplied by a replaceable discovery component;
|
||||
- automatic first contact through the public Mainline DHT (BEP44);
|
||||
- real iroh connections plus an explicit mutual proof of network membership;
|
||||
- a small versioned control protocol: handshake, hostname/capability
|
||||
announcement, ping/pong;
|
||||
- automatic reconnect with bounded exponential backoff and jitter;
|
||||
- status snapshots, an event stream and honest diagnostics;
|
||||
- configuration restored after a restart;
|
||||
- correct behaviour when the disposable cache is missing or corrupt;
|
||||
- a **WireGuard data plane**, in userspace: its own key per network, an IPv6
|
||||
overlay with deterministically derived addresses and optional IPv4, real
|
||||
tunnels carried over iroh, and address ownership enforced rather than
|
||||
believed;
|
||||
- a **command line agent**, `tsng`, with a local control socket.
|
||||
|
||||
### What it deliberately does **not** do
|
||||
|
||||
Not implemented, and not pretended to be: DNS, routing through
|
||||
intermediate participants, a full CRDT, dynamically loaded plugins, a system
|
||||
service, a complete CLI, or a local control socket. Snapshot synchronisation
|
||||
and signed revocations are designed for but not implemented — see
|
||||
[docs/sync-model.md](docs/sync-model.md). The WireGuard plugin's own limits,
|
||||
including that its system backend is Linux-only, are in
|
||||
[docs/wireguard.md](docs/wireguard.md#limits-and-future-work).
|
||||
|
||||
**Control and data are separated logically, not physically.** Both ride on
|
||||
iroh, on different ALPNs and different connections, so the data plane inherits
|
||||
iroh's hole punching and relay fallback instead of reimplementing them — while
|
||||
the control protocol still knows nothing about packets and can keep a different
|
||||
transport underneath it later. Filtering user traffic remains the operating
|
||||
system's and the user's responsibility, not this library's.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Rust 1.91 or newer (iroh 1.2 requires it), edition 2024. Pinned dependencies
|
||||
in `Cargo.lock`.
|
||||
- No internet, no public DHT, no public relay, no administrator rights and no changes
|
||||
to OS network settings are needed to build or test.
|
||||
- WireGuard runs in userspace (boringtun): **no kernel module and no `wg`
|
||||
tool**. Only creating a real network interface needs `CAP_NET_ADMIN`, and
|
||||
`--no-tun` skips even that.
|
||||
|
||||
## Trying it on two machines
|
||||
|
||||
On the first machine:
|
||||
|
||||
```bash
|
||||
cargo build --release
|
||||
./target/release/tsng status # this device, the agent, and this host
|
||||
./target/release/tsng network # the networks this device belongs to
|
||||
```
|
||||
|
||||
Two commands do the work, and they are separate on purpose: `up` runs the
|
||||
agent — one per device, for as long as you want it — and `join` decides
|
||||
what it belongs to, at any time, while it runs.
|
||||
|
||||
```bash
|
||||
# Terminal one: the agent. It prints its endpoint id and then serves.
|
||||
./target/release/tsng up
|
||||
|
||||
# Terminal two: make a network. With no --secret it invents one and prints
|
||||
# it, along with the line to send the other machine.
|
||||
./target/release/tsng join --network lab
|
||||
```
|
||||
|
||||
On the second machine, start its agent and join the same network with the
|
||||
secret that was printed:
|
||||
|
||||
```bash
|
||||
./target/release/tsng up
|
||||
./target/release/tsng join --network lab --secret tsn1...
|
||||
```
|
||||
|
||||
**Name and secret are enough for the first meeting.** Mainline DHT discovery
|
||||
is enabled by default. Each active network publishes this device's endpoint
|
||||
and searches until the first authenticated connection. While connected it
|
||||
only republishes; after 60 seconds without any authenticated connection it
|
||||
searches again. Publication is approximately every five minutes, and records
|
||||
older than 15 minutes are ignored. DHT storage nodes may retain them longer.
|
||||
|
||||
Use `--no-dht` to disable DHT, or `--peer <endpoint-id>` for optional manual
|
||||
bootstrap. `--reach local` disables public DHT regardless of the DHT flag.
|
||||
Endpoint records are public, signed, and contain no network secret. Discovery
|
||||
is not authentication: every connection must still prove network membership.
|
||||
Details and failure behavior are in [docs/mainline-dht.md](docs/mainline-dht.md).
|
||||
|
||||
**One introduction is enough for the whole network.** Members tell each
|
||||
other about the members they know, and every author of a signed record is
|
||||
somebody to try, so a device pointed at one member ends up talking to all
|
||||
of them rather than to the one that happened to be on its command line.
|
||||
What travels is a candidate — an address somebody has seen — and it is
|
||||
authenticated by the handshake like any other; being introduced grants
|
||||
nothing. It is deliberately the same shape a lookup in a distributed hash
|
||||
table would return, so that is a source to add beside this one rather than
|
||||
a redesign.
|
||||
|
||||
An agent with nobody to contact says so in `status` rather than sitting
|
||||
there looking patient.
|
||||
|
||||
Within a few seconds both print something like:
|
||||
|
||||
```text
|
||||
+ peer b47c958462 connected over direct rtt=Some(4.5ms)
|
||||
+ data link to b47c958462 for wg-quic: direct via 192.0.2.7:41234, datagram 1382
|
||||
|
||||
--- status ---
|
||||
control: 1 peer(s), 0 dial failure(s), 0 handshake failure(s)
|
||||
wg-quic: tsun0 on 10.13.37.69/24 mtu 1280, 1/1 tunnel(s) established
|
||||
4jO4kx9Z 10.13.37.237 handshake 3s ago tx=0 rx=0 dropped=0 path=direct via 192.0.2.7:41234
|
||||
```
|
||||
|
||||
`1/1 tunnel(s) established` means a real WireGuard handshake completed.
|
||||
|
||||
## Checking that it works
|
||||
|
||||
From another shell on either machine:
|
||||
|
||||
```bash
|
||||
tsng status
|
||||
```
|
||||
|
||||
```text
|
||||
endpoint 7d76ccbbc21bf30767e14422c0494740a2cecc02aa9f82d5b8d57bdae350e7fc
|
||||
hostname tsunagi-7d76ccbbc2
|
||||
bound 0.0.0.0:41641
|
||||
|
||||
network lab (z2o4qwrvnj3zb6st2aoqg4abf342j662q2ujttqrsmz22argk2ba) active
|
||||
peer b345d5271b tsunagi-b345d5271b Direct rtt 24ms
|
||||
overlay tsunz2o4qwrvnj3 fd09:…:c1c6/64 and 100.110.49.177 mtu 1280 1/1 tunnel(s) up
|
||||
SDsEb/WF fd09:…:c4b / 100.65.243.53 handshake 4s ago tx 0 rx 0 Direct via Ip(…)
|
||||
```
|
||||
|
||||
`1/1 tunnel(s) up` and a recent handshake mean the tunnel is live. Then send
|
||||
real traffic to the peer's overlay address:
|
||||
|
||||
```bash
|
||||
ping6 fd09:…:c4b # or
|
||||
ping 100.65.243.53
|
||||
```
|
||||
|
||||
`tx` and `rx` in the status should start moving.
|
||||
|
||||
**Addresses are allocated and then remembered.** The default range is
|
||||
`10.13.37.0/24`; the first member to join settles it and later members adopt
|
||||
what they find, so `--ipv4-range` only matters for whoever starts the network:
|
||||
|
||||
```bash
|
||||
tsng up --ipv4-range 10.44.0.0/16
|
||||
tsng up --ipv4-range none # no data plane at all
|
||||
```
|
||||
|
||||
One agent has one interface, so two of its networks cannot both use that
|
||||
range. The second takes the range **derived from its own network id**: not
|
||||
picked locally — every member derives the same one from something they all
|
||||
already have — so it is an agreement rather than a guess, and a device in
|
||||
several networks gets an address in each.
|
||||
|
||||
An address is claimed with a record signed by that member's persistent device
|
||||
key, stored, and merged between every replica. A member that disappears for a
|
||||
month comes back to the same address, because the claim outlived the session.
|
||||
No vote is involved — see
|
||||
[docs/wireguard.md](docs/wireguard.md#ipv4-allocated-signed-and-kept) and
|
||||
[docs/sync-model.md](docs/sync-model.md).
|
||||
|
||||
Because the address is allocated at run time rather than derived, it is not
|
||||
known until the agent has started and agreed with its peers. The agent then
|
||||
assigns it to the interface itself.
|
||||
|
||||
## Levels
|
||||
|
||||
The command line is split the way the design is. A bare `tsng up` needs
|
||||
only a network name and a secret; everything else sits under the level it
|
||||
belongs to, which `--help` shows as two sections:
|
||||
|
||||
* **System** — what the agent itself does: how it reaches peers (`--reach`),
|
||||
the one overlay interface it owns (`--interface`, `--mtu`, `--no-tun`), the
|
||||
address range (`--ipv4-range`), and the local resolver (`--no-dns` to disable).
|
||||
* **Transport** — which protocols carry packets (`--protocol`, a list) and
|
||||
their own settings (`-o key=value`, or `-o protocol:key=value`).
|
||||
|
||||
```bash
|
||||
tsng protocols # what this build can carry packets with
|
||||
```
|
||||
|
||||
```text
|
||||
wg-quic (wire version 4)
|
||||
what WireGuard's cryptography carried in iroh's QUIC datagrams
|
||||
-o keepalive=SECONDS keeps a tunnel and its link warm through a NAT
|
||||
-o mtu=BYTES largest packet a tunnel will carry, at least 576
|
||||
```
|
||||
|
||||
Each protocol declares its own settings, so the agent can list them without
|
||||
knowing anything about the protocol, and a setting no selected protocol takes
|
||||
is refused rather than ignored. `--protocol none` runs the control plane by
|
||||
itself.
|
||||
|
||||
A pair of peers uses a protocol they both have **at the same wire version**.
|
||||
That is not the software version: two peers on different builds carry traffic
|
||||
for each other for as long as the bytes between them have not changed. A peer
|
||||
with nothing in common keeps its control plane — messages and signed state
|
||||
still flow — and simply has no data plane, which `tsng status` shows as a
|
||||
session with no agreed protocol.
|
||||
|
||||
## Names
|
||||
|
||||
DNS is enabled by default and serves a local zone for **every network this device is in**,
|
||||
each named after the network, so members can be reached by name instead of
|
||||
by address:
|
||||
|
||||
```bash
|
||||
tsng up # DNS is already enabled
|
||||
dig @127.0.0.1 -p 5354 music.lab
|
||||
```
|
||||
|
||||
An explicit choice is remembered across restarts. `--no-dns` or `tsng dns off`
|
||||
disables it; `--dns` or `tsng dns on` enables it again. Existing saved opt-outs
|
||||
remain respected. The `dns on` and `dns off` commands also take effect immediately
|
||||
on a running agent. `--dns` and `--no-dns` cannot be combined.
|
||||
|
||||
The zone of a network is its name — `--dns-zone` is gone, because with
|
||||
several networks there is no single zone to name. A network name may contain
|
||||
dots, so `--network lab.internal` is how you get `music.lab.internal`.
|
||||
|
||||
Names come from signed state, which is the point: **a member that is
|
||||
switched off still resolves**, because its claim outlived the session.
|
||||
|
||||
The answers are IPv4 addresses, because that is what the overlay is. The
|
||||
*questions* are taken on `127.0.0.1` and `[::1]`, over UDP and TCP, so a
|
||||
resolver reaches it over whichever family it uses; both are published to the
|
||||
system resolver together. Loopback and nothing else: the zones are a view
|
||||
for the host running the agent, and binding an overlay address would put
|
||||
them in front of the whole mesh — an agent in two networks would then answer
|
||||
one network's questions about the other's names.
|
||||
|
||||
The zone is the network name, and it is yours to choose. A name that
|
||||
shadows a real public domain is reported and then used: a network called
|
||||
`ru` warns that every public `.ru` name becomes unreachable from this host,
|
||||
and then does it. Anything that collides with nothing — which is most names
|
||||
— is said nothing about, because a warning that fires on every private name
|
||||
anybody picks is how people learn to ignore warnings.
|
||||
|
||||
On Linux the agent tells systemd-resolved to send questions for that suffix
|
||||
here, over D-Bus, scoped to the overlay interface and as a *routing* domain
|
||||
so it never becomes the resolver for anything else. resolved drops the whole
|
||||
setting when the interface goes, and the interface goes with the agent.
|
||||
|
||||
Only an interface the agent created, though. Under `--no-tun` there is no
|
||||
host interface at all, and the agent says so and serves the zone on
|
||||
loopback rather than configuring whatever else on the host happens to share
|
||||
the name.
|
||||
|
||||
That last step needs permission that `CAP_NET_ADMIN` does not give:
|
||||
systemd-resolved asks polkit, and polkit decides by **user**, not by
|
||||
capability, so there is no way for the agent to arrange it from inside. On
|
||||
a desktop the refusal reads `Interactive authentication required`.
|
||||
|
||||
Running as a system service is enough. Otherwise the agent prints the rule
|
||||
that grants it — the four actions it calls and nothing else — ready to
|
||||
paste:
|
||||
|
||||
```bash
|
||||
sudo tee /etc/polkit-1/rules.d/50-tsunagi-resolved.rules > /dev/null <<'RULE'
|
||||
polkit.addRule(function(action, subject) {
|
||||
var allowed = [
|
||||
"org.freedesktop.resolve1.set-dns-servers",
|
||||
"org.freedesktop.resolve1.set-domains",
|
||||
"org.freedesktop.resolve1.set-default-route",
|
||||
"org.freedesktop.resolve1.revert"
|
||||
];
|
||||
if (allowed.indexOf(action.id) >= 0 && subject.user == "YOUR-USER") {
|
||||
return polkit.Result.YES;
|
||||
}
|
||||
});
|
||||
RULE
|
||||
```
|
||||
|
||||
**Without it the server still runs** — `tsng status` prints where it is
|
||||
listening and the exact `dig` line — so the automatic part is missing, not
|
||||
the feature. The refusal is said once rather than on every pass, and retried
|
||||
slowly, because nothing but a person will change it.
|
||||
|
||||
The server is authoritative for its zone and nothing else. No recursion, no
|
||||
forwarding, no cache: pointing a resolver at it can never make it a route to
|
||||
the outside.
|
||||
|
||||
## Privileges
|
||||
|
||||
On Linux the agent **manages its own overlay interface**. It creates the TUN
|
||||
interface, sets the MTU, brings it up and assigns both overlay addresses, all
|
||||
over netlink in process — no `ip` invocation, no shell, nothing that a
|
||||
remote peer could influence.
|
||||
|
||||
That needs `CAP_NET_ADMIN`, granted once:
|
||||
|
||||
```bash
|
||||
sudo setcap cap_net_admin+p /usr/local/bin/tsng
|
||||
```
|
||||
|
||||
`+p` rather than `+ep`: the capability is then *permitted* but not
|
||||
*effective*, and the agent raises it only around the handful of netlink calls
|
||||
that need it — a few milliseconds at startup, and again if its address
|
||||
allocation changes. Everything else, including every byte from the network,
|
||||
is handled with it lowered. `+ep` works too; the agent lowers it on the way
|
||||
in.
|
||||
|
||||
`tsng status` says which of these applies on the host it runs on, along
|
||||
with what the agent is doing. It grades each finding: **ok** for what works,
|
||||
**warn** for what the agent runs without and you can fix from the line it
|
||||
prints, **FAIL** for what it cannot work around. The words carry the grade as
|
||||
well as the colour, so the report reads the same piped to a file or on a
|
||||
terminal without colour, and it honours `NO_COLOR`.
|
||||
|
||||
Members are listed online first, then the ones that are away. A member that
|
||||
is away is reported plainly rather than flagged: in a mesh of laptops it is
|
||||
the ordinary condition, not a fault. The signed state is what makes that
|
||||
sayable — it remembers who belongs while they are gone, so the report can say
|
||||
"offline, 10.13.37.99 still reserved for it" instead of leaving a
|
||||
dial-failure counter to imply it. Counters are history and are never graded:
|
||||
a peer that left and came back should not leave the report looking broken.
|
||||
|
||||
The other two commands split along the line the system itself draws. `id` is
|
||||
this **device**: the key it signs with and the name it answers to. `network`
|
||||
is what it **belongs to**: which networks, their secrets, joining and
|
||||
leaving. A device outlives every network it is in, and a network outlives
|
||||
any device in it, so a command that mixed them had to be read twice.
|
||||
|
||||
Every item takes the same shape, so there is nothing to remember: name it to
|
||||
see it, name it with a value to change it.
|
||||
|
||||
```
|
||||
tsng id everything about this device
|
||||
tsng id hostname the name it answers to
|
||||
tsng id hostname mango change it
|
||||
tsng id key the key it signs with
|
||||
tsng id key rotate replace that key
|
||||
|
||||
tsng network the networks this device belongs to
|
||||
tsng network join -n lab -s tsn1… join one; adds it to a running agent
|
||||
tsng network join -n lab resume one this device has, or make it on the spot
|
||||
tsng network stop lab stop serving it, keeping everything
|
||||
tsng network start lab serve it again, from where it left off
|
||||
tsng network leave lab give up the address and name, then forget it
|
||||
tsng network secret the secret of each joined network
|
||||
tsng network secret lab just that one, for copying
|
||||
tsng network secret generate a fresh secret for a network that does not exist yet
|
||||
|
||||
tsng dns whether the local resolver is serving, and what
|
||||
tsng dns on start it, now and after every restart
|
||||
tsng dns off stop it, now and after every restart
|
||||
```
|
||||
|
||||
**A network without a secret makes one.** `tsng join --network lab`
|
||||
resolves a bare name in the obvious way: if this device is already in
|
||||
exactly one network called `lab`, that one — so the name alone resumes what
|
||||
you have; if it is in none, a fresh random secret, printed in full with the
|
||||
line to send the other machine:
|
||||
|
||||
```
|
||||
joined `lab` (k2on43wadbi5x267vp6z3ogkm7nbjedfdoxyauxhtxhwqrprylba)
|
||||
<p align="center">
|
||||
<img src="dist/linux/tsunagi.svg" width="96" alt="">
|
||||
</p>
|
||||
|
||||
<h1 align="center">tsunagi</h1>
|
||||
|
||||
<p align="center">
|
||||
<b>A mesh VPN with no server.</b><br>
|
||||
Pick a network name, share a secret, and your machines are on one private network.
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/house-of-vanity/tsunagi/releases/latest"><img alt="release" src="https://img.shields.io/github/v/release/house-of-vanity/tsunagi?include_prereleases"></a>
|
||||
<a href="https://github.com/house-of-vanity/tsunagi/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/house-of-vanity/tsunagi/actions/workflows/ci.yml/badge.svg"></a>
|
||||
<a href="LICENSE"><img alt="license" src="https://img.shields.io/badge/license-WTFPL-blue"></a>
|
||||
</p>
|
||||
|
||||
```console
|
||||
laptop$ tsng join -n home
|
||||
joined `home`
|
||||
secret tsn1u7c…
|
||||
|
||||
Run this on the other machine:
|
||||
server$ tsng join -n home -s tsn1u7c…
|
||||
server$ tsng id hostname server
|
||||
|
||||
tsng join --network lab --secret tsn1u7c…
|
||||
|
||||
Its agent has to be running. If it is not:
|
||||
|
||||
tsng up --peer 91e83a6e2b7a…
|
||||
laptop$ ssh server.home
|
||||
```
|
||||
|
||||
Two networks of one name is a thing that happens — a mistyped secret makes
|
||||
one — so joining says whether the network was already here, and warns when
|
||||
another configured network answers to the same name. A name is a label and
|
||||
the id is the identity.
|
||||
No account, no coordination server, no admin console, no keys to approve.
|
||||
Anyone who knows the **name** and the **secret** is in. Nobody else can find
|
||||
the network, let alone enter it.
|
||||
|
||||
That is the ad-hoc case: one person makes a network and sends the command
|
||||
round. The secret is printed *only* when the agent invented it — there is
|
||||
nowhere else to read it from — and never when it was supplied, because then
|
||||
it is already yours. Two networks of one name and no secret is the one case
|
||||
with no answer, and it says so instead of choosing.
|
||||
## Why
|
||||
|
||||
A secret is printed by `network secret` and nowhere else — not by `id`, not
|
||||
by `status`, not in a log, a `Debug` rendering or anything sent to a peer.
|
||||
Asking for it is deliberate, because these reports get pasted into chats.
|
||||
Tailscale, Headscale and ZeroTier are great, but each needs a control server:
|
||||
theirs, or one you have to run and keep alive. tsunagi doesn't. Machines find
|
||||
each other through the public BitTorrent DHT, connect directly (hole punching,
|
||||
with relay fallback), and keep the network state among themselves.
|
||||
|
||||
**`up` takes no network at all** if you would rather decide later: it
|
||||
brings up the agent and whatever it is already configured for, and waits.
|
||||
That is the shape of a daemon in one terminal and `tsng network join`
|
||||
in another.
|
||||
| | Tailscale | Headscale | ZeroTier | **tsunagi** |
|
||||
| ---------------------- | ------------------ | ---------------- | ------------------- | ------------------- |
|
||||
| Control server | hosted by vendor | you run it | hosted or self-run | **none** |
|
||||
| Account / sign-in | yes | yes | yes | **no** |
|
||||
| To join a network | sign in, get approved | pre-auth key | network ID, approval | **name + secret** |
|
||||
| Direct connections | yes | yes | yes | yes |
|
||||
| Exit nodes | yes | yes | via routes | yes |
|
||||
|
||||
`network join` is also the answer to a question `up` cannot: a state
|
||||
directory belongs to one live agent, so a second `tsng up` cannot add a
|
||||
network to the one already running. This adds it over the control socket and
|
||||
it starts at once. With no agent running it is written to the configuration
|
||||
and starts with the next `up`.
|
||||
## Features
|
||||
|
||||
The name is part of the signed state, so changing it revokes the previous
|
||||
one: there is one record per author, a new version replaces the whole claim,
|
||||
and no replica can keep the old name standing. Changing it while the agent
|
||||
runs goes through the agent, which republishes and tells its peers straight
|
||||
away.
|
||||
- **Two things to know:** a network name and a secret. A secret is generated for you when you make a network.
|
||||
- **Direct, encrypted links** between members (WireGuard cryptography), with relay fallback behind strict NATs.
|
||||
- **Names for machines:** `server.home` just works, even while `server` is switched off. A name belongs to whoever claimed it first; a later device with the same name keeps working by address and is told so.
|
||||
- **Exit nodes:** send all your internet traffic through any member that offers one.
|
||||
- **Multi-hop:** members that can't reach each other directly are routed through a member that can, over whatever kind of link each hop has. A member reached that way is online, and shown as relayed; traffic moves to a direct link by itself the moment one comes up.
|
||||
- **LAN games:** UDP broadcasts are relayed, so "LAN" lobbies show up across the mesh.
|
||||
- **Several networks at once** on one device.
|
||||
- **Tray app** for Windows, macOS and Linux; a command line for everything.
|
||||
|
||||
Replacing the signing key makes this device a different member, and it loses
|
||||
the address and name the old key held — nothing can sign on a retired key's
|
||||
behalf, and by design there is no authority that could overrule an author. So
|
||||
the outgoing key signs a release for every network on its way out, which
|
||||
frees them for whoever wants them next, and the whole thing commits at once.
|
||||
## Install
|
||||
|
||||
`status` and `id` both prefer a running agent, which is live and
|
||||
authoritative, and fall back to reading the state store when there is none.
|
||||
Reading takes no directory lock, so neither has to wait for the agent it is
|
||||
asking about — nor does either need one to be running.
|
||||
Grab a build from the [latest release](https://github.com/house-of-vanity/tsunagi/releases/latest).
|
||||
|
||||
### It cleans up after itself
|
||||
| Platform | How |
|
||||
| --------------------- | -------------------------------------------------------------------------------- |
|
||||
| **Windows** | Run `tsunagi-setup-<version>-x86_64.exe`. Installs the service, the tray and `tsng`. |
|
||||
| **macOS** | `brew tap house-of-vanity/tap && brew install --cask tsunagi-gui`<br>CLI only: `brew install tsunagi` |
|
||||
| **Debian / Ubuntu** | `sudo apt install ./tsunagi-gui_<version>_amd64.deb` (`tsunagi_…` for no tray) |
|
||||
| **Arch** | `sudo pacman -U tsunagi-gui-<version>-1-x86_64.pkg.tar.zst` |
|
||||
| **NixOS** | [Flake and module](#nixos) below |
|
||||
| **Any other Linux** | Unpack the tarball, then `sudo ./install.sh && sudo systemctl enable --now tsunagi` |
|
||||
|
||||
The interface is tied to an open file descriptor and is deliberately **not**
|
||||
made persistent, so the kernel removes it when the agent exits — on a clean
|
||||
shutdown, on a panic, on `SIGKILL`, on power loss alike. Keeping it is what
|
||||
would take an action; removing it is the default.
|
||||
Linux packages run the agent as an unprivileged service user. Add yourself to
|
||||
the `tsunagi` group (`sudo usermod -aG tsunagi $USER`, then log in again) to
|
||||
use `tsng` and the tray without `sudo`.
|
||||
|
||||
If something is left behind anyway — an interface made by an older version's
|
||||
manual recipe, or one from a run killed in the instant between creating it and
|
||||
recording it — the next start **replaces it**, along with any stale addresses
|
||||
it carried. Two things are never touched:
|
||||
## Quick start
|
||||
|
||||
* an interface that is not a TUN, because the name colliding with somebody's
|
||||
bridge is not a reason to destroy the bridge;
|
||||
* a TUN that another process is holding open, because that is a working
|
||||
overlay belonging to somebody else — most likely a second agent on this
|
||||
host, which should be given a different `--wg-prefix`.
|
||||
|
||||
Both of those refuse with an explanation rather than guessing.
|
||||
|
||||
Two settings the manual recipe used to need are gone with it.
|
||||
`keep_addr_on_down` existed only because an interface nobody held open lost
|
||||
carrier and had its IPv6 addresses flushed, and `nodad` only because duplicate
|
||||
address detection can never finish without carrier. An interface held open for
|
||||
its whole life has carrier for its whole life.
|
||||
|
||||
### The MTU is 1280
|
||||
|
||||
The default interface MTU stays at 1280 on direct paths, relays and paths
|
||||
inside another VPN. Large encrypted packets are split into smaller QUIC
|
||||
datagrams and reassembled before reaching WireGuard. SSH and full-size TCP
|
||||
segments therefore do not require a manual MTU override or MSS adjustment.
|
||||
The original IP packet, including its DF bit, is preserved. See
|
||||
[docs/wireguard.md](docs/wireguard.md#mtu) for limits and compatibility.
|
||||
|
||||
### Summary
|
||||
|
||||
| approach | agent runs as | notes |
|
||||
|---|---|---|
|
||||
| `setcap cap_net_admin+p` | ordinary user, one capability | recommended: nothing to prepare, nothing left behind. Lost on every rebuild or copy of the binary. |
|
||||
| systemd service | `User=`, `AmbientCapabilities=CAP_NET_ADMIN` | the same, for an installed service |
|
||||
| `sudo tsng up` | root | everything works, nothing is isolated |
|
||||
| macOS root LaunchDaemon | root | the native macOS deployment; see below |
|
||||
| `--no-tun` | ordinary user, no capabilities | tunnels run and handshake, traffic never reaches the OS |
|
||||
|
||||
On Windows, put `wintun.dll` beside the executable and start `tsng up`
|
||||
from PowerShell or Command Prompt opened with **Run as administrator**.
|
||||
Run commands controlling that agent (`join`, `status`, `dns`, `network`)
|
||||
as the same Windows user with the same elevation. If Windows denies access
|
||||
to the agent's control pipe, the command reports a permission error instead
|
||||
of claiming the agent is absent or trying to edit its locked state.
|
||||
Creating a TUN without sufficient privileges also explains how to restart
|
||||
the agent with the required permissions.
|
||||
|
||||
On macOS the agent creates a `utun` interface, configures it with `ifconfig`
|
||||
and `route`, and points the system resolver at the overlay by writing
|
||||
`/etc/resolver/<network>` files (which carry a port, so the DNS server needs no
|
||||
privileged socket). All of this needs **root**: macOS has no per-capability
|
||||
grant like Linux, and a `utun` plus `/etc/resolver` are root-only. The kernel
|
||||
assigns the interface its `utunN` name, so the name derived from the network id
|
||||
is only a request and the real one is adopted once the interface is up.
|
||||
|
||||
Run it as root — `sudo tsng up` for a quick start, or install it as a root
|
||||
**LaunchDaemon** for the native always-on deployment. As a daemon the agent runs
|
||||
unprivileged commands (`join`, `status`, `dns`, `network`) over its control
|
||||
socket like any other platform; the socket lives at `/var/run/tsunagi/agent.sock`
|
||||
and the stores at `/var/db/tsunagi`, both fixed system paths so they do not
|
||||
depend on whose shell started `sudo`. A ready-to-edit plist and install notes
|
||||
are in [dist/macos/](dist/macos/). The broadcast route is installed; the
|
||||
inbound-UDP firewall allowance (a `pf` anchor) is not yet, so `status` reports
|
||||
that half as incomplete and names what to allow by hand. The decision logic that
|
||||
says *what* to change is shared and tested on every platform.
|
||||
|
||||
## Checks
|
||||
|
||||
```bash
|
||||
cargo fmt --all -- --check
|
||||
cargo clippy --locked --workspace --all-targets -- -D warnings
|
||||
cargo test --locked --workspace --all-targets
|
||||
```sh
|
||||
tsng join -n home # first machine: makes the network and prints a secret
|
||||
tsng join -n home -s tsn1… # every other machine
|
||||
tsng status # who is online, over which path
|
||||
```
|
||||
|
||||
The whole suite runs offline on loopback. Set `TSUNAGI_TEST_LOG=tsunagi=debug`
|
||||
to see agent logs while a test runs.
|
||||
Every member gets an address in `10.13.37.0/24` and a name like `<host>.home`.
|
||||
Machines are called `tsunagi-<id>` until you rename them: `tsng id hostname laptop`.
|
||||
|
||||
There are also two runnable demos, which are demos and not substitutes for the
|
||||
tests:
|
||||
### Exit nodes
|
||||
|
||||
```bash
|
||||
cargo run --example two_agents # control plane only
|
||||
```sh
|
||||
tsng network exit-node home on # on the machine that should be the exit
|
||||
tsng exit-node # on another one: who offers an exit
|
||||
tsng exit-node laptop # send everything through it
|
||||
tsng exit-node off
|
||||
```
|
||||
|
||||
Both run with no privileges and change nothing on the host.
|
||||
The tray does the same with a checkbox. An exit node sees everything you send
|
||||
through it, so use only members you trust.
|
||||
[Details and per-OS notes](docs/exit-node.md).
|
||||
|
||||
## Usage
|
||||
## Platforms
|
||||
|
||||
```rust,no_run
|
||||
use std::sync::Arc;
|
||||
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::identity::{NetworkName, NetworkSecret};
|
||||
use tsunagi::proto::ControlMessage;
|
||||
use tsunagi::{Agent, Result};
|
||||
| Platform | Build | CLI `tsng` | Tray | Exit node: offer / use | Packages |
|
||||
| ------------------------- | ------------ | :--------: | :--: | :--------------------: | ----------------------------------- |
|
||||
| Linux x86_64 | release | ✔ | ✔ | ✔ / ✔ | `.deb`, `.pkg.tar.zst`, tarball, Nix |
|
||||
| Linux aarch64 | release | ✔ | –¹ | ✔ / ✔ | `.deb`, `.pkg.tar.zst`, tarball, Nix |
|
||||
| Linux armv7 | release | ✔ | –¹ | ✔ / ✔ | `.deb`, `.pkg.tar.zst`, tarball |
|
||||
| macOS, Apple silicon | release | ✔ | ✔ | ✔ / ✔² | Homebrew, tarball |
|
||||
| macOS, Intel | from source | ✔³ | ✔³ | ✔ / ✔²³ | – |
|
||||
| Windows x86_64 | release | ✔ | ✔ | ✔ / ✔² | installer, zip |
|
||||
|
||||
// The library never starts a runtime, installs a logger, handles signals,
|
||||
// forks, or calls process::exit. The binary owns all of that.
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<()> {
|
||||
let config = AgentConfig::new(StoragePaths::user_default()?)
|
||||
.with_transport(TransportPolicy::N0Defaults)
|
||||
.with_discovery(Arc::new(SharedMemoryDiscovery::new()));
|
||||
<sup>¹ Not in the release; builds from source (and through Nix).
|
||||
² Newer and less battle-tested than on Linux.
|
||||
³ Builds from source, not covered by CI.</sup>
|
||||
|
||||
let agent = Agent::spawn(config).await?;
|
||||
The Linux CLI is a single static binary: it runs on any distribution.
|
||||
|
||||
let name = NetworkName::new("kitchen-table")?;
|
||||
let secret = NetworkSecret::generate(); // 32 random bytes
|
||||
println!("share this: {}", secret.encode().as_str());
|
||||
## NixOS
|
||||
|
||||
let network = agent.join_network(&name, &secret).await?;
|
||||
The repository is a flake with a package and a module.
|
||||
|
||||
let mut events = agent.subscribe();
|
||||
tokio::spawn(async move {
|
||||
while let Ok(event) = events.recv().await {
|
||||
println!("{event:?}");
|
||||
```nix
|
||||
# flake.nix
|
||||
{
|
||||
inputs.tsunagi.url = "github:house-of-vanity/tsunagi";
|
||||
|
||||
outputs = { nixpkgs, tsunagi, ... }: {
|
||||
nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
|
||||
modules = [
|
||||
tsunagi.nixosModules.default
|
||||
{
|
||||
services.tsunagi = {
|
||||
enable = true;
|
||||
tray.enable = true; # optional, for desktops
|
||||
users = [ "alice" ]; # may use tsng and the tray without sudo
|
||||
};
|
||||
}
|
||||
});
|
||||
|
||||
for peer in agent.network_status(network).await?.connected_peers() {
|
||||
agent
|
||||
.send(network, peer, ControlMessage::Ping { seq: 1, payload: vec![] })
|
||||
.await?;
|
||||
}
|
||||
|
||||
agent.shutdown().await;
|
||||
Ok(())
|
||||
];
|
||||
};
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
`TransportPolicy::LocalOnly` is the default, so a plain `AgentConfig::new` never
|
||||
reaches the internet by accident. Opt into `DirectOnly` or `N0Defaults`
|
||||
explicitly.
|
||||
Just trying it: `nix run github:house-of-vanity/tsunagi#tsunagi-cli -- --help`.
|
||||
`packages.<system>.tsunagi` (with the tray) and `tsunagi-cli` are also exported,
|
||||
along with `overlays.default`.
|
||||
|
||||
### Through somebody in the middle
|
||||
## Build from source
|
||||
|
||||
Tsunagi routes the overlay in userspace. A direct data link always wins;
|
||||
otherwise the shortest available path can pass through several members.
|
||||
Each authenticated member advertises only its own live, protocol-specific
|
||||
transport links. The control plane collects these into a graph and replaces
|
||||
the routing table when connectivity changes; announcements expire after
|
||||
90 seconds without refresh.
|
||||
|
||||
Transit goes transport → router → transport, without entering WireGuard or
|
||||
the intermediate host's TUN. The payload stays encrypted between the original
|
||||
endpoints. TUN connects only the local OS to Tsunagi; the kernel needs only
|
||||
the overlay route through that interface, with no IP forwarding configuration.
|
||||
|
||||
A 16-hop limit bounds loops while topology updates converge. Equal shortest
|
||||
paths use a stable flow hash derived before encryption, so packets of a TCP
|
||||
or UDP flow keep their path while topology is unchanged. Physical link changes
|
||||
do not replace end-to-end WireGuard tunnels or change the 1280-byte default MTU.
|
||||
`status` reports the hop count and first next hop for relayed paths.
|
||||
|
||||
The hot path reads an immutable routing snapshot and sends directly to a cached
|
||||
transport handle. It takes no routing mutex, walks no graph and does not parse
|
||||
the encrypted payload. See [routing.md](docs/routing.md) for the architecture,
|
||||
limits and reproducible forwarding microbenchmark.
|
||||
|
||||
This wire format requires all members to upgrade together (control ALPN 3,
|
||||
data ALPN 4); saved identities, network names, secrets and addresses survive.
|
||||
|
||||
### LAN game discovery
|
||||
|
||||
IPv4 UDP broadcast relay is enabled by default for each network. The IP router
|
||||
sends one encrypted copy to each participating peer, including through multihop
|
||||
paths. It supports `255.255.255.255` and the overlay subnet's broadcast address;
|
||||
received broadcasts never trigger another fanout.
|
||||
Rust 1.91 or newer.
|
||||
|
||||
```sh
|
||||
tsng join -n games --no-broadcast
|
||||
tsng join -n games --broadcast
|
||||
tsng network broadcast games off
|
||||
tsng network broadcast games on
|
||||
cargo build --release --bin tsng # the agent and CLI
|
||||
cargo build --release --bin tsunagi-tray # the tray (Linux needs GTK 3 and libayatana-appindicator)
|
||||
```
|
||||
|
||||
The choice persists across restarts and a plain `join`. It can be changed while
|
||||
the agent runs; `status` shows it for each network. While broadcast is on, the
|
||||
agent routes `255.255.255.255` through the Tsunagi interface and allows inbound
|
||||
UDP from the overlay range on it (Linux and Windows, needs the same privilege as
|
||||
the interface), and removes both when it is turned off or the agent stops. Games
|
||||
bound to a physical address still bypass it. Physical LAN capture/subnet sharing
|
||||
is not implemented.
|
||||
See [broadcast.md](docs/broadcast.md) for domain isolation and future LAN gateways.
|
||||
Windows needs [Wintun](https://www.wintun.net) next to `tsng.exe`.
|
||||
Per-platform notes: [Linux](dist/linux/README.md), [Arch](dist/arch/README.md),
|
||||
[macOS](dist/macos/README.md), [Windows](dist/windows/README.md).
|
||||
|
||||
## How peers find each other
|
||||
## How it works
|
||||
|
||||
Two different lookups are involved, and only one of them is this project's:
|
||||
- Your name and secret derive a network identity. Members publish signed
|
||||
contact records to the public Mainline DHT under it, and find each other there.
|
||||
- Every connection proves knowledge of the secret before it's accepted.
|
||||
Traffic between members is end-to-end encrypted, relays included.
|
||||
- Addresses and names are claimed with each device's own key and replicated
|
||||
between members. There is no one in charge, so nothing to be taken down.
|
||||
|
||||
**1. Resolving one endpoint's address — iroh's, and it works today.**
|
||||
With `--reach relay` or `--reach direct`, iroh publishes a signed
|
||||
record of this endpoint's addresses, keyed by its endpoint id, to the public
|
||||
service run by Number 0 — "n0", the company behind iroh — at `dns.iroh.link`,
|
||||
over pkarr and DNS, and resolves other endpoints the same way. That is why `--peer <endpoint-id>` works with no address attached:
|
||||
iroh looks it up. None of that code is ours.
|
||||
|
||||
**2. Finding who is in a network — Mainline DHT by default.**
|
||||
`NetworkDiscovery` supplies unverified candidates. The Mainline backend uses
|
||||
a signing key derived from the network name and secret to publish signed
|
||||
BEP44 records containing each writer's endpoint ID and addresses. Its 16
|
||||
slots provide starting candidates, not a complete membership list or a limit
|
||||
on network size. After the first authenticated connection, introductions and
|
||||
signed state reveal the other members. `StaticBootstrap` (`--peer`) remains
|
||||
available, and tests use either a local Mainline Testnet or in-memory discovery.
|
||||
|
||||
What this means in practice:
|
||||
|
||||
- With `relay` or `direct`, **your endpoint id and IP addresses are published
|
||||
to a public third-party service** (Number 0's, unless you change it). They are not secret, and the network secret is
|
||||
never published, but an observer of that service learns that your endpoint
|
||||
exists and where it is. `--reach local` publishes nothing.
|
||||
- With DHT enabled, endpoint records are also published to the public Mainline
|
||||
DHT. They are signed, not encrypted; membership still requires the secret.
|
||||
- A relay, when one is needed, sees the volume and timing of your traffic — not
|
||||
its contents. The default relays are Number 0's, in the US, EU and
|
||||
Asia-Pacific.
|
||||
|
||||
## Storage
|
||||
|
||||
Two physically separate SQLite files, placed wherever the library's
|
||||
configuration says (`StoragePaths`). A future system service supplies its own
|
||||
paths; tests always use temporary directories.
|
||||
|
||||
| file | holds | when damaged |
|
||||
|----------------|--------------------------------------------------|-------------------------|
|
||||
| `state.sqlite` | device identity, network configuration, hostname | clear error, never reset |
|
||||
| `cache.sqlite` | address hints and other recoverable data | discarded and recreated |
|
||||
|
||||
The `wg-quic` protocol keeps its own keys in its own store under `wg-quic/`,
|
||||
because a protocol's keys are neither the iroh identity nor the network
|
||||
secret.
|
||||
|
||||
The command line agent puts everything under the platform's per-user
|
||||
directories by default; `--state-dir` and `--cache-dir` override them.
|
||||
|
||||
One state directory belongs to one live agent, enforced with a real OS file
|
||||
lock rather than an existence check. That directory *is* the identity: one
|
||||
agent, one device key, one interface, and as many networks as you like on it
|
||||
— `tsng network join` adds them to the agent that is already running,
|
||||
which is why a second `tsng up` on the same directory is refused rather
|
||||
than made to work. A second agent on the same host is a second identity, and
|
||||
needs everything of its own: its own state and cache directories, its own
|
||||
interface name and its own overlay range. The library owns no globals, so
|
||||
several of them run side by side in one process as readily as in one host.
|
||||
|
||||
### Leaving, and starting over
|
||||
|
||||
Membership outlives a session, so it also has to be possible to end it.
|
||||
|
||||
```bash
|
||||
tsng network # what this device belongs to
|
||||
tsng network leave lab # give up the address and the name, then forget it
|
||||
tsng wipe --yes # remove everything and be a stranger again
|
||||
```
|
||||
|
||||
**Stopping is not leaving.** `stop` closes this network's sessions, takes
|
||||
its address off the interface and keeps it from starting again, and that is
|
||||
all: the configuration, the secret, the signed state and the protocol key
|
||||
stay exactly as they are, nothing is announced, and to the others this
|
||||
device is simply away — an ordinary condition they already handle, with its
|
||||
address and name still reserved for it. `start` picks it up where it left
|
||||
off. A network named on the `up` command line is started by that command
|
||||
whatever its stored state, and the start-up banner says so rather than
|
||||
letting a `stop` quietly come back.
|
||||
|
||||
`leave` is the other one, and it removes, locally: the configuration and
|
||||
the secret, this network's signed records, its cached address hints and the
|
||||
protocol key it used there. What it keeps is this author's version counter
|
||||
for that network — a rejoin has to continue above the release, or every
|
||||
replica would treat the new claim as stale. Everything about *other*
|
||||
networks, and the device identity itself, is untouched.
|
||||
|
||||
`leave` publishes a signed `Release` **first**, while the agent is running and
|
||||
its sessions are up, so the address and the name are freed for the others
|
||||
instead of staying reserved to a member that has gone. They pass the tombstone
|
||||
on, so a member that was away hears it from them. With no agent running,
|
||||
nothing can sign or send it: the command says so and refuses, and `--offline`
|
||||
drops the network locally while leaving the others holding the old claim. The
|
||||
protocol key for that network goes too — rejoining is joining, not resuming.
|
||||
|
||||
`leave`, `stop`, `start`, `secret` and `broadcast` accept an exact stored network
|
||||
name, its full id, or a unique id prefix (including a copied report ellipsis).
|
||||
Names are case-sensitive. If several networks share a name, or a name matches
|
||||
another network's id prefix, the command lists matching names and full ids and
|
||||
changes nothing. Use a full id to choose explicitly; a full id always takes
|
||||
precedence over a name. No secret is needed to select a configured network.
|
||||
|
||||
`wipe` removes both directories' contents: the device identity, every network,
|
||||
every signed record and everything a protocol kept beside them. It refuses
|
||||
while an agent is running, and refuses a directory with no `state.sqlite` in
|
||||
it, so a mistyped `--state-dir` cannot take somebody's documents with it.
|
||||
Without `--yes` it only says what it would remove. It is not a goodbye: nobody
|
||||
is told, because after it there is no key left to sign anything with. Leave the
|
||||
networks first if the addresses should be freed.
|
||||
The secret is the only key, so make it a good one (the generated ones are).
|
||||
The [threat model](docs/threat-model.md) says what this does and doesn't protect against.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [docs/architecture.md](docs/architecture.md) — module boundaries and runtime.
|
||||
- [docs/wireguard.md](docs/wireguard.md) — the WireGuard plugin: overlay
|
||||
addressing, announcements, backends, reconciliation.
|
||||
- [docs/protocol.md](docs/protocol.md) — identity derivation, framing, handshake.
|
||||
- [docs/sync-model.md](docs/sync-model.md) — the planned signed-state model and
|
||||
what is deliberately not built yet.
|
||||
- [docs/exit-node.md](docs/exit-node.md) — exit nodes: sending all traffic through a member.
|
||||
- [docs/threat-model.md](docs/threat-model.md) — threat model and known limits.
|
||||
- [docs/testing.md](docs/testing.md) — what the suite covers and what it does not.
|
||||
- [AGENTS.md](AGENTS.md) — rules for anyone (human or agent) changing this repo.
|
||||
|
||||
## Security in one paragraph
|
||||
|
||||
Membership is proved by an HMAC over a transcript keyed by a value derived from
|
||||
the shared secret, bound to the specific iroh connection through the TLS
|
||||
exporter, to the network id, to both endpoint identities and to distinct role
|
||||
labels. This targets high-entropy secrets: there is no PAKE here, so a short
|
||||
human passphrase is guessable offline by anyone who can reach the handshake.
|
||||
Anyone who knows the secret is a full participant and can create many
|
||||
identities. Read [docs/threat-model.md](docs/threat-model.md) before relying on
|
||||
any of this.
|
||||
[Architecture](docs/architecture.md) ·
|
||||
[Protocol](docs/protocol.md) ·
|
||||
[WireGuard data plane](docs/wireguard.md) ·
|
||||
[Routing](docs/routing.md) ·
|
||||
[Exit nodes](docs/exit-node.md) ·
|
||||
[Threat model](docs/threat-model.md) ·
|
||||
[Contributing](AGENTS.md)
|
||||
|
||||
## Licence
|
||||
|
||||
|
||||
@@ -129,6 +129,19 @@ pub enum Event {
|
||||
/// Why it is not up.
|
||||
reason: String,
|
||||
},
|
||||
/// The name this agent claimed was claimed first by another member.
|
||||
///
|
||||
/// Nothing stops working: this agent keeps its address and every
|
||||
/// connection. It just is not found by that name until it is renamed or
|
||||
/// the member that holds the name lets it go.
|
||||
HostnameTaken {
|
||||
/// The network.
|
||||
network: NetworkId,
|
||||
/// The name that is taken.
|
||||
name: String,
|
||||
/// The member that holds it.
|
||||
holder: EndpointId,
|
||||
},
|
||||
/// An IP plugin reported an error. Never fatal.
|
||||
PluginError {
|
||||
/// The network the call was scoped to.
|
||||
|
||||
@@ -26,8 +26,8 @@ mod status;
|
||||
|
||||
pub use events::Event;
|
||||
pub use status::{
|
||||
AgentStatus, CandidateStatus, MemberStatus, NetworkMetrics, NetworkState, NetworkStatus,
|
||||
OverlayStatus, PeerStatus,
|
||||
AgentStatus, CandidateStatus, DataPath, HostnameConflict, MemberStatus, NetworkMetrics,
|
||||
NetworkState, NetworkStatus, OverlayStatus, PeerStatus,
|
||||
};
|
||||
|
||||
use std::collections::HashMap;
|
||||
@@ -981,6 +981,7 @@ impl Agent {
|
||||
// An inactive network has no runtime to ask; the roster and
|
||||
// the range come from one. Empty, not invented.
|
||||
members: Vec::new(),
|
||||
hostname_conflict: None,
|
||||
range: None,
|
||||
range_conflict: None,
|
||||
metrics: NetworkMetrics::default(),
|
||||
|
||||
@@ -32,7 +32,8 @@ use super::events::Event;
|
||||
use super::session::{self, Session, SessionEvent};
|
||||
use super::shutdown::Shutdown;
|
||||
use super::status::{
|
||||
CandidateStatus, MemberStatus, NetworkMetrics, NetworkState, NetworkStatus, PeerStatus,
|
||||
CandidateStatus, DataPath, HostnameConflict, MemberStatus, NetworkMetrics, NetworkState,
|
||||
NetworkStatus, PeerStatus,
|
||||
};
|
||||
|
||||
/// An inbound connection that already passed the handshake.
|
||||
@@ -320,6 +321,10 @@ struct Runtime {
|
||||
std::time::Instant,
|
||||
),
|
||||
>,
|
||||
/// The protocols this agent last announced, in the order it announced
|
||||
/// them. The order is its preference, and the lower of two ids settles
|
||||
/// which protocol a pair runs.
|
||||
local_capabilities: Vec<PluginCapability>,
|
||||
/// What this agent last told its peers it could reach, so it is only
|
||||
/// said again when it changed.
|
||||
announced_reach: HashSet<crate::proto::message::ReachableLink>,
|
||||
@@ -338,6 +343,9 @@ struct Runtime {
|
||||
/// publish a claim afterwards — the periodic check, a peer's records
|
||||
/// arriving — would silently take it back.
|
||||
released: bool,
|
||||
/// The name this agent last said was taken, so it is said once per
|
||||
/// change rather than on every round.
|
||||
reported_name_taken: Option<HostnameConflict>,
|
||||
/// The address last reported as absent from the interface, so it is said
|
||||
/// once rather than for ever.
|
||||
reported_missing: Option<std::net::Ipv4Addr>,
|
||||
@@ -366,6 +374,16 @@ impl Runtime {
|
||||
let (link_results_tx, link_results_rx) = mpsc::channel(64);
|
||||
let (candidate_tx, discovery_candidates) =
|
||||
mpsc::channel(params.limits.max_discovery_candidates.max(1));
|
||||
let hub = crate::dataplane::relay::RelayHub::new(network_id, local_id);
|
||||
// The order the protocols were configured in is also how links are
|
||||
// preferred when a neighbour can be reached over more than one.
|
||||
hub.set_priority(
|
||||
params
|
||||
.plugins
|
||||
.iter()
|
||||
.map(|plugin| plugin.protocol_id().to_string())
|
||||
.collect(),
|
||||
);
|
||||
let discovery_worker = params.discovery.as_ref().map(|backend| {
|
||||
DiscoveryWorker::spawn(
|
||||
backend.clone(),
|
||||
@@ -393,8 +411,9 @@ impl Runtime {
|
||||
dial_results_tx,
|
||||
dial_results_rx,
|
||||
links: HashMap::new(),
|
||||
hub: crate::dataplane::relay::RelayHub::new(network_id, local_id),
|
||||
hub,
|
||||
reachable: HashMap::new(),
|
||||
local_capabilities: Vec::new(),
|
||||
announced_reach: HashSet::new(),
|
||||
opening: HashSet::new(),
|
||||
link_results_tx,
|
||||
@@ -403,6 +422,7 @@ impl Runtime {
|
||||
reported_mismatch: HashSet::new(),
|
||||
released: false,
|
||||
reported_missing: None,
|
||||
reported_name_taken: None,
|
||||
state: StateSet::new(),
|
||||
pending_state: Vec::new(),
|
||||
own_version: 0,
|
||||
@@ -617,6 +637,8 @@ impl Runtime {
|
||||
tracing::debug!(%err, "could not queue re-announcement");
|
||||
}
|
||||
}
|
||||
// What was announced decides which protocol each pair runs.
|
||||
self.update_paths();
|
||||
}
|
||||
|
||||
/// Queues a message without blocking the runtime loop.
|
||||
@@ -669,6 +691,7 @@ impl Runtime {
|
||||
// alone, so something has to look again; this runs on a timer and
|
||||
// the check is a comparison when nothing has changed.
|
||||
self.ensure_own_claim().await;
|
||||
self.report_hostname_taken();
|
||||
// The same timer repairs the relay picture: reachability is soft
|
||||
// state, repeated while it holds and forgotten when it stops.
|
||||
self.update_paths();
|
||||
@@ -1031,12 +1054,25 @@ impl Runtime {
|
||||
}
|
||||
};
|
||||
|
||||
// A name keeps the time it was first claimed for as long as this
|
||||
// member keeps the name, so a later claimant cannot overtake it by
|
||||
// the passage of time, and a new or changed name starts the clock
|
||||
// afresh.
|
||||
let hostname_since = wanted_hostname.as_ref().map(|name| {
|
||||
let kept = self
|
||||
.state
|
||||
.get(&self.local_id)
|
||||
.filter(|record| record.body.hostname() == Some(name.as_str()))
|
||||
.and_then(|record| record.body.hostname_since());
|
||||
kept.unwrap_or_else(unix_millis)
|
||||
});
|
||||
let body = RecordBody::Claim {
|
||||
address: wanted_address,
|
||||
// The range only travels with an address, so a member of an
|
||||
// IPv6-only network does not assert one.
|
||||
range: wanted_address.and(range),
|
||||
hostname: wanted_hostname,
|
||||
hostname_since,
|
||||
};
|
||||
|
||||
// Nothing to say is not the same as saying nothing changed: a member
|
||||
@@ -1100,6 +1136,7 @@ impl Runtime {
|
||||
for (peer, records) in std::mem::take(&mut self.pending_state) {
|
||||
self.receive_state(peer, records).await;
|
||||
}
|
||||
self.report_hostname_taken();
|
||||
}
|
||||
|
||||
/// Sends everything we know to every peer.
|
||||
@@ -1658,68 +1695,93 @@ impl Runtime {
|
||||
}
|
||||
}
|
||||
|
||||
/// The protocol this agent and a member run their tunnel in, if they have
|
||||
/// one in common.
|
||||
///
|
||||
/// Both ends must name the same one without asking each other, and what
|
||||
/// each side was configured to prefer can differ. So the member with the
|
||||
/// lower id decides, by the order it announces its protocols in, and the
|
||||
/// other side reads that order from the announcement it already has.
|
||||
fn tunnel_protocol(&self, session: &Session) -> Option<String> {
|
||||
let usable = |ours: &[PluginCapability], theirs: &[PluginCapability], name: &str| {
|
||||
ours.iter().any(|capability| {
|
||||
capability.enabled
|
||||
&& capability.protocol == name
|
||||
&& theirs.iter().any(|other| {
|
||||
other.enabled
|
||||
&& other.protocol == name
|
||||
&& other.version == capability.version
|
||||
})
|
||||
})
|
||||
};
|
||||
let (decides, other) = if self.local_id.as_bytes() < session.peer.as_bytes() {
|
||||
(&self.local_capabilities, &session.capabilities)
|
||||
} else {
|
||||
(&session.capabilities, &self.local_capabilities)
|
||||
};
|
||||
decides
|
||||
.iter()
|
||||
.find(|capability| usable(decides, other, &capability.protocol))
|
||||
.map(|capability| capability.protocol.clone())
|
||||
}
|
||||
|
||||
/// Builds the graph off the packet path. Every remote row comes from that
|
||||
/// member's authenticated control session; stale or incompatible rows never
|
||||
/// enter the graph. The hub replaces an immutable table only on changes.
|
||||
/// member's authenticated control session, and a row says only who that
|
||||
/// member has a link to, never what kind of link: a member in the middle
|
||||
/// needs no protocol in common with either end. Stale rows never enter the
|
||||
/// graph. The hub replaces an immutable table only on changes.
|
||||
fn update_paths(&mut self) {
|
||||
self.reachable.retain(|peer, (_, heard)| {
|
||||
heard.elapsed() < REACH_EXPIRY && self.sessions.contains_key(peer)
|
||||
});
|
||||
for (protocol, version) in self.served_protocols() {
|
||||
let peers: Vec<_> = self
|
||||
.sessions
|
||||
.values()
|
||||
.filter(|session| {
|
||||
session.capabilities.iter().any(|capability| {
|
||||
capability.enabled
|
||||
&& capability.protocol == protocol
|
||||
&& capability.version == version
|
||||
})
|
||||
})
|
||||
.map(|session| session.peer)
|
||||
.collect();
|
||||
let mut members: HashSet<_> = peers.iter().map(|peer| *peer.as_bytes()).collect();
|
||||
members.insert(*self.local_id.as_bytes());
|
||||
let graph = self
|
||||
.reachable
|
||||
.iter()
|
||||
.filter(|(peer, _)| members.contains(peer.as_bytes()))
|
||||
.map(|(peer, (links, _))| {
|
||||
let mut neighbors: Vec<_> = links
|
||||
.iter()
|
||||
.filter(|link| {
|
||||
link.protocol == protocol
|
||||
&& members.contains(&link.peer)
|
||||
&& link.peer != *peer.as_bytes()
|
||||
})
|
||||
.map(|link| link.peer)
|
||||
.collect();
|
||||
neighbors.sort_unstable();
|
||||
(*peer.as_bytes(), neighbors)
|
||||
})
|
||||
.collect();
|
||||
self.hub.set_topology(&protocol, graph, members);
|
||||
for peer in peers {
|
||||
if self.hub.has_link(peer, &protocol) || !self.hub.reachable(peer, &protocol) {
|
||||
continue;
|
||||
}
|
||||
let Some(plugin) = self
|
||||
.params
|
||||
.plugins
|
||||
let mut members: HashSet<_> = self.sessions.keys().map(|peer| *peer.as_bytes()).collect();
|
||||
members.insert(*self.local_id.as_bytes());
|
||||
let graph = self
|
||||
.reachable
|
||||
.iter()
|
||||
.filter(|(peer, _)| members.contains(peer.as_bytes()))
|
||||
.map(|(peer, (links, _))| {
|
||||
let mut neighbors: Vec<_> = links
|
||||
.iter()
|
||||
.find(|plugin| plugin.protocol_id() == protocol)
|
||||
.cloned()
|
||||
else {
|
||||
continue;
|
||||
};
|
||||
let link = self.hub.link(peer, &protocol);
|
||||
let path = link.path_description();
|
||||
let max_datagram = link.max_datagram_size();
|
||||
plugin.on_peer_link(self.network_id, peer, link);
|
||||
.filter(|link| members.contains(&link.peer) && link.peer != *peer.as_bytes())
|
||||
.map(|link| link.peer)
|
||||
.collect();
|
||||
neighbors.sort_unstable();
|
||||
neighbors.dedup();
|
||||
(*peer.as_bytes(), neighbors)
|
||||
})
|
||||
.collect();
|
||||
self.hub.set_topology(graph, members);
|
||||
|
||||
let wanted: Vec<(EndpointId, String)> = self
|
||||
.sessions
|
||||
.values()
|
||||
.filter_map(|session| Some((session.peer, self.tunnel_protocol(session)?)))
|
||||
.collect();
|
||||
for (peer, protocol) in wanted {
|
||||
if self.hub.has_link(peer, &protocol) || !self.hub.reachable(peer) {
|
||||
continue;
|
||||
}
|
||||
let Some(plugin) = self
|
||||
.params
|
||||
.plugins
|
||||
.iter()
|
||||
.find(|plugin| plugin.protocol_id() == protocol)
|
||||
.cloned()
|
||||
else {
|
||||
continue;
|
||||
};
|
||||
let link = self.hub.link(peer, &protocol);
|
||||
let relayed = self.hub.route(peer).is_some_and(|route| route.hops > 1);
|
||||
let path = link.path_description();
|
||||
let max_datagram = link.max_datagram_size();
|
||||
plugin.on_peer_link(self.network_id, peer, link);
|
||||
// A direct link has already been reported as one.
|
||||
if relayed {
|
||||
self.emit(Event::DataLinkUp {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
protocol: protocol.clone(),
|
||||
protocol,
|
||||
path,
|
||||
max_datagram,
|
||||
});
|
||||
@@ -1786,30 +1848,25 @@ impl Runtime {
|
||||
|
||||
/// Hands a link to the plugin that owns its protocol.
|
||||
fn adopt_link(&mut self, peer: EndpointId, protocol: String, link: SharedLink) {
|
||||
let Some(plugin) = self
|
||||
if !self
|
||||
.params
|
||||
.plugins
|
||||
.iter()
|
||||
.find(|plugin| plugin.protocol_id() == protocol)
|
||||
.cloned()
|
||||
else {
|
||||
.any(|plugin| plugin.protocol_id() == protocol)
|
||||
{
|
||||
return;
|
||||
};
|
||||
}
|
||||
|
||||
let path = link.path_description();
|
||||
let max_datagram = link.max_datagram_size();
|
||||
self.links
|
||||
.insert((peer, protocol.clone()), Arc::clone(&link));
|
||||
// The protocol is handed the hub's link, not this one: the same
|
||||
// object for as long as the peer is a peer, whatever happens to
|
||||
// the path under it. A tunnel is not torn down because a direct
|
||||
// link came or went.
|
||||
let first_time = !self.hub.has_link(peer, &protocol);
|
||||
// What this is, is a way to reach the peer. Which protocol's
|
||||
// tunnel runs over it is decided in `update_paths`, and that tunnel
|
||||
// is handed the hub's link, not this one: the same object for as
|
||||
// long as the peer is a peer, whatever happens to the path under
|
||||
// it. A tunnel is not torn down because a direct link came or went.
|
||||
self.hub.set_direct(peer, &protocol, link);
|
||||
let shared = self.hub.link(peer, &protocol);
|
||||
let max_datagram = shared.max_datagram_size();
|
||||
if first_time {
|
||||
plugin.on_peer_link(self.network_id, peer, shared);
|
||||
}
|
||||
self.metrics.data_links_established += 1;
|
||||
self.emit(Event::DataLinkUp {
|
||||
network: self.network_id,
|
||||
@@ -1941,6 +1998,13 @@ impl Runtime {
|
||||
});
|
||||
}
|
||||
capabilities.truncate(self.params.limits.max_capabilities);
|
||||
self.local_capabilities = capabilities
|
||||
.iter()
|
||||
.map(|capability| PluginCapability {
|
||||
data: Vec::new(),
|
||||
..capability.clone()
|
||||
})
|
||||
.collect();
|
||||
let exit_node = self.exit_ready();
|
||||
self.announced_exit = exit_node;
|
||||
Announcement {
|
||||
@@ -2175,6 +2239,69 @@ impl Runtime {
|
||||
|
||||
// ----------------------------------------------------------------- status
|
||||
|
||||
/// Whether a member can be reached now, by any path.
|
||||
///
|
||||
/// The single definition every client reports: a live control session,
|
||||
/// or a data path to it through other members. Being reached over a long
|
||||
/// path is being online; how well is `data_path`'s to say.
|
||||
fn is_online(&self, peer: EndpointId) -> bool {
|
||||
peer == self.local_id || self.sessions.contains_key(&peer) || self.hub.reachable(peer)
|
||||
}
|
||||
|
||||
/// How packets reach a member, which says nothing about whether it is
|
||||
/// there.
|
||||
fn data_path(&self, peer: EndpointId) -> DataPath {
|
||||
match self.hub.route(peer) {
|
||||
None => DataPath::None,
|
||||
Some(route) if route.hops <= 1 => DataPath::Direct {
|
||||
transport: self
|
||||
.hub
|
||||
.direct_kinds(peer)
|
||||
.into_iter()
|
||||
.next()
|
||||
.unwrap_or_default(),
|
||||
},
|
||||
Some(route) => DataPath::Relayed {
|
||||
hops: route.hops,
|
||||
via: route.via,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/// Says so when this agent's own name is held by someone else, and
|
||||
/// again if that changes. Never anything but a statement: the agent keeps
|
||||
/// working by address either way.
|
||||
fn report_hostname_taken(&mut self) {
|
||||
let taken = self.hostname_conflict_of(&self.local_id);
|
||||
if taken == self.reported_name_taken {
|
||||
return;
|
||||
}
|
||||
if let Some(conflict) = &taken {
|
||||
tracing::warn!(
|
||||
network = %self.network_id,
|
||||
name = %conflict.name,
|
||||
holder = %conflict.holder.fmt_short(),
|
||||
"this name was claimed first by another member; it stays reachable by \
|
||||
address but is not found by this name until it is renamed"
|
||||
);
|
||||
self.emit(Event::HostnameTaken {
|
||||
network: self.network_id,
|
||||
name: conflict.name.clone(),
|
||||
holder: conflict.holder,
|
||||
});
|
||||
}
|
||||
self.reported_name_taken = taken;
|
||||
}
|
||||
|
||||
fn hostname_conflict_of(&self, author: &EndpointId) -> Option<HostnameConflict> {
|
||||
self.state
|
||||
.hostname_taken(author)
|
||||
.map(|(name, holder)| HostnameConflict {
|
||||
name: name.to_string(),
|
||||
holder,
|
||||
})
|
||||
}
|
||||
|
||||
fn status(&self) -> NetworkStatus {
|
||||
let served = self.served_protocols();
|
||||
let mut peers: Vec<PeerStatus> = self
|
||||
@@ -2212,6 +2339,7 @@ impl Runtime {
|
||||
control_messages_received: session.messages_received,
|
||||
control_bytes_sent: session.bytes_sent,
|
||||
control_bytes_received: session.bytes_received,
|
||||
data_path: self.data_path(session.peer),
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
@@ -2249,6 +2377,9 @@ impl Runtime {
|
||||
overlay_address_v4: self.state.address_of(&endpoint_id),
|
||||
hostname: self.state.hostname_of(&endpoint_id).map(str::to_string),
|
||||
last_hostname: self.known_hostnames.get(&endpoint_id).cloned(),
|
||||
hostname_conflict: self.hostname_conflict_of(&endpoint_id),
|
||||
online: self.is_online(endpoint_id),
|
||||
data_path: self.data_path(endpoint_id),
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
@@ -2276,6 +2407,7 @@ impl Runtime {
|
||||
peers,
|
||||
candidates,
|
||||
members,
|
||||
hostname_conflict: self.hostname_conflict_of(&self.local_id),
|
||||
range: self.effective_range(),
|
||||
// Only worth reporting while it is actually stuck: with a
|
||||
// fallback in play the configured range being another
|
||||
@@ -2290,6 +2422,15 @@ impl Runtime {
|
||||
}
|
||||
}
|
||||
|
||||
/// Milliseconds since the Unix epoch, for the time a name was first claimed.
|
||||
fn unix_millis() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map_or(u64::MAX, |elapsed| {
|
||||
u64::try_from(elapsed.as_millis()).unwrap_or(u64::MAX)
|
||||
})
|
||||
}
|
||||
|
||||
/// Which side dialled, given a role and the two identities.
|
||||
fn initiator_of(role: Role, local: EndpointId, peer: EndpointId) -> EndpointId {
|
||||
match role {
|
||||
|
||||
@@ -59,6 +59,43 @@ pub struct OverlayStatus {
|
||||
pub broadcast_rules: Option<crate::overlay::BroadcastRulesReport>,
|
||||
}
|
||||
|
||||
/// How packets reach a member right now.
|
||||
///
|
||||
/// Being reachable is one thing and being reachable well is another, and
|
||||
/// the second is never a reason to call a member away: a member that can only
|
||||
/// be reached through somebody else is online, over a longer path.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum DataPath {
|
||||
/// Nothing carries packets to it yet. The member is connected for
|
||||
/// control only, which is the state of a data path being set up.
|
||||
None,
|
||||
/// A direct link, of the named kind of transport.
|
||||
Direct {
|
||||
/// The kind of transport carrying it, e.g. `wg`.
|
||||
transport: String,
|
||||
},
|
||||
/// Through other members.
|
||||
Relayed {
|
||||
/// Number of links on the way, at least two.
|
||||
hops: u8,
|
||||
/// The member the first of them goes to.
|
||||
via: EndpointId,
|
||||
},
|
||||
}
|
||||
|
||||
/// A name that a member claimed and somebody else holds.
|
||||
///
|
||||
/// A name belongs to whoever claimed it first. The member that claimed it
|
||||
/// later is still a member and still reachable by its address; it is just
|
||||
/// not found by that name, on any member of the network.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct HostnameConflict {
|
||||
/// The name that is taken.
|
||||
pub name: String,
|
||||
/// The member that holds it.
|
||||
pub holder: EndpointId,
|
||||
}
|
||||
|
||||
/// A member the signed state knows about, connected or not.
|
||||
///
|
||||
/// This is the durable roster: it comes from signed records, so a member that
|
||||
@@ -85,6 +122,15 @@ pub struct MemberStatus {
|
||||
/// Not signed, so it is a convenience for naming a member that is away
|
||||
/// and never a claim: the zone is served from `hostname`.
|
||||
pub last_hostname: Option<String>,
|
||||
/// The name it claimed, when somebody else claimed it first.
|
||||
pub hostname_conflict: Option<HostnameConflict>,
|
||||
/// Whether this agent can reach it now, by any path: a live control
|
||||
/// session, or a data path through other members. The one answer to "is
|
||||
/// it online" for every client, so that none of them has to work it out
|
||||
/// from a different part of the picture. This agent counts as online.
|
||||
pub online: bool,
|
||||
/// How packets reach it. [`DataPath::None`] while it is offline.
|
||||
pub data_path: DataPath,
|
||||
}
|
||||
|
||||
/// Status of one authenticated session.
|
||||
@@ -128,6 +174,8 @@ pub struct PeerStatus {
|
||||
pub control_bytes_sent: u64,
|
||||
/// Control payload bytes read from this session.
|
||||
pub control_bytes_received: u64,
|
||||
/// How packets reach this peer right now.
|
||||
pub data_path: DataPath,
|
||||
}
|
||||
|
||||
/// Counters scoped to one logical network.
|
||||
@@ -188,6 +236,10 @@ pub struct NetworkStatus {
|
||||
pub candidates: Vec<CandidateStatus>,
|
||||
/// Members the signed state knows about, whether connected or not.
|
||||
pub members: Vec<MemberStatus>,
|
||||
/// This agent's own name, when another member claimed it first and so
|
||||
/// holds it. This agent keeps working by address; nobody finds it by
|
||||
/// this name until it is renamed or the other member lets it go.
|
||||
pub hostname_conflict: Option<HostnameConflict>,
|
||||
/// The overlay range this network uses, once it has one.
|
||||
pub range: Option<crate::state::Ipv4Range>,
|
||||
/// The range it could not have, because another network on this agent
|
||||
|
||||
@@ -67,10 +67,41 @@ impl Tally {
|
||||
}
|
||||
}
|
||||
|
||||
/// The links to one neighbour, best first. Which transport carries a frame
|
||||
/// is decided here and nowhere else: the end-to-end protocol is in the
|
||||
/// envelope and never depends on it.
|
||||
#[derive(Debug)]
|
||||
struct NextHop {
|
||||
peer: PeerId,
|
||||
link: SharedLink,
|
||||
links: Box<[SharedLink]>,
|
||||
}
|
||||
|
||||
impl NextHop {
|
||||
fn usable(&self) -> bool {
|
||||
self.links.iter().any(|link| !link.is_closed())
|
||||
}
|
||||
|
||||
fn first_open(&self) -> Option<&SharedLink> {
|
||||
self.links.iter().find(|link| !link.is_closed())
|
||||
}
|
||||
|
||||
/// Hands a frame to the best open link that accepts it. A link that
|
||||
/// cannot carry the frame (too large for its path, closed a moment ago)
|
||||
/// leaves it to the next one; the frame is only copied when there is
|
||||
/// another to try.
|
||||
fn send(&self, frame: Bytes) -> Result<(), TransportError> {
|
||||
let mut open = self.links.iter().filter(|link| !link.is_closed());
|
||||
let Some(mut current) = open.next() else {
|
||||
return Err(TransportError::Closed);
|
||||
};
|
||||
for next in open {
|
||||
if current.send(frame.clone()).is_ok() {
|
||||
return Ok(());
|
||||
}
|
||||
current = next;
|
||||
}
|
||||
current.send(frame)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
@@ -82,21 +113,22 @@ struct ForwardRoute {
|
||||
impl ForwardRoute {
|
||||
fn select(&self, source: &PeerId, flow: FlowId) -> Option<&NextHop> {
|
||||
if self.next.len() == 1 {
|
||||
return self.next.first().filter(|hop| !hop.link.is_closed());
|
||||
return self.next.first().filter(|hop| hop.usable());
|
||||
}
|
||||
let seed = u64::from_le_bytes(source[..8].try_into().ok()?);
|
||||
let start = mix64(flow ^ seed) as usize % self.next.len();
|
||||
// Stable fallback if a link closed just before its table was replaced.
|
||||
(0..self.next.len())
|
||||
.map(|i| &self.next[(start + i) % self.next.len()])
|
||||
.find(|hop| !hop.link.is_closed())
|
||||
.find(|hop| hop.usable())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct ForwardingTable {
|
||||
routes: HashMap<PeerId, ForwardRoute>,
|
||||
inboxes: HashMap<PeerId, mpsc::Sender<Bytes>>,
|
||||
/// Local delivery, by who sent it and which end-to-end protocol it is.
|
||||
inboxes: HashMap<(PeerId, u64), mpsc::Sender<Bytes>>,
|
||||
members: HashSet<PeerId>,
|
||||
}
|
||||
|
||||
@@ -121,7 +153,7 @@ impl Plane {
|
||||
return;
|
||||
}
|
||||
if header.destination == self.local {
|
||||
let Some(inbox) = table.inboxes.get(&header.source) else {
|
||||
let Some(inbox) = table.inboxes.get(&(header.source, header.protocol)) else {
|
||||
self.tally.dropped_unknown.fetch_add(1, Ordering::Relaxed);
|
||||
return;
|
||||
};
|
||||
@@ -144,7 +176,7 @@ impl Plane {
|
||||
self.tally.dropped_no_link.fetch_add(1, Ordering::Relaxed);
|
||||
return;
|
||||
};
|
||||
if hop.link.send(envelope::decrement(frame)).is_ok() {
|
||||
if hop.send(envelope::decrement(frame)).is_ok() {
|
||||
self.tally.forwarded.fetch_add(1, Ordering::Relaxed);
|
||||
} else {
|
||||
self.tally.dropped_no_link.fetch_add(1, Ordering::Relaxed);
|
||||
@@ -152,27 +184,65 @@ impl Plane {
|
||||
}
|
||||
}
|
||||
|
||||
/// One physical link to a neighbour, of whatever kind of transport.
|
||||
#[derive(Debug)]
|
||||
struct Protocol {
|
||||
plane: Arc<Plane>,
|
||||
graph: HashMap<PeerId, Vec<PeerId>>,
|
||||
members: HashSet<PeerId>,
|
||||
raw: HashMap<PeerId, SharedLink>,
|
||||
readers: HashMap<PeerId, JoinHandle<()>>,
|
||||
links: HashMap<PeerId, Arc<PeerLink>>,
|
||||
struct RawLink {
|
||||
kind: String,
|
||||
link: SharedLink,
|
||||
reader: JoinHandle<()>,
|
||||
}
|
||||
|
||||
impl Protocol {
|
||||
/// How a destination is reached right now.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct RouteSummary {
|
||||
/// Number of transport links to it. One is a direct link.
|
||||
pub hops: u8,
|
||||
/// The neighbour the first link goes to. For a direct link that is the
|
||||
/// destination itself.
|
||||
pub via: EndpointId,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct State {
|
||||
plane: Arc<Plane>,
|
||||
/// Transport kinds, best first. A neighbour with several links is
|
||||
/// reached over the first of these it has.
|
||||
priority: Vec<String>,
|
||||
graph: HashMap<PeerId, Vec<PeerId>>,
|
||||
members: HashSet<PeerId>,
|
||||
raw: HashMap<PeerId, Vec<RawLink>>,
|
||||
links: HashMap<(PeerId, String), Arc<PeerLink>>,
|
||||
}
|
||||
|
||||
impl State {
|
||||
fn rank(&self, kind: &str) -> usize {
|
||||
self.priority
|
||||
.iter()
|
||||
.position(|known| known == kind)
|
||||
.unwrap_or(usize::MAX)
|
||||
}
|
||||
|
||||
/// A neighbour's links, best kind first.
|
||||
fn ordered(&self, peer: &PeerId) -> Vec<&RawLink> {
|
||||
let mut links: Vec<&RawLink> = self
|
||||
.raw
|
||||
.get(peer)
|
||||
.map(|links| links.iter().collect())
|
||||
.unwrap_or_default();
|
||||
links.sort_by_key(|raw| (self.rank(&raw.kind), raw.kind.clone()));
|
||||
links
|
||||
}
|
||||
|
||||
fn publish(&self) {
|
||||
let mut graph = self.graph.clone();
|
||||
graph.insert(
|
||||
self.plane.local,
|
||||
self.raw
|
||||
.iter()
|
||||
.filter(|(_, link)| !link.is_closed())
|
||||
.map(|(peer, _)| *peer)
|
||||
.collect(),
|
||||
);
|
||||
let mut neighbours: Vec<PeerId> = self
|
||||
.raw
|
||||
.iter()
|
||||
.filter(|(_, links)| links.iter().any(|raw| !raw.link.is_closed()))
|
||||
.map(|(peer, _)| *peer)
|
||||
.collect();
|
||||
neighbours.sort_unstable();
|
||||
graph.insert(self.plane.local, neighbours);
|
||||
let routing = RoutingTable::build(self.plane.local, &graph, ROUTING_HOP_LIMIT);
|
||||
let routes = routing
|
||||
.iter()
|
||||
@@ -181,10 +251,12 @@ impl Protocol {
|
||||
.next_hops
|
||||
.iter()
|
||||
.filter_map(|id| {
|
||||
self.raw.get(id).map(|link| NextHop {
|
||||
peer: *id,
|
||||
link: link.clone(),
|
||||
})
|
||||
let links: Box<[_]> = self
|
||||
.ordered(id)
|
||||
.into_iter()
|
||||
.map(|raw| raw.link.clone())
|
||||
.collect();
|
||||
(!links.is_empty()).then_some(NextHop { peer: *id, links })
|
||||
})
|
||||
.collect();
|
||||
(!next.is_empty()).then_some((
|
||||
@@ -204,17 +276,21 @@ impl Protocol {
|
||||
inboxes: self
|
||||
.links
|
||||
.iter()
|
||||
.map(|(peer, link)| (*peer, link.inbox_tx.clone()))
|
||||
.map(|((peer, _), link)| ((*peer, link.tag), link.inbox_tx.clone()))
|
||||
.collect(),
|
||||
}));
|
||||
}
|
||||
|
||||
fn stop_readers(&mut self) {
|
||||
for raw in self.raw.values().flatten() {
|
||||
raw.reader.abort();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Protocol {
|
||||
impl Drop for State {
|
||||
fn drop(&mut self) {
|
||||
for task in self.readers.values() {
|
||||
task.abort();
|
||||
}
|
||||
self.stop_readers();
|
||||
for link in self.links.values() {
|
||||
link.close();
|
||||
}
|
||||
@@ -222,12 +298,17 @@ impl Drop for Protocol {
|
||||
}
|
||||
}
|
||||
|
||||
/// One network's transport links and independently published protocol tables.
|
||||
/// One network's transport links and its published forwarding table.
|
||||
///
|
||||
/// There is one table for the whole network, whatever protocols run over it.
|
||||
/// A link is a way to reach a neighbour; a protocol is an end-to-end channel
|
||||
/// to a peer. The two meet only in the envelope, which names the protocol, so
|
||||
/// a tunnel keeps working over any mix of transports and a middle member
|
||||
/// needs no protocol in common with either end.
|
||||
#[derive(Debug)]
|
||||
pub struct RelayHub {
|
||||
network: NetworkId,
|
||||
local: PeerId,
|
||||
protocols: Mutex<HashMap<String, Protocol>>,
|
||||
state: Mutex<State>,
|
||||
tally: Arc<Tally>,
|
||||
changed: Arc<Notify>,
|
||||
}
|
||||
@@ -235,40 +316,38 @@ pub struct RelayHub {
|
||||
impl RelayHub {
|
||||
/// Creates the router at this network's local endpoint.
|
||||
pub fn new(network: NetworkId, local: EndpointId) -> Arc<Self> {
|
||||
let tally = Arc::new(Tally::default());
|
||||
Arc::new(Self {
|
||||
network,
|
||||
local: *local.as_bytes(),
|
||||
protocols: Mutex::default(),
|
||||
tally: Arc::default(),
|
||||
state: Mutex::new(State {
|
||||
plane: Arc::new(Plane {
|
||||
local: *local.as_bytes(),
|
||||
table: ArcSwap::from_pointee(ForwardingTable::default()),
|
||||
tally: tally.clone(),
|
||||
}),
|
||||
priority: Vec::new(),
|
||||
graph: HashMap::new(),
|
||||
members: HashSet::new(),
|
||||
raw: HashMap::new(),
|
||||
links: HashMap::new(),
|
||||
}),
|
||||
tally,
|
||||
changed: Arc::default(),
|
||||
})
|
||||
}
|
||||
|
||||
fn protocols(&self) -> std::sync::MutexGuard<'_, HashMap<String, Protocol>> {
|
||||
self.protocols
|
||||
.lock()
|
||||
.unwrap_or_else(|error| error.into_inner())
|
||||
fn state(&self) -> std::sync::MutexGuard<'_, State> {
|
||||
self.state.lock().unwrap_or_else(|error| error.into_inner())
|
||||
}
|
||||
|
||||
fn protocol<'a>(
|
||||
&self,
|
||||
protocols: &'a mut HashMap<String, Protocol>,
|
||||
name: &str,
|
||||
) -> &'a mut Protocol {
|
||||
protocols
|
||||
.entry(name.to_owned())
|
||||
.or_insert_with(|| Protocol {
|
||||
plane: Arc::new(Plane {
|
||||
local: self.local,
|
||||
table: ArcSwap::from_pointee(ForwardingTable::default()),
|
||||
tally: self.tally.clone(),
|
||||
}),
|
||||
graph: HashMap::new(),
|
||||
members: HashSet::new(),
|
||||
raw: HashMap::new(),
|
||||
readers: HashMap::new(),
|
||||
links: HashMap::new(),
|
||||
})
|
||||
/// Sets which kinds of transport are preferred, best first, when a
|
||||
/// neighbour can be reached over more than one.
|
||||
pub fn set_priority(&self, kinds: Vec<String>) {
|
||||
let mut state = self.state();
|
||||
if state.priority != kinds {
|
||||
state.priority = kinds;
|
||||
state.publish();
|
||||
}
|
||||
}
|
||||
|
||||
/// Wakes the control loop promptly when a transport reader ends.
|
||||
@@ -282,14 +361,11 @@ impl RelayHub {
|
||||
}
|
||||
|
||||
/// Publishes authenticated topology. Refreshing identical state is cheap.
|
||||
pub fn set_topology(
|
||||
&self,
|
||||
protocol: &str,
|
||||
graph: HashMap<PeerId, Vec<PeerId>>,
|
||||
members: HashSet<PeerId>,
|
||||
) {
|
||||
let mut protocols = self.protocols();
|
||||
let state = self.protocol(&mut protocols, protocol);
|
||||
///
|
||||
/// Every row of `graph` is one member's own account of who it has a link
|
||||
/// to; what kind of link it is does not matter here.
|
||||
pub fn set_topology(&self, graph: HashMap<PeerId, Vec<PeerId>>, members: HashSet<PeerId>) {
|
||||
let mut state = self.state();
|
||||
if state.graph != graph || state.members != members {
|
||||
state.graph = graph;
|
||||
state.members = members;
|
||||
@@ -298,22 +374,44 @@ impl RelayHub {
|
||||
}
|
||||
|
||||
/// Whether the current table has a path to this destination.
|
||||
pub fn reachable(&self, peer: EndpointId, protocol: &str) -> bool {
|
||||
self.protocols().get(protocol).is_some_and(|state| {
|
||||
state
|
||||
.plane
|
||||
.table
|
||||
.load()
|
||||
.routes
|
||||
.contains_key(peer.as_bytes())
|
||||
pub fn reachable(&self, peer: EndpointId) -> bool {
|
||||
self.state()
|
||||
.plane
|
||||
.table
|
||||
.load()
|
||||
.routes
|
||||
.contains_key(peer.as_bytes())
|
||||
}
|
||||
|
||||
/// How a destination is reached right now, if it is.
|
||||
pub fn route(&self, peer: EndpointId) -> Option<RouteSummary> {
|
||||
let table = self.state().plane.table.load_full();
|
||||
let route = table.routes.get(peer.as_bytes())?;
|
||||
let via = EndpointId::from_bytes(&route.next.first()?.peer).ok()?;
|
||||
Some(RouteSummary {
|
||||
hops: route.hops,
|
||||
via,
|
||||
})
|
||||
}
|
||||
|
||||
/// Stable logical link, preserved across topology changes.
|
||||
/// The kinds of transport with an open direct link to a peer, best
|
||||
/// first.
|
||||
pub fn direct_kinds(&self, peer: EndpointId) -> Vec<String> {
|
||||
let state = self.state();
|
||||
state
|
||||
.ordered(peer.as_bytes())
|
||||
.into_iter()
|
||||
.filter(|raw| !raw.link.is_closed())
|
||||
.map(|raw| raw.kind.clone())
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Stable end-to-end channel for a protocol, preserved across topology
|
||||
/// and transport changes.
|
||||
pub fn link(&self, peer: EndpointId, protocol: &str) -> Arc<PeerLink> {
|
||||
let mut protocols = self.protocols();
|
||||
let state = self.protocol(&mut protocols, protocol);
|
||||
if let Some(link) = state.links.get(peer.as_bytes()) {
|
||||
let mut state = self.state();
|
||||
let key = (*peer.as_bytes(), protocol.to_owned());
|
||||
if let Some(link) = state.links.get(&key) {
|
||||
return link.clone();
|
||||
}
|
||||
let (inbox_tx, inbox_rx) = mpsc::channel(INBOX);
|
||||
@@ -321,88 +419,118 @@ impl RelayHub {
|
||||
let link = Arc::new(PeerLink {
|
||||
network: self.network,
|
||||
peer,
|
||||
tag: envelope::protocol_tag(protocol),
|
||||
plane: state.plane.clone(),
|
||||
inbox_tx,
|
||||
inbox_rx: tokio::sync::Mutex::new(inbox_rx),
|
||||
closed,
|
||||
});
|
||||
state.links.insert(*peer.as_bytes(), link.clone());
|
||||
state.links.insert(key, link.clone());
|
||||
state.publish();
|
||||
link
|
||||
}
|
||||
|
||||
/// Whether the protocol already owns its logical link.
|
||||
/// Whether the protocol already owns its end-to-end channel to a peer.
|
||||
pub fn has_link(&self, peer: EndpointId, protocol: &str) -> bool {
|
||||
self.protocols()
|
||||
.get(protocol)
|
||||
.is_some_and(|state| state.links.contains_key(peer.as_bytes()))
|
||||
self.state()
|
||||
.links
|
||||
.contains_key(&(*peer.as_bytes(), protocol.to_owned()))
|
||||
}
|
||||
|
||||
/// Installs a raw transport link and starts its independent ingress reader.
|
||||
pub fn set_direct(&self, peer: EndpointId, protocol: &str, raw: SharedLink) {
|
||||
///
|
||||
/// A neighbour may have one link per kind of transport; installing a
|
||||
/// second of the same kind replaces the first.
|
||||
pub fn set_direct(&self, peer: EndpointId, kind: &str, raw: SharedLink) {
|
||||
if raw.network() != self.network || raw.peer() != peer {
|
||||
return;
|
||||
}
|
||||
let mut protocols = self.protocols();
|
||||
let state = self.protocol(&mut protocols, protocol);
|
||||
let mut state = self.state();
|
||||
let id = *peer.as_bytes();
|
||||
if let Some(old) = state.readers.remove(&id) {
|
||||
old.abort();
|
||||
}
|
||||
state.raw.insert(id, raw.clone());
|
||||
state.publish();
|
||||
let plane = state.plane.clone();
|
||||
let changed = self.changed.clone();
|
||||
state.readers.insert(
|
||||
id,
|
||||
tokio::spawn(async move {
|
||||
let mut batch = 0;
|
||||
while let Some(frame) = raw.recv().await {
|
||||
plane.receive(id, frame);
|
||||
batch += 1;
|
||||
if batch == 64 {
|
||||
tokio::task::yield_now().await;
|
||||
batch = 0;
|
||||
}
|
||||
let reader_link = raw.clone();
|
||||
let reader = tokio::spawn(async move {
|
||||
let mut batch = 0;
|
||||
while let Some(frame) = reader_link.recv().await {
|
||||
plane.receive(id, frame);
|
||||
batch += 1;
|
||||
if batch == 64 {
|
||||
tokio::task::yield_now().await;
|
||||
batch = 0;
|
||||
}
|
||||
changed.notify_one();
|
||||
}),
|
||||
);
|
||||
}
|
||||
changed.notify_one();
|
||||
});
|
||||
let links = state.raw.entry(id).or_default();
|
||||
if let Some(position) = links.iter().position(|known| known.kind == kind) {
|
||||
links.remove(position).reader.abort();
|
||||
}
|
||||
links.push(RawLink {
|
||||
kind: kind.to_owned(),
|
||||
link: raw,
|
||||
reader,
|
||||
});
|
||||
state.publish();
|
||||
}
|
||||
|
||||
/// Removes a physical link without tearing down end-to-end protocol state.
|
||||
pub fn clear_direct(&self, peer: EndpointId, protocol: &str) {
|
||||
if let Some(state) = self.protocols().get_mut(protocol) {
|
||||
state.raw.remove(peer.as_bytes());
|
||||
if let Some(task) = state.readers.remove(peer.as_bytes()) {
|
||||
task.abort();
|
||||
/// Removes one physical link without tearing down end-to-end protocol state.
|
||||
pub fn clear_direct(&self, peer: EndpointId, kind: &str) {
|
||||
let mut state = self.state();
|
||||
let mut changed = false;
|
||||
if let Some(links) = state.raw.get_mut(peer.as_bytes()) {
|
||||
if let Some(position) = links.iter().position(|known| known.kind == kind) {
|
||||
links.remove(position).reader.abort();
|
||||
changed = true;
|
||||
}
|
||||
if links.is_empty() {
|
||||
state.raw.remove(peer.as_bytes());
|
||||
}
|
||||
}
|
||||
if changed {
|
||||
state.publish();
|
||||
}
|
||||
}
|
||||
|
||||
/// Revokes all links and topology involving a departed authenticated peer.
|
||||
pub fn remove_peer(&self, peer: EndpointId) {
|
||||
for state in self.protocols().values_mut() {
|
||||
state.raw.remove(peer.as_bytes());
|
||||
if let Some(task) = state.readers.remove(peer.as_bytes()) {
|
||||
task.abort();
|
||||
let mut state = self.state();
|
||||
if let Some(links) = state.raw.remove(peer.as_bytes()) {
|
||||
for raw in links {
|
||||
raw.reader.abort();
|
||||
}
|
||||
if let Some(link) = state.links.remove(peer.as_bytes()) {
|
||||
}
|
||||
let gone: Vec<_> = state
|
||||
.links
|
||||
.keys()
|
||||
.filter(|(id, _)| id == peer.as_bytes())
|
||||
.cloned()
|
||||
.collect();
|
||||
for key in gone {
|
||||
if let Some(link) = state.links.remove(&key) {
|
||||
link.close();
|
||||
}
|
||||
state.members.remove(peer.as_bytes());
|
||||
state.graph.remove(peer.as_bytes());
|
||||
for neighbors in state.graph.values_mut() {
|
||||
neighbors.retain(|id| id != peer.as_bytes());
|
||||
}
|
||||
state.publish();
|
||||
}
|
||||
state.members.remove(peer.as_bytes());
|
||||
state.graph.remove(peer.as_bytes());
|
||||
for neighbors in state.graph.values_mut() {
|
||||
neighbors.retain(|id| id != peer.as_bytes());
|
||||
}
|
||||
state.publish();
|
||||
}
|
||||
|
||||
/// Stops every reader and releases all transport handles.
|
||||
pub fn close(&self) {
|
||||
self.protocols().clear();
|
||||
let mut state = self.state();
|
||||
state.stop_readers();
|
||||
state.raw.clear();
|
||||
for link in state.links.values() {
|
||||
link.close();
|
||||
}
|
||||
state.links.clear();
|
||||
state.graph.clear();
|
||||
state.members.clear();
|
||||
state.plane.table.store(Arc::default());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -411,6 +539,7 @@ impl RelayHub {
|
||||
pub struct PeerLink {
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
tag: u64,
|
||||
plane: Arc<Plane>,
|
||||
inbox_tx: mpsc::Sender<Bytes>,
|
||||
inbox_rx: tokio::sync::Mutex<mpsc::Receiver<Bytes>>,
|
||||
@@ -454,10 +583,11 @@ impl PacketLink for PeerLink {
|
||||
let hop = route
|
||||
.select(&self.plane.local, flow)
|
||||
.ok_or(TransportError::Closed)?;
|
||||
hop.link.send(envelope::encode(
|
||||
hop.send(envelope::encode(
|
||||
self.plane.local,
|
||||
*self.peer.as_bytes(),
|
||||
flow,
|
||||
self.tag,
|
||||
&payload,
|
||||
))?;
|
||||
if route.hops > 1 {
|
||||
@@ -489,7 +619,9 @@ impl PacketLink for PeerLink {
|
||||
fn path_description(&self) -> String {
|
||||
let table = self.plane.table.load();
|
||||
match table.routes.get(self.peer.as_bytes()) {
|
||||
Some(route) if route.hops == 1 => route.next[0].link.path_description(),
|
||||
Some(route) if route.hops == 1 => route.next[0]
|
||||
.first_open()
|
||||
.map_or_else(|| "unreachable".into(), |link| link.path_description()),
|
||||
Some(route) => format!(
|
||||
"relay {} hops via {} ({} equal paths)",
|
||||
route.hops,
|
||||
|
||||
@@ -16,6 +16,9 @@ fn peer(seed: u8) -> EndpointId {
|
||||
fn id(seed: u8) -> PeerId {
|
||||
*peer(seed).as_bytes()
|
||||
}
|
||||
fn tag() -> u64 {
|
||||
envelope::protocol_tag("ip")
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct Wire {
|
||||
@@ -78,9 +81,8 @@ impl PacketLink for Wire {
|
||||
}
|
||||
}
|
||||
|
||||
fn topology(hub: &RelayHub, protocol: &str, edges: &[(u8, &[u8])], members: &[u8]) {
|
||||
fn topology(hub: &RelayHub, edges: &[(u8, &[u8])], members: &[u8]) {
|
||||
hub.set_topology(
|
||||
protocol,
|
||||
edges
|
||||
.iter()
|
||||
.map(|(a, bs)| (id(*a), bs.iter().map(|b| id(*b)).collect()))
|
||||
@@ -96,7 +98,7 @@ async fn ecmp_is_per_flow_direct_wins_and_existing_tunnel_survives_changes() {
|
||||
let c = Wire::new(3, true);
|
||||
hub.set_direct(peer(2), "ip", b.clone());
|
||||
hub.set_direct(peer(3), "ip", c.clone());
|
||||
topology(&hub, "ip", &[(2, &[4]), (3, &[4])], &[1, 2, 3, 4]);
|
||||
topology(&hub, &[(2, &[4]), (3, &[4])], &[1, 2, 3, 4]);
|
||||
let link = hub.link(peer(4), "ip");
|
||||
let mut chosen = HashMap::new();
|
||||
for _ in 0..8 {
|
||||
@@ -132,7 +134,7 @@ async fn ecmp_is_per_flow_direct_wins_and_existing_tunnel_survives_changes() {
|
||||
}
|
||||
assert_eq!(c.take().len(), 100);
|
||||
hub.clear_direct(peer(2), "ip");
|
||||
topology(&hub, "ip", &[(3, &[])], &[1, 2, 3, 4]);
|
||||
topology(&hub, &[(3, &[])], &[1, 2, 3, 4]);
|
||||
assert!(link.send(Bytes::new()).is_err());
|
||||
assert!(!link.is_closed());
|
||||
assert!(Arc::ptr_eq(&link, &hub.link(peer(4), "ip")));
|
||||
@@ -152,8 +154,8 @@ async fn transport_reader_forwards_without_any_plugin_reader_and_isolates_protoc
|
||||
let c = Wire::new(3, true);
|
||||
hub.set_direct(peer(1), "ip", a.clone());
|
||||
hub.set_direct(peer(3), "ip", c.clone());
|
||||
topology(&hub, "ip", &[(3, &[4])], &[1, 2, 3, 4]);
|
||||
let frame = envelope::encode(id(1), id(4), 91, b"end-to-end encrypted");
|
||||
topology(&hub, &[(3, &[4])], &[1, 2, 3, 4]);
|
||||
let frame = envelope::encode(id(1), id(4), 91, tag(), b"end-to-end encrypted");
|
||||
let pointer = frame.as_ptr();
|
||||
a.feed.send(frame).await.unwrap();
|
||||
tokio::time::timeout(std::time::Duration::from_secs(1), async {
|
||||
@@ -170,12 +172,136 @@ async fn transport_reader_forwards_without_any_plugin_reader_and_isolates_protoc
|
||||
ROUTING_HOP_LIMIT - 1
|
||||
);
|
||||
assert_eq!(&sent[0][RELAY_OVERHEAD..], b"end-to-end encrypted");
|
||||
let isolated = hub.link(peer(4), "other-ip");
|
||||
assert!(isolated.send(Bytes::new()).is_err());
|
||||
assert_eq!(hub.counters().forwarded, 1);
|
||||
hub.close();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_frame_is_delivered_only_to_the_protocol_it_names() {
|
||||
let hub = RelayHub::new(network(), peer(2));
|
||||
hub.set_direct(peer(1), "ip", Wire::new(1, true));
|
||||
let ip = hub.link(peer(1), "ip");
|
||||
let other = hub.link(peer(1), "other-ip");
|
||||
let plane = ip.plane.clone();
|
||||
plane.receive(id(1), envelope::encode(id(1), id(2), 0, tag(), b"for ip"));
|
||||
plane.receive(
|
||||
id(1),
|
||||
envelope::encode(
|
||||
id(1),
|
||||
id(2),
|
||||
0,
|
||||
envelope::protocol_tag("other-ip"),
|
||||
b"for other",
|
||||
),
|
||||
);
|
||||
plane.receive(
|
||||
id(1),
|
||||
envelope::encode(
|
||||
id(1),
|
||||
id(2),
|
||||
0,
|
||||
envelope::protocol_tag("nobody-runs-this"),
|
||||
b"for nobody",
|
||||
),
|
||||
);
|
||||
assert_eq!(ip.recv().await.unwrap(), Bytes::from_static(b"for ip"));
|
||||
assert_eq!(
|
||||
other.recv().await.unwrap(),
|
||||
Bytes::from_static(b"for other")
|
||||
);
|
||||
assert_eq!(hub.counters().dropped_unknown, 1);
|
||||
hub.close();
|
||||
}
|
||||
|
||||
/// A middle member joins two neighbours over different transports, and the
|
||||
/// two ends share a protocol the middle has never heard of.
|
||||
#[tokio::test]
|
||||
async fn a_tunnel_crosses_a_middle_member_whose_links_are_of_different_kinds() {
|
||||
let a = RelayHub::new(network(), peer(1));
|
||||
let c = RelayHub::new(network(), peer(3));
|
||||
let b = RelayHub::new(network(), peer(2));
|
||||
let a_to_c = Wire::new(3, true);
|
||||
let c_to_a = Wire::new(1, true);
|
||||
let c_to_b = Wire::new(2, true);
|
||||
let b_to_c = Wire::new(3, true);
|
||||
a.set_direct(peer(3), "wg", a_to_c.clone());
|
||||
c.set_direct(peer(1), "wg", c_to_a);
|
||||
c.set_direct(peer(2), "wg-quic", c_to_b.clone());
|
||||
b.set_direct(peer(3), "wg-quic", b_to_c.clone());
|
||||
topology(&a, &[(3, &[1, 2])], &[1, 2, 3]);
|
||||
topology(&b, &[(3, &[1, 2])], &[1, 2, 3]);
|
||||
topology(&c, &[(1, &[3]), (2, &[3])], &[1, 2, 3]);
|
||||
|
||||
let outgoing = a.link(peer(2), "tunnel");
|
||||
let incoming = b.link(peer(1), "tunnel");
|
||||
outgoing
|
||||
.send_flow(Bytes::from_static(b"sealed"), 5)
|
||||
.unwrap();
|
||||
let frame = a_to_c.take().remove(0);
|
||||
|
||||
// The middle forwards it onto its other kind of link without looking
|
||||
// inside.
|
||||
c.state().plane.receive(id(1), frame);
|
||||
let frame = c_to_b.take().remove(0);
|
||||
assert_eq!(c.counters().forwarded, 1);
|
||||
|
||||
b.state().plane.receive(id(3), frame);
|
||||
assert_eq!(
|
||||
incoming.recv().await.unwrap(),
|
||||
Bytes::from_static(b"sealed")
|
||||
);
|
||||
assert_eq!(b.counters().received_via, 1);
|
||||
assert_eq!(
|
||||
b.route(peer(1)),
|
||||
Some(RouteSummary {
|
||||
hops: 2,
|
||||
via: peer(3)
|
||||
})
|
||||
);
|
||||
a.close();
|
||||
b.close();
|
||||
c.close();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_neighbour_is_reached_over_the_preferred_kind_and_falls_back_when_it_is_lost() {
|
||||
let hub = RelayHub::new(network(), peer(1));
|
||||
hub.set_priority(vec!["wg".into(), "tcp-tls".into(), "wg-quic".into()]);
|
||||
let slow = Wire::new(2, true);
|
||||
let fast = Wire::new(2, true);
|
||||
hub.set_direct(peer(2), "wg-quic", slow.clone());
|
||||
let link = hub.link(peer(2), "tunnel");
|
||||
link.send(Bytes::from_static(b"one")).unwrap();
|
||||
assert_eq!(slow.take().len(), 1);
|
||||
assert_eq!(hub.direct_kinds(peer(2)), ["wg-quic"]);
|
||||
|
||||
// A better kind appears: traffic moves to it with nothing to redo.
|
||||
hub.set_direct(peer(2), "wg", fast.clone());
|
||||
assert_eq!(hub.direct_kinds(peer(2)), ["wg", "wg-quic"]);
|
||||
link.send(Bytes::from_static(b"two")).unwrap();
|
||||
assert_eq!(fast.take().len(), 1);
|
||||
assert!(slow.take().is_empty());
|
||||
|
||||
// It dies: the worse one carries on, and the logical link is untouched.
|
||||
fast.dead.store(true, Ordering::Relaxed);
|
||||
link.send(Bytes::from_static(b"three")).unwrap();
|
||||
assert_eq!(slow.take().len(), 1);
|
||||
assert!(!link.is_closed());
|
||||
assert_eq!(
|
||||
hub.route(peer(2)),
|
||||
Some(RouteSummary {
|
||||
hops: 1,
|
||||
via: peer(2)
|
||||
})
|
||||
);
|
||||
|
||||
hub.clear_direct(peer(2), "wg-quic");
|
||||
hub.clear_direct(peer(2), "wg");
|
||||
assert!(!hub.reachable(peer(2)));
|
||||
assert!(Arc::ptr_eq(&link, &hub.link(peer(2), "tunnel")));
|
||||
hub.close();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn local_delivery_is_bounded_and_malformed_or_unknown_sources_are_dropped() {
|
||||
let hub = RelayHub::new(network(), peer(2));
|
||||
@@ -184,18 +310,21 @@ async fn local_delivery_is_bounded_and_malformed_or_unknown_sources_are_dropped(
|
||||
let link = hub.link(peer(1), "ip");
|
||||
let plane = link.plane.clone();
|
||||
for _ in 0..INBOX + 1 {
|
||||
plane.receive(id(1), envelope::encode(id(1), id(2), 0, b"ip"));
|
||||
plane.receive(id(1), envelope::encode(id(1), id(2), 0, tag(), b"ip"));
|
||||
}
|
||||
assert_eq!(hub.counters().dropped_congested, 1);
|
||||
for _ in 0..INBOX {
|
||||
assert_eq!(link.recv().await.unwrap(), Bytes::from_static(b"ip"));
|
||||
}
|
||||
plane.receive(id(1), Bytes::from_static(b"broken"));
|
||||
plane.receive(id(1), envelope::encode(id(9), id(2), 0, b"unknown member"));
|
||||
plane.receive(
|
||||
id(1),
|
||||
envelope::encode(id(9), id(2), 0, tag(), b"unknown member"),
|
||||
);
|
||||
assert_eq!(hub.counters().dropped_unknown, 2);
|
||||
plane.receive(
|
||||
id(1),
|
||||
envelope::encode(id(1), id(9), 0, b"unknown destination"),
|
||||
envelope::encode(id(1), id(9), 0, tag(), b"unknown destination"),
|
||||
);
|
||||
assert_eq!(hub.counters().dropped_no_link, 1);
|
||||
hub.close();
|
||||
@@ -210,11 +339,11 @@ async fn inconsistent_tables_cannot_loop_beyond_the_hop_limit() {
|
||||
a.set_direct(peer(2), "ip", ab.clone());
|
||||
b.set_direct(peer(1), "ip", ba.clone());
|
||||
// Deliberately inconsistent snapshots while updates are in flight.
|
||||
topology(&a, "ip", &[(2, &[4])], &[1, 2, 3, 4]);
|
||||
topology(&b, "ip", &[(1, &[4])], &[1, 2, 3, 4]);
|
||||
topology(&a, &[(2, &[4])], &[1, 2, 3, 4]);
|
||||
topology(&b, &[(1, &[4])], &[1, 2, 3, 4]);
|
||||
let ap = a.link(peer(4), "ip").plane.clone();
|
||||
let bp = b.link(peer(4), "ip").plane.clone();
|
||||
let mut frame = envelope::encode(id(3), id(4), 7, b"transit");
|
||||
let mut frame = envelope::encode(id(3), id(4), 7, tag(), b"transit");
|
||||
for turn in 0..ROUTING_HOP_LIMIT {
|
||||
let (plane, incoming, wire) = if turn % 2 == 0 {
|
||||
(&ap, id(2), &ab)
|
||||
@@ -253,7 +382,7 @@ async fn forwarding_benchmark() {
|
||||
hub.set_direct(peer(3), "ip", wire.clone());
|
||||
let alternate = Wire::new(5, false);
|
||||
hub.set_direct(peer(5), "ip", alternate.clone());
|
||||
topology(&hub, "ip", &[(3, &[4]), (5, &[4])], &[1, 2, 3, 4, 5]);
|
||||
topology(&hub, &[(3, &[4]), (5, &[4])], &[1, 2, 3, 4, 5]);
|
||||
let plane = hub.link(peer(4), "ip").plane.clone();
|
||||
let source = id(1);
|
||||
let destination = id(4);
|
||||
@@ -264,7 +393,9 @@ async fn forwarding_benchmark() {
|
||||
const BATCHES: usize = 100;
|
||||
for _ in 0..BATCHES {
|
||||
let frames: Vec<_> = (0..COUNT)
|
||||
.map(|flow| envelope::encode(source, destination, flow as u64, &vec![42; size]))
|
||||
.map(|flow| {
|
||||
envelope::encode(source, destination, flow as u64, tag(), &vec![42; size])
|
||||
})
|
||||
.collect();
|
||||
let start = std::time::Instant::now();
|
||||
for frame in frames {
|
||||
|
||||
@@ -3,8 +3,8 @@ use super::{FlowId, PeerId};
|
||||
use crate::config::{MAX_DATA_DATAGRAM, ROUTING_HOP_LIMIT};
|
||||
use bytes::{BufMut, Bytes, BytesMut};
|
||||
|
||||
pub(crate) const HEADER: usize = 2 + 32 + 32 + 8;
|
||||
const VERSION: u8 = 1;
|
||||
pub(crate) const HEADER: usize = 2 + 32 + 32 + 8 + 8;
|
||||
const VERSION: u8 = 2;
|
||||
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub(crate) struct Header {
|
||||
@@ -12,6 +12,15 @@ pub(crate) struct Header {
|
||||
pub destination: PeerId,
|
||||
pub remaining: u8,
|
||||
pub flow: FlowId,
|
||||
/// Which end-to-end protocol the payload belongs to, whatever links carry it.
|
||||
pub protocol: u64,
|
||||
}
|
||||
|
||||
/// A stable 64-bit tag for a protocol id (FNV-1a), the same on every member.
|
||||
pub(crate) fn protocol_tag(name: &str) -> u64 {
|
||||
name.bytes().fold(0xcbf2_9ce4_8422_2325, |hash, byte| {
|
||||
(hash ^ u64::from(byte)).wrapping_mul(0x0000_0100_0000_01b3)
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn decode(bytes: &[u8]) -> Option<Header> {
|
||||
@@ -28,16 +37,24 @@ pub(crate) fn decode(bytes: &[u8]) -> Option<Header> {
|
||||
destination: bytes[34..66].try_into().ok()?,
|
||||
remaining: bytes[1],
|
||||
flow: u64::from_be_bytes(bytes[66..74].try_into().ok()?),
|
||||
protocol: u64::from_be_bytes(bytes[74..82].try_into().ok()?),
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn encode(source: PeerId, destination: PeerId, flow: FlowId, payload: &[u8]) -> Bytes {
|
||||
pub(crate) fn encode(
|
||||
source: PeerId,
|
||||
destination: PeerId,
|
||||
flow: FlowId,
|
||||
protocol: u64,
|
||||
payload: &[u8],
|
||||
) -> Bytes {
|
||||
let mut bytes = BytesMut::with_capacity(HEADER + payload.len());
|
||||
bytes.put_u8(VERSION);
|
||||
bytes.put_u8(ROUTING_HOP_LIMIT);
|
||||
bytes.extend_from_slice(&source);
|
||||
bytes.extend_from_slice(&destination);
|
||||
bytes.put_u64(flow);
|
||||
bytes.put_u64(protocol);
|
||||
bytes.extend_from_slice(payload);
|
||||
bytes.freeze()
|
||||
}
|
||||
@@ -58,7 +75,7 @@ mod tests {
|
||||
use super::*;
|
||||
#[test]
|
||||
fn transit_reuses_the_buffer_and_rejects_invalid_headers() {
|
||||
let frame = encode([1; 32], [2; 32], 17, &[42; 1280]);
|
||||
let frame = encode([1; 32], [2; 32], 17, protocol_tag("wg"), &[42; 1280]);
|
||||
let address = frame.as_ptr();
|
||||
let frame = decrement(frame);
|
||||
assert_eq!(
|
||||
@@ -66,7 +83,10 @@ mod tests {
|
||||
address,
|
||||
"no payload copy for an owned buffer"
|
||||
);
|
||||
assert_eq!(decode(&frame).unwrap().remaining, ROUTING_HOP_LIMIT - 1);
|
||||
let header = decode(&frame).unwrap();
|
||||
assert_eq!(header.remaining, ROUTING_HOP_LIMIT - 1);
|
||||
assert_eq!(header.protocol, protocol_tag("wg"));
|
||||
assert_ne!(protocol_tag("wg"), protocol_tag("wg-quic"));
|
||||
assert_eq!(&frame[HEADER..], &[42; 1280]);
|
||||
for length in 0..HEADER {
|
||||
assert!(decode(&frame[..length]).is_none());
|
||||
|
||||
@@ -484,11 +484,43 @@ pub struct NetworkReport {
|
||||
pub range_conflict: Option<String>,
|
||||
/// The overlay, when a protocol is running one.
|
||||
pub overlay: Option<OverlayReport>,
|
||||
/// This agent's own name, when another member claimed it first.
|
||||
pub hostname_conflict: Option<HostnameConflictReport>,
|
||||
/// Exit-node settings: whether this agent offers one, and whether it
|
||||
/// uses one.
|
||||
pub exit: ExitReport,
|
||||
}
|
||||
|
||||
/// How packets reach a member right now.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum DataPathReport {
|
||||
/// Nothing carries packets to it: it is away, or only connected for
|
||||
/// control while a data path is being set up.
|
||||
#[default]
|
||||
None,
|
||||
/// A direct link.
|
||||
Direct {
|
||||
/// The kind of transport carrying it, e.g. `wg`.
|
||||
transport: String,
|
||||
},
|
||||
/// Through other members.
|
||||
Relayed {
|
||||
/// Number of links on the way, at least two.
|
||||
hops: u8,
|
||||
/// The member the first of them goes to, as an endpoint id.
|
||||
via: String,
|
||||
},
|
||||
}
|
||||
|
||||
/// A name somebody else claimed first.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct HostnameConflictReport {
|
||||
/// The name that is taken.
|
||||
pub name: String,
|
||||
/// The member that holds it, as an endpoint id.
|
||||
pub holder: String,
|
||||
}
|
||||
|
||||
/// One member of the network, from signed state.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct MemberReport {
|
||||
@@ -500,6 +532,15 @@ pub struct MemberReport {
|
||||
pub failed_dials: u32,
|
||||
/// The last hostname this agent saw it announce, kept while it is away.
|
||||
pub hostname: Option<String>,
|
||||
/// Whether this agent can reach it now, by any path. The one answer
|
||||
/// every client gives to "is it online": a member reached through
|
||||
/// others is online, over a longer path.
|
||||
pub online: bool,
|
||||
/// How packets reach it.
|
||||
pub path: DataPathReport,
|
||||
/// The name it claimed, when somebody else claimed it first. It still
|
||||
/// works by address; it is not found by that name.
|
||||
pub hostname_conflict: Option<HostnameConflictReport>,
|
||||
}
|
||||
|
||||
/// One control plane peer.
|
||||
@@ -516,6 +557,8 @@ pub struct PeerReport {
|
||||
pub rtt_ms: Option<u64>,
|
||||
/// The peer offers itself as an exit node in this network.
|
||||
pub exit_node: bool,
|
||||
/// How packets reach it.
|
||||
pub path: DataPathReport,
|
||||
}
|
||||
|
||||
/// The WireGuard overlay of one network.
|
||||
@@ -779,7 +822,7 @@ pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
///
|
||||
/// Bump it whenever [`Request`], [`Response`] or anything they contain
|
||||
/// changes shape.
|
||||
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 17]);
|
||||
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 18]);
|
||||
|
||||
/// Reads one request off an accepted stream, answers it, writes the response.
|
||||
///
|
||||
|
||||
@@ -20,7 +20,7 @@ use crate::error::ProtocolError;
|
||||
/// The version in the ALPN is the wire-compatibility version of the control
|
||||
/// protocol. It is independent of the network identity scheme version, so
|
||||
/// bumping it must not change any existing [`crate::NetworkId`].
|
||||
pub const ALPN: &[u8] = b"tsunagi/ctrl/4";
|
||||
pub const ALPN: &[u8] = b"tsunagi/ctrl/5";
|
||||
|
||||
/// ALPN of the tsunagi data plane.
|
||||
///
|
||||
@@ -30,9 +30,11 @@ pub const ALPN: &[u8] = b"tsunagi/ctrl/4";
|
||||
/// transport underneath can be replaced without touching the control protocol.
|
||||
///
|
||||
/// Version 4 adds source, destination, hop limit and stable flow id above
|
||||
/// transport fragmentation. All members must upgrade together. Persistent
|
||||
/// identities and network configuration do not change.
|
||||
pub const DATA_ALPN: &[u8] = b"tsunagi/data/4";
|
||||
/// transport fragmentation. Version 5 adds the end-to-end protocol to that
|
||||
/// envelope, so a tunnel no longer depends on the kind of link under it, and
|
||||
/// a first-claimed time to a signed hostname claim. All members must upgrade
|
||||
/// together. Persistent identities and network configuration do not change.
|
||||
pub const DATA_ALPN: &[u8] = b"tsunagi/data/5";
|
||||
|
||||
/// Largest plugin protocol identifier accepted when opening a data channel.
|
||||
pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
|
||||
@@ -41,7 +43,7 @@ pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
|
||||
pub const MAX_SIGNATURE_LEN: usize = 64;
|
||||
|
||||
/// Control protocol version carried inside the handshake.
|
||||
pub const PROTOCOL_VERSION: u16 = 4;
|
||||
pub const PROTOCOL_VERSION: u16 = 5;
|
||||
|
||||
/// First message of the handshake, sent by the initiator.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
|
||||
@@ -150,7 +150,7 @@ pub fn derived_ipv4_range(network: NetworkId) -> Ipv4Range {
|
||||
}
|
||||
|
||||
/// Frozen domain separator for the bytes a record signature covers.
|
||||
pub const RECORD_DOMAIN: &str = "tsunagi-signed-record-v2";
|
||||
pub const RECORD_DOMAIN: &str = "tsunagi-signed-record-v3";
|
||||
|
||||
/// Longest hostname a record may carry.
|
||||
///
|
||||
@@ -207,6 +207,18 @@ pub enum RecordBody {
|
||||
range: Option<Ipv4Range>,
|
||||
/// The name this author answers to.
|
||||
hostname: Option<String>,
|
||||
/// When this author first claimed that name, in milliseconds since
|
||||
/// the Unix epoch, kept across every later version that keeps the
|
||||
/// name. It is what makes a name belong to whoever claimed it first
|
||||
/// rather than to whoever happens to sort lowest, so a member that
|
||||
/// arrives later with the same name cannot take it over.
|
||||
///
|
||||
/// The author's own word: a member with a wrong clock can claim an
|
||||
/// earlier time. That costs it nothing it could not do by other
|
||||
/// means, since anybody who knows the network secret is a member,
|
||||
/// and it cannot take an address or reach anything it was not
|
||||
/// already entitled to.
|
||||
hostname_since: Option<u64>,
|
||||
},
|
||||
/// This author gives up everything it claimed.
|
||||
///
|
||||
@@ -243,6 +255,14 @@ impl RecordBody {
|
||||
}
|
||||
}
|
||||
|
||||
/// When the author first claimed the hostname, if it has one.
|
||||
pub fn hostname_since(&self) -> Option<u64> {
|
||||
match self {
|
||||
RecordBody::Claim { hostname_since, .. } => *hostname_since,
|
||||
RecordBody::Release => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn canonical(&self) -> Vec<u8> {
|
||||
let mut out = Vec::with_capacity(96);
|
||||
match self {
|
||||
@@ -250,6 +270,7 @@ impl RecordBody {
|
||||
address,
|
||||
range,
|
||||
hostname,
|
||||
hostname_since,
|
||||
} => {
|
||||
push_lp(&mut out, b"claim");
|
||||
push_opt(
|
||||
@@ -270,6 +291,13 @@ impl RecordBody {
|
||||
.as_deref(),
|
||||
);
|
||||
push_opt(&mut out, hostname.as_deref().map(str::as_bytes));
|
||||
push_opt(
|
||||
&mut out,
|
||||
hostname_since
|
||||
.map(|since| since.to_be_bytes())
|
||||
.as_ref()
|
||||
.map(|bytes| &bytes[..]),
|
||||
);
|
||||
}
|
||||
RecordBody::Release => {
|
||||
push_lp(&mut out, b"release");
|
||||
@@ -391,8 +419,12 @@ impl SignedRecord {
|
||||
address,
|
||||
range,
|
||||
hostname,
|
||||
hostname_since,
|
||||
} = &self.body
|
||||
{
|
||||
if hostname.is_none() && hostname_since.is_some() {
|
||||
return Err(StateError::Malformed("a time was claimed for no hostname"));
|
||||
}
|
||||
// Bounds before anything is believed, because all of this came
|
||||
// off the network.
|
||||
if let Some(range) = range {
|
||||
@@ -585,25 +617,33 @@ impl StateSet {
|
||||
|
||||
/// Who currently holds each claimed hostname.
|
||||
///
|
||||
/// The same rule as addresses, for the same reason: a name is owned, two
|
||||
/// members may claim one, and every replica has to reach the same answer
|
||||
/// about who has it without asking anybody.
|
||||
/// A name belongs to the member that claimed it first, and the others
|
||||
/// that claim it keep working by address but are not found by that name.
|
||||
/// "First" is the time each author signed into its own claim, with the
|
||||
/// lower id deciding a tie and a claim with no time counting as last, so
|
||||
/// every replica reaches the same answer about who has it without
|
||||
/// asking anybody. It is a pure function of the records: nothing depends
|
||||
/// on the order they arrived in, so there is no race to lose.
|
||||
pub fn hostname_holders(&self) -> HashMap<String, EndpointId> {
|
||||
let mut holders: HashMap<String, EndpointId> = HashMap::new();
|
||||
let mut holders: HashMap<String, (u64, EndpointId)> = HashMap::new();
|
||||
for (author, record) in &self.records {
|
||||
let Some(hostname) = record.body.hostname() else {
|
||||
continue;
|
||||
};
|
||||
let claim = (record.body.hostname_since().unwrap_or(u64::MAX), *author);
|
||||
holders
|
||||
.entry(hostname.to_string())
|
||||
.and_modify(|held| {
|
||||
if author.as_bytes() < held.as_bytes() {
|
||||
*held = *author;
|
||||
if (claim.0, claim.1.as_bytes()) < (held.0, held.1.as_bytes()) {
|
||||
*held = claim;
|
||||
}
|
||||
})
|
||||
.or_insert(*author);
|
||||
.or_insert(claim);
|
||||
}
|
||||
holders
|
||||
.into_iter()
|
||||
.map(|(hostname, (_, author))| (hostname, author))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The hostname an author holds, if it holds one uncontested.
|
||||
@@ -612,6 +652,19 @@ impl StateSet {
|
||||
(self.hostname_holders().get(hostname) == Some(author)).then_some(hostname)
|
||||
}
|
||||
|
||||
/// The name an author claimed and the member that holds it instead, when
|
||||
/// somebody who claimed it earlier does.
|
||||
pub fn hostname_taken(&self, author: &EndpointId) -> Option<(&str, EndpointId)> {
|
||||
let hostname = self.records.get(author)?.body.hostname()?;
|
||||
let holder = *self.hostname_holders().get(hostname)?;
|
||||
(holder != *author).then_some((hostname, holder))
|
||||
}
|
||||
|
||||
/// When an author first claimed the name it holds now, if it has one.
|
||||
pub fn hostname_since_of(&self, author: &EndpointId) -> Option<u64> {
|
||||
self.records.get(author)?.body.hostname_since()
|
||||
}
|
||||
|
||||
/// The address an author holds, if it holds one uncontested.
|
||||
pub fn address_of(&self, author: &EndpointId) -> Option<Ipv4Addr> {
|
||||
let address = self.records.get(author)?.body.claimed_address()?;
|
||||
@@ -661,6 +714,7 @@ mod tests {
|
||||
address: Some(address.parse().unwrap()),
|
||||
range: Some(range()),
|
||||
hostname: None,
|
||||
hostname_since: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -720,6 +774,7 @@ mod tests {
|
||||
address: Some("10.99.0.1".parse().unwrap()),
|
||||
range: Some(range()),
|
||||
hostname: None,
|
||||
hostname_since: None,
|
||||
},
|
||||
);
|
||||
assert!(matches!(outside.verify(id), Err(StateError::Malformed(_))));
|
||||
@@ -735,6 +790,7 @@ mod tests {
|
||||
prefix_len: 31,
|
||||
}),
|
||||
hostname: None,
|
||||
hostname_since: None,
|
||||
},
|
||||
);
|
||||
assert!(matches!(no_hosts.verify(id), Err(StateError::Malformed(_))));
|
||||
@@ -776,6 +832,7 @@ mod tests {
|
||||
address: None,
|
||||
range: None,
|
||||
hostname: Some(bad.to_string()),
|
||||
hostname_since: None,
|
||||
},
|
||||
);
|
||||
assert!(
|
||||
@@ -797,6 +854,7 @@ mod tests {
|
||||
address: Some("10.13.37.5".parse().unwrap()),
|
||||
range: None,
|
||||
hostname: None,
|
||||
hostname_since: None,
|
||||
},
|
||||
);
|
||||
assert!(matches!(record.verify(id), Err(StateError::Malformed(_))));
|
||||
@@ -818,6 +876,7 @@ mod tests {
|
||||
address: None,
|
||||
range: None,
|
||||
hostname: Some("music".into()),
|
||||
hostname_since: None,
|
||||
},
|
||||
)
|
||||
};
|
||||
@@ -860,6 +919,7 @@ mod tests {
|
||||
address: None,
|
||||
range: None,
|
||||
hostname: Some(name.to_string()),
|
||||
hostname_since: None,
|
||||
},
|
||||
)
|
||||
};
|
||||
@@ -879,6 +939,109 @@ mod tests {
|
||||
assert_eq!(set.hostname_of(&secret.public()), Some("new"));
|
||||
}
|
||||
|
||||
fn named_at(
|
||||
secret: &SecretKey,
|
||||
id: NetworkId,
|
||||
version: u64,
|
||||
name: &str,
|
||||
since: u64,
|
||||
) -> SignedRecord {
|
||||
SignedRecord::sign(
|
||||
secret,
|
||||
id,
|
||||
version,
|
||||
RecordBody::Claim {
|
||||
address: None,
|
||||
range: None,
|
||||
hostname: Some(name.into()),
|
||||
hostname_since: Some(since),
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_name_belongs_to_whoever_claimed_it_first_whatever_their_ids_or_arrival_order() {
|
||||
let id = network("naming");
|
||||
let mut keys = [SecretKey::generate(), SecretKey::generate()];
|
||||
keys.sort_by_key(|key| *key.public().as_bytes());
|
||||
let [low, high] = keys;
|
||||
|
||||
// The member with the higher id claimed first, so it holds the name,
|
||||
// and the lower id does not take it back by sorting first.
|
||||
let early = named_at(&high, id, 1, "music", 1_000);
|
||||
let late = named_at(&low, id, 1, "music", 2_000);
|
||||
for order in [[&early, &late], [&late, &early]] {
|
||||
let mut set = StateSet::new();
|
||||
for record in order {
|
||||
set.merge(id, (*record).clone()).unwrap();
|
||||
}
|
||||
assert_eq!(set.hostname_of(&high.public()), Some("music"));
|
||||
assert_eq!(set.hostname_of(&low.public()), None);
|
||||
assert_eq!(
|
||||
set.hostname_taken(&low.public()),
|
||||
Some(("music", high.public()))
|
||||
);
|
||||
assert_eq!(set.hostname_taken(&high.public()), None);
|
||||
}
|
||||
|
||||
// At the same instant the lower id decides, the same way everywhere.
|
||||
let mut set = StateSet::new();
|
||||
set.merge(id, named_at(&high, id, 1, "music", 5)).unwrap();
|
||||
set.merge(id, named_at(&low, id, 1, "music", 5)).unwrap();
|
||||
assert_eq!(set.hostname_of(&low.public()), Some("music"));
|
||||
assert_eq!(set.hostname_of(&high.public()), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_name_that_loses_is_found_again_once_the_holder_lets_it_go() {
|
||||
let id = network("naming");
|
||||
let first = SecretKey::generate();
|
||||
let second = SecretKey::generate();
|
||||
let mut set = StateSet::new();
|
||||
set.merge(id, named_at(&first, id, 1, "music", 1)).unwrap();
|
||||
set.merge(id, named_at(&second, id, 1, "music", 2)).unwrap();
|
||||
assert_eq!(set.hostname_of(&second.public()), None);
|
||||
|
||||
// Renaming starts the clock afresh, so it cannot be used to jump the
|
||||
// queue for a name somebody else already holds.
|
||||
set.merge(id, named_at(&first, id, 2, "other", 3)).unwrap();
|
||||
assert_eq!(set.hostname_of(&second.public()), Some("music"));
|
||||
set.merge(id, named_at(&first, id, 3, "music", 4)).unwrap();
|
||||
assert_eq!(set.hostname_of(&second.public()), Some("music"));
|
||||
assert_eq!(set.hostname_of(&first.public()), None);
|
||||
|
||||
// And a release hands it on.
|
||||
set.merge(id, SignedRecord::sign(&second, id, 2, RecordBody::Release))
|
||||
.unwrap();
|
||||
assert_eq!(set.hostname_of(&first.public()), Some("music"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_time_with_no_name_is_not_a_valid_claim() {
|
||||
let id = network("naming");
|
||||
let secret = SecretKey::generate();
|
||||
let record = SignedRecord::sign(
|
||||
&secret,
|
||||
id,
|
||||
1,
|
||||
RecordBody::Claim {
|
||||
address: None,
|
||||
range: None,
|
||||
hostname: None,
|
||||
hostname_since: Some(1),
|
||||
},
|
||||
);
|
||||
assert!(matches!(record.verify(id), Err(StateError::Malformed(_))));
|
||||
|
||||
// The time is signed: moving it breaks the signature, so nobody can
|
||||
// improve another member's place in the queue.
|
||||
let mut forged = named_at(&secret, id, 1, "music", 9_000);
|
||||
if let RecordBody::Claim { hostname_since, .. } = &mut forged.body {
|
||||
*hostname_since = Some(1);
|
||||
}
|
||||
assert_eq!(forged.verify(id).unwrap_err(), StateError::BadSignature);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_release_gives_up_the_name_as_well_as_the_address() {
|
||||
let secret = SecretKey::generate();
|
||||
@@ -894,6 +1057,7 @@ mod tests {
|
||||
address: Some("10.13.37.5".parse().unwrap()),
|
||||
range: Some(range()),
|
||||
hostname: Some("music".into()),
|
||||
hostname_since: None,
|
||||
},
|
||||
),
|
||||
)
|
||||
@@ -1057,6 +1221,7 @@ mod tests {
|
||||
address: Some("10.99.0.7".parse().unwrap()),
|
||||
range: Some(custom),
|
||||
hostname: None,
|
||||
hostname_since: None,
|
||||
},
|
||||
),
|
||||
)
|
||||
|
||||
@@ -21,7 +21,7 @@ use crate::identity::{DeviceIdentity, NetworkId, NetworkName, NetworkSecret};
|
||||
use crate::state::{RecordBody, SignedRecord};
|
||||
|
||||
/// Schema version written by this build.
|
||||
pub const SCHEMA_VERSION: i64 = 6;
|
||||
pub const SCHEMA_VERSION: i64 = 7;
|
||||
|
||||
/// Key of the stored hostname setting.
|
||||
const SETTING_HOSTNAME: &str = "hostname";
|
||||
@@ -221,6 +221,23 @@ impl StateStore {
|
||||
)
|
||||
.map_err(|err| self.corrupt(format!("cannot migrate schema to 6: {err}")))?;
|
||||
}
|
||||
// Migration 6 -> 7: a claim gained the time its hostname was first
|
||||
// claimed, which changed the signing domain. Records signed before
|
||||
// no longer verify, so they are discarded rather than kept as rows
|
||||
// every read has to reject; each member re-publishes its own claim on
|
||||
// its next run and the others arrive again from their authors. The
|
||||
// version counter is kept, so nothing is ever published twice at one
|
||||
// version.
|
||||
if found < 7 {
|
||||
self.conn
|
||||
.execute_batch(
|
||||
"BEGIN;
|
||||
DELETE FROM signed_records;
|
||||
PRAGMA user_version = 7;
|
||||
COMMIT;",
|
||||
)
|
||||
.map_err(|err| self.corrupt(format!("cannot migrate schema to 7: {err}")))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,308 @@
|
||||
//! Connectivity does not depend on what kind of link each hop happens to be,
|
||||
//! and a member is online however far away it is.
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use bytes::Bytes;
|
||||
use iroh::EndpointId;
|
||||
use tsunagi::NetworkId;
|
||||
use tsunagi::agent::DataPath;
|
||||
use tsunagi::dataplane::transport::SharedLink;
|
||||
use tsunagi::dataplane::{IpPlugin, PluginCapability, PluginError};
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::testing::{TestAgent, network, wait_for_peers, wait_until};
|
||||
|
||||
/// A protocol that moves bytes between two members unchanged and keeps the
|
||||
/// link it was handed, so a test can use the link directly.
|
||||
#[derive(Debug)]
|
||||
struct Carrier {
|
||||
protocol: String,
|
||||
links: Mutex<HashMap<EndpointId, SharedLink>>,
|
||||
}
|
||||
|
||||
impl Carrier {
|
||||
fn new(protocol: &str) -> Arc<Self> {
|
||||
Arc::new(Self {
|
||||
protocol: protocol.into(),
|
||||
links: Mutex::default(),
|
||||
})
|
||||
}
|
||||
|
||||
fn link(&self, peer: EndpointId) -> Option<SharedLink> {
|
||||
self.links.lock().unwrap().get(&peer).cloned()
|
||||
}
|
||||
}
|
||||
|
||||
impl IpPlugin for Carrier {
|
||||
fn protocol_id(&self) -> &str {
|
||||
&self.protocol
|
||||
}
|
||||
|
||||
fn protocol_version(&self) -> u16 {
|
||||
1
|
||||
}
|
||||
|
||||
fn local_capability(
|
||||
&self,
|
||||
_network: NetworkId,
|
||||
) -> Result<Option<PluginCapability>, PluginError> {
|
||||
Ok(Some(PluginCapability {
|
||||
protocol: self.protocol.clone(),
|
||||
version: 1,
|
||||
enabled: true,
|
||||
data: Vec::new(),
|
||||
}))
|
||||
}
|
||||
|
||||
fn on_peer_capability(
|
||||
&self,
|
||||
_network: NetworkId,
|
||||
_peer: EndpointId,
|
||||
_capability: &PluginCapability,
|
||||
) -> Result<(), PluginError> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn on_peer_link(&self, _network: NetworkId, peer: EndpointId, link: SharedLink) {
|
||||
self.links.lock().unwrap().insert(peer, link);
|
||||
}
|
||||
|
||||
fn on_peer_gone(&self, _network: NetworkId, peer: EndpointId) {
|
||||
self.links.lock().unwrap().remove(&peer);
|
||||
}
|
||||
|
||||
fn on_network_deactivated(&self, _network: NetworkId) {}
|
||||
}
|
||||
|
||||
type Blocked = Arc<Mutex<HashSet<EndpointId>>>;
|
||||
|
||||
async fn member(
|
||||
discovery: &SharedMemoryDiscovery,
|
||||
plugins: &[Arc<Carrier>],
|
||||
blocked: &Blocked,
|
||||
) -> TestAgent {
|
||||
let plugins: Vec<_> = plugins.to_vec();
|
||||
let blocked = Arc::clone(blocked);
|
||||
TestAgent::spawn_with(
|
||||
move |config| {
|
||||
let config = config.with_unreachable_data_peers(blocked);
|
||||
plugins
|
||||
.into_iter()
|
||||
.fold(config, |config, plugin| config.with_plugin(plugin))
|
||||
},
|
||||
discovery,
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
async fn exchange(from: &SharedLink, to: &SharedLink, payload: &'static [u8]) {
|
||||
from.send(Bytes::from_static(payload)).unwrap();
|
||||
let got = tokio::time::timeout(tsunagi::testing::DEADLINE, to.recv())
|
||||
.await
|
||||
.expect("the datagram should arrive")
|
||||
.expect("the link stays open");
|
||||
assert_eq!(got, Bytes::from_static(payload));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_tunnel_crosses_a_member_that_shares_no_protocol_with_either_end_and_moves_to_a_direct_link_by_itself()
|
||||
{
|
||||
// A and B share one protocol and nothing else. C shares a different
|
||||
// one with each, so every link here is of a different kind and the
|
||||
// middle has never heard of what A and B speak. A and B cannot reach
|
||||
// each other directly, which is the case a blocked UDP path produces.
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("mixed-transports");
|
||||
|
||||
let blocked_a: Blocked = Arc::default();
|
||||
let blocked_b: Blocked = Arc::default();
|
||||
let a_shared = Carrier::new("shared");
|
||||
let b_shared = Carrier::new("shared");
|
||||
let a = member(
|
||||
&discovery,
|
||||
&[Carrier::new("a-and-c"), Arc::clone(&a_shared)],
|
||||
&blocked_a,
|
||||
)
|
||||
.await;
|
||||
let b = member(
|
||||
&discovery,
|
||||
&[Carrier::new("c-and-b"), Arc::clone(&b_shared)],
|
||||
&blocked_b,
|
||||
)
|
||||
.await;
|
||||
let c = member(
|
||||
&discovery,
|
||||
&[Carrier::new("a-and-c"), Carrier::new("c-and-b")],
|
||||
&Blocked::default(),
|
||||
)
|
||||
.await;
|
||||
let (a_id, b_id, c_id) = (
|
||||
a.agent.endpoint_id(),
|
||||
b.agent.endpoint_id(),
|
||||
c.agent.endpoint_id(),
|
||||
);
|
||||
blocked_a.lock().unwrap().insert(b_id);
|
||||
blocked_b.lock().unwrap().insert(a_id);
|
||||
|
||||
let network_id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
c.agent.join_network(&name, &secret).await.unwrap();
|
||||
for agent in [&a, &b, &c] {
|
||||
wait_for_peers(&agent.agent, network_id, 2).await;
|
||||
}
|
||||
|
||||
// The tunnel comes up through C, and data goes both ways.
|
||||
let to_b = wait_until("a has a link to b", || async { a_shared.link(b_id) }).await;
|
||||
let to_a = wait_until("b has a link to a", || async { b_shared.link(a_id) }).await;
|
||||
exchange(&to_b, &to_a, b"a to b, sealed").await;
|
||||
exchange(&to_a, &to_b, b"b to a, sealed").await;
|
||||
|
||||
// The status says what is true: online, and through whom.
|
||||
let status = a.agent.network_status(network_id).await.unwrap();
|
||||
let b_peer = status.peers.iter().find(|p| p.endpoint_id == b_id).unwrap();
|
||||
assert_eq!(b_peer.data_path, DataPath::Relayed { hops: 2, via: c_id });
|
||||
let c_peer = status.peers.iter().find(|p| p.endpoint_id == c_id).unwrap();
|
||||
assert!(matches!(&c_peer.data_path, DataPath::Direct { transport } if transport == "a-and-c"));
|
||||
let b_member = wait_until("a lists b as a member", || async {
|
||||
let status = a.agent.network_status(network_id).await.ok()?;
|
||||
status.members.into_iter().find(|m| m.endpoint_id == b_id)
|
||||
})
|
||||
.await;
|
||||
assert!(b_member.online);
|
||||
let carried = c.agent.network_status(network_id).await.unwrap().relay;
|
||||
assert!(carried.forwarded >= 2, "{carried:?}");
|
||||
|
||||
// The direct path becomes possible: the same link carries on, now
|
||||
// without C, and nobody has to do anything about it.
|
||||
blocked_a.lock().unwrap().clear();
|
||||
blocked_b.lock().unwrap().clear();
|
||||
wait_until("a reaches b directly", || async {
|
||||
let status = a.agent.network_status(network_id).await.ok()?;
|
||||
let peer = status.peers.iter().find(|p| p.endpoint_id == b_id)?;
|
||||
matches!(&peer.data_path, DataPath::Direct { transport } if transport == "shared")
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
let forwarded = c
|
||||
.agent
|
||||
.network_status(network_id)
|
||||
.await
|
||||
.unwrap()
|
||||
.relay
|
||||
.forwarded;
|
||||
exchange(&to_b, &to_a, b"direct now").await;
|
||||
assert!(
|
||||
Arc::ptr_eq(&to_b, &a_shared.link(b_id).unwrap()),
|
||||
"the tunnel's link is the one it always had"
|
||||
);
|
||||
assert_eq!(
|
||||
c.agent
|
||||
.network_status(network_id)
|
||||
.await
|
||||
.unwrap()
|
||||
.relay
|
||||
.forwarded,
|
||||
forwarded,
|
||||
"nothing goes through C any more"
|
||||
);
|
||||
|
||||
a.stop().await;
|
||||
b.stop().await;
|
||||
c.stop().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_name_belongs_to_the_member_that_claimed_it_first_on_every_member() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("first-claim");
|
||||
let spawn = |hostname: &'static str| {
|
||||
let discovery = discovery.clone();
|
||||
async move {
|
||||
TestAgent::spawn_with(move |config| config.with_hostname(hostname), &discovery)
|
||||
.await
|
||||
.unwrap()
|
||||
}
|
||||
};
|
||||
|
||||
let first = spawn("music").await;
|
||||
let network_id = first.agent.join_network(&name, &secret).await.unwrap();
|
||||
let first_id = first.agent.endpoint_id();
|
||||
wait_until("the first claim is signed", || async {
|
||||
let status = first.agent.network_status(network_id).await.ok()?;
|
||||
status
|
||||
.members
|
||||
.iter()
|
||||
.any(|m| m.endpoint_id == first_id && m.hostname.as_deref() == Some("music"))
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
|
||||
// Whatever the ids turn out to be, the later one does not take it.
|
||||
let second = spawn("music").await;
|
||||
let third = spawn("studio").await;
|
||||
second.agent.join_network(&name, &secret).await.unwrap();
|
||||
third.agent.join_network(&name, &secret).await.unwrap();
|
||||
let second_id = second.agent.endpoint_id();
|
||||
for agent in [&first, &second, &third] {
|
||||
wait_for_peers(&agent.agent, network_id, 2).await;
|
||||
}
|
||||
|
||||
for (viewer, label) in [(&first, "first"), (&second, "second"), (&third, "third")] {
|
||||
let status = wait_until(&format!("{label} sees both claims"), || async {
|
||||
let status = viewer.agent.network_status(network_id).await.ok()?;
|
||||
// The roster is the authors of signed records, so a member
|
||||
// being in it means its claim has arrived.
|
||||
let both = [first_id, second_id]
|
||||
.iter()
|
||||
.all(|id| status.members.iter().any(|m| m.endpoint_id == *id));
|
||||
both.then_some(status)
|
||||
})
|
||||
.await;
|
||||
let holder = status
|
||||
.members
|
||||
.iter()
|
||||
.find(|m| m.endpoint_id == first_id)
|
||||
.unwrap();
|
||||
let loser = status
|
||||
.members
|
||||
.iter()
|
||||
.find(|m| m.endpoint_id == second_id)
|
||||
.unwrap();
|
||||
assert_eq!(holder.hostname.as_deref(), Some("music"), "{label}");
|
||||
assert_eq!(loser.hostname, None, "{label}: no name for the later one");
|
||||
let conflict = loser.hostname_conflict.as_ref().expect(label);
|
||||
assert_eq!(conflict.name, "music");
|
||||
assert_eq!(conflict.holder, first_id);
|
||||
assert!(holder.hostname_conflict.is_none(), "{label}");
|
||||
// Nothing about being second makes it less of a member.
|
||||
assert!(
|
||||
loser.online || viewer.agent.endpoint_id() == second_id,
|
||||
"{label}"
|
||||
);
|
||||
}
|
||||
|
||||
// The later one is told so itself, and keeps working.
|
||||
let own = second.agent.network_status(network_id).await.unwrap();
|
||||
let conflict = own.hostname_conflict.expect("told its name is taken");
|
||||
assert_eq!(
|
||||
(conflict.name.as_str(), conflict.holder),
|
||||
("music", first_id)
|
||||
);
|
||||
assert!(
|
||||
first
|
||||
.agent
|
||||
.network_status(network_id)
|
||||
.await
|
||||
.unwrap()
|
||||
.hostname_conflict
|
||||
.is_none()
|
||||
);
|
||||
|
||||
first.stop().await;
|
||||
second.stop().await;
|
||||
third.stop().await;
|
||||
}
|
||||
@@ -9,13 +9,17 @@ use std::collections::HashMap;
|
||||
|
||||
use eframe::egui;
|
||||
|
||||
use tsunagi::ipc::{NetworkReport, OverlayPeerReport};
|
||||
use tsunagi::ipc::{DataPathReport, NetworkReport, OverlayPeerReport};
|
||||
|
||||
use crate::agent::{AgentClient, Command};
|
||||
use crate::exit;
|
||||
use crate::format;
|
||||
use crate::stats::{Traffic, Unit};
|
||||
|
||||
/// A device that works but not as well as it could: reached through others,
|
||||
/// or with a name somebody else holds.
|
||||
const AMBER: egui::Color32 = egui::Color32::from_rgb(0xd9, 0xa4, 0x2c);
|
||||
|
||||
/// A readable title for a network's window.
|
||||
pub(crate) fn title(network: &NetworkReport) -> String {
|
||||
format!("tsunagi · {}", network.name)
|
||||
@@ -23,14 +27,23 @@ pub(crate) fn title(network: &NetworkReport) -> String {
|
||||
|
||||
/// One row of the device table, built from a tunnel and/or a signed member.
|
||||
struct Device<'a> {
|
||||
/// The agent's own answer to whether it can be reached by any path.
|
||||
online: bool,
|
||||
/// How packets reach it.
|
||||
path: Option<&'a DataPathReport>,
|
||||
/// What to call the member its traffic goes through, when it does.
|
||||
via_name: Option<String>,
|
||||
/// The name it claimed that another member holds, and what to call that
|
||||
/// member.
|
||||
name_taken: Option<(&'a str, String)>,
|
||||
/// The last hostname seen for it, kept while it is offline.
|
||||
hostname: Option<&'a str>,
|
||||
/// What clicking the hostname copies: `hostname.network`, so it resolves.
|
||||
copy: Option<String>,
|
||||
/// The full endpoint id, which clicking the short form copies.
|
||||
id: &'a str,
|
||||
proto: &'a str,
|
||||
/// End-to-end tunnel protocol, independent of the path carrying it.
|
||||
tunnel_protocol: &'a str,
|
||||
address: Option<String>,
|
||||
handshake_secs: Option<u64>,
|
||||
overlay: Option<&'a OverlayPeerReport>,
|
||||
@@ -93,6 +106,13 @@ fn name_cell(ui: &mut egui::Ui, device: &Device<'_>) {
|
||||
}
|
||||
_ => format::copy_field(ui, &short_id(device.id), device.id),
|
||||
}
|
||||
if let Some((name, holder)) = &device.name_taken {
|
||||
ui.label(egui::RichText::new(egui_phosphor::regular::WARNING).color(AMBER))
|
||||
.on_hover_text(format!(
|
||||
"`{name}` was claimed first by {holder}. This device works by address, \
|
||||
but nobody finds it by that name."
|
||||
));
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -276,6 +296,11 @@ fn known_ids<'a>(network: &'a NetworkReport, own_id: &str) -> Vec<&'a str> {
|
||||
ids
|
||||
}
|
||||
|
||||
/// How many other devices can be reached now, by any path.
|
||||
pub(crate) fn online_count(network: &NetworkReport, own_id: &str) -> usize {
|
||||
devices(network, own_id).iter().filter(|d| d.online).count()
|
||||
}
|
||||
|
||||
/// How many other devices this network knows of, connected or not.
|
||||
pub(crate) fn known_count(network: &NetworkReport, own_id: &str) -> usize {
|
||||
known_ids(network, own_id).len()
|
||||
@@ -310,14 +335,33 @@ fn devices<'a>(network: &'a NetworkReport, own_id: &str) -> Vec<Device<'a>> {
|
||||
.map(|id| {
|
||||
let overlay = overlay_peers.iter().find(|p| p.endpoint_id == id);
|
||||
let member = network.members.iter().find(|m| m.endpoint_id == id);
|
||||
let online = overlay.is_some_and(|p| p.handshake_secs_ago.is_some());
|
||||
let online = member.is_some_and(|m| m.online)
|
||||
|| network.peers.iter().any(|p| p.endpoint_id == id);
|
||||
let path = network
|
||||
.peers
|
||||
.iter()
|
||||
.find(|p| p.endpoint_id == id)
|
||||
.map(|p| &p.path)
|
||||
.or_else(|| member.map(|m| &m.path));
|
||||
let via_name = match path {
|
||||
Some(DataPathReport::Relayed { via, .. }) => Some(exit::name_of(network, via)),
|
||||
_ => None,
|
||||
};
|
||||
let name_taken = member.and_then(|m| {
|
||||
m.hostname_conflict
|
||||
.as_ref()
|
||||
.map(|c| (c.name.as_str(), exit::name_of(network, &c.holder)))
|
||||
});
|
||||
let hostname = hostnames.get(id).copied();
|
||||
Device {
|
||||
online,
|
||||
path,
|
||||
via_name,
|
||||
name_taken,
|
||||
hostname,
|
||||
copy: hostname.map(|h| format!("{h}.{}", network.name)),
|
||||
id,
|
||||
proto: overlay.map_or("—", |p| p.protocol.as_str()),
|
||||
tunnel_protocol: overlay.map_or("—", |p| p.protocol.as_str()),
|
||||
address: overlay
|
||||
.and_then(|p| p.address.clone())
|
||||
.or_else(|| member.and_then(|m| m.overlay_address_v4.clone())),
|
||||
@@ -346,9 +390,9 @@ fn devices<'a>(network: &'a NetworkReport, own_id: &str) -> Vec<Device<'a>> {
|
||||
const COLUMN_SAMPLES: [&str; 9] = [
|
||||
"",
|
||||
"exit",
|
||||
"tcp-tls",
|
||||
"wg-quic · waiting for handshake",
|
||||
"255.255.255.255",
|
||||
"23h 59m",
|
||||
"direct over tcp-tls",
|
||||
"99999/99999",
|
||||
"1023.9 MB / 1023.9 MB",
|
||||
"1023.9 kB/s",
|
||||
@@ -465,9 +509,9 @@ fn device_table(
|
||||
for (column, header) in [
|
||||
"device",
|
||||
"exit",
|
||||
"proto",
|
||||
"end-to-end",
|
||||
"address",
|
||||
"handshake",
|
||||
"data path",
|
||||
"pkts",
|
||||
"bytes",
|
||||
"rate",
|
||||
@@ -477,7 +521,22 @@ fn device_table(
|
||||
.enumerate()
|
||||
{
|
||||
cell(ui, column, &mut widths, |ui| {
|
||||
ui.weak(header);
|
||||
let response = ui.weak(header);
|
||||
match column {
|
||||
2 => {
|
||||
response.on_hover_text(
|
||||
"Protocol that encrypts traffic between this device and the \
|
||||
destination. It is independent of the route carrying it.",
|
||||
);
|
||||
}
|
||||
4 => {
|
||||
response.on_hover_text(
|
||||
"Current route carrying the encrypted tunnel traffic. `direct \
|
||||
over tcp-tls` means one Tsunagi hop with no mesh relay.",
|
||||
);
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
});
|
||||
}
|
||||
ui.end_row();
|
||||
@@ -488,7 +547,7 @@ fn device_table(
|
||||
exit_cell(ui, device, network_id, confirm);
|
||||
});
|
||||
cell(ui, 2, &mut widths, |ui| {
|
||||
ui.label(device.proto);
|
||||
tunnel_cell(ui, device);
|
||||
});
|
||||
cell(ui, 3, &mut widths, |ui| match &device.address {
|
||||
Some(address) => format::copy_field(ui, address, address),
|
||||
@@ -496,13 +555,7 @@ fn device_table(
|
||||
ui.weak("—");
|
||||
}
|
||||
});
|
||||
cell(ui, 4, &mut widths, |ui| {
|
||||
ui.label(
|
||||
device
|
||||
.handshake_secs
|
||||
.map_or_else(|| "offline".to_string(), format::age),
|
||||
);
|
||||
});
|
||||
cell(ui, 4, &mut widths, |ui| data_path_cell(ui, device));
|
||||
|
||||
match device.overlay.filter(|_| device.online) {
|
||||
Some(peer) => {
|
||||
@@ -580,6 +633,211 @@ fn device_table(
|
||||
ui.memory_mut(|m| m.data.insert_temp(widths_id, widths));
|
||||
}
|
||||
|
||||
/// A protocol name as it should appear in the table.
|
||||
fn displayed_protocol(protocol: &str) -> &str {
|
||||
if protocol.is_empty() {
|
||||
"wg-quic"
|
||||
} else {
|
||||
protocol
|
||||
}
|
||||
}
|
||||
|
||||
/// The end-to-end tunnel and its own health, without transport details.
|
||||
fn tunnel_text(protocol: &str, handshake_secs: Option<u64>) -> String {
|
||||
let protocol = displayed_protocol(protocol);
|
||||
if protocol == "—" {
|
||||
return protocol.to_string();
|
||||
}
|
||||
match handshake_secs {
|
||||
Some(secs) if protocol == "tcp-tls" => {
|
||||
format!("{protocol} · up {}", format::age(secs))
|
||||
}
|
||||
Some(secs) => format!("{protocol} · handshake {} ago", format::age(secs)),
|
||||
None => format!("{protocol} · waiting for handshake"),
|
||||
}
|
||||
}
|
||||
|
||||
fn tunnel_cell(ui: &mut egui::Ui, device: &Device<'_>) {
|
||||
let protocol = displayed_protocol(device.tunnel_protocol);
|
||||
let text = tunnel_text(protocol, device.handshake_secs);
|
||||
if protocol == "—" {
|
||||
ui.weak(text);
|
||||
return;
|
||||
}
|
||||
let hint = match device.path {
|
||||
Some(DataPathReport::Direct { transport }) => format!(
|
||||
"The end-to-end {protocol} tunnel is currently carried over a direct {transport} \
|
||||
transport."
|
||||
),
|
||||
Some(DataPathReport::Relayed { hops, .. }) => format!(
|
||||
"The end-to-end {protocol} tunnel is currently carried through other devices over \
|
||||
{hops} transport hops."
|
||||
),
|
||||
Some(DataPathReport::None) | None => {
|
||||
format!("The end-to-end {protocol} tunnel does not have a data path carrying it yet.")
|
||||
}
|
||||
};
|
||||
ui.label(text).on_hover_text(hint);
|
||||
}
|
||||
|
||||
/// Current route text and whether that route is the preferred direct form.
|
||||
fn data_path_text(
|
||||
path: Option<&DataPathReport>,
|
||||
via_name: Option<&str>,
|
||||
online: bool,
|
||||
) -> (String, bool) {
|
||||
if !online {
|
||||
return ("offline".into(), false);
|
||||
}
|
||||
match path {
|
||||
Some(DataPathReport::Direct { transport }) => (format!("direct over {transport}"), true),
|
||||
Some(DataPathReport::Relayed { hops, .. }) => (
|
||||
format!("via {} · {hops} hops", via_name.unwrap_or("another device")),
|
||||
false,
|
||||
),
|
||||
Some(DataPathReport::None) | None => ("no data path yet".into(), false),
|
||||
}
|
||||
}
|
||||
|
||||
/// How a device is reached right now.
|
||||
///
|
||||
/// A device reached through others is online, over a longer path, and says
|
||||
/// so: the colour is the only difference in how it is treated.
|
||||
fn data_path_cell(ui: &mut egui::Ui, device: &Device<'_>) {
|
||||
let (text, good) = data_path_text(device.path, device.via_name.as_deref(), device.online);
|
||||
let response = if good {
|
||||
ui.label(text)
|
||||
} else {
|
||||
ui.label(egui::RichText::new(text).color(AMBER))
|
||||
};
|
||||
response.on_hover_text(match device.path {
|
||||
Some(DataPathReport::Direct { .. }) => {
|
||||
"One Tsunagi hop with no mesh relay. The end-to-end tunnel may use a different \
|
||||
protocol from this transport."
|
||||
}
|
||||
Some(DataPathReport::Relayed { .. }) => {
|
||||
"No direct transport yet: traffic goes through another device, and moves to a \
|
||||
direct path by itself as soon as one comes up."
|
||||
}
|
||||
_ if device.online => "Connected, but no data path carries packets yet.",
|
||||
_ => "This device is not reachable now.",
|
||||
});
|
||||
}
|
||||
|
||||
fn yes_no(value: bool) -> &'static str {
|
||||
if value { "yes" } else { "no" }
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
use tsunagi::ipc::{HostnameConflictReport, MemberReport};
|
||||
|
||||
fn member(id: &str, online: bool, path: DataPathReport) -> MemberReport {
|
||||
MemberReport {
|
||||
endpoint_id: id.into(),
|
||||
online,
|
||||
path,
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_device_reached_through_another_is_online_whatever_its_tunnel_says() {
|
||||
// The tray used to call a device online only while its tunnel had
|
||||
// handshaken, which disagreed with the command line about the same
|
||||
// device. Both now read the agent's answer.
|
||||
let network = NetworkReport {
|
||||
members: vec![
|
||||
member("own", true, DataPathReport::None),
|
||||
member(
|
||||
"far",
|
||||
true,
|
||||
DataPathReport::Relayed {
|
||||
hops: 2,
|
||||
via: "near".into(),
|
||||
},
|
||||
),
|
||||
member("away", false, DataPathReport::None),
|
||||
],
|
||||
..Default::default()
|
||||
};
|
||||
let all = devices(&network, "own");
|
||||
let far = all.iter().find(|d| d.id == "far").unwrap();
|
||||
assert!(far.online);
|
||||
assert!(matches!(
|
||||
far.path,
|
||||
Some(DataPathReport::Relayed { hops: 2, .. })
|
||||
));
|
||||
assert!(!all.iter().find(|d| d.id == "away").unwrap().online);
|
||||
assert_eq!(online_count(&network, "own"), 1);
|
||||
assert_eq!(known_count(&network, "own"), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_name_somebody_else_holds_is_carried_to_the_row() {
|
||||
let mut taken = member("late", true, DataPathReport::None);
|
||||
taken.hostname_conflict = Some(HostnameConflictReport {
|
||||
name: "music".into(),
|
||||
holder: "early".into(),
|
||||
});
|
||||
let network = NetworkReport {
|
||||
members: vec![member("early", true, DataPathReport::None), taken],
|
||||
..Default::default()
|
||||
};
|
||||
let all = devices(&network, "own");
|
||||
let late = all.iter().find(|d| d.id == "late").unwrap();
|
||||
assert_eq!(
|
||||
late.name_taken.as_ref().map(|(name, _)| *name),
|
||||
Some("music")
|
||||
);
|
||||
assert!(
|
||||
all.iter()
|
||||
.find(|d| d.id == "early")
|
||||
.unwrap()
|
||||
.name_taken
|
||||
.is_none()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tunnel_and_direct_transport_are_named_as_different_layers() {
|
||||
assert_eq!(tunnel_text("wg", Some(1)), "wg · handshake 1s ago");
|
||||
assert_eq!(
|
||||
data_path_text(
|
||||
Some(&DataPathReport::Direct {
|
||||
transport: "tcp-tls".into(),
|
||||
}),
|
||||
None,
|
||||
true,
|
||||
),
|
||||
("direct over tcp-tls".into(), true)
|
||||
);
|
||||
assert_eq!(tunnel_text("tcp-tls", Some(42)), "tcp-tls · up 42s");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn incomplete_and_longer_paths_stay_explicit() {
|
||||
assert_eq!(
|
||||
tunnel_text("wg-quic", None),
|
||||
"wg-quic · waiting for handshake"
|
||||
);
|
||||
assert_eq!(
|
||||
data_path_text(
|
||||
Some(&DataPathReport::Relayed {
|
||||
hops: 2,
|
||||
via: "near".into(),
|
||||
}),
|
||||
Some("iris"),
|
||||
true,
|
||||
),
|
||||
("via iris · 2 hops".into(), false)
|
||||
);
|
||||
assert_eq!(
|
||||
data_path_text(Some(&DataPathReport::None), None, true),
|
||||
("no data path yet".into(), false)
|
||||
);
|
||||
assert_eq!(data_path_text(None, None, false), ("offline".into(), false));
|
||||
}
|
||||
}
|
||||
@@ -243,8 +243,8 @@ fn draw_network(
|
||||
}
|
||||
ui.weak("peers");
|
||||
ui.label(format!(
|
||||
"{} connected · {} known",
|
||||
network.peers.len(),
|
||||
"{} online · {} known",
|
||||
devices::online_count(network, own_id),
|
||||
devices::known_count(network, own_id)
|
||||
));
|
||||
ui.end_row();
|
||||
|
||||
@@ -42,6 +42,7 @@ fn source(agent: Agent, plugin: Arc<WireguardPlugin>) -> Arc<dyn tsunagi::ipc::R
|
||||
transport: format!("{:?}", peer.transport),
|
||||
rtt_ms: peer.rtt.map(|rtt| rtt.as_millis() as u64),
|
||||
exit_node: peer.exit_node,
|
||||
path: Default::default(),
|
||||
})
|
||||
.collect(),
|
||||
overlay: plugin.overview(net.network_id).map(|view| {
|
||||
|
||||
@@ -1661,8 +1661,10 @@ async fn lan_broadcast_fanout_is_scoped_opt_in_and_does_not_reflood() {
|
||||
bytes[28..].copy_from_slice(b"LAN GAME");
|
||||
// UDP checksum zero is valid for IPv4.
|
||||
let sum: u32 = bytes[..20]
|
||||
.chunks_exact(2)
|
||||
.map(|w| u16::from_be_bytes([w[0], w[1]]) as u32)
|
||||
.as_chunks::<2>()
|
||||
.0
|
||||
.iter()
|
||||
.map(|w| u16::from_be_bytes(*w) as u32)
|
||||
.sum();
|
||||
let checksum = !((sum & 0xffff) + (sum >> 16)) as u16;
|
||||
bytes[10..12].copy_from_slice(&checksum.to_be_bytes());
|
||||
|
||||
+231
-20
@@ -3156,7 +3156,18 @@ fn dns_absent_section() -> report::Section {
|
||||
struct MemberRow<'a> {
|
||||
endpoint_id: &'a str,
|
||||
hostname: Option<&'a str>,
|
||||
/// `Some` exactly when there is an authenticated session right now.
|
||||
/// Whether it can be reached now, by any path: the agent's own answer,
|
||||
/// the same one every other client gives.
|
||||
online: bool,
|
||||
/// How packets reach it.
|
||||
path: Option<&'a tsunagi::ipc::DataPathReport>,
|
||||
/// What to call the member the first of those links goes to.
|
||||
via_label: Option<String>,
|
||||
/// The name it claimed that another member holds, and what to call the
|
||||
/// holder.
|
||||
name_taken: Option<(&'a str, String)>,
|
||||
/// `Some` exactly when there is an authenticated session right now: how
|
||||
/// that control connection reaches the peer.
|
||||
transport: Option<&'a str>,
|
||||
rtt_ms: Option<u64>,
|
||||
overlay_address_v4: Option<&'a str>,
|
||||
@@ -3170,7 +3181,7 @@ struct MemberRow<'a> {
|
||||
|
||||
impl MemberRow<'_> {
|
||||
fn online(&self) -> bool {
|
||||
self.transport.is_some()
|
||||
self.online
|
||||
}
|
||||
|
||||
/// What to call it: the name it announced — the last one, when it is
|
||||
@@ -3194,6 +3205,10 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve
|
||||
rows.entry(id).or_insert_with(|| MemberRow {
|
||||
endpoint_id: id,
|
||||
hostname: None,
|
||||
online: false,
|
||||
path: None,
|
||||
via_label: None,
|
||||
name_taken: None,
|
||||
transport: None,
|
||||
rtt_ms: None,
|
||||
overlay_address_v4: None,
|
||||
@@ -3210,11 +3225,19 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve
|
||||
row.overlay_address_v4 = member.overlay_address_v4.as_deref();
|
||||
row.failed_dials = member.failed_dials;
|
||||
row.hostname = member.hostname.as_deref();
|
||||
row.online = member.online;
|
||||
row.path = Some(&member.path);
|
||||
row.name_taken = member
|
||||
.hostname_conflict
|
||||
.as_ref()
|
||||
.map(|conflict| (conflict.name.as_str(), exit_name(network, &conflict.holder)));
|
||||
}
|
||||
for peer in &network.peers {
|
||||
let row = entry(&mut rows, &peer.endpoint_id);
|
||||
row.hostname = peer.hostname.as_deref().or(row.hostname);
|
||||
row.exit_node = peer.exit_node;
|
||||
row.online = true;
|
||||
row.path = Some(&peer.path);
|
||||
row.transport = Some(&peer.transport);
|
||||
row.rtt_ms = peer.rtt_ms;
|
||||
}
|
||||
@@ -3230,6 +3253,9 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve
|
||||
|
||||
for row in rows.values_mut() {
|
||||
row.exit_in_use = network.exit.via.as_deref() == Some(row.endpoint_id);
|
||||
if let Some(tsunagi::ipc::DataPathReport::Relayed { via, .. }) = row.path {
|
||||
row.via_label = Some(exit_name(network, via));
|
||||
}
|
||||
}
|
||||
|
||||
// This agent is in the roster too — it signs claims like everyone else —
|
||||
@@ -3285,6 +3311,23 @@ fn network_section(
|
||||
),
|
||||
);
|
||||
}
|
||||
if let Some(taken) = &network.hostname_conflict {
|
||||
section.push(
|
||||
Row::new(
|
||||
Health::Degraded,
|
||||
"hostname",
|
||||
format!(
|
||||
"`{}` is held by {}, which claimed it first",
|
||||
taken.name,
|
||||
exit_name(network, &taken.holder)
|
||||
),
|
||||
)
|
||||
.with_note(
|
||||
"this device works by address, but nobody finds it by that name. \
|
||||
Pick another with `tsng id hostname <name>`.",
|
||||
),
|
||||
);
|
||||
}
|
||||
if let Some(conflict) = &network.range_conflict {
|
||||
section.push(
|
||||
Row::new(
|
||||
@@ -3547,7 +3590,10 @@ fn exit_name(network: &tsunagi::ipc::NetworkReport, id: &str) -> String {
|
||||
fn member_row(row: &MemberRow<'_>) -> report::Row {
|
||||
use report::{Health, Row};
|
||||
|
||||
let Some(transport) = row.transport else {
|
||||
let taken = row.name_taken.as_ref().map(|(name, holder)| {
|
||||
format!(" · name `{name}` is held by {holder}: reachable by address only")
|
||||
});
|
||||
if !row.online {
|
||||
// Away. Not a fault of this agent, and in a mesh of laptops it is the
|
||||
// ordinary condition, so it is stated rather than flagged.
|
||||
let mut detail = "offline".to_string();
|
||||
@@ -3557,6 +3603,7 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
|
||||
if row.exit_in_use {
|
||||
detail.push_str(" · your exit node");
|
||||
}
|
||||
detail.extend(taken);
|
||||
let out = Row::new(Health::Info, row.label(), detail);
|
||||
return if row.failed_dials > 0 {
|
||||
// Attributed to the member it concerns, rather than left as a
|
||||
@@ -3568,17 +3615,38 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
|
||||
} else {
|
||||
out
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
let direct = transport.eq_ignore_ascii_case("direct");
|
||||
use tsunagi::ipc::DataPathReport;
|
||||
let direct_control = row
|
||||
.transport
|
||||
.is_none_or(|transport| transport.eq_ignore_ascii_case("direct"));
|
||||
let tunnel_up = row.tunnel.is_some_and(|tunnel| tunnel.is_up());
|
||||
let has_overlay = row.tunnel.is_some();
|
||||
// Online is online: a member reached through others is as much here as
|
||||
// one reached directly, and is said to be reached that way.
|
||||
let (path, data_path_good) = match row.path {
|
||||
Some(DataPathReport::Direct { transport }) => {
|
||||
(format!("data path direct over {transport}"), true)
|
||||
}
|
||||
Some(DataPathReport::Relayed { hops, .. }) => (
|
||||
format!(
|
||||
"data path via {} · {hops} hops",
|
||||
row.via_label.as_deref().unwrap_or("another member")
|
||||
),
|
||||
false,
|
||||
),
|
||||
Some(DataPathReport::None) | None => ("connected · no data path yet".to_string(), false),
|
||||
};
|
||||
|
||||
let mut detail = String::new();
|
||||
if let Some(v4) = row.overlay_address_v4 {
|
||||
detail.push_str(&format!("{v4:<15} "));
|
||||
}
|
||||
detail.push_str(&transport.to_lowercase());
|
||||
detail.push_str(&path);
|
||||
if let Some(transport) = row.transport {
|
||||
detail.push_str(&format!(" · control path {}", transport.to_lowercase()));
|
||||
}
|
||||
if let Some(rtt) = row.rtt_ms {
|
||||
detail.push_str(&format!(" rtt {rtt}ms"));
|
||||
}
|
||||
@@ -3587,8 +3655,9 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
|
||||
} else if row.exit_node {
|
||||
detail.push_str(" · exit node");
|
||||
}
|
||||
detail.extend(taken);
|
||||
|
||||
let health = if !direct || (has_overlay && !tunnel_up) {
|
||||
let health = if !data_path_good || !direct_control || (has_overlay && !tunnel_up) {
|
||||
Health::Degraded
|
||||
} else {
|
||||
Health::Good
|
||||
@@ -3603,13 +3672,13 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
|
||||
};
|
||||
out = out.with_note(match tunnel.handshake_secs_ago {
|
||||
Some(secs) => {
|
||||
let stats = format!(
|
||||
"tunnel up ({proto}), {} tx {} rx {}{}",
|
||||
if proto == "tcp-tls" {
|
||||
format!("uptime {secs}s,")
|
||||
} else {
|
||||
format!("handshake {secs}s ago,")
|
||||
},
|
||||
let health = if proto == "tcp-tls" {
|
||||
format!("uptime {secs}s")
|
||||
} else {
|
||||
format!("handshake {secs}s ago")
|
||||
};
|
||||
format!(
|
||||
"end-to-end tunnel {proto} · {health} · tx {} rx {}{} · transport detail: {}",
|
||||
tunnel.tx_packets,
|
||||
tunnel.rx_packets,
|
||||
if tunnel.dropped > 0 {
|
||||
@@ -3617,14 +3686,20 @@ fn member_row(row: &MemberRow<'_>) -> report::Row {
|
||||
} else {
|
||||
String::new()
|
||||
},
|
||||
);
|
||||
format!("{stats:<48} · {}", tunnel.path)
|
||||
tunnel.path
|
||||
)
|
||||
}
|
||||
None => {
|
||||
if proto == "wg-quic" || proto == "wg" {
|
||||
"no WireGuard handshake yet; the tunnel cannot carry traffic".to_string()
|
||||
format!(
|
||||
"end-to-end tunnel {proto} · no WireGuard handshake yet; the tunnel \
|
||||
cannot carry traffic"
|
||||
)
|
||||
} else {
|
||||
format!("no {proto} handshake yet; the tunnel cannot carry traffic")
|
||||
format!(
|
||||
"end-to-end tunnel {proto} · no {proto} handshake yet; the tunnel \
|
||||
cannot carry traffic"
|
||||
)
|
||||
}
|
||||
}
|
||||
});
|
||||
@@ -4890,6 +4965,7 @@ async fn build_report(
|
||||
transport: peer.transport.to_string(),
|
||||
rtt_ms: peer.rtt.map(|rtt| rtt.as_millis() as u64),
|
||||
exit_node: peer.exit_node,
|
||||
path: data_path_report(&peer.data_path),
|
||||
})
|
||||
.collect(),
|
||||
members: network
|
||||
@@ -4911,6 +4987,9 @@ async fn build_report(
|
||||
.last_hostname
|
||||
.clone()
|
||||
.or_else(|| member.hostname.clone()),
|
||||
online: member.online,
|
||||
path: data_path_report(&member.data_path),
|
||||
hostname_conflict: member.hostname_conflict.as_ref().map(conflict_report),
|
||||
})
|
||||
.collect(),
|
||||
range: network.range.map(|range| range.to_string()),
|
||||
@@ -4922,6 +5001,7 @@ async fn build_report(
|
||||
network.metrics.control_messages_received,
|
||||
),
|
||||
overlay,
|
||||
hostname_conflict: network.hostname_conflict.as_ref().map(conflict_report),
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
@@ -4955,6 +5035,28 @@ async fn build_report(
|
||||
}
|
||||
}
|
||||
|
||||
fn data_path_report(path: &tsunagi::agent::DataPath) -> tsunagi::ipc::DataPathReport {
|
||||
match path {
|
||||
tsunagi::agent::DataPath::None => tsunagi::ipc::DataPathReport::None,
|
||||
tsunagi::agent::DataPath::Direct { transport } => tsunagi::ipc::DataPathReport::Direct {
|
||||
transport: transport.clone(),
|
||||
},
|
||||
tsunagi::agent::DataPath::Relayed { hops, via } => tsunagi::ipc::DataPathReport::Relayed {
|
||||
hops: *hops,
|
||||
via: via.to_string(),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn conflict_report(
|
||||
conflict: &tsunagi::agent::HostnameConflict,
|
||||
) -> tsunagi::ipc::HostnameConflictReport {
|
||||
tsunagi::ipc::HostnameConflictReport {
|
||||
name: conflict.name.clone(),
|
||||
holder: conflict.holder.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// One network's exit-node settings, as the control interface reports them.
|
||||
fn exit_report(network: &tsunagi::agent::NetworkStatus) -> tsunagi::ipc::ExitReport {
|
||||
let rules = |report: &Option<tsunagi::overlay::RuleSetReport>| {
|
||||
@@ -5260,6 +5362,7 @@ mod status_tests {
|
||||
fn network_after_a_peer_returned() -> NetworkReport {
|
||||
NetworkReport {
|
||||
exit: Default::default(),
|
||||
hostname_conflict: None,
|
||||
broadcast: true,
|
||||
name: "LAB".into(),
|
||||
network_id: "xa7gyz".into(),
|
||||
@@ -5274,6 +5377,9 @@ mod status_tests {
|
||||
transport: "direct".into(),
|
||||
rtt_ms: Some(24),
|
||||
exit_node: false,
|
||||
path: tsunagi::ipc::DataPathReport::Direct {
|
||||
transport: "wg".into(),
|
||||
},
|
||||
}],
|
||||
members: vec![
|
||||
MemberReport {
|
||||
@@ -5281,12 +5387,18 @@ mod status_tests {
|
||||
overlay_address_v4: Some("10.13.37.69".into()),
|
||||
failed_dials: 0,
|
||||
hostname: None,
|
||||
..Default::default()
|
||||
},
|
||||
MemberReport {
|
||||
endpoint_id: ONLINE.into(),
|
||||
overlay_address_v4: Some("10.13.37.237".into()),
|
||||
failed_dials: 0,
|
||||
hostname: None,
|
||||
online: true,
|
||||
path: tsunagi::ipc::DataPathReport::Direct {
|
||||
transport: "wg".into(),
|
||||
},
|
||||
..Default::default()
|
||||
},
|
||||
],
|
||||
range: Some("10.13.37.0/24".into()),
|
||||
@@ -5528,6 +5640,7 @@ mod status_tests {
|
||||
overlay_address_v4: Some("10.13.37.99".into()),
|
||||
failed_dials: 9,
|
||||
hostname: None,
|
||||
..Default::default()
|
||||
});
|
||||
|
||||
let rows = member_rows(&network, OWN);
|
||||
@@ -5648,6 +5761,7 @@ mod status_tests {
|
||||
overlay_address_v4: Some("10.13.37.99".into()),
|
||||
failed_dials: 0,
|
||||
hostname: Some("laptop".into()),
|
||||
..Default::default()
|
||||
});
|
||||
|
||||
let rows = member_rows(&network, OWN);
|
||||
@@ -5668,6 +5782,7 @@ mod status_tests {
|
||||
overlay_address_v4: Some("10.13.37.99".into()),
|
||||
failed_dials: 9,
|
||||
hostname: None,
|
||||
..Default::default()
|
||||
});
|
||||
|
||||
let mut out = report::Report::new();
|
||||
@@ -5681,6 +5796,69 @@ mod status_tests {
|
||||
assert!(text.contains("1 of 2 online"), "{text}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_member_reached_through_another_is_online_and_says_through_whom() {
|
||||
// No direct link, a path through somebody else: online all the same,
|
||||
// and said to be, rather than being shown as away or as a fault.
|
||||
let mut network = network_after_a_peer_returned();
|
||||
network.members.push(MemberReport {
|
||||
endpoint_id: AWAY.into(),
|
||||
overlay_address_v4: Some("10.13.37.99".into()),
|
||||
hostname: Some("laptop".into()),
|
||||
online: true,
|
||||
path: tsunagi::ipc::DataPathReport::Relayed {
|
||||
hops: 2,
|
||||
via: ONLINE.into(),
|
||||
},
|
||||
..Default::default()
|
||||
});
|
||||
|
||||
let rows = member_rows(&network, OWN);
|
||||
let away = rows.iter().find(|row| row.endpoint_id == AWAY).unwrap();
|
||||
assert!(away.online());
|
||||
|
||||
let mut out = report::Report::new();
|
||||
out.push(network_section(&network, OWN, false));
|
||||
let text = out.render(false);
|
||||
assert!(text.contains("2 of 2 online"), "{text}");
|
||||
assert!(
|
||||
text.contains("data path via music · 2 hops"),
|
||||
"names the member it goes through: {text}"
|
||||
);
|
||||
assert!(!text.contains("offline"), "{text}");
|
||||
assert_eq!(out.worst(), Health::Degraded, "{text}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_name_somebody_else_holds_is_said_on_both_the_member_and_this_device() {
|
||||
let mut network = network_after_a_peer_returned();
|
||||
network.members[1].hostname_conflict = Some(tsunagi::ipc::HostnameConflictReport {
|
||||
name: "music".into(),
|
||||
holder: OWN.into(),
|
||||
});
|
||||
let text = {
|
||||
let mut out = report::Report::new();
|
||||
out.push(network_section(&network, OWN, false));
|
||||
out.render(false)
|
||||
};
|
||||
assert!(
|
||||
text.contains("name `music` is held by") && text.contains("reachable by address only"),
|
||||
"{text}"
|
||||
);
|
||||
|
||||
let mut network = network_after_a_peer_returned();
|
||||
network.hostname_conflict = Some(tsunagi::ipc::HostnameConflictReport {
|
||||
name: "studio".into(),
|
||||
holder: ONLINE.into(),
|
||||
});
|
||||
let mut out = report::Report::new();
|
||||
out.push(network_section(&network, OWN, false));
|
||||
let text = out.render(false);
|
||||
assert!(text.contains("`studio` is held by music"), "{text}");
|
||||
assert!(text.contains("tsng id hostname"), "{text}");
|
||||
assert_eq!(out.worst(), Health::Degraded, "{text}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_relayed_peer_is_graded_as_degraded_quality() {
|
||||
let mut network = network_after_a_peer_returned();
|
||||
@@ -5710,6 +5888,9 @@ mod status_tests {
|
||||
// With a peer connected the same counter is just history.
|
||||
let mut network = network_after_a_peer_returned();
|
||||
network.peers.clear();
|
||||
for member in &mut network.members {
|
||||
member.online = false;
|
||||
}
|
||||
network.overlay = Some(overlay(Vec::new()));
|
||||
network.handshake_failures = 4;
|
||||
|
||||
@@ -5734,10 +5915,40 @@ mod status_tests {
|
||||
out.push(network_section(&network, OWN, false));
|
||||
let text = out.render(false);
|
||||
assert_eq!(out.worst(), Health::Good, "{text}");
|
||||
assert!(text.contains("tunnel up (tcp-tls)"), "{text}");
|
||||
assert!(text.contains("end-to-end tunnel tcp-tls"), "{text}");
|
||||
assert!(text.contains("uptime 42s"), "{text}");
|
||||
assert!(text.contains("tx 100 rx 200"), "{text}");
|
||||
assert!(text.contains("tcp-tls via 192.0.2.1:443"), "{text}");
|
||||
assert!(
|
||||
text.contains("transport detail: tcp-tls via 192.0.2.1:443"),
|
||||
"{text}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn end_to_end_tunnel_and_data_transport_are_named_separately() {
|
||||
let mut network = network_after_a_peer_returned();
|
||||
network.peers[0].path = tsunagi::ipc::DataPathReport::Direct {
|
||||
transport: "tcp-tls".into(),
|
||||
};
|
||||
network.members[1].path = network.peers[0].path.clone();
|
||||
let mut t = tunnel(ONLINE, Some(1));
|
||||
t.protocol = "wg".into();
|
||||
t.path = "tcp-tls via 192.0.2.1:443".into();
|
||||
network.overlay = Some(overlay(vec![t]));
|
||||
|
||||
let mut out = report::Report::new();
|
||||
out.push(network_section(&network, OWN, false));
|
||||
let text = out.render(false);
|
||||
assert!(text.contains("data path direct over tcp-tls"), "{text}");
|
||||
assert!(text.contains("control path direct"), "{text}");
|
||||
assert!(
|
||||
text.contains("end-to-end tunnel wg · handshake 1s ago"),
|
||||
"{text}"
|
||||
);
|
||||
assert!(
|
||||
text.contains("transport detail: tcp-tls via 192.0.2.1:443"),
|
||||
"{text}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Vendored
+1
-1
@@ -22,7 +22,7 @@ service's socket at `/run/tsunagi/agent.sock`):
|
||||
|
||||
```sh
|
||||
sudo tsng status
|
||||
sudo tsng join <network-name> <tsn1…secret>
|
||||
sudo tsng join -n <network-name> -s <tsn1…secret>
|
||||
```
|
||||
|
||||
## How it runs
|
||||
|
||||
Vendored
+1
-1
@@ -28,7 +28,7 @@ group, so after `sudo usermod -aG tsunagi "$USER"` (and logging in again) no
|
||||
|
||||
```sh
|
||||
sudo tsng status
|
||||
sudo tsng join <network-name> <tsn1…secret>
|
||||
sudo tsng join -n <network-name> -s <tsn1…secret>
|
||||
```
|
||||
|
||||
## As your own user — one capability, no service
|
||||
|
||||
Vendored
+1
-1
@@ -48,7 +48,7 @@ Done. Start the agent with:
|
||||
Control it with sudo (the CLI finds the service's socket automatically):
|
||||
|
||||
sudo tsng status
|
||||
sudo tsng join <network-name> <tsn1…secret>
|
||||
sudo tsng join -n <network-name> -s <tsn1…secret>
|
||||
EOF
|
||||
else
|
||||
cat <<EOF
|
||||
|
||||
Vendored
+1
-1
@@ -15,7 +15,7 @@ socket and do **not** need `sudo`:
|
||||
|
||||
```sh
|
||||
tsng status
|
||||
tsng join <network-name> <tsn1…secret>
|
||||
tsng join -n <network-name> -s <tsn1…secret>
|
||||
```
|
||||
|
||||
The agent uses fixed system paths, so the commands find it regardless of whose
|
||||
|
||||
+29
-15
@@ -9,7 +9,7 @@ Two versions exist and are independent:
|
||||
|
||||
- **Identity scheme**, `tsunagi-network-id-v1`. Frozen. Changing it creates a
|
||||
different network space for the same name and secret.
|
||||
- **Control protocol**, ALPN `tsunagi/ctrl/4`, `PROTOCOL_VERSION = 4`.
|
||||
- **Control protocol**, ALPN `tsunagi/ctrl/5`, `PROTOCOL_VERSION = 5`.
|
||||
|
||||
Upgrading the crate or bumping the control protocol must never change an
|
||||
existing `NetworkId`.
|
||||
@@ -135,7 +135,7 @@ event does not report a network id.
|
||||
## The data plane protocol
|
||||
|
||||
IP plugin packets never travel on a control connection. They use their own
|
||||
ALPN, `tsunagi/data/4`, on their own iroh connection:
|
||||
ALPN, `tsunagi/data/5`, on their own iroh connection:
|
||||
|
||||
```text
|
||||
initiator -> responder : (the same membership handshake as above)
|
||||
@@ -172,32 +172,46 @@ holding up subsequent packets. Packet IDs are scoped to a QUIC connection;
|
||||
reassembly is created only after the membership handshake. All limits live in
|
||||
`config.rs`. There is no transport-layer retransmission added by this framing.
|
||||
|
||||
Version 4 adds a routing envelope inside the transport fragmentation framing:
|
||||
Version 4 added a routing envelope inside the transport fragmentation framing,
|
||||
and version 5 added the end-to-end protocol to it:
|
||||
|
||||
| field | bytes | encoding |
|
||||
|---|---:|---|
|
||||
| envelope version | 1 | `1` |
|
||||
| envelope version | 1 | `2` |
|
||||
| remaining hops | 1 | initially `16`; transit decrements, drops at `1` |
|
||||
| source peer | 32 | original endpoint public key |
|
||||
| destination peer | 32 | final endpoint public key |
|
||||
| flow id | 8 | big endian opaque stable hash supplied by the plugin |
|
||||
| protocol | 8 | big endian FNV-1a of the end-to-end protocol id (`wg`, `tcp-tls`, ...) |
|
||||
| payload | remaining | end-to-end encrypted plugin bytes |
|
||||
|
||||
The fixed header is 74 bytes. Zero/oversized hop limits, unknown versions,
|
||||
The fixed header is 82 bytes. Zero/oversized hop limits, unknown versions,
|
||||
short frames, oversized datagrams and unknown sources are rejected. A frame
|
||||
addressed here is delivered to its source's protocol inbox; transit goes straight
|
||||
to its next transport. The network and protocol are bound to the authenticated
|
||||
transport connection, not supplied by the frame. Intermediate nodes cannot
|
||||
decrypt or authenticate the inner WireGuard payload; the destination does that.
|
||||
Flow ids are routing hints, not authorization proofs.
|
||||
addressed here is delivered to the inbox of its source and protocol; transit
|
||||
goes straight to its next transport, whatever kind of transport that is. The
|
||||
network is bound to the authenticated transport connection, not supplied by
|
||||
the frame. The protocol names which end-to-end tunnel the payload belongs to
|
||||
and is independent of the links it crosses: a datagram of one protocol can
|
||||
travel over a link opened for another, and a middle member needs no protocol
|
||||
in common with either end. Intermediate nodes cannot decrypt or authenticate
|
||||
the inner payload; the destination's tunnel does that, so a frame naming the
|
||||
wrong protocol or a forged source is simply discarded there. Flow ids are
|
||||
routing hints, not authorization proofs.
|
||||
|
||||
Control ALPN 3 carries `Reachable { links: [{ peer, protocol }] }`. Each row
|
||||
belongs to the authenticated sender and is replaced atomically, expires after
|
||||
90 seconds, and is withdrawn on session closure. Only compatible authenticated
|
||||
members enter a protocol's graph; local edges always come from actual links.
|
||||
`Reachable { links: [{ peer, protocol }] }` lists the sender's direct links;
|
||||
`protocol` is the kind of transport the link is, for diagnostics, and routing
|
||||
ignores it. Each row belongs to the authenticated sender and is replaced
|
||||
atomically, expires after 90 seconds, and is withdrawn on session closure. Only
|
||||
authenticated members enter the graph; local edges always come from actual
|
||||
links.
|
||||
Announcements refresh on the existing control maintenance interval and are
|
||||
sent immediately on link changes. Topology stays volatile, outside signed state.
|
||||
|
||||
Version 5 also adds `hostname_since` to a signed hostname claim (signing domain
|
||||
`tsunagi-signed-record-v3`): a name belongs to whoever claimed it first. Stored
|
||||
records signed before are discarded on upgrade and re-published by their
|
||||
authors.
|
||||
|
||||
Upgrade both endpoints and intermediate peers together. Older control and data
|
||||
ALPNs cannot interoperate. The network secret, device identity, identity
|
||||
derivation, transcript encoding and stored network configuration are unchanged.
|
||||
@@ -220,7 +234,7 @@ not affect other networks.
|
||||
| `Pong { seq, payload }` | the echoed reply |
|
||||
| `State { records }` | a snapshot of signed records, merged into what the receiver holds |
|
||||
| `Peers { peers }` | unverified member address hints, requiring their own handshake |
|
||||
| `Reachable { links }` | sender's current direct data links, scoped by protocol |
|
||||
| `Reachable { links }` | sender's current direct data links, by kind of transport |
|
||||
| `Bye { reason }` | graceful goodbye; not a revocation of anything |
|
||||
|
||||
`broadcast` is a per-network local opt-in, enabled by default and treated as
|
||||
|
||||
+39
-9
@@ -15,12 +15,27 @@ the default 1280-byte interface MTU also works on small QUIC paths.
|
||||
## Graph and lifecycle
|
||||
|
||||
Each authenticated control session advertises that peer's own direct data
|
||||
links, including the plugin protocol. Tables are separate for each network
|
||||
and protocol; only members offering a matching enabled protocol version enter
|
||||
the graph. A snapshot refreshes on the maintenance interval, expires after
|
||||
90 seconds, and is withdrawn when the session ends. Link arrival, closure and
|
||||
changed announcements rebuild routes immediately. Unchanged announcements
|
||||
refresh their age without republishing the forwarding snapshot.
|
||||
links. There is one table for each network, whatever protocols run over it: a
|
||||
link is a way to reach a neighbour, and a protocol is an end-to-end tunnel to a
|
||||
peer. The envelope names the protocol, so the two are independent. A neighbour
|
||||
reached over several kinds of transport (UDP, TCP, QUIC datagrams) is sent to
|
||||
over the best open one, in the order the protocols were configured, and over
|
||||
the next if that one fails or cannot carry the frame. Every authenticated
|
||||
member enters the graph; a member in the middle needs no protocol in common
|
||||
with either end, only a link to each. A snapshot refreshes on the maintenance
|
||||
interval, expires after 90 seconds, and is withdrawn when the session ends.
|
||||
Link arrival, closure and changed announcements rebuild routes immediately.
|
||||
Unchanged announcements refresh their age without republishing the forwarding
|
||||
snapshot.
|
||||
|
||||
Which protocol a pair of members runs their tunnel in is settled without
|
||||
asking: the member with the lower endpoint id decides, by the order it
|
||||
announces its protocols in, among those both have enabled at the same wire
|
||||
version. Both ends read the same answer from what they already hold. The
|
||||
tunnel is created as soon as the peer is reachable by any path, and it does not
|
||||
change when a direct link comes or goes underneath it: as soon as a direct
|
||||
link comes up it is shorter than the path through others, and traffic moves to
|
||||
it with nothing to redo. The user sees only a lower round-trip time.
|
||||
|
||||
Breadth-first search computes the shortest directed paths and every equal-cost
|
||||
first hop in sorted order. An actual direct link always wins. The first row is
|
||||
@@ -30,14 +45,29 @@ loops if different nodes have not yet received the same topology update.
|
||||
|
||||
The existing control plane still forms authenticated pairwise sessions.
|
||||
Routing provides multihop **data** paths among these members; it does not add
|
||||
control-plane flooding or carry control sessions through the data plane.
|
||||
control-plane flooding or carry control sessions through the data plane, so a
|
||||
member that has no control session with another has no data path to it either.
|
||||
Announcements are volatile, not durable membership or availability guarantees.
|
||||
|
||||
## What "online" means
|
||||
|
||||
One definition, computed by the agent and reported identically by `tsng status`
|
||||
and the tray: a member is online when this agent can reach it by any path — a
|
||||
live authenticated control session (QUIC keep-alive every 5 seconds, so a
|
||||
vanished peer is noticed within about 15), or a data path to it through other
|
||||
members. It is never inferred from ping, which a firewall may block, or from a
|
||||
tunnel's last handshake, which says only when traffic last flowed.
|
||||
|
||||
How it is reached is reported separately: `direct` over a named transport,
|
||||
`relayed` through a named member over N hops, or no data path yet while one is
|
||||
being set up. Being relayed is not being degraded in any way but distance, and
|
||||
is shown so it is not a surprise.
|
||||
|
||||
## Packet path
|
||||
|
||||
The control loop binds next hops to transport handles in an immutable table
|
||||
and publishes it through `ArcSwap`. Readers do not acquire the topology mutex.
|
||||
The transit routine validates the fixed 74-byte envelope, looks up the source
|
||||
The transit routine validates the fixed 82-byte envelope, looks up the source
|
||||
and destination, selects a cached next hop, decrements one byte, and calls the
|
||||
transport's synchronous datagram send. It never searches the graph, formats an
|
||||
endpoint string, validates an Ed25519 key, decrypts, or touches TUN.
|
||||
@@ -86,5 +116,5 @@ counts sends without storing frames. Results are CPU forwarding cost, **not**
|
||||
end-to-end network latency or VPN throughput; encryption, fragmentation,
|
||||
sockets, congestion and scheduling contribute separately.
|
||||
|
||||
Wire compatibility: control ALPN `tsunagi/ctrl/4`, data ALPN `tsunagi/data/4`.
|
||||
Wire compatibility: control ALPN `tsunagi/ctrl/5`, data ALPN `tsunagi/data/5`.
|
||||
Upgrade every participant together; saved identities and network state persist.
|
||||
+19
-1
@@ -43,7 +43,9 @@ document trusted merely because the neighbour who forwarded it signed it.
|
||||
- Duplicates do not change the result.
|
||||
- Two conflicting signed records at the same version from the same author need
|
||||
explicit handling; they are not resolved by luck.
|
||||
- Neither arrival order nor system clocks decide a winner.
|
||||
- Neither arrival order nor the receiving replica's clock decides a winner. The
|
||||
one thing an author's own clock is used for is the time it first claimed a
|
||||
hostname, signed into its record (see below).
|
||||
- Compaction must not drop what is needed to stop revoked records being
|
||||
resurrected.
|
||||
|
||||
@@ -53,6 +55,22 @@ A hostname is a mutable binding to a persistent author, not an identity. A
|
||||
rename must be a signed record that revokes the specific old binding and
|
||||
announces the new one, ideally atomically in one record.
|
||||
|
||||
A hostname belongs to the member that claimed it **first**. Each claim carries
|
||||
`hostname_since`, the author's own time in milliseconds when it first claimed
|
||||
that exact name, signed with the rest of the record and kept across every later
|
||||
version that keeps the name; renaming starts it afresh. Replicas order claims
|
||||
by `(hostname_since, author id)`, a pure function of the records, so every
|
||||
replica names the same holder whatever order the records arrived in and there
|
||||
is no race to lose. A member that arrives later with a name already held keeps
|
||||
working by address and keeps its claim, but is not found by that name on any
|
||||
member; it is told so in its status and in `tsng status`, and finds the name
|
||||
again if the holder renames or releases it.
|
||||
|
||||
It is the author's word, so a member with a clock set far back can claim an
|
||||
older time. That can reorder who holds a *name* and nothing else — addresses,
|
||||
keys and everything the data plane trusts are decided without it — and anybody
|
||||
who knows the secret could take a name by other means anyway.
|
||||
|
||||
Turning a computer off is not a revocation of its hostname and does not remove
|
||||
the participant. Revocations are not dropped merely because they are old, and
|
||||
no acknowledgement from offline peers is required to keep working.
|
||||
|
||||
+3
-3
@@ -26,8 +26,8 @@ system can hand us IP packets, and even that is behind a trait
|
||||
| `ManagedTunFactory` | `CAP_NET_ADMIN` | traffic actually reaches the OS |
|
||||
|
||||
`ManagedTunFactory` creates the interface and configures it; see
|
||||
*Provisioning the interface* below and *Privileges* in
|
||||
[../README.md](../README.md#privileges). With no capability the agent runs
|
||||
*Provisioning the interface* below and Privileges in
|
||||
[../dist/linux/README.md](../dist/linux/README.md). With no capability the agent runs
|
||||
with `--no-tun`: everything but the last hop into the kernel still works.
|
||||
|
||||
[boringtun]: https://docs.rs/boringtun
|
||||
@@ -184,7 +184,7 @@ QUIC DATAGRAM frames themselves cannot fragment.
|
||||
Reassembly is bounded and expires incomplete packets; one lost fragment loses
|
||||
one packet, without blocking unrelated traffic. The logical data payload limit
|
||||
is 64 KiB, with the relay envelope subtracted before it reaches the plugin.
|
||||
The new framing uses data ALPN `tsunagi/data/4`; both ends and intermediate
|
||||
The new framing uses data ALPN `tsunagi/data/5`; both ends and intermediate
|
||||
peers need the updated binary. Network identities and saved state do not change.
|
||||
See [protocol.md](protocol.md#the-data-plane-protocol) for the wire format.
|
||||
|
||||
|
||||
Generated
+27
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"nodes": {
|
||||
"nixpkgs": {
|
||||
"locked": {
|
||||
"lastModified": 1790769526,
|
||||
"narHash": "sha256-5AIVFLRcx8YAwbFTVPTs7YhwJH7JyHfAQsvXmpovcXI=",
|
||||
"owner": "NixOS",
|
||||
"repo": "nixpkgs",
|
||||
"rev": "c9fe7d12cd78d1adcd12dd15e24432dde5b155a0",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "NixOS",
|
||||
"ref": "nixpkgs-unstable",
|
||||
"repo": "nixpkgs",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"root": {
|
||||
"inputs": {
|
||||
"nixpkgs": "nixpkgs"
|
||||
}
|
||||
}
|
||||
},
|
||||
"root": "root",
|
||||
"version": 7
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
{
|
||||
description = "tsunagi: serverless mesh VPN. A network name and a secret are all it takes.";
|
||||
|
||||
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
|
||||
|
||||
outputs =
|
||||
{ self, nixpkgs }:
|
||||
let
|
||||
systems = [
|
||||
"x86_64-linux"
|
||||
"aarch64-linux"
|
||||
];
|
||||
forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f nixpkgs.legacyPackages.${system});
|
||||
in
|
||||
{
|
||||
packages = forAllSystems (pkgs: rec {
|
||||
tsunagi = pkgs.callPackage ./nix/package.nix { };
|
||||
tsunagi-cli = pkgs.callPackage ./nix/package.nix { withTray = false; };
|
||||
default = tsunagi;
|
||||
});
|
||||
|
||||
overlays.default = final: _prev: {
|
||||
tsunagi = final.callPackage ./nix/package.nix { };
|
||||
tsunagi-cli = final.callPackage ./nix/package.nix { withTray = false; };
|
||||
};
|
||||
|
||||
nixosModules.default = import ./nix/module.nix self;
|
||||
|
||||
devShells = forAllSystems (pkgs: {
|
||||
default = pkgs.mkShell {
|
||||
inputsFrom = [ self.packages.${pkgs.stdenv.hostPlatform.system}.tsunagi ];
|
||||
packages = [
|
||||
pkgs.rustfmt
|
||||
pkgs.clippy
|
||||
];
|
||||
};
|
||||
});
|
||||
};
|
||||
}
|
||||
+132
@@ -0,0 +1,132 @@
|
||||
flake:
|
||||
{
|
||||
config,
|
||||
lib,
|
||||
pkgs,
|
||||
...
|
||||
}:
|
||||
|
||||
let
|
||||
cfg = config.services.tsunagi;
|
||||
socket = "/run/tsunagi/agent.sock";
|
||||
in
|
||||
{
|
||||
options.services.tsunagi = {
|
||||
enable = lib.mkEnableOption "the tsunagi mesh agent";
|
||||
|
||||
package = lib.mkOption {
|
||||
type = lib.types.package;
|
||||
default =
|
||||
if cfg.tray.enable then
|
||||
flake.packages.${pkgs.stdenv.hostPlatform.system}.tsunagi
|
||||
else
|
||||
flake.packages.${pkgs.stdenv.hostPlatform.system}.tsunagi-cli;
|
||||
defaultText = lib.literalExpression "tsunagi.packages.\${system}.tsunagi (tsunagi-cli without the tray)";
|
||||
description = "The tsunagi package to run.";
|
||||
};
|
||||
|
||||
tray.enable = lib.mkEnableOption "the tray client, started with every graphical session";
|
||||
|
||||
users = lib.mkOption {
|
||||
type = lib.types.listOf lib.types.str;
|
||||
default = [ ];
|
||||
example = [ "alice" ];
|
||||
description = ''
|
||||
Users allowed to talk to the agent without sudo (`tsng status`, `tsng join`,
|
||||
the tray). They are added to the `tsunagi` group.
|
||||
'';
|
||||
};
|
||||
|
||||
protocols = lib.mkOption {
|
||||
type = lib.types.nullOr (lib.types.listOf lib.types.str);
|
||||
default = null;
|
||||
example = [
|
||||
"wg-quic"
|
||||
"wg"
|
||||
];
|
||||
description = "Packet protocols for `tsng up --protocol`, best first. Null keeps the agent's default.";
|
||||
};
|
||||
};
|
||||
|
||||
config = lib.mkIf cfg.enable {
|
||||
users.groups.tsunagi = { };
|
||||
users.users = {
|
||||
tsunagi = {
|
||||
isSystemUser = true;
|
||||
group = "tsunagi";
|
||||
home = "/var/lib/tsunagi";
|
||||
description = "tsunagi agent";
|
||||
};
|
||||
}
|
||||
// lib.genAttrs cfg.users (_: {
|
||||
extraGroups = [ "tsunagi" ];
|
||||
});
|
||||
|
||||
systemd.services.tsunagi = {
|
||||
description = "tsunagi mesh agent";
|
||||
documentation = [ "https://github.com/house-of-vanity/tsunagi" ];
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
wants = [ "network-online.target" ];
|
||||
after = [ "network-online.target" ];
|
||||
environment = {
|
||||
TSUNAGI_STATE_DIR = "/var/lib/tsunagi";
|
||||
TSUNAGI_CACHE_DIR = "/var/cache/tsunagi";
|
||||
TSUNAGI_CONTROL_SOCKET = socket;
|
||||
TSUNAGI_CONTROL_GROUP = "tsunagi";
|
||||
};
|
||||
serviceConfig = {
|
||||
ExecStart = lib.concatStringsSep " " (
|
||||
[ "${cfg.package}/bin/tsng up" ]
|
||||
++ lib.optional (cfg.protocols != null) "--protocol ${lib.concatStringsSep "," cfg.protocols}"
|
||||
);
|
||||
User = "tsunagi";
|
||||
Group = "tsunagi";
|
||||
AmbientCapabilities = [
|
||||
"CAP_NET_ADMIN"
|
||||
"CAP_NET_BIND_SERVICE"
|
||||
];
|
||||
CapabilityBoundingSet = [
|
||||
"CAP_NET_ADMIN"
|
||||
"CAP_NET_BIND_SERVICE"
|
||||
];
|
||||
StateDirectory = "tsunagi";
|
||||
StateDirectoryMode = "0700";
|
||||
CacheDirectory = "tsunagi";
|
||||
RuntimeDirectory = "tsunagi";
|
||||
RuntimeDirectoryMode = "0755";
|
||||
Restart = "on-failure";
|
||||
RestartSec = 5;
|
||||
};
|
||||
};
|
||||
|
||||
# Exit nodes: the agent looks for iptables only in fixed FHS directories.
|
||||
systemd.tmpfiles.rules = [
|
||||
"L+ /usr/bin/iptables - - - - ${pkgs.iptables}/bin/iptables"
|
||||
];
|
||||
|
||||
# Names of mesh members are served locally and handed to the system resolver.
|
||||
services.resolved.enable = lib.mkDefault true;
|
||||
|
||||
# polkit decides by group, not capability, who may configure systemd-resolved.
|
||||
security.polkit.extraConfig = ''
|
||||
polkit.addRule(function(action, subject) {
|
||||
var allowed = [
|
||||
"org.freedesktop.resolve1.set-dns-servers",
|
||||
"org.freedesktop.resolve1.set-domains",
|
||||
"org.freedesktop.resolve1.set-default-route",
|
||||
"org.freedesktop.resolve1.revert"
|
||||
];
|
||||
if (allowed.indexOf(action.id) >= 0 && subject.isInGroup("tsunagi")) {
|
||||
return polkit.Result.YES;
|
||||
}
|
||||
});
|
||||
'';
|
||||
|
||||
environment.systemPackages = [ cfg.package ];
|
||||
environment.sessionVariables.TSUNAGI_CONTROL_SOCKET = socket;
|
||||
|
||||
environment.etc."xdg/autostart/tsunagi-tray.desktop" = lib.mkIf cfg.tray.enable {
|
||||
source = "${cfg.package}/share/applications/tsunagi-tray.desktop";
|
||||
};
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
{
|
||||
withTray ? true,
|
||||
lib,
|
||||
rustPlatform,
|
||||
pkg-config,
|
||||
copyDesktopItems,
|
||||
wrapGAppsHook3,
|
||||
gtk3,
|
||||
libayatana-appindicator,
|
||||
libglvnd,
|
||||
libxkbcommon,
|
||||
vulkan-loader,
|
||||
wayland,
|
||||
xdotool,
|
||||
libx11,
|
||||
libxcursor,
|
||||
libxi,
|
||||
libxrandr,
|
||||
}:
|
||||
|
||||
let
|
||||
# The tray is an egui app that loads graphics, windowing and appindicator
|
||||
# libraries at run time, so they have to be put on its library path by hand.
|
||||
trayRuntimeLibs = [
|
||||
libayatana-appindicator
|
||||
libglvnd
|
||||
libxkbcommon
|
||||
vulkan-loader
|
||||
wayland
|
||||
libx11
|
||||
libxcursor
|
||||
libxi
|
||||
libxrandr
|
||||
];
|
||||
in
|
||||
rustPlatform.buildRustPackage {
|
||||
pname = if withTray then "tsunagi" else "tsunagi-cli";
|
||||
version = "0.1.0";
|
||||
|
||||
src = lib.cleanSource ../.;
|
||||
cargoLock.lockFile = ../Cargo.lock;
|
||||
|
||||
cargoBuildFlags = [
|
||||
"--bin"
|
||||
"tsng"
|
||||
]
|
||||
++ lib.optionals withTray [
|
||||
"--bin"
|
||||
"tsunagi-tray"
|
||||
];
|
||||
|
||||
# The suite opens real sockets and interfaces; it runs in CI instead.
|
||||
doCheck = false;
|
||||
|
||||
nativeBuildInputs = [ pkg-config ] ++ lib.optionals withTray [
|
||||
copyDesktopItems
|
||||
wrapGAppsHook3
|
||||
];
|
||||
buildInputs = lib.optionals withTray [
|
||||
gtk3
|
||||
libayatana-appindicator
|
||||
xdotool
|
||||
];
|
||||
|
||||
dontWrapGApps = true;
|
||||
|
||||
postInstall = ''
|
||||
install -Dm644 dist/linux/50-tsunagi-resolved.rules \
|
||||
"$out/share/doc/tsunagi/50-tsunagi-resolved.rules"
|
||||
''
|
||||
+ lib.optionalString withTray ''
|
||||
install -Dm644 dist/linux/tsunagi.svg \
|
||||
"$out/share/icons/hicolor/scalable/apps/tsunagi.svg"
|
||||
install -Dm644 dist/linux/tsunagi-tray.desktop \
|
||||
"$out/share/applications/tsunagi-tray.desktop"
|
||||
'';
|
||||
|
||||
postFixup = lib.optionalString withTray ''
|
||||
wrapProgram "$out/bin/tsunagi-tray" \
|
||||
"''${gappsWrapperArgs[@]}" \
|
||||
--prefix LD_LIBRARY_PATH : ${lib.makeLibraryPath trayRuntimeLibs}:/run/opengl-driver/lib
|
||||
'';
|
||||
|
||||
meta = {
|
||||
description = "Serverless mesh VPN: join a network with a name and a secret";
|
||||
homepage = "https://github.com/house-of-vanity/tsunagi";
|
||||
license = lib.licenses.wtfpl;
|
||||
mainProgram = "tsng";
|
||||
platforms = [
|
||||
"x86_64-linux"
|
||||
"aarch64-linux"
|
||||
];
|
||||
};
|
||||
}
|
||||
Reference in new issue
Block a user