ab/tsunagi
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Rewrite README as a short project front page; add a Nix flake

Browse Source 
The README is now a landing page: the idea (a mesh VPN with no server, joined
with a network name and a secret), install and platform tables, exit nodes,
and a short build section. Implementation detail stays in docs/.

The flake builds tsng and the tray from source and ships a NixOS module
(services.tsunagi). Dist install notes now show the real `join -n -s` syntax,
and two links that pointed into the old README are redirected.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Aleksandr BogomiakovandClaude committed 2026-10-03 17:47:44 +01:00
1 parent e6ec62bcba
commit c184ef0914
11 files changed
+449 -740
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+1 -1
View File
@@ -8,7 +8,7 @@ before touching the code. Do not restate the full design here; follow the links.
A proof-of-concept agent library for small private mesh networks. A user
supplies a network name and one shared secret; agents derive the same network
space independently, find each other, prove membership and exchange control
messages. Scope and non-scope are in [README.md](README.md).
messages. What it does and does not do is in [docs/architecture.md](docs/architecture.md) and [docs/threat-model.md](docs/threat-model.md).
There is no central server, no owner, no registration and no majority vote.
Design accordingly: a majority is not a root of trust.
README.md
+150 -733
View File
@@ -1,766 +1,183 @@
# tsunagi
A proof-of-concept Rust library for **small private mesh networks** — a handful
of friends, home machines, a few servers. Units to dozens of participants, not
thousands.
The end user configures exactly two things:
```text
network_name
secret # one shared secret; "password" and "secret" mean the same value
```
From those, every agent independently derives the same network space. There is
no central server, no network owner with special powers, no registration and no
majority vote. Anyone who knows the parameters can join; nobody has to trust
anybody else.
## What this proof of concept actually does
A working library with **real iroh connections** and integration tests:
- persistent device identity stored in SQLite, stable across restarts;
- several independent networks at once in one agent;
- deterministic network identity derived from name + secret;
- candidates supplied by a replaceable discovery component;
- automatic first contact through the public Mainline DHT (BEP44);
- real iroh connections plus an explicit mutual proof of network membership;
- a small versioned control protocol: handshake, hostname/capability
announcement, ping/pong;
- automatic reconnect with bounded exponential backoff and jitter;
- status snapshots, an event stream and honest diagnostics;
- configuration restored after a restart;
- correct behaviour when the disposable cache is missing or corrupt;
- a **WireGuard data plane**, in userspace: its own key per network, an IPv6
overlay with deterministically derived addresses and optional IPv4, real
tunnels carried over iroh, and address ownership enforced rather than
believed;
- a **command line agent**, `tsng`, with a local control socket.
### What it deliberately does **not** do
Not implemented, and not pretended to be: DNS, routing through
intermediate participants, a full CRDT, dynamically loaded plugins, a system
service, a complete CLI, or a local control socket. Snapshot synchronisation
and signed revocations are designed for but not implemented — see
[docs/sync-model.md](docs/sync-model.md). The WireGuard plugin's own limits,
including that its system backend is Linux-only, are in
[docs/wireguard.md](docs/wireguard.md#limits-and-future-work).
**Control and data are separated logically, not physically.** Both ride on
iroh, on different ALPNs and different connections, so the data plane inherits
iroh's hole punching and relay fallback instead of reimplementing them — while
the control protocol still knows nothing about packets and can keep a different
transport underneath it later. Filtering user traffic remains the operating
system's and the user's responsibility, not this library's.
## Requirements
- Rust 1.91 or newer (iroh 1.2 requires it), edition 2024. Pinned dependencies
in `Cargo.lock`.
- No internet, no public DHT, no public relay, no administrator rights and no changes
to OS network settings are needed to build or test.
- WireGuard runs in userspace (boringtun): **no kernel module and no `wg`
tool**. Only creating a real network interface needs `CAP_NET_ADMIN`, and
`--no-tun` skips even that.
## Trying it on two machines
On the first machine:
```bash
cargo build --release
./target/release/tsng status # this device, the agent, and this host
./target/release/tsng network # the networks this device belongs to
```
Two commands do the work, and they are separate on purpose: `up` runs the
agent — one per device, for as long as you want it — and `join` decides
what it belongs to, at any time, while it runs.
```bash
# Terminal one: the agent. It prints its endpoint id and then serves.
./target/release/tsng up
# Terminal two: make a network. With no --secret it invents one and prints
# it, along with the line to send the other machine.
./target/release/tsng join --network lab
```
On the second machine, start its agent and join the same network with the
secret that was printed:
```bash
./target/release/tsng up
./target/release/tsng join --network lab --secret tsn1...
```
**Name and secret are enough for the first meeting.** Mainline DHT discovery
is enabled by default. Each active network publishes this device's endpoint
and searches until the first authenticated connection. While connected it
only republishes; after 60 seconds without any authenticated connection it
searches again. Publication is approximately every five minutes, and records
older than 15 minutes are ignored. DHT storage nodes may retain them longer.
Use `--no-dht` to disable DHT, or `--peer <endpoint-id>` for optional manual
bootstrap. `--reach local` disables public DHT regardless of the DHT flag.
Endpoint records are public, signed, and contain no network secret. Discovery
is not authentication: every connection must still prove network membership.
Details and failure behavior are in [docs/mainline-dht.md](docs/mainline-dht.md).
**One introduction is enough for the whole network.** Members tell each
other about the members they know, and every author of a signed record is
somebody to try, so a device pointed at one member ends up talking to all
of them rather than to the one that happened to be on its command line.
What travels is a candidate — an address somebody has seen — and it is
authenticated by the handshake like any other; being introduced grants
nothing. It is deliberately the same shape a lookup in a distributed hash
table would return, so that is a source to add beside this one rather than
a redesign.
An agent with nobody to contact says so in `status` rather than sitting
there looking patient.
Within a few seconds both print something like:
```text
+ peer b47c958462 connected over direct rtt=Some(4.5ms)
+ data link to b47c958462 for wg-quic: direct via 192.0.2.7:41234, datagram 1382
--- status ---
control: 1 peer(s), 0 dial failure(s), 0 handshake failure(s)
wg-quic: tsun0 on 10.13.37.69/24 mtu 1280, 1/1 tunnel(s) established
4jO4kx9Z 10.13.37.237 handshake 3s ago tx=0 rx=0 dropped=0 path=direct via 192.0.2.7:41234
```
`1/1 tunnel(s) established` means a real WireGuard handshake completed.
## Checking that it works
From another shell on either machine:
```bash
tsng status
```
```text
endpoint 7d76ccbbc21bf30767e14422c0494740a2cecc02aa9f82d5b8d57bdae350e7fc
hostname tsunagi-7d76ccbbc2
bound 0.0.0.0:41641
network lab (z2o4qwrvnj3zb6st2aoqg4abf342j662q2ujttqrsmz22argk2ba) active
peer b345d5271b tsunagi-b345d5271b Direct rtt 24ms
overlay tsunz2o4qwrvnj3 fd09:…:c1c6/64 and 100.110.49.177 mtu 1280 1/1 tunnel(s) up
SDsEb/WF fd09:…:c4b / 100.65.243.53 handshake 4s ago tx 0 rx 0 Direct via Ip(…)
```
`1/1 tunnel(s) up` and a recent handshake mean the tunnel is live. Then send
real traffic to the peer's overlay address:
```bash
ping6 fd09:…:c4b # or
ping 100.65.243.53
```
`tx` and `rx` in the status should start moving.
**Addresses are allocated and then remembered.** The default range is
`10.13.37.0/24`; the first member to join settles it and later members adopt
what they find, so `--ipv4-range` only matters for whoever starts the network:
```bash
tsng up --ipv4-range 10.44.0.0/16
tsng up --ipv4-range none # no data plane at all
```
One agent has one interface, so two of its networks cannot both use that
range. The second takes the range **derived from its own network id**: not
picked locally — every member derives the same one from something they all
already have — so it is an agreement rather than a guess, and a device in
several networks gets an address in each.
An address is claimed with a record signed by that member's persistent device
key, stored, and merged between every replica. A member that disappears for a
month comes back to the same address, because the claim outlived the session.
No vote is involved — see
[docs/wireguard.md](docs/wireguard.md#ipv4-allocated-signed-and-kept) and
[docs/sync-model.md](docs/sync-model.md).
Because the address is allocated at run time rather than derived, it is not
known until the agent has started and agreed with its peers. The agent then
assigns it to the interface itself.
## Levels
The command line is split the way the design is. A bare `tsng up` needs
only a network name and a secret; everything else sits under the level it
belongs to, which `--help` shows as two sections:
* **System** — what the agent itself does: how it reaches peers (`--reach`),
the one overlay interface it owns (`--interface`, `--mtu`, `--no-tun`), the
address range (`--ipv4-range`), and the local resolver (`--no-dns` to disable).
* **Transport** — which protocols carry packets (`--protocol`, a list) and
their own settings (`-o key=value`, or `-o protocol:key=value`).
```bash
tsng protocols # what this build can carry packets with
```
```text
wg-quic (wire version 4)
what WireGuard's cryptography carried in iroh's QUIC datagrams
-o keepalive=SECONDS keeps a tunnel and its link warm through a NAT
-o mtu=BYTES largest packet a tunnel will carry, at least 576
```
Each protocol declares its own settings, so the agent can list them without
knowing anything about the protocol, and a setting no selected protocol takes
is refused rather than ignored. `--protocol none` runs the control plane by
itself.
A pair of peers uses a protocol they both have **at the same wire version**.
That is not the software version: two peers on different builds carry traffic
for each other for as long as the bytes between them have not changed. A peer
with nothing in common keeps its control plane — messages and signed state
still flow — and simply has no data plane, which `tsng status` shows as a
session with no agreed protocol.
## Names
DNS is enabled by default and serves a local zone for **every network this device is in**,
each named after the network, so members can be reached by name instead of
by address:
```bash
tsng up # DNS is already enabled
dig @127.0.0.1 -p 5354 music.lab
```
An explicit choice is remembered across restarts. `--no-dns` or `tsng dns off`
disables it; `--dns` or `tsng dns on` enables it again. Existing saved opt-outs
remain respected. The `dns on` and `dns off` commands also take effect immediately
on a running agent. `--dns` and `--no-dns` cannot be combined.
The zone of a network is its name — `--dns-zone` is gone, because with
several networks there is no single zone to name. A network name may contain
dots, so `--network lab.internal` is how you get `music.lab.internal`.
Names come from signed state, which is the point: **a member that is
switched off still resolves**, because its claim outlived the session.
The answers are IPv4 addresses, because that is what the overlay is. The
*questions* are taken on `127.0.0.1` and `[::1]`, over UDP and TCP, so a
resolver reaches it over whichever family it uses; both are published to the
system resolver together. Loopback and nothing else: the zones are a view
for the host running the agent, and binding an overlay address would put
them in front of the whole mesh — an agent in two networks would then answer
one network's questions about the other's names.
The zone is the network name, and it is yours to choose. A name that
shadows a real public domain is reported and then used: a network called
`ru` warns that every public `.ru` name becomes unreachable from this host,
and then does it. Anything that collides with nothing — which is most names
— is said nothing about, because a warning that fires on every private name
anybody picks is how people learn to ignore warnings.
On Linux the agent tells systemd-resolved to send questions for that suffix
here, over D-Bus, scoped to the overlay interface and as a *routing* domain
so it never becomes the resolver for anything else. resolved drops the whole
setting when the interface goes, and the interface goes with the agent.
Only an interface the agent created, though. Under `--no-tun` there is no
host interface at all, and the agent says so and serves the zone on
loopback rather than configuring whatever else on the host happens to share
the name.
That last step needs permission that `CAP_NET_ADMIN` does not give:
systemd-resolved asks polkit, and polkit decides by **user**, not by
capability, so there is no way for the agent to arrange it from inside. On
a desktop the refusal reads `Interactive authentication required`.
Running as a system service is enough. Otherwise the agent prints the rule
that grants it — the four actions it calls and nothing else — ready to
paste:
```bash
sudo tee /etc/polkit-1/rules.d/50-tsunagi-resolved.rules > /dev/null <<'RULE'
polkit.addRule(function(action, subject) {
var allowed = [
"org.freedesktop.resolve1.set-dns-servers",
"org.freedesktop.resolve1.set-domains",
"org.freedesktop.resolve1.set-default-route",
"org.freedesktop.resolve1.revert"
];
if (allowed.indexOf(action.id) >= 0 && subject.user == "YOUR-USER") {
return polkit.Result.YES;
}
});
RULE
```
**Without it the server still runs** — `tsng status` prints where it is
listening and the exact `dig` line — so the automatic part is missing, not
the feature. The refusal is said once rather than on every pass, and retried
slowly, because nothing but a person will change it.
The server is authoritative for its zone and nothing else. No recursion, no
forwarding, no cache: pointing a resolver at it can never make it a route to
the outside.
## Privileges
On Linux the agent **manages its own overlay interface**. It creates the TUN
interface, sets the MTU, brings it up and assigns both overlay addresses, all
over netlink in process — no `ip` invocation, no shell, nothing that a
remote peer could influence.
That needs `CAP_NET_ADMIN`, granted once:
```bash
sudo setcap cap_net_admin+p /usr/local/bin/tsng
```
`+p` rather than `+ep`: the capability is then *permitted* but not
*effective*, and the agent raises it only around the handful of netlink calls
that need it — a few milliseconds at startup, and again if its address
allocation changes. Everything else, including every byte from the network,
is handled with it lowered. `+ep` works too; the agent lowers it on the way
in.
`tsng status` says which of these applies on the host it runs on, along
with what the agent is doing. It grades each finding: **ok** for what works,
**warn** for what the agent runs without and you can fix from the line it
prints, **FAIL** for what it cannot work around. The words carry the grade as
well as the colour, so the report reads the same piped to a file or on a
terminal without colour, and it honours `NO_COLOR`.
Members are listed online first, then the ones that are away. A member that
is away is reported plainly rather than flagged: in a mesh of laptops it is
the ordinary condition, not a fault. The signed state is what makes that
sayable — it remembers who belongs while they are gone, so the report can say
"offline, 10.13.37.99 still reserved for it" instead of leaving a
dial-failure counter to imply it. Counters are history and are never graded:
a peer that left and came back should not leave the report looking broken.
The other two commands split along the line the system itself draws. `id` is
this **device**: the key it signs with and the name it answers to. `network`
is what it **belongs to**: which networks, their secrets, joining and
leaving. A device outlives every network it is in, and a network outlives
any device in it, so a command that mixed them had to be read twice.
Every item takes the same shape, so there is nothing to remember: name it to
see it, name it with a value to change it.
```
tsng id everything about this device
tsng id hostname the name it answers to
tsng id hostname mango change it
tsng id key the key it signs with
tsng id key rotate replace that key
tsng network the networks this device belongs to
tsng network join -n lab -s tsn1… join one; adds it to a running agent
tsng network join -n lab resume one this device has, or make it on the spot
tsng network stop lab stop serving it, keeping everything
tsng network start lab serve it again, from where it left off
tsng network leave lab give up the address and name, then forget it
tsng network secret the secret of each joined network
tsng network secret lab just that one, for copying
tsng network secret generate a fresh secret for a network that does not exist yet
tsng dns whether the local resolver is serving, and what
tsng dns on start it, now and after every restart
tsng dns off stop it, now and after every restart
```
**A network without a secret makes one.** `tsng join --network lab`
resolves a bare name in the obvious way: if this device is already in
exactly one network called `lab`, that one — so the name alone resumes what
you have; if it is in none, a fresh random secret, printed in full with the
line to send the other machine:
```
joined `lab` (k2on43wadbi5x267vp6z3ogkm7nbjedfdoxyauxhtxhwqrprylba)
<p align="center">
<img src="dist/linux/tsunagi.svg" width="96" alt="">
</p>
<h1 align="center">tsunagi</h1>
<p align="center">
<b>A mesh VPN with no server.</b><br>
Pick a network name, share a secret, and your machines are on one private network.
</p>
<p align="center">
<a href="https://github.com/house-of-vanity/tsunagi/releases/latest"><img alt="release" src="https://img.shields.io/github/v/release/house-of-vanity/tsunagi?include_prereleases"></a>
<a href="https://github.com/house-of-vanity/tsunagi/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/house-of-vanity/tsunagi/actions/workflows/ci.yml/badge.svg"></a>
<a href="LICENSE"><img alt="license" src="https://img.shields.io/badge/license-WTFPL-blue"></a>
</p>
```console
laptop$ tsng join -n home
joined `home`
secret tsn1u7c…
Run this on the other machine:
server$ tsng join -n home -s tsn1u7c…
server$ tsng id hostname server
tsng join --network lab --secret tsn1u7c…
Its agent has to be running. If it is not:
tsng up --peer 91e83a6e2b7a…
laptop$ ssh server.home
```
Two networks of one name is a thing that happens — a mistyped secret makes
one — so joining says whether the network was already here, and warns when
another configured network answers to the same name. A name is a label and
the id is the identity.
No account, no coordination server, no admin console, no keys to approve.
Anyone who knows the **name** and the **secret** is in. Nobody else can find
the network, let alone enter it.
That is the ad-hoc case: one person makes a network and sends the command
round. The secret is printed *only* when the agent invented it — there is
nowhere else to read it from — and never when it was supplied, because then
it is already yours. Two networks of one name and no secret is the one case
with no answer, and it says so instead of choosing.
## Why
A secret is printed by `network secret` and nowhere else — not by `id`, not
by `status`, not in a log, a `Debug` rendering or anything sent to a peer.
Asking for it is deliberate, because these reports get pasted into chats.
Tailscale, Headscale and ZeroTier are great, but each needs a control server:
theirs, or one you have to run and keep alive. tsunagi doesn't. Machines find
each other through the public BitTorrent DHT, connect directly (hole punching,
with relay fallback), and keep the network state among themselves.
**`up` takes no network at all** if you would rather decide later: it
brings up the agent and whatever it is already configured for, and waits.
That is the shape of a daemon in one terminal and `tsng network join`
in another.
| | Tailscale | Headscale | ZeroTier | **tsunagi** |
| ---------------------- | ------------------ | ---------------- | ------------------- | ------------------- |
| Control server | hosted by vendor | you run it | hosted or self-run | **none** |
| Account / sign-in | yes | yes | yes | **no** |
| To join a network | sign in, get approved | pre-auth key | network ID, approval | **name + secret** |
| Direct connections | yes | yes | yes | yes |
| Exit nodes | yes | yes | via routes | yes |
`network join` is also the answer to a question `up` cannot: a state
directory belongs to one live agent, so a second `tsng up` cannot add a
network to the one already running. This adds it over the control socket and
it starts at once. With no agent running it is written to the configuration
and starts with the next `up`.
## Features
The name is part of the signed state, so changing it revokes the previous
one: there is one record per author, a new version replaces the whole claim,
and no replica can keep the old name standing. Changing it while the agent
runs goes through the agent, which republishes and tells its peers straight
away.
- **Two things to know:** a network name and a secret. A secret is generated for you when you make a network.
- **Direct, encrypted links** between members (WireGuard cryptography), with relay fallback behind strict NATs.
- **Names for machines:** `server.home` just works, even while `server` is switched off.
- **Exit nodes:** send all your internet traffic through any member that offers one.
- **Multi-hop:** members that can't reach each other directly are routed through a member that can.
- **LAN games:** UDP broadcasts are relayed, so "LAN" lobbies show up across the mesh.
- **Several networks at once** on one device.
- **Tray app** for Windows, macOS and Linux; a command line for everything.
Replacing the signing key makes this device a different member, and it loses
the address and name the old key held — nothing can sign on a retired key's
behalf, and by design there is no authority that could overrule an author. So
the outgoing key signs a release for every network on its way out, which
frees them for whoever wants them next, and the whole thing commits at once.
## Install
`status` and `id` both prefer a running agent, which is live and
authoritative, and fall back to reading the state store when there is none.
Reading takes no directory lock, so neither has to wait for the agent it is
asking about — nor does either need one to be running.
Grab a build from the [latest release](https://github.com/house-of-vanity/tsunagi/releases/latest).
### It cleans up after itself
| Platform | How |
| --------------------- | -------------------------------------------------------------------------------- |
| **Windows** | Run `tsunagi-setup-<version>-x86_64.exe`. Installs the service, the tray and `tsng`. |
| **macOS** | `brew tap house-of-vanity/tap && brew install --cask tsunagi-gui`<br>CLI only: `brew install tsunagi` |
| **Debian / Ubuntu** | `sudo apt install ./tsunagi-gui_<version>_amd64.deb` (`tsunagi_…` for no tray) |
| **Arch** | `sudo pacman -U tsunagi-gui-<version>-1-x86_64.pkg.tar.zst` |
| **NixOS** | [Flake and module](#nixos) below |
| **Any other Linux** | Unpack the tarball, then `sudo ./install.sh && sudo systemctl enable --now tsunagi` |
The interface is tied to an open file descriptor and is deliberately **not**
made persistent, so the kernel removes it when the agent exits — on a clean
shutdown, on a panic, on `SIGKILL`, on power loss alike. Keeping it is what
would take an action; removing it is the default.
Linux packages run the agent as an unprivileged service user. Add yourself to
the `tsunagi` group (`sudo usermod -aG tsunagi $USER`, then log in again) to
use `tsng` and the tray without `sudo`.
If something is left behind anyway — an interface made by an older version's
manual recipe, or one from a run killed in the instant between creating it and
recording it — the next start **replaces it**, along with any stale addresses
it carried. Two things are never touched:
* an interface that is not a TUN, because the name colliding with somebody's
bridge is not a reason to destroy the bridge;
* a TUN that another process is holding open, because that is a working
overlay belonging to somebody else — most likely a second agent on this
host, which should be given a different `--wg-prefix`.
Both of those refuse with an explanation rather than guessing.
Two settings the manual recipe used to need are gone with it.
`keep_addr_on_down` existed only because an interface nobody held open lost
carrier and had its IPv6 addresses flushed, and `nodad` only because duplicate
address detection can never finish without carrier. An interface held open for
its whole life has carrier for its whole life.
### The MTU is 1280
The default interface MTU stays at 1280 on direct paths, relays and paths
inside another VPN. Large encrypted packets are split into smaller QUIC
datagrams and reassembled before reaching WireGuard. SSH and full-size TCP
segments therefore do not require a manual MTU override or MSS adjustment.
The original IP packet, including its DF bit, is preserved. See
[docs/wireguard.md](docs/wireguard.md#mtu) for limits and compatibility.
### Summary
| approach | agent runs as | notes |
|---|---|---|
| `setcap cap_net_admin+p` | ordinary user, one capability | recommended: nothing to prepare, nothing left behind. Lost on every rebuild or copy of the binary. |
| systemd service | `User=`, `AmbientCapabilities=CAP_NET_ADMIN` | the same, for an installed service |
| `sudo tsng up` | root | everything works, nothing is isolated |
| macOS root LaunchDaemon | root | the native macOS deployment; see below |
| `--no-tun` | ordinary user, no capabilities | tunnels run and handshake, traffic never reaches the OS |
On Windows, put `wintun.dll` beside the executable and start `tsng up`
from PowerShell or Command Prompt opened with **Run as administrator**.
Run commands controlling that agent (`join`, `status`, `dns`, `network`)
as the same Windows user with the same elevation. If Windows denies access
to the agent's control pipe, the command reports a permission error instead
of claiming the agent is absent or trying to edit its locked state.
Creating a TUN without sufficient privileges also explains how to restart
the agent with the required permissions.
On macOS the agent creates a `utun` interface, configures it with `ifconfig`
and `route`, and points the system resolver at the overlay by writing
`/etc/resolver/<network>` files (which carry a port, so the DNS server needs no
privileged socket). All of this needs **root**: macOS has no per-capability
grant like Linux, and a `utun` plus `/etc/resolver` are root-only. The kernel
assigns the interface its `utunN` name, so the name derived from the network id
is only a request and the real one is adopted once the interface is up.
Run it as root — `sudo tsng up` for a quick start, or install it as a root
**LaunchDaemon** for the native always-on deployment. As a daemon the agent runs
unprivileged commands (`join`, `status`, `dns`, `network`) over its control
socket like any other platform; the socket lives at `/var/run/tsunagi/agent.sock`
and the stores at `/var/db/tsunagi`, both fixed system paths so they do not
depend on whose shell started `sudo`. A ready-to-edit plist and install notes
are in [dist/macos/](dist/macos/). The broadcast route is installed; the
inbound-UDP firewall allowance (a `pf` anchor) is not yet, so `status` reports
that half as incomplete and names what to allow by hand. The decision logic that
says *what* to change is shared and tested on every platform.
## Checks
```bash
cargo fmt --all -- --check
cargo clippy --locked --workspace --all-targets -- -D warnings
cargo test --locked --workspace --all-targets
```
The whole suite runs offline on loopback. Set `TSUNAGI_TEST_LOG=tsunagi=debug`
to see agent logs while a test runs.
There are also two runnable demos, which are demos and not substitutes for the
tests:
```bash
cargo run --example two_agents # control plane only
```
Both run with no privileges and change nothing on the host.
## Usage
```rust,no_run
use std::sync::Arc;
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
use tsunagi::discovery::SharedMemoryDiscovery;
use tsunagi::identity::{NetworkName, NetworkSecret};
use tsunagi::proto::ControlMessage;
use tsunagi::{Agent, Result};
// The library never starts a runtime, installs a logger, handles signals,
// forks, or calls process::exit. The binary owns all of that.
#[tokio::main]
async fn main() -> Result<()> {
let config = AgentConfig::new(StoragePaths::user_default()?)
.with_transport(TransportPolicy::N0Defaults)
.with_discovery(Arc::new(SharedMemoryDiscovery::new()));
let agent = Agent::spawn(config).await?;
let name = NetworkName::new("kitchen-table")?;
let secret = NetworkSecret::generate(); // 32 random bytes
println!("share this: {}", secret.encode().as_str());
let network = agent.join_network(&name, &secret).await?;
let mut events = agent.subscribe();
tokio::spawn(async move {
while let Ok(event) = events.recv().await {
println!("{event:?}");
}
});
for peer in agent.network_status(network).await?.connected_peers() {
agent
.send(network, peer, ControlMessage::Ping { seq: 1, payload: vec![] })
.await?;
}
agent.shutdown().await;
Ok(())
}
```
`TransportPolicy::LocalOnly` is the default, so a plain `AgentConfig::new` never
reaches the internet by accident. Opt into `DirectOnly` or `N0Defaults`
explicitly.
### Through somebody in the middle
Tsunagi routes the overlay in userspace. A direct data link always wins;
otherwise the shortest available path can pass through several members.
Each authenticated member advertises only its own live, protocol-specific
transport links. The control plane collects these into a graph and replaces
the routing table when connectivity changes; announcements expire after
90 seconds without refresh.
Transit goes transport → router → transport, without entering WireGuard or
the intermediate host's TUN. The payload stays encrypted between the original
endpoints. TUN connects only the local OS to Tsunagi; the kernel needs only
the overlay route through that interface, with no IP forwarding configuration.
A 16-hop limit bounds loops while topology updates converge. Equal shortest
paths use a stable flow hash derived before encryption, so packets of a TCP
or UDP flow keep their path while topology is unchanged. Physical link changes
do not replace end-to-end WireGuard tunnels or change the 1280-byte default MTU.
`status` reports the hop count and first next hop for relayed paths.
The hot path reads an immutable routing snapshot and sends directly to a cached
transport handle. It takes no routing mutex, walks no graph and does not parse
the encrypted payload. See [routing.md](docs/routing.md) for the architecture,
limits and reproducible forwarding microbenchmark.
This wire format requires all members to upgrade together (control ALPN 3,
data ALPN 4); saved identities, network names, secrets and addresses survive.
### LAN game discovery
IPv4 UDP broadcast relay is enabled by default for each network. The IP router
sends one encrypted copy to each participating peer, including through multihop
paths. It supports `255.255.255.255` and the overlay subnet's broadcast address;
received broadcasts never trigger another fanout.
## Quick start
```sh
tsng join -n games --no-broadcast
tsng join -n games --broadcast
tsng network broadcast games off
tsng network broadcast games on
tsng join -n home # first machine: makes the network and prints a secret
tsng join -n home -s tsn1… # every other machine
tsng status # who is online, over which path
```
The choice persists across restarts and a plain `join`. It can be changed while
the agent runs; `status` shows it for each network. While broadcast is on, the
agent routes `255.255.255.255` through the Tsunagi interface and allows inbound
UDP from the overlay range on it (Linux and Windows, needs the same privilege as
the interface), and removes both when it is turned off or the agent stops. Games
bound to a physical address still bypass it. Physical LAN capture/subnet sharing
is not implemented.
See [broadcast.md](docs/broadcast.md) for domain isolation and future LAN gateways.
Every member gets an address in `10.13.37.0/24` and a name like `<host>.home`.
Machines are called `tsunagi-<id>` until you rename them: `tsng id hostname laptop`.
## How peers find each other
### Exit nodes
Two different lookups are involved, and only one of them is this project's:
**1. Resolving one endpoint's address — iroh's, and it works today.**
With `--reach relay` or `--reach direct`, iroh publishes a signed
record of this endpoint's addresses, keyed by its endpoint id, to the public
service run by Number 0 — "n0", the company behind iroh — at `dns.iroh.link`,
over pkarr and DNS, and resolves other endpoints the same way. That is why `--peer <endpoint-id>` works with no address attached:
iroh looks it up. None of that code is ours.
**2. Finding who is in a network — Mainline DHT by default.**
`NetworkDiscovery` supplies unverified candidates. The Mainline backend uses
a signing key derived from the network name and secret to publish signed
BEP44 records containing each writer's endpoint ID and addresses. Its 16
slots provide starting candidates, not a complete membership list or a limit
on network size. After the first authenticated connection, introductions and
signed state reveal the other members. `StaticBootstrap` (`--peer`) remains
available, and tests use either a local Mainline Testnet or in-memory discovery.
What this means in practice:
- With `relay` or `direct`, **your endpoint id and IP addresses are published
to a public third-party service** (Number 0's, unless you change it). They are not secret, and the network secret is
never published, but an observer of that service learns that your endpoint
exists and where it is. `--reach local` publishes nothing.
- With DHT enabled, endpoint records are also published to the public Mainline
DHT. They are signed, not encrypted; membership still requires the secret.
- A relay, when one is needed, sees the volume and timing of your traffic — not
its contents. The default relays are Number 0's, in the US, EU and
Asia-Pacific.
## Storage
Two physically separate SQLite files, placed wherever the library's
configuration says (`StoragePaths`). A future system service supplies its own
paths; tests always use temporary directories.
| file | holds | when damaged |
|----------------|--------------------------------------------------|-------------------------|
| `state.sqlite` | device identity, network configuration, hostname | clear error, never reset |
| `cache.sqlite` | address hints and other recoverable data | discarded and recreated |
The `wg-quic` protocol keeps its own keys in its own store under `wg-quic/`,
because a protocol's keys are neither the iroh identity nor the network
secret.
The command line agent puts everything under the platform's per-user
directories by default; `--state-dir` and `--cache-dir` override them.
One state directory belongs to one live agent, enforced with a real OS file
lock rather than an existence check. That directory *is* the identity: one
agent, one device key, one interface, and as many networks as you like on it
— `tsng network join` adds them to the agent that is already running,
which is why a second `tsng up` on the same directory is refused rather
than made to work. A second agent on the same host is a second identity, and
needs everything of its own: its own state and cache directories, its own
interface name and its own overlay range. The library owns no globals, so
several of them run side by side in one process as readily as in one host.
### Leaving, and starting over
Membership outlives a session, so it also has to be possible to end it.
```bash
tsng network # what this device belongs to
tsng network leave lab # give up the address and the name, then forget it
tsng wipe --yes # remove everything and be a stranger again
```sh
tsng network exit-node home on # on the machine that should be the exit
tsng exit-node # on another one: who offers an exit
tsng exit-node laptop # send everything through it
tsng exit-node off
```
**Stopping is not leaving.** `stop` closes this network's sessions, takes
its address off the interface and keeps it from starting again, and that is
all: the configuration, the secret, the signed state and the protocol key
stay exactly as they are, nothing is announced, and to the others this
device is simply away — an ordinary condition they already handle, with its
address and name still reserved for it. `start` picks it up where it left
off. A network named on the `up` command line is started by that command
whatever its stored state, and the start-up banner says so rather than
letting a `stop` quietly come back.
The tray does the same with a checkbox. An exit node sees everything you send
through it, so use only members you trust.
[Details and per-OS notes](docs/exit-node.md).
`leave` is the other one, and it removes, locally: the configuration and
the secret, this network's signed records, its cached address hints and the
protocol key it used there. What it keeps is this author's version counter
for that network — a rejoin has to continue above the release, or every
replica would treat the new claim as stale. Everything about *other*
networks, and the device identity itself, is untouched.
## Platforms
`leave` publishes a signed `Release` **first**, while the agent is running and
its sessions are up, so the address and the name are freed for the others
instead of staying reserved to a member that has gone. They pass the tombstone
on, so a member that was away hears it from them. With no agent running,
nothing can sign or send it: the command says so and refuses, and `--offline`
drops the network locally while leaving the others holding the old claim. The
protocol key for that network goes too — rejoining is joining, not resuming.
| Platform | Build | CLI `tsng` | Tray | Exit node: offer / use | Packages |
| ------------------------- | ------------ | :--------: | :--: | :--------------------: | ----------------------------------- |
| Linux x86_64 | release | ✔ | ✔ | ✔ / ✔ | `.deb`, `.pkg.tar.zst`, tarball, Nix |
| Linux aarch64 | release | ✔ | –¹ | ✔ / ✔ | `.deb`, `.pkg.tar.zst`, tarball, Nix |
| Linux armv7 | release | ✔ | –¹ | ✔ / ✔ | `.deb`, `.pkg.tar.zst`, tarball |
| macOS, Apple silicon | release | ✔ | ✔ | ✔ / ✔² | Homebrew, tarball |
| macOS, Intel | from source | ✔³ | ✔³ | ✔ / ✔²³ | – |
| Windows x86_64 | release | ✔ | ✔ | ✔ / ✔² | installer, zip |
`leave`, `stop`, `start`, `secret` and `broadcast` accept an exact stored network
name, its full id, or a unique id prefix (including a copied report ellipsis).
Names are case-sensitive. If several networks share a name, or a name matches
another network's id prefix, the command lists matching names and full ids and
changes nothing. Use a full id to choose explicitly; a full id always takes
precedence over a name. No secret is needed to select a configured network.
<sup>¹ Not in the release; builds from source (and through Nix).
² Newer and less battle-tested than on Linux.
³ Builds from source, not covered by CI.</sup>
`wipe` removes both directories' contents: the device identity, every network,
every signed record and everything a protocol kept beside them. It refuses
while an agent is running, and refuses a directory with no `state.sqlite` in
it, so a mistyped `--state-dir` cannot take somebody's documents with it.
Without `--yes` it only says what it would remove. It is not a goodbye: nobody
is told, because after it there is no key left to sign anything with. Leave the
networks first if the addresses should be freed.
The Linux CLI is a single static binary: it runs on any distribution.
## NixOS
The repository is a flake with a package and a module.
```nix
# flake.nix
{
inputs.tsunagi.url = "github:house-of-vanity/tsunagi";
outputs = { nixpkgs, tsunagi, ... }: {
nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
modules = [
tsunagi.nixosModules.default
{
services.tsunagi = {
enable = true;
tray.enable = true; # optional, for desktops
users = [ "alice" ]; # may use tsng and the tray without sudo
};
}
];
};
};
}
```
Just trying it: `nix run github:house-of-vanity/tsunagi#tsunagi-cli -- --help`.
`packages.<system>.tsunagi` (with the tray) and `tsunagi-cli` are also exported,
along with `overlays.default`.
## Build from source
Rust 1.91 or newer.
```sh
cargo build --release --bin tsng # the agent and CLI
cargo build --release --bin tsunagi-tray # the tray (Linux needs GTK 3 and libayatana-appindicator)
```
Windows needs [Wintun](https://www.wintun.net) next to `tsng.exe`.
Per-platform notes: [Linux](dist/linux/README.md), [Arch](dist/arch/README.md),
[macOS](dist/macos/README.md), [Windows](dist/windows/README.md).
## How it works
- Your name and secret derive a network identity. Members publish signed
contact records to the public Mainline DHT under it, and find each other there.
- Every connection proves knowledge of the secret before it's accepted.
Traffic between members is end-to-end encrypted, relays included.
- Addresses and names are claimed with each device's own key and replicated
between members. There is no one in charge, so nothing to be taken down.
The secret is the only key, so make it a good one (the generated ones are).
The [threat model](docs/threat-model.md) says what this does and doesn't protect against.
## Documentation
- [docs/architecture.md](docs/architecture.md) — module boundaries and runtime.
- [docs/wireguard.md](docs/wireguard.md) — the WireGuard plugin: overlay
addressing, announcements, backends, reconciliation.
- [docs/protocol.md](docs/protocol.md) — identity derivation, framing, handshake.
- [docs/sync-model.md](docs/sync-model.md) — the planned signed-state model and
what is deliberately not built yet.
- [docs/exit-node.md](docs/exit-node.md) — exit nodes: sending all traffic through a member.
- [docs/threat-model.md](docs/threat-model.md) — threat model and known limits.
- [docs/testing.md](docs/testing.md) — what the suite covers and what it does not.
- [AGENTS.md](AGENTS.md) — rules for anyone (human or agent) changing this repo.
## Security in one paragraph
Membership is proved by an HMAC over a transcript keyed by a value derived from
the shared secret, bound to the specific iroh connection through the TLS
exporter, to the network id, to both endpoint identities and to distinct role
labels. This targets high-entropy secrets: there is no PAKE here, so a short
human passphrase is guessable offline by anyone who can reach the handshake.
Anyone who knows the secret is a full participant and can create many
identities. Read [docs/threat-model.md](docs/threat-model.md) before relying on
any of this.
[Architecture](docs/architecture.md) ·
[Protocol](docs/protocol.md) ·
[WireGuard data plane](docs/wireguard.md) ·
[Routing](docs/routing.md) ·
[Exit nodes](docs/exit-node.md) ·
[Threat model](docs/threat-model.md) ·
[Contributing](AGENTS.md)
## Licence
dist/arch/README.md
Vendored
+1 -1
View File
@@ -22,7 +22,7 @@ service's socket at `/run/tsunagi/agent.sock`):
```sh
sudo tsng status
sudo tsng join <network-name> <tsn1…secret>
sudo tsng join -n <network-name> -s <tsn1…secret>
```
## How it runs
dist/linux/README.md
Vendored
+1 -1
View File
@@ -28,7 +28,7 @@ group, so after `sudo usermod -aG tsunagi "$USER"` (and logging in again) no
```sh
sudo tsng status
sudo tsng join <network-name> <tsn1…secret>
sudo tsng join -n <network-name> -s <tsn1…secret>
```
## As your own user — one capability, no service
dist/linux/install.sh
Vendored
+1 -1
View File
@@ -48,7 +48,7 @@ Done. Start the agent with:
Control it with sudo (the CLI finds the service's socket automatically):
sudo tsng status
sudo tsng join <network-name> <tsn1…secret>
sudo tsng join -n <network-name> -s <tsn1…secret>
EOF
else
cat <<EOF
dist/macos/README.md
Vendored
+1 -1
View File
@@ -15,7 +15,7 @@ socket and do **not** need `sudo`:
```sh
tsng status
tsng join <network-name> <tsn1…secret>
tsng join -n <network-name> -s <tsn1…secret>
```
The agent uses fixed system paths, so the commands find it regardless of whose
docs/wireguard.md
+2 -2
View File
@@ -26,8 +26,8 @@ system can hand us IP packets, and even that is behind a trait
| `ManagedTunFactory` | `CAP_NET_ADMIN` | traffic actually reaches the OS |
`ManagedTunFactory` creates the interface and configures it; see
*Provisioning the interface* below and *Privileges* in
[../README.md](../README.md#privileges). With no capability the agent runs
*Provisioning the interface* below and Privileges in
[../dist/linux/README.md](../dist/linux/README.md). With no capability the agent runs
with `--no-tun`: everything but the last hop into the kernel still works.
[boringtun]: https://docs.rs/boringtun
flake.lock
Generated
+27
View File
@@ -0,0 +1,27 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1790769526,
"narHash": "sha256-5AIVFLRcx8YAwbFTVPTs7YhwJH7JyHfAQsvXmpovcXI=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "c9fe7d12cd78d1adcd12dd15e24432dde5b155a0",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}
flake.nix
+39
View File
@@ -0,0 +1,39 @@
{
description = "tsunagi: serverless mesh VPN. A network name and a secret are all it takes.";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
outputs =
{ self, nixpkgs }:
let
systems = [
"x86_64-linux"
"aarch64-linux"
];
forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f nixpkgs.legacyPackages.${system});
in
{
packages = forAllSystems (pkgs: rec {
tsunagi = pkgs.callPackage ./nix/package.nix { };
tsunagi-cli = pkgs.callPackage ./nix/package.nix { withTray = false; };
default = tsunagi;
});
overlays.default = final: _prev: {
tsunagi = final.callPackage ./nix/package.nix { };
tsunagi-cli = final.callPackage ./nix/package.nix { withTray = false; };
};
nixosModules.default = import ./nix/module.nix self;
devShells = forAllSystems (pkgs: {
default = pkgs.mkShell {
inputsFrom = [ self.packages.${pkgs.stdenv.hostPlatform.system}.tsunagi ];
packages = [
pkgs.rustfmt
pkgs.clippy
];
};
});
};
}
nix/module.nix
+132
View File
@@ -0,0 +1,132 @@
flake:
{
config,
lib,
pkgs,
...
}:
let
cfg = config.services.tsunagi;
socket = "/run/tsunagi/agent.sock";
in
{
options.services.tsunagi = {
enable = lib.mkEnableOption "the tsunagi mesh agent";
package = lib.mkOption {
type = lib.types.package;
default =
if cfg.tray.enable then
flake.packages.${pkgs.stdenv.hostPlatform.system}.tsunagi
else
flake.packages.${pkgs.stdenv.hostPlatform.system}.tsunagi-cli;
defaultText = lib.literalExpression "tsunagi.packages.\${system}.tsunagi (tsunagi-cli without the tray)";
description = "The tsunagi package to run.";
};
tray.enable = lib.mkEnableOption "the tray client, started with every graphical session";
users = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [ ];
example = [ "alice" ];
description = ''
Users allowed to talk to the agent without sudo (`tsng status`, `tsng join`,
the tray). They are added to the `tsunagi` group.
'';
};
protocols = lib.mkOption {
type = lib.types.nullOr (lib.types.listOf lib.types.str);
default = null;
example = [
"wg-quic"
"wg"
];
description = "Packet protocols for `tsng up --protocol`, best first. Null keeps the agent's default.";
};
};
config = lib.mkIf cfg.enable {
users.groups.tsunagi = { };
users.users = {
tsunagi = {
isSystemUser = true;
group = "tsunagi";
home = "/var/lib/tsunagi";
description = "tsunagi agent";
};
}
// lib.genAttrs cfg.users (_: {
extraGroups = [ "tsunagi" ];
});
systemd.services.tsunagi = {
description = "tsunagi mesh agent";
documentation = [ "https://github.com/house-of-vanity/tsunagi" ];
wantedBy = [ "multi-user.target" ];
wants = [ "network-online.target" ];
after = [ "network-online.target" ];
environment = {
TSUNAGI_STATE_DIR = "/var/lib/tsunagi";
TSUNAGI_CACHE_DIR = "/var/cache/tsunagi";
TSUNAGI_CONTROL_SOCKET = socket;
TSUNAGI_CONTROL_GROUP = "tsunagi";
};
serviceConfig = {
ExecStart = lib.concatStringsSep " " (
[ "${cfg.package}/bin/tsng up" ]
++ lib.optional (cfg.protocols != null) "--protocol ${lib.concatStringsSep "," cfg.protocols}"
);
User = "tsunagi";
Group = "tsunagi";
AmbientCapabilities = [
"CAP_NET_ADMIN"
"CAP_NET_BIND_SERVICE"
];
CapabilityBoundingSet = [
"CAP_NET_ADMIN"
"CAP_NET_BIND_SERVICE"
];
StateDirectory = "tsunagi";
StateDirectoryMode = "0700";
CacheDirectory = "tsunagi";
RuntimeDirectory = "tsunagi";
RuntimeDirectoryMode = "0755";
Restart = "on-failure";
RestartSec = 5;
};
};
# Exit nodes: the agent looks for iptables only in fixed FHS directories.
systemd.tmpfiles.rules = [
"L+ /usr/bin/iptables - - - - ${pkgs.iptables}/bin/iptables"
];
# Names of mesh members are served locally and handed to the system resolver.
services.resolved.enable = lib.mkDefault true;
# polkit decides by group, not capability, who may configure systemd-resolved.
security.polkit.extraConfig = ''
polkit.addRule(function(action, subject) {
var allowed = [
"org.freedesktop.resolve1.set-dns-servers",
"org.freedesktop.resolve1.set-domains",
"org.freedesktop.resolve1.set-default-route",
"org.freedesktop.resolve1.revert"
];
if (allowed.indexOf(action.id) >= 0 && subject.isInGroup("tsunagi")) {
return polkit.Result.YES;
}
});
'';
environment.systemPackages = [ cfg.package ];
environment.sessionVariables.TSUNAGI_CONTROL_SOCKET = socket;
environment.etc."xdg/autostart/tsunagi-tray.desktop" = lib.mkIf cfg.tray.enable {
source = "${cfg.package}/share/applications/tsunagi-tray.desktop";
};
};
}
nix/package.nix
+94
View File
@@ -0,0 +1,94 @@
{
withTray ? true,
lib,
rustPlatform,
pkg-config,
copyDesktopItems,
wrapGAppsHook3,
gtk3,
libayatana-appindicator,
libglvnd,
libxkbcommon,
vulkan-loader,
wayland,
xdotool,
libx11,
libxcursor,
libxi,
libxrandr,
}:
let
# The tray is an egui app that loads graphics, windowing and appindicator
# libraries at run time, so they have to be put on its library path by hand.
trayRuntimeLibs = [
libayatana-appindicator
libglvnd
libxkbcommon
vulkan-loader
wayland
libx11
libxcursor
libxi
libxrandr
];
in
rustPlatform.buildRustPackage {
pname = if withTray then "tsunagi" else "tsunagi-cli";
version = "0.1.0";
src = lib.cleanSource ../.;
cargoLock.lockFile = ../Cargo.lock;
cargoBuildFlags = [
"--bin"
"tsng"
]
++ lib.optionals withTray [
"--bin"
"tsunagi-tray"
];
# The suite opens real sockets and interfaces; it runs in CI instead.
doCheck = false;
nativeBuildInputs = [ pkg-config ] ++ lib.optionals withTray [
copyDesktopItems
wrapGAppsHook3
];
buildInputs = lib.optionals withTray [
gtk3
libayatana-appindicator
xdotool
];
dontWrapGApps = true;
postInstall = ''
install -Dm644 dist/linux/50-tsunagi-resolved.rules \
"$out/share/doc/tsunagi/50-tsunagi-resolved.rules"
''
+ lib.optionalString withTray ''
install -Dm644 dist/linux/tsunagi.svg \
"$out/share/icons/hicolor/scalable/apps/tsunagi.svg"
install -Dm644 dist/linux/tsunagi-tray.desktop \
"$out/share/applications/tsunagi-tray.desktop"
'';
postFixup = lib.optionalString withTray ''
wrapProgram "$out/bin/tsunagi-tray" \
"''${gappsWrapperArgs[@]}" \
--prefix LD_LIBRARY_PATH : ${lib.makeLibraryPath trayRuntimeLibs}:/run/opengl-driver/lib
'';
meta = {
description = "Serverless mesh VPN: join a network with a name and a secret";
homepage = "https://github.com/house-of-vanity/tsunagi";
license = lib.licenses.wtfpl;
mainProgram = "tsng";
platforms = [
"x86_64-linux"
"aarch64-linux"
];
};
}