Implement the WireGuard data plane plugin
The first IP plugin, built on the data plane boundary the core already had. Plugin: - one X25519 key per network in the plugin's own wireguard.sqlite, separate from the iroh identity and from the network secret; a damaged store is an error, never a silently regenerated identity - deterministic IPv6 ULA overlay: every member derives the same /64 from the network id and its own /128 from its WireGuard public key, so no coordinator allocates addresses - AllowedIPs are derived locally, never taken from a peer's announcement, so a member cannot claim another member's overlay address; a mismatched claim is rejected - bounded, versioned, validated announcement carried as the existing opaque capability payload, which the core still never parses - each agent builds its own full-mesh configuration (N-1 peers) and reconciles on every change and on a timer, repairing drift - WireguardBackend abstraction: RecordingBackend in memory, and WgToolBackend driving real wg/ip on Linux, split into a pure planner plus parsers and a thin executor so everything interesting is testable without root Core, three generic additions the plugin needed: - IpPlugin::on_network_activated, so per-network state is ready before peers - PluginContext for re-announcements and error reports from plugin tasks, with errors counted by the owning network runtime - IpPlugin::shutdown, awaited with a grace period, so system objects go away 94 tests pass offline with no privileges: 35 new WireGuard unit tests and 12 integration tests over real iroh connections. The real wg/ip backend needs root and is behind --ignored in tests/wireguard_system.rs; it was not run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+12
-7
@@ -12,12 +12,16 @@ authentication, participant announcements, capability exchange and — later —
|
||||
state synchronisation and delivery of IP-plugin data.
|
||||
|
||||
**Data plane.** Separate plugins create IP connectivity. WireGuard is the first
|
||||
planned one; none exists yet. Plugin keys, configuration and lifecycle are
|
||||
separate from iroh identity and from the network secret. The core moves an
|
||||
opaque, bounded payload and never parses it.
|
||||
one and is implemented — see [wireguard.md](wireguard.md). Plugin keys,
|
||||
configuration and lifecycle are separate from iroh identity and from the
|
||||
network secret. The core moves an opaque, bounded payload and never parses it.
|
||||
|
||||
Only control messages travel over iroh. User IP traffic is not tunnelled
|
||||
through it.
|
||||
through it; WireGuard packets travel over WireGuard's own UDP sockets.
|
||||
|
||||
A plugin talks to the core through three narrow hooks — `on_network_activated`,
|
||||
a `PluginContext` for re-announcements and error reports, and a bounded
|
||||
`shutdown` — so the core never learns anything protocol-specific.
|
||||
|
||||
A data plane failure never stops the daemon: the control plane keeps running
|
||||
and the agent stays manageable.
|
||||
@@ -32,11 +36,12 @@ and the agent stays manageable.
|
||||
| `proto` | message format, handshake, membership proof, protocol limits |
|
||||
| `agent` | agent and per-network lifecycle, reconnect, in-process message routing |
|
||||
| `storage` | mandatory state and the separately recoverable cache |
|
||||
| `dataplane` | the minimal contract future IP plugins implement |
|
||||
| `dataplane` | the contract IP plugins implement, plus the WireGuard plugin |
|
||||
|
||||
Abstractions exist only where something is really substituted or really needs
|
||||
isolating for tests: `NetworkDiscovery` and `IpPlugin`. Everything else is a
|
||||
concrete type.
|
||||
isolating for tests: `NetworkDiscovery`, `IpPlugin`, and `WireguardBackend`
|
||||
(which is what lets the plugin be tested in full without root). Everything else
|
||||
is a concrete type.
|
||||
|
||||
## Runtime shape
|
||||
|
||||
|
||||
+3
-1
@@ -2,7 +2,9 @@
|
||||
|
||||
**Nothing in this document is implemented.** The proof of concept exchanges
|
||||
hostname and capability announcements over live sessions and keeps no
|
||||
replicated history. This file records the intended direction so the module
|
||||
replicated history. That is also why WireGuard peer membership is
|
||||
session-scoped today: a peer disappears from the overlay configuration when its
|
||||
control session ends, because there is no agreed durable state to keep it. This file records the intended direction so the module
|
||||
boundaries in [architecture.md](architecture.md) stay compatible with it, and so
|
||||
nobody mistakes the current announcements for synchronisation.
|
||||
|
||||
|
||||
@@ -38,6 +38,19 @@ of several system processes, and is not presented as one.
|
||||
| 9 | wrong version, a message before authentication, a proof replayed on another connection, an oversized frame and a `Hello` for an inactive network are all rejected without taking the agent down | `tests/authentication.rs` |
|
||||
| 10 | a second agent on the same state directory gets a clear error; after a clean stop the directory reopens; shutdown ends background tasks and refuses further work; independent agents coexist in one process | `tests/resilience.rs` |
|
||||
|
||||
`tests/wireguard.rs` drives the WireGuard plugin over real iroh connections
|
||||
with the in-memory backend: a pair and a three-agent mesh converge to `N - 1`
|
||||
peers with locally derived `AllowedIPs`, a departing peer is removed, two
|
||||
networks get separate interfaces, keys and overlays, reconciliation repairs a
|
||||
configuration edited by hand, a backend failure leaves the control plane
|
||||
untouched, a restart keeps the WireGuard identity, shutdown removes every
|
||||
interface, a member claiming another member's overlay address is rejected, and
|
||||
the core carries the payload without interpreting it.
|
||||
|
||||
`tests/wireguard_system.rs` exercises the real `wg`/`ip` backend. It is
|
||||
**ignored by default** because it changes the host's network and needs Linux,
|
||||
wireguard-tools and `CAP_NET_ADMIN`.
|
||||
|
||||
`tests/discovery.rs` covers the discovery contract itself: a static bootstrap
|
||||
candidate is enough to join, several backends compose, entries are withdrawn
|
||||
when a network stops, and a forgotten network stays forgotten across a restart.
|
||||
@@ -46,6 +59,12 @@ Unit tests in `src/proto/handshake.rs` cover the transcript construction
|
||||
itself: role separation, channel binding, identity and network binding,
|
||||
unambiguous encoding, and rejection under the wrong key.
|
||||
|
||||
Unit tests in `src/dataplane/wireguard/` cover the parts that would otherwise
|
||||
need root: key clamping against the RFC 7748 vector, overlay derivation,
|
||||
announcement validation including the address-hijack attempt, configuration
|
||||
building and rendering, the exact command plan the real backend would run, and
|
||||
parsing `wg showconf` and `ip address show` output.
|
||||
|
||||
`tests/end_to_end.rs` is the vertical slice: persistent identity → network
|
||||
space → discovery → iroh → authentication → message exchange.
|
||||
|
||||
|
||||
+18
-2
@@ -19,6 +19,10 @@ Read this before relying on anything here. The protocol is in
|
||||
messages for network B, even over a shared physical connection.
|
||||
- **Confidentiality and integrity in transit.** Provided by QUIC/TLS. This
|
||||
crate adds no encryption of its own.
|
||||
- **Overlay address ownership.** A WireGuard peer's `AllowedIPs` are derived
|
||||
from its public key, not taken from its announcement, so a member cannot
|
||||
claim another member's overlay address and receive its traffic. See
|
||||
[wireguard.md](wireguard.md#deterministic-overlay-addressing).
|
||||
- **Resource bounds.** Frame lengths are validated before allocation; strings,
|
||||
lists, queues, concurrent dials and in-flight handshakes are all bounded;
|
||||
handshakes, dials and writes have timeouts.
|
||||
@@ -45,8 +49,20 @@ Read this before relying on anything here. The protocol is in
|
||||
permissions are owner-only where the platform supports it, and the state
|
||||
directory takes an ownership lock, but neither defends against a user who can
|
||||
read the file or against malware running as that user.
|
||||
- **User IP traffic.** Not carried here at all. Filtering it is the operating
|
||||
system's and the user's job.
|
||||
- **User IP traffic.** Carried by the WireGuard plugin, not by iroh, and
|
||||
encrypted by WireGuard itself. *Filtering* it is still the operating system's
|
||||
and the user's job: the plugin creates connectivity between members and does
|
||||
not police what flows over it.
|
||||
- **Overlay address squatting.** A member can mint many WireGuard keys and
|
||||
therefore occupy many overlay addresses. It cannot pick which ones, but it
|
||||
can consume them and appear as many participants.
|
||||
- **Plugin keys on disk.** The WireGuard private keys live in the plugin's own
|
||||
`wireguard.sqlite`, owner-only where the platform supports it. Copying that
|
||||
file copies this agent's overlay identity, exactly as copying `state.sqlite`
|
||||
copies its control plane identity.
|
||||
- **What the data plane does not police.** The plugin sets `AllowedIPs` per
|
||||
peer, which stops a member impersonating another member's overlay address.
|
||||
It does not stop a member sending whatever it likes *from its own* address.
|
||||
- **Denial of service.** Bounds and timeouts stop trivial resource exhaustion
|
||||
from a single peer. They do not make the agent resistant to a determined
|
||||
attacker who knows the secret, and no rate limiting per identity exists yet.
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
# The WireGuard plugin
|
||||
|
||||
WireGuard is the first IP plugin. It creates real IP connectivity between
|
||||
participants while the control plane keeps doing its own job: deciding who is
|
||||
in the network and carrying each participant's opaque announcement.
|
||||
|
||||
Module boundaries are in [architecture.md](architecture.md), the control
|
||||
protocol in [protocol.md](protocol.md), and the security consequences in
|
||||
[threat-model.md](threat-model.md).
|
||||
|
||||
## What stays separate
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **User IP traffic** | Never goes through iroh. iroh carries announcements; packets travel over WireGuard's own UDP sockets. |
|
||||
| **Addresses** | An iroh address is an address for iroh. The plugin gathers and advertises its own reachability. |
|
||||
| **Payloads** | The core moves a bounded opaque blob. Only `dataplane::wireguard::announcement` interprets it. |
|
||||
| **Keys** | One WireGuard key per network, in the plugin's own store. Unrelated to the iroh device key and to the network secret. |
|
||||
| **Failures** | A data plane error is reported and retried. The control plane keeps running and the agent stays manageable. |
|
||||
|
||||
## Deterministic overlay addressing
|
||||
|
||||
A mesh with no coordinator cannot hand out addresses, so everyone derives their
|
||||
own. The result is an IPv6 unique local address (RFC 4193):
|
||||
|
||||
```text
|
||||
prefix (/64) = 0xfd || SHA-256( LP(domain) || LP("prefix") || LP(network_id) )[0..7]
|
||||
iid (64b) = SHA-256( LP(domain) || LP("interface") || LP(network_id) || LP(wg_public_key) )[0..8]
|
||||
address = prefix || iid
|
||||
```
|
||||
|
||||
with `domain = "tsunagi-wireguard-overlay-v1"` and `LP(x) = u32_be(len(x)) || x`,
|
||||
the same unambiguous encoding the rest of the project uses.
|
||||
|
||||
Two consequences matter:
|
||||
|
||||
* every member of a network derives the **same `/64`**, so the overlay is one
|
||||
subnet that nobody had to allocate;
|
||||
* a member's address is bound to its WireGuard public key, so
|
||||
**`AllowedIPs` are derived locally and never taken from what a peer claims**.
|
||||
|
||||
That second point is the plugin's central security property. A participant who
|
||||
knows the network secret can mint as many WireGuard keys — and therefore as
|
||||
many overlay addresses — as it likes, but it cannot choose to collide with an
|
||||
existing member's address without finding a hash preimage. An announcement
|
||||
whose claimed address does not match the derivation is rejected outright.
|
||||
|
||||
## The announcement
|
||||
|
||||
Carried as the opaque `PluginCapability { protocol: "wireguard", .. }` payload,
|
||||
encoded with postcard:
|
||||
|
||||
| field | meaning |
|
||||
|---|---|
|
||||
| `version` | announcement format version, currently 1 |
|
||||
| `public_key` | the peer's X25519 WireGuard key |
|
||||
| `listen_port` | the UDP port its interface listens on |
|
||||
| `endpoints` | reachability the plugin gathered for itself, at most 8 |
|
||||
| `overlay_address` | what the peer believes its address is — cross-checked, never used |
|
||||
|
||||
Validation, all before anything reaches a configuration: the version must
|
||||
match, the key must not be zero and must not be our own, the port must not be
|
||||
zero, the endpoint list must be within bounds, unusable endpoints
|
||||
(unspecified, multicast, broadcast, documentation, port zero) are dropped, and
|
||||
the claimed overlay address must equal the derived one.
|
||||
|
||||
## Building the configuration
|
||||
|
||||
Each agent builds its **own** configuration from the agreed set of
|
||||
participants: for a full mesh of `N` members that is `N - 1` peers locally.
|
||||
Nobody hands a configuration to anybody else and no participant is
|
||||
authoritative.
|
||||
|
||||
* **Interface name** — `prefix + base32(network_id)`, truncated to the
|
||||
platform's 15 characters. Stable across restarts. Two agents on one host in
|
||||
the same network need different prefixes.
|
||||
* **Port** — `PortPolicy::Derived` picks a stable port from the network id
|
||||
inside a range, so a peer's cached endpoint keeps working across restarts and
|
||||
two networks on one host do not collide. `PortPolicy::Fixed` pins it.
|
||||
* **Addresses** — the agent's own `/128` plus the shared `/64`.
|
||||
* **Peer entries** — public key, derived `AllowedIPs`, the peer's first usable
|
||||
advertised endpoint, and a keepalive.
|
||||
|
||||
Nothing free-form from the network reaches a command argument or a
|
||||
configuration directive: peer keys, endpoints, prefixes and keepalives are
|
||||
typed values that the plugin re-serialises itself.
|
||||
|
||||
## Backends
|
||||
|
||||
The plugin computes *what* the interface should look like; a backend makes it
|
||||
so. Splitting them is what keeps every interesting decision testable without
|
||||
root.
|
||||
|
||||
* **`RecordingBackend`** — applies configurations in memory. The default test
|
||||
suite and the example use it, so neither needs privileges nor touches the
|
||||
host. It can also be told to fail, or to report drift.
|
||||
* **`WgToolBackend`** — drives the real `wg` and `ip` tools. Linux only,
|
||||
requires `CAP_NET_ADMIN`. It is split into a **pure planner** and pure
|
||||
parsers, which are unit tested on every platform, plus a thin executor. The
|
||||
WireGuard configuration is piped to `wg setconf` / `wg syncconf` on standard
|
||||
input, so the private key never reaches the filesystem.
|
||||
|
||||
Ownership is explicit: the plugin creates the interface and the plugin removes
|
||||
it. An interface that already exists and is not a WireGuard device is
|
||||
**refused, not adopted**, so the agent never takes over something it did not
|
||||
create. It changes no routing, DNS or firewall settings.
|
||||
|
||||
## Reconciliation
|
||||
|
||||
The plugin reconciles on every change — a peer announcement, a peer leaving —
|
||||
coalesced over a short debounce, and again on a timer. Each pass reads the
|
||||
interface back, compares it with the desired state, and applies only if they
|
||||
differ. A configuration edited by hand is therefore put back the way it should
|
||||
be, which is exactly what `reconciliation_repairs_a_configuration_edited_by_hand`
|
||||
in `tests/wireguard.rs` checks.
|
||||
|
||||
Deactivating a network removes its interface but **keeps its key**, so coming
|
||||
back later keeps the same overlay address. Agent shutdown removes every
|
||||
interface the plugin created.
|
||||
|
||||
## What the plugin needs from the core
|
||||
|
||||
Three small additions to the `IpPlugin` contract, all generic rather than
|
||||
WireGuard-specific:
|
||||
|
||||
* `on_network_activated` — prepare per-network state before any peer appears;
|
||||
* `attach(PluginContext)` — a handle to ask for a re-announcement when the
|
||||
plugin's own capability changes, and to report an error from its own tasks;
|
||||
* `shutdown` — remove system objects during a bounded agent shutdown.
|
||||
|
||||
Errors reported through the context are counted by the owning network's
|
||||
runtime, so `NetworkMetrics::plugin_errors` and `Event::PluginError` always
|
||||
agree.
|
||||
|
||||
## Using it
|
||||
|
||||
```rust,no_run
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
|
||||
use tsunagi::dataplane::IpPlugin;
|
||||
use tsunagi::dataplane::wireguard::{WgToolBackend, WireguardConfig, WireguardPlugin};
|
||||
use tsunagi::identity::{NetworkName, NetworkSecret};
|
||||
use tsunagi::{Agent, Result};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<()> {
|
||||
let paths = StoragePaths::user_default()?;
|
||||
|
||||
// The plugin's own state, separate from the agent's.
|
||||
let wireguard = WireguardConfig::new(paths.state_dir.join("wireguard"));
|
||||
let backend = WgToolBackend::new().expect("Linux with wg and CAP_NET_ADMIN");
|
||||
let plugin = WireguardPlugin::open(wireguard, Arc::new(backend))
|
||||
.await
|
||||
.expect("wireguard plugin");
|
||||
|
||||
let agent = Agent::spawn(
|
||||
AgentConfig::new(paths)
|
||||
.with_transport(TransportPolicy::N0Defaults)
|
||||
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
|
||||
)
|
||||
.await?;
|
||||
|
||||
let network = agent
|
||||
.join_network(&NetworkName::new("kitchen-table")?, &NetworkSecret::generate())
|
||||
.await?;
|
||||
|
||||
if let Some(view) = plugin.overview(network) {
|
||||
println!("{} on {} at {}", view.interface, view.overlay_prefix, view.overlay_address);
|
||||
}
|
||||
|
||||
tokio::time::sleep(Duration::from_secs(60)).await;
|
||||
agent.shutdown().await; // removes the interface
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
There is a runnable version in `examples/wireguard_mesh.rs`, which uses the
|
||||
in-memory backend by default and the real one with `--real`.
|
||||
|
||||
## Limits and future work
|
||||
|
||||
* **Full mesh only.** Every member configures every other member. Routing
|
||||
through an intermediate participant is not implemented.
|
||||
* **No IPv4 overlay.** Addressing is IPv6 ULA, because it can be derived
|
||||
collision-free. An IPv4 overlay would need an allocator, which needs the
|
||||
agreed state described in [sync-model.md](sync-model.md).
|
||||
* **Peer membership is session-scoped.** A peer disappears from the
|
||||
configuration when its control session ends. Persisting membership across a
|
||||
long absence is part of the same future work.
|
||||
* **`WgToolBackend` is Linux only.** A netlink backend, and backends for macOS
|
||||
and Windows, are not implemented. `WgToolBackend::new()` fails with a clear
|
||||
message elsewhere.
|
||||
* **No MTU or path discovery.** The MTU is a configured constant.
|
||||
* **The real backend is not exercised by the default suite.** It needs root,
|
||||
so its tests live in `tests/wireguard_system.rs` behind `--ignored`.
|
||||
Reference in New Issue
Block a user