Separate internal WireGuard encryption from data carriers

AWG-only policy also selected a competing end-to-end plugin, so compatible
external links could still have no compatible encrypted data channel. Carrier
changes replaced tunnel state and status selected counters from different
engines. Use one internal WireGuard plugin and a separate DataCarrier contract
for external capabilities/lifecycle. Keep the logical link, keys, handshake,
interface and counter epoch across policy, fallback and direct/relay changes.
Control/discovery transport selection and membership remain independent.

Share the bounded per-neighbour UDP WireGuard engine between ordinary WG and
Amnezia masking. Encrypt the relay envelope inside the outer hop channel with
fresh pair keys/PSK. Remove duplicate WG key/device code and the TCP/TLS inner
cipher. TLS carries opaque internal WireGuard frames. Refuse missing native
transports instead of letting default iroh impersonate their protocol names.
Probe the actual authenticated carrier, independently of the shared IP tunnel;
failed local probe submissions count as loss too.

Apply the vendored iroh interface exclusion to actual direct candidates, not
just public discovery records. Exclude overlay subnets from introductions and
propagate the actual owned interface name, including kernel-assigned utun names,
to every carrier. Production logs showed paths through the overlay itself.
This removes a recursive transport path; it does not establish the historical
cause of the reported 100 GB absolute counter. Exit-node traffic is also real
peer IP traffic even when local applications are idle.

Hide inner cipher choices/handshake clutter in CLI and tray. Report external
routes, readiness and actionable policy/version failures. Count original IP
packets only when their ciphertext is accepted by the logical transport,
including queued packets' exact unpadded lengths. Exclude physical probes and
keepalives. Export counter epochs; sum per-peer rate deltas and discard stale
histories so joins/restarts cannot turn old totals into fresh traffic. Status
also prints exact IP byte totals for comparison with the tray.

Coordinated upgrade: data ALPN 6, internal wireguard-e2e capability v4,
wg/awg v2, QUIC carrier v1, TLS v2, local IPC v23. Control ALPN 8, device/network
identity derivation, signed records and saved network choices stay unchanged.
Keep the historical wg-quic key store; transactionally import missing keys from
wg then awg without replacing canonical keys. Forget removes legacy copies too,
on the blocking pool, to prevent resurrection after restart. Preserve historic
cipher options as shared settings, rejecting contradictory values.

Validation on Linux in the Nix development shell:
- cargo fmt --all -- --check
- cargo clippy --locked --workspace --all-targets -- -D warnings
- cargo test --locked --workspace --all-targets (562 passed, 3 ignored)
The inherited service socket/secret/group environment overrides were removed
for isolated CLI process tests. A new carrier test initially raced the first
peer announcement; its deadline polling now waits for a peer row explicitly.
Real loopback endpoints, membership, SQLite, WG/AWG/TLS encryption and memory
TUNs cover mixed-carrier relaying and one smoltcp TCP connection surviving
relay/direct/policy changes and a dropped transmission. Counter tests check
exact plaintext lengths, refused sends, epochs, joins and stale histories.
Public DHT, privileged kernel and manual benchmark tests remain ignored;
macOS/Windows native TUNs, desktop GUI and WAN/DPI behavior were not run.
This commit is contained in:
Aleksandr Bogomiakov committed 2026-10-22 01:07:52 +01:00
1 parent 8a6216bccb
commit 2166ba4800
76 files changed
+3903 -6811

No files matched your search

+29 -14
View File
@@ -17,13 +17,16 @@ tsng protocols home awg
```
Changes are local to this device, persist across restarts, and retire existing
data links and end-to-end channels for that network before renegotiation.
physical data links for that network before renegotiation. The single internal
WireGuard channel, its keys, session and counters survive.
Authenticated control sessions, membership, addresses and other networks stay
live. Removing every protocol (`none` in the CLI, all checkboxes off in the tray)
leaves the network control-only. This is different from automatic carrier
fallback, which preserves the selected end-to-end channel.
leaves the network control-only. Re-enabling a carrier resumes the existing
internal channel, subject to ordinary WireGuard and application timeouts. The
choice applies to both incoming and outgoing local data links; a relay may use
a different transport on its next hop. Control and discovery are independent.
`awg` is Tsunagi AWG wire version 1. Its packet masking engine is adapted from
`awg` is Tsunagi AWG wire version 2. Its packet masking engine is adapted from
[RunaRoo/AmneziaWG-RS](https://github.com/RunaRoo/AmneziaWG-RS), with the pinned
source and license recorded in
[UPSTREAM.md](../crates/tsunagi-awg/UPSTREAM.md). It uses the existing boringtun
@@ -36,7 +39,7 @@ not implemented: this carrier connects authenticated Tsunagi members.
There is one independent UDP socket and cancellable packet-engine task per
`(network, peer)` pair. The smaller endpoint id initiates, following the same
rule as other data carriers. Both sides must announce enabled AWG wire version
1. An authenticated iroh control session is required throughout; AWG supplies
2. An authenticated iroh control session is required throughout; AWG supplies
no alternate discovery or membership mechanism.
Every `open` generates new masking parameters, a header-protection key, an
@@ -100,25 +103,37 @@ candidates. IPv6-only paths, symmetric NAT and networks blocking arbitrary UDP
may fail AWG and fall back; there is no AWG-over-QUIC substitution or AWG TURN
relay hidden behind its name.
Data delivery checks go through the actual end-to-end encrypted tunnel, once
per second, independently of control-plane RTT. Requests and replies bind a
Data delivery checks go through the specific authenticated physical carrier,
once per second, independently of control-plane RTT or the shared internal
tunnel. A successful relay path cannot mask a broken direct carrier. Requests and replies bind a
fresh private marker/key to the sender's endpoint id. Unanswered requests expire
after three seconds. After a 20-second warmup, at least 15 completed observations
and at least 60% loss in the last 20 observations retire a carrier. Probes are
consumed locally after source/destination admission and never enter the TUN.
consumed at the carrier boundary and never enter the TUN or user IP counters.
AWG additionally checks encrypted outer delivery throughout the pair lifetime.
Setup failure or persistent runtime loss advances to the next mutually supported
carrier in the initiator's order. Earlier expired cooldowns do not jump ahead of
untried carriers. Retry cooldowns remain bounded; reaching AWG again negotiates
new parameters. Control sessions and the logical end-to-end tunnel survive a
carrier change. Status names the end-to-end protocol and the actual carrier
separately, so `wg` over `awg` is a valid result.
carrier change. Status and the tray show the actual external transport or relay
route and data readiness. There is no user-selectable internal cipher.
Control wire version 6 introduces the private transport message. All members
must upgrade together; device identities, network ids, handshake transcript
encoding, signed state and stored network configuration are unchanged. Data
ALPN remains `tsunagi/data/5`; the relay envelope is unchanged.
The internal capability is `wireguard-e2e` v4; external `wg` and `awg` use v2,
`wg-quic` v1 and `tcp-tls` v2. All members must upgrade together. Control ALPN
`tsunagi/ctrl/8`, discovery, membership and signed state are unchanged. The data
ALPN is `tsunagi/data/6`; the relay envelope format is unchanged. Local IPC is v23,
so agent, CLI and tray must match. Historical `wg-quic/wireguard.sqlite` is the
canonical internal key store. Missing keys are imported transactionally from
`wg` then `awg`, without replacing an existing canonical key or modifying a
source database. Corrupt key stores fail rather than silently regenerating keys.
Device identities, network ids, addresses and saved transport choices survive.
Ordinary `wg` uses the same independent pair engine with standard WireGuard
headers, no masking/junk, fresh hop keys and PSK, and a preferred local UDP port
(default 51820; additional pairs use dynamic ports). The whole relay envelope is
inside the hop encryption. It is separate from the internal WireGuard channel.
Neither UDP mode is a client for an externally configured WireGuard server.
## Verification
+13 -8
View File
@@ -11,14 +11,15 @@ messages. On top of it, this crate's protocol handles membership
authentication, participant announcements, capability exchange and — later —
state synchronisation and delivery of IP-plugin data.
**Data plane.** Separate plugins create IP connectivity. WireGuard is the first
one and is implemented in userspace — see [wireguard.md](wireguard.md). Plugin
**Data plane.** One internal WireGuard plugin creates IP connectivity in userspace — see [wireguard.md](wireguard.md). Plugin
keys, configuration and lifecycle are separate from iroh identity and from the
network secret. The core moves an opaque, bounded payload and never parses it.
**The transport in between.** Plugins do not open connections. They are handed
a `PacketLink` — an authenticated, unreliable datagram channel to one peer for
one protocol — and never learn how it is carried.
the internal cipher — and never learns how it is carried. External `DataCarrier`
adapters own their reachability capabilities and physical link lifecycle; they
never receive plaintext overlay IP packets.
The separation between the two planes is **logical, not physical**. The control plane uses
iroh; the data plane can use iroh on its own ALPN and connection, or independent
@@ -73,8 +74,10 @@ And in crates of their own:
| crate | responsibility |
|---|---|
| `tsunagi-awg` | ephemeral pairwise Amnezia masking, private negotiation and independently encrypted UDP carriers; reuses the end-to-end WireGuard engine |
| `tsunagi-wg-quic` | the `wg-quic` protocol: its keys, its announcement, its tunnels |
| `tsunagi-wireguard` | independent ordinary WG and masked UDP hop engines, private pair negotiation and local candidates |
| `tsunagi-tcp-tls` | TLS hop encryption and bounded reachability announcements; no inner IP cipher |
| `tsunagi-awg` | ephemeral pairwise Amnezia masking, private negotiation and independently encrypted UDP carriers; shares the independent UDP hop engine with `wg` |
| `tsunagi-wg-quic` | the single internal WireGuard cipher, keys and opaque announcement (historical crate name) |
| `tsunagi` | the command line agent, built as the `tsng` executable: the only place that owns a runtime, a logger and signals |
The core library is the package `tsunagi-core`; its library target is still called `tsunagi`, so code says `use tsunagi::…`.
@@ -85,7 +88,7 @@ they compare the *wire* version, which moves only when the bytes do, so two
peers on different releases keep working.
Abstractions exist only where something is really substituted or really needs
isolating for tests: `NetworkDiscovery`, `IpPlugin`, `PacketTransport` /
isolating for tests: `NetworkDiscovery`, one `IpPlugin`, external `DataCarrier`, `PacketTransport` /
`PacketLink` (so a protocol's carrier can change), `TunFactory` (which is what
lets the whole data plane be tested without privileges), `InterfaceProvisioner`
and `DnsPublisher` (which are where the platforms differ). Everything else is a
@@ -161,8 +164,10 @@ Values that cannot honestly be attributed to one network stay at the endpoint
level. A value iroh does not report is `None`.
An iroh address is an address for iroh. It is not assumed usable by any other
protocol; a future WireGuard plugin gathers its own reachability data and ships
it through the control plane as an opaque announcement.
protocol; each external carrier gathers its own reachability data and ships it through
the control plane as an opaque announcement or private pair negotiation.
Local iroh candidates and introductions exclude the agent's own overlay, so
transport traffic cannot recursively tunnel through itself.
### Relays
+4 -4
View File
@@ -15,9 +15,9 @@ Upgrading the crate or bumping the control protocol must never change an
existing `NetworkId`.
Control version 8 adds signed, multi-writer VIP field snapshots to version 7's
node retirements. Data ALPN remains `tsunagi/data/5`. All members must upgrade
node retirements. Data ALPN is `tsunagi/data/6` for the single internal WireGuard channel. All members must upgrade
together. Network ids, derivation labels, handshake encoding and existing record
signatures are unchanged. Local CLI/tray IPC is version 22; restart the agent
signatures are unchanged. Local CLI/tray IPC is version 23; restart the agent
with matching clients. SQLite schema 10 adds `vip_fields`, preserving identity,
network configuration, signed node records and retirement proofs.
See [virtual services](vip.md), [state sync](sync-model.md) and [AWG](amnezia.md).
@@ -145,7 +145,7 @@ event does not report a network id.
## The data plane protocol
IP plugin packets never travel on a control connection. They use their own
ALPN, `tsunagi/data/5`, on their own iroh connection:
ALPN, `tsunagi/data/6`, on their own iroh connection:
```text
initiator -> responder : (the same membership handshake as above)
@@ -192,7 +192,7 @@ and version 5 added the end-to-end protocol to it:
| source peer | 32 | original endpoint public key |
| destination peer | 32 | final endpoint public key |
| flow id | 8 | big endian opaque stable hash supplied by the plugin |
| protocol | 8 | big endian FNV-1a of the end-to-end protocol id (`wg`, `tcp-tls`, ...) |
| protocol | 8 | big endian FNV-1a of the internal protocol id (`wireguard-e2e`) |
| payload | remaining | end-to-end encrypted plugin bytes |
The fixed header is 82 bytes. Zero/oversized hop limits, unknown versions,
+13 -11
View File
@@ -16,8 +16,8 @@ the default 1280-byte interface MTU also works on small QUIC paths.
Each authenticated control session advertises that peer's own direct data
links. There is one table for each network, whatever protocols run over it: a
link is a way to reach a neighbour, and a protocol is an end-to-end tunnel to a
peer. The envelope names the protocol, so the two are independent. A neighbour
link is a way to reach a neighbour, while the single internal protocol is an
end-to-end tunnel to a peer. The envelope names the protocol, so the two are independent. A neighbour
reached over several kinds of transport (UDP, TCP, QUIC datagrams) is sent to
over the best open one, in the order the protocols were configured, and over
the next if that one fails or cannot carry the frame. Every authenticated
@@ -28,14 +28,16 @@ Link arrival, closure and changed announcements rebuild routes immediately.
Unchanged announcements refresh their age without republishing the forwarding
snapshot.
Which protocol a pair of members runs their tunnel in is settled without
asking: the member with the lower endpoint id decides, by the order it
announces its protocols in, among those both have enabled at the same wire
version. Both ends read the same answer from what they already hold. The
tunnel is created as soon as the peer is reachable by any path, and it does not
change when a direct link comes or goes underneath it: as soon as a direct
link comes up it is shorter than the path through others, and traffic moves to
it with nothing to redo. The user sees only a lower round-trip time.
Every pair uses one internal WireGuard channel, matched by its fixed opaque
capability and independent of the selected external protocols. The tunnel is
created when a member and its key are known; its logical link survives temporary
loss of all physical paths. Carrier selection changes, automatic fallback and
shorter routes keep the same keys, interface and counters. The lower endpoint
id still chooses the mutually enabled external carrier in its preference order.
A relayed AWG first hop can lead to a QUIC next hop: the user's choice governs
this device's adjacent links, while the inner ciphertext stays unchanged.
An existing TCP connection continues after transient loss if its own timeout
has not expired; a sufficiently long outage can still terminate it.
Breadth-first search computes the shortest directed paths and every equal-cost
first hop in sorted order. An actual direct link always wins. The first row is
@@ -124,5 +126,5 @@ counts sends without storing frames. Results are CPU forwarding cost, **not**
end-to-end network latency or VPN throughput; encryption, fragmentation,
sockets, congestion and scheduling contribute separately.
Wire compatibility: control ALPN `tsunagi/ctrl/8`, data ALPN `tsunagi/data/5`.
Wire compatibility: control ALPN `tsunagi/ctrl/8`, data ALPN `tsunagi/data/6`.
Upgrade every participant together; saved identities and network state persist.
+23 -3
View File
@@ -185,7 +185,8 @@ validation. Tests are offline; public STUN/DPI/NAT effectiveness needs separate
real-network validation.
Runtime protocol selection tests switch real AWG and iroh/QUIC carriers and
end-to-end WireGuard tunnels from either endpoint-id role, carry full-size
external policies from either endpoint-id role while preserving the one internal
WireGuard tunnel, key, handshake and counter epoch, carry full-size
packets, keep another active network's tunnels unchanged, disable and re-enable
the data plane without losing control sessions, and preserve selection after
restart. CLI tests use a real agent process and control socket to exercise
@@ -225,8 +226,8 @@ to test the entire mesh inside these namespaces.
Devices-window tests render the egui grid at multiple widths with its real
scroll area. They check stretching and shrinking without an expanding first
column, separate handshake/control RTT values, unknown measurements and short
end-to-end hints that do not substitute the carrier protocol. This does not
column, control RTT without internal handshake/protocol clutter, unknown measurements
and data readiness/error hints. This does not
establish native desktop rendering on every platform.
@@ -252,3 +253,22 @@ widths, click a diagram backend, and distinguish the new-flow selection from
traffic on an existing binding. Sampling checks cover idle open flows, stale
observations, counter resets, recreated listeners and removal of cached
histories. These are headless egui checks, not native desktop screenshots.
`tsunagi-wg-quic/tests/carriers.rs` uses the same internal WireGuard plugin over
real ordinary WG UDP and TCP/TLS, with loopback address selection and memory
TUNs. Exact original IP byte/packet counts exclude health probes and encryption
overhead. The AWG suite connects AWG-only and QUIC-only endpoints through a
mixed relay. One real smoltcp TCP connection stays established through relay →
direct QUIC → a dropped packet with all local carriers off → AWG relay; both
middle TUNs and endpoint keys are checked. This exercises a real userspace TCP
stack, not a host SSH server. Legacy key migration tests cover missing keys,
canonical precedence, idempotence and damaged source files.
Candidate tests exercise the vendored iroh local address filter on real bound
endpoints and exclude overlay subnets from introductions. A renaming memory TUN checks
that carriers receive the actual interface name, as required for macOS utun.
A missing native WG transport is refused rather than silently carried by QUIC. Tray traffic tests
cover a joining peer with 100 GB of prior counters, departed peers, removed
networks and new counter epochs, including resets hidden by growth in the other
direction. These distinguish rates from absolute volumes; they do not establish
the cause of an earlier unrecorded high absolute counter on a real host.
+3 -3
View File
@@ -130,7 +130,7 @@ AWG profiles, PSKs and outer keys are private to one authenticated pair attempt,
not shared network state. Address candidates are unverified until authenticated
outer WireGuard data arrives. Header protection and structured signature packets
reduce fixed wire markers; they do not guarantee admission by DPI or hide traffic
analysis. iroh control availability remains a prerequisite. Encrypted end-to-end
delivery probes detect sustained carrier loss, never confer membership and pass
the same signed source/destination admission as ordinary IP traffic. See
analysis. iroh control availability remains a prerequisite. Authenticated physical-link
delivery probes detect sustained loss on that carrier, independently of a
working end-to-end relay path. They never confer membership or enter the TUN. See
[amnezia.md](amnezia.md) for limits and tested scope.
+64 -95
View File
@@ -1,6 +1,6 @@
# The WireGuard data plane
WireGuard is the first IP plugin. It carries user traffic between
WireGuard is the single internal IP cipher. It carries user traffic between
participants while the control plane keeps doing its own job: deciding who is
in the network and carrying each participant's opaque announcement.
@@ -46,20 +46,26 @@ peer by the agent and runs a WireGuard tunnel over it:
source address checked <--Tunn.decapsulate-- ciphertext <- transport
```
Reachability — hole punching, relay fallback — belongs to the transport, which
today is iroh. That is the whole reason the plugin's announcement says *who* it
is and never *where* it is: there is no address for a peer to advertise, get
wrong, or lie about.
Reachability belongs to external carriers: ordinary WireGuard UDP (`wg`),
Amnezia-masked UDP (`awg`), iroh QUIC (`wg-quic`), or TCP/TLS (`tcp-tls`). The
user's ordered choice applies only to those physical data links. The internal
capability `wireguard-e2e` v4 is fixed and hidden from the CLI/tray controls.
All carriers move the same inner encrypted bytes, including through relays.
The encryption plugin's announcement identifies a key, never an IP endpoint.
**Two peers behind NAT work exactly as well as iroh does.** iroh hole punches a
direct path when it can and falls back to a relay when it cannot; the tunnel
rides on whichever it got. There is no separate STUN, no separate hole punching
and no second set of NAT problems to solve for WireGuard.
A relay removes its adjacent carrier's encryption to read routing metadata,
then sends the inner ciphertext on another carrier. It cannot decrypt the
endpoint IP packet. WireGuard session keys, counters and the interface survive
carrier changes and direct/relay changes. TCP inside the overlay retransmits
after transient loss; ordinary application and WireGuard timeouts still apply.
The UDP modes have their own bounded STUN/pair discovery and can fail on NATs
that iroh can traverse; selecting only AWG does not permit QUIC fallback.
## Checking it from outside
`tsng status` asks a running agent over its local control socket and prints
what it sees, including whether each tunnel has actually handshaken. See
the external transport or relay route, data readiness and plaintext counters.
An unavailable channel includes a reason such as no common permitted transport. See
[../README.md](../README.md#checking-that-it-works).
## Deterministic overlay addressing
@@ -207,26 +213,40 @@ QUIC DATAGRAM frames themselves cannot fragment.
Reassembly is bounded and expires incomplete packets; one lost fragment loses
one packet, without blocking unrelated traffic. The logical data payload limit
is 64 KiB, with the relay envelope subtracted before it reaches the plugin.
The new framing uses data ALPN `tsunagi/data/5`; both ends and intermediate
The new framing uses data ALPN `tsunagi/data/6`; both ends and intermediate
peers need the updated binary. Network identities and saved state do not change.
See [protocol.md](protocol.md#the-data-plane-protocol) for the wire format.
## Lifecycle
## Lifecycle and counters
* A network is activated → the plugin loads or creates its key for that
network, derives the interface name, and creates the packet interface. If
that fails — no privileges, for instance — the key and the announcement still
work and the interface is retried on the next reconcile.
* A peer announces its key → recorded.
* A data link to that peer arrives → recorded.
* Reconciliation starts a tunnel for every peer that has **both**, and removes
tunnels for peers that lost either.
* A network is deactivated, or the agent shuts down → the interface and every
tunnel go away. The key stays, so coming back keeps the same overlay address.
The agent owns one interface across every network. On network activation the
internal plugin loads a network-specific key, announces it and reconciles
peer tunnels. It owns no interface or host route. Local IP ingress resolves
the destination owner and sends only through this one plugin. Source admission
is enforced after decryption by the agent, not by trusting a remote address.
There is no external configuration file and no command line tool, so unlike a
kernel-WireGuard setup there is nothing outside this process for anybody to
edit. Reconciliation is purely "do the running tunnels match what is known".
A carrier policy change retires physical links while retaining the logical
peer link and the internal tunnel. A peer's control membership ending removes
its tunnel; network deactivation removes that network's channels. Its persistent
key remains. The interface lives until agent shutdown, independently of carrier
availability and individual network lifetimes.
The historical `wg-quic/wireguard.sqlite` remains the canonical key store.
Missing keys migrate from `wg` then `awg` without replacing canonical keys or
modifying the source files. Corrupt legacy databases are hard errors. Old
`-o keepalive=…`, `-o wg:keepalive=…` and `mtu` settings configure the shared
internal channel; contradictory values are rejected. Interface MTU is selected
once with `--mtu`, independent of carrier selection.
User traffic counters count original IP bytes once when the encrypted frame is
accepted by the logical transport, and once when decrypted on receipt. They do
not promise remote delivery. They exclude carrier overhead, handshakes,
keepalives, relay transit and carrier delivery probes. Packets queued before a
handshake retain their exact original lengths. Each new tunnel gets an opaque
counter epoch so tray deltas cannot cross a restart, including when its new
counter already exceeds the old one. A carrier switch does not change that
epoch. Network rates sum per-peer deltas; peers arriving with prior traffic
never add that history to the current rate.
## Using it
@@ -238,71 +258,21 @@ tsng join --network lab --secret "$SECRET"
See the two-machine walkthrough in [../README.md](../README.md#trying-it-on-two-machines).
From the library:
From the library, configure one `WireguardPlugin` with
`AgentConfig::with_tunnel`, the agent's `with_interface`, and independent
`with_carrier` adapters. UDP/TLS carriers also register their physical
`PacketTransport` using `with_custom_transport`. Their own accept tasks pass
real links to `Agent::install_inbound_link`. The `carriers.rs` integration test
is a rootless example for ordinary WG and TLS; `tsunagi-awg/tests/pairs.rs`
covers mixed transports and relay transitions.
```rust,no_run
use std::sync::Arc;
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
use tsunagi::dataplane::IpPlugin;
use tsunagi::dataplane::wireguard::{MemoryTunFactory, WireguardConfig, WireguardPlugin};
use tsunagi::identity::{NetworkName, NetworkSecret};
use tsunagi::{Agent, Result};
#[tokio::main]
async fn main() -> Result<()> {
let paths = StoragePaths::user_default()?;
// MemoryTunFactory needs no privileges; swap in ManagedTunFactory for a
// real interface.
let plugin = WireguardPlugin::open(
WireguardConfig::new(paths.state_dir.join("wireguard")),
Arc::new(MemoryTunFactory::new()),
)
.await
.expect("wireguard plugin");
let agent = Agent::spawn(
AgentConfig::new(paths)
.with_transport(TransportPolicy::N0Defaults)
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
)
.await?;
let network = agent
.join_network(&NetworkName::new("lab")?, &NetworkSecret::generate())
.await?;
if let Some(view) = plugin.overview(network) {
println!("{} on {}", view.interface, view.overlay_address);
}
agent.shutdown().await;
Ok(())
}
```
## Limits and future work
* **Full mesh only.** Every member runs a tunnel to every other member.
Routing through an intermediate participant is not implemented.
* **Nothing frees an address yet.** The release record exists and merges, but
no command emits one.
* **A snapshot grows with the number of members ever seen**, and is capped per
message rather than compacted.
* **No routes, DNS or firewall rules.** The plugin creates its interface and
nothing else. Anything beyond the overlay `/64` is the operator's business.
* **Membership is session-scoped.** A peer leaves the overlay when its control
session ends; surviving a long absence is the same future work.
* **Userspace costs CPU.** Kernel WireGuard is faster. A kernel backend could
return behind the same boundary, but it would give up transport-provided NAT
traversal unless paired with a local proxy.
* **The interface is managed, not prepared.** On Linux the agent creates the
TUN interface and configures it over netlink, in process. See
*Provisioning* below.
* **No provisioner exists for macOS or Windows yet.** There the agent attaches
to an interface prepared by hand and says so.
* **The system interface path is not exercised by the default suite**, because
it needs privileges. Everything else about the data plane is.
## Limits
The default suite exercises real cryptography, sockets, handshakes, storage and
IP routing using memory TUNs. It does not establish public NAT traversal, DPI
effectiveness, actual host TUN performance, or native GUI behavior on every OS.
Userspace encryption costs CPU. Reassembly and packet queues are bounded and
may drop packets; host TCP handles retransmission within its own timeouts.
## Provisioning the interface
@@ -313,7 +283,7 @@ should carry and no others — observes what is actually on the host, and
applies the difference. Running it twice changes nothing the second time.
The decision of *what* to change is
`dataplane::wireguard::provision::plan_changes`: pure, platform-independent
`overlay::provision::plan_changes`: pure, platform-independent
and unit-tested on every platform. Only the execution is behind
`InterfaceProvisioner`, which has three implementations — netlink on Linux, a
`MockProvisioner` over a pretend host for the tests, and one that refuses with
@@ -342,13 +312,12 @@ addresses with them.
The two refusals are the interesting part:
* **A link that is not a TUN is never touched.** The name is derived from the
network id, so colliding with a real device is unlikely rather than
impossible, and deleting somebody's bridge is not a recoverable mistake.
* **A link that is not a TUN is never touched.** The interface name is supplied locally, so colliding with a real device is
possible, and deleting somebody's bridge is not a recoverable mistake.
* **A TUN another process holds open is never deleted.** Carrier is the
signal: a TUN has it exactly while something is attached. An attached one is
a working overlay, almost certainly a second agent on this host, and it is
told to use a different `--wg-prefix` instead.
told to use a different `--interface` instead.
### Privilege
@@ -374,6 +343,6 @@ construction.
### What it cannot be told to do
Nothing here takes a name, an address or a command from the network. The
interface name is derived from the network id, the addresses come from the
local plugin and the signed allocation records, and no external program is
interface name comes from local configuration, the addresses come from the
signed allocation records, and no external program is
executed at any point — there is no `ip`, no shell and no `PATH` involved.