Compare commits
11
Commits
ca8759c023
...
990b9f2e0f
| Author | SHA1 | Date | |
|---|---|---|---|
  tsunagiandClaude Opus 5
|
990b9f2e0f | ||
|
tsunagiandClaude Opus 5
|
142fdf995c | ||
|
tsunagiandClaude Opus 5
|
ff7e235414 | ||
|
tsunagiandClaude Opus 5
|
7c1be332e3 | ||
|
tsunagiandClaude Opus 5
|
9415866193 | ||
|
tsunagiandClaude Opus 5
|
c852de2c78 | ||
|
tsunagiandClaude Opus 5
|
a80a014f20 | ||
|
tsunagiandClaude Opus 5
|
41d5f554c7 | ||
|
tsunagiandClaude Opus 5
|
745bbaea06 | ||
|
tsunagiandClaude Opus 5
|
5bee148497 | ||
|
tsunagiandClaude Opus 5
|
60e6b263d1 |
No files matched your search
@@ -18,18 +18,31 @@ Design accordingly: a majority is not a root of trust.
|
||||
Keep these separate. Crossing them is the main thing to review for.
|
||||
|
||||
- **Control plane vs data plane.** The separation is **logical, not physical**.
|
||||
The control protocol in `src/proto/` knows nothing about packets, and
|
||||
`src/dataplane/` knows nothing about the control protocol; either can be
|
||||
replaced on its own. Both may ride on iroh — refusing to would throw away
|
||||
iroh's NAT traversal and force the data plane to reimplement it. They use
|
||||
different ALPNs and different connections, so a busy or broken data plane
|
||||
cannot disturb control traffic.
|
||||
The control protocol in `crates/tsunagi/src/proto/` knows nothing about
|
||||
packets, and a protocol crate knows nothing about the control protocol;
|
||||
either can be replaced on its own. Both may ride on iroh — refusing to
|
||||
would throw away iroh's NAT traversal and force the data plane to
|
||||
reimplement it. They use different ALPNs and different connections, so a
|
||||
busy or broken data plane cannot disturb control traffic.
|
||||
- **One agent, one interface.** It belongs to the system level, along with
|
||||
the addresses on it and the decision of whose packet is whose. Several
|
||||
protocols may be carrying traffic at once and none of them owns the thing
|
||||
they carry it for. A protocol is handed a routed packet and hands back a
|
||||
decrypted one; it never creates an interface and never picks an address.
|
||||
- **A protocol is a separate crate with its own version.** The version peers
|
||||
compare is the *wire* version, never the software version: two peers on
|
||||
different releases work together for as long as the bytes between them
|
||||
have not changed. Nothing negotiated may be derived from anything that
|
||||
moves with a release.
|
||||
- **A protocol never sees the network secret.** It proves who is at the other
|
||||
end of a tunnel; proving membership of a network stays in the core, which
|
||||
is why the authenticated transport does too.
|
||||
- **Plugins never learn reachability.** An `IpPlugin` is handed a `PacketLink`
|
||||
per peer and moves datagrams over it. Addresses, hole punching and relays
|
||||
belong to `src/dataplane/transport/`. A plugin announcement says *who*, never
|
||||
belong to `crates/tsunagi/src/dataplane/transport/`. A plugin announcement says *who*, never
|
||||
*where*.
|
||||
- **The core never parses a plugin payload.** See `src/dataplane/mod.rs`. Only
|
||||
`src/dataplane/wireguard/announcement.rs` interprets WireGuard payloads, and
|
||||
- **The core never parses a plugin payload.** See `crates/tsunagi/src/dataplane/mod.rs`. Only
|
||||
`crates/tsunagi-wg-quic/src/announcement.rs` interprets `wg-quic` payloads, and
|
||||
only after bounding every field. A data plane failure must never stop the
|
||||
control plane.
|
||||
- **Derived, not claimed.** A peer's overlay address is derived from its public
|
||||
@@ -37,7 +50,7 @@ Keep these separate. Crossing them is the main thing to review for.
|
||||
inbound packets are dropped unless their source is the address derived for
|
||||
the peer that sent them. Never trust an address a peer announces.
|
||||
- **Signed state is the only durable agreement.** A fact that must survive a
|
||||
participant being away goes in `src/state/` as a record signed by its
|
||||
participant being away goes in `crates/tsunagi/src/state/` as a record signed by its
|
||||
author, never in a session. Merging is deterministic, an older version never
|
||||
rolls back a newer one, and absence from a snapshot is not deletion. Never
|
||||
add a vote or a quorum: a majority is not a trust root here, and it would
|
||||
@@ -65,7 +78,7 @@ Keep these separate. Crossing them is the main thing to review for.
|
||||
normal path *and* the important failures. See [docs/testing.md](docs/testing.md).
|
||||
Do not add tests for getters or to move a coverage number.
|
||||
2. **Never change `IDENTITY_SCHEME`, the derivation labels, or the transcript
|
||||
encoding** in `src/identity/network.rs` and `src/proto/handshake.rs` without
|
||||
encoding** in `crates/tsunagi/src/identity/network.rs` and `crates/tsunagi/src/proto/handshake.rs` without
|
||||
treating it as an incompatible protocol change. Bumping the crate version or
|
||||
the control protocol version must not change an existing `NetworkId`.
|
||||
3. **Secrets never leak.** Not into logs, not into `Debug`, not into status
|
||||
@@ -76,7 +89,7 @@ Keep these separate. Crossing them is the main thing to review for.
|
||||
the library; tests opt out explicitly at the top of each file.
|
||||
5. **Bounds before allocation.** Frame lengths are checked against
|
||||
`Limits::max_frame_len` before a buffer is allocated. Every string, list and
|
||||
queue has a limit in `src/config.rs`.
|
||||
queue has a limit in `crates/tsunagi/src/config.rs`.
|
||||
6. **Failure is contained.** A bad signature, wrong secret, malformed packet or
|
||||
unknown version rejects one message or one session. It never stops another
|
||||
network and never stops the agent. There is no irreversible global error
|
||||
@@ -99,19 +112,28 @@ Keep these separate. Crossing them is the main thing to review for.
|
||||
|
||||
| path | responsibility |
|
||||
|---------------------|----------------|
|
||||
| `src/identity/` | device identity; deterministic network space identity and derived keys |
|
||||
| `src/storage/` | `state.sqlite`, `cache.sqlite`, directory ownership lock |
|
||||
| `src/discovery.rs` | candidate sources; test and static backends |
|
||||
| `src/proto/` | framing, message formats, membership handshake |
|
||||
| `src/net.rs` | iroh endpoint adapter and observability snapshots |
|
||||
| `src/agent/` | agent lifecycle, per-network runtimes, sessions, events, status |
|
||||
| `src/state/` | signed records that outlive a session, their merge rules and address allocation |
|
||||
| `src/dataplane/` | the plugin contract, the packet transport, and the WireGuard plugin |
|
||||
| `src/bin/tsunagi.rs`| the command line agent; the only place that owns a runtime, a logger and signals |
|
||||
| `crates/tsunagi/src/identity/` | device identity; deterministic network space identity and derived keys |
|
||||
| `crates/tsunagi/src/storage/` | `state.sqlite`, `cache.sqlite`, directory ownership lock |
|
||||
| `crates/tsunagi/src/discovery.rs` | candidate sources; test and static backends |
|
||||
| `crates/tsunagi/src/proto/` | framing, message formats, membership handshake |
|
||||
| `crates/tsunagi/src/net.rs` | iroh endpoint adapter and observability snapshots |
|
||||
| `crates/tsunagi/src/agent/` | agent lifecycle, per-network runtimes, sessions, events, status |
|
||||
| `crates/tsunagi/src/state/` | signed records that outlive a session, their merge rules and address allocation |
|
||||
| `crates/tsunagi/src/overlay/` | the one interface an agent owns: provisioning, the TUN, routing, source checks |
|
||||
| `crates/tsunagi/src/dataplane/` | the protocol contract and the authenticated packet transport |
|
||||
| `crates/tsunagi/src/dns/` | the DNS view of a network: zone, server, resolver publication |
|
||||
| `crates/tsunagi-wg-quic/` | the `wg-quic` protocol: its keys, its announcement, its tunnels |
|
||||
| `crates/tsunagi-cli/` | the command line agent; the only place that owns a runtime, a logger and signals |
|
||||
| `tests/` | integration tests; `tests/common/` is the shared harness |
|
||||
|
||||
Add abstractions only at real substitution or testing boundaries. Do not add a
|
||||
trait per struct. Prefer one crate with clear modules over many small crates.
|
||||
trait per struct.
|
||||
|
||||
**The system level and its plugins are separate crates**, so that boundary is
|
||||
checked by the compiler and not by discipline: a plugin can reach only what
|
||||
`tsunagi` makes public, and carries its own version. Everything else stays
|
||||
one crate with clear modules — do not split further without a reason of that
|
||||
kind.
|
||||
|
||||
## Testing rules
|
||||
|
||||
|
||||
Generated
+44
-4
@@ -3821,12 +3821,8 @@ checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
|
||||
name = "tsunagi"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"anstream",
|
||||
"anstyle",
|
||||
"boringtun",
|
||||
"bytes",
|
||||
"caps",
|
||||
"clap",
|
||||
"data-encoding",
|
||||
"directories",
|
||||
"fs4",
|
||||
@@ -3851,11 +3847,55 @@ dependencies = [
|
||||
"tokio",
|
||||
"tracing",
|
||||
"tracing-subscriber",
|
||||
"tsunagi",
|
||||
"tun",
|
||||
"zbus",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tsunagi-cli"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"anstream",
|
||||
"anstyle",
|
||||
"clap",
|
||||
"iroh",
|
||||
"netwatch",
|
||||
"simple-dns",
|
||||
"tempfile",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"tracing-subscriber",
|
||||
"tsunagi",
|
||||
"tsunagi-wg-quic",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tsunagi-wg-quic"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"boringtun",
|
||||
"bytes",
|
||||
"data-encoding",
|
||||
"hex",
|
||||
"hkdf",
|
||||
"iroh",
|
||||
"postcard",
|
||||
"rand 0.10.3",
|
||||
"rusqlite",
|
||||
"serde",
|
||||
"sha2",
|
||||
"subtle",
|
||||
"tempfile",
|
||||
"thiserror 2.0.20",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"tracing-subscriber",
|
||||
"tsunagi",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tun"
|
||||
version = "0.8.14"
|
||||
|
||||
+23
-83
@@ -1,94 +1,34 @@
|
||||
[package]
|
||||
name = "tsunagi"
|
||||
version = "0.1.0"
|
||||
# The system level and its plugins are separate crates, so the boundary
|
||||
# between them is checked by the compiler rather than by discipline: a plugin
|
||||
# can reach only what the core makes public, and carries its own version.
|
||||
[workspace]
|
||||
resolver = "3"
|
||||
members = ["crates/*"]
|
||||
|
||||
[workspace.package]
|
||||
edition = "2024"
|
||||
rust-version = "1.91"
|
||||
license = "MIT OR Apache-2.0"
|
||||
description = "Proof-of-concept library for small private mesh networks: persistent agent identity, deterministic network spaces, iroh-based control plane."
|
||||
repository = "https://github.com/tsunagi-net/tsunagi"
|
||||
readme = "README.md"
|
||||
keywords = ["mesh", "p2p", "iroh", "networking"]
|
||||
categories = ["network-programming"]
|
||||
|
||||
[features]
|
||||
default = ["cli"]
|
||||
# The `tsunagi` command line binary. Library users can opt out.
|
||||
cli = ["dep:clap", "dep:anstream", "dep:anstyle", "dep:tracing-subscriber", "tokio/signal", "tun-device", "dns-publish"]
|
||||
# A real TUN device, so the WireGuard plugin can carry actual IP traffic.
|
||||
# Needs CAP_NET_ADMIN at run time; without it the plugin still runs and its
|
||||
# in-memory device can be used for tests.
|
||||
tun-device = ["dep:tun", "dep:rtnetlink", "dep:caps", "dep:futures-util"]
|
||||
# Telling the operating system where to send its questions. Linux only for
|
||||
# now; the zone and the server work without it.
|
||||
dns-publish = ["dep:zbus"]
|
||||
|
||||
[[bin]]
|
||||
name = "tsunagi"
|
||||
path = "src/bin/tsunagi.rs"
|
||||
required-features = ["cli"]
|
||||
|
||||
[dependencies]
|
||||
clap = { version = "4.5", features = ["derive", "env"], optional = true }
|
||||
# Already in the tree through clap. `anstream` strips the escapes when stdout
|
||||
# is not a terminal and turns on virtual terminal processing on Windows, so
|
||||
# colour is never written where it would show up as rubbish.
|
||||
anstream = { version = "1.0", optional = true }
|
||||
anstyle = { version = "1.0", optional = true }
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"], optional = true }
|
||||
iroh = { version = "1.2", default-features = false, features = ["tls-ring"] }
|
||||
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros"] }
|
||||
rusqlite = { version = "0.40", features = ["bundled"] }
|
||||
hkdf = "0.13"
|
||||
hmac = "0.13"
|
||||
sha2 = "0.11"
|
||||
subtle = "2.6"
|
||||
zeroize = { version = "1.9", features = ["derive"] }
|
||||
rand = "0.10"
|
||||
serde = { version = "1.0", features = ["derive"] }
|
||||
postcard = { version = "1.1", default-features = false, features = ["use-std"] }
|
||||
# Already in the tree through iroh. A packet codec, not a DNS server: the
|
||||
# zone logic is ours and a full server framework would be a large dependency
|
||||
# for answering A records from memory.
|
||||
simple-dns = "0.12"
|
||||
# The IANA top-level domain list, compiled in: one function, no
|
||||
# dependencies, no network. Used only to warn that a zone name shadows a
|
||||
# real public domain, never to refuse one.
|
||||
tld = "2.40"
|
||||
data-encoding = "2.11"
|
||||
hex = "0.4"
|
||||
thiserror = "2.0"
|
||||
tracing = "0.1"
|
||||
fs4 = { version = "1.1", features = ["sync"] }
|
||||
directories = "6.0"
|
||||
# The real system hostname, without a libc call of our own: this crate is
|
||||
# forbidden `unsafe` and will not make one.
|
||||
gethostname = "1.1"
|
||||
netwatch = "0.19.3"
|
||||
bytes = "1.12.1"
|
||||
boringtun = { version = "0.7.1", default-features = false }
|
||||
tun = { version = "0.8", features = ["async"], optional = true }
|
||||
|
||||
# Linux-only interface provisioning. `rtnetlink` configures the interface in
|
||||
# process, so no `ip` invocation is ever needed; `caps` keeps CAP_NET_ADMIN
|
||||
# out of the effective set except during the moments it is used.
|
||||
[target.'cfg(target_os = "linux")'.dependencies]
|
||||
rtnetlink = { version = "0.23", optional = true }
|
||||
# Pure Rust D-Bus, no libdbus to link against. `tokio` rather than the
|
||||
# default reactor, because the agent brings its own.
|
||||
zbus = { version = "5.19", default-features = false, features = ["tokio"], optional = true }
|
||||
caps = { version = "0.5", optional = true }
|
||||
futures-util = { version = "0.3", default-features = false, optional = true }
|
||||
|
||||
[dev-dependencies]
|
||||
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros", "process"] }
|
||||
tempfile = "3.24"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
|
||||
[lints.rust]
|
||||
[workspace.lints.rust]
|
||||
missing_docs = "warn"
|
||||
unsafe_code = "forbid"
|
||||
|
||||
[lints.clippy]
|
||||
[workspace.lints.clippy]
|
||||
unwrap_used = "warn"
|
||||
expect_used = "warn"
|
||||
panic = "warn"
|
||||
|
||||
[workspace.dependencies]
|
||||
iroh = { version = "1.2", default-features = false, features = ["tls-ring"] }
|
||||
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros"] }
|
||||
serde = { version = "1.0", features = ["derive"] }
|
||||
postcard = { version = "1.1", default-features = false, features = ["use-std"] }
|
||||
bytes = "1.12.1"
|
||||
thiserror = "2.0"
|
||||
tracing = "0.1"
|
||||
hex = "0.4"
|
||||
data-encoding = "2.11"
|
||||
tempfile = "3.24"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
@@ -73,26 +73,26 @@ cargo build --release
|
||||
./target/release/tsunagi id secret generate # prints tsn1...; share it privately
|
||||
./target/release/tsunagi status # this device, the agent, and this host
|
||||
|
||||
./target/release/tsunagi up --network lab --secret "$SECRET" --wireguard
|
||||
./target/release/tsunagi up --network lab --secret "$SECRET"
|
||||
```
|
||||
|
||||
It prints its endpoint id and then waits. On the second machine, pass that id:
|
||||
|
||||
```bash
|
||||
./target/release/tsunagi up --network lab --secret "$SECRET" --wireguard \
|
||||
./target/release/tsunagi up --network lab --secret "$SECRET" \
|
||||
--peer <endpoint-id-from-the-first-machine>
|
||||
```
|
||||
|
||||
Within a few seconds both print something like:
|
||||
|
||||
```text
|
||||
+ peer b47c958462 connected over Direct rtt=Some(4.5ms)
|
||||
+ data link to b47c958462 for wireguard: Direct via Ip(…), datagram 1382
|
||||
+ peer b47c958462 connected over direct rtt=Some(4.5ms)
|
||||
+ data link to b47c958462 for wg-quic: direct via 192.0.2.7:41234, datagram 1382
|
||||
|
||||
--- status ---
|
||||
control: 1 peer(s), 0 dial failure(s), 0 handshake failure(s)
|
||||
wireguard: tsunkkcp43lmdje on fd15:1d9e:fa21:f201:…/64 mtu 1280, 1/1 tunnel(s) established
|
||||
4jO4kx9Z fd15:1d9e:fa21:f201:… handshake 3s ago tx=0 rx=0 dropped=0 path=Direct via Ip(…)
|
||||
wg-quic: tsun0 on 10.13.37.69/24 mtu 1280, 1/1 tunnel(s) established
|
||||
4jO4kx9Z 10.13.37.237 handshake 3s ago tx=0 rx=0 dropped=0 path=direct via 192.0.2.7:41234
|
||||
```
|
||||
|
||||
`1/1 tunnel(s) established` means a real WireGuard handshake completed.
|
||||
@@ -134,8 +134,8 @@ id and collides with essentially nothing.
|
||||
what they find, so `--ipv4-range` only matters for whoever starts the network:
|
||||
|
||||
```bash
|
||||
tsunagi up --network lab --secret "$SECRET" --wireguard --ipv4-range 10.44.0.0/16
|
||||
tsunagi up --network lab --secret "$SECRET" --wireguard --ipv4-range none # IPv6 only
|
||||
tsunagi up --network lab --secret "$SECRET" --ipv4-range 10.44.0.0/16
|
||||
tsunagi up --network lab --secret "$SECRET" --ipv4-range none # IPv6 only
|
||||
```
|
||||
|
||||
An address is claimed with a record signed by that member's persistent device
|
||||
@@ -149,13 +149,48 @@ Because the address is allocated at run time rather than derived, it is not
|
||||
known until the agent has started and agreed with its peers. The agent then
|
||||
assigns it to the interface itself.
|
||||
|
||||
## Levels
|
||||
|
||||
The command line is split the way the design is. A bare `tsunagi up` needs
|
||||
only a network name and a secret; everything else sits under the level it
|
||||
belongs to, which `--help` shows as two sections:
|
||||
|
||||
* **System** — what the agent itself does: how it reaches peers (`--reach`),
|
||||
the one overlay interface it owns (`--interface`, `--mtu`, `--no-tun`), the
|
||||
address range (`--ipv4-range`), and the local resolver (`--dns`).
|
||||
* **Transport** — which protocols carry packets (`--protocol`, a list) and
|
||||
their own settings (`-o key=value`, or `-o protocol:key=value`).
|
||||
|
||||
```bash
|
||||
tsunagi protocols # what this build can carry packets with
|
||||
```
|
||||
|
||||
```text
|
||||
wg-quic (wire version 4)
|
||||
what WireGuard's cryptography carried in iroh's QUIC datagrams
|
||||
-o keepalive=SECONDS keeps a tunnel and its link warm through a NAT
|
||||
-o mtu=BYTES largest packet a tunnel will carry, at least 576
|
||||
```
|
||||
|
||||
Each protocol declares its own settings, so the agent can list them without
|
||||
knowing anything about the protocol, and a setting no selected protocol takes
|
||||
is refused rather than ignored. `--protocol none` runs the control plane by
|
||||
itself.
|
||||
|
||||
A pair of peers uses a protocol they both have **at the same wire version**.
|
||||
That is not the software version: two peers on different builds carry traffic
|
||||
for each other for as long as the bytes between them have not changed. A peer
|
||||
with nothing in common keeps its control plane — messages and signed state
|
||||
still flow — and simply has no data plane, which `tsunagi status` shows as a
|
||||
session with no agreed protocol.
|
||||
|
||||
## Names
|
||||
|
||||
`--dns` serves a local DNS zone for the network's members, so they can be
|
||||
reached by name instead of by address:
|
||||
|
||||
```bash
|
||||
tsunagi up --network lab --secret "$SECRET" --wireguard --dns
|
||||
tsunagi up --network lab --secret "$SECRET" --dns
|
||||
dig @10.13.37.69 -p 5354 music.lab
|
||||
```
|
||||
|
||||
@@ -349,7 +384,6 @@ tests:
|
||||
|
||||
```bash
|
||||
cargo run --example two_agents # control plane only
|
||||
cargo run --example wireguard_mesh # a WireGuard overlay carrying a real packet
|
||||
```
|
||||
|
||||
Both run with no privileges and change nothing on the host.
|
||||
@@ -407,7 +441,7 @@ explicitly.
|
||||
Two different lookups are involved, and only one of them is this project's:
|
||||
|
||||
**1. Resolving one endpoint's address — iroh's, and it works today.**
|
||||
With `--transport relay` or `--transport direct`, iroh publishes a signed
|
||||
With `--reach relay` or `--reach direct`, iroh publishes a signed
|
||||
record of this endpoint's addresses, keyed by its endpoint id, to the public
|
||||
service run by Number 0 — "n0", the company behind iroh — at `dns.iroh.link`,
|
||||
over pkarr and DNS, and resolves other endpoints the same way. That is why `--peer <endpoint-id>` works with no address attached:
|
||||
@@ -426,7 +460,7 @@ What this means in practice:
|
||||
- With `relay` or `direct`, **your endpoint id and IP addresses are published
|
||||
to a public third-party service** (Number 0's, unless you change it). They are not secret, and the network secret is
|
||||
never published, but an observer of that service learns that your endpoint
|
||||
exists and where it is. `--transport local` publishes nothing.
|
||||
exists and where it is. `--reach local` publishes nothing.
|
||||
- A relay, when one is needed, sees the volume and timing of your traffic — not
|
||||
its contents. The default relays are Number 0's, in the US, EU and
|
||||
Asia-Pacific.
|
||||
@@ -442,9 +476,9 @@ paths; tests always use temporary directories.
|
||||
| `state.sqlite` | device identity, network configuration, hostname | clear error, never reset |
|
||||
| `cache.sqlite` | address hints and other recoverable data | discarded and recreated |
|
||||
|
||||
The WireGuard plugin keeps its own keys in its own `wireguard.sqlite`, wherever
|
||||
its configuration points, because plugin keys are neither the iroh identity nor
|
||||
the network secret.
|
||||
The `wg-quic` protocol keeps its own keys in its own store under `wg-quic/`,
|
||||
because a protocol's keys are neither the iroh identity nor the network
|
||||
secret.
|
||||
|
||||
The command line agent puts everything under the platform's per-user
|
||||
directories by default; `--state-dir` and `--cache-dir` override them.
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
[package]
|
||||
name = "tsunagi-cli"
|
||||
version = "0.1.0"
|
||||
description = "The tsunagi command line agent."
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
repository.workspace = true
|
||||
|
||||
[[bin]]
|
||||
name = "tsunagi"
|
||||
path = "src/main.rs"
|
||||
|
||||
[dependencies]
|
||||
tsunagi = { path = "../tsunagi", version = "0.1.0", features = ["tun-device", "dns-publish"] }
|
||||
tsunagi-wg-quic = { path = "../tsunagi-wg-quic", version = "0.1.0" }
|
||||
iroh.workspace = true
|
||||
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros", "signal"] }
|
||||
tracing.workspace = true
|
||||
tracing-subscriber.workspace = true
|
||||
clap = { version = "4.5", features = ["derive", "env"] }
|
||||
# Already in the tree through clap. `anstream` strips the escapes when stdout
|
||||
# is not a terminal and turns on virtual terminal processing on Windows, so
|
||||
# colour is never written where it would show up as rubbish.
|
||||
anstream = "1.0"
|
||||
anstyle = "1.0"
|
||||
# Local interface addresses, for the diagnostics in `status`.
|
||||
netwatch = "0.19.3"
|
||||
|
||||
[dev-dependencies]
|
||||
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros", "process"] }
|
||||
tempfile.workspace = true
|
||||
simple-dns = "0.12"
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
@@ -15,14 +15,13 @@ use clap::{Args, Parser, Subcommand, ValueEnum};
|
||||
use tsunagi::agent::Event;
|
||||
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
|
||||
use tsunagi::dataplane::IpPlugin;
|
||||
use tsunagi::dataplane::wireguard::{
|
||||
MemoryTunFactory, TunFactory, WireguardConfig, WireguardPlugin,
|
||||
};
|
||||
use tsunagi::discovery::{CompositeDiscovery, NetworkDiscovery, StaticBootstrap};
|
||||
use tsunagi::identity::{NetworkName, NetworkSecret};
|
||||
use tsunagi::iroh_types::EndpointAddr;
|
||||
use tsunagi::overlay::{MemoryTunFactory, TunFactory};
|
||||
use tsunagi::state::Ipv4Range;
|
||||
use tsunagi::{Agent, NetworkId};
|
||||
use tsunagi_wg_quic::{WireguardConfig, WireguardPlugin};
|
||||
|
||||
/// A small agent for private mesh networks.
|
||||
#[derive(Debug, Parser)]
|
||||
@@ -50,6 +49,8 @@ enum Command {
|
||||
Up(Box<UpArgs>),
|
||||
/// Reports this device, what the agent is doing, and what this host can do.
|
||||
Status(StatusArgs),
|
||||
/// Shows the protocols this build can carry packets with.
|
||||
Protocols,
|
||||
}
|
||||
|
||||
#[derive(Debug, Args)]
|
||||
@@ -160,7 +161,7 @@ impl PathArgs {
|
||||
/// behind iroh — at `dns.iroh.link`, and resolve peers through it. That is
|
||||
/// what makes `--peer <endpoint-id>` work without an address.
|
||||
#[derive(Debug, Clone, Copy, ValueEnum)]
|
||||
enum Transport {
|
||||
enum Reach {
|
||||
/// Loopback and the local network only. Publishes nothing.
|
||||
Local,
|
||||
/// Public address lookup, direct paths only, no relays.
|
||||
@@ -170,12 +171,12 @@ enum Transport {
|
||||
Relay,
|
||||
}
|
||||
|
||||
impl From<Transport> for TransportPolicy {
|
||||
fn from(value: Transport) -> Self {
|
||||
impl From<Reach> for TransportPolicy {
|
||||
fn from(value: Reach) -> Self {
|
||||
match value {
|
||||
Transport::Local => TransportPolicy::LocalOnly,
|
||||
Transport::Direct => TransportPolicy::DirectOnly,
|
||||
Transport::Relay => TransportPolicy::N0Defaults,
|
||||
Reach::Local => TransportPolicy::LocalOnly,
|
||||
Reach::Direct => TransportPolicy::DirectOnly,
|
||||
Reach::Relay => TransportPolicy::N0Defaults,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -203,44 +204,70 @@ struct UpArgs {
|
||||
secret_file: Option<PathBuf>,
|
||||
|
||||
/// Hostname to announce. Defaults to the machine's.
|
||||
#[arg(long)]
|
||||
#[arg(long, help_heading = "System")]
|
||||
hostname: Option<String>,
|
||||
|
||||
/// How much external connectivity to use.
|
||||
#[arg(long, value_enum, default_value_t = Transport::Relay)]
|
||||
transport: Transport,
|
||||
/// How much of iroh's reachability to use.
|
||||
///
|
||||
/// About how the *control plane* finds peers, not about which protocol
|
||||
/// carries packets — that is `--protocol`.
|
||||
#[arg(long, value_enum, default_value_t = Reach::Relay, help_heading = "System")]
|
||||
reach: Reach,
|
||||
|
||||
/// A peer to contact, as `<endpoint-id>` or `<endpoint-id>@<ip:port>,...`.
|
||||
///
|
||||
/// One agent needs to know another to begin with. Repeat for several.
|
||||
#[arg(long = "peer", value_name = "PEER")]
|
||||
#[arg(long = "peer", value_name = "PEER", help_heading = "System")]
|
||||
peers: Vec<String>,
|
||||
|
||||
/// Local address to bind. Repeat for several; defaults to iroh's choice.
|
||||
#[arg(long = "bind", value_name = "ADDR")]
|
||||
#[arg(long = "bind", value_name = "ADDR", help_heading = "System")]
|
||||
binds: Vec<SocketAddr>,
|
||||
|
||||
/// Run the WireGuard data plane.
|
||||
#[arg(long)]
|
||||
wireguard: bool,
|
||||
/// Name of the overlay interface. One agent has one, whatever carries it.
|
||||
#[arg(
|
||||
long,
|
||||
default_value = "tsun0",
|
||||
value_name = "NAME",
|
||||
help_heading = "System"
|
||||
)]
|
||||
interface: String,
|
||||
|
||||
/// Largest packet the overlay carries, at least 576.
|
||||
#[arg(long, value_name = "BYTES", help_heading = "System")]
|
||||
mtu: Option<u32>,
|
||||
|
||||
/// Do not create a real network interface.
|
||||
///
|
||||
/// The WireGuard tunnels still run and handshake, so the mesh can be
|
||||
/// verified with no privileges; traffic just does not reach the
|
||||
/// operating system.
|
||||
#[arg(long)]
|
||||
/// Tunnels still run and handshake, so a mesh can be verified with no
|
||||
/// privileges; traffic just does not reach the operating system.
|
||||
#[arg(long, help_heading = "System")]
|
||||
no_tun: bool,
|
||||
|
||||
/// Interface name prefix for the WireGuard data plane.
|
||||
#[arg(long, default_value = "tsun")]
|
||||
wg_prefix: String,
|
||||
|
||||
/// Interface MTU for the WireGuard data plane.
|
||||
/// Protocols to carry packets with, best first.
|
||||
///
|
||||
/// Must be at least 1280, the minimum IPv6 requires.
|
||||
#[arg(long)]
|
||||
wg_mtu: Option<u32>,
|
||||
/// A pair of peers uses one they both have at the same wire version. A
|
||||
/// peer with none in common keeps its control plane and gets no data
|
||||
/// plane. `none` runs the control plane alone.
|
||||
#[arg(
|
||||
long = "protocol",
|
||||
value_name = "LIST",
|
||||
value_delimiter = ',',
|
||||
default_value = "wg-quic",
|
||||
help_heading = "Transport"
|
||||
)]
|
||||
protocols: Vec<String>,
|
||||
|
||||
/// A protocol setting, as `key=value` or `protocol:key=value`.
|
||||
///
|
||||
/// Repeat for several. `tsunagi protocols` lists what each one takes.
|
||||
#[arg(
|
||||
short = 'o',
|
||||
long = "protocol-option",
|
||||
value_name = "KEY=VALUE",
|
||||
help_heading = "Transport"
|
||||
)]
|
||||
protocol_options: Vec<String>,
|
||||
|
||||
/// IPv4 overlay range, as `address/prefix`, or `none` to disable IPv4.
|
||||
///
|
||||
@@ -249,22 +276,22 @@ struct UpArgs {
|
||||
/// agent adopts what it finds. Addresses are allocated from it and
|
||||
/// recorded in signed state, so each member keeps its own across
|
||||
/// restarts and long absences.
|
||||
#[arg(long, value_name = "CIDR")]
|
||||
#[arg(long, value_name = "CIDR", help_heading = "System")]
|
||||
ipv4_range: Option<String>,
|
||||
|
||||
/// Serve a local DNS zone for this network's members.
|
||||
///
|
||||
/// Members resolve as `<hostname>.<zone>`, from signed state, so a
|
||||
/// member that is switched off still resolves. IPv4 only.
|
||||
#[arg(long)]
|
||||
#[arg(long, help_heading = "System")]
|
||||
dns: bool,
|
||||
|
||||
/// The zone to answer for. Defaults to the network name.
|
||||
#[arg(long, value_name = "NAME")]
|
||||
#[arg(long, value_name = "NAME", help_heading = "System")]
|
||||
dns_zone: Option<String>,
|
||||
|
||||
/// Port for the local DNS server.
|
||||
#[arg(long, default_value_t = 5354)]
|
||||
#[arg(long, default_value_t = 5354, help_heading = "System")]
|
||||
dns_port: u16,
|
||||
|
||||
/// How often to print a status summary, in seconds. Zero disables it.
|
||||
@@ -353,6 +380,7 @@ async fn run(command: Command) -> Result<(), Box<dyn std::error::Error>> {
|
||||
Command::Id(args) => id(args).await,
|
||||
Command::Up(args) => up(*args).await,
|
||||
Command::Status(args) => status(args).await,
|
||||
Command::Protocols => show_protocols(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -526,11 +554,11 @@ impl DnsService {
|
||||
|
||||
/// Picks the publisher for this platform.
|
||||
fn dns_publisher() -> Arc<dyn tsunagi::dns::DnsPublisher> {
|
||||
#[cfg(all(feature = "dns-publish", target_os = "linux"))]
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
Arc::new(tsunagi::dns::publish::ResolvedPublisher::new())
|
||||
}
|
||||
#[cfg(not(all(feature = "dns-publish", target_os = "linux")))]
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
{
|
||||
Arc::new(tsunagi::dns::publish::UnsupportedPublisher::new())
|
||||
}
|
||||
@@ -539,7 +567,6 @@ fn dns_publisher() -> Arc<dyn tsunagi::dns::DnsPublisher> {
|
||||
/// Starts the DNS service for one network and keeps it in step with state.
|
||||
fn spawn_dns(
|
||||
agent: Agent,
|
||||
wireguard: Option<Arc<WireguardPlugin>>,
|
||||
network: NetworkId,
|
||||
zone: tsunagi::dns::ZoneName,
|
||||
port: u16,
|
||||
@@ -599,10 +626,11 @@ fn spawn_dns(
|
||||
.iter()
|
||||
.find(|member| member.endpoint_id == own)
|
||||
.and_then(|member| member.overlay_address_v4);
|
||||
let interface = wireguard
|
||||
.as_ref()
|
||||
.and_then(|plugin| plugin.overview(network))
|
||||
.map(|view| view.interface)
|
||||
// The interface belongs to the agent, so the resolver
|
||||
// setting attaches to that one and not to a protocol's.
|
||||
let interface = agent
|
||||
.overlay()
|
||||
.map(|overlay| overlay.interface)
|
||||
.filter(|name| !name.is_empty());
|
||||
let wanted = listen_addresses(overlay, port);
|
||||
if attempted != wanted {
|
||||
@@ -727,6 +755,110 @@ fn update(state: &Arc<std::sync::Mutex<DnsState>>, edit: impl FnOnce(&mut DnsSta
|
||||
}
|
||||
}
|
||||
|
||||
/// What a protocol is called, what it speaks, and what it takes.
|
||||
///
|
||||
/// A registry rather than a lookup on the plugins themselves, because
|
||||
/// `tsunagi protocols` has to answer before anything is constructed, and
|
||||
/// because this is the list `--protocol` resolves against.
|
||||
struct ProtocolSpec {
|
||||
/// The name on the wire, which is what peers compare.
|
||||
name: &'static str,
|
||||
/// The wire version. Not the software version: two peers on different
|
||||
/// builds carry traffic for each other as long as this matches.
|
||||
version: u16,
|
||||
/// One line about what it is.
|
||||
summary: &'static str,
|
||||
/// The settings it accepts.
|
||||
options: &'static [tsunagi::dataplane::ProtocolOption],
|
||||
}
|
||||
|
||||
/// Every protocol this build has.
|
||||
const PROTOCOLS: &[ProtocolSpec] = &[ProtocolSpec {
|
||||
name: tsunagi_wg_quic::WIREGUARD_PROTOCOL,
|
||||
version: tsunagi_wg_quic::ANNOUNCEMENT_VERSION,
|
||||
summary: "WireGuard's cryptography carried in iroh's QUIC datagrams, so it \
|
||||
crosses NAT and survives where plain WireGuard is blocked",
|
||||
options: WireguardPlugin::OPTIONS,
|
||||
}];
|
||||
|
||||
/// One `-o` setting, and the protocol it was aimed at.
|
||||
struct Setting {
|
||||
/// `Some` when written as `protocol:key=value`.
|
||||
protocol: Option<String>,
|
||||
key: String,
|
||||
value: String,
|
||||
}
|
||||
|
||||
/// Parses `-o` settings, which are `key=value` or `protocol:key=value`.
|
||||
fn parse_settings(raw: &[String]) -> Result<Vec<Setting>, Box<dyn std::error::Error>> {
|
||||
raw.iter()
|
||||
.map(|entry| {
|
||||
let (left, value) = entry
|
||||
.split_once('=')
|
||||
.ok_or_else(|| format!("`{entry}` is not a setting; write it as key=value"))?;
|
||||
let (protocol, key) = match left.split_once(':') {
|
||||
Some((protocol, key)) => (Some(protocol.to_string()), key),
|
||||
None => (None, left),
|
||||
};
|
||||
if key.is_empty() {
|
||||
return Err(format!("`{entry}` has no key").into());
|
||||
}
|
||||
Ok(Setting {
|
||||
protocol,
|
||||
key: key.to_string(),
|
||||
value: value.to_string(),
|
||||
})
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The settings meant for one protocol, refusing any that fit nowhere.
|
||||
fn settings_for(spec: &ProtocolSpec, settings: &[Setting]) -> Vec<(String, String)> {
|
||||
let mut taken = Vec::new();
|
||||
for setting in settings {
|
||||
let aimed_here = match &setting.protocol {
|
||||
Some(name) => name == spec.name,
|
||||
// Unqualified settings go to whichever protocol declares the
|
||||
// key. With one selected that is the obvious reading; with
|
||||
// several, write `protocol:key=value`.
|
||||
None => spec.options.iter().any(|option| option.key == setting.key),
|
||||
};
|
||||
if aimed_here {
|
||||
taken.push((setting.key.clone(), setting.value.clone()));
|
||||
}
|
||||
}
|
||||
taken
|
||||
}
|
||||
|
||||
/// Shows the protocols this build has, and what each one takes.
|
||||
fn show_protocols() -> Result<(), Box<dyn std::error::Error>> {
|
||||
use report::{Health, Report, Row, Section};
|
||||
|
||||
let mut out = Report::new();
|
||||
for spec in PROTOCOLS {
|
||||
let mut section = Section::new(format!("{} (wire version {})", spec.name, spec.version));
|
||||
section.push(Row::new(Health::Info, "what", spec.summary));
|
||||
if spec.options.is_empty() {
|
||||
section.push(Row::new(Health::Info, "settings", "none"));
|
||||
}
|
||||
for option in spec.options {
|
||||
section.push(
|
||||
Row::new(
|
||||
Health::Info,
|
||||
format!("-o {}={}", option.key, option.value),
|
||||
option.help,
|
||||
)
|
||||
.with_note(match option.default {
|
||||
Some(default) => format!("default {default}"),
|
||||
None => "no default".to_string(),
|
||||
}),
|
||||
);
|
||||
}
|
||||
out.push(section);
|
||||
}
|
||||
print_report("tsunagi protocols", &out)
|
||||
}
|
||||
|
||||
/// Serves the local control socket from the running agent.
|
||||
///
|
||||
/// A struct rather than a closure because this end both answers questions and
|
||||
@@ -1152,7 +1284,7 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve
|
||||
let row = entry(&mut rows, &peer.endpoint_id);
|
||||
row.tunnel = Some(peer);
|
||||
if row.overlay_address_v4.is_none() {
|
||||
row.overlay_address_v4 = peer.address_v4.as_deref();
|
||||
row.overlay_address_v4 = peer.address.as_deref();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1197,13 +1329,11 @@ fn network_section(network: &tsunagi::ipc::NetworkReport, own_id: &str) -> repor
|
||||
Health::Info,
|
||||
"overlay",
|
||||
format!(
|
||||
"{} {}/{}{} mtu {}",
|
||||
"{} {} mtu {}",
|
||||
overlay.interface,
|
||||
overlay.address,
|
||||
overlay.prefix_len,
|
||||
match &overlay.address_v4 {
|
||||
Some(v4) => format!(" and {v4}"),
|
||||
None => String::new(),
|
||||
match &overlay.address {
|
||||
Some(address) => format!("{address}/{}", overlay.prefix_len),
|
||||
None => "no address agreed yet".to_string(),
|
||||
},
|
||||
overlay.mtu
|
||||
),
|
||||
@@ -1355,7 +1485,6 @@ fn host_section() -> report::Section {
|
||||
"implementation",
|
||||
"userspace WireGuard (boringtun); no kernel module needed",
|
||||
));
|
||||
#[cfg(feature = "tun-device")]
|
||||
{
|
||||
if cfg!(target_os = "linux") {
|
||||
let tun_path = std::path::Path::new("/dev/net/tun");
|
||||
@@ -1379,7 +1508,7 @@ fn host_section() -> report::Section {
|
||||
});
|
||||
}
|
||||
|
||||
use tsunagi::dataplane::wireguard::{Privilege, probe_net_admin};
|
||||
use tsunagi::overlay::{Privilege, probe_net_admin};
|
||||
match probe_net_admin() {
|
||||
Privilege::Available => {
|
||||
host.push(Row::new(Health::Good, "privileges", "CAP_NET_ADMIN held"));
|
||||
@@ -1420,15 +1549,6 @@ fn host_section() -> report::Section {
|
||||
}
|
||||
}
|
||||
}
|
||||
#[cfg(not(feature = "tun-device"))]
|
||||
host.push(
|
||||
Row::new(
|
||||
Health::Degraded,
|
||||
"interface",
|
||||
"not built in; the tunnels run but cannot reach the OS",
|
||||
)
|
||||
.with_note("rebuild with the `tun-device` feature, or run with `--no-tun`"),
|
||||
);
|
||||
host
|
||||
}
|
||||
|
||||
@@ -1884,7 +2004,7 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
|
||||
let mut config = AgentConfig::new(paths.clone())
|
||||
.with_overlay_ipv4_range(ipv4_range)
|
||||
.with_transport(args.transport.into())
|
||||
.with_transport(args.reach.into())
|
||||
.with_discovery(discovery)
|
||||
.with_discovery_interval(Duration::from_secs(5));
|
||||
if let Some(hostname) = &args.hostname {
|
||||
@@ -1894,24 +2014,75 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
config = config.with_bind_addrs(args.binds.clone());
|
||||
}
|
||||
|
||||
// The data plane is optional and never required for the control plane.
|
||||
let wireguard = if args.wireguard {
|
||||
// What the user asked for is checked first, before anything that could
|
||||
// fail for a reason of its own: a misspelled protocol or setting is
|
||||
// their mistake to see, not something to bury under a privilege error.
|
||||
let wanted: Vec<&ProtocolSpec> = {
|
||||
let mut wanted = Vec::new();
|
||||
for name in args
|
||||
.protocols
|
||||
.iter()
|
||||
.filter(|name| !name.eq_ignore_ascii_case("none"))
|
||||
{
|
||||
let Some(spec) = PROTOCOLS.iter().find(|spec| spec.name == name.as_str()) else {
|
||||
let known: Vec<&str> = PROTOCOLS.iter().map(|spec| spec.name).collect();
|
||||
return Err(format!(
|
||||
"this build has no protocol called `{name}`; it has {}. \
|
||||
Run `tsunagi protocols` to see what each one takes.",
|
||||
known.join(", ")
|
||||
)
|
||||
.into());
|
||||
};
|
||||
wanted.push(spec);
|
||||
}
|
||||
wanted
|
||||
};
|
||||
|
||||
let settings = parse_settings(&args.protocol_options)?;
|
||||
// A setting nobody takes is a mistake, not a preference: one that was
|
||||
// silently dropped looks exactly like one that did not work.
|
||||
for setting in &settings {
|
||||
if !wanted
|
||||
.iter()
|
||||
.any(|spec| !settings_for(spec, std::slice::from_ref(setting)).is_empty())
|
||||
{
|
||||
return Err(format!(
|
||||
"no selected protocol takes `{}`; run `tsunagi protocols` to see what they do",
|
||||
setting.key
|
||||
)
|
||||
.into());
|
||||
}
|
||||
}
|
||||
|
||||
// The interface belongs to the agent, so it is configured once whatever
|
||||
// was selected to carry traffic over it.
|
||||
let mut wireguard = None;
|
||||
if !wanted.is_empty() {
|
||||
let tun_factory: Arc<dyn TunFactory> = if args.no_tun {
|
||||
Arc::new(MemoryTunFactory::new())
|
||||
} else {
|
||||
system_tun_factory()?
|
||||
};
|
||||
let mut wg = WireguardConfig::new(paths.state_dir.join("wireguard"))
|
||||
.with_interface_prefix(args.wg_prefix.clone());
|
||||
if let Some(mtu) = args.wg_mtu {
|
||||
wg = wg.with_mtu(mtu);
|
||||
let mtu = args.mtu.unwrap_or(tsunagi_wg_quic::DEFAULT_MTU);
|
||||
config = config.with_interface(tun_factory, args.interface.clone(), mtu);
|
||||
}
|
||||
|
||||
for spec in &wanted {
|
||||
let options = settings_for(spec, &settings);
|
||||
match spec.name {
|
||||
tsunagi_wg_quic::WIREGUARD_PROTOCOL => {
|
||||
let mut wg = WireguardConfig::new(paths.state_dir.join("wg-quic"));
|
||||
if let Some(mtu) = args.mtu {
|
||||
wg = wg.with_mtu(mtu);
|
||||
}
|
||||
let wg = WireguardPlugin::configure(wg, &options)?;
|
||||
let plugin = WireguardPlugin::open(wg).await?;
|
||||
config = config.with_plugin(plugin.clone() as Arc<dyn IpPlugin>);
|
||||
wireguard = Some(plugin);
|
||||
}
|
||||
other => return Err(format!("`{other}` is listed but not built in").into()),
|
||||
}
|
||||
let plugin = WireguardPlugin::open(wg, tun_factory).await?;
|
||||
config = config.with_plugin(plugin.clone() as Arc<dyn IpPlugin>);
|
||||
Some(plugin)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
}
|
||||
|
||||
let agent = Agent::spawn(config).await?;
|
||||
// From here on every exit goes through `agent.shutdown()`, so the endpoint
|
||||
@@ -1948,13 +2119,7 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
tracing::warn!("{warning}");
|
||||
}
|
||||
println!(" dns zone {}", zone.as_str());
|
||||
Some(spawn_dns(
|
||||
agent.clone(),
|
||||
wireguard.clone(),
|
||||
network,
|
||||
zone,
|
||||
args.dns_port,
|
||||
))
|
||||
Some(spawn_dns(agent.clone(), network, zone, args.dns_port))
|
||||
}
|
||||
Err(err) => {
|
||||
agent.shutdown().await;
|
||||
@@ -2042,6 +2207,7 @@ async fn build_report(
|
||||
StatusReport,
|
||||
};
|
||||
|
||||
let overlay = agent.overlay();
|
||||
let dns = dns.map(|dns| DnsReport {
|
||||
zone: dns.zone,
|
||||
listening: dns.listening.map(|address| address.to_string()),
|
||||
@@ -2063,20 +2229,19 @@ async fn build_report(
|
||||
let overlay = wireguard
|
||||
.and_then(|plugin| plugin.overview(network.network_id))
|
||||
.map(|view| OverlayReport {
|
||||
interface: view.interface.clone(),
|
||||
mtu: view.mtu,
|
||||
address: view.overlay_address.to_string(),
|
||||
address_v4: view.overlay_address_v4.map(|addr| addr.to_string()),
|
||||
prefix: view.overlay_prefix.to_string(),
|
||||
prefix_len: view.overlay_prefix_len,
|
||||
interface: overlay
|
||||
.as_ref()
|
||||
.map_or_else(String::new, |overlay| overlay.interface.clone()),
|
||||
mtu: overlay.as_ref().map_or(0, |overlay| overlay.mtu),
|
||||
address: view.overlay_address_v4.map(|addr| addr.to_string()),
|
||||
prefix_len: view.ipv4_range.map_or(0, |range| range.prefix_len),
|
||||
peers: view
|
||||
.peers
|
||||
.iter()
|
||||
.map(|peer| OverlayPeerReport {
|
||||
endpoint_id: peer.endpoint_id.to_string(),
|
||||
public_key: peer.public_key.to_string(),
|
||||
address: peer.overlay_address.to_string(),
|
||||
address_v4: peer.overlay_address_v4.map(|addr| addr.to_string()),
|
||||
address: peer.overlay_address_v4.map(|addr| addr.to_string()),
|
||||
handshake_secs_ago: peer
|
||||
.tunnel
|
||||
.as_ref()
|
||||
@@ -2104,9 +2269,21 @@ async fn build_report(
|
||||
.unwrap_or_else(|| "no data link".into()),
|
||||
})
|
||||
.collect(),
|
||||
unroutable_packets: view.unroutable_packets,
|
||||
multicast_packets: view.multicast_packets,
|
||||
unroutable_sample: view.unroutable_sample.map(|address| address.to_string()),
|
||||
// The interface belongs to the agent, so the counters
|
||||
// about it come from there and are the same for every
|
||||
// network sharing it.
|
||||
unroutable_packets: overlay
|
||||
.as_ref()
|
||||
.map_or(0, |overlay| overlay.counters.unroutable),
|
||||
multicast_packets: overlay
|
||||
.as_ref()
|
||||
.map_or(0, |overlay| overlay.counters.multicast),
|
||||
unroutable_sample: overlay.as_ref().and_then(|overlay| {
|
||||
overlay
|
||||
.counters
|
||||
.unroutable_sample
|
||||
.map(|address| address.to_string())
|
||||
}),
|
||||
});
|
||||
|
||||
NetworkReport {
|
||||
@@ -2199,9 +2376,9 @@ async fn stop_signal() -> &'static str {
|
||||
/// also the one that cleans up after itself, because the interface is tied to
|
||||
/// an open file descriptor and goes away with the agent, however the agent
|
||||
/// goes away.
|
||||
#[cfg(all(feature = "tun-device", target_os = "linux"))]
|
||||
#[cfg(target_os = "linux")]
|
||||
fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
|
||||
use tsunagi::dataplane::wireguard::{ManagedTunFactory, NetlinkProvisioner};
|
||||
use tsunagi::overlay::{ManagedTunFactory, NetlinkProvisioner};
|
||||
let provisioner = NetlinkProvisioner::new()?;
|
||||
Ok(Arc::new(ManagedTunFactory::new(Arc::new(provisioner))))
|
||||
}
|
||||
@@ -2210,7 +2387,7 @@ fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error
|
||||
///
|
||||
/// Refused here rather than at the first packet, and with the one thing that
|
||||
/// does work on every platform named.
|
||||
#[cfg(all(feature = "tun-device", not(target_os = "linux")))]
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
|
||||
Err(format!(
|
||||
"managing the overlay interface is not implemented on {} yet. \
|
||||
@@ -2220,11 +2397,6 @@ fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error
|
||||
.into())
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "tun-device"))]
|
||||
fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
|
||||
Err("this build has no interface support; rebuild with the `tun-device` feature or pass --no-tun".into())
|
||||
}
|
||||
|
||||
fn print_event(event: &Event) {
|
||||
match event {
|
||||
Event::PeerConnected {
|
||||
@@ -2296,10 +2468,14 @@ async fn print_status(agent: &Agent, network: NetworkId, wireguard: Option<&Wire
|
||||
&& let Some(view) = plugin.overview(network)
|
||||
{
|
||||
println!(
|
||||
"wireguard: {} on {}/{} mtu {}, {}/{} tunnel(s) established",
|
||||
view.interface,
|
||||
view.overlay_address,
|
||||
view.overlay_prefix_len,
|
||||
"{}: {} on {}/{} mtu {}, {}/{} tunnel(s) established",
|
||||
plugin.protocol_id(),
|
||||
agent
|
||||
.overlay()
|
||||
.map_or_else(|| "no interface".to_string(), |overlay| overlay.interface),
|
||||
view.overlay_address_v4
|
||||
.map_or_else(|| "no address yet".to_string(), |addr| addr.to_string()),
|
||||
view.ipv4_range.map_or(0, |range| range.prefix_len),
|
||||
view.mtu,
|
||||
view.established_peers(),
|
||||
view.peers.len()
|
||||
@@ -2309,7 +2485,8 @@ async fn print_status(agent: &Agent, network: NetworkId, wireguard: Option<&Wire
|
||||
Some(tunnel) => println!(
|
||||
" {} {} {} tx={} rx={} dropped={} path={}",
|
||||
peer.public_key.fmt_short(),
|
||||
peer.overlay_address,
|
||||
peer.overlay_address_v4
|
||||
.map_or_else(|| "no address".to_string(), |addr| addr.to_string()),
|
||||
match tunnel.health.since_handshake {
|
||||
Some(since) => format!("handshake {}s ago", since.as_secs()),
|
||||
None => "NOT HANDSHAKEN".to_string(),
|
||||
@@ -2322,16 +2499,23 @@ async fn print_status(agent: &Agent, network: NetworkId, wireguard: Option<&Wire
|
||||
None => println!(
|
||||
" {} {} waiting for a data link",
|
||||
peer.public_key.fmt_short(),
|
||||
peer.overlay_address
|
||||
peer.overlay_address_v4
|
||||
.map_or_else(|| "no address".to_string(), |addr| addr.to_string())
|
||||
),
|
||||
}
|
||||
}
|
||||
if view.unroutable_packets > 0 {
|
||||
println!(
|
||||
" {} packet(s) for unknown addresses",
|
||||
view.unroutable_packets
|
||||
);
|
||||
}
|
||||
}
|
||||
if let Some(overlay) = agent.overlay()
|
||||
&& overlay.counters.unroutable > 0
|
||||
{
|
||||
println!(
|
||||
" {} packet(s) for unknown addresses{}",
|
||||
overlay.counters.unroutable,
|
||||
match overlay.counters.unroutable_sample {
|
||||
Some(sample) => format!(" (for example {sample})"),
|
||||
None => String::new(),
|
||||
}
|
||||
);
|
||||
}
|
||||
println!();
|
||||
}
|
||||
@@ -2352,10 +2536,8 @@ mod status_tests {
|
||||
OverlayReport {
|
||||
interface: "tsundemo".into(),
|
||||
mtu: 1280,
|
||||
address: "fd55::1".into(),
|
||||
address_v4: Some("10.13.37.69".into()),
|
||||
prefix: "fd55::".into(),
|
||||
prefix_len: 64,
|
||||
address: Some("10.13.37.69".into()),
|
||||
prefix_len: 24,
|
||||
peers,
|
||||
..Default::default()
|
||||
}
|
||||
@@ -2365,8 +2547,7 @@ mod status_tests {
|
||||
OverlayPeerReport {
|
||||
endpoint_id: endpoint_id.into(),
|
||||
public_key: "keykeykey".into(),
|
||||
address: "fd55::2".into(),
|
||||
address_v4: Some("10.13.37.237".into()),
|
||||
address: Some("10.13.37.237".into()),
|
||||
handshake_secs_ago: handshake,
|
||||
tx_packets: 32,
|
||||
rx_packets: 887,
|
||||
@@ -92,7 +92,7 @@ fn start(zone: &str, port: u16) -> Running {
|
||||
.arg("--cache-dir")
|
||||
.arg(dir.path().join("cache"))
|
||||
// No real interface and no internet: this is about the wiring.
|
||||
.args(["--transport", "local", "--no-tun", "--wireguard", "--dns"])
|
||||
.args(["--reach", "local", "--no-tun", "--dns"])
|
||||
.args(["--dns-zone", zone])
|
||||
.args(["--dns-port", &port.to_string()])
|
||||
.args(["--log", "error", "--status-interval", "0"])
|
||||
@@ -0,0 +1,56 @@
|
||||
//! What the command line says about this device, against a running agent.
|
||||
//!
|
||||
//! These drive the real binary, which is why they live beside it rather
|
||||
//! than with the library's own tests: the library cannot depend on a binary
|
||||
//! built from a crate that depends on it.
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use tempfile::TempDir;
|
||||
use tsunagi::Agent;
|
||||
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
|
||||
|
||||
/// Asking who this device is must not need the directory lock.
|
||||
///
|
||||
/// The lock belongs to the one agent allowed to *write* the state. `id` only
|
||||
/// reads, so making it take the lock would mean the question could never be
|
||||
/// answered while an agent was running — which is exactly when you want to
|
||||
/// ask it.
|
||||
#[tokio::test]
|
||||
async fn identity_can_be_read_while_an_agent_holds_the_directory() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
|
||||
// Hold the directory the way a running agent does.
|
||||
let agent = Agent::spawn(
|
||||
AgentConfig::new(StoragePaths::under(dir.path()))
|
||||
.with_transport(TransportPolicy::LocalOnly)
|
||||
.with_loopback_bind(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let expected = agent.endpoint_id().to_string();
|
||||
|
||||
let output = tokio::process::Command::new(env!("CARGO_BIN_EXE_tsunagi"))
|
||||
.arg("id")
|
||||
// The same layout `StoragePaths::under` gives the agent above.
|
||||
.arg("--state-dir")
|
||||
.arg(dir.path().join("state"))
|
||||
.arg("--cache-dir")
|
||||
.arg(dir.path().join("cache"))
|
||||
.output()
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
||||
assert!(
|
||||
output.status.success(),
|
||||
"`id` failed while an agent was running:\n{stderr}"
|
||||
);
|
||||
assert!(
|
||||
stdout.contains(&expected),
|
||||
"expected {expected} in:\n{stdout}"
|
||||
);
|
||||
|
||||
agent.shutdown().await;
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
[package]
|
||||
name = "tsunagi-wg-quic"
|
||||
# Its own version, and deliberately separate from the wire version it
|
||||
# negotiates: a release here does not stop it talking to a peer on an older
|
||||
# build, because what peers compare is `ANNOUNCEMENT_VERSION`.
|
||||
version = "0.1.0"
|
||||
description = "The wg-quic protocol for tsunagi: WireGuard's cryptography carried in iroh's QUIC datagrams."
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
repository.workspace = true
|
||||
|
||||
[dependencies]
|
||||
tsunagi = { path = "../tsunagi", version = "0.1.0" }
|
||||
iroh.workspace = true
|
||||
tokio.workspace = true
|
||||
serde.workspace = true
|
||||
postcard.workspace = true
|
||||
bytes.workspace = true
|
||||
thiserror.workspace = true
|
||||
tracing.workspace = true
|
||||
data-encoding.workspace = true
|
||||
hex.workspace = true
|
||||
rusqlite = { version = "0.40", features = ["bundled"] }
|
||||
sha2 = "0.11"
|
||||
hkdf = "0.13"
|
||||
subtle = "2.6"
|
||||
zeroize = { version = "1.9", features = ["derive"] }
|
||||
rand = "0.10"
|
||||
boringtun = { version = "0.7.1", default-features = false }
|
||||
|
||||
[dev-dependencies]
|
||||
tsunagi = { path = "../tsunagi", features = ["testing"] }
|
||||
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros"] }
|
||||
tempfile.workspace = true
|
||||
tracing-subscriber.workspace = true
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
+43
-42
@@ -1,45 +1,49 @@
|
||||
//! What a WireGuard peer tells the network about itself.
|
||||
//!
|
||||
//! This is the opaque payload the control plane carries in a
|
||||
//! [`crate::dataplane::PluginCapability`]. The agent core never parses it —
|
||||
//! [`tsunagi::dataplane::PluginCapability`]. The agent core never parses it —
|
||||
//! only this module does, and only after bounding every field.
|
||||
//!
|
||||
//! The announcement is deliberately tiny: a participant says **who it is**,
|
||||
//! not **where it is**. Reachability is the data plane transport's job, and
|
||||
//! the transport already solves it — see
|
||||
//! [`crate::dataplane::transport`]. A plugin that also tried to advertise
|
||||
//! [`tsunagi::dataplane::transport`]. A plugin that also tried to advertise
|
||||
//! addresses would be reimplementing NAT traversal badly.
|
||||
|
||||
use std::net::Ipv6Addr;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::identity::NetworkId;
|
||||
use tsunagi::dataplane::PluginError;
|
||||
use tsunagi::identity::NetworkId;
|
||||
|
||||
use super::keys::WgPublicKey;
|
||||
use super::overlay::overlay_address;
|
||||
use crate::keys::WgPublicKey;
|
||||
|
||||
/// Version of the announcement format.
|
||||
///
|
||||
/// Version 3 dropped the IPv4 range again: overlay addressing moved to the
|
||||
/// signed records in [`crate::state`], which carry the range and survive a
|
||||
/// signed records in [`tsunagi::state`], which carry the range and survive a
|
||||
/// participant being away. postcard is not self-describing, so an older peer
|
||||
/// cannot read a newer announcement; the mismatch is reported, not misparsed.
|
||||
pub const ANNOUNCEMENT_VERSION: u16 = 3;
|
||||
pub const ANNOUNCEMENT_VERSION: u16 = 4;
|
||||
|
||||
/// What one participant advertises for the WireGuard data plane.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct WgAnnouncement {
|
||||
/// Announcement format version.
|
||||
pub version: u16,
|
||||
/// The peer's WireGuard public key. Its overlay address is derived from it.
|
||||
pub public_key: [u8; 32],
|
||||
/// The overlay address the peer believes it has.
|
||||
/// The peer's WireGuard public key.
|
||||
///
|
||||
/// Carried for diagnostics and cross-checking only. Addresses are always
|
||||
/// derived locally, never taken from this field.
|
||||
pub overlay_address: Ipv6Addr,
|
||||
/// The whole announcement, now that addresses belong to the system
|
||||
/// level: this says *who* is at the other end of a tunnel, and nothing
|
||||
/// about where.
|
||||
pub public_key: [u8; 32],
|
||||
/// The network this key is for.
|
||||
///
|
||||
/// Strictly redundant — a capability arrives on a session that already
|
||||
/// proved membership of one network — and kept anyway, because the
|
||||
/// binding used to be a side effect of checking a derived address and
|
||||
/// losing it silently when that check went would be the wrong way to
|
||||
/// lose it.
|
||||
pub network: [u8; 32],
|
||||
}
|
||||
|
||||
/// A peer announcement that has been validated against a specific network.
|
||||
@@ -47,8 +51,6 @@ pub struct WgAnnouncement {
|
||||
pub struct ValidatedAnnouncement {
|
||||
/// The peer's WireGuard public key.
|
||||
pub public_key: WgPublicKey,
|
||||
/// The overlay address derived locally for this key. Authoritative.
|
||||
pub overlay_address: Ipv6Addr,
|
||||
}
|
||||
|
||||
impl WgAnnouncement {
|
||||
@@ -57,7 +59,7 @@ impl WgAnnouncement {
|
||||
Self {
|
||||
version: ANNOUNCEMENT_VERSION,
|
||||
public_key: *public_key.as_bytes(),
|
||||
overlay_address: overlay_address(network, public_key),
|
||||
network: *network.as_bytes(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -105,20 +107,13 @@ impl WgAnnouncement {
|
||||
"peer announced this agent's own WireGuard key".into(),
|
||||
));
|
||||
}
|
||||
// AllowedIPs are derived, never trusted. A mismatch means the peer is
|
||||
// confused or lying, and either way its own claim is discarded.
|
||||
let derived = overlay_address(network, &public_key);
|
||||
if self.overlay_address != derived {
|
||||
if self.network != *network.as_bytes() {
|
||||
return Err(PluginError::Rejected(
|
||||
"announced overlay address does not match the one derived from the peer's key"
|
||||
.into(),
|
||||
"announcement is for a different network".into(),
|
||||
));
|
||||
}
|
||||
|
||||
Ok(ValidatedAnnouncement {
|
||||
public_key,
|
||||
overlay_address: derived,
|
||||
})
|
||||
Ok(ValidatedAnnouncement { public_key })
|
||||
}
|
||||
}
|
||||
|
||||
@@ -127,7 +122,7 @@ mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use super::*;
|
||||
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
use tsunagi::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
|
||||
use super::super::keys::WgSecretKey;
|
||||
|
||||
@@ -149,7 +144,6 @@ mod tests {
|
||||
let validated = WgAnnouncement::decode_and_validate(&payload, id, &local).unwrap();
|
||||
|
||||
assert_eq!(validated.public_key, peer);
|
||||
assert_eq!(validated.overlay_address, overlay_address(id, &peer));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -167,20 +161,27 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn allowed_ips_are_derived_not_taken_from_the_peer() {
|
||||
fn there_is_nothing_address_like_to_forge() {
|
||||
// Addresses belong to the system level, are allocated there and are
|
||||
// signed by the member that holds one. A protocol announcement
|
||||
// carries no address at all, so this is not a thing a peer can lie
|
||||
// about here — and a peer sending traffic from an address it does
|
||||
// not hold is rejected by the agreed address, not by anything it
|
||||
// said in this message.
|
||||
let id = network("no-hijack");
|
||||
let victim = WgSecretKey::generate().public();
|
||||
let attacker = WgSecretKey::generate().public();
|
||||
let peer = WgSecretKey::generate().public();
|
||||
let local = WgSecretKey::generate().public();
|
||||
|
||||
// An attacker claims the victim's overlay address with its own key.
|
||||
let mut forged = WgAnnouncement::new(id, &attacker);
|
||||
forged.overlay_address = overlay_address(id, &victim);
|
||||
let announcement = WgAnnouncement::new(id, &peer);
|
||||
let validated =
|
||||
WgAnnouncement::decode_and_validate(&announcement.encode().unwrap(), id, &local)
|
||||
.unwrap();
|
||||
assert_eq!(validated.public_key, peer);
|
||||
|
||||
let result = WgAnnouncement::decode_and_validate(&forged.encode().unwrap(), id, &local);
|
||||
assert!(
|
||||
matches!(result, Err(PluginError::Rejected(ref reason)) if reason.contains("does not match")),
|
||||
"claiming another member's overlay address must be rejected: {result:?}"
|
||||
// The validated form has one field, and it is an identity.
|
||||
assert_eq!(
|
||||
std::mem::size_of_val(&validated),
|
||||
std::mem::size_of::<WgPublicKey>()
|
||||
);
|
||||
}
|
||||
|
||||
@@ -236,7 +237,7 @@ mod tests {
|
||||
let peer = WgSecretKey::generate().public();
|
||||
let payload = WgAnnouncement::new(id, &peer).encode().unwrap();
|
||||
assert!(
|
||||
payload.len() < crate::config::Limits::default().max_capability_data_len,
|
||||
payload.len() < tsunagi::config::Limits::default().max_capability_data_len,
|
||||
"announcement is {} bytes",
|
||||
payload.len()
|
||||
);
|
||||
@@ -18,16 +18,18 @@
|
||||
//! ourselves, and we do:
|
||||
//!
|
||||
//! * outbound, a packet is routed to the peer that **owns** its destination
|
||||
//! address, where ownership is the derivation in [`super::overlay`];
|
||||
//! address, where ownership is the signed claim the system level agreed;
|
||||
//! * inbound, a decrypted packet is dropped unless its **source** is exactly
|
||||
//! the address derived for the peer whose tunnel decrypted it.
|
||||
//! the address that peer holds.
|
||||
//!
|
||||
//! So a participant cannot receive traffic addressed to someone else, and
|
||||
//! cannot forge traffic that appears to come from someone else, no matter
|
||||
//! what it announced.
|
||||
//! cannot forge traffic that appears to come from someone else. Neither
|
||||
//! check consults anything the peer said here: an address is claimed at the
|
||||
//! system level and signed by its holder, and that is what is compared
|
||||
//! against.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
|
||||
use std::net::Ipv4Addr;
|
||||
use std::sync::atomic::{AtomicU32, AtomicU64, Ordering};
|
||||
use std::sync::{Arc, Mutex, RwLock};
|
||||
use std::time::Duration;
|
||||
@@ -37,15 +39,11 @@ use bytes::Bytes;
|
||||
use iroh::EndpointId;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::dataplane::transport::{SharedLink, TransportError};
|
||||
use crate::identity::NetworkId;
|
||||
use tsunagi::dataplane::transport::{SharedLink, TransportError};
|
||||
use tsunagi::dataplane::{PacketSink, PluginError};
|
||||
use tsunagi::identity::NetworkId;
|
||||
|
||||
use super::keys::{WgPublicKey, WgSecretKey};
|
||||
use super::overlay::overlay_address;
|
||||
use super::packet::IpHeader;
|
||||
use super::tun::TunDevice;
|
||||
use crate::state::Ipv4Range;
|
||||
use crate::keys::{WgPublicKey, WgSecretKey};
|
||||
|
||||
/// How often WireGuard's own timers are driven.
|
||||
///
|
||||
@@ -65,6 +63,8 @@ struct PeerCounters {
|
||||
rx_bytes: AtomicU64,
|
||||
dropped_wrong_source: AtomicU64,
|
||||
dropped_oversize: AtomicU64,
|
||||
/// Packets there was no session to encrypt with yet.
|
||||
dropped_no_session: AtomicU64,
|
||||
protocol_errors: AtomicU64,
|
||||
}
|
||||
|
||||
@@ -85,6 +85,11 @@ pub struct PeerStats {
|
||||
pub dropped_wrong_source: u64,
|
||||
/// Packets dropped because they did not fit in one link datagram.
|
||||
pub dropped_oversize: u64,
|
||||
/// Packets dropped because there was no session to encrypt with yet.
|
||||
///
|
||||
/// A handful while a tunnel comes up is normal; a number that keeps
|
||||
/// climbing means the handshake is not completing.
|
||||
pub dropped_no_session: u64,
|
||||
/// WireGuard protocol errors, including packets that failed to decrypt.
|
||||
pub protocol_errors: u64,
|
||||
}
|
||||
@@ -109,9 +114,11 @@ impl PeerHealth {
|
||||
struct Peer {
|
||||
endpoint_id: EndpointId,
|
||||
public_key: WgPublicKey,
|
||||
overlay: Ipv6Addr,
|
||||
/// The IPv4 address this peer owns, when the overlay is dual stack and
|
||||
/// nobody else derived the same one.
|
||||
/// The overlay address this peer holds, as the control plane agreed it.
|
||||
///
|
||||
/// Not derived here and not taken from the peer: the system level
|
||||
/// allocates it, the peer signs the claim, and every protocol carries
|
||||
/// traffic for the same address.
|
||||
overlay_v4: Mutex<Option<Ipv4Addr>>,
|
||||
tunn: Mutex<Tunn>,
|
||||
link: SharedLink,
|
||||
@@ -124,7 +131,6 @@ impl std::fmt::Debug for Peer {
|
||||
f.debug_struct("Peer")
|
||||
.field("peer", &self.endpoint_id.fmt_short().to_string())
|
||||
.field("public_key", &self.public_key)
|
||||
.field("overlay", &self.overlay)
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
@@ -148,6 +154,7 @@ impl Peer {
|
||||
rx_bytes: self.counters.rx_bytes.load(Ordering::Relaxed),
|
||||
dropped_wrong_source: self.counters.dropped_wrong_source.load(Ordering::Relaxed),
|
||||
dropped_oversize: self.counters.dropped_oversize.load(Ordering::Relaxed),
|
||||
dropped_no_session: self.counters.dropped_no_session.load(Ordering::Relaxed),
|
||||
protocol_errors: self.counters.protocol_errors.load(Ordering::Relaxed),
|
||||
}
|
||||
}
|
||||
@@ -170,15 +177,8 @@ pub struct PeerSummary {
|
||||
pub endpoint_id: EndpointId,
|
||||
/// The peer's WireGuard public key.
|
||||
pub public_key: WgPublicKey,
|
||||
/// The overlay address this agent derived for it.
|
||||
pub overlay_address: Ipv6Addr,
|
||||
/// Its IPv4 overlay address, when the overlay is dual stack.
|
||||
///
|
||||
/// `None` with `ipv4_conflict` set means another member derived the same
|
||||
/// address and won it; that peer is still fully reachable over IPv6.
|
||||
/// The overlay address it holds, once the control plane has agreed one.
|
||||
pub overlay_address_v4: Option<Ipv4Addr>,
|
||||
/// Whether this peer lost an IPv4 address to a derivation collision.
|
||||
pub ipv4_conflict: bool,
|
||||
/// Whether the tunnel has handshaken.
|
||||
pub health: PeerHealth,
|
||||
/// Traffic counters.
|
||||
@@ -192,30 +192,25 @@ pub struct PeerSummary {
|
||||
struct Inner {
|
||||
network: NetworkId,
|
||||
private_key: WgSecretKey,
|
||||
tun: Arc<dyn TunDevice>,
|
||||
/// The IPv4 overlay range, when the overlay is dual stack.
|
||||
ipv4_range: Option<Ipv4Range>,
|
||||
peers: RwLock<HashMap<WgPublicKey, Arc<Peer>>>,
|
||||
/// Both families, so one lookup routes any packet.
|
||||
routes: RwLock<HashMap<IpAddr, WgPublicKey>>,
|
||||
/// Where decrypted packets go.
|
||||
///
|
||||
/// The interface belongs to the system level, so this hands a packet up
|
||||
/// rather than writing it out: only that level knows which addresses the
|
||||
/// sending member is entitled to use.
|
||||
sink: Arc<dyn PacketSink>,
|
||||
next_index: AtomicU32,
|
||||
unroutable: AtomicU64,
|
||||
/// One destination nobody owned, kept so the counter can be acted on.
|
||||
unroutable_sample: Mutex<Option<IpAddr>>,
|
||||
multicast: AtomicU64,
|
||||
ipv4_conflicts: AtomicU64,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for Inner {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("Inner")
|
||||
.field("network", &self.network.fmt_short())
|
||||
.field("tun", &self.tun.name())
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
/// A userspace WireGuard interface for one network.
|
||||
/// The userspace WireGuard tunnels of one network.
|
||||
#[derive(Debug)]
|
||||
pub struct WireguardDevice {
|
||||
inner: Arc<Inner>,
|
||||
@@ -223,44 +218,73 @@ pub struct WireguardDevice {
|
||||
}
|
||||
|
||||
impl WireguardDevice {
|
||||
/// Starts a device on top of `tun`.
|
||||
pub fn start(
|
||||
network: NetworkId,
|
||||
private_key: WgSecretKey,
|
||||
tun: Arc<dyn TunDevice>,
|
||||
ipv4_range: Option<Ipv4Range>,
|
||||
) -> Self {
|
||||
/// Starts the tunnels for one network.
|
||||
///
|
||||
/// No interface is involved: packets arrive through [`Self::carry`] and
|
||||
/// leave through the sink. Which address belongs to whom, and therefore
|
||||
/// where a packet should go, is decided above this.
|
||||
pub fn start(network: NetworkId, private_key: WgSecretKey, sink: Arc<dyn PacketSink>) -> Self {
|
||||
let inner = Arc::new(Inner {
|
||||
network,
|
||||
private_key,
|
||||
tun,
|
||||
ipv4_range,
|
||||
peers: RwLock::new(HashMap::new()),
|
||||
routes: RwLock::new(HashMap::new()),
|
||||
sink,
|
||||
next_index: AtomicU32::new(1),
|
||||
unroutable: AtomicU64::new(0),
|
||||
unroutable_sample: Mutex::new(None),
|
||||
multicast: AtomicU64::new(0),
|
||||
ipv4_conflicts: AtomicU64::new(0),
|
||||
});
|
||||
|
||||
let reader = tokio::spawn(read_from_os(Arc::clone(&inner)));
|
||||
let timers = tokio::spawn(drive_timers(Arc::clone(&inner)));
|
||||
|
||||
Self {
|
||||
inner,
|
||||
tasks: vec![reader, timers],
|
||||
tasks: vec![timers],
|
||||
}
|
||||
}
|
||||
|
||||
/// The interface name in use.
|
||||
pub fn interface(&self) -> &str {
|
||||
self.inner.tun.name()
|
||||
}
|
||||
/// Encrypts a packet and sends it to a peer.
|
||||
///
|
||||
/// `false` when there is no tunnel for that peer, which is a state the
|
||||
/// caller reports rather than an error: a peer whose link has not come
|
||||
/// up yet is normal.
|
||||
pub fn carry(&self, peer: iroh::EndpointId, packet: &[u8]) -> bool {
|
||||
let found = read_lock(&self.inner.peers)
|
||||
.values()
|
||||
.find(|candidate| candidate.endpoint_id == peer)
|
||||
.map(Arc::clone);
|
||||
let Some(peer) = found else {
|
||||
return false;
|
||||
};
|
||||
|
||||
/// The interface MTU.
|
||||
pub fn mtu(&self) -> u32 {
|
||||
self.inner.tun.mtu()
|
||||
let mut scratch = vec![0u8; SCRATCH];
|
||||
// The encryption is the whole of what this protocol contributes, so
|
||||
// it happens here rather than anywhere the packet passes through.
|
||||
// The lock is released before the send: a slow link must not hold up
|
||||
// the tunnel's timers.
|
||||
let len = {
|
||||
let mut tunn = match peer.tunn.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
match tunn.encapsulate(packet, &mut scratch) {
|
||||
TunnResult::WriteToNetwork(out) => Some(out.len()),
|
||||
// No session yet, so nothing to send. Counted as dropped
|
||||
// rather than reported: the handshake is in flight and the
|
||||
// next packet will go.
|
||||
_ => None,
|
||||
}
|
||||
};
|
||||
let Some(len) = len else {
|
||||
peer.counters
|
||||
.dropped_no_session
|
||||
.fetch_add(1, Ordering::Relaxed);
|
||||
return false;
|
||||
};
|
||||
|
||||
send_to_peer(&peer, &scratch[..len]);
|
||||
peer.counters.tx_packets.fetch_add(1, Ordering::Relaxed);
|
||||
peer.counters
|
||||
.tx_bytes
|
||||
.fetch_add(packet.len() as u64, Ordering::Relaxed);
|
||||
true
|
||||
}
|
||||
|
||||
/// Adds or replaces a peer and starts its tunnel.
|
||||
@@ -292,12 +316,9 @@ impl WireguardDevice {
|
||||
None,
|
||||
);
|
||||
|
||||
let overlay = overlay_address(self.inner.network, &public_key);
|
||||
let overlay_v4 = self.claim_ipv4(&public_key, overlay_v4);
|
||||
let peer = Arc::new(Peer {
|
||||
endpoint_id,
|
||||
public_key,
|
||||
overlay,
|
||||
overlay_v4: Mutex::new(overlay_v4),
|
||||
tunn: Mutex::new(tunn),
|
||||
link,
|
||||
@@ -311,10 +332,6 @@ impl WireguardDevice {
|
||||
}
|
||||
|
||||
write_lock(&self.inner.peers).insert(public_key, Arc::clone(&peer));
|
||||
write_lock(&self.inner.routes).insert(IpAddr::V6(overlay), public_key);
|
||||
if let Some(v4) = overlay_v4 {
|
||||
write_lock(&self.inner.routes).insert(IpAddr::V4(v4), public_key);
|
||||
}
|
||||
|
||||
// Start the handshake now instead of waiting for the next timer tick,
|
||||
// so the tunnel is usable as soon as the link exists.
|
||||
@@ -322,61 +339,11 @@ impl WireguardDevice {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Decides which IPv4 address a new peer gets, if any.
|
||||
///
|
||||
/// IPv4 has far too little room for a derived address to be collision
|
||||
/// free. When two members derive the same one, the member whose public
|
||||
/// key sorts lower keeps it — a rule every member computes identically,
|
||||
/// so they all agree on the outcome without talking about it. The other
|
||||
/// member simply has no IPv4 address; it is still fully reachable over
|
||||
/// IPv6, which never collides.
|
||||
fn claim_ipv4(&self, public_key: &WgPublicKey, wanted: Option<Ipv4Addr>) -> Option<Ipv4Addr> {
|
||||
let wanted = wanted?;
|
||||
|
||||
let holder = read_lock(&self.inner.routes)
|
||||
.get(&IpAddr::V4(wanted))
|
||||
.copied();
|
||||
let Some(holder) = holder else {
|
||||
return Some(wanted);
|
||||
};
|
||||
if holder == *public_key {
|
||||
return Some(wanted);
|
||||
}
|
||||
|
||||
self.inner.ipv4_conflicts.fetch_add(1, Ordering::Relaxed);
|
||||
if holder.as_bytes() <= public_key.as_bytes() {
|
||||
// The peer already holding it wins.
|
||||
return None;
|
||||
}
|
||||
// The newcomer wins; take the address away from the other peer.
|
||||
if let Some(loser) = read_lock(&self.inner.peers).get(&holder).cloned() {
|
||||
let mut slot = match loser.overlay_v4.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
*slot = None;
|
||||
}
|
||||
Some(wanted)
|
||||
}
|
||||
|
||||
/// Removes a peer and stops its tunnel.
|
||||
pub fn remove_peer(&self, public_key: &WgPublicKey) {
|
||||
if let Some(peer) = write_lock(&self.inner.peers).remove(public_key) {
|
||||
let mut routes = write_lock(&self.inner.routes);
|
||||
routes.remove(&IpAddr::V6(peer.overlay));
|
||||
let v4 = match peer.overlay_v4.lock() {
|
||||
Ok(guard) => *guard,
|
||||
Err(poisoned) => *poisoned.into_inner(),
|
||||
};
|
||||
if let Some(v4) = v4 {
|
||||
routes.remove(&IpAddr::V4(v4));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// How many IPv4 derivation collisions have been resolved.
|
||||
pub fn ipv4_conflicts(&self) -> u64 {
|
||||
self.inner.ipv4_conflicts.load(Ordering::Relaxed)
|
||||
// Dropping it stops the task and closes the link; there is no route
|
||||
// to withdraw, because routes are not kept here.
|
||||
write_lock(&self.inner.peers).remove(public_key);
|
||||
}
|
||||
|
||||
/// Removes every peer whose key is not in `keep`.
|
||||
@@ -408,9 +375,7 @@ impl WireguardDevice {
|
||||
PeerSummary {
|
||||
endpoint_id: peer.endpoint_id,
|
||||
public_key: peer.public_key,
|
||||
overlay_address: peer.overlay,
|
||||
overlay_address_v4: overlay_v4,
|
||||
ipv4_conflict: overlay_v4.is_none() && self.inner.ipv4_range.is_some(),
|
||||
health: peer.health(),
|
||||
stats: peer.stats(),
|
||||
path: peer.link.path_description(),
|
||||
@@ -421,34 +386,6 @@ impl WireguardDevice {
|
||||
peers.sort_by_key(|peer| peer.public_key);
|
||||
peers
|
||||
}
|
||||
|
||||
/// Unicast packets the operating system sent to an address no peer owns.
|
||||
///
|
||||
/// A non-zero value means something tried to reach a host that is not in
|
||||
/// the overlay.
|
||||
pub fn unroutable_packets(&self) -> u64 {
|
||||
self.inner.unroutable.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
/// One destination that nobody owned, if there was one.
|
||||
///
|
||||
/// A bare count says something is wrong but not what; the address usually
|
||||
/// says it outright.
|
||||
pub fn unroutable_sample(&self) -> Option<IpAddr> {
|
||||
match self.inner.unroutable_sample.lock() {
|
||||
Ok(guard) => *guard,
|
||||
Err(poisoned) => *poisoned.into_inner(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Multicast packets dropped.
|
||||
///
|
||||
/// Expected and harmless: Linux emits multicast listener and router
|
||||
/// solicitation traffic on any IPv6 interface, and this overlay is
|
||||
/// unicast only. Counted separately so it does not look like a fault.
|
||||
pub fn multicast_packets(&self) -> u64 {
|
||||
self.inner.multicast.load(Ordering::Relaxed)
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for WireguardDevice {
|
||||
@@ -511,77 +448,6 @@ fn send_to_peer(peer: &Peer, payload: &[u8]) {
|
||||
}
|
||||
}
|
||||
|
||||
/// Operating system -> peer.
|
||||
async fn read_from_os(inner: Arc<Inner>) {
|
||||
loop {
|
||||
let Some(packet) = inner.tun.recv().await else {
|
||||
return;
|
||||
};
|
||||
|
||||
// Route by destination: only the peer that owns that overlay address
|
||||
// may receive it. Both families go through the same table.
|
||||
let Some(destination) = IpHeader::parse(&packet).map(|header| header.destination()) else {
|
||||
inner.unroutable.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
};
|
||||
// The kernel emits multicast on every IPv6 interface. The overlay is
|
||||
// unicast only, so this is dropped, but it is not a fault.
|
||||
if destination.is_multicast() {
|
||||
inner.multicast.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
}
|
||||
let target = read_lock(&inner.routes).get(&destination).copied();
|
||||
let Some(target) = target else {
|
||||
note_unroutable(&inner, destination);
|
||||
continue;
|
||||
};
|
||||
let peer = read_lock(&inner.peers).get(&target).cloned();
|
||||
let Some(peer) = peer else {
|
||||
note_unroutable(&inner, destination);
|
||||
continue;
|
||||
};
|
||||
|
||||
let mut scratch = vec![0u8; SCRATCH];
|
||||
let outcome = {
|
||||
let mut tunn = match peer.tunn.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
match tunn.encapsulate(&packet, &mut scratch) {
|
||||
TunnResult::WriteToNetwork(out) => Some(out.len()),
|
||||
TunnResult::Done => None,
|
||||
TunnResult::Err(err) => {
|
||||
tracing::trace!(?err, "wireguard encapsulation failed");
|
||||
peer.counters
|
||||
.protocol_errors
|
||||
.fetch_add(1, Ordering::Relaxed);
|
||||
None
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
};
|
||||
|
||||
if let Some(len) = outcome {
|
||||
send_to_peer(&peer, &scratch[..len]);
|
||||
peer.counters.tx_packets.fetch_add(1, Ordering::Relaxed);
|
||||
peer.counters
|
||||
.tx_bytes
|
||||
.fetch_add(packet.len() as u64, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Counts a packet nobody owned the destination of, keeping one example.
|
||||
fn note_unroutable(inner: &Inner, destination: IpAddr) {
|
||||
inner.unroutable.fetch_add(1, Ordering::Relaxed);
|
||||
let mut sample = match inner.unroutable_sample.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
*sample = Some(destination);
|
||||
}
|
||||
|
||||
/// Peer -> operating system.
|
||||
async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
|
||||
loop {
|
||||
let Some(datagram) = peer.link.recv().await else {
|
||||
@@ -600,12 +466,11 @@ async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
|
||||
};
|
||||
match tunn.decapsulate(None, input.unwrap_or(&[]), &mut scratch) {
|
||||
TunnResult::WriteToNetwork(out) => Outcome::ToNetwork(out.len()),
|
||||
TunnResult::WriteToTunnelV6(out, source) => {
|
||||
Outcome::ToTunnel(out.len(), IpAddr::V6(source))
|
||||
}
|
||||
TunnResult::WriteToTunnelV4(out, source) => {
|
||||
Outcome::ToTunnel(out.len(), IpAddr::V4(source))
|
||||
}
|
||||
// The source boringtun reports is not consulted here:
|
||||
// whether the peer may use it is checked where the
|
||||
// claims live.
|
||||
TunnResult::WriteToTunnelV6(out, _) => Outcome::ToTunnel(out.len()),
|
||||
TunnResult::WriteToTunnelV4(out, _) => Outcome::ToTunnel(out.len()),
|
||||
TunnResult::Done => Outcome::Done,
|
||||
TunnResult::Err(err) => {
|
||||
tracing::trace!(?err, "wireguard decapsulation failed");
|
||||
@@ -621,32 +486,20 @@ async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
|
||||
input = None;
|
||||
continue;
|
||||
}
|
||||
Outcome::ToTunnel(len, source) => {
|
||||
Outcome::ToTunnel(len) => {
|
||||
// Handed up, not written out. This end has proved *who*
|
||||
// sent the packet; whether that member may use the source
|
||||
// address it chose is a question about a signed claim,
|
||||
// and only the system level holds those.
|
||||
let payload = Bytes::copy_from_slice(&scratch[..len]);
|
||||
// Enforce address ownership: a peer may only send from an
|
||||
// address derived for its own key, in either family.
|
||||
let owned = match source {
|
||||
IpAddr::V6(addr) => addr == peer.overlay,
|
||||
IpAddr::V4(addr) => {
|
||||
let held = match peer.overlay_v4.lock() {
|
||||
Ok(guard) => *guard,
|
||||
Err(poisoned) => *poisoned.into_inner(),
|
||||
};
|
||||
held == Some(addr)
|
||||
}
|
||||
};
|
||||
if !owned {
|
||||
peer.counters
|
||||
.dropped_wrong_source
|
||||
.fetch_add(1, Ordering::Relaxed);
|
||||
break;
|
||||
}
|
||||
if inner.tun.send(payload).await.is_ok() {
|
||||
peer.counters.rx_packets.fetch_add(1, Ordering::Relaxed);
|
||||
peer.counters
|
||||
.rx_bytes
|
||||
.fetch_add(len as u64, Ordering::Relaxed);
|
||||
}
|
||||
inner
|
||||
.sink
|
||||
.deliver(inner.network, peer.endpoint_id, payload)
|
||||
.await;
|
||||
peer.counters.rx_packets.fetch_add(1, Ordering::Relaxed);
|
||||
peer.counters
|
||||
.rx_bytes
|
||||
.fetch_add(len as u64, Ordering::Relaxed);
|
||||
break;
|
||||
}
|
||||
Outcome::Failed => {
|
||||
@@ -663,7 +516,7 @@ async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
|
||||
|
||||
enum Outcome {
|
||||
ToNetwork(usize),
|
||||
ToTunnel(usize, IpAddr),
|
||||
ToTunnel(usize),
|
||||
Done,
|
||||
Failed,
|
||||
}
|
||||
@@ -11,7 +11,7 @@ use boringtun::x25519;
|
||||
use data_encoding::BASE64;
|
||||
use zeroize::{Zeroize, Zeroizing};
|
||||
|
||||
use crate::dataplane::PluginError;
|
||||
use tsunagi::dataplane::PluginError;
|
||||
|
||||
/// Length of a raw WireGuard key, in bytes.
|
||||
pub const KEY_LEN: usize = 32;
|
||||
@@ -0,0 +1,50 @@
|
||||
//! The `wg-quic` protocol: WireGuard's cryptography in iroh's QUIC datagrams.
|
||||
//!
|
||||
//! A crate of its own, so the line between a protocol and the system level
|
||||
//! is drawn by the compiler rather than by discipline: nothing here can
|
||||
//! reach into `tsunagi` beyond what it makes public. It also carries its own
|
||||
//! version, which is **not** the version peers compare — that is
|
||||
//! [`ANNOUNCEMENT_VERSION`], and it moves only when the bytes on the wire
|
||||
//! do, so two peers on different releases keep working.
|
||||
//!
|
||||
//! That it uses iroh is a convenience, not a requirement of the design: the
|
||||
//! system level already has an iroh endpoint that crosses NAT, and plain
|
||||
//! WireGuard is blocked on some networks where this is not.
|
||||
//!
|
||||
//! Where the line falls:
|
||||
//!
|
||||
//! * **It knows nothing about reachability.** It is handed a
|
||||
//! [`PacketLink`](tsunagi::dataplane::transport::PacketLink) per peer and
|
||||
//! moves datagrams over it. Hole punching and relaying are the transport's.
|
||||
//! * **It knows nothing about addresses, and owns no interface.** One agent
|
||||
//! has one interface at the system level, and every protocol carries
|
||||
//! traffic for the same addresses on it. A packet arrives here already
|
||||
//! routed and leaves here already decrypted, to be checked against the
|
||||
//! signed claim by the level that holds those.
|
||||
//! * **Its announcement says who, not where.** A public key and the network
|
||||
//! it is for, so there is nothing about reachability to lie about.
|
||||
//! * **The core never parses that announcement.** It moves a bounded opaque
|
||||
//! blob; only [`announcement`] reads it.
|
||||
//! * **Its keys are its own.** One per network, in its own store, unrelated
|
||||
//! to the iroh device key and to the network secret — which it never sees.
|
||||
//! * **WireGuard's own crypto is untouched.** The handshake and encryption
|
||||
//! run end to end between the two ends of a tunnel, via [`boringtun`]'s
|
||||
//! state machine in this process: no kernel module, no `wg` tool, the same
|
||||
//! code on every platform.
|
||||
//!
|
||||
//! See `docs/wireguard.md` for the full picture.
|
||||
|
||||
pub mod announcement;
|
||||
pub mod device;
|
||||
pub mod keys;
|
||||
pub mod plugin;
|
||||
pub mod store;
|
||||
|
||||
pub use announcement::{ANNOUNCEMENT_VERSION, ValidatedAnnouncement, WgAnnouncement};
|
||||
pub use device::{PeerHealth, PeerStats, PeerSummary, WireguardDevice};
|
||||
pub use keys::{WgPublicKey, WgSecretKey};
|
||||
pub use plugin::{
|
||||
DEFAULT_MTU, MIN_MTU, NetworkOverview, PeerOverview, WIREGUARD_OVERHEAD, WIREGUARD_PROTOCOL,
|
||||
WireguardConfig, WireguardPlugin,
|
||||
};
|
||||
pub use store::WgKeyStore;
|
||||
@@ -1,4 +1,4 @@
|
||||
//! The WireGuard IP plugin.
|
||||
//! The WireGuard protocol plugin.
|
||||
//!
|
||||
//! Each agent builds its own view of the overlay from the set of participants
|
||||
//! the control plane agreed on. For a full mesh of `N` members that is `N - 1`
|
||||
@@ -8,24 +8,24 @@
|
||||
//! # What this plugin does and does not know
|
||||
//!
|
||||
//! * It does **not** know where a peer is. It is handed a
|
||||
//! [`PacketLink`](crate::dataplane::transport::PacketLink) per peer and runs
|
||||
//! [`PacketLink`](tsunagi::dataplane::transport::PacketLink) per peer and runs
|
||||
//! a WireGuard tunnel over it. Reachability, hole punching and relaying are
|
||||
//! the transport's problem.
|
||||
//! * It does **not** know which addresses anybody holds, and owns no
|
||||
//! interface. One agent has one interface, at the system level, and every
|
||||
//! protocol carries traffic for the same addresses on it. A packet arrives
|
||||
//! here already routed and leaves here already decrypted.
|
||||
//! * It owns one WireGuard key per network, in its own store, unrelated to the
|
||||
//! iroh device key and to the network secret.
|
||||
//! * It owns one packet interface per network, named deterministically.
|
||||
//! * It never touches an interface it did not create, and never changes
|
||||
//! routing, DNS or firewall settings beyond its own device.
|
||||
//!
|
||||
//! WireGuard runs in userspace via [`boringtun`], so there is no kernel module
|
||||
//! and no `wg` tool to depend on. The only privileged step is creating the
|
||||
//! packet interface, and even that is behind [`TunFactory`] so the whole data
|
||||
//! plane can run unprivileged in tests.
|
||||
//! and no `wg` tool to depend on, and nothing this plugin does needs
|
||||
//! privileges: creating the interface is somebody else's job now.
|
||||
//!
|
||||
//! A failure here is reported and retried. It never stops the control plane.
|
||||
|
||||
use std::collections::{BTreeSet, HashMap};
|
||||
use std::net::{IpAddr, Ipv4Addr};
|
||||
use std::net::Ipv4Addr;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
use std::time::Duration;
|
||||
@@ -34,37 +34,40 @@ use iroh::EndpointId;
|
||||
use tokio::sync::{mpsc, oneshot};
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::transport::SharedLink;
|
||||
use crate::dataplane::{IpPlugin, PluginCapability, PluginContext, PluginError};
|
||||
use crate::identity::NetworkId;
|
||||
use tsunagi::BoxFuture;
|
||||
use tsunagi::dataplane::PacketSink;
|
||||
use tsunagi::dataplane::transport::SharedLink;
|
||||
use tsunagi::dataplane::{IpPlugin, PluginCapability, PluginContext, PluginError};
|
||||
use tsunagi::identity::NetworkId;
|
||||
|
||||
use super::announcement::{ValidatedAnnouncement, WgAnnouncement};
|
||||
use super::config::{DEFAULT_INTERFACE_PREFIX, interface_name};
|
||||
use super::device::{PeerSummary, WireguardDevice};
|
||||
use super::keys::{WgPublicKey, WgSecretKey};
|
||||
use super::overlay::{OVERLAY_PREFIX_LEN, overlay_address, overlay_prefix};
|
||||
use super::store::WgKeyStore;
|
||||
use super::tun::{TunFactory, TunRequest};
|
||||
use crate::state::Ipv4Range;
|
||||
use crate::announcement::{ValidatedAnnouncement, WgAnnouncement};
|
||||
use crate::device::{PeerSummary, WireguardDevice};
|
||||
use crate::keys::{WgPublicKey, WgSecretKey};
|
||||
use crate::store::WgKeyStore;
|
||||
use tsunagi::state::Ipv4Range;
|
||||
|
||||
/// The protocol identifier this plugin announces.
|
||||
pub const WIREGUARD_PROTOCOL: &str = "wireguard";
|
||||
|
||||
/// Smallest interface MTU IPv6 permits, from RFC 8200.
|
||||
/// The protocol id of this plugin.
|
||||
///
|
||||
/// This is not advice, it is a hard limit. Linux tears IPv6 down entirely on
|
||||
/// an interface whose MTU is below it — the per-device `/proc/sys/net/ipv6`
|
||||
/// entries disappear and `ip -6 address add` fails with `Invalid argument` —
|
||||
/// so the overlay address could never be assigned. Anything smaller is
|
||||
/// rejected up front instead of failing obscurely later.
|
||||
pub const MIN_MTU: u32 = 1280;
|
||||
/// `wg-quic`, because that is what it is: WireGuard's cryptography carried
|
||||
/// in QUIC datagrams. The name is on the wire, so it is a protocol name and
|
||||
/// not a description of the implementation.
|
||||
pub const WIREGUARD_PROTOCOL: &str = "wg-quic";
|
||||
|
||||
/// Smallest interface MTU the overlay accepts.
|
||||
///
|
||||
/// 576 bytes is what IPv4 guarantees every host can reassemble (RFC 1122),
|
||||
/// so nothing below it is worth offering. The floor used to be 1280 because
|
||||
/// Linux tears IPv6 down on an interface below that; the overlay is IPv4
|
||||
/// now, so that constraint is gone and a path with small datagrams — a
|
||||
/// relay, typically — can be matched instead of warned about.
|
||||
pub const MIN_MTU: u32 = 576;
|
||||
|
||||
/// Default interface MTU.
|
||||
///
|
||||
/// Equal to [`MIN_MTU`], because the overlay is IPv6 and there is no room
|
||||
/// below it.
|
||||
pub const DEFAULT_MTU: u32 = MIN_MTU;
|
||||
/// Comfortably under what a direct path carries, and the same number the
|
||||
/// overlay used before, so an existing network does not have to change.
|
||||
pub const DEFAULT_MTU: u32 = 1280;
|
||||
|
||||
/// Bytes WireGuard adds to a packet: type and reserved, receiver index,
|
||||
/// counter and the Poly1305 tag.
|
||||
@@ -78,14 +81,12 @@ pub const WIREGUARD_OVERHEAD: u32 = 32;
|
||||
pub struct WireguardConfig {
|
||||
/// Directory for the plugin's own key store. Separate from agent state.
|
||||
pub state_dir: PathBuf,
|
||||
/// Prefix of the interface names this plugin creates.
|
||||
///
|
||||
/// Two agents on one host in the same network need different prefixes,
|
||||
/// because the rest of the name is derived from the network id.
|
||||
pub interface_prefix: String,
|
||||
/// WireGuard keepalive, which keeps tunnels and their links warm.
|
||||
pub keepalive: Option<u16>,
|
||||
/// Interface MTU. See [`DEFAULT_MTU`].
|
||||
/// The largest packet a tunnel will carry.
|
||||
///
|
||||
/// Not the interface MTU, which belongs to the agent: this is what this
|
||||
/// protocol refuses to encrypt because it would not fit one datagram.
|
||||
pub mtu: u32,
|
||||
/// How long to coalesce changes before reconciling.
|
||||
pub reconcile_debounce: Duration,
|
||||
@@ -99,7 +100,6 @@ impl WireguardConfig {
|
||||
pub fn new(state_dir: impl Into<PathBuf>) -> Self {
|
||||
Self {
|
||||
state_dir: state_dir.into(),
|
||||
interface_prefix: DEFAULT_INTERFACE_PREFIX.to_string(),
|
||||
keepalive: Some(25),
|
||||
mtu: DEFAULT_MTU,
|
||||
reconcile_debounce: Duration::from_millis(200),
|
||||
@@ -107,12 +107,6 @@ impl WireguardConfig {
|
||||
}
|
||||
}
|
||||
|
||||
/// Sets the interface name prefix.
|
||||
pub fn with_interface_prefix(mut self, prefix: impl Into<String>) -> Self {
|
||||
self.interface_prefix = prefix.into();
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the interface MTU.
|
||||
///
|
||||
/// Validated when the plugin is opened; see [`MIN_MTU`].
|
||||
@@ -139,30 +133,16 @@ impl WireguardConfig {
|
||||
pub struct NetworkOverview {
|
||||
/// The network.
|
||||
pub network: NetworkId,
|
||||
/// Packet interface this plugin created for it.
|
||||
pub interface: String,
|
||||
/// Interface MTU.
|
||||
/// The largest packet a tunnel in this network will carry.
|
||||
pub mtu: u32,
|
||||
/// This agent's WireGuard public key in this network.
|
||||
pub public_key: WgPublicKey,
|
||||
/// This agent's overlay address.
|
||||
pub overlay_address: IpAddr,
|
||||
/// The overlay subnet every member shares.
|
||||
pub overlay_prefix: IpAddr,
|
||||
/// Prefix length of the overlay subnet.
|
||||
pub overlay_prefix_len: u8,
|
||||
/// This agent's IPv4 overlay address, when the overlay is dual stack.
|
||||
/// This agent's overlay address, once the network has agreed one.
|
||||
pub overlay_address_v4: Option<Ipv4Addr>,
|
||||
/// The IPv4 overlay range in use.
|
||||
pub ipv4_range: Option<Ipv4Range>,
|
||||
/// Peers this agent knows about.
|
||||
pub peers: Vec<PeerOverview>,
|
||||
/// Unicast packets the operating system sent to an address no peer owns.
|
||||
pub unroutable_packets: u64,
|
||||
/// Multicast packets dropped. Expected, not a fault.
|
||||
pub multicast_packets: u64,
|
||||
/// One destination nobody owned, if there was one.
|
||||
pub unroutable_sample: Option<IpAddr>,
|
||||
}
|
||||
|
||||
impl NetworkOverview {
|
||||
@@ -179,9 +159,7 @@ pub struct PeerOverview {
|
||||
pub endpoint_id: EndpointId,
|
||||
/// The peer's WireGuard public key.
|
||||
pub public_key: WgPublicKey,
|
||||
/// The overlay address derived for it locally.
|
||||
pub overlay_address: IpAddr,
|
||||
/// Its IPv4 overlay address, once a tunnel exists and it won the address.
|
||||
/// The overlay address the network agreed it holds.
|
||||
pub overlay_address_v4: Option<Ipv4Addr>,
|
||||
/// Whether a data plane link to it exists.
|
||||
pub has_link: bool,
|
||||
@@ -201,7 +179,6 @@ impl PeerOverview {
|
||||
#[derive(Debug)]
|
||||
struct NetworkState {
|
||||
key: WgSecretKey,
|
||||
interface: String,
|
||||
device: Option<Arc<WireguardDevice>>,
|
||||
announcements: HashMap<EndpointId, ValidatedAnnouncement>,
|
||||
links: HashMap<EndpointId, SharedLink>,
|
||||
@@ -209,14 +186,6 @@ struct NetworkState {
|
||||
allocations: HashMap<EndpointId, Ipv4Addr>,
|
||||
/// The range those allocations came from.
|
||||
ipv4_range: Option<Ipv4Range>,
|
||||
/// The address last reported as missing, so it is said once, not forever.
|
||||
reported_missing_v4: Option<Ipv4Addr>,
|
||||
/// What was last applied to the host interface.
|
||||
///
|
||||
/// The overlay IPv4 address is allocated at run time and can change while
|
||||
/// the agent runs, so the interface has to be brought back in line
|
||||
/// without being recreated — recreating it would drop every tunnel.
|
||||
applied: Option<TunRequest>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
@@ -241,16 +210,16 @@ struct Worker {
|
||||
config: WireguardConfig,
|
||||
/// This agent's endpoint id, learned when the plugin is attached.
|
||||
local_id: OnceLock<EndpointId>,
|
||||
tun_factory: Arc<dyn TunFactory>,
|
||||
store: WgKeyStore,
|
||||
shared: Mutex<Shared>,
|
||||
context: OnceLock<PluginContext>,
|
||||
/// Where decrypted packets go, once the agent has attached one.
|
||||
sink: OnceLock<Arc<dyn PacketSink>>,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for Worker {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("Worker")
|
||||
.field("tun", &self.tun_factory.name())
|
||||
.field("store", &self.store.path())
|
||||
.finish()
|
||||
}
|
||||
@@ -265,22 +234,65 @@ pub struct WireguardPlugin {
|
||||
}
|
||||
|
||||
impl WireguardPlugin {
|
||||
/// The settings this protocol accepts.
|
||||
pub const OPTIONS: &'static [tsunagi::dataplane::ProtocolOption] = &[
|
||||
tsunagi::dataplane::ProtocolOption {
|
||||
key: "keepalive",
|
||||
value: "SECONDS",
|
||||
help: "keeps a tunnel and its link warm through a NAT; 0 turns it off",
|
||||
default: Some("25"),
|
||||
},
|
||||
tsunagi::dataplane::ProtocolOption {
|
||||
key: "mtu",
|
||||
value: "BYTES",
|
||||
help: "largest packet a tunnel will carry, at least 576",
|
||||
default: Some("1280"),
|
||||
},
|
||||
];
|
||||
|
||||
/// Applies `key=value` settings to a configuration.
|
||||
///
|
||||
/// An unknown key is refused rather than ignored: a setting that was
|
||||
/// silently dropped looks exactly like one that did not work.
|
||||
pub fn configure(
|
||||
mut config: WireguardConfig,
|
||||
options: &[(String, String)],
|
||||
) -> Result<WireguardConfig, PluginError> {
|
||||
for (key, value) in options {
|
||||
match key.as_str() {
|
||||
"keepalive" => {
|
||||
let seconds: u16 = value.parse().map_err(|_| {
|
||||
PluginError::Other(format!("keepalive={value} is not a number of seconds"))
|
||||
})?;
|
||||
config.keepalive = (seconds > 0).then_some(seconds);
|
||||
}
|
||||
"mtu" => {
|
||||
let mtu: u32 = value.parse().map_err(|_| {
|
||||
PluginError::Other(format!("mtu={value} is not a number of bytes"))
|
||||
})?;
|
||||
config = config.with_mtu(mtu);
|
||||
}
|
||||
other => {
|
||||
let known: Vec<&str> = Self::OPTIONS.iter().map(|spec| spec.key).collect();
|
||||
return Err(PluginError::Other(format!(
|
||||
"`{other}` is not a setting of {WIREGUARD_PROTOCOL}; it takes {}",
|
||||
known.join(", ")
|
||||
)));
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(config)
|
||||
}
|
||||
|
||||
/// Opens the plugin's key store and starts its reconciliation task.
|
||||
///
|
||||
/// Must be called from inside a tokio runtime; the plugin starts no
|
||||
/// runtime of its own.
|
||||
pub async fn open(
|
||||
config: WireguardConfig,
|
||||
tun_factory: Arc<dyn TunFactory>,
|
||||
) -> Result<Arc<Self>, PluginError> {
|
||||
// Validate the prefix once, here, rather than failing per network.
|
||||
interface_name(&config.interface_prefix, NetworkId::from_bytes([0u8; 32]))?;
|
||||
|
||||
pub async fn open(config: WireguardConfig) -> Result<Arc<Self>, PluginError> {
|
||||
if config.mtu < MIN_MTU {
|
||||
return Err(PluginError::Other(format!(
|
||||
"an MTU of {} is below the {MIN_MTU} bytes IPv6 requires (RFC 8200). \
|
||||
Linux disables IPv6 on an interface below that, so the overlay address \
|
||||
could never be assigned.",
|
||||
"an MTU of {} is below the {MIN_MTU} bytes every IPv4 host must be able \
|
||||
to reassemble (RFC 1122)",
|
||||
config.mtu
|
||||
)));
|
||||
}
|
||||
@@ -293,10 +305,10 @@ impl WireguardPlugin {
|
||||
let worker = Arc::new(Worker {
|
||||
config,
|
||||
local_id: OnceLock::new(),
|
||||
tun_factory,
|
||||
store,
|
||||
shared: Mutex::new(Shared::default()),
|
||||
context: OnceLock::new(),
|
||||
sink: OnceLock::new(),
|
||||
});
|
||||
|
||||
let (commands, receiver) = mpsc::channel(64);
|
||||
@@ -332,7 +344,6 @@ impl WireguardPlugin {
|
||||
.map(|(endpoint_id, announcement)| PeerOverview {
|
||||
endpoint_id: *endpoint_id,
|
||||
public_key: announcement.public_key,
|
||||
overlay_address: IpAddr::V6(announcement.overlay_address),
|
||||
overlay_address_v4: state.allocations.get(endpoint_id).copied(),
|
||||
has_link: state.links.contains_key(endpoint_id),
|
||||
tunnel: tunnels.get(&announcement.public_key).cloned(),
|
||||
@@ -342,29 +353,11 @@ impl WireguardPlugin {
|
||||
|
||||
Some(NetworkOverview {
|
||||
network,
|
||||
interface: state.interface.clone(),
|
||||
mtu: self.worker.config.mtu,
|
||||
public_key: state.key.public(),
|
||||
overlay_address: IpAddr::V6(overlay_address(network, &state.key.public())),
|
||||
overlay_prefix: IpAddr::V6(overlay_prefix(network)),
|
||||
overlay_prefix_len: OVERLAY_PREFIX_LEN,
|
||||
overlay_address_v4: state.allocations.get(&self.worker.local_id()).copied(),
|
||||
ipv4_range: state.ipv4_range,
|
||||
peers,
|
||||
unroutable_packets: state
|
||||
.device
|
||||
.as_ref()
|
||||
.map(|device| device.unroutable_packets())
|
||||
.unwrap_or(0),
|
||||
multicast_packets: state
|
||||
.device
|
||||
.as_ref()
|
||||
.map(|device| device.multicast_packets())
|
||||
.unwrap_or(0),
|
||||
unroutable_sample: state
|
||||
.device
|
||||
.as_ref()
|
||||
.and_then(|device| device.unroutable_sample()),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -433,7 +426,6 @@ impl Worker {
|
||||
return Ok(false);
|
||||
}
|
||||
|
||||
let name = interface_name(&self.config.interface_prefix, network)?;
|
||||
let worker = Arc::clone(self);
|
||||
let key = tokio::task::spawn_blocking(move || worker.store.load_or_create(network))
|
||||
.await
|
||||
@@ -443,14 +435,11 @@ impl Worker {
|
||||
let mut shared = self.lock_shared();
|
||||
shared.networks.entry(network).or_insert(NetworkState {
|
||||
key,
|
||||
interface: name,
|
||||
device: None,
|
||||
announcements: HashMap::new(),
|
||||
links: HashMap::new(),
|
||||
allocations: HashMap::new(),
|
||||
ipv4_range: None,
|
||||
reported_missing_v4: None,
|
||||
applied: None,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -463,79 +452,35 @@ impl Worker {
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// What the host interface for a network should look like.
|
||||
fn desired_request(&self, state: &NetworkState, network: NetworkId) -> TunRequest {
|
||||
let own_range = state.ipv4_range;
|
||||
TunRequest {
|
||||
name: state.interface.clone(),
|
||||
address: overlay_address(network, &state.key.public()),
|
||||
prefix_len: OVERLAY_PREFIX_LEN,
|
||||
address_v4: state.allocations.get(&self.local_id()).copied(),
|
||||
prefix_len_v4: own_range.map_or(0, |range| range.prefix_len),
|
||||
mtu: self.config.mtu,
|
||||
}
|
||||
}
|
||||
|
||||
/// Creates the packet interface and starts the WireGuard device.
|
||||
/// Starts this network's tunnels.
|
||||
///
|
||||
/// No interface is created: one agent has one, it belongs to the system
|
||||
/// level, and decrypted packets are handed there rather than written
|
||||
/// out.
|
||||
async fn ensure_device(&self, network: NetworkId) -> Result<(), PluginError> {
|
||||
let (request, key, own_range) = {
|
||||
let key = {
|
||||
let shared = self.lock_shared();
|
||||
match shared.networks.get(&network) {
|
||||
Some(state) if state.device.is_none() => (
|
||||
self.desired_request(state, network),
|
||||
state.key.clone(),
|
||||
state.ipv4_range,
|
||||
),
|
||||
Some(state) if state.device.is_none() => state.key.clone(),
|
||||
_ => return Ok(()),
|
||||
}
|
||||
};
|
||||
|
||||
let applied = request.clone();
|
||||
let tun = self.tun_factory.create(request).await?;
|
||||
let device = Arc::new(WireguardDevice::start(network, key, tun, own_range));
|
||||
let sink = match self.sink.get() {
|
||||
Some(sink) => Arc::clone(sink),
|
||||
// Not attached to an agent: the protocol still runs, and its
|
||||
// packets have nowhere to go.
|
||||
None => Arc::new(tsunagi::dataplane::DiscardPackets) as Arc<dyn PacketSink>,
|
||||
};
|
||||
let device = Arc::new(WireguardDevice::start(network, key, sink));
|
||||
|
||||
let mut shared = self.lock_shared();
|
||||
if let Some(state) = shared.networks.get_mut(&network) {
|
||||
state.interface = device.interface().to_string();
|
||||
state.device = Some(device);
|
||||
state.applied = Some(applied);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Brings a live interface back in line after the overlay changed its
|
||||
/// mind about this agent's address.
|
||||
async fn ensure_addresses(&self, network: NetworkId) {
|
||||
let wanted = {
|
||||
let shared = self.lock_shared();
|
||||
match shared.networks.get(&network) {
|
||||
Some(state) if state.device.is_some() => {
|
||||
let wanted = self.desired_request(state, network);
|
||||
if state.applied.as_ref() == Some(&wanted) {
|
||||
return;
|
||||
}
|
||||
wanted
|
||||
}
|
||||
_ => return,
|
||||
}
|
||||
};
|
||||
|
||||
match self.tun_factory.reconfigure(wanted.clone()).await {
|
||||
Ok(()) => {
|
||||
let mut shared = self.lock_shared();
|
||||
if let Some(state) = shared.networks.get_mut(&network) {
|
||||
state.applied = Some(wanted);
|
||||
}
|
||||
}
|
||||
Err(err) => {
|
||||
// Not fatal: the tunnels keep running on the addresses that
|
||||
// are there, and the next reconciliation tries again.
|
||||
tracing::warn!(%err, "cannot update the overlay interface addresses");
|
||||
self.report(network, err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Brings the running tunnels in line with what is known.
|
||||
///
|
||||
/// A peer gets a tunnel once both halves have arrived: its announcement,
|
||||
@@ -550,10 +495,6 @@ impl Worker {
|
||||
};
|
||||
|
||||
let allocations = state.allocations.clone();
|
||||
let own_v4 = state.allocations.get(&self.local_id()).copied();
|
||||
let interface = state.device.as_ref().map(|_| state.interface.clone());
|
||||
let range = state.ipv4_range;
|
||||
let state_reported = state.reported_missing_v4;
|
||||
let mut wanted: Vec<WgPublicKey> = Vec::new();
|
||||
let mut too_small: Vec<(usize, usize)> = Vec::new();
|
||||
for (endpoint_id, announcement) in &state.announcements {
|
||||
@@ -594,53 +535,15 @@ impl Worker {
|
||||
}
|
||||
device.retain_peers(&wanted);
|
||||
|
||||
// The agent assigns this address itself, so finding it absent means
|
||||
// the assignment did not take — something outside removed it, or the
|
||||
// provisioner reported a success it did not achieve. Left unsaid it
|
||||
// looks like a broken network: the kernel would send packets with the
|
||||
// wrong source address and every peer would drop them. So it is
|
||||
// checked rather than assumed, because the assumption is exactly the
|
||||
// kind that has been wrong here before.
|
||||
let missing_v4 = match (own_v4, interface.as_deref(), range) {
|
||||
(Some(address), Some(interface), Some(range))
|
||||
if !super::tun::address_is_local(IpAddr::V4(address)) =>
|
||||
{
|
||||
let already = state_reported == Some(address);
|
||||
if let Some(state) = shared.networks.get_mut(&network) {
|
||||
state.reported_missing_v4 = Some(address);
|
||||
}
|
||||
(!already).then_some((address, interface.to_string(), range))
|
||||
}
|
||||
_ => {
|
||||
if let Some(state) = shared.networks.get_mut(&network) {
|
||||
state.reported_missing_v4 = None;
|
||||
}
|
||||
None
|
||||
}
|
||||
};
|
||||
drop(shared);
|
||||
|
||||
if let Some((address, interface, range)) = missing_v4 {
|
||||
self.report(
|
||||
network,
|
||||
format!(
|
||||
"this agent was allocated {address}/{} but the address is not on any \
|
||||
interface, so IPv4 cannot work: packets would leave with the wrong \
|
||||
source and every peer would drop them. It should have been assigned \
|
||||
to `{interface}` automatically; check whether something else removed \
|
||||
it.",
|
||||
range.prefix_len
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
for (available, needed) in too_small {
|
||||
self.report(
|
||||
network,
|
||||
format!(
|
||||
"this path carries only {available} byte datagrams but a {} byte MTU needs \
|
||||
{needed}; packets larger than {} bytes will be dropped. Lower the MTU only \
|
||||
if you can stay at or above {MIN_MTU}, which IPv6 requires.",
|
||||
{needed}; packets larger than {} bytes will be dropped. The floor is \
|
||||
{MIN_MTU} bytes, what every IPv4 host must be able to reassemble.",
|
||||
self.config.mtu,
|
||||
available.saturating_sub(WIREGUARD_OVERHEAD as usize)
|
||||
),
|
||||
@@ -655,11 +558,10 @@ impl Worker {
|
||||
// kernel to remove an interface this agent created; the explicit
|
||||
// destroy makes that immediate and definite rather than dependent on
|
||||
// the last reader letting go.
|
||||
let removed = self.lock_shared().networks.remove(&network);
|
||||
if let Some(state) = removed {
|
||||
drop(state.device);
|
||||
self.tun_factory.destroy(&state.interface).await;
|
||||
}
|
||||
// Dropping the state drops the tunnels, which stops their tasks and
|
||||
// closes their links. There is no interface to remove: the agent owns
|
||||
// it, and it outlives any one network.
|
||||
self.lock_shared().networks.remove(&network);
|
||||
}
|
||||
|
||||
fn known_networks(&self) -> Vec<NetworkId> {
|
||||
@@ -725,7 +627,6 @@ async fn run(worker: Arc<Worker>, mut commands: mpsc::Receiver<Command>) {
|
||||
}, if wait_until.is_some() => {
|
||||
deadline = None;
|
||||
for network in std::mem::take(&mut pending) {
|
||||
worker.ensure_addresses(network).await;
|
||||
worker.sync(network);
|
||||
}
|
||||
}
|
||||
@@ -736,7 +637,6 @@ async fn run(worker: Arc<Worker>, mut commands: mpsc::Receiver<Command>) {
|
||||
if let Err(err) = worker.ensure_device(network).await {
|
||||
tracing::debug!(%err, "packet interface still unavailable");
|
||||
}
|
||||
worker.ensure_addresses(network).await;
|
||||
worker.sync(network);
|
||||
}
|
||||
}
|
||||
@@ -749,10 +649,19 @@ impl IpPlugin for WireguardPlugin {
|
||||
WIREGUARD_PROTOCOL
|
||||
}
|
||||
|
||||
fn protocol_version(&self) -> u16 {
|
||||
crate::announcement::ANNOUNCEMENT_VERSION
|
||||
}
|
||||
|
||||
fn options(&self) -> &'static [tsunagi::dataplane::ProtocolOption] {
|
||||
Self::OPTIONS
|
||||
}
|
||||
|
||||
fn attach(&self, context: PluginContext) {
|
||||
if let Some(local) = context.local_endpoint_id() {
|
||||
let _ = self.worker.local_id.set(local);
|
||||
}
|
||||
let _ = self.worker.sink.set(context.packet_sink());
|
||||
let _ = self.worker.context.set(context);
|
||||
}
|
||||
|
||||
@@ -777,7 +686,7 @@ impl IpPlugin for WireguardPlugin {
|
||||
let announcement = WgAnnouncement::new(network, &state.key.public());
|
||||
Ok(Some(PluginCapability {
|
||||
protocol: WIREGUARD_PROTOCOL.to_string(),
|
||||
version: super::announcement::ANNOUNCEMENT_VERSION,
|
||||
version: crate::announcement::ANNOUNCEMENT_VERSION,
|
||||
enabled: true,
|
||||
data: announcement.encode()?,
|
||||
}))
|
||||
@@ -845,6 +754,15 @@ impl IpPlugin for WireguardPlugin {
|
||||
}
|
||||
}
|
||||
|
||||
fn carry(&self, network: NetworkId, peer: EndpointId, packet: bytes::Bytes) -> bool {
|
||||
let shared = self.worker.lock_shared();
|
||||
shared
|
||||
.networks
|
||||
.get(&network)
|
||||
.and_then(|state| state.device.as_ref())
|
||||
.is_some_and(|device| device.carry(peer, &packet))
|
||||
}
|
||||
|
||||
fn on_peer_link(&self, network: NetworkId, peer: EndpointId, link: SharedLink) {
|
||||
self.nudge(Command::Link {
|
||||
network,
|
||||
@@ -17,10 +17,10 @@ use std::sync::Mutex;
|
||||
|
||||
use rusqlite::{Connection, OptionalExtension, params};
|
||||
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::identity::NetworkId;
|
||||
use tsunagi::dataplane::PluginError;
|
||||
use tsunagi::identity::NetworkId;
|
||||
|
||||
use super::keys::{KEY_LEN, WgSecretKey};
|
||||
use crate::keys::{KEY_LEN, WgSecretKey};
|
||||
|
||||
/// Schema version written by this build.
|
||||
pub const SCHEMA_VERSION: i64 = 1;
|
||||
@@ -37,14 +37,14 @@ impl WgKeyStore {
|
||||
pub fn open(path: impl AsRef<Path>) -> Result<Self, PluginError> {
|
||||
let path = path.as_ref().to_path_buf();
|
||||
if let Some(parent) = path.parent() {
|
||||
crate::storage::create_dir(parent)
|
||||
tsunagi::storage::create_dir(parent)
|
||||
.map_err(|err| PluginError::Other(format!("cannot create {parent:?}: {err}")))?;
|
||||
}
|
||||
let existed = path.exists();
|
||||
let conn = Connection::open(&path).map_err(|err| {
|
||||
PluginError::Other(format!("cannot open the WireGuard key store: {err}"))
|
||||
})?;
|
||||
crate::storage::restrict_path_permissions(&path)
|
||||
tsunagi::storage::restrict_path_permissions(&path)
|
||||
.map_err(|err| PluginError::Other(format!("cannot secure the key store: {err}")))?;
|
||||
|
||||
conn.busy_timeout(std::time::Duration::from_secs(5))
|
||||
@@ -169,7 +169,7 @@ mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use super::*;
|
||||
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
use tsunagi::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
|
||||
fn network(name: &str) -> NetworkId {
|
||||
NetworkKeys::derive(
|
||||
+43
-36
@@ -11,29 +11,27 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use std::net::{IpAddr, Ipv4Addr};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use common::{config_with, network, wait_until};
|
||||
use tempfile::TempDir;
|
||||
use tsunagi::dataplane::IpPlugin;
|
||||
use tsunagi::dataplane::wireguard::{
|
||||
Cidr, InterfaceState, LinkKind, ManagedTunFactory, MockHost, MockProvisioner, WireguardConfig,
|
||||
WireguardPlugin,
|
||||
};
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::identity::NetworkId;
|
||||
use tsunagi::overlay::{
|
||||
Cidr, InterfaceState, LinkKind, ManagedTunFactory, MockHost, MockProvisioner,
|
||||
};
|
||||
use tsunagi::state::DEFAULT_IPV4_RANGE;
|
||||
use tsunagi::testing::{config_with, network, wait_until};
|
||||
use tsunagi::{Agent, NetworkStatus};
|
||||
use tsunagi_wg_quic::{WireguardConfig, WireguardPlugin};
|
||||
|
||||
/// An agent whose overlay interface is applied to a pretend host.
|
||||
struct HostedAgent {
|
||||
_dir: TempDir,
|
||||
agent: Agent,
|
||||
plugin: Arc<WireguardPlugin>,
|
||||
_plugin: Arc<WireguardPlugin>,
|
||||
host: MockHost,
|
||||
}
|
||||
|
||||
@@ -43,12 +41,12 @@ impl HostedAgent {
|
||||
let provisioner = Arc::new(MockProvisioner::new(host.clone()));
|
||||
let factory = Arc::new(ManagedTunFactory::new(provisioner));
|
||||
let config = WireguardConfig::new(dir.path().join("wireguard"))
|
||||
.with_interface_prefix(tag)
|
||||
.with_reconcile(Duration::from_millis(20), Duration::from_millis(100));
|
||||
let plugin = WireguardPlugin::open(config, factory).await.unwrap();
|
||||
let plugin = WireguardPlugin::open(config).await.unwrap();
|
||||
let agent = Agent::spawn(
|
||||
config_with(dir.path(), discovery)
|
||||
.with_overlay_ipv4_range(Some(DEFAULT_IPV4_RANGE))
|
||||
.with_interface(factory, tag, 1280)
|
||||
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
|
||||
)
|
||||
.await
|
||||
@@ -56,17 +54,18 @@ impl HostedAgent {
|
||||
Self {
|
||||
_dir: dir,
|
||||
agent,
|
||||
plugin,
|
||||
_plugin: plugin,
|
||||
host,
|
||||
}
|
||||
}
|
||||
|
||||
/// The interface name the plugin settled on for a network.
|
||||
async fn interface(&self, network: NetworkId) -> String {
|
||||
wait_until("the plugin named its interface", || async {
|
||||
self.plugin
|
||||
.overview(network)
|
||||
.map(|view| view.interface)
|
||||
/// The one interface this agent owns. Not per network.
|
||||
async fn interface(&self, _network: NetworkId) -> String {
|
||||
wait_until("the agent named its interface", || async {
|
||||
self.agent
|
||||
.overlay()
|
||||
.map(|overlay| overlay.interface)
|
||||
.filter(|name| !name.is_empty())
|
||||
})
|
||||
.await
|
||||
@@ -94,11 +93,12 @@ fn v4(state: &InterfaceState) -> Vec<Ipv4Addr> {
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn has_v6(state: &InterfaceState) -> bool {
|
||||
state
|
||||
.addresses
|
||||
.iter()
|
||||
.any(|cidr| matches!(cidr.addr, IpAddr::V6(_)))
|
||||
/// Whether the overlay address has been put on the interface yet.
|
||||
///
|
||||
/// It is allocated and signed at the system level, so it arrives on a later
|
||||
/// reconciliation than the interface itself rather than with it.
|
||||
fn addressed(state: &InterfaceState) -> bool {
|
||||
!state.addresses.is_empty()
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -111,18 +111,13 @@ async fn an_agent_creates_and_configures_its_own_overlay_interface() {
|
||||
let interface = agent.interface(network_id).await;
|
||||
|
||||
let state = agent
|
||||
.wait_for_host("the interface to be created", &interface, |state| {
|
||||
state.filter(|state| !state.addresses.is_empty())
|
||||
})
|
||||
.wait_for_host("the interface to be created", &interface, |state| state)
|
||||
.await;
|
||||
|
||||
assert_eq!(state.kind, LinkKind::Tun);
|
||||
assert!(state.up, "the agent brought the link up itself");
|
||||
assert_eq!(state.mtu, 1280);
|
||||
assert!(has_v6(&state), "the derived overlay address is assigned");
|
||||
|
||||
// The IPv4 address is allocated at run time, so it arrives on a later
|
||||
// reconciliation than the interface itself.
|
||||
let addresses = agent
|
||||
.wait_for_host("the allocated IPv4 address", &interface, |state| {
|
||||
state.map(|state| v4(&state)).filter(|v4| !v4.is_empty())
|
||||
@@ -160,9 +155,14 @@ async fn an_interface_left_by_a_crashed_run_is_replaced_rather_than_tripped_over
|
||||
let network_id = agent.agent.join_network(&name, &secret).await.unwrap();
|
||||
assert_eq!(agent.interface(network_id).await, interface);
|
||||
|
||||
// Waited on directly rather than on "it has some address": the address
|
||||
// this agent was allocated arrives a reconciliation after the interface
|
||||
// does, and in between the leftover is still the only one there.
|
||||
let state = agent
|
||||
.wait_for_host("the interface to be rebuilt", &interface, |state| {
|
||||
state.filter(|state| state.attached && has_v6(state))
|
||||
.wait_for_host("the stale address to be replaced", &interface, |state| {
|
||||
state.filter(|state| {
|
||||
state.attached && addressed(state) && !state.addresses.contains(&stale)
|
||||
})
|
||||
})
|
||||
.await;
|
||||
assert!(
|
||||
@@ -216,7 +216,10 @@ async fn an_interface_belonging_to_something_else_is_left_alone() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn leaving_a_network_removes_the_interface_from_the_host() {
|
||||
async fn leaving_a_network_takes_its_address_off_the_interface_but_not_the_interface() {
|
||||
// The interface belongs to the agent, so it outlives any one network:
|
||||
// another may still be using it. What a network takes with it is its own
|
||||
// address.
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("provision-cleanup");
|
||||
let agent = HostedAgent::spawn(&discovery, "tsunx", MockHost::new()).await;
|
||||
@@ -224,21 +227,25 @@ async fn leaving_a_network_removes_the_interface_from_the_host() {
|
||||
let network_id = agent.agent.join_network(&name, &secret).await.unwrap();
|
||||
let interface = agent.interface(network_id).await;
|
||||
agent
|
||||
.wait_for_host("the interface to exist", &interface, |state| state)
|
||||
.wait_for_host("the address to be assigned", &interface, |state| {
|
||||
state.filter(addressed)
|
||||
})
|
||||
.await;
|
||||
|
||||
agent.agent.deactivate_network(network_id).await.unwrap();
|
||||
|
||||
agent
|
||||
.wait_for_host("the interface to be removed", &interface, |state| {
|
||||
state.is_none().then_some(())
|
||||
let state = agent
|
||||
.wait_for_host("the address to be withdrawn", &interface, |state| {
|
||||
state.filter(|state| !addressed(state))
|
||||
})
|
||||
.await;
|
||||
assert_eq!(state.kind, LinkKind::Tun, "the interface is still there");
|
||||
|
||||
// And it goes when the agent does.
|
||||
agent.agent.shutdown().await;
|
||||
assert!(
|
||||
agent.host.names().is_empty(),
|
||||
"nothing is left behind: {:?}",
|
||||
agent.host.names()
|
||||
);
|
||||
|
||||
agent.agent.shutdown().await;
|
||||
}
|
||||
@@ -6,19 +6,18 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use common::{config_with, network, wait_for_peers, wait_until};
|
||||
use tempfile::TempDir;
|
||||
use tsunagi::dataplane::IpPlugin;
|
||||
use tsunagi::dataplane::wireguard::{MemoryTunFactory, WireguardConfig, WireguardPlugin};
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::ipc::unix::{ControlSocket, request_status};
|
||||
use tsunagi::ipc::{StatusReport, control_socket_path};
|
||||
use tsunagi::overlay::MemoryTunFactory;
|
||||
use tsunagi::testing::{config_with, network, wait_for_peers, wait_until};
|
||||
use tsunagi::{Agent, BoxFuture};
|
||||
use tsunagi_wg_quic::{WireguardConfig, WireguardPlugin};
|
||||
|
||||
/// Builds the report the way the binary does, from the agent plus the plugin.
|
||||
fn source(agent: Agent, plugin: Arc<WireguardPlugin>) -> Arc<dyn tsunagi::ipc::unix::ReportSource> {
|
||||
@@ -46,17 +45,19 @@ fn source(agent: Agent, plugin: Arc<WireguardPlugin>) -> Arc<dyn tsunagi::ipc::u
|
||||
.collect(),
|
||||
overlay: plugin.overview(net.network_id).map(|view| {
|
||||
tsunagi::ipc::OverlayReport {
|
||||
interface: view.interface.clone(),
|
||||
mtu: view.mtu,
|
||||
address: view.overlay_address.to_string(),
|
||||
prefix: view.overlay_prefix.to_string(),
|
||||
prefix_len: view.overlay_prefix_len,
|
||||
// The interface belongs to the agent now.
|
||||
interface: agent
|
||||
.overlay()
|
||||
.map_or_else(String::new, |overlay| overlay.interface),
|
||||
mtu: agent.overlay().map_or(0, |overlay| overlay.mtu),
|
||||
address: view.overlay_address_v4.map(|a| a.to_string()),
|
||||
prefix_len: view.ipv4_range.map_or(0, |range| range.prefix_len),
|
||||
peers: view
|
||||
.peers
|
||||
.iter()
|
||||
.map(|peer| tsunagi::ipc::OverlayPeerReport {
|
||||
public_key: peer.public_key.to_string(),
|
||||
address: peer.overlay_address.to_string(),
|
||||
address: peer.overlay_address_v4.map(|a| a.to_string()),
|
||||
handshake_secs_ago: peer
|
||||
.tunnel
|
||||
.as_ref()
|
||||
@@ -96,14 +97,14 @@ async fn a_client_sees_the_agent_and_its_overlay() {
|
||||
let tuns = MemoryTunFactory::new();
|
||||
let plugin = WireguardPlugin::open(
|
||||
WireguardConfig::new(dir_a.path().join("wg"))
|
||||
.with_interface_prefix("tca")
|
||||
.with_reconcile(Duration::from_millis(20), Duration::from_millis(250)),
|
||||
Arc::new(tuns),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let agent = Agent::spawn(
|
||||
config_with(dir_a.path(), &discovery).with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
|
||||
config_with(dir_a.path(), &discovery)
|
||||
.with_interface(Arc::new(tuns), "tca0", 1280)
|
||||
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
@@ -111,14 +112,14 @@ async fn a_client_sees_the_agent_and_its_overlay() {
|
||||
let dir_b = TempDir::new().unwrap();
|
||||
let plugin_b = WireguardPlugin::open(
|
||||
WireguardConfig::new(dir_b.path().join("wg"))
|
||||
.with_interface_prefix("tcb")
|
||||
.with_reconcile(Duration::from_millis(20), Duration::from_millis(250)),
|
||||
Arc::new(MemoryTunFactory::new()),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let agent_b = Agent::spawn(
|
||||
config_with(dir_b.path(), &discovery).with_plugin(plugin_b.clone() as Arc<dyn IpPlugin>),
|
||||
config_with(dir_b.path(), &discovery)
|
||||
.with_interface(Arc::new(MemoryTunFactory::new()), "tcb0", 1280)
|
||||
.with_plugin(plugin_b.clone() as Arc<dyn IpPlugin>),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
@@ -168,7 +169,7 @@ async fn a_client_sees_the_agent_and_its_overlay() {
|
||||
"the tunnel is up: {:?}",
|
||||
overlay.peers
|
||||
);
|
||||
assert!(!overlay.address.is_empty());
|
||||
assert!(overlay.address.is_some());
|
||||
|
||||
control.shutdown().await;
|
||||
assert!(!socket_path.exists(), "the socket is removed on shutdown");
|
||||
@@ -232,46 +233,3 @@ fn the_socket_path_is_derived_and_short_enough() {
|
||||
control_socket_path(&std::path::PathBuf::from("/somewhere/else"))
|
||||
);
|
||||
}
|
||||
|
||||
/// Asking who this device is must not need the directory lock.
|
||||
///
|
||||
/// The lock belongs to the one agent allowed to *write* the state. `id` only
|
||||
/// reads, so making it take the lock would mean the question could never be
|
||||
/// answered while an agent was running — which is exactly when you want to
|
||||
/// ask it.
|
||||
#[cfg(feature = "cli")]
|
||||
#[tokio::test]
|
||||
async fn identity_can_be_read_while_an_agent_holds_the_directory() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
|
||||
// Hold the directory the way a running agent does.
|
||||
let agent = Agent::spawn(config_with(dir.path(), &discovery))
|
||||
.await
|
||||
.unwrap();
|
||||
let expected = agent.endpoint_id().to_string();
|
||||
|
||||
let output = tokio::process::Command::new(env!("CARGO_BIN_EXE_tsunagi"))
|
||||
.arg("id")
|
||||
// The same layout `StoragePaths::under` gives the agent above.
|
||||
.arg("--state-dir")
|
||||
.arg(dir.path().join("state"))
|
||||
.arg("--cache-dir")
|
||||
.arg(dir.path().join("cache"))
|
||||
.output()
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
||||
assert!(
|
||||
output.status.success(),
|
||||
"`id` failed while an agent was running:\n{stderr}"
|
||||
);
|
||||
assert!(
|
||||
stdout.contains(&expected),
|
||||
"expected {expected} in:\n{stdout}"
|
||||
);
|
||||
|
||||
agent.shutdown().await;
|
||||
}
|
||||
@@ -8,25 +8,24 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
|
||||
use std::net::Ipv4Addr;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use bytes::Bytes;
|
||||
use common::{config_with, network, settle, wait_event, wait_for_peers, wait_until};
|
||||
use iroh::EndpointId;
|
||||
use tempfile::TempDir;
|
||||
use tsunagi::agent::Event;
|
||||
use tsunagi::dataplane::IpPlugin;
|
||||
use tsunagi::dataplane::wireguard::{
|
||||
Ipv4Range, MemoryTun, MemoryTunFactory, WIREGUARD_PROTOCOL, WgAnnouncement, WgSecretKey,
|
||||
WireguardConfig, WireguardPlugin, overlay_address, overlay_prefix,
|
||||
};
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::identity::{NetworkId, NetworkName, NetworkSecret};
|
||||
use tsunagi::overlay::{MemoryTun, MemoryTunFactory};
|
||||
use tsunagi::state::Ipv4Range;
|
||||
use tsunagi::testing::{config_with, network, settle, wait_event, wait_for_peers, wait_until};
|
||||
use tsunagi::{Agent, NetworkStatus};
|
||||
use tsunagi_wg_quic::{
|
||||
WIREGUARD_PROTOCOL, WgAnnouncement, WgSecretKey, WireguardConfig, WireguardPlugin,
|
||||
};
|
||||
|
||||
/// An agent with a WireGuard plugin backed by an in-memory packet interface.
|
||||
struct WgAgent {
|
||||
@@ -106,15 +105,15 @@ impl WgAgent {
|
||||
let tuns = MemoryTunFactory::new();
|
||||
let wg = tune(
|
||||
WireguardConfig::new(root.join("wireguard"))
|
||||
.with_interface_prefix(tag)
|
||||
.with_reconcile(Duration::from_millis(20), Duration::from_millis(250)),
|
||||
);
|
||||
let plugin = WireguardPlugin::open(wg, Arc::new(tuns.clone()))
|
||||
.await
|
||||
.unwrap();
|
||||
let plugin = WireguardPlugin::open(wg).await.unwrap();
|
||||
// The interface belongs to the agent now, so the tag names it
|
||||
// directly rather than prefixing one per network.
|
||||
let agent = Agent::spawn(
|
||||
config_with(root, discovery)
|
||||
.with_overlay_ipv4_range(range)
|
||||
.with_interface(Arc::new(tuns.clone()), tag, 1280)
|
||||
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
|
||||
)
|
||||
.await
|
||||
@@ -127,25 +126,26 @@ impl WgAgent {
|
||||
}
|
||||
|
||||
/// This agent's overlay address in a network.
|
||||
async fn overlay(&self, network: NetworkId) -> Ipv6Addr {
|
||||
let view = wait_until("the plugin prepared the network", || async {
|
||||
self.plugin.overview(network)
|
||||
///
|
||||
/// Allocated and signed at the system level, so it appears once the
|
||||
/// network has agreed on it rather than the moment the plugin starts.
|
||||
async fn overlay(&self, network: NetworkId) -> Ipv4Addr {
|
||||
wait_until("the network agreed an overlay address", || async {
|
||||
self.plugin.overview(network)?.overlay_address_v4
|
||||
})
|
||||
.await;
|
||||
match view.overlay_address {
|
||||
IpAddr::V6(addr) => addr,
|
||||
IpAddr::V4(_) => panic!("the overlay is IPv6"),
|
||||
}
|
||||
.await
|
||||
}
|
||||
|
||||
/// The in-memory packet interface for a network.
|
||||
async fn tun(&self, network: NetworkId) -> Arc<MemoryTun> {
|
||||
/// The one in-memory packet interface this agent owns.
|
||||
///
|
||||
/// Not per network: one agent has one interface, and which network a
|
||||
/// packet on it belongs to is decided by its address.
|
||||
async fn tun(&self, _network: NetworkId) -> Arc<MemoryTun> {
|
||||
let name = wait_until("the packet interface exists", || async {
|
||||
let view = self.plugin.overview(network)?;
|
||||
self.tuns.device(&view.interface).map(|_| view.interface)
|
||||
self.agent.overlay().map(|overlay| overlay.interface)
|
||||
})
|
||||
.await;
|
||||
self.tuns.device(&name).unwrap()
|
||||
self.tuns.device(&name).expect("the device was created")
|
||||
}
|
||||
|
||||
/// Waits until `count` tunnels have completed a WireGuard handshake.
|
||||
@@ -184,20 +184,6 @@ fn ipv4_packet(source: Ipv4Addr, destination: Ipv4Addr, payload: &[u8]) -> Bytes
|
||||
Bytes::from(packet)
|
||||
}
|
||||
|
||||
/// Builds a minimal well-formed IPv6 packet.
|
||||
fn ipv6_packet(source: Ipv6Addr, destination: Ipv6Addr, payload: &[u8]) -> Bytes {
|
||||
let mut packet = Vec::with_capacity(40 + payload.len());
|
||||
packet.push(6 << 4); // version 6
|
||||
packet.extend_from_slice(&[0, 0, 0]); // traffic class and flow label
|
||||
packet.extend_from_slice(&(payload.len() as u16).to_be_bytes());
|
||||
packet.push(59); // "no next header"
|
||||
packet.push(64); // hop limit
|
||||
packet.extend_from_slice(&source.octets());
|
||||
packet.extend_from_slice(&destination.octets());
|
||||
packet.extend_from_slice(payload);
|
||||
Bytes::from(packet)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn two_agents_carry_real_ip_packets_through_a_wireguard_tunnel() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
@@ -217,12 +203,10 @@ async fn two_agents_carry_real_ip_packets_through_a_wireguard_tunnel() {
|
||||
let addr_a = a.overlay(network_id).await;
|
||||
let addr_b = b.overlay(network_id).await;
|
||||
assert_ne!(addr_a, addr_b);
|
||||
// One shared /64, derived by both sides independently.
|
||||
assert_eq!(addr_a.octets()[0..8], addr_b.octets()[0..8]);
|
||||
assert_eq!(
|
||||
&overlay_prefix(network_id).octets()[0..8],
|
||||
&addr_a.octets()[0..8]
|
||||
);
|
||||
// One shared range, agreed at the system level and signed by each
|
||||
// member, not derived from anything either protocol holds.
|
||||
let range = a.plugin.overview(network_id).unwrap().ipv4_range.unwrap();
|
||||
assert!(range.contains(addr_a) && range.contains(addr_b));
|
||||
|
||||
let tun_a = a.tun(network_id).await;
|
||||
let tun_b = b.tun(network_id).await;
|
||||
@@ -230,23 +214,23 @@ async fn two_agents_carry_real_ip_packets_through_a_wireguard_tunnel() {
|
||||
// A real IP packet, encrypted by WireGuard, carried over iroh, decrypted
|
||||
// on the other side and handed to that host's packet interface.
|
||||
let payload = b"hello over the overlay";
|
||||
tun_a.push_from_os(ipv6_packet(addr_a, addr_b, payload));
|
||||
tun_a.push_from_os(ipv4_packet(addr_a, addr_b, payload));
|
||||
|
||||
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
|
||||
let received = tokio::time::timeout(tsunagi::testing::DEADLINE, tun_b.pop_to_os())
|
||||
.await
|
||||
.expect("the packet should arrive")
|
||||
.expect("the interface should still be open");
|
||||
assert_eq!(&received[40..], payload);
|
||||
assert_eq!(&received[8..24], &addr_a.octets(), "source preserved");
|
||||
assert_eq!(&received[24..40], &addr_b.octets(), "destination preserved");
|
||||
assert_eq!(&received[20..], payload);
|
||||
assert_eq!(&received[12..16], &addr_a.octets(), "source preserved");
|
||||
assert_eq!(&received[16..20], &addr_b.octets(), "destination preserved");
|
||||
|
||||
// And back the other way.
|
||||
tun_b.push_from_os(ipv6_packet(addr_b, addr_a, b"and back"));
|
||||
let back = tokio::time::timeout(common::DEADLINE, tun_a.pop_to_os())
|
||||
tun_b.push_from_os(ipv4_packet(addr_b, addr_a, b"and back"));
|
||||
let back = tokio::time::timeout(tsunagi::testing::DEADLINE, tun_a.pop_to_os())
|
||||
.await
|
||||
.expect("the reply should arrive")
|
||||
.unwrap();
|
||||
assert_eq!(&back[40..], b"and back");
|
||||
assert_eq!(&back[20..], b"and back");
|
||||
|
||||
let view = a.plugin.overview(network_id).unwrap();
|
||||
let tunnel = view.peers[0].tunnel.as_ref().unwrap();
|
||||
@@ -278,35 +262,36 @@ async fn a_peer_cannot_send_from_an_address_it_does_not_own() {
|
||||
let tun_b = b.tun(network_id).await;
|
||||
|
||||
// A sends a packet claiming to come from a third party's address.
|
||||
let someone_else: Ipv6Addr = {
|
||||
let someone_else = {
|
||||
let mut octets = addr_a.octets();
|
||||
octets[15] ^= 0xff;
|
||||
Ipv6Addr::from(octets)
|
||||
octets[3] ^= 0xff;
|
||||
Ipv4Addr::from(octets)
|
||||
};
|
||||
tun_a.push_from_os(ipv6_packet(someone_else, addr_b, b"spoofed"));
|
||||
tun_a.push_from_os(ipv4_packet(someone_else, addr_b, b"spoofed"));
|
||||
|
||||
// B must drop it: the source is not the address derived for A's key.
|
||||
// B must drop it: the source is not the address A holds.
|
||||
// Counted on the interface, not on the tunnel: the protocol proved who
|
||||
// sent the packet, and whether that member may use the address it chose
|
||||
// is a question about a signed claim, which the system level holds.
|
||||
wait_until("the spoofed packet is dropped", || async {
|
||||
let view = b.plugin.overview(network_id)?;
|
||||
let tunnel = view.peers.first()?.tunnel.as_ref()?;
|
||||
(tunnel.stats.dropped_wrong_source >= 1).then_some(())
|
||||
(b.agent.overlay()?.counters.wrong_source >= 1).then_some(())
|
||||
})
|
||||
.await;
|
||||
|
||||
// A legitimate packet still goes through, so the tunnel is not broken.
|
||||
tun_a.push_from_os(ipv6_packet(addr_a, addr_b, b"honest"));
|
||||
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
|
||||
tun_a.push_from_os(ipv4_packet(addr_a, addr_b, b"honest"));
|
||||
let received = tokio::time::timeout(tsunagi::testing::DEADLINE, tun_b.pop_to_os())
|
||||
.await
|
||||
.expect("the honest packet should arrive")
|
||||
.unwrap();
|
||||
assert_eq!(&received[40..], b"honest");
|
||||
assert_eq!(&received[20..], b"honest");
|
||||
|
||||
a.shutdown().await;
|
||||
b.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_overlay_carries_ipv4_alongside_ipv6() {
|
||||
async fn the_overlay_uses_a_range_the_network_was_told_to_use() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("wg-dual-stack");
|
||||
|
||||
@@ -332,17 +317,16 @@ async fn the_overlay_carries_ipv4_alongside_ipv6() {
|
||||
u32::from(Ipv4Addr::new(10, 77, 0, 0))
|
||||
);
|
||||
}
|
||||
// Each side derived the other's address identically.
|
||||
// Each side learned the other's address from the same signed state.
|
||||
assert_eq!(view_a.peers[0].overlay_address_v4, Some(v4_b));
|
||||
assert_eq!(view_b.peers[0].overlay_address_v4, Some(v4_a));
|
||||
assert!(!view_a.peers[0].tunnel.as_ref().unwrap().ipv4_conflict);
|
||||
|
||||
let tun_a = a.tun(network_id).await;
|
||||
let tun_b = b.tun(network_id).await;
|
||||
|
||||
// A real IPv4 packet through the same tunnel.
|
||||
tun_a.push_from_os(ipv4_packet(v4_a, v4_b, b"ipv4 over the overlay"));
|
||||
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
|
||||
let received = tokio::time::timeout(tsunagi::testing::DEADLINE, tun_b.pop_to_os())
|
||||
.await
|
||||
.expect("the IPv4 packet should arrive")
|
||||
.unwrap();
|
||||
@@ -351,17 +335,6 @@ async fn the_overlay_carries_ipv4_alongside_ipv6() {
|
||||
assert_eq!(&received[16..20], &v4_b.octets());
|
||||
assert_eq!(&received[20..], b"ipv4 over the overlay");
|
||||
|
||||
// IPv6 keeps working on the same tunnel.
|
||||
let addr_a = a.overlay(network_id).await;
|
||||
let addr_b = b.overlay(network_id).await;
|
||||
tun_a.push_from_os(ipv6_packet(addr_a, addr_b, b"and ipv6 too"));
|
||||
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
|
||||
.await
|
||||
.expect("the IPv6 packet should arrive")
|
||||
.unwrap();
|
||||
assert_eq!(received[0] >> 4, 6);
|
||||
assert_eq!(&received[40..], b"and ipv6 too");
|
||||
|
||||
a.shutdown().await;
|
||||
b.shutdown().await;
|
||||
}
|
||||
@@ -399,15 +372,13 @@ async fn an_ipv4_source_a_peer_does_not_own_is_dropped() {
|
||||
tun_a.push_from_os(ipv4_packet(forged, v4_b, b"spoofed v4"));
|
||||
|
||||
wait_until("the spoofed IPv4 packet is dropped", || async {
|
||||
let view = b.plugin.overview(network_id)?;
|
||||
let tunnel = view.peers.first()?.tunnel.as_ref()?;
|
||||
(tunnel.stats.dropped_wrong_source >= 1).then_some(())
|
||||
(b.agent.overlay()?.counters.wrong_source >= 1).then_some(())
|
||||
})
|
||||
.await;
|
||||
|
||||
// The honest one still gets through.
|
||||
tun_a.push_from_os(ipv4_packet(v4_a, v4_b, b"honest v4"));
|
||||
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
|
||||
let received = tokio::time::timeout(tsunagi::testing::DEADLINE, tun_b.pop_to_os())
|
||||
.await
|
||||
.expect("the honest packet should arrive")
|
||||
.unwrap();
|
||||
@@ -417,41 +388,6 @@ async fn an_ipv4_source_a_peer_does_not_own_is_dropped() {
|
||||
b.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_ipv6_only_overlay_can_be_asked_for() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("wg-v6-only");
|
||||
|
||||
let a = WgAgent::spawn_range(&discovery, "ta", None).await;
|
||||
let b = WgAgent::spawn_range(&discovery, "tb", None).await;
|
||||
|
||||
let network_id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
a.wait_for_tunnels(network_id, 1).await;
|
||||
|
||||
let view = a.plugin.overview(network_id).unwrap();
|
||||
assert_eq!(view.overlay_address_v4, None);
|
||||
assert_eq!(view.ipv4_range, None);
|
||||
assert_eq!(view.peers[0].overlay_address_v4, None);
|
||||
// No IPv4 configured is not a conflict.
|
||||
assert!(!view.peers[0].tunnel.as_ref().unwrap().ipv4_conflict);
|
||||
|
||||
// IPv6 is unaffected.
|
||||
let addr_a = a.overlay(network_id).await;
|
||||
let addr_b = b.overlay(network_id).await;
|
||||
a.tun(network_id)
|
||||
.await
|
||||
.push_from_os(ipv6_packet(addr_a, addr_b, b"v6 only"));
|
||||
let received = tokio::time::timeout(common::DEADLINE, b.tun(network_id).await.pop_to_os())
|
||||
.await
|
||||
.expect("the packet should arrive")
|
||||
.unwrap();
|
||||
assert_eq!(&received[40..], b"v6 only");
|
||||
|
||||
a.shutdown().await;
|
||||
b.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_joining_member_adopts_the_range_the_network_already_uses() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
@@ -537,10 +473,10 @@ async fn an_allocated_address_missing_from_the_host_is_reported() {
|
||||
reason.contains(&allocated.to_string()),
|
||||
"unexpected: {reason}"
|
||||
);
|
||||
// The agent assigns the address itself, so the report says which
|
||||
// interface should have had it rather than a command to run.
|
||||
// The agent owns the interface, so it is the agent that notices and the
|
||||
// report names the interface the address should have been on.
|
||||
assert!(
|
||||
reason.contains(&a.plugin.overview(network_id).unwrap().interface),
|
||||
reason.contains(&a.agent.overlay().unwrap().interface),
|
||||
"must name the interface: {reason}"
|
||||
);
|
||||
|
||||
@@ -670,12 +606,13 @@ async fn packets_for_an_unknown_address_are_counted_not_broadcast() {
|
||||
let tun_b = b.tun(network_id).await;
|
||||
|
||||
// Nobody owns this address, so it must not be sent to anybody.
|
||||
let nowhere: Ipv6Addr = "fd00:dead:beef::1".parse().unwrap();
|
||||
tun_a.push_from_os(ipv6_packet(addr_a, nowhere, b"lost"));
|
||||
let nowhere: Ipv4Addr = "192.0.2.111".parse().unwrap();
|
||||
tun_a.push_from_os(ipv4_packet(addr_a, nowhere, b"lost"));
|
||||
|
||||
wait_until("the packet is counted as unroutable", || async {
|
||||
let view = a.plugin.overview(network_id)?;
|
||||
(view.unroutable_packets >= 1).then_some(())
|
||||
let _ = view;
|
||||
(a.agent.overlay()?.counters.unroutable >= 1).then_some(())
|
||||
})
|
||||
.await;
|
||||
|
||||
@@ -714,13 +651,12 @@ async fn a_mesh_of_three_establishes_every_tunnel() {
|
||||
let mut addresses = Vec::new();
|
||||
for agent in [&a, &b, &c] {
|
||||
let view = agent.plugin.overview(network_id).unwrap();
|
||||
assert_eq!(view.overlay_prefix, IpAddr::V6(overlay_prefix(network_id)));
|
||||
assert!(
|
||||
view.peers
|
||||
.iter()
|
||||
.all(|peer| peer.public_key != view.public_key)
|
||||
);
|
||||
addresses.push(view.overlay_address);
|
||||
addresses.push(agent.overlay(network_id).await);
|
||||
}
|
||||
addresses.sort();
|
||||
addresses.dedup();
|
||||
@@ -731,12 +667,15 @@ async fn a_mesh_of_three_establishes_every_tunnel() {
|
||||
let addr_c = c.overlay(network_id).await;
|
||||
a.tun(network_id)
|
||||
.await
|
||||
.push_from_os(ipv6_packet(addr_a, addr_c, b"a to c"));
|
||||
let received = tokio::time::timeout(common::DEADLINE, c.tun(network_id).await.pop_to_os())
|
||||
.await
|
||||
.expect("the packet should arrive")
|
||||
.unwrap();
|
||||
assert_eq!(&received[40..], b"a to c");
|
||||
.push_from_os(ipv4_packet(addr_a, addr_c, b"a to c"));
|
||||
let received = tokio::time::timeout(
|
||||
tsunagi::testing::DEADLINE,
|
||||
c.tun(network_id).await.pop_to_os(),
|
||||
)
|
||||
.await
|
||||
.expect("the packet should arrive")
|
||||
.unwrap();
|
||||
assert_eq!(&received[20..], b"a to c");
|
||||
|
||||
a.shutdown().await;
|
||||
b.shutdown().await;
|
||||
@@ -777,47 +716,73 @@ async fn a_departing_peer_loses_its_tunnel() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn two_networks_get_separate_interfaces_keys_and_overlays() {
|
||||
async fn two_networks_share_one_interface_with_keys_and_ranges_of_their_own() {
|
||||
// One agent, one interface — so two networks on it must use different
|
||||
// ranges, or an address would belong to both. Each network's range is
|
||||
// settled by whoever got there first, and the agent adopts what it
|
||||
// finds; see the routing table's own tests for the refusal when they
|
||||
// overlap.
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name_a, secret_a) = network("wg-left");
|
||||
let (name_b, secret_b) = network("wg-right");
|
||||
let beta_range = Some("10.99.0.0/16".parse::<Ipv4Range>().unwrap());
|
||||
|
||||
let hub = WgAgent::spawn(&discovery, "th").await;
|
||||
let left = WgAgent::spawn(&discovery, "tl").await;
|
||||
let right = WgAgent::spawn(&discovery, "tr").await;
|
||||
let right = WgAgent::spawn_range(&discovery, "tr", beta_range).await;
|
||||
|
||||
let alpha = hub.agent.join_network(&name_a, &secret_a).await.unwrap();
|
||||
let beta = hub.agent.join_network(&name_b, &secret_b).await.unwrap();
|
||||
left.agent.join_network(&name_a, &secret_a).await.unwrap();
|
||||
right.agent.join_network(&name_b, &secret_b).await.unwrap();
|
||||
// The other members settle each range before the hub joins, so it has
|
||||
// something to adopt rather than a default to collide with.
|
||||
let alpha = left.agent.join_network(&name_a, &secret_a).await.unwrap();
|
||||
let beta = right.agent.join_network(&name_b, &secret_b).await.unwrap();
|
||||
left.overlay(alpha).await;
|
||||
right.overlay(beta).await;
|
||||
|
||||
hub.agent.join_network(&name_a, &secret_a).await.unwrap();
|
||||
hub.agent.join_network(&name_b, &secret_b).await.unwrap();
|
||||
hub.wait_for_tunnels(alpha, 1).await;
|
||||
hub.wait_for_tunnels(beta, 1).await;
|
||||
|
||||
let view_alpha = hub.plugin.overview(alpha).unwrap();
|
||||
let view_beta = hub.plugin.overview(beta).unwrap();
|
||||
assert_ne!(view_alpha.interface, view_beta.interface);
|
||||
assert_ne!(
|
||||
view_alpha.public_key, view_beta.public_key,
|
||||
hub.plugin.overview(alpha).unwrap().public_key,
|
||||
hub.plugin.overview(beta).unwrap().public_key,
|
||||
"one WireGuard identity per network, not one per host"
|
||||
);
|
||||
assert_ne!(view_alpha.overlay_prefix, view_beta.overlay_prefix);
|
||||
assert_eq!(hub.tuns.devices().len(), 2);
|
||||
assert_eq!(
|
||||
hub.tuns.devices().len(),
|
||||
1,
|
||||
"one agent has one interface, whatever it is a member of"
|
||||
);
|
||||
|
||||
// Traffic in one overlay never surfaces in the other.
|
||||
let hub_alpha = match view_alpha.overlay_address {
|
||||
IpAddr::V6(addr) => addr,
|
||||
IpAddr::V4(_) => panic!("ipv6"),
|
||||
};
|
||||
// Each network's address comes from its own range.
|
||||
let hub_alpha = wait_until("the hub settles into alpha's range", || async {
|
||||
let address = hub.plugin.overview(alpha)?.overlay_address_v4?;
|
||||
tsunagi::state::DEFAULT_IPV4_RANGE
|
||||
.contains(address)
|
||||
.then_some(address)
|
||||
})
|
||||
.await;
|
||||
let hub_beta = wait_until("the hub settles into beta's range", || async {
|
||||
let address = hub.plugin.overview(beta)?.overlay_address_v4?;
|
||||
beta_range?.contains(address).then_some(address)
|
||||
})
|
||||
.await;
|
||||
assert_ne!(hub_alpha, hub_beta);
|
||||
|
||||
// Traffic in one overlay never surfaces in the other, though both cross
|
||||
// the same interface.
|
||||
let left_addr = left.overlay(alpha).await;
|
||||
hub.tun(alpha)
|
||||
.await
|
||||
.push_from_os(ipv6_packet(hub_alpha, left_addr, b"alpha only"));
|
||||
let seen = tokio::time::timeout(common::DEADLINE, left.tun(alpha).await.pop_to_os())
|
||||
.await
|
||||
.expect("the packet should arrive")
|
||||
.unwrap();
|
||||
assert_eq!(&seen[40..], b"alpha only");
|
||||
.push_from_os(ipv4_packet(hub_alpha, left_addr, b"alpha only"));
|
||||
let seen = tokio::time::timeout(
|
||||
tsunagi::testing::DEADLINE,
|
||||
left.tun(alpha).await.pop_to_os(),
|
||||
)
|
||||
.await
|
||||
.expect("the packet should arrive")
|
||||
.unwrap();
|
||||
assert_eq!(&seen[20..], b"alpha only");
|
||||
assert!(
|
||||
tokio::time::timeout(
|
||||
Duration::from_millis(200),
|
||||
@@ -828,14 +793,6 @@ async fn two_networks_get_separate_interfaces_keys_and_overlays() {
|
||||
"the other overlay must see nothing"
|
||||
);
|
||||
|
||||
// Deactivating one network removes only its interface.
|
||||
hub.agent.deactivate_network(alpha).await.unwrap();
|
||||
wait_until("the alpha interface is gone", || async {
|
||||
hub.plugin.overview(alpha).is_none().then_some(())
|
||||
})
|
||||
.await;
|
||||
assert!(hub.plugin.overview(beta).is_some());
|
||||
|
||||
hub.shutdown().await;
|
||||
left.shutdown().await;
|
||||
right.shutdown().await;
|
||||
@@ -863,8 +820,6 @@ async fn restarting_keeps_the_wireguard_identity_and_overlay_address() {
|
||||
})
|
||||
.await;
|
||||
assert_eq!(after.public_key, before.public_key);
|
||||
assert_eq!(after.overlay_address, before.overlay_address);
|
||||
assert_eq!(after.interface, before.interface);
|
||||
|
||||
// The tunnel comes back on its own.
|
||||
wait_until("the tunnel is re-established", || async {
|
||||
@@ -930,20 +885,25 @@ async fn the_core_carries_the_payload_without_interpreting_it() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_forged_overlay_claim_is_rejected_and_never_reaches_a_tunnel() {
|
||||
async fn an_announcement_for_another_network_never_reaches_a_tunnel() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let name = NetworkName::new("wg-hijack").unwrap();
|
||||
let secret = NetworkSecret::generate();
|
||||
|
||||
let victim = WgAgent::spawn(&discovery, "tv").await;
|
||||
let network_id = victim.agent.join_network(&name, &secret).await.unwrap();
|
||||
let victim_address = victim.overlay(network_id).await;
|
||||
|
||||
// A legitimate member — it knows the secret — claims the victim's overlay
|
||||
// address with its own WireGuard key.
|
||||
// Addresses are not in an announcement any more, so there is no address
|
||||
// to forge. What is left to lie about is which network the key is for —
|
||||
// and a capability is scoped to the session it arrived on, so claiming
|
||||
// another network's is the shape a confused or hostile member takes.
|
||||
let attacker_key = WgSecretKey::generate().public();
|
||||
let mut forged = WgAnnouncement::new(network_id, &attacker_key);
|
||||
forged.overlay_address = victim_address;
|
||||
let elsewhere = tsunagi::identity::NetworkKeys::derive(
|
||||
&NetworkName::new("somewhere-else").unwrap(),
|
||||
&NetworkSecret::generate(),
|
||||
)
|
||||
.network_id();
|
||||
let forged = WgAnnouncement::new(elsewhere, &attacker_key);
|
||||
|
||||
let forger = Arc::new(ForgingPlugin {
|
||||
payload: std::sync::Mutex::new(Some(forged.encode().unwrap())),
|
||||
@@ -960,14 +920,18 @@ async fn a_forged_overlay_claim_is_rejected_and_never_reaches_a_tunnel() {
|
||||
attacker.join_network(&name, &secret).await.unwrap();
|
||||
wait_for_peers(&victim.agent, network_id, 1).await;
|
||||
|
||||
// Filtered on the reason: this agent reports other things too, and the
|
||||
// first plugin error to arrive is not necessarily this one.
|
||||
let reason = wait_event(&mut events, |event| match event {
|
||||
Event::PluginError {
|
||||
protocol, reason, ..
|
||||
} if protocol == WIREGUARD_PROTOCOL => Some(reason.clone()),
|
||||
} if protocol == WIREGUARD_PROTOCOL && reason.contains("different network") => {
|
||||
Some(reason.clone())
|
||||
}
|
||||
_ => None,
|
||||
})
|
||||
.await;
|
||||
assert!(reason.contains("does not match"), "unexpected: {reason}");
|
||||
assert!(reason.contains("different network"), "unexpected: {reason}");
|
||||
|
||||
settle().await;
|
||||
let view = victim.plugin.overview(network_id).unwrap();
|
||||
@@ -977,13 +941,160 @@ async fn a_forged_overlay_claim_is_rejected_and_never_reaches_a_tunnel() {
|
||||
.all(|peer| peer.public_key != attacker_key),
|
||||
"a rejected announcement must never become a tunnel"
|
||||
);
|
||||
assert_eq!(view.overlay_address, IpAddr::V6(victim_address));
|
||||
|
||||
attacker.shutdown().await;
|
||||
victim.shutdown().await;
|
||||
drop(attacker_dir);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_peer_at_another_protocol_version_gets_no_data_plane_and_keeps_the_control_plane() {
|
||||
// Both sides must have the protocol at the same version. There is no
|
||||
// middle ground to negotiate: either the words mean the same thing at
|
||||
// both ends or they do not.
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("wg-version");
|
||||
|
||||
let here = WgAgent::spawn(&discovery, "tvh").await;
|
||||
let network_id = here.agent.join_network(&name, &secret).await.unwrap();
|
||||
|
||||
let ahead = Arc::new(FromTheFuture);
|
||||
let dir = TempDir::new().unwrap();
|
||||
let other = Agent::spawn(
|
||||
config_with(dir.path(), &discovery).with_plugin(ahead.clone() as Arc<dyn IpPlugin>),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let mut events = here.agent.subscribe();
|
||||
other.join_network(&name, &secret).await.unwrap();
|
||||
wait_for_peers(&here.agent, network_id, 1).await;
|
||||
|
||||
let reason = wait_event(&mut events, |event| match event {
|
||||
Event::PluginError { reason, .. } if reason.contains("version") => Some(reason.clone()),
|
||||
_ => None,
|
||||
})
|
||||
.await;
|
||||
assert!(
|
||||
reason.contains("control plane is unaffected"),
|
||||
"the message should say what still works: {reason}"
|
||||
);
|
||||
|
||||
// No tunnel, and no attempt at one.
|
||||
settle().await;
|
||||
assert!(
|
||||
here.plugin.overview(network_id).unwrap().peers.is_empty(),
|
||||
"a version that cannot match must not become a tunnel"
|
||||
);
|
||||
let status = here.agent.network_status(network_id).await.unwrap();
|
||||
assert_eq!(status.peers.len(), 1, "the session is up all the same");
|
||||
assert!(
|
||||
status.peers[0].protocols.is_empty(),
|
||||
"nothing was agreed with it: {:?}",
|
||||
status.peers[0].protocols
|
||||
);
|
||||
|
||||
// And the control plane carries a message to it regardless.
|
||||
assert_eq!(
|
||||
here.agent
|
||||
.broadcast(
|
||||
network_id,
|
||||
tsunagi::proto::ControlMessage::Ping {
|
||||
seq: 1,
|
||||
payload: b"still talking".to_vec(),
|
||||
},
|
||||
)
|
||||
.await
|
||||
.unwrap(),
|
||||
1
|
||||
);
|
||||
|
||||
other.shutdown().await;
|
||||
here.shutdown().await;
|
||||
drop(dir);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn agreement_turns_on_the_protocol_version_and_nothing_else() {
|
||||
// A different build is not a different protocol. What is compared is the
|
||||
// wire version and the name; everything else about a peer — its release,
|
||||
// and whatever opaque payload its announcement carries — has no say.
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("wg-same-wire");
|
||||
|
||||
let here = WgAgent::spawn(&discovery, "tsw").await;
|
||||
let network_id = here.agent.join_network(&name, &secret).await.unwrap();
|
||||
|
||||
// Announces the right protocol at the right version, with a payload
|
||||
// this build has never seen.
|
||||
let stranger = Arc::new(ForgingPlugin {
|
||||
payload: std::sync::Mutex::new(Some(b"from some other release".to_vec())),
|
||||
});
|
||||
let dir = TempDir::new().unwrap();
|
||||
let other = Agent::spawn(
|
||||
config_with(dir.path(), &discovery).with_plugin(stranger.clone() as Arc<dyn IpPlugin>),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
other.join_network(&name, &secret).await.unwrap();
|
||||
wait_for_peers(&here.agent, network_id, 1).await;
|
||||
|
||||
// Agreed, because the wire matches. The payload is rejected by the
|
||||
// protocol afterwards, which is a separate matter and says nothing about
|
||||
// whether the two agreed to talk.
|
||||
wait_until("the protocol is agreed with it", || async {
|
||||
let status = here.agent.network_status(network_id).await.ok()?;
|
||||
status
|
||||
.peers
|
||||
.first()?
|
||||
.protocols
|
||||
.contains(&WIREGUARD_PROTOCOL.to_string())
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
|
||||
other.shutdown().await;
|
||||
here.shutdown().await;
|
||||
drop(dir);
|
||||
}
|
||||
|
||||
/// A plugin claiming the same protocol at a version this build does not speak.
|
||||
#[derive(Debug)]
|
||||
struct FromTheFuture;
|
||||
|
||||
impl IpPlugin for FromTheFuture {
|
||||
fn protocol_id(&self) -> &str {
|
||||
WIREGUARD_PROTOCOL
|
||||
}
|
||||
|
||||
fn protocol_version(&self) -> u16 {
|
||||
tsunagi_wg_quic::ANNOUNCEMENT_VERSION + 1
|
||||
}
|
||||
|
||||
fn local_capability(
|
||||
&self,
|
||||
_network: NetworkId,
|
||||
) -> Result<Option<tsunagi::dataplane::PluginCapability>, tsunagi::dataplane::PluginError> {
|
||||
Ok(Some(tsunagi::dataplane::PluginCapability {
|
||||
protocol: WIREGUARD_PROTOCOL.to_string(),
|
||||
version: self.protocol_version(),
|
||||
enabled: true,
|
||||
data: Vec::new(),
|
||||
}))
|
||||
}
|
||||
|
||||
fn on_peer_capability(
|
||||
&self,
|
||||
_network: NetworkId,
|
||||
_peer: EndpointId,
|
||||
_capability: &tsunagi::dataplane::PluginCapability,
|
||||
) -> Result<(), tsunagi::dataplane::PluginError> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn on_peer_gone(&self, _network: NetworkId, _peer: EndpointId) {}
|
||||
fn on_network_deactivated(&self, _network: NetworkId) {}
|
||||
}
|
||||
|
||||
/// A plugin that announces whatever bytes it is told to, under the WireGuard
|
||||
/// protocol id. Used to test what a hostile member can do.
|
||||
#[derive(Debug)]
|
||||
@@ -996,6 +1107,12 @@ impl IpPlugin for ForgingPlugin {
|
||||
WIREGUARD_PROTOCOL
|
||||
}
|
||||
|
||||
/// The same version, so the announcement is looked at rather than
|
||||
/// dismissed for the wrong reason.
|
||||
fn protocol_version(&self) -> u16 {
|
||||
tsunagi_wg_quic::ANNOUNCEMENT_VERSION
|
||||
}
|
||||
|
||||
fn local_capability(
|
||||
&self,
|
||||
_network: NetworkId,
|
||||
@@ -1006,7 +1123,9 @@ impl IpPlugin for ForgingPlugin {
|
||||
};
|
||||
Ok(payload.map(|data| tsunagi::dataplane::PluginCapability {
|
||||
protocol: WIREGUARD_PROTOCOL.to_string(),
|
||||
version: 1,
|
||||
// The version it actually speaks, so the payload is examined
|
||||
// rather than set aside for the wrong reason.
|
||||
version: tsunagi_wg_quic::ANNOUNCEMENT_VERSION,
|
||||
enabled: true,
|
||||
data,
|
||||
}))
|
||||
@@ -1026,49 +1145,31 @@ impl IpPlugin for ForgingPlugin {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_mtu_below_the_ipv6_minimum_is_refused() {
|
||||
use tsunagi::dataplane::wireguard::{DEFAULT_MTU, MIN_MTU, WIREGUARD_OVERHEAD};
|
||||
async fn an_mtu_below_what_ipv4_guarantees_is_refused() {
|
||||
use tsunagi_wg_quic::{DEFAULT_MTU, MIN_MTU, WIREGUARD_OVERHEAD};
|
||||
|
||||
// Linux disables IPv6 outright on an interface below 1280 bytes, so the
|
||||
// overlay address could never be assigned. Catch it here rather than as
|
||||
// an obscure RTNETLINK error much later.
|
||||
// 576 bytes is what every IPv4 host must be able to reassemble, so
|
||||
// nothing below it is worth offering. The floor used to be 1280 for
|
||||
// IPv6's sake; the overlay is IPv4 now and a relayed path with small
|
||||
// datagrams can be matched instead of warned about.
|
||||
let dir = TempDir::new().unwrap();
|
||||
let result = WireguardPlugin::open(
|
||||
WireguardConfig::new(dir.path()).with_mtu(MIN_MTU - 1),
|
||||
Arc::new(MemoryTunFactory::new()),
|
||||
)
|
||||
.await;
|
||||
let result =
|
||||
WireguardPlugin::open(WireguardConfig::new(dir.path()).with_mtu(MIN_MTU - 1)).await;
|
||||
match result {
|
||||
Err(err) => {
|
||||
let text = err.to_string();
|
||||
assert!(text.contains("1280"), "unexpected message: {text}");
|
||||
assert!(text.contains("IPv6"), "unexpected message: {text}");
|
||||
assert!(text.contains("576"), "unexpected message: {text}");
|
||||
}
|
||||
Ok(_) => panic!("an MTU below the IPv6 minimum must be refused"),
|
||||
Ok(_) => panic!("an MTU below what IPv4 guarantees must be refused"),
|
||||
}
|
||||
|
||||
// The default is exactly the minimum, and a link has to carry it plus
|
||||
// WireGuard's own overhead.
|
||||
assert_eq!(DEFAULT_MTU, MIN_MTU);
|
||||
// The default leaves room below it now, which is the point of lowering
|
||||
// the floor, and a link has to carry it plus WireGuard's own overhead.
|
||||
const { assert!(DEFAULT_MTU > MIN_MTU) };
|
||||
assert_eq!(WIREGUARD_OVERHEAD, 32);
|
||||
assert!(
|
||||
WireguardPlugin::open(
|
||||
WireguardConfig::new(dir.path().join("ok")),
|
||||
Arc::new(MemoryTunFactory::new()),
|
||||
)
|
||||
.await
|
||||
.is_ok()
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_overlay_address_is_derived_from_the_key_alone() {
|
||||
let (name, secret) = network("wg-derivation");
|
||||
let id = tsunagi::identity::NetworkKeys::derive(&name, &secret).network_id();
|
||||
let key = WgSecretKey::generate().public();
|
||||
assert_eq!(overlay_address(id, &key), overlay_address(id, &key));
|
||||
assert_ne!(
|
||||
overlay_address(id, &key),
|
||||
overlay_address(id, &WgSecretKey::generate().public())
|
||||
WireguardPlugin::open(WireguardConfig::new(dir.path().join("ok")))
|
||||
.await
|
||||
.is_ok()
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
[package]
|
||||
name = "tsunagi"
|
||||
version = "0.1.0"
|
||||
description = "Small private mesh networks: persistent agent identity, deterministic network spaces, an iroh control plane, signed state and a local DNS view."
|
||||
readme = "../../README.md"
|
||||
keywords = ["mesh", "p2p", "iroh", "networking"]
|
||||
categories = ["network-programming"]
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
repository.workspace = true
|
||||
|
||||
[features]
|
||||
default = []
|
||||
# A real TUN device, so a plugin can carry actual IP traffic. Needs
|
||||
# CAP_NET_ADMIN at run time; without it the in-memory device serves tests.
|
||||
tun-device = ["dep:tun", "dep:rtnetlink", "dep:caps", "dep:futures-util"]
|
||||
# Telling the operating system where to send its DNS questions. Linux only
|
||||
# for now; the zone and the server work without it.
|
||||
dns-publish = ["dep:zbus"]
|
||||
# The test harness in `testing`, for this crate's own tests and for a
|
||||
# protocol crate's. Not on by default: it is only of use to a test.
|
||||
testing = ["dep:tempfile", "dep:tracing-subscriber"]
|
||||
|
||||
[dependencies]
|
||||
iroh.workspace = true
|
||||
tokio.workspace = true
|
||||
serde.workspace = true
|
||||
postcard.workspace = true
|
||||
bytes.workspace = true
|
||||
thiserror.workspace = true
|
||||
tracing.workspace = true
|
||||
hex.workspace = true
|
||||
data-encoding.workspace = true
|
||||
rusqlite = { version = "0.40", features = ["bundled"] }
|
||||
hkdf = "0.13"
|
||||
hmac = "0.13"
|
||||
sha2 = "0.11"
|
||||
subtle = "2.6"
|
||||
zeroize = { version = "1.9", features = ["derive"] }
|
||||
rand = "0.10"
|
||||
# Already in the tree through iroh. A packet codec, not a DNS server: the
|
||||
# zone logic is ours and a full server framework would be a large dependency
|
||||
# for answering A records from memory.
|
||||
simple-dns = "0.12"
|
||||
# The IANA top-level domain list, compiled in: one function, no
|
||||
# dependencies, no network. Used only to warn that a zone name shadows a
|
||||
# real public domain, never to refuse one.
|
||||
tld = "2.40"
|
||||
fs4 = { version = "1.1", features = ["sync"] }
|
||||
directories = "6.0"
|
||||
# The real system hostname, without a libc call of our own: this crate is
|
||||
# forbidden `unsafe` and will not make one.
|
||||
gethostname = "1.1"
|
||||
netwatch = "0.19.3"
|
||||
tun = { version = "0.8", features = ["async"], optional = true }
|
||||
# Only for the `testing` harness, which a protocol crate's tests use too.
|
||||
tempfile = { version = "3.24", optional = true }
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"], optional = true }
|
||||
|
||||
# Linux-only interface provisioning. `rtnetlink` configures the interface in
|
||||
# process, so no `ip` invocation is ever needed; `caps` keeps CAP_NET_ADMIN
|
||||
# out of the effective set except during the moments it is used.
|
||||
[target.'cfg(target_os = "linux")'.dependencies]
|
||||
rtnetlink = { version = "0.23", optional = true }
|
||||
# Pure Rust D-Bus, no libdbus to link against. `tokio` rather than the
|
||||
# default reactor, because the agent brings its own.
|
||||
zbus = { version = "5.19", default-features = false, features = ["tokio"], optional = true }
|
||||
caps = { version = "0.5", optional = true }
|
||||
futures-util = { version = "0.3", default-features = false, optional = true }
|
||||
|
||||
[dev-dependencies]
|
||||
# Its own tests use the harness it publishes.
|
||||
tsunagi = { path = ".", features = ["testing"] }
|
||||
tokio = { version = "1.53", features = ["rt", "rt-multi-thread", "sync", "time", "macros", "process"] }
|
||||
tempfile.workspace = true
|
||||
tracing-subscriber.workspace = true
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
File renamed without changes.
@@ -27,7 +27,7 @@ mod status;
|
||||
pub use events::Event;
|
||||
pub use status::{
|
||||
AgentStatus, CandidateStatus, MemberStatus, NetworkMetrics, NetworkState, NetworkStatus,
|
||||
PeerStatus,
|
||||
OverlayStatus, PeerStatus,
|
||||
};
|
||||
|
||||
use std::collections::HashMap;
|
||||
@@ -37,13 +37,15 @@ use iroh::{EndpointAddr, EndpointId};
|
||||
use tokio::sync::{RwLock, broadcast, mpsc, oneshot};
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::config::{AgentConfig, Limits};
|
||||
use crate::dataplane::transport::PacketTransport;
|
||||
use crate::dataplane::transport::iroh_link::{IrohTransport, TransportContext};
|
||||
use crate::dataplane::{PluginContext, PluginRequest};
|
||||
use crate::dataplane::{PacketSink, PluginContext, PluginRequest};
|
||||
use crate::error::{Error, Result};
|
||||
use crate::identity::{DeviceIdentity, NetworkId, NetworkKeys, NetworkName, NetworkSecret};
|
||||
use crate::net::EndpointAdapter;
|
||||
use crate::overlay::PacketCarrier;
|
||||
use crate::proto::handshake;
|
||||
use crate::proto::message::ControlMessage;
|
||||
use crate::storage::{CacheOutcome, Storage};
|
||||
@@ -87,6 +89,10 @@ struct Inner {
|
||||
accept_task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
plugin_task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
transport: std::sync::OnceLock<Arc<dyn PacketTransport>>,
|
||||
/// Who holds which overlay address, across every network.
|
||||
routes: Arc<crate::overlay::RoutingTable>,
|
||||
/// The one interface, when the agent was given a way to make one.
|
||||
interface: std::sync::OnceLock<Arc<crate::overlay::Interface>>,
|
||||
}
|
||||
|
||||
impl Inner {
|
||||
@@ -107,11 +113,69 @@ impl Inner {
|
||||
}
|
||||
}
|
||||
|
||||
/// Routes an outbound packet to whichever protocol can carry it.
|
||||
///
|
||||
/// Holds a weak reference: the interface belongs to the agent, and a strong
|
||||
/// one here would keep the agent alive for as long as its own interface.
|
||||
struct Carrier(Weak<Inner>);
|
||||
|
||||
impl std::fmt::Debug for Carrier {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str("Carrier")
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for Sink {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str("Sink")
|
||||
}
|
||||
}
|
||||
|
||||
impl PacketCarrier for Carrier {
|
||||
fn carry(&self, route: crate::overlay::Route, packet: bytes::Bytes) -> bool {
|
||||
let Some(inner) = self.0.upgrade() else {
|
||||
return false;
|
||||
};
|
||||
// Offered to each protocol in turn. With one configured this is a
|
||||
// single call; the shape is what allows several to be live at once,
|
||||
// each carrying the peers it has a link to.
|
||||
inner
|
||||
.config
|
||||
.plugins
|
||||
.iter()
|
||||
.any(|plugin| plugin.carry(route.network, route.peer, packet.clone()))
|
||||
}
|
||||
}
|
||||
|
||||
/// Writes a packet a protocol decrypted to the interface.
|
||||
struct Sink(Weak<Inner>);
|
||||
|
||||
impl PacketSink for Sink {
|
||||
fn deliver<'a>(
|
||||
&'a self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
packet: bytes::Bytes,
|
||||
) -> BoxFuture<'a, ()> {
|
||||
Box::pin(async move {
|
||||
let Some(inner) = self.0.upgrade() else {
|
||||
return;
|
||||
};
|
||||
let Some(interface) = inner.interface.get() else {
|
||||
return;
|
||||
};
|
||||
// The rejection is already counted on the interface; there is
|
||||
// nothing useful to do with it here.
|
||||
let _ = interface.deliver(network, peer, packet).await;
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Answers the data plane transport's questions about the agent.
|
||||
///
|
||||
/// Holds a weak reference on purpose: the transport lives inside the agent, so
|
||||
/// a strong one would be a cycle and the agent — with its open databases and
|
||||
/// its directory lock — would never be released.
|
||||
/// Holds a weak reference for the same reason as [`Carrier`]: the transport
|
||||
/// lives inside the agent, so a strong one would be a cycle and the agent —
|
||||
/// with its open databases and its directory lock — would never be released.
|
||||
#[derive(Debug)]
|
||||
struct TransportCtx(Weak<Inner>);
|
||||
|
||||
@@ -178,9 +242,38 @@ impl Agent {
|
||||
accept_task: std::sync::Mutex::new(None),
|
||||
plugin_task: std::sync::Mutex::new(None),
|
||||
transport: std::sync::OnceLock::new(),
|
||||
routes: Arc::new(crate::overlay::RoutingTable::new()),
|
||||
interface: std::sync::OnceLock::new(),
|
||||
config,
|
||||
});
|
||||
|
||||
// One interface for the agent, if it was given a way to make one.
|
||||
// Failing to is reported and not fatal: the control plane works, and
|
||||
// so do the protocols, they just have nowhere to put packets.
|
||||
if let Some(factory) = inner.config.tun_factory.clone() {
|
||||
let carrier = Arc::new(Carrier(Arc::downgrade(&inner))) as Arc<dyn PacketCarrier>;
|
||||
match crate::overlay::Interface::start(
|
||||
factory,
|
||||
inner.config.interface_name.clone(),
|
||||
inner.config.interface_mtu,
|
||||
Arc::clone(&inner.routes),
|
||||
carrier,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(interface) => {
|
||||
let _ = inner.interface.set(Arc::new(interface));
|
||||
}
|
||||
Err(err) => {
|
||||
let _ = inner.events.send(Event::PluginError {
|
||||
network: NetworkId::from_bytes([0u8; 32]),
|
||||
protocol: "overlay".into(),
|
||||
reason: err.to_string(),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The data plane rides on iroh too, which is where it gets hole
|
||||
// punching and relay fallback from. It is a separate ALPN and a
|
||||
// separate connection, so the two planes stay independent.
|
||||
@@ -205,7 +298,11 @@ impl Agent {
|
||||
// bound; overflow drops the request rather than stalling the plugin.
|
||||
if !inner.config.plugins.is_empty() {
|
||||
let (plugin_tx, plugin_rx) = mpsc::channel(64);
|
||||
let context = PluginContext::new(plugin_tx, inner.identity.endpoint_id());
|
||||
let sink = inner
|
||||
.interface
|
||||
.get()
|
||||
.map(|_| Arc::new(Sink(Arc::downgrade(&inner))) as Arc<dyn PacketSink>);
|
||||
let context = PluginContext::new(plugin_tx, inner.identity.endpoint_id(), sink);
|
||||
for plugin in &inner.config.plugins {
|
||||
plugin.attach(context.clone());
|
||||
}
|
||||
@@ -245,6 +342,47 @@ impl Agent {
|
||||
self.inner.adapter.loopback_addr()
|
||||
}
|
||||
|
||||
/// Reserves the configured overlay range for a network, if it can.
|
||||
///
|
||||
/// Done here, where activations are serialised, rather than inside the
|
||||
/// runtime: whether a range is free depends on what the other networks
|
||||
/// took, and deciding that in a task would make the answer depend on
|
||||
/// which task ran first.
|
||||
///
|
||||
/// `None` means this agent will not propose a range for that network and
|
||||
/// waits to adopt whatever it settles on. One agent has one interface, so
|
||||
/// proposing a range it could not route would be worse than having none:
|
||||
/// the lowest author's range wins, and the collision would spread.
|
||||
fn reserve_range(&self, network: NetworkId) -> Option<crate::state::Ipv4Range> {
|
||||
let wanted = self.inner.config.overlay_ipv4_range?;
|
||||
let reservation = crate::overlay::NetworkRoutes {
|
||||
range: Some(wanted),
|
||||
local: None,
|
||||
peers: Vec::new(),
|
||||
};
|
||||
match self.inner.routes.set_network(network, reservation) {
|
||||
Ok(()) => Some(wanted),
|
||||
Err(err) => {
|
||||
tracing::info!(%err, "not proposing a range for this network");
|
||||
None
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The overlay interface this agent owns, when it has one.
|
||||
///
|
||||
/// One agent, one interface, so this is not asked per network: a packet
|
||||
/// on it may belong to any of them.
|
||||
pub fn overlay(&self) -> Option<OverlayStatus> {
|
||||
let interface = self.inner.interface.get()?;
|
||||
Some(OverlayStatus {
|
||||
interface: interface.name().to_string(),
|
||||
mtu: interface.mtu(),
|
||||
addresses: interface.wanted_addresses(),
|
||||
counters: interface.counters(),
|
||||
})
|
||||
}
|
||||
|
||||
/// The hostname announced to peers.
|
||||
pub fn hostname(&self) -> String {
|
||||
self.inner.read_hostname()
|
||||
@@ -367,10 +505,12 @@ impl Agent {
|
||||
discovery: self.inner.config.discovery.clone(),
|
||||
discovery_interval: self.inner.config.discovery_interval,
|
||||
plugins: self.inner.config.plugins.clone(),
|
||||
routes: Arc::clone(&self.inner.routes),
|
||||
interface: self.inner.interface.get().cloned(),
|
||||
hostname: self.inner.read_hostname(),
|
||||
transport: self.inner.transport.get().cloned(),
|
||||
device_secret: self.inner.identity.signing_key(),
|
||||
ipv4_range: self.inner.config.overlay_ipv4_range,
|
||||
ipv4_range: self.reserve_range(network_id),
|
||||
});
|
||||
networks.insert(network_id, handle);
|
||||
drop(networks);
|
||||
@@ -401,6 +541,15 @@ impl Agent {
|
||||
for plugin in &self.inner.config.plugins {
|
||||
plugin.on_network_deactivated(network_id);
|
||||
}
|
||||
// The network's addresses come off the interface; the interface
|
||||
// itself stays, because the agent owns it and other networks may
|
||||
// still be using it.
|
||||
self.inner.routes.remove_network(network_id);
|
||||
if let Some(interface) = self.inner.interface.get()
|
||||
&& let Err(err) = interface.sync_addresses().await
|
||||
{
|
||||
tracing::warn!(%err, "cannot update the overlay addresses");
|
||||
}
|
||||
self.inner.storage.set_auto_start(network_id, false).await?;
|
||||
Ok(())
|
||||
}
|
||||
@@ -583,6 +732,12 @@ impl Agent {
|
||||
}
|
||||
}
|
||||
|
||||
// The interface goes with the agent that owns it. Before the
|
||||
// plugins, so nothing is still trying to write to it.
|
||||
if let Some(interface) = self.inner.interface.get() {
|
||||
interface.remove().await;
|
||||
}
|
||||
|
||||
// Plugins remove whatever system objects they created. A plugin that
|
||||
// misbehaves here must not hold up the agent, so this is bounded.
|
||||
for plugin in &self.inner.config.plugins {
|
||||
@@ -120,6 +120,10 @@ pub(crate) struct RuntimeParams {
|
||||
pub(crate) discovery: Option<Arc<dyn NetworkDiscovery>>,
|
||||
pub(crate) discovery_interval: Duration,
|
||||
pub(crate) plugins: Vec<SharedPlugin>,
|
||||
/// Who holds which overlay address, shared with every other network.
|
||||
pub(crate) routes: Arc<crate::overlay::RoutingTable>,
|
||||
/// The one interface, when the agent has one.
|
||||
pub(crate) interface: Option<Arc<crate::overlay::Interface>>,
|
||||
pub(crate) hostname: String,
|
||||
/// Signing key for this agent's own records.
|
||||
pub(crate) device_secret: iroh::SecretKey,
|
||||
@@ -206,6 +210,12 @@ struct Runtime {
|
||||
opening: HashSet<(EndpointId, String)>,
|
||||
link_results_tx: mpsc::Sender<LinkOutcome>,
|
||||
link_results_rx: mpsc::Receiver<LinkOutcome>,
|
||||
/// Peers already told about a protocol version that cannot match, so it
|
||||
/// is said once rather than on every announcement.
|
||||
reported_mismatch: HashSet<(EndpointId, String)>,
|
||||
/// The address last reported as absent from the interface, so it is said
|
||||
/// once rather than for ever.
|
||||
reported_missing: Option<std::net::Ipv4Addr>,
|
||||
/// Signed records, merged from every replica we have talked to.
|
||||
state: StateSet,
|
||||
/// Snapshots received while dispatching, handled on the next loop pass.
|
||||
@@ -238,6 +248,8 @@ impl Runtime {
|
||||
opening: HashSet::new(),
|
||||
link_results_tx,
|
||||
link_results_rx,
|
||||
reported_mismatch: HashSet::new(),
|
||||
reported_missing: None,
|
||||
state: StateSet::new(),
|
||||
pending_state: Vec::new(),
|
||||
own_version: 0,
|
||||
@@ -615,7 +627,7 @@ impl Runtime {
|
||||
.unwrap_or(0);
|
||||
|
||||
self.ensure_own_claim().await;
|
||||
self.publish_allocations();
|
||||
self.publish_allocations().await;
|
||||
}
|
||||
|
||||
/// The range this network uses: whatever it has already settled on, else
|
||||
@@ -624,7 +636,20 @@ impl Runtime {
|
||||
/// Adopting the agreed one is what lets a participant join without being
|
||||
/// told the range out of band.
|
||||
fn effective_range(&self) -> Option<Ipv4Range> {
|
||||
self.state.agreed_range().or(self.params.ipv4_range)
|
||||
if let Some(agreed) = self.state.agreed_range() {
|
||||
return Some(agreed);
|
||||
}
|
||||
// Nobody has settled one yet, so this agent would be proposing its
|
||||
// own. It must not propose a range it cannot route: one agent has
|
||||
// one interface, and claiming an address in a range another network
|
||||
// already owns would spread the collision rather than contain it —
|
||||
// "the lowest author's range wins" would carry it to everybody.
|
||||
// Better to hold off and adopt whatever the network settles on.
|
||||
let wanted = self.params.ipv4_range?;
|
||||
match self.params.routes.would_overlap(self.network_id, wanted) {
|
||||
None => Some(wanted),
|
||||
Some(_) => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Makes sure this agent holds an address, claiming one if it does not.
|
||||
@@ -810,14 +835,14 @@ impl Runtime {
|
||||
|
||||
// Somebody may have taken the address we were using.
|
||||
self.ensure_own_claim().await;
|
||||
self.publish_allocations();
|
||||
self.publish_allocations().await;
|
||||
if self.state.records() != before {
|
||||
self.broadcast_state();
|
||||
}
|
||||
}
|
||||
|
||||
/// Tells the plugins who holds which overlay address.
|
||||
fn publish_allocations(&mut self) {
|
||||
async fn publish_allocations(&mut self) {
|
||||
let Some(range) = self.effective_range() else {
|
||||
return;
|
||||
};
|
||||
@@ -832,16 +857,86 @@ impl Runtime {
|
||||
for plugin in &self.params.plugins {
|
||||
plugin.on_address_allocation(self.network_id, range, &allocations);
|
||||
}
|
||||
|
||||
// And the system level's own view, which is what decides whose
|
||||
// packet is whose. The local address is kept out of the peer list:
|
||||
// a packet for ourselves does not go over a tunnel.
|
||||
let local = self.state.address_of(&self.local_id);
|
||||
let routes = crate::overlay::NetworkRoutes {
|
||||
range: Some(range),
|
||||
local,
|
||||
peers: allocations
|
||||
.iter()
|
||||
.filter(|(holder, _)| *holder != self.local_id)
|
||||
.map(|(holder, address)| (*address, *holder))
|
||||
.collect(),
|
||||
};
|
||||
if let Err(err) = self.params.routes.set_network(self.network_id, routes) {
|
||||
// Reported once per change rather than swallowed: two networks
|
||||
// wanting the same addresses is a thing the user has to settle.
|
||||
self.metrics.plugin_errors += 1;
|
||||
self.emit(Event::PluginError {
|
||||
network: self.network_id,
|
||||
protocol: "overlay".into(),
|
||||
reason: err.to_string(),
|
||||
});
|
||||
return;
|
||||
}
|
||||
let Some(interface) = self.params.interface.clone() else {
|
||||
return;
|
||||
};
|
||||
if let Err(err) = interface.sync_addresses().await {
|
||||
self.emit(Event::PluginError {
|
||||
network: self.network_id,
|
||||
protocol: "overlay".into(),
|
||||
reason: err.to_string(),
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// The agent assigns this address itself, so finding it absent means
|
||||
// the assignment did not take — something outside removed it, or the
|
||||
// provisioner reported a success it did not achieve. Left unsaid it
|
||||
// looks like a broken network: packets would leave with the wrong
|
||||
// source and every peer would drop them. Checked rather than
|
||||
// assumed, because the assumption is exactly the kind that has been
|
||||
// wrong here before. Said once per address, not every round.
|
||||
let missing = match local {
|
||||
Some(address) if !crate::overlay::address_is_local(std::net::IpAddr::V4(address)) => {
|
||||
let already = self.reported_missing == Some(address);
|
||||
self.reported_missing = Some(address);
|
||||
(!already).then_some(address)
|
||||
}
|
||||
_ => {
|
||||
self.reported_missing = None;
|
||||
None
|
||||
}
|
||||
};
|
||||
if let Some(address) = missing {
|
||||
self.emit(Event::PluginError {
|
||||
network: self.network_id,
|
||||
protocol: "overlay".into(),
|
||||
reason: format!(
|
||||
"this agent was allocated {address}/{} but the address is not on any \
|
||||
interface, so the overlay cannot work: packets would leave with the \
|
||||
wrong source and every peer would drop them. It should have been \
|
||||
assigned to `{}` automatically; check whether something else removed \
|
||||
it.",
|
||||
range.prefix_len,
|
||||
interface.name()
|
||||
),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------ data plane
|
||||
|
||||
/// Protocol ids this agent has a plugin for.
|
||||
fn served_protocols(&self) -> Vec<String> {
|
||||
/// The protocols this agent speaks, with the version it speaks them at.
|
||||
fn served_protocols(&self) -> Vec<(String, u16)> {
|
||||
self.params
|
||||
.plugins
|
||||
.iter()
|
||||
.map(|plugin| plugin.protocol_id().to_string())
|
||||
.map(|plugin| (plugin.protocol_id().to_string(), plugin.protocol_version()))
|
||||
.collect()
|
||||
}
|
||||
|
||||
@@ -874,6 +969,10 @@ impl Runtime {
|
||||
});
|
||||
}
|
||||
|
||||
// Both the name and the version have to match. A peer offering
|
||||
// `wg-quic` at a version this build does not speak is not a peer to
|
||||
// carry traffic with, and a link opened anyway would fail later and
|
||||
// say less about why.
|
||||
let wanted: Vec<(EndpointId, String)> = self
|
||||
.sessions
|
||||
.values()
|
||||
@@ -883,9 +982,14 @@ impl Runtime {
|
||||
.capabilities
|
||||
.iter()
|
||||
.filter(|capability| capability.enabled)
|
||||
.map(move |capability| (peer, capability.protocol.clone()))
|
||||
.map(move |capability| (peer, capability.protocol.clone(), capability.version))
|
||||
})
|
||||
.filter(|(_, protocol)| served.contains(protocol))
|
||||
.filter(|(_, protocol, version)| {
|
||||
served
|
||||
.iter()
|
||||
.any(|(name, ours)| name == protocol && ours == version)
|
||||
})
|
||||
.map(|(peer, protocol, _)| (peer, protocol))
|
||||
.collect();
|
||||
|
||||
for (peer, protocol) in wanted {
|
||||
@@ -1220,6 +1324,26 @@ impl Runtime {
|
||||
if plugin.protocol_id() != capability.protocol {
|
||||
continue;
|
||||
}
|
||||
if plugin.protocol_version() != capability.version {
|
||||
// Said once per peer and protocol. A version that will
|
||||
// not match this build will not match it on the next
|
||||
// announcement either, and repeating it every round
|
||||
// would bury everything else.
|
||||
let key = (peer, capability.protocol.clone());
|
||||
if self.reported_mismatch.insert(key) {
|
||||
errors.push((
|
||||
capability.protocol.clone(),
|
||||
format!(
|
||||
"peer speaks {} version {} and this build speaks {}, so there \
|
||||
is no data plane with it; the control plane is unaffected",
|
||||
capability.protocol,
|
||||
capability.version,
|
||||
plugin.protocol_version()
|
||||
),
|
||||
));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// The core hands the opaque payload over without interpreting it.
|
||||
if let Err(err) = plugin.on_peer_capability(self.network_id, peer, capability) {
|
||||
errors.push((plugin.protocol_id().to_string(), err.to_string()));
|
||||
@@ -1239,6 +1363,7 @@ impl Runtime {
|
||||
// ----------------------------------------------------------------- status
|
||||
|
||||
fn status(&self) -> NetworkStatus {
|
||||
let served = self.served_protocols();
|
||||
let mut peers: Vec<PeerStatus> = self
|
||||
.sessions
|
||||
.values()
|
||||
@@ -1248,6 +1373,17 @@ impl Runtime {
|
||||
endpoint_id: session.peer,
|
||||
role: session.role,
|
||||
hostname: session.hostname.clone(),
|
||||
protocols: session
|
||||
.capabilities
|
||||
.iter()
|
||||
.filter(|capability| capability.enabled)
|
||||
.filter(|capability| {
|
||||
served.iter().any(|(name, ours)| {
|
||||
*name == capability.protocol && *ours == capability.version
|
||||
})
|
||||
})
|
||||
.map(|capability| capability.protocol.clone())
|
||||
.collect(),
|
||||
capabilities: session.capabilities.clone(),
|
||||
connected_for: session.established.elapsed(),
|
||||
paths: snapshot.paths,
|
||||
File renamed without changes.
File renamed without changes.
@@ -36,6 +36,19 @@ pub struct CandidateStatus {
|
||||
pub consecutive_failures: u32,
|
||||
}
|
||||
|
||||
/// The overlay interface, as the agent sees it.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct OverlayStatus {
|
||||
/// The interface name the operating system gave.
|
||||
pub interface: String,
|
||||
/// Its MTU.
|
||||
pub mtu: u32,
|
||||
/// The addresses it should be carrying.
|
||||
pub addresses: Vec<crate::overlay::Cidr>,
|
||||
/// What has happened on it.
|
||||
pub counters: crate::overlay::Counters,
|
||||
}
|
||||
|
||||
/// A member the signed state knows about, connected or not.
|
||||
///
|
||||
/// This is the durable roster: it comes from signed records, so a member that
|
||||
@@ -70,6 +83,12 @@ pub struct PeerStatus {
|
||||
///
|
||||
/// A mutable binding, not an identity.
|
||||
pub hostname: Option<String>,
|
||||
/// Protocols agreed with this peer: both sides have them at the same
|
||||
/// version.
|
||||
///
|
||||
/// Empty means no data plane with that peer, which leaves the control
|
||||
/// plane working — messages and state still flow.
|
||||
pub protocols: Vec<String>,
|
||||
/// Capabilities the peer announced. Payloads stay opaque.
|
||||
pub capabilities: Vec<PluginCapability>,
|
||||
/// How long the session has been up.
|
||||
@@ -237,6 +237,17 @@ pub struct AgentConfig {
|
||||
pub reconnect: ReconnectPolicy,
|
||||
/// IP plugins whose capabilities are announced and dispatched.
|
||||
pub plugins: Vec<SharedPlugin>,
|
||||
/// Where the overlay interface comes from, when the agent should have one.
|
||||
///
|
||||
/// One agent, one interface: it belongs here rather than to a protocol,
|
||||
/// because every protocol carries traffic for the same addresses on it.
|
||||
/// `None` means no interface — the protocols still run and their packets
|
||||
/// are discarded.
|
||||
pub tun_factory: Option<Arc<dyn crate::overlay::TunFactory>>,
|
||||
/// The name to ask the operating system for.
|
||||
pub interface_name: String,
|
||||
/// The interface MTU.
|
||||
pub interface_mtu: u32,
|
||||
/// The IPv4 overlay range this agent proposes.
|
||||
///
|
||||
/// Addresses are allocated from it and recorded in signed state, so a
|
||||
@@ -259,6 +270,9 @@ impl AgentConfig {
|
||||
limits: Limits::default(),
|
||||
reconnect: ReconnectPolicy::default(),
|
||||
plugins: Vec::new(),
|
||||
tun_factory: None,
|
||||
interface_name: "tsun0".to_string(),
|
||||
interface_mtu: 1280,
|
||||
overlay_ipv4_range: Some(crate::state::DEFAULT_IPV4_RANGE),
|
||||
}
|
||||
}
|
||||
@@ -314,6 +328,19 @@ impl AgentConfig {
|
||||
self
|
||||
}
|
||||
|
||||
/// Gives the agent an overlay interface, from this factory.
|
||||
pub fn with_interface(
|
||||
mut self,
|
||||
factory: Arc<dyn crate::overlay::TunFactory>,
|
||||
name: impl Into<String>,
|
||||
mtu: u32,
|
||||
) -> Self {
|
||||
self.tun_factory = Some(factory);
|
||||
self.interface_name = name.into();
|
||||
self.interface_mtu = mtu;
|
||||
self
|
||||
}
|
||||
|
||||
/// Replaces the limits.
|
||||
pub fn with_limits(mut self, limits: Limits) -> Self {
|
||||
self.limits = limits;
|
||||
@@ -20,7 +20,6 @@
|
||||
//! recorded and surfaced, the control plane keeps running.
|
||||
|
||||
pub mod transport;
|
||||
pub mod wireguard;
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
@@ -42,7 +41,7 @@ pub const MAX_PROTOCOL_ID_LEN: usize = 32;
|
||||
/// chooses to do so must validate it itself.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct PluginCapability {
|
||||
/// Protocol identifier, e.g. `wireguard`. Bounded by [`MAX_PROTOCOL_ID_LEN`].
|
||||
/// Protocol identifier, e.g. `wg-quic`. Bounded by [`MAX_PROTOCOL_ID_LEN`].
|
||||
pub protocol: String,
|
||||
/// Version of the plugin's announcement format.
|
||||
pub version: u16,
|
||||
@@ -52,6 +51,51 @@ pub struct PluginCapability {
|
||||
pub data: Vec<u8>,
|
||||
}
|
||||
|
||||
/// Where a protocol hands the packets it has decrypted.
|
||||
///
|
||||
/// A protocol proves *who* sent a packet; it does not know what that member
|
||||
/// is entitled to say, because entitlement is an address claim the system
|
||||
/// level holds. So a decrypted packet goes here rather than straight to an
|
||||
/// interface, and is checked on the way.
|
||||
pub trait PacketSink: Send + Sync + 'static {
|
||||
/// Hands over one packet, attributed to the peer whose tunnel decrypted
|
||||
/// it.
|
||||
fn deliver<'a>(
|
||||
&'a self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
packet: bytes::Bytes,
|
||||
) -> BoxFuture<'a, ()>;
|
||||
}
|
||||
|
||||
/// A sink that drops everything, for a protocol running without an interface.
|
||||
#[derive(Debug, Clone, Copy, Default)]
|
||||
pub struct DiscardPackets;
|
||||
|
||||
impl PacketSink for DiscardPackets {
|
||||
fn deliver<'a>(
|
||||
&'a self,
|
||||
_network: NetworkId,
|
||||
_peer: EndpointId,
|
||||
_packet: bytes::Bytes,
|
||||
) -> BoxFuture<'a, ()> {
|
||||
Box::pin(async move {})
|
||||
}
|
||||
}
|
||||
|
||||
/// One setting a protocol accepts.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct ProtocolOption {
|
||||
/// The key, as written in `key=value`.
|
||||
pub key: &'static str,
|
||||
/// A word for the value, for the help line: `BYTES`, `NAME`.
|
||||
pub value: &'static str,
|
||||
/// What it does.
|
||||
pub help: &'static str,
|
||||
/// What happens when it is not given.
|
||||
pub default: Option<&'static str>,
|
||||
}
|
||||
|
||||
/// Errors a plugin may return. They are recorded, never fatal for the agent.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[non_exhaustive]
|
||||
@@ -95,13 +139,32 @@ pub(crate) enum PluginRequest {
|
||||
pub struct PluginContext {
|
||||
sender: Option<mpsc::Sender<PluginRequest>>,
|
||||
local: Option<EndpointId>,
|
||||
/// Where decrypted packets go. Absent when nothing is carrying traffic,
|
||||
/// in which case a protocol still runs and its packets are discarded.
|
||||
sink: Option<Arc<dyn PacketSink>>,
|
||||
}
|
||||
|
||||
impl PluginContext {
|
||||
pub(crate) fn new(sender: mpsc::Sender<PluginRequest>, local: EndpointId) -> Self {
|
||||
pub(crate) fn new(
|
||||
sender: mpsc::Sender<PluginRequest>,
|
||||
local: EndpointId,
|
||||
sink: Option<Arc<dyn PacketSink>>,
|
||||
) -> Self {
|
||||
Self {
|
||||
sender: Some(sender),
|
||||
local: Some(local),
|
||||
sink,
|
||||
}
|
||||
}
|
||||
|
||||
/// Where to hand a decrypted packet.
|
||||
///
|
||||
/// Always answers: with no interface to write to, the packets are
|
||||
/// discarded, which is what `--no-tun` means and is not an error.
|
||||
pub fn packet_sink(&self) -> Arc<dyn PacketSink> {
|
||||
match &self.sink {
|
||||
Some(sink) => Arc::clone(sink),
|
||||
None => Arc::new(DiscardPackets),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -110,6 +173,7 @@ impl PluginContext {
|
||||
Self {
|
||||
sender: None,
|
||||
local: None,
|
||||
sink: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -170,11 +234,34 @@ impl std::fmt::Debug for PluginContext {
|
||||
/// Implementations must be cheap and non-blocking: the agent calls them from
|
||||
/// its runtime tasks. Anything slow belongs in the plugin's own tasks.
|
||||
pub trait IpPlugin: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// Stable protocol identifier, e.g. `wireguard`.
|
||||
/// Stable protocol identifier, e.g. `wg-quic`.
|
||||
///
|
||||
/// Must be non-empty and at most [`MAX_PROTOCOL_ID_LEN`] bytes.
|
||||
fn protocol_id(&self) -> &str;
|
||||
|
||||
/// The wire version of this protocol.
|
||||
///
|
||||
/// Two peers carry traffic for each other only when they have the same
|
||||
/// protocol at the same version. There is no negotiating a middle
|
||||
/// ground: a protocol either speaks the same words at both ends or it
|
||||
/// does not, and pretending otherwise produces a link that fails later
|
||||
/// and less clearly.
|
||||
///
|
||||
/// **Not the software version.** A plugin crate has its own version and
|
||||
/// it is nobody else's business: two peers on different builds work
|
||||
/// together for as long as the wire between them has not changed. This
|
||||
/// number moves only when the bytes do, so it must never be derived from
|
||||
/// `CARGO_PKG_VERSION` or anything else that moves with a release.
|
||||
fn protocol_version(&self) -> u16;
|
||||
|
||||
/// The settings this protocol accepts, and what they mean.
|
||||
///
|
||||
/// Declared rather than documented elsewhere, so the agent can list them
|
||||
/// without knowing anything about the protocol.
|
||||
fn options(&self) -> &'static [ProtocolOption] {
|
||||
&[]
|
||||
}
|
||||
|
||||
/// Called once, when the agent starts, before any network is activated.
|
||||
///
|
||||
/// The plugin keeps the context to ask for re-announcements and to report
|
||||
@@ -224,6 +311,20 @@ pub trait IpPlugin: Send + Sync + std::fmt::Debug + 'static {
|
||||
let _ = (network, range, allocations);
|
||||
}
|
||||
|
||||
/// Carries one packet to a peer, encrypting it however this protocol
|
||||
/// does.
|
||||
///
|
||||
/// `false` when it cannot right now — no link, no tunnel, not this
|
||||
/// protocol's peer — which the caller reports rather than treats as an
|
||||
/// error. The packet came off the one interface the agent owns, and
|
||||
/// which protocol takes it is settled by asking.
|
||||
///
|
||||
/// The default carries nothing, which is right for a plugin that only
|
||||
/// announces something.
|
||||
fn carry(&self, _network: NetworkId, _peer: EndpointId, _packet: bytes::Bytes) -> bool {
|
||||
false
|
||||
}
|
||||
|
||||
/// A data plane link to a peer is available for this plugin's protocol.
|
||||
///
|
||||
/// The plugin moves its packets over this link and never learns how the
|
||||
@@ -290,6 +391,10 @@ impl IpPlugin for TestCapabilityPlugin {
|
||||
&self.protocol
|
||||
}
|
||||
|
||||
fn protocol_version(&self) -> u16 {
|
||||
1
|
||||
}
|
||||
|
||||
fn local_capability(
|
||||
&self,
|
||||
_network: NetworkId,
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -185,13 +185,9 @@ pub struct OverlayReport {
|
||||
pub interface: String,
|
||||
/// Interface MTU.
|
||||
pub mtu: u32,
|
||||
/// This agent's overlay address.
|
||||
pub address: String,
|
||||
/// This agent's IPv4 overlay address, when the overlay is dual stack.
|
||||
pub address_v4: Option<String>,
|
||||
/// The subnet every member shares.
|
||||
pub prefix: String,
|
||||
/// Prefix length of that subnet.
|
||||
/// This agent's overlay address, once the network has agreed one.
|
||||
pub address: Option<String>,
|
||||
/// Prefix length of the overlay range every member shares.
|
||||
pub prefix_len: u8,
|
||||
/// One entry per overlay peer.
|
||||
pub peers: Vec<OverlayPeerReport>,
|
||||
@@ -211,10 +207,8 @@ pub struct OverlayPeerReport {
|
||||
pub endpoint_id: String,
|
||||
/// The peer's WireGuard public key.
|
||||
pub public_key: String,
|
||||
/// Its overlay address.
|
||||
pub address: String,
|
||||
/// Its IPv4 overlay address, when it has one.
|
||||
pub address_v4: Option<String>,
|
||||
/// The overlay address the network agreed it holds.
|
||||
pub address: Option<String>,
|
||||
/// Seconds since the last WireGuard handshake.
|
||||
///
|
||||
/// `None` means the tunnel has never handshaken and cannot carry traffic.
|
||||
@@ -205,7 +205,7 @@ pub async fn set_hostname(path: impl AsRef<Path>, hostname: &str) -> Result<Stri
|
||||
///
|
||||
/// Bump it whenever [`Request`], [`Response`] or anything they contain
|
||||
/// changes shape.
|
||||
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 4]);
|
||||
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 5]);
|
||||
|
||||
async fn write_message<T: serde::Serialize>(stream: &mut UnixStream, value: &T) -> Result<()> {
|
||||
let encoded = postcard::to_stdvec(value)
|
||||
@@ -44,9 +44,12 @@ pub mod error;
|
||||
pub mod identity;
|
||||
pub mod ipc;
|
||||
pub mod net;
|
||||
pub mod overlay;
|
||||
pub mod proto;
|
||||
pub mod state;
|
||||
pub mod storage;
|
||||
#[cfg(feature = "testing")]
|
||||
pub mod testing;
|
||||
|
||||
pub use agent::{Agent, AgentStatus, Event, NetworkStatus, PeerStatus};
|
||||
pub use config::{AgentConfig, Limits, ReconnectPolicy, StoragePaths, TransportPolicy};
|
||||
File renamed without changes.
@@ -11,10 +11,11 @@
|
||||
|
||||
use std::net::{IpAddr, Ipv6Addr};
|
||||
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::identity::NetworkId;
|
||||
use crate::overlay::OverlayError;
|
||||
|
||||
use super::overlay::OVERLAY_HOST_PREFIX_LEN;
|
||||
/// Prefix length of a single IPv6 host address.
|
||||
const HOST_PREFIX_LEN: u8 = 128;
|
||||
|
||||
/// Longest interface name Linux accepts, excluding the terminating NUL.
|
||||
pub const MAX_INTERFACE_NAME_LEN: usize = 15;
|
||||
@@ -33,13 +34,13 @@ pub struct Cidr {
|
||||
|
||||
impl Cidr {
|
||||
/// Builds a CIDR, rejecting an impossible prefix length.
|
||||
pub fn new(addr: IpAddr, prefix_len: u8) -> Result<Self, PluginError> {
|
||||
pub fn new(addr: IpAddr, prefix_len: u8) -> Result<Self, OverlayError> {
|
||||
let max = match addr {
|
||||
IpAddr::V4(_) => 32,
|
||||
IpAddr::V6(_) => 128,
|
||||
};
|
||||
if prefix_len > max {
|
||||
return Err(PluginError::Other(format!(
|
||||
return Err(OverlayError::Other(format!(
|
||||
"prefix length /{prefix_len} is impossible for {addr}"
|
||||
)));
|
||||
}
|
||||
@@ -50,7 +51,7 @@ impl Cidr {
|
||||
pub fn host(addr: Ipv6Addr) -> Self {
|
||||
Self {
|
||||
addr: IpAddr::V6(addr),
|
||||
prefix_len: OVERLAY_HOST_PREFIX_LEN,
|
||||
prefix_len: HOST_PREFIX_LEN,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -66,9 +67,9 @@ impl std::fmt::Display for Cidr {
|
||||
/// The name is stable across restarts and short enough for the platform. Two
|
||||
/// agents on the same host in the same network must be given different
|
||||
/// prefixes, or they would derive the same name.
|
||||
pub fn interface_name(prefix: &str, network: NetworkId) -> Result<String, PluginError> {
|
||||
pub fn interface_name(prefix: &str, network: NetworkId) -> Result<String, OverlayError> {
|
||||
if prefix.is_empty() {
|
||||
return Err(PluginError::Other(
|
||||
return Err(OverlayError::Other(
|
||||
"interface prefix must not be empty".into(),
|
||||
));
|
||||
}
|
||||
@@ -76,12 +77,12 @@ pub fn interface_name(prefix: &str, network: NetworkId) -> Result<String, Plugin
|
||||
.chars()
|
||||
.all(|c| c.is_ascii_lowercase() || c.is_ascii_digit())
|
||||
{
|
||||
return Err(PluginError::Other(
|
||||
return Err(OverlayError::Other(
|
||||
"interface prefix must be lowercase ASCII letters and digits".into(),
|
||||
));
|
||||
}
|
||||
if prefix.len() >= MAX_INTERFACE_NAME_LEN {
|
||||
return Err(PluginError::Other(format!(
|
||||
return Err(OverlayError::Other(format!(
|
||||
"interface prefix must be shorter than {MAX_INTERFACE_NAME_LEN} characters"
|
||||
)));
|
||||
}
|
||||
@@ -0,0 +1,577 @@
|
||||
//! The one interface an agent owns, and the loop that moves packets across it.
|
||||
//!
|
||||
//! Outbound, a packet read from the interface is looked up in the
|
||||
//! [`RoutingTable`] and handed to whatever can carry it to that peer.
|
||||
//! Inbound, a packet a protocol has decrypted is checked against the same
|
||||
//! table and written to the interface.
|
||||
//!
|
||||
//! Neither direction knows which protocol is involved, and that is the
|
||||
//! point: the interface, the addresses on it and the decision of whose
|
||||
//! packet this is all belong here, so several protocols can be carrying
|
||||
//! traffic at once without any of them owning the thing they are carrying it
|
||||
//! for.
|
||||
//!
|
||||
//! # What is counted rather than hidden
|
||||
//!
|
||||
//! A packet with nowhere to go, a packet from a peer that does not hold its
|
||||
//! source address, a multicast packet the operating system emitted anyway:
|
||||
//! each is dropped and counted, with a sample kept for the first kind,
|
||||
//! because "the overlay is quiet" and "the overlay is dropping everything"
|
||||
//! look identical without them.
|
||||
|
||||
use std::net::{IpAddr, Ipv4Addr};
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
|
||||
use bytes::Bytes;
|
||||
use iroh::EndpointId;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
use super::packet::IpHeader;
|
||||
use super::router::{Route, RoutingTable};
|
||||
use super::tun::{TunDevice, TunFactory, TunRequest};
|
||||
use super::{Cidr, OverlayError};
|
||||
|
||||
/// Carries one packet to a peer, by whatever protocol currently can.
|
||||
///
|
||||
/// Implemented by the layer that knows which protocols are live for which
|
||||
/// peers. The interface does not: it asks for a packet to reach a peer and
|
||||
/// is told whether that was possible.
|
||||
pub trait PacketCarrier: Send + Sync + 'static {
|
||||
/// Carries a packet, answering `false` if nothing can right now.
|
||||
fn carry(&self, route: Route, packet: Bytes) -> bool;
|
||||
}
|
||||
|
||||
/// Why a decrypted packet was not written to the interface.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum Rejected {
|
||||
/// It was not a packet this agent could read.
|
||||
Malformed,
|
||||
/// The overlay is IPv4; anything else has no owner here.
|
||||
WrongFamily,
|
||||
/// The sending peer does not hold the source address it used.
|
||||
WrongSource,
|
||||
}
|
||||
|
||||
/// Counters for one interface.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct Counters {
|
||||
/// Packets handed to a protocol.
|
||||
pub sent: u64,
|
||||
/// Packets written to the operating system.
|
||||
pub received: u64,
|
||||
/// Packets for an address nobody in any network holds.
|
||||
pub unroutable: u64,
|
||||
/// One such destination, so the number can be acted on.
|
||||
pub unroutable_sample: Option<IpAddr>,
|
||||
/// Packets nothing could carry, though their destination was known.
|
||||
pub undeliverable: u64,
|
||||
/// Multicast and broadcast packets, which the overlay does not carry.
|
||||
pub multicast: u64,
|
||||
/// Packets from a peer that does not hold the source address used.
|
||||
pub wrong_source: u64,
|
||||
/// Packets that could not be read at all.
|
||||
pub malformed: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct Tally {
|
||||
sent: AtomicU64,
|
||||
received: AtomicU64,
|
||||
unroutable: AtomicU64,
|
||||
undeliverable: AtomicU64,
|
||||
multicast: AtomicU64,
|
||||
wrong_source: AtomicU64,
|
||||
malformed: AtomicU64,
|
||||
sample: std::sync::Mutex<Option<IpAddr>>,
|
||||
}
|
||||
|
||||
impl Tally {
|
||||
fn snapshot(&self) -> Counters {
|
||||
Counters {
|
||||
sent: self.sent.load(Ordering::Relaxed),
|
||||
received: self.received.load(Ordering::Relaxed),
|
||||
unroutable: self.unroutable.load(Ordering::Relaxed),
|
||||
unroutable_sample: match self.sample.lock() {
|
||||
Ok(guard) => *guard,
|
||||
Err(poisoned) => *poisoned.into_inner(),
|
||||
},
|
||||
undeliverable: self.undeliverable.load(Ordering::Relaxed),
|
||||
multicast: self.multicast.load(Ordering::Relaxed),
|
||||
wrong_source: self.wrong_source.load(Ordering::Relaxed),
|
||||
malformed: self.malformed.load(Ordering::Relaxed),
|
||||
}
|
||||
}
|
||||
|
||||
fn note_unroutable(&self, destination: IpAddr) {
|
||||
self.unroutable.fetch_add(1, Ordering::Relaxed);
|
||||
// The first one is kept. A later sample would keep overwriting the
|
||||
// one a reader is looking at.
|
||||
let mut guard = match self.sample.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
if guard.is_none() {
|
||||
*guard = Some(destination);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The overlay interface.
|
||||
#[derive(Debug)]
|
||||
pub struct Interface {
|
||||
device: Arc<dyn TunDevice>,
|
||||
factory: Arc<dyn TunFactory>,
|
||||
name: String,
|
||||
mtu: u32,
|
||||
routes: Arc<RoutingTable>,
|
||||
tally: Arc<Tally>,
|
||||
/// What was last applied to the host, so an unchanged table is not
|
||||
/// re-applied on every pass.
|
||||
applied: std::sync::Mutex<Vec<Cidr>>,
|
||||
task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
}
|
||||
|
||||
impl Interface {
|
||||
/// Creates the interface and starts moving packets.
|
||||
pub async fn start(
|
||||
factory: Arc<dyn TunFactory>,
|
||||
name: impl Into<String>,
|
||||
mtu: u32,
|
||||
routes: Arc<RoutingTable>,
|
||||
carrier: Arc<dyn PacketCarrier>,
|
||||
) -> Result<Self, OverlayError> {
|
||||
let name = name.into();
|
||||
let device = factory.create(TunRequest::bare(name.clone(), mtu)).await?;
|
||||
let tally = Arc::new(Tally::default());
|
||||
|
||||
let task = {
|
||||
let device = Arc::clone(&device);
|
||||
let routes = Arc::clone(&routes);
|
||||
let tally = Arc::clone(&tally);
|
||||
tokio::spawn(async move {
|
||||
while let Some(packet) = device.recv().await {
|
||||
let Some(header) = IpHeader::parse(&packet) else {
|
||||
tally.malformed.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
};
|
||||
let destination = header.destination();
|
||||
if is_multicast_or_broadcast(destination) {
|
||||
// The operating system emits these on any interface.
|
||||
// The overlay is a set of point-to-point tunnels and
|
||||
// has nowhere to put them; expected, not a fault.
|
||||
tally.multicast.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
}
|
||||
let Some(route) = routes.route_v4(destination) else {
|
||||
tally.note_unroutable(destination);
|
||||
continue;
|
||||
};
|
||||
if carrier.carry(route, packet) {
|
||||
tally.sent.fetch_add(1, Ordering::Relaxed);
|
||||
} else {
|
||||
tally.undeliverable.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
})
|
||||
};
|
||||
|
||||
Ok(Self {
|
||||
device,
|
||||
factory,
|
||||
name,
|
||||
mtu,
|
||||
routes,
|
||||
tally,
|
||||
applied: std::sync::Mutex::new(Vec::new()),
|
||||
task: std::sync::Mutex::new(Some(task)),
|
||||
})
|
||||
}
|
||||
|
||||
/// Brings the addresses on the host in line with the routing table.
|
||||
///
|
||||
/// Called whenever a network agrees a different address for this agent.
|
||||
/// The interface is never recreated for it: that would drop every tunnel
|
||||
/// riding on it for the sake of one address.
|
||||
pub async fn sync_addresses(&self) -> Result<(), OverlayError> {
|
||||
let wanted = self.wanted_addresses();
|
||||
{
|
||||
let applied = match self.applied.lock() {
|
||||
Ok(guard) => guard.clone(),
|
||||
Err(poisoned) => poisoned.into_inner().clone(),
|
||||
};
|
||||
if applied == wanted {
|
||||
return Ok(());
|
||||
}
|
||||
}
|
||||
|
||||
// One address per network, so at most a handful; the request carries
|
||||
// the first and the provisioner reconciles the rest.
|
||||
let request = TunRequest {
|
||||
name: self.name.clone(),
|
||||
address: wanted.first().and_then(|cidr| match cidr.addr {
|
||||
IpAddr::V4(address) => Some(address),
|
||||
IpAddr::V6(_) => None,
|
||||
}),
|
||||
prefix_len: wanted.first().map_or(0, |cidr| cidr.prefix_len),
|
||||
mtu: self.mtu,
|
||||
};
|
||||
self.factory.reconfigure(request).await?;
|
||||
match self.applied.lock() {
|
||||
Ok(mut guard) => *guard = wanted,
|
||||
Err(poisoned) => *poisoned.into_inner() = wanted,
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The interface name the operating system gave.
|
||||
pub fn name(&self) -> &str {
|
||||
self.device.name()
|
||||
}
|
||||
|
||||
/// The interface MTU.
|
||||
pub fn mtu(&self) -> u32 {
|
||||
self.device.mtu()
|
||||
}
|
||||
|
||||
/// Removes the interface from the host.
|
||||
pub async fn remove(&self) {
|
||||
self.factory.destroy(self.device.name()).await;
|
||||
}
|
||||
|
||||
/// The counters as they stand.
|
||||
pub fn counters(&self) -> Counters {
|
||||
self.tally.snapshot()
|
||||
}
|
||||
|
||||
/// Writes a packet a protocol decrypted to the operating system.
|
||||
///
|
||||
/// The source is checked against what the network agreed that peer
|
||||
/// holds. A protocol proved *who* sent the packet; only this level knows
|
||||
/// what that member is entitled to say.
|
||||
pub async fn deliver(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
packet: Bytes,
|
||||
) -> Result<(), Rejected> {
|
||||
let Some(header) = IpHeader::parse(&packet) else {
|
||||
self.tally.malformed.fetch_add(1, Ordering::Relaxed);
|
||||
return Err(Rejected::Malformed);
|
||||
};
|
||||
let IpAddr::V4(source) = header.source() else {
|
||||
self.tally.wrong_source.fetch_add(1, Ordering::Relaxed);
|
||||
return Err(Rejected::WrongFamily);
|
||||
};
|
||||
if !self.routes.may_send_from(network, peer, source) {
|
||||
self.tally.wrong_source.fetch_add(1, Ordering::Relaxed);
|
||||
return Err(Rejected::WrongSource);
|
||||
}
|
||||
if self.device.send(packet).await.is_ok() {
|
||||
self.tally.received.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The addresses the interface should be carrying, from the table.
|
||||
pub fn wanted_addresses(&self) -> Vec<Cidr> {
|
||||
self.routes
|
||||
.local_addresses()
|
||||
.into_iter()
|
||||
.filter_map(|(address, prefix_len)| Cidr::new(address.into(), prefix_len).ok())
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Interface {
|
||||
fn drop(&mut self) {
|
||||
if let Ok(mut guard) = self.task.lock()
|
||||
&& let Some(task) = guard.take()
|
||||
{
|
||||
task.abort();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl RoutingTable {
|
||||
/// The route for a destination, when it is one the overlay carries.
|
||||
fn route_v4(&self, destination: IpAddr) -> Option<Route> {
|
||||
match destination {
|
||||
IpAddr::V4(address) => self.route(address),
|
||||
// The overlay is IPv4. An IPv6 packet has no owner here, and
|
||||
// counting it as unroutable is the truth rather than a fault.
|
||||
IpAddr::V6(_) => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether an address is one the overlay has nowhere to send.
|
||||
fn is_multicast_or_broadcast(address: IpAddr) -> bool {
|
||||
match address {
|
||||
IpAddr::V4(address) => {
|
||||
address.is_multicast() || address.is_broadcast() || address == Ipv4Addr::UNSPECIFIED
|
||||
}
|
||||
IpAddr::V6(address) => address.is_multicast(),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use super::*;
|
||||
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
use crate::overlay::router::NetworkRoutes;
|
||||
use crate::overlay::tun::{MemoryTun, MemoryTunFactory};
|
||||
|
||||
fn network(name: &str) -> NetworkId {
|
||||
NetworkKeys::derive(
|
||||
&NetworkName::new(name).unwrap(),
|
||||
&NetworkSecret::from_bytes(vec![7u8; 32]).unwrap(),
|
||||
)
|
||||
.network_id()
|
||||
}
|
||||
|
||||
fn peer(seed: u8) -> EndpointId {
|
||||
iroh::SecretKey::from_bytes(&[seed; 32]).public()
|
||||
}
|
||||
|
||||
fn addr(last: u8) -> Ipv4Addr {
|
||||
Ipv4Addr::new(10, 13, 37, last)
|
||||
}
|
||||
|
||||
fn ipv4(source: Ipv4Addr, destination: Ipv4Addr, payload: &[u8]) -> Bytes {
|
||||
let mut packet = vec![0u8; 20];
|
||||
packet[0] = 0x45;
|
||||
let total = (20 + payload.len()) as u16;
|
||||
packet[2..4].copy_from_slice(&total.to_be_bytes());
|
||||
packet[9] = 17;
|
||||
packet[12..16].copy_from_slice(&source.octets());
|
||||
packet[16..20].copy_from_slice(&destination.octets());
|
||||
packet.extend_from_slice(payload);
|
||||
Bytes::from(packet)
|
||||
}
|
||||
|
||||
/// Records what it was asked to carry, and can refuse.
|
||||
#[derive(Debug, Default)]
|
||||
struct Recorder {
|
||||
carried: std::sync::Mutex<Vec<(Route, Bytes)>>,
|
||||
refuse: bool,
|
||||
}
|
||||
|
||||
impl PacketCarrier for Recorder {
|
||||
fn carry(&self, route: Route, packet: Bytes) -> bool {
|
||||
if self.refuse {
|
||||
return false;
|
||||
}
|
||||
match self.carried.lock() {
|
||||
Ok(mut guard) => guard.push((route, packet)),
|
||||
Err(poisoned) => poisoned.into_inner().push((route, packet)),
|
||||
}
|
||||
true
|
||||
}
|
||||
}
|
||||
|
||||
impl Recorder {
|
||||
fn carried(&self) -> Vec<(Route, Bytes)> {
|
||||
match self.carried.lock() {
|
||||
Ok(guard) => guard.clone(),
|
||||
Err(poisoned) => poisoned.into_inner().clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn interface(
|
||||
refuse: bool,
|
||||
) -> (
|
||||
Interface,
|
||||
Arc<MemoryTun>,
|
||||
Arc<RoutingTable>,
|
||||
Arc<Recorder>,
|
||||
NetworkId,
|
||||
) {
|
||||
let id = network("interface");
|
||||
let routes = Arc::new(RoutingTable::new());
|
||||
routes
|
||||
.set_network(
|
||||
id,
|
||||
NetworkRoutes {
|
||||
range: Some("10.13.37.0/24".parse().unwrap()),
|
||||
local: Some(addr(1)),
|
||||
peers: vec![(addr(2), peer(2)), (addr(3), peer(3))],
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
let carrier = Arc::new(Recorder {
|
||||
carried: std::sync::Mutex::new(Vec::new()),
|
||||
refuse,
|
||||
});
|
||||
let factory = Arc::new(MemoryTunFactory::new());
|
||||
let interface = Interface::start(
|
||||
Arc::clone(&factory) as Arc<dyn TunFactory>,
|
||||
"tsuntest",
|
||||
1280,
|
||||
Arc::clone(&routes),
|
||||
Arc::clone(&carrier) as Arc<dyn PacketCarrier>,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
// The factory keeps what it made, which is how a test reaches the
|
||||
// device the interface is using without exposing it.
|
||||
let device = factory.device("tsuntest").expect("the device was created");
|
||||
(interface, device, routes, carrier, id)
|
||||
}
|
||||
|
||||
/// Waits for a counter to move, so the interface's own task has run.
|
||||
async fn wait_for(interface: &Interface, pick: impl Fn(&Counters) -> u64) {
|
||||
for _ in 0..400 {
|
||||
if pick(&interface.counters()) > 0 {
|
||||
return;
|
||||
}
|
||||
tokio::time::sleep(std::time::Duration::from_millis(5)).await;
|
||||
}
|
||||
panic!("the counter never moved: {:?}", interface.counters());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_packet_leaving_the_interface_goes_to_the_peer_that_owns_it() {
|
||||
let (interface, device, _routes, carrier, id) = interface(false).await;
|
||||
device.push_from_os(ipv4(addr(1), addr(2), b"hello"));
|
||||
|
||||
wait_for(&interface, |counters| counters.sent).await;
|
||||
let carried = carrier.carried();
|
||||
assert_eq!(carried.len(), 1);
|
||||
assert_eq!(carried[0].0.peer, peer(2));
|
||||
assert_eq!(carried[0].0.network, id);
|
||||
assert_eq!(&carried[0].1[20..], b"hello");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_packet_for_nobody_is_counted_with_a_sample_and_not_sent() {
|
||||
// Counted rather than dropped quietly: without the number and an
|
||||
// example of it, "the overlay is quiet" and "the overlay is dropping
|
||||
// everything" look the same.
|
||||
let (interface, device, _routes, carrier, _id) = interface(false).await;
|
||||
let nowhere = Ipv4Addr::new(10, 13, 37, 200);
|
||||
device.push_from_os(ipv4(addr(1), nowhere, b"lost"));
|
||||
|
||||
wait_for(&interface, |counters| counters.unroutable).await;
|
||||
assert_eq!(
|
||||
interface.counters().unroutable_sample,
|
||||
Some(IpAddr::V4(nowhere))
|
||||
);
|
||||
assert!(carrier.carried().is_empty(), "nothing was flooded");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn multicast_is_expected_and_never_carried() {
|
||||
let (interface, device, _routes, carrier, _id) = interface(false).await;
|
||||
device.push_from_os(ipv4(addr(1), Ipv4Addr::new(224, 0, 0, 251), b"mdns"));
|
||||
|
||||
wait_for(&interface, |counters| counters.multicast).await;
|
||||
assert_eq!(interface.counters().unroutable, 0, "not a routing failure");
|
||||
assert!(carrier.carried().is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_packet_nothing_can_carry_is_counted_separately() {
|
||||
// Distinct from unroutable: the destination is known, there is just
|
||||
// no live protocol for it. The two want different answers.
|
||||
let (interface, device, _routes, _carrier, _id) = interface(true).await;
|
||||
device.push_from_os(ipv4(addr(1), addr(2), b"no link"));
|
||||
|
||||
wait_for(&interface, |counters| counters.undeliverable).await;
|
||||
let counters = interface.counters();
|
||||
assert_eq!(counters.sent, 0);
|
||||
assert_eq!(counters.unroutable, 0);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_decrypted_packet_from_its_owner_reaches_the_operating_system() {
|
||||
let (interface, device, _routes, _carrier, id) = interface(false).await;
|
||||
interface
|
||||
.deliver(id, peer(2), ipv4(addr(2), addr(1), b"inbound"))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let written = tokio::time::timeout(std::time::Duration::from_secs(5), device.pop_to_os())
|
||||
.await
|
||||
.expect("the packet reaches the interface")
|
||||
.unwrap();
|
||||
assert_eq!(&written[20..], b"inbound");
|
||||
assert_eq!(interface.counters().received, 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_peer_cannot_send_from_an_address_it_does_not_hold() {
|
||||
let (interface, _device, _routes, _carrier, id) = interface(false).await;
|
||||
|
||||
// Another member's address.
|
||||
assert_eq!(
|
||||
interface
|
||||
.deliver(id, peer(2), ipv4(addr(3), addr(1), b"spoofed"))
|
||||
.await,
|
||||
Err(Rejected::WrongSource)
|
||||
);
|
||||
// And this agent's own.
|
||||
assert_eq!(
|
||||
interface
|
||||
.deliver(id, peer(2), ipv4(addr(1), addr(1), b"spoofed"))
|
||||
.await,
|
||||
Err(Rejected::WrongSource)
|
||||
);
|
||||
assert_eq!(interface.counters().wrong_source, 2);
|
||||
assert_eq!(interface.counters().received, 0);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_unreadable_or_wrong_family_packet_is_refused_without_panicking() {
|
||||
let (interface, _device, _routes, _carrier, id) = interface(false).await;
|
||||
|
||||
assert_eq!(
|
||||
interface.deliver(id, peer(2), Bytes::new()).await,
|
||||
Err(Rejected::Malformed)
|
||||
);
|
||||
assert_eq!(
|
||||
interface
|
||||
.deliver(id, peer(2), Bytes::from_static(&[0xff; 8]))
|
||||
.await,
|
||||
Err(Rejected::Malformed)
|
||||
);
|
||||
|
||||
// An IPv6 packet: readable, but the overlay has no owner for it.
|
||||
let mut six = vec![0u8; 40];
|
||||
six[0] = 0x60;
|
||||
assert_eq!(
|
||||
interface.deliver(id, peer(2), Bytes::from(six)).await,
|
||||
Err(Rejected::WrongFamily)
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_wanted_addresses_follow_the_table() {
|
||||
let (interface, _device, routes, _carrier, id) = interface(false).await;
|
||||
assert_eq!(
|
||||
interface.wanted_addresses(),
|
||||
vec![Cidr::new(addr(1).into(), 24).unwrap()]
|
||||
);
|
||||
|
||||
// The network agreed a different address for us.
|
||||
routes
|
||||
.set_network(
|
||||
id,
|
||||
NetworkRoutes {
|
||||
range: Some("10.13.37.0/24".parse().unwrap()),
|
||||
local: Some(addr(9)),
|
||||
peers: vec![(addr(2), peer(2))],
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
interface.wanted_addresses(),
|
||||
vec![Cidr::new(addr(9).into(), 24).unwrap()]
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
//! The overlay interface: the one network interface an agent owns.
|
||||
//!
|
||||
//! One agent, one interface. It belongs to the system level rather than to
|
||||
//! any protocol, and that is the whole point: a packet leaving it is routed
|
||||
//! to whichever peer owns its destination address, over whichever protocol
|
||||
//! currently has a link to that peer. Several protocols can be live at once
|
||||
//! without arguing over who holds the address, because neither of them
|
||||
//! holds it — the agent does.
|
||||
//!
|
||||
//! What lives here:
|
||||
//!
|
||||
//! * [`provision`] creates the interface and configures it, and removes it
|
||||
//! again. Platform mechanics behind one decision function.
|
||||
//! * [`tun`] is the packet interface itself, real or in memory.
|
||||
//! * [`packet`] reads just enough of an IP header to route by it.
|
||||
//! * [`config`] names interfaces and describes addresses.
|
||||
//!
|
||||
//! What does not live here: encryption, peer discovery, and how a packet
|
||||
//! actually reaches another machine. Those belong to a protocol plugin,
|
||||
//! which this level hands packets to and takes packets from.
|
||||
|
||||
/// Why the overlay interface cannot be brought into the state asked for.
|
||||
///
|
||||
/// Its own error rather than the plugin one it used to borrow: this level
|
||||
/// owns the interface now, and a plugin failing is a different event from
|
||||
/// the interface failing.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[non_exhaustive]
|
||||
pub enum OverlayError {
|
||||
/// The interface cannot be created or configured right now, with the
|
||||
/// reason and, where there is one, what to do about it.
|
||||
#[error("overlay unavailable: {0}")]
|
||||
Unavailable(String),
|
||||
/// Anything else.
|
||||
#[error("{0}")]
|
||||
Other(String),
|
||||
}
|
||||
|
||||
pub mod config;
|
||||
pub mod interface;
|
||||
pub mod packet;
|
||||
pub mod provision;
|
||||
pub mod router;
|
||||
pub mod tun;
|
||||
|
||||
pub use config::{Cidr, DEFAULT_INTERFACE_PREFIX, MAX_INTERFACE_NAME_LEN, interface_name};
|
||||
pub use interface::{Counters, Interface, PacketCarrier, Rejected};
|
||||
pub use packet::IpHeader;
|
||||
pub use provision::{
|
||||
Changes, InterfacePlan, InterfaceProvisioner, InterfaceState, LinkKind, ManagedTunFactory,
|
||||
MockHost, MockProvisioner, Privilege, Provisioned, UnsupportedProvisioner, plan_changes,
|
||||
probe_net_admin,
|
||||
};
|
||||
pub use router::{NetworkRoutes, Route, RouteError, RoutingTable};
|
||||
pub use tun::{MemoryTun, MemoryTunFactory, TunDevice, TunFactory, TunRequest, address_is_local};
|
||||
|
||||
#[cfg(all(feature = "tun-device", target_os = "linux"))]
|
||||
pub use provision::NetlinkProvisioner;
|
||||
File renamed without changes.
+12
-14
@@ -9,17 +9,17 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::overlay::OverlayError;
|
||||
|
||||
use super::super::config::Cidr;
|
||||
use super::super::tun::{TunDevice, TunFactory, TunRequest};
|
||||
use super::{InterfacePlan, InterfaceProvisioner};
|
||||
|
||||
/// Turns a [`TunRequest`] into the plan for a host interface.
|
||||
fn plan_for(request: &TunRequest) -> Result<InterfacePlan, PluginError> {
|
||||
let mut addresses = vec![Cidr::new(request.address.into(), request.prefix_len)?];
|
||||
if let Some(address) = request.address_v4 {
|
||||
addresses.push(Cidr::new(address.into(), request.prefix_len_v4)?);
|
||||
fn plan_for(request: &TunRequest) -> Result<InterfacePlan, OverlayError> {
|
||||
let mut addresses = Vec::new();
|
||||
if let Some(address) = request.address {
|
||||
addresses.push(Cidr::new(address.into(), request.prefix_len)?);
|
||||
}
|
||||
Ok(InterfacePlan::new(
|
||||
request.name.clone(),
|
||||
@@ -54,7 +54,7 @@ impl TunFactory for ManagedTunFactory {
|
||||
fn create<'a>(
|
||||
&'a self,
|
||||
request: TunRequest,
|
||||
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, PluginError>> {
|
||||
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, OverlayError>> {
|
||||
Box::pin(async move {
|
||||
let plan = plan_for(&request)?;
|
||||
let provisioned = self.provisioner.reconcile(&plan).await?;
|
||||
@@ -67,7 +67,7 @@ impl TunFactory for ManagedTunFactory {
|
||||
// Reaching here would mean the interface already existed and
|
||||
// was held open by us, which cannot be true on the path that
|
||||
// creates a device.
|
||||
PluginError::Other(format!(
|
||||
OverlayError::Other(format!(
|
||||
"interface `{}` was reconciled but no device came back",
|
||||
plan.name
|
||||
))
|
||||
@@ -75,7 +75,7 @@ impl TunFactory for ManagedTunFactory {
|
||||
})
|
||||
}
|
||||
|
||||
fn reconfigure<'a>(&'a self, request: TunRequest) -> BoxFuture<'a, Result<(), PluginError>> {
|
||||
fn reconfigure<'a>(&'a self, request: TunRequest) -> BoxFuture<'a, Result<(), OverlayError>> {
|
||||
Box::pin(async move {
|
||||
let plan = plan_for(&request)?;
|
||||
let provisioned = self.provisioner.reconcile(&plan).await?;
|
||||
@@ -106,7 +106,7 @@ impl TunFactory for ManagedTunFactory {
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use std::net::{Ipv4Addr, Ipv6Addr};
|
||||
use std::net::Ipv4Addr;
|
||||
|
||||
use super::super::{LinkKind, MockHost, MockProvisioner};
|
||||
use super::*;
|
||||
@@ -114,10 +114,8 @@ mod tests {
|
||||
fn request(v4: Option<Ipv4Addr>) -> TunRequest {
|
||||
TunRequest {
|
||||
name: "tsunfactory".into(),
|
||||
address: Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0, 1),
|
||||
prefix_len: 64,
|
||||
address_v4: v4,
|
||||
prefix_len_v4: 24,
|
||||
address: v4,
|
||||
prefix_len: 24,
|
||||
mtu: 1280,
|
||||
}
|
||||
}
|
||||
@@ -137,7 +135,7 @@ mod tests {
|
||||
let state = provisioner.host().get("tsunfactory").unwrap();
|
||||
assert_eq!(state.kind, LinkKind::Tun);
|
||||
assert_eq!(state.mtu, 1280);
|
||||
assert_eq!(state.addresses.len(), 2, "both families are assigned");
|
||||
assert_eq!(state.addresses.len(), 1, "the overlay address is assigned");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
+27
-27
@@ -50,7 +50,7 @@ use rtnetlink::packet_route::link::{InfoKind, LinkAttribute, LinkFlags, LinkInfo
|
||||
use rtnetlink::{LinkMessageBuilder, LinkUnspec};
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::overlay::OverlayError;
|
||||
|
||||
use super::super::config::Cidr;
|
||||
use super::super::tun::{TunDevice, TunRequest};
|
||||
@@ -71,7 +71,7 @@ enum Command {
|
||||
Stop,
|
||||
}
|
||||
|
||||
type Reply<T> = mpsc::Sender<Result<T, PluginError>>;
|
||||
type Reply<T> = mpsc::Sender<Result<T, OverlayError>>;
|
||||
|
||||
/// The configuration half of a reconciliation.
|
||||
struct Configure {
|
||||
@@ -97,17 +97,17 @@ pub struct NetlinkProvisioner {
|
||||
|
||||
impl NetlinkProvisioner {
|
||||
/// Starts the netlink thread, after checking this process can use it.
|
||||
pub fn new() -> Result<Self, PluginError> {
|
||||
pub fn new() -> Result<Self, OverlayError> {
|
||||
match probe_net_admin() {
|
||||
Privilege::Available => {}
|
||||
Privilege::Missing(reason) => {
|
||||
return Err(PluginError::Unavailable(format!(
|
||||
return Err(OverlayError::Unavailable(format!(
|
||||
"{reason}. {}",
|
||||
Privilege::how_to_grant(¤t_program())
|
||||
)));
|
||||
}
|
||||
Privilege::Unsupported => {
|
||||
return Err(PluginError::Unavailable(
|
||||
return Err(OverlayError::Unavailable(
|
||||
"interface management is not compiled in".to_string(),
|
||||
));
|
||||
}
|
||||
@@ -118,7 +118,7 @@ impl NetlinkProvisioner {
|
||||
.name("tsunagi-netlink".to_string())
|
||||
.spawn(move || netlink_thread(requests))
|
||||
.map_err(|err| {
|
||||
PluginError::Unavailable(format!("cannot start the netlink thread: {err}"))
|
||||
OverlayError::Unavailable(format!("cannot start the netlink thread: {err}"))
|
||||
})?;
|
||||
|
||||
Ok(Self {
|
||||
@@ -132,13 +132,13 @@ impl NetlinkProvisioner {
|
||||
fn call<T: Send + 'static>(
|
||||
&self,
|
||||
make: impl FnOnce(Reply<T>) -> Command,
|
||||
) -> Result<T, PluginError> {
|
||||
) -> Result<T, OverlayError> {
|
||||
let (reply_tx, reply_rx) = mpsc::channel();
|
||||
self.commands
|
||||
.send(make(reply_tx))
|
||||
.map_err(|_| PluginError::Unavailable("the netlink thread has stopped".to_string()))?;
|
||||
.map_err(|_| OverlayError::Unavailable("the netlink thread has stopped".to_string()))?;
|
||||
reply_rx.recv().map_err(|_| {
|
||||
PluginError::Unavailable("the netlink thread stopped mid-request".to_string())
|
||||
OverlayError::Unavailable("the netlink thread stopped mid-request".to_string())
|
||||
})?
|
||||
}
|
||||
|
||||
@@ -150,7 +150,7 @@ impl NetlinkProvisioner {
|
||||
///
|
||||
/// Synchronous on purpose: the capability guard is raised and lowered
|
||||
/// without an `await` in between, so it cannot outlive this thread.
|
||||
fn create_device(&self, plan: &InterfacePlan) -> Result<Arc<dyn TunDevice>, PluginError> {
|
||||
fn create_device(&self, plan: &InterfacePlan) -> Result<Arc<dyn TunDevice>, OverlayError> {
|
||||
let request = TunRequest::bare(plan.name.clone(), plan.mtu);
|
||||
let _guard = NetAdmin::acquire()?;
|
||||
super::super::tun::open_tun(&request)
|
||||
@@ -174,7 +174,7 @@ impl InterfaceProvisioner for NetlinkProvisioner {
|
||||
fn reconcile<'a>(
|
||||
&'a self,
|
||||
plan: &'a InterfacePlan,
|
||||
) -> BoxFuture<'a, Result<Provisioned, PluginError>> {
|
||||
) -> BoxFuture<'a, Result<Provisioned, OverlayError>> {
|
||||
Box::pin(async move {
|
||||
let current = self.call(|reply| Command::Observe(plan.name.clone(), reply))?;
|
||||
let changes = plan_changes(¤t, plan, self.is_ours(&plan.name))?;
|
||||
@@ -224,7 +224,7 @@ impl InterfaceProvisioner for NetlinkProvisioner {
|
||||
})
|
||||
}
|
||||
|
||||
fn remove<'a>(&'a self, name: &'a str) -> BoxFuture<'a, Result<(), PluginError>> {
|
||||
fn remove<'a>(&'a self, name: &'a str) -> BoxFuture<'a, Result<(), OverlayError>> {
|
||||
Box::pin(async move {
|
||||
// Dropping the device is what removes the interface; the explicit
|
||||
// delete is only so it is gone by the time this returns rather
|
||||
@@ -271,10 +271,10 @@ fn netlink_thread(requests: mpsc::Receiver<Command>) {
|
||||
let message = format!("the netlink thread has no runtime: {err}");
|
||||
match command {
|
||||
Command::Observe(_, reply) => {
|
||||
let _ = reply.send(Err(PluginError::Unavailable(message)));
|
||||
let _ = reply.send(Err(OverlayError::Unavailable(message)));
|
||||
}
|
||||
Command::Delete(_, reply) | Command::Configure(_, reply) => {
|
||||
let _ = reply.send(Err(PluginError::Unavailable(message)));
|
||||
let _ = reply.send(Err(OverlayError::Unavailable(message)));
|
||||
}
|
||||
Command::Stop => return,
|
||||
}
|
||||
@@ -311,12 +311,12 @@ fn netlink_thread(requests: mpsc::Receiver<Command>) {
|
||||
}
|
||||
|
||||
/// Opens a netlink connection and runs one unit of work over it.
|
||||
async fn with_netlink<T, F>(work: impl FnOnce(rtnetlink::Handle) -> F) -> Result<T, PluginError>
|
||||
async fn with_netlink<T, F>(work: impl FnOnce(rtnetlink::Handle) -> F) -> Result<T, OverlayError>
|
||||
where
|
||||
F: std::future::Future<Output = Result<T, PluginError>>,
|
||||
F: std::future::Future<Output = Result<T, OverlayError>>,
|
||||
{
|
||||
let (connection, handle, _messages) = rtnetlink::new_connection()
|
||||
.map_err(|err| PluginError::Unavailable(format!("cannot open netlink: {err}")))?;
|
||||
.map_err(|err| OverlayError::Unavailable(format!("cannot open netlink: {err}")))?;
|
||||
let pump = tokio::spawn(connection);
|
||||
let result = work(handle).await;
|
||||
pump.abort();
|
||||
@@ -361,7 +361,7 @@ fn link_kind(message: &LinkMessage, name: &str) -> LinkKind {
|
||||
}
|
||||
}
|
||||
|
||||
async fn observe(name: &str) -> Result<InterfaceState, PluginError> {
|
||||
async fn observe(name: &str) -> Result<InterfaceState, OverlayError> {
|
||||
with_netlink(|handle| async move {
|
||||
let mut links = handle.link().get().match_name(name.to_string()).execute();
|
||||
let message = match links.try_next().await {
|
||||
@@ -373,7 +373,7 @@ async fn observe(name: &str) -> Result<InterfaceState, PluginError> {
|
||||
if !std::path::Path::new(&format!("/sys/class/net/{name}")).exists() {
|
||||
return Ok(InterfaceState::absent());
|
||||
}
|
||||
return Err(PluginError::Unavailable(format!(
|
||||
return Err(OverlayError::Unavailable(format!(
|
||||
"cannot read interface `{name}`: {err}"
|
||||
)));
|
||||
}
|
||||
@@ -397,7 +397,7 @@ async fn observe(name: &str) -> Result<InterfaceState, PluginError> {
|
||||
.set_link_index_filter(index)
|
||||
.execute();
|
||||
while let Some(message) = stream.try_next().await.map_err(|err| {
|
||||
PluginError::Unavailable(format!("cannot read the addresses of `{name}`: {err}"))
|
||||
OverlayError::Unavailable(format!("cannot read the addresses of `{name}`: {err}"))
|
||||
})? {
|
||||
if let Some(cidr) = address_of(&message)
|
||||
&& !is_link_local(cidr.addr)
|
||||
@@ -440,7 +440,7 @@ fn cidr(addr: IpAddr, prefix_len: u8) -> Option<Cidr> {
|
||||
Cidr::new(addr, prefix_len).ok()
|
||||
}
|
||||
|
||||
async fn delete_link(name: &str) -> Result<(), PluginError> {
|
||||
async fn delete_link(name: &str) -> Result<(), OverlayError> {
|
||||
with_netlink(|handle| async move {
|
||||
let mut links = handle.link().get().match_name(name.to_string()).execute();
|
||||
let index = match links.try_next().await {
|
||||
@@ -449,13 +449,13 @@ async fn delete_link(name: &str) -> Result<(), PluginError> {
|
||||
Ok(None) | Err(_) => return Ok(()),
|
||||
};
|
||||
handle.link().del(index).execute().await.map_err(|err| {
|
||||
PluginError::Unavailable(format!("cannot remove interface `{name}`: {err}"))
|
||||
OverlayError::Unavailable(format!("cannot remove interface `{name}`: {err}"))
|
||||
})
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
async fn configure_link(configure: &Configure) -> Result<(), PluginError> {
|
||||
async fn configure_link(configure: &Configure) -> Result<(), OverlayError> {
|
||||
let name = configure.name.as_str();
|
||||
with_netlink(|handle| async move {
|
||||
let mut links = handle.link().get().match_name(name.to_string()).execute();
|
||||
@@ -466,7 +466,7 @@ async fn configure_link(configure: &Configure) -> Result<(), PluginError> {
|
||||
.flatten()
|
||||
.map(|message| message.header.index)
|
||||
.ok_or_else(|| {
|
||||
PluginError::Unavailable(format!(
|
||||
OverlayError::Unavailable(format!(
|
||||
"interface `{name}` disappeared before it could be configured"
|
||||
))
|
||||
})?;
|
||||
@@ -485,7 +485,7 @@ async fn configure_link(configure: &Configure) -> Result<(), PluginError> {
|
||||
.execute()
|
||||
.await
|
||||
.map_err(|err| {
|
||||
PluginError::Unavailable(format!("cannot configure interface `{name}`: {err}"))
|
||||
OverlayError::Unavailable(format!("cannot configure interface `{name}`: {err}"))
|
||||
})?;
|
||||
}
|
||||
|
||||
@@ -512,7 +512,7 @@ async fn configure_link(configure: &Configure) -> Result<(), PluginError> {
|
||||
.execute()
|
||||
.await
|
||||
.map_err(|err| {
|
||||
PluginError::Unavailable(format!(
|
||||
OverlayError::Unavailable(format!(
|
||||
"cannot remove {cidr} from interface `{name}`: {err}"
|
||||
))
|
||||
})?;
|
||||
@@ -526,7 +526,7 @@ async fn configure_link(configure: &Configure) -> Result<(), PluginError> {
|
||||
.execute()
|
||||
.await
|
||||
.map_err(|err| {
|
||||
PluginError::Unavailable(format!(
|
||||
OverlayError::Unavailable(format!(
|
||||
"cannot add {cidr} to interface `{name}`: {err}"
|
||||
))
|
||||
})?;
|
||||
+5
-5
@@ -9,7 +9,7 @@ use std::collections::HashMap;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::overlay::OverlayError;
|
||||
|
||||
use super::super::config::Cidr;
|
||||
use super::super::tun::{MemoryTun, MemoryTunFactory, TunFactory, TunRequest};
|
||||
@@ -125,9 +125,9 @@ impl MockProvisioner {
|
||||
}
|
||||
}
|
||||
|
||||
fn apply(&self, plan: &InterfacePlan) -> Result<Changes, PluginError> {
|
||||
fn apply(&self, plan: &InterfacePlan) -> Result<Changes, OverlayError> {
|
||||
if let Some(reason) = &self.failure {
|
||||
return Err(PluginError::Unavailable(reason.clone()));
|
||||
return Err(OverlayError::Unavailable(reason.clone()));
|
||||
}
|
||||
|
||||
let ours = self.owned().iter().any(|name| name == &plan.name);
|
||||
@@ -179,7 +179,7 @@ impl InterfaceProvisioner for MockProvisioner {
|
||||
fn reconcile<'a>(
|
||||
&'a self,
|
||||
plan: &'a InterfacePlan,
|
||||
) -> BoxFuture<'a, Result<Provisioned, PluginError>> {
|
||||
) -> BoxFuture<'a, Result<Provisioned, OverlayError>> {
|
||||
Box::pin(async move {
|
||||
let changes = self.apply(plan)?;
|
||||
let device = if changes.create_link {
|
||||
@@ -195,7 +195,7 @@ impl InterfaceProvisioner for MockProvisioner {
|
||||
})
|
||||
}
|
||||
|
||||
fn remove<'a>(&'a self, name: &'a str) -> BoxFuture<'a, Result<(), PluginError>> {
|
||||
fn remove<'a>(&'a self, name: &'a str) -> BoxFuture<'a, Result<(), OverlayError>> {
|
||||
Box::pin(async move {
|
||||
self.host.lock().remove(name);
|
||||
self.owned().retain(|owned| owned != name);
|
||||
+6
-6
@@ -38,7 +38,7 @@ use std::collections::BTreeSet;
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::overlay::OverlayError;
|
||||
|
||||
use super::config::Cidr;
|
||||
use super::tun::TunDevice;
|
||||
@@ -203,19 +203,19 @@ pub fn plan_changes(
|
||||
current: &InterfaceState,
|
||||
plan: &InterfacePlan,
|
||||
ours: bool,
|
||||
) -> Result<Changes, PluginError> {
|
||||
) -> Result<Changes, OverlayError> {
|
||||
let mut changes = Changes::default();
|
||||
|
||||
match ¤t.kind {
|
||||
LinkKind::Foreign(kind) => {
|
||||
return Err(PluginError::Unavailable(format!(
|
||||
return Err(OverlayError::Unavailable(format!(
|
||||
"`{}` already exists and is a {kind} interface, not one of ours. \
|
||||
Refusing to touch it. Run with a different interface prefix.",
|
||||
plan.name
|
||||
)));
|
||||
}
|
||||
LinkKind::Tun if !ours && current.attached => {
|
||||
return Err(PluginError::Unavailable(format!(
|
||||
return Err(OverlayError::Unavailable(format!(
|
||||
"`{}` already exists and another process is attached to it. \
|
||||
That is most likely a second agent on this host in the same \
|
||||
network; give one of them a different interface prefix.",
|
||||
@@ -293,13 +293,13 @@ pub trait InterfaceProvisioner: Send + Sync + std::fmt::Debug + 'static {
|
||||
fn reconcile<'a>(
|
||||
&'a self,
|
||||
plan: &'a InterfacePlan,
|
||||
) -> BoxFuture<'a, Result<Provisioned, PluginError>>;
|
||||
) -> BoxFuture<'a, Result<Provisioned, OverlayError>>;
|
||||
|
||||
/// Removes an interface this provisioner created.
|
||||
///
|
||||
/// Removing one that is already gone succeeds: this runs on the shutdown
|
||||
/// path, where the interface having vanished is the desired outcome.
|
||||
fn remove<'a>(&'a self, name: &'a str) -> BoxFuture<'a, Result<(), PluginError>>;
|
||||
fn remove<'a>(&'a self, name: &'a str) -> BoxFuture<'a, Result<(), OverlayError>>;
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
+4
-4
@@ -78,7 +78,7 @@ mod linux_impl {
|
||||
use caps::{CapSet, Capability};
|
||||
|
||||
use super::Privilege;
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::overlay::OverlayError;
|
||||
|
||||
/// Whether this thread holds `CAP_NET_ADMIN` in its permitted set.
|
||||
pub fn probe_net_admin() -> Privilege {
|
||||
@@ -108,16 +108,16 @@ mod linux_impl {
|
||||
|
||||
impl NetAdmin {
|
||||
/// Raises `CAP_NET_ADMIN` into the effective set.
|
||||
pub fn acquire() -> Result<Self, PluginError> {
|
||||
pub fn acquire() -> Result<Self, OverlayError> {
|
||||
let already = caps::has_cap(None, CapSet::Effective, Capability::CAP_NET_ADMIN)
|
||||
.map_err(|err| {
|
||||
PluginError::Unavailable(format!("cannot read capabilities: {err}"))
|
||||
OverlayError::Unavailable(format!("cannot read capabilities: {err}"))
|
||||
})?;
|
||||
if already {
|
||||
return Ok(Self { raised: false });
|
||||
}
|
||||
caps::raise(None, CapSet::Effective, Capability::CAP_NET_ADMIN).map_err(|err| {
|
||||
PluginError::Unavailable(format!(
|
||||
OverlayError::Unavailable(format!(
|
||||
"cannot raise CAP_NET_ADMIN: {err}. {}",
|
||||
Privilege::how_to_grant("tsunagi")
|
||||
))
|
||||
+5
-5
@@ -7,7 +7,7 @@
|
||||
//! of provisioning and says what to do instead.
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::overlay::OverlayError;
|
||||
|
||||
use super::{InterfacePlan, InterfaceProvisioner, Provisioned};
|
||||
|
||||
@@ -31,8 +31,8 @@ impl UnsupportedProvisioner {
|
||||
}
|
||||
}
|
||||
|
||||
fn refusal(&self) -> PluginError {
|
||||
PluginError::Unavailable(format!(
|
||||
fn refusal(&self) -> OverlayError {
|
||||
OverlayError::Unavailable(format!(
|
||||
"managing the overlay interface is not implemented on {} yet. \
|
||||
Run with `--no-tun` until it is: the tunnels still form, they just \
|
||||
do not reach the operating system.",
|
||||
@@ -49,11 +49,11 @@ impl InterfaceProvisioner for UnsupportedProvisioner {
|
||||
fn reconcile<'a>(
|
||||
&'a self,
|
||||
_plan: &'a InterfacePlan,
|
||||
) -> BoxFuture<'a, Result<Provisioned, PluginError>> {
|
||||
) -> BoxFuture<'a, Result<Provisioned, OverlayError>> {
|
||||
Box::pin(async move { Err(self.refusal()) })
|
||||
}
|
||||
|
||||
fn remove<'a>(&'a self, _name: &'a str) -> BoxFuture<'a, Result<(), PluginError>> {
|
||||
fn remove<'a>(&'a self, _name: &'a str) -> BoxFuture<'a, Result<(), OverlayError>> {
|
||||
// Nothing was ever created, so there is nothing to clean up and no
|
||||
// reason to fail a shutdown path.
|
||||
Box::pin(async move { Ok(()) })
|
||||
@@ -0,0 +1,440 @@
|
||||
//! Which peer a packet is for, and whether a peer may have sent it.
|
||||
//!
|
||||
//! One agent has one interface, so a packet coming off it could belong to any
|
||||
//! network the agent is in and to any protocol currently carrying traffic.
|
||||
//! Deciding which is this table's job, and it is the reason the interface
|
||||
//! belongs to the system level: the answer comes from signed state, which no
|
||||
//! protocol owns.
|
||||
//!
|
||||
//! Two questions, and they are not the same one:
|
||||
//!
|
||||
//! * **Outbound**: this destination address — whose is it? A packet for
|
||||
//! nobody is dropped rather than broadcast.
|
||||
//! * **Inbound**: this peer decrypted a packet with this source address —
|
||||
//! is that address actually its? A peer may not speak for anybody else,
|
||||
//! and the check consults the signed claim rather than anything the peer
|
||||
//! said.
|
||||
//!
|
||||
//! Nothing here is derived from a protocol's key. An address is allocated at
|
||||
//! the system level and signed by the member that holds it, so every protocol
|
||||
//! carries traffic for the same addresses and the table is the same whichever
|
||||
//! one is in use.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::net::Ipv4Addr;
|
||||
use std::sync::RwLock;
|
||||
|
||||
use iroh::EndpointId;
|
||||
|
||||
use crate::identity::NetworkId;
|
||||
use crate::state::Ipv4Range;
|
||||
|
||||
/// Where an outbound packet is going.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct Route {
|
||||
/// The network the destination belongs to.
|
||||
pub network: NetworkId,
|
||||
/// The member that holds the destination address.
|
||||
pub peer: EndpointId,
|
||||
}
|
||||
|
||||
/// What one network contributes to the table.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct NetworkRoutes {
|
||||
/// The range this network allocates from, once it has agreed one.
|
||||
pub range: Option<Ipv4Range>,
|
||||
/// This agent's own address in the network.
|
||||
pub local: Option<Ipv4Addr>,
|
||||
/// Every other member's address, as the signed state has it.
|
||||
pub peers: Vec<(Ipv4Addr, EndpointId)>,
|
||||
}
|
||||
|
||||
/// Why a network cannot be added to the table.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
|
||||
pub enum RouteError {
|
||||
/// Two networks would answer for the same addresses.
|
||||
#[error(
|
||||
"network {network} uses {range}, which overlaps {other_range} already in use by \
|
||||
{other}. One agent has one interface, so an address can belong to only one of \
|
||||
them: give one network a different range."
|
||||
)]
|
||||
Overlap {
|
||||
/// The network being added.
|
||||
network: NetworkId,
|
||||
/// The range it wants.
|
||||
range: Ipv4Range,
|
||||
/// The network already using an overlapping range.
|
||||
other: NetworkId,
|
||||
/// That network's range.
|
||||
other_range: Ipv4Range,
|
||||
},
|
||||
}
|
||||
|
||||
/// Address ownership across every network this agent is in.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct RoutingTable {
|
||||
networks: RwLock<HashMap<NetworkId, NetworkRoutes>>,
|
||||
}
|
||||
|
||||
impl RoutingTable {
|
||||
/// An empty table.
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
fn read(&self) -> std::sync::RwLockReadGuard<'_, HashMap<NetworkId, NetworkRoutes>> {
|
||||
match self.networks.read() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
}
|
||||
}
|
||||
|
||||
fn write(&self) -> std::sync::RwLockWriteGuard<'_, HashMap<NetworkId, NetworkRoutes>> {
|
||||
match self.networks.write() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Replaces what one network contributes.
|
||||
///
|
||||
/// Rejected when its range overlaps another network's. With one
|
||||
/// interface an address can belong to only one network, and guessing
|
||||
/// which would send somebody's traffic to a stranger.
|
||||
pub fn set_network(&self, network: NetworkId, routes: NetworkRoutes) -> Result<(), RouteError> {
|
||||
let mut networks = self.write();
|
||||
if let Some(range) = routes.range {
|
||||
for (other, existing) in networks.iter() {
|
||||
if *other == network {
|
||||
continue;
|
||||
}
|
||||
let Some(other_range) = existing.range else {
|
||||
continue;
|
||||
};
|
||||
if ranges_overlap(range, other_range) {
|
||||
return Err(RouteError::Overlap {
|
||||
network,
|
||||
range,
|
||||
other: *other,
|
||||
other_range,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
networks.insert(network, routes);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Forgets a network.
|
||||
pub fn remove_network(&self, network: NetworkId) {
|
||||
self.write().remove(&network);
|
||||
}
|
||||
|
||||
/// The peer that holds a destination address, if anybody does.
|
||||
///
|
||||
/// This agent's own address returns `None`: a packet for ourselves does
|
||||
/// not go over a tunnel, and answering with a peer would send it to one.
|
||||
pub fn route(&self, destination: Ipv4Addr) -> Option<Route> {
|
||||
let networks = self.read();
|
||||
for (network, routes) in networks.iter() {
|
||||
if routes.local == Some(destination) {
|
||||
return None;
|
||||
}
|
||||
if let Some((_, peer)) = routes
|
||||
.peers
|
||||
.iter()
|
||||
.find(|(address, _)| *address == destination)
|
||||
{
|
||||
return Some(Route {
|
||||
network: *network,
|
||||
peer: *peer,
|
||||
});
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Whether a peer may send from a source address.
|
||||
pub fn may_send_from(&self, network: NetworkId, peer: EndpointId, source: Ipv4Addr) -> bool {
|
||||
self.read().get(&network).is_some_and(|routes| {
|
||||
routes
|
||||
.peers
|
||||
.iter()
|
||||
.any(|(address, holder)| *address == source && *holder == peer)
|
||||
})
|
||||
}
|
||||
|
||||
/// Every address this agent should answer to, with its prefix length.
|
||||
pub fn local_addresses(&self) -> Vec<(Ipv4Addr, u8)> {
|
||||
let mut addresses: Vec<(Ipv4Addr, u8)> = self
|
||||
.read()
|
||||
.values()
|
||||
.filter_map(|routes| Some((routes.local?, routes.range?.prefix_len)))
|
||||
.collect();
|
||||
addresses.sort();
|
||||
addresses
|
||||
}
|
||||
|
||||
/// Whether a range would collide with one another network already uses.
|
||||
///
|
||||
/// Asked before proposing a range rather than after: with one interface
|
||||
/// an agent that claimed an address it could not route would also be
|
||||
/// telling everybody else to use that range, and "the range of the
|
||||
/// lowest author wins" would spread the collision instead of containing
|
||||
/// it.
|
||||
pub fn would_overlap(&self, network: NetworkId, range: Ipv4Range) -> Option<Ipv4Range> {
|
||||
self.read().iter().find_map(|(other, existing)| {
|
||||
if *other == network {
|
||||
return None;
|
||||
}
|
||||
let other_range = existing.range?;
|
||||
ranges_overlap(range, other_range).then_some(other_range)
|
||||
})
|
||||
}
|
||||
|
||||
/// How many networks the table covers.
|
||||
pub fn len(&self) -> usize {
|
||||
self.read().len()
|
||||
}
|
||||
|
||||
/// Whether it covers none.
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.read().is_empty()
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether two ranges share any address.
|
||||
fn ranges_overlap(one: Ipv4Range, other: Ipv4Range) -> bool {
|
||||
// A range contains the other's base, or the other way round. With
|
||||
// prefix-aligned ranges that is the whole of it: two blocks either nest
|
||||
// or are disjoint.
|
||||
one.contains(other.base) || other.contains(one.base)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use super::*;
|
||||
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
|
||||
fn network(name: &str) -> NetworkId {
|
||||
NetworkKeys::derive(
|
||||
&NetworkName::new(name).unwrap(),
|
||||
&NetworkSecret::from_bytes(vec![4u8; 32]).unwrap(),
|
||||
)
|
||||
.network_id()
|
||||
}
|
||||
|
||||
fn peer(seed: u8) -> EndpointId {
|
||||
iroh::SecretKey::from_bytes(&[seed; 32]).public()
|
||||
}
|
||||
|
||||
fn addr(last: u8) -> Ipv4Addr {
|
||||
Ipv4Addr::new(10, 13, 37, last)
|
||||
}
|
||||
|
||||
fn routes() -> NetworkRoutes {
|
||||
NetworkRoutes {
|
||||
range: Some("10.13.37.0/24".parse().unwrap()),
|
||||
local: Some(addr(1)),
|
||||
peers: vec![(addr(2), peer(2)), (addr(3), peer(3))],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_packet_goes_to_the_member_that_holds_its_destination() {
|
||||
let table = RoutingTable::new();
|
||||
let id = network("routing");
|
||||
table.set_network(id, routes()).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
table.route(addr(2)),
|
||||
Some(Route {
|
||||
network: id,
|
||||
peer: peer(2)
|
||||
})
|
||||
);
|
||||
assert_eq!(
|
||||
table.route(addr(3)),
|
||||
Some(Route {
|
||||
network: id,
|
||||
peer: peer(3)
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_packet_for_nobody_has_nowhere_to_go() {
|
||||
// Dropped rather than sent to everybody: a tunnel is not a broadcast
|
||||
// domain, and flooding would hand one member another's traffic.
|
||||
let table = RoutingTable::new();
|
||||
table.set_network(network("routing"), routes()).unwrap();
|
||||
assert_eq!(table.route(addr(200)), None);
|
||||
assert_eq!(table.route(Ipv4Addr::new(192, 0, 2, 1)), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_packet_for_ourselves_is_not_sent_to_a_peer() {
|
||||
let table = RoutingTable::new();
|
||||
table.set_network(network("routing"), routes()).unwrap();
|
||||
assert_eq!(table.route(addr(1)), None, "that address is ours");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_peer_may_only_send_from_the_address_it_holds() {
|
||||
let table = RoutingTable::new();
|
||||
let id = network("ownership");
|
||||
table.set_network(id, routes()).unwrap();
|
||||
|
||||
assert!(table.may_send_from(id, peer(2), addr(2)));
|
||||
// Somebody else's address, and its own network's.
|
||||
assert!(!table.may_send_from(id, peer(2), addr(3)));
|
||||
assert!(!table.may_send_from(id, peer(2), addr(1)));
|
||||
assert!(!table.may_send_from(id, peer(2), addr(99)));
|
||||
// A peer that holds nothing here.
|
||||
assert!(!table.may_send_from(id, peer(9), addr(2)));
|
||||
// And the question is per network, not global.
|
||||
assert!(!table.may_send_from(network("elsewhere"), peer(2), addr(2)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn two_networks_with_overlapping_ranges_are_refused() {
|
||||
// One interface means an address belongs to one network. Accepting
|
||||
// both and guessing would send somebody's traffic to a stranger.
|
||||
let table = RoutingTable::new();
|
||||
let first = network("first");
|
||||
let second = network("second");
|
||||
table.set_network(first, routes()).unwrap();
|
||||
|
||||
let err = table.set_network(second, routes()).unwrap_err();
|
||||
let RouteError::Overlap { other, .. } = err;
|
||||
assert_eq!(other, first);
|
||||
assert!(err.to_string().contains("different range"), "{err}");
|
||||
assert_eq!(table.len(), 1, "the overlapping network is not added");
|
||||
|
||||
// A range of its own is fine.
|
||||
table
|
||||
.set_network(
|
||||
second,
|
||||
NetworkRoutes {
|
||||
range: Some("10.99.0.0/16".parse().unwrap()),
|
||||
local: Some(Ipv4Addr::new(10, 99, 0, 1)),
|
||||
peers: vec![(Ipv4Addr::new(10, 99, 0, 2), peer(4))],
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(table.len(), 2);
|
||||
assert_eq!(
|
||||
table.route(Ipv4Addr::new(10, 99, 0, 2)),
|
||||
Some(Route {
|
||||
network: second,
|
||||
peer: peer(4)
|
||||
})
|
||||
);
|
||||
// And the first network still answers for its own.
|
||||
assert_eq!(table.route(addr(2)).map(|route| route.network), Some(first));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_range_can_be_asked_about_before_it_is_proposed() {
|
||||
// The point of asking first: an agent that claims an address it
|
||||
// cannot route also tells everybody else to use that range.
|
||||
let table = RoutingTable::new();
|
||||
let first = network("first");
|
||||
let second = network("second");
|
||||
table.set_network(first, routes()).unwrap();
|
||||
|
||||
let mine: Ipv4Range = "10.13.37.0/24".parse().unwrap();
|
||||
assert_eq!(table.would_overlap(second, mine), Some(mine));
|
||||
// Its own range is not a collision with itself.
|
||||
assert_eq!(table.would_overlap(first, mine), None);
|
||||
assert_eq!(
|
||||
table.would_overlap(second, "10.99.0.0/16".parse().unwrap()),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_nested_range_counts_as_overlapping() {
|
||||
let table = RoutingTable::new();
|
||||
table
|
||||
.set_network(
|
||||
network("wide"),
|
||||
NetworkRoutes {
|
||||
range: Some("10.0.0.0/8".parse().unwrap()),
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
// Inside the /8, in either direction.
|
||||
assert!(
|
||||
table
|
||||
.set_network(
|
||||
network("narrow"),
|
||||
NetworkRoutes {
|
||||
range: Some("10.13.37.0/24".parse().unwrap()),
|
||||
..Default::default()
|
||||
}
|
||||
)
|
||||
.is_err()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replacing_a_network_does_not_count_as_overlapping_itself() {
|
||||
let table = RoutingTable::new();
|
||||
let id = network("refresh");
|
||||
table.set_network(id, routes()).unwrap();
|
||||
|
||||
let mut updated = routes();
|
||||
updated.peers.push((addr(4), peer(4)));
|
||||
table.set_network(id, updated).unwrap();
|
||||
assert_eq!(table.len(), 1);
|
||||
assert!(table.route(addr(4)).is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn forgetting_a_network_takes_its_addresses_with_it() {
|
||||
let table = RoutingTable::new();
|
||||
let id = network("leaving");
|
||||
table.set_network(id, routes()).unwrap();
|
||||
table.remove_network(id);
|
||||
|
||||
assert!(table.is_empty());
|
||||
assert_eq!(table.route(addr(2)), None);
|
||||
assert!(!table.may_send_from(id, peer(2), addr(2)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_local_addresses_are_what_the_interface_should_carry() {
|
||||
let table = RoutingTable::new();
|
||||
table.set_network(network("one"), routes()).unwrap();
|
||||
table
|
||||
.set_network(
|
||||
network("two"),
|
||||
NetworkRoutes {
|
||||
range: Some("10.99.0.0/16".parse().unwrap()),
|
||||
local: Some(Ipv4Addr::new(10, 99, 0, 1)),
|
||||
peers: Vec::new(),
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
table.local_addresses(),
|
||||
vec![(addr(1), 24), (Ipv4Addr::new(10, 99, 0, 1), 16)]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_network_with_no_address_yet_contributes_nothing() {
|
||||
let table = RoutingTable::new();
|
||||
let id = network("waiting");
|
||||
table.set_network(id, NetworkRoutes::default()).unwrap();
|
||||
|
||||
assert!(table.local_addresses().is_empty());
|
||||
assert_eq!(table.route(addr(2)), None);
|
||||
assert_eq!(table.len(), 1, "the network is known, it just has nothing");
|
||||
}
|
||||
}
|
||||
@@ -1,8 +1,8 @@
|
||||
//! The boundary to the operating system's packet interface.
|
||||
//!
|
||||
//! The WireGuard implementation in [`super::device`] is pure userspace and
|
||||
//! needs no kernel WireGuard module and no `wg` tool. It does still need a way
|
||||
//! to hand IP packets to the operating system, which is what this trait is.
|
||||
//! One agent has one of these, and it belongs to the agent rather than to
|
||||
//! any protocol: a packet arriving here is routed to whichever peer owns its
|
||||
//! destination, over whichever protocol has a link to that peer.
|
||||
//!
|
||||
//! Two implementations:
|
||||
//!
|
||||
@@ -13,27 +13,25 @@
|
||||
//! Creating one needs `CAP_NET_ADMIN`, and it is
|
||||
//! [`provision`](super::provision) that holds that and creates it.
|
||||
|
||||
use std::net::Ipv6Addr;
|
||||
use std::sync::Arc;
|
||||
|
||||
use bytes::Bytes;
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::overlay::OverlayError;
|
||||
|
||||
/// What a device should look like once created.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct TunRequest {
|
||||
/// Interface name to ask for.
|
||||
pub name: String,
|
||||
/// The overlay address this host answers to.
|
||||
pub address: Ipv6Addr,
|
||||
/// Prefix length of the overlay subnet, so the OS routes it here.
|
||||
/// The overlay address this host answers to, and its prefix length.
|
||||
///
|
||||
/// Allocated and signed at the system level, never derived from a
|
||||
/// protocol's key: every protocol carries traffic for the same address.
|
||||
pub address: Option<std::net::Ipv4Addr>,
|
||||
/// Prefix length of the overlay range.
|
||||
pub prefix_len: u8,
|
||||
/// The IPv4 overlay address this host answers to, when dual stack.
|
||||
pub address_v4: Option<std::net::Ipv4Addr>,
|
||||
/// Prefix length of the IPv4 overlay range.
|
||||
pub prefix_len_v4: u8,
|
||||
/// Interface MTU.
|
||||
pub mtu: u32,
|
||||
}
|
||||
@@ -46,10 +44,8 @@ impl TunRequest {
|
||||
pub fn bare(name: impl Into<String>, mtu: u32) -> Self {
|
||||
Self {
|
||||
name: name.into(),
|
||||
address: Ipv6Addr::UNSPECIFIED,
|
||||
address: None,
|
||||
prefix_len: 0,
|
||||
address_v4: None,
|
||||
prefix_len_v4: 0,
|
||||
mtu,
|
||||
}
|
||||
}
|
||||
@@ -71,7 +67,7 @@ pub trait TunDevice: Send + Sync + std::fmt::Debug + 'static {
|
||||
fn recv(&self) -> BoxFuture<'_, Option<Bytes>>;
|
||||
|
||||
/// Delivers a packet to the operating system.
|
||||
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), PluginError>>;
|
||||
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), OverlayError>>;
|
||||
}
|
||||
|
||||
/// Creates packet interfaces.
|
||||
@@ -83,7 +79,7 @@ pub trait TunFactory: Send + Sync + std::fmt::Debug + 'static {
|
||||
fn create<'a>(
|
||||
&'a self,
|
||||
request: TunRequest,
|
||||
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, PluginError>>;
|
||||
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, OverlayError>>;
|
||||
|
||||
/// Applies a changed request to an interface that already exists.
|
||||
///
|
||||
@@ -94,7 +90,7 @@ pub trait TunFactory: Send + Sync + std::fmt::Debug + 'static {
|
||||
///
|
||||
/// The default does nothing, which is right for a factory that only
|
||||
/// attaches to an interface somebody else prepared.
|
||||
fn reconfigure<'a>(&'a self, _request: TunRequest) -> BoxFuture<'a, Result<(), PluginError>> {
|
||||
fn reconfigure<'a>(&'a self, _request: TunRequest) -> BoxFuture<'a, Result<(), OverlayError>> {
|
||||
Box::pin(async move { Ok(()) })
|
||||
}
|
||||
|
||||
@@ -162,7 +158,7 @@ impl TunDevice for MemoryTun {
|
||||
Box::pin(async move { self.from_os_rx.lock().await.recv().await })
|
||||
}
|
||||
|
||||
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), PluginError>> {
|
||||
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), OverlayError>> {
|
||||
Box::pin(async move {
|
||||
let _ = self.to_os_tx.send(packet);
|
||||
Ok(())
|
||||
@@ -223,7 +219,7 @@ impl TunFactory for MemoryTunFactory {
|
||||
fn create<'a>(
|
||||
&'a self,
|
||||
request: TunRequest,
|
||||
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, PluginError>> {
|
||||
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, OverlayError>> {
|
||||
Box::pin(async move {
|
||||
let device = MemoryTun::new(request.name, request.mtu);
|
||||
let mut guard = match self.created.lock() {
|
||||
@@ -248,7 +244,7 @@ mod system {
|
||||
|
||||
use super::{TunDevice, TunRequest};
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::overlay::OverlayError;
|
||||
|
||||
/// A real TUN interface.
|
||||
///
|
||||
@@ -303,14 +299,14 @@ mod system {
|
||||
})
|
||||
}
|
||||
|
||||
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), PluginError>> {
|
||||
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), OverlayError>> {
|
||||
Box::pin(async move {
|
||||
use tokio::io::AsyncWriteExt;
|
||||
let mut writer = self.writer.lock().await;
|
||||
writer
|
||||
.write_all(&packet)
|
||||
.await
|
||||
.map_err(|err| PluginError::Other(format!("tun write failed: {err}")))
|
||||
.map_err(|err| OverlayError::Other(format!("tun write failed: {err}")))
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -320,7 +316,7 @@ mod system {
|
||||
/// Synchronous, and deliberately so: the caller holds a capability guard
|
||||
/// across this call, and such a guard must not span an `await` because
|
||||
/// Linux capabilities are per thread.
|
||||
pub(crate) fn open_tun(request: &TunRequest) -> Result<Arc<dyn TunDevice>, PluginError> {
|
||||
pub(crate) fn open_tun(request: &TunRequest) -> Result<Arc<dyn TunDevice>, OverlayError> {
|
||||
let mut config = tun::Configuration::default();
|
||||
config.tun_name(&request.name);
|
||||
config.platform_config(|platform| {
|
||||
@@ -334,7 +330,7 @@ mod system {
|
||||
// information, so the flags match when attaching to one.
|
||||
|
||||
let device = tun::create_as_async(&config).map_err(|err| {
|
||||
PluginError::Unavailable(format!(
|
||||
OverlayError::Unavailable(format!(
|
||||
"cannot create the TUN interface `{}`: {err}. Creating one needs \
|
||||
CAP_NET_ADMIN; grant it with `setcap cap_net_admin+p`, or run with \
|
||||
`--no-tun` to keep the tunnels off the operating system.",
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -56,8 +56,13 @@ fn restrict_permissions(_file: &std::fs::File, _path: &Path) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Restricts a file to the user that owns it.
|
||||
///
|
||||
/// Part of what the system level offers a protocol: a plugin keeping keys of
|
||||
/// its own on disk has the same obligation as the agent, and should not have
|
||||
/// to work out the platform details again to meet it.
|
||||
#[cfg(unix)]
|
||||
pub(crate) fn restrict_path_permissions(path: &Path) -> Result<()> {
|
||||
pub fn restrict_path_permissions(path: &Path) -> Result<()> {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)).map_err(|source| {
|
||||
Error::Io {
|
||||
@@ -67,13 +72,15 @@ pub(crate) fn restrict_path_permissions(path: &Path) -> Result<()> {
|
||||
})
|
||||
}
|
||||
|
||||
/// Restricts a file to the user that owns it. A no-op off Unix.
|
||||
#[cfg(not(unix))]
|
||||
pub(crate) fn restrict_path_permissions(_path: &Path) -> Result<()> {
|
||||
pub fn restrict_path_permissions(_path: &Path) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Restricts a directory to the user that owns it.
|
||||
#[cfg(unix)]
|
||||
pub(crate) fn restrict_dir_permissions(path: &Path) -> Result<()> {
|
||||
pub fn restrict_dir_permissions(path: &Path) -> Result<()> {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o700)).map_err(|source| {
|
||||
Error::Io {
|
||||
@@ -83,12 +90,14 @@ pub(crate) fn restrict_dir_permissions(path: &Path) -> Result<()> {
|
||||
})
|
||||
}
|
||||
|
||||
/// Restricts a directory to the user that owns it. A no-op off Unix.
|
||||
#[cfg(not(unix))]
|
||||
pub(crate) fn restrict_dir_permissions(_path: &Path) -> Result<()> {
|
||||
pub fn restrict_dir_permissions(_path: &Path) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) fn create_dir(path: &Path) -> Result<()> {
|
||||
/// Creates a directory and every parent, restricted to this user.
|
||||
pub fn create_dir(path: &Path) -> Result<()> {
|
||||
std::fs::create_dir_all(path).map_err(|source| Error::Io {
|
||||
path: path.to_path_buf(),
|
||||
source,
|
||||
File renamed without changes.
@@ -1,26 +1,31 @@
|
||||
//! Shared helpers for the integration tests.
|
||||
//! A harness for testing against a real agent.
|
||||
//!
|
||||
//! Every test uses real iroh endpoints on loopback, its own temporary SQLite
|
||||
//! files and independent agent instances. Only discovery is substituted; iroh,
|
||||
//! the handshake, message passing and persistent storage are not.
|
||||
//! Behind the `testing` feature, because it is only of use to a test and
|
||||
//! pulls in `tempfile` and a tracing subscriber. It lives here rather than
|
||||
//! being copied into each crate's `tests/` directory: a protocol plugin in
|
||||
//! its own crate needs exactly the same harness, and two copies of it would
|
||||
//! drift.
|
||||
//!
|
||||
//! Synchronisation is always "wait for a specific event or condition, under one
|
||||
//! overall deadline", never a fixed multi-second sleep.
|
||||
//! Everything here builds agents that touch nothing outside a temporary
|
||||
//! directory: loopback only, no relay, no address publication.
|
||||
|
||||
#![allow(dead_code, clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
// A harness fails a test loudly and at once; that is the whole job. The
|
||||
// crate forbids these elsewhere for the opposite reason — nothing off the
|
||||
// network may panic — and this module never sees anything off the network.
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use std::future::Future;
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use crate::agent::Event;
|
||||
use crate::config::{AgentConfig, StoragePaths, TransportPolicy};
|
||||
use crate::discovery::SharedMemoryDiscovery;
|
||||
use crate::identity::{NetworkName, NetworkSecret};
|
||||
use crate::{Agent, Result};
|
||||
use tempfile::TempDir;
|
||||
use tokio::sync::broadcast::error::RecvError;
|
||||
use tsunagi::agent::Event;
|
||||
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::identity::{NetworkName, NetworkSecret};
|
||||
use tsunagi::{Agent, Result};
|
||||
|
||||
/// Installs a tracing subscriber when `TSUNAGI_TEST_LOG` is set.
|
||||
///
|
||||
@@ -50,7 +55,7 @@ const POLL_INTERVAL: Duration = Duration::from_millis(25);
|
||||
/// port mapping, so the suite needs neither the internet nor privileges.
|
||||
/// Timeouts are shortened so that failure paths finish quickly.
|
||||
pub fn local_config(dir: &Path) -> AgentConfig {
|
||||
let limits = tsunagi::Limits {
|
||||
let limits = crate::Limits {
|
||||
dial_timeout: Duration::from_millis(1500),
|
||||
handshake_timeout: Duration::from_secs(5),
|
||||
..Default::default()
|
||||
@@ -69,7 +74,9 @@ pub fn config_with(dir: &Path, discovery: &SharedMemoryDiscovery) -> AgentConfig
|
||||
|
||||
/// A temporary directory plus the agent running on it.
|
||||
pub struct TestAgent {
|
||||
/// The directory it keeps its state in, removed when this is dropped.
|
||||
pub dir: TempDir,
|
||||
/// The agent itself.
|
||||
pub agent: Agent,
|
||||
}
|
||||
|
||||
@@ -170,7 +177,7 @@ pub async fn settle() {
|
||||
/// Waits until `agent` has `count` authenticated peers in `network`.
|
||||
pub async fn wait_for_peers(
|
||||
agent: &Agent,
|
||||
network: tsunagi::NetworkId,
|
||||
network: crate::NetworkId,
|
||||
count: usize,
|
||||
) -> Vec<iroh::EndpointId> {
|
||||
wait_until(&format!("{count} peers in {network}"), || async move {
|
||||
@@ -8,9 +8,6 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use common::{TestAgent, network, settle, wait_event};
|
||||
use iroh::endpoint::{Connection, PortmapperConfig, RecvStream, SendStream, presets};
|
||||
use iroh::{Endpoint, EndpointAddr, RelayMode};
|
||||
use tsunagi::agent::Event;
|
||||
@@ -22,6 +19,7 @@ use tsunagi::proto::message::{
|
||||
};
|
||||
use tsunagi::proto::{read_frame, write_frame};
|
||||
use tsunagi::test_support;
|
||||
use tsunagi::testing::{TestAgent, network, settle, wait_event};
|
||||
|
||||
/// A bare iroh endpoint with no tsunagi agent behind it.
|
||||
async fn raw_endpoint() -> Endpoint {
|
||||
@@ -2,14 +2,12 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use std::io::Write;
|
||||
|
||||
use common::{TestAgent, config_with, local_config, network, wait_for_peers};
|
||||
use tsunagi::config::StoragePaths;
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::storage::CacheOutcome;
|
||||
use tsunagi::testing::{TestAgent, config_with, local_config, network, wait_for_peers};
|
||||
use tsunagi::{Agent, Error};
|
||||
|
||||
/// Overwrites a file with bytes that are definitely not a SQLite database.
|
||||
@@ -8,9 +8,6 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use common::{config_with, network, wait_until};
|
||||
use tempfile::TempDir;
|
||||
use tsunagi::Agent;
|
||||
use tsunagi::config::StoragePaths;
|
||||
@@ -18,6 +15,7 @@ use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
use tsunagi::state::{RecordBody, StateSet};
|
||||
use tsunagi::storage::StateStore;
|
||||
use tsunagi::testing::{config_with, network, wait_until};
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_changed_name_reaches_peers_and_replaces_the_old_claim() {
|
||||
@@ -3,16 +3,14 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use common::{TestAgent, local_config, network, settle, wait_for_peers};
|
||||
use tsunagi::Agent;
|
||||
use tsunagi::discovery::{
|
||||
CandidateSource, CompositeDiscovery, NetworkDiscovery, SharedMemoryDiscovery, StaticBootstrap,
|
||||
};
|
||||
use tsunagi::identity::NetworkKeys;
|
||||
use tsunagi::testing::{TestAgent, local_config, network, settle, wait_for_peers};
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_static_bootstrap_candidate_is_enough_to_join() {
|
||||
@@ -3,12 +3,10 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use common::{TestAgent, network, wait_event, wait_for_peers};
|
||||
use tsunagi::agent::Event;
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::proto::ControlMessage;
|
||||
use tsunagi::testing::{TestAgent, network, wait_event, wait_for_peers};
|
||||
|
||||
#[tokio::test]
|
||||
async fn two_agents_authenticate_and_exchange_messages() {
|
||||
@@ -5,12 +5,10 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use common::{TestAgent, config_with, network};
|
||||
use tsunagi::Agent;
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
use tsunagi::testing::{TestAgent, config_with, network};
|
||||
|
||||
#[test]
|
||||
fn derivation_is_a_pure_function_of_name_and_secret() {
|
||||
@@ -3,16 +3,14 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use std::collections::HashSet;
|
||||
use std::sync::Arc;
|
||||
|
||||
use common::{TestAgent, network, wait_event, wait_for_peers, wait_until};
|
||||
use tsunagi::agent::Event;
|
||||
use tsunagi::dataplane::TestCapabilityPlugin;
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::proto::ControlMessage;
|
||||
use tsunagi::testing::{TestAgent, network, wait_event, wait_for_peers, wait_until};
|
||||
|
||||
#[tokio::test]
|
||||
async fn four_agents_form_a_mesh_and_exchange_distinguishable_messages() {
|
||||
@@ -2,9 +2,6 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use common::{TestAgent, network, settle, wait_event, wait_for_peers};
|
||||
use iroh::endpoint::{PortmapperConfig, presets};
|
||||
use iroh::{Endpoint, RelayMode};
|
||||
use tsunagi::agent::Event;
|
||||
@@ -16,6 +13,7 @@ use tsunagi::proto::message::{
|
||||
};
|
||||
use tsunagi::proto::{read_frame, write_frame};
|
||||
use tsunagi::test_support;
|
||||
use tsunagi::testing::{TestAgent, network, settle, wait_event, wait_for_peers};
|
||||
|
||||
const LIMIT: usize = 64 * 1024;
|
||||
|
||||
@@ -3,18 +3,18 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use std::net::SocketAddr;
|
||||
use std::time::Duration;
|
||||
|
||||
use common::{TestAgent, config_with, local_config, network, settle, wait_event, wait_for_peers};
|
||||
use iroh::{EndpointAddr, SecretKey};
|
||||
use tsunagi::agent::Event;
|
||||
use tsunagi::config::ReconnectPolicy;
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::identity::NetworkKeys;
|
||||
use tsunagi::proto::ControlMessage;
|
||||
use tsunagi::testing::{
|
||||
TestAgent, config_with, local_config, network, settle, wait_event, wait_for_peers,
|
||||
};
|
||||
use tsunagi::{Agent, Error};
|
||||
|
||||
/// An endpoint id nobody is listening for, at an address nothing answers on.
|
||||
@@ -2,13 +2,11 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
mod common;
|
||||
|
||||
use common::{TestAgent, config_with, network, settle, wait_event, wait_for_peers};
|
||||
use tsunagi::agent::Event;
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::identity::{NetworkName, NetworkSecret};
|
||||
use tsunagi::proto::ControlMessage;
|
||||
use tsunagi::testing::{TestAgent, config_with, network, settle, wait_event, wait_for_peers};
|
||||
use tsunagi::{Agent, Error};
|
||||
|
||||
#[tokio::test]
|
||||
+19
-4
@@ -47,13 +47,28 @@ and the agent stays manageable.
|
||||
| `storage` | mandatory state and the separately recoverable cache |
|
||||
| `state` | signed records that outlive a session, merged between replicas |
|
||||
| `dataplane::transport` | authenticated datagram links to peers; where reachability lives |
|
||||
| `dataplane` | the contract IP plugins implement, plus the WireGuard plugin |
|
||||
| `dataplane` | the contract a protocol implements, and nothing else |
|
||||
| `overlay` | the one interface an agent owns: provisioning, the TUN, whose packet is whose |
|
||||
| `dns` | the DNS view of a network, and telling the system resolver about it |
|
||||
|
||||
And in crates of their own:
|
||||
|
||||
| crate | responsibility |
|
||||
|---|---|
|
||||
| `tsunagi-wg-quic` | the `wg-quic` protocol: its keys, its announcement, its tunnels |
|
||||
| `tsunagi-cli` | the command line agent: the only place that owns a runtime, a logger and signals |
|
||||
|
||||
A protocol is a separate crate so the boundary is the compiler's to enforce,
|
||||
and so it can carry its own version. That version is not what peers compare:
|
||||
they compare the *wire* version, which moves only when the bytes do, so two
|
||||
peers on different releases keep working.
|
||||
|
||||
Abstractions exist only where something is really substituted or really needs
|
||||
isolating for tests: `NetworkDiscovery`, `IpPlugin`, `PacketTransport` /
|
||||
`PacketLink` (so the data plane's carrier can change), and `TunFactory` (which
|
||||
is what lets the whole data plane be tested without privileges). Everything
|
||||
else is a concrete type.
|
||||
`PacketLink` (so a protocol's carrier can change), `TunFactory` (which is what
|
||||
lets the whole data plane be tested without privileges), `InterfaceProvisioner`
|
||||
and `DnsPublisher` (which are where the platforms differ). Everything else is a
|
||||
concrete type.
|
||||
|
||||
## Runtime shape
|
||||
|
||||
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
# Planned state synchronisation
|
||||
|
||||
**The first slice of this model is now implemented**, in `src/state/`, and is
|
||||
**The first slice of this model is now implemented**, in `crates/tsunagi/src/state/`, and is
|
||||
used for one thing: IPv4 overlay addresses. What follows describes the whole
|
||||
model; the section at the end says exactly which parts exist.
|
||||
|
||||
@@ -77,7 +77,7 @@ migrations for this.
|
||||
|
||||
## What exists today
|
||||
|
||||
Implemented, in `src/state/`:
|
||||
Implemented, in `crates/tsunagi/src/state/`:
|
||||
|
||||
* signed records, one per author per network, each holding that author's
|
||||
complete current statement rather than a delta;
|
||||
|
||||
+3
-3
@@ -50,7 +50,7 @@ keeping the WireGuard identity, shutdown removing every interface, a forged
|
||||
overlay claim being rejected, and the core carrying the payload without
|
||||
interpreting it.
|
||||
|
||||
Unit tests in `src/state/` cover the signed record model directly: tampering
|
||||
Unit tests in `crates/tsunagi/src/state/` cover the signed record model directly: tampering
|
||||
with any field breaks verification, a newer version wins while an older one
|
||||
never rolls back, two authors claiming one address resolve the same way no
|
||||
matter the merge order, one key used in two places is reported rather than
|
||||
@@ -68,11 +68,11 @@ staying short enough to bind.
|
||||
candidate is enough to join, several backends compose, entries are withdrawn
|
||||
when a network stops, and a forgotten network stays forgotten across a restart.
|
||||
|
||||
Unit tests in `src/proto/handshake.rs` cover the transcript construction
|
||||
Unit tests in `crates/tsunagi/src/proto/handshake.rs` cover the transcript construction
|
||||
itself: role separation, channel binding, identity and network binding,
|
||||
unambiguous encoding, and rejection under the wrong key.
|
||||
|
||||
Unit tests in `src/dataplane/wireguard/` cover key clamping against the RFC
|
||||
Unit tests in `crates/tsunagi-wg-quic/src/` cover key clamping against the RFC
|
||||
7748 vector, overlay derivation, announcement validation including the
|
||||
address-hijack attempt, interface naming, and IP header parsing against
|
||||
truncated and nonsense input.
|
||||
|
||||
@@ -1,75 +0,0 @@
|
||||
//! The WireGuard data plane plugin.
|
||||
//!
|
||||
//! WireGuard is the first IP plugin. It creates real IP connectivity between
|
||||
//! participants, while the control plane keeps doing what it does: agreeing on
|
||||
//! who is in the network and carrying each participant's opaque announcement.
|
||||
//!
|
||||
//! The two planes stay separate:
|
||||
//!
|
||||
//! * **The plugin knows nothing about reachability.** It is handed a
|
||||
//! [`PacketLink`](crate::dataplane::transport::PacketLink) per peer and
|
||||
//! bridges the kernel WireGuard device onto it. Hole punching and relaying
|
||||
//! belong to the transport.
|
||||
//! * **The announcement says who, not where.** It carries a public key, so
|
||||
//! there is no address for a peer to lie about.
|
||||
//! * **The core never parses these announcements.** It moves a bounded opaque
|
||||
//! blob; only [`announcement`] interprets it.
|
||||
//! * **Keys are separate.** The plugin has its own key per network, in its own
|
||||
//! store, unrelated to the iroh device key and to the network secret.
|
||||
//! * **WireGuard's own crypto is untouched.** The bridge is a pipe; the
|
||||
//! handshake and encryption run end to end between the two kernels.
|
||||
//!
|
||||
//! # How a mesh forms
|
||||
//!
|
||||
//! Every participant derives its own overlay address from the network id and
|
||||
//! its own WireGuard public key ([`overlay`]), so no coordinator hands out
|
||||
//! addresses. Because that derivation is public, each agent computes every
|
||||
//! peer's `AllowedIPs` itself instead of believing what the peer claims — a
|
||||
//! member cannot route another member's traffic to itself.
|
||||
//!
|
||||
//! WireGuard itself is [`boringtun`]'s protocol state machine, running in this
|
||||
//! process: no kernel module, no `wg` tool, the same code on every platform.
|
||||
//! [`device::WireguardDevice`] drives one tunnel per peer and routes packets
|
||||
//! between them and a [`tun::TunDevice`].
|
||||
//!
|
||||
//! The only part that needs privileges is the packet interface. With
|
||||
//! [`tun::MemoryTunFactory`] the whole data plane — handshake, encryption,
|
||||
//! routing, address ownership — runs and is tested with no privileges at all.
|
||||
//! For real traffic there is [`provision::ManagedTunFactory`], where the
|
||||
//! agent creates and configures the interface itself over netlink and
|
||||
//! removes it again on exit.
|
||||
//!
|
||||
//! See `docs/wireguard.md` for the full picture.
|
||||
|
||||
pub mod announcement;
|
||||
pub mod config;
|
||||
pub mod device;
|
||||
pub mod keys;
|
||||
pub mod overlay;
|
||||
pub mod packet;
|
||||
pub mod plugin;
|
||||
pub mod provision;
|
||||
pub mod store;
|
||||
pub mod tun;
|
||||
|
||||
pub use crate::state::Ipv4Range;
|
||||
pub use announcement::{ValidatedAnnouncement, WgAnnouncement};
|
||||
pub use config::{Cidr, DEFAULT_INTERFACE_PREFIX, MAX_INTERFACE_NAME_LEN, interface_name};
|
||||
pub use device::{PeerHealth, PeerStats, PeerSummary, WireguardDevice};
|
||||
pub use keys::{WgPublicKey, WgSecretKey};
|
||||
pub use overlay::{OVERLAY_PREFIX_LEN, overlay_address, overlay_address_v4, overlay_prefix};
|
||||
pub use packet::IpHeader;
|
||||
pub use plugin::{
|
||||
DEFAULT_MTU, MIN_MTU, NetworkOverview, PeerOverview, WIREGUARD_OVERHEAD, WIREGUARD_PROTOCOL,
|
||||
WireguardConfig, WireguardPlugin,
|
||||
};
|
||||
pub use provision::{
|
||||
Changes, InterfacePlan, InterfaceProvisioner, InterfaceState, LinkKind, ManagedTunFactory,
|
||||
MockHost, MockProvisioner, Privilege, Provisioned, UnsupportedProvisioner, plan_changes,
|
||||
probe_net_admin,
|
||||
};
|
||||
pub use store::WgKeyStore;
|
||||
pub use tun::{MemoryTun, MemoryTunFactory, TunDevice, TunFactory, TunRequest, address_is_local};
|
||||
|
||||
#[cfg(all(feature = "tun-device", target_os = "linux"))]
|
||||
pub use provision::NetlinkProvisioner;
|
||||
@@ -1,266 +0,0 @@
|
||||
//! Deterministic overlay addressing.
|
||||
//!
|
||||
//! A mesh with no coordinator cannot hand out addresses, so every participant
|
||||
//! derives its own from values everybody already knows. The result is an IPv6
|
||||
//! unique local address (RFC 4193):
|
||||
//!
|
||||
//! ```text
|
||||
//! prefix (/64) = 0xfd || SHA-256( LP(domain) || LP("prefix") || LP(network_id) )[0..7]
|
||||
//! iid (64b) = SHA-256( LP(domain) || LP("interface") || LP(network_id) || LP(wg_public_key) )[0..8]
|
||||
//! address = prefix || iid
|
||||
//! ```
|
||||
//!
|
||||
//! Two properties matter:
|
||||
//!
|
||||
//! * Every member of a network derives the **same** `/64`, so the overlay is
|
||||
//! one subnet without anybody allocating it.
|
||||
//! * A member's address is bound to its WireGuard public key, so a peer's
|
||||
//! `AllowedIPs` can be **derived locally and never taken from what the peer
|
||||
//! claims**. A participant can mint many keys and therefore many addresses,
|
||||
//! but it cannot choose to collide with an existing member's address without
|
||||
//! finding a hash preimage.
|
||||
|
||||
use std::net::{Ipv4Addr, Ipv6Addr};
|
||||
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
use crate::state::Ipv4Range;
|
||||
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
use super::keys::WgPublicKey;
|
||||
|
||||
/// Frozen domain separator for overlay address derivation.
|
||||
pub const OVERLAY_DOMAIN: &str = "tsunagi-wireguard-overlay-v1";
|
||||
|
||||
/// Prefix length of the overlay subnet.
|
||||
pub const OVERLAY_PREFIX_LEN: u8 = 64;
|
||||
|
||||
/// Prefix length of one member's address inside the overlay.
|
||||
pub const OVERLAY_HOST_PREFIX_LEN: u8 = 128;
|
||||
|
||||
fn push_lp(out: &mut Vec<u8>, bytes: &[u8]) {
|
||||
let len = u32::try_from(bytes.len()).unwrap_or(u32::MAX);
|
||||
out.extend_from_slice(&len.to_be_bytes());
|
||||
out.extend_from_slice(bytes);
|
||||
}
|
||||
|
||||
fn digest(label: &str, network: NetworkId, key: Option<&WgPublicKey>) -> [u8; 32] {
|
||||
let mut input = Vec::with_capacity(128);
|
||||
push_lp(&mut input, OVERLAY_DOMAIN.as_bytes());
|
||||
push_lp(&mut input, label.as_bytes());
|
||||
push_lp(&mut input, network.as_bytes());
|
||||
if let Some(key) = key {
|
||||
push_lp(&mut input, key.as_bytes());
|
||||
}
|
||||
Sha256::digest(&input).into()
|
||||
}
|
||||
|
||||
/// The `/64` every member of `network` shares.
|
||||
///
|
||||
/// Returned as the network address of the prefix, i.e. with a zero interface
|
||||
/// identifier.
|
||||
pub fn overlay_prefix(network: NetworkId) -> Ipv6Addr {
|
||||
let hash = digest("prefix", network, None);
|
||||
let mut octets = [0u8; 16];
|
||||
// fd00::/8 marks a locally assigned unique local address.
|
||||
octets[0] = 0xfd;
|
||||
// 40 bits of global id followed by a 16 bit subnet id fill the rest of /64.
|
||||
octets[1..8].copy_from_slice(&hash[0..7]);
|
||||
Ipv6Addr::from(octets)
|
||||
}
|
||||
|
||||
/// The address a member with `key` has in `network`.
|
||||
pub fn overlay_address(network: NetworkId, key: &WgPublicKey) -> Ipv6Addr {
|
||||
let prefix = overlay_prefix(network).octets();
|
||||
let hash = digest("interface", network, Some(key));
|
||||
|
||||
let mut octets = [0u8; 16];
|
||||
octets[0..8].copy_from_slice(&prefix[0..8]);
|
||||
octets[8..16].copy_from_slice(&hash[0..8]);
|
||||
|
||||
// The all-zero interface identifier is the subnet-router anycast address
|
||||
// and must not be handed to a host.
|
||||
if octets[8..16] == [0u8; 8] {
|
||||
octets[15] = 1;
|
||||
}
|
||||
Ipv6Addr::from(octets)
|
||||
}
|
||||
|
||||
/// The IPv4 address a member with `key` has in `network`.
|
||||
///
|
||||
/// # Why this is weaker than the IPv6 derivation
|
||||
///
|
||||
/// A 64 bit interface identifier makes an IPv6 collision impossible in
|
||||
/// practice. IPv4 has nothing like that much room, so two members *can* derive
|
||||
/// the same address. In a `/10` with 50 members the chance is roughly 0.03%,
|
||||
/// which is small but real, so it is detected and resolved rather than
|
||||
/// assumed away — see [`super::device`]. IPv6 remains the address that always
|
||||
/// works.
|
||||
///
|
||||
/// Returns `None` when the range has no room for hosts.
|
||||
pub fn overlay_address_v4(
|
||||
network: NetworkId,
|
||||
key: &WgPublicKey,
|
||||
range: Ipv4Range,
|
||||
) -> Option<Ipv4Addr> {
|
||||
let Ipv4Range { base, prefix_len } = range;
|
||||
if prefix_len > 32 {
|
||||
return None;
|
||||
}
|
||||
let host_bits = 32 - u32::from(prefix_len);
|
||||
// A usable range needs a network address, a broadcast address and at
|
||||
// least one host between them.
|
||||
if host_bits < 2 {
|
||||
return None;
|
||||
}
|
||||
|
||||
let hash = digest("ipv4", network, Some(key));
|
||||
let raw = u32::from_be_bytes([hash[0], hash[1], hash[2], hash[3]]);
|
||||
|
||||
let usable = (1u64 << host_bits) - 2;
|
||||
let offset = (u64::from(raw) % usable) + 1;
|
||||
|
||||
let mask = if host_bits == 32 {
|
||||
0
|
||||
} else {
|
||||
u32::MAX << host_bits
|
||||
};
|
||||
let network_part = u32::from(base) & mask;
|
||||
Some(Ipv4Addr::from(network_part | offset as u32))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use super::*;
|
||||
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
|
||||
fn network(name: &str) -> NetworkId {
|
||||
NetworkKeys::derive(
|
||||
&NetworkName::new(name).unwrap(),
|
||||
&NetworkSecret::from_bytes(vec![7u8; 32]).unwrap(),
|
||||
)
|
||||
.network_id()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_prefix_is_a_unique_local_address() {
|
||||
let prefix = overlay_prefix(network("home"));
|
||||
assert_eq!(prefix.octets()[0], 0xfd);
|
||||
assert!(prefix.is_unique_local());
|
||||
assert_eq!(&prefix.octets()[8..16], &[0u8; 8], "a /64 network address");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn everyone_in_a_network_shares_one_prefix() {
|
||||
let id = network("shared");
|
||||
let a = overlay_address(id, &WgPublicKey::from_bytes([1u8; 32]));
|
||||
let b = overlay_address(id, &WgPublicKey::from_bytes([2u8; 32]));
|
||||
assert_eq!(a.octets()[0..8], b.octets()[0..8]);
|
||||
assert_ne!(a, b, "different keys get different addresses");
|
||||
assert_eq!(&overlay_prefix(id).octets()[0..8], &a.octets()[0..8]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn derivation_is_deterministic_and_network_scoped() {
|
||||
let key = WgPublicKey::from_bytes([9u8; 32]);
|
||||
let first = network("one");
|
||||
let second = network("two");
|
||||
assert_eq!(overlay_address(first, &key), overlay_address(first, &key));
|
||||
assert_ne!(
|
||||
overlay_address(first, &key),
|
||||
overlay_address(second, &key),
|
||||
"the same key in a different network gets a different address"
|
||||
);
|
||||
assert_ne!(overlay_prefix(first), overlay_prefix(second));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ipv4_addresses_land_inside_the_range_and_avoid_its_edges() {
|
||||
let id = network("v4");
|
||||
let range: Ipv4Range = "100.64.0.0/10".parse().unwrap();
|
||||
for byte in 0..64u8 {
|
||||
let key = WgPublicKey::from_bytes([byte; 32]);
|
||||
let addr = overlay_address_v4(id, &key, range).unwrap();
|
||||
let raw = u32::from(addr);
|
||||
assert_eq!(
|
||||
raw & 0xffc0_0000,
|
||||
u32::from(range.base),
|
||||
"outside 100.64.0.0/10"
|
||||
);
|
||||
// Never the network address and never the broadcast address.
|
||||
assert_ne!(raw & 0x003f_ffff, 0);
|
||||
assert_ne!(raw & 0x003f_ffff, 0x003f_ffff);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_range_parses_and_prints_round_trip() {
|
||||
let range: Ipv4Range = "10.77.0.0/16".parse().unwrap();
|
||||
assert_eq!(range.to_string(), "10.77.0.0/16");
|
||||
assert!(range.contains("10.77.3.4".parse().unwrap()));
|
||||
assert!(!range.contains("10.78.3.4".parse().unwrap()));
|
||||
|
||||
assert!("10.77.0.0".parse::<Ipv4Range>().is_err());
|
||||
assert!("nonsense/16".parse::<Ipv4Range>().is_err());
|
||||
assert!("10.77.0.0/zz".parse::<Ipv4Range>().is_err());
|
||||
assert!("10.77.0.0/31".parse::<Ipv4Range>().is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ipv4_derivation_is_deterministic_and_scoped_like_ipv6() {
|
||||
let key = WgPublicKey::from_bytes([9u8; 32]);
|
||||
let first = network("one");
|
||||
let second = network("two");
|
||||
let range: Ipv4Range = "100.64.0.0/10".parse().unwrap();
|
||||
|
||||
assert_eq!(
|
||||
overlay_address_v4(first, &key, range),
|
||||
overlay_address_v4(first, &key, range)
|
||||
);
|
||||
assert_ne!(
|
||||
overlay_address_v4(first, &key, range),
|
||||
overlay_address_v4(second, &key, range)
|
||||
);
|
||||
assert_ne!(
|
||||
overlay_address_v4(first, &key, range),
|
||||
overlay_address_v4(first, &WgPublicKey::from_bytes([10u8; 32]), range)
|
||||
);
|
||||
// A different range moves everybody.
|
||||
assert_ne!(
|
||||
overlay_address_v4(first, &key, range),
|
||||
overlay_address_v4(
|
||||
first,
|
||||
&key,
|
||||
Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 0), 8).unwrap()
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_range_with_no_room_yields_nothing() {
|
||||
let id = network("tiny");
|
||||
let key = WgPublicKey::from_bytes([1u8; 32]);
|
||||
// /31 and /32 have no usable host addresses, so they are refused.
|
||||
assert!(Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 0), 31).is_err());
|
||||
assert!(Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 1), 32).is_err());
|
||||
assert!(Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 0), 33).is_err());
|
||||
// A /30 has two usable addresses.
|
||||
let small = Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 0), 30).unwrap();
|
||||
assert!(overlay_address_v4(id, &key, small).is_some());
|
||||
// A /0 must not overflow.
|
||||
let everything = Ipv4Range::new(Ipv4Addr::UNSPECIFIED, 0).unwrap();
|
||||
assert!(overlay_address_v4(id, &key, everything).is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn addresses_are_never_the_subnet_router_anycast_address() {
|
||||
let id = network("anycast");
|
||||
for byte in 0..64u8 {
|
||||
let address = overlay_address(id, &WgPublicKey::from_bytes([byte; 32]));
|
||||
assert_ne!(&address.octets()[8..16], &[0u8; 8]);
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user