Make the overlay dual stack

Every member now also derives an IPv4 address, from the same inputs as its
IPv6 one, into 100.64.0.0/10 by default. The range is configurable and IPv4
can be turned off with --no-ipv4.

IPv4 is honestly weaker than IPv6 here and the code says so. A 64 bit
interface identifier makes an IPv6 collision impossible in practice; IPv4
has nothing like that room, and in a /10 with 50 members two will derive the
same address about 0.03% of the time. A mesh with no coordinator cannot
allocate around that, so a collision is detected and resolved instead: the
member whose public key sorts lower keeps the address, a rule every member
computes identically and therefore agrees on. The other keeps IPv6 and is
flagged in the status. IPv6 always works; IPv4 almost always works and
degrades predictably.

Routing and address-ownership enforcement now cover both families: a packet
goes to the peer that owns its destination, and a decrypted packet is
dropped unless its source is an address derived for the peer that sent it,
IPv4 included.

Six new tests, among them a real IPv4 packet crossing a tunnel next to an
IPv6 one, a spoofed IPv4 source being dropped, and an IPv6-only overlay.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
2026-09-21 13:00:35 +01:00
co-authored by Claude Opus 5
parent be459e5bd0
commit cfab38824d
12 changed files with 638 additions and 46 deletions
+40 -6
View File
@@ -31,10 +31,10 @@ A working library with **real iroh connections** and integration tests:
- status snapshots, an event stream and honest diagnostics; - status snapshots, an event stream and honest diagnostics;
- configuration restored after a restart; - configuration restored after a restart;
- correct behaviour when the disposable cache is missing or corrupt; - correct behaviour when the disposable cache is missing or corrupt;
- a **WireGuard data plane**, in userspace: its own key per network, - a **WireGuard data plane**, in userspace: its own key per network, a dual
deterministic IPv6 overlay addressing, real tunnels carried over iroh, and stack overlay with deterministically derived addresses, real tunnels carried
address ownership enforced rather than believed; over iroh, and address ownership enforced rather than believed;
- a **command line agent**, `tsunagi`. - a **command line agent**, `tsunagi`, with a local control socket.
### What it deliberately does **not** do ### What it deliberately does **not** do
@@ -94,8 +94,41 @@ wireguard: tsunkkcp43lmdje on fd15:1d9e:fa21:f201:…/64 mtu 1280, 1/1 tunnel(s)
4jO4kx9Z fd15:1d9e:fa21:f201:… handshake 3s ago tx=0 rx=0 dropped=0 path=Direct via Ip(…) 4jO4kx9Z fd15:1d9e:fa21:f201:… handshake 3s ago tx=0 rx=0 dropped=0 path=Direct via Ip(…)
``` ```
`1/1 tunnel(s) established` means a real WireGuard handshake completed. Then `1/1 tunnel(s) established` means a real WireGuard handshake completed.
`ping6` the peer's overlay address.
## Checking that it works
From another shell on either machine:
```bash
tsunagi status
```
```text
endpoint 7d76ccbbc21bf30767e14422c0494740a2cecc02aa9f82d5b8d57bdae350e7fc
hostname tsunagi-7d76ccbbc2
bound 0.0.0.0:41641
network lab (z2o4qwrvnj3zb6st2aoqg4abf342j662q2ujttqrsmz22argk2ba) active
peer b345d5271b tsunagi-b345d5271b Direct rtt 24ms
overlay tsunz2o4qwrvnj3 fd09:…:c1c6/64 and 100.110.49.177 mtu 1280 1/1 tunnel(s) up
SDsEb/WF fd09:…:c4b / 100.65.243.53 handshake 4s ago tx 0 rx 0 Direct via Ip(…)
```
`1/1 tunnel(s) up` and a recent handshake mean the tunnel is live. Then send
real traffic to the peer's overlay address:
```bash
ping6 fd09:…:c4b # or
ping 100.65.243.53
```
`tx` and `rx` in the status should start moving.
The overlay is dual stack: every member derives both an IPv6 address, which
can never collide, and an IPv4 one in `100.64.0.0/10`, which very rarely can —
see [docs/wireguard.md](docs/wireguard.md#ipv4-alongside-ipv6). `--no-ipv4`
runs IPv6 only, `--ipv4-range` moves the range.
Notes: Notes:
@@ -130,6 +163,7 @@ sudo ip tuntap add dev tsunjwc6dcrtmo5 mode tun user ab
sudo ip link set dev tsunjwc6dcrtmo5 mtu 1280 up sudo ip link set dev tsunjwc6dcrtmo5 mtu 1280 up
sudo sysctl -qw net.ipv6.conf.tsunjwc6dcrtmo5.keep_addr_on_down=1 sudo sysctl -qw net.ipv6.conf.tsunjwc6dcrtmo5.keep_addr_on_down=1
sudo ip -6 address add fd80:1210:f724:f620:d1bb:f982:3b6e:19bd/64 dev tsunjwc6dcrtmo5 nodad sudo ip -6 address add fd80:1210:f724:f620:d1bb:f982:3b6e:19bd/64 dev tsunjwc6dcrtmo5 nodad
sudo ip address add 100.110.49.177/10 dev tsunjwc6dcrtmo5
``` ```
The MTU is 1280 because that is the minimum IPv6 requires (RFC 8200). Linux The MTU is 1280 because that is the minimum IPv6 requires (RFC 8200). Linux
+5
View File
@@ -50,6 +50,11 @@ keeping the WireGuard identity, shutdown removing every interface, a forged
overlay claim being rejected, and the core carrying the payload without overlay claim being rejected, and the core carrying the payload without
interpreting it. interpreting it.
`tests/local_control.rs` covers the local control socket end to end: a client
asking a running agent for status over a real Unix socket, a leftover socket
file being replaced while a live one is not, and the derived socket path
staying short enough to bind.
`tests/discovery.rs` covers the discovery contract itself: a static bootstrap `tests/discovery.rs` covers the discovery contract itself: a static bootstrap
candidate is enough to join, several backends compose, entries are withdrawn candidate is enough to join, several backends compose, entries are withdrawn
when a network stops, and a forgotten network stays forgotten across a restart. when a network stops, and a forgotten network stays forgotten across a restart.
+33 -3
View File
@@ -56,6 +56,12 @@ direct path when it can and falls back to a relay when it cannot; the tunnel
rides on whichever it got. There is no separate STUN, no separate hole punching rides on whichever it got. There is no separate STUN, no separate hole punching
and no second set of NAT problems to solve for WireGuard. and no second set of NAT problems to solve for WireGuard.
## Checking it from outside
`tsunagi status` asks a running agent over its local control socket and prints
what it sees, including whether each tunnel has actually handshaken. See
[../README.md](../README.md#checking-that-it-works).
## Deterministic overlay addressing ## Deterministic overlay addressing
A mesh with no coordinator cannot hand out addresses, so everyone derives their A mesh with no coordinator cannot hand out addresses, so everyone derives their
@@ -77,6 +83,29 @@ Two consequences matter:
* a member's address is bound to its WireGuard public key, so address * a member's address is bound to its WireGuard public key, so address
ownership can be checked locally rather than believed. ownership can be checked locally rather than believed.
## IPv4 alongside IPv6
The overlay is dual stack by default: every member also derives an IPv4
address, from the same inputs, into `100.64.0.0/10` (RFC 6598 shared address
space — deliberately not RFC 1918, so it rarely clashes with the network the
machine is already on). The range is configurable, and IPv4 can be turned off.
**IPv4 is weaker than IPv6 here, and the difference is not cosmetic.** A 64 bit
interface identifier makes an IPv6 collision impossible in practice. IPv4 has
nothing like that much room: in a `/10` with 50 members the chance that two
derive the same address is roughly 0.03%. Small, but not zero, and a mesh with
no coordinator cannot simply allocate around it.
So a collision is detected and resolved rather than assumed away: the member
whose WireGuard public key sorts lower keeps the address, a rule every member
computes identically and therefore agrees on without exchanging anything. The
other member ends up with **no IPv4 address** and is still fully reachable over
IPv6. The status output flags it.
That is the honest summary: **IPv6 always works; IPv4 almost always works and
degrades predictably when it does not.** Allocating IPv4 properly needs the
agreed state described in [sync-model.md](sync-model.md).
## Address ownership is enforced, not announced ## Address ownership is enforced, not announced
Kernel WireGuard enforces `AllowedIPs`. In userspace that is our job, and Kernel WireGuard enforces `AllowedIPs`. In userspace that is our job, and
@@ -87,6 +116,8 @@ Kernel WireGuard enforces `AllowedIPs`. In userspace that is our job, and
* **inbound**, a decrypted packet is dropped unless its *source* is exactly the * **inbound**, a decrypted packet is dropped unless its *source* is exactly the
address derived for the peer whose tunnel decrypted it. address derived for the peer whose tunnel decrypted it.
Both apply to IPv4 and IPv6 alike.
So a participant cannot receive traffic addressed to somebody else and cannot So a participant cannot receive traffic addressed to somebody else and cannot
forge traffic that appears to come from somebody else. A participant who knows forge traffic that appears to come from somebody else. A participant who knows
the network secret can mint many keys and therefore occupy many addresses, but the network secret can mint many keys and therefore occupy many addresses, but
@@ -192,9 +223,8 @@ async fn main() -> Result<()> {
* **Full mesh only.** Every member runs a tunnel to every other member. * **Full mesh only.** Every member runs a tunnel to every other member.
Routing through an intermediate participant is not implemented. Routing through an intermediate participant is not implemented.
* **IPv6 overlay only.** Addressing is IPv6 ULA because it can be derived * **IPv4 addressing can collide.** See above: it is resolved deterministically
collision-free. An IPv4 overlay would need an allocator, which needs the and the loser keeps IPv6, but a proper allocator needs agreed state.
agreed state described in [sync-model.md](sync-model.md).
* **No routes, DNS or firewall rules.** The plugin creates its interface and * **No routes, DNS or firewall rules.** The plugin creates its interface and
nothing else. Anything beyond the overlay `/64` is the operator's business. nothing else. Anything beyond the overlay `/64` is the operator's business.
* **Membership is session-scoped.** A peer leaves the overlay when its control * **Membership is session-scoped.** A peer leaves the overlay when its control
+63 -1
View File
@@ -98,6 +98,45 @@ struct TunSetupArgs {
/// Interface MTU, matching `tsunagi up --wg-mtu`. At least 1280. /// Interface MTU, matching `tsunagi up --wg-mtu`. At least 1280.
#[arg(long)] #[arg(long)]
wg_mtu: Option<u32>, wg_mtu: Option<u32>,
/// Match `tsunagi up --no-ipv4`.
#[arg(long)]
no_ipv4: bool,
/// Match `tsunagi up --ipv4-range`.
#[arg(long, value_name = "CIDR", conflicts_with = "no_ipv4")]
ipv4_range: Option<String>,
}
/// Parses `address/prefix` into an IPv4 range.
fn parse_ipv4_range(text: &str) -> Result<(std::net::Ipv4Addr, u8), String> {
let (address, prefix) = text
.split_once('/')
.ok_or_else(|| format!("`{text}` is not an address with a prefix, e.g. 100.64.0.0/10"))?;
let address = address
.parse()
.map_err(|err| format!("`{address}` is not an IPv4 address: {err}"))?;
let prefix: u8 = prefix
.parse()
.map_err(|err| format!("`{prefix}` is not a prefix length: {err}"))?;
if prefix > 30 {
return Err(format!("a /{prefix} has no room for hosts"));
}
Ok((address, prefix))
}
/// Resolves the IPv4 overlay range from the flags.
fn resolve_ipv4_range(
no_ipv4: bool,
range: Option<&String>,
) -> Result<Option<(std::net::Ipv4Addr, u8)>, Box<dyn std::error::Error>> {
if no_ipv4 {
return Ok(None);
}
match range {
Some(text) => Ok(Some(parse_ipv4_range(text)?)),
None => Ok(Some(tsunagi::dataplane::wireguard::DEFAULT_IPV4_RANGE)),
}
} }
#[derive(Debug, Args, Clone)] #[derive(Debug, Args, Clone)]
@@ -212,6 +251,14 @@ struct UpArgs {
#[arg(long)] #[arg(long)]
wg_mtu: Option<u32>, wg_mtu: Option<u32>,
/// Run an IPv6-only overlay instead of dual stack.
#[arg(long)]
no_ipv4: bool,
/// IPv4 overlay range, as `address/prefix`. Defaults to 100.64.0.0/10.
#[arg(long, value_name = "CIDR", conflicts_with = "no_ipv4")]
ipv4_range: Option<String>,
/// How often to print a status summary, in seconds. Zero disables it. /// How often to print a status summary, in seconds. Zero disables it.
#[arg(long, default_value_t = 15)] #[arg(long, default_value_t = 15)]
status_interval: u64, status_interval: u64,
@@ -346,6 +393,7 @@ async fn status(args: StatusArgs) -> Result<(), Box<dyn std::error::Error>> {
async fn tun_setup(args: TunSetupArgs) -> Result<(), Box<dyn std::error::Error>> { async fn tun_setup(args: TunSetupArgs) -> Result<(), Box<dyn std::error::Error>> {
use tsunagi::dataplane::wireguard::{ use tsunagi::dataplane::wireguard::{
DEFAULT_MTU, OVERLAY_PREFIX_LEN, WgKeyStore, interface_name, overlay_address, DEFAULT_MTU, OVERLAY_PREFIX_LEN, WgKeyStore, interface_name, overlay_address,
overlay_address_v4,
}; };
use tsunagi::identity::NetworkKeys; use tsunagi::identity::NetworkKeys;
@@ -364,6 +412,7 @@ async fn tun_setup(args: TunSetupArgs) -> Result<(), Box<dyn std::error::Error>>
let interface = interface_name(&args.wg_prefix, network)?; let interface = interface_name(&args.wg_prefix, network)?;
let address = overlay_address(network, &key.public()); let address = overlay_address(network, &key.public());
let ipv4_range = resolve_ipv4_range(args.no_ipv4, args.ipv4_range.as_ref())?;
let mtu = args.wg_mtu.unwrap_or(DEFAULT_MTU); let mtu = args.wg_mtu.unwrap_or(DEFAULT_MTU);
let user = args.user.unwrap_or_else(|| { let user = args.user.unwrap_or_else(|| {
std::env::var("SUDO_USER") std::env::var("SUDO_USER")
@@ -373,6 +422,11 @@ async fn tun_setup(args: TunSetupArgs) -> Result<(), Box<dyn std::error::Error>>
println!("# Network {name} ({network})"); println!("# Network {name} ({network})");
println!("# Interface {interface}, address {address}/{OVERLAY_PREFIX_LEN}, mtu {mtu}"); println!("# Interface {interface}, address {address}/{OVERLAY_PREFIX_LEN}, mtu {mtu}");
if let Some((base, prefix)) = ipv4_range
&& let Some(v4) = overlay_address_v4(network, &key.public(), (base, prefix))
{
println!("# IPv4 overlay address {v4}/{prefix}");
}
println!("# Run once as root; then run `tsunagi up` as {user}."); println!("# Run once as root; then run `tsunagi up` as {user}.");
println!( println!(
"#\n\ "#\n\
@@ -386,6 +440,11 @@ async fn tun_setup(args: TunSetupArgs) -> Result<(), Box<dyn std::error::Error>>
println!("sudo ip link set dev {interface} mtu {mtu} up"); println!("sudo ip link set dev {interface} mtu {mtu} up");
println!("sudo sysctl -qw net.ipv6.conf.{interface}.keep_addr_on_down=1"); println!("sudo sysctl -qw net.ipv6.conf.{interface}.keep_addr_on_down=1");
println!("sudo ip -6 address add {address}/{OVERLAY_PREFIX_LEN} dev {interface} nodad"); println!("sudo ip -6 address add {address}/{OVERLAY_PREFIX_LEN} dev {interface} nodad");
if let Some((base, prefix)) = ipv4_range
&& let Some(v4) = overlay_address_v4(network, &key.public(), (base, prefix))
{
println!("sudo ip address add {v4}/{prefix} dev {interface}");
}
println!("\n# To check it afterwards:"); println!("\n# To check it afterwards:");
println!("ip -6 addr show dev {interface}"); println!("ip -6 addr show dev {interface}");
println!("\n# To remove it again:"); println!("\n# To remove it again:");
@@ -516,7 +575,8 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
system_tun_factory()? system_tun_factory()?
}; };
let mut wg = WireguardConfig::new(paths.state_dir.join("wireguard")) let mut wg = WireguardConfig::new(paths.state_dir.join("wireguard"))
.with_interface_prefix(args.wg_prefix.clone()); .with_interface_prefix(args.wg_prefix.clone())
.with_ipv4_range(resolve_ipv4_range(args.no_ipv4, args.ipv4_range.as_ref())?);
if let Some(mtu) = args.wg_mtu { if let Some(mtu) = args.wg_mtu {
wg = wg.with_mtu(mtu); wg = wg.with_mtu(mtu);
} }
@@ -637,6 +697,7 @@ async fn build_report(
interface: view.interface.clone(), interface: view.interface.clone(),
mtu: view.mtu, mtu: view.mtu,
address: view.overlay_address.to_string(), address: view.overlay_address.to_string(),
address_v4: view.overlay_address_v4.map(|addr| addr.to_string()),
prefix: view.overlay_prefix.to_string(), prefix: view.overlay_prefix.to_string(),
prefix_len: view.overlay_prefix_len, prefix_len: view.overlay_prefix_len,
peers: view peers: view
@@ -645,6 +706,7 @@ async fn build_report(
.map(|peer| OverlayPeerReport { .map(|peer| OverlayPeerReport {
public_key: peer.public_key.to_string(), public_key: peer.public_key.to_string(),
address: peer.overlay_address.to_string(), address: peer.overlay_address.to_string(),
address_v4: peer.overlay_address_v4.map(|addr| addr.to_string()),
handshake_secs_ago: peer handshake_secs_ago: peer
.tunnel .tunnel
.as_ref() .as_ref()
+112 -16
View File
@@ -27,7 +27,7 @@
//! what it announced. //! what it announced.
use std::collections::HashMap; use std::collections::HashMap;
use std::net::Ipv6Addr; use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use std::sync::atomic::{AtomicU32, AtomicU64, Ordering}; use std::sync::atomic::{AtomicU32, AtomicU64, Ordering};
use std::sync::{Arc, Mutex, RwLock}; use std::sync::{Arc, Mutex, RwLock};
use std::time::Duration; use std::time::Duration;
@@ -42,7 +42,7 @@ use crate::dataplane::transport::{SharedLink, TransportError};
use crate::identity::NetworkId; use crate::identity::NetworkId;
use super::keys::{WgPublicKey, WgSecretKey}; use super::keys::{WgPublicKey, WgSecretKey};
use super::overlay::overlay_address; use super::overlay::{overlay_address, overlay_address_v4};
use super::packet::IpHeader; use super::packet::IpHeader;
use super::tun::TunDevice; use super::tun::TunDevice;
@@ -109,6 +109,9 @@ struct Peer {
endpoint_id: EndpointId, endpoint_id: EndpointId,
public_key: WgPublicKey, public_key: WgPublicKey,
overlay: Ipv6Addr, overlay: Ipv6Addr,
/// The IPv4 address this peer owns, when the overlay is dual stack and
/// nobody else derived the same one.
overlay_v4: Mutex<Option<Ipv4Addr>>,
tunn: Mutex<Tunn>, tunn: Mutex<Tunn>,
link: SharedLink, link: SharedLink,
counters: Arc<PeerCounters>, counters: Arc<PeerCounters>,
@@ -168,6 +171,13 @@ pub struct PeerSummary {
pub public_key: WgPublicKey, pub public_key: WgPublicKey,
/// The overlay address this agent derived for it. /// The overlay address this agent derived for it.
pub overlay_address: Ipv6Addr, pub overlay_address: Ipv6Addr,
/// Its IPv4 overlay address, when the overlay is dual stack.
///
/// `None` with `ipv4_conflict` set means another member derived the same
/// address and won it; that peer is still fully reachable over IPv6.
pub overlay_address_v4: Option<Ipv4Addr>,
/// Whether this peer lost an IPv4 address to a derivation collision.
pub ipv4_conflict: bool,
/// Whether the tunnel has handshaken. /// Whether the tunnel has handshaken.
pub health: PeerHealth, pub health: PeerHealth,
/// Traffic counters. /// Traffic counters.
@@ -182,11 +192,15 @@ struct Inner {
network: NetworkId, network: NetworkId,
private_key: WgSecretKey, private_key: WgSecretKey,
tun: Arc<dyn TunDevice>, tun: Arc<dyn TunDevice>,
/// The IPv4 overlay range, when the overlay is dual stack.
ipv4_range: Option<(Ipv4Addr, u8)>,
peers: RwLock<HashMap<WgPublicKey, Arc<Peer>>>, peers: RwLock<HashMap<WgPublicKey, Arc<Peer>>>,
routes: RwLock<HashMap<Ipv6Addr, WgPublicKey>>, /// Both families, so one lookup routes any packet.
routes: RwLock<HashMap<IpAddr, WgPublicKey>>,
next_index: AtomicU32, next_index: AtomicU32,
unroutable: AtomicU64, unroutable: AtomicU64,
multicast: AtomicU64, multicast: AtomicU64,
ipv4_conflicts: AtomicU64,
} }
impl std::fmt::Debug for Inner { impl std::fmt::Debug for Inner {
@@ -207,16 +221,23 @@ pub struct WireguardDevice {
impl WireguardDevice { impl WireguardDevice {
/// Starts a device on top of `tun`. /// Starts a device on top of `tun`.
pub fn start(network: NetworkId, private_key: WgSecretKey, tun: Arc<dyn TunDevice>) -> Self { pub fn start(
network: NetworkId,
private_key: WgSecretKey,
tun: Arc<dyn TunDevice>,
ipv4_range: Option<(Ipv4Addr, u8)>,
) -> Self {
let inner = Arc::new(Inner { let inner = Arc::new(Inner {
network, network,
private_key, private_key,
tun, tun,
ipv4_range,
peers: RwLock::new(HashMap::new()), peers: RwLock::new(HashMap::new()),
routes: RwLock::new(HashMap::new()), routes: RwLock::new(HashMap::new()),
next_index: AtomicU32::new(1), next_index: AtomicU32::new(1),
unroutable: AtomicU64::new(0), unroutable: AtomicU64::new(0),
multicast: AtomicU64::new(0), multicast: AtomicU64::new(0),
ipv4_conflicts: AtomicU64::new(0),
}); });
let reader = tokio::spawn(read_from_os(Arc::clone(&inner))); let reader = tokio::spawn(read_from_os(Arc::clone(&inner)));
@@ -263,10 +284,12 @@ impl WireguardDevice {
); );
let overlay = overlay_address(self.inner.network, &public_key); let overlay = overlay_address(self.inner.network, &public_key);
let overlay_v4 = self.claim_ipv4(&public_key);
let peer = Arc::new(Peer { let peer = Arc::new(Peer {
endpoint_id, endpoint_id,
public_key, public_key,
overlay, overlay,
overlay_v4: Mutex::new(overlay_v4),
tunn: Mutex::new(tunn), tunn: Mutex::new(tunn),
link, link,
counters: Arc::new(PeerCounters::default()), counters: Arc::new(PeerCounters::default()),
@@ -279,7 +302,10 @@ impl WireguardDevice {
} }
write_lock(&self.inner.peers).insert(public_key, Arc::clone(&peer)); write_lock(&self.inner.peers).insert(public_key, Arc::clone(&peer));
write_lock(&self.inner.routes).insert(overlay, public_key); write_lock(&self.inner.routes).insert(IpAddr::V6(overlay), public_key);
if let Some(v4) = overlay_v4 {
write_lock(&self.inner.routes).insert(IpAddr::V4(v4), public_key);
}
// Start the handshake now instead of waiting for the next timer tick, // Start the handshake now instead of waiting for the next timer tick,
// so the tunnel is usable as soon as the link exists. // so the tunnel is usable as soon as the link exists.
@@ -287,12 +313,63 @@ impl WireguardDevice {
Ok(()) Ok(())
} }
/// Decides which IPv4 address a new peer gets, if any.
///
/// IPv4 has far too little room for a derived address to be collision
/// free. When two members derive the same one, the member whose public
/// key sorts lower keeps it — a rule every member computes identically,
/// so they all agree on the outcome without talking about it. The other
/// member simply has no IPv4 address; it is still fully reachable over
/// IPv6, which never collides.
fn claim_ipv4(&self, public_key: &WgPublicKey) -> Option<Ipv4Addr> {
let range = self.inner.ipv4_range?;
let wanted = overlay_address_v4(self.inner.network, public_key, range)?;
let holder = read_lock(&self.inner.routes)
.get(&IpAddr::V4(wanted))
.copied();
let Some(holder) = holder else {
return Some(wanted);
};
if holder == *public_key {
return Some(wanted);
}
self.inner.ipv4_conflicts.fetch_add(1, Ordering::Relaxed);
if holder.as_bytes() <= public_key.as_bytes() {
// The peer already holding it wins.
return None;
}
// The newcomer wins; take the address away from the other peer.
if let Some(loser) = read_lock(&self.inner.peers).get(&holder).cloned() {
let mut slot = match loser.overlay_v4.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
};
*slot = None;
}
Some(wanted)
}
/// Removes a peer and stops its tunnel. /// Removes a peer and stops its tunnel.
pub fn remove_peer(&self, public_key: &WgPublicKey) { pub fn remove_peer(&self, public_key: &WgPublicKey) {
if let Some(peer) = write_lock(&self.inner.peers).remove(public_key) { if let Some(peer) = write_lock(&self.inner.peers).remove(public_key) {
write_lock(&self.inner.routes).remove(&peer.overlay); let mut routes = write_lock(&self.inner.routes);
routes.remove(&IpAddr::V6(peer.overlay));
let v4 = match peer.overlay_v4.lock() {
Ok(guard) => *guard,
Err(poisoned) => *poisoned.into_inner(),
};
if let Some(v4) = v4 {
routes.remove(&IpAddr::V4(v4));
} }
} }
}
/// How many IPv4 derivation collisions have been resolved.
pub fn ipv4_conflicts(&self) -> u64 {
self.inner.ipv4_conflicts.load(Ordering::Relaxed)
}
/// Removes every peer whose key is not in `keep`. /// Removes every peer whose key is not in `keep`.
pub fn retain_peers(&self, keep: &[WgPublicKey]) { pub fn retain_peers(&self, keep: &[WgPublicKey]) {
@@ -315,14 +392,22 @@ impl WireguardDevice {
pub fn peers(&self) -> Vec<PeerSummary> { pub fn peers(&self) -> Vec<PeerSummary> {
let mut peers: Vec<PeerSummary> = read_lock(&self.inner.peers) let mut peers: Vec<PeerSummary> = read_lock(&self.inner.peers)
.values() .values()
.map(|peer| PeerSummary { .map(|peer| {
let overlay_v4 = match peer.overlay_v4.lock() {
Ok(guard) => *guard,
Err(poisoned) => *poisoned.into_inner(),
};
PeerSummary {
endpoint_id: peer.endpoint_id, endpoint_id: peer.endpoint_id,
public_key: peer.public_key, public_key: peer.public_key,
overlay_address: peer.overlay, overlay_address: peer.overlay,
overlay_address_v4: overlay_v4,
ipv4_conflict: overlay_v4.is_none() && self.inner.ipv4_range.is_some(),
health: peer.health(), health: peer.health(),
stats: peer.stats(), stats: peer.stats(),
path: peer.link.path_description(), path: peer.link.path_description(),
max_datagram: peer.link.max_datagram_size(), max_datagram: peer.link.max_datagram_size(),
}
}) })
.collect(); .collect();
peers.sort_by_key(|peer| peer.public_key); peers.sort_by_key(|peer| peer.public_key);
@@ -415,9 +500,8 @@ async fn read_from_os(inner: Arc<Inner>) {
}; };
// Route by destination: only the peer that owns that overlay address // Route by destination: only the peer that owns that overlay address
// may receive it. // may receive it. Both families go through the same table.
let Some(destination) = IpHeader::parse(&packet).and_then(|header| header.v6_destination()) let Some(destination) = IpHeader::parse(&packet).map(|header| header.destination()) else {
else {
inner.unroutable.fetch_add(1, Ordering::Relaxed); inner.unroutable.fetch_add(1, Ordering::Relaxed);
continue; continue;
}; };
@@ -488,9 +572,11 @@ async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
match tunn.decapsulate(None, input.unwrap_or(&[]), &mut scratch) { match tunn.decapsulate(None, input.unwrap_or(&[]), &mut scratch) {
TunnResult::WriteToNetwork(out) => Outcome::ToNetwork(out.len()), TunnResult::WriteToNetwork(out) => Outcome::ToNetwork(out.len()),
TunnResult::WriteToTunnelV6(out, source) => { TunnResult::WriteToTunnelV6(out, source) => {
Outcome::ToTunnel(out.len(), Some(source)) Outcome::ToTunnel(out.len(), IpAddr::V6(source))
}
TunnResult::WriteToTunnelV4(out, source) => {
Outcome::ToTunnel(out.len(), IpAddr::V4(source))
} }
TunnResult::WriteToTunnelV4(out, _) => Outcome::ToTunnel(out.len(), None),
TunnResult::Done => Outcome::Done, TunnResult::Done => Outcome::Done,
TunnResult::Err(err) => { TunnResult::Err(err) => {
tracing::trace!(?err, "wireguard decapsulation failed"); tracing::trace!(?err, "wireguard decapsulation failed");
@@ -508,9 +594,19 @@ async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
} }
Outcome::ToTunnel(len, source) => { Outcome::ToTunnel(len, source) => {
let payload = Bytes::copy_from_slice(&scratch[..len]); let payload = Bytes::copy_from_slice(&scratch[..len]);
// Enforce address ownership: a peer may only send from the // Enforce address ownership: a peer may only send from an
// address derived for its own key. // address derived for its own key, in either family.
if source != Some(peer.overlay) { let owned = match source {
IpAddr::V6(addr) => addr == peer.overlay,
IpAddr::V4(addr) => {
let held = match peer.overlay_v4.lock() {
Ok(guard) => *guard,
Err(poisoned) => *poisoned.into_inner(),
};
held == Some(addr)
}
};
if !owned {
peer.counters peer.counters
.dropped_wrong_source .dropped_wrong_source
.fetch_add(1, Ordering::Relaxed); .fetch_add(1, Ordering::Relaxed);
@@ -538,7 +634,7 @@ async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
enum Outcome { enum Outcome {
ToNetwork(usize), ToNetwork(usize),
ToTunnel(usize, Option<Ipv6Addr>), ToTunnel(usize, IpAddr),
Done, Done,
Failed, Failed,
} }
+3 -1
View File
@@ -54,7 +54,9 @@ pub use announcement::{ValidatedAnnouncement, WgAnnouncement};
pub use config::{Cidr, DEFAULT_INTERFACE_PREFIX, MAX_INTERFACE_NAME_LEN, interface_name}; pub use config::{Cidr, DEFAULT_INTERFACE_PREFIX, MAX_INTERFACE_NAME_LEN, interface_name};
pub use device::{PeerHealth, PeerStats, PeerSummary, WireguardDevice}; pub use device::{PeerHealth, PeerStats, PeerSummary, WireguardDevice};
pub use keys::{WgPublicKey, WgSecretKey}; pub use keys::{WgPublicKey, WgSecretKey};
pub use overlay::{OVERLAY_PREFIX_LEN, overlay_address, overlay_prefix}; pub use overlay::{
DEFAULT_IPV4_RANGE, OVERLAY_PREFIX_LEN, overlay_address, overlay_address_v4, overlay_prefix,
};
pub use packet::IpHeader; pub use packet::IpHeader;
pub use plugin::{ pub use plugin::{
DEFAULT_MTU, MIN_MTU, NetworkOverview, PeerOverview, WIREGUARD_OVERHEAD, WIREGUARD_PROTOCOL, DEFAULT_MTU, MIN_MTU, NetworkOverview, PeerOverview, WIREGUARD_OVERHEAD, WIREGUARD_PROTOCOL,
+110 -1
View File
@@ -20,7 +20,7 @@
//! but it cannot choose to collide with an existing member's address without //! but it cannot choose to collide with an existing member's address without
//! finding a hash preimage. //! finding a hash preimage.
use std::net::Ipv6Addr; use std::net::{Ipv4Addr, Ipv6Addr};
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
@@ -37,6 +37,13 @@ pub const OVERLAY_PREFIX_LEN: u8 = 64;
/// Prefix length of one member's address inside the overlay. /// Prefix length of one member's address inside the overlay.
pub const OVERLAY_HOST_PREFIX_LEN: u8 = 128; pub const OVERLAY_HOST_PREFIX_LEN: u8 = 128;
/// Default IPv4 overlay range: RFC 6598 shared address space.
///
/// Deliberately not RFC 1918, so it rarely collides with the home or office
/// network the machine is already on. It can collide with a carrier-grade NAT
/// that uses the same range, which is why it is configurable.
pub const DEFAULT_IPV4_RANGE: (Ipv4Addr, u8) = (Ipv4Addr::new(100, 64, 0, 0), 10);
fn push_lp(out: &mut Vec<u8>, bytes: &[u8]) { fn push_lp(out: &mut Vec<u8>, bytes: &[u8]) {
let len = u32::try_from(bytes.len()).unwrap_or(u32::MAX); let len = u32::try_from(bytes.len()).unwrap_or(u32::MAX);
out.extend_from_slice(&len.to_be_bytes()); out.extend_from_slice(&len.to_be_bytes());
@@ -85,6 +92,49 @@ pub fn overlay_address(network: NetworkId, key: &WgPublicKey) -> Ipv6Addr {
Ipv6Addr::from(octets) Ipv6Addr::from(octets)
} }
/// The IPv4 address a member with `key` has in `network`.
///
/// # Why this is weaker than the IPv6 derivation
///
/// A 64 bit interface identifier makes an IPv6 collision impossible in
/// practice. IPv4 has nothing like that much room, so two members *can* derive
/// the same address. In a `/10` with 50 members the chance is roughly 0.03%,
/// which is small but real, so it is detected and resolved rather than
/// assumed away — see [`super::device`]. IPv6 remains the address that always
/// works.
///
/// Returns `None` when the range has no room for hosts.
pub fn overlay_address_v4(
network: NetworkId,
key: &WgPublicKey,
range: (Ipv4Addr, u8),
) -> Option<Ipv4Addr> {
let (base, prefix_len) = range;
if prefix_len > 32 {
return None;
}
let host_bits = 32 - u32::from(prefix_len);
// A usable range needs a network address, a broadcast address and at
// least one host between them.
if host_bits < 2 {
return None;
}
let hash = digest("ipv4", network, Some(key));
let raw = u32::from_be_bytes([hash[0], hash[1], hash[2], hash[3]]);
let usable = (1u64 << host_bits) - 2;
let offset = (u64::from(raw) % usable) + 1;
let mask = if host_bits == 32 {
0
} else {
u32::MAX << host_bits
};
let network_part = u32::from(base) & mask;
Some(Ipv4Addr::from(network_part | offset as u32))
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)] #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
@@ -132,6 +182,65 @@ mod tests {
assert_ne!(overlay_prefix(first), overlay_prefix(second)); assert_ne!(overlay_prefix(first), overlay_prefix(second));
} }
#[test]
fn ipv4_addresses_land_inside_the_range_and_avoid_its_edges() {
let id = network("v4");
let range = DEFAULT_IPV4_RANGE;
for byte in 0..64u8 {
let key = WgPublicKey::from_bytes([byte; 32]);
let addr = overlay_address_v4(id, &key, range).unwrap();
let raw = u32::from(addr);
assert_eq!(
raw & 0xffc0_0000,
u32::from(range.0),
"outside 100.64.0.0/10"
);
// Never the network address and never the broadcast address.
assert_ne!(raw & 0x003f_ffff, 0);
assert_ne!(raw & 0x003f_ffff, 0x003f_ffff);
}
}
#[test]
fn ipv4_derivation_is_deterministic_and_scoped_like_ipv6() {
let key = WgPublicKey::from_bytes([9u8; 32]);
let first = network("one");
let second = network("two");
let range = DEFAULT_IPV4_RANGE;
assert_eq!(
overlay_address_v4(first, &key, range),
overlay_address_v4(first, &key, range)
);
assert_ne!(
overlay_address_v4(first, &key, range),
overlay_address_v4(second, &key, range)
);
assert_ne!(
overlay_address_v4(first, &key, range),
overlay_address_v4(first, &WgPublicKey::from_bytes([10u8; 32]), range)
);
// A different range moves everybody.
assert_ne!(
overlay_address_v4(first, &key, range),
overlay_address_v4(first, &key, (Ipv4Addr::new(10, 0, 0, 0), 8))
);
}
#[test]
fn a_range_with_no_room_yields_nothing() {
let id = network("tiny");
let key = WgPublicKey::from_bytes([1u8; 32]);
// /31 and /32 have no usable host addresses.
assert!(overlay_address_v4(id, &key, (Ipv4Addr::new(10, 0, 0, 0), 31)).is_none());
assert!(overlay_address_v4(id, &key, (Ipv4Addr::new(10, 0, 0, 1), 32)).is_none());
assert!(overlay_address_v4(id, &key, (Ipv4Addr::new(10, 0, 0, 0), 33)).is_none());
// A /30 has two usable addresses.
assert!(overlay_address_v4(id, &key, (Ipv4Addr::new(10, 0, 0, 0), 30)).is_some());
// A /0 must not overflow.
assert!(overlay_address_v4(id, &key, (Ipv4Addr::UNSPECIFIED, 0)).is_some());
}
#[test] #[test]
fn addresses_are_never_the_subnet_router_anycast_address() { fn addresses_are_never_the_subnet_router_anycast_address() {
let id = network("anycast"); let id = network("anycast");
+16
View File
@@ -65,6 +65,22 @@ impl IpHeader {
IpHeader::V4 { .. } => None, IpHeader::V4 { .. } => None,
} }
} }
/// The destination, whichever family it is.
pub fn destination(&self) -> std::net::IpAddr {
match self {
IpHeader::V4 { destination, .. } => std::net::IpAddr::V4(*destination),
IpHeader::V6 { destination, .. } => std::net::IpAddr::V6(*destination),
}
}
/// The source, whichever family it is.
pub fn source(&self) -> std::net::IpAddr {
match self {
IpHeader::V4 { source, .. } => std::net::IpAddr::V4(*source),
IpHeader::V6 { source, .. } => std::net::IpAddr::V6(*source),
}
}
} }
#[cfg(test)] #[cfg(test)]
+41 -3
View File
@@ -25,7 +25,7 @@
//! A failure here is reported and retried. It never stops the control plane. //! A failure here is reported and retried. It never stops the control plane.
use std::collections::{BTreeSet, HashMap}; use std::collections::{BTreeSet, HashMap};
use std::net::IpAddr; use std::net::{IpAddr, Ipv4Addr};
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::{Arc, Mutex, OnceLock}; use std::sync::{Arc, Mutex, OnceLock};
use std::time::Duration; use std::time::Duration;
@@ -43,7 +43,9 @@ use super::announcement::{ValidatedAnnouncement, WgAnnouncement};
use super::config::{DEFAULT_INTERFACE_PREFIX, interface_name}; use super::config::{DEFAULT_INTERFACE_PREFIX, interface_name};
use super::device::{PeerSummary, WireguardDevice}; use super::device::{PeerSummary, WireguardDevice};
use super::keys::{WgPublicKey, WgSecretKey}; use super::keys::{WgPublicKey, WgSecretKey};
use super::overlay::{OVERLAY_PREFIX_LEN, overlay_address, overlay_prefix}; use super::overlay::{
DEFAULT_IPV4_RANGE, OVERLAY_PREFIX_LEN, overlay_address, overlay_address_v4, overlay_prefix,
};
use super::store::WgKeyStore; use super::store::WgKeyStore;
use super::tun::{TunFactory, TunRequest}; use super::tun::{TunFactory, TunRequest};
@@ -86,6 +88,12 @@ pub struct WireguardConfig {
pub keepalive: Option<u16>, pub keepalive: Option<u16>,
/// Interface MTU. See [`DEFAULT_MTU`]. /// Interface MTU. See [`DEFAULT_MTU`].
pub mtu: u32, pub mtu: u32,
/// IPv4 overlay range, or `None` for an IPv6-only overlay.
///
/// IPv6 addresses are derived collision-free; IPv4 ones cannot be, so a
/// collision is detected and resolved deterministically instead. See
/// `docs/wireguard.md`.
pub ipv4_range: Option<(Ipv4Addr, u8)>,
/// How long to coalesce changes before reconciling. /// How long to coalesce changes before reconciling.
pub reconcile_debounce: Duration, pub reconcile_debounce: Duration,
/// How often to reconcile anyway, which is also when a packet interface /// How often to reconcile anyway, which is also when a packet interface
@@ -101,6 +109,7 @@ impl WireguardConfig {
interface_prefix: DEFAULT_INTERFACE_PREFIX.to_string(), interface_prefix: DEFAULT_INTERFACE_PREFIX.to_string(),
keepalive: Some(25), keepalive: Some(25),
mtu: DEFAULT_MTU, mtu: DEFAULT_MTU,
ipv4_range: Some(DEFAULT_IPV4_RANGE),
reconcile_debounce: Duration::from_millis(200), reconcile_debounce: Duration::from_millis(200),
reconcile_interval: Duration::from_secs(15), reconcile_interval: Duration::from_secs(15),
} }
@@ -120,6 +129,12 @@ impl WireguardConfig {
self self
} }
/// Sets the IPv4 overlay range, or disables IPv4 with `None`.
pub fn with_ipv4_range(mut self, range: Option<(Ipv4Addr, u8)>) -> Self {
self.ipv4_range = range;
self
}
/// Sets the reconciliation timings. /// Sets the reconciliation timings.
pub fn with_reconcile(mut self, debounce: Duration, interval: Duration) -> Self { pub fn with_reconcile(mut self, debounce: Duration, interval: Duration) -> Self {
self.reconcile_debounce = debounce; self.reconcile_debounce = debounce;
@@ -150,6 +165,10 @@ pub struct NetworkOverview {
pub overlay_prefix: IpAddr, pub overlay_prefix: IpAddr,
/// Prefix length of the overlay subnet. /// Prefix length of the overlay subnet.
pub overlay_prefix_len: u8, pub overlay_prefix_len: u8,
/// This agent's IPv4 overlay address, when the overlay is dual stack.
pub overlay_address_v4: Option<Ipv4Addr>,
/// The IPv4 overlay range in use.
pub ipv4_range: Option<(Ipv4Addr, u8)>,
/// Peers this agent knows about. /// Peers this agent knows about.
pub peers: Vec<PeerOverview>, pub peers: Vec<PeerOverview>,
/// Unicast packets the operating system sent to an address no peer owns. /// Unicast packets the operating system sent to an address no peer owns.
@@ -174,6 +193,8 @@ pub struct PeerOverview {
pub public_key: WgPublicKey, pub public_key: WgPublicKey,
/// The overlay address derived for it locally. /// The overlay address derived for it locally.
pub overlay_address: IpAddr, pub overlay_address: IpAddr,
/// Its IPv4 overlay address, once a tunnel exists and it won the address.
pub overlay_address_v4: Option<Ipv4Addr>,
/// Whether a data plane link to it exists. /// Whether a data plane link to it exists.
pub has_link: bool, pub has_link: bool,
/// The running tunnel, once there is a link. /// The running tunnel, once there is a link.
@@ -309,6 +330,9 @@ impl WireguardPlugin {
endpoint_id: *endpoint_id, endpoint_id: *endpoint_id,
public_key: announcement.public_key, public_key: announcement.public_key,
overlay_address: IpAddr::V6(announcement.overlay_address), overlay_address: IpAddr::V6(announcement.overlay_address),
overlay_address_v4: tunnels
.get(&announcement.public_key)
.and_then(|tunnel| tunnel.overlay_address_v4),
has_link: state.links.contains_key(endpoint_id), has_link: state.links.contains_key(endpoint_id),
tunnel: tunnels.get(&announcement.public_key).cloned(), tunnel: tunnels.get(&announcement.public_key).cloned(),
}) })
@@ -323,6 +347,12 @@ impl WireguardPlugin {
overlay_address: IpAddr::V6(overlay_address(network, &state.key.public())), overlay_address: IpAddr::V6(overlay_address(network, &state.key.public())),
overlay_prefix: IpAddr::V6(overlay_prefix(network)), overlay_prefix: IpAddr::V6(overlay_prefix(network)),
overlay_prefix_len: OVERLAY_PREFIX_LEN, overlay_prefix_len: OVERLAY_PREFIX_LEN,
overlay_address_v4: self
.worker
.config
.ipv4_range
.and_then(|range| overlay_address_v4(network, &state.key.public(), range)),
ipv4_range: self.worker.config.ipv4_range,
peers, peers,
unroutable_packets: state unroutable_packets: state
.device .device
@@ -437,10 +467,18 @@ impl Worker {
name: name.clone(), name: name.clone(),
address: overlay_address(network, &key.public()), address: overlay_address(network, &key.public()),
prefix_len: OVERLAY_PREFIX_LEN, prefix_len: OVERLAY_PREFIX_LEN,
address_v4: self.config.ipv4_range.and_then(|range| {
overlay_address_v4(network, &key.public(), range).map(|address| (address, range.1))
}),
mtu: self.config.mtu, mtu: self.config.mtu,
}; };
let tun = self.tun_factory.create(request).await?; let tun = self.tun_factory.create(request).await?;
let device = Arc::new(WireguardDevice::start(network, key, tun)); let device = Arc::new(WireguardDevice::start(
network,
key,
tun,
self.config.ipv4_range,
));
let mut shared = self.lock_shared(); let mut shared = self.lock_shared();
if let Some(state) = shared.networks.get_mut(&network) { if let Some(state) = shared.networks.get_mut(&network) {
+30 -5
View File
@@ -12,7 +12,7 @@
//! * `SystemTun`, behind the `tun-device` feature, is a real TUN interface. //! * `SystemTun`, behind the `tun-device` feature, is a real TUN interface.
//! Creating one needs `CAP_NET_ADMIN` on Linux or the equivalent elsewhere. //! Creating one needs `CAP_NET_ADMIN` on Linux or the equivalent elsewhere.
use std::net::Ipv6Addr; use std::net::{Ipv4Addr, Ipv6Addr};
use std::sync::Arc; use std::sync::Arc;
use bytes::Bytes; use bytes::Bytes;
@@ -29,6 +29,8 @@ pub struct TunRequest {
pub address: Ipv6Addr, pub address: Ipv6Addr,
/// Prefix length of the overlay subnet, so the OS routes it here. /// Prefix length of the overlay subnet, so the OS routes it here.
pub prefix_len: u8, pub prefix_len: u8,
/// The IPv4 overlay address and its prefix length, when dual stack.
pub address_v4: Option<(Ipv4Addr, u8)>,
/// Interface MTU. /// Interface MTU.
pub mtu: u32, pub mtu: u32,
} }
@@ -381,7 +383,7 @@ mod system {
/// reason: duplicate address detection can never finish with no carrier, /// reason: duplicate address detection can never finish with no carrier,
/// and the address would stay tentative and unusable. /// and the address would stay tentative and unusable.
pub fn setup_commands(request: &TunRequest, user: &str) -> Vec<String> { pub fn setup_commands(request: &TunRequest, user: &str) -> Vec<String> {
vec![ let mut commands = vec![
format!( format!(
"sudo ip tuntap add dev {} mode tun user {user}", "sudo ip tuntap add dev {} mode tun user {user}",
request.name request.name
@@ -398,7 +400,16 @@ mod system {
"sudo ip -6 address add {}/{} dev {} nodad", "sudo ip -6 address add {}/{} dev {} nodad",
request.address, request.prefix_len, request.name request.address, request.prefix_len, request.name
), ),
] ];
if let Some((address, prefix_len)) = request.address_v4 {
// IPv4 is not sensitive to carrier the way IPv6 is, so it needs
// no extra settings.
commands.push(format!(
"sudo ip address add {address}/{prefix_len} dev {}",
request.name
));
}
commands
} }
fn current_user() -> String { fn current_user() -> String {
@@ -578,7 +589,8 @@ fd559caf9652cb86321feac65c73bd84 05 40 00 08 tsun0
name: "tsun0".into(), name: "tsun0".into(),
address: "fd00::1".parse().unwrap(), address: "fd00::1".parse().unwrap(),
prefix_len: 64, prefix_len: 64,
mtu: 1100, address_v4: Some(("100.64.1.2".parse().unwrap(), 10)),
mtu: 1280,
}; };
let commands = setup_commands(&request, "someone"); let commands = setup_commands(&request, "someone");
@@ -586,11 +598,24 @@ fd559caf9652cb86321feac65c73bd84 05 40 00 08 tsun0
// must survive losing carrier, and it must not wait for duplicate // must survive losing carrier, and it must not wait for duplicate
// address detection that can never complete. // address detection that can never complete.
let joined = commands.join("\n"); let joined = commands.join("\n");
let up = joined.find("link set dev tsun0 mtu 1100 up").unwrap(); let up = joined.find("link set dev tsun0 mtu 1280 up").unwrap();
let keep = joined.find("keep_addr_on_down=1").unwrap(); let keep = joined.find("keep_addr_on_down=1").unwrap();
let add = joined.find("address add fd00::1/64").unwrap(); let add = joined.find("address add fd00::1/64").unwrap();
assert!(up < keep && keep < add, "wrong order:\n{joined}"); assert!(up < keep && keep < add, "wrong order:\n{joined}");
assert!(joined.contains("nodad")); assert!(joined.contains("nodad"));
assert!(joined.contains("user someone")); assert!(joined.contains("user someone"));
// IPv4 needs no carrier tricks, just the address.
assert!(joined.contains("ip address add 100.64.1.2/10 dev tsun0"));
// An IPv6-only overlay says nothing about IPv4.
let v6_only = TunRequest {
address_v4: None,
..request
};
assert!(
!setup_commands(&v6_only, "someone")
.join("\n")
.contains("100.64")
);
} }
} }
+14 -2
View File
@@ -136,6 +136,8 @@ pub struct OverlayReport {
pub mtu: u32, pub mtu: u32,
/// This agent's overlay address. /// This agent's overlay address.
pub address: String, pub address: String,
/// This agent's IPv4 overlay address, when the overlay is dual stack.
pub address_v4: Option<String>,
/// The subnet every member shares. /// The subnet every member shares.
pub prefix: String, pub prefix: String,
/// Prefix length of that subnet. /// Prefix length of that subnet.
@@ -155,6 +157,8 @@ pub struct OverlayPeerReport {
pub public_key: String, pub public_key: String,
/// Its overlay address. /// Its overlay address.
pub address: String, pub address: String,
/// Its IPv4 overlay address, when it has one.
pub address_v4: Option<String>,
/// Seconds since the last WireGuard handshake. /// Seconds since the last WireGuard handshake.
/// ///
/// `None` means the tunnel has never handshaken and cannot carry traffic. /// `None` means the tunnel has never handshaken and cannot carry traffic.
@@ -229,10 +233,14 @@ impl StatusReport {
let up = overlay.peers.iter().filter(|peer| peer.is_up()).count(); let up = overlay.peers.iter().filter(|peer| peer.is_up()).count();
let _ = writeln!( let _ = writeln!(
out, out,
" overlay {} {}/{} mtu {} {}/{} tunnel(s) up", " overlay {} {}/{}{} mtu {} {}/{} tunnel(s) up",
overlay.interface, overlay.interface,
overlay.address, overlay.address,
overlay.prefix_len, overlay.prefix_len,
match &overlay.address_v4 {
Some(v4) => format!(" and {v4}"),
None => String::new(),
},
overlay.mtu, overlay.mtu,
up, up,
overlay.peers.len() overlay.peers.len()
@@ -240,9 +248,13 @@ impl StatusReport {
for peer in &overlay.peers { for peer in &overlay.peers {
let _ = writeln!( let _ = writeln!(
out, out,
" {} {} {} tx {} rx {}{} {}", " {} {}{} {} tx {} rx {}{} {}",
&peer.public_key[..8.min(peer.public_key.len())], &peer.public_key[..8.min(peer.public_key.len())],
peer.address, peer.address,
match &peer.address_v4 {
Some(v4) => format!(" / {v4}"),
None => String::new(),
},
match peer.handshake_secs_ago { match peer.handshake_secs_ago {
Some(secs) => format!("handshake {secs}s ago"), Some(secs) => format!("handshake {secs}s ago"),
None => "NOT HANDSHAKEN".to_string(), None => "NOT HANDSHAKEN".to_string(),
+164 -1
View File
@@ -10,7 +10,7 @@
mod common; mod common;
use std::net::{IpAddr, Ipv6Addr}; use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
@@ -123,6 +123,24 @@ impl WgAgent {
} }
} }
/// Builds a minimal well-formed IPv4 packet.
fn ipv4_packet(source: Ipv4Addr, destination: Ipv4Addr, payload: &[u8]) -> Bytes {
let total = 20 + payload.len();
let mut packet = Vec::with_capacity(total);
packet.push((4 << 4) | 5); // version 4, header length 5 words
packet.push(0); // dscp/ecn
packet.extend_from_slice(&(total as u16).to_be_bytes());
packet.extend_from_slice(&[0, 0]); // identification
packet.extend_from_slice(&[0, 0]); // flags and fragment offset
packet.push(64); // ttl
packet.push(253); // an experimental protocol number
packet.extend_from_slice(&[0, 0]); // checksum, not verified by the overlay
packet.extend_from_slice(&source.octets());
packet.extend_from_slice(&destination.octets());
packet.extend_from_slice(payload);
Bytes::from(packet)
}
/// Builds a minimal well-formed IPv6 packet. /// Builds a minimal well-formed IPv6 packet.
fn ipv6_packet(source: Ipv6Addr, destination: Ipv6Addr, payload: &[u8]) -> Bytes { fn ipv6_packet(source: Ipv6Addr, destination: Ipv6Addr, payload: &[u8]) -> Bytes {
let mut packet = Vec::with_capacity(40 + payload.len()); let mut packet = Vec::with_capacity(40 + payload.len());
@@ -244,6 +262,151 @@ async fn a_peer_cannot_send_from_an_address_it_does_not_own() {
b.shutdown().await; b.shutdown().await;
} }
#[tokio::test]
async fn the_overlay_carries_ipv4_alongside_ipv6() {
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("wg-dual-stack");
let a = WgAgent::spawn(&discovery, "ta").await;
let b = WgAgent::spawn(&discovery, "tb").await;
let network_id = a.agent.join_network(&name, &secret).await.unwrap();
b.agent.join_network(&name, &secret).await.unwrap();
a.wait_for_tunnels(network_id, 1).await;
b.wait_for_tunnels(network_id, 1).await;
let view_a = a.plugin.overview(network_id).unwrap();
let view_b = b.plugin.overview(network_id).unwrap();
let v4_a = view_a.overlay_address_v4.expect("dual stack by default");
let v4_b = view_b.overlay_address_v4.expect("dual stack by default");
assert_ne!(v4_a, v4_b);
// Both inside the configured range.
for addr in [v4_a, v4_b] {
assert_eq!(
u32::from(addr) & 0xffc0_0000,
u32::from(Ipv4Addr::new(100, 64, 0, 0))
);
}
// Each side derived the other's address identically.
assert_eq!(view_a.peers[0].overlay_address_v4, Some(v4_b));
assert_eq!(view_b.peers[0].overlay_address_v4, Some(v4_a));
assert!(!view_a.peers[0].tunnel.as_ref().unwrap().ipv4_conflict);
let tun_a = a.tun(network_id).await;
let tun_b = b.tun(network_id).await;
// A real IPv4 packet through the same tunnel.
tun_a.push_from_os(ipv4_packet(v4_a, v4_b, b"ipv4 over the overlay"));
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
.await
.expect("the IPv4 packet should arrive")
.unwrap();
assert_eq!(received[0] >> 4, 4, "still an IPv4 packet");
assert_eq!(&received[12..16], &v4_a.octets());
assert_eq!(&received[16..20], &v4_b.octets());
assert_eq!(&received[20..], b"ipv4 over the overlay");
// IPv6 keeps working on the same tunnel.
let addr_a = a.overlay(network_id).await;
let addr_b = b.overlay(network_id).await;
tun_a.push_from_os(ipv6_packet(addr_a, addr_b, b"and ipv6 too"));
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
.await
.expect("the IPv6 packet should arrive")
.unwrap();
assert_eq!(received[0] >> 4, 6);
assert_eq!(&received[40..], b"and ipv6 too");
a.shutdown().await;
b.shutdown().await;
}
#[tokio::test]
async fn an_ipv4_source_a_peer_does_not_own_is_dropped() {
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("wg-v4-spoof");
let a = WgAgent::spawn(&discovery, "ta").await;
let b = WgAgent::spawn(&discovery, "tb").await;
let network_id = a.agent.join_network(&name, &secret).await.unwrap();
b.agent.join_network(&name, &secret).await.unwrap();
a.wait_for_tunnels(network_id, 1).await;
b.wait_for_tunnels(network_id, 1).await;
let v4_a = a
.plugin
.overview(network_id)
.unwrap()
.overlay_address_v4
.unwrap();
let v4_b = b
.plugin
.overview(network_id)
.unwrap()
.overlay_address_v4
.unwrap();
let tun_a = a.tun(network_id).await;
let tun_b = b.tun(network_id).await;
// A claims an IPv4 address that is not the one derived from its key.
let forged = Ipv4Addr::from(u32::from(v4_a) ^ 0x0000_00ff);
tun_a.push_from_os(ipv4_packet(forged, v4_b, b"spoofed v4"));
wait_until("the spoofed IPv4 packet is dropped", || async {
let view = b.plugin.overview(network_id)?;
let tunnel = view.peers.first()?.tunnel.as_ref()?;
(tunnel.stats.dropped_wrong_source >= 1).then_some(())
})
.await;
// The honest one still gets through.
tun_a.push_from_os(ipv4_packet(v4_a, v4_b, b"honest v4"));
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
.await
.expect("the honest packet should arrive")
.unwrap();
assert_eq!(&received[20..], b"honest v4");
a.shutdown().await;
b.shutdown().await;
}
#[tokio::test]
async fn an_ipv6_only_overlay_can_be_asked_for() {
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("wg-v6-only");
let a = WgAgent::spawn_with(&discovery, "ta", |config| config.with_ipv4_range(None)).await;
let b = WgAgent::spawn_with(&discovery, "tb", |config| config.with_ipv4_range(None)).await;
let network_id = a.agent.join_network(&name, &secret).await.unwrap();
b.agent.join_network(&name, &secret).await.unwrap();
a.wait_for_tunnels(network_id, 1).await;
let view = a.plugin.overview(network_id).unwrap();
assert_eq!(view.overlay_address_v4, None);
assert_eq!(view.ipv4_range, None);
assert_eq!(view.peers[0].overlay_address_v4, None);
// No IPv4 configured is not a conflict.
assert!(!view.peers[0].tunnel.as_ref().unwrap().ipv4_conflict);
// IPv6 is unaffected.
let addr_a = a.overlay(network_id).await;
let addr_b = b.overlay(network_id).await;
a.tun(network_id)
.await
.push_from_os(ipv6_packet(addr_a, addr_b, b"v6 only"));
let received = tokio::time::timeout(common::DEADLINE, b.tun(network_id).await.pop_to_os())
.await
.expect("the packet should arrive")
.unwrap();
assert_eq!(&received[40..], b"v6 only");
a.shutdown().await;
b.shutdown().await;
}
#[tokio::test] #[tokio::test]
async fn packets_for_an_unknown_address_are_counted_not_broadcast() { async fn packets_for_an_unknown_address_are_counted_not_broadcast() {
let discovery = SharedMemoryDiscovery::new(); let discovery = SharedMemoryDiscovery::new();