ab/tsunagi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Drop IPv6 from the overlay

Browse Source 
The overlay address was derived from a WireGuard key, which makes it the
protocol's address — and the whole point of the interface belonging to
the system level is that every protocol carries traffic for the *same*
addresses. A derived-per-protocol address cannot be that.

So the overlay is IPv4 only: allocated at the system level, signed by the
member that holds it, and the same address whichever protocol happens to
be moving the packets. The derivation, its ULA prefix and its constants
are gone, along with the collision rule that existed only because a
derived IPv4 address has too little room to be unique — an allocated one
is unique by construction.

A real loss came with it and is restored explicitly. The announcement was
bound to its network only as a side effect of checking the derived
address, so removing that check removed the binding. It now carries the
network id and rejects a mismatch. Strictly redundant, because a
capability arrives on a session that already proved membership, and kept
because losing a property silently is the wrong way to lose one.

An unlock falls out: the MTU floor of 1280 existed because Linux tears
IPv6 down below it. Without IPv6 the floor is 576, what every IPv4 host
must be able to reassemble, so a relayed path with small datagrams can be
matched rather than warned about. The default stays 1280.

The test that forged an overlay address now forges a network id, which is
what is left to lie about. One flaky assertion fixed while passing: it
waited for "the interface has some address", which is briefly true of the
leftover it was meant to see replaced.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
2026-09-21 18:40:49 +01:00
co-authored by Claude Opus 5
parent 5bee148497
commit 745bbaea06
13 changed files with 193 additions and 624 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/tsunagi-cli/src/main.rs
+18 -23
View File
@@ -1152,7 +1152,7 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve
let row = entry(&mut rows, &peer.endpoint_id);
row.tunnel = Some(peer);
if row.overlay_address_v4.is_none() {
row.overlay_address_v4 = peer.address_v4.as_deref();
row.overlay_address_v4 = peer.address.as_deref();
}
}
}
@@ -1197,13 +1197,11 @@ fn network_section(network: &tsunagi::ipc::NetworkReport, own_id: &str) -> repor
Health::Info,
"overlay",
format!(
"{} {}/{}{} mtu {}",
"{} {} mtu {}",
overlay.interface,
overlay.address,
overlay.prefix_len,
match &overlay.address_v4 {
Some(v4) => format!(" and {v4}"),
None => String::new(),
match &overlay.address {
Some(address) => format!("{address}/{}", overlay.prefix_len),
None => "no address agreed yet".to_string(),
},
overlay.mtu
),
@@ -2055,18 +2053,15 @@ async fn build_report(
.map(|view| OverlayReport {
interface: view.interface.clone(),
mtu: view.mtu,
address: view.overlay_address.to_string(),
address_v4: view.overlay_address_v4.map(|addr| addr.to_string()),
prefix: view.overlay_prefix.to_string(),
prefix_len: view.overlay_prefix_len,
address: view.overlay_address_v4.map(|addr| addr.to_string()),
prefix_len: view.ipv4_range.map_or(0, |range| range.prefix_len),
peers: view
.peers
.iter()
.map(|peer| OverlayPeerReport {
endpoint_id: peer.endpoint_id.to_string(),
public_key: peer.public_key.to_string(),
address: peer.overlay_address.to_string(),
address_v4: peer.overlay_address_v4.map(|addr| addr.to_string()),
address: peer.overlay_address_v4.map(|addr| addr.to_string()),
handshake_secs_ago: peer
.tunnel
.as_ref()
@@ -2283,8 +2278,9 @@ async fn print_status(agent: &Agent, network: NetworkId, wireguard: Option<&Wire
println!(
"wireguard: {} on {}/{} mtu {}, {}/{} tunnel(s) established",
view.interface,
view.overlay_address,
view.overlay_prefix_len,
view.overlay_address_v4
.map_or_else(|| "no address yet".to_string(), |addr| addr.to_string()),
view.ipv4_range.map_or(0, |range| range.prefix_len),
view.mtu,
view.established_peers(),
view.peers.len()
@@ -2294,7 +2290,8 @@ async fn print_status(agent: &Agent, network: NetworkId, wireguard: Option<&Wire
Some(tunnel) => println!(
" {} {} {} tx={} rx={} dropped={} path={}",
peer.public_key.fmt_short(),
peer.overlay_address,
peer.overlay_address_v4
.map_or_else(|| "no address".to_string(), |addr| addr.to_string()),
match tunnel.health.since_handshake {
Some(since) => format!("handshake {}s ago", since.as_secs()),
None => "NOT HANDSHAKEN".to_string(),
@@ -2307,7 +2304,8 @@ async fn print_status(agent: &Agent, network: NetworkId, wireguard: Option<&Wire
None => println!(
" {} {} waiting for a data link",
peer.public_key.fmt_short(),
peer.overlay_address
peer.overlay_address_v4
.map_or_else(|| "no address".to_string(), |addr| addr.to_string())
),
}
}
@@ -2337,10 +2335,8 @@ mod status_tests {
OverlayReport {
interface: "tsundemo".into(),
mtu: 1280,
address: "fd55::1".into(),
address_v4: Some("10.13.37.69".into()),
prefix: "fd55::".into(),
prefix_len: 64,
address: Some("10.13.37.69".into()),
prefix_len: 24,
peers,
..Default::default()
}
@@ -2350,8 +2346,7 @@ mod status_tests {
OverlayPeerReport {
endpoint_id: endpoint_id.into(),
public_key: "keykeykey".into(),
address: "fd55::2".into(),
address_v4: Some("10.13.37.237".into()),
address: Some("10.13.37.237".into()),
handshake_secs_ago: handshake,
tx_packets: 32,
rx_packets: 887,
crates/tsunagi/src/dataplane/wireguard/announcement.rs
+35 -34
View File
@@ -10,15 +10,12 @@
//! [`crate::dataplane::transport`]. A plugin that also tried to advertise
//! addresses would be reimplementing NAT traversal badly.
use std::net::Ipv6Addr;
use serde::{Deserialize, Serialize};
use crate::dataplane::PluginError;
use crate::identity::NetworkId;
use super::keys::WgPublicKey;
use super::overlay::overlay_address;
/// Version of the announcement format.
///
@@ -26,20 +23,27 @@ use super::overlay::overlay_address;
/// signed records in [`crate::state`], which carry the range and survive a
/// participant being away. postcard is not self-describing, so an older peer
/// cannot read a newer announcement; the mismatch is reported, not misparsed.
pub const ANNOUNCEMENT_VERSION: u16 = 3;
pub const ANNOUNCEMENT_VERSION: u16 = 4;
/// What one participant advertises for the WireGuard data plane.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct WgAnnouncement {
/// Announcement format version.
pub version: u16,
/// The peer's WireGuard public key. Its overlay address is derived from it.
pub public_key: [u8; 32],
/// The overlay address the peer believes it has.
/// The peer's WireGuard public key.
///
/// Carried for diagnostics and cross-checking only. Addresses are always
/// derived locally, never taken from this field.
pub overlay_address: Ipv6Addr,
/// The whole announcement, now that addresses belong to the system
/// level: this says *who* is at the other end of a tunnel, and nothing
/// about where.
pub public_key: [u8; 32],
/// The network this key is for.
///
/// Strictly redundant — a capability arrives on a session that already
/// proved membership of one network — and kept anyway, because the
/// binding used to be a side effect of checking a derived address and
/// losing it silently when that check went would be the wrong way to
/// lose it.
pub network: [u8; 32],
}
/// A peer announcement that has been validated against a specific network.
@@ -47,8 +51,6 @@ pub struct WgAnnouncement {
pub struct ValidatedAnnouncement {
/// The peer's WireGuard public key.
pub public_key: WgPublicKey,
/// The overlay address derived locally for this key. Authoritative.
pub overlay_address: Ipv6Addr,
}
impl WgAnnouncement {
@@ -57,7 +59,7 @@ impl WgAnnouncement {
Self {
version: ANNOUNCEMENT_VERSION,
public_key: *public_key.as_bytes(),
overlay_address: overlay_address(network, public_key),
network: *network.as_bytes(),
}
}
@@ -105,20 +107,13 @@ impl WgAnnouncement {
"peer announced this agent's own WireGuard key".into(),
));
}
// AllowedIPs are derived, never trusted. A mismatch means the peer is
// confused or lying, and either way its own claim is discarded.
let derived = overlay_address(network, &public_key);
if self.overlay_address != derived {
if self.network != *network.as_bytes() {
return Err(PluginError::Rejected(
"announced overlay address does not match the one derived from the peer's key"
.into(),
"announcement is for a different network".into(),
));
}
Ok(ValidatedAnnouncement {
public_key,
overlay_address: derived,
})
Ok(ValidatedAnnouncement { public_key })
}
}
@@ -149,7 +144,6 @@ mod tests {
let validated = WgAnnouncement::decode_and_validate(&payload, id, &local).unwrap();
assert_eq!(validated.public_key, peer);
assert_eq!(validated.overlay_address, overlay_address(id, &peer));
}
#[test]
@@ -167,20 +161,27 @@ mod tests {
}
#[test]
fn allowed_ips_are_derived_not_taken_from_the_peer() {
fn there_is_nothing_address_like_to_forge() {
// Addresses belong to the system level, are allocated there and are
// signed by the member that holds one. A protocol announcement
// carries no address at all, so this is not a thing a peer can lie
// about here — and a peer sending traffic from an address it does
// not hold is rejected by the agreed address, not by anything it
// said in this message.
let id = network("no-hijack");
let victim = WgSecretKey::generate().public();
let attacker = WgSecretKey::generate().public();
let peer = WgSecretKey::generate().public();
let local = WgSecretKey::generate().public();
// An attacker claims the victim's overlay address with its own key.
let mut forged = WgAnnouncement::new(id, &attacker);
forged.overlay_address = overlay_address(id, &victim);
let announcement = WgAnnouncement::new(id, &peer);
let validated =
WgAnnouncement::decode_and_validate(&announcement.encode().unwrap(), id, &local)
.unwrap();
assert_eq!(validated.public_key, peer);
let result = WgAnnouncement::decode_and_validate(&forged.encode().unwrap(), id, &local);
assert!(
matches!(result, Err(PluginError::Rejected(ref reason)) if reason.contains("does not match")),
"claiming another member's overlay address must be rejected: {result:?}"
// The validated form has one field, and it is an identity.
assert_eq!(
std::mem::size_of_val(&validated),
std::mem::size_of::<WgPublicKey>()
);
}
crates/tsunagi/src/dataplane/wireguard/device.rs
+16 -70
View File
@@ -27,7 +27,7 @@
//! what it announced.
use std::collections::HashMap;
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use std::net::{IpAddr, Ipv4Addr};
use std::sync::atomic::{AtomicU32, AtomicU64, Ordering};
use std::sync::{Arc, Mutex, RwLock};
use std::time::Duration;
@@ -42,10 +42,8 @@ use crate::dataplane::transport::{SharedLink, TransportError};
use crate::identity::NetworkId;
use super::keys::{WgPublicKey, WgSecretKey};
use super::overlay::overlay_address;
use crate::overlay::packet::IpHeader;
use crate::overlay::tun::TunDevice;
use crate::state::Ipv4Range;
/// How often WireGuard's own timers are driven.
///
@@ -109,9 +107,11 @@ impl PeerHealth {
struct Peer {
endpoint_id: EndpointId,
public_key: WgPublicKey,
overlay: Ipv6Addr,
/// The IPv4 address this peer owns, when the overlay is dual stack and
/// nobody else derived the same one.
/// The overlay address this peer holds, as the control plane agreed it.
///
/// Not derived here and not taken from the peer: the system level
/// allocates it, the peer signs the claim, and every protocol carries
/// traffic for the same address.
overlay_v4: Mutex<Option<Ipv4Addr>>,
tunn: Mutex<Tunn>,
link: SharedLink,
@@ -124,7 +124,6 @@ impl std::fmt::Debug for Peer {
f.debug_struct("Peer")
.field("peer", &self.endpoint_id.fmt_short().to_string())
.field("public_key", &self.public_key)
.field("overlay", &self.overlay)
.finish()
}
}
@@ -170,15 +169,8 @@ pub struct PeerSummary {
pub endpoint_id: EndpointId,
/// The peer's WireGuard public key.
pub public_key: WgPublicKey,
/// The overlay address this agent derived for it.
pub overlay_address: Ipv6Addr,
/// Its IPv4 overlay address, when the overlay is dual stack.
///
/// `None` with `ipv4_conflict` set means another member derived the same
/// address and won it; that peer is still fully reachable over IPv6.
/// The overlay address it holds, once the control plane has agreed one.
pub overlay_address_v4: Option<Ipv4Addr>,
/// Whether this peer lost an IPv4 address to a derivation collision.
pub ipv4_conflict: bool,
/// Whether the tunnel has handshaken.
pub health: PeerHealth,
/// Traffic counters.
@@ -194,7 +186,6 @@ struct Inner {
private_key: WgSecretKey,
tun: Arc<dyn TunDevice>,
/// The IPv4 overlay range, when the overlay is dual stack.
ipv4_range: Option<Ipv4Range>,
peers: RwLock<HashMap<WgPublicKey, Arc<Peer>>>,
/// Both families, so one lookup routes any packet.
routes: RwLock<HashMap<IpAddr, WgPublicKey>>,
@@ -224,17 +215,11 @@ pub struct WireguardDevice {
impl WireguardDevice {
/// Starts a device on top of `tun`.
pub fn start(
network: NetworkId,
private_key: WgSecretKey,
tun: Arc<dyn TunDevice>,
ipv4_range: Option<Ipv4Range>,
) -> Self {
pub fn start(network: NetworkId, private_key: WgSecretKey, tun: Arc<dyn TunDevice>) -> Self {
let inner = Arc::new(Inner {
network,
private_key,
tun,
ipv4_range,
peers: RwLock::new(HashMap::new()),
routes: RwLock::new(HashMap::new()),
next_index: AtomicU32::new(1),
@@ -292,12 +277,9 @@ impl WireguardDevice {
None,
);
let overlay = overlay_address(self.inner.network, &public_key);
let overlay_v4 = self.claim_ipv4(&public_key, overlay_v4);
let peer = Arc::new(Peer {
endpoint_id,
public_key,
overlay,
overlay_v4: Mutex::new(overlay_v4),
tunn: Mutex::new(tunn),
link,
@@ -311,7 +293,6 @@ impl WireguardDevice {
}
write_lock(&self.inner.peers).insert(public_key, Arc::clone(&peer));
write_lock(&self.inner.routes).insert(IpAddr::V6(overlay), public_key);
if let Some(v4) = overlay_v4 {
write_lock(&self.inner.routes).insert(IpAddr::V4(v4), public_key);
}
@@ -322,48 +303,10 @@ impl WireguardDevice {
Ok(())
}
/// Decides which IPv4 address a new peer gets, if any.
///
/// IPv4 has far too little room for a derived address to be collision
/// free. When two members derive the same one, the member whose public
/// key sorts lower keeps it — a rule every member computes identically,
/// so they all agree on the outcome without talking about it. The other
/// member simply has no IPv4 address; it is still fully reachable over
/// IPv6, which never collides.
fn claim_ipv4(&self, public_key: &WgPublicKey, wanted: Option<Ipv4Addr>) -> Option<Ipv4Addr> {
let wanted = wanted?;
let holder = read_lock(&self.inner.routes)
.get(&IpAddr::V4(wanted))
.copied();
let Some(holder) = holder else {
return Some(wanted);
};
if holder == *public_key {
return Some(wanted);
}
self.inner.ipv4_conflicts.fetch_add(1, Ordering::Relaxed);
if holder.as_bytes() <= public_key.as_bytes() {
// The peer already holding it wins.
return None;
}
// The newcomer wins; take the address away from the other peer.
if let Some(loser) = read_lock(&self.inner.peers).get(&holder).cloned() {
let mut slot = match loser.overlay_v4.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
};
*slot = None;
}
Some(wanted)
}
/// Removes a peer and stops its tunnel.
pub fn remove_peer(&self, public_key: &WgPublicKey) {
if let Some(peer) = write_lock(&self.inner.peers).remove(public_key) {
let mut routes = write_lock(&self.inner.routes);
routes.remove(&IpAddr::V6(peer.overlay));
let v4 = match peer.overlay_v4.lock() {
Ok(guard) => *guard,
Err(poisoned) => *poisoned.into_inner(),
@@ -408,9 +351,7 @@ impl WireguardDevice {
PeerSummary {
endpoint_id: peer.endpoint_id,
public_key: peer.public_key,
overlay_address: peer.overlay,
overlay_address_v4: overlay_v4,
ipv4_conflict: overlay_v4.is_none() && self.inner.ipv4_range.is_some(),
health: peer.health(),
stats: peer.stats(),
path: peer.link.path_description(),
@@ -623,10 +564,12 @@ async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
}
Outcome::ToTunnel(len, source) => {
let payload = Bytes::copy_from_slice(&scratch[..len]);
// Enforce address ownership: a peer may only send from an
// address derived for its own key, in either family.
// Enforce address ownership: a peer may only send from
// the address the control plane agreed it holds. An
// overlay address is signed by its holder, so this is
// checked against the agreement and never against
// anything the peer said here.
let owned = match source {
IpAddr::V6(addr) => addr == peer.overlay,
IpAddr::V4(addr) => {
let held = match peer.overlay_v4.lock() {
Ok(guard) => *guard,
@@ -634,6 +577,9 @@ async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
};
held == Some(addr)
}
// The overlay is IPv4. Anything else has no owner
// here and is dropped rather than guessed at.
IpAddr::V6(_) => false,
};
if !owned {
peer.counters
crates/tsunagi/src/dataplane/wireguard/mod.rs
-2
View File
@@ -44,7 +44,6 @@
pub mod announcement;
pub mod device;
pub mod keys;
pub mod overlay;
pub mod plugin;
pub mod store;
@@ -64,7 +63,6 @@ pub use crate::overlay::{
};
pub use device::{PeerHealth, PeerStats, PeerSummary, WireguardDevice};
pub use keys::{WgPublicKey, WgSecretKey};
pub use overlay::{OVERLAY_PREFIX_LEN, overlay_address, overlay_address_v4, overlay_prefix};
pub use plugin::{
DEFAULT_MTU, MIN_MTU, NetworkOverview, PeerOverview, WIREGUARD_OVERHEAD, WIREGUARD_PROTOCOL,
WireguardConfig, WireguardPlugin,
crates/tsunagi/src/dataplane/wireguard/overlay.rs
-266
View File
@@ -1,266 +0,0 @@
//! Deterministic overlay addressing.
//!
//! A mesh with no coordinator cannot hand out addresses, so every participant
//! derives its own from values everybody already knows. The result is an IPv6
//! unique local address (RFC 4193):
//!
//! ```text
//! prefix (/64) = 0xfd || SHA-256( LP(domain) || LP("prefix") || LP(network_id) )[0..7]
//! iid (64b) = SHA-256( LP(domain) || LP("interface") || LP(network_id) || LP(wg_public_key) )[0..8]
//! address = prefix || iid
//! ```
//!
//! Two properties matter:
//!
//! * Every member of a network derives the **same** `/64`, so the overlay is
//! one subnet without anybody allocating it.
//! * A member's address is bound to its WireGuard public key, so a peer's
//! `AllowedIPs` can be **derived locally and never taken from what the peer
//! claims**. A participant can mint many keys and therefore many addresses,
//! but it cannot choose to collide with an existing member's address without
//! finding a hash preimage.
use std::net::{Ipv4Addr, Ipv6Addr};
use sha2::{Digest, Sha256};
use crate::state::Ipv4Range;
use crate::identity::NetworkId;
use super::keys::WgPublicKey;
/// Frozen domain separator for overlay address derivation.
pub const OVERLAY_DOMAIN: &str = "tsunagi-wireguard-overlay-v1";
/// Prefix length of the overlay subnet.
pub const OVERLAY_PREFIX_LEN: u8 = 64;
/// Prefix length of one member's address inside the overlay.
pub const OVERLAY_HOST_PREFIX_LEN: u8 = 128;
fn push_lp(out: &mut Vec<u8>, bytes: &[u8]) {
let len = u32::try_from(bytes.len()).unwrap_or(u32::MAX);
out.extend_from_slice(&len.to_be_bytes());
out.extend_from_slice(bytes);
}
fn digest(label: &str, network: NetworkId, key: Option<&WgPublicKey>) -> [u8; 32] {
let mut input = Vec::with_capacity(128);
push_lp(&mut input, OVERLAY_DOMAIN.as_bytes());
push_lp(&mut input, label.as_bytes());
push_lp(&mut input, network.as_bytes());
if let Some(key) = key {
push_lp(&mut input, key.as_bytes());
}
Sha256::digest(&input).into()
}
/// The `/64` every member of `network` shares.
///
/// Returned as the network address of the prefix, i.e. with a zero interface
/// identifier.
pub fn overlay_prefix(network: NetworkId) -> Ipv6Addr {
let hash = digest("prefix", network, None);
let mut octets = [0u8; 16];
// fd00::/8 marks a locally assigned unique local address.
octets[0] = 0xfd;
// 40 bits of global id followed by a 16 bit subnet id fill the rest of /64.
octets[1..8].copy_from_slice(&hash[0..7]);
Ipv6Addr::from(octets)
}
/// The address a member with `key` has in `network`.
pub fn overlay_address(network: NetworkId, key: &WgPublicKey) -> Ipv6Addr {
let prefix = overlay_prefix(network).octets();
let hash = digest("interface", network, Some(key));
let mut octets = [0u8; 16];
octets[0..8].copy_from_slice(&prefix[0..8]);
octets[8..16].copy_from_slice(&hash[0..8]);
// The all-zero interface identifier is the subnet-router anycast address
// and must not be handed to a host.
if octets[8..16] == [0u8; 8] {
octets[15] = 1;
}
Ipv6Addr::from(octets)
}
/// The IPv4 address a member with `key` has in `network`.
///
/// # Why this is weaker than the IPv6 derivation
///
/// A 64 bit interface identifier makes an IPv6 collision impossible in
/// practice. IPv4 has nothing like that much room, so two members *can* derive
/// the same address. In a `/10` with 50 members the chance is roughly 0.03%,
/// which is small but real, so it is detected and resolved rather than
/// assumed away — see [`super::device`]. IPv6 remains the address that always
/// works.
///
/// Returns `None` when the range has no room for hosts.
pub fn overlay_address_v4(
network: NetworkId,
key: &WgPublicKey,
range: Ipv4Range,
) -> Option<Ipv4Addr> {
let Ipv4Range { base, prefix_len } = range;
if prefix_len > 32 {
return None;
}
let host_bits = 32 - u32::from(prefix_len);
// A usable range needs a network address, a broadcast address and at
// least one host between them.
if host_bits < 2 {
return None;
}
let hash = digest("ipv4", network, Some(key));
let raw = u32::from_be_bytes([hash[0], hash[1], hash[2], hash[3]]);
let usable = (1u64 << host_bits) - 2;
let offset = (u64::from(raw) % usable) + 1;
let mask = if host_bits == 32 {
0
} else {
u32::MAX << host_bits
};
let network_part = u32::from(base) & mask;
Some(Ipv4Addr::from(network_part | offset as u32))
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
fn network(name: &str) -> NetworkId {
NetworkKeys::derive(
&NetworkName::new(name).unwrap(),
&NetworkSecret::from_bytes(vec![7u8; 32]).unwrap(),
)
.network_id()
}
#[test]
fn the_prefix_is_a_unique_local_address() {
let prefix = overlay_prefix(network("home"));
assert_eq!(prefix.octets()[0], 0xfd);
assert!(prefix.is_unique_local());
assert_eq!(&prefix.octets()[8..16], &[0u8; 8], "a /64 network address");
}
#[test]
fn everyone_in_a_network_shares_one_prefix() {
let id = network("shared");
let a = overlay_address(id, &WgPublicKey::from_bytes([1u8; 32]));
let b = overlay_address(id, &WgPublicKey::from_bytes([2u8; 32]));
assert_eq!(a.octets()[0..8], b.octets()[0..8]);
assert_ne!(a, b, "different keys get different addresses");
assert_eq!(&overlay_prefix(id).octets()[0..8], &a.octets()[0..8]);
}
#[test]
fn derivation_is_deterministic_and_network_scoped() {
let key = WgPublicKey::from_bytes([9u8; 32]);
let first = network("one");
let second = network("two");
assert_eq!(overlay_address(first, &key), overlay_address(first, &key));
assert_ne!(
overlay_address(first, &key),
overlay_address(second, &key),
"the same key in a different network gets a different address"
);
assert_ne!(overlay_prefix(first), overlay_prefix(second));
}
#[test]
fn ipv4_addresses_land_inside_the_range_and_avoid_its_edges() {
let id = network("v4");
let range: Ipv4Range = "100.64.0.0/10".parse().unwrap();
for byte in 0..64u8 {
let key = WgPublicKey::from_bytes([byte; 32]);
let addr = overlay_address_v4(id, &key, range).unwrap();
let raw = u32::from(addr);
assert_eq!(
raw & 0xffc0_0000,
u32::from(range.base),
"outside 100.64.0.0/10"
);
// Never the network address and never the broadcast address.
assert_ne!(raw & 0x003f_ffff, 0);
assert_ne!(raw & 0x003f_ffff, 0x003f_ffff);
}
}
#[test]
fn a_range_parses_and_prints_round_trip() {
let range: Ipv4Range = "10.77.0.0/16".parse().unwrap();
assert_eq!(range.to_string(), "10.77.0.0/16");
assert!(range.contains("10.77.3.4".parse().unwrap()));
assert!(!range.contains("10.78.3.4".parse().unwrap()));
assert!("10.77.0.0".parse::<Ipv4Range>().is_err());
assert!("nonsense/16".parse::<Ipv4Range>().is_err());
assert!("10.77.0.0/zz".parse::<Ipv4Range>().is_err());
assert!("10.77.0.0/31".parse::<Ipv4Range>().is_err());
}
#[test]
fn ipv4_derivation_is_deterministic_and_scoped_like_ipv6() {
let key = WgPublicKey::from_bytes([9u8; 32]);
let first = network("one");
let second = network("two");
let range: Ipv4Range = "100.64.0.0/10".parse().unwrap();
assert_eq!(
overlay_address_v4(first, &key, range),
overlay_address_v4(first, &key, range)
);
assert_ne!(
overlay_address_v4(first, &key, range),
overlay_address_v4(second, &key, range)
);
assert_ne!(
overlay_address_v4(first, &key, range),
overlay_address_v4(first, &WgPublicKey::from_bytes([10u8; 32]), range)
);
// A different range moves everybody.
assert_ne!(
overlay_address_v4(first, &key, range),
overlay_address_v4(
first,
&key,
Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 0), 8).unwrap()
)
);
}
#[test]
fn a_range_with_no_room_yields_nothing() {
let id = network("tiny");
let key = WgPublicKey::from_bytes([1u8; 32]);
// /31 and /32 have no usable host addresses, so they are refused.
assert!(Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 0), 31).is_err());
assert!(Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 1), 32).is_err());
assert!(Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 0), 33).is_err());
// A /30 has two usable addresses.
let small = Ipv4Range::new(Ipv4Addr::new(10, 0, 0, 0), 30).unwrap();
assert!(overlay_address_v4(id, &key, small).is_some());
// A /0 must not overflow.
let everything = Ipv4Range::new(Ipv4Addr::UNSPECIFIED, 0).unwrap();
assert!(overlay_address_v4(id, &key, everything).is_some());
}
#[test]
fn addresses_are_never_the_subnet_router_anycast_address() {
let id = network("anycast");
for byte in 0..64u8 {
let address = overlay_address(id, &WgPublicKey::from_bytes([byte; 32]));
assert_ne!(&address.octets()[8..16], &[0u8; 8]);
}
}
}
crates/tsunagi/src/dataplane/wireguard/plugin.rs
+23 -41
View File
@@ -42,7 +42,6 @@ use crate::identity::NetworkId;
use super::announcement::{ValidatedAnnouncement, WgAnnouncement};
use super::device::{PeerSummary, WireguardDevice};
use super::keys::{WgPublicKey, WgSecretKey};
use super::overlay::{OVERLAY_PREFIX_LEN, overlay_address, overlay_prefix};
use super::store::WgKeyStore;
use crate::overlay::config::{DEFAULT_INTERFACE_PREFIX, interface_name};
use crate::overlay::tun::{TunFactory, TunRequest};
@@ -51,20 +50,20 @@ use crate::state::Ipv4Range;
/// The protocol identifier this plugin announces.
pub const WIREGUARD_PROTOCOL: &str = "wireguard";
/// Smallest interface MTU IPv6 permits, from RFC 8200.
/// Smallest interface MTU the overlay accepts.
///
/// This is not advice, it is a hard limit. Linux tears IPv6 down entirely on
/// an interface whose MTU is below it — the per-device `/proc/sys/net/ipv6`
/// entries disappear and `ip -6 address add` fails with `Invalid argument` —
/// so the overlay address could never be assigned. Anything smaller is
/// rejected up front instead of failing obscurely later.
pub const MIN_MTU: u32 = 1280;
/// 576 bytes is what IPv4 guarantees every host can reassemble (RFC 1122),
/// so nothing below it is worth offering. The floor used to be 1280 because
/// Linux tears IPv6 down on an interface below that; the overlay is IPv4
/// now, so that constraint is gone and a path with small datagrams — a
/// relay, typically — can be matched instead of warned about.
pub const MIN_MTU: u32 = 576;
/// Default interface MTU.
///
/// Equal to [`MIN_MTU`], because the overlay is IPv6 and there is no room
/// below it.
pub const DEFAULT_MTU: u32 = MIN_MTU;
/// Comfortably under what a direct path carries, and the same number the
/// overlay used before, so an existing network does not have to change.
pub const DEFAULT_MTU: u32 = 1280;
/// Bytes WireGuard adds to a packet: type and reserved, receiver index,
/// counter and the Poly1305 tag.
@@ -145,13 +144,7 @@ pub struct NetworkOverview {
pub mtu: u32,
/// This agent's WireGuard public key in this network.
pub public_key: WgPublicKey,
/// This agent's overlay address.
pub overlay_address: IpAddr,
/// The overlay subnet every member shares.
pub overlay_prefix: IpAddr,
/// Prefix length of the overlay subnet.
pub overlay_prefix_len: u8,
/// This agent's IPv4 overlay address, when the overlay is dual stack.
/// This agent's overlay address, once the network has agreed one.
pub overlay_address_v4: Option<Ipv4Addr>,
/// The IPv4 overlay range in use.
pub ipv4_range: Option<Ipv4Range>,
@@ -179,9 +172,7 @@ pub struct PeerOverview {
pub endpoint_id: EndpointId,
/// The peer's WireGuard public key.
pub public_key: WgPublicKey,
/// The overlay address derived for it locally.
pub overlay_address: IpAddr,
/// Its IPv4 overlay address, once a tunnel exists and it won the address.
/// The overlay address the network agreed it holds.
pub overlay_address_v4: Option<Ipv4Addr>,
/// Whether a data plane link to it exists.
pub has_link: bool,
@@ -278,9 +269,8 @@ impl WireguardPlugin {
if config.mtu < MIN_MTU {
return Err(PluginError::Other(format!(
"an MTU of {} is below the {MIN_MTU} bytes IPv6 requires (RFC 8200). \
Linux disables IPv6 on an interface below that, so the overlay address \
could never be assigned.",
"an MTU of {} is below the {MIN_MTU} bytes every IPv4 host must be able \
to reassemble (RFC 1122)",
config.mtu
)));
}
@@ -332,7 +322,6 @@ impl WireguardPlugin {
.map(|(endpoint_id, announcement)| PeerOverview {
endpoint_id: *endpoint_id,
public_key: announcement.public_key,
overlay_address: IpAddr::V6(announcement.overlay_address),
overlay_address_v4: state.allocations.get(endpoint_id).copied(),
has_link: state.links.contains_key(endpoint_id),
tunnel: tunnels.get(&announcement.public_key).cloned(),
@@ -345,9 +334,6 @@ impl WireguardPlugin {
interface: state.interface.clone(),
mtu: self.worker.config.mtu,
public_key: state.key.public(),
overlay_address: IpAddr::V6(overlay_address(network, &state.key.public())),
overlay_prefix: IpAddr::V6(overlay_prefix(network)),
overlay_prefix_len: OVERLAY_PREFIX_LEN,
overlay_address_v4: state.allocations.get(&self.worker.local_id()).copied(),
ipv4_range: state.ipv4_range,
peers,
@@ -464,35 +450,31 @@ impl Worker {
}
/// What the host interface for a network should look like.
fn desired_request(&self, state: &NetworkState, network: NetworkId) -> TunRequest {
fn desired_request(&self, state: &NetworkState) -> TunRequest {
let own_range = state.ipv4_range;
TunRequest {
name: state.interface.clone(),
address: overlay_address(network, &state.key.public()),
prefix_len: OVERLAY_PREFIX_LEN,
address_v4: state.allocations.get(&self.local_id()).copied(),
prefix_len_v4: own_range.map_or(0, |range| range.prefix_len),
address: state.allocations.get(&self.local_id()).copied(),
prefix_len: own_range.map_or(0, |range| range.prefix_len),
mtu: self.config.mtu,
}
}
/// Creates the packet interface and starts the WireGuard device.
async fn ensure_device(&self, network: NetworkId) -> Result<(), PluginError> {
let (request, key, own_range) = {
let (request, key) = {
let shared = self.lock_shared();
match shared.networks.get(&network) {
Some(state) if state.device.is_none() => (
self.desired_request(state, network),
state.key.clone(),
state.ipv4_range,
),
Some(state) if state.device.is_none() => {
(self.desired_request(state), state.key.clone())
}
_ => return Ok(()),
}
};
let applied = request.clone();
let tun = self.tun_factory.create(request).await?;
let device = Arc::new(WireguardDevice::start(network, key, tun, own_range));
let device = Arc::new(WireguardDevice::start(network, key, tun));
let mut shared = self.lock_shared();
if let Some(state) = shared.networks.get_mut(&network) {
@@ -510,7 +492,7 @@ impl Worker {
let shared = self.lock_shared();
match shared.networks.get(&network) {
Some(state) if state.device.is_some() => {
let wanted = self.desired_request(state, network);
let wanted = self.desired_request(state);
if state.applied.as_ref() == Some(&wanted) {
return;
}
crates/tsunagi/src/ipc/mod.rs
+5 -11
View File
@@ -185,13 +185,9 @@ pub struct OverlayReport {
pub interface: String,
/// Interface MTU.
pub mtu: u32,
/// This agent's overlay address.
pub address: String,
/// This agent's IPv4 overlay address, when the overlay is dual stack.
pub address_v4: Option<String>,
/// The subnet every member shares.
pub prefix: String,
/// Prefix length of that subnet.
/// This agent's overlay address, once the network has agreed one.
pub address: Option<String>,
/// Prefix length of the overlay range every member shares.
pub prefix_len: u8,
/// One entry per overlay peer.
pub peers: Vec<OverlayPeerReport>,
@@ -211,10 +207,8 @@ pub struct OverlayPeerReport {
pub endpoint_id: String,
/// The peer's WireGuard public key.
pub public_key: String,
/// Its overlay address.
pub address: String,
/// Its IPv4 overlay address, when it has one.
pub address_v4: Option<String>,
/// The overlay address the network agreed it holds.
pub address: Option<String>,
/// Seconds since the last WireGuard handshake.
///
/// `None` means the tunnel has never handshaken and cannot carry traffic.
crates/tsunagi/src/ipc/unix.rs
+1 -1
View File
@@ -205,7 +205,7 @@ pub async fn set_hostname(path: impl AsRef<Path>, hostname: &str) -> Result<Stri
///
/// Bump it whenever [`Request`], [`Response`] or anything they contain
/// changes shape.
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 4]);
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 5]);
async fn write_message<T: serde::Serialize>(stream: &mut UnixStream, value: &T) -> Result<()> {
let encoded = postcard::to_stdvec(value)
crates/tsunagi/src/overlay/provision/factory.rs
+7 -9
View File
@@ -17,9 +17,9 @@ use super::{InterfacePlan, InterfaceProvisioner};
/// Turns a [`TunRequest`] into the plan for a host interface.
fn plan_for(request: &TunRequest) -> Result<InterfacePlan, OverlayError> {
let mut addresses = vec![Cidr::new(request.address.into(), request.prefix_len)?];
if let Some(address) = request.address_v4 {
addresses.push(Cidr::new(address.into(), request.prefix_len_v4)?);
let mut addresses = Vec::new();
if let Some(address) = request.address {
addresses.push(Cidr::new(address.into(), request.prefix_len)?);
}
Ok(InterfacePlan::new(
request.name.clone(),
@@ -106,7 +106,7 @@ impl TunFactory for ManagedTunFactory {
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::{Ipv4Addr, Ipv6Addr};
use std::net::Ipv4Addr;
use super::super::{LinkKind, MockHost, MockProvisioner};
use super::*;
@@ -114,10 +114,8 @@ mod tests {
fn request(v4: Option<Ipv4Addr>) -> TunRequest {
TunRequest {
name: "tsunfactory".into(),
address: Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0, 1),
prefix_len: 64,
address_v4: v4,
prefix_len_v4: 24,
address: v4,
prefix_len: 24,
mtu: 1280,
}
}
@@ -137,7 +135,7 @@ mod tests {
let state = provisioner.host().get("tsunfactory").unwrap();
assert_eq!(state.kind, LinkKind::Tun);
assert_eq!(state.mtu, 1280);
assert_eq!(state.addresses.len(), 2, "both families are assigned");
assert_eq!(state.addresses.len(), 1, "the overlay address is assigned");
}
#[tokio::test]
crates/tsunagi/src/overlay/tun.rs
+7 -11
View File
@@ -13,7 +13,6 @@
//! Creating one needs `CAP_NET_ADMIN`, and it is
//! [`provision`](super::provision) that holds that and creates it.
use std::net::Ipv6Addr;
use std::sync::Arc;
use bytes::Bytes;
@@ -26,14 +25,13 @@ use crate::overlay::OverlayError;
pub struct TunRequest {
/// Interface name to ask for.
pub name: String,
/// The overlay address this host answers to.
pub address: Ipv6Addr,
/// Prefix length of the overlay subnet, so the OS routes it here.
/// The overlay address this host answers to, and its prefix length.
///
/// Allocated and signed at the system level, never derived from a
/// protocol's key: every protocol carries traffic for the same address.
pub address: Option<std::net::Ipv4Addr>,
/// Prefix length of the overlay range.
pub prefix_len: u8,
/// The IPv4 overlay address this host answers to, when dual stack.
pub address_v4: Option<std::net::Ipv4Addr>,
/// Prefix length of the IPv4 overlay range.
pub prefix_len_v4: u8,
/// Interface MTU.
pub mtu: u32,
}
@@ -46,10 +44,8 @@ impl TunRequest {
pub fn bare(name: impl Into<String>, mtu: u32) -> Self {
Self {
name: name.into(),
address: Ipv6Addr::UNSPECIFIED,
address: None,
prefix_len: 0,
address_v4: None,
prefix_len_v4: 0,
mtu,
}
}
crates/tsunagi/tests/interface_provisioning.rs
+14 -13
View File
@@ -94,11 +94,12 @@ fn v4(state: &InterfaceState) -> Vec<Ipv4Addr> {
.collect()
}
fn has_v6(state: &InterfaceState) -> bool {
state
.addresses
.iter()
.any(|cidr| matches!(cidr.addr, IpAddr::V6(_)))
/// Whether the overlay address has been put on the interface yet.
///
/// It is allocated and signed at the system level, so it arrives on a later
/// reconciliation than the interface itself rather than with it.
fn addressed(state: &InterfaceState) -> bool {
!state.addresses.is_empty()
}
#[tokio::test]
@@ -111,18 +112,13 @@ async fn an_agent_creates_and_configures_its_own_overlay_interface() {
let interface = agent.interface(network_id).await;
let state = agent
.wait_for_host("the interface to be created", &interface, |state| {
state.filter(|state| !state.addresses.is_empty())
})
.wait_for_host("the interface to be created", &interface, |state| state)
.await;
assert_eq!(state.kind, LinkKind::Tun);
assert!(state.up, "the agent brought the link up itself");
assert_eq!(state.mtu, 1280);
assert!(has_v6(&state), "the derived overlay address is assigned");
// The IPv4 address is allocated at run time, so it arrives on a later
// reconciliation than the interface itself.
let addresses = agent
.wait_for_host("the allocated IPv4 address", &interface, |state| {
state.map(|state| v4(&state)).filter(|v4| !v4.is_empty())
@@ -160,9 +156,14 @@ async fn an_interface_left_by_a_crashed_run_is_replaced_rather_than_tripped_over
let network_id = agent.agent.join_network(&name, &secret).await.unwrap();
assert_eq!(agent.interface(network_id).await, interface);
// Waited on directly rather than on "it has some address": the address
// this agent was allocated arrives a reconciliation after the interface
// does, and in between the leftover is still the only one there.
let state = agent
.wait_for_host("the interface to be rebuilt", &interface, |state| {
state.filter(|state| state.attached && has_v6(state))
.wait_for_host("the stale address to be replaced", &interface, |state| {
state.filter(|state| {
state.attached && addressed(state) && !state.addresses.contains(&stale)
})
})
.await;
assert!(
crates/tsunagi/tests/local_control.rs
+4 -5
View File
@@ -48,15 +48,14 @@ fn source(agent: Agent, plugin: Arc<WireguardPlugin>) -> Arc<dyn tsunagi::ipc::u
tsunagi::ipc::OverlayReport {
interface: view.interface.clone(),
mtu: view.mtu,
address: view.overlay_address.to_string(),
prefix: view.overlay_prefix.to_string(),
prefix_len: view.overlay_prefix_len,
address: view.overlay_address_v4.map(|a| a.to_string()),
prefix_len: view.ipv4_range.map_or(0, |range| range.prefix_len),
peers: view
.peers
.iter()
.map(|peer| tsunagi::ipc::OverlayPeerReport {
public_key: peer.public_key.to_string(),
address: peer.overlay_address.to_string(),
address: peer.overlay_address_v4.map(|a| a.to_string()),
handshake_secs_ago: peer
.tunnel
.as_ref()
@@ -168,7 +167,7 @@ async fn a_client_sees_the_agent_and_its_overlay() {
"the tunnel is up: {:?}",
overlay.peers
);
assert!(!overlay.address.is_empty());
assert!(overlay.address.is_some());
control.shutdown().await;
assert!(!socket_path.exists(), "the socket is removed on shutdown");
crates/tsunagi/tests/wireguard.rs
+63 -138
View File
@@ -10,7 +10,7 @@
mod common;
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use std::net::Ipv4Addr;
use std::sync::Arc;
use std::time::Duration;
@@ -22,7 +22,7 @@ use tsunagi::agent::Event;
use tsunagi::dataplane::IpPlugin;
use tsunagi::dataplane::wireguard::{
Ipv4Range, MemoryTun, MemoryTunFactory, WIREGUARD_PROTOCOL, WgAnnouncement, WgSecretKey,
WireguardConfig, WireguardPlugin, overlay_address, overlay_prefix,
WireguardConfig, WireguardPlugin,
};
use tsunagi::discovery::SharedMemoryDiscovery;
use tsunagi::identity::{NetworkId, NetworkName, NetworkSecret};
@@ -127,15 +127,14 @@ impl WgAgent {
}
/// This agent's overlay address in a network.
async fn overlay(&self, network: NetworkId) -> Ipv6Addr {
let view = wait_until("the plugin prepared the network", || async {
self.plugin.overview(network)
///
/// Allocated and signed at the system level, so it appears once the
/// network has agreed on it rather than the moment the plugin starts.
async fn overlay(&self, network: NetworkId) -> Ipv4Addr {
wait_until("the network agreed an overlay address", || async {
self.plugin.overview(network)?.overlay_address_v4
})
.await;
match view.overlay_address {
IpAddr::V6(addr) => addr,
IpAddr::V4(_) => panic!("the overlay is IPv6"),
}
.await
}
/// The in-memory packet interface for a network.
@@ -184,20 +183,6 @@ fn ipv4_packet(source: Ipv4Addr, destination: Ipv4Addr, payload: &[u8]) -> Bytes
Bytes::from(packet)
}
/// Builds a minimal well-formed IPv6 packet.
fn ipv6_packet(source: Ipv6Addr, destination: Ipv6Addr, payload: &[u8]) -> Bytes {
let mut packet = Vec::with_capacity(40 + payload.len());
packet.push(6 << 4); // version 6
packet.extend_from_slice(&[0, 0, 0]); // traffic class and flow label
packet.extend_from_slice(&(payload.len() as u16).to_be_bytes());
packet.push(59); // "no next header"
packet.push(64); // hop limit
packet.extend_from_slice(&source.octets());
packet.extend_from_slice(&destination.octets());
packet.extend_from_slice(payload);
Bytes::from(packet)
}
#[tokio::test]
async fn two_agents_carry_real_ip_packets_through_a_wireguard_tunnel() {
let discovery = SharedMemoryDiscovery::new();
@@ -217,12 +202,10 @@ async fn two_agents_carry_real_ip_packets_through_a_wireguard_tunnel() {
let addr_a = a.overlay(network_id).await;
let addr_b = b.overlay(network_id).await;
assert_ne!(addr_a, addr_b);
// One shared /64, derived by both sides independently.
assert_eq!(addr_a.octets()[0..8], addr_b.octets()[0..8]);
assert_eq!(
&overlay_prefix(network_id).octets()[0..8],
&addr_a.octets()[0..8]
);
// One shared range, agreed at the system level and signed by each
// member, not derived from anything either protocol holds.
let range = a.plugin.overview(network_id).unwrap().ipv4_range.unwrap();
assert!(range.contains(addr_a) && range.contains(addr_b));
let tun_a = a.tun(network_id).await;
let tun_b = b.tun(network_id).await;
@@ -230,23 +213,23 @@ async fn two_agents_carry_real_ip_packets_through_a_wireguard_tunnel() {
// A real IP packet, encrypted by WireGuard, carried over iroh, decrypted
// on the other side and handed to that host's packet interface.
let payload = b"hello over the overlay";
tun_a.push_from_os(ipv6_packet(addr_a, addr_b, payload));
tun_a.push_from_os(ipv4_packet(addr_a, addr_b, payload));
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
.await
.expect("the packet should arrive")
.expect("the interface should still be open");
assert_eq!(&received[40..], payload);
assert_eq!(&received[8..24], &addr_a.octets(), "source preserved");
assert_eq!(&received[24..40], &addr_b.octets(), "destination preserved");
assert_eq!(&received[20..], payload);
assert_eq!(&received[12..16], &addr_a.octets(), "source preserved");
assert_eq!(&received[16..20], &addr_b.octets(), "destination preserved");
// And back the other way.
tun_b.push_from_os(ipv6_packet(addr_b, addr_a, b"and back"));
tun_b.push_from_os(ipv4_packet(addr_b, addr_a, b"and back"));
let back = tokio::time::timeout(common::DEADLINE, tun_a.pop_to_os())
.await
.expect("the reply should arrive")
.unwrap();
assert_eq!(&back[40..], b"and back");
assert_eq!(&back[20..], b"and back");
let view = a.plugin.overview(network_id).unwrap();
let tunnel = view.peers[0].tunnel.as_ref().unwrap();
@@ -278,14 +261,14 @@ async fn a_peer_cannot_send_from_an_address_it_does_not_own() {
let tun_b = b.tun(network_id).await;
// A sends a packet claiming to come from a third party's address.
let someone_else: Ipv6Addr = {
let someone_else = {
let mut octets = addr_a.octets();
octets[15] ^= 0xff;
Ipv6Addr::from(octets)
octets[3] ^= 0xff;
Ipv4Addr::from(octets)
};
tun_a.push_from_os(ipv6_packet(someone_else, addr_b, b"spoofed"));
tun_a.push_from_os(ipv4_packet(someone_else, addr_b, b"spoofed"));
// B must drop it: the source is not the address derived for A's key.
// B must drop it: the source is not the address A holds.
wait_until("the spoofed packet is dropped", || async {
let view = b.plugin.overview(network_id)?;
let tunnel = view.peers.first()?.tunnel.as_ref()?;
@@ -294,19 +277,19 @@ async fn a_peer_cannot_send_from_an_address_it_does_not_own() {
.await;
// A legitimate packet still goes through, so the tunnel is not broken.
tun_a.push_from_os(ipv6_packet(addr_a, addr_b, b"honest"));
tun_a.push_from_os(ipv4_packet(addr_a, addr_b, b"honest"));
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
.await
.expect("the honest packet should arrive")
.unwrap();
assert_eq!(&received[40..], b"honest");
assert_eq!(&received[20..], b"honest");
a.shutdown().await;
b.shutdown().await;
}
#[tokio::test]
async fn the_overlay_carries_ipv4_alongside_ipv6() {
async fn the_overlay_uses_a_range_the_network_was_told_to_use() {
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("wg-dual-stack");
@@ -332,10 +315,9 @@ async fn the_overlay_carries_ipv4_alongside_ipv6() {
u32::from(Ipv4Addr::new(10, 77, 0, 0))
);
}
// Each side derived the other's address identically.
// Each side learned the other's address from the same signed state.
assert_eq!(view_a.peers[0].overlay_address_v4, Some(v4_b));
assert_eq!(view_b.peers[0].overlay_address_v4, Some(v4_a));
assert!(!view_a.peers[0].tunnel.as_ref().unwrap().ipv4_conflict);
let tun_a = a.tun(network_id).await;
let tun_b = b.tun(network_id).await;
@@ -351,17 +333,6 @@ async fn the_overlay_carries_ipv4_alongside_ipv6() {
assert_eq!(&received[16..20], &v4_b.octets());
assert_eq!(&received[20..], b"ipv4 over the overlay");
// IPv6 keeps working on the same tunnel.
let addr_a = a.overlay(network_id).await;
let addr_b = b.overlay(network_id).await;
tun_a.push_from_os(ipv6_packet(addr_a, addr_b, b"and ipv6 too"));
let received = tokio::time::timeout(common::DEADLINE, tun_b.pop_to_os())
.await
.expect("the IPv6 packet should arrive")
.unwrap();
assert_eq!(received[0] >> 4, 6);
assert_eq!(&received[40..], b"and ipv6 too");
a.shutdown().await;
b.shutdown().await;
}
@@ -417,41 +388,6 @@ async fn an_ipv4_source_a_peer_does_not_own_is_dropped() {
b.shutdown().await;
}
#[tokio::test]
async fn an_ipv6_only_overlay_can_be_asked_for() {
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("wg-v6-only");
let a = WgAgent::spawn_range(&discovery, "ta", None).await;
let b = WgAgent::spawn_range(&discovery, "tb", None).await;
let network_id = a.agent.join_network(&name, &secret).await.unwrap();
b.agent.join_network(&name, &secret).await.unwrap();
a.wait_for_tunnels(network_id, 1).await;
let view = a.plugin.overview(network_id).unwrap();
assert_eq!(view.overlay_address_v4, None);
assert_eq!(view.ipv4_range, None);
assert_eq!(view.peers[0].overlay_address_v4, None);
// No IPv4 configured is not a conflict.
assert!(!view.peers[0].tunnel.as_ref().unwrap().ipv4_conflict);
// IPv6 is unaffected.
let addr_a = a.overlay(network_id).await;
let addr_b = b.overlay(network_id).await;
a.tun(network_id)
.await
.push_from_os(ipv6_packet(addr_a, addr_b, b"v6 only"));
let received = tokio::time::timeout(common::DEADLINE, b.tun(network_id).await.pop_to_os())
.await
.expect("the packet should arrive")
.unwrap();
assert_eq!(&received[40..], b"v6 only");
a.shutdown().await;
b.shutdown().await;
}
#[tokio::test]
async fn a_joining_member_adopts_the_range_the_network_already_uses() {
let discovery = SharedMemoryDiscovery::new();
@@ -670,8 +606,8 @@ async fn packets_for_an_unknown_address_are_counted_not_broadcast() {
let tun_b = b.tun(network_id).await;
// Nobody owns this address, so it must not be sent to anybody.
let nowhere: Ipv6Addr = "fd00:dead:beef::1".parse().unwrap();
tun_a.push_from_os(ipv6_packet(addr_a, nowhere, b"lost"));
let nowhere: Ipv4Addr = "192.0.2.111".parse().unwrap();
tun_a.push_from_os(ipv4_packet(addr_a, nowhere, b"lost"));
wait_until("the packet is counted as unroutable", || async {
let view = a.plugin.overview(network_id)?;
@@ -714,13 +650,12 @@ async fn a_mesh_of_three_establishes_every_tunnel() {
let mut addresses = Vec::new();
for agent in [&a, &b, &c] {
let view = agent.plugin.overview(network_id).unwrap();
assert_eq!(view.overlay_prefix, IpAddr::V6(overlay_prefix(network_id)));
assert!(
view.peers
.iter()
.all(|peer| peer.public_key != view.public_key)
);
addresses.push(view.overlay_address);
addresses.push(agent.overlay(network_id).await);
}
addresses.sort();
addresses.dedup();
@@ -731,12 +666,12 @@ async fn a_mesh_of_three_establishes_every_tunnel() {
let addr_c = c.overlay(network_id).await;
a.tun(network_id)
.await
.push_from_os(ipv6_packet(addr_a, addr_c, b"a to c"));
.push_from_os(ipv4_packet(addr_a, addr_c, b"a to c"));
let received = tokio::time::timeout(common::DEADLINE, c.tun(network_id).await.pop_to_os())
.await
.expect("the packet should arrive")
.unwrap();
assert_eq!(&received[40..], b"a to c");
assert_eq!(&received[20..], b"a to c");
a.shutdown().await;
b.shutdown().await;
@@ -801,23 +736,19 @@ async fn two_networks_get_separate_interfaces_keys_and_overlays() {
view_alpha.public_key, view_beta.public_key,
"one WireGuard identity per network, not one per host"
);
assert_ne!(view_alpha.overlay_prefix, view_beta.overlay_prefix);
assert_eq!(hub.tuns.devices().len(), 2);
// Traffic in one overlay never surfaces in the other.
let hub_alpha = match view_alpha.overlay_address {
IpAddr::V6(addr) => addr,
IpAddr::V4(_) => panic!("ipv6"),
};
let hub_alpha = hub.overlay(alpha).await;
let left_addr = left.overlay(alpha).await;
hub.tun(alpha)
.await
.push_from_os(ipv6_packet(hub_alpha, left_addr, b"alpha only"));
.push_from_os(ipv4_packet(hub_alpha, left_addr, b"alpha only"));
let seen = tokio::time::timeout(common::DEADLINE, left.tun(alpha).await.pop_to_os())
.await
.expect("the packet should arrive")
.unwrap();
assert_eq!(&seen[40..], b"alpha only");
assert_eq!(&seen[20..], b"alpha only");
assert!(
tokio::time::timeout(
Duration::from_millis(200),
@@ -863,7 +794,6 @@ async fn restarting_keeps_the_wireguard_identity_and_overlay_address() {
})
.await;
assert_eq!(after.public_key, before.public_key);
assert_eq!(after.overlay_address, before.overlay_address);
assert_eq!(after.interface, before.interface);
// The tunnel comes back on its own.
@@ -930,20 +860,25 @@ async fn the_core_carries_the_payload_without_interpreting_it() {
}
#[tokio::test]
async fn a_forged_overlay_claim_is_rejected_and_never_reaches_a_tunnel() {
async fn an_announcement_for_another_network_never_reaches_a_tunnel() {
let discovery = SharedMemoryDiscovery::new();
let name = NetworkName::new("wg-hijack").unwrap();
let secret = NetworkSecret::generate();
let victim = WgAgent::spawn(&discovery, "tv").await;
let network_id = victim.agent.join_network(&name, &secret).await.unwrap();
let victim_address = victim.overlay(network_id).await;
// A legitimate member — it knows the secret — claims the victim's overlay
// address with its own WireGuard key.
// Addresses are not in an announcement any more, so there is no address
// to forge. What is left to lie about is which network the key is for —
// and a capability is scoped to the session it arrived on, so claiming
// another network's is the shape a confused or hostile member takes.
let attacker_key = WgSecretKey::generate().public();
let mut forged = WgAnnouncement::new(network_id, &attacker_key);
forged.overlay_address = victim_address;
let elsewhere = tsunagi::identity::NetworkKeys::derive(
&NetworkName::new("somewhere-else").unwrap(),
&NetworkSecret::generate(),
)
.network_id();
let forged = WgAnnouncement::new(elsewhere, &attacker_key);
let forger = Arc::new(ForgingPlugin {
payload: std::sync::Mutex::new(Some(forged.encode().unwrap())),
@@ -960,14 +895,18 @@ async fn a_forged_overlay_claim_is_rejected_and_never_reaches_a_tunnel() {
attacker.join_network(&name, &secret).await.unwrap();
wait_for_peers(&victim.agent, network_id, 1).await;
// Filtered on the reason: this agent reports other things too, and the
// first plugin error to arrive is not necessarily this one.
let reason = wait_event(&mut events, |event| match event {
Event::PluginError {
protocol, reason, ..
} if protocol == WIREGUARD_PROTOCOL => Some(reason.clone()),
} if protocol == WIREGUARD_PROTOCOL && reason.contains("different network") => {
Some(reason.clone())
}
_ => None,
})
.await;
assert!(reason.contains("does not match"), "unexpected: {reason}");
assert!(reason.contains("different network"), "unexpected: {reason}");
settle().await;
let view = victim.plugin.overview(network_id).unwrap();
@@ -977,8 +916,6 @@ async fn a_forged_overlay_claim_is_rejected_and_never_reaches_a_tunnel() {
.all(|peer| peer.public_key != attacker_key),
"a rejected announcement must never become a tunnel"
);
assert_eq!(view.overlay_address, IpAddr::V6(victim_address));
attacker.shutdown().await;
victim.shutdown().await;
drop(attacker_dir);
@@ -1026,12 +963,13 @@ impl IpPlugin for ForgingPlugin {
}
#[tokio::test]
async fn an_mtu_below_the_ipv6_minimum_is_refused() {
async fn an_mtu_below_what_ipv4_guarantees_is_refused() {
use tsunagi::dataplane::wireguard::{DEFAULT_MTU, MIN_MTU, WIREGUARD_OVERHEAD};
// Linux disables IPv6 outright on an interface below 1280 bytes, so the
// overlay address could never be assigned. Catch it here rather than as
// an obscure RTNETLINK error much later.
// 576 bytes is what every IPv4 host must be able to reassemble, so
// nothing below it is worth offering. The floor used to be 1280 for
// IPv6's sake; the overlay is IPv4 now and a relayed path with small
// datagrams can be matched instead of warned about.
let dir = TempDir::new().unwrap();
let result = WireguardPlugin::open(
WireguardConfig::new(dir.path()).with_mtu(MIN_MTU - 1),
@@ -1041,15 +979,14 @@ async fn an_mtu_below_the_ipv6_minimum_is_refused() {
match result {
Err(err) => {
let text = err.to_string();
assert!(text.contains("1280"), "unexpected message: {text}");
assert!(text.contains("IPv6"), "unexpected message: {text}");
assert!(text.contains("576"), "unexpected message: {text}");
}
Ok(_) => panic!("an MTU below the IPv6 minimum must be refused"),
Ok(_) => panic!("an MTU below what IPv4 guarantees must be refused"),
}
// The default is exactly the minimum, and a link has to carry it plus
// WireGuard's own overhead.
assert_eq!(DEFAULT_MTU, MIN_MTU);
// The default leaves room below it now, which is the point of lowering
// the floor, and a link has to carry it plus WireGuard's own overhead.
const { assert!(DEFAULT_MTU > MIN_MTU) };
assert_eq!(WIREGUARD_OVERHEAD, 32);
assert!(
WireguardPlugin::open(
@@ -1060,15 +997,3 @@ async fn an_mtu_below_the_ipv6_minimum_is_refused() {
.is_ok()
);
}
#[tokio::test]
async fn the_overlay_address_is_derived_from_the_key_alone() {
let (name, secret) = network("wg-derivation");
let id = tsunagi::identity::NetworkKeys::derive(&name, &secret).network_id();
let key = WgSecretKey::generate().public();
assert_eq!(overlay_address(id, &key), overlay_address(id, &key));
assert_ne!(
overlay_address(id, &key),
overlay_address(id, &WgSecretKey::generate().public())
);
}