Separate control and data logically, move WireGuard into userspace, add a CLI
Corrects the architecture on two points raised in review, while the project is still small enough to change cheaply. 1. Control and data are separated *logically*, not physically. The old reading — "nothing but control may ride on iroh" — threw away iroh's whole value and would have forced the data plane to reimplement STUN, ICE and a relay. Now both planes ride on iroh with different ALPNs and different connections, so the data plane inherits hole punching and relay fallback, while proto/ still knows nothing about packets and dataplane/ knows nothing about the control protocol. New boundary: PacketTransport / PacketLink, an authenticated unreliable datagram channel per (network, peer, protocol). tsunagi/data/1 runs the same membership handshake, then DataOpen/DataOpenAck, then QUIC datagrams. Only the smaller endpoint id dials, so exactly one link exists per pair. A plugin is handed links and never learns reachability, so the WireGuard announcement shrank to a public key: there is no address left to lie about. 2. WireGuard now runs in userspace, on boringtun's protocol state machine. No kernel module, no wg tool, no ip shell-out, no loopback proxy: the wgtool, backend and bridge modules are gone. Only creating a TUN device needs privileges, and that sits behind TunFactory, so the entire data plane — handshake, encryption, routing, address ownership — is tested with none. Address ownership is enforced rather than believed: outbound packets go to the owner of the destination address, inbound packets are dropped unless their source is the address derived for the peer that sent them. 3. A `tsunagi` binary: secret, doctor, id, up. It owns the runtime, the logging subscriber and Ctrl-C, which the library still refuses to. Also fixes a reference cycle where IrohTransport held Arc<Inner>, which kept the databases open and the directory lock held after shutdown; two storage tests caught it once the cycle existed. 81 tests pass offline with no privileges, including real IPv6 packets crossing a real WireGuard tunnel over real iroh connections. Verified by hand: two CLI processes forming a mesh both on loopback and via n0 discovery using only an endpoint id. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
Claude Opus 5
@@ -100,6 +100,35 @@ pub enum Event {
|
||||
/// Why it was discarded. Free of secrets.
|
||||
reason: String,
|
||||
},
|
||||
/// A data plane link to a peer is up.
|
||||
///
|
||||
/// The data plane is a separate connection from the control plane; this
|
||||
/// says nothing about the control session, and vice versa.
|
||||
DataLinkUp {
|
||||
/// The network.
|
||||
network: NetworkId,
|
||||
/// The peer.
|
||||
peer: EndpointId,
|
||||
/// Plugin protocol the link carries.
|
||||
protocol: String,
|
||||
/// What the transport reports about the path in use.
|
||||
path: String,
|
||||
/// Largest datagram the link can carry.
|
||||
max_datagram: usize,
|
||||
},
|
||||
/// A data plane link went away or could not be opened.
|
||||
///
|
||||
/// Never fatal: the control plane keeps running and the link is retried.
|
||||
DataLinkDown {
|
||||
/// The network.
|
||||
network: NetworkId,
|
||||
/// The peer.
|
||||
peer: EndpointId,
|
||||
/// Plugin protocol the link would have carried.
|
||||
protocol: String,
|
||||
/// Why it is not up.
|
||||
reason: String,
|
||||
},
|
||||
/// An IP plugin reported an error. Never fatal.
|
||||
PluginError {
|
||||
/// The network the call was scoped to.
|
||||
|
||||
+97
-11
@@ -37,6 +37,8 @@ use tokio::sync::{RwLock, broadcast, mpsc, oneshot};
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::config::{AgentConfig, Limits};
|
||||
use crate::dataplane::transport::PacketTransport;
|
||||
use crate::dataplane::transport::iroh_link::{IrohTransport, TransportContext};
|
||||
use crate::dataplane::{PluginContext, PluginRequest};
|
||||
use crate::error::{Error, Result};
|
||||
use crate::identity::{DeviceIdentity, NetworkId, NetworkKeys, NetworkName, NetworkSecret};
|
||||
@@ -82,20 +84,47 @@ struct Inner {
|
||||
shutdown: Shutdown,
|
||||
accept_task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
plugin_task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
transport: std::sync::OnceLock<Arc<dyn PacketTransport>>,
|
||||
}
|
||||
|
||||
impl Drop for Inner {
|
||||
fn drop(&mut self) {
|
||||
// Nothing here awaits; this is only a safety net for a handle that was
|
||||
// dropped without an explicit shutdown.
|
||||
self.shutdown.trigger();
|
||||
for guard in [&self.accept_task, &self.plugin_task] {
|
||||
if let Ok(mut guard) = guard.lock()
|
||||
&& let Some(task) = guard.take()
|
||||
{
|
||||
task.abort();
|
||||
/// Answers the data plane transport's questions about the agent.
|
||||
///
|
||||
/// Holds a weak reference on purpose: the transport lives inside the agent, so
|
||||
/// a strong one would be a cycle and the agent — with its open databases and
|
||||
/// its directory lock — would never be released.
|
||||
#[derive(Debug)]
|
||||
struct TransportCtx(Weak<Inner>);
|
||||
|
||||
impl TransportContext for TransportCtx {
|
||||
fn snapshot(&self) -> crate::BoxFuture<'_, HashMap<NetworkId, NetworkKeys>> {
|
||||
Box::pin(async move {
|
||||
let Some(inner) = self.0.upgrade() else {
|
||||
return HashMap::new();
|
||||
};
|
||||
inner
|
||||
.networks
|
||||
.read()
|
||||
.await
|
||||
.iter()
|
||||
.map(|(id, handle)| (*id, handle.keys.clone()))
|
||||
.collect()
|
||||
})
|
||||
}
|
||||
|
||||
fn serves<'a>(&'a self, network: NetworkId, protocol: &'a str) -> crate::BoxFuture<'a, bool> {
|
||||
Box::pin(async move {
|
||||
let Some(inner) = self.0.upgrade() else {
|
||||
return false;
|
||||
};
|
||||
if !inner.networks.read().await.contains_key(&network) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
inner
|
||||
.config
|
||||
.plugins
|
||||
.iter()
|
||||
.any(|plugin| plugin.protocol_id() == protocol)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -127,9 +156,20 @@ impl Agent {
|
||||
shutdown: Shutdown::new(),
|
||||
accept_task: std::sync::Mutex::new(None),
|
||||
plugin_task: std::sync::Mutex::new(None),
|
||||
transport: std::sync::OnceLock::new(),
|
||||
config,
|
||||
});
|
||||
|
||||
// The data plane rides on iroh too, which is where it gets hole
|
||||
// punching and relay fallback from. It is a separate ALPN and a
|
||||
// separate connection, so the two planes stay independent.
|
||||
let transport: Arc<dyn PacketTransport> = Arc::new(IrohTransport::new(
|
||||
inner.adapter.clone(),
|
||||
Arc::clone(&inner.limits),
|
||||
Arc::new(TransportCtx(Arc::downgrade(&inner))) as Arc<dyn TransportContext>,
|
||||
));
|
||||
let _ = inner.transport.set(transport);
|
||||
|
||||
if let CacheOutcome::Reset(reason) = inner.storage.cache_outcome().clone() {
|
||||
let _ = inner.events.send(Event::CacheReset { reason });
|
||||
}
|
||||
@@ -257,6 +297,7 @@ impl Agent {
|
||||
discovery_interval: self.inner.config.discovery_interval,
|
||||
plugins: self.inner.config.plugins.clone(),
|
||||
hostname: self.inner.hostname.clone(),
|
||||
transport: self.inner.transport.get().cloned(),
|
||||
});
|
||||
networks.insert(network_id, handle);
|
||||
drop(networks);
|
||||
@@ -625,6 +666,12 @@ async fn handle_incoming(inner: Arc<Inner>, incoming: iroh::endpoint::Incoming)
|
||||
};
|
||||
let peer = conn.remote_id();
|
||||
|
||||
// Two protocols share the endpoint; they are told apart here and never mix.
|
||||
if conn.alpn() == crate::proto::message::DATA_ALPN {
|
||||
handle_inbound_data(inner, conn).await;
|
||||
return;
|
||||
}
|
||||
|
||||
let (mut send, mut recv) = match conn.accept_bi().await {
|
||||
Ok(streams) => streams,
|
||||
Err(err) => {
|
||||
@@ -696,6 +743,45 @@ async fn handle_incoming(inner: Arc<Inner>, incoming: iroh::endpoint::Incoming)
|
||||
}
|
||||
}
|
||||
|
||||
/// Completes an inbound data plane connection and routes it to its network.
|
||||
async fn handle_inbound_data(inner: Arc<Inner>, conn: iroh::endpoint::Connection) {
|
||||
let Some(transport) = inner.transport.get().cloned() else {
|
||||
conn.close(5u32.into(), b"data plane not ready");
|
||||
return;
|
||||
};
|
||||
// Downcasting is avoided by keeping the accept side on the concrete type.
|
||||
let Some(iroh_transport) = transport.as_ref().as_any().downcast_ref::<IrohTransport>() else {
|
||||
conn.close(5u32.into(), b"unsupported data transport");
|
||||
return;
|
||||
};
|
||||
|
||||
let inbound = match iroh_transport.accept(conn).await {
|
||||
Ok(inbound) => inbound,
|
||||
Err(err) => {
|
||||
tracing::debug!(%err, "inbound data channel rejected");
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let sender = {
|
||||
let networks = inner.networks.read().await;
|
||||
networks
|
||||
.get(&inbound.network)
|
||||
.map(|handle| handle.commands.clone())
|
||||
};
|
||||
let Some(sender) = sender else {
|
||||
// The network went away while the channel was being set up.
|
||||
return;
|
||||
};
|
||||
if sender
|
||||
.send(NetCommand::InboundLink(Box::new(inbound)))
|
||||
.await
|
||||
.is_err()
|
||||
{
|
||||
tracing::debug!("network runtime stopped before the data link was installed");
|
||||
}
|
||||
}
|
||||
|
||||
/// Picks the hostname to announce.
|
||||
///
|
||||
/// Order: explicit configuration, then what the state store already holds, then
|
||||
|
||||
+195
-2
@@ -5,7 +5,7 @@
|
||||
//! explicit [`NetworkId`], so deactivating or breaking one network cannot
|
||||
//! disturb another and cannot stop the agent.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
@@ -15,6 +15,7 @@ use tokio::sync::{broadcast, mpsc, oneshot};
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::config::{Limits, ReconnectPolicy};
|
||||
use crate::dataplane::transport::{InboundLink, PacketTransport, SharedLink};
|
||||
use crate::dataplane::{PluginCapability, SharedPlugin};
|
||||
use crate::discovery::{Candidate, CandidateSource, NetworkDiscovery};
|
||||
use crate::error::{Error, Result};
|
||||
@@ -50,6 +51,8 @@ pub(crate) enum NetCommand {
|
||||
message: ControlMessage,
|
||||
reply: oneshot::Sender<usize>,
|
||||
},
|
||||
/// A peer opened a data plane link towards us.
|
||||
InboundLink(Box<InboundLink>),
|
||||
Status {
|
||||
reply: oneshot::Sender<Box<NetworkStatus>>,
|
||||
},
|
||||
@@ -69,6 +72,7 @@ impl std::fmt::Debug for NetCommand {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
NetCommand::Inbound(_) => f.write_str("Inbound"),
|
||||
NetCommand::InboundLink(link) => write!(f, "InboundLink({})", link.protocol),
|
||||
NetCommand::Send { peer, message, .. } => {
|
||||
write!(f, "Send({}, {})", peer.fmt_short(), kind(message))
|
||||
}
|
||||
@@ -110,6 +114,15 @@ pub(crate) struct RuntimeParams {
|
||||
pub(crate) discovery_interval: Duration,
|
||||
pub(crate) plugins: Vec<SharedPlugin>,
|
||||
pub(crate) hostname: String,
|
||||
/// How data plane links are opened. `None` disables the data plane.
|
||||
pub(crate) transport: Option<Arc<dyn PacketTransport>>,
|
||||
}
|
||||
|
||||
/// Outcome of one attempt to open a data plane link.
|
||||
struct LinkOutcome {
|
||||
peer: EndpointId,
|
||||
protocol: String,
|
||||
result: Result<SharedLink, String>,
|
||||
}
|
||||
|
||||
/// Outcome of one outbound dial.
|
||||
@@ -175,6 +188,12 @@ struct Runtime {
|
||||
session_events_rx: mpsc::Receiver<SessionEvent>,
|
||||
dial_results_tx: mpsc::Sender<DialOutcome>,
|
||||
dial_results_rx: mpsc::Receiver<DialOutcome>,
|
||||
/// Live data plane links, keyed by peer and plugin protocol.
|
||||
links: HashMap<(EndpointId, String), SharedLink>,
|
||||
/// Links currently being opened, so we do not start two.
|
||||
opening: HashSet<(EndpointId, String)>,
|
||||
link_results_tx: mpsc::Sender<LinkOutcome>,
|
||||
link_results_rx: mpsc::Receiver<LinkOutcome>,
|
||||
}
|
||||
|
||||
impl Runtime {
|
||||
@@ -183,6 +202,7 @@ impl Runtime {
|
||||
let local_id = params.adapter.endpoint_id();
|
||||
let (session_events_tx, session_events_rx) = mpsc::channel(256);
|
||||
let (dial_results_tx, dial_results_rx) = mpsc::channel(64);
|
||||
let (link_results_tx, link_results_rx) = mpsc::channel(64);
|
||||
Self {
|
||||
params,
|
||||
network_id,
|
||||
@@ -196,6 +216,10 @@ impl Runtime {
|
||||
session_events_rx,
|
||||
dial_results_tx,
|
||||
dial_results_rx,
|
||||
links: HashMap::new(),
|
||||
opening: HashSet::new(),
|
||||
link_results_tx,
|
||||
link_results_rx,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -226,7 +250,15 @@ impl Runtime {
|
||||
self.handle_dial_result(result).await;
|
||||
}
|
||||
}
|
||||
_ = ticker.tick() => self.discovery_round().await,
|
||||
result = self.link_results_rx.recv() => {
|
||||
if let Some(result) = result {
|
||||
self.handle_link_result(result);
|
||||
}
|
||||
}
|
||||
_ = ticker.tick() => {
|
||||
self.discovery_round().await;
|
||||
self.ensure_links();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -242,6 +274,7 @@ impl Runtime {
|
||||
.unpublish(self.params.keys.discovery_key(), self.local_id)
|
||||
.await;
|
||||
}
|
||||
self.links.clear();
|
||||
let peers: Vec<EndpointId> = self.sessions.keys().copied().collect();
|
||||
for peer in peers {
|
||||
if let Some(session) = self.sessions.remove(&peer) {
|
||||
@@ -260,6 +293,7 @@ impl Runtime {
|
||||
NetCommand::Inbound(inbound) => {
|
||||
self.install_session(*inbound).await;
|
||||
}
|
||||
NetCommand::InboundLink(inbound) => self.install_link(*inbound),
|
||||
NetCommand::Send {
|
||||
peer,
|
||||
message,
|
||||
@@ -516,6 +550,163 @@ impl Runtime {
|
||||
}
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------ data plane
|
||||
|
||||
/// Protocol ids this agent has a plugin for.
|
||||
fn served_protocols(&self) -> Vec<String> {
|
||||
self.params
|
||||
.plugins
|
||||
.iter()
|
||||
.map(|plugin| plugin.protocol_id().to_string())
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Opens whatever data plane links are missing, and forgets dead ones.
|
||||
///
|
||||
/// Only one side dials, chosen by a rule both sides compute the same way,
|
||||
/// so two agents never open two links for the same thing.
|
||||
fn ensure_links(&mut self) {
|
||||
let Some(transport) = self.params.transport.clone() else {
|
||||
return;
|
||||
};
|
||||
let served = self.served_protocols();
|
||||
if served.is_empty() {
|
||||
return;
|
||||
}
|
||||
|
||||
let mut dead: Vec<(EndpointId, String)> = Vec::new();
|
||||
for (key, link) in &self.links {
|
||||
if link.is_closed() {
|
||||
dead.push(key.clone());
|
||||
}
|
||||
}
|
||||
for (peer, protocol) in dead {
|
||||
self.links.remove(&(peer, protocol.clone()));
|
||||
self.emit(Event::DataLinkDown {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
protocol,
|
||||
reason: "link closed".into(),
|
||||
});
|
||||
}
|
||||
|
||||
let wanted: Vec<(EndpointId, String)> = self
|
||||
.sessions
|
||||
.values()
|
||||
.flat_map(|session| {
|
||||
let peer = session.peer;
|
||||
session
|
||||
.capabilities
|
||||
.iter()
|
||||
.filter(|capability| capability.enabled)
|
||||
.map(move |capability| (peer, capability.protocol.clone()))
|
||||
})
|
||||
.filter(|(_, protocol)| served.contains(protocol))
|
||||
.collect();
|
||||
|
||||
for (peer, protocol) in wanted {
|
||||
let key = (peer, protocol.clone());
|
||||
if self.links.contains_key(&key) || self.opening.contains(&key) {
|
||||
continue;
|
||||
}
|
||||
// The smaller endpoint id dials; the other side accepts. Both
|
||||
// compute this identically, so exactly one link is created.
|
||||
if self.local_id.as_bytes() >= peer.as_bytes() {
|
||||
continue;
|
||||
}
|
||||
self.opening.insert(key);
|
||||
|
||||
let results = self.link_results_tx.clone();
|
||||
let transport = Arc::clone(&transport);
|
||||
let network = self.network_id;
|
||||
let shutdown = self.shutdown.clone();
|
||||
tokio::spawn(async move {
|
||||
let result = tokio::select! {
|
||||
biased;
|
||||
_ = shutdown.wait() => Err("network deactivated".to_string()),
|
||||
result = transport.open(network, peer, &protocol) => {
|
||||
result.map_err(|err| err.to_string())
|
||||
}
|
||||
};
|
||||
let _ = results
|
||||
.send(LinkOutcome {
|
||||
peer,
|
||||
protocol,
|
||||
result,
|
||||
})
|
||||
.await;
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_link_result(&mut self, outcome: LinkOutcome) {
|
||||
let key = (outcome.peer, outcome.protocol.clone());
|
||||
self.opening.remove(&key);
|
||||
match outcome.result {
|
||||
Ok(link) => self.adopt_link(outcome.peer, outcome.protocol, link),
|
||||
Err(reason) => {
|
||||
// A data plane that cannot be set up is reported, never fatal.
|
||||
self.metrics.data_link_failures += 1;
|
||||
self.emit(Event::DataLinkDown {
|
||||
network: self.network_id,
|
||||
peer: outcome.peer,
|
||||
protocol: outcome.protocol,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn install_link(&mut self, inbound: InboundLink) {
|
||||
self.adopt_link(inbound.peer, inbound.protocol, inbound.link);
|
||||
}
|
||||
|
||||
/// Hands a link to the plugin that owns its protocol.
|
||||
fn adopt_link(&mut self, peer: EndpointId, protocol: String, link: SharedLink) {
|
||||
let Some(plugin) = self
|
||||
.params
|
||||
.plugins
|
||||
.iter()
|
||||
.find(|plugin| plugin.protocol_id() == protocol)
|
||||
.cloned()
|
||||
else {
|
||||
return;
|
||||
};
|
||||
|
||||
let path = link.path_description();
|
||||
let max_datagram = link.max_datagram_size();
|
||||
self.links
|
||||
.insert((peer, protocol.clone()), Arc::clone(&link));
|
||||
plugin.on_peer_link(self.network_id, peer, link);
|
||||
self.metrics.data_links_established += 1;
|
||||
self.emit(Event::DataLinkUp {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
protocol,
|
||||
path,
|
||||
max_datagram,
|
||||
});
|
||||
}
|
||||
|
||||
/// Drops every link to a peer.
|
||||
fn drop_links_for(&mut self, peer: EndpointId) {
|
||||
let keys: Vec<(EndpointId, String)> = self
|
||||
.links
|
||||
.keys()
|
||||
.filter(|(id, _)| *id == peer)
|
||||
.cloned()
|
||||
.collect();
|
||||
for key in keys {
|
||||
self.links.remove(&key);
|
||||
self.emit(Event::DataLinkDown {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
protocol: key.1,
|
||||
reason: "peer session ended".into(),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------- sessions
|
||||
|
||||
async fn install_session(&mut self, inbound: InboundSession) {
|
||||
@@ -682,6 +873,7 @@ impl Runtime {
|
||||
session.conn.close(0u32.into(), b"session ended");
|
||||
}
|
||||
self.metrics.disconnects += 1;
|
||||
self.drop_links_for(peer);
|
||||
for plugin in &self.params.plugins {
|
||||
plugin.on_peer_gone(self.network_id, peer);
|
||||
}
|
||||
@@ -711,6 +903,7 @@ impl Runtime {
|
||||
session.capabilities = capabilities.clone();
|
||||
}
|
||||
self.dispatch_capabilities(peer, &capabilities);
|
||||
self.ensure_links();
|
||||
}
|
||||
ControlMessage::Ping { seq, payload } => {
|
||||
let pong = ControlMessage::Pong {
|
||||
|
||||
@@ -94,6 +94,10 @@ pub struct NetworkMetrics {
|
||||
pub protocol_violations: u64,
|
||||
/// Errors reported by IP plugins. Never fatal.
|
||||
pub plugin_errors: u64,
|
||||
/// Data plane links that were established.
|
||||
pub data_links_established: u64,
|
||||
/// Attempts to open a data plane link that failed.
|
||||
pub data_link_failures: u64,
|
||||
}
|
||||
|
||||
/// Status of one network.
|
||||
|
||||
@@ -0,0 +1,554 @@
|
||||
//! The `tsunagi` command line agent.
|
||||
//!
|
||||
//! This binary owns everything the library deliberately refuses to do: it
|
||||
//! starts the tokio runtime, installs a logging subscriber and handles
|
||||
//! Ctrl-C. The library itself does none of that.
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use std::net::SocketAddr;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use clap::{Args, Parser, Subcommand, ValueEnum};
|
||||
use tsunagi::agent::Event;
|
||||
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
|
||||
use tsunagi::dataplane::IpPlugin;
|
||||
use tsunagi::dataplane::wireguard::{
|
||||
MemoryTunFactory, TunFactory, WireguardConfig, WireguardPlugin,
|
||||
};
|
||||
use tsunagi::discovery::{CompositeDiscovery, NetworkDiscovery, StaticBootstrap};
|
||||
use tsunagi::identity::{NetworkName, NetworkSecret};
|
||||
use tsunagi::iroh_types::EndpointAddr;
|
||||
use tsunagi::{Agent, NetworkId};
|
||||
|
||||
/// A small agent for private mesh networks.
|
||||
#[derive(Debug, Parser)]
|
||||
#[command(name = "tsunagi", version, about, long_about = None)]
|
||||
struct Cli {
|
||||
/// Log filter, for example `info` or `tsunagi=debug`.
|
||||
#[arg(long, global = true, env = "TSUNAGI_LOG", default_value = "warn")]
|
||||
log: String,
|
||||
|
||||
#[command(subcommand)]
|
||||
command: Command,
|
||||
}
|
||||
|
||||
#[derive(Debug, Subcommand)]
|
||||
enum Command {
|
||||
/// Generates a fresh network secret and prints it.
|
||||
Secret,
|
||||
/// Reports what this machine can and cannot do.
|
||||
Doctor(PathArgs),
|
||||
/// Shows this device's identity without joining anything.
|
||||
Id(PathArgs),
|
||||
/// Joins a network and runs until interrupted.
|
||||
Up(UpArgs),
|
||||
}
|
||||
|
||||
#[derive(Debug, Args, Clone)]
|
||||
struct PathArgs {
|
||||
/// Directory for the mandatory state. Defaults to the platform location.
|
||||
#[arg(long, env = "TSUNAGI_STATE_DIR")]
|
||||
state_dir: Option<PathBuf>,
|
||||
/// Directory for the disposable cache. Defaults to the platform location.
|
||||
#[arg(long, env = "TSUNAGI_CACHE_DIR")]
|
||||
cache_dir: Option<PathBuf>,
|
||||
}
|
||||
|
||||
impl PathArgs {
|
||||
fn resolve(&self) -> Result<StoragePaths, tsunagi::Error> {
|
||||
let mut paths = StoragePaths::user_default()?;
|
||||
if let Some(dir) = &self.state_dir {
|
||||
paths.state_dir = dir.clone();
|
||||
}
|
||||
if let Some(dir) = &self.cache_dir {
|
||||
paths.cache_dir = dir.clone();
|
||||
}
|
||||
Ok(paths)
|
||||
}
|
||||
}
|
||||
|
||||
/// How much external connectivity machinery the endpoint may use.
|
||||
#[derive(Debug, Clone, Copy, ValueEnum)]
|
||||
enum Transport {
|
||||
/// Loopback and the local network only. No relays, no address lookup.
|
||||
Local,
|
||||
/// Public address lookup, but no relays.
|
||||
Direct,
|
||||
/// iroh's defaults: address lookup plus the public n0 relays.
|
||||
N0,
|
||||
}
|
||||
|
||||
impl From<Transport> for TransportPolicy {
|
||||
fn from(value: Transport) -> Self {
|
||||
match value {
|
||||
Transport::Local => TransportPolicy::LocalOnly,
|
||||
Transport::Direct => TransportPolicy::DirectOnly,
|
||||
Transport::N0 => TransportPolicy::N0Defaults,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Args)]
|
||||
struct UpArgs {
|
||||
#[command(flatten)]
|
||||
paths: PathArgs,
|
||||
|
||||
/// Network name. Must be identical on every participant.
|
||||
#[arg(long, short = 'n')]
|
||||
network: String,
|
||||
|
||||
/// The shared secret, as printed by `tsunagi secret`.
|
||||
#[arg(
|
||||
long,
|
||||
short = 's',
|
||||
env = "TSUNAGI_SECRET",
|
||||
conflicts_with = "secret_file"
|
||||
)]
|
||||
secret: Option<String>,
|
||||
|
||||
/// Read the shared secret from a file instead of the command line.
|
||||
#[arg(long)]
|
||||
secret_file: Option<PathBuf>,
|
||||
|
||||
/// Hostname to announce. Defaults to the machine's.
|
||||
#[arg(long)]
|
||||
hostname: Option<String>,
|
||||
|
||||
/// How much external connectivity to use.
|
||||
#[arg(long, value_enum, default_value_t = Transport::N0)]
|
||||
transport: Transport,
|
||||
|
||||
/// A peer to contact, as `<endpoint-id>` or `<endpoint-id>@<ip:port>,...`.
|
||||
///
|
||||
/// One agent needs to know another to begin with. Repeat for several.
|
||||
#[arg(long = "peer", value_name = "PEER")]
|
||||
peers: Vec<String>,
|
||||
|
||||
/// Local address to bind. Repeat for several; defaults to iroh's choice.
|
||||
#[arg(long = "bind", value_name = "ADDR")]
|
||||
binds: Vec<SocketAddr>,
|
||||
|
||||
/// Run the WireGuard data plane.
|
||||
#[arg(long)]
|
||||
wireguard: bool,
|
||||
|
||||
/// Do not create a real network interface.
|
||||
///
|
||||
/// The WireGuard tunnels still run and handshake, so the mesh can be
|
||||
/// verified with no privileges; traffic just does not reach the
|
||||
/// operating system.
|
||||
#[arg(long)]
|
||||
no_tun: bool,
|
||||
|
||||
/// Interface name prefix for the WireGuard data plane.
|
||||
#[arg(long, default_value = "tsun")]
|
||||
wg_prefix: String,
|
||||
|
||||
/// Interface MTU for the WireGuard data plane.
|
||||
#[arg(long)]
|
||||
wg_mtu: Option<u32>,
|
||||
|
||||
/// How often to print a status summary, in seconds. Zero disables it.
|
||||
#[arg(long, default_value_t = 15)]
|
||||
status_interval: u64,
|
||||
}
|
||||
|
||||
impl UpArgs {
|
||||
fn load_secret(&self) -> Result<NetworkSecret, Box<dyn std::error::Error>> {
|
||||
let text = match (&self.secret, &self.secret_file) {
|
||||
(Some(secret), _) => secret.clone(),
|
||||
(None, Some(path)) => std::fs::read_to_string(path)?,
|
||||
(None, None) => {
|
||||
return Err("provide --secret, --secret-file or TSUNAGI_SECRET".into());
|
||||
}
|
||||
};
|
||||
let text = text.trim();
|
||||
// The canonical form is preferred, but a raw high-entropy value is
|
||||
// accepted so an existing secret can be reused.
|
||||
match NetworkSecret::decode(text) {
|
||||
Ok(secret) => Ok(secret),
|
||||
Err(_) => Ok(NetworkSecret::from_bytes(text.as_bytes().to_vec())?),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Parses `<endpoint-id>` or `<endpoint-id>@<ip:port>,<ip:port>`.
|
||||
fn parse_peer(text: &str) -> Result<EndpointAddr, String> {
|
||||
let (id_text, addr_text) = match text.split_once('@') {
|
||||
Some((id, addrs)) => (id, Some(addrs)),
|
||||
None => (text, None),
|
||||
};
|
||||
let id: tsunagi::iroh_types::EndpointId = id_text
|
||||
.parse()
|
||||
.map_err(|err| format!("`{id_text}` is not an endpoint id: {err}"))?;
|
||||
let mut addr = EndpointAddr::new(id);
|
||||
if let Some(addrs) = addr_text {
|
||||
for entry in addrs.split(',') {
|
||||
let socket: SocketAddr = entry
|
||||
.trim()
|
||||
.parse()
|
||||
.map_err(|err| format!("`{entry}` is not an address: {err}"))?;
|
||||
addr = addr.with_ip_addr(socket);
|
||||
}
|
||||
}
|
||||
Ok(addr)
|
||||
}
|
||||
|
||||
fn main() -> std::process::ExitCode {
|
||||
let cli = Cli::parse();
|
||||
|
||||
tracing_subscriber::fmt()
|
||||
.with_env_filter(tracing_subscriber::EnvFilter::new(&cli.log))
|
||||
.with_writer(std::io::stderr)
|
||||
.init();
|
||||
|
||||
// The library never starts a runtime; this binary owns it.
|
||||
let runtime = match tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
{
|
||||
Ok(runtime) => runtime,
|
||||
Err(err) => {
|
||||
eprintln!("cannot start the async runtime: {err}");
|
||||
return std::process::ExitCode::FAILURE;
|
||||
}
|
||||
};
|
||||
|
||||
match runtime.block_on(run(cli.command)) {
|
||||
Ok(()) => std::process::ExitCode::SUCCESS,
|
||||
Err(err) => {
|
||||
eprintln!("error: {err}");
|
||||
std::process::ExitCode::FAILURE
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn run(command: Command) -> Result<(), Box<dyn std::error::Error>> {
|
||||
match command {
|
||||
Command::Secret => {
|
||||
let secret = NetworkSecret::generate();
|
||||
println!("{}", secret.encode().as_str());
|
||||
eprintln!(
|
||||
"\nShare this with every participant, over a channel you trust.\n\
|
||||
Anyone who has it can join the network."
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
Command::Doctor(paths) => doctor(paths).await,
|
||||
Command::Id(paths) => show_id(paths).await,
|
||||
Command::Up(args) => up(args).await,
|
||||
}
|
||||
}
|
||||
|
||||
async fn show_id(paths: PathArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let paths = paths.resolve()?;
|
||||
println!("state directory {}", paths.state_dir.display());
|
||||
println!("cache directory {}", paths.cache_dir.display());
|
||||
|
||||
let agent =
|
||||
Agent::spawn(AgentConfig::new(paths).with_transport(TransportPolicy::LocalOnly)).await?;
|
||||
println!("endpoint id {}", agent.endpoint_id());
|
||||
println!("hostname {}", agent.hostname());
|
||||
for network in agent.list_networks().await? {
|
||||
println!(
|
||||
"network {} ({}) auto-start={}",
|
||||
network.name, network.network_id, network.auto_start
|
||||
);
|
||||
}
|
||||
agent.shutdown().await;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn doctor(paths: PathArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let paths = paths.resolve()?;
|
||||
println!("tsunagi doctor\n");
|
||||
|
||||
println!("state directory {}", paths.state_dir.display());
|
||||
println!("cache directory {}", paths.cache_dir.display());
|
||||
match std::fs::create_dir_all(&paths.state_dir) {
|
||||
Ok(()) => println!(" writable yes"),
|
||||
Err(err) => println!(" writable NO ({err})"),
|
||||
}
|
||||
|
||||
println!("\ncontrol plane");
|
||||
println!(" needs outbound UDP; no privileges");
|
||||
println!(" status always available");
|
||||
|
||||
println!("\ndata plane (WireGuard)");
|
||||
println!(" implementation userspace (boringtun); no kernel module needed");
|
||||
#[cfg(feature = "tun-device")]
|
||||
{
|
||||
let tun_path = std::path::Path::new("/dev/net/tun");
|
||||
if cfg!(target_os = "linux") {
|
||||
if tun_path.exists() {
|
||||
match std::fs::OpenOptions::new()
|
||||
.read(true)
|
||||
.write(true)
|
||||
.open(tun_path)
|
||||
{
|
||||
Ok(_) => println!(" /dev/net/tun openable"),
|
||||
Err(err) => println!(" /dev/net/tun present but not openable ({err})"),
|
||||
}
|
||||
} else {
|
||||
println!(" /dev/net/tun missing (load the `tun` module)");
|
||||
}
|
||||
}
|
||||
println!(" interfaces supported on this build");
|
||||
}
|
||||
#[cfg(not(feature = "tun-device"))]
|
||||
println!(" interfaces not built in (enable the `tun-device` feature)");
|
||||
|
||||
#[cfg(unix)]
|
||||
{
|
||||
// Creating a network interface needs CAP_NET_ADMIN, which in practice
|
||||
// means root unless capabilities were granted explicitly.
|
||||
let euid = std::fs::metadata("/proc/self").ok().map(|_| ());
|
||||
let _ = euid;
|
||||
println!(
|
||||
" privileges creating an interface needs CAP_NET_ADMIN; \
|
||||
use --no-tun to run without it"
|
||||
);
|
||||
}
|
||||
|
||||
println!("\nlocal addresses");
|
||||
let state = netwatch_addresses().await;
|
||||
if state.is_empty() {
|
||||
println!(" none found");
|
||||
}
|
||||
for addr in state {
|
||||
println!(" {addr}");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn netwatch_addresses() -> Vec<std::net::IpAddr> {
|
||||
// Best effort; used for diagnostics only.
|
||||
let state = netwatch::interfaces::State::new().await;
|
||||
let mut addresses = state.local_addresses.regular;
|
||||
addresses.sort();
|
||||
addresses.dedup();
|
||||
addresses
|
||||
}
|
||||
|
||||
async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let name = NetworkName::new(args.network.clone())?;
|
||||
let secret = args.load_secret()?;
|
||||
let paths = args.paths.resolve()?;
|
||||
|
||||
let mut bootstrap: Vec<EndpointAddr> = Vec::new();
|
||||
for peer in &args.peers {
|
||||
bootstrap.push(parse_peer(peer)?);
|
||||
}
|
||||
let discovery: Arc<dyn NetworkDiscovery> =
|
||||
Arc::new(CompositeDiscovery::new([
|
||||
Arc::new(StaticBootstrap::new(bootstrap)) as Arc<dyn NetworkDiscovery>,
|
||||
]));
|
||||
|
||||
let mut config = AgentConfig::new(paths.clone())
|
||||
.with_transport(args.transport.into())
|
||||
.with_discovery(discovery)
|
||||
.with_discovery_interval(Duration::from_secs(5));
|
||||
if let Some(hostname) = &args.hostname {
|
||||
config = config.with_hostname(hostname.clone());
|
||||
}
|
||||
if !args.binds.is_empty() {
|
||||
config = config.with_bind_addrs(args.binds.clone());
|
||||
}
|
||||
|
||||
// The data plane is optional and never required for the control plane.
|
||||
let wireguard = if args.wireguard {
|
||||
let tun_factory: Arc<dyn TunFactory> = if args.no_tun {
|
||||
Arc::new(MemoryTunFactory::new())
|
||||
} else {
|
||||
system_tun_factory()?
|
||||
};
|
||||
let mut wg = WireguardConfig::new(paths.state_dir.join("wireguard"))
|
||||
.with_interface_prefix(args.wg_prefix.clone());
|
||||
if let Some(mtu) = args.wg_mtu {
|
||||
wg = wg.with_mtu(mtu);
|
||||
}
|
||||
let plugin = WireguardPlugin::open(wg, tun_factory).await?;
|
||||
config = config.with_plugin(plugin.clone() as Arc<dyn IpPlugin>);
|
||||
Some(plugin)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let agent = Agent::spawn(config).await?;
|
||||
let mut events = agent.subscribe();
|
||||
let network = agent.join_network(&name, &secret).await?;
|
||||
|
||||
println!("tsunagi is up");
|
||||
println!(" endpoint id {}", agent.endpoint_id());
|
||||
println!(" hostname {}", agent.hostname());
|
||||
println!(" network {name} ({network})");
|
||||
println!(" state {}", paths.state_dir.display());
|
||||
if args.peers.is_empty() {
|
||||
println!(
|
||||
"\nNo --peer was given, so this agent waits to be contacted.\n\
|
||||
On the other machine run:\n\n tsunagi up --network {name} --secret <secret> \\\n --peer {}\n",
|
||||
agent.endpoint_id()
|
||||
);
|
||||
}
|
||||
println!("Press Ctrl-C to stop.\n");
|
||||
|
||||
let status_every =
|
||||
(args.status_interval > 0).then(|| Duration::from_secs(args.status_interval));
|
||||
let mut ticker = status_every.map(tokio::time::interval);
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
signal = tokio::signal::ctrl_c() => {
|
||||
if let Err(err) = signal {
|
||||
eprintln!("cannot listen for Ctrl-C: {err}");
|
||||
}
|
||||
println!("\nstopping...");
|
||||
break;
|
||||
}
|
||||
event = events.recv() => match event {
|
||||
Ok(event) => print_event(&event),
|
||||
Err(tokio::sync::broadcast::error::RecvError::Lagged(skipped)) => {
|
||||
println!(" (missed {skipped} events)");
|
||||
}
|
||||
Err(tokio::sync::broadcast::error::RecvError::Closed) => break,
|
||||
},
|
||||
_ = async {
|
||||
match ticker.as_mut() {
|
||||
Some(ticker) => { ticker.tick().await; }
|
||||
None => std::future::pending::<()>().await,
|
||||
}
|
||||
}, if ticker.is_some() => {
|
||||
print_status(&agent, network, wireguard.as_deref()).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
agent.shutdown().await;
|
||||
println!("stopped.");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(feature = "tun-device")]
|
||||
fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
|
||||
use tsunagi::dataplane::wireguard::SystemTunFactory;
|
||||
Ok(Arc::new(SystemTunFactory::new()))
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "tun-device"))]
|
||||
fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
|
||||
Err("this build has no interface support; rebuild with the `tun-device` feature or pass --no-tun".into())
|
||||
}
|
||||
|
||||
fn print_event(event: &Event) {
|
||||
match event {
|
||||
Event::PeerConnected {
|
||||
peer,
|
||||
transport,
|
||||
rtt,
|
||||
..
|
||||
} => println!(
|
||||
" + peer {} connected over {transport:?} rtt={rtt:?}",
|
||||
peer.fmt_short()
|
||||
),
|
||||
Event::PeerDisconnected { peer, reason, .. } => {
|
||||
println!(" - peer {} gone: {reason}", peer.fmt_short())
|
||||
}
|
||||
Event::DataLinkUp {
|
||||
peer,
|
||||
protocol,
|
||||
path,
|
||||
max_datagram,
|
||||
..
|
||||
} => println!(
|
||||
" + data link to {} for {protocol}: {path}, datagram {max_datagram}",
|
||||
peer.fmt_short()
|
||||
),
|
||||
Event::DataLinkDown {
|
||||
peer,
|
||||
protocol,
|
||||
reason,
|
||||
..
|
||||
} => println!(
|
||||
" - data link to {} for {protocol}: {reason}",
|
||||
peer.fmt_short()
|
||||
),
|
||||
Event::HandshakeRejected { peer, reason, .. } => println!(
|
||||
" ! rejected {}: {reason}",
|
||||
peer.map(|peer| peer.fmt_short().to_string())
|
||||
.unwrap_or_else(|| "a caller".into())
|
||||
),
|
||||
Event::PluginError {
|
||||
protocol, reason, ..
|
||||
} => println!(" ! {protocol}: {reason}"),
|
||||
Event::CacheReset { reason } => println!(" ! cache was reset: {reason}"),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
async fn print_status(agent: &Agent, network: NetworkId, wireguard: Option<&WireguardPlugin>) {
|
||||
let Ok(status) = agent.network_status(network).await else {
|
||||
return;
|
||||
};
|
||||
println!("\n--- status ---");
|
||||
println!(
|
||||
"control: {} peer(s), {} dial failure(s), {} handshake failure(s)",
|
||||
status.peers.len(),
|
||||
status.metrics.dial_failures,
|
||||
status.metrics.handshake_failures
|
||||
);
|
||||
for peer in &status.peers {
|
||||
println!(
|
||||
" {} {} {:?} rtt={:?}",
|
||||
peer.endpoint_id.fmt_short(),
|
||||
peer.hostname.as_deref().unwrap_or("?"),
|
||||
peer.transport,
|
||||
peer.rtt
|
||||
);
|
||||
}
|
||||
|
||||
if let Some(plugin) = wireguard
|
||||
&& let Some(view) = plugin.overview(network)
|
||||
{
|
||||
println!(
|
||||
"wireguard: {} on {}/{} mtu {}, {}/{} tunnel(s) established",
|
||||
view.interface,
|
||||
view.overlay_address,
|
||||
view.overlay_prefix_len,
|
||||
view.mtu,
|
||||
view.established_peers(),
|
||||
view.peers.len()
|
||||
);
|
||||
for peer in &view.peers {
|
||||
match &peer.tunnel {
|
||||
Some(tunnel) => println!(
|
||||
" {} {} {} tx={} rx={} dropped={} path={}",
|
||||
peer.public_key.fmt_short(),
|
||||
peer.overlay_address,
|
||||
match tunnel.health.since_handshake {
|
||||
Some(since) => format!("handshake {}s ago", since.as_secs()),
|
||||
None => "NOT HANDSHAKEN".to_string(),
|
||||
},
|
||||
tunnel.stats.tx_packets,
|
||||
tunnel.stats.rx_packets,
|
||||
tunnel.stats.dropped_wrong_source + tunnel.stats.dropped_oversize,
|
||||
tunnel.path
|
||||
),
|
||||
None => println!(
|
||||
" {} {} waiting for a data link",
|
||||
peer.public_key.fmt_short(),
|
||||
peer.overlay_address
|
||||
),
|
||||
}
|
||||
}
|
||||
if view.unroutable_packets > 0 {
|
||||
println!(
|
||||
" {} packet(s) for unknown addresses",
|
||||
view.unroutable_packets
|
||||
);
|
||||
}
|
||||
}
|
||||
println!();
|
||||
}
|
||||
@@ -19,6 +19,7 @@
|
||||
//! A data plane failure never stops the daemon: errors returned here are
|
||||
//! recorded and surfaced, the control plane keeps running.
|
||||
|
||||
pub mod transport;
|
||||
pub mod wireguard;
|
||||
|
||||
use std::sync::Arc;
|
||||
@@ -29,6 +30,8 @@ use tokio::sync::mpsc;
|
||||
use crate::BoxFuture;
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
pub use transport::{PacketLink, PacketTransport, SharedLink, TransportError};
|
||||
|
||||
/// Maximum length of a plugin protocol identifier.
|
||||
pub const MAX_PROTOCOL_ID_LEN: usize = 32;
|
||||
|
||||
@@ -194,7 +197,17 @@ pub trait IpPlugin: Send + Sync + std::fmt::Debug + 'static {
|
||||
capability: &PluginCapability,
|
||||
) -> std::result::Result<(), PluginError>;
|
||||
|
||||
/// A data plane link to a peer is available for this plugin's protocol.
|
||||
///
|
||||
/// The plugin moves its packets over this link and never learns how the
|
||||
/// link is carried. A new link for a peer replaces any previous one.
|
||||
fn on_peer_link(&self, network: NetworkId, peer: EndpointId, link: SharedLink) {
|
||||
let _ = (network, peer, link);
|
||||
}
|
||||
|
||||
/// Called when a peer's session in a network goes away.
|
||||
///
|
||||
/// Any link handed to the plugin for that peer must be dropped here.
|
||||
fn on_peer_gone(&self, network: NetworkId, peer: EndpointId);
|
||||
|
||||
/// Called when a network is deactivated locally.
|
||||
|
||||
@@ -0,0 +1,317 @@
|
||||
//! A data plane transport built on iroh.
|
||||
//!
|
||||
//! This is where the data plane gets NAT traversal from. iroh hole punches a
|
||||
//! direct path between two peers when it can and falls back to a relay when it
|
||||
//! cannot, so every plugin inherits that without implementing STUN, ICE or a
|
||||
//! relay of its own.
|
||||
//!
|
||||
//! Data connections are separate from control connections in every way that
|
||||
//! matters: their own ALPN ([`DATA_ALPN`]), their own QUIC connection, their
|
||||
//! own congestion control. They carry one plugin protocol for one network.
|
||||
//! A data connection that breaks or floods cannot disturb the control plane.
|
||||
//!
|
||||
//! Packets travel as QUIC datagrams: unreliable and unordered, which is what a
|
||||
//! tunnelled UDP protocol wants, and free of the head-of-line blocking a
|
||||
//! stream would add.
|
||||
//!
|
||||
//! The channel is authenticated exactly like a control connection — the same
|
||||
//! membership handshake, bound to the same network — so a data link cannot be
|
||||
//! opened by someone who does not know the network secret.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
use bytes::Bytes;
|
||||
use iroh::EndpointId;
|
||||
use iroh::endpoint::{Connection, RecvStream, SendStream};
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::config::Limits;
|
||||
use crate::error::ProtocolError;
|
||||
use crate::identity::{NetworkId, NetworkKeys};
|
||||
use crate::net::EndpointAdapter;
|
||||
use crate::proto::handshake;
|
||||
use crate::proto::message::{
|
||||
DATA_ALPN, DataOpen, DataOpenAck, MAX_DATA_PROTOCOL_LEN, decode, encode,
|
||||
};
|
||||
use crate::proto::{read_frame, write_frame};
|
||||
|
||||
use super::{InboundLink, PacketLink, PacketTransport, SharedLink, TransportError};
|
||||
|
||||
/// What the iroh transport needs from the agent.
|
||||
///
|
||||
/// Implemented by the agent, which is the only thing that knows which networks
|
||||
/// are active and which plugin protocols are served.
|
||||
pub trait TransportContext: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// Key material of every network that is active right now.
|
||||
///
|
||||
/// Taken as one snapshot because the membership handshake resolves the
|
||||
/// requested network synchronously, exactly as the control plane's accept
|
||||
/// path does.
|
||||
fn snapshot<'a>(&'a self) -> BoxFuture<'a, HashMap<NetworkId, NetworkKeys>>;
|
||||
|
||||
/// Whether a plugin protocol is served in a network.
|
||||
fn serves<'a>(&'a self, network: NetworkId, protocol: &'a str) -> BoxFuture<'a, bool>;
|
||||
}
|
||||
|
||||
/// One authenticated datagram channel over an iroh connection.
|
||||
#[derive(Debug)]
|
||||
pub struct IrohLink {
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
conn: Connection,
|
||||
max_datagram: usize,
|
||||
// Kept alive so the peer sees the channel as open; the connection closes
|
||||
// when the link is dropped.
|
||||
_send: tokio::sync::Mutex<SendStream>,
|
||||
_recv: tokio::sync::Mutex<RecvStream>,
|
||||
}
|
||||
|
||||
impl IrohLink {
|
||||
fn new(
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
conn: Connection,
|
||||
peer_limit: usize,
|
||||
send: SendStream,
|
||||
recv: RecvStream,
|
||||
) -> Self {
|
||||
let local_limit = conn.max_datagram_size().unwrap_or(0);
|
||||
// Both ends must agree, so the smaller limit wins.
|
||||
let max_datagram = local_limit.min(peer_limit);
|
||||
Self {
|
||||
network,
|
||||
peer,
|
||||
conn,
|
||||
max_datagram,
|
||||
_send: tokio::sync::Mutex::new(send),
|
||||
_recv: tokio::sync::Mutex::new(recv),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl PacketLink for IrohLink {
|
||||
fn network(&self) -> NetworkId {
|
||||
self.network
|
||||
}
|
||||
|
||||
fn peer(&self) -> EndpointId {
|
||||
self.peer
|
||||
}
|
||||
|
||||
fn max_datagram_size(&self) -> usize {
|
||||
self.max_datagram
|
||||
}
|
||||
|
||||
fn send(&self, payload: Bytes) -> Result<(), TransportError> {
|
||||
if payload.len() > self.max_datagram {
|
||||
return Err(TransportError::TooLarge {
|
||||
size: payload.len(),
|
||||
limit: self.max_datagram,
|
||||
});
|
||||
}
|
||||
self.conn.send_datagram(payload).map_err(|err| {
|
||||
use iroh::endpoint::SendDatagramError;
|
||||
match err {
|
||||
SendDatagramError::ConnectionLost(_) => TransportError::Closed,
|
||||
other => TransportError::Other(other.to_string()),
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
|
||||
Box::pin(async move { self.conn.read_datagram().await.ok() })
|
||||
}
|
||||
|
||||
fn closed(&self) -> BoxFuture<'_, ()> {
|
||||
Box::pin(async move {
|
||||
let _ = self.conn.closed().await;
|
||||
})
|
||||
}
|
||||
|
||||
fn is_closed(&self) -> bool {
|
||||
self.conn.close_reason().is_some()
|
||||
}
|
||||
|
||||
fn path_description(&self) -> String {
|
||||
// Report what iroh actually knows, never a guess.
|
||||
let snapshot = crate::net::snapshot_connection(&self.conn);
|
||||
match snapshot.paths.iter().find(|path| path.is_selected) {
|
||||
Some(path) => format!("{:?} via {:?}", snapshot.transport, path.remote),
|
||||
None => format!("{:?}, no selected path yet", snapshot.transport),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Opens and accepts data plane links over iroh.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct IrohTransport {
|
||||
adapter: EndpointAdapter,
|
||||
limits: Arc<Limits>,
|
||||
lookup: Arc<dyn TransportContext>,
|
||||
}
|
||||
|
||||
impl IrohTransport {
|
||||
/// Creates a transport on an existing endpoint.
|
||||
pub fn new(
|
||||
adapter: EndpointAdapter,
|
||||
limits: Arc<Limits>,
|
||||
lookup: Arc<dyn TransportContext>,
|
||||
) -> Self {
|
||||
Self {
|
||||
adapter,
|
||||
limits,
|
||||
lookup,
|
||||
}
|
||||
}
|
||||
|
||||
/// Completes an inbound data connection that the accept loop routed here.
|
||||
///
|
||||
/// The membership handshake runs first, exactly as on a control
|
||||
/// connection, so an unauthenticated caller never reaches a plugin.
|
||||
pub async fn accept(&self, conn: Connection) -> Result<InboundLink, TransportError> {
|
||||
let peer = conn.remote_id();
|
||||
let (mut send, mut recv) = conn
|
||||
.accept_bi()
|
||||
.await
|
||||
.map_err(|err| TransportError::Other(format!("no data channel stream: {err}")))?;
|
||||
|
||||
let local_id = self.adapter.endpoint_id();
|
||||
let known = self.lookup.snapshot().await;
|
||||
let outcome = handshake::respond(
|
||||
&conn,
|
||||
&mut send,
|
||||
&mut recv,
|
||||
local_id,
|
||||
&self.limits,
|
||||
|network| known.get(&network).cloned(),
|
||||
)
|
||||
.await
|
||||
.map_err(|err| match err {
|
||||
ProtocolError::UnknownNetwork => {
|
||||
TransportError::Other("network is not active for the data plane".into())
|
||||
}
|
||||
other => TransportError::Other(other.to_string()),
|
||||
})?;
|
||||
|
||||
let open: DataOpen = decode(
|
||||
&read_frame(&mut recv, self.limits.max_frame_len)
|
||||
.await
|
||||
.map_err(|err| TransportError::Other(err.to_string()))?,
|
||||
)
|
||||
.map_err(|err| TransportError::Other(err.to_string()))?;
|
||||
|
||||
if open.protocol.is_empty() || open.protocol.len() > MAX_DATA_PROTOCOL_LEN {
|
||||
return Err(TransportError::Other(
|
||||
"data channel protocol id is out of bounds".into(),
|
||||
));
|
||||
}
|
||||
|
||||
let serves = self.lookup.serves(outcome.network_id, &open.protocol).await;
|
||||
let max_datagram = conn.max_datagram_size().unwrap_or(0);
|
||||
let ack = DataOpenAck {
|
||||
accepted: serves,
|
||||
max_datagram: max_datagram as u32,
|
||||
};
|
||||
write_frame(
|
||||
&mut send,
|
||||
&encode(&ack).map_err(|err| TransportError::Other(err.to_string()))?,
|
||||
self.limits.max_frame_len,
|
||||
)
|
||||
.await
|
||||
.map_err(|err| TransportError::Other(err.to_string()))?;
|
||||
|
||||
if !serves {
|
||||
conn.close(4u32.into(), b"no plugin for this protocol");
|
||||
return Err(TransportError::Declined(open.protocol));
|
||||
}
|
||||
|
||||
let link = IrohLink::new(outcome.network_id, peer, conn, usize::MAX, send, recv);
|
||||
Ok(InboundLink {
|
||||
network: outcome.network_id,
|
||||
peer,
|
||||
protocol: open.protocol,
|
||||
link: Arc::new(link),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl PacketTransport for IrohTransport {
|
||||
fn name(&self) -> &str {
|
||||
"iroh"
|
||||
}
|
||||
|
||||
fn as_any(&self) -> &dyn std::any::Any {
|
||||
self
|
||||
}
|
||||
|
||||
fn open<'a>(
|
||||
&'a self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
protocol: &'a str,
|
||||
) -> BoxFuture<'a, Result<SharedLink, TransportError>> {
|
||||
Box::pin(async move {
|
||||
if protocol.is_empty() || protocol.len() > MAX_DATA_PROTOCOL_LEN {
|
||||
return Err(TransportError::Other(
|
||||
"data channel protocol id is out of bounds".into(),
|
||||
));
|
||||
}
|
||||
let keys = self
|
||||
.lookup
|
||||
.snapshot()
|
||||
.await
|
||||
.remove(&network)
|
||||
.ok_or_else(|| TransportError::Other("network is not active".into()))?;
|
||||
|
||||
let addr = iroh::EndpointAddr::new(peer);
|
||||
let conn = self
|
||||
.adapter
|
||||
.endpoint()
|
||||
.connect(addr, DATA_ALPN)
|
||||
.await
|
||||
.map_err(|err| TransportError::Unreachable(err.to_string()))?;
|
||||
let (mut send, mut recv) = conn
|
||||
.open_bi()
|
||||
.await
|
||||
.map_err(|err| TransportError::Unreachable(err.to_string()))?;
|
||||
|
||||
handshake::initiate(
|
||||
&conn,
|
||||
&mut send,
|
||||
&mut recv,
|
||||
self.adapter.endpoint_id(),
|
||||
&keys,
|
||||
&self.limits,
|
||||
)
|
||||
.await
|
||||
.map_err(|err| TransportError::Other(err.to_string()))?;
|
||||
|
||||
let open = DataOpen {
|
||||
protocol: protocol.to_string(),
|
||||
};
|
||||
write_frame(
|
||||
&mut send,
|
||||
&encode(&open).map_err(|err| TransportError::Other(err.to_string()))?,
|
||||
self.limits.max_frame_len,
|
||||
)
|
||||
.await
|
||||
.map_err(|err| TransportError::Other(err.to_string()))?;
|
||||
|
||||
let ack: DataOpenAck = decode(
|
||||
&read_frame(&mut recv, self.limits.max_frame_len)
|
||||
.await
|
||||
.map_err(|err| TransportError::Other(err.to_string()))?,
|
||||
)
|
||||
.map_err(|err| TransportError::Other(err.to_string()))?;
|
||||
|
||||
if !ack.accepted {
|
||||
conn.close(4u32.into(), b"declined");
|
||||
return Err(TransportError::Declined(protocol.to_string()));
|
||||
}
|
||||
|
||||
let link = IrohLink::new(network, peer, conn, ack.max_datagram as usize, send, recv);
|
||||
Ok(Arc::new(link) as SharedLink)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
//! The data plane transport boundary.
|
||||
//!
|
||||
//! This is the seam that keeps the control protocol and the data plane
|
||||
//! independent. An [`IpPlugin`] never learns how its packets are moved: it is
|
||||
//! handed a [`PacketLink`] to a peer and writes datagrams into it. Whether
|
||||
//! that link runs over iroh today, a raw UDP socket, or something else
|
||||
//! entirely tomorrow is the transport's business alone.
|
||||
//!
|
||||
//! [`IpPlugin`]: crate::dataplane::IpPlugin
|
||||
//!
|
||||
//! # Why the transport may use iroh
|
||||
//!
|
||||
//! The separation between control and data is **logical**, not a ban on
|
||||
//! sharing technology. Refusing to use iroh for data would throw away exactly
|
||||
//! what iroh is good at — hole punching a direct path between two peers behind
|
||||
//! NAT, with a relay as fallback — and force the data plane to reimplement it.
|
||||
//! So the default transport is [`iroh_link::IrohTransport`], which gives every
|
||||
//! plugin that connectivity for free.
|
||||
//!
|
||||
//! What the separation does buy is that the control protocol in
|
||||
//! [`crate::proto`] knows nothing about packets, and this module knows nothing
|
||||
//! about WireGuard. Either side can be replaced on its own.
|
||||
//!
|
||||
//! # Semantics
|
||||
//!
|
||||
//! A link is an **unreliable, unordered datagram** channel, because that is
|
||||
//! what a tunnelled UDP protocol needs: no retransmission, no head-of-line
|
||||
//! blocking, loss is normal rather than an error. It is authenticated and
|
||||
//! encrypted by the transport, and scoped to exactly one network, one peer and
|
||||
//! one plugin protocol.
|
||||
|
||||
pub mod iroh_link;
|
||||
|
||||
use bytes::Bytes;
|
||||
use iroh::EndpointId;
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
/// Why a data plane link failed.
|
||||
///
|
||||
/// None of these ever stop the control plane.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[non_exhaustive]
|
||||
pub enum TransportError {
|
||||
/// The peer is not reachable for the data plane right now.
|
||||
#[error("peer is unreachable: {0}")]
|
||||
Unreachable(String),
|
||||
/// The peer declined to open a channel for this protocol.
|
||||
#[error("peer declined a data channel for protocol `{0}`")]
|
||||
Declined(String),
|
||||
/// The link is closed.
|
||||
#[error("data link is closed")]
|
||||
Closed,
|
||||
/// A datagram was larger than the link can carry.
|
||||
#[error("datagram of {size} bytes exceeds the {limit} byte link limit")]
|
||||
TooLarge {
|
||||
/// Size that was attempted.
|
||||
size: usize,
|
||||
/// Largest datagram this link accepts.
|
||||
limit: usize,
|
||||
},
|
||||
/// Anything else.
|
||||
#[error("data transport error: {0}")]
|
||||
Other(String),
|
||||
}
|
||||
|
||||
/// An authenticated datagram channel to one peer, for one plugin protocol.
|
||||
///
|
||||
/// Dropping the link closes it.
|
||||
pub trait PacketLink: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// The network this link belongs to.
|
||||
fn network(&self) -> NetworkId;
|
||||
|
||||
/// The authenticated peer on the other end.
|
||||
fn peer(&self) -> EndpointId;
|
||||
|
||||
/// The largest datagram this link can carry, in bytes.
|
||||
///
|
||||
/// A plugin must size its own packets to fit, because there is no
|
||||
/// fragmentation here.
|
||||
fn max_datagram_size(&self) -> usize;
|
||||
|
||||
/// Sends one datagram.
|
||||
///
|
||||
/// Delivery is not guaranteed. Returning `Ok` means the datagram was
|
||||
/// handed to the transport, nothing more.
|
||||
fn send(&self, payload: Bytes) -> Result<(), TransportError>;
|
||||
|
||||
/// Receives the next datagram, or `None` once the link is finished.
|
||||
fn recv(&self) -> BoxFuture<'_, Option<Bytes>>;
|
||||
|
||||
/// Resolves once the link is closed, for whatever reason.
|
||||
fn closed(&self) -> BoxFuture<'_, ()>;
|
||||
|
||||
/// Whether the link is already closed.
|
||||
///
|
||||
/// Lets the owner notice a dead link and ask for a new one without
|
||||
/// keeping a task parked on [`PacketLink::closed`].
|
||||
fn is_closed(&self) -> bool;
|
||||
|
||||
/// A short description of the path in use, for diagnostics.
|
||||
///
|
||||
/// Reports what the transport actually knows. It must not invent a value.
|
||||
fn path_description(&self) -> String;
|
||||
}
|
||||
|
||||
/// A shared handle to a link.
|
||||
pub type SharedLink = std::sync::Arc<dyn PacketLink>;
|
||||
|
||||
/// An inbound link a peer opened towards us.
|
||||
#[derive(Debug)]
|
||||
pub struct InboundLink {
|
||||
/// The network it belongs to.
|
||||
pub network: NetworkId,
|
||||
/// The peer that opened it.
|
||||
pub peer: EndpointId,
|
||||
/// The plugin protocol it carries.
|
||||
pub protocol: String,
|
||||
/// The link itself.
|
||||
pub link: SharedLink,
|
||||
}
|
||||
|
||||
/// Opens and accepts data plane links.
|
||||
///
|
||||
/// The agent owns one of these and hands links to plugins; plugins never call
|
||||
/// it directly.
|
||||
pub trait PacketTransport: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// A short name used in diagnostics.
|
||||
fn name(&self) -> &str;
|
||||
|
||||
/// Lets the agent recover the concrete transport to drive its accept side.
|
||||
///
|
||||
/// Accepting is inherently transport-specific — it starts from whatever
|
||||
/// the transport's own listener produced — so it is not part of this
|
||||
/// trait's uniform interface.
|
||||
fn as_any(&self) -> &dyn std::any::Any;
|
||||
|
||||
/// Opens a link to `peer` in `network` for `protocol`.
|
||||
fn open<'a>(
|
||||
&'a self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
protocol: &'a str,
|
||||
) -> BoxFuture<'a, Result<SharedLink, TransportError>>;
|
||||
}
|
||||
@@ -4,11 +4,13 @@
|
||||
//! [`crate::dataplane::PluginCapability`]. The agent core never parses it —
|
||||
//! only this module does, and only after bounding every field.
|
||||
//!
|
||||
//! An iroh address is an address for iroh. It is **not** reused here: the
|
||||
//! plugin advertises its own reachability, gathered by itself, for its own
|
||||
//! listening port.
|
||||
//! The announcement is deliberately tiny: a participant says **who it is**,
|
||||
//! not **where it is**. Reachability is the data plane transport's job, and
|
||||
//! the transport already solves it — see
|
||||
//! [`crate::dataplane::transport`]. A plugin that also tried to advertise
|
||||
//! addresses would be reimplementing NAT traversal badly.
|
||||
|
||||
use std::net::{IpAddr, Ipv6Addr, SocketAddr};
|
||||
use std::net::Ipv6Addr;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
@@ -21,9 +23,6 @@ use super::overlay::overlay_address;
|
||||
/// Version of the announcement format.
|
||||
pub const ANNOUNCEMENT_VERSION: u16 = 1;
|
||||
|
||||
/// Largest number of advertised endpoints accepted from a peer.
|
||||
pub const MAX_ENDPOINTS: usize = 8;
|
||||
|
||||
/// What one participant advertises for the WireGuard data plane.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct WgAnnouncement {
|
||||
@@ -31,10 +30,6 @@ pub struct WgAnnouncement {
|
||||
pub version: u16,
|
||||
/// The peer's WireGuard public key. Its overlay address is derived from it.
|
||||
pub public_key: [u8; 32],
|
||||
/// The UDP port the peer's WireGuard interface listens on.
|
||||
pub listen_port: u16,
|
||||
/// Reachability the plugin gathered for itself. Advisory, may be empty.
|
||||
pub endpoints: Vec<SocketAddr>,
|
||||
/// The overlay address the peer believes it has.
|
||||
///
|
||||
/// Carried for diagnostics and cross-checking only. `AllowedIPs` are
|
||||
@@ -47,40 +42,16 @@ pub struct WgAnnouncement {
|
||||
pub struct ValidatedAnnouncement {
|
||||
/// The peer's WireGuard public key.
|
||||
pub public_key: WgPublicKey,
|
||||
/// The peer's listening port.
|
||||
pub listen_port: u16,
|
||||
/// Usable endpoints, filtered.
|
||||
pub endpoints: Vec<SocketAddr>,
|
||||
/// The overlay address derived locally for this key. Authoritative.
|
||||
pub overlay_address: Ipv6Addr,
|
||||
}
|
||||
|
||||
impl ValidatedAnnouncement {
|
||||
/// The endpoint to configure for this peer, if any is usable.
|
||||
///
|
||||
/// WireGuard takes a single endpoint. The first usable one wins, and
|
||||
/// WireGuard itself will re-learn the peer's real source address from the
|
||||
/// first authenticated packet it receives.
|
||||
pub fn preferred_endpoint(&self) -> Option<SocketAddr> {
|
||||
self.endpoints.first().copied()
|
||||
}
|
||||
}
|
||||
|
||||
impl WgAnnouncement {
|
||||
/// Builds this agent's announcement.
|
||||
pub fn new(
|
||||
network: NetworkId,
|
||||
public_key: &WgPublicKey,
|
||||
listen_port: u16,
|
||||
endpoints: Vec<SocketAddr>,
|
||||
) -> Self {
|
||||
let mut endpoints = endpoints;
|
||||
endpoints.truncate(MAX_ENDPOINTS);
|
||||
pub fn new(network: NetworkId, public_key: &WgPublicKey) -> Self {
|
||||
Self {
|
||||
version: ANNOUNCEMENT_VERSION,
|
||||
public_key: *public_key.as_bytes(),
|
||||
listen_port,
|
||||
endpoints,
|
||||
overlay_address: overlay_address(network, public_key),
|
||||
}
|
||||
}
|
||||
@@ -128,18 +99,6 @@ impl WgAnnouncement {
|
||||
"peer announced this agent's own WireGuard key".into(),
|
||||
));
|
||||
}
|
||||
if self.listen_port == 0 {
|
||||
return Err(PluginError::Rejected(
|
||||
"WireGuard listen port must not be zero".into(),
|
||||
));
|
||||
}
|
||||
if self.endpoints.len() > MAX_ENDPOINTS {
|
||||
return Err(PluginError::Rejected(format!(
|
||||
"announcement carries {} endpoints, at most {MAX_ENDPOINTS} are accepted",
|
||||
self.endpoints.len()
|
||||
)));
|
||||
}
|
||||
|
||||
// AllowedIPs are derived, never trusted. A mismatch means the peer is
|
||||
// confused or lying, and either way its own claim is discarded.
|
||||
let derived = overlay_address(network, &public_key);
|
||||
@@ -150,40 +109,13 @@ impl WgAnnouncement {
|
||||
));
|
||||
}
|
||||
|
||||
let endpoints: Vec<SocketAddr> = self
|
||||
.endpoints
|
||||
.into_iter()
|
||||
.filter(is_usable_endpoint)
|
||||
.collect();
|
||||
|
||||
Ok(ValidatedAnnouncement {
|
||||
public_key,
|
||||
listen_port: self.listen_port,
|
||||
endpoints,
|
||||
overlay_address: derived,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether an advertised endpoint is worth trying.
|
||||
///
|
||||
/// Nothing here is trusted; this only discards addresses that cannot be a
|
||||
/// peer, so the plugin does not waste a WireGuard endpoint slot on them.
|
||||
fn is_usable_endpoint(endpoint: &SocketAddr) -> bool {
|
||||
if endpoint.port() == 0 {
|
||||
return false;
|
||||
}
|
||||
match endpoint.ip() {
|
||||
IpAddr::V4(ip) => {
|
||||
!ip.is_unspecified()
|
||||
&& !ip.is_multicast()
|
||||
&& !ip.is_broadcast()
|
||||
&& !ip.is_documentation()
|
||||
}
|
||||
IpAddr::V6(ip) => !ip.is_unspecified() && !ip.is_multicast(),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
@@ -201,26 +133,31 @@ mod tests {
|
||||
.network_id()
|
||||
}
|
||||
|
||||
fn endpoint(text: &str) -> SocketAddr {
|
||||
text.parse().unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_well_formed_announcement_round_trips() {
|
||||
let id = network("round-trip");
|
||||
let peer = WgSecretKey::generate().public();
|
||||
let local = WgSecretKey::generate().public();
|
||||
|
||||
let announcement =
|
||||
WgAnnouncement::new(id, &peer, 51820, vec![endpoint("192.0.2.10:51820")]);
|
||||
let payload = announcement.encode().unwrap();
|
||||
let payload = WgAnnouncement::new(id, &peer).encode().unwrap();
|
||||
let validated = WgAnnouncement::decode_and_validate(&payload, id, &local).unwrap();
|
||||
|
||||
assert_eq!(validated.public_key, peer);
|
||||
assert_eq!(validated.listen_port, 51820);
|
||||
assert_eq!(validated.overlay_address, overlay_address(id, &peer));
|
||||
// 192.0.2.0/24 is documentation space and is filtered out.
|
||||
assert!(validated.endpoints.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_announcement_says_who_not_where() {
|
||||
// Reachability belongs to the transport. Nothing address-like is
|
||||
// carried here, so there is nothing for a peer to lie about.
|
||||
let id = network("identity-only");
|
||||
let peer = WgSecretKey::generate().public();
|
||||
let payload = WgAnnouncement::new(id, &peer).encode().unwrap();
|
||||
assert!(
|
||||
payload.len() < 80,
|
||||
"the announcement should stay tiny, got {} bytes",
|
||||
payload.len()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -231,7 +168,7 @@ mod tests {
|
||||
let local = WgSecretKey::generate().public();
|
||||
|
||||
// An attacker claims the victim's overlay address with its own key.
|
||||
let mut forged = WgAnnouncement::new(id, &attacker, 51820, Vec::new());
|
||||
let mut forged = WgAnnouncement::new(id, &attacker);
|
||||
forged.overlay_address = overlay_address(id, &victim);
|
||||
|
||||
let result = WgAnnouncement::decode_and_validate(&forged.encode().unwrap(), id, &local);
|
||||
@@ -248,9 +185,7 @@ mod tests {
|
||||
let peer = WgSecretKey::generate().public();
|
||||
let local = WgSecretKey::generate().public();
|
||||
|
||||
let payload = WgAnnouncement::new(there, &peer, 51820, Vec::new())
|
||||
.encode()
|
||||
.unwrap();
|
||||
let payload = WgAnnouncement::new(there, &peer).encode().unwrap();
|
||||
assert!(WgAnnouncement::decode_and_validate(&payload, here, &local).is_err());
|
||||
}
|
||||
|
||||
@@ -260,13 +195,12 @@ mod tests {
|
||||
let local = WgSecretKey::generate().public();
|
||||
let peer = WgSecretKey::generate().public();
|
||||
|
||||
// Not postcard at all.
|
||||
assert!(WgAnnouncement::decode_and_validate(&[0xff; 64], id, &local).is_err());
|
||||
assert!(WgAnnouncement::decode_and_validate(&[], id, &local).is_err());
|
||||
|
||||
let wrong_version = WgAnnouncement {
|
||||
version: ANNOUNCEMENT_VERSION + 1,
|
||||
..WgAnnouncement::new(id, &peer, 51820, Vec::new())
|
||||
..WgAnnouncement::new(id, &peer)
|
||||
};
|
||||
assert!(
|
||||
WgAnnouncement::decode_and_validate(&wrong_version.encode().unwrap(), id, &local)
|
||||
@@ -275,79 +209,26 @@ mod tests {
|
||||
|
||||
let zero_key = WgAnnouncement {
|
||||
public_key: [0u8; 32],
|
||||
..WgAnnouncement::new(id, &peer, 51820, Vec::new())
|
||||
..WgAnnouncement::new(id, &peer)
|
||||
};
|
||||
assert!(
|
||||
WgAnnouncement::decode_and_validate(&zero_key.encode().unwrap(), id, &local).is_err()
|
||||
);
|
||||
|
||||
let zero_port = WgAnnouncement::new(id, &peer, 0, Vec::new());
|
||||
assert!(
|
||||
WgAnnouncement::decode_and_validate(&zero_port.encode().unwrap(), id, &local).is_err()
|
||||
);
|
||||
|
||||
let too_many = WgAnnouncement {
|
||||
endpoints: (0..MAX_ENDPOINTS + 1)
|
||||
.map(|index| endpoint(&format!("10.0.0.1:{}", 1000 + index)))
|
||||
.collect(),
|
||||
..WgAnnouncement::new(id, &peer, 51820, Vec::new())
|
||||
};
|
||||
assert!(
|
||||
WgAnnouncement::decode_and_validate(&too_many.encode().unwrap(), id, &local).is_err()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_peer_cannot_claim_our_own_key() {
|
||||
let id = network("self");
|
||||
let local = WgSecretKey::generate().public();
|
||||
let payload = WgAnnouncement::new(id, &local, 51820, Vec::new())
|
||||
.encode()
|
||||
.unwrap();
|
||||
let payload = WgAnnouncement::new(id, &local).encode().unwrap();
|
||||
assert!(WgAnnouncement::decode_and_validate(&payload, id, &local).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unusable_endpoints_are_filtered_and_the_rest_kept() {
|
||||
let id = network("filter");
|
||||
let peer = WgSecretKey::generate().public();
|
||||
let local = WgSecretKey::generate().public();
|
||||
|
||||
let announcement = WgAnnouncement::new(
|
||||
id,
|
||||
&peer,
|
||||
51820,
|
||||
vec![
|
||||
endpoint("0.0.0.0:51820"),
|
||||
endpoint("224.0.0.1:51820"),
|
||||
endpoint("10.1.2.3:0"),
|
||||
endpoint("10.1.2.3:51820"),
|
||||
endpoint("[2001:db8::1]:51820"),
|
||||
],
|
||||
);
|
||||
let validated =
|
||||
WgAnnouncement::decode_and_validate(&announcement.encode().unwrap(), id, &local)
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
validated.endpoints,
|
||||
vec![endpoint("10.1.2.3:51820"), endpoint("[2001:db8::1]:51820")]
|
||||
);
|
||||
assert_eq!(
|
||||
validated.preferred_endpoint(),
|
||||
Some(endpoint("10.1.2.3:51820"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn announcements_stay_well_under_the_capability_payload_limit() {
|
||||
let id = network("size");
|
||||
let peer = WgSecretKey::generate().public();
|
||||
let endpoints = (0..MAX_ENDPOINTS)
|
||||
.map(|index| endpoint(&format!("[2001:db8::{index}]:51820")))
|
||||
.collect();
|
||||
let payload = WgAnnouncement::new(id, &peer, 51820, endpoints)
|
||||
.encode()
|
||||
.unwrap();
|
||||
let payload = WgAnnouncement::new(id, &peer).encode().unwrap();
|
||||
assert!(
|
||||
payload.len() < crate::config::Limits::default().max_capability_data_len,
|
||||
"announcement is {} bytes",
|
||||
|
||||
@@ -1,214 +0,0 @@
|
||||
//! How a desired configuration reaches the operating system.
|
||||
//!
|
||||
//! The plugin computes *what* the interface should look like; a backend makes
|
||||
//! it so. Splitting them keeps every interesting decision testable without
|
||||
//! root and without touching the host's network.
|
||||
//!
|
||||
//! A backend only ever touches the interface named in the configuration it is
|
||||
//! given. It never enumerates, adopts or modifies anything else.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use crate::dataplane::PluginError;
|
||||
|
||||
use super::config::{InterfaceConfig, InterfaceState};
|
||||
|
||||
/// Applies a desired WireGuard configuration.
|
||||
///
|
||||
/// Implementations are synchronous and may block; the plugin calls them from a
|
||||
/// blocking task, never from the async runtime.
|
||||
pub trait WireguardBackend: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// A short name used in diagnostics.
|
||||
fn name(&self) -> &str;
|
||||
|
||||
/// Reads back the current state of an interface.
|
||||
///
|
||||
/// `Ok(None)` means the interface does not exist, which is different from
|
||||
/// an error.
|
||||
fn inspect(&self, interface: &str) -> Result<Option<InterfaceState>, PluginError>;
|
||||
|
||||
/// Creates or updates the interface so that it matches `desired`.
|
||||
fn apply(&self, desired: &InterfaceConfig) -> Result<(), PluginError>;
|
||||
|
||||
/// Removes an interface this plugin created. Removing an absent interface
|
||||
/// succeeds.
|
||||
fn remove(&self, interface: &str) -> Result<(), PluginError>;
|
||||
}
|
||||
|
||||
/// What a [`RecordingBackend`] was asked to do.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum BackendCall {
|
||||
/// An interface was inspected.
|
||||
Inspect(String),
|
||||
/// An interface was created or updated.
|
||||
Apply(String),
|
||||
/// An interface was removed.
|
||||
Remove(String),
|
||||
}
|
||||
|
||||
/// An in-memory backend for tests and dry runs.
|
||||
///
|
||||
/// It behaves like a working WireGuard implementation without needing root or
|
||||
/// touching the host: applied configurations are remembered and can be read
|
||||
/// back, drift can be injected, and failures can be simulated.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct RecordingBackend {
|
||||
inner: Arc<Mutex<Recorded>>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct Recorded {
|
||||
interfaces: HashMap<String, InterfaceState>,
|
||||
calls: Vec<BackendCall>,
|
||||
fail_next_apply: Option<String>,
|
||||
}
|
||||
|
||||
impl RecordingBackend {
|
||||
/// Creates an empty backend.
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
fn with<T>(&self, f: impl FnOnce(&mut Recorded) -> T) -> T {
|
||||
let mut guard = match self.inner.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
f(&mut guard)
|
||||
}
|
||||
|
||||
/// The state currently configured for an interface, if any.
|
||||
pub fn state(&self, interface: &str) -> Option<InterfaceState> {
|
||||
self.with(|recorded| recorded.interfaces.get(interface).cloned())
|
||||
}
|
||||
|
||||
/// Every interface currently configured.
|
||||
pub fn interfaces(&self) -> Vec<String> {
|
||||
self.with(|recorded| {
|
||||
let mut names: Vec<String> = recorded.interfaces.keys().cloned().collect();
|
||||
names.sort();
|
||||
names
|
||||
})
|
||||
}
|
||||
|
||||
/// Everything the backend was asked to do, in order.
|
||||
pub fn calls(&self) -> Vec<BackendCall> {
|
||||
self.with(|recorded| recorded.calls.clone())
|
||||
}
|
||||
|
||||
/// How many times an interface was applied.
|
||||
pub fn apply_count(&self, interface: &str) -> usize {
|
||||
self.with(|recorded| {
|
||||
recorded
|
||||
.calls
|
||||
.iter()
|
||||
.filter(|call| matches!(call, BackendCall::Apply(name) if name == interface))
|
||||
.count()
|
||||
})
|
||||
}
|
||||
|
||||
/// Replaces an interface's state, simulating someone editing it by hand.
|
||||
pub fn inject_drift(&self, interface: &str, state: InterfaceState) {
|
||||
self.with(|recorded| {
|
||||
recorded.interfaces.insert(interface.to_string(), state);
|
||||
});
|
||||
}
|
||||
|
||||
/// Makes the next `apply` fail, simulating a data plane error.
|
||||
pub fn fail_next_apply(&self, reason: impl Into<String>) {
|
||||
let reason = reason.into();
|
||||
self.with(|recorded| recorded.fail_next_apply = Some(reason));
|
||||
}
|
||||
|
||||
/// Forgets the recorded call history, keeping configured interfaces.
|
||||
pub fn clear_calls(&self) {
|
||||
self.with(|recorded| recorded.calls.clear());
|
||||
}
|
||||
}
|
||||
|
||||
impl WireguardBackend for RecordingBackend {
|
||||
fn name(&self) -> &str {
|
||||
"recording"
|
||||
}
|
||||
|
||||
fn inspect(&self, interface: &str) -> Result<Option<InterfaceState>, PluginError> {
|
||||
self.with(|recorded| {
|
||||
recorded
|
||||
.calls
|
||||
.push(BackendCall::Inspect(interface.to_string()));
|
||||
Ok(recorded.interfaces.get(interface).cloned())
|
||||
})
|
||||
}
|
||||
|
||||
fn apply(&self, desired: &InterfaceConfig) -> Result<(), PluginError> {
|
||||
let state = desired.to_state();
|
||||
self.with(|recorded| {
|
||||
recorded
|
||||
.calls
|
||||
.push(BackendCall::Apply(desired.name.clone()));
|
||||
if let Some(reason) = recorded.fail_next_apply.take() {
|
||||
return Err(PluginError::Unavailable(reason));
|
||||
}
|
||||
recorded.interfaces.insert(desired.name.clone(), state);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
fn remove(&self, interface: &str) -> Result<(), PluginError> {
|
||||
self.with(|recorded| {
|
||||
recorded
|
||||
.calls
|
||||
.push(BackendCall::Remove(interface.to_string()));
|
||||
recorded.interfaces.remove(interface);
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use super::*;
|
||||
use crate::dataplane::wireguard::config::{InterfaceParams, build_interface};
|
||||
use crate::dataplane::wireguard::keys::WgSecretKey;
|
||||
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
|
||||
#[test]
|
||||
fn the_recording_backend_behaves_like_a_working_one() {
|
||||
let network = NetworkKeys::derive(
|
||||
&NetworkName::new("backend").unwrap(),
|
||||
&NetworkSecret::from_bytes(vec![2u8; 32]).unwrap(),
|
||||
)
|
||||
.network_id();
|
||||
let backend = RecordingBackend::new();
|
||||
let config = build_interface(
|
||||
InterfaceParams {
|
||||
network,
|
||||
name: "tsun0".into(),
|
||||
private_key: WgSecretKey::generate(),
|
||||
listen_port: 51820,
|
||||
mtu: None,
|
||||
keepalive: None,
|
||||
},
|
||||
[WgSecretKey::generate().public()],
|
||||
|_| None,
|
||||
);
|
||||
|
||||
assert_eq!(backend.inspect("tsun0").unwrap(), None);
|
||||
backend.apply(&config).unwrap();
|
||||
assert_eq!(backend.inspect("tsun0").unwrap(), Some(config.to_state()));
|
||||
assert_eq!(backend.interfaces(), vec!["tsun0".to_string()]);
|
||||
|
||||
backend.fail_next_apply("no permission");
|
||||
assert!(backend.apply(&config).is_err());
|
||||
backend.apply(&config).unwrap();
|
||||
|
||||
backend.remove("tsun0").unwrap();
|
||||
assert_eq!(backend.inspect("tsun0").unwrap(), None);
|
||||
// Removing something absent is not an error.
|
||||
backend.remove("tsun0").unwrap();
|
||||
assert_eq!(backend.apply_count("tsun0"), 3);
|
||||
}
|
||||
}
|
||||
@@ -9,19 +9,12 @@
|
||||
//! module re-serialises itself, so a hostile announcement cannot inject a
|
||||
//! configuration directive or a command argument.
|
||||
|
||||
use std::fmt::Write as _;
|
||||
use std::net::{IpAddr, Ipv6Addr, SocketAddr};
|
||||
|
||||
use sha2::{Digest, Sha256};
|
||||
use zeroize::Zeroizing;
|
||||
use std::net::{IpAddr, Ipv6Addr};
|
||||
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
use super::keys::{WgPublicKey, WgSecretKey};
|
||||
use super::overlay::{
|
||||
OVERLAY_HOST_PREFIX_LEN, OVERLAY_PREFIX_LEN, overlay_address, overlay_prefix,
|
||||
};
|
||||
use super::overlay::OVERLAY_HOST_PREFIX_LEN;
|
||||
|
||||
/// Longest interface name Linux accepts, excluding the terminating NUL.
|
||||
pub const MAX_INTERFACE_NAME_LEN: usize = 15;
|
||||
@@ -68,150 +61,6 @@ impl std::fmt::Display for Cidr {
|
||||
}
|
||||
}
|
||||
|
||||
/// One remote participant, as this agent will configure it.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct PeerConfig {
|
||||
/// The peer's WireGuard public key.
|
||||
pub public_key: WgPublicKey,
|
||||
/// Where to send the first packet, when the peer advertised somewhere.
|
||||
pub endpoint: Option<SocketAddr>,
|
||||
/// Prefixes accepted from and routed to this peer.
|
||||
///
|
||||
/// Always derived locally from the peer's key. Never taken from what the
|
||||
/// peer claims.
|
||||
pub allowed_ips: Vec<Cidr>,
|
||||
/// Keepalive interval, needed to hold a NAT mapping open.
|
||||
pub persistent_keepalive: Option<u16>,
|
||||
}
|
||||
|
||||
/// The complete local configuration for one network's overlay interface.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct InterfaceConfig {
|
||||
/// Interface name this plugin owns.
|
||||
pub name: String,
|
||||
/// This agent's private key for this network.
|
||||
pub private_key: WgSecretKey,
|
||||
/// UDP port the interface listens on.
|
||||
pub listen_port: u16,
|
||||
/// Addresses assigned to the interface.
|
||||
pub addresses: Vec<Cidr>,
|
||||
/// Interface MTU, when one is configured.
|
||||
pub mtu: Option<u32>,
|
||||
/// Remote participants.
|
||||
pub peers: Vec<PeerConfig>,
|
||||
}
|
||||
|
||||
/// Observable state of a configured interface, without any private key.
|
||||
///
|
||||
/// This is what desired and actual are compared on, so reconciliation never
|
||||
/// needs to move a private key around.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct InterfaceState {
|
||||
/// Interface name.
|
||||
pub name: String,
|
||||
/// Public key currently configured on the interface.
|
||||
pub public_key: WgPublicKey,
|
||||
/// Port currently listened on.
|
||||
pub listen_port: u16,
|
||||
/// Addresses currently assigned.
|
||||
pub addresses: Vec<Cidr>,
|
||||
/// Peers currently configured, sorted by public key.
|
||||
pub peers: Vec<PeerState>,
|
||||
}
|
||||
|
||||
/// Observable state of one configured peer.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct PeerState {
|
||||
/// The peer's public key.
|
||||
pub public_key: WgPublicKey,
|
||||
/// Endpoint currently configured.
|
||||
pub endpoint: Option<SocketAddr>,
|
||||
/// Allowed prefixes currently configured, sorted.
|
||||
pub allowed_ips: Vec<Cidr>,
|
||||
/// Keepalive currently configured.
|
||||
pub persistent_keepalive: Option<u16>,
|
||||
}
|
||||
|
||||
impl PeerState {
|
||||
/// Puts the state in its canonical, comparable form.
|
||||
pub fn normalised(mut self) -> Self {
|
||||
self.allowed_ips.sort();
|
||||
self.allowed_ips.dedup();
|
||||
// WireGuard reports a disabled keepalive as zero.
|
||||
if self.persistent_keepalive == Some(0) {
|
||||
self.persistent_keepalive = None;
|
||||
}
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
impl InterfaceState {
|
||||
/// Puts the state in its canonical, comparable form.
|
||||
pub fn normalised(mut self) -> Self {
|
||||
self.addresses.sort();
|
||||
self.addresses.dedup();
|
||||
self.peers = self.peers.into_iter().map(PeerState::normalised).collect();
|
||||
self.peers.sort_by_key(|peer| peer.public_key);
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
impl InterfaceConfig {
|
||||
/// The state this configuration is expected to produce.
|
||||
pub fn to_state(&self) -> InterfaceState {
|
||||
InterfaceState {
|
||||
name: self.name.clone(),
|
||||
public_key: self.private_key.public(),
|
||||
listen_port: self.listen_port,
|
||||
addresses: self.addresses.clone(),
|
||||
peers: self
|
||||
.peers
|
||||
.iter()
|
||||
.map(|peer| PeerState {
|
||||
public_key: peer.public_key,
|
||||
endpoint: peer.endpoint,
|
||||
allowed_ips: peer.allowed_ips.clone(),
|
||||
persistent_keepalive: peer.persistent_keepalive,
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
.normalised()
|
||||
}
|
||||
|
||||
/// Renders the configuration in the format `wg setconf` and `wg syncconf`
|
||||
/// read.
|
||||
///
|
||||
/// Only WireGuard's own directives appear here. Addresses and MTU are not
|
||||
/// part of this format — they belong to the network interface and are
|
||||
/// applied separately.
|
||||
///
|
||||
/// The result contains the private key and is zeroized on drop.
|
||||
pub fn render(&self) -> Zeroizing<String> {
|
||||
let mut out = String::with_capacity(256 + self.peers.len() * 192);
|
||||
out.push_str("[Interface]\n");
|
||||
let _ = writeln!(out, "PrivateKey = {}", self.private_key.encode().as_str());
|
||||
let _ = writeln!(out, "ListenPort = {}", self.listen_port);
|
||||
|
||||
let mut peers = self.peers.clone();
|
||||
peers.sort_by_key(|peer| peer.public_key);
|
||||
for peer in &peers {
|
||||
out.push_str("\n[Peer]\n");
|
||||
let _ = writeln!(out, "PublicKey = {}", peer.public_key.encode());
|
||||
let mut allowed = peer.allowed_ips.clone();
|
||||
allowed.sort();
|
||||
let rendered: Vec<String> = allowed.iter().map(Cidr::to_string).collect();
|
||||
let _ = writeln!(out, "AllowedIPs = {}", rendered.join(", "));
|
||||
if let Some(endpoint) = peer.endpoint {
|
||||
let _ = writeln!(out, "Endpoint = {endpoint}");
|
||||
}
|
||||
if let Some(keepalive) = peer.persistent_keepalive {
|
||||
let _ = writeln!(out, "PersistentKeepalive = {keepalive}");
|
||||
}
|
||||
}
|
||||
Zeroizing::new(out)
|
||||
}
|
||||
}
|
||||
|
||||
/// Derives this plugin's interface name for a network.
|
||||
///
|
||||
/// The name is stable across restarts and short enough for the platform. Two
|
||||
@@ -244,132 +93,6 @@ pub fn interface_name(prefix: &str, network: NetworkId) -> Result<String, Plugin
|
||||
Ok(format!("{prefix}{suffix}"))
|
||||
}
|
||||
|
||||
/// How the plugin chooses its UDP port.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum PortPolicy {
|
||||
/// Always this port. Only usable with a single network.
|
||||
Fixed(u16),
|
||||
/// A port derived from the network id inside `base .. base + span`.
|
||||
///
|
||||
/// Stable across restarts, so a peer's cached endpoint keeps working, and
|
||||
/// different networks on one host land on different ports.
|
||||
Derived {
|
||||
/// First port of the range.
|
||||
base: u16,
|
||||
/// How many ports the range covers.
|
||||
span: u16,
|
||||
},
|
||||
}
|
||||
|
||||
impl Default for PortPolicy {
|
||||
fn default() -> Self {
|
||||
Self::Derived {
|
||||
base: 51820,
|
||||
span: 64,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl PortPolicy {
|
||||
/// The port to listen on for `network`.
|
||||
pub fn port_for(&self, network: NetworkId) -> Result<u16, PluginError> {
|
||||
match *self {
|
||||
PortPolicy::Fixed(port) => {
|
||||
if port == 0 {
|
||||
return Err(PluginError::Other(
|
||||
"a fixed WireGuard port must not be zero".into(),
|
||||
));
|
||||
}
|
||||
Ok(port)
|
||||
}
|
||||
PortPolicy::Derived { base, span } => {
|
||||
if base == 0 || span == 0 {
|
||||
return Err(PluginError::Other(
|
||||
"a derived WireGuard port range must not be empty or start at zero".into(),
|
||||
));
|
||||
}
|
||||
let room = u16::MAX - base;
|
||||
if span - 1 > room {
|
||||
return Err(PluginError::Other(
|
||||
"the derived WireGuard port range runs past port 65535".into(),
|
||||
));
|
||||
}
|
||||
let hash = Sha256::digest(network.as_bytes());
|
||||
let offset = u16::from_be_bytes([hash[0], hash[1]]) % span;
|
||||
Ok(base + offset)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything about the local side of one network's interface.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct InterfaceParams {
|
||||
/// The network the interface serves.
|
||||
pub network: NetworkId,
|
||||
/// Interface name, derived by [`interface_name`].
|
||||
pub name: String,
|
||||
/// This agent's private key for this network.
|
||||
pub private_key: WgSecretKey,
|
||||
/// Port to listen on.
|
||||
pub listen_port: u16,
|
||||
/// Interface MTU.
|
||||
pub mtu: Option<u32>,
|
||||
/// Keepalive applied to every peer.
|
||||
pub keepalive: Option<u16>,
|
||||
}
|
||||
|
||||
/// Builds this agent's interface configuration for one network.
|
||||
///
|
||||
/// `peers` is the set of participants the control plane agreed on; `endpoints`
|
||||
/// supplies whatever reachability each of them advertised.
|
||||
pub fn build_interface(
|
||||
params: InterfaceParams,
|
||||
peers: impl IntoIterator<Item = WgPublicKey>,
|
||||
endpoints: impl Fn(&WgPublicKey) -> Option<SocketAddr>,
|
||||
) -> InterfaceConfig {
|
||||
let InterfaceParams {
|
||||
network,
|
||||
name,
|
||||
private_key,
|
||||
listen_port,
|
||||
mtu,
|
||||
keepalive,
|
||||
} = params;
|
||||
let local = overlay_address(network, &private_key.public());
|
||||
|
||||
let mut peer_configs: Vec<PeerConfig> = peers
|
||||
.into_iter()
|
||||
.filter(|key| !key.is_zero() && *key != private_key.public())
|
||||
.map(|key| PeerConfig {
|
||||
endpoint: endpoints(&key),
|
||||
// Derived locally. This is the whole reason a hostile member
|
||||
// cannot route another member's traffic to itself.
|
||||
allowed_ips: vec![Cidr::host(overlay_address(network, &key))],
|
||||
public_key: key,
|
||||
persistent_keepalive: keepalive,
|
||||
})
|
||||
.collect();
|
||||
peer_configs.sort_by_key(|peer| peer.public_key);
|
||||
peer_configs.dedup_by(|a, b| a.public_key == b.public_key);
|
||||
|
||||
InterfaceConfig {
|
||||
name,
|
||||
private_key,
|
||||
listen_port,
|
||||
addresses: vec![
|
||||
Cidr::host(local),
|
||||
// The shared /64 gives the interface a route for the overlay.
|
||||
Cidr {
|
||||
addr: IpAddr::V6(overlay_prefix(network)),
|
||||
prefix_len: OVERLAY_PREFIX_LEN,
|
||||
},
|
||||
],
|
||||
mtu,
|
||||
peers: peer_configs,
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
@@ -385,132 +108,6 @@ mod tests {
|
||||
.network_id()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_full_mesh_of_n_members_yields_n_minus_one_peers() {
|
||||
let id = network("mesh");
|
||||
let me = WgSecretKey::generate();
|
||||
let others: Vec<WgPublicKey> = (0..3).map(|_| WgSecretKey::generate().public()).collect();
|
||||
|
||||
let config = build_interface(
|
||||
InterfaceParams {
|
||||
network: id,
|
||||
name: "tsun0".into(),
|
||||
private_key: me.clone(),
|
||||
listen_port: 51820,
|
||||
mtu: None,
|
||||
keepalive: Some(25),
|
||||
},
|
||||
others.clone().into_iter().chain([me.public()]),
|
||||
|_| None,
|
||||
);
|
||||
|
||||
assert_eq!(config.peers.len(), 3, "our own key is never a peer");
|
||||
for peer in &config.peers {
|
||||
assert_eq!(peer.allowed_ips.len(), 1);
|
||||
assert_eq!(
|
||||
peer.allowed_ips[0],
|
||||
Cidr::host(overlay_address(id, &peer.public_key))
|
||||
);
|
||||
assert_eq!(peer.persistent_keepalive, Some(25));
|
||||
}
|
||||
assert!(
|
||||
config
|
||||
.addresses
|
||||
.contains(&Cidr::host(overlay_address(id, &me.public())))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn duplicate_and_zero_peer_keys_are_dropped() {
|
||||
let id = network("dupes");
|
||||
let me = WgSecretKey::generate();
|
||||
let other = WgSecretKey::generate().public();
|
||||
|
||||
let config = build_interface(
|
||||
InterfaceParams {
|
||||
network: id,
|
||||
name: "tsun0".into(),
|
||||
private_key: me,
|
||||
listen_port: 51820,
|
||||
mtu: None,
|
||||
keepalive: None,
|
||||
},
|
||||
[other, other, WgPublicKey::from_bytes([0u8; 32])],
|
||||
|_| None,
|
||||
);
|
||||
assert_eq!(config.peers.len(), 1);
|
||||
assert_eq!(config.peers[0].public_key, other);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_rendered_config_is_the_wg_setconf_format() {
|
||||
let id = network("render");
|
||||
let me = WgSecretKey::generate();
|
||||
let peer = WgSecretKey::generate().public();
|
||||
let config = build_interface(
|
||||
InterfaceParams {
|
||||
network: id,
|
||||
name: "tsun0".into(),
|
||||
private_key: me.clone(),
|
||||
listen_port: 51820,
|
||||
mtu: Some(1380),
|
||||
keepalive: Some(25),
|
||||
},
|
||||
[peer],
|
||||
|_| Some("10.0.0.7:51820".parse().unwrap()),
|
||||
);
|
||||
|
||||
let rendered = config.render();
|
||||
let text = rendered.as_str();
|
||||
assert!(text.starts_with("[Interface]\n"));
|
||||
assert!(text.contains(&format!("PrivateKey = {}", me.encode().as_str())));
|
||||
assert!(text.contains("ListenPort = 51820"));
|
||||
assert!(text.contains(&format!("PublicKey = {}", peer.encode())));
|
||||
assert!(text.contains("Endpoint = 10.0.0.7:51820"));
|
||||
assert!(text.contains("PersistentKeepalive = 25"));
|
||||
assert!(text.contains(&format!(
|
||||
"AllowedIPs = {}",
|
||||
Cidr::host(overlay_address(id, &peer))
|
||||
)));
|
||||
// Address and MTU belong to the interface, not to wg's own format.
|
||||
assert!(!text.contains("Address"));
|
||||
assert!(!text.contains("MTU"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rendering_is_deterministic_regardless_of_peer_order() {
|
||||
let id = network("stable");
|
||||
let me = WgSecretKey::generate();
|
||||
let keys: Vec<WgPublicKey> = (0..5).map(|_| WgSecretKey::generate().public()).collect();
|
||||
|
||||
let forward = build_interface(
|
||||
InterfaceParams {
|
||||
network: id,
|
||||
name: "tsun0".into(),
|
||||
private_key: me.clone(),
|
||||
listen_port: 51820,
|
||||
mtu: None,
|
||||
keepalive: None,
|
||||
},
|
||||
keys.clone(),
|
||||
|_| None,
|
||||
);
|
||||
let reversed = build_interface(
|
||||
InterfaceParams {
|
||||
network: id,
|
||||
name: "tsun0".into(),
|
||||
private_key: me,
|
||||
listen_port: 51820,
|
||||
mtu: None,
|
||||
keepalive: None,
|
||||
},
|
||||
keys.into_iter().rev().collect::<Vec<_>>(),
|
||||
|_| None,
|
||||
);
|
||||
assert_eq!(forward.render().as_str(), reversed.render().as_str());
|
||||
assert_eq!(forward.to_state(), reversed.to_state());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn interface_names_fit_the_platform_limit_and_are_stable() {
|
||||
let id = network("naming");
|
||||
@@ -532,56 +129,12 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn derived_ports_are_stable_and_inside_the_range() {
|
||||
let policy = PortPolicy::default();
|
||||
let id = network("ports");
|
||||
let port = policy.port_for(id).unwrap();
|
||||
assert_eq!(port, policy.port_for(id).unwrap());
|
||||
assert!(
|
||||
(51820..51884).contains(&port),
|
||||
"port {port} outside the range"
|
||||
);
|
||||
|
||||
assert_eq!(PortPolicy::Fixed(1234).port_for(id).unwrap(), 1234);
|
||||
assert!(PortPolicy::Fixed(0).port_for(id).is_err());
|
||||
assert!(
|
||||
PortPolicy::Derived {
|
||||
base: 65500,
|
||||
span: 1000
|
||||
}
|
||||
.port_for(id)
|
||||
.is_err()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn state_comparison_ignores_ordering_and_a_zero_keepalive() {
|
||||
let peer_a = WgSecretKey::generate().public();
|
||||
let peer_b = WgSecretKey::generate().public();
|
||||
let make = |order: [WgPublicKey; 2], keepalive: Option<u16>| {
|
||||
InterfaceState {
|
||||
name: "tsun0".into(),
|
||||
public_key: peer_a,
|
||||
listen_port: 51820,
|
||||
addresses: vec![
|
||||
Cidr::host("fd00::2".parse().unwrap()),
|
||||
Cidr::host("fd00::1".parse().unwrap()),
|
||||
],
|
||||
peers: order
|
||||
.into_iter()
|
||||
.map(|public_key| PeerState {
|
||||
public_key,
|
||||
endpoint: None,
|
||||
allowed_ips: Vec::new(),
|
||||
persistent_keepalive: keepalive,
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
.normalised()
|
||||
};
|
||||
fn a_cidr_rejects_an_impossible_prefix_length() {
|
||||
assert!(Cidr::new("10.0.0.1".parse().unwrap(), 33).is_err());
|
||||
assert!(Cidr::new("fd00::1".parse().unwrap(), 129).is_err());
|
||||
assert_eq!(
|
||||
make([peer_a, peer_b], None),
|
||||
make([peer_b, peer_a], Some(0))
|
||||
Cidr::host("fd00::1".parse().unwrap()).to_string(),
|
||||
"fd00::1/128"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,554 @@
|
||||
//! Userspace WireGuard.
|
||||
//!
|
||||
//! The protocol itself is [`boringtun::noise::Tunn`], which is pure state
|
||||
//! machine: no sockets, no TUN, no kernel module. That is what lets this work
|
||||
//! the same way on any platform and be tested end to end without privileges.
|
||||
//!
|
||||
//! ```text
|
||||
//! TunDevice (IP packets) PacketLink per peer
|
||||
//! | |
|
||||
//! v v
|
||||
//! destination address -> peer --Tunn.encapsulate--> ciphertext
|
||||
//! source address checked <--Tunn.decapsulate-- ciphertext
|
||||
//! ```
|
||||
//!
|
||||
//! # Address ownership is enforced here
|
||||
//!
|
||||
//! Kernel WireGuard enforces `AllowedIPs`; in userspace we must do it
|
||||
//! ourselves, and we do:
|
||||
//!
|
||||
//! * outbound, a packet is routed to the peer that **owns** its destination
|
||||
//! address, where ownership is the derivation in [`super::overlay`];
|
||||
//! * inbound, a decrypted packet is dropped unless its **source** is exactly
|
||||
//! the address derived for the peer whose tunnel decrypted it.
|
||||
//!
|
||||
//! So a participant cannot receive traffic addressed to someone else, and
|
||||
//! cannot forge traffic that appears to come from someone else, no matter
|
||||
//! what it announced.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::net::Ipv6Addr;
|
||||
use std::sync::atomic::{AtomicU32, AtomicU64, Ordering};
|
||||
use std::sync::{Arc, Mutex, RwLock};
|
||||
use std::time::Duration;
|
||||
|
||||
use boringtun::noise::{Tunn, TunnResult};
|
||||
use bytes::Bytes;
|
||||
use iroh::EndpointId;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::dataplane::PluginError;
|
||||
use crate::dataplane::transport::{SharedLink, TransportError};
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
use super::keys::{WgPublicKey, WgSecretKey};
|
||||
use super::overlay::overlay_address;
|
||||
use super::packet::IpHeader;
|
||||
use super::tun::TunDevice;
|
||||
|
||||
/// How often WireGuard's own timers are driven.
|
||||
///
|
||||
/// boringtun expects this at least every few hundred milliseconds; it is what
|
||||
/// drives handshakes, rekeying and keepalives.
|
||||
const TIMER_INTERVAL: Duration = Duration::from_millis(250);
|
||||
|
||||
/// Scratch space for one encapsulate or decapsulate call.
|
||||
const SCRATCH: usize = 4096;
|
||||
|
||||
/// Counters for one peer's tunnel.
|
||||
#[derive(Debug, Default)]
|
||||
struct PeerCounters {
|
||||
tx_packets: AtomicU64,
|
||||
tx_bytes: AtomicU64,
|
||||
rx_packets: AtomicU64,
|
||||
rx_bytes: AtomicU64,
|
||||
dropped_wrong_source: AtomicU64,
|
||||
dropped_oversize: AtomicU64,
|
||||
protocol_errors: AtomicU64,
|
||||
}
|
||||
|
||||
/// A snapshot of one peer's tunnel counters.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub struct PeerStats {
|
||||
/// Plaintext packets encrypted and sent to this peer.
|
||||
pub tx_packets: u64,
|
||||
/// Plaintext bytes encrypted and sent to this peer.
|
||||
pub tx_bytes: u64,
|
||||
/// Plaintext packets decrypted from this peer and given to the OS.
|
||||
pub rx_packets: u64,
|
||||
/// Plaintext bytes decrypted from this peer and given to the OS.
|
||||
pub rx_bytes: u64,
|
||||
/// Packets dropped because their source was not this peer's address.
|
||||
///
|
||||
/// A non-zero value means a peer tried to use an address it does not own.
|
||||
pub dropped_wrong_source: u64,
|
||||
/// Packets dropped because they did not fit in one link datagram.
|
||||
pub dropped_oversize: u64,
|
||||
/// WireGuard protocol errors, including packets that failed to decrypt.
|
||||
pub protocol_errors: u64,
|
||||
}
|
||||
|
||||
/// Whether a peer's tunnel has completed a handshake.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct PeerHealth {
|
||||
/// Time since the last successful WireGuard handshake.
|
||||
///
|
||||
/// `None` means no handshake has completed yet, so the tunnel is not
|
||||
/// carrying traffic. This is reported as it is, never guessed.
|
||||
pub since_handshake: Option<Duration>,
|
||||
}
|
||||
|
||||
impl PeerHealth {
|
||||
/// Whether the tunnel has ever completed a handshake.
|
||||
pub fn is_up(&self) -> bool {
|
||||
self.since_handshake.is_some()
|
||||
}
|
||||
}
|
||||
|
||||
struct Peer {
|
||||
endpoint_id: EndpointId,
|
||||
public_key: WgPublicKey,
|
||||
overlay: Ipv6Addr,
|
||||
tunn: Mutex<Tunn>,
|
||||
link: SharedLink,
|
||||
counters: Arc<PeerCounters>,
|
||||
task: Mutex<Option<JoinHandle<()>>>,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for Peer {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("Peer")
|
||||
.field("peer", &self.endpoint_id.fmt_short().to_string())
|
||||
.field("public_key", &self.public_key)
|
||||
.field("overlay", &self.overlay)
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Peer {
|
||||
fn drop(&mut self) {
|
||||
if let Ok(mut guard) = self.task.lock()
|
||||
&& let Some(task) = guard.take()
|
||||
{
|
||||
task.abort();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Peer {
|
||||
fn stats(&self) -> PeerStats {
|
||||
PeerStats {
|
||||
tx_packets: self.counters.tx_packets.load(Ordering::Relaxed),
|
||||
tx_bytes: self.counters.tx_bytes.load(Ordering::Relaxed),
|
||||
rx_packets: self.counters.rx_packets.load(Ordering::Relaxed),
|
||||
rx_bytes: self.counters.rx_bytes.load(Ordering::Relaxed),
|
||||
dropped_wrong_source: self.counters.dropped_wrong_source.load(Ordering::Relaxed),
|
||||
dropped_oversize: self.counters.dropped_oversize.load(Ordering::Relaxed),
|
||||
protocol_errors: self.counters.protocol_errors.load(Ordering::Relaxed),
|
||||
}
|
||||
}
|
||||
|
||||
fn health(&self) -> PeerHealth {
|
||||
let guard = match self.tunn.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
PeerHealth {
|
||||
since_handshake: guard.time_since_last_handshake(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// What a peer's tunnel looks like from outside.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PeerSummary {
|
||||
/// The peer's control plane identity.
|
||||
pub endpoint_id: EndpointId,
|
||||
/// The peer's WireGuard public key.
|
||||
pub public_key: WgPublicKey,
|
||||
/// The overlay address this agent derived for it.
|
||||
pub overlay_address: Ipv6Addr,
|
||||
/// Whether the tunnel has handshaken.
|
||||
pub health: PeerHealth,
|
||||
/// Traffic counters.
|
||||
pub stats: PeerStats,
|
||||
/// What the transport reports about the path carrying this tunnel.
|
||||
pub path: String,
|
||||
/// Largest datagram the link accepts.
|
||||
pub max_datagram: usize,
|
||||
}
|
||||
|
||||
struct Inner {
|
||||
network: NetworkId,
|
||||
private_key: WgSecretKey,
|
||||
tun: Arc<dyn TunDevice>,
|
||||
peers: RwLock<HashMap<WgPublicKey, Arc<Peer>>>,
|
||||
routes: RwLock<HashMap<Ipv6Addr, WgPublicKey>>,
|
||||
next_index: AtomicU32,
|
||||
unroutable: AtomicU64,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for Inner {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("Inner")
|
||||
.field("network", &self.network.fmt_short())
|
||||
.field("tun", &self.tun.name())
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
/// A userspace WireGuard interface for one network.
|
||||
#[derive(Debug)]
|
||||
pub struct WireguardDevice {
|
||||
inner: Arc<Inner>,
|
||||
tasks: Vec<JoinHandle<()>>,
|
||||
}
|
||||
|
||||
impl WireguardDevice {
|
||||
/// Starts a device on top of `tun`.
|
||||
pub fn start(network: NetworkId, private_key: WgSecretKey, tun: Arc<dyn TunDevice>) -> Self {
|
||||
let inner = Arc::new(Inner {
|
||||
network,
|
||||
private_key,
|
||||
tun,
|
||||
peers: RwLock::new(HashMap::new()),
|
||||
routes: RwLock::new(HashMap::new()),
|
||||
next_index: AtomicU32::new(1),
|
||||
unroutable: AtomicU64::new(0),
|
||||
});
|
||||
|
||||
let reader = tokio::spawn(read_from_os(Arc::clone(&inner)));
|
||||
let timers = tokio::spawn(drive_timers(Arc::clone(&inner)));
|
||||
|
||||
Self {
|
||||
inner,
|
||||
tasks: vec![reader, timers],
|
||||
}
|
||||
}
|
||||
|
||||
/// The interface name in use.
|
||||
pub fn interface(&self) -> &str {
|
||||
self.inner.tun.name()
|
||||
}
|
||||
|
||||
/// The interface MTU.
|
||||
pub fn mtu(&self) -> u32 {
|
||||
self.inner.tun.mtu()
|
||||
}
|
||||
|
||||
/// Adds or replaces a peer and starts its tunnel.
|
||||
pub fn add_peer(
|
||||
&self,
|
||||
endpoint_id: EndpointId,
|
||||
public_key: WgPublicKey,
|
||||
link: SharedLink,
|
||||
keepalive: Option<u16>,
|
||||
) -> Result<(), PluginError> {
|
||||
if public_key == self.inner.private_key.public() {
|
||||
return Err(PluginError::Rejected(
|
||||
"refusing to add ourselves as a WireGuard peer".into(),
|
||||
));
|
||||
}
|
||||
|
||||
let index = self.inner.next_index.fetch_add(1, Ordering::Relaxed);
|
||||
let tunn = Tunn::new(
|
||||
self.inner.private_key.to_static_secret(),
|
||||
public_key.into_x25519(),
|
||||
None,
|
||||
keepalive,
|
||||
index,
|
||||
None,
|
||||
);
|
||||
|
||||
let overlay = overlay_address(self.inner.network, &public_key);
|
||||
let peer = Arc::new(Peer {
|
||||
endpoint_id,
|
||||
public_key,
|
||||
overlay,
|
||||
tunn: Mutex::new(tunn),
|
||||
link,
|
||||
counters: Arc::new(PeerCounters::default()),
|
||||
task: Mutex::new(None),
|
||||
});
|
||||
|
||||
let task = tokio::spawn(read_from_link(Arc::clone(&self.inner), Arc::clone(&peer)));
|
||||
if let Ok(mut guard) = peer.task.lock() {
|
||||
*guard = Some(task);
|
||||
}
|
||||
|
||||
write_lock(&self.inner.peers).insert(public_key, Arc::clone(&peer));
|
||||
write_lock(&self.inner.routes).insert(overlay, public_key);
|
||||
|
||||
// Start the handshake now instead of waiting for the next timer tick,
|
||||
// so the tunnel is usable as soon as the link exists.
|
||||
kick_handshake(&peer);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Removes a peer and stops its tunnel.
|
||||
pub fn remove_peer(&self, public_key: &WgPublicKey) {
|
||||
if let Some(peer) = write_lock(&self.inner.peers).remove(public_key) {
|
||||
write_lock(&self.inner.routes).remove(&peer.overlay);
|
||||
}
|
||||
}
|
||||
|
||||
/// Removes every peer whose key is not in `keep`.
|
||||
pub fn retain_peers(&self, keep: &[WgPublicKey]) {
|
||||
let stale: Vec<WgPublicKey> = read_lock(&self.inner.peers)
|
||||
.keys()
|
||||
.filter(|key| !keep.contains(key))
|
||||
.copied()
|
||||
.collect();
|
||||
for key in stale {
|
||||
self.remove_peer(&key);
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a peer's tunnel exists.
|
||||
pub fn has_peer(&self, public_key: &WgPublicKey) -> bool {
|
||||
read_lock(&self.inner.peers).contains_key(public_key)
|
||||
}
|
||||
|
||||
/// A snapshot of every peer.
|
||||
pub fn peers(&self) -> Vec<PeerSummary> {
|
||||
let mut peers: Vec<PeerSummary> = read_lock(&self.inner.peers)
|
||||
.values()
|
||||
.map(|peer| PeerSummary {
|
||||
endpoint_id: peer.endpoint_id,
|
||||
public_key: peer.public_key,
|
||||
overlay_address: peer.overlay,
|
||||
health: peer.health(),
|
||||
stats: peer.stats(),
|
||||
path: peer.link.path_description(),
|
||||
max_datagram: peer.link.max_datagram_size(),
|
||||
})
|
||||
.collect();
|
||||
peers.sort_by_key(|peer| peer.public_key);
|
||||
peers
|
||||
}
|
||||
|
||||
/// Packets the operating system sent that no peer owns the address for.
|
||||
pub fn unroutable_packets(&self) -> u64 {
|
||||
self.inner.unroutable.load(Ordering::Relaxed)
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for WireguardDevice {
|
||||
fn drop(&mut self) {
|
||||
for task in &self.tasks {
|
||||
task.abort();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn read_lock<T>(lock: &RwLock<T>) -> std::sync::RwLockReadGuard<'_, T> {
|
||||
match lock.read() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
}
|
||||
}
|
||||
|
||||
fn write_lock<T>(lock: &RwLock<T>) -> std::sync::RwLockWriteGuard<'_, T> {
|
||||
match lock.write() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Asks boringtun for a handshake initiation and sends it.
|
||||
///
|
||||
/// Encapsulating an empty packet is how the protocol state machine is told
|
||||
/// "there is something to say"; with no session yet it answers with the
|
||||
/// handshake initiation.
|
||||
fn kick_handshake(peer: &Peer) {
|
||||
let mut scratch = vec![0u8; SCRATCH];
|
||||
let len = {
|
||||
let mut tunn = match peer.tunn.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
match tunn.encapsulate(&[], &mut scratch) {
|
||||
TunnResult::WriteToNetwork(out) => Some(out.len()),
|
||||
_ => None,
|
||||
}
|
||||
};
|
||||
if let Some(len) = len {
|
||||
send_to_peer(peer, &scratch[..len]);
|
||||
}
|
||||
}
|
||||
|
||||
/// Sends whatever boringtun produced, without holding the tunnel lock.
|
||||
fn send_to_peer(peer: &Peer, payload: &[u8]) {
|
||||
match peer.link.send(Bytes::copy_from_slice(payload)) {
|
||||
Ok(()) => {}
|
||||
Err(TransportError::TooLarge { .. }) => {
|
||||
peer.counters
|
||||
.dropped_oversize
|
||||
.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
Err(TransportError::Closed) => {}
|
||||
Err(err) => {
|
||||
tracing::trace!(%err, "dropping a WireGuard packet the link refused");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Operating system -> peer.
|
||||
async fn read_from_os(inner: Arc<Inner>) {
|
||||
loop {
|
||||
let Some(packet) = inner.tun.recv().await else {
|
||||
return;
|
||||
};
|
||||
|
||||
// Route by destination: only the peer that owns that overlay address
|
||||
// may receive it.
|
||||
let Some(destination) = IpHeader::parse(&packet).and_then(|header| header.v6_destination())
|
||||
else {
|
||||
inner.unroutable.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
};
|
||||
let target = read_lock(&inner.routes).get(&destination).copied();
|
||||
let Some(target) = target else {
|
||||
inner.unroutable.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
};
|
||||
let peer = read_lock(&inner.peers).get(&target).cloned();
|
||||
let Some(peer) = peer else {
|
||||
inner.unroutable.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
};
|
||||
|
||||
let mut scratch = vec![0u8; SCRATCH];
|
||||
let outcome = {
|
||||
let mut tunn = match peer.tunn.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
match tunn.encapsulate(&packet, &mut scratch) {
|
||||
TunnResult::WriteToNetwork(out) => Some(out.len()),
|
||||
TunnResult::Done => None,
|
||||
TunnResult::Err(err) => {
|
||||
tracing::trace!(?err, "wireguard encapsulation failed");
|
||||
peer.counters
|
||||
.protocol_errors
|
||||
.fetch_add(1, Ordering::Relaxed);
|
||||
None
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
};
|
||||
|
||||
if let Some(len) = outcome {
|
||||
send_to_peer(&peer, &scratch[..len]);
|
||||
peer.counters.tx_packets.fetch_add(1, Ordering::Relaxed);
|
||||
peer.counters
|
||||
.tx_bytes
|
||||
.fetch_add(packet.len() as u64, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Peer -> operating system.
|
||||
async fn read_from_link(inner: Arc<Inner>, peer: Arc<Peer>) {
|
||||
loop {
|
||||
let Some(datagram) = peer.link.recv().await else {
|
||||
return;
|
||||
};
|
||||
|
||||
let mut scratch = vec![0u8; SCRATCH];
|
||||
// boringtun may need several passes: a handshake reply first, then
|
||||
// any packets that were queued while the session was coming up.
|
||||
let mut input: Option<&[u8]> = Some(&datagram);
|
||||
loop {
|
||||
let outcome = {
|
||||
let mut tunn = match peer.tunn.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
match tunn.decapsulate(None, input.unwrap_or(&[]), &mut scratch) {
|
||||
TunnResult::WriteToNetwork(out) => Outcome::ToNetwork(out.len()),
|
||||
TunnResult::WriteToTunnelV6(out, source) => {
|
||||
Outcome::ToTunnel(out.len(), Some(source))
|
||||
}
|
||||
TunnResult::WriteToTunnelV4(out, _) => Outcome::ToTunnel(out.len(), None),
|
||||
TunnResult::Done => Outcome::Done,
|
||||
TunnResult::Err(err) => {
|
||||
tracing::trace!(?err, "wireguard decapsulation failed");
|
||||
Outcome::Failed
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
match outcome {
|
||||
Outcome::ToNetwork(len) => {
|
||||
send_to_peer(&peer, &scratch[..len]);
|
||||
// Keep draining with an empty datagram, as boringtun asks.
|
||||
input = None;
|
||||
continue;
|
||||
}
|
||||
Outcome::ToTunnel(len, source) => {
|
||||
let payload = Bytes::copy_from_slice(&scratch[..len]);
|
||||
// Enforce address ownership: a peer may only send from the
|
||||
// address derived for its own key.
|
||||
if source != Some(peer.overlay) {
|
||||
peer.counters
|
||||
.dropped_wrong_source
|
||||
.fetch_add(1, Ordering::Relaxed);
|
||||
break;
|
||||
}
|
||||
if inner.tun.send(payload).await.is_ok() {
|
||||
peer.counters.rx_packets.fetch_add(1, Ordering::Relaxed);
|
||||
peer.counters
|
||||
.rx_bytes
|
||||
.fetch_add(len as u64, Ordering::Relaxed);
|
||||
}
|
||||
break;
|
||||
}
|
||||
Outcome::Failed => {
|
||||
peer.counters
|
||||
.protocol_errors
|
||||
.fetch_add(1, Ordering::Relaxed);
|
||||
break;
|
||||
}
|
||||
Outcome::Done => break,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
enum Outcome {
|
||||
ToNetwork(usize),
|
||||
ToTunnel(usize, Option<Ipv6Addr>),
|
||||
Done,
|
||||
Failed,
|
||||
}
|
||||
|
||||
/// Drives WireGuard's handshake, rekey and keepalive timers.
|
||||
async fn drive_timers(inner: Arc<Inner>) {
|
||||
let mut ticker = tokio::time::interval(TIMER_INTERVAL);
|
||||
ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
|
||||
loop {
|
||||
ticker.tick().await;
|
||||
let peers: Vec<Arc<Peer>> = read_lock(&inner.peers).values().cloned().collect();
|
||||
for peer in peers {
|
||||
let mut scratch = vec![0u8; SCRATCH];
|
||||
let len = {
|
||||
let mut tunn = match peer.tunn.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
match tunn.update_timers(&mut scratch) {
|
||||
TunnResult::WriteToNetwork(out) => Some(out.len()),
|
||||
TunnResult::Err(err) => {
|
||||
tracing::trace!(?err, "wireguard timer produced an error");
|
||||
None
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
};
|
||||
if let Some(len) = len {
|
||||
send_to_peer(&peer, &scratch[..len]);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -7,6 +7,7 @@
|
||||
//! Keys are X25519, encoded the way WireGuard encodes them: standard base64
|
||||
//! with padding, 44 characters.
|
||||
|
||||
use boringtun::x25519;
|
||||
use data_encoding::BASE64;
|
||||
use zeroize::{Zeroize, Zeroizing};
|
||||
|
||||
@@ -46,6 +47,11 @@ impl WgPublicKey {
|
||||
&self.0
|
||||
}
|
||||
|
||||
/// The key in the form the WireGuard implementation expects.
|
||||
pub(crate) fn into_x25519(self) -> x25519::PublicKey {
|
||||
x25519::PublicKey::from(self.0)
|
||||
}
|
||||
|
||||
/// Whether this is the all-zero key, which is never a valid peer.
|
||||
pub fn is_zero(&self) -> bool {
|
||||
self.0 == [0u8; KEY_LEN]
|
||||
@@ -121,9 +127,12 @@ impl WgSecretKey {
|
||||
|
||||
/// The matching public key.
|
||||
pub fn public(&self) -> WgPublicKey {
|
||||
let secret = x25519_dalek::StaticSecret::from(*self.0);
|
||||
let public = x25519_dalek::PublicKey::from(&secret);
|
||||
WgPublicKey(public.to_bytes())
|
||||
WgPublicKey(x25519::PublicKey::from(&self.to_static_secret()).to_bytes())
|
||||
}
|
||||
|
||||
/// The key in the form the WireGuard implementation expects.
|
||||
pub(crate) fn to_static_secret(&self) -> x25519::StaticSecret {
|
||||
x25519::StaticSecret::from(*self.0)
|
||||
}
|
||||
|
||||
/// The base64 form, for the WireGuard configuration. Zeroized on drop.
|
||||
|
||||
@@ -6,15 +6,18 @@
|
||||
//!
|
||||
//! The two planes stay separate:
|
||||
//!
|
||||
//! * **No user IP traffic goes through iroh.** iroh carries this plugin's
|
||||
//! announcements and nothing else; the packets themselves travel over
|
||||
//! WireGuard's own UDP sockets.
|
||||
//! * **An iroh address is not a WireGuard address.** The plugin gathers its
|
||||
//! own reachability and advertises that.
|
||||
//! * **The plugin knows nothing about reachability.** It is handed a
|
||||
//! [`PacketLink`](crate::dataplane::transport::PacketLink) per peer and
|
||||
//! bridges the kernel WireGuard device onto it. Hole punching and relaying
|
||||
//! belong to the transport.
|
||||
//! * **The announcement says who, not where.** It carries a public key, so
|
||||
//! there is no address for a peer to lie about.
|
||||
//! * **The core never parses these announcements.** It moves a bounded opaque
|
||||
//! blob; only [`announcement`] interprets it.
|
||||
//! * **Keys are separate.** The plugin has its own key per network, in its own
|
||||
//! store, unrelated to the iroh device key and to the network secret.
|
||||
//! * **WireGuard's own crypto is untouched.** The bridge is a pipe; the
|
||||
//! handshake and encryption run end to end between the two kernels.
|
||||
//!
|
||||
//! # How a mesh forms
|
||||
//!
|
||||
@@ -24,34 +27,41 @@
|
||||
//! peer's `AllowedIPs` itself instead of believing what the peer claims — a
|
||||
//! member cannot route another member's traffic to itself.
|
||||
//!
|
||||
//! Each agent then builds its own local configuration with one peer entry per
|
||||
//! other participant ([`config`]) and hands it to a [`backend`]. The
|
||||
//! [`backend::RecordingBackend`] applies it in memory, which is what the test
|
||||
//! suite uses; [`wgtool::WgToolBackend`] drives the real `wg` and `ip` tools
|
||||
//! and needs Linux with `CAP_NET_ADMIN`.
|
||||
//! WireGuard itself is [`boringtun`]'s protocol state machine, running in this
|
||||
//! process: no kernel module, no `wg` tool, the same code on every platform.
|
||||
//! [`device::WireguardDevice`] drives one tunnel per peer and routes packets
|
||||
//! between them and a [`tun::TunDevice`].
|
||||
//!
|
||||
//! The only part that needs privileges is the packet interface. With
|
||||
//! [`tun::MemoryTunFactory`] the whole data plane — handshake, encryption,
|
||||
//! routing, address ownership — runs and is tested with no privileges at all;
|
||||
//! `SystemTunFactory` swaps in a real interface when you want traffic to
|
||||
//! reach the operating system.
|
||||
//!
|
||||
//! See `docs/wireguard.md` for the full picture.
|
||||
|
||||
pub mod announcement;
|
||||
pub mod backend;
|
||||
pub mod config;
|
||||
pub mod device;
|
||||
pub mod keys;
|
||||
pub mod overlay;
|
||||
pub mod packet;
|
||||
pub mod plugin;
|
||||
pub mod store;
|
||||
pub mod wgtool;
|
||||
pub mod tun;
|
||||
|
||||
pub use announcement::{ValidatedAnnouncement, WgAnnouncement};
|
||||
pub use backend::{BackendCall, RecordingBackend, WireguardBackend};
|
||||
pub use config::{
|
||||
Cidr, InterfaceConfig, InterfaceParams, InterfaceState, PeerConfig, PeerState, PortPolicy,
|
||||
build_interface, interface_name,
|
||||
};
|
||||
pub use config::{Cidr, DEFAULT_INTERFACE_PREFIX, MAX_INTERFACE_NAME_LEN, interface_name};
|
||||
pub use device::{PeerHealth, PeerStats, PeerSummary, WireguardDevice};
|
||||
pub use keys::{WgPublicKey, WgSecretKey};
|
||||
pub use overlay::{overlay_address, overlay_prefix};
|
||||
pub use overlay::{OVERLAY_PREFIX_LEN, overlay_address, overlay_prefix};
|
||||
pub use packet::IpHeader;
|
||||
pub use plugin::{
|
||||
AdvertisePolicy, NetworkOverview, PeerOverview, WIREGUARD_PROTOCOL, WireguardConfig,
|
||||
DEFAULT_MTU, NetworkOverview, PeerOverview, WIREGUARD_PROTOCOL, WireguardConfig,
|
||||
WireguardPlugin,
|
||||
};
|
||||
pub use store::WgKeyStore;
|
||||
pub use wgtool::{WgToolBackend, plan_apply, plan_remove};
|
||||
pub use tun::{MemoryTun, MemoryTunFactory, TunDevice, TunFactory, TunRequest};
|
||||
|
||||
#[cfg(feature = "tun-device")]
|
||||
pub use tun::SystemTunFactory;
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
//! The little bit of IP parsing the data plane needs.
|
||||
//!
|
||||
//! Two questions only: which peer should carry this packet, and did the packet
|
||||
//! that came back really come from that peer? Everything is bounds checked and
|
||||
//! nothing here can panic on a hostile packet.
|
||||
|
||||
use std::net::{Ipv4Addr, Ipv6Addr};
|
||||
|
||||
/// The addresses of an IP packet, as far as routing cares.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum IpHeader {
|
||||
/// An IPv4 packet.
|
||||
V4 {
|
||||
/// Source address.
|
||||
source: Ipv4Addr,
|
||||
/// Destination address.
|
||||
destination: Ipv4Addr,
|
||||
},
|
||||
/// An IPv6 packet.
|
||||
V6 {
|
||||
/// Source address.
|
||||
source: Ipv6Addr,
|
||||
/// Destination address.
|
||||
destination: Ipv6Addr,
|
||||
},
|
||||
}
|
||||
|
||||
impl IpHeader {
|
||||
/// Reads the addresses out of a packet, or `None` if it is not one.
|
||||
pub fn parse(packet: &[u8]) -> Option<Self> {
|
||||
let version = packet.first()? >> 4;
|
||||
match version {
|
||||
4 => {
|
||||
let source: [u8; 4] = packet.get(12..16)?.try_into().ok()?;
|
||||
let destination: [u8; 4] = packet.get(16..20)?.try_into().ok()?;
|
||||
Some(IpHeader::V4 {
|
||||
source: Ipv4Addr::from(source),
|
||||
destination: Ipv4Addr::from(destination),
|
||||
})
|
||||
}
|
||||
6 => {
|
||||
let source: [u8; 16] = packet.get(8..24)?.try_into().ok()?;
|
||||
let destination: [u8; 16] = packet.get(24..40)?.try_into().ok()?;
|
||||
Some(IpHeader::V6 {
|
||||
source: Ipv6Addr::from(source),
|
||||
destination: Ipv6Addr::from(destination),
|
||||
})
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// The destination, when the packet is IPv6.
|
||||
pub fn v6_destination(&self) -> Option<Ipv6Addr> {
|
||||
match self {
|
||||
IpHeader::V6 { destination, .. } => Some(*destination),
|
||||
IpHeader::V4 { .. } => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// The source, when the packet is IPv6.
|
||||
pub fn v6_source(&self) -> Option<Ipv6Addr> {
|
||||
match self {
|
||||
IpHeader::V6 { source, .. } => Some(*source),
|
||||
IpHeader::V4 { .. } => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use super::*;
|
||||
|
||||
fn ipv6_packet(source: Ipv6Addr, destination: Ipv6Addr) -> Vec<u8> {
|
||||
let mut packet = vec![0u8; 48];
|
||||
packet[0] = 6 << 4;
|
||||
packet[8..24].copy_from_slice(&source.octets());
|
||||
packet[24..40].copy_from_slice(&destination.octets());
|
||||
packet
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ipv6_addresses_are_read_correctly() {
|
||||
let source: Ipv6Addr = "fd00::1".parse().unwrap();
|
||||
let destination: Ipv6Addr = "fd00::2".parse().unwrap();
|
||||
let header = IpHeader::parse(&ipv6_packet(source, destination)).unwrap();
|
||||
assert_eq!(header.v6_source(), Some(source));
|
||||
assert_eq!(header.v6_destination(), Some(destination));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ipv4_addresses_are_read_correctly() {
|
||||
let mut packet = vec![0u8; 20];
|
||||
packet[0] = 4 << 4;
|
||||
packet[12..16].copy_from_slice(&[10, 0, 0, 1]);
|
||||
packet[16..20].copy_from_slice(&[10, 0, 0, 2]);
|
||||
let header = IpHeader::parse(&packet).unwrap();
|
||||
assert_eq!(
|
||||
header,
|
||||
IpHeader::V4 {
|
||||
source: Ipv4Addr::new(10, 0, 0, 1),
|
||||
destination: Ipv4Addr::new(10, 0, 0, 2),
|
||||
}
|
||||
);
|
||||
// The overlay is IPv6, so the v6 accessors correctly report nothing.
|
||||
assert_eq!(header.v6_destination(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn truncated_and_nonsense_packets_are_rejected_without_panicking() {
|
||||
assert!(IpHeader::parse(&[]).is_none());
|
||||
assert!(IpHeader::parse(&[0x60]).is_none());
|
||||
assert!(IpHeader::parse(&[0x40; 19]).is_none(), "short IPv4");
|
||||
assert!(IpHeader::parse(&[0x60; 39]).is_none(), "short IPv6");
|
||||
assert!(IpHeader::parse(&[0x00; 64]).is_none(), "version 0");
|
||||
assert!(IpHeader::parse(&[0xf0; 64]).is_none(), "version 15");
|
||||
// Every possible first byte is safe to feed in.
|
||||
for byte in 0..=u8::MAX {
|
||||
let _ = IpHeader::parse(&[byte; 64]);
|
||||
let _ = IpHeader::parse(&[byte]);
|
||||
}
|
||||
}
|
||||
}
|
||||
+237
-238
@@ -1,26 +1,31 @@
|
||||
//! The WireGuard IP plugin.
|
||||
//!
|
||||
//! Each agent builds its **own** local configuration from the set of
|
||||
//! participants the control plane agreed on. For a full mesh of `N` members
|
||||
//! that is `N - 1` peers locally. Nobody is handed a configuration by anybody
|
||||
//! else, and no participant is authoritative.
|
||||
//! Each agent builds its own view of the overlay from the set of participants
|
||||
//! the control plane agreed on. For a full mesh of `N` members that is `N - 1`
|
||||
//! tunnels locally. Nobody is handed a configuration by anybody else, and no
|
||||
//! participant is authoritative.
|
||||
//!
|
||||
//! What the plugin owns and what it never touches:
|
||||
//! # What this plugin does and does not know
|
||||
//!
|
||||
//! * it owns one WireGuard key per network, in its own store;
|
||||
//! * it owns one interface per network, named deterministically from the
|
||||
//! network id and its configured prefix;
|
||||
//! * it never enumerates, adopts or edits an interface it did not create, and
|
||||
//! it never changes routing, DNS or firewall settings.
|
||||
//! * It does **not** know where a peer is. It is handed a
|
||||
//! [`PacketLink`](crate::dataplane::transport::PacketLink) per peer and runs
|
||||
//! a WireGuard tunnel over it. Reachability, hole punching and relaying are
|
||||
//! the transport's problem.
|
||||
//! * It owns one WireGuard key per network, in its own store, unrelated to the
|
||||
//! iroh device key and to the network secret.
|
||||
//! * It owns one packet interface per network, named deterministically.
|
||||
//! * It never touches an interface it did not create, and never changes
|
||||
//! routing, DNS or firewall settings beyond its own device.
|
||||
//!
|
||||
//! Reconciliation runs on every change and on a timer, so a configuration
|
||||
//! edited by hand is put back the way it should be.
|
||||
//! WireGuard runs in userspace via [`boringtun`], so there is no kernel module
|
||||
//! and no `wg` tool to depend on. The only privileged step is creating the
|
||||
//! packet interface, and even that is behind [`TunFactory`] so the whole data
|
||||
//! plane can run unprivileged in tests.
|
||||
//!
|
||||
//! A failure here is reported and retried. It never stops the control plane:
|
||||
//! the agent keeps receiving state and stays manageable.
|
||||
//! A failure here is reported and retried. It never stops the control plane.
|
||||
|
||||
use std::collections::{BTreeSet, HashMap};
|
||||
use std::net::{IpAddr, SocketAddr};
|
||||
use std::net::IpAddr;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
use std::time::Duration;
|
||||
@@ -30,39 +35,28 @@ use tokio::sync::{mpsc, oneshot};
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::transport::SharedLink;
|
||||
use crate::dataplane::{IpPlugin, PluginCapability, PluginContext, PluginError};
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
use super::announcement::{ValidatedAnnouncement, WgAnnouncement};
|
||||
use super::backend::WireguardBackend;
|
||||
use super::config::{
|
||||
DEFAULT_INTERFACE_PREFIX, InterfaceConfig, InterfaceParams, PortPolicy, build_interface,
|
||||
interface_name,
|
||||
};
|
||||
use super::config::{DEFAULT_INTERFACE_PREFIX, interface_name};
|
||||
use super::device::{PeerSummary, WireguardDevice};
|
||||
use super::keys::{WgPublicKey, WgSecretKey};
|
||||
use super::overlay::{overlay_address, overlay_prefix};
|
||||
use super::overlay::{OVERLAY_PREFIX_LEN, overlay_address, overlay_prefix};
|
||||
use super::store::WgKeyStore;
|
||||
use super::tun::{TunFactory, TunRequest};
|
||||
|
||||
/// The protocol identifier this plugin announces.
|
||||
pub const WIREGUARD_PROTOCOL: &str = "wireguard";
|
||||
|
||||
/// How the plugin advertises its own reachability.
|
||||
/// Default interface MTU.
|
||||
///
|
||||
/// An iroh address is an address for iroh. WireGuard needs its own, so the
|
||||
/// plugin gathers its own rather than reusing the control plane's.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum AdvertisePolicy {
|
||||
/// Advertise nothing.
|
||||
///
|
||||
/// Peers can still reach this agent if they are reachable themselves:
|
||||
/// WireGuard learns a peer's real source address from the first
|
||||
/// authenticated packet it receives.
|
||||
None,
|
||||
/// Advertise exactly these addresses, combined with the listening port.
|
||||
Explicit(Vec<IpAddr>),
|
||||
/// Advertise the host's own non-loopback addresses.
|
||||
LocalInterfaces,
|
||||
}
|
||||
/// Every packet rides in one transport datagram, and WireGuard adds 32 bytes.
|
||||
/// A QUIC datagram on a relayed path can be as small as roughly 1160 bytes, so
|
||||
/// 1100 leaves headroom instead of relying on the best case. Packets that do
|
||||
/// not fit are dropped and counted, never truncated.
|
||||
pub const DEFAULT_MTU: u32 = 1100;
|
||||
|
||||
/// Configuration of the WireGuard plugin.
|
||||
#[derive(Debug, Clone)]
|
||||
@@ -74,18 +68,14 @@ pub struct WireguardConfig {
|
||||
/// Two agents on one host in the same network need different prefixes,
|
||||
/// because the rest of the name is derived from the network id.
|
||||
pub interface_prefix: String,
|
||||
/// How the listening port is chosen.
|
||||
pub ports: PortPolicy,
|
||||
/// What reachability to advertise.
|
||||
pub advertise: AdvertisePolicy,
|
||||
/// Keepalive interval, which holds a NAT mapping open.
|
||||
/// WireGuard keepalive, which keeps tunnels and their links warm.
|
||||
pub keepalive: Option<u16>,
|
||||
/// Interface MTU.
|
||||
pub mtu: Option<u32>,
|
||||
/// Interface MTU. See [`DEFAULT_MTU`].
|
||||
pub mtu: u32,
|
||||
/// How long to coalesce changes before reconciling.
|
||||
pub reconcile_debounce: Duration,
|
||||
/// How often to reconcile even when nothing changed, which is what
|
||||
/// corrects a configuration someone edited by hand.
|
||||
/// How often to reconcile anyway, which is also when a packet interface
|
||||
/// that could not be created before is retried.
|
||||
pub reconcile_interval: Duration,
|
||||
}
|
||||
|
||||
@@ -95,12 +85,10 @@ impl WireguardConfig {
|
||||
Self {
|
||||
state_dir: state_dir.into(),
|
||||
interface_prefix: DEFAULT_INTERFACE_PREFIX.to_string(),
|
||||
ports: PortPolicy::default(),
|
||||
advertise: AdvertisePolicy::LocalInterfaces,
|
||||
keepalive: Some(25),
|
||||
mtu: Some(1380),
|
||||
mtu: DEFAULT_MTU,
|
||||
reconcile_debounce: Duration::from_millis(200),
|
||||
reconcile_interval: Duration::from_secs(30),
|
||||
reconcile_interval: Duration::from_secs(15),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -110,15 +98,9 @@ impl WireguardConfig {
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the port policy.
|
||||
pub fn with_ports(mut self, ports: PortPolicy) -> Self {
|
||||
self.ports = ports;
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets what reachability to advertise.
|
||||
pub fn with_advertise(mut self, advertise: AdvertisePolicy) -> Self {
|
||||
self.advertise = advertise;
|
||||
/// Sets the interface MTU.
|
||||
pub fn with_mtu(mut self, mtu: u32) -> Self {
|
||||
self.mtu = mtu;
|
||||
self
|
||||
}
|
||||
|
||||
@@ -140,23 +122,32 @@ impl WireguardConfig {
|
||||
pub struct NetworkOverview {
|
||||
/// The network.
|
||||
pub network: NetworkId,
|
||||
/// Interface this plugin created for it.
|
||||
/// Packet interface this plugin created for it.
|
||||
pub interface: String,
|
||||
/// Interface MTU.
|
||||
pub mtu: u32,
|
||||
/// This agent's WireGuard public key in this network.
|
||||
pub public_key: WgPublicKey,
|
||||
/// This agent's overlay address.
|
||||
pub overlay_address: IpAddr,
|
||||
/// The overlay subnet every member shares.
|
||||
pub overlay_prefix: IpAddr,
|
||||
/// Port the interface listens on.
|
||||
pub listen_port: u16,
|
||||
/// Reachability advertised to peers.
|
||||
pub advertised: Vec<SocketAddr>,
|
||||
/// Peers whose announcements were accepted.
|
||||
/// Prefix length of the overlay subnet.
|
||||
pub overlay_prefix_len: u8,
|
||||
/// Peers this agent knows about.
|
||||
pub peers: Vec<PeerOverview>,
|
||||
/// Packets the operating system sent to an address no peer owns.
|
||||
pub unroutable_packets: u64,
|
||||
}
|
||||
|
||||
/// One accepted peer.
|
||||
impl NetworkOverview {
|
||||
/// Peers whose tunnel has completed a handshake.
|
||||
pub fn established_peers(&self) -> usize {
|
||||
self.peers.iter().filter(|peer| peer.is_up()).count()
|
||||
}
|
||||
}
|
||||
|
||||
/// One peer of the overlay.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PeerOverview {
|
||||
/// The peer's control plane identity.
|
||||
@@ -165,17 +156,28 @@ pub struct PeerOverview {
|
||||
pub public_key: WgPublicKey,
|
||||
/// The overlay address derived for it locally.
|
||||
pub overlay_address: IpAddr,
|
||||
/// Endpoint that will be configured for it, if any.
|
||||
pub endpoint: Option<SocketAddr>,
|
||||
/// Whether a data plane link to it exists.
|
||||
pub has_link: bool,
|
||||
/// The running tunnel, once there is a link.
|
||||
pub tunnel: Option<PeerSummary>,
|
||||
}
|
||||
|
||||
impl PeerOverview {
|
||||
/// Whether the tunnel to this peer has handshaken and can carry traffic.
|
||||
pub fn is_up(&self) -> bool {
|
||||
self.tunnel
|
||||
.as_ref()
|
||||
.is_some_and(|tunnel| tunnel.health.is_up())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct NetworkState {
|
||||
key: WgSecretKey,
|
||||
interface: String,
|
||||
listen_port: u16,
|
||||
advertised: Vec<SocketAddr>,
|
||||
peers: HashMap<EndpointId, ValidatedAnnouncement>,
|
||||
device: Option<Arc<WireguardDevice>>,
|
||||
announcements: HashMap<EndpointId, ValidatedAnnouncement>,
|
||||
links: HashMap<EndpointId, SharedLink>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
@@ -185,19 +187,20 @@ struct Shared {
|
||||
|
||||
#[derive(Debug)]
|
||||
enum Command {
|
||||
/// Make sure a network has keys, a name and a port.
|
||||
Prepare(NetworkId),
|
||||
/// Bring the interface in line with the known peers.
|
||||
Sync(NetworkId),
|
||||
/// Remove the interface for a network.
|
||||
Link {
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
link: SharedLink,
|
||||
},
|
||||
Teardown(NetworkId),
|
||||
/// Tear everything down and stop.
|
||||
Stop(oneshot::Sender<()>),
|
||||
}
|
||||
|
||||
struct Worker {
|
||||
config: WireguardConfig,
|
||||
backend: Arc<dyn WireguardBackend>,
|
||||
tun_factory: Arc<dyn TunFactory>,
|
||||
store: WgKeyStore,
|
||||
shared: Mutex<Shared>,
|
||||
context: OnceLock<PluginContext>,
|
||||
@@ -206,7 +209,7 @@ struct Worker {
|
||||
impl std::fmt::Debug for Worker {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("Worker")
|
||||
.field("backend", &self.backend.name())
|
||||
.field("tun", &self.tun_factory.name())
|
||||
.field("store", &self.store.path())
|
||||
.finish()
|
||||
}
|
||||
@@ -227,7 +230,7 @@ impl WireguardPlugin {
|
||||
/// runtime of its own.
|
||||
pub async fn open(
|
||||
config: WireguardConfig,
|
||||
backend: Arc<dyn WireguardBackend>,
|
||||
tun_factory: Arc<dyn TunFactory>,
|
||||
) -> Result<Arc<Self>, PluginError> {
|
||||
// Validate the prefix once, here, rather than failing per network.
|
||||
interface_name(&config.interface_prefix, NetworkId::from_bytes([0u8; 32]))?;
|
||||
@@ -239,7 +242,7 @@ impl WireguardPlugin {
|
||||
|
||||
let worker = Arc::new(Worker {
|
||||
config,
|
||||
backend,
|
||||
tun_factory,
|
||||
store,
|
||||
shared: Mutex::new(Shared::default()),
|
||||
context: OnceLock::new(),
|
||||
@@ -259,14 +262,28 @@ impl WireguardPlugin {
|
||||
pub fn overview(&self, network: NetworkId) -> Option<NetworkOverview> {
|
||||
let shared = self.worker.lock_shared();
|
||||
let state = shared.networks.get(&network)?;
|
||||
|
||||
let tunnels: HashMap<WgPublicKey, PeerSummary> = state
|
||||
.device
|
||||
.as_ref()
|
||||
.map(|device| {
|
||||
device
|
||||
.peers()
|
||||
.into_iter()
|
||||
.map(|summary| (summary.public_key, summary))
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
|
||||
let mut peers: Vec<PeerOverview> = state
|
||||
.peers
|
||||
.announcements
|
||||
.iter()
|
||||
.map(|(endpoint_id, announcement)| PeerOverview {
|
||||
endpoint_id: *endpoint_id,
|
||||
public_key: announcement.public_key,
|
||||
overlay_address: IpAddr::V6(announcement.overlay_address),
|
||||
endpoint: announcement.preferred_endpoint(),
|
||||
has_link: state.links.contains_key(endpoint_id),
|
||||
tunnel: tunnels.get(&announcement.public_key).cloned(),
|
||||
})
|
||||
.collect();
|
||||
peers.sort_by_key(|peer| peer.public_key);
|
||||
@@ -274,18 +291,21 @@ impl WireguardPlugin {
|
||||
Some(NetworkOverview {
|
||||
network,
|
||||
interface: state.interface.clone(),
|
||||
mtu: self.worker.config.mtu,
|
||||
public_key: state.key.public(),
|
||||
overlay_address: IpAddr::V6(overlay_address(network, &state.key.public())),
|
||||
overlay_prefix: IpAddr::V6(overlay_prefix(network)),
|
||||
listen_port: state.listen_port,
|
||||
advertised: state.advertised.clone(),
|
||||
overlay_prefix_len: OVERLAY_PREFIX_LEN,
|
||||
peers,
|
||||
unroutable_packets: state
|
||||
.device
|
||||
.as_ref()
|
||||
.map(|device| device.unroutable_packets())
|
||||
.unwrap_or(0),
|
||||
})
|
||||
}
|
||||
|
||||
/// Asks the reconciliation task to run now, and waits for it to be queued.
|
||||
///
|
||||
/// Tests use it to avoid waiting for the periodic tick.
|
||||
/// Asks the reconciliation task to run now.
|
||||
pub async fn reconcile_now(&self, network: NetworkId) {
|
||||
let _ = self.commands.send(Command::Sync(network)).await;
|
||||
}
|
||||
@@ -293,7 +313,7 @@ impl WireguardPlugin {
|
||||
fn nudge(&self, command: Command) {
|
||||
if let Err(err) = self.commands.try_send(command) {
|
||||
// A full queue means work is already scheduled; the periodic
|
||||
// reconcile will pick anything up that was missed.
|
||||
// reconcile picks up anything that was missed.
|
||||
tracing::debug!(%err, "wireguard command queue is busy");
|
||||
}
|
||||
}
|
||||
@@ -320,55 +340,27 @@ impl Worker {
|
||||
}
|
||||
}
|
||||
|
||||
/// Gathers the addresses to advertise for our own listening port.
|
||||
async fn advertised_endpoints(&self, listen_port: u16) -> Vec<SocketAddr> {
|
||||
let addresses: Vec<IpAddr> = match &self.config.advertise {
|
||||
AdvertisePolicy::None => Vec::new(),
|
||||
AdvertisePolicy::Explicit(addresses) => addresses.clone(),
|
||||
AdvertisePolicy::LocalInterfaces => {
|
||||
let state = netwatch::interfaces::State::new().await;
|
||||
state.local_addresses.regular
|
||||
}
|
||||
};
|
||||
|
||||
let mut endpoints: Vec<SocketAddr> = addresses
|
||||
.into_iter()
|
||||
.filter(|addr| !addr.is_loopback() && !addr.is_unspecified() && !is_link_local(addr))
|
||||
.map(|addr| SocketAddr::new(addr, listen_port))
|
||||
.collect();
|
||||
endpoints.sort();
|
||||
endpoints.dedup();
|
||||
endpoints.truncate(super::announcement::MAX_ENDPOINTS);
|
||||
endpoints
|
||||
}
|
||||
|
||||
/// Makes sure a network has a key, an interface name and a port.
|
||||
/// Makes sure a network has a key, a name and a running packet interface.
|
||||
///
|
||||
/// Returns `true` when something changed and peers should be told.
|
||||
/// Returns `true` when the key became available now, so peers should be
|
||||
/// told. Creating the interface may fail without privileges; the key and
|
||||
/// the announcement still work, and the interface is retried.
|
||||
async fn prepare(self: &Arc<Self>, network: NetworkId) -> Result<bool, PluginError> {
|
||||
let existing = {
|
||||
let shared = self.lock_shared();
|
||||
shared
|
||||
.networks
|
||||
.get(&network)
|
||||
.map(|state| (state.listen_port, state.advertised.clone()))
|
||||
.map(|state| state.device.is_some())
|
||||
};
|
||||
|
||||
let listen_port = match existing {
|
||||
Some((port, _)) => port,
|
||||
None => self.config.ports.port_for(network)?,
|
||||
};
|
||||
let advertised = self.advertised_endpoints(listen_port).await;
|
||||
|
||||
if let Some((_, previous)) = existing {
|
||||
if previous == advertised {
|
||||
if let Some(has_device) = existing {
|
||||
if has_device {
|
||||
return Ok(false);
|
||||
}
|
||||
let mut shared = self.lock_shared();
|
||||
if let Some(state) = shared.networks.get_mut(&network) {
|
||||
state.advertised = advertised;
|
||||
}
|
||||
return Ok(true);
|
||||
// The key is there but the interface is not. Try again.
|
||||
self.ensure_device(network).await?;
|
||||
return Ok(false);
|
||||
}
|
||||
|
||||
let name = interface_name(&self.config.interface_prefix, network)?;
|
||||
@@ -377,86 +369,97 @@ impl Worker {
|
||||
.await
|
||||
.map_err(|err| PluginError::Other(format!("key store task failed: {err}")))??;
|
||||
|
||||
let mut shared = self.lock_shared();
|
||||
shared.networks.entry(network).or_insert(NetworkState {
|
||||
key,
|
||||
interface: name,
|
||||
listen_port,
|
||||
advertised,
|
||||
peers: HashMap::new(),
|
||||
});
|
||||
{
|
||||
let mut shared = self.lock_shared();
|
||||
shared.networks.entry(network).or_insert(NetworkState {
|
||||
key,
|
||||
interface: name,
|
||||
device: None,
|
||||
announcements: HashMap::new(),
|
||||
links: HashMap::new(),
|
||||
});
|
||||
}
|
||||
|
||||
// The announcement only needs the key, so peers can be told even if
|
||||
// the interface is not up yet.
|
||||
let device = self.ensure_device(network).await;
|
||||
if let Err(err) = device {
|
||||
self.report(network, err);
|
||||
}
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// Builds the configuration this agent wants for a network.
|
||||
fn desired_config(&self, network: NetworkId) -> Option<InterfaceConfig> {
|
||||
let shared = self.lock_shared();
|
||||
let state = shared.networks.get(&network)?;
|
||||
|
||||
let endpoints: HashMap<WgPublicKey, SocketAddr> = state
|
||||
.peers
|
||||
.values()
|
||||
.filter_map(|announcement| {
|
||||
announcement
|
||||
.preferred_endpoint()
|
||||
.map(|endpoint| (announcement.public_key, endpoint))
|
||||
})
|
||||
.collect();
|
||||
let keys: Vec<WgPublicKey> = state
|
||||
.peers
|
||||
.values()
|
||||
.map(|announcement| announcement.public_key)
|
||||
.collect();
|
||||
|
||||
Some(build_interface(
|
||||
InterfaceParams {
|
||||
network,
|
||||
name: state.interface.clone(),
|
||||
private_key: state.key.clone(),
|
||||
listen_port: state.listen_port,
|
||||
mtu: self.config.mtu,
|
||||
keepalive: self.config.keepalive,
|
||||
},
|
||||
keys,
|
||||
|key| endpoints.get(key).copied(),
|
||||
))
|
||||
}
|
||||
|
||||
/// Brings the interface in line with the desired configuration.
|
||||
async fn sync(&self, network: NetworkId) -> Result<(), PluginError> {
|
||||
let Some(desired) = self.desired_config(network) else {
|
||||
return Ok(());
|
||||
};
|
||||
let backend = Arc::clone(&self.backend);
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let current = backend.inspect(&desired.name)?;
|
||||
// Reconciliation: anything that drifted, including an edit made by
|
||||
// hand, is corrected here.
|
||||
if current.as_ref() == Some(&desired.to_state()) {
|
||||
return Ok(());
|
||||
/// Creates the packet interface and starts the WireGuard device.
|
||||
async fn ensure_device(&self, network: NetworkId) -> Result<(), PluginError> {
|
||||
let (name, key) = {
|
||||
let shared = self.lock_shared();
|
||||
match shared.networks.get(&network) {
|
||||
Some(state) if state.device.is_none() => {
|
||||
(state.interface.clone(), state.key.clone())
|
||||
}
|
||||
_ => return Ok(()),
|
||||
}
|
||||
backend.apply(&desired)
|
||||
})
|
||||
.await
|
||||
.map_err(|err| PluginError::Other(format!("wireguard apply task failed: {err}")))?
|
||||
};
|
||||
|
||||
let request = TunRequest {
|
||||
name: name.clone(),
|
||||
address: overlay_address(network, &key.public()),
|
||||
prefix_len: OVERLAY_PREFIX_LEN,
|
||||
mtu: self.config.mtu,
|
||||
};
|
||||
let tun = self.tun_factory.create(request).await?;
|
||||
let device = Arc::new(WireguardDevice::start(network, key, tun));
|
||||
|
||||
let mut shared = self.lock_shared();
|
||||
if let Some(state) = shared.networks.get_mut(&network) {
|
||||
state.interface = device.interface().to_string();
|
||||
state.device = Some(device);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Removes the interface for a network, keeping its key.
|
||||
async fn teardown(&self, network: NetworkId) -> Result<(), PluginError> {
|
||||
let interface = {
|
||||
let mut shared = self.lock_shared();
|
||||
shared
|
||||
.networks
|
||||
.remove(&network)
|
||||
.map(|state| state.interface)
|
||||
/// Brings the running tunnels in line with what is known.
|
||||
///
|
||||
/// A peer gets a tunnel once both halves have arrived: its announcement,
|
||||
/// which says who it is, and a link, which says packets can reach it.
|
||||
fn sync(&self, network: NetworkId) {
|
||||
let mut shared = self.lock_shared();
|
||||
let Some(state) = shared.networks.get_mut(&network) else {
|
||||
return;
|
||||
};
|
||||
let Some(interface) = interface else {
|
||||
return Ok(());
|
||||
let Some(device) = state.device.clone() else {
|
||||
return;
|
||||
};
|
||||
let backend = Arc::clone(&self.backend);
|
||||
tokio::task::spawn_blocking(move || backend.remove(&interface))
|
||||
.await
|
||||
.map_err(|err| PluginError::Other(format!("wireguard remove task failed: {err}")))?
|
||||
|
||||
let mut wanted: Vec<WgPublicKey> = Vec::new();
|
||||
for (endpoint_id, announcement) in &state.announcements {
|
||||
let Some(link) = state.links.get(endpoint_id) else {
|
||||
continue;
|
||||
};
|
||||
if link.is_closed() {
|
||||
continue;
|
||||
}
|
||||
wanted.push(announcement.public_key);
|
||||
if device.has_peer(&announcement.public_key) {
|
||||
continue;
|
||||
}
|
||||
if let Err(err) = device.add_peer(
|
||||
*endpoint_id,
|
||||
announcement.public_key,
|
||||
Arc::clone(link),
|
||||
self.config.keepalive,
|
||||
) {
|
||||
tracing::debug!(%err, "cannot start a WireGuard tunnel");
|
||||
}
|
||||
}
|
||||
device.retain_peers(&wanted);
|
||||
}
|
||||
|
||||
/// Removes a network's interface and tunnels, keeping its key.
|
||||
fn teardown(&self, network: NetworkId) {
|
||||
// Dropping the state drops the device, which stops its tasks and
|
||||
// closes the packet interface.
|
||||
self.lock_shared().networks.remove(&network);
|
||||
}
|
||||
|
||||
fn known_networks(&self) -> Vec<NetworkId> {
|
||||
@@ -465,16 +468,11 @@ impl Worker {
|
||||
}
|
||||
|
||||
/// The reconciliation task.
|
||||
///
|
||||
/// Changes are coalesced over a short debounce so that a burst of peer
|
||||
/// announcements produces one apply, and a periodic tick reconciles even when
|
||||
/// nothing changed locally.
|
||||
async fn run(worker: Arc<Worker>, mut commands: mpsc::Receiver<Command>) {
|
||||
let mut pending: BTreeSet<NetworkId> = BTreeSet::new();
|
||||
let mut deadline: Option<tokio::time::Instant> = None;
|
||||
let mut ticker = tokio::time::interval(worker.config.reconcile_interval);
|
||||
ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
|
||||
// The first tick fires immediately and would reconcile nothing.
|
||||
ticker.tick().await;
|
||||
|
||||
loop {
|
||||
@@ -495,18 +493,23 @@ async fn run(worker: Arc<Worker>, mut commands: mpsc::Receiver<Command>) {
|
||||
Command::Sync(network) => {
|
||||
pending.insert(network);
|
||||
}
|
||||
Command::Link { network, peer, link } => {
|
||||
{
|
||||
let mut shared = worker.lock_shared();
|
||||
if let Some(state) = shared.networks.get_mut(&network) {
|
||||
state.links.insert(peer, link);
|
||||
}
|
||||
}
|
||||
pending.insert(network);
|
||||
}
|
||||
Command::Teardown(network) => {
|
||||
pending.remove(&network);
|
||||
if let Err(err) = worker.teardown(network).await {
|
||||
worker.report(network, err);
|
||||
}
|
||||
worker.teardown(network);
|
||||
continue;
|
||||
}
|
||||
Command::Stop(reply) => {
|
||||
for network in worker.known_networks() {
|
||||
if let Err(err) = worker.teardown(network).await {
|
||||
tracing::warn!(%err, "wireguard teardown failed during shutdown");
|
||||
}
|
||||
worker.teardown(network);
|
||||
}
|
||||
let _ = reply.send(());
|
||||
return;
|
||||
@@ -517,37 +520,28 @@ async fn run(worker: Arc<Worker>, mut commands: mpsc::Receiver<Command>) {
|
||||
_ = async {
|
||||
match wait_until {
|
||||
Some(at) => tokio::time::sleep_until(at).await,
|
||||
// Never resolves; the branch is disabled by the guard.
|
||||
None => std::future::pending::<()>().await,
|
||||
}
|
||||
}, if wait_until.is_some() => {
|
||||
deadline = None;
|
||||
for network in std::mem::take(&mut pending) {
|
||||
if let Err(err) = worker.sync(network).await {
|
||||
worker.report(network, err);
|
||||
}
|
||||
worker.sync(network);
|
||||
}
|
||||
}
|
||||
_ = ticker.tick() => {
|
||||
// Periodic reconciliation is what corrects drift nobody told
|
||||
// us about.
|
||||
for network in worker.known_networks() {
|
||||
if let Err(err) = worker.sync(network).await {
|
||||
worker.report(network, err);
|
||||
// Also the retry for an interface that could not be
|
||||
// created earlier.
|
||||
if let Err(err) = worker.ensure_device(network).await {
|
||||
tracing::debug!(%err, "packet interface still unavailable");
|
||||
}
|
||||
worker.sync(network);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn is_link_local(addr: &IpAddr) -> bool {
|
||||
match addr {
|
||||
IpAddr::V4(ip) => ip.is_link_local(),
|
||||
IpAddr::V6(ip) => (ip.segments()[0] & 0xffc0) == 0xfe80,
|
||||
}
|
||||
}
|
||||
|
||||
impl IpPlugin for WireguardPlugin {
|
||||
fn protocol_id(&self) -> &str {
|
||||
WIREGUARD_PROTOCOL
|
||||
@@ -567,19 +561,15 @@ impl IpPlugin for WireguardPlugin {
|
||||
) -> Result<Option<PluginCapability>, PluginError> {
|
||||
let shared = self.worker.lock_shared();
|
||||
let Some(state) = shared.networks.get(&network) else {
|
||||
// Not ready yet. Ask for preparation; once it finishes the plugin
|
||||
// asks the agent to re-announce, so peers are not left waiting.
|
||||
// Not ready yet. Ask for preparation; once the key exists the
|
||||
// plugin asks the agent to re-announce.
|
||||
drop(shared);
|
||||
self.nudge(Command::Prepare(network));
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
let announcement = WgAnnouncement::new(
|
||||
network,
|
||||
&state.key.public(),
|
||||
state.listen_port,
|
||||
state.advertised.clone(),
|
||||
);
|
||||
// Identity only. Where to send packets is the transport's business.
|
||||
let announcement = WgAnnouncement::new(network, &state.key.public());
|
||||
Ok(Some(PluginCapability {
|
||||
protocol: WIREGUARD_PROTOCOL.to_string(),
|
||||
version: super::announcement::ANNOUNCEMENT_VERSION,
|
||||
@@ -614,7 +604,7 @@ impl IpPlugin for WireguardPlugin {
|
||||
let mut shared = self.worker.lock_shared();
|
||||
match shared.networks.get_mut(&network) {
|
||||
Some(state) => {
|
||||
state.peers.insert(peer, validated) != state.peers.get(&peer).cloned()
|
||||
state.announcements.insert(peer, validated.clone()) != Some(validated)
|
||||
}
|
||||
None => false,
|
||||
}
|
||||
@@ -625,14 +615,24 @@ impl IpPlugin for WireguardPlugin {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn on_peer_link(&self, network: NetworkId, peer: EndpointId, link: SharedLink) {
|
||||
self.nudge(Command::Link {
|
||||
network,
|
||||
peer,
|
||||
link,
|
||||
});
|
||||
}
|
||||
|
||||
fn on_peer_gone(&self, network: NetworkId, peer: EndpointId) {
|
||||
let removed = {
|
||||
let mut shared = self.worker.lock_shared();
|
||||
shared
|
||||
.networks
|
||||
.get_mut(&network)
|
||||
.and_then(|state| state.peers.remove(&peer))
|
||||
.is_some()
|
||||
match shared.networks.get_mut(&network) {
|
||||
Some(state) => {
|
||||
let had_link = state.links.remove(&peer).is_some();
|
||||
state.announcements.remove(&peer).is_some() || had_link
|
||||
}
|
||||
None => false,
|
||||
}
|
||||
};
|
||||
if removed {
|
||||
self.nudge(Command::Sync(network));
|
||||
@@ -659,7 +659,6 @@ impl IpPlugin for WireguardPlugin {
|
||||
|
||||
impl Drop for WireguardPlugin {
|
||||
fn drop(&mut self) {
|
||||
// Safety net for a plugin dropped without an explicit shutdown.
|
||||
if let Ok(mut guard) = self.task.lock()
|
||||
&& let Some(task) = guard.take()
|
||||
{
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
//! The boundary to the operating system's packet interface.
|
||||
//!
|
||||
//! The WireGuard implementation in [`super::device`] is pure userspace and
|
||||
//! needs no kernel WireGuard module and no `wg` tool. It does still need a way
|
||||
//! to hand IP packets to the operating system, which is what this trait is.
|
||||
//!
|
||||
//! Two implementations:
|
||||
//!
|
||||
//! * [`MemoryTun`] keeps packets in memory. It needs no privileges at all and
|
||||
//! is what the test suite uses, so the entire data plane — handshake,
|
||||
//! encryption, routing — is exercised without touching the host.
|
||||
//! * `SystemTun`, behind the `tun-device` feature, is a real TUN interface.
|
||||
//! Creating one needs `CAP_NET_ADMIN` on Linux or the equivalent elsewhere.
|
||||
|
||||
use std::net::Ipv6Addr;
|
||||
use std::sync::Arc;
|
||||
|
||||
use bytes::Bytes;
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::PluginError;
|
||||
|
||||
/// What a device should look like once created.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct TunRequest {
|
||||
/// Interface name to ask for.
|
||||
pub name: String,
|
||||
/// The overlay address this host answers to.
|
||||
pub address: Ipv6Addr,
|
||||
/// Prefix length of the overlay subnet, so the OS routes it here.
|
||||
pub prefix_len: u8,
|
||||
/// Interface MTU.
|
||||
pub mtu: u32,
|
||||
}
|
||||
|
||||
/// A packet interface.
|
||||
///
|
||||
/// `recv` yields packets the operating system wants sent; `send` delivers
|
||||
/// packets that arrived from a peer.
|
||||
pub trait TunDevice: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// The interface name the operating system actually gave us.
|
||||
fn name(&self) -> &str;
|
||||
|
||||
/// The interface MTU.
|
||||
fn mtu(&self) -> u32;
|
||||
|
||||
/// The next packet the operating system wants to send, or `None` once the
|
||||
/// device is gone.
|
||||
fn recv(&self) -> BoxFuture<'_, Option<Bytes>>;
|
||||
|
||||
/// Delivers a packet to the operating system.
|
||||
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), PluginError>>;
|
||||
}
|
||||
|
||||
/// Creates packet interfaces.
|
||||
pub trait TunFactory: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// A short name used in diagnostics.
|
||||
fn name(&self) -> &str;
|
||||
|
||||
/// Creates a device.
|
||||
fn create<'a>(
|
||||
&'a self,
|
||||
request: TunRequest,
|
||||
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, PluginError>>;
|
||||
}
|
||||
|
||||
/// An in-memory packet interface.
|
||||
///
|
||||
/// Nothing reaches the operating system. Packets the device "sends" can be
|
||||
/// read back with [`MemoryTun::pop_to_os`], and packets can be injected as if
|
||||
/// the operating system produced them with [`MemoryTun::push_from_os`].
|
||||
#[derive(Debug)]
|
||||
pub struct MemoryTun {
|
||||
name: String,
|
||||
mtu: u32,
|
||||
from_os_tx: tokio::sync::mpsc::UnboundedSender<Bytes>,
|
||||
from_os_rx: tokio::sync::Mutex<tokio::sync::mpsc::UnboundedReceiver<Bytes>>,
|
||||
to_os_tx: tokio::sync::mpsc::UnboundedSender<Bytes>,
|
||||
to_os_rx: tokio::sync::Mutex<tokio::sync::mpsc::UnboundedReceiver<Bytes>>,
|
||||
}
|
||||
|
||||
impl MemoryTun {
|
||||
/// Creates a device with the given name and MTU.
|
||||
pub fn new(name: impl Into<String>, mtu: u32) -> Arc<Self> {
|
||||
let (from_os_tx, from_os_rx) = tokio::sync::mpsc::unbounded_channel();
|
||||
let (to_os_tx, to_os_rx) = tokio::sync::mpsc::unbounded_channel();
|
||||
Arc::new(Self {
|
||||
name: name.into(),
|
||||
mtu,
|
||||
from_os_tx,
|
||||
from_os_rx: tokio::sync::Mutex::new(from_os_rx),
|
||||
to_os_tx,
|
||||
to_os_rx: tokio::sync::Mutex::new(to_os_rx),
|
||||
})
|
||||
}
|
||||
|
||||
/// Injects a packet as if the operating system had produced it.
|
||||
pub fn push_from_os(&self, packet: Bytes) {
|
||||
let _ = self.from_os_tx.send(packet);
|
||||
}
|
||||
|
||||
/// Takes the next packet the device delivered to the operating system.
|
||||
pub async fn pop_to_os(&self) -> Option<Bytes> {
|
||||
self.to_os_rx.lock().await.recv().await
|
||||
}
|
||||
}
|
||||
|
||||
impl TunDevice for MemoryTun {
|
||||
fn name(&self) -> &str {
|
||||
&self.name
|
||||
}
|
||||
|
||||
fn mtu(&self) -> u32 {
|
||||
self.mtu
|
||||
}
|
||||
|
||||
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
|
||||
Box::pin(async move { self.from_os_rx.lock().await.recv().await })
|
||||
}
|
||||
|
||||
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), PluginError>> {
|
||||
Box::pin(async move {
|
||||
let _ = self.to_os_tx.send(packet);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Creates [`MemoryTun`] devices.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct MemoryTunFactory {
|
||||
created: Arc<std::sync::Mutex<Vec<Arc<MemoryTun>>>>,
|
||||
}
|
||||
|
||||
impl MemoryTunFactory {
|
||||
/// Creates a factory.
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// The device created for an interface name, if any.
|
||||
pub fn device(&self, name: &str) -> Option<Arc<MemoryTun>> {
|
||||
let guard = match self.created.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
guard
|
||||
.iter()
|
||||
.find(|device| device.name() == name)
|
||||
.map(Arc::clone)
|
||||
}
|
||||
|
||||
/// Every device created so far.
|
||||
pub fn devices(&self) -> Vec<Arc<MemoryTun>> {
|
||||
let guard = match self.created.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
guard.clone()
|
||||
}
|
||||
}
|
||||
|
||||
impl TunFactory for MemoryTunFactory {
|
||||
fn name(&self) -> &str {
|
||||
"memory"
|
||||
}
|
||||
|
||||
fn create<'a>(
|
||||
&'a self,
|
||||
request: TunRequest,
|
||||
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, PluginError>> {
|
||||
Box::pin(async move {
|
||||
let device = MemoryTun::new(request.name, request.mtu);
|
||||
let mut guard = match self.created.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
guard.push(Arc::clone(&device));
|
||||
Ok(device as Arc<dyn TunDevice>)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "tun-device")]
|
||||
pub use system::SystemTunFactory;
|
||||
|
||||
#[cfg(feature = "tun-device")]
|
||||
mod system {
|
||||
use std::sync::Arc;
|
||||
|
||||
use bytes::Bytes;
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
use super::{TunDevice, TunFactory, TunRequest};
|
||||
use crate::BoxFuture;
|
||||
use crate::dataplane::PluginError;
|
||||
|
||||
/// A real TUN interface.
|
||||
///
|
||||
/// Creating one needs `CAP_NET_ADMIN` on Linux, or the platform
|
||||
/// equivalent. Failure is reported, never fatal for the agent.
|
||||
pub struct SystemTun {
|
||||
name: String,
|
||||
mtu: u32,
|
||||
reader: Mutex<tokio::io::ReadHalf<tun::AsyncDevice>>,
|
||||
writer: Mutex<tokio::io::WriteHalf<tun::AsyncDevice>>,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for SystemTun {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("SystemTun")
|
||||
.field("name", &self.name)
|
||||
.field("mtu", &self.mtu)
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
impl TunDevice for SystemTun {
|
||||
fn name(&self) -> &str {
|
||||
&self.name
|
||||
}
|
||||
|
||||
fn mtu(&self) -> u32 {
|
||||
self.mtu
|
||||
}
|
||||
|
||||
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
|
||||
Box::pin(async move {
|
||||
use tokio::io::AsyncReadExt;
|
||||
let mut buffer = vec![0u8; self.mtu as usize + 64];
|
||||
let mut reader = self.reader.lock().await;
|
||||
match reader.read(&mut buffer).await {
|
||||
Ok(0) => None,
|
||||
Ok(read) => {
|
||||
buffer.truncate(read);
|
||||
Some(Bytes::from(buffer))
|
||||
}
|
||||
Err(err) => {
|
||||
tracing::debug!(%err, "tun read failed");
|
||||
None
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), PluginError>> {
|
||||
Box::pin(async move {
|
||||
use tokio::io::AsyncWriteExt;
|
||||
let mut writer = self.writer.lock().await;
|
||||
writer
|
||||
.write_all(&packet)
|
||||
.await
|
||||
.map_err(|err| PluginError::Other(format!("tun write failed: {err}")))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Creates real TUN interfaces.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct SystemTunFactory;
|
||||
|
||||
impl SystemTunFactory {
|
||||
/// Creates the factory.
|
||||
pub fn new() -> Self {
|
||||
Self
|
||||
}
|
||||
}
|
||||
|
||||
impl TunFactory for SystemTunFactory {
|
||||
fn name(&self) -> &str {
|
||||
"system"
|
||||
}
|
||||
|
||||
fn create<'a>(
|
||||
&'a self,
|
||||
request: TunRequest,
|
||||
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, PluginError>> {
|
||||
Box::pin(async move {
|
||||
let mut config = tun::Configuration::default();
|
||||
config.tun_name(&request.name).mtu(request.mtu as u16).up();
|
||||
// The overlay address and its subnet, so the operating system
|
||||
// routes overlay traffic into this interface.
|
||||
let _ = (&request.address, request.prefix_len);
|
||||
|
||||
let device = tun::create_as_async(&config).map_err(|err| {
|
||||
PluginError::Unavailable(format!(
|
||||
"cannot create the TUN interface `{}`: {err}. \
|
||||
This needs CAP_NET_ADMIN (try running as root).",
|
||||
request.name
|
||||
))
|
||||
})?;
|
||||
|
||||
// The name was requested explicitly; creation fails rather
|
||||
// than silently picking another one.
|
||||
let name = request.name.clone();
|
||||
let (reader, writer) = tokio::io::split(device);
|
||||
|
||||
Ok(Arc::new(SystemTun {
|
||||
name,
|
||||
mtu: request.mtu,
|
||||
reader: Mutex::new(reader),
|
||||
writer: Mutex::new(writer),
|
||||
}) as Arc<dyn TunDevice>)
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,667 +0,0 @@
|
||||
//! A backend that drives the standard `wg` and `ip` tools.
|
||||
//!
|
||||
//! This is the one part of the plugin that changes the operating system. It is
|
||||
//! split in two deliberately:
|
||||
//!
|
||||
//! * a **pure planner** that turns a desired configuration into an exact list
|
||||
//! of commands, and pure **parsers** for the tools' output — both fully
|
||||
//! unit tested on every platform;
|
||||
//! * a thin executor that runs the plan, which needs Linux and
|
||||
//! `CAP_NET_ADMIN`.
|
||||
//!
|
||||
//! Nothing that arrives from the network is ever passed through as text. Peer
|
||||
//! keys, endpoints, allowed prefixes and keepalives are typed values that this
|
||||
//! module re-serialises itself, so an announcement cannot inject an argument
|
||||
//! or a configuration directive. The only names involved are derived locally.
|
||||
//!
|
||||
//! The interface is created by this plugin and removed by this plugin. An
|
||||
//! interface that already exists and is not a WireGuard device is refused, not
|
||||
//! adopted, so the agent never takes over something it did not create.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
use std::net::SocketAddr;
|
||||
|
||||
use zeroize::Zeroizing;
|
||||
|
||||
use crate::dataplane::PluginError;
|
||||
|
||||
use super::config::{Cidr, InterfaceConfig, InterfaceState, PeerState};
|
||||
use super::keys::{WgPublicKey, WgSecretKey};
|
||||
|
||||
/// Which external programs to use.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Tools {
|
||||
/// The `wg` executable.
|
||||
pub wg: String,
|
||||
/// The `ip` executable.
|
||||
pub ip: String,
|
||||
}
|
||||
|
||||
impl Default for Tools {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
wg: "wg".into(),
|
||||
ip: "ip".into(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One command to run, with optional data for its standard input.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct WgCommand {
|
||||
/// Program to execute.
|
||||
pub program: String,
|
||||
/// Arguments, already separated. Never a shell string.
|
||||
pub args: Vec<String>,
|
||||
/// Data piped to the program's standard input.
|
||||
///
|
||||
/// Used for the WireGuard configuration so the private key never reaches
|
||||
/// the filesystem. Zeroized on drop.
|
||||
pub stdin: Option<Zeroizing<String>>,
|
||||
}
|
||||
|
||||
impl WgCommand {
|
||||
fn new(program: &str, args: &[&str]) -> Self {
|
||||
Self {
|
||||
program: program.to_string(),
|
||||
args: args.iter().map(|arg| arg.to_string()).collect(),
|
||||
stdin: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// A redacted rendering, safe for logs.
|
||||
pub fn describe(&self) -> String {
|
||||
format!("{} {}", self.program, self.args.join(" "))
|
||||
}
|
||||
}
|
||||
|
||||
/// Builds the commands that bring `desired` into being.
|
||||
///
|
||||
/// `current` is what the interface looks like now, or `None` if it does not
|
||||
/// exist yet. The plan is minimal: an interface that already matches produces
|
||||
/// only the idempotent link-up command.
|
||||
pub fn plan_apply(
|
||||
desired: &InterfaceConfig,
|
||||
current: Option<&InterfaceState>,
|
||||
tools: &Tools,
|
||||
) -> Vec<WgCommand> {
|
||||
let mut plan = Vec::new();
|
||||
let name = desired.name.as_str();
|
||||
|
||||
if current.is_none() {
|
||||
plan.push(WgCommand::new(
|
||||
&tools.ip,
|
||||
&["link", "add", "dev", name, "type", "wireguard"],
|
||||
));
|
||||
}
|
||||
|
||||
// `setconf` replaces everything, `syncconf` applies a difference without
|
||||
// tearing down live peers. Use each where it belongs.
|
||||
let subcommand = if current.is_none() {
|
||||
"setconf"
|
||||
} else {
|
||||
"syncconf"
|
||||
};
|
||||
let mut configure = WgCommand::new(&tools.wg, &[subcommand, name, "/dev/stdin"]);
|
||||
configure.stdin = Some(desired.render());
|
||||
plan.push(configure);
|
||||
|
||||
let desired_addrs: BTreeSet<Cidr> = desired.addresses.iter().copied().collect();
|
||||
let current_addrs: BTreeSet<Cidr> = current
|
||||
.map(|state| state.addresses.iter().copied().collect())
|
||||
.unwrap_or_default();
|
||||
|
||||
for addr in desired_addrs.difference(¤t_addrs) {
|
||||
plan.push(WgCommand::new(
|
||||
&tools.ip,
|
||||
&["address", "add", &addr.to_string(), "dev", name],
|
||||
));
|
||||
}
|
||||
// Addresses on an interface this plugin owns that are not wanted any more
|
||||
// were either put there by an older configuration or by hand. Either way
|
||||
// reconciliation removes them.
|
||||
for addr in current_addrs.difference(&desired_addrs) {
|
||||
plan.push(WgCommand::new(
|
||||
&tools.ip,
|
||||
&["address", "del", &addr.to_string(), "dev", name],
|
||||
));
|
||||
}
|
||||
|
||||
if let Some(mtu) = desired.mtu {
|
||||
plan.push(WgCommand::new(
|
||||
&tools.ip,
|
||||
&["link", "set", "mtu", &mtu.to_string(), "dev", name],
|
||||
));
|
||||
}
|
||||
|
||||
plan.push(WgCommand::new(
|
||||
&tools.ip,
|
||||
&["link", "set", "up", "dev", name],
|
||||
));
|
||||
plan
|
||||
}
|
||||
|
||||
/// Builds the commands that remove an interface this plugin created.
|
||||
pub fn plan_remove(interface: &str, tools: &Tools) -> Vec<WgCommand> {
|
||||
vec![WgCommand::new(
|
||||
&tools.ip,
|
||||
&["link", "del", "dev", interface],
|
||||
)]
|
||||
}
|
||||
|
||||
/// Parses the output of `wg showconf <interface>`.
|
||||
///
|
||||
/// The private key present in that output is used only to derive the
|
||||
/// interface's public key and is dropped immediately.
|
||||
pub fn parse_showconf(interface: &str, text: &str) -> Result<InterfaceState, PluginError> {
|
||||
#[derive(Default)]
|
||||
struct PartialPeer {
|
||||
public_key: Option<WgPublicKey>,
|
||||
endpoint: Option<SocketAddr>,
|
||||
allowed_ips: Vec<Cidr>,
|
||||
persistent_keepalive: Option<u16>,
|
||||
}
|
||||
|
||||
let mut public_key: Option<WgPublicKey> = None;
|
||||
let mut listen_port = 0u16;
|
||||
let mut peers: Vec<PeerState> = Vec::new();
|
||||
let mut current: Option<PartialPeer> = None;
|
||||
|
||||
let finish = |peer: PartialPeer, peers: &mut Vec<PeerState>| -> Result<(), PluginError> {
|
||||
let key = peer
|
||||
.public_key
|
||||
.ok_or_else(|| PluginError::Other("wg showconf peer without a public key".into()))?;
|
||||
peers.push(
|
||||
PeerState {
|
||||
public_key: key,
|
||||
endpoint: peer.endpoint,
|
||||
allowed_ips: peer.allowed_ips,
|
||||
persistent_keepalive: peer.persistent_keepalive,
|
||||
}
|
||||
.normalised(),
|
||||
);
|
||||
Ok(())
|
||||
};
|
||||
|
||||
for raw in text.lines() {
|
||||
let line = raw.trim();
|
||||
if line.is_empty() || line.starts_with('#') {
|
||||
continue;
|
||||
}
|
||||
if line.eq_ignore_ascii_case("[interface]") {
|
||||
if let Some(peer) = current.take() {
|
||||
finish(peer, &mut peers)?;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if line.eq_ignore_ascii_case("[peer]") {
|
||||
if let Some(peer) = current.take() {
|
||||
finish(peer, &mut peers)?;
|
||||
}
|
||||
current = Some(PartialPeer::default());
|
||||
continue;
|
||||
}
|
||||
|
||||
let Some((key, value)) = line.split_once('=') else {
|
||||
continue;
|
||||
};
|
||||
let name = key.trim().to_ascii_lowercase();
|
||||
let value = value.trim();
|
||||
|
||||
match current.as_mut() {
|
||||
None => match name.as_str() {
|
||||
"privatekey" => {
|
||||
// Derive the public key, then let the secret drop.
|
||||
let raw = data_encoding::BASE64
|
||||
.decode(value.as_bytes())
|
||||
.map_err(|_| {
|
||||
PluginError::Other("wg showconf private key is not base64".into())
|
||||
})?;
|
||||
let bytes = <[u8; 32]>::try_from(raw.as_slice()).map_err(|_| {
|
||||
PluginError::Other("wg showconf private key is not 32 bytes".into())
|
||||
})?;
|
||||
public_key = Some(WgSecretKey::from_bytes(&bytes).public());
|
||||
}
|
||||
"listenport" => {
|
||||
listen_port = value
|
||||
.parse()
|
||||
.map_err(|_| PluginError::Other(format!("bad listen port {value:?}")))?;
|
||||
}
|
||||
_ => {}
|
||||
},
|
||||
Some(peer) => match name.as_str() {
|
||||
"publickey" => peer.public_key = Some(WgPublicKey::decode(value)?),
|
||||
"endpoint" => peer.endpoint = value.parse().ok(),
|
||||
"allowedips" => {
|
||||
for entry in value.split(',') {
|
||||
let entry = entry.trim();
|
||||
if entry.is_empty() {
|
||||
continue;
|
||||
}
|
||||
peer.allowed_ips.push(parse_cidr(entry)?);
|
||||
}
|
||||
}
|
||||
"persistentkeepalive" => {
|
||||
peer.persistent_keepalive =
|
||||
match value {
|
||||
"off" => None,
|
||||
other => Some(other.parse().map_err(|_| {
|
||||
PluginError::Other(format!("bad keepalive {other:?}"))
|
||||
})?),
|
||||
};
|
||||
}
|
||||
_ => {}
|
||||
},
|
||||
}
|
||||
}
|
||||
if let Some(peer) = current.take() {
|
||||
finish(peer, &mut peers)?;
|
||||
}
|
||||
|
||||
let public_key = public_key
|
||||
.ok_or_else(|| PluginError::Other("wg showconf did not report a private key".into()))?;
|
||||
|
||||
Ok(InterfaceState {
|
||||
name: interface.to_string(),
|
||||
public_key,
|
||||
listen_port,
|
||||
addresses: Vec::new(),
|
||||
peers,
|
||||
}
|
||||
.normalised())
|
||||
}
|
||||
|
||||
/// Parses the addresses out of `ip -o address show dev <interface>`.
|
||||
pub fn parse_ip_addresses(text: &str) -> Result<Vec<Cidr>, PluginError> {
|
||||
let mut out = Vec::new();
|
||||
for line in text.lines() {
|
||||
let mut tokens = line.split_whitespace();
|
||||
while let Some(token) = tokens.next() {
|
||||
if token != "inet" && token != "inet6" {
|
||||
continue;
|
||||
}
|
||||
let Some(value) = tokens.next() else {
|
||||
continue;
|
||||
};
|
||||
// A link-local address is added by the kernel, not by us.
|
||||
let cidr = parse_cidr(value)?;
|
||||
if cidr.addr.is_loopback() {
|
||||
continue;
|
||||
}
|
||||
if let std::net::IpAddr::V6(ip) = cidr.addr
|
||||
&& (ip.segments()[0] & 0xffc0) == 0xfe80
|
||||
{
|
||||
continue;
|
||||
}
|
||||
out.push(cidr);
|
||||
}
|
||||
}
|
||||
out.sort();
|
||||
out.dedup();
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn parse_cidr(text: &str) -> Result<Cidr, PluginError> {
|
||||
let (addr, prefix) = text
|
||||
.split_once('/')
|
||||
.ok_or_else(|| PluginError::Other(format!("{text:?} is not an address with a prefix")))?;
|
||||
let addr = addr
|
||||
.parse()
|
||||
.map_err(|_| PluginError::Other(format!("{addr:?} is not an IP address")))?;
|
||||
let prefix_len = prefix
|
||||
.parse()
|
||||
.map_err(|_| PluginError::Other(format!("{prefix:?} is not a prefix length")))?;
|
||||
Cidr::new(addr, prefix_len)
|
||||
}
|
||||
|
||||
pub use executor::WgToolBackend;
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
mod executor {
|
||||
use std::io::Write;
|
||||
use std::process::{Command, Stdio};
|
||||
|
||||
use super::*;
|
||||
use crate::dataplane::wireguard::backend::WireguardBackend;
|
||||
|
||||
/// Drives the real `wg` and `ip` tools.
|
||||
///
|
||||
/// Requires Linux and `CAP_NET_ADMIN`, so it is never used by the default
|
||||
/// test suite.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct WgToolBackend {
|
||||
tools: Tools,
|
||||
}
|
||||
|
||||
impl WgToolBackend {
|
||||
/// Creates a backend using `wg` and `ip` from `PATH`.
|
||||
pub fn new() -> Result<Self, PluginError> {
|
||||
Self::with_tools(Tools::default())
|
||||
}
|
||||
|
||||
/// Creates a backend using explicitly located tools.
|
||||
pub fn with_tools(tools: Tools) -> Result<Self, PluginError> {
|
||||
Ok(Self { tools })
|
||||
}
|
||||
|
||||
fn run(&self, command: &WgCommand) -> Result<String, PluginError> {
|
||||
let mut child = Command::new(&command.program)
|
||||
.args(&command.args)
|
||||
.stdin(if command.stdin.is_some() {
|
||||
Stdio::piped()
|
||||
} else {
|
||||
Stdio::null()
|
||||
})
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.spawn()
|
||||
.map_err(|err| {
|
||||
PluginError::Unavailable(format!("cannot run `{}`: {err}", command.program))
|
||||
})?;
|
||||
|
||||
if let Some(stdin) = &command.stdin {
|
||||
let mut handle = child.stdin.take().ok_or_else(|| {
|
||||
PluginError::Other("could not open the child's standard input".into())
|
||||
})?;
|
||||
handle.write_all(stdin.as_bytes()).map_err(|err| {
|
||||
PluginError::Other(format!("cannot write the WireGuard configuration: {err}"))
|
||||
})?;
|
||||
drop(handle);
|
||||
}
|
||||
|
||||
let output = child.wait_with_output().map_err(|err| {
|
||||
PluginError::Other(format!("`{}` did not complete: {err}", command.describe()))
|
||||
})?;
|
||||
if !output.status.success() {
|
||||
// The configuration went to stdin, so stderr cannot contain it.
|
||||
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
|
||||
return Err(PluginError::Unavailable(format!(
|
||||
"`{}` failed: {stderr}",
|
||||
command.describe()
|
||||
)));
|
||||
}
|
||||
Ok(String::from_utf8_lossy(&output.stdout).into_owned())
|
||||
}
|
||||
|
||||
fn link_exists(&self, interface: &str) -> bool {
|
||||
Command::new(&self.tools.ip)
|
||||
.args(["link", "show", "dev", interface])
|
||||
.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.status()
|
||||
.map(|status| status.success())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
}
|
||||
|
||||
impl WireguardBackend for WgToolBackend {
|
||||
fn name(&self) -> &str {
|
||||
"wg-tools"
|
||||
}
|
||||
|
||||
fn inspect(&self, interface: &str) -> Result<Option<InterfaceState>, PluginError> {
|
||||
if !self.link_exists(interface) {
|
||||
return Ok(None);
|
||||
}
|
||||
let showconf = WgCommand::new(&self.tools.wg, &["showconf", interface]);
|
||||
let text = match self.run(&showconf) {
|
||||
Ok(text) => text,
|
||||
Err(_) => {
|
||||
// The link exists but is not a WireGuard device. It is not
|
||||
// ours, so it is refused rather than adopted or modified.
|
||||
return Err(PluginError::Rejected(format!(
|
||||
"interface `{interface}` already exists and is not a WireGuard device; \
|
||||
refusing to touch it"
|
||||
)));
|
||||
}
|
||||
};
|
||||
let mut state = parse_showconf(interface, &text)?;
|
||||
let addresses = self.run(&WgCommand::new(
|
||||
&self.tools.ip,
|
||||
&["-o", "address", "show", "dev", interface],
|
||||
))?;
|
||||
state.addresses = parse_ip_addresses(&addresses)?;
|
||||
Ok(Some(state.normalised()))
|
||||
}
|
||||
|
||||
fn apply(&self, desired: &InterfaceConfig) -> Result<(), PluginError> {
|
||||
let current = self.inspect(&desired.name)?;
|
||||
for command in plan_apply(desired, current.as_ref(), &self.tools) {
|
||||
self.run(&command)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn remove(&self, interface: &str) -> Result<(), PluginError> {
|
||||
if !self.link_exists(interface) {
|
||||
return Ok(());
|
||||
}
|
||||
for command in plan_remove(interface, &self.tools) {
|
||||
self.run(&command)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
mod executor {
|
||||
use super::*;
|
||||
|
||||
/// Placeholder on platforms where this backend is not implemented.
|
||||
///
|
||||
/// The planner and the parsers in this module work everywhere; only
|
||||
/// applying a configuration is Linux-specific.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct WgToolBackend {
|
||||
_private: (),
|
||||
}
|
||||
|
||||
impl WgToolBackend {
|
||||
/// Always fails: this backend drives `ip link ... type wireguard`,
|
||||
/// which exists on Linux only.
|
||||
pub fn new() -> Result<Self, PluginError> {
|
||||
Self::with_tools(Tools::default())
|
||||
}
|
||||
|
||||
/// Always fails, see [`WgToolBackend::new`].
|
||||
pub fn with_tools(_tools: Tools) -> Result<Self, PluginError> {
|
||||
Err(PluginError::Unavailable(
|
||||
"the wg-tools backend is implemented for Linux only".into(),
|
||||
))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use super::*;
|
||||
use crate::dataplane::wireguard::config::{InterfaceParams, build_interface};
|
||||
use crate::dataplane::wireguard::keys::WgSecretKey;
|
||||
use crate::dataplane::wireguard::overlay::overlay_address;
|
||||
use crate::identity::{NetworkId, NetworkKeys, NetworkName, NetworkSecret};
|
||||
|
||||
fn network(name: &str) -> NetworkId {
|
||||
NetworkKeys::derive(
|
||||
&NetworkName::new(name).unwrap(),
|
||||
&NetworkSecret::from_bytes(vec![4u8; 32]).unwrap(),
|
||||
)
|
||||
.network_id()
|
||||
}
|
||||
|
||||
fn sample() -> (NetworkId, InterfaceConfig, WgPublicKey) {
|
||||
let id = network("plan");
|
||||
let peer = WgSecretKey::generate().public();
|
||||
let config = build_interface(
|
||||
InterfaceParams {
|
||||
network: id,
|
||||
name: "tsun0".into(),
|
||||
private_key: WgSecretKey::generate(),
|
||||
listen_port: 51820,
|
||||
mtu: Some(1380),
|
||||
keepalive: Some(25),
|
||||
},
|
||||
[peer],
|
||||
|_| Some("10.0.0.5:51820".parse().unwrap()),
|
||||
);
|
||||
(id, config, peer)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn creating_an_interface_plans_every_step_in_order() {
|
||||
let (_, config, _) = sample();
|
||||
let plan = plan_apply(&config, None, &Tools::default());
|
||||
let described: Vec<String> = plan.iter().map(WgCommand::describe).collect();
|
||||
|
||||
assert_eq!(described[0], "ip link add dev tsun0 type wireguard");
|
||||
assert_eq!(described[1], "wg setconf tsun0 /dev/stdin");
|
||||
assert!(plan[1].stdin.is_some(), "the config goes over stdin");
|
||||
assert!(described.iter().any(|c| c.starts_with("ip address add")));
|
||||
assert!(described.contains(&"ip link set mtu 1380 dev tsun0".to_string()));
|
||||
assert_eq!(described.last().unwrap(), "ip link set up dev tsun0");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn updating_an_existing_interface_syncs_instead_of_replacing() {
|
||||
let (_, config, _) = sample();
|
||||
let current = config.to_state();
|
||||
let plan = plan_apply(&config, Some(¤t), &Tools::default());
|
||||
let described: Vec<String> = plan.iter().map(WgCommand::describe).collect();
|
||||
|
||||
assert!(
|
||||
!described.iter().any(|c| c.contains("link add")),
|
||||
"an existing interface must not be recreated"
|
||||
);
|
||||
assert_eq!(described[0], "wg syncconf tsun0 /dev/stdin");
|
||||
assert!(
|
||||
!described.iter().any(|c| c.starts_with("ip address add")),
|
||||
"matching addresses need no change: {described:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn addresses_that_should_not_be_there_are_removed() {
|
||||
let (_, config, _) = sample();
|
||||
let mut current = config.to_state();
|
||||
current.addresses.push(Cidr {
|
||||
addr: "192.0.2.1".parse().unwrap(),
|
||||
prefix_len: 32,
|
||||
});
|
||||
let plan = plan_apply(&config, Some(¤t), &Tools::default());
|
||||
let described: Vec<String> = plan.iter().map(WgCommand::describe).collect();
|
||||
assert!(
|
||||
described.contains(&"ip address del 192.0.2.1/32 dev tsun0".to_string()),
|
||||
"{described:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nothing_from_the_network_reaches_an_argument_as_text() {
|
||||
let (id, config, peer) = sample();
|
||||
let plan = plan_apply(&config, None, &Tools::default());
|
||||
for command in &plan {
|
||||
for arg in &command.args {
|
||||
assert!(
|
||||
!arg.contains(' ') && !arg.contains(';') && !arg.contains('\n'),
|
||||
"argument {arg:?} is not a single clean token"
|
||||
);
|
||||
}
|
||||
}
|
||||
// The peer's key and derived prefix travel in the piped configuration,
|
||||
// which is a value this crate rendered itself.
|
||||
let rendered = plan[1].stdin.as_ref().unwrap();
|
||||
assert!(rendered.contains(&peer.encode()));
|
||||
assert!(rendered.contains(&Cidr::host(overlay_address(id, &peer)).to_string()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn removal_only_touches_the_named_interface() {
|
||||
let plan = plan_remove("tsun0", &Tools::default());
|
||||
assert_eq!(
|
||||
plan.iter().map(WgCommand::describe).collect::<Vec<_>>(),
|
||||
vec!["ip link del dev tsun0".to_string()]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn showconf_output_parses_into_comparable_state() {
|
||||
let secret = WgSecretKey::generate();
|
||||
let peer_a = WgSecretKey::generate().public();
|
||||
let peer_b = WgSecretKey::generate().public();
|
||||
let text = format!(
|
||||
"[Interface]\n\
|
||||
ListenPort = 51821\n\
|
||||
PrivateKey = {}\n\
|
||||
\n\
|
||||
[Peer]\n\
|
||||
PublicKey = {}\n\
|
||||
AllowedIPs = fd00::2/128, fd00::3/128\n\
|
||||
Endpoint = 10.0.0.9:51820\n\
|
||||
PersistentKeepalive = 25\n\
|
||||
\n\
|
||||
[Peer]\n\
|
||||
PublicKey = {}\n\
|
||||
AllowedIPs = fd00::4/128\n\
|
||||
PersistentKeepalive = off\n",
|
||||
secret.encode().as_str(),
|
||||
peer_a.encode(),
|
||||
peer_b.encode(),
|
||||
);
|
||||
|
||||
let state = parse_showconf("tsun0", &text).unwrap();
|
||||
assert_eq!(state.public_key, secret.public());
|
||||
assert_eq!(state.listen_port, 51821);
|
||||
assert_eq!(state.peers.len(), 2);
|
||||
|
||||
let a = state
|
||||
.peers
|
||||
.iter()
|
||||
.find(|peer| peer.public_key == peer_a)
|
||||
.unwrap();
|
||||
assert_eq!(a.endpoint, Some("10.0.0.9:51820".parse().unwrap()));
|
||||
assert_eq!(a.allowed_ips.len(), 2);
|
||||
assert_eq!(a.persistent_keepalive, Some(25));
|
||||
|
||||
let b = state
|
||||
.peers
|
||||
.iter()
|
||||
.find(|peer| peer.public_key == peer_b)
|
||||
.unwrap();
|
||||
assert_eq!(b.endpoint, None);
|
||||
assert_eq!(b.persistent_keepalive, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_tool_output_is_an_error_not_a_panic() {
|
||||
assert!(parse_showconf("tsun0", "").is_err());
|
||||
assert!(parse_showconf("tsun0", "[Interface]\nListenPort = nope\n").is_err());
|
||||
assert!(parse_showconf("tsun0", "[Peer]\nAllowedIPs = fd00::1/128\n").is_err());
|
||||
assert!(parse_showconf("tsun0", "[Interface]\nPrivateKey = zzzz\n").is_err());
|
||||
assert!(parse_ip_addresses("1: tsun0 inet6 not-an-address scope global").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn interface_addresses_parse_and_skip_kernel_managed_ones() {
|
||||
let text = "3: tsun0 inet6 fd12:3456::1/128 scope global \\ valid_lft forever\n\
|
||||
3: tsun0 inet6 fd12:3456::/64 scope global \\ valid_lft forever\n\
|
||||
3: tsun0 inet6 fe80::1/64 scope link \\ valid_lft forever\n";
|
||||
let addresses = parse_ip_addresses(text).unwrap();
|
||||
assert_eq!(
|
||||
addresses.iter().map(Cidr::to_string).collect::<Vec<_>>(),
|
||||
vec!["fd12:3456::/64".to_string(), "fd12:3456::1/128".to_string()]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_rendered_config_round_trips_through_the_parser() {
|
||||
let (_, config, _) = sample();
|
||||
let rendered = config.render();
|
||||
let parsed = parse_showconf(&config.name, &rendered).unwrap();
|
||||
let mut expected = config.to_state();
|
||||
// showconf does not report interface addresses.
|
||||
expected.addresses.clear();
|
||||
assert_eq!(parsed, expected);
|
||||
}
|
||||
}
|
||||
+6
-2
@@ -32,7 +32,7 @@ use iroh::{Endpoint, EndpointAddr, EndpointId, RelayMode};
|
||||
use crate::config::{AgentConfig, TransportPolicy};
|
||||
use crate::error::{Error, Result};
|
||||
use crate::identity::DeviceIdentity;
|
||||
use crate::proto::message::ALPN;
|
||||
use crate::proto::message::{ALPN, DATA_ALPN};
|
||||
|
||||
/// A network path address as reported by iroh.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
@@ -193,6 +193,10 @@ fn local_addr_string(addr: &iroh::endpoint::LocalTransportAddr) -> Option<String
|
||||
}
|
||||
|
||||
/// Thin wrapper around the iroh endpoint.
|
||||
///
|
||||
/// The endpoint serves two ALPNs: the control protocol and the data plane.
|
||||
/// They are separate connections with separate congestion control, so a busy
|
||||
/// or broken data plane cannot disturb control traffic.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct EndpointAdapter {
|
||||
endpoint: Endpoint,
|
||||
@@ -203,7 +207,7 @@ impl EndpointAdapter {
|
||||
pub async fn bind(config: &AgentConfig, identity: &DeviceIdentity) -> Result<Self> {
|
||||
let mut builder = Endpoint::builder(presets::Minimal)
|
||||
.secret_key(identity.secret_key())
|
||||
.alpns(vec![ALPN.to_vec()]);
|
||||
.alpns(vec![ALPN.to_vec(), DATA_ALPN.to_vec()]);
|
||||
|
||||
builder = match config.transport {
|
||||
TransportPolicy::LocalOnly => builder
|
||||
|
||||
@@ -22,6 +22,17 @@ use crate::error::ProtocolError;
|
||||
/// bumping it must not change any existing [`crate::NetworkId`].
|
||||
pub const ALPN: &[u8] = b"tsunagi/ctrl/1";
|
||||
|
||||
/// ALPN of the tsunagi data plane.
|
||||
///
|
||||
/// Data plane connections are deliberately separate from control plane ones.
|
||||
/// They carry one IP plugin's packets for one network and nothing else, so a
|
||||
/// saturated or broken data plane cannot disturb control traffic, and the
|
||||
/// transport underneath can be replaced without touching the control protocol.
|
||||
pub const DATA_ALPN: &[u8] = b"tsunagi/data/1";
|
||||
|
||||
/// Largest plugin protocol identifier accepted when opening a data channel.
|
||||
pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
|
||||
|
||||
/// Control protocol version carried inside the handshake.
|
||||
pub const PROTOCOL_VERSION: u16 = 1;
|
||||
|
||||
@@ -61,6 +72,26 @@ pub struct Announcement {
|
||||
pub capabilities: Vec<PluginCapability>,
|
||||
}
|
||||
|
||||
/// Opens a data channel, sent by the initiator right after the membership
|
||||
/// handshake on a [`DATA_ALPN`] connection.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct DataOpen {
|
||||
/// Which IP plugin's packets this channel will carry.
|
||||
pub protocol: String,
|
||||
}
|
||||
|
||||
/// The responder's answer to [`DataOpen`].
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct DataOpenAck {
|
||||
/// Whether the channel was accepted.
|
||||
///
|
||||
/// A channel is refused when the responder has no plugin for that
|
||||
/// protocol in that network. That is an ordinary outcome, not an error.
|
||||
pub accepted: bool,
|
||||
/// Largest datagram the responder is willing to receive, in bytes.
|
||||
pub max_datagram: u32,
|
||||
}
|
||||
|
||||
/// A control message exchanged after a successful handshake.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[non_exhaustive]
|
||||
|
||||
+9
-1
@@ -15,6 +15,13 @@
|
||||
//! re-checked against the session's network id.
|
||||
//!
|
||||
//! Nothing here adds its own encryption on top of iroh.
|
||||
//!
|
||||
//! # The data plane speaks a different protocol
|
||||
//!
|
||||
//! IP plugin packets never travel on a control connection. They use their own
|
||||
//! ALPN, [`message::DATA_ALPN`], with the same membership handshake followed by
|
||||
//! [`message::DataOpen`]. Keeping them apart is what lets the data plane's
|
||||
//! transport be replaced without touching anything above.
|
||||
|
||||
pub mod frame;
|
||||
pub mod handshake;
|
||||
@@ -23,5 +30,6 @@ pub mod message;
|
||||
pub use frame::{read_frame, write_frame};
|
||||
pub use handshake::{HandshakeOutcome, Role};
|
||||
pub use message::{
|
||||
ALPN, Announcement, AuthProof, ControlMessage, Envelope, Hello, HelloAck, PROTOCOL_VERSION,
|
||||
ALPN, Announcement, AuthProof, ControlMessage, DATA_ALPN, DataOpen, DataOpenAck, Envelope,
|
||||
Hello, HelloAck, PROTOCOL_VERSION,
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user