Proof-of-concept mesh agent library over iroh
Working library with real iroh connections, not an interface sketch: - persistent device identity in state.sqlite, stable across restarts - deterministic network space derived from name + secret via HKDF-SHA256, with frozen labels and unambiguous length-prefixed encoding - replaceable discovery returning unverified candidates only; static bootstrap, in-memory test backend and a composite - real iroh connections plus an explicit mutual membership proof: HMAC-SHA256 over a role-separated transcript bound to the TLS exporter, the network id and both endpoint identities - small versioned control protocol: handshake, announcement, ping/pong - multiple networks per agent with enforced isolation - automatic reconnect with bounded backoff and jitter - mandatory state vs disposable cache, with a real directory ownership lock - status snapshots, event stream and honest diagnostics 47 integration and unit tests cover the required scenarios offline on loopback. Snapshots, revocations and WireGuard are designed for and documented, not implemented. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
Claude Opus 5
@@ -0,0 +1,112 @@
|
||||
//! Events published by a running agent.
|
||||
//!
|
||||
//! Events are delivered through a bounded [`tokio::sync::broadcast`] channel.
|
||||
//! A slow subscriber is lagged, never allowed to stall the runtime.
|
||||
//!
|
||||
//! Nothing here ever carries a secret, a derived key or a handshake proof.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use iroh::EndpointId;
|
||||
|
||||
use crate::identity::NetworkId;
|
||||
use crate::net::TransportKind;
|
||||
use crate::proto::ControlMessage;
|
||||
use crate::proto::handshake::Role;
|
||||
|
||||
/// Something that happened inside the agent.
|
||||
#[derive(Debug, Clone)]
|
||||
#[non_exhaustive]
|
||||
pub enum Event {
|
||||
/// A network was activated locally.
|
||||
NetworkActivated {
|
||||
/// The network.
|
||||
network: NetworkId,
|
||||
},
|
||||
/// A network was deactivated locally.
|
||||
///
|
||||
/// This is a local deactivation only. It is not a signed revocation of
|
||||
/// membership, and it says nothing about the network's other participants.
|
||||
NetworkDeactivated {
|
||||
/// The network.
|
||||
network: NetworkId,
|
||||
},
|
||||
/// A peer completed the handshake and has an authenticated session.
|
||||
PeerConnected {
|
||||
/// The network the session belongs to.
|
||||
network: NetworkId,
|
||||
/// Authenticated peer endpoint id.
|
||||
peer: EndpointId,
|
||||
/// Which side this agent played.
|
||||
role: Role,
|
||||
/// How the connection currently reaches the peer.
|
||||
transport: TransportKind,
|
||||
/// RTT of the selected path, if iroh reported one.
|
||||
rtt: Option<Duration>,
|
||||
},
|
||||
/// A peer's session ended.
|
||||
PeerDisconnected {
|
||||
/// The network the session belonged to.
|
||||
network: NetworkId,
|
||||
/// The peer.
|
||||
peer: EndpointId,
|
||||
/// Why the session ended. Free of secrets.
|
||||
reason: String,
|
||||
},
|
||||
/// A control message arrived on an authenticated session.
|
||||
MessageReceived {
|
||||
/// The network.
|
||||
network: NetworkId,
|
||||
/// The peer that sent it.
|
||||
peer: EndpointId,
|
||||
/// The message.
|
||||
message: ControlMessage,
|
||||
},
|
||||
/// An outbound dial failed.
|
||||
///
|
||||
/// A dead candidate produces these and nothing else; other peers keep
|
||||
/// connecting normally.
|
||||
DialFailed {
|
||||
/// The network.
|
||||
network: NetworkId,
|
||||
/// The candidate that could not be reached.
|
||||
peer: EndpointId,
|
||||
/// Why it failed.
|
||||
reason: String,
|
||||
},
|
||||
/// A handshake was rejected.
|
||||
///
|
||||
/// The network is `None` when the failure happened before the peer's
|
||||
/// requested network could be resolved.
|
||||
HandshakeRejected {
|
||||
/// The network, when known.
|
||||
network: Option<NetworkId>,
|
||||
/// The peer, when known.
|
||||
peer: Option<EndpointId>,
|
||||
/// Why it was rejected.
|
||||
reason: String,
|
||||
},
|
||||
/// A message or session was rejected for violating the protocol.
|
||||
ProtocolViolation {
|
||||
/// The network, when known.
|
||||
network: Option<NetworkId>,
|
||||
/// The peer, when known.
|
||||
peer: Option<EndpointId>,
|
||||
/// What was wrong.
|
||||
reason: String,
|
||||
},
|
||||
/// The disposable cache was discarded and recreated at startup.
|
||||
CacheReset {
|
||||
/// Why it was discarded. Free of secrets.
|
||||
reason: String,
|
||||
},
|
||||
/// An IP plugin reported an error. Never fatal.
|
||||
PluginError {
|
||||
/// The network the call was scoped to.
|
||||
network: NetworkId,
|
||||
/// Plugin protocol id.
|
||||
protocol: String,
|
||||
/// The reported error.
|
||||
reason: String,
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,611 @@
|
||||
//! The agent runtime.
|
||||
//!
|
||||
//! An [`Agent`] owns one persistent identity, one iroh endpoint, one state
|
||||
//! directory and any number of networks. It is started explicitly with
|
||||
//! [`Agent::spawn`] and stopped explicitly with [`Agent::shutdown`]; it starts
|
||||
//! no runtime of its own, installs no global logger, handles no signals and
|
||||
//! never calls `process::exit`. Several agents can therefore run side by side in
|
||||
//! one process, which is exactly what the integration tests do.
|
||||
//!
|
||||
//! # Local readiness
|
||||
//!
|
||||
//! [`Agent::spawn`] returns as soon as the local agent is ready. It never waits
|
||||
//! for other participants to appear or for a relay to become reachable.
|
||||
//!
|
||||
//! # Failure containment
|
||||
//!
|
||||
//! A bad signature, a wrong secret, a malformed frame or an unknown version
|
||||
//! rejects that message or that session. It never stops another network and
|
||||
//! never stops the agent. There is no global, irreversible error flag.
|
||||
|
||||
mod events;
|
||||
mod network;
|
||||
mod session;
|
||||
mod shutdown;
|
||||
mod status;
|
||||
|
||||
pub use events::Event;
|
||||
pub use status::{
|
||||
AgentStatus, CandidateStatus, NetworkMetrics, NetworkState, NetworkStatus, PeerStatus,
|
||||
};
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Weak};
|
||||
|
||||
use iroh::{EndpointAddr, EndpointId};
|
||||
use tokio::sync::{RwLock, broadcast, mpsc, oneshot};
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::config::{AgentConfig, Limits};
|
||||
use crate::error::{Error, Result};
|
||||
use crate::identity::{DeviceIdentity, NetworkId, NetworkKeys, NetworkName, NetworkSecret};
|
||||
use crate::net::EndpointAdapter;
|
||||
use crate::proto::handshake;
|
||||
use crate::proto::message::ControlMessage;
|
||||
use crate::storage::{CacheOutcome, Storage};
|
||||
|
||||
use network::{InboundSession, NetCommand, NetworkHandle, RuntimeParams};
|
||||
use shutdown::Shutdown;
|
||||
|
||||
/// Summary of a configured network, whether or not it is running.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ConfiguredNetwork {
|
||||
/// Public network identifier.
|
||||
pub network_id: NetworkId,
|
||||
/// Network name.
|
||||
pub name: NetworkName,
|
||||
/// Whether it is activated automatically at startup.
|
||||
pub auto_start: bool,
|
||||
/// Whether it is currently running.
|
||||
pub active: bool,
|
||||
}
|
||||
|
||||
/// A running agent.
|
||||
///
|
||||
/// Cloning gives another handle to the same agent.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Agent {
|
||||
inner: Arc<Inner>,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct Inner {
|
||||
config: AgentConfig,
|
||||
limits: Arc<Limits>,
|
||||
storage: Storage,
|
||||
identity: DeviceIdentity,
|
||||
adapter: EndpointAdapter,
|
||||
hostname: String,
|
||||
events: broadcast::Sender<Event>,
|
||||
networks: RwLock<HashMap<NetworkId, NetworkHandle>>,
|
||||
shutdown: Shutdown,
|
||||
accept_task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
}
|
||||
|
||||
impl Drop for Inner {
|
||||
fn drop(&mut self) {
|
||||
// Nothing here awaits; this is only a safety net for a handle that was
|
||||
// dropped without an explicit shutdown.
|
||||
self.shutdown.trigger();
|
||||
if let Ok(mut guard) = self.accept_task.lock()
|
||||
&& let Some(task) = guard.take()
|
||||
{
|
||||
task.abort();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Agent {
|
||||
/// Starts an agent.
|
||||
///
|
||||
/// Opens the state store (taking its ownership lock), restores the
|
||||
/// persistent device identity, binds the iroh endpoint and activates every
|
||||
/// configured network whose auto-start flag is set.
|
||||
pub async fn spawn(config: AgentConfig) -> Result<Self> {
|
||||
let storage = Storage::open(&config.paths)?;
|
||||
let identity = storage.device_identity().await?;
|
||||
let adapter = EndpointAdapter::bind(&config, &identity).await?;
|
||||
|
||||
let hostname = resolve_hostname(&config, &storage, identity.endpoint_id())?;
|
||||
storage.set_hostname(hostname.clone()).await?;
|
||||
|
||||
let (events, _) = broadcast::channel(config.limits.event_buffer);
|
||||
let limits = Arc::new(config.limits.clone());
|
||||
|
||||
let inner = Arc::new(Inner {
|
||||
limits,
|
||||
storage,
|
||||
identity,
|
||||
adapter,
|
||||
hostname,
|
||||
events,
|
||||
networks: RwLock::new(HashMap::new()),
|
||||
shutdown: Shutdown::new(),
|
||||
accept_task: std::sync::Mutex::new(None),
|
||||
config,
|
||||
});
|
||||
|
||||
if let CacheOutcome::Reset(reason) = inner.storage.cache_outcome().clone() {
|
||||
let _ = inner.events.send(Event::CacheReset { reason });
|
||||
}
|
||||
|
||||
let accept = tokio::spawn(accept_loop(Arc::downgrade(&inner)));
|
||||
if let Ok(mut guard) = inner.accept_task.lock() {
|
||||
*guard = Some(accept);
|
||||
}
|
||||
|
||||
let agent = Self { inner };
|
||||
|
||||
for stored in agent.inner.storage.list_networks().await? {
|
||||
if stored.auto_start {
|
||||
let keys = NetworkKeys::derive(&stored.name, &stored.secret);
|
||||
agent.activate_with_keys(keys).await?;
|
||||
}
|
||||
}
|
||||
|
||||
Ok(agent)
|
||||
}
|
||||
|
||||
/// This device's persistent endpoint id.
|
||||
pub fn endpoint_id(&self) -> EndpointId {
|
||||
self.inner.identity.endpoint_id()
|
||||
}
|
||||
|
||||
/// This endpoint's dialable address as iroh currently reports it.
|
||||
pub fn endpoint_addr(&self) -> EndpointAddr {
|
||||
self.inner.adapter.addr()
|
||||
}
|
||||
|
||||
/// An address containing only the locally bound sockets.
|
||||
///
|
||||
/// Handy when relays and address lookup are disabled and peers must be
|
||||
/// handed literal addresses, as in the test suite.
|
||||
pub fn local_addr(&self) -> EndpointAddr {
|
||||
self.inner.adapter.loopback_addr()
|
||||
}
|
||||
|
||||
/// The hostname announced to peers.
|
||||
pub fn hostname(&self) -> &str {
|
||||
&self.inner.hostname
|
||||
}
|
||||
|
||||
/// The underlying iroh endpoint, for callers that need more detail.
|
||||
pub fn endpoint(&self) -> &iroh::Endpoint {
|
||||
self.inner.adapter.endpoint()
|
||||
}
|
||||
|
||||
/// Subscribes to agent events.
|
||||
///
|
||||
/// The channel is bounded; a subscriber that falls behind is lagged rather
|
||||
/// than allowed to stall the runtime.
|
||||
pub fn subscribe(&self) -> broadcast::Receiver<Event> {
|
||||
self.inner.events.subscribe()
|
||||
}
|
||||
|
||||
/// Adds a network to the persistent configuration and activates it.
|
||||
///
|
||||
/// The same `(name, secret)` always produces the same [`NetworkId`], on
|
||||
/// every device.
|
||||
pub async fn join_network(
|
||||
&self,
|
||||
name: &NetworkName,
|
||||
secret: &NetworkSecret,
|
||||
) -> Result<NetworkId> {
|
||||
let keys = NetworkKeys::derive(name, secret);
|
||||
let network_id = keys.network_id();
|
||||
self.inner
|
||||
.storage
|
||||
.upsert_network(network_id, name.clone(), secret.clone(), true)
|
||||
.await?;
|
||||
self.activate_with_keys(keys).await?;
|
||||
Ok(network_id)
|
||||
}
|
||||
|
||||
/// Activates a configured network that is currently inactive.
|
||||
pub async fn activate_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
let stored = self
|
||||
.inner
|
||||
.storage
|
||||
.list_networks()
|
||||
.await?
|
||||
.into_iter()
|
||||
.find(|stored| stored.network_id == network_id)
|
||||
.ok_or(Error::NetworkUnknown(network_id))?;
|
||||
let keys = NetworkKeys::derive(&stored.name, &stored.secret);
|
||||
self.inner.storage.set_auto_start(network_id, true).await?;
|
||||
self.activate_with_keys(keys).await
|
||||
}
|
||||
|
||||
async fn activate_with_keys(&self, keys: NetworkKeys) -> Result<()> {
|
||||
if self.inner.shutdown.is_triggered() {
|
||||
return Err(Error::Stopped);
|
||||
}
|
||||
let network_id = keys.network_id();
|
||||
let mut networks = self.inner.networks.write().await;
|
||||
if networks.contains_key(&network_id) {
|
||||
return Err(Error::NetworkAlreadyActive(network_id));
|
||||
}
|
||||
|
||||
let handle = network::spawn(RuntimeParams {
|
||||
keys,
|
||||
adapter: self.inner.adapter.clone(),
|
||||
storage: self.inner.storage.clone(),
|
||||
events: self.inner.events.clone(),
|
||||
limits: Arc::clone(&self.inner.limits),
|
||||
reconnect: self.inner.config.reconnect.clone(),
|
||||
discovery: self.inner.config.discovery.clone(),
|
||||
discovery_interval: self.inner.config.discovery_interval,
|
||||
plugins: self.inner.config.plugins.clone(),
|
||||
hostname: self.inner.hostname.clone(),
|
||||
});
|
||||
networks.insert(network_id, handle);
|
||||
drop(networks);
|
||||
|
||||
let _ = self.inner.events.send(Event::NetworkActivated {
|
||||
network: network_id,
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Deactivates a running network, leaving its configuration in place.
|
||||
///
|
||||
/// This is a local action. It is not a signed revocation of membership and
|
||||
/// it does not remove this agent from anyone else's view of the network.
|
||||
/// Other networks keep running.
|
||||
pub async fn deactivate_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
let handle = {
|
||||
let mut networks = self.inner.networks.write().await;
|
||||
networks
|
||||
.remove(&network_id)
|
||||
.ok_or(Error::NetworkNotActive(network_id))?
|
||||
};
|
||||
handle.stop().await;
|
||||
for plugin in &self.inner.config.plugins {
|
||||
plugin.on_network_deactivated(network_id);
|
||||
}
|
||||
self.inner.storage.set_auto_start(network_id, false).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Deactivates a network if it is running and removes it from the state
|
||||
/// store together with its cached hints.
|
||||
pub async fn forget_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
if self.is_active(network_id).await {
|
||||
self.deactivate_network(network_id).await?;
|
||||
}
|
||||
self.inner.storage.remove_network(network_id).await
|
||||
}
|
||||
|
||||
/// Whether a network is currently running.
|
||||
pub async fn is_active(&self, network_id: NetworkId) -> bool {
|
||||
self.inner.networks.read().await.contains_key(&network_id)
|
||||
}
|
||||
|
||||
/// Lists configured networks and whether each is running.
|
||||
pub async fn list_networks(&self) -> Result<Vec<ConfiguredNetwork>> {
|
||||
let active: Vec<NetworkId> = self.inner.networks.read().await.keys().copied().collect();
|
||||
Ok(self
|
||||
.inner
|
||||
.storage
|
||||
.list_networks()
|
||||
.await?
|
||||
.into_iter()
|
||||
.map(|stored| ConfiguredNetwork {
|
||||
network_id: stored.network_id,
|
||||
name: stored.name,
|
||||
auto_start: stored.auto_start,
|
||||
active: active.contains(&stored.network_id),
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
/// Sends a control message to one authenticated peer in one network.
|
||||
///
|
||||
/// Fails if that network is not active or if there is no authenticated
|
||||
/// session with that peer *in that network*. Being authenticated in network
|
||||
/// A never grants the right to send into network B.
|
||||
pub async fn send(
|
||||
&self,
|
||||
network_id: NetworkId,
|
||||
peer: EndpointId,
|
||||
message: ControlMessage,
|
||||
) -> Result<()> {
|
||||
let (reply_tx, reply_rx) = oneshot::channel();
|
||||
self.command(
|
||||
network_id,
|
||||
NetCommand::Send {
|
||||
peer,
|
||||
message,
|
||||
reply: reply_tx,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
reply_rx.await.map_err(|_| Error::Stopped)?
|
||||
}
|
||||
|
||||
/// Sends a control message to every authenticated peer in a network.
|
||||
///
|
||||
/// Returns how many sessions accepted it into their outbound queue.
|
||||
pub async fn broadcast(&self, network_id: NetworkId, message: ControlMessage) -> Result<usize> {
|
||||
let (reply_tx, reply_rx) = oneshot::channel();
|
||||
self.command(
|
||||
network_id,
|
||||
NetCommand::Broadcast {
|
||||
message,
|
||||
reply: reply_tx,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
reply_rx.await.map_err(|_| Error::Stopped)
|
||||
}
|
||||
|
||||
/// Status of one running network.
|
||||
pub async fn network_status(&self, network_id: NetworkId) -> Result<NetworkStatus> {
|
||||
let (reply_tx, reply_rx) = oneshot::channel();
|
||||
self.command(network_id, NetCommand::Status { reply: reply_tx })
|
||||
.await?;
|
||||
reply_rx
|
||||
.await
|
||||
.map(|boxed| *boxed)
|
||||
.map_err(|_| Error::Stopped)
|
||||
}
|
||||
|
||||
/// Status of the whole agent, including configured but inactive networks.
|
||||
pub async fn status(&self) -> Result<AgentStatus> {
|
||||
let endpoint = self.inner.adapter.snapshot();
|
||||
let active: Vec<NetworkId> = self.inner.networks.read().await.keys().copied().collect();
|
||||
|
||||
let mut networks = Vec::new();
|
||||
for stored in self.inner.storage.list_networks().await? {
|
||||
if active.contains(&stored.network_id)
|
||||
&& let Ok(status) = self.network_status(stored.network_id).await
|
||||
{
|
||||
networks.push(status);
|
||||
continue;
|
||||
}
|
||||
let keys = NetworkKeys::derive(&stored.name, &stored.secret);
|
||||
networks.push(NetworkStatus {
|
||||
descriptor: keys.descriptor(),
|
||||
name: stored.name,
|
||||
network_id: stored.network_id,
|
||||
state: NetworkState::Inactive,
|
||||
peers: Vec::new(),
|
||||
candidates: Vec::new(),
|
||||
metrics: NetworkMetrics::default(),
|
||||
});
|
||||
}
|
||||
|
||||
Ok(AgentStatus {
|
||||
endpoint_id: endpoint.endpoint_id,
|
||||
hostname: self.inner.hostname.clone(),
|
||||
bound_sockets: endpoint.bound_sockets,
|
||||
observed_addrs: endpoint.observed_addrs,
|
||||
endpoint_addr: self.inner.adapter.addr(),
|
||||
cache_outcome: self.inner.storage.cache_outcome().clone(),
|
||||
cache_healthy: self.inner.storage.cache_healthy(),
|
||||
networks,
|
||||
})
|
||||
}
|
||||
|
||||
/// Asks one network to re-run discovery and re-evaluate dials right now.
|
||||
///
|
||||
/// Call this when the host's network environment changed. Platform wake-up
|
||||
/// notifications can be wired to it later.
|
||||
pub async fn recheck_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
self.command(network_id, NetCommand::Recheck).await
|
||||
}
|
||||
|
||||
/// Asks every running network to re-run discovery right now.
|
||||
pub async fn recheck(&self) {
|
||||
let senders: Vec<mpsc::Sender<NetCommand>> = self
|
||||
.inner
|
||||
.networks
|
||||
.read()
|
||||
.await
|
||||
.values()
|
||||
.map(|handle| handle.commands.clone())
|
||||
.collect();
|
||||
for sender in senders {
|
||||
let _ = sender.send(NetCommand::Recheck).await;
|
||||
}
|
||||
}
|
||||
|
||||
/// Stops every network, the accept loop and the endpoint.
|
||||
///
|
||||
/// After this returns, the state directory can be opened by another agent.
|
||||
pub async fn shutdown(&self) {
|
||||
self.inner.shutdown.trigger();
|
||||
|
||||
let handles: Vec<NetworkHandle> = {
|
||||
let mut networks = self.inner.networks.write().await;
|
||||
networks.drain().map(|(_, handle)| handle).collect()
|
||||
};
|
||||
for handle in handles {
|
||||
handle.stop().await;
|
||||
}
|
||||
|
||||
self.inner.adapter.close().await;
|
||||
|
||||
let task = self
|
||||
.inner
|
||||
.accept_task
|
||||
.lock()
|
||||
.ok()
|
||||
.and_then(|mut guard| guard.take());
|
||||
if let Some(task) = task {
|
||||
let _ = task.await;
|
||||
}
|
||||
|
||||
// Release the directory so another instance can claim it right away.
|
||||
self.inner.storage.release_ownership_lock();
|
||||
}
|
||||
|
||||
async fn command(&self, network_id: NetworkId, command: NetCommand) -> Result<()> {
|
||||
let sender = {
|
||||
let networks = self.inner.networks.read().await;
|
||||
networks
|
||||
.get(&network_id)
|
||||
.map(|handle| handle.commands.clone())
|
||||
.ok_or(Error::NetworkNotActive(network_id))?
|
||||
};
|
||||
sender.send(command).await.map_err(|_| Error::Stopped)
|
||||
}
|
||||
}
|
||||
|
||||
/// Accepts inbound connections and routes authenticated sessions to networks.
|
||||
///
|
||||
/// Holds only a weak reference, so dropping every [`Agent`] handle lets the
|
||||
/// runtime state be released and this loop exit.
|
||||
async fn accept_loop(weak: Weak<Inner>) {
|
||||
let Some(inner) = weak.upgrade() else {
|
||||
return;
|
||||
};
|
||||
let endpoint = inner.adapter.endpoint().clone();
|
||||
let shutdown = inner.shutdown.clone();
|
||||
let permits = Arc::new(tokio::sync::Semaphore::new(
|
||||
inner.limits.max_inbound_handshakes,
|
||||
));
|
||||
drop(inner);
|
||||
|
||||
loop {
|
||||
let incoming = tokio::select! {
|
||||
biased;
|
||||
_ = shutdown.wait() => break,
|
||||
incoming = endpoint.accept() => match incoming {
|
||||
Some(incoming) => incoming,
|
||||
None => break,
|
||||
},
|
||||
};
|
||||
|
||||
let Some(inner) = weak.upgrade() else {
|
||||
break;
|
||||
};
|
||||
let Ok(permit) = Arc::clone(&permits).try_acquire_owned() else {
|
||||
// Too many handshakes in flight: refuse cheaply instead of queueing.
|
||||
incoming.refuse();
|
||||
continue;
|
||||
};
|
||||
|
||||
tokio::spawn(async move {
|
||||
let _permit = permit;
|
||||
handle_incoming(inner, incoming).await;
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async fn handle_incoming(inner: Arc<Inner>, incoming: iroh::endpoint::Incoming) {
|
||||
let connecting = match incoming.accept() {
|
||||
Ok(connecting) => connecting,
|
||||
Err(err) => {
|
||||
tracing::debug!(%err, "inbound connection could not be accepted");
|
||||
return;
|
||||
}
|
||||
};
|
||||
let conn = match connecting.await {
|
||||
Ok(conn) => conn,
|
||||
Err(err) => {
|
||||
tracing::debug!(%err, "inbound connection failed during setup");
|
||||
return;
|
||||
}
|
||||
};
|
||||
let peer = conn.remote_id();
|
||||
|
||||
let (mut send, mut recv) = match conn.accept_bi().await {
|
||||
Ok(streams) => streams,
|
||||
Err(err) => {
|
||||
tracing::debug!(%err, "peer did not open a control stream");
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// Snapshot the active networks so the handshake's lookup stays synchronous.
|
||||
let known: HashMap<NetworkId, NetworkKeys> = inner
|
||||
.networks
|
||||
.read()
|
||||
.await
|
||||
.iter()
|
||||
.map(|(id, handle)| (*id, handle.keys.clone()))
|
||||
.collect();
|
||||
|
||||
let local_id = inner.identity.endpoint_id();
|
||||
let outcome = handshake::respond(
|
||||
&conn,
|
||||
&mut send,
|
||||
&mut recv,
|
||||
local_id,
|
||||
&inner.limits,
|
||||
|network_id| known.get(&network_id).cloned(),
|
||||
)
|
||||
.await;
|
||||
|
||||
let outcome = match outcome {
|
||||
Ok(outcome) => outcome,
|
||||
Err(err) => {
|
||||
// The network id is deliberately not reported here: before a
|
||||
// successful handshake the peer's claim is unverified.
|
||||
let network = None;
|
||||
conn.close(2u32.into(), b"handshake rejected");
|
||||
let _ = inner.events.send(Event::HandshakeRejected {
|
||||
network,
|
||||
peer: Some(peer),
|
||||
reason: err.to_string(),
|
||||
});
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let sender = {
|
||||
let networks = inner.networks.read().await;
|
||||
networks
|
||||
.get(&outcome.network_id)
|
||||
.map(|handle| handle.commands.clone())
|
||||
};
|
||||
let Some(sender) = sender else {
|
||||
// The network was deactivated while the handshake ran.
|
||||
conn.close(3u32.into(), b"network no longer active");
|
||||
return;
|
||||
};
|
||||
|
||||
let inbound = InboundSession {
|
||||
conn,
|
||||
send,
|
||||
recv,
|
||||
outcome,
|
||||
};
|
||||
if sender
|
||||
.send(NetCommand::Inbound(Box::new(inbound)))
|
||||
.await
|
||||
.is_err()
|
||||
{
|
||||
tracing::debug!("network runtime stopped before the session could be installed");
|
||||
}
|
||||
}
|
||||
|
||||
/// Picks the hostname to announce.
|
||||
///
|
||||
/// Order: explicit configuration, then what the state store already holds, then
|
||||
/// a best-effort environment variable, then a stable fallback derived from the
|
||||
/// endpoint id. The library does not shell out to discover a hostname.
|
||||
fn resolve_hostname(
|
||||
config: &AgentConfig,
|
||||
storage: &Storage,
|
||||
endpoint_id: EndpointId,
|
||||
) -> Result<String> {
|
||||
if let Some(hostname) = &config.hostname {
|
||||
return Ok(hostname.clone());
|
||||
}
|
||||
if let Some(stored) = storage.hostname_blocking()?
|
||||
&& !stored.is_empty()
|
||||
{
|
||||
return Ok(stored);
|
||||
}
|
||||
for key in ["HOSTNAME", "COMPUTERNAME"] {
|
||||
if let Ok(value) = std::env::var(key) {
|
||||
let value = value.trim();
|
||||
if !value.is_empty() {
|
||||
return Ok(value.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(format!("tsunagi-{}", endpoint_id.fmt_short()))
|
||||
}
|
||||
@@ -0,0 +1,856 @@
|
||||
//! The per-network runtime.
|
||||
//!
|
||||
//! One of these runs for every locally active network. It owns that network's
|
||||
//! sessions, its dial loop and its counters. Everything it touches carries an
|
||||
//! explicit [`NetworkId`], so deactivating or breaking one network cannot
|
||||
//! disturb another and cannot stop the agent.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use iroh::EndpointId;
|
||||
use iroh::endpoint::{Connection, RecvStream, SendStream};
|
||||
use tokio::sync::{broadcast, mpsc, oneshot};
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::config::{Limits, ReconnectPolicy};
|
||||
use crate::dataplane::{PluginCapability, SharedPlugin};
|
||||
use crate::discovery::{Candidate, CandidateSource, NetworkDiscovery};
|
||||
use crate::error::{Error, Result};
|
||||
use crate::identity::{NetworkId, NetworkKeys};
|
||||
use crate::net::{EndpointAdapter, PathAddr, snapshot_connection};
|
||||
use crate::proto::handshake::{self, HandshakeOutcome, Role};
|
||||
use crate::proto::message::{Announcement, ControlMessage, Envelope, encode, kind};
|
||||
use crate::storage::Storage;
|
||||
|
||||
use super::events::Event;
|
||||
use super::session::{self, Session, SessionEvent};
|
||||
use super::shutdown::Shutdown;
|
||||
use super::status::{CandidateStatus, NetworkMetrics, NetworkState, NetworkStatus, PeerStatus};
|
||||
|
||||
/// An inbound connection that already passed the handshake.
|
||||
#[derive(Debug)]
|
||||
pub(crate) struct InboundSession {
|
||||
pub(crate) conn: Connection,
|
||||
pub(crate) send: SendStream,
|
||||
pub(crate) recv: RecvStream,
|
||||
pub(crate) outcome: HandshakeOutcome,
|
||||
}
|
||||
|
||||
/// Commands accepted by a network runtime.
|
||||
pub(crate) enum NetCommand {
|
||||
Inbound(Box<InboundSession>),
|
||||
Send {
|
||||
peer: EndpointId,
|
||||
message: ControlMessage,
|
||||
reply: oneshot::Sender<Result<()>>,
|
||||
},
|
||||
Broadcast {
|
||||
message: ControlMessage,
|
||||
reply: oneshot::Sender<usize>,
|
||||
},
|
||||
Status {
|
||||
reply: oneshot::Sender<Box<NetworkStatus>>,
|
||||
},
|
||||
Recheck,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for NetCommand {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
NetCommand::Inbound(_) => f.write_str("Inbound"),
|
||||
NetCommand::Send { peer, message, .. } => {
|
||||
write!(f, "Send({}, {})", peer.fmt_short(), kind(message))
|
||||
}
|
||||
NetCommand::Broadcast { message, .. } => write!(f, "Broadcast({})", kind(message)),
|
||||
NetCommand::Status { .. } => f.write_str("Status"),
|
||||
NetCommand::Recheck => f.write_str("Recheck"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Handle to a running network runtime.
|
||||
#[derive(Debug)]
|
||||
pub(crate) struct NetworkHandle {
|
||||
pub(crate) keys: NetworkKeys,
|
||||
pub(crate) commands: mpsc::Sender<NetCommand>,
|
||||
shutdown: Shutdown,
|
||||
task: JoinHandle<()>,
|
||||
}
|
||||
|
||||
impl NetworkHandle {
|
||||
/// Stops the runtime and waits for its task to finish.
|
||||
pub(crate) async fn stop(self) {
|
||||
self.shutdown.trigger();
|
||||
let _ = self.task.await;
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything a network runtime needs to run.
|
||||
pub(crate) struct RuntimeParams {
|
||||
pub(crate) keys: NetworkKeys,
|
||||
pub(crate) adapter: EndpointAdapter,
|
||||
pub(crate) storage: Storage,
|
||||
pub(crate) events: broadcast::Sender<Event>,
|
||||
pub(crate) limits: Arc<Limits>,
|
||||
pub(crate) reconnect: ReconnectPolicy,
|
||||
pub(crate) discovery: Option<Arc<dyn NetworkDiscovery>>,
|
||||
pub(crate) discovery_interval: Duration,
|
||||
pub(crate) plugins: Vec<SharedPlugin>,
|
||||
pub(crate) hostname: String,
|
||||
}
|
||||
|
||||
/// Outcome of one outbound dial.
|
||||
enum DialOutcome {
|
||||
Established(Box<InboundSession>),
|
||||
Failed {
|
||||
peer: EndpointId,
|
||||
reason: String,
|
||||
during_handshake: bool,
|
||||
},
|
||||
}
|
||||
|
||||
/// Backoff bookkeeping for one candidate.
|
||||
#[derive(Debug)]
|
||||
struct DialState {
|
||||
consecutive_failures: u32,
|
||||
next_attempt: Instant,
|
||||
in_flight: bool,
|
||||
source: CandidateSource,
|
||||
}
|
||||
|
||||
impl DialState {
|
||||
fn new(source: CandidateSource) -> Self {
|
||||
Self {
|
||||
consecutive_failures: 0,
|
||||
next_attempt: Instant::now(),
|
||||
in_flight: false,
|
||||
source,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Starts a network runtime.
|
||||
pub(crate) fn spawn(params: RuntimeParams) -> NetworkHandle {
|
||||
let keys = params.keys.clone();
|
||||
let shutdown = Shutdown::new();
|
||||
let (commands_tx, commands_rx) = mpsc::channel(64);
|
||||
|
||||
let runtime_shutdown = shutdown.clone();
|
||||
let task = tokio::spawn(async move {
|
||||
let mut runtime = Runtime::new(params, runtime_shutdown);
|
||||
runtime.run(commands_rx).await;
|
||||
});
|
||||
|
||||
NetworkHandle {
|
||||
keys,
|
||||
commands: commands_tx,
|
||||
shutdown,
|
||||
task,
|
||||
}
|
||||
}
|
||||
|
||||
struct Runtime {
|
||||
params: RuntimeParams,
|
||||
network_id: NetworkId,
|
||||
local_id: EndpointId,
|
||||
shutdown: Shutdown,
|
||||
sessions: HashMap<EndpointId, Session>,
|
||||
dial_states: HashMap<EndpointId, DialState>,
|
||||
candidate_addrs: HashMap<EndpointId, iroh::EndpointAddr>,
|
||||
metrics: NetworkMetrics,
|
||||
session_events_tx: mpsc::Sender<SessionEvent>,
|
||||
session_events_rx: mpsc::Receiver<SessionEvent>,
|
||||
dial_results_tx: mpsc::Sender<DialOutcome>,
|
||||
dial_results_rx: mpsc::Receiver<DialOutcome>,
|
||||
}
|
||||
|
||||
impl Runtime {
|
||||
fn new(params: RuntimeParams, shutdown: Shutdown) -> Self {
|
||||
let network_id = params.keys.network_id();
|
||||
let local_id = params.adapter.endpoint_id();
|
||||
let (session_events_tx, session_events_rx) = mpsc::channel(256);
|
||||
let (dial_results_tx, dial_results_rx) = mpsc::channel(64);
|
||||
Self {
|
||||
params,
|
||||
network_id,
|
||||
local_id,
|
||||
shutdown,
|
||||
sessions: HashMap::new(),
|
||||
dial_states: HashMap::new(),
|
||||
candidate_addrs: HashMap::new(),
|
||||
metrics: NetworkMetrics::default(),
|
||||
session_events_tx,
|
||||
session_events_rx,
|
||||
dial_results_tx,
|
||||
dial_results_rx,
|
||||
}
|
||||
}
|
||||
|
||||
fn emit(&self, event: Event) {
|
||||
// A broadcast with no subscribers is not an error.
|
||||
let _ = self.params.events.send(event);
|
||||
}
|
||||
|
||||
async fn run(&mut self, mut commands: mpsc::Receiver<NetCommand>) {
|
||||
let mut ticker = tokio::time::interval(self.params.discovery_interval);
|
||||
ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
biased;
|
||||
_ = self.shutdown.wait() => break,
|
||||
command = commands.recv() => match command {
|
||||
Some(command) => self.handle_command(command).await,
|
||||
None => break,
|
||||
},
|
||||
event = self.session_events_rx.recv() => {
|
||||
if let Some(event) = event {
|
||||
self.handle_session_event(event).await;
|
||||
}
|
||||
}
|
||||
result = self.dial_results_rx.recv() => {
|
||||
if let Some(result) = result {
|
||||
self.handle_dial_result(result).await;
|
||||
}
|
||||
}
|
||||
_ = ticker.tick() => self.discovery_round().await,
|
||||
}
|
||||
}
|
||||
|
||||
self.teardown().await;
|
||||
}
|
||||
|
||||
async fn teardown(&mut self) {
|
||||
// Stop accepting session events first: nothing is going to act on them
|
||||
// any more, and a sender blocked on a full queue would stall shutdown.
|
||||
self.session_events_rx.close();
|
||||
if let Some(discovery) = &self.params.discovery {
|
||||
let _ = discovery
|
||||
.unpublish(self.params.keys.discovery_key(), self.local_id)
|
||||
.await;
|
||||
}
|
||||
let peers: Vec<EndpointId> = self.sessions.keys().copied().collect();
|
||||
for peer in peers {
|
||||
if let Some(session) = self.sessions.remove(&peer) {
|
||||
session.stop().await;
|
||||
}
|
||||
}
|
||||
self.emit(Event::NetworkDeactivated {
|
||||
network: self.network_id,
|
||||
});
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- commands
|
||||
|
||||
async fn handle_command(&mut self, command: NetCommand) {
|
||||
match command {
|
||||
NetCommand::Inbound(inbound) => {
|
||||
self.install_session(*inbound).await;
|
||||
}
|
||||
NetCommand::Send {
|
||||
peer,
|
||||
message,
|
||||
reply,
|
||||
} => {
|
||||
let _ = reply.send(self.send_to(peer, message));
|
||||
}
|
||||
NetCommand::Broadcast { message, reply } => {
|
||||
let peers: Vec<EndpointId> = self.sessions.keys().copied().collect();
|
||||
let mut delivered = 0;
|
||||
for peer in peers {
|
||||
if self.send_to(peer, message.clone()).is_ok() {
|
||||
delivered += 1;
|
||||
}
|
||||
}
|
||||
let _ = reply.send(delivered);
|
||||
}
|
||||
NetCommand::Status { reply } => {
|
||||
let _ = reply.send(Box::new(self.status()));
|
||||
}
|
||||
NetCommand::Recheck => self.discovery_round().await,
|
||||
}
|
||||
}
|
||||
|
||||
/// Queues a message without blocking the runtime loop.
|
||||
///
|
||||
/// The envelope is encoded here so that the exact number of control bytes
|
||||
/// handed to the transport is known and can be reported honestly.
|
||||
///
|
||||
/// A full queue is backpressure: the send fails rather than stalling every
|
||||
/// other peer in this network.
|
||||
fn send_to(&mut self, peer: EndpointId, message: ControlMessage) -> Result<()> {
|
||||
let network = self.network_id;
|
||||
let envelope = Envelope {
|
||||
network_id: *network.as_bytes(),
|
||||
message,
|
||||
};
|
||||
let encoded = encode(&envelope)?;
|
||||
let bytes = encoded.len() as u64;
|
||||
|
||||
let session = self.sessions.get_mut(&peer).ok_or(Error::NoSuchPeer {
|
||||
network,
|
||||
peer: peer.fmt_short().to_string(),
|
||||
})?;
|
||||
|
||||
match session.outbound.try_send(encoded) {
|
||||
Ok(()) => {
|
||||
session.messages_sent += 1;
|
||||
session.bytes_sent += bytes;
|
||||
self.metrics.control_messages_sent += 1;
|
||||
self.metrics.control_bytes_sent += bytes;
|
||||
Ok(())
|
||||
}
|
||||
Err(mpsc::error::TrySendError::Full(_)) => Err(Error::Storage(format!(
|
||||
"outbound queue for peer {} is full",
|
||||
peer.fmt_short()
|
||||
))),
|
||||
Err(mpsc::error::TrySendError::Closed(_)) => Err(Error::NoSuchPeer {
|
||||
network,
|
||||
peer: peer.fmt_short().to_string(),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------- discovery
|
||||
|
||||
async fn discovery_round(&mut self) {
|
||||
if self.shutdown.is_triggered() {
|
||||
return;
|
||||
}
|
||||
|
||||
let mut candidates: Vec<Candidate> = Vec::new();
|
||||
|
||||
if let Some(discovery) = self.params.discovery.clone() {
|
||||
let key = self.params.keys.discovery_key();
|
||||
// Publishing every round keeps a restarted agent reachable at its
|
||||
// new local port without any special case.
|
||||
if let Err(err) = discovery.publish(key, self.params.adapter.addr()).await {
|
||||
tracing::debug!(%err, "discovery publish failed");
|
||||
}
|
||||
if let Err(err) = discovery
|
||||
.publish(key, self.params.adapter.loopback_addr())
|
||||
.await
|
||||
{
|
||||
tracing::debug!(%err, "discovery publish of bound sockets failed");
|
||||
}
|
||||
match discovery.resolve(key).await {
|
||||
Ok(found) => candidates.extend(found),
|
||||
Err(err) => tracing::debug!(%err, "discovery resolve failed"),
|
||||
}
|
||||
}
|
||||
|
||||
// A stale or missing cache only changes which candidates we try first.
|
||||
// It never bypasses authentication.
|
||||
for hint in self.params.storage.hints_for_network(self.network_id).await {
|
||||
let Ok(endpoint_id) = EndpointId::from_bytes(&hint.endpoint_id) else {
|
||||
continue;
|
||||
};
|
||||
let Some(addr) = decode_hint(endpoint_id, &hint.addr) else {
|
||||
continue;
|
||||
};
|
||||
candidates.push(Candidate::new(addr, CandidateSource::Cache));
|
||||
}
|
||||
|
||||
for candidate in candidates {
|
||||
let peer = candidate.endpoint_id();
|
||||
if peer == self.local_id {
|
||||
continue;
|
||||
}
|
||||
self.candidate_addrs
|
||||
.entry(peer)
|
||||
.and_modify(|existing| merge_addr(existing, &candidate.addr))
|
||||
.or_insert_with(|| candidate.addr.clone());
|
||||
self.dial_states
|
||||
.entry(peer)
|
||||
.or_insert_with(|| DialState::new(candidate.source));
|
||||
}
|
||||
|
||||
self.start_dials();
|
||||
}
|
||||
|
||||
fn start_dials(&mut self) {
|
||||
let now = Instant::now();
|
||||
let in_flight = self
|
||||
.dial_states
|
||||
.values()
|
||||
.filter(|state| state.in_flight)
|
||||
.count();
|
||||
let mut budget = self
|
||||
.params
|
||||
.limits
|
||||
.max_concurrent_dials
|
||||
.saturating_sub(in_flight);
|
||||
if budget == 0 || self.sessions.len() >= self.params.limits.max_sessions_per_network {
|
||||
return;
|
||||
}
|
||||
|
||||
let ready: Vec<EndpointId> = self
|
||||
.dial_states
|
||||
.iter()
|
||||
.filter(|(peer, state)| {
|
||||
!state.in_flight && state.next_attempt <= now && !self.sessions.contains_key(*peer)
|
||||
})
|
||||
.map(|(peer, _)| *peer)
|
||||
.collect();
|
||||
|
||||
for peer in ready {
|
||||
if budget == 0 {
|
||||
break;
|
||||
}
|
||||
let Some(addr) = self.candidate_addrs.get(&peer).cloned() else {
|
||||
continue;
|
||||
};
|
||||
if let Some(state) = self.dial_states.get_mut(&peer) {
|
||||
state.in_flight = true;
|
||||
}
|
||||
budget -= 1;
|
||||
self.metrics.dial_attempts += 1;
|
||||
|
||||
let adapter = self.params.adapter.clone();
|
||||
let keys = self.params.keys.clone();
|
||||
let limits = Arc::clone(&self.params.limits);
|
||||
let results = self.dial_results_tx.clone();
|
||||
let local_id = self.local_id;
|
||||
let shutdown = self.shutdown.clone();
|
||||
|
||||
tokio::spawn(async move {
|
||||
let outcome = tokio::select! {
|
||||
biased;
|
||||
_ = shutdown.wait() => DialOutcome::Failed {
|
||||
peer,
|
||||
reason: "network deactivated".into(),
|
||||
during_handshake: false,
|
||||
},
|
||||
outcome = dial(adapter, addr, keys, limits, local_id, peer) => outcome,
|
||||
};
|
||||
let _ = results.send(outcome).await;
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async fn handle_dial_result(&mut self, result: DialOutcome) {
|
||||
match result {
|
||||
DialOutcome::Established(inbound) => {
|
||||
let peer = inbound.outcome.peer;
|
||||
if let Some(state) = self.dial_states.get_mut(&peer) {
|
||||
state.in_flight = false;
|
||||
state.consecutive_failures = 0;
|
||||
state.next_attempt = Instant::now();
|
||||
}
|
||||
self.install_session(*inbound).await;
|
||||
}
|
||||
DialOutcome::Failed {
|
||||
peer,
|
||||
reason,
|
||||
during_handshake,
|
||||
} => {
|
||||
self.metrics.dial_failures += 1;
|
||||
if during_handshake {
|
||||
self.metrics.handshake_failures += 1;
|
||||
}
|
||||
let give_up = {
|
||||
let policy = &self.params.reconnect;
|
||||
let state = self
|
||||
.dial_states
|
||||
.entry(peer)
|
||||
.or_insert_with(|| DialState::new(CandidateSource::Discovery));
|
||||
state.in_flight = false;
|
||||
state.consecutive_failures = state.consecutive_failures.saturating_add(1);
|
||||
let delay = policy.delay_for(state.consecutive_failures);
|
||||
state.next_attempt = Instant::now() + delay;
|
||||
policy
|
||||
.max_consecutive_failures
|
||||
.is_some_and(|max| state.consecutive_failures >= max)
|
||||
};
|
||||
if give_up {
|
||||
// Keep the backoff entry but push it far out; discovery
|
||||
// seeing the peer again resets it.
|
||||
if let Some(state) = self.dial_states.get_mut(&peer) {
|
||||
state.next_attempt = Instant::now() + self.params.reconnect.max_delay;
|
||||
}
|
||||
}
|
||||
self.emit(Event::DialFailed {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------- sessions
|
||||
|
||||
async fn install_session(&mut self, inbound: InboundSession) {
|
||||
let InboundSession {
|
||||
conn,
|
||||
send,
|
||||
recv,
|
||||
outcome,
|
||||
} = inbound;
|
||||
let peer = outcome.peer;
|
||||
|
||||
if self.sessions.len() >= self.params.limits.max_sessions_per_network
|
||||
&& !self.sessions.contains_key(&peer)
|
||||
{
|
||||
conn.close(1u32.into(), b"session limit reached");
|
||||
self.emit(Event::ProtocolViolation {
|
||||
network: Some(self.network_id),
|
||||
peer: Some(peer),
|
||||
reason: "session limit for this network reached".into(),
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// Two agents may dial each other at the same time. Both sides apply the
|
||||
// same deterministic rule, so they converge on the same session.
|
||||
if let Some(existing) = self.sessions.get(&peer) {
|
||||
let existing_initiator = initiator_of(existing.role, self.local_id, peer);
|
||||
let new_initiator = initiator_of(outcome.role, self.local_id, peer);
|
||||
if existing_initiator.as_bytes() <= new_initiator.as_bytes() {
|
||||
conn.close(0u32.into(), b"duplicate session");
|
||||
return;
|
||||
}
|
||||
if let Some(old) = self.sessions.remove(&peer) {
|
||||
old.abort();
|
||||
old.conn
|
||||
.close(0u32.into(), b"replaced by preferred session");
|
||||
}
|
||||
}
|
||||
|
||||
self.record_hints(&conn).await;
|
||||
|
||||
let session = session::spawn(
|
||||
self.network_id,
|
||||
peer,
|
||||
outcome.role,
|
||||
conn.clone(),
|
||||
send,
|
||||
recv,
|
||||
Arc::clone(&self.params.limits),
|
||||
self.session_events_tx.clone(),
|
||||
self.shutdown.clone(),
|
||||
);
|
||||
|
||||
let snapshot = snapshot_connection(&conn);
|
||||
self.sessions.insert(peer, session);
|
||||
self.metrics.sessions_established += 1;
|
||||
|
||||
// Announce ourselves straight away so the peer learns our hostname and
|
||||
// capabilities without another round of discovery.
|
||||
let announcement = ControlMessage::Announce(self.local_announcement());
|
||||
if let Err(err) = self.send_to(peer, announcement) {
|
||||
tracing::debug!(%err, "could not queue initial announcement");
|
||||
}
|
||||
|
||||
self.emit(Event::PeerConnected {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
role: outcome.role,
|
||||
transport: snapshot.transport,
|
||||
rtt: snapshot.rtt,
|
||||
});
|
||||
}
|
||||
|
||||
fn local_announcement(&mut self) -> Announcement {
|
||||
let mut capabilities = Vec::new();
|
||||
let mut errors = Vec::new();
|
||||
for plugin in &self.params.plugins {
|
||||
match plugin.local_capability(self.network_id) {
|
||||
Ok(Some(capability)) => capabilities.push(capability),
|
||||
Ok(None) => {}
|
||||
Err(err) => errors.push((plugin.protocol_id().to_string(), err.to_string())),
|
||||
}
|
||||
}
|
||||
for (protocol, reason) in errors {
|
||||
self.metrics.plugin_errors += 1;
|
||||
self.emit(Event::PluginError {
|
||||
network: self.network_id,
|
||||
protocol,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
capabilities.truncate(self.params.limits.max_capabilities);
|
||||
Announcement {
|
||||
hostname: self.params.hostname.clone(),
|
||||
capabilities,
|
||||
}
|
||||
}
|
||||
|
||||
async fn record_hints(&self, conn: &Connection) {
|
||||
let snapshot = snapshot_connection(conn);
|
||||
let peer_bytes = *snapshot.remote_id.as_bytes();
|
||||
for path in &snapshot.paths {
|
||||
if let Some(encoded) = encode_hint(&path.remote) {
|
||||
self.params
|
||||
.storage
|
||||
.record_hint(
|
||||
self.network_id,
|
||||
peer_bytes,
|
||||
encoded,
|
||||
self.params.limits.max_hints_per_peer,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn handle_session_event(&mut self, event: SessionEvent) {
|
||||
match event {
|
||||
SessionEvent::Message {
|
||||
session_id,
|
||||
peer,
|
||||
message,
|
||||
bytes,
|
||||
} => {
|
||||
let current = self.sessions.get(&peer).map(|session| session.id);
|
||||
if current != Some(session_id) {
|
||||
return;
|
||||
}
|
||||
self.metrics.control_messages_received += 1;
|
||||
self.metrics.control_bytes_received += bytes as u64;
|
||||
if let Some(session) = self.sessions.get_mut(&peer) {
|
||||
session.messages_received += 1;
|
||||
session.bytes_received += bytes as u64;
|
||||
}
|
||||
self.dispatch_message(peer, message);
|
||||
}
|
||||
SessionEvent::Violation {
|
||||
session_id,
|
||||
peer,
|
||||
error,
|
||||
} => {
|
||||
let current = self.sessions.get(&peer).map(|session| session.id);
|
||||
if current != Some(session_id) {
|
||||
return;
|
||||
}
|
||||
self.metrics.protocol_violations += 1;
|
||||
self.emit(Event::ProtocolViolation {
|
||||
network: Some(self.network_id),
|
||||
peer: Some(peer),
|
||||
reason: error.to_string(),
|
||||
});
|
||||
}
|
||||
SessionEvent::Closed {
|
||||
session_id,
|
||||
peer,
|
||||
reason,
|
||||
} => {
|
||||
let current = self.sessions.get(&peer).map(|session| session.id);
|
||||
if current != Some(session_id) {
|
||||
return;
|
||||
}
|
||||
if let Some(session) = self.sessions.remove(&peer) {
|
||||
session.abort();
|
||||
session.conn.close(0u32.into(), b"session ended");
|
||||
}
|
||||
self.metrics.disconnects += 1;
|
||||
for plugin in &self.params.plugins {
|
||||
plugin.on_peer_gone(self.network_id, peer);
|
||||
}
|
||||
// Retry promptly, then back off if it keeps failing.
|
||||
let policy = &self.params.reconnect;
|
||||
let state = self
|
||||
.dial_states
|
||||
.entry(peer)
|
||||
.or_insert_with(|| DialState::new(CandidateSource::Discovery));
|
||||
state.in_flight = false;
|
||||
state.next_attempt = Instant::now() + policy.initial_delay;
|
||||
self.emit(Event::PeerDisconnected {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn dispatch_message(&mut self, peer: EndpointId, message: ControlMessage) {
|
||||
match &message {
|
||||
ControlMessage::Announce(announcement) => {
|
||||
let capabilities = announcement.capabilities.clone();
|
||||
if let Some(session) = self.sessions.get_mut(&peer) {
|
||||
session.hostname = Some(announcement.hostname.clone());
|
||||
session.capabilities = capabilities.clone();
|
||||
}
|
||||
self.dispatch_capabilities(peer, &capabilities);
|
||||
}
|
||||
ControlMessage::Ping { seq, payload } => {
|
||||
let pong = ControlMessage::Pong {
|
||||
seq: *seq,
|
||||
payload: payload.clone(),
|
||||
};
|
||||
if let Err(err) = self.send_to(peer, pong) {
|
||||
tracing::debug!(%err, "could not queue pong");
|
||||
}
|
||||
}
|
||||
ControlMessage::Pong { .. } | ControlMessage::Bye { .. } => {}
|
||||
}
|
||||
|
||||
self.emit(Event::MessageReceived {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
message,
|
||||
});
|
||||
}
|
||||
|
||||
fn dispatch_capabilities(&mut self, peer: EndpointId, capabilities: &[PluginCapability]) {
|
||||
let mut errors = Vec::new();
|
||||
for capability in capabilities {
|
||||
for plugin in &self.params.plugins {
|
||||
if plugin.protocol_id() != capability.protocol {
|
||||
continue;
|
||||
}
|
||||
// The core hands the opaque payload over without interpreting it.
|
||||
if let Err(err) = plugin.on_peer_capability(self.network_id, peer, capability) {
|
||||
errors.push((plugin.protocol_id().to_string(), err.to_string()));
|
||||
}
|
||||
}
|
||||
}
|
||||
for (protocol, reason) in errors {
|
||||
self.metrics.plugin_errors += 1;
|
||||
self.emit(Event::PluginError {
|
||||
network: self.network_id,
|
||||
protocol,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------- status
|
||||
|
||||
fn status(&self) -> NetworkStatus {
|
||||
let mut peers: Vec<PeerStatus> = self
|
||||
.sessions
|
||||
.values()
|
||||
.map(|session| {
|
||||
let snapshot = snapshot_connection(&session.conn);
|
||||
PeerStatus {
|
||||
endpoint_id: session.peer,
|
||||
role: session.role,
|
||||
hostname: session.hostname.clone(),
|
||||
capabilities: session.capabilities.clone(),
|
||||
connected_for: session.established.elapsed(),
|
||||
paths: snapshot.paths,
|
||||
transport: snapshot.transport,
|
||||
rtt: snapshot.rtt,
|
||||
connection: snapshot.counters,
|
||||
control_messages_sent: session.messages_sent,
|
||||
control_messages_received: session.messages_received,
|
||||
control_bytes_sent: session.bytes_sent,
|
||||
control_bytes_received: session.bytes_received,
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
peers.sort_by(|a, b| a.endpoint_id.as_bytes().cmp(b.endpoint_id.as_bytes()));
|
||||
|
||||
let mut candidates: Vec<CandidateStatus> = self
|
||||
.dial_states
|
||||
.iter()
|
||||
.map(|(peer, state)| CandidateStatus {
|
||||
endpoint_id: *peer,
|
||||
source: state.source,
|
||||
consecutive_failures: state.consecutive_failures,
|
||||
})
|
||||
.collect();
|
||||
candidates.sort_by(|a, b| a.endpoint_id.as_bytes().cmp(b.endpoint_id.as_bytes()));
|
||||
|
||||
NetworkStatus {
|
||||
descriptor: self.params.keys.descriptor(),
|
||||
name: self.params.keys.name().clone(),
|
||||
network_id: self.network_id,
|
||||
state: NetworkState::Active,
|
||||
peers,
|
||||
candidates,
|
||||
metrics: self.metrics.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Which side dialled, given a role and the two identities.
|
||||
fn initiator_of(role: Role, local: EndpointId, peer: EndpointId) -> EndpointId {
|
||||
match role {
|
||||
Role::Initiator => local,
|
||||
Role::Responder => peer,
|
||||
}
|
||||
}
|
||||
|
||||
/// Performs one dial and handshake.
|
||||
async fn dial(
|
||||
adapter: EndpointAdapter,
|
||||
addr: iroh::EndpointAddr,
|
||||
keys: NetworkKeys,
|
||||
limits: Arc<Limits>,
|
||||
local_id: EndpointId,
|
||||
peer: EndpointId,
|
||||
) -> DialOutcome {
|
||||
let connect = tokio::time::timeout(limits.dial_timeout, adapter.connect(addr)).await;
|
||||
let (conn, mut send, mut recv) = match connect {
|
||||
Ok(Ok(parts)) => parts,
|
||||
Ok(Err(err)) => {
|
||||
return DialOutcome::Failed {
|
||||
peer,
|
||||
reason: err.to_string(),
|
||||
during_handshake: false,
|
||||
};
|
||||
}
|
||||
Err(_) => {
|
||||
return DialOutcome::Failed {
|
||||
peer,
|
||||
reason: "dial timed out".into(),
|
||||
during_handshake: false,
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
match handshake::initiate(&conn, &mut send, &mut recv, local_id, &keys, &limits).await {
|
||||
Ok(outcome) => DialOutcome::Established(Box::new(InboundSession {
|
||||
conn,
|
||||
send,
|
||||
recv,
|
||||
outcome,
|
||||
})),
|
||||
Err(err) => {
|
||||
conn.close(2u32.into(), b"handshake failed");
|
||||
DialOutcome::Failed {
|
||||
peer,
|
||||
reason: err.to_string(),
|
||||
during_handshake: true,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Encodes a path address as a cache hint.
|
||||
fn encode_hint(addr: &PathAddr) -> Option<String> {
|
||||
match addr {
|
||||
PathAddr::Ip(socket) => Some(format!("ip:{socket}")),
|
||||
PathAddr::Relay(url) => Some(format!("relay:{url}")),
|
||||
PathAddr::Other(_) => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Decodes a cache hint back into an address. Malformed hints are ignored.
|
||||
fn decode_hint(endpoint_id: EndpointId, hint: &str) -> Option<iroh::EndpointAddr> {
|
||||
if let Some(rest) = hint.strip_prefix("ip:") {
|
||||
let socket: std::net::SocketAddr = rest.parse().ok()?;
|
||||
return Some(iroh::EndpointAddr::new(endpoint_id).with_ip_addr(socket));
|
||||
}
|
||||
if let Some(rest) = hint.strip_prefix("relay:") {
|
||||
let url: iroh::RelayUrl = rest.parse().ok()?;
|
||||
return Some(iroh::EndpointAddr::new(endpoint_id).with_relay_url(url));
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Folds newly learned addresses into a known candidate address.
|
||||
fn merge_addr(existing: &mut iroh::EndpointAddr, incoming: &iroh::EndpointAddr) {
|
||||
if existing.id != incoming.id {
|
||||
*existing = incoming.clone();
|
||||
return;
|
||||
}
|
||||
for addr in &incoming.addrs {
|
||||
existing.addrs.insert(addr.clone());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,337 @@
|
||||
//! One authenticated session with one peer, in one network.
|
||||
//!
|
||||
//! A session owns a reader task and a writer task over a single QUIC
|
||||
//! bidirectional stream. Splitting them keeps both halves simple and avoids
|
||||
//! cancelling a partially consumed frame, which stream reads do not tolerate.
|
||||
//!
|
||||
//! Every inbound message is re-checked against the session's network id, so an
|
||||
//! authenticated session for one network can never deliver into another.
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::time::Instant;
|
||||
|
||||
use iroh::EndpointId;
|
||||
use iroh::endpoint::{Connection, RecvStream, SendStream};
|
||||
use tokio::sync::mpsc;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::config::Limits;
|
||||
use crate::dataplane::PluginCapability;
|
||||
use crate::error::ProtocolError;
|
||||
use crate::identity::NetworkId;
|
||||
use crate::proto::handshake::Role;
|
||||
use crate::proto::message::{ControlMessage, Envelope, decode, validate};
|
||||
use crate::proto::{read_frame, write_frame};
|
||||
|
||||
use super::shutdown::Shutdown;
|
||||
|
||||
/// What a session task reports back to its network runtime.
|
||||
#[derive(Debug)]
|
||||
pub(crate) enum SessionEvent {
|
||||
/// A valid control message arrived.
|
||||
Message {
|
||||
/// Which session instance produced this.
|
||||
session_id: u64,
|
||||
/// The peer.
|
||||
peer: EndpointId,
|
||||
/// The decoded message.
|
||||
message: ControlMessage,
|
||||
/// Payload bytes read off the wire.
|
||||
bytes: usize,
|
||||
},
|
||||
/// The peer sent something the protocol does not allow.
|
||||
Violation {
|
||||
/// Which session instance produced this.
|
||||
session_id: u64,
|
||||
/// The peer.
|
||||
peer: EndpointId,
|
||||
/// What was wrong.
|
||||
error: ProtocolError,
|
||||
},
|
||||
/// The session ended.
|
||||
Closed {
|
||||
/// Which session instance ended.
|
||||
session_id: u64,
|
||||
/// The peer.
|
||||
peer: EndpointId,
|
||||
/// Why it ended.
|
||||
reason: String,
|
||||
},
|
||||
}
|
||||
|
||||
/// A live session, as held by the network runtime.
|
||||
#[derive(Debug)]
|
||||
pub(crate) struct Session {
|
||||
pub(crate) id: u64,
|
||||
pub(crate) peer: EndpointId,
|
||||
pub(crate) role: Role,
|
||||
pub(crate) established: Instant,
|
||||
pub(crate) conn: Connection,
|
||||
/// Already encoded frame payloads, so the runtime can account for the exact
|
||||
/// number of control bytes it queues.
|
||||
pub(crate) outbound: mpsc::Sender<Vec<u8>>,
|
||||
pub(crate) hostname: Option<String>,
|
||||
pub(crate) capabilities: Vec<PluginCapability>,
|
||||
pub(crate) messages_sent: u64,
|
||||
pub(crate) messages_received: u64,
|
||||
pub(crate) bytes_sent: u64,
|
||||
pub(crate) bytes_received: u64,
|
||||
reader: JoinHandle<()>,
|
||||
writer: JoinHandle<()>,
|
||||
shutdown: Shutdown,
|
||||
}
|
||||
|
||||
impl Session {
|
||||
/// Signals both tasks to stop and waits for them, with a bounded grace
|
||||
/// period.
|
||||
///
|
||||
/// A peer that stops reading must not be able to hold up shutdown, so the
|
||||
/// tasks are aborted if they do not wind down in time.
|
||||
pub(crate) async fn stop(self) {
|
||||
let Session {
|
||||
conn,
|
||||
reader,
|
||||
writer,
|
||||
shutdown,
|
||||
..
|
||||
} = self;
|
||||
shutdown.trigger();
|
||||
// Closing the connection unblocks a reader parked on the stream.
|
||||
conn.close(0u32.into(), b"session stopped by local agent");
|
||||
|
||||
let reader_abort = reader.abort_handle();
|
||||
let writer_abort = writer.abort_handle();
|
||||
let joined = tokio::time::timeout(STOP_GRACE, async move {
|
||||
let _ = reader.await;
|
||||
let _ = writer.await;
|
||||
})
|
||||
.await;
|
||||
if joined.is_err() {
|
||||
tracing::debug!("session tasks did not wind down in time; aborting them");
|
||||
reader_abort.abort();
|
||||
writer_abort.abort();
|
||||
}
|
||||
}
|
||||
|
||||
/// Aborts both tasks without waiting. Used on replacement.
|
||||
pub(crate) fn abort(&self) {
|
||||
self.shutdown.trigger();
|
||||
self.reader.abort();
|
||||
self.writer.abort();
|
||||
}
|
||||
}
|
||||
|
||||
/// How long a stopping session may take to wind down before its tasks are
|
||||
/// aborted. Shutdown must be bounded even if a peer stops reading.
|
||||
const STOP_GRACE: std::time::Duration = std::time::Duration::from_millis(500);
|
||||
|
||||
/// Source of monotonically increasing session instance ids.
|
||||
static NEXT_SESSION_ID: AtomicU64 = AtomicU64::new(1);
|
||||
|
||||
/// Starts the reader and writer tasks for an authenticated stream.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub(crate) fn spawn(
|
||||
network_id: NetworkId,
|
||||
peer: EndpointId,
|
||||
role: Role,
|
||||
conn: Connection,
|
||||
send: SendStream,
|
||||
recv: RecvStream,
|
||||
limits: Arc<Limits>,
|
||||
events: mpsc::Sender<SessionEvent>,
|
||||
parent_shutdown: Shutdown,
|
||||
) -> Session {
|
||||
let id = NEXT_SESSION_ID.fetch_add(1, Ordering::Relaxed);
|
||||
let shutdown = Shutdown::new();
|
||||
let (outbound_tx, outbound_rx) = mpsc::channel(limits.session_send_queue);
|
||||
|
||||
let writer = tokio::spawn(writer_task(
|
||||
send,
|
||||
outbound_rx,
|
||||
Arc::clone(&limits),
|
||||
shutdown.clone(),
|
||||
parent_shutdown.clone(),
|
||||
));
|
||||
|
||||
let reader = tokio::spawn(reader_task(
|
||||
id,
|
||||
network_id,
|
||||
peer,
|
||||
recv,
|
||||
limits,
|
||||
events,
|
||||
shutdown.clone(),
|
||||
parent_shutdown,
|
||||
));
|
||||
|
||||
Session {
|
||||
id,
|
||||
peer,
|
||||
role,
|
||||
established: Instant::now(),
|
||||
conn,
|
||||
outbound: outbound_tx,
|
||||
hostname: None,
|
||||
capabilities: Vec::new(),
|
||||
messages_sent: 0,
|
||||
messages_received: 0,
|
||||
bytes_sent: 0,
|
||||
bytes_received: 0,
|
||||
reader,
|
||||
writer,
|
||||
shutdown,
|
||||
}
|
||||
}
|
||||
|
||||
async fn writer_task(
|
||||
mut send: SendStream,
|
||||
mut outbound: mpsc::Receiver<Vec<u8>>,
|
||||
limits: Arc<Limits>,
|
||||
shutdown: Shutdown,
|
||||
parent: Shutdown,
|
||||
) {
|
||||
loop {
|
||||
let encoded = tokio::select! {
|
||||
biased;
|
||||
_ = shutdown.wait() => break,
|
||||
_ = parent.wait() => break,
|
||||
encoded = outbound.recv() => match encoded {
|
||||
Some(encoded) => encoded,
|
||||
None => break,
|
||||
},
|
||||
};
|
||||
|
||||
// The write must stay cancellable: shutting the session down cannot
|
||||
// wait for a peer that has stopped reading. Abandoning a half written
|
||||
// frame is fine, because the session is going away with it.
|
||||
let write = tokio::time::timeout(
|
||||
limits.write_timeout,
|
||||
write_frame(&mut send, &encoded, limits.max_frame_len),
|
||||
);
|
||||
let result = tokio::select! {
|
||||
biased;
|
||||
_ = shutdown.wait() => break,
|
||||
_ = parent.wait() => break,
|
||||
result = write => result,
|
||||
};
|
||||
match result {
|
||||
Ok(Ok(())) => {}
|
||||
Ok(Err(err)) => {
|
||||
tracing::debug!(%err, "control stream write failed");
|
||||
break;
|
||||
}
|
||||
Err(_) => {
|
||||
tracing::debug!("control stream write timed out");
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
let _ = send.finish();
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn reader_task(
|
||||
session_id: u64,
|
||||
network_id: NetworkId,
|
||||
peer: EndpointId,
|
||||
mut recv: RecvStream,
|
||||
limits: Arc<Limits>,
|
||||
events: mpsc::Sender<SessionEvent>,
|
||||
shutdown: Shutdown,
|
||||
parent: Shutdown,
|
||||
) {
|
||||
let reason = loop {
|
||||
let frame = tokio::select! {
|
||||
biased;
|
||||
_ = shutdown.wait() => break "stopped locally".to_string(),
|
||||
_ = parent.wait() => break "network deactivated".to_string(),
|
||||
frame = read_frame(&mut recv, limits.max_frame_len) => frame,
|
||||
};
|
||||
|
||||
let payload = match frame {
|
||||
Ok(payload) => payload,
|
||||
Err(ProtocolError::StreamClosed) => break "peer closed the control stream".to_string(),
|
||||
Err(ProtocolError::Stream(err)) => {
|
||||
// The transport went away. That is a disconnect, not a peer
|
||||
// misbehaving, so it ends the session without being counted as
|
||||
// a protocol violation.
|
||||
break format!("control stream error: {err}");
|
||||
}
|
||||
Err(err) => {
|
||||
// A framing violation ends this session. Framing errors are not
|
||||
// recoverable mid-stream: the next bytes have no known meaning.
|
||||
let text = err.to_string();
|
||||
let _ = events
|
||||
.send(SessionEvent::Violation {
|
||||
session_id,
|
||||
peer,
|
||||
error: err,
|
||||
})
|
||||
.await;
|
||||
break text;
|
||||
}
|
||||
};
|
||||
|
||||
let bytes = payload.len();
|
||||
let envelope: Envelope = match decode(&payload) {
|
||||
Ok(envelope) => envelope,
|
||||
Err(err) => {
|
||||
let _ = events
|
||||
.send(SessionEvent::Violation {
|
||||
session_id,
|
||||
peer,
|
||||
error: err,
|
||||
})
|
||||
.await;
|
||||
break "malformed control frame".to_string();
|
||||
}
|
||||
};
|
||||
|
||||
// Network isolation: a session authenticated for one network must never
|
||||
// deliver a message belonging to another.
|
||||
if envelope.network_id != *network_id.as_bytes() {
|
||||
let _ = events
|
||||
.send(SessionEvent::Violation {
|
||||
session_id,
|
||||
peer,
|
||||
error: ProtocolError::NetworkMismatch,
|
||||
})
|
||||
.await;
|
||||
break "network id mismatch on an authenticated session".to_string();
|
||||
}
|
||||
|
||||
if let Err(err) = validate(&envelope.message, &limits) {
|
||||
let _ = events
|
||||
.send(SessionEvent::Violation {
|
||||
session_id,
|
||||
peer,
|
||||
error: err,
|
||||
})
|
||||
.await;
|
||||
continue;
|
||||
}
|
||||
|
||||
if events
|
||||
.send(SessionEvent::Message {
|
||||
session_id,
|
||||
peer,
|
||||
message: envelope.message,
|
||||
bytes,
|
||||
})
|
||||
.await
|
||||
.is_err()
|
||||
{
|
||||
break "network runtime stopped".to_string();
|
||||
}
|
||||
};
|
||||
|
||||
// Best effort: the runtime may already have stopped draining this channel
|
||||
// while it tears the network down, and a closing session must not block on
|
||||
// that.
|
||||
let _ = events.try_send(SessionEvent::Closed {
|
||||
session_id,
|
||||
peer,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
//! A minimal cancellation primitive.
|
||||
//!
|
||||
//! Kept local so the crate does not pull in a utility dependency for one type,
|
||||
//! and so that no global state is involved: every agent and every network
|
||||
//! runtime owns its own token.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use tokio::sync::watch;
|
||||
|
||||
/// A clonable cancellation token.
|
||||
#[derive(Debug, Clone)]
|
||||
pub(crate) struct Shutdown {
|
||||
tx: Arc<watch::Sender<bool>>,
|
||||
rx: watch::Receiver<bool>,
|
||||
}
|
||||
|
||||
impl Shutdown {
|
||||
/// Creates an untriggered token.
|
||||
pub(crate) fn new() -> Self {
|
||||
let (tx, rx) = watch::channel(false);
|
||||
Self {
|
||||
tx: Arc::new(tx),
|
||||
rx,
|
||||
}
|
||||
}
|
||||
|
||||
/// Triggers cancellation. Idempotent.
|
||||
pub(crate) fn trigger(&self) {
|
||||
let _ = self.tx.send(true);
|
||||
}
|
||||
|
||||
/// Whether cancellation has been triggered.
|
||||
pub(crate) fn is_triggered(&self) -> bool {
|
||||
*self.rx.borrow()
|
||||
}
|
||||
|
||||
/// Resolves once cancellation has been triggered.
|
||||
pub(crate) async fn wait(&self) {
|
||||
let mut rx = self.rx.clone();
|
||||
{
|
||||
if *rx.borrow() {
|
||||
return;
|
||||
}
|
||||
}
|
||||
while rx.changed().await.is_ok() {
|
||||
if *rx.borrow() {
|
||||
return;
|
||||
}
|
||||
}
|
||||
// The sender is gone, which for our purposes means "stop".
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,153 @@
|
||||
//! Status snapshots.
|
||||
//!
|
||||
//! Metrics are reported at the level they actually belong to. Values that
|
||||
//! genuinely cannot be attributed to a single network — everything the iroh
|
||||
//! endpoint aggregates, for instance — stay at the endpoint level rather than
|
||||
//! being split between networks with invented precision.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use iroh::{EndpointAddr, EndpointId};
|
||||
|
||||
use crate::dataplane::PluginCapability;
|
||||
use crate::discovery::CandidateSource;
|
||||
use crate::identity::{NetworkDescriptor, NetworkId, NetworkName};
|
||||
use crate::net::{ConnectionCounters, PathAddr, PathInfo, TransportKind};
|
||||
use crate::proto::handshake::Role;
|
||||
use crate::storage::CacheOutcome;
|
||||
|
||||
/// Whether a configured network is running locally.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum NetworkState {
|
||||
/// Configured and running.
|
||||
Active,
|
||||
/// Configured but not running.
|
||||
Inactive,
|
||||
}
|
||||
|
||||
/// An unverified candidate as seen by a network runtime.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CandidateStatus {
|
||||
/// Candidate endpoint id.
|
||||
pub endpoint_id: EndpointId,
|
||||
/// Where it came from.
|
||||
pub source: CandidateSource,
|
||||
/// Consecutive failed dial attempts since the last success.
|
||||
pub consecutive_failures: u32,
|
||||
}
|
||||
|
||||
/// Status of one authenticated session.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PeerStatus {
|
||||
/// Authenticated endpoint id.
|
||||
pub endpoint_id: EndpointId,
|
||||
/// Which side this agent played in the handshake.
|
||||
pub role: Role,
|
||||
/// Hostname the peer announced, if it has announced one yet.
|
||||
///
|
||||
/// A mutable binding, not an identity.
|
||||
pub hostname: Option<String>,
|
||||
/// Capabilities the peer announced. Payloads stay opaque.
|
||||
pub capabilities: Vec<PluginCapability>,
|
||||
/// How long the session has been up.
|
||||
pub connected_for: Duration,
|
||||
/// Verified paths of the underlying connection.
|
||||
pub paths: Vec<PathInfo>,
|
||||
/// How the connection currently reaches the peer.
|
||||
pub transport: TransportKind,
|
||||
/// RTT of the selected path, when iroh reported one.
|
||||
pub rtt: Option<Duration>,
|
||||
/// Per-connection counters.
|
||||
pub connection: ConnectionCounters,
|
||||
/// Control messages sent on this session.
|
||||
pub control_messages_sent: u64,
|
||||
/// Control messages received on this session.
|
||||
pub control_messages_received: u64,
|
||||
/// Control payload bytes queued for this session.
|
||||
pub control_bytes_sent: u64,
|
||||
/// Control payload bytes read from this session.
|
||||
pub control_bytes_received: u64,
|
||||
}
|
||||
|
||||
/// Counters scoped to one logical network.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct NetworkMetrics {
|
||||
/// Outbound dial attempts started.
|
||||
pub dial_attempts: u64,
|
||||
/// Outbound dials that failed before or during the handshake.
|
||||
pub dial_failures: u64,
|
||||
/// Handshakes rejected, in either direction.
|
||||
pub handshake_failures: u64,
|
||||
/// Sessions that reached the authenticated state.
|
||||
pub sessions_established: u64,
|
||||
/// Sessions that ended.
|
||||
pub disconnects: u64,
|
||||
/// Control messages sent in this network.
|
||||
pub control_messages_sent: u64,
|
||||
/// Control messages received in this network.
|
||||
pub control_messages_received: u64,
|
||||
/// Control bytes sent in this network, payload only.
|
||||
pub control_bytes_sent: u64,
|
||||
/// Control bytes received in this network, payload only.
|
||||
pub control_bytes_received: u64,
|
||||
/// Messages or sessions rejected for protocol violations.
|
||||
pub protocol_violations: u64,
|
||||
/// Errors reported by IP plugins. Never fatal.
|
||||
pub plugin_errors: u64,
|
||||
}
|
||||
|
||||
/// Status of one network.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NetworkStatus {
|
||||
/// Immutable deterministic description of the network space.
|
||||
pub descriptor: NetworkDescriptor,
|
||||
/// Network name, for convenience.
|
||||
pub name: NetworkName,
|
||||
/// Public network identifier.
|
||||
pub network_id: NetworkId,
|
||||
/// Whether the network is running locally.
|
||||
pub state: NetworkState,
|
||||
/// Authenticated sessions.
|
||||
pub peers: Vec<PeerStatus>,
|
||||
/// Unverified candidates currently known. Not peers.
|
||||
pub candidates: Vec<CandidateStatus>,
|
||||
/// Per-network counters.
|
||||
pub metrics: NetworkMetrics,
|
||||
}
|
||||
|
||||
impl NetworkStatus {
|
||||
/// Endpoint ids of peers with an authenticated session.
|
||||
pub fn connected_peers(&self) -> Vec<EndpointId> {
|
||||
self.peers.iter().map(|peer| peer.endpoint_id).collect()
|
||||
}
|
||||
}
|
||||
|
||||
/// Status of the whole agent.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct AgentStatus {
|
||||
/// This device's persistent endpoint id.
|
||||
pub endpoint_id: EndpointId,
|
||||
/// Hostname announced to peers.
|
||||
pub hostname: String,
|
||||
/// Sockets actually bound.
|
||||
pub bound_sockets: Vec<std::net::SocketAddr>,
|
||||
/// Addresses iroh believes this endpoint has. Observed, not verified.
|
||||
pub observed_addrs: Vec<PathAddr>,
|
||||
/// The dialable address of this endpoint, as iroh currently reports it.
|
||||
pub endpoint_addr: EndpointAddr,
|
||||
/// What happened to the disposable cache at startup.
|
||||
pub cache_outcome: CacheOutcome,
|
||||
/// Whether the cache is currently usable.
|
||||
pub cache_healthy: bool,
|
||||
/// Per-network status, including configured but inactive networks.
|
||||
pub networks: Vec<NetworkStatus>,
|
||||
}
|
||||
|
||||
impl AgentStatus {
|
||||
/// Looks up one network's status.
|
||||
pub fn network(&self, network_id: &NetworkId) -> Option<&NetworkStatus> {
|
||||
self.networks
|
||||
.iter()
|
||||
.find(|status| &status.network_id == network_id)
|
||||
}
|
||||
}
|
||||
+327
@@ -0,0 +1,327 @@
|
||||
//! Library configuration.
|
||||
//!
|
||||
//! Everything the agent needs is passed in explicitly. The library reads no
|
||||
//! environment variables, installs no global state and picks no default
|
||||
//! directories behind the caller's back — [`StoragePaths::user_default`] exists
|
||||
//! but must be called on purpose.
|
||||
|
||||
use std::net::SocketAddr;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use crate::dataplane::SharedPlugin;
|
||||
use crate::discovery::NetworkDiscovery;
|
||||
use crate::error::{Error, Result};
|
||||
|
||||
/// Qualifier/organisation/application triple used for platform directories.
|
||||
const APP_NAME: &str = "tsunagi";
|
||||
|
||||
/// Where the two stores live.
|
||||
///
|
||||
/// The mandatory state and the disposable cache are separate both logically and
|
||||
/// physically, so that the cache can be deleted at any time without touching
|
||||
/// identity or network configuration.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct StoragePaths {
|
||||
/// Directory holding `state.sqlite` and the ownership lock.
|
||||
pub state_dir: PathBuf,
|
||||
/// Directory holding `cache.sqlite`.
|
||||
pub cache_dir: PathBuf,
|
||||
}
|
||||
|
||||
impl StoragePaths {
|
||||
/// Uses explicit directories. Tests always use temporary directories.
|
||||
pub fn new(state_dir: impl Into<PathBuf>, cache_dir: impl Into<PathBuf>) -> Self {
|
||||
Self {
|
||||
state_dir: state_dir.into(),
|
||||
cache_dir: cache_dir.into(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Puts both stores under one root, in `state/` and `cache/` subdirectories.
|
||||
pub fn under(root: impl AsRef<Path>) -> Self {
|
||||
let root = root.as_ref();
|
||||
Self {
|
||||
state_dir: root.join("state"),
|
||||
cache_dir: root.join("cache"),
|
||||
}
|
||||
}
|
||||
|
||||
/// The per-user platform directories.
|
||||
///
|
||||
/// A future system service can supply its own paths instead.
|
||||
pub fn user_default() -> Result<Self> {
|
||||
let dirs = directories::ProjectDirs::from("", "", APP_NAME).ok_or_else(|| {
|
||||
Error::Storage("no valid home directory for platform config paths".into())
|
||||
})?;
|
||||
Ok(Self {
|
||||
state_dir: dirs.data_dir().to_path_buf(),
|
||||
cache_dir: dirs.cache_dir().to_path_buf(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Path of the mandatory state database.
|
||||
pub fn state_db(&self) -> PathBuf {
|
||||
self.state_dir.join("state.sqlite")
|
||||
}
|
||||
|
||||
/// Path of the disposable cache database.
|
||||
pub fn cache_db(&self) -> PathBuf {
|
||||
self.cache_dir.join("cache.sqlite")
|
||||
}
|
||||
|
||||
/// Path of the ownership lock file.
|
||||
pub fn lock_file(&self) -> PathBuf {
|
||||
self.state_dir.join("state.lock")
|
||||
}
|
||||
}
|
||||
|
||||
/// How the iroh endpoint is allowed to reach the outside world.
|
||||
///
|
||||
/// The default is [`TransportPolicy::LocalOnly`] so that a plain
|
||||
/// `AgentConfig::new(...)` never reaches the internet by accident. Callers that
|
||||
/// want public connectivity must opt in.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub enum TransportPolicy {
|
||||
/// No relays, no address lookup service, no port mapping.
|
||||
///
|
||||
/// Suitable for tests and for fully local deployments.
|
||||
#[default]
|
||||
LocalOnly,
|
||||
/// No relays, but the n0 DNS/pkarr address lookup is enabled.
|
||||
DirectOnly,
|
||||
/// iroh's standard behaviour, including the public n0 relays.
|
||||
///
|
||||
/// Public relays are fine for development; they carry no availability
|
||||
/// guarantee.
|
||||
N0Defaults,
|
||||
}
|
||||
|
||||
/// Bounds applied to everything that comes off the network.
|
||||
///
|
||||
/// Each of these is enforced before memory is allocated for the corresponding
|
||||
/// object where that is possible (notably [`Limits::max_frame_len`]).
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Limits {
|
||||
/// Largest accepted control frame payload, in bytes.
|
||||
pub max_frame_len: usize,
|
||||
/// Largest accepted hostname, in bytes.
|
||||
pub max_hostname_len: usize,
|
||||
/// Largest number of plugin capabilities in one announcement.
|
||||
pub max_capabilities: usize,
|
||||
/// Largest opaque plugin payload, in bytes.
|
||||
pub max_capability_data_len: usize,
|
||||
/// Largest accepted echo payload in a ping/pong exchange, in bytes.
|
||||
pub max_echo_payload_len: usize,
|
||||
/// Largest accepted free-text reason string, in bytes.
|
||||
pub max_reason_len: usize,
|
||||
/// Deadline for the whole handshake.
|
||||
pub handshake_timeout: Duration,
|
||||
/// Deadline for one outbound dial attempt.
|
||||
pub dial_timeout: Duration,
|
||||
/// Deadline for writing one control frame.
|
||||
///
|
||||
/// Liveness of an established session is delegated to QUIC: iroh configures
|
||||
/// keep-alives and an idle timeout, so a dead peer surfaces as a read error
|
||||
/// rather than needing a protocol-level heartbeat here.
|
||||
pub write_timeout: Duration,
|
||||
/// Maximum simultaneous outbound dials per network.
|
||||
pub max_concurrent_dials: usize,
|
||||
/// Maximum simultaneous authenticated sessions per network.
|
||||
pub max_sessions_per_network: usize,
|
||||
/// Maximum simultaneous inbound connections being handshaken.
|
||||
pub max_inbound_handshakes: usize,
|
||||
/// Capacity of a session's outbound queue, providing backpressure.
|
||||
pub session_send_queue: usize,
|
||||
/// Capacity of the event broadcast channel.
|
||||
pub event_buffer: usize,
|
||||
/// Maximum address hints kept per peer in the cache.
|
||||
pub max_hints_per_peer: usize,
|
||||
}
|
||||
|
||||
impl Default for Limits {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
max_frame_len: 64 * 1024,
|
||||
max_hostname_len: 255,
|
||||
max_capabilities: 16,
|
||||
max_capability_data_len: 4 * 1024,
|
||||
max_echo_payload_len: 4 * 1024,
|
||||
max_reason_len: 256,
|
||||
handshake_timeout: Duration::from_secs(10),
|
||||
dial_timeout: Duration::from_secs(10),
|
||||
write_timeout: Duration::from_secs(30),
|
||||
max_concurrent_dials: 8,
|
||||
max_sessions_per_network: 64,
|
||||
max_inbound_handshakes: 32,
|
||||
session_send_queue: 64,
|
||||
event_buffer: 512,
|
||||
max_hints_per_peer: 8,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Bounded exponential backoff with jitter for reconnect attempts.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct ReconnectPolicy {
|
||||
/// Delay before the first retry.
|
||||
pub initial_delay: Duration,
|
||||
/// Upper bound on the delay.
|
||||
pub max_delay: Duration,
|
||||
/// Multiplier applied after each failed attempt.
|
||||
pub factor: f64,
|
||||
/// Fraction of the delay applied as random jitter, in `0.0..=1.0`.
|
||||
pub jitter: f64,
|
||||
/// Give up on a peer after this many consecutive failures until it is seen
|
||||
/// again by discovery. `None` means never give up while the network is up.
|
||||
pub max_consecutive_failures: Option<u32>,
|
||||
}
|
||||
|
||||
impl Default for ReconnectPolicy {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
initial_delay: Duration::from_millis(250),
|
||||
max_delay: Duration::from_secs(30),
|
||||
factor: 2.0,
|
||||
jitter: 0.3,
|
||||
max_consecutive_failures: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl ReconnectPolicy {
|
||||
/// Delay to wait before retry number `attempt` (1-based), with jitter.
|
||||
pub(crate) fn delay_for(&self, attempt: u32) -> Duration {
|
||||
let exp = self.factor.powi(attempt.saturating_sub(1).min(32) as i32);
|
||||
let base = self.initial_delay.as_secs_f64() * exp;
|
||||
let capped = base.min(self.max_delay.as_secs_f64());
|
||||
let jitter = self.jitter.clamp(0.0, 1.0);
|
||||
let factor = 1.0 - jitter + jitter * 2.0 * rand::random::<f64>();
|
||||
Duration::from_secs_f64((capped * factor).max(0.0))
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything needed to start an [`crate::Agent`].
|
||||
#[derive(Clone)]
|
||||
pub struct AgentConfig {
|
||||
/// Where the mandatory state and the disposable cache live.
|
||||
pub paths: StoragePaths,
|
||||
/// Explicit local bind addresses. Empty means iroh's defaults.
|
||||
///
|
||||
/// Tests bind to `127.0.0.1:0` so each agent gets a dynamic port.
|
||||
pub bind_addrs: Vec<SocketAddr>,
|
||||
/// How much external connectivity machinery the endpoint may use.
|
||||
pub transport: TransportPolicy,
|
||||
/// Hostname announced to peers. `None` keeps whatever the state store holds,
|
||||
/// falling back to the OS hostname and finally to a short endpoint id.
|
||||
pub hostname: Option<String>,
|
||||
/// Discovery backend. `None` disables discovery-driven dialling; static
|
||||
/// bootstrap candidates still work.
|
||||
pub discovery: Option<Arc<dyn NetworkDiscovery>>,
|
||||
/// How often each active network re-runs discovery and re-evaluates dials.
|
||||
pub discovery_interval: Duration,
|
||||
/// Bounds applied to network input.
|
||||
pub limits: Limits,
|
||||
/// Reconnect backoff policy.
|
||||
pub reconnect: ReconnectPolicy,
|
||||
/// IP plugins whose capabilities are announced and dispatched.
|
||||
pub plugins: Vec<SharedPlugin>,
|
||||
}
|
||||
|
||||
impl AgentConfig {
|
||||
/// Creates a configuration with local-only transport and default limits.
|
||||
pub fn new(paths: StoragePaths) -> Self {
|
||||
Self {
|
||||
paths,
|
||||
bind_addrs: Vec::new(),
|
||||
transport: TransportPolicy::default(),
|
||||
hostname: None,
|
||||
discovery: None,
|
||||
discovery_interval: Duration::from_secs(5),
|
||||
limits: Limits::default(),
|
||||
reconnect: ReconnectPolicy::default(),
|
||||
plugins: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Binds to loopback with a dynamic port. Used by the test suite.
|
||||
pub fn with_loopback_bind(mut self) -> Self {
|
||||
self.bind_addrs = vec![
|
||||
SocketAddr::from(([127, 0, 0, 1], 0)),
|
||||
SocketAddr::from(([0, 0, 0, 0, 0, 0, 0, 1], 0)),
|
||||
];
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets an explicit list of bind addresses.
|
||||
pub fn with_bind_addrs(mut self, addrs: impl IntoIterator<Item = SocketAddr>) -> Self {
|
||||
self.bind_addrs = addrs.into_iter().collect();
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the transport policy.
|
||||
pub fn with_transport(mut self, transport: TransportPolicy) -> Self {
|
||||
self.transport = transport;
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the discovery backend.
|
||||
pub fn with_discovery(mut self, discovery: Arc<dyn NetworkDiscovery>) -> Self {
|
||||
self.discovery = Some(discovery);
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets how often discovery runs.
|
||||
pub fn with_discovery_interval(mut self, interval: Duration) -> Self {
|
||||
self.discovery_interval = interval;
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the announced hostname.
|
||||
pub fn with_hostname(mut self, hostname: impl Into<String>) -> Self {
|
||||
self.hostname = Some(hostname.into());
|
||||
self
|
||||
}
|
||||
|
||||
/// Registers an IP plugin.
|
||||
pub fn with_plugin(mut self, plugin: SharedPlugin) -> Self {
|
||||
self.plugins.push(plugin);
|
||||
self
|
||||
}
|
||||
|
||||
/// Replaces the limits.
|
||||
pub fn with_limits(mut self, limits: Limits) -> Self {
|
||||
self.limits = limits;
|
||||
self
|
||||
}
|
||||
|
||||
/// Replaces the reconnect policy.
|
||||
pub fn with_reconnect(mut self, reconnect: ReconnectPolicy) -> Self {
|
||||
self.reconnect = reconnect;
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for AgentConfig {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("AgentConfig")
|
||||
.field("paths", &self.paths)
|
||||
.field("bind_addrs", &self.bind_addrs)
|
||||
.field("transport", &self.transport)
|
||||
.field("hostname", &self.hostname)
|
||||
.field("discovery", &self.discovery.as_ref().map(|d| d.name()))
|
||||
.field("discovery_interval", &self.discovery_interval)
|
||||
.field("limits", &self.limits)
|
||||
.field("reconnect", &self.reconnect)
|
||||
.field(
|
||||
"plugins",
|
||||
&self
|
||||
.plugins
|
||||
.iter()
|
||||
.map(|p| p.protocol_id().to_string())
|
||||
.collect::<Vec<_>>(),
|
||||
)
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,168 @@
|
||||
//! Boundary between the control plane core and future IP plugins.
|
||||
//!
|
||||
//! The data plane is where actual IP connectivity is created. WireGuard is the
|
||||
//! first planned plugin; none is implemented here.
|
||||
//!
|
||||
//! Two rules shape this module:
|
||||
//!
|
||||
//! 1. **The core never parses plugin payloads.** A [`PluginCapability`] carries
|
||||
//! a protocol id, a version, an enabled flag and a bounded opaque blob. The
|
||||
//! core transports the blob and hands it to the matching plugin. It does not
|
||||
//! know what a WireGuard configuration looks like.
|
||||
//! 2. **Plugin keys and lifecycle are separate from iroh identity and from the
|
||||
//! network secret.** A plugin owns its own keys and its own system objects.
|
||||
//!
|
||||
//! An iroh address is *not* automatically a WireGuard address. A future plugin
|
||||
//! is expected to gather its own reachability information and ship it through
|
||||
//! the control plane as its announcement payload.
|
||||
//!
|
||||
//! A data plane failure never stops the daemon: errors returned here are
|
||||
//! recorded and surfaced, the control plane keeps running.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use iroh::EndpointId;
|
||||
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
/// Maximum length of a plugin protocol identifier.
|
||||
pub const MAX_PROTOCOL_ID_LEN: usize = 32;
|
||||
|
||||
/// An announcement of one IP plugin's capability.
|
||||
///
|
||||
/// `data` is opaque to the core. Nothing in it may be interpreted as a shell
|
||||
/// command, a filesystem path or an OS setting by the core; a plugin that
|
||||
/// chooses to do so must validate it itself.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct PluginCapability {
|
||||
/// Protocol identifier, e.g. `wireguard`. Bounded by [`MAX_PROTOCOL_ID_LEN`].
|
||||
pub protocol: String,
|
||||
/// Version of the plugin's announcement format.
|
||||
pub version: u16,
|
||||
/// Whether the peer currently has this plugin enabled.
|
||||
pub enabled: bool,
|
||||
/// Opaque, bounded, plugin-defined payload.
|
||||
pub data: Vec<u8>,
|
||||
}
|
||||
|
||||
/// Errors a plugin may return. They are recorded, never fatal for the agent.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[non_exhaustive]
|
||||
pub enum PluginError {
|
||||
/// The plugin is not currently able to produce or apply configuration.
|
||||
#[error("plugin unavailable: {0}")]
|
||||
Unavailable(String),
|
||||
/// A peer announcement was not acceptable to the plugin.
|
||||
#[error("rejected peer announcement: {0}")]
|
||||
Rejected(String),
|
||||
/// Anything else.
|
||||
#[error("plugin error: {0}")]
|
||||
Other(String),
|
||||
}
|
||||
|
||||
/// The minimal contract a future IP plugin implements.
|
||||
///
|
||||
/// Implementations must be cheap and non-blocking: the agent calls them from
|
||||
/// its runtime tasks. Anything slow belongs in the plugin's own tasks.
|
||||
pub trait IpPlugin: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// Stable protocol identifier, e.g. `wireguard`.
|
||||
///
|
||||
/// Must be non-empty and at most [`MAX_PROTOCOL_ID_LEN`] bytes.
|
||||
fn protocol_id(&self) -> &str;
|
||||
|
||||
/// Produces this agent's announcement for a given network.
|
||||
///
|
||||
/// Returning `Ok(None)` means "nothing to announce right now", which is
|
||||
/// different from an error.
|
||||
fn local_capability(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
) -> std::result::Result<Option<PluginCapability>, PluginError>;
|
||||
|
||||
/// Called when a peer announces a capability for this plugin's protocol.
|
||||
///
|
||||
/// The core has already bounded the payload size but has not interpreted it.
|
||||
fn on_peer_capability(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
capability: &PluginCapability,
|
||||
) -> std::result::Result<(), PluginError>;
|
||||
|
||||
/// Called when a peer's session in a network goes away.
|
||||
fn on_peer_gone(&self, network: NetworkId, peer: EndpointId);
|
||||
|
||||
/// Called when a network is deactivated locally.
|
||||
///
|
||||
/// This is a local deactivation, not a signed revocation of membership.
|
||||
fn on_network_deactivated(&self, network: NetworkId);
|
||||
}
|
||||
|
||||
/// A shared handle to a plugin.
|
||||
pub type SharedPlugin = Arc<dyn IpPlugin>;
|
||||
|
||||
/// A plugin used in tests and examples.
|
||||
///
|
||||
/// It announces an explicitly test-only protocol id, so nothing in this crate
|
||||
/// ever advertises WireGuard as an available transport before it exists.
|
||||
#[derive(Debug)]
|
||||
pub struct TestCapabilityPlugin {
|
||||
protocol: String,
|
||||
payload: Vec<u8>,
|
||||
seen: std::sync::Mutex<Vec<(NetworkId, EndpointId, PluginCapability)>>,
|
||||
}
|
||||
|
||||
impl TestCapabilityPlugin {
|
||||
/// Creates a plugin announcing `protocol` with a fixed opaque payload.
|
||||
pub fn new(protocol: impl Into<String>, payload: impl Into<Vec<u8>>) -> Self {
|
||||
Self {
|
||||
protocol: protocol.into(),
|
||||
payload: payload.into(),
|
||||
seen: std::sync::Mutex::new(Vec::new()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns everything this plugin was handed so far.
|
||||
pub fn observed(&self) -> Vec<(NetworkId, EndpointId, PluginCapability)> {
|
||||
match self.seen.lock() {
|
||||
Ok(guard) => guard.clone(),
|
||||
Err(poisoned) => poisoned.into_inner().clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl IpPlugin for TestCapabilityPlugin {
|
||||
fn protocol_id(&self) -> &str {
|
||||
&self.protocol
|
||||
}
|
||||
|
||||
fn local_capability(
|
||||
&self,
|
||||
_network: NetworkId,
|
||||
) -> std::result::Result<Option<PluginCapability>, PluginError> {
|
||||
Ok(Some(PluginCapability {
|
||||
protocol: self.protocol.clone(),
|
||||
version: 1,
|
||||
enabled: true,
|
||||
data: self.payload.clone(),
|
||||
}))
|
||||
}
|
||||
|
||||
fn on_peer_capability(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
capability: &PluginCapability,
|
||||
) -> std::result::Result<(), PluginError> {
|
||||
let mut guard = match self.seen.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
guard.push((network, peer, capability.clone()));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn on_peer_gone(&self, _network: NetworkId, _peer: EndpointId) {}
|
||||
|
||||
fn on_network_deactivated(&self, _network: NetworkId) {}
|
||||
}
|
||||
@@ -0,0 +1,303 @@
|
||||
//! Finding *candidates*, and nothing more.
|
||||
//!
|
||||
//! Discovery answers one question: "which iroh endpoints might currently be
|
||||
//! participating in the network behind this [`DiscoveryKey`], and at which
|
||||
//! addresses?". Its answers are **unverified candidates**. Membership is decided
|
||||
//! later, by the control protocol handshake in [`crate::proto::handshake`].
|
||||
//!
|
||||
//! A discovery backend must not carry control messages between agents, must not
|
||||
//! confirm authentication and must not mutate agent state directly.
|
||||
//!
|
||||
//! Two concerns are kept apart:
|
||||
//!
|
||||
//! * *Finding members of a network* — [`NetworkDiscovery::resolve`], keyed by
|
||||
//! the secret-derived [`DiscoveryKey`].
|
||||
//! * *Resolving the address of one iroh endpoint* — an
|
||||
//! [`iroh::EndpointAddr`] either already carries addresses, or iroh's own
|
||||
//! address lookup service must be enabled. Dialling a bare [`EndpointId`]
|
||||
//! with neither is expected to fail.
|
||||
//!
|
||||
//! No empty result ever proves a network is empty. It only means "nobody found
|
||||
//! yet".
|
||||
//!
|
||||
//! Mainline DHT discovery is future work and is not implemented here.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::future::Future;
|
||||
use std::pin::Pin;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use iroh::{EndpointAddr, EndpointId};
|
||||
|
||||
use crate::error::Result;
|
||||
use crate::identity::DiscoveryKey;
|
||||
|
||||
/// A boxed future, so that [`NetworkDiscovery`] stays object safe.
|
||||
pub type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;
|
||||
|
||||
/// Where a candidate came from. Purely informational.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum CandidateSource {
|
||||
/// A statically configured bootstrap entry.
|
||||
Bootstrap,
|
||||
/// A discovery backend lookup.
|
||||
Discovery,
|
||||
/// An address hint restored from the disposable cache.
|
||||
Cache,
|
||||
}
|
||||
|
||||
/// An unverified candidate peer.
|
||||
///
|
||||
/// Holding one grants nothing: the peer still has to pass the handshake.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Candidate {
|
||||
/// iroh address of the candidate, including whatever addressing info exists.
|
||||
pub addr: EndpointAddr,
|
||||
/// Where this candidate came from.
|
||||
pub source: CandidateSource,
|
||||
}
|
||||
|
||||
impl Candidate {
|
||||
/// Creates a candidate.
|
||||
pub fn new(addr: EndpointAddr, source: CandidateSource) -> Self {
|
||||
Self { addr, source }
|
||||
}
|
||||
|
||||
/// The candidate's endpoint id.
|
||||
pub fn endpoint_id(&self) -> EndpointId {
|
||||
self.addr.id
|
||||
}
|
||||
}
|
||||
|
||||
/// A replaceable source of candidates.
|
||||
///
|
||||
/// Implementations must be cheap to clone behind an [`Arc`] and must never
|
||||
/// block the async executor.
|
||||
pub trait NetworkDiscovery: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// A short name used in diagnostics.
|
||||
fn name(&self) -> &str;
|
||||
|
||||
/// Publishes this agent's address under `key`.
|
||||
///
|
||||
/// Backends that cannot publish (static bootstrap lists) return `Ok(())`.
|
||||
fn publish<'a>(&'a self, key: DiscoveryKey, addr: EndpointAddr) -> BoxFuture<'a, Result<()>>;
|
||||
|
||||
/// Withdraws a previously published address.
|
||||
fn unpublish<'a>(
|
||||
&'a self,
|
||||
key: DiscoveryKey,
|
||||
endpoint: EndpointId,
|
||||
) -> BoxFuture<'a, Result<()>>;
|
||||
|
||||
/// Returns the candidates currently known for `key`.
|
||||
fn resolve<'a>(&'a self, key: DiscoveryKey) -> BoxFuture<'a, Result<Vec<Candidate>>>;
|
||||
}
|
||||
|
||||
/// A statically configured list of bootstrap candidates.
|
||||
///
|
||||
/// Each entry must carry enough addressing information to be dialled, i.e. an
|
||||
/// iroh endpoint id plus direct addresses or a relay URL, unless iroh's own
|
||||
/// address lookup is enabled in [`crate::config::TransportPolicy`].
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct StaticBootstrap {
|
||||
entries: Vec<EndpointAddr>,
|
||||
}
|
||||
|
||||
impl StaticBootstrap {
|
||||
/// Creates a bootstrap list.
|
||||
pub fn new(entries: impl IntoIterator<Item = EndpointAddr>) -> Self {
|
||||
Self {
|
||||
entries: entries.into_iter().collect(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl NetworkDiscovery for StaticBootstrap {
|
||||
fn name(&self) -> &str {
|
||||
"static-bootstrap"
|
||||
}
|
||||
|
||||
fn publish<'a>(&'a self, _key: DiscoveryKey, _addr: EndpointAddr) -> BoxFuture<'a, Result<()>> {
|
||||
Box::pin(async { Ok(()) })
|
||||
}
|
||||
|
||||
fn unpublish<'a>(
|
||||
&'a self,
|
||||
_key: DiscoveryKey,
|
||||
_endpoint: EndpointId,
|
||||
) -> BoxFuture<'a, Result<()>> {
|
||||
Box::pin(async { Ok(()) })
|
||||
}
|
||||
|
||||
fn resolve<'a>(&'a self, _key: DiscoveryKey) -> BoxFuture<'a, Result<Vec<Candidate>>> {
|
||||
let candidates: Vec<Candidate> = self
|
||||
.entries
|
||||
.iter()
|
||||
.cloned()
|
||||
.map(|addr| Candidate::new(addr, CandidateSource::Bootstrap))
|
||||
.collect();
|
||||
Box::pin(async move { Ok(candidates) })
|
||||
}
|
||||
}
|
||||
|
||||
/// An in-process discovery backend used by tests and examples.
|
||||
///
|
||||
/// It stores a mapping from [`DiscoveryKey`] to endpoint addresses and nothing
|
||||
/// else. It carries no messages, performs no authentication and cannot touch an
|
||||
/// agent's state. Clone it to hand the same rendezvous table to several agents;
|
||||
/// create a new one per test so that tests stay independent — there is no global
|
||||
/// mutable state here.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct SharedMemoryDiscovery {
|
||||
inner: Arc<Mutex<HashMap<DiscoveryKey, HashMap<EndpointId, EndpointAddr>>>>,
|
||||
}
|
||||
|
||||
impl SharedMemoryDiscovery {
|
||||
/// Creates an empty rendezvous table.
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Number of entries published under `key`. Useful in tests.
|
||||
pub fn len(&self, key: &DiscoveryKey) -> usize {
|
||||
self.with_inner(|map| map.get(key).map_or(0, HashMap::len))
|
||||
}
|
||||
|
||||
/// Whether nothing is published under `key`.
|
||||
pub fn is_empty(&self, key: &DiscoveryKey) -> bool {
|
||||
self.len(key) == 0
|
||||
}
|
||||
|
||||
/// Removes every entry under `key`, simulating a discovery outage.
|
||||
pub fn clear(&self, key: &DiscoveryKey) {
|
||||
self.with_inner(|map| {
|
||||
map.remove(key);
|
||||
});
|
||||
}
|
||||
|
||||
/// Replaces an entry with a deliberately wrong address, simulating a stale
|
||||
/// or poisoned record.
|
||||
pub fn insert_raw(&self, key: DiscoveryKey, addr: EndpointAddr) {
|
||||
self.with_inner(|map| {
|
||||
map.entry(key).or_default().insert(addr.id, addr);
|
||||
});
|
||||
}
|
||||
|
||||
fn with_inner<T>(
|
||||
&self,
|
||||
f: impl FnOnce(&mut HashMap<DiscoveryKey, HashMap<EndpointId, EndpointAddr>>) -> T,
|
||||
) -> T {
|
||||
let mut guard = match self.inner.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
f(&mut guard)
|
||||
}
|
||||
}
|
||||
|
||||
impl NetworkDiscovery for SharedMemoryDiscovery {
|
||||
fn name(&self) -> &str {
|
||||
"shared-memory"
|
||||
}
|
||||
|
||||
fn publish<'a>(&'a self, key: DiscoveryKey, addr: EndpointAddr) -> BoxFuture<'a, Result<()>> {
|
||||
self.with_inner(|map| {
|
||||
map.entry(key).or_default().insert(addr.id, addr);
|
||||
});
|
||||
Box::pin(async { Ok(()) })
|
||||
}
|
||||
|
||||
fn unpublish<'a>(
|
||||
&'a self,
|
||||
key: DiscoveryKey,
|
||||
endpoint: EndpointId,
|
||||
) -> BoxFuture<'a, Result<()>> {
|
||||
self.with_inner(|map| {
|
||||
if let Some(entries) = map.get_mut(&key) {
|
||||
entries.remove(&endpoint);
|
||||
if entries.is_empty() {
|
||||
map.remove(&key);
|
||||
}
|
||||
}
|
||||
});
|
||||
Box::pin(async { Ok(()) })
|
||||
}
|
||||
|
||||
fn resolve<'a>(&'a self, key: DiscoveryKey) -> BoxFuture<'a, Result<Vec<Candidate>>> {
|
||||
let candidates: Vec<Candidate> = self.with_inner(|map| {
|
||||
map.get(&key)
|
||||
.map(|entries| {
|
||||
entries
|
||||
.values()
|
||||
.cloned()
|
||||
.map(|addr| Candidate::new(addr, CandidateSource::Discovery))
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default()
|
||||
});
|
||||
Box::pin(async move { Ok(candidates) })
|
||||
}
|
||||
}
|
||||
|
||||
/// Combines several backends, concatenating their candidates.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CompositeDiscovery {
|
||||
backends: Vec<Arc<dyn NetworkDiscovery>>,
|
||||
}
|
||||
|
||||
impl CompositeDiscovery {
|
||||
/// Creates a composite over the given backends.
|
||||
pub fn new(backends: impl IntoIterator<Item = Arc<dyn NetworkDiscovery>>) -> Self {
|
||||
Self {
|
||||
backends: backends.into_iter().collect(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl NetworkDiscovery for CompositeDiscovery {
|
||||
fn name(&self) -> &str {
|
||||
"composite"
|
||||
}
|
||||
|
||||
fn publish<'a>(&'a self, key: DiscoveryKey, addr: EndpointAddr) -> BoxFuture<'a, Result<()>> {
|
||||
Box::pin(async move {
|
||||
for backend in &self.backends {
|
||||
// One failing backend must not stop the others.
|
||||
if let Err(err) = backend.publish(key, addr.clone()).await {
|
||||
tracing::debug!(backend = backend.name(), %err, "publish failed");
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
fn unpublish<'a>(
|
||||
&'a self,
|
||||
key: DiscoveryKey,
|
||||
endpoint: EndpointId,
|
||||
) -> BoxFuture<'a, Result<()>> {
|
||||
Box::pin(async move {
|
||||
for backend in &self.backends {
|
||||
if let Err(err) = backend.unpublish(key, endpoint).await {
|
||||
tracing::debug!(backend = backend.name(), %err, "unpublish failed");
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
fn resolve<'a>(&'a self, key: DiscoveryKey) -> BoxFuture<'a, Result<Vec<Candidate>>> {
|
||||
Box::pin(async move {
|
||||
let mut out = Vec::new();
|
||||
for backend in &self.backends {
|
||||
match backend.resolve(key).await {
|
||||
Ok(mut found) => out.append(&mut found),
|
||||
Err(err) => {
|
||||
tracing::debug!(backend = backend.name(), %err, "resolve failed");
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
})
|
||||
}
|
||||
}
|
||||
+191
@@ -0,0 +1,191 @@
|
||||
//! Error types for the whole library.
|
||||
//!
|
||||
//! The library never panics on untrusted network input: every decoding and
|
||||
//! validation failure is represented as a [`ProtocolError`] and surfaced as a
|
||||
//! rejected message or session, never as a process abort.
|
||||
|
||||
use std::path::PathBuf;
|
||||
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
/// Convenient result alias used across the crate.
|
||||
pub type Result<T, E = Error> = std::result::Result<T, E>;
|
||||
|
||||
/// Top level error type of the library.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[non_exhaustive]
|
||||
pub enum Error {
|
||||
/// The supplied network name does not satisfy the documented rules.
|
||||
#[error("invalid network name: {0}")]
|
||||
InvalidNetworkName(&'static str),
|
||||
|
||||
/// The supplied network secret does not satisfy the documented rules.
|
||||
///
|
||||
/// The secret itself is never included in the message.
|
||||
#[error("invalid network secret: {0}")]
|
||||
InvalidNetworkSecret(&'static str),
|
||||
|
||||
/// A textual identifier could not be parsed.
|
||||
#[error("invalid {kind}: {reason}")]
|
||||
InvalidEncoding {
|
||||
/// What was being parsed, e.g. `network id`.
|
||||
kind: &'static str,
|
||||
/// Why parsing failed.
|
||||
reason: &'static str,
|
||||
},
|
||||
|
||||
/// The mandatory state directory is already owned by another live agent.
|
||||
///
|
||||
/// This is an ownership lock, not a "file exists" check.
|
||||
#[error("state directory {path} is owned by another running agent instance")]
|
||||
StateLocked {
|
||||
/// Directory that could not be locked.
|
||||
path: PathBuf,
|
||||
},
|
||||
|
||||
/// The mandatory state store is unusable. It is never silently recreated.
|
||||
#[error("mandatory state store at {path} is unusable and was NOT reset: {reason}")]
|
||||
StateCorrupted {
|
||||
/// Path of the unusable store.
|
||||
path: PathBuf,
|
||||
/// Human readable reason, free of secrets.
|
||||
reason: String,
|
||||
},
|
||||
|
||||
/// The mandatory state store has a schema this build cannot handle.
|
||||
#[error(
|
||||
"state store schema version {found} is not supported by this build (supported: {supported})"
|
||||
)]
|
||||
UnsupportedSchema {
|
||||
/// Version found on disk.
|
||||
found: i64,
|
||||
/// Version this build writes.
|
||||
supported: i64,
|
||||
},
|
||||
|
||||
/// A storage operation failed.
|
||||
#[error("storage error: {0}")]
|
||||
Storage(String),
|
||||
|
||||
/// A filesystem operation failed.
|
||||
#[error("io error at {path}: {source}")]
|
||||
Io {
|
||||
/// Path involved in the failure.
|
||||
path: PathBuf,
|
||||
/// Underlying error.
|
||||
#[source]
|
||||
source: std::io::Error,
|
||||
},
|
||||
|
||||
/// Binding or driving the iroh endpoint failed.
|
||||
#[error("iroh endpoint error: {0}")]
|
||||
Endpoint(String),
|
||||
|
||||
/// A control protocol violation.
|
||||
#[error(transparent)]
|
||||
Protocol(#[from] ProtocolError),
|
||||
|
||||
/// The requested network is not currently active on this agent.
|
||||
#[error("network {0} is not active")]
|
||||
NetworkNotActive(NetworkId),
|
||||
|
||||
/// The requested network is already active on this agent.
|
||||
#[error("network {0} is already active")]
|
||||
NetworkAlreadyActive(NetworkId),
|
||||
|
||||
/// The requested network is not configured in the state store.
|
||||
#[error("network {0} is not configured")]
|
||||
NetworkUnknown(NetworkId),
|
||||
|
||||
/// No session with that peer exists in the given network.
|
||||
#[error("no authenticated session with peer {peer} in network {network}")]
|
||||
NoSuchPeer {
|
||||
/// Network the lookup was scoped to.
|
||||
network: NetworkId,
|
||||
/// Peer that was looked up, short form.
|
||||
peer: String,
|
||||
},
|
||||
|
||||
/// The agent is shutting down or already stopped.
|
||||
#[error("agent is stopped")]
|
||||
Stopped,
|
||||
|
||||
/// Discovery backend failure. Never fatal for the agent.
|
||||
#[error("discovery error: {0}")]
|
||||
Discovery(String),
|
||||
}
|
||||
|
||||
/// Errors produced while speaking the control protocol.
|
||||
///
|
||||
/// These always result in rejecting a single message or a single session. They
|
||||
/// never stop other networks, other peers, or the agent itself.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
|
||||
#[non_exhaustive]
|
||||
pub enum ProtocolError {
|
||||
/// The peer announced an unsupported control protocol version.
|
||||
#[error("unsupported control protocol version {found} (this build speaks {supported})")]
|
||||
UnsupportedVersion {
|
||||
/// Version announced by the peer.
|
||||
found: u16,
|
||||
/// Version this build speaks.
|
||||
supported: u16,
|
||||
},
|
||||
|
||||
/// A frame header announced more bytes than the configured limit allows.
|
||||
///
|
||||
/// Checked *before* any buffer of that size is allocated.
|
||||
#[error("frame of {announced} bytes exceeds the {limit} byte limit")]
|
||||
FrameTooLarge {
|
||||
/// Length announced in the frame header.
|
||||
announced: u64,
|
||||
/// Configured limit.
|
||||
limit: usize,
|
||||
},
|
||||
|
||||
/// A frame could not be decoded.
|
||||
#[error("malformed frame: {0}")]
|
||||
Malformed(&'static str),
|
||||
|
||||
/// The stream ended before a complete frame was read.
|
||||
#[error("stream closed while reading a frame")]
|
||||
StreamClosed,
|
||||
|
||||
/// An underlying stream read or write failed.
|
||||
#[error("stream error: {0}")]
|
||||
Stream(String),
|
||||
|
||||
/// The peer failed to prove knowledge of the derived network secret.
|
||||
#[error("network authentication failed")]
|
||||
AuthenticationFailed,
|
||||
|
||||
/// The peer asked for a network this agent does not have active.
|
||||
#[error("peer requested an unknown or inactive network")]
|
||||
UnknownNetwork,
|
||||
|
||||
/// A message carried a network id different from the session's network.
|
||||
#[error("message network id does not match the authenticated session network")]
|
||||
NetworkMismatch,
|
||||
|
||||
/// A regular control message arrived before the handshake completed.
|
||||
#[error("control message received before authentication completed")]
|
||||
NotAuthenticated,
|
||||
|
||||
/// A handshake step did not complete within the configured timeout.
|
||||
#[error("handshake timed out")]
|
||||
HandshakeTimeout,
|
||||
|
||||
/// A field exceeded its configured bound.
|
||||
#[error("field `{field}` exceeds its limit ({len} > {limit})")]
|
||||
FieldTooLarge {
|
||||
/// Name of the offending field.
|
||||
field: &'static str,
|
||||
/// Observed length.
|
||||
len: usize,
|
||||
/// Configured limit.
|
||||
limit: usize,
|
||||
},
|
||||
|
||||
/// The connection is not usable for deriving channel binding material.
|
||||
#[error("connection does not provide TLS exporter material: {0}")]
|
||||
NoChannelBinding(String),
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
//! Device identity and network space identity.
|
||||
//!
|
||||
//! These are two independent things and must not be confused:
|
||||
//!
|
||||
//! * [`DeviceIdentity`] wraps the persistent iroh [`SecretKey`]. Its public key
|
||||
//! *is* the iroh [`EndpointId`]. It survives restarts and survives a change of
|
||||
//! the network secret.
|
||||
//! * [`NetworkId`] identifies a network space and is derived purely from the
|
||||
//! network name and shared secret. It is unrelated to any device key.
|
||||
|
||||
mod network;
|
||||
|
||||
pub use network::{
|
||||
DiscoveryKey, IDENTITY_SCHEME, MAX_NETWORK_NAME_LEN, MAX_NETWORK_SECRET_LEN,
|
||||
MIN_NETWORK_SECRET_LEN, NetworkDescriptor, NetworkId, NetworkKeys, NetworkName, NetworkSecret,
|
||||
SECRET_TEXT_PREFIX,
|
||||
};
|
||||
|
||||
use iroh::{EndpointId, SecretKey};
|
||||
|
||||
/// The persistent identity of this device.
|
||||
///
|
||||
/// Created once and stored in the mandatory state store. Restarting the agent
|
||||
/// must not produce a new peer, so the stored secret key is always reused.
|
||||
/// Corruption of the stored key is reported as an error and never silently
|
||||
/// replaced by a fresh key.
|
||||
#[derive(Clone)]
|
||||
pub struct DeviceIdentity {
|
||||
secret: SecretKey,
|
||||
}
|
||||
|
||||
impl DeviceIdentity {
|
||||
/// Generates a brand new device identity.
|
||||
pub fn generate() -> Self {
|
||||
Self {
|
||||
secret: SecretKey::generate(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Reconstructs a device identity from its stored 32 secret key bytes.
|
||||
pub fn from_secret_bytes(bytes: &[u8; 32]) -> Self {
|
||||
Self {
|
||||
secret: SecretKey::from_bytes(bytes),
|
||||
}
|
||||
}
|
||||
|
||||
/// The iroh endpoint id, i.e. the public key of this device.
|
||||
pub fn endpoint_id(&self) -> EndpointId {
|
||||
self.secret.public()
|
||||
}
|
||||
|
||||
/// The raw secret key bytes, for persistence only.
|
||||
pub(crate) fn secret_bytes(&self) -> [u8; 32] {
|
||||
self.secret.to_bytes()
|
||||
}
|
||||
|
||||
/// A clone of the iroh secret key, for endpoint construction only.
|
||||
pub(crate) fn secret_key(&self) -> SecretKey {
|
||||
self.secret.clone()
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for DeviceIdentity {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("DeviceIdentity")
|
||||
.field("endpoint_id", &self.endpoint_id().fmt_short().to_string())
|
||||
.field("secret", &"<redacted>")
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,419 @@
|
||||
//! Deterministic network space identity.
|
||||
//!
|
||||
//! A *network space* is fully determined by a [`NetworkName`] and a
|
||||
//! [`NetworkSecret`]. Two agents that were given the same pair derive the same
|
||||
//! [`NetworkId`], [`DiscoveryKey`] and handshake authentication key, with no
|
||||
//! coordination, no creator, no timestamp and no leader election.
|
||||
//!
|
||||
//! # Derivation scheme (`tsunagi-network-id-v1`)
|
||||
//!
|
||||
//! All inputs are encoded with an unambiguous length-prefixed encoding, written
|
||||
//! here as `LP(x) = u32_be(x.len()) || x`. String concatenation is never used.
|
||||
//!
|
||||
//! ```text
|
||||
//! salt = SHA-256( LP("tsunagi-network-id-v1") || LP(name_utf8) )
|
||||
//! prk = HKDF-SHA256-Extract(salt, ikm = secret_bytes)
|
||||
//! info(label) = LP("tsunagi-network-id-v1") || LP(label)
|
||||
//! network_id = HKDF-Expand(prk, info("network-id"), 32)
|
||||
//! discovery_key = HKDF-Expand(prk, info("discovery-key"), 32)
|
||||
//! auth_key = HKDF-Expand(prk, info("handshake-auth"), 32)
|
||||
//! ```
|
||||
//!
|
||||
//! HKDF's `info` parameter is what separates the three derived values
|
||||
//! (RFC 5869 §3.2). Learning `discovery_key` — which is published to a
|
||||
//! discovery backend and is therefore semi-public — does not reveal `auth_key`,
|
||||
//! so the discovery key must never be used as a password or bearer token.
|
||||
//!
|
||||
//! The scheme label is versioned and frozen. Upgrading this crate or bumping
|
||||
//! the control protocol version must not change an existing [`NetworkId`].
|
||||
|
||||
use hkdf::Hkdf;
|
||||
use sha2::{Digest, Sha256};
|
||||
use zeroize::{Zeroize, Zeroizing};
|
||||
|
||||
use crate::error::{Error, Result};
|
||||
|
||||
/// Frozen label identifying the network identity derivation scheme.
|
||||
///
|
||||
/// Changing this string creates a different, incompatible network space for the
|
||||
/// same name and secret. It must never be changed casually.
|
||||
pub const IDENTITY_SCHEME: &str = "tsunagi-network-id-v1";
|
||||
|
||||
/// Maximum length of a network name in UTF-8 bytes.
|
||||
pub const MAX_NETWORK_NAME_LEN: usize = 64;
|
||||
|
||||
/// Minimum length of a network secret in bytes.
|
||||
///
|
||||
/// The proof-of-concept targets high-entropy shared secrets. See
|
||||
/// [`NetworkSecret::generate`].
|
||||
pub const MIN_NETWORK_SECRET_LEN: usize = 16;
|
||||
|
||||
/// Maximum length of a network secret in bytes.
|
||||
pub const MAX_NETWORK_SECRET_LEN: usize = 1024;
|
||||
|
||||
/// Human-readable prefix of the canonical secret text encoding.
|
||||
pub const SECRET_TEXT_PREFIX: &str = "tsn1";
|
||||
|
||||
/// Appends `LP(bytes) = u32_be(len) || bytes` to `out`.
|
||||
///
|
||||
/// Panics are impossible here: the caller-provided slices are already bounded by
|
||||
/// [`MAX_NETWORK_NAME_LEN`] / [`MAX_NETWORK_SECRET_LEN`], and the cast is
|
||||
/// saturating for anything larger.
|
||||
fn push_lp(out: &mut Vec<u8>, bytes: &[u8]) {
|
||||
let len = u32::try_from(bytes.len()).unwrap_or(u32::MAX);
|
||||
out.extend_from_slice(&len.to_be_bytes());
|
||||
out.extend_from_slice(bytes);
|
||||
}
|
||||
|
||||
/// The name half of a network space.
|
||||
///
|
||||
/// Rules, deliberately strict and never silently applied:
|
||||
///
|
||||
/// * 1..=[`MAX_NETWORK_NAME_LEN`] bytes of UTF-8.
|
||||
/// * No ASCII control characters.
|
||||
/// * No leading or trailing ASCII whitespace — such a name is **rejected**,
|
||||
/// not trimmed.
|
||||
/// * Used verbatim. No case folding and no Unicode normalisation is performed,
|
||||
/// so `Home` and `home` are different network spaces.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
|
||||
pub struct NetworkName(String);
|
||||
|
||||
impl NetworkName {
|
||||
/// Validates and wraps a network name.
|
||||
pub fn new(name: impl Into<String>) -> Result<Self> {
|
||||
let name = name.into();
|
||||
if name.is_empty() {
|
||||
return Err(Error::InvalidNetworkName("must not be empty"));
|
||||
}
|
||||
if name.len() > MAX_NETWORK_NAME_LEN {
|
||||
return Err(Error::InvalidNetworkName(
|
||||
"must not exceed 64 bytes of UTF-8",
|
||||
));
|
||||
}
|
||||
if name.chars().any(|c| c.is_control()) {
|
||||
return Err(Error::InvalidNetworkName(
|
||||
"must not contain control characters",
|
||||
));
|
||||
}
|
||||
let trimmed = name.trim_matches(|c: char| c.is_ascii_whitespace());
|
||||
if trimmed.len() != name.len() {
|
||||
return Err(Error::InvalidNetworkName(
|
||||
"must not have leading or trailing ASCII whitespace",
|
||||
));
|
||||
}
|
||||
Ok(Self(name))
|
||||
}
|
||||
|
||||
/// Returns the name as a string slice.
|
||||
pub fn as_str(&self) -> &str {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for NetworkName {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str(&self.0)
|
||||
}
|
||||
}
|
||||
|
||||
impl std::str::FromStr for NetworkName {
|
||||
type Err = Error;
|
||||
fn from_str(s: &str) -> Result<Self> {
|
||||
Self::new(s)
|
||||
}
|
||||
}
|
||||
|
||||
/// The shared secret half of a network space.
|
||||
///
|
||||
/// This is the single shared secret the end user configures; "password" and
|
||||
/// "secret" refer to the same value. The bytes are used **verbatim**: never
|
||||
/// trimmed, case-folded, normalised or truncated.
|
||||
///
|
||||
/// The value is zeroized on drop and redacted from [`Debug`].
|
||||
#[derive(Clone, PartialEq, Eq)]
|
||||
pub struct NetworkSecret(Zeroizing<Vec<u8>>);
|
||||
|
||||
impl NetworkSecret {
|
||||
/// Wraps raw secret bytes.
|
||||
///
|
||||
/// Requires at least [`MIN_NETWORK_SECRET_LEN`] bytes. This crate makes no
|
||||
/// security promises for short, low-entropy human passphrases: there is no
|
||||
/// PAKE here, so an offline guessing attack against a weak secret is cheap
|
||||
/// for anyone who can reach the handshake.
|
||||
pub fn from_bytes(bytes: impl Into<Vec<u8>>) -> Result<Self> {
|
||||
let bytes = Zeroizing::new(bytes.into());
|
||||
if bytes.len() < MIN_NETWORK_SECRET_LEN {
|
||||
return Err(Error::InvalidNetworkSecret(
|
||||
"must be at least 16 bytes; use NetworkSecret::generate()",
|
||||
));
|
||||
}
|
||||
if bytes.len() > MAX_NETWORK_SECRET_LEN {
|
||||
return Err(Error::InvalidNetworkSecret("must not exceed 1024 bytes"));
|
||||
}
|
||||
Ok(Self(bytes))
|
||||
}
|
||||
|
||||
/// Generates a fresh 32-byte random secret.
|
||||
///
|
||||
/// This is the recommended way to create a network secret.
|
||||
pub fn generate() -> Self {
|
||||
let mut buf = vec![0u8; 32];
|
||||
rand::fill(&mut buf[..]);
|
||||
Self(Zeroizing::new(buf))
|
||||
}
|
||||
|
||||
/// Parses the canonical text form produced by [`NetworkSecret::encode`].
|
||||
///
|
||||
/// The format is `tsn1` followed by lowercase unpadded RFC 4648 base32.
|
||||
/// Parsing is strict: no whitespace, no case mixing in the payload.
|
||||
pub fn decode(text: &str) -> Result<Self> {
|
||||
let payload = text
|
||||
.strip_prefix(SECRET_TEXT_PREFIX)
|
||||
.ok_or(Error::InvalidNetworkSecret(
|
||||
"canonical secrets start with `tsn1`",
|
||||
))?;
|
||||
let bytes = data_encoding::BASE32_NOPAD
|
||||
.decode(payload.to_ascii_uppercase().as_bytes())
|
||||
.map_err(|_| Error::InvalidNetworkSecret("not valid base32"))?;
|
||||
Self::from_bytes(bytes)
|
||||
}
|
||||
|
||||
/// Encodes the secret in its canonical text form.
|
||||
///
|
||||
/// The returned string is zeroized on drop. Never log it.
|
||||
pub fn encode(&self) -> Zeroizing<String> {
|
||||
let mut encoded = data_encoding::BASE32_NOPAD.encode(&self.0);
|
||||
encoded.make_ascii_lowercase();
|
||||
let out = Zeroizing::new(format!("{SECRET_TEXT_PREFIX}{encoded}"));
|
||||
encoded.zeroize();
|
||||
out
|
||||
}
|
||||
|
||||
/// Exposes the raw secret bytes.
|
||||
///
|
||||
/// Callers must not log, serialise or copy these bytes into diagnostics.
|
||||
pub(crate) fn expose(&self) -> &[u8] {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for NetworkSecret {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str("NetworkSecret(<redacted>)")
|
||||
}
|
||||
}
|
||||
|
||||
/// Encodes 32 bytes as lowercase unpadded base32.
|
||||
fn b32(bytes: &[u8; 32]) -> String {
|
||||
let mut s = data_encoding::BASE32_NOPAD.encode(bytes);
|
||||
s.make_ascii_lowercase();
|
||||
s
|
||||
}
|
||||
|
||||
/// Decodes lowercase unpadded base32 into 32 bytes.
|
||||
fn unb32(kind: &'static str, s: &str) -> Result<[u8; 32]> {
|
||||
let bytes = data_encoding::BASE32_NOPAD
|
||||
.decode(s.to_ascii_uppercase().as_bytes())
|
||||
.map_err(|_| Error::InvalidEncoding {
|
||||
kind,
|
||||
reason: "not valid base32",
|
||||
})?;
|
||||
<[u8; 32]>::try_from(bytes.as_slice()).map_err(|_| Error::InvalidEncoding {
|
||||
kind,
|
||||
reason: "expected 32 bytes",
|
||||
})
|
||||
}
|
||||
|
||||
/// Public, non-secret identifier of a network space.
|
||||
///
|
||||
/// Safe to log, publish and put into status output. It does not authorise
|
||||
/// anything on its own: an attacker who knows a `NetworkId` still cannot pass
|
||||
/// the handshake without the secret.
|
||||
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
|
||||
pub struct NetworkId([u8; 32]);
|
||||
|
||||
impl NetworkId {
|
||||
/// Returns the raw 32 bytes.
|
||||
pub fn as_bytes(&self) -> &[u8; 32] {
|
||||
&self.0
|
||||
}
|
||||
|
||||
/// Builds a network id from raw bytes.
|
||||
pub fn from_bytes(bytes: [u8; 32]) -> Self {
|
||||
Self(bytes)
|
||||
}
|
||||
|
||||
/// Returns a short prefix useful for logs.
|
||||
pub fn fmt_short(&self) -> String {
|
||||
b32(&self.0).chars().take(10).collect()
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for NetworkId {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str(&b32(&self.0))
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for NetworkId {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(f, "NetworkId({})", self.fmt_short())
|
||||
}
|
||||
}
|
||||
|
||||
impl std::str::FromStr for NetworkId {
|
||||
type Err = Error;
|
||||
fn from_str(s: &str) -> Result<Self> {
|
||||
unb32("network id", s).map(Self)
|
||||
}
|
||||
}
|
||||
|
||||
/// Lookup key used to find candidates for a network in a discovery backend.
|
||||
///
|
||||
/// Derived from the secret, so it is not published in the clear the way a
|
||||
/// [`NetworkId`] is. It is nevertheless **not** a credential: a discovery
|
||||
/// backend, or anyone observing it, learns nothing that helps pass the
|
||||
/// handshake.
|
||||
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
|
||||
pub struct DiscoveryKey([u8; 32]);
|
||||
|
||||
impl DiscoveryKey {
|
||||
/// Returns the raw 32 bytes.
|
||||
pub fn as_bytes(&self) -> &[u8; 32] {
|
||||
&self.0
|
||||
}
|
||||
|
||||
/// Builds a discovery key from raw bytes.
|
||||
pub fn from_bytes(bytes: [u8; 32]) -> Self {
|
||||
Self(bytes)
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for DiscoveryKey {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str(&b32(&self.0))
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for DiscoveryKey {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(
|
||||
f,
|
||||
"DiscoveryKey({}…)",
|
||||
b32(&self.0).chars().take(10).collect::<String>()
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// The deterministic, immutable description of a network space.
|
||||
///
|
||||
/// There is no competing genesis: this value contains no creator identity, no
|
||||
/// creation time and no owner signature, so two agents started independently
|
||||
/// with the same parameters produce byte-identical descriptors. The secret is
|
||||
/// never part of it.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct NetworkDescriptor {
|
||||
/// Frozen derivation scheme label, see [`IDENTITY_SCHEME`].
|
||||
pub scheme: &'static str,
|
||||
/// The network name.
|
||||
pub name: NetworkName,
|
||||
/// The derived public network identifier.
|
||||
pub network_id: NetworkId,
|
||||
}
|
||||
|
||||
impl NetworkDescriptor {
|
||||
/// Canonical, unambiguous byte encoding of the descriptor.
|
||||
pub fn to_canonical_bytes(&self) -> Vec<u8> {
|
||||
let mut out = Vec::new();
|
||||
push_lp(&mut out, self.scheme.as_bytes());
|
||||
push_lp(&mut out, self.name.as_str().as_bytes());
|
||||
push_lp(&mut out, &self.network_id.0);
|
||||
out
|
||||
}
|
||||
}
|
||||
|
||||
/// All key material derived from a [`NetworkName`] and [`NetworkSecret`].
|
||||
///
|
||||
/// The authentication key is zeroized on drop and never leaves this crate.
|
||||
#[derive(Clone)]
|
||||
pub struct NetworkKeys {
|
||||
network_id: NetworkId,
|
||||
discovery_key: DiscoveryKey,
|
||||
auth_key: Zeroizing<[u8; 32]>,
|
||||
name: NetworkName,
|
||||
}
|
||||
|
||||
impl NetworkKeys {
|
||||
/// Derives all network key material.
|
||||
///
|
||||
/// This is a pure function of `(name, secret)`. It does not depend on the
|
||||
/// device key, the hostname, the wall clock or the order in which agents
|
||||
/// start.
|
||||
pub fn derive(name: &NetworkName, secret: &NetworkSecret) -> Self {
|
||||
let mut salt_input = Vec::new();
|
||||
push_lp(&mut salt_input, IDENTITY_SCHEME.as_bytes());
|
||||
push_lp(&mut salt_input, name.as_str().as_bytes());
|
||||
let salt = Sha256::digest(&salt_input);
|
||||
|
||||
let hk = Hkdf::<Sha256>::new(Some(&salt), secret.expose());
|
||||
|
||||
let expand = |label: &str| -> [u8; 32] {
|
||||
let mut info = Vec::new();
|
||||
push_lp(&mut info, IDENTITY_SCHEME.as_bytes());
|
||||
push_lp(&mut info, label.as_bytes());
|
||||
let mut okm = [0u8; 32];
|
||||
// 32 bytes is far below HKDF-SHA256's 255*32 limit, so this cannot fail.
|
||||
match hk.expand(&info, &mut okm) {
|
||||
Ok(()) => okm,
|
||||
Err(_) => unreachable!("HKDF-SHA256 expand of 32 bytes cannot fail"),
|
||||
}
|
||||
};
|
||||
|
||||
Self {
|
||||
network_id: NetworkId(expand("network-id")),
|
||||
discovery_key: DiscoveryKey(expand("discovery-key")),
|
||||
auth_key: Zeroizing::new(expand("handshake-auth")),
|
||||
name: name.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
/// The public network identifier.
|
||||
pub fn network_id(&self) -> NetworkId {
|
||||
self.network_id
|
||||
}
|
||||
|
||||
/// The discovery lookup key.
|
||||
pub fn discovery_key(&self) -> DiscoveryKey {
|
||||
self.discovery_key
|
||||
}
|
||||
|
||||
/// The network name.
|
||||
pub fn name(&self) -> &NetworkName {
|
||||
&self.name
|
||||
}
|
||||
|
||||
/// The immutable descriptor of this network space.
|
||||
pub fn descriptor(&self) -> NetworkDescriptor {
|
||||
NetworkDescriptor {
|
||||
scheme: IDENTITY_SCHEME,
|
||||
name: self.name.clone(),
|
||||
network_id: self.network_id,
|
||||
}
|
||||
}
|
||||
|
||||
/// The handshake authentication key. Crate-internal on purpose.
|
||||
pub(crate) fn auth_key(&self) -> &[u8; 32] {
|
||||
&self.auth_key
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for NetworkKeys {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("NetworkKeys")
|
||||
.field("name", &self.name)
|
||||
.field("network_id", &self.network_id)
|
||||
.field("discovery_key", &self.discovery_key)
|
||||
.field("auth_key", &"<redacted>")
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
+105
@@ -0,0 +1,105 @@
|
||||
//! Tsunagi: a proof-of-concept agent for small private mesh networks.
|
||||
//!
|
||||
//! See `README.md` for the exact scope of this proof of concept, and
|
||||
//! `docs/architecture.md` / `docs/protocol.md` for the design.
|
||||
//!
|
||||
//! # Shape of the library
|
||||
//!
|
||||
//! * [`identity`] — persistent device identity and deterministic network space
|
||||
//! identity.
|
||||
//! * [`storage`] — mandatory state (`state.sqlite`) and separately disposable
|
||||
//! cache (`cache.sqlite`).
|
||||
//! * [`discovery`] — pluggable sources of *candidate* addresses. Candidates are
|
||||
//! never trusted peers.
|
||||
//! * [`proto`] — the control protocol: framing, messages, handshake.
|
||||
//! * [`net`] — the iroh connectivity adapter and its observability surface.
|
||||
//! * [`agent`] — the runtime: agent lifecycle, per-network runtimes, reconnect.
|
||||
//! * [`dataplane`] — the minimal contract future IP plugins must satisfy.
|
||||
//!
|
||||
//! # What this library deliberately does not do
|
||||
//!
|
||||
//! It never starts a global tokio runtime, never installs a global tracing
|
||||
//! subscriber, never handles process signals, never forks and never calls
|
||||
//! `process::exit`. Several independent agents can run in one process.
|
||||
//!
|
||||
//! Only control messages travel over iroh. User IP traffic is not tunnelled
|
||||
//! through it.
|
||||
|
||||
#![deny(rustdoc::broken_intra_doc_links)]
|
||||
|
||||
pub mod agent;
|
||||
pub mod config;
|
||||
pub mod dataplane;
|
||||
pub mod discovery;
|
||||
pub mod error;
|
||||
pub mod identity;
|
||||
pub mod net;
|
||||
pub mod proto;
|
||||
pub mod storage;
|
||||
|
||||
pub use agent::{Agent, AgentStatus, Event, NetworkStatus, PeerStatus};
|
||||
pub use config::{AgentConfig, Limits, ReconnectPolicy, StoragePaths, TransportPolicy};
|
||||
pub use error::{Error, ProtocolError, Result};
|
||||
pub use identity::{
|
||||
DeviceIdentity, DiscoveryKey, NetworkDescriptor, NetworkId, NetworkName, NetworkSecret,
|
||||
};
|
||||
|
||||
/// Re-exported iroh types that appear in this crate's public API.
|
||||
pub mod iroh_types {
|
||||
pub use iroh::{EndpointAddr, EndpointId, RelayUrl};
|
||||
}
|
||||
|
||||
#[doc(hidden)]
|
||||
pub mod test_support {
|
||||
//! Internals exposed for this crate's own negative tests.
|
||||
//!
|
||||
//! **Not part of the stable API.** It exists so the integration tests can
|
||||
//! hand-craft handshakes — a valid proof to replay on another connection, a
|
||||
//! message sent before authentication, an oversized frame — which is the
|
||||
//! only way to test those rejections against a real agent.
|
||||
|
||||
use iroh::endpoint::Connection;
|
||||
|
||||
use crate::error::ProtocolError;
|
||||
use crate::identity::NetworkKeys;
|
||||
use crate::proto::handshake;
|
||||
|
||||
/// Exposes the derived handshake authentication key.
|
||||
pub fn auth_key(keys: &NetworkKeys) -> [u8; 32] {
|
||||
*keys.auth_key()
|
||||
}
|
||||
|
||||
/// Derives this connection's channel binding material.
|
||||
pub fn channel_binding(
|
||||
conn: &Connection,
|
||||
network_id: &[u8; 32],
|
||||
) -> Result<[u8; 32], ProtocolError> {
|
||||
handshake::channel_binding_for_test(conn, network_id)
|
||||
}
|
||||
|
||||
/// Computes a handshake proof for the given role.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn compute_proof(
|
||||
auth_key: &[u8; 32],
|
||||
role: &str,
|
||||
version: u16,
|
||||
network_id: &[u8; 32],
|
||||
initiator: &[u8; 32],
|
||||
responder: &[u8; 32],
|
||||
channel_binding: &[u8],
|
||||
nonce_initiator: &[u8; 16],
|
||||
nonce_responder: &[u8; 16],
|
||||
) -> [u8; 32] {
|
||||
handshake::proof_for_test(
|
||||
auth_key,
|
||||
role,
|
||||
version,
|
||||
network_id,
|
||||
initiator,
|
||||
responder,
|
||||
channel_binding,
|
||||
nonce_initiator,
|
||||
nonce_responder,
|
||||
)
|
||||
}
|
||||
}
|
||||
+304
@@ -0,0 +1,304 @@
|
||||
//! The iroh connectivity adapter and its observability surface.
|
||||
//!
|
||||
//! This module builds and owns the [`Endpoint`], and turns iroh's runtime
|
||||
//! information into plain owned snapshots that the rest of the library and the
|
||||
//! library's users can read.
|
||||
//!
|
||||
//! # Honest reporting
|
||||
//!
|
||||
//! Three different things are kept distinct and never conflated:
|
||||
//!
|
||||
//! * an **unverified candidate** — something discovery handed us
|
||||
//! ([`crate::discovery::Candidate`]);
|
||||
//! * an **observed address** — an address this endpoint believes it has
|
||||
//! ([`EndpointSnapshot::observed_addrs`]);
|
||||
//! * a **verified path** — a network path QUIC has actually validated and is
|
||||
//! using or can use ([`PathInfo`]).
|
||||
//!
|
||||
//! A value that is not available is reported as `None`. It is never invented.
|
||||
//!
|
||||
//! An iroh address is an address for *iroh*. It must not be assumed to be usable
|
||||
//! by any other protocol; a future WireGuard plugin is expected to collect its
|
||||
//! own reachability data.
|
||||
|
||||
use std::net::SocketAddr;
|
||||
use std::time::Duration;
|
||||
|
||||
use iroh::endpoint::{
|
||||
Connection, ConnectionStats, PortmapperConfig, RecvStream, SendStream, presets,
|
||||
};
|
||||
use iroh::{Endpoint, EndpointAddr, EndpointId, RelayMode};
|
||||
|
||||
use crate::config::{AgentConfig, TransportPolicy};
|
||||
use crate::error::{Error, Result};
|
||||
use crate::identity::DeviceIdentity;
|
||||
use crate::proto::message::ALPN;
|
||||
|
||||
/// A network path address as reported by iroh.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum PathAddr {
|
||||
/// A direct IP path.
|
||||
Ip(SocketAddr),
|
||||
/// A path through a relay server.
|
||||
Relay(String),
|
||||
/// A custom transport iroh reported but this crate does not model.
|
||||
Other(String),
|
||||
}
|
||||
|
||||
/// One verified network path of a connection.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PathInfo {
|
||||
/// Remote address of the path.
|
||||
pub remote: PathAddr,
|
||||
/// Local address of the path, when the OS reports one.
|
||||
pub local: Option<String>,
|
||||
/// Whether QUIC currently transmits application data over this path.
|
||||
pub is_selected: bool,
|
||||
/// Round-trip time estimate for this path.
|
||||
pub rtt: Duration,
|
||||
}
|
||||
|
||||
impl PathInfo {
|
||||
/// Whether this is a direct IP path.
|
||||
pub fn is_direct(&self) -> bool {
|
||||
matches!(self.remote, PathAddr::Ip(_))
|
||||
}
|
||||
|
||||
/// Whether this path goes through a relay.
|
||||
pub fn is_relay(&self) -> bool {
|
||||
matches!(self.remote, PathAddr::Relay(_))
|
||||
}
|
||||
}
|
||||
|
||||
/// How a connection currently reaches its peer.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum TransportKind {
|
||||
/// A direct IP path is selected.
|
||||
Direct,
|
||||
/// A relay path is selected.
|
||||
Relay,
|
||||
/// iroh has not reported a selected path yet.
|
||||
Unknown,
|
||||
}
|
||||
|
||||
/// Counters for one connection.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct ConnectionCounters {
|
||||
/// UDP bytes sent on this connection.
|
||||
pub udp_tx_bytes: u64,
|
||||
/// UDP bytes received on this connection.
|
||||
pub udp_rx_bytes: u64,
|
||||
/// UDP datagrams sent.
|
||||
pub udp_tx_datagrams: u64,
|
||||
/// UDP datagrams received.
|
||||
pub udp_rx_datagrams: u64,
|
||||
/// Packets declared lost.
|
||||
pub lost_packets: u64,
|
||||
}
|
||||
|
||||
impl From<ConnectionStats> for ConnectionCounters {
|
||||
fn from(stats: ConnectionStats) -> Self {
|
||||
Self {
|
||||
udp_tx_bytes: stats.udp_tx.bytes,
|
||||
udp_rx_bytes: stats.udp_rx.bytes,
|
||||
udp_tx_datagrams: stats.udp_tx.datagrams,
|
||||
udp_rx_datagrams: stats.udp_rx.datagrams,
|
||||
lost_packets: stats.lost_packets,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// An owned snapshot of one live connection.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ConnectionSnapshot {
|
||||
/// Authenticated endpoint id of the remote side.
|
||||
pub remote_id: EndpointId,
|
||||
/// Verified paths, as reported by iroh at snapshot time.
|
||||
pub paths: Vec<PathInfo>,
|
||||
/// How the connection currently reaches the peer.
|
||||
pub transport: TransportKind,
|
||||
/// RTT of the selected path, when there is one.
|
||||
pub rtt: Option<Duration>,
|
||||
/// Per-connection counters.
|
||||
pub counters: ConnectionCounters,
|
||||
}
|
||||
|
||||
/// Snapshot of this endpoint, independent of any particular network.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct EndpointSnapshot {
|
||||
/// This endpoint's id.
|
||||
pub endpoint_id: EndpointId,
|
||||
/// Sockets actually bound locally.
|
||||
pub bound_sockets: Vec<SocketAddr>,
|
||||
/// Addresses iroh believes this endpoint is reachable at.
|
||||
///
|
||||
/// These are *observed*, not verified by any remote peer.
|
||||
pub observed_addrs: Vec<PathAddr>,
|
||||
/// Relay URLs this endpoint currently considers usable, if any.
|
||||
pub relay_urls: Vec<String>,
|
||||
}
|
||||
|
||||
fn path_addr(addr: &iroh::TransportAddr) -> PathAddr {
|
||||
match addr {
|
||||
iroh::TransportAddr::Ip(socket) => PathAddr::Ip(*socket),
|
||||
iroh::TransportAddr::Relay(url) => PathAddr::Relay(url.to_string()),
|
||||
other => PathAddr::Other(format!("{other:?}")),
|
||||
}
|
||||
}
|
||||
|
||||
/// Builds an owned snapshot of a live connection.
|
||||
pub fn snapshot_connection(conn: &Connection) -> ConnectionSnapshot {
|
||||
let mut paths = Vec::new();
|
||||
let mut transport = TransportKind::Unknown;
|
||||
let mut rtt = None;
|
||||
|
||||
for path in conn.paths().iter() {
|
||||
let info = PathInfo {
|
||||
remote: path_addr(path.remote_addr()),
|
||||
local: local_addr_string(path.local_addr()),
|
||||
is_selected: path.is_selected(),
|
||||
rtt: path.rtt(),
|
||||
};
|
||||
if info.is_selected {
|
||||
transport = if info.is_relay() {
|
||||
TransportKind::Relay
|
||||
} else if info.is_direct() {
|
||||
TransportKind::Direct
|
||||
} else {
|
||||
TransportKind::Unknown
|
||||
};
|
||||
rtt = Some(info.rtt);
|
||||
}
|
||||
paths.push(info);
|
||||
}
|
||||
|
||||
ConnectionSnapshot {
|
||||
remote_id: conn.remote_id(),
|
||||
paths,
|
||||
transport,
|
||||
rtt,
|
||||
counters: conn.stats().into(),
|
||||
}
|
||||
}
|
||||
|
||||
fn local_addr_string(addr: &iroh::endpoint::LocalTransportAddr) -> Option<String> {
|
||||
match addr {
|
||||
iroh::endpoint::LocalTransportAddr::Ip(Some(ip)) => Some(ip.to_string()),
|
||||
iroh::endpoint::LocalTransportAddr::Ip(None) => None,
|
||||
iroh::endpoint::LocalTransportAddr::Relay(url) => Some(url.to_string()),
|
||||
iroh::endpoint::LocalTransportAddr::Custom(Some(custom)) => Some(format!("{custom:?}")),
|
||||
iroh::endpoint::LocalTransportAddr::Custom(None) => None,
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Thin wrapper around the iroh endpoint.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct EndpointAdapter {
|
||||
endpoint: Endpoint,
|
||||
}
|
||||
|
||||
impl EndpointAdapter {
|
||||
/// Binds an endpoint according to `config`, reusing the persistent device key.
|
||||
pub async fn bind(config: &AgentConfig, identity: &DeviceIdentity) -> Result<Self> {
|
||||
let mut builder = Endpoint::builder(presets::Minimal)
|
||||
.secret_key(identity.secret_key())
|
||||
.alpns(vec![ALPN.to_vec()]);
|
||||
|
||||
builder = match config.transport {
|
||||
TransportPolicy::LocalOnly => builder
|
||||
.relay_mode(RelayMode::Disabled)
|
||||
.clear_address_lookup()
|
||||
.portmapper_config(PortmapperConfig::Disabled)
|
||||
.net_report_config(iroh::NetReportConfig::minimal()),
|
||||
TransportPolicy::DirectOnly => builder.preset(presets::N0DisableRelay),
|
||||
TransportPolicy::N0Defaults => builder.preset(presets::N0),
|
||||
};
|
||||
|
||||
if !config.bind_addrs.is_empty() {
|
||||
builder = builder.clear_ip_transports();
|
||||
for addr in &config.bind_addrs {
|
||||
builder = builder
|
||||
.bind_addr(*addr)
|
||||
.map_err(|err| Error::Endpoint(format!("invalid bind address: {err}")))?;
|
||||
}
|
||||
}
|
||||
|
||||
let endpoint = builder
|
||||
.bind()
|
||||
.await
|
||||
.map_err(|err| Error::Endpoint(format!("cannot bind endpoint: {err}")))?;
|
||||
|
||||
Ok(Self { endpoint })
|
||||
}
|
||||
|
||||
/// The underlying iroh endpoint.
|
||||
pub fn endpoint(&self) -> &Endpoint {
|
||||
&self.endpoint
|
||||
}
|
||||
|
||||
/// This endpoint's id.
|
||||
pub fn endpoint_id(&self) -> EndpointId {
|
||||
self.endpoint.id()
|
||||
}
|
||||
|
||||
/// This endpoint's dialable address, as currently known.
|
||||
pub fn addr(&self) -> EndpointAddr {
|
||||
self.endpoint.addr()
|
||||
}
|
||||
|
||||
/// Builds an address containing only the locally bound sockets.
|
||||
///
|
||||
/// Useful when address lookup and relays are disabled and peers must be
|
||||
/// given literal addresses.
|
||||
pub fn loopback_addr(&self) -> EndpointAddr {
|
||||
self.endpoint
|
||||
.bound_sockets()
|
||||
.into_iter()
|
||||
.fold(EndpointAddr::new(self.endpoint.id()), |addr, socket| {
|
||||
addr.with_ip_addr(socket)
|
||||
})
|
||||
}
|
||||
|
||||
/// Snapshot of endpoint-level information.
|
||||
pub fn snapshot(&self) -> EndpointSnapshot {
|
||||
let addr = self.endpoint.addr();
|
||||
let mut observed = Vec::new();
|
||||
let mut relays = Vec::new();
|
||||
for socket in addr.ip_addrs() {
|
||||
observed.push(PathAddr::Ip(*socket));
|
||||
}
|
||||
for url in addr.relay_urls() {
|
||||
relays.push(url.to_string());
|
||||
observed.push(PathAddr::Relay(url.to_string()));
|
||||
}
|
||||
EndpointSnapshot {
|
||||
endpoint_id: self.endpoint.id(),
|
||||
bound_sockets: self.endpoint.bound_sockets(),
|
||||
observed_addrs: observed,
|
||||
relay_urls: relays,
|
||||
}
|
||||
}
|
||||
|
||||
/// Dials a candidate and opens the control stream.
|
||||
pub async fn connect(
|
||||
&self,
|
||||
addr: EndpointAddr,
|
||||
) -> Result<(Connection, SendStream, RecvStream), Error> {
|
||||
let conn = self
|
||||
.endpoint
|
||||
.connect(addr, ALPN)
|
||||
.await
|
||||
.map_err(|err| Error::Endpoint(format!("connect failed: {err}")))?;
|
||||
let (send, recv) = conn
|
||||
.open_bi()
|
||||
.await
|
||||
.map_err(|err| Error::Endpoint(format!("cannot open control stream: {err}")))?;
|
||||
Ok((conn, send, recv))
|
||||
}
|
||||
|
||||
/// Closes the endpoint and waits for it to finish.
|
||||
pub async fn close(&self) {
|
||||
self.endpoint.close().await;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
//! Length-prefixed framing over one QUIC bidirectional stream.
|
||||
//!
|
||||
//! A frame is `u32_be(len) || payload`. The announced length is validated
|
||||
//! against the configured limit **before** a buffer of that size is allocated,
|
||||
//! so a hostile peer cannot make the agent allocate arbitrary memory with a
|
||||
//! four byte header.
|
||||
//!
|
||||
//! No custom encryption is layered on top: the iroh/QUIC connection already
|
||||
//! provides confidentiality, integrity and endpoint authentication.
|
||||
|
||||
use iroh::endpoint::{RecvStream, SendStream};
|
||||
|
||||
use crate::error::ProtocolError;
|
||||
|
||||
/// Size of the frame length prefix, in bytes.
|
||||
pub const LENGTH_PREFIX_LEN: usize = 4;
|
||||
|
||||
/// Writes one frame.
|
||||
pub async fn write_frame(
|
||||
stream: &mut SendStream,
|
||||
payload: &[u8],
|
||||
max_frame_len: usize,
|
||||
) -> Result<(), ProtocolError> {
|
||||
if payload.len() > max_frame_len {
|
||||
return Err(ProtocolError::FrameTooLarge {
|
||||
announced: payload.len() as u64,
|
||||
limit: max_frame_len,
|
||||
});
|
||||
}
|
||||
let len = u32::try_from(payload.len()).map_err(|_| ProtocolError::FrameTooLarge {
|
||||
announced: payload.len() as u64,
|
||||
limit: max_frame_len,
|
||||
})?;
|
||||
stream
|
||||
.write_all(&len.to_be_bytes())
|
||||
.await
|
||||
.map_err(|err| ProtocolError::Stream(err.to_string()))?;
|
||||
stream
|
||||
.write_all(payload)
|
||||
.await
|
||||
.map_err(|err| ProtocolError::Stream(err.to_string()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Reads one frame, rejecting oversized headers before allocating.
|
||||
pub async fn read_frame(
|
||||
stream: &mut RecvStream,
|
||||
max_frame_len: usize,
|
||||
) -> Result<Vec<u8>, ProtocolError> {
|
||||
let mut header = [0u8; LENGTH_PREFIX_LEN];
|
||||
match stream.read_exact(&mut header).await {
|
||||
Ok(()) => {}
|
||||
Err(err) => return Err(classify_read_error(err)),
|
||||
}
|
||||
|
||||
let announced = u32::from_be_bytes(header) as u64;
|
||||
if announced > max_frame_len as u64 {
|
||||
return Err(ProtocolError::FrameTooLarge {
|
||||
announced,
|
||||
limit: max_frame_len,
|
||||
});
|
||||
}
|
||||
|
||||
// Safe: `announced` was just bounded by `max_frame_len`, a usize.
|
||||
let mut payload = vec![0u8; announced as usize];
|
||||
if !payload.is_empty() {
|
||||
match stream.read_exact(&mut payload).await {
|
||||
Ok(()) => {}
|
||||
Err(err) => return Err(classify_read_error(err)),
|
||||
}
|
||||
}
|
||||
Ok(payload)
|
||||
}
|
||||
|
||||
fn classify_read_error(err: iroh::endpoint::ReadExactError) -> ProtocolError {
|
||||
match err {
|
||||
iroh::endpoint::ReadExactError::FinishedEarly(_) => ProtocolError::StreamClosed,
|
||||
other => ProtocolError::Stream(other.to_string()),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,575 @@
|
||||
//! Mutual proof that both ends belong to the same network space.
|
||||
//!
|
||||
//! # Why a successful iroh connection is not enough
|
||||
//!
|
||||
//! iroh authenticates *endpoints*: after the QUIC/TLS handshake each side knows
|
||||
//! the other's [`EndpointId`], because that id is the public key in the
|
||||
//! certificate. It says nothing about network membership — anybody can dial us.
|
||||
//! So on top of the authenticated connection we run an explicit mutual proof of
|
||||
//! knowledge of the derived network authentication key.
|
||||
//!
|
||||
//! # Channel binding is not a proof by itself
|
||||
//!
|
||||
//! iroh exposes the TLS exporter (RFC 5705) via
|
||||
//! [`Connection::export_keying_material`]. That gives both sides the same
|
||||
//! secret bytes for *this* connection, which is exactly what is needed to stop
|
||||
//! a proof being replayed on another connection. It proves nothing about the
|
||||
//! shared network secret on its own, because both ends of any connection can
|
||||
//! compute it. The proof of membership is the HMAC keyed by `auth_key`; the
|
||||
//! exporter output is only one of its inputs.
|
||||
//!
|
||||
//! # The scheme
|
||||
//!
|
||||
//! ```text
|
||||
//! cb = TLS-Exporter(label = "tsunagi/handshake/v1", context = network_id, 32)
|
||||
//! LP(x)= u32_be(len(x)) || x
|
||||
//!
|
||||
//! transcript(role) = LP("tsunagi-handshake-v1")
|
||||
//! || LP(role) // "initiator-proof" | "responder-proof"
|
||||
//! || LP(u16_be(protocol_version))
|
||||
//! || LP(network_id) // 32 bytes
|
||||
//! || LP(initiator_endpoint_id) // 32 bytes
|
||||
//! || LP(responder_endpoint_id) // 32 bytes
|
||||
//! || LP(cb) // 32 bytes
|
||||
//! || LP(nonce_initiator) // 16 bytes
|
||||
//! || LP(nonce_responder) // 16 bytes
|
||||
//!
|
||||
//! proof(role) = HMAC-SHA256(auth_key, transcript(role))
|
||||
//! ```
|
||||
//!
|
||||
//! What each input buys:
|
||||
//!
|
||||
//! * `auth_key` — membership. Derived from name+secret only, see
|
||||
//! [`crate::identity`].
|
||||
//! * `cb` — binding to this connection. A proof captured elsewhere is useless
|
||||
//! here, because `cb` differs per TLS session.
|
||||
//! * `network_id` — binding to this network space.
|
||||
//! * both endpoint ids — binding to these two identities.
|
||||
//! * distinct `role` labels — no reflection: the responder cannot bounce the
|
||||
//! initiator's own proof back at it.
|
||||
//! * both nonces — freshness contributed by each side.
|
||||
//!
|
||||
//! # Message order
|
||||
//!
|
||||
//! ```text
|
||||
//! initiator -> responder : Hello { version, network_id, nonce_i }
|
||||
//! initiator <- responder : HelloAck { version, nonce_r }
|
||||
//! initiator -> responder : AuthProof{ proof(initiator) }
|
||||
//! initiator <- responder : AuthProof{ proof(responder) } // only if the first proof verified
|
||||
//! ```
|
||||
//!
|
||||
//! The responder emits nothing derived from `auth_key` until the initiator's
|
||||
//! proof has verified, so a caller who does not know the secret learns nothing.
|
||||
//! Until both steps complete, no regular control message is accepted in either
|
||||
//! direction.
|
||||
//!
|
||||
//! [`Connection::export_keying_material`]: iroh::endpoint::Connection::export_keying_material
|
||||
|
||||
use hmac::{Hmac, KeyInit, Mac};
|
||||
use iroh::EndpointId;
|
||||
use iroh::endpoint::{Connection, RecvStream, SendStream};
|
||||
use sha2::Sha256;
|
||||
use zeroize::Zeroizing;
|
||||
|
||||
use crate::config::Limits;
|
||||
use crate::error::ProtocolError;
|
||||
use crate::identity::{NetworkId, NetworkKeys};
|
||||
use crate::proto::frame::{read_frame, write_frame};
|
||||
use crate::proto::message::{AuthProof, Hello, HelloAck, PROTOCOL_VERSION, decode, encode};
|
||||
|
||||
/// Frozen domain separator of the handshake transcript.
|
||||
pub const TRANSCRIPT_DOMAIN: &str = "tsunagi-handshake-v1";
|
||||
|
||||
/// TLS exporter label used for channel binding.
|
||||
pub const EXPORTER_LABEL: &[u8] = b"tsunagi/handshake/v1";
|
||||
|
||||
/// Transcript role label of the side that dialled.
|
||||
pub const ROLE_INITIATOR: &str = "initiator-proof";
|
||||
|
||||
/// Transcript role label of the side that accepted.
|
||||
pub const ROLE_RESPONDER: &str = "responder-proof";
|
||||
|
||||
/// Length of the channel binding material, in bytes.
|
||||
pub const CHANNEL_BINDING_LEN: usize = 32;
|
||||
|
||||
/// Which side of the handshake this agent played.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum Role {
|
||||
/// This agent dialled.
|
||||
Initiator,
|
||||
/// This agent accepted.
|
||||
Responder,
|
||||
}
|
||||
|
||||
impl Role {
|
||||
/// Short label for diagnostics.
|
||||
pub fn as_str(&self) -> &'static str {
|
||||
match self {
|
||||
Role::Initiator => "initiator",
|
||||
Role::Responder => "responder",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Result of a completed handshake.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct HandshakeOutcome {
|
||||
/// Network both sides proved membership of.
|
||||
pub network_id: NetworkId,
|
||||
/// Authenticated endpoint id of the peer, taken from the TLS certificate.
|
||||
pub peer: EndpointId,
|
||||
/// Which side this agent played.
|
||||
pub role: Role,
|
||||
}
|
||||
|
||||
fn push_lp(out: &mut Vec<u8>, bytes: &[u8]) {
|
||||
let len = u32::try_from(bytes.len()).unwrap_or(u32::MAX);
|
||||
out.extend_from_slice(&len.to_be_bytes());
|
||||
out.extend_from_slice(bytes);
|
||||
}
|
||||
|
||||
/// Builds the role-specific transcript. Pure function, unit tested.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn transcript(
|
||||
role: &str,
|
||||
version: u16,
|
||||
network_id: &[u8; 32],
|
||||
initiator: &[u8; 32],
|
||||
responder: &[u8; 32],
|
||||
channel_binding: &[u8],
|
||||
nonce_initiator: &[u8; 16],
|
||||
nonce_responder: &[u8; 16],
|
||||
) -> Vec<u8> {
|
||||
let mut out = Vec::with_capacity(256);
|
||||
push_lp(&mut out, TRANSCRIPT_DOMAIN.as_bytes());
|
||||
push_lp(&mut out, role.as_bytes());
|
||||
push_lp(&mut out, &version.to_be_bytes());
|
||||
push_lp(&mut out, network_id);
|
||||
push_lp(&mut out, initiator);
|
||||
push_lp(&mut out, responder);
|
||||
push_lp(&mut out, channel_binding);
|
||||
push_lp(&mut out, nonce_initiator);
|
||||
push_lp(&mut out, nonce_responder);
|
||||
out
|
||||
}
|
||||
|
||||
/// Computes one proof.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn proof(
|
||||
auth_key: &[u8; 32],
|
||||
role: &str,
|
||||
version: u16,
|
||||
network_id: &[u8; 32],
|
||||
initiator: &[u8; 32],
|
||||
responder: &[u8; 32],
|
||||
channel_binding: &[u8],
|
||||
nonce_initiator: &[u8; 16],
|
||||
nonce_responder: &[u8; 16],
|
||||
) -> [u8; 32] {
|
||||
let message = transcript(
|
||||
role,
|
||||
version,
|
||||
network_id,
|
||||
initiator,
|
||||
responder,
|
||||
channel_binding,
|
||||
nonce_initiator,
|
||||
nonce_responder,
|
||||
);
|
||||
let mut mac = match <Hmac<Sha256> as KeyInit>::new_from_slice(auth_key) {
|
||||
Ok(mac) => mac,
|
||||
// HMAC-SHA256 accepts keys of any length, so a 32 byte key cannot fail.
|
||||
Err(_) => unreachable!("HMAC-SHA256 accepts a 32 byte key"),
|
||||
};
|
||||
mac.update(&message);
|
||||
let tag = mac.finalize().into_bytes();
|
||||
let mut out = [0u8; 32];
|
||||
out.copy_from_slice(&tag);
|
||||
out
|
||||
}
|
||||
|
||||
/// Verifies a proof in constant time.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn verify(
|
||||
auth_key: &[u8; 32],
|
||||
role: &str,
|
||||
version: u16,
|
||||
network_id: &[u8; 32],
|
||||
initiator: &[u8; 32],
|
||||
responder: &[u8; 32],
|
||||
channel_binding: &[u8],
|
||||
nonce_initiator: &[u8; 16],
|
||||
nonce_responder: &[u8; 16],
|
||||
candidate: &[u8; 32],
|
||||
) -> Result<(), ProtocolError> {
|
||||
let message = transcript(
|
||||
role,
|
||||
version,
|
||||
network_id,
|
||||
initiator,
|
||||
responder,
|
||||
channel_binding,
|
||||
nonce_initiator,
|
||||
nonce_responder,
|
||||
);
|
||||
let mut mac = match <Hmac<Sha256> as KeyInit>::new_from_slice(auth_key) {
|
||||
Ok(mac) => mac,
|
||||
Err(_) => unreachable!("HMAC-SHA256 accepts a 32 byte key"),
|
||||
};
|
||||
mac.update(&message);
|
||||
mac.verify_slice(candidate)
|
||||
.map_err(|_| ProtocolError::AuthenticationFailed)
|
||||
}
|
||||
|
||||
/// Extracts channel binding material from the connection.
|
||||
fn channel_binding(
|
||||
conn: &Connection,
|
||||
network_id: &[u8; 32],
|
||||
) -> Result<Zeroizing<[u8; CHANNEL_BINDING_LEN]>, ProtocolError> {
|
||||
let mut out = Zeroizing::new([0u8; CHANNEL_BINDING_LEN]);
|
||||
conn.export_keying_material(out.as_mut(), EXPORTER_LABEL, network_id)
|
||||
.map_err(|err| ProtocolError::NoChannelBinding(format!("{err:?}")))?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn fresh_nonce() -> [u8; 16] {
|
||||
let mut nonce = [0u8; 16];
|
||||
rand::fill(&mut nonce);
|
||||
nonce
|
||||
}
|
||||
|
||||
fn check_version(found: u16) -> Result<(), ProtocolError> {
|
||||
if found != PROTOCOL_VERSION {
|
||||
return Err(ProtocolError::UnsupportedVersion {
|
||||
found,
|
||||
supported: PROTOCOL_VERSION,
|
||||
});
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Runs the initiator side of the handshake.
|
||||
///
|
||||
/// Bounded by [`Limits::handshake_timeout`].
|
||||
pub async fn initiate(
|
||||
conn: &Connection,
|
||||
send: &mut SendStream,
|
||||
recv: &mut RecvStream,
|
||||
local_id: EndpointId,
|
||||
keys: &NetworkKeys,
|
||||
limits: &Limits,
|
||||
) -> Result<HandshakeOutcome, ProtocolError> {
|
||||
tokio::time::timeout(
|
||||
limits.handshake_timeout,
|
||||
initiate_inner(conn, send, recv, local_id, keys, limits),
|
||||
)
|
||||
.await
|
||||
.unwrap_or(Err(ProtocolError::HandshakeTimeout))
|
||||
}
|
||||
|
||||
async fn initiate_inner(
|
||||
conn: &Connection,
|
||||
send: &mut SendStream,
|
||||
recv: &mut RecvStream,
|
||||
local_id: EndpointId,
|
||||
keys: &NetworkKeys,
|
||||
limits: &Limits,
|
||||
) -> Result<HandshakeOutcome, ProtocolError> {
|
||||
let network_id = keys.network_id();
|
||||
let network_bytes = *network_id.as_bytes();
|
||||
let peer = conn.remote_id();
|
||||
|
||||
let initiator = *local_id.as_bytes();
|
||||
let responder = *peer.as_bytes();
|
||||
let cb = channel_binding(conn, &network_bytes)?;
|
||||
|
||||
let nonce_i = fresh_nonce();
|
||||
let hello = Hello {
|
||||
version: PROTOCOL_VERSION,
|
||||
network_id: network_bytes,
|
||||
nonce: nonce_i,
|
||||
};
|
||||
write_frame(send, &encode(&hello)?, limits.max_frame_len).await?;
|
||||
|
||||
let ack: HelloAck = decode(&read_frame(recv, limits.max_frame_len).await?)?;
|
||||
check_version(ack.version)?;
|
||||
let nonce_r = ack.nonce;
|
||||
|
||||
let mine = proof(
|
||||
keys.auth_key(),
|
||||
ROLE_INITIATOR,
|
||||
PROTOCOL_VERSION,
|
||||
&network_bytes,
|
||||
&initiator,
|
||||
&responder,
|
||||
cb.as_ref(),
|
||||
&nonce_i,
|
||||
&nonce_r,
|
||||
);
|
||||
write_frame(
|
||||
send,
|
||||
&encode(&AuthProof { proof: mine })?,
|
||||
limits.max_frame_len,
|
||||
)
|
||||
.await?;
|
||||
|
||||
let theirs: AuthProof = decode(&read_frame(recv, limits.max_frame_len).await?)?;
|
||||
verify(
|
||||
keys.auth_key(),
|
||||
ROLE_RESPONDER,
|
||||
PROTOCOL_VERSION,
|
||||
&network_bytes,
|
||||
&initiator,
|
||||
&responder,
|
||||
cb.as_ref(),
|
||||
&nonce_i,
|
||||
&nonce_r,
|
||||
&theirs.proof,
|
||||
)?;
|
||||
|
||||
Ok(HandshakeOutcome {
|
||||
network_id,
|
||||
peer,
|
||||
role: Role::Initiator,
|
||||
})
|
||||
}
|
||||
|
||||
/// Runs the responder side of the handshake.
|
||||
///
|
||||
/// `lookup` maps the network id the peer asked for to the local key material,
|
||||
/// returning `None` if this agent does not have that network active. Routing
|
||||
/// stays in the agent; the protocol stays here.
|
||||
///
|
||||
/// Bounded by [`Limits::handshake_timeout`].
|
||||
pub async fn respond<F>(
|
||||
conn: &Connection,
|
||||
send: &mut SendStream,
|
||||
recv: &mut RecvStream,
|
||||
local_id: EndpointId,
|
||||
limits: &Limits,
|
||||
lookup: F,
|
||||
) -> Result<HandshakeOutcome, ProtocolError>
|
||||
where
|
||||
F: FnOnce(NetworkId) -> Option<NetworkKeys>,
|
||||
{
|
||||
tokio::time::timeout(
|
||||
limits.handshake_timeout,
|
||||
respond_inner(conn, send, recv, local_id, limits, lookup),
|
||||
)
|
||||
.await
|
||||
.unwrap_or(Err(ProtocolError::HandshakeTimeout))
|
||||
}
|
||||
|
||||
async fn respond_inner<F>(
|
||||
conn: &Connection,
|
||||
send: &mut SendStream,
|
||||
recv: &mut RecvStream,
|
||||
local_id: EndpointId,
|
||||
limits: &Limits,
|
||||
lookup: F,
|
||||
) -> Result<HandshakeOutcome, ProtocolError>
|
||||
where
|
||||
F: FnOnce(NetworkId) -> Option<NetworkKeys>,
|
||||
{
|
||||
let hello: Hello = decode(&read_frame(recv, limits.max_frame_len).await?)?;
|
||||
check_version(hello.version)?;
|
||||
|
||||
let network_id = NetworkId::from_bytes(hello.network_id);
|
||||
let keys = lookup(network_id).ok_or(ProtocolError::UnknownNetwork)?;
|
||||
|
||||
let peer = conn.remote_id();
|
||||
let network_bytes = hello.network_id;
|
||||
let initiator = *peer.as_bytes();
|
||||
let responder = *local_id.as_bytes();
|
||||
let cb = channel_binding(conn, &network_bytes)?;
|
||||
|
||||
let nonce_i = hello.nonce;
|
||||
let nonce_r = fresh_nonce();
|
||||
let ack = HelloAck {
|
||||
version: PROTOCOL_VERSION,
|
||||
nonce: nonce_r,
|
||||
};
|
||||
write_frame(send, &encode(&ack)?, limits.max_frame_len).await?;
|
||||
|
||||
let theirs: AuthProof = decode(&read_frame(recv, limits.max_frame_len).await?)?;
|
||||
verify(
|
||||
keys.auth_key(),
|
||||
ROLE_INITIATOR,
|
||||
PROTOCOL_VERSION,
|
||||
&network_bytes,
|
||||
&initiator,
|
||||
&responder,
|
||||
cb.as_ref(),
|
||||
&nonce_i,
|
||||
&nonce_r,
|
||||
&theirs.proof,
|
||||
)?;
|
||||
|
||||
// Only now, after the peer proved membership, do we emit our own proof.
|
||||
let mine = proof(
|
||||
keys.auth_key(),
|
||||
ROLE_RESPONDER,
|
||||
PROTOCOL_VERSION,
|
||||
&network_bytes,
|
||||
&initiator,
|
||||
&responder,
|
||||
cb.as_ref(),
|
||||
&nonce_i,
|
||||
&nonce_r,
|
||||
);
|
||||
write_frame(
|
||||
send,
|
||||
&encode(&AuthProof { proof: mine })?,
|
||||
limits.max_frame_len,
|
||||
)
|
||||
.await?;
|
||||
|
||||
Ok(HandshakeOutcome {
|
||||
network_id,
|
||||
peer,
|
||||
role: Role::Responder,
|
||||
})
|
||||
}
|
||||
|
||||
/// Test-only re-export of [`channel_binding`]. See [`crate::test_support`].
|
||||
#[doc(hidden)]
|
||||
pub fn channel_binding_for_test(
|
||||
conn: &Connection,
|
||||
network_id: &[u8; 32],
|
||||
) -> Result<[u8; 32], ProtocolError> {
|
||||
channel_binding(conn, network_id).map(|cb| *cb)
|
||||
}
|
||||
|
||||
/// Test-only re-export of the proof function. See [`crate::test_support`].
|
||||
#[doc(hidden)]
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn proof_for_test(
|
||||
auth_key: &[u8; 32],
|
||||
role: &str,
|
||||
version: u16,
|
||||
network_id: &[u8; 32],
|
||||
initiator: &[u8; 32],
|
||||
responder: &[u8; 32],
|
||||
channel_binding: &[u8],
|
||||
nonce_initiator: &[u8; 16],
|
||||
nonce_responder: &[u8; 16],
|
||||
) -> [u8; 32] {
|
||||
proof(
|
||||
auth_key,
|
||||
role,
|
||||
version,
|
||||
network_id,
|
||||
initiator,
|
||||
responder,
|
||||
channel_binding,
|
||||
nonce_initiator,
|
||||
nonce_responder,
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use super::*;
|
||||
|
||||
/// `(auth_key, network_id, initiator_id, responder_id, nonce_i, nonce_r)`
|
||||
type Fixture = ([u8; 32], [u8; 32], [u8; 32], [u8; 32], [u8; 16], [u8; 16]);
|
||||
|
||||
fn fixture() -> Fixture {
|
||||
(
|
||||
[1u8; 32], [2u8; 32], [3u8; 32], [4u8; 32], [5u8; 16], [6u8; 16],
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn role_labels_produce_different_transcripts() {
|
||||
let (key, net, ini, res, ni, nr) = fixture();
|
||||
let cb = [7u8; 32];
|
||||
let a = proof(&key, ROLE_INITIATOR, 1, &net, &ini, &res, &cb, &ni, &nr);
|
||||
let b = proof(&key, ROLE_RESPONDER, 1, &net, &ini, &res, &cb, &ni, &nr);
|
||||
assert_ne!(a, b, "reflecting a proof back must not verify");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn channel_binding_changes_the_proof() {
|
||||
let (key, net, ini, res, ni, nr) = fixture();
|
||||
let a = proof(
|
||||
&key,
|
||||
ROLE_INITIATOR,
|
||||
1,
|
||||
&net,
|
||||
&ini,
|
||||
&res,
|
||||
&[7u8; 32],
|
||||
&ni,
|
||||
&nr,
|
||||
);
|
||||
let b = proof(
|
||||
&key,
|
||||
ROLE_INITIATOR,
|
||||
1,
|
||||
&net,
|
||||
&ini,
|
||||
&res,
|
||||
&[8u8; 32],
|
||||
&ni,
|
||||
&nr,
|
||||
);
|
||||
assert_ne!(a, b, "a proof must not be replayable on another connection");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn identities_and_network_are_bound() {
|
||||
let (key, net, ini, res, ni, nr) = fixture();
|
||||
let cb = [7u8; 32];
|
||||
let base = proof(&key, ROLE_INITIATOR, 1, &net, &ini, &res, &cb, &ni, &nr);
|
||||
let other_net = proof(
|
||||
&key,
|
||||
ROLE_INITIATOR,
|
||||
1,
|
||||
&[9u8; 32],
|
||||
&ini,
|
||||
&res,
|
||||
&cb,
|
||||
&ni,
|
||||
&nr,
|
||||
);
|
||||
let swapped = proof(&key, ROLE_INITIATOR, 1, &net, &res, &ini, &cb, &ni, &nr);
|
||||
assert_ne!(base, other_net);
|
||||
assert_ne!(base, swapped);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn transcript_encoding_is_unambiguous() {
|
||||
// Two different field splits that would collide under naive concatenation.
|
||||
let a = transcript(
|
||||
"ab", 1, &[0u8; 32], &[0u8; 32], &[0u8; 32], b"cd", &[0u8; 16], &[0u8; 16],
|
||||
);
|
||||
let b = transcript(
|
||||
"a", 1, &[0u8; 32], &[0u8; 32], &[0u8; 32], b"bcd", &[0u8; 16], &[0u8; 16],
|
||||
);
|
||||
assert_ne!(a, b);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn wrong_key_fails_verification() {
|
||||
let (key, net, ini, res, ni, nr) = fixture();
|
||||
let cb = [7u8; 32];
|
||||
let tag = proof(&key, ROLE_INITIATOR, 1, &net, &ini, &res, &cb, &ni, &nr);
|
||||
let wrong = [0xAAu8; 32];
|
||||
let result = verify(
|
||||
&wrong,
|
||||
ROLE_INITIATOR,
|
||||
1,
|
||||
&net,
|
||||
&ini,
|
||||
&res,
|
||||
&cb,
|
||||
&ni,
|
||||
&nr,
|
||||
&tag,
|
||||
);
|
||||
assert!(matches!(result, Err(ProtocolError::AuthenticationFailed)));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
//! Control message formats.
|
||||
//!
|
||||
//! This is deliberately a small, closed set of messages, not a general RPC
|
||||
//! framework. Bodies are encoded with [postcard], a compact, deterministic,
|
||||
//! non-self-describing serde format.
|
||||
//!
|
||||
//! Every message that arrives from the network passes [`validate`] against the
|
||||
//! configured [`Limits`] before it reaches anything else.
|
||||
//!
|
||||
//! [postcard]: https://docs.rs/postcard
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::config::Limits;
|
||||
use crate::dataplane::{MAX_PROTOCOL_ID_LEN, PluginCapability};
|
||||
use crate::error::ProtocolError;
|
||||
|
||||
/// ALPN of the tsunagi control plane.
|
||||
///
|
||||
/// The version in the ALPN is the wire-compatibility version of the control
|
||||
/// protocol. It is independent of the network identity scheme version, so
|
||||
/// bumping it must not change any existing [`crate::NetworkId`].
|
||||
pub const ALPN: &[u8] = b"tsunagi/ctrl/1";
|
||||
|
||||
/// Control protocol version carried inside the handshake.
|
||||
pub const PROTOCOL_VERSION: u16 = 1;
|
||||
|
||||
/// First message of the handshake, sent by the initiator.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct Hello {
|
||||
/// Control protocol version the initiator speaks.
|
||||
pub version: u16,
|
||||
/// Public network identifier the initiator wants to join.
|
||||
pub network_id: [u8; 32],
|
||||
/// Initiator's fresh handshake nonce.
|
||||
pub nonce: [u8; 16],
|
||||
}
|
||||
|
||||
/// Responder's reply to [`Hello`]. Carries no proof yet.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct HelloAck {
|
||||
/// Control protocol version the responder speaks.
|
||||
pub version: u16,
|
||||
/// Responder's fresh handshake nonce.
|
||||
pub nonce: [u8; 16],
|
||||
}
|
||||
|
||||
/// A handshake proof, i.e. one HMAC tag over a role-specific transcript.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct AuthProof {
|
||||
/// HMAC-SHA256 tag.
|
||||
pub proof: [u8; 32],
|
||||
}
|
||||
|
||||
/// What this agent tells a peer about itself.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct Announcement {
|
||||
/// Human-readable hostname. A mutable binding, not an identity.
|
||||
pub hostname: String,
|
||||
/// Announced IP plugin capabilities. Opaque to the core.
|
||||
pub capabilities: Vec<PluginCapability>,
|
||||
}
|
||||
|
||||
/// A control message exchanged after a successful handshake.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[non_exhaustive]
|
||||
pub enum ControlMessage {
|
||||
/// Hostname and capability announcement.
|
||||
Announce(Announcement),
|
||||
/// A small request used to verify that the exchange works.
|
||||
Ping {
|
||||
/// Caller-chosen sequence number, echoed back.
|
||||
seq: u64,
|
||||
/// Opaque bounded payload, echoed back.
|
||||
payload: Vec<u8>,
|
||||
},
|
||||
/// The reply to a [`ControlMessage::Ping`].
|
||||
Pong {
|
||||
/// Sequence number of the request being answered.
|
||||
seq: u64,
|
||||
/// Echoed payload.
|
||||
payload: Vec<u8>,
|
||||
},
|
||||
/// Graceful goodbye.
|
||||
///
|
||||
/// A peer going away is not a revocation of anything.
|
||||
Bye {
|
||||
/// Short free-text reason.
|
||||
reason: String,
|
||||
},
|
||||
}
|
||||
|
||||
/// A control message together with the network it belongs to.
|
||||
///
|
||||
/// Every session is bound to exactly one network at handshake time. The
|
||||
/// `network_id` here is re-checked on every message, so an authenticated
|
||||
/// session for network A can never be used to speak to network B, even over a
|
||||
/// shared physical connection.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct Envelope {
|
||||
/// Network this message belongs to.
|
||||
pub network_id: [u8; 32],
|
||||
/// The message itself.
|
||||
pub message: ControlMessage,
|
||||
}
|
||||
|
||||
/// Encodes a value into a postcard byte vector.
|
||||
pub fn encode<T: Serialize>(value: &T) -> Result<Vec<u8>, ProtocolError> {
|
||||
postcard::to_stdvec(value).map_err(|_| ProtocolError::Malformed("cannot encode message"))
|
||||
}
|
||||
|
||||
/// Decodes a value from postcard bytes.
|
||||
pub fn decode<T: for<'de> Deserialize<'de>>(bytes: &[u8]) -> Result<T, ProtocolError> {
|
||||
postcard::from_bytes(bytes).map_err(|_| ProtocolError::Malformed("cannot decode message"))
|
||||
}
|
||||
|
||||
fn check_len(field: &'static str, len: usize, limit: usize) -> Result<(), ProtocolError> {
|
||||
if len > limit {
|
||||
return Err(ProtocolError::FieldTooLarge { field, len, limit });
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Validates a decoded capability against the configured limits.
|
||||
pub fn validate_capability(
|
||||
capability: &PluginCapability,
|
||||
limits: &Limits,
|
||||
) -> Result<(), ProtocolError> {
|
||||
if capability.protocol.is_empty() {
|
||||
return Err(ProtocolError::Malformed("empty plugin protocol id"));
|
||||
}
|
||||
check_len(
|
||||
"capability.protocol",
|
||||
capability.protocol.len(),
|
||||
MAX_PROTOCOL_ID_LEN,
|
||||
)?;
|
||||
check_len(
|
||||
"capability.data",
|
||||
capability.data.len(),
|
||||
limits.max_capability_data_len,
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Validates a decoded control message against the configured limits.
|
||||
///
|
||||
/// Returning an error rejects that single message. It never stops the session's
|
||||
/// network, the other networks or the agent.
|
||||
pub fn validate(message: &ControlMessage, limits: &Limits) -> Result<(), ProtocolError> {
|
||||
match message {
|
||||
ControlMessage::Announce(announcement) => {
|
||||
check_len(
|
||||
"announce.hostname",
|
||||
announcement.hostname.len(),
|
||||
limits.max_hostname_len,
|
||||
)?;
|
||||
check_len(
|
||||
"announce.capabilities",
|
||||
announcement.capabilities.len(),
|
||||
limits.max_capabilities,
|
||||
)?;
|
||||
for capability in &announcement.capabilities {
|
||||
validate_capability(capability, limits)?;
|
||||
}
|
||||
}
|
||||
ControlMessage::Ping { payload, .. } | ControlMessage::Pong { payload, .. } => {
|
||||
check_len("echo.payload", payload.len(), limits.max_echo_payload_len)?;
|
||||
}
|
||||
ControlMessage::Bye { reason } => {
|
||||
check_len("bye.reason", reason.len(), limits.max_reason_len)?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A short, stable label for a message kind, for metrics and diagnostics.
|
||||
pub fn kind(message: &ControlMessage) -> &'static str {
|
||||
match message {
|
||||
ControlMessage::Announce(_) => "announce",
|
||||
ControlMessage::Ping { .. } => "ping",
|
||||
ControlMessage::Pong { .. } => "pong",
|
||||
ControlMessage::Bye { .. } => "bye",
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
//! The control protocol: framing, messages and the network membership
|
||||
//! handshake.
|
||||
//!
|
||||
//! Only control messages travel over iroh. User IP traffic is never tunnelled
|
||||
//! through this protocol.
|
||||
//!
|
||||
//! Layering, outermost first:
|
||||
//!
|
||||
//! 1. iroh/QUIC connection with ALPN [`message::ALPN`] — endpoint
|
||||
//! authentication, confidentiality and integrity.
|
||||
//! 2. One bidirectional stream per session, carrying length-prefixed frames
|
||||
//! ([`frame`]).
|
||||
//! 3. The [`handshake`], which must complete before anything else is accepted.
|
||||
//! 4. [`message::Envelope`]s carrying [`message::ControlMessage`]s, each
|
||||
//! re-checked against the session's network id.
|
||||
//!
|
||||
//! Nothing here adds its own encryption on top of iroh.
|
||||
|
||||
pub mod frame;
|
||||
pub mod handshake;
|
||||
pub mod message;
|
||||
|
||||
pub use frame::{read_frame, write_frame};
|
||||
pub use handshake::{HandshakeOutcome, Role};
|
||||
pub use message::{
|
||||
ALPN, Announcement, AuthProof, ControlMessage, Envelope, Hello, HelloAck, PROTOCOL_VERSION,
|
||||
};
|
||||
@@ -0,0 +1,231 @@
|
||||
//! The disposable cache store, `cache.sqlite`.
|
||||
//!
|
||||
//! Everything here is recoverable. A missing cache is recreated, a corrupt one
|
||||
//! is thrown away and recreated, and a stale one is simply wrong data that the
|
||||
//! rest of the system is expected to tolerate.
|
||||
//!
|
||||
//! Crucially, a stale cache never bypasses identity or network authentication:
|
||||
//! cached hints only produce *candidates*, which still have to pass the
|
||||
//! handshake.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use rusqlite::{Connection, params};
|
||||
|
||||
use crate::error::{Error, Result};
|
||||
use crate::identity::NetworkId;
|
||||
|
||||
/// Schema version written by this build.
|
||||
pub const SCHEMA_VERSION: i64 = 1;
|
||||
|
||||
/// A cached address hint for one peer in one network.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct AddressHint {
|
||||
/// Network the hint belongs to.
|
||||
pub network_id: NetworkId,
|
||||
/// Peer endpoint id, 32 bytes.
|
||||
pub endpoint_id: [u8; 32],
|
||||
/// Serialised address, currently `ip:<socketaddr>` or `relay:<url>`.
|
||||
pub addr: String,
|
||||
/// Unix seconds when this hint was last confirmed.
|
||||
pub last_seen: i64,
|
||||
}
|
||||
|
||||
/// Why the cache had to be recreated, if it did.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum CacheOutcome {
|
||||
/// Opened normally.
|
||||
Opened,
|
||||
/// Created because nothing was there.
|
||||
Created,
|
||||
/// Discarded and recreated. The reason is free of secrets.
|
||||
Reset(String),
|
||||
}
|
||||
|
||||
/// The disposable cache store.
|
||||
#[derive(Debug)]
|
||||
pub struct CacheStore {
|
||||
conn: Connection,
|
||||
path: PathBuf,
|
||||
}
|
||||
|
||||
impl CacheStore {
|
||||
/// Opens the cache, discarding and recreating it if it is unusable.
|
||||
pub fn open_or_reset(path: impl AsRef<Path>) -> Result<(Self, CacheOutcome)> {
|
||||
let path = path.as_ref().to_path_buf();
|
||||
let existed = path.exists();
|
||||
|
||||
match Self::try_open(&path, existed) {
|
||||
Ok(store) => Ok((
|
||||
store,
|
||||
if existed {
|
||||
CacheOutcome::Opened
|
||||
} else {
|
||||
CacheOutcome::Created
|
||||
},
|
||||
)),
|
||||
Err(reason) => {
|
||||
tracing::warn!(path = %path.display(), %reason, "discarding unusable cache");
|
||||
Self::remove_files(&path);
|
||||
let store = Self::try_open(&path, false)
|
||||
.map_err(|err| Error::Storage(format!("cannot recreate cache: {err}")))?;
|
||||
Ok((store, CacheOutcome::Reset(reason)))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn try_open(path: &Path, check_integrity: bool) -> std::result::Result<Self, String> {
|
||||
if let Some(parent) = path.parent() {
|
||||
std::fs::create_dir_all(parent)
|
||||
.map_err(|err| format!("cannot create cache directory: {err}"))?;
|
||||
}
|
||||
let conn = Connection::open(path).map_err(|err| format!("cannot open: {err}"))?;
|
||||
super::restrict_path_permissions(path).map_err(|err| format!("{err}"))?;
|
||||
super::apply_common_pragmas(&conn).map_err(|err| format!("cannot configure: {err}"))?;
|
||||
|
||||
if check_integrity {
|
||||
let integrity: String = conn
|
||||
.query_row("PRAGMA integrity_check", [], |row| row.get(0))
|
||||
.map_err(|err| format!("integrity check failed: {err}"))?;
|
||||
if integrity != "ok" {
|
||||
return Err(format!("integrity check reported: {integrity}"));
|
||||
}
|
||||
}
|
||||
|
||||
let found: i64 = conn
|
||||
.query_row("PRAGMA user_version", [], |row| row.get(0))
|
||||
.map_err(|err| format!("cannot read schema version: {err}"))?;
|
||||
if found > SCHEMA_VERSION {
|
||||
return Err(format!(
|
||||
"cache schema version {found} is newer than {SCHEMA_VERSION}"
|
||||
));
|
||||
}
|
||||
if found < SCHEMA_VERSION {
|
||||
conn.execute_batch(
|
||||
"BEGIN;
|
||||
DROP TABLE IF EXISTS address_hints;
|
||||
CREATE TABLE address_hints (
|
||||
network_id BLOB NOT NULL,
|
||||
endpoint_id BLOB NOT NULL,
|
||||
addr TEXT NOT NULL,
|
||||
last_seen INTEGER NOT NULL,
|
||||
PRIMARY KEY (network_id, endpoint_id, addr)
|
||||
);
|
||||
PRAGMA user_version = 1;
|
||||
COMMIT;",
|
||||
)
|
||||
.map_err(|err| format!("cannot create cache schema: {err}"))?;
|
||||
} else {
|
||||
conn.query_row("SELECT count(*) FROM address_hints", [], |row| {
|
||||
row.get::<_, i64>(0)
|
||||
})
|
||||
.map_err(|err| format!("cache schema is unusable: {err}"))?;
|
||||
}
|
||||
|
||||
Ok(Self {
|
||||
conn,
|
||||
path: path.to_path_buf(),
|
||||
})
|
||||
}
|
||||
|
||||
fn remove_files(path: &Path) {
|
||||
for suffix in ["", "-wal", "-shm", "-journal"] {
|
||||
let mut name = path.as_os_str().to_os_string();
|
||||
name.push(suffix);
|
||||
let _ = std::fs::remove_file(PathBuf::from(name));
|
||||
}
|
||||
}
|
||||
|
||||
/// Path of the underlying file.
|
||||
pub fn path(&self) -> &Path {
|
||||
&self.path
|
||||
}
|
||||
|
||||
/// Records an address hint, keeping at most `max_per_peer` newest entries.
|
||||
pub fn record_hint(
|
||||
&self,
|
||||
network_id: NetworkId,
|
||||
endpoint_id: &[u8; 32],
|
||||
addr: &str,
|
||||
max_per_peer: usize,
|
||||
) -> Result<()> {
|
||||
self.conn
|
||||
.execute(
|
||||
"INSERT INTO address_hints (network_id, endpoint_id, addr, last_seen)
|
||||
VALUES (?1, ?2, ?3, ?4)
|
||||
ON CONFLICT(network_id, endpoint_id, addr)
|
||||
DO UPDATE SET last_seen = excluded.last_seen",
|
||||
params![
|
||||
network_id.as_bytes().as_slice(),
|
||||
endpoint_id.as_slice(),
|
||||
addr,
|
||||
super::state::now_unix()
|
||||
],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot record address hint: {err}")))?;
|
||||
|
||||
self.conn
|
||||
.execute(
|
||||
"DELETE FROM address_hints
|
||||
WHERE network_id = ?1 AND endpoint_id = ?2 AND addr NOT IN (
|
||||
SELECT addr FROM address_hints
|
||||
WHERE network_id = ?1 AND endpoint_id = ?2
|
||||
ORDER BY last_seen DESC LIMIT ?3
|
||||
)",
|
||||
params![
|
||||
network_id.as_bytes().as_slice(),
|
||||
endpoint_id.as_slice(),
|
||||
max_per_peer as i64
|
||||
],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot prune address hints: {err}")))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Returns every hint known for a network.
|
||||
pub fn hints_for_network(&self, network_id: NetworkId) -> Result<Vec<AddressHint>> {
|
||||
let mut stmt = self
|
||||
.conn
|
||||
.prepare(
|
||||
"SELECT endpoint_id, addr, last_seen FROM address_hints
|
||||
WHERE network_id = ?1 ORDER BY last_seen DESC",
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot read address hints: {err}")))?;
|
||||
let rows = stmt
|
||||
.query_map(params![network_id.as_bytes().as_slice()], |row| {
|
||||
let endpoint_id: Vec<u8> = row.get(0)?;
|
||||
let addr: String = row.get(1)?;
|
||||
let last_seen: i64 = row.get(2)?;
|
||||
Ok((endpoint_id, addr, last_seen))
|
||||
})
|
||||
.map_err(|err| Error::Storage(format!("cannot read address hints: {err}")))?;
|
||||
|
||||
let mut out = Vec::new();
|
||||
for row in rows {
|
||||
let (endpoint_id, addr, last_seen) =
|
||||
row.map_err(|err| Error::Storage(format!("cannot read hint row: {err}")))?;
|
||||
// A malformed row in a disposable store is skipped, not fatal.
|
||||
let Ok(endpoint_id) = <[u8; 32]>::try_from(endpoint_id.as_slice()) else {
|
||||
continue;
|
||||
};
|
||||
out.push(AddressHint {
|
||||
network_id,
|
||||
endpoint_id,
|
||||
addr,
|
||||
last_seen,
|
||||
});
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Drops all hints for a network.
|
||||
pub fn forget_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
self.conn
|
||||
.execute(
|
||||
"DELETE FROM address_hints WHERE network_id = ?1",
|
||||
params![network_id.as_bytes().as_slice()],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot clear address hints: {err}")))?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
//! Ownership lock for a state directory.
|
||||
//!
|
||||
//! One persistent state directory belongs to exactly one live agent instance.
|
||||
//! Checking whether a file exists is not enough — a stale file from a crashed
|
||||
//! process must not block a restart, and two concurrently starting agents must
|
||||
//! not both win. An advisory OS file lock gives both properties.
|
||||
|
||||
use std::fs::{File, OpenOptions};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use fs4::{FileExt, TryLockError};
|
||||
|
||||
use crate::error::{Error, Result};
|
||||
|
||||
/// An exclusive lock held for the lifetime of an agent.
|
||||
///
|
||||
/// Dropping it releases the lock, so a cleanly stopped agent leaves the
|
||||
/// directory immediately reopenable.
|
||||
#[derive(Debug)]
|
||||
pub struct DirectoryLock {
|
||||
file: File,
|
||||
path: PathBuf,
|
||||
}
|
||||
|
||||
impl DirectoryLock {
|
||||
/// Acquires the lock, failing fast if another live agent holds it.
|
||||
pub fn acquire(path: impl AsRef<Path>) -> Result<Self> {
|
||||
let path = path.as_ref().to_path_buf();
|
||||
let file = OpenOptions::new()
|
||||
.create(true)
|
||||
.read(true)
|
||||
.write(true)
|
||||
.truncate(false)
|
||||
.open(&path)
|
||||
.map_err(|source| Error::Io {
|
||||
path: path.clone(),
|
||||
source,
|
||||
})?;
|
||||
super::restrict_permissions(&file, &path)?;
|
||||
|
||||
match FileExt::try_lock(&file) {
|
||||
Ok(()) => Ok(Self { file, path }),
|
||||
Err(TryLockError::WouldBlock) => Err(Error::StateLocked {
|
||||
path: path
|
||||
.parent()
|
||||
.map(Path::to_path_buf)
|
||||
.unwrap_or_else(|| path.clone()),
|
||||
}),
|
||||
Err(TryLockError::Error(source)) => Err(Error::Io { path, source }),
|
||||
}
|
||||
}
|
||||
|
||||
/// The path of the lock file.
|
||||
pub fn path(&self) -> &Path {
|
||||
&self.path
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for DirectoryLock {
|
||||
fn drop(&mut self) {
|
||||
// Best effort: the OS releases the lock when the descriptor closes anyway.
|
||||
let _ = FileExt::unlock(&self.file);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,310 @@
|
||||
//! Persistence, split into mandatory state and a disposable cache.
|
||||
//!
|
||||
//! | store | contents | on damage |
|
||||
//! |----------------|--------------------------------------------------------|-----------|
|
||||
//! | `state.sqlite` | device identity, network configuration, hostname | hard error |
|
||||
//! | `cache.sqlite` | address hints and other recoverable data | discarded and recreated |
|
||||
//!
|
||||
//! Both files are created with owner-only permissions where the platform
|
||||
//! supports it. The state directory additionally carries an ownership lock, see
|
||||
//! [`DirectoryLock`].
|
||||
//!
|
||||
//! SQLite is synchronous. Every call that touches a database therefore runs on
|
||||
//! a blocking pool via [`tokio::task::spawn_blocking`], and no database lock is
|
||||
//! ever held across a network `await`.
|
||||
|
||||
mod cache;
|
||||
mod lock;
|
||||
mod state;
|
||||
|
||||
pub use cache::{AddressHint, CacheOutcome, CacheStore};
|
||||
pub use lock::DirectoryLock;
|
||||
pub use state::{SCHEMA_VERSION, StateStore, StoredNetwork};
|
||||
|
||||
use std::path::Path;
|
||||
use std::sync::{Arc, Mutex, MutexGuard};
|
||||
|
||||
use crate::config::StoragePaths;
|
||||
use crate::error::{Error, Result};
|
||||
use crate::identity::{DeviceIdentity, NetworkId, NetworkName, NetworkSecret};
|
||||
|
||||
/// Applies the pragmas both stores share.
|
||||
fn apply_common_pragmas(conn: &rusqlite::Connection) -> rusqlite::Result<()> {
|
||||
conn.busy_timeout(std::time::Duration::from_secs(5))?;
|
||||
conn.pragma_update(None, "journal_mode", "WAL")?;
|
||||
conn.pragma_update(None, "synchronous", "NORMAL")?;
|
||||
conn.pragma_update(None, "foreign_keys", "ON")?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Restricts a file to the current user where the platform supports it.
|
||||
#[cfg(unix)]
|
||||
fn restrict_permissions(file: &std::fs::File, path: &Path) -> Result<()> {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
let perms = std::fs::Permissions::from_mode(0o600);
|
||||
file.set_permissions(perms).map_err(|source| Error::Io {
|
||||
path: path.to_path_buf(),
|
||||
source,
|
||||
})
|
||||
}
|
||||
|
||||
/// On Windows, files inherit the parent directory's ACL, which for the
|
||||
/// per-user application data directory is already restricted to that user.
|
||||
#[cfg(not(unix))]
|
||||
fn restrict_permissions(_file: &std::fs::File, _path: &Path) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
fn restrict_path_permissions(path: &Path) -> Result<()> {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)).map_err(|source| {
|
||||
Error::Io {
|
||||
path: path.to_path_buf(),
|
||||
source,
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(not(unix))]
|
||||
fn restrict_path_permissions(_path: &Path) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
fn restrict_dir_permissions(path: &Path) -> Result<()> {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o700)).map_err(|source| {
|
||||
Error::Io {
|
||||
path: path.to_path_buf(),
|
||||
source,
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(not(unix))]
|
||||
fn restrict_dir_permissions(_path: &Path) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn create_dir(path: &Path) -> Result<()> {
|
||||
std::fs::create_dir_all(path).map_err(|source| Error::Io {
|
||||
path: path.to_path_buf(),
|
||||
source,
|
||||
})?;
|
||||
restrict_dir_permissions(path)
|
||||
}
|
||||
|
||||
/// Async facade over both stores, holding the directory ownership lock.
|
||||
///
|
||||
/// Cloning shares the same underlying connections and the same lock.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Storage {
|
||||
inner: Arc<Inner>,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct Inner {
|
||||
state: Mutex<StateStore>,
|
||||
cache: Mutex<Option<CacheStore>>,
|
||||
cache_outcome: CacheOutcome,
|
||||
paths: StoragePaths,
|
||||
lock: Mutex<Option<DirectoryLock>>,
|
||||
}
|
||||
|
||||
impl Storage {
|
||||
/// Opens both stores and takes the ownership lock on the state directory.
|
||||
///
|
||||
/// Fails with [`Error::StateLocked`] if another live agent owns the state
|
||||
/// directory, and with [`Error::StateCorrupted`] if the mandatory state is
|
||||
/// unusable. A broken cache is silently discarded and reported through
|
||||
/// [`Storage::cache_outcome`].
|
||||
pub fn open(paths: &StoragePaths) -> Result<Self> {
|
||||
create_dir(&paths.state_dir)?;
|
||||
create_dir(&paths.cache_dir)?;
|
||||
|
||||
let lock = DirectoryLock::acquire(paths.lock_file())?;
|
||||
let state = StateStore::open(paths.state_db())?;
|
||||
let (cache, cache_outcome) = CacheStore::open_or_reset(paths.cache_db())?;
|
||||
|
||||
Ok(Self {
|
||||
inner: Arc::new(Inner {
|
||||
state: Mutex::new(state),
|
||||
cache: Mutex::new(Some(cache)),
|
||||
cache_outcome,
|
||||
paths: paths.clone(),
|
||||
lock: Mutex::new(Some(lock)),
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
/// What happened to the cache when the agent started.
|
||||
pub fn cache_outcome(&self) -> &CacheOutcome {
|
||||
&self.inner.cache_outcome
|
||||
}
|
||||
|
||||
/// The configured paths.
|
||||
pub fn paths(&self) -> &StoragePaths {
|
||||
&self.inner.paths
|
||||
}
|
||||
|
||||
fn lock_state(&self) -> MutexGuard<'_, StateStore> {
|
||||
match self.inner.state.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
}
|
||||
}
|
||||
|
||||
fn lock_cache(&self) -> MutexGuard<'_, Option<CacheStore>> {
|
||||
match self.inner.cache.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Runs a closure against the mandatory state store on the blocking pool.
|
||||
async fn with_state<T, F>(&self, f: F) -> Result<T>
|
||||
where
|
||||
T: Send + 'static,
|
||||
F: FnOnce(&StateStore) -> Result<T> + Send + 'static,
|
||||
{
|
||||
let inner = Arc::clone(&self.inner);
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = match inner.state.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
f(&guard)
|
||||
})
|
||||
.await
|
||||
.map_err(|err| Error::Storage(format!("state task failed: {err}")))?
|
||||
}
|
||||
|
||||
/// Runs a closure against the cache, tolerating an unavailable cache.
|
||||
///
|
||||
/// If the cache has been disabled because it misbehaved, the closure is
|
||||
/// skipped and `default` is returned.
|
||||
async fn with_cache<T, F>(&self, default: T, f: F) -> T
|
||||
where
|
||||
T: Send + 'static,
|
||||
F: FnOnce(&CacheStore) -> Result<T> + Send + 'static,
|
||||
{
|
||||
let inner = Arc::clone(&self.inner);
|
||||
let joined = tokio::task::spawn_blocking(move || {
|
||||
let guard = match inner.cache.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
match guard.as_ref() {
|
||||
Some(cache) => f(cache),
|
||||
None => Err(Error::Storage("cache is unavailable".into())),
|
||||
}
|
||||
})
|
||||
.await;
|
||||
|
||||
match joined {
|
||||
Ok(Ok(value)) => value,
|
||||
Ok(Err(err)) => {
|
||||
tracing::debug!(%err, "cache operation failed; continuing without it");
|
||||
default
|
||||
}
|
||||
Err(err) => {
|
||||
tracing::debug!(%err, "cache task failed; continuing without it");
|
||||
default
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Loads or creates the persistent device identity.
|
||||
pub async fn device_identity(&self) -> Result<DeviceIdentity> {
|
||||
self.with_state(|state| state.load_or_create_device_identity())
|
||||
.await
|
||||
}
|
||||
|
||||
/// Lists configured networks.
|
||||
pub async fn list_networks(&self) -> Result<Vec<StoredNetwork>> {
|
||||
self.with_state(|state| state.list_networks()).await
|
||||
}
|
||||
|
||||
/// Stores or updates a network configuration.
|
||||
pub async fn upsert_network(
|
||||
&self,
|
||||
network_id: NetworkId,
|
||||
name: NetworkName,
|
||||
secret: NetworkSecret,
|
||||
auto_start: bool,
|
||||
) -> Result<()> {
|
||||
self.with_state(move |state| state.upsert_network(network_id, &name, &secret, auto_start))
|
||||
.await
|
||||
}
|
||||
|
||||
/// Updates the auto-start flag of a network.
|
||||
pub async fn set_auto_start(&self, network_id: NetworkId, auto_start: bool) -> Result<()> {
|
||||
self.with_state(move |state| state.set_auto_start(network_id, auto_start))
|
||||
.await
|
||||
}
|
||||
|
||||
/// Removes a network configuration and its cached hints.
|
||||
pub async fn remove_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
self.with_state(move |state| state.remove_network(network_id))
|
||||
.await?;
|
||||
self.with_cache((), move |cache| cache.forget_network(network_id))
|
||||
.await;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Reads the stored hostname.
|
||||
pub async fn hostname(&self) -> Result<Option<String>> {
|
||||
self.with_state(|state| state.hostname()).await
|
||||
}
|
||||
|
||||
/// Writes the stored hostname.
|
||||
pub async fn set_hostname(&self, hostname: String) -> Result<()> {
|
||||
self.with_state(move |state| state.set_hostname(&hostname))
|
||||
.await
|
||||
}
|
||||
|
||||
/// Records an address hint. Failures are non-fatal.
|
||||
pub async fn record_hint(
|
||||
&self,
|
||||
network_id: NetworkId,
|
||||
endpoint_id: [u8; 32],
|
||||
addr: String,
|
||||
max_per_peer: usize,
|
||||
) {
|
||||
self.with_cache((), move |cache| {
|
||||
cache.record_hint(network_id, &endpoint_id, &addr, max_per_peer)
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
/// Reads cached address hints. Returns an empty list if the cache is gone.
|
||||
pub async fn hints_for_network(&self, network_id: NetworkId) -> Vec<AddressHint> {
|
||||
self.with_cache(Vec::new(), move |cache| cache.hints_for_network(network_id))
|
||||
.await
|
||||
}
|
||||
|
||||
/// Synchronously reads the hostname. Used only during startup.
|
||||
pub(crate) fn hostname_blocking(&self) -> Result<Option<String>> {
|
||||
self.lock_state().hostname()
|
||||
}
|
||||
|
||||
/// Whether the cache is currently usable.
|
||||
pub fn cache_healthy(&self) -> bool {
|
||||
self.lock_cache().is_some()
|
||||
}
|
||||
|
||||
/// Releases the state directory ownership lock.
|
||||
///
|
||||
/// Called by [`crate::Agent::shutdown`] so that a cleanly stopped agent
|
||||
/// leaves its directory immediately claimable by another instance. The
|
||||
/// databases stay open and readable, but this handle no longer owns the
|
||||
/// directory and must not be used to write after this point.
|
||||
pub fn release_ownership_lock(&self) {
|
||||
let mut guard = match self.inner.lock.lock() {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
guard.take();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,331 @@
|
||||
//! The mandatory state store, `state.sqlite`.
|
||||
//!
|
||||
//! Holds the persistent device identity, configured networks (including their
|
||||
//! shared secrets, which are needed to re-derive keys after a restart), the
|
||||
//! stored hostname and auto-start flags.
|
||||
//!
|
||||
//! Corruption is reported, never silently repaired: a damaged state store must
|
||||
//! not quietly turn into a brand new identity.
|
||||
//!
|
||||
//! Future work will add per-author record versions, accepted signed states and
|
||||
//! revocations here. When that lands, writing an event and bumping the author's
|
||||
//! own counter must happen in one SQLite transaction *before* the change is
|
||||
//! published to the network.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use rusqlite::{Connection, OptionalExtension, params};
|
||||
|
||||
use crate::error::{Error, Result};
|
||||
use crate::identity::{DeviceIdentity, NetworkId, NetworkName, NetworkSecret};
|
||||
|
||||
/// Schema version written by this build.
|
||||
pub const SCHEMA_VERSION: i64 = 1;
|
||||
|
||||
/// Key of the stored hostname setting.
|
||||
const SETTING_HOSTNAME: &str = "hostname";
|
||||
|
||||
/// A network as persisted in the state store.
|
||||
///
|
||||
/// The secret is held in a [`NetworkSecret`], which redacts itself from `Debug`
|
||||
/// and zeroizes on drop.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct StoredNetwork {
|
||||
/// Derived public network identifier.
|
||||
pub network_id: NetworkId,
|
||||
/// Network name.
|
||||
pub name: NetworkName,
|
||||
/// Shared secret, needed to re-derive keys after restart.
|
||||
pub secret: NetworkSecret,
|
||||
/// Whether the network is activated automatically at agent startup.
|
||||
pub auto_start: bool,
|
||||
}
|
||||
|
||||
/// The mandatory state store.
|
||||
#[derive(Debug)]
|
||||
pub struct StateStore {
|
||||
conn: Connection,
|
||||
path: PathBuf,
|
||||
}
|
||||
|
||||
impl StateStore {
|
||||
/// Opens (creating if absent) the state store at `path`.
|
||||
///
|
||||
/// Returns [`Error::StateCorrupted`] if the file exists but is not a usable
|
||||
/// database. The file is never deleted or recreated by this function.
|
||||
pub fn open(path: impl AsRef<Path>) -> Result<Self> {
|
||||
let path = path.as_ref().to_path_buf();
|
||||
let existed = path.exists();
|
||||
let conn = Connection::open(&path).map_err(|err| Error::StateCorrupted {
|
||||
path: path.clone(),
|
||||
reason: format!("cannot open database: {err}"),
|
||||
})?;
|
||||
|
||||
super::restrict_path_permissions(&path)?;
|
||||
super::apply_common_pragmas(&conn).map_err(|err| Error::StateCorrupted {
|
||||
path: path.clone(),
|
||||
reason: format!("cannot configure database: {err}"),
|
||||
})?;
|
||||
|
||||
if existed {
|
||||
let integrity: String = conn
|
||||
.query_row("PRAGMA integrity_check", [], |row| row.get(0))
|
||||
.map_err(|err| Error::StateCorrupted {
|
||||
path: path.clone(),
|
||||
reason: format!("integrity check failed: {err}"),
|
||||
})?;
|
||||
if integrity != "ok" {
|
||||
return Err(Error::StateCorrupted {
|
||||
path,
|
||||
reason: format!("integrity check reported: {integrity}"),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
let store = Self { conn, path };
|
||||
store.migrate()?;
|
||||
Ok(store)
|
||||
}
|
||||
|
||||
/// Path of the underlying file.
|
||||
pub fn path(&self) -> &Path {
|
||||
&self.path
|
||||
}
|
||||
|
||||
fn corrupt(&self, reason: impl std::fmt::Display) -> Error {
|
||||
Error::StateCorrupted {
|
||||
path: self.path.clone(),
|
||||
reason: reason.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
fn migrate(&self) -> Result<()> {
|
||||
let found: i64 = self
|
||||
.conn
|
||||
.query_row("PRAGMA user_version", [], |row| row.get(0))
|
||||
.map_err(|err| self.corrupt(format!("cannot read schema version: {err}")))?;
|
||||
|
||||
if found > SCHEMA_VERSION {
|
||||
return Err(Error::UnsupportedSchema {
|
||||
found,
|
||||
supported: SCHEMA_VERSION,
|
||||
});
|
||||
}
|
||||
if found == SCHEMA_VERSION {
|
||||
return self.verify_shape();
|
||||
}
|
||||
|
||||
// Migration 0 -> 1: initial schema.
|
||||
if found < 1 {
|
||||
self.conn
|
||||
.execute_batch(
|
||||
"BEGIN;
|
||||
CREATE TABLE device_identity (
|
||||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||
secret_key BLOB NOT NULL,
|
||||
created_at INTEGER NOT NULL
|
||||
);
|
||||
CREATE TABLE networks (
|
||||
network_id BLOB PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
secret BLOB NOT NULL,
|
||||
auto_start INTEGER NOT NULL DEFAULT 1,
|
||||
created_at INTEGER NOT NULL
|
||||
);
|
||||
CREATE TABLE settings (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL
|
||||
);
|
||||
PRAGMA user_version = 1;
|
||||
COMMIT;",
|
||||
)
|
||||
.map_err(|err| self.corrupt(format!("cannot create schema: {err}")))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Confirms the expected tables exist, so that a truncated or foreign
|
||||
/// database is reported rather than used.
|
||||
fn verify_shape(&self) -> Result<()> {
|
||||
for table in ["device_identity", "networks", "settings"] {
|
||||
let present: Option<String> = self
|
||||
.conn
|
||||
.query_row(
|
||||
"SELECT name FROM sqlite_master WHERE type = 'table' AND name = ?1",
|
||||
params![table],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.optional()
|
||||
.map_err(|err| self.corrupt(format!("cannot inspect schema: {err}")))?;
|
||||
if present.is_none() {
|
||||
return Err(self.corrupt(format!("table `{table}` is missing")));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Loads the stored device identity, creating one on first use.
|
||||
///
|
||||
/// A stored key of the wrong length is a corruption error, never a reason to
|
||||
/// silently mint a new identity.
|
||||
pub fn load_or_create_device_identity(&self) -> Result<DeviceIdentity> {
|
||||
let stored: Option<Vec<u8>> = self
|
||||
.conn
|
||||
.query_row(
|
||||
"SELECT secret_key FROM device_identity WHERE id = 1",
|
||||
[],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.optional()
|
||||
.map_err(|err| self.corrupt(format!("cannot read device identity: {err}")))?;
|
||||
|
||||
if let Some(bytes) = stored {
|
||||
let bytes: [u8; 32] = bytes.as_slice().try_into().map_err(|_| {
|
||||
self.corrupt(format!(
|
||||
"stored device key has {} bytes, expected 32; refusing to replace it",
|
||||
bytes.len()
|
||||
))
|
||||
})?;
|
||||
return Ok(DeviceIdentity::from_secret_bytes(&bytes));
|
||||
}
|
||||
|
||||
let identity = DeviceIdentity::generate();
|
||||
self.conn
|
||||
.execute(
|
||||
"INSERT INTO device_identity (id, secret_key, created_at) VALUES (1, ?1, ?2)",
|
||||
params![identity.secret_bytes().as_slice(), now_unix()],
|
||||
)
|
||||
.map_err(|err| self.corrupt(format!("cannot store device identity: {err}")))?;
|
||||
Ok(identity)
|
||||
}
|
||||
|
||||
/// Inserts or updates a network configuration.
|
||||
pub fn upsert_network(
|
||||
&self,
|
||||
network_id: NetworkId,
|
||||
name: &NetworkName,
|
||||
secret: &NetworkSecret,
|
||||
auto_start: bool,
|
||||
) -> Result<()> {
|
||||
self.conn
|
||||
.execute(
|
||||
"INSERT INTO networks (network_id, name, secret, auto_start, created_at)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5)
|
||||
ON CONFLICT(network_id) DO UPDATE SET
|
||||
name = excluded.name,
|
||||
secret = excluded.secret,
|
||||
auto_start = excluded.auto_start",
|
||||
params![
|
||||
network_id.as_bytes().as_slice(),
|
||||
name.as_str(),
|
||||
secret.expose(),
|
||||
auto_start as i64,
|
||||
now_unix()
|
||||
],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot store network: {err}")))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Sets the auto-start flag of a configured network.
|
||||
pub fn set_auto_start(&self, network_id: NetworkId, auto_start: bool) -> Result<()> {
|
||||
self.conn
|
||||
.execute(
|
||||
"UPDATE networks SET auto_start = ?2 WHERE network_id = ?1",
|
||||
params![network_id.as_bytes().as_slice(), auto_start as i64],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot update network: {err}")))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Removes a network configuration entirely.
|
||||
pub fn remove_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
self.conn
|
||||
.execute(
|
||||
"DELETE FROM networks WHERE network_id = ?1",
|
||||
params![network_id.as_bytes().as_slice()],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot remove network: {err}")))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Lists every configured network.
|
||||
pub fn list_networks(&self) -> Result<Vec<StoredNetwork>> {
|
||||
let mut stmt = self
|
||||
.conn
|
||||
.prepare("SELECT network_id, name, secret, auto_start FROM networks ORDER BY name")
|
||||
.map_err(|err| Error::Storage(format!("cannot list networks: {err}")))?;
|
||||
let rows = stmt
|
||||
.query_map([], |row| {
|
||||
let id: Vec<u8> = row.get(0)?;
|
||||
let name: String = row.get(1)?;
|
||||
let secret: Vec<u8> = row.get(2)?;
|
||||
let auto_start: i64 = row.get(3)?;
|
||||
Ok((id, name, secret, auto_start != 0))
|
||||
})
|
||||
.map_err(|err| Error::Storage(format!("cannot list networks: {err}")))?;
|
||||
|
||||
let mut out = Vec::new();
|
||||
for row in rows {
|
||||
let (id, name, secret, auto_start) =
|
||||
row.map_err(|err| Error::Storage(format!("cannot read network row: {err}")))?;
|
||||
let id: [u8; 32] = id
|
||||
.as_slice()
|
||||
.try_into()
|
||||
.map_err(|_| self.corrupt("stored network id is not 32 bytes"))?;
|
||||
out.push(StoredNetwork {
|
||||
network_id: NetworkId::from_bytes(id),
|
||||
name: NetworkName::new(name)?,
|
||||
secret: NetworkSecret::from_bytes(secret)?,
|
||||
auto_start,
|
||||
});
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Reads the stored hostname, if any.
|
||||
pub fn hostname(&self) -> Result<Option<String>> {
|
||||
self.get_setting(SETTING_HOSTNAME)
|
||||
}
|
||||
|
||||
/// Stores the hostname.
|
||||
///
|
||||
/// Today this is a local setting. In the future a rename must be a signed
|
||||
/// record that revokes the specific old binding and announces the new one,
|
||||
/// ideally atomically in one record.
|
||||
pub fn set_hostname(&self, hostname: &str) -> Result<()> {
|
||||
self.set_setting(SETTING_HOSTNAME, hostname)
|
||||
}
|
||||
|
||||
/// Reads an arbitrary setting.
|
||||
pub fn get_setting(&self, key: &str) -> Result<Option<String>> {
|
||||
self.conn
|
||||
.query_row(
|
||||
"SELECT value FROM settings WHERE key = ?1",
|
||||
params![key],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.optional()
|
||||
.map_err(|err| Error::Storage(format!("cannot read setting `{key}`: {err}")))
|
||||
}
|
||||
|
||||
/// Writes an arbitrary setting.
|
||||
pub fn set_setting(&self, key: &str, value: &str) -> Result<()> {
|
||||
self.conn
|
||||
.execute(
|
||||
"INSERT INTO settings (key, value) VALUES (?1, ?2)
|
||||
ON CONFLICT(key) DO UPDATE SET value = excluded.value",
|
||||
params![key, value],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot write setting `{key}`: {err}")))?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Seconds since the Unix epoch, saturating at 0 before it.
|
||||
pub(crate) fn now_unix() -> i64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_secs() as i64)
|
||||
.unwrap_or(0)
|
||||
}
|
||||
Reference in New Issue
Block a user