Files
tsunagi/crates/tsunagi-core/src/config.rs
T
Aleksandr Bogomiakov e701986a39 Add ephemeral pairwise Amnezia transport and runtime fallback
Plain WG and QUIC can be blocked or severely degraded while the iroh control
connection still works. Add an awg carrier with one independent UDP socket,
WireGuard engine and randomized masking profile per network/peer attempt.
The smaller endpoint id negotiates fresh outer keys, PSK, header protection,
header ranges, padding and signature templates through private authenticated
control messages. Restart or fallback reentry generates a new profile;
ordinary rekeying preserves the current profile. No pair parameters enter
signed state, capabilities, persistent storage or public message events.
Private payloads and serialized control buffers redact and zeroize themselves.

Adapt the bounded packet masking engine from RunaRoo/AmneziaWG-RS at commit
51579a7d6e8882d1d8d6d00822f09e18a9423b73, retaining its Apache license and
provenance. Declare WTFPL AND Apache-2.0 in package metadata to cover both
new glue and the adapted source. Reuse boringtun instead of upstream's Node/global executors and
interface machinery. This is Tsunagi AWG wire version 1, with standard WG
headers restored before authentication; external Amnezia server compatibility
is not implemented. DNS/SIP signature generators vary fields instead of
replaying captures. Each attempt closes independently, bounds allocation,
queues, candidates and fragment assemblies, and supports the full 64 KiB
carrier contract using UDP packets at most 1200 bytes. Yield between fragment
batches so large messages do not starve local socket readers.

Keep carrier encryption separate from end-to-end tunnels so multihop relays
still forward opaque ciphertext. The system owns the sole interface, signed
addresses and admission. Plugins receive neither network secrets nor paths.
Expose a bounded opaque control sender at the transport boundary; core control
messages remain outside protocol crates. Optional STUN on the pair's actual
socket adds IPv4 public candidates in CLI direct/relay mode, with a finite
query deadline and an off/alternate-server option. Local mode remains offline.
Symmetric NAT, IPv6-only paths and arbitrary UDP blocking can still require
fallback; no AWG-over-QUIC or hidden AWG relay is introduced.

Default carrier order is wg, awg, wg-quic, tcp-tls. Advance the fallback cycle
after setup failures and sustained runtime loss, even after earlier cooldowns
expire. Authenticated delivery probes traverse the actual encrypted tunnel,
fit the interface MTU and never enter the TUN. Retire after a 20-second warmup
and at least 60% loss over at least 15 completed recent observations. Preserve
logical tunnels and control sessions while explicitly closing physical links,
including TCP reader/writer tasks. Retain the newest private probe settings
across independently queued control negotiation and inbound installation.
CLI/tray reports keep end-to-end protocol and actual carrier separate.

Control ALPN/version becomes tsunagi/ctrl/6 for the new private message;
upgrade all members together. Data ALPN/envelopes remain version 5. Network
identity derivation, transcript encoding, device identities, signed records,
saved network configuration and release versions are unchanged.

Validation on Linux through nix develop:
- cargo fmt --all -- --check
- cargo clippy --locked --workspace --all-targets -- -D warnings
- cargo test --locked --workspace --all-targets, with the inherited
  TSUNAGI_CONTROL_SOCKET override removed: 504 passed, 2 pre-existing ignored

Tests use real iroh membership, SQLite, UDP sockets and WireGuard encryption
with memory TUNs. Cover three independent pairs, full-size IP and carrier
frames, restart freshness/socket release, 80% carrier datagram loss, fallback
to real iroh, fresh retries with working delivery checks, unequal small MTUs,
malformed negotiations/profiles, role and resource bounds, probe forgery and
STUN parsing. Earlier compile/lint iterations and focused tests caught RNG
API/initializer mistakes, a retry assertion racing path installation, and a
large-fragment burst overflowing loopback buffers; corrected before these
successful checks. Normalized upstream license line endings for diff checks.
Public NAT/STUN connectivity, actual DPI effectiveness, WAN performance,
per-pair memory measurement, macOS/Windows, privileged TUN operations and
manual tray interaction were not exercised.
2026-10-11 13:52:10 +01:00

529 lines
20 KiB
Rust

//! Library configuration.
//!
//! Everything the agent needs is passed in explicitly. The library reads no
//! environment variables, installs no global state and picks no default
//! directories behind the caller's back — [`StoragePaths::user_default`] exists
//! but must be called on purpose.
use std::net::SocketAddr;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::Duration;
use crate::dataplane::SharedPlugin;
use crate::discovery::{MainlineDiscovery, NetworkDiscovery};
use crate::error::Result;
/// Qualifier/organisation/application triple used for platform directories.
///
/// Unused on macOS, where the default paths are fixed system locations rather
/// than per-user directories derived from this.
#[cfg_attr(target_os = "macos", allow(dead_code))]
const APP_NAME: &str = "tsunagi";
/// Rendezvous slots per network; this bounds a discovery sample, not membership.
pub const DHT_SLOTS: u8 = 16;
/// BEP44 permits 1000 bencoded bytes; a 996-byte string has a four-byte prefix.
pub const DHT_MAX_VALUE: usize = 996;
/// Maximum direct addresses in one rendezvous record.
pub const DHT_MAX_ADDRS: usize = 8;
/// Maximum relay URL bytes in a rendezvous record.
pub const DHT_MAX_RELAY_LEN: usize = 512;
/// Application freshness, independent of storage nodes' own expiration.
pub const DHT_RECORD_TTL: Duration = Duration::from_secs(900);
/// Clock skew tolerated when reading a recently published record.
pub const DHT_CLOCK_SKEW: Duration = Duration::from_secs(120);
/// Maximum reassembled data-transport payload, independent of the path MTU.
pub const MAX_DATA_DATAGRAM: usize = 64 * 1024;
/// Maximum incomplete datagrams held by one authenticated data link.
pub const MAX_DATA_ASSEMBLIES: usize = 64;
/// Maximum allocated payload bytes for incomplete datagrams on one data link.
pub const MAX_DATA_REASSEMBLY_BYTES: usize = 256 * 1024;
/// Maximum fragments of one logical datagram, including after a path MTU change.
pub const MAX_DATA_FRAGMENTS: usize = 128;
/// Time allowed to assemble a datagram; loss never stalls later datagrams.
pub const DATA_REASSEMBLY_TTL: Duration = Duration::from_secs(5);
/// Completed or discarded packet IDs retained to ignore late duplicate fragments.
pub const DATA_RECENT_IDS: usize = 128;
/// Where the two stores live.
///
/// The mandatory state and the disposable cache are separate both logically and
/// physically, so that the cache can be deleted at any time without touching
/// identity or network configuration.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct StoragePaths {
/// Directory holding `state.sqlite` and the ownership lock.
pub state_dir: PathBuf,
/// Directory holding `cache.sqlite`.
pub cache_dir: PathBuf,
}
impl StoragePaths {
/// Uses explicit directories. Tests always use temporary directories.
pub fn new(state_dir: impl Into<PathBuf>, cache_dir: impl Into<PathBuf>) -> Self {
Self {
state_dir: state_dir.into(),
cache_dir: cache_dir.into(),
}
}
/// Puts both stores under one root, in `state/` and `cache/` subdirectories.
pub fn under(root: impl AsRef<Path>) -> Self {
let root = root.as_ref();
Self {
state_dir: root.join("state"),
cache_dir: root.join("cache"),
}
}
/// The default platform directories.
///
/// Per user on Linux and Windows. On macOS the agent runs as a root
/// LaunchDaemon (creating a utun interface needs root, and `$HOME` under
/// `sudo` points at the wrong place), so the stores live in the system
/// locations instead — `/var/db/tsunagi` for the mandatory state and a
/// disposable cache beside it. A caller that wants somewhere else passes
/// explicit paths to [`StoragePaths::new`] or [`StoragePaths::under`].
pub fn user_default() -> Result<Self> {
#[cfg(target_os = "macos")]
{
Ok(Self {
state_dir: PathBuf::from("/var/db/tsunagi"),
cache_dir: PathBuf::from("/var/db/tsunagi/cache"),
})
}
#[cfg(not(target_os = "macos"))]
{
let dirs = directories::ProjectDirs::from("", "", APP_NAME).ok_or_else(|| {
crate::error::Error::Storage(
"no valid home directory for platform config paths".into(),
)
})?;
Ok(Self {
state_dir: dirs.data_dir().to_path_buf(),
cache_dir: dirs.cache_dir().to_path_buf(),
})
}
}
/// Path of the mandatory state database.
pub fn state_db(&self) -> PathBuf {
self.state_dir.join("state.sqlite")
}
/// Path of the disposable cache database.
pub fn cache_db(&self) -> PathBuf {
self.cache_dir.join("cache.sqlite")
}
/// Path of the ownership lock file.
pub fn lock_file(&self) -> PathBuf {
self.state_dir.join("state.lock")
}
}
/// How the iroh endpoint is allowed to reach the outside world.
///
/// The default is [`TransportPolicy::LocalOnly`] so that a plain
/// `AgentConfig::new(...)` never reaches the internet by accident. Callers that
/// want public connectivity must opt in.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub enum TransportPolicy {
/// No relays, no address lookup service, no port mapping.
///
/// Suitable for tests and for fully local deployments.
#[default]
LocalOnly,
/// Public address lookup, but no relays.
///
/// Enables iroh's DNS/pkarr address lookup against the public service run
/// by Number 0 (the company behind iroh) at `dns.iroh.link`. This endpoint
/// publishes a signed record of its own addresses there, so peers can dial
/// it by endpoint id alone.
DirectOnly,
/// iroh's standard behaviour: public address lookup plus public relays.
///
/// Maps to iroh's own `presets::N0`. As well as the address lookup above,
/// it uses Number 0's public relay servers as a fallback when a direct
/// path cannot be hole punched. They are fine for development and carry no
/// availability guarantee.
N0Defaults,
}
/// Bounds applied to everything that comes off the network.
///
/// Each of these is enforced before memory is allocated for the corresponding
/// object where that is possible (notably [`Limits::max_frame_len`]).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Limits {
/// Largest accepted control frame payload, in bytes.
pub max_frame_len: usize,
/// Largest accepted hostname, in bytes.
pub max_hostname_len: usize,
/// Largest number of plugin capabilities in one announcement.
pub max_capabilities: usize,
/// Largest opaque plugin payload, in bytes.
pub max_capability_data_len: usize,
/// Largest private transport negotiation payload (also hard-capped at 4096 bytes).
pub max_transport_payload_len: usize,
/// Largest accepted echo payload in a ping/pong exchange, in bytes.
pub max_echo_payload_len: usize,
/// Largest accepted free-text reason string, in bytes.
pub max_reason_len: usize,
/// Largest number of signed records accepted in one snapshot.
pub max_state_records: usize,
/// Deadline for the whole handshake.
pub handshake_timeout: Duration,
/// Deadline for one outbound dial attempt.
pub dial_timeout: Duration,
/// Deadline for writing one control frame.
///
/// Liveness of an established session is delegated to QUIC: iroh configures
/// keep-alives and an idle timeout, so a dead peer surfaces as a read error
/// rather than needing a protocol-level heartbeat here.
pub write_timeout: Duration,
/// Maximum simultaneous outbound dials per network.
pub max_concurrent_dials: usize,
/// Maximum unverified discovery candidates retained per network.
pub max_discovery_candidates: usize,
/// Maximum simultaneous authenticated sessions per network.
pub max_sessions_per_network: usize,
/// Maximum simultaneous inbound connections being handshaken.
pub max_inbound_handshakes: usize,
/// Capacity of a session's outbound queue, providing backpressure.
pub session_send_queue: usize,
/// Capacity of the event broadcast channel.
pub event_buffer: usize,
/// Maximum address hints kept per peer in the cache.
pub max_hints_per_peer: usize,
}
impl Default for Limits {
fn default() -> Self {
Self {
max_frame_len: 64 * 1024,
max_hostname_len: 255,
max_capabilities: 16,
max_transport_payload_len: 4096,
max_capability_data_len: 4 * 1024,
max_echo_payload_len: 4 * 1024,
max_reason_len: 256,
max_state_records: crate::state::MAX_RECORDS_PER_MESSAGE,
handshake_timeout: Duration::from_secs(10),
dial_timeout: Duration::from_secs(10),
write_timeout: Duration::from_secs(30),
max_concurrent_dials: 8,
max_discovery_candidates: 16,
max_sessions_per_network: 64,
max_inbound_handshakes: 32,
session_send_queue: 64,
event_buffer: 512,
max_hints_per_peer: 8,
}
}
}
/// Bounded exponential backoff with jitter for reconnect attempts.
#[derive(Debug, Clone, PartialEq)]
pub struct ReconnectPolicy {
/// Delay before the first retry.
pub initial_delay: Duration,
/// Upper bound on the delay.
pub max_delay: Duration,
/// Multiplier applied after each failed attempt.
pub factor: f64,
/// Fraction of the delay applied as random jitter, in `0.0..=1.0`.
pub jitter: f64,
/// Give up on a peer after this many consecutive failures until it is seen
/// again by discovery. `None` means never give up while the network is up.
pub max_consecutive_failures: Option<u32>,
}
impl Default for ReconnectPolicy {
fn default() -> Self {
Self {
initial_delay: Duration::from_millis(250),
max_delay: Duration::from_secs(30),
factor: 2.0,
jitter: 0.3,
max_consecutive_failures: None,
}
}
}
impl ReconnectPolicy {
/// Delay to wait before retry number `attempt` (1-based), with jitter.
pub(crate) fn delay_for(&self, attempt: u32) -> Duration {
let exp = self.factor.powi(attempt.saturating_sub(1).min(32) as i32);
let base = self.initial_delay.as_secs_f64() * exp;
let capped = base.min(self.max_delay.as_secs_f64());
let jitter = self.jitter.clamp(0.0, 1.0);
let factor = 1.0 - jitter + jitter * 2.0 * rand::random::<f64>();
Duration::from_secs_f64((capped * factor).max(0.0))
}
}
/// Scheduling of candidate lookup and independent self-publication.
#[derive(Debug, Clone)]
pub struct DiscoveryPolicy {
/// Mean interval between successful self-publications (20% jitter).
pub publish_interval: Duration,
/// Initial delay between unsuccessful bootstrap lookups.
pub lookup_interval: Duration,
/// Maximum delay between bootstrap lookups.
pub max_lookup_interval: Duration,
/// Continuous isolation before a previously connected network searches again.
pub reconnect_delay: Duration,
/// Deadline for one backend operation.
pub request_timeout: Duration,
}
impl Default for DiscoveryPolicy {
fn default() -> Self {
Self {
publish_interval: Duration::from_secs(300),
lookup_interval: Duration::from_secs(5),
max_lookup_interval: Duration::from_secs(30),
reconnect_delay: Duration::from_secs(60),
request_timeout: Duration::from_secs(60),
}
}
}
/// Everything needed to start an [`crate::Agent`].
#[derive(Clone)]
pub struct AgentConfig {
/// Where the mandatory state and the disposable cache live.
pub paths: StoragePaths,
/// Explicit local bind addresses. Empty means iroh's defaults.
///
/// Tests bind to `127.0.0.1:0` so each agent gets a dynamic port.
pub bind_addrs: Vec<SocketAddr>,
/// How much external connectivity machinery the endpoint may use.
pub transport: TransportPolicy,
/// Real QUIC transport settings for constrained-path integration tests.
#[cfg(feature = "testing")]
pub test_quic_transport: Option<iroh::endpoint::QuicTransportConfig>,
/// Peers with no direct data path, for tests. See
/// [`AgentConfig::with_unreachable_data_peers`].
#[cfg(feature = "testing")]
pub unreachable_data_peers: Arc<std::sync::Mutex<std::collections::HashSet<iroh::EndpointId>>>,
/// Hostname announced to peers. `None` keeps whatever the state store holds,
/// falling back to the OS hostname and finally to a short endpoint id.
pub hostname: Option<String>,
/// Additional discovery backend, composed with Mainline when it is enabled.
pub discovery: Option<Arc<dyn NetworkDiscovery>>,
/// Optional Mainline client, shared by this agent's networks. The CLI enables
/// it by default except in local-only mode; library callers opt in explicitly.
pub dht: Option<MainlineDiscovery>,
/// Lookup, publication and recovery scheduling.
pub discovery_policy: DiscoveryPolicy,
/// How often each active network maintains known peers and observes address changes.
pub discovery_interval: Duration,
/// Bounds applied to network input.
pub limits: Limits,
/// Reconnect backoff policy.
pub reconnect: ReconnectPolicy,
/// IP plugins whose capabilities are announced and dispatched.
pub plugins: Vec<SharedPlugin>,
/// Custom packet transports registered per protocol.
pub custom_transports:
std::collections::HashMap<String, Arc<dyn crate::dataplane::PacketTransport>>,
/// Where the overlay interface comes from, when the agent should have one.
///
/// One agent, one interface: it belongs here rather than to a protocol,
/// because every protocol carries traffic for the same addresses on it.
/// `None` means no interface — the protocols still run and their packets
/// are discarded.
pub tun_factory: Option<Arc<dyn crate::overlay::TunFactory>>,
/// The name to ask the operating system for.
pub interface_name: String,
/// The interface MTU.
pub interface_mtu: u32,
/// Whether new networks derive their IPv4 range from their network id.
///
/// Enabled by default. Explicit ranges and IPv4 opt-out disable this.
pub overlay_ipv4_auto: bool,
/// The IPv4 overlay range this agent proposes when automatic selection is off.
///
/// Addresses are allocated from it and recorded in signed state, so a
/// participant keeps the same one across restarts. A network that has
/// already settled on another range wins: a joining agent adopts what it
/// finds rather than imposing this.
pub overlay_ipv4_range: Option<crate::state::Ipv4Range>,
}
impl AgentConfig {
/// Creates a configuration with local-only transport and default limits.
pub fn new(paths: StoragePaths) -> Self {
Self {
paths,
bind_addrs: Vec::new(),
transport: TransportPolicy::default(),
#[cfg(feature = "testing")]
test_quic_transport: None,
#[cfg(feature = "testing")]
unreachable_data_peers: Arc::new(std::sync::Mutex::new(
std::collections::HashSet::new(),
)),
hostname: None,
discovery: None,
dht: None,
discovery_policy: DiscoveryPolicy::default(),
discovery_interval: Duration::from_secs(5),
limits: Limits::default(),
reconnect: ReconnectPolicy::default(),
plugins: Vec::new(),
custom_transports: std::collections::HashMap::new(),
tun_factory: None,
interface_name: "tsun0".to_string(),
interface_mtu: 1280,
overlay_ipv4_auto: true,
overlay_ipv4_range: None,
}
}
/// Binds to loopback with a dynamic port. Used by the test suite.
pub fn with_loopback_bind(mut self) -> Self {
self.bind_addrs = vec![
SocketAddr::from(([127, 0, 0, 1], 0)),
SocketAddr::from(([0, 0, 0, 0, 0, 0, 0, 1], 0)),
];
self
}
/// Sets an explicit list of bind addresses.
pub fn with_bind_addrs(mut self, addrs: impl IntoIterator<Item = SocketAddr>) -> Self {
self.bind_addrs = addrs.into_iter().collect();
self
}
/// Peers this agent will not open or accept a **direct** data link with.
///
/// Test-only, and behind the `testing` feature so a release build has
/// no such switch. The one arrangement that cannot be made on a single
/// host is two agents that both reach a third and not each other,
/// which is exactly the case a relay exists for.
#[cfg(feature = "testing")]
pub fn with_unreachable_data_peers(
mut self,
peers: Arc<std::sync::Mutex<std::collections::HashSet<iroh::EndpointId>>>,
) -> Self {
self.unreachable_data_peers = peers;
self
}
/// Sets the transport policy.
pub fn with_transport(mut self, transport: TransportPolicy) -> Self {
self.transport = transport;
self
}
/// Sets the discovery backend.
pub fn with_discovery(mut self, discovery: Arc<dyn NetworkDiscovery>) -> Self {
self.discovery = Some(discovery);
self
}
/// Adds Mainline discovery alongside the configured candidate backend.
pub fn with_dht(mut self, dht: MainlineDiscovery) -> Self {
self.dht = Some(dht);
self
}
/// Sets lookup, publication and isolation recovery timing.
pub fn with_discovery_policy(mut self, policy: DiscoveryPolicy) -> Self {
self.discovery_policy = policy;
self
}
/// Sets the known-peer maintenance and endpoint-address observation interval.
pub fn with_discovery_interval(mut self, interval: Duration) -> Self {
self.discovery_interval = interval;
self
}
/// Sets the announced hostname.
pub fn with_hostname(mut self, hostname: impl Into<String>) -> Self {
self.hostname = Some(hostname.into());
self
}
/// Sets the IPv4 overlay range this agent proposes, or disables IPv4.
pub fn with_overlay_ipv4_range(mut self, range: Option<crate::state::Ipv4Range>) -> Self {
self.overlay_ipv4_auto = false;
self.overlay_ipv4_range = range;
self
}
/// Registers an IP plugin.
pub fn with_plugin(mut self, plugin: SharedPlugin) -> Self {
self.plugins.push(plugin);
self
}
/// Registers a custom packet transport for a specific protocol.
pub fn with_custom_transport(
mut self,
protocol: impl Into<String>,
transport: Arc<dyn crate::dataplane::PacketTransport>,
) -> Self {
self.custom_transports.insert(protocol.into(), transport);
self
}
/// Gives the agent an overlay interface, from this factory.
pub fn with_interface(
mut self,
factory: Arc<dyn crate::overlay::TunFactory>,
name: impl Into<String>,
mtu: u32,
) -> Self {
self.tun_factory = Some(factory);
self.interface_name = name.into();
self.interface_mtu = mtu;
self
}
/// Replaces the limits.
pub fn with_limits(mut self, limits: Limits) -> Self {
self.limits = limits;
self
}
/// Replaces the reconnect policy.
pub fn with_reconnect(mut self, reconnect: ReconnectPolicy) -> Self {
self.reconnect = reconnect;
self
}
}
impl std::fmt::Debug for AgentConfig {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("AgentConfig")
.field("paths", &self.paths)
.field("bind_addrs", &self.bind_addrs)
.field("transport", &self.transport)
.field("hostname", &self.hostname)
.field("discovery", &self.discovery.as_ref().map(|d| d.name()))
.field("dht", &self.dht.is_some())
.field("discovery_policy", &self.discovery_policy)
.field("discovery_interval", &self.discovery_interval)
.field("limits", &self.limits)
.field("reconnect", &self.reconnect)
.field(
"plugins",
&self
.plugins
.iter()
.map(|p| p.protocol_id().to_string())
.collect::<Vec<_>>(),
)
.finish()
}
}
/// Maximum number of transport links a routed datagram may traverse.
pub const ROUTING_HOP_LIMIT: u8 = 16;