From ed28e86748804a8972f850a8c83d3bb6ac23209c Mon Sep 17 00:00:00 2001 From: Aleksandr Bogomiakov Date: Sun, 11 Oct 2026 11:58:27 +0100 Subject: [PATCH] Derive default network ranges and switch overlapping networks Every first network previously proposed 10.13.37.0/24. Independently created networks therefore collided when a device joined more than one of them. Use the existing stable NetworkId-derived /24 in 10/8 for every new network, independent of activation order or peer availability. Legacy and explicit signed ranges outrank the derived default; remaining range disagreements still use author-id ordering. The derivation, identity labels, handshake and signed record encoding stay unchanged. There is no creator, timestamp or vote. Reserve ranges before starting network actors and serialise start, stop and switching through the agent lifecycle lock. Refuse equal or nested overlaps without changing a network's subnet locally. A late authenticated range that collides suspends only its network before updating plugins or the interface. Coalesced watch notifications let the agent clean up outside the network actor and reconsider suspended networks when ranges change. An explicit switch stops every blocker before activating the chosen network. Suspended networks retain their startup request and resume whenever their ranges become free, including switching from a wide range to a narrow one. A manual stop cancels that request even for an already suspended network. Persist preference order and enabling together in an SQLite transaction so the user's selection survives restart; no database schema migration is needed. CLI status and network listing show blocker names, ids, ranges and the start command. Network start switches explicitly and reports suspended networks; joining an overlapping network reports it as configured but inactive. The tray adds conflict details, an amber icon, Switch to this network and Keep stopped actions. Local IPC wire version is 19 for the extended reports, so restart the agent and tray together. Release versions and peer wire formats are unchanged. Update the user documentation and compatibility tests. Validation on Linux through nix develop: - cargo fmt --all -- --check - cargo clippy --locked --workspace --all-targets -- -D warnings - cargo test --locked --workspace --all-targets with TSUNAGI_CONTROL_SOCKET removed from the test environment: 486 passed, 2 pre-existing ignored tests - final focused range_conflicts integration suite after making discovery fixtures deterministic: 6 passed Earlier validation attempts caught outdated diagnostic assertions, a new test helper scope error, and the inherited system control socket override. Those were corrected before the successful runs. Plain cargo was unavailable in the host shell, so validation used the Nix development toolchain. macOS, Windows, privileged host TUN operations and manual tray interaction were not exercised. --- README.md | 12 +- crates/tsunagi-core/src/agent/mod.rs | 362 +++++++++++------- crates/tsunagi-core/src/agent/network.rs | 128 +++---- crates/tsunagi-core/src/agent/status.rs | 22 +- crates/tsunagi-core/src/config.rs | 10 +- crates/tsunagi-core/src/error.rs | 13 + crates/tsunagi-core/src/ipc/mod.rs | 29 +- crates/tsunagi-core/src/overlay/router.rs | 23 +- crates/tsunagi-core/src/state/mod.rs | 74 ++-- crates/tsunagi-core/src/storage/mod.rs | 27 ++ crates/tsunagi-core/src/storage/state.rs | 44 ++- crates/tsunagi-core/tests/leaving.rs | 2 +- .../tsunagi-core/tests/network_isolation.rs | 11 +- crates/tsunagi-core/tests/range_conflicts.rs | 336 ++++++++++++++++ crates/tsunagi-tray/src/agent.rs | 28 +- crates/tsunagi-tray/src/icon.rs | 1 + crates/tsunagi-tray/src/ui.rs | 29 +- crates/tsunagi-tray/src/watch.rs | 28 ++ crates/tsunagi-wg-quic/tests/local_control.rs | 1 + crates/tsunagi-wg-quic/tests/wireguard.rs | 26 +- crates/tsunagi/src/main.rs | 184 ++++++--- crates/tsunagi/tests/network_cli.rs | 175 ++++++++- docs/testing.md | 7 + docs/wireguard.md | 37 +- 24 files changed, 1291 insertions(+), 318 deletions(-) create mode 100644 crates/tsunagi-core/tests/range_conflicts.rs diff --git a/README.md b/README.md index ce41f95..539f322 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,17 @@ tsng join -n home -s tsn1… # every other machine tsng status # who is online, over which path ``` -Every member gets an address in `10.13.37.0/24` and a name like `.home`. +Every new network gets a stable pseudorandom `/24` in `10.0.0.0/8`, derived +from its network id. Its members agree on that range even when starting offline, +and each gets a name like `.home`. Existing signed ranges are preserved. + +If two configured networks have overlapping ranges, the agent runs only one. +`tsng status`, `tsng network` and the tray show which network blocks the +other. `tsng network start ` or the tray’s “Switch to this network” +button suspends the conflicting networks. They resume automatically when the +selected network stops; the selection survives restarting the agent. Explicitly +stopping a suspended network cancels its automatic resume. + Machines are called `tsunagi-` until you rename them: `tsng id hostname laptop`. ### Exit nodes diff --git a/crates/tsunagi-core/src/agent/mod.rs b/crates/tsunagi-core/src/agent/mod.rs index a11c543..848edb2 100644 --- a/crates/tsunagi-core/src/agent/mod.rs +++ b/crates/tsunagi-core/src/agent/mod.rs @@ -27,14 +27,14 @@ mod status; pub use events::Event; pub use status::{ AgentStatus, CandidateStatus, DataPath, HostnameConflict, MemberStatus, NetworkMetrics, - NetworkState, NetworkStatus, OverlayStatus, PeerStatus, + NetworkState, NetworkStatus, OverlayStatus, PeerStatus, RangeConflict, }; use std::collections::HashMap; use std::sync::{Arc, Weak}; use iroh::{EndpointAddr, EndpointId}; -use tokio::sync::{RwLock, broadcast, mpsc, oneshot}; +use tokio::sync::{Mutex, RwLock, broadcast, mpsc, oneshot, watch}; use tokio::task::JoinHandle; use crate::BoxFuture; @@ -90,6 +90,10 @@ struct Inner { events: broadcast::Sender, networks: RwLock>, shutdown: Shutdown, + /// Serialises start, stop, switching and conflict cleanup, including teardown. + lifecycle: Mutex<()>, + range_changed: watch::Sender<()>, + range_task: std::sync::Mutex>>, accept_task: std::sync::Mutex>>, plugin_task: std::sync::Mutex>>, transport: std::sync::OnceLock>, @@ -234,6 +238,7 @@ impl Agent { let (events, _) = broadcast::channel(config.limits.event_buffer); let limits = Arc::new(config.limits.clone()); + let (range_changed, range_updates) = watch::channel(()); let inner = Arc::new(Inner { limits, storage, @@ -243,6 +248,9 @@ impl Agent { events, networks: RwLock::new(HashMap::new()), shutdown: Shutdown::new(), + lifecycle: Mutex::new(()), + range_changed, + range_task: std::sync::Mutex::new(None), accept_task: std::sync::Mutex::new(None), plugin_task: std::sync::Mutex::new(None), transport: std::sync::OnceLock::new(), @@ -326,11 +334,16 @@ impl Agent { let agent = Self { inner }; - for stored in agent.inner.storage.list_networks().await? { - if stored.auto_start { - let keys = NetworkKeys::derive(&stored.name, &stored.secret); - agent.activate_with_keys(keys).await?; - } + { + let _guard = agent.inner.lifecycle.lock().await; + agent.resume_networks_locked().await?; + } + let task = tokio::spawn(range_conflict_loop( + Arc::downgrade(&agent.inner), + range_updates, + )); + if let Ok(mut guard) = agent.inner.range_task.lock() { + *guard = Some(task); } Ok(agent) @@ -374,100 +387,65 @@ impl Agent { Ok(()) } - /// Reserves the configured overlay range for a network, if it can. - /// - /// Done here, where activations are serialised, rather than inside the - /// runtime: whether a range is free depends on what the other networks - /// took, and deciding that in a task would make the answer depend on - /// which task ran first. - /// - /// `None` means this agent will not propose a range for that network and - /// waits to adopt whatever it settles on. One agent has one interface, so - /// proposing a range it could not route would be worse than having none: - /// the lowest author's range wins, and the collision would spread. - async fn reserve_range(&self, network: NetworkId) -> RangePlan { - let reserve = |wanted: Ipv4Range| { - let reservation = crate::overlay::NetworkRoutes { - exit: Default::default(), - broadcast: Default::default(), - range: Some(wanted), - local: None, - peers: Vec::new(), - }; - self.inner.routes.set_network(network, reservation).is_ok() - }; - - // If this network already has signed records in storage, check if its - // members have already agreed on a range. If so, preserve and reserve - // THAT range rather than preemptively seizing the default configured range. - let stored_records = self - .inner - .storage - .signed_records(network) - .await - .unwrap_or_default(); - let mut state_set = crate::state::StateSet::new(); - state_set.merge_all(network, stored_records); - if let Some(agreed) = state_set.agreed_range() { - if reserve(agreed) { - return RangePlan { - propose: if Some(agreed) == self.inner.config.overlay_ipv4_range { - Some(agreed) - } else { - None - }, - ..RangePlan::default() - }; + /// Signed state always wins over a default or an explicit proposal. + async fn desired_range(&self, network: NetworkId) -> Result> { + let records = self.inner.storage.signed_records(network).await?; + let mut state = crate::state::StateSet::new(); + state.merge_all(network, records); + Ok(state.agreed_range().or_else(|| { + if self.inner.config.overlay_ipv4_auto { + Some(crate::state::derived_ipv4_range(network)) } else { - tracing::warn!( - %network, - range = %agreed, - "agreed range for this network overlaps another network already active on this interface" - ); - return RangePlan { - conflict: Some(agreed), - ..RangePlan::default() - }; + self.inner.config.overlay_ipv4_range + } + })) + } + + async fn range_conflicts( + &self, + network: NetworkId, + range: Option, + ) -> Result> { + let Some(range) = range else { + return Ok(Vec::new()); + }; + let overlapping = self.inner.routes.overlapping_networks(network, range); + let stored = self.inner.storage.list_networks().await?; + Ok(overlapping + .into_iter() + .filter_map(|(id, range)| { + let name = stored.iter().find(|n| n.network_id == id)?.name.clone(); + Some(RangeConflict { + network_id: id, + name, + range, + }) + }) + .collect()) + } + + /// Starts requested networks in saved preference order, skipping overlaps. + /// Called with the lifecycle lock held; skipped networks remain requested. + async fn resume_networks_locked(&self) -> Result<()> { + let order = self.inner.storage.network_start_order().await?; + let mut stored = self.inner.storage.list_networks().await?; + stored.sort_by_key(|n| { + order + .iter() + .position(|id| *id == n.network_id) + .unwrap_or(usize::MAX) + }); + for network in stored { + if !network.auto_start || self.is_active(network.network_id).await { + continue; + } + let keys = NetworkKeys::derive(&network.name, &network.secret); + match self.activate_with_keys_locked(keys).await { + Ok(()) | Err(Error::NetworkRangeConflict { .. }) => {} + Err(err) => return Err(err), } } - - let Some(configured) = self.inner.config.overlay_ipv4_range else { - return RangePlan::default(); - }; - - // The configured range first, so a network a device has always had - // keeps the addresses it has always had. - if reserve(configured) { - return RangePlan { - propose: Some(configured), - ..RangePlan::default() - }; - } - - // A second network on the same agent cannot have it — one agent, - // one interface — so it falls back to the range derived from its - // own id, which every one of its members derives identically - // without being told. - let derived = crate::state::derived_ipv4_range(network); - if reserve(derived) { - tracing::info!( - %network, - range = %derived, - "another network here holds the configured range; this one uses the \ - range derived from its id unless its members settled another" - ); - return RangePlan { - fallback: Some(derived), - conflict: Some(configured), - ..RangePlan::default() - }; - } - - tracing::info!(%network, "not proposing a range for this network"); - RangePlan { - conflict: Some(configured), - ..RangePlan::default() - } + Ok(()) } /// The overlay interface this agent owns, when it has one. @@ -546,6 +524,9 @@ impl Agent { /// The same `(name, secret)` always produces the same [`NetworkId`], on /// every device. /// + /// An overlapping network stays configured and requested, but inactive; + /// use [`Agent::switch_network`] to suspend the blockers explicitly. + /// /// This is declarative and therefore **idempotent**: joining a network /// that is already active succeeds and changes nothing. That matters /// because a configured network is activated automatically at startup, so @@ -599,7 +580,9 @@ impl Agent { } match self.activate_with_keys(keys).await { // Already a member of exactly this network space: nothing to do. - Ok(()) | Err(Error::NetworkAlreadyActive(_)) => Ok(network_id), + Ok(()) + | Err(Error::NetworkAlreadyActive(_)) + | Err(Error::NetworkRangeConflict { .. }) => Ok(network_id), Err(err) => Err(err), } } @@ -703,7 +686,44 @@ impl Agent { self.activate_with_keys(keys).await } + /// Switches to a configured network, suspending every overlapping network. + /// Suspended networks retain their auto-start request and resume when free. + /// Returns the networks suspended by this request. + pub async fn switch_network(&self, network_id: NetworkId) -> Result> { + let _guard = self.inner.lifecycle.lock().await; + if self.inner.shutdown.is_triggered() { + return Err(Error::Stopped); + } + let stored = self + .inner + .storage + .list_networks() + .await? + .into_iter() + .find(|n| n.network_id == network_id) + .ok_or(Error::NetworkUnknown(network_id))?; + let range = self.desired_range(network_id).await?; + let conflicts = self.range_conflicts(network_id, range).await?; + self.inner.storage.prefer_network(network_id).await?; + let mut stopped = Vec::new(); + for conflict in conflicts { + self.stop_network_locked(conflict.network_id).await?; + stopped.push(conflict.network_id); + } + if !self.is_active(network_id).await { + self.activate_with_keys_locked(NetworkKeys::derive(&stored.name, &stored.secret)) + .await?; + } + self.resume_networks_locked().await?; + Ok(stopped) + } + async fn activate_with_keys(&self, keys: NetworkKeys) -> Result<()> { + let _guard = self.inner.lifecycle.lock().await; + self.activate_with_keys_locked(keys).await + } + + async fn activate_with_keys_locked(&self, keys: NetworkKeys) -> Result<()> { if self.inner.shutdown.is_triggered() { return Err(Error::Stopped); } @@ -712,8 +732,36 @@ impl Agent { if networks.contains_key(&network_id) { return Err(Error::NetworkAlreadyActive(network_id)); } - - let range = self.reserve_range(network_id).await; + let stored = self + .inner + .storage + .list_networks() + .await? + .into_iter() + .find(|stored| stored.network_id == network_id); + let range = self.desired_range(network_id).await?; + if let Some(range) = range { + let conflicts = self.inner.routes.overlapping_networks(network_id, range); + if !conflicts.is_empty() { + return Err(Error::NetworkRangeConflict { + network: network_id, + range, + conflicts: conflicts.into_iter().map(|(id, _)| id).collect(), + }); + } + self.inner + .routes + .set_network( + network_id, + crate::overlay::NetworkRoutes { + range: Some(range), + ..Default::default() + }, + ) + .map_err(|err| Error::Rejected { + reason: err.to_string(), + })?; + } let mut backends = Vec::new(); if let Some(discovery) = &self.inner.config.discovery { backends.push(discovery.clone()); @@ -729,13 +777,7 @@ impl Agent { as Arc, ) }; - let stored = self - .inner - .storage - .list_networks() - .await? - .into_iter() - .find(|stored| stored.network_id == network_id); + let broadcast = stored.as_ref().is_none_or(|stored| stored.broadcast); let exit_offer = stored.as_ref().is_some_and(|stored| stored.exit_node); let exit_via = stored @@ -763,9 +805,8 @@ impl Agent { device_secret: self.inner.identity.signing_key(), #[cfg(feature = "testing")] unreachable_data_peers: Arc::clone(&self.inner.config.unreachable_data_peers), - ipv4_range: range.propose, - ipv4_fallback: range.fallback, - range_conflict: range.conflict, + ipv4_range: range, + range_changed: self.inner.range_changed.clone(), }); networks.insert(network_id, handle); drop(networks); @@ -786,6 +827,30 @@ impl Agent { /// it does not remove this agent from anyone else's view of the network. /// Other networks keep running. pub async fn deactivate_network(&self, network_id: NetworkId) -> Result<()> { + let _guard = self.inner.lifecycle.lock().await; + if !self.is_active(network_id).await { + return Err(Error::NetworkNotActive(network_id)); + } + self.inner.storage.set_auto_start(network_id, false).await?; + self.stop_network_locked(network_id).await?; + self.resume_networks_locked().await + } + + /// Stops a configured network, including one already suspended by overlap. + /// Clears its auto-start request so it will not resume behind the user. + pub async fn pause_network(&self, network_id: NetworkId) -> Result<()> { + let _guard = self.inner.lifecycle.lock().await; + if self.inner.shutdown.is_triggered() { + return Err(Error::Stopped); + } + self.inner.storage.set_auto_start(network_id, false).await?; + if self.is_active(network_id).await { + self.stop_network_locked(network_id).await?; + } + self.resume_networks_locked().await + } + + async fn stop_network_locked(&self, network_id: NetworkId) -> Result<()> { let handle = { let mut networks = self.inner.networks.write().await; networks @@ -814,7 +879,6 @@ impl Agent { interface.sync_broadcast_rules().await; interface.sync_exit_rules().await; } - self.inner.storage.set_auto_start(network_id, false).await?; Ok(()) } @@ -824,17 +888,15 @@ impl Agent { /// Local and silent: nobody is told. [`Agent::leave_network`] is the one /// that says goodbye first, and is what a person means by leaving. pub async fn forget_network(&self, network_id: NetworkId) -> Result<()> { + let _guard = self.inner.lifecycle.lock().await; if self.is_active(network_id).await { - self.deactivate_network(network_id).await?; + self.stop_network_locked(network_id).await?; } self.inner.storage.remove_network(network_id).await?; - // Whatever a protocol kept for this network goes with it. The - // agent cannot know what that is, only that there is no longer any - // reason to hold it. for plugin in &self.inner.config.plugins { plugin.on_network_forgotten(network_id); } - Ok(()) + self.resume_networks_locked().await } /// Leaves a network: gives up what was claimed, then forgets it. @@ -958,13 +1020,17 @@ impl Agent { let mut networks = Vec::new(); for stored in self.inner.storage.list_networks().await? { if active.contains(&stored.network_id) - && let Ok(status) = self.network_status(stored.network_id).await + && let Ok(mut status) = self.network_status(stored.network_id).await { + status.auto_start = stored.auto_start; networks.push(status); continue; } let keys = NetworkKeys::derive(&stored.name, &stored.secret); + let range = self.desired_range(stored.network_id).await?; + let range_conflicts = self.range_conflicts(stored.network_id, range).await?; networks.push(NetworkStatus { + auto_start: stored.auto_start, broadcast: stored.broadcast, exit_offer: stored.exit_node, exit_via: stored @@ -978,12 +1044,13 @@ impl Agent { state: NetworkState::Inactive, peers: Vec::new(), candidates: Vec::new(), - // An inactive network has no runtime to ask; the roster and - // the range come from one. Empty, not invented. + // The range is available from signed state or the deterministic + // default even while the network has no runtime. members: Vec::new(), hostname_conflict: None, - range: None, - range_conflict: None, + range, + range_conflict: range_conflicts.first().map(|conflict| conflict.range), + range_conflicts, metrics: NetworkMetrics::default(), relay: Default::default(), }); @@ -1039,6 +1106,7 @@ impl Agent { /// After this returns, the state directory can be opened by another agent. pub async fn shutdown(&self) { self.inner.shutdown.trigger(); + let lifecycle_guard = self.inner.lifecycle.lock().await; let handles: Vec = { let mut networks = self.inner.networks.write().await; @@ -1048,6 +1116,7 @@ impl Agent { handle.stop().await; } + drop(lifecycle_guard); if let Some(dht) = &self.inner.config.dht { dht.shutdown().await; } @@ -1064,6 +1133,7 @@ impl Agent { for (what, handle) in [ ("inbound connections", &self.inner.accept_task), ("plugin requests", &self.inner.plugin_task), + ("range conflicts", &self.inner.range_task), ] { let task = handle.lock().ok().and_then(|mut guard| guard.take()); if let Some(task) = task { @@ -1107,16 +1177,44 @@ impl Agent { } } -/// What a network may propose as its overlay range, and what it cannot have. -#[derive(Debug, Clone, Copy, Default)] -struct RangePlan { - /// Proposed straight away: the configured range, when it is free here. - propose: Option, - /// Proposed after a moment, when the configured range is another - /// network's: the range derived from this network's own id. - fallback: Option, - /// The configured range, when this network cannot have it. - conflict: Option, +/// Processes coalesced conflict notifications without blocking a network actor. +async fn range_conflict_loop(inner: Weak, mut updates: watch::Receiver<()>) { + loop { + let Some(owner) = inner.upgrade() else { + return; + }; + let shutdown = owner.shutdown.clone(); + drop(owner); + tokio::select! { + _ = shutdown.wait() => return, + result = updates.changed() => if result.is_err() { return; }, + } + let Some(inner) = inner.upgrade() else { + return; + }; + let agent = Agent { inner }; + let _guard = agent.inner.lifecycle.lock().await; + if agent.inner.shutdown.is_triggered() { + return; + } + let blocked: Vec<_> = agent + .inner + .networks + .read() + .await + .iter() + .filter(|(_, h)| h.range_blocked.load(std::sync::atomic::Ordering::Acquire)) + .map(|(id, _)| *id) + .collect(); + for id in blocked { + if let Err(err) = agent.stop_network_locked(id).await { + tracing::warn!(%err, "cannot suspend overlapping network"); + } + } + if let Err(err) = agent.resume_networks_locked().await { + tracing::warn!(%err, "cannot resume suspended networks"); + } + } } /// How long a release is given to reach the sessions it was queued on. diff --git a/crates/tsunagi-core/src/agent/network.rs b/crates/tsunagi-core/src/agent/network.rs index 7d0f94a..9440d80 100644 --- a/crates/tsunagi-core/src/agent/network.rs +++ b/crates/tsunagi-core/src/agent/network.rs @@ -137,6 +137,7 @@ pub(crate) struct NetworkHandle { pub(crate) commands: mpsc::Sender, shutdown: Shutdown, task: JoinHandle<()>, + pub(crate) range_blocked: Arc, } impl NetworkHandle { @@ -177,17 +178,8 @@ pub(crate) struct RuntimeParams { /// The IPv4 overlay range this agent would use, if the network has not /// already settled on another one. pub(crate) ipv4_range: Option, - /// The range it was configured with but cannot have, because another - /// network on this agent holds it. - pub(crate) range_conflict: Option, - /// The range derived from this network's own id, proposed when the - /// configured one belongs to another network here. - /// - /// Held back at first: a network that already exists has a range of its - /// own, and a member that proposed before hearing anything would be - /// arguing with it instead of adopting it. Every member derives the - /// same one, so once the wait is over there is nothing to argue about. - pub(crate) ipv4_fallback: Option, + /// Coalesced notifications to the agent's lifecycle manager. + pub(crate) range_changed: tokio::sync::watch::Sender<()>, /// How data plane links are opened. `None` disables the data plane. pub(crate) transport: Option>, } @@ -236,8 +228,10 @@ pub(crate) fn spawn(params: RuntimeParams) -> NetworkHandle { let (commands_tx, commands_rx) = mpsc::channel(64); let runtime_shutdown = shutdown.clone(); + let range_blocked = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let runtime_blocked = Arc::clone(&range_blocked); let task = tokio::spawn(async move { - let mut runtime = Runtime::new(params, runtime_shutdown); + let mut runtime = Runtime::new(params, runtime_shutdown, runtime_blocked); runtime.run(commands_rx).await; }); @@ -246,6 +240,7 @@ pub(crate) fn spawn(params: RuntimeParams) -> NetworkHandle { commands: commands_tx, shutdown, task, + range_blocked, } } @@ -257,14 +252,6 @@ pub(crate) fn spawn(params: RuntimeParams) -> NetworkHandle { /// itself, a few announcements later. const REACH_EXPIRY: Duration = Duration::from_secs(90); -/// How long a network waits to be told its range before proposing the one -/// derived from its id. -/// -/// Only for that fallback: the configured range is proposed at once, as it -/// always was. This is the window in which an existing network's records -/// can arrive and be adopted instead. -const RANGE_PROPOSAL_GRACE: Duration = Duration::from_secs(3); - #[derive(Debug, Clone)] struct ProtocolDialState { failures: u32, @@ -287,8 +274,9 @@ struct Runtime { discovery_worker: Option, discovery_candidates: mpsc::Receiver, params: RuntimeParams, - /// When this runtime started, for the fallback range's grace period. - activated: std::time::Instant, + range_blocked: Arc, + /// Last range successfully published to the agent's routing table. + published_range: Option, network_id: NetworkId, local_id: EndpointId, shutdown: Shutdown, @@ -366,7 +354,11 @@ struct Runtime { } impl Runtime { - fn new(params: RuntimeParams, shutdown: Shutdown) -> Self { + fn new( + params: RuntimeParams, + shutdown: Shutdown, + range_blocked: Arc, + ) -> Self { let network_id = params.keys.network_id(); let local_id = params.adapter.endpoint_id(); let (session_events_tx, session_events_rx) = mpsc::channel(256); @@ -398,7 +390,8 @@ impl Runtime { discovery_worker, discovery_candidates, params, - activated: std::time::Instant::now(), + range_blocked, + published_range: None, network_id, local_id, shutdown, @@ -968,35 +961,27 @@ impl Runtime { if let Some(agreed) = self.state.agreed_range() { return Some(agreed); } - // Nobody has settled one yet, so this agent would be proposing its - // own. It must not propose a range it cannot route: one agent has - // one interface, and claiming an address in a range another network - // already owns would spread the collision rather than contain it — - // "the lowest author's range wins" would carry it to everybody. - if let Some(wanted) = self.params.ipv4_range + self.params.ipv4_range + } + + /// A late signed range can conflict even though activation was safe. + /// Fail closed before touching claims, plugins or the host interface. + fn suspend_if_overlapping(&self) -> bool { + if let Some(range) = self.effective_range() && self .params .routes - .would_overlap(self.network_id, wanted) - .is_none() + .would_overlap(self.network_id, range) + .is_some() { - return Some(wanted); + self.range_blocked + .store(true, std::sync::atomic::Ordering::Release); + self.params.routes.remove_network(self.network_id); + self.shutdown.trigger(); + self.params.range_changed.send_replace(()); + return true; } - - // The configured range is another network's here, so this one falls - // back to the range derived from its id — after a moment's wait, so - // that a network which already exists gets to say what it uses - // first. Every member derives the same one, so members that reach - // this point agree without negotiating. - let fallback = self.params.ipv4_fallback?; - if self.activated.elapsed() < RANGE_PROPOSAL_GRACE { - return None; - } - self.params - .routes - .would_overlap(self.network_id, fallback) - .is_none() - .then_some(fallback) + false } /// Makes sure this agent holds an address, claiming one if it does not. @@ -1006,7 +991,7 @@ impl Runtime { async fn ensure_own_claim(&mut self) { // On the way out. Claiming again here is how a goodbye becomes a // hello nobody asked for. - if self.released { + if self.released || self.suspend_if_overlapping() { return; } let wanted_hostname = { @@ -1296,6 +1281,9 @@ impl Runtime { /// Tells the plugins who holds which overlay address. async fn publish_allocations(&mut self) { + if self.shutdown.is_triggered() || self.suspend_if_overlapping() { + return; + } let Some(range) = self.effective_range() else { return; }; @@ -1307,10 +1295,6 @@ impl Runtime { .collect(); allocations.sort_by_key(|(holder, _)| *holder.as_bytes()); - for plugin in &self.params.plugins { - plugin.on_address_allocation(self.network_id, range, &allocations); - } - // And the system level's own view, which is what decides whose // packet is whose. The local address is kept out of the peer list: // a packet for ourselves does not go over a tunnel. @@ -1326,17 +1310,27 @@ impl Runtime { .map(|(holder, address)| (*address, *holder)) .collect(), }; - if let Err(err) = self.params.routes.set_network(self.network_id, routes) { - // Reported once per change rather than swallowed: two networks - // wanting the same addresses is a thing the user has to settle. - self.metrics.plugin_errors += 1; - self.emit(Event::PluginError { - network: self.network_id, - protocol: "overlay".into(), - reason: err.to_string(), - }); + if self + .params + .routes + .set_network(self.network_id, routes) + .is_err() + { + self.range_blocked + .store(true, std::sync::atomic::Ordering::Release); + self.params.routes.remove_network(self.network_id); + self.shutdown.trigger(); + self.params.range_changed.send_replace(()); return; } + for plugin in &self.params.plugins { + plugin.on_address_allocation(self.network_id, range, &allocations); + } + if self.published_range != Some(range) { + self.published_range = Some(range); + self.params.range_changed.send_replace(()); + } + let Some(interface) = self.params.interface.clone() else { return; }; @@ -2387,6 +2381,7 @@ impl Runtime { members.dedup_by(|a, b| a.endpoint_id == b.endpoint_id); NetworkStatus { + auto_start: true, broadcast: self.params.broadcast, exit_offer: self.params.exit_offer, exit_via: self.params.exit_via, @@ -2409,13 +2404,8 @@ impl Runtime { members, hostname_conflict: self.hostname_conflict_of(&self.local_id), range: self.effective_range(), - // Only worth reporting while it is actually stuck: with a - // fallback in play the configured range being another - // network's is how it is meant to work, not a fault. - range_conflict: self - .params - .range_conflict - .filter(|_| self.effective_range().is_none()), + range_conflict: None, + range_conflicts: Vec::new(), metrics: self.metrics.clone(), relay: self.hub.counters(), } diff --git a/crates/tsunagi-core/src/agent/status.rs b/crates/tsunagi-core/src/agent/status.rs index 570a468..9e39688 100644 --- a/crates/tsunagi-core/src/agent/status.rs +++ b/crates/tsunagi-core/src/agent/status.rs @@ -209,9 +209,22 @@ pub struct NetworkMetrics { pub data_link_failures: u64, } +/// An active network preventing another network from starting locally. +#[derive(Debug, Clone)] +pub struct RangeConflict { + /// The blocking network's public identifier. + pub network_id: NetworkId, + /// Its configured name. + pub name: NetworkName, + /// Its overlapping overlay range. + pub range: crate::state::Ipv4Range, +} + /// Status of one network. #[derive(Debug, Clone)] pub struct NetworkStatus { + /// Whether this network is requested to start or resume automatically. + pub auto_start: bool, /// Local broadcast participation in this network. pub broadcast: bool, /// This agent offers itself as an exit node in this network. @@ -242,13 +255,10 @@ pub struct NetworkStatus { pub hostname_conflict: Option, /// The overlay range this network uses, once it has one. pub range: Option, - /// The range it could not have, because another network on this agent - /// already holds it. - /// - /// One agent has one interface, so an address belongs to one network. - /// This network waits to adopt whatever its members settle on instead of - /// proposing something it could not route. + /// The first overlapping active range; `range_conflicts` identifies all blockers. pub range_conflict: Option, + /// Active networks that would be suspended by switching to this network. + pub range_conflicts: Vec, /// Per-network counters. pub metrics: NetworkMetrics, /// What has gone through a peer in the middle, in both directions. diff --git a/crates/tsunagi-core/src/config.rs b/crates/tsunagi-core/src/config.rs index 5193f87..9e3149b 100644 --- a/crates/tsunagi-core/src/config.rs +++ b/crates/tsunagi-core/src/config.rs @@ -339,7 +339,11 @@ pub struct AgentConfig { pub interface_name: String, /// The interface MTU. pub interface_mtu: u32, - /// The IPv4 overlay range this agent proposes. + /// Whether new networks derive their IPv4 range from their network id. + /// + /// Enabled by default. Explicit ranges and IPv4 opt-out disable this. + pub overlay_ipv4_auto: bool, + /// The IPv4 overlay range this agent proposes when automatic selection is off. /// /// Addresses are allocated from it and recorded in signed state, so a /// participant keeps the same one across restarts. A network that has @@ -373,7 +377,8 @@ impl AgentConfig { tun_factory: None, interface_name: "tsun0".to_string(), interface_mtu: 1280, - overlay_ipv4_range: Some(crate::state::DEFAULT_IPV4_RANGE), + overlay_ipv4_auto: true, + overlay_ipv4_range: None, } } @@ -445,6 +450,7 @@ impl AgentConfig { /// Sets the IPv4 overlay range this agent proposes, or disables IPv4. pub fn with_overlay_ipv4_range(mut self, range: Option) -> Self { + self.overlay_ipv4_auto = false; self.overlay_ipv4_range = range; self } diff --git a/crates/tsunagi-core/src/error.rs b/crates/tsunagi-core/src/error.rs index d6c3bbf..66f79df 100644 --- a/crates/tsunagi-core/src/error.rs +++ b/crates/tsunagi-core/src/error.rs @@ -97,6 +97,19 @@ pub enum Error { #[error("network {0} is not configured")] NetworkUnknown(NetworkId), + /// An overlay range overlaps networks already active on this agent. + #[error( + "network {network} cannot start: {range} overlaps active networks {conflicts:?}; switch to it to suspend those networks" + )] + NetworkRangeConflict { + /// Network that could not start. + network: NetworkId, + /// Its agreed or proposed range. + range: crate::state::Ipv4Range, + /// Active networks that must be suspended first. + conflicts: Vec, + }, + /// No session with that peer exists in the given network. #[error("no authenticated session with peer {peer} in network {network}")] NoSuchPeer { diff --git a/crates/tsunagi-core/src/ipc/mod.rs b/crates/tsunagi-core/src/ipc/mod.rs index 47b5ec2..8b2b574 100644 --- a/crates/tsunagi-core/src/ipc/mod.rs +++ b/crates/tsunagi-core/src/ipc/mod.rs @@ -324,6 +324,9 @@ pub struct ActiveReport { /// Stopping something already stopped is not an error — it is the state /// asked for — but it is worth saying which happened. pub changed: bool, + /// Networks suspended by switching to this network. + #[serde(default)] + pub suspended: Vec, } /// What happened when a network was joined. @@ -345,6 +348,13 @@ pub struct JoinedReport { /// Almost always a mistyped secret, and the one thing that makes two /// sections of a report look like one network. pub name_shared_with: Option, + /// Whether it started; false means it is configured but blocked by overlap. + #[serde(default = "joined_active_default")] + pub active: bool, +} + +fn joined_active_default() -> bool { + true } /// What happened when a network was left. @@ -437,9 +447,23 @@ pub struct DnsReport { pub names: u32, } +/// An active network with an overlapping overlay range. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct RangeConflictReport { + /// Public network id, for unambiguous selection. + pub network_id: String, + /// Configured network name. + pub name: String, + /// The overlapping range. + pub range: String, +} + /// One network. #[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] pub struct NetworkReport { + /// Whether the network starts or resumes automatically when its range is free. + #[serde(default)] + pub auto_start: bool, /// Whether this agent participates in LAN broadcasts in this network. #[serde(default)] pub broadcast: bool, @@ -482,6 +506,9 @@ pub struct NetworkReport { /// The range it could not have, because another network on this agent /// already holds it. pub range_conflict: Option, + /// Networks that switching to this network would suspend. + #[serde(default)] + pub range_conflicts: Vec, /// The overlay, when a protocol is running one. pub overlay: Option, /// This agent's own name, when another member claimed it first. @@ -822,7 +849,7 @@ pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5); /// /// Bump it whenever [`Request`], [`Response`] or anything they contain /// changes shape. -pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 18]); +pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 19]); /// Reads one request off an accepted stream, answers it, writes the response. /// diff --git a/crates/tsunagi-core/src/overlay/router.rs b/crates/tsunagi-core/src/overlay/router.rs index 938d3bb..a894c5d 100644 --- a/crates/tsunagi-core/src/overlay/router.rs +++ b/crates/tsunagi-core/src/overlay/router.rs @@ -72,7 +72,7 @@ pub enum RouteError { #[error( "network {network} uses {range}, which overlaps {other_range} already in use by \ {other}. One agent has one interface, so an address can belong to only one of \ - them: give one network a different range." + them: stop the other network before starting this one." )] Overlap { /// The network being added. @@ -365,6 +365,25 @@ impl RoutingTable { }) } + /// All active networks whose reserved ranges intersect this range. + pub fn overlapping_networks( + &self, + network: NetworkId, + range: Ipv4Range, + ) -> Vec<(NetworkId, Ipv4Range)> { + let mut conflicts: Vec<_> = self + .read() + .iter() + .filter_map(|(other, routes)| { + let other_range = routes.range?; + (*other != network && ranges_overlap(range, other_range)) + .then_some((*other, other_range)) + }) + .collect(); + conflicts.sort_by_key(|(id, _)| *id); + conflicts + } + /// How many networks the table covers. pub fn len(&self) -> usize { self.read().len() @@ -510,7 +529,7 @@ mod tests { let err = table.set_network(second, routes()).unwrap_err(); let RouteError::Overlap { other, .. } = err; assert_eq!(other, first); - assert!(err.to_string().contains("different range"), "{err}"); + assert!(err.to_string().contains("stop the other network"), "{err}"); assert_eq!(table.len(), 1, "the overlapping network is not added"); // A range of its own is fine. diff --git a/crates/tsunagi-core/src/state/mod.rs b/crates/tsunagi-core/src/state/mod.rs index 7d843dc..6bcdcdb 100644 --- a/crates/tsunagi-core/src/state/mod.rs +++ b/crates/tsunagi-core/src/state/mod.rs @@ -109,12 +109,8 @@ impl std::str::FromStr for Ipv4Range { } } -/// The IPv4 overlay range used unless something else is configured or agreed. -/// -/// A small, specific `/24`: memorable, and far less likely to overlap a -/// network the machine is already on than taking a whole `/8` or `/10` would -/// be. Because addresses are allocated rather than derived, 254 of them is -/// plenty for the size of network this is for. +/// The legacy default range, retained for existing networks and explicit use. +/// New networks use [`derived_ipv4_range`] by default. pub const DEFAULT_IPV4_RANGE: Ipv4Range = Ipv4Range { base: Ipv4Addr::new(10, 13, 37, 0), prefix_len: 24, @@ -122,15 +118,10 @@ pub const DEFAULT_IPV4_RANGE: Ipv4Range = Ipv4Range { /// A range derived from a network's own identity. /// -/// One agent has one interface, so two of its networks cannot both use the -/// configured range. The second needs another one — and it cannot be picked -/// locally, because every member has to arrive at the same answer without -/// being told. So it comes from the network id, which every member already -/// has and nobody can choose: the same network derives the same range on -/// every device. -/// -/// Inside 10/8 like the default, and never equal to it, so the first -/// network keeps what it has always had. +/// Every member independently derives the same pseudorandom `/24` in 10/8, +/// including isolated first members. Stored signed ranges take precedence. +/// The derivation is kept stable for networks already using it; the legacy +/// default is excluded. Distinct network ids can still produce collisions. pub fn derived_ipv4_range(network: NetworkId) -> Ipv4Range { let bytes = network.as_bytes(); let second = bytes[0]; @@ -673,15 +664,20 @@ impl StateSet { /// The range the network settled on, if anybody has said. /// - /// When claims disagree, the one from the lowest author id wins, so every - /// replica reaches the same answer. A participant joining later therefore - /// adopts the range already in use instead of imposing its own. + /// A legacy or explicit range takes precedence over the derived default, + /// so a fresh member's default cannot renumber an established network. + /// Other disagreements use the lowest author id, independently of arrival + /// order. The signed record format and network identity remain unchanged. pub fn agreed_range(&self) -> Option { - let mut authors: Vec<&EndpointId> = self.records.keys().collect(); - authors.sort_by_key(|author| *author.as_bytes()); - authors - .into_iter() - .find_map(|author| self.records.get(author)?.body.range()) + self.records + .iter() + .filter_map(|(author, record)| { + let range = record.body.range()?; + let derived = derived_ipv4_range(NetworkId::from_bytes(record.network)); + Some(((range == derived, *author.as_bytes()), range)) + }) + .min_by_key(|(priority, _)| *priority) + .map(|(_, range)| range) } /// The highest version this author has published, as far as is known. @@ -1263,7 +1259,7 @@ mod tests { assert_eq!(once.base.octets()[0], 10, "inside 10/8 like the default"); assert_ne!( once, DEFAULT_IPV4_RANGE, - "the default belongs to whichever network asked first" + "the legacy default remains excluded" ); // A different network derives a different range, which is the @@ -1275,4 +1271,34 @@ mod tests { .network_id(); assert_ne!(once, derived_ipv4_range(other)); } + + #[test] + fn a_derived_default_cannot_displace_a_legacy_or_explicit_range() { + let id = network("range-migration"); + let a = SecretKey::generate(); + let b = SecretKey::generate(); + let (low, high) = if a.public().as_bytes() < b.public().as_bytes() { + (a, b) + } else { + (b, a) + }; + for established in [DEFAULT_IPV4_RANGE, "10.90.0.0/16".parse().unwrap()] { + let body = |range: Ipv4Range| RecordBody::Claim { + address: Some(Ipv4Addr::from(u32::from(range.base) + 1)), + range: Some(range), + hostname: None, + hostname_since: None, + }; + let fresh = SignedRecord::sign(&low, id, 1, body(derived_ipv4_range(id))); + let existing = SignedRecord::sign(&high, id, 1, body(established)); + for records in [ + vec![fresh.clone(), existing.clone()], + vec![existing.clone(), fresh.clone()], + ] { + let mut state = StateSet::new(); + state.merge_all(id, records); + assert_eq!(state.agreed_range(), Some(established)); + } + } + } } diff --git a/crates/tsunagi-core/src/storage/mod.rs b/crates/tsunagi-core/src/storage/mod.rs index 806c90d..8d402c9 100644 --- a/crates/tsunagi-core/src/storage/mod.rs +++ b/crates/tsunagi-core/src/storage/mod.rs @@ -272,6 +272,33 @@ impl Storage { .await } + /// Configured networks in the user's most recently selected order. + pub async fn network_start_order(&self) -> Result> { + self.with_state(|state| { + let configured = state.list_networks()?; + let text = state + .get_setting("network_start_order")? + .unwrap_or_default(); + let mut order = Vec::new(); + for word in text.split(',') { + if let Ok(id) = word.parse::() + && configured.iter().any(|n| n.network_id == id) + && !order.contains(&id) + { + order.push(id); + } + } + Ok(order) + }) + .await + } + + /// Prefers this network at startup without disabling suspended networks. + pub async fn prefer_network(&self, network: NetworkId) -> Result<()> { + self.with_state(move |state| state.prefer_network(network)) + .await + } + /// Removes a network configuration and its cached hints. pub async fn remove_network(&self, network_id: NetworkId) -> Result<()> { self.with_state(move |state| { diff --git a/crates/tsunagi-core/src/storage/state.rs b/crates/tsunagi-core/src/storage/state.rs index fcf8109..8f0104a 100644 --- a/crates/tsunagi-core/src/storage/state.rs +++ b/crates/tsunagi-core/src/storage/state.rs @@ -448,12 +448,16 @@ impl StateStore { /// Sets the auto-start flag of a configured network. pub fn set_auto_start(&self, network_id: NetworkId, auto_start: bool) -> Result<()> { - self.conn + let changed = self + .conn .execute( "UPDATE networks SET auto_start = ?2 WHERE network_id = ?1", params![network_id.as_bytes().as_slice(), auto_start as i64], ) .map_err(|err| Error::Storage(format!("cannot update network: {err}")))?; + if changed == 0 { + return Err(Error::NetworkUnknown(network_id)); + } Ok(()) } @@ -798,6 +802,44 @@ impl StateStore { Ok(()) } + /// Atomically enables a network and puts it first in the startup order. + pub fn prefer_network(&self, network: NetworkId) -> Result<()> { + let configured = self.list_networks()?; + if !configured.iter().any(|n| n.network_id == network) { + return Err(Error::NetworkUnknown(network)); + } + let existing = self.get_setting("network_start_order")?.unwrap_or_default(); + let mut order = vec![network]; + for word in existing.split(',') { + if let Ok(id) = word.parse::() + && configured.iter().any(|n| n.network_id == id) + && !order.contains(&id) + { + order.push(id); + } + } + let value = order + .iter() + .map(ToString::to_string) + .collect::>() + .join(","); + let transaction = self + .conn + .unchecked_transaction() + .map_err(|err| Error::Storage(format!("cannot begin network preference: {err}")))?; + transaction + .execute( + "UPDATE networks SET auto_start = 1 WHERE network_id = ?1", + params![network.as_bytes().as_slice()], + ) + .map_err(|err| Error::Storage(format!("cannot enable network: {err}")))?; + transaction.execute("INSERT INTO settings (key, value) VALUES ('network_start_order', ?1) ON CONFLICT(key) DO UPDATE SET value = excluded.value", params![value]) + .map_err(|err| Error::Storage(format!("cannot store network preference: {err}")))?; + transaction + .commit() + .map_err(|err| Error::Storage(format!("cannot commit network preference: {err}"))) + } + /// Reads an arbitrary setting. pub fn get_setting(&self, key: &str) -> Result> { self.conn diff --git a/crates/tsunagi-core/tests/leaving.rs b/crates/tsunagi-core/tests/leaving.rs index 563fca4..b523c81 100644 --- a/crates/tsunagi-core/tests/leaving.rs +++ b/crates/tsunagi-core/tests/leaving.rs @@ -162,7 +162,7 @@ async fn rejoining_after_leaving_is_not_taken_for_a_stale_record() { }) .await; assert!( - tsunagi::state::DEFAULT_IPV4_RANGE.contains(again), + tsunagi::state::derived_ipv4_range(network_id).contains(again), "an address in the network's range: {again}" ); diff --git a/crates/tsunagi-core/tests/network_isolation.rs b/crates/tsunagi-core/tests/network_isolation.rs index 37e87fc..af06038 100644 --- a/crates/tsunagi-core/tests/network_isolation.rs +++ b/crates/tsunagi-core/tests/network_isolation.rs @@ -265,10 +265,7 @@ async fn deactivating_one_network_leaves_the_agent_and_others_running() { #[tokio::test] async fn two_networks_on_one_agent_each_get_a_range_of_their_own() { - // One agent, one interface: the configured range can only belong to one - // of them. The second does not go without — it takes the range derived - // from its own id, which every one of its members derives identically, - // so it is an agreement and not a local invention. + // Each network derives its range independently of activation order. let discovery = SharedMemoryDiscovery::new(); let (first, first_secret) = network("two-ranges-one"); let (second, second_secret) = network("two-ranges-two"); @@ -309,8 +306,8 @@ async fn two_networks_on_one_agent_each_get_a_range_of_their_own() { "different networks, different addresses" ); assert!( - tsunagi::state::DEFAULT_IPV4_RANGE.contains(addresses.0), - "the first keeps the configured range: {}", + tsunagi::state::derived_ipv4_range(one).contains(addresses.0), + "the first uses its derived range: {}", addresses.0 ); assert!( @@ -319,7 +316,7 @@ async fn two_networks_on_one_agent_each_get_a_range_of_their_own() { addresses.1 ); assert!( - !tsunagi::state::DEFAULT_IPV4_RANGE.contains(addresses.1), + !tsunagi::state::derived_ipv4_range(one).contains(addresses.1), "and not in the one the first network holds" ); diff --git a/crates/tsunagi-core/tests/range_conflicts.rs b/crates/tsunagi-core/tests/range_conflicts.rs new file mode 100644 index 0000000..9c01edb --- /dev/null +++ b/crates/tsunagi-core/tests/range_conflicts.rs @@ -0,0 +1,336 @@ +//! Derived defaults and local exclusion of overlapping overlay networks. +#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)] + +use tsunagi::Agent; +use tsunagi::discovery::SharedMemoryDiscovery; +use tsunagi::state::{DEFAULT_IPV4_RANGE, Ipv4Range, derived_ipv4_range}; +use tsunagi::testing::{TestAgent, config_with, network, wait_until}; + +// These tests need discovery to happen before a collision, so keep the initial +// default outside the established ranges. Derived collisions have their own test. +fn network_outside( + name: &str, + ranges: &[Ipv4Range], +) -> ( + tsunagi::identity::NetworkName, + tsunagi::identity::NetworkSecret, +) { + use tsunagi::identity::{NetworkKeys, NetworkName, NetworkSecret}; + let name = NetworkName::new(name).unwrap(); + let secret = (0..=255u8) + .map(|seed| NetworkSecret::from_bytes([seed; 32]).unwrap()) + .find(|secret| { + let derived = derived_ipv4_range(NetworkKeys::derive(&name, secret).network_id()); + ranges.iter().all(|range| !range.contains(derived.base)) + }) + .expect("a deterministic test secret with a non-overlapping default"); + (name, secret) +} + +#[tokio::test] +async fn colliding_derived_defaults_are_blocked_instead_of_locally_renumbered() { + use std::collections::HashMap; + use tsunagi::identity::{NetworkKeys, NetworkName, NetworkSecret}; + let secret = NetworkSecret::from_bytes([19; 32]).unwrap(); + let mut seen = HashMap::new(); + let (first, second, range) = (0..=65_536) + .find_map(|index| { + let name = NetworkName::new(format!("derived-collision-{index}")).unwrap(); + let range = derived_ipv4_range(NetworkKeys::derive(&name, &secret).network_id()); + seen.insert(range.base, name.clone()) + .map(|first| (first, name, range)) + }) + .expect("more network names than possible /24 ranges"); + let agent = TestAgent::spawn(&SharedMemoryDiscovery::new()) + .await + .unwrap(); + let first = agent.agent.join_network(&first, &secret).await.unwrap(); + let second = agent.agent.join_network(&second, &secret).await.unwrap(); + assert!(agent.agent.is_active(first).await); + assert!(!agent.agent.is_active(second).await); + let status = agent.agent.status().await.unwrap(); + let blocked = status.network(&second).unwrap(); + assert_eq!(blocked.range, Some(range)); + assert_eq!(blocked.range_conflicts[0].network_id, first); + assert_eq!( + agent.agent.switch_network(second).await.unwrap(), + vec![first] + ); + assert_eq!( + agent.agent.network_status(second).await.unwrap().range, + Some(range) + ); + agent.agent.shutdown().await; +} + +#[tokio::test] +async fn isolated_first_members_choose_the_same_range_and_keep_it_after_restart() { + let discovery = SharedMemoryDiscovery::new(); + let (name, secret) = network("derived-first"); + let a = TestAgent::spawn(&discovery).await.unwrap(); + let b = TestAgent::spawn(&SharedMemoryDiscovery::new()) + .await + .unwrap(); + let id = a.agent.join_network(&name, &secret).await.unwrap(); + b.agent.join_network(&name, &secret).await.unwrap(); + let expected = derived_ipv4_range(id); + for agent in [&a.agent, &b.agent] { + wait_until("isolated member allocates in its derived range", || async { + let status = agent.network_status(id).await.ok()?; + (status.range == Some(expected) && !status.members.is_empty()).then_some(()) + }) + .await; + } + let dir = a.stop().await; + let a = Agent::spawn(config_with(dir.path(), &discovery)) + .await + .unwrap(); + assert_eq!(a.network_status(id).await.unwrap().range, Some(expected)); + a.shutdown().await; + b.agent.shutdown().await; +} + +#[tokio::test] +async fn overlap_blocks_activation_switches_and_resumes_across_restart() { + let discovery = SharedMemoryDiscovery::new(); + let a = TestAgent::spawn_with( + |c| c.with_overlay_ipv4_range(Some(DEFAULT_IPV4_RANGE)), + &discovery, + ) + .await + .unwrap(); + let (one_name, one_secret) = network("conflict-one"); + let (two_name, two_secret) = network("conflict-two"); + let one = a.agent.join_network(&one_name, &one_secret).await.unwrap(); + a.agent.network_status(one).await.unwrap(); + let two = a.agent.join_network(&two_name, &two_secret).await.unwrap(); + assert!(a.agent.is_active(one).await); + assert!(!a.agent.is_active(two).await); + assert!(matches!( + a.agent.activate_network(two).await, + Err(tsunagi::Error::NetworkRangeConflict { .. }) + )); + let status = a.agent.status().await.unwrap(); + let blocked = status.network(&two).unwrap(); + assert_eq!(blocked.range, Some(DEFAULT_IPV4_RANGE)); + assert_eq!(blocked.range_conflicts[0].network_id, one); + assert_eq!(blocked.range_conflicts[0].name, one_name); + assert_eq!(a.agent.switch_network(two).await.unwrap(), vec![one]); + assert!(a.agent.is_active(two).await); + assert!(!a.agent.is_active(one).await); + + a.agent.network_status(two).await.unwrap(); + let dir = a.stop().await; + let a = Agent::spawn(config_with(dir.path(), &discovery)) + .await + .unwrap(); + assert!(a.is_active(two).await, "the user's choice survives restart"); + assert!(!a.is_active(one).await); + a.deactivate_network(two).await.unwrap(); + assert!( + a.is_active(one).await, + "suspended networks resume when the blocker stops" + ); + assert_eq!( + a.network_status(one).await.unwrap().range, + Some(DEFAULT_IPV4_RANGE), + "legacy ranges survive the new default" + ); + a.shutdown().await; +} + +#[tokio::test] +async fn late_authenticated_state_with_a_nested_range_suspends_only_that_network() { + let discovery = SharedMemoryDiscovery::new(); + let broad: Ipv4Range = "10.90.0.0/16".parse().unwrap(); + let narrow: Ipv4Range = "10.90.12.0/24".parse().unwrap(); + let (name, secret) = network_outside("late-conflict", &[broad]); + let remote = TestAgent::spawn_with(|c| c.with_overlay_ipv4_range(Some(narrow)), &discovery) + .await + .unwrap(); + let target = remote.agent.join_network(&name, &secret).await.unwrap(); + wait_until("remote range is persisted", || async { + remote + .agent + .network_status(target) + .await + .ok()? + .members + .first() + .map(|_| ()) + }) + .await; + + let (other_name, other_secret) = network("existing-broad"); + // Seed an existing network using a real agent and its signed claim. + let seeded = TestAgent::spawn_with( + |c| c.with_overlay_ipv4_range(Some(broad)), + &SharedMemoryDiscovery::new(), + ) + .await + .unwrap(); + let other = seeded + .agent + .join_network(&other_name, &other_secret) + .await + .unwrap(); + wait_until("broad range is persisted", || async { + seeded + .agent + .network_status(other) + .await + .ok()? + .members + .first() + .map(|_| ()) + }) + .await; + let dir = seeded.stop().await; + let seeded = Agent::spawn(config_with(dir.path(), &discovery)) + .await + .unwrap(); + seeded.join_network(&name, &secret).await.unwrap(); + wait_until("late overlap suspends the joining network", || async { + let status = seeded.status().await.ok()?; + let blocked = status.network(&target)?; + (!seeded.is_active(target).await + && blocked + .range_conflicts + .iter() + .any(|c| c.network_id == other)) + .then_some(()) + }) + .await; + assert!(seeded.is_active(other).await); + assert_eq!(seeded.switch_network(target).await.unwrap(), vec![other]); + assert_eq!( + seeded.network_status(target).await.unwrap().range, + Some(narrow) + ); + seeded.shutdown().await; + remote.agent.shutdown().await; +} + +#[tokio::test] +async fn a_wide_range_suspends_all_overlaps_and_manual_stop_cancels_resume() { + let discovery = SharedMemoryDiscovery::new(); + let hub = TestAgent::spawn(&discovery).await.unwrap(); + let mut members = Vec::new(); + let mut ids = Vec::new(); + for (name, range) in [ + ("narrow-one", "10.81.0.0/24"), + ("narrow-two", "10.81.1.0/24"), + ] { + let peer = TestAgent::spawn_with( + |c| c.with_overlay_ipv4_range(Some(range.parse().unwrap())), + &discovery, + ) + .await + .unwrap(); + let (name, secret) = network_outside( + name, + &[ + "10.81.0.0/24".parse().unwrap(), + "10.81.1.0/24".parse().unwrap(), + ], + ); + let id = peer.agent.join_network(&name, &secret).await.unwrap(); + peer.agent.network_status(id).await.unwrap(); + hub.agent.join_network(&name, &secret).await.unwrap(); + wait_until("hub adopts the established narrow range", || async { + (hub.agent.network_status(id).await.ok()?.range == Some(range.parse().unwrap())) + .then_some(()) + }) + .await; + members.push(peer); + ids.push(id); + } + let (name, secret) = network_outside( + "wide-overlap", + &[ + "10.81.0.0/24".parse().unwrap(), + "10.81.1.0/24".parse().unwrap(), + ], + ); + let wide = TestAgent::spawn_with( + |c| c.with_overlay_ipv4_range(Some("10.0.0.0/8".parse().unwrap())), + &discovery, + ) + .await + .unwrap(); + let id = wide.agent.join_network(&name, &secret).await.unwrap(); + wide.agent.network_status(id).await.unwrap(); + hub.agent.join_network(&name, &secret).await.unwrap(); + wait_until("wide range is blocked by both narrow networks", || async { + let status = hub.agent.status().await.ok()?; + let blocked = status.network(&id)?; + (!hub.agent.is_active(id).await && blocked.range_conflicts.len() == 2).then_some(()) + }) + .await; + let stopped = hub.agent.switch_network(id).await.unwrap(); + assert_eq!(stopped.len(), 2); + assert!(ids.iter().all(|id| stopped.contains(id))); + assert_eq!(hub.agent.switch_network(ids[0]).await.unwrap(), vec![id]); + assert!( + hub.agent.is_active(ids[1]).await, + "switching away from the wide range frees the other narrow range too" + ); + assert_eq!(hub.agent.switch_network(id).await.unwrap().len(), 2); + hub.agent.pause_network(ids[0]).await.unwrap(); + hub.agent.deactivate_network(id).await.unwrap(); + assert!( + !hub.agent.is_active(ids[0]).await, + "explicitly stopped networks do not resume" + ); + assert!(hub.agent.is_active(ids[1]).await); + assert!( + hub.agent + .switch_network(tsunagi::NetworkId::from_bytes([0; 32])) + .await + .is_err() + ); + assert!( + hub.agent.is_active(ids[1]).await, + "unknown selectors do not stop any network" + ); + hub.agent.shutdown().await; + wide.agent.shutdown().await; + for peer in members { + peer.agent.shutdown().await; + } +} + +#[tokio::test] +async fn simultaneous_switches_never_leave_overlapping_networks_active() { + let discovery = SharedMemoryDiscovery::new(); + let agent = TestAgent::spawn_with( + |c| c.with_overlay_ipv4_range(Some(DEFAULT_IPV4_RANGE)), + &discovery, + ) + .await + .unwrap(); + let (a, a_secret) = network("switch-a"); + let (b, b_secret) = network("switch-b"); + let a = agent.agent.join_network(&a, &a_secret).await.unwrap(); + agent.agent.network_status(a).await.unwrap(); + let b = agent.agent.join_network(&b, &b_secret).await.unwrap(); + let (left, right) = tokio::join!(agent.agent.switch_network(b), agent.agent.switch_network(a)); + left.unwrap(); + right.unwrap(); + assert_ne!( + agent.agent.is_active(a).await, + agent.agent.is_active(b).await + ); + assert_eq!( + agent + .agent + .list_networks() + .await + .unwrap() + .iter() + .filter(|n| n.active) + .count(), + 1 + ); + agent.agent.shutdown().await; +} diff --git a/crates/tsunagi-tray/src/agent.rs b/crates/tsunagi-tray/src/agent.rs index ba9d60b..2e5dd6a 100644 --- a/crates/tsunagi-tray/src/agent.rs +++ b/crates/tsunagi-tray/src/agent.rs @@ -183,7 +183,21 @@ async fn run( .await .map(|report| { let state = if report.active { "started" } else { "stopped" }; - format!("{} {}", state, report.name) + if report.suspended.is_empty() { + format!("{} {}", state, report.name) + } else { + format!( + "{} {}; suspended {} until it stops", + state, + report.name, + report + .suspended + .iter() + .map(|n| format!("{} ({})", n.name, n.network_id)) + .collect::>() + .join(", ") + ) + } }) .map_err(|err| err.to_string()), Command::SetBroadcast { @@ -211,7 +225,17 @@ async fn run( } Command::Join { name, secret } => ipc::join_network(socket, &name, &secret) .await - .map(|report| format!("joined {}", report.name)) + .map(|report| { + format!( + "{} {}", + if report.active { + "joined" + } else { + "configured; stopped by range conflict:" + }, + report.name + ) + }) .map_err(|err| err.to_string()), Command::Leave { network_id } => ipc::leave_network(socket, &network_id) .await diff --git a/crates/tsunagi-tray/src/icon.rs b/crates/tsunagi-tray/src/icon.rs index 4ae59b2..9ebc999 100644 --- a/crates/tsunagi-tray/src/icon.rs +++ b/crates/tsunagi-tray/src/icon.rs @@ -78,6 +78,7 @@ fn colour_of(health: Health) -> [u8; 3] { match health { Health::Disconnected => DARK_RED, Health::Connected => BLUE, + Health::RangeConflict => [0xd9, 0x91, 0x20], Health::ExitNode => GREEN, } } diff --git a/crates/tsunagi-tray/src/ui.rs b/crates/tsunagi-tray/src/ui.rs index 235b392..525f516 100644 --- a/crates/tsunagi-tray/src/ui.rs +++ b/crates/tsunagi-tray/src/ui.rs @@ -252,13 +252,30 @@ fn draw_network( format::sparkline(ui, series, state.unit, 34.0); + if !network.range_conflicts.is_empty() { + ui.colored_label(BAD, "Stopped: overlapping network ranges"); + for conflict in &network.range_conflicts { + ui.label(format!("{} ({}) uses {}", conflict.name, conflict.network_id, conflict.range)); + } + ui.label("Switching suspends these networks until this network stops, then resumes them automatically."); + if network.auto_start { ui.weak("This network will resume automatically when its range is free."); } + } ui.horizontal(|ui| { - let mut active = network.active; - if ui.checkbox(&mut active, "active").changed() { - agent.send(Command::SetActive { - network_id: network.network_id.clone(), - active, - }); + if !network.active && !network.range_conflicts.is_empty() { + if ui.button("Switch to this network").clicked() { + agent.send(Command::SetActive { network_id: network.network_id.clone(), active: true }); + } + if network.auto_start && ui.button("Keep stopped").clicked() { + agent.send(Command::SetActive { network_id: network.network_id.clone(), active: false }); + } + } else { + let mut active = network.active; + if ui.checkbox(&mut active, "active").changed() { + agent.send(Command::SetActive { + network_id: network.network_id.clone(), + active, + }); + } } let mut broadcast = network.broadcast; if ui.checkbox(&mut broadcast, "broadcast").changed() { diff --git a/crates/tsunagi-tray/src/watch.rs b/crates/tsunagi-tray/src/watch.rs index 43c3e86..ff5b82e 100644 --- a/crates/tsunagi-tray/src/watch.rs +++ b/crates/tsunagi-tray/src/watch.rs @@ -22,6 +22,8 @@ pub(crate) enum Health { Disconnected, /// Connected, nothing out of the ordinary. Connected, + /// At least one configured network is blocked by an overlapping active range. + RangeConflict, /// This device's traffic leaves through an exit node, and that works. ExitNode, } @@ -33,6 +35,13 @@ impl Health { let Some(report) = report else { return Self::Disconnected; }; + if report + .networks + .iter() + .any(|network| network.auto_start && !network.range_conflicts.is_empty()) + { + return Self::RangeConflict; + } if !report .networks .iter() @@ -56,6 +65,7 @@ impl Health { match self { Self::Disconnected => "tsunagi: not connected", Self::Connected => "tsunagi: connected", + Self::RangeConflict => "tsunagi: network range conflict; open to switch networks", Self::ExitNode => "tsunagi: using an exit node", } } @@ -108,6 +118,24 @@ mod tests { } } + #[test] + fn overlapping_networks_are_visible_even_while_other_peers_are_connected() { + let mut blocked = NetworkReport { + auto_start: true, + ..Default::default() + }; + blocked + .range_conflicts + .push(tsunagi::ipc::RangeConflictReport { + name: "other".into(), + network_id: "other-id".into(), + range: "10.90.0.0/16".into(), + }); + let status = report(vec![network(2, ExitReport::default()), blocked]); + assert_eq!(Health::of(Some(&status)), Health::RangeConflict); + assert!(Health::RangeConflict.label().contains("switch networks")); + } + #[test] fn no_agent_or_nobody_connected_is_the_red_state() { assert_eq!(Health::of(None), Health::Disconnected); diff --git a/crates/tsunagi-wg-quic/tests/local_control.rs b/crates/tsunagi-wg-quic/tests/local_control.rs index 5643d38..f6c5591 100644 --- a/crates/tsunagi-wg-quic/tests/local_control.rs +++ b/crates/tsunagi-wg-quic/tests/local_control.rs @@ -277,6 +277,7 @@ impl tsunagi::ipc::ReportSource for Control { network_id: network_id.to_string(), already_configured: already, name_shared_with: None, + active: self.0.is_active(network_id).await, }) }) } diff --git a/crates/tsunagi-wg-quic/tests/wireguard.rs b/crates/tsunagi-wg-quic/tests/wireguard.rs index 2172229..d936cf7 100644 --- a/crates/tsunagi-wg-quic/tests/wireguard.rs +++ b/crates/tsunagi-wg-quic/tests/wireguard.rs @@ -109,6 +109,30 @@ impl WgAgent { .await } + async fn spawn_auto(discovery: &SharedMemoryDiscovery, tag: &str) -> Self { + let dir = TempDir::new().unwrap(); + let tuns = MemoryTunFactory::new(); + let plugin = WireguardPlugin::open( + WireguardConfig::new(dir.path().join("wireguard")) + .with_reconcile(Duration::from_millis(20), Duration::from_millis(250)), + ) + .await + .unwrap(); + let agent = Agent::spawn( + config_with(dir.path(), discovery) + .with_interface(Arc::new(tuns.clone()), tag, 1280) + .with_plugin(plugin.clone() as Arc), + ) + .await + .unwrap(); + Self { + dir, + agent, + plugin, + tuns, + } + } + /// Starts an agent with an explicit overlay range, or none at all. async fn spawn_range( discovery: &SharedMemoryDiscovery, @@ -989,7 +1013,7 @@ async fn two_networks_share_one_interface_with_keys_and_ranges_of_their_own() { let (name_b, secret_b) = network("wg-right"); let beta_range = Some("10.99.0.0/16".parse::().unwrap()); - let hub = WgAgent::spawn(&discovery, "th").await; + let hub = WgAgent::spawn_auto(&discovery, "th").await; let left = WgAgent::spawn(&discovery, "tl").await; let right = WgAgent::spawn_range(&discovery, "tr", beta_range).await; diff --git a/crates/tsunagi/src/main.rs b/crates/tsunagi/src/main.rs index 6befb47..1911611 100644 --- a/crates/tsunagi/src/main.rs +++ b/crates/tsunagi/src/main.rs @@ -226,7 +226,9 @@ enum NetworkAction { /// Exact network name, full id or unique id prefix. network: String, }, - /// Serves a stopped network again, from where it left off. + /// Serves a stopped network again, suspending networks with overlapping ranges. + /// + /// Suspended networks resume when this one stops. The choice survives restart. Start { /// Exact network name, full id or unique id prefix. network: String, @@ -351,7 +353,7 @@ fn resolve_ipv4_range( text.parse::() .map_err(|err| format!("--ipv4-range {text}: {err}"))?, )), - None => Ok(Some(tsunagi::state::DEFAULT_IPV4_RANGE)), + None => Ok(None), } } @@ -489,8 +491,8 @@ struct UpArgs { /// IPv4 overlay range, as `address/prefix`, or `none` to disable IPv4. /// - /// Defaults to 10.13.37.0/24. Only the first member to join decides: - /// a network that has already settled on a range wins, and a joining + /// Defaults to a stable pseudorandom /24 derived from the network id. + /// A network that has already settled on a range wins, and a joining /// agent adopts what it finds. Addresses are allocated from it and /// recorded in signed state, so each member keeps its own across /// restarts and long absences. @@ -1672,6 +1674,7 @@ impl tsunagi::ipc::ReportSource for AgentControl { network_id: network_id.to_string(), already_configured: already, name_shared_with: shared, + active: self.agent.is_active(network_id).await, }) }) } @@ -1743,36 +1746,50 @@ impl tsunagi::ipc::ReportSource for AgentControl { let wanted: tsunagi::NetworkId = network_id .parse() .map_err(|err| format!("`{network_id}` is not a network id: {err}"))?; - let name = self + let configured = self .agent .list_networks() .await .map_err(|err| err.to_string())? .into_iter() .find(|network| network.network_id == wanted) - .map(|network| network.name.as_str().to_string()) .ok_or_else(|| format!("this agent is not in {wanted}"))?; + let name = configured.name.to_string(); let was = self.agent.is_active(wanted).await; - if was != active { + let mut suspended = Vec::new(); + if active && !was { // Both of these also remember the answer, so a restart // does what the last instruction said. - if active { - self.agent - .activate_network(wanted) - .await - .map_err(|err| err.to_string())?; - } else { - self.agent - .deactivate_network(wanted) - .await - .map_err(|err| err.to_string())?; - } + let before = self.agent.status().await.map_err(|err| err.to_string())?; + let stopped = self + .agent + .switch_network(wanted) + .await + .map_err(|err| err.to_string())?; + suspended = before + .networks + .iter() + .filter(|n| stopped.contains(&n.network_id)) + .filter_map(|n| { + Some(tsunagi::ipc::RangeConflictReport { + network_id: n.network_id.to_string(), + name: n.name.to_string(), + range: n.range?.to_string(), + }) + }) + .collect(); + } else if !active { + self.agent + .pause_network(wanted) + .await + .map_err(|err| err.to_string())?; } Ok(tsunagi::ipc::ActiveReport { name, active, - changed: was != active, + changed: was != active || configured.auto_start != active, + suspended, }) }) } @@ -1972,21 +1989,28 @@ async fn join_network( .await?; // The id in full either way: it is what every other command takes, // and the shortened form in a report is for reading, not copying. - match standing { - NetworkStanding::New => { - println!("joined `{}` ({})", report.name, report.network_id) + if !report.active { + println!( + "configured `{}` ({}); stopped because its range overlaps an active network. `tsng network start {}` switches to it and suspends conflicting networks", + report.name, report.network_id, report.network_id + ); + } else { + match standing { + NetworkStanding::New => { + println!("joined `{}` ({})", report.name, report.network_id) + } + NetworkStanding::Known => println!( + "`{}` ({}) was already here; it is running", + report.name, report.network_id + ), + // Joining is an instruction to run it, so it undoes a stop — + // said out loud, because a pause that ends without a word is + // a pause nobody can rely on. + NetworkStanding::Stopped => println!( + "`{}` ({}) was stopped; it is running again", + report.name, report.network_id + ), } - NetworkStanding::Known => println!( - "`{}` ({}) was already here; it is running", - report.name, report.network_id - ), - // Joining is an instruction to run it, so it undoes a stop — - // said out loud, because a pause that ends without a word is - // a pause nobody can rely on. - NetworkStanding::Stopped => println!( - "`{}` ({}) was stopped; it is running again", - report.name, report.network_id - ), } if let Some(other) = &report.name_shared_with { eprintln!( @@ -2112,6 +2136,13 @@ async fn show_networks( short(&id, 10) ), ), + Some(live) if !live.range_conflicts.is_empty() => ( + "stopped · range conflict".to_string(), + format!( + "{}; `tsng network start {id}` switches to this network and suspends them until it stops", + conflict_description(live) + ), + ), Some(_) => ( "stopped".to_string(), format!( @@ -2517,6 +2548,12 @@ async fn set_active( (true, true) => println!("started `{}` ({})", report.name, short(&id, 10)), (true, false) => println!("`{}` ({}) was already running", report.name, short(&id, 10)), } + for suspended in &report.suspended { + println!( + "suspended `{}` ({}): {} overlaps; it resumes when the conflicting network stops", + suspended.name, suspended.network_id, suspended.range + ); + } if !report.active { eprintln!( "\nNothing was announced: to the others this device is away, and the address \ @@ -2530,7 +2567,11 @@ async fn set_active( // No agent: the stored flag is what the next start reads. let storage = tsunagi::storage::Storage::open(paths)?; - storage.set_auto_start(network.network_id, active).await?; + if active { + storage.prefer_network(network.network_id).await?; + } else { + storage.set_auto_start(network.network_id, false).await?; + } storage.release_ownership_lock(); println!( "`{name}` ({}) will {} with the next `tsng up`", @@ -3274,6 +3315,15 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve rows } +fn conflict_description(network: &tsunagi::ipc::NetworkReport) -> String { + network + .range_conflicts + .iter() + .map(|c| format!("`{}` ({}) uses {}", c.name, c.network_id, c.range)) + .collect::>() + .join("; ") +} + /// One network: what it is, who is in it, and what has happened since start. fn network_section( network: &tsunagi::ipc::NetworkReport, @@ -3328,19 +3378,12 @@ fn network_section( ), ); } - if let Some(conflict) = &network.range_conflict { - section.push( - Row::new( - Health::Degraded, - "range", - format!("cannot use {conflict}: another network here already does"), - ) - .with_note( - "one agent has one interface, so an address belongs to one network. \ - This one waits to adopt whatever its members settle on; give it \ - `--ipv4-range` of its own to propose one.", - ), - ); + if !network.range_conflicts.is_empty() { + section.push(Row::new(Health::Degraded, "range conflict", conflict_description(network)) + .with_note(format!("`tsng network start {}` switches to this network and suspends the listed networks; they resume when it stops", network.network_id))); + } else if let Some(conflict) = &network.range_conflict { + section.push(Row::new(Health::Degraded, "range", format!("cannot use {conflict}: another network here already does")) + .with_note(format!("`tsng network start {}` switches to this network and suspends conflicting networks", network.network_id))); } if let Some(overlay) = &network.overlay { @@ -4375,10 +4418,12 @@ async fn up(args: UpArgs) -> Result<(), Box> { ])); let mut config = AgentConfig::new(paths.clone()) - .with_overlay_ipv4_range(ipv4_range) .with_transport(args.reach.into()) .with_discovery(discovery) .with_discovery_interval(Duration::from_secs(5)); + if args.ipv4_range.is_some() { + config = config.with_overlay_ipv4_range(ipv4_range); + } if dht_enabled(&args) { config = config.with_dht(MainlineDiscovery::default()); } @@ -4947,6 +4992,7 @@ async fn build_report( }; NetworkReport { + auto_start: network.auto_start, exit: exit_report(network), broadcast: network.broadcast, name: network.name.to_string(), @@ -4994,6 +5040,15 @@ async fn build_report( .collect(), range: network.range.map(|range| range.to_string()), range_conflict: network.range_conflict.map(|range| range.to_string()), + range_conflicts: network + .range_conflicts + .iter() + .map(|c| tsunagi::ipc::RangeConflictReport { + network_id: c.network_id.to_string(), + name: c.name.to_string(), + range: c.range.to_string(), + }) + .collect(), dial_failures: network.metrics.dial_failures, handshake_failures: network.metrics.handshake_failures, control_messages: ( @@ -5361,6 +5416,7 @@ mod status_tests { /// The situation that prompted this: one peer left and came back. fn network_after_a_peer_returned() -> NetworkReport { NetworkReport { + auto_start: true, exit: Default::default(), hostname_conflict: None, broadcast: true, @@ -5403,6 +5459,7 @@ mod status_tests { ], range: Some("10.13.37.0/24".into()), range_conflict: None, + range_conflicts: Vec::new(), // Everything below happened while the peer was away. dial_failures: 9, handshake_failures: 0, @@ -5534,7 +5591,36 @@ mod status_tests { let text = out.render(false); assert!(text.contains("no range to allocate from"), "{text}"); assert!(text.contains("another network here already does"), "{text}"); - assert!(text.contains("--ipv4-range"), "the fix is named: {text}"); + assert!( + text.contains("tsng network start"), + "the fix is named: {text}" + ); + } + + #[test] + fn a_blocked_network_names_all_blockers_and_the_switch_action() { + let mut network = network_after_a_peer_returned(); + network.active = false; + network.overlay = None; + network.range_conflicts = vec![tsunagi::ipc::RangeConflictReport { + name: "other-network".into(), + network_id: "full-other-network-id".into(), + range: "10.13.0.0/16".into(), + }]; + let mut out = report::Report::new(); + out.push(network_section(&network, OWN, false)); + let text = out.render(false); + let health = out.worst(); + for detail in [ + "other-network", + "full-other-network-id", + "10.13.0.0/16", + "tsng network start xa7gyz", + "resume when it stops", + ] { + assert!(text.contains(detail), "missing {detail}: {text}"); + } + assert_eq!(health, Health::Degraded); } #[test] diff --git a/crates/tsunagi/tests/network_cli.rs b/crates/tsunagi/tests/network_cli.rs index 53f84bf..192d00a 100644 --- a/crates/tsunagi/tests/network_cli.rs +++ b/crates/tsunagi/tests/network_cli.rs @@ -6,7 +6,7 @@ #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)] -use std::time::Duration; +use std::time::{Duration, Instant}; use tempfile::TempDir; @@ -50,8 +50,13 @@ fn read_networks(dir: &TempDir) -> Vec { /// An agent with no network at all, the way a daemon is started before /// anything has been decided. fn start_bare(port: u16) -> Running { + start_bare_range(port, None) +} + +fn start_bare_range(port: u16, range: Option<&str>) -> Running { let dir = TempDir::new().unwrap(); - let child = std::process::Command::new(env!("CARGO_BIN_EXE_tsng")) + let mut command = std::process::Command::new(env!("CARGO_BIN_EXE_tsng")); + command .arg("up") .arg("--state-dir") .arg(dir.path().join("state")) @@ -62,11 +67,31 @@ fn start_bare(port: u16) -> Running { .arg(format!("127.0.0.1:{port}")) .args(["--log", "error", "--status-interval", "0"]) .stdout(std::process::Stdio::null()) - .stderr(std::process::Stdio::null()) - .spawn() - .expect("the agent binary starts"); - std::thread::sleep(Duration::from_secs(2)); - Running { child, dir } + .stderr(std::process::Stdio::null()); + if let Some(range) = range { + command.args(["--ipv4-range", range]); + } + let child = command.spawn().expect("the agent binary starts"); + let mut agent = Running { child, dir }; + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let socket = tsunagi::ipc::control_socket_path(&agent.dir.path().join("state")); + runtime.block_on(async { + tokio::time::timeout(Duration::from_secs(30), async { + while !tsunagi::ipc::is_serving(&socket).await { + assert!( + agent.child.try_wait().unwrap().is_none(), + "agent exited before becoming ready" + ); + tokio::time::sleep(Duration::from_millis(25)).await; + } + }) + .await + .expect("agent becomes ready"); + }); + agent } /// An agent already in one network, the way most of these start. @@ -461,3 +486,139 @@ fn an_exit_node_is_offered_per_network_only_when_asked_and_nobody_offers_by_defa String::from_utf8_lossy(&unknown.stderr) ); } + +#[test] +fn overlapping_networks_are_reported_and_cli_start_switches_then_resumes() { + let agent = start_bare_range(0, Some("10.90.0.0/16")); + for name in ["resident", "games"] { + let joined = agent.run(&["join", "-n", name, "-s", "a-secret-for-the-conflict-test"]); + assert!( + joined.status.success(), + "{}", + String::from_utf8_lossy(&joined.stderr) + ); + if name == "games" { + let text = String::from_utf8_lossy(&joined.stdout); + assert!( + text.contains("stopped because its range overlaps"), + "{text}" + ); + assert!(!text.contains("it is running"), "{text}"); + } + } + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let socket = tsunagi::ipc::control_socket_path(&agent.dir.path().join("state")); + let snapshot = || { + runtime + .block_on(tsunagi::ipc::request_status(&socket)) + .unwrap() + }; + let before = snapshot(); + let resident = before + .networks + .iter() + .find(|n| n.name == "resident") + .unwrap(); + let games = before.networks.iter().find(|n| n.name == "games").unwrap(); + assert!(resident.active); + assert!(!games.active); + assert_eq!(games.range_conflicts[0].network_id, resident.network_id); + assert_eq!(games.range_conflicts[0].name, "resident"); + for args in [vec!["status"], vec!["network"]] { + let result = agent.run(&args); + let text = String::from_utf8_lossy(&result.stdout); + assert!(text.contains("range conflict"), "{text}"); + assert!(text.contains(&resident.network_id), "{text}"); + assert!(text.contains("tsng network start"), "{text}"); + } + let switched = agent.run(&["network", "start", "games"]); + assert!( + switched.status.success(), + "{}", + String::from_utf8_lossy(&switched.stderr) + ); + let text = String::from_utf8_lossy(&switched.stdout); + assert!( + text.contains("started `games`") && text.contains("suspended `resident`"), + "{text}" + ); + let after = snapshot(); + assert!( + after + .networks + .iter() + .find(|n| n.name == "games") + .unwrap() + .active + ); + assert!( + !after + .networks + .iter() + .find(|n| n.name == "resident") + .unwrap() + .active + ); + assert!(agent.run(&["network", "stop", "games"]).status.success()); + assert!( + snapshot() + .networks + .iter() + .find(|n| n.name == "resident") + .unwrap() + .active + ); + + assert!(agent.run(&["network", "start", "games"]).status.success()); + let stop_suspended = agent.run(&["network", "stop", "resident"]); + assert!(stop_suspended.status.success()); + assert!(String::from_utf8_lossy(&stop_suspended.stdout).contains("stopped `resident`")); + assert!(agent.run(&["network", "stop", "games"]).status.success()); + assert!( + snapshot() + .networks + .iter() + .all(|n| !n.active && !n.auto_start) + ); +} + +#[test] +fn cli_defaults_derive_each_networks_range() { + let agent = start_bare(0); + for name in ["derived-one", "derived-two"] { + assert!( + agent + .run(&["join", "-n", name, "-s", "a-secret-for-the-derived-test"]) + .status + .success() + ); + } + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let socket = tsunagi::ipc::control_socket_path(&agent.dir.path().join("state")); + let deadline = Instant::now() + Duration::from_secs(30); + loop { + let report = runtime + .block_on(tsunagi::ipc::request_status(&socket)) + .unwrap(); + if report.networks.iter().all(|n| n.range.is_some()) { + assert_eq!(report.networks.len(), 2); + for network in report.networks { + let id = network.network_id.parse().unwrap(); + assert!(network.active); + assert_eq!( + network.range, + Some(tsunagi::state::derived_ipv4_range(id).to_string()) + ); + } + break; + } + assert!(Instant::now() < deadline, "networks allocate ranges"); + std::thread::sleep(Duration::from_millis(25)); + } +} diff --git a/docs/testing.md b/docs/testing.md index 04de202..8f65284 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -98,6 +98,13 @@ a batch does not stop the rest, and allocation is deterministic, spread out, walks past everything taken and reports a full range instead of handing out a duplicate. +`tests/range_conflicts.rs` checks isolated first members choosing the same +range, persistence across restart, preservation of legacy ranges, exclusion +of equal and nested ranges, late authenticated conflicts, switching all +blockers, automatic resume, cancelling a suspended network's resume and +concurrent switch requests. These use real agents, loopback handshakes and +SQLite; they do not change host network settings. + `tests/local_control.rs` covers the local control socket end to end: a client asking a running agent for status over a real Unix socket, joining and leaving a network through it, a leftover socket file being replaced while a diff --git a/docs/wireguard.md b/docs/wireguard.md index b620f48..ad08f58 100644 --- a/docs/wireguard.md +++ b/docs/wireguard.md @@ -92,8 +92,8 @@ and then recorded as a signed fact**, using the model in [sync-model.md](sync-model.md). ```text -default range 10.13.37.0/24 (override with --ipv4-range) -who decides the first member to claim; later ones adopt what they find +default range stable pseudorandom /24 in 10/8, derived from NetworkId +who decides all members derive the default; signed explicit ranges win who signs the claiming member, with its persistent device key where it is kept state.sqlite, and every replica that has seen it what a return costs nothing: the old address is reclaimed @@ -120,11 +120,34 @@ the same address at the same moment are resolved by a rule both compute identically — the lower endpoint id keeps it — and the loser simply allocates again with a higher version. -**The range is agreed, not configured per member.** `--ipv4-range` says what -this agent would use; a network that has already settled on something else -wins, and the agent adopts it. So the flag matters for whoever starts the -network and is harmless afterwards. Pass `--ipv4-range none` for an IPv6-only -overlay. +**The range is agreed, not configured per member.** New networks use the +same deterministic `/24` on every device, independently of which network a +member activates first or whether any peers are reachable. The derivation +is unchanged from the former fallback; network identity and signed record +encoding are unchanged. Distinct network ids can still derive the same range. + +`--ipv4-range` proposes an explicit range; existing signed ranges take +precedence over the new derived default, including the former `10.13.37.0/24`. +Among disagreeing non-default ranges the lowest author id still wins. Updated +members propagate their adopted range in their signed claims; older releases +still apply their original selection rule until those claims converge. +`--ipv4-range none` disables local proposals, as before. + +An agent reserves each active network's range before starting its runtime. +Overlapping ranges, including nested blocks such as a `/24` inside a `/16`, +cannot run simultaneously. A late authenticated state update that introduces +an overlap suspends that network before publishing new allocations to plugins +or the interface. Other networks continue running. + +Status includes every blocking network's name, id and range. `tsng network +start ` and the tray's switch button stop all blockers before +starting the selected network. Suspended networks keep their startup request +and resume when the blocker stops or is forgotten. A manual stop cancels that +request even for a network already suspended. The user's preference order is +stored locally, so the same selection wins at the next startup; it is never +advertised or treated as a remote network setting. Local IPC uses wire version +19 for the new reports: restart the agent and tray together after upgrading. +The peer control protocol and signed record encoding do not change. The agent puts the address on the interface itself, as soon as the network has agreed on it — no restart, and the interface is not recreated, which would