From 533921059afdc7a20a9ead7983f2148009155b34 Mon Sep 17 00:00:00 2001 From: Aleksandr Bogomiakov Date: Thu, 1 Oct 2026 15:03:43 +0100 Subject: [PATCH] Install broadcast route and firewall allowance on the host while broadcast is on LAN discovery (CS 1.6 and similar) only worked after two manual steps on every host: route 255.255.255.255 through the TUN, and allow inbound traffic on it. Without the route the game's limited broadcast leaves via the physical adapter and never reaches the overlay; without the firewall allowance the server's unicast reply (from a different address than the one the request went to, so never matched by conntrack) is dropped before the game's socket. Verified by hand on Linux: with both in place the server answered A2S_INFO over the overlay. The system level (not a plugin) now installs exactly those two things for the interface it owns, and removes them when broadcast is turned off, the network is left or stopped, or the agent exits. Design: - New boundary overlay::hostrules (BroadcastHostRules: idempotent apply/clear, with Linux, Windows, Mock and Unsupported implementations). TunFactory gains host_rules(); ManagedTunFactory::with_host_rules wires it in the CLI. - Interface::sync_broadcast_rules reconciles the host against the routing table (RoutingTable::broadcast_host_target) and is called after address sync, on live `network broadcast on/off`, on network deactivation and by remove(). Failure is contained: the outcome is kept and logged, never fatal. - One interface carries several networks but the host has one 255.255.255.255 route, so the source is the lowest NetworkId among broadcast-enabled networks with an address (documented in docs/broadcast.md). - Linux: the route is added on the existing netlink thread (preferred source = overlay address, replace semantics). The firewall rule is `iptables -I INPUT -i -p udp -s ` tagged with a comment `tsunagi:`; it is UDP-only and range-limited on purpose so TCP services stay unexposed. iptables is located in fixed system directories, never via PATH; stale tagged rules (e.g. from a crash) are replaced. A missing iptables leaves the route in place and is reported. rp_filter is not touched. - Windows: PowerShell (absolute path) New-NetRoute metric 1 and a tagged inbound UDP Defender rule scoped to the interface alias and overlay range. - status shows a `broadcast rules` row (ok, or which half failed and why). IPC control protocol bumped 14 -> 15 because OverlayReport gained a field. - AGENTS.md records the narrow exception to "never touch routing/firewall". Compatibility: restart the agent after upgrading (control protocol 15). No wire or schema change. Checks run (Linux only): cargo fmt --check, cargo clippy --locked --workspace --all-targets -D warnings, cargo test --locked --workspace --all-targets, all passing. Tests use a mock host and a scripted iptables; no OS setting is changed. NOT done: the Windows code was never compiled for or run on Windows (cross-compiling failed for lack of a C toolchain), only its script building is unit-tested; the real netlink route and iptables calls were not exercised against a live kernel in the suite; CS 1.6 was not launched from this change. --- AGENTS.md | 11 +- README.md | 8 +- crates/tsunagi-cli/src/main.rs | 103 +++- crates/tsunagi/src/agent/mod.rs | 4 + crates/tsunagi/src/agent/network.rs | 9 + crates/tsunagi/src/agent/status.rs | 4 + crates/tsunagi/src/ipc/mod.rs | 17 +- crates/tsunagi/src/overlay/hostrules/linux.rs | 566 ++++++++++++++++++ crates/tsunagi/src/overlay/hostrules/mod.rs | 325 ++++++++++ .../tsunagi/src/overlay/hostrules/windows.rs | 272 +++++++++ crates/tsunagi/src/overlay/interface.rs | 220 +++++++ crates/tsunagi/src/overlay/mod.rs | 11 + .../tsunagi/src/overlay/provision/factory.rs | 18 +- crates/tsunagi/src/overlay/provision/linux.rs | 161 ++++- crates/tsunagi/src/overlay/provision/mod.rs | 2 + crates/tsunagi/src/overlay/router.rs | 76 +++ crates/tsunagi/src/overlay/tun.rs | 8 + docs/broadcast.md | 42 +- 18 files changed, 1834 insertions(+), 23 deletions(-) create mode 100644 crates/tsunagi/src/overlay/hostrules/linux.rs create mode 100644 crates/tsunagi/src/overlay/hostrules/mod.rs create mode 100644 crates/tsunagi/src/overlay/hostrules/windows.rs diff --git a/AGENTS.md b/AGENTS.md index fe224ee..241f358 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -90,6 +90,12 @@ Keep these separate. Crossing them is the main thing to review for. transaction, **before** the record is announced. - **Plugins own their system objects.** A plugin creates and removes its own interface and nothing else. Never touch routing, DNS or firewall settings. + The one exception is the system level, not a plugin: while a network's + broadcast is on, `overlay::hostrules` may install exactly a + `255.255.255.255/32` route and an inbound UDP firewall allowance (overlay range + only) for the interface the agent owns. Both are tagged `tsunagi:`, + reversible, removed on broadcast off, leave and shutdown, and computed from + local state, never from a remote announcement. - **Device identity vs network identity.** The iroh endpoint id is the device's public key. `NetworkId` is derived from name + secret only. Never conflate them, and never let one change the other. @@ -137,7 +143,8 @@ Keep these separate. Crossing them is the main thing to review for. up shutdown. Wind tasks down with a grace period and then abort. 10. **Nothing from a remote announcement becomes a shell command, a filesystem path or an OS setting.** The agent never touches interfaces or OS settings - it did not create. + it did not create (the tagged broadcast host rules above are for its own + interface). ## Where things live @@ -150,7 +157,7 @@ Keep these separate. Crossing them is the main thing to review for. | `crates/tsunagi/src/net.rs` | iroh endpoint adapter and observability snapshots | | `crates/tsunagi/src/agent/` | agent lifecycle, per-network runtimes, sessions, events, status | | `crates/tsunagi/src/state/` | signed records that outlive a session, their merge rules and address allocation | -| `crates/tsunagi/src/overlay/` | the one interface an agent owns: provisioning, the TUN, routing, source checks | +| `crates/tsunagi/src/overlay/` | the one interface an agent owns: provisioning, the TUN, routing, source checks, broadcast host rules | | `crates/tsunagi/src/dataplane/` | the protocol contract and the authenticated packet transport | | `crates/tsunagi/src/dns/` | the DNS view of a network: zone, server, resolver publication | | `crates/tsunagi-wg-quic/` | the `wg-quic` protocol: its keys, its announcement, its tunnels | diff --git a/README.md b/README.md index a997ddf..0aea736 100644 --- a/README.md +++ b/README.md @@ -604,8 +604,12 @@ tsunagi network broadcast games on ``` The choice persists across restarts and a plain `join`. It can be changed while -the agent runs; `status` shows it for each network. The game must send through -the Tsunagi interface. Physical LAN capture/subnet sharing is not implemented. +the agent runs; `status` shows it for each network. While broadcast is on, the +agent routes `255.255.255.255` through the Tsunagi interface and allows inbound +UDP from the overlay range on it (Linux and Windows, needs the same privilege as +the interface), and removes both when it is turned off or the agent stops. Games +bound to a physical address still bypass it. Physical LAN capture/subnet sharing +is not implemented. See [broadcast.md](docs/broadcast.md) for domain isolation and future LAN gateways. ## How peers find each other diff --git a/crates/tsunagi-cli/src/main.rs b/crates/tsunagi-cli/src/main.rs index acbad93..01856d3 100644 --- a/crates/tsunagi-cli/src/main.rs +++ b/crates/tsunagi-cli/src/main.rs @@ -2722,6 +2722,25 @@ fn network_section( "broadcast", if network.broadcast { "on" } else { "off" }, )); + if network.broadcast + && let Some(rules) = network + .overlay + .as_ref() + .and_then(|o| o.broadcast_rules.as_ref()) + { + section.push(Row::new( + if rules.ok { + Health::Good + } else { + Health::Degraded + }, + "broadcast rules", + broadcast_rules_line( + rules, + network.overlay.as_ref().and_then(|o| o.address.as_deref()), + ), + )); + } let relayed = network.relay_forwarded + network.relay_sent_via + network.relay_received_via; if relayed > 0 { section.push(Row::new( @@ -4087,6 +4106,14 @@ async fn build_report( unroutable_sample: overlay .as_ref() .and_then(|o| o.counters.unroutable_sample.map(|a| a.to_string())), + broadcast_rules: overlay + .as_ref() + .and_then(|o| o.broadcast_rules.as_ref()) + .map(|rules| tsunagi::ipc::HostRulesReport { + ok: rules.is_ok(), + detail: rules.summary(), + source: rules.plan.source.to_string(), + }), }) } else { None @@ -4155,6 +4182,21 @@ async fn build_report( } } +/// The status line for the host rules, saying so when they are bound to +/// another network's address. +fn broadcast_rules_line( + rules: &tsunagi::ipc::HostRulesReport, + own_address: Option<&str>, +) -> String { + match own_address { + Some(own) if own != rules.source => format!( + "{} · route source is {}, another network's address", + rules.detail, rules.source + ), + _ => rules.detail.clone(), + } +} + /// Resolves when the process is asked to stop. /// /// Both Ctrl-C and `SIGTERM` are handled, so a service manager stopping the @@ -4193,16 +4235,22 @@ async fn stop_signal() -> &'static str { fn system_tun_factory() -> Result, Box> { use tsunagi::overlay::{ManagedTunFactory, NetlinkProvisioner}; let provisioner = NetlinkProvisioner::new()?; - Ok(Arc::new(ManagedTunFactory::new(Arc::new(provisioner)))) + let rules = Arc::new(provisioner.host_rules()); + Ok(Arc::new( + ManagedTunFactory::new(Arc::new(provisioner)).with_host_rules(rules), + )) } /// The Wintun adapter, created and configured by the agent and removed when it /// exits, the same as the Linux one. #[cfg(target_os = "windows")] fn system_tun_factory() -> Result, Box> { - use tsunagi::overlay::{ManagedTunFactory, WintunProvisioner}; + use tsunagi::overlay::{ManagedTunFactory, WindowsHostRules, WintunProvisioner}; let provisioner = WintunProvisioner::new()?; - Ok(Arc::new(ManagedTunFactory::new(Arc::new(provisioner)))) + Ok(Arc::new( + ManagedTunFactory::new(Arc::new(provisioner)) + .with_host_rules(Arc::new(WindowsHostRules::new())), + )) } /// There is no provisioner for this platform yet. @@ -4452,6 +4500,55 @@ mod status_tests { assert!(text.contains("mistyped secret"), "{text}"); } + #[test] + fn broadcast_rules_are_reported_with_the_reason_when_incomplete() { + let mut network = network_after_a_peer_returned(); + network.broadcast = true; + let render = |network: &tsunagi::ipc::NetworkReport| { + let mut out = report::Report::new(); + out.push(network_section(network, OWN, false)); + (out.render(false), out.worst()) + }; + + // Nothing installed (in-memory interface, or broadcast just off): no row. + let (text, _) = render(&network); + assert!(!text.contains("broadcast rules"), "{text}"); + + let overlay = network.overlay.as_mut().unwrap(); + overlay.broadcast_rules = Some(tsunagi::ipc::HostRulesReport { + ok: true, + detail: "ok (route + firewall)".into(), + source: "10.13.37.69".into(), + }); + let (text, worst) = render(&network); + assert!(text.contains("broadcast rules"), "{text}"); + assert!(text.contains("ok (route + firewall)"), "{text}"); + assert_ne!(worst, Health::Degraded, "{text}"); + + let overlay = network.overlay.as_mut().unwrap(); + overlay.broadcast_rules = Some(tsunagi::ipc::HostRulesReport { + ok: false, + detail: "incomplete; route ok; firewall failed: iptables not found".into(), + source: "10.13.37.69".into(), + }); + let (text, worst) = render(&network); + assert!( + text.contains("firewall failed: iptables not found"), + "{text}" + ); + assert_eq!(worst, Health::Degraded, "{text}"); + + // Bound to another network's address: said, not hidden. + network.overlay.as_mut().unwrap().address = Some("10.99.0.5".into()); + let (text, _) = render(&network); + assert!(text.contains("another network's address"), "{text}"); + + // Broadcast off for this network: the line is not shown. + network.broadcast = false; + let (text, _) = render(&network); + assert!(!text.contains("broadcast rules"), "{text}"); + } + #[test] fn an_agent_with_nobody_to_contact_says_so_instead_of_waiting_quietly() { // The wiped-and-restarted case: both devices join the same network diff --git a/crates/tsunagi/src/agent/mod.rs b/crates/tsunagi/src/agent/mod.rs index ad300ca..d0ae627 100644 --- a/crates/tsunagi/src/agent/mod.rs +++ b/crates/tsunagi/src/agent/mod.rs @@ -481,6 +481,7 @@ impl Agent { mtu: interface.mtu(), addresses: interface.wanted_addresses(), counters: interface.counters(), + broadcast_rules: interface.broadcast_rules(), }) } @@ -730,6 +731,9 @@ impl Agent { { tracing::warn!(%err, "cannot update the overlay addresses"); } + if let Some(interface) = self.inner.interface.get() { + interface.sync_broadcast_rules().await; + } self.inner.storage.set_auto_start(network_id, false).await?; Ok(()) } diff --git a/crates/tsunagi/src/agent/network.rs b/crates/tsunagi/src/agent/network.rs index d8304bb..b633715 100644 --- a/crates/tsunagi/src/agent/network.rs +++ b/crates/tsunagi/src/agent/network.rs @@ -495,6 +495,9 @@ impl Runtime { NetCommand::SetBroadcast { enabled, reply } => { self.params.broadcast = enabled; self.update_broadcast(); + if let Some(interface) = self.params.interface.clone() { + interface.sync_broadcast_rules().await; + } self.reannounce(); let _ = reply.send(()); } @@ -1211,6 +1214,12 @@ impl Runtime { ), }); } + + // The broadcast route names this address as its source, so it goes in + // only once the address is really on the interface. + if !is_missing { + interface.sync_broadcast_rules().await; + } } // ------------------------------------------------------------ data plane diff --git a/crates/tsunagi/src/agent/status.rs b/crates/tsunagi/src/agent/status.rs index bdcf392..0f02391 100644 --- a/crates/tsunagi/src/agent/status.rs +++ b/crates/tsunagi/src/agent/status.rs @@ -53,6 +53,10 @@ pub struct OverlayStatus { pub addresses: Vec, /// What has happened on it. pub counters: crate::overlay::Counters, + /// The route and firewall allowance installed for LAN broadcast, when + /// some network has broadcast on and the interface is on the host. + /// `None` is "none installed", not "unknown". + pub broadcast_rules: Option, } /// A member the signed state knows about, connected or not. diff --git a/crates/tsunagi/src/ipc/mod.rs b/crates/tsunagi/src/ipc/mod.rs index ccd5e29..af0736b 100644 --- a/crates/tsunagi/src/ipc/mod.rs +++ b/crates/tsunagi/src/ipc/mod.rs @@ -413,6 +413,21 @@ pub struct OverlayReport { pub multicast_packets: u64, /// One destination nobody owned, if there was one. pub unroutable_sample: Option, + /// The broadcast route and firewall allowance on this host, when the + /// agent installed them. `None` means none are installed. + pub broadcast_rules: Option, +} + +/// What the agent installed on the host so LAN broadcast reaches the overlay. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct HostRulesReport { + /// Whether both the route and the firewall allowance are in place. + pub ok: bool, + /// One line: `ok (route + firewall)`, or what is missing and why. + pub detail: String, + /// The overlay address the `255.255.255.255` route is bound to. With + /// several networks on one interface only one address can be its source. + pub source: String, } /// One overlay peer. @@ -568,7 +583,7 @@ pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5); /// /// Bump it whenever [`Request`], [`Response`] or anything they contain /// changes shape. -pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 14]); +pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 15]); /// Reads one request off an accepted stream, answers it, writes the response. /// diff --git a/crates/tsunagi/src/overlay/hostrules/linux.rs b/crates/tsunagi/src/overlay/hostrules/linux.rs new file mode 100644 index 0000000..d5bdf84 --- /dev/null +++ b/crates/tsunagi/src/overlay/hostrules/linux.rs @@ -0,0 +1,566 @@ +//! Broadcast host rules on Linux: a netlink route and an `iptables` rule. +//! +//! The route is added by the netlink thread the interface provisioner already +//! owns (see [`super::super::provision`]), because that thread is the only one +//! that ever holds `CAP_NET_ADMIN`. It belongs to the interface and goes away +//! with it. +//! +//! The firewall allowance is a rule in the `INPUT` chain, inserted at the top +//! so a default-deny chain further down cannot drop discovery replies first: +//! +//! ```text +//! iptables -I INPUT 1 -i -p udp -s -m comment --comment tsunagi: -j ACCEPT +//! ``` +//! +//! `iptables` is a program, not a netlink call, so it is looked up in fixed +//! system directories rather than on `PATH`, and its arguments are built from +//! values this agent computed. It works with both the legacy backend and +//! `iptables-nft`. A host with only `nft` or `firewalld` has no `iptables`; +//! that is reported on the firewall half and the route is still installed. +//! +//! The rule carries a comment tag. That is how a rule left behind by a crashed +//! run is found and replaced, and why nothing else in the chain is ever +//! deleted. + +use std::net::Ipv4Addr; + +use crate::BoxFuture; +use crate::overlay::provision::RouteHandle; + +use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome, tag_for}; + +const CHAIN: &str = "INPUT"; + +/// Fixed places `iptables` is installed. Never `PATH`: the agent is +/// privileged and a directory a user controls must not decide what it runs. +const IPTABLES_DIRECTORIES: [&str; 5] = [ + "/usr/sbin/iptables", + "/sbin/iptables", + "/usr/bin/iptables", + "/bin/iptables", + "/usr/local/sbin/iptables", +]; + +/// The first candidate that exists. +fn pick_iptables(exists: impl Fn(&str) -> bool) -> Option<&'static str> { + IPTABLES_DIRECTORIES + .into_iter() + .find(|candidate| exists(candidate)) +} + +/// What running `iptables` produced. +#[derive(Debug, Clone, PartialEq, Eq)] +enum Ran { + /// There is no `iptables` to run. + Missing, + /// It ran, and exited like this. + Exited { + success: bool, + stdout: String, + stderr: String, + }, +} + +/// Runs `iptables` with these arguments. A function value so the flow below +/// can be tested without touching the host. +type Run<'a> = &'a dyn Fn(&[String]) -> Ran; + +/// The match and target of the rule, without the chain or position. +fn rule_spec(plan: &BroadcastRulesPlan) -> Vec { + [ + "-i", + plan.interface.as_str(), + "-p", + "udp", + "-s", + &plan.range.to_string(), + "-m", + "comment", + "--comment", + &plan.tag(), + "-j", + "ACCEPT", + ] + .into_iter() + .map(str::to_string) + .collect() +} + +/// Waits briefly for the xtables lock instead of failing when another tool +/// holds it. +fn locked(args: impl IntoIterator) -> Vec { + ["-w", "5"] + .into_iter() + .map(str::to_string) + .chain(args) + .collect() +} + +fn check_args(plan: &BroadcastRulesPlan) -> Vec { + locked( + ["-C".to_string(), CHAIN.to_string()] + .into_iter() + .chain(rule_spec(plan)), + ) +} + +fn insert_args(plan: &BroadcastRulesPlan) -> Vec { + locked( + ["-I".to_string(), CHAIN.to_string(), "1".to_string()] + .into_iter() + .chain(rule_spec(plan)), + ) +} + +fn list_args() -> Vec { + locked(["-S".to_string(), CHAIN.to_string()]) +} + +/// Splits one `iptables -S` line into words, honouring double quotes, which +/// `iptables` puts around comments that need them. +fn split_rule_line(line: &str) -> Vec { + let mut words = Vec::new(); + let mut current = String::new(); + let mut quoted = false; + let mut started = false; + for c in line.chars() { + match c { + '"' => { + quoted = !quoted; + started = true; + } + c if c.is_whitespace() && !quoted => { + if started { + words.push(std::mem::take(&mut current)); + started = false; + } + } + c => { + current.push(c); + started = true; + } + } + } + if started { + words.push(current); + } + words +} + +/// The delete commands for every rule in an `iptables -S INPUT` listing that +/// carries exactly this tag. Anything else in the chain is left alone. +fn tagged_deletions(listing: &str, tag: &str) -> Vec> { + listing + .lines() + .filter_map(|line| { + let words = split_rule_line(line); + let [append, chain, rest @ ..] = words.as_slice() else { + return None; + }; + if append != "-A" || chain != CHAIN { + return None; + } + let tagged = rest + .windows(2) + .any(|pair| pair[0] == "--comment" && pair[1] == tag); + tagged.then(|| { + locked( + ["-D".to_string(), CHAIN.to_string()] + .into_iter() + .chain(rest.iter().cloned()), + ) + }) + }) + .collect() +} + +/// A failure message with the likely fix for the likely cause. +fn failure(stderr: &str) -> String { + let text = stderr.trim(); + let lower = text.to_ascii_lowercase(); + if lower.contains("permission denied") || lower.contains("must be root") { + format!("iptables needs root: {text}") + } else if text.is_empty() { + "iptables failed without saying why".to_string() + } else { + format!("iptables failed: {text}") + } +} + +fn missing(plan: &BroadcastRulesPlan) -> String { + format!( + "iptables not found; install it (the iptables-nft package is enough) or allow inbound \ + UDP from {} on `{}` in your firewall", + plan.range, plan.interface + ) +} + +/// Makes the firewall hold exactly one rule for this interface, the planned one. +fn apply_firewall(run: Run<'_>, plan: &BroadcastRulesPlan) -> RuleOutcome { + let present = match run(&check_args(plan)) { + Ran::Missing => return RuleOutcome::Failed(missing(plan)), + Ran::Exited { success, .. } => success, + }; + let listing = match run(&list_args()) { + Ran::Missing => return RuleOutcome::Failed(missing(plan)), + Ran::Exited { + success: true, + stdout, + .. + } => stdout, + Ran::Exited { stderr, .. } => return RuleOutcome::Failed(failure(&stderr)), + }; + let stale = tagged_deletions(&listing, &plan.tag()); + // `-C` matched an identical rule and nothing else carries our tag, so + // there is nothing to do. Otherwise start from a clean slate for this + // interface: a rule from an earlier run may name another range. + if present && stale.len() == 1 { + return RuleOutcome::Applied; + } + for delete in &stale { + if let Ran::Exited { + success: false, + stderr, + .. + } = run(delete) + { + return RuleOutcome::Failed(failure(&stderr)); + } + } + match run(&insert_args(plan)) { + Ran::Missing => RuleOutcome::Failed(missing(plan)), + Ran::Exited { success: true, .. } => RuleOutcome::Applied, + Ran::Exited { stderr, .. } => RuleOutcome::Failed(failure(&stderr)), + } +} + +/// Removes every rule tagged for this interface. Best effort. +fn clear_firewall(run: Run<'_>, interface: &str) { + let Ran::Exited { + success: true, + stdout, + .. + } = run(&list_args()) + else { + return; + }; + for delete in tagged_deletions(&stdout, &tag_for(interface)) { + if let Ran::Exited { + success: false, + stderr, + .. + } = run(&delete) + { + tracing::warn!( + interface, + "cannot remove the broadcast firewall rule: {}", + failure(&stderr) + ); + } + } +} + +/// Runs the real `iptables`. +fn run_iptables(args: &[String]) -> Ran { + let Some(program) = pick_iptables(|path| std::path::Path::new(path).is_file()) else { + return Ran::Missing; + }; + match std::process::Command::new(program).args(args).output() { + Ok(output) => Ran::Exited { + success: output.status.success(), + stdout: String::from_utf8_lossy(&output.stdout).into_owned(), + stderr: String::from_utf8_lossy(&output.stderr).into_owned(), + }, + Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ran::Missing, + Err(err) => Ran::Exited { + success: false, + stdout: String::new(), + stderr: err.to_string(), + }, + } +} + +/// The Linux implementation: netlink for the route, `iptables` for the rule. +#[derive(Debug)] +pub struct LinuxHostRules { + route: RouteHandle, +} + +impl LinuxHostRules { + pub(crate) fn new(route: RouteHandle) -> Self { + Self { route } + } + + async fn set_route(&self, interface: String, source: Ipv4Addr) -> RuleOutcome { + let route = self.route.clone(); + match tokio::task::spawn_blocking(move || route.set(&interface, source)).await { + Ok(Ok(())) => RuleOutcome::Applied, + Ok(Err(err)) => RuleOutcome::Failed(err.to_string()), + Err(err) => RuleOutcome::Failed(format!("the route task failed: {err}")), + } + } +} + +impl BroadcastHostRules for LinuxHostRules { + fn name(&self) -> &str { + "netlink+iptables" + } + + fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> { + Box::pin(async move { + let route = self.set_route(plan.interface.clone(), plan.source).await; + let owned = plan.clone(); + let firewall = + tokio::task::spawn_blocking(move || apply_firewall(&run_iptables, &owned)) + .await + .unwrap_or_else(|err| { + RuleOutcome::Failed(format!("the firewall task failed: {err}")) + }); + BroadcastRulesReport { + plan: plan.clone(), + route, + firewall, + } + }) + } + + fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> { + Box::pin(async move { + let route = self.route.clone(); + let name = interface.to_string(); + let route_name = name.clone(); + if let Ok(Err(err)) = + tokio::task::spawn_blocking(move || route.clear(&route_name)).await + { + tracing::warn!(interface = %name, %err, "cannot remove the broadcast route"); + } + let name = interface.to_string(); + let _ = tokio::task::spawn_blocking(move || clear_firewall(&run_iptables, &name)).await; + }) + } +} + +#[cfg(test)] +mod tests { + #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)] + + use super::*; + use std::cell::RefCell; + + fn plan() -> BroadcastRulesPlan { + BroadcastRulesPlan { + interface: "tsun0".into(), + source: Ipv4Addr::new(10, 13, 37, 142), + range: "10.13.37.0/24".parse().unwrap(), + } + } + + fn words(args: &[String]) -> String { + args.join(" ") + } + + #[test] + fn the_rule_is_udp_only_from_the_overlay_range_on_this_interface() { + let spec = words(&rule_spec(&plan())); + assert_eq!( + spec, + "-i tsun0 -p udp -s 10.13.37.0/24 -m comment --comment tsunagi:tsun0 -j ACCEPT" + ); + // Nothing here may widen to other protocols or every source. + assert!(spec.contains("-p udp")); + assert!(!spec.contains("0.0.0.0/0")); + assert!(words(&insert_args(&plan())).starts_with("-w 5 -I INPUT 1 -i tsun0")); + assert!(words(&check_args(&plan())).starts_with("-w 5 -C INPUT -i tsun0")); + } + + #[test] + fn iptables_is_found_in_fixed_directories_never_on_path() { + assert_eq!(pick_iptables(|_| false), None); + assert_eq!( + pick_iptables(|p| p == "/sbin/iptables"), + Some("/sbin/iptables") + ); + assert_eq!(pick_iptables(|_| true), Some("/usr/sbin/iptables")); + } + + #[test] + fn only_rules_with_our_exact_tag_are_selected_for_deletion() { + let listing = "\ +-P INPUT DROP +-A INPUT -s 10.13.37.0/24 -i tsun0 -p udp -m comment --comment \"tsunagi:tsun0\" -j ACCEPT +-A INPUT -i tsun0 -m comment --comment tsunagi:tsun0 -j ACCEPT +-A INPUT -i tsun1 -p udp -m comment --comment tsunagi:tsun1 -j ACCEPT +-A INPUT -p tcp --dport 22 -m comment --comment \"allow ssh tsunagi:tsun0\" -j ACCEPT +-A FORWARD -i tsun0 -m comment --comment tsunagi:tsun0 -j ACCEPT +-A INPUT -p tcp --dport 22 -j ACCEPT +"; + let deletions = tagged_deletions(listing, "tsunagi:tsun0"); + assert_eq!(deletions.len(), 2, "{deletions:?}"); + assert_eq!( + words(&deletions[0]), + "-w 5 -D INPUT -s 10.13.37.0/24 -i tsun0 -p udp -m comment --comment tsunagi:tsun0 -j ACCEPT" + ); + assert!(words(&deletions[1]).contains("-D INPUT -i tsun0 -m comment")); + assert!(tagged_deletions("", "tsunagi:tsun0").is_empty()); + } + + /// A scripted `iptables`: answers by the leading verb, records every call. + struct Fake { + calls: RefCell>, + check_ok: bool, + listing: String, + insert_stderr: Option<&'static str>, + missing: bool, + } + + impl Fake { + fn new(check_ok: bool, listing: &str) -> Self { + Self { + calls: RefCell::new(Vec::new()), + check_ok, + listing: listing.to_string(), + insert_stderr: None, + missing: false, + } + } + fn run(&self, args: &[String]) -> Ran { + self.calls.borrow_mut().push(words(args)); + if self.missing { + return Ran::Missing; + } + let ok = |stdout: &str| Ran::Exited { + success: true, + stdout: stdout.to_string(), + stderr: String::new(), + }; + match args.get(2).map(String::as_str) { + Some("-C") if self.check_ok => ok(""), + Some("-C") => Ran::Exited { + success: false, + stdout: String::new(), + stderr: "Bad rule".into(), + }, + Some("-S") => ok(&self.listing), + Some("-I") => match self.insert_stderr { + Some(stderr) => Ran::Exited { + success: false, + stdout: String::new(), + stderr: stderr.into(), + }, + None => ok(""), + }, + _ => ok(""), + } + } + fn verbs(&self) -> Vec { + self.calls + .borrow() + .iter() + .map(|call| call.split(' ').nth(2).unwrap().to_string()) + .collect() + } + } + + const ONE_OURS: &str = + "-A INPUT -s 10.13.37.0/24 -i tsun0 -p udp -m comment --comment tsunagi:tsun0 -j ACCEPT\n"; + + #[test] + fn an_identical_rule_already_in_place_is_left_alone() { + let fake = Fake::new(true, ONE_OURS); + assert_eq!( + apply_firewall(&|a| fake.run(a), &plan()), + RuleOutcome::Applied + ); + assert_eq!(fake.verbs(), ["-C", "-S"], "no insert, no delete"); + } + + #[test] + fn a_missing_rule_is_inserted_once() { + let fake = Fake::new(false, "-P INPUT ACCEPT\n"); + assert_eq!( + apply_firewall(&|a| fake.run(a), &plan()), + RuleOutcome::Applied + ); + assert_eq!(fake.verbs(), ["-C", "-S", "-I"]); + } + + #[test] + fn a_stale_rule_for_another_range_is_replaced_not_stacked() { + // A crashed run left a rule for an old range under our tag. + let stale = "-A INPUT -s 10.99.0.0/16 -i tsun0 -p udp -m comment --comment tsunagi:tsun0 -j ACCEPT\n"; + let fake = Fake::new(false, stale); + assert_eq!( + apply_firewall(&|a| fake.run(a), &plan()), + RuleOutcome::Applied + ); + assert_eq!(fake.verbs(), ["-C", "-S", "-D", "-I"]); + } + + #[test] + fn duplicates_of_the_rule_collapse_to_one() { + let doubled = format!("{ONE_OURS}{ONE_OURS}"); + let fake = Fake::new(true, &doubled); + assert_eq!( + apply_firewall(&|a| fake.run(a), &plan()), + RuleOutcome::Applied + ); + assert_eq!(fake.verbs(), ["-C", "-S", "-D", "-D", "-I"]); + } + + #[test] + fn a_missing_iptables_says_what_to_install_and_what_to_allow() { + let mut fake = Fake::new(false, ""); + fake.missing = true; + let RuleOutcome::Failed(reason) = apply_firewall(&|a| fake.run(a), &plan()) else { + panic!("expected a failure"); + }; + assert!(reason.contains("iptables not found"), "{reason}"); + assert!(reason.contains("10.13.37.0/24"), "{reason}"); + assert!(reason.contains("tsun0"), "{reason}"); + } + + #[test] + fn a_refused_insert_is_reported_with_the_cause() { + let mut fake = Fake::new(false, ""); + fake.insert_stderr = Some( + "iptables v1.8.10: can't initialize iptables table `filter': Permission denied (you must be root)", + ); + let RuleOutcome::Failed(reason) = apply_firewall(&|a| fake.run(a), &plan()) else { + panic!("expected a failure"); + }; + assert!(reason.starts_with("iptables needs root"), "{reason}"); + } + + #[test] + fn clearing_removes_only_tagged_rules_and_ignores_a_missing_iptables() { + let listing = format!( + "-A INPUT -p tcp --dport 22 -j ACCEPT\n{ONE_OURS}-A INPUT -i tsun1 -m comment --comment tsunagi:tsun1 -j ACCEPT\n" + ); + let fake = Fake::new(true, &listing); + clear_firewall(&|a| fake.run(a), "tsun0"); + assert_eq!(fake.verbs(), ["-S", "-D"]); + + let mut gone = Fake::new(true, ""); + gone.missing = true; + clear_firewall(&|a| gone.run(a), "tsun0"); + assert_eq!(gone.verbs(), ["-S"]); + } + + #[test] + fn rule_listing_words_honour_quotes() { + assert_eq!( + split_rule_line("-A INPUT -m comment --comment \"a b\" -j ACCEPT"), + [ + "-A", + "INPUT", + "-m", + "comment", + "--comment", + "a b", + "-j", + "ACCEPT" + ] + ); + assert!(split_rule_line(" ").is_empty()); + } +} diff --git a/crates/tsunagi/src/overlay/hostrules/mod.rs b/crates/tsunagi/src/overlay/hostrules/mod.rs new file mode 100644 index 0000000..6d453b5 --- /dev/null +++ b/crates/tsunagi/src/overlay/hostrules/mod.rs @@ -0,0 +1,325 @@ +//! Host rules that let a game's LAN discovery reach the overlay interface. +//! +//! Broadcast fanout happens at local TUN ingress (see [`super::broadcast`]), +//! so a limited broadcast only reaches Tsunagi if the operating system routes +//! it through the overlay interface, and a discovery *reply* only reaches the +//! game if the host firewall lets it in. Left to the user that is two manual +//! steps per host that nobody remembers, so when a network's broadcast is on +//! the agent installs exactly these two things for the interface it owns: +//! +//! * a `255.255.255.255/32` route through the overlay interface, with the +//! overlay address as its preferred source, so the game's discovery packet +//! leaves with a source address the router recognises; +//! * an inbound firewall allowance for **UDP only**, from the overlay range +//! only, on that interface only. Replies to a broadcast come from a +//! different address than the one the request was sent to, so a stateful +//! firewall never matches them to the request. Opening every protocol +//! would expose services such as SSH to every member of the network, which +//! is not what a game needs. +//! +//! # Why this is allowed here +//! +//! The architecture says plugins never touch routing or firewall settings, +//! and nothing a peer announces becomes an OS setting. This is neither: it is +//! the system level, which owns the interface, configuring that interface from +//! values it computed itself (the interface name is derived locally, the +//! address and range come from signed state it already verified). Every object +//! is tagged `tsunagi:`, reversible, and removed when broadcast is +//! turned off, the network leaves, or the agent stops. A crash can leave a +//! firewall rule behind; the next apply for the same interface removes it. +//! +//! # Failure is contained +//! +//! Neither step can stop the agent. Each reports its own outcome, so a missing +//! firewall tool still leaves the route in place and says what is missing. +//! +//! # Why a trait +//! +//! Linux, Windows and the tests do this very differently, while *what* to +//! install is the same everywhere and lives in [`BroadcastRulesPlan`] and +//! [`rules_summary`], tested on every platform. + +use std::net::Ipv4Addr; +use std::sync::{Arc, Mutex}; + +use crate::BoxFuture; +use crate::state::Ipv4Range; + +#[cfg(all(feature = "tun-device", target_os = "linux"))] +mod linux; +#[cfg(all(feature = "tun-device", target_os = "linux"))] +pub use linux::LinuxHostRules; + +// Compiled everywhere so the scripts it builds are tested everywhere; only +// the type that runs them is exported on Windows. +mod windows; +#[cfg(all(feature = "tun-device", target_os = "windows"))] +pub use windows::WindowsHostRules; + +/// What the host should be told for one interface. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct BroadcastRulesPlan { + /// The overlay interface. Derived locally from the network id, never from + /// anything a peer said. + pub interface: String, + /// This agent's overlay address, the route's preferred source. + pub source: Ipv4Addr, + /// The overlay range inbound UDP is admitted from. + pub range: Ipv4Range, +} + +impl BroadcastRulesPlan { + /// The tag every object carries, so it can be found again and nothing + /// else is ever touched. + pub fn tag(&self) -> String { + tag_for(&self.interface) + } +} + +/// The tag for an interface's objects. +pub fn tag_for(interface: &str) -> String { + format!("tsunagi:{interface}") +} + +/// What happened to one of the two host objects. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum RuleOutcome { + /// It is in place. + Applied, + /// It is not, with a reason a person can act on. + Failed(String), +} + +impl RuleOutcome { + /// Whether the object is in place. + pub fn is_applied(&self) -> bool { + matches!(self, Self::Applied) + } +} + +/// What applying a plan achieved. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct BroadcastRulesReport { + /// What was asked for. + pub plan: BroadcastRulesPlan, + /// The `255.255.255.255` route. + pub route: RuleOutcome, + /// The inbound firewall allowance. + pub firewall: RuleOutcome, +} + +impl BroadcastRulesReport { + /// Whether both objects are in place. + pub fn is_ok(&self) -> bool { + self.route.is_applied() && self.firewall.is_applied() + } + + /// One line for `status`. + pub fn summary(&self) -> String { + rules_summary(&self.route, &self.firewall) + } +} + +/// The status line for a pair of outcomes. +pub fn rules_summary(route: &RuleOutcome, firewall: &RuleOutcome) -> String { + match (route, firewall) { + (RuleOutcome::Applied, RuleOutcome::Applied) => "ok (route + firewall)".to_string(), + (route, firewall) => { + let part = |what: &str, outcome: &RuleOutcome| match outcome { + RuleOutcome::Applied => format!("{what} ok"), + RuleOutcome::Failed(reason) => format!("{what} failed: {reason}"), + }; + format!( + "incomplete; {}; {}", + part("route", route), + part("firewall", firewall) + ) + } + } +} + +/// Installs and removes the host rules for an overlay interface. +/// +/// Both calls are idempotent. Neither can fail as a whole: a step that cannot +/// be done is reported in the [`BroadcastRulesReport`], because the caller's +/// only useful response is to say so. +pub trait BroadcastHostRules: Send + Sync + std::fmt::Debug + 'static { + /// A short name used in diagnostics. + fn name(&self) -> &str; + + /// Makes the host match the plan, replacing whatever this agent put there + /// for the same interface before. + fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport>; + + /// Removes what this agent put there for the interface. Succeeds when + /// nothing is there: it runs on the shutdown path. + fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()>; +} + +/// The platform has no implementation, which is said rather than pretended. +#[derive(Debug, Default)] +pub struct UnsupportedHostRules; + +impl BroadcastHostRules for UnsupportedHostRules { + fn name(&self) -> &str { + "unsupported" + } + + fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> { + Box::pin(async move { + let reason = "host rules are not implemented on this platform; route \ + 255.255.255.255 through the overlay interface and allow inbound \ + UDP from the overlay range yourself" + .to_string(); + BroadcastRulesReport { + plan: plan.clone(), + route: RuleOutcome::Failed(reason.clone()), + firewall: RuleOutcome::Failed(reason), + } + }) + } + + fn clear<'a>(&'a self, _interface: &'a str) -> BoxFuture<'a, ()> { + Box::pin(async move {}) + } +} + +/// What a [`MockHostRules`] currently holds, per interface. +pub type MockRulesState = std::collections::BTreeMap; + +/// An in-memory host, for tests: nothing is touched. +#[derive(Debug, Clone, Default)] +pub struct MockHostRules { + state: Arc>, + calls: Arc>>, + firewall_failure: Arc>>, +} + +impl MockHostRules { + /// An empty host. + pub fn new() -> Self { + Self::default() + } + + /// Makes the firewall step fail with this reason from now on. + pub fn fail_firewall(&self, reason: impl Into) { + *lock(&self.firewall_failure) = Some(reason.into()); + } + + /// What is installed now. + pub fn installed(&self) -> MockRulesState { + lock(&self.state).clone() + } + + /// Every `apply:` / `clear:` call, in order. + pub fn calls(&self) -> Vec { + lock(&self.calls).clone() + } +} + +fn lock(mutex: &Mutex) -> std::sync::MutexGuard<'_, T> { + match mutex.lock() { + Ok(guard) => guard, + Err(poisoned) => poisoned.into_inner(), + } +} + +impl BroadcastHostRules for MockHostRules { + fn name(&self) -> &str { + "mock" + } + + fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> { + Box::pin(async move { + lock(&self.calls).push(format!("apply:{}", plan.interface)); + lock(&self.state).insert(plan.interface.clone(), plan.clone()); + let firewall = match lock(&self.firewall_failure).clone() { + Some(reason) => RuleOutcome::Failed(reason), + None => RuleOutcome::Applied, + }; + BroadcastRulesReport { + plan: plan.clone(), + route: RuleOutcome::Applied, + firewall, + } + }) + } + + fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> { + Box::pin(async move { + lock(&self.calls).push(format!("clear:{interface}")); + lock(&self.state).remove(interface); + }) + } +} + +#[cfg(test)] +mod tests { + #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)] + + use super::*; + + fn plan() -> BroadcastRulesPlan { + BroadcastRulesPlan { + interface: "tsun0".into(), + source: Ipv4Addr::new(10, 13, 37, 142), + range: "10.13.37.0/24".parse().unwrap(), + } + } + + #[test] + fn every_object_is_tagged_with_its_interface() { + assert_eq!(plan().tag(), "tsunagi:tsun0"); + assert_eq!(tag_for("tsun7"), "tsunagi:tsun7"); + } + + #[test] + fn the_summary_says_which_half_is_missing() { + assert_eq!( + rules_summary(&RuleOutcome::Applied, &RuleOutcome::Applied), + "ok (route + firewall)" + ); + let line = rules_summary( + &RuleOutcome::Applied, + &RuleOutcome::Failed("iptables not found".into()), + ); + assert!(line.contains("route ok"), "{line}"); + assert!( + line.contains("firewall failed: iptables not found"), + "{line}" + ); + let report = BroadcastRulesReport { + plan: plan(), + route: RuleOutcome::Failed("denied".into()), + firewall: RuleOutcome::Applied, + }; + assert!(!report.is_ok()); + assert!(report.summary().contains("route failed: denied")); + } + + #[tokio::test] + async fn the_mock_applies_replaces_and_clears_idempotently() { + let rules = MockHostRules::new(); + let first = rules.apply(&plan()).await; + assert!(first.is_ok()); + assert_eq!(rules.installed().len(), 1); + + let mut moved = plan(); + moved.source = Ipv4Addr::new(10, 13, 37, 7); + rules.apply(&moved).await; + assert_eq!(rules.installed()["tsun0"].source, moved.source); + + rules.clear("tsun0").await; + rules.clear("tsun0").await; + assert!(rules.installed().is_empty()); + assert_eq!(rules.calls().len(), 4); + } + + #[tokio::test] + async fn an_unsupported_platform_reports_both_steps_failed() { + let report = UnsupportedHostRules.apply(&plan()).await; + assert!(!report.route.is_applied()); + assert!(!report.firewall.is_applied()); + UnsupportedHostRules.clear("tsun0").await; + } +} diff --git a/crates/tsunagi/src/overlay/hostrules/windows.rs b/crates/tsunagi/src/overlay/hostrules/windows.rs new file mode 100644 index 0000000..522a238 --- /dev/null +++ b/crates/tsunagi/src/overlay/hostrules/windows.rs @@ -0,0 +1,272 @@ +//! Broadcast host rules on Windows: a route and a Defender Firewall rule. +//! +//! Both are written with the `NetTCPIP` and `NetSecurity` PowerShell modules, +//! run by absolute path from `%SystemRoot%` like the DNS publisher does: the +//! in-process alternatives are `unsafe` IP Helper calls this crate forbids. +//! +//! * The route is `255.255.255.255/32` on the overlay interface, on-link, with +//! metric 1, written to the active store only so it cannot outlive a reboot. +//! Windows picks the interface's own address as the source of a packet sent +//! along it; with several networks on one interface that is whichever +//! address Windows treats as primary, which this cannot choose. +//! * The firewall rule allows inbound UDP on that interface alias from the +//! overlay range only, in every profile. Wintun adapters are usually +//! classified as a public network, where inbound UDP is otherwise dropped. +//! Its display name is the tag `tsunagi:`, which is how it is +//! found again, replaced and removed. +//! +//! Every value in a script is one this agent produced: the interface name is +//! derived from the network id and the range comes from signed state. They are +//! still single-quoted and escaped, so nothing could be read as PowerShell +//! rather than as data. Changing routes and firewall rules needs an elevated +//! process, which is reported as its own cause. +//! +//! This module is compiled everywhere so its script building is tested +//! everywhere. It could not be run against a real Windows host while it was +//! written. + +#![cfg_attr(not(target_os = "windows"), allow(dead_code))] + +use crate::BoxFuture; + +use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome, tag_for}; + +/// A PowerShell single-quoted literal, with any embedded quote doubled. +fn ps_literal(value: &str) -> String { + format!("'{}'", value.replace('\'', "''")) +} + +/// The step that makes the route match the plan. +/// +/// A leftover route (from a crashed run, or for an old interface index) is +/// removed first so the result is exactly one route. +fn route_script(plan: &BroadcastRulesPlan) -> String { + let alias = ps_literal(&plan.interface); + format!( + "$ErrorActionPreference = 'Stop'\n\ + $alias = {alias}\n\ + Get-NetRoute -DestinationPrefix '255.255.255.255/32' -InterfaceAlias $alias \ + -PolicyStore ActiveStore -ErrorAction SilentlyContinue | \ + Remove-NetRoute -Confirm:$false -ErrorAction SilentlyContinue\n\ + New-NetRoute -DestinationPrefix '255.255.255.255/32' -InterfaceAlias $alias \ + -NextHop '0.0.0.0' -RouteMetric 1 -PolicyStore ActiveStore | Out-Null\n" + ) +} + +/// The step that makes the firewall hold exactly one rule for the interface. +fn firewall_script(plan: &BroadcastRulesPlan) -> String { + let alias = ps_literal(&plan.interface); + let tag = ps_literal(&plan.tag()); + let range = ps_literal(&plan.range.to_string()); + format!( + "$ErrorActionPreference = 'Stop'\n\ + $alias = {alias}\n\ + $tag = {tag}\n\ + Get-NetFirewallRule -DisplayName $tag -ErrorAction SilentlyContinue | \ + Remove-NetFirewallRule -ErrorAction SilentlyContinue\n\ + New-NetFirewallRule -DisplayName $tag -Direction Inbound -Action Allow -Protocol UDP \ + -InterfaceAlias $alias -RemoteAddress {range} -Profile Any | Out-Null\n" + ) +} + +/// The script that removes both, ignoring what is not there. +fn clear_script(interface: &str) -> String { + let alias = ps_literal(interface); + let tag = ps_literal(&tag_for(interface)); + format!( + "$ErrorActionPreference = 'SilentlyContinue'\n\ + Get-NetRoute -DestinationPrefix '255.255.255.255/32' -InterfaceAlias {alias} \ + -PolicyStore ActiveStore | Remove-NetRoute -Confirm:$false\n\ + Get-NetFirewallRule -DisplayName {tag} | Remove-NetFirewallRule\n\ + exit 0\n" + ) +} + +/// Turns what PowerShell said into a cause a person can act on. +fn classify(what: &str, text: &str) -> String { + let lower = text.to_ascii_lowercase(); + if lower.contains("access is denied") + || lower.contains("requires elevation") + || lower.contains("run as administrator") + || lower.contains("permissiondenied") + { + format!("{what}: Windows refused; run tsunagi from an elevated (administrator) process") + } else if lower.contains("is not recognized") || lower.contains("commandnotfound") { + format!("{what}: the NetTCPIP/NetSecurity PowerShell modules are not available: {text}") + } else if text.is_empty() { + format!("{what}: failed without saying why") + } else { + format!("{what}: {text}") + } +} + +/// The absolute path to Windows PowerShell, so nothing on `PATH` can stand in +/// for it. +fn powershell_path() -> std::path::PathBuf { + let root = std::env::var_os("SystemRoot").unwrap_or_else(|| r"C:\Windows".into()); + std::path::Path::new(&root) + .join("System32") + .join("WindowsPowerShell") + .join("v1.0") + .join("powershell.exe") +} + +/// Runs a script and returns what it said on failure. +fn run_powershell(script: &str) -> Result<(), String> { + let output = std::process::Command::new(powershell_path()) + .args([ + "-NoProfile", + "-NonInteractive", + "-ExecutionPolicy", + "Bypass", + "-Command", + script, + ]) + .output() + .map_err(|err| format!("could not run PowerShell: {err}"))?; + if output.status.success() { + return Ok(()); + } + let stderr = String::from_utf8_lossy(&output.stderr); + let text = if stderr.trim().is_empty() { + String::from_utf8_lossy(&output.stdout).trim().to_string() + } else { + stderr.trim().to_string() + }; + Err(text) +} + +async fn step(what: &'static str, script: String) -> RuleOutcome { + match tokio::task::spawn_blocking(move || run_powershell(&script)).await { + Ok(Ok(())) => RuleOutcome::Applied, + Ok(Err(text)) => RuleOutcome::Failed(classify(what, &text)), + Err(err) => RuleOutcome::Failed(format!("{what}: the task failed: {err}")), + } +} + +/// The Windows implementation, through PowerShell. +#[derive(Debug, Default)] +pub struct WindowsHostRules; + +impl WindowsHostRules { + /// Creates it. Nothing is run until [`BroadcastHostRules::apply`]. + pub fn new() -> Self { + Self + } +} + +impl BroadcastHostRules for WindowsHostRules { + fn name(&self) -> &str { + "windows-netsecurity" + } + + fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> { + Box::pin(async move { + let route = step("route", route_script(plan)).await; + let firewall = step("firewall", firewall_script(plan)).await; + BroadcastRulesReport { + plan: plan.clone(), + route, + firewall, + } + }) + } + + fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> { + Box::pin(async move { + if let RuleOutcome::Failed(reason) = step("clear", clear_script(interface)).await { + tracing::warn!( + interface, + "cannot remove the broadcast host rules: {reason}" + ); + } + }) + } +} + +#[cfg(test)] +mod tests { + #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)] + + use super::*; + use std::net::Ipv4Addr; + + fn plan() -> BroadcastRulesPlan { + BroadcastRulesPlan { + interface: "tsun0".into(), + source: Ipv4Addr::new(10, 13, 37, 142), + range: "10.13.37.0/24".parse().unwrap(), + } + } + + #[test] + fn a_quote_in_a_value_cannot_escape_the_literal() { + assert_eq!(ps_literal("a'b"), "'a''b'"); + let mut hostile = plan(); + hostile.interface = "x'; Remove-Item C:\\ #".into(); + let script = route_script(&hostile); + assert!( + script.contains("$alias = 'x''; Remove-Item C:\\ #'"), + "{script}" + ); + assert!(firewall_script(&hostile).contains("'x''; Remove-Item C:\\ #'")); + } + + #[test] + fn the_route_is_on_link_metric_one_in_the_active_store_and_replaces_its_own() { + let script = route_script(&plan()); + assert!(script.contains("'255.255.255.255/32'")); + assert!(script.contains("-NextHop '0.0.0.0'")); + assert!(script.contains("-RouteMetric 1")); + assert!(script.contains("-PolicyStore ActiveStore")); + assert!( + script.find("Remove-NetRoute").unwrap() < script.find("New-NetRoute").unwrap(), + "a stale route goes before the new one:\n{script}" + ); + } + + #[test] + fn the_firewall_rule_is_udp_only_from_the_overlay_range_on_this_interface() { + let script = firewall_script(&plan()); + assert!(script.contains("-Protocol UDP")); + assert!(script.contains("-Direction Inbound -Action Allow")); + assert!(script.contains("-InterfaceAlias $alias")); + assert!(script.contains("-RemoteAddress '10.13.37.0/24'")); + assert!(script.contains("$tag = 'tsunagi:tsun0'")); + assert!(!script.contains("-Protocol Any") && !script.contains("-Protocol TCP")); + assert!( + script.find("Remove-NetFirewallRule").unwrap() + < script.find("New-NetFirewallRule").unwrap(), + "{script}" + ); + } + + #[test] + fn clearing_touches_only_our_route_and_our_tagged_rule() { + let script = clear_script("tsun0"); + assert!(script.contains("-InterfaceAlias 'tsun0'")); + assert!(script.contains("Get-NetFirewallRule -DisplayName 'tsunagi:tsun0'")); + assert!(script.contains("exit 0")); + } + + #[test] + fn a_refusal_names_elevation_and_a_missing_module_names_the_module() { + assert!(classify("route", "Access is denied").contains("elevated")); + assert!( + classify( + "firewall", + "The term 'New-NetFirewallRule' is not recognized" + ) + .contains("modules are not available") + ); + assert_eq!(classify("route", "boom"), "route: boom"); + assert!(classify("route", "").contains("without saying why")); + } + + #[test] + fn powershell_is_found_by_absolute_path() { + let path = powershell_path(); + assert!(path.ends_with("powershell.exe")); + assert!(path.to_string_lossy().contains("System32"), "{path:?}"); + } +} diff --git a/crates/tsunagi/src/overlay/interface.rs b/crates/tsunagi/src/overlay/interface.rs index 3ae5f49..39b21a9 100644 --- a/crates/tsunagi/src/overlay/interface.rs +++ b/crates/tsunagi/src/overlay/interface.rs @@ -29,6 +29,7 @@ use tokio::task::JoinHandle; use crate::identity::NetworkId; +use super::hostrules::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport}; use super::packet::IpHeader; use super::router::{Route, RoutingTable}; use super::tun::{TunDevice, TunFactory, TunRequest}; @@ -153,6 +154,10 @@ pub struct Interface { applied: std::sync::Mutex>, task: std::sync::Mutex>>, sync_lock: Arc>, + /// The host rules for this interface, when the factory manages the host. + host_rules: Option>, + /// What those rules last reported, `None` while none are installed. + rules_report: std::sync::Mutex>, } impl Interface { @@ -165,6 +170,7 @@ impl Interface { carrier: Arc, ) -> Result { let name = name.into(); + let host_rules = factory.host_rules(); let device = factory.create(TunRequest::bare(name.clone(), mtu)).await?; let tally = Arc::new(Tally::default()); @@ -230,9 +236,81 @@ impl Interface { applied: std::sync::Mutex::new(Vec::new()), task: std::sync::Mutex::new(Some(task)), sync_lock: Arc::new(tokio::sync::Mutex::new(())), + host_rules, + rules_report: std::sync::Mutex::new(None), }) } + /// Brings the broadcast route and firewall allowance in line with the + /// routing table: installed while some network has broadcast on and an + /// address, removed when none does. + /// + /// Called after the addresses are synced, because the route's preferred + /// source has to be on the interface already. A plan that has not changed + /// is not re-applied, so this is cheap to call whenever the table moves. + /// It never fails: the outcome is kept for [`Self::broadcast_rules`]. + pub async fn sync_broadcast_rules(&self) { + let Some(rules) = &self.host_rules else { + return; + }; + let _guard = self.sync_lock.lock().await; + let desired = + self.routes + .broadcast_host_target() + .map(|(source, range)| BroadcastRulesPlan { + interface: self.name.clone(), + source, + range, + }); + let current = self.current_rules_report(); + match (desired, current) { + (None, None) => {} + (None, Some(_)) => { + rules.clear(&self.name).await; + self.store_rules_report(None); + tracing::info!(interface = %self.name, "broadcast host rules removed"); + } + (Some(plan), Some(report)) if report.plan == plan => {} + (Some(plan), _) => { + let report = rules.apply(&plan).await; + if report.is_ok() { + tracing::info!( + interface = %self.name, + source = %plan.source, + "broadcast host rules applied" + ); + } else { + tracing::warn!( + interface = %self.name, + "broadcast host rules are incomplete: {}", + report.summary() + ); + } + self.store_rules_report(Some(report)); + } + } + } + + /// What the broadcast host rules achieved, `None` when none are installed + /// or this interface has no host to configure. + pub fn broadcast_rules(&self) -> Option { + self.current_rules_report() + } + + fn current_rules_report(&self) -> Option { + match self.rules_report.lock() { + Ok(guard) => guard.clone(), + Err(poisoned) => poisoned.into_inner().clone(), + } + } + + fn store_rules_report(&self, report: Option) { + match self.rules_report.lock() { + Ok(mut guard) => *guard = report, + Err(poisoned) => *poisoned.into_inner() = report, + } + } + /// Brings the addresses on the host in line with the routing table. /// /// Called whenever a network agrees a different address for this agent. @@ -289,6 +367,12 @@ impl Interface { /// Removes the interface from the host. pub async fn remove(&self) { + // The route goes with the interface; the firewall rule does not, so + // it is taken out explicitly while the interface is still known. + if let Some(rules) = &self.host_rules { + rules.clear(&self.name).await; + self.store_rules_report(None); + } // The packet loop holds the device open, and it ends only when the // device reports end of stream — which a real interface never does // while it exists. Stop it directly before destroying the interface. @@ -431,6 +515,9 @@ mod tests { interface.remove().await; } use crate::identity::{NetworkKeys, NetworkName, NetworkSecret}; + use crate::overlay::broadcast::BroadcastPolicy; + use crate::overlay::hostrules::{MockHostRules, RuleOutcome}; + use crate::overlay::provision::{ManagedTunFactory, MockProvisioner}; use crate::overlay::router::NetworkRoutes; use crate::overlay::tun::{MemoryTun, MemoryTunFactory}; @@ -751,4 +838,137 @@ mod tests { vec![Cidr::new(addr(9).into(), 24).unwrap()] ); } + + async fn managed_interface( + rules: &MockHostRules, + broadcast_enabled: bool, + ) -> (Interface, Arc, NetworkId) { + let id = network("rules"); + let routes = Arc::new(RoutingTable::new()); + routes + .set_network( + id, + NetworkRoutes { + broadcast: BroadcastPolicy { + enabled: broadcast_enabled, + ..Default::default() + }, + range: Some("10.13.37.0/24".parse().unwrap()), + local: Some(addr(1)), + peers: vec![(addr(2), peer(2))], + }, + ) + .unwrap(); + let factory = Arc::new( + ManagedTunFactory::new(Arc::new(MockProvisioner::default())) + .with_host_rules(Arc::new(rules.clone())), + ); + let interface = Interface::start( + factory as Arc, + "tsunrules", + 1280, + Arc::clone(&routes), + Arc::new(Recorder::default()) as Arc, + ) + .await + .unwrap(); + (interface, routes, id) + } + + #[tokio::test] + async fn host_rules_follow_broadcast_policy_address_and_interface_lifetime() { + let rules = MockHostRules::new(); + let (interface, routes, id) = managed_interface(&rules, true).await; + assert!( + interface.broadcast_rules().is_none(), + "nothing before a sync" + ); + + interface.sync_broadcast_rules().await; + let installed = rules.installed(); + assert_eq!(installed["tsunrules"].source, addr(1)); + assert_eq!( + installed["tsunrules"].range, + "10.13.37.0/24".parse().unwrap() + ); + assert!(interface.broadcast_rules().unwrap().is_ok()); + + // Unchanged: not applied a second time. + interface.sync_broadcast_rules().await; + interface.sync_broadcast_rules().await; + assert_eq!(rules.calls(), ["apply:tsunrules"]); + + // A new address replaces the plan, it does not stack a second one. + routes + .set_network( + id, + NetworkRoutes { + broadcast: BroadcastPolicy::default(), + range: Some("10.13.37.0/24".parse().unwrap()), + local: Some(addr(9)), + peers: vec![(addr(2), peer(2))], + }, + ) + .unwrap(); + interface.sync_broadcast_rules().await; + assert_eq!(rules.installed()["tsunrules"].source, addr(9)); + assert_eq!(rules.installed().len(), 1); + + // Turning broadcast off takes the rules away. + routes.set_broadcast( + id, + BroadcastPolicy { + enabled: false, + ..Default::default() + }, + ); + interface.sync_broadcast_rules().await; + assert!(rules.installed().is_empty()); + assert!(interface.broadcast_rules().is_none()); + + // And on again puts them back; leaving the agent removes them once more. + routes.set_broadcast(id, BroadcastPolicy::default()); + interface.sync_broadcast_rules().await; + assert_eq!(rules.installed().len(), 1); + interface.remove().await; + assert!(rules.installed().is_empty(), "shutdown clears the host"); + assert!(interface.broadcast_rules().is_none()); + } + + #[tokio::test] + async fn host_rules_are_not_installed_while_broadcast_is_off() { + let rules = MockHostRules::new(); + let (interface, _routes, _id) = managed_interface(&rules, false).await; + interface.sync_broadcast_rules().await; + assert!(rules.installed().is_empty()); + assert!(rules.calls().is_empty(), "the host was not even asked"); + interface.remove().await; + } + + #[tokio::test] + async fn a_failed_firewall_step_is_reported_and_does_not_undo_the_route() { + let rules = MockHostRules::new(); + rules.fail_firewall("iptables not found"); + let (interface, _routes, _id) = managed_interface(&rules, true).await; + interface.sync_broadcast_rules().await; + let report = interface.broadcast_rules().unwrap(); + assert!(report.route.is_applied()); + assert_eq!( + report.firewall, + RuleOutcome::Failed("iptables not found".into()) + ); + assert!(!report.is_ok()); + // Not retried on every sync: the plan is the same. + interface.sync_broadcast_rules().await; + assert_eq!(rules.calls(), ["apply:tsunrules"]); + interface.remove().await; + } + + #[tokio::test] + async fn an_interface_without_a_host_has_no_host_rules() { + let (interface, _device, _routes, _carrier, _id) = interface(false).await; + interface.sync_broadcast_rules().await; + assert!(interface.broadcast_rules().is_none()); + interface.remove().await; + } } diff --git a/crates/tsunagi/src/overlay/mod.rs b/crates/tsunagi/src/overlay/mod.rs index e0f33f9..3282881 100644 --- a/crates/tsunagi/src/overlay/mod.rs +++ b/crates/tsunagi/src/overlay/mod.rs @@ -11,6 +11,8 @@ //! //! * [`provision`] creates the interface and configures it, and removes it //! again. Platform mechanics behind one decision function. +//! * [`hostrules`] installs the route and firewall allowance that let a +//! game's LAN discovery reach the interface, while broadcast is on. //! * [`tun`] is the packet interface itself, real or in memory. //! * [`packet`] reads just enough of an IP header to route by it. //! * [`config`] names interfaces and describes addresses. @@ -38,6 +40,7 @@ pub enum OverlayError { pub mod broadcast; pub mod config; +pub mod hostrules; pub mod interface; pub mod packet; pub mod provision; @@ -45,6 +48,10 @@ pub mod router; pub mod tun; pub use config::{Cidr, DEFAULT_INTERFACE_PREFIX, MAX_INTERFACE_NAME_LEN, interface_name}; +pub use hostrules::{ + BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, MockHostRules, RuleOutcome, + UnsupportedHostRules, +}; pub use interface::{Counters, Interface, PacketCarrier, Rejected}; pub use packet::IpHeader; pub use provision::{ @@ -55,6 +62,10 @@ pub use provision::{ pub use router::{NetworkRoutes, Route, RouteError, RoutingTable}; pub use tun::{MemoryTun, MemoryTunFactory, TunDevice, TunFactory, TunRequest, address_is_local}; +#[cfg(all(feature = "tun-device", target_os = "linux"))] +pub use hostrules::LinuxHostRules; +#[cfg(all(feature = "tun-device", target_os = "windows"))] +pub use hostrules::WindowsHostRules; #[cfg(all(feature = "tun-device", target_os = "linux"))] pub use provision::NetlinkProvisioner; diff --git a/crates/tsunagi/src/overlay/provision/factory.rs b/crates/tsunagi/src/overlay/provision/factory.rs index 3771035..1a3e45c 100644 --- a/crates/tsunagi/src/overlay/provision/factory.rs +++ b/crates/tsunagi/src/overlay/provision/factory.rs @@ -11,6 +11,7 @@ use std::sync::Arc; use crate::BoxFuture; use crate::overlay::OverlayError; +use super::super::hostrules::BroadcastHostRules; use super::super::tun::{TunDevice, TunFactory, TunRequest}; use super::{InterfacePlan, InterfaceProvisioner}; @@ -30,12 +31,23 @@ fn plan_for(request: &TunRequest) -> Result { #[derive(Debug)] pub struct ManagedTunFactory { provisioner: Arc, + host_rules: Option>, } impl ManagedTunFactory { /// Wraps a provisioner. pub fn new(provisioner: Arc) -> Self { - Self { provisioner } + Self { + provisioner, + host_rules: None, + } + } + + /// Lets the agent install the broadcast route and firewall allowance for + /// the interface this factory manages. + pub fn with_host_rules(mut self, rules: Arc) -> Self { + self.host_rules = Some(rules); + self } /// The provisioner underneath. @@ -54,6 +66,10 @@ impl TunFactory for ManagedTunFactory { true } + fn host_rules(&self) -> Option> { + self.host_rules.clone() + } + fn create<'a>( &'a self, request: TunRequest, diff --git a/crates/tsunagi/src/overlay/provision/linux.rs b/crates/tsunagi/src/overlay/provision/linux.rs index b50623c..4de2cd1 100644 --- a/crates/tsunagi/src/overlay/provision/linux.rs +++ b/crates/tsunagi/src/overlay/provision/linux.rs @@ -39,7 +39,7 @@ //! capability is raised immediately before that call and lowered immediately //! after, and the connection task lives and dies inside it. -use std::net::IpAddr; +use std::net::{IpAddr, Ipv4Addr}; use std::sync::{Arc, Mutex, mpsc}; use futures_util::TryStreamExt; @@ -47,7 +47,8 @@ use futures_util::TryStreamExt; // of step with the client that sends them. use rtnetlink::packet_route::address::{AddressAttribute, AddressMessage}; use rtnetlink::packet_route::link::{InfoKind, LinkAttribute, LinkFlags, LinkInfo, LinkMessage}; -use rtnetlink::{LinkMessageBuilder, LinkUnspec}; +use rtnetlink::packet_route::route::RouteScope; +use rtnetlink::{LinkMessageBuilder, LinkUnspec, RouteMessageBuilder}; use crate::BoxFuture; use crate::overlay::OverlayError; @@ -67,6 +68,10 @@ enum Command { Delete(String, Reply<()>), /// Apply MTU, link state and addresses. Configure(Box, Reply<()>), + /// Route `255.255.255.255/32` through this interface from this address. + SetBroadcastRoute(String, Ipv4Addr, Reply<()>), + /// Remove that route from this interface. + DelBroadcastRoute(String, Reply<()>), /// Stop the thread. Stop, } @@ -133,13 +138,18 @@ impl NetlinkProvisioner { &self, make: impl FnOnce(Reply) -> Command, ) -> Result { - let (reply_tx, reply_rx) = mpsc::channel(); - self.commands - .send(make(reply_tx)) - .map_err(|_| OverlayError::Unavailable("the netlink thread has stopped".to_string()))?; - reply_rx.recv().map_err(|_| { - OverlayError::Unavailable("the netlink thread stopped mid-request".to_string()) - })? + call_thread(&self.commands, make) + } + + /// The broadcast host rules for this host. + /// + /// The route is added by this provisioner's netlink thread, because it is + /// the only one that ever holds `CAP_NET_ADMIN`, and only for the duration + /// of a call. Keep the provisioner alive for as long as the rules are used. + pub fn host_rules(&self) -> crate::overlay::hostrules::LinuxHostRules { + crate::overlay::hostrules::LinuxHostRules::new(RouteHandle { + commands: self.commands.clone(), + }) } fn is_ours(&self, name: &str) -> bool { @@ -157,6 +167,44 @@ impl NetlinkProvisioner { } } +fn call_thread( + commands: &mpsc::Sender, + make: impl FnOnce(Reply) -> Command, +) -> Result { + let (reply_tx, reply_rx) = mpsc::channel(); + commands + .send(make(reply_tx)) + .map_err(|_| OverlayError::Unavailable("the netlink thread has stopped".to_string()))?; + reply_rx.recv().map_err(|_| { + OverlayError::Unavailable("the netlink thread stopped mid-request".to_string()) + })? +} + +/// Adds and removes the limited-broadcast route on the netlink thread. +#[derive(Debug, Clone)] +pub(crate) struct RouteHandle { + commands: mpsc::Sender, +} + +impl RouteHandle { + /// Routes `255.255.255.255/32` through `interface`, preferring `source`. + /// Replaces a route this agent added earlier, so it can be repeated. + pub(crate) fn set(&self, interface: &str, source: Ipv4Addr) -> Result<(), OverlayError> { + let interface = interface.to_string(); + call_thread(&self.commands, |reply| { + Command::SetBroadcastRoute(interface, source, reply) + }) + } + + /// Removes the route. Succeeds when it, or the interface, is gone. + pub(crate) fn clear(&self, interface: &str) -> Result<(), OverlayError> { + let interface = interface.to_string(); + call_thread(&self.commands, |reply| { + Command::DelBroadcastRoute(interface, reply) + }) + } +} + impl Drop for NetlinkProvisioner { fn drop(&mut self) { let _ = self.commands.send(Command::Stop); @@ -273,7 +321,10 @@ fn netlink_thread(requests: mpsc::Receiver) { Command::Observe(_, reply) => { let _ = reply.send(Err(OverlayError::Unavailable(message))); } - Command::Delete(_, reply) | Command::Configure(_, reply) => { + Command::Delete(_, reply) + | Command::Configure(_, reply) + | Command::SetBroadcastRoute(_, _, reply) + | Command::DelBroadcastRoute(_, reply) => { let _ = reply.send(Err(OverlayError::Unavailable(message))); } Command::Stop => return, @@ -306,6 +357,22 @@ fn netlink_thread(requests: mpsc::Receiver) { }); let _ = reply.send(result); } + Command::SetBroadcastRoute(name, source, reply) => { + let result = NetAdmin::acquire().and_then(|guard| { + let result = runtime.block_on(set_broadcast_route(&name, source)); + drop(guard); + result + }); + let _ = reply.send(result); + } + Command::DelBroadcastRoute(name, reply) => { + let result = NetAdmin::acquire().and_then(|guard| { + let result = runtime.block_on(del_broadcast_route(&name)); + drop(guard); + result + }); + let _ = reply.send(result); + } } } } @@ -455,6 +522,80 @@ async fn delete_link(name: &str) -> Result<(), OverlayError> { .await } +/// The message that names the limited-broadcast route on an interface. +/// +/// On-link scope is what the kernel requires of a route with no gateway, and +/// what `ip route add 255.255.255.255/32 dev ` produces. Deleting needs the +/// same scope or the kernel finds no match. +fn broadcast_route_message( + index: u32, + source: Option, +) -> rtnetlink::packet_route::route::RouteMessage { + let mut builder = RouteMessageBuilder::::new() + .destination_prefix(Ipv4Addr::BROADCAST, 32) + .output_interface(index) + .scope(RouteScope::Link); + if let Some(source) = source { + builder = builder.pref_source(source); + } + builder.build() +} + +async fn link_index(handle: &rtnetlink::Handle, name: &str) -> Option { + let mut links = handle.link().get().match_name(name.to_string()).execute(); + links + .try_next() + .await + .ok() + .flatten() + .map(|message| message.header.index) +} + +async fn set_broadcast_route(name: &str, source: Ipv4Addr) -> Result<(), OverlayError> { + with_netlink(|handle| async move { + let index = link_index(&handle, name).await.ok_or_else(|| { + OverlayError::Unavailable(format!( + "interface `{name}` disappeared before its broadcast route could be added" + )) + })?; + handle + .route() + .add(broadcast_route_message(index, Some(source))) + .replace() + .execute() + .await + .map_err(|err| { + OverlayError::Unavailable(format!( + "cannot route 255.255.255.255 through `{name}` from {source}: {err}" + )) + }) + }) + .await +} + +async fn del_broadcast_route(name: &str) -> Result<(), OverlayError> { + with_netlink(|handle| async move { + // An interface that is gone took its routes with it. + let Some(index) = link_index(&handle, name).await else { + return Ok(()); + }; + match handle + .route() + .del(broadcast_route_message(index, None)) + .execute() + .await + { + Ok(()) => Ok(()), + // "No such process": there was no such route, which is the goal. + Err(err) if err.to_string().contains("No such process") => Ok(()), + Err(err) => Err(OverlayError::Unavailable(format!( + "cannot remove the 255.255.255.255 route from `{name}`: {err}" + ))), + } + }) + .await +} + async fn configure_link(configure: &Configure) -> Result<(), OverlayError> { let name = configure.name.as_str(); with_netlink(|handle| async move { diff --git a/crates/tsunagi/src/overlay/provision/mod.rs b/crates/tsunagi/src/overlay/provision/mod.rs index 1cd0769..e520257 100644 --- a/crates/tsunagi/src/overlay/provision/mod.rs +++ b/crates/tsunagi/src/overlay/provision/mod.rs @@ -52,6 +52,8 @@ pub use mock::{MockHost, MockProvisioner}; mod linux; #[cfg(all(feature = "tun-device", target_os = "linux"))] pub use linux::NetlinkProvisioner; +#[cfg(all(feature = "tun-device", target_os = "linux"))] +pub(crate) use linux::RouteHandle; #[cfg(all(feature = "tun-device", target_os = "windows"))] mod windows; diff --git a/crates/tsunagi/src/overlay/router.rs b/crates/tsunagi/src/overlay/router.rs index 302473c..6b882f9 100644 --- a/crates/tsunagi/src/overlay/router.rs +++ b/crates/tsunagi/src/overlay/router.rs @@ -230,6 +230,23 @@ impl RoutingTable { addresses } + /// The one network whose address the limited broadcast route is bound to. + /// + /// The host has a single `255.255.255.255` route per interface, and one + /// interface carries every network, so only one overlay address can be its + /// source. The choice is the lowest [`NetworkId`] among networks that have + /// broadcast enabled, a range and a local address: deterministic, so two + /// runs with the same networks pick the same one. + pub fn broadcast_host_target(&self) -> Option<(Ipv4Addr, Ipv4Range)> { + let networks = self.read(); + networks + .iter() + .filter(|(_, routes)| routes.broadcast.enabled) + .filter_map(|(network, routes)| Some((*network, routes.local?, routes.range?))) + .min_by_key(|(network, _, _)| *network) + .map(|(_, local, range)| (local, range)) + } + /// Whether a range would collide with one another network already uses. /// /// Asked before proposing a range rather than after: with one interface @@ -393,6 +410,65 @@ mod tests { assert_eq!(table.route(addr(2)).map(|route| route.network), Some(first)); } + #[test] + fn the_host_broadcast_target_is_the_lowest_enabled_network_with_an_address() { + let table = RoutingTable::new(); + assert_eq!(table.broadcast_host_target(), None); + + let (low, high) = { + let (a, b) = (network("first"), network("second")); + if a < b { (a, b) } else { (b, a) } + }; + let mk = |enabled: bool, base: [u8; 4], local: Option| NetworkRoutes { + broadcast: BroadcastPolicy { + enabled, + ..Default::default() + }, + range: Some( + format!("{}.{}.{}.0/24", base[0], base[1], base[2]) + .parse() + .unwrap(), + ), + local, + peers: Vec::new(), + }; + let a = Ipv4Addr::new(10, 1, 0, 1); + let b = Ipv4Addr::new(10, 2, 0, 1); + + // The lower id has broadcast off, so the higher one is chosen. + table + .set_network(low, mk(false, [10, 1, 0, 0], Some(a))) + .unwrap(); + table + .set_network(high, mk(true, [10, 2, 0, 0], Some(b))) + .unwrap(); + assert_eq!( + table.broadcast_host_target(), + Some((b, "10.2.0.0/24".parse().unwrap())) + ); + + // Both enabled: the lowest id wins, whatever the insertion order. + table + .set_network(low, mk(true, [10, 1, 0, 0], Some(a))) + .unwrap(); + assert_eq!( + table.broadcast_host_target(), + Some((a, "10.1.0.0/24".parse().unwrap())) + ); + + // No local address yet: it cannot be the route's source. + table + .set_network(low, mk(true, [10, 1, 0, 0], None)) + .unwrap(); + assert_eq!( + table.broadcast_host_target(), + Some((b, "10.2.0.0/24".parse().unwrap())) + ); + + table.remove_network(high); + assert_eq!(table.broadcast_host_target(), None); + } + #[test] fn a_range_can_be_asked_about_before_it_is_proposed() { // The point of asking first: an agent that claims an address it diff --git a/crates/tsunagi/src/overlay/tun.rs b/crates/tsunagi/src/overlay/tun.rs index f10e9a5..3e705e3 100644 --- a/crates/tsunagi/src/overlay/tun.rs +++ b/crates/tsunagi/src/overlay/tun.rs @@ -109,6 +109,14 @@ pub trait TunFactory: Send + Sync + std::fmt::Debug + 'static { Box::pin(async move { Ok(()) }) } + /// The host rules for this factory's interface, when it manages the host. + /// + /// `None` for an in-memory device or an interface somebody else prepared: + /// there is nothing of ours to route or to open in a firewall. + fn host_rules(&self) -> Option> { + None + } + /// Removes an interface this factory created. /// /// Runs on the teardown path, so it reports rather than fails: there is diff --git a/docs/broadcast.md b/docs/broadcast.md index eeca189..b7a3a8c 100644 --- a/docs/broadcast.md +++ b/docs/broadcast.md @@ -61,13 +61,47 @@ routers cannot inspect another pair's encrypted IP payload. ## Host behavior and verification -The game must send discovery through the Tsunagi interface. Traffic bound to a -physical adapter never reaches this TUN, and this feature does not capture it. +A game's discovery packet only reaches Tsunagi if the operating system sends it +through the Tsunagi interface, and its reply only reaches the game if the host +firewall admits it. While a network has broadcast on, the agent installs +exactly two things for the interface it owns, and removes them when broadcast +is turned off, the network is left or stopped, or the agent exits: + +- **Route** `255.255.255.255/32` through the overlay interface, with the + overlay address as its source (Linux: netlink, `ip route add 255.255.255.255/32 + dev tsun0 src `; Windows: `New-NetRoute`, metric 1, active store). + Limited broadcasts from every program then go to the overlay and not to the + physical LAN while this is installed; this is the price of the feature. +- **Firewall allowance** for inbound **UDP only**, from the overlay range only, + on that interface only (Linux: an `iptables -I INPUT` rule tagged + `tsunagi:`; Windows: a Defender rule of that name). Replies to a + broadcast come from another address than the request went to, so a stateful + firewall never matches them to it. TCP services such as SSH stay unexposed. + +Both are tagged, idempotent and reversible, and are never derived from a remote +announcement. A rule left by a crashed run is replaced on the next apply. If +`iptables` is missing (for example a host with only `nft`) or the process is not +privileged, the route is still installed and `status` prints a +`broadcast rules` line saying which half is missing and why. To undo it all, +turn broadcast off for the network: `tsunagi network broadcast off`. + +**One interface, several networks.** The host has one `255.255.255.255` route +per interface, so only one overlay address can be its source. The lowest network +id among broadcast-enabled networks with an address is used; `status` says when +that is not the network being shown. Directed broadcasts (`10.13.37.255`) follow +the ordinary connected route and are not affected. + +**What it cannot fix.** A game that binds its socket to a physical address or +device, or sends to the physical subnet's broadcast address, still bypasses the +TUN. The Windows implementation could not be run while it was written; its +scripts are unit-tested only. + The suite tests real agents, authenticated iroh links and WireGuard encryption with memory TUNs: limited/directed UDP discovery, multihop fanout, single-copy delivery, opt-out, unicast replies, malformed input, source validation and -network isolation. It does not launch CS 1.6 or Warcraft III or establish that -every game/host chooses the virtual adapter automatically. +network isolation. Host rules are tested through a mock host and a scripted +`iptables`; the default suite changes no OS setting. It does not launch CS 1.6 or +Warcraft III. ## Future exported LANs