diff --git a/README.md b/README.md index aa3b8c4..4a03069 100644 --- a/README.md +++ b/README.md @@ -146,9 +146,8 @@ No vote is involved — see [docs/sync-model.md](docs/sync-model.md). Because the address is allocated at run time rather than derived, it is not -known until the agent has started and agreed with its peers. On Linux the -agent assigns it itself as soon as it has one; elsewhere `tsunagi tun-setup` -prints the `ip address add` line once the claim is in `state.sqlite`. +known until the agent has started and agreed with its peers. The agent then +assigns it to the interface itself. ## Privileges @@ -192,64 +191,37 @@ it carried. Two things are never touched: Both of those refuse with an explanation rather than guessing. -### Running without the capability +Two settings the manual recipe used to need are gone with it. +`keep_addr_on_down` existed only because an interface nobody held open lost +carrier and had its IPv6 addresses flushed, and `nodad` only because duplicate +address detection can never finish without carrier. An interface held open for +its whole life has carrier for its whole life. -`--interface attach` (or `auto`, which falls back on its own) opens an -interface prepared beforehand and needs **no privileges at all**. Ask the -agent what to run: +### The MTU is 1280 -```bash -tsunagi tun-setup --network lab --secret "$SECRET" -``` - -```text -# Interface tsunjwc6dcrtmo5, address fd80:1210:f724:f620:d1bb:f982:3b6e:19bd/64, mtu 1280 -# Run once as root; then run `tsunagi up` as ab. - -sudo ip tuntap add dev tsunjwc6dcrtmo5 mode tun user ab -sudo ip link set dev tsunjwc6dcrtmo5 mtu 1280 up -sudo sysctl -qw net.ipv6.conf.tsunjwc6dcrtmo5.keep_addr_on_down=1 -sudo ip -6 address add fd80:1210:f724:f620:d1bb:f982:3b6e:19bd/64 dev tsunjwc6dcrtmo5 nodad -``` - -`user ab` is the point: the interface is persistent and owned by that user, so -`tsunagi up` afterwards opens it with no privileges and no capabilities. - -The last two settings are what the managed path does not need. A persistent -TUN has **no carrier** until a process attaches to it; Linux flushes IPv6 -addresses from an interface that loses carrier unless `keep_addr_on_down` is -set, and duplicate address detection can never finish without carrier, so the -address would sit there tentative and unusable without `nodad`. An interface -the agent creates and holds open has carrier for its whole life, so neither -applies. IPv4 needs neither in either case: Linux keeps IPv4 addresses across -carrier loss and IPv4 has no duplicate address detection. - -With IPv4 enabled, `tun-setup` adds an `ip address add` line once there is an -address to print — it reads the signed claim back out of `state.sqlite`, which -does not disturb a running agent. - -The MTU is 1280 because that is the minimum IPv6 requires (RFC 8200). Linux -disables IPv6 entirely on an interface below it — the per-device +That is the minimum IPv6 requires (RFC 8200), and Linux enforces it by +disabling IPv6 outright on an interface below it — the per-device `/proc/sys/net/ipv6` entries vanish and adding an address fails with -`Invalid argument` — so a smaller MTU cannot work at all. The agent refuses -one rather than letting it fail later. +`Invalid argument`. A smaller MTU cannot work at all, so the agent refuses one +rather than letting it fail later. See +[docs/wireguard.md](docs/wireguard.md#mtu) for the ceiling that pushes back +from the other side. ### Summary | approach | agent runs as | notes | |---|---|---| -| `setcap cap_net_admin+p` | ordinary user, one capability | recommended on Linux: nothing to prepare, nothing left behind. Lost on every rebuild or copy of the binary. | +| `setcap cap_net_admin+p` | ordinary user, one capability | recommended: nothing to prepare, nothing left behind. Lost on every rebuild or copy of the binary. | | systemd service | `User=`, `AmbientCapabilities=CAP_NET_ADMIN` | the same, for an installed service | -| `tsunagi tun-setup` then `--interface attach` | ordinary user, no capabilities | one privileged setup per host; needed on macOS and Windows, where no provisioner is implemented yet | | `sudo tsunagi up` | root | everything works, nothing is isolated | | `--no-tun` | ordinary user, no capabilities | tunnels run and handshake, traffic never reaches the OS | **Not implemented yet.** macOS and Windows have no provisioner: both need real platform work — `utun` and `SystemConfiguration` on one, the IP Helper -API and a Wintun adapter on the other. On those the agent says so and falls -back to attaching to a prepared interface. The decision logic that says *what* -to change is shared and tested on every platform; only the execution is -per-platform. +API and a Wintun adapter on the other. There the agent says so and `--no-tun` +is the way to run it; the control plane and the tunnels are unaffected. The +decision logic that says *what* to change is shared and tested on every +platform, so only the execution is left to write. ## Checks diff --git a/docs/wireguard.md b/docs/wireguard.md index 8f21026..bad7509 100644 --- a/docs/wireguard.md +++ b/docs/wireguard.md @@ -22,13 +22,13 @@ system can hand us IP packets, and even that is behind a trait | | needs privileges | what it proves | |---|---|---| | `MemoryTunFactory` | no | handshake, encryption, routing, address ownership | -| `SystemTunFactory`, attaching | none, if the interface was prepared | traffic actually reaches the OS | -| `SystemTunFactory`, creating | `CAP_NET_ADMIN` | the same, at the cost of a capability | +| `MockProvisioner` | no | the above, plus what would have been done to the host | +| `ManagedTunFactory` | `CAP_NET_ADMIN` | traffic actually reaches the OS | -`SystemTunFactory` attaches to an interface that already exists and only -creates one when it does not. A persistent interface created by root and owned -by the user lets the agent run with no privileges at all; see *Running -unprivileged* in [../README.md](../README.md#running-unprivileged). +`ManagedTunFactory` creates the interface and configures it; see +*Provisioning the interface* below and *Privileges* in +[../README.md](../README.md#privileges). With no capability the agent runs +with `--no-tun`: everything but the last hop into the kernel still works. [boringtun]: https://docs.rs/boringtun [`TunFactory`]: https://docs.rs/tsunagi @@ -126,14 +126,16 @@ wins, and the agent adopts it. So the flag matters for whoever starts the network and is harmless afterwards. Pass `--ipv4-range none` for an IPv6-only overlay. -Putting the address on the interface still needs privileges, and the agent -cannot do it. Since the address is only known once the agent has agreed with -its peers, `tsunagi tun-setup` cannot print it up front either. So the agent -checks whether the address is assigned anywhere on the host — by binding a UDP -socket to it, which needs no privileges — and reports the exact command until -it is. This matters: with the wrong address on the interface, packets leave -with the wrong source and every peer drops them as not belonging to us, which -looks like a broken network rather than a missing command. +The agent puts the address on the interface itself, as soon as the network has +agreed on it — no restart, and the interface is not recreated, which would +drop every tunnel riding on it. + +It then checks that it is really there, by binding a UDP socket to it, which +needs no privileges. That check is deliberately independent of the code that +did the assigning: the failure it guards against is a silent one. With the +wrong address on the interface, packets leave with the wrong source and every +peer drops them as not belonging to us, which looks like a broken network +rather than a broken assumption. A release tombstone exists in the record type and merges correctly, but nothing emits one yet, so an address stays claimed until the network is @@ -224,7 +226,7 @@ use tsunagi::{Agent, Result}; async fn main() -> Result<()> { let paths = StoragePaths::user_default()?; - // MemoryTunFactory needs no privileges; swap in SystemTunFactory for a + // MemoryTunFactory needs no privileges; swap in ManagedTunFactory for a // real interface. let plugin = WireguardPlugin::open( WireguardConfig::new(paths.state_dir.join("wireguard")), diff --git a/src/bin/tsunagi.rs b/src/bin/tsunagi.rs index 104524e..da2fb34 100644 --- a/src/bin/tsunagi.rs +++ b/src/bin/tsunagi.rs @@ -45,15 +45,12 @@ enum Command { /// Shows this device's identity without joining anything. Id(PathArgs), /// Joins a network and runs until interrupted. - Up(UpArgs), + // Boxed: it is much larger than the other variants, and every command + // but this one would otherwise pay for its size. A `//` comment, not a + // `///` one, or clap would print it as help. + Up(Box), /// Asks a running agent what it is doing. Status(StatusArgs), - /// Prints the one-time privileged setup for the overlay interface. - /// - /// Run its output once as root, then run `tsunagi up` as an ordinary - /// user: the agent attaches to the prepared interface and needs no - /// privileges of its own. - TunSetup(TunSetupArgs), } #[derive(Debug, Args)] @@ -66,45 +63,6 @@ struct StatusArgs { control_socket: Option, } -#[derive(Debug, Args)] -struct TunSetupArgs { - #[command(flatten)] - paths: PathArgs, - - /// Network name, exactly as passed to `tsunagi up`. - #[arg(long, short = 'n')] - network: String, - - /// The shared secret. - #[arg( - long, - short = 's', - env = "TSUNAGI_SECRET", - conflicts_with = "secret_file" - )] - secret: Option, - - /// Read the shared secret from a file instead of the command line. - #[arg(long)] - secret_file: Option, - - /// The user that should own the interface. Defaults to the current one. - #[arg(long)] - user: Option, - - /// Interface name prefix, matching `tsunagi up --wg-prefix`. - #[arg(long, default_value = "tsun")] - wg_prefix: String, - - /// Interface MTU, matching `tsunagi up --wg-mtu`. At least 1280. - #[arg(long)] - wg_mtu: Option, - - /// Match `tsunagi up --ipv4-range`. - #[arg(long, value_name = "CIDR")] - ipv4_range: Option, -} - /// Resolves the IPv4 overlay range from the flag. /// /// Absent means the built-in default. A network that already settled on @@ -224,15 +182,6 @@ struct UpArgs { #[arg(long)] no_tun: bool, - /// How the overlay interface is obtained. - /// - /// `managed` has the agent create and configure it itself, which needs - /// CAP_NET_ADMIN and cleans up on exit. `attach` opens an interface that - /// was prepared beforehand (see `tsunagi tun-setup`) and needs no - /// privileges. `auto` manages it when it can and attaches when it cannot. - #[arg(long, value_enum, default_value_t = InterfaceMode::Auto)] - interface: InterfaceMode, - /// Interface name prefix for the WireGuard data plane. #[arg(long, default_value = "tsun")] wg_prefix: String, @@ -347,37 +296,11 @@ async fn run(command: Command) -> Result<(), Box> { } Command::Doctor(paths) => doctor(paths).await, Command::Id(paths) => show_id(paths).await, - Command::Up(args) => up(args).await, - Command::TunSetup(args) => tun_setup(args).await, + Command::Up(args) => up(*args).await, Command::Status(args) => status(args).await, } } -/// The IPv4 address this agent has already been allocated, if any. -/// -/// Read straight from the mandatory state store. Opening it for reading does -/// not take the directory lock, so this works while the agent is running. -/// Records are verified here too: the database is not a trust boundary. -fn allocated_ipv4( - paths: &StoragePaths, - network: tsunagi::NetworkId, -) -> Result, Box> { - use tsunagi::state::RecordBody; - use tsunagi::storage::StateStore; - - let store = StateStore::open(paths.state_db())?; - let author = store.load_or_create_device_identity()?.endpoint_id(); - for record in store.signed_records(network)? { - if record.author != *author.as_bytes() || record.verify(network).is_err() { - continue; - } - if let RecordBody::Ipv4Claim { address, range } = record.body { - return Ok(Some((address, range.prefix_len))); - } - } - Ok(None) -} - /// Path of the local control socket for a state directory. fn control_socket(paths: &StoragePaths, override_path: Option<&PathBuf>) -> PathBuf { match override_path { @@ -404,86 +327,6 @@ async fn status(args: StatusArgs) -> Result<(), Box> { Ok(()) } -/// Works out the interface name and overlay address, then prints the -/// privileged commands that prepare it. -/// -/// The address depends on this agent's WireGuard key for the network, so the -/// key store is opened (and the key created on first use) to compute it. -async fn tun_setup(args: TunSetupArgs) -> Result<(), Box> { - use tsunagi::dataplane::wireguard::{ - DEFAULT_MTU, OVERLAY_PREFIX_LEN, WgKeyStore, interface_name, overlay_address, - }; - use tsunagi::identity::NetworkKeys; - - let name = NetworkName::new(args.network.clone())?; - let secret = load_secret(args.secret.as_deref(), args.secret_file.as_deref())?; - let paths = args.paths.resolve()?; - let network = NetworkKeys::derive(&name, &secret).network_id(); - - let store_path = paths.state_dir.join("wireguard").join("wireguard.sqlite"); - let store = tokio::task::spawn_blocking({ - let store_path = store_path.clone(); - move || WgKeyStore::open(store_path) - }) - .await??; - let key = tokio::task::spawn_blocking(move || store.load_or_create(network)).await??; - - let interface = interface_name(&args.wg_prefix, network)?; - let address = overlay_address(network, &key.public()); - let ipv4_range = resolve_ipv4_range(args.ipv4_range.as_ref())?; - let mtu = args.wg_mtu.unwrap_or(DEFAULT_MTU); - let user = args.user.unwrap_or_else(|| { - std::env::var("SUDO_USER") - .or_else(|_| std::env::var("USER")) - .unwrap_or_else(|_| "$USER".to_string()) - }); - - println!("# Network {name} ({network})"); - println!("# Interface {interface}, address {address}/{OVERLAY_PREFIX_LEN}, mtu {mtu}"); - // The IPv4 address is allocated at run time, so it can only be shown once - // the agent has one. Reading the state store does not disturb a running - // agent: the directory lock belongs to the agent, not to this reader. - let allocated_v4 = ipv4_range.and_then(|_| allocated_ipv4(&paths, network).ok().flatten()); - match (ipv4_range, allocated_v4) { - (Some(_), Some((address, prefix_len))) => { - println!("# IPv4 overlay address {address}/{prefix_len}, allocated and signed"); - } - (Some(_), None) => { - println!( - "# IPv4 is allocated once the agent runs and agrees with its peers, so\n\ - # there is nothing to print yet. Start `tsunagi up`: it prints the exact\n\ - # `ip address add` command for the address it was given, and this\n\ - # command will include it from then on." - ); - } - (None, _) => {} - } - println!("# Run once as root; then run `tsunagi up` as {user}."); - println!( - "#\n\ - # keep_addr_on_down matters: a persistent TUN interface has no carrier\n\ - # until a process attaches, and Linux flushes IPv6 addresses from an\n\ - # interface that loses carrier unless it is set. `nodad` matters for the\n\ - # same reason: duplicate address detection can never finish without a\n\ - # carrier, leaving the address tentative and unusable.\n" - ); - println!("sudo ip tuntap add dev {interface} mode tun user {user}"); - println!("sudo ip link set dev {interface} mtu {mtu} up"); - println!("sudo sysctl -qw net.ipv6.conf.{interface}.keep_addr_on_down=1"); - println!("sudo ip -6 address add {address}/{OVERLAY_PREFIX_LEN} dev {interface} nodad"); - if let Some((address, prefix_len)) = allocated_v4 { - // IPv4 addresses are not flushed when an interface loses carrier, so - // this one needs none of the treatment IPv6 does. - println!("sudo ip address add {address}/{prefix_len} dev {interface}"); - } - - println!("\n# To check it afterwards:"); - println!("ip addr show dev {interface}"); - println!("\n# To remove it again:"); - println!("sudo ip link del dev {interface}"); - Ok(()) -} - async fn show_id(paths: PathArgs) -> Result<(), Box> { let paths = paths.resolve()?; println!("state directory {}", paths.state_dir.display()); @@ -555,9 +398,9 @@ async fn doctor(paths: PathArgs) -> Result<(), Box> { } Privilege::Missing(reason) => { println!(" privileges no CAP_NET_ADMIN ({reason})"); - println!(" interface must be prepared first; run `tsunagi tun-setup`"); + println!(" interface cannot be created; run with `--no-tun` meanwhile"); println!( - " to manage it {}", + " to grant it {}", Privilege::how_to_grant(&program_path()) ); } @@ -566,7 +409,7 @@ async fn doctor(paths: PathArgs) -> Result<(), Box> { " privileges managing interfaces is not implemented on {} yet", std::env::consts::OS ); - println!(" interface must be prepared first; run `tsunagi tun-setup`"); + println!(" interface cannot be created; run with `--no-tun`"); } } } @@ -634,7 +477,7 @@ async fn up(args: UpArgs) -> Result<(), Box> { let tun_factory: Arc = if args.no_tun { Arc::new(MemoryTunFactory::new()) } else { - system_tun_factory(args.interface)? + system_tun_factory()? }; let mut wg = WireguardConfig::new(paths.state_dir.join("wireguard")) .with_interface_prefix(args.wg_prefix.clone()); @@ -866,47 +709,14 @@ async fn stop_signal() -> &'static str { } } -/// How the overlay interface is obtained. -#[derive(Debug, Clone, Copy, PartialEq, Eq, clap::ValueEnum)] -enum InterfaceMode { - /// Manage it when possible, attach to a prepared one otherwise. - Auto, - /// Create and configure it in process. Needs CAP_NET_ADMIN. - Managed, - /// Open an interface prepared beforehand. Needs no privileges. - Attach, -} - -/// Builds the interface factory for the chosen mode. +/// Builds the interface factory. /// -/// The managed path is preferred because it is the one that cleans up after -/// itself: the interface is tied to an open file descriptor, so it goes away -/// when the agent does, however the agent goes away. -#[cfg(feature = "tun-device")] -fn system_tun_factory( - mode: InterfaceMode, -) -> Result, Box> { - use tsunagi::dataplane::wireguard::SystemTunFactory; - - if mode == InterfaceMode::Attach { - return Ok(Arc::new(SystemTunFactory::new())); - } - - match managed_tun_factory() { - Ok(factory) => Ok(factory), - Err(err) if mode == InterfaceMode::Managed => Err(err), - Err(err) => { - tracing::warn!( - "{err} Falling back to attaching to a prepared interface; \ - `tsunagi tun-setup` prints how to make one." - ); - Ok(Arc::new(SystemTunFactory::new())) - } - } -} - +/// One path: the agent creates and configures the interface itself. It is +/// also the one that cleans up after itself, because the interface is tied to +/// an open file descriptor and goes away with the agent, however the agent +/// goes away. #[cfg(all(feature = "tun-device", target_os = "linux"))] -fn managed_tun_factory() -> Result, Box> { +fn system_tun_factory() -> Result, Box> { use tsunagi::dataplane::wireguard::{ManagedTunFactory, NetlinkProvisioner}; let provisioner = NetlinkProvisioner::new()?; Ok(Arc::new(ManagedTunFactory::new(Arc::new(provisioner)))) @@ -914,21 +724,20 @@ fn managed_tun_factory() -> Result, Box Result, Box> { +fn system_tun_factory() -> Result, Box> { Err(format!( - "managing the overlay interface is not implemented on {} yet.", + "managing the overlay interface is not implemented on {} yet. \ + Run with `--no-tun` to keep the tunnels off the operating system.", std::env::consts::OS ) .into()) } #[cfg(not(feature = "tun-device"))] -fn system_tun_factory( - _mode: InterfaceMode, -) -> Result, Box> { +fn system_tun_factory() -> Result, Box> { Err("this build has no interface support; rebuild with the `tun-device` feature or pass --no-tun".into()) } diff --git a/src/dataplane/wireguard/mod.rs b/src/dataplane/wireguard/mod.rs index 56e69f0..7c0f82f 100644 --- a/src/dataplane/wireguard/mod.rs +++ b/src/dataplane/wireguard/mod.rs @@ -35,10 +35,9 @@ //! The only part that needs privileges is the packet interface. With //! [`tun::MemoryTunFactory`] the whole data plane — handshake, encryption, //! routing, address ownership — runs and is tested with no privileges at all. -//! For real traffic there are two ways in: [`provision::ManagedTunFactory`], -//! where the agent creates and configures the interface itself over netlink -//! and removes it again on exit, and `SystemTunFactory`, which attaches to an -//! interface somebody else prepared and needs no privileges. +//! For real traffic there is [`provision::ManagedTunFactory`], where the +//! agent creates and configures the interface itself over netlink and +//! removes it again on exit. //! //! See `docs/wireguard.md` for the full picture. @@ -74,9 +73,3 @@ pub use tun::{MemoryTun, MemoryTunFactory, TunDevice, TunFactory, TunRequest, ad #[cfg(all(feature = "tun-device", target_os = "linux"))] pub use provision::NetlinkProvisioner; - -#[cfg(feature = "tun-device")] -pub use tun::{ - Assigned, SystemTunFactory, interface_addresses, interface_exists, parse_if_inet6, - setup_commands, -}; diff --git a/src/dataplane/wireguard/plugin.rs b/src/dataplane/wireguard/plugin.rs index 1b76e66..cdf1ef5 100644 --- a/src/dataplane/wireguard/plugin.rs +++ b/src/dataplane/wireguard/plugin.rs @@ -594,11 +594,13 @@ impl Worker { } device.retain_peers(&wanted); - // The address is allocated at run time, but putting it on the - // interface needs privileges we do not have. Without it the kernel - // sends our packets with the wrong source address and every peer - // drops them, which looks like a broken network rather than a missing - // command. So say exactly what is wrong. + // The agent assigns this address itself, so finding it absent means + // the assignment did not take — something outside removed it, or the + // provisioner reported a success it did not achieve. Left unsaid it + // looks like a broken network: the kernel would send packets with the + // wrong source address and every peer would drop them. So it is + // checked rather than assumed, because the assumption is exactly the + // kind that has been wrong here before. let missing_v4 = match (own_v4, interface.as_deref(), range) { (Some(address), Some(interface), Some(range)) if !super::tun::address_is_local(IpAddr::V4(address)) => @@ -622,11 +624,11 @@ impl Worker { self.report( network, format!( - "this agent was allocated {address} but that address is not on any \ + "this agent was allocated {address}/{} but the address is not on any \ interface, so IPv4 cannot work: packets would leave with the wrong \ - source and every peer would drop them. Run:\n \ - sudo ip address add {address}/{} dev {interface}\n \ - and remove any other address of that range from it.", + source and every peer would drop them. It should have been assigned \ + to `{interface}` automatically; check whether something else removed \ + it.", range.prefix_len ), ); diff --git a/src/dataplane/wireguard/provision/linux.rs b/src/dataplane/wireguard/provision/linux.rs index 0748446..8bd2afd 100644 --- a/src/dataplane/wireguard/provision/linux.rs +++ b/src/dataplane/wireguard/provision/linux.rs @@ -153,7 +153,7 @@ impl NetlinkProvisioner { fn create_device(&self, plan: &InterfacePlan) -> Result, PluginError> { let request = TunRequest::bare(plan.name.clone(), plan.mtu); let _guard = NetAdmin::acquire()?; - super::super::tun::open_tun(&request, false) + super::super::tun::open_tun(&request) } } diff --git a/src/dataplane/wireguard/provision/privilege.rs b/src/dataplane/wireguard/provision/privilege.rs index 882ada2..0e6fba6 100644 --- a/src/dataplane/wireguard/provision/privilege.rs +++ b/src/dataplane/wireguard/provision/privilege.rs @@ -50,8 +50,9 @@ impl Privilege { pub fn how_to_grant(program: &str) -> String { format!( "Grant it once with `sudo setcap cap_net_admin+p {program}` \ - and the agent manages its own interface. Without it, prepare the \ - interface by hand with `tsunagi tun-setup`." + and the agent manages its own interface. Without it, run with \ + `--no-tun`: the tunnels still form, they just do not reach the \ + operating system." ) } } @@ -153,7 +154,7 @@ mod tests { fn the_grant_instructions_name_the_program() { let text = Privilege::how_to_grant("/usr/local/bin/tsunagi"); assert!(text.contains("setcap cap_net_admin+p /usr/local/bin/tsunagi")); - assert!(text.contains("tun-setup"), "the fallback is offered too"); + assert!(text.contains("--no-tun"), "the fallback is offered too"); } #[test] diff --git a/src/dataplane/wireguard/provision/unsupported.rs b/src/dataplane/wireguard/provision/unsupported.rs index e630f96..35b056a 100644 --- a/src/dataplane/wireguard/provision/unsupported.rs +++ b/src/dataplane/wireguard/provision/unsupported.rs @@ -34,8 +34,8 @@ impl UnsupportedProvisioner { fn refusal(&self) -> PluginError { PluginError::Unavailable(format!( "managing the overlay interface is not implemented on {} yet. \ - Prepare the interface by hand — `tsunagi tun-setup` prints what to run — \ - and the agent will attach to it.", + Run with `--no-tun` until it is: the tunnels still form, they just \ + do not reach the operating system.", self.platform )) } @@ -72,7 +72,7 @@ mod tests { let plan = InterfacePlan::new("tsuntest", 1280, Vec::new()); let err = provisioner.reconcile(&plan).await.unwrap_err(); let message = err.to_string(); - assert!(message.contains("tun-setup"), "{message}"); + assert!(message.contains("--no-tun"), "{message}"); assert!(message.contains(std::env::consts::OS), "{message}"); provisioner.remove("tsuntest").await.unwrap(); diff --git a/src/dataplane/wireguard/tun.rs b/src/dataplane/wireguard/tun.rs index 69e33f4..f95f892 100644 --- a/src/dataplane/wireguard/tun.rs +++ b/src/dataplane/wireguard/tun.rs @@ -10,8 +10,8 @@ //! is what the test suite uses, so the entire data plane — handshake, //! encryption, routing — is exercised without touching the host. //! * `SystemTun`, behind the `tun-device` feature, is a real TUN interface. -//! Creating one needs `CAP_NET_ADMIN`; attaching to one somebody else -//! prepared needs nothing. +//! Creating one needs `CAP_NET_ADMIN`, and it is +//! [`provision`](super::provision) that holds that and creates it. use std::net::Ipv6Addr; use std::sync::Arc; @@ -238,43 +238,27 @@ impl TunFactory for MemoryTunFactory { #[cfg(feature = "tun-device")] pub(crate) use system::open_tun; -#[cfg(feature = "tun-device")] -pub use system::{ - Assigned, SystemTunFactory, interface_addresses, interface_exists, parse_if_inet6, - setup_commands, -}; #[cfg(feature = "tun-device")] mod system { - use std::net::Ipv6Addr; use std::sync::Arc; use bytes::Bytes; use tokio::sync::Mutex; - use super::{TunDevice, TunFactory, TunRequest}; + use super::{TunDevice, TunRequest}; use crate::BoxFuture; use crate::dataplane::PluginError; /// A real TUN interface. /// - /// Two ways to get one, and the difference is who needs privileges: + /// Created by opening `/dev/net/tun`, which needs `CAP_NET_ADMIN` and is + /// why [`open_tun`] is only ever called from + /// [`provision`](super::super::provision), where that capability is + /// raised for the length of the call and no longer. /// - /// * **Attach** to an interface that already exists. Needs no privileges - /// at all, as long as the interface was created persistent and owned by - /// this user. This is the recommended way to run the agent unprivileged. - /// * **Create** it here, which needs `CAP_NET_ADMIN`. - /// - /// `SystemTunFactory` is the **attach** path, for a host where the agent - /// has no privileges at all: the interface and its addresses were put - /// there by something else, so it checks they are present and says - /// exactly what to run if they are not, rather than coming up in a state - /// where no traffic could ever arrive. - /// - /// The other path is - /// [`ManagedTunFactory`](super::super::provision::ManagedTunFactory), - /// where the agent creates and configures the interface itself. That is - /// the default on Linux and needs no preparation at all. + /// It is deliberately **not** made persistent, so the kernel removes the + /// interface when this value is dropped — however the process ends. pub struct SystemTun { name: String, mtu: u32, @@ -331,168 +315,18 @@ mod system { } } - /// Whether an interface of this name exists. - pub fn interface_exists(name: &str) -> bool { - std::path::Path::new(&format!("/sys/class/net/{name}")).exists() - } - - /// `IFA_F_TENTATIVE`: the address is not usable until DAD finishes, which - /// never happens on an interface with no carrier. - const IFA_F_TENTATIVE: u32 = 0x40; - /// `IFA_F_DADFAILED`: duplicate address detection rejected it. - const IFA_F_DADFAILED: u32 = 0x08; - - /// One IPv6 address assigned to an interface. - #[derive(Debug, Clone, Copy, PartialEq, Eq)] - pub struct Assigned { - /// The address. - pub address: Ipv6Addr, - /// Raw `IFA_F_*` flags as the kernel reports them. - pub flags: u32, - } - - impl Assigned { - /// Whether the address can actually carry traffic. - pub fn is_usable(&self) -> bool { - self.flags & (IFA_F_TENTATIVE | IFA_F_DADFAILED) == 0 - } - - /// A short explanation when it cannot. - pub fn why_unusable(&self) -> Option<&'static str> { - if self.flags & IFA_F_DADFAILED != 0 { - Some("duplicate address detection failed") - } else if self.flags & IFA_F_TENTATIVE != 0 { - Some( - "still tentative; duplicate address detection cannot finish \ - on an interface with no carrier, so add it with `nodad`", - ) - } else { - None - } - } - } - - /// Parses the IPv6 addresses of one interface out of `/proc/net/if_inet6`. + /// Creates the TUN interface by opening it. /// - /// Each line is `<32 hex address> - /// `, all hexadecimal. - pub fn parse_if_inet6(contents: &str, name: &str) -> Vec { - contents - .lines() - .filter_map(|line| { - let fields: Vec<&str> = line.split_whitespace().collect(); - if fields.len() < 6 || fields[5] != name { - return None; - } - let raw = <[u8; 16]>::try_from(hex::decode(fields[0]).ok()?.as_slice()).ok()?; - Some(Assigned { - address: Ipv6Addr::from(raw), - flags: u32::from_str_radix(fields[4], 16).unwrap_or(0), - }) - }) - .collect() - } - - /// The IPv6 addresses of an interface, or `None` if that cannot be read. - /// - /// Reads `/proc/net/if_inet6`, which needs no privileges. - pub fn interface_addresses(name: &str) -> Option> { - let contents = std::fs::read_to_string("/proc/net/if_inet6").ok()?; - Some(parse_if_inet6(&contents, name)) - } - - /// What is wrong with an interface's addressing, if anything. - pub(crate) fn check_address(name: &str, wanted: Ipv6Addr) -> Result<(), String> { - let Some(assigned) = interface_addresses(name) else { - // Cannot tell. Carry on rather than block on a guess. - return Ok(()); - }; - match assigned.iter().find(|entry| entry.address == wanted) { - Some(entry) if entry.is_usable() => Ok(()), - Some(entry) => Err(format!( - "interface `{name}` has {wanted} but it is unusable: {}", - entry.why_unusable().unwrap_or("unknown reason") - )), - None => { - let present = if assigned.is_empty() { - "it currently has no IPv6 address at all".to_string() - } else { - format!( - "it currently has: {}", - assigned - .iter() - .map(|entry| entry.address.to_string()) - .collect::>() - .join(", ") - ) - }; - Err(format!( - "interface `{name}` has no {wanted} address, {present}" - )) - } - } - } - - /// The commands a privileged user runs once to prepare an interface. - /// - /// The order and the two extra settings matter. A persistent TUN - /// interface has no carrier until a process attaches to it, and Linux - /// flushes IPv6 addresses from an interface that loses carrier unless - /// `keep_addr_on_down` is set — so an address added without it silently - /// disappears before the agent ever starts. `nodad` is needed for the same - /// reason: duplicate address detection can never finish with no carrier, - /// and the address would stay tentative and unusable. - pub fn setup_commands(request: &TunRequest, user: &str) -> Vec { - let mut commands = vec![ - format!( - "sudo ip tuntap add dev {} mode tun user {user}", - request.name - ), - format!( - "sudo ip link set dev {} mtu {} up", - request.name, request.mtu - ), - format!( - "sudo sysctl -qw net.ipv6.conf.{}.keep_addr_on_down=1", - request.name - ), - format!( - "sudo ip -6 address add {}/{} dev {} nodad", - request.address, request.prefix_len, request.name - ), - ]; - if let Some(address) = request.address_v4 { - let prefix_len = request.prefix_len_v4; - // IPv4 is not sensitive to carrier the way IPv6 is, so it needs - // no extra settings. - commands.push(format!( - "sudo ip address add {address}/{prefix_len} dev {}", - request.name - )); - } - commands - } - - /// Opens the TUN interface, creating it if it is not already there. - /// - /// Synchronous, and deliberately so: on the managed path the caller holds - /// a capability guard across this call, and a guard must not span an - /// `await` because Linux capabilities are per thread. - /// - /// `attach_only` says the interface already exists and was prepared by - /// something else, so nothing beyond `TUNSETIFF` is issued — reconfiguring - /// it would need exactly the privileges that path is avoiding. - pub(crate) fn open_tun( - request: &TunRequest, - attach_only: bool, - ) -> Result, PluginError> { + /// Synchronous, and deliberately so: the caller holds a capability guard + /// across this call, and such a guard must not span an `await` because + /// Linux capabilities are per thread. + pub(crate) fn open_tun(request: &TunRequest) -> Result, PluginError> { let mut config = tun::Configuration::default(); config.tun_name(&request.name); config.platform_config(|platform| { - // The crate's own root check is not the check we want: the - // managed path holds CAP_NET_ADMIN without being root, and the - // attach path needs no privileges at all. Whether the open - // succeeds is the honest answer either way. + // The crate's own root check is not the check we want: this holds + // CAP_NET_ADMIN without being root. Whether the open succeeds is + // the honest answer. platform.ensure_root_privileges(false); }); // Packet information stays off, so reads and writes are raw IP @@ -500,22 +334,12 @@ mod system { // information, so the flags match when attaching to one. let device = tun::create_as_async(&config).map_err(|err| { - let hint = if attach_only { - format!( - "interface `{}` exists but could not be opened: {err}. \ - It must be a persistent TUN interface owned by this user.", - request.name - ) - } else { - format!( - "cannot create the TUN interface `{}`: {err}. \ - Creating one needs CAP_NET_ADMIN. Either grant it with \ - `setcap cap_net_admin+p`, or prepare the interface once as root \ - (see `tsunagi tun-setup`) and run unprivileged.", - request.name - ) - }; - PluginError::Unavailable(hint) + PluginError::Unavailable(format!( + "cannot create the TUN interface `{}`: {err}. Creating one needs \ + CAP_NET_ADMIN; grant it with `setcap cap_net_admin+p`, or run with \ + `--no-tun` to keep the tunnels off the operating system.", + request.name + )) })?; let (reader, writer) = tokio::io::split(device); @@ -526,172 +350,4 @@ mod system { writer: Mutex::new(writer), }) as Arc) } - - fn current_user() -> String { - std::env::var("SUDO_USER") - .or_else(|_| std::env::var("USER")) - .unwrap_or_else(|_| "$USER".to_string()) - } - - /// Opens real TUN interfaces. - #[derive(Debug, Clone, Default)] - pub struct SystemTunFactory; - - impl SystemTunFactory { - /// Creates the factory. - pub fn new() -> Self { - Self - } - } - - impl TunFactory for SystemTunFactory { - fn name(&self) -> &str { - "system" - } - - fn create<'a>( - &'a self, - request: TunRequest, - ) -> BoxFuture<'a, Result, PluginError>> { - Box::pin(async move { - let existed = interface_exists(&request.name); - - // Check before attaching. Opening and then dropping the - // device toggles the carrier, and with the default - // `keep_addr_on_down=0` that is enough to flush the very - // address we are looking for. - if existed && let Err(reason) = check_address(&request.name, request.address) { - let commands = setup_commands(&request, ¤t_user()).join("\n "); - return Err(PluginError::Unavailable(format!( - "{reason}.\nAssigning an IPv6 address needs privileges. \ - Remove the interface and prepare it again:\n sudo ip link del dev {}\n {commands}", - request.name - ))); - } - - let device = open_tun(&request, existed)?; - - // An interface we just created has no address yet either. - if let Err(reason) = check_address(&request.name, request.address) { - let commands = setup_commands(&request, ¤t_user()).join("\n "); - return Err(PluginError::Unavailable(format!( - "{reason}.\nAssigning an IPv6 address needs privileges. Run:\n {commands}" - ))); - } - - Ok(device) - }) - } - } -} - -#[cfg(all(test, feature = "tun-device", target_os = "linux"))] -mod system_tests { - #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)] - - use std::net::Ipv6Addr; - - use super::TunRequest; - use super::system::{interface_addresses, parse_if_inet6, setup_commands}; - - const SAMPLE: &str = "\ -fe800000000000008baaeb0c433b635a 04 40 20 80 tailscale0 -00000000000000000000000000000001 01 80 10 80 lo -fd559caf9652cb86321feac65c73bd82 05 40 00 80 tsun0 -fd559caf9652cb86321feac65c73bd83 05 40 00 40 tsun0 -fd559caf9652cb86321feac65c73bd84 05 40 00 08 tsun0 -"; - - #[test] - fn addresses_are_read_per_interface_with_their_flags() { - let found = parse_if_inet6(SAMPLE, "tsun0"); - assert_eq!(found.len(), 3); - assert_eq!( - found[0].address, - "fd55:9caf:9652:cb86:321f:eac6:5c73:bd82" - .parse::() - .unwrap() - ); - assert!(found[0].is_usable(), "permanent address is usable"); - - // Tentative: duplicate address detection never finishes without a - // carrier, so the address exists but cannot carry traffic. - assert!(!found[1].is_usable()); - assert!(found[1].why_unusable().unwrap().contains("tentative")); - - // Duplicate address detection failed outright. - assert!(!found[2].is_usable()); - assert!(found[2].why_unusable().unwrap().contains("duplicate")); - - assert!(parse_if_inet6(SAMPLE, "nosuchdev").is_empty()); - // An interface name that is a prefix of another must not match. - assert!(parse_if_inet6(SAMPLE, "tsun").is_empty()); - } - - #[test] - fn malformed_lines_are_skipped_rather_than_panicking() { - assert!(parse_if_inet6("", "tsun0").is_empty()); - assert!(parse_if_inet6("garbage", "tsun0").is_empty()); - assert!(parse_if_inet6("zz 01 40 00 80 tsun0", "tsun0").is_empty()); - assert!(parse_if_inet6("00 01 40 00 80 tsun0", "tsun0").is_empty()); - // Flags that do not parse fall back to zero rather than dropping the - // address, so a usable address is never hidden by a formatting change. - let odd = parse_if_inet6("00000000000000000000000000000001 01 80 10 zz lo", "lo"); - assert_eq!(odd.len(), 1); - assert!(odd[0].is_usable()); - } - - #[test] - fn loopback_is_found_on_this_host() { - // A real read of /proc/net/if_inet6: every Linux host has ::1 on lo. - let found = interface_addresses("lo").expect("/proc/net/if_inet6 should be readable"); - assert!( - found - .iter() - .any(|entry| entry.address == Ipv6Addr::LOCALHOST), - "expected ::1 on lo, got {found:?}" - ); - assert!( - interface_addresses("definitely-not-an-interface") - .unwrap() - .is_empty() - ); - } - - #[test] - fn the_setup_recipe_survives_a_carrier_drop() { - let request = TunRequest { - name: "tsun0".into(), - address: "fd00::1".parse().unwrap(), - prefix_len: 64, - address_v4: Some("100.64.1.2".parse().unwrap()), - prefix_len_v4: 10, - mtu: 1280, - }; - let commands = setup_commands(&request, "someone"); - - // The interface must be up before the address is added, the address - // must survive losing carrier, and it must not wait for duplicate - // address detection that can never complete. - let joined = commands.join("\n"); - let up = joined.find("link set dev tsun0 mtu 1280 up").unwrap(); - let keep = joined.find("keep_addr_on_down=1").unwrap(); - let add = joined.find("address add fd00::1/64").unwrap(); - assert!(up < keep && keep < add, "wrong order:\n{joined}"); - assert!(joined.contains("nodad")); - assert!(joined.contains("user someone")); - // IPv4 needs no carrier tricks, just the address. - assert!(joined.contains("ip address add 100.64.1.2/10 dev tsun0")); - - // An IPv6-only overlay says nothing about IPv4. - let v6_only = TunRequest { - address_v4: None, - ..request - }; - assert!( - !setup_commands(&v6_only, "someone") - .join("\n") - .contains("100.64") - ); - } } diff --git a/tests/wireguard.rs b/tests/wireguard.rs index 9b82a84..ad2801b 100644 --- a/tests/wireguard.rs +++ b/tests/wireguard.rs @@ -537,9 +537,11 @@ async fn an_allocated_address_missing_from_the_host_is_reported() { reason.contains(&allocated.to_string()), "unexpected: {reason}" ); + // The agent assigns the address itself, so the report says which + // interface should have had it rather than a command to run. assert!( - reason.contains("ip address add"), - "must name the fix: {reason}" + reason.contains(&a.plugin.overview(network_id).unwrap().interface), + "must name the interface: {reason}" ); a.shutdown().await;