Issue no privileged ioctl when attaching to a prepared interface

The tun crate runs configure() by default (ensure_root_privileges is true),
which is harmless today because it only acts on fields that were set, and
the attach path sets none. Saying so explicitly documents the intent and
keeps it true if the crate changes.

Also records why packet information stays off: `ip tuntap add ... mode tun`
defaults to no packet information too, so the TUNSETIFF flags match when
attaching and reads and writes stay raw IP packets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

README.md

@@ -100,15 +100,60 @@ wireguard: tsunkkcp43lmdje on fd15:1d9e:fa21:f201:…/64 mtu 1100, 1/1 tunnel(s)
 Notes:
 
 - Only one side needs `--peer`; the link is bidirectional.
-- The default `--transport n0` uses iroh's public address lookup and relays, so
-  two machines behind NAT find each other. `--transport local` keeps everything
+- The default `--transport relay` uses iroh's public address lookup and relays,
+  so two machines behind NAT find each other. `--transport local` keeps everything
   on the local network. See *How peers find each other* below — it is worth
   understanding what gets published.
-- Without `CAP_NET_ADMIN`, add `--no-tun`: the mesh, the data links and the
+- Without a network interface, add `--no-tun`: the mesh, the data links and the
   WireGuard handshakes all still run and are visible in the status output, only
   traffic does not reach the operating system. That is the quickest way to
   confirm the network forms.
-- Run as root (or grant `CAP_NET_ADMIN`) to get a real interface.
+- For real traffic, see *Running unprivileged* below.
+
+## Running unprivileged
+
+The agent does not need to run as root. Creating a network interface and
+giving it an address do need privileges, but they are a **one-time setup step**
+that can be done separately.
+
+Ask the agent what it needs, then run that once as root:
+
+```bash
+tsunagi tun-setup --network lab --secret "$SECRET"
+```
+
+```text
+# Network  lab (jwc6dcrtmo5zzdk7f6wfpkcqrpvqwr6po7vz3q2fvttgolt4ijfa)
+# Interface tsunjwc6dcrtmo5, address fd80:1210:f724:f620:d1bb:f982:3b6e:19bd/64, mtu 1100
+# Run once as root; then run `tsunagi up` as ab.
+
+sudo ip tuntap add dev tsunjwc6dcrtmo5 mode tun user ab
+sudo ip -6 address add fd80:1210:f724:f620:d1bb:f982:3b6e:19bd/64 dev tsunjwc6dcrtmo5
+sudo ip link set dev tsunjwc6dcrtmo5 mtu 1100 up
+```
+
+`user ab` is the point: the interface is persistent and owned by that user, so
+`tsunagi up` afterwards opens it with **no privileges and no capabilities at
+all**. The interface name and address are derived, so they are stable — the
+setup survives restarts and only has to be redone if the network name, the
+secret or this agent's WireGuard key changes.
+
+Other ways, and their trade-offs:
+
+| approach | agent runs as | notes |
+|---|---|---|
+| `tsunagi tun-setup` (above) | ordinary user, no capabilities | recommended |
+| `sudo setcap cap_net_admin+ep ./tsunagi` | ordinary user, one capability | the agent can then create the interface itself, but **still cannot assign the IPv6 address** (see below), so the `ip -6 address add` line is needed anyway. The capability is lost on every rebuild or copy. |
+| systemd service | `User=`, `AmbientCapabilities=CAP_NET_ADMIN` | same caveat about the address |
+| plain `sudo tsunagi up` | root | everything works, nothing is isolated |
+
+**Known limitation.** The agent cannot assign the IPv6 overlay address itself:
+the `tun` crate sets addresses through an IPv4-only ioctl, so an IPv6 address
+has to come from `ip -6 address add` or an equivalent. Rather than start with
+an interface that could never receive anything, the agent checks for the
+address in `/proc/net/if_inet6` and refuses with the exact command to run.
+Doing it in-process would mean talking netlink directly, which is possible but
+not implemented.
 
 ## Checks
 
@@ -184,10 +229,10 @@ explicitly.
 Two different lookups are involved, and only one of them is this project's:
 
 **1. Resolving one endpoint's address — iroh's, and it works today.**
-With `--transport n0` or `--transport direct`, iroh publishes a signed record
-of this endpoint's addresses, keyed by its endpoint id, to Number 0's public
-service (`dns.iroh.link`, over pkarr and DNS) and resolves other endpoints the
-same way. That is why `--peer <endpoint-id>` works with no address attached:
+With `--transport relay` or `--transport direct`, iroh publishes a signed
+record of this endpoint's addresses, keyed by its endpoint id, to the public
+service run by Number 0 — "n0", the company behind iroh — at `dns.iroh.link`,
+over pkarr and DNS, and resolves other endpoints the same way. That is why `--peer <endpoint-id>` works with no address attached:
 iroh looks it up. None of that code is ours.
 
 **2. Finding who is in a network — ours, and it is still manual.**
@@ -200,12 +245,13 @@ whatever those agents reach.
 
 What this means in practice:
 
-- With `n0` or `direct`, **your endpoint id and IP addresses are published to a
-  public third-party service.** They are not secret, and the network secret is
+- With `relay` or `direct`, **your endpoint id and IP addresses are published
+  to a public third-party service** (Number 0's, unless you change it). They are not secret, and the network secret is
   never published, but an observer of that service learns that your endpoint
   exists and where it is. `--transport local` publishes nothing.
 - A relay, when one is needed, sees the volume and timing of your traffic — not
-  its contents.
+  its contents. The default relays are Number 0's, in the US, EU and
+  Asia-Pacific.
 
 ## Storage
 

docs/threat-model.md

@@ -43,8 +43,9 @@ Read this before relying on anything here. The protocol is in
   handshake. Use `NetworkSecret::generate()`.
 - **Public address publication.** With `TransportPolicy::N0Defaults` or
   `DirectOnly`, iroh publishes a signed record of this endpoint's addresses,
-  keyed by its endpoint id, to Number 0's public pkarr/DNS service, and
-  resolves peers through it. The network secret is never published and
+  keyed by its endpoint id, to the public pkarr/DNS service run by Number 0
+  ("n0", the company behind iroh) at `dns.iroh.link`, and resolves peers
+  through it. The network secret is never published and
   membership cannot be inferred from a single record, but the endpoint's
   existence and its addresses become public. `LocalOnly` publishes nothing.
 - **Addresses and metadata are observable.** Anyone able to watch the network

docs/wireguard.md

@@ -22,7 +22,13 @@ system can hand us IP packets, and even that is behind a trait
 | | needs privileges | what it proves |
 |---|---|---|
 | `MemoryTunFactory` | no | handshake, encryption, routing, address ownership |
-| `SystemTunFactory` | `CAP_NET_ADMIN` | traffic actually reaches the OS |
+| `SystemTunFactory`, attaching | none, if the interface was prepared | traffic actually reaches the OS |
+| `SystemTunFactory`, creating | `CAP_NET_ADMIN` | the same, at the cost of a capability |
+
+`SystemTunFactory` attaches to an interface that already exists and only
+creates one when it does not. A persistent interface created by root and owned
+by the user lets the agent run with no privileges at all; see *Running
+unprivileged* in [../README.md](../README.md#running-unprivileged).
 
 [boringtun]: https://docs.rs/boringtun
 [`TunFactory`]: https://docs.rs/tsunagi
@@ -184,5 +190,11 @@ async fn main() -> Result<()> {
 * **Userspace costs CPU.** Kernel WireGuard is faster. A kernel backend could
   return behind the same boundary, but it would give up transport-provided NAT
   traversal unless paired with a local proxy.
-* **The system interface path is barely exercised by the default suite**,
-  because it needs privileges. Everything else about the data plane is.
+* **The agent cannot assign the overlay address itself.** The `tun` crate sets
+  addresses through an IPv4-only ioctl, so the IPv6 overlay address must come
+  from `ip -6 address add` or an equivalent. The agent verifies the address is
+  present, via `/proc/net/if_inet6`, and refuses with the exact command rather
+  than running an interface that could never receive anything. Doing it
+  in-process would mean speaking netlink, which is not implemented.
+* **The system interface path is not exercised by the default suite**, because
+  it needs privileges. Everything else about the data plane is.

src/bin/tsunagi.rs

@@ -45,6 +45,47 @@ enum Command {
     Id(PathArgs),
     /// Joins a network and runs until interrupted.
     Up(UpArgs),
+    /// Prints the one-time privileged setup for the overlay interface.
+    ///
+    /// Run its output once as root, then run `tsunagi up` as an ordinary
+    /// user: the agent attaches to the prepared interface and needs no
+    /// privileges of its own.
+    TunSetup(TunSetupArgs),
+}
+
+#[derive(Debug, Args)]
+struct TunSetupArgs {
+    #[command(flatten)]
+    paths: PathArgs,
+
+    /// Network name, exactly as passed to `tsunagi up`.
+    #[arg(long, short = 'n')]
+    network: String,
+
+    /// The shared secret.
+    #[arg(
+        long,
+        short = 's',
+        env = "TSUNAGI_SECRET",
+        conflicts_with = "secret_file"
+    )]
+    secret: Option<String>,
+
+    /// Read the shared secret from a file instead of the command line.
+    #[arg(long)]
+    secret_file: Option<PathBuf>,
+
+    /// The user that should own the interface. Defaults to the current one.
+    #[arg(long)]
+    user: Option<String>,
+
+    /// Interface name prefix, matching `tsunagi up --wg-prefix`.
+    #[arg(long, default_value = "tsun")]
+    wg_prefix: String,
+
+    /// Interface MTU, matching `tsunagi up --wg-mtu`.
+    #[arg(long)]
+    wg_mtu: Option<u32>,
 }
 
 #[derive(Debug, Args, Clone)]
@@ -72,17 +113,19 @@ impl PathArgs {
 
 /// How much external connectivity machinery the endpoint may use.
 ///
-/// `direct` and `n0` publish this endpoint's addresses, keyed by its endpoint
-/// id, to Number 0's public lookup service, and resolve peers through it.
-/// That is what makes `--peer <endpoint-id>` work without an address.
+/// `direct` and `relay` publish this endpoint's addresses, keyed by its
+/// endpoint id, to the public lookup service run by Number 0 — the company
+/// behind iroh — at `dns.iroh.link`, and resolve peers through it. That is
+/// what makes `--peer <endpoint-id>` work without an address.
 #[derive(Debug, Clone, Copy, ValueEnum)]
 enum Transport {
     /// Loopback and the local network only. Publishes nothing.
     Local,
-    /// Public address lookup, but no relays.
+    /// Public address lookup, direct paths only, no relays.
     Direct,
-    /// iroh's defaults: public address lookup plus the public n0 relays.
-    N0,
+    /// Public address lookup plus public relay fallback. The default.
+    #[value(alias = "n0")]
+    Relay,
 }
 
 impl From<Transport> for TransportPolicy {
@@ -90,7 +133,7 @@ impl From<Transport> for TransportPolicy {
         match value {
             Transport::Local => TransportPolicy::LocalOnly,
             Transport::Direct => TransportPolicy::DirectOnly,
-            Transport::N0 => TransportPolicy::N0Defaults,
+            Transport::Relay => TransportPolicy::N0Defaults,
         }
     }
 }
@@ -122,7 +165,7 @@ struct UpArgs {
     hostname: Option<String>,
 
     /// How much external connectivity to use.
-    #[arg(long, value_enum, default_value_t = Transport::N0)]
+    #[arg(long, value_enum, default_value_t = Transport::Relay)]
     transport: Transport,
 
     /// A peer to contact, as `<endpoint-id>` or `<endpoint-id>@<ip:port>,...`.
@@ -160,22 +203,24 @@ struct UpArgs {
     status_interval: u64,
 }
 
-impl UpArgs {
-    fn load_secret(&self) -> Result<NetworkSecret, Box<dyn std::error::Error>> {
-        let text = match (&self.secret, &self.secret_file) {
-            (Some(secret), _) => secret.clone(),
-            (None, Some(path)) => std::fs::read_to_string(path)?,
-            (None, None) => {
-                return Err("provide --secret, --secret-file or TSUNAGI_SECRET".into());
-            }
-        };
-        let text = text.trim();
-        // The canonical form is preferred, but a raw high-entropy value is
-        // accepted so an existing secret can be reused.
-        match NetworkSecret::decode(text) {
-            Ok(secret) => Ok(secret),
-            Err(_) => Ok(NetworkSecret::from_bytes(text.as_bytes().to_vec())?),
+/// Reads the shared secret from an argument or a file.
+fn load_secret(
+    secret: Option<&str>,
+    secret_file: Option<&std::path::Path>,
+) -> Result<NetworkSecret, Box<dyn std::error::Error>> {
+    let text = match (secret, secret_file) {
+        (Some(secret), _) => secret.to_string(),
+        (None, Some(path)) => std::fs::read_to_string(path)?,
+        (None, None) => {
+            return Err("provide --secret, --secret-file or TSUNAGI_SECRET".into());
         }
+    };
+    let text = text.trim();
+    // The canonical form is preferred, but a raw high-entropy value is
+    // accepted so an existing secret can be reused.
+    match NetworkSecret::decode(text) {
+        Ok(secret) => Ok(secret),
+        Err(_) => Ok(NetworkSecret::from_bytes(text.as_bytes().to_vec())?),
     }
 }
 
@@ -244,9 +289,54 @@ async fn run(command: Command) -> Result<(), Box<dyn std::error::Error>> {
         Command::Doctor(paths) => doctor(paths).await,
         Command::Id(paths) => show_id(paths).await,
         Command::Up(args) => up(args).await,
+        Command::TunSetup(args) => tun_setup(args).await,
     }
 }
 
+/// Works out the interface name and overlay address, then prints the
+/// privileged commands that prepare it.
+///
+/// The address depends on this agent's WireGuard key for the network, so the
+/// key store is opened (and the key created on first use) to compute it.
+async fn tun_setup(args: TunSetupArgs) -> Result<(), Box<dyn std::error::Error>> {
+    use tsunagi::dataplane::wireguard::{
+        DEFAULT_MTU, OVERLAY_PREFIX_LEN, WgKeyStore, interface_name, overlay_address,
+    };
+    use tsunagi::identity::NetworkKeys;
+
+    let name = NetworkName::new(args.network.clone())?;
+    let secret = load_secret(args.secret.as_deref(), args.secret_file.as_deref())?;
+    let paths = args.paths.resolve()?;
+    let network = NetworkKeys::derive(&name, &secret).network_id();
+
+    let store_path = paths.state_dir.join("wireguard").join("wireguard.sqlite");
+    let store = tokio::task::spawn_blocking({
+        let store_path = store_path.clone();
+        move || WgKeyStore::open(store_path)
+    })
+    .await??;
+    let key = tokio::task::spawn_blocking(move || store.load_or_create(network)).await??;
+
+    let interface = interface_name(&args.wg_prefix, network)?;
+    let address = overlay_address(network, &key.public());
+    let mtu = args.wg_mtu.unwrap_or(DEFAULT_MTU);
+    let user = args.user.unwrap_or_else(|| {
+        std::env::var("SUDO_USER")
+            .or_else(|_| std::env::var("USER"))
+            .unwrap_or_else(|_| "$USER".to_string())
+    });
+
+    println!("# Network  {name} ({network})");
+    println!("# Interface {interface}, address {address}/{OVERLAY_PREFIX_LEN}, mtu {mtu}");
+    println!("# Run once as root; then run `tsunagi up` as {user}.\n");
+    println!("sudo ip tuntap add dev {interface} mode tun user {user}");
+    println!("sudo ip -6 address add {address}/{OVERLAY_PREFIX_LEN} dev {interface}");
+    println!("sudo ip link set dev {interface} mtu {mtu} up");
+    println!("\n# To remove it again:");
+    println!("sudo ip link del dev {interface}");
+    Ok(())
+}
+
 async fn show_id(paths: PathArgs) -> Result<(), Box<dyn std::error::Error>> {
     let paths = paths.resolve()?;
     println!("state directory  {}", paths.state_dir.display());
@@ -339,7 +429,7 @@ async fn netwatch_addresses() -> Vec<std::net::IpAddr> {
 
 async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
     let name = NetworkName::new(args.network.clone())?;
-    let secret = args.load_secret()?;
+    let secret = load_secret(args.secret.as_deref(), args.secret_file.as_deref())?;
     let paths = args.paths.resolve()?;
 
     let mut bootstrap: Vec<EndpointAddr> = Vec::new();

src/config.rs

@@ -89,12 +89,19 @@ pub enum TransportPolicy {
     /// Suitable for tests and for fully local deployments.
     #[default]
     LocalOnly,
-    /// No relays, but the n0 DNS/pkarr address lookup is enabled.
-    DirectOnly,
-    /// iroh's standard behaviour, including the public n0 relays.
+    /// Public address lookup, but no relays.
     ///
-    /// Public relays are fine for development; they carry no availability
-    /// guarantee.
+    /// Enables iroh's DNS/pkarr address lookup against the public service run
+    /// by Number 0 (the company behind iroh) at `dns.iroh.link`. This endpoint
+    /// publishes a signed record of its own addresses there, so peers can dial
+    /// it by endpoint id alone.
+    DirectOnly,
+    /// iroh's standard behaviour: public address lookup plus public relays.
+    ///
+    /// Maps to iroh's own `presets::N0`. As well as the address lookup above,
+    /// it uses Number 0's public relay servers as a fallback when a direct
+    /// path cannot be hole punched. They are fine for development and carry no
+    /// availability guarantee.
     N0Defaults,
 }
 

src/dataplane/wireguard/mod.rs

@@ -64,4 +64,4 @@ pub use store::WgKeyStore;
 pub use tun::{MemoryTun, MemoryTunFactory, TunDevice, TunFactory, TunRequest};
 
 #[cfg(feature = "tun-device")]
-pub use tun::SystemTunFactory;
+pub use tun::{SystemTunFactory, interface_exists, interface_has_address, setup_commands};

src/dataplane/wireguard/tun.rs

@@ -182,10 +182,11 @@ impl TunFactory for MemoryTunFactory {
 }
 
 #[cfg(feature = "tun-device")]
-pub use system::SystemTunFactory;
+pub use system::{SystemTunFactory, interface_exists, interface_has_address, setup_commands};
 
 #[cfg(feature = "tun-device")]
 mod system {
+    use std::net::Ipv6Addr;
     use std::sync::Arc;
 
     use bytes::Bytes;
@@ -197,8 +198,18 @@ mod system {
 
     /// A real TUN interface.
     ///
-    /// Creating one needs `CAP_NET_ADMIN` on Linux, or the platform
-    /// equivalent. Failure is reported, never fatal for the agent.
+    /// Two ways to get one, and the difference is who needs privileges:
+    ///
+    /// * **Attach** to an interface that already exists. Needs no privileges
+    ///   at all, as long as the interface was created persistent and owned by
+    ///   this user. This is the recommended way to run the agent unprivileged.
+    /// * **Create** it here, which needs `CAP_NET_ADMIN`.
+    ///
+    /// Either way the overlay address has to be assigned by something
+    /// privileged: assigning an IPv6 address to an interface is not something
+    /// this crate's dependencies can do, so the agent checks that it is there
+    /// and says exactly what to run if it is not, rather than coming up in a
+    /// state where no traffic could ever arrive.
     pub struct SystemTun {
         name: String,
         mtu: u32,
@@ -255,7 +266,53 @@ mod system {
         }
     }
 
-    /// Creates real TUN interfaces.
+    /// Whether an interface of this name exists.
+    pub fn interface_exists(name: &str) -> bool {
+        std::path::Path::new(&format!("/sys/class/net/{name}")).exists()
+    }
+
+    /// Whether an interface already carries an IPv6 address.
+    ///
+    /// Reads `/proc/net/if_inet6`, which needs no privileges.
+    pub fn interface_has_address(name: &str, address: Ipv6Addr) -> bool {
+        let Ok(contents) = std::fs::read_to_string("/proc/net/if_inet6") else {
+            // Cannot tell. Assume it is there rather than block on a guess.
+            return true;
+        };
+        let wanted = hex::encode(address.octets());
+        contents.lines().any(|line| {
+            let mut fields = line.split_whitespace();
+            let addr = fields.next().unwrap_or_default();
+            let iface = fields.last().unwrap_or_default();
+            addr.eq_ignore_ascii_case(&wanted) && iface == name
+        })
+    }
+
+    /// The commands a privileged user runs once to prepare an interface.
+    pub fn setup_commands(request: &TunRequest, user: &str) -> Vec<String> {
+        vec![
+            format!(
+                "sudo ip tuntap add dev {} mode tun user {user}",
+                request.name
+            ),
+            format!(
+                "sudo ip -6 address add {}/{} dev {}",
+                request.address, request.prefix_len, request.name
+            ),
+            format!(
+                "sudo ip link set dev {} mtu {} up",
+                request.name, request.mtu
+            ),
+        ]
+    }
+
+    fn current_user() -> String {
+        std::env::var("SUDO_USER")
+            .or_else(|_| std::env::var("USER"))
+            .unwrap_or_else(|_| "$USER".to_string())
+    }
+
+    /// Opens real TUN interfaces.
     #[derive(Debug, Clone, Default)]
     pub struct SystemTunFactory;
 
@@ -276,27 +333,58 @@ mod system {
             request: TunRequest,
         ) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, PluginError>> {
             Box::pin(async move {
+                let existed = interface_exists(&request.name);
+
                 let mut config = tun::Configuration::default();
-                config.tun_name(&request.name).mtu(request.mtu as u16).up();
-                // The overlay address and its subnet, so the operating system
-                // routes overlay traffic into this interface.
-                let _ = (&request.address, request.prefix_len);
+                config.tun_name(&request.name);
+                if existed {
+                    // Attach only. Reconfiguring an interface somebody
+                    // prepared for us would need exactly the privileges we
+                    // are avoiding, so no ioctl beyond TUNSETIFF is issued.
+                    config.platform_config(|platform| {
+                        platform.ensure_root_privileges(false);
+                    });
+                } else {
+                    // We are creating it, so we configure it.
+                    config.mtu(request.mtu as u16).up();
+                }
+                // Packet information stays off, so reads and writes are raw IP
+                // packets. `ip tuntap add ... mode tun` also defaults to no
+                // packet information, so the flags match when attaching.
 
                 let device = tun::create_as_async(&config).map_err(|err| {
-                    PluginError::Unavailable(format!(
-                        "cannot create the TUN interface `{}`: {err}. \
-                         This needs CAP_NET_ADMIN (try running as root).",
-                        request.name
-                    ))
+                    let hint = if existed {
+                        format!(
+                            "interface `{}` exists but could not be opened: {err}. \
+                             It must be a persistent TUN interface owned by this user.",
+                            request.name
+                        )
+                    } else {
+                        format!(
+                            "cannot create the TUN interface `{}`: {err}. \
+                             Creating one needs CAP_NET_ADMIN. Either prepare it once as root \
+                             (see `tsunagi tun-setup`) and run unprivileged, or grant the \
+                             capability.",
+                            request.name
+                        )
+                    };
+                    PluginError::Unavailable(hint)
                 })?;
 
-                // The name was requested explicitly; creation fails rather
-                // than silently picking another one.
-                let name = request.name.clone();
-                let (reader, writer) = tokio::io::split(device);
+                // Without its overlay address the interface can never receive
+                // anything, so say so instead of pretending to be up.
+                if !interface_has_address(&request.name, request.address) {
+                    let commands = setup_commands(&request, &current_user()).join("\n  ");
+                    return Err(PluginError::Unavailable(format!(
+                        "interface `{}` has no {} address. Assigning an IPv6 address needs \
+                         privileges. Run:\n  {commands}",
+                        request.name, request.address
+                    )));
+                }
 
+                let (reader, writer) = tokio::io::split(device);
                 Ok(Arc::new(SystemTun {
-                    name,
+                    name: request.name,
                     mtu: request.mtu,
                     reader: Mutex::new(reader),
                     writer: Mutex::new(writer),