Remove tun-setup and the attach path it served

The managed interface supersedes both. They go together because apart
they are useless: attaching needs an interface somebody prepared, and
tun-setup existed only to say how to prepare one.

This also corrects what the last commit's README claimed. It said the
manual route was needed on macOS and Windows; it was not, and could not
be. The recipe printed Linux `ip` commands, and a persistent TUN that a
second process can attach to is a Linux concept — macOS creates a utun
by opening a control socket and there is nothing to hand over. So those
platforms were never served by this path, and their honest state is that
a real interface waits on a provisioner, with --no-tun meanwhile.

Gone with it: the interface-existence check, the /proc/net/if_inet6
address inspection and its DAD flag decoding, and the --interface flag,
which had one mode left.

Kept: the check that the allocated IPv4 address is really on a local
interface. The agent now assigns that address itself, so the check is no
longer telling a user what to run — it verifies the outcome instead of
trusting it, which is worth keeping precisely because the assumptions
around Linux address behaviour have been wrong here more than once. Its
message says which interface should have had the address rather than a
command to run.

Boxing Up(UpArgs) is fallout: TunSetupArgs had been masking how much
larger that variant is than its siblings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
2026-09-21 14:46:39 +01:00
co-authored by Claude Opus 5
parent b23e832a73
commit 4759e47e31
10 changed files with 106 additions and 669 deletions
+19 -47
View File
@@ -146,9 +146,8 @@ No vote is involved — see
[docs/sync-model.md](docs/sync-model.md). [docs/sync-model.md](docs/sync-model.md).
Because the address is allocated at run time rather than derived, it is not Because the address is allocated at run time rather than derived, it is not
known until the agent has started and agreed with its peers. On Linux the known until the agent has started and agreed with its peers. The agent then
agent assigns it itself as soon as it has one; elsewhere `tsunagi tun-setup` assigns it to the interface itself.
prints the `ip address add` line once the claim is in `state.sqlite`.
## Privileges ## Privileges
@@ -192,64 +191,37 @@ it carried. Two things are never touched:
Both of those refuse with an explanation rather than guessing. Both of those refuse with an explanation rather than guessing.
### Running without the capability Two settings the manual recipe used to need are gone with it.
`keep_addr_on_down` existed only because an interface nobody held open lost
carrier and had its IPv6 addresses flushed, and `nodad` only because duplicate
address detection can never finish without carrier. An interface held open for
its whole life has carrier for its whole life.
`--interface attach` (or `auto`, which falls back on its own) opens an ### The MTU is 1280
interface prepared beforehand and needs **no privileges at all**. Ask the
agent what to run:
```bash That is the minimum IPv6 requires (RFC 8200), and Linux enforces it by
tsunagi tun-setup --network lab --secret "$SECRET" disabling IPv6 outright on an interface below it — the per-device
```
```text
# Interface tsunjwc6dcrtmo5, address fd80:1210:f724:f620:d1bb:f982:3b6e:19bd/64, mtu 1280
# Run once as root; then run `tsunagi up` as ab.
sudo ip tuntap add dev tsunjwc6dcrtmo5 mode tun user ab
sudo ip link set dev tsunjwc6dcrtmo5 mtu 1280 up
sudo sysctl -qw net.ipv6.conf.tsunjwc6dcrtmo5.keep_addr_on_down=1
sudo ip -6 address add fd80:1210:f724:f620:d1bb:f982:3b6e:19bd/64 dev tsunjwc6dcrtmo5 nodad
```
`user ab` is the point: the interface is persistent and owned by that user, so
`tsunagi up` afterwards opens it with no privileges and no capabilities.
The last two settings are what the managed path does not need. A persistent
TUN has **no carrier** until a process attaches to it; Linux flushes IPv6
addresses from an interface that loses carrier unless `keep_addr_on_down` is
set, and duplicate address detection can never finish without carrier, so the
address would sit there tentative and unusable without `nodad`. An interface
the agent creates and holds open has carrier for its whole life, so neither
applies. IPv4 needs neither in either case: Linux keeps IPv4 addresses across
carrier loss and IPv4 has no duplicate address detection.
With IPv4 enabled, `tun-setup` adds an `ip address add` line once there is an
address to print — it reads the signed claim back out of `state.sqlite`, which
does not disturb a running agent.
The MTU is 1280 because that is the minimum IPv6 requires (RFC 8200). Linux
disables IPv6 entirely on an interface below it — the per-device
`/proc/sys/net/ipv6` entries vanish and adding an address fails with `/proc/sys/net/ipv6` entries vanish and adding an address fails with
`Invalid argument` — so a smaller MTU cannot work at all. The agent refuses `Invalid argument`. A smaller MTU cannot work at all, so the agent refuses one
one rather than letting it fail later. rather than letting it fail later. See
[docs/wireguard.md](docs/wireguard.md#mtu) for the ceiling that pushes back
from the other side.
### Summary ### Summary
| approach | agent runs as | notes | | approach | agent runs as | notes |
|---|---|---| |---|---|---|
| `setcap cap_net_admin+p` | ordinary user, one capability | recommended on Linux: nothing to prepare, nothing left behind. Lost on every rebuild or copy of the binary. | | `setcap cap_net_admin+p` | ordinary user, one capability | recommended: nothing to prepare, nothing left behind. Lost on every rebuild or copy of the binary. |
| systemd service | `User=`, `AmbientCapabilities=CAP_NET_ADMIN` | the same, for an installed service | | systemd service | `User=`, `AmbientCapabilities=CAP_NET_ADMIN` | the same, for an installed service |
| `tsunagi tun-setup` then `--interface attach` | ordinary user, no capabilities | one privileged setup per host; needed on macOS and Windows, where no provisioner is implemented yet |
| `sudo tsunagi up` | root | everything works, nothing is isolated | | `sudo tsunagi up` | root | everything works, nothing is isolated |
| `--no-tun` | ordinary user, no capabilities | tunnels run and handshake, traffic never reaches the OS | | `--no-tun` | ordinary user, no capabilities | tunnels run and handshake, traffic never reaches the OS |
**Not implemented yet.** macOS and Windows have no provisioner: both need **Not implemented yet.** macOS and Windows have no provisioner: both need
real platform work — `utun` and `SystemConfiguration` on one, the IP Helper real platform work — `utun` and `SystemConfiguration` on one, the IP Helper
API and a Wintun adapter on the other. On those the agent says so and falls API and a Wintun adapter on the other. There the agent says so and `--no-tun`
back to attaching to a prepared interface. The decision logic that says *what* is the way to run it; the control plane and the tunnels are unaffected. The
to change is shared and tested on every platform; only the execution is decision logic that says *what* to change is shared and tested on every
per-platform. platform, so only the execution is left to write.
## Checks ## Checks
+17 -15
View File
@@ -22,13 +22,13 @@ system can hand us IP packets, and even that is behind a trait
| | needs privileges | what it proves | | | needs privileges | what it proves |
|---|---|---| |---|---|---|
| `MemoryTunFactory` | no | handshake, encryption, routing, address ownership | | `MemoryTunFactory` | no | handshake, encryption, routing, address ownership |
| `SystemTunFactory`, attaching | none, if the interface was prepared | traffic actually reaches the OS | | `MockProvisioner` | no | the above, plus what would have been done to the host |
| `SystemTunFactory`, creating | `CAP_NET_ADMIN` | the same, at the cost of a capability | | `ManagedTunFactory` | `CAP_NET_ADMIN` | traffic actually reaches the OS |
`SystemTunFactory` attaches to an interface that already exists and only `ManagedTunFactory` creates the interface and configures it; see
creates one when it does not. A persistent interface created by root and owned *Provisioning the interface* below and *Privileges* in
by the user lets the agent run with no privileges at all; see *Running [../README.md](../README.md#privileges). With no capability the agent runs
unprivileged* in [../README.md](../README.md#running-unprivileged). with `--no-tun`: everything but the last hop into the kernel still works.
[boringtun]: https://docs.rs/boringtun [boringtun]: https://docs.rs/boringtun
[`TunFactory`]: https://docs.rs/tsunagi [`TunFactory`]: https://docs.rs/tsunagi
@@ -126,14 +126,16 @@ wins, and the agent adopts it. So the flag matters for whoever starts the
network and is harmless afterwards. Pass `--ipv4-range none` for an IPv6-only network and is harmless afterwards. Pass `--ipv4-range none` for an IPv6-only
overlay. overlay.
Putting the address on the interface still needs privileges, and the agent The agent puts the address on the interface itself, as soon as the network has
cannot do it. Since the address is only known once the agent has agreed with agreed on it — no restart, and the interface is not recreated, which would
its peers, `tsunagi tun-setup` cannot print it up front either. So the agent drop every tunnel riding on it.
checks whether the address is assigned anywhere on the host — by binding a UDP
socket to it, which needs no privileges — and reports the exact command until It then checks that it is really there, by binding a UDP socket to it, which
it is. This matters: with the wrong address on the interface, packets leave needs no privileges. That check is deliberately independent of the code that
with the wrong source and every peer drops them as not belonging to us, which did the assigning: the failure it guards against is a silent one. With the
looks like a broken network rather than a missing command. wrong address on the interface, packets leave with the wrong source and every
peer drops them as not belonging to us, which looks like a broken network
rather than a broken assumption.
A release tombstone exists in the record type and merges correctly, but A release tombstone exists in the record type and merges correctly, but
nothing emits one yet, so an address stays claimed until the network is nothing emits one yet, so an address stays claimed until the network is
@@ -224,7 +226,7 @@ use tsunagi::{Agent, Result};
async fn main() -> Result<()> { async fn main() -> Result<()> {
let paths = StoragePaths::user_default()?; let paths = StoragePaths::user_default()?;
// MemoryTunFactory needs no privileges; swap in SystemTunFactory for a // MemoryTunFactory needs no privileges; swap in ManagedTunFactory for a
// real interface. // real interface.
let plugin = WireguardPlugin::open( let plugin = WireguardPlugin::open(
WireguardConfig::new(paths.state_dir.join("wireguard")), WireguardConfig::new(paths.state_dir.join("wireguard")),
+21 -212
View File
@@ -45,15 +45,12 @@ enum Command {
/// Shows this device's identity without joining anything. /// Shows this device's identity without joining anything.
Id(PathArgs), Id(PathArgs),
/// Joins a network and runs until interrupted. /// Joins a network and runs until interrupted.
Up(UpArgs), // Boxed: it is much larger than the other variants, and every command
// but this one would otherwise pay for its size. A `//` comment, not a
// `///` one, or clap would print it as help.
Up(Box<UpArgs>),
/// Asks a running agent what it is doing. /// Asks a running agent what it is doing.
Status(StatusArgs), Status(StatusArgs),
/// Prints the one-time privileged setup for the overlay interface.
///
/// Run its output once as root, then run `tsunagi up` as an ordinary
/// user: the agent attaches to the prepared interface and needs no
/// privileges of its own.
TunSetup(TunSetupArgs),
} }
#[derive(Debug, Args)] #[derive(Debug, Args)]
@@ -66,45 +63,6 @@ struct StatusArgs {
control_socket: Option<PathBuf>, control_socket: Option<PathBuf>,
} }
#[derive(Debug, Args)]
struct TunSetupArgs {
#[command(flatten)]
paths: PathArgs,
/// Network name, exactly as passed to `tsunagi up`.
#[arg(long, short = 'n')]
network: String,
/// The shared secret.
#[arg(
long,
short = 's',
env = "TSUNAGI_SECRET",
conflicts_with = "secret_file"
)]
secret: Option<String>,
/// Read the shared secret from a file instead of the command line.
#[arg(long)]
secret_file: Option<PathBuf>,
/// The user that should own the interface. Defaults to the current one.
#[arg(long)]
user: Option<String>,
/// Interface name prefix, matching `tsunagi up --wg-prefix`.
#[arg(long, default_value = "tsun")]
wg_prefix: String,
/// Interface MTU, matching `tsunagi up --wg-mtu`. At least 1280.
#[arg(long)]
wg_mtu: Option<u32>,
/// Match `tsunagi up --ipv4-range`.
#[arg(long, value_name = "CIDR")]
ipv4_range: Option<String>,
}
/// Resolves the IPv4 overlay range from the flag. /// Resolves the IPv4 overlay range from the flag.
/// ///
/// Absent means the built-in default. A network that already settled on /// Absent means the built-in default. A network that already settled on
@@ -224,15 +182,6 @@ struct UpArgs {
#[arg(long)] #[arg(long)]
no_tun: bool, no_tun: bool,
/// How the overlay interface is obtained.
///
/// `managed` has the agent create and configure it itself, which needs
/// CAP_NET_ADMIN and cleans up on exit. `attach` opens an interface that
/// was prepared beforehand (see `tsunagi tun-setup`) and needs no
/// privileges. `auto` manages it when it can and attaches when it cannot.
#[arg(long, value_enum, default_value_t = InterfaceMode::Auto)]
interface: InterfaceMode,
/// Interface name prefix for the WireGuard data plane. /// Interface name prefix for the WireGuard data plane.
#[arg(long, default_value = "tsun")] #[arg(long, default_value = "tsun")]
wg_prefix: String, wg_prefix: String,
@@ -347,37 +296,11 @@ async fn run(command: Command) -> Result<(), Box<dyn std::error::Error>> {
} }
Command::Doctor(paths) => doctor(paths).await, Command::Doctor(paths) => doctor(paths).await,
Command::Id(paths) => show_id(paths).await, Command::Id(paths) => show_id(paths).await,
Command::Up(args) => up(args).await, Command::Up(args) => up(*args).await,
Command::TunSetup(args) => tun_setup(args).await,
Command::Status(args) => status(args).await, Command::Status(args) => status(args).await,
} }
} }
/// The IPv4 address this agent has already been allocated, if any.
///
/// Read straight from the mandatory state store. Opening it for reading does
/// not take the directory lock, so this works while the agent is running.
/// Records are verified here too: the database is not a trust boundary.
fn allocated_ipv4(
paths: &StoragePaths,
network: tsunagi::NetworkId,
) -> Result<Option<(std::net::Ipv4Addr, u8)>, Box<dyn std::error::Error>> {
use tsunagi::state::RecordBody;
use tsunagi::storage::StateStore;
let store = StateStore::open(paths.state_db())?;
let author = store.load_or_create_device_identity()?.endpoint_id();
for record in store.signed_records(network)? {
if record.author != *author.as_bytes() || record.verify(network).is_err() {
continue;
}
if let RecordBody::Ipv4Claim { address, range } = record.body {
return Ok(Some((address, range.prefix_len)));
}
}
Ok(None)
}
/// Path of the local control socket for a state directory. /// Path of the local control socket for a state directory.
fn control_socket(paths: &StoragePaths, override_path: Option<&PathBuf>) -> PathBuf { fn control_socket(paths: &StoragePaths, override_path: Option<&PathBuf>) -> PathBuf {
match override_path { match override_path {
@@ -404,86 +327,6 @@ async fn status(args: StatusArgs) -> Result<(), Box<dyn std::error::Error>> {
Ok(()) Ok(())
} }
/// Works out the interface name and overlay address, then prints the
/// privileged commands that prepare it.
///
/// The address depends on this agent's WireGuard key for the network, so the
/// key store is opened (and the key created on first use) to compute it.
async fn tun_setup(args: TunSetupArgs) -> Result<(), Box<dyn std::error::Error>> {
use tsunagi::dataplane::wireguard::{
DEFAULT_MTU, OVERLAY_PREFIX_LEN, WgKeyStore, interface_name, overlay_address,
};
use tsunagi::identity::NetworkKeys;
let name = NetworkName::new(args.network.clone())?;
let secret = load_secret(args.secret.as_deref(), args.secret_file.as_deref())?;
let paths = args.paths.resolve()?;
let network = NetworkKeys::derive(&name, &secret).network_id();
let store_path = paths.state_dir.join("wireguard").join("wireguard.sqlite");
let store = tokio::task::spawn_blocking({
let store_path = store_path.clone();
move || WgKeyStore::open(store_path)
})
.await??;
let key = tokio::task::spawn_blocking(move || store.load_or_create(network)).await??;
let interface = interface_name(&args.wg_prefix, network)?;
let address = overlay_address(network, &key.public());
let ipv4_range = resolve_ipv4_range(args.ipv4_range.as_ref())?;
let mtu = args.wg_mtu.unwrap_or(DEFAULT_MTU);
let user = args.user.unwrap_or_else(|| {
std::env::var("SUDO_USER")
.or_else(|_| std::env::var("USER"))
.unwrap_or_else(|_| "$USER".to_string())
});
println!("# Network {name} ({network})");
println!("# Interface {interface}, address {address}/{OVERLAY_PREFIX_LEN}, mtu {mtu}");
// The IPv4 address is allocated at run time, so it can only be shown once
// the agent has one. Reading the state store does not disturb a running
// agent: the directory lock belongs to the agent, not to this reader.
let allocated_v4 = ipv4_range.and_then(|_| allocated_ipv4(&paths, network).ok().flatten());
match (ipv4_range, allocated_v4) {
(Some(_), Some((address, prefix_len))) => {
println!("# IPv4 overlay address {address}/{prefix_len}, allocated and signed");
}
(Some(_), None) => {
println!(
"# IPv4 is allocated once the agent runs and agrees with its peers, so\n\
# there is nothing to print yet. Start `tsunagi up`: it prints the exact\n\
# `ip address add` command for the address it was given, and this\n\
# command will include it from then on."
);
}
(None, _) => {}
}
println!("# Run once as root; then run `tsunagi up` as {user}.");
println!(
"#\n\
# keep_addr_on_down matters: a persistent TUN interface has no carrier\n\
# until a process attaches, and Linux flushes IPv6 addresses from an\n\
# interface that loses carrier unless it is set. `nodad` matters for the\n\
# same reason: duplicate address detection can never finish without a\n\
# carrier, leaving the address tentative and unusable.\n"
);
println!("sudo ip tuntap add dev {interface} mode tun user {user}");
println!("sudo ip link set dev {interface} mtu {mtu} up");
println!("sudo sysctl -qw net.ipv6.conf.{interface}.keep_addr_on_down=1");
println!("sudo ip -6 address add {address}/{OVERLAY_PREFIX_LEN} dev {interface} nodad");
if let Some((address, prefix_len)) = allocated_v4 {
// IPv4 addresses are not flushed when an interface loses carrier, so
// this one needs none of the treatment IPv6 does.
println!("sudo ip address add {address}/{prefix_len} dev {interface}");
}
println!("\n# To check it afterwards:");
println!("ip addr show dev {interface}");
println!("\n# To remove it again:");
println!("sudo ip link del dev {interface}");
Ok(())
}
async fn show_id(paths: PathArgs) -> Result<(), Box<dyn std::error::Error>> { async fn show_id(paths: PathArgs) -> Result<(), Box<dyn std::error::Error>> {
let paths = paths.resolve()?; let paths = paths.resolve()?;
println!("state directory {}", paths.state_dir.display()); println!("state directory {}", paths.state_dir.display());
@@ -555,9 +398,9 @@ async fn doctor(paths: PathArgs) -> Result<(), Box<dyn std::error::Error>> {
} }
Privilege::Missing(reason) => { Privilege::Missing(reason) => {
println!(" privileges no CAP_NET_ADMIN ({reason})"); println!(" privileges no CAP_NET_ADMIN ({reason})");
println!(" interface must be prepared first; run `tsunagi tun-setup`"); println!(" interface cannot be created; run with `--no-tun` meanwhile");
println!( println!(
" to manage it {}", " to grant it {}",
Privilege::how_to_grant(&program_path()) Privilege::how_to_grant(&program_path())
); );
} }
@@ -566,7 +409,7 @@ async fn doctor(paths: PathArgs) -> Result<(), Box<dyn std::error::Error>> {
" privileges managing interfaces is not implemented on {} yet", " privileges managing interfaces is not implemented on {} yet",
std::env::consts::OS std::env::consts::OS
); );
println!(" interface must be prepared first; run `tsunagi tun-setup`"); println!(" interface cannot be created; run with `--no-tun`");
} }
} }
} }
@@ -634,7 +477,7 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
let tun_factory: Arc<dyn TunFactory> = if args.no_tun { let tun_factory: Arc<dyn TunFactory> = if args.no_tun {
Arc::new(MemoryTunFactory::new()) Arc::new(MemoryTunFactory::new())
} else { } else {
system_tun_factory(args.interface)? system_tun_factory()?
}; };
let mut wg = WireguardConfig::new(paths.state_dir.join("wireguard")) let mut wg = WireguardConfig::new(paths.state_dir.join("wireguard"))
.with_interface_prefix(args.wg_prefix.clone()); .with_interface_prefix(args.wg_prefix.clone());
@@ -866,47 +709,14 @@ async fn stop_signal() -> &'static str {
} }
} }
/// How the overlay interface is obtained. /// Builds the interface factory.
#[derive(Debug, Clone, Copy, PartialEq, Eq, clap::ValueEnum)]
enum InterfaceMode {
/// Manage it when possible, attach to a prepared one otherwise.
Auto,
/// Create and configure it in process. Needs CAP_NET_ADMIN.
Managed,
/// Open an interface prepared beforehand. Needs no privileges.
Attach,
}
/// Builds the interface factory for the chosen mode.
/// ///
/// The managed path is preferred because it is the one that cleans up after /// One path: the agent creates and configures the interface itself. It is
/// itself: the interface is tied to an open file descriptor, so it goes away /// also the one that cleans up after itself, because the interface is tied to
/// when the agent does, however the agent goes away. /// an open file descriptor and goes away with the agent, however the agent
#[cfg(feature = "tun-device")] /// goes away.
fn system_tun_factory(
mode: InterfaceMode,
) -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
use tsunagi::dataplane::wireguard::SystemTunFactory;
if mode == InterfaceMode::Attach {
return Ok(Arc::new(SystemTunFactory::new()));
}
match managed_tun_factory() {
Ok(factory) => Ok(factory),
Err(err) if mode == InterfaceMode::Managed => Err(err),
Err(err) => {
tracing::warn!(
"{err} Falling back to attaching to a prepared interface; \
`tsunagi tun-setup` prints how to make one."
);
Ok(Arc::new(SystemTunFactory::new()))
}
}
}
#[cfg(all(feature = "tun-device", target_os = "linux"))] #[cfg(all(feature = "tun-device", target_os = "linux"))]
fn managed_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> { fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
use tsunagi::dataplane::wireguard::{ManagedTunFactory, NetlinkProvisioner}; use tsunagi::dataplane::wireguard::{ManagedTunFactory, NetlinkProvisioner};
let provisioner = NetlinkProvisioner::new()?; let provisioner = NetlinkProvisioner::new()?;
Ok(Arc::new(ManagedTunFactory::new(Arc::new(provisioner)))) Ok(Arc::new(ManagedTunFactory::new(Arc::new(provisioner))))
@@ -914,21 +724,20 @@ fn managed_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Erro
/// There is no provisioner for this platform yet. /// There is no provisioner for this platform yet.
/// ///
/// Refused here rather than at the first packet, so `auto` falls back to /// Refused here rather than at the first packet, and with the one thing that
/// attaching and `--interface managed` says plainly why it cannot. /// does work on every platform named.
#[cfg(all(feature = "tun-device", not(target_os = "linux")))] #[cfg(all(feature = "tun-device", not(target_os = "linux")))]
fn managed_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> { fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
Err(format!( Err(format!(
"managing the overlay interface is not implemented on {} yet.", "managing the overlay interface is not implemented on {} yet. \
Run with `--no-tun` to keep the tunnels off the operating system.",
std::env::consts::OS std::env::consts::OS
) )
.into()) .into())
} }
#[cfg(not(feature = "tun-device"))] #[cfg(not(feature = "tun-device"))]
fn system_tun_factory( fn system_tun_factory() -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
_mode: InterfaceMode,
) -> Result<Arc<dyn TunFactory>, Box<dyn std::error::Error>> {
Err("this build has no interface support; rebuild with the `tun-device` feature or pass --no-tun".into()) Err("this build has no interface support; rebuild with the `tun-device` feature or pass --no-tun".into())
} }
+3 -10
View File
@@ -35,10 +35,9 @@
//! The only part that needs privileges is the packet interface. With //! The only part that needs privileges is the packet interface. With
//! [`tun::MemoryTunFactory`] the whole data plane — handshake, encryption, //! [`tun::MemoryTunFactory`] the whole data plane — handshake, encryption,
//! routing, address ownership — runs and is tested with no privileges at all. //! routing, address ownership — runs and is tested with no privileges at all.
//! For real traffic there are two ways in: [`provision::ManagedTunFactory`], //! For real traffic there is [`provision::ManagedTunFactory`], where the
//! where the agent creates and configures the interface itself over netlink //! agent creates and configures the interface itself over netlink and
//! and removes it again on exit, and `SystemTunFactory`, which attaches to an //! removes it again on exit.
//! interface somebody else prepared and needs no privileges.
//! //!
//! See `docs/wireguard.md` for the full picture. //! See `docs/wireguard.md` for the full picture.
@@ -74,9 +73,3 @@ pub use tun::{MemoryTun, MemoryTunFactory, TunDevice, TunFactory, TunRequest, ad
#[cfg(all(feature = "tun-device", target_os = "linux"))] #[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use provision::NetlinkProvisioner; pub use provision::NetlinkProvisioner;
#[cfg(feature = "tun-device")]
pub use tun::{
Assigned, SystemTunFactory, interface_addresses, interface_exists, parse_if_inet6,
setup_commands,
};
+11 -9
View File
@@ -594,11 +594,13 @@ impl Worker {
} }
device.retain_peers(&wanted); device.retain_peers(&wanted);
// The address is allocated at run time, but putting it on the // The agent assigns this address itself, so finding it absent means
// interface needs privileges we do not have. Without it the kernel // the assignment did not take — something outside removed it, or the
// sends our packets with the wrong source address and every peer // provisioner reported a success it did not achieve. Left unsaid it
// drops them, which looks like a broken network rather than a missing // looks like a broken network: the kernel would send packets with the
// command. So say exactly what is wrong. // wrong source address and every peer would drop them. So it is
// checked rather than assumed, because the assumption is exactly the
// kind that has been wrong here before.
let missing_v4 = match (own_v4, interface.as_deref(), range) { let missing_v4 = match (own_v4, interface.as_deref(), range) {
(Some(address), Some(interface), Some(range)) (Some(address), Some(interface), Some(range))
if !super::tun::address_is_local(IpAddr::V4(address)) => if !super::tun::address_is_local(IpAddr::V4(address)) =>
@@ -622,11 +624,11 @@ impl Worker {
self.report( self.report(
network, network,
format!( format!(
"this agent was allocated {address} but that address is not on any \ "this agent was allocated {address}/{} but the address is not on any \
interface, so IPv4 cannot work: packets would leave with the wrong \ interface, so IPv4 cannot work: packets would leave with the wrong \
source and every peer would drop them. Run:\n \ source and every peer would drop them. It should have been assigned \
sudo ip address add {address}/{} dev {interface}\n \ to `{interface}` automatically; check whether something else removed \
and remove any other address of that range from it.", it.",
range.prefix_len range.prefix_len
), ),
); );
+1 -1
View File
@@ -153,7 +153,7 @@ impl NetlinkProvisioner {
fn create_device(&self, plan: &InterfacePlan) -> Result<Arc<dyn TunDevice>, PluginError> { fn create_device(&self, plan: &InterfacePlan) -> Result<Arc<dyn TunDevice>, PluginError> {
let request = TunRequest::bare(plan.name.clone(), plan.mtu); let request = TunRequest::bare(plan.name.clone(), plan.mtu);
let _guard = NetAdmin::acquire()?; let _guard = NetAdmin::acquire()?;
super::super::tun::open_tun(&request, false) super::super::tun::open_tun(&request)
} }
} }
@@ -50,8 +50,9 @@ impl Privilege {
pub fn how_to_grant(program: &str) -> String { pub fn how_to_grant(program: &str) -> String {
format!( format!(
"Grant it once with `sudo setcap cap_net_admin+p {program}` \ "Grant it once with `sudo setcap cap_net_admin+p {program}` \
and the agent manages its own interface. Without it, prepare the \ and the agent manages its own interface. Without it, run with \
interface by hand with `tsunagi tun-setup`." `--no-tun`: the tunnels still form, they just do not reach the \
operating system."
) )
} }
} }
@@ -153,7 +154,7 @@ mod tests {
fn the_grant_instructions_name_the_program() { fn the_grant_instructions_name_the_program() {
let text = Privilege::how_to_grant("/usr/local/bin/tsunagi"); let text = Privilege::how_to_grant("/usr/local/bin/tsunagi");
assert!(text.contains("setcap cap_net_admin+p /usr/local/bin/tsunagi")); assert!(text.contains("setcap cap_net_admin+p /usr/local/bin/tsunagi"));
assert!(text.contains("tun-setup"), "the fallback is offered too"); assert!(text.contains("--no-tun"), "the fallback is offered too");
} }
#[test] #[test]
@@ -34,8 +34,8 @@ impl UnsupportedProvisioner {
fn refusal(&self) -> PluginError { fn refusal(&self) -> PluginError {
PluginError::Unavailable(format!( PluginError::Unavailable(format!(
"managing the overlay interface is not implemented on {} yet. \ "managing the overlay interface is not implemented on {} yet. \
Prepare the interface by hand `tsunagi tun-setup` prints what to run \ Run with `--no-tun` until it is: the tunnels still form, they just \
and the agent will attach to it.", do not reach the operating system.",
self.platform self.platform
)) ))
} }
@@ -72,7 +72,7 @@ mod tests {
let plan = InterfacePlan::new("tsuntest", 1280, Vec::new()); let plan = InterfacePlan::new("tsuntest", 1280, Vec::new());
let err = provisioner.reconcile(&plan).await.unwrap_err(); let err = provisioner.reconcile(&plan).await.unwrap_err();
let message = err.to_string(); let message = err.to_string();
assert!(message.contains("tun-setup"), "{message}"); assert!(message.contains("--no-tun"), "{message}");
assert!(message.contains(std::env::consts::OS), "{message}"); assert!(message.contains(std::env::consts::OS), "{message}");
provisioner.remove("tsuntest").await.unwrap(); provisioner.remove("tsuntest").await.unwrap();
+23 -367
View File
@@ -10,8 +10,8 @@
//! is what the test suite uses, so the entire data plane — handshake, //! is what the test suite uses, so the entire data plane — handshake,
//! encryption, routing — is exercised without touching the host. //! encryption, routing — is exercised without touching the host.
//! * `SystemTun`, behind the `tun-device` feature, is a real TUN interface. //! * `SystemTun`, behind the `tun-device` feature, is a real TUN interface.
//! Creating one needs `CAP_NET_ADMIN`; attaching to one somebody else //! Creating one needs `CAP_NET_ADMIN`, and it is
//! prepared needs nothing. //! [`provision`](super::provision) that holds that and creates it.
use std::net::Ipv6Addr; use std::net::Ipv6Addr;
use std::sync::Arc; use std::sync::Arc;
@@ -238,43 +238,27 @@ impl TunFactory for MemoryTunFactory {
#[cfg(feature = "tun-device")] #[cfg(feature = "tun-device")]
pub(crate) use system::open_tun; pub(crate) use system::open_tun;
#[cfg(feature = "tun-device")]
pub use system::{
Assigned, SystemTunFactory, interface_addresses, interface_exists, parse_if_inet6,
setup_commands,
};
#[cfg(feature = "tun-device")] #[cfg(feature = "tun-device")]
mod system { mod system {
use std::net::Ipv6Addr;
use std::sync::Arc; use std::sync::Arc;
use bytes::Bytes; use bytes::Bytes;
use tokio::sync::Mutex; use tokio::sync::Mutex;
use super::{TunDevice, TunFactory, TunRequest}; use super::{TunDevice, TunRequest};
use crate::BoxFuture; use crate::BoxFuture;
use crate::dataplane::PluginError; use crate::dataplane::PluginError;
/// A real TUN interface. /// A real TUN interface.
/// ///
/// Two ways to get one, and the difference is who needs privileges: /// Created by opening `/dev/net/tun`, which needs `CAP_NET_ADMIN` and is
/// why [`open_tun`] is only ever called from
/// [`provision`](super::super::provision), where that capability is
/// raised for the length of the call and no longer.
/// ///
/// * **Attach** to an interface that already exists. Needs no privileges /// It is deliberately **not** made persistent, so the kernel removes the
/// at all, as long as the interface was created persistent and owned by /// interface when this value is dropped — however the process ends.
/// this user. This is the recommended way to run the agent unprivileged.
/// * **Create** it here, which needs `CAP_NET_ADMIN`.
///
/// `SystemTunFactory` is the **attach** path, for a host where the agent
/// has no privileges at all: the interface and its addresses were put
/// there by something else, so it checks they are present and says
/// exactly what to run if they are not, rather than coming up in a state
/// where no traffic could ever arrive.
///
/// The other path is
/// [`ManagedTunFactory`](super::super::provision::ManagedTunFactory),
/// where the agent creates and configures the interface itself. That is
/// the default on Linux and needs no preparation at all.
pub struct SystemTun { pub struct SystemTun {
name: String, name: String,
mtu: u32, mtu: u32,
@@ -331,168 +315,18 @@ mod system {
} }
} }
/// Whether an interface of this name exists. /// Creates the TUN interface by opening it.
pub fn interface_exists(name: &str) -> bool {
std::path::Path::new(&format!("/sys/class/net/{name}")).exists()
}
/// `IFA_F_TENTATIVE`: the address is not usable until DAD finishes, which
/// never happens on an interface with no carrier.
const IFA_F_TENTATIVE: u32 = 0x40;
/// `IFA_F_DADFAILED`: duplicate address detection rejected it.
const IFA_F_DADFAILED: u32 = 0x08;
/// One IPv6 address assigned to an interface.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Assigned {
/// The address.
pub address: Ipv6Addr,
/// Raw `IFA_F_*` flags as the kernel reports them.
pub flags: u32,
}
impl Assigned {
/// Whether the address can actually carry traffic.
pub fn is_usable(&self) -> bool {
self.flags & (IFA_F_TENTATIVE | IFA_F_DADFAILED) == 0
}
/// A short explanation when it cannot.
pub fn why_unusable(&self) -> Option<&'static str> {
if self.flags & IFA_F_DADFAILED != 0 {
Some("duplicate address detection failed")
} else if self.flags & IFA_F_TENTATIVE != 0 {
Some(
"still tentative; duplicate address detection cannot finish \
on an interface with no carrier, so add it with `nodad`",
)
} else {
None
}
}
}
/// Parses the IPv6 addresses of one interface out of `/proc/net/if_inet6`.
/// ///
/// Each line is `<32 hex address> <ifindex> <prefixlen> <scope> <flags> /// Synchronous, and deliberately so: the caller holds a capability guard
/// <device>`, all hexadecimal. /// across this call, and such a guard must not span an `await` because
pub fn parse_if_inet6(contents: &str, name: &str) -> Vec<Assigned> { /// Linux capabilities are per thread.
contents pub(crate) fn open_tun(request: &TunRequest) -> Result<Arc<dyn TunDevice>, PluginError> {
.lines()
.filter_map(|line| {
let fields: Vec<&str> = line.split_whitespace().collect();
if fields.len() < 6 || fields[5] != name {
return None;
}
let raw = <[u8; 16]>::try_from(hex::decode(fields[0]).ok()?.as_slice()).ok()?;
Some(Assigned {
address: Ipv6Addr::from(raw),
flags: u32::from_str_radix(fields[4], 16).unwrap_or(0),
})
})
.collect()
}
/// The IPv6 addresses of an interface, or `None` if that cannot be read.
///
/// Reads `/proc/net/if_inet6`, which needs no privileges.
pub fn interface_addresses(name: &str) -> Option<Vec<Assigned>> {
let contents = std::fs::read_to_string("/proc/net/if_inet6").ok()?;
Some(parse_if_inet6(&contents, name))
}
/// What is wrong with an interface's addressing, if anything.
pub(crate) fn check_address(name: &str, wanted: Ipv6Addr) -> Result<(), String> {
let Some(assigned) = interface_addresses(name) else {
// Cannot tell. Carry on rather than block on a guess.
return Ok(());
};
match assigned.iter().find(|entry| entry.address == wanted) {
Some(entry) if entry.is_usable() => Ok(()),
Some(entry) => Err(format!(
"interface `{name}` has {wanted} but it is unusable: {}",
entry.why_unusable().unwrap_or("unknown reason")
)),
None => {
let present = if assigned.is_empty() {
"it currently has no IPv6 address at all".to_string()
} else {
format!(
"it currently has: {}",
assigned
.iter()
.map(|entry| entry.address.to_string())
.collect::<Vec<_>>()
.join(", ")
)
};
Err(format!(
"interface `{name}` has no {wanted} address, {present}"
))
}
}
}
/// The commands a privileged user runs once to prepare an interface.
///
/// The order and the two extra settings matter. A persistent TUN
/// interface has no carrier until a process attaches to it, and Linux
/// flushes IPv6 addresses from an interface that loses carrier unless
/// `keep_addr_on_down` is set — so an address added without it silently
/// disappears before the agent ever starts. `nodad` is needed for the same
/// reason: duplicate address detection can never finish with no carrier,
/// and the address would stay tentative and unusable.
pub fn setup_commands(request: &TunRequest, user: &str) -> Vec<String> {
let mut commands = vec![
format!(
"sudo ip tuntap add dev {} mode tun user {user}",
request.name
),
format!(
"sudo ip link set dev {} mtu {} up",
request.name, request.mtu
),
format!(
"sudo sysctl -qw net.ipv6.conf.{}.keep_addr_on_down=1",
request.name
),
format!(
"sudo ip -6 address add {}/{} dev {} nodad",
request.address, request.prefix_len, request.name
),
];
if let Some(address) = request.address_v4 {
let prefix_len = request.prefix_len_v4;
// IPv4 is not sensitive to carrier the way IPv6 is, so it needs
// no extra settings.
commands.push(format!(
"sudo ip address add {address}/{prefix_len} dev {}",
request.name
));
}
commands
}
/// Opens the TUN interface, creating it if it is not already there.
///
/// Synchronous, and deliberately so: on the managed path the caller holds
/// a capability guard across this call, and a guard must not span an
/// `await` because Linux capabilities are per thread.
///
/// `attach_only` says the interface already exists and was prepared by
/// something else, so nothing beyond `TUNSETIFF` is issued — reconfiguring
/// it would need exactly the privileges that path is avoiding.
pub(crate) fn open_tun(
request: &TunRequest,
attach_only: bool,
) -> Result<Arc<dyn TunDevice>, PluginError> {
let mut config = tun::Configuration::default(); let mut config = tun::Configuration::default();
config.tun_name(&request.name); config.tun_name(&request.name);
config.platform_config(|platform| { config.platform_config(|platform| {
// The crate's own root check is not the check we want: the // The crate's own root check is not the check we want: this holds
// managed path holds CAP_NET_ADMIN without being root, and the // CAP_NET_ADMIN without being root. Whether the open succeeds is
// attach path needs no privileges at all. Whether the open // the honest answer.
// succeeds is the honest answer either way.
platform.ensure_root_privileges(false); platform.ensure_root_privileges(false);
}); });
// Packet information stays off, so reads and writes are raw IP // Packet information stays off, so reads and writes are raw IP
@@ -500,22 +334,12 @@ mod system {
// information, so the flags match when attaching to one. // information, so the flags match when attaching to one.
let device = tun::create_as_async(&config).map_err(|err| { let device = tun::create_as_async(&config).map_err(|err| {
let hint = if attach_only { PluginError::Unavailable(format!(
format!( "cannot create the TUN interface `{}`: {err}. Creating one needs \
"interface `{}` exists but could not be opened: {err}. \ CAP_NET_ADMIN; grant it with `setcap cap_net_admin+p`, or run with \
It must be a persistent TUN interface owned by this user.", `--no-tun` to keep the tunnels off the operating system.",
request.name request.name
) ))
} else {
format!(
"cannot create the TUN interface `{}`: {err}. \
Creating one needs CAP_NET_ADMIN. Either grant it with \
`setcap cap_net_admin+p`, or prepare the interface once as root \
(see `tsunagi tun-setup`) and run unprivileged.",
request.name
)
};
PluginError::Unavailable(hint)
})?; })?;
let (reader, writer) = tokio::io::split(device); let (reader, writer) = tokio::io::split(device);
@@ -526,172 +350,4 @@ mod system {
writer: Mutex::new(writer), writer: Mutex::new(writer),
}) as Arc<dyn TunDevice>) }) as Arc<dyn TunDevice>)
} }
fn current_user() -> String {
std::env::var("SUDO_USER")
.or_else(|_| std::env::var("USER"))
.unwrap_or_else(|_| "$USER".to_string())
}
/// Opens real TUN interfaces.
#[derive(Debug, Clone, Default)]
pub struct SystemTunFactory;
impl SystemTunFactory {
/// Creates the factory.
pub fn new() -> Self {
Self
}
}
impl TunFactory for SystemTunFactory {
fn name(&self) -> &str {
"system"
}
fn create<'a>(
&'a self,
request: TunRequest,
) -> BoxFuture<'a, Result<Arc<dyn TunDevice>, PluginError>> {
Box::pin(async move {
let existed = interface_exists(&request.name);
// Check before attaching. Opening and then dropping the
// device toggles the carrier, and with the default
// `keep_addr_on_down=0` that is enough to flush the very
// address we are looking for.
if existed && let Err(reason) = check_address(&request.name, request.address) {
let commands = setup_commands(&request, &current_user()).join("\n ");
return Err(PluginError::Unavailable(format!(
"{reason}.\nAssigning an IPv6 address needs privileges. \
Remove the interface and prepare it again:\n sudo ip link del dev {}\n {commands}",
request.name
)));
}
let device = open_tun(&request, existed)?;
// An interface we just created has no address yet either.
if let Err(reason) = check_address(&request.name, request.address) {
let commands = setup_commands(&request, &current_user()).join("\n ");
return Err(PluginError::Unavailable(format!(
"{reason}.\nAssigning an IPv6 address needs privileges. Run:\n {commands}"
)));
}
Ok(device)
})
}
}
}
#[cfg(all(test, feature = "tun-device", target_os = "linux"))]
mod system_tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::net::Ipv6Addr;
use super::TunRequest;
use super::system::{interface_addresses, parse_if_inet6, setup_commands};
const SAMPLE: &str = "\
fe800000000000008baaeb0c433b635a 04 40 20 80 tailscale0
00000000000000000000000000000001 01 80 10 80 lo
fd559caf9652cb86321feac65c73bd82 05 40 00 80 tsun0
fd559caf9652cb86321feac65c73bd83 05 40 00 40 tsun0
fd559caf9652cb86321feac65c73bd84 05 40 00 08 tsun0
";
#[test]
fn addresses_are_read_per_interface_with_their_flags() {
let found = parse_if_inet6(SAMPLE, "tsun0");
assert_eq!(found.len(), 3);
assert_eq!(
found[0].address,
"fd55:9caf:9652:cb86:321f:eac6:5c73:bd82"
.parse::<Ipv6Addr>()
.unwrap()
);
assert!(found[0].is_usable(), "permanent address is usable");
// Tentative: duplicate address detection never finishes without a
// carrier, so the address exists but cannot carry traffic.
assert!(!found[1].is_usable());
assert!(found[1].why_unusable().unwrap().contains("tentative"));
// Duplicate address detection failed outright.
assert!(!found[2].is_usable());
assert!(found[2].why_unusable().unwrap().contains("duplicate"));
assert!(parse_if_inet6(SAMPLE, "nosuchdev").is_empty());
// An interface name that is a prefix of another must not match.
assert!(parse_if_inet6(SAMPLE, "tsun").is_empty());
}
#[test]
fn malformed_lines_are_skipped_rather_than_panicking() {
assert!(parse_if_inet6("", "tsun0").is_empty());
assert!(parse_if_inet6("garbage", "tsun0").is_empty());
assert!(parse_if_inet6("zz 01 40 00 80 tsun0", "tsun0").is_empty());
assert!(parse_if_inet6("00 01 40 00 80 tsun0", "tsun0").is_empty());
// Flags that do not parse fall back to zero rather than dropping the
// address, so a usable address is never hidden by a formatting change.
let odd = parse_if_inet6("00000000000000000000000000000001 01 80 10 zz lo", "lo");
assert_eq!(odd.len(), 1);
assert!(odd[0].is_usable());
}
#[test]
fn loopback_is_found_on_this_host() {
// A real read of /proc/net/if_inet6: every Linux host has ::1 on lo.
let found = interface_addresses("lo").expect("/proc/net/if_inet6 should be readable");
assert!(
found
.iter()
.any(|entry| entry.address == Ipv6Addr::LOCALHOST),
"expected ::1 on lo, got {found:?}"
);
assert!(
interface_addresses("definitely-not-an-interface")
.unwrap()
.is_empty()
);
}
#[test]
fn the_setup_recipe_survives_a_carrier_drop() {
let request = TunRequest {
name: "tsun0".into(),
address: "fd00::1".parse().unwrap(),
prefix_len: 64,
address_v4: Some("100.64.1.2".parse().unwrap()),
prefix_len_v4: 10,
mtu: 1280,
};
let commands = setup_commands(&request, "someone");
// The interface must be up before the address is added, the address
// must survive losing carrier, and it must not wait for duplicate
// address detection that can never complete.
let joined = commands.join("\n");
let up = joined.find("link set dev tsun0 mtu 1280 up").unwrap();
let keep = joined.find("keep_addr_on_down=1").unwrap();
let add = joined.find("address add fd00::1/64").unwrap();
assert!(up < keep && keep < add, "wrong order:\n{joined}");
assert!(joined.contains("nodad"));
assert!(joined.contains("user someone"));
// IPv4 needs no carrier tricks, just the address.
assert!(joined.contains("ip address add 100.64.1.2/10 dev tsun0"));
// An IPv6-only overlay says nothing about IPv4.
let v6_only = TunRequest {
address_v4: None,
..request
};
assert!(
!setup_commands(&v6_only, "someone")
.join("\n")
.contains("100.64")
);
}
} }
+4 -2
View File
@@ -537,9 +537,11 @@ async fn an_allocated_address_missing_from_the_host_is_reported() {
reason.contains(&allocated.to_string()), reason.contains(&allocated.to_string()),
"unexpected: {reason}" "unexpected: {reason}"
); );
// The agent assigns the address itself, so the report says which
// interface should have had it rather than a command to run.
assert!( assert!(
reason.contains("ip address add"), reason.contains(&a.plugin.overview(network_id).unwrap().interface),
"must name the fix: {reason}" "must name the interface: {reason}"
); );
a.shutdown().await; a.shutdown().await;