Compare commits

...
Author SHA1 Message Date
Ultradesu 4e235cbac6 Keep device data paths on one line
The tray used a fixed sample for the data-path column, so longer relay names wrapped and made device rows uneven. Measure each rendered path and retain the widest value, then render path labels without wrapping. Replace direct/via words with embedded Phosphor route icons while preserving transport, relay name, hop count and explanatory tooltips. The app already ships the font; keep its setup in one function and verify registration alongside a long-relay-width regression test. No wire or stored-state changes. Checks: cargo fmt --all -- --check; cargo clippy --locked --workspace --all-targets -- -D warnings; cargo test --locked --workspace --all-targets.
2026-10-14 13:50:05 +01:00
Aleksandr Bogomiakov 2631eb6975 Protect Linux exit-node connections and tidy the device table
Using an exit node could destroy direct peer connections: the agent's UID
bypass protected outbound routing, but reverse-path checks look up replies
without that UID and selected the overlay default. A real-host A/B check
confirmed that relaxing only strict reverse-path filtering restored direct
connections and exit HTTP traffic; the diagnostic host changes were removed.

Keep strict filtering and protect the Linux underlay with a dedicated masked
mark bit, 0x40000000. Mark the dedicated service UID in OUTPUT, save only that
bit in conntrack and restore it before PREROUTING reverse-path checks. Install
IPv4/IPv6 mark and UID main-table bypass rules before Tailscale's priorities;
leave application-specific and exit-default routing after them. Preserve
foreign mark bits, main-table routes and foreign firewall/policy rules.

Protect the first inbound TCP/UDP packet using bounded, locally owned socket
addresses/ports from /proc and an actual socket match. Refresh socket protection
before sending private transport negotiations and on discovery ticks, without
firewall writes for an unchanged socket set. Remote announcements never become
OS arguments. A failed refresh leaves active exit routing in place. Install
protection before routes, remove routes before protection, roll back partial
activation, and remove only exact interface tags, including legacy UID rules.

Reject strict kernel rp_filter without src_valid_mark with an actionable error;
never weaken filtering or change host sysctls. Document mark-aware Netfilter
requirements and provide the trusted ip6tables path in the NixOS module.
Other platforms retain their existing exit implementation. Peer wire formats,
identity derivation, local IPC and persistent storage are unchanged.

Make the tray devices grid fill and shrink with the window, constrain the first
cell and give remaining width to traffic plots. Keep displayed end-to-end
protocol and its short encryption tooltip consistent, independent of carrier
protocol. Move handshake/session age into its own column beside the existing
iroh control RTT. Use stable per-device hover identities and concise tooltips;
missing protocols/measurements remain explicit rather than invented.

Validation on Linux through nix develop:
- cargo fmt --all -- --check
- cargo clippy --locked --workspace --all-targets -- -D warnings
- cargo test --locked --workspace --all-targets: 529 passed, 3 ignored;
  inherited TSUNAGI_CONTROL_SOCKET and TSUNAGI_CONTROL_GROUP were unset
- explicit privileged kernel regression: 1 passed, using disposable namespaces,
  real TCP/UDP IPv4/IPv6, strict kernel and Netfilter RPF, first inbound packets,
  dynamic UDP binding, application exit routing/IPv6 blocking, repeated apply
  and exact cleanup with foreign Tailscale-priority rules retained
- headless egui grid/scroll-area rendering at multiple sizes and control RTT,
  mark masks/order, passive socket parsing, partial failure and cleanup tests
- git diff --check

Earlier iterations caught rtnetlink family API and nonexhaustive initializer
errors, Clippy warnings, egui's implicit minimum column width, and kernel-test
fixture issues (wildcard echo source, blocked IPv6 neighbour discovery and a
closed child stdout pipe). Corrected before final checks; the stdout-only test
fix was checked with fmt, Clippy and the explicit kernel regression afterward.
Native tray rendering and macOS/Windows checks were not run. The running host
agent was not replaced or restarted. Requested release tag: v0.1.0-rc.32.
2026-10-14 10:40:30 +01:00
Aleksandr Bogomiakov 90ef5cefec Retire offline node claims with equal network-wide cleanup rights
Offline devices reserved names and IPv4 addresses indefinitely, leaving stale
members in the device history. Give every authenticated network participant
the same right to retire another device's observed signed claim without an
owner, quorum, permanent identity ban or secret change.

Add a separately signed retirement containing the target's original signed
claim as proof. Verify both signatures and network scopes, retain the strongest
proved cutoff per target with deterministic ties, and suppress claim versions
at or below it in membership, hostname ownership and IPv4 allocation. Keep the
old signed records and range agreement, and retain markers after a newer claim
returns so old replicas cannot resurrect obsolete reservations. A returning
device signs a newer claim, allocates against current holders and resets its
previous hostname priority rather than taking a reused name with an old time.

Commit local retirement and the requester's author counter atomically before
announcing it; receivers persist proofs before applying them. Bound history to
1024 targets by default, reject additional targets without evicting protection,
and treat corrupt or over-limit mandatory storage as an explicit error. Repeat
bounded marker batches with a per-session rotating cursor so queue pressure
cannot indefinitely starve later markers. Exclude retired claims and cached
hints from history-derived discovery without preventing a fresh authenticated
device from returning.

Expose Agent::forget_node through the shared local control API. CLI command
tsng network forget-node accepts exact network names, full ids or unique id
prefixes, and exact node hostnames, ids, prefixes or overlay IPv4 addresses.
Ambiguous selections change nothing. Recheck the observed version and refuse
self, online nodes, the selected exit node and absent or changed claims.
Add Forget with an identity confirmation to offline tray device rows. Widen
Show devices to 1440x640 with an 800x400 minimum to fit its contents.

Control ALPN/version advances to tsunagi/ctrl/7 and local IPC to 21; upgrade
all peers and restart the agent with matching CLI/tray clients. Data ALPN 5,
network identity derivation, handshake encoding and existing record signing
bytes remain unchanged. SQLite schema 9 adds retirement storage while retaining
identity, secrets, settings and signed claims. Document behavior and limits.

Validation on Linux through nix develop:
- cargo fmt --all -- --check
- cargo clippy --locked --workspace --all-targets -- -D warnings
- cargo test --locked --workspace --all-targets: 519 passed, 2 existing ignored
  tests; inherited TSUNAGI_CONTROL_SOCKET and TSUNAGI_CONTROL_GROUP were unset
- git diff --check and the built CLI forget-node help

Tests use real iroh authentication and SQLite. Cover restart, stale-replica
recovery, equal rights and network isolation, hostname reuse and return, twelve
marker batches through a two-message queue, malformed signatures and proofs,
invented future cutoffs, deterministic merge, capacity, migration and injected
counter-write rollback, selectors and cleanup through two real CLI processes.
Earlier iterations caught a private test API, a collapsible tray conditional
and a crash-based CLI test waiting for QUIC's idle timeout; corrected before
the final successful suite. Manual tray rendering, macOS/Windows, privileged
TUNs and public-network traversal were not run. The requested RC tag is
v0.1.0-rc.31; no push was requested.
2026-10-12 18:04:29 +01:00
Aleksandr Bogomiakov 13bdcaf98d Render protocol priority buttons with Phosphor icons
The tray protocol selection used ordinary Unicode up/down arrows for its
priority buttons. The default UI fonts lack these glyphs, so both buttons
rendered missing-glyph boxes. Use the regular Phosphor ARROW_UP and ARROW_DOWN
icons already installed in the application's proportional font fallback.

Validation on Linux through nix develop:
- cargo fmt --all -- --check
- cargo clippy --locked --workspace --all-targets -- -D warnings
- cargo test --locked --workspace --all-targets, with inherited control socket
  and group overrides removed: 509 passed, 2 existing ignored tests

No new tests were added for the two glyph substitutions. Manual tray rendering
and macOS/Windows checks were not performed. No release tag was requested for
this fix.
2026-10-12 17:19:53 +01:00
34 changed files with 3457 additions and 196 deletions

No files matched your search

+18
View File
@@ -129,6 +129,24 @@ a common protocol at the same wire version. `tsng up --protocol` supplies the
initial defaults for networks without a saved choice; it does not restrict
which built-in protocols can be enabled later.
Remove an obsolete offline device from a network's shared history:
```sh
tsng network forget-node home old-laptop
tsng network forget-node home 10.13.37.143
```
Networks accept exact names, full ids or unique id prefixes; nodes accept those
id selectors, exact hostnames or overlay IPv4 addresses. Ambiguous selectors
change nothing. Start the network first if it is stopped. In **Show devices**,
click **Forget** beside an offline node and confirm its identity.
Every member has the same cleanup right. The signed cleanup frees the observed
claim's name and IPv4 reservation, survives restarts and reaches offline replicas
when they reconnect. It does not ban the device: a newer signed claim can return.
Online nodes and the currently selected exit node cannot be forgotten.
[State and limits](docs/sync-model.md#offline-node-cleanup).
## Platforms
| Platform | Build | CLI `tsng` | Tray | Exit node: offer / use | Packages |
+39
View File
@@ -72,6 +72,17 @@ pub struct ConfiguredNetwork {
pub active: bool,
}
/// A durable version-scoped cleanup, not an identity ban.
#[derive(Debug, Clone)]
pub struct ForgetNodeOutcome {
/// Identity whose observed claim was retired.
pub target: EndpointId,
/// The retired claim version; future signed claims remain admissible.
pub record_version: u64,
/// Authenticated peers to which cleanup was queued immediately.
pub peers_told: usize,
}
/// A running agent.
///
/// Cloning gives another handle to the same agent.
@@ -298,6 +309,27 @@ impl TransportContext for TransportCtx {
}
impl Agent {
/// Retires one offline member's observed claim without revoking membership.
/// Every member has this right. The version must still match the selection.
pub async fn forget_node(
&self,
network: NetworkId,
peer: EndpointId,
expected_version: u64,
) -> Result<ForgetNodeOutcome> {
let _lifecycle = self.inner.lifecycle.lock().await;
let (reply, receive) = oneshot::channel();
self.command(
network,
NetCommand::ForgetNode {
peer,
expected_version,
reply,
},
)
.await?;
receive.await.map_err(|_| Error::Stopped)?
}
fn effective_protocols(&self, saved: Option<&Vec<String>>) -> Vec<String> {
saved
.or(self.inner.config.default_protocols.as_ref())
@@ -878,6 +910,12 @@ impl Agent {
.await?
.into_iter()
.find(|stored| stored.network_id == network_id);
// Mandatory cleanup proofs must never be silently dropped on startup.
let retirements = self
.inner
.storage
.retirements(network_id, self.inner.limits.max_retirements)
.await?;
let range = self.desired_range(network_id).await?;
if let Some(range) = range {
let conflicts = self.inner.routes.overlapping_networks(network_id, range);
@@ -930,6 +968,7 @@ impl Agent {
.and_then(|stored| stored.exit_via)
.and_then(|bytes| EndpointId::from_bytes(&bytes).ok());
let handle = network::spawn(RuntimeParams {
retirements,
broadcast,
exit_offer,
exit_via,
+243 -6
View File
@@ -26,7 +26,9 @@ use crate::net::{EndpointAdapter, PathAddr, snapshot_connection};
use crate::proto::handshake::{self, HandshakeOutcome, Role};
use crate::proto::message::{Announcement, ControlMessage, Envelope, encode, kind};
use crate::state::allocator::allocate;
use crate::state::{Ipv4Range, Merged, RecordBody, SignedRecord, StateSet};
use crate::state::{
Ipv4Range, MAX_RETIREMENT_LEN, Merged, RecordBody, SignedRecord, SignedRetirement, StateSet,
};
use crate::storage::Storage;
use super::events::Event;
@@ -48,6 +50,11 @@ pub(crate) struct InboundSession {
/// Commands accepted by a network runtime.
pub(crate) enum NetCommand {
ForgetNode {
peer: EndpointId,
expected_version: u64,
reply: oneshot::Sender<Result<super::ForgetNodeOutcome>>,
},
Inbound(Box<InboundSession>),
Send {
peer: EndpointId,
@@ -118,6 +125,7 @@ pub(crate) enum NetCommand {
impl std::fmt::Debug for NetCommand {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
NetCommand::ForgetNode { peer, .. } => write!(f, "ForgetNode({peer})"),
NetCommand::Inbound(_) => f.write_str("Inbound"),
NetCommand::InboundLink(link) => write!(f, "InboundLink({})", link.protocol),
NetCommand::Send { peer, message, .. } => {
@@ -162,6 +170,7 @@ impl NetworkHandle {
/// Everything a network runtime needs to run.
pub(crate) struct RuntimeParams {
pub(crate) retirements: Vec<SignedRetirement>,
pub(crate) broadcast: bool,
/// This agent offers itself as an exit node here.
pub(crate) exit_offer: bool,
@@ -366,6 +375,9 @@ struct Runtime {
state: StateSet,
/// Snapshots received while dispatching, handled on the next loop pass.
pending_state: Vec<(EndpointId, Vec<SignedRecord>)>,
pending_retirements: Vec<(EndpointId, Vec<SignedRetirement>)>,
/// Next cleanup batch per live session; prevents queue pressure starving a suffix.
retirement_cursor: HashMap<EndpointId, usize>,
/// The highest version this agent has ever published for this network.
own_version: u64,
/// Whether the exit node in use was reachable when last looked at, so
@@ -448,6 +460,8 @@ impl Runtime {
reported_name_taken: None,
state: StateSet::new(),
pending_state: Vec::new(),
pending_retirements: Vec::new(),
retirement_cursor: HashMap::new(),
own_version: 0,
exit_via_was_online: false,
announced_exit: false,
@@ -547,6 +561,14 @@ impl Runtime {
async fn handle_command(&mut self, command: NetCommand) {
match command {
NetCommand::ForgetNode {
peer,
expected_version,
reply,
} => {
let result = self.forget_node(peer, expected_version).await;
let _ = reply.send(result);
}
NetCommand::Inbound(inbound) => {
self.install_session(*inbound).await;
}
@@ -556,6 +578,13 @@ impl Runtime {
message,
reply,
} => {
if matches!(&message, ControlMessage::Transport { .. })
&& let Some(interface) = self.params.interface.clone()
{
// Locally bound pair sockets must be protected before their
// private offer/accept invites the first inbound datagram.
interface.sync_exit_rules().await;
}
let _ = reply.send(self.send_to(peer, message));
}
NetCommand::Broadcast { message, reply } => {
@@ -801,6 +830,7 @@ impl Runtime {
// the check is a comparison when nothing has changed.
self.ensure_own_claim().await;
self.report_hostname_taken();
self.broadcast_retirements();
// The same timer repairs the relay picture: reachability is soft
// state, repeated while it holds and forgotten when it stops.
self.update_paths();
@@ -814,7 +844,7 @@ impl Runtime {
// still dialable when the endpoint's own discovery can resolve
// one. This is what makes a network a mesh rather than a star
// around whoever was named on a command line.
for record in self.state.records() {
for record in self.state.active_records() {
if let Ok(author) = record.author_id()
&& author != self.local_id
{
@@ -831,6 +861,9 @@ impl Runtime {
let Ok(endpoint_id) = EndpointId::from_bytes(&hint.endpoint_id) else {
continue;
};
if self.state.is_retired(&endpoint_id) {
continue;
}
let Some(addr) = decode_hint(endpoint_id, &hint.addr) else {
continue;
};
@@ -845,6 +878,9 @@ impl Runtime {
// Last, so it carries what this round learned.
self.introduce_peers();
self.refresh_underlay().await;
if let Some(interface) = self.params.interface.clone() {
interface.sync_exit_rules().await;
}
}
/// Hands the interface the underlay destinations this network depends
@@ -1036,6 +1072,22 @@ impl Runtime {
/// boundary, because a restored backup or a copied file could hold
/// anything.
async fn load_state(&mut self) {
for marker in std::mem::take(&mut self.params.retirements) {
// Storage verified these proofs and activation enforced capacity.
if let Err(err) = self.state.merge_retirement(
self.network_id,
marker,
self.params.limits.max_retirements,
) {
self.emit(Event::PluginError {
network: self.network_id,
protocol: "state".into(),
reason: err.to_string(),
});
self.shutdown.trigger();
return;
}
}
let stored = self
.params
.storage
@@ -1051,7 +1103,8 @@ impl Runtime {
.storage
.own_record_version(self.network_id, self.local_id)
.await
.unwrap_or(0);
.unwrap_or(0)
.max(self.state.authored_version(&self.local_id));
for (id, hostname) in self
.params
.storage
@@ -1163,6 +1216,7 @@ impl Runtime {
let kept = self
.state
.get(&self.local_id)
.filter(|_| !self.state.is_retired(&self.local_id))
.filter(|record| record.body.hostname() == Some(name.as_str()))
.and_then(|record| record.body.hostname_since());
kept.unwrap_or_else(unix_millis)
@@ -1195,7 +1249,7 @@ impl Runtime {
if self
.state
.get(&self.local_id)
.is_some_and(|record| record.body == body)
.is_some_and(|record| record.body == body && !self.state.is_retired(&self.local_id))
{
return;
}
@@ -1203,13 +1257,187 @@ impl Runtime {
self.publish_record(body).await;
}
async fn forget_node(
&mut self,
peer: EndpointId,
expected_version: u64,
) -> Result<super::ForgetNodeOutcome> {
if peer == self.local_id {
return Err(Error::ForgetNode(
"use network leave for this device".into(),
));
}
if self.is_online(peer) {
return Err(Error::ForgetNode(
"the node is online; only offline history can be forgotten".into(),
));
}
if self.params.exit_via == Some(peer) {
return Err(Error::ForgetNode(
"stop using this exit node before forgetting it".into(),
));
}
let target = self
.state
.get(&peer)
.filter(|_| !self.state.is_retired(&peer))
.filter(|record| matches!(record.body, RecordBody::Claim { .. }))
.cloned()
.ok_or_else(|| Error::ForgetNode("no current signed claim for this node".into()))?;
if target.version != expected_version {
return Err(Error::ForgetNode(
"the node's claim changed; refresh and select it again".into(),
));
}
let version = self
.own_version
.max(self.state.authored_version(&self.local_id))
.checked_add(1)
.filter(|v| *v <= i64::MAX as u64)
.ok_or_else(|| Error::ForgetNode("author counter exhausted".into()))?;
let marker =
SignedRetirement::sign(&self.params.device_secret, self.network_id, version, target);
let mut candidate = self.state.clone();
candidate
.merge_retirement(
self.network_id,
marker.clone(),
self.params.limits.max_retirements,
)
.map_err(|err| Error::ForgetNode(err.to_string()))?;
// Signed cleanup and our author counter commit together before visibility.
self.params
.storage
.publish_own_retirement(marker.clone())
.await?;
self.own_version = version;
self.state = candidate;
self.known_hostnames.remove(&peer);
self.dial_states.remove(&peer);
self.candidate_addrs.remove(&peer);
self.publish_allocations().await;
self.report_hostname_taken();
let message = ControlMessage::Retirements {
markers: vec![marker],
};
let peers: Vec<_> = self.sessions.keys().copied().collect();
let peers_told = peers
.into_iter()
.filter(|peer| self.send_to(*peer, message.clone()).is_ok())
.count();
Ok(super::ForgetNodeOutcome {
target: peer,
record_version: expected_version,
peers_told,
})
}
fn broadcast_retirements(&mut self) -> usize {
let markers = self.state.retirements();
if markers.is_empty() {
return 0;
}
// Each proof has an encoded bound; reserve envelope/list overhead and
// chunk every marker, so a low-id prefix cannot starve later cleanup.
let batch = (self.params.limits.max_frame_len.saturating_sub(128) / MAX_RETIREMENT_LEN)
.max(1)
.min(self.params.limits.max_state_records.max(1));
let mut delivered = HashSet::new();
let peers: Vec<_> = self.sessions.keys().copied().collect();
let chunks: Vec<_> = markers.chunks(batch).collect();
for peer in peers {
let start = self.retirement_cursor.get(&peer).copied().unwrap_or(0) % chunks.len();
for offset in 0..chunks.len() {
let index = (start + offset) % chunks.len();
let message = ControlMessage::Retirements {
markers: chunks[index].to_vec(),
};
if self.send_to(peer, message).is_err() {
self.retirement_cursor.insert(peer, index);
break;
}
delivered.insert(peer);
self.retirement_cursor
.insert(peer, (index + 1) % chunks.len());
}
}
delivered.len()
}
async fn receive_retirements(&mut self, peer: EndpointId, markers: Vec<SignedRetirement>) {
let mut changed = false;
for marker in markers {
let mut candidate = self.state.clone();
match candidate.merge_retirement(
self.network_id,
marker.clone(),
self.params.limits.max_retirements,
) {
Ok(false) => continue,
Ok(true) => {}
Err(err) => {
self.metrics.protocol_violations += 1;
self.emit(Event::ProtocolViolation {
network: Some(self.network_id),
peer: Some(peer),
reason: format!("unusable cleanup proof: {err}"),
});
continue;
}
}
let target = marker.target.author_id().ok();
if let Err(err) = self.params.storage.put_retirement(marker).await {
self.emit(Event::PluginError {
network: self.network_id,
protocol: "state".into(),
reason: format!("cannot persist cleanup proof: {err}"),
});
continue;
}
self.state = candidate;
if let Some(target) = target
&& self.state.is_retired(&target)
&& !self.sessions.contains_key(&target)
{
self.known_hostnames.remove(&target);
self.dial_states.remove(&target);
self.candidate_addrs.remove(&target);
}
changed = true;
}
if changed {
// A live target answers cleanup with a fresh claim; its identity
// is still authenticated, and old hostname priority is not reused.
self.ensure_own_claim().await;
self.publish_allocations().await;
self.broadcast_retirements();
}
}
/// Signs, stores and announces one of this agent's own records.
///
/// Stored before it is announced, in one transaction with the version
/// counter, so a crash can never let us reuse a version we already put on
/// the wire.
async fn publish_record(&mut self, body: RecordBody) {
let version = self.own_version.saturating_add(1);
let observed = self.state.authored_version(&self.local_id).max(
self.state
.retirement_of(&self.local_id)
.map_or(0, |m| m.target.version),
);
let Some(version) = self
.own_version
.max(observed)
.checked_add(1)
.filter(|v| *v <= i64::MAX as u64)
else {
self.emit(Event::PluginError {
network: self.network_id,
protocol: "state".into(),
reason: "author counter exhausted".into(),
});
return;
};
let record = SignedRecord::sign(&self.params.device_secret, self.network_id, version, body);
if let Err(err) = self.params.storage.publish_own_record(record.clone()).await {
@@ -1234,6 +1462,9 @@ impl Runtime {
/// Handles snapshots collected while dispatching messages.
async fn drain_pending_state(&mut self) {
for (peer, markers) in std::mem::take(&mut self.pending_retirements) {
self.receive_retirements(peer, markers).await;
}
for (peer, records) in std::mem::take(&mut self.pending_state) {
self.receive_state(peer, records).await;
}
@@ -1242,6 +1473,7 @@ impl Runtime {
/// Sends everything we know to every peer.
fn broadcast_state(&mut self) {
self.broadcast_retirements();
let mut records = self.state.records();
records.truncate(self.params.limits.max_state_records);
if records.is_empty() {
@@ -2071,6 +2303,7 @@ impl Runtime {
/// Drops every link to a peer.
fn drop_links_for(&mut self, peer: EndpointId) {
self.retirement_cursor.remove(&peer);
self.reset_peer_data(peer, "peer session ended");
self.hub.remove_peer(peer);
self.reachable.remove(&peer);
@@ -2434,6 +2667,9 @@ impl Runtime {
ControlMessage::State { records } => {
self.pending_state.push((peer, records.clone()));
}
ControlMessage::Retirements { markers } => {
self.pending_retirements.push((peer, markers.clone()));
}
// Members somebody else knows of. Candidates, nothing more:
// each still has to pass the handshake, and each is tried the
// same way as one from any other source.
@@ -2631,12 +2867,13 @@ impl Runtime {
// listing it as one turns leaving into a peer that looks broken.
let mut members: Vec<MemberStatus> = self
.state
.records()
.active_records()
.into_iter()
.filter(|record| !matches!(record.body, RecordBody::Release))
.filter_map(|record| {
let endpoint_id = record.author_id().ok()?;
Some(MemberStatus {
record_version: record.version,
endpoint_id,
// Only uncontested claims are reported as held: two
// members may have claimed the same thing, and saying
+2
View File
@@ -108,6 +108,8 @@ pub struct HostnameConflict {
/// connected.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MemberStatus {
/// Current signed claim version, used to bind cleanup confirmation.
pub record_version: u64,
/// The member's device identity.
pub endpoint_id: EndpointId,
/// The IPv4 overlay address it claimed and signed for.
+3
View File
@@ -174,6 +174,8 @@ pub struct Limits {
pub max_reason_len: usize,
/// Largest number of signed records accepted in one snapshot.
pub max_state_records: usize,
/// Maximum durable retirement markers retained per network.
pub max_retirements: usize,
/// Deadline for the whole handshake.
pub handshake_timeout: Duration,
/// Deadline for one outbound dial attempt.
@@ -211,6 +213,7 @@ impl Default for Limits {
max_echo_payload_len: 4 * 1024,
max_reason_len: 256,
max_state_records: crate::state::MAX_RECORDS_PER_MESSAGE,
max_retirements: 1024,
handshake_timeout: Duration::from_secs(10),
dial_timeout: Duration::from_secs(10),
write_timeout: Duration::from_secs(30),
+3
View File
@@ -15,6 +15,9 @@ pub type Result<T, E = Error> = std::result::Result<T, E>;
#[derive(Debug, thiserror::Error)]
#[non_exhaustive]
pub enum Error {
/// A node-history cleanup is unsafe, stale or cannot be performed.
#[error("cannot forget node: {0}")]
ForgetNode(String),
/// An ordered local data-plane protocol selection is invalid.
#[error("invalid protocol selection: {0}")]
InvalidProtocols(String),
+174 -1
View File
@@ -171,6 +171,15 @@ pub fn default_control_socket_path() -> PathBuf {
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
#[non_exhaustive]
pub enum Request {
/// Retire the selected offline member's observed claim.
ForgetNode {
/// Exact public network id.
network_id: String,
/// Exact device identity selected by the client.
peer: String,
/// Claim version displayed when the user selected the node.
expected_version: u64,
},
/// Replace one network's ordered local data-plane selection.
SetProtocols {
/// Public network identifier.
@@ -262,6 +271,14 @@ pub enum Request {
impl std::fmt::Debug for Request {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Request::ForgetNode {
network_id,
peer,
expected_version,
} => write!(
f,
"ForgetNode {{ {network_id}, peer: {peer}, version: {expected_version} }}"
),
Request::SetProtocols {
network_id,
protocols,
@@ -300,6 +317,8 @@ impl std::fmt::Debug for Request {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[non_exhaustive]
pub enum Response {
/// An observed offline claim was durably retired.
NodeForgotten(ForgottenNodeReport),
/// Accepted ordered protocol selection.
Protocols(Vec<String>),
/// A status report.
@@ -328,6 +347,141 @@ pub enum Response {
Error(String),
}
/// The durable result of an offline-node cleanup.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct ForgottenNodeReport {
/// Selected device identity.
pub peer: String,
/// The retired claim version, not a future-version ban.
pub record_version: u64,
/// Peers to which cleanup was immediately queued.
pub peers_told: u32,
}
/// Resolves a node by exact hostname, full id, unique id prefix or overlay IPv4.
/// Ambiguity is reported with names and full ids and never guessed.
pub fn resolve_node<'a>(
network: &'a NetworkReport,
selector: &str,
) -> std::result::Result<&'a MemberReport, String> {
let selector = selector.trim();
if selector.is_empty() {
return Err("select a node by name, id or IPv4 address".into());
}
if let Some(member) = network.members.iter().find(|m| m.endpoint_id == selector) {
return Ok(member);
}
let prefix = selector.trim_end_matches('…');
let matches: Vec<_> = network
.members
.iter()
.filter(|m| {
m.hostname.as_deref() == Some(selector)
|| m.overlay_address_v4.as_deref() == Some(selector)
|| (!prefix.is_empty() && m.endpoint_id.starts_with(prefix))
})
.collect();
match matches.as_slice() {
[one] => Ok(one),
[] => Err(format!(
"no signed node matches `{selector}` in `{}`",
network.name
)),
many => Err(format!(
"`{selector}` is ambiguous; use a full id or unique id prefix:\n{}",
many.iter()
.map(|m| format!(
" {} {} {}",
m.hostname.as_deref().unwrap_or("unnamed"),
m.endpoint_id,
m.overlay_address_v4.as_deref().unwrap_or("no IPv4")
))
.collect::<Vec<_>>()
.join("\n")
)),
}
}
/// Retires an exact selected claim; an online or changed target is rejected.
pub async fn forget_node(
socket: &Path,
network: &str,
peer: &str,
expected_version: u64,
) -> Result<ForgottenNodeReport> {
let request = Request::ForgetNode {
network_id: network.into(),
peer: peer.into(),
expected_version,
};
match exchange(socket, &request, EXCHANGE_TIMEOUT).await? {
Response::NodeForgotten(report) => Ok(report),
Response::Error(reason) => Err(Error::ForgetNode(reason)),
other => Err(Error::Storage(format!("unexpected response: {other:?}"))),
}
}
#[cfg(test)]
mod node_selector_tests {
use super::*;
#[test]
fn exact_names_addresses_and_unique_prefixes_select_the_same_signed_node() {
let node = MemberReport {
endpoint_id: "abcdef01".into(),
hostname: Some("laptop".into()),
overlay_address_v4: Some("10.2.0.3".into()),
record_version: 42,
..Default::default()
};
let network = NetworkReport {
name: "home".into(),
members: vec![node.clone()],
..Default::default()
};
for selector in ["laptop", "10.2.0.3", "abcdef01", "abc…"] {
assert_eq!(resolve_node(&network, selector), Ok(&node));
}
for selector in ["", "LAPTOP", "missing"] {
assert!(resolve_node(&network, selector).is_err());
}
}
#[test]
fn collisions_between_names_addresses_and_prefixes_require_an_explicit_identity() {
let first = MemberReport {
endpoint_id: "abcdef01".into(),
hostname: Some("duplicate".into()),
overlay_address_v4: Some("10.2.0.3".into()),
..Default::default()
};
let second = MemberReport {
endpoint_id: "abcdef02".into(),
hostname: Some("duplicate".into()),
overlay_address_v4: Some("10.2.0.3".into()),
..Default::default()
};
let third = MemberReport {
endpoint_id: "12345678".into(),
hostname: Some("abc".into()),
..Default::default()
};
let network = NetworkReport {
name: "home".into(),
members: vec![first.clone(), second, third],
..Default::default()
};
for selector in ["duplicate", "10.2.0.3", "abc"] {
let result = resolve_node(&network, selector);
assert!(result.is_err_and(|error| error.contains("ambiguous")
&& error.contains("abcdef01")
&& error.contains("abcdef02")
&& error.contains("duplicate")));
}
assert_eq!(resolve_node(&network, "abcdef01"), Ok(&first));
}
}
/// What happened when a network was stopped or started.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct ActiveReport {
@@ -580,6 +734,8 @@ pub struct HostnameConflictReport {
/// One member of the network, from signed state.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct MemberReport {
/// Signed claim version used to reject stale cleanup requests.
pub record_version: u64,
/// The member's device identity.
pub endpoint_id: String,
/// The IPv4 overlay address it claimed and signed for.
@@ -749,6 +905,15 @@ impl OverlayPeerReport {
/// running and what they can report or change. That is what keeps this module
/// free of any knowledge of them.
pub trait ReportSource: Send + Sync + 'static {
/// Persists and announces a version-scoped node cleanup.
fn forget_node(
&self,
_network: String,
_peer: String,
_expected_version: u64,
) -> BoxFuture<'_, std::result::Result<ForgottenNodeReport, String>> {
Box::pin(async { Err("this agent cannot forget node history".into()) })
}
/// Saves and applies a local ordered protocol set for one network.
fn set_protocols(
&self,
@@ -886,7 +1051,7 @@ pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5);
///
/// Bump it whenever [`Request`], [`Response`] or anything they contain
/// changes shape.
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 20]);
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 21]);
/// Reads one request off an accepted stream, answers it, writes the response.
///
@@ -910,6 +1075,14 @@ where
}
};
let response = match request {
Request::ForgetNode {
network_id,
peer,
expected_version,
} => match source.forget_node(network_id, peer, expected_version).await {
Ok(report) => Response::NodeForgotten(report),
Err(reason) => Response::Error(reason),
},
Request::SetProtocols {
network_id,
protocols,
@@ -84,6 +84,11 @@ pub trait ExitHostRules: Send + Sync + std::fmt::Debug + 'static {
/// startup to take away what a crashed run left.
fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()>;
/// Refreshes volatile local socket protection without replacing exit routes.
fn refresh<'a>(&'a self, _plan: &'a ExitHostPlan) -> BoxFuture<'a, Option<RuleOutcome>> {
Box::pin(async { None })
}
/// Whether using an exit node on this host needs the underlay addresses
/// the agent talks to, because the host cannot exempt the agent's own
/// traffic from the default route by user id (Windows). The default is no.
@@ -33,12 +33,13 @@ const CHAIN: &str = "INPUT";
/// Fixed places `iptables` is installed. Never `PATH`: the agent is
/// privileged and a directory a user controls must not decide what it runs.
const IPTABLES_DIRECTORIES: [&str; 5] = [
const IPTABLES_DIRECTORIES: [&str; 6] = [
"/usr/sbin/iptables",
"/sbin/iptables",
"/usr/bin/iptables",
"/bin/iptables",
"/usr/local/sbin/iptables",
"/run/current-system/sw/bin/iptables",
];
/// The first candidate that exists.
@@ -328,7 +329,23 @@ fn clear_firewall(run: Run<'_>, interface: &str) {
/// Runs the real `iptables`.
pub(super) fn run_iptables(args: &[String]) -> Ran {
let Some(program) = pick_iptables(|path| std::path::Path::new(path).is_file()) else {
run_firewall(false, args)
}
pub(super) fn run_firewall(ipv6: bool, args: &[String]) -> Ran {
let v6_paths: Vec<_> = IPTABLES_DIRECTORIES
.iter()
.map(|p| p.replace("iptables", "ip6tables"))
.collect();
let v6_program = v6_paths
.iter()
.find(|path| std::path::Path::new(path).is_file());
let program = if ipv6 {
v6_program.map(String::as_str)
} else {
pick_iptables(|path| std::path::Path::new(path).is_file())
};
let Some(program) = program else {
return Ran::Missing;
};
match std::process::Command::new(program).args(args).output() {
@@ -350,11 +367,17 @@ pub(super) fn run_iptables(args: &[String]) -> Ran {
#[derive(Debug)]
pub struct LinuxHostRules {
pub(super) route: RouteHandle,
/// Interface, sorted socket snapshot and whether exit routes committed.
pub(super) underlay_sockets:
std::sync::Mutex<Option<(String, Vec<super::linux_exit_mark::Listener>, bool)>>,
}
impl LinuxHostRules {
pub(crate) fn new(route: RouteHandle) -> Self {
Self { route }
Self {
route,
underlay_sockets: std::sync::Mutex::new(None),
}
}
async fn set_route(&self, interface: String, source: Ipv4Addr) -> RuleOutcome {
@@ -25,18 +25,19 @@
//!
//! ```text
//! table 0x7473: default dev <if>
//! 5280: from all uidrange <euid>-<euid> lookup main
//! 5205: from all fwmark 0x40000000/0x40000000 lookup main
//! 5206: from all uidrange <euid>-<euid> lookup main
//! 5290: from all lookup main suppress_prefixlength 0
//! 5300: from all lookup 0x7473
//! ```
//!
//! The first keeps the agent's *own* sockets off the tunnel: its QUIC, relay,
//! The first two keep the agent's *own* sockets and marked replies off the tunnel: its QUIC, relay,
//! DHT and DNS traffic would otherwise be routed into the interface it is
//! carrying, a loop. The second lets every specific route — the LAN, another
//! carrying, a loop. The specific-route rule lets the LAN, another
//! VPN, a tunnel to somewhere else — keep working, because only the default
//! route is suppressed. The priorities sit after Tailscale's 5270.
//! route is suppressed. Mark/UID bypass precedes Tailscale; application rules follow it.
//!
//! The first rule exempts a *user id*, so it is only right when the agent has
//! The UID rule exempts a *user id*, so it is only right when the agent has
//! one of its own, as the packaged service does. An agent started from a login
//! session would take that user's own applications out of the tunnel with it,
//! and the exit node would silently do nothing for them. That is refused, not
@@ -48,9 +49,10 @@ use crate::state::Ipv4Range;
use super::RuleOutcome;
use super::exit::{ExitHostPlan, ExitHostReport, ExitHostRules, exit_tag, exit_tag_prefix};
use super::linux::{
LinuxHostRules, Ran, Run, comment_supported, failure, locked, run_iptables, same_rule,
split_rule_line,
LinuxHostRules, Ran, Run, comment_supported, failure, locked, run_firewall, run_iptables,
same_rule, split_rule_line,
};
use super::linux_exit_mark;
/// The chains the offering rules live in: `(table, chain)`.
const CHAINS: [(&str, &str); 2] = [("nat", "POSTROUTING"), ("filter", "FORWARD")];
@@ -377,6 +379,32 @@ fn read_sysctl(path: &str) -> Option<String> {
std::fs::read_to_string(path).ok()
}
fn check_kernel_rpf(overlay: &str) -> Result<(), String> {
let directory = "/proc/sys/net/ipv4/conf";
let value = |name: &str, setting: &str| -> Result<u32, String> {
read_sysctl(&format!("{directory}/{name}/{setting}"))
.and_then(|s| s.trim().parse().ok())
.ok_or_else(|| format!("cannot check {name}/{setting} before installing exit routes"))
};
let mut interfaces = Vec::new();
for entry in std::fs::read_dir(directory).map_err(|err| err.to_string())? {
let entry = entry.map_err(|err| err.to_string())?;
let name = entry.file_name().to_string_lossy().into_owned();
if name != "all" && name != "default" && name != "lo" && name != overlay {
interfaces.push((
name.clone(),
value(&name, "rp_filter")?,
value(&name, "src_valid_mark")?,
));
}
}
linux_exit_mark::check_kernel_rpf(
value("all", "rp_filter")?,
value("all", "src_valid_mark")?,
&interfaces,
)
}
/// The effective user id from the text of `/proc/self/status`.
fn parse_euid(status: &str) -> Option<u32> {
status
@@ -417,7 +445,7 @@ impl LinuxHostRules {
let xdg = std::env::var("XDG_RUNTIME_DIR").ok();
if let Some(reason) = client_refusal(xdg.as_deref()) {
// Whatever an earlier run left must not stay in force.
self.clear_client().await;
let _ = self.clear_client(interface).await;
return RuleOutcome::Failed(reason);
}
let Some(euid) = std::fs::read_to_string("/proc/self/status")
@@ -426,24 +454,147 @@ impl LinuxHostRules {
else {
return RuleOutcome::Failed("cannot tell which user this agent runs as".to_string());
};
let installed = self
.underlay_sockets
.lock()
.unwrap_or_else(|p| p.into_inner())
.as_ref()
.is_some_and(|(name, _, routed)| name == interface && *routed);
if installed {
// A failed socket refresh must never remove an active exit route.
return self
.refresh_sockets(interface)
.await
.unwrap_or(RuleOutcome::Applied);
}
if let Err(reason) = self.clear_client(interface).await {
return RuleOutcome::Failed(reason);
}
let name = interface.to_string();
let prepared = Self::blocking(move || {
check_kernel_rpf(&name)?;
linux_exit_mark::install(
&run_firewall,
&name,
euid,
std::path::Path::new("/proc/sys/net/ipv6").is_dir(),
)
})
.await;
match prepared {
Ok(Ok(sockets)) => {
*self
.underlay_sockets
.lock()
.unwrap_or_else(|p| p.into_inner()) = Some((interface.into(), sockets, false));
}
Ok(Err(reason)) | Err(reason) => return RuleOutcome::Failed(reason),
}
let route = self.route.clone();
let interface = interface.to_string();
match Self::blocking(move || route.set_exit_client(&interface, euid)).await {
let name = interface.to_string();
let outcome = match Self::blocking(move || route.set_exit_client(&name, euid)).await {
Ok(Ok(())) => RuleOutcome::Applied,
Ok(Err(err)) => RuleOutcome::Failed(err.to_string()),
Err(reason) => RuleOutcome::Failed(reason),
};
if let RuleOutcome::Failed(reason) = outcome {
let reason = match self.clear_client(interface).await {
Ok(()) => reason,
Err(cleanup) => format!("{reason}; cannot roll back exit rules: {cleanup}"),
};
RuleOutcome::Failed(reason)
} else {
if let Some((_, _, routed)) = self
.underlay_sockets
.lock()
.unwrap_or_else(|p| p.into_inner())
.as_mut()
{
*routed = true;
}
outcome
}
}
async fn clear_client(&self) {
async fn clear_client(&self, interface: &str) -> Result<(), String> {
let route = self.route.clone();
if let Ok(Err(err)) = Self::blocking(move || route.clear_exit_client()).await {
tracing::warn!(%err, "cannot remove the exit node routing rules");
Self::blocking(move || route.clear_exit_client())
.await?
.map_err(|err| err.to_string())?;
// Route removal must precede removing reply protection.
let name = interface.to_string();
Self::blocking(move || {
linux_exit_mark::clear(
&run_firewall,
&name,
std::path::Path::new("/proc/sys/net/ipv6").is_dir(),
)
})
.await??;
*self
.underlay_sockets
.lock()
.unwrap_or_else(|p| p.into_inner()) = None;
Ok(())
}
async fn refresh_sockets(&self, interface: &str) -> Option<RuleOutcome> {
let previous = self
.underlay_sockets
.lock()
.unwrap_or_else(|p| p.into_inner())
.clone();
if previous
.as_ref()
.is_none_or(|(name, _, routed)| name != interface || !routed)
{
return None;
}
let name = interface.to_string();
let work_name = name.clone();
let result = Self::blocking(move || {
let uid = std::fs::read_to_string("/proc/self/status")
.ok()
.and_then(|s| parse_euid(&s))
.ok_or("cannot read agent UID")?;
check_kernel_rpf(&work_name)?;
let v6 = std::path::Path::new("/proc/sys/net/ipv6").is_dir();
let sockets = linux_exit_mark::local_sockets(uid, v6)?;
if previous
.as_ref()
.is_some_and(|(name, old, _)| name == &work_name && old == &sockets)
{
return Ok(None);
}
linux_exit_mark::refresh(&run_firewall, &work_name, v6, &sockets)?;
Ok(Some(sockets))
})
.await;
match result {
Ok(Ok(None)) => None,
Ok(Ok(Some(sockets))) => {
*self
.underlay_sockets
.lock()
.unwrap_or_else(|p| p.into_inner()) = Some((name, sockets, true));
Some(RuleOutcome::Applied)
}
Ok(Err(reason)) | Err(reason) => Some(RuleOutcome::Failed(reason)),
}
}
}
impl ExitHostRules for LinuxHostRules {
fn refresh<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, Option<RuleOutcome>> {
Box::pin(async move {
if plan.client {
self.refresh_sockets(&plan.interface).await
} else {
None
}
})
}
fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport> {
Box::pin(async move {
let interface = plan.interface.clone();
@@ -467,7 +618,9 @@ impl ExitHostRules for LinuxHostRules {
let client = if plan.client {
Some(self.apply_client(&interface).await)
} else {
self.clear_client().await;
if let Err(reason) = self.clear_client(&interface).await {
tracing::warn!(%reason, "cannot remove exit-client rules");
}
None
};
ExitHostReport {
@@ -482,7 +635,9 @@ impl ExitHostRules for LinuxHostRules {
Box::pin(async move {
let name = interface.to_string();
let _ = Self::blocking(move || clear_offer(&run_iptables, &name)).await;
self.clear_client().await;
if let Err(reason) = self.clear_client(interface).await {
tracing::warn!(%reason, "cannot remove exit-client rules");
}
})
}
}
@@ -0,0 +1,439 @@
//! Privileged routing regression, isolated from the host in named namespaces.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use std::io::{BufRead, BufReader, Read, Write};
use std::net::{SocketAddr, TcpListener, TcpStream, UdpSocket};
use std::os::unix::process::CommandExt;
use std::process::{Child, Command};
use std::time::Duration;
use super::linux::run_firewall;
use super::linux_exit_mark::{clear, install, local_sockets, refresh};
use crate::overlay::NetlinkProvisioner;
const TEST: &str = "overlay::hostrules::linux_exit_kernel::strict_firewall_preserves_underlay_and_application_exit_routing";
const ROLE: &str = "TSUNAGI_EXIT_KERNEL_TEST_ROLE";
fn ip(arguments: &[&str]) -> String {
let result = Command::new("ip").args(arguments).output().unwrap();
assert!(
result.status.success(),
"ip {arguments:?}: {}",
String::from_utf8_lossy(&result.stderr)
);
String::from_utf8_lossy(&result.stdout).into_owned()
}
struct Namespace(String);
impl Drop for Namespace {
fn drop(&mut self) {
let _ = Command::new("ip").args(["netns", "del", &self.0]).status();
}
}
struct Echo(Child);
impl Drop for Echo {
fn drop(&mut self) {
let _ = self.0.kill();
let _ = self.0.wait();
}
}
fn probes() {
let should_pass = std::env::var("TSUNAGI_EXIT_KERNEL_EXPECT").unwrap() == "pass";
for address in ["198.51.100.2:42000", "[2001:db8:2::2]:42000"] {
let address: SocketAddr = address.parse().unwrap();
let socket = UdpSocket::bind(if address.is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
})
.unwrap();
socket
.set_read_timeout(Some(Duration::from_secs(1)))
.unwrap();
socket.send_to(b"rpf", address).unwrap();
let mut packet = [0u8; 3];
assert_eq!(
socket.recv(&mut packet).is_ok(),
should_pass,
"UDP {address}"
);
if should_pass {
assert_eq!(&packet, b"rpf");
}
let connected = TcpStream::connect_timeout(&address, Duration::from_secs(1));
assert_eq!(connected.is_ok(), should_pass, "TCP {address}");
if let Ok(mut stream) = connected {
stream
.set_read_timeout(Some(Duration::from_secs(1)))
.unwrap();
stream.write_all(b"rpf").unwrap();
stream.read_exact(&mut packet).unwrap();
assert_eq!(&packet, b"rpf");
}
}
}
fn run_probes(expect: &str) {
let result = Command::new(std::env::current_exe().unwrap())
.args(["--ignored", "--exact", TEST, "--nocapture"])
.env(ROLE, "probe")
.env("TSUNAGI_EXIT_KERNEL_EXPECT", expect)
.uid(990)
.gid(990)
.output()
.unwrap();
assert!(
result.status.success(),
"{}\n{}",
String::from_utf8_lossy(&result.stdout),
String::from_utf8_lossy(&result.stderr)
);
}
fn passive_listener() {
let listeners: Vec<_> = ["192.0.2.1:42010", "[2001:db8:1::1]:42010"]
.into_iter()
.map(|address| TcpListener::bind(address).unwrap())
.collect();
println!("PASSIVE_READY");
let tasks: Vec<_> = listeners
.into_iter()
.map(|listener| {
std::thread::spawn(move || {
let (mut socket, _) = listener.accept().unwrap();
socket
.set_read_timeout(Some(Duration::from_secs(2)))
.unwrap();
let mut bytes = [0; 3];
socket.read_exact(&mut bytes).unwrap();
socket.write_all(&bytes).unwrap();
})
})
.collect();
for task in tasks {
task.join().unwrap();
}
}
fn udp_listener() {
let sockets: Vec<_> = ["192.0.2.1:42011", "[2001:db8:1::1]:42011"]
.into_iter()
.map(|a| UdpSocket::bind(a).unwrap())
.collect();
println!("PASSIVE_READY");
let tasks: Vec<_> = sockets
.into_iter()
.map(|socket| {
std::thread::spawn(move || {
socket
.set_read_timeout(Some(Duration::from_secs(5)))
.unwrap();
let mut bytes = [0; 3];
let (len, peer) = socket.recv_from(&mut bytes).unwrap();
socket.send_to(&bytes[..len], peer).unwrap();
})
})
.collect();
for task in tasks {
task.join().unwrap();
}
}
fn spawn_listener(role: &str) -> Echo {
let process = Command::new(std::env::current_exe().unwrap())
.args(["--ignored", "--exact", TEST, "--nocapture"])
.env(ROLE, role)
.uid(990)
.gid(990)
.stdout(std::process::Stdio::piped())
.spawn()
.unwrap();
let mut child = Echo(process);
let mut output = BufReader::new(child.0.stdout.take().unwrap());
loop {
let mut line = String::new();
assert_ne!(
output.read_line(&mut line).unwrap(),
0,
"listener exited before readiness"
);
if line.contains("PASSIVE_READY") {
// Keep the reader open for libtest's final status after the socket
// worker finishes; closing it here makes success a broken pipe.
child.0.stdout = Some(output.into_inner());
return child;
}
}
}
fn client() {
for setting in ["rp_filter", "src_valid_mark"] {
std::fs::write(format!("/proc/sys/net/ipv4/conf/all/{setting}"), b"1").unwrap();
}
std::fs::write("/proc/sys/net/ipv4/conf/rpf-c/rp_filter", b"1").unwrap();
for family in [false, true] {
let args: Vec<_> = [
"-t",
"mangle",
"-A",
"PREROUTING",
"-m",
"rpfilter",
"--invert",
"--validmark",
"-j",
"DROP",
]
.into_iter()
.map(str::to_string)
.collect();
let result = run_firewall(family, &args);
assert!(
matches!(result, super::linux::Ran::Exited { success: true, .. }),
"{result:?}"
);
}
let provisioner = NetlinkProvisioner::new().unwrap();
let route = provisioner.host_rules().route;
// Model Tailscale's earlier lookup, without running another VPN daemon.
ip(&["-4", "rule", "add", "priority", "5270", "table", "52"]);
ip(&[
"-4", "route", "add", "default", "dev", "tsun0", "table", "52",
]);
ip(&["-6", "rule", "add", "priority", "5270", "table", "52"]);
ip(&[
"-6",
"route",
"add",
"unreachable",
"default",
"table",
"52",
]);
// A foreign VPN must leave neighbour discovery and connected LAN paths
// usable; otherwise the fixture itself cannot resolve its uplink neighbour.
for prefix in ["fe80::/64", "2001:db8:1::/64"] {
ip(&["-6", "route", "add", "throw", prefix, "table", "52"]);
}
let mut passive = spawn_listener("listener");
route.set_exit_client("tsun0", 990).unwrap();
// uid bypass alone cannot protect reverse lookups, reproducing the bug.
run_probes("blocked");
install(&run_firewall, "tsun0", 990, true).unwrap();
run_probes("pass");
// Pair transports may bind fresh UDP ports after exit activation.
let mut dynamic_udp = spawn_listener("udp-listener");
refresh(
&run_firewall,
"tsun0",
true,
&local_sockets(990, true).unwrap(),
)
.unwrap();
let server = std::env::var("TSUNAGI_EXIT_KERNEL_SERVER").unwrap();
let incoming = Command::new("ip").args(["netns", "exec", &server, "python3", "-c", r#"import socket
for family,source,target in [(socket.AF_INET,'198.51.100.2','192.0.2.1'),(socket.AF_INET6,'2001:db8:2::2','2001:db8:1::1')]:
for kind,port in [(socket.SOCK_STREAM,42010),(socket.SOCK_DGRAM,42011)]:
s=socket.socket(family,kind); s.settimeout(2); s.bind((source,0)); s.connect((target,port)); s.sendall(b'rpf'); assert s.recv(3)==b'rpf'; s.close()
"#]).output().unwrap();
assert!(
incoming.status.success(),
"first inbound TCP/UDP must be protected: {}",
String::from_utf8_lossy(&incoming.stderr)
);
assert!(passive.0.wait().unwrap().success());
assert!(dynamic_udp.0.wait().unwrap().success());
// Repeated apply remains usable, and does not redirect application sockets.
install(&run_firewall, "tsun0", 990, true).unwrap();
run_probes("pass");
assert!(ip(&["-4", "route", "get", "198.51.100.2", "uid", "991"]).contains("dev tsun0"));
let blocked = Command::new("ip")
.args(["-6", "route", "get", "2001:db8:2::2", "uid", "991"])
.output()
.unwrap();
assert!(
!blocked.status.success(),
"application IPv6 must stay blocked"
);
route.clear_exit_client().unwrap();
clear(&run_firewall, "tsun0", true).unwrap();
for family in ["-4", "-6"] {
assert!(
ip(&[family, "rule", "show"]).contains("5270:"),
"foreign routing rules survive cleanup"
);
ip(&[family, "rule", "del", "priority", "5270"]);
}
assert!(ip(&["-4", "route", "get", "198.51.100.2", "uid", "991"]).contains("dev rpf-c"));
assert!(ip(&["-6", "route", "get", "2001:db8:2::2", "uid", "991"]).contains("dev rpf-c"));
for family in [false, true] {
let result = run_firewall(family, &["-t".into(), "mangle".into(), "-S".into()]);
if let super::linux::Ran::Exited { stdout, .. } = result {
assert!(!stdout.contains("tsunagi-exit-underlay:"));
assert!(!stdout.contains("tsunagi-exit-sockets:"));
assert!(
stdout.contains("rpfilter"),
"strict host filtering is retained"
);
}
}
}
#[test]
#[ignore = "needs root, iproute2, iptables/ip6tables and Python; changes only temporary namespaces"]
fn strict_firewall_preserves_underlay_and_application_exit_routing() {
match std::env::var(ROLE).as_deref() {
Ok("probe") => {
probes();
return;
}
Ok("client") => {
client();
return;
}
Ok("listener") => {
passive_listener();
return;
}
Ok("udp-listener") => {
udp_listener();
return;
}
_ => {}
}
let suffix = std::process::id();
let client_name = format!("tsng-rpf-c-{suffix}");
let server_name = format!("tsng-rpf-s-{suffix}");
ip(&["netns", "add", &client_name]);
let _client = Namespace(client_name.clone());
ip(&["netns", "add", &server_name]);
let _server = Namespace(server_name.clone());
ip(&[
"link",
"add",
"rpf-c",
"netns",
&client_name,
"type",
"veth",
"peer",
"name",
"rpf-s",
"netns",
&server_name,
]);
for (namespace, interface, v4, v6) in [
(&client_name, "rpf-c", "192.0.2.1/24", "2001:db8:1::1/64"),
(&server_name, "rpf-s", "192.0.2.2/24", "2001:db8:1::2/64"),
] {
ip(&["-n", namespace, "link", "set", "lo", "up"]);
ip(&["-n", namespace, "addr", "add", v4, "dev", interface]);
ip(&[
"-n", namespace, "-6", "addr", "add", v6, "dev", interface, "nodad",
]);
ip(&["-n", namespace, "link", "set", interface, "up"]);
}
ip(&[
"-n",
&server_name,
"addr",
"add",
"198.51.100.2/32",
"dev",
"lo",
]);
ip(&[
"-n",
&server_name,
"-6",
"addr",
"add",
"2001:db8:2::2/128",
"dev",
"lo",
"nodad",
]);
ip(&[
"-n",
&client_name,
"route",
"add",
"default",
"via",
"192.0.2.2",
]);
ip(&[
"-n",
&client_name,
"-6",
"route",
"add",
"default",
"via",
"2001:db8:1::2",
]);
ip(&["-n", &client_name, "link", "add", "tsun0", "type", "dummy"]);
ip(&[
"-n",
&client_name,
"addr",
"add",
"10.13.37.1/24",
"dev",
"tsun0",
]);
ip(&["-n", &client_name, "link", "set", "tsun0", "up"]);
let echo_code = r#"import socket,selectors,sys
selector=selectors.DefaultSelector()
for family in [socket.AF_INET,socket.AF_INET6]:
for kind in [socket.SOCK_DGRAM,socket.SOCK_STREAM]:
sock=socket.socket(family,kind)
if family==socket.AF_INET6: sock.setsockopt(socket.IPPROTO_IPV6,socket.IPV6_V6ONLY,1)
sock.bind(('198.51.100.2' if family==socket.AF_INET else '2001:db8:2::2',42000))
if kind==socket.SOCK_STREAM: sock.listen()
selector.register(sock,selectors.EVENT_READ,kind)
print('ready',flush=True)
while True:
for key,_ in selector.select():
if key.data==socket.SOCK_DGRAM:
data,address=key.fileobj.recvfrom(128); key.fileobj.sendto(data,address)
else:
conn,_=key.fileobj.accept(); conn.settimeout(2)
try: conn.sendall(conn.recv(128))
finally: conn.close()
"#;
let child = Command::new("ip")
.args([
"netns",
"exec",
&server_name,
"python3",
"-u",
"-c",
echo_code,
])
.stdout(std::process::Stdio::piped())
.spawn()
.unwrap();
let mut echo = Echo(child);
let mut ready = String::new();
BufReader::new(echo.0.stdout.take().unwrap())
.read_line(&mut ready)
.unwrap();
assert_eq!(ready.trim(), "ready");
let result = Command::new("ip")
.args(["netns", "exec", &client_name])
.arg(std::env::current_exe().unwrap())
.args(["--ignored", "--exact", TEST, "--nocapture"])
.env(ROLE, "client")
.env("TSUNAGI_EXIT_KERNEL_SERVER", &server_name)
.output()
.unwrap();
assert!(
result.status.success(),
"{}\n{}",
String::from_utf8_lossy(&result.stdout),
String::from_utf8_lossy(&result.stderr)
);
}
@@ -0,0 +1,535 @@
//! Own-bit connection marks for exit-client underlay traffic.
//! Reply marks are restored before host reverse-path filtering, never accepted
//! around it. No peer address becomes a host route or firewall argument.
use super::linux::{Ran, failure, locked, split_rule_line};
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
/// Outside Tailscale's 0xff0000 mask and ordinary low-bit WireGuard marks.
pub(crate) const UNDERLAY_MARK: u32 = 0x4000_0000;
pub(crate) const UNDERLAY_MASK: u32 = UNDERLAY_MARK;
const MARK: &str = "0x40000000/0x40000000";
const MASK: &str = "0x40000000";
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub(super) struct Listener {
address: IpAddr,
port: u16,
tcp: bool,
}
fn parse_listeners(listing: &str, uid: u32, v6: bool, tcp: bool) -> Result<Vec<Listener>, String> {
let mut result = Vec::new();
for line in listing.lines().skip(1) {
let fields: Vec<_> = line.split_whitespace().collect();
if (tcp && fields.get(3) != Some(&"0A"))
|| fields.get(7).and_then(|s| s.parse::<u32>().ok()) != Some(uid)
{
continue;
}
let (address, port) = fields
.get(1)
.and_then(|s| s.split_once(':'))
.ok_or("malformed local socket")?;
let port = u16::from_str_radix(port, 16).map_err(|e| e.to_string())?;
let address = if v6 {
let mut bytes = [0; 16];
for (word, chunk) in bytes.as_chunks_mut::<4>().0.iter_mut().enumerate() {
let hex = address
.get(word * 8..word * 8 + 8)
.ok_or("malformed local IPv6 listener")?;
chunk.copy_from_slice(
&u32::from_str_radix(hex, 16)
.map_err(|e| e.to_string())?
.to_ne_bytes(),
);
}
IpAddr::V6(Ipv6Addr::from(bytes))
} else {
IpAddr::V4(Ipv4Addr::from(
u32::from_str_radix(address, 16)
.map_err(|e| e.to_string())?
.to_ne_bytes(),
))
};
if result.len() == 128 {
return Err("too many local sockets for exit protection".into());
}
if port != 0 {
result.push(Listener { address, port, tcp });
}
}
Ok(result)
}
pub(super) fn local_sockets(uid: u32, v6: bool) -> Result<Vec<Listener>, String> {
let mut listeners = Vec::new();
for (protocol, tcp) in [("tcp", true), ("udp", false)] {
for family in [false, true].into_iter().filter(|f| !f || v6) {
let path = format!("/proc/net/{protocol}{}", if family { "6" } else { "" });
listeners.extend(parse_listeners(
&std::fs::read_to_string(path).map_err(|e| e.to_string())?,
uid,
family,
tcp,
)?);
}
}
listeners.sort();
listeners.dedup();
Ok(listeners)
}
fn listener_rules(interface: &str, listener: &Listener) -> Vec<Vec<String>> {
let mut base = words(&[
"!",
"-i",
interface,
"-p",
if listener.tcp { "tcp" } else { "udp" },
"--dport",
&listener.port.to_string(),
]);
if !listener.address.is_unspecified() {
base.extend(words(&["-d", &listener.address.to_string()]));
}
// Match an actual local socket, not just a destination port. Listening
// sockets need protection on the first packet, before any outbound connmark.
base.extend(words(&[
"-m",
"socket",
"--nowildcard",
"-m",
"comment",
"--comment",
&listener_tag(interface),
]));
let save = base
.iter()
.cloned()
.chain(words(&[
"-m",
"mark",
"--mark",
MARK,
"-j",
"CONNMARK",
"--save-mark",
"--nfmask",
MASK,
"--ctmask",
MASK,
]))
.collect();
let mark = base
.into_iter()
.chain(words(&["-j", "MARK", "--set-xmark", MARK]))
.collect();
vec![save, mark]
}
pub(super) type RunFamily<'a> = &'a dyn Fn(bool, &[String]) -> Ran;
fn tag(interface: &str) -> String {
format!("tsunagi-exit-underlay:{interface}")
}
fn listener_tag(interface: &str) -> String {
format!("tsunagi-exit-sockets:{interface}")
}
fn words(values: &[&str]) -> Vec<String> {
values.iter().map(|s| (*s).into()).collect()
}
fn rules(interface: &str, uid: u32) -> Vec<(&'static str, Vec<String>)> {
let tag = tag(interface);
let uid = uid.to_string();
// Insert at position one in this order: final OUTPUT order is MARK, SAVE.
vec![
(
"PREROUTING",
words(&[
"!",
"-i",
interface,
"-m",
"connmark",
"--mark",
MARK,
"-m",
"comment",
"--comment",
&tag,
"-j",
"CONNMARK",
"--restore-mark",
"--nfmask",
MASK,
"--ctmask",
MASK,
]),
),
(
"OUTPUT",
words(&[
"-m",
"owner",
"--uid-owner",
&uid,
"-m",
"mark",
"--mark",
MARK,
"-m",
"comment",
"--comment",
&tag,
"-j",
"CONNMARK",
"--save-mark",
"--nfmask",
MASK,
"--ctmask",
MASK,
]),
),
(
"OUTPUT",
words(&[
"-m",
"owner",
"--uid-owner",
&uid,
"-m",
"comment",
"--comment",
&tag,
"-j",
"MARK",
"--set-xmark",
MARK,
]),
),
]
}
fn execute(run: RunFamily<'_>, v6: bool, args: Vec<String>) -> Result<String, String> {
match run(v6, &locked(args)) {
Ran::Exited {
success: true,
stdout,
..
} => Ok(stdout),
Ran::Exited { stderr, .. } => Err(failure(&stderr)),
Ran::Missing => Err(format!(
"{} is required for exit-node connection marks",
if v6 { "ip6tables" } else { "iptables" }
)),
}
}
/// Removes only exact interface tags, retaining other agents and host rules.
pub(super) fn clear(run: RunFamily<'_>, interface: &str, v6: bool) -> Result<(), String> {
clear_tags(run, interface, v6, false)
}
fn clear_tags(
run: RunFamily<'_>,
interface: &str,
v6: bool,
sockets_only: bool,
) -> Result<(), String> {
let wanted = tag(interface);
let sockets = listener_tag(interface);
for family in [false, true].into_iter().filter(|family| !family || v6) {
let chains: &[&str] = if sockets_only {
&["PREROUTING"]
} else {
&["OUTPUT", "PREROUTING"]
};
for chain in chains {
let listing = execute(run, family, words(&["-t", "mangle", "-S", chain]))?;
for line in listing.lines() {
let tokens = split_rule_line(line);
let [append, listed_chain, rest @ ..] = tokens.as_slice() else {
continue;
};
if append == "-A"
&& listed_chain == *chain
&& rest.windows(2).any(|pair| {
pair[0] == "--comment"
&& (pair[1] == sockets || (!sockets_only && pair[1] == wanted))
})
{
let args = words(&["-t", "mangle", "-D", chain])
.into_iter()
.chain(rest.iter().cloned())
.collect();
execute(run, family, args)?;
}
}
}
}
Ok(())
}
/// Establishes protection before routing may redirect application traffic.
pub(super) fn install(
run: RunFamily<'_>,
interface: &str,
uid: u32,
v6: bool,
) -> Result<Vec<Listener>, String> {
let listeners = local_sockets(uid, v6)?;
install_with_listeners(run, interface, uid, v6, &listeners)?;
Ok(listeners)
}
fn install_with_listeners(
run: RunFamily<'_>,
interface: &str,
uid: u32,
v6: bool,
listeners: &[Listener],
) -> Result<(), String> {
clear(run, interface, v6)?;
let result = (|| {
for family in [false, true].into_iter().filter(|family| !family || v6) {
for (chain, spec) in rules(interface, uid) {
let args = words(&["-t", "mangle", "-I", chain, "1"])
.into_iter()
.chain(spec)
.collect();
execute(run, family, args)?;
}
for listener in listeners.iter().filter(|s| s.address.is_ipv6() == family) {
for spec in listener_rules(interface, listener) {
execute(
run,
family,
words(&["-t", "mangle", "-I", "PREROUTING", "1"])
.into_iter()
.chain(spec)
.collect(),
)?;
}
}
}
Ok(())
})();
if result.is_err()
&& let Err(cleanup) = clear(run, interface, v6)
{
return Err(format!(
"{}; cannot roll back connection marks: {cleanup}",
result.err().unwrap_or_default()
));
}
result
}
/// Updates passive protection without interrupting established connmark rules.
pub(super) fn refresh(
run: RunFamily<'_>,
interface: &str,
v6: bool,
listeners: &[Listener],
) -> Result<(), String> {
clear_tags(run, interface, v6, true)?;
for family in [false, true].into_iter().filter(|f| !f || v6) {
for listener in listeners.iter().filter(|s| s.address.is_ipv6() == family) {
for spec in listener_rules(interface, listener) {
execute(
run,
family,
words(&["-t", "mangle", "-I", "PREROUTING", "1"])
.into_iter()
.chain(spec)
.collect(),
)?;
}
}
}
Ok(())
}
/// Kernel strict RPF must include the restored mark, independently of iptables.
pub(super) fn check_kernel_rpf(
all_rpf: u32,
all_valid: u32,
interfaces: &[(String, u32, u32)],
) -> Result<(), String> {
for (interface, rpf, valid) in interfaces {
if all_rpf.max(*rpf) == 1 && all_valid.max(*valid) == 0 {
return Err(format!(
"strict rp_filter on `{interface}` ignores connection marks; enable net.ipv4.conf.all.src_valid_mark=1 before using an exit node (rp_filter stays enabled)"
));
}
}
Ok(())
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use std::cell::RefCell;
#[test]
fn both_families_restore_before_firewall_and_preserve_foreign_mark_bits() {
let calls = RefCell::new(Vec::new());
let run = |v6, args: &[String]| {
calls.borrow_mut().push((v6, args.to_vec()));
Ran::Exited {
success: true,
stdout: String::new(),
stderr: String::new(),
}
};
install_with_listeners(&run, "tsun0", 990, true, &[]).unwrap();
let calls = calls.borrow();
for family in [false, true] {
let inserts: Vec<_> = calls
.iter()
.filter(|(v6, a)| *v6 == family && a.iter().any(|s| s == "-I"))
.map(|(_, a)| a.join(" "))
.collect();
assert_eq!(inserts.len(), 3);
assert!(inserts[0].contains("-I PREROUTING 1 ! -i tsun0 -m connmark"));
assert!(inserts[0].contains("--restore-mark --nfmask 0x40000000 --ctmask 0x40000000"));
assert!(inserts[1].contains("--uid-owner 990"));
assert!(inserts[1].contains("--save-mark"));
assert!(inserts[2].contains("--set-xmark 0x40000000/0x40000000"));
assert!(
!inserts
.iter()
.any(|s| s.contains("-j ACCEPT") || s.contains("-j RETURN"))
);
}
assert_eq!(UNDERLAY_MASK & 0x00ff_0000, 0);
}
#[test]
fn cleanup_retains_other_interfaces_and_rolls_back_partial_ipv6_installation() {
let calls = RefCell::new(Vec::new());
let run = |v6, args: &[String]| {
calls.borrow_mut().push((v6, args.to_vec()));
let listing = "-A OUTPUT -m comment --comment tsunagi-exit-underlay:tsun0 -j MARK --set-xmark 0x40000000/0x40000000\n-A OUTPUT -m comment --comment tsunagi-exit-underlay:tsun1 -j MARK --set-xmark 0x40000000/0x40000000\n-A OUTPUT -m comment --comment unrelated -j RETURN\n";
Ran::Exited {
success: !(v6 && args.iter().any(|s| s == "-I")),
stdout: if args.iter().any(|s| s == "-S") {
listing.into()
} else {
String::new()
},
stderr: "injected IPv6 failure".into(),
}
};
assert!(install_with_listeners(&run, "tsun0", 990, true, &[]).is_err());
let calls = calls.borrow();
let deletions: Vec<_> = calls
.iter()
.filter(|(_, a)| a.iter().any(|s| s == "-D"))
.collect();
assert!(
deletions.len() >= 4,
"both initial and rollback cleanup run"
);
assert!(
deletions
.iter()
.all(|(_, a)| a.iter().any(|s| s == "tsunagi-exit-underlay:tsun0"))
);
}
#[test]
fn kernel_strict_validation_requires_mark_awareness_without_weakening_rpf() {
let interfaces = vec![("eth0".into(), 1, 0)];
assert!(check_kernel_rpf(0, 0, &interfaces).is_err());
assert!(check_kernel_rpf(0, 1, &interfaces).is_ok());
assert!(check_kernel_rpf(2, 0, &interfaces).is_ok());
assert!(check_kernel_rpf(0, 0, &[("eth0".into(), 1, 1)]).is_ok());
}
#[test]
fn passive_tcp_protection_uses_only_locally_owned_listeners() {
let listing = "header\n0: 00000000:01BB 00000000:0000 0A 0:0 0:0 0 990 0 1\n1: 0100007F:1111 00000000:0000 0A 0:0 0:0 0 1000 0 2\n2: 0100007F:2222 00000000:0000 01 0:0 0:0 0 990 0 3\n";
let listeners = parse_listeners(listing, 990, false, true).unwrap();
assert_eq!(
listeners,
vec![Listener {
address: IpAddr::V4(Ipv4Addr::UNSPECIFIED),
port: 443,
tcp: true
}]
);
let rules = listener_rules("tsun0", &listeners[0]);
assert!(
rules[0]
.join(" ")
.contains("-p tcp --dport 443 -m socket --nowildcard")
);
assert!(rules[0].join(" ").contains("--save-mark"));
assert!(rules[1].join(" ").contains("--set-xmark"));
}
#[test]
fn listener_parser_checks_family_encoding_and_bounds() {
let loopback = Ipv6Addr::LOCALHOST.octets();
let hex: String = loopback
.as_chunks::<4>()
.0
.iter()
.map(|bytes| format!("{:08X}", u32::from_ne_bytes(*bytes)))
.collect();
let listing = format!("header\n0: {hex}:01BB 0:0 0A 0:0 0:0 0 990 0 1\n");
assert_eq!(
parse_listeners(&listing, 990, true, true).unwrap(),
vec![Listener {
address: IpAddr::V6(Ipv6Addr::LOCALHOST),
port: 443,
tcp: true
}]
);
assert!(
parse_listeners(
"header\n0: BAD:01BB 0:0 0A 0:0 0:0 0 990 0 1",
990,
true,
true
)
.is_err()
);
let huge = format!(
"header\n{}",
"0: 00000000:01BB 0:0 0A 0:0 0:0 0 990 0 1\n".repeat(129)
);
assert!(parse_listeners(&huge, 990, false, true).is_err());
}
#[test]
fn udp_refresh_preserves_general_connmark_protection_and_other_interfaces() {
let listing = "header\n0: 00000000:ABA0 0:0 07 0:0 0:0 0 990 0 1\n1: 00000000:0000 0:0 07 0:0 0:0 0 990 0 2\n";
let sockets = parse_listeners(listing, 990, false, false).unwrap();
assert_eq!(sockets.len(), 1);
assert!(!sockets[0].tcp);
let calls = RefCell::new(Vec::new());
let run = |v6, args: &[String]| {
calls.borrow_mut().push((v6, args.join(" ")));
Ran::Exited { success: true, stdout: "-A PREROUTING -m comment --comment tsunagi-exit-underlay:tsun0 -j CONNMARK --restore-mark\n-A PREROUTING -m comment --comment tsunagi-exit-sockets:tsun0 -j MARK\n-A PREROUTING -m comment --comment tsunagi-exit-sockets:tsun1 -j MARK\n".into(), stderr: String::new() }
};
refresh(&run, "tsun0", false, &sockets).unwrap();
let calls = calls.borrow();
assert!(
calls
.iter()
.any(|(_, a)| a.contains("-I PREROUTING 1") && a.contains("-p udp"))
);
assert!(
calls
.iter()
.filter(|(_, a)| a.contains("-D"))
.all(|(_, a)| a.contains("tsunagi-exit-sockets:tsun0"))
);
}
}
@@ -55,6 +55,10 @@ pub use exit::{
mod linux;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
mod linux_exit;
#[cfg(all(test, feature = "tun-device", target_os = "linux"))]
mod linux_exit_kernel;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub(in crate::overlay) mod linux_exit_mark;
#[cfg(all(feature = "tun-device", target_os = "linux"))]
pub use linux::LinuxHostRules;
@@ -397,6 +397,14 @@ impl Interface {
return;
}
if planned.as_ref() == Some(&wanted) && ok {
if let Some(outcome) = exit.refresh(&wanted).await {
if let RuleOutcome::Failed(reason) = &outcome {
tracing::warn!(interface = %self.name, %reason, "cannot refresh exit-client socket protection");
}
if let Some(report) = self.exit_state().report.as_mut() {
report.client = Some(outcome);
}
}
return;
}
let report = exit.apply(&wanted).await;
@@ -55,6 +55,7 @@ use crate::BoxFuture;
use crate::overlay::OverlayError;
use super::super::config::Cidr;
use super::super::hostrules::linux_exit_mark::{UNDERLAY_MARK, UNDERLAY_MASK};
use super::super::tun::{TunDevice, TunRequest};
use super::privilege::{NetAdmin, Privilege, probe_net_admin};
use super::{
@@ -641,8 +642,10 @@ async fn del_broadcast_route(name: &str) -> Result<(), OverlayError> {
const EXIT_TABLE: u32 = 0x7473;
/// The main table.
const MAIN_TABLE: u32 = 254;
/// Where the exit rules sit, after Tailscale's 5270.
const EXIT_BYPASS_PRIORITY: u32 = 5280;
/// Protect marked underlay and initial socket routing before Tailscale's rules.
const EXIT_MARK_PRIORITY: u32 = 5205;
const EXIT_BYPASS_PRIORITY: u32 = 5206;
const LEGACY_EXIT_BYPASS_PRIORITY: u32 = 5280;
const EXIT_SPECIFIC_PRIORITY: u32 = 5290;
const EXIT_DEFAULT_PRIORITY: u32 = 5300;
@@ -666,14 +669,23 @@ fn rule_priority(rule: &RuleMessage) -> Option<u32> {
})
}
/// Whether a rule is one of the three this agent adds for an exit node.
/// Whether a rule is one this agent adds for an exit node, including old UID priority.
///
/// Recognised by shape as well as priority: those numbers are not reserved,
/// and a rule somebody else put at one of them must not be removed.
fn is_exit_rule(rule: &RuleMessage) -> bool {
let table = rule_table(rule);
match rule_priority(rule) {
Some(EXIT_BYPASS_PRIORITY) => {
Some(EXIT_MARK_PRIORITY) => {
table == MAIN_TABLE
&& rule
.attributes
.contains(&RuleAttribute::FwMark(UNDERLAY_MARK))
&& rule
.attributes
.contains(&RuleAttribute::FwMask(UNDERLAY_MASK))
}
Some(EXIT_BYPASS_PRIORITY | LEGACY_EXIT_BYPASS_PRIORITY) => {
table == MAIN_TABLE
&& rule
.attributes
@@ -789,7 +801,8 @@ async fn remove_exit_client(handle: &rtnetlink::Handle) -> Result<(), OverlayErr
///
/// ```text
/// ip route replace default dev <if> table 0x7473
/// ip rule add priority 5280 uidrange <uid>-<uid> lookup main
/// ip rule add priority 5205 fwmark 0x40000000/0x40000000 lookup main
/// ip rule add priority 5206 uidrange <uid>-<uid> lookup main
/// ip rule add priority 5290 lookup main suppress_prefixlength 0
/// ip rule add priority 5300 lookup 0x7473
/// ```
@@ -819,6 +832,8 @@ async fn set_exit_client(name: &str, uid: u32) -> Result<(), OverlayError> {
.await
.map_err(|err| fail("default route", err))?;
add_mark_bypass(&handle, false).await?;
let mut bypass = handle
.rule()
.add()
@@ -891,6 +906,7 @@ async fn set_exit_block_v6(handle: &rtnetlink::Handle, uid: u32) -> Result<(), O
};
}
add_mark_bypass(handle, true).await?;
let mut bypass = handle
.rule()
.add()
@@ -938,6 +954,27 @@ async fn set_exit_block_v6(handle: &rtnetlink::Handle, uid: u32) -> Result<(), O
.map_err(|err| fail("rule for the default route", err))
}
async fn add_mark_bypass(handle: &rtnetlink::Handle, v6: bool) -> Result<(), OverlayError> {
let mut rule = handle
.rule()
.add()
.table_id(MAIN_TABLE)
.action(RuleAction::ToTable)
.priority(EXIT_MARK_PRIORITY)
.fw_mark(UNDERLAY_MARK);
rule.message_mut().header.family = if v6 {
rtnetlink::packet_route::AddressFamily::Inet6
} else {
rtnetlink::packet_route::AddressFamily::Inet
};
rule.message_mut()
.attributes
.push(RuleAttribute::FwMask(UNDERLAY_MASK));
rule.execute().await.map_err(|err| {
OverlayError::Unavailable(format!("cannot protect marked underlay traffic: {err}"))
})
}
async fn configure_link(configure: &Configure) -> Result<(), OverlayError> {
let name = configure.name.as_str();
with_netlink(|handle| async move {
@@ -1110,6 +1147,28 @@ mod tests {
assert!(!is_exit_rule(&rule(5280, 100, vec![uid])));
}
#[test]
fn underlay_bypass_recognises_only_our_exact_bit_and_mask() {
let rule = |table, mark, mask| {
let mut message = RuleMessage::default();
message.attributes.extend([
RuleAttribute::Table(table),
RuleAttribute::Priority(EXIT_MARK_PRIORITY),
RuleAttribute::FwMark(mark),
RuleAttribute::FwMask(mask),
]);
message
};
assert!(is_exit_rule(&rule(
MAIN_TABLE,
UNDERLAY_MARK,
UNDERLAY_MASK
)));
assert!(!is_exit_rule(&rule(MAIN_TABLE, 0x80000, 0xff0000)));
assert!(!is_exit_rule(&rule(MAIN_TABLE, UNDERLAY_MARK, u32::MAX)));
assert!(!is_exit_rule(&rule(52, UNDERLAY_MARK, UNDERLAY_MASK)));
}
#[test]
fn the_ipv6_default_of_the_exit_table_refuses_instead_of_routing() {
let route = exit_blocking_route_v6();
+30 -2
View File
@@ -20,7 +20,7 @@ use crate::error::ProtocolError;
/// The version in the ALPN is the wire-compatibility version of the control
/// protocol. It is independent of the network identity scheme version, so
/// bumping it must not change any existing [`crate::NetworkId`].
pub const ALPN: &[u8] = b"tsunagi/ctrl/6";
pub const ALPN: &[u8] = b"tsunagi/ctrl/7";
/// ALPN of the tsunagi data plane.
///
@@ -43,7 +43,7 @@ pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
pub const MAX_SIGNATURE_LEN: usize = 64;
/// Control protocol version carried inside the handshake.
pub const PROTOCOL_VERSION: u16 = 6;
pub const PROTOCOL_VERSION: u16 = 7;
pub use crate::dataplane::transport::{MAX_TRANSPORT_PAYLOAD, TransportPayload};
@@ -181,6 +181,11 @@ pub enum ControlMessage {
/// Short free-text reason.
reason: String,
},
/// Signed cleanup markers, each proving the retired claim version.
Retirements {
/// Bounded by the per-message signed-state limit.
markers: Vec<crate::state::SignedRetirement>,
},
}
/// Somewhere a member has been seen.
@@ -273,6 +278,28 @@ pub fn validate_capability(
/// network, the other networks or the agent.
pub fn validate(message: &ControlMessage, limits: &Limits) -> Result<(), ProtocolError> {
match message {
ControlMessage::Retirements { markers } => {
check_len("retirements", markers.len(), limits.max_state_records)?;
for marker in markers {
check_len(
"retirement.signature",
marker.signature.len(),
MAX_SIGNATURE_LEN,
)?;
check_len(
"retirement.target.signature",
marker.target.signature.len(),
MAX_SIGNATURE_LEN,
)?;
if let Some(name) = marker.target.body.hostname() {
check_len(
"retirement.target.hostname",
name.len(),
limits.max_hostname_len,
)?;
}
}
}
ControlMessage::Transport { protocol, payload } => {
if protocol.is_empty() {
return Err(ProtocolError::Malformed("empty transport protocol"));
@@ -345,6 +372,7 @@ pub fn kind(message: &ControlMessage) -> &'static str {
ControlMessage::Ping { .. } => "ping",
ControlMessage::Pong { .. } => "pong",
ControlMessage::State { .. } => "state",
ControlMessage::Retirements { .. } => "retirements",
ControlMessage::Reachable { .. } => "reachable",
ControlMessage::Peers { .. } => "peers",
ControlMessage::Bye { .. } => "bye",
+84 -2
View File
@@ -28,16 +28,23 @@
//! replica that has the record needs nothing else to interpret it, and
//! recovery never depends on replaying a chain from the beginning.
//!
//! Any member may also sign a separate retirement of another author's proved
//! claim version. It suppresses stale reservations without changing that
//! author's record, and a newer claim remains valid.
//!
//! # What this slice does not do yet
//!
//! Compaction, revocation of a whole author, and snapshots covering more than
//! one record type. See `docs/sync-model.md` for the shape those take.
//! Compaction and permanent revocation of a whole author. See
//! `docs/sync-model.md` for the implemented cleanup semantics.
pub mod allocator;
use std::collections::HashMap;
use std::net::Ipv4Addr;
mod retirement;
pub use retirement::{MAX_RETIREMENT_LEN, SignedRetirement};
use iroh::{EndpointId, SecretKey, Signature};
use serde::{Deserialize, Serialize};
@@ -488,6 +495,7 @@ pub enum Merged {
#[derive(Debug, Clone, Default)]
pub struct StateSet {
records: HashMap<EndpointId, SignedRecord>,
retirements: HashMap<EndpointId, SignedRetirement>,
}
impl StateSet {
@@ -500,6 +508,7 @@ impl StateSet {
pub fn from_verified(records: impl IntoIterator<Item = (EndpointId, SignedRecord)>) -> Self {
Self {
records: records.into_iter().collect(),
retirements: HashMap::new(),
}
}
@@ -528,6 +537,52 @@ impl StateSet {
.collect()
}
/// Visible records, excluding versions retired by authenticated members.
pub fn active_records(&self) -> Vec<SignedRecord> {
self.records()
.into_iter()
.filter(|r| r.author_id().is_ok_and(|id| !self.is_retired(&id)))
.collect()
}
/// Whether the current known version is at or below a proved retirement.
pub fn is_retired(&self, author: &EndpointId) -> bool {
self.retirements
.get(author)
.is_some_and(|m| self.version_of(author) <= m.target.version)
}
/// Strongest retained cleanup proof for an identity.
pub fn retirement_of(&self, target: &EndpointId) -> Option<&SignedRetirement> {
self.retirements.get(target)
}
/// Durable markers in target-id order, including markers surpassed by a fresh claim.
pub fn retirements(&self) -> Vec<SignedRetirement> {
let mut markers: Vec<_> = self.retirements.values().cloned().collect();
markers.sort_by_key(|m| m.target.author);
markers
}
/// Accepts any member's signed cleanup after checking both signatures.
pub fn merge_retirement(
&mut self,
network: NetworkId,
marker: SignedRetirement,
max: usize,
) -> Result<bool, StateError> {
let target = marker.verify(network)?;
if let Some(old) = self.retirements.get(&target) {
if !marker.supersedes(old) {
return Ok(false);
}
} else if self.retirements.len() >= max {
return Err(StateError::Malformed("retirement capacity reached"));
}
self.retirements.insert(target, marker);
Ok(true)
}
/// Merges one record, verifying it first.
///
/// Merging is into the existing set, never a wholesale replacement, and an
@@ -591,6 +646,9 @@ impl StateSet {
pub fn address_holders(&self) -> HashMap<Ipv4Addr, EndpointId> {
let mut holders: HashMap<Ipv4Addr, EndpointId> = HashMap::new();
for (author, record) in &self.records {
if self.is_retired(author) {
continue;
}
let Some(address) = record.body.claimed_address() else {
continue;
};
@@ -618,6 +676,9 @@ impl StateSet {
pub fn hostname_holders(&self) -> HashMap<String, EndpointId> {
let mut holders: HashMap<String, (u64, EndpointId)> = HashMap::new();
for (author, record) in &self.records {
if self.is_retired(author) {
continue;
}
let Some(hostname) = record.body.hostname() else {
continue;
};
@@ -646,6 +707,9 @@ impl StateSet {
/// The name an author claimed and the member that holds it instead, when
/// somebody who claimed it earlier does.
pub fn hostname_taken(&self, author: &EndpointId) -> Option<(&str, EndpointId)> {
if self.is_retired(author) {
return None;
}
let hostname = self.records.get(author)?.body.hostname()?;
let holder = *self.hostname_holders().get(hostname)?;
(holder != *author).then_some((hostname, holder))
@@ -671,6 +735,12 @@ impl StateSet {
pub fn agreed_range(&self) -> Option<Ipv4Range> {
self.records
.iter()
.chain(
self.retirements
.iter()
.filter(|(author, _)| !self.records.contains_key(*author))
.map(|(author, marker)| (author, &marker.target)),
)
.filter_map(|(author, record)| {
let range = record.body.range()?;
let derived = derived_ipv4_range(NetworkId::from_bytes(record.network));
@@ -684,6 +754,18 @@ impl StateSet {
pub fn version_of(&self, author: &EndpointId) -> u64 {
self.records.get(author).map_or(0, |record| record.version)
}
/// Highest witnessed author counter across claims and authored cleanup.
/// This differs from the claim version used to decide retirement.
pub fn authored_version(&self, author: &EndpointId) -> u64 {
self.retirements
.values()
.filter(|m| m.author == *author.as_bytes())
.map(|m| m.version)
.chain(std::iter::once(self.version_of(author)))
.max()
.unwrap_or(0)
}
}
#[cfg(test)]
+219
View File
@@ -0,0 +1,219 @@
//! A member may retire an observed claim, never another identity's future.
use iroh::{EndpointId, SecretKey, Signature};
use serde::{Deserialize, Serialize};
use super::{RecordBody, SignedRecord, StateError, push_lp};
use crate::NetworkId;
/// Maximum encoded retirement size, including its bounded claim proof.
pub const MAX_RETIREMENT_LEN: usize = 1024;
/// A network member's signed retirement of a proved claim version.
///
/// The original author's signed claim proves the cutoff really existed.
/// Newer claims remain valid. This is neither a forged Release nor a ban.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct SignedRetirement {
/// Device identity of the member requesting cleanup.
pub author: [u8; 32],
/// Network in which cleanup applies.
pub network: [u8; 32],
/// Requester's durable author counter, committed with this marker.
pub version: u64,
/// The target's original signed claim; bounds the retired version.
pub target: SignedRecord,
/// Requester's Ed25519 signature over this marker and its proof.
pub signature: Vec<u8>,
}
impl SignedRetirement {
fn canonical_bytes(&self) -> Vec<u8> {
let mut out = Vec::with_capacity(768);
push_lp(&mut out, b"tsunagi-node-retirement/v1");
push_lp(&mut out, &self.network);
push_lp(&mut out, &self.author);
push_lp(&mut out, &self.version.to_be_bytes());
// Bind every field of the original signed claim, including its signature.
push_lp(&mut out, &self.target.network);
push_lp(&mut out, &self.target.author);
push_lp(&mut out, &self.target.version.to_be_bytes());
push_lp(&mut out, &self.target.body.canonical());
push_lp(&mut out, &self.target.signature);
out
}
/// Signs a marker with the requesting member's persistent device key.
pub fn sign(
secret: &SecretKey,
network: NetworkId,
version: u64,
target: SignedRecord,
) -> Self {
let mut marker = Self {
author: *secret.public().as_bytes(),
network: *network.as_bytes(),
version,
target,
signature: Vec::new(),
};
marker.signature = secret.sign(&marker.canonical_bytes()).to_bytes().to_vec();
marker
}
/// Verifies both authors, scope and a finite, genuinely signed cutoff.
pub fn verify(&self, network: NetworkId) -> Result<EndpointId, StateError> {
if self.network != *network.as_bytes() {
return Err(StateError::WrongNetwork);
}
if self.version == 0 || self.version > i64::MAX as u64 {
return Err(StateError::Malformed("retirement version is out of range"));
}
if !matches!(self.target.body, RecordBody::Claim { .. }) {
return Err(StateError::Malformed("only a claim can be retired"));
}
let target = self.target.verify(network)?;
let author = EndpointId::from_bytes(&self.author).map_err(|_| StateError::BadAuthor)?;
if author == target {
return Err(StateError::Malformed(
"use Release to retire your own claim",
));
}
let raw: [u8; Signature::LENGTH] =
self.signature
.as_slice()
.try_into()
.map_err(|_| StateError::BadSignatureLength {
expected: Signature::LENGTH,
})?;
author
.verify(&self.canonical_bytes(), &Signature::from_bytes(&raw))
.map_err(|_| StateError::BadSignature)?;
Ok(target)
}
/// A deterministic strongest marker per target, independent of arrival order.
pub(crate) fn supersedes(&self, other: &Self) -> bool {
self.target.version > other.target.version
|| (self.target.version == other.target.version
&& (self.canonical_bytes(), &self.signature)
< (other.canonical_bytes(), &other.signature))
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
use crate::state::{Ipv4Range, StateSet};
fn claim(key: &SecretKey, id: NetworkId, version: u64) -> SignedRecord {
SignedRecord::sign(
key,
id,
version,
RecordBody::Claim {
address: Some("10.23.0.2".parse().unwrap()),
range: Some("10.23.0.0/24".parse::<Ipv4Range>().unwrap()),
hostname: Some("retired".into()),
hostname_since: Some(123),
},
)
}
#[test]
fn markers_release_claims_in_any_merge_order_and_older_snapshots_cannot_restore_them() {
let id = NetworkId::from_bytes([7; 32]);
let target = SecretKey::generate();
let cleaner = SecretKey::generate();
let record = claim(&target, id, 5);
let marker = SignedRetirement::sign(&cleaner, id, 8, record.clone());
for first in [true, false] {
let mut state = StateSet::new();
if first {
state.merge_retirement(id, marker.clone(), 16).unwrap();
}
state.merge(id, record.clone()).unwrap();
if !first {
state.merge_retirement(id, marker.clone(), 16).unwrap();
}
assert!(state.address_holders().is_empty());
assert!(state.hostname_holders().is_empty());
state.merge(id, claim(&target, id, 4)).unwrap();
assert!(state.is_retired(&target.public()));
assert!(
state.agreed_range().is_some(),
"cleanup does not renumber the network"
);
state.merge(id, claim(&target, id, 6)).unwrap();
assert!(!state.is_retired(&target.public()));
assert_eq!(state.hostname_of(&target.public()), Some("retired"));
}
}
#[test]
fn proof_signature_scope_tampering_future_cutoffs_and_capacity_are_checked() {
let id = NetworkId::from_bytes([7; 32]);
let target = SecretKey::generate();
let cleaner = SecretKey::generate();
let marker = SignedRetirement::sign(&cleaner, id, 8, claim(&target, id, 5));
assert!(marker.verify(NetworkId::from_bytes([8; 32])).is_err());
let mut forged = marker.clone();
forged.target.version = 99;
assert!(forged.verify(id).is_err());
let mut forged = marker.clone();
forged.signature[0] ^= 1;
assert!(forged.verify(id).is_err());
let mut proof = marker.target.clone();
proof.version = 99;
assert!(
SignedRetirement::sign(&cleaner, id, 8, proof)
.verify(id)
.is_err()
);
let mut state = StateSet::new();
assert!(state.merge_retirement(id, marker.clone(), 0).is_err());
assert!(state.merge_retirement(id, marker.clone(), 1).unwrap());
assert!(!state.merge_retirement(id, marker.clone(), 1).unwrap());
assert!(postcard::to_stdvec(&marker).unwrap().len() <= MAX_RETIREMENT_LEN);
let another = SecretKey::generate();
assert!(
state
.merge_retirement(
id,
SignedRetirement::sign(&cleaner, id, 9, claim(&another, id, 1)),
1
)
.is_err()
);
}
#[test]
fn competing_cleanup_markers_converge_and_keep_the_highest_observed_cutoff() {
let id = NetworkId::from_bytes([7; 32]);
let target = SecretKey::generate();
let a = SecretKey::generate();
let b = SecretKey::generate();
let markers = [
SignedRetirement::sign(&a, id, 1, claim(&target, id, 4)),
SignedRetirement::sign(&a, id, 2, claim(&target, id, 5)),
SignedRetirement::sign(&b, id, 3, claim(&target, id, 5)),
];
let mut forward = StateSet::new();
let mut reverse = StateSet::new();
for marker in &markers {
forward.merge_retirement(id, marker.clone(), 16).unwrap();
}
for marker in markers.iter().rev() {
reverse.merge_retirement(id, marker.clone(), 16).unwrap();
}
assert_eq!(forward.retirements(), reverse.retirements());
assert_eq!(
forward
.retirement_of(&target.public())
.unwrap()
.target
.version,
5
);
}
}
+26
View File
@@ -310,6 +310,7 @@ impl Storage {
self.with_state(move |state| {
state.remove_network(network_id)?;
state.forget_signed_records(network_id)?;
state.forget_retirements(network_id)?;
state.forget_peer_hostnames(network_id)
})
.await?;
@@ -369,6 +370,31 @@ impl Storage {
.await
}
/// Loads verified durable cleanup proofs off the executor.
pub async fn retirements(
&self,
network: NetworkId,
max: usize,
) -> Result<Vec<crate::state::SignedRetirement>> {
self.with_state(move |state| state.retirements(network, max))
.await
}
/// Persists a received cleanup proof before applying it.
pub async fn put_retirement(&self, marker: crate::state::SignedRetirement) -> Result<()> {
self.with_state(move |state| state.put_retirement(&marker))
.await
}
/// Commits our cleanup and author counter together before announcement.
pub async fn publish_own_retirement(
&self,
marker: crate::state::SignedRetirement,
) -> Result<()> {
self.with_state(move |state| state.publish_own_retirement(&marker))
.await
}
/// Stores a record received from another replica.
pub async fn put_signed_record(&self, record: SignedRecord) -> Result<()> {
self.with_state(move |state| state.put_signed_record(&record))
+212 -2
View File
@@ -18,10 +18,10 @@ use rusqlite::{Connection, OptionalExtension, params};
use crate::error::{Error, Result};
use crate::identity::{DeviceIdentity, NetworkId, NetworkName, NetworkSecret};
use crate::state::{RecordBody, SignedRecord};
use crate::state::{MAX_RETIREMENT_LEN, RecordBody, SignedRecord, SignedRetirement};
/// Schema version written by this build.
pub const SCHEMA_VERSION: i64 = 8;
pub const SCHEMA_VERSION: i64 = 9;
/// Key of the stored hostname setting.
const SETTING_HOSTNAME: &str = "hostname";
@@ -250,6 +250,19 @@ impl StateStore {
)
.map_err(|err| self.corrupt(format!("cannot migrate schema to 8: {err}")))?;
}
if found < 9 {
self.conn
.execute_batch(
"BEGIN;
CREATE TABLE IF NOT EXISTS retirements (
network_id BLOB NOT NULL, target BLOB NOT NULL, data BLOB NOT NULL,
PRIMARY KEY (network_id, target)
);
PRAGMA user_version = 9;
COMMIT;",
)
.map_err(|err| self.corrupt(format!("cannot migrate schema to 9: {err}")))?;
}
Ok(())
}
@@ -262,6 +275,7 @@ impl StateStore {
"settings",
"signed_records",
"peer_hostnames",
"retirements",
] {
let present: Option<String> = self
.conn
@@ -785,6 +799,112 @@ impl StateStore {
Ok(version.unwrap_or(0).max(0) as u64)
}
/// Loads durable cleanup markers without allocating oversized stored blobs.
pub fn retirements(&self, network: NetworkId, max: usize) -> Result<Vec<SignedRetirement>> {
let mut stmt = self
.conn
.prepare(
"SELECT CASE WHEN length(data) <= ?2 THEN data ELSE NULL END
FROM retirements WHERE network_id = ?1 ORDER BY target",
)
.map_err(|err| self.corrupt(err))?;
let rows = stmt
.query_map(
params![network.as_bytes().as_slice(), MAX_RETIREMENT_LEN as i64],
|row| row.get::<_, Option<Vec<u8>>>(0),
)
.map_err(|err| self.corrupt(err))?;
let mut out = Vec::new();
for row in rows {
if out.len() >= max {
return Err(Error::Storage(
"stored retirements exceed the configured limit".into(),
));
}
let bytes = row
.map_err(|err| self.corrupt(err))?
.ok_or_else(|| self.corrupt("oversized retirement"))?;
let marker: SignedRetirement =
postcard::from_bytes(&bytes).map_err(|err| self.corrupt(err))?;
marker.verify(network).map_err(|err| self.corrupt(err))?;
out.push(marker);
}
Ok(out)
}
/// Persists a strongest verified marker received from an authenticated session.
pub fn put_retirement(&self, marker: &SignedRetirement) -> Result<()> {
let network = NetworkId::from_bytes(marker.network);
marker
.verify(network)
.map_err(|err| Error::Storage(err.to_string()))?;
let old: Option<Option<Vec<u8>>> = self.conn.query_row(
"SELECT CASE WHEN length(data) <= ?3 THEN data ELSE NULL END FROM retirements WHERE network_id = ?1 AND target = ?2",
params![marker.network.as_slice(), marker.target.author.as_slice(), MAX_RETIREMENT_LEN as i64], |row| row.get(0))
.optional().map_err(|err| self.corrupt(err))?;
if let Some(bytes) = old {
let bytes = bytes.ok_or_else(|| self.corrupt("oversized retirement"))?;
let old: SignedRetirement =
postcard::from_bytes(&bytes).map_err(|err| self.corrupt(err))?;
old.verify(network).map_err(|err| self.corrupt(err))?;
if !marker.supersedes(&old) {
return Ok(());
}
}
let bytes = postcard::to_stdvec(marker).map_err(|err| Error::Storage(err.to_string()))?;
if bytes.len() > MAX_RETIREMENT_LEN {
return Err(Error::Storage(
"retirement exceeds its encoded bound".into(),
));
}
self.conn
.execute(
"INSERT INTO retirements (network_id, target, data) VALUES (?1, ?2, ?3)
ON CONFLICT(network_id, target) DO UPDATE SET data = excluded.data",
params![
marker.network.as_slice(),
marker.target.author.as_slice(),
bytes
],
)
.map_err(|err| Error::Storage(format!("cannot store retirement: {err}")))?;
Ok(())
}
/// Commits our signed cleanup marker and author counter in one transaction.
pub fn publish_own_retirement(&self, marker: &SignedRetirement) -> Result<()> {
let transaction = self
.conn
.unchecked_transaction()
.map_err(|err| Error::Storage(err.to_string()))?;
self.put_retirement(marker)?;
transaction
.execute(
"INSERT INTO own_record_version (network_id, author, version) VALUES (?1, ?2, ?3)
ON CONFLICT(network_id, author) DO UPDATE SET version = max(version, excluded.version)",
params![
marker.network.as_slice(),
marker.author.as_slice(),
marker.version as i64
],
)
.map_err(|err| Error::Storage(format!("cannot store retirement counter: {err}")))?;
transaction
.commit()
.map_err(|err| Error::Storage(format!("cannot commit retirement: {err}")))
}
/// Drops a forgotten network's local cleanup proofs.
pub fn forget_retirements(&self, network: NetworkId) -> Result<()> {
self.conn
.execute(
"DELETE FROM retirements WHERE network_id = ?1",
params![network.as_bytes().as_slice()],
)
.map_err(|err| Error::Storage(err.to_string()))?;
Ok(())
}
/// Forgets every record of a network.
pub fn forget_signed_records(&self, network_id: NetworkId) -> Result<()> {
self.conn
@@ -1006,6 +1126,96 @@ mod broadcast_storage_tests {
assert!(store.peer_hostnames(id).unwrap().is_empty());
}
#[test]
fn retirement_migration_and_atomic_counter_commit_preserve_existing_state() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("state.sqlite");
let store = StateStore::open(&path).unwrap();
let identity = store.load_or_create_device_identity().unwrap();
let name = NetworkName::new("retirement-migration").unwrap();
let secret = NetworkSecret::from_bytes([6; 32]).unwrap();
let id = NetworkKeys::derive(&name, &secret).network_id();
store.upsert_network(id, &name, &secret, true).unwrap();
store.set_protocols(id, &["awg".into()]).unwrap();
let own = SignedRecord::sign(&identity.signing_key(), id, 8, RecordBody::Release);
store.publish_own_record(&own).unwrap();
store
.conn
.execute_batch("DROP TABLE retirements; PRAGMA user_version = 8;")
.unwrap();
drop(store);
let store = StateStore::open(&path).unwrap();
assert_eq!(
store.device_identity().unwrap().unwrap().endpoint_id(),
identity.endpoint_id()
);
assert_eq!(
store.list_networks().unwrap()[0].protocols,
Some(vec!["awg".into()])
);
assert_eq!(store.signed_records(id).unwrap(), vec![own]);
let target_key = iroh::SecretKey::generate();
let target = SignedRecord::sign(
&target_key,
id,
3,
RecordBody::Claim {
address: Some("10.13.37.2".parse().unwrap()),
range: Some(crate::state::DEFAULT_IPV4_RANGE),
hostname: Some("old-device".into()),
hostname_since: Some(123),
},
);
let marker = SignedRetirement::sign(&identity.signing_key(), id, 9, target);
store
.conn
.execute_batch(
"CREATE TRIGGER fail_retirement_counter BEFORE INSERT ON own_record_version
WHEN NEW.version = 9 BEGIN SELECT RAISE(ABORT, 'counter write refused'); END;",
)
.unwrap();
assert!(store.publish_own_retirement(&marker).is_err());
assert!(
store.retirements(id, 16).unwrap().is_empty(),
"marker must roll back with counter failure"
);
assert_eq!(
store
.own_record_version(id, identity.endpoint_id())
.unwrap(),
8
);
store
.conn
.execute_batch("DROP TRIGGER fail_retirement_counter;")
.unwrap();
store.publish_own_retirement(&marker).unwrap();
drop(store);
let store = StateStore::open(&path).unwrap();
assert_eq!(store.retirements(id, 16).unwrap(), vec![marker.clone()]);
assert_eq!(
store
.own_record_version(id, identity.endpoint_id())
.unwrap(),
9
);
assert!(
store.retirements(id, 0).is_err(),
"capacity changes must not silently drop markers"
);
store
.conn
.execute(
"UPDATE retirements SET data = ?1",
params![vec![0u8; MAX_RETIREMENT_LEN + 1]],
)
.unwrap();
assert!(matches!(
store.retirements(id, 16),
Err(Error::StateCorrupted { .. })
));
}
#[test]
fn v7_migration_keeps_signed_state_and_protocol_overrides_are_local_and_durable() {
let dir = tempfile::tempdir().unwrap();
+358
View File
@@ -0,0 +1,358 @@
//! Equal-rights retirement uses real membership, signatures and persistent state.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use tsunagi::discovery::SharedMemoryDiscovery;
use tsunagi::testing::{TestAgent, config_with, network, wait_for_peers, wait_until};
use tsunagi::{Agent, NetworkId};
#[tokio::test]
async fn bounded_marker_batches_eventually_reach_the_tail_of_a_small_send_queue() {
use tsunagi::config::StoragePaths;
use tsunagi::state::{RecordBody, SignedRecord, SignedRetirement};
use tsunagi::storage::StateStore;
let discovery = SharedMemoryDiscovery::new();
let cleaner = TestAgent::spawn(&discovery).await.unwrap();
let replica = TestAgent::spawn(&discovery).await.unwrap();
let (name, secret) = network("forget-batched-history");
let id = cleaner.agent.join_network(&name, &secret).await.unwrap();
replica.agent.join_network(&name, &secret).await.unwrap();
let cleaner_dir = cleaner.stop().await;
let replica_dir = replica.stop().await;
let cleaner_path = StoragePaths::under(cleaner_dir.path()).state_db();
let replica_path = StoragePaths::under(replica_dir.path()).state_db();
// Seed genuine signed offline history in closed stores. Delivery still uses
// real agents, authenticated loopback sessions and the persistent stores.
let targets = tokio::task::spawn_blocking(move || {
let cleaner = StateStore::open(&cleaner_path).unwrap();
let replica = StateStore::open(&replica_path).unwrap();
let author = cleaner.device_identity().unwrap().unwrap();
let signing_key = iroh::SecretKey::from_bytes(&author.secret_bytes());
let mut targets = Vec::new();
for index in 0..12 {
let target = iroh::SecretKey::generate();
let record = SignedRecord::sign(
&target,
id,
1,
RecordBody::Claim {
address: None,
range: None,
hostname: Some(format!("offline-{index}")),
hostname_since: Some(1),
},
);
cleaner.put_signed_record(&record).unwrap();
replica.put_signed_record(&record).unwrap();
cleaner
.publish_own_retirement(&SignedRetirement::sign(
&signing_key,
id,
100 + index,
record,
))
.unwrap();
targets.push(target.public());
}
targets
})
.await
.unwrap();
let replica = Agent::spawn(config_with(replica_dir.path(), &discovery))
.await
.unwrap();
assert_eq!(
replica
.network_status(id)
.await
.unwrap()
.members
.iter()
.filter(|m| targets.contains(&m.endpoint_id))
.count(),
12
);
let mut config = config_with(cleaner_dir.path(), &discovery);
config.limits.max_state_records = 1;
config.limits.session_send_queue = 2;
let cleaner = Agent::spawn(config).await.unwrap();
wait_for_peers(&cleaner, id, 1).await;
wait_until(
"all cleanup batches arrive despite send queue pressure",
|| async {
replica
.network_status(id)
.await
.ok()?
.members
.iter()
.all(|m| !targets.contains(&m.endpoint_id))
.then_some(())
},
)
.await;
cleaner.shutdown().await;
replica.shutdown().await;
let replica_path = StoragePaths::under(replica_dir.path()).state_db();
tokio::task::spawn_blocking(move || {
let replica = StateStore::open(&replica_path).unwrap();
assert_eq!(replica.retirements(id, 16).unwrap().len(), 12);
})
.await
.unwrap();
}
async fn member(
agent: &Agent,
network: NetworkId,
peer: iroh::EndpointId,
) -> tsunagi::agent::MemberStatus {
wait_until("signed member", || async {
agent
.network_status(network)
.await
.ok()?
.members
.into_iter()
.find(|m| m.endpoint_id == peer)
})
.await
}
#[tokio::test]
async fn retirement_survives_restart_and_stale_replicas_but_allows_a_new_claim() {
let discovery = SharedMemoryDiscovery::new();
let cleaner = TestAgent::spawn_with(|c| c.with_hostname("cleaner"), &discovery)
.await
.unwrap();
let target = TestAgent::spawn_with(|c| c.with_hostname("old-device"), &discovery)
.await
.unwrap();
let replica = TestAgent::spawn_with(|c| c.with_hostname("replica"), &discovery)
.await
.unwrap();
let (name, secret) = network("forget-propagation");
let id = cleaner.agent.join_network(&name, &secret).await.unwrap();
target.agent.join_network(&name, &secret).await.unwrap();
replica.agent.join_network(&name, &secret).await.unwrap();
wait_for_peers(&cleaner.agent, id, 2).await;
let peer = target.agent.endpoint_id();
let original = wait_until("all three agents agree on allocated claims", || async {
let a = cleaner.agent.network_status(id).await.ok()?;
let b = target.agent.network_status(id).await.ok()?;
let c = replica.agent.network_status(id).await.ok()?;
for status in [&a, &b, &c] {
if status.members.len() != 3 {
return None;
}
let addresses: std::collections::HashSet<_> = status
.members
.iter()
.filter_map(|m| m.overlay_address_v4)
.collect();
if addresses.len() != 3 {
return None;
}
}
let record = a.members.iter().find(|m| m.endpoint_id == peer)?;
(b.members
.iter()
.find(|m| m.endpoint_id == peer)?
.record_version
== record.record_version
&& c.members
.iter()
.find(|m| m.endpoint_id == peer)?
.record_version
== record.record_version)
.then(|| record.clone())
})
.await;
assert!(
cleaner
.agent
.forget_node(id, peer, original.record_version)
.await
.is_err(),
"online nodes are not history"
);
assert!(
cleaner
.agent
.forget_node(id, cleaner.agent.endpoint_id(), 1)
.await
.is_err()
);
let replica_dir = replica.stop().await;
let target_dir = target.stop().await;
wait_until("target offline", || async {
cleaner
.agent
.network_status(id)
.await
.ok()?
.members
.iter()
.find(|m| m.endpoint_id == peer)
.filter(|m| !m.online)
.map(|_| ())
})
.await;
assert!(
cleaner
.agent
.forget_node(id, peer, original.record_version + 1)
.await
.is_err(),
"stale confirmation changes nothing"
);
let result = cleaner
.agent
.forget_node(id, peer, original.record_version)
.await
.unwrap();
assert_eq!(result.target, peer);
assert_eq!(result.record_version, original.record_version);
assert_eq!(result.peers_told, 0);
assert!(
!cleaner
.agent
.network_status(id)
.await
.unwrap()
.members
.iter()
.any(|m| m.endpoint_id == peer)
);
let cleaner_dir = cleaner.stop().await;
let cleaner =
Agent::spawn(config_with(cleaner_dir.path(), &discovery).with_hostname("cleaner"))
.await
.unwrap();
let replica =
Agent::spawn(config_with(replica_dir.path(), &discovery).with_hostname("replica"))
.await
.unwrap();
wait_for_peers(&cleaner, id, 1).await;
wait_until("stale replica learns durable retirement", || async {
replica
.network_status(id)
.await
.ok()?
.members
.iter()
.all(|m| m.endpoint_id != peer)
.then_some(())
})
.await;
assert!(
!cleaner
.network_status(id)
.await
.unwrap()
.members
.iter()
.any(|m| m.endpoint_id == peer)
);
replica.set_hostname("old-device").await.unwrap();
let replica_id = replica.endpoint_id();
wait_until("released hostname is reclaimed", || async {
cleaner
.network_status(id)
.await
.ok()?
.members
.iter()
.find(|m| m.endpoint_id == replica_id)
.filter(|m| m.hostname.as_deref() == Some("old-device"))?
.hostname_conflict
.is_none()
.then_some(())
})
.await;
let returned =
Agent::spawn(config_with(target_dir.path(), &discovery).with_hostname("old-device"))
.await
.unwrap();
wait_for_peers(&returned, id, 2).await;
wait_until("returning identity publishes a fresh version", || async {
let m = cleaner
.network_status(id)
.await
.ok()?
.members
.into_iter()
.find(|m| m.endpoint_id == peer)?;
(m.online && m.record_version > original.record_version && m.hostname_conflict.is_some())
.then_some(())
})
.await;
cleaner.shutdown().await;
replica.shutdown().await;
returned.shutdown().await;
}
#[tokio::test]
async fn forgetting_is_network_scoped_and_every_member_has_the_same_right() {
let discovery = SharedMemoryDiscovery::new();
let a = TestAgent::spawn(&discovery).await.unwrap();
let b = TestAgent::spawn(&discovery).await.unwrap();
let target = TestAgent::spawn(&discovery).await.unwrap();
let (name, secret) = network("forget-one-network");
let (other_name, other_secret) = network("keep-other-network");
let id = a.agent.join_network(&name, &secret).await.unwrap();
b.agent.join_network(&name, &secret).await.unwrap();
target.agent.join_network(&name, &secret).await.unwrap();
let other = a
.agent
.join_network(&other_name, &other_secret)
.await
.unwrap();
target
.agent
.join_network(&other_name, &other_secret)
.await
.unwrap();
let peer = target.agent.endpoint_id();
let record = member(&b.agent, id, peer).await;
member(&a.agent, other, peer).await;
target.agent.shutdown().await;
wait_until("offline on B", || async {
b.agent
.network_status(id)
.await
.ok()?
.members
.iter()
.find(|m| m.endpoint_id == peer)
.filter(|m| !m.online)
.map(|_| ())
})
.await;
b.agent
.forget_node(id, peer, record.record_version)
.await
.unwrap();
wait_until("A learns B's retirement", || async {
a.agent
.network_status(id)
.await
.ok()?
.members
.iter()
.all(|m| m.endpoint_id != peer)
.then_some(())
})
.await;
assert!(
a.agent
.network_status(other)
.await
.unwrap()
.members
.iter()
.any(|m| m.endpoint_id == peer)
);
a.agent.shutdown().await;
b.agent.shutdown().await;
}
+19
View File
@@ -21,6 +21,12 @@ const POLL_INTERVAL: Duration = Duration::from_secs(2);
/// A request from the UI to the agent.
pub(crate) enum Command {
/// Retire the exact offline-node claim selected in the devices window.
ForgetNode {
network_id: String,
peer: String,
expected_version: u64,
},
/// Replace the network's enabled protocols in preference order.
SetProtocols {
network_id: String,
@@ -192,6 +198,19 @@ async fn run(
ctx.request_repaint();
let outcome: Result<String, String> = match command {
Command::ForgetNode {
network_id,
peer,
expected_version,
} => ipc::forget_node(socket, &network_id, &peer, expected_version)
.await
.map(|report| {
format!(
"forgot node claim v{}; cleanup queued to {} peers",
report.record_version, report.peers_told
)
})
.map_err(|err| err.to_string()),
Command::SetProtocols {
network_id,
protocols,
+28 -6
View File
@@ -15,6 +15,12 @@ use crate::devices;
use crate::stats::Traffic;
use crate::ui::{self, UiState};
fn app_fonts() -> egui::FontDefinitions {
let mut fonts = egui::FontDefinitions::default();
egui_phosphor::add_to_fonts(&mut fonts, egui_phosphor::Variant::Regular);
fonts
}
/// The window's state.
pub(crate) struct App {
agent: AgentClient,
@@ -34,9 +40,7 @@ impl App {
let ctx = cc.egui_ctx.clone();
// Add the Phosphor icon glyphs to the font set so the icon buttons
// render instead of showing as tofu.
let mut fonts = egui::FontDefinitions::default();
egui_phosphor::add_to_fonts(&mut fonts, egui_phosphor::Variant::Regular);
ctx.set_fonts(fonts);
ctx.set_fonts(app_fonts());
let agent = AgentClient::spawn(&runtime, ctx);
Ok(Self {
@@ -116,20 +120,21 @@ impl App {
.with_title(devices::title(network))
.with_app_id(crate::icon::APP_ID)
.with_icon(crate::icon::window())
.with_inner_size([940.0, 540.0])
.with_min_inner_size([480.0, 320.0]);
.with_inner_size([1440.0, 640.0])
.with_min_inner_size([800.0, 400.0]);
let agent = &self.agent;
let traffic = &self.traffic;
// The window shares the single unit setting, so its toggle and the
// main window's stay in step.
let unit = &mut self.state.unit;
let confirm = &mut self.state.exit_confirm;
let forget_confirm = &mut self.state.forget_confirm;
let keep = ctx.show_viewport_immediate(viewport_id, builder, |vctx, _class| {
devices::show(
vctx,
agent,
unit,
confirm,
(confirm, forget_confirm),
network,
&report.endpoint_id,
traffic,
@@ -142,3 +147,20 @@ impl App {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn route_icons_have_an_embedded_font_in_the_app() {
let fonts = app_fonts();
assert!(fonts.font_data.contains_key("phosphor"));
assert!(
fonts
.families
.get(&egui::FontFamily::Proportional)
.is_some_and(|family| family.iter().any(|name| name == "phosphor"))
);
}
}
+406 -108
View File
@@ -8,6 +8,7 @@
use std::collections::HashMap;
use eframe::egui;
use egui_phosphor::regular as icon;
use tsunagi::ipc::{DataPathReport, NetworkReport, OverlayPeerReport};
@@ -27,6 +28,8 @@ pub(crate) fn title(network: &NetworkReport) -> String {
/// One row of the device table, built from a tunnel and/or a signed member.
struct Device<'a> {
/// Observed claim version, if this row came from signed membership.
record_version: Option<u64>,
/// The agent's own answer to whether it can be reached by any path.
online: bool,
/// How packets reach it.
@@ -46,6 +49,7 @@ struct Device<'a> {
tunnel_protocol: &'a str,
address: Option<String>,
handshake_secs: Option<u64>,
control_rtt_ms: Option<u64>,
overlay: Option<&'a OverlayPeerReport>,
/// It is connected and offers to be an exit node.
exit_node: bool,
@@ -65,6 +69,15 @@ pub(crate) struct ExitConfirm {
main: bool,
}
/// An exact offline claim selected for durable cleanup.
pub(crate) struct ForgetConfirm {
network_id: String,
peer: String,
version: u64,
name: String,
address: Option<String>,
}
impl ExitConfirm {
/// A question about stopping the exit node a network's tile shows.
pub(crate) fn stop_from_tile(network: &NetworkReport, peer_id: &str) -> Self {
@@ -109,8 +122,7 @@ fn name_cell(ui: &mut egui::Ui, device: &Device<'_>) {
if let Some((name, holder)) = &device.name_taken {
ui.label(egui::RichText::new(egui_phosphor::regular::WARNING).color(AMBER))
.on_hover_text(format!(
"`{name}` was claimed first by {holder}. This device works by address, \
but nobody finds it by that name."
"Name `{name}` belongs to {holder}. Use this device's address."
));
}
});
@@ -121,11 +133,12 @@ pub(crate) fn show(
ctx: &egui::Context,
agent: &AgentClient,
unit: &mut Unit,
confirm: &mut Option<ExitConfirm>,
confirmations: (&mut Option<ExitConfirm>, &mut Option<ForgetConfirm>),
network: &NetworkReport,
own_id: &str,
traffic: &Traffic,
) {
let (confirm, forget_confirm) = confirmations;
egui::CentralPanel::default().show(ctx, |ui| {
ui.horizontal(|ui| {
ui.heading(&network.name);
@@ -158,10 +171,68 @@ pub(crate) fn show(
egui::ScrollArea::both()
.auto_shrink([false, false])
.show(ui, |ui| {
device_table(ui, unit, network, &devices, traffic, confirm);
device_table(
ui,
unit,
network,
&devices,
traffic,
confirm,
forget_confirm,
);
});
});
confirm_exit(ctx, agent, confirm, Some(&network.network_id));
confirm_forget(ctx, agent, forget_confirm, &network.network_id);
}
fn confirm_forget(
ctx: &egui::Context,
agent: &AgentClient,
confirm: &mut Option<ForgetConfirm>,
network: &str,
) {
let Some(pending) = confirm.as_ref().filter(|c| c.network_id == network) else {
return;
};
egui::Area::new(egui::Id::new("forget node backdrop"))
.order(egui::Order::Middle)
.fixed_pos(egui::Pos2::ZERO)
.show(ctx, |ui| {
let screen = ctx.screen_rect();
ui.allocate_response(screen.size(), egui::Sense::click_and_drag());
ui.painter()
.rect_filled(screen, 0.0, egui::Color32::from_black_alpha(140));
});
let (mut accepted, mut cancelled) = (false, false);
egui::Window::new("Forget offline node")
.order(egui::Order::Foreground)
.collapsible(false)
.resizable(false)
.anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0])
.show(ctx, |ui| {
ui.set_max_width(440.0);
ui.label(format!("Forget {}?", pending.name));
ui.monospace(&pending.peer);
if let Some(address) = &pending.address {
ui.label(format!("IP: {address}"));
}
ui.label("The network will forget this claim and free its name and IP. A returning device can publish a new claim; this does not ban it.");
ui.horizontal(|ui| {
accepted = ui.add_enabled(!agent.snapshot().busy, egui::Button::new("Forget")).clicked();
cancelled = ui.button("Cancel").clicked();
});
});
if accepted {
agent.send(Command::ForgetNode {
network_id: pending.network_id.clone(),
peer: pending.peer.clone(),
expected_version: pending.version,
});
}
if accepted || cancelled || ctx.input(|i| i.key_pressed(egui::Key::Escape)) {
*confirm = None;
}
}
/// The confirmation for an exit-node choice, drawn in the window it was
@@ -354,6 +425,7 @@ fn devices<'a>(network: &'a NetworkReport, own_id: &str) -> Vec<Device<'a>> {
});
let hostname = hostnames.get(id).copied();
Device {
record_version: member.map(|m| m.record_version),
online,
path,
via_name,
@@ -366,6 +438,11 @@ fn devices<'a>(network: &'a NetworkReport, own_id: &str) -> Vec<Device<'a>> {
.and_then(|p| p.address.clone())
.or_else(|| member.and_then(|m| m.overlay_address_v4.clone())),
handshake_secs: overlay.and_then(|p| p.handshake_secs_ago),
control_rtt_ms: network
.peers
.iter()
.find(|p| p.endpoint_id == id)
.and_then(|p| p.rtt_ms),
overlay,
exit_node: online
&& network
@@ -387,16 +464,17 @@ fn devices<'a>(network: &'a NetworkReport, own_id: &str) -> Vec<Device<'a>> {
/// The widest value each column is expected to hold, so the table starts out
/// wide enough and does not reflow as numbers grow.
const COLUMN_SAMPLES: [&str; 9] = [
"",
const COLUMN_SAMPLES: [&str; 10] = [
"device",
"exit",
"wg-quic · waiting for handshake",
"end-to-end",
"999d / 9999 ms",
"255.255.255.255",
"direct over tcp-tls",
"data path",
"99999/99999",
"1023.9 MB / 1023.9 MB",
"1023.9 kB/s",
"",
"plot",
];
/// Column widths that only ever grow, kept across frames.
@@ -405,11 +483,11 @@ const COLUMN_SAMPLES: [&str; 9] = [
/// gets shorter — `14.4 MB/s` giving way to `382 pkt/s` — would pull the
/// column in and push every cell after it. Remembering the widest seen keeps
/// the table where it is.
fn column_widths(ui: &egui::Ui, id: egui::Id) -> [f32; 9] {
let stored: Option<[f32; 9]> = ui.memory(|m| m.data.get_temp(id));
fn column_widths(ui: &egui::Ui, id: egui::Id) -> [f32; 10] {
let stored: Option<[f32; 10]> = ui.memory(|m| m.data.get_temp(id));
stored.unwrap_or_else(|| {
let font = egui::TextStyle::Body.resolve(ui.style());
let mut widths = [0.0; 9];
let mut widths = [0.0; 10];
for (width, sample) in widths.iter_mut().zip(COLUMN_SAMPLES) {
if !sample.is_empty() {
*width = ui.fonts(|fonts| {
@@ -425,13 +503,24 @@ fn column_widths(ui: &egui::Ui, id: egui::Id) -> [f32; 9] {
}
/// One cell, at least as wide as its column has ever been.
fn cell(ui: &mut egui::Ui, column: usize, widths: &mut [f32; 9], add: impl FnOnce(&mut egui::Ui)) {
fn cell(ui: &mut egui::Ui, column: usize, widths: &[f32; 10], add: impl FnOnce(&mut egui::Ui)) {
let floor = widths[column];
let response = ui.scope(|ui| {
ui.set_min_width(floor);
add(ui);
});
widths[column] = widths[column].max(response.response.rect.width());
ui.allocate_ui_with_layout(
egui::vec2(floor, 0.0),
egui::Layout::top_down(egui::Align::Min),
|ui| {
ui.set_min_width(floor);
ui.set_max_width(floor);
add(ui);
},
);
}
fn stretch_columns(mut widths: [f32; 10], available: f32) -> [f32; 10] {
let gaps = 10.0 * (widths.len() - 1) as f32;
widths[9] = widths[9].max(100.0);
widths[9] += (available - gaps - widths.iter().sum::<f32>()).max(0.0);
widths
}
/// The exit-node mark of a device: shown when it offers to be one, bright
@@ -457,10 +546,7 @@ fn exit_cell(
};
(color, "in use as your exit node — click to stop")
} else {
(
ui.visuals().text_color(),
"offers to be an exit node — click to send all your internet traffic through it",
)
(ui.visuals().text_color(), "Use this exit node")
};
let response = ui
.add(
@@ -493,6 +579,7 @@ fn device_table(
devices: &[Device<'_>],
traffic: &Traffic,
confirm: &mut Option<ExitConfirm>,
forget_confirm: &mut Option<ForgetConfirm>,
) {
let network_id = network.network_id.as_str();
if devices.is_empty() {
@@ -501,8 +588,35 @@ fn device_table(
}
let widths_id = egui::Id::new(("devices column widths", network_id));
let mut widths = column_widths(ui, widths_id);
let font = egui::TextStyle::Body.resolve(ui.style());
for device in devices {
let name = format!(
"{} ({}){}",
device.name(),
short_id(device.id),
if !device.online { " Forget" } else { "" }
);
let natural = ui.fonts(|fonts| {
fonts
.layout_no_wrap(name, font.clone(), egui::Color32::WHITE)
.size()
.x
});
widths[0] = widths[0].max(natural + 24.0);
let (path, _) = data_path_text(device.path, device.via_name.as_deref(), device.online);
let path_width = ui.fonts(|fonts| {
fonts
.layout_no_wrap(path, font.clone(), egui::Color32::WHITE)
.size()
.x
});
widths[5] = widths[5].max(path_width);
}
ui.memory_mut(|m| m.data.insert_temp(widths_id, widths));
let widths = stretch_columns(widths, ui.available_width());
egui::Grid::new("devices")
.num_columns(9)
.num_columns(10)
.min_col_width(0.0)
.striped(true)
.spacing([10.0, 4.0])
.show(ui, |ui| {
@@ -510,6 +624,7 @@ fn device_table(
"device",
"exit",
"end-to-end",
"handshake / RTT",
"address",
"data path",
"pkts",
@@ -520,20 +635,17 @@ fn device_table(
.into_iter()
.enumerate()
{
cell(ui, column, &mut widths, |ui| {
cell(ui, column, &widths, |ui| {
let response = ui.weak(header);
match column {
2 => {
response.on_hover_text(
"Protocol that encrypts traffic between this device and the \
destination. It is independent of the route carrying it.",
);
response.on_hover_text("End-to-end encryption.");
}
4 => {
response.on_hover_text(
"Current route carrying the encrypted tunnel traffic. `direct \
over tcp-tls` means one Tsunagi hop with no mesh relay.",
);
3 => {
response.on_hover_text("Handshake age / control-connection RTT.");
}
5 => {
response.on_hover_text("Transport carrying the encrypted tunnel.");
}
_ => {}
}
@@ -542,51 +654,89 @@ fn device_table(
ui.end_row();
for device in devices {
cell(ui, 0, &mut widths, |ui| name_cell(ui, device));
cell(ui, 1, &mut widths, |ui| {
cell(ui, 0, &widths, |ui| {
ui.horizontal(|ui| {
name_cell(ui, device);
if !device.online
&& !device.in_use
&& let Some(version) = device.record_version
&& ui
.add_enabled(network.active, egui::Button::new("Forget"))
.on_hover_text("Free this offline device's name and IPv4")
.clicked()
{
*forget_confirm = Some(ForgetConfirm {
network_id: network.network_id.clone(),
peer: device.id.into(),
version,
name: device.name(),
address: device.address.clone(),
});
}
});
});
cell(ui, 1, &widths, |ui| {
exit_cell(ui, device, network_id, confirm);
});
cell(ui, 2, &mut widths, |ui| {
tunnel_cell(ui, device);
cell(ui, 2, &widths, |ui| {
ui.push_id((device.id, "end-to-end"), |ui| {
tunnel_cell(ui, device);
});
});
cell(ui, 3, &mut widths, |ui| match &device.address {
cell(ui, 3, &widths, |ui| {
ui.label(handshake_rtt_text(
device.handshake_secs,
device.control_rtt_ms,
displayed_protocol(device.tunnel_protocol) != "—",
))
.on_hover_text(if device.tunnel_protocol == "tcp-tls" {
"Session age / control RTT."
} else {
"Last handshake age / control RTT."
});
});
cell(ui, 4, &widths, |ui| match &device.address {
Some(address) => format::copy_field(ui, address, address),
None => {
ui.weak("—");
}
});
cell(ui, 4, &mut widths, |ui| data_path_cell(ui, device));
cell(ui, 5, &widths, |ui| {
ui.push_id((device.id, "data-path"), |ui| data_path_cell(ui, device));
});
match device.overlay.filter(|_| device.online) {
Some(peer) => {
let series = traffic.peer(network_id, &peer.public_key);
cell(ui, 5, &mut widths, |ui| {
cell(ui, 6, &widths, |ui| {
ui.label(format!("{}/{}", peer.tx_packets, peer.rx_packets));
});
cell(ui, 6, &mut widths, |ui| {
cell(ui, 7, &widths, |ui| {
ui.label(format!(
"{} / {}",
format::bytes(peer.tx_bytes as f64),
format::bytes(peer.rx_bytes as f64)
));
});
cell(ui, 7, &mut widths, |ui| {
cell(ui, 8, &widths, |ui| {
ui.add(
egui::Label::new(format::series_rate(series, unit))
.wrap_mode(egui::TextWrapMode::Extend),
);
});
ui.allocate_ui(egui::vec2(100.0, 20.0), |ui| {
ui.allocate_ui(egui::vec2(widths[9], 20.0), |ui| {
format::sparkline(ui, series, unit, 18.0);
});
}
None => {
for column in 5..8 {
cell(ui, column, &mut widths, |ui| {
for column in 6..9 {
cell(ui, column, &widths, |ui| {
ui.weak("—");
});
}
ui.weak("—");
cell(ui, 9, &widths, |ui| {
ui.weak("—");
});
}
}
ui.end_row();
@@ -601,25 +751,25 @@ fn device_table(
tx_bytes += peer.tx_bytes;
rx_bytes += peer.rx_bytes;
}
cell(ui, 0, &mut widths, |ui| {
cell(ui, 0, &widths, |ui| {
ui.strong("total");
});
for column in 1..5 {
cell(ui, column, &mut widths, |ui| {
for column in 1..6 {
cell(ui, column, &widths, |ui| {
ui.label("");
});
}
cell(ui, 5, &mut widths, |ui| {
cell(ui, 6, &widths, |ui| {
ui.strong(format!("{tx_packets}/{rx_packets}"));
});
cell(ui, 6, &mut widths, |ui| {
cell(ui, 7, &widths, |ui| {
ui.strong(format!(
"{} / {}",
format::bytes(tx_bytes as f64),
format::bytes(rx_bytes as f64)
));
});
cell(ui, 7, &mut widths, |ui| {
cell(ui, 8, &widths, |ui| {
ui.add(
egui::Label::new(
egui::RichText::new(format::series_rate(traffic.network(network_id), unit))
@@ -630,57 +780,49 @@ fn device_table(
});
ui.end_row();
});
ui.memory_mut(|m| m.data.insert_temp(widths_id, widths));
}
/// A protocol name as it should appear in the table.
fn displayed_protocol(protocol: &str) -> &str {
if protocol.is_empty() {
"wg-quic"
} else {
protocol
}
if protocol.is_empty() { "—" } else { protocol }
}
/// The end-to-end tunnel and its own health, without transport details.
fn tunnel_text(protocol: &str, handshake_secs: Option<u64>) -> String {
let protocol = displayed_protocol(protocol);
if protocol == "—" {
return protocol.to_string();
}
match handshake_secs {
Some(secs) if protocol == "tcp-tls" => {
format!("{protocol} · up {}", format::age(secs))
}
Some(secs) => format!("{protocol} · handshake {} ago", format::age(secs)),
None => format!("{protocol} · waiting for handshake"),
fn handshake_rtt_text(handshake: Option<u64>, rtt: Option<u64>, has_tunnel: bool) -> String {
let age = handshake.map_or_else(
|| {
if has_tunnel {
"waiting".into()
} else {
"—".into()
}
},
format::age,
);
let rtt = rtt.map_or_else(|| "—".into(), |value| format!("{value} ms"));
format!("{age} / {rtt}")
}
fn tunnel_hint(protocol: &str) -> String {
let name = displayed_protocol(protocol);
if name == "tcp-tls" {
format!("{name}: Noise end-to-end encryption")
} else {
format!("{name}: WireGuard end-to-end encryption")
}
}
fn tunnel_cell(ui: &mut egui::Ui, device: &Device<'_>) {
let protocol = displayed_protocol(device.tunnel_protocol);
let text = tunnel_text(protocol, device.handshake_secs);
if protocol == "—" {
ui.weak(text);
ui.weak(protocol);
return;
}
let hint = match device.path {
Some(DataPathReport::Direct { transport }) => format!(
"The end-to-end {protocol} tunnel is currently carried over a direct {transport} \
transport."
),
Some(DataPathReport::Relayed { hops, .. }) => format!(
"The end-to-end {protocol} tunnel is currently carried through other devices over \
{hops} transport hops."
),
Some(DataPathReport::None) | None => {
format!("The end-to-end {protocol} tunnel does not have a data path carrying it yet.")
}
};
ui.label(text).on_hover_text(hint);
ui.label(protocol).on_hover_text(tunnel_hint(protocol));
}
/// Current route text and whether that route is the preferred direct form.
/// Compact route label and whether that route is the preferred direct form.
/// The icon font is embedded by egui-phosphor and installed in `App::new`.
fn data_path_text(
path: Option<&DataPathReport>,
via_name: Option<&str>,
@@ -690,9 +832,13 @@ fn data_path_text(
return ("offline".into(), false);
}
match path {
Some(DataPathReport::Direct { transport }) => (format!("direct over {transport}"), true),
Some(DataPathReport::Direct { transport }) => (format!("{} {transport}", icon::LINK), true),
Some(DataPathReport::Relayed { hops, .. }) => (
format!("via {} · {hops} hops", via_name.unwrap_or("another device")),
format!(
"{} {} · {hops} hops",
icon::SHARE_NETWORK,
via_name.unwrap_or("another device")
),
false,
),
Some(DataPathReport::None) | None => ("no data path yet".into(), false),
@@ -706,21 +852,18 @@ fn data_path_text(
fn data_path_cell(ui: &mut egui::Ui, device: &Device<'_>) {
let (text, good) = data_path_text(device.path, device.via_name.as_deref(), device.online);
let response = if good {
ui.label(text)
ui.add(egui::Label::new(text).wrap_mode(egui::TextWrapMode::Extend))
} else {
ui.label(egui::RichText::new(text).color(AMBER))
ui.add(
egui::Label::new(egui::RichText::new(text).color(AMBER))
.wrap_mode(egui::TextWrapMode::Extend),
)
};
response.on_hover_text(match device.path {
Some(DataPathReport::Direct { .. }) => {
"One Tsunagi hop with no mesh relay. The end-to-end tunnel may use a different \
protocol from this transport."
}
Some(DataPathReport::Relayed { .. }) => {
"No direct transport yet: traffic goes through another device, and moves to a \
direct path by itself as soon as one comes up."
}
_ if device.online => "Connected, but no data path carries packets yet.",
_ => "This device is not reachable now.",
Some(DataPathReport::Direct { .. }) => "Direct transport; no mesh relay.",
Some(DataPathReport::Relayed { .. }) => "Traffic forwarded through another device.",
_ if device.online => "Control connected; data transport unavailable.",
_ => "Unreachable.",
});
}
@@ -736,6 +879,7 @@ mod tests {
fn member(id: &str, online: bool, path: DataPathReport) -> MemberReport {
MemberReport {
record_version: 1,
endpoint_id: id.into(),
online,
path,
@@ -803,7 +947,8 @@ mod tests {
#[test]
fn tunnel_and_direct_transport_are_named_as_different_layers() {
assert_eq!(tunnel_text("wg", Some(1)), "wg · handshake 1s ago");
assert_eq!(displayed_protocol("wg"), "wg");
assert_eq!(tunnel_hint("wg"), "wg: WireGuard end-to-end encryption");
assert_eq!(
data_path_text(
Some(&DataPathReport::Direct {
@@ -812,17 +957,17 @@ mod tests {
None,
true,
),
("direct over tcp-tls".into(), true)
(format!("{} tcp-tls", icon::LINK), true)
);
assert_eq!(
tunnel_hint("tcp-tls"),
"tcp-tls: Noise end-to-end encryption"
);
assert_eq!(tunnel_text("tcp-tls", Some(42)), "tcp-tls · up 42s");
}
#[test]
fn incomplete_and_longer_paths_stay_explicit() {
assert_eq!(
tunnel_text("wg-quic", None),
"wg-quic · waiting for handshake"
);
assert_eq!(handshake_rtt_text(None, Some(25), true), "waiting / 25 ms");
assert_eq!(
data_path_text(
Some(&DataPathReport::Relayed {
@@ -832,7 +977,7 @@ mod tests {
Some("iris"),
true,
),
("via iris · 2 hops".into(), false)
(format!("{} iris · 2 hops", icon::SHARE_NETWORK), false)
);
assert_eq!(
data_path_text(Some(&DataPathReport::None), None, true),
@@ -840,4 +985,157 @@ mod tests {
);
assert_eq!(data_path_text(None, None, false), ("offline".into(), false));
}
#[test]
fn handshake_rtt_has_no_invented_values_and_uses_control_measurements() {
use tsunagi::ipc::{OverlayReport, PeerReport};
let network = NetworkReport {
peers: vec![PeerReport {
endpoint_id: "peer".into(),
rtt_ms: Some(27),
..Default::default()
}],
overlay: Some(OverlayReport {
peers: vec![OverlayPeerReport {
endpoint_id: "peer".into(),
protocol: "wg".into(),
handshake_secs_ago: Some(5),
..Default::default()
}],
..Default::default()
}),
..Default::default()
};
let rows = devices(&network, "own");
assert_eq!(
handshake_rtt_text(rows[0].handshake_secs, rows[0].control_rtt_ms, true),
"5s / 27 ms"
);
assert_eq!(handshake_rtt_text(None, None, false), "— / —");
assert_eq!(rows[0].tunnel_protocol, "wg");
assert!(!tunnel_hint(rows[0].tunnel_protocol).contains("wg-quic"));
}
#[test]
fn resized_table_fills_the_window_without_growing_fixed_columns() {
let minimum = [100.0; 10];
let wide = stretch_columns(minimum, 1440.0);
assert!((wide.iter().sum::<f32>() + 90.0 - 1440.0).abs() < 0.1);
assert_eq!(&wide[..9], &minimum[..9]);
let narrow = stretch_columns(minimum, 800.0);
assert_eq!(
narrow, minimum,
"narrow windows scroll rather than clipping columns"
);
assert_eq!(
stretch_columns(minimum, 1440.0),
wide,
"a resize does not accumulate padding"
);
}
#[test]
fn a_long_relay_name_gets_its_full_column_width() {
let mut relay = member("near", true, DataPathReport::None);
relay.hostname = Some("a-relay-with-a-long-device-name".into());
let network = NetworkReport {
network_id: "network".into(),
members: vec![
relay,
member(
"far",
true,
DataPathReport::Relayed {
hops: 2,
via: "near".into(),
},
),
],
..Default::default()
};
let rows = devices(&network, "own");
let far = rows.iter().find(|device| device.id == "far").unwrap();
let (label, _) = data_path_text(far.path, far.via_name.as_deref(), far.online);
let context = egui::Context::default();
let mut fonts = egui::FontDefinitions::default();
egui_phosphor::add_to_fonts(&mut fonts, egui_phosphor::Variant::Regular);
context.set_fonts(fonts);
let _ = context.run(egui::RawInput::default(), |ctx| {
egui::CentralPanel::default().show(ctx, |ui| {
device_table(
ui,
Unit::Bytes,
&network,
&rows,
&Traffic::default(),
&mut None,
&mut None,
);
let stored: [f32; 10] = ui
.memory(|memory| {
memory
.data
.get_temp(egui::Id::new(("devices column widths", "network")))
})
.unwrap();
let font = egui::TextStyle::Body.resolve(ui.style());
let needed = ui.fonts(|fonts| {
fonts
.layout_no_wrap(label.clone(), font, egui::Color32::WHITE)
.size()
.x
});
assert!(stored[5] >= needed);
});
});
}
#[test]
fn rendered_table_uses_window_width_without_an_expanding_device_cell() {
let context = egui::Context::default();
let network = NetworkReport {
members: vec![MemberReport {
endpoint_id: "0123456789abcdef".into(),
hostname: Some("laptop".into()),
online: true,
..Default::default()
}],
..Default::default()
};
let rows = devices(&network, "own");
for width in [1440.0, 1200.0, 1440.0] {
for _ in 0..3 {
let input = egui::RawInput {
screen_rect: Some(egui::Rect::from_min_size(
egui::Pos2::ZERO,
egui::vec2(width, 640.0),
)),
..Default::default()
};
let _output = context.run(input, |ctx| {
egui::CentralPanel::default().show(ctx, |ui| {
let right = ui.max_rect().right();
egui::ScrollArea::both()
.auto_shrink([false, false])
.show(ui, |ui| {
device_table(
ui,
Unit::Bytes,
&network,
&rows,
&Traffic::default(),
&mut None,
&mut None,
);
});
assert!(
(ui.min_rect().right() - right).abs() < 2.0,
"table width after resize: {} vs {right}",
ui.min_rect().right()
);
});
});
}
}
}
}
+8 -2
View File
@@ -45,6 +45,8 @@ pub(crate) struct UiState {
pub(crate) about_open: bool,
/// An exit-node choice waiting for the user to confirm it.
pub(crate) exit_confirm: Option<devices::ExitConfirm>,
/// A selected offline claim waiting for cleanup confirmation.
pub(crate) forget_confirm: Option<devices::ForgetConfirm>,
}
impl Default for UiState {
@@ -56,6 +58,7 @@ impl Default for UiState {
open_devices: BTreeSet::new(),
about_open: false,
exit_confirm: None,
forget_confirm: None,
}
}
}
@@ -390,7 +393,7 @@ fn draw_protocols(
}
if let Some(index) = selected.iter().position(|p| p == &protocol.name) {
if ui
.add_enabled(index > 0, egui::Button::new("↑"))
.add_enabled(index > 0, egui::Button::new(icon::ARROW_UP))
.on_hover_text("prefer earlier")
.clicked()
{
@@ -398,7 +401,10 @@ fn draw_protocols(
changed = true;
}
if ui
.add_enabled(index + 1 < selected.len(), egui::Button::new("↓"))
.add_enabled(
index + 1 < selected.len(),
egui::Button::new(icon::ARROW_DOWN),
)
.on_hover_text("prefer later")
.clicked()
{
+82
View File
@@ -204,6 +204,13 @@ struct JoinArgs {
#[derive(Debug, Subcommand)]
enum NetworkAction {
/// Forgets an offline node's observed claim on every replica; newer claims may return.
ForgetNode {
/// Exact network name, full id or unique id prefix.
network: String,
/// Exact node hostname, full endpoint id, unique id prefix or overlay IPv4.
node: String,
},
/// Shows or changes this network's local LAN broadcast participation.
Broadcast {
/// Exact network name, full id or unique id prefix.
@@ -1570,6 +1577,31 @@ struct AgentControl {
}
impl tsunagi::ipc::ReportSource for AgentControl {
fn forget_node(
&self,
network: String,
peer: String,
expected_version: u64,
) -> tsunagi::BoxFuture<'_, Result<tsunagi::ipc::ForgottenNodeReport, String>> {
Box::pin(async move {
let network = network
.parse()
.map_err(|err| format!("invalid network id: {err}"))?;
let peer = peer
.parse()
.map_err(|err| format!("invalid endpoint id: {err}"))?;
let result = self
.agent
.forget_node(network, peer, expected_version)
.await
.map_err(|err| err.to_string())?;
Ok(tsunagi::ipc::ForgottenNodeReport {
peer: result.target.to_string(),
record_version: result.record_version,
peers_told: result.peers_told as u32,
})
})
}
fn report(&self) -> tsunagi::BoxFuture<'_, tsunagi::ipc::StatusReport> {
Box::pin(async move {
let dns = self.dns_state().await;
@@ -2035,6 +2067,46 @@ async fn protocols_command(args: ProtocolsArgs) -> Result<(), Box<dyn std::error
Ok(())
}
async fn forget_node_command(
socket: &std::path::Path,
network: &str,
node: &str,
) -> Result<(), Box<dyn std::error::Error>> {
let report = tsunagi::ipc::request_status(socket)
.await
.map_err(|err| format!("a reachable running agent is required to forget nodes: {err}"))?;
let entries: Vec<_> = report
.networks
.iter()
.map(|n| (n.network_id.clone(), n.name.clone()))
.collect();
let network = &report.networks[resolve_network_index(&entries, network)?];
if !network.active {
return Err(format!(
"start `{}` first: tsng network start {}",
network.name, network.network_id
)
.into());
}
let member = tsunagi::ipc::resolve_node(network, node)?;
let result = tsunagi::ipc::forget_node(
socket,
&network.network_id,
&member.endpoint_id,
member.record_version,
)
.await?;
println!(
"forgot {} ({}) claim v{} in `{}`; cleanup queued to {} peers. A newer signed claim can return.",
member.hostname.as_deref().unwrap_or("unnamed"),
result.peer,
result.record_version,
network.name,
result.peers_told
);
Ok(())
}
/// `tsng network`: what this device belongs to, and leaving it.
async fn network_command(args: NetworkArgs) -> Result<(), Box<dyn std::error::Error>> {
let paths = args.paths.resolve()?;
@@ -2042,6 +2114,9 @@ async fn network_command(args: NetworkArgs) -> Result<(), Box<dyn std::error::Er
let socket = control_socket(&paths, args.control_socket.as_ref(), is_custom_state);
let paths = agent_paths(paths, &socket, is_custom_state).await;
match args.action {
Some(NetworkAction::ForgetNode { network, node }) => {
forget_node_command(&socket, &network, &node).await
}
None => show_networks(&paths, &socket).await,
Some(NetworkAction::Broadcast { network, choice }) => {
broadcast_command(&paths, &socket, &network, choice).await
@@ -5259,6 +5334,7 @@ async fn build_report(
.members
.iter()
.map(|member| MemberReport {
record_version: member.record_version,
endpoint_id: member.endpoint_id.to_string(),
overlay_address_v4: member.overlay_address_v4.map(|addr| addr.to_string()),
// What this agent is currently experiencing trying to
@@ -5688,6 +5764,7 @@ mod status_tests {
}],
members: vec![
MemberReport {
record_version: 1,
endpoint_id: OWN.into(),
overlay_address_v4: Some("10.13.37.69".into()),
failed_dials: 0,
@@ -5695,6 +5772,7 @@ mod status_tests {
..Default::default()
},
MemberReport {
record_version: 1,
endpoint_id: ONLINE.into(),
overlay_address_v4: Some("10.13.37.237".into()),
failed_dials: 0,
@@ -5971,6 +6049,7 @@ mod status_tests {
fn offline_members_are_listed_after_online_ones() {
let mut network = network_after_a_peer_returned();
network.members.push(MemberReport {
record_version: 1,
endpoint_id: AWAY.into(),
overlay_address_v4: Some("10.13.37.99".into()),
failed_dials: 9,
@@ -6092,6 +6171,7 @@ mod status_tests {
fn an_offline_member_keeps_the_last_name_it_announced() {
let mut network = network_after_a_peer_returned();
network.members.push(MemberReport {
record_version: 1,
endpoint_id: AWAY.into(),
overlay_address_v4: Some("10.13.37.99".into()),
failed_dials: 0,
@@ -6113,6 +6193,7 @@ mod status_tests {
// are attributed to it instead of floating free as a counter.
let mut network = network_after_a_peer_returned();
network.members.push(MemberReport {
record_version: 1,
endpoint_id: AWAY.into(),
overlay_address_v4: Some("10.13.37.99".into()),
failed_dials: 9,
@@ -6137,6 +6218,7 @@ mod status_tests {
// and said to be, rather than being shown as away or as a fault.
let mut network = network_after_a_peer_returned();
network.members.push(MemberReport {
record_version: 1,
endpoint_id: AWAY.into(),
overlay_address_v4: Some("10.13.37.99".into()),
hostname: Some("laptop".into()),
+119
View File
@@ -58,6 +58,15 @@ fn start_bare_range(port: u16, range: Option<&str>) -> Running {
}
fn start_bare_config(port: u16, range: Option<&str>, protocols: Option<&str>) -> Running {
start_bare_peers(port, range, protocols, &[])
}
fn start_bare_peers(
port: u16,
range: Option<&str>,
protocols: Option<&str>,
peers: &[String],
) -> Running {
let dir = TempDir::new().unwrap();
let mut command = std::process::Command::new(env!("CARGO_BIN_EXE_tsng"));
command
@@ -78,6 +87,9 @@ fn start_bare_config(port: u16, range: Option<&str>, protocols: Option<&str>) ->
if let Some(protocols) = protocols {
command.args(["--protocol", protocols]);
}
for peer in peers {
command.args(["--peer", peer]);
}
let child = command.spawn().expect("the agent binary starts");
let mut agent = Running { child, dir };
let runtime = tokio::runtime::Builder::new_current_thread()
@@ -119,6 +131,113 @@ fn start(network: &str, port: u16) -> Running {
agent
}
#[test]
fn cli_forgets_only_the_selected_offline_claim_over_real_control() {
let cleaner = start("forget-cli", 0);
let runtime = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
let socket = tsunagi::ipc::control_socket_path(&cleaner.dir.path().join("state"));
let read = || {
runtime
.block_on(tsunagi::ipc::request_status(&socket))
.unwrap()
};
let before = read();
let bootstrap = format!("{}@{}", before.endpoint_id, before.bound_sockets[0]);
let target = start_bare_peers(0, None, Some("none"), &[bootstrap]);
assert!(
target
.run(&["id", "hostname", "old-cli-node"])
.status
.success()
);
assert!(
target
.run(&[
"join",
"--network",
"forget-cli",
"--secret",
"a-secret-for-the-cli-test"
])
.status
.success()
);
runtime.block_on(async {
tokio::time::timeout(Duration::from_secs(30), async {
loop {
let status = tsunagi::ipc::request_status(&socket).await.unwrap();
if status.networks[0]
.members
.iter()
.any(|m| m.hostname.as_deref() == Some("old-cli-node") && m.online)
{
break;
}
tokio::time::sleep(Duration::from_millis(25)).await;
}
})
.await
.unwrap();
});
assert!(
!cleaner
.run(&["network", "forget-node", "forget-cli", "old-cli-node"])
.status
.success()
);
assert!(
!cleaner
.run(&["network", "forget-node", "forget-cli", &before.endpoint_id])
.status
.success()
);
assert!(
target
.run(&["network", "stop", "forget-cli"])
.status
.success()
);
runtime.block_on(async {
tokio::time::timeout(Duration::from_secs(30), async {
loop {
let status = tsunagi::ipc::request_status(&socket).await.unwrap();
if status.networks[0]
.members
.iter()
.any(|m| m.hostname.as_deref() == Some("old-cli-node") && !m.online)
{
break;
}
tokio::time::sleep(Duration::from_millis(25)).await;
}
})
.await
.unwrap();
});
let forgotten = cleaner.run(&["network", "forget-node", "forget-cli", "old-cli-node"]);
assert!(
forgotten.status.success(),
"{}",
String::from_utf8_lossy(&forgotten.stderr)
);
assert!(String::from_utf8_lossy(&forgotten.stdout).contains("newer signed claim can return"));
assert!(
!read().networks[0]
.members
.iter()
.any(|m| m.hostname.as_deref() == Some("old-cli-node"))
);
assert!(
!cleaner
.run(&["network", "forget-node", "forget-cli", "old-cli-node"])
.status
.success()
);
}
#[test]
fn protocols_absent_from_startup_can_be_enabled_and_changed_offline() {
let mut agent = start_bare_config(0, None, Some("none"));
+30 -6
View File
@@ -43,12 +43,36 @@ be an exit node in one network and not in another.
`tsunagi-exit:<interface>:<range>` and are removed when the offer ends, the
network stops, and when the agent exits. Only networks that offer have
rules, so the others' traffic is not forwarded.
- **Client:** policy routing, with no change to the main table. Table `28787`
holds `default dev <interface>`, and three rules sit after Tailscale's:
`5280 uidrange <agent uid> lookup main` (the agent's own connections never
go through the tunnel, which would be a loop), `5290 lookup main
suppress_prefixlength 0` (LAN and other specific routes keep working) and
`5300 lookup 28787`.
- **Client:** policy routing, with no change to the main table. Table `29811`
holds `default dev <interface>`. Rules `5205 fwmark 0x40000000/0x40000000
lookup main` and `5206 uidrange <agent uid> lookup main` keep the agent's
sockets and their replies outside the tunnel, before Tailscale's rules.
`5290 lookup main suppress_prefixlength 0` preserves LAN and other specific
routes, and `5300 lookup 29811` directs other traffic through the exit.
The exit table is `29811` (`0x7473`).
- **Linux reverse-path filtering:** only the agent's dedicated UID is marked
in mangle `OUTPUT`. Its own bit, `0x40000000`, is saved to conntrack and
restored in `PREROUTING` before the host firewall checks reply routing.
Other mark bits, including Tailscale's `0xff0000` mask, are preserved. The
rules are tagged `tsunagi-exit-underlay:<interface>` and never accept packets
around the host firewall. IPv4 and IPv6 protection is installed before exit
routes; routes are removed before protection, including rollback on failure.
Locally owned TCP listeners and bound UDP sockets also protect the first
incoming packet, before any outgoing packet has created a connection mark.
Socket addresses and ports
come from local kernel socket tables and match an actual socket; remote
announcements never supply these firewall arguments.
The socket set is refreshed before private pair negotiation is sent and on
discovery ticks, so fresh AWG ports are included. An unchanged set causes no
firewall writes; a socket-refresh failure leaves active exit routes in place.
Linux needs both `iptables` and `ip6tables` when IPv6 is present.
- **Kernel strict `rp_filter`:** if an interface uses strict mode (`1`), the
kernel must include marks in reverse lookups: set
`net.ipv4.conf.all.src_valid_mark=1`, or enable it on those interfaces.
The agent checks this before installing exit routes and reports an actionable
error if marks would be ignored. It never disables `rp_filter` or changes
host sysctls. Netfilter strict reverse checks also need their mark-aware
form, such as `rpfilter --validmark`, which NixOS already uses.
- Packets for addresses outside every overlay range are sent to the exit node
by the router; the exit node's replies are accepted from any source. A member
that is not your exit node cannot do that.
+9 -4
View File
@@ -9,14 +9,18 @@ Two versions exist and are independent:
- **Identity scheme**, `tsunagi-network-id-v1`. Frozen. Changing it creates a
different network space for the same name and secret.
- **Control protocol**, ALPN `tsunagi/ctrl/6`, `PROTOCOL_VERSION = 6`.
- **Control protocol**, ALPN `tsunagi/ctrl/7`, `PROTOCOL_VERSION = 7`.
Upgrading the crate or bumping the control protocol must never change an
existing `NetworkId`.
Control version 6 adds private transport negotiation for pairwise AWG and data
delivery checks. Data ALPN remains `tsunagi/data/5`. All members must upgrade
together; network ids and stored state are unchanged. See [amnezia.md](amnezia.md).
Control version 7 adds signed offline-node retirement messages to version 6's
private AWG transport negotiation and delivery checks. Data ALPN remains
`tsunagi/data/5`. All members must upgrade together; network ids, handshake
encoding and existing record signatures are unchanged. Local CLI/tray IPC is
version 21, so restart the agent with matching clients. SQLite schema 9 adds
retirement storage while preserving existing identities, settings and records.
See [sync-model.md](sync-model.md#offline-node-cleanup) and [amnezia.md](amnezia.md).
## Canonical encoding
@@ -238,6 +242,7 @@ not affect other networks.
| `Ping { seq, payload }` | small request used to verify the exchange |
| `Pong { seq, payload }` | the echoed reply |
| `State { records }` | a snapshot of signed records, merged into what the receiver holds |
| `Retirements { markers }` | signed cleanup markers with the target's original signed claim as proof |
| `Peers { peers }` | unverified member address hints, requiring their own handshake |
| `Reachable { links }` | sender's current direct data links, by kind of transport |
| `Bye { reason }` | graceful goodbye; not a revocation of anything |
+1 -1
View File
@@ -124,5 +124,5 @@ counts sends without storing frames. Results are CPU forwarding cost, **not**
end-to-end network latency or VPN throughput; encryption, fragmentation,
sockets, congestion and scheduling contribute separately.
Wire compatibility: control ALPN `tsunagi/ctrl/6`, data ALPN `tsunagi/data/5`.
Wire compatibility: control ALPN `tsunagi/ctrl/7`, data ALPN `tsunagi/data/5`.
Upgrade every participant together; saved identities and network state persist.
+46 -27
View File
@@ -1,18 +1,9 @@
# Planned state synchronisation
# State synchronisation
**The first slice of this model is now implemented**, in `crates/tsunagi-core/src/state/`, and is
used for one thing: IPv4 overlay addresses. What follows describes the whole
model; the section at the end says exactly which parts exist.
The proof of concept still exchanges hostname and capability announcements
over live sessions and keeps no replicated history for those, which is why
WireGuard peer *membership* remains session-scoped even though a peer's
*address* no longer is. This file records the intended direction so the module
boundaries in [architecture.md](architecture.md) stay compatible with it, and so
nobody mistakes the current announcements for synchronisation.
There is no fake "ready CRDT" here and no snapshots that are not actually
verified.
Signed claims in `crates/tsunagi-core/src/state/` preserve IPv4 reservations and
hostnames across an absence. Signed releases and offline-node cleanup markers
free those reservations durably. Capabilities, links and authenticated sessions
remain volatile; a stored claim never authenticates a live connection.
## The problem
@@ -75,6 +66,39 @@ Turning a computer off is not a revocation of its hostname and does not remove
the participant. Revocations are not dropped merely because they are old, and
no acknowledgement from offline peers is required to keep working.
## Offline-node cleanup
Any authenticated member can sign a `SignedRetirement` for another member's
observed `Claim`. It includes the original signed claim as proof and uses the
separate `tsunagi-node-retirement/v1` signing domain. Both signatures and both
network scopes are checked. No owner, vote or quorum is involved, and no member
can fabricate the target's release or invent a future version cutoff.
The strongest observed target version wins; ties use a deterministic canonical
byte order. Claims at or below that version do not occupy an address, hostname
or visible member row. Original signed records remain stored, including their
range, so cleanup does not renumber the network. Markers remain even after a
newer claim returns: absence from a snapshot is never deletion and replaying an
old snapshot cannot restore the retired claim.
The local `Agent::forget_node` API checks the exact observed version again and
refuses self, online nodes, the selected exit node and missing/changed claims.
CLI and tray share this API. A returning live node can sign a newer claim and
allocate against the current holders; it resets the old hostname priority, so
it cannot reclaim a reused name merely by keeping its old timestamp. Cleanup
is not a permanent identity ban or a change of secret.
The marker and requester's own author counter commit in one SQLite transaction
before publication. Receivers persist a verified marker before applying it.
Invalid proofs are rejected individually. Periodic bounded marker batches
rotate through each peer's send queue, including when a queue fills, so a
replica that missed an update can catch up without the requester being online.
One strongest marker per target is retained, with a default limit of 1024 per
network and 1024 bytes per encoded marker. New targets beyond that limit are
refused rather than evicting resurrection protection. Corrupt mandatory marker
storage or a configured limit smaller than the stored set is an explicit error.
## Storage requirement this creates
When signed records land, writing the event and bumping the author's own
@@ -104,9 +128,9 @@ Implemented, in `crates/tsunagi-core/src/state/`:
* the merge rules above: higher version wins, an older version never rolls
back a newer one, duplicates are idempotent, a same-version conflict is
resolved identically on every replica and reported;
* a release tombstone, which merges correctly and is not undone by a replica
that has not heard of it — though nothing emits one yet, so freeing an
address still means forgetting the network;
* a release tombstone, emitted when leaving or replacing the device key and
not undone by a replica that has not heard of it;
* durable hostnames and equal-rights offline-node retirement described above;
* persistence in `state.sqlite`, with the record and the author's own version
counter committed in **one transaction before the record is announced**;
* distribution as a `State` control message, merged into what the receiver
@@ -114,21 +138,16 @@ Implemented, in `crates/tsunagi-core/src/state/`:
* allocation of a free IPv4 address against what everybody else holds, which
is what makes an address stable across an absence.
Deliberately not implemented: compaction, revoking a whole author, record
types beyond addressing, and any bound on how large a snapshot may grow
beyond the per-message limit.
Deliberately not implemented: compaction and permanent author revocation.
Claims have a per-message bound; retirement history also has a per-network bound.
## Future tests
These are **not implemented and must not be reported as passing**:
- snapshot merge against the rules above, including conflicting same-version
records;
- revocation propagation and resistance to resurrection after compaction;
- resistance to resurrection after compaction (compaction is not implemented);
- long network partitions and rejoin after an extended absence;
- hostname rename with atomic revoke-and-announce;
- recovery of a returning participant from a replica that is not the author;
- NAT traversal and hole punching between real hosts;
- relay fallback behaviour against a self-hosted relay;
- multi-process and multi-host deployment, as opposed to several library
instances inside one test process.
- multi-host deployment. CLI tests do run separate local agent processes;
library tests run several independent instances inside one process.
+34 -4
View File
@@ -164,10 +164,9 @@ paused clock. The public Mainline/relay smoke test is ignored by default.
## Not covered, and not claimed to be
Listed in [sync-model.md](sync-model.md#future-tests): snapshots, revocations,
long partitions, hostname renames, recovery of a returning participant, NAT
traversal, relay fallback, and multi-process or multi-host deployment. None of
these are implemented, and none are marked as passing.
Listed in [sync-model.md](sync-model.md#future-tests): compaction, long partitions,
NAT traversal, relay fallback and multi-host deployment. These scenarios are
not established by the default loopback suite.
## Debugging a test
@@ -198,3 +197,34 @@ SQLite tests migrate schema 7 to 8 without changing
identity, secrets or signed records, preserve order and explicit empty choices,
and reject malformed stored selections. These checks use memory TUNs and do
not establish GUI rendering or behavior on an actual OS interface.
`tests/forget_node.rs` uses real agents, handshakes and SQLite to check equal
cleanup rights, network isolation, online/self/stale-version refusal, restart,
recovery of a stale replica, reuse of a hostname and a returning identity's
newer claim. Bounded batch delivery is checked with a two-message send queue
and twelve retained markers. Retirement unit tests check signatures, proof tampering, invented
future cutoffs, bounds, deterministic merge and resistance to stale snapshots.
SQLite tests migrate schema 8 to 9 and inject counter-write failure to prove
atomic rollback. The CLI test starts two real agent processes and exercises
offline cleanup by network and hostname through the control socket.
Native tray rendering and physical screen sizing are not exercised automatically.
Linux exit-client unit tests cover preservation of foreign mark bits, matching
only the agent's UID and locally owned TCP/UDP sockets, exact tagged cleanup,
partial IPv6 failure and mark-aware strict kernel RPF validation. The ignored
`overlay::hostrules::linux_exit_kernel::strict_firewall_preserves_underlay_and_application_exit_routing`
test creates two disposable network namespaces. It reproduces the UID-only
failure with real sockets, then runs TCP/UDP IPv4/IPv6 and first inbound TCP/UDP
packets, including a UDP port bound after activation, through real mark/conntrack
rules with both strict kernel and
Netfilter reverse-path checks. It verifies application exit routing, IPv6
blocking, repeated apply and cleanup with foreign rules at Tailscale's priority
retained. It requires root, iproute2,
iptables/ip6tables and Python; it does not run in the default suite or claim
to test the entire mesh inside these namespaces.
Devices-window tests render the egui grid at multiple widths with its real
scroll area. They check stretching and shrinking without an expanding first
column, separate handshake/control RTT values, unknown measurements and short
end-to-end hints that do not substitute the carrier protocol. This does not
establish native desktop rendering on every platform.
+12 -2
View File
@@ -38,6 +38,14 @@ Read this before relying on anything here. The protocol is in
arbitrarily many identities, flood the network with records and collide with
other participants' names. This is why a majority is not a root of trust.
Signatures protect authorship; they do not make a participant honest.
- **Cleanup rights are equal.** Any member can retire an observed signed claim
of another device, freeing its hostname and IPv4 reservation. A valid original
claim bounds the cutoff; a member cannot invent a future cutoff or forge the
target's release. The local API refuses online nodes, but a remote signature
cannot prove that a target was offline. A live target may publish a newer
claim; a malicious member can repeatedly disrupt reservations. Cleanup is
not expulsion or secret revocation. See
[sync-model.md](sync-model.md#offline-node-cleanup).
- **Weak secrets.** This targets high-entropy secrets. There is no PAKE, so a
short human passphrase can be guessed offline by anyone who can reach the
handshake. Use `NetworkSecret::generate()`.
@@ -55,8 +63,10 @@ Read this before relying on anything here. The protocol is in
having iroh under it does not make it so.
- **A cloned state directory is a cloned identity.** `state.sqlite` holds the
device secret key and the network secrets. Copying it copies the participant.
Restoring an old backup rolls the agent's state back, which — once signed
records exist — can resurrect revoked information or replay stale versions.
Restoring an old backup rolls the agent's state back and can replay stale
versions until it meets a replica holding the newer signed state. Retained
cleanup markers prevent old claims from resurrecting after that merge; they
cannot help if every copy of a marker has been lost.
- **A compromised host.** The secret is on disk to survive restarts. File
permissions are owner-only where the platform supports it, and the state
directory takes an ownership lock, but neither defends against a user who can
+1
View File
@@ -102,6 +102,7 @@ in
# Exit nodes: the agent looks for iptables only in fixed FHS directories.
systemd.tmpfiles.rules = [
"L+ /usr/bin/iptables - - - - ${pkgs.iptables}/bin/iptables"
"L+ /usr/bin/ip6tables - - - - ${pkgs.iptables}/bin/ip6tables"
];
# Names of mesh members are served locally and handed to the system resolver.