Exit nodes on Windows: use a member as exit node via destination bypass routes

Choosing an exit node on Windows did nothing: the using side was refused,
because Windows has no per-user routing and the agent's own traffic would loop
into the tunnel it carries.

Now the overlay interface gets 0.0.0.0/1 and 128.0.0.0/1 routes (and ::/1,
8000::/1 as an IPv6 blackhole), and every underlay address the agent talks to
directly gets a /32 route through the physical default gateway, marked with
route metric 7373 so stale ones are found and removed. Bypass routes are
written before the halves and removed after them; a failed step removes what
was written. All routes are in the active store.

The addresses come from the runtime: each network runtime hands the interface
the verified direct IPv4 paths of its control connections and the relay hosts
in use (Interface::set_underlay, refreshed every dial round); relay names are
resolved when the rules are applied. ExitHostRules::needs_bypass (true only on
Windows) gates it, so Linux and macOS plans never carry or re-apply on path
changes. ExitHostPlan gains `bypass` and `bypass_hosts`; networks leaving
forget their entry.

Limits: a path that appears after the last apply goes through the tunnel until
the next apply; data-plane-only addresses are not collected separately (the
data links ride the same iroh paths). Nothing was run on Windows.

Checks: cargo fmt --check, cargo clippy --locked --workspace --all-targets
-D warnings and cargo test --locked --workspace --all-targets passed on macOS;
scripts and ordering are unit-tested with an injected runner and resolver, and
Interface::set_underlay with the mock host.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

.gitea/workflows/release.yml

@@ -45,6 +45,16 @@ jobs:
             deb: true
             deb_arch: arm64
             pkg_arch: aarch64
+          # 32-bit ARM with hardware float (ARMv7, `v7l`): single-board and edge
+          # devices. Statically linked like the others.
+          - os: ubuntu-latest
+            target: armv7-unknown-linux-musleabihf
+            asset: tsunagi-linux-armv7
+            archive: tar.gz
+            zig: true
+            deb: true
+            deb_arch: armhf
+            pkg_arch: armv7h
           - os: macos-latest
             target: aarch64-apple-darwin
             asset: tsunagi-macos-aarch64
@@ -163,6 +173,10 @@ jobs:
             macOS)
               cp dist/macos/cy.hexor.tsunagi.plist "$stage/"
               cp dist/macos/README.md "$stage/INSTALL.md"
+              # The tray as an application bundle, for the Homebrew cask.
+              bash dist/macos/build-app.sh \
+                "target/${{ matrix.target }}/release/tsunagi-tray" \
+                "$version" "$stage/Tsunagi.app"
               ;;
             Windows)
               cp dist/windows/README.md "$stage/INSTALL.md"

.github/workflows/release.yml

@@ -47,6 +47,16 @@ jobs:
             deb: true
             deb_arch: arm64
             pkg_arch: aarch64
+          # 32-bit ARM with hardware float (ARMv7, `v7l`): single-board and edge
+          # devices. Statically linked like the others.
+          - os: ubuntu-latest
+            target: armv7-unknown-linux-musleabihf
+            asset: tsunagi-linux-armv7
+            archive: tar.gz
+            zig: true
+            deb: true
+            deb_arch: armhf
+            pkg_arch: armv7h
           - os: macos-latest
             target: aarch64-apple-darwin
             asset: tsunagi-macos-aarch64
@@ -165,6 +175,10 @@ jobs:
             macOS)
               cp dist/macos/cy.hexor.tsunagi.plist "$stage/"
               cp dist/macos/README.md "$stage/INSTALL.md"
+              # The tray as an application bundle, for the Homebrew cask.
+              bash dist/macos/build-app.sh \
+                "target/${{ matrix.target }}/release/tsunagi-tray" \
+                "$version" "$stage/Tsunagi.app"
               ;;
             Windows)
               cp dist/windows/README.md "$stage/INSTALL.md"

crates/tsunagi-core/src/agent/mod.rs

@@ -800,6 +800,11 @@ impl Agent {
         // itself stays, because the agent owns it and other networks may
         // still be using it.
         self.inner.routes.remove_network(network_id);
+        if let Some(interface) = self.inner.interface.get() {
+            interface
+                .set_underlay(network_id, Vec::new(), Vec::new())
+                .await;
+        }
         if let Some(interface) = self.inner.interface.get()
             && let Err(err) = interface.sync_addresses().await
         {

crates/tsunagi-core/src/agent/network.rs

@@ -712,6 +712,34 @@ impl Runtime {
         self.start_dials();
         // Last, so it carries what this round learned.
         self.introduce_peers();
+        self.refresh_underlay().await;
+    }
+
+    /// Hands the interface the underlay destinations this network depends
+    /// on: the verified direct IPv4 paths of every control connection, and
+    /// the relays in use. Only hosts that cannot exempt the agent from an
+    /// exit node's default route read it.
+    async fn refresh_underlay(&self) {
+        let Some(interface) = self.params.interface.clone() else {
+            return;
+        };
+        let mut addresses = Vec::new();
+        let mut hosts = Vec::new();
+        for session in self.sessions.values() {
+            for path in snapshot_connection(&session.conn).paths {
+                match path.remote {
+                    PathAddr::Ip(std::net::SocketAddr::V4(socket)) => addresses.push(*socket.ip()),
+                    PathAddr::Relay(url) => hosts.extend(relay_host(&url)),
+                    _ => {}
+                }
+            }
+        }
+        for url in self.params.adapter.snapshot().relay_urls {
+            hosts.extend(relay_host(&url));
+        }
+        interface
+            .set_underlay(self.network_id, addresses, hosts)
+            .await;
     }
 
     fn add_candidate(&mut self, candidate: Candidate) {
@@ -2448,3 +2476,48 @@ mod tests {
         assert_eq!(chosen_restored.copied(), Some("wg"));
     }
 }
+
+/// The host name of a relay URL such as `https://relay.example:443/`.
+fn relay_host(url: &str) -> Option<String> {
+    let rest = url.split_once("://").map_or(url, |(_, rest)| rest);
+    let authority = rest.split(['/', '?', '#']).next()?;
+    let host = authority.rsplit_once('@').map_or(authority, |(_, h)| h);
+    let host = if host.starts_with('[') {
+        host.split_once(']')?.0.trim_start_matches('[')
+    } else {
+        host.split(':').next()?
+    };
+    (!host.is_empty()
+        && host.len() <= 253
+        && host
+            .chars()
+            .all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == ':'))
+    .then(|| host.to_ascii_lowercase())
+}
+
+#[cfg(test)]
+mod relay_host_tests {
+    use super::relay_host;
+
+    #[test]
+    fn the_host_is_taken_out_of_a_relay_url() {
+        assert_eq!(
+            relay_host("https://relay.example.com./").as_deref(),
+            Some("relay.example.com.")
+        );
+        assert_eq!(
+            relay_host("https://Euw1-1.relay.iroh.network:443/x").as_deref(),
+            Some("euw1-1.relay.iroh.network")
+        );
+        assert_eq!(
+            relay_host("http://192.0.2.7:3340").as_deref(),
+            Some("192.0.2.7")
+        );
+        assert_eq!(
+            relay_host("https://[2001:db8::1]:443/").as_deref(),
+            Some("2001:db8::1")
+        );
+        assert_eq!(relay_host("https:///").as_deref(), None);
+        assert_eq!(relay_host("https://bad host/").as_deref(), None);
+    }
+}

crates/tsunagi-core/src/overlay/hostrules/exit.rs

@@ -35,6 +35,17 @@ pub struct ExitHostPlan {
     pub offer: Vec<Ipv4Range>,
     /// Whether this device sends its internet traffic through a member.
     pub client: bool,
+    /// The range of every network on the interface. Only the using side
+    /// reads it, to keep this agent's own traffic *to* the overlay inside it
+    /// where the host has no per-user routing (macOS).
+    pub overlay: Vec<Ipv4Range>,
+    /// Underlay addresses this agent talks to directly and must keep reaching
+    /// outside the overlay's default route. Only filled while `client` is set
+    /// and only for hosts that report [`ExitHostRules::needs_bypass`].
+    pub bypass: Vec<std::net::Ipv4Addr>,
+    /// Host names (relay servers) that must stay reachable the same way; the
+    /// host rules resolve them.
+    pub bypass_hosts: Vec<String>,
 }
 
 impl ExitHostPlan {
@@ -72,6 +83,13 @@ pub trait ExitHostRules: Send + Sync + std::fmt::Debug + 'static {
     /// when nothing is there: it runs on the shutdown path, and once at
     /// startup to take away what a crashed run left.
     fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()>;
+
+    /// Whether using an exit node on this host needs the underlay addresses
+    /// the agent talks to, because the host cannot exempt the agent's own
+    /// traffic from the default route by user id (Windows). The default is no.
+    fn needs_bypass(&self) -> bool {
+        false
+    }
 }
 
 /// The tag every firewall object of one offered range carries.
@@ -99,6 +117,7 @@ pub struct MockExitRules {
     calls: Arc<Mutex<Vec<String>>>,
     forwarding: Arc<Mutex<Option<bool>>>,
     failure: Arc<Mutex<Option<String>>>,
+    needs_bypass: Arc<Mutex<bool>>,
 }
 
 impl MockExitRules {
@@ -114,6 +133,11 @@ impl MockExitRules {
         *lock(&self.forwarding) = forwarding;
     }
 
+    /// Whether this host asks for the underlay addresses, like Windows.
+    pub fn set_needs_bypass(&self, needs: bool) {
+        *lock(&self.needs_bypass) = needs;
+    }
+
     /// Makes every step fail with this reason from now on.
     pub fn fail_with(&self, reason: Option<String>) {
         *lock(&self.failure) = reason;
@@ -138,6 +162,10 @@ fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
 }
 
 impl ExitHostRules for MockExitRules {
+    fn needs_bypass(&self) -> bool {
+        *lock(&self.needs_bypass)
+    }
+
     fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport> {
         Box::pin(async move {
             lock(&self.calls).push(format!("apply:{}", plan.interface));
@@ -204,6 +232,8 @@ mod tests {
             interface: "tsun0".into(),
             offer: vec![range()],
             client: true,
+            overlay: vec![range()],
+            ..Default::default()
         };
         let report = rules.apply(&plan).await;
         assert_eq!(report.offer[0].1, RuleOutcome::Applied);
@@ -218,6 +248,8 @@ mod tests {
                 interface: "tsun0".into(),
                 offer: vec![range()],
                 client: false,
+                overlay: Vec::new(),
+                ..Default::default()
             })
             .await;
         assert_eq!(report.forwarding, Some(false));

crates/tsunagi-core/src/overlay/hostrules/linux.rs

@@ -66,24 +66,68 @@ pub(super) enum Ran {
 pub(super) type Run<'a> = &'a dyn Fn(&[String]) -> Ran;
 
 /// The match and target of the rule, without the chain or position.
-fn rule_spec(plan: &BroadcastRulesPlan) -> Vec<String> {
-    [
-        "-i",
-        plan.interface.as_str(),
-        "-p",
-        "udp",
-        "-s",
-        &plan.range.to_string(),
-        "-m",
-        "comment",
-        "--comment",
-        &plan.tag(),
-        "-j",
-        "ACCEPT",
-    ]
-    .into_iter()
-    .map(str::to_string)
-    .collect()
+///
+/// `comment` false leaves the tag off, for a kernel that has no `comment`
+/// match; the rule is then recognised by its shape, see [`tagged_deletions`].
+fn rule_spec(plan: &BroadcastRulesPlan, comment: bool) -> Vec<String> {
+    let range = plan.range.to_string();
+    let tag = plan.tag();
+    let mut words = vec!["-i", plan.interface.as_str(), "-p", "udp", "-s", &range];
+    if comment {
+        words.extend(["-m", "comment", "--comment", &tag]);
+    }
+    words.extend(["-j", "ACCEPT"]);
+    words.into_iter().map(str::to_string).collect()
+}
+
+/// Whether the kernel has the `comment` match, which a rule needs to carry
+/// its tag. Asked of iptables itself: checking a rule that does not exist
+/// fails either way, but only a missing match says so.
+pub(super) fn comment_supported(run: Run<'_>) -> bool {
+    let probe = locked(
+        [
+            "-t",
+            "filter",
+            "-C",
+            "FORWARD",
+            "-m",
+            "comment",
+            "--comment",
+            "tsunagi-probe",
+            "-j",
+            "ACCEPT",
+        ]
+        .map(str::to_string),
+    );
+    match run(&probe) {
+        Ran::Exited {
+            success: false,
+            stderr,
+            ..
+        } => !stderr.contains("No chain/target/match by that name"),
+        _ => true,
+    }
+}
+
+/// `iptables -S` words as option clauses: an option with its value, a negated
+/// one with its `!`. It prints the options in its own order, not the one a
+/// rule was written in, so rules are compared by their clauses, sorted.
+fn clauses(words: &[String]) -> Vec<String> {
+    let mut out = Vec::new();
+    let mut at = 0;
+    while at < words.len() {
+        let take = if words[at] == "!" { 3 } else { 2 };
+        let end = (at + take).min(words.len());
+        out.push(words[at..end].join(" "));
+        at = end;
+    }
+    out.sort();
+    out
+}
+
+/// Whether two rules say the same thing, whatever order they list it in.
+pub(super) fn same_rule(a: &[String], b: &[String]) -> bool {
+    clauses(a) == clauses(b)
 }
 
 /// Waits briefly for the xtables lock instead of failing when another tool
@@ -96,19 +140,19 @@ pub(super) fn locked(args: impl IntoIterator<Item = String>) -> Vec<String> {
         .collect()
 }
 
-fn check_args(plan: &BroadcastRulesPlan) -> Vec<String> {
+fn check_args(plan: &BroadcastRulesPlan, comment: bool) -> Vec<String> {
     locked(
         ["-C".to_string(), CHAIN.to_string()]
             .into_iter()
-            .chain(rule_spec(plan)),
+            .chain(rule_spec(plan, comment)),
     )
 }
 
-fn insert_args(plan: &BroadcastRulesPlan) -> Vec<String> {
+fn insert_args(plan: &BroadcastRulesPlan, comment: bool) -> Vec<String> {
     locked(
         ["-I".to_string(), CHAIN.to_string(), "1".to_string()]
             .into_iter()
-            .chain(rule_spec(plan)),
+            .chain(rule_spec(plan, comment)),
     )
 }
 
@@ -147,8 +191,28 @@ pub(super) fn split_rule_line(line: &str) -> Vec<String> {
     words
 }
 
+/// Whether a rule is the one this module writes for some range of the
+/// interface the tag belongs to, with no comment on it.
+fn is_uncommented_rule(tag: &str, rest: &[String]) -> bool {
+    let Some(interface) = tag.strip_prefix("tsunagi:") else {
+        return false;
+    };
+    let Some(range) = rest
+        .windows(2)
+        .find(|pair| pair[0] == "-s")
+        .map(|pair| pair[1].as_str())
+    else {
+        return false;
+    };
+    let ours: Vec<String> = ["-i", interface, "-p", "udp", "-s", range, "-j", "ACCEPT"]
+        .map(str::to_string)
+        .to_vec();
+    same_rule(&ours, rest)
+}
+
 /// The delete commands for every rule in an `iptables -S INPUT` listing that
-/// carries exactly this tag. Anything else in the chain is left alone.
+/// carries exactly this tag, or, with no `comment` match in the kernel, is
+/// exactly the rule written for some range. Anything else is left alone.
 fn tagged_deletions(listing: &str, tag: &str) -> Vec<Vec<String>> {
     listing
         .lines()
@@ -162,7 +226,8 @@ fn tagged_deletions(listing: &str, tag: &str) -> Vec<Vec<String>> {
             }
             let tagged = rest
                 .windows(2)
-                .any(|pair| pair[0] == "--comment" && pair[1] == tag);
+                .any(|pair| pair[0] == "--comment" && pair[1] == tag)
+                || is_uncommented_rule(tag, rest);
             tagged.then(|| {
                 locked(
                     ["-D".to_string(), CHAIN.to_string()]
@@ -197,7 +262,8 @@ fn missing(plan: &BroadcastRulesPlan) -> String {
 
 /// Makes the firewall hold exactly one rule for this interface, the planned one.
 fn apply_firewall(run: Run<'_>, plan: &BroadcastRulesPlan) -> RuleOutcome {
-    let present = match run(&check_args(plan)) {
+    let comment = comment_supported(run);
+    let present = match run(&check_args(plan, comment)) {
         Ran::Missing => return RuleOutcome::Failed(missing(plan)),
         Ran::Exited { success, .. } => success,
     };
@@ -227,7 +293,7 @@ fn apply_firewall(run: Run<'_>, plan: &BroadcastRulesPlan) -> RuleOutcome {
             return RuleOutcome::Failed(failure(&stderr));
         }
     }
-    match run(&insert_args(plan)) {
+    match run(&insert_args(plan, comment)) {
         Ran::Missing => RuleOutcome::Failed(missing(plan)),
         Ran::Exited { success: true, .. } => RuleOutcome::Applied,
         Ran::Exited { stderr, .. } => RuleOutcome::Failed(failure(&stderr)),
@@ -365,7 +431,7 @@ mod tests {
 
     #[test]
     fn the_rule_is_udp_only_from_the_overlay_range_on_this_interface() {
-        let spec = words(&rule_spec(&plan()));
+        let spec = words(&rule_spec(&plan(), true));
         assert_eq!(
             spec,
             "-i tsun0 -p udp -s 10.13.37.0/24 -m comment --comment tsunagi:tsun0 -j ACCEPT"
@@ -373,8 +439,8 @@ mod tests {
         // Nothing here may widen to other protocols or every source.
         assert!(spec.contains("-p udp"));
         assert!(!spec.contains("0.0.0.0/0"));
-        assert!(words(&insert_args(&plan())).starts_with("-w 5 -I INPUT 1 -i tsun0"));
-        assert!(words(&check_args(&plan())).starts_with("-w 5 -C INPUT -i tsun0"));
+        assert!(words(&insert_args(&plan(), true)).starts_with("-w 5 -I INPUT 1 -i tsun0"));
+        assert!(words(&check_args(&plan(), true)).starts_with("-w 5 -C INPUT -i tsun0"));
     }
 
     #[test]
@@ -408,6 +474,49 @@ mod tests {
         assert!(tagged_deletions("", "tsunagi:tsun0").is_empty());
     }
 
+    #[test]
+    fn a_kernel_without_the_comment_match_says_so_in_its_error() {
+        let answer = |stderr: &'static str| {
+            move |_: &[String]| Ran::Exited {
+                success: false,
+                stdout: String::new(),
+                stderr: stderr.to_string(),
+            }
+        };
+        assert!(!comment_supported(&answer(
+            "iptables: No chain/target/match by that name."
+        )));
+        assert!(comment_supported(&answer(
+            "iptables: Bad rule (does a matching rule exist in that chain?)."
+        )));
+        assert!(comment_supported(&|_: &[String]| Ran::Missing));
+    }
+
+    #[test]
+    fn a_rule_written_without_a_comment_is_found_by_its_shape() {
+        // Listed the way iptables prints it: its own option order.
+        let listing = "\
+-P INPUT ACCEPT
+-A INPUT -s 10.13.37.0/24 -i tsun0 -p udp -j ACCEPT
+-A INPUT -s 10.99.0.0/16 -i tsun0 -p udp -j ACCEPT
+-A INPUT -s 10.13.37.0/24 -i tsun1 -p udp -j ACCEPT
+-A INPUT -s 10.13.37.0/24 -i tsun0 -p udp -j DROP
+-A INPUT -s 10.13.37.0/24 -i tsun0 -p tcp -j ACCEPT
+";
+        let deletions = tagged_deletions(listing, "tsunagi:tsun0");
+        assert_eq!(deletions.len(), 2, "{deletions:?}");
+        assert!(words(&deletions[0]).ends_with("-s 10.13.37.0/24 -i tsun0 -p udp -j ACCEPT"));
+        assert!(words(&deletions[1]).contains("10.99.0.0/16"));
+    }
+
+    #[test]
+    fn without_a_comment_the_rule_has_none() {
+        let with = words(&rule_spec(&plan(), true));
+        let without = words(&rule_spec(&plan(), false));
+        assert!(with.contains("-m comment --comment tsunagi:tsun0"));
+        assert_eq!(without, "-i tsun0 -p udp -s 10.13.37.0/24 -j ACCEPT");
+    }
+
     /// A scripted `iptables`: answers by the leading verb, records every call.
     struct Fake {
         calls: RefCell<Vec<String>>,
@@ -428,6 +537,14 @@ mod tests {
             }
         }
         fn run(&self, args: &[String]) -> Ran {
+            // The probe for the `comment` match: not part of what is tested.
+            if args.iter().any(|word| word == "tsunagi-probe") {
+                return Ran::Exited {
+                    success: false,
+                    stdout: String::new(),
+                    stderr: "Bad rule".into(),
+                };
+            }
             self.calls.borrow_mut().push(words(args));
             if self.missing {
                 return Ran::Missing;

crates/tsunagi-core/src/overlay/hostrules/linux_exit.rs

@@ -47,7 +47,10 @@ use crate::state::Ipv4Range;
 
 use super::RuleOutcome;
 use super::exit::{ExitHostPlan, ExitHostReport, ExitHostRules, exit_tag, exit_tag_prefix};
-use super::linux::{LinuxHostRules, Ran, Run, failure, locked, run_iptables, split_rule_line};
+use super::linux::{
+    LinuxHostRules, Ran, Run, comment_supported, failure, locked, run_iptables, same_rule,
+    split_rule_line,
+};
 
 /// The chains the offering rules live in: `(table, chain)`.
 const CHAINS: [(&str, &str); 2] = [("nat", "POSTROUTING"), ("filter", "FORWARD")];
@@ -65,14 +68,23 @@ fn owned(words: &[&str]) -> Vec<String> {
 }
 
 /// The three rules of one offered range.
-fn offer_rules(interface: &str, range: &Ipv4Range) -> Vec<Rule> {
+///
+/// Each carries a comment naming its interface and range, which is how it is
+/// found again. A kernel without the `comment` match (small router builds
+/// leave it out) cannot take that, so `comment` false leaves it off and the
+/// rules are recognised by their shape instead; see [`tagged_rules`].
+fn offer_rules(interface: &str, range: &Ipv4Range, comment: bool) -> Vec<Rule> {
     let range_text = range.to_string();
     let tag = exit_tag(interface, range);
-    let comment = ["-m", "comment", "--comment", tag.as_str()];
+    let comment: Vec<String> = if comment {
+        owned(&["-m", "comment", "--comment", tag.as_str()])
+    } else {
+        Vec::new()
+    };
     let spec = |head: &[&str], tail: &[&str]| -> Vec<String> {
         owned(head)
             .into_iter()
-            .chain(owned(&comment))
+            .chain(comment.iter().cloned())
             .chain(owned(tail))
             .collect()
     };
@@ -142,9 +154,16 @@ struct Found {
     rest: Vec<String>,
 }
 
-/// Every rule in an `iptables -S <chain>` listing carrying a tag that starts
-/// with `prefix`. Anything else in the chain is left alone.
+/// Every rule in an `iptables -S <chain>` listing that is ours: carrying a
+/// tag that starts with `prefix`, or, on a kernel with no `comment` match,
+/// being exactly one of the three rules this module writes for some range.
+/// Anything else in the chain is left alone.
 fn tagged_rules(listing: &str, chain: &str, prefix: &str) -> Vec<Found> {
+    // `tsunagi-exit:<interface>:`
+    let interface = prefix
+        .strip_prefix("tsunagi-exit:")
+        .and_then(|rest| rest.strip_suffix(':'))
+        .unwrap_or_default();
     listing
         .lines()
         .filter_map(|line| {
@@ -158,7 +177,8 @@ fn tagged_rules(listing: &str, chain: &str, prefix: &str) -> Vec<Found> {
             let tag = rest
                 .windows(2)
                 .find(|pair| pair[0] == "--comment" && pair[1].starts_with(prefix))
-                .map(|pair| pair[1].clone())?;
+                .map(|pair| pair[1].clone())
+                .or_else(|| uncommented_tag(interface, chain, rest))?;
             Some(Found {
                 tag,
                 rest: rest.to_vec(),
@@ -167,6 +187,22 @@ fn tagged_rules(listing: &str, chain: &str, prefix: &str) -> Vec<Found> {
         .collect()
 }
 
+/// The tag a rule would have carried, when it is one of ours written without
+/// a comment: its words are those of the rule for the range it names.
+fn uncommented_tag(interface: &str, chain: &str, rest: &[String]) -> Option<String> {
+    if interface.is_empty() {
+        return None;
+    }
+    let range: Ipv4Range = rest
+        .windows(2)
+        .find(|pair| pair[0] == "-s" || pair[0] == "-d")
+        .and_then(|pair| pair[1].parse().ok())?;
+    offer_rules(interface, &range, false)
+        .iter()
+        .any(|rule| rule.chain == chain && same_rule(&rule.spec, rest))
+        .then(|| exit_tag(interface, &range))
+}
+
 fn delete_args(table: &str, chain: &str, found: &Found) -> Vec<String> {
     locked(
         owned(&["-t", table, "-D", chain])
@@ -236,6 +272,7 @@ fn apply_offer(
     };
     let prefix = exit_tag_prefix(interface);
     let wanted: Vec<String> = ranges.iter().map(|r| exit_tag(interface, r)).collect();
+    let comment = comment_supported(run);
 
     // What is there now, per chain.
     let mut present: Vec<(&str, &str, Vec<Found>)> = Vec::new();
@@ -263,7 +300,7 @@ fn apply_offer(
     ranges
         .iter()
         .map(|range| {
-            let rules = offer_rules(interface, range);
+            let rules = offer_rules(interface, range, comment);
             let tag = exit_tag(interface, range);
             let in_place = rules
                 .iter()
@@ -468,7 +505,7 @@ mod tests {
 
     #[test]
     fn the_rules_masquerade_the_range_out_of_every_other_interface_and_forward_it() {
-        let rules = offer_rules("tsun0", &range());
+        let rules = offer_rules("tsun0", &range(), true);
         assert_eq!(rules.len(), 3);
         assert_eq!(
             words(&rules[0].spec),
@@ -521,6 +558,8 @@ mod tests {
         listing: RefCell<Vec<(String, String)>>,
         missing: bool,
         check_ok: bool,
+        /// A kernel with no `comment` match, as small router builds have.
+        no_comment: bool,
     }
 
     impl Fake {
@@ -530,6 +569,7 @@ mod tests {
                 listing: RefCell::new(Vec::new()),
                 missing: false,
                 check_ok,
+                no_comment: false,
             }
         }
 
@@ -556,6 +596,13 @@ mod tests {
                 stdout,
                 stderr: String::new(),
             };
+            if self.no_comment && line.contains("-m comment") {
+                return Ran::Exited {
+                    success: false,
+                    stdout: String::new(),
+                    stderr: "iptables: No chain/target/match by that name.".to_string(),
+                };
+            }
             if line.contains(" -S ") {
                 let listing: Vec<String> = self
                     .listing
@@ -586,6 +633,44 @@ mod tests {
         }
     }
 
+    #[test]
+    fn a_kernel_without_the_comment_match_gets_the_rules_without_comments() {
+        let mut fake = Fake::new(false);
+        fake.no_comment = true;
+        let outcome = apply_offer(&|args| fake.run(args), "tsun0", &[range()]);
+        assert_eq!(outcome, vec![(range(), RuleOutcome::Applied)]);
+        let inserted = fake.verbs("-I");
+        assert_eq!(inserted.len(), 3, "{inserted:?}");
+        assert!(
+            inserted.iter().all(|call| !call.contains("comment")),
+            "{inserted:?}"
+        );
+    }
+
+    #[test]
+    fn rules_written_without_a_comment_are_ours_and_are_removed_with_them() {
+        // The way iptables prints them: its own option order.
+        let listing = "\
+-P FORWARD ACCEPT
+-A FORWARD -s 10.13.37.0/24 -i tsun0 ! -o tsun0 -j ACCEPT
+-A FORWARD -d 10.13.37.0/24 -o tsun0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+-A FORWARD -i tsun0 -j ACCEPT
+-A FORWARD -s 10.13.37.0/24 -i tsun1 ! -o tsun1 -j ACCEPT
+";
+        let found = tagged_rules(listing, "FORWARD", &exit_tag_prefix("tsun0"));
+        assert_eq!(found.len(), 2, "{found:?}");
+        assert!(
+            found
+                .iter()
+                .all(|rule| rule.tag == exit_tag("tsun0", &range()))
+        );
+        let nat = "-A POSTROUTING -s 10.13.37.0/24 ! -o tsun0 -j MASQUERADE\n";
+        assert_eq!(
+            tagged_rules(nat, "POSTROUTING", &exit_tag_prefix("tsun0")).len(),
+            1
+        );
+    }
+
     #[test]
     fn a_fresh_host_gets_the_three_rules() {
         let fake = Fake::new(false);

crates/tsunagi-core/src/overlay/hostrules/macos.rs

@@ -19,6 +19,10 @@
 
 use crate::BoxFuture;
 
+use std::sync::{Arc, Mutex};
+
+use super::exit::{ExitHostPlan, ExitHostReport, ExitHostRules};
+use super::macos_exit::{apply_plan, clear_all, run_tool};
 use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome};
 
 /// The limited-broadcast destination routed through the overlay interface.
@@ -60,14 +64,19 @@ fn firewall_todo(plan: &BroadcastRulesPlan) -> String {
     )
 }
 
-/// The macOS implementation: `route` for the route, no firewall yet.
+/// The macOS implementation: `route` for the route, no firewall yet, and
+/// `pf` plus `route` for exit nodes (see [`super::macos_exit`]).
 #[derive(Debug, Default)]
-pub struct MacosHostRules;
+pub struct MacosHostRules {
+    /// The reference `pfctl -E` handed out, held while exit-node rules need
+    /// `pf` on and released when they are removed.
+    pub(super) pf_token: Arc<Mutex<Option<String>>>,
+}
 
 impl MacosHostRules {
     /// Creates the host rules.
     pub fn new() -> Self {
-        Self
+        Self::default()
     }
 
     /// Installs the route, replacing one this agent added earlier so the call
@@ -82,11 +91,46 @@ impl MacosHostRules {
     }
 }
 
+impl ExitHostRules for MacosHostRules {
+    fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport> {
+        Box::pin(async move {
+            let job_plan = plan.clone();
+            let token = Arc::clone(&self.pf_token);
+            tokio::task::spawn_blocking(move || apply_plan(&run_tool, &token, &job_plan))
+                .await
+                .unwrap_or_else(|err| {
+                    let failed = RuleOutcome::Failed(format!("the task failed: {err}"));
+                    ExitHostReport {
+                        offer: plan
+                            .offer
+                            .iter()
+                            .map(|range| (*range, failed.clone()))
+                            .collect(),
+                        forwarding: None,
+                        client: plan.client.then_some(failed),
+                    }
+                })
+        })
+    }
+
+    fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
+        Box::pin(async move {
+            let name = interface.to_string();
+            let token = Arc::clone(&self.pf_token);
+            let _ = tokio::task::spawn_blocking(move || clear_all(&run_tool, &token, &name)).await;
+        })
+    }
+}
+
 impl BroadcastHostRules for MacosHostRules {
     fn name(&self) -> &str {
         "route"
     }
 
+    fn exit_rules(&self) -> Option<&dyn ExitHostRules> {
+        Some(self)
+    }
+
     fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
         Box::pin(async move {
             let route = self.set_route(plan.interface.clone()).await;

crates/tsunagi-core/src/overlay/hostrules/macos_exit.rs

@@ -0,0 +1,915 @@
+//! Exit-node host rules on macOS: a `pf` anchor and `route` entries.
+//!
+//! Compiled on every platform so the rule text and the parsers are tested
+//! everywhere; only `MacosHostRules` runs them, and only on macOS.
+//!
+//! # Offering
+//!
+//! Per offered range, in the anchor `com.apple/tsunagi-exit-<if>`:
+//!
+//! ```text
+//! nat on <egress> inet from <range> to any -> (<egress>)
+//! pass in quick on <if> inet from <range> to ! <range>
+//! ```
+//!
+//! The anchor lives under `com.apple/` on purpose: the stock `/etc/pf.conf`
+//! evaluates `nat-anchor "com.apple/*"` and `anchor "com.apple/*"`, so the
+//! rules take effect without editing the user's ruleset. `pf` cannot say
+//! "every interface but one" in a translation address, so the egress is the
+//! interface of the default route **when the rules are applied**; after the
+//! default route moves (Wi-Fi to Ethernet) the rules are re-applied by
+//! switching the exit node off and on, or restarting the agent.
+//!
+//! The kernel's `net.inet.ip.forwarding` is only read, never written: it
+//! changes how the whole host behaves, which is the owner's decision.
+//!
+//! # Using
+//!
+//! macOS has no `uidrange` policy routing, and the agent is root, so the
+//! Linux scheme does not carry over. Instead:
+//!
+//! ```text
+//! route add -net 0.0.0.0/1   -interface <if>
+//! route add -net 128.0.0.0/1 -interface <if>
+//! pass out quick on <if> route-to (<egress> <gateway>) inet proto { tcp udp }
+//!     from any to ! <overlay ranges> user <agent uid>
+//! nat on <egress> inet from (<if>) to any -> (<egress>)
+//! ```
+//!
+//! The two halves outrank the default route without replacing it, and every
+//! more specific route (LAN, another VPN) keeps working. The `route-to` rule
+//! sends the *agent's own* traffic — QUIC, relay, DHT, which would otherwise
+//! loop into the tunnel it carries — back out the physical interface, and the
+//! translation gives it that interface's address, since an unbound socket
+//! picked the overlay address when the route pointed at the overlay. Like the
+//! Linux exemption this is by user id, so anything else running as root also
+//! bypasses the tunnel.
+//!
+//! IPv6 is blocked while an exit node is in use, as on Linux: `::/1` and
+//! `8000::/1` are rejecting routes, so applications fall back to IPv4 at once
+//! instead of leaking.
+//!
+//! Every object is derived from local state and the interface the agent owns;
+//! none of it comes from a peer.
+
+use std::net::Ipv4Addr;
+use std::sync::Mutex;
+
+use crate::state::Ipv4Range;
+
+use super::RuleOutcome;
+use super::exit::{ExitHostPlan, ExitHostReport};
+
+/// The programs this module runs, always by absolute path.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(super) enum Tool {
+    Route,
+    Pfctl,
+    Sysctl,
+    Id,
+}
+
+impl Tool {
+    fn path(self) -> &'static str {
+        match self {
+            Self::Route => "/sbin/route",
+            Self::Pfctl => "/sbin/pfctl",
+            Self::Sysctl => "/usr/sbin/sysctl",
+            Self::Id => "/usr/bin/id",
+        }
+    }
+}
+
+/// What running a tool came to.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub(super) enum Ran {
+    /// The program is not there.
+    Missing,
+    /// It ran.
+    Exited {
+        success: bool,
+        stdout: String,
+        stderr: String,
+    },
+}
+
+/// Runs a tool with optional standard input; injectable so the logic is
+/// tested without touching the host.
+pub(super) type Run<'a> = &'a dyn Fn(Tool, &[String], Option<&str>) -> Ran;
+
+/// Runs a real tool. Blocking: callers use `spawn_blocking`.
+pub(super) fn run_tool(tool: Tool, args: &[String], stdin: Option<&str>) -> Ran {
+    use std::io::Write;
+    use std::process::{Command, Stdio};
+
+    let mut command = Command::new(tool.path());
+    command
+        .args(args)
+        .stdin(if stdin.is_some() {
+            Stdio::piped()
+        } else {
+            Stdio::null()
+        })
+        .stdout(Stdio::piped())
+        .stderr(Stdio::piped());
+    let mut child = match command.spawn() {
+        Ok(child) => child,
+        Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ran::Missing,
+        Err(err) => {
+            return Ran::Exited {
+                success: false,
+                stdout: String::new(),
+                stderr: format!("could not run {}: {err}", tool.path()),
+            };
+        }
+    };
+    if let (Some(text), Some(mut pipe)) = (stdin, child.stdin.take()) {
+        // A write error shows up as the tool's own failure below.
+        let _ = pipe.write_all(text.as_bytes());
+    }
+    match child.wait_with_output() {
+        Ok(output) => Ran::Exited {
+            success: output.status.success(),
+            stdout: String::from_utf8_lossy(&output.stdout).into_owned(),
+            stderr: String::from_utf8_lossy(&output.stderr).into_owned(),
+        },
+        Err(err) => Ran::Exited {
+            success: false,
+            stdout: String::new(),
+            stderr: format!("could not run {}: {err}", tool.path()),
+        },
+    }
+}
+
+fn owned(words: &[&str]) -> Vec<String> {
+    words.iter().map(|word| word.to_string()).collect()
+}
+
+/// A name that is safe in a `pf` rule, an anchor path and a table name.
+fn safe_name(name: &str) -> bool {
+    !name.is_empty() && name.len() <= 32 && name.chars().all(|c| c.is_ascii_alphanumeric())
+}
+
+/// The anchor holding every rule of one interface.
+fn anchor_name(interface: &str) -> Result<String, String> {
+    if safe_name(interface) {
+        Ok(format!("com.apple/tsunagi-exit-{interface}"))
+    } else {
+        Err(format!(
+            "`{interface}` is not an interface name this agent can write a pf rule for"
+        ))
+    }
+}
+
+/// The default route as `route -n get default` reports it.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub(super) struct DefaultRoute {
+    pub interface: String,
+    pub gateway: Option<Ipv4Addr>,
+}
+
+/// Reads `interface:` and `gateway:` out of `route -n get default`.
+fn parse_default_route(text: &str) -> Option<DefaultRoute> {
+    let field = |name: &str| {
+        text.lines().find_map(|line| {
+            let (key, value) = line.split_once(':')?;
+            (key.trim() == name).then(|| value.trim().to_string())
+        })
+    };
+    let interface = field("interface").filter(|name| safe_name(name))?;
+    let gateway = field("gateway").and_then(|value| value.parse().ok());
+    Some(DefaultRoute { interface, gateway })
+}
+
+fn failure(stderr: &str, stdout: &str) -> String {
+    let text = if stderr.trim().is_empty() {
+        stdout.trim()
+    } else {
+        stderr.trim()
+    };
+    if text.is_empty() {
+        "the command failed without saying why".to_string()
+    } else {
+        text.to_string()
+    }
+}
+
+/// Runs a tool and turns a failure into a sentence a person can act on.
+fn run_ok(
+    run: Run<'_>,
+    tool: Tool,
+    args: &[String],
+    stdin: Option<&str>,
+) -> Result<String, String> {
+    match run(tool, args, stdin) {
+        Ran::Missing => Err(format!("{} not found", tool.path())),
+        Ran::Exited {
+            success: true,
+            stdout,
+            ..
+        } => Ok(stdout),
+        Ran::Exited { stderr, stdout, .. } => Err(format!(
+            "{} {} failed: {}",
+            tool.path(),
+            args.join(" "),
+            failure(&stderr, &stdout)
+        )),
+    }
+}
+
+/// The route traffic leaves by now, which is the egress for translation and
+/// for the agent's own sockets. Read *before* the overlay routes are added.
+fn default_route(run: Run<'_>, overlay: &str) -> Result<DefaultRoute, String> {
+    let out = run_ok(run, Tool::Route, &owned(&["-n", "get", "default"]), None)
+        .map_err(|why| format!("cannot find the default route: {why}"))?;
+    let route = parse_default_route(&out)
+        .ok_or_else(|| "the host has no default route to send traffic out of".to_string())?;
+    if route.interface == overlay {
+        return Err(format!(
+            "the default route already goes through `{overlay}`"
+        ));
+    }
+    Ok(route)
+}
+
+/// Whether the kernel forwards IPv4. Read only.
+fn forwarding_state(run: Run<'_>) -> Option<bool> {
+    let out = run_ok(
+        run,
+        Tool::Sysctl,
+        &owned(&["-n", "net.inet.ip.forwarding"]),
+        None,
+    )
+    .ok()?;
+    match out.trim() {
+        "1" => Some(true),
+        "0" => Some(false),
+        _ => None,
+    }
+}
+
+/// The effective user id of this process, from `id -u`.
+fn agent_uid(run: Run<'_>) -> Result<u32, String> {
+    run_ok(run, Tool::Id, &owned(&["-u"]), None)
+        .ok()
+        .and_then(|out| out.trim().parse().ok())
+        .ok_or_else(|| "cannot tell which user this agent runs as".to_string())
+}
+
+/// What the using side adds to the rules.
+#[derive(Debug, Clone, PartialEq, Eq)]
+struct ClientRules<'a> {
+    uid: u32,
+    /// Every overlay range: the agent's own traffic *to* those still goes
+    /// through the interface.
+    overlay: &'a [Ipv4Range],
+}
+
+/// The text of the anchor: translation first, filtering after, which is the
+/// order `pf` insists on.
+fn pf_rules(
+    interface: &str,
+    egress: &DefaultRoute,
+    offer: &[Ipv4Range],
+    client: Option<&ClientRules<'_>>,
+) -> String {
+    let out = &egress.interface;
+    let mut nat = Vec::new();
+    let mut filter = Vec::new();
+    for range in offer {
+        nat.push(format!("nat on {out} inet from {range} to any -> ({out})"));
+        filter.push(format!(
+            "pass in quick on {interface} inet from {range} to ! {range}"
+        ));
+    }
+    let mut tables = Vec::new();
+    if let Some(client) = client {
+        nat.push(format!(
+            "nat on {out} inet from ({interface}) to any -> ({out})"
+        ));
+        let target = if client.overlay.is_empty() {
+            "any".to_string()
+        } else {
+            let ranges: Vec<String> = client.overlay.iter().map(|r| r.to_string()).collect();
+            // A table, because `! { a, b }` expands to one rule per element
+            // in older `pf`, which matches everything.
+            tables.push(format!(
+                "table <tsunagi_overlay_{interface}> const {{ {} }}",
+                ranges.join(", ")
+            ));
+            format!("! <tsunagi_overlay_{interface}>")
+        };
+        let via = match egress.gateway {
+            Some(gateway) => format!("({out} {gateway})"),
+            None => out.clone(),
+        };
+        filter.push(format!(
+            "pass out quick on {interface} route-to {via} inet proto {{ tcp udp }} \
+             from any to {target} user {}",
+            client.uid
+        ));
+    }
+    tables
+        .into_iter()
+        .chain(nat)
+        .chain(filter)
+        .collect::<Vec<_>>()
+        .join("\n")
+        + "\n"
+}
+
+/// Turns `pf` on if nobody has, holding the reference `pfctl` hands out so it
+/// can be released when the agent is done. One reference at most.
+fn acquire_pf(run: Run<'_>, slot: &Mutex<Option<String>>) -> Result<(), String> {
+    if lock(slot).is_some() {
+        return Ok(());
+    }
+    match run(Tool::Pfctl, &owned(&["-E"]), None) {
+        Ran::Missing => Err(format!("{} not found", Tool::Pfctl.path())),
+        Ran::Exited {
+            success,
+            stdout,
+            stderr,
+        } => {
+            // `pfctl -E` reports `Token : <n>` on stderr, and also says so
+            // when pf is already enabled.
+            let token = stdout.lines().chain(stderr.lines()).find_map(parse_token);
+            if !success && token.is_none() {
+                return Err(format!("cannot enable pf: {}", failure(&stderr, &stdout)));
+            }
+            *lock(slot) = token;
+            Ok(())
+        }
+    }
+}
+
+fn parse_token(line: &str) -> Option<String> {
+    let (key, value) = line.split_once(':')?;
+    let value = value.trim();
+    (key.trim().eq_ignore_ascii_case("token")
+        && !value.is_empty()
+        && value.chars().all(|c| c.is_ascii_digit()))
+    .then(|| value.to_string())
+}
+
+fn release_pf(run: Run<'_>, slot: &Mutex<Option<String>>) {
+    if let Some(token) = lock(slot).take()
+        && let Err(why) = run_ok(run, Tool::Pfctl, &["-X".to_string(), token], None)
+    {
+        tracing::debug!(%why, "cannot release the pf reference");
+    }
+}
+
+fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
+    match mutex.lock() {
+        Ok(guard) => guard,
+        Err(poisoned) => poisoned.into_inner(),
+    }
+}
+
+fn flush_args(anchor: &str) -> Vec<String> {
+    owned(&["-q", "-a", anchor, "-F", "all"])
+}
+
+fn load_args(anchor: &str) -> Vec<String> {
+    owned(&["-q", "-a", anchor, "-f", "-"])
+}
+
+/// The two halves that out-rank the default route.
+const HALVES: [&str; 2] = ["0.0.0.0/1", "128.0.0.0/1"];
+/// The IPv6 halves, made unreachable.
+const HALVES_V6: [&str; 2] = ["::/1", "8000::/1"];
+
+fn route_add_args(half: &str, interface: &str) -> Vec<String> {
+    owned(&["-q", "-n", "add", "-net", half, "-interface", interface])
+}
+
+fn route_del_args(half: &str, interface: &str) -> Vec<String> {
+    owned(&["-q", "-n", "delete", "-net", half, "-interface", interface])
+}
+
+fn route_add_v6_args(half: &str) -> Vec<String> {
+    owned(&["-q", "-n", "add", "-inet6", "-net", half, "::1", "-reject"])
+}
+
+fn route_del_v6_args(half: &str) -> Vec<String> {
+    owned(&["-q", "-n", "delete", "-inet6", "-net", half, "::1"])
+}
+
+/// Takes away the overlay default routes. Best effort: what is not there is
+/// the state wanted.
+fn clear_routes(run: Run<'_>, interface: &str) {
+    for half in HALVES {
+        let _ = run(Tool::Route, &route_del_args(half, interface), None);
+    }
+    for half in HALVES_V6 {
+        let _ = run(Tool::Route, &route_del_v6_args(half), None);
+    }
+}
+
+fn set_routes(run: Run<'_>, interface: &str) -> Result<(), String> {
+    clear_routes(run, interface);
+    for half in HALVES {
+        if let Err(why) = run_ok(run, Tool::Route, &route_add_args(half, interface), None) {
+            clear_routes(run, interface);
+            return Err(why);
+        }
+    }
+    for half in HALVES_V6 {
+        // A host without IPv6 has nothing to block; not a failure.
+        if let Err(why) = run_ok(run, Tool::Route, &route_add_v6_args(half), None) {
+            tracing::debug!(%why, "cannot block IPv6 for the exit node");
+        }
+    }
+    Ok(())
+}
+
+/// Makes the host match the plan.
+pub(super) fn apply_plan(
+    run: Run<'_>,
+    token: &Mutex<Option<String>>,
+    plan: &ExitHostPlan,
+) -> ExitHostReport {
+    let forwarding = if plan.offer.is_empty() {
+        None
+    } else {
+        forwarding_state(run)
+    };
+    let outcome = apply_rules(run, token, plan);
+    let verdict = |ok: bool, why: &str| {
+        if ok {
+            RuleOutcome::Applied
+        } else {
+            RuleOutcome::Failed(why.to_string())
+        }
+    };
+    match outcome {
+        Ok(()) => ExitHostReport {
+            offer: plan
+                .offer
+                .iter()
+                .map(|range| (*range, RuleOutcome::Applied))
+                .collect(),
+            forwarding,
+            client: plan.client.then_some(RuleOutcome::Applied),
+        },
+        Err(why) => ExitHostReport {
+            offer: plan
+                .offer
+                .iter()
+                .map(|range| (*range, verdict(false, &why)))
+                .collect(),
+            forwarding,
+            client: plan.client.then(|| verdict(false, &why)),
+        },
+    }
+}
+
+fn apply_rules(
+    run: Run<'_>,
+    token: &Mutex<Option<String>>,
+    plan: &ExitHostPlan,
+) -> Result<(), String> {
+    let interface = plan.interface.as_str();
+    let anchor = anchor_name(interface)?;
+    let egress = default_route(run, interface)?;
+    let uid = if plan.client {
+        Some(agent_uid(run)?)
+    } else {
+        None
+    };
+    let client = uid.map(|uid| ClientRules {
+        uid,
+        overlay: &plan.overlay,
+    });
+    let rules = pf_rules(interface, &egress, &plan.offer, client.as_ref());
+
+    acquire_pf(run, token)?;
+    run_ok(run, Tool::Pfctl, &load_args(&anchor), Some(&rules))
+        .map_err(|why| format!("cannot load the pf rules: {why}"))?;
+    // The routes come after the rules that keep the agent's own traffic off
+    // them: the other order would loop for as long as it takes.
+    if plan.client {
+        set_routes(run, interface)?;
+    } else {
+        clear_routes(run, interface);
+    }
+    Ok(())
+}
+
+/// Removes everything this agent put there for the interface.
+pub(super) fn clear_all(run: Run<'_>, token: &Mutex<Option<String>>, interface: &str) {
+    clear_routes(run, interface);
+    if let Ok(anchor) = anchor_name(interface) {
+        // Flushing an anchor nothing was ever loaded into is not an error
+        // worth reporting.
+        if let Err(why) = run_ok(run, Tool::Pfctl, &flush_args(&anchor), None) {
+            tracing::debug!(%why, "cannot flush the exit node pf anchor");
+        }
+    }
+    release_pf(run, token);
+}
+
+#[cfg(test)]
+mod tests {
+    #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
+
+    use std::cell::RefCell;
+
+    use super::*;
+
+    fn range() -> Ipv4Range {
+        "10.13.37.0/24".parse().unwrap()
+    }
+
+    const DEFAULT_ROUTE: &str = "   route to: default\n\
+destination: default\n\
+       mask: default\n\
+    gateway: 192.168.1.1\n\
+  interface: en0\n\
+      flags: <UP,GATEWAY,DONE,STATIC,PRCLONING,GLOBAL>\n";
+
+    fn egress() -> DefaultRoute {
+        DefaultRoute {
+            interface: "en0".into(),
+            gateway: Some(Ipv4Addr::new(192, 168, 1, 1)),
+        }
+    }
+
+    /// A scripted host that records every call.
+    struct Fake {
+        calls: RefCell<Vec<String>>,
+        stdins: RefCell<Vec<String>>,
+        default_route: Option<String>,
+        forwarding: &'static str,
+        pf_missing: bool,
+        pf_load_fails: bool,
+        route_add_fails: bool,
+        pf_enable_output: &'static str,
+    }
+
+    impl Fake {
+        fn new() -> Self {
+            Self {
+                calls: RefCell::new(Vec::new()),
+                stdins: RefCell::new(Vec::new()),
+                default_route: Some(DEFAULT_ROUTE.to_string()),
+                forwarding: "1\n",
+                pf_missing: false,
+                pf_load_fails: false,
+                route_add_fails: false,
+                pf_enable_output: "pf enabled\nToken : 4242\n",
+            }
+        }
+
+        fn run(&self, tool: Tool, args: &[String], stdin: Option<&str>) -> Ran {
+            let line = format!("{:?} {}", tool, args.join(" "));
+            self.calls.borrow_mut().push(line);
+            if let Some(text) = stdin {
+                self.stdins.borrow_mut().push(text.to_string());
+            }
+            let ok = |stdout: &str, stderr: &str| Ran::Exited {
+                success: true,
+                stdout: stdout.to_string(),
+                stderr: stderr.to_string(),
+            };
+            let fail = |why: &str| Ran::Exited {
+                success: false,
+                stdout: String::new(),
+                stderr: why.to_string(),
+            };
+            match tool {
+                Tool::Id => ok("0\n", ""),
+                Tool::Sysctl => ok(self.forwarding, ""),
+                Tool::Route if args.first().map(String::as_str) == Some("-n") => {
+                    match &self.default_route {
+                        Some(text) => ok(text, ""),
+                        None => fail("route: writing to routing socket: not in table"),
+                    }
+                }
+                Tool::Route if args.contains(&"add".to_string()) && self.route_add_fails => {
+                    fail("route: writing to routing socket: File exists")
+                }
+                Tool::Route => ok("", ""),
+                Tool::Pfctl if self.pf_missing => Ran::Missing,
+                Tool::Pfctl if args == ["-E"] => ok("", self.pf_enable_output),
+                Tool::Pfctl if args.contains(&"-f".to_string()) && self.pf_load_fails => {
+                    fail("pfctl: Syntax error")
+                }
+                Tool::Pfctl => ok("", ""),
+            }
+        }
+
+        fn calls(&self) -> Vec<String> {
+            self.calls.borrow().clone()
+        }
+
+        fn position(&self, needle: &str) -> Option<usize> {
+            self.calls.borrow().iter().position(|c| c.contains(needle))
+        }
+
+        fn count(&self, needle: &str) -> usize {
+            self.calls
+                .borrow()
+                .iter()
+                .filter(|c| c.contains(needle))
+                .count()
+        }
+    }
+
+    fn plan(offer: bool, client: bool) -> ExitHostPlan {
+        ExitHostPlan {
+            interface: "utun5".into(),
+            offer: if offer { vec![range()] } else { Vec::new() },
+            client,
+            overlay: vec![range()],
+            ..Default::default()
+        }
+    }
+
+    #[test]
+    fn the_default_route_is_read_from_route_output() {
+        assert_eq!(parse_default_route(DEFAULT_ROUTE), Some(egress()));
+        // A point-to-point default has no gateway line to use.
+        let ppp = "destination: default\n  interface: ppp0\n";
+        assert_eq!(
+            parse_default_route(ppp),
+            Some(DefaultRoute {
+                interface: "ppp0".into(),
+                gateway: None
+            })
+        );
+        assert_eq!(parse_default_route("route: not in table"), None);
+        // An interface name that is not plain letters and digits never ends
+        // up in a rule.
+        assert_eq!(parse_default_route("interface: en0; pass all\n"), None);
+    }
+
+    #[test]
+    fn the_anchor_is_under_com_apple_and_only_for_plain_names() {
+        assert_eq!(
+            anchor_name("utun5").unwrap(),
+            "com.apple/tsunagi-exit-utun5"
+        );
+        assert!(anchor_name("").is_err());
+        assert!(anchor_name("utun5 pass all").is_err());
+        assert!(anchor_name("../x").is_err());
+    }
+
+    #[test]
+    fn offering_translates_the_range_and_admits_it_in() {
+        let text = pf_rules("utun5", &egress(), &[range()], None);
+        assert_eq!(
+            text,
+            "nat on en0 inet from 10.13.37.0/24 to any -> (en0)\n\
+             pass in quick on utun5 inet from 10.13.37.0/24 to ! 10.13.37.0/24\n"
+        );
+        // Translation precedes filtering, as pf requires.
+        assert!(text.find("nat on").unwrap() < text.find("pass in").unwrap());
+    }
+
+    #[test]
+    fn using_sends_the_agents_own_traffic_back_out_the_physical_interface() {
+        let other: Ipv4Range = "10.99.0.0/16".parse().unwrap();
+        let overlay = [range(), other];
+        let client = ClientRules {
+            uid: 0,
+            overlay: &overlay,
+        };
+        let text = pf_rules("utun5", &egress(), &[], Some(&client));
+        assert!(
+            text.starts_with(
+                "table <tsunagi_overlay_utun5> const { 10.13.37.0/24, 10.99.0.0/16 }\n"
+            ),
+            "{text}"
+        );
+        assert!(
+            text.contains("nat on en0 inet from (utun5) to any -> (en0)\n"),
+            "{text}"
+        );
+        assert!(
+            text.contains(
+                "pass out quick on utun5 route-to (en0 192.168.1.1) inet proto { tcp udp } \
+                 from any to ! <tsunagi_overlay_utun5> user 0\n"
+            ),
+            "{text}"
+        );
+        // Only the agent's user, and only what is not the overlay.
+        assert!(!text.contains("from any to any"));
+
+        let direct = DefaultRoute {
+            interface: "ppp0".into(),
+            gateway: None,
+        };
+        let text = pf_rules(
+            "utun5",
+            &direct,
+            &[],
+            Some(&ClientRules {
+                uid: 501,
+                overlay: &[],
+            }),
+        );
+        assert!(text.contains("route-to ppp0 inet"), "{text}");
+        assert!(text.contains("to any user 501"), "{text}");
+        assert!(!text.contains("table"), "{text}");
+    }
+
+    #[test]
+    fn the_overlay_default_routes_are_halves_and_ipv6_is_rejected() {
+        assert_eq!(
+            route_add_args("0.0.0.0/1", "utun5").join(" "),
+            "-q -n add -net 0.0.0.0/1 -interface utun5"
+        );
+        assert_eq!(
+            route_del_args("128.0.0.0/1", "utun5").join(" "),
+            "-q -n delete -net 128.0.0.0/1 -interface utun5"
+        );
+        assert_eq!(
+            route_add_v6_args("8000::/1").join(" "),
+            "-q -n add -inet6 -net 8000::/1 ::1 -reject"
+        );
+        assert_eq!(
+            route_del_v6_args("::/1").join(" "),
+            "-q -n delete -inet6 -net ::/1 ::1"
+        );
+    }
+
+    #[test]
+    fn the_pf_token_is_found_in_either_stream() {
+        assert_eq!(parse_token("Token : 1234"), Some("1234".into()));
+        assert_eq!(parse_token("token: 9"), Some("9".into()));
+        assert_eq!(parse_token("pf enabled"), None);
+        assert_eq!(parse_token("Token : x; rm"), None);
+    }
+
+    #[test]
+    fn offering_loads_the_anchor_and_reads_but_never_writes_forwarding() {
+        let fake = Fake::new();
+        let token = Mutex::new(None);
+        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
+        assert_eq!(report.offer, vec![(range(), RuleOutcome::Applied)]);
+        assert_eq!(report.forwarding, Some(true));
+        assert_eq!(report.client, None);
+        let calls = fake.calls();
+        assert!(
+            calls
+                .iter()
+                .any(|c| c == "Pfctl -q -a com.apple/tsunagi-exit-utun5 -f -"),
+            "{calls:?}"
+        );
+        assert_eq!(
+            fake.stdins.borrow()[0],
+            "nat on en0 inet from 10.13.37.0/24 to any -> (en0)\n\
+             pass in quick on utun5 inet from 10.13.37.0/24 to ! 10.13.37.0/24\n"
+        );
+        // sysctl is only ever asked, never told.
+        assert!(
+            calls
+                .iter()
+                .filter(|c| c.starts_with("Sysctl"))
+                .all(|c| c == "Sysctl -n net.inet.ip.forwarding"),
+            "{calls:?}"
+        );
+        // No client routes were added for an offer alone.
+        assert_eq!(fake.count(" add "), 0);
+        assert_eq!(token.lock().unwrap().as_deref(), Some("4242"));
+    }
+
+    #[test]
+    fn forwarding_off_is_reported_and_unreadable_is_none() {
+        let mut fake = Fake::new();
+        fake.forwarding = "0\n";
+        let token = Mutex::new(None);
+        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
+        assert_eq!(report.forwarding, Some(false));
+        fake.forwarding = "junk";
+        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
+        assert_eq!(report.forwarding, None);
+    }
+
+    #[test]
+    fn the_client_rules_are_loaded_before_the_routes_that_need_them() {
+        let fake = Fake::new();
+        let token = Mutex::new(None);
+        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(false, true));
+        assert_eq!(report.client, Some(RuleOutcome::Applied));
+        assert!(report.offer.is_empty());
+        let load = fake.position("-f -").unwrap();
+        let add = fake.position(" add -net 0.0.0.0/1").unwrap();
+        assert!(load < add, "{:?}", fake.calls());
+        assert!(fake.position("add -net 128.0.0.0/1").is_some());
+        assert!(fake.position("-inet6 -net ::/1 ::1 -reject").is_some());
+        assert!(fake.stdins.borrow()[0].contains("user 0"));
+        // The default route was read before any overlay route existed.
+        assert!(fake.position("-n get default").unwrap() < add);
+    }
+
+    #[test]
+    fn no_routes_are_added_when_the_rules_did_not_load() {
+        let mut fake = Fake::new();
+        fake.pf_load_fails = true;
+        let token = Mutex::new(None);
+        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, true));
+        let Some(RuleOutcome::Failed(why)) = report.client else {
+            panic!("{report:?}");
+        };
+        assert!(why.contains("cannot load the pf rules"), "{why}");
+        assert!(why.contains("Syntax error"), "{why}");
+        assert!(matches!(report.offer[0].1, RuleOutcome::Failed(_)));
+        // Nothing sent the host's traffic into a tunnel it could not exempt
+        // the agent from.
+        assert_eq!(fake.count(" add "), 0);
+    }
+
+    #[test]
+    fn a_failing_route_is_a_failure_and_leaves_nothing_behind() {
+        let mut fake = Fake::new();
+        fake.route_add_fails = true;
+        let token = Mutex::new(None);
+        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(false, true));
+        let Some(RuleOutcome::Failed(why)) = report.client else {
+            panic!("{report:?}");
+        };
+        assert!(why.contains("File exists"), "{why}");
+        // The cleanup after the failed add removed whatever did land.
+        let last_add = fake
+            .calls()
+            .iter()
+            .rposition(|c| c.contains(" add "))
+            .unwrap();
+        assert!(
+            fake.calls()[last_add..]
+                .iter()
+                .any(|c| c.contains("delete"))
+        );
+    }
+
+    #[test]
+    fn a_missing_pfctl_or_default_route_says_what_is_wrong() {
+        let mut fake = Fake::new();
+        fake.pf_missing = true;
+        let token = Mutex::new(None);
+        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
+        let RuleOutcome::Failed(why) = &report.offer[0].1 else {
+            panic!("{report:?}");
+        };
+        assert!(why.contains("/sbin/pfctl not found"), "{why}");
+
+        let mut fake = Fake::new();
+        fake.default_route = None;
+        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
+        let RuleOutcome::Failed(why) = &report.offer[0].1 else {
+            panic!("{report:?}");
+        };
+        assert!(why.contains("default route"), "{why}");
+        assert_eq!(fake.count("-f -"), 0);
+    }
+
+    #[test]
+    fn a_default_route_through_the_overlay_is_refused() {
+        let mut fake = Fake::new();
+        fake.default_route = Some("gateway: 10.13.37.1\ninterface: utun5\n".into());
+        let token = Mutex::new(None);
+        let report = apply_plan(&|t, a, s| fake.run(t, a, s), &token, &plan(true, false));
+        let RuleOutcome::Failed(why) = &report.offer[0].1 else {
+            panic!("{report:?}");
+        };
+        assert!(why.contains("already goes through"), "{why}");
+    }
+
+    #[test]
+    fn pf_is_enabled_once_and_released_once_on_clear() {
+        let fake = Fake::new();
+        let token = Mutex::new(None);
+        let run = |t, a: &[String], s: Option<&str>| fake.run(t, a, s);
+        apply_plan(&run, &token, &plan(true, false));
+        apply_plan(&run, &token, &plan(true, true));
+        assert_eq!(fake.count("Pfctl -E"), 1, "{:?}", fake.calls());
+
+        clear_all(&run, &token, "utun5");
+        assert_eq!(fake.count("Pfctl -X 4242"), 1);
+        assert!(
+            fake.position("-a com.apple/tsunagi-exit-utun5 -F all")
+                .is_some()
+        );
+        assert!(
+            fake.position("delete -net 0.0.0.0/1 -interface utun5")
+                .is_some()
+        );
+        assert_eq!(*token.lock().unwrap(), None);
+        // A second clear has nothing to release and still succeeds.
+        clear_all(&run, &token, "utun5");
+        assert_eq!(fake.count("Pfctl -X"), 1);
+    }
+
+    #[test]
+    fn dropping_the_client_removes_its_routes() {
+        let fake = Fake::new();
+        let token = Mutex::new(None);
+        let run = |t, a: &[String], s: Option<&str>| fake.run(t, a, s);
+        apply_plan(&run, &token, &plan(true, false));
+        assert!(fake.position("delete -net 0.0.0.0/1").is_some());
+        assert_eq!(fake.count(" add "), 0);
+    }
+}

crates/tsunagi-core/src/overlay/hostrules/mod.rs

@@ -61,11 +61,16 @@ pub use linux::LinuxHostRules;
 // Compiled everywhere so the scripts it builds are tested everywhere; only
 // the type that runs them is exported on Windows.
 mod windows;
+#[cfg_attr(not(target_os = "windows"), allow(dead_code))]
+mod windows_exit;
 #[cfg(all(feature = "tun-device", target_os = "windows"))]
 pub use windows::WindowsHostRules;
 
+// Compiled everywhere so its rule text and parsers are tested everywhere.
 #[cfg(all(feature = "tun-device", target_os = "macos"))]
 mod macos;
+#[cfg_attr(not(target_os = "macos"), allow(dead_code))]
+mod macos_exit;
 #[cfg(all(feature = "tun-device", target_os = "macos"))]
 pub use macos::MacosHostRules;
 

crates/tsunagi-core/src/overlay/hostrules/windows.rs

@@ -29,6 +29,8 @@
 
 use crate::BoxFuture;
 
+use super::exit::{ExitHostPlan, ExitHostReport, ExitHostRules};
+use super::windows_exit;
 use super::{BroadcastHostRules, BroadcastRulesPlan, BroadcastRulesReport, RuleOutcome, tag_for};
 
 /// A PowerShell single-quoted literal, with any embedded quote doubled.
@@ -113,6 +115,11 @@ fn powershell_path() -> std::path::PathBuf {
 
 /// Runs a script and returns what it said on failure.
 fn run_powershell(script: &str) -> Result<(), String> {
+    run_powershell_output(script).map(|_| ())
+}
+
+/// Runs a script and returns what it printed, or what it said on failure.
+fn run_powershell_output(script: &str) -> Result<String, String> {
     let output = std::process::Command::new(powershell_path())
         .args([
             "-NoProfile",
@@ -125,7 +132,7 @@ fn run_powershell(script: &str) -> Result<(), String> {
         .output()
         .map_err(|err| format!("could not run PowerShell: {err}"))?;
     if output.status.success() {
-        return Ok(());
+        return Ok(String::from_utf8_lossy(&output.stdout).into_owned());
     }
     let stderr = String::from_utf8_lossy(&output.stderr);
     let text = if stderr.trim().is_empty() {
@@ -155,11 +162,73 @@ impl WindowsHostRules {
     }
 }
 
+/// The IPv4 addresses a host name resolves to; empty when it does not.
+fn resolve_host(host: &str) -> Vec<std::net::Ipv4Addr> {
+    use std::net::{IpAddr, ToSocketAddrs};
+    (host, 443)
+        .to_socket_addrs()
+        .map(|addrs| {
+            addrs
+                .filter_map(|addr| match addr.ip() {
+                    IpAddr::V4(ip) => Some(ip),
+                    IpAddr::V6(_) => None,
+                })
+                .collect()
+        })
+        .unwrap_or_default()
+}
+
+impl ExitHostRules for WindowsHostRules {
+    fn needs_bypass(&self) -> bool {
+        true
+    }
+
+    fn apply<'a>(&'a self, plan: &'a ExitHostPlan) -> BoxFuture<'a, ExitHostReport> {
+        Box::pin(async move {
+            let job = plan.clone();
+            tokio::task::spawn_blocking(move || {
+                windows_exit::apply_plan(
+                    &|script| run_powershell_output(script),
+                    &resolve_host,
+                    &job,
+                )
+            })
+            .await
+            .unwrap_or_else(|err| {
+                let failed = RuleOutcome::Failed(format!("the task failed: {err}"));
+                ExitHostReport {
+                    offer: plan
+                        .offer
+                        .iter()
+                        .map(|range| (*range, failed.clone()))
+                        .collect(),
+                    forwarding: None,
+                    client: plan.client.then_some(failed),
+                }
+            })
+        })
+    }
+
+    fn clear<'a>(&'a self, interface: &'a str) -> BoxFuture<'a, ()> {
+        Box::pin(async move {
+            let name = interface.to_string();
+            let _ = tokio::task::spawn_blocking(move || {
+                windows_exit::clear_all(&|script| run_powershell_output(script), &name)
+            })
+            .await;
+        })
+    }
+}
+
 impl BroadcastHostRules for WindowsHostRules {
     fn name(&self) -> &str {
         "windows-netsecurity"
     }
 
+    fn exit_rules(&self) -> Option<&dyn ExitHostRules> {
+        Some(self)
+    }
+
     fn apply<'a>(&'a self, plan: &'a BroadcastRulesPlan) -> BoxFuture<'a, BroadcastRulesReport> {
         Box::pin(async move {
             let route = step("route", route_script(plan)).await;

crates/tsunagi-core/src/overlay/hostrules/windows_exit.rs

@@ -0,0 +1,603 @@
+//! Exit-node host rules on Windows: WinNAT through PowerShell.
+//!
+//! Compiled on every platform so the scripts and the plan logic are tested
+//! everywhere; only `WindowsHostRules` runs them, and only on Windows. It
+//! could not be run against a real Windows host while it was written.
+//!
+//! # Offering
+//!
+//! One NAT object per offered range, named with the usual tag
+//! `tsunagi-exit:<if>:<range>`:
+//!
+//! ```text
+//! New-NetNat -Name <tag> -InternalIPInterfaceAddressPrefix <range>
+//! ```
+//!
+//! WinNAT translates that range out of whichever interface the traffic leaves
+//! by, which is the Windows counterpart of the Linux masquerade. Windows
+//! forwards per interface, and the egress is not known here, so forwarding is
+//! only *read* for the overlay interface; turning it on for the interfaces
+//! involved is the owner's decision (`Set-NetIPInterface -Forwarding Enabled`).
+//! Some Windows versions allow a single NAT per host: a second offered range
+//! then reports its own failure.
+//!
+//! # Using
+//!
+//! Windows has no per-user or per-process routing, so the Linux scheme (an
+//! exemption for the agent's user) does not carry over. Instead the agent's
+//! own underlay traffic is exempted by *destination*:
+//!
+//! ```text
+//! New-NetRoute 0.0.0.0/1   on the overlay interface, metric 1
+//! New-NetRoute 128.0.0.0/1 on the overlay interface, metric 1
+//! New-NetRoute <ip>/32 via the physical default gateway, metric 7373
+//! ```
+//!
+//! The halves out-rank the default route without replacing it, so every more
+//! specific route (LAN, another VPN) keeps working. A `/32` per address the
+//! agent talks to directly (the verified direct paths of its connections, and
+//! the relays it uses, resolved when the rules are applied) keeps that traffic
+//! on the physical network instead of looping into the tunnel it carries. The
+//! metric 7373 marks those routes as ours, which is how stale ones are found
+//! and removed. Everything is written to the active store, so a reboot clears
+//! it. IPv6 is blackholed with `::/1` and `8000::/1` on the overlay interface,
+//! which has no IPv6: applications fall back to IPv4 after a delay, where
+//! Linux and macOS answer at once. The bypass follows the agent's paths as
+//! they change; a path that appears after the last apply is reached through
+//! the tunnel until the next one.
+//!
+//! The bypass routes are written before the halves, and removed after them.
+
+use super::RuleOutcome;
+use std::net::Ipv4Addr;
+
+use super::exit::{ExitHostPlan, ExitHostReport, exit_tag, exit_tag_prefix};
+use crate::state::Ipv4Range;
+
+/// The route metric that marks a bypass route as this agent's.
+const BYPASS_METRIC: u32 = 7373;
+
+/// Runs a PowerShell script and returns what it printed.
+pub(super) type RunScript<'a> = &'a dyn Fn(&str) -> Result<String, String>;
+
+/// Resolves a host name to its IPv4 addresses.
+pub(super) type Resolve<'a> = &'a dyn Fn(&str) -> Vec<Ipv4Addr>;
+
+/// A PowerShell single-quoted literal, with any embedded quote doubled.
+fn ps_literal(value: &str) -> String {
+    format!("'{}'", value.replace('\'', "''"))
+}
+
+/// Creates the NAT of one range, replacing one this agent made before.
+fn nat_script(interface: &str, range: &Ipv4Range) -> String {
+    let name = ps_literal(&exit_tag(interface, range));
+    let prefix = ps_literal(&range.to_string());
+    format!(
+        "$ErrorActionPreference = 'Stop'\n\
+         $name = {name}\n\
+         Get-NetNat -Name $name -ErrorAction SilentlyContinue | \
+         Remove-NetNat -Confirm:$false -ErrorAction SilentlyContinue\n\
+         New-NetNat -Name $name -InternalIPInterfaceAddressPrefix {prefix} | Out-Null\n"
+    )
+}
+
+/// Removes this interface's NAT objects except those named in `keep`.
+fn prune_script(interface: &str, keep: &[Ipv4Range]) -> String {
+    let prefix = ps_literal(&exit_tag_prefix(interface));
+    let keep: Vec<String> = keep
+        .iter()
+        .map(|range| ps_literal(&exit_tag(interface, range)))
+        .collect();
+    format!(
+        "$ErrorActionPreference = 'SilentlyContinue'\n\
+         $prefix = {prefix}\n\
+         $keep = @({})\n\
+         Get-NetNat | Where-Object {{ $_.Name.StartsWith($prefix) -and \
+         ($keep -notcontains $_.Name) }} | Remove-NetNat -Confirm:$false\n\
+         exit 0\n",
+        keep.join(", ")
+    )
+}
+
+/// Prints `Enabled` or `Disabled` for the interface's IPv4 forwarding.
+fn forwarding_script(interface: &str) -> String {
+    let alias = ps_literal(interface);
+    format!(
+        "$ErrorActionPreference = 'Stop'\n\
+         (Get-NetIPInterface -InterfaceAlias {alias} -AddressFamily IPv4).Forwarding\n"
+    )
+}
+
+/// Prints `<interface index> <next hop>` of the best default route that is
+/// not through the overlay interface.
+fn gateway_script(interface: &str) -> String {
+    let alias = ps_literal(interface);
+    format!(
+        "$ErrorActionPreference = 'Stop'\n\
+         $alias = {alias}\n\
+         $best = Get-NetRoute -DestinationPrefix '0.0.0.0/0' -AddressFamily IPv4 \
+         -PolicyStore ActiveStore | Where-Object {{ $_.InterfaceAlias -ne $alias -and \
+         $_.NextHop -ne '0.0.0.0' }} | Sort-Object {{ $_.RouteMetric + \
+         (Get-NetIPInterface -InterfaceIndex $_.InterfaceIndex -AddressFamily IPv4).InterfaceMetric }} | \
+         Select-Object -First 1\n\
+         if (-not $best) {{ throw 'the host has no default gateway outside the overlay' }}\n\
+         \"$($best.InterfaceIndex) $($best.NextHop)\"\n"
+    )
+}
+
+/// The default gateway as the script above printed it.
+fn parse_gateway(text: &str) -> Option<(u32, Ipv4Addr)> {
+    let mut words = text.split_whitespace();
+    let index = words.next()?.parse().ok()?;
+    let hop = words.next()?.parse().ok()?;
+    words.next().is_none().then_some((index, hop))
+}
+
+/// Replaces the bypass routes with one per address, through the gateway.
+fn bypass_script(index: u32, gateway: Ipv4Addr, addresses: &[Ipv4Addr]) -> String {
+    let list: Vec<String> = addresses
+        .iter()
+        .map(|address| ps_literal(&address.to_string()))
+        .collect();
+    format!(
+        "$ErrorActionPreference = 'Stop'\n\
+         Get-NetRoute -AddressFamily IPv4 -PolicyStore ActiveStore -ErrorAction SilentlyContinue | \
+         Where-Object {{ $_.RouteMetric -eq {BYPASS_METRIC} -and $_.DestinationPrefix.EndsWith('/32') }} | \
+         Remove-NetRoute -Confirm:$false -ErrorAction SilentlyContinue\n\
+         foreach ($ip in @({})) {{\n\
+         New-NetRoute -DestinationPrefix \"$ip/32\" -InterfaceIndex {index} -NextHop '{gateway}' \
+         -RouteMetric {BYPASS_METRIC} -PolicyStore ActiveStore -ErrorAction SilentlyContinue | Out-Null\n\
+         }}\n",
+        list.join(", ")
+    )
+}
+
+/// Sends everything else through the overlay interface.
+fn halves_script(interface: &str) -> String {
+    let alias = ps_literal(interface);
+    format!(
+        "$ErrorActionPreference = 'Stop'\n\
+         $alias = {alias}\n\
+         foreach ($p in '0.0.0.0/1', '128.0.0.0/1') {{\n\
+         Get-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -PolicyStore ActiveStore \
+         -ErrorAction SilentlyContinue | Remove-NetRoute -Confirm:$false -ErrorAction SilentlyContinue\n\
+         New-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -NextHop '0.0.0.0' \
+         -RouteMetric 1 -PolicyStore ActiveStore | Out-Null\n\
+         }}\n\
+         foreach ($p in '::/1', '8000::/1') {{\n\
+         Get-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -PolicyStore ActiveStore \
+         -ErrorAction SilentlyContinue | Remove-NetRoute -Confirm:$false -ErrorAction SilentlyContinue\n\
+         New-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -NextHop '::' \
+         -RouteMetric 1 -PolicyStore ActiveStore -ErrorAction SilentlyContinue | Out-Null\n\
+         }}\n"
+    )
+}
+
+/// Removes the halves and every bypass route. What is not there is the state
+/// wanted.
+fn remove_client_script(interface: &str) -> String {
+    let alias = ps_literal(interface);
+    format!(
+        "$ErrorActionPreference = 'SilentlyContinue'\n\
+         $alias = {alias}\n\
+         foreach ($p in '0.0.0.0/1', '128.0.0.0/1', '::/1', '8000::/1') {{\n\
+         Get-NetRoute -DestinationPrefix $p -InterfaceAlias $alias -PolicyStore ActiveStore | \
+         Remove-NetRoute -Confirm:$false\n\
+         }}\n\
+         Get-NetRoute -AddressFamily IPv4 -PolicyStore ActiveStore | \
+         Where-Object {{ $_.RouteMetric -eq {BYPASS_METRIC} -and $_.DestinationPrefix.EndsWith('/32') }} | \
+         Remove-NetRoute -Confirm:$false\n\
+         exit 0\n"
+    )
+}
+
+/// Every address to keep direct: the ones given and what the names resolve
+/// to, sorted and without duplicates.
+fn bypass_addresses(
+    resolve: Resolve<'_>,
+    addresses: &[Ipv4Addr],
+    hosts: &[String],
+) -> Vec<Ipv4Addr> {
+    let mut all: std::collections::BTreeSet<Ipv4Addr> = addresses.iter().copied().collect();
+    for host in hosts {
+        all.extend(resolve(host));
+    }
+    all.into_iter()
+        .filter(|a| !a.is_loopback() && !a.is_unspecified() && !a.is_multicast())
+        .collect()
+}
+
+fn apply_client(
+    run: RunScript<'_>,
+    resolve: Resolve<'_>,
+    plan: &ExitHostPlan,
+) -> Result<(), String> {
+    let interface = plan.interface.as_str();
+    let gateway = run(&gateway_script(interface))?;
+    let (index, hop) = parse_gateway(&gateway)
+        .ok_or_else(|| format!("cannot read the default gateway from `{}`", gateway.trim()))?;
+    let addresses = bypass_addresses(resolve, &plan.bypass, &plan.bypass_hosts);
+    run(&bypass_script(index, hop, &addresses))?;
+    if let Err(why) = run(&halves_script(interface)) {
+        // Do not leave a half-installed set: bypass routes alone are harmless
+        // but the next apply starts clean.
+        let _ = run(&remove_client_script(interface));
+        return Err(why);
+    }
+    Ok(())
+}
+
+fn parse_forwarding(text: &str) -> Option<bool> {
+    match text.trim().to_ascii_lowercase().as_str() {
+        "enabled" | "1" => Some(true),
+        "disabled" | "0" => Some(false),
+        _ => None,
+    }
+}
+
+/// Makes the host match the plan.
+pub(super) fn apply_plan(
+    run: RunScript<'_>,
+    resolve: Resolve<'_>,
+    plan: &ExitHostPlan,
+) -> ExitHostReport {
+    let interface = plan.interface.as_str();
+    // What an earlier plan left and this one no longer wants.
+    let pruned = run(&prune_script(interface, &plan.offer));
+    let offer = plan
+        .offer
+        .iter()
+        .map(|range| {
+            let outcome = match &pruned {
+                Err(why) => RuleOutcome::Failed(why.clone()),
+                Ok(_) => match run(&nat_script(interface, range)) {
+                    Ok(_) => RuleOutcome::Applied,
+                    Err(why) => RuleOutcome::Failed(why),
+                },
+            };
+            (*range, outcome)
+        })
+        .collect();
+    let client = if plan.client {
+        Some(match apply_client(run, resolve, plan) {
+            Ok(()) => RuleOutcome::Applied,
+            Err(why) => RuleOutcome::Failed(why),
+        })
+    } else {
+        // Whatever an earlier plan left must not stay in force.
+        let _ = run(&remove_client_script(interface));
+        None
+    };
+    let forwarding = if plan.offer.is_empty() {
+        None
+    } else {
+        run(&forwarding_script(interface))
+            .ok()
+            .and_then(|text| parse_forwarding(&text))
+    };
+    ExitHostReport {
+        offer,
+        forwarding,
+        client,
+    }
+}
+
+/// Removes every NAT object of this agent's for the interface.
+pub(super) fn clear_all(run: RunScript<'_>, interface: &str) {
+    if let Err(why) = run(&prune_script(interface, &[])) {
+        tracing::warn!(interface, "cannot remove the exit node NAT: {why}");
+    }
+    if let Err(why) = run(&remove_client_script(interface)) {
+        tracing::warn!(interface, "cannot remove the exit node routes: {why}");
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    #![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
+
+    use std::cell::RefCell;
+
+    use super::*;
+
+    fn range() -> Ipv4Range {
+        "10.13.37.0/24".parse().unwrap()
+    }
+
+    fn plan(client: bool) -> ExitHostPlan {
+        ExitHostPlan {
+            interface: "tsun0".into(),
+            offer: vec![range()],
+            client,
+            overlay: vec![range()],
+            ..Default::default()
+        }
+    }
+
+    #[test]
+    fn the_nat_is_named_with_the_tag_and_replaces_its_own() {
+        let script = nat_script("tsun0", &range());
+        assert!(
+            script.contains("$name = 'tsunagi-exit:tsun0:10.13.37.0/24'"),
+            "{script}"
+        );
+        assert!(script.contains("-InternalIPInterfaceAddressPrefix '10.13.37.0/24'"));
+        assert!(
+            script.find("Remove-NetNat").unwrap() < script.find("New-NetNat").unwrap(),
+            "{script}"
+        );
+    }
+
+    #[test]
+    fn a_quote_in_a_value_cannot_escape_the_literal() {
+        let script = nat_script("x'; Remove-Item C:\\ #", &range());
+        assert!(script.contains("'tsunagi-exit:x''; Remove-Item C:\\ #:10.13.37.0/24'"));
+        assert!(prune_script("x'y", &[]).contains("'tsunagi-exit:x''y:'"));
+        assert!(forwarding_script("x'y").contains("'x''y'"));
+    }
+
+    #[test]
+    fn pruning_keeps_wanted_ranges_and_only_selects_this_interfaces_nats() {
+        let script = prune_script("tsun0", &[range()]);
+        assert!(script.contains("$prefix = 'tsunagi-exit:tsun0:'"));
+        assert!(script.contains("$keep = @('tsunagi-exit:tsun0:10.13.37.0/24')"));
+        assert!(script.contains("StartsWith($prefix)"));
+        assert!(script.contains("exit 0"));
+        assert!(prune_script("tsun0", &[]).contains("$keep = @()"));
+    }
+
+    #[test]
+    fn forwarding_text_is_read_never_written() {
+        assert_eq!(parse_forwarding("Enabled\r\n"), Some(true));
+        assert_eq!(parse_forwarding("Disabled"), Some(false));
+        assert_eq!(parse_forwarding("what"), None);
+        let script = forwarding_script("tsun0");
+        assert!(script.contains("Get-NetIPInterface"));
+        assert!(!script.contains("Set-NetIPInterface"));
+    }
+
+    #[test]
+    fn offering_prunes_then_creates_and_reads_forwarding() {
+        let calls = RefCell::new(Vec::new());
+        let run = |script: &str| -> Result<String, String> {
+            calls.borrow_mut().push(script.to_string());
+            Ok(if script.contains("Get-NetIPInterface") {
+                "Disabled\n".to_string()
+            } else {
+                String::new()
+            })
+        };
+        let report = apply_plan(&run, &no_names, &plan(false));
+        assert_eq!(report.offer, vec![(range(), RuleOutcome::Applied)]);
+        assert_eq!(report.forwarding, Some(false));
+        assert_eq!(report.client, None);
+        let calls = calls.borrow();
+        assert!(calls[0].contains("StartsWith"), "{calls:?}");
+        assert!(calls[1].contains("New-NetNat"), "{calls:?}");
+        // Without a client, the routes of an earlier plan are taken away.
+        assert!(calls[2].contains("RouteMetric -eq 7373"), "{calls:?}");
+        assert!(calls[3].contains("Get-NetIPInterface"), "{calls:?}");
+    }
+
+    #[test]
+    fn a_failing_nat_is_that_ranges_failure() {
+        let run = |script: &str| -> Result<String, String> {
+            if script.contains("New-NetNat") {
+                Err("A NAT with this prefix already exists".into())
+            } else {
+                Ok(String::new())
+            }
+        };
+        let report = apply_plan(&run, &no_names, &plan(false));
+        let RuleOutcome::Failed(why) = &report.offer[0].1 else {
+            panic!("{report:?}");
+        };
+        assert!(why.contains("already exists"), "{why}");
+    }
+
+    fn no_names(_: &str) -> Vec<Ipv4Addr> {
+        Vec::new()
+    }
+
+    fn client_plan() -> ExitHostPlan {
+        ExitHostPlan {
+            interface: "tsun0".into(),
+            offer: Vec::new(),
+            client: true,
+            overlay: vec![range()],
+            bypass: vec![Ipv4Addr::new(138, 201, 61, 182)],
+            bypass_hosts: vec!["relay.example".into()],
+        }
+    }
+
+    #[test]
+    fn the_gateway_is_read_from_the_script_output() {
+        assert_eq!(
+            parse_gateway("12 192.168.1.1\r\n"),
+            Some((12, Ipv4Addr::new(192, 168, 1, 1)))
+        );
+        assert_eq!(parse_gateway(""), None);
+        assert_eq!(parse_gateway("12"), None);
+        assert_eq!(parse_gateway("12 gw"), None);
+        assert_eq!(parse_gateway("12 1.1.1.1 extra"), None);
+        let script = gateway_script("tsun0");
+        assert!(script.contains("-ne $alias"));
+        assert!(script.contains("-ne '0.0.0.0'"));
+    }
+
+    #[test]
+    fn bypass_routes_are_marked_host_routes_through_the_physical_gateway() {
+        let script = bypass_script(
+            12,
+            Ipv4Addr::new(192, 168, 1, 1),
+            &[Ipv4Addr::new(1, 2, 3, 4), Ipv4Addr::new(5, 6, 7, 8)],
+        );
+        assert!(script.contains("@('1.2.3.4', '5.6.7.8')"), "{script}");
+        assert!(script.contains("-InterfaceIndex 12 -NextHop '192.168.1.1'"));
+        assert!(script.contains("-RouteMetric 7373"));
+        assert!(script.contains("-PolicyStore ActiveStore"));
+        // Stale marked routes go first, and only /32 ones with our metric.
+        assert!(script.find("Remove-NetRoute").unwrap() < script.find("New-NetRoute").unwrap());
+        assert!(script.contains("RouteMetric -eq 7373"));
+        assert!(script.contains("EndsWith('/32')"));
+    }
+
+    #[test]
+    fn the_halves_go_through_the_overlay_and_ipv6_is_blackholed() {
+        let script = halves_script("tsun0");
+        assert!(script.contains("'0.0.0.0/1', '128.0.0.0/1'"));
+        assert!(script.contains("-InterfaceAlias $alias -NextHop '0.0.0.0'"));
+        assert!(script.contains("'::/1', '8000::/1'"));
+        assert!(script.contains("-PolicyStore ActiveStore"));
+        assert!(
+            !script.contains("0.0.0.0/0"),
+            "the default route is left alone"
+        );
+        assert!(halves_script("x'y").contains("'x''y'"));
+    }
+
+    #[test]
+    fn addresses_are_resolved_merged_and_cleaned() {
+        let resolve = |host: &str| -> Vec<Ipv4Addr> {
+            match host {
+                "relay.example" => vec![Ipv4Addr::new(9, 9, 9, 9), Ipv4Addr::new(1, 2, 3, 4)],
+                _ => Vec::new(),
+            }
+        };
+        let all = bypass_addresses(
+            &resolve,
+            &[
+                Ipv4Addr::new(1, 2, 3, 4),
+                Ipv4Addr::LOCALHOST,
+                Ipv4Addr::UNSPECIFIED,
+            ],
+            &["relay.example".into(), "gone.example".into()],
+        );
+        assert_eq!(
+            all,
+            vec![Ipv4Addr::new(1, 2, 3, 4), Ipv4Addr::new(9, 9, 9, 9)]
+        );
+    }
+
+    #[test]
+    fn the_bypass_is_written_before_the_halves_that_need_it() {
+        let calls = RefCell::new(Vec::new());
+        let run = |script: &str| -> Result<String, String> {
+            calls.borrow_mut().push(script.to_string());
+            Ok(if script.contains("Select-Object -First 1") {
+                "12 192.168.1.1\n".to_string()
+            } else {
+                String::new()
+            })
+        };
+        let resolve = |_: &str| vec![Ipv4Addr::new(9, 9, 9, 9)];
+        let report = apply_plan(&run, &resolve, &client_plan());
+        assert_eq!(report.client, Some(RuleOutcome::Applied));
+        let calls = calls.borrow();
+        let at = |needle: &str| calls.iter().position(|c| c.contains(needle)).unwrap();
+        assert!(at("Select-Object -First 1") < at("-RouteMetric 7373"));
+        assert!(at("-RouteMetric 7373") < at("'0.0.0.0/1', '128.0.0.0/1'"));
+        let bypass = &calls[at("-RouteMetric 7373")];
+        assert!(
+            bypass.contains("'9.9.9.9'") && bypass.contains("'138.201.61.182'"),
+            "{bypass}"
+        );
+    }
+
+    #[test]
+    fn nothing_goes_through_the_overlay_when_the_bypass_cannot_be_written() {
+        let run = |script: &str| -> Result<String, String> {
+            if script.contains("Select-Object -First 1") {
+                Ok("12 192.168.1.1".to_string())
+            } else if script.contains("-RouteMetric 7373") && script.contains("foreach ($ip") {
+                Err("Access is denied".to_string())
+            } else {
+                assert!(!script.contains("-NextHop '0.0.0.0'"), "{script}");
+                Ok(String::new())
+            }
+        };
+        let report = apply_plan(&run, &no_names, &client_plan());
+        assert_eq!(
+            report.client,
+            Some(RuleOutcome::Failed("Access is denied".into()))
+        );
+    }
+
+    #[test]
+    fn a_missing_gateway_is_a_failure_that_installs_no_route() {
+        let run = |script: &str| -> Result<String, String> {
+            assert!(!script.contains("New-NetRoute"), "{script}");
+            if script.contains("Select-Object -First 1") {
+                Err("the host has no default gateway outside the overlay".into())
+            } else {
+                Ok(String::new())
+            }
+        };
+        let report = apply_plan(&run, &no_names, &client_plan());
+        let Some(RuleOutcome::Failed(why)) = report.client else {
+            panic!("{report:?}");
+        };
+        assert!(why.contains("no default gateway"), "{why}");
+    }
+
+    #[test]
+    fn a_failed_halves_step_removes_what_was_written() {
+        let calls = RefCell::new(Vec::new());
+        let run = |script: &str| -> Result<String, String> {
+            calls.borrow_mut().push(script.to_string());
+            if script.contains("Select-Object -First 1") {
+                Ok("12 192.168.1.1".to_string())
+            } else if script.contains("'128.0.0.0/1'") && script.contains("New-NetRoute") {
+                Err("boom".to_string())
+            } else {
+                Ok(String::new())
+            }
+        };
+        let report = apply_plan(&run, &no_names, &client_plan());
+        assert!(matches!(report.client, Some(RuleOutcome::Failed(_))));
+        assert!(calls.borrow().last().unwrap().contains("Remove-NetRoute"));
+    }
+
+    #[test]
+    fn dropping_the_client_removes_its_routes_and_clearing_does_too() {
+        let calls = RefCell::new(Vec::new());
+        let run = |script: &str| -> Result<String, String> {
+            calls.borrow_mut().push(script.to_string());
+            Ok(String::new())
+        };
+        let report = apply_plan(&run, &no_names, &plan(false));
+        assert_eq!(report.client, None);
+        assert!(
+            calls
+                .borrow()
+                .iter()
+                .any(|c| c.contains("RouteMetric -eq 7373"))
+        );
+        let script = remove_client_script("tsun0");
+        assert!(script.contains("'0.0.0.0/1', '128.0.0.0/1', '::/1', '8000::/1'"));
+        assert!(script.contains("exit 0"));
+        // Only routes on our interface, or marked ours, are touched.
+        assert!(script.contains("-InterfaceAlias $alias"));
+        calls.borrow_mut().clear();
+        clear_all(&run, "tsun0");
+        assert!(
+            calls
+                .borrow()
+                .iter()
+                .any(|c| c.contains("RouteMetric -eq 7373"))
+        );
+    }
+
+    #[test]
+    fn clearing_prunes_everything_for_the_interface() {
+        let seen = RefCell::new(Vec::<String>::new());
+        let run = |script: &str| -> Result<String, String> {
+            seen.borrow_mut().push(script.to_string());
+            Ok(String::new())
+        };
+        clear_all(&run, "tsun0");
+        let seen = seen.borrow();
+        assert!(seen[0].contains("$keep = @()"));
+        assert!(seen[0].contains("'tsunagi-exit:tsun0:'"));
+    }
+}

crates/tsunagi-core/src/overlay/interface.rs

@@ -164,6 +164,17 @@ pub struct Interface {
     rules_report: std::sync::Mutex<Option<BroadcastRulesReport>>,
     /// What the exit-node rules were last asked for, and what came of it.
     exit_state: std::sync::Mutex<ExitState>,
+    /// Where each network's runtime reaches its peers and relays outside the
+    /// overlay; read only by hosts that cannot exempt the agent from the
+    /// exit node's default route.
+    underlay: std::sync::Mutex<std::collections::HashMap<NetworkId, Underlay>>,
+}
+
+/// Underlay destinations one network's runtime currently depends on.
+#[derive(Debug, Default, Clone, PartialEq, Eq)]
+struct Underlay {
+    addresses: std::collections::BTreeSet<std::net::Ipv4Addr>,
+    hosts: std::collections::BTreeSet<String>,
 }
 
 /// The exit-node host rules as this interface last left them.
@@ -275,6 +286,7 @@ impl Interface {
             host_rules,
             rules_report: std::sync::Mutex::new(None),
             exit_state: std::sync::Mutex::new(ExitState::default()),
+            underlay: std::sync::Mutex::new(std::collections::HashMap::new()),
         })
     }
 
@@ -350,10 +362,20 @@ impl Interface {
             .collect();
         ranges.sort_by_key(|range| (range.base, range.prefix_len));
         ranges.dedup();
+        let client = self.routes.exit_via().is_some();
+        let overlay = self.routes.overlay_ranges();
+        let (bypass, bypass_hosts) = if client && exit.needs_bypass() {
+            self.underlay_targets(&overlay)
+        } else {
+            (Vec::new(), Vec::new())
+        };
         let wanted = ExitHostPlan {
             interface: self.name.clone(),
             offer: ranges,
-            client: self.routes.exit_via().is_some(),
+            client,
+            overlay,
+            bypass,
+            bypass_hosts,
         };
 
         let first = !std::mem::replace(&mut self.exit_state().cleaned, true);
@@ -398,9 +420,15 @@ impl Interface {
             tracing::warn!(
                 interface = %self.name,
                 "kernel forwarding is off for this interface, so nothing is forwarded for the \
-                 exit node yet: `sysctl -w net.ipv4.conf.{0}.forwarding=1` (or \
-                 `net.ipv4.ip_forward=1`), and set it in sysctl.d to keep it",
-                self.name
+                 exit node yet: {}",
+                if cfg!(target_os = "windows") {
+                    "`Set-NetIPInterface -Forwarding Enabled` on the interfaces involved"
+                } else if cfg!(target_os = "macos") {
+                    "`sysctl -w net.inet.ip.forwarding=1`, set again after every reboot"
+                } else {
+                    "`sysctl -w net.ipv4.ip_forward=1` (or `net.ipv4.conf.<interface>.forwarding=1`), \
+                     and set it in sysctl.d to keep it"
+                }
             );
         }
         match &report.client {
@@ -419,6 +447,68 @@ impl Interface {
         state.report = Some(report);
     }
 
+    /// Tells the interface where a network's runtime reaches peers and
+    /// relays outside the overlay. When the host needs those kept out of the
+    /// exit node's default route and the set changed, the rules follow.
+    pub async fn set_underlay(
+        &self,
+        network: NetworkId,
+        addresses: impl IntoIterator<Item = std::net::Ipv4Addr>,
+        hosts: impl IntoIterator<Item = String>,
+    ) {
+        let new = Underlay {
+            addresses: addresses.into_iter().collect(),
+            hosts: hosts.into_iter().collect(),
+        };
+        let changed = {
+            let mut map = match self.underlay.lock() {
+                Ok(guard) => guard,
+                Err(poisoned) => poisoned.into_inner(),
+            };
+            if new == Underlay::default() {
+                map.remove(&network).is_some()
+            } else {
+                map.insert(network, new.clone()) != Some(new)
+            }
+        };
+        let needed = changed
+            && self.routes.exit_via().is_some()
+            && self
+                .host_rules
+                .as_ref()
+                .and_then(|rules| rules.exit_rules())
+                .is_some_and(|exit| exit.needs_bypass());
+        if needed {
+            self.sync_exit_rules().await;
+        }
+    }
+
+    /// Every underlay destination worth keeping direct, minus anything that
+    /// is the overlay itself, loopback or link-local.
+    fn underlay_targets(
+        &self,
+        overlay: &[crate::state::Ipv4Range],
+    ) -> (Vec<std::net::Ipv4Addr>, Vec<String>) {
+        let map = match self.underlay.lock() {
+            Ok(guard) => guard,
+            Err(poisoned) => poisoned.into_inner(),
+        };
+        let mut addresses = std::collections::BTreeSet::new();
+        let mut hosts = std::collections::BTreeSet::new();
+        for underlay in map.values() {
+            addresses.extend(underlay.addresses.iter().copied().filter(|address| {
+                !address.is_loopback()
+                    && !address.is_link_local()
+                    && !address.is_multicast()
+                    && !address.is_unspecified()
+                    && !address.is_broadcast()
+                    && !overlay.iter().any(|range| range.contains(*address))
+            }));
+            hosts.extend(underlay.hosts.iter().cloned());
+        }
+        (addresses.into_iter().collect(), hosts.into_iter().collect())
+    }
+
     fn exit_state(&self) -> std::sync::MutexGuard<'_, ExitState> {
         match self.exit_state.lock() {
             Ok(guard) => guard,
@@ -443,7 +533,7 @@ impl Interface {
                 "this overlay interface is not on the host (no TUN device), so there is \
                  nothing to forward to",
             ),
-            Some(false) => Some("exit nodes are only supported on Linux yet"),
+            Some(false) => Some("exit nodes are not supported on this platform yet"),
             Some(true) => None,
         };
         let failed = |detail: &str| RuleSetReport {
@@ -1336,6 +1426,65 @@ mod tests {
         assert_eq!(interface.counters().received, 1);
     }
 
+    #[tokio::test]
+    async fn the_underlay_reaches_the_rules_only_for_hosts_that_need_it_and_only_while_using_one() {
+        let rules = MockHostRules::new();
+        let (interface, routes, id) = managed_interface(&rules, false).await;
+        let relay = || vec!["relay.example".to_string()];
+        let direct = || {
+            vec![
+                Ipv4Addr::new(138, 201, 61, 182),
+                Ipv4Addr::new(10, 13, 37, 9), // the overlay itself
+                Ipv4Addr::LOCALHOST,
+                Ipv4Addr::new(169, 254, 1, 1),
+            ]
+        };
+
+        // A host that does not ask never gets them, so Linux and macOS do not
+        // re-apply their rules whenever a path moves.
+        routes.set_exit(id, using(peer(2)));
+        interface.set_underlay(id, direct(), relay()).await;
+        interface.sync_exit_rules().await;
+        let plan = rules.exit().installed().unwrap();
+        assert!(plan.client);
+        assert!(plan.bypass.is_empty() && plan.bypass_hosts.is_empty());
+
+        // A host that does gets the usable ones, and follows changes.
+        rules.exit().set_needs_bypass(true);
+        interface.set_underlay(id, direct(), relay()).await;
+        interface.sync_exit_rules().await;
+        let plan = rules.exit().installed().unwrap();
+        assert_eq!(plan.bypass, vec![Ipv4Addr::new(138, 201, 61, 182)]);
+        assert_eq!(plan.bypass_hosts, relay());
+        let applies = || {
+            rules
+                .exit()
+                .calls()
+                .iter()
+                .filter(|c| c.starts_with("apply"))
+                .count()
+        };
+        let before = applies();
+        interface.set_underlay(id, direct(), relay()).await;
+        assert_eq!(applies(), before, "an unchanged set is not applied again");
+        interface
+            .set_underlay(id, vec![Ipv4Addr::new(5, 6, 7, 8)], relay())
+            .await;
+        assert_eq!(applies(), before + 1);
+        assert_eq!(
+            rules.exit().installed().unwrap().bypass,
+            vec![Ipv4Addr::new(5, 6, 7, 8)]
+        );
+
+        // Not using an exit node: nothing to keep direct.
+        routes.set_exit(id, Default::default());
+        interface.sync_exit_rules().await;
+        interface
+            .set_underlay(id, vec![Ipv4Addr::new(1, 1, 1, 1)], relay())
+            .await;
+        assert!(rules.exit().installed().is_none());
+    }
+
     #[tokio::test]
     async fn the_exit_rules_follow_the_routing_table_and_leave_with_the_interface() {
         let rules = MockHostRules::new();

crates/tsunagi-core/src/overlay/router.rs

@@ -293,6 +293,18 @@ impl RoutingTable {
         offers
     }
 
+    /// The range of every network in the table that has agreed one, sorted.
+    pub fn overlay_ranges(&self) -> Vec<Ipv4Range> {
+        let mut ranges: Vec<Ipv4Range> = self
+            .read()
+            .values()
+            .filter_map(|routes| routes.range)
+            .collect();
+        ranges.sort_by_key(|range| (range.base, range.prefix_len));
+        ranges.dedup();
+        ranges
+    }
+
     /// One network's exit-node settings and its range, if it is in the table.
     pub fn exit_settings(&self, network: NetworkId) -> Option<(ExitPolicy, Option<Ipv4Range>)> {
         self.read()

crates/tsunagi-tray/src/app.rs

@@ -81,6 +81,8 @@ impl App {
         }
         let builder = egui::ViewportBuilder::default()
             .with_title("About tsunagi")
+            .with_app_id(crate::icon::APP_ID)
+            .with_icon(crate::icon::window())
             .with_inner_size([560.0, 520.0])
             .with_min_inner_size([380.0, 280.0]);
         let socket = self.agent.socket();
@@ -112,6 +114,8 @@ impl App {
             let viewport_id = egui::ViewportId::from_hash_of(("devices", &id));
             let builder = egui::ViewportBuilder::default()
                 .with_title(devices::title(network))
+                .with_app_id(crate::icon::APP_ID)
+                .with_icon(crate::icon::window())
                 .with_inner_size([940.0, 540.0])
                 .with_min_inner_size([480.0, 320.0]);
             let agent = &self.agent;

crates/tsunagi-tray/src/exit.rs

@@ -13,6 +13,12 @@ pub(crate) const GOOD: egui::Color32 = egui::Color32::from_rgb(0x3c, 0xb0, 0x4a)
 pub(crate) const LOUD: egui::Color32 = egui::Color32::from_rgb(0xe0, 0x4a, 0x3c);
 
 /// What to run to turn kernel forwarding on, which the agent never does.
+#[cfg(target_os = "macos")]
+pub(crate) const FORWARDING_COMMAND: &str = "sudo sysctl -w net.inet.ip.forwarding=1";
+#[cfg(target_os = "windows")]
+pub(crate) const FORWARDING_COMMAND: &str =
+    "Get-NetIPInterface | Set-NetIPInterface -Forwarding Enabled";
+#[cfg(not(any(target_os = "macos", target_os = "windows")))]
 pub(crate) const FORWARDING_COMMAND: &str = "sudo sysctl -w net.ipv4.ip_forward=1";
 
 /// One thing wrong, loud enough to read at a glance.

crates/tsunagi-tray/src/icon.rs

@@ -0,0 +1,135 @@
+//! The application's picture: the logo drawn in code, in the colour of the
+//! state it stands for, so there is no asset to ship or to go missing.
+//!
+//! It is the same shape as `dist/linux/tsunagi.svg` (a disc, a ring, a dot).
+
+use crate::watch::Health;
+
+/// Normal operation. The colour of the logo itself.
+const BLUE: [u8; 3] = [0x2f, 0x80, 0xd8];
+/// An exit node is in use.
+const GREEN: [u8; 3] = [0x3c, 0xb0, 0x4a];
+/// Nothing is connected, or the exit node in use is gone.
+const DARK_RED: [u8; 3] = [0x9a, 0x1f, 0x1f];
+
+/// The window application id, which is also the name of the desktop entry
+/// (`tsunagi-tray.desktop`). The compositor finds the icon through it.
+pub(crate) const APP_ID: &str = "tsunagi-tray";
+
+const SIZE: u32 = 64;
+const SAMPLES: u32 = 4;
+
+/// What the logo looks like at a point of its 128 × 128 design, as
+/// premultiplied-free RGBA in `0.0..=1.0`.
+fn shade(x: f32, y: f32, color: [u8; 3]) -> [f32; 4] {
+    let distance = ((x - 64.0).powi(2) + (y - 64.0).powi(2)).sqrt();
+    if distance > 56.0 {
+        return [0.0; 4];
+    }
+    let mut pixel = [
+        f32::from(color[0]) / 255.0,
+        f32::from(color[1]) / 255.0,
+        f32::from(color[2]) / 255.0,
+    ];
+    let mut white = 0.0;
+    if (26.5..=33.5).contains(&distance) {
+        white = 0.9;
+    }
+    if distance <= 9.0 {
+        white = 1.0;
+    }
+    for channel in &mut pixel {
+        *channel = *channel * (1.0 - white) + white;
+    }
+    [pixel[0], pixel[1], pixel[2], 1.0]
+}
+
+/// The logo as RGBA pixels, antialiased by supersampling.
+fn rgba(color: [u8; 3]) -> Vec<u8> {
+    let scale = 128.0 / SIZE as f32;
+    let mut out = Vec::with_capacity((SIZE * SIZE * 4) as usize);
+    for py in 0..SIZE {
+        for px in 0..SIZE {
+            let mut sum = [0.0f32; 4];
+            for sy in 0..SAMPLES {
+                for sx in 0..SAMPLES {
+                    let x = (px as f32 + (sx as f32 + 0.5) / SAMPLES as f32) * scale;
+                    let y = (py as f32 + (sy as f32 + 0.5) / SAMPLES as f32) * scale;
+                    let [r, g, b, a] = shade(x, y, color);
+                    sum[0] += r * a;
+                    sum[1] += g * a;
+                    sum[2] += b * a;
+                    sum[3] += a;
+                }
+            }
+            let count = (SAMPLES * SAMPLES) as f32;
+            let alpha = sum[3] / count;
+            let unmultiply = if sum[3] > 0.0 { 1.0 / sum[3] } else { 0.0 };
+            for channel in &sum[..3] {
+                out.push((channel * unmultiply * 255.0).round() as u8);
+            }
+            out.push((alpha * 255.0).round() as u8);
+        }
+    }
+    out
+}
+
+fn colour_of(health: Health) -> [u8; 3] {
+    match health {
+        Health::Disconnected => DARK_RED,
+        Health::Connected => BLUE,
+        Health::ExitNode => GREEN,
+    }
+}
+
+/// The tray icon for a state.
+pub(crate) fn tray(health: Health) -> Option<tray_icon::Icon> {
+    tray_icon::Icon::from_rgba(rgba(colour_of(health)), SIZE, SIZE).ok()
+}
+
+/// The icon of the application's windows.
+pub(crate) fn window() -> eframe::egui::IconData {
+    eframe::egui::IconData {
+        rgba: rgba(BLUE),
+        width: SIZE,
+        height: SIZE,
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    fn pixel(pixels: &[u8], x: u32, y: u32) -> [u8; 4] {
+        let at = ((y * SIZE + x) * 4) as usize;
+        [pixels[at], pixels[at + 1], pixels[at + 2], pixels[at + 3]]
+    }
+
+    #[test]
+    fn the_logo_is_a_coloured_disc_with_a_white_centre_and_clear_corners() {
+        let pixels = rgba(GREEN);
+        assert_eq!(pixels.len(), (SIZE * SIZE * 4) as usize);
+        assert_eq!(pixel(&pixels, 0, 0)[3], 0, "the corner is transparent");
+        let centre = pixel(&pixels, SIZE / 2, SIZE / 2);
+        assert_eq!(
+            centre,
+            [255, 255, 255, 255],
+            "the dot in the middle is white"
+        );
+        // Halfway between the ring and the edge is the colour itself.
+        let body = pixel(&pixels, SIZE / 2, SIZE / 2 - (SIZE * 44 / 128));
+        assert_eq!(body, [GREEN[0], GREEN[1], GREEN[2], 255]);
+    }
+
+    #[test]
+    fn each_state_has_its_own_colour() {
+        let colours = [
+            colour_of(Health::Disconnected),
+            colour_of(Health::Connected),
+            colour_of(Health::ExitNode),
+        ];
+        assert_ne!(colours[0], colours[1]);
+        assert_ne!(colours[1], colours[2]);
+        assert_ne!(colours[0], colours[2]);
+    }
+}

crates/tsunagi-tray/src/main.rs

@@ -15,9 +15,11 @@ mod app;
 mod devices;
 mod exit;
 mod format;
+mod icon;
 mod stats;
 mod tray;
 mod ui;
+mod watch;
 
 use eframe::egui;
 
@@ -53,7 +55,8 @@ fn window() -> Result<(), Box<dyn std::error::Error>> {
         // A fixed, small window; the whole layout targets this size.
         viewport: egui::ViewportBuilder::default()
             .with_title("tsunagi")
-            .with_app_id("tsunagi")
+            .with_app_id(icon::APP_ID)
+            .with_icon(icon::window())
             .with_inner_size([360.0, 600.0])
             .with_resizable(false),
         ..Default::default()

crates/tsunagi-tray/src/tray.rs

@@ -8,7 +8,10 @@ use std::process::{Child, Command};
 use std::sync::{Arc, Mutex};
 
 use tray_icon::menu::{Menu, MenuEvent, MenuId, MenuItem, PredefinedMenuItem};
-use tray_icon::{Icon, TrayIcon, TrayIconBuilder};
+use tray_icon::{TrayIcon, TrayIconBuilder};
+
+use crate::icon;
+use crate::watch::{self, Health};
 
 /// The window process, when there is one.
 struct Window {
@@ -69,9 +72,9 @@ fn build() -> Result<Built, String> {
     menu.append(&quit).map_err(|err| err.to_string())?;
 
     let icon = TrayIconBuilder::new()
-        .with_tooltip("tsunagi")
+        .with_tooltip(Health::Disconnected.label())
         .with_menu(Box::new(menu))
-        .with_icon(make_icon()?)
+        .with_icon(icon::tray(Health::Disconnected).ok_or("cannot draw the tray icon")?)
         .build()
         .map_err(|err| err.to_string())?;
     Ok(Built {
@@ -101,6 +104,16 @@ fn build_checked() -> Result<Built, String> {
     })
 }
 
+/// Shows a state on the icon, if it is a different one than is showing.
+fn show(icon: &TrayIcon, shown: &mut Option<Health>, health: Health) {
+    if *shown == Some(health) {
+        return;
+    }
+    *shown = Some(health);
+    let _ = icon.set_icon(icon::tray(health));
+    let _ = icon.set_tooltip(Some(health.label()));
+}
+
 /// Acts on menu choices as the platform's loop delivers them.
 fn handle_menu(built: &Built, window: Arc<Window>) {
     let (open, quit) = (built.open.clone(), built.quit.clone());
@@ -128,8 +141,22 @@ pub(crate) fn run() -> Result<(), Box<dyn std::error::Error>> {
             child: Mutex::new(None),
         }),
     );
+
+    // The icon belongs to this thread, so the answers come to it: the poller
+    // queues them and the loop here takes them off once a second.
+    let (tx, rx) = std::sync::mpsc::channel();
+    watch::spawn(move |health| {
+        let _ = tx.send(health);
+    });
+    let icon = built.icon;
+    let mut shown = None;
+    gtk::glib::timeout_add_seconds_local(1, move || {
+        while let Ok(health) = rx.try_recv() {
+            show(&icon, &mut shown, health);
+        }
+        gtk::glib::ControlFlow::Continue
+    });
     gtk::main();
-    drop(built.icon);
     Ok(())
 }
 
@@ -148,9 +175,16 @@ pub(crate) fn run() -> Result<(), Box<dyn std::error::Error>> {
     struct Pump {
         window: Arc<Window>,
         icon: Option<TrayIcon>,
+        shown: Option<Health>,
     }
 
-    impl ApplicationHandler for Pump {
+    impl ApplicationHandler<Health> for Pump {
+        fn user_event(&mut self, _event_loop: &ActiveEventLoop, health: Health) {
+            if let Some(icon) = &self.icon {
+                show(icon, &mut self.shown, health);
+            }
+        }
+
         fn resumed(&mut self, _event_loop: &ActiveEventLoop) {
             if self.icon.is_some() {
                 return;
@@ -170,7 +204,7 @@ pub(crate) fn run() -> Result<(), Box<dyn std::error::Error>> {
         fn window_event(&mut self, _: &ActiveEventLoop, _: WindowId, _: WindowEvent) {}
     }
 
-    let mut builder = EventLoop::builder();
+    let mut builder = EventLoop::<Health>::with_user_event();
     // macOS: live in the menu bar with no Dock icon.
     #[cfg(target_os = "macos")]
     {
@@ -178,33 +212,17 @@ pub(crate) fn run() -> Result<(), Box<dyn std::error::Error>> {
         builder.with_activation_policy(ActivationPolicy::Accessory);
     }
     let event_loop = builder.build()?;
+    let proxy = event_loop.create_proxy();
+    watch::spawn(move |health| {
+        let _ = proxy.send_event(health);
+    });
     let mut pump = Pump {
         window: Arc::new(Window {
             child: Mutex::new(None),
         }),
         icon: None,
+        shown: None,
     };
     event_loop.run_app(&mut pump)?;
     Ok(())
 }
-
-/// A small round icon, drawn in code so there is no asset to ship yet.
-fn make_icon() -> Result<Icon, String> {
-    const SIZE: u32 = 32;
-    let centre = (SIZE as f32 - 1.0) / 2.0;
-    let radius = SIZE as f32 * 0.46;
-    let mut rgba = vec![0u8; (SIZE * SIZE * 4) as usize];
-    for y in 0..SIZE {
-        for x in 0..SIZE {
-            let (dx, dy) = (x as f32 - centre, y as f32 - centre);
-            if dx * dx + dy * dy <= radius * radius {
-                let i = ((y * SIZE + x) * 4) as usize;
-                rgba[i] = 0x2f;
-                rgba[i + 1] = 0x80;
-                rgba[i + 2] = 0xd8;
-                rgba[i + 3] = 0xff;
-            }
-        }
-    }
-    Icon::from_rgba(rgba, SIZE, SIZE).map_err(|err| err.to_string())
-}

crates/tsunagi-tray/src/ui.rs

@@ -283,32 +283,31 @@ fn draw_network(
             }
         });
         ui.horizontal(|ui| {
-            ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
-                // In the corner while this device sends its traffic through
-                // somebody: green while that works, red while it does not.
-                if let Some(via) = &network.exit.via {
-                    let color = if network.exit.via_online {
-                        exit::GOOD
-                    } else {
-                        exit::LOUD
-                    };
-                    let response = ui
-                        .add(
-                            egui::Label::new(
-                                egui::RichText::new(icon::SIGN_OUT).size(18.0).color(color),
-                            )
-                            .sense(egui::Sense::click()),
+            // In the bottom left corner while this device sends its traffic through
+            // somebody: green while that works, red while it does not.
+            if let Some(via) = &network.exit.via {
+                let color = if network.exit.via_online {
+                    exit::GOOD
+                } else {
+                    exit::LOUD
+                };
+                let response = ui
+                    .add(
+                        egui::Label::new(
+                            egui::RichText::new(icon::SIGN_OUT).size(18.0).color(color),
                         )
-                        .on_hover_cursor(egui::CursorIcon::PointingHand)
-                        .on_hover_text(format!(
-                            "exit node: {} — click to stop",
-                            exit::name_of(network, via)
-                        ));
-                    if response.clicked() {
-                        state.exit_confirm =
-                            Some(devices::ExitConfirm::stop_from_tile(network, via));
-                    }
+                        .sense(egui::Sense::click()),
+                    )
+                    .on_hover_cursor(egui::CursorIcon::PointingHand)
+                    .on_hover_text(format!(
+                        "exit node: {} — click to stop",
+                        exit::name_of(network, via)
+                    ));
+                if response.clicked() {
+                    state.exit_confirm = Some(devices::ExitConfirm::stop_from_tile(network, via));
                 }
+            }
+            ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
                 if ui.button("Show devices").clicked() {
                     state.open_devices.insert(network.network_id.clone());
                 }

crates/tsunagi-tray/src/watch.rs

@@ -0,0 +1,146 @@
+//! What the tray icon shows: whether the agent has anybody to talk to, and
+//! whether this device sends its traffic through an exit node.
+//!
+//! The tray process has no window and no event loop of its own to poll from,
+//! so a thread asks the agent on an interval and hands each answer to a
+//! callback, which puts it where the platform wants it.
+
+use std::time::Duration;
+
+use tsunagi::ipc::{self, StatusReport};
+
+use crate::agent::resolve_socket;
+
+/// How often the agent is asked.
+const INTERVAL: Duration = Duration::from_secs(3);
+
+/// What the icon says.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum Health {
+    /// Nobody is connected, or the agent cannot be reached, or the exit node
+    /// this device uses is gone, which blocks its traffic.
+    Disconnected,
+    /// Connected, nothing out of the ordinary.
+    Connected,
+    /// This device's traffic leaves through an exit node, and that works.
+    ExitNode,
+}
+
+impl Health {
+    /// The state a status report amounts to; `None` is an agent that did not
+    /// answer.
+    pub(crate) fn of(report: Option<&StatusReport>) -> Self {
+        let Some(report) = report else {
+            return Self::Disconnected;
+        };
+        if !report
+            .networks
+            .iter()
+            .any(|network| !network.peers.is_empty())
+        {
+            return Self::Disconnected;
+        }
+        match report
+            .networks
+            .iter()
+            .find(|network| network.exit.via.is_some())
+        {
+            Some(network) if network.exit.via_online => Self::ExitNode,
+            Some(_) => Self::Disconnected,
+            None => Self::Connected,
+        }
+    }
+
+    /// The tooltip of the icon.
+    pub(crate) fn label(self) -> &'static str {
+        match self {
+            Self::Disconnected => "tsunagi: not connected",
+            Self::Connected => "tsunagi: connected",
+            Self::ExitNode => "tsunagi: using an exit node",
+        }
+    }
+}
+
+/// Starts asking the agent, and calls `notify` with every answer.
+pub(crate) fn spawn(notify: impl Fn(Health) + Send + 'static) {
+    let spawned = std::thread::Builder::new()
+        .name("tray-status".into())
+        .spawn(move || {
+            let Ok(runtime) = tokio::runtime::Builder::new_current_thread()
+                .enable_all()
+                .build()
+            else {
+                return;
+            };
+            loop {
+                let report = runtime.block_on(async {
+                    tokio::time::timeout(INTERVAL, ipc::request_status(resolve_socket()))
+                        .await
+                        .ok()
+                        .and_then(Result::ok)
+                });
+                notify(Health::of(report.as_ref()));
+                std::thread::sleep(INTERVAL);
+            }
+        });
+    if let Err(err) = spawned {
+        eprintln!("cannot watch the agent for the tray icon: {err}");
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use tsunagi::ipc::{ExitReport, NetworkReport, PeerReport};
+
+    fn network(peers: usize, exit: ExitReport) -> NetworkReport {
+        NetworkReport {
+            peers: vec![PeerReport::default(); peers],
+            exit,
+            ..Default::default()
+        }
+    }
+
+    fn report(networks: Vec<NetworkReport>) -> StatusReport {
+        StatusReport {
+            networks,
+            ..Default::default()
+        }
+    }
+
+    #[test]
+    fn no_agent_or_nobody_connected_is_the_red_state() {
+        assert_eq!(Health::of(None), Health::Disconnected);
+        assert_eq!(Health::of(Some(&report(vec![]))), Health::Disconnected);
+        assert_eq!(
+            Health::of(Some(&report(vec![network(0, ExitReport::default())]))),
+            Health::Disconnected
+        );
+    }
+
+    #[test]
+    fn a_connection_in_any_network_is_the_blue_state() {
+        let status = report(vec![
+            network(0, ExitReport::default()),
+            network(2, ExitReport::default()),
+        ]);
+        assert_eq!(Health::of(Some(&status)), Health::Connected);
+    }
+
+    #[test]
+    fn an_exit_node_in_use_is_green_while_it_works_and_red_once_it_is_gone() {
+        let using = |online| ExitReport {
+            via: Some("abc".into()),
+            via_online: online,
+            ..Default::default()
+        };
+        assert_eq!(
+            Health::of(Some(&report(vec![network(1, using(true))]))),
+            Health::ExitNode
+        );
+        assert_eq!(
+            Health::of(Some(&report(vec![network(1, using(false))]))),
+            Health::Disconnected
+        );
+    }
+}

crates/tsunagi/src/main.rs

@@ -2085,15 +2085,29 @@ async fn broadcast_command(
 }
 
 /// What to run to turn kernel forwarding on, which the agent never does.
+#[cfg(target_os = "macos")]
+const FORWARDING_COMMAND: &str = "sudo sysctl -w net.inet.ip.forwarding=1";
+#[cfg(target_os = "windows")]
+const FORWARDING_COMMAND: &str = "Get-NetIPInterface | Set-NetIPInterface -Forwarding Enabled   # in an administrator PowerShell";
+#[cfg(not(any(target_os = "macos", target_os = "windows")))]
 const FORWARDING_COMMAND: &str = "sudo sysctl -w net.ipv4.ip_forward=1";
 
+/// How to keep forwarding on across reboots.
+#[cfg(target_os = "macos")]
+const FORWARDING_PERSIST: &str = "macOS forgets it at reboot; set it again from a boot script \
+                                  such as a LaunchDaemon";
+#[cfg(target_os = "windows")]
+const FORWARDING_PERSIST: &str = "the setting is kept across reboots";
+#[cfg(not(any(target_os = "macos", target_os = "windows")))]
+const FORWARDING_PERSIST: &str =
+    "keep it across reboots with `net.ipv4.ip_forward = 1` in /etc/sysctl.d/99-tsunagi.conf";
+
 /// The warning for an exit node whose rules are in place while the kernel is
 /// not forwarding, so that nothing passes.
 fn forwarding_off_note() -> String {
     format!(
         "kernel forwarding is OFF, so nothing passes yet: run `{FORWARDING_COMMAND}` \
-         (and keep it across reboots with `net.ipv4.ip_forward = 1` in \
-         /etc/sysctl.d/99-tsunagi.conf)"
+         ({FORWARDING_PERSIST})"
     )
 }
 
@@ -5435,10 +5449,7 @@ mod status_tests {
         out.push(network_section(&network, OWN, false));
         let text = out.render(false);
         assert_eq!(out.worst(), Health::Degraded, "{text}");
-        assert!(
-            text.contains("sudo sysctl -w net.ipv4.ip_forward=1"),
-            "{text}"
-        );
+        assert!(text.contains(FORWARDING_COMMAND), "{text}");
         assert!(text.contains("ok (nat + forward)"), "{text}");
 
         network.exit.forwarding = Some(true);

dist/arch/build-pkg.sh

@@ -4,7 +4,7 @@
 #
 # Usage: build-pkg.sh <binary> <arch> <version> <output.pkg.tar.zst> [tray_binary]
 #   <binary>      path to the built tsunagi (agent/CLI) executable
-#   <arch>        pacman architecture: x86_64 or aarch64
+#   <arch>        pacman architecture: x86_64, aarch64 or armv7h
 #   <version>     package version (the tag without its leading v)
 #   <output>      path to write the package to
 #   [tray_binary] optional tsunagi-tray GUI executable; when given, the package

dist/deb/build-deb.sh

@@ -4,7 +4,7 @@
 #
 # Usage: build-deb.sh <binary> <deb_arch> <version> <output.deb> [tray_binary]
 #   <binary>      path to the built tsunagi (agent/CLI) executable
-#   <deb_arch>    Debian architecture: amd64 or arm64
+#   <deb_arch>    Debian architecture: amd64, arm64 or armhf
 #   <version>     package version (the tag without its leading v)
 #   <output>      path to write the .deb to
 #   [tray_binary] optional path to the tsunagi-tray GUI executable; when given,

dist/linux/tsunagi-tray.desktop

@@ -9,3 +9,4 @@ Terminal=false
 Categories=Network;Utility;
 Keywords=mesh;vpn;overlay;network;
 StartupNotify=false
+StartupWMClass=tsunagi-tray

dist/macos/README.md

@@ -27,6 +27,20 @@ shell started `sudo`:
 | disposable cache | `/var/db/tsunagi/cache` |
 | control socket | `/var/run/tsunagi/agent.sock` |
 
+## Homebrew
+
+```sh
+brew tap house-of-vanity/tap
+brew install --cask tsunagi-gui    # the tray app, and the agent it needs
+# or just the command line agent:
+brew install tsunagi
+sudo brew services start tsunagi   # the agent, as a root daemon at boot
+```
+
+The cask starts the service for you. The agent runs as root, and its control
+socket is open to the `admin` group, so the CLI and the tray work without
+`sudo` for an administrator.
+
 ## Native deployment — root LaunchDaemon
 
 For an always-on agent that starts at boot, install the bundled

dist/macos/build-app.sh

@@ -0,0 +1,63 @@
+#!/usr/bin/env bash
+# Wrap the tray GUI in a macOS application bundle, so it can be installed as an
+# app (a Homebrew cask, or dragging it to /Applications) instead of being a bare
+# executable.
+#
+# Usage: build-app.sh <tray_binary> <version> <output.app>
+#   <tray_binary>  the built tsunagi-tray executable
+#   <version>      the release version (the tag without its leading v)
+#   <output.app>   the bundle to create; replaced if it exists
+#
+# It is a menu-bar app: LSUIElement keeps it out of the Dock and the app
+# switcher, and the tray starts the window as a second process of the same
+# executable, which inherits that. The bundle is signed ad hoc where `codesign`
+# exists (the macOS runner), because arm64 will not run unsigned code; it is not
+# notarised, so a downloaded copy needs its quarantine flag cleared, which the
+# cask does.
+set -euo pipefail
+
+tray="$1"
+version="$2"
+app="$3"
+
+# CFBundleShortVersionString is dotted numbers only; 0.1.0-rc.22 becomes 0.1.0.
+short="${version%%-*}"
+
+rm -rf "$app"
+mkdir -p "$app/Contents/MacOS" "$app/Contents/Resources"
+install -m 0755 "$tray" "$app/Contents/MacOS/tsunagi-tray"
+
+cat > "$app/Contents/Info.plist" <<PLIST
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>CFBundleName</key>
+    <string>Tsunagi</string>
+    <key>CFBundleDisplayName</key>
+    <string>Tsunagi</string>
+    <key>CFBundleIdentifier</key>
+    <string>cy.hexor.tsunagi.tray</string>
+    <key>CFBundleExecutable</key>
+    <string>tsunagi-tray</string>
+    <key>CFBundlePackageType</key>
+    <string>APPL</string>
+    <key>CFBundleVersion</key>
+    <string>${short}</string>
+    <key>CFBundleShortVersionString</key>
+    <string>${short}</string>
+    <key>LSMinimumSystemVersion</key>
+    <string>11.0</string>
+    <key>LSUIElement</key>
+    <true/>
+    <key>NSHighResolutionCapable</key>
+    <true/>
+</dict>
+</plist>
+PLIST
+
+if command -v codesign >/dev/null 2>&1; then
+    codesign --force --deep --sign - "$app"
+fi
+
+echo "built $app"

dist/macos/cy.hexor.tsunagi.plist

@@ -36,6 +36,15 @@
     <key>ThrottleInterval</key>
     <integer>10</integer>
 
+    <!-- The control socket belongs to the admin group, so the CLI and the tray
+         work without sudo for any administrator. A plain account is not in it
+         and cannot manage the agent. -->
+    <key>EnvironmentVariables</key>
+    <dict>
+        <key>TSUNAGI_CONTROL_GROUP</key>
+        <string>admin</string>
+    </dict>
+
     <!-- Logs. launchd creates these as root; rotate them with newsyslog if you
          keep the daemon around. -->
     <key>StandardOutPath</key>

docs/exit-node.md

@@ -59,6 +59,49 @@ be an exit node in one network and not in another.
   specific IPv6 routes keep working, and so does the agent's own IPv6. A host
   with no IPv6 has nothing to block and is left alone.
 
+## macOS
+
+macOS has no `iptables` and no per-user policy routing, so the same behaviour
+is built from `pf` and `route`. The agent runs as root there.
+
+- **Offering:** a `pf` anchor `com.apple/tsunagi-exit-<interface>` (the stock
+  `/etc/pf.conf` already evaluates `com.apple/*`) holds
+  `nat on <egress> from <range> to any -> (<egress>)` and a `pass in` rule.
+  `pf` is enabled if it was off, and that reference is released when the rules
+  go. Forwarding is only read: `sudo sysctl -w net.inet.ip.forwarding=1`, which
+  macOS forgets at reboot. The egress is the default route's interface **at the
+  time the rules are applied**; after it changes (Wi-Fi to Ethernet) switch the
+  offer off and on.
+- **Using:** `0.0.0.0/1` and `128.0.0.0/1` routes through the overlay interface,
+  and a `pf` `route-to` rule that sends the agent's own traffic (matched by its
+  user id, root) back out the physical interface, with translation to that
+  interface's address. Anything else you run as root also bypasses the tunnel.
+  IPv6 is blocked with rejecting `::/1` and `8000::/1` routes.
+- Not verified on a live host when this was written: run `tsng exit-node` and
+  check `sudo pfctl -a com.apple/tsunagi-exit-utunN -s rules -s nat`.
+
+## Windows
+
+- **Offering:** each offered range gets a WinNAT object
+  `tsunagi-exit:<interface>:<range>` (`New-NetNat`), removed with the offer, the
+  network and the agent. Forwarding is read for the overlay interface and never
+  changed; Windows forwards per interface, so enable it on the ones involved
+  (`Get-NetIPInterface | Set-NetIPInterface -Forwarding Enabled`, administrator
+  PowerShell). Some Windows versions allow one NAT per host; a second range then
+  reports its own failure.
+- **Using:** Windows has no per-user routing, so the agent's own traffic is kept
+  off the tunnel *by destination*. `0.0.0.0/1` and `128.0.0.0/1` go through the
+  overlay interface; a `/32` route with metric 7373 (the marker for "ours")
+  through the physical default gateway is added for every address the agent
+  talks to directly: verified direct paths of its connections, plus the relays
+  it uses, resolved when the rules are applied. The set follows path changes. A
+  path that appears after the last apply goes through the tunnel until the next
+  one. IPv6 is blackholed with `::/1` and `8000::/1` on the overlay interface,
+  so applications fall back to IPv4 after a delay rather than at once.
+  Everything is in the active store: a reboot clears it.
+- Not verified on a live host when this was written. To inspect:
+  `Get-NetRoute -PolicyStore ActiveStore | ? { $_.RouteMetric -eq 7373 -or $_.InterfaceAlias -like 'tsun*' }`.
+
 ## What it does not do
 
 - **The agent must run as a system service** (its own user). The uid exemption