Reach a peer through one that can reach both

Two members of a mesh could both reach a third and not each other, and
that pair was simply lost to one another: a packet for a peer with no data
link was counted undeliverable and dropped. Now it goes through a member
that has both.

What travels is not routes. Each agent says only which peers *it* has a
live link with — first-hand, over the control plane, one hop, never a
claim about somebody else's reachability — and everybody computes their
own way through from that. The choice is local and deterministic (the
lowest endpoint id among the peers that have a link to the destination),
so there is nothing to agree, nothing to elect, and two agents may well
route each direction differently. It is soft state: repeated while it
holds, expired when it stops, so a relay that disappears stops being
chosen without anybody revoking anything.

The one in the middle carries bytes it cannot read. A datagram is wrapped
with the peer it is for, and unwrapped on the other side into the link for
the peer it came *from* — which matters, because a packet attributed to
the carrier would be dropped as coming from an address the carrier does
not hold. The tunnel stays end to end, and the relayed datagram goes link
in, link out: it never reaches the middle's interface, so no routing,
forwarding or firewall setting of that host is involved. One hop, so a
loop cannot form without counting anything.

A protocol is handed one link per peer that now outlives the paths under
it. A direct link that dies, a hop that changes, a direct link that comes
back: none of it tears down a tunnel any more, and the size a protocol may
use does not change with the path. Where there was never a direct link at
all, the link exists anyway as long as a hop does, so a peer reachable
only through somebody still gets a tunnel.

The data ALPN is `tsunagi/data/2`: every datagram now carries a tag saying
whether it is direct, for somebody else, or from somebody else. The local
control protocol is 13, for the relay counters — what this device carried
for others is their traffic on its uplink, and that should not be
invisible. `status` says `via <peer>` on a path through somebody.

Fairness between the peers a relay carries for is deliberately not here
yet: the queues are bounded and the counters are what a limit would be
built on.

Tested with fake links for the mechanics, and end to end with three real
agents — two that cannot reach each other directly, a real WireGuard
packet crossing through the middle. The one arrangement a single host
cannot produce by itself is a pair that cannot see each other, so that is
a `testing`-only switch on the agent config and exists in no release
build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
2026-09-22 01:32:43 +01:00
co-authored by Claude Opus 5
parent 3044e592bd
commit 3581feb9b9
14 changed files with 1262 additions and 6 deletions
+112
View File
@@ -162,6 +162,43 @@ impl WgAgent {
(agent, plugin, tuns)
}
/// An agent that refuses a direct data path to some peers.
///
/// The one arrangement a single host cannot produce by itself: two
/// agents that both reach a third and not each other. The control
/// plane is untouched — they are members and they talk — only the
/// direct data link is refused, which is the real-world case of a
/// blocked or unreachable data path.
async fn spawn_cut_off(
discovery: &SharedMemoryDiscovery,
tag: &str,
blocked: Arc<std::sync::Mutex<std::collections::HashSet<EndpointId>>>,
) -> Self {
let dir = TempDir::new().unwrap();
let tuns = MemoryTunFactory::new();
let plugin = WireguardPlugin::open(
WireguardConfig::new(dir.path().join("wireguard"))
.with_reconcile(Duration::from_millis(20), Duration::from_millis(250)),
)
.await
.unwrap();
let agent = Agent::spawn(
config_with(dir.path(), discovery)
.with_overlay_ipv4_range(Some(tsunagi::state::DEFAULT_IPV4_RANGE))
.with_interface(Arc::new(tuns.clone()), tag, 1280)
.with_unreachable_data_peers(blocked)
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
)
.await
.unwrap();
Self {
dir,
agent,
plugin,
tuns,
}
}
/// An agent whose interface claims to be on the host but is not.
///
/// Used only by the missing-address test: everywhere else the in-memory
@@ -1306,3 +1343,78 @@ async fn an_mtu_below_what_ipv4_guarantees_is_refused() {
.is_ok()
);
}
#[tokio::test]
async fn a_peer_with_no_direct_path_is_reached_through_one_that_has_both() {
// A and B can each reach C and not each other. Without a way through
// the middle they are lost to one another while sitting in the same
// mesh; with one, C carries their datagrams without being able to read
// a byte of them — the WireGuard tunnel is still end to end.
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("wg-relay");
let a_blocks = Arc::new(std::sync::Mutex::new(std::collections::HashSet::new()));
let b_blocks = Arc::new(std::sync::Mutex::new(std::collections::HashSet::new()));
let a = WgAgent::spawn_cut_off(&discovery, "tra", Arc::clone(&a_blocks)).await;
let b = WgAgent::spawn_cut_off(&discovery, "trb", Arc::clone(&b_blocks)).await;
let middle = WgAgent::spawn(&discovery, "trc").await;
a_blocks.lock().unwrap().insert(b.endpoint_id());
b_blocks.lock().unwrap().insert(a.endpoint_id());
let network_id = middle.agent.join_network(&name, &secret).await.unwrap();
a.agent.join_network(&name, &secret).await.unwrap();
b.agent.join_network(&name, &secret).await.unwrap();
// All three are members and all three talk: only the data path
// between A and B is missing.
wait_for_peers(&a.agent, network_id, 2).await;
wait_for_peers(&b.agent, network_id, 2).await;
middle.wait_for_tunnels(network_id, 2).await;
let a_addr = a.overlay(network_id).await;
let b_addr = b.overlay(network_id).await;
assert_ne!(a_addr, b_addr);
// A's tunnel to B comes up through C.
wait_until("a's tunnel to b is established", || async {
let view = a.plugin.overview(network_id)?;
view.peers
.iter()
.find(|peer| peer.endpoint_id == b.endpoint_id())?
.tunnel
.as_ref()
.map(|tunnel| tunnel.health.since_handshake)?
.map(|_| ())
})
.await;
let path = a
.plugin
.overview(network_id)
.unwrap()
.peers
.iter()
.find(|peer| peer.endpoint_id == b.endpoint_id())
.and_then(|peer| peer.tunnel.as_ref().map(|tunnel| tunnel.path.clone()))
.unwrap_or_default();
assert!(
path.contains("via"),
"the path should say it goes through somebody: {path}"
);
// And a real packet crosses: A's interface to B's interface, through C.
a.tun(network_id)
.await
.push_from_os(ipv4_packet(a_addr, b_addr, b"through the middle"));
let seen = tokio::time::timeout(
tsunagi::testing::DEADLINE,
b.tun(network_id).await.pop_to_os(),
)
.await
.expect("the packet should arrive")
.unwrap();
assert_eq!(&seen[20..], b"through the middle");
a.shutdown().await;
b.shutdown().await;
middle.shutdown().await;
}