Reach a peer through one that can reach both
Two members of a mesh could both reach a third and not each other, and that pair was simply lost to one another: a packet for a peer with no data link was counted undeliverable and dropped. Now it goes through a member that has both. What travels is not routes. Each agent says only which peers *it* has a live link with — first-hand, over the control plane, one hop, never a claim about somebody else's reachability — and everybody computes their own way through from that. The choice is local and deterministic (the lowest endpoint id among the peers that have a link to the destination), so there is nothing to agree, nothing to elect, and two agents may well route each direction differently. It is soft state: repeated while it holds, expired when it stops, so a relay that disappears stops being chosen without anybody revoking anything. The one in the middle carries bytes it cannot read. A datagram is wrapped with the peer it is for, and unwrapped on the other side into the link for the peer it came *from* — which matters, because a packet attributed to the carrier would be dropped as coming from an address the carrier does not hold. The tunnel stays end to end, and the relayed datagram goes link in, link out: it never reaches the middle's interface, so no routing, forwarding or firewall setting of that host is involved. One hop, so a loop cannot form without counting anything. A protocol is handed one link per peer that now outlives the paths under it. A direct link that dies, a hop that changes, a direct link that comes back: none of it tears down a tunnel any more, and the size a protocol may use does not change with the path. Where there was never a direct link at all, the link exists anyway as long as a hop does, so a peer reachable only through somebody still gets a tunnel. The data ALPN is `tsunagi/data/2`: every datagram now carries a tag saying whether it is direct, for somebody else, or from somebody else. The local control protocol is 13, for the relay counters — what this device carried for others is their traffic on its uplink, and that should not be invisible. `status` says `via <peer>` on a path through somebody. Fairness between the peers a relay carries for is deliberately not here yet: the queues are bounded and the counters are what a limit would be built on. Tested with fake links for the mechanics, and end to end with three real agents — two that cannot reach each other directly, a real WireGuard packet crossing through the middle. The one arrangement a single host cannot produce by itself is a pair that cannot see each other, so that is a `testing`-only switch on the agent config and exists in no release build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -162,6 +162,43 @@ impl WgAgent {
|
||||
(agent, plugin, tuns)
|
||||
}
|
||||
|
||||
/// An agent that refuses a direct data path to some peers.
|
||||
///
|
||||
/// The one arrangement a single host cannot produce by itself: two
|
||||
/// agents that both reach a third and not each other. The control
|
||||
/// plane is untouched — they are members and they talk — only the
|
||||
/// direct data link is refused, which is the real-world case of a
|
||||
/// blocked or unreachable data path.
|
||||
async fn spawn_cut_off(
|
||||
discovery: &SharedMemoryDiscovery,
|
||||
tag: &str,
|
||||
blocked: Arc<std::sync::Mutex<std::collections::HashSet<EndpointId>>>,
|
||||
) -> Self {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let tuns = MemoryTunFactory::new();
|
||||
let plugin = WireguardPlugin::open(
|
||||
WireguardConfig::new(dir.path().join("wireguard"))
|
||||
.with_reconcile(Duration::from_millis(20), Duration::from_millis(250)),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let agent = Agent::spawn(
|
||||
config_with(dir.path(), discovery)
|
||||
.with_overlay_ipv4_range(Some(tsunagi::state::DEFAULT_IPV4_RANGE))
|
||||
.with_interface(Arc::new(tuns.clone()), tag, 1280)
|
||||
.with_unreachable_data_peers(blocked)
|
||||
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
Self {
|
||||
dir,
|
||||
agent,
|
||||
plugin,
|
||||
tuns,
|
||||
}
|
||||
}
|
||||
|
||||
/// An agent whose interface claims to be on the host but is not.
|
||||
///
|
||||
/// Used only by the missing-address test: everywhere else the in-memory
|
||||
@@ -1306,3 +1343,78 @@ async fn an_mtu_below_what_ipv4_guarantees_is_refused() {
|
||||
.is_ok()
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_peer_with_no_direct_path_is_reached_through_one_that_has_both() {
|
||||
// A and B can each reach C and not each other. Without a way through
|
||||
// the middle they are lost to one another while sitting in the same
|
||||
// mesh; with one, C carries their datagrams without being able to read
|
||||
// a byte of them — the WireGuard tunnel is still end to end.
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("wg-relay");
|
||||
|
||||
let a_blocks = Arc::new(std::sync::Mutex::new(std::collections::HashSet::new()));
|
||||
let b_blocks = Arc::new(std::sync::Mutex::new(std::collections::HashSet::new()));
|
||||
let a = WgAgent::spawn_cut_off(&discovery, "tra", Arc::clone(&a_blocks)).await;
|
||||
let b = WgAgent::spawn_cut_off(&discovery, "trb", Arc::clone(&b_blocks)).await;
|
||||
let middle = WgAgent::spawn(&discovery, "trc").await;
|
||||
a_blocks.lock().unwrap().insert(b.endpoint_id());
|
||||
b_blocks.lock().unwrap().insert(a.endpoint_id());
|
||||
|
||||
let network_id = middle.agent.join_network(&name, &secret).await.unwrap();
|
||||
a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
|
||||
// All three are members and all three talk: only the data path
|
||||
// between A and B is missing.
|
||||
wait_for_peers(&a.agent, network_id, 2).await;
|
||||
wait_for_peers(&b.agent, network_id, 2).await;
|
||||
middle.wait_for_tunnels(network_id, 2).await;
|
||||
|
||||
let a_addr = a.overlay(network_id).await;
|
||||
let b_addr = b.overlay(network_id).await;
|
||||
assert_ne!(a_addr, b_addr);
|
||||
|
||||
// A's tunnel to B comes up through C.
|
||||
wait_until("a's tunnel to b is established", || async {
|
||||
let view = a.plugin.overview(network_id)?;
|
||||
view.peers
|
||||
.iter()
|
||||
.find(|peer| peer.endpoint_id == b.endpoint_id())?
|
||||
.tunnel
|
||||
.as_ref()
|
||||
.map(|tunnel| tunnel.health.since_handshake)?
|
||||
.map(|_| ())
|
||||
})
|
||||
.await;
|
||||
let path = a
|
||||
.plugin
|
||||
.overview(network_id)
|
||||
.unwrap()
|
||||
.peers
|
||||
.iter()
|
||||
.find(|peer| peer.endpoint_id == b.endpoint_id())
|
||||
.and_then(|peer| peer.tunnel.as_ref().map(|tunnel| tunnel.path.clone()))
|
||||
.unwrap_or_default();
|
||||
assert!(
|
||||
path.contains("via"),
|
||||
"the path should say it goes through somebody: {path}"
|
||||
);
|
||||
|
||||
// And a real packet crosses: A's interface to B's interface, through C.
|
||||
a.tun(network_id)
|
||||
.await
|
||||
.push_from_os(ipv4_packet(a_addr, b_addr, b"through the middle"));
|
||||
let seen = tokio::time::timeout(
|
||||
tsunagi::testing::DEADLINE,
|
||||
b.tun(network_id).await.pop_to_os(),
|
||||
)
|
||||
.await
|
||||
.expect("the packet should arrive")
|
||||
.unwrap();
|
||||
assert_eq!(&seen[20..], b"through the middle");
|
||||
|
||||
a.shutdown().await;
|
||||
b.shutdown().await;
|
||||
middle.shutdown().await;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user