ab/tsunagi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Answer DNS over both families, and only for our own interface

Browse Source 
Questions now arrive over IPv4 or IPv6, whichever the resolver uses. The
server opens one listener per family — the overlay address or `127.0.0.1`,
and `[::1]` — and all of them are published to systemd-resolved in one
call, which is what that call requires: sending them one at a time leaves
only the last. A family that cannot be bound, IPv6 switched off in the
kernel for instance, no longer stops the other from answering.

The answers stay IPv4, because that is what the overlay is. A listening
address is disposable, unlike an address a member holds in signed state,
so serving one family over the overlay and the other over loopback costs
nothing and loses nothing.

`--no-tun` was also configuring the host. An in-memory interface has a
name and an MTU and nothing else, but everything downstream read that name
as a host interface: the resolver setting was pushed onto whatever else on
the host happened to be called `tsun0` — which, with two agents on one
machine, is another agent's live interface. A factory now says whether
what it creates is on the host, and the resolver setting goes only to an
interface the agent created. For the same reason the complaint that "the
allocated address is not on any interface" no longer fires under
`--no-tun`, where there was never going to be one; it had people looking
for something that had removed their address. The status line says
`tsun0 (in memory, --no-tun)` rather than printing an address beside a name
the operating system does not have.

That distinction also corrected a test that used the in-memory interface
as a stand-in for an unconfigured host interface. They are not the same
case, so the test now uses a factory that claims the host and puts nothing
there — a provisioner that reported a success it did not achieve — and a
second test covers `--no-tun` being an arrangement rather than a fault.

An in-memory device now reports end of stream when it is destroyed. It
never did, so the packet loop reading it could not end, and since shutdown
became bounded that cost every `--no-tun` agent the full five-second grace
before the loop was aborted instead of finishing.

And `status` says when there is no local resolver at all. Its absence is
the answer to "why does this name not resolve?", and leaving the section
out made a report with DNS switched off look exactly like one where it was
running.

The local control protocol is 7: `DnsReport` carries a list of listening
addresses and `OverlayReport` says whether the interface is on the host.

Verified against the running systemd-resolved: it takes
`127.0.0.1:5354 [::1]:5354` on one link in a single call, and forward and
reverse questions for a peer's name are answered identically over both
transports.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
tsunagi
2026-09-21 21:21:55 +01:00
co-authored by Claude Opus 5
parent 0b3915d52b
commit fae3816bbf
17 changed files with 587 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+14 -5
View File
@@ -195,11 +195,15 @@ dig @10.13.37.69 -p 5354 music.lab
```
Names come from signed state, which is the point: **a member that is
switched off still resolves**, because its claim outlived the session. IPv4
only — the IPv6 overlay address derives from a key that travels in live
announcements, so it cannot be answered for a member that is away, and
answering for some members and not others depending on who is online is
worse than not answering.
switched off still resolves**, because its claim outlived the session.
The answers are IPv4 addresses, because that is what the overlay is. The
*questions* are taken over both families, on UDP and TCP: the server listens
on the overlay address and on `127.0.0.1`, and on `[::1]` for IPv6, so a
resolver reaches it over whichever it uses. All of those are published to
the system resolver together. A listening address is disposable — unlike an
address a member holds in signed state — so serving one family over the
overlay and the other over loopback costs nothing.
The zone is the network name unless `--dns-zone` says otherwise. It is
yours to choose, so a name that shadows a real public domain is reported and
@@ -212,6 +216,11 @@ here, over D-Bus, scoped to the overlay interface and as a *routing* domain
so it never becomes the resolver for anything else. resolved drops the whole
setting when the interface goes, and the interface goes with the agent.
Only an interface the agent created, though. Under `--no-tun` there is no
host interface at all, and the agent says so and serves the zone on
loopback rather than configuring whatever else on the host happens to share
the name.
That last step needs permission that `CAP_NET_ADMIN` does not give:
systemd-resolved asks polkit, and polkit decides by **user**, not by
capability, so there is no way for the agent to arrange it from inside. On
crates/tsunagi-cli/src/main.rs
+184 -51
View File
@@ -282,7 +282,9 @@ struct UpArgs {
/// Serve a local DNS zone for this network's members.
///
/// Members resolve as `<hostname>.<zone>`, from signed state, so a
/// member that is switched off still resolves. IPv4 only.
/// member that is switched off still resolves. The answers are the
/// overlay's IPv4 addresses; questions are taken over both IPv4 and
/// IPv6, on UDP and TCP.
#[arg(long, help_heading = "System")]
dns: bool,
@@ -290,7 +292,7 @@ struct UpArgs {
#[arg(long, value_name = "NAME", help_heading = "System")]
dns_zone: Option<String>,
/// Port for the local DNS server.
/// Port for the local DNS server, on every address it listens on.
#[arg(long, default_value_t = 5354, help_heading = "System")]
dns_port: u16,
@@ -522,7 +524,7 @@ fn configured_networks_section(paths: &StoragePaths) -> report::Section {
#[derive(Debug, Clone, Default)]
struct DnsState {
zone: String,
listening: Option<SocketAddr>,
listening: Vec<SocketAddr>,
bind_error: Option<String>,
publish_error: Option<String>,
publish_remedy: Option<String>,
@@ -571,7 +573,7 @@ fn spawn_dns(
zone: tsunagi::dns::ZoneName,
port: u16,
) -> DnsService {
use tsunagi::dns::{DnsServer, SharedZone, Zone, listen_addresses};
use tsunagi::dns::{DnsServer, SharedZone, Zone, listen_plan};
let state = Arc::new(std::sync::Mutex::new(DnsState {
zone: zone.as_str().to_string(),
@@ -585,15 +587,17 @@ fn spawn_dns(
let publisher = Arc::clone(&publisher);
tokio::spawn(async move {
let shared = SharedZone::new(Zone::new(zone.clone(), []));
// Held for its `Drop`, which stops the server: the value is
// never read, but letting it go is what closes the socket.
let mut _server: Option<DnsServer> = None;
let mut bound: Option<SocketAddr> = None;
// Held for their `Drop`, which stops each server: the values are
// never read, but letting them go is what closes the sockets.
// One per address family, so a question is answered over
// whichever the resolver uses.
let mut _servers: Vec<DnsServer> = Vec::new();
let mut bound: Vec<SocketAddr> = Vec::new();
// What was tried last time, not what was got. Comparing against
// what was got would rebind on every tick whenever the preferred
// address is one that cannot be bound, closing the port each
// time for no reason.
let mut attempted: Vec<SocketAddr> = Vec::new();
let mut attempted: Option<tsunagi::dns::ListenPlan> = None;
let mut published: Option<tsunagi::dns::Published> = None;
// A condition that persists is worth saying once, not every
// pass; and a refusal will not lift without somebody acting, so
@@ -627,65 +631,79 @@ fn spawn_dns(
.find(|member| member.endpoint_id == own)
.and_then(|member| member.overlay_address_v4);
// The interface belongs to the agent, so the resolver
// setting attaches to that one and not to a protocol's.
// setting attaches to that one and not to a protocol's. Only
// one that is really on the host: an in-memory interface has
// a name and nothing else, and telling the operating system
// about that name would configure whatever else happens to
// be called it.
let interface = agent
.overlay()
.filter(|overlay| overlay.on_host)
.map(|overlay| overlay.interface)
.filter(|name| !name.is_empty());
let wanted = listen_addresses(overlay, port);
if attempted != wanted {
attempted = wanted.clone();
// Dropping the old one first releases the port, so the
let wanted = listen_plan(overlay, port);
if attempted.as_ref() != Some(&wanted) {
attempted = Some(wanted.clone());
// Dropping the old ones first releases the port, so the
// rebind is not racing itself.
_server = None;
_servers.clear();
let mut last: Option<std::io::Error> = None;
bound = None;
for candidate in &wanted {
match DnsServer::bind(*candidate, shared.clone()).await {
Ok(fresh) => {
tracing::info!(
address = %fresh.local_addr(),
zone = %zone.as_str(),
"dns listening"
);
bound = Some(fresh.local_addr());
_server = Some(fresh);
break;
bound.clear();
// Each family on its own: one of them being unavailable
// — IPv6 switched off, an address not on an interface —
// is no reason to answer on neither.
for family in wanted.families() {
for candidate in family {
match DnsServer::bind(*candidate, shared.clone()).await {
Ok(fresh) => {
tracing::info!(
address = %fresh.local_addr(),
zone = %zone.as_str(),
"dns listening"
);
bound.push(fresh.local_addr());
_servers.push(fresh);
break;
}
Err(err) => last = Some(err),
}
Err(err) => last = Some(err),
}
}
let bind_error = bound.is_none().then(|| {
let bind_error = bound.is_empty().then(|| {
last.map_or_else(
|| "no address to listen on".to_string(),
|err| err.to_string(),
)
});
let listening = bound.clone();
update(&state, |state| {
state.listening = bound;
state.listening = listening;
state.bind_error = bind_error;
});
// The address moved, so whatever the resolver was told
// The addresses moved, so whatever the resolver was told
// is now wrong.
published = None;
}
let Some(address) = bound else { continue };
if bound.is_empty() {
continue;
}
let Some(interface) = interface else {
update(&state, |state| {
state.publish_error = Some(
"there is no overlay interface to attach the resolver setting to"
"there is no overlay interface on this host to attach the resolver \
setting to"
.to_string(),
);
state.publish_remedy = None;
state.names = names;
});
update(&state, |state| state.names = names);
continue;
};
let want_published = tsunagi::dns::Published {
interface,
server: address,
servers: bound.clone(),
domains: vec![zone.as_str().to_string()],
};
let due = retry_after.is_none_or(|at| tokio::time::Instant::now() >= at);
@@ -1156,10 +1174,11 @@ async fn status(args: StatusArgs) -> Result<(), Box<dyn std::error::Error>> {
Observed::Stored { .. } => out.push(configured_networks_section(&paths)),
}
if let Observed::Agent(report) = &observed
&& let Some(dns) = &report.dns
{
out.push(dns_section(dns));
if let Observed::Agent(report) = &observed {
out.push(match &report.dns {
Some(dns) => dns_section(dns),
None => dns_absent_section(),
});
}
out.push(host_section());
@@ -1181,20 +1200,23 @@ fn dns_section(dns: &tsunagi::ipc::DnsReport) -> report::Section {
section.push(Row::new(Health::Degraded, "zone name", warning.clone()));
}
match (&dns.listening, &dns.bind_error) {
(Some(address), _) => {
section.push(Row::new(Health::Good, "listening", address.clone()));
}
(None, Some(err)) => {
match (dns.listening.as_slice(), &dns.bind_error) {
([], Some(err)) => {
section.push(Row::new(Health::Broken, "listening", err.clone()));
}
(None, None) => {
([], None) => {
section.push(Row::new(Health::Degraded, "listening", "not yet"));
}
// One address per family it could open. Both is the ordinary case;
// one is worth seeing rather than hiding, because then a question
// over the other family goes unanswered.
(addresses, _) => {
section.push(Row::new(Health::Good, "listening", addresses.join(", ")));
}
}
match &dns.publish_error {
None if dns.listening.is_some() => {
None if !dns.listening.is_empty() => {
section.push(Row::new(
Health::Good,
"system resolver",
@@ -1206,7 +1228,7 @@ fn dns_section(dns: &tsunagi::ipc::DnsReport) -> report::Section {
// The server still answers, so this is a degraded overlay and
// not a broken one; what is missing is the automatic part.
let row = Row::new(Health::Degraded, "system resolver", err.clone());
section.push(match (&dns.publish_remedy, &dns.listening) {
section.push(match (&dns.publish_remedy, dns.listening.first()) {
(Some(remedy), _) => row.with_note(remedy.clone()),
(None, Some(address)) => row.with_note(format!(
"resolve names yourself with `dig @{} -p {} <name>.{}`",
@@ -1221,6 +1243,30 @@ fn dns_section(dns: &tsunagi::ipc::DnsReport) -> report::Section {
section
}
/// Says that there is no local resolver, when there is none.
///
/// Its absence is why a name does not resolve, and nothing else in the
/// report says so: a missing section reads as nothing to report rather than
/// as a feature that was never asked for.
fn dns_absent_section() -> report::Section {
use report::{Health, Row, Section};
let mut section = Section::new("dns");
section.push(
Row::new(
Health::Info,
"not serving",
"no local resolver for any network",
)
.with_note(
"members resolve by address only. `tsunagi up --dns` serves \
`<hostname>.<network>` from signed state, so a member that is switched \
off still resolves.",
),
);
section
}
/// One member of a network, from every source that knows something about it.
///
/// The three sources answer different questions and none of them answers the
@@ -1366,12 +1412,23 @@ fn network_section(
}
if let Some(overlay) = &network.overlay {
let interface = if overlay.on_host {
overlay.interface.clone()
} else {
// The name is real to the agent and to nothing else. Said here,
// because an address on an interface the operating system does
// not have explains every ping that goes nowhere.
format!("{} (in memory, --no-tun)", overlay.interface)
};
section.push(Row::new(
Health::Info,
if overlay.on_host {
Health::Info
} else {
Health::Degraded
},
"overlay",
format!(
"{} {} mtu {}",
overlay.interface,
"{interface} {} mtu {}",
match &overlay.address {
Some(address) => format!("{address}/{}", overlay.prefix_len),
None => "no address agreed yet".to_string(),
@@ -2256,7 +2313,11 @@ async fn build_report(
let overlay = agent.overlay();
let dns = dns.map(|dns| DnsReport {
zone: dns.zone,
listening: dns.listening.map(|address| address.to_string()),
listening: dns
.listening
.iter()
.map(|address| address.to_string())
.collect(),
bind_error: dns.bind_error,
publish_error: dns.publish_error,
publish_remedy: dns.publish_remedy,
@@ -2278,6 +2339,7 @@ async fn build_report(
interface: overlay
.as_ref()
.map_or_else(String::new, |overlay| overlay.interface.clone()),
on_host: overlay.as_ref().is_some_and(|overlay| overlay.on_host),
mtu: overlay.as_ref().map_or(0, |overlay| overlay.mtu),
address: view.overlay_address_v4.map(|addr| addr.to_string()),
prefix_len: view.ipv4_range.map_or(0, |range| range.prefix_len),
@@ -2583,6 +2645,9 @@ mod status_tests {
fn overlay(peers: Vec<OverlayPeerReport>) -> OverlayReport {
OverlayReport {
interface: "tsundemo".into(),
// A real interface, which is the ordinary case; the in-memory
// one has a test of its own.
on_host: true,
mtu: 1280,
address: Some("10.13.37.69".into()),
prefix_len: 24,
@@ -2678,6 +2743,74 @@ mod status_tests {
assert!(text.contains("--ipv4-range"), "the fix is named: {text}");
}
#[test]
fn an_in_memory_interface_is_not_presented_as_a_host_interface() {
// `--no-tun` runs the tunnels and moves packets between agents, but
// the operating system has no interface, no address and no route. An
// address printed beside a name the host does not have is what makes
// a ping that goes nowhere look like a network fault.
let mut network = network_after_a_peer_returned();
if let Some(overlay) = &mut network.overlay {
overlay.on_host = false;
}
let mut out = report::Report::new();
out.push(network_section(&network, OWN, false));
let text = out.render(false);
assert!(text.contains("in memory"), "{text}");
assert!(text.contains("--no-tun"), "the reason is named: {text}");
assert_eq!(out.worst(), Health::Degraded, "{text}");
}
#[test]
fn with_no_local_resolver_the_report_says_so_rather_than_nothing() {
// The absence is the answer to "why does the name not resolve?".
// Left out, the report looked the same as one where DNS was running.
let mut out = report::Report::new();
out.push(dns_absent_section());
let text = out.render(false);
assert!(text.contains("not serving"), "{text}");
assert!(text.contains("--dns"), "the flag that starts it: {text}");
// Nothing is wrong with an agent that was never asked to serve DNS.
assert_eq!(out.worst(), Health::Good, "{text}");
}
#[test]
fn both_families_are_listed_while_only_one_bound_is_still_good() {
use tsunagi::ipc::DnsReport;
let both = DnsReport {
zone: "lab".into(),
listening: vec!["10.13.37.69:5354".into(), "[::1]:5354".into()],
names: 2,
..Default::default()
};
let mut out = report::Report::new();
out.push(dns_section(&both));
let text = out.render(false);
assert!(text.contains("10.13.37.69:5354, [::1]:5354"), "{text}");
// One family is worth seeing rather than hiding: a question over the
// other one goes unanswered.
let one = DnsReport {
listening: vec!["127.0.0.1:5354".into()],
..both.clone()
};
let mut out = report::Report::new();
out.push(dns_section(&one));
assert!(out.render(false).contains("127.0.0.1:5354"));
// Neither, with a reason, is broken.
let none = DnsReport {
listening: Vec::new(),
bind_error: Some("address already in use".into()),
..both
};
let mut out = report::Report::new();
out.push(dns_section(&none));
assert_eq!(out.worst(), Health::Broken, "{}", out.render(false));
}
#[test]
fn counters_from_the_past_do_not_grade_the_present() {
// A peer that left and returned leaves dial failures and a packet
crates/tsunagi-wg-quic/tests/wireguard.rs
+112 -7
View File
@@ -19,7 +19,9 @@ use tsunagi::agent::Event;
use tsunagi::dataplane::IpPlugin;
use tsunagi::discovery::SharedMemoryDiscovery;
use tsunagi::identity::{NetworkId, NetworkName, NetworkSecret};
use tsunagi::overlay::{MemoryTun, MemoryTunFactory};
use tsunagi::overlay::{
MemoryTun, MemoryTunFactory, OverlayError, TunDevice, TunFactory, TunRequest,
};
use tsunagi::state::Ipv4Range;
use tsunagi::testing::{config_with, network, settle, wait_event, wait_for_peers, wait_until};
use tsunagi::{Agent, NetworkStatus};
@@ -27,6 +29,45 @@ use tsunagi_wg_quic::{
WIREGUARD_PROTOCOL, WgAnnouncement, WgSecretKey, WireguardConfig, WireguardPlugin,
};
/// A factory that claims the host and puts nothing on it.
///
/// This is the case the missing-address report exists for: a provisioner
/// that returned success without achieving it, or something outside that
/// removed the address afterwards. An in-memory interface is *not* that
/// case — it has no host side at all, and treating the two as one is what
/// had the agent reporting a fault about `--no-tun` working as intended,
/// and configuring host interfaces it never created.
#[derive(Debug, Clone)]
struct PretendHostTuns(MemoryTunFactory);
impl TunFactory for PretendHostTuns {
fn name(&self) -> &str {
"pretend-host"
}
fn on_host(&self) -> bool {
true
}
fn create<'a>(
&'a self,
request: TunRequest,
) -> tsunagi::BoxFuture<'a, Result<Arc<dyn TunDevice>, OverlayError>> {
self.0.create(request)
}
fn reconfigure<'a>(
&'a self,
request: TunRequest,
) -> tsunagi::BoxFuture<'a, Result<(), OverlayError>> {
self.0.reconfigure(request)
}
fn destroy<'a>(&'a self, name: &'a str) -> tsunagi::BoxFuture<'a, ()> {
self.0.destroy(name)
}
}
/// An agent with a WireGuard plugin backed by an in-memory packet interface.
struct WgAgent {
dir: TempDir,
@@ -121,6 +162,35 @@ impl WgAgent {
(agent, plugin, tuns)
}
/// An agent whose interface claims to be on the host but is not.
///
/// Used only by the missing-address test: everywhere else the in-memory
/// interface is honest about what it is.
async fn spawn_pretending_host(discovery: &SharedMemoryDiscovery, tag: &str) -> Self {
let dir = TempDir::new().unwrap();
let tuns = MemoryTunFactory::new();
let plugin = WireguardPlugin::open(
WireguardConfig::new(dir.path().join("wireguard"))
.with_reconcile(Duration::from_millis(20), Duration::from_millis(250)),
)
.await
.unwrap();
let agent = Agent::spawn(
config_with(dir.path(), discovery)
.with_overlay_ipv4_range(Some(tsunagi::state::DEFAULT_IPV4_RANGE))
.with_interface(Arc::new(PretendHostTuns(tuns.clone())), tag, 1280)
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
)
.await
.unwrap();
Self {
dir,
agent,
plugin,
tuns,
}
}
fn endpoint_id(&self) -> EndpointId {
self.agent.endpoint_id()
}
@@ -445,12 +515,13 @@ async fn an_allocated_address_missing_from_the_host_is_reported() {
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("wg-missing-address");
// The in-memory interface never carries the address, which is exactly
// the situation of a real interface the operator has not configured yet.
// Left unsaid, packets leave with the wrong source and every peer drops
// them, which looks like a broken network rather than a missing command.
let a = WgAgent::spawn(&discovery, "ta").await;
let b = WgAgent::spawn(&discovery, "tb").await;
// An interface that says it is on the host and never carries the
// address: a provisioner that reported a success it did not achieve, or
// something outside that took the address away. Left unsaid, packets
// leave with the wrong source and every peer drops them, which looks
// like a broken network rather than a missing command.
let a = WgAgent::spawn_pretending_host(&discovery, "ta").await;
let b = WgAgent::spawn_pretending_host(&discovery, "tb").await;
let mut events = a.agent.subscribe();
let network_id = a.agent.join_network(&name, &secret).await.unwrap();
@@ -484,6 +555,40 @@ async fn an_allocated_address_missing_from_the_host_is_reported() {
b.shutdown().await;
}
#[tokio::test]
async fn an_interface_that_is_only_in_memory_is_not_reported_as_a_fault() {
let discovery = SharedMemoryDiscovery::new();
let (name, secret) = network("wg-memory-interface");
// `--no-tun` is an arrangement, not a failure: the tunnels run and the
// packets move between agents, and nothing was ever going to put an
// address on a host interface that does not exist. Complaining about it
// sent people looking for something that had removed their address.
let a = WgAgent::spawn(&discovery, "ta").await;
let b = WgAgent::spawn(&discovery, "tb").await;
let mut events = a.agent.subscribe();
let network_id = a.agent.join_network(&name, &secret).await.unwrap();
b.agent.join_network(&name, &secret).await.unwrap();
a.wait_for_tunnels(network_id, 1).await;
// An address was allocated, so the report had every other reason to fire.
let address = a.overlay(network_id).await;
assert!(tsunagi::state::DEFAULT_IPV4_RANGE.contains(address));
settle().await;
let complaint = std::iter::from_fn(|| events.try_recv().ok()).find(
|event| matches!(event, Event::PluginError { reason, .. } if reason.contains("not on any")),
);
assert!(complaint.is_none(), "unexpected: {complaint:?}");
// And the interface says plainly what it is, which is what keeps the
// resolver setting off a host interface of the same name.
assert!(!a.agent.overlay().unwrap().on_host);
a.shutdown().await;
b.shutdown().await;
}
#[tokio::test]
async fn an_address_is_kept_across_a_restart() {
let discovery = SharedMemoryDiscovery::new();
crates/tsunagi/src/agent/mod.rs
+1
View File
@@ -381,6 +381,7 @@ impl Agent {
let interface = self.inner.interface.get()?;
Some(OverlayStatus {
interface: interface.name().to_string(),
on_host: interface.on_host(),
mtu: interface.mtu(),
addresses: interface.wanted_addresses(),
counters: interface.counters(),
crates/tsunagi/src/agent/network.rs
+6 -1
View File
@@ -904,8 +904,13 @@ impl Runtime {
// source and every peer would drop them. Checked rather than
// assumed, because the assumption is exactly the kind that has been
// wrong here before. Said once per address, not every round.
// An in-memory interface has no addresses to be missing from: with
// `--no-tun` this is the arrangement asked for, not a fault.
let missing = match local {
Some(address) if !crate::overlay::address_is_local(std::net::IpAddr::V4(address)) => {
Some(address)
if interface.on_host()
&& !crate::overlay::address_is_local(std::net::IpAddr::V4(address)) =>
{
let already = self.reported_missing == Some(address);
self.reported_missing = Some(address);
(!already).then_some(address)
crates/tsunagi/src/agent/status.rs
+6
View File
@@ -41,6 +41,12 @@ pub struct CandidateStatus {
pub struct OverlayStatus {
/// The interface name the operating system gave.
pub interface: String,
/// Whether that interface exists on the host.
///
/// `false` with an in-memory device: the name is real to the agent and
/// to nothing else. Reported, because everything that would configure
/// the operating system for this interface must not when it is this.
pub on_host: bool,
/// Its MTU.
pub mtu: u32,
/// The addresses it should be carrying.
crates/tsunagi/src/dns/mod.rs
+55 -13
View File
@@ -18,13 +18,19 @@ pub mod zone;
pub use publish::{DnsPublisher, PublishError, Published};
use std::net::{Ipv4Addr, SocketAddr};
use std::net::{Ipv4Addr, Ipv6Addr, SocketAddr};
/// Where the server should try to listen, best first.
/// Where the server should try to listen, one list per address family.
///
/// The overlay address comes first: that is what the system resolver is
/// pointed at, and it is reachable only over the overlay interface, so a
/// question for these names cannot arrive from anywhere else.
/// Both families are served, and independently: a question arrives over
/// whichever one the resolver happens to use, and on a host where one of
/// them is switched off the other must still answer. So neither list failing
/// says anything about the other, and a listener is opened from each.
///
/// Within a list the order is best first. The overlay address comes first:
/// that is what the system resolver is pointed at, and it is reachable only
/// over the overlay interface, so a question for these names cannot arrive
/// from anywhere else.
///
/// Loopback second, and it is not merely a fallback for having no overlay
/// address. An address this agent has been *allocated* is not necessarily an
@@ -32,13 +38,38 @@ use std::net::{Ipv4Addr, SocketAddr};
/// or in the moment before the interface is configured, it is not — and
/// binding to one that is not there fails. Trying loopback afterwards is
/// what keeps the promise that the port comes up regardless.
pub fn listen_addresses(overlay: Option<Ipv4Addr>, port: u16) -> Vec<SocketAddr> {
let mut candidates = Vec::with_capacity(2);
///
/// The overlay itself is IPv4, so there is no overlay address to offer for
/// IPv6 and that list is loopback alone. That is a property of the overlay
/// and not of this: a listening address is disposable, unlike an address a
/// member holds in signed state, so serving one family over loopback and the
/// other over the overlay costs nothing and loses nothing.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ListenPlan {
/// IPv4 candidates, best first.
pub v4: Vec<SocketAddr>,
/// IPv6 candidates, best first.
pub v6: Vec<SocketAddr>,
}
impl ListenPlan {
/// The two lists, so a caller can bind one listener from each.
pub fn families(&self) -> [&[SocketAddr]; 2] {
[&self.v4, &self.v6]
}
}
/// Builds the listen plan for one port.
pub fn listen_plan(overlay: Option<Ipv4Addr>, port: u16) -> ListenPlan {
let mut v4 = Vec::with_capacity(2);
if let Some(overlay) = overlay {
candidates.push(SocketAddr::from((overlay, port)));
v4.push(SocketAddr::from((overlay, port)));
}
v4.push(SocketAddr::from((Ipv4Addr::LOCALHOST, port)));
ListenPlan {
v4,
v6: vec![SocketAddr::from((Ipv6Addr::LOCALHOST, port))],
}
candidates.push(SocketAddr::from((Ipv4Addr::LOCALHOST, port)));
candidates
}
pub use server::{DnsServer, SharedZone};
@@ -51,20 +82,31 @@ mod tests {
use super::*;
#[test]
fn the_overlay_is_preferred_and_loopback_is_always_offered() {
fn both_families_are_planned_for_and_the_overlay_is_preferred() {
let plan = listen_plan(Some(Ipv4Addr::new(10, 13, 37, 69)), 5354);
// Loopback is in the list even when there is an overlay address,
// because being allocated one is not the same as it being on an
// interface — and binding to one that is not there fails.
assert_eq!(
listen_addresses(Some(Ipv4Addr::new(10, 13, 37, 69)), 5354),
plan.v4,
vec![
"10.13.37.69:5354".parse::<SocketAddr>().unwrap(),
"127.0.0.1:5354".parse::<SocketAddr>().unwrap(),
]
);
// IPv6 is served too, over loopback: the overlay has no IPv6
// address to offer, and that is no reason to answer only one family.
assert_eq!(plan.v6, vec!["[::1]:5354".parse::<SocketAddr>().unwrap()]);
assert_eq!(plan.families().len(), 2);
}
#[test]
fn with_no_overlay_address_each_family_still_has_somewhere_to_listen() {
let plan = listen_plan(None, 5354);
assert_eq!(
listen_addresses(None, 5354),
plan.v4,
vec!["127.0.0.1:5354".parse::<SocketAddr>().unwrap()]
);
assert_eq!(plan.v6, vec!["[::1]:5354".parse::<SocketAddr>().unwrap()]);
}
}
crates/tsunagi/src/dns/publish/mock.rs
+4 -1
View File
@@ -74,7 +74,10 @@ mod tests {
fn published() -> Published {
Published {
interface: "tsundemo".into(),
server: "10.13.37.69:5354".parse().unwrap(),
servers: vec![
"10.13.37.69:5354".parse().unwrap(),
"[::1]:5354".parse().unwrap(),
],
domains: vec!["lab".into()],
}
}
crates/tsunagi/src/dns/publish/mod.rs
+9 -3
View File
@@ -32,10 +32,16 @@ pub struct Published {
/// The interface questions should be sent through.
///
/// The server listens on an overlay address, which is only reachable
/// over the overlay interface, so the two travel together.
/// over the overlay interface, so the two travel together. It must be an
/// interface this agent created: configuring one it did not is
/// configuring somebody else's.
pub interface: String,
/// Where the server is listening.
pub server: SocketAddr,
/// Every address the server is listening on, best first.
///
/// One per address family, so a resolver reaches it over whichever it
/// uses. All of them are published together and none is preferred by
/// this: which one a resolver picks is its business.
pub servers: Vec<SocketAddr>,
/// The suffixes that belong to this server.
///
/// Routing suffixes only: they say *which questions* come here, never
crates/tsunagi/src/dns/publish/resolved.rs
+31 -14
View File
@@ -135,36 +135,53 @@ impl DnsPublisher for ResolvedPublisher {
published.interface
))
})?;
if published.servers.is_empty() {
return Err(PublishError::Unavailable(
"there is no listening address to send questions to".to_string(),
));
}
let proxy = Self::proxy().await?;
let index = ifindex as i32;
let (family, octets) = wire_address(published.server.ip());
let port = published.server.port();
// Every family in one call, because this replaces the link's
// whole list: sending them one at a time would leave only the
// last. A resolver then picks whichever it can reach.
let servers: Vec<(i32, Vec<u8>, u16, String)> = published
.servers
.iter()
.map(|server| {
let (family, octets) = wire_address(server.ip());
(family, octets, server.port(), String::new())
})
.collect();
match proxy
.set_link_dns_ex(index, &[(family, octets.clone(), port, String::new())])
.await
{
match proxy.set_link_dns_ex(index, &servers).await {
Ok(()) => {}
Err(err) => {
let classified = classify(err, "setting the link's DNS server");
let classified = classify(err, "setting the link's DNS servers");
// Older systemd has no `Ex` form, and the plain one is
// always port 53. Falling back to it when the server is
// somewhere else would point resolved at nothing.
if !matches!(classified, PublishError::Unavailable(_)) {
return Err(classified);
}
if port != 53 {
if let Some(elsewhere) =
published.servers.iter().find(|server| server.port() != 53)
{
return Err(PublishError::Unavailable(format!(
"this systemd-resolved cannot be given a port, and the server is on \
{port}. Run the server on port 53, or point your resolver at \
{} yourself.",
published.server
{}. Run the server on port 53, or point your resolver at \
{elsewhere} yourself.",
elsewhere.port()
)));
}
let plain: Vec<(i32, Vec<u8>)> = servers
.into_iter()
.map(|(family, octets, _, _)| (family, octets))
.collect();
proxy
.set_link_dns(index, &[(family, octets)])
.set_link_dns(index, &plain)
.await
.map_err(|err| classify(err, "setting the link's DNS server"))?;
.map_err(|err| classify(err, "setting the link's DNS servers"))?;
}
}
@@ -243,7 +260,7 @@ mod tests {
let publisher = ResolvedPublisher::new();
let published = Published {
interface: "tsunagi-no-such-interface".into(),
server: SocketAddr::from(([10, 13, 37, 69], 5354)),
servers: vec![SocketAddr::from(([10, 13, 37, 69], 5354))],
domains: vec!["lab".into()],
};
let err = publisher.apply(&published).await.unwrap_err();
crates/tsunagi/src/dns/publish/unsupported.rs
+1 -1
View File
@@ -62,7 +62,7 @@ mod tests {
let publisher = UnsupportedPublisher::new();
let published = Published {
interface: "tsundemo".into(),
server: "10.0.0.1:5354".parse().unwrap(),
servers: vec!["10.0.0.1:5354".parse().unwrap()],
domains: vec!["lab".into()],
};
let err = publisher.apply(&published).await.unwrap_err();
crates/tsunagi/src/dns/server.rs
+55
View File
@@ -518,6 +518,61 @@ mod tests {
assert_eq!(reply.answers.len(), 1);
}
#[tokio::test]
async fn the_same_zone_is_answered_over_ipv6_as_over_ipv4() {
// A question arrives over whichever family the resolver chooses, so
// a listener is opened from each list in the plan and both have to
// answer the same thing. Answering one family only leaves the other
// timing out, which looks like a broken overlay.
let shared = SharedZone::new(zone());
let plan = crate::dns::listen_plan(None, 0);
let mut answered = 0;
for family in plan.families() {
let candidate = family[0];
let server = match DnsServer::bind(candidate, shared.clone()).await {
Ok(server) => server,
// A host with IPv6 switched off in the kernel cannot bind
// `::1`, and the agent copes with that by design; so does
// this. The other family still has to work.
Err(err) if candidate.is_ipv6() => {
eprintln!("no IPv6 loopback on this host ({err}); skipping that family");
continue;
}
Err(err) => panic!("cannot listen on {candidate}: {err}"),
};
let addr = server.local_addr();
assert_eq!(addr.is_ipv6(), candidate.is_ipv6());
let client = tokio::net::UdpSocket::bind(if addr.is_ipv6() {
"[::1]:0"
} else {
"127.0.0.1:0"
})
.await
.unwrap();
client
.send_to(&ask("music.lab", TYPE::A), addr)
.await
.unwrap();
let mut buffer = vec![0u8; MAX_MESSAGE_LEN];
let read = client.recv(&mut buffer).await.unwrap();
let reply = simple_dns::Packet::parse(&buffer[..read]).unwrap();
assert_eq!(reply.rcode(), RCODE::NoError, "over {addr}");
match &reply.answers[0].rdata {
// The same IPv4 answer either way: the family a question
// travelled over says nothing about what the answer is.
RData::A(a) => {
assert_eq!(Ipv4Addr::from(a.address), Ipv4Addr::new(10, 13, 37, 237))
}
other => panic!("expected an A record over {addr}, got {other:?}"),
}
answered += 1;
}
assert!(answered > 0, "at least one family must have answered");
}
#[tokio::test]
async fn replacing_the_zone_changes_what_the_running_server_answers() {
let shared = SharedZone::new(Zone::new(ZoneName::new("lab").unwrap(), []));
crates/tsunagi/src/ipc/mod.rs
+10 -2
View File
@@ -108,8 +108,10 @@ pub struct StatusReport {
pub struct DnsReport {
/// The zone it answers for.
pub zone: String,
/// Where it is listening, if it managed to bind.
pub listening: Option<String>,
/// Every address it is listening on, one per family where it could.
///
/// Empty means it answers nowhere, and `bind_error` says why.
pub listening: Vec<String>,
/// Why it could not bind, if it did not.
pub bind_error: Option<String>,
/// Why the system resolver was not told, if it was not.
@@ -188,6 +190,12 @@ pub struct PeerReport {
pub struct OverlayReport {
/// Packet interface name.
pub interface: String,
/// Whether that interface exists on the host.
///
/// `false` under `--no-tun`: tunnels run and packets move between
/// agents, but the operating system has no interface, no address and no
/// route, so nothing local reaches the overlay.
pub on_host: bool,
/// Interface MTU.
pub mtu: u32,
/// This agent's overlay address, once the network has agreed one.
crates/tsunagi/src/ipc/unix.rs
+1 -1
View File
@@ -237,7 +237,7 @@ async fn exchange(path: &Path, request: &Request, within: Duration) -> Result<Re
///
/// Bump it whenever [`Request`], [`Response`] or anything they contain
/// changes shape.
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 6]);
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 7]);
async fn write_message<T: serde::Serialize>(stream: &mut UnixStream, value: &T) -> Result<()> {
let encoded = postcard::to_stdvec(value)
crates/tsunagi/src/overlay/interface.rs
+32
View File
@@ -236,6 +236,16 @@ impl Interface {
self.device.mtu()
}
/// Whether this interface exists on the host.
///
/// `false` with an in-memory device, where tunnels run and packets move
/// but the operating system knows nothing about any of it. Anything that
/// would configure the host — a route, a resolver setting, a complaint
/// that an address is missing from a link — has to ask first.
pub fn on_host(&self) -> bool {
self.factory.on_host()
}
/// Removes the interface from the host.
pub async fn remove(&self) {
// The packet loop holds the device open, and it ends only when the
@@ -332,6 +342,28 @@ mod tests {
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use super::*;
#[tokio::test]
async fn an_in_memory_interface_says_it_is_not_on_the_host() {
// Everything that would configure the operating system for this
// interface asks first, and the answer here is no: the name is real
// to the agent and to nothing else. A real `tsun0` belonging to
// another agent looks identical from here, so acting on the name
// alone would configure that one.
let interface = Interface::start(
Arc::new(crate::overlay::MemoryTunFactory::new()),
"tsun0",
1280,
Arc::new(RoutingTable::new()),
Arc::new(Recorder::default()),
)
.await
.unwrap();
assert_eq!(interface.name(), "tsun0");
assert!(!interface.on_host());
interface.remove().await;
}
use crate::identity::{NetworkKeys, NetworkName, NetworkSecret};
use crate::overlay::router::NetworkRoutes;
use crate::overlay::tun::{MemoryTun, MemoryTunFactory};
crates/tsunagi/src/overlay/provision/factory.rs
+5
View File
@@ -51,6 +51,11 @@ impl TunFactory for ManagedTunFactory {
self.provisioner.name()
}
/// It creates the interface on the host and holds it open.
fn on_host(&self) -> bool {
true
}
fn create<'a>(
&'a self,
request: TunRequest,
crates/tsunagi/src/overlay/tun.rs
+61 -1
View File
@@ -75,6 +75,17 @@ pub trait TunFactory: Send + Sync + std::fmt::Debug + 'static {
/// A short name used in diagnostics.
fn name(&self) -> &str;
/// Whether what this factory creates exists on the host.
///
/// `false` for an in-memory device: it has a name and an MTU and nothing
/// else — no link, no addresses, nothing for the operating system to
/// route through or to attach a resolver setting to. Stated rather than
/// guessed from the name, because a name is not evidence: an in-memory
/// `tsun0` and a real `tsun0` belonging to another agent look identical
/// from here, and configuring the operating system for the first would
/// land on the second.
fn on_host(&self) -> bool;
/// Creates a device.
fn create<'a>(
&'a self,
@@ -117,6 +128,16 @@ pub struct MemoryTun {
from_os_rx: tokio::sync::Mutex<tokio::sync::mpsc::UnboundedReceiver<Bytes>>,
to_os_tx: tokio::sync::mpsc::UnboundedSender<Bytes>,
to_os_rx: tokio::sync::Mutex<tokio::sync::mpsc::UnboundedReceiver<Bytes>>,
/// Set once the device is gone, so a reader stops rather than waiting
/// for a packet that will never come.
///
/// A `watch` rather than a `Notify`: closing must be seen by a reader
/// that has not started waiting yet as well as by one already waiting —
/// a device can be removed before the loop reading it has been polled
/// even once. The receiver is kept for the same reason: a `watch` send
/// with nobody subscribed does nothing at all.
closed: tokio::sync::watch::Sender<bool>,
closed_rx: tokio::sync::watch::Receiver<bool>,
}
impl MemoryTun {
@@ -124,6 +145,7 @@ impl MemoryTun {
pub fn new(name: impl Into<String>, mtu: u32) -> Arc<Self> {
let (from_os_tx, from_os_rx) = tokio::sync::mpsc::unbounded_channel();
let (to_os_tx, to_os_rx) = tokio::sync::mpsc::unbounded_channel();
let (closed, closed_rx) = tokio::sync::watch::channel(false);
Arc::new(Self {
name: name.into(),
mtu,
@@ -131,9 +153,20 @@ impl MemoryTun {
from_os_rx: tokio::sync::Mutex::new(from_os_rx),
to_os_tx,
to_os_rx: tokio::sync::Mutex::new(to_os_rx),
closed,
closed_rx,
})
}
/// Marks the device gone, so the packet loop reading it ends.
///
/// A real interface reports end of stream when it is removed; this is
/// how that happens here. Without it teardown waits out the whole grace
/// period for a reader that had no way of knowing.
pub fn close(&self) {
let _ = self.closed.send(true);
}
/// Injects a packet as if the operating system had produced it.
pub fn push_from_os(&self, packet: Bytes) {
let _ = self.from_os_tx.send(packet);
@@ -155,7 +188,17 @@ impl TunDevice for MemoryTun {
}
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
Box::pin(async move { self.from_os_rx.lock().await.recv().await })
Box::pin(async move {
let mut closed = self.closed_rx.clone();
if *closed.borrow_and_update() {
return None;
}
let mut queue = self.from_os_rx.lock().await;
tokio::select! {
packet = queue.recv() => packet,
_ = closed.changed() => None,
}
})
}
fn send<'a>(&'a self, packet: Bytes) -> BoxFuture<'a, Result<(), OverlayError>> {
@@ -216,6 +259,23 @@ impl TunFactory for MemoryTunFactory {
"memory"
}
/// Nothing here is on the host, which is the whole point of it.
fn on_host(&self) -> bool {
false
}
/// Closes the device so whatever is reading it stops.
///
/// The device itself is kept, because a test asks what went through it
/// after the agent that owned it has gone.
fn destroy<'a>(&'a self, name: &'a str) -> BoxFuture<'a, ()> {
Box::pin(async move {
if let Some(device) = self.device(name) {
device.close();
}
})
}
fn create<'a>(
&'a self,
request: TunRequest,