Nine orphaned test processes were found still running from the day before,
three of them spinning on a core each for twenty hours. The code they ran
is several changes old and the mesh test passes twenty times over now, so
the wedge itself is gone — but nothing in the way it was waited on was
bounded, which is why a wedge lasted a day instead of failing a run.
The harness enforced its deadline only between probes. A probe that never
returned — one call into a wedged runtime, which is exactly what a status
request is — waited for ever inside the deadline it was supposed to obey.
The probe is now bounded too, so the same wedge fails the test in thirty
seconds.
Shutdown claimed to be bounded and was not. The plugins had a grace
period; the network runtimes, the accept loop, the plugin request loop and
the endpoint close did not, and a peer that stops reading is enough to
hold any of them open. Each now gets a grace period and is aborted after
it. The overlay packet loop was not stopped at all: it ends when the
device reports end of stream, which a live interface never does, so it
outlived the interface it was reading. And a plugin's grace period
abandoned the future without stopping the task behind it, so the helper
is public and `wg-quic` uses it on its own runtime.
The local control socket was unbounded in both directions. A wedged agent
left `tsunagi status` hanging with nothing on screen and no way out but
Ctrl-C; it now says the agent did not answer, after five seconds, and
falls back to the state store as it already did for a socket that refuses
a connection. On the serving side, a connection that sends no request no
longer holds a task open.
Tests cover the mechanism — a task that stops on its own is not aborted,
one that ignores the grace is cut off and drops what it held — and both
sides of the change in behaviour: a probe that never answers fails its
deadline, and a silent agent is reported rather than waited out.
Also: the binary opts out of rustdoc, since it shares a name with the
library and `cargo doc` cannot put both in one directory.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A status report with two sections both headed "network LAB" reads as one
network that is somehow working and empty at once. It was two networks:
the same name with different secrets, which is two different networks
that share nothing, because a network's identity is its name *and* its
secret.
Three fixes for the one confusion.
The heading now carries the network id, so the sections are plainly
different things. A name is a label the user chose; the id is the
identity.
Joining a name that is already configured with another secret says so, at
the moment it happens, because that is almost always a mistyped secret
and until now it silently produced an empty network sitting beside a
working one. `status` flags it too, for the case where it already
happened.
And the second network's emptiness now says why. It had no address
because the only configured range was already taken by the first — one
agent has one interface, so an address belongs to one network — and
"nobody else has joined" pointed at the wrong thing entirely. It now
names the range it cannot have, the reason, and the flag that gives it
one of its own.
Nothing was wrong with the connectivity: the working network's tunnel was
up and its ping was answering throughout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tsunagi-wg-quic. The line between a protocol and the system level is now
drawn by the compiler: nothing in it can reach into tsunagi beyond what
tsunagi makes public, and it carries its own version — which is not the
version peers compare.
Two things the compiler found the moment the boundary was real. The key
store was reaching into the core's `pub(crate)` file-permission helpers;
those are a legitimate service of the system level, because a protocol
keeping keys on disk has the same obligation the agent does, so they are
public now with that said. And the test harness was about to be copied
into a second crate, which is how two copies start to drift; it is a
`testing` feature of the core instead, which is also what anybody writing
a protocol would need.
The bridges put up while things were moving are gone: the error
conversion between the two levels, and the re-exports of the system
level's types from the protocol crate. Imports now say which level they
come from, which is the point.
One deliberate deviation, stated rather than hidden. The authenticated
transport stayed in the core. Moving it would have meant handing a
protocol the network's keys so it could prove membership itself, and a
plugin that can authenticate on the control plane is a worse trade than
a module boundary is worth. So the core proves who is at the other end
and the protocol owns what is said over it — the same separation, without
the secret crossing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`up` now says which level each setting belongs to, and `--help` shows the
two sections. System: how the agent reaches peers, the one interface it
owns, the address range, the resolver. Transport: which protocols carry
packets and what they take.
`--wireguard` is gone. `--protocol` takes a list and defaults to
`wg-quic`, which is what the protocol is now called — WireGuard's
cryptography in QUIC datagrams, so the name says what is on the wire
rather than what the implementation borrows. `--protocol none` runs the
control plane alone.
Protocol settings moved to `-o key=value`, or `-o protocol:key=value`
when several are selected. Each protocol declares its own settings and
their help, so `tsunagi protocols` can list them without the agent
knowing anything about any protocol, and a setting nobody takes is
refused rather than dropped — a dropped setting looks exactly like one
that did not work. What the user asked for is checked before anything
that could fail on its own, so a misspelled protocol is not buried under
a privilege error.
`--wg-prefix` and `--wg-mtu` became `--interface` and `--mtu`: they were
never the protocol's, and the interface they describe belongs to the
agent. `--transport` became `--reach`, because "transport" now means the
protocol level and using the word for iroh's path policy as well would
be a collision of meaning rather than a shortage of words.
The plugin gave up the last things that were not its own: the interface
name it carried in its own state, and the check that this agent's
address is really on an interface. Both are the agent's, and the check
is now the agent's too, still said once per address rather than every
round.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A peer's announcement already carried a protocol and a version; only the
name was being checked. Now both are, and a peer offering a protocol at a
version this build does not speak simply has no data plane — the control
plane keeps working, messages and signed state still flow, and the
difference is reported once rather than on every announcement.
The version compared is the *wire* version, not the software version, and
the trait says so: a plugin crate has its own version and it is nobody
else's business. Two peers on different releases work together for as
long as the bytes between them have not changed, and nothing in the
negotiation may be derived from anything that moves with a release. A
test pins that agreement turns on the name and version alone, with a
peer whose announcement carries a payload this build has never seen.
`PeerStatus` gained the protocols agreed with each peer, so an empty list
is visible as what it is: a session that is up, carrying control traffic,
with no protocol in common.
The forging test plugin was announcing version 1 while claiming to speak
the current one, so its payload was being set aside for the wrong reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One agent, one interface. The plugin no longer creates one, no longer
holds a TUN factory, no longer keeps a routing table and no longer
decides who owns an address. What is left of it is the protocol: a
WireGuard key per network, a tunnel per peer, encryption on the way out
and decryption on the way in.
The packet path is now explicit about where each decision lives. Out: the
agent's interface reads a packet, the routing table says whose
destination it is, and each protocol is asked in turn whether it can
carry it there. In: the protocol decrypts and hands the packet up with
the peer it came from attached, and the agent checks that peer is
entitled to the source address before writing it out. A protocol proves
who; only the system level knows what they may say.
Two things found by making it work.
`carry` sent the packet unencrypted at first. The encryption had lived in
the interface loop that moved to the core, so taking that out quietly
removed it — the receiving end rejected plaintext as a bad WireGuard
datagram and the counters said nothing at all. Encryption belongs with
the protocol and is now there, with packets dropped for having no session
yet counted apart, because a handful while a tunnel comes up is normal
and a number that keeps climbing is not.
An agent could impose a range it could not itself route. With one
interface two networks need different ranges, and "the lowest author's
range wins" would have carried one agent's colliding default to
everybody. The configured range is now reserved when a network is
activated — on the serialised path, so the answer does not depend on
which task ran first — and an agent that cannot have it proposes nothing
and adopts whatever the network settles on.
Leaving a network takes its address off the interface and leaves the
interface; the interface goes when the agent does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The loop that reads the interface, decides whose packet it is and hands
it to whatever can carry it — and the other direction, where a packet a
protocol decrypted is checked and written out.
Neither direction knows which protocol is involved. The interface asks
for a packet to reach a peer and is told whether that was possible;
`PacketCarrier` is the whole of what it knows about protocols. That is
what lets several be live at once without any of them owning the
interface they are carrying traffic for.
The source check stays here rather than in a protocol, and the split is
the point: a protocol proves *who* sent a packet, and only this level
knows what that member is entitled to say, because entitlement is the
signed claim.
Four kinds of drop are counted apart, because they want different
answers: a destination nobody holds (with the first such address kept, so
the number can be acted on), a destination that is known but has no live
protocol, multicast the operating system emitted anyway, and a packet
that could not be read. Rolled into one counter they would say only that
something is wrong.
Not wired in yet: the WireGuard plugin still owns its own interface, and
that is the next commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The routing table, which is what makes one interface able to serve
several protocols. A packet coming off it could belong to any network the
agent is in and to any protocol currently carrying traffic; the answer
comes from signed state, which no protocol owns, so this is the right
level for it.
Two questions, deliberately not the same one. Outbound: whose is this
destination — and a packet for nobody is dropped rather than flooded,
because a tunnel is not a broadcast domain. Inbound: this peer decrypted
a packet claiming this source, is that address actually its — checked
against the signed claim and never against anything the peer said.
A packet addressed to this agent itself routes nowhere, rather than to
whichever peer happens to be listed.
One interface means an address belongs to one network, so two networks
whose ranges overlap are refused with the reason and the fix. Guessing
between them would hand somebody's traffic to a stranger. Nested ranges
count as overlapping, which is the case the obvious comparison misses.
Not wired in yet: the interface loop that uses it comes next.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The module documentation still said addresses were derived from a
WireGuard key and that the plugin owned the interface. Both are now
wrong, and rustdoc caught it as broken links to modules that had moved.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The overlay address was derived from a WireGuard key, which makes it the
protocol's address — and the whole point of the interface belonging to
the system level is that every protocol carries traffic for the *same*
addresses. A derived-per-protocol address cannot be that.
So the overlay is IPv4 only: allocated at the system level, signed by the
member that holds it, and the same address whichever protocol happens to
be moving the packets. The derivation, its ULA prefix and its constants
are gone, along with the collision rule that existed only because a
derived IPv4 address has too little room to be unique — an allocated one
is unique by construction.
A real loss came with it and is restored explicitly. The announcement was
bound to its network only as a side effect of checking the derived
address, so removing that check removed the binding. It now carries the
network id and rejects a mismatch. Strictly redundant, because a
capability arrives on a session that already proved membership, and kept
because losing a property silently is the wrong way to lose one.
An unlock falls out: the MTU floor of 1280 existed because Linux tears
IPv6 down below it. Without IPv6 the floor is 576, what every IPv4 host
must be able to reassemble, so a relayed path with small datagrams can be
matched rather than warned about. The default stays 1280.
The test that forged an overlay address now forges a network id, which is
what is left to lie about. One flaky assertion fixed while passing: it
waited for "the interface has some address", which is briefly true of the
leftover it was meant to see replaced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One agent, one interface, owned by the agent rather than by a protocol.
That is what makes several protocols able to be live at once: a packet
leaving the interface is routed to whichever peer owns its destination,
over whichever protocol has a link to that peer, and neither protocol
has to hold the address because the agent holds it.
This commit moves the pieces without changing behaviour: provisioning,
the TUN itself, IP header parsing and interface naming are now
crates/tsunagi/src/overlay, and the WireGuard module re-exports them
while its callers are moved over. They were never WireGuard-specific —
netlink, capabilities and `ip tuntap` have nothing to do with the
protocol running on top.
They also get their own error type. An interface that cannot be created
is not a plugin failing, and now that the two belong to different levels
they should not share a word for it.
Routing, addressing and the reduced plugin contract come next.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
First step of separating the layers. The library and the binary are now
crates/tsunagi and crates/tsunagi-cli, which means the plugin crate to
come can be told apart from the core by the compiler rather than by
discipline.
Falls out of it immediately: the CLI's dependencies stop being features
of the library. clap, anstream and tracing-subscriber were optional
dependencies behind a `cli` feature that every library user had to
remember to turn off; now they belong to the crate that uses them, and
the library defaults to no features at all.
The one test that drives the binary moved beside it — a library cannot
depend on a binary built from a crate that depends on the library — and
was rewritten against the public API instead of the test harness.
AGENTS.md said to prefer one crate. It now says the system level and its
plugins are separate crates, for the reason above, and that everything
else stays one crate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tsunagiandClaude Opus 5