Retire offline node claims with equal network-wide cleanup rights
Offline devices reserved names and IPv4 addresses indefinitely, leaving stale members in the device history. Give every authenticated network participant the same right to retire another device's observed signed claim without an owner, quorum, permanent identity ban or secret change. Add a separately signed retirement containing the target's original signed claim as proof. Verify both signatures and network scopes, retain the strongest proved cutoff per target with deterministic ties, and suppress claim versions at or below it in membership, hostname ownership and IPv4 allocation. Keep the old signed records and range agreement, and retain markers after a newer claim returns so old replicas cannot resurrect obsolete reservations. A returning device signs a newer claim, allocates against current holders and resets its previous hostname priority rather than taking a reused name with an old time. Commit local retirement and the requester's author counter atomically before announcing it; receivers persist proofs before applying them. Bound history to 1024 targets by default, reject additional targets without evicting protection, and treat corrupt or over-limit mandatory storage as an explicit error. Repeat bounded marker batches with a per-session rotating cursor so queue pressure cannot indefinitely starve later markers. Exclude retired claims and cached hints from history-derived discovery without preventing a fresh authenticated device from returning. Expose Agent::forget_node through the shared local control API. CLI command tsng network forget-node accepts exact network names, full ids or unique id prefixes, and exact node hostnames, ids, prefixes or overlay IPv4 addresses. Ambiguous selections change nothing. Recheck the observed version and refuse self, online nodes, the selected exit node and absent or changed claims. Add Forget with an identity confirmation to offline tray device rows. Widen Show devices to 1440x640 with an 800x400 minimum to fit its contents. Control ALPN/version advances to tsunagi/ctrl/7 and local IPC to 21; upgrade all peers and restart the agent with matching CLI/tray clients. Data ALPN 5, network identity derivation, handshake encoding and existing record signing bytes remain unchanged. SQLite schema 9 adds retirement storage while retaining identity, secrets, settings and signed claims. Document behavior and limits. Validation on Linux through nix develop: - cargo fmt --all -- --check - cargo clippy --locked --workspace --all-targets -- -D warnings - cargo test --locked --workspace --all-targets: 519 passed, 2 existing ignored tests; inherited TSUNAGI_CONTROL_SOCKET and TSUNAGI_CONTROL_GROUP were unset - git diff --check and the built CLI forget-node help Tests use real iroh authentication and SQLite. Cover restart, stale-replica recovery, equal rights and network isolation, hostname reuse and return, twelve marker batches through a two-message queue, malformed signatures and proofs, invented future cutoffs, deterministic merge, capacity, migration and injected counter-write rollback, selectors and cleanup through two real CLI processes. Earlier iterations caught a private test API, a collapsible tray conditional and a crash-based CLI test waiting for QUIC's idle timeout; corrected before the final successful suite. Manual tray rendering, macOS/Windows, privileged TUNs and public-network traversal were not run. The requested RC tag is v0.1.0-rc.31; no push was requested.
This commit is contained in:
1 parent
13bdcaf98d
commit
90ef5cefec
24 files changed
+1805
-57
No files matched your search
+9
-4
@@ -9,14 +9,18 @@ Two versions exist and are independent:
|
||||
|
||||
- **Identity scheme**, `tsunagi-network-id-v1`. Frozen. Changing it creates a
|
||||
different network space for the same name and secret.
|
||||
- **Control protocol**, ALPN `tsunagi/ctrl/6`, `PROTOCOL_VERSION = 6`.
|
||||
- **Control protocol**, ALPN `tsunagi/ctrl/7`, `PROTOCOL_VERSION = 7`.
|
||||
|
||||
Upgrading the crate or bumping the control protocol must never change an
|
||||
existing `NetworkId`.
|
||||
|
||||
Control version 6 adds private transport negotiation for pairwise AWG and data
|
||||
delivery checks. Data ALPN remains `tsunagi/data/5`. All members must upgrade
|
||||
together; network ids and stored state are unchanged. See [amnezia.md](amnezia.md).
|
||||
Control version 7 adds signed offline-node retirement messages to version 6's
|
||||
private AWG transport negotiation and delivery checks. Data ALPN remains
|
||||
`tsunagi/data/5`. All members must upgrade together; network ids, handshake
|
||||
encoding and existing record signatures are unchanged. Local CLI/tray IPC is
|
||||
version 21, so restart the agent with matching clients. SQLite schema 9 adds
|
||||
retirement storage while preserving existing identities, settings and records.
|
||||
See [sync-model.md](sync-model.md#offline-node-cleanup) and [amnezia.md](amnezia.md).
|
||||
|
||||
## Canonical encoding
|
||||
|
||||
@@ -238,6 +242,7 @@ not affect other networks.
|
||||
| `Ping { seq, payload }` | small request used to verify the exchange |
|
||||
| `Pong { seq, payload }` | the echoed reply |
|
||||
| `State { records }` | a snapshot of signed records, merged into what the receiver holds |
|
||||
| `Retirements { markers }` | signed cleanup markers with the target's original signed claim as proof |
|
||||
| `Peers { peers }` | unverified member address hints, requiring their own handshake |
|
||||
| `Reachable { links }` | sender's current direct data links, by kind of transport |
|
||||
| `Bye { reason }` | graceful goodbye; not a revocation of anything |
|
||||
|
||||
+1
-1
@@ -124,5 +124,5 @@ counts sends without storing frames. Results are CPU forwarding cost, **not**
|
||||
end-to-end network latency or VPN throughput; encryption, fragmentation,
|
||||
sockets, congestion and scheduling contribute separately.
|
||||
|
||||
Wire compatibility: control ALPN `tsunagi/ctrl/6`, data ALPN `tsunagi/data/5`.
|
||||
Wire compatibility: control ALPN `tsunagi/ctrl/7`, data ALPN `tsunagi/data/5`.
|
||||
Upgrade every participant together; saved identities and network state persist.
|
||||
+46
-27
@@ -1,18 +1,9 @@
|
||||
# Planned state synchronisation
|
||||
# State synchronisation
|
||||
|
||||
**The first slice of this model is now implemented**, in `crates/tsunagi-core/src/state/`, and is
|
||||
used for one thing: IPv4 overlay addresses. What follows describes the whole
|
||||
model; the section at the end says exactly which parts exist.
|
||||
|
||||
The proof of concept still exchanges hostname and capability announcements
|
||||
over live sessions and keeps no replicated history for those, which is why
|
||||
WireGuard peer *membership* remains session-scoped even though a peer's
|
||||
*address* no longer is. This file records the intended direction so the module
|
||||
boundaries in [architecture.md](architecture.md) stay compatible with it, and so
|
||||
nobody mistakes the current announcements for synchronisation.
|
||||
|
||||
There is no fake "ready CRDT" here and no snapshots that are not actually
|
||||
verified.
|
||||
Signed claims in `crates/tsunagi-core/src/state/` preserve IPv4 reservations and
|
||||
hostnames across an absence. Signed releases and offline-node cleanup markers
|
||||
free those reservations durably. Capabilities, links and authenticated sessions
|
||||
remain volatile; a stored claim never authenticates a live connection.
|
||||
|
||||
## The problem
|
||||
|
||||
@@ -75,6 +66,39 @@ Turning a computer off is not a revocation of its hostname and does not remove
|
||||
the participant. Revocations are not dropped merely because they are old, and
|
||||
no acknowledgement from offline peers is required to keep working.
|
||||
|
||||
## Offline-node cleanup
|
||||
|
||||
Any authenticated member can sign a `SignedRetirement` for another member's
|
||||
observed `Claim`. It includes the original signed claim as proof and uses the
|
||||
separate `tsunagi-node-retirement/v1` signing domain. Both signatures and both
|
||||
network scopes are checked. No owner, vote or quorum is involved, and no member
|
||||
can fabricate the target's release or invent a future version cutoff.
|
||||
|
||||
The strongest observed target version wins; ties use a deterministic canonical
|
||||
byte order. Claims at or below that version do not occupy an address, hostname
|
||||
or visible member row. Original signed records remain stored, including their
|
||||
range, so cleanup does not renumber the network. Markers remain even after a
|
||||
newer claim returns: absence from a snapshot is never deletion and replaying an
|
||||
old snapshot cannot restore the retired claim.
|
||||
|
||||
The local `Agent::forget_node` API checks the exact observed version again and
|
||||
refuses self, online nodes, the selected exit node and missing/changed claims.
|
||||
CLI and tray share this API. A returning live node can sign a newer claim and
|
||||
allocate against the current holders; it resets the old hostname priority, so
|
||||
it cannot reclaim a reused name merely by keeping its old timestamp. Cleanup
|
||||
is not a permanent identity ban or a change of secret.
|
||||
|
||||
The marker and requester's own author counter commit in one SQLite transaction
|
||||
before publication. Receivers persist a verified marker before applying it.
|
||||
Invalid proofs are rejected individually. Periodic bounded marker batches
|
||||
rotate through each peer's send queue, including when a queue fills, so a
|
||||
replica that missed an update can catch up without the requester being online.
|
||||
|
||||
One strongest marker per target is retained, with a default limit of 1024 per
|
||||
network and 1024 bytes per encoded marker. New targets beyond that limit are
|
||||
refused rather than evicting resurrection protection. Corrupt mandatory marker
|
||||
storage or a configured limit smaller than the stored set is an explicit error.
|
||||
|
||||
## Storage requirement this creates
|
||||
|
||||
When signed records land, writing the event and bumping the author's own
|
||||
@@ -104,9 +128,9 @@ Implemented, in `crates/tsunagi-core/src/state/`:
|
||||
* the merge rules above: higher version wins, an older version never rolls
|
||||
back a newer one, duplicates are idempotent, a same-version conflict is
|
||||
resolved identically on every replica and reported;
|
||||
* a release tombstone, which merges correctly and is not undone by a replica
|
||||
that has not heard of it — though nothing emits one yet, so freeing an
|
||||
address still means forgetting the network;
|
||||
* a release tombstone, emitted when leaving or replacing the device key and
|
||||
not undone by a replica that has not heard of it;
|
||||
* durable hostnames and equal-rights offline-node retirement described above;
|
||||
* persistence in `state.sqlite`, with the record and the author's own version
|
||||
counter committed in **one transaction before the record is announced**;
|
||||
* distribution as a `State` control message, merged into what the receiver
|
||||
@@ -114,21 +138,16 @@ Implemented, in `crates/tsunagi-core/src/state/`:
|
||||
* allocation of a free IPv4 address against what everybody else holds, which
|
||||
is what makes an address stable across an absence.
|
||||
|
||||
Deliberately not implemented: compaction, revoking a whole author, record
|
||||
types beyond addressing, and any bound on how large a snapshot may grow
|
||||
beyond the per-message limit.
|
||||
Deliberately not implemented: compaction and permanent author revocation.
|
||||
Claims have a per-message bound; retirement history also has a per-network bound.
|
||||
|
||||
## Future tests
|
||||
|
||||
These are **not implemented and must not be reported as passing**:
|
||||
|
||||
- snapshot merge against the rules above, including conflicting same-version
|
||||
records;
|
||||
- revocation propagation and resistance to resurrection after compaction;
|
||||
- resistance to resurrection after compaction (compaction is not implemented);
|
||||
- long network partitions and rejoin after an extended absence;
|
||||
- hostname rename with atomic revoke-and-announce;
|
||||
- recovery of a returning participant from a replica that is not the author;
|
||||
- NAT traversal and hole punching between real hosts;
|
||||
- relay fallback behaviour against a self-hosted relay;
|
||||
- multi-process and multi-host deployment, as opposed to several library
|
||||
instances inside one test process.
|
||||
- multi-host deployment. CLI tests do run separate local agent processes;
|
||||
library tests run several independent instances inside one process.
|
||||
+14
-4
@@ -164,10 +164,9 @@ paused clock. The public Mainline/relay smoke test is ignored by default.
|
||||
|
||||
## Not covered, and not claimed to be
|
||||
|
||||
Listed in [sync-model.md](sync-model.md#future-tests): snapshots, revocations,
|
||||
long partitions, hostname renames, recovery of a returning participant, NAT
|
||||
traversal, relay fallback, and multi-process or multi-host deployment. None of
|
||||
these are implemented, and none are marked as passing.
|
||||
Listed in [sync-model.md](sync-model.md#future-tests): compaction, long partitions,
|
||||
NAT traversal, relay fallback and multi-host deployment. These scenarios are
|
||||
not established by the default loopback suite.
|
||||
|
||||
## Debugging a test
|
||||
|
||||
@@ -198,3 +197,14 @@ SQLite tests migrate schema 7 to 8 without changing
|
||||
identity, secrets or signed records, preserve order and explicit empty choices,
|
||||
and reject malformed stored selections. These checks use memory TUNs and do
|
||||
not establish GUI rendering or behavior on an actual OS interface.
|
||||
|
||||
`tests/forget_node.rs` uses real agents, handshakes and SQLite to check equal
|
||||
cleanup rights, network isolation, online/self/stale-version refusal, restart,
|
||||
recovery of a stale replica, reuse of a hostname and a returning identity's
|
||||
newer claim. Bounded batch delivery is checked with a two-message send queue
|
||||
and twelve retained markers. Retirement unit tests check signatures, proof tampering, invented
|
||||
future cutoffs, bounds, deterministic merge and resistance to stale snapshots.
|
||||
SQLite tests migrate schema 8 to 9 and inject counter-write failure to prove
|
||||
atomic rollback. The CLI test starts two real agent processes and exercises
|
||||
offline cleanup by network and hostname through the control socket.
|
||||
Tray rendering and physical screen sizing are not exercised automatically.
|
||||
+12
-2
@@ -38,6 +38,14 @@ Read this before relying on anything here. The protocol is in
|
||||
arbitrarily many identities, flood the network with records and collide with
|
||||
other participants' names. This is why a majority is not a root of trust.
|
||||
Signatures protect authorship; they do not make a participant honest.
|
||||
- **Cleanup rights are equal.** Any member can retire an observed signed claim
|
||||
of another device, freeing its hostname and IPv4 reservation. A valid original
|
||||
claim bounds the cutoff; a member cannot invent a future cutoff or forge the
|
||||
target's release. The local API refuses online nodes, but a remote signature
|
||||
cannot prove that a target was offline. A live target may publish a newer
|
||||
claim; a malicious member can repeatedly disrupt reservations. Cleanup is
|
||||
not expulsion or secret revocation. See
|
||||
[sync-model.md](sync-model.md#offline-node-cleanup).
|
||||
- **Weak secrets.** This targets high-entropy secrets. There is no PAKE, so a
|
||||
short human passphrase can be guessed offline by anyone who can reach the
|
||||
handshake. Use `NetworkSecret::generate()`.
|
||||
@@ -55,8 +63,10 @@ Read this before relying on anything here. The protocol is in
|
||||
having iroh under it does not make it so.
|
||||
- **A cloned state directory is a cloned identity.** `state.sqlite` holds the
|
||||
device secret key and the network secrets. Copying it copies the participant.
|
||||
Restoring an old backup rolls the agent's state back, which — once signed
|
||||
records exist — can resurrect revoked information or replay stale versions.
|
||||
Restoring an old backup rolls the agent's state back and can replay stale
|
||||
versions until it meets a replica holding the newer signed state. Retained
|
||||
cleanup markers prevent old claims from resurrecting after that merge; they
|
||||
cannot help if every copy of a marker has been lost.
|
||||
- **A compromised host.** The secret is on disk to survive restarts. File
|
||||
permissions are owner-only where the platform supports it, and the state
|
||||
directory takes an ownership lock, but neither defends against a user who can
|
||||
|
||||
Reference in new issue
Block a user