Add shared TCP and UDP virtual services

Provide named VIPs with independent protocol/port listeners and node/port
backends so applications can use one DNS endpoint without a separate load
balancer. Every authenticated member may create, edit or delete any service.
Client agents select the eligible backend with the lowest measured encrypted
data-plane RTT, including relayed paths, and pin existing flows. UDP recovers
on another backend after failure; established TCP is never replayed or moved.
An unanswered empty TCP SYN may retry another backend before establishment.

Keep signed multi-writer field registers in state/vip.rs. Independent backend
edits merge, same-field conflicts use Lamport/author/canonical ordering, and
listener generations prevent old backend entries returning on port recreation.
Permanent service tombstones protect replay while allowing obsolete fields to
be compacted. Active-service and deletion-history limits are separate. Commit
complete snapshots and the author's durable counter atomically before local
routing changes or publication. Restore counters from newer genuine author
updates learned from replicas after an old backup is restored.

Implement bounded client-side address/port NAT at overlay ingress and reverse
translation only after authenticated source/destination admission. Reserve OS
source ports to avoid direct-flow collisions and quarantine retired ports for
late replies. Handle a backend on the client itself, scoped fragment assembly,
checksums, re-fragmentation and quoted ICMP/PMTU headers. Plugins and encrypted
transit retain their existing contracts. Data-path probes measure RTT; bounded
fair TCP connect checks are enabled by default and configurable per listener.
Arbitrary UDP application readiness remains unknown; path checks and explicit
port errors drive eligibility without protocol-specific probe payloads.

Expose the shared draft API through local IPC, simple vip/listener/backend CLI
commands, JSON, editor-based atomic updates and watch output. Add a VIP button
on each network tile and a separate tray editor. Edits and removals stay local
until Save; stale revisions retain the draft, and closing a dirty window offers
explicit discard. Report local routing choices, health reasons, RTT age, active
and draining bindings, IP counters, rates, drops and UDP failovers. Extend DNS
with service names; keep numeric addresses secondary and preserve existing
bindings while VIP addresses or listeners change.

This changes control wire ALPN/version to tsunagi/ctrl/8 and local IPC to 22;
all members and local clients must upgrade together. Data ALPN remains 5.
SQLite schema 10 migrates existing mandatory state without changing identity,
secrets, network ids, derivation labels, handshake encoding or node records.
Crate versions and release tags are unchanged. No installed agent or tray was
replaced. Commands, limits and operational semantics are in docs/vip.md.

Checks on Linux in nix develop:
- cargo fmt --all -- --check
- cargo clippy --locked --workspace --all-targets -- -D warnings
- cargo test --locked --workspace --all-targets: 559 passed, 3 ignored
- git diff --cached --check

The full suite includes real encrypted TCP/UDP VIP traffic, backend-stop UDP
failover, real loopback TCP check recovery, source admission, signed merges,
stale/deleted state, SQLite migration and rollback, restart/author recovery,
network isolation, actual CLI/DNS processes, and headless text-entry/Save-button
interaction. Discovery and memory TUNs are the existing test substitutions;
authentication, transport, encryption and SQLite remain real.

Initial checks outside Nix failed for missing GTK pkg-config metadata, and an
initial CLI test inherited TSUNAGI_CONTROL_SOCKET. Final checks use the Nix
environment with that variable unset so tests address their temporary sockets.
Root/internet-dependent tests remain ignored. No live host TUN, native desktop
session, macOS or Windows validation was performed.
This commit is contained in:
Aleksandr Bogomiakov committed 2026-10-21 15:40:45 +01:00
1 parent 94318e7c16
commit 23af1a2d9d
35 files changed
+5055 -21

No files matched your search

+13 -11
View File
@@ -9,24 +9,26 @@ Two versions exist and are independent:
- **Identity scheme**, `tsunagi-network-id-v1`. Frozen. Changing it creates a
different network space for the same name and secret.
- **Control protocol**, ALPN `tsunagi/ctrl/7`, `PROTOCOL_VERSION = 7`.
- **Control protocol**, ALPN `tsunagi/ctrl/8`, `PROTOCOL_VERSION = 8`.
Upgrading the crate or bumping the control protocol must never change an
existing `NetworkId`.
Control version 7 adds signed offline-node retirement messages to version 6's
private AWG transport negotiation and delivery checks. Data ALPN remains
`tsunagi/data/5`. All members must upgrade together; network ids, handshake
encoding and existing record signatures are unchanged. Local CLI/tray IPC is
version 21, so restart the agent with matching clients. SQLite schema 9 adds
retirement storage while preserving existing identities, settings and records.
See [sync-model.md](sync-model.md#offline-node-cleanup) and [amnezia.md](amnezia.md).
Control version 8 adds signed, multi-writer VIP field snapshots to version 7's
node retirements. Data ALPN remains `tsunagi/data/5`. All members must upgrade
together. Network ids, derivation labels, handshake encoding and existing record
signatures are unchanged. Local CLI/tray IPC is version 22; restart the agent
with matching clients. SQLite schema 10 adds `vip_fields`, preserving identity,
network configuration, signed node records and retirement proofs.
See [virtual services](vip.md), [state sync](sync-model.md) and [AWG](amnezia.md).
## Canonical encoding
Everything that is hashed or MAC'd uses length-prefixed fields, written here as
`LP(x) = u32_be(len(x)) || x`. String concatenation is never used, so no two
different field splits can produce the same bytes.
Network identity derivation and the membership transcript use length-prefixed
fields, written here as `LP(x) = u32_be(len(x)) || x`. String concatenation is
never used, so different field splits cannot produce the same bytes. VIP field
signatures use their separate domain and canonical postcard encoding described
in [virtual services](vip.md#shared-state-and-addresses).
## Network space identity
+1 -1
View File
@@ -124,5 +124,5 @@ counts sends without storing frames. Results are CPU forwarding cost, **not**
end-to-end network latency or VPN throughput; encryption, fragmentation,
sockets, congestion and scheduling contribute separately.
Wire compatibility: control ALPN `tsunagi/ctrl/7`, data ALPN `tsunagi/data/5`.
Wire compatibility: control ALPN `tsunagi/ctrl/8`, data ALPN `tsunagi/data/5`.
Upgrade every participant together; saved identities and network state persist.
+15
View File
@@ -228,3 +228,18 @@ scroll area. They check stretching and shrinking without an expanding first
column, separate handshake/control RTT values, unknown measurements and short
end-to-end hints that do not substitute the carrier protocol. This does not
establish native desktop rendering on every platform.
VIP tests cover signed multi-writer backend merging, stale draft rejection,
permanent deletion and replay, separate active/deleted capacity, and mandatory
SQLite migration/counter rollback. Real agent instances verify equal editing
rights, network isolation, restart and recovery without the creator. Packet
checks exercise RTT selection, port translation, UDP failover, a local backend,
fragment reassembly/checksums and ICMP quoted-header/PMTU translation. TCP check
recovery uses real loopback listening sockets. The WireGuard suite moves VIP
TCP/UDP packets through real encrypted tunnels, rejects forged VIP sources and
checks UDP failover after the selected agent stops. CLI/DNS tests run the real
binary, configure independent listeners, disable checks, reject an invalid
change atomically and observe DNS creation/deletion. GUI tests cover draft
isolation and form rendering. These do not establish live host TUN behavior or
native desktop behavior on Linux, macOS or Windows. See [VIP semantics](vip.md).
+171
View File
@@ -0,0 +1,171 @@
# Named virtual services
A VIP is a shared name such as `api.home`, with independent TCP/UDP listeners.
Each listener maps its port to a set of node identities and application ports.
The application uses the name; the automatically allocated IPv4 address is an
internal reservation. The current overlay carries IPv4 only.
## CLI and tray
```sh
tsng vip list home
tsng vip create home api
tsng vip port add home api tcp/443 --backend server1:6443 --backend server2:7443
tsng vip port add home api udp/53 --backend server1:5353
tsng vip backend add home api tcp/443 server3:6443
tsng vip backend remove home api tcp/443 server2:7443
tsng vip port set home api tcp/443 --port-check off
tsng vip port set home api tcp/443 --check-interval-ms 3000 --check-timeout-ms 2000 --failures 2 --successes 1 --idle-timeout 7200
tsng vip show home api
tsng vip status home api --watch
tsng vip edit home api
tsng vip port remove home api udp/53
tsng vip remove home api
```
`list` without a network lists all configured networks. Network and VIP
selectors accept exact names, full ids and unique id prefixes; backend node
selectors also accept stored node names. Ambiguous selections change nothing.
Names resolve to identities when saved, so a renamed node remains a backend.
`--json` provides configuration, revision, counters and observations;
`list`/`watch` emit one JSON object per service/sample. `edit`
opens JSON in `VISUAL`/`EDITOR` and submits one complete version-checked draft.
An invalid or rejected edited file is retained with its path reported.
Each mutating command is atomic. A listener can be added with its full initial
backend set, and `edit` can replace the entire configuration at once. Empty
VIPs/listeners are valid and resolve, but cannot accept traffic. Configuration
updates currently require the network to be active locally. Saved configuration remains visible while a
network is stopped.
The tray's network tile has a **VIP** button. Its separate window edits names,
listeners, backend nodes/ports, check settings and flow timeouts. All edits,
including deletion, remain drafts until **Save**. Status updates do not
replace a dirty draft. A save conflict retains the draft; **Discard / reload**
loads the latest configuration. Closing a dirty window offers explicit discard
or continued editing. Addresses and ids are available in a collapsed detail.
## Delivery and recovery
The client agent selects the eligible backend with the lowest smoothed full
round-trip latency through the actual encrypted data plane, including relays.
Unknown or stale RTT is not treated as zero. Local backends have zero path RTT.
A new flow is pinned to its selected backend; a faster RTT does not move it.
This can concentrate new traffic on the closest node.
At local ingress, NAT replaces destination address/port and reserves an OS
source port for an unambiguous reverse mapping. Return traffic passes normal
network/peer source and destination admission **before** reverse NAT restores
the virtual source address/port. Plugins and encrypted transit stay unchanged.
A peer never gains permission to claim a VIP source. Fragment assembly is
bounded, checksums are updated, packets are fragmented back to the overlay MTU,
and ICMP errors have their quoted NAT headers restored. Fragment caches are
scoped to network and authenticated sender, with idle expiry independent of
further traffic.
For a backend on the client itself, the request is injected into the same TUN
with the nonlocal VIP as source, and the host reply is reverse-translated at
local ingress. The local backend therefore sees the VIP as source; remote
backends see the client's ordinary overlay address. Firewalls and application
bind addresses remain the administrator's responsibility.
Removing a backend/listener/VIP prevents new bindings while existing bindings
continue until idle expiry. Established TCP connections are not migrated. A retransmitted empty SYN may
choose another backend before a SYN-ACK has arrived; SYNs carrying application
data are not retargeted. A new SYN sequence creates a new binding even if the
client reuses its address/port tuple. TCP RST shortens binding retention. An unavailable UDP backend is replaced for subsequent
datagrams; already sent datagrams are never replayed. Stateful UDP applications
may need to re-establish their application session. Protocols that embed IP
addresses or negotiate additional ports require their own configuration; no
application payloads are parsed or rewritten.
Idle expiry defaults to 7200 seconds for TCP and 120 seconds for UDP. Shadow
ports are quarantined for 120 seconds after retirement to reject late replies
and avoid immediate reuse. Active and quarantined bindings share a bounded
capacity. State and sockets are released on network stop or agent shutdown.
A restart drops bindings; shared service configuration remains intact.
## Health
One probe exchange per second measures each used peer through the encrypted
IP tunnel. The EWMA uses 7/8 of the previous sample and 1/8 of the new one.
A probe expires after three seconds; two consecutive missing responses or a
missing data path exclude the peer. Measurements expire after ten seconds.
Probes continue during failure so recovery is automatic.
TCP connect checks default to on. Defaults are a three-second interval,
two-second timeout, two failures to exclude and one success to recover.
Checks are shared for the same network/node/address/TCP-port, run off the
packet path with at most 32 concurrent connects, and are scheduled fairly.
A 250ms worker tick services timers. Connect checks send no application data;
they confirm port acceptance and may appear in server logs. They cannot prove
application readiness after a successful handshake. Disabling port checks
retains data-plane probes.
UDP uses encrypted path checks and explicit ICMP port errors. No arbitrary
probe datagrams are injected into an unknown application protocol. Silence
cannot prove failure of an arbitrary UDP application. An explicit unreachable
port is excluded for ten seconds and then becomes eligible to retry.
## Shared state and addresses
Any authenticated member can create, edit or remove any VIP. Each field is a
self-contained signed register: network, service id, field key, optional encoded
value, Lamport clock, author, durable author counter and signature. Signing uses
the separate `tsunagi-vip-field/v1\0` domain followed by postcard encoding of
those fields with the signature vector empty. The encoding/order is a wire
contract; the identity scheme and handshake transcript are unchanged.
Concurrent edits to different backend fields merge. A conflict on one field
uses `(Lamport clock, author id, canonical bytes)` ordering, independent of wall
time and arrival order. Listener generations prevent backend fields from an
old removed listener reappearing when the port is recreated. Service deletion
is a permanent signed tombstone for its id; recreation gets a fresh id.
Missing snapshot fields are never deletion.
Saving validates the entire draft and checks the revision shown by the editor.
The merged snapshot and author's counter commit together in SQLite before local
routing changes or publication. Receivers verify and persist complete service
snapshots before exposing them. Periodic rotating snapshots repair queue loss
and returning replicas without requiring the creator online. Concurrent edits
made before participants have seen each other can still merge after a save.
A merged configuration exceeding listener/backend limits is reported as an
error and excluded from routing rather than selected according to arrival order.
VIP allocation walks the agreed overlay range from a service-id-derived seed,
excluding known node claims and other allocated VIPs. Nodes avoid known VIP
reservations when choosing a new address. Names colliding with node names or
another service are reported; node names take precedence and service ties use
id order. A partition can temporarily have incomplete state; after merge an
address or conflicting name's effective holder can change. Use DNS names.
DNS TTL is 30 seconds; DNS does not migrate established connections or force
applications to discard cached addresses. Existing NAT bindings retain their
original VIP address while draining.
`Limits` bounds active services (64 by default), permanent deletion markers
(1024), signed fields per active service (192), listeners (16), backends per
listener (32), active TCP checks (32), flow/port reservations (4096) and incomplete
IPv4 datagrams (64). A deleted service retains only its signed permanent
marker; obsolete fields can be dropped because replay cannot override deletion.
A full deletion history is an explicit error; protection is never silently
evicted. Deleted services do not consume the active-service limit. Whole service
snapshots must also fit the configured control frame limit. Corrupt mandatory VIP storage
is a hard error on activation, never reset as a cache.
## Observations and verification
Status is this client's observation. Per VIP/listener/backend it reports the
next selected backend, health reason, RTT and sample age, active bindings,
user IP packet/byte counters, drops and UDP failovers. Probe bytes do not enter
user traffic counters. The GUI and CLI watch derive rates from cumulative
counters; counters reset on restart. These are IP forwarding counters, not
application acknowledgements or guaranteed delivered-byte totals.
Automated tests cover field merging, stale drafts, signed deletion, tampering,
network isolation, SQLite migration and transactional rollback, real port
connect failure/recovery, RTT selection, UDP failover, local hairpin NAT,
fragmentation/checksums, real encrypted TCP/UDP packet delivery and source
admission, real CLI/DNS wiring and draft form rendering. In-memory TUN tests
exercise packets, not an operating system TCP stack. Native desktop behavior,
physical interfaces and cross-platform kernel/firewall behavior need live-host
validation; automated tests do not establish those.