Add shared TCP and UDP virtual services
Provide named VIPs with independent protocol/port listeners and node/port backends so applications can use one DNS endpoint without a separate load balancer. Every authenticated member may create, edit or delete any service. Client agents select the eligible backend with the lowest measured encrypted data-plane RTT, including relayed paths, and pin existing flows. UDP recovers on another backend after failure; established TCP is never replayed or moved. An unanswered empty TCP SYN may retry another backend before establishment. Keep signed multi-writer field registers in state/vip.rs. Independent backend edits merge, same-field conflicts use Lamport/author/canonical ordering, and listener generations prevent old backend entries returning on port recreation. Permanent service tombstones protect replay while allowing obsolete fields to be compacted. Active-service and deletion-history limits are separate. Commit complete snapshots and the author's durable counter atomically before local routing changes or publication. Restore counters from newer genuine author updates learned from replicas after an old backup is restored. Implement bounded client-side address/port NAT at overlay ingress and reverse translation only after authenticated source/destination admission. Reserve OS source ports to avoid direct-flow collisions and quarantine retired ports for late replies. Handle a backend on the client itself, scoped fragment assembly, checksums, re-fragmentation and quoted ICMP/PMTU headers. Plugins and encrypted transit retain their existing contracts. Data-path probes measure RTT; bounded fair TCP connect checks are enabled by default and configurable per listener. Arbitrary UDP application readiness remains unknown; path checks and explicit port errors drive eligibility without protocol-specific probe payloads. Expose the shared draft API through local IPC, simple vip/listener/backend CLI commands, JSON, editor-based atomic updates and watch output. Add a VIP button on each network tile and a separate tray editor. Edits and removals stay local until Save; stale revisions retain the draft, and closing a dirty window offers explicit discard. Report local routing choices, health reasons, RTT age, active and draining bindings, IP counters, rates, drops and UDP failovers. Extend DNS with service names; keep numeric addresses secondary and preserve existing bindings while VIP addresses or listeners change. This changes control wire ALPN/version to tsunagi/ctrl/8 and local IPC to 22; all members and local clients must upgrade together. Data ALPN remains 5. SQLite schema 10 migrates existing mandatory state without changing identity, secrets, network ids, derivation labels, handshake encoding or node records. Crate versions and release tags are unchanged. No installed agent or tray was replaced. Commands, limits and operational semantics are in docs/vip.md. Checks on Linux in nix develop: - cargo fmt --all -- --check - cargo clippy --locked --workspace --all-targets -- -D warnings - cargo test --locked --workspace --all-targets: 559 passed, 3 ignored - git diff --cached --check The full suite includes real encrypted TCP/UDP VIP traffic, backend-stop UDP failover, real loopback TCP check recovery, source admission, signed merges, stale/deleted state, SQLite migration and rollback, restart/author recovery, network isolation, actual CLI/DNS processes, and headless text-entry/Save-button interaction. Discovery and memory TUNs are the existing test substitutions; authentication, transport, encryption and SQLite remain real. Initial checks outside Nix failed for missing GTK pkg-config metadata, and an initial CLI test inherited TSUNAGI_CONTROL_SOCKET. Final checks use the Nix environment with that variable unset so tests address their temporary sockets. Root/internet-dependent tests remain ignored. No live host TUN, native desktop session, macOS or Windows validation was performed.
This commit is contained in:
Aleksandr Bogomiakov committed
1 parent
94318e7c16
commit
23af1a2d9d
35 files changed
+5055
-21
No files matched your search
@@ -309,6 +309,28 @@ impl TransportContext for TransportCtx {
|
||||
}
|
||||
|
||||
impl Agent {
|
||||
/// Atomically saves or removes a version-checked shared VIP draft.
|
||||
/// Every authenticated network member may edit any service.
|
||||
pub async fn save_vip(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
config: crate::vip::VipConfig,
|
||||
expected: Option<String>,
|
||||
delete: bool,
|
||||
) -> Result<Vec<crate::vip::VipStatus>> {
|
||||
let (reply, receive) = oneshot::channel();
|
||||
self.command(
|
||||
network,
|
||||
NetCommand::SaveVip {
|
||||
config,
|
||||
expected,
|
||||
delete,
|
||||
reply,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
receive.await.map_err(|_| Error::Stopped)?
|
||||
}
|
||||
/// Retires one offline member's observed claim without revoking membership.
|
||||
/// Every member has this right. The version must still match the selection.
|
||||
pub async fn forget_node(
|
||||
@@ -916,6 +938,34 @@ impl Agent {
|
||||
.storage
|
||||
.retirements(network_id, self.inner.limits.max_retirements)
|
||||
.await?;
|
||||
let vip_entries = self
|
||||
.inner
|
||||
.storage
|
||||
.vip_fields(
|
||||
network_id,
|
||||
self.inner
|
||||
.limits
|
||||
.max_vips
|
||||
.saturating_mul(self.inner.limits.max_vip_fields)
|
||||
.saturating_add(self.inner.limits.max_vip_tombstones),
|
||||
)
|
||||
.await?;
|
||||
let mut vips = crate::vip::VipSet::default();
|
||||
for id in vip_entries
|
||||
.iter()
|
||||
.map(|e| e.vip.clone())
|
||||
.collect::<std::collections::BTreeSet<_>>()
|
||||
{
|
||||
vips.merge(
|
||||
network_id,
|
||||
vip_entries
|
||||
.iter()
|
||||
.filter(|e| e.vip == id)
|
||||
.cloned()
|
||||
.collect(),
|
||||
&self.inner.limits,
|
||||
)?;
|
||||
}
|
||||
let range = self.desired_range(network_id).await?;
|
||||
if let Some(range) = range {
|
||||
let conflicts = self.inner.routes.overlapping_networks(network_id, range);
|
||||
@@ -968,6 +1018,7 @@ impl Agent {
|
||||
.and_then(|stored| stored.exit_via)
|
||||
.and_then(|bytes| EndpointId::from_bytes(&bytes).ok());
|
||||
let handle = network::spawn(RuntimeParams {
|
||||
vips,
|
||||
retirements,
|
||||
broadcast,
|
||||
exit_offer,
|
||||
@@ -1215,7 +1266,43 @@ impl Agent {
|
||||
let keys = NetworkKeys::derive(&stored.name, &stored.secret);
|
||||
let range = self.desired_range(stored.network_id).await?;
|
||||
let range_conflicts = self.range_conflicts(stored.network_id, range).await?;
|
||||
let fields = self
|
||||
.inner
|
||||
.storage
|
||||
.vip_fields(
|
||||
stored.network_id,
|
||||
self.inner
|
||||
.limits
|
||||
.max_vips
|
||||
.saturating_mul(self.inner.limits.max_vip_fields)
|
||||
.saturating_add(self.inner.limits.max_vip_tombstones),
|
||||
)
|
||||
.await?;
|
||||
let mut service_state = crate::vip::VipSet::default();
|
||||
for id in fields
|
||||
.iter()
|
||||
.map(|e| e.vip.clone())
|
||||
.collect::<std::collections::BTreeSet<_>>()
|
||||
{
|
||||
service_state.merge(
|
||||
stored.network_id,
|
||||
fields.iter().filter(|e| e.vip == id).cloned().collect(),
|
||||
&self.inner.limits,
|
||||
)?;
|
||||
}
|
||||
let vips = service_state
|
||||
.configs()
|
||||
.into_iter()
|
||||
.map(|config| crate::vip::VipStatus {
|
||||
revision: service_state.revision(&config.id),
|
||||
config,
|
||||
address: None,
|
||||
error: Some("network is inactive".into()),
|
||||
listeners: vec![],
|
||||
})
|
||||
.collect();
|
||||
networks.push(NetworkStatus {
|
||||
vips,
|
||||
protocols: self.effective_protocols(stored.protocols.as_ref()),
|
||||
auto_start: stored.auto_start,
|
||||
broadcast: stored.broadcast,
|
||||
|
||||
@@ -50,6 +50,12 @@ pub(crate) struct InboundSession {
|
||||
|
||||
/// Commands accepted by a network runtime.
|
||||
pub(crate) enum NetCommand {
|
||||
SaveVip {
|
||||
config: crate::vip::VipConfig,
|
||||
expected: Option<String>,
|
||||
delete: bool,
|
||||
reply: oneshot::Sender<Result<Vec<crate::vip::VipStatus>>>,
|
||||
},
|
||||
ForgetNode {
|
||||
peer: EndpointId,
|
||||
expected_version: u64,
|
||||
@@ -125,6 +131,7 @@ pub(crate) enum NetCommand {
|
||||
impl std::fmt::Debug for NetCommand {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
NetCommand::SaveVip { .. } => f.write_str("SaveVip"),
|
||||
NetCommand::ForgetNode { peer, .. } => write!(f, "ForgetNode({peer})"),
|
||||
NetCommand::Inbound(_) => f.write_str("Inbound"),
|
||||
NetCommand::InboundLink(link) => write!(f, "InboundLink({})", link.protocol),
|
||||
@@ -170,6 +177,7 @@ impl NetworkHandle {
|
||||
|
||||
/// Everything a network runtime needs to run.
|
||||
pub(crate) struct RuntimeParams {
|
||||
pub(crate) vips: crate::vip::VipSet,
|
||||
pub(crate) retirements: Vec<SignedRetirement>,
|
||||
pub(crate) broadcast: bool,
|
||||
/// This agent offers itself as an exit node here.
|
||||
@@ -376,6 +384,8 @@ struct Runtime {
|
||||
/// Snapshots received while dispatching, handled on the next loop pass.
|
||||
pending_state: Vec<(EndpointId, Vec<SignedRecord>)>,
|
||||
pending_retirements: Vec<(EndpointId, Vec<SignedRetirement>)>,
|
||||
pending_vips: Vec<Vec<crate::vip::VipEntry>>,
|
||||
vip_cursor: HashMap<EndpointId, usize>,
|
||||
/// Next cleanup batch per live session; prevents queue pressure starving a suffix.
|
||||
retirement_cursor: HashMap<EndpointId, usize>,
|
||||
/// The highest version this agent has ever published for this network.
|
||||
@@ -461,6 +471,8 @@ impl Runtime {
|
||||
state: StateSet::new(),
|
||||
pending_state: Vec::new(),
|
||||
pending_retirements: Vec::new(),
|
||||
pending_vips: Vec::new(),
|
||||
vip_cursor: HashMap::new(),
|
||||
retirement_cursor: HashMap::new(),
|
||||
own_version: 0,
|
||||
exit_via_was_online: false,
|
||||
@@ -561,6 +573,15 @@ impl Runtime {
|
||||
|
||||
async fn handle_command(&mut self, command: NetCommand) {
|
||||
match command {
|
||||
NetCommand::SaveVip {
|
||||
config,
|
||||
expected,
|
||||
delete,
|
||||
reply,
|
||||
} => {
|
||||
let result = self.save_vip(config, expected, delete).await;
|
||||
let _ = reply.send(result);
|
||||
}
|
||||
NetCommand::ForgetNode {
|
||||
peer,
|
||||
expected_version,
|
||||
@@ -830,6 +851,7 @@ impl Runtime {
|
||||
// the check is a comparison when nothing has changed.
|
||||
self.ensure_own_claim().await;
|
||||
self.report_hostname_taken();
|
||||
self.broadcast_vips();
|
||||
self.broadcast_retirements();
|
||||
// The same timer repairs the relay picture: reachability is soft
|
||||
// state, repeated while it holds and forgotten when it stops.
|
||||
@@ -1104,7 +1126,8 @@ impl Runtime {
|
||||
.own_record_version(self.network_id, self.local_id)
|
||||
.await
|
||||
.unwrap_or(0)
|
||||
.max(self.state.authored_version(&self.local_id));
|
||||
.max(self.state.authored_version(&self.local_id))
|
||||
.max(self.params.vips.authored_version(self.local_id));
|
||||
for (id, hostname) in self
|
||||
.params
|
||||
.storage
|
||||
@@ -1178,11 +1201,26 @@ impl Runtime {
|
||||
match self.state.address_of(&self.local_id) {
|
||||
Some(mine) if range.contains(mine) => Some(mine),
|
||||
_ => {
|
||||
let taken: std::collections::HashSet<std::net::Ipv4Addr> = holders
|
||||
let mut taken: std::collections::HashSet<std::net::Ipv4Addr> = holders
|
||||
.iter()
|
||||
.filter(|(_, holder)| **holder != self.local_id)
|
||||
.map(|(address, _)| *address)
|
||||
.collect();
|
||||
for vip in self.params.vips.views(
|
||||
Some(range),
|
||||
taken.clone(),
|
||||
&std::collections::BTreeSet::new(),
|
||||
&self.params.limits,
|
||||
) {
|
||||
if let Some(address) = vip.address {
|
||||
taken.insert(address);
|
||||
}
|
||||
}
|
||||
for vip in self.params.routes.vips.statuses(self.network_id) {
|
||||
if let Some(address) = vip.address {
|
||||
taken.insert(address);
|
||||
}
|
||||
}
|
||||
match allocate(
|
||||
self.network_id,
|
||||
self.local_id,
|
||||
@@ -1460,8 +1498,188 @@ impl Runtime {
|
||||
self.broadcast_state();
|
||||
}
|
||||
|
||||
fn update_vips(&self) {
|
||||
let taken = self.state.address_holders().into_keys().collect();
|
||||
let names = self
|
||||
.state
|
||||
.active_records()
|
||||
.into_iter()
|
||||
.filter_map(|r| {
|
||||
r.author_id()
|
||||
.ok()
|
||||
.and_then(|p| self.state.hostname_of(&p).map(str::to_owned))
|
||||
})
|
||||
.collect();
|
||||
let views =
|
||||
self.params
|
||||
.vips
|
||||
.views(self.effective_range(), taken, &names, &self.params.limits);
|
||||
let peers = self
|
||||
.state
|
||||
.address_holders()
|
||||
.into_iter()
|
||||
.map(|(a, p)| (p, a, self.hub.reachable(p)))
|
||||
.collect();
|
||||
self.params.routes.vips.configure(
|
||||
self.network_id,
|
||||
self.local_id,
|
||||
self.state.address_of(&self.local_id),
|
||||
views,
|
||||
peers,
|
||||
&self.params.limits,
|
||||
);
|
||||
}
|
||||
async fn save_vip(
|
||||
&mut self,
|
||||
config: crate::vip::VipConfig,
|
||||
expected: Option<String>,
|
||||
delete: bool,
|
||||
) -> Result<Vec<crate::vip::VipStatus>> {
|
||||
if self.released || self.shutdown.is_triggered() {
|
||||
return Err(Error::Stopped);
|
||||
}
|
||||
let version = self
|
||||
.own_version
|
||||
.checked_add(1)
|
||||
.filter(|n| *n <= i64::MAX as u64)
|
||||
.ok_or_else(|| crate::vip::invalid("author counter exhausted"))?;
|
||||
let entries = if delete {
|
||||
self.params.vips.delete(
|
||||
self.network_id,
|
||||
&self.params.device_secret,
|
||||
version,
|
||||
&config.id,
|
||||
expected.as_deref().unwrap_or(""),
|
||||
)?
|
||||
} else {
|
||||
self.params.vips.edit(
|
||||
self.network_id,
|
||||
&self.params.device_secret,
|
||||
version,
|
||||
&config,
|
||||
expected.as_deref(),
|
||||
&self.params.limits,
|
||||
)?
|
||||
};
|
||||
let mut candidate = self.params.vips.clone();
|
||||
candidate.merge(self.network_id, entries, &self.params.limits)?;
|
||||
if !delete {
|
||||
let duplicate = candidate
|
||||
.configs()
|
||||
.iter()
|
||||
.any(|v| v.id != config.id && v.name == config.name)
|
||||
|| self.state.active_records().iter().any(|r| {
|
||||
r.author_id().ok().and_then(|p| self.state.hostname_of(&p))
|
||||
== Some(config.name.as_str())
|
||||
});
|
||||
if duplicate {
|
||||
return Err(crate::vip::invalid(
|
||||
"VIP name is already used by a node or another VIP",
|
||||
));
|
||||
}
|
||||
}
|
||||
let snapshot = candidate.entries(&config.id);
|
||||
self.validate_vip_frame(&snapshot)?;
|
||||
self.params
|
||||
.storage
|
||||
.put_vip_fields(
|
||||
self.network_id,
|
||||
snapshot.clone(),
|
||||
Some((*self.local_id.as_bytes(), version)),
|
||||
)
|
||||
.await?;
|
||||
self.own_version = version;
|
||||
self.params.vips = candidate;
|
||||
self.update_vips();
|
||||
let peers: Vec<_> = self.sessions.keys().copied().collect();
|
||||
for peer in peers {
|
||||
let _ = self.send_to(
|
||||
peer,
|
||||
ControlMessage::Vips {
|
||||
entries: snapshot.clone(),
|
||||
},
|
||||
);
|
||||
}
|
||||
Ok(self.params.routes.vips.statuses(self.network_id))
|
||||
}
|
||||
async fn receive_vips(&mut self, entries: Vec<crate::vip::VipEntry>) -> Result<()> {
|
||||
let Some(id) = entries.first().map(|e| e.vip.clone()) else {
|
||||
return Ok(());
|
||||
};
|
||||
if entries.iter().any(|e| e.vip != id) {
|
||||
return Err(crate::vip::invalid("VIP snapshot mixes service identities"));
|
||||
}
|
||||
let own_seen = entries
|
||||
.iter()
|
||||
.filter(|e| e.author == *self.local_id.as_bytes())
|
||||
.map(|e| e.version)
|
||||
.max()
|
||||
.unwrap_or(0);
|
||||
let mut candidate = self.params.vips.clone();
|
||||
let changed = candidate.merge(self.network_id, entries, &self.params.limits)?;
|
||||
if !changed && own_seen <= self.own_version {
|
||||
return Ok(());
|
||||
}
|
||||
let counter =
|
||||
(own_seen > self.own_version).then_some((*self.local_id.as_bytes(), own_seen));
|
||||
let snapshot = candidate.entries(&id);
|
||||
self.validate_vip_frame(&snapshot)?;
|
||||
self.params
|
||||
.storage
|
||||
.put_vip_fields(self.network_id, snapshot, counter)
|
||||
.await?;
|
||||
self.own_version = self.own_version.max(own_seen);
|
||||
self.params.vips = candidate;
|
||||
self.update_vips();
|
||||
self.broadcast_vips();
|
||||
Ok(())
|
||||
}
|
||||
fn validate_vip_frame(&self, entries: &[crate::vip::VipEntry]) -> Result<()> {
|
||||
let frame = Envelope {
|
||||
network_id: *self.network_id.as_bytes(),
|
||||
message: ControlMessage::Vips {
|
||||
entries: entries.to_vec(),
|
||||
},
|
||||
};
|
||||
let bytes = encode(&frame).map_err(|e| crate::vip::invalid(e.to_string()))?;
|
||||
if bytes.len() > self.params.limits.max_frame_len {
|
||||
return Err(crate::vip::invalid(
|
||||
"VIP snapshot exceeds the control frame limit",
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
fn broadcast_vips(&mut self) {
|
||||
let ids = self.params.vips.ids();
|
||||
if ids.is_empty() {
|
||||
return;
|
||||
}
|
||||
let peers: Vec<_> = self.sessions.keys().copied().collect();
|
||||
for peer in peers {
|
||||
let start = self.vip_cursor.get(&peer).copied().unwrap_or(0) % ids.len();
|
||||
for step in 0..ids.len() {
|
||||
let index = (start + step) % ids.len();
|
||||
let entries = self.params.vips.entries(&ids[index]);
|
||||
if self
|
||||
.send_to(peer, ControlMessage::Vips { entries })
|
||||
.is_err()
|
||||
{
|
||||
self.vip_cursor.insert(peer, index);
|
||||
break;
|
||||
}
|
||||
self.vip_cursor.insert(peer, (index + 1) % ids.len());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Handles snapshots collected while dispatching messages.
|
||||
async fn drain_pending_state(&mut self) {
|
||||
for entries in std::mem::take(&mut self.pending_vips) {
|
||||
if let Err(err) = self.receive_vips(entries).await {
|
||||
self.metrics.protocol_violations += 1;
|
||||
tracing::warn!(%err,"rejecting VIP snapshot");
|
||||
}
|
||||
}
|
||||
for (peer, markers) in std::mem::take(&mut self.pending_retirements) {
|
||||
self.receive_retirements(peer, markers).await;
|
||||
}
|
||||
@@ -1473,6 +1691,7 @@ impl Runtime {
|
||||
|
||||
/// Sends everything we know to every peer.
|
||||
fn broadcast_state(&mut self) {
|
||||
self.broadcast_vips();
|
||||
self.broadcast_retirements();
|
||||
let mut records = self.state.records();
|
||||
records.truncate(self.params.limits.max_state_records);
|
||||
@@ -1671,6 +1890,7 @@ impl Runtime {
|
||||
self.params.range_changed.send_replace(());
|
||||
return;
|
||||
}
|
||||
self.update_vips();
|
||||
for plugin in &self.params.plugins {
|
||||
plugin.on_address_allocation(self.network_id, range, &allocations);
|
||||
}
|
||||
@@ -2154,6 +2374,7 @@ impl Runtime {
|
||||
});
|
||||
}
|
||||
}
|
||||
self.update_vips();
|
||||
}
|
||||
|
||||
fn handle_link_result(&mut self, outcome: LinkOutcome) {
|
||||
@@ -2304,6 +2525,7 @@ impl Runtime {
|
||||
/// Drops every link to a peer.
|
||||
fn drop_links_for(&mut self, peer: EndpointId) {
|
||||
self.retirement_cursor.remove(&peer);
|
||||
self.vip_cursor.remove(&peer);
|
||||
self.reset_peer_data(peer, "peer session ended");
|
||||
self.hub.remove_peer(peer);
|
||||
self.reachable.remove(&peer);
|
||||
@@ -2664,6 +2886,7 @@ impl Runtime {
|
||||
tracing::debug!(%err, "could not queue pong");
|
||||
}
|
||||
}
|
||||
ControlMessage::Vips { entries } => self.pending_vips.push(entries.clone()),
|
||||
ControlMessage::State { records } => {
|
||||
self.pending_state.push((peer, records.clone()));
|
||||
}
|
||||
@@ -2891,6 +3114,7 @@ impl Runtime {
|
||||
members.dedup_by(|a, b| a.endpoint_id == b.endpoint_id);
|
||||
|
||||
NetworkStatus {
|
||||
vips: self.params.routes.vips.statuses(self.network_id),
|
||||
protocols: self
|
||||
.params
|
||||
.plugins
|
||||
|
||||
@@ -225,6 +225,8 @@ pub struct RangeConflict {
|
||||
/// Status of one network.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NetworkStatus {
|
||||
/// Shared service configuration with this client's routing observations.
|
||||
pub vips: Vec<crate::vip::VipStatus>,
|
||||
/// Enabled local data-plane protocols, in preference order.
|
||||
pub protocols: Vec<String>,
|
||||
/// Whether this network is requested to start or resume automatically.
|
||||
|
||||
@@ -158,6 +158,22 @@ pub enum TransportPolicy {
|
||||
/// object where that is possible (notably [`Limits::max_frame_len`]).
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Limits {
|
||||
/// Maximum active VIP identities per network.
|
||||
pub max_vips: usize,
|
||||
/// Maximum permanent VIP deletion tombstones per network.
|
||||
pub max_vip_tombstones: usize,
|
||||
/// Maximum concurrent active TCP port checks across the agent.
|
||||
pub max_vip_checks: usize,
|
||||
/// Maximum signed fields per VIP, including removed fields.
|
||||
pub max_vip_fields: usize,
|
||||
/// Maximum listeners per VIP.
|
||||
pub max_vip_listeners: usize,
|
||||
/// Maximum backend endpoints per listener.
|
||||
pub max_vip_backends: usize,
|
||||
/// Maximum simultaneous local VIP flow bindings.
|
||||
pub max_vip_flows: usize,
|
||||
/// Maximum incomplete IPv4 datagrams per network in VIP reassembly.
|
||||
pub max_vip_fragments: usize,
|
||||
/// Largest accepted control frame payload, in bytes.
|
||||
pub max_frame_len: usize,
|
||||
/// Largest accepted hostname, in bytes.
|
||||
@@ -205,6 +221,14 @@ pub struct Limits {
|
||||
impl Default for Limits {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
max_vips: 64,
|
||||
max_vip_tombstones: 1024,
|
||||
max_vip_checks: 32,
|
||||
max_vip_fields: 192,
|
||||
max_vip_listeners: 16,
|
||||
max_vip_backends: 32,
|
||||
max_vip_flows: 4096,
|
||||
max_vip_fragments: 64,
|
||||
max_frame_len: 64 * 1024,
|
||||
max_hostname_len: 255,
|
||||
max_capabilities: 16,
|
||||
|
||||
@@ -171,6 +171,17 @@ pub fn default_control_socket_path() -> PathBuf {
|
||||
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[non_exhaustive]
|
||||
pub enum Request {
|
||||
/// Save a complete version-checked service draft or its deletion.
|
||||
SaveVip {
|
||||
/// Exact network id.
|
||||
network_id: String,
|
||||
/// Entire desired configuration.
|
||||
config: crate::vip::VipConfig,
|
||||
/// Revision originally displayed; absent for creation.
|
||||
expected: Option<String>,
|
||||
/// Remove the service permanently.
|
||||
delete: bool,
|
||||
},
|
||||
/// Retire the selected offline member's observed claim.
|
||||
ForgetNode {
|
||||
/// Exact public network id.
|
||||
@@ -271,6 +282,9 @@ pub enum Request {
|
||||
impl std::fmt::Debug for Request {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Request::SaveVip {
|
||||
network_id, config, ..
|
||||
} => write!(f, "SaveVip {{ {network_id}, {} }}", config.id),
|
||||
Request::ForgetNode {
|
||||
network_id,
|
||||
peer,
|
||||
@@ -317,6 +331,8 @@ impl std::fmt::Debug for Request {
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[non_exhaustive]
|
||||
pub enum Response {
|
||||
/// Accepted configuration with fresh local routing observations.
|
||||
Vips(Vec<crate::vip::VipStatus>),
|
||||
/// An observed offline claim was durably retired.
|
||||
NodeForgotten(ForgottenNodeReport),
|
||||
/// Accepted ordered protocol selection.
|
||||
@@ -358,6 +374,57 @@ pub struct ForgottenNodeReport {
|
||||
pub peers_told: u32,
|
||||
}
|
||||
|
||||
/// Saves a whole VIP draft through the running agent.
|
||||
pub async fn save_vip(
|
||||
socket: &Path,
|
||||
network_id: &str,
|
||||
config: crate::vip::VipConfig,
|
||||
expected: Option<String>,
|
||||
delete: bool,
|
||||
) -> Result<Vec<crate::vip::VipStatus>> {
|
||||
match exchange(
|
||||
socket,
|
||||
&Request::SaveVip {
|
||||
network_id: network_id.into(),
|
||||
config,
|
||||
expected,
|
||||
delete,
|
||||
},
|
||||
EXCHANGE_TIMEOUT,
|
||||
)
|
||||
.await?
|
||||
{
|
||||
Response::Vips(vips) => Ok(vips),
|
||||
Response::Error(reason) => Err(Error::Rejected { reason }),
|
||||
_ => Err(Error::Rejected {
|
||||
reason: "unexpected VIP response".into(),
|
||||
}),
|
||||
}
|
||||
}
|
||||
/// Resolves a service by exact name, full id or unique id prefix.
|
||||
pub fn resolve_vip<'a>(
|
||||
network: &'a NetworkReport,
|
||||
selector: &str,
|
||||
) -> std::result::Result<&'a crate::vip::VipStatus, String> {
|
||||
let matches: Vec<_> = network
|
||||
.vips
|
||||
.iter()
|
||||
.filter(|v| v.config.name == selector || v.config.id.starts_with(selector))
|
||||
.collect();
|
||||
match matches.as_slice() {
|
||||
[vip] => Ok(vip),
|
||||
[] => Err(format!("no VIP matches `{selector}`")),
|
||||
_ => Err(format!(
|
||||
"ambiguous VIP `{selector}`: {}",
|
||||
matches
|
||||
.iter()
|
||||
.map(|v| format!("{} ({})", v.config.name, v.config.id))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ")
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolves a node by exact hostname, full id, unique id prefix or overlay IPv4.
|
||||
/// Ambiguity is reported with names and full ids and never guessed.
|
||||
pub fn resolve_node<'a>(
|
||||
@@ -642,6 +709,9 @@ pub struct ProtocolReport {
|
||||
/// One network.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct NetworkReport {
|
||||
/// Shared VIPs and this client's routing observations.
|
||||
#[serde(default)]
|
||||
pub vips: Vec<crate::vip::VipStatus>,
|
||||
/// Enabled local data-plane protocols, in preference order.
|
||||
pub protocols: Vec<String>,
|
||||
/// Whether the network starts or resumes automatically when its range is free.
|
||||
@@ -905,6 +975,16 @@ impl OverlayPeerReport {
|
||||
/// running and what they can report or change. That is what keeps this module
|
||||
/// free of any knowledge of them.
|
||||
pub trait ReportSource: Send + Sync + 'static {
|
||||
/// Atomically saves a complete VIP draft using its original revision.
|
||||
fn save_vip(
|
||||
&self,
|
||||
_network: String,
|
||||
_config: crate::vip::VipConfig,
|
||||
_expected: Option<String>,
|
||||
_delete: bool,
|
||||
) -> BoxFuture<'_, std::result::Result<Vec<crate::vip::VipStatus>, String>> {
|
||||
Box::pin(async { Err("this agent cannot manage VIPs".into()) })
|
||||
}
|
||||
/// Persists and announces a version-scoped node cleanup.
|
||||
fn forget_node(
|
||||
&self,
|
||||
@@ -1051,7 +1131,7 @@ pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
///
|
||||
/// Bump it whenever [`Request`], [`Response`] or anything they contain
|
||||
/// changes shape.
|
||||
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 21]);
|
||||
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 22]);
|
||||
|
||||
/// Reads one request off an accepted stream, answers it, writes the response.
|
||||
///
|
||||
@@ -1075,6 +1155,15 @@ where
|
||||
}
|
||||
};
|
||||
let response = match request {
|
||||
Request::SaveVip {
|
||||
network_id,
|
||||
config,
|
||||
expected,
|
||||
delete,
|
||||
} => match source.save_vip(network_id, config, expected, delete).await {
|
||||
Ok(vips) => Response::Vips(vips),
|
||||
Err(reason) => Response::Error(reason),
|
||||
},
|
||||
Request::ForgetNode {
|
||||
network_id,
|
||||
peer,
|
||||
|
||||
@@ -51,6 +51,7 @@ pub mod storage;
|
||||
pub mod task;
|
||||
#[cfg(feature = "testing")]
|
||||
pub mod testing;
|
||||
pub mod vip;
|
||||
|
||||
pub use agent::{Agent, AgentStatus, Event, NetworkStatus, PeerStatus};
|
||||
pub use config::{AgentConfig, Limits, ReconnectPolicy, StoragePaths, TransportPolicy};
|
||||
|
||||
@@ -145,18 +145,19 @@ impl Tally {
|
||||
}
|
||||
|
||||
/// The overlay interface.
|
||||
#[derive(Debug)]
|
||||
pub struct Interface {
|
||||
device: Arc<dyn TunDevice>,
|
||||
factory: Arc<dyn TunFactory>,
|
||||
name: String,
|
||||
mtu: u32,
|
||||
routes: Arc<RoutingTable>,
|
||||
carrier: Arc<dyn PacketCarrier>,
|
||||
tally: Arc<Tally>,
|
||||
/// What was last applied to the host, so an unchanged table is not
|
||||
/// re-applied on every pass.
|
||||
applied: std::sync::Mutex<Vec<Cidr>>,
|
||||
task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
vip_task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
sync_lock: Arc<tokio::sync::Mutex<()>>,
|
||||
/// The host rules for this interface, when the factory manages the host.
|
||||
host_rules: Option<Arc<dyn BroadcastHostRules>>,
|
||||
@@ -200,6 +201,14 @@ impl ExitState {
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for Interface {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("Interface")
|
||||
.field("name", &self.name)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
|
||||
impl Interface {
|
||||
/// Creates the interface and starts moving packets.
|
||||
pub async fn start(
|
||||
@@ -221,7 +230,29 @@ impl Interface {
|
||||
let name = device.name().to_string();
|
||||
let tally = Arc::new(Tally::default());
|
||||
|
||||
let vip_task = {
|
||||
let routes = Arc::clone(&routes);
|
||||
let carrier = Arc::clone(&carrier);
|
||||
tokio::spawn(async move {
|
||||
let mut interval = tokio::time::interval(std::time::Duration::from_millis(250));
|
||||
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
|
||||
let mut checks = tokio::task::JoinSet::new();
|
||||
loop {
|
||||
tokio::select! {
|
||||
_=interval.tick()=>{
|
||||
let (probes,jobs)=routes.vips.tick(routes.vips.check_capacity().saturating_sub(checks.len()));
|
||||
for (route,packet) in probes {carrier.carry(route,packet);}
|
||||
for job in jobs {checks.spawn(job.run());}
|
||||
},
|
||||
Some(result)=checks.join_next(),if !checks.is_empty()=>{
|
||||
if let Ok((job,ok))=result {routes.vips.checked(job,ok);}
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
};
|
||||
let task = {
|
||||
let carrier = Arc::clone(&carrier);
|
||||
let device = Arc::clone(&device);
|
||||
let routes = Arc::clone(&routes);
|
||||
let tally = Arc::clone(&tally);
|
||||
@@ -231,6 +262,26 @@ impl Interface {
|
||||
tally.malformed.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
};
|
||||
match routes.vips.outgoing(packet.clone()) {
|
||||
crate::vip::router::Decision::Send(route, packet) => {
|
||||
for part in crate::vip::packet::fragments(packet, mtu) {
|
||||
if carrier.carry(route, part) {
|
||||
tally.sent.fetch_add(1, Ordering::Relaxed);
|
||||
} else {
|
||||
tally.undeliverable.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
crate::vip::router::Decision::Local(packet) => {
|
||||
for part in crate::vip::packet::fragments(packet, mtu) {
|
||||
let _ = device.send(part).await;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
crate::vip::router::Decision::Drop => continue,
|
||||
crate::vip::router::Decision::Pass => {}
|
||||
}
|
||||
let destination = header.destination();
|
||||
if let (IpAddr::V4(source), IpAddr::V4(destination)) =
|
||||
(header.source(), destination)
|
||||
@@ -279,9 +330,11 @@ impl Interface {
|
||||
name,
|
||||
mtu,
|
||||
routes,
|
||||
carrier,
|
||||
tally,
|
||||
applied: std::sync::Mutex::new(Vec::new()),
|
||||
task: std::sync::Mutex::new(Some(task)),
|
||||
vip_task: std::sync::Mutex::new(Some(vip_task)),
|
||||
sync_lock: Arc::new(tokio::sync::Mutex::new(())),
|
||||
host_rules,
|
||||
rules_report: std::sync::Mutex::new(None),
|
||||
@@ -716,6 +769,15 @@ impl Interface {
|
||||
task.abort();
|
||||
let _ = task.await;
|
||||
}
|
||||
let vip_task = self
|
||||
.vip_task
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.take();
|
||||
if let Some(task) = vip_task {
|
||||
task.abort();
|
||||
let _ = task.await;
|
||||
}
|
||||
self.factory.destroy(self.device.name()).await;
|
||||
}
|
||||
|
||||
@@ -770,6 +832,24 @@ impl Interface {
|
||||
return Err(Rejected::WrongDestination);
|
||||
}
|
||||
};
|
||||
// Source ownership was checked before any VIP reverse translation.
|
||||
let packet = match self.routes.vips.incoming(network, peer, packet.clone()) {
|
||||
crate::vip::router::Decision::Send(route, packet) => {
|
||||
// Probe replies use the ordinary peer carrier, never the TUN.
|
||||
self.carrier.carry(route, packet);
|
||||
return Ok(());
|
||||
}
|
||||
crate::vip::router::Decision::Local(packet) => {
|
||||
for part in crate::vip::packet::fragments(packet, self.mtu) {
|
||||
if self.device.send(part).await.is_ok() {
|
||||
self.tally.received.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
return Ok(());
|
||||
}
|
||||
crate::vip::router::Decision::Drop => return Ok(()),
|
||||
crate::vip::router::Decision::Pass => packet,
|
||||
};
|
||||
// Remote broadcasts terminate here. Only local TUN ingress can fan out.
|
||||
if self.device.send(packet).await.is_ok() {
|
||||
if broadcast {
|
||||
@@ -794,6 +874,14 @@ impl Interface {
|
||||
|
||||
impl Drop for Interface {
|
||||
fn drop(&mut self) {
|
||||
if let Some(task) = self
|
||||
.vip_task
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.take()
|
||||
{
|
||||
task.abort();
|
||||
}
|
||||
if let Ok(mut guard) = self.task.lock()
|
||||
&& let Some(task) = guard.take()
|
||||
{
|
||||
|
||||
@@ -89,6 +89,7 @@ pub enum RouteError {
|
||||
/// Address ownership across every network this agent is in.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct RoutingTable {
|
||||
pub(crate) vips: Arc<crate::vip::VipRouter>,
|
||||
broadcast: ArcSwap<BroadcastTable>,
|
||||
networks: RwLock<HashMap<NetworkId, NetworkRoutes>>,
|
||||
}
|
||||
@@ -146,6 +147,7 @@ impl RoutingTable {
|
||||
|
||||
/// Forgets a network.
|
||||
pub fn remove_network(&self, network: NetworkId) {
|
||||
self.vips.remove_network(network);
|
||||
let mut networks = self.write();
|
||||
networks.remove(&network);
|
||||
self.broadcast
|
||||
|
||||
@@ -20,7 +20,7 @@ use crate::error::ProtocolError;
|
||||
/// The version in the ALPN is the wire-compatibility version of the control
|
||||
/// protocol. It is independent of the network identity scheme version, so
|
||||
/// bumping it must not change any existing [`crate::NetworkId`].
|
||||
pub const ALPN: &[u8] = b"tsunagi/ctrl/7";
|
||||
pub const ALPN: &[u8] = b"tsunagi/ctrl/8";
|
||||
|
||||
/// ALPN of the tsunagi data plane.
|
||||
///
|
||||
@@ -43,7 +43,7 @@ pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
|
||||
pub const MAX_SIGNATURE_LEN: usize = 64;
|
||||
|
||||
/// Control protocol version carried inside the handshake.
|
||||
pub const PROTOCOL_VERSION: u16 = 7;
|
||||
pub const PROTOCOL_VERSION: u16 = 8;
|
||||
|
||||
pub use crate::dataplane::transport::{MAX_TRANSPORT_PAYLOAD, TransportPayload};
|
||||
|
||||
@@ -181,6 +181,11 @@ pub enum ControlMessage {
|
||||
/// Short free-text reason.
|
||||
reason: String,
|
||||
},
|
||||
/// One complete signed VIP field snapshot, merged atomically.
|
||||
Vips {
|
||||
/// Signed registers bounded by the per-VIP field limit.
|
||||
entries: Vec<crate::vip::VipEntry>,
|
||||
},
|
||||
/// Signed cleanup markers, each proving the retired claim version.
|
||||
Retirements {
|
||||
/// Bounded by the per-message signed-state limit.
|
||||
@@ -278,6 +283,15 @@ pub fn validate_capability(
|
||||
/// network, the other networks or the agent.
|
||||
pub fn validate(message: &ControlMessage, limits: &Limits) -> Result<(), ProtocolError> {
|
||||
match message {
|
||||
ControlMessage::Vips { entries } => {
|
||||
check_len("vip.fields", entries.len(), limits.max_vip_fields)?;
|
||||
for e in entries {
|
||||
check_len("vip.id", e.vip.len(), 32)?;
|
||||
check_len("vip.key", e.key.len(), 160)?;
|
||||
check_len("vip.value", e.value.as_ref().map_or(0, Vec::len), 256)?;
|
||||
check_len("vip.signature", e.signature.len(), 64)?;
|
||||
}
|
||||
}
|
||||
ControlMessage::Retirements { markers } => {
|
||||
check_len("retirements", markers.len(), limits.max_state_records)?;
|
||||
for marker in markers {
|
||||
@@ -373,6 +387,7 @@ pub fn kind(message: &ControlMessage) -> &'static str {
|
||||
ControlMessage::Pong { .. } => "pong",
|
||||
ControlMessage::State { .. } => "state",
|
||||
ControlMessage::Retirements { .. } => "retirements",
|
||||
ControlMessage::Vips { .. } => "vips",
|
||||
ControlMessage::Reachable { .. } => "reachable",
|
||||
ControlMessage::Peers { .. } => "peers",
|
||||
ControlMessage::Bye { .. } => "bye",
|
||||
|
||||
@@ -38,6 +38,8 @@
|
||||
//! `docs/sync-model.md` for the implemented cleanup semantics.
|
||||
|
||||
pub mod allocator;
|
||||
/// Signed shared virtual service state and its merge rules.
|
||||
pub mod vip;
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::net::Ipv4Addr;
|
||||
|
||||
@@ -0,0 +1,439 @@
|
||||
//! Signed multi-writer virtual service registers and deterministic merge rules.
|
||||
use crate::error::Result;
|
||||
use crate::vip::{Backend, Listener, Protocol, VipConfig, VipStatus, invalid};
|
||||
use crate::{Limits, NetworkId};
|
||||
use iroh::{EndpointId, SecretKey, Signature};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sha2::{Digest, Sha256};
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::net::Ipv4Addr;
|
||||
|
||||
/// A signed, independently mergeable field. A removed field keeps its tombstone.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct VipEntry {
|
||||
/// Network scope.
|
||||
pub network: [u8; 32],
|
||||
/// Stable service id.
|
||||
pub vip: String,
|
||||
/// Canonical field key.
|
||||
pub key: String,
|
||||
/// Encoded field value; absent means deletion.
|
||||
pub value: Option<Vec<u8>>,
|
||||
/// Lamport clock, independent of wall time.
|
||||
pub clock: u64,
|
||||
/// Persistent signing identity.
|
||||
pub author: [u8; 32],
|
||||
/// Author's durable network counter.
|
||||
pub version: u64,
|
||||
/// Ed25519 signature.
|
||||
pub signature: Vec<u8>,
|
||||
}
|
||||
impl VipEntry {
|
||||
fn bytes(&self) -> Vec<u8> {
|
||||
let mut e = self.clone();
|
||||
e.signature.clear();
|
||||
let mut out = b"tsunagi-vip-field/v1\0".to_vec();
|
||||
if let Ok(bytes) = postcard::to_stdvec(&e) {
|
||||
out.extend(bytes);
|
||||
}
|
||||
out
|
||||
}
|
||||
/// Verifies bounds, network and authorship before the entry is merged.
|
||||
pub fn verify(&self, network: NetworkId) -> Result<()> {
|
||||
if self.network != *network.as_bytes()
|
||||
|| self.vip.len() != 32
|
||||
|| hex::decode(&self.vip).is_err()
|
||||
|| self.key.len() > 160
|
||||
|| self.value.as_ref().is_some_and(|v| v.len() > 256)
|
||||
|| self.version == 0
|
||||
|| self.version > i64::MAX as u64
|
||||
|| self.clock == 0
|
||||
|| self.clock > i64::MAX as u64
|
||||
{
|
||||
return Err(invalid("invalid VIP field bounds or network"));
|
||||
}
|
||||
let author =
|
||||
EndpointId::from_bytes(&self.author).map_err(|_| invalid("invalid VIP author"))?;
|
||||
let raw: [u8; 64] = self
|
||||
.signature
|
||||
.as_slice()
|
||||
.try_into()
|
||||
.map_err(|_| invalid("invalid VIP signature length"))?;
|
||||
author
|
||||
.verify(&self.bytes(), &Signature::from_bytes(&raw))
|
||||
.map_err(|_| invalid("invalid VIP signature"))?;
|
||||
if self.key != "deleted"
|
||||
&& self.key != "name"
|
||||
&& !self.key.starts_with("l/")
|
||||
&& !self.key.starts_with("b/")
|
||||
{
|
||||
return Err(invalid("unknown VIP field"));
|
||||
}
|
||||
match self.key.as_str() {
|
||||
"deleted" => {
|
||||
if self.value.as_deref() != Some(&[1][..]) {
|
||||
return Err(invalid("invalid VIP deletion tombstone"));
|
||||
}
|
||||
}
|
||||
"name" => {
|
||||
let value = self
|
||||
.value
|
||||
.as_ref()
|
||||
.ok_or_else(|| invalid("VIP name cannot be absent"))?;
|
||||
let name: String =
|
||||
postcard::from_bytes(value).map_err(|_| invalid("invalid VIP name field"))?;
|
||||
VipConfig {
|
||||
id: self.vip.clone(),
|
||||
name,
|
||||
listeners: vec![],
|
||||
}
|
||||
.validate(&Limits::default())?;
|
||||
}
|
||||
key if key.starts_with("l/") => {
|
||||
let parts: Vec<_> = key.split('/').collect();
|
||||
if parts.len() != 3
|
||||
|| parts[1].parse::<Protocol>().is_err()
|
||||
|| parts[2].parse::<u16>().ok().is_none_or(|p| p == 0)
|
||||
{
|
||||
return Err(invalid("invalid listener field key"));
|
||||
}
|
||||
if let Some(value) = &self.value {
|
||||
let l: Listener = postcard::from_bytes(value)
|
||||
.map_err(|_| invalid("invalid listener field"))?;
|
||||
if !l.backends.is_empty() || key != format!("l/{}/{}", l.protocol, l.port) {
|
||||
return Err(invalid("listener key does not match its value"));
|
||||
}
|
||||
VipConfig {
|
||||
id: self.vip.clone(),
|
||||
name: "validation".into(),
|
||||
listeners: vec![l],
|
||||
}
|
||||
.validate(&Limits::default())?;
|
||||
}
|
||||
}
|
||||
key if key.starts_with("b/") => {
|
||||
let parts: Vec<_> = key.split('/').collect();
|
||||
if parts.len() != 6
|
||||
|| parts[1].parse::<Protocol>().is_err()
|
||||
|| parts[2].parse::<u16>().ok().is_none_or(|p| p == 0)
|
||||
|| parts[3].len() != 32
|
||||
|| hex::decode(parts[3]).is_err()
|
||||
|| parts[4].parse::<EndpointId>().is_err()
|
||||
|| parts[5].parse::<u16>().ok().is_none_or(|p| p == 0)
|
||||
{
|
||||
return Err(invalid("invalid backend field key"));
|
||||
}
|
||||
if let Some(value) = &self.value {
|
||||
let b: Backend = postcard::from_bytes(value)
|
||||
.map_err(|_| invalid("invalid backend field"))?;
|
||||
if b.peer != parts[4] || b.port.to_string() != parts[5] {
|
||||
return Err(invalid("backend key does not match its value"));
|
||||
}
|
||||
}
|
||||
}
|
||||
_ => return Err(invalid("unknown VIP field")),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
fn rank(&self) -> (u64, [u8; 32], Vec<u8>) {
|
||||
(self.clock, self.author, self.bytes())
|
||||
}
|
||||
}
|
||||
/// Signed registers retained across absences, including deletion tombstones.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct VipSet {
|
||||
entries: BTreeMap<(String, String), VipEntry>,
|
||||
}
|
||||
impl VipSet {
|
||||
/// Highest retained version genuinely signed by this author.
|
||||
pub fn authored_version(&self, author: EndpointId) -> u64 {
|
||||
self.entries
|
||||
.values()
|
||||
.filter(|e| e.author == *author.as_bytes())
|
||||
.map(|e| e.version)
|
||||
.max()
|
||||
.unwrap_or(0)
|
||||
}
|
||||
/// Complete independently verifiable state of one service, including tombstones.
|
||||
pub fn entries(&self, id: &str) -> Vec<VipEntry> {
|
||||
self.entries
|
||||
.values()
|
||||
.filter(|e| e.vip == id)
|
||||
.cloned()
|
||||
.collect()
|
||||
}
|
||||
/// All retained service ids, including deleted services.
|
||||
pub fn ids(&self) -> Vec<String> {
|
||||
self.entries
|
||||
.keys()
|
||||
.map(|(id, _)| id.clone())
|
||||
.collect::<BTreeSet<_>>()
|
||||
.into_iter()
|
||||
.collect()
|
||||
}
|
||||
/// Opaque optimistic concurrency token for a whole service.
|
||||
pub fn revision(&self, id: &str) -> String {
|
||||
let mut hash = Sha256::new();
|
||||
for e in self.entries(id) {
|
||||
hash.update(e.bytes());
|
||||
hash.update(&e.signature);
|
||||
}
|
||||
hex::encode(hash.finalize())
|
||||
}
|
||||
/// Atomically verifies and merges one complete service snapshot.
|
||||
pub fn merge(
|
||||
&mut self,
|
||||
network: NetworkId,
|
||||
entries: Vec<VipEntry>,
|
||||
limits: &Limits,
|
||||
) -> Result<bool> {
|
||||
if entries.len() > limits.max_vip_fields {
|
||||
return Err(invalid("too many VIP fields"));
|
||||
}
|
||||
let mut next = self.clone();
|
||||
for e in entries {
|
||||
e.verify(network)?;
|
||||
let key = (e.vip.clone(), e.key.clone());
|
||||
if next
|
||||
.entries
|
||||
.get(&key)
|
||||
.is_none_or(|old| e.rank() > old.rank())
|
||||
{
|
||||
next.entries.insert(key, e);
|
||||
}
|
||||
}
|
||||
let deleted: BTreeSet<_> = next
|
||||
.entries
|
||||
.values()
|
||||
.filter(|e| e.key == "deleted" && e.value.is_some())
|
||||
.map(|e| e.vip.clone())
|
||||
.collect();
|
||||
// A permanent service tombstone alone prevents every former field's replay.
|
||||
next.entries
|
||||
.retain(|(id, key), _| !deleted.contains(id) || key == "deleted");
|
||||
let ids = next.ids();
|
||||
if ids.len().saturating_sub(deleted.len()) > limits.max_vips
|
||||
|| deleted.len() > limits.max_vip_tombstones
|
||||
|| ids
|
||||
.iter()
|
||||
.any(|id| next.entries(id).len() > limits.max_vip_fields)
|
||||
{
|
||||
return Err(invalid(
|
||||
"VIP state capacity exceeded; deletion protection is retained",
|
||||
));
|
||||
}
|
||||
let changed = self.entries != next.entries;
|
||||
*self = next;
|
||||
Ok(changed)
|
||||
}
|
||||
/// Materialized configuration. A service tombstone permanently retires its id.
|
||||
pub fn configs(&self) -> Vec<VipConfig> {
|
||||
self.ids()
|
||||
.into_iter()
|
||||
.filter_map(|id| self.config(&id))
|
||||
.collect()
|
||||
}
|
||||
fn config(&self, id: &str) -> Option<VipConfig> {
|
||||
let fields = self.entries(id);
|
||||
if fields
|
||||
.iter()
|
||||
.any(|e| e.key == "deleted" && e.value.is_some())
|
||||
{
|
||||
return None;
|
||||
}
|
||||
let name = fields
|
||||
.iter()
|
||||
.find(|e| e.key == "name")?
|
||||
.value
|
||||
.as_ref()
|
||||
.and_then(|v| postcard::from_bytes::<String>(v).ok())?;
|
||||
let mut listeners: Vec<Listener> = fields
|
||||
.iter()
|
||||
.filter(|e| e.key.starts_with("l/"))
|
||||
.filter_map(|e| e.value.as_ref().and_then(|v| postcard::from_bytes(v).ok()))
|
||||
.collect();
|
||||
for l in &mut listeners {
|
||||
l.backends = fields
|
||||
.iter()
|
||||
.filter(|e| {
|
||||
e.key
|
||||
.starts_with(&format!("b/{}/{}/{}/", l.protocol, l.port, l.id))
|
||||
})
|
||||
.filter_map(|e| e.value.as_ref().and_then(|v| postcard::from_bytes(v).ok()))
|
||||
.collect();
|
||||
l.backends.sort();
|
||||
}
|
||||
listeners.sort_by_key(|l| (l.protocol, l.port));
|
||||
Some(VipConfig {
|
||||
id: id.into(),
|
||||
name,
|
||||
listeners,
|
||||
})
|
||||
}
|
||||
/// Builds a signed patch from the user's complete, version-checked draft.
|
||||
pub fn edit(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
secret: &SecretKey,
|
||||
version: u64,
|
||||
config: &VipConfig,
|
||||
expected: Option<&str>,
|
||||
limits: &Limits,
|
||||
) -> Result<Vec<VipEntry>> {
|
||||
config.validate(limits)?;
|
||||
if let Some(revision) = expected {
|
||||
if revision != self.revision(&config.id) {
|
||||
return Err(invalid("VIP changed while editing; reload before saving"));
|
||||
}
|
||||
} else if !self.entries(&config.id).is_empty() {
|
||||
return Err(invalid("VIP id already exists"));
|
||||
}
|
||||
if self
|
||||
.entries
|
||||
.get(&(config.id.clone(), "deleted".into()))
|
||||
.is_some_and(|e| e.value.is_some())
|
||||
{
|
||||
return Err(invalid("VIP was deleted; create a new VIP"));
|
||||
}
|
||||
let before = self
|
||||
.config(&config.id)
|
||||
.map(|c| flatten(&c))
|
||||
.transpose()?
|
||||
.unwrap_or_default();
|
||||
let after = flatten(config)?;
|
||||
let keys: BTreeSet<_> = before.keys().chain(after.keys()).cloned().collect();
|
||||
let changes = keys
|
||||
.into_iter()
|
||||
.filter_map(|key| {
|
||||
let value = after.get(&key).cloned();
|
||||
(before.get(&key) != value.as_ref()).then_some((key, value))
|
||||
})
|
||||
.collect();
|
||||
self.sign(network, secret, version, &config.id, changes)
|
||||
}
|
||||
/// Builds a permanent service tombstone after checking the selected revision.
|
||||
pub fn delete(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
secret: &SecretKey,
|
||||
version: u64,
|
||||
id: &str,
|
||||
expected: &str,
|
||||
) -> Result<Vec<VipEntry>> {
|
||||
if self.config(id).is_none() || self.revision(id) != expected {
|
||||
return Err(invalid("VIP missing or changed while editing"));
|
||||
}
|
||||
self.sign(
|
||||
network,
|
||||
secret,
|
||||
version,
|
||||
id,
|
||||
vec![("deleted".into(), Some(vec![1]))],
|
||||
)
|
||||
}
|
||||
fn sign(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
secret: &SecretKey,
|
||||
version: u64,
|
||||
id: &str,
|
||||
changes: Vec<(String, Option<Vec<u8>>)>,
|
||||
) -> Result<Vec<VipEntry>> {
|
||||
let clock = self
|
||||
.entries
|
||||
.values()
|
||||
.map(|e| e.clock)
|
||||
.max()
|
||||
.unwrap_or(0)
|
||||
.checked_add(1)
|
||||
.filter(|n| *n <= i64::MAX as u64)
|
||||
.ok_or_else(|| invalid("VIP clock exhausted"))?;
|
||||
Ok(changes
|
||||
.into_iter()
|
||||
.map(|(key, value)| {
|
||||
let mut e = VipEntry {
|
||||
network: *network.as_bytes(),
|
||||
vip: id.into(),
|
||||
key,
|
||||
value,
|
||||
clock,
|
||||
author: *secret.public().as_bytes(),
|
||||
version,
|
||||
signature: vec![],
|
||||
};
|
||||
e.signature = secret.sign(&e.bytes()).to_bytes().to_vec();
|
||||
e
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
}
|
||||
pub(crate) fn flatten(cfg: &VipConfig) -> Result<BTreeMap<String, Vec<u8>>> {
|
||||
let mut fields = BTreeMap::new();
|
||||
fields.insert(
|
||||
"name".into(),
|
||||
postcard::to_stdvec(&cfg.name).map_err(|e| invalid(e.to_string()))?,
|
||||
);
|
||||
for l in &cfg.listeners {
|
||||
let mut settings = l.clone();
|
||||
settings.backends.clear();
|
||||
fields.insert(
|
||||
format!("l/{}/{}", l.protocol, l.port),
|
||||
postcard::to_stdvec(&settings).map_err(|e| invalid(e.to_string()))?,
|
||||
);
|
||||
for b in &l.backends {
|
||||
fields.insert(
|
||||
format!("b/{}/{}/{}/{}/{}", l.protocol, l.port, l.id, b.peer, b.port),
|
||||
postcard::to_stdvec(b).map_err(|e| invalid(e.to_string()))?,
|
||||
);
|
||||
}
|
||||
}
|
||||
Ok(fields)
|
||||
}
|
||||
impl VipSet {
|
||||
/// Allocates internal addresses deterministically against durable node claims.
|
||||
pub(crate) fn views(
|
||||
&self,
|
||||
range: Option<crate::state::Ipv4Range>,
|
||||
mut taken: std::collections::HashSet<Ipv4Addr>,
|
||||
hostnames: &BTreeSet<String>,
|
||||
limits: &Limits,
|
||||
) -> Vec<VipStatus> {
|
||||
let mut names = hostnames.clone();
|
||||
let mut result = Vec::new();
|
||||
for config in self.configs() {
|
||||
let mut error = config.validate(limits).err().map(|e| e.to_string());
|
||||
if !names.insert(config.name.clone()) {
|
||||
error = Some("name conflicts with another VIP or node".into());
|
||||
}
|
||||
let address = range.and_then(|r| {
|
||||
let usable = crate::state::allocator::usable_addresses(r);
|
||||
if usable == 0 {
|
||||
return None;
|
||||
}
|
||||
let digest = Sha256::digest(config.id.as_bytes());
|
||||
let seed = u64::from_be_bytes([
|
||||
digest[0], digest[1], digest[2], digest[3], digest[4], digest[5], digest[6],
|
||||
digest[7],
|
||||
]);
|
||||
let bits = 32 - u32::from(r.prefix_len);
|
||||
let mask = if bits == 32 { 0 } else { u32::MAX << bits };
|
||||
(0..usable).find_map(|step| {
|
||||
let offset = (seed.wrapping_add(step) % usable + 1) as u32;
|
||||
let ip = Ipv4Addr::from((u32::from(r.base) & mask) | offset);
|
||||
taken.insert(ip).then_some(ip)
|
||||
})
|
||||
});
|
||||
if address.is_none() {
|
||||
error = Some("no free overlay address".into());
|
||||
}
|
||||
result.push(VipStatus {
|
||||
revision: self.revision(&config.id),
|
||||
config,
|
||||
address,
|
||||
error,
|
||||
listeners: vec![],
|
||||
});
|
||||
}
|
||||
result
|
||||
}
|
||||
}
|
||||
@@ -370,6 +370,25 @@ impl Storage {
|
||||
.await
|
||||
}
|
||||
|
||||
/// Loads mandatory signed VIP fields with a strict capacity bound.
|
||||
pub async fn vip_fields(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
max: usize,
|
||||
) -> Result<Vec<crate::vip::VipEntry>> {
|
||||
self.with_state(move |state| state.vip_fields(network, max))
|
||||
.await
|
||||
}
|
||||
/// Stores a complete merged service snapshot and optional own counter.
|
||||
pub async fn put_vip_fields(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
entries: Vec<crate::vip::VipEntry>,
|
||||
own: Option<([u8; 32], u64)>,
|
||||
) -> Result<()> {
|
||||
self.with_state(move |state| state.put_vip_fields(network, &entries, own))
|
||||
.await
|
||||
}
|
||||
/// Loads verified durable cleanup proofs off the executor.
|
||||
pub async fn retirements(
|
||||
&self,
|
||||
|
||||
@@ -21,7 +21,7 @@ use crate::identity::{DeviceIdentity, NetworkId, NetworkName, NetworkSecret};
|
||||
use crate::state::{MAX_RETIREMENT_LEN, RecordBody, SignedRecord, SignedRetirement};
|
||||
|
||||
/// Schema version written by this build.
|
||||
pub const SCHEMA_VERSION: i64 = 9;
|
||||
pub const SCHEMA_VERSION: i64 = 10;
|
||||
|
||||
/// Key of the stored hostname setting.
|
||||
const SETTING_HOSTNAME: &str = "hostname";
|
||||
@@ -263,6 +263,12 @@ impl StateStore {
|
||||
)
|
||||
.map_err(|err| self.corrupt(format!("cannot migrate schema to 9: {err}")))?;
|
||||
}
|
||||
if found < 10 {
|
||||
self.conn.execute_batch("BEGIN; CREATE TABLE IF NOT EXISTS vip_fields (
|
||||
network_id BLOB NOT NULL, vip TEXT NOT NULL, field TEXT NOT NULL, data BLOB NOT NULL,
|
||||
PRIMARY KEY(network_id, vip, field)); PRAGMA user_version = 10; COMMIT;")
|
||||
.map_err(|err| self.corrupt(format!("cannot migrate VIP state: {err}")))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -276,6 +282,7 @@ impl StateStore {
|
||||
"signed_records",
|
||||
"peer_hostnames",
|
||||
"retirements",
|
||||
"vip_fields",
|
||||
] {
|
||||
let present: Option<String> = self
|
||||
.conn
|
||||
@@ -894,8 +901,77 @@ impl StateStore {
|
||||
.map_err(|err| Error::Storage(format!("cannot commit retirement: {err}")))
|
||||
}
|
||||
|
||||
/// Reads bounded mandatory VIP state; corrupt records fail activation.
|
||||
pub fn vip_fields(&self, network: NetworkId, max: usize) -> Result<Vec<crate::vip::VipEntry>> {
|
||||
let mut stmt = self.conn.prepare("SELECT CASE WHEN length(data) <= 1024 THEN data ELSE NULL END FROM vip_fields WHERE network_id=?1 ORDER BY vip, field")
|
||||
.map_err(|e| self.corrupt(e.to_string()))?;
|
||||
let mut rows = stmt
|
||||
.query(params![network.as_bytes().as_slice()])
|
||||
.map_err(|e| self.corrupt(e.to_string()))?;
|
||||
let mut entries = Vec::new();
|
||||
while let Some(row) = rows.next().map_err(|e| self.corrupt(e.to_string()))? {
|
||||
if entries.len() >= max {
|
||||
return Err(self.corrupt("stored VIP state exceeds configured limits"));
|
||||
}
|
||||
let bytes: Option<Vec<u8>> = row.get(0).map_err(|e| self.corrupt(e.to_string()))?;
|
||||
let bytes = bytes.ok_or_else(|| self.corrupt("oversized VIP field"))?;
|
||||
let entry: crate::vip::VipEntry =
|
||||
postcard::from_bytes(&bytes).map_err(|e| self.corrupt(e.to_string()))?;
|
||||
entry
|
||||
.verify(network)
|
||||
.map_err(|e| self.corrupt(e.to_string()))?;
|
||||
entries.push(entry);
|
||||
}
|
||||
Ok(entries)
|
||||
}
|
||||
|
||||
/// Commits a validated VIP snapshot and optional author counter atomically.
|
||||
pub fn put_vip_fields(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
entries: &[crate::vip::VipEntry],
|
||||
own: Option<([u8; 32], u64)>,
|
||||
) -> Result<()> {
|
||||
let tx = self
|
||||
.conn
|
||||
.unchecked_transaction()
|
||||
.map_err(|e| Error::Storage(e.to_string()))?;
|
||||
if let Some(first) = entries.first() {
|
||||
if entries.iter().any(|e| e.vip != first.vip) {
|
||||
return Err(Error::Storage(
|
||||
"VIP snapshot mixes service identities".into(),
|
||||
));
|
||||
}
|
||||
tx.execute(
|
||||
"DELETE FROM vip_fields WHERE network_id=?1 AND vip=?2",
|
||||
params![network.as_bytes().as_slice(), first.vip],
|
||||
)
|
||||
.map_err(|e| Error::Storage(e.to_string()))?;
|
||||
}
|
||||
for e in entries {
|
||||
e.verify(network)?;
|
||||
let bytes = postcard::to_stdvec(e).map_err(|e| Error::Storage(e.to_string()))?;
|
||||
if bytes.len() > 1024 {
|
||||
return Err(Error::Storage("oversized VIP field".into()));
|
||||
}
|
||||
tx.execute("INSERT INTO vip_fields(network_id,vip,field,data) VALUES(?1,?2,?3,?4) ON CONFLICT(network_id,vip,field) DO UPDATE SET data=excluded.data",
|
||||
params![network.as_bytes().as_slice(),e.vip,e.key,bytes]).map_err(|e|Error::Storage(e.to_string()))?;
|
||||
}
|
||||
if let Some((author, version)) = own {
|
||||
tx.execute("INSERT INTO own_record_version(network_id,author,version) VALUES(?1,?2,?3) ON CONFLICT(network_id,author) DO UPDATE SET version=max(version,excluded.version)",
|
||||
params![network.as_bytes().as_slice(),author.as_slice(),version as i64]).map_err(|e|Error::Storage(e.to_string()))?;
|
||||
}
|
||||
tx.commit().map_err(|e| Error::Storage(e.to_string()))
|
||||
}
|
||||
|
||||
/// Drops a forgotten network's local cleanup proofs.
|
||||
pub fn forget_retirements(&self, network: NetworkId) -> Result<()> {
|
||||
self.conn
|
||||
.execute(
|
||||
"DELETE FROM vip_fields WHERE network_id = ?1",
|
||||
params![network.as_bytes().as_slice()],
|
||||
)
|
||||
.map_err(|e| Error::Storage(e.to_string()))?;
|
||||
self.conn
|
||||
.execute(
|
||||
"DELETE FROM retirements WHERE network_id = ?1",
|
||||
@@ -1333,3 +1409,59 @@ mod broadcast_storage_tests {
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod vip_tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
#[test]
|
||||
fn vip_migration_and_counter_failure_preserve_identity_and_rollback_fields() {
|
||||
let dir = tempfile::TempDir::new().unwrap();
|
||||
let path = dir.path().join("state.sqlite");
|
||||
let store = StateStore::open(&path).unwrap();
|
||||
let identity = store.load_or_create_device_identity().unwrap();
|
||||
store
|
||||
.conn
|
||||
.execute_batch("DROP TABLE vip_fields; PRAGMA user_version=9;")
|
||||
.unwrap();
|
||||
drop(store);
|
||||
let store = StateStore::open(&path).unwrap();
|
||||
assert_eq!(
|
||||
store.device_identity().unwrap().unwrap().endpoint_id(),
|
||||
identity.endpoint_id()
|
||||
);
|
||||
let net = NetworkId::from_bytes([7; 32]);
|
||||
let key = identity.signing_key();
|
||||
let config = crate::vip::VipConfig::new("api");
|
||||
let fields = crate::vip::VipSet::default()
|
||||
.edit(net, &key, 7, &config, None, &crate::Limits::default())
|
||||
.unwrap();
|
||||
store.conn.execute_batch("CREATE TRIGGER fail_vip_counter BEFORE INSERT ON own_record_version BEGIN SELECT RAISE(ABORT,'counter unavailable'); END;").unwrap();
|
||||
assert!(
|
||||
store
|
||||
.put_vip_fields(net, &fields, Some((*key.public().as_bytes(), 7)))
|
||||
.is_err()
|
||||
);
|
||||
assert!(store.vip_fields(net, 192).unwrap().is_empty());
|
||||
store
|
||||
.conn
|
||||
.execute_batch("DROP TRIGGER fail_vip_counter;")
|
||||
.unwrap();
|
||||
store
|
||||
.put_vip_fields(net, &fields, Some((*key.public().as_bytes(), 7)))
|
||||
.unwrap();
|
||||
assert_eq!(store.vip_fields(net, 192).unwrap(), fields);
|
||||
assert_eq!(store.own_record_version(net, key.public()).unwrap(), 7);
|
||||
drop(store);
|
||||
let reopened = StateStore::open(path).unwrap();
|
||||
assert_eq!(reopened.vip_fields(net, 192).unwrap(), fields);
|
||||
reopened
|
||||
.conn
|
||||
.execute("UPDATE vip_fields SET data=?1", params![vec![0xffu8; 1025]])
|
||||
.unwrap();
|
||||
assert!(matches!(
|
||||
reopened.vip_fields(net, 192),
|
||||
Err(Error::StateCorrupted { .. })
|
||||
));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,239 @@
|
||||
//! Named, shared virtual services. Signed field registers converge without an
|
||||
//! owner; routing, health and connection state remain local to each client.
|
||||
use crate::Limits;
|
||||
use crate::error::{Error, Result};
|
||||
use iroh::EndpointId;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::BTreeSet;
|
||||
use std::net::Ipv4Addr;
|
||||
pub(crate) mod packet;
|
||||
pub(crate) mod router;
|
||||
pub use crate::state::vip::{VipEntry, VipSet};
|
||||
pub(crate) use router::VipRouter;
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
|
||||
/// IP transport carried by a listener.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum Protocol {
|
||||
/// TCP.
|
||||
Tcp,
|
||||
/// UDP.
|
||||
Udp,
|
||||
}
|
||||
impl std::fmt::Display for Protocol {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str(match self {
|
||||
Self::Tcp => "tcp",
|
||||
Self::Udp => "udp",
|
||||
})
|
||||
}
|
||||
}
|
||||
impl std::str::FromStr for Protocol {
|
||||
type Err = String;
|
||||
fn from_str(s: &str) -> std::result::Result<Self, String> {
|
||||
match s {
|
||||
"tcp" => Ok(Self::Tcp),
|
||||
"udp" => Ok(Self::Udp),
|
||||
_ => Err("protocol must be tcp or udp".into()),
|
||||
}
|
||||
}
|
||||
}
|
||||
/// A node identity and its application port.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct Backend {
|
||||
/// Full device identity, resolved when editing.
|
||||
pub peer: String,
|
||||
/// Destination port.
|
||||
pub port: u16,
|
||||
}
|
||||
/// One independently routed port of a VIP.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct Listener {
|
||||
/// Stable listener generation; prevents removed backend fields resurfacing on re-creation.
|
||||
pub id: String,
|
||||
/// TCP or UDP.
|
||||
pub protocol: Protocol,
|
||||
/// Client-facing port.
|
||||
pub port: u16,
|
||||
/// Enable active TCP connect checks; UDP uses path checks and ICMP errors.
|
||||
pub port_check: bool,
|
||||
/// Period between checks in milliseconds.
|
||||
pub check_interval_ms: u32,
|
||||
/// Deadline of a port check in milliseconds.
|
||||
pub check_timeout_ms: u32,
|
||||
/// Consecutive failed checks before exclusion.
|
||||
pub failure_threshold: u8,
|
||||
/// Consecutive successes before recovery.
|
||||
pub recovery_threshold: u8,
|
||||
/// Idle expiry for this listener's flows, in seconds.
|
||||
pub idle_timeout_secs: u32,
|
||||
/// Allowed destination nodes and ports.
|
||||
pub backends: Vec<Backend>,
|
||||
}
|
||||
impl Listener {
|
||||
/// Creates a listener with automatic recovery and bounded checks.
|
||||
pub fn new(protocol: Protocol, port: u16) -> Self {
|
||||
Self {
|
||||
id: hex::encode(rand::random::<[u8; 16]>()),
|
||||
protocol,
|
||||
port,
|
||||
port_check: true,
|
||||
check_interval_ms: 3000,
|
||||
check_timeout_ms: 2000,
|
||||
failure_threshold: 2,
|
||||
recovery_threshold: 1,
|
||||
idle_timeout_secs: if protocol == Protocol::Tcp { 7200 } else { 120 },
|
||||
backends: vec![],
|
||||
}
|
||||
}
|
||||
}
|
||||
/// Entire editable service; its address is an internal derived reservation.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct VipConfig {
|
||||
/// Stable random 128-bit id in hexadecimal.
|
||||
pub id: String,
|
||||
/// One DNS label, unique within the network.
|
||||
pub name: String,
|
||||
/// Independent listeners.
|
||||
pub listeners: Vec<Listener>,
|
||||
}
|
||||
impl VipConfig {
|
||||
/// Creates an empty named service with a fresh stable identity.
|
||||
pub fn new(name: impl Into<String>) -> Self {
|
||||
Self {
|
||||
id: hex::encode(rand::random::<[u8; 16]>()),
|
||||
name: name.into(),
|
||||
listeners: vec![],
|
||||
}
|
||||
}
|
||||
/// Validates a complete draft before any state is changed.
|
||||
pub fn validate(&self, limits: &Limits) -> Result<()> {
|
||||
if self.id.len() != 32 || hex::decode(&self.id).is_err() {
|
||||
return Err(invalid("invalid VIP id"));
|
||||
}
|
||||
if self.name.is_empty()
|
||||
|| self.name.len() > 63
|
||||
|| self.name.starts_with('-')
|
||||
|| self.name.ends_with('-')
|
||||
|| !self
|
||||
.name
|
||||
.bytes()
|
||||
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-')
|
||||
{
|
||||
return Err(invalid(
|
||||
"VIP name must be a lowercase DNS label (letters, digits and hyphens)",
|
||||
));
|
||||
}
|
||||
if self.listeners.len() > limits.max_vip_listeners {
|
||||
return Err(invalid("too many VIP listeners"));
|
||||
}
|
||||
let mut seen = BTreeSet::new();
|
||||
for l in &self.listeners {
|
||||
if l.id.len() != 32 || hex::decode(&l.id).is_err() {
|
||||
return Err(invalid("invalid listener identity"));
|
||||
}
|
||||
if l.port == 0 || !seen.insert((l.protocol, l.port)) {
|
||||
return Err(invalid(
|
||||
"listener ports must be nonzero and unique per protocol",
|
||||
));
|
||||
}
|
||||
if !(250..=3_600_000).contains(&l.check_interval_ms)
|
||||
|| !(100..=60_000).contains(&l.check_timeout_ms)
|
||||
|| l.failure_threshold == 0
|
||||
|| l.recovery_threshold == 0
|
||||
|| !(1..=604800).contains(&l.idle_timeout_secs)
|
||||
{
|
||||
return Err(invalid(
|
||||
"invalid check interval, timeout, thresholds or flow idle timeout",
|
||||
));
|
||||
}
|
||||
if l.backends.len() > limits.max_vip_backends {
|
||||
return Err(invalid("too many listener backends"));
|
||||
}
|
||||
let mut backends = BTreeSet::new();
|
||||
for b in &l.backends {
|
||||
if b.port == 0 || b.peer.parse::<EndpointId>().is_err() || !backends.insert(b) {
|
||||
return Err(invalid(
|
||||
"backend must have a unique node identity and a nonzero port",
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
if crate::state::vip::flatten(self)?.len() > limits.max_vip_fields {
|
||||
return Err(invalid("VIP configuration exceeds signed field capacity"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
pub(crate) fn invalid(reason: impl Into<String>) -> Error {
|
||||
Error::Rejected {
|
||||
reason: reason.into(),
|
||||
}
|
||||
}
|
||||
/// Cumulative IP traffic, excluding probes. Reset on agent restart.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct Counters {
|
||||
/// Outbound packets.
|
||||
pub tx_packets: u64,
|
||||
/// Outbound bytes.
|
||||
pub tx_bytes: u64,
|
||||
/// Inbound packets.
|
||||
pub rx_packets: u64,
|
||||
/// Inbound bytes.
|
||||
pub rx_bytes: u64,
|
||||
/// Created flow bindings.
|
||||
pub flows: u64,
|
||||
/// UDP failovers.
|
||||
pub failovers: u64,
|
||||
/// Packets rejected or lacking an eligible backend.
|
||||
pub dropped: u64,
|
||||
}
|
||||
/// One backend, as seen by this client.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct BackendStatus {
|
||||
/// Configured endpoint.
|
||||
pub backend: Backend,
|
||||
/// Eligibility and reason.
|
||||
pub state: String,
|
||||
/// Smoothed full data-plane RTT, microseconds; unknown stays absent.
|
||||
pub rtt_us: Option<u64>,
|
||||
/// Last successful path measurement age.
|
||||
pub measurement_age_ms: Option<u64>,
|
||||
/// Current bindings.
|
||||
pub active_flows: u32,
|
||||
/// Local traffic counters.
|
||||
pub counters: Counters,
|
||||
}
|
||||
/// Runtime view of one listener.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct ListenerStatus {
|
||||
/// Protocol.
|
||||
pub protocol: Protocol,
|
||||
/// Frontend port.
|
||||
pub port: u16,
|
||||
/// Best backend for a new flow, if measured and eligible.
|
||||
pub selected: Option<Backend>,
|
||||
/// Configured and draining endpoints.
|
||||
pub backends: Vec<BackendStatus>,
|
||||
/// Local counters.
|
||||
pub counters: Counters,
|
||||
}
|
||||
/// Configuration and local diagnostics returned through the same API to CLI and GUI.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct VipStatus {
|
||||
/// Persisted configuration.
|
||||
pub config: VipConfig,
|
||||
/// Opaque revision used when saving a draft.
|
||||
pub revision: String,
|
||||
/// Internal service address.
|
||||
pub address: Option<Ipv4Addr>,
|
||||
/// Conflict or allocation issue, if any.
|
||||
pub error: Option<String>,
|
||||
/// Local listener status.
|
||||
pub listeners: Vec<ListenerStatus>,
|
||||
}
|
||||
@@ -0,0 +1,566 @@
|
||||
//! Bounded IPv4 parsing, NAT checksums and fragment reassembly at local ingress.
|
||||
use super::Protocol;
|
||||
use bytes::Bytes;
|
||||
use std::collections::HashMap;
|
||||
use std::net::Ipv4Addr;
|
||||
use std::time::{Duration, Instant};
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub(crate) struct Tuple {
|
||||
pub src: Ipv4Addr,
|
||||
pub dst: Ipv4Addr,
|
||||
pub sport: u16,
|
||||
pub dport: u16,
|
||||
pub protocol: Protocol,
|
||||
}
|
||||
impl Tuple {
|
||||
pub fn reverse(self) -> Self {
|
||||
Self {
|
||||
src: self.dst,
|
||||
dst: self.src,
|
||||
sport: self.dport,
|
||||
dport: self.sport,
|
||||
protocol: self.protocol,
|
||||
}
|
||||
}
|
||||
}
|
||||
pub(crate) fn header(p: &[u8]) -> Option<usize> {
|
||||
if p.len() < 20 || p[0] >> 4 != 4 {
|
||||
return None;
|
||||
}
|
||||
let n = usize::from(p[0] & 15) * 4;
|
||||
let total = usize::from(u16::from_be_bytes([p[2], p[3]]));
|
||||
(n >= 20 && n <= p.len() && total == p.len() && checksum(&p[..n]) == 0).then_some(n)
|
||||
}
|
||||
pub(crate) fn addresses(p: &[u8]) -> Option<(Ipv4Addr, Ipv4Addr)> {
|
||||
header(p)?;
|
||||
Some((
|
||||
Ipv4Addr::new(p[12], p[13], p[14], p[15]),
|
||||
Ipv4Addr::new(p[16], p[17], p[18], p[19]),
|
||||
))
|
||||
}
|
||||
pub(crate) fn tuple(p: &[u8]) -> Option<Tuple> {
|
||||
let n = header(p)?;
|
||||
if u16::from_be_bytes([p[6], p[7]]) & 0x3fff != 0 {
|
||||
return None;
|
||||
}
|
||||
let protocol = match p[9] {
|
||||
6 => Protocol::Tcp,
|
||||
17 => Protocol::Udp,
|
||||
_ => return None,
|
||||
};
|
||||
let min = if protocol == Protocol::Tcp { 20 } else { 8 };
|
||||
if p.len() < n + min {
|
||||
return None;
|
||||
}
|
||||
if protocol == Protocol::Tcp {
|
||||
let h = usize::from(p[n + 12] >> 4) * 4;
|
||||
if h < 20 || n + h > p.len() {
|
||||
return None;
|
||||
}
|
||||
} else if usize::from(u16::from_be_bytes([p[n + 4], p[n + 5]])) != p.len() - n {
|
||||
return None;
|
||||
}
|
||||
let (src, dst) = addresses(p)?;
|
||||
Some(Tuple {
|
||||
src,
|
||||
dst,
|
||||
sport: u16::from_be_bytes([p[n], p[n + 1]]),
|
||||
dport: u16::from_be_bytes([p[n + 2], p[n + 3]]),
|
||||
protocol,
|
||||
})
|
||||
}
|
||||
pub(crate) fn rewrite(p: &[u8], t: Tuple) -> Option<Bytes> {
|
||||
let n = header(p)?;
|
||||
if p.len() < n + if t.protocol == Protocol::Tcp { 20 } else { 8 } {
|
||||
return None;
|
||||
}
|
||||
let mut out = p.to_vec();
|
||||
out[12..16].copy_from_slice(&t.src.octets());
|
||||
out[16..20].copy_from_slice(&t.dst.octets());
|
||||
out[n..n + 2].copy_from_slice(&t.sport.to_be_bytes());
|
||||
out[n + 2..n + 4].copy_from_slice(&t.dport.to_be_bytes());
|
||||
finish(&mut out, n, t.protocol);
|
||||
Some(Bytes::from(out))
|
||||
}
|
||||
fn finish(p: &mut [u8], n: usize, protocol: Protocol) {
|
||||
p[10..12].fill(0);
|
||||
let c = checksum(&p[..n]);
|
||||
p[10..12].copy_from_slice(&c.to_be_bytes());
|
||||
let offset = n + if protocol == Protocol::Tcp { 16 } else { 6 };
|
||||
p[offset..offset + 2].fill(0);
|
||||
let mut sum = 0u32;
|
||||
for pair in p[12..20].as_chunks::<2>().0 {
|
||||
sum += u32::from(u16::from_be_bytes([pair[0], pair[1]]));
|
||||
}
|
||||
sum += u32::from(p[9]) + (p.len() - n) as u32;
|
||||
let c = checksum_seed(&p[n..], sum);
|
||||
let c = if c == 0 && protocol == Protocol::Udp {
|
||||
0xffff
|
||||
} else {
|
||||
c
|
||||
};
|
||||
p[offset..offset + 2].copy_from_slice(&c.to_be_bytes());
|
||||
}
|
||||
pub(crate) fn checksum(p: &[u8]) -> u16 {
|
||||
checksum_seed(p, 0)
|
||||
}
|
||||
fn checksum_seed(p: &[u8], mut sum: u32) -> u16 {
|
||||
for pair in p.chunks(2) {
|
||||
sum += u32::from(u16::from_be_bytes([pair[0], *pair.get(1).unwrap_or(&0)]));
|
||||
}
|
||||
while sum >> 16 != 0 {
|
||||
sum = (sum & 0xffff) + (sum >> 16);
|
||||
}
|
||||
!(sum as u16)
|
||||
}
|
||||
pub(crate) fn tcp_syn(p: &[u8]) -> bool {
|
||||
header(p).is_some_and(|n| p[9] == 6 && p.get(n + 13).is_some_and(|f| f & 0x12 == 2))
|
||||
}
|
||||
pub(crate) fn syn_sequence(p: &[u8]) -> Option<u32> {
|
||||
if !tcp_syn(p) {
|
||||
return None;
|
||||
}
|
||||
let n = header(p)?;
|
||||
let bytes = p.get(n + 4..n + 8)?;
|
||||
Some(u32::from_be_bytes(bytes.try_into().ok()?))
|
||||
}
|
||||
pub(crate) fn empty_syn(p: &[u8]) -> bool {
|
||||
header(p).is_some_and(|n| {
|
||||
tcp_syn(p)
|
||||
&& p.get(n + 12)
|
||||
.is_some_and(|h| n + usize::from(h >> 4) * 4 == p.len())
|
||||
})
|
||||
}
|
||||
pub(crate) fn syn_ack(p: &[u8]) -> bool {
|
||||
header(p).is_some_and(|n| p[9] == 6 && p.get(n + 13).is_some_and(|f| f & 0x12 == 0x12))
|
||||
}
|
||||
pub(crate) fn tcp_reset(p: &[u8]) -> bool {
|
||||
header(p).is_some_and(|n| p[9] == 6 && p.get(n + 13).is_some_and(|f| f & 4 != 0))
|
||||
}
|
||||
/// Rewrites an ICMP error and its quoted original header after NAT lookup.
|
||||
pub(crate) fn icmp_quote(p: &[u8]) -> Option<Tuple> {
|
||||
let n = header(p)?;
|
||||
if p[9] != 1 || p.len() < n + 8 + 28 || ![3, 11, 12].contains(&p[n]) {
|
||||
return None;
|
||||
}
|
||||
let q = &p[n + 8..];
|
||||
let h = usize::from(q[0] & 15) * 4;
|
||||
if q[0] >> 4 != 4 || h < 20 || q.len() < h + 8 {
|
||||
return None;
|
||||
}
|
||||
let protocol = match q[9] {
|
||||
6 => Protocol::Tcp,
|
||||
17 => Protocol::Udp,
|
||||
_ => return None,
|
||||
};
|
||||
Some(Tuple {
|
||||
src: Ipv4Addr::new(q[12], q[13], q[14], q[15]),
|
||||
dst: Ipv4Addr::new(q[16], q[17], q[18], q[19]),
|
||||
sport: u16::from_be_bytes([q[h], q[h + 1]]),
|
||||
dport: u16::from_be_bytes([q[h + 2], q[h + 3]]),
|
||||
protocol,
|
||||
})
|
||||
}
|
||||
pub(crate) fn rewrite_icmp(p: &[u8], original: Tuple) -> Option<Bytes> {
|
||||
let n = header(p)?;
|
||||
icmp_quote(p)?;
|
||||
let mut out = p.to_vec();
|
||||
out[12..16].copy_from_slice(&original.dst.octets());
|
||||
out[16..20].copy_from_slice(&original.src.octets());
|
||||
let q = &mut out[n + 8..];
|
||||
let h = usize::from(q[0] & 15) * 4;
|
||||
// Incrementally update the quoted transport checksum when it is present.
|
||||
let off = h + if original.protocol == Protocol::Tcp {
|
||||
16
|
||||
} else {
|
||||
6
|
||||
};
|
||||
if q.len() >= off + 2 {
|
||||
let old = u16::from_be_bytes([q[off], q[off + 1]]);
|
||||
if old != 0 || original.protocol == Protocol::Tcp {
|
||||
let mut sum = u32::from(!old);
|
||||
let mut replacement = Vec::from(original.src.octets());
|
||||
replacement.extend(original.dst.octets());
|
||||
replacement.extend(original.sport.to_be_bytes());
|
||||
replacement.extend(original.dport.to_be_bytes());
|
||||
let previous = [&q[12..20], &q[h..h + 4]].concat();
|
||||
for (a, b) in previous
|
||||
.as_chunks::<2>()
|
||||
.0
|
||||
.iter()
|
||||
.zip(replacement.as_chunks::<2>().0)
|
||||
{
|
||||
sum += u32::from(!u16::from_be_bytes([a[0], a[1]]))
|
||||
+ u32::from(u16::from_be_bytes([b[0], b[1]]));
|
||||
}
|
||||
while sum >> 16 != 0 {
|
||||
sum = (sum & 0xffff) + (sum >> 16);
|
||||
}
|
||||
let c = !(sum as u16);
|
||||
q[off..off + 2].copy_from_slice(
|
||||
&(if c == 0 && original.protocol == Protocol::Udp {
|
||||
0xffff
|
||||
} else {
|
||||
c
|
||||
})
|
||||
.to_be_bytes(),
|
||||
);
|
||||
}
|
||||
}
|
||||
q[12..16].copy_from_slice(&original.src.octets());
|
||||
q[16..20].copy_from_slice(&original.dst.octets());
|
||||
q[h..h + 2].copy_from_slice(&original.sport.to_be_bytes());
|
||||
q[h + 2..h + 4].copy_from_slice(&original.dport.to_be_bytes());
|
||||
q[10..12].fill(0);
|
||||
let c = checksum(&q[..h]);
|
||||
q[10..12].copy_from_slice(&c.to_be_bytes());
|
||||
out[n + 2..n + 4].fill(0);
|
||||
let c = checksum(&out[n..]);
|
||||
out[n + 2..n + 4].copy_from_slice(&c.to_be_bytes());
|
||||
out[10..12].fill(0);
|
||||
let c = checksum(&out[..n]);
|
||||
out[10..12].copy_from_slice(&c.to_be_bytes());
|
||||
Some(Bytes::from(out))
|
||||
}
|
||||
pub(crate) fn reject(p: &[u8]) -> Option<Bytes> {
|
||||
let t = tuple(p)?;
|
||||
let n = header(p)?;
|
||||
if t.protocol == Protocol::Tcp {
|
||||
if !tcp_syn(p) {
|
||||
return None;
|
||||
}
|
||||
let mut out = vec![0; 40];
|
||||
out[0] = 0x45;
|
||||
out[2..4].copy_from_slice(&40u16.to_be_bytes());
|
||||
out[8] = 64;
|
||||
out[9] = 6;
|
||||
out[12..16].copy_from_slice(&t.dst.octets());
|
||||
out[16..20].copy_from_slice(&t.src.octets());
|
||||
out[20..22].copy_from_slice(&t.dport.to_be_bytes());
|
||||
out[22..24].copy_from_slice(&t.sport.to_be_bytes());
|
||||
let seq = u32::from_be_bytes(p[n + 4..n + 8].try_into().ok()?);
|
||||
out[28..32].copy_from_slice(&seq.wrapping_add(1).to_be_bytes());
|
||||
out[32] = 0x50;
|
||||
out[33] = 0x14;
|
||||
finish(&mut out, 20, Protocol::Tcp);
|
||||
Some(Bytes::from(out))
|
||||
} else {
|
||||
let q = p.len().min(n + 8);
|
||||
let mut out = vec![0; 28 + q];
|
||||
out[0] = 0x45;
|
||||
let len = out.len() as u16;
|
||||
out[2..4].copy_from_slice(&len.to_be_bytes());
|
||||
out[8] = 64;
|
||||
out[9] = 1;
|
||||
out[12..16].copy_from_slice(&t.dst.octets());
|
||||
out[16..20].copy_from_slice(&t.src.octets());
|
||||
out[20] = 3;
|
||||
out[21] = 3;
|
||||
out[28..].copy_from_slice(&p[..q]);
|
||||
let c = checksum(&out[20..]);
|
||||
out[22..24].copy_from_slice(&c.to_be_bytes());
|
||||
let c = checksum(&out[..20]);
|
||||
out[10..12].copy_from_slice(&c.to_be_bytes());
|
||||
Some(Bytes::from(out))
|
||||
}
|
||||
}
|
||||
#[derive(Debug)]
|
||||
struct Datagram {
|
||||
at: Instant,
|
||||
header: Option<Vec<u8>>,
|
||||
parts: std::collections::BTreeMap<usize, Vec<u8>>,
|
||||
end: Option<usize>,
|
||||
bytes: usize,
|
||||
}
|
||||
type FragmentKey = (Option<[u8; 32]>, Ipv4Addr, Ipv4Addr, u16, u8);
|
||||
#[derive(Debug, Default)]
|
||||
pub(crate) struct Reassembly {
|
||||
pending: HashMap<FragmentKey, Datagram>,
|
||||
}
|
||||
impl Reassembly {
|
||||
pub fn expire(&mut self, now: Instant) {
|
||||
self.pending
|
||||
.retain(|_, d| now.duration_since(d.at) < Duration::from_secs(5));
|
||||
}
|
||||
pub fn accept(&mut self, p: Bytes, max: usize, scope: Option<[u8; 32]>) -> Option<Bytes> {
|
||||
let n = header(&p)?;
|
||||
let flags = u16::from_be_bytes([p[6], p[7]]);
|
||||
if flags & 0x3fff == 0 {
|
||||
return Some(p);
|
||||
}
|
||||
let now = Instant::now();
|
||||
self.pending
|
||||
.retain(|_, d| now.duration_since(d.at) < Duration::from_secs(5));
|
||||
let (src, dst) = addresses(&p)?;
|
||||
let key = (scope, src, dst, u16::from_be_bytes([p[4], p[5]]), p[9]);
|
||||
let start = usize::from(flags & 0x1fff) * 8;
|
||||
let payload = &p[n..];
|
||||
let end = start.checked_add(payload.len())?;
|
||||
if payload.is_empty()
|
||||
|| end > 65515
|
||||
|| flags & 0x2000 != 0 && !payload.len().is_multiple_of(8)
|
||||
{
|
||||
self.pending.remove(&key);
|
||||
return None;
|
||||
}
|
||||
if !self.pending.contains_key(&key) && self.pending.len() >= max {
|
||||
return None;
|
||||
}
|
||||
let d = self.pending.entry(key).or_insert_with(|| Datagram {
|
||||
at: now,
|
||||
header: None,
|
||||
parts: Default::default(),
|
||||
end: None,
|
||||
bytes: 0,
|
||||
});
|
||||
if d.parts
|
||||
.get(&start)
|
||||
.is_some_and(|bytes| bytes.as_slice() == payload)
|
||||
{
|
||||
return None;
|
||||
}
|
||||
if d.parts.iter().any(|(s, b)| *s < end && start < s + b.len()) {
|
||||
self.pending.remove(&key);
|
||||
return None;
|
||||
}
|
||||
if flags & 0x2000 == 0 {
|
||||
if d.end.is_some_and(|e| e != end) {
|
||||
self.pending.remove(&key);
|
||||
return None;
|
||||
}
|
||||
d.end = Some(end);
|
||||
}
|
||||
if d.end.is_some_and(|e| end > e) {
|
||||
self.pending.remove(&key);
|
||||
return None;
|
||||
}
|
||||
if start == 0 {
|
||||
d.header = Some(p[..n].to_vec());
|
||||
}
|
||||
d.bytes += payload.len();
|
||||
d.parts.insert(start, payload.to_vec());
|
||||
let complete = d.end.is_some_and(|e| e == d.bytes) && d.header.is_some();
|
||||
if !complete {
|
||||
return None;
|
||||
}
|
||||
let d = self.pending.remove(&key)?;
|
||||
let mut out = d.header?;
|
||||
for part in d.parts.values() {
|
||||
out.extend(part);
|
||||
}
|
||||
if out.len() > 65535 {
|
||||
return None;
|
||||
}
|
||||
let len = out.len() as u16;
|
||||
out[2..4].copy_from_slice(&len.to_be_bytes());
|
||||
out[6..8].fill(0);
|
||||
out[10..12].fill(0);
|
||||
let h = usize::from(out[0] & 15) * 4;
|
||||
let c = checksum(&out[..h]);
|
||||
out[10..12].copy_from_slice(&c.to_be_bytes());
|
||||
Some(Bytes::from(out))
|
||||
}
|
||||
}
|
||||
|
||||
/// Re-fragments an assembled NAT datagram for the unchanged overlay MTU.
|
||||
pub(crate) fn fragments(packet: Bytes, mtu: u32) -> Vec<Bytes> {
|
||||
if packet.len() <= mtu as usize {
|
||||
return vec![packet];
|
||||
}
|
||||
let Some(n) = header(&packet) else {
|
||||
return vec![];
|
||||
};
|
||||
if u16::from_be_bytes([packet[6], packet[7]]) & 0x4000 != 0 {
|
||||
return vec![];
|
||||
}
|
||||
let size = (mtu as usize).saturating_sub(n) / 8 * 8;
|
||||
if size == 0 {
|
||||
return vec![];
|
||||
}
|
||||
let mut copied = Vec::new();
|
||||
let mut offset = 20;
|
||||
while offset < n {
|
||||
let kind = packet[offset];
|
||||
if kind == 0 {
|
||||
break;
|
||||
}
|
||||
if kind == 1 {
|
||||
offset += 1;
|
||||
continue;
|
||||
}
|
||||
let Some(&len) = packet.get(offset + 1) else {
|
||||
return vec![];
|
||||
};
|
||||
let len = usize::from(len);
|
||||
if len < 2 || offset + len > n {
|
||||
return vec![];
|
||||
}
|
||||
if kind & 0x80 != 0 {
|
||||
copied.extend_from_slice(&packet[offset..offset + len]);
|
||||
}
|
||||
offset += len;
|
||||
}
|
||||
while !copied.len().is_multiple_of(4) {
|
||||
copied.push(0);
|
||||
}
|
||||
let mut result = Vec::new();
|
||||
let payload = &packet[n..];
|
||||
for (start, part) in payload.chunks(size).enumerate() {
|
||||
let offset = start * size;
|
||||
let mut out = if offset == 0 {
|
||||
packet[..n].to_vec()
|
||||
} else {
|
||||
let mut h = packet[..20].to_vec();
|
||||
h.extend_from_slice(&copied);
|
||||
h[0] = 0x40 | (h.len() / 4) as u8;
|
||||
h
|
||||
};
|
||||
let header_len = out.len();
|
||||
out.extend(part);
|
||||
let len = out.len() as u16;
|
||||
out[2..4].copy_from_slice(&len.to_be_bytes());
|
||||
let flags = (offset / 8) as u16
|
||||
| if offset + part.len() < payload.len() {
|
||||
0x2000
|
||||
} else {
|
||||
0
|
||||
};
|
||||
out[6..8].copy_from_slice(&flags.to_be_bytes());
|
||||
out[10..12].fill(0);
|
||||
let c = checksum(&out[..header_len]);
|
||||
out[10..12].copy_from_slice(&c.to_be_bytes());
|
||||
result.push(Bytes::from(out));
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
fn datagram() -> Bytes {
|
||||
let mut p = vec![0; 3028];
|
||||
p[0] = 0x45;
|
||||
p[2..4].copy_from_slice(&3028u16.to_be_bytes());
|
||||
p[8] = 64;
|
||||
p[9] = 17;
|
||||
p[12..16].copy_from_slice(&[10, 0, 0, 1]);
|
||||
p[16..20].copy_from_slice(&[10, 0, 0, 2]);
|
||||
p[20..22].copy_from_slice(&1000u16.to_be_bytes());
|
||||
p[22..24].copy_from_slice(&2000u16.to_be_bytes());
|
||||
p[24..26].copy_from_slice(&3008u16.to_be_bytes());
|
||||
finish(&mut p, 20, Protocol::Udp);
|
||||
Bytes::from(p)
|
||||
}
|
||||
#[test]
|
||||
fn fragmented_udp_reassembles_out_of_order_and_nat_checksums_are_valid() {
|
||||
let p = datagram();
|
||||
let parts = fragments(p.clone(), 1280);
|
||||
assert_eq!(parts.len(), 3);
|
||||
let mut reassembly = Reassembly::default();
|
||||
let mut result = None;
|
||||
for part in parts.into_iter().rev() {
|
||||
if let Some(p) = reassembly.accept(part, 8, None) {
|
||||
result = Some(p);
|
||||
}
|
||||
}
|
||||
assert_eq!(result, Some(p.clone()));
|
||||
let mut t = tuple(&p).unwrap();
|
||||
t.dst = Ipv4Addr::new(10, 0, 0, 240);
|
||||
t.dport = 53;
|
||||
let out = rewrite(&p, t).unwrap();
|
||||
assert_eq!(header(&out), Some(20));
|
||||
let mut sum = 0u32;
|
||||
for pair in out[12..20].as_chunks::<2>().0 {
|
||||
sum += u32::from(u16::from_be_bytes([pair[0], pair[1]]));
|
||||
}
|
||||
sum += 17 + (out.len() - 20) as u32;
|
||||
assert_eq!(checksum_seed(&out[20..], sum), 0);
|
||||
}
|
||||
#[test]
|
||||
fn overlaps_and_capacity_are_rejected_without_allocating_unbounded_state() {
|
||||
let parts = fragments(datagram(), 1280);
|
||||
let mut r = Reassembly::default();
|
||||
assert!(r.accept(parts[0].clone(), 0, None).is_none());
|
||||
assert!(r.pending.is_empty());
|
||||
assert!(r.accept(parts[0].clone(), 1, None).is_none());
|
||||
assert!(r.accept(parts[0].clone(), 1, None).is_none());
|
||||
assert_eq!(r.pending.len(), 1);
|
||||
let mut conflict = parts[0].to_vec();
|
||||
conflict[20] ^= 1;
|
||||
assert!(r.accept(Bytes::from(conflict), 1, None).is_none());
|
||||
assert!(r.pending.is_empty());
|
||||
for size in 0..80 {
|
||||
let p = vec![0xff; size];
|
||||
assert!(tuple(&p).is_none());
|
||||
assert!(icmp_quote(&p).is_none());
|
||||
}
|
||||
}
|
||||
#[test]
|
||||
fn fragment_assembly_is_scoped_to_authenticated_sender_and_expires_when_idle() {
|
||||
let packet = datagram();
|
||||
let parts = fragments(packet.clone(), 1280);
|
||||
let mut r = Reassembly::default();
|
||||
assert!(r.accept(parts[0].clone(), 8, Some([1; 32])).is_none());
|
||||
assert!(r.accept(parts[1].clone(), 8, Some([2; 32])).is_none());
|
||||
assert!(r.accept(parts[2].clone(), 8, Some([1; 32])).is_none());
|
||||
assert_eq!(r.accept(parts[1].clone(), 8, Some([1; 32])), Some(packet));
|
||||
assert_eq!(r.pending.len(), 1);
|
||||
r.expire(Instant::now() + Duration::from_secs(6));
|
||||
assert!(r.pending.is_empty());
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod icmp_tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
#[test]
|
||||
fn pmtu_errors_restore_quoted_addresses_ports_and_checksums() {
|
||||
let original = Tuple {
|
||||
src: Ipv4Addr::new(10, 0, 0, 1),
|
||||
dst: Ipv4Addr::new(10, 0, 0, 240),
|
||||
sport: 40000,
|
||||
dport: 53,
|
||||
protocol: Protocol::Udp,
|
||||
};
|
||||
let mut p = vec![0; 36];
|
||||
p[0] = 0x45;
|
||||
p[2..4].copy_from_slice(&36u16.to_be_bytes());
|
||||
p[8] = 64;
|
||||
p[9] = 17;
|
||||
p[24..26].copy_from_slice(&16u16.to_be_bytes());
|
||||
let c = checksum(&p[..20]);
|
||||
p[10..12].copy_from_slice(&c.to_be_bytes());
|
||||
let p = rewrite(&p, original).unwrap();
|
||||
let wire = Tuple {
|
||||
dst: Ipv4Addr::new(10, 0, 0, 2),
|
||||
sport: 50000,
|
||||
dport: 5353,
|
||||
..original
|
||||
};
|
||||
let translated = rewrite(&p, wire).unwrap();
|
||||
let mut error = vec![0; 56];
|
||||
error[0] = 0x45;
|
||||
error[2..4].copy_from_slice(&56u16.to_be_bytes());
|
||||
error[8] = 64;
|
||||
error[9] = 1;
|
||||
error[12..16].copy_from_slice(&wire.dst.octets());
|
||||
error[16..20].copy_from_slice(&wire.src.octets());
|
||||
error[20] = 3;
|
||||
error[21] = 4;
|
||||
error[26..28].copy_from_slice(&1200u16.to_be_bytes());
|
||||
error[28..].copy_from_slice(&translated[..28]);
|
||||
let c = checksum(&error[20..]);
|
||||
error[22..24].copy_from_slice(&c.to_be_bytes());
|
||||
let c = checksum(&error[..20]);
|
||||
error[10..12].copy_from_slice(&c.to_be_bytes());
|
||||
let restored = rewrite_icmp(&error, original).unwrap();
|
||||
assert_eq!(icmp_quote(&restored), Some(original));
|
||||
assert_eq!(checksum(&restored[20..]), 0);
|
||||
assert_eq!(checksum(&restored[28..48]), 0);
|
||||
assert_eq!(&restored[54..56], &p[26..28]);
|
||||
assert_eq!(&restored[26..28], &1200u16.to_be_bytes());
|
||||
assert_eq!(&restored[12..16], &original.dst.octets());
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,144 @@
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
use crate::NetworkId;
|
||||
|
||||
#[test]
|
||||
fn concurrent_backend_edits_merge_and_deletion_cannot_resurrect() {
|
||||
let net = NetworkId::from_bytes([7; 32]);
|
||||
let a = iroh::SecretKey::generate();
|
||||
let b = iroh::SecretKey::generate();
|
||||
let limits = Limits::default();
|
||||
let mut state = VipSet::default();
|
||||
let id = hex::encode([9; 16]);
|
||||
let initial = VipConfig {
|
||||
id: id.clone(),
|
||||
name: "api".into(),
|
||||
listeners: vec![Listener::new(Protocol::Tcp, 443)],
|
||||
};
|
||||
let changes = state.edit(net, &a, 1, &initial, None, &limits).unwrap();
|
||||
state.merge(net, changes, &limits).unwrap();
|
||||
let revision = state.revision(&id);
|
||||
let mut first = initial.clone();
|
||||
first.listeners[0].backends.push(Backend {
|
||||
peer: a.public().to_string(),
|
||||
port: 6443,
|
||||
});
|
||||
let mut second = initial.clone();
|
||||
second.listeners[0].backends.push(Backend {
|
||||
peer: b.public().to_string(),
|
||||
port: 7443,
|
||||
});
|
||||
let x = state
|
||||
.edit(net, &a, 2, &first, Some(&revision), &limits)
|
||||
.unwrap();
|
||||
let y = state
|
||||
.edit(net, &b, 1, &second, Some(&revision), &limits)
|
||||
.unwrap();
|
||||
let mut reverse = state.clone();
|
||||
state.merge(net, x.clone(), &limits).unwrap();
|
||||
state.merge(net, y.clone(), &limits).unwrap();
|
||||
reverse.merge(net, y, &limits).unwrap();
|
||||
reverse.merge(net, x, &limits).unwrap();
|
||||
assert_eq!(state.configs(), reverse.configs());
|
||||
assert_eq!(state.configs()[0].listeners[0].backends.len(), 2);
|
||||
assert!(
|
||||
state
|
||||
.edit(net, &a, 3, &initial, Some(&revision), &limits)
|
||||
.is_err()
|
||||
);
|
||||
let removed = state.delete(net, &a, 3, &id, &state.revision(&id)).unwrap();
|
||||
let old = state.entries(&id);
|
||||
state.merge(net, removed, &limits).unwrap();
|
||||
state.merge(net, old, &limits).unwrap();
|
||||
assert!(state.configs().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn signed_fields_reject_tampering_wrong_network_and_limits() {
|
||||
let net = NetworkId::from_bytes([2; 32]);
|
||||
let key = iroh::SecretKey::generate();
|
||||
let limits = Limits::default();
|
||||
let state = VipSet::default();
|
||||
let cfg = VipConfig {
|
||||
id: hex::encode([3; 16]),
|
||||
name: "dns".into(),
|
||||
..Default::default()
|
||||
};
|
||||
let entries = state.edit(net, &key, 1, &cfg, None, &limits).unwrap();
|
||||
let mut bad = entries.clone();
|
||||
bad[0].clock += 1;
|
||||
assert!(VipSet::default().merge(net, bad, &limits).is_err());
|
||||
assert!(
|
||||
VipSet::default()
|
||||
.merge(NetworkId::from_bytes([4; 32]), entries.clone(), &limits)
|
||||
.is_err()
|
||||
);
|
||||
let mut small = limits.clone();
|
||||
small.max_vips = 0;
|
||||
assert!(VipSet::default().merge(net, entries, &small).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deletion_history_does_not_exhaust_active_capacity_or_accept_stale_fields() {
|
||||
let network = NetworkId::from_bytes([9; 32]);
|
||||
let key = iroh::SecretKey::generate();
|
||||
let limits = Limits {
|
||||
max_vips: 1,
|
||||
max_vip_tombstones: 4,
|
||||
..Default::default()
|
||||
};
|
||||
let mut state = VipSet::default();
|
||||
for index in 0..4 {
|
||||
let config = VipConfig::new("reused-name");
|
||||
let fields = state
|
||||
.edit(network, &key, index * 2 + 1, &config, None, &limits)
|
||||
.unwrap();
|
||||
state.merge(network, fields.clone(), &limits).unwrap();
|
||||
let deletion = state
|
||||
.delete(
|
||||
network,
|
||||
&key,
|
||||
index * 2 + 2,
|
||||
&config.id,
|
||||
&state.revision(&config.id),
|
||||
)
|
||||
.unwrap();
|
||||
state.merge(network, deletion, &limits).unwrap();
|
||||
assert_eq!(state.entries(&config.id).len(), 1);
|
||||
assert!(!state.merge(network, fields, &limits).unwrap());
|
||||
}
|
||||
let next = VipConfig::new("next");
|
||||
let fields = state.edit(network, &key, 9, &next, None, &limits).unwrap();
|
||||
state.merge(network, fields, &limits).unwrap();
|
||||
assert_eq!(state.configs().len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reservations_avoid_nodes_and_name_conflicts_have_a_deterministic_holder() {
|
||||
let network = NetworkId::from_bytes([4; 32]);
|
||||
let key = iroh::SecretKey::generate();
|
||||
let limits = Limits::default();
|
||||
let mut state = VipSet::default();
|
||||
for version in 1..=2 {
|
||||
let config = VipConfig::new("shared-name");
|
||||
let fields = state
|
||||
.edit(network, &key, version, &config, None, &limits)
|
||||
.unwrap();
|
||||
state.merge(network, fields, &limits).unwrap();
|
||||
}
|
||||
let range = "10.0.0.0/29".parse().unwrap();
|
||||
let taken = ["10.0.0.1".parse().unwrap()].into_iter().collect();
|
||||
let views = state.views(Some(range), taken, &Default::default(), &limits);
|
||||
assert!(views[0].error.is_none());
|
||||
assert!(
|
||||
views[1]
|
||||
.error
|
||||
.as_deref()
|
||||
.unwrap()
|
||||
.contains("name conflicts")
|
||||
);
|
||||
assert_ne!(views[0].address, views[1].address);
|
||||
for v in views {
|
||||
assert_ne!(v.address, Some("10.0.0.1".parse().unwrap()));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,207 @@
|
||||
//! Shared services use real membership, signed state and mandatory SQLite.
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::testing::{TestAgent, config_with, network, wait_for_peers, wait_until};
|
||||
use tsunagi::vip::{Backend, Listener, Protocol, VipConfig, VipStatus};
|
||||
use tsunagi::{Agent, NetworkId};
|
||||
async fn view(agent: &Agent, network: NetworkId, id: &str) -> VipStatus {
|
||||
wait_until("VIP configuration reaches the replica", || async {
|
||||
agent
|
||||
.network_status(network)
|
||||
.await
|
||||
.ok()?
|
||||
.vips
|
||||
.into_iter()
|
||||
.find(|v| v.config.id == id)
|
||||
})
|
||||
.await
|
||||
}
|
||||
#[tokio::test]
|
||||
async fn any_member_can_edit_services_and_an_offline_creator_is_not_required() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let a = TestAgent::spawn(&discovery).await.unwrap();
|
||||
let b = TestAgent::spawn(&discovery).await.unwrap();
|
||||
let (name, secret) = network("shared-vips");
|
||||
let net = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
wait_for_peers(&a.agent, net, 1).await;
|
||||
let cfg = VipConfig::new("api");
|
||||
a.agent
|
||||
.save_vip(net, cfg.clone(), None, false)
|
||||
.await
|
||||
.unwrap();
|
||||
let before = view(&b.agent, net, &cfg.id).await;
|
||||
let mut edited = before.config.clone();
|
||||
let mut listener = Listener::new(Protocol::Tcp, 443);
|
||||
listener.backends.push(Backend {
|
||||
peer: b.agent.endpoint_id().to_string(),
|
||||
port: 6443,
|
||||
});
|
||||
edited.listeners.push(listener);
|
||||
b.agent
|
||||
.save_vip(net, edited.clone(), Some(before.revision.clone()), false)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(
|
||||
b.agent
|
||||
.save_vip(net, cfg.clone(), Some(before.revision), false)
|
||||
.await
|
||||
.is_err()
|
||||
);
|
||||
wait_until("creator observes another member's edit", || async {
|
||||
(view(&a.agent, net, &cfg.id).await.config == edited).then_some(())
|
||||
})
|
||||
.await;
|
||||
let a_dir = a.stop().await;
|
||||
let c = TestAgent::spawn(&discovery).await.unwrap();
|
||||
c.agent.join_network(&name, &secret).await.unwrap();
|
||||
assert_eq!(view(&c.agent, net, &cfg.id).await.config, edited);
|
||||
let a = Agent::spawn(config_with(a_dir.path(), &discovery))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(view(&a, net, &cfg.id).await.config, edited);
|
||||
let current = view(&c.agent, net, &cfg.id).await;
|
||||
c.agent
|
||||
.save_vip(net, current.config, Some(current.revision), true)
|
||||
.await
|
||||
.unwrap();
|
||||
wait_until("deletion reaches the original creator", || async {
|
||||
a.network_status(net)
|
||||
.await
|
||||
.ok()?
|
||||
.vips
|
||||
.is_empty()
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
a.shutdown().await;
|
||||
b.stop().await;
|
||||
c.stop().await;
|
||||
}
|
||||
#[tokio::test]
|
||||
async fn invalid_drafts_are_atomic_and_services_do_not_cross_networks() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let a = TestAgent::spawn(&discovery).await.unwrap();
|
||||
let (name, secret) = network("vip-one");
|
||||
let net = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
let (other, secret) = network("vip-two");
|
||||
let second = a.agent.join_network(&other, &secret).await.unwrap();
|
||||
let mut cfg = VipConfig::new("api");
|
||||
cfg.listeners.push(Listener::new(Protocol::Udp, 53));
|
||||
a.agent
|
||||
.save_vip(net, cfg.clone(), None, false)
|
||||
.await
|
||||
.unwrap();
|
||||
let before = view(&a.agent, net, &cfg.id).await;
|
||||
let mut invalid = before.config.clone();
|
||||
invalid.name = "renamed".into();
|
||||
invalid.listeners.push(invalid.listeners[0].clone());
|
||||
assert!(
|
||||
a.agent
|
||||
.save_vip(net, invalid, Some(before.revision.clone()), false)
|
||||
.await
|
||||
.is_err()
|
||||
);
|
||||
let after = view(&a.agent, net, &cfg.id).await;
|
||||
assert_eq!(after.config, before.config);
|
||||
assert_eq!(after.revision, before.revision);
|
||||
assert!(
|
||||
a.agent
|
||||
.network_status(second)
|
||||
.await
|
||||
.unwrap()
|
||||
.vips
|
||||
.is_empty()
|
||||
);
|
||||
a.stop().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_restored_author_advances_its_counter_past_its_own_updates_held_by_a_replica() {
|
||||
use tsunagi::config::StoragePaths;
|
||||
use tsunagi::storage::StateStore;
|
||||
use tsunagi::vip::VipSet;
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let a = TestAgent::spawn(&discovery).await.unwrap();
|
||||
let b = TestAgent::spawn(&discovery).await.unwrap();
|
||||
let (name, secret) = network("vip-author-recovery");
|
||||
let net = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
wait_for_peers(&a.agent, net, 1).await;
|
||||
let config = VipConfig::new("api");
|
||||
a.agent
|
||||
.save_vip(net, config.clone(), None, false)
|
||||
.await
|
||||
.unwrap();
|
||||
view(&b.agent, net, &config.id).await;
|
||||
let author = a.agent.endpoint_id();
|
||||
let adir = a.stop().await;
|
||||
let bdir = b.stop().await;
|
||||
let apath = StoragePaths::under(adir.path()).state_db();
|
||||
let bpath = StoragePaths::under(bdir.path()).state_db();
|
||||
// Model an old author backup with a newer, genuinely signed update retained
|
||||
// by another offline replica. Connections and recovery below remain real.
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let a = StateStore::open(apath).unwrap();
|
||||
let b = StateStore::open(bpath).unwrap();
|
||||
let mut set = VipSet::default();
|
||||
set.merge(
|
||||
net,
|
||||
a.vip_fields(net, 192).unwrap(),
|
||||
&tsunagi::Limits::default(),
|
||||
)
|
||||
.unwrap();
|
||||
let mut cfg = set.configs().remove(0);
|
||||
cfg.name = "newer".into();
|
||||
let key =
|
||||
iroh::SecretKey::from_bytes(&a.device_identity().unwrap().unwrap().secret_bytes());
|
||||
let fields = set
|
||||
.edit(
|
||||
net,
|
||||
&key,
|
||||
100,
|
||||
&cfg,
|
||||
Some(&set.revision(&cfg.id)),
|
||||
&tsunagi::Limits::default(),
|
||||
)
|
||||
.unwrap();
|
||||
set.merge(net, fields, &tsunagi::Limits::default()).unwrap();
|
||||
b.put_vip_fields(net, &set.entries(&cfg.id), None).unwrap();
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
let b = Agent::spawn(config_with(bdir.path(), &discovery))
|
||||
.await
|
||||
.unwrap();
|
||||
let a = Agent::spawn(config_with(adir.path(), &discovery))
|
||||
.await
|
||||
.unwrap();
|
||||
let learned = wait_until(
|
||||
"the restored author learns its newer signed service",
|
||||
|| async {
|
||||
let v = view(&a, net, &config.id).await;
|
||||
(v.config.name == "newer").then_some(v)
|
||||
},
|
||||
)
|
||||
.await;
|
||||
let mut next = learned.config;
|
||||
next.name = "latest".into();
|
||||
a.save_vip(net, next, Some(learned.revision), false)
|
||||
.await
|
||||
.unwrap();
|
||||
a.shutdown().await;
|
||||
let path = StoragePaths::under(adir.path()).state_db();
|
||||
let version = tokio::task::spawn_blocking(move || {
|
||||
StateStore::open(path)
|
||||
.unwrap()
|
||||
.own_record_version(net, author)
|
||||
.unwrap()
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(
|
||||
version > 100,
|
||||
"the observed author counter must not be reused"
|
||||
);
|
||||
b.shutdown().await;
|
||||
}
|
||||
@@ -21,6 +21,13 @@ const POLL_INTERVAL: Duration = Duration::from_secs(2);
|
||||
|
||||
/// A request from the UI to the agent.
|
||||
pub(crate) enum Command {
|
||||
SaveVip {
|
||||
token: String,
|
||||
network_id: String,
|
||||
config: tsunagi::vip::VipConfig,
|
||||
expected: Option<String>,
|
||||
delete: bool,
|
||||
},
|
||||
/// Retire the exact offline-node claim selected in the devices window.
|
||||
ForgetNode {
|
||||
network_id: String,
|
||||
@@ -55,8 +62,14 @@ pub(crate) enum Command {
|
||||
}
|
||||
|
||||
/// The latest the worker knows, read by the UI each frame.
|
||||
#[derive(Clone)]
|
||||
pub(crate) struct VipSaveResult {
|
||||
pub token: String,
|
||||
pub result: Result<Vec<tsunagi::vip::VipStatus>, String>,
|
||||
}
|
||||
#[derive(Clone, Default)]
|
||||
pub(crate) struct Snapshot {
|
||||
pub vip_result: Option<VipSaveResult>,
|
||||
/// The last status, or the reason it could not be read (agent down, no
|
||||
/// permission on the socket, …).
|
||||
pub status: Option<Result<StatusReport, String>>,
|
||||
@@ -84,6 +97,18 @@ pub(crate) struct AgentClient {
|
||||
}
|
||||
|
||||
impl AgentClient {
|
||||
#[cfg(test)]
|
||||
pub(crate) fn test_client() -> (Self, mpsc::UnboundedReceiver<Command>) {
|
||||
let (tx, rx) = mpsc::unbounded_channel();
|
||||
(
|
||||
Self {
|
||||
tx,
|
||||
shared: Arc::new(Mutex::new(Snapshot::default())),
|
||||
pending_copy: Arc::new(Mutex::new(None)),
|
||||
},
|
||||
rx,
|
||||
)
|
||||
}
|
||||
/// Starts the worker on `runtime`, reporting to `ctx` (for repaints).
|
||||
pub(crate) fn spawn(runtime: &tokio::runtime::Handle, ctx: egui::Context) -> Self {
|
||||
let (tx, rx) = mpsc::unbounded_channel();
|
||||
@@ -198,6 +223,22 @@ async fn run(
|
||||
ctx.request_repaint();
|
||||
|
||||
let outcome: Result<String, String> = match command {
|
||||
Command::SaveVip {
|
||||
token,
|
||||
network_id,
|
||||
config,
|
||||
expected,
|
||||
delete,
|
||||
} => {
|
||||
let result = ipc::save_vip(socket, &network_id, config, expected, delete)
|
||||
.await
|
||||
.map_err(|e| e.to_string());
|
||||
lock(shared).vip_result = Some(VipSaveResult {
|
||||
token,
|
||||
result: result.clone(),
|
||||
});
|
||||
result.map(|_| "VIP configuration saved".into())
|
||||
}
|
||||
Command::ForgetNode {
|
||||
network_id,
|
||||
peer,
|
||||
|
||||
@@ -64,6 +64,7 @@ impl eframe::App for App {
|
||||
|
||||
ui::draw(ctx, &self.agent, &mut self.state, &snapshot, &self.traffic);
|
||||
self.show_devices_windows(ctx, &snapshot);
|
||||
self.show_vip_windows(ctx, &snapshot);
|
||||
self.show_about_window(ctx, &snapshot);
|
||||
|
||||
// A secret the worker fetched for the clipboard: copy it here, on the
|
||||
@@ -78,6 +79,51 @@ impl eframe::App for App {
|
||||
}
|
||||
|
||||
impl App {
|
||||
fn show_vip_windows(&mut self, ctx: &egui::Context, snapshot: &crate::agent::Snapshot) {
|
||||
let Some(Ok(report)) = &snapshot.status else {
|
||||
return;
|
||||
};
|
||||
let ids: Vec<_> = self.state.vip_windows.keys().cloned().collect();
|
||||
for id in ids {
|
||||
let Some(network) = report.networks.iter().find(|n| n.network_id == id) else {
|
||||
self.state.vip_windows.remove(&id);
|
||||
continue;
|
||||
};
|
||||
let Some(state) = self.state.vip_windows.get_mut(&id) else {
|
||||
continue;
|
||||
};
|
||||
let builder = egui::ViewportBuilder::default()
|
||||
.with_title(format!("tsunagi · VIP · {}", network.name))
|
||||
.with_app_id(crate::icon::APP_ID)
|
||||
.with_icon(crate::icon::window())
|
||||
.with_inner_size([1000., 720.])
|
||||
.with_min_inner_size([650., 400.]);
|
||||
let keep = ctx.show_viewport_immediate(
|
||||
egui::ViewportId::from_hash_of(("vip", &id)),
|
||||
builder,
|
||||
|vctx, _| {
|
||||
let close = crate::vip::show(vctx, &self.agent, state, network, snapshot);
|
||||
if close {
|
||||
return false;
|
||||
}
|
||||
if vctx.input(|i| i.viewport().close_requested()) {
|
||||
if state.dirty() {
|
||||
state.request_close();
|
||||
vctx.send_viewport_cmd(egui::ViewportCommand::CancelClose);
|
||||
true
|
||||
} else {
|
||||
false
|
||||
}
|
||||
} else {
|
||||
true
|
||||
}
|
||||
},
|
||||
);
|
||||
if !keep {
|
||||
self.state.vip_windows.remove(&id);
|
||||
}
|
||||
}
|
||||
}
|
||||
/// Renders the About window while it is open.
|
||||
fn show_about_window(&mut self, ctx: &egui::Context, snapshot: &crate::agent::Snapshot) {
|
||||
if !self.state.about_open {
|
||||
|
||||
@@ -19,6 +19,7 @@ mod icon;
|
||||
mod stats;
|
||||
mod tray;
|
||||
mod ui;
|
||||
mod vip;
|
||||
mod watch;
|
||||
|
||||
use eframe::egui;
|
||||
|
||||
@@ -34,7 +34,7 @@ pub(crate) struct Series {
|
||||
}
|
||||
|
||||
impl Series {
|
||||
fn sample(&mut self, now: Instant, packets: u64, bytes: u64) {
|
||||
pub(crate) fn sample(&mut self, now: Instant, packets: u64, bytes: u64) {
|
||||
if let Some((then, prev_packets, prev_bytes)) = self.last {
|
||||
let dt = now.saturating_duration_since(then).as_secs_f32();
|
||||
if dt > 0.0 {
|
||||
|
||||
@@ -41,6 +41,7 @@ pub(crate) struct UiState {
|
||||
pub(crate) host_edit: Option<String>,
|
||||
/// Network ids whose devices window is open.
|
||||
pub(crate) open_devices: BTreeSet<String>,
|
||||
pub(crate) vip_windows: std::collections::BTreeMap<String, crate::vip::Window>,
|
||||
/// Whether the About window is open.
|
||||
pub(crate) about_open: bool,
|
||||
/// An exit-node choice waiting for the user to confirm it.
|
||||
@@ -56,6 +57,7 @@ impl Default for UiState {
|
||||
unit: Unit::Packets,
|
||||
host_edit: None,
|
||||
open_devices: BTreeSet::new(),
|
||||
vip_windows: Default::default(),
|
||||
about_open: false,
|
||||
exit_confirm: None,
|
||||
forget_confirm: None,
|
||||
@@ -211,6 +213,7 @@ fn draw_network(
|
||||
egui::Frame::group(ui.style()).show(ui, |ui| {
|
||||
ui.horizontal(|ui| {
|
||||
format::copy_field(ui, network.name.as_str(), &network.name);
|
||||
if ui.button("VIP").clicked(){state.vip_windows.entry(network.network_id.clone()).or_default();}
|
||||
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
||||
let leave =
|
||||
egui::Button::new(egui::RichText::new("Leave").color(egui::Color32::WHITE))
|
||||
|
||||
@@ -0,0 +1,461 @@
|
||||
//! Per-network VIP editor. Drafts never issue commands until Save is pressed.
|
||||
use crate::agent::{AgentClient, Command, Snapshot};
|
||||
use crate::stats::{Series, Unit};
|
||||
use eframe::egui;
|
||||
use std::collections::BTreeMap;
|
||||
use tsunagi::ipc::NetworkReport;
|
||||
use tsunagi::vip::{Backend, Listener, Protocol, VipConfig, VipStatus};
|
||||
#[derive(Clone)]
|
||||
struct Draft {
|
||||
config: VipConfig,
|
||||
original: VipConfig,
|
||||
revision: Option<String>,
|
||||
delete: bool,
|
||||
}
|
||||
impl Draft {
|
||||
fn new(config: VipConfig, revision: Option<String>) -> Self {
|
||||
Self {
|
||||
original: config.clone(),
|
||||
config,
|
||||
revision,
|
||||
delete: false,
|
||||
}
|
||||
}
|
||||
fn dirty(&self) -> bool {
|
||||
self.revision.is_none() || self.delete || self.config != self.original
|
||||
}
|
||||
}
|
||||
#[derive(Default)]
|
||||
pub(crate) struct Window {
|
||||
draft: Option<Draft>,
|
||||
pending: Option<String>,
|
||||
error: Option<String>,
|
||||
close_question: bool,
|
||||
close_after_save: bool,
|
||||
rates: BTreeMap<String, (Series, Series)>,
|
||||
generation: u64,
|
||||
acknowledged_generation: Option<u64>,
|
||||
}
|
||||
impl Window {
|
||||
pub fn request_close(&mut self) {
|
||||
self.close_question = true;
|
||||
self.close_after_save = true;
|
||||
}
|
||||
pub fn dirty(&self) -> bool {
|
||||
self.draft.as_ref().is_some_and(Draft::dirty) || self.pending.is_some()
|
||||
}
|
||||
}
|
||||
fn peer_name<'a>(n: &'a NetworkReport, id: &'a str) -> &'a str {
|
||||
n.members
|
||||
.iter()
|
||||
.find(|m| m.endpoint_id == id)
|
||||
.and_then(|m| m.hostname.as_deref())
|
||||
.unwrap_or(id)
|
||||
}
|
||||
pub(crate) fn show(
|
||||
ctx: &egui::Context,
|
||||
agent: &AgentClient,
|
||||
state: &mut Window,
|
||||
n: &NetworkReport,
|
||||
snapshot: &Snapshot,
|
||||
) -> bool {
|
||||
if state.generation != snapshot.generation {
|
||||
state.generation = snapshot.generation;
|
||||
if let Some(at) = snapshot.at {
|
||||
for v in &n.vips {
|
||||
let counters = v.listeners.iter().fold((0, 0, 0, 0), |a, l| {
|
||||
(
|
||||
a.0 + l.counters.tx_packets,
|
||||
a.1 + l.counters.tx_bytes,
|
||||
a.2 + l.counters.rx_packets,
|
||||
a.3 + l.counters.rx_bytes,
|
||||
)
|
||||
});
|
||||
let (tx, rx) = state.rates.entry(v.config.id.clone()).or_default();
|
||||
tx.sample(at, counters.0, counters.1);
|
||||
rx.sample(at, counters.2, counters.3);
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(result) = &snapshot.vip_result
|
||||
&& state.pending.as_deref() == Some(result.token.as_str())
|
||||
{
|
||||
state.pending = None;
|
||||
state.acknowledged_generation = Some(snapshot.generation);
|
||||
match &result.result {
|
||||
Ok(vips) => {
|
||||
if let Some(draft) = &state.draft {
|
||||
state.draft = vips
|
||||
.iter()
|
||||
.find(|v| v.config.id == draft.config.id)
|
||||
.map(|v| Draft::new(v.config.clone(), Some(v.revision.clone())));
|
||||
}
|
||||
state.error = None;
|
||||
}
|
||||
Err(error) => {
|
||||
state.error = Some(error.clone());
|
||||
state.close_after_save = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut close = state.close_after_save && !state.dirty();
|
||||
let mut discard_new = false;
|
||||
egui::CentralPanel::default().show(ctx, |ui| {
|
||||
ui.heading(format!("VIP · {}", n.name));
|
||||
ui.weak("Configuration is shared with every member. Changes take effect only after Save.");
|
||||
if state.close_question {
|
||||
ui.horizontal(|ui| {
|
||||
ui.label("Unsaved changes.");
|
||||
if ui
|
||||
.add_enabled(
|
||||
state.pending.is_none(),
|
||||
egui::Button::new("Discard and close"),
|
||||
)
|
||||
.clicked()
|
||||
{
|
||||
state.draft = None;
|
||||
state.close_question = false;
|
||||
close = true;
|
||||
}
|
||||
if ui.button("Keep editing").clicked() {
|
||||
state.close_question = false;
|
||||
state.close_after_save = false;
|
||||
}
|
||||
});
|
||||
}
|
||||
if let Some(error) = &state.error {
|
||||
ui.colored_label(egui::Color32::LIGHT_RED, error);
|
||||
}
|
||||
ui.separator();
|
||||
ui.horizontal_wrapped(|ui| {
|
||||
for vip in &n.vips {
|
||||
let selected = state
|
||||
.draft
|
||||
.as_ref()
|
||||
.is_some_and(|d| d.config.id == vip.config.id);
|
||||
if ui.selectable_label(selected, &vip.config.name).clicked() {
|
||||
if state.dirty() {
|
||||
state.error = Some(
|
||||
"Save or discard the current draft before selecting another VIP."
|
||||
.into(),
|
||||
);
|
||||
} else {
|
||||
state.draft =
|
||||
Some(Draft::new(vip.config.clone(), Some(vip.revision.clone())));
|
||||
state.error = None;
|
||||
}
|
||||
}
|
||||
}
|
||||
if ui.button("+ New VIP").clicked() {
|
||||
if state.dirty() {
|
||||
state.error = Some("Save or discard the current draft first.".into());
|
||||
} else {
|
||||
state.draft = Some(Draft::new(VipConfig::new(""), None));
|
||||
state.error = None;
|
||||
}
|
||||
}
|
||||
});
|
||||
let Some(draft) = &mut state.draft else {
|
||||
ui.label("Select a VIP or create one.");
|
||||
return;
|
||||
};
|
||||
let current = n.vips.iter().find(|v| v.config.id == draft.config.id);
|
||||
if state.pending.is_none()
|
||||
&& state.acknowledged_generation != Some(snapshot.generation)
|
||||
&& !draft.dirty()
|
||||
&& let Some(current) = current
|
||||
&& draft.revision.as_ref() != Some(¤t.revision)
|
||||
{
|
||||
*draft = Draft::new(current.config.clone(), Some(current.revision.clone()));
|
||||
}
|
||||
let changed =
|
||||
current.is_some_and(|v| draft.revision.as_ref().is_some_and(|r| r != &v.revision));
|
||||
if changed {
|
||||
ui.colored_label(
|
||||
egui::Color32::YELLOW,
|
||||
"This VIP changed on the network. Reload before saving; your draft is retained.",
|
||||
);
|
||||
}
|
||||
if draft.revision.is_some() && current.is_none() {
|
||||
ui.colored_label(
|
||||
egui::Color32::YELLOW,
|
||||
"This VIP was removed on the network. The draft cannot overwrite its deletion.",
|
||||
);
|
||||
}
|
||||
let busy = state.pending.is_some();
|
||||
ui.horizontal(|ui| {
|
||||
let valid = draft.config.validate(&tsunagi::Limits::default());
|
||||
if !n.active {
|
||||
ui.weak("Start the network to save changes.");
|
||||
}
|
||||
if ui
|
||||
.add_enabled(
|
||||
n.active
|
||||
&& !busy
|
||||
&& draft.dirty()
|
||||
&& !changed
|
||||
&& (valid.is_ok() || draft.delete),
|
||||
egui::Button::new("Save"),
|
||||
)
|
||||
.clicked()
|
||||
{
|
||||
let token = VipConfig::new("").id;
|
||||
state.pending = Some(token.clone());
|
||||
agent.send(Command::SaveVip {
|
||||
token,
|
||||
network_id: n.network_id.clone(),
|
||||
config: draft.config.clone(),
|
||||
expected: draft.revision.clone(),
|
||||
delete: draft.delete,
|
||||
});
|
||||
}
|
||||
if ui
|
||||
.add_enabled(!busy, egui::Button::new("Discard / reload"))
|
||||
.clicked()
|
||||
{
|
||||
if let Some(v) = current {
|
||||
*draft = Draft::new(v.config.clone(), Some(v.revision.clone()));
|
||||
} else {
|
||||
discard_new = true;
|
||||
}
|
||||
state.error = None;
|
||||
}
|
||||
if busy {
|
||||
ui.spinner();
|
||||
ui.label("Saving…");
|
||||
} else if draft.dirty() {
|
||||
ui.weak("Unsaved changes");
|
||||
}
|
||||
if let Err(error) = valid {
|
||||
ui.colored_label(egui::Color32::LIGHT_RED, error.to_string());
|
||||
}
|
||||
});
|
||||
ui.separator();
|
||||
egui::ScrollArea::vertical().show(ui, |ui| {
|
||||
ui.add_enabled_ui(!busy, |ui| {
|
||||
ui.horizontal(|ui| {
|
||||
ui.label("Name");
|
||||
ui.add(
|
||||
egui::TextEdit::singleline(&mut draft.config.name)
|
||||
.id(egui::Id::new(("vip-name", &draft.config.id))),
|
||||
);
|
||||
ui.weak(format!(".{}", n.name));
|
||||
});
|
||||
if draft.revision.is_some() {
|
||||
ui.checkbox(&mut draft.delete, "Delete this VIP when saved");
|
||||
}
|
||||
if !draft.delete {
|
||||
edit_listeners(ui, &mut draft.config, n);
|
||||
}
|
||||
});
|
||||
if let Some(current) = current {
|
||||
ui.separator();
|
||||
status(ui, current, n, state.rates.get(¤t.config.id));
|
||||
}
|
||||
});
|
||||
});
|
||||
if discard_new {
|
||||
state.draft = None;
|
||||
state.error = None;
|
||||
}
|
||||
close
|
||||
}
|
||||
fn edit_listeners(ui: &mut egui::Ui, config: &mut VipConfig, n: &NetworkReport) {
|
||||
let mut remove = None;
|
||||
for (index, l) in config.listeners.iter_mut().enumerate() {
|
||||
ui.push_id(index,|ui|{egui::Frame::group(ui.style()).show(ui,|ui|{
|
||||
ui.horizontal(|ui|{egui::ComboBox::from_id_salt("protocol").selected_text(l.protocol.to_string()).show_ui(ui,|ui|{ui.selectable_value(&mut l.protocol,Protocol::Tcp,"TCP");ui.selectable_value(&mut l.protocol,Protocol::Udp,"UDP");});ui.label("Port");ui.add(egui::DragValue::new(&mut l.port).range(1..=65535));if ui.button("Remove listener").clicked(){remove=Some(index);}});
|
||||
if l.protocol==Protocol::Tcp{ui.checkbox(&mut l.port_check,"Active TCP port checks");}else{ui.weak("UDP health: encrypted path probes and explicit port errors; application readiness is unknown.");}
|
||||
ui.collapsing("Checks and flow expiry",|ui|{egui::Grid::new("settings").show(ui,|ui|{
|
||||
ui.label("Check interval, ms");ui.add(egui::DragValue::new(&mut l.check_interval_ms).range(250..=3_600_000));ui.end_row();
|
||||
ui.label("Check timeout, ms");ui.add(egui::DragValue::new(&mut l.check_timeout_ms).range(100..=60_000));ui.end_row();
|
||||
ui.label("Failures before exclusion");ui.add(egui::DragValue::new(&mut l.failure_threshold).range(1..=255));ui.end_row();
|
||||
ui.label("Successes before recovery");ui.add(egui::DragValue::new(&mut l.recovery_threshold).range(1..=255));ui.end_row();
|
||||
ui.label("Flow idle timeout, seconds");ui.add(egui::DragValue::new(&mut l.idle_timeout_secs).range(1..=604800));ui.end_row();
|
||||
});});
|
||||
let mut remove_backend=None;for(i,b)in l.backends.iter_mut().enumerate(){ui.push_id(i,|ui|{ui.horizontal(|ui|{
|
||||
egui::ComboBox::from_id_salt("node").width(240.).selected_text(peer_name(n,&b.peer)).show_ui(ui,|ui|{for m in &n.members{let label=format!("{} ({})",m.hostname.as_deref().unwrap_or("unnamed"),m.endpoint_id.chars().take(8).collect::<String>());ui.selectable_value(&mut b.peer,m.endpoint_id.clone(),label);}});
|
||||
ui.label("Target port");ui.add(egui::DragValue::new(&mut b.port).range(1..=65535));if ui.button("Remove").clicked(){remove_backend=Some(i);}
|
||||
});});}if let Some(i)=remove_backend{l.backends.remove(i);}
|
||||
if ui.button("+ Backend").clicked(){l.backends.push(Backend{peer:n.members.first().map(|m|m.endpoint_id.clone()).unwrap_or_default(),port:l.port});}
|
||||
});});
|
||||
}
|
||||
if let Some(index) = remove {
|
||||
config.listeners.remove(index);
|
||||
}
|
||||
if ui.button("+ Listener").clicked() {
|
||||
config.listeners.push(Listener::new(Protocol::Tcp, 443));
|
||||
}
|
||||
}
|
||||
fn status(ui: &mut egui::Ui, v: &VipStatus, n: &NetworkReport, rates: Option<&(Series, Series)>) {
|
||||
ui.heading("Status · this device");
|
||||
if let Some(error) = &v.error {
|
||||
ui.colored_label(egui::Color32::LIGHT_RED, error);
|
||||
}
|
||||
if let Some((tx, rx)) = rates {
|
||||
ui.label(format!(
|
||||
"TX {:.1} KiB/s · RX {:.1} KiB/s",
|
||||
tx.rate(Unit::Bytes) / 1024.,
|
||||
rx.rate(Unit::Bytes) / 1024.
|
||||
));
|
||||
}
|
||||
for l in &v.listeners {
|
||||
ui.strong(format!("{}/{}", l.protocol, l.port));
|
||||
let selected = l
|
||||
.selected
|
||||
.as_ref()
|
||||
.map(|b| format!("{}:{}", peer_name(n, &b.peer), b.port))
|
||||
.unwrap_or_else(|| "none".into());
|
||||
ui.label(format!("Next new flow: {selected}"));
|
||||
ui.label(format!(
|
||||
"TX {} bytes / {} packets · RX {} bytes / {} packets · drops {} · UDP failovers {}",
|
||||
l.counters.tx_bytes,
|
||||
l.counters.tx_packets,
|
||||
l.counters.rx_bytes,
|
||||
l.counters.rx_packets,
|
||||
l.counters.dropped,
|
||||
l.counters.failovers
|
||||
));
|
||||
for b in &l.backends {
|
||||
let rtt = b
|
||||
.rtt_us
|
||||
.map(|v| format!("{:.2} ms", v as f64 / 1000.))
|
||||
.unwrap_or_else(|| "unknown".into());
|
||||
let age = b
|
||||
.measurement_age_ms
|
||||
.map(|v| format!("{:.1}s", v as f64 / 1000.))
|
||||
.unwrap_or_else(|| "unknown".into());
|
||||
ui.label(format!(
|
||||
"{}:{} · {} · RTT {} (age {}) · {} active · TX/RX {}/{} bytes",
|
||||
peer_name(n, &b.backend.peer),
|
||||
b.backend.port,
|
||||
b.state,
|
||||
rtt,
|
||||
age,
|
||||
b.active_flows,
|
||||
b.counters.tx_bytes,
|
||||
b.counters.rx_bytes
|
||||
));
|
||||
}
|
||||
}
|
||||
ui.collapsing("Internal address and identity", |ui| {
|
||||
ui.monospace(&v.config.id);
|
||||
if let Some(address) = v.address {
|
||||
ui.monospace(address.to_string());
|
||||
}
|
||||
});
|
||||
}
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
#[test]
|
||||
fn editing_the_actual_form_sends_nothing_until_save_is_clicked() {
|
||||
let (agent, mut commands) = AgentClient::test_client();
|
||||
let ctx = egui::Context::default();
|
||||
let mut config = VipConfig::new("api");
|
||||
config.listeners.push(Listener::new(Protocol::Tcp, 443));
|
||||
let mut state = Window {
|
||||
draft: Some(Draft::new(config.clone(), Some("revision".into()))),
|
||||
..Default::default()
|
||||
};
|
||||
let network = NetworkReport {
|
||||
active: true,
|
||||
name: "lab".into(),
|
||||
vips: vec![VipStatus {
|
||||
config: config.clone(),
|
||||
revision: "revision".into(),
|
||||
address: None,
|
||||
error: None,
|
||||
listeners: vec![],
|
||||
}],
|
||||
..Default::default()
|
||||
};
|
||||
let snapshot = Snapshot::default();
|
||||
let frame = |events: Vec<egui::Event>, state: &mut Window| {
|
||||
ctx.run(
|
||||
egui::RawInput {
|
||||
screen_rect: Some(egui::Rect::from_min_size(
|
||||
egui::Pos2::ZERO,
|
||||
egui::vec2(1000., 800.),
|
||||
)),
|
||||
events,
|
||||
..Default::default()
|
||||
},
|
||||
|ctx| {
|
||||
show(ctx, &agent, state, &network, &snapshot);
|
||||
},
|
||||
)
|
||||
};
|
||||
frame(vec![], &mut state);
|
||||
ctx.memory_mut(|m| m.request_focus(egui::Id::new(("vip-name", &config.id))));
|
||||
frame(vec![egui::Event::Text("new".into())], &mut state);
|
||||
assert!(state.dirty());
|
||||
assert_ne!(state.draft.as_ref().unwrap().config.name, config.name);
|
||||
assert!(commands.try_recv().is_err());
|
||||
assert_eq!(network.vips[0].config, config);
|
||||
fn save_position(shape: &egui::Shape) -> Option<egui::Pos2> {
|
||||
match shape {
|
||||
egui::Shape::Text(text) if text.galley.text() == "Save" => {
|
||||
Some(text.pos + text.galley.rect.center().to_vec2())
|
||||
}
|
||||
egui::Shape::Vec(shapes) => shapes.iter().find_map(save_position),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
// Save was disabled before this edit; allow the next layout to register
|
||||
// its now-enabled hit target before dispatching the pointer press.
|
||||
let ready = frame(vec![], &mut state);
|
||||
let position = ready
|
||||
.shapes
|
||||
.iter()
|
||||
.find_map(|s| save_position(&s.shape))
|
||||
.unwrap();
|
||||
frame(
|
||||
vec![
|
||||
egui::Event::PointerMoved(position),
|
||||
egui::Event::PointerButton {
|
||||
pos: position,
|
||||
button: egui::PointerButton::Primary,
|
||||
pressed: true,
|
||||
modifiers: egui::Modifiers::NONE,
|
||||
},
|
||||
],
|
||||
&mut state,
|
||||
);
|
||||
frame(
|
||||
vec![egui::Event::PointerButton {
|
||||
pos: position,
|
||||
button: egui::PointerButton::Primary,
|
||||
pressed: false,
|
||||
modifiers: egui::Modifiers::NONE,
|
||||
}],
|
||||
&mut state,
|
||||
);
|
||||
let Command::SaveVip {
|
||||
config: saved,
|
||||
expected,
|
||||
delete,
|
||||
..
|
||||
} = commands.try_recv().unwrap()
|
||||
else {
|
||||
panic!("Save must submit one complete VIP draft");
|
||||
};
|
||||
assert_ne!(saved.name, config.name);
|
||||
assert_eq!(expected.as_deref(), Some("revision"));
|
||||
assert!(!delete);
|
||||
assert!(commands.try_recv().is_err());
|
||||
}
|
||||
#[test]
|
||||
fn all_listener_settings_render_without_a_live_agent() {
|
||||
let ctx = egui::Context::default();
|
||||
let mut cfg = VipConfig::new("api");
|
||||
cfg.listeners.push(Listener::new(Protocol::Tcp, 443));
|
||||
cfg.listeners.push(Listener::new(Protocol::Udp, 53));
|
||||
let n = NetworkReport::default();
|
||||
let _ = ctx.run(egui::RawInput::default(), |ctx| {
|
||||
egui::CentralPanel::default().show(ctx, |ui| edit_listeners(ui, &mut cfg, &n));
|
||||
});
|
||||
assert_eq!(cfg.listeners.len(), 2);
|
||||
}
|
||||
}
|
||||
@@ -1770,3 +1770,188 @@ async fn lan_broadcast_fanout_is_scoped_opt_in_and_does_not_reflood() {
|
||||
agent.shutdown().await;
|
||||
}
|
||||
}
|
||||
|
||||
fn vip_segment(
|
||||
source: Ipv4Addr,
|
||||
destination: Ipv4Addr,
|
||||
sport: u16,
|
||||
dport: u16,
|
||||
udp: bool,
|
||||
flags: u8,
|
||||
) -> Bytes {
|
||||
fn sum(bytes: &[u8]) -> u16 {
|
||||
let mut s: u32 = bytes
|
||||
.chunks(2)
|
||||
.map(|b| u32::from(u16::from_be_bytes([b[0], *b.get(1).unwrap_or(&0)])))
|
||||
.sum();
|
||||
while s >> 16 != 0 {
|
||||
s = (s & 0xffff) + (s >> 16);
|
||||
}
|
||||
!(s as u16)
|
||||
}
|
||||
let mut p = vec![0; if udp { 36 } else { 40 }];
|
||||
p[0] = 0x45;
|
||||
let len = p.len() as u16;
|
||||
p[2..4].copy_from_slice(&len.to_be_bytes());
|
||||
p[8] = 64;
|
||||
p[9] = if udp { 17 } else { 6 };
|
||||
p[12..16].copy_from_slice(&source.octets());
|
||||
p[16..20].copy_from_slice(&destination.octets());
|
||||
p[20..22].copy_from_slice(&sport.to_be_bytes());
|
||||
p[22..24].copy_from_slice(&dport.to_be_bytes());
|
||||
if udp {
|
||||
p[24..26].copy_from_slice(&16u16.to_be_bytes());
|
||||
} else {
|
||||
p[32] = 0x50;
|
||||
p[33] = flags;
|
||||
p[24..28].copy_from_slice(&123u32.to_be_bytes());
|
||||
}
|
||||
let mut pseudo = p[12..20].to_vec();
|
||||
pseudo.extend([0, p[9]]);
|
||||
pseudo.extend(((p.len() - 20) as u16).to_be_bytes());
|
||||
pseudo.extend(&p[20..]);
|
||||
let c = sum(&pseudo);
|
||||
let offset = if udp { 26 } else { 36 };
|
||||
p[offset..offset + 2].copy_from_slice(&c.to_be_bytes());
|
||||
let c = sum(&p[..20]);
|
||||
p[10..12].copy_from_slice(&c.to_be_bytes());
|
||||
Bytes::from(p)
|
||||
}
|
||||
#[tokio::test]
|
||||
async fn vip_tcp_and_udp_remap_ports_over_real_encryption_and_preserve_source_admission() {
|
||||
use tsunagi::vip::{Backend, Listener, Protocol, VipConfig};
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("vip-encrypted");
|
||||
let a = WgAgent::spawn(&discovery, "vipa").await;
|
||||
let b = WgAgent::spawn(&discovery, "vipb").await;
|
||||
let net = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
a.wait_for_tunnels(net, 1).await;
|
||||
b.wait_for_tunnels(net, 1).await;
|
||||
let aa = a.overlay(net).await;
|
||||
let ab = b.overlay(net).await;
|
||||
let ta = a.tun(net).await;
|
||||
let tb = b.tun(net).await;
|
||||
let mut cfg = VipConfig::new("service");
|
||||
for (protocol, port, backend) in [(Protocol::Tcp, 443, 6443), (Protocol::Udp, 53, 5353)] {
|
||||
let mut l = Listener::new(protocol, port);
|
||||
l.port_check = false;
|
||||
l.backends.push(Backend {
|
||||
peer: b.endpoint_id().to_string(),
|
||||
port: backend,
|
||||
});
|
||||
cfg.listeners.push(l);
|
||||
}
|
||||
a.agent
|
||||
.save_vip(net, cfg.clone(), None, false)
|
||||
.await
|
||||
.unwrap();
|
||||
let vip = wait_until("data-plane probes measure the backend RTT", || async {
|
||||
let status = a.agent.network_status(net).await.ok()?;
|
||||
let v = status.vips.iter().find(|v| v.config.id == cfg.id)?;
|
||||
(v.listeners.iter().all(|l| l.selected.is_some()))
|
||||
.then_some(v.address?)
|
||||
.filter(|_| v.listeners.len() == 2)
|
||||
})
|
||||
.await;
|
||||
for (udp, port, target) in [(false, 443, 6443), (true, 53, 5353)] {
|
||||
ta.push_from_os(vip_segment(aa, vip, 40000, port, udp, 2));
|
||||
let received = tokio::time::timeout(Duration::from_secs(5), tb.pop_to_os())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert_eq!(&received[12..16], &aa.octets());
|
||||
assert_eq!(&received[16..20], &ab.octets());
|
||||
assert_eq!(u16::from_be_bytes([received[22], received[23]]), target);
|
||||
let translated = u16::from_be_bytes([received[20], received[21]]);
|
||||
tb.push_from_os(vip_segment(ab, aa, target, translated, udp, 0x12));
|
||||
let reply = tokio::time::timeout(Duration::from_secs(5), ta.pop_to_os())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert_eq!(&reply[12..16], &vip.octets());
|
||||
assert_eq!(u16::from_be_bytes([reply[20], reply[21]]), port);
|
||||
assert_eq!(u16::from_be_bytes([reply[22], reply[23]]), 40000);
|
||||
// A backend cannot send a VIP source over its authenticated tunnel.
|
||||
let forged = vip_segment(vip, aa, port, 40000, udp, 0x12);
|
||||
tb.push_from_os(forged);
|
||||
}
|
||||
wait_until("forged VIP sources are rejected before NAT", || async {
|
||||
(a.agent.overlay()?.counters.wrong_source >= 2).then_some(())
|
||||
})
|
||||
.await;
|
||||
a.shutdown().await;
|
||||
b.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn vip_udp_reaches_another_backend_after_the_selected_agent_stops() {
|
||||
use tsunagi::vip::{Backend, Listener, Protocol, VipConfig};
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("vip-failover");
|
||||
let a = WgAgent::spawn(&discovery, "vipfa").await;
|
||||
let b = WgAgent::spawn(&discovery, "vipfb").await;
|
||||
let c = WgAgent::spawn(&discovery, "vipfc").await;
|
||||
let net = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
for node in [&b, &c] {
|
||||
node.agent.join_network(&name, &secret).await.unwrap();
|
||||
}
|
||||
for node in [&a, &b, &c] {
|
||||
node.wait_for_tunnels(net, 2).await;
|
||||
}
|
||||
let mut config = VipConfig::new("datagrams");
|
||||
let mut listener = Listener::new(Protocol::Udp, 8888);
|
||||
listener.backends = vec![
|
||||
Backend {
|
||||
peer: b.endpoint_id().to_string(),
|
||||
port: 9000,
|
||||
},
|
||||
Backend {
|
||||
peer: c.endpoint_id().to_string(),
|
||||
port: 9001,
|
||||
},
|
||||
];
|
||||
config.listeners.push(listener);
|
||||
a.agent
|
||||
.save_vip(net, config.clone(), None, false)
|
||||
.await
|
||||
.unwrap();
|
||||
let first = wait_until("both UDP backends have measured paths", || async {
|
||||
let s = a.agent.network_status(net).await.ok()?;
|
||||
let v = s.vips.iter().find(|v| v.config.id == config.id)?;
|
||||
let l = v.listeners.first()?;
|
||||
(l.backends.iter().all(|b| b.rtt_us.is_some())).then_some((v.address?, l.selected.clone()?))
|
||||
})
|
||||
.await;
|
||||
let (dead, alive) = if first.1.peer == b.endpoint_id().to_string() {
|
||||
(b, c)
|
||||
} else {
|
||||
(c, b)
|
||||
};
|
||||
let address = a.overlay(net).await;
|
||||
let tun = a.tun(net).await;
|
||||
let dest = dead.tun(net).await;
|
||||
tun.push_from_os(vip_segment(address, first.0, 41000, 8888, true, 0));
|
||||
tokio::time::timeout(Duration::from_secs(5), dest.pop_to_os())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
dead.shutdown().await;
|
||||
wait_until("the remaining backend becomes the next choice", || async {
|
||||
let s = a.agent.network_status(net).await.ok()?;
|
||||
let v = s.vips.iter().find(|v| v.config.id == config.id)?;
|
||||
(v.listeners.first()?.selected.as_ref()?.peer == alive.endpoint_id().to_string())
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
tun.push_from_os(vip_segment(address, first.0, 41000, 8888, true, 0));
|
||||
let packet = tokio::time::timeout(Duration::from_secs(5), alive.tun(net).await.pop_to_os())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert_eq!(&packet[16..20], &alive.overlay(net).await.octets());
|
||||
let view = a.agent.network_status(net).await.unwrap();
|
||||
assert_eq!(view.vips[0].listeners[0].counters.failovers, 1);
|
||||
a.shutdown().await;
|
||||
alive.shutdown().await;
|
||||
}
|
||||
@@ -27,6 +27,8 @@ clap = { version = "4.5", features = ["derive", "env"] }
|
||||
# colour is never written where it would show up as rubbish.
|
||||
anstream = "1.0"
|
||||
anstyle = "1.0"
|
||||
serde_json = "1.0"
|
||||
tempfile.workspace = true
|
||||
# Local interface addresses, for the diagnostics in `status`.
|
||||
netwatch = "0.19.3"
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ use std::time::Duration;
|
||||
#[cfg(windows)]
|
||||
mod winservice;
|
||||
|
||||
mod vip_cli;
|
||||
use clap::{Args, Parser, Subcommand, ValueEnum};
|
||||
use tsunagi::agent::Event;
|
||||
use tsunagi::config::{AgentConfig, StoragePaths, TransportPolicy};
|
||||
@@ -52,6 +53,8 @@ struct Cli {
|
||||
|
||||
#[derive(Debug, Subcommand)]
|
||||
enum Command {
|
||||
/// Manage named shared TCP/UDP virtual services.
|
||||
Vip(vip_cli::VipArgs),
|
||||
/// Shows this device's identity, and changes it.
|
||||
///
|
||||
/// Every item follows the same shape: name it to see it, name it with a
|
||||
@@ -927,6 +930,7 @@ async fn run(command: Command) -> Result<(), Box<dyn std::error::Error>> {
|
||||
Command::Join(args) => join_command(args).await,
|
||||
Command::Network(args) => network_command(args).await,
|
||||
Command::Dns(args) => dns_command(args).await,
|
||||
Command::Vip(args) => vip_cli::run(args).await,
|
||||
Command::ExitNode(args) => exit_node_command(args).await,
|
||||
Command::Wipe(args) => wipe(args).await,
|
||||
// Taken in `main`, before there is a runtime: the service manager
|
||||
@@ -1317,7 +1321,12 @@ fn spawn_dns(agent: Agent, port: u16) -> DnsService {
|
||||
let members = network.members.iter().filter_map(|member| {
|
||||
Some((member.hostname.clone()?, member.overlay_address_v4?))
|
||||
});
|
||||
zones.push(Zone::new(zone, members));
|
||||
let services = network
|
||||
.vips
|
||||
.iter()
|
||||
.filter(|v| v.error.is_none())
|
||||
.filter_map(|v| Some((v.config.name.clone(), v.address?)));
|
||||
zones.push(Zone::new(zone, members.chain(services)));
|
||||
}
|
||||
let zones = Zones::new(zones);
|
||||
let names = zones.names() as u32;
|
||||
@@ -1577,6 +1586,23 @@ struct AgentControl {
|
||||
}
|
||||
|
||||
impl tsunagi::ipc::ReportSource for AgentControl {
|
||||
fn save_vip(
|
||||
&self,
|
||||
network: String,
|
||||
config: tsunagi::vip::VipConfig,
|
||||
expected: Option<String>,
|
||||
delete: bool,
|
||||
) -> tsunagi::BoxFuture<'_, Result<Vec<tsunagi::vip::VipStatus>, String>> {
|
||||
Box::pin(async move {
|
||||
let network = network
|
||||
.parse()
|
||||
.map_err(|e| format!("invalid network id: {e}"))?;
|
||||
self.agent
|
||||
.save_vip(network, config, expected, delete)
|
||||
.await
|
||||
.map_err(|e| e.to_string())
|
||||
})
|
||||
}
|
||||
fn forget_node(
|
||||
&self,
|
||||
network: String,
|
||||
@@ -5307,6 +5333,7 @@ async fn build_report(
|
||||
};
|
||||
|
||||
NetworkReport {
|
||||
vips: network.vips.clone(),
|
||||
protocols: network.protocols.clone(),
|
||||
auto_start: network.auto_start,
|
||||
exit: exit_report(network),
|
||||
@@ -5740,6 +5767,7 @@ mod status_tests {
|
||||
/// The situation that prompted this: one peer left and came back.
|
||||
fn network_after_a_peer_returned() -> NetworkReport {
|
||||
NetworkReport {
|
||||
vips: Vec::new(),
|
||||
protocols: vec!["wg-quic".into()],
|
||||
auto_start: true,
|
||||
exit: Default::default(),
|
||||
|
||||
@@ -0,0 +1,506 @@
|
||||
//! Named virtual service administration; every mutation submits one complete draft.
|
||||
use super::{PathArgs, control_socket, resolve_network_index};
|
||||
use clap::{Args, Subcommand};
|
||||
use std::path::PathBuf;
|
||||
use tsunagi::ipc::{self, NetworkReport};
|
||||
use tsunagi::vip::{Backend, Listener, Protocol, VipConfig, VipStatus};
|
||||
type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;
|
||||
#[derive(Debug, Args)]
|
||||
#[command(
|
||||
after_help = "Examples:\n tsng vip create lab k8s\n tsng vip port add lab k8s tcp/443 --backend cp1:6443 --backend cp2:7443\n tsng vip backend add lab k8s tcp/443 cp3:6443\n tsng vip status lab k8s --watch\n tsng vip edit lab k8s\n\nNames, full ids and unique id prefixes are accepted. Changes are shared with the network.\nTCP port checks default to on; UDP uses data-path probes and explicit ICMP errors."
|
||||
)]
|
||||
pub(crate) struct VipArgs {
|
||||
#[command(flatten)]
|
||||
paths: PathArgs,
|
||||
/// Agent control socket; defaults to the running service.
|
||||
#[arg(long, global = true, env = "TSUNAGI_CONTROL_SOCKET")]
|
||||
control_socket: Option<PathBuf>,
|
||||
/// Machine-readable configuration and local observations.
|
||||
#[arg(long, global = true)]
|
||||
json: bool,
|
||||
#[command(subcommand)]
|
||||
action: Action,
|
||||
}
|
||||
#[derive(Debug, Args)]
|
||||
struct Target {
|
||||
/// Network exact name, id or unique id prefix.
|
||||
network: String,
|
||||
/// VIP exact name, id or unique id prefix.
|
||||
vip: String,
|
||||
}
|
||||
#[derive(Debug, Args)]
|
||||
struct PortTarget {
|
||||
#[command(flatten)]
|
||||
target: Target,
|
||||
/// Listener as tcp/443 or udp/53.
|
||||
listener: String,
|
||||
}
|
||||
#[derive(Debug, Subcommand)]
|
||||
enum Action {
|
||||
/// List services in one network, or every configured network.
|
||||
List { network: Option<String> },
|
||||
/// Create a named VIP with an automatically reserved internal address.
|
||||
Create { network: String, name: String },
|
||||
/// Show the complete saved configuration and local status.
|
||||
Show(Target),
|
||||
/// Show local health, routing choices and cumulative traffic counters.
|
||||
Status {
|
||||
#[command(flatten)]
|
||||
target: Target,
|
||||
/// Refresh once a second until interrupted.
|
||||
#[arg(long)]
|
||||
watch: bool,
|
||||
},
|
||||
/// Edit the entire configuration in VISUAL/EDITOR and save it atomically.
|
||||
Edit(Target),
|
||||
/// Remove a VIP; existing flows drain until completion or idle expiry.
|
||||
Remove(Target),
|
||||
/// Add, configure or remove listeners independently of the VIP.
|
||||
Port {
|
||||
#[command(subcommand)]
|
||||
action: PortAction,
|
||||
},
|
||||
/// Change a listener's destination nodes and application ports.
|
||||
Backend {
|
||||
#[command(subcommand)]
|
||||
action: BackendAction,
|
||||
},
|
||||
}
|
||||
#[derive(Debug, Subcommand)]
|
||||
enum PortAction {
|
||||
/// Add a listener, optionally with its complete initial backend set.
|
||||
Add {
|
||||
#[command(flatten)]
|
||||
target: PortTarget,
|
||||
/// Node name/id followed by :port. Repeat for multiple destinations.
|
||||
#[arg(long)]
|
||||
backend: Vec<String>,
|
||||
},
|
||||
/// Change check settings and flow idle expiry.
|
||||
Set {
|
||||
#[command(flatten)]
|
||||
target: PortTarget,
|
||||
/// Active TCP connect checks: on or off.
|
||||
#[arg(long,value_parser=["on","off"])]
|
||||
port_check: Option<String>,
|
||||
/// Milliseconds between active checks (250..3600000).
|
||||
#[arg(long)]
|
||||
check_interval_ms: Option<u32>,
|
||||
/// TCP check deadline in milliseconds (100..60000).
|
||||
#[arg(long)]
|
||||
check_timeout_ms: Option<u32>,
|
||||
/// Consecutive failures before exclusion.
|
||||
#[arg(long)]
|
||||
failures: Option<u8>,
|
||||
/// Consecutive successes before recovery.
|
||||
#[arg(long)]
|
||||
successes: Option<u8>,
|
||||
/// Flow idle expiry in seconds (1..604800).
|
||||
#[arg(long)]
|
||||
idle_timeout: Option<u32>,
|
||||
},
|
||||
/// Remove a listener from new-flow selection; existing flows drain.
|
||||
Remove(PortTarget),
|
||||
}
|
||||
#[derive(Debug, Subcommand)]
|
||||
enum BackendAction {
|
||||
/// Add a destination node and port to a listener.
|
||||
Add {
|
||||
#[command(flatten)]
|
||||
target: PortTarget,
|
||||
backend: String,
|
||||
},
|
||||
/// Remove a destination from new-flow selection.
|
||||
Remove {
|
||||
#[command(flatten)]
|
||||
target: PortTarget,
|
||||
backend: String,
|
||||
},
|
||||
}
|
||||
impl Action {
|
||||
fn network(&self) -> Option<&str> {
|
||||
match self {
|
||||
Self::List { network } => network.as_deref(),
|
||||
Self::Create { network, .. } => Some(network),
|
||||
Self::Show(t) | Self::Edit(t) | Self::Remove(t) | Self::Status { target: t, .. } => {
|
||||
Some(&t.network)
|
||||
}
|
||||
Self::Port { action } => Some(
|
||||
&match action {
|
||||
PortAction::Add { target, .. }
|
||||
| PortAction::Set { target, .. }
|
||||
| PortAction::Remove(target) => target,
|
||||
}
|
||||
.target
|
||||
.network,
|
||||
),
|
||||
Self::Backend { action } => Some(
|
||||
&match action {
|
||||
BackendAction::Add { target, .. } | BackendAction::Remove { target, .. } => {
|
||||
target
|
||||
}
|
||||
}
|
||||
.target
|
||||
.network,
|
||||
),
|
||||
}
|
||||
}
|
||||
}
|
||||
fn parse_listener(s: &str) -> Result<(Protocol, u16)> {
|
||||
let (p, n) = s
|
||||
.split_once('/')
|
||||
.ok_or("listener must be tcp/PORT or udp/PORT")?;
|
||||
let port = n.parse::<u16>()?;
|
||||
if port == 0 {
|
||||
return Err("port must not be zero".into());
|
||||
}
|
||||
Ok((p.parse()?, port))
|
||||
}
|
||||
fn backend(n: &NetworkReport, s: &str) -> Result<Backend> {
|
||||
let (node, port) = s.rsplit_once(':').ok_or("backend must be NODE:PORT")?;
|
||||
let peer = match ipc::resolve_node(n, node) {
|
||||
Ok(member) => member.endpoint_id.clone(),
|
||||
Err(error) => {
|
||||
let matches = n
|
||||
.members
|
||||
.iter()
|
||||
.any(|m| m.hostname.as_deref() == Some(node) || m.endpoint_id.starts_with(node));
|
||||
if matches {
|
||||
return Err(error.into());
|
||||
}
|
||||
node.parse::<iroh::EndpointId>()
|
||||
.map_err(|_| error)?
|
||||
.to_string()
|
||||
}
|
||||
};
|
||||
let port = port.parse()?;
|
||||
if port == 0 {
|
||||
return Err("backend port must not be zero".into());
|
||||
}
|
||||
Ok(Backend { peer, port })
|
||||
}
|
||||
fn selected(n: &NetworkReport, t: &Target) -> Result<VipStatus> {
|
||||
Ok(ipc::resolve_vip(n, &t.vip)?.clone())
|
||||
}
|
||||
fn listener<'a>(c: &'a mut VipConfig, target: &PortTarget) -> Result<&'a mut Listener> {
|
||||
let (p, n) = parse_listener(&target.listener)?;
|
||||
c.listeners
|
||||
.iter_mut()
|
||||
.find(|l| l.protocol == p && l.port == n)
|
||||
.ok_or_else(|| format!("no listener {} on {}", target.listener, c.name).into())
|
||||
}
|
||||
fn name_of<'a>(n: &'a NetworkReport, peer: &'a str) -> &'a str {
|
||||
n.members
|
||||
.iter()
|
||||
.find(|m| m.endpoint_id == peer)
|
||||
.and_then(|m| m.hostname.as_deref())
|
||||
.unwrap_or(peer)
|
||||
}
|
||||
fn show(n: &NetworkReport, v: &VipStatus, json: bool) -> Result<()> {
|
||||
if json {
|
||||
println!("{}", serde_json::to_string(v)?);
|
||||
return Ok(());
|
||||
}
|
||||
println!(
|
||||
"{}.{} {}",
|
||||
v.config.name,
|
||||
n.name,
|
||||
v.error.as_deref().unwrap_or("configured")
|
||||
);
|
||||
for l in &v.listeners {
|
||||
let choice = l
|
||||
.selected
|
||||
.as_ref()
|
||||
.map(|b| format!("{}:{}", name_of(n, &b.peer), b.port))
|
||||
.unwrap_or_else(|| "none".into());
|
||||
println!(
|
||||
" {}/{} next: {} flows: {} tx/rx: {}/{} bytes drops: {} UDP failovers: {}",
|
||||
l.protocol,
|
||||
l.port,
|
||||
choice,
|
||||
l.backends.iter().map(|b| b.active_flows).sum::<u32>(),
|
||||
l.counters.tx_bytes,
|
||||
l.counters.rx_bytes,
|
||||
l.counters.dropped,
|
||||
l.counters.failovers
|
||||
);
|
||||
for b in &l.backends {
|
||||
let rtt = b
|
||||
.rtt_us
|
||||
.map(|n| format!("{:.2} ms", n as f64 / 1000.))
|
||||
.unwrap_or_else(|| "unknown".into());
|
||||
println!(
|
||||
" {}:{} {} RTT {} active {}",
|
||||
name_of(n, &b.backend.peer),
|
||||
b.backend.port,
|
||||
b.state,
|
||||
rtt,
|
||||
b.active_flows
|
||||
);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
pub(crate) async fn run(args: VipArgs) -> Result<()> {
|
||||
let custom = args.paths.state_dir.is_some();
|
||||
let paths = args.paths.resolve()?;
|
||||
let socket = control_socket(&paths, args.control_socket.as_ref(), custom);
|
||||
let report = ipc::request_status(&socket).await?;
|
||||
if let Action::List { network: None } = &args.action {
|
||||
for n in &report.networks {
|
||||
for v in &n.vips {
|
||||
show(n, v, args.json)?;
|
||||
}
|
||||
}
|
||||
return Ok(());
|
||||
}
|
||||
let selector = args.action.network().ok_or("network is required")?;
|
||||
let names: Vec<_> = report
|
||||
.networks
|
||||
.iter()
|
||||
.map(|n| (n.network_id.clone(), n.name.clone()))
|
||||
.collect();
|
||||
let network = &report.networks[resolve_network_index(&names, selector)?];
|
||||
if let Action::Status {
|
||||
target,
|
||||
watch: true,
|
||||
} = &args.action
|
||||
{
|
||||
let mut previous: Option<(std::time::Instant, u64, u64)> = None;
|
||||
loop {
|
||||
let report = ipc::request_status(&socket).await?;
|
||||
let n = report
|
||||
.networks
|
||||
.iter()
|
||||
.find(|n| n.network_id == network.network_id)
|
||||
.ok_or("network was removed")?;
|
||||
let vip = selected(n, target)?;
|
||||
let (tx, rx) = vip.listeners.iter().fold((0u64, 0u64), |a, l| {
|
||||
(
|
||||
a.0.saturating_add(l.counters.tx_bytes),
|
||||
a.1.saturating_add(l.counters.rx_bytes),
|
||||
)
|
||||
});
|
||||
let now = std::time::Instant::now();
|
||||
if let Some((at, old_tx, old_rx)) = previous
|
||||
&& !args.json
|
||||
{
|
||||
let secs = now.duration_since(at).as_secs_f64();
|
||||
if secs > 0. {
|
||||
println!(
|
||||
"TX {:.1} KiB/s · RX {:.1} KiB/s",
|
||||
tx.saturating_sub(old_tx) as f64 / secs / 1024.,
|
||||
rx.saturating_sub(old_rx) as f64 / secs / 1024.
|
||||
);
|
||||
}
|
||||
}
|
||||
previous = Some((now, tx, rx));
|
||||
show(n, &vip, args.json)?;
|
||||
tokio::select! {_=tokio::signal::ctrl_c()=>break,_=tokio::time::sleep(std::time::Duration::from_secs(1))=>{}}
|
||||
}
|
||||
return Ok(());
|
||||
}
|
||||
let mut edited_file = None;
|
||||
let (mut config, expected, delete) = match &args.action {
|
||||
Action::List { .. } => {
|
||||
for v in &network.vips {
|
||||
show(network, v, args.json)?;
|
||||
}
|
||||
return Ok(());
|
||||
}
|
||||
Action::Show(t) | Action::Status { target: t, .. } => {
|
||||
let v = selected(network, t)?;
|
||||
show(network, &v, args.json)?;
|
||||
if matches!(args.action, Action::Show(_)) && !args.json {
|
||||
println!("{}", serde_json::to_string_pretty(&v.config)?);
|
||||
}
|
||||
return Ok(());
|
||||
}
|
||||
Action::Create { name, .. } => (VipConfig::new(name), None, false),
|
||||
Action::Remove(t) => {
|
||||
let v = selected(network, t)?;
|
||||
(v.config, Some(v.revision), true)
|
||||
}
|
||||
Action::Edit(t) => {
|
||||
let v = selected(network, t)?;
|
||||
let mut file = tempfile::Builder::new()
|
||||
.prefix("tsunagi-vip-")
|
||||
.suffix(".json")
|
||||
.tempfile()?;
|
||||
use std::io::Write;
|
||||
file.write_all(serde_json::to_string_pretty(&v.config)?.as_bytes())?;
|
||||
file.flush()?;
|
||||
let editor = std::env::var("VISUAL")
|
||||
.or_else(|_| std::env::var("EDITOR"))
|
||||
.unwrap_or_else(|_| "vi".into());
|
||||
let mut words = editor.split_whitespace();
|
||||
let program = words.next().ok_or("empty EDITOR")?;
|
||||
let status = std::process::Command::new(program)
|
||||
.args(words)
|
||||
.arg(file.path())
|
||||
.status()?;
|
||||
if !status.success() {
|
||||
return Err("editor failed; VIP unchanged".into());
|
||||
}
|
||||
let bytes = std::fs::read(file.path())?;
|
||||
let cfg: VipConfig = match serde_json::from_slice(&bytes) {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
let (_, path) = file.keep()?;
|
||||
return Err(
|
||||
format!("invalid draft: {e}; retained at {}", path.display()).into(),
|
||||
);
|
||||
}
|
||||
};
|
||||
if cfg.id != v.config.id {
|
||||
return Err("VIP id cannot be edited".into());
|
||||
}
|
||||
edited_file = Some(file);
|
||||
(cfg, Some(v.revision), false)
|
||||
}
|
||||
Action::Port { action } => {
|
||||
let target = match action {
|
||||
PortAction::Add { target, .. }
|
||||
| PortAction::Set { target, .. }
|
||||
| PortAction::Remove(target) => target,
|
||||
};
|
||||
let v = selected(network, &target.target)?;
|
||||
let mut cfg = v.config;
|
||||
match action {
|
||||
PortAction::Add {
|
||||
target,
|
||||
backend: destinations,
|
||||
} => {
|
||||
let (p, port) = parse_listener(&target.listener)?;
|
||||
let mut l = Listener::new(p, port);
|
||||
l.backends = destinations
|
||||
.iter()
|
||||
.map(|b| backend(network, b))
|
||||
.collect::<Result<_>>()?;
|
||||
cfg.listeners.push(l);
|
||||
}
|
||||
PortAction::Remove(target) => {
|
||||
let (p, port) = parse_listener(&target.listener)?;
|
||||
listener(&mut cfg, target)?;
|
||||
cfg.listeners.retain(|l| l.protocol != p || l.port != port);
|
||||
}
|
||||
PortAction::Set {
|
||||
target,
|
||||
port_check,
|
||||
check_interval_ms,
|
||||
check_timeout_ms,
|
||||
failures,
|
||||
successes,
|
||||
idle_timeout,
|
||||
} => {
|
||||
let l = listener(&mut cfg, target)?;
|
||||
if let Some(v) = port_check {
|
||||
l.port_check = v == "on";
|
||||
}
|
||||
if let Some(v) = check_interval_ms {
|
||||
l.check_interval_ms = *v;
|
||||
}
|
||||
if let Some(v) = check_timeout_ms {
|
||||
l.check_timeout_ms = *v;
|
||||
}
|
||||
if let Some(v) = failures {
|
||||
l.failure_threshold = *v;
|
||||
}
|
||||
if let Some(v) = successes {
|
||||
l.recovery_threshold = *v;
|
||||
}
|
||||
if let Some(v) = idle_timeout {
|
||||
l.idle_timeout_secs = *v;
|
||||
}
|
||||
}
|
||||
}
|
||||
(cfg, Some(v.revision), false)
|
||||
}
|
||||
Action::Backend { action } => {
|
||||
let (target, destination, remove) = match action {
|
||||
BackendAction::Add { target, backend } => (target, backend, false),
|
||||
BackendAction::Remove { target, backend } => (target, backend, true),
|
||||
};
|
||||
let v = selected(network, &target.target)?;
|
||||
let mut cfg = v.config;
|
||||
let b = backend(network, destination)?;
|
||||
let l = listener(&mut cfg, target)?;
|
||||
if remove {
|
||||
if !l.backends.contains(&b) {
|
||||
return Err("backend is not configured".into());
|
||||
}
|
||||
l.backends.retain(|x| x != &b);
|
||||
} else {
|
||||
l.backends.push(b);
|
||||
}
|
||||
(cfg, Some(v.revision), false)
|
||||
}
|
||||
};
|
||||
// Canonical order makes displayed drafts and shared state deterministic.
|
||||
config.listeners.sort_by_key(|l| (l.protocol, l.port));
|
||||
for l in &mut config.listeners {
|
||||
l.backends.sort();
|
||||
}
|
||||
let id = config.id.clone();
|
||||
let vips = match ipc::save_vip(&socket, &network.network_id, config, expected, delete).await {
|
||||
Ok(vips) => vips,
|
||||
Err(error) => {
|
||||
if let Some(file) = edited_file {
|
||||
let (_, path) = file.keep()?;
|
||||
return Err(format!("{error}; draft retained at {}", path.display()).into());
|
||||
}
|
||||
return Err(error.into());
|
||||
}
|
||||
};
|
||||
if delete {
|
||||
if args.json {
|
||||
println!("{{\"removed\":\"{id}\"}}");
|
||||
} else {
|
||||
println!("VIP removed; existing flows drain.");
|
||||
}
|
||||
} else if let Some(v) = vips.iter().find(|v| v.config.id == id) {
|
||||
show(network, v, args.json)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use clap::Parser;
|
||||
#[test]
|
||||
fn vip_commands_parse_complete_listener_and_settings() {
|
||||
let cli = super::super::Cli::try_parse_from([
|
||||
"tsng",
|
||||
"vip",
|
||||
"port",
|
||||
"add",
|
||||
"lab",
|
||||
"k8s",
|
||||
"tcp/443",
|
||||
"--backend",
|
||||
"cp1:6443",
|
||||
"--backend",
|
||||
"cp2:7443",
|
||||
]);
|
||||
assert!(cli.is_ok(), "{cli:?}");
|
||||
assert!(
|
||||
super::super::Cli::try_parse_from([
|
||||
"tsng",
|
||||
"vip",
|
||||
"port",
|
||||
"set",
|
||||
"lab",
|
||||
"k8s",
|
||||
"tcp/443",
|
||||
"--port-check",
|
||||
"off",
|
||||
"--failures",
|
||||
"3"
|
||||
])
|
||||
.is_ok()
|
||||
);
|
||||
assert!(parse_listener("tcp/0").is_err());
|
||||
assert!(parse_listener("sctp/53").is_err());
|
||||
}
|
||||
}
|
||||
@@ -345,3 +345,95 @@ fn opposing_dns_flags_are_rejected() {
|
||||
assert!(!result.status.success());
|
||||
assert!(String::from_utf8_lossy(&result.stderr).contains("cannot be used with"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn vip_cli_creates_multiple_ports_updates_dns_and_rejects_partial_invalid_changes() {
|
||||
let agent = start("vip-test", 15367);
|
||||
let created = agent.run(&["vip", "create", "vip-test", "api", "--json"]);
|
||||
assert!(
|
||||
created.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&created.stderr)
|
||||
);
|
||||
let vip: tsunagi::vip::VipStatus = serde_json::from_slice(&created.stdout).unwrap();
|
||||
assert!(vip.config.listeners.is_empty());
|
||||
let reply = wait_for_answer("127.0.0.1:15367".parse().unwrap(), "api.vip-test");
|
||||
assert!(has_answer(&reply));
|
||||
let socket = tsunagi::ipc::control_socket_path(&agent.dir.path().join("state"));
|
||||
let runtime = tokio::runtime::Builder::new_current_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
.unwrap();
|
||||
let status = runtime
|
||||
.block_on(tsunagi::ipc::request_status(socket))
|
||||
.unwrap();
|
||||
let backend = format!("{}:6443", status.endpoint_id);
|
||||
let added = agent.run(&[
|
||||
"vip",
|
||||
"port",
|
||||
"add",
|
||||
"vip-test",
|
||||
"api",
|
||||
"tcp/443",
|
||||
"--backend",
|
||||
&backend,
|
||||
]);
|
||||
assert!(
|
||||
added.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&added.stderr)
|
||||
);
|
||||
assert!(
|
||||
agent
|
||||
.run(&["vip", "port", "add", "vip-test", "api", "udp/53"])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
assert!(
|
||||
!agent
|
||||
.run(&["vip", "port", "add", "vip-test", "api", "tcp/443"])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
assert!(
|
||||
agent
|
||||
.run(&[
|
||||
"vip",
|
||||
"port",
|
||||
"set",
|
||||
"vip-test",
|
||||
"api",
|
||||
"tcp/443",
|
||||
"--port-check",
|
||||
"off"
|
||||
])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
let shown = agent.run(&["vip", "show", "vip-test", "api", "--json"]);
|
||||
assert!(shown.status.success());
|
||||
let shown: tsunagi::vip::VipStatus = serde_json::from_slice(&shown.stdout).unwrap();
|
||||
assert_eq!(shown.config.listeners.len(), 2);
|
||||
assert!(
|
||||
!shown
|
||||
.config
|
||||
.listeners
|
||||
.iter()
|
||||
.find(|l| l.protocol == tsunagi::vip::Protocol::Tcp)
|
||||
.unwrap()
|
||||
.port_check
|
||||
);
|
||||
assert_eq!(shown.config.listeners[0].backends[0].port, 6443);
|
||||
let removed = agent.run(&["vip", "remove", "vip-test", "api"]);
|
||||
assert!(removed.status.success());
|
||||
let deadline = Instant::now() + Duration::from_secs(10);
|
||||
loop {
|
||||
if let Some(reply) = query("127.0.0.1:15367".parse().unwrap(), "api.vip-test", TYPE::A)
|
||||
&& !has_answer(&reply)
|
||||
{
|
||||
break;
|
||||
}
|
||||
assert!(Instant::now() < deadline);
|
||||
std::thread::sleep(Duration::from_millis(25));
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user