A long-running rc.33 daemon reached OOM with roughly 22 GiB resident and 24 GiB swapped. Live inspection found 40 GiB and 10/20 GiB anonymous container mappings; the active worker stack and tracing metadata identified iroh's scheduled path-open retry loop. In iroh 1.2.0 every failed connection requeues the same path, while each retry visits every connection again. Separate control and data connections multiply the queue geometrically. Vendor the published iroh 1.2.0 source and select it with a workspace Cargo patch. Deduplicate retry admission and cap distinct pending paths at 64, evicting the oldest before inserting a new candidate. Keep the current and next retry batches bounded without combining control/data connections or changing wire protocols, identity derivation, authentication or state. Document source provenance, the live evidence and removal conditions. Upstream PR 4551 merged on October 6, but inspection of the official 1.3.0 crate still shows the unconditional enqueue, so that upgrade alone is not a fix. No version bump, release tag or installed-service change. The default workspace suite includes the exact vendored admission module. Three regressions failed with the original enqueue and pass with the fix: repeated failures across connections, duplicate admission at capacity, and 10,000 distinct candidates with bounded length/capacity and FIFO eviction. Transport integration tests continue using real endpoints and encryption. Checks on Linux in nix develop: - cargo fmt --all -- --check - cargo clippy --locked --workspace --all-targets -- -D warnings - cargo test --locked --workspace --all-targets, with only the inherited TSUNAGI_CONTROL_SOCKET removed from the test environment - git diff --cached --check Initial full-suite runs failed six DNS readiness checks because the inherited system socket path overrode their temporary state directories. Clearing that environment variable resolves those failures without changing application code. Root/network-dependent ignored tests and other operating systems were not run.
127 lines
5.5 KiB
Rust
127 lines
5.5 KiB
Rust
use iroh_metrics::{Counter, MetricsGroup};
|
|
use serde::{Deserialize, Serialize};
|
|
|
|
/// Metrics collected by the iroh socket.
|
|
#[derive(Debug, Serialize, Deserialize, MetricsGroup)]
|
|
#[non_exhaustive]
|
|
#[metrics(name = "socket", default)]
|
|
pub struct Metrics {
|
|
/// Intended to count updates to the local direct address set, but currently unused
|
|
/// (never incremented).
|
|
pub update_direct_addrs: Counter,
|
|
|
|
/// Number of bytes sent over IPv4.
|
|
pub send_ipv4: Counter,
|
|
/// Number of bytes sent over IPv6.
|
|
pub send_ipv6: Counter,
|
|
/// Number of bytes sent over the relay transport.
|
|
pub send_relay: Counter,
|
|
|
|
/// Number of data bytes received over the relay transport.
|
|
pub recv_data_relay: Counter,
|
|
/// Number of data bytes received over any custom transport.
|
|
pub recv_data_custom: Counter,
|
|
/// Number of data bytes received over IPv4.
|
|
pub recv_data_ipv4: Counter,
|
|
/// Number of data bytes received over IPv6.
|
|
pub recv_data_ipv6: Counter,
|
|
/// Number of QUIC datagrams received.
|
|
pub recv_datagrams: Counter,
|
|
/// Number of receive events that used GRO (coalesced datagram batches).
|
|
///
|
|
/// This counts batches, not the individual datagrams within them. See
|
|
/// [`Self::recv_datagrams`] for the datagram count.
|
|
pub recv_gro_datagrams: Counter,
|
|
|
|
/// Number of times the home relay changed to a different relay.
|
|
///
|
|
/// This includes the initial assignment from no home relay to a home relay.
|
|
pub relay_home_change: Counter,
|
|
|
|
/// Number of connections to a relay server that were established.
|
|
///
|
|
/// Failed dial attempts are not counted. A relay that is reconnected to is counted
|
|
/// once per connection, so this grows on every reconnect.
|
|
pub relay_conns_success: Counter,
|
|
/// Number of failed attempts to connect to a relay server.
|
|
///
|
|
/// Each attempt is counted, so an unreachable relay increments this on every retry.
|
|
pub relay_conns_failed: Counter,
|
|
/// Number of connections to a relay server that ended.
|
|
///
|
|
/// Paired with [`Self::relay_conns_success`], so the number of current connections is
|
|
/// `relay_conns_success` - `relay_conns_closed`.
|
|
pub relay_conns_closed: Counter,
|
|
/// Number of relay connections on which the relay reported rate limiting.
|
|
///
|
|
/// This is incremented at most once per connection, so it counts the affected
|
|
/// connections and not how often the relay reported the problem. Relate it to
|
|
/// [`Self::relay_conns_success`] to see how many connections were affected.
|
|
pub relay_conns_ratelimited: Counter,
|
|
|
|
/*
|
|
* Holepunching metrics
|
|
*/
|
|
/// The number of times holepunching is initiated on a connection.
|
|
///
|
|
/// This can be incremented multiple times for a single connection. Note that only the
|
|
/// client-side of a connection will increment this counter.
|
|
pub holepunch_attempts: Counter,
|
|
/// The number of network paths to peers that are direct.
|
|
///
|
|
/// This can be incremented multiple times for a single connection.
|
|
pub paths_direct: Counter,
|
|
/// The number of network paths to peers that are relayed.
|
|
///
|
|
/// This would typically only be incremented once for a single connection.
|
|
pub paths_relay: Counter,
|
|
/// The number of network paths to peers that are user defined.
|
|
///
|
|
/// This would typically only be incremented once for a single connection.
|
|
pub paths_custom: Counter,
|
|
/// The number of connections that have been direct connections.
|
|
///
|
|
/// This is only incremented once for each opened connection. See `num_conns_opened` for
|
|
/// the number of opened connections.
|
|
pub num_conns_direct: Counter,
|
|
|
|
/*
|
|
* Connection Metrics
|
|
*
|
|
* These all only count connections that completed the TLS handshake successfully. This means
|
|
* that short lived 0RTT connections are potentially not included in these counts.
|
|
*/
|
|
/// Number of connections opened (only handshaked connections are counted).
|
|
pub num_conns_opened: Counter,
|
|
/// Number of connections closed (only handshaked connections are counted).
|
|
pub num_conns_closed: Counter,
|
|
|
|
/// Number of IP transport paths opened.
|
|
pub transport_ip_paths_added: Counter,
|
|
/// Number of IP transport paths closed.
|
|
pub transport_ip_paths_removed: Counter,
|
|
/// Number of relay transport paths opened.
|
|
pub transport_relay_paths_added: Counter,
|
|
/// Number of relay transport paths closed.
|
|
pub transport_relay_paths_removed: Counter,
|
|
/// Number of custom transport paths opened.
|
|
pub transport_custom_paths_added: Counter,
|
|
/// Number of custom transport paths closed.
|
|
pub transport_custom_paths_removed: Counter,
|
|
|
|
/// Number of iterations of the main socket actor loop.
|
|
pub actor_tick_main: Counter,
|
|
/// Number of actor messages processed by the socket actor loop.
|
|
pub actor_tick_msg: Counter,
|
|
/// Number of periodic re-STUN timer ticks handled by the socket actor loop.
|
|
pub actor_tick_re_stun: Counter,
|
|
/// Number of port-mapping change events handled by the socket actor loop.
|
|
pub actor_tick_portmap_changed: Counter,
|
|
/// Intended to count direct address heartbeat ticks, but currently unused (never incremented).
|
|
pub actor_tick_direct_addr_heartbeat: Counter,
|
|
/// Number of local network interface (link) change events handled by the socket actor loop.
|
|
pub actor_link_change: Counter,
|
|
/// Number of times an input watcher or receiver closed in the socket actor loop.
|
|
pub actor_tick_other: Counter,
|
|
}
|