Files
Aleksandr Bogomiakov 2b0c938487 Bound iroh path retries to prevent multi-connection OOM
A long-running rc.33 daemon reached OOM with roughly 22 GiB resident and
24 GiB swapped. Live inspection found 40 GiB and 10/20 GiB anonymous
container mappings; the active worker stack and tracing metadata identified
iroh's scheduled path-open retry loop. In iroh 1.2.0 every failed connection
requeues the same path, while each retry visits every connection again.
Separate control and data connections multiply the queue geometrically.

Vendor the published iroh 1.2.0 source and select it with a workspace Cargo
patch. Deduplicate retry admission and cap distinct pending paths at 64,
evicting the oldest before inserting a new candidate. Keep the current and
next retry batches bounded without combining control/data connections or
changing wire protocols, identity derivation, authentication or state.
Document source provenance, the live evidence and removal conditions.
Upstream PR 4551 merged on October 6, but inspection of the official 1.3.0
crate still shows the unconditional enqueue, so that upgrade alone is not
a fix. No version bump, release tag or installed-service change.

The default workspace suite includes the exact vendored admission module.
Three regressions failed with the original enqueue and pass with the fix:
repeated failures across connections, duplicate admission at capacity, and
10,000 distinct candidates with bounded length/capacity and FIFO eviction.
Transport integration tests continue using real endpoints and encryption.

Checks on Linux in nix develop:
- cargo fmt --all -- --check
- cargo clippy --locked --workspace --all-targets -- -D warnings
- cargo test --locked --workspace --all-targets, with only the inherited
  TSUNAGI_CONTROL_SOCKET removed from the test environment
- git diff --cached --check

Initial full-suite runs failed six DNS readiness checks because the inherited
system socket path overrode their temporary state directories. Clearing that
environment variable resolves those failures without changing application code.
Root/network-dependent ignored tests and other operating systems were not run.
2026-10-14 23:28:53 +01:00

6.9 KiB

iroh

less net work for networks

Documentation Crates.io Chat Youtube License: MIT License: Apache 2.0


Iroh is a Rust library to establish direct connections between endpoints. It gives you an API for dialing by public key. You say "connect to that endpoint", and iroh finds and maintains the best connection for you.

Under the hood iroh establishes peer-to-peer QUIC connections between endpoints. The fastest route is a direct connection, so iroh tries to hole-punch one whenever it can. If that fails it falls back to using relay servers.

Because iroh is built on QUIC, all connections are end-to-end encrypted and may carry any number of concurrent streams. Dialing by public key also makes them mutually authenticated, because each endpoint's public key is its TLS identity.

Overview

An iroh endpoint is created and controlled by the Endpoint. Each endpoint has a unique SecretKey, whose public key is the endpoint's identity, the EndpointId. Connections are authenticated against this key, which means an EndpointId can't be impersonated.

A connection is usually established with the help of a relay server. When an endpoint is created it connects to the closest relay and designates it as its home relay. Other endpoints reach it first through this relay, then both sides use QUIC NAT traversal to establish a direct connection. In the rare cases where a direct connection is not possible, traffic keeps flowing over the relay.

Relay servers only forward encrypted packets addressed to Endpoint IDs, they cannot read any traffic between endpoints.

Endpoints can also connect directly without a relay, as long as the accepting endpoint is directly reachable at one of its addresses.

To discover addressing information for an endpoint, iroh uses address lookup services. With address lookup, you can connect to other endpoints with only their EndpointId. Addressing information will then be resolved on-demand.

The N0 preset installs the DNS/Pkarr address lookup service, which uses servers hosted by n0 to provide global lookup for endpoints.

Example

This is an echo protocol: the accepting side copies back whatever it receives. The full, commented version is in echo.rs.

use iroh::{
    Endpoint,
    endpoint::{Connection, presets},
    protocol::{AcceptError, ProtocolHandler, Router},
};

/// Each protocol is identified by its ALPN, exchanged during the handshake.
const ALPN: &[u8] = b"iroh-example/echo/0";

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // The accepting side: bind an endpoint and route the ALPN to a handler.
    let endpoint = Endpoint::bind(presets::N0).await?;
    let router = Router::builder(endpoint.clone()).accept(ALPN, Echo).spawn();
    endpoint.online().await;
    // Get the endpoint's address so that we can connect to it.
    let addr = endpoint.addr();

    // The connecting side: dial the accepting endpoint by its address.
    {
        let other_endpoint = Endpoint::bind(presets::N0).await?;

        let conn = other_endpoint.connect(addr, ALPN).await?;
        let (mut send, mut recv) = conn.open_bi().await?;
        send.write_all(b"Hello, world!").await?;
        send.finish()?;
        let response = recv.read_to_end(1000).await?;
        assert_eq!(&response, b"Hello, world!");
        conn.close(0u32.into(), b"bye!");

        other_endpoint.close().await;
    }

    router.shutdown().await?;
    Ok(())
}

#[derive(Debug, Clone)]
struct Echo;

impl ProtocolHandler for Echo {
    async fn accept(&self, connection: Connection) -> Result<(), AcceptError> {
        let (mut send, mut recv) = connection.accept_bi().await?;
        // Echo bytes back until the sender signals the end of data.
        tokio::io::copy(&mut recv, &mut send).await?;
        send.finish()?;
        // Wait until the other endpoint closes the connection.
        connection.closed().await;
        Ok(())
    }
}

More examples live in iroh/examples. Run them with cargo run --example NAME. Details for each are in the file itself.

Compose protocols

Instead of writing your own, you can build on protocols that already exist on top of iroh:

  • iroh-blobs for BLAKE3-based content-addressed blob transfer, scaling from kilobytes to terabytes.
  • iroh-gossip for publish-subscribe overlay networks that scale down to what an average phone can handle.
  • and many more.

To use iroh from other languages, see iroh-ffi.

Development

For notes on iroh's structured events and how to build the documentation, see DEVELOPMENT.md.

License

This project is licensed under either of

at your option.

Contribution

See CONTRIBUTING.md for how to get involved.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.