Bound iroh path retries to prevent multi-connection OOM

A long-running rc.33 daemon reached OOM with roughly 22 GiB resident and
24 GiB swapped. Live inspection found 40 GiB and 10/20 GiB anonymous
container mappings; the active worker stack and tracing metadata identified
iroh's scheduled path-open retry loop. In iroh 1.2.0 every failed connection
requeues the same path, while each retry visits every connection again.
Separate control and data connections multiply the queue geometrically.

Vendor the published iroh 1.2.0 source and select it with a workspace Cargo
patch. Deduplicate retry admission and cap distinct pending paths at 64,
evicting the oldest before inserting a new candidate. Keep the current and
next retry batches bounded without combining control/data connections or
changing wire protocols, identity derivation, authentication or state.
Document source provenance, the live evidence and removal conditions.
Upstream PR 4551 merged on October 6, but inspection of the official 1.3.0
crate still shows the unconditional enqueue, so that upgrade alone is not
a fix. No version bump, release tag or installed-service change.

The default workspace suite includes the exact vendored admission module.
Three regressions failed with the original enqueue and pass with the fix:
repeated failures across connections, duplicate admission at capacity, and
10,000 distinct candidates with bounded length/capacity and FIFO eviction.
Transport integration tests continue using real endpoints and encryption.

Checks on Linux in nix develop:
- cargo fmt --all -- --check
- cargo clippy --locked --workspace --all-targets -- -D warnings
- cargo test --locked --workspace --all-targets, with only the inherited
  TSUNAGI_CONTROL_SOCKET removed from the test environment
- git diff --cached --check

Initial full-suite runs failed six DNS readiness checks because the inherited
system socket path overrode their temporary state directories. Clearing that
environment variable resolves those failures without changing application code.
Root/network-dependent ignored tests and other operating systems were not run.
This commit is contained in:
Aleksandr Bogomiakov committed 2026-10-14 23:28:53 +01:00
1 parent 4e235cbac6
commit 2b0c938487
86 files changed
+35810 -3

No files matched your search

+125
View File
@@ -0,0 +1,125 @@
use ctutils::CtEq;
use noq_proto::crypto;
use rand::RngExt;
use rustls::crypto::cipher::{
AeadKey, InboundOpaqueMessage, Iv, NONCE_LEN, OutboundPlainMessage, Tls13AeadAlgorithm,
};
/// Implements [`crypto::HandshakeTokenKey`] using a [`Tls13AeadAlgorithm`].
///
/// This can be obtained from looking through available ciphers from a
/// [`rustls::crypto::CryptoProvider`].
pub(crate) struct RustlsTokenKey {
key: [u8; 32],
aead: &'static dyn Tls13AeadAlgorithm,
}
impl RustlsTokenKey {
/// Constructs [`crypto::HandshakeTokenKey`] from a [`rustls::crypto::CryptoProvider`].
///
/// Tries to find a suitable TLS 1.3 cipher suite from the provided crypto provider,
/// then uses it to extract an AEAD to use as the token key encryption method.
///
/// Then generates a random master key to use.
///
/// Returns `None` when this can't find a suitable TLS cipher suite in the given crypto
/// provider.
pub(crate) fn new(
rng: &mut impl rand::CryptoRng,
crypto_provider: &rustls::crypto::CryptoProvider,
) -> Option<Self> {
let suite = crypto_provider
.cipher_suites
.iter()
.filter_map(|suite| suite.tls13())
.next()?;
let aead = suite.aead_alg;
Some(Self {
key: rng.random(),
aead,
})
}
}
impl crypto::HandshakeTokenKey for RustlsTokenKey {
fn seal(&self, token_nonce: u128, data: &mut Vec<u8>) -> Result<(), crypto::CryptoError> {
let key = AeadKey::from(self.key);
let nonce: [u8; NONCE_LEN] = *token_nonce
.to_le_bytes()
.first_chunk()
.expect("expected u128 > 96 bit");
let iv = Iv::from(nonce);
let msg = OutboundPlainMessage {
typ: rustls::ContentType::ApplicationData,
version: rustls::ProtocolVersion::TLSv1_3,
payload: rustls::crypto::cipher::OutboundChunks::Single(&*data),
};
let out = self
.aead
.encrypter(key, iv)
.encrypt(msg, 0)
.map_err(|_| crypto::CryptoError)?;
data.clear();
data.extend(out.payload.as_ref());
Ok(())
}
fn open<'a>(
&self,
token_nonce: u128,
data: &'a mut [u8],
) -> Result<&'a [u8], crypto::CryptoError> {
let key = AeadKey::from(self.key);
let nonce: [u8; NONCE_LEN] = *token_nonce
.to_le_bytes()
.first_chunk()
.expect("expected u128 > 96 bit");
let iv = Iv::from(nonce);
let msg = InboundOpaqueMessage::new(
rustls::ContentType::ApplicationData,
rustls::ProtocolVersion::TLSv1_3,
data,
);
let plain = self
.aead
.decrypter(key, iv)
.decrypt(msg, 0)
.map_err(|_| crypto::CryptoError)?;
Ok(plain.payload)
}
}
pub(crate) struct Blake3HmacKey([u8; 32]);
impl Blake3HmacKey {
pub(crate) fn new(rng: &mut impl rand::CryptoRng) -> Self {
let mut key = [0u8; 32];
rng.fill_bytes(&mut key);
Self(key)
}
}
impl crypto::HmacKey for Blake3HmacKey {
fn sign(&self, data: &[u8], signature_out: &mut [u8]) {
signature_out.copy_from_slice(blake3::keyed_hash(&self.0, data).as_slice());
}
fn signature_len(&self) -> usize {
blake3::OUT_LEN // 32 bytes
}
fn verify(&self, data: &[u8], signature: &[u8]) -> Result<(), crypto::CryptoError> {
let reference = blake3::keyed_hash(&self.0, data);
// to_bool is fine here, because it's the last thing we do to
// distinguish success or failure (see to_bool documentation)
if signature.ct_eq(reference.as_slice()).to_bool() {
Ok(())
} else {
Err(crypto::CryptoError)
}
}
}
+67
View File
@@ -0,0 +1,67 @@
//! Implementation of encoding iroh EndpointIds as domain names.
//!
//! We used to send this name via SNI up until iroh version 1.0.3, but stopped doing so in
//! versions after that. It is now used only locally by the `ServerCertificateVerifier` and
//! to separate the buckets for 0-RTT session tickets.
//!
//! We used to use a constant "localhost" for the TLS server name - however, that affects
//! 0-RTT and would put all of the TLS session tickets we receive into the same bucket in
//! the TLS session ticket cache.
//! So we choose something that'd dependent on the EndpointId.
//! We cannot use hex to encode the EndpointId, as that'd encode to 64 characters, but we only
//! have 63 maximum per DNS subdomain. Base32 is the next best alternative.
//! We use the `.invalid` TLD, as that's specified (in RFC 2606) to never actually resolve
//! "for real", unlike `.localhost` which is allowed to resolve to `127.0.0.1`.
//! We also add "iroh" as a subdomain, although those 5 bytes might not be necessary.
//! We *could* decide to remove that indicator in the future likely without breakage.
use data_encoding::BASE32_DNSSEC;
use iroh_base::EndpointId;
pub(crate) fn encode(endpoint_id: EndpointId) -> String {
format!(
"{}.iroh.invalid",
BASE32_DNSSEC.encode(endpoint_id.as_bytes())
)
}
pub(crate) fn decode(name: &str) -> Option<EndpointId> {
let [base32_endpoint_id, "iroh", "invalid"] = name.split(".").collect::<Vec<_>>()[..] else {
return None;
};
EndpointId::from_bytes(
&BASE32_DNSSEC
.decode(base32_endpoint_id.as_bytes())
.ok()?
.try_into()
.ok()?,
)
.ok()
}
#[cfg(test)]
mod tests {
use iroh_base::SecretKey;
use rand::{RngExt, SeedableRng};
#[test]
fn test_roundtrip() {
let mut rng = rand_chacha::ChaCha8Rng::seed_from_u64(0u64);
let key = SecretKey::from_bytes(&rng.random());
let endpoint_id = key.public();
println!("{}", super::encode(endpoint_id));
assert_eq!(
Some(endpoint_id),
super::decode(&super::encode(endpoint_id))
);
}
#[test]
fn test_snapshot() {
let key = SecretKey::from_bytes(&[0; 32]);
assert_eq!(
super::encode(key.public()),
"7dl2ff6emqi2qol3l382krodedij45bn3nh479hqo14a32qpr8kg.iroh.invalid",
);
}
}
+100
View File
@@ -0,0 +1,100 @@
use std::sync::Arc;
use iroh_base::SecretKey;
use webpki_types::CertificateDer;
#[derive(Debug)]
pub(super) struct ResolveRawPublicKeyCert {
key: Arc<rustls::sign::CertifiedKey>,
}
impl ResolveRawPublicKeyCert {
pub(super) fn new(secret_key: &SecretKey) -> Self {
let client_private_key = Arc::new(IrohSecretKey::from(secret_key.clone()));
let client_public_key = client_private_key.spki_public_key();
let client_public_key_as_cert = CertificateDer::from(client_public_key.to_vec());
let certified_key =
rustls::sign::CertifiedKey::new(vec![client_public_key_as_cert], client_private_key);
let key = Arc::new(certified_key);
Self { key }
}
}
impl rustls::client::ResolvesClientCert for ResolveRawPublicKeyCert {
fn resolve(
&self,
_root_hint_subjects: &[&[u8]],
_sigschemes: &[rustls::SignatureScheme],
) -> Option<Arc<rustls::sign::CertifiedKey>> {
Some(Arc::clone(&self.key))
}
fn only_raw_public_keys(&self) -> bool {
true
}
fn has_certs(&self) -> bool {
true
}
}
impl rustls::server::ResolvesServerCert for ResolveRawPublicKeyCert {
fn resolve(
&self,
_client_hello: rustls::server::ClientHello<'_>,
) -> Option<Arc<rustls::sign::CertifiedKey>> {
Some(Arc::clone(&self.key))
}
fn only_raw_public_keys(&self) -> bool {
true
}
}
#[derive(Debug, Clone, derive_more::From)]
struct IrohSecretKey {
#[from]
key: SecretKey,
}
impl IrohSecretKey {
fn spki_public_key(&self) -> webpki_types::SubjectPublicKeyInfoDer<'static> {
rustls::sign::public_key_to_spki(
&webpki_types::alg_id::ED25519,
self.key.public().as_bytes(),
)
}
}
impl rustls::sign::SigningKey for IrohSecretKey {
fn choose_scheme(
&self,
offered: &[rustls::SignatureScheme],
) -> Option<Box<dyn rustls::sign::Signer>> {
if offered.contains(&rustls::SignatureScheme::ED25519) {
Some(Box::new(self.clone()))
} else {
None
}
}
fn algorithm(&self) -> rustls::SignatureAlgorithm {
rustls::SignatureAlgorithm::ED25519
}
fn public_key(&self) -> Option<webpki_types::SubjectPublicKeyInfoDer<'_>> {
Some(self.spki_public_key())
}
}
impl rustls::sign::Signer for IrohSecretKey {
fn sign(&self, message: &[u8]) -> Result<Vec<u8>, rustls::Error> {
Ok(self.key.sign(message).to_bytes().to_vec())
}
fn scheme(&self) -> rustls::SignatureScheme {
rustls::SignatureScheme::ED25519
}
}
+211
View File
@@ -0,0 +1,211 @@
//! TLS 1.3 certificates and handshakes handling.
//!
//! This module handles a verification of a client/server certificate chain
//! and signatures allegedly by the given certificates, or using raw public keys.
use iroh_base::{PublicKey, Signature};
use rustls::{
CertificateError, DigitallySignedStruct, DistinguishedName, SignatureScheme,
SupportedProtocolVersion,
client::danger::{HandshakeSignatureValid, ServerCertVerified, ServerCertVerifier},
crypto::{WebPkiSupportedAlgorithms, verify_tls13_signature_with_raw_key},
pki_types::CertificateDer as Certificate,
server::danger::{ClientCertVerified, ClientCertVerifier},
};
use webpki_types::SubjectPublicKeyInfoDer;
/// The only TLS version we support is 1.3
pub(super) const PROTOCOL_VERSIONS: &[&SupportedProtocolVersion] = &[&rustls::version::TLS13];
const ED25519_DALEK: Ed25519Dalek = Ed25519Dalek;
const SUPPORTED_SIG_ALGS: WebPkiSupportedAlgorithms = WebPkiSupportedAlgorithms {
all: &[&ED25519_DALEK],
mapping: &[(SignatureScheme::ED25519, &[&ED25519_DALEK])],
};
/// Implementation of the `rustls` certificate verification traits
///
/// Only TLS 1.3 is supported. TLS 1.2 should be disabled in the configuration of `rustls`.
#[derive(Default, Debug)]
pub(super) struct ServerCertificateVerifier;
impl ServerCertVerifier for ServerCertificateVerifier {
fn verify_server_cert(
&self,
end_entity: &Certificate,
intermediates: &[Certificate],
server_name: &rustls::pki_types::ServerName,
_ocsp_response: &[u8],
_now: rustls::pki_types::UnixTime,
) -> Result<ServerCertVerified, rustls::Error> {
let rustls::pki_types::ServerName::DnsName(dns_name) = server_name else {
return Err(rustls::Error::UnsupportedNameType);
};
let Some(remote_peer_id) = super::name::decode(dns_name.as_ref()) else {
return Err(rustls::Error::InvalidCertificate(
CertificateError::NotValidForName,
));
};
if !intermediates.is_empty() {
return Err(rustls::Error::InvalidCertificate(
CertificateError::UnknownIssuer,
));
}
let end_entity_as_spki = SubjectPublicKeyInfoDer::from(end_entity.as_ref());
let remote_public_spki = rustls::sign::public_key_to_spki(
&webpki_types::alg_id::ED25519,
remote_peer_id.as_bytes(),
);
// This effectively checks that the `end_entity_as_spki` bytes have the expected
// (constant) 12 byte prefix (consisting of the Ed25519 public key ASN.1 object
// identifier, some ASN.1 DER encoding bytes signaling that this is a SPKI and
// consists of the object identifier and a bit sequence, a zero byte indicating
// that the bit sequence is padded with 0 additional bits) matches, as well as
// the public key bytes match the `remote_peer_id` public key bytes.
if remote_public_spki != end_entity_as_spki {
return Err(rustls::Error::InvalidCertificate(
CertificateError::UnknownIssuer,
));
}
Ok(ServerCertVerified::assertion())
}
fn verify_tls12_signature(
&self,
_message: &[u8],
_cert: &Certificate,
_dss: &DigitallySignedStruct,
) -> Result<HandshakeSignatureValid, rustls::Error> {
Err(rustls::Error::PeerIncompatible(
rustls::PeerIncompatible::Tls12NotOffered,
))
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &Certificate,
dss: &DigitallySignedStruct,
) -> Result<HandshakeSignatureValid, rustls::Error> {
verify_tls13_signature_with_raw_key(
message,
&SubjectPublicKeyInfoDer::from(cert.as_ref()),
dss,
&SUPPORTED_SIG_ALGS,
)
}
fn supported_verify_schemes(&self) -> Vec<SignatureScheme> {
SUPPORTED_SIG_ALGS.supported_schemes()
}
fn requires_raw_public_keys(&self) -> bool {
true
}
}
/// Implementation of the `rustls` certificate verification traits.
///
/// Only TLS 1.3 is supported. TLS 1.2 should be disabled in the configuration of `rustls`.
#[derive(Default, Debug)]
pub(super) struct ClientCertificateVerifier;
/// We require one of the following client certificate configurations:
///
/// - a valid raw public key configuration
impl ClientCertVerifier for ClientCertificateVerifier {
fn offer_client_auth(&self) -> bool {
true
}
fn verify_client_cert(
&self,
_end_entity: &Certificate,
intermediates: &[Certificate],
_now: rustls::pki_types::UnixTime,
) -> Result<ClientCertVerified, rustls::Error> {
if !intermediates.is_empty() {
return Err(rustls::Error::InvalidCertificate(
CertificateError::UnknownIssuer,
));
}
// Beyond checking for no intermediates, we don't check the client certificate.
// The actual signatures are already verified - this ensures authentication.
Ok(ClientCertVerified::assertion())
}
fn verify_tls12_signature(
&self,
_message: &[u8],
_cert: &Certificate,
_dss: &DigitallySignedStruct,
) -> Result<HandshakeSignatureValid, rustls::Error> {
Err(rustls::Error::PeerIncompatible(
rustls::PeerIncompatible::Tls12NotOffered,
))
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &Certificate,
dss: &DigitallySignedStruct,
) -> Result<HandshakeSignatureValid, rustls::Error> {
verify_tls13_signature_with_raw_key(
message,
&SubjectPublicKeyInfoDer::from(cert.as_ref()),
dss,
&SUPPORTED_SIG_ALGS,
)
}
fn supported_verify_schemes(&self) -> Vec<SignatureScheme> {
SUPPORTED_SIG_ALGS.supported_schemes()
}
fn root_hint_subjects(&self) -> &[DistinguishedName] {
&[][..]
}
fn requires_raw_public_keys(&self) -> bool {
true
}
}
#[derive(Debug)]
struct Ed25519Dalek;
impl webpki_types::SignatureVerificationAlgorithm for Ed25519Dalek {
fn verify_signature(
&self,
public_key: &[u8],
message: &[u8],
signature: &[u8],
) -> Result<(), webpki_types::InvalidSignature> {
let public_key =
PublicKey::try_from(public_key).map_err(|_| webpki_types::InvalidSignature)?;
let signature =
Signature::try_from(signature).map_err(|_| webpki_types::InvalidSignature)?;
public_key
.verify(message, &signature)
.map_err(|_| webpki_types::InvalidSignature)
}
fn public_key_alg_id(&self) -> webpki_types::AlgorithmIdentifier {
webpki_types::alg_id::ED25519
}
fn signature_alg_id(&self) -> webpki_types::AlgorithmIdentifier {
webpki_types::alg_id::ED25519
}
fn fips(&self) -> bool {
false
}
}