Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d941cdac7a | ||
|
|
e701986a39 | ||
|
|
ed28e86748 |
No files matched your search
Generated
+21
@@ -6246,6 +6246,7 @@ dependencies = [
|
||||
"tokio",
|
||||
"tracing",
|
||||
"tracing-subscriber",
|
||||
"tsunagi-awg",
|
||||
"tsunagi-core",
|
||||
"tsunagi-tcp-tls",
|
||||
"tsunagi-wg-quic",
|
||||
@@ -6253,6 +6254,26 @@ dependencies = [
|
||||
"windows-service",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tsunagi-awg"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"boringtun",
|
||||
"bytes",
|
||||
"chacha20 0.9.1",
|
||||
"iroh",
|
||||
"postcard",
|
||||
"rand 0.10.3",
|
||||
"serde",
|
||||
"tempfile",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"tsunagi-core",
|
||||
"tsunagi-wg-quic",
|
||||
"tsunagi-wireguard",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tsunagi-core"
|
||||
version = "0.1.0"
|
||||
|
||||
@@ -81,7 +81,17 @@ tsng join -n home -s tsn1… # every other machine
|
||||
tsng status # who is online, over which path
|
||||
```
|
||||
|
||||
Every member gets an address in `10.13.37.0/24` and a name like `<host>.home`.
|
||||
Every new network gets a stable pseudorandom `/24` in `10.0.0.0/8`, derived
|
||||
from its network id. Its members agree on that range even when starting offline,
|
||||
and each gets a name like `<host>.home`. Existing signed ranges are preserved.
|
||||
|
||||
If two configured networks have overlapping ranges, the agent runs only one.
|
||||
`tsng status`, `tsng network` and the tray show which network blocks the
|
||||
other. `tsng network start <name-or-id>` or the tray’s “Switch to this network”
|
||||
button suspends the conflicting networks. They resume automatically when the
|
||||
selected network stops; the selection survives restarting the agent. Explicitly
|
||||
stopping a suspended network cancels its automatic resume.
|
||||
|
||||
Machines are called `tsunagi-<id>` until you rename them: `tsng id hostname laptop`.
|
||||
|
||||
### Exit nodes
|
||||
@@ -97,6 +107,28 @@ The tray does the same with a checkbox. An exit node sees everything you send
|
||||
through it, so use only members you trust.
|
||||
[Details and per-OS notes](docs/exit-node.md).
|
||||
|
||||
The default data transport order is `wg,awg,wg-quic,tcp-tls`. AWG negotiates a
|
||||
fresh masking profile and outer keys for each pair connection attempt, including
|
||||
reentry after fallback. Sustained data loss triggers switching independently of
|
||||
the iroh control connection. [Configuration and limits](docs/amnezia.md).
|
||||
|
||||
Change a network's protocol selection without editing its service or restarting:
|
||||
|
||||
```sh
|
||||
tsng protocols # available protocols and settings
|
||||
tsng protocols home # this device's choice for home
|
||||
tsng protocols home awg,wg-quic # prefer AWG, retain QUIC as fallback
|
||||
tsng protocols home none # control plane only
|
||||
```
|
||||
|
||||
Exact network names, full ids and unique id prefixes work here. The choice is
|
||||
local to this device, saved per network and applied immediately; an offline
|
||||
change applies on the next start. In the tray, expand **protocols** on a network
|
||||
tile to toggle protocols or change their preference order. Both endpoints need
|
||||
a common protocol at the same wire version. `tsng up --protocol` supplies the
|
||||
initial defaults for networks without a saved choice; it does not restrict
|
||||
which built-in protocols can be enabled later.
|
||||
|
||||
## Platforms
|
||||
|
||||
| Platform | Build | CLI `tsng` | Tray | Exit node: offer / use | Packages |
|
||||
@@ -174,6 +206,7 @@ The [threat model](docs/threat-model.md) says what this does and doesn't protect
|
||||
[Architecture](docs/architecture.md) ·
|
||||
[Protocol](docs/protocol.md) ·
|
||||
[WireGuard data plane](docs/wireguard.md) ·
|
||||
[Pairwise Amnezia transport](docs/amnezia.md) ·
|
||||
[Routing](docs/routing.md) ·
|
||||
[Exit nodes](docs/exit-node.md) ·
|
||||
[Threat model](docs/threat-model.md) ·
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
[package]
|
||||
name = "tsunagi-awg"
|
||||
version = "0.1.0"
|
||||
description = "Ephemeral pairwise Amnezia transport for Tsunagi."
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
license = "WTFPL AND Apache-2.0"
|
||||
repository.workspace = true
|
||||
|
||||
[dependencies]
|
||||
tsunagi-core = { path = "../tsunagi-core" }
|
||||
tsunagi-wg-quic = { path = "../tsunagi-wg-quic" }
|
||||
tsunagi-wireguard = { path = "../tsunagi-wireguard" }
|
||||
iroh.workspace = true
|
||||
tokio = { workspace = true, features = ["net", "time", "sync", "macros"] }
|
||||
serde.workspace = true
|
||||
postcard.workspace = true
|
||||
bytes.workspace = true
|
||||
tracing.workspace = true
|
||||
rand = "0.10"
|
||||
zeroize = { version = "1.9", features = ["derive"] }
|
||||
boringtun = { version = "0.7.1", default-features = false }
|
||||
chacha20 = "0.9"
|
||||
|
||||
[dev-dependencies]
|
||||
tsunagi-core = { path = "../tsunagi-core", features = ["testing"] }
|
||||
tempfile.workspace = true
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright 2026 Friderika Coulson (NanoBee)
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Amnezia packet engine
|
||||
|
||||
`src/masking.rs` is adapted from `src/core/amnezia.rs` in
|
||||
https://github.com/RunaRoo/AmneziaWG-RS at
|
||||
`51579a7d6e8882d1d8d6d00822f09e18a9423b73`, under Apache-2.0.
|
||||
The upstream license is retained in `LICENSE-AmneziaWG-RS`.
|
||||
|
||||
The adaptation replaces configuration-file loading with bounded typed pair
|
||||
profiles, updates RNG calls to rand 0.10, removes an unnecessary unwrap, and
|
||||
does not embed upstream's Node, global executors, OS interface operations or
|
||||
detached tasks. Noise/WireGuard is supplied by boringtun, already used by
|
||||
Tsunagi. Each pair owns one socket and one cancellable packet-engine task.
|
||||
|
||||
This is Tsunagi AWG wire version 1, not a claim of compatibility with external
|
||||
Amnezia servers. The masking engine substitutes message headers outside the
|
||||
standard WireGuard state machine and restores them before Noise validation.
|
||||
There are no remote file paths, shell hooks, or persisted pair profiles.
|
||||
|
||||
The adapted masking source remains Apache-2.0. Newly written Tsunagi code
|
||||
follows the repository's WTFPL. This package declares both licenses in Cargo
|
||||
metadata; the workspace license alone does not describe its copied source.
|
||||
@@ -0,0 +1,16 @@
|
||||
//! Pairwise, ephemeral Amnezia packet transport with independent end-to-end tunnels.
|
||||
//! All negotiation travels over authenticated Tsunagi control sessions.
|
||||
mod masking;
|
||||
mod plugin;
|
||||
pub mod profile;
|
||||
mod session;
|
||||
mod stun;
|
||||
mod transport;
|
||||
|
||||
pub use plugin::AwgPlugin;
|
||||
pub use transport::{AwgTransport, PairOverview};
|
||||
|
||||
/// Protocol identifier and default second-choice direct transport.
|
||||
pub const AWG_PROTOCOL: &str = "awg";
|
||||
/// Tsunagi AWG wire version, independent of software releases.
|
||||
pub const AWG_WIRE_VERSION: u16 = 1;
|
||||
@@ -0,0 +1,336 @@
|
||||
// Adapted from RunaRoo/AmneziaWG-RS, commit 51579a7d6e8882d1d8d6d00822f09e18a9423b73.
|
||||
// Apache-2.0; see LICENSE-AmneziaWG-RS and UPSTREAM.md.
|
||||
#![allow(dead_code, missing_docs)]
|
||||
// AmneziaWG configuration, header protection, and traffic shaping.
|
||||
|
||||
use bytes::{Bytes, BytesMut};
|
||||
use chacha20::{
|
||||
ChaCha20,
|
||||
cipher::{KeyIvInit, StreamCipher},
|
||||
};
|
||||
use rand::RngExt;
|
||||
|
||||
pub const HEADER_PROTECTION_NONCE_SIZE: usize = 12;
|
||||
pub const DEFAULT_UDP_WINDOW: usize = 500;
|
||||
|
||||
#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
|
||||
pub struct Range {
|
||||
pub min: u32,
|
||||
pub max: u32,
|
||||
}
|
||||
|
||||
impl Range {
|
||||
pub fn contains(&self, val: u32) -> bool {
|
||||
val >= self.min && val <= self.max
|
||||
}
|
||||
|
||||
pub fn random(&self) -> u32 {
|
||||
if self.min >= self.max {
|
||||
self.min
|
||||
} else {
|
||||
rand::rng().random_range(self.min..=self.max)
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_zero(&self) -> bool {
|
||||
self.min == 0 && self.max == 0
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub enum AwgKind {
|
||||
Initiation,
|
||||
Response,
|
||||
Cookie,
|
||||
Transport,
|
||||
}
|
||||
|
||||
impl AwgKind {
|
||||
pub fn fixed_size(self) -> usize {
|
||||
match self {
|
||||
Self::Initiation => 148,
|
||||
Self::Response => 92,
|
||||
Self::Cookie => 64,
|
||||
Self::Transport => 32,
|
||||
}
|
||||
}
|
||||
|
||||
fn standard_type(self) -> u32 {
|
||||
match self {
|
||||
Self::Initiation => 1,
|
||||
Self::Response => 2,
|
||||
Self::Cookie => 3,
|
||||
Self::Transport => 4,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct DecodedPacket {
|
||||
pub kind: AwgKind,
|
||||
pub message: Bytes,
|
||||
pub wire_len: usize,
|
||||
}
|
||||
|
||||
pub struct AmneziaConfig {
|
||||
pub jc: u32,
|
||||
pub jmin: usize,
|
||||
pub jmax: usize,
|
||||
pub s1: usize,
|
||||
pub s2: usize,
|
||||
pub s3: usize,
|
||||
pub s4: usize,
|
||||
pub h1: Range,
|
||||
pub h2: Range,
|
||||
pub h3: Range,
|
||||
pub h4: Range,
|
||||
pub i1: String,
|
||||
pub i2: String,
|
||||
pub i3: String,
|
||||
pub i4: String,
|
||||
pub i5: String,
|
||||
pub header_protection_key: Option<[u8; 32]>,
|
||||
pub content_padding_addition: Range,
|
||||
pub random_trailers: bool,
|
||||
pub mtu: usize,
|
||||
}
|
||||
|
||||
impl AmneziaConfig {
|
||||
pub fn random_header(&self, range: &Range) -> u32 {
|
||||
range.random()
|
||||
}
|
||||
|
||||
pub fn random_padding(&self, len: usize) -> Vec<u8> {
|
||||
let mut padding = vec![0u8; len];
|
||||
if len > 0 {
|
||||
rand::fill(&mut padding);
|
||||
}
|
||||
padding
|
||||
}
|
||||
|
||||
fn shape(&self, kind: AwgKind) -> (usize, Range) {
|
||||
match kind {
|
||||
AwgKind::Initiation => (self.s1, self.h1),
|
||||
AwgKind::Response => (self.s2, self.h2),
|
||||
AwgKind::Cookie => (self.s3, self.h3),
|
||||
AwgKind::Transport => (self.s4, self.h4),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn encode(&self, kind: AwgKind, payload: Bytes, udp_window: usize) -> Bytes {
|
||||
let (padding_len, _) = self.shape(kind);
|
||||
let trailer_len = if kind != AwgKind::Transport && self.random_trailers {
|
||||
let base = padding_len.saturating_add(payload.len());
|
||||
let room = udp_window.max(DEFAULT_UDP_WINDOW).saturating_sub(base);
|
||||
if room == 0 {
|
||||
0
|
||||
} else {
|
||||
rand::rng().random_range(0..room)
|
||||
}
|
||||
} else {
|
||||
0
|
||||
};
|
||||
|
||||
let mut out = BytesMut::zeroed(padding_len + payload.len() + trailer_len);
|
||||
if padding_len > 0 {
|
||||
rand::fill(&mut out[..padding_len]);
|
||||
}
|
||||
out[padding_len..padding_len + payload.len()].copy_from_slice(&payload);
|
||||
if trailer_len > 0 {
|
||||
rand::fill(&mut out[padding_len + payload.len()..]);
|
||||
}
|
||||
|
||||
if self.header_protection_key.is_some() {
|
||||
let protected_len = if kind == AwgKind::Transport {
|
||||
16
|
||||
} else {
|
||||
kind.fixed_size()
|
||||
};
|
||||
self.apply_header_cipher(&mut out, padding_len, protected_len);
|
||||
}
|
||||
|
||||
out.freeze()
|
||||
}
|
||||
|
||||
pub fn decode(&self, datagram: &Bytes) -> Option<DecodedPacket> {
|
||||
if datagram.len() < 4 {
|
||||
return None;
|
||||
}
|
||||
|
||||
for kind in [
|
||||
AwgKind::Initiation,
|
||||
AwgKind::Response,
|
||||
AwgKind::Cookie,
|
||||
AwgKind::Transport,
|
||||
] {
|
||||
let (padding_len, headers) = self.shape(kind);
|
||||
let minimum = padding_len + kind.fixed_size();
|
||||
let size_ok = match kind {
|
||||
AwgKind::Transport | AwgKind::Initiation => datagram.len() >= minimum,
|
||||
AwgKind::Response | AwgKind::Cookie if self.random_trailers => {
|
||||
datagram.len() >= minimum
|
||||
}
|
||||
AwgKind::Response | AwgKind::Cookie => datagram.len() == minimum,
|
||||
};
|
||||
if !size_ok || datagram.len() < padding_len + 4 {
|
||||
continue;
|
||||
}
|
||||
|
||||
let protected_header =
|
||||
u32::from_le_bytes(datagram[padding_len..padding_len + 4].try_into().ok()?);
|
||||
let candidate = if self.header_protection_key.is_some() {
|
||||
protected_header ^ self.header_type_hash(datagram)?
|
||||
} else {
|
||||
protected_header
|
||||
};
|
||||
if !headers.contains(candidate) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let message_len = match kind {
|
||||
AwgKind::Transport | AwgKind::Initiation => datagram.len() - padding_len,
|
||||
AwgKind::Response | AwgKind::Cookie => kind.fixed_size(),
|
||||
};
|
||||
let mut payload = BytesMut::from(&datagram[padding_len..padding_len + message_len]);
|
||||
if self.header_protection_key.is_some() {
|
||||
let protected_len = if kind == AwgKind::Transport {
|
||||
16
|
||||
} else {
|
||||
kind.fixed_size()
|
||||
};
|
||||
self.apply_header_cipher_with_nonce(
|
||||
&mut payload,
|
||||
0,
|
||||
protected_len,
|
||||
&datagram[..HEADER_PROTECTION_NONCE_SIZE],
|
||||
);
|
||||
}
|
||||
|
||||
return Some(DecodedPacket {
|
||||
kind,
|
||||
message: payload.freeze(),
|
||||
wire_len: datagram.len(),
|
||||
});
|
||||
}
|
||||
|
||||
None
|
||||
}
|
||||
|
||||
pub fn prepare_transport_plaintext(&self, packet: Bytes, udp_window: usize) -> Bytes {
|
||||
if packet.is_empty() {
|
||||
return packet;
|
||||
}
|
||||
|
||||
let requested = if !self.content_padding_addition.is_zero() {
|
||||
self.content_padding_addition.random() as usize
|
||||
} else if self.random_trailers {
|
||||
let wire_without_padding = self.s4 + AwgKind::Transport.fixed_size() + packet.len();
|
||||
let room = udp_window
|
||||
.max(DEFAULT_UDP_WINDOW)
|
||||
.saturating_sub(wire_without_padding);
|
||||
if room == 0 {
|
||||
0
|
||||
} else {
|
||||
rand::rng().random_range(0..room)
|
||||
}
|
||||
} else {
|
||||
let aligned = (packet.len() + 15) & !15;
|
||||
aligned.saturating_sub(packet.len())
|
||||
};
|
||||
|
||||
let padding = self.clamp_padding_to_mtu(packet.len(), requested);
|
||||
if padding == 0 {
|
||||
return packet;
|
||||
}
|
||||
|
||||
let mut out = BytesMut::with_capacity(packet.len() + padding);
|
||||
out.extend_from_slice(&packet);
|
||||
out.resize(packet.len() + padding, 0);
|
||||
out.freeze()
|
||||
}
|
||||
|
||||
pub fn transport_content_length(&self, plaintext: &[u8]) -> Option<usize> {
|
||||
if plaintext.is_empty() || plaintext[0] == 0 {
|
||||
return Some(0);
|
||||
}
|
||||
match plaintext[0] >> 4 {
|
||||
4 if plaintext.len() >= 20 => {
|
||||
let total = u16::from_be_bytes([plaintext[2], plaintext[3]]) as usize;
|
||||
(total >= 20 && total <= plaintext.len()).then_some(total)
|
||||
}
|
||||
6 if plaintext.len() >= 40 => {
|
||||
let payload = u16::from_be_bytes([plaintext[4], plaintext[5]]) as usize;
|
||||
let total = 40 + payload;
|
||||
(total <= plaintext.len()).then_some(total)
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn clamp_padding_to_mtu(&self, packet_len: usize, requested: usize) -> usize {
|
||||
if self.mtu == 0 {
|
||||
return requested;
|
||||
}
|
||||
let normalized = if packet_len > self.mtu {
|
||||
packet_len % self.mtu
|
||||
} else {
|
||||
packet_len
|
||||
};
|
||||
requested.min(self.mtu.saturating_sub(normalized))
|
||||
}
|
||||
|
||||
fn header_type_hash(&self, datagram: &Bytes) -> Option<u32> {
|
||||
if datagram.len() < HEADER_PROTECTION_NONCE_SIZE {
|
||||
return None;
|
||||
}
|
||||
let mut hash = [0u8; 4];
|
||||
self.apply_keystream(&mut hash, &datagram[..HEADER_PROTECTION_NONCE_SIZE]);
|
||||
Some(u32::from_le_bytes(hash))
|
||||
}
|
||||
|
||||
fn apply_header_cipher(&self, packet: &mut [u8], offset: usize, len: usize) {
|
||||
if packet.len() < HEADER_PROTECTION_NONCE_SIZE || packet.len() < offset + len {
|
||||
return;
|
||||
}
|
||||
let nonce: [u8; HEADER_PROTECTION_NONCE_SIZE] =
|
||||
match packet[..HEADER_PROTECTION_NONCE_SIZE].try_into() {
|
||||
Ok(nonce) => nonce,
|
||||
Err(_) => return,
|
||||
};
|
||||
self.apply_header_cipher_with_nonce(packet, offset, len, &nonce);
|
||||
}
|
||||
|
||||
fn apply_header_cipher_with_nonce(
|
||||
&self,
|
||||
packet: &mut [u8],
|
||||
offset: usize,
|
||||
len: usize,
|
||||
nonce: &[u8],
|
||||
) {
|
||||
if packet.len() < offset + len {
|
||||
return;
|
||||
}
|
||||
self.apply_keystream(&mut packet[offset..offset + len], nonce);
|
||||
}
|
||||
|
||||
fn apply_keystream(&self, data: &mut [u8], nonce: &[u8]) {
|
||||
let Some(key) = self.header_protection_key.as_ref() else {
|
||||
return;
|
||||
};
|
||||
if nonce.len() != HEADER_PROTECTION_NONCE_SIZE {
|
||||
return;
|
||||
}
|
||||
let Ok(nonce) = <&[u8; HEADER_PROTECTION_NONCE_SIZE]>::try_from(nonce) else {
|
||||
return;
|
||||
};
|
||||
let mut cipher = ChaCha20::new(key.into(), nonce.into());
|
||||
cipher.apply_keystream(data);
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for AmneziaConfig {
|
||||
fn drop(&mut self) {
|
||||
use zeroize::Zeroize;
|
||||
self.header_protection_key.zeroize();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
//! End-to-end encryption remains independent of physical Amnezia pair links.
|
||||
use crate::{AWG_PROTOCOL, AWG_WIRE_VERSION, AwgTransport};
|
||||
use iroh::EndpointId;
|
||||
use std::sync::Arc;
|
||||
use tsunagi::dataplane::transport::SharedLink;
|
||||
use tsunagi::dataplane::{IpPlugin, PluginCapability, PluginContext, PluginError, ProtocolOption};
|
||||
use tsunagi::state::Ipv4Range;
|
||||
use tsunagi::{BoxFuture, NetworkId};
|
||||
use tsunagi_wg_quic::{NetworkOverview, WireguardConfig, WireguardPlugin};
|
||||
|
||||
/// AWG capability and an independent end-to-end WireGuard engine.
|
||||
/// Pair transport keys/profiles never enter its durable end-to-end key store.
|
||||
#[derive(Debug)]
|
||||
pub struct AwgPlugin {
|
||||
inner: Arc<WireguardPlugin>,
|
||||
transport: Arc<AwgTransport>,
|
||||
}
|
||||
impl AwgPlugin {
|
||||
/// Uses standard tunnel MTU/keepalive options; pair ports and profiles are ephemeral.
|
||||
pub const OPTIONS: &'static [ProtocolOption] = &[
|
||||
ProtocolOption {
|
||||
key: "keepalive",
|
||||
value: "SECONDS",
|
||||
help: "end-to-end tunnel keepalive; 0 disables it",
|
||||
default: Some("25"),
|
||||
},
|
||||
ProtocolOption {
|
||||
key: "mtu",
|
||||
value: "BYTES",
|
||||
help: "maximum overlay packet size",
|
||||
default: Some("1280"),
|
||||
},
|
||||
ProtocolOption {
|
||||
key: "stun",
|
||||
value: "HOST:PORT|off",
|
||||
help: "UDP public mapping discovery; disabled in local reach mode",
|
||||
default: Some("stun.l.google.com:19302"),
|
||||
},
|
||||
];
|
||||
/// Opens the end-to-end engine, retaining separate pair transport state.
|
||||
pub async fn open(
|
||||
mut config: WireguardConfig,
|
||||
transport: Arc<AwgTransport>,
|
||||
) -> Result<Arc<Self>, PluginError> {
|
||||
config.protocol = AWG_PROTOCOL;
|
||||
Ok(Arc::new(Self {
|
||||
inner: WireguardPlugin::open(config).await?,
|
||||
transport,
|
||||
}))
|
||||
}
|
||||
/// Reports end-to-end tunnel health and traffic counters.
|
||||
pub fn overview(&self, network: NetworkId) -> Option<NetworkOverview> {
|
||||
self.inner.overview(network)
|
||||
}
|
||||
}
|
||||
impl IpPlugin for AwgPlugin {
|
||||
fn protocol_id(&self) -> &str {
|
||||
AWG_PROTOCOL
|
||||
}
|
||||
fn protocol_version(&self) -> u16 {
|
||||
AWG_WIRE_VERSION
|
||||
}
|
||||
fn options(&self) -> &'static [ProtocolOption] {
|
||||
Self::OPTIONS
|
||||
}
|
||||
fn attach(&self, context: PluginContext) {
|
||||
self.inner.attach(context);
|
||||
}
|
||||
fn local_capability(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
) -> Result<Option<PluginCapability>, PluginError> {
|
||||
self.inner.local_capability(network).map(|cap| {
|
||||
cap.map(|mut cap| {
|
||||
cap.protocol = AWG_PROTOCOL.into();
|
||||
cap.version = AWG_WIRE_VERSION;
|
||||
cap
|
||||
})
|
||||
})
|
||||
}
|
||||
fn on_network_activated(&self, network: NetworkId) {
|
||||
self.inner.on_network_activated(network);
|
||||
}
|
||||
fn on_peer_capability(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
capability: &PluginCapability,
|
||||
) -> Result<(), PluginError> {
|
||||
if capability.protocol != AWG_PROTOCOL
|
||||
|| capability.version != AWG_WIRE_VERSION
|
||||
|| !capability.enabled
|
||||
{
|
||||
return Err(PluginError::Rejected("unsupported AWG capability".into()));
|
||||
}
|
||||
self.inner.on_peer_capability(network, peer, capability)?;
|
||||
self.transport.allow_peer(network, peer);
|
||||
Ok(())
|
||||
}
|
||||
fn on_address_allocation(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
range: Ipv4Range,
|
||||
allocations: &[(EndpointId, std::net::Ipv4Addr)],
|
||||
) {
|
||||
self.inner
|
||||
.on_address_allocation(network, range, allocations);
|
||||
}
|
||||
fn carry(&self, network: NetworkId, peer: EndpointId, packet: bytes::Bytes) -> bool {
|
||||
self.inner.carry(network, peer, packet)
|
||||
}
|
||||
fn on_peer_link(&self, network: NetworkId, peer: EndpointId, link: SharedLink) {
|
||||
self.inner.on_peer_link(network, peer, link);
|
||||
}
|
||||
fn on_peer_gone(&self, network: NetworkId, peer: EndpointId) {
|
||||
self.transport.forget_peer(network, peer);
|
||||
self.inner.on_peer_gone(network, peer);
|
||||
}
|
||||
fn on_network_deactivated(&self, network: NetworkId) {
|
||||
self.transport.stop_network(network);
|
||||
self.inner.on_network_deactivated(network);
|
||||
}
|
||||
fn on_network_forgotten(&self, network: NetworkId) {
|
||||
self.inner.on_network_forgotten(network);
|
||||
}
|
||||
fn shutdown(&self) -> BoxFuture<'_, ()> {
|
||||
Box::pin(async move {
|
||||
self.transport.shutdown();
|
||||
self.inner.shutdown().await;
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
//! Bounded profiles, regenerated for every pair connection attempt.
|
||||
use crate::masking::{AmneziaConfig, AwgKind, Range};
|
||||
use bytes::Bytes;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// An ephemeral masking profile. Secret fields are redacted and erased on drop.
|
||||
#[derive(Clone, Serialize, Deserialize, zeroize::ZeroizeOnDrop)]
|
||||
pub struct PairProfile {
|
||||
id: [u8; 16],
|
||||
prefix: u16,
|
||||
headers: [[u32; 2]; 4],
|
||||
header_key: [u8; 32],
|
||||
pub(crate) psk: [u8; 32],
|
||||
junk_count: u8,
|
||||
junk_min: u16,
|
||||
junk_max: u16,
|
||||
padding: u16,
|
||||
signature: u8,
|
||||
}
|
||||
impl std::fmt::Debug for PairProfile {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str("PairProfile([REDACTED])")
|
||||
}
|
||||
}
|
||||
impl PairProfile {
|
||||
/// Generates a new safe profile; no deterministic seed or durable state.
|
||||
pub fn generate() -> Self {
|
||||
let headers = std::array::from_fn(|i| {
|
||||
let base = (i as u32) * 0x4000_0000 + rand::random_range(1024..0x3fff_0000);
|
||||
[base, base + rand::random_range(64..1024)]
|
||||
});
|
||||
Self {
|
||||
id: rand::random(),
|
||||
prefix: rand::random_range(12..49),
|
||||
headers,
|
||||
header_key: rand::random(),
|
||||
psk: rand::random(),
|
||||
junk_count: rand::random_range(4..9),
|
||||
junk_min: rand::random_range(24..65),
|
||||
junk_max: rand::random_range(80..161),
|
||||
padding: rand::random_range(16..65),
|
||||
signature: rand::random_range(0..2),
|
||||
}
|
||||
}
|
||||
/// Public attempt identifier, useful for distinguishing retries without revealing secrets.
|
||||
pub fn id(&self) -> [u8; 16] {
|
||||
self.id
|
||||
}
|
||||
/// Header key, for packet engines and protocol tests; never log this value.
|
||||
pub fn header_key(&self) -> &[u8; 32] {
|
||||
&self.header_key
|
||||
}
|
||||
/// Rejects invalid remote profiles before allocating packets or entering the codec.
|
||||
pub fn validate(&self) -> Result<(), &'static str> {
|
||||
if !(12..=48).contains(&self.prefix)
|
||||
|| !(4..=8).contains(&self.junk_count)
|
||||
|| self.junk_min < 24
|
||||
|| self.junk_min > self.junk_max
|
||||
|| self.junk_max > 160
|
||||
|| self.padding > 64
|
||||
|| self.signature > 1
|
||||
|| self.header_key == [0; 32]
|
||||
|| self.psk == [0; 32]
|
||||
{
|
||||
return Err("invalid AWG profile bounds");
|
||||
}
|
||||
for (i, [lo, hi]) in self.headers.iter().copied().enumerate() {
|
||||
if lo < 5 || lo > hi || hi - lo > 1024 {
|
||||
return Err("invalid AWG header range");
|
||||
}
|
||||
for [other_lo, other_hi] in &self.headers[..i] {
|
||||
if lo <= *other_hi && *other_lo <= hi {
|
||||
return Err("overlapping AWG header ranges");
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
fn config(&self) -> AmneziaConfig {
|
||||
let range = |i: usize| Range {
|
||||
min: self.headers[i][0],
|
||||
max: self.headers[i][1],
|
||||
};
|
||||
AmneziaConfig {
|
||||
jc: self.junk_count.into(),
|
||||
jmin: self.junk_min.into(),
|
||||
jmax: self.junk_max.into(),
|
||||
s1: self.prefix.into(),
|
||||
s2: self.prefix.into(),
|
||||
s3: self.prefix.into(),
|
||||
s4: self.prefix.into(),
|
||||
h1: range(0),
|
||||
h2: range(1),
|
||||
h3: range(2),
|
||||
h4: range(3),
|
||||
i1: String::new(),
|
||||
i2: String::new(),
|
||||
i3: String::new(),
|
||||
i4: String::new(),
|
||||
i5: String::new(),
|
||||
header_protection_key: Some(self.header_key),
|
||||
content_padding_addition: Range {
|
||||
min: 0,
|
||||
max: self.padding.into(),
|
||||
},
|
||||
random_trailers: true,
|
||||
mtu: 1050,
|
||||
}
|
||||
}
|
||||
/// Masks one standard WireGuard message; invalid types and oversized packets are refused.
|
||||
pub fn encode(&self, raw: &[u8]) -> Option<Bytes> {
|
||||
let kind = match raw.get(..4)? {
|
||||
[1, 0, 0, 0] => AwgKind::Initiation,
|
||||
[2, 0, 0, 0] => AwgKind::Response,
|
||||
[3, 0, 0, 0] => AwgKind::Cookie,
|
||||
[4, 0, 0, 0] => AwgKind::Transport,
|
||||
_ => return None,
|
||||
};
|
||||
if raw.len() < kind.fixed_size() || raw.len() > 1050 {
|
||||
return None;
|
||||
}
|
||||
let config = self.config();
|
||||
let range = match kind {
|
||||
AwgKind::Initiation => config.h1,
|
||||
AwgKind::Response => config.h2,
|
||||
AwgKind::Cookie => config.h3,
|
||||
AwgKind::Transport => config.h4,
|
||||
};
|
||||
let mut payload = raw.to_vec();
|
||||
payload[..4].copy_from_slice(&range.random().to_le_bytes());
|
||||
Some(config.encode(kind, Bytes::from(payload), 1200))
|
||||
}
|
||||
/// Unmasks a packet and restores standard headers before WireGuard validation.
|
||||
pub fn decode(&self, wire: &[u8]) -> Option<Vec<u8>> {
|
||||
if wire.len() > 1200 {
|
||||
return None;
|
||||
}
|
||||
let decoded = self.config().decode(&Bytes::copy_from_slice(wire))?;
|
||||
let (kind, len) = match decoded.kind {
|
||||
AwgKind::Initiation => (1, 148),
|
||||
AwgKind::Response => (2, 92),
|
||||
AwgKind::Cookie => (3, 64),
|
||||
AwgKind::Transport => (4, decoded.message.len()),
|
||||
};
|
||||
let mut message = decoded.message.get(..len)?.to_vec();
|
||||
message[..4].copy_from_slice(&(kind as u32).to_le_bytes());
|
||||
Some(message)
|
||||
}
|
||||
pub(crate) fn pad(&self, packet: Bytes) -> Bytes {
|
||||
self.config().prepare_transport_plaintext(packet, 1200)
|
||||
}
|
||||
/// Generates structured signature packets with fresh fields, never static captures.
|
||||
/// These target the other Tsunagi peer, not DNS or SIP services.
|
||||
pub fn signature_packets(&self) -> Vec<Vec<u8>> {
|
||||
if self.signature == 0 {
|
||||
// A complete DNS query. Counts, label lengths and qtype remain valid.
|
||||
let label = format!("n{:016x}", rand::random::<u64>());
|
||||
let mut packet = Vec::from(rand::random::<u16>().to_be_bytes());
|
||||
packet.extend_from_slice(&[1, 0, 0, 1, 0, 0, 0, 0, 0, 0]);
|
||||
packet.push(label.len() as u8);
|
||||
packet.extend_from_slice(label.as_bytes());
|
||||
packet.extend_from_slice(b"\x07example\x03net\0\0");
|
||||
packet.push(if rand::random() { 1 } else { 28 });
|
||||
packet.extend_from_slice(&[0, 1]);
|
||||
vec![packet]
|
||||
} else {
|
||||
let id = rand::random::<u64>();
|
||||
vec![format!("OPTIONS sip:n{id:x}@example.net SIP/2.0\r\nVia: SIP/2.0/UDP example.net;branch=z9hG4bK{id:x}\r\nMax-Forwards: 70\r\nFrom: <sip:a@example.net>;tag={id:x}\r\nTo: <sip:n{id:x}@example.net>\r\nCall-ID: {id:x}@example.net\r\nCSeq: 1 OPTIONS\r\nContent-Length: 0\r\n\r\n").into_bytes()]
|
||||
}
|
||||
}
|
||||
pub(crate) fn junk_packets(&self) -> Vec<Vec<u8>> {
|
||||
(0..self.junk_count)
|
||||
.map(|_| {
|
||||
let mut packet = vec![0; rand::random_range(self.junk_min..=self.junk_max).into()];
|
||||
rand::fill(&mut packet[..]);
|
||||
packet
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
#[test]
|
||||
fn unsafe_peer_profiles_are_rejected_before_the_codec() {
|
||||
let valid = PairProfile::generate();
|
||||
let mut bad = valid.clone();
|
||||
bad.prefix = 11;
|
||||
assert!(bad.validate().is_err());
|
||||
let mut bad = valid.clone();
|
||||
bad.headers[1] = bad.headers[0];
|
||||
assert!(bad.validate().is_err());
|
||||
let mut bad = valid.clone();
|
||||
bad.junk_max = u16::MAX;
|
||||
assert!(bad.validate().is_err());
|
||||
let mut bad = valid.clone();
|
||||
bad.padding = u16::MAX;
|
||||
assert!(bad.validate().is_err());
|
||||
let mut bad = valid.clone();
|
||||
bad.psk = [0; 32];
|
||||
assert!(bad.validate().is_err());
|
||||
let mut bad = valid.clone();
|
||||
bad.signature = 2;
|
||||
assert!(bad.validate().is_err());
|
||||
}
|
||||
#[test]
|
||||
fn signature_templates_have_consistent_framing_and_fresh_fields() {
|
||||
let mut profile = PairProfile::generate();
|
||||
profile.signature = 0;
|
||||
let dns = profile.signature_packets().remove(0);
|
||||
assert_eq!(&dns[2..12], &[1, 0, 0, 1, 0, 0, 0, 0, 0, 0]);
|
||||
let mut at = 12;
|
||||
while dns[at] != 0 {
|
||||
at += 1 + usize::from(dns[at]);
|
||||
}
|
||||
assert_eq!(at + 5, dns.len());
|
||||
assert_eq!(&dns[dns.len() - 2..], &[0, 1]);
|
||||
profile.signature = 1;
|
||||
let sip = profile.signature_packets().remove(0);
|
||||
assert!(sip.starts_with(b"OPTIONS sip:"));
|
||||
assert!(sip.ends_with(b"Content-Length: 0\r\n\r\n"));
|
||||
assert_ne!(sip, profile.signature_packets().remove(0));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,402 @@
|
||||
//! One cancellable actor and UDP socket per pair. All payload and health frames
|
||||
//! are inside the Amnezia-masked authenticated WireGuard tunnel.
|
||||
use crate::profile::PairProfile;
|
||||
use boringtun::noise::{Tunn, TunnResult};
|
||||
use bytes::Bytes;
|
||||
use iroh::EndpointId;
|
||||
use std::collections::{HashMap, VecDeque};
|
||||
use std::net::{Ipv4Addr, SocketAddr};
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::{Duration, Instant};
|
||||
use tokio::net::UdpSocket;
|
||||
use tokio::sync::{mpsc, watch};
|
||||
use tsunagi::dataplane::transport::{PacketLink, TransportError};
|
||||
use tsunagi::{BoxFuture, NetworkId};
|
||||
|
||||
pub(crate) const MAX_PAYLOAD: usize = tsunagi::config::MAX_DATA_DATAGRAM;
|
||||
const CHUNK: usize = 700;
|
||||
const QUEUE: usize = 64;
|
||||
const WINDOW: Duration = Duration::from_secs(20);
|
||||
|
||||
pub(crate) struct PairLink {
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
pub(crate) attempt: [u8; 16],
|
||||
tx: mpsc::Sender<Bytes>,
|
||||
rx: tokio::sync::Mutex<mpsc::Receiver<Bytes>>,
|
||||
stop: watch::Sender<bool>,
|
||||
closed: AtomicBool,
|
||||
remote: Mutex<Option<SocketAddr>>,
|
||||
reason: Mutex<Option<String>>,
|
||||
}
|
||||
impl std::fmt::Debug for PairLink {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("AwgPair")
|
||||
.field("network", &self.network)
|
||||
.field("peer", &self.peer)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
impl PairLink {
|
||||
pub(crate) fn has_authenticated_path(&self) -> bool {
|
||||
match self.remote.lock() {
|
||||
Ok(remote) => remote.is_some(),
|
||||
Err(_) => false,
|
||||
}
|
||||
}
|
||||
}
|
||||
impl PacketLink for PairLink {
|
||||
fn network(&self) -> NetworkId {
|
||||
self.network
|
||||
}
|
||||
fn peer(&self) -> EndpointId {
|
||||
self.peer
|
||||
}
|
||||
fn max_datagram_size(&self) -> usize {
|
||||
MAX_PAYLOAD
|
||||
}
|
||||
fn send(&self, payload: Bytes) -> Result<(), TransportError> {
|
||||
if self.is_closed() {
|
||||
return Err(TransportError::Closed);
|
||||
}
|
||||
if payload.len() > MAX_PAYLOAD {
|
||||
return Err(TransportError::TooLarge {
|
||||
size: payload.len(),
|
||||
limit: MAX_PAYLOAD,
|
||||
});
|
||||
}
|
||||
self.tx
|
||||
.try_send(payload)
|
||||
.map_err(|_| TransportError::Other("AWG send queue full or closed".into()))
|
||||
}
|
||||
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
|
||||
Box::pin(async move {
|
||||
let mut stop = self.stop.subscribe();
|
||||
if *stop.borrow() {
|
||||
return None;
|
||||
}
|
||||
let mut rx = self.rx.lock().await;
|
||||
tokio::select! {biased; _=stop.changed()=>None,packet=rx.recv()=>packet}
|
||||
})
|
||||
}
|
||||
fn closed(&self) -> BoxFuture<'_, ()> {
|
||||
Box::pin(async move {
|
||||
let mut stop = self.stop.subscribe();
|
||||
if !*stop.borrow() {
|
||||
let _ = stop.changed().await;
|
||||
}
|
||||
})
|
||||
}
|
||||
fn is_closed(&self) -> bool {
|
||||
self.closed.load(Ordering::Acquire) || *self.stop.borrow()
|
||||
}
|
||||
fn close(&self) {
|
||||
self.stop.send_replace(true);
|
||||
}
|
||||
fn failure_reason(&self) -> Option<String> {
|
||||
match self.reason.lock() {
|
||||
Ok(reason) => reason.clone(),
|
||||
Err(_) => None,
|
||||
}
|
||||
}
|
||||
fn path_description(&self) -> String {
|
||||
match self.remote.lock() {
|
||||
Ok(addr) => match *addr {
|
||||
Some(addr) => format!("awg UDP via {addr}"),
|
||||
None => "awg UDP, establishing".into(),
|
||||
},
|
||||
Err(_) => "awg UDP".into(),
|
||||
}
|
||||
}
|
||||
}
|
||||
impl Drop for PairLink {
|
||||
fn drop(&mut self) {
|
||||
self.stop.send_replace(true);
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn start(
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
socket: UdpSocket,
|
||||
tunn: Tunn,
|
||||
profile: PairProfile,
|
||||
candidates: Vec<SocketAddr>,
|
||||
parent: watch::Receiver<bool>,
|
||||
) -> Arc<PairLink> {
|
||||
let (tx, rx) = mpsc::channel(QUEUE);
|
||||
let (in_tx, in_rx) = mpsc::channel(QUEUE);
|
||||
let (stop, _) = watch::channel(false);
|
||||
let link = Arc::new(PairLink {
|
||||
network,
|
||||
peer,
|
||||
attempt: profile.id(),
|
||||
tx,
|
||||
rx: tokio::sync::Mutex::new(in_rx),
|
||||
stop: stop.clone(),
|
||||
closed: AtomicBool::new(false),
|
||||
remote: Mutex::new(None),
|
||||
reason: Mutex::new(None),
|
||||
});
|
||||
let weak = Arc::downgrade(&link);
|
||||
tokio::spawn(async move {
|
||||
let mut engine = Engine {
|
||||
stop: stop.subscribe(),
|
||||
socket,
|
||||
tunn,
|
||||
profile,
|
||||
candidates,
|
||||
remote: None,
|
||||
inbox: in_tx,
|
||||
rx,
|
||||
counter: rand::random(),
|
||||
sequence: 0,
|
||||
pending: VecDeque::new(),
|
||||
samples: VecDeque::new(),
|
||||
assemblies: HashMap::new(),
|
||||
started: Instant::now(),
|
||||
last_authenticated: Instant::now(),
|
||||
};
|
||||
engine.run(parent, &weak).await;
|
||||
if let Some(link) = weak.upgrade() {
|
||||
link.closed.store(true, Ordering::Release);
|
||||
link.stop.send_replace(true);
|
||||
}
|
||||
});
|
||||
link
|
||||
}
|
||||
|
||||
struct Assembly {
|
||||
created: Instant,
|
||||
count: u16,
|
||||
chunks: Vec<Option<Bytes>>,
|
||||
}
|
||||
struct Engine {
|
||||
stop: watch::Receiver<bool>,
|
||||
socket: UdpSocket,
|
||||
tunn: Tunn,
|
||||
profile: PairProfile,
|
||||
candidates: Vec<SocketAddr>,
|
||||
remote: Option<SocketAddr>,
|
||||
inbox: mpsc::Sender<Bytes>,
|
||||
rx: mpsc::Receiver<Bytes>,
|
||||
counter: u64,
|
||||
sequence: u64,
|
||||
pending: VecDeque<(u64, Instant)>,
|
||||
samples: VecDeque<bool>,
|
||||
assemblies: HashMap<u64, Assembly>,
|
||||
started: Instant,
|
||||
last_authenticated: Instant,
|
||||
}
|
||||
enum ResultOwned {
|
||||
Wire(Vec<u8>),
|
||||
Plain(Vec<u8>),
|
||||
Done,
|
||||
}
|
||||
fn own(result: TunnResult<'_>) -> ResultOwned {
|
||||
match result {
|
||||
TunnResult::WriteToNetwork(p) => ResultOwned::Wire(p.to_vec()),
|
||||
TunnResult::WriteToTunnelV4(p, _) => ResultOwned::Plain(p.to_vec()),
|
||||
_ => ResultOwned::Done,
|
||||
}
|
||||
}
|
||||
fn ip(payload: &[u8]) -> Bytes {
|
||||
let mut packet = vec![0; 20];
|
||||
packet[0] = 0x45;
|
||||
packet[8] = 64;
|
||||
packet[9] = 253;
|
||||
packet[12..16].copy_from_slice(&Ipv4Addr::new(192, 0, 2, 1).octets());
|
||||
packet[16..20].copy_from_slice(&Ipv4Addr::new(192, 0, 2, 2).octets());
|
||||
packet[2..4].copy_from_slice(&((20 + payload.len()) as u16).to_be_bytes());
|
||||
let sum = packet
|
||||
.as_chunks::<2>()
|
||||
.0
|
||||
.iter()
|
||||
.map(|b| u32::from(u16::from_be_bytes([b[0], b[1]])))
|
||||
.sum::<u32>();
|
||||
let sum = (sum & 0xffff) + (sum >> 16);
|
||||
packet[10..12].copy_from_slice(&(!(sum as u16)).to_be_bytes());
|
||||
packet.extend_from_slice(payload);
|
||||
Bytes::from(packet)
|
||||
}
|
||||
impl Engine {
|
||||
async fn run(&mut self, mut stop: watch::Receiver<bool>, link: &std::sync::Weak<PairLink>) {
|
||||
if *self.stop.borrow() || *stop.borrow() {
|
||||
return;
|
||||
}
|
||||
let mut buffer = [0; 1201];
|
||||
let mut scratch = [0; 4096];
|
||||
let mut timer = tokio::time::interval(Duration::from_millis(250));
|
||||
timer.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
|
||||
let mut last_probe = Instant::now() - Duration::from_secs(2);
|
||||
loop {
|
||||
tokio::select! {
|
||||
biased;
|
||||
_=stop.changed()=>break,
|
||||
_=self.stop.changed()=>break,
|
||||
_=timer.tick()=>{
|
||||
let result=own(self.tunn.update_timers(&mut scratch));self.process(result,None).await;
|
||||
if last_probe.elapsed()>=Duration::from_secs(1) {
|
||||
last_probe=Instant::now();self.sequence=self.sequence.wrapping_add(1);
|
||||
self.pending.push_back((self.sequence,Instant::now()));
|
||||
let mut probe=vec![1];probe.extend_from_slice(&self.sequence.to_be_bytes());
|
||||
probe.resize(713,0);rand::fill(&mut probe[9..]);self.encrypt(&probe).await;
|
||||
}
|
||||
while self.pending.front().is_some_and(|(_,at)|at.elapsed()>=Duration::from_secs(3)) {
|
||||
self.pending.pop_front();self.samples.push_back(false);
|
||||
}
|
||||
while self.samples.len()>20 {self.samples.pop_front();}
|
||||
self.assemblies.retain(|_,a|a.created.elapsed()<Duration::from_secs(5));
|
||||
if self.started.elapsed()>WINDOW && (self.last_authenticated.elapsed()>Duration::from_secs(10)
|
||||
|| (self.samples.len()>=15&&self.samples.iter().filter(|ok|!**ok).count()*100>=self.samples.len()*60)) {
|
||||
if let Some(link)=link.upgrade() && let Ok(mut reason)=link.reason.lock() {*reason=Some("AWG data delivery checks: persistent loss or timeout".into());}
|
||||
break;
|
||||
}
|
||||
}
|
||||
packet=self.socket.recv_from(&mut buffer)=>{
|
||||
if let Ok((len,source))=packet && let Some(message)=self.profile.decode(&buffer[..len]) {
|
||||
let result=own(self.tunn.decapsulate(None,&message,&mut scratch));
|
||||
if matches!(result,ResultOwned::Plain(_)) {
|
||||
self.remote=Some(source);self.last_authenticated=Instant::now();
|
||||
if let Some(link)=link.upgrade() && let Ok(mut remote)=link.remote.lock(){*remote=Some(source);}
|
||||
}
|
||||
self.process(result,Some(source)).await;
|
||||
for _ in 0..16 {
|
||||
let result=own(self.tunn.decapsulate(None,&[],&mut scratch));
|
||||
if matches!(result,ResultOwned::Done){break;}
|
||||
self.process(result,Some(source)).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
packet=self.rx.recv()=>{
|
||||
let Some(packet)=packet else{break;};
|
||||
let count=packet.len().max(1).div_ceil(CHUNK) as u16;
|
||||
self.counter=self.counter.wrapping_add(1);
|
||||
for (index,chunk) in packet.chunks(CHUNK).enumerate() {
|
||||
let mut payload=vec![3];payload.extend_from_slice(&self.counter.to_be_bytes());
|
||||
payload.extend_from_slice(&(index as u16).to_be_bytes());payload.extend_from_slice(&count.to_be_bytes());
|
||||
payload.extend_from_slice(chunk);self.encrypt(&payload).await;
|
||||
// Large logical datagrams must let local readers and other
|
||||
// pairs drain their bounded socket/command queues.
|
||||
if (index + 1).is_multiple_of(16) {tokio::task::yield_now().await;}
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
async fn encrypt(&mut self, payload: &[u8]) {
|
||||
let packet = self.profile.pad(ip(payload));
|
||||
let mut scratch = [0; 4096];
|
||||
let result = own(self.tunn.encapsulate(&packet, &mut scratch));
|
||||
self.process(result, None).await;
|
||||
}
|
||||
async fn wire(&self, raw: &[u8], target: Option<SocketAddr>) {
|
||||
let Some(wire) = self.profile.encode(raw) else {
|
||||
return;
|
||||
};
|
||||
let targets: Vec<_> = target
|
||||
.or(self.remote)
|
||||
.map(|a| vec![a])
|
||||
.unwrap_or_else(|| self.candidates.clone());
|
||||
for addr in targets {
|
||||
if raw.first() == Some(&1) {
|
||||
for signature in self.profile.signature_packets() {
|
||||
let _ = self.send_wire(&signature, addr).await;
|
||||
}
|
||||
for junk in self.profile.junk_packets() {
|
||||
let _ = self.send_wire(&junk, addr).await;
|
||||
}
|
||||
}
|
||||
let _ = self.send_wire(&wire, addr).await;
|
||||
}
|
||||
}
|
||||
async fn send_wire(&self, bytes: &[u8], address: SocketAddr) {
|
||||
let mut stop = self.stop.clone();
|
||||
if *stop.borrow() {
|
||||
return;
|
||||
}
|
||||
tokio::select! {biased; _=stop.changed()=>{}, _=tokio::time::timeout(Duration::from_millis(100),self.socket.send_to(bytes,address))=>{}}
|
||||
}
|
||||
async fn process(&mut self, result: ResultOwned, target: Option<SocketAddr>) {
|
||||
match result {
|
||||
ResultOwned::Wire(raw) => self.wire(&raw, target).await,
|
||||
ResultOwned::Plain(packet) => {
|
||||
let Some(payload) = packet.get(20..) else {
|
||||
return;
|
||||
};
|
||||
let Some(seq) = payload
|
||||
.get(1..9)
|
||||
.and_then(|b| b.try_into().ok())
|
||||
.map(u64::from_be_bytes)
|
||||
else {
|
||||
return;
|
||||
};
|
||||
match payload.first() {
|
||||
Some(1) => {
|
||||
let mut pong = vec![2];
|
||||
pong.extend_from_slice(&seq.to_be_bytes());
|
||||
pong.resize(713, 0);
|
||||
let packet = self.profile.pad(ip(&pong));
|
||||
let mut scratch = [0; 4096];
|
||||
if let ResultOwned::Wire(raw) =
|
||||
own(self.tunn.encapsulate(&packet, &mut scratch))
|
||||
{
|
||||
self.wire(&raw, target).await;
|
||||
}
|
||||
}
|
||||
Some(2) => {
|
||||
if let Some(index) =
|
||||
self.pending.iter().position(|(pending, _)| *pending == seq)
|
||||
{
|
||||
self.pending.remove(index);
|
||||
self.samples.push_back(true);
|
||||
}
|
||||
}
|
||||
Some(3) => self.fragment(seq, payload),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
ResultOwned::Done => {}
|
||||
}
|
||||
}
|
||||
fn fragment(&mut self, id: u64, payload: &[u8]) {
|
||||
let Some(header) = payload.get(9..13) else {
|
||||
return;
|
||||
};
|
||||
let index = u16::from_be_bytes([header[0], header[1]]);
|
||||
let count = u16::from_be_bytes([header[2], header[3]]);
|
||||
let chunk = &payload[13..];
|
||||
if count == 0
|
||||
|| usize::from(count) > MAX_PAYLOAD.div_ceil(CHUNK)
|
||||
|| index >= count
|
||||
|| chunk.len() > CHUNK
|
||||
{
|
||||
return;
|
||||
}
|
||||
if !self.assemblies.contains_key(&id) && self.assemblies.len() >= 8 {
|
||||
return;
|
||||
}
|
||||
let assembly = self.assemblies.entry(id).or_insert_with(|| Assembly {
|
||||
created: Instant::now(),
|
||||
count,
|
||||
chunks: vec![None; count.into()],
|
||||
});
|
||||
if assembly.count != count {
|
||||
return;
|
||||
}
|
||||
assembly.chunks[usize::from(index)] = Some(Bytes::copy_from_slice(chunk));
|
||||
if assembly.chunks.iter().all(Option::is_some)
|
||||
&& let Some(assembly) = self.assemblies.remove(&id)
|
||||
{
|
||||
let mut packet = Vec::new();
|
||||
for chunk in assembly.chunks.into_iter().flatten() {
|
||||
packet.extend_from_slice(&chunk);
|
||||
}
|
||||
if packet.len() <= MAX_PAYLOAD {
|
||||
let _ = self.inbox.try_send(Bytes::from(packet));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
//! Optional, bounded server-reflexive discovery on the pair's own UDP socket.
|
||||
//! A STUN answer is a candidate only; WireGuard authentication verifies a path.
|
||||
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
|
||||
use std::time::Duration;
|
||||
use tokio::net::UdpSocket;
|
||||
const COOKIE: [u8; 4] = [0x21, 0x12, 0xa4, 0x42];
|
||||
pub(crate) async fn mapped(socket: &UdpSocket, server: &str) -> Option<SocketAddr> {
|
||||
tokio::time::timeout(Duration::from_millis(1200), async {
|
||||
let address = tokio::net::lookup_host(server)
|
||||
.await
|
||||
.ok()?
|
||||
.find(SocketAddr::is_ipv4)?;
|
||||
let transaction: [u8; 12] = rand::random();
|
||||
let mut request = [0; 20];
|
||||
request[..2].copy_from_slice(&[0, 1]);
|
||||
request[4..8].copy_from_slice(&COOKIE);
|
||||
request[8..].copy_from_slice(&transaction);
|
||||
socket.send_to(&request, address).await.ok()?;
|
||||
let mut bytes = [0; 1024];
|
||||
loop {
|
||||
let (len, source) = socket.recv_from(&mut bytes).await.ok()?;
|
||||
if source == address
|
||||
&& let Some(mapped) = parse(&bytes[..len], &transaction)
|
||||
{
|
||||
return Some(mapped);
|
||||
}
|
||||
}
|
||||
})
|
||||
.await
|
||||
.ok()
|
||||
.flatten()
|
||||
}
|
||||
fn parse(bytes: &[u8], transaction: &[u8; 12]) -> Option<SocketAddr> {
|
||||
if bytes.len() < 20
|
||||
|| bytes[..2] != [1, 1]
|
||||
|| bytes[4..8] != COOKIE
|
||||
|| &bytes[8..20] != transaction
|
||||
{
|
||||
return None;
|
||||
}
|
||||
let declared = u16::from_be_bytes(bytes[2..4].try_into().ok()?) as usize;
|
||||
if !declared.is_multiple_of(4) || declared != bytes.len() - 20 {
|
||||
return None;
|
||||
}
|
||||
let mut at = 20;
|
||||
while at + 4 <= bytes.len() {
|
||||
let kind = u16::from_be_bytes(bytes[at..at + 2].try_into().ok()?);
|
||||
let len = u16::from_be_bytes(bytes[at + 2..at + 4].try_into().ok()?) as usize;
|
||||
at += 4;
|
||||
let attribute = bytes.get(at..at.checked_add(len)?)?;
|
||||
if kind == 0x0020 && attribute.len() == 8 && attribute[0..2] == [0, 1] {
|
||||
let port = u16::from_be_bytes([attribute[2] ^ COOKIE[0], attribute[3] ^ COOKIE[1]]);
|
||||
let ip = Ipv4Addr::new(
|
||||
attribute[4] ^ COOKIE[0],
|
||||
attribute[5] ^ COOKIE[1],
|
||||
attribute[6] ^ COOKIE[2],
|
||||
attribute[7] ^ COOKIE[3],
|
||||
);
|
||||
if port == 0 || ip.is_unspecified() || ip.is_multicast() || ip.is_broadcast() {
|
||||
return None;
|
||||
}
|
||||
return Some(SocketAddr::new(IpAddr::V4(ip), port));
|
||||
}
|
||||
at = at.checked_add((len + 3) & !3)?;
|
||||
}
|
||||
None
|
||||
}
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
#[tokio::test]
|
||||
async fn mapping_uses_the_actual_socket_and_rejects_unrelated_or_truncated_answers() {
|
||||
let server = UdpSocket::bind((Ipv4Addr::LOCALHOST, 0)).await.unwrap();
|
||||
let address = server.local_addr().unwrap();
|
||||
let socket = UdpSocket::bind((Ipv4Addr::LOCALHOST, 0)).await.unwrap();
|
||||
let expected = socket.local_addr().unwrap();
|
||||
let task = tokio::spawn(async move {
|
||||
let mut request = [0; 128];
|
||||
let (len, source) = server.recv_from(&mut request).await.unwrap();
|
||||
assert_eq!(len, 20);
|
||||
assert_eq!(source, expected);
|
||||
let tx: [u8; 12] = request[8..20].try_into().unwrap();
|
||||
let mut reply = request[..20].to_vec();
|
||||
reply[..4].copy_from_slice(&[1, 1, 0, 12]);
|
||||
reply.extend_from_slice(&[0, 0x20, 0, 8, 0, 1]);
|
||||
let port = source.port().to_be_bytes();
|
||||
reply.extend_from_slice(&[port[0] ^ COOKIE[0], port[1] ^ COOKIE[1]]);
|
||||
for (i, b) in Ipv4Addr::LOCALHOST.octets().into_iter().enumerate() {
|
||||
reply.push(b ^ COOKIE[i]);
|
||||
}
|
||||
assert!(parse(&reply, &[0; 12]).is_none());
|
||||
for len in 0..reply.len() {
|
||||
assert!(parse(&reply[..len], &tx).is_none());
|
||||
}
|
||||
server.send_to(&reply, source).await.unwrap();
|
||||
});
|
||||
assert_eq!(mapped(&socket, &address.to_string()).await, Some(expected));
|
||||
task.await.unwrap();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,720 @@
|
||||
//! Private pair negotiation. No secret profiles in capabilities or durable state.
|
||||
use crate::{
|
||||
AWG_PROTOCOL, AWG_WIRE_VERSION,
|
||||
profile::PairProfile,
|
||||
session::{self, PairLink},
|
||||
};
|
||||
use boringtun::{
|
||||
noise::Tunn,
|
||||
x25519::{PublicKey, StaticSecret},
|
||||
};
|
||||
use iroh::EndpointId;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::net::{Ipv4Addr, SocketAddr};
|
||||
use std::sync::{Arc, Mutex, OnceLock, Weak};
|
||||
use std::time::Duration;
|
||||
use tokio::net::UdpSocket;
|
||||
use tokio::sync::{mpsc, oneshot, watch};
|
||||
use tsunagi::dataplane::transport::TransportPayload;
|
||||
use tsunagi::dataplane::transport::{
|
||||
ControlSender, InboundLink, PacketLink, PacketTransport, SharedLink, TransportError,
|
||||
};
|
||||
use tsunagi::{BoxFuture, NetworkId};
|
||||
|
||||
type Pair = (NetworkId, EndpointId);
|
||||
#[derive(Serialize, Deserialize)]
|
||||
enum Negotiation {
|
||||
Offer {
|
||||
version: u16,
|
||||
profile: PairProfile,
|
||||
key: [u8; 32],
|
||||
#[serde(deserialize_with = "bounded_addresses")]
|
||||
addresses: Vec<SocketAddr>,
|
||||
},
|
||||
Answer {
|
||||
attempt: [u8; 16],
|
||||
key: [u8; 32],
|
||||
#[serde(deserialize_with = "bounded_addresses")]
|
||||
addresses: Vec<SocketAddr>,
|
||||
},
|
||||
}
|
||||
fn bounded_addresses<'de, D: serde::Deserializer<'de>>(
|
||||
deserializer: D,
|
||||
) -> Result<Vec<SocketAddr>, D::Error> {
|
||||
struct Visitor;
|
||||
impl<'de> serde::de::Visitor<'de> for Visitor {
|
||||
type Value = Vec<SocketAddr>;
|
||||
fn expecting(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str("at most 32 UDP candidates")
|
||||
}
|
||||
fn visit_seq<A: serde::de::SeqAccess<'de>>(
|
||||
self,
|
||||
mut sequence: A,
|
||||
) -> Result<Self::Value, A::Error> {
|
||||
if sequence.size_hint().is_some_and(|n| n > 32) {
|
||||
return Err(serde::de::Error::custom("too many UDP candidates"));
|
||||
}
|
||||
let mut addresses = Vec::with_capacity(sequence.size_hint().unwrap_or(0).min(32));
|
||||
while let Some(address) = sequence.next_element()? {
|
||||
if addresses.len() == 32 {
|
||||
return Err(serde::de::Error::custom("too many UDP candidates"));
|
||||
}
|
||||
addresses.push(address);
|
||||
}
|
||||
Ok(addresses)
|
||||
}
|
||||
}
|
||||
deserializer.deserialize_seq(Visitor)
|
||||
}
|
||||
struct Pending {
|
||||
profile: PairProfile,
|
||||
key: StaticSecret,
|
||||
socket: UdpSocket,
|
||||
reply: oneshot::Sender<Result<SharedLink, TransportError>>,
|
||||
}
|
||||
struct Inner {
|
||||
stopped: watch::Sender<bool>,
|
||||
local: EndpointId,
|
||||
bind: Ipv4Addr,
|
||||
stun: Option<String>,
|
||||
busy: Mutex<HashSet<Pair>>,
|
||||
control: OnceLock<Arc<dyn ControlSender>>,
|
||||
allowed: Mutex<HashSet<Pair>>,
|
||||
pending: Mutex<HashMap<Pair, Pending>>,
|
||||
links: Mutex<HashMap<Pair, Arc<PairLink>>>,
|
||||
inbound: mpsc::Sender<InboundLink>,
|
||||
port_by_pair: Mutex<HashMap<Pair, u16>>,
|
||||
}
|
||||
/// Public diagnostics for one pair. No key or masking parameters are exposed.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PairOverview {
|
||||
/// Other endpoint's authenticated identity.
|
||||
pub peer: EndpointId,
|
||||
/// Fresh connection-attempt identifier.
|
||||
pub attempt: [u8; 16],
|
||||
/// Last known UDP port; None if diagnostics race with pair retirement.
|
||||
pub local_port: Option<u16>,
|
||||
/// Actual transport path, if established.
|
||||
pub path: String,
|
||||
/// Whether this attempt has been retired.
|
||||
pub closed: bool,
|
||||
}
|
||||
/// A bounded set of independently configured pair engines.
|
||||
pub struct AwgTransport {
|
||||
inner: Arc<Inner>,
|
||||
commands: mpsc::Sender<(NetworkId, EndpointId, Negotiation)>,
|
||||
incoming: tokio::sync::Mutex<mpsc::Receiver<InboundLink>>,
|
||||
}
|
||||
impl std::fmt::Debug for AwgTransport {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("AwgTransport")
|
||||
.field("local", &self.inner.local)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
|
||||
match mutex.lock() {
|
||||
Ok(g) => g,
|
||||
Err(p) => p.into_inner(),
|
||||
}
|
||||
}
|
||||
fn error(text: &str) -> TransportError {
|
||||
TransportError::Other(text.into())
|
||||
}
|
||||
async fn addresses(inner: &Inner, socket: &UdpSocket) -> Result<Vec<SocketAddr>, TransportError> {
|
||||
let local = socket
|
||||
.local_addr()
|
||||
.map_err(|_| error("AWG socket address unavailable"))?;
|
||||
let mut candidates = if inner.bind.is_loopback() {
|
||||
vec![local]
|
||||
} else {
|
||||
tsunagi_wireguard::local_ip_candidates()
|
||||
.into_iter()
|
||||
.filter(|ip| ip.is_ipv4())
|
||||
.take(31)
|
||||
.map(|ip| SocketAddr::new(ip, local.port()))
|
||||
.collect()
|
||||
};
|
||||
if let Some(server) = &inner.stun
|
||||
&& let Some(mapped) = crate::stun::mapped(socket, server).await
|
||||
&& !candidates.contains(&mapped)
|
||||
{
|
||||
candidates.push(mapped);
|
||||
}
|
||||
if candidates.is_empty() {
|
||||
return Err(error("AWG has no UDP address candidates"));
|
||||
}
|
||||
Ok(candidates)
|
||||
}
|
||||
fn validate_endpoint(key: &[u8; 32], addresses: &[SocketAddr]) -> Result<(), TransportError> {
|
||||
if *key == [0; 32]
|
||||
|| addresses.is_empty()
|
||||
|| addresses.len() > 32
|
||||
|| addresses.iter().any(|address| {
|
||||
address.port() == 0
|
||||
|| !address.is_ipv4()
|
||||
|| address.ip().is_unspecified()
|
||||
|| address.ip().is_multicast()
|
||||
|| matches!(address.ip(), std::net::IpAddr::V4(ip) if ip.is_broadcast())
|
||||
})
|
||||
{
|
||||
return Err(error("invalid AWG endpoint"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
impl AwgTransport {
|
||||
/// Starts the coordinator. Production binds wildcard IPv4; tests use loopback.
|
||||
pub fn new(local: EndpointId, bind: Ipv4Addr) -> Arc<Self> {
|
||||
Self::new_with_stun(local, bind, None)
|
||||
}
|
||||
/// Enables an optional STUN server (host:port) for this pair's own socket.
|
||||
/// Local-only callers pass None; failure still permits local UDP candidates.
|
||||
pub fn new_with_stun(local: EndpointId, bind: Ipv4Addr, stun: Option<String>) -> Arc<Self> {
|
||||
let (commands, mut rx) = mpsc::channel(64);
|
||||
let (inbound, incoming) = mpsc::channel(64);
|
||||
let (stopped, _) = watch::channel(false);
|
||||
let inner = Arc::new(Inner {
|
||||
stopped,
|
||||
local,
|
||||
bind,
|
||||
stun,
|
||||
busy: Mutex::default(),
|
||||
control: OnceLock::new(),
|
||||
allowed: Mutex::default(),
|
||||
pending: Mutex::default(),
|
||||
links: Mutex::default(),
|
||||
inbound,
|
||||
port_by_pair: Mutex::default(),
|
||||
});
|
||||
let weak = Arc::downgrade(&inner);
|
||||
let mut stopped = inner.stopped.subscribe();
|
||||
tokio::spawn(async move {
|
||||
while let Some((network, peer, message)) =
|
||||
tokio::select! { biased; _=stopped.changed()=>None, message=rx.recv()=>message }
|
||||
{
|
||||
let Some(inner) = weak.upgrade() else {
|
||||
break;
|
||||
};
|
||||
if !lock(&inner.allowed).contains(&(network, peer))
|
||||
|| !lock(&inner.busy).insert((network, peer))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
tokio::spawn(async move {
|
||||
let mut stop = inner.stopped.subscribe();
|
||||
tokio::select! {biased; _=stop.changed()=>{}, _=async {
|
||||
if let Err(err)=receive(&inner,(network,peer),message).await {tracing::debug!(%err,"AWG pair negotiation rejected");}
|
||||
}=>{}}
|
||||
lock(&inner.busy).remove(&(network, peer));
|
||||
});
|
||||
}
|
||||
});
|
||||
Arc::new(Self {
|
||||
inner,
|
||||
commands,
|
||||
incoming: tokio::sync::Mutex::new(incoming),
|
||||
})
|
||||
}
|
||||
/// Registers a peer after its authenticated capability has been validated.
|
||||
pub fn allow_peer(&self, network: NetworkId, peer: EndpointId) {
|
||||
let mut allowed = lock(&self.inner.allowed);
|
||||
if !*self.inner.stopped.borrow() && allowed.len() < 128 {
|
||||
allowed.insert((network, peer));
|
||||
}
|
||||
}
|
||||
/// Retires a pair when its network membership session goes away.
|
||||
pub fn forget_peer(&self, network: NetworkId, peer: EndpointId) {
|
||||
let pair = (network, peer);
|
||||
lock(&self.inner.allowed).remove(&pair);
|
||||
lock(&self.inner.pending).remove(&pair);
|
||||
lock(&self.inner.port_by_pair).remove(&pair);
|
||||
if let Some(link) = lock(&self.inner.links).remove(&pair) {
|
||||
link.close();
|
||||
}
|
||||
}
|
||||
/// Stops all pair engines in this network.
|
||||
pub fn stop_network(&self, network: NetworkId) {
|
||||
let peers: Vec<_> = lock(&self.inner.allowed)
|
||||
.iter()
|
||||
.filter(|(n, _)| *n == network)
|
||||
.map(|(_, p)| *p)
|
||||
.collect();
|
||||
for peer in peers {
|
||||
self.forget_peer(network, peer);
|
||||
}
|
||||
}
|
||||
/// Stops all sockets and coordinator work without waiting for remote peers.
|
||||
pub fn shutdown(&self) {
|
||||
self.inner.stopped.send_replace(true);
|
||||
lock(&self.inner.allowed).clear();
|
||||
lock(&self.inner.pending).clear();
|
||||
for (_, link) in lock(&self.inner.links).drain() {
|
||||
link.close();
|
||||
}
|
||||
lock(&self.inner.port_by_pair).clear();
|
||||
}
|
||||
/// Receives a successfully negotiated and authenticated inbound pair link.
|
||||
pub async fn next_inbound(&self) -> Option<InboundLink> {
|
||||
let mut stopped = self.inner.stopped.subscribe();
|
||||
if *stopped.borrow() {
|
||||
return None;
|
||||
}
|
||||
let mut incoming = self.incoming.lock().await;
|
||||
tokio::select! {biased; _=stopped.changed()=>None, link=incoming.recv()=>link}
|
||||
}
|
||||
/// Takes a public snapshot of this network's pair attempts.
|
||||
pub fn overview(&self, network: NetworkId) -> Vec<PairOverview> {
|
||||
let links = lock(&self.inner.links);
|
||||
let ports = lock(&self.inner.port_by_pair);
|
||||
links
|
||||
.iter()
|
||||
.filter(|((n, _), _)| *n == network)
|
||||
.map(|((_, peer), link)| PairOverview {
|
||||
peer: *peer,
|
||||
attempt: link.attempt,
|
||||
local_port: ports.get(&(network, *peer)).copied(),
|
||||
path: link.path_description(),
|
||||
closed: link.is_closed(),
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
impl PacketTransport for AwgTransport {
|
||||
fn name(&self) -> &str {
|
||||
AWG_PROTOCOL
|
||||
}
|
||||
fn as_any(&self) -> &dyn std::any::Any {
|
||||
self
|
||||
}
|
||||
fn attach_control(&self, control: Arc<dyn ControlSender>) {
|
||||
let _ = self.inner.control.set(control);
|
||||
}
|
||||
fn on_control(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
protocol: &str,
|
||||
payload: &[u8],
|
||||
) -> Result<(), TransportError> {
|
||||
if protocol != AWG_PROTOCOL || !lock(&self.inner.allowed).contains(&(network, peer)) {
|
||||
return Err(TransportError::Declined(protocol.into()));
|
||||
}
|
||||
if payload.len() > 4096 {
|
||||
return Err(error("AWG negotiation too large"));
|
||||
}
|
||||
let message: Negotiation =
|
||||
postcard::from_bytes(payload).map_err(|_| error("malformed AWG negotiation"))?;
|
||||
match &message {
|
||||
Negotiation::Offer {
|
||||
version,
|
||||
profile,
|
||||
key,
|
||||
addresses,
|
||||
} => {
|
||||
if *version != AWG_WIRE_VERSION || peer.as_bytes() >= self.inner.local.as_bytes() {
|
||||
return Err(error("invalid AWG offer role/version"));
|
||||
}
|
||||
profile.validate().map_err(error)?;
|
||||
validate_endpoint(key, addresses)?;
|
||||
}
|
||||
Negotiation::Answer { key, addresses, .. } => validate_endpoint(key, addresses)?,
|
||||
}
|
||||
self.commands
|
||||
.try_send((network, peer, message))
|
||||
.map_err(|_| error("AWG negotiation queue full"))
|
||||
}
|
||||
fn open<'a>(
|
||||
&'a self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
protocol: &'a str,
|
||||
) -> BoxFuture<'a, Result<SharedLink, TransportError>> {
|
||||
Box::pin(async move {
|
||||
let pair = (network, peer);
|
||||
if protocol != AWG_PROTOCOL
|
||||
|| self.inner.local.as_bytes() >= peer.as_bytes()
|
||||
|| !lock(&self.inner.allowed).contains(&pair)
|
||||
{
|
||||
return Err(TransportError::Declined(protocol.into()));
|
||||
}
|
||||
let control = self
|
||||
.inner
|
||||
.control
|
||||
.get()
|
||||
.ok_or_else(|| error("AWG control plane unavailable"))?;
|
||||
let socket = UdpSocket::bind((self.inner.bind, 0))
|
||||
.await
|
||||
.map_err(|_| error("AWG UDP bind failed"))?;
|
||||
let profile = PairProfile::generate();
|
||||
let attempt = profile.id();
|
||||
let key = StaticSecret::from(rand::random::<[u8; 32]>());
|
||||
let public = PublicKey::from(&key).to_bytes();
|
||||
let message = Negotiation::Offer {
|
||||
version: AWG_WIRE_VERSION,
|
||||
profile: profile.clone(),
|
||||
key: public,
|
||||
addresses: addresses(&self.inner, &socket).await?,
|
||||
};
|
||||
let (reply, rx) = oneshot::channel();
|
||||
if lock(&self.inner.pending).len() >= 128 {
|
||||
return Err(error("too many AWG pair negotiations"));
|
||||
}
|
||||
lock(&self.inner.pending).insert(
|
||||
pair,
|
||||
Pending {
|
||||
profile,
|
||||
key,
|
||||
socket,
|
||||
reply,
|
||||
},
|
||||
);
|
||||
// A cancellation guard also retires partially started attempts.
|
||||
let mut guard = AttemptGuard {
|
||||
inner: Arc::downgrade(&self.inner),
|
||||
pair,
|
||||
attempt,
|
||||
completed: false,
|
||||
};
|
||||
let bytes =
|
||||
postcard::to_stdvec(&message).map_err(|_| error("AWG offer encoding failed"))?;
|
||||
control
|
||||
.send(network, peer, AWG_PROTOCOL, TransportPayload::new(bytes))
|
||||
.await?;
|
||||
let result = tokio::time::timeout(Duration::from_secs(15), rx)
|
||||
.await
|
||||
.map_err(|_| error("AWG negotiation/handshake timed out"))?
|
||||
.map_err(|_| error("AWG attempt cancelled"))?;
|
||||
guard.completed = result.is_ok();
|
||||
result
|
||||
})
|
||||
}
|
||||
}
|
||||
struct AttemptGuard {
|
||||
inner: Weak<Inner>,
|
||||
pair: Pair,
|
||||
attempt: [u8; 16],
|
||||
completed: bool,
|
||||
}
|
||||
impl Drop for AttemptGuard {
|
||||
fn drop(&mut self) {
|
||||
if let Some(inner) = self.inner.upgrade() {
|
||||
let mut pending = lock(&inner.pending);
|
||||
if pending
|
||||
.get(&self.pair)
|
||||
.is_some_and(|p| p.profile.id() == self.attempt)
|
||||
{
|
||||
pending.remove(&self.pair);
|
||||
}
|
||||
drop(pending);
|
||||
if !self.completed
|
||||
&& let Some(link) = lock(&inner.links).get(&self.pair)
|
||||
&& link.attempt == self.attempt
|
||||
{
|
||||
link.close();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
async fn receive(
|
||||
inner: &Arc<Inner>,
|
||||
pair: Pair,
|
||||
message: Negotiation,
|
||||
) -> Result<(), TransportError> {
|
||||
if !lock(&inner.allowed).contains(&pair) {
|
||||
return Err(error("AWG peer no longer authenticated"));
|
||||
}
|
||||
match message {
|
||||
Negotiation::Offer {
|
||||
profile,
|
||||
key,
|
||||
addresses: candidates,
|
||||
..
|
||||
} => {
|
||||
if let Some(link) = lock(&inner.links).get(&pair) {
|
||||
if link.attempt == profile.id() {
|
||||
return Err(error("duplicate AWG offer"));
|
||||
}
|
||||
link.close();
|
||||
}
|
||||
let socket = UdpSocket::bind((inner.bind, 0))
|
||||
.await
|
||||
.map_err(|_| error("AWG UDP bind failed"))?;
|
||||
let local_port = socket
|
||||
.local_addr()
|
||||
.map_err(|_| error("AWG socket address unavailable"))?
|
||||
.port();
|
||||
let local_key = StaticSecret::from(rand::random::<[u8; 32]>());
|
||||
let public = PublicKey::from(&local_key).to_bytes();
|
||||
let answer = Negotiation::Answer {
|
||||
attempt: profile.id(),
|
||||
key: public,
|
||||
addresses: addresses(inner, &socket).await?,
|
||||
};
|
||||
let tunn = Tunn::new(
|
||||
local_key,
|
||||
PublicKey::from(key),
|
||||
Some(profile.psk),
|
||||
Some(1),
|
||||
rand::random(),
|
||||
None,
|
||||
);
|
||||
|
||||
let link = session::start(
|
||||
pair.0,
|
||||
pair.1,
|
||||
socket,
|
||||
tunn,
|
||||
profile,
|
||||
candidates,
|
||||
inner.stopped.subscribe(),
|
||||
);
|
||||
if !lock(&inner.allowed).contains(&pair) {
|
||||
link.close();
|
||||
return Err(error("AWG membership ended"));
|
||||
}
|
||||
lock(&inner.links).insert(pair, link.clone());
|
||||
lock(&inner.port_by_pair).insert(pair, local_port);
|
||||
let control = inner
|
||||
.control
|
||||
.get()
|
||||
.ok_or_else(|| error("AWG control unavailable"))?;
|
||||
if let Err(err) = control
|
||||
.send(
|
||||
pair.0,
|
||||
pair.1,
|
||||
AWG_PROTOCOL,
|
||||
TransportPayload::new(
|
||||
postcard::to_stdvec(&answer)
|
||||
.map_err(|_| error("AWG answer encoding failed"))?,
|
||||
),
|
||||
)
|
||||
.await
|
||||
{
|
||||
link.close();
|
||||
return Err(err);
|
||||
}
|
||||
publish_when_up(Arc::downgrade(inner), pair, link, None);
|
||||
}
|
||||
Negotiation::Answer {
|
||||
attempt,
|
||||
key,
|
||||
addresses: candidates,
|
||||
} => {
|
||||
if !lock(&inner.pending)
|
||||
.get(&pair)
|
||||
.is_some_and(|p| p.profile.id() == attempt)
|
||||
{
|
||||
return Err(error("stale AWG answer"));
|
||||
}
|
||||
let pending = lock(&inner.pending)
|
||||
.remove(&pair)
|
||||
.ok_or_else(|| error("AWG attempt cancelled"))?;
|
||||
let local_port = pending
|
||||
.socket
|
||||
.local_addr()
|
||||
.map_err(|_| error("AWG socket address unavailable"))?
|
||||
.port();
|
||||
let tunn = Tunn::new(
|
||||
pending.key,
|
||||
PublicKey::from(key),
|
||||
Some(pending.profile.psk),
|
||||
Some(1),
|
||||
rand::random(),
|
||||
None,
|
||||
);
|
||||
|
||||
let link = session::start(
|
||||
pair.0,
|
||||
pair.1,
|
||||
pending.socket,
|
||||
tunn,
|
||||
pending.profile,
|
||||
candidates,
|
||||
inner.stopped.subscribe(),
|
||||
);
|
||||
if !lock(&inner.allowed).contains(&pair) {
|
||||
link.close();
|
||||
return Err(error("AWG membership ended"));
|
||||
}
|
||||
lock(&inner.links).insert(pair, link.clone());
|
||||
lock(&inner.port_by_pair).insert(pair, local_port);
|
||||
publish_when_up(Arc::downgrade(inner), pair, link, Some(pending.reply));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
fn publish_when_up(
|
||||
inner: Weak<Inner>,
|
||||
pair: Pair,
|
||||
link: Arc<PairLink>,
|
||||
reply: Option<oneshot::Sender<Result<SharedLink, TransportError>>>,
|
||||
) {
|
||||
tokio::spawn(async move {
|
||||
let mut reply = reply;
|
||||
let result = tokio::time::timeout(Duration::from_secs(14), async {
|
||||
loop {
|
||||
if link.is_closed() || reply.as_ref().is_some_and(oneshot::Sender::is_closed) {
|
||||
return Err(error("AWG pair closed during handshake"));
|
||||
}
|
||||
if link.has_authenticated_path() {
|
||||
return Ok(());
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(25)).await;
|
||||
}
|
||||
})
|
||||
.await;
|
||||
if !matches!(result, Ok(Ok(()))) {
|
||||
link.close();
|
||||
if let Some(reply) = reply.take() {
|
||||
let _ = reply.send(Err(error("AWG authenticated data path timed out")));
|
||||
}
|
||||
return;
|
||||
}
|
||||
let Some(inner) = inner.upgrade() else {
|
||||
link.close();
|
||||
return;
|
||||
};
|
||||
if !lock(&inner.allowed).contains(&pair) {
|
||||
link.close();
|
||||
return;
|
||||
}
|
||||
if link.is_closed()
|
||||
|| !lock(&inner.links)
|
||||
.get(&pair)
|
||||
.is_some_and(|current| current.attempt == link.attempt)
|
||||
{
|
||||
link.close();
|
||||
return;
|
||||
}
|
||||
let shared: SharedLink = link.clone();
|
||||
if let Some(reply) = reply {
|
||||
if reply.send(Ok(shared)).is_err() {
|
||||
link.close();
|
||||
}
|
||||
} else if inner
|
||||
.inbound
|
||||
.try_send(InboundLink {
|
||||
network: pair.0,
|
||||
peer: pair.1,
|
||||
protocol: AWG_PROTOCOL.into(),
|
||||
link: shared,
|
||||
})
|
||||
.is_err()
|
||||
{
|
||||
link.close();
|
||||
}
|
||||
});
|
||||
}
|
||||
impl Drop for AwgTransport {
|
||||
fn drop(&mut self) {
|
||||
self.shutdown();
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
#[tokio::test]
|
||||
async fn concurrent_admission_is_bounded_across_networks_and_retirement_releases_slots() {
|
||||
let identity = tsunagi::identity::DeviceIdentity::generate();
|
||||
let transport = AwgTransport::new(identity.endpoint_id(), Ipv4Addr::LOCALHOST);
|
||||
let peer = tsunagi::identity::DeviceIdentity::generate().endpoint_id();
|
||||
std::thread::scope(|scope| {
|
||||
for thread in 0..8u8 {
|
||||
let transport = &transport;
|
||||
scope.spawn(move || {
|
||||
for item in 0..32u8 {
|
||||
let mut id = [0; 32];
|
||||
id[0] = thread;
|
||||
id[1] = item;
|
||||
transport.allow_peer(NetworkId::from_bytes(id), peer);
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
assert_eq!(lock(&transport.inner.allowed).len(), 128);
|
||||
let network = lock(&transport.inner.allowed).iter().next().unwrap().0;
|
||||
transport.stop_network(network);
|
||||
let new = NetworkId::from_bytes([255; 32]);
|
||||
transport.allow_peer(new, peer);
|
||||
assert!(lock(&transport.inner.allowed).contains(&(new, peer)));
|
||||
assert_eq!(lock(&transport.inner.allowed).len(), 128);
|
||||
transport.shutdown();
|
||||
assert!(lock(&transport.inner.allowed).is_empty());
|
||||
}
|
||||
#[tokio::test]
|
||||
async fn private_negotiation_bounds_roles_and_membership_before_starting_engines() {
|
||||
let mut ids = [
|
||||
tsunagi::identity::DeviceIdentity::generate().endpoint_id(),
|
||||
tsunagi::identity::DeviceIdentity::generate().endpoint_id(),
|
||||
];
|
||||
ids.sort_by_key(|id| *id.as_bytes());
|
||||
let peer = ids[0];
|
||||
let local = ids[1];
|
||||
let network = NetworkId::from_bytes([7; 32]);
|
||||
let transport = AwgTransport::new(local, Ipv4Addr::LOCALHOST);
|
||||
let offer = |version, key, addresses| Negotiation::Offer {
|
||||
version,
|
||||
profile: PairProfile::generate(),
|
||||
key,
|
||||
addresses,
|
||||
};
|
||||
let address = SocketAddr::from((Ipv4Addr::LOCALHOST, 12345));
|
||||
let encode = |message: &Negotiation| postcard::to_stdvec(message).unwrap();
|
||||
let valid = encode(&offer(AWG_WIRE_VERSION, [1; 32], vec![address]));
|
||||
assert!(
|
||||
transport
|
||||
.on_control(network, peer, AWG_PROTOCOL, &valid)
|
||||
.is_err()
|
||||
);
|
||||
transport.allow_peer(network, peer);
|
||||
assert!(
|
||||
transport
|
||||
.on_control(network, peer, "wrong-protocol", &valid)
|
||||
.is_err()
|
||||
);
|
||||
assert!(
|
||||
transport
|
||||
.on_control(network, peer, AWG_PROTOCOL, &[0; 4097])
|
||||
.is_err()
|
||||
);
|
||||
for bytes in [
|
||||
encode(&offer(99, [1; 32], vec![address])),
|
||||
encode(&offer(AWG_WIRE_VERSION, [0; 32], vec![address])),
|
||||
encode(&offer(AWG_WIRE_VERSION, [1; 32], vec![])),
|
||||
encode(&offer(AWG_WIRE_VERSION, [1; 32], vec![address; 33])),
|
||||
] {
|
||||
assert!(
|
||||
transport
|
||||
.on_control(network, peer, AWG_PROTOCOL, &bytes)
|
||||
.is_err()
|
||||
);
|
||||
}
|
||||
for len in 0..valid.len() {
|
||||
assert!(
|
||||
transport
|
||||
.on_control(network, peer, AWG_PROTOCOL, &valid[..len])
|
||||
.is_err()
|
||||
);
|
||||
}
|
||||
let wrong_role = AwgTransport::new(peer, Ipv4Addr::LOCALHOST);
|
||||
wrong_role.allow_peer(network, local);
|
||||
assert!(
|
||||
wrong_role
|
||||
.on_control(network, local, AWG_PROTOCOL, &valid)
|
||||
.is_err()
|
||||
);
|
||||
transport.forget_peer(network, peer);
|
||||
assert!(
|
||||
transport
|
||||
.on_control(network, peer, AWG_PROTOCOL, &valid)
|
||||
.is_err()
|
||||
);
|
||||
assert!(transport.overview(network).is_empty());
|
||||
transport.shutdown();
|
||||
wrong_role.shutdown();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,773 @@
|
||||
//! Real membership, UDP sockets, pair negotiation and nested WireGuard encryption.
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use bytes::Bytes;
|
||||
use std::{
|
||||
net::Ipv4Addr,
|
||||
path::Path,
|
||||
sync::{
|
||||
Arc,
|
||||
atomic::{AtomicBool, AtomicU64, Ordering},
|
||||
},
|
||||
time::Duration,
|
||||
};
|
||||
use tempfile::TempDir;
|
||||
use tsunagi::dataplane::IpPlugin;
|
||||
use tsunagi::dataplane::transport::{
|
||||
ControlSender, PacketLink, PacketTransport, SharedLink, TransportError,
|
||||
};
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::overlay::{MemoryTun, MemoryTunFactory};
|
||||
use tsunagi::testing::{config_with, network, wait_for_peers, wait_until};
|
||||
use tsunagi::{Agent, BoxFuture, NetworkId};
|
||||
use tsunagi_awg::{AWG_PROTOCOL, AwgPlugin, AwgTransport};
|
||||
use tsunagi_wg_quic::WireguardConfig;
|
||||
|
||||
struct Node {
|
||||
agent: Agent,
|
||||
plugin: Arc<AwgPlugin>,
|
||||
quic: Option<Arc<tsunagi_wg_quic::WireguardPlugin>>,
|
||||
transport: Arc<AwgTransport>,
|
||||
tuns: MemoryTunFactory,
|
||||
loss: Arc<AtomicBool>,
|
||||
}
|
||||
impl Node {
|
||||
async fn open(root: &Path, discovery: &SharedMemoryDiscovery, interface: &str) -> Self {
|
||||
Self::open_with_fallback(root, discovery, interface, false).await
|
||||
}
|
||||
async fn open_with_fallback(
|
||||
root: &Path,
|
||||
discovery: &SharedMemoryDiscovery,
|
||||
interface: &str,
|
||||
fallback: bool,
|
||||
) -> Self {
|
||||
Self::open_config(root, discovery, interface, fallback, 1280).await
|
||||
}
|
||||
async fn open_config(
|
||||
root: &Path,
|
||||
discovery: &SharedMemoryDiscovery,
|
||||
interface: &str,
|
||||
fallback: bool,
|
||||
mtu: u32,
|
||||
) -> Self {
|
||||
Self::open_selected(root, discovery, interface, fallback, mtu, None).await
|
||||
}
|
||||
async fn open_selected(
|
||||
root: &Path,
|
||||
discovery: &SharedMemoryDiscovery,
|
||||
interface: &str,
|
||||
fallback: bool,
|
||||
mtu: u32,
|
||||
default_protocols: Option<Vec<String>>,
|
||||
) -> Self {
|
||||
let identity =
|
||||
tsunagi::storage::StateStore::open(tsunagi::StoragePaths::under(root).state_db())
|
||||
.unwrap()
|
||||
.load_or_create_device_identity()
|
||||
.unwrap();
|
||||
let transport = AwgTransport::new(identity.endpoint_id(), Ipv4Addr::LOCALHOST);
|
||||
let plugin = AwgPlugin::open(
|
||||
WireguardConfig::new(root.join("awg"))
|
||||
.with_mtu(mtu)
|
||||
.with_reconcile(Duration::from_millis(20), Duration::from_millis(250)),
|
||||
transport.clone(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let loss = Arc::new(AtomicBool::new(false));
|
||||
let faults = Arc::new(FaultTransport {
|
||||
inner: transport.clone(),
|
||||
loss: loss.clone(),
|
||||
});
|
||||
let tuns = MemoryTunFactory::new();
|
||||
let mut config = config_with(root, discovery)
|
||||
.with_interface(Arc::new(tuns.clone()), interface, mtu)
|
||||
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>)
|
||||
.with_custom_transport(AWG_PROTOCOL, faults);
|
||||
let mut quic_plugin = None;
|
||||
if fallback {
|
||||
let quic = tsunagi_wg_quic::WireguardPlugin::open(
|
||||
WireguardConfig::new(root.join("wg-quic"))
|
||||
.with_mtu(mtu)
|
||||
.with_reconcile(Duration::from_millis(20), Duration::from_millis(250)),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
config = config.with_plugin(quic.clone());
|
||||
quic_plugin = Some(quic);
|
||||
}
|
||||
config.default_protocols = default_protocols;
|
||||
let agent = Agent::spawn(config).await.unwrap();
|
||||
let inbound_agent = agent.clone();
|
||||
let inbound_transport = transport.clone();
|
||||
let inbound_loss = loss.clone();
|
||||
tokio::spawn(async move {
|
||||
while let Some(mut link) = inbound_transport.next_inbound().await {
|
||||
link.link = impaired(link.link, inbound_loss.clone());
|
||||
let _ = inbound_agent.install_inbound_link(link).await;
|
||||
}
|
||||
});
|
||||
Self {
|
||||
agent,
|
||||
plugin,
|
||||
quic: quic_plugin,
|
||||
transport,
|
||||
tuns,
|
||||
loss,
|
||||
}
|
||||
}
|
||||
async fn ready(&self, id: NetworkId, n: usize) {
|
||||
wait_until("AWG outer paths and end-to-end handshakes", || async {
|
||||
(self
|
||||
.transport
|
||||
.overview(id)
|
||||
.iter()
|
||||
.filter(|p| !p.closed && !p.path.contains("establishing"))
|
||||
.count()
|
||||
== n
|
||||
&& self
|
||||
.plugin
|
||||
.overview(id)?
|
||||
.peers
|
||||
.iter()
|
||||
.filter(|p| {
|
||||
p.tunnel
|
||||
.as_ref()
|
||||
.is_some_and(|t| t.health.is_up() && t.path.contains("awg UDP via"))
|
||||
})
|
||||
.count()
|
||||
== n)
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
}
|
||||
fn tun(&self) -> Arc<MemoryTun> {
|
||||
self.tuns
|
||||
.device(&self.agent.overlay().unwrap().interface)
|
||||
.unwrap()
|
||||
}
|
||||
fn addr(&self, id: NetworkId) -> Ipv4Addr {
|
||||
self.plugin
|
||||
.overview(id)
|
||||
.unwrap()
|
||||
.overlay_address_v4
|
||||
.unwrap()
|
||||
}
|
||||
}
|
||||
fn packet(source: Ipv4Addr, dest: Ipv4Addr, len: usize) -> Bytes {
|
||||
let mut bytes = vec![0; len];
|
||||
bytes[0] = 0x45;
|
||||
bytes[2..4].copy_from_slice(&(len as u16).to_be_bytes());
|
||||
bytes[8] = 64;
|
||||
bytes[9] = 253;
|
||||
bytes[12..16].copy_from_slice(&source.octets());
|
||||
bytes[16..20].copy_from_slice(&dest.octets());
|
||||
rand::fill(&mut bytes[20..]);
|
||||
Bytes::from(bytes)
|
||||
}
|
||||
async fn exchange(a: &Node, b: &Node, id: NetworkId) {
|
||||
for len in [64, 1097, 1280] {
|
||||
let bytes = packet(a.addr(id), b.addr(id), len);
|
||||
a.tun().push_from_os(bytes.clone());
|
||||
assert_eq!(
|
||||
tokio::time::timeout(Duration::from_secs(5), b.tun().pop_to_os())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap(),
|
||||
bytes
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_control_only_agent_enables_real_data_without_restart_and_cancels_stale_attempts() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let roots = [TempDir::new().unwrap(), TempDir::new().unwrap()];
|
||||
let a = Node::open_selected(
|
||||
roots[0].path(),
|
||||
&discovery,
|
||||
"initial-none-a",
|
||||
true,
|
||||
1280,
|
||||
Some(vec![]),
|
||||
)
|
||||
.await;
|
||||
let b = Node::open_selected(
|
||||
roots[1].path(),
|
||||
&discovery,
|
||||
"initial-none-b",
|
||||
true,
|
||||
1280,
|
||||
Some(vec![]),
|
||||
)
|
||||
.await;
|
||||
let (name, secret) = network("enable-after-control-only");
|
||||
let id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
wait_for_peers(&a.agent, id, 1).await;
|
||||
wait_for_peers(&b.agent, id, 1).await;
|
||||
assert!(a.agent.overlay().is_none());
|
||||
assert!(b.agent.overlay().is_none());
|
||||
for node in [&a, &b] {
|
||||
assert!(
|
||||
node.agent
|
||||
.network_status(id)
|
||||
.await
|
||||
.unwrap()
|
||||
.protocols
|
||||
.is_empty()
|
||||
);
|
||||
node.agent
|
||||
.set_protocols(id, vec!["awg".into()])
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(node.agent.overlay().is_some());
|
||||
}
|
||||
a.ready(id, 1).await;
|
||||
b.ready(id, 1).await;
|
||||
exchange(&a, &b, id).await;
|
||||
let before = a.agent.overlay().unwrap().interface;
|
||||
// Changes race unfinished opens and prepared plugin callbacks; the last
|
||||
// selection must win even if an earlier open has completed in its queue.
|
||||
for selection in [
|
||||
vec!["wg-quic".into()],
|
||||
vec![],
|
||||
vec!["awg".into()],
|
||||
vec![],
|
||||
vec!["awg".into()],
|
||||
] {
|
||||
a.agent.set_protocols(id, selection).await.unwrap();
|
||||
}
|
||||
a.ready(id, 1).await;
|
||||
b.ready(id, 1).await;
|
||||
exchange(&a, &b, id).await;
|
||||
assert_eq!(a.agent.overlay().unwrap().interface, before);
|
||||
for node in [&a, &b] {
|
||||
let status = node.agent.network_status(id).await.unwrap();
|
||||
assert_eq!(status.protocols, ["awg"]);
|
||||
assert_eq!(status.metrics.sessions_established, 1);
|
||||
assert_eq!(status.metrics.disconnects, 0);
|
||||
}
|
||||
a.agent.shutdown().await;
|
||||
b.agent.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn runtime_protocol_selection_replaces_real_tunnels_and_survives_restart() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let roots = [TempDir::new().unwrap(), TempDir::new().unwrap()];
|
||||
let a = Node::open_with_fallback(roots[0].path(), &discovery, "runtime-a", true).await;
|
||||
let b = Node::open_with_fallback(roots[1].path(), &discovery, "runtime-b", true).await;
|
||||
let (name, secret) = network("runtime-protocols");
|
||||
let id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
a.ready(id, 1).await;
|
||||
b.ready(id, 1).await;
|
||||
exchange(&a, &b, id).await;
|
||||
|
||||
let (other_name, other_secret, other_id) = loop {
|
||||
let (name, secret) = network("unchanged-runtime-network");
|
||||
let other_id = tsunagi::identity::NetworkKeys::derive(&name, &secret).network_id();
|
||||
if tsunagi::state::derived_ipv4_range(id) != tsunagi::state::derived_ipv4_range(other_id) {
|
||||
break (name, secret, other_id);
|
||||
}
|
||||
};
|
||||
a.agent
|
||||
.join_network(&other_name, &other_secret)
|
||||
.await
|
||||
.unwrap();
|
||||
b.agent
|
||||
.join_network(&other_name, &other_secret)
|
||||
.await
|
||||
.unwrap();
|
||||
a.ready(other_id, 1).await;
|
||||
b.ready(other_id, 1).await;
|
||||
let other_before = a
|
||||
.agent
|
||||
.network_status(other_id)
|
||||
.await
|
||||
.unwrap()
|
||||
.metrics
|
||||
.data_links_established;
|
||||
|
||||
for node in [&a, &b] {
|
||||
let before = node.agent.network_status(id).await.unwrap().protocols;
|
||||
for invalid in [vec!["unknown".into()], vec!["awg".into(), "awg".into()]] {
|
||||
assert!(node.agent.set_protocols(id, invalid).await.is_err());
|
||||
assert_eq!(
|
||||
node.agent.network_status(id).await.unwrap().protocols,
|
||||
before
|
||||
);
|
||||
}
|
||||
}
|
||||
// Exercise updates initiated by either endpoint-id role, including an
|
||||
// empty set. Membership and the control session stay alive throughout.
|
||||
for node in [&a, &b] {
|
||||
node.agent.set_protocols(id, Vec::new()).await.unwrap();
|
||||
wait_until("peer accepts control-only selection", || async {
|
||||
let other = if node.agent.endpoint_id() == a.agent.endpoint_id() {
|
||||
&b
|
||||
} else {
|
||||
&a
|
||||
};
|
||||
let status = other.agent.network_status(id).await.ok()?;
|
||||
(status.peers.len() == 1
|
||||
&& status.peers[0].protocols.is_empty()
|
||||
&& matches!(status.peers[0].data_path, tsunagi::agent::DataPath::None))
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
assert_eq!(node.agent.network_status(id).await.unwrap().peers.len(), 1);
|
||||
node.agent
|
||||
.set_protocols(id, vec!["wg-quic".into(), "awg".into()])
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
a.agent
|
||||
.set_protocols(id, vec!["wg-quic".into()])
|
||||
.await
|
||||
.unwrap();
|
||||
wait_until("real QUIC replaces AWG", || async {
|
||||
let a_status = a.agent.network_status(id).await.ok()?;
|
||||
let b_status = b.agent.network_status(id).await.ok()?;
|
||||
(a_status.peers[0].protocols == ["wg-quic"]
|
||||
&& b_status.peers[0].protocols == ["wg-quic"]
|
||||
&& !matches!(a_status.peers[0].data_path, tsunagi::agent::DataPath::None)
|
||||
&& !matches!(b_status.peers[0].data_path, tsunagi::agent::DataPath::None)
|
||||
&& a.transport.overview(id).iter().all(|p| p.closed)
|
||||
&& b.transport.overview(id).iter().all(|p| p.closed))
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
// Get addresses from the system allocation: the disabled AWG plugin no
|
||||
// longer has a live view of this network.
|
||||
for (source, dest) in [(&a, &b), (&b, &a)] {
|
||||
wait_until("QUIC end-to-end handshake", || async {
|
||||
source
|
||||
.quic
|
||||
.as_ref()?
|
||||
.overview(id)?
|
||||
.peers
|
||||
.iter()
|
||||
.any(|p| p.tunnel.as_ref().is_some_and(|t| t.health.is_up()))
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
let addr = |n: &Node| {
|
||||
n.quic
|
||||
.as_ref()
|
||||
.unwrap()
|
||||
.overview(id)
|
||||
.unwrap()
|
||||
.overlay_address_v4
|
||||
.unwrap()
|
||||
};
|
||||
let bytes = packet(addr(source), addr(dest), 1280);
|
||||
source.tun().push_from_os(bytes.clone());
|
||||
assert_eq!(
|
||||
tokio::time::timeout(Duration::from_secs(5), dest.tun().pop_to_os())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap(),
|
||||
bytes
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
a.agent
|
||||
.network_status(other_id)
|
||||
.await
|
||||
.unwrap()
|
||||
.metrics
|
||||
.data_links_established,
|
||||
other_before
|
||||
);
|
||||
exchange(&a, &b, other_id).await;
|
||||
a.agent.shutdown().await;
|
||||
let restarted = Node::open_with_fallback(roots[0].path(), &discovery, "runtime-a", true).await;
|
||||
wait_for_peers(&restarted.agent, id, 1).await;
|
||||
assert_eq!(
|
||||
restarted.agent.network_status(id).await.unwrap().protocols,
|
||||
["wg-quic"]
|
||||
);
|
||||
assert_eq!(
|
||||
b.agent.network_status(other_id).await.unwrap().protocols,
|
||||
["awg", "wg-quic"]
|
||||
);
|
||||
restarted.ready(other_id, 1).await;
|
||||
b.ready(other_id, 1).await;
|
||||
exchange(&restarted, &b, other_id).await;
|
||||
restarted
|
||||
.agent
|
||||
.set_protocols(id, vec!["awg".into()])
|
||||
.await
|
||||
.unwrap();
|
||||
restarted.ready(id, 1).await;
|
||||
b.ready(id, 1).await;
|
||||
exchange(&restarted, &b, id).await;
|
||||
restarted.agent.shutdown().await;
|
||||
b.agent.shutdown().await;
|
||||
}
|
||||
#[tokio::test]
|
||||
async fn real_pairs_carry_full_size_packets_and_restart_with_fresh_profiles() {
|
||||
tsunagi::testing::init_tracing();
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let roots = [
|
||||
TempDir::new().unwrap(),
|
||||
TempDir::new().unwrap(),
|
||||
TempDir::new().unwrap(),
|
||||
];
|
||||
let a = Node::open(roots[0].path(), &discovery, "awga").await;
|
||||
let b = Node::open(roots[1].path(), &discovery, "awgb").await;
|
||||
let c = Node::open(roots[2].path(), &discovery, "awgc").await;
|
||||
let (name, secret) = network("awg-pairs");
|
||||
let id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
c.agent.join_network(&name, &secret).await.unwrap();
|
||||
for node in [&a, &b, &c] {
|
||||
wait_for_peers(&node.agent, id, 2).await;
|
||||
node.ready(id, 2).await;
|
||||
}
|
||||
let ids: std::collections::HashSet<_> = [&a, &b, &c]
|
||||
.into_iter()
|
||||
.flat_map(|n| n.transport.overview(id).into_iter().map(|p| p.attempt))
|
||||
.collect();
|
||||
assert_eq!(
|
||||
ids.len(),
|
||||
3,
|
||||
"one independent profile for each of three pairs"
|
||||
);
|
||||
exchange(&a, &b, id).await;
|
||||
exchange(&b, &a, id).await;
|
||||
exchange(&a, &c, id).await;
|
||||
let old_ports: Vec<_> = a
|
||||
.transport
|
||||
.overview(id)
|
||||
.into_iter()
|
||||
.filter_map(|p| p.local_port)
|
||||
.collect();
|
||||
a.agent.shutdown().await;
|
||||
for port in old_ports {
|
||||
wait_until("old pair socket released", || async {
|
||||
tokio::net::UdpSocket::bind((Ipv4Addr::LOCALHOST, port))
|
||||
.await
|
||||
.ok()
|
||||
})
|
||||
.await;
|
||||
}
|
||||
let restarted = Node::open(roots[0].path(), &discovery, "awga").await;
|
||||
restarted.ready(id, 2).await;
|
||||
assert!(
|
||||
restarted
|
||||
.transport
|
||||
.overview(id)
|
||||
.iter()
|
||||
.all(|p| !ids.contains(&p.attempt))
|
||||
);
|
||||
exchange(&restarted, &b, id).await;
|
||||
exchange(&c, &restarted, id).await;
|
||||
restarted.agent.shutdown().await;
|
||||
b.agent.shutdown().await;
|
||||
c.agent.shutdown().await;
|
||||
assert!(
|
||||
tokio::time::timeout(Duration::from_secs(1), restarted.transport.next_inbound())
|
||||
.await
|
||||
.unwrap()
|
||||
.is_none()
|
||||
);
|
||||
}
|
||||
|
||||
// Drops datagrams at the link boundary. Membership, control, sockets, crypto,
|
||||
// addressing and storage remain real. This substitutes a lossy physical path.
|
||||
#[derive(Debug)]
|
||||
struct FaultTransport {
|
||||
inner: Arc<AwgTransport>,
|
||||
loss: Arc<AtomicBool>,
|
||||
}
|
||||
impl PacketTransport for FaultTransport {
|
||||
fn name(&self) -> &str {
|
||||
AWG_PROTOCOL
|
||||
}
|
||||
fn as_any(&self) -> &dyn std::any::Any {
|
||||
self
|
||||
}
|
||||
fn attach_control(&self, sender: Arc<dyn ControlSender>) {
|
||||
self.inner.attach_control(sender);
|
||||
}
|
||||
fn on_control(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
peer: iroh::EndpointId,
|
||||
protocol: &str,
|
||||
bytes: &[u8],
|
||||
) -> Result<(), TransportError> {
|
||||
self.inner.on_control(network, peer, protocol, bytes)
|
||||
}
|
||||
fn open<'a>(
|
||||
&'a self,
|
||||
network: NetworkId,
|
||||
peer: iroh::EndpointId,
|
||||
protocol: &'a str,
|
||||
) -> BoxFuture<'a, Result<SharedLink, TransportError>> {
|
||||
Box::pin(async move {
|
||||
Ok(impaired(
|
||||
self.inner.open(network, peer, protocol).await?,
|
||||
self.loss.clone(),
|
||||
))
|
||||
})
|
||||
}
|
||||
}
|
||||
fn impaired(raw: SharedLink, loss: Arc<AtomicBool>) -> SharedLink {
|
||||
Arc::new(FaultLink {
|
||||
raw,
|
||||
loss,
|
||||
sequence: AtomicU64::new(0),
|
||||
})
|
||||
}
|
||||
#[derive(Debug)]
|
||||
struct FaultLink {
|
||||
raw: SharedLink,
|
||||
loss: Arc<AtomicBool>,
|
||||
sequence: AtomicU64,
|
||||
}
|
||||
impl PacketLink for FaultLink {
|
||||
fn network(&self) -> NetworkId {
|
||||
self.raw.network()
|
||||
}
|
||||
fn peer(&self) -> iroh::EndpointId {
|
||||
self.raw.peer()
|
||||
}
|
||||
fn max_datagram_size(&self) -> usize {
|
||||
self.raw.max_datagram_size()
|
||||
}
|
||||
fn send(&self, bytes: Bytes) -> Result<(), TransportError> {
|
||||
if self.loss.load(Ordering::Relaxed)
|
||||
&& !self
|
||||
.sequence
|
||||
.fetch_add(1, Ordering::Relaxed)
|
||||
.is_multiple_of(5)
|
||||
{
|
||||
Ok(())
|
||||
} else {
|
||||
self.raw.send(bytes)
|
||||
}
|
||||
}
|
||||
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
|
||||
self.raw.recv()
|
||||
}
|
||||
fn closed(&self) -> BoxFuture<'_, ()> {
|
||||
self.raw.closed()
|
||||
}
|
||||
fn is_closed(&self) -> bool {
|
||||
self.raw.is_closed()
|
||||
}
|
||||
fn close(&self) {
|
||||
self.raw.close();
|
||||
}
|
||||
fn path_description(&self) -> String {
|
||||
self.raw.path_description()
|
||||
}
|
||||
}
|
||||
#[tokio::test]
|
||||
async fn persistent_runtime_loss_retires_the_pair_and_negotiates_fresh_parameters() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let roots = [TempDir::new().unwrap(), TempDir::new().unwrap()];
|
||||
let a = Node::open(roots[0].path(), &discovery, "lossa").await;
|
||||
let b = Node::open(roots[1].path(), &discovery, "lossb").await;
|
||||
let (name, secret) = network("awg-loss");
|
||||
let id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
a.ready(id, 1).await;
|
||||
b.ready(id, 1).await;
|
||||
exchange(&a, &b, id).await;
|
||||
let old = a.transport.overview(id)[0].attempt;
|
||||
let mut events = a.agent.subscribe();
|
||||
a.loss.store(true, Ordering::Relaxed);
|
||||
tsunagi::testing::wait_event(&mut events, |event| match event {
|
||||
tsunagi::Event::DataLinkDown {
|
||||
network, reason, ..
|
||||
} if *network == id && reason.contains("delivery checks") => Some(()),
|
||||
_ => None,
|
||||
})
|
||||
.await;
|
||||
a.loss.store(false, Ordering::Relaxed);
|
||||
wait_until("fresh AWG profile after runtime loss", || async {
|
||||
a.transport
|
||||
.overview(id)
|
||||
.iter()
|
||||
.any(|p| p.attempt != old && !p.closed && !p.path.contains("establishing"))
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
a.ready(id, 1).await;
|
||||
b.ready(id, 1).await;
|
||||
let initiator = if a.agent.endpoint_id().as_bytes() < b.agent.endpoint_id().as_bytes() {
|
||||
&a
|
||||
} else {
|
||||
&b
|
||||
};
|
||||
let received = initiator.plugin.overview(id).unwrap().peers[0]
|
||||
.tunnel
|
||||
.as_ref()
|
||||
.unwrap()
|
||||
.stats
|
||||
.rx_packets;
|
||||
let attempt = initiator.transport.overview(id)[0].attempt;
|
||||
wait_until(
|
||||
"fresh pair delivery checks answered after installation",
|
||||
|| async {
|
||||
let view = initiator.plugin.overview(id)?;
|
||||
(view.peers.first()?.tunnel.as_ref()?.stats.rx_packets >= received + 3).then_some(())
|
||||
},
|
||||
)
|
||||
.await;
|
||||
assert_eq!(initiator.transport.overview(id)[0].attempt, attempt);
|
||||
wait_for_peers(&a.agent, id, 1).await;
|
||||
exchange(&a, &b, id).await;
|
||||
exchange(&b, &a, id).await;
|
||||
a.agent.shutdown().await;
|
||||
b.agent.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn runtime_loss_falls_back_to_real_iroh_without_interrupting_control_or_ip() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let roots = [TempDir::new().unwrap(), TempDir::new().unwrap()];
|
||||
let a = Node::open_with_fallback(roots[0].path(), &discovery, "fallba", true).await;
|
||||
let b = Node::open_with_fallback(roots[1].path(), &discovery, "fallbb", true).await;
|
||||
let (name, secret) = network("awg-fallback");
|
||||
let id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
a.ready(id, 1).await;
|
||||
b.ready(id, 1).await;
|
||||
exchange(&a, &b, id).await;
|
||||
let mut events = a.agent.subscribe();
|
||||
a.loss.store(true, Ordering::Relaxed);
|
||||
tsunagi::testing::wait_event(&mut events, |event| match event {
|
||||
tsunagi::Event::DataLinkDown {
|
||||
network, reason, ..
|
||||
} if *network == id && reason.contains("delivery checks") => Some(()),
|
||||
_ => None,
|
||||
})
|
||||
.await;
|
||||
for node in [&a, &b] {
|
||||
wait_until("fallback carrier is wg-quic",||async {
|
||||
let status=node.agent.status().await.ok()?;
|
||||
let network=status.networks.iter().find(|n|n.network_id==id)?;
|
||||
(network.members.iter().any(|m|matches!(&m.data_path,tsunagi::agent::DataPath::Direct{transport} if transport=="wg-quic"))).then_some(())
|
||||
}).await;
|
||||
wait_for_peers(&node.agent, id, 1).await;
|
||||
}
|
||||
exchange(&a, &b, id).await;
|
||||
exchange(&b, &a, id).await;
|
||||
a.agent.shutdown().await;
|
||||
b.agent.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn opaque_pair_carrier_fragments_its_full_advertised_capacity() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let roots = [TempDir::new().unwrap(), TempDir::new().unwrap()];
|
||||
let mut nodes = Vec::new();
|
||||
for root in &roots {
|
||||
let identity = tsunagi::storage::StateStore::open(
|
||||
tsunagi::StoragePaths::under(root.path()).state_db(),
|
||||
)
|
||||
.unwrap()
|
||||
.load_or_create_device_identity()
|
||||
.unwrap();
|
||||
let transport = AwgTransport::new(identity.endpoint_id(), Ipv4Addr::LOCALHOST);
|
||||
let agent = Agent::spawn(
|
||||
config_with(root.path(), &discovery)
|
||||
.with_custom_transport(AWG_PROTOCOL, transport.clone()),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
nodes.push((agent, transport));
|
||||
}
|
||||
let (name, secret) = network("awg-capacity");
|
||||
let id = nodes[0].0.join_network(&name, &secret).await.unwrap();
|
||||
nodes[1].0.join_network(&name, &secret).await.unwrap();
|
||||
for (i, (agent, transport)) in nodes.iter().enumerate() {
|
||||
wait_for_peers(agent, id, 1).await;
|
||||
// This fixture tests only the transport contract: register the member
|
||||
// after the real core has authenticated it, without an IP plugin/TUN.
|
||||
transport.allow_peer(id, nodes[1 - i].0.endpoint_id());
|
||||
}
|
||||
nodes.sort_by_key(|(agent, _)| *agent.endpoint_id().as_bytes());
|
||||
let outgoing = nodes[0]
|
||||
.1
|
||||
.open(id, nodes[1].0.endpoint_id(), AWG_PROTOCOL)
|
||||
.await
|
||||
.unwrap();
|
||||
let incoming = tokio::time::timeout(Duration::from_secs(5), nodes[1].1.next_inbound())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap()
|
||||
.link;
|
||||
for (from, to) in [(&outgoing, &incoming), (&incoming, &outgoing)] {
|
||||
let mut bytes = vec![0; from.max_datagram_size()];
|
||||
rand::fill(&mut bytes[..]);
|
||||
let bytes = Bytes::from(bytes);
|
||||
from.send(bytes.clone()).unwrap();
|
||||
assert_eq!(
|
||||
tokio::time::timeout(Duration::from_secs(5), to.recv())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap(),
|
||||
bytes
|
||||
);
|
||||
assert!(matches!(
|
||||
from.send(Bytes::from(vec![0; from.max_datagram_size() + 1])),
|
||||
Err(TransportError::TooLarge { .. })
|
||||
));
|
||||
}
|
||||
outgoing.close();
|
||||
incoming.close();
|
||||
for (agent, transport) in nodes {
|
||||
transport.shutdown();
|
||||
agent.shutdown().await;
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn encrypted_health_checks_fit_a_smaller_peers_configured_interface_mtu() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let roots = [TempDir::new().unwrap(), TempDir::new().unwrap()];
|
||||
let a = Node::open_config(roots[0].path(), &discovery, "smalla", false, 1280).await;
|
||||
let b = Node::open_config(roots[1].path(), &discovery, "smallb", false, 576).await;
|
||||
let (name, secret) = network("awg-small-mtu");
|
||||
let id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
a.ready(id, 1).await;
|
||||
b.ready(id, 1).await;
|
||||
let bytes = packet(a.addr(id), b.addr(id), 576);
|
||||
a.tun().push_from_os(bytes.clone());
|
||||
assert_eq!(
|
||||
tokio::time::timeout(Duration::from_secs(5), b.tun().pop_to_os())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap(),
|
||||
bytes
|
||||
);
|
||||
wait_until(
|
||||
"encrypted delivery checks answered with a small MTU",
|
||||
|| async {
|
||||
let view = a.plugin.overview(id)?;
|
||||
let peer = view.peers.first()?.tunnel.as_ref()?;
|
||||
(peer.stats.rx_packets >= 3).then_some(())
|
||||
},
|
||||
)
|
||||
.await;
|
||||
for node in [&a, &b] {
|
||||
assert_eq!(
|
||||
node.plugin.overview(id).unwrap().peers[0]
|
||||
.tunnel
|
||||
.as_ref()
|
||||
.unwrap()
|
||||
.stats
|
||||
.dropped_oversize,
|
||||
0
|
||||
);
|
||||
}
|
||||
a.agent.shutdown().await;
|
||||
b.agent.shutdown().await;
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
//! Fresh and bounded masking profiles, and rejection of foreign/truncated wire packets.
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use tsunagi_awg::profile::PairProfile;
|
||||
|
||||
#[test]
|
||||
fn profiles_are_fresh_bounded_and_redacted() {
|
||||
let profiles: Vec<_> = (0..128).map(|_| PairProfile::generate()).collect();
|
||||
for (i, profile) in profiles.iter().enumerate() {
|
||||
profile.validate().unwrap();
|
||||
assert!(!format!("{profile:?}").contains(&format!("{:?}", profile.header_key())));
|
||||
for other in &profiles[..i] {
|
||||
assert_ne!(profile.header_key(), other.header_key());
|
||||
assert_ne!(profile.id(), other.id());
|
||||
}
|
||||
for packet in profile.signature_packets() {
|
||||
assert!(packet.len() < 1200);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn masked_packets_round_trip_and_reject_other_profiles_and_truncation() {
|
||||
let profile = PairProfile::generate();
|
||||
let other = PairProfile::generate();
|
||||
for (kind, size) in [(1, 148), (2, 92), (3, 64), (4, 800)] {
|
||||
let mut message = vec![0x7b; size];
|
||||
message[..4].copy_from_slice(&(kind as u32).to_le_bytes());
|
||||
let wire = profile.encode(&message).unwrap();
|
||||
assert_ne!(&wire[..4], &message[..4]);
|
||||
assert_eq!(profile.decode(&wire).unwrap(), message);
|
||||
assert!(other.decode(&wire).is_none());
|
||||
for length in 0..44 {
|
||||
assert!(profile.decode(&wire[..length]).is_none());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -27,14 +27,14 @@ mod status;
|
||||
pub use events::Event;
|
||||
pub use status::{
|
||||
AgentStatus, CandidateStatus, DataPath, HostnameConflict, MemberStatus, NetworkMetrics,
|
||||
NetworkState, NetworkStatus, OverlayStatus, PeerStatus,
|
||||
NetworkState, NetworkStatus, OverlayStatus, PeerStatus, RangeConflict,
|
||||
};
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Weak};
|
||||
|
||||
use iroh::{EndpointAddr, EndpointId};
|
||||
use tokio::sync::{RwLock, broadcast, mpsc, oneshot};
|
||||
use tokio::sync::{Mutex, RwLock, broadcast, mpsc, oneshot, watch};
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::BoxFuture;
|
||||
@@ -58,6 +58,8 @@ use shutdown::Shutdown;
|
||||
/// Summary of a configured network, whether or not it is running.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ConfiguredNetwork {
|
||||
/// Enabled local data-plane protocols, in preference order.
|
||||
pub protocols: Vec<String>,
|
||||
/// Saved local broadcast participation.
|
||||
pub broadcast: bool,
|
||||
/// Public network identifier.
|
||||
@@ -90,6 +92,10 @@ struct Inner {
|
||||
events: broadcast::Sender<Event>,
|
||||
networks: RwLock<HashMap<NetworkId, NetworkHandle>>,
|
||||
shutdown: Shutdown,
|
||||
/// Serialises start, stop, switching and conflict cleanup, including teardown.
|
||||
lifecycle: Mutex<()>,
|
||||
range_changed: watch::Sender<()>,
|
||||
range_task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
accept_task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
plugin_task: std::sync::Mutex<Option<JoinHandle<()>>>,
|
||||
transport: std::sync::OnceLock<Arc<dyn PacketTransport>>,
|
||||
@@ -100,6 +106,37 @@ struct Inner {
|
||||
}
|
||||
|
||||
impl Inner {
|
||||
/// Called under the lifecycle lock after startup: an initially
|
||||
/// control-only agent creates its one interface on first enablement.
|
||||
async fn ensure_interface(self: &Arc<Self>) {
|
||||
if self.interface.get().is_some() {
|
||||
return;
|
||||
}
|
||||
if let Some(factory) = self.config.tun_factory.clone() {
|
||||
let carrier = Arc::new(Carrier(Arc::downgrade(self))) as Arc<dyn PacketCarrier>;
|
||||
match crate::overlay::Interface::start(
|
||||
factory,
|
||||
self.config.interface_name.clone(),
|
||||
self.config.interface_mtu,
|
||||
Arc::clone(&self.routes),
|
||||
carrier,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(interface) => {
|
||||
let _ = self.interface.set(Arc::new(interface));
|
||||
}
|
||||
Err(err) => {
|
||||
let _ = self.events.send(Event::PluginError {
|
||||
network: NetworkId::from_bytes([0u8; 32]),
|
||||
protocol: "overlay".into(),
|
||||
reason: err.to_string(),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The name this agent currently answers to.
|
||||
fn read_hostname(&self) -> String {
|
||||
match self.hostname.read() {
|
||||
@@ -165,6 +202,21 @@ impl PacketSink for Sink {
|
||||
let Some(inner) = self.0.upgrade() else {
|
||||
return;
|
||||
};
|
||||
if let Some(payload) = crate::dataplane::transport::health::ip_payload(&packet) {
|
||||
if let Some(header) = crate::overlay::IpHeader::parse(&packet)
|
||||
&& let (std::net::IpAddr::V4(source), std::net::IpAddr::V4(destination)) =
|
||||
(header.source(), header.destination())
|
||||
&& inner.routes.may_send_from(network, peer, source)
|
||||
&& inner.routes.is_local_destination(network, destination)
|
||||
&& let Some(handle) = inner.networks.read().await.get(&network)
|
||||
{
|
||||
let _ = handle.commands.try_send(NetCommand::HealthProbe {
|
||||
peer,
|
||||
packet: bytes::Bytes::copy_from_slice(payload),
|
||||
});
|
||||
}
|
||||
return;
|
||||
}
|
||||
let Some(interface) = inner.interface.get() else {
|
||||
return;
|
||||
};
|
||||
@@ -183,6 +235,35 @@ impl PacketSink for Sink {
|
||||
#[derive(Debug)]
|
||||
struct TransportCtx(Weak<Inner>);
|
||||
|
||||
impl crate::dataplane::transport::ControlSender for TransportCtx {
|
||||
fn send<'a>(
|
||||
&'a self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
protocol: &'a str,
|
||||
payload: crate::proto::message::TransportPayload,
|
||||
) -> crate::BoxFuture<'a, std::result::Result<(), crate::dataplane::transport::TransportError>>
|
||||
{
|
||||
Box::pin(async move {
|
||||
let inner = self
|
||||
.0
|
||||
.upgrade()
|
||||
.ok_or(crate::dataplane::transport::TransportError::Closed)?;
|
||||
Agent { inner }
|
||||
.send(
|
||||
network,
|
||||
peer,
|
||||
crate::proto::message::ControlMessage::Transport {
|
||||
protocol: protocol.into(),
|
||||
payload,
|
||||
},
|
||||
)
|
||||
.await
|
||||
.map_err(|err| crate::dataplane::transport::TransportError::Other(err.to_string()))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl TransportContext for TransportCtx {
|
||||
fn snapshot(&self) -> crate::BoxFuture<'_, HashMap<NetworkId, NetworkKeys>> {
|
||||
Box::pin(async move {
|
||||
@@ -217,12 +298,90 @@ impl TransportContext for TransportCtx {
|
||||
}
|
||||
|
||||
impl Agent {
|
||||
fn effective_protocols(&self, saved: Option<&Vec<String>>) -> Vec<String> {
|
||||
saved
|
||||
.or(self.inner.config.default_protocols.as_ref())
|
||||
.cloned()
|
||||
.unwrap_or_else(|| {
|
||||
self.inner
|
||||
.config
|
||||
.plugins
|
||||
.iter()
|
||||
.map(|p| p.protocol_id().to_owned())
|
||||
.collect()
|
||||
})
|
||||
}
|
||||
|
||||
fn selected_plugins(&self, protocols: &[String]) -> Vec<crate::dataplane::SharedPlugin> {
|
||||
protocols
|
||||
.iter()
|
||||
.filter_map(|name| {
|
||||
self.inner
|
||||
.config
|
||||
.plugins
|
||||
.iter()
|
||||
.find(|p| p.protocol_id() == name)
|
||||
.cloned()
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Saves and immediately applies a network's ordered local protocol set.
|
||||
/// An empty set disables data traffic while retaining control sessions.
|
||||
pub async fn set_protocols(&self, network_id: NetworkId, protocols: Vec<String>) -> Result<()> {
|
||||
crate::dataplane::validate_protocols(&protocols, self.inner.limits.max_capabilities)?;
|
||||
for name in &protocols {
|
||||
if !self
|
||||
.inner
|
||||
.config
|
||||
.plugins
|
||||
.iter()
|
||||
.any(|p| p.protocol_id() == name)
|
||||
{
|
||||
return Err(Error::InvalidProtocols(format!(
|
||||
"this agent has no `{name}` plugin"
|
||||
)));
|
||||
}
|
||||
}
|
||||
let _lifecycle = self.inner.lifecycle.lock().await;
|
||||
self.inner
|
||||
.storage
|
||||
.set_protocols(network_id, protocols.clone())
|
||||
.await?;
|
||||
if self.is_active(network_id).await {
|
||||
let (reply, receive) = oneshot::channel();
|
||||
if !protocols.is_empty() {
|
||||
self.inner.ensure_interface().await;
|
||||
}
|
||||
self.command(
|
||||
network_id,
|
||||
NetCommand::SetProtocols {
|
||||
protocols,
|
||||
interface: self.inner.interface.get().cloned(),
|
||||
reply,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
receive.await.map_err(|_| Error::Stopped)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
/// Starts an agent.
|
||||
///
|
||||
/// Opens the state store (taking its ownership lock), restores the
|
||||
/// persistent device identity, binds the iroh endpoint and activates every
|
||||
/// configured network whose auto-start flag is set.
|
||||
pub async fn spawn(config: AgentConfig) -> Result<Self> {
|
||||
if let Some(protocols) = &config.default_protocols {
|
||||
crate::dataplane::validate_protocols(protocols, config.limits.max_capabilities)?;
|
||||
for name in protocols {
|
||||
if !config.plugins.iter().any(|p| p.protocol_id() == name) {
|
||||
return Err(Error::InvalidProtocols(format!(
|
||||
"this agent has no `{name}` plugin"
|
||||
)));
|
||||
}
|
||||
}
|
||||
}
|
||||
let storage = Storage::open(&config.paths)?;
|
||||
let identity = storage.device_identity().await?;
|
||||
let adapter = EndpointAdapter::bind(&config, &identity).await?;
|
||||
@@ -234,6 +393,7 @@ impl Agent {
|
||||
let (events, _) = broadcast::channel(config.limits.event_buffer);
|
||||
let limits = Arc::new(config.limits.clone());
|
||||
|
||||
let (range_changed, range_updates) = watch::channel(());
|
||||
let inner = Arc::new(Inner {
|
||||
limits,
|
||||
storage,
|
||||
@@ -243,6 +403,9 @@ impl Agent {
|
||||
events,
|
||||
networks: RwLock::new(HashMap::new()),
|
||||
shutdown: Shutdown::new(),
|
||||
lifecycle: Mutex::new(()),
|
||||
range_changed,
|
||||
range_task: std::sync::Mutex::new(None),
|
||||
accept_task: std::sync::Mutex::new(None),
|
||||
plugin_task: std::sync::Mutex::new(None),
|
||||
transport: std::sync::OnceLock::new(),
|
||||
@@ -251,31 +414,13 @@ impl Agent {
|
||||
config,
|
||||
});
|
||||
|
||||
// One interface for the agent, if it was given a way to make one.
|
||||
// Failing to is reported and not fatal: the control plane works, and
|
||||
// so do the protocols, they just have nowhere to put packets.
|
||||
if let Some(factory) = inner.config.tun_factory.clone() {
|
||||
let carrier = Arc::new(Carrier(Arc::downgrade(&inner))) as Arc<dyn PacketCarrier>;
|
||||
match crate::overlay::Interface::start(
|
||||
factory,
|
||||
inner.config.interface_name.clone(),
|
||||
inner.config.interface_mtu,
|
||||
Arc::clone(&inner.routes),
|
||||
carrier,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(interface) => {
|
||||
let _ = inner.interface.set(Arc::new(interface));
|
||||
}
|
||||
Err(err) => {
|
||||
let _ = inner.events.send(Event::PluginError {
|
||||
network: NetworkId::from_bytes([0u8; 32]),
|
||||
protocol: "overlay".into(),
|
||||
reason: err.to_string(),
|
||||
});
|
||||
}
|
||||
}
|
||||
if inner
|
||||
.config
|
||||
.default_protocols
|
||||
.as_ref()
|
||||
.is_none_or(|names| !names.is_empty())
|
||||
{
|
||||
inner.ensure_interface().await;
|
||||
}
|
||||
|
||||
// The data plane rides on iroh too, which is where it gets hole
|
||||
@@ -294,6 +439,7 @@ impl Agent {
|
||||
inner.config.custom_transports.clone(),
|
||||
))
|
||||
};
|
||||
transport.attach_control(Arc::new(TransportCtx(Arc::downgrade(&inner))));
|
||||
let _ = inner.transport.set(transport);
|
||||
|
||||
if let CacheOutcome::Reset(reason) = inner.storage.cache_outcome().clone() {
|
||||
@@ -311,8 +457,9 @@ impl Agent {
|
||||
if !inner.config.plugins.is_empty() {
|
||||
let (plugin_tx, plugin_rx) = mpsc::channel(64);
|
||||
let sink = inner
|
||||
.interface
|
||||
.get()
|
||||
.config
|
||||
.tun_factory
|
||||
.as_ref()
|
||||
.map(|_| Arc::new(Sink(Arc::downgrade(&inner))) as Arc<dyn PacketSink>);
|
||||
let context = PluginContext::new(plugin_tx, inner.identity.endpoint_id(), sink);
|
||||
for plugin in &inner.config.plugins {
|
||||
@@ -326,11 +473,16 @@ impl Agent {
|
||||
|
||||
let agent = Self { inner };
|
||||
|
||||
for stored in agent.inner.storage.list_networks().await? {
|
||||
if stored.auto_start {
|
||||
let keys = NetworkKeys::derive(&stored.name, &stored.secret);
|
||||
agent.activate_with_keys(keys).await?;
|
||||
}
|
||||
{
|
||||
let _guard = agent.inner.lifecycle.lock().await;
|
||||
agent.resume_networks_locked().await?;
|
||||
}
|
||||
let task = tokio::spawn(range_conflict_loop(
|
||||
Arc::downgrade(&agent.inner),
|
||||
range_updates,
|
||||
));
|
||||
if let Ok(mut guard) = agent.inner.range_task.lock() {
|
||||
*guard = Some(task);
|
||||
}
|
||||
|
||||
Ok(agent)
|
||||
@@ -374,100 +526,65 @@ impl Agent {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Reserves the configured overlay range for a network, if it can.
|
||||
///
|
||||
/// Done here, where activations are serialised, rather than inside the
|
||||
/// runtime: whether a range is free depends on what the other networks
|
||||
/// took, and deciding that in a task would make the answer depend on
|
||||
/// which task ran first.
|
||||
///
|
||||
/// `None` means this agent will not propose a range for that network and
|
||||
/// waits to adopt whatever it settles on. One agent has one interface, so
|
||||
/// proposing a range it could not route would be worse than having none:
|
||||
/// the lowest author's range wins, and the collision would spread.
|
||||
async fn reserve_range(&self, network: NetworkId) -> RangePlan {
|
||||
let reserve = |wanted: Ipv4Range| {
|
||||
let reservation = crate::overlay::NetworkRoutes {
|
||||
exit: Default::default(),
|
||||
broadcast: Default::default(),
|
||||
range: Some(wanted),
|
||||
local: None,
|
||||
peers: Vec::new(),
|
||||
};
|
||||
self.inner.routes.set_network(network, reservation).is_ok()
|
||||
};
|
||||
|
||||
// If this network already has signed records in storage, check if its
|
||||
// members have already agreed on a range. If so, preserve and reserve
|
||||
// THAT range rather than preemptively seizing the default configured range.
|
||||
let stored_records = self
|
||||
.inner
|
||||
.storage
|
||||
.signed_records(network)
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
let mut state_set = crate::state::StateSet::new();
|
||||
state_set.merge_all(network, stored_records);
|
||||
if let Some(agreed) = state_set.agreed_range() {
|
||||
if reserve(agreed) {
|
||||
return RangePlan {
|
||||
propose: if Some(agreed) == self.inner.config.overlay_ipv4_range {
|
||||
Some(agreed)
|
||||
} else {
|
||||
None
|
||||
},
|
||||
..RangePlan::default()
|
||||
};
|
||||
/// Signed state always wins over a default or an explicit proposal.
|
||||
async fn desired_range(&self, network: NetworkId) -> Result<Option<Ipv4Range>> {
|
||||
let records = self.inner.storage.signed_records(network).await?;
|
||||
let mut state = crate::state::StateSet::new();
|
||||
state.merge_all(network, records);
|
||||
Ok(state.agreed_range().or_else(|| {
|
||||
if self.inner.config.overlay_ipv4_auto {
|
||||
Some(crate::state::derived_ipv4_range(network))
|
||||
} else {
|
||||
tracing::warn!(
|
||||
%network,
|
||||
range = %agreed,
|
||||
"agreed range for this network overlaps another network already active on this interface"
|
||||
);
|
||||
return RangePlan {
|
||||
conflict: Some(agreed),
|
||||
..RangePlan::default()
|
||||
};
|
||||
self.inner.config.overlay_ipv4_range
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
async fn range_conflicts(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
range: Option<Ipv4Range>,
|
||||
) -> Result<Vec<RangeConflict>> {
|
||||
let Some(range) = range else {
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
let overlapping = self.inner.routes.overlapping_networks(network, range);
|
||||
let stored = self.inner.storage.list_networks().await?;
|
||||
Ok(overlapping
|
||||
.into_iter()
|
||||
.filter_map(|(id, range)| {
|
||||
let name = stored.iter().find(|n| n.network_id == id)?.name.clone();
|
||||
Some(RangeConflict {
|
||||
network_id: id,
|
||||
name,
|
||||
range,
|
||||
})
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
/// Starts requested networks in saved preference order, skipping overlaps.
|
||||
/// Called with the lifecycle lock held; skipped networks remain requested.
|
||||
async fn resume_networks_locked(&self) -> Result<()> {
|
||||
let order = self.inner.storage.network_start_order().await?;
|
||||
let mut stored = self.inner.storage.list_networks().await?;
|
||||
stored.sort_by_key(|n| {
|
||||
order
|
||||
.iter()
|
||||
.position(|id| *id == n.network_id)
|
||||
.unwrap_or(usize::MAX)
|
||||
});
|
||||
for network in stored {
|
||||
if !network.auto_start || self.is_active(network.network_id).await {
|
||||
continue;
|
||||
}
|
||||
let keys = NetworkKeys::derive(&network.name, &network.secret);
|
||||
match self.activate_with_keys_locked(keys).await {
|
||||
Ok(()) | Err(Error::NetworkRangeConflict { .. }) => {}
|
||||
Err(err) => return Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
let Some(configured) = self.inner.config.overlay_ipv4_range else {
|
||||
return RangePlan::default();
|
||||
};
|
||||
|
||||
// The configured range first, so a network a device has always had
|
||||
// keeps the addresses it has always had.
|
||||
if reserve(configured) {
|
||||
return RangePlan {
|
||||
propose: Some(configured),
|
||||
..RangePlan::default()
|
||||
};
|
||||
}
|
||||
|
||||
// A second network on the same agent cannot have it — one agent,
|
||||
// one interface — so it falls back to the range derived from its
|
||||
// own id, which every one of its members derives identically
|
||||
// without being told.
|
||||
let derived = crate::state::derived_ipv4_range(network);
|
||||
if reserve(derived) {
|
||||
tracing::info!(
|
||||
%network,
|
||||
range = %derived,
|
||||
"another network here holds the configured range; this one uses the \
|
||||
range derived from its id unless its members settled another"
|
||||
);
|
||||
return RangePlan {
|
||||
fallback: Some(derived),
|
||||
conflict: Some(configured),
|
||||
..RangePlan::default()
|
||||
};
|
||||
}
|
||||
|
||||
tracing::info!(%network, "not proposing a range for this network");
|
||||
RangePlan {
|
||||
conflict: Some(configured),
|
||||
..RangePlan::default()
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The overlay interface this agent owns, when it has one.
|
||||
@@ -546,6 +663,9 @@ impl Agent {
|
||||
/// The same `(name, secret)` always produces the same [`NetworkId`], on
|
||||
/// every device.
|
||||
///
|
||||
/// An overlapping network stays configured and requested, but inactive;
|
||||
/// use [`Agent::switch_network`] to suspend the blockers explicitly.
|
||||
///
|
||||
/// This is declarative and therefore **idempotent**: joining a network
|
||||
/// that is already active succeeds and changes nothing. That matters
|
||||
/// because a configured network is activated automatically at startup, so
|
||||
@@ -599,7 +719,9 @@ impl Agent {
|
||||
}
|
||||
match self.activate_with_keys(keys).await {
|
||||
// Already a member of exactly this network space: nothing to do.
|
||||
Ok(()) | Err(Error::NetworkAlreadyActive(_)) => Ok(network_id),
|
||||
Ok(())
|
||||
| Err(Error::NetworkAlreadyActive(_))
|
||||
| Err(Error::NetworkRangeConflict { .. }) => Ok(network_id),
|
||||
Err(err) => Err(err),
|
||||
}
|
||||
}
|
||||
@@ -703,7 +825,44 @@ impl Agent {
|
||||
self.activate_with_keys(keys).await
|
||||
}
|
||||
|
||||
/// Switches to a configured network, suspending every overlapping network.
|
||||
/// Suspended networks retain their auto-start request and resume when free.
|
||||
/// Returns the networks suspended by this request.
|
||||
pub async fn switch_network(&self, network_id: NetworkId) -> Result<Vec<NetworkId>> {
|
||||
let _guard = self.inner.lifecycle.lock().await;
|
||||
if self.inner.shutdown.is_triggered() {
|
||||
return Err(Error::Stopped);
|
||||
}
|
||||
let stored = self
|
||||
.inner
|
||||
.storage
|
||||
.list_networks()
|
||||
.await?
|
||||
.into_iter()
|
||||
.find(|n| n.network_id == network_id)
|
||||
.ok_or(Error::NetworkUnknown(network_id))?;
|
||||
let range = self.desired_range(network_id).await?;
|
||||
let conflicts = self.range_conflicts(network_id, range).await?;
|
||||
self.inner.storage.prefer_network(network_id).await?;
|
||||
let mut stopped = Vec::new();
|
||||
for conflict in conflicts {
|
||||
self.stop_network_locked(conflict.network_id).await?;
|
||||
stopped.push(conflict.network_id);
|
||||
}
|
||||
if !self.is_active(network_id).await {
|
||||
self.activate_with_keys_locked(NetworkKeys::derive(&stored.name, &stored.secret))
|
||||
.await?;
|
||||
}
|
||||
self.resume_networks_locked().await?;
|
||||
Ok(stopped)
|
||||
}
|
||||
|
||||
async fn activate_with_keys(&self, keys: NetworkKeys) -> Result<()> {
|
||||
let _guard = self.inner.lifecycle.lock().await;
|
||||
self.activate_with_keys_locked(keys).await
|
||||
}
|
||||
|
||||
async fn activate_with_keys_locked(&self, keys: NetworkKeys) -> Result<()> {
|
||||
if self.inner.shutdown.is_triggered() {
|
||||
return Err(Error::Stopped);
|
||||
}
|
||||
@@ -712,8 +871,36 @@ impl Agent {
|
||||
if networks.contains_key(&network_id) {
|
||||
return Err(Error::NetworkAlreadyActive(network_id));
|
||||
}
|
||||
|
||||
let range = self.reserve_range(network_id).await;
|
||||
let stored = self
|
||||
.inner
|
||||
.storage
|
||||
.list_networks()
|
||||
.await?
|
||||
.into_iter()
|
||||
.find(|stored| stored.network_id == network_id);
|
||||
let range = self.desired_range(network_id).await?;
|
||||
if let Some(range) = range {
|
||||
let conflicts = self.inner.routes.overlapping_networks(network_id, range);
|
||||
if !conflicts.is_empty() {
|
||||
return Err(Error::NetworkRangeConflict {
|
||||
network: network_id,
|
||||
range,
|
||||
conflicts: conflicts.into_iter().map(|(id, _)| id).collect(),
|
||||
});
|
||||
}
|
||||
self.inner
|
||||
.routes
|
||||
.set_network(
|
||||
network_id,
|
||||
crate::overlay::NetworkRoutes {
|
||||
range: Some(range),
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.map_err(|err| Error::Rejected {
|
||||
reason: err.to_string(),
|
||||
})?;
|
||||
}
|
||||
let mut backends = Vec::new();
|
||||
if let Some(discovery) = &self.inner.config.discovery {
|
||||
backends.push(discovery.clone());
|
||||
@@ -729,14 +916,14 @@ impl Agent {
|
||||
as Arc<dyn crate::discovery::NetworkDiscovery>,
|
||||
)
|
||||
};
|
||||
let stored = self
|
||||
.inner
|
||||
.storage
|
||||
.list_networks()
|
||||
.await?
|
||||
.into_iter()
|
||||
.find(|stored| stored.network_id == network_id);
|
||||
|
||||
let broadcast = stored.as_ref().is_none_or(|stored| stored.broadcast);
|
||||
let protocols =
|
||||
self.effective_protocols(stored.as_ref().and_then(|s| s.protocols.as_ref()));
|
||||
let plugins = self.selected_plugins(&protocols);
|
||||
if !plugins.is_empty() {
|
||||
self.inner.ensure_interface().await;
|
||||
}
|
||||
let exit_offer = stored.as_ref().is_some_and(|stored| stored.exit_node);
|
||||
let exit_via = stored
|
||||
.as_ref()
|
||||
@@ -755,7 +942,8 @@ impl Agent {
|
||||
discovery,
|
||||
discovery_interval: self.inner.config.discovery_interval,
|
||||
discovery_policy: self.inner.config.discovery_policy.clone(),
|
||||
plugins: self.inner.config.plugins.clone(),
|
||||
available_plugins: self.inner.config.plugins.clone(),
|
||||
plugins: plugins.clone(),
|
||||
routes: Arc::clone(&self.inner.routes),
|
||||
interface: self.inner.interface.get().cloned(),
|
||||
hostname: self.inner.read_hostname(),
|
||||
@@ -763,14 +951,13 @@ impl Agent {
|
||||
device_secret: self.inner.identity.signing_key(),
|
||||
#[cfg(feature = "testing")]
|
||||
unreachable_data_peers: Arc::clone(&self.inner.config.unreachable_data_peers),
|
||||
ipv4_range: range.propose,
|
||||
ipv4_fallback: range.fallback,
|
||||
range_conflict: range.conflict,
|
||||
ipv4_range: range,
|
||||
range_changed: self.inner.range_changed.clone(),
|
||||
});
|
||||
networks.insert(network_id, handle);
|
||||
drop(networks);
|
||||
|
||||
for plugin in &self.inner.config.plugins {
|
||||
for plugin in &plugins {
|
||||
plugin.on_network_activated(network_id);
|
||||
}
|
||||
|
||||
@@ -786,6 +973,30 @@ impl Agent {
|
||||
/// it does not remove this agent from anyone else's view of the network.
|
||||
/// Other networks keep running.
|
||||
pub async fn deactivate_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
let _guard = self.inner.lifecycle.lock().await;
|
||||
if !self.is_active(network_id).await {
|
||||
return Err(Error::NetworkNotActive(network_id));
|
||||
}
|
||||
self.inner.storage.set_auto_start(network_id, false).await?;
|
||||
self.stop_network_locked(network_id).await?;
|
||||
self.resume_networks_locked().await
|
||||
}
|
||||
|
||||
/// Stops a configured network, including one already suspended by overlap.
|
||||
/// Clears its auto-start request so it will not resume behind the user.
|
||||
pub async fn pause_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
let _guard = self.inner.lifecycle.lock().await;
|
||||
if self.inner.shutdown.is_triggered() {
|
||||
return Err(Error::Stopped);
|
||||
}
|
||||
self.inner.storage.set_auto_start(network_id, false).await?;
|
||||
if self.is_active(network_id).await {
|
||||
self.stop_network_locked(network_id).await?;
|
||||
}
|
||||
self.resume_networks_locked().await
|
||||
}
|
||||
|
||||
async fn stop_network_locked(&self, network_id: NetworkId) -> Result<()> {
|
||||
let handle = {
|
||||
let mut networks = self.inner.networks.write().await;
|
||||
networks
|
||||
@@ -814,7 +1025,6 @@ impl Agent {
|
||||
interface.sync_broadcast_rules().await;
|
||||
interface.sync_exit_rules().await;
|
||||
}
|
||||
self.inner.storage.set_auto_start(network_id, false).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -824,17 +1034,15 @@ impl Agent {
|
||||
/// Local and silent: nobody is told. [`Agent::leave_network`] is the one
|
||||
/// that says goodbye first, and is what a person means by leaving.
|
||||
pub async fn forget_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
let _guard = self.inner.lifecycle.lock().await;
|
||||
if self.is_active(network_id).await {
|
||||
self.deactivate_network(network_id).await?;
|
||||
self.stop_network_locked(network_id).await?;
|
||||
}
|
||||
self.inner.storage.remove_network(network_id).await?;
|
||||
// Whatever a protocol kept for this network goes with it. The
|
||||
// agent cannot know what that is, only that there is no longer any
|
||||
// reason to hold it.
|
||||
for plugin in &self.inner.config.plugins {
|
||||
plugin.on_network_forgotten(network_id);
|
||||
}
|
||||
Ok(())
|
||||
self.resume_networks_locked().await
|
||||
}
|
||||
|
||||
/// Leaves a network: gives up what was claimed, then forgets it.
|
||||
@@ -890,6 +1098,7 @@ impl Agent {
|
||||
.await?
|
||||
.into_iter()
|
||||
.map(|stored| ConfiguredNetwork {
|
||||
protocols: self.effective_protocols(stored.protocols.as_ref()),
|
||||
broadcast: stored.broadcast,
|
||||
network_id: stored.network_id,
|
||||
name: stored.name,
|
||||
@@ -958,13 +1167,18 @@ impl Agent {
|
||||
let mut networks = Vec::new();
|
||||
for stored in self.inner.storage.list_networks().await? {
|
||||
if active.contains(&stored.network_id)
|
||||
&& let Ok(status) = self.network_status(stored.network_id).await
|
||||
&& let Ok(mut status) = self.network_status(stored.network_id).await
|
||||
{
|
||||
status.auto_start = stored.auto_start;
|
||||
networks.push(status);
|
||||
continue;
|
||||
}
|
||||
let keys = NetworkKeys::derive(&stored.name, &stored.secret);
|
||||
let range = self.desired_range(stored.network_id).await?;
|
||||
let range_conflicts = self.range_conflicts(stored.network_id, range).await?;
|
||||
networks.push(NetworkStatus {
|
||||
protocols: self.effective_protocols(stored.protocols.as_ref()),
|
||||
auto_start: stored.auto_start,
|
||||
broadcast: stored.broadcast,
|
||||
exit_offer: stored.exit_node,
|
||||
exit_via: stored
|
||||
@@ -978,12 +1192,13 @@ impl Agent {
|
||||
state: NetworkState::Inactive,
|
||||
peers: Vec::new(),
|
||||
candidates: Vec::new(),
|
||||
// An inactive network has no runtime to ask; the roster and
|
||||
// the range come from one. Empty, not invented.
|
||||
// The range is available from signed state or the deterministic
|
||||
// default even while the network has no runtime.
|
||||
members: Vec::new(),
|
||||
hostname_conflict: None,
|
||||
range: None,
|
||||
range_conflict: None,
|
||||
range,
|
||||
range_conflict: range_conflicts.first().map(|conflict| conflict.range),
|
||||
range_conflicts,
|
||||
metrics: NetworkMetrics::default(),
|
||||
relay: Default::default(),
|
||||
});
|
||||
@@ -1039,6 +1254,7 @@ impl Agent {
|
||||
/// After this returns, the state directory can be opened by another agent.
|
||||
pub async fn shutdown(&self) {
|
||||
self.inner.shutdown.trigger();
|
||||
let lifecycle_guard = self.inner.lifecycle.lock().await;
|
||||
|
||||
let handles: Vec<NetworkHandle> = {
|
||||
let mut networks = self.inner.networks.write().await;
|
||||
@@ -1048,6 +1264,7 @@ impl Agent {
|
||||
handle.stop().await;
|
||||
}
|
||||
|
||||
drop(lifecycle_guard);
|
||||
if let Some(dht) = &self.inner.config.dht {
|
||||
dht.shutdown().await;
|
||||
}
|
||||
@@ -1064,6 +1281,7 @@ impl Agent {
|
||||
for (what, handle) in [
|
||||
("inbound connections", &self.inner.accept_task),
|
||||
("plugin requests", &self.inner.plugin_task),
|
||||
("range conflicts", &self.inner.range_task),
|
||||
] {
|
||||
let task = handle.lock().ok().and_then(|mut guard| guard.take());
|
||||
if let Some(task) = task {
|
||||
@@ -1107,16 +1325,44 @@ impl Agent {
|
||||
}
|
||||
}
|
||||
|
||||
/// What a network may propose as its overlay range, and what it cannot have.
|
||||
#[derive(Debug, Clone, Copy, Default)]
|
||||
struct RangePlan {
|
||||
/// Proposed straight away: the configured range, when it is free here.
|
||||
propose: Option<Ipv4Range>,
|
||||
/// Proposed after a moment, when the configured range is another
|
||||
/// network's: the range derived from this network's own id.
|
||||
fallback: Option<Ipv4Range>,
|
||||
/// The configured range, when this network cannot have it.
|
||||
conflict: Option<Ipv4Range>,
|
||||
/// Processes coalesced conflict notifications without blocking a network actor.
|
||||
async fn range_conflict_loop(inner: Weak<Inner>, mut updates: watch::Receiver<()>) {
|
||||
loop {
|
||||
let Some(owner) = inner.upgrade() else {
|
||||
return;
|
||||
};
|
||||
let shutdown = owner.shutdown.clone();
|
||||
drop(owner);
|
||||
tokio::select! {
|
||||
_ = shutdown.wait() => return,
|
||||
result = updates.changed() => if result.is_err() { return; },
|
||||
}
|
||||
let Some(inner) = inner.upgrade() else {
|
||||
return;
|
||||
};
|
||||
let agent = Agent { inner };
|
||||
let _guard = agent.inner.lifecycle.lock().await;
|
||||
if agent.inner.shutdown.is_triggered() {
|
||||
return;
|
||||
}
|
||||
let blocked: Vec<_> = agent
|
||||
.inner
|
||||
.networks
|
||||
.read()
|
||||
.await
|
||||
.iter()
|
||||
.filter(|(_, h)| h.range_blocked.load(std::sync::atomic::Ordering::Acquire))
|
||||
.map(|(id, _)| *id)
|
||||
.collect();
|
||||
for id in blocked {
|
||||
if let Err(err) = agent.stop_network_locked(id).await {
|
||||
tracing::warn!(%err, "cannot suspend overlapping network");
|
||||
}
|
||||
}
|
||||
if let Err(err) = agent.resume_networks_locked().await {
|
||||
tracing::warn!(%err, "cannot resume suspended networks");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// How long a release is given to reach the sessions it was queued on.
|
||||
|
||||
@@ -15,6 +15,7 @@ use tokio::sync::{broadcast, mpsc, oneshot};
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::config::{DiscoveryPolicy, Limits, ReconnectPolicy};
|
||||
use crate::dataplane::transport::health::{HealthLink, ProbeRoute, Settings as HealthSettings};
|
||||
use crate::dataplane::transport::{InboundLink, PacketLink, PacketTransport, SharedLink};
|
||||
use crate::dataplane::{PluginCapability, SharedPlugin};
|
||||
use crate::discovery::worker::DiscoveryWorker;
|
||||
@@ -59,6 +60,10 @@ pub(crate) enum NetCommand {
|
||||
},
|
||||
/// A peer opened a data plane link towards us.
|
||||
InboundLink(Box<InboundLink>),
|
||||
HealthProbe {
|
||||
peer: EndpointId,
|
||||
packet: bytes::Bytes,
|
||||
},
|
||||
Status {
|
||||
reply: oneshot::Sender<Box<NetworkStatus>>,
|
||||
},
|
||||
@@ -67,6 +72,11 @@ pub(crate) enum NetCommand {
|
||||
enabled: bool,
|
||||
reply: oneshot::Sender<()>,
|
||||
},
|
||||
SetProtocols {
|
||||
interface: Option<Arc<crate::overlay::Interface>>,
|
||||
protocols: Vec<String>,
|
||||
reply: oneshot::Sender<()>,
|
||||
},
|
||||
/// Offer this agent as an exit node in this network, or stop.
|
||||
SetExitOffer {
|
||||
enabled: bool,
|
||||
@@ -117,12 +127,14 @@ impl std::fmt::Debug for NetCommand {
|
||||
NetCommand::Status { .. } => f.write_str("Status"),
|
||||
NetCommand::Recheck => f.write_str("Recheck"),
|
||||
NetCommand::SetBroadcast { enabled, .. } => write!(f, "SetBroadcast({enabled})"),
|
||||
NetCommand::SetProtocols { protocols, .. } => write!(f, "SetProtocols({protocols:?})"),
|
||||
NetCommand::SetExitOffer { enabled, .. } => write!(f, "SetExitOffer({enabled})"),
|
||||
NetCommand::SetExitVia { via, .. } => write!(f, "SetExitVia({})", via.is_some()),
|
||||
NetCommand::Reannounce => f.write_str("Reannounce"),
|
||||
NetCommand::SetHostname(_) => f.write_str("SetHostname"),
|
||||
NetCommand::Release { .. } => f.write_str("Release"),
|
||||
NetCommand::PluginError { protocol, .. } => write!(f, "PluginError({protocol})"),
|
||||
NetCommand::HealthProbe { peer, .. } => write!(f, "HealthProbe({})", peer.fmt_short()),
|
||||
NetCommand::ProtocolFailure { peer, protocol, .. } => {
|
||||
write!(f, "ProtocolFailure({}, {protocol})", peer.fmt_short())
|
||||
}
|
||||
@@ -137,6 +149,7 @@ pub(crate) struct NetworkHandle {
|
||||
pub(crate) commands: mpsc::Sender<NetCommand>,
|
||||
shutdown: Shutdown,
|
||||
task: JoinHandle<()>,
|
||||
pub(crate) range_blocked: Arc<std::sync::atomic::AtomicBool>,
|
||||
}
|
||||
|
||||
impl NetworkHandle {
|
||||
@@ -164,6 +177,7 @@ pub(crate) struct RuntimeParams {
|
||||
pub(crate) discovery_interval: Duration,
|
||||
pub(crate) discovery_policy: DiscoveryPolicy,
|
||||
pub(crate) plugins: Vec<SharedPlugin>,
|
||||
pub(crate) available_plugins: Vec<SharedPlugin>,
|
||||
/// Who holds which overlay address, shared with every other network.
|
||||
pub(crate) routes: Arc<crate::overlay::RoutingTable>,
|
||||
/// The one interface, when the agent has one.
|
||||
@@ -177,23 +191,15 @@ pub(crate) struct RuntimeParams {
|
||||
/// The IPv4 overlay range this agent would use, if the network has not
|
||||
/// already settled on another one.
|
||||
pub(crate) ipv4_range: Option<Ipv4Range>,
|
||||
/// The range it was configured with but cannot have, because another
|
||||
/// network on this agent holds it.
|
||||
pub(crate) range_conflict: Option<Ipv4Range>,
|
||||
/// The range derived from this network's own id, proposed when the
|
||||
/// configured one belongs to another network here.
|
||||
///
|
||||
/// Held back at first: a network that already exists has a range of its
|
||||
/// own, and a member that proposed before hearing anything would be
|
||||
/// arguing with it instead of adopting it. Every member derives the
|
||||
/// same one, so once the wait is over there is nothing to argue about.
|
||||
pub(crate) ipv4_fallback: Option<Ipv4Range>,
|
||||
/// Coalesced notifications to the agent's lifecycle manager.
|
||||
pub(crate) range_changed: tokio::sync::watch::Sender<()>,
|
||||
/// How data plane links are opened. `None` disables the data plane.
|
||||
pub(crate) transport: Option<Arc<dyn PacketTransport>>,
|
||||
}
|
||||
|
||||
/// Outcome of one attempt to open a data plane link.
|
||||
struct LinkOutcome {
|
||||
generation: u64,
|
||||
peer: EndpointId,
|
||||
protocol: String,
|
||||
result: Result<SharedLink, String>,
|
||||
@@ -236,8 +242,10 @@ pub(crate) fn spawn(params: RuntimeParams) -> NetworkHandle {
|
||||
let (commands_tx, commands_rx) = mpsc::channel(64);
|
||||
|
||||
let runtime_shutdown = shutdown.clone();
|
||||
let range_blocked = Arc::new(std::sync::atomic::AtomicBool::new(false));
|
||||
let runtime_blocked = Arc::clone(&range_blocked);
|
||||
let task = tokio::spawn(async move {
|
||||
let mut runtime = Runtime::new(params, runtime_shutdown);
|
||||
let mut runtime = Runtime::new(params, runtime_shutdown, runtime_blocked);
|
||||
runtime.run(commands_rx).await;
|
||||
});
|
||||
|
||||
@@ -246,6 +254,7 @@ pub(crate) fn spawn(params: RuntimeParams) -> NetworkHandle {
|
||||
commands: commands_tx,
|
||||
shutdown,
|
||||
task,
|
||||
range_blocked,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -257,14 +266,6 @@ pub(crate) fn spawn(params: RuntimeParams) -> NetworkHandle {
|
||||
/// itself, a few announcements later.
|
||||
const REACH_EXPIRY: Duration = Duration::from_secs(90);
|
||||
|
||||
/// How long a network waits to be told its range before proposing the one
|
||||
/// derived from its id.
|
||||
///
|
||||
/// Only for that fallback: the configured range is proposed at once, as it
|
||||
/// always was. This is the window in which an existing network's records
|
||||
/// can arrive and be adopted instead.
|
||||
const RANGE_PROPOSAL_GRACE: Duration = Duration::from_secs(3);
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
struct ProtocolDialState {
|
||||
failures: u32,
|
||||
@@ -274,6 +275,12 @@ struct ProtocolDialState {
|
||||
const DATA_LINK_BASE_BACKOFF: Duration = Duration::from_secs(3);
|
||||
const DATA_LINK_MAX_BACKOFF: Duration = Duration::from_secs(60);
|
||||
|
||||
fn next_protocol_index(supported: &[String], last_failed: Option<&str>) -> usize {
|
||||
last_failed
|
||||
.and_then(|failed| supported.iter().position(|p| p == failed))
|
||||
.map_or(0, |i| (i + 1) % supported.len().max(1))
|
||||
}
|
||||
|
||||
fn data_link_backoff(failures: u32) -> Duration {
|
||||
if failures == 0 {
|
||||
return Duration::ZERO;
|
||||
@@ -287,8 +294,9 @@ struct Runtime {
|
||||
discovery_worker: Option<DiscoveryWorker>,
|
||||
discovery_candidates: mpsc::Receiver<Candidate>,
|
||||
params: RuntimeParams,
|
||||
/// When this runtime started, for the fallback range's grace period.
|
||||
activated: std::time::Instant,
|
||||
range_blocked: Arc<std::sync::atomic::AtomicBool>,
|
||||
/// Last range successfully published to the agent's routing table.
|
||||
published_range: Option<Ipv4Range>,
|
||||
network_id: NetworkId,
|
||||
local_id: EndpointId,
|
||||
shutdown: Shutdown,
|
||||
@@ -304,7 +312,9 @@ struct Runtime {
|
||||
///
|
||||
/// The direct ones. What a protocol holds is the hub's link for that
|
||||
/// peer, which outlives any particular path.
|
||||
links: HashMap<(EndpointId, String), SharedLink>,
|
||||
links: HashMap<(EndpointId, String), Arc<HealthLink>>,
|
||||
pending_health: HashMap<(EndpointId, String), HealthSettings>,
|
||||
last_failed_protocol: HashMap<EndpointId, String>,
|
||||
/// Every data link of this network, and what is reached through what.
|
||||
hub: Arc<crate::dataplane::relay::RelayHub>,
|
||||
/// Which peers each peer says it has a live link with, and when it
|
||||
@@ -330,6 +340,9 @@ struct Runtime {
|
||||
announced_reach: HashSet<crate::proto::message::ReachableLink>,
|
||||
/// Links currently being opened, so we do not start two.
|
||||
opening: HashSet<(EndpointId, String)>,
|
||||
link_tasks: HashMap<(EndpointId, String), tokio::task::AbortHandle>,
|
||||
link_attempts: HashMap<(EndpointId, String), u64>,
|
||||
next_link_attempt: u64,
|
||||
link_results_tx: mpsc::Sender<LinkOutcome>,
|
||||
link_results_rx: mpsc::Receiver<LinkOutcome>,
|
||||
/// Protocol dial failure count and backoff cooldown per peer and protocol.
|
||||
@@ -366,7 +379,11 @@ struct Runtime {
|
||||
}
|
||||
|
||||
impl Runtime {
|
||||
fn new(params: RuntimeParams, shutdown: Shutdown) -> Self {
|
||||
fn new(
|
||||
params: RuntimeParams,
|
||||
shutdown: Shutdown,
|
||||
range_blocked: Arc<std::sync::atomic::AtomicBool>,
|
||||
) -> Self {
|
||||
let network_id = params.keys.network_id();
|
||||
let local_id = params.adapter.endpoint_id();
|
||||
let (session_events_tx, session_events_rx) = mpsc::channel(256);
|
||||
@@ -398,7 +415,8 @@ impl Runtime {
|
||||
discovery_worker,
|
||||
discovery_candidates,
|
||||
params,
|
||||
activated: std::time::Instant::now(),
|
||||
range_blocked,
|
||||
published_range: None,
|
||||
network_id,
|
||||
local_id,
|
||||
shutdown,
|
||||
@@ -411,11 +429,16 @@ impl Runtime {
|
||||
dial_results_tx,
|
||||
dial_results_rx,
|
||||
links: HashMap::new(),
|
||||
pending_health: HashMap::new(),
|
||||
last_failed_protocol: HashMap::new(),
|
||||
hub,
|
||||
reachable: HashMap::new(),
|
||||
local_capabilities: Vec::new(),
|
||||
announced_reach: HashSet::new(),
|
||||
opening: HashSet::new(),
|
||||
link_tasks: HashMap::new(),
|
||||
link_attempts: HashMap::new(),
|
||||
next_link_attempt: 0,
|
||||
link_results_tx,
|
||||
link_results_rx,
|
||||
protocol_dial_states: HashMap::new(),
|
||||
@@ -498,7 +521,16 @@ impl Runtime {
|
||||
// any more, and a sender blocked on a full queue would stall shutdown.
|
||||
self.session_events_rx.close();
|
||||
self.hub.close();
|
||||
for link in self.links.values() {
|
||||
link.close();
|
||||
}
|
||||
self.links.clear();
|
||||
for (_, task) in self.link_tasks.drain() {
|
||||
task.abort();
|
||||
}
|
||||
self.opening.clear();
|
||||
self.pending_health.clear();
|
||||
self.last_failed_protocol.clear();
|
||||
self.protocol_dial_states.clear();
|
||||
let peers: Vec<EndpointId> = self.sessions.keys().copied().collect();
|
||||
for peer in peers {
|
||||
@@ -539,6 +571,60 @@ impl Runtime {
|
||||
NetCommand::Status { reply } => {
|
||||
let _ = reply.send(Box::new(self.status()));
|
||||
}
|
||||
NetCommand::SetProtocols {
|
||||
protocols,
|
||||
interface,
|
||||
reply,
|
||||
} => {
|
||||
self.params.interface = interface;
|
||||
let current: Vec<_> = self
|
||||
.params
|
||||
.plugins
|
||||
.iter()
|
||||
.map(|p| p.protocol_id().to_owned())
|
||||
.collect();
|
||||
if current != protocols {
|
||||
let peers: Vec<_> = self.sessions.keys().copied().collect();
|
||||
for peer in &peers {
|
||||
self.reset_peer_data(*peer, "local protocol selection changed");
|
||||
}
|
||||
for plugin in &self.params.plugins {
|
||||
if !protocols.iter().any(|name| name == plugin.protocol_id()) {
|
||||
plugin.on_network_deactivated(self.network_id);
|
||||
}
|
||||
}
|
||||
self.params.plugins = protocols
|
||||
.iter()
|
||||
.filter_map(|name| {
|
||||
self.params
|
||||
.available_plugins
|
||||
.iter()
|
||||
.find(|p| p.protocol_id() == name)
|
||||
.cloned()
|
||||
})
|
||||
.collect();
|
||||
for plugin in &self.params.plugins {
|
||||
if !current.iter().any(|name| name == plugin.protocol_id()) {
|
||||
plugin.on_network_activated(self.network_id);
|
||||
}
|
||||
}
|
||||
self.hub.set_priority(protocols);
|
||||
self.publish_allocations().await;
|
||||
for peer in peers {
|
||||
let caps = self
|
||||
.sessions
|
||||
.get(&peer)
|
||||
.map(|s| s.capabilities.clone())
|
||||
.unwrap_or_default();
|
||||
self.dispatch_capabilities(peer, &caps);
|
||||
}
|
||||
self.reannounce();
|
||||
self.ensure_links();
|
||||
self.update_paths();
|
||||
self.announce_reach(true);
|
||||
}
|
||||
let _ = reply.send(());
|
||||
}
|
||||
NetCommand::SetBroadcast { enabled, reply } => {
|
||||
self.params.broadcast = enabled;
|
||||
self.update_broadcast();
|
||||
@@ -570,7 +656,22 @@ impl Runtime {
|
||||
let _ = reply.send(outcome);
|
||||
}
|
||||
NetCommand::Recheck => self.discovery_round().await,
|
||||
NetCommand::Reannounce => self.reannounce(),
|
||||
NetCommand::Reannounce => {
|
||||
// A newly enabled plugin may prepare its key asynchronously.
|
||||
// Replay allocations and peer identities once it is ready.
|
||||
self.publish_allocations().await;
|
||||
let peers: Vec<_> = self
|
||||
.sessions
|
||||
.iter()
|
||||
.map(|(peer, s)| (*peer, s.capabilities.clone()))
|
||||
.collect();
|
||||
for (peer, caps) in peers {
|
||||
self.dispatch_capabilities(peer, &caps);
|
||||
}
|
||||
self.reannounce();
|
||||
self.ensure_links();
|
||||
self.update_paths();
|
||||
}
|
||||
NetCommand::Release { reply } => {
|
||||
// A tombstone, so the address and the name this agent held
|
||||
// are freed for somebody else instead of staying reserved
|
||||
@@ -602,12 +703,20 @@ impl Runtime {
|
||||
reason,
|
||||
});
|
||||
}
|
||||
NetCommand::HealthProbe { peer, packet } => {
|
||||
for ((p, _), link) in &self.links {
|
||||
if *p == peer {
|
||||
link.receive_probe(packet.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
NetCommand::ProtocolFailure {
|
||||
peer,
|
||||
protocol,
|
||||
reason,
|
||||
} => {
|
||||
if self.links.remove(&(peer, protocol.clone())).is_some() {
|
||||
if let Some(link) = self.links.remove(&(peer, protocol.clone())) {
|
||||
link.close();
|
||||
self.hub.clear_direct(peer, &protocol);
|
||||
}
|
||||
self.metrics.data_link_failures += 1;
|
||||
@@ -662,7 +771,7 @@ impl Runtime {
|
||||
peer: peer.fmt_short().to_string(),
|
||||
})?;
|
||||
|
||||
match session.outbound.try_send(encoded) {
|
||||
match session.outbound.try_send(zeroize::Zeroizing::new(encoded)) {
|
||||
Ok(()) => {
|
||||
session.messages_sent += 1;
|
||||
session.bytes_sent += bytes;
|
||||
@@ -968,35 +1077,27 @@ impl Runtime {
|
||||
if let Some(agreed) = self.state.agreed_range() {
|
||||
return Some(agreed);
|
||||
}
|
||||
// Nobody has settled one yet, so this agent would be proposing its
|
||||
// own. It must not propose a range it cannot route: one agent has
|
||||
// one interface, and claiming an address in a range another network
|
||||
// already owns would spread the collision rather than contain it —
|
||||
// "the lowest author's range wins" would carry it to everybody.
|
||||
if let Some(wanted) = self.params.ipv4_range
|
||||
self.params.ipv4_range
|
||||
}
|
||||
|
||||
/// A late signed range can conflict even though activation was safe.
|
||||
/// Fail closed before touching claims, plugins or the host interface.
|
||||
fn suspend_if_overlapping(&self) -> bool {
|
||||
if let Some(range) = self.effective_range()
|
||||
&& self
|
||||
.params
|
||||
.routes
|
||||
.would_overlap(self.network_id, wanted)
|
||||
.is_none()
|
||||
.would_overlap(self.network_id, range)
|
||||
.is_some()
|
||||
{
|
||||
return Some(wanted);
|
||||
self.range_blocked
|
||||
.store(true, std::sync::atomic::Ordering::Release);
|
||||
self.params.routes.remove_network(self.network_id);
|
||||
self.shutdown.trigger();
|
||||
self.params.range_changed.send_replace(());
|
||||
return true;
|
||||
}
|
||||
|
||||
// The configured range is another network's here, so this one falls
|
||||
// back to the range derived from its id — after a moment's wait, so
|
||||
// that a network which already exists gets to say what it uses
|
||||
// first. Every member derives the same one, so members that reach
|
||||
// this point agree without negotiating.
|
||||
let fallback = self.params.ipv4_fallback?;
|
||||
if self.activated.elapsed() < RANGE_PROPOSAL_GRACE {
|
||||
return None;
|
||||
}
|
||||
self.params
|
||||
.routes
|
||||
.would_overlap(self.network_id, fallback)
|
||||
.is_none()
|
||||
.then_some(fallback)
|
||||
false
|
||||
}
|
||||
|
||||
/// Makes sure this agent holds an address, claiming one if it does not.
|
||||
@@ -1006,7 +1107,7 @@ impl Runtime {
|
||||
async fn ensure_own_claim(&mut self) {
|
||||
// On the way out. Claiming again here is how a goodbye becomes a
|
||||
// hello nobody asked for.
|
||||
if self.released {
|
||||
if self.released || self.suspend_if_overlapping() {
|
||||
return;
|
||||
}
|
||||
let wanted_hostname = {
|
||||
@@ -1296,6 +1397,9 @@ impl Runtime {
|
||||
|
||||
/// Tells the plugins who holds which overlay address.
|
||||
async fn publish_allocations(&mut self) {
|
||||
if self.shutdown.is_triggered() || self.suspend_if_overlapping() {
|
||||
return;
|
||||
}
|
||||
let Some(range) = self.effective_range() else {
|
||||
return;
|
||||
};
|
||||
@@ -1307,10 +1411,6 @@ impl Runtime {
|
||||
.collect();
|
||||
allocations.sort_by_key(|(holder, _)| *holder.as_bytes());
|
||||
|
||||
for plugin in &self.params.plugins {
|
||||
plugin.on_address_allocation(self.network_id, range, &allocations);
|
||||
}
|
||||
|
||||
// And the system level's own view, which is what decides whose
|
||||
// packet is whose. The local address is kept out of the peer list:
|
||||
// a packet for ourselves does not go over a tunnel.
|
||||
@@ -1326,17 +1426,27 @@ impl Runtime {
|
||||
.map(|(holder, address)| (*address, *holder))
|
||||
.collect(),
|
||||
};
|
||||
if let Err(err) = self.params.routes.set_network(self.network_id, routes) {
|
||||
// Reported once per change rather than swallowed: two networks
|
||||
// wanting the same addresses is a thing the user has to settle.
|
||||
self.metrics.plugin_errors += 1;
|
||||
self.emit(Event::PluginError {
|
||||
network: self.network_id,
|
||||
protocol: "overlay".into(),
|
||||
reason: err.to_string(),
|
||||
});
|
||||
if self
|
||||
.params
|
||||
.routes
|
||||
.set_network(self.network_id, routes)
|
||||
.is_err()
|
||||
{
|
||||
self.range_blocked
|
||||
.store(true, std::sync::atomic::Ordering::Release);
|
||||
self.params.routes.remove_network(self.network_id);
|
||||
self.shutdown.trigger();
|
||||
self.params.range_changed.send_replace(());
|
||||
return;
|
||||
}
|
||||
for plugin in &self.params.plugins {
|
||||
plugin.on_address_allocation(self.network_id, range, &allocations);
|
||||
}
|
||||
if self.published_range != Some(range) {
|
||||
self.published_range = Some(range);
|
||||
self.params.range_changed.send_replace(());
|
||||
}
|
||||
|
||||
let Some(interface) = self.params.interface.clone() else {
|
||||
return;
|
||||
};
|
||||
@@ -1429,6 +1539,7 @@ impl Runtime {
|
||||
}
|
||||
|
||||
fn record_protocol_failure(&mut self, peer: EndpointId, protocol: &str) {
|
||||
self.last_failed_protocol.insert(peer, protocol.to_string());
|
||||
let key = (peer, protocol.to_string());
|
||||
let entry = self
|
||||
.protocol_dial_states
|
||||
@@ -1467,7 +1578,17 @@ impl Runtime {
|
||||
}
|
||||
}
|
||||
for (peer, protocol) in dead {
|
||||
self.links.remove(&(peer, protocol.clone()));
|
||||
let reason = self
|
||||
.links
|
||||
.remove(&(peer, protocol.clone()))
|
||||
.map(|link| {
|
||||
let reason = link
|
||||
.failure_reason()
|
||||
.unwrap_or_else(|| "link closed".into());
|
||||
link.close();
|
||||
reason
|
||||
})
|
||||
.unwrap_or_else(|| "link closed".into());
|
||||
// Only the direct path is gone. Whether the peer is still
|
||||
// reachable through somebody is decided below; the protocol's
|
||||
// link stays either way.
|
||||
@@ -1477,7 +1598,7 @@ impl Runtime {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
protocol,
|
||||
reason: "link closed".into(),
|
||||
reason,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1518,12 +1639,21 @@ impl Runtime {
|
||||
continue;
|
||||
}
|
||||
|
||||
let chosen = supported.iter().find(|proto| {
|
||||
match self.protocol_dial_states.get(&(peer, (*proto).clone())) {
|
||||
Some(state) => state.cooldown_until <= now,
|
||||
None => true,
|
||||
}
|
||||
});
|
||||
let start = next_protocol_index(
|
||||
&supported,
|
||||
self.last_failed_protocol.get(&peer).map(String::as_str),
|
||||
);
|
||||
let chosen = supported
|
||||
.iter()
|
||||
.cycle()
|
||||
.skip(start)
|
||||
.take(supported.len())
|
||||
.find(
|
||||
|proto| match self.protocol_dial_states.get(&(peer, (*proto).clone())) {
|
||||
Some(state) => state.cooldown_until <= now,
|
||||
None => true,
|
||||
},
|
||||
);
|
||||
|
||||
if let Some(proto) = chosen {
|
||||
wanted.push((peer, proto.clone()));
|
||||
@@ -1535,13 +1665,16 @@ impl Runtime {
|
||||
if self.links.contains_key(&key) || self.opening.contains(&key) {
|
||||
continue;
|
||||
}
|
||||
self.opening.insert(key);
|
||||
self.opening.insert(key.clone());
|
||||
|
||||
let results = self.link_results_tx.clone();
|
||||
let transport = Arc::clone(&transport);
|
||||
let network = self.network_id;
|
||||
let shutdown = self.shutdown.clone();
|
||||
tokio::spawn(async move {
|
||||
self.next_link_attempt = self.next_link_attempt.wrapping_add(1);
|
||||
let generation = self.next_link_attempt;
|
||||
self.link_attempts.insert(key.clone(), generation);
|
||||
let task = tokio::spawn(async move {
|
||||
let result = tokio::select! {
|
||||
biased;
|
||||
_ = shutdown.wait() => Err("network deactivated".to_string()),
|
||||
@@ -1551,12 +1684,14 @@ impl Runtime {
|
||||
};
|
||||
let _ = results
|
||||
.send(LinkOutcome {
|
||||
generation,
|
||||
peer,
|
||||
protocol,
|
||||
result,
|
||||
})
|
||||
.await;
|
||||
});
|
||||
self.link_tasks.insert(key, task.abort_handle());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1791,7 +1926,15 @@ impl Runtime {
|
||||
|
||||
fn handle_link_result(&mut self, outcome: LinkOutcome) {
|
||||
let key = (outcome.peer, outcome.protocol.clone());
|
||||
if self.link_attempts.get(&key) != Some(&outcome.generation) {
|
||||
if let Ok(link) = outcome.result {
|
||||
link.close();
|
||||
}
|
||||
return;
|
||||
}
|
||||
self.link_attempts.remove(&key);
|
||||
self.opening.remove(&key);
|
||||
self.link_tasks.remove(&key);
|
||||
match outcome.result {
|
||||
Ok(link) => {
|
||||
self.record_protocol_success(outcome.peer, &outcome.protocol);
|
||||
@@ -1837,7 +1980,11 @@ impl Runtime {
|
||||
}
|
||||
|
||||
fn install_link(&mut self, inbound: InboundLink) {
|
||||
if self.no_direct_data(inbound.peer) {
|
||||
if self.no_direct_data(inbound.peer)
|
||||
|| inbound.link.is_closed()
|
||||
|| !self.sessions.contains_key(&inbound.peer)
|
||||
{
|
||||
inbound.link.close();
|
||||
return;
|
||||
}
|
||||
self.record_protocol_success(inbound.peer, &inbound.protocol);
|
||||
@@ -1848,19 +1995,64 @@ impl Runtime {
|
||||
|
||||
/// Hands a link to the plugin that owns its protocol.
|
||||
fn adopt_link(&mut self, peer: EndpointId, protocol: String, link: SharedLink) {
|
||||
if !self
|
||||
.params
|
||||
.plugins
|
||||
.iter()
|
||||
.any(|plugin| plugin.protocol_id() == protocol)
|
||||
if link.is_closed()
|
||||
|| link.network() != self.network_id
|
||||
|| link.peer() != peer
|
||||
|| !self.sessions.contains_key(&peer)
|
||||
{
|
||||
link.close();
|
||||
return;
|
||||
}
|
||||
if !self.params.plugins.iter().any(|plugin| {
|
||||
plugin.protocol_id() == protocol
|
||||
&& self.sessions.get(&peer).is_some_and(|s| {
|
||||
s.capabilities.iter().any(|c| {
|
||||
c.enabled
|
||||
&& c.protocol == protocol
|
||||
&& c.version == plugin.protocol_version()
|
||||
})
|
||||
})
|
||||
}) {
|
||||
link.close();
|
||||
return;
|
||||
}
|
||||
|
||||
let key = (peer, protocol.clone());
|
||||
let settings = if self.local_id.as_bytes() < peer.as_bytes() {
|
||||
let settings = HealthSettings::fresh();
|
||||
if let Ok(bytes) = postcard::to_stdvec(&(protocol.clone(), settings.clone())) {
|
||||
let _ = self.send_to(
|
||||
peer,
|
||||
ControlMessage::Transport {
|
||||
protocol: "link-health".into(),
|
||||
payload: crate::proto::message::TransportPayload::new(bytes),
|
||||
},
|
||||
);
|
||||
}
|
||||
Some(settings)
|
||||
} else {
|
||||
self.pending_health.remove(&key)
|
||||
};
|
||||
let route = self
|
||||
.sessions
|
||||
.get(&peer)
|
||||
.and_then(|s| self.tunnel_protocol(s))
|
||||
.and_then(|name| self.params.plugins.iter().find(|p| p.protocol_id() == name))
|
||||
.map(|plugin| ProbeRoute {
|
||||
plugin: plugin.clone(),
|
||||
routes: self.params.routes.clone(),
|
||||
network: self.network_id,
|
||||
peer,
|
||||
max_probe_size: self.params.interface.as_ref().map_or(540, |interface| {
|
||||
(interface.mtu() as usize).saturating_sub(36)
|
||||
}),
|
||||
});
|
||||
let link = HealthLink::new(link, self.local_id, settings, route);
|
||||
let path = link.path_description();
|
||||
let max_datagram = link.max_datagram_size();
|
||||
self.links
|
||||
.insert((peer, protocol.clone()), Arc::clone(&link));
|
||||
if let Some(old) = self.links.insert(key, Arc::clone(&link)) {
|
||||
old.close();
|
||||
}
|
||||
// What this is, is a way to reach the peer. Which protocol's
|
||||
// tunnel runs over it is decided in `update_paths`, and that tunnel
|
||||
// is handed the hub's link, not this one: the same object for as
|
||||
@@ -1879,26 +2071,47 @@ impl Runtime {
|
||||
|
||||
/// Drops every link to a peer.
|
||||
fn drop_links_for(&mut self, peer: EndpointId) {
|
||||
// The peer itself is going, not just a path to it, so the
|
||||
// protocol's link goes with it.
|
||||
self.reset_peer_data(peer, "peer session ended");
|
||||
self.hub.remove_peer(peer);
|
||||
self.reachable.remove(&peer);
|
||||
self.protocol_dial_states.retain(|(p, _), _| *p != peer);
|
||||
let keys: Vec<(EndpointId, String)> = self
|
||||
}
|
||||
|
||||
/// Retire only the data plane; authenticated membership and topology remain.
|
||||
fn reset_peer_data(&mut self, peer: EndpointId, reason: &str) {
|
||||
self.link_attempts.retain(|(id, _), _| *id != peer);
|
||||
self.opening.retain(|(id, _)| *id != peer);
|
||||
self.link_tasks.retain(|(id, _), task| {
|
||||
if *id == peer {
|
||||
task.abort();
|
||||
false
|
||||
} else {
|
||||
true
|
||||
}
|
||||
});
|
||||
self.hub.clear_peer_links(peer);
|
||||
let keys: Vec<_> = self
|
||||
.links
|
||||
.keys()
|
||||
.filter(|(id, _)| *id == peer)
|
||||
.cloned()
|
||||
.collect();
|
||||
for key in keys {
|
||||
self.links.remove(&key);
|
||||
if let Some(link) = self.links.remove(&key) {
|
||||
link.close();
|
||||
}
|
||||
self.emit(Event::DataLinkDown {
|
||||
network: self.network_id,
|
||||
peer,
|
||||
protocol: key.1,
|
||||
reason: "peer session ended".into(),
|
||||
reason: reason.into(),
|
||||
});
|
||||
}
|
||||
self.protocol_dial_states.retain(|(id, _), _| *id != peer);
|
||||
self.last_failed_protocol.remove(&peer);
|
||||
self.pending_health.retain(|(id, _), _| *id != peer);
|
||||
for plugin in &self.params.plugins {
|
||||
plugin.on_peer_gone(self.network_id, peer);
|
||||
}
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------- sessions
|
||||
@@ -2091,9 +2304,6 @@ impl Runtime {
|
||||
self.watch_exit_via();
|
||||
self.update_paths();
|
||||
self.announce_reach(false);
|
||||
for plugin in &self.params.plugins {
|
||||
plugin.on_peer_gone(self.network_id, peer);
|
||||
}
|
||||
// Retry promptly, then back off if it keeps failing.
|
||||
let policy = &self.params.reconnect;
|
||||
let state = self
|
||||
@@ -2140,8 +2350,64 @@ impl Runtime {
|
||||
|
||||
fn dispatch_message(&mut self, peer: EndpointId, message: ControlMessage) {
|
||||
match &message {
|
||||
ControlMessage::Transport { protocol, payload } => {
|
||||
if protocol == "link-health" {
|
||||
// Only the elected initiator supplies a profile, for an enabled,
|
||||
// mutually supported physical protocol in this authenticated session.
|
||||
if peer.as_bytes() >= self.local_id.as_bytes() {
|
||||
return;
|
||||
}
|
||||
let Ok((name, settings)) =
|
||||
postcard::from_bytes::<(String, HealthSettings)>(payload.as_bytes())
|
||||
else {
|
||||
return;
|
||||
};
|
||||
if name.len() > crate::dataplane::MAX_PROTOCOL_ID_LEN
|
||||
|| !self.served_protocols().iter().any(|(p, v)| {
|
||||
p == &name
|
||||
&& self.sessions.get(&peer).is_some_and(|s| {
|
||||
s.capabilities
|
||||
.iter()
|
||||
.any(|c| c.enabled && c.protocol == name && c.version == *v)
|
||||
})
|
||||
})
|
||||
{
|
||||
return;
|
||||
}
|
||||
let key = (peer, name);
|
||||
// Negotiation and inbound installation are separate queues.
|
||||
// Keep the newest settings even if the previous physical
|
||||
// link is still in the map when this control frame arrives.
|
||||
if let Some(link) = self.links.get(&key)
|
||||
&& !link.is_closed()
|
||||
{
|
||||
link.configure(settings.clone());
|
||||
}
|
||||
self.pending_health.insert(key, settings);
|
||||
return;
|
||||
}
|
||||
if let Some(transport) = &self.params.transport
|
||||
&& let Err(err) =
|
||||
transport.on_control(self.network_id, peer, protocol, payload.as_bytes())
|
||||
{
|
||||
tracing::debug!(%err, "transport negotiation rejected");
|
||||
}
|
||||
return;
|
||||
}
|
||||
ControlMessage::Announce(announcement) => {
|
||||
let capabilities = announcement.capabilities.clone();
|
||||
let changed = self.sessions.get(&peer).is_some_and(|session| {
|
||||
let selection = |caps: &[PluginCapability]| {
|
||||
caps.iter()
|
||||
.filter(|c| c.enabled)
|
||||
.map(|c| (c.protocol.clone(), c.version))
|
||||
.collect::<Vec<_>>()
|
||||
};
|
||||
selection(&session.capabilities) != selection(&capabilities)
|
||||
});
|
||||
if changed {
|
||||
self.reset_peer_data(peer, "peer protocol selection changed");
|
||||
}
|
||||
if let Some(session) = self.sessions.get_mut(&peer) {
|
||||
session.hostname = Some(announcement.hostname.clone());
|
||||
session.broadcast = announcement.broadcast;
|
||||
@@ -2154,6 +2420,7 @@ impl Runtime {
|
||||
self.dispatch_capabilities(peer, &capabilities);
|
||||
self.ensure_links();
|
||||
self.update_paths();
|
||||
self.announce_reach(false);
|
||||
}
|
||||
ControlMessage::Ping { seq, payload } => {
|
||||
let pong = ControlMessage::Pong {
|
||||
@@ -2387,6 +2654,13 @@ impl Runtime {
|
||||
members.dedup_by(|a, b| a.endpoint_id == b.endpoint_id);
|
||||
|
||||
NetworkStatus {
|
||||
protocols: self
|
||||
.params
|
||||
.plugins
|
||||
.iter()
|
||||
.map(|p| p.protocol_id().to_owned())
|
||||
.collect(),
|
||||
auto_start: true,
|
||||
broadcast: self.params.broadcast,
|
||||
exit_offer: self.params.exit_offer,
|
||||
exit_via: self.params.exit_via,
|
||||
@@ -2409,13 +2683,8 @@ impl Runtime {
|
||||
members,
|
||||
hostname_conflict: self.hostname_conflict_of(&self.local_id),
|
||||
range: self.effective_range(),
|
||||
// Only worth reporting while it is actually stuck: with a
|
||||
// fallback in play the configured range being another
|
||||
// network's is how it is meant to work, not a fault.
|
||||
range_conflict: self
|
||||
.params
|
||||
.range_conflict
|
||||
.filter(|_| self.effective_range().is_none()),
|
||||
range_conflict: None,
|
||||
range_conflicts: Vec::new(),
|
||||
metrics: self.metrics.clone(),
|
||||
relay: self.hub.counters(),
|
||||
}
|
||||
@@ -2662,3 +2931,17 @@ mod relay_host_tests {
|
||||
assert_eq!(relay_host("https://bad host/").as_deref(), None);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod fallback_order_tests {
|
||||
use super::next_protocol_index;
|
||||
#[test]
|
||||
fn degradation_advances_the_cycle_even_after_earlier_cooldowns_expire() {
|
||||
let protocols = ["wg", "awg", "wg-quic", "tcp-tls"].map(str::to_owned);
|
||||
assert_eq!(next_protocol_index(&protocols, None), 0);
|
||||
for (i, p) in protocols.iter().enumerate() {
|
||||
assert_eq!(next_protocol_index(&protocols, Some(p)), (i + 1) % 4);
|
||||
}
|
||||
assert_eq!(next_protocol_index(&protocols, Some("removed")), 0);
|
||||
}
|
||||
}
|
||||
@@ -70,7 +70,7 @@ pub(crate) struct Session {
|
||||
pub(crate) conn: Connection,
|
||||
/// Already encoded frame payloads, so the runtime can account for the exact
|
||||
/// number of control bytes it queues.
|
||||
pub(crate) outbound: mpsc::Sender<Vec<u8>>,
|
||||
pub(crate) outbound: mpsc::Sender<zeroize::Zeroizing<Vec<u8>>>,
|
||||
pub(crate) hostname: Option<String>,
|
||||
pub(crate) capabilities: Vec<PluginCapability>,
|
||||
pub(crate) broadcast: bool,
|
||||
@@ -244,7 +244,7 @@ pub(crate) fn spawn(
|
||||
|
||||
async fn writer_task(
|
||||
mut send: SendStream,
|
||||
mut outbound: mpsc::Receiver<Vec<u8>>,
|
||||
mut outbound: mpsc::Receiver<zeroize::Zeroizing<Vec<u8>>>,
|
||||
limits: Arc<Limits>,
|
||||
shutdown: Shutdown,
|
||||
parent: Shutdown,
|
||||
@@ -331,6 +331,7 @@ async fn reader_task(
|
||||
}
|
||||
};
|
||||
|
||||
let payload = zeroize::Zeroizing::new(payload);
|
||||
let bytes = payload.len();
|
||||
let envelope: Envelope = match decode(&payload) {
|
||||
Ok(envelope) => envelope,
|
||||
|
||||
@@ -209,9 +209,24 @@ pub struct NetworkMetrics {
|
||||
pub data_link_failures: u64,
|
||||
}
|
||||
|
||||
/// An active network preventing another network from starting locally.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RangeConflict {
|
||||
/// The blocking network's public identifier.
|
||||
pub network_id: NetworkId,
|
||||
/// Its configured name.
|
||||
pub name: NetworkName,
|
||||
/// Its overlapping overlay range.
|
||||
pub range: crate::state::Ipv4Range,
|
||||
}
|
||||
|
||||
/// Status of one network.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NetworkStatus {
|
||||
/// Enabled local data-plane protocols, in preference order.
|
||||
pub protocols: Vec<String>,
|
||||
/// Whether this network is requested to start or resume automatically.
|
||||
pub auto_start: bool,
|
||||
/// Local broadcast participation in this network.
|
||||
pub broadcast: bool,
|
||||
/// This agent offers itself as an exit node in this network.
|
||||
@@ -242,13 +257,10 @@ pub struct NetworkStatus {
|
||||
pub hostname_conflict: Option<HostnameConflict>,
|
||||
/// The overlay range this network uses, once it has one.
|
||||
pub range: Option<crate::state::Ipv4Range>,
|
||||
/// The range it could not have, because another network on this agent
|
||||
/// already holds it.
|
||||
///
|
||||
/// One agent has one interface, so an address belongs to one network.
|
||||
/// This network waits to adopt whatever its members settle on instead of
|
||||
/// proposing something it could not route.
|
||||
/// The first overlapping active range; `range_conflicts` identifies all blockers.
|
||||
pub range_conflict: Option<crate::state::Ipv4Range>,
|
||||
/// Active networks that would be suspended by switching to this network.
|
||||
pub range_conflicts: Vec<RangeConflict>,
|
||||
/// Per-network counters.
|
||||
pub metrics: NetworkMetrics,
|
||||
/// What has gone through a peer in the middle, in both directions.
|
||||
|
||||
@@ -166,6 +166,8 @@ pub struct Limits {
|
||||
pub max_capabilities: usize,
|
||||
/// Largest opaque plugin payload, in bytes.
|
||||
pub max_capability_data_len: usize,
|
||||
/// Largest private transport negotiation payload (also hard-capped at 4096 bytes).
|
||||
pub max_transport_payload_len: usize,
|
||||
/// Largest accepted echo payload in a ping/pong exchange, in bytes.
|
||||
pub max_echo_payload_len: usize,
|
||||
/// Largest accepted free-text reason string, in bytes.
|
||||
@@ -204,6 +206,7 @@ impl Default for Limits {
|
||||
max_frame_len: 64 * 1024,
|
||||
max_hostname_len: 255,
|
||||
max_capabilities: 16,
|
||||
max_transport_payload_len: 4096,
|
||||
max_capability_data_len: 4 * 1024,
|
||||
max_echo_payload_len: 4 * 1024,
|
||||
max_reason_len: 256,
|
||||
@@ -325,6 +328,9 @@ pub struct AgentConfig {
|
||||
pub reconnect: ReconnectPolicy,
|
||||
/// IP plugins whose capabilities are announced and dispatched.
|
||||
pub plugins: Vec<SharedPlugin>,
|
||||
/// Initial ordered selection for networks without a saved override.
|
||||
/// `None` enables all registered plugins; an empty list is control-only.
|
||||
pub default_protocols: Option<Vec<String>>,
|
||||
/// Custom packet transports registered per protocol.
|
||||
pub custom_transports:
|
||||
std::collections::HashMap<String, Arc<dyn crate::dataplane::PacketTransport>>,
|
||||
@@ -339,7 +345,11 @@ pub struct AgentConfig {
|
||||
pub interface_name: String,
|
||||
/// The interface MTU.
|
||||
pub interface_mtu: u32,
|
||||
/// The IPv4 overlay range this agent proposes.
|
||||
/// Whether new networks derive their IPv4 range from their network id.
|
||||
///
|
||||
/// Enabled by default. Explicit ranges and IPv4 opt-out disable this.
|
||||
pub overlay_ipv4_auto: bool,
|
||||
/// The IPv4 overlay range this agent proposes when automatic selection is off.
|
||||
///
|
||||
/// Addresses are allocated from it and recorded in signed state, so a
|
||||
/// participant keeps the same one across restarts. A network that has
|
||||
@@ -369,11 +379,13 @@ impl AgentConfig {
|
||||
limits: Limits::default(),
|
||||
reconnect: ReconnectPolicy::default(),
|
||||
plugins: Vec::new(),
|
||||
default_protocols: None,
|
||||
custom_transports: std::collections::HashMap::new(),
|
||||
tun_factory: None,
|
||||
interface_name: "tsun0".to_string(),
|
||||
interface_mtu: 1280,
|
||||
overlay_ipv4_range: Some(crate::state::DEFAULT_IPV4_RANGE),
|
||||
overlay_ipv4_auto: true,
|
||||
overlay_ipv4_range: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -445,6 +457,7 @@ impl AgentConfig {
|
||||
|
||||
/// Sets the IPv4 overlay range this agent proposes, or disables IPv4.
|
||||
pub fn with_overlay_ipv4_range(mut self, range: Option<crate::state::Ipv4Range>) -> Self {
|
||||
self.overlay_ipv4_auto = false;
|
||||
self.overlay_ipv4_range = range;
|
||||
self
|
||||
}
|
||||
|
||||
@@ -40,6 +40,33 @@ pub use tunnel::{GenericTunnelPlugin, TunnelCodec};
|
||||
/// Maximum length of a plugin protocol identifier.
|
||||
pub const MAX_PROTOCOL_ID_LEN: usize = 32;
|
||||
|
||||
/// Validates bounded local protocol preferences before storing or applying them.
|
||||
pub(crate) fn validate_protocols(protocols: &[String], max: usize) -> crate::error::Result<()> {
|
||||
if protocols.len() > max {
|
||||
return Err(crate::Error::InvalidProtocols(format!(
|
||||
"at most {max} protocols are allowed"
|
||||
)));
|
||||
}
|
||||
for (index, name) in protocols.iter().enumerate() {
|
||||
if name.is_empty()
|
||||
|| name.len() > MAX_PROTOCOL_ID_LEN
|
||||
|| !name
|
||||
.bytes()
|
||||
.all(|c| c.is_ascii_alphanumeric() || c == b'-' || c == b'_')
|
||||
{
|
||||
return Err(crate::Error::InvalidProtocols(
|
||||
"names must be 1–32 ASCII letters, digits, hyphens or underscores".into(),
|
||||
));
|
||||
}
|
||||
if protocols[..index].contains(name) {
|
||||
return Err(crate::Error::InvalidProtocols(format!(
|
||||
"`{name}` is listed more than once"
|
||||
)));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// An announcement of one IP plugin's capability.
|
||||
///
|
||||
/// `data` is opaque to the core. Nothing in it may be interpreted as a shell
|
||||
|
||||
@@ -492,11 +492,12 @@ impl RelayHub {
|
||||
}
|
||||
}
|
||||
|
||||
/// Revokes all links and topology involving a departed authenticated peer.
|
||||
pub fn remove_peer(&self, peer: EndpointId) {
|
||||
/// Retires data channels without removing an authenticated peer's topology.
|
||||
pub fn clear_peer_links(&self, peer: EndpointId) {
|
||||
let mut state = self.state();
|
||||
if let Some(links) = state.raw.remove(peer.as_bytes()) {
|
||||
for raw in links {
|
||||
raw.link.close();
|
||||
raw.reader.abort();
|
||||
}
|
||||
}
|
||||
@@ -511,6 +512,13 @@ impl RelayHub {
|
||||
link.close();
|
||||
}
|
||||
}
|
||||
state.publish();
|
||||
}
|
||||
|
||||
/// Revokes all links and topology involving a departed authenticated peer.
|
||||
pub fn remove_peer(&self, peer: EndpointId) {
|
||||
self.clear_peer_links(peer);
|
||||
let mut state = self.state();
|
||||
state.members.remove(peer.as_bytes());
|
||||
state.graph.remove(peer.as_bytes());
|
||||
for neighbors in state.graph.values_mut() {
|
||||
|
||||
@@ -0,0 +1,350 @@
|
||||
//! Authenticated probes of the actual data link, independent of control RTT.
|
||||
//! Keys are negotiated privately for each physical link. No fixed wire magic.
|
||||
use super::{PacketLink, SharedLink, TransportError};
|
||||
use crate::dataplane::SharedPlugin;
|
||||
use crate::overlay::RoutingTable;
|
||||
use crate::{BoxFuture, NetworkId};
|
||||
use bytes::Bytes;
|
||||
use hmac::{Hmac, KeyInit, Mac};
|
||||
use iroh::EndpointId;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sha2::Sha256;
|
||||
use std::collections::VecDeque;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::{Duration, Instant};
|
||||
use tokio::sync::{mpsc, watch};
|
||||
|
||||
#[derive(Clone, Serialize, Deserialize, zeroize::ZeroizeOnDrop)]
|
||||
pub(crate) struct Settings {
|
||||
marker: [u8; 16],
|
||||
key: [u8; 32],
|
||||
}
|
||||
impl Settings {
|
||||
pub(crate) fn fresh() -> Self {
|
||||
Self {
|
||||
marker: rand::random(),
|
||||
key: rand::random(),
|
||||
}
|
||||
}
|
||||
}
|
||||
impl std::fmt::Debug for Settings {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str("HealthSettings([REDACTED])")
|
||||
}
|
||||
}
|
||||
struct Samples {
|
||||
pending: VecDeque<(u64, Instant)>,
|
||||
recent: VecDeque<bool>,
|
||||
}
|
||||
impl Samples {
|
||||
fn new() -> Self {
|
||||
Self {
|
||||
pending: VecDeque::new(),
|
||||
recent: VecDeque::new(),
|
||||
}
|
||||
}
|
||||
fn sent(&mut self, seq: u64, now: Instant) {
|
||||
self.pending.push_back((seq, now));
|
||||
}
|
||||
fn acknowledge(&mut self, seq: u64) {
|
||||
if let Some(i) = self.pending.iter().position(|(n, _)| *n == seq) {
|
||||
self.pending.remove(i);
|
||||
self.recent.push_back(true);
|
||||
}
|
||||
}
|
||||
fn expire(&mut self, now: Instant) {
|
||||
while self
|
||||
.pending
|
||||
.front()
|
||||
.is_some_and(|(_, at)| now.duration_since(*at) >= Duration::from_secs(3))
|
||||
{
|
||||
self.pending.pop_front();
|
||||
self.recent.push_back(false);
|
||||
}
|
||||
while self.recent.len() > 20 {
|
||||
self.recent.pop_front();
|
||||
}
|
||||
}
|
||||
fn degraded(&self) -> bool {
|
||||
self.recent.len() >= 15
|
||||
&& self.recent.iter().filter(|ok| !**ok).count() * 100 >= self.recent.len() * 60
|
||||
}
|
||||
}
|
||||
// This prefix is inside end-to-end encryption. It never appears on the UDP wire.
|
||||
const IP_MAGIC: &[u8; 16] = b"tsunagi-health01";
|
||||
pub(crate) fn ip_payload(packet: &[u8]) -> Option<&[u8]> {
|
||||
(packet.len() >= 36 && packet[0] == 0x45 && packet[9] == 253 && &packet[20..36] == IP_MAGIC)
|
||||
.then(|| &packet[36..])
|
||||
}
|
||||
/// Inject probes through the same encrypted tunnel as application IP packets.
|
||||
/// This matters for pure WG: raw UDP probes could pass DPI while WG is blocked.
|
||||
pub(crate) struct ProbeRoute {
|
||||
pub(crate) plugin: SharedPlugin,
|
||||
pub(crate) routes: Arc<RoutingTable>,
|
||||
pub(crate) network: NetworkId,
|
||||
pub(crate) peer: EndpointId,
|
||||
pub(crate) max_probe_size: usize,
|
||||
}
|
||||
impl ProbeRoute {
|
||||
fn send(&self, payload: Bytes) -> bool {
|
||||
let Some((source, destination)) = self.routes.probe_addresses(self.network, self.peer)
|
||||
else {
|
||||
return false;
|
||||
};
|
||||
let mut packet = vec![0; 36];
|
||||
packet[0] = 0x45;
|
||||
packet[8] = 64;
|
||||
packet[9] = 253;
|
||||
packet[12..16].copy_from_slice(&source.octets());
|
||||
packet[16..20].copy_from_slice(&destination.octets());
|
||||
packet[20..36].copy_from_slice(IP_MAGIC);
|
||||
let length = (packet.len() + payload.len()) as u16;
|
||||
packet[2..4].copy_from_slice(&length.to_be_bytes());
|
||||
let sum = packet[..20]
|
||||
.as_chunks::<2>()
|
||||
.0
|
||||
.iter()
|
||||
.map(|b| u32::from(u16::from_be_bytes(*b)))
|
||||
.sum::<u32>();
|
||||
let sum = (sum & 0xffff) + (sum >> 16);
|
||||
packet[10..12].copy_from_slice(&(!(sum as u16)).to_be_bytes());
|
||||
packet.extend_from_slice(&payload);
|
||||
self.plugin
|
||||
.carry(self.network, self.peer, Bytes::from(packet))
|
||||
}
|
||||
}
|
||||
/// A monitored physical link. It forwards application bytes unchanged.
|
||||
pub(crate) struct HealthLink {
|
||||
probes: mpsc::Sender<Bytes>,
|
||||
raw: SharedLink,
|
||||
settings: Mutex<Option<Settings>>,
|
||||
rx: tokio::sync::Mutex<mpsc::Receiver<Bytes>>,
|
||||
stop: watch::Sender<bool>,
|
||||
reason: Mutex<Option<String>>,
|
||||
}
|
||||
impl std::fmt::Debug for HealthLink {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("HealthLink")
|
||||
.field("raw", &self.raw)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
fn lock<T>(m: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
|
||||
match m.lock() {
|
||||
Ok(g) => g,
|
||||
Err(p) => p.into_inner(),
|
||||
}
|
||||
}
|
||||
impl HealthLink {
|
||||
pub(crate) fn new(
|
||||
raw: SharedLink,
|
||||
local: EndpointId,
|
||||
settings: Option<Settings>,
|
||||
route: Option<ProbeRoute>,
|
||||
) -> Arc<Self> {
|
||||
let (tx, rx) = mpsc::channel(64);
|
||||
let (probe_tx, mut probe_rx) = mpsc::channel(64);
|
||||
let (stop, _) = watch::channel(false);
|
||||
let link = Arc::new(Self {
|
||||
probes: probe_tx,
|
||||
raw: raw.clone(),
|
||||
settings: Mutex::new(settings),
|
||||
rx: tokio::sync::Mutex::new(rx),
|
||||
stop,
|
||||
reason: Mutex::new(None),
|
||||
});
|
||||
let weak = Arc::downgrade(&link);
|
||||
let mut stopped = link.stop.subscribe();
|
||||
tokio::spawn(async move {
|
||||
let mut interval = tokio::time::interval(Duration::from_secs(1));
|
||||
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
|
||||
let mut samples = Samples::new();
|
||||
let mut sequence = rand::random::<u64>();
|
||||
let mut since: Option<Instant> = None;
|
||||
let mut marker = None;
|
||||
loop {
|
||||
tokio::select! {
|
||||
biased;
|
||||
_=stopped.changed()=>break,
|
||||
_=interval.tick()=>{
|
||||
let Some(link)=weak.upgrade()else{break;};let Some(settings)=lock(&link.settings).clone()else{continue;};
|
||||
if marker != Some(settings.marker) { marker = Some(settings.marker); samples = Samples::new(); since = None; }
|
||||
let start=*since.get_or_insert_with(Instant::now);let now=Instant::now();samples.expire(now);
|
||||
if now.duration_since(start)>=Duration::from_secs(20) && samples.degraded() {
|
||||
*lock(&link.reason)=Some("data delivery checks: at least 60% loss over the last 20 probes".into());break;
|
||||
}
|
||||
sequence=sequence.wrapping_add(1);
|
||||
let size=route.as_ref().map_or(raw.max_datagram_size(),|route|route.max_probe_size);
|
||||
let packet=probe(&settings,local,1,sequence,size.min(1000));
|
||||
if let Some(route)=&route { if route.send(packet) {samples.sent(sequence,now);} }
|
||||
else { let _=raw.send(packet); samples.sent(sequence,now); }
|
||||
}
|
||||
packet=raw.recv()=>{
|
||||
let Some(packet)=packet else{break;};let Some(link)=weak.upgrade()else{break;};
|
||||
let settings=lock(&link.settings).clone();
|
||||
if route.is_none() && let Some(settings)=settings && packet.starts_with(&settings.marker) {
|
||||
if let Some((kind,seq))=verify(&settings,raw.peer(),&packet) {
|
||||
if kind==1 {let _=raw.send(probe(&settings,local,2,seq,packet.len()));}
|
||||
else if kind==2 {samples.acknowledge(seq);}
|
||||
}
|
||||
} else {let _=tx.try_send(packet);}
|
||||
}
|
||||
packet=probe_rx.recv(),if route.is_some()=>{
|
||||
let Some(packet)=packet else {break;};let Some(link)=weak.upgrade()else{break;};
|
||||
let Some(settings)=lock(&link.settings).clone()else{continue;};
|
||||
if let Some((kind,seq))=verify(&settings,raw.peer(),&packet) {
|
||||
if kind==1 { if let Some(route)=&route { route.send(probe(&settings,local,2,seq,packet.len().min(route.max_probe_size))); } }
|
||||
else if kind==2 {samples.acknowledge(seq);}
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
raw.close();
|
||||
if let Some(link) = weak.upgrade() {
|
||||
link.stop.send_replace(true);
|
||||
}
|
||||
});
|
||||
link
|
||||
}
|
||||
pub(crate) fn receive_probe(&self, packet: Bytes) {
|
||||
let _ = self.probes.try_send(packet);
|
||||
}
|
||||
pub(crate) fn configure(&self, settings: Settings) {
|
||||
*lock(&self.settings) = Some(settings);
|
||||
}
|
||||
}
|
||||
fn probe(settings: &Settings, sender: EndpointId, kind: u8, seq: u64, size: usize) -> Bytes {
|
||||
let mut bytes = vec![0; size.max(57)];
|
||||
bytes[..16].copy_from_slice(&settings.marker);
|
||||
bytes[16] = kind;
|
||||
bytes[17..25].copy_from_slice(&seq.to_be_bytes());
|
||||
rand::fill(&mut bytes[25..size.max(57) - 32]);
|
||||
let tail = bytes.len() - 32;
|
||||
if let Ok(mut mac) = <Hmac<Sha256> as KeyInit>::new_from_slice(&settings.key) {
|
||||
mac.update(sender.as_bytes());
|
||||
mac.update(&bytes[..tail]);
|
||||
bytes[tail..].copy_from_slice(&mac.finalize().into_bytes());
|
||||
}
|
||||
Bytes::from(bytes)
|
||||
}
|
||||
fn verify(settings: &Settings, sender: EndpointId, bytes: &[u8]) -> Option<(u8, u64)> {
|
||||
if bytes.len() < 57 || !bytes.starts_with(&settings.marker) {
|
||||
return None;
|
||||
}
|
||||
let mut mac = <Hmac<Sha256> as KeyInit>::new_from_slice(&settings.key).ok()?;
|
||||
let tail = bytes.len() - 32;
|
||||
mac.update(sender.as_bytes());
|
||||
mac.update(&bytes[..tail]);
|
||||
mac.verify_slice(&bytes[tail..]).ok()?;
|
||||
Some((
|
||||
bytes[16],
|
||||
u64::from_be_bytes(bytes.get(17..25)?.try_into().ok()?),
|
||||
))
|
||||
}
|
||||
impl PacketLink for HealthLink {
|
||||
fn network(&self) -> NetworkId {
|
||||
self.raw.network()
|
||||
}
|
||||
fn peer(&self) -> EndpointId {
|
||||
self.raw.peer()
|
||||
}
|
||||
fn max_datagram_size(&self) -> usize {
|
||||
self.raw.max_datagram_size()
|
||||
}
|
||||
fn send(&self, payload: Bytes) -> Result<(), TransportError> {
|
||||
if self.is_closed() {
|
||||
Err(TransportError::Closed)
|
||||
} else {
|
||||
self.raw.send(payload)
|
||||
}
|
||||
}
|
||||
fn send_flow(
|
||||
&self,
|
||||
payload: Bytes,
|
||||
flow: super::super::routing::FlowId,
|
||||
) -> Result<(), TransportError> {
|
||||
if self.is_closed() {
|
||||
Err(TransportError::Closed)
|
||||
} else {
|
||||
self.raw.send_flow(payload, flow)
|
||||
}
|
||||
}
|
||||
fn recv(&self) -> BoxFuture<'_, Option<Bytes>> {
|
||||
Box::pin(async move {
|
||||
let mut stop = self.stop.subscribe();
|
||||
if *stop.borrow() {
|
||||
return None;
|
||||
}
|
||||
let mut rx = self.rx.lock().await;
|
||||
tokio::select! {biased;_=stop.changed()=>None,packet=rx.recv()=>packet}
|
||||
})
|
||||
}
|
||||
fn closed(&self) -> BoxFuture<'_, ()> {
|
||||
Box::pin(async move {
|
||||
let mut stop = self.stop.subscribe();
|
||||
if !*stop.borrow() {
|
||||
let _ = stop.changed().await;
|
||||
}
|
||||
})
|
||||
}
|
||||
fn is_closed(&self) -> bool {
|
||||
*self.stop.borrow() || self.raw.is_closed()
|
||||
}
|
||||
fn close(&self) {
|
||||
self.stop.send_replace(true);
|
||||
self.raw.close();
|
||||
}
|
||||
fn failure_reason(&self) -> Option<String> {
|
||||
lock(&self.reason)
|
||||
.clone()
|
||||
.or_else(|| self.raw.failure_reason())
|
||||
}
|
||||
fn path_description(&self) -> String {
|
||||
self.raw.path_description()
|
||||
}
|
||||
}
|
||||
impl Drop for HealthLink {
|
||||
fn drop(&mut self) {
|
||||
self.stop.send_replace(true);
|
||||
self.raw.close();
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
use super::*;
|
||||
#[test]
|
||||
fn no_false_failure_before_window_but_persistent_loss_is_detected() {
|
||||
let mut samples = Samples::new();
|
||||
let now = Instant::now();
|
||||
for seq in 0..20 {
|
||||
samples.sent(seq, now);
|
||||
if seq % 5 == 0 {
|
||||
samples.acknowledge(seq);
|
||||
}
|
||||
}
|
||||
assert!(!samples.degraded());
|
||||
samples.expire(now + Duration::from_secs(4));
|
||||
assert!(samples.degraded());
|
||||
for seq in 20..40 {
|
||||
samples.sent(seq, now);
|
||||
samples.acknowledge(seq);
|
||||
}
|
||||
samples.expire(now + Duration::from_secs(4));
|
||||
assert!(!samples.degraded());
|
||||
}
|
||||
#[test]
|
||||
fn a_forged_or_reflected_probe_cannot_claim_delivery() {
|
||||
let a = crate::identity::DeviceIdentity::generate().endpoint_id();
|
||||
let b = crate::identity::DeviceIdentity::generate().endpoint_id();
|
||||
let settings = Settings::fresh();
|
||||
let packet = probe(&settings, a, 1, 42, 1100);
|
||||
assert_eq!(verify(&settings, a, &packet), Some((1, 42)));
|
||||
assert!(verify(&settings, b, &packet).is_none());
|
||||
let mut forged = packet.to_vec();
|
||||
forged[16] = 2;
|
||||
assert!(verify(&settings, a, &forged).is_none());
|
||||
}
|
||||
}
|
||||
@@ -97,6 +97,9 @@ impl IrohLink {
|
||||
}
|
||||
|
||||
impl PacketLink for IrohLink {
|
||||
fn close(&self) {
|
||||
self.conn.close(0u32.into(), b"data link retired");
|
||||
}
|
||||
fn network(&self) -> NetworkId {
|
||||
self.network
|
||||
}
|
||||
|
||||
@@ -30,12 +30,14 @@
|
||||
//! one plugin protocol.
|
||||
|
||||
mod fragments;
|
||||
pub(crate) mod health;
|
||||
pub mod iroh_link;
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use bytes::Bytes;
|
||||
use iroh::EndpointId;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::BoxFuture;
|
||||
use crate::identity::NetworkId;
|
||||
@@ -112,6 +114,14 @@ pub trait PacketLink: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// keeping a task parked on [`PacketLink::closed`].
|
||||
fn is_closed(&self) -> bool;
|
||||
|
||||
/// Explicitly retires a physical link. Implementations should wake readers.
|
||||
fn close(&self) {}
|
||||
|
||||
/// Observed reason for retiring this link, if available.
|
||||
fn failure_reason(&self) -> Option<String> {
|
||||
None
|
||||
}
|
||||
|
||||
/// A short description of the path in use, for diagnostics.
|
||||
///
|
||||
/// Reports what the transport actually knows. It must not invent a value.
|
||||
@@ -121,6 +131,76 @@ pub trait PacketLink: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// A shared handle to a link.
|
||||
pub type SharedLink = std::sync::Arc<dyn PacketLink>;
|
||||
|
||||
/// Maximum opaque pair-negotiation payload.
|
||||
pub const MAX_TRANSPORT_PAYLOAD: usize = 4096;
|
||||
|
||||
/// Private transport bytes, never included in Debug output.
|
||||
#[derive(Clone, PartialEq, Eq, Serialize, zeroize::ZeroizeOnDrop)]
|
||||
pub struct TransportPayload(Vec<u8>);
|
||||
|
||||
impl<'de> Deserialize<'de> for TransportPayload {
|
||||
fn deserialize<D: serde::Deserializer<'de>>(
|
||||
deserializer: D,
|
||||
) -> std::result::Result<Self, D::Error> {
|
||||
struct Visitor;
|
||||
impl<'de> serde::de::Visitor<'de> for Visitor {
|
||||
type Value = TransportPayload;
|
||||
fn expecting(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(f, "at most {MAX_TRANSPORT_PAYLOAD} private transport bytes")
|
||||
}
|
||||
fn visit_seq<A: serde::de::SeqAccess<'de>>(
|
||||
self,
|
||||
mut sequence: A,
|
||||
) -> std::result::Result<Self::Value, A::Error> {
|
||||
if sequence
|
||||
.size_hint()
|
||||
.is_some_and(|size| size > MAX_TRANSPORT_PAYLOAD)
|
||||
{
|
||||
return Err(serde::de::Error::custom("transport payload too large"));
|
||||
}
|
||||
let mut payload = TransportPayload(Vec::with_capacity(
|
||||
sequence.size_hint().unwrap_or(0).min(MAX_TRANSPORT_PAYLOAD),
|
||||
));
|
||||
while let Some(byte) = sequence.next_element::<u8>()? {
|
||||
if payload.0.len() == MAX_TRANSPORT_PAYLOAD {
|
||||
return Err(serde::de::Error::custom("transport payload too large"));
|
||||
}
|
||||
payload.0.push(byte);
|
||||
}
|
||||
Ok(payload)
|
||||
}
|
||||
}
|
||||
deserializer.deserialize_seq(Visitor)
|
||||
}
|
||||
}
|
||||
impl TransportPayload {
|
||||
/// Wraps bytes for one authenticated control session.
|
||||
pub fn new(bytes: Vec<u8>) -> Self {
|
||||
Self(bytes)
|
||||
}
|
||||
/// Borrows private bytes; callers must not log them.
|
||||
pub fn as_bytes(&self) -> &[u8] {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
impl std::fmt::Debug for TransportPayload {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str("TransportPayload([REDACTED])")
|
||||
}
|
||||
}
|
||||
|
||||
/// Private control-plane sender for transports. Contains no network secret.
|
||||
pub trait ControlSender: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// Queues a bounded message to one authenticated member of one network.
|
||||
fn send<'a>(
|
||||
&'a self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
protocol: &'a str,
|
||||
payload: TransportPayload,
|
||||
) -> BoxFuture<'a, Result<(), TransportError>>;
|
||||
}
|
||||
|
||||
/// An inbound link a peer opened towards us.
|
||||
#[derive(Debug)]
|
||||
pub struct InboundLink {
|
||||
@@ -139,6 +219,18 @@ pub struct InboundLink {
|
||||
/// The agent owns one of these and hands links to plugins; plugins never call
|
||||
/// it directly.
|
||||
pub trait PacketTransport: Send + Sync + std::fmt::Debug + 'static {
|
||||
/// Supplies a private sender before network activation.
|
||||
fn attach_control(&self, _control: Arc<dyn ControlSender>) {}
|
||||
/// Validates and queues private negotiation received from an authenticated member.
|
||||
fn on_control(
|
||||
&self,
|
||||
_network: NetworkId,
|
||||
_peer: EndpointId,
|
||||
_protocol: &str,
|
||||
_payload: &[u8],
|
||||
) -> Result<(), TransportError> {
|
||||
Err(TransportError::Declined(self.name().into()))
|
||||
}
|
||||
/// A short name used in diagnostics.
|
||||
fn name(&self) -> &str;
|
||||
|
||||
@@ -184,6 +276,24 @@ impl MultiPacketTransport {
|
||||
}
|
||||
|
||||
impl PacketTransport for MultiPacketTransport {
|
||||
fn attach_control(&self, control: Arc<dyn ControlSender>) {
|
||||
self.default.attach_control(control.clone());
|
||||
for transport in self.by_protocol.values() {
|
||||
transport.attach_control(control.clone());
|
||||
}
|
||||
}
|
||||
fn on_control(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
protocol: &str,
|
||||
payload: &[u8],
|
||||
) -> Result<(), TransportError> {
|
||||
match self.by_protocol.get(protocol) {
|
||||
Some(transport) => transport.on_control(network, peer, protocol, payload),
|
||||
None => Err(TransportError::Declined(protocol.into())),
|
||||
}
|
||||
}
|
||||
fn name(&self) -> &str {
|
||||
"multi"
|
||||
}
|
||||
|
||||
@@ -15,6 +15,9 @@ pub type Result<T, E = Error> = std::result::Result<T, E>;
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[non_exhaustive]
|
||||
pub enum Error {
|
||||
/// An ordered local data-plane protocol selection is invalid.
|
||||
#[error("invalid protocol selection: {0}")]
|
||||
InvalidProtocols(String),
|
||||
/// The supplied network name does not satisfy the documented rules.
|
||||
#[error("invalid network name: {0}")]
|
||||
InvalidNetworkName(&'static str),
|
||||
@@ -97,6 +100,19 @@ pub enum Error {
|
||||
#[error("network {0} is not configured")]
|
||||
NetworkUnknown(NetworkId),
|
||||
|
||||
/// An overlay range overlaps networks already active on this agent.
|
||||
#[error(
|
||||
"network {network} cannot start: {range} overlaps active networks {conflicts:?}; switch to it to suspend those networks"
|
||||
)]
|
||||
NetworkRangeConflict {
|
||||
/// Network that could not start.
|
||||
network: NetworkId,
|
||||
/// Its agreed or proposed range.
|
||||
range: crate::state::Ipv4Range,
|
||||
/// Active networks that must be suspended first.
|
||||
conflicts: Vec<NetworkId>,
|
||||
},
|
||||
|
||||
/// No session with that peer exists in the given network.
|
||||
#[error("no authenticated session with peer {peer} in network {network}")]
|
||||
NoSuchPeer {
|
||||
|
||||
@@ -171,6 +171,13 @@ pub fn default_control_socket_path() -> PathBuf {
|
||||
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[non_exhaustive]
|
||||
pub enum Request {
|
||||
/// Replace one network's ordered local data-plane selection.
|
||||
SetProtocols {
|
||||
/// Public network identifier.
|
||||
network_id: String,
|
||||
/// Protocol names, best first; empty selects control only.
|
||||
protocols: Vec<String>,
|
||||
},
|
||||
/// Report what the agent is doing.
|
||||
Status,
|
||||
/// Set local broadcast participation in one configured network.
|
||||
@@ -255,6 +262,13 @@ pub enum Request {
|
||||
impl std::fmt::Debug for Request {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Request::SetProtocols {
|
||||
network_id,
|
||||
protocols,
|
||||
} => write!(
|
||||
f,
|
||||
"SetProtocols {{ {network_id}, protocols: {protocols:?} }}"
|
||||
),
|
||||
Request::Status => f.write_str("Status"),
|
||||
Request::SetBroadcast {
|
||||
network_id,
|
||||
@@ -286,6 +300,8 @@ impl std::fmt::Debug for Request {
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[non_exhaustive]
|
||||
pub enum Response {
|
||||
/// Accepted ordered protocol selection.
|
||||
Protocols(Vec<String>),
|
||||
/// A status report.
|
||||
Status(Box<StatusReport>),
|
||||
/// Accepted local broadcast participation.
|
||||
@@ -324,6 +340,9 @@ pub struct ActiveReport {
|
||||
/// Stopping something already stopped is not an error — it is the state
|
||||
/// asked for — but it is worth saying which happened.
|
||||
pub changed: bool,
|
||||
/// Networks suspended by switching to this network.
|
||||
#[serde(default)]
|
||||
pub suspended: Vec<RangeConflictReport>,
|
||||
}
|
||||
|
||||
/// What happened when a network was joined.
|
||||
@@ -345,6 +364,13 @@ pub struct JoinedReport {
|
||||
/// Almost always a mistyped secret, and the one thing that makes two
|
||||
/// sections of a report look like one network.
|
||||
pub name_shared_with: Option<String>,
|
||||
/// Whether it started; false means it is configured but blocked by overlap.
|
||||
#[serde(default = "joined_active_default")]
|
||||
pub active: bool,
|
||||
}
|
||||
|
||||
fn joined_active_default() -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
/// What happened when a network was left.
|
||||
@@ -367,6 +393,8 @@ pub struct LeftReport {
|
||||
/// Everything the agent is doing, in one snapshot.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct StatusReport {
|
||||
/// Data-plane protocols this agent can enable at runtime.
|
||||
pub available_protocols: Vec<ProtocolReport>,
|
||||
/// This device's persistent endpoint id.
|
||||
pub endpoint_id: String,
|
||||
/// Hostname announced to peers.
|
||||
@@ -437,9 +465,34 @@ pub struct DnsReport {
|
||||
pub names: u32,
|
||||
}
|
||||
|
||||
/// An active network with an overlapping overlay range.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct RangeConflictReport {
|
||||
/// Public network id, for unambiguous selection.
|
||||
pub network_id: String,
|
||||
/// Configured network name.
|
||||
pub name: String,
|
||||
/// The overlapping range.
|
||||
pub range: String,
|
||||
}
|
||||
|
||||
/// A data-plane protocol available in the running agent.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct ProtocolReport {
|
||||
/// Stable protocol name used by the CLI and network negotiation.
|
||||
pub name: String,
|
||||
/// A short description of encryption and transport.
|
||||
pub summary: String,
|
||||
}
|
||||
|
||||
/// One network.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct NetworkReport {
|
||||
/// Enabled local data-plane protocols, in preference order.
|
||||
pub protocols: Vec<String>,
|
||||
/// Whether the network starts or resumes automatically when its range is free.
|
||||
#[serde(default)]
|
||||
pub auto_start: bool,
|
||||
/// Whether this agent participates in LAN broadcasts in this network.
|
||||
#[serde(default)]
|
||||
pub broadcast: bool,
|
||||
@@ -482,6 +535,9 @@ pub struct NetworkReport {
|
||||
/// The range it could not have, because another network on this agent
|
||||
/// already holds it.
|
||||
pub range_conflict: Option<String>,
|
||||
/// Networks that switching to this network would suspend.
|
||||
#[serde(default)]
|
||||
pub range_conflicts: Vec<RangeConflictReport>,
|
||||
/// The overlay, when a protocol is running one.
|
||||
pub overlay: Option<OverlayReport>,
|
||||
/// This agent's own name, when another member claimed it first.
|
||||
@@ -693,6 +749,14 @@ impl OverlayPeerReport {
|
||||
/// running and what they can report or change. That is what keeps this module
|
||||
/// free of any knowledge of them.
|
||||
pub trait ReportSource: Send + Sync + 'static {
|
||||
/// Saves and applies a local ordered protocol set for one network.
|
||||
fn set_protocols(
|
||||
&self,
|
||||
_network_id: String,
|
||||
_protocols: Vec<String>,
|
||||
) -> BoxFuture<'_, std::result::Result<Vec<String>, String>> {
|
||||
Box::pin(async { Err("this agent cannot change protocols".into()) })
|
||||
}
|
||||
/// Produces a fresh report.
|
||||
fn report(&self) -> BoxFuture<'_, StatusReport>;
|
||||
|
||||
@@ -822,7 +886,7 @@ pub const EXCHANGE_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
///
|
||||
/// Bump it whenever [`Request`], [`Response`] or anything they contain
|
||||
/// changes shape.
|
||||
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 18]);
|
||||
pub const CONTROL_PROTOCOL: u32 = u32::from_be_bytes([b'T', b'S', b'N', 20]);
|
||||
|
||||
/// Reads one request off an accepted stream, answers it, writes the response.
|
||||
///
|
||||
@@ -846,6 +910,13 @@ where
|
||||
}
|
||||
};
|
||||
let response = match request {
|
||||
Request::SetProtocols {
|
||||
network_id,
|
||||
protocols,
|
||||
} => match source.set_protocols(network_id, protocols).await {
|
||||
Ok(protocols) => Response::Protocols(protocols),
|
||||
Err(reason) => Response::Error(reason),
|
||||
},
|
||||
Request::Status => Response::Status(Box::new(source.report().await)),
|
||||
Request::SetBroadcast {
|
||||
network_id,
|
||||
@@ -1001,7 +1072,24 @@ pub async fn join_network_with_broadcast(
|
||||
}
|
||||
}
|
||||
|
||||
/// Changes one network's broadcast participation while the agent runs.
|
||||
/// Replaces the enabled data-plane protocols of one configured network.
|
||||
pub async fn set_protocols(
|
||||
socket: &Path,
|
||||
network_id: &str,
|
||||
protocols: Vec<String>,
|
||||
) -> Result<Vec<String>> {
|
||||
let request = Request::SetProtocols {
|
||||
network_id: network_id.to_owned(),
|
||||
protocols,
|
||||
};
|
||||
match exchange(socket, &request, EXCHANGE_TIMEOUT).await? {
|
||||
Response::Protocols(protocols) => Ok(protocols),
|
||||
Response::Error(reason) => Err(Error::Storage(reason)),
|
||||
other => Err(Error::Storage(format!("unexpected response: {other:?}"))),
|
||||
}
|
||||
}
|
||||
|
||||
/// Changes local broadcast participation in a network.
|
||||
pub async fn set_broadcast(
|
||||
path: impl AsRef<Path>,
|
||||
network_id: &str,
|
||||
|
||||
@@ -72,7 +72,7 @@ pub enum RouteError {
|
||||
#[error(
|
||||
"network {network} uses {range}, which overlaps {other_range} already in use by \
|
||||
{other}. One agent has one interface, so an address can belong to only one of \
|
||||
them: give one network a different range."
|
||||
them: stop the other network before starting this one."
|
||||
)]
|
||||
Overlap {
|
||||
/// The network being added.
|
||||
@@ -365,6 +365,43 @@ impl RoutingTable {
|
||||
})
|
||||
}
|
||||
|
||||
/// All active networks whose reserved ranges intersect this range.
|
||||
pub fn overlapping_networks(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
range: Ipv4Range,
|
||||
) -> Vec<(NetworkId, Ipv4Range)> {
|
||||
let mut conflicts: Vec<_> = self
|
||||
.read()
|
||||
.iter()
|
||||
.filter_map(|(other, routes)| {
|
||||
let other_range = routes.range?;
|
||||
(*other != network && ranges_overlap(range, other_range))
|
||||
.then_some((*other, other_range))
|
||||
})
|
||||
.collect();
|
||||
conflicts.sort_by_key(|(id, _)| *id);
|
||||
conflicts
|
||||
}
|
||||
|
||||
/// Current signed allocations used for encrypted data delivery checks.
|
||||
pub(crate) fn probe_addresses(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
peer: EndpointId,
|
||||
) -> Option<(Ipv4Addr, Ipv4Addr)> {
|
||||
let table = self.read();
|
||||
let routes = table.get(&network)?;
|
||||
Some((
|
||||
routes.local?,
|
||||
routes
|
||||
.peers
|
||||
.iter()
|
||||
.find(|(_, holder)| *holder == peer)
|
||||
.map(|(address, _)| *address)?,
|
||||
))
|
||||
}
|
||||
|
||||
/// How many networks the table covers.
|
||||
pub fn len(&self) -> usize {
|
||||
self.read().len()
|
||||
@@ -510,7 +547,7 @@ mod tests {
|
||||
let err = table.set_network(second, routes()).unwrap_err();
|
||||
let RouteError::Overlap { other, .. } = err;
|
||||
assert_eq!(other, first);
|
||||
assert!(err.to_string().contains("different range"), "{err}");
|
||||
assert!(err.to_string().contains("stop the other network"), "{err}");
|
||||
assert_eq!(table.len(), 1, "the overlapping network is not added");
|
||||
|
||||
// A range of its own is fine.
|
||||
|
||||
@@ -20,7 +20,7 @@ use crate::error::ProtocolError;
|
||||
/// The version in the ALPN is the wire-compatibility version of the control
|
||||
/// protocol. It is independent of the network identity scheme version, so
|
||||
/// bumping it must not change any existing [`crate::NetworkId`].
|
||||
pub const ALPN: &[u8] = b"tsunagi/ctrl/5";
|
||||
pub const ALPN: &[u8] = b"tsunagi/ctrl/6";
|
||||
|
||||
/// ALPN of the tsunagi data plane.
|
||||
///
|
||||
@@ -43,7 +43,9 @@ pub const MAX_DATA_PROTOCOL_LEN: usize = 32;
|
||||
pub const MAX_SIGNATURE_LEN: usize = 64;
|
||||
|
||||
/// Control protocol version carried inside the handshake.
|
||||
pub const PROTOCOL_VERSION: u16 = 5;
|
||||
pub const PROTOCOL_VERSION: u16 = 6;
|
||||
|
||||
pub use crate::dataplane::transport::{MAX_TRANSPORT_PAYLOAD, TransportPayload};
|
||||
|
||||
/// First message of the handshake, sent by the initiator.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
@@ -114,6 +116,13 @@ pub struct DataOpenAck {
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[non_exhaustive]
|
||||
pub enum ControlMessage {
|
||||
/// Private transport negotiation; never persisted or rebroadcast.
|
||||
Transport {
|
||||
/// Transport protocol id; the payload stays opaque to the core.
|
||||
protocol: String,
|
||||
/// Secret negotiation data, redacted in diagnostics.
|
||||
payload: TransportPayload,
|
||||
},
|
||||
/// Hostname and capability announcement.
|
||||
Announce(Announcement),
|
||||
/// A small request used to verify that the exchange works.
|
||||
@@ -264,6 +273,17 @@ pub fn validate_capability(
|
||||
/// network, the other networks or the agent.
|
||||
pub fn validate(message: &ControlMessage, limits: &Limits) -> Result<(), ProtocolError> {
|
||||
match message {
|
||||
ControlMessage::Transport { protocol, payload } => {
|
||||
if protocol.is_empty() {
|
||||
return Err(ProtocolError::Malformed("empty transport protocol"));
|
||||
}
|
||||
check_len("transport.protocol", protocol.len(), MAX_PROTOCOL_ID_LEN)?;
|
||||
check_len(
|
||||
"transport.payload",
|
||||
payload.as_bytes().len(),
|
||||
limits.max_transport_payload_len.min(MAX_TRANSPORT_PAYLOAD),
|
||||
)?;
|
||||
}
|
||||
ControlMessage::Peers { peers } => {
|
||||
check_len("peers", peers.len(), limits.max_state_records)?;
|
||||
for hint in peers {
|
||||
@@ -320,6 +340,7 @@ pub fn validate(message: &ControlMessage, limits: &Limits) -> Result<(), Protoco
|
||||
/// A short, stable label for a message kind, for metrics and diagnostics.
|
||||
pub fn kind(message: &ControlMessage) -> &'static str {
|
||||
match message {
|
||||
ControlMessage::Transport { .. } => "transport",
|
||||
ControlMessage::Announce(_) => "announce",
|
||||
ControlMessage::Ping { .. } => "ping",
|
||||
ControlMessage::Pong { .. } => "pong",
|
||||
|
||||
@@ -109,12 +109,8 @@ impl std::str::FromStr for Ipv4Range {
|
||||
}
|
||||
}
|
||||
|
||||
/// The IPv4 overlay range used unless something else is configured or agreed.
|
||||
///
|
||||
/// A small, specific `/24`: memorable, and far less likely to overlap a
|
||||
/// network the machine is already on than taking a whole `/8` or `/10` would
|
||||
/// be. Because addresses are allocated rather than derived, 254 of them is
|
||||
/// plenty for the size of network this is for.
|
||||
/// The legacy default range, retained for existing networks and explicit use.
|
||||
/// New networks use [`derived_ipv4_range`] by default.
|
||||
pub const DEFAULT_IPV4_RANGE: Ipv4Range = Ipv4Range {
|
||||
base: Ipv4Addr::new(10, 13, 37, 0),
|
||||
prefix_len: 24,
|
||||
@@ -122,15 +118,10 @@ pub const DEFAULT_IPV4_RANGE: Ipv4Range = Ipv4Range {
|
||||
|
||||
/// A range derived from a network's own identity.
|
||||
///
|
||||
/// One agent has one interface, so two of its networks cannot both use the
|
||||
/// configured range. The second needs another one — and it cannot be picked
|
||||
/// locally, because every member has to arrive at the same answer without
|
||||
/// being told. So it comes from the network id, which every member already
|
||||
/// has and nobody can choose: the same network derives the same range on
|
||||
/// every device.
|
||||
///
|
||||
/// Inside 10/8 like the default, and never equal to it, so the first
|
||||
/// network keeps what it has always had.
|
||||
/// Every member independently derives the same pseudorandom `/24` in 10/8,
|
||||
/// including isolated first members. Stored signed ranges take precedence.
|
||||
/// The derivation is kept stable for networks already using it; the legacy
|
||||
/// default is excluded. Distinct network ids can still produce collisions.
|
||||
pub fn derived_ipv4_range(network: NetworkId) -> Ipv4Range {
|
||||
let bytes = network.as_bytes();
|
||||
let second = bytes[0];
|
||||
@@ -673,15 +664,20 @@ impl StateSet {
|
||||
|
||||
/// The range the network settled on, if anybody has said.
|
||||
///
|
||||
/// When claims disagree, the one from the lowest author id wins, so every
|
||||
/// replica reaches the same answer. A participant joining later therefore
|
||||
/// adopts the range already in use instead of imposing its own.
|
||||
/// A legacy or explicit range takes precedence over the derived default,
|
||||
/// so a fresh member's default cannot renumber an established network.
|
||||
/// Other disagreements use the lowest author id, independently of arrival
|
||||
/// order. The signed record format and network identity remain unchanged.
|
||||
pub fn agreed_range(&self) -> Option<Ipv4Range> {
|
||||
let mut authors: Vec<&EndpointId> = self.records.keys().collect();
|
||||
authors.sort_by_key(|author| *author.as_bytes());
|
||||
authors
|
||||
.into_iter()
|
||||
.find_map(|author| self.records.get(author)?.body.range())
|
||||
self.records
|
||||
.iter()
|
||||
.filter_map(|(author, record)| {
|
||||
let range = record.body.range()?;
|
||||
let derived = derived_ipv4_range(NetworkId::from_bytes(record.network));
|
||||
Some(((range == derived, *author.as_bytes()), range))
|
||||
})
|
||||
.min_by_key(|(priority, _)| *priority)
|
||||
.map(|(_, range)| range)
|
||||
}
|
||||
|
||||
/// The highest version this author has published, as far as is known.
|
||||
@@ -1263,7 +1259,7 @@ mod tests {
|
||||
assert_eq!(once.base.octets()[0], 10, "inside 10/8 like the default");
|
||||
assert_ne!(
|
||||
once, DEFAULT_IPV4_RANGE,
|
||||
"the default belongs to whichever network asked first"
|
||||
"the legacy default remains excluded"
|
||||
);
|
||||
|
||||
// A different network derives a different range, which is the
|
||||
@@ -1275,4 +1271,34 @@ mod tests {
|
||||
.network_id();
|
||||
assert_ne!(once, derived_ipv4_range(other));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_derived_default_cannot_displace_a_legacy_or_explicit_range() {
|
||||
let id = network("range-migration");
|
||||
let a = SecretKey::generate();
|
||||
let b = SecretKey::generate();
|
||||
let (low, high) = if a.public().as_bytes() < b.public().as_bytes() {
|
||||
(a, b)
|
||||
} else {
|
||||
(b, a)
|
||||
};
|
||||
for established in [DEFAULT_IPV4_RANGE, "10.90.0.0/16".parse().unwrap()] {
|
||||
let body = |range: Ipv4Range| RecordBody::Claim {
|
||||
address: Some(Ipv4Addr::from(u32::from(range.base) + 1)),
|
||||
range: Some(range),
|
||||
hostname: None,
|
||||
hostname_since: None,
|
||||
};
|
||||
let fresh = SignedRecord::sign(&low, id, 1, body(derived_ipv4_range(id)));
|
||||
let existing = SignedRecord::sign(&high, id, 1, body(established));
|
||||
for records in [
|
||||
vec![fresh.clone(), existing.clone()],
|
||||
vec![existing.clone(), fresh.clone()],
|
||||
] {
|
||||
let mut state = StateSet::new();
|
||||
state.merge_all(id, records);
|
||||
assert_eq!(state.agreed_range(), Some(established));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -254,6 +254,12 @@ impl Storage {
|
||||
.await
|
||||
}
|
||||
|
||||
/// Persists the ordered protocol selection off the async executor.
|
||||
pub async fn set_protocols(&self, network_id: NetworkId, protocols: Vec<String>) -> Result<()> {
|
||||
self.with_state(move |state| state.set_protocols(network_id, &protocols))
|
||||
.await
|
||||
}
|
||||
|
||||
/// Persists whether this agent offers itself as an exit node in a network.
|
||||
pub async fn set_exit_node(&self, network_id: NetworkId, enabled: bool) -> Result<()> {
|
||||
self.with_state(move |state| state.set_exit_node(network_id, enabled))
|
||||
@@ -272,6 +278,33 @@ impl Storage {
|
||||
.await
|
||||
}
|
||||
|
||||
/// Configured networks in the user's most recently selected order.
|
||||
pub async fn network_start_order(&self) -> Result<Vec<NetworkId>> {
|
||||
self.with_state(|state| {
|
||||
let configured = state.list_networks()?;
|
||||
let text = state
|
||||
.get_setting("network_start_order")?
|
||||
.unwrap_or_default();
|
||||
let mut order = Vec::new();
|
||||
for word in text.split(',') {
|
||||
if let Ok(id) = word.parse::<NetworkId>()
|
||||
&& configured.iter().any(|n| n.network_id == id)
|
||||
&& !order.contains(&id)
|
||||
{
|
||||
order.push(id);
|
||||
}
|
||||
}
|
||||
Ok(order)
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
/// Prefers this network at startup without disabling suspended networks.
|
||||
pub async fn prefer_network(&self, network: NetworkId) -> Result<()> {
|
||||
self.with_state(move |state| state.prefer_network(network))
|
||||
.await
|
||||
}
|
||||
|
||||
/// Removes a network configuration and its cached hints.
|
||||
pub async fn remove_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
self.with_state(move |state| {
|
||||
|
||||
@@ -21,7 +21,7 @@ use crate::identity::{DeviceIdentity, NetworkId, NetworkName, NetworkSecret};
|
||||
use crate::state::{RecordBody, SignedRecord};
|
||||
|
||||
/// Schema version written by this build.
|
||||
pub const SCHEMA_VERSION: i64 = 7;
|
||||
pub const SCHEMA_VERSION: i64 = 8;
|
||||
|
||||
/// Key of the stored hostname setting.
|
||||
const SETTING_HOSTNAME: &str = "hostname";
|
||||
@@ -32,6 +32,8 @@ const SETTING_HOSTNAME: &str = "hostname";
|
||||
/// and zeroizes on drop.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct StoredNetwork {
|
||||
/// Ordered local protocol override; absent inherits the agent's defaults.
|
||||
pub protocols: Option<Vec<String>>,
|
||||
/// Derived public network identifier.
|
||||
pub network_id: NetworkId,
|
||||
/// Network name.
|
||||
@@ -238,6 +240,16 @@ impl StateStore {
|
||||
)
|
||||
.map_err(|err| self.corrupt(format!("cannot migrate schema to 7: {err}")))?;
|
||||
}
|
||||
if found < 8 {
|
||||
self.conn
|
||||
.execute_batch(
|
||||
"BEGIN;
|
||||
ALTER TABLE networks ADD COLUMN protocols TEXT;
|
||||
PRAGMA user_version = 8;
|
||||
COMMIT;",
|
||||
)
|
||||
.map_err(|err| self.corrupt(format!("cannot migrate schema to 8: {err}")))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -448,12 +460,16 @@ impl StateStore {
|
||||
|
||||
/// Sets the auto-start flag of a configured network.
|
||||
pub fn set_auto_start(&self, network_id: NetworkId, auto_start: bool) -> Result<()> {
|
||||
self.conn
|
||||
let changed = self
|
||||
.conn
|
||||
.execute(
|
||||
"UPDATE networks SET auto_start = ?2 WHERE network_id = ?1",
|
||||
params![network_id.as_bytes().as_slice(), auto_start as i64],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot update network: {err}")))?;
|
||||
if changed == 0 {
|
||||
return Err(Error::NetworkUnknown(network_id));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -472,6 +488,25 @@ impl StateStore {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Saves a local ordered protocol selection without changing membership.
|
||||
pub fn set_protocols(&self, network_id: NetworkId, protocols: &[String]) -> Result<()> {
|
||||
crate::dataplane::validate_protocols(
|
||||
protocols,
|
||||
crate::config::Limits::default().max_capabilities,
|
||||
)?;
|
||||
let changed = self
|
||||
.conn
|
||||
.execute(
|
||||
"UPDATE networks SET protocols = ?2 WHERE network_id = ?1",
|
||||
params![network_id.as_bytes().as_slice(), protocols.join(",")],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot update protocols: {err}")))?;
|
||||
if changed == 0 {
|
||||
return Err(Error::NetworkUnknown(network_id));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Removes a network configuration entirely.
|
||||
pub fn remove_network(&self, network_id: NetworkId) -> Result<()> {
|
||||
self.conn
|
||||
@@ -532,7 +567,7 @@ impl StateStore {
|
||||
pub fn list_networks(&self) -> Result<Vec<StoredNetwork>> {
|
||||
let mut stmt = self
|
||||
.conn
|
||||
.prepare("SELECT network_id, name, secret, auto_start, broadcast, exit_node, exit_via FROM networks ORDER BY name")
|
||||
.prepare("SELECT network_id, name, secret, auto_start, broadcast, exit_node, exit_via, protocols FROM networks ORDER BY name")
|
||||
.map_err(|err| Error::Storage(format!("cannot list networks: {err}")))?;
|
||||
let rows = stmt
|
||||
.query_map([], |row| {
|
||||
@@ -548,19 +583,37 @@ impl StateStore {
|
||||
row.get::<_, bool>(4)?,
|
||||
row.get::<_, bool>(5)?,
|
||||
row.get::<_, Option<Vec<u8>>>(6)?,
|
||||
row.get::<_, Option<String>>(7)?,
|
||||
))
|
||||
})
|
||||
.map_err(|err| Error::Storage(format!("cannot list networks: {err}")))?;
|
||||
|
||||
let mut out = Vec::new();
|
||||
for row in rows {
|
||||
let (id, name, secret, auto_start, broadcast, exit_node, exit_via) =
|
||||
let (id, name, secret, auto_start, broadcast, exit_node, exit_via, protocols) =
|
||||
row.map_err(|err| Error::Storage(format!("cannot read network row: {err}")))?;
|
||||
let protocols = protocols
|
||||
.map(|text| {
|
||||
let max = crate::config::Limits::default().max_capabilities;
|
||||
if text.len() > max * (crate::dataplane::MAX_PROTOCOL_ID_LEN + 1) {
|
||||
return Err(self.corrupt("stored protocol selection is too long"));
|
||||
}
|
||||
let names = if text.is_empty() {
|
||||
Vec::new()
|
||||
} else {
|
||||
text.split(',').map(str::to_owned).collect()
|
||||
};
|
||||
crate::dataplane::validate_protocols(&names, max)
|
||||
.map_err(|err| self.corrupt(err))?;
|
||||
Ok(names)
|
||||
})
|
||||
.transpose()?;
|
||||
let id: [u8; 32] = id
|
||||
.as_slice()
|
||||
.try_into()
|
||||
.map_err(|_| self.corrupt("stored network id is not 32 bytes"))?;
|
||||
out.push(StoredNetwork {
|
||||
protocols,
|
||||
network_id: NetworkId::from_bytes(id),
|
||||
name: NetworkName::new(name)?,
|
||||
secret: NetworkSecret::from_bytes(secret)?,
|
||||
@@ -798,6 +851,44 @@ impl StateStore {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Atomically enables a network and puts it first in the startup order.
|
||||
pub fn prefer_network(&self, network: NetworkId) -> Result<()> {
|
||||
let configured = self.list_networks()?;
|
||||
if !configured.iter().any(|n| n.network_id == network) {
|
||||
return Err(Error::NetworkUnknown(network));
|
||||
}
|
||||
let existing = self.get_setting("network_start_order")?.unwrap_or_default();
|
||||
let mut order = vec![network];
|
||||
for word in existing.split(',') {
|
||||
if let Ok(id) = word.parse::<NetworkId>()
|
||||
&& configured.iter().any(|n| n.network_id == id)
|
||||
&& !order.contains(&id)
|
||||
{
|
||||
order.push(id);
|
||||
}
|
||||
}
|
||||
let value = order
|
||||
.iter()
|
||||
.map(ToString::to_string)
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
let transaction = self
|
||||
.conn
|
||||
.unchecked_transaction()
|
||||
.map_err(|err| Error::Storage(format!("cannot begin network preference: {err}")))?;
|
||||
transaction
|
||||
.execute(
|
||||
"UPDATE networks SET auto_start = 1 WHERE network_id = ?1",
|
||||
params![network.as_bytes().as_slice()],
|
||||
)
|
||||
.map_err(|err| Error::Storage(format!("cannot enable network: {err}")))?;
|
||||
transaction.execute("INSERT INTO settings (key, value) VALUES ('network_start_order', ?1) ON CONFLICT(key) DO UPDATE SET value = excluded.value", params![value])
|
||||
.map_err(|err| Error::Storage(format!("cannot store network preference: {err}")))?;
|
||||
transaction
|
||||
.commit()
|
||||
.map_err(|err| Error::Storage(format!("cannot commit network preference: {err}")))
|
||||
}
|
||||
|
||||
/// Reads an arbitrary setting.
|
||||
pub fn get_setting(&self, key: &str) -> Result<Option<String>> {
|
||||
self.conn
|
||||
@@ -915,6 +1006,68 @@ mod broadcast_storage_tests {
|
||||
assert!(store.peer_hostnames(id).unwrap().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn v7_migration_keeps_signed_state_and_protocol_overrides_are_local_and_durable() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let path = dir.path().join("state.sqlite");
|
||||
let store = StateStore::open(&path).unwrap();
|
||||
let identity = store.load_or_create_device_identity().unwrap();
|
||||
let name = NetworkName::new("protocol-migration").unwrap();
|
||||
let secret = NetworkSecret::from_bytes([4; 32]).unwrap();
|
||||
let id = NetworkKeys::derive(&name, &secret).network_id();
|
||||
store.upsert_network(id, &name, &secret, true).unwrap();
|
||||
store.set_broadcast(id, false).unwrap();
|
||||
let record = SignedRecord::sign(&identity.signing_key(), id, 17, RecordBody::Release);
|
||||
store.publish_own_record(&record).unwrap();
|
||||
store
|
||||
.conn
|
||||
.execute_batch("ALTER TABLE networks DROP COLUMN protocols; PRAGMA user_version=7;")
|
||||
.unwrap();
|
||||
drop(store);
|
||||
let store = StateStore::open(&path).unwrap();
|
||||
assert_eq!(
|
||||
store
|
||||
.load_or_create_device_identity()
|
||||
.unwrap()
|
||||
.endpoint_id(),
|
||||
identity.endpoint_id()
|
||||
);
|
||||
assert_eq!(store.signed_records(id).unwrap(), vec![record]);
|
||||
let saved = store.list_networks().unwrap();
|
||||
assert!(saved[0].protocols.is_none());
|
||||
assert!(!saved[0].broadcast);
|
||||
assert!(saved[0].auto_start);
|
||||
assert_eq!(saved[0].secret, secret);
|
||||
let chosen = vec!["awg".into(), "wg-quic".into()];
|
||||
store.set_protocols(id, &chosen).unwrap();
|
||||
assert!(
|
||||
store
|
||||
.set_protocols(id, &["awg".into(), "awg".into()])
|
||||
.is_err()
|
||||
);
|
||||
assert!(
|
||||
store
|
||||
.set_protocols(NetworkId::from_bytes([1; 32]), &[])
|
||||
.is_err()
|
||||
);
|
||||
drop(store);
|
||||
let store = StateStore::open(&path).unwrap();
|
||||
assert_eq!(store.list_networks().unwrap()[0].protocols, Some(chosen));
|
||||
store.upsert_network(id, &name, &secret, true).unwrap();
|
||||
store.set_protocols(id, &[]).unwrap();
|
||||
drop(store);
|
||||
let store = StateStore::open(&path).unwrap();
|
||||
assert_eq!(store.list_networks().unwrap()[0].protocols, Some(vec![]));
|
||||
store
|
||||
.conn
|
||||
.execute("UPDATE networks SET protocols = 'awg,awg'", [])
|
||||
.unwrap();
|
||||
assert!(matches!(
|
||||
store.list_networks(),
|
||||
Err(Error::StateCorrupted { .. })
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn v3_migration_preserves_identity_and_networks_and_defaults_broadcast_on() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
@@ -935,6 +1088,7 @@ mod broadcast_storage_tests {
|
||||
"ALTER TABLE networks DROP COLUMN broadcast; \
|
||||
ALTER TABLE networks DROP COLUMN exit_node; \
|
||||
ALTER TABLE networks DROP COLUMN exit_via; \
|
||||
ALTER TABLE networks DROP COLUMN protocols; \
|
||||
PRAGMA user_version=3;",
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
@@ -162,7 +162,7 @@ async fn rejoining_after_leaving_is_not_taken_for_a_stale_record() {
|
||||
})
|
||||
.await;
|
||||
assert!(
|
||||
tsunagi::state::DEFAULT_IPV4_RANGE.contains(again),
|
||||
tsunagi::state::derived_ipv4_range(network_id).contains(again),
|
||||
"an address in the network's range: {again}"
|
||||
);
|
||||
|
||||
|
||||
@@ -265,10 +265,7 @@ async fn deactivating_one_network_leaves_the_agent_and_others_running() {
|
||||
|
||||
#[tokio::test]
|
||||
async fn two_networks_on_one_agent_each_get_a_range_of_their_own() {
|
||||
// One agent, one interface: the configured range can only belong to one
|
||||
// of them. The second does not go without — it takes the range derived
|
||||
// from its own id, which every one of its members derives identically,
|
||||
// so it is an agreement and not a local invention.
|
||||
// Each network derives its range independently of activation order.
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (first, first_secret) = network("two-ranges-one");
|
||||
let (second, second_secret) = network("two-ranges-two");
|
||||
@@ -309,8 +306,8 @@ async fn two_networks_on_one_agent_each_get_a_range_of_their_own() {
|
||||
"different networks, different addresses"
|
||||
);
|
||||
assert!(
|
||||
tsunagi::state::DEFAULT_IPV4_RANGE.contains(addresses.0),
|
||||
"the first keeps the configured range: {}",
|
||||
tsunagi::state::derived_ipv4_range(one).contains(addresses.0),
|
||||
"the first uses its derived range: {}",
|
||||
addresses.0
|
||||
);
|
||||
assert!(
|
||||
@@ -319,7 +316,7 @@ async fn two_networks_on_one_agent_each_get_a_range_of_their_own() {
|
||||
addresses.1
|
||||
);
|
||||
assert!(
|
||||
!tsunagi::state::DEFAULT_IPV4_RANGE.contains(addresses.1),
|
||||
!tsunagi::state::derived_ipv4_range(one).contains(addresses.1),
|
||||
"and not in the one the first network holds"
|
||||
);
|
||||
|
||||
|
||||
@@ -0,0 +1,336 @@
|
||||
//! Derived defaults and local exclusion of overlapping overlay networks.
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use tsunagi::Agent;
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::state::{DEFAULT_IPV4_RANGE, Ipv4Range, derived_ipv4_range};
|
||||
use tsunagi::testing::{TestAgent, config_with, network, wait_until};
|
||||
|
||||
// These tests need discovery to happen before a collision, so keep the initial
|
||||
// default outside the established ranges. Derived collisions have their own test.
|
||||
fn network_outside(
|
||||
name: &str,
|
||||
ranges: &[Ipv4Range],
|
||||
) -> (
|
||||
tsunagi::identity::NetworkName,
|
||||
tsunagi::identity::NetworkSecret,
|
||||
) {
|
||||
use tsunagi::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
let name = NetworkName::new(name).unwrap();
|
||||
let secret = (0..=255u8)
|
||||
.map(|seed| NetworkSecret::from_bytes([seed; 32]).unwrap())
|
||||
.find(|secret| {
|
||||
let derived = derived_ipv4_range(NetworkKeys::derive(&name, secret).network_id());
|
||||
ranges.iter().all(|range| !range.contains(derived.base))
|
||||
})
|
||||
.expect("a deterministic test secret with a non-overlapping default");
|
||||
(name, secret)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn colliding_derived_defaults_are_blocked_instead_of_locally_renumbered() {
|
||||
use std::collections::HashMap;
|
||||
use tsunagi::identity::{NetworkKeys, NetworkName, NetworkSecret};
|
||||
let secret = NetworkSecret::from_bytes([19; 32]).unwrap();
|
||||
let mut seen = HashMap::new();
|
||||
let (first, second, range) = (0..=65_536)
|
||||
.find_map(|index| {
|
||||
let name = NetworkName::new(format!("derived-collision-{index}")).unwrap();
|
||||
let range = derived_ipv4_range(NetworkKeys::derive(&name, &secret).network_id());
|
||||
seen.insert(range.base, name.clone())
|
||||
.map(|first| (first, name, range))
|
||||
})
|
||||
.expect("more network names than possible /24 ranges");
|
||||
let agent = TestAgent::spawn(&SharedMemoryDiscovery::new())
|
||||
.await
|
||||
.unwrap();
|
||||
let first = agent.agent.join_network(&first, &secret).await.unwrap();
|
||||
let second = agent.agent.join_network(&second, &secret).await.unwrap();
|
||||
assert!(agent.agent.is_active(first).await);
|
||||
assert!(!agent.agent.is_active(second).await);
|
||||
let status = agent.agent.status().await.unwrap();
|
||||
let blocked = status.network(&second).unwrap();
|
||||
assert_eq!(blocked.range, Some(range));
|
||||
assert_eq!(blocked.range_conflicts[0].network_id, first);
|
||||
assert_eq!(
|
||||
agent.agent.switch_network(second).await.unwrap(),
|
||||
vec![first]
|
||||
);
|
||||
assert_eq!(
|
||||
agent.agent.network_status(second).await.unwrap().range,
|
||||
Some(range)
|
||||
);
|
||||
agent.agent.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn isolated_first_members_choose_the_same_range_and_keep_it_after_restart() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let (name, secret) = network("derived-first");
|
||||
let a = TestAgent::spawn(&discovery).await.unwrap();
|
||||
let b = TestAgent::spawn(&SharedMemoryDiscovery::new())
|
||||
.await
|
||||
.unwrap();
|
||||
let id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
let expected = derived_ipv4_range(id);
|
||||
for agent in [&a.agent, &b.agent] {
|
||||
wait_until("isolated member allocates in its derived range", || async {
|
||||
let status = agent.network_status(id).await.ok()?;
|
||||
(status.range == Some(expected) && !status.members.is_empty()).then_some(())
|
||||
})
|
||||
.await;
|
||||
}
|
||||
let dir = a.stop().await;
|
||||
let a = Agent::spawn(config_with(dir.path(), &discovery))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(a.network_status(id).await.unwrap().range, Some(expected));
|
||||
a.shutdown().await;
|
||||
b.agent.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn overlap_blocks_activation_switches_and_resumes_across_restart() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let a = TestAgent::spawn_with(
|
||||
|c| c.with_overlay_ipv4_range(Some(DEFAULT_IPV4_RANGE)),
|
||||
&discovery,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let (one_name, one_secret) = network("conflict-one");
|
||||
let (two_name, two_secret) = network("conflict-two");
|
||||
let one = a.agent.join_network(&one_name, &one_secret).await.unwrap();
|
||||
a.agent.network_status(one).await.unwrap();
|
||||
let two = a.agent.join_network(&two_name, &two_secret).await.unwrap();
|
||||
assert!(a.agent.is_active(one).await);
|
||||
assert!(!a.agent.is_active(two).await);
|
||||
assert!(matches!(
|
||||
a.agent.activate_network(two).await,
|
||||
Err(tsunagi::Error::NetworkRangeConflict { .. })
|
||||
));
|
||||
let status = a.agent.status().await.unwrap();
|
||||
let blocked = status.network(&two).unwrap();
|
||||
assert_eq!(blocked.range, Some(DEFAULT_IPV4_RANGE));
|
||||
assert_eq!(blocked.range_conflicts[0].network_id, one);
|
||||
assert_eq!(blocked.range_conflicts[0].name, one_name);
|
||||
assert_eq!(a.agent.switch_network(two).await.unwrap(), vec![one]);
|
||||
assert!(a.agent.is_active(two).await);
|
||||
assert!(!a.agent.is_active(one).await);
|
||||
|
||||
a.agent.network_status(two).await.unwrap();
|
||||
let dir = a.stop().await;
|
||||
let a = Agent::spawn(config_with(dir.path(), &discovery))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(a.is_active(two).await, "the user's choice survives restart");
|
||||
assert!(!a.is_active(one).await);
|
||||
a.deactivate_network(two).await.unwrap();
|
||||
assert!(
|
||||
a.is_active(one).await,
|
||||
"suspended networks resume when the blocker stops"
|
||||
);
|
||||
assert_eq!(
|
||||
a.network_status(one).await.unwrap().range,
|
||||
Some(DEFAULT_IPV4_RANGE),
|
||||
"legacy ranges survive the new default"
|
||||
);
|
||||
a.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn late_authenticated_state_with_a_nested_range_suspends_only_that_network() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let broad: Ipv4Range = "10.90.0.0/16".parse().unwrap();
|
||||
let narrow: Ipv4Range = "10.90.12.0/24".parse().unwrap();
|
||||
let (name, secret) = network_outside("late-conflict", &[broad]);
|
||||
let remote = TestAgent::spawn_with(|c| c.with_overlay_ipv4_range(Some(narrow)), &discovery)
|
||||
.await
|
||||
.unwrap();
|
||||
let target = remote.agent.join_network(&name, &secret).await.unwrap();
|
||||
wait_until("remote range is persisted", || async {
|
||||
remote
|
||||
.agent
|
||||
.network_status(target)
|
||||
.await
|
||||
.ok()?
|
||||
.members
|
||||
.first()
|
||||
.map(|_| ())
|
||||
})
|
||||
.await;
|
||||
|
||||
let (other_name, other_secret) = network("existing-broad");
|
||||
// Seed an existing network using a real agent and its signed claim.
|
||||
let seeded = TestAgent::spawn_with(
|
||||
|c| c.with_overlay_ipv4_range(Some(broad)),
|
||||
&SharedMemoryDiscovery::new(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let other = seeded
|
||||
.agent
|
||||
.join_network(&other_name, &other_secret)
|
||||
.await
|
||||
.unwrap();
|
||||
wait_until("broad range is persisted", || async {
|
||||
seeded
|
||||
.agent
|
||||
.network_status(other)
|
||||
.await
|
||||
.ok()?
|
||||
.members
|
||||
.first()
|
||||
.map(|_| ())
|
||||
})
|
||||
.await;
|
||||
let dir = seeded.stop().await;
|
||||
let seeded = Agent::spawn(config_with(dir.path(), &discovery))
|
||||
.await
|
||||
.unwrap();
|
||||
seeded.join_network(&name, &secret).await.unwrap();
|
||||
wait_until("late overlap suspends the joining network", || async {
|
||||
let status = seeded.status().await.ok()?;
|
||||
let blocked = status.network(&target)?;
|
||||
(!seeded.is_active(target).await
|
||||
&& blocked
|
||||
.range_conflicts
|
||||
.iter()
|
||||
.any(|c| c.network_id == other))
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
assert!(seeded.is_active(other).await);
|
||||
assert_eq!(seeded.switch_network(target).await.unwrap(), vec![other]);
|
||||
assert_eq!(
|
||||
seeded.network_status(target).await.unwrap().range,
|
||||
Some(narrow)
|
||||
);
|
||||
seeded.shutdown().await;
|
||||
remote.agent.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_wide_range_suspends_all_overlaps_and_manual_stop_cancels_resume() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let hub = TestAgent::spawn(&discovery).await.unwrap();
|
||||
let mut members = Vec::new();
|
||||
let mut ids = Vec::new();
|
||||
for (name, range) in [
|
||||
("narrow-one", "10.81.0.0/24"),
|
||||
("narrow-two", "10.81.1.0/24"),
|
||||
] {
|
||||
let peer = TestAgent::spawn_with(
|
||||
|c| c.with_overlay_ipv4_range(Some(range.parse().unwrap())),
|
||||
&discovery,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let (name, secret) = network_outside(
|
||||
name,
|
||||
&[
|
||||
"10.81.0.0/24".parse().unwrap(),
|
||||
"10.81.1.0/24".parse().unwrap(),
|
||||
],
|
||||
);
|
||||
let id = peer.agent.join_network(&name, &secret).await.unwrap();
|
||||
peer.agent.network_status(id).await.unwrap();
|
||||
hub.agent.join_network(&name, &secret).await.unwrap();
|
||||
wait_until("hub adopts the established narrow range", || async {
|
||||
(hub.agent.network_status(id).await.ok()?.range == Some(range.parse().unwrap()))
|
||||
.then_some(())
|
||||
})
|
||||
.await;
|
||||
members.push(peer);
|
||||
ids.push(id);
|
||||
}
|
||||
let (name, secret) = network_outside(
|
||||
"wide-overlap",
|
||||
&[
|
||||
"10.81.0.0/24".parse().unwrap(),
|
||||
"10.81.1.0/24".parse().unwrap(),
|
||||
],
|
||||
);
|
||||
let wide = TestAgent::spawn_with(
|
||||
|c| c.with_overlay_ipv4_range(Some("10.0.0.0/8".parse().unwrap())),
|
||||
&discovery,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let id = wide.agent.join_network(&name, &secret).await.unwrap();
|
||||
wide.agent.network_status(id).await.unwrap();
|
||||
hub.agent.join_network(&name, &secret).await.unwrap();
|
||||
wait_until("wide range is blocked by both narrow networks", || async {
|
||||
let status = hub.agent.status().await.ok()?;
|
||||
let blocked = status.network(&id)?;
|
||||
(!hub.agent.is_active(id).await && blocked.range_conflicts.len() == 2).then_some(())
|
||||
})
|
||||
.await;
|
||||
let stopped = hub.agent.switch_network(id).await.unwrap();
|
||||
assert_eq!(stopped.len(), 2);
|
||||
assert!(ids.iter().all(|id| stopped.contains(id)));
|
||||
assert_eq!(hub.agent.switch_network(ids[0]).await.unwrap(), vec![id]);
|
||||
assert!(
|
||||
hub.agent.is_active(ids[1]).await,
|
||||
"switching away from the wide range frees the other narrow range too"
|
||||
);
|
||||
assert_eq!(hub.agent.switch_network(id).await.unwrap().len(), 2);
|
||||
hub.agent.pause_network(ids[0]).await.unwrap();
|
||||
hub.agent.deactivate_network(id).await.unwrap();
|
||||
assert!(
|
||||
!hub.agent.is_active(ids[0]).await,
|
||||
"explicitly stopped networks do not resume"
|
||||
);
|
||||
assert!(hub.agent.is_active(ids[1]).await);
|
||||
assert!(
|
||||
hub.agent
|
||||
.switch_network(tsunagi::NetworkId::from_bytes([0; 32]))
|
||||
.await
|
||||
.is_err()
|
||||
);
|
||||
assert!(
|
||||
hub.agent.is_active(ids[1]).await,
|
||||
"unknown selectors do not stop any network"
|
||||
);
|
||||
hub.agent.shutdown().await;
|
||||
wide.agent.shutdown().await;
|
||||
for peer in members {
|
||||
peer.agent.shutdown().await;
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn simultaneous_switches_never_leave_overlapping_networks_active() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let agent = TestAgent::spawn_with(
|
||||
|c| c.with_overlay_ipv4_range(Some(DEFAULT_IPV4_RANGE)),
|
||||
&discovery,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let (a, a_secret) = network("switch-a");
|
||||
let (b, b_secret) = network("switch-b");
|
||||
let a = agent.agent.join_network(&a, &a_secret).await.unwrap();
|
||||
agent.agent.network_status(a).await.unwrap();
|
||||
let b = agent.agent.join_network(&b, &b_secret).await.unwrap();
|
||||
let (left, right) = tokio::join!(agent.agent.switch_network(b), agent.agent.switch_network(a));
|
||||
left.unwrap();
|
||||
right.unwrap();
|
||||
assert_ne!(
|
||||
agent.agent.is_active(a).await,
|
||||
agent.agent.is_active(b).await
|
||||
);
|
||||
assert_eq!(
|
||||
agent
|
||||
.agent
|
||||
.list_networks()
|
||||
.await
|
||||
.unwrap()
|
||||
.iter()
|
||||
.filter(|n| n.active)
|
||||
.count(),
|
||||
1
|
||||
);
|
||||
agent.agent.shutdown().await;
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
//! Opaque transport negotiation stays inside authenticated network sessions.
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use std::sync::{Arc, Mutex};
|
||||
use tsunagi::dataplane::transport::{ControlSender, PacketTransport, SharedLink, TransportError};
|
||||
use tsunagi::discovery::SharedMemoryDiscovery;
|
||||
use tsunagi::proto::message::{ControlMessage, TransportPayload};
|
||||
use tsunagi::testing::{TestAgent, network, wait_for_peers, wait_until};
|
||||
use tsunagi::{BoxFuture, NetworkId};
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct Negotiator {
|
||||
control: Mutex<Option<Arc<dyn ControlSender>>>,
|
||||
received: Mutex<Vec<(NetworkId, iroh::EndpointId, Vec<u8>)>>,
|
||||
}
|
||||
impl PacketTransport for Negotiator {
|
||||
fn name(&self) -> &str {
|
||||
"negotiator"
|
||||
}
|
||||
fn as_any(&self) -> &dyn std::any::Any {
|
||||
self
|
||||
}
|
||||
fn attach_control(&self, control: Arc<dyn ControlSender>) {
|
||||
*self.control.lock().unwrap() = Some(control);
|
||||
}
|
||||
fn on_control(
|
||||
&self,
|
||||
network: NetworkId,
|
||||
peer: iroh::EndpointId,
|
||||
_: &str,
|
||||
payload: &[u8],
|
||||
) -> Result<(), TransportError> {
|
||||
self.received
|
||||
.lock()
|
||||
.unwrap()
|
||||
.push((network, peer, payload.to_vec()));
|
||||
Ok(())
|
||||
}
|
||||
fn open<'a>(
|
||||
&'a self,
|
||||
_: NetworkId,
|
||||
_: iroh::EndpointId,
|
||||
_: &'a str,
|
||||
) -> BoxFuture<'a, Result<SharedLink, TransportError>> {
|
||||
Box::pin(async { Err(TransportError::Closed) })
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn negotiated_secrets_are_redacted_scoped_and_delivered_over_real_control() {
|
||||
let discovery = SharedMemoryDiscovery::new();
|
||||
let a_transport = Arc::new(Negotiator::default());
|
||||
let b_transport = Arc::new(Negotiator::default());
|
||||
let a = TestAgent::spawn_with(
|
||||
|c| c.with_custom_transport("negotiator", a_transport.clone()),
|
||||
&discovery,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let b = TestAgent::spawn_with(
|
||||
|c| c.with_custom_transport("negotiator", b_transport.clone()),
|
||||
&discovery,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let (name, secret) = network("private-negotiation");
|
||||
let id = a.agent.join_network(&name, &secret).await.unwrap();
|
||||
b.agent.join_network(&name, &secret).await.unwrap();
|
||||
wait_for_peers(&a.agent, id, 1).await;
|
||||
let message = ControlMessage::Transport {
|
||||
protocol: "negotiator".into(),
|
||||
payload: TransportPayload::new(b"never-export-this-secret".to_vec()),
|
||||
};
|
||||
assert!(!format!("{message:?}").contains("never-export"));
|
||||
let sender = a_transport.control.lock().unwrap().clone().unwrap();
|
||||
sender
|
||||
.send(
|
||||
id,
|
||||
b.agent.endpoint_id(),
|
||||
"negotiator",
|
||||
TransportPayload::new(b"never-export-this-secret".to_vec()),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
wait_until("private transport message", || async {
|
||||
(!b_transport.received.lock().unwrap().is_empty()).then_some(())
|
||||
})
|
||||
.await;
|
||||
assert_eq!(
|
||||
b_transport.received.lock().unwrap()[0],
|
||||
(
|
||||
id,
|
||||
a.agent.endpoint_id(),
|
||||
b"never-export-this-secret".to_vec()
|
||||
)
|
||||
);
|
||||
let unknown = NetworkId::from_bytes([0xa5; 32]);
|
||||
assert!(
|
||||
sender
|
||||
.send(
|
||||
unknown,
|
||||
b.agent.endpoint_id(),
|
||||
"negotiator",
|
||||
TransportPayload::new(vec![1])
|
||||
)
|
||||
.await
|
||||
.is_err()
|
||||
);
|
||||
a.agent.shutdown().await;
|
||||
b.agent.shutdown().await;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn private_bytes_are_bounded_on_decode_and_by_local_policy() {
|
||||
let limits = tsunagi::Limits {
|
||||
max_transport_payload_len: 4,
|
||||
..Default::default()
|
||||
};
|
||||
let message = |len| ControlMessage::Transport {
|
||||
protocol: "awg".into(),
|
||||
payload: TransportPayload::new(vec![42; len]),
|
||||
};
|
||||
assert!(tsunagi::proto::message::validate(&message(4), &limits).is_ok());
|
||||
assert!(tsunagi::proto::message::validate(&message(5), &limits).is_err());
|
||||
let bytes = postcard::to_stdvec(&TransportPayload::new(vec![1; 4097])).unwrap();
|
||||
assert!(postcard::from_bytes::<TransportPayload>(&bytes).is_err());
|
||||
let bytes = postcard::to_stdvec(&message(4)).unwrap();
|
||||
let decoded: ControlMessage = postcard::from_bytes(&bytes).unwrap();
|
||||
assert_eq!(decoded, message(4));
|
||||
}
|
||||
@@ -115,6 +115,7 @@ pub struct TcpTlsLink {
|
||||
peer: EndpointId,
|
||||
tx: mpsc::Sender<Bytes>,
|
||||
rx: tokio::sync::Mutex<mpsc::Receiver<Bytes>>,
|
||||
tasks: [tokio::task::AbortHandle; 2],
|
||||
closed_notify: Arc<Notify>,
|
||||
is_closed: Arc<AtomicBool>,
|
||||
remote_addr: String,
|
||||
@@ -136,7 +137,7 @@ impl TcpTlsLink {
|
||||
|
||||
let closed_flag_w = Arc::clone(&is_closed);
|
||||
let closed_notify_w = Arc::clone(&closed_notify);
|
||||
tokio::spawn(async move {
|
||||
let writer_task = tokio::spawn(async move {
|
||||
while let Some(msg) = rx_out.recv().await {
|
||||
if msg.len() > MAX_FRAME_SIZE {
|
||||
continue;
|
||||
@@ -158,7 +159,7 @@ impl TcpTlsLink {
|
||||
|
||||
let closed_flag_r = Arc::clone(&is_closed);
|
||||
let closed_notify_r = Arc::clone(&closed_notify);
|
||||
tokio::spawn(async move {
|
||||
let reader_task = tokio::spawn(async move {
|
||||
let mut len_buf = [0u8; 2];
|
||||
loop {
|
||||
if reader.read_exact(&mut len_buf).await.is_err() {
|
||||
@@ -185,6 +186,7 @@ impl TcpTlsLink {
|
||||
peer,
|
||||
tx: tx_out,
|
||||
rx: tokio::sync::Mutex::new(rx_in),
|
||||
tasks: [writer_task.abort_handle(), reader_task.abort_handle()],
|
||||
closed_notify,
|
||||
is_closed,
|
||||
remote_addr,
|
||||
@@ -192,7 +194,19 @@ impl TcpTlsLink {
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TcpTlsLink {
|
||||
fn drop(&mut self) {
|
||||
self.close();
|
||||
}
|
||||
}
|
||||
impl PacketLink for TcpTlsLink {
|
||||
fn close(&self) {
|
||||
self.is_closed.store(true, Ordering::Release);
|
||||
self.closed_notify.notify_waiters();
|
||||
for task in &self.tasks {
|
||||
task.abort();
|
||||
}
|
||||
}
|
||||
fn network(&self) -> NetworkId {
|
||||
self.network
|
||||
}
|
||||
|
||||
@@ -21,6 +21,11 @@ const POLL_INTERVAL: Duration = Duration::from_secs(2);
|
||||
|
||||
/// A request from the UI to the agent.
|
||||
pub(crate) enum Command {
|
||||
/// Replace the network's enabled protocols in preference order.
|
||||
SetProtocols {
|
||||
network_id: String,
|
||||
protocols: Vec<String>,
|
||||
},
|
||||
/// Stop serving a network, or start serving it again.
|
||||
SetActive { network_id: String, active: bool },
|
||||
/// Turn local broadcast participation on or off for a network.
|
||||
@@ -106,7 +111,13 @@ impl AgentClient {
|
||||
|
||||
/// Sends a command; a dropped worker simply means nothing happens.
|
||||
pub(crate) fn send(&self, command: Command) {
|
||||
let _ = self.tx.send(command);
|
||||
// Disable protocol editing until the refreshed snapshot arrives;
|
||||
// otherwise a second checkbox click could replace the first edit
|
||||
// using the same stale network selection.
|
||||
lock(&self.shared).busy = true;
|
||||
if self.tx.send(command).is_err() {
|
||||
lock(&self.shared).busy = false;
|
||||
}
|
||||
}
|
||||
|
||||
/// A copy of the latest snapshot.
|
||||
@@ -142,6 +153,8 @@ async fn worker(
|
||||
let Some(command) = command else { return };
|
||||
run(&resolve_socket(), command, &shared, &pending_copy, &ctx).await;
|
||||
refresh(&shared, &ctx).await;
|
||||
lock(&shared).busy = false;
|
||||
ctx.request_repaint();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -179,11 +192,41 @@ async fn run(
|
||||
ctx.request_repaint();
|
||||
|
||||
let outcome: Result<String, String> = match command {
|
||||
Command::SetProtocols {
|
||||
network_id,
|
||||
protocols,
|
||||
} => ipc::set_protocols(socket, &network_id, protocols)
|
||||
.await
|
||||
.map(|names| {
|
||||
format!(
|
||||
"protocols: {}",
|
||||
if names.is_empty() {
|
||||
"none".into()
|
||||
} else {
|
||||
names.join(", ")
|
||||
}
|
||||
)
|
||||
})
|
||||
.map_err(|err| err.to_string()),
|
||||
Command::SetActive { network_id, active } => ipc::set_active(socket, &network_id, active)
|
||||
.await
|
||||
.map(|report| {
|
||||
let state = if report.active { "started" } else { "stopped" };
|
||||
format!("{} {}", state, report.name)
|
||||
if report.suspended.is_empty() {
|
||||
format!("{} {}", state, report.name)
|
||||
} else {
|
||||
format!(
|
||||
"{} {}; suspended {} until it stops",
|
||||
state,
|
||||
report.name,
|
||||
report
|
||||
.suspended
|
||||
.iter()
|
||||
.map(|n| format!("{} ({})", n.name, n.network_id))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ")
|
||||
)
|
||||
}
|
||||
})
|
||||
.map_err(|err| err.to_string()),
|
||||
Command::SetBroadcast {
|
||||
@@ -211,7 +254,17 @@ async fn run(
|
||||
}
|
||||
Command::Join { name, secret } => ipc::join_network(socket, &name, &secret)
|
||||
.await
|
||||
.map(|report| format!("joined {}", report.name))
|
||||
.map(|report| {
|
||||
format!(
|
||||
"{} {}",
|
||||
if report.active {
|
||||
"joined"
|
||||
} else {
|
||||
"configured; stopped by range conflict:"
|
||||
},
|
||||
report.name
|
||||
)
|
||||
})
|
||||
.map_err(|err| err.to_string()),
|
||||
Command::Leave { network_id } => ipc::leave_network(socket, &network_id)
|
||||
.await
|
||||
@@ -236,7 +289,6 @@ async fn run(
|
||||
|
||||
{
|
||||
let mut guard = lock(shared);
|
||||
guard.busy = false;
|
||||
guard.last_action = if matches!(&outcome, Ok(message) if message.is_empty()) {
|
||||
None
|
||||
} else {
|
||||
|
||||
@@ -78,6 +78,7 @@ fn colour_of(health: Health) -> [u8; 3] {
|
||||
match health {
|
||||
Health::Disconnected => DARK_RED,
|
||||
Health::Connected => BLUE,
|
||||
Health::RangeConflict => [0xd9, 0x91, 0x20],
|
||||
Health::ExitNode => GREEN,
|
||||
}
|
||||
}
|
||||
|
||||
+106
-10
@@ -123,7 +123,7 @@ pub(crate) fn draw(
|
||||
ui.weak("No networks yet — join one below.");
|
||||
}
|
||||
for network in &report.networks {
|
||||
draw_network(ui, agent, state, traffic, network, &report.endpoint_id);
|
||||
draw_network(ui, agent, state, traffic, network, report);
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -202,7 +202,7 @@ fn draw_network(
|
||||
state: &mut UiState,
|
||||
traffic: &Traffic,
|
||||
network: &NetworkReport,
|
||||
own_id: &str,
|
||||
report: &StatusReport,
|
||||
) {
|
||||
ui.add_space(6.0);
|
||||
egui::Frame::group(ui.style()).show(ui, |ui| {
|
||||
@@ -244,21 +244,40 @@ fn draw_network(
|
||||
ui.weak("peers");
|
||||
ui.label(format!(
|
||||
"{} online · {} known",
|
||||
devices::online_count(network, own_id),
|
||||
devices::known_count(network, own_id)
|
||||
devices::online_count(network, &report.endpoint_id),
|
||||
devices::known_count(network, &report.endpoint_id)
|
||||
));
|
||||
ui.end_row();
|
||||
});
|
||||
|
||||
draw_protocols(ui, agent, network, &report.available_protocols);
|
||||
|
||||
format::sparkline(ui, series, state.unit, 34.0);
|
||||
|
||||
if !network.range_conflicts.is_empty() {
|
||||
ui.colored_label(BAD, "Stopped: overlapping network ranges");
|
||||
for conflict in &network.range_conflicts {
|
||||
ui.label(format!("{} ({}) uses {}", conflict.name, conflict.network_id, conflict.range));
|
||||
}
|
||||
ui.label("Switching suspends these networks until this network stops, then resumes them automatically.");
|
||||
if network.auto_start { ui.weak("This network will resume automatically when its range is free."); }
|
||||
}
|
||||
ui.horizontal(|ui| {
|
||||
let mut active = network.active;
|
||||
if ui.checkbox(&mut active, "active").changed() {
|
||||
agent.send(Command::SetActive {
|
||||
network_id: network.network_id.clone(),
|
||||
active,
|
||||
});
|
||||
if !network.active && !network.range_conflicts.is_empty() {
|
||||
if ui.button("Switch to this network").clicked() {
|
||||
agent.send(Command::SetActive { network_id: network.network_id.clone(), active: true });
|
||||
}
|
||||
if network.auto_start && ui.button("Keep stopped").clicked() {
|
||||
agent.send(Command::SetActive { network_id: network.network_id.clone(), active: false });
|
||||
}
|
||||
} else {
|
||||
let mut active = network.active;
|
||||
if ui.checkbox(&mut active, "active").changed() {
|
||||
agent.send(Command::SetActive {
|
||||
network_id: network.network_id.clone(),
|
||||
active,
|
||||
});
|
||||
}
|
||||
}
|
||||
let mut broadcast = network.broadcast;
|
||||
if ui.checkbox(&mut broadcast, "broadcast").changed() {
|
||||
@@ -328,6 +347,83 @@ fn draw_network(
|
||||
});
|
||||
}
|
||||
|
||||
fn draw_protocols(
|
||||
ui: &mut egui::Ui,
|
||||
agent: &AgentClient,
|
||||
network: &NetworkReport,
|
||||
available: &[tsunagi::ipc::ProtocolReport],
|
||||
) {
|
||||
let summary = if network.protocols.is_empty() {
|
||||
"none".into()
|
||||
} else {
|
||||
network.protocols.join(", ")
|
||||
};
|
||||
egui::CollapsingHeader::new(format!("protocols {summary}"))
|
||||
.id_salt(("protocols", &network.network_id))
|
||||
.show(ui, |ui| {
|
||||
ui.weak("Preference order; changes apply immediately and are saved.");
|
||||
let mut selected = network.protocols.clone();
|
||||
let mut changed = false;
|
||||
ui.add_enabled_ui(!agent.snapshot().busy, |ui| {
|
||||
// Enabled items follow the user's preference; unchecked items
|
||||
// follow the agent's catalog. The worker refreshes after edits.
|
||||
let ordered: Vec<_> = network
|
||||
.protocols
|
||||
.iter()
|
||||
.filter_map(|name| available.iter().find(|p| &p.name == name))
|
||||
.chain(
|
||||
available
|
||||
.iter()
|
||||
.filter(|p| !network.protocols.contains(&p.name)),
|
||||
)
|
||||
.collect();
|
||||
for protocol in ordered {
|
||||
ui.horizontal(|ui| {
|
||||
let mut on = selected.contains(&protocol.name);
|
||||
if ui.checkbox(&mut on, &protocol.name).changed() {
|
||||
if on {
|
||||
selected.push(protocol.name.clone());
|
||||
} else {
|
||||
selected.retain(|p| p != &protocol.name);
|
||||
}
|
||||
changed = true;
|
||||
}
|
||||
if let Some(index) = selected.iter().position(|p| p == &protocol.name) {
|
||||
if ui
|
||||
.add_enabled(index > 0, egui::Button::new("↑"))
|
||||
.on_hover_text("prefer earlier")
|
||||
.clicked()
|
||||
{
|
||||
selected.swap(index, index - 1);
|
||||
changed = true;
|
||||
}
|
||||
if ui
|
||||
.add_enabled(index + 1 < selected.len(), egui::Button::new("↓"))
|
||||
.on_hover_text("prefer later")
|
||||
.clicked()
|
||||
{
|
||||
selected.swap(index, index + 1);
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
});
|
||||
ui.indent(("description", &network.network_id, &protocol.name), |ui| {
|
||||
ui.weak(&protocol.summary);
|
||||
});
|
||||
}
|
||||
});
|
||||
if selected.is_empty() {
|
||||
ui.weak("Control only; no IP traffic in this network.");
|
||||
}
|
||||
if changed {
|
||||
agent.send(Command::SetProtocols {
|
||||
network_id: network.network_id.clone(),
|
||||
protocols: selected,
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/// A prominent line about an exit node. One that carries a command copies it
|
||||
/// when clicked.
|
||||
fn draw_exit_warning(ui: &mut egui::Ui, warning: &exit::Warning) {
|
||||
|
||||
@@ -22,6 +22,8 @@ pub(crate) enum Health {
|
||||
Disconnected,
|
||||
/// Connected, nothing out of the ordinary.
|
||||
Connected,
|
||||
/// At least one configured network is blocked by an overlapping active range.
|
||||
RangeConflict,
|
||||
/// This device's traffic leaves through an exit node, and that works.
|
||||
ExitNode,
|
||||
}
|
||||
@@ -33,6 +35,13 @@ impl Health {
|
||||
let Some(report) = report else {
|
||||
return Self::Disconnected;
|
||||
};
|
||||
if report
|
||||
.networks
|
||||
.iter()
|
||||
.any(|network| network.auto_start && !network.range_conflicts.is_empty())
|
||||
{
|
||||
return Self::RangeConflict;
|
||||
}
|
||||
if !report
|
||||
.networks
|
||||
.iter()
|
||||
@@ -56,6 +65,7 @@ impl Health {
|
||||
match self {
|
||||
Self::Disconnected => "tsunagi: not connected",
|
||||
Self::Connected => "tsunagi: connected",
|
||||
Self::RangeConflict => "tsunagi: network range conflict; open to switch networks",
|
||||
Self::ExitNode => "tsunagi: using an exit node",
|
||||
}
|
||||
}
|
||||
@@ -108,6 +118,24 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overlapping_networks_are_visible_even_while_other_peers_are_connected() {
|
||||
let mut blocked = NetworkReport {
|
||||
auto_start: true,
|
||||
..Default::default()
|
||||
};
|
||||
blocked
|
||||
.range_conflicts
|
||||
.push(tsunagi::ipc::RangeConflictReport {
|
||||
name: "other".into(),
|
||||
network_id: "other-id".into(),
|
||||
range: "10.90.0.0/16".into(),
|
||||
});
|
||||
let status = report(vec![network(2, ExitReport::default()), blocked]);
|
||||
assert_eq!(Health::of(Some(&status)), Health::RangeConflict);
|
||||
assert!(Health::RangeConflict.label().contains("switch networks"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_agent_or_nobody_connected_is_the_red_state() {
|
||||
assert_eq!(Health::of(None), Health::Disconnected);
|
||||
|
||||
@@ -79,6 +79,8 @@ pub const WIREGUARD_OVERHEAD: u32 = 32;
|
||||
/// Configuration of the WireGuard plugin.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct WireguardConfig {
|
||||
/// Diagnostic protocol id when this end-to-end engine is reused by another carrier.
|
||||
pub protocol: &'static str,
|
||||
/// Directory for the plugin's own key store. Separate from agent state.
|
||||
pub state_dir: PathBuf,
|
||||
/// WireGuard keepalive, which keeps tunnels and their links warm.
|
||||
@@ -100,6 +102,7 @@ impl WireguardConfig {
|
||||
/// Creates a configuration rooted at `state_dir` with sensible defaults.
|
||||
pub fn new(state_dir: impl Into<PathBuf>) -> Self {
|
||||
Self {
|
||||
protocol: WIREGUARD_PROTOCOL,
|
||||
state_dir: state_dir.into(),
|
||||
keepalive: Some(25),
|
||||
mtu: DEFAULT_MTU,
|
||||
@@ -394,7 +397,7 @@ impl Worker {
|
||||
fn report(&self, network: NetworkId, reason: impl std::fmt::Display) {
|
||||
tracing::warn!(network = %network.fmt_short(), %reason, "wireguard plugin error");
|
||||
if let Some(context) = self.context.get() {
|
||||
context.report_error(network, WIREGUARD_PROTOCOL, reason.to_string());
|
||||
context.report_error(network, self.config.protocol, reason.to_string());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -647,7 +650,7 @@ async fn run(worker: Arc<Worker>, mut commands: mpsc::Receiver<Command>) {
|
||||
|
||||
impl IpPlugin for WireguardPlugin {
|
||||
fn protocol_id(&self) -> &str {
|
||||
WIREGUARD_PROTOCOL
|
||||
self.worker.config.protocol
|
||||
}
|
||||
|
||||
fn protocol_version(&self) -> u16 {
|
||||
@@ -686,7 +689,7 @@ impl IpPlugin for WireguardPlugin {
|
||||
// Identity only. Where to send packets is the transport's business.
|
||||
let announcement = WgAnnouncement::new(network, &state.key.public());
|
||||
Ok(Some(PluginCapability {
|
||||
protocol: WIREGUARD_PROTOCOL.to_string(),
|
||||
protocol: self.worker.config.protocol.to_string(),
|
||||
version: crate::announcement::ANNOUNCEMENT_VERSION,
|
||||
enabled: true,
|
||||
data: announcement.encode()?,
|
||||
|
||||
@@ -277,6 +277,7 @@ impl tsunagi::ipc::ReportSource for Control {
|
||||
network_id: network_id.to_string(),
|
||||
already_configured: already,
|
||||
name_shared_with: None,
|
||||
active: self.0.is_active(network_id).await,
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
@@ -109,6 +109,30 @@ impl WgAgent {
|
||||
.await
|
||||
}
|
||||
|
||||
async fn spawn_auto(discovery: &SharedMemoryDiscovery, tag: &str) -> Self {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let tuns = MemoryTunFactory::new();
|
||||
let plugin = WireguardPlugin::open(
|
||||
WireguardConfig::new(dir.path().join("wireguard"))
|
||||
.with_reconcile(Duration::from_millis(20), Duration::from_millis(250)),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let agent = Agent::spawn(
|
||||
config_with(dir.path(), discovery)
|
||||
.with_interface(Arc::new(tuns.clone()), tag, 1280)
|
||||
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
Self {
|
||||
dir,
|
||||
agent,
|
||||
plugin,
|
||||
tuns,
|
||||
}
|
||||
}
|
||||
|
||||
/// Starts an agent with an explicit overlay range, or none at all.
|
||||
async fn spawn_range(
|
||||
discovery: &SharedMemoryDiscovery,
|
||||
@@ -989,7 +1013,7 @@ async fn two_networks_share_one_interface_with_keys_and_ranges_of_their_own() {
|
||||
let (name_b, secret_b) = network("wg-right");
|
||||
let beta_range = Some("10.99.0.0/16".parse::<Ipv4Range>().unwrap());
|
||||
|
||||
let hub = WgAgent::spawn(&discovery, "th").await;
|
||||
let hub = WgAgent::spawn_auto(&discovery, "th").await;
|
||||
let left = WgAgent::spawn(&discovery, "tl").await;
|
||||
let right = WgAgent::spawn_range(&discovery, "tr", beta_range).await;
|
||||
|
||||
|
||||
@@ -231,6 +231,9 @@ impl WireguardLink {
|
||||
}
|
||||
|
||||
impl PacketLink for WireguardLink {
|
||||
fn close(&self) {
|
||||
WireguardLink::close(self);
|
||||
}
|
||||
fn network(&self) -> NetworkId {
|
||||
self.network
|
||||
}
|
||||
|
||||
@@ -14,6 +14,7 @@ path = "src/main.rs"
|
||||
[dependencies]
|
||||
tsunagi-core = { path = "../tsunagi-core", version = "0.1.0", features = ["tun-device", "dns-publish"] }
|
||||
tsunagi-wireguard = { path = "../tsunagi-wireguard", version = "0.1.0" }
|
||||
tsunagi-awg = { path = "../tsunagi-awg", version = "0.1.0" }
|
||||
tsunagi-wg-quic = { path = "../tsunagi-wg-quic", version = "0.1.0" }
|
||||
tsunagi-tcp-tls = { path = "../tsunagi-tcp-tls", version = "0.1.0" }
|
||||
iroh.workspace = true
|
||||
|
||||
+486
-121
@@ -26,6 +26,7 @@ use tsunagi::iroh_types::{EndpointAddr, EndpointId};
|
||||
use tsunagi::overlay::{MemoryTunFactory, TunFactory};
|
||||
use tsunagi::state::Ipv4Range;
|
||||
use tsunagi::{Agent, NetworkId};
|
||||
use tsunagi_awg::{AWG_PROTOCOL, AWG_WIRE_VERSION, AwgPlugin, AwgTransport};
|
||||
use tsunagi_tcp_tls::{
|
||||
TCP_TLS_PROTOCOL, TCP_TLS_VERSION, TcpTlsCodec, TcpTlsConfig, TcpTlsPlugin, TcpTlsTransport,
|
||||
};
|
||||
@@ -68,8 +69,8 @@ enum Command {
|
||||
Up(Box<UpArgs>),
|
||||
/// Reports this device, what the agent is doing, and what this host can do.
|
||||
Status(StatusArgs),
|
||||
/// Shows the protocols this build can carry packets with.
|
||||
Protocols,
|
||||
/// Lists protocols, or shows and changes one network's selection at runtime.
|
||||
Protocols(ProtocolsArgs),
|
||||
/// Makes a network, or joins one — the command that does the work.
|
||||
///
|
||||
/// The same as `tsng network join`, at the top level because it is
|
||||
@@ -106,6 +107,23 @@ struct DnsArgs {
|
||||
action: Option<DnsAction>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Args)]
|
||||
#[command(
|
||||
after_help = "Examples:\n tsng protocols\n tsng protocols home\n tsng protocols home awg,wg-quic\n tsng protocols home none\n\nThe selection is local to this device, saved per network and applied immediately.\nBoth endpoints need a common protocol at the same wire version."
|
||||
)]
|
||||
struct ProtocolsArgs {
|
||||
#[command(flatten)]
|
||||
paths: PathArgs,
|
||||
/// Control socket to talk to; uses the running service by default.
|
||||
#[arg(long, env = "TSUNAGI_CONTROL_SOCKET")]
|
||||
control_socket: Option<PathBuf>,
|
||||
/// Network's exact name, id or unique id prefix.
|
||||
network: Option<String>,
|
||||
/// Replace the selection, best first: awg,wg-quic; none disables data traffic.
|
||||
#[arg(value_name = "LIST", requires = "network")]
|
||||
protocols: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Subcommand)]
|
||||
enum DnsAction {
|
||||
/// Starts serving, now and after every restart.
|
||||
@@ -226,7 +244,9 @@ enum NetworkAction {
|
||||
/// Exact network name, full id or unique id prefix.
|
||||
network: String,
|
||||
},
|
||||
/// Serves a stopped network again, from where it left off.
|
||||
/// Serves a stopped network again, suspending networks with overlapping ranges.
|
||||
///
|
||||
/// Suspended networks resume when this one stops. The choice survives restart.
|
||||
Start {
|
||||
/// Exact network name, full id or unique id prefix.
|
||||
network: String,
|
||||
@@ -351,7 +371,7 @@ fn resolve_ipv4_range(
|
||||
text.parse::<Ipv4Range>()
|
||||
.map_err(|err| format!("--ipv4-range {text}: {err}"))?,
|
||||
)),
|
||||
None => Ok(Some(tsunagi::state::DEFAULT_IPV4_RANGE)),
|
||||
None => Ok(None),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -462,16 +482,18 @@ struct UpArgs {
|
||||
#[arg(long, help_heading = "System")]
|
||||
no_tun: bool,
|
||||
|
||||
/// Protocols to carry packets with, best first.
|
||||
/// Initial protocol selection for networks without a saved choice, best first.
|
||||
///
|
||||
/// A pair of peers uses one they both have at the same wire version. A
|
||||
/// peer with none in common keeps its control plane and gets no data
|
||||
/// plane. `none` runs the control plane alone.
|
||||
/// plane. `none` runs the control plane alone. `tsng protocols NETWORK LIST`
|
||||
/// changes and saves each network's choice while this agent runs; any
|
||||
/// built-in protocol can be enabled, even if absent from this initial list.
|
||||
#[arg(
|
||||
long = "protocol",
|
||||
value_name = "LIST",
|
||||
value_delimiter = ',',
|
||||
default_value = "wg,tcp-tls,wg-quic",
|
||||
default_value = "wg,awg,wg-quic,tcp-tls",
|
||||
help_heading = "Transport"
|
||||
)]
|
||||
protocols: Vec<String>,
|
||||
@@ -489,8 +511,8 @@ struct UpArgs {
|
||||
|
||||
/// IPv4 overlay range, as `address/prefix`, or `none` to disable IPv4.
|
||||
///
|
||||
/// Defaults to 10.13.37.0/24. Only the first member to join decides:
|
||||
/// a network that has already settled on a range wins, and a joining
|
||||
/// Defaults to a stable pseudorandom /24 derived from the network id.
|
||||
/// A network that has already settled on a range wins, and a joining
|
||||
/// agent adopts what it finds. Addresses are allocated from it and
|
||||
/// recorded in signed state, so each member keeps its own across
|
||||
/// restarts and long absences.
|
||||
@@ -894,7 +916,7 @@ async fn run(command: Command) -> Result<(), Box<dyn std::error::Error>> {
|
||||
Command::Id(args) => id(args).await,
|
||||
Command::Up(args) => up(*args).await,
|
||||
Command::Status(args) => status(args).await,
|
||||
Command::Protocols => show_protocols(),
|
||||
Command::Protocols(args) => protocols_command(args).await,
|
||||
Command::Join(args) => join_command(args).await,
|
||||
Command::Network(args) => network_command(args).await,
|
||||
Command::Dns(args) => dns_command(args).await,
|
||||
@@ -1427,25 +1449,31 @@ const PROTOCOLS: &[ProtocolSpec] = &[
|
||||
ProtocolSpec {
|
||||
name: WG_PROTOCOL,
|
||||
version: WG_WIRE_VERSION,
|
||||
summary: "Pure WireGuard direct UDP transport for uncensored networks and maximum performance",
|
||||
summary: "Pure WireGuard direct UDP transport",
|
||||
options: PureWgPlugin::OPTIONS,
|
||||
},
|
||||
ProtocolSpec {
|
||||
name: TCP_TLS_PROTOCOL,
|
||||
version: TCP_TLS_VERSION,
|
||||
summary: "Noise E2EE carried over TCP TLS 1.3 (port 443) disguised as HTTPS traffic",
|
||||
options: TcpTlsCodec::OPTIONS,
|
||||
name: AWG_PROTOCOL,
|
||||
version: AWG_WIRE_VERSION,
|
||||
summary: "Pairwise Amnezia-masked UDP with fresh private profiles on every connection attempt",
|
||||
options: AwgPlugin::OPTIONS,
|
||||
},
|
||||
ProtocolSpec {
|
||||
name: tsunagi_wg_quic::WIREGUARD_PROTOCOL,
|
||||
version: tsunagi_wg_quic::ANNOUNCEMENT_VERSION,
|
||||
summary: "WireGuard's cryptography carried in iroh's QUIC datagrams, so it \
|
||||
crosses NAT and survives where plain WireGuard is blocked",
|
||||
summary: "WireGuard encryption over iroh QUIC datagrams with NAT traversal and relays",
|
||||
options: WireguardPlugin::OPTIONS,
|
||||
},
|
||||
ProtocolSpec {
|
||||
name: TCP_TLS_PROTOCOL,
|
||||
version: TCP_TLS_VERSION,
|
||||
summary: "Noise E2EE carried over TCP TLS 1.3",
|
||||
options: TcpTlsCodec::OPTIONS,
|
||||
},
|
||||
];
|
||||
|
||||
/// One `-o` setting, and the protocol it was aimed at.
|
||||
#[derive(Clone)]
|
||||
struct Setting {
|
||||
/// `Some` when written as `protocol:key=value`.
|
||||
protocol: Option<String>,
|
||||
@@ -1532,6 +1560,7 @@ fn show_protocols() -> Result<(), Box<dyn std::error::Error>> {
|
||||
struct AgentControl {
|
||||
agent: Agent,
|
||||
pure_wg: Option<Arc<PureWgPlugin>>,
|
||||
awg: Option<Arc<AwgPlugin>>,
|
||||
plugin: Option<Arc<WireguardPlugin>>,
|
||||
tcp_tls: Option<Arc<TcpTlsPlugin>>,
|
||||
/// The resolver, which this owns so it can be switched while running.
|
||||
@@ -1547,6 +1576,7 @@ impl tsunagi::ipc::ReportSource for AgentControl {
|
||||
build_report(
|
||||
&self.agent,
|
||||
self.pure_wg.as_deref(),
|
||||
self.awg.as_deref(),
|
||||
self.plugin.as_deref(),
|
||||
self.tcp_tls.as_deref(),
|
||||
dns,
|
||||
@@ -1672,6 +1702,7 @@ impl tsunagi::ipc::ReportSource for AgentControl {
|
||||
network_id: network_id.to_string(),
|
||||
already_configured: already,
|
||||
name_shared_with: shared,
|
||||
active: self.agent.is_active(network_id).await,
|
||||
})
|
||||
})
|
||||
}
|
||||
@@ -1693,6 +1724,23 @@ impl tsunagi::ipc::ReportSource for AgentControl {
|
||||
})
|
||||
}
|
||||
|
||||
fn set_protocols(
|
||||
&self,
|
||||
network_id: String,
|
||||
protocols: Vec<String>,
|
||||
) -> tsunagi::BoxFuture<'_, Result<Vec<String>, String>> {
|
||||
Box::pin(async move {
|
||||
let id = network_id
|
||||
.parse()
|
||||
.map_err(|err| format!("invalid network id: {err}"))?;
|
||||
self.agent
|
||||
.set_protocols(id, protocols.clone())
|
||||
.await
|
||||
.map_err(|err| err.to_string())?;
|
||||
Ok(protocols)
|
||||
})
|
||||
}
|
||||
|
||||
fn set_exit_offer(
|
||||
&self,
|
||||
network_id: String,
|
||||
@@ -1743,36 +1791,50 @@ impl tsunagi::ipc::ReportSource for AgentControl {
|
||||
let wanted: tsunagi::NetworkId = network_id
|
||||
.parse()
|
||||
.map_err(|err| format!("`{network_id}` is not a network id: {err}"))?;
|
||||
let name = self
|
||||
let configured = self
|
||||
.agent
|
||||
.list_networks()
|
||||
.await
|
||||
.map_err(|err| err.to_string())?
|
||||
.into_iter()
|
||||
.find(|network| network.network_id == wanted)
|
||||
.map(|network| network.name.as_str().to_string())
|
||||
.ok_or_else(|| format!("this agent is not in {wanted}"))?;
|
||||
|
||||
let name = configured.name.to_string();
|
||||
let was = self.agent.is_active(wanted).await;
|
||||
if was != active {
|
||||
let mut suspended = Vec::new();
|
||||
if active && !was {
|
||||
// Both of these also remember the answer, so a restart
|
||||
// does what the last instruction said.
|
||||
if active {
|
||||
self.agent
|
||||
.activate_network(wanted)
|
||||
.await
|
||||
.map_err(|err| err.to_string())?;
|
||||
} else {
|
||||
self.agent
|
||||
.deactivate_network(wanted)
|
||||
.await
|
||||
.map_err(|err| err.to_string())?;
|
||||
}
|
||||
let before = self.agent.status().await.map_err(|err| err.to_string())?;
|
||||
let stopped = self
|
||||
.agent
|
||||
.switch_network(wanted)
|
||||
.await
|
||||
.map_err(|err| err.to_string())?;
|
||||
suspended = before
|
||||
.networks
|
||||
.iter()
|
||||
.filter(|n| stopped.contains(&n.network_id))
|
||||
.filter_map(|n| {
|
||||
Some(tsunagi::ipc::RangeConflictReport {
|
||||
network_id: n.network_id.to_string(),
|
||||
name: n.name.to_string(),
|
||||
range: n.range?.to_string(),
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
} else if !active {
|
||||
self.agent
|
||||
.pause_network(wanted)
|
||||
.await
|
||||
.map_err(|err| err.to_string())?;
|
||||
}
|
||||
Ok(tsunagi::ipc::ActiveReport {
|
||||
name,
|
||||
active,
|
||||
changed: was != active,
|
||||
changed: was != active || configured.auto_start != active,
|
||||
suspended,
|
||||
})
|
||||
})
|
||||
}
|
||||
@@ -1841,45 +1903,138 @@ fn resolve_network<'a>(
|
||||
networks: &'a [tsunagi::storage::StoredNetwork],
|
||||
wanted: &str,
|
||||
) -> Result<&'a tsunagi::storage::StoredNetwork, String> {
|
||||
let entries: Vec<_> = networks
|
||||
.iter()
|
||||
.map(|n| (n.network_id.to_string(), n.name.as_str().to_owned()))
|
||||
.collect();
|
||||
resolve_network_index(&entries, wanted).map(|i| &networks[i])
|
||||
}
|
||||
|
||||
/// Shared selector rules for stored networks and secret-free live reports.
|
||||
fn resolve_network_index(networks: &[(String, String)], wanted: &str) -> Result<usize, String> {
|
||||
let wanted = wanted.trim_matches(|c: char| c.is_ascii_whitespace());
|
||||
if wanted.is_empty() {
|
||||
return Err("select a network by name or id; `tsng network` lists them".to_string());
|
||||
return Err("select a network by name or id; `tsng network` lists them".into());
|
||||
}
|
||||
if let Some(network) = networks
|
||||
.iter()
|
||||
.find(|network| network.network_id.to_string() == wanted)
|
||||
{
|
||||
return Ok(network);
|
||||
if let Some(index) = networks.iter().position(|(id, _)| id == wanted) {
|
||||
return Ok(index);
|
||||
}
|
||||
// Only id prefixes lose a copied report ellipsis; names are used verbatim.
|
||||
let prefix = wanted.trim_end_matches('…');
|
||||
let matched: Vec<&tsunagi::storage::StoredNetwork> = networks
|
||||
let matches: Vec<_> = networks
|
||||
.iter()
|
||||
.filter(|network| {
|
||||
network.name.as_str() == wanted
|
||||
|| (!prefix.is_empty() && network.network_id.to_string().starts_with(prefix))
|
||||
})
|
||||
.enumerate()
|
||||
.filter(|(_, (id, name))| name == wanted || (!prefix.is_empty() && id.starts_with(prefix)))
|
||||
.collect();
|
||||
match matched.as_slice() {
|
||||
[one] => Ok(one),
|
||||
match matches.as_slice() {
|
||||
[(index, _)] => Ok(*index),
|
||||
[] => Err(format!(
|
||||
"no configured network matches name or id `{wanted}`; \
|
||||
`tsng network` lists them"
|
||||
"no configured network matches name or id `{wanted}`; `tsng network` lists them"
|
||||
)),
|
||||
several => {
|
||||
let candidates = several
|
||||
several => Err(format!(
|
||||
"`{wanted}` matches {} networks; use the full id or a unique id prefix:\n{}",
|
||||
several.len(),
|
||||
several
|
||||
.iter()
|
||||
.map(|network| format!(" `{}` {}", network.name, network.network_id))
|
||||
.map(|(_, (id, name))| format!(" `{name}` {id}"))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n");
|
||||
Err(format!(
|
||||
"`{wanted}` matches {} networks; use the full id or a unique id prefix:\n{candidates}",
|
||||
several.len()
|
||||
))
|
||||
}
|
||||
.join("\n")
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_protocol_list(text: &str) -> Result<Vec<String>, String> {
|
||||
if text.eq_ignore_ascii_case("none") {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let mut names = Vec::new();
|
||||
for name in text.split(',') {
|
||||
let name = name.trim();
|
||||
let name = if name.eq_ignore_ascii_case("wireguard") {
|
||||
WG_PROTOCOL
|
||||
} else {
|
||||
name
|
||||
};
|
||||
if !PROTOCOLS.iter().any(|spec| spec.name == name) {
|
||||
return Err(format!(
|
||||
"unknown protocol `{name}`; use wg,awg,wg-quic,tcp-tls or `none` alone. Run `tsng protocols` for details"
|
||||
));
|
||||
}
|
||||
if names.iter().any(|n| n == name) {
|
||||
return Err(format!("protocol `{name}` is listed more than once"));
|
||||
}
|
||||
names.push(name.to_owned());
|
||||
}
|
||||
Ok(names)
|
||||
}
|
||||
|
||||
async fn protocols_command(args: ProtocolsArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let Some(selector) = args.network else {
|
||||
return show_protocols();
|
||||
};
|
||||
let requested = args
|
||||
.protocols
|
||||
.as_deref()
|
||||
.map(parse_protocol_list)
|
||||
.transpose()?;
|
||||
let custom = args.paths.state_dir.is_some();
|
||||
let paths = args.paths.resolve()?;
|
||||
let socket = control_socket(&paths, args.control_socket.as_ref(), custom);
|
||||
if tsunagi::ipc::is_serving(&socket).await {
|
||||
let report = tsunagi::ipc::request_status(&socket).await?;
|
||||
let entries: Vec<_> = report
|
||||
.networks
|
||||
.iter()
|
||||
.map(|n| (n.network_id.clone(), n.name.clone()))
|
||||
.collect();
|
||||
let network = &report.networks[resolve_network_index(&entries, &selector)?];
|
||||
let protocols = match requested {
|
||||
Some(protocols) => {
|
||||
tsunagi::ipc::set_protocols(&socket, &network.network_id, protocols).await?
|
||||
}
|
||||
None => network.protocols.clone(),
|
||||
};
|
||||
println!(
|
||||
"protocols {} for `{}` ({}) · preference order",
|
||||
if protocols.is_empty() {
|
||||
"none".into()
|
||||
} else {
|
||||
protocols.join(",")
|
||||
},
|
||||
network.name,
|
||||
network.network_id
|
||||
);
|
||||
} else {
|
||||
// Take the directory ownership lock even for an offline edit. Never
|
||||
// write behind a live agent whose socket cannot be reached.
|
||||
let storage = tsunagi::storage::Storage::open(&paths)?;
|
||||
let configured = storage.list_networks().await?;
|
||||
let network = resolve_network(&configured, &selector)?;
|
||||
let protocols = match requested {
|
||||
Some(protocols) => {
|
||||
storage
|
||||
.set_protocols(network.network_id, protocols.clone())
|
||||
.await?;
|
||||
Some(protocols)
|
||||
}
|
||||
None => network.protocols.clone(),
|
||||
};
|
||||
let shown = protocols
|
||||
.map(|names| {
|
||||
if names.is_empty() {
|
||||
"none".into()
|
||||
} else {
|
||||
names.join(",")
|
||||
}
|
||||
})
|
||||
.unwrap_or_else(|| "defaults from tsng up --protocol".into());
|
||||
println!(
|
||||
"protocols {shown} for `{}` ({}) · agent stopped; applies on next start",
|
||||
network.name, network.network_id
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `tsng network`: what this device belongs to, and leaving it.
|
||||
async fn network_command(args: NetworkArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let paths = args.paths.resolve()?;
|
||||
@@ -1972,21 +2127,28 @@ async fn join_network(
|
||||
.await?;
|
||||
// The id in full either way: it is what every other command takes,
|
||||
// and the shortened form in a report is for reading, not copying.
|
||||
match standing {
|
||||
NetworkStanding::New => {
|
||||
println!("joined `{}` ({})", report.name, report.network_id)
|
||||
if !report.active {
|
||||
println!(
|
||||
"configured `{}` ({}); stopped because its range overlaps an active network. `tsng network start {}` switches to it and suspends conflicting networks",
|
||||
report.name, report.network_id, report.network_id
|
||||
);
|
||||
} else {
|
||||
match standing {
|
||||
NetworkStanding::New => {
|
||||
println!("joined `{}` ({})", report.name, report.network_id)
|
||||
}
|
||||
NetworkStanding::Known => println!(
|
||||
"`{}` ({}) was already here; it is running",
|
||||
report.name, report.network_id
|
||||
),
|
||||
// Joining is an instruction to run it, so it undoes a stop —
|
||||
// said out loud, because a pause that ends without a word is
|
||||
// a pause nobody can rely on.
|
||||
NetworkStanding::Stopped => println!(
|
||||
"`{}` ({}) was stopped; it is running again",
|
||||
report.name, report.network_id
|
||||
),
|
||||
}
|
||||
NetworkStanding::Known => println!(
|
||||
"`{}` ({}) was already here; it is running",
|
||||
report.name, report.network_id
|
||||
),
|
||||
// Joining is an instruction to run it, so it undoes a stop —
|
||||
// said out loud, because a pause that ends without a word is
|
||||
// a pause nobody can rely on.
|
||||
NetworkStanding::Stopped => println!(
|
||||
"`{}` ({}) was stopped; it is running again",
|
||||
report.name, report.network_id
|
||||
),
|
||||
}
|
||||
if let Some(other) = &report.name_shared_with {
|
||||
eprintln!(
|
||||
@@ -2112,6 +2274,13 @@ async fn show_networks(
|
||||
short(&id, 10)
|
||||
),
|
||||
),
|
||||
Some(live) if !live.range_conflicts.is_empty() => (
|
||||
"stopped · range conflict".to_string(),
|
||||
format!(
|
||||
"{}; `tsng network start {id}` switches to this network and suspends them until it stops",
|
||||
conflict_description(live)
|
||||
),
|
||||
),
|
||||
Some(_) => (
|
||||
"stopped".to_string(),
|
||||
format!(
|
||||
@@ -2517,6 +2686,12 @@ async fn set_active(
|
||||
(true, true) => println!("started `{}` ({})", report.name, short(&id, 10)),
|
||||
(true, false) => println!("`{}` ({}) was already running", report.name, short(&id, 10)),
|
||||
}
|
||||
for suspended in &report.suspended {
|
||||
println!(
|
||||
"suspended `{}` ({}): {} overlaps; it resumes when the conflicting network stops",
|
||||
suspended.name, suspended.network_id, suspended.range
|
||||
);
|
||||
}
|
||||
if !report.active {
|
||||
eprintln!(
|
||||
"\nNothing was announced: to the others this device is away, and the address \
|
||||
@@ -2530,7 +2705,11 @@ async fn set_active(
|
||||
|
||||
// No agent: the stored flag is what the next start reads.
|
||||
let storage = tsunagi::storage::Storage::open(paths)?;
|
||||
storage.set_auto_start(network.network_id, active).await?;
|
||||
if active {
|
||||
storage.prefer_network(network.network_id).await?;
|
||||
} else {
|
||||
storage.set_auto_start(network.network_id, false).await?;
|
||||
}
|
||||
storage.release_ownership_lock();
|
||||
println!(
|
||||
"`{name}` ({}) will {} with the next `tsng up`",
|
||||
@@ -3274,6 +3453,15 @@ fn member_rows<'a>(network: &'a tsunagi::ipc::NetworkReport, own_id: &str) -> Ve
|
||||
rows
|
||||
}
|
||||
|
||||
fn conflict_description(network: &tsunagi::ipc::NetworkReport) -> String {
|
||||
network
|
||||
.range_conflicts
|
||||
.iter()
|
||||
.map(|c| format!("`{}` ({}) uses {}", c.name, c.network_id, c.range))
|
||||
.collect::<Vec<_>>()
|
||||
.join("; ")
|
||||
}
|
||||
|
||||
/// One network: what it is, who is in it, and what has happened since start.
|
||||
fn network_section(
|
||||
network: &tsunagi::ipc::NetworkReport,
|
||||
@@ -3328,19 +3516,12 @@ fn network_section(
|
||||
),
|
||||
);
|
||||
}
|
||||
if let Some(conflict) = &network.range_conflict {
|
||||
section.push(
|
||||
Row::new(
|
||||
Health::Degraded,
|
||||
"range",
|
||||
format!("cannot use {conflict}: another network here already does"),
|
||||
)
|
||||
.with_note(
|
||||
"one agent has one interface, so an address belongs to one network. \
|
||||
This one waits to adopt whatever its members settle on; give it \
|
||||
`--ipv4-range` of its own to propose one.",
|
||||
),
|
||||
);
|
||||
if !network.range_conflicts.is_empty() {
|
||||
section.push(Row::new(Health::Degraded, "range conflict", conflict_description(network))
|
||||
.with_note(format!("`tsng network start {}` switches to this network and suspends the listed networks; they resume when it stops", network.network_id)));
|
||||
} else if let Some(conflict) = &network.range_conflict {
|
||||
section.push(Row::new(Health::Degraded, "range", format!("cannot use {conflict}: another network here already does"))
|
||||
.with_note(format!("`tsng network start {}` switches to this network and suspends conflicting networks", network.network_id)));
|
||||
}
|
||||
|
||||
if let Some(overlay) = &network.overlay {
|
||||
@@ -4375,10 +4556,12 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
]));
|
||||
|
||||
let mut config = AgentConfig::new(paths.clone())
|
||||
.with_overlay_ipv4_range(ipv4_range)
|
||||
.with_transport(args.reach.into())
|
||||
.with_discovery(discovery)
|
||||
.with_discovery_interval(Duration::from_secs(5));
|
||||
if args.ipv4_range.is_some() {
|
||||
config = config.with_overlay_ipv4_range(ipv4_range);
|
||||
}
|
||||
if dht_enabled(&args) {
|
||||
config = config.with_dht(MainlineDiscovery::default());
|
||||
}
|
||||
@@ -4392,43 +4575,24 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
// What the user asked for is checked first, before anything that could
|
||||
// fail for a reason of its own: a misspelled protocol or setting is
|
||||
// their mistake to see, not something to bury under a privilege error.
|
||||
let wanted: Vec<&ProtocolSpec> = {
|
||||
let mut wanted = Vec::new();
|
||||
for name in args
|
||||
.protocols
|
||||
.iter()
|
||||
.filter(|name| !name.eq_ignore_ascii_case("none"))
|
||||
{
|
||||
// Accept "wireguard" as a convenience alias for the canonical "wg".
|
||||
let canonical = if name.eq_ignore_ascii_case("wireguard") {
|
||||
WG_PROTOCOL
|
||||
} else {
|
||||
name.as_str()
|
||||
};
|
||||
let Some(spec) = PROTOCOLS.iter().find(|spec| spec.name == canonical) else {
|
||||
let known: Vec<&str> = PROTOCOLS.iter().map(|spec| spec.name).collect();
|
||||
return Err(format!(
|
||||
"this build has no protocol called `{name}`; it has {}. \
|
||||
Run `tsng protocols` to see what each one takes.",
|
||||
known.join(", ")
|
||||
)
|
||||
.into());
|
||||
};
|
||||
wanted.push(spec);
|
||||
}
|
||||
wanted
|
||||
};
|
||||
let initial = parse_protocol_list(&args.protocols.join(","))?;
|
||||
let wanted: Vec<&ProtocolSpec> = initial
|
||||
.iter()
|
||||
.filter_map(|name| PROTOCOLS.iter().find(|spec| spec.name == name))
|
||||
.collect();
|
||||
config.default_protocols = Some(initial);
|
||||
|
||||
let settings = parse_settings(&args.protocol_options)?;
|
||||
// A setting nobody takes is a mistake, not a preference: one that was
|
||||
// silently dropped looks exactly like one that did not work.
|
||||
for setting in &settings {
|
||||
if !wanted
|
||||
if !PROTOCOLS
|
||||
.iter()
|
||||
.filter(|spec| setting.protocol.is_some() || wanted.iter().any(|p| p.name == spec.name))
|
||||
.any(|spec| !settings_for(spec, std::slice::from_ref(setting)).is_empty())
|
||||
{
|
||||
return Err(format!(
|
||||
"no selected protocol takes `{}`; run `tsng protocols` to see what they do",
|
||||
"no configured protocol takes `{}`; qualify disabled protocol settings as protocol:key=value. Run `tsng protocols` for details",
|
||||
setting.key
|
||||
)
|
||||
.into());
|
||||
@@ -4438,11 +4602,13 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
// The interface belongs to the agent, so it is configured once whatever
|
||||
// was selected to carry traffic over it.
|
||||
let mut wireguard = None;
|
||||
let mut awg_transport = None;
|
||||
let mut awg_plugin = None;
|
||||
let mut tcp_tls_transport = None;
|
||||
let mut tcp_tls_plugin = None;
|
||||
let mut pure_wg_plugin: Option<Arc<PureWgPlugin>> = None;
|
||||
let mut pure_wg_transport: Option<Arc<PureWgTransport>> = None;
|
||||
if !wanted.is_empty() {
|
||||
if !PROTOCOLS.is_empty() {
|
||||
let tun_factory: Arc<dyn TunFactory> = if args.no_tun {
|
||||
Arc::new(MemoryTunFactory::new())
|
||||
} else {
|
||||
@@ -4452,9 +4618,60 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
config = config.with_interface(tun_factory, args.interface.clone(), mtu);
|
||||
}
|
||||
|
||||
for spec in &wanted {
|
||||
let options = settings_for(spec, &settings);
|
||||
for spec in PROTOCOLS {
|
||||
// Unqualified options keep targeting the initial selection; explicitly
|
||||
// scoped options also configure protocols that will be enabled later.
|
||||
let scoped: Vec<_> = settings
|
||||
.iter()
|
||||
.filter(|setting| {
|
||||
setting.protocol.is_some() || wanted.iter().any(|p| p.name == spec.name)
|
||||
})
|
||||
.cloned()
|
||||
.collect();
|
||||
let options = settings_for(spec, &scoped);
|
||||
match spec.name {
|
||||
AWG_PROTOCOL => {
|
||||
let mut wg = WireguardConfig::new(paths.state_dir.join("awg"));
|
||||
if let Some(mtu) = args.mtu {
|
||||
wg = wg.with_mtu(mtu);
|
||||
}
|
||||
let tunnel_options: Vec<_> = options
|
||||
.iter()
|
||||
.filter(|(key, _)| key != "stun")
|
||||
.cloned()
|
||||
.collect();
|
||||
let wg = WireguardPlugin::configure(wg, &tunnel_options)?;
|
||||
let server = options
|
||||
.iter()
|
||||
.find(|(key, _)| key == "stun")
|
||||
.map(|(_, value)| value.as_str())
|
||||
.unwrap_or("stun.l.google.com:19302");
|
||||
if server.len() > 255 || (server != "off" && !server.contains(':')) {
|
||||
return Err("awg:stun must be HOST:PORT or off".into());
|
||||
}
|
||||
let stun = (!matches!(args.reach, Reach::Local) && server != "off")
|
||||
.then(|| server.to_owned());
|
||||
let state_db = paths.state_db();
|
||||
let identity = tokio::task::spawn_blocking(move || {
|
||||
tsunagi::storage::StateStore::open(state_db)?.load_or_create_device_identity()
|
||||
})
|
||||
.await??;
|
||||
let transport = AwgTransport::new_with_stun(
|
||||
identity.endpoint_id(),
|
||||
std::net::Ipv4Addr::UNSPECIFIED,
|
||||
stun,
|
||||
);
|
||||
let plugin = AwgPlugin::open(wg, transport.clone()).await?;
|
||||
config = config
|
||||
.with_plugin(plugin.clone() as Arc<dyn IpPlugin>)
|
||||
.with_custom_transport(
|
||||
AWG_PROTOCOL,
|
||||
transport.clone()
|
||||
as Arc<dyn tsunagi::dataplane::transport::PacketTransport>,
|
||||
);
|
||||
awg_transport = Some(transport);
|
||||
awg_plugin = Some(plugin);
|
||||
}
|
||||
tsunagi_wg_quic::WIREGUARD_PROTOCOL => {
|
||||
let mut wg = WireguardConfig::new(paths.state_dir.join("wg-quic"));
|
||||
if let Some(mtu) = args.mtu {
|
||||
@@ -4558,6 +4775,16 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
// is never dropped without being closed.
|
||||
let mut events = agent.subscribe();
|
||||
|
||||
if let Some(transport) = awg_transport {
|
||||
let inbound_agent = agent.clone();
|
||||
tokio::spawn(async move {
|
||||
while let Some(inbound) = transport.next_inbound().await {
|
||||
if let Err(err) = inbound_agent.install_inbound_link(inbound).await {
|
||||
tracing::debug!(%err, "failed to install inbound awg link");
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
if let Some(transport) = tcp_tls_transport {
|
||||
let agent_inbound = agent.clone();
|
||||
tokio::spawn(async move {
|
||||
@@ -4625,12 +4852,14 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let control = {
|
||||
let agent = agent.clone();
|
||||
let pure_wg = pure_wg_plugin.clone();
|
||||
let awg = awg_plugin.clone();
|
||||
let plugin = wireguard.clone();
|
||||
let tcp_tls = tcp_tls_plugin.clone();
|
||||
let dns = Arc::clone(&dns);
|
||||
let source: Arc<dyn tsunagi::ipc::ReportSource> = Arc::new(AgentControl {
|
||||
agent,
|
||||
pure_wg,
|
||||
awg,
|
||||
plugin,
|
||||
tcp_tls,
|
||||
dns,
|
||||
@@ -4716,6 +4945,7 @@ async fn up(args: UpArgs) -> Result<(), Box<dyn std::error::Error>> {
|
||||
async fn build_report(
|
||||
agent: &Agent,
|
||||
pure_wg: Option<&PureWgPlugin>,
|
||||
awg: Option<&AwgPlugin>,
|
||||
wireguard: Option<&WireguardPlugin>,
|
||||
tcp_tls: Option<&TcpTlsPlugin>,
|
||||
dns: Option<DnsState>,
|
||||
@@ -4736,6 +4966,7 @@ async fn build_report(
|
||||
.networks
|
||||
.iter()
|
||||
.map(|network| {
|
||||
let awg_view = awg.and_then(|plugin| plugin.overview(network.network_id));
|
||||
let wg_view = wireguard.and_then(|plugin| plugin.overview(network.network_id));
|
||||
let pure_wg_view = pure_wg.and_then(|plugin| plugin.overview(network.network_id));
|
||||
let tcp_tls_reports = tcp_tls
|
||||
@@ -4748,14 +4979,18 @@ async fn build_report(
|
||||
let wg_overlay_addr = wg_view
|
||||
.as_ref()
|
||||
.and_then(|v| v.overlay_address_v4)
|
||||
.or_else(|| pure_wg_view.as_ref().and_then(|v| v.overlay_address_v4));
|
||||
.or_else(|| pure_wg_view.as_ref().and_then(|v| v.overlay_address_v4))
|
||||
.or_else(|| awg_view.as_ref().and_then(|v| v.overlay_address_v4));
|
||||
let wg_prefix_len = wg_view
|
||||
.as_ref()
|
||||
.and_then(|v| v.ipv4_range)
|
||||
.or_else(|| pure_wg_view.as_ref().and_then(|v| v.ipv4_range))
|
||||
.or_else(|| awg_view.as_ref().and_then(|v| v.ipv4_range))
|
||||
.map_or(0, |r| r.prefix_len);
|
||||
let has_any_plugin =
|
||||
wg_view.is_some() || pure_wg_view.is_some() || !tcp_tls_reports.is_empty();
|
||||
let has_any_plugin = wg_view.is_some()
|
||||
|| pure_wg_view.is_some()
|
||||
|| awg_view.is_some()
|
||||
|| !tcp_tls_reports.is_empty();
|
||||
|
||||
let overlay = if has_any_plugin && overlay.is_some() {
|
||||
let mut peers: Vec<OverlayPeerReport> = Vec::new();
|
||||
@@ -4783,6 +5018,13 @@ async fn build_report(
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(ref view) = awg_view {
|
||||
for peer in &view.peers {
|
||||
if !all_peer_ids.contains(&peer.endpoint_id) {
|
||||
all_peer_ids.push(peer.endpoint_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
for report in &tcp_tls_reports {
|
||||
if !all_peer_ids.contains(&report.peer) {
|
||||
all_peer_ids.push(report.peer);
|
||||
@@ -4793,6 +5035,9 @@ async fn build_report(
|
||||
let pure_wg_match = pure_wg_view
|
||||
.as_ref()
|
||||
.and_then(|v| v.peers.iter().find(|p| p.endpoint_id == peer_id));
|
||||
let awg_match = awg_view
|
||||
.as_ref()
|
||||
.and_then(|v| v.peers.iter().find(|p| p.endpoint_id == peer_id));
|
||||
let tcp_tls_match = tcp_tls_reports.iter().find(|r| r.peer == peer_id);
|
||||
let wg_quic_match = wg_view
|
||||
.as_ref()
|
||||
@@ -4830,6 +5075,7 @@ async fn build_report(
|
||||
let pubkey = pure_wg_match
|
||||
.map(|p| p.public_key.to_string())
|
||||
.or_else(|| wg_quic_match.map(|p| p.public_key.to_string()))
|
||||
.or_else(|| awg_match.map(|p| p.public_key.to_string()))
|
||||
.unwrap_or_default();
|
||||
peers.push(OverlayPeerReport {
|
||||
endpoint_id: peer_id.to_string(),
|
||||
@@ -4845,6 +5091,27 @@ async fn build_report(
|
||||
protocol_errors: 0,
|
||||
path: tcp_tls.path.clone(),
|
||||
});
|
||||
} else if let Some(awg_peer) = awg_match
|
||||
&& let Some(tunnel) = &awg_peer.tunnel
|
||||
{
|
||||
peers.push(OverlayPeerReport {
|
||||
endpoint_id: peer_id.to_string(),
|
||||
protocol: AWG_PROTOCOL.into(),
|
||||
public_key: awg_peer.public_key.to_string(),
|
||||
address: awg_peer
|
||||
.overlay_address_v4
|
||||
.map(|a| a.to_string())
|
||||
.or(member_addr),
|
||||
handshake_secs_ago: tunnel.health.since_handshake.map(|s| s.as_secs()),
|
||||
tx_packets: tunnel.stats.tx_packets,
|
||||
rx_packets: tunnel.stats.rx_packets,
|
||||
tx_bytes: tunnel.stats.tx_bytes,
|
||||
rx_bytes: tunnel.stats.rx_bytes,
|
||||
dropped: tunnel.stats.dropped_wrong_source
|
||||
+ tunnel.stats.dropped_oversize,
|
||||
protocol_errors: tunnel.stats.protocol_errors,
|
||||
path: tunnel.path.clone(),
|
||||
});
|
||||
} else if let Some(wg_peer) = wg_quic_match
|
||||
&& let Some(tunnel) = &wg_peer.tunnel
|
||||
{
|
||||
@@ -4884,6 +5151,24 @@ async fn build_report(
|
||||
protocol_errors: 0,
|
||||
path: "no data link".into(),
|
||||
});
|
||||
} else if let Some(awg_peer) = awg_match {
|
||||
peers.push(OverlayPeerReport {
|
||||
endpoint_id: peer_id.to_string(),
|
||||
protocol: AWG_PROTOCOL.into(),
|
||||
public_key: awg_peer.public_key.to_string(),
|
||||
address: awg_peer
|
||||
.overlay_address_v4
|
||||
.map(|a| a.to_string())
|
||||
.or(member_addr),
|
||||
handshake_secs_ago: None,
|
||||
tx_packets: 0,
|
||||
rx_packets: 0,
|
||||
tx_bytes: 0,
|
||||
rx_bytes: 0,
|
||||
dropped: 0,
|
||||
protocol_errors: 0,
|
||||
path: "no data link".into(),
|
||||
});
|
||||
} else if let Some(wg_peer) = wg_quic_match {
|
||||
peers.push(OverlayPeerReport {
|
||||
endpoint_id: peer_id.to_string(),
|
||||
@@ -4947,6 +5232,8 @@ async fn build_report(
|
||||
};
|
||||
|
||||
NetworkReport {
|
||||
protocols: network.protocols.clone(),
|
||||
auto_start: network.auto_start,
|
||||
exit: exit_report(network),
|
||||
broadcast: network.broadcast,
|
||||
name: network.name.to_string(),
|
||||
@@ -4994,6 +5281,15 @@ async fn build_report(
|
||||
.collect(),
|
||||
range: network.range.map(|range| range.to_string()),
|
||||
range_conflict: network.range_conflict.map(|range| range.to_string()),
|
||||
range_conflicts: network
|
||||
.range_conflicts
|
||||
.iter()
|
||||
.map(|c| tsunagi::ipc::RangeConflictReport {
|
||||
network_id: c.network_id.to_string(),
|
||||
name: c.name.to_string(),
|
||||
range: c.range.to_string(),
|
||||
})
|
||||
.collect(),
|
||||
dial_failures: network.metrics.dial_failures,
|
||||
handshake_failures: network.metrics.handshake_failures,
|
||||
control_messages: (
|
||||
@@ -5007,6 +5303,13 @@ async fn build_report(
|
||||
.collect();
|
||||
|
||||
StatusReport {
|
||||
available_protocols: PROTOCOLS
|
||||
.iter()
|
||||
.map(|p| tsunagi::ipc::ProtocolReport {
|
||||
name: p.name.into(),
|
||||
summary: p.summary.into(),
|
||||
})
|
||||
.collect(),
|
||||
endpoint_id: status.endpoint_id.to_string(),
|
||||
hostname: status.hostname.clone(),
|
||||
bound_sockets: status
|
||||
@@ -5361,6 +5664,8 @@ mod status_tests {
|
||||
/// The situation that prompted this: one peer left and came back.
|
||||
fn network_after_a_peer_returned() -> NetworkReport {
|
||||
NetworkReport {
|
||||
protocols: vec!["wg-quic".into()],
|
||||
auto_start: true,
|
||||
exit: Default::default(),
|
||||
hostname_conflict: None,
|
||||
broadcast: true,
|
||||
@@ -5403,6 +5708,7 @@ mod status_tests {
|
||||
],
|
||||
range: Some("10.13.37.0/24".into()),
|
||||
range_conflict: None,
|
||||
range_conflicts: Vec::new(),
|
||||
// Everything below happened while the peer was away.
|
||||
dial_failures: 9,
|
||||
handshake_failures: 0,
|
||||
@@ -5534,7 +5840,36 @@ mod status_tests {
|
||||
let text = out.render(false);
|
||||
assert!(text.contains("no range to allocate from"), "{text}");
|
||||
assert!(text.contains("another network here already does"), "{text}");
|
||||
assert!(text.contains("--ipv4-range"), "the fix is named: {text}");
|
||||
assert!(
|
||||
text.contains("tsng network start"),
|
||||
"the fix is named: {text}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_blocked_network_names_all_blockers_and_the_switch_action() {
|
||||
let mut network = network_after_a_peer_returned();
|
||||
network.active = false;
|
||||
network.overlay = None;
|
||||
network.range_conflicts = vec![tsunagi::ipc::RangeConflictReport {
|
||||
name: "other-network".into(),
|
||||
network_id: "full-other-network-id".into(),
|
||||
range: "10.13.0.0/16".into(),
|
||||
}];
|
||||
let mut out = report::Report::new();
|
||||
out.push(network_section(&network, OWN, false));
|
||||
let text = out.render(false);
|
||||
let health = out.worst();
|
||||
for detail in [
|
||||
"other-network",
|
||||
"full-other-network-id",
|
||||
"10.13.0.0/16",
|
||||
"tsng network start xa7gyz",
|
||||
"resume when it stops",
|
||||
] {
|
||||
assert!(text.contains(detail), "missing {detail}: {text}");
|
||||
}
|
||||
assert_eq!(health, Health::Degraded);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -5955,6 +6290,34 @@ mod status_tests {
|
||||
#[cfg(test)]
|
||||
mod network_tests {
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
#[test]
|
||||
fn default_carriers_include_awg_and_explicit_order_is_preserved() {
|
||||
use clap::Parser;
|
||||
for (words, expected) in [
|
||||
(vec!["tsng", "up"], vec!["wg", "awg", "wg-quic", "tcp-tls"]),
|
||||
(
|
||||
vec![
|
||||
"tsng",
|
||||
"up",
|
||||
"--protocol",
|
||||
"awg,wg-quic",
|
||||
"-o",
|
||||
"awg:stun=off",
|
||||
],
|
||||
vec!["awg", "wg-quic"],
|
||||
),
|
||||
] {
|
||||
let super::Command::Up(args) = super::Cli::try_parse_from(words).unwrap().command
|
||||
else {
|
||||
panic!("expected up");
|
||||
};
|
||||
assert_eq!(args.protocols, expected);
|
||||
for name in expected {
|
||||
assert!(super::PROTOCOLS.iter().any(|spec| spec.name == name));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dht_defaults_and_local_mode_do_not_require_manual_peers() {
|
||||
use clap::Parser;
|
||||
@@ -5985,6 +6348,7 @@ mod network_tests {
|
||||
NetworkSecret::from_bytes(&[secret.as_bytes(), &[0u8; 32]].concat()[..32]).unwrap();
|
||||
let keys = NetworkKeys::derive(&name, &secret);
|
||||
StoredNetwork {
|
||||
protocols: None,
|
||||
exit_node: false,
|
||||
exit_via: None,
|
||||
broadcast: true,
|
||||
@@ -6208,6 +6572,7 @@ mod network_context_tests {
|
||||
let secret = NetworkSecret::from_bytes([seed; 32]).unwrap();
|
||||
let keys = NetworkKeys::derive(&name, &secret);
|
||||
StoredNetwork {
|
||||
protocols: None,
|
||||
exit_node: false,
|
||||
exit_via: None,
|
||||
broadcast: true,
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
|
||||
|
||||
use std::time::Duration;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use tempfile::TempDir;
|
||||
|
||||
@@ -50,8 +50,17 @@ fn read_networks(dir: &TempDir) -> Vec<tsunagi::storage::StoredNetwork> {
|
||||
/// An agent with no network at all, the way a daemon is started before
|
||||
/// anything has been decided.
|
||||
fn start_bare(port: u16) -> Running {
|
||||
start_bare_range(port, None)
|
||||
}
|
||||
|
||||
fn start_bare_range(port: u16, range: Option<&str>) -> Running {
|
||||
start_bare_config(port, range, None)
|
||||
}
|
||||
|
||||
fn start_bare_config(port: u16, range: Option<&str>, protocols: Option<&str>) -> Running {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let child = std::process::Command::new(env!("CARGO_BIN_EXE_tsng"))
|
||||
let mut command = std::process::Command::new(env!("CARGO_BIN_EXE_tsng"));
|
||||
command
|
||||
.arg("up")
|
||||
.arg("--state-dir")
|
||||
.arg(dir.path().join("state"))
|
||||
@@ -62,11 +71,34 @@ fn start_bare(port: u16) -> Running {
|
||||
.arg(format!("127.0.0.1:{port}"))
|
||||
.args(["--log", "error", "--status-interval", "0"])
|
||||
.stdout(std::process::Stdio::null())
|
||||
.stderr(std::process::Stdio::null())
|
||||
.spawn()
|
||||
.expect("the agent binary starts");
|
||||
std::thread::sleep(Duration::from_secs(2));
|
||||
Running { child, dir }
|
||||
.stderr(std::process::Stdio::null());
|
||||
if let Some(range) = range {
|
||||
command.args(["--ipv4-range", range]);
|
||||
}
|
||||
if let Some(protocols) = protocols {
|
||||
command.args(["--protocol", protocols]);
|
||||
}
|
||||
let child = command.spawn().expect("the agent binary starts");
|
||||
let mut agent = Running { child, dir };
|
||||
let runtime = tokio::runtime::Builder::new_current_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
.unwrap();
|
||||
let socket = tsunagi::ipc::control_socket_path(&agent.dir.path().join("state"));
|
||||
runtime.block_on(async {
|
||||
tokio::time::timeout(Duration::from_secs(30), async {
|
||||
while !tsunagi::ipc::is_serving(&socket).await {
|
||||
assert!(
|
||||
agent.child.try_wait().unwrap().is_none(),
|
||||
"agent exited before becoming ready"
|
||||
);
|
||||
tokio::time::sleep(Duration::from_millis(25)).await;
|
||||
}
|
||||
})
|
||||
.await
|
||||
.expect("agent becomes ready");
|
||||
});
|
||||
agent
|
||||
}
|
||||
|
||||
/// An agent already in one network, the way most of these start.
|
||||
@@ -87,6 +119,147 @@ fn start(network: &str, port: u16) -> Running {
|
||||
agent
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn protocols_absent_from_startup_can_be_enabled_and_changed_offline() {
|
||||
let mut agent = start_bare_config(0, None, Some("none"));
|
||||
assert!(
|
||||
agent
|
||||
.run(&[
|
||||
"join",
|
||||
"--network",
|
||||
"from-control-only",
|
||||
"--secret",
|
||||
"a-control-only-test-secret"
|
||||
])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
let initial = agent.run(&["protocols", "from-control-only"]);
|
||||
assert!(initial.status.success());
|
||||
assert!(String::from_utf8_lossy(&initial.stdout).contains("protocols none"));
|
||||
let enabled = agent.run(&["protocols", "from-control-only", "awg"]);
|
||||
assert!(
|
||||
enabled.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&enabled.stderr)
|
||||
);
|
||||
assert_eq!(
|
||||
read_networks(&agent.dir)[0].protocols,
|
||||
Some(vec!["awg".into()])
|
||||
);
|
||||
agent.child.kill().unwrap();
|
||||
agent.child.wait().unwrap();
|
||||
let saved = agent.run(&["protocols", "from-control-only", "wg-quic,awg"]);
|
||||
assert!(
|
||||
saved.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&saved.stderr)
|
||||
);
|
||||
assert!(String::from_utf8_lossy(&saved.stdout).contains("agent stopped"));
|
||||
assert_eq!(
|
||||
read_networks(&agent.dir)[0].protocols,
|
||||
Some(vec!["wg-quic".into(), "awg".into()])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn protocols_are_changed_live_per_network_and_saved_without_changing_the_service() {
|
||||
let agent = start("protocol-choice", 0);
|
||||
assert!(
|
||||
agent
|
||||
.run(&[
|
||||
"join",
|
||||
"--network",
|
||||
"other-choice",
|
||||
"--secret",
|
||||
"other-secret-for-cli-test"
|
||||
])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
let before = read_networks(&agent.dir);
|
||||
let other_id = before
|
||||
.iter()
|
||||
.find(|n| n.name.as_str() == "other-choice")
|
||||
.unwrap()
|
||||
.network_id;
|
||||
let changed = agent.run(&["protocols", "protocol-choice", "awg,wg-quic"]);
|
||||
assert!(
|
||||
changed.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&changed.stderr)
|
||||
);
|
||||
let shown = agent.run(&["protocols", "protocol-choice"]);
|
||||
assert!(shown.status.success());
|
||||
assert!(String::from_utf8_lossy(&shown.stdout).contains("awg,wg-quic"));
|
||||
let saved = read_networks(&agent.dir);
|
||||
assert_eq!(
|
||||
saved
|
||||
.iter()
|
||||
.find(|n| n.name.as_str() == "protocol-choice")
|
||||
.unwrap()
|
||||
.protocols,
|
||||
Some(vec!["awg".into(), "wg-quic".into()])
|
||||
);
|
||||
assert_eq!(
|
||||
saved
|
||||
.iter()
|
||||
.find(|n| n.network_id == other_id)
|
||||
.unwrap()
|
||||
.protocols,
|
||||
before
|
||||
.iter()
|
||||
.find(|n| n.network_id == other_id)
|
||||
.unwrap()
|
||||
.protocols
|
||||
);
|
||||
for invalid in ["unknown", "awg,awg", "none,awg"] {
|
||||
assert!(
|
||||
!agent
|
||||
.run(&["protocols", "protocol-choice", invalid])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
}
|
||||
assert!(!agent.run(&["protocols", "missing", "awg"]).status.success());
|
||||
let disabled = agent.run(&["protocols", "protocol-choice", "none"]);
|
||||
assert!(
|
||||
disabled.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&disabled.stderr)
|
||||
);
|
||||
assert_eq!(
|
||||
read_networks(&agent.dir)
|
||||
.iter()
|
||||
.find(|n| n.name.as_str() == "protocol-choice")
|
||||
.unwrap()
|
||||
.protocols,
|
||||
Some(Vec::new())
|
||||
);
|
||||
assert!(
|
||||
agent
|
||||
.run(&["network", "stop", "protocol-choice"])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
assert!(
|
||||
agent
|
||||
.run(&["protocols", "protocol-choice", "wg-quic"])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
assert!(
|
||||
agent
|
||||
.run(&["network", "start", "protocol-choice"])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
assert!(
|
||||
String::from_utf8_lossy(&agent.run(&["protocols", "protocol-choice"]).stdout)
|
||||
.contains("wg-quic")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn joining_with_no_secret_makes_one_and_prints_what_to_send() {
|
||||
// The hurry case: a network with somebody, for as long as it is needed,
|
||||
@@ -461,3 +634,139 @@ fn an_exit_node_is_offered_per_network_only_when_asked_and_nobody_offers_by_defa
|
||||
String::from_utf8_lossy(&unknown.stderr)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overlapping_networks_are_reported_and_cli_start_switches_then_resumes() {
|
||||
let agent = start_bare_range(0, Some("10.90.0.0/16"));
|
||||
for name in ["resident", "games"] {
|
||||
let joined = agent.run(&["join", "-n", name, "-s", "a-secret-for-the-conflict-test"]);
|
||||
assert!(
|
||||
joined.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&joined.stderr)
|
||||
);
|
||||
if name == "games" {
|
||||
let text = String::from_utf8_lossy(&joined.stdout);
|
||||
assert!(
|
||||
text.contains("stopped because its range overlaps"),
|
||||
"{text}"
|
||||
);
|
||||
assert!(!text.contains("it is running"), "{text}");
|
||||
}
|
||||
}
|
||||
let runtime = tokio::runtime::Builder::new_current_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
.unwrap();
|
||||
let socket = tsunagi::ipc::control_socket_path(&agent.dir.path().join("state"));
|
||||
let snapshot = || {
|
||||
runtime
|
||||
.block_on(tsunagi::ipc::request_status(&socket))
|
||||
.unwrap()
|
||||
};
|
||||
let before = snapshot();
|
||||
let resident = before
|
||||
.networks
|
||||
.iter()
|
||||
.find(|n| n.name == "resident")
|
||||
.unwrap();
|
||||
let games = before.networks.iter().find(|n| n.name == "games").unwrap();
|
||||
assert!(resident.active);
|
||||
assert!(!games.active);
|
||||
assert_eq!(games.range_conflicts[0].network_id, resident.network_id);
|
||||
assert_eq!(games.range_conflicts[0].name, "resident");
|
||||
for args in [vec!["status"], vec!["network"]] {
|
||||
let result = agent.run(&args);
|
||||
let text = String::from_utf8_lossy(&result.stdout);
|
||||
assert!(text.contains("range conflict"), "{text}");
|
||||
assert!(text.contains(&resident.network_id), "{text}");
|
||||
assert!(text.contains("tsng network start"), "{text}");
|
||||
}
|
||||
let switched = agent.run(&["network", "start", "games"]);
|
||||
assert!(
|
||||
switched.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&switched.stderr)
|
||||
);
|
||||
let text = String::from_utf8_lossy(&switched.stdout);
|
||||
assert!(
|
||||
text.contains("started `games`") && text.contains("suspended `resident`"),
|
||||
"{text}"
|
||||
);
|
||||
let after = snapshot();
|
||||
assert!(
|
||||
after
|
||||
.networks
|
||||
.iter()
|
||||
.find(|n| n.name == "games")
|
||||
.unwrap()
|
||||
.active
|
||||
);
|
||||
assert!(
|
||||
!after
|
||||
.networks
|
||||
.iter()
|
||||
.find(|n| n.name == "resident")
|
||||
.unwrap()
|
||||
.active
|
||||
);
|
||||
assert!(agent.run(&["network", "stop", "games"]).status.success());
|
||||
assert!(
|
||||
snapshot()
|
||||
.networks
|
||||
.iter()
|
||||
.find(|n| n.name == "resident")
|
||||
.unwrap()
|
||||
.active
|
||||
);
|
||||
|
||||
assert!(agent.run(&["network", "start", "games"]).status.success());
|
||||
let stop_suspended = agent.run(&["network", "stop", "resident"]);
|
||||
assert!(stop_suspended.status.success());
|
||||
assert!(String::from_utf8_lossy(&stop_suspended.stdout).contains("stopped `resident`"));
|
||||
assert!(agent.run(&["network", "stop", "games"]).status.success());
|
||||
assert!(
|
||||
snapshot()
|
||||
.networks
|
||||
.iter()
|
||||
.all(|n| !n.active && !n.auto_start)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cli_defaults_derive_each_networks_range() {
|
||||
let agent = start_bare(0);
|
||||
for name in ["derived-one", "derived-two"] {
|
||||
assert!(
|
||||
agent
|
||||
.run(&["join", "-n", name, "-s", "a-secret-for-the-derived-test"])
|
||||
.status
|
||||
.success()
|
||||
);
|
||||
}
|
||||
let runtime = tokio::runtime::Builder::new_current_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
.unwrap();
|
||||
let socket = tsunagi::ipc::control_socket_path(&agent.dir.path().join("state"));
|
||||
let deadline = Instant::now() + Duration::from_secs(30);
|
||||
loop {
|
||||
let report = runtime
|
||||
.block_on(tsunagi::ipc::request_status(&socket))
|
||||
.unwrap();
|
||||
if report.networks.iter().all(|n| n.range.is_some()) {
|
||||
assert_eq!(report.networks.len(), 2);
|
||||
for network in report.networks {
|
||||
let id = network.network_id.parse().unwrap();
|
||||
assert!(network.active);
|
||||
assert_eq!(
|
||||
network.range,
|
||||
Some(tsunagi::state::derived_ipv4_range(id).to_string())
|
||||
);
|
||||
}
|
||||
break;
|
||||
}
|
||||
assert!(Instant::now() < deadline, "networks allocate ranges");
|
||||
std::thread::sleep(Duration::from_millis(25));
|
||||
}
|
||||
}
|
||||
+131
@@ -0,0 +1,131 @@
|
||||
# Pairwise Amnezia transport
|
||||
|
||||
The CLI's default carrier preference is `wg,awg,wg-quic,tcp-tls`. To select
|
||||
AWG explicitly:
|
||||
|
||||
```sh
|
||||
tsng up --protocol awg
|
||||
tsng up --protocol wg,awg,wg-quic,tcp-tls
|
||||
```
|
||||
|
||||
The startup list is the initial per-network preference. A saved network choice
|
||||
overrides it and can enable any built-in protocol at runtime:
|
||||
|
||||
```sh
|
||||
tsng protocols home awg,wg-quic
|
||||
tsng protocols home awg
|
||||
```
|
||||
|
||||
Changes are local to this device, persist across restarts, and retire existing
|
||||
data links and end-to-end channels for that network before renegotiation.
|
||||
Authenticated control sessions, membership, addresses and other networks stay
|
||||
live. Removing every protocol (`none` in the CLI, all checkboxes off in the tray)
|
||||
leaves the network control-only. This is different from automatic carrier
|
||||
fallback, which preserves the selected end-to-end channel.
|
||||
|
||||
`awg` is Tsunagi AWG wire version 1. Its packet masking engine is adapted from
|
||||
[RunaRoo/AmneziaWG-RS](https://github.com/RunaRoo/AmneziaWG-RS), with the pinned
|
||||
source and license recorded in
|
||||
[UPSTREAM.md](../crates/tsunagi-awg/UPSTREAM.md). It uses the existing boringtun
|
||||
Noise/WireGuard implementation instead of upstream's Node, global executors
|
||||
and OS interface machinery. Compatibility with ordinary Amnezia servers is
|
||||
not implemented: this carrier connects authenticated Tsunagi members.
|
||||
|
||||
## Negotiation and lifetime
|
||||
|
||||
There is one independent UDP socket and cancellable packet-engine task per
|
||||
`(network, peer)` pair. The smaller endpoint id initiates, following the same
|
||||
rule as other data carriers. Both sides must announce enabled AWG wire version
|
||||
1. An authenticated iroh control session is required throughout; AWG supplies
|
||||
no alternate discovery or membership mechanism.
|
||||
|
||||
Every `open` generates new masking parameters, a header-protection key, an
|
||||
outer WireGuard key pair and a PSK. The initiator sends a private `Offer` with
|
||||
a random attempt id, its public key, candidate socket addresses and the shared
|
||||
profile. The responder validates it, creates its own fresh key pair and socket,
|
||||
and returns an `Answer` naming that attempt. Both send masked handshakes and
|
||||
encrypted delivery probes to the candidates. Only authenticated WireGuard
|
||||
**data**, not a STUN response or an incoming UDP address alone, verifies a path.
|
||||
|
||||
Negotiation payloads are bounded at 4096 bytes (a caller can lower the limit),
|
||||
redacted in Debug, excluded from public message events and erased on drop.
|
||||
They are never capabilities, signed records, cache entries or exported status.
|
||||
At most 128 pairs are admitted per transport, with bounded command/data queues,
|
||||
at most 32 candidates per attempt and finite establishment deadlines.
|
||||
Stale answers and duplicate offers are rejected; a new attempt replaces and
|
||||
closes an older pair engine. Cancellation, peer departure, network deactivation
|
||||
and shutdown retire sockets without waiting for a remote reader.
|
||||
|
||||
A process restart, a failed setup, or a complete fallback cycle returning to AWG
|
||||
starts fresh negotiation. Normal WireGuard rekeying within a live attempt keeps
|
||||
that attempt's masking profile. A failed pair does not reconfigure other pairs
|
||||
or other networks. A full mesh of 15 members uses 14 engines per member and 105
|
||||
independent pairs in total; it does not create additional system interfaces.
|
||||
|
||||
## Masking
|
||||
|
||||
Each profile randomizes disjoint H1–H4 header ranges, a 12–48-byte S1–S4 prefix,
|
||||
a ChaCha20 header-protection key, handshake trailers, junk packet count/lengths,
|
||||
and extra content padding. Prefix lengths are equal within one profile so
|
||||
header detection has one offset. A fresh random nonce is in every packet's
|
||||
prefix; handshake headers and transport headers/counters are protected.
|
||||
The standard WireGuard header is restored before boringtun authenticates a
|
||||
message. This preserves boringtun's standard MAC calculations, which is another
|
||||
reason this is a Tsunagi-specific AWG wire version.
|
||||
|
||||
Before a handshake, an I1 signature packet is generated from either a complete
|
||||
DNS query or SIP OPTIONS template, with fresh identifiers and consistent length
|
||||
fields, followed by random junk packets. I2–I5 are unused in this version.
|
||||
The templates are structural generators, not replayed captures. QUIC imitation
|
||||
is not selected. Masking can change DPI classification; it is not a promise of
|
||||
indistinguishability or that any particular firewall will admit it.
|
||||
|
||||
## Packets, NAT and fallback
|
||||
|
||||
The pair engine carries opaque relay envelopes inside authenticated outer WG
|
||||
packets. It fragments those envelopes into bounded chunks and expires incomplete
|
||||
assemblies; a logical datagram is bounded at 64 KiB and an outer UDP packet is
|
||||
at most 1200 bytes. The encapsulated IPv4 headers are internal framing only: no interface or OS address is created for a
|
||||
pair. Overlay addresses, routing, ownership checks and end-to-end encryption
|
||||
remain independent. A relay decrypts its own adjacent AWG carrier to forward
|
||||
opaque end-to-end ciphertext, never the destination's IP packet.
|
||||
|
||||
Local IPv4 candidates and simultaneous encrypted probes provide direct UDP
|
||||
connectivity. In `direct`/`relay` reach mode the CLI also uses
|
||||
`stun.l.google.com:19302` to discover a server-reflexive address on **this pair's
|
||||
socket**, with a 1.2-second bound. A STUN result is only a candidate. Local reach
|
||||
mode makes no STUN request. Configure or disable it with
|
||||
`-o awg:stun=HOST:PORT` or `-o awg:stun=off`. Failed discovery still permits local
|
||||
candidates. IPv6-only paths, symmetric NAT and networks blocking arbitrary UDP
|
||||
may fail AWG and fall back; there is no AWG-over-QUIC substitution or AWG TURN
|
||||
relay hidden behind its name.
|
||||
|
||||
Data delivery checks go through the actual end-to-end encrypted tunnel, once
|
||||
per second, independently of control-plane RTT. Requests and replies bind a
|
||||
fresh private marker/key to the sender's endpoint id. Unanswered requests expire
|
||||
after three seconds. After a 20-second warmup, at least 15 completed observations
|
||||
and at least 60% loss in the last 20 observations retire a carrier. Probes are
|
||||
consumed locally after source/destination admission and never enter the TUN.
|
||||
AWG additionally checks encrypted outer delivery throughout the pair lifetime.
|
||||
|
||||
Setup failure or persistent runtime loss advances to the next mutually supported
|
||||
carrier in the initiator's order. Earlier expired cooldowns do not jump ahead of
|
||||
untried carriers. Retry cooldowns remain bounded; reaching AWG again negotiates
|
||||
new parameters. Control sessions and the logical end-to-end tunnel survive a
|
||||
carrier change. Status names the end-to-end protocol and the actual carrier
|
||||
separately, so `wg` over `awg` is a valid result.
|
||||
|
||||
Control wire version 6 introduces the private transport message. All members
|
||||
must upgrade together; device identities, network ids, handshake transcript
|
||||
encoding, signed state and stored network configuration are unchanged. Data
|
||||
ALPN remains `tsunagi/data/5`; the relay envelope is unchanged.
|
||||
|
||||
## Verification
|
||||
|
||||
The default rootless, offline tests use real iroh endpoints, membership proofs,
|
||||
SQLite, UDP sockets, masked WireGuard handshakes, end-to-end encryption and
|
||||
memory TUNs. They cover independent three-member pairs, full-size bidirectional
|
||||
packets, restart freshness/socket release, 80% runtime datagram loss, fallback
|
||||
to real iroh and fresh negotiation on retry, plus malformed profiles/messages,
|
||||
role checks, probe forgery and STUN response parsing. These tests do not establish
|
||||
DPI effectiveness, WAN throughput, public NAT traversal or per-pair memory cost.
|
||||
@@ -20,8 +20,9 @@ network secret. The core moves an opaque, bounded payload and never parses it.
|
||||
a `PacketLink` — an authenticated, unreliable datagram channel to one peer for
|
||||
one protocol — and never learn how it is carried.
|
||||
|
||||
The separation between the two planes is **logical, not physical**. Both ride
|
||||
on iroh, on different ALPNs and different connections. That is deliberate:
|
||||
The separation between the two planes is **logical, not physical**. The control plane uses
|
||||
iroh; the data plane can use iroh on its own ALPN and connection, or independent
|
||||
UDP/TLS carriers. That is deliberate:
|
||||
iroh's whole value is hole punching a direct path between peers behind NAT,
|
||||
with a relay as fallback, and a data plane that refused to use it would have to
|
||||
reimplement all of it. What the separation buys is that `proto` knows nothing
|
||||
@@ -32,6 +33,10 @@ A plugin talks to the core through three narrow hooks — `on_network_activated`
|
||||
a `PluginContext` for re-announcements and error reports, and a bounded
|
||||
`shutdown` — so the core never learns anything protocol-specific.
|
||||
|
||||
Private pair negotiation uses a bounded transport sender supplied by the core.
|
||||
Protocol crates do not encode or parse control messages and never receive the
|
||||
network secret. No pair profiles enter signed state.
|
||||
|
||||
A data plane failure never stops the daemon: the control plane keeps running
|
||||
and the agent stays manageable.
|
||||
|
||||
@@ -68,6 +73,7 @@ And in crates of their own:
|
||||
|
||||
| crate | responsibility |
|
||||
|---|---|
|
||||
| `tsunagi-awg` | ephemeral pairwise Amnezia masking, private negotiation and independently encrypted UDP carriers; reuses the end-to-end WireGuard engine |
|
||||
| `tsunagi-wg-quic` | the `wg-quic` protocol: its keys, its announcement, its tunnels |
|
||||
| `tsunagi` | the command line agent, built as the `tsng` executable: the only place that owns a runtime, a logger and signals |
|
||||
|
||||
|
||||
+6
-1
@@ -9,11 +9,15 @@ Two versions exist and are independent:
|
||||
|
||||
- **Identity scheme**, `tsunagi-network-id-v1`. Frozen. Changing it creates a
|
||||
different network space for the same name and secret.
|
||||
- **Control protocol**, ALPN `tsunagi/ctrl/5`, `PROTOCOL_VERSION = 5`.
|
||||
- **Control protocol**, ALPN `tsunagi/ctrl/6`, `PROTOCOL_VERSION = 6`.
|
||||
|
||||
Upgrading the crate or bumping the control protocol must never change an
|
||||
existing `NetworkId`.
|
||||
|
||||
Control version 6 adds private transport negotiation for pairwise AWG and data
|
||||
delivery checks. Data ALPN remains `tsunagi/data/5`. All members must upgrade
|
||||
together; network ids and stored state are unchanged. See [amnezia.md](amnezia.md).
|
||||
|
||||
## Canonical encoding
|
||||
|
||||
Everything that is hashed or MAC'd uses length-prefixed fields, written here as
|
||||
@@ -230,6 +234,7 @@ not affect other networks.
|
||||
| message | meaning |
|
||||
|---|---|
|
||||
| `Announce { hostname, capabilities, broadcast }` | this agent's hostname, IP-plugin capabilities and local broadcast participation |
|
||||
| `Transport { protocol, payload }` | private, bounded per-peer transport negotiation; never state or a public message event |
|
||||
| `Ping { seq, payload }` | small request used to verify the exchange |
|
||||
| `Pong { seq, payload }` | the echoed reply |
|
||||
| `State { records }` | a snapshot of signed records, merged into what the receiver holds |
|
||||
|
||||
+9
-1
@@ -96,6 +96,14 @@ topology change can move a flow; datagrams remain unreliable and unordered,
|
||||
and there is no promise to preserve order across a physical path failure.
|
||||
Logical peer links and end-to-end encryption state survive these changes.
|
||||
|
||||
## Carrier health and switching
|
||||
|
||||
Carrier setup failures and persistent encrypted delivery loss advance the
|
||||
initiator's ordered fallback cycle. Logical end-to-end links survive while
|
||||
physical links are retired explicitly. The AWG carrier negotiates a fresh
|
||||
private pair profile on every reentry, without changing topology's durable
|
||||
state. See [amnezia.md](amnezia.md) for thresholds and bounds.
|
||||
|
||||
## Verification and measurement
|
||||
|
||||
The default offline suite exercises four real agents, iroh transports and
|
||||
@@ -116,5 +124,5 @@ counts sends without storing frames. Results are CPU forwarding cost, **not**
|
||||
end-to-end network latency or VPN throughput; encryption, fragmentation,
|
||||
sockets, congestion and scheduling contribute separately.
|
||||
|
||||
Wire compatibility: control ALPN `tsunagi/ctrl/5`, data ALPN `tsunagi/data/5`.
|
||||
Wire compatibility: control ALPN `tsunagi/ctrl/6`, data ALPN `tsunagi/data/5`.
|
||||
Upgrade every participant together; saved identities and network state persist.
|
||||
@@ -98,6 +98,13 @@ a batch does not stop the rest, and allocation is deterministic, spread out,
|
||||
walks past everything taken and reports a full range instead of handing out a
|
||||
duplicate.
|
||||
|
||||
`tests/range_conflicts.rs` checks isolated first members choosing the same
|
||||
range, persistence across restart, preservation of legacy ranges, exclusion
|
||||
of equal and nested ranges, late authenticated conflicts, switching all
|
||||
blockers, automatic resume, cancelling a suspended network's resume and
|
||||
concurrent switch requests. These use real agents, loopback handshakes and
|
||||
SQLite; they do not change host network settings.
|
||||
|
||||
`tests/local_control.rs` covers the local control socket end to end: a client
|
||||
asking a running agent for status over a real Unix socket, joining and
|
||||
leaving a network through it, a leftover socket file being replaced while a
|
||||
@@ -170,3 +177,24 @@ TSUNAGI_TEST_LOG=tsunagi=debug cargo test --test multi_peer -- --nocapture
|
||||
|
||||
The library never installs a global subscriber; the harness opts in only when
|
||||
that variable is set.
|
||||
|
||||
The `tsunagi-awg` suite runs real pair negotiation and masked UDP WireGuard
|
||||
engines with memory TUNs. A loss wrapper drops datagrams at the carrier boundary
|
||||
without substituting iroh, membership, encryption or SQLite. It checks runtime
|
||||
loss detection, fallback and fresh profiles, in addition to restart and input
|
||||
validation. Tests are offline; public STUN/DPI/NAT effectiveness needs separate
|
||||
real-network validation.
|
||||
|
||||
Runtime protocol selection tests switch real AWG and iroh/QUIC carriers and
|
||||
end-to-end WireGuard tunnels from either endpoint-id role, carry full-size
|
||||
packets, keep another active network's tunnels unchanged, disable and re-enable
|
||||
the data plane without losing control sessions, and preserve selection after
|
||||
restart. CLI tests use a real agent process and control socket to exercise
|
||||
selection by network name, stopped networks, invalid lists, and persisted
|
||||
control-only configuration, offline updates, and enabling a protocol absent
|
||||
from the startup list. A real AWG test starts without a TUN, creates the single
|
||||
memory interface on enablement, and races rapid changes against pending opens.
|
||||
SQLite tests migrate schema 7 to 8 without changing
|
||||
identity, secrets or signed records, preserve order and explicit empty choices,
|
||||
and reject malformed stored selections. These checks use memory TUNs and do
|
||||
not establish GUI rendering or behavior on an actual OS interface.
|
||||
+13
-2
@@ -26,8 +26,8 @@ Read this before relying on anything here. The protocol is in
|
||||
neither receive nor forge another member's traffic. See
|
||||
[wireguard.md](wireguard.md#address-ownership-is-enforced-not-announced).
|
||||
- **Tunnelled traffic is end-to-end encrypted by WireGuard**, independently of
|
||||
this crate. The transport underneath is also encrypted by iroh, but the
|
||||
tunnel's confidentiality does not depend on that.
|
||||
this crate. Carriers may be iroh, TLS, direct WG or
|
||||
pairwise AWG; confidentiality does not depend on a particular carrier.
|
||||
- **Resource bounds.** Frame lengths are validated before allocation; strings,
|
||||
lists, queues, concurrent dials and in-flight handshakes are all bounded;
|
||||
handshakes, dials and writes have timeouts.
|
||||
@@ -113,3 +113,14 @@ for key separation, HMAC-SHA256 for the membership proof, constant-time
|
||||
verification, iroh's Ed25519 endpoint keys and QUIC/TLS for the transport, and
|
||||
RFC 5705 TLS exporter output for channel binding. There is no custom encryption
|
||||
layer and no custom PAKE.
|
||||
|
||||
## Pairwise AWG masking
|
||||
|
||||
AWG profiles, PSKs and outer keys are private to one authenticated pair attempt,
|
||||
not shared network state. Address candidates are unverified until authenticated
|
||||
outer WireGuard data arrives. Header protection and structured signature packets
|
||||
reduce fixed wire markers; they do not guarantee admission by DPI or hide traffic
|
||||
analysis. iroh control availability remains a prerequisite. Encrypted end-to-end
|
||||
delivery probes detect sustained carrier loss, never confer membership and pass
|
||||
the same signed source/destination admission as ordinary IP traffic. See
|
||||
[amnezia.md](amnezia.md) for limits and tested scope.
|
||||
+30
-7
@@ -92,8 +92,8 @@ and then recorded as a signed fact**, using the model in
|
||||
[sync-model.md](sync-model.md).
|
||||
|
||||
```text
|
||||
default range 10.13.37.0/24 (override with --ipv4-range)
|
||||
who decides the first member to claim; later ones adopt what they find
|
||||
default range stable pseudorandom /24 in 10/8, derived from NetworkId
|
||||
who decides all members derive the default; signed explicit ranges win
|
||||
who signs the claiming member, with its persistent device key
|
||||
where it is kept state.sqlite, and every replica that has seen it
|
||||
what a return costs nothing: the old address is reclaimed
|
||||
@@ -120,11 +120,34 @@ the same address at the same moment are resolved by a rule both compute
|
||||
identically — the lower endpoint id keeps it — and the loser simply allocates
|
||||
again with a higher version.
|
||||
|
||||
**The range is agreed, not configured per member.** `--ipv4-range` says what
|
||||
this agent would use; a network that has already settled on something else
|
||||
wins, and the agent adopts it. So the flag matters for whoever starts the
|
||||
network and is harmless afterwards. Pass `--ipv4-range none` for an IPv6-only
|
||||
overlay.
|
||||
**The range is agreed, not configured per member.** New networks use the
|
||||
same deterministic `/24` on every device, independently of which network a
|
||||
member activates first or whether any peers are reachable. The derivation
|
||||
is unchanged from the former fallback; network identity and signed record
|
||||
encoding are unchanged. Distinct network ids can still derive the same range.
|
||||
|
||||
`--ipv4-range` proposes an explicit range; existing signed ranges take
|
||||
precedence over the new derived default, including the former `10.13.37.0/24`.
|
||||
Among disagreeing non-default ranges the lowest author id still wins. Updated
|
||||
members propagate their adopted range in their signed claims; older releases
|
||||
still apply their original selection rule until those claims converge.
|
||||
`--ipv4-range none` disables local proposals, as before.
|
||||
|
||||
An agent reserves each active network's range before starting its runtime.
|
||||
Overlapping ranges, including nested blocks such as a `/24` inside a `/16`,
|
||||
cannot run simultaneously. A late authenticated state update that introduces
|
||||
an overlap suspends that network before publishing new allocations to plugins
|
||||
or the interface. Other networks continue running.
|
||||
|
||||
Status includes every blocking network's name, id and range. `tsng network
|
||||
start <name-or-id>` and the tray's switch button stop all blockers before
|
||||
starting the selected network. Suspended networks keep their startup request
|
||||
and resume when the blocker stops or is forgotten. A manual stop cancels that
|
||||
request even for a network already suspended. The user's preference order is
|
||||
stored locally, so the same selection wins at the next startup; it is never
|
||||
advertised or treated as a remote network setting. Local IPC uses wire version
|
||||
19 for the new reports: restart the agent and tray together after upgrading.
|
||||
The peer control protocol and signed record encoding do not change.
|
||||
|
||||
The agent puts the address on the interface itself, as soon as the network has
|
||||
agreed on it — no restart, and the interface is not recreated, which would
|
||||
|
||||
Reference in new issue
Block a user