Document podcast contract and bump crate versions
Add PODCASTS.md as the normative podcast contract: identity and URL canonicalization, the no-GUID rule, synced versus local versus public data, operation kinds and merge rules with examples, snapshot recovery, version gating for peers below v4, Jam privacy and the reserved playback_rate. Update PLAYBACK_PROTOCOL.md, ARCHITECTURE.md, the crate READMEs and CHANGELOG.md. Bump music-dht to 0.6.0 and furumi-library to 0.3.0: new public enum variants and struct fields are breaking changes for 0.x, and furumi-library re-exposes music-dht types. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
2d4782819d
commit
4f7e999280
12 files changed
+563
-10
No files matched your search
+32
-1
@@ -177,13 +177,39 @@ their local output is playing. The web gateway uses a passive server profile;
|
||||
browser startup and explicit selection are separate inputs.
|
||||
|
||||
This register provides eventual exclusivity after a partition heals, not a
|
||||
quorum lock during a partition. Device-sync profiles now advertise version 3;
|
||||
quorum lock during a partition. Device-sync profiles now advertise version 4
|
||||
(version 3 introduced coordination, version 4 adds podcast queue items);
|
||||
the additive coordination envelope is independently versioned and transported
|
||||
over the existing JSON `furumi/sync/2` stream. Legacy library records remain
|
||||
readable, but legacy unfenced personal playback messages cannot control an
|
||||
upgraded output. See [PLAYBACK_PROTOCOL.md](PLAYBACK_PROTOCOL.md) for the
|
||||
contract, adapter obligations, configuration and rollout rules.
|
||||
|
||||
Podcasts are a separate content type with a pure shared contract in
|
||||
`music_dht::podcast` (normative description: [PODCASTS.md](PODCASTS.md)). It
|
||||
owns canonical feed URLs, domain-separated BLAKE3 podcast and episode ids,
|
||||
feed aliases, the no-GUID episode rule, bounded metadata DTOs, the
|
||||
self-contained `PodcastEpisodeRef` queue item, and convergent merge functions
|
||||
for subscriptions, share decisions, progress registers and listening
|
||||
sessions. The tolerant RSS/Atom parser is shared behind the `podcast-feed`
|
||||
feature because episode identity depends on it. HTTP fetching, scheduling,
|
||||
buffering and decoding remain client runtime work, and persistence lives in
|
||||
client storage that implements the same rules. Podcast audio is fetched from
|
||||
the publisher: it is never imported, content-addressed or offered for P2P
|
||||
transfer.
|
||||
|
||||
Device-sync protocol v4 carries podcast state between trusted devices only:
|
||||
four `podcast_*` operations, bounded snapshot sections (the recovery path,
|
||||
because sync vectors never re-request operations hidden from an older peer),
|
||||
and an explicit `PlaybackTrack.source`. Operation and source enums are closed,
|
||||
so senders gate on the version each peer reported itself; podcast state is
|
||||
never sent to peers below v4, handoff to them is refused, and their playback
|
||||
commands are ignored while the queue contains episodes. Jam snapshots replace
|
||||
episodes with opaque placeholders. Progress ordering uses a Lamport
|
||||
generation rather than wall-clock `hlc_ms`, so a device with a fast clock
|
||||
cannot undo an observed seek or restart. Public podcast discovery is a future
|
||||
separate overlay; the music DHT schema and `federation-net` are unchanged.
|
||||
|
||||
Music discovery identifies an owner and content; it does not force all domain
|
||||
traffic through DHT messages. `music-dht` exposes the underlying authenticated
|
||||
stream capability and defines shared wire models for Furumi catalog and device
|
||||
@@ -283,3 +309,8 @@ architectural constraint:
|
||||
- Local durable state survives ordinary peer and network failure.
|
||||
- Additional stream protocols reuse authenticated connectivity without
|
||||
coupling bulk data to the typed control channel.
|
||||
- Private personal state (podcast subscriptions, progress, history) stays in
|
||||
the trusted-device group; Jam and public overlays only see explicit public
|
||||
projections or opaque placeholders.
|
||||
- A device's protocol version is what that device reported; relays never
|
||||
substitute their own.
|
||||
@@ -1,5 +1,23 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased — podcast contract
|
||||
|
||||
- `music-dht 0.6.0` (breaking): new `music_dht::podcast` module with the shared
|
||||
podcast contract: canonical `FeedUrl`, `PodcastId`/`EpisodeId` derivation
|
||||
(`furumi-podcast:*:v1`), feed aliases, the no-GUID episode identity rule,
|
||||
`PodcastMeta`/`EpisodeMeta`/`PodcastEpisodeRef`, and convergent merge rules
|
||||
for subscriptions, shares, progress registers and listening sessions.
|
||||
- `music-dht 0.6.0`: optional `podcast-feed` feature with the shared tolerant
|
||||
RSS/Atom parser (`quick-xml`, `encoding_rs`, `url`).
|
||||
- `music-dht 0.6.0`: device-sync protocol version 4 (ALPN unchanged): four
|
||||
`podcast_*` operation kinds, bounded podcast snapshot sections,
|
||||
`PlaybackTrack.source`, reserved `PlaybackStateWire.playback_rate`,
|
||||
`DeviceProfileSet.protocol_version`, version-gating helpers and Jam
|
||||
redaction. New struct fields and enum variants break exhaustive
|
||||
construction and matching in clients.
|
||||
- `furumi-library 0.3.0`: follows `music-dht 0.6.0`, whose types it re-exposes.
|
||||
- See `PODCASTS.md` for the normative contract.
|
||||
|
||||
## v0.4.1 — 2026-09-02
|
||||
|
||||
- `federation-net 0.3.1`: recover timed-out or failed Mainline-DHT rendezvous clients and expose network health to applications.
|
||||
|
||||
Generated
+2
-2
@@ -932,7 +932,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "furumi-library"
|
||||
version = "0.2.0"
|
||||
version = "0.3.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"blake3",
|
||||
@@ -2099,7 +2099,7 @@ checksum = "743fb55ba31b18fb1ecef6bdc9aa2743314978ac084044301a7eee33fb99a20d"
|
||||
|
||||
[[package]]
|
||||
name = "music-dht"
|
||||
version = "0.5.0"
|
||||
version = "0.6.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
|
||||
+1
-1
@@ -14,7 +14,7 @@ repository = "https://gt.hexor.cy/ab/frid"
|
||||
|
||||
[workspace.dependencies]
|
||||
federation-net = { path = "crates/federation-net", version = "0.3.1" }
|
||||
music-dht = { path = "crates/music-dht", version = "0.5.0" }
|
||||
music-dht = { path = "crates/music-dht", version = "0.6.0" }
|
||||
iroh = "1"
|
||||
iroh-base = "1"
|
||||
iroh-tickets = "1"
|
||||
|
||||
+44
-2
@@ -97,8 +97,9 @@ is a runtime input, not a persisted setting.
|
||||
## Versioning and rollout
|
||||
|
||||
The library-sync JSON ALPN remains `furumi/sync/2`; catalog, pairing and library
|
||||
data do not require a new transport. Device profiles advertise protocol version
|
||||
3. Coordination envelopes independently carry version 1. Snapshot coordination
|
||||
data do not require a new transport. Device profiles advertised protocol
|
||||
version 3 when coordination was introduced (version 4 adds podcast queue items,
|
||||
see below). Coordination envelopes independently carry version 1. Snapshot coordination
|
||||
and operation authority are additive optional JSON fields so library-sync
|
||||
records from older clients still deserialize. Upgraded clients ignore legacy
|
||||
personal playback snapshots/commands without the coordination envelope.
|
||||
@@ -109,6 +110,47 @@ applications while preventing divergent control message definitions. These
|
||||
envelopes use JSON, not a backwards-compatible promise for postcard layouts.
|
||||
Do not change Jam's independent authority by interpreting personal stamps there.
|
||||
|
||||
## Protocol v4: podcast queue items
|
||||
|
||||
Device profiles advertise protocol version 4. The JSON ALPN stays
|
||||
`furumi/sync/2` and the coordination envelope stays version 1. Version 4 adds
|
||||
to the portable queue DTO:
|
||||
|
||||
- `PlaybackTrack.source`: `music`, `podcast_episode` with a self-contained
|
||||
`PodcastEpisodeRef`, or `private` (Jam placeholder). It is omitted for
|
||||
music, so music-only state serializes exactly as in v3. Podcast episodes are
|
||||
never encoded as negative ids or URLs in `file_path`.
|
||||
- `PlaybackStateWire.playback_rate`: reserved, omitted when unset. Treat a
|
||||
missing or unsupported value as 1.0.
|
||||
|
||||
`PlaybackSource` is a closed enum: a v3 client fails to parse any message
|
||||
with a podcast item. Adapters therefore apply the shared helpers from
|
||||
`music_dht::device_sync`:
|
||||
|
||||
1. Send a playback snapshot to a peer only when
|
||||
`playback_allowed_for_peer(state, peer_version)` holds; otherwise send
|
||||
`None`.
|
||||
2. Refuse an explicit transfer to a device below v4 while the queue contains
|
||||
podcast items and show that the target needs an update. The decision uses
|
||||
the version that device reported itself (`profile_protocol_version`),
|
||||
never the local constant. Do not strip episodes from the queue.
|
||||
3. Ignore `SetState`/`ActiveChanged` from peers below v4 while the local
|
||||
queue contains podcast items (`playback_command_allowed_from_peer`).
|
||||
4. A stamped `playback_command` operation with podcast items is itself gated
|
||||
by `op_allowed_for_peer`.
|
||||
|
||||
A handoff of a podcast episode records the position and the play/pause intent
|
||||
first, transfers the queue with its sources, and lets the receiver open the
|
||||
publisher's HTTP source and seek. Ownership still changes only through the
|
||||
claim, so a late response cannot make the previous owner resume audio. The
|
||||
durable episode progress is a separate register; see `PODCASTS.md`.
|
||||
|
||||
Jam hosts send `music_dht::jam::redact_jam_snapshot`, which replaces podcast
|
||||
items with opaque `private` placeholders and drops the personal coordination
|
||||
envelope. They reject participant commands with real episodes
|
||||
(`jam_command_allowed`) and map placeholders back with
|
||||
`restore_jam_private_items` before applying a participant's state.
|
||||
|
||||
For local integration checks, patch the three frid crates with Cargo's external
|
||||
`--config` mechanism. Keep released dependency declarations and lockfiles in
|
||||
the applications; publish frid and then bump client dependencies for rollout.
|
||||
+400
@@ -0,0 +1,400 @@
|
||||
# Podcast contract
|
||||
|
||||
Status: normative for `music-dht` 0.6 and device-sync protocol v4.
|
||||
|
||||
Podcasts are a separate content type shared by every Furumi client. This
|
||||
document defines what all clients must agree on: identity, normalization,
|
||||
private state and its merge rules, trusted-device sync operations, snapshot
|
||||
recovery, queue items, version gating and Jam privacy. The Rust definitions
|
||||
live in `music_dht::podcast` (pure types and functions) and
|
||||
`music_dht::device_sync` (wire DTOs and gating helpers). Clients must use
|
||||
them instead of local copies.
|
||||
|
||||
Public discovery (signed podcast cards in a separate DHT overlay) is not part
|
||||
of this version of the contract; it will be specified with the overlay.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
| Layer | Owns |
|
||||
| --- | --- |
|
||||
| `music_dht::podcast` | ids, URL canonicalization, no-GUID rule, metadata DTOs and bounds, merge functions, RSS/Atom parser (feature `podcast-feed`) |
|
||||
| `music_dht::device_sync` | sync operation kinds, snapshot sections, `PlaybackSource`, version gating, Jam redaction |
|
||||
| Client storage (`furumi-library`, web PostgreSQL) | persistence and migrations implementing the rules below |
|
||||
| Client runtime | HTTP fetching, conditional requests, scheduling, audio buffering and decoding, UI |
|
||||
|
||||
Audio is always fetched from the publisher. It is never imported into the
|
||||
music library, never content-addressed with a BLAKE3 content id and never
|
||||
offered for P2P transfer. RSS refreshes fetch metadata only.
|
||||
|
||||
## Data classes
|
||||
|
||||
| Data | Trusted devices | Public federation | Local only |
|
||||
| --- | --- | --- | --- |
|
||||
| Subscription state, channel metadata, feed aliases | `podcast_subscription_set` + snapshot | never | |
|
||||
| Share decision | `podcast_share_set` + snapshot | only the published card (future overlay) | |
|
||||
| Episode progress | `podcast_progress_set` + snapshot | never | |
|
||||
| Listening sessions, play counts | `podcast_listen_recorded` + snapshot | never | |
|
||||
| Episode catalog | each device parses RSS itself; `PodcastEpisodeRef` travels with progress and queue items | never | |
|
||||
| ETag/Last-Modified, last check time, fetch errors, artwork cache, audio buffer | | | yes |
|
||||
|
||||
Synchronizing to a trusted device is not permission to publish. Private
|
||||
podcasts, their URLs, descriptions and episodes must not reach DHT records,
|
||||
public catalogs, public search results or Jam participants.
|
||||
|
||||
## Identity
|
||||
|
||||
### Feed URL canonicalization
|
||||
|
||||
`FeedUrl::parse` produces the canonical form. Every client must use it (or
|
||||
an implementation that passes the same tests) before deriving ids or
|
||||
comparing URLs.
|
||||
|
||||
1. Only `http://` and `https://` are accepted (case-insensitive) and the
|
||||
scheme is lowercased.
|
||||
2. Userinfo (`user@`, `user:pass@`), whitespace, control characters and `\`
|
||||
are rejected.
|
||||
3. The host is lowercased. Non-ASCII hosts are rejected; pass the IDNA
|
||||
(`xn--`) form as produced by WHATWG URL parsers. Bracketed IPv6 literals
|
||||
are accepted.
|
||||
4. An empty port and the default port (80 for http, 443 for https) are
|
||||
removed; other ports lose leading zeros.
|
||||
5. The fragment is removed.
|
||||
6. An empty path becomes `/`. The rest of the path and the **entire** query
|
||||
string are kept byte-for-byte: parameter order, case, percent-encoding and
|
||||
dot segments are preserved. Query parameters can be part of the source
|
||||
identity (for example a per-user token) and must never be dropped. Only
|
||||
non-ASCII characters are percent-encoded (UTF-8, uppercase hex).
|
||||
7. The result is at most 2048 bytes. Canonicalization is idempotent;
|
||||
deserialization accepts only canonical strings.
|
||||
|
||||
| Input | Canonical |
|
||||
| --- | --- |
|
||||
| `HTTPS://Example.COM:443/Feed.xml?b=2&a=1#top` | `https://example.com/Feed.xml?b=2&a=1` |
|
||||
| `http://example.com:80` | `http://example.com/` |
|
||||
| `https://example.com/rss?token=XyZ&x=%2f` | unchanged |
|
||||
| `https://user@example.com/rss` | rejected |
|
||||
|
||||
`FeedUrl::from_user_input` is for typed or pasted input only: it trims, maps
|
||||
`feed:`, `podcast:`, `pcast:`, `itpc:` to `https://` and adds `https://` when
|
||||
the scheme is missing, then canonicalizes.
|
||||
|
||||
### Podcast id
|
||||
|
||||
```text
|
||||
podcast_id = "pod1:" + base32_lower_nopad(BLAKE3("furumi-podcast:podcast:v1" || 0x00 || canonical_feed_url))
|
||||
```
|
||||
|
||||
The id is derived **once**, from the canonical URL of the first subscription.
|
||||
It never changes afterwards.
|
||||
|
||||
### Feed aliases
|
||||
|
||||
Redirect targets (permanent redirects), `itunes:new-feed-url` and URLs used to
|
||||
subscribe again are feed aliases of an existing podcast:
|
||||
|
||||
* the subscription record carries `feed_aliases`, a grow-only set merged
|
||||
independently of the last-writer-wins register (`merge_feed_aliases`). The
|
||||
URL that derives the podcast id is always kept, plus the 8 smallest other
|
||||
URLs; this bounded union is commutative, associative and idempotent;
|
||||
* `podcast.feed_url` is the URL currently fetched and is always added to the
|
||||
aliases;
|
||||
* before deriving a new id for a URL, clients call `resolve_podcast_id` with
|
||||
every known `(podcast_id, url)` pair. A match reuses the existing podcast,
|
||||
its progress and its history. With several matches the smallest id wins.
|
||||
|
||||
Limitation: two devices that subscribe offline, through different URLs of
|
||||
the same feed, before either learns the alias, create two podcast ids. The
|
||||
contract does not merge podcast ids automatically in this version.
|
||||
|
||||
### Episode id
|
||||
|
||||
```text
|
||||
episode_id = "ep1:" + base32_lower_nopad(BLAKE3("furumi-podcast:episode:v1" || 0x00 || podcast_id || key))
|
||||
```
|
||||
|
||||
| Key | Material |
|
||||
| --- | --- |
|
||||
| GUID | `guid:` + RSS `<guid>` / Atom `<id>` trimmed, case-sensitive |
|
||||
| link | `link:` + item `<link>` trimmed |
|
||||
| date + title | `date-title:` + publication unix seconds + `\0` + normalized title |
|
||||
| enclosure | `enclosure:` + enclosure URL trimmed |
|
||||
|
||||
The normalized title is `music_dht::normalize_name` (NFKC, lowercase,
|
||||
non-alphanumeric runs collapsed). GUIDs longer than 2048 bytes are treated as
|
||||
absent. Changing the title, artwork, description or audio URL of an episode
|
||||
never changes its id and never resets its progress.
|
||||
|
||||
### No-GUID rule
|
||||
|
||||
`resolve_episode_ids(podcast_id, channel_link, items, known)` is normative.
|
||||
For one parsed feed document:
|
||||
|
||||
1. Items with a GUID get the GUID id. A GUID never matches another known
|
||||
episode, so the result does not depend on local history.
|
||||
2. An item without GUID first searches the known episodes of the podcast,
|
||||
stage by stage: same enclosure URL, then same `<link>`, then same
|
||||
publication second and normalized title. The first stage with candidates
|
||||
wins and the smallest candidate id is reused. A known episode can be
|
||||
claimed by only one item of the document.
|
||||
3. Otherwise a new id is derived from the first available key: `<link>`,
|
||||
then date + title, then enclosure URL.
|
||||
|
||||
A `<link>` is identifying only if it is unique in the document and differs
|
||||
from the channel `<link>` (many feeds repeat the homepage in every item).
|
||||
Enclosure URLs and date + title pairs shared by several GUID-less items are
|
||||
not identifying either. Items without any usable key, and duplicates of an
|
||||
earlier item of the same document, get no id and are skipped.
|
||||
|
||||
"Known episodes" include episodes from earlier refreshes **and** episodes
|
||||
learned through sync (`PodcastEpisodeRef::identity_hints`). Feed GUID-less
|
||||
episodes into it so a device that sees an episode first through sync keeps
|
||||
the id used by the other devices.
|
||||
|
||||
Limitation: for GUID-less episodes the id can depend on what a device saw
|
||||
first. If a publisher changes several identifying fields of a GUID-less
|
||||
episode at once, a device that never saw the old version can derive a
|
||||
different id.
|
||||
|
||||
## Metadata and bounds
|
||||
|
||||
| Field | Bound |
|
||||
| --- | --- |
|
||||
| titles, author | 1000 characters, non-empty titles |
|
||||
| descriptions | 4000 characters (plain text) |
|
||||
| URLs | 2048 bytes, audio and artwork URLs absolute `http(s)` |
|
||||
| GUID | 2048 bytes |
|
||||
| MIME | 128 printable ASCII bytes |
|
||||
| device, session, operation ids | 1..256 bytes, no control characters |
|
||||
| feed aliases | 8 plus the origin URL |
|
||||
|
||||
`PodcastMeta` (channel), `EpisodeMeta` (episode, including season, episode
|
||||
number, `itunes:episodeType`, publication time and duration in milliseconds,
|
||||
audio URL, MIME, byte length, artwork and link) and `PodcastEpisodeRef`
|
||||
(queue item) validate these bounds. `PodcastMeta::bounded` and
|
||||
`EpisodeMeta::to_ref` truncate publisher data to fit. Receivers drop records
|
||||
that fail validation.
|
||||
|
||||
Episodes without a season are shown in their own group. Unknown
|
||||
`itunes:episodeType` values are `full`.
|
||||
|
||||
## Personal state and merge rules
|
||||
|
||||
All merge functions are commutative, associative and idempotent per entity;
|
||||
ties are broken by a total order over all fields. The same set of operations
|
||||
therefore yields the same state in any delivery order, with duplicates.
|
||||
`hlc_ms` is the sync operation timestamp; it is not a true hybrid logical
|
||||
clock, so ordering of user intent on progress uses a Lamport generation.
|
||||
|
||||
### Subscription
|
||||
|
||||
* Register: `subscribed` + `podcast` metadata, last-writer-wins by
|
||||
`(hlc_ms, op_id)`.
|
||||
* Unsubscribing writes `subscribed = false`. It is state, not a tombstone,
|
||||
and is never compacted, so it beats a stale subscription from a device that
|
||||
returns after being offline.
|
||||
* Unsubscribing stops monitoring but keeps progress and history.
|
||||
Subscribing again continues where the user stopped. Erasing history is a
|
||||
separate explicit action (not part of this version).
|
||||
* Only explicit subscribe and unsubscribe actions write the register. RSS
|
||||
refreshes keep their metadata locally; a background refresh that wrote the
|
||||
register could resurrect an unsubscribed podcast.
|
||||
* Subscribing to a podcast already subscribed locally must not emit a new
|
||||
operation.
|
||||
|
||||
### Share
|
||||
|
||||
* `ShareRecord`, last-writer-wins by `(hlc_ms, op_id)`. `shared = false` is a
|
||||
tombstone that revokes the publication.
|
||||
* Unsubscribing also writes `shared = false`.
|
||||
* `effective_share(subscription, share)` decides whether to publish: the user
|
||||
is subscribed, chose to share, **and** the share decision is newer than the
|
||||
latest subscription write. Every new subscription, including subscribing
|
||||
again or subscribing from someone else's public card, is private until the
|
||||
user shares it again, and a stale share decision cannot republish.
|
||||
|
||||
### Progress
|
||||
|
||||
`ProgressRecord { episode_id, podcast_id, position_ms, status, generation,
|
||||
session_id, hlc_ms, device_id, op_id }`, with `status` one of `not_started`,
|
||||
`in_progress`, `completed`. The winner is the greatest
|
||||
`(generation, hlc_ms, device_id, op_id)`. Progress is never the maximum
|
||||
position: seeking back and "start over" are real changes.
|
||||
|
||||
* Explicit actions use `generation = max_seen + 1`
|
||||
(`ProgressRecord::explicit_generation`): starting a listening session,
|
||||
seeking, "start over", "mark played", "mark unplayed", handing playback to
|
||||
another device. `max_seen` is the generation of the current register,
|
||||
including received records.
|
||||
* Periodic checkpoints and pause/stop saves within a session keep the
|
||||
generation of that session's latest explicit action.
|
||||
* Write progress periodically (every 15–30 s) and on pause, seek, stop,
|
||||
episode change and handoff, in the background. Coalesce pending checkpoints
|
||||
so the outgoing log keeps only the latest per episode.
|
||||
|
||||
| Events | Result |
|
||||
| --- | --- |
|
||||
| A checkpoints session s1 at g3; B seeks back at g4 with a slower clock; A's late g3 checkpoint arrives | B's seek (g4) wins |
|
||||
| A restarts (g3, position 0); a queued g2 checkpoint of the old session arrives later | restart wins |
|
||||
| A and B both saw g5 and listen offline; both start sessions at g6 | the later `hlc_ms` (then `device_id`, `op_id`) wins on every device; both sessions are kept |
|
||||
|
||||
Ordinary Play resumes from `resume_position_ms()` (the position when in
|
||||
progress, otherwise 0). "Start over" is a separate explicit action. Publisher
|
||||
metadata changes never touch progress; if the duration or audio changed, the
|
||||
client keeps the stored position and lets the user decide.
|
||||
|
||||
### Listening sessions and counts
|
||||
|
||||
`ListenSession { session_id, episode_id, podcast_id, device_id,
|
||||
started_at_ms, ended_at_ms, start_position_ms, end_position_ms, listened_ms,
|
||||
completed }` forms a grow-only set keyed by `session_id` (generated at
|
||||
playback start). A device may resend its session with more listening time;
|
||||
versions merge to the greatest `(completed, listened_ms, ended_at_ms, ...)`.
|
||||
|
||||
* A session is recorded in history when `completed` or
|
||||
`listened_ms >= 30 000` (`should_record`).
|
||||
* `completed` is set when the session reaches the end of the stream or
|
||||
`reaches_completion(position_ms, duration_ms)`: within the smaller of 30 s
|
||||
and 5 % of the duration from the end. Completing also writes progress with
|
||||
status `completed` (explicit generation).
|
||||
* The play count of an episode is the number of distinct completed sessions
|
||||
(`completed_listen_count`). Each session counts at most once, so
|
||||
re-delivery never double counts, and listening again is a new session.
|
||||
* "Mark played" only writes progress; it does not create a session or invent
|
||||
listening time.
|
||||
* Concurrent offline sessions are all kept; there is no promise of a single
|
||||
global session during a partition.
|
||||
|
||||
## Trusted-device sync (protocol v4)
|
||||
|
||||
The ALPN remains `furumi/sync/2`; device profiles advertise
|
||||
`protocol_version = 4`. New operation kinds:
|
||||
|
||||
| Kind | Payload | Materialization | Tombstone |
|
||||
| --- | --- | --- | --- |
|
||||
| `podcast_subscription_set` | `podcast_id`, `subscribed`, `podcast`, `feed_aliases` | `SyncOpWire::podcast_subscription` | no |
|
||||
| `podcast_share_set` | `podcast_id`, `shared` | `SyncOpWire::podcast_share` | `shared = false` |
|
||||
| `podcast_progress_set` | `episode_id`, `podcast_id`, `position_ms`, `status`, `generation`, `session_id`, optional `episode` ref | `SyncOpWire::podcast_progress` | no |
|
||||
| `podcast_listen_recorded` | `session`, optional `episode` ref | `SyncOpWire::podcast_listen_session` | no |
|
||||
|
||||
Register stamps come from the operation: `hlc_ms`, `op_id`, and for progress
|
||||
`device_id = origin_device_id`. A session's `device_id` must equal the
|
||||
operation origin. `SyncOpWire::validate_podcast` checks this and the bounds;
|
||||
receivers skip invalid podcast operations but still acknowledge them, so the
|
||||
log keeps moving.
|
||||
|
||||
`DeviceProfileSet` gains an optional `protocol_version`. Each device reports
|
||||
its own version. Receivers store
|
||||
`profile_protocol_version(reported, previously_known)` and relays forward the
|
||||
stored version of other devices in `DeviceProfileWire.protocol_version`. They
|
||||
must never substitute their own constant: version gating would then send
|
||||
podcast state to an old client. A device that never reported a version is
|
||||
treated as version 1.
|
||||
|
||||
### Snapshot recovery
|
||||
|
||||
Sync vectors only grow, so operations hidden from a peer while it ran an
|
||||
older version are never re-requested after it upgrades. Podcast state is
|
||||
therefore part of `SyncSnapshot`:
|
||||
|
||||
| Section | Content | Limit |
|
||||
| --- | --- | --- |
|
||||
| `podcast_subscriptions` | all subscription registers, including unsubscribed | 1000 |
|
||||
| `podcast_shares` | all share registers, including `shared = false` | 1000 |
|
||||
| `podcast_progress` | progress registers | 5000 |
|
||||
| `podcast_listen_sessions` | most recent sessions | 2000 |
|
||||
|
||||
`SyncSnapshot::bound_podcast_state` orders each section newest first, caps it
|
||||
and fits all sections into a 4 MiB JSON budget in the priority order above,
|
||||
leaving room for music state inside the 8 MiB line limit. Records that do not
|
||||
fit still propagate through operations. Receivers ignore podcast sections
|
||||
that exceed the counts (`podcast_state_within_limits`), validate each record
|
||||
and merge it with the same functions as operations. A new or long-absent
|
||||
device thus converges from one snapshot.
|
||||
|
||||
Empty sections are not serialized, so snapshots without podcast state are
|
||||
byte-identical to v3.
|
||||
|
||||
## Queue and playback
|
||||
|
||||
`PlaybackTrack.source: Option<PlaybackSource>` makes the item source
|
||||
explicit: absent or `music` for music, `podcast_episode` with a
|
||||
`PodcastEpisodeRef`, or `private` for a Jam placeholder. Negative ids,
|
||||
placeholder releases and URLs in `file_path` are not part of the contract.
|
||||
`PlaybackTrack::podcast_episode` fills the legacy display fields (episode
|
||||
title, podcast title as artist and release, `id = 0`, empty `file_path`, no
|
||||
`content_id`); receivers read `source`.
|
||||
|
||||
`PodcastEpisodeRef` is self-contained: podcast and episode ids, canonical
|
||||
feed URL, titles, audio URL, MIME, duration, publication time, season,
|
||||
episode number, artwork. A device without the subscription can display,
|
||||
resume and play the item; if the audio URL is stale it refreshes the feed and
|
||||
looks the episode up by id. Mixed queues keep order, position and identity of
|
||||
every item.
|
||||
|
||||
Handoff follows `PLAYBACK_PROTOCOL.md`: record position and play/pause
|
||||
intent, transfer the queue with its sources, the receiver opens the HTTP
|
||||
source and seeks, and ownership moves only through the coordination claim.
|
||||
|
||||
`PlaybackStateWire.playback_rate: Option<f32>` is reserved. No client
|
||||
implements variable speed yet; receivers treat a missing or unsupported value
|
||||
as 1.0. When implemented, position extrapolation and handoff must use it.
|
||||
|
||||
## Version gating for peers below v4
|
||||
|
||||
`SyncOpPayload` and `PlaybackSource` are closed serde enums. A v3 client that
|
||||
receives an unknown `kind` fails to parse the **whole** message, not one
|
||||
operation, which would stall synchronization. Additive `serde(default)`
|
||||
fields alone do not make this compatible. Therefore:
|
||||
|
||||
1. Never send podcast operations to a peer below v4
|
||||
(`op_allowed_for_peer`).
|
||||
2. Never send podcast snapshot sections (`SyncSnapshot::for_peer`) or a
|
||||
playback snapshot whose queue contains podcast items
|
||||
(`playback_allowed_for_peer`) to a peer below v4.
|
||||
3. Refuse a handoff to a device below v4 while the queue contains podcast
|
||||
items and tell the user that the target must be updated. Never strip the
|
||||
episodes to make the queue fit.
|
||||
4. Reject `SetState`/`ActiveChanged` from a peer below v4 while the local
|
||||
queue contains podcast items (`playback_command_allowed_from_peer`): its
|
||||
state would silently drop them.
|
||||
5. Music operations and music-only playback keep working with v1–v3 peers.
|
||||
|
||||
## Jam privacy
|
||||
|
||||
Jam participants are outside the trusted-device group.
|
||||
|
||||
* Hosts send `jam::redact_jam_snapshot` (or `redact_for_jam` on the state):
|
||||
every podcast item becomes a `private` placeholder titled "Private item"
|
||||
without URL, feed, ids, titles or duration; music items, order and position
|
||||
are kept.
|
||||
* Commands containing real podcast episodes are rejected
|
||||
(`jam_command_allowed`): participants cannot add episodes.
|
||||
* Before applying a participant's state, hosts call
|
||||
`restore_jam_private_items`: the k-th placeholder maps to the host's k-th
|
||||
private item. States that drop, duplicate or inject private items are
|
||||
rejected. Participants may still pause, seek and reorder music.
|
||||
|
||||
## Feed parser
|
||||
|
||||
With the `podcast-feed` feature, `music_dht::podcast::feed` parses RSS 2.0,
|
||||
RSS 1.0 and Atom tolerantly: BOM and declared single-byte encodings, HTML
|
||||
entities, CDATA, mismatched end tags, undeclared prefixes, depth and size
|
||||
limits, lenient dates and durations, relative media URLs resolved against
|
||||
the fetch URL. Items without an absolute `http(s)` audio URL are dropped. It
|
||||
extracts `itunes:season`, `itunes:episode`, `itunes:episodeType`,
|
||||
`guid isPermaLink` and `itunes:new-feed-url`, and produces identity hints for
|
||||
`resolve_episode_ids`. It performs no I/O.
|
||||
|
||||
## Versioning
|
||||
|
||||
* `music-dht` 0.6.0 and `furumi-library` 0.3.0: new public types, enum
|
||||
variants and struct fields are breaking for 0.x.
|
||||
* Device-sync protocol 4, ALPN unchanged. Podcast payloads are JSON; no
|
||||
postcard layout changes.
|
||||
* Id domains `furumi-podcast:podcast:v1` and `furumi-podcast:episode:v1` and
|
||||
the `pod1:`/`ep1:` prefixes are frozen. A different derivation needs new
|
||||
domains, new prefixes, a migration and a new protocol version.
|
||||
* Adding an operation kind or a `PlaybackSource` variant requires a new
|
||||
protocol version and gating as above.
|
||||
* Public podcast cards will use a separate signed overlay with its own ALPN
|
||||
and version; the music DHT schema and `federation-net` do not change.
|
||||
@@ -13,7 +13,7 @@ The workspace has three layers:
|
||||
- [`music-dht`](crates/music-dht) builds a Kademlia-style distributed music
|
||||
directory on top of that transport, including catalog discovery, content-id
|
||||
lookup, signed similarity-LSH routing, direct byte streams, and shared Furumi
|
||||
wire types.
|
||||
wire types, including the shared podcast contract ([PODCASTS.md](PODCASTS.md)).
|
||||
- [`furumi-library`](crates/furumi-library) owns the shared local `library.db`
|
||||
contract, migrations, catalog operations, metadata import, artwork,
|
||||
playlists, likes, history, and similarity persistence used by Furumi clients.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "furumi-library"
|
||||
version = "0.2.0"
|
||||
version = "0.3.0"
|
||||
description = "Client-independent SQLite music library for Furumi players"
|
||||
readme = "README.md"
|
||||
documentation = "https://docs.rs/furumi-library"
|
||||
|
||||
@@ -19,3 +19,7 @@ publication, clients can depend on the released crate version instead.
|
||||
`Library::open(default_db_path()?)` opens the same platform-native Furumi
|
||||
library used by every client. Connections use WAL, foreign keys, and a busy
|
||||
timeout so independently running clients can safely share the database.
|
||||
|
||||
Version 0.3 follows `music-dht` 0.6, whose device-sync and podcast contract
|
||||
types it re-exposes. Podcast persistence is not part of this release; it will
|
||||
implement the rules of the workspace `PODCASTS.md`.
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "music-dht"
|
||||
version = "0.5.0"
|
||||
version = "0.6.0"
|
||||
description = "Distributed music library search: a Kademlia-style DHT on top of federation-net"
|
||||
readme = "README.md"
|
||||
documentation = "https://docs.rs/music-dht"
|
||||
|
||||
@@ -185,7 +185,9 @@ new clients should accept both ALPNs and only send `ListenRecorded` when
|
||||
`supports_listen_history(peer.protocol_version)` returns true. Likes,
|
||||
playlists and membership remain compatible with v1. Profile version 3 adds
|
||||
versioned personal playback coordination over the existing JSON sync stream;
|
||||
legacy unfenced playback messages are ignored by upgraded clients.
|
||||
legacy unfenced playback messages are ignored by upgraded clients. Profile
|
||||
version 4 adds podcasts (see below); `op_allowed_for_peer` applies all of
|
||||
these rules to one operation.
|
||||
|
||||
`music_dht::playback` provides the shared pure ownership engine and generic
|
||||
snapshot/command envelopes. Configure startup and failover with `Config`, feed
|
||||
@@ -193,12 +195,64 @@ actual output reports, and persist `Checkpoint` before applying decisions.
|
||||
`Config::passive()` is intended for always-running gateways. See the workspace
|
||||
`PLAYBACK_PROTOCOL.md` for invariants, configuration and migration details.
|
||||
|
||||
## Podcasts
|
||||
|
||||
`music_dht::podcast` is the shared, pure podcast contract (no SQLite, HTTP or
|
||||
clock). Clients use it for identity, merge rules and wire DTOs; see the
|
||||
workspace `PODCASTS.md` for the normative rules.
|
||||
|
||||
```rust
|
||||
use music_dht::podcast::{
|
||||
EpisodeIdentityHints, FeedUrl, PodcastId, ProgressRecord, resolve_episode_ids,
|
||||
};
|
||||
|
||||
let url = FeedUrl::from_user_input("feed://example.com/rss?token=abc")?;
|
||||
let podcast = PodcastId::from_feed_url(&url); // "pod1:..."
|
||||
let hints = [EpisodeIdentityHints {
|
||||
guid: Some("episode-1".into()),
|
||||
..Default::default()
|
||||
}];
|
||||
let ids = resolve_episode_ids(&podcast, None, &hints, &[]); // "ep1:..."
|
||||
|
||||
// Explicit actions (seek, start over, new session) bump the generation.
|
||||
let generation = ProgressRecord::explicit_generation(Some(4)); // 5
|
||||
```
|
||||
|
||||
* `FeedUrl` canonicalizes feed URLs and keeps every query parameter.
|
||||
* `PodcastId`, `EpisodeId`, `EpisodeKey`, `resolve_podcast_id`,
|
||||
`merge_feed_aliases`, `resolve_episode_ids`: identity, aliases and the
|
||||
no-GUID rule.
|
||||
* `PodcastMeta`, `EpisodeMeta`, `PodcastEpisodeRef`: bounded metadata and the
|
||||
self-contained queue item.
|
||||
* `SubscriptionRecord`, `ShareRecord`, `ProgressRecord`, `ListenSession` with
|
||||
`merge_subscription`, `merge_share`, `merge_progress`,
|
||||
`merge_listen_session`, `effective_share`, `reaches_completion` and
|
||||
`completed_listen_count`.
|
||||
|
||||
Enable the `podcast-feed` feature for `music_dht::podcast::feed`, the shared
|
||||
tolerant RSS 2.0 / RSS 1.0 / Atom parser:
|
||||
|
||||
```toml
|
||||
music-dht = { version = "0.6", features = ["podcast-feed"] }
|
||||
```
|
||||
|
||||
Device-sync protocol v4 adds `podcast_subscription_set`, `podcast_share_set`,
|
||||
`podcast_progress_set` and `podcast_listen_recorded`, podcast snapshot
|
||||
sections, `PlaybackTrack.source` and a reserved `playback_rate`. Old clients
|
||||
cannot parse these, so always filter with `op_allowed_for_peer`,
|
||||
`playback_allowed_for_peer`, `SyncSnapshot::for_peer` and
|
||||
`playback_command_allowed_from_peer`, using the protocol version each device
|
||||
reported itself (`profile_protocol_version`). Jam hosts use
|
||||
`jam::redact_jam_snapshot`, `jam_command_allowed` and
|
||||
`restore_jam_private_items`.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
cargo fmt --check
|
||||
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||
cargo test -p music-dht
|
||||
cargo test -p music-dht --features podcast-feed
|
||||
```
|
||||
|
||||
The integration test starts three real nodes in one process (they use Iroh's
|
||||
|
||||
@@ -23,6 +23,10 @@
|
||||
//! * [`similarity_lsh`] turns those embeddings into deterministic compact
|
||||
//! routing summaries, while [`similarity_dht`] stores and discovers signed
|
||||
//! peer summaries without owning model inference or an embedding database.
|
||||
//! * [`podcast`] is the pure, shared podcast contract (canonical feed URLs,
|
||||
//! podcast/episode ids, metadata DTOs and convergent merge rules), with an
|
||||
//! optional RSS/Atom parser behind the `podcast-feed` feature;
|
||||
//! [`device_sync`] protocol v4 carries it between trusted devices.
|
||||
//! * Records are replicated to the `K` nodes whose ids are XOR-closest to
|
||||
//! each key. Publishers pick the targets from their routing table and send
|
||||
//! batched store requests (one pipeline per peer), so even a large library
|
||||
|
||||
Reference in new issue
Block a user