From c6d7848ba4e8bd4f95924a81bf9b621ff5d6be5a Mon Sep 17 00:00:00 2001 From: Sanapach Date: Thu, 19 Nov 2026 21:02:45 +0200 Subject: [PATCH] Document podcast sync between devices and release 0.4.0 README explains that podcast state is private by default, syncs only between trusted devices and streams audio from the publisher, plus handoff, Jam and older devices. ARCHITECTURE, AGENTS and the changelog describe protocol v4, version gating, the podcast store bridge and the invariants (no numeric ids for episodes, versions only from the device itself). Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 28 ++++++++++++++---- ARCHITECTURE.md | 75 ++++++++++++++++++++++++++++++++++++++----------- CHANGELOG.md | 38 +++++++++++++++++++++---- Cargo.lock | 2 +- Cargo.toml | 2 +- README.md | 42 +++++++++++++++++++++++---- 6 files changed, 154 insertions(+), 33 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 89c2eb7..3e2cf2d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -66,10 +66,12 @@ update it only when dependencies change. podcast identity, merge rules and the SQLite store come from FRID (`music_dht::podcast`, `furumi_library::podcasts`) and must not be re-derived here. Queue items are identified by `state::track_key` / - `TrackItem::key`, never by numeric id; music-only code must skip - `TrackItem::is_podcast()` items. + `TrackItem::key`, never by numeric id (every episode has the same sentinel + id); music-only code must skip `TrackItem::is_podcast()` items. - `src/devices.rs` owns trusted-device replication, membership, and playback - coordination. + coordination. Its wire types are the shared `music_dht::device_sync` + definitions; do not add local copies (golden tests in + `devices/wire_compat_tests.rs` keep the v3 JSON). - `src/ui/` renders `AppState`; rendering code must not start background work or mutate persistence. - `src/media.rs` owns OS media controls. @@ -162,7 +164,21 @@ When changing it: - keep snapshots able to repair peers that missed older operations; - treat membership and revocation as replicated durable state; - keep playback commands targeted and deduplicated; -- test merge behavior from at least two operation orders. +- test merge behavior from at least two operation orders; +- never store this build's protocol version for another device: use the + device's own report (`profile_protocol_version`) and only raise a stored + version from relayed profiles (`relayed_protocol_version`); +- gate everything a peer cannot parse by the version it reported: podcast + operations, podcast snapshot sections and playback state with podcast items + go only to v4 peers (`op_allowed_for_peer`, `SyncSnapshot::for_peer`, + `playback_allowed_for_peer`), a podcast queue is never handed to an older + device and its playback commands are ignored while the queue has episodes; + never strip episodes from a queue to make it fit an old peer; +- apply podcast operations and snapshot sections only through + `PodcastStore` (`apply_remote_op`, `apply_snapshot`); never call the + podcast store while holding the sync-log connection; +- keep Jam snapshots redacted (`redact_for_jam`) and map guest placeholders + back with `restore_jam_private_items`. Do not replace eventual reconciliation with assumptions about a continuously connected leader. @@ -228,7 +244,9 @@ Add tests at the closest stable boundary: - pure state transitions in `app/update_tests.rs`; - SQLite behavior and migrations in `library/tests.rs`; - operation merge, tombstone, membership, and playback sync behavior in - `devices/tests.rs`; + `devices/tests.rs`; podcast sync between device stores and simulated v3 + peers in `devices/podcast_sync_tests.rs`; wire shapes in + `devices/wire_compat_tests.rs`; - search/ranking/federation conversion in `federation/tests.rs`; - protocol-specific tests beside `federation/audio.rs` or `federation/catalog.rs` when appropriate. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 1e03885..5819e04 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -66,12 +66,27 @@ This layer connects devices owned or trusted by the same user. It carries personal state such as: - likes and playlist operations; +- podcast subscriptions, episode progress and listening sessions; - device membership and revocation; - acknowledgements and synchronization progress; - playback state, commands, and handoff information. Pairing establishes the trust relationship. After that, changes are replicated through an append-only operation log and applied to materialized local tables. +The wire types are the shared `music_dht::device_sync` definitions (protocol +v4); golden tests keep the v3 JSON of furumi-tui 0.3.x byte for byte. + +Each device stores the protocol version that the other devices report +themselves (their hello and pairing profiles, the sender's own entry of a +`devices` list, their own profile announcements). A relayed profile only +raises a stored version, because older relays list every device with their +own constant; this build never stores its own version for another device. +Everything a peer cannot parse is gated by that version: podcast operations, +podcast snapshot sections and playback state that contains podcast items +reach only v4 peers (`op_allowed_for_peer`, `SyncSnapshot::for_peer`, +`playback_allowed_for_peer`). An older client would reject a whole message +over one unknown operation kind, or silently drop an episode it does not +understand. ### Music federation @@ -97,8 +112,12 @@ audio. Other TUI peers use a dedicated Jam ALPN to submit the same portable playback commands used by connected-device control and receive the host's playback snapshot. They receive queue metadata, not audio. -Jam never exchanges trusted membership, likes, playlists, or listening -history. Volume remains local. Commands carry unique IDs and are retried until +Jam never exchanges trusted membership, likes, playlists, listening history +or podcast state. The host's podcast episodes are private: the published +snapshot replaces them with opaque placeholders (`redact_for_jam`), guest +commands that add episodes are dropped, and a guest's placeholders are mapped +back to the host's episodes (`restore_jam_private_items`); a change that +drops, duplicates or seeks in them is refused. Volume remains local. Commands carry unique IDs and are retried until the host acknowledges them, while inactive participants expire from the runtime session. Regenerating the capability or restarting the host invalidates the previous link. @@ -189,6 +208,12 @@ The synchronization model combines: - per-peer acknowledgements to determine when old tombstones can be compacted; - snapshots to repair a peer that missed older operations. +Vectors only grow, so operations hidden from a peer while it ran an older +protocol are never sent again after it upgrades. Snapshots are the recovery +path for that case; the podcast sections carry the subscription, share and +progress registers and the most recent listening sessions, bounded by the +shared FRID limits and by the line budget of each message. + This is eventual consistency scoped to a trusted device group. A temporarily offline laptop can modify playlists, another device can continue playback, and both can later converge without a permanently available coordinator. @@ -212,7 +237,12 @@ The synchronized state describes the queue, current item, position, pause state, volume, shuffle/repeat mode, and the active playback owner. Commands are targeted and deduplicated. Handoff transfers intent and position; the receiving device resolves the track against its own library or federation sources before -starting its audio engine. +starting its audio engine. Each queue item names its source: a podcast episode +travels as `PlaybackSource::PodcastEpisode` with the reference needed to play +it (the receiving device streams it from the publisher at the handed-off +position, with or without a subscription). A queue with episodes is not handed +to a device below protocol v4, and playback commands from such a device are +ignored while the local queue has episodes. Active-device leases and idle timing prevent stale snapshots from immediately taking control after a device reconnects. This provides practical coordination @@ -296,23 +326,33 @@ The TUI owns what the contract leaves to clients: pool or as a task and reports back through `AppEvent`s; `update::podcasts` holds the pure state transitions. Store open failures (newer schema, foreign file) disable only the Podcasts tab and leave the file untouched. -- **Sync log.** Local actions (subscribe, unsubscribe, mark played) draw their - operation stamps from the device-sync log through +- **Sync log.** Local actions (subscribe, unsubscribe, mark played, progress) + draw their operation stamps from the device-sync log through `DeviceSync::run_podcast_action`, and the operations are appended with exactly those stamps, so the store's registers and the log agree. The lock - order is podcast store, then sync log. Podcast operations are protocol v4 - payloads; until this client speaks sync v4 they stay in the log and are not - sent to peers. + order is podcast store, then sync log; device sync never calls the store + while it holds the log. Only the newest progress operation of an origin per + episode stays in the log (later operations of one origin always win the + register comparison). +- **Device sync.** Received podcast operations are kept in the log (and + relayed) and merged into the store with `PodcastStore::apply_remote_op`; + invalid ones are skipped but acknowledged. Operations that arrive before + the store opens wait in `sync_podcast_pending` and are applied when it is + attached. Snapshot sections are exported with `fill_snapshot` and merged + with `apply_snapshot` after the operations of the same message. The UI + reloads what it shows after such changes and checks the feed of a + subscription that came from another device. - **Queue identity.** An episode is a `TrackItem` with an explicit `podcast` reference (`TrackItem::podcast_episode`); its numeric id is a shared sentinel and never identifies it. Queue keys check the episode id first (`ep:…`), then music content ids, federated sources and library rows. Music-only paths (likes, playlists, history, content-id resolution, federation download, similarity, share links, metadata edits, release and - artist jumps) check `is_podcast()` and skip episodes. Sync protocol 3 - cannot describe an episode, so peers and Jam guests receive the shared - private placeholder, a queue with episodes is not handed off, and remote - `SetState` commands do not replace it. + artist jumps) check `is_podcast()` and skip episodes. On the wire an + episode is `PlaybackSource::PodcastEpisode`; received ones become episodes + again (`TrackItem::from_playback_track`) and the library source mode never + drops them. Jam private placeholders are numbered queue entries that go + back on the wire as `PlaybackSource::Private`. - **Playback.** Starting an episode resolves its current audio URL and resume position from the store, opens `podcasts::stream` (a bounded Range-request reader on its own thread), decodes and positions it with @@ -328,9 +368,12 @@ The TUI owns what the contract leaves to clients: "start over"), checkpoints every 20 s and on pause, explicit seeks, completion (`reaches_completion` or end of stream) and an end with the last seen position on stop, track change, handoff and exit. When the store - reports a newer generation, the playback stops checkpointing until the - user's next explicit action. Listening sessions are recorded at pause, - completion and end; the store keeps those that pass the shared threshold. + reports a newer generation (another device started over, for example), the + playback stops checkpointing until the user's next explicit action. + Listening sessions are recorded at pause, completion and end; the store + keeps those that pass the shared threshold. Queue items carry the latest + progress generation this device saw, so a handoff target starts its + session above it. ## Scripted visualizations @@ -459,7 +502,7 @@ Furumi stores different kinds of state according to their lifetime: | State | Storage | Role | | --- | --- | --- | | Library, playlists, likes, history | SQLite | Durable local source of truth | -| Podcast subscriptions, episodes, progress | SQLite (`podcasts.sqlite3`, FRID store) | Durable personal state and a refreshable episode catalog | +| Podcast subscriptions, episodes, progress | SQLite (`podcasts.sqlite3`, FRID store) | Durable personal state (synced with trusted devices) and a refreshable episode catalog | | Device operation log and replicas | SQLite | Offline synchronization | | Versioned track embeddings | SQLite | Durable, locally rebuildable similarity data | | Federation catalog cache | SQLite/cache | Faster network browsing | diff --git a/CHANGELOG.md b/CHANGELOG.md index 8acdd5c..68bfcdd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.4.0] - unreleased + ### Added - A Podcasts tab on `4`: subscribe to RSS/Atom feeds by URL, browse @@ -18,8 +20,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Podcast data uses the shared FRID store (`furumi_library::podcasts`) in `podcasts.sqlite3`; a database of the earlier podcast test build is migrated in place, and a newer or foreign file is refused without changes. -- Podcast subscription and progress changes are recorded in the trusted-device - sync log with their own stamps. They are not sent to other devices yet. +- Podcasts sync between trusted devices (device-sync protocol 4): + subscriptions, unsubscribing, feed addresses, episode progress ("played", + position, "start over") and listening sessions. A podcast subscribed on one + device appears on the others without its URL; each device checks the feed + itself. Podcast operations are applied to the podcast store, kept in the + sync log and relayed; snapshots carry the podcast state (bounded) so new + devices and devices that upgrade from an older version catch up. Periodic + progress saves are coalesced in the log. +- A queue with podcast episodes is handed to other devices with the episode, + its position and pause; the receiving device does not need the + subscription. - Podcast episodes play from the Podcasts tab and the shared queue (mixed with music): audio streams from the publisher through a bounded HTTP Range buffer, Play continues from the saved position, `b` starts over, and the @@ -37,9 +48,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Queue items are keyed by their source: podcast episodes by episode id, library rows as `lib:` instead of `local:`. Likes, playlists, music history, similarity, share links and metadata edits skip episodes. -- Peers and Jam guests see podcast episodes as "Private item" (sync - protocol 3 cannot describe them); a queue with episodes is not handed to - another device and remote queue changes do not replace it. +- Device sync speaks protocol 4 and uses the shared `music_dht::device_sync` + wire types instead of private copies; v3 messages are unchanged byte for + byte (golden tests). Devices below protocol 4 never receive podcast + operations, podcast snapshot sections or a queue with episodes: handing + such a queue to them is refused ("needs a Furumi update"), and their + playback commands are ignored while the queue has episodes. +- Jam guests see the host's podcast episodes as "Private item" (no title, + feed or address). Guest changes that add, drop or duplicate them, or seek + in them, are refused; a guest leaving a Jam drops the placeholders. +- Controlling another v4 device may queue and play episodes there; an + episode started by another device's command at position 0 resumes from its + saved position. - Tab order is now `1` Library, `2` Playlists, `3` Queue, `4` Podcasts, `5` Settings, `6` Logs, for the tab bar, Tab/Shift+Tab, and the default number keys. @@ -56,6 +76,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - The help window names the target tab for number-key shortcuts ("Go to Settings") instead of "Go to tab N". +### Fixed + +- The protocol version stored for another device came from this build's own + constant (on profile announcements and accepted pairings), not from the + device. Versions now come from the device's own reports; relayed profiles + only raise them. Invites and profile announcements carry the version, and + the first start of a new version announces it. + ## [0.2.8] - 2026-09-02 ### Added diff --git a/Cargo.lock b/Cargo.lock index 494da1e..ea18f58 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1711,7 +1711,7 @@ dependencies = [ [[package]] name = "furumi_tui" -version = "0.3.5" +version = "0.4.0" dependencies = [ "anyhow", "base64", diff --git a/Cargo.toml b/Cargo.toml index 0da4f4e..f071f79 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "furumi_tui" -version = "0.3.5" +version = "0.4.0" edition = "2024" rust-version = "1.97" description = "A federated P2P player for personal music libraries" diff --git a/README.md b/README.md index 7edcda4..ef06887 100644 --- a/README.md +++ b/README.md @@ -231,12 +231,39 @@ Episodes play in the same queue as music: - Likes, playlists, music history, similarity search and share links are for music only and skip episodes; `Shift+J` on an episode opens its podcast. -Episodes play on this device only for now: a queue with episodes cannot be -handed to another device, and other devices or Jam guests see episodes as -"Private item". Podcast data lives in `podcasts.sqlite3` in the platform data directory (the +Podcasts and your connected devices: + +- **Private by default.** Subscriptions, episode progress and listening + history are personal. They are synchronized only between your trusted + devices (Connected Devices), never published to the federation: no DHT + record, public catalog or search result contains them. Publishing a + podcast for discovery is a separate, explicit choice that this version + does not offer yet. +- **Synchronized between trusted devices.** A podcast you subscribe to on + one device appears on the others without typing its URL again; each + device then checks the feed itself. Unsubscribing, "played", the saved + position and "start over" follow the same way, also through a device + that relays them and after a device was offline for a while. When two + devices played the same episode offline, they agree on one position and + keep both listening sessions. +- **Handoff.** A queue with episodes moves to another device like music: + the episode continues there at the handed-off position, paused or + playing, even if that device is not subscribed. Devices that run an older + Furumi (device sync before v4) cannot keep episodes: handing such a queue + to them is refused with "needs a Furumi update", their remote control is + ignored while the queue has episodes, and they get no podcast data. +- **Audio from the publisher.** Every device streams the episode from the + publisher's server itself. Furumi peers never pass podcast audio to each + other, and nothing is kept after playback. +- **Jam.** Guests see the host's episodes as "Private item" (no title, feed + or address), cannot add episodes to the host's queue and cannot seek in a + private item. They can still reorder the queue and skip. + +Podcast data lives in `podcasts.sqlite3` in the platform data directory (the same file the earlier podcast test build used; it is upgraded in place). If that file was written by a newer Furumi or is not a podcast database, the tab -explains why it is unavailable and the file is left unchanged. +explains why it is unavailable and the file is left unchanged; podcast +changes from other devices then wait in the device-sync log until it opens. ### Now playing in tmux @@ -325,11 +352,16 @@ Run the local protocol checks from the TUI repository: ```bash cargo test localhost_devices_exchange_state_and_handoff +cargo test localhost_devices_sync_podcasts +cargo test devices::podcast_sync_tests python scripts/test_device_interop.py ../furumusic ``` The first test uses real iroh streams, isolated SQLite databases, ownership -handoff, and a stalled peer. The second builds both player test binaries and +handoff, and a stalled peer. The podcast tests sync subscriptions, progress +and "start over" between TUI device stores (over iroh streams and through +the JSON messages), check a handoff with an episode and simulate a device +sync v3 peer with the exact JSON types of furumi-tui 0.3.x. The second builds both player test binaries and exchanges their actual JSON messages over `127.0.0.1`: queue metadata, bidirectional handoff, pause/seek and duplicate command fencing. It exercises the TUI command adapter and web player hub, but does not start a browser or