Document the SQLite podcast store

PODCASTS.md gains the "SQLite reference store" section (migrations,
prototype import, tables, invariants); ARCHITECTURE.md, the
furumi-library README and the changelog describe PodcastStore and
TrackItem.podcast.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
sanapachandClaude Opus 5.5 committed 2026-11-19 16:42:58 +02:00
1 parent e22c470ea7
commit 49ae6423f6
4 files changed
+170 -4

No files matched your search

+16 -1
View File
@@ -12,6 +12,7 @@ application
furumi-library
├── local catalog, import, migrations
├── playlists, likes, history, similarity persistence
├── podcast store (podcasts.sqlite3)
│
music-dht
├── Kademlia routing and iterative lookup
@@ -41,9 +42,23 @@ metadata import, artwork resolution, playlists, likes, listening history, and
similarity index storage. Connections enable foreign keys, WAL, and a bounded
busy timeout so independently running clients can safely share the database.
Podcasts use a separate store, `furumi_library::podcasts::PodcastStore`, in
its own `podcasts.sqlite3` file next to the library database. It implements
the podcast contract of `music_dht::podcast` for SQLite clients: the episode
catalog refreshed from RSS, the subscription, share and progress registers,
grow-only listening sessions, feed aliases and local-only fetch state.
Local actions, received operations and snapshot sections share one apply
path built on the `music-dht` materializers and merge functions, so the store
converges independently of delivery order. Migrations follow
`PRAGMA user_version`; the prototype podcast database is imported and its
tables are kept as `legacy_*`; newer or foreign files are refused unchanged.
`TrackItem.podcast` marks episode queue items, which music-only code must
skip. See the "SQLite reference store" section of [PODCASTS.md](PODCASTS.md).
Filesystem scanning schedules, playback, federation lifecycle, and UI state
remain application responsibilities. Database and metadata work must stay off
interactive and async reactor threads.
interactive and async reactor threads. RSS fetching and refresh scheduling are
application work as well.
## Transport: federation-net
+14
View File
@@ -18,6 +18,20 @@
Jam redaction. New struct fields and enum variants break exhaustive
construction and matching in clients.
- `furumi-library 0.3.0`: follows `music-dht 0.6.0`, whose types it re-exposes.
- `furumi-library 0.3.0`: new `furumi_library::podcasts::PodcastStore`, the
SQLite reference store of the podcast contract in its own
`podcasts.sqlite3`: `user_version` migrations that import the prototype
podcast database (tables kept as `legacy_*`, unimportable rows in
`legacy_quarantine`) and refuse newer or foreign files unchanged; podcasts,
feed aliases, seasons, episodes, subscription/share/progress registers,
listening sessions and local-only fetch state; local actions that return
the `podcast_*` operations for the client's sync log; `apply_remote_op`,
bounded snapshot export and merge; RSS refresh ingestion with stable
episode ids; UI queries. Enables the `music-dht` `podcast-feed` feature.
- `furumi-library 0.3.0` (breaking): `TrackItem.podcast:
Option<PodcastEpisodeRef>` and `TrackItem::podcast_episode`/`is_podcast`.
Exhaustive `TrackItem` literals in clients need the new field; music-only
code must skip podcast items. The library database is unchanged.
- See `PODCASTS.md` for the normative contract.
## v0.4.1 — 2026-09-02
+96 -1
View File
@@ -19,7 +19,7 @@ of this version of the contract; it will be specified with the overlay.
| --- | --- |
| `music_dht::podcast` | ids, URL canonicalization, no-GUID rule, metadata DTOs and bounds, merge functions, RSS/Atom parser (feature `podcast-feed`) |
| `music_dht::device_sync` | sync operation kinds, snapshot sections, `PlaybackSource`, version gating, Jam redaction |
| Client storage (`furumi-library`, web PostgreSQL) | persistence and migrations implementing the rules below |
| Client storage (`furumi_library::podcasts::PodcastStore`, web PostgreSQL) | persistence and migrations implementing the rules below (SQLite reference: see "SQLite reference store") |
| Client runtime | HTTP fetching, conditional requests, scheduling, audio buffering and decoding, UI |
Audio is always fetched from the publisher. It is never imported into the
@@ -511,6 +511,101 @@ extracts `itunes:season`, `itunes:episode`, `itunes:episodeType`,
Links are kept verbatim (trimmed only) because they are identity material.
Text cleanup is linear in the input size. It performs no I/O.
## SQLite reference store
`furumi_library::podcasts::PodcastStore` (`furumi-library` 0.3) implements
this contract for SQLite clients. Other stores (the web client's PostgreSQL)
must keep the same invariants.
### File and migrations
* Own file, conventionally `podcasts.sqlite3` next to `library.db`; the caller
passes the path. WAL, foreign keys, 5 s busy timeout.
* `PRAGMA user_version` drives migrations. Each runs in a `BEGIN IMMEDIATE`
transaction that re-reads the version, so concurrent openers and
interrupted upgrades are safe and reopening is idempotent.
| Version | Content |
| --- | --- |
| 0 | empty file |
| 1 | prototype schema of the original podcast PR (`podcasts`, `podcast_episodes`), verbatim |
| 2 | the model below; the prototype tables are renamed and imported |
* Before writing anything (including the WAL switch), the store refuses a
file with a `user_version` above its own, and a file whose tables are not
the expected ones for its version (for example `library.db` passed by
mistake). The file is left byte-for-byte unchanged.
* Prototype import (migration 2). Tables are renamed `legacy_podcasts` and
`legacy_podcast_episodes`, never dropped.
* Each feed becomes a podcast with `podcast_id` from the canonical feed URL
(`FeedUrl::parse`, then `FeedUrl::from_user_input`) and a subscription
register `subscribed = true`, generation 0, `hlc_ms` = `added_at`,
`op_id` = `legacy-import:<podcast_id>`, not shared. Every contract write
orders after it; other devices learn it through the snapshot sections.
Rows with the same canonical URL merge (first row wins).
* URLs that cannot be canonicalized, episodes with a non-`http(s)` audio URL,
invalid metadata or no identity are listed in `legacy_quarantine`; the
rows stay in the `legacy_*` tables.
* Episode ids come from `resolve_episode_ids` over the stored items. The
prototype stored the enclosure URL as `guid` when an item had none: such
rows are imported as GUID-less items and keep the value in
`episodes.legacy_guid`. When a later refresh shows that value as a real
GUID, the episode is re-keyed to its GUID id if it has no progress and no
session (otherwise the GUID item becomes a new episode).
* ETag, Last-Modified, check time, last error and the cover cache move to the
fetch state. The prototype stores no progress or history, so none is
created.
### Tables (version 2)
| Table | Key | Content | Synced |
| --- | --- | --- | --- |
| `podcasts` | `podcast_id` | local channel metadata, `feed_url` = URL this device fetches, `meta_from_feed` | catalog, no |
| `podcast_feed_aliases` | `(podcast_id, feed_url)` | bounded grow-only alias set (`merge_feed_aliases`) | yes, inside subscription records and `podcast_feed_aliases_added` |
| `episodes` | `episode_id` | `EpisodeMeta` columns (season, number, type, published, duration, audio URL, MIME, length, artwork, link), `source` (`feed`/`sync`), `in_feed`, `legacy_guid` | catalog, no |
| `podcast_seasons` | `(podcast_id, season)` | derived from `episodes`: episode count, first/last publication; `season` NULL groups episodes without a season | no |
| `podcast_subscriptions` | `podcast_id` | subscription register: `subscribed`, `PodcastMeta` JSON, `generation`, `hlc_ms`, `op_id` | yes |
| `podcast_shares` | `podcast_id` | share register | yes |
| `episode_progress` | `episode_id` | progress register (`ProgressRecord`) | yes |
| `podcast_listen_sessions` | `session_id` | grow-only set of `ListenSession` | yes |
| `podcast_fetch_state` | `podcast_id` | ETag, Last-Modified, checked / last success / next check times, last error, consecutive failures, cover cache | **never** |
| `legacy_quarantine` | `(legacy_table, legacy_id)` | prototype rows that could not be imported | no |
### Invariants
* Registers and sessions have no foreign keys to the catalog: operations for
podcasts or episodes the device has not fetched are stored as they arrive,
in any order. An operation's `PodcastEpisodeRef` adds a `source = 'sync'`
catalog row only when the episode is unknown; feed metadata always wins.
* One apply path. Local actions validate the payload, draw a stamp from the
client's sync log, build the `SyncOpWire`, and apply it like a received
operation: `SyncOpWire::podcast_*` materializers, then `merge_*`. The
returned operation must be appended with exactly that stamp. Snapshot
records go through the same functions after validation. The result is
therefore commutative, associative and idempotent, and a snapshot never
replaces newer local state.
* Invalid operations (`validate_podcast`, a generation above `i64::MAX`, a
session whose identity differs from the stored one) are skipped and
reported as rejected, so the client can still acknowledge them.
* Only `subscribe`/`unsubscribe` write the subscription register and only
`set_shared`/`unsubscribe` the share register, with
`next_podcast_generation`. No-op actions draw no stamp and emit nothing.
`unsubscribe` emits `subscribed = false` and `shared = false` and keeps
progress, sessions and the catalog.
* Explicit progress actions use `explicit_generation(max(local, carried))`;
checkpoints keep the session generation and are refused (`Superseded`)
once the register holds a greater one, or after the session completed.
* RSS refreshes update the catalog (channel metadata, episode metadata and
audio URLs, `in_feed`) and the alias set only; episodes missing from the
feed are kept. Permanent redirects and `itunes:new-feed-url` become the
fetch URL and aliases, and emit `podcast_feed_aliases_added` only when the
stored alias set changes.
* Snapshot export reads each section newest first up to its count, then
applies `bound_podcast_state_to(budget)`. Fetch state and the catalog
(descriptions, episodes) are never exported.
* Feed URLs resolve through `resolve_podcast_id` over the alias set, the
catalog fetch URL and the register's `podcast.feed_url`.
## Versioning
* `music-dht` 0.6.0 and `furumi-library` 0.3.0: new public types, enum
+44 -2
View File
@@ -21,5 +21,47 @@ library used by every client. Connections use WAL, foreign keys, and a busy
timeout so independently running clients can safely share the database.
Version 0.3 follows `music-dht` 0.6, whose device-sync and podcast contract
types it re-exposes. Podcast persistence is not part of this release; it will
implement the rules of the workspace `PODCASTS.md`.
types it re-exposes.
## Podcasts
`furumi_library::podcasts::PodcastStore` is the SQLite reference store of the
podcast contract (workspace `PODCASTS.md`, section "SQLite reference store").
It lives in its own file, conventionally `podcasts.sqlite3`
(`PODCAST_DB_FILE`) next to `library.db`; the caller passes the path.
- **Migrations** run on `PRAGMA user_version`, each in its own transaction.
A database of the original podcast prototype (`user_version` 1) is imported:
its tables are renamed `legacy_podcasts` / `legacy_podcast_episodes` (never
dropped), feeds become subscriptions with contract ids, rows that cannot be
imported are listed in `legacy_quarantine`. A database from a newer Furumi,
or a file that is not a podcast database, is refused without being touched.
- **Local actions** (`subscribe`, `unsubscribe`, `set_shared`,
`start_session`, `checkpoint`, `seek`, `restart`, `complete_session`,
`mark_played`, `mark_unplayed`, `record_listen_session`) draw stamps from a
`StampSource` (usually the client's sync log), apply the operation and
return `PodcastLocalOp`s to append to the log with exactly those stamps.
- **Sync**: `apply_remote_op` applies the five `podcast_*` operations,
`fill_snapshot` / `export_snapshot` produce the bounded snapshot sections
and `apply_snapshot` merges them. Everything goes through the `music-dht`
materializers and merge functions, so the result does not depend on
delivery order and never loses newer local state.
- **RSS**: `apply_feed` ingests a document parsed by
`music_dht::podcast::feed` (the crate enables the `podcast-feed` feature):
channel metadata, episodes with stable ids and current audio URLs, feed
aliases from permanent redirects and `itunes:new-feed-url`. It never
touches subscriptions, progress or history. HTTP, conditional requests and
scheduling stay in the client; `record_fetch_*`, `fetch_state` and
`podcasts_due_for_refresh` keep the local-only fetch state, which is never
synced.
- **Queries**: `list_podcasts`, `seasons`, `episodes` (season, played-state
and text filters, date or episode-number order), `resume_position_ms`,
`episode_ref` / `episode_track` for queueing, `listen_history`.
The store is synchronous: methods take `&self` and lock one connection, so
`Arc<PodcastStore>` can be shared; run calls on a blocking pool.
`TrackItem.podcast` marks a queue item as a podcast episode
(`TrackItem::podcast_episode`). Music-only code (likes, playlists, history,
content ids, federation downloads) must check `TrackItem::is_podcast`; the
music library never stores episodes.