Merge pull request 'Подкасты в FRID: SQLite-хранилище PodcastStore, импорт прототипа и TrackItem.podcast (MR-F2)' (#2) from sanapach/frid:podcasts/store into main
CI / check (push) Successful in 1m14s
CI / check (push) Successful in 1m14s
This commit was merged in pull request #2.
This commit is contained in:
commit
3046c3f924
15 files changed
+6735
-7
No files matched your search
+23
-1
@@ -12,6 +12,7 @@ application
|
||||
furumi-library
|
||||
├── local catalog, import, migrations
|
||||
├── playlists, likes, history, similarity persistence
|
||||
├── podcast store (podcasts.sqlite3)
|
||||
│
|
||||
music-dht
|
||||
├── Kademlia routing and iterative lookup
|
||||
@@ -41,9 +42,30 @@ metadata import, artwork resolution, playlists, likes, listening history, and
|
||||
similarity index storage. Connections enable foreign keys, WAL, and a bounded
|
||||
busy timeout so independently running clients can safely share the database.
|
||||
|
||||
Podcasts use a separate store, `furumi_library::podcasts::PodcastStore`, in
|
||||
its own `podcasts.sqlite3` file next to the library database. It implements
|
||||
the podcast contract of `music_dht::podcast` for SQLite clients: the episode
|
||||
catalog refreshed from RSS, the subscription, share and progress registers,
|
||||
grow-only listening sessions, feed aliases and local-only fetch state.
|
||||
Local actions, received operations and snapshot sections share one apply
|
||||
path built on the `music-dht` materializers and merge functions, so the store
|
||||
converges independently of delivery order. Migrations follow
|
||||
`PRAGMA user_version`; the prototype podcast database is imported and its
|
||||
tables are kept as `legacy_*`; newer or foreign files are refused unchanged.
|
||||
`TrackItem.podcast` marks episode queue items, which music-only code must
|
||||
skip. An episode queue item is identified by its episode id
|
||||
(`TrackItem::key`, a typed source identity: library row, federated owner and
|
||||
item id, or episode id), never by the numeric id: episodes use the fixed
|
||||
sentinel id, `release_id` and artist id -1, which carries no identity and
|
||||
only keeps code written for library tracks (`id >= 0`) away from them.
|
||||
Received playback items are converted with `TrackItem::from_playback_track`,
|
||||
which never copies the wire ids (0 for episodes and private placeholders). See the "SQLite reference store" section
|
||||
of [PODCASTS.md](PODCASTS.md).
|
||||
|
||||
Filesystem scanning schedules, playback, federation lifecycle, and UI state
|
||||
remain application responsibilities. Database and metadata work must stay off
|
||||
interactive and async reactor threads.
|
||||
interactive and async reactor threads. RSS fetching and refresh scheduling are
|
||||
application work as well.
|
||||
|
||||
## Transport: federation-net
|
||||
|
||||
|
||||
@@ -18,6 +18,37 @@
|
||||
Jam redaction. New struct fields and enum variants break exhaustive
|
||||
construction and matching in clients.
|
||||
- `furumi-library 0.3.0`: follows `music-dht 0.6.0`, whose types it re-exposes.
|
||||
- `furumi-library 0.3.0`: new `furumi_library::podcasts::PodcastStore`, the
|
||||
SQLite reference store of the podcast contract in its own
|
||||
`podcasts.sqlite3`: `user_version` migrations that import the prototype
|
||||
podcast database (tables kept as `legacy_*`, unimportable rows in
|
||||
`legacy_quarantine`) and refuse newer or foreign files unchanged; podcasts,
|
||||
feed aliases, seasons, episodes, subscription/share/progress registers,
|
||||
listening sessions and local-only fetch state; local actions that return
|
||||
the `podcast_*` operations for the client's sync log; `apply_remote_op`,
|
||||
bounded snapshot export and merge; RSS refresh ingestion with stable
|
||||
episode ids; UI queries. Enables the `music-dht` `podcast-feed` feature.
|
||||
- `furumi-library 0.3.0` (breaking): `TrackItem.podcast:
|
||||
Option<PodcastEpisodeRef>` and `TrackItem::podcast_episode`/`is_podcast`.
|
||||
Exhaustive `TrackItem` literals in clients need the new field; music-only
|
||||
code must skip podcast items. Podcast items use the fixed sentinel
|
||||
`TrackItem::NOT_A_LIBRARY_ROW` (-1) as `id`, `release_id` and artist id,
|
||||
so existing `id < 0` / `release_id < 0` checks already treat them as
|
||||
non-library items; the sentinel carries no identity. New
|
||||
`TrackItem::key` / `TrackItemKey` (`lib:<id>`, `fed:<owner>:<item_id>`,
|
||||
`ep:<episode id>`), the source identity: episodes must be keyed by it (or
|
||||
`TrackItem.podcast`), never by `id`; content-first music matching stays
|
||||
as it is. New `TrackItem::from_playback_track` / `PlaybackItem` convert
|
||||
received playback items without copying wire ids (0 for episodes and Jam
|
||||
private placeholders). The library database is unchanged.
|
||||
- `furumi-library 0.3.0`: prototype import collapses rows of one GUID-less
|
||||
item under several enclosure URLs, and the first refresh re-keys imported
|
||||
GUID-less episodes without personal state to the ids the current feed
|
||||
gives, so they match devices that never had the prototype; when another
|
||||
episode (for example one created by a remote operation) already holds
|
||||
that id, the imported copy is deleted instead of kept as a duplicate.
|
||||
Fetch-state
|
||||
writes are atomic across threads and processes.
|
||||
- See `PODCASTS.md` for the normative contract.
|
||||
|
||||
## v0.4.1 — 2026-09-02
|
||||
|
||||
Generated
+1
@@ -942,6 +942,7 @@ dependencies = [
|
||||
"rusqlite",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"tempfile",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
|
||||
+143
-1
@@ -19,7 +19,7 @@ of this version of the contract; it will be specified with the overlay.
|
||||
| --- | --- |
|
||||
| `music_dht::podcast` | ids, URL canonicalization, no-GUID rule, metadata DTOs and bounds, merge functions, RSS/Atom parser (feature `podcast-feed`) |
|
||||
| `music_dht::device_sync` | sync operation kinds, snapshot sections, `PlaybackSource`, version gating, Jam redaction |
|
||||
| Client storage (`furumi-library`, web PostgreSQL) | persistence and migrations implementing the rules below |
|
||||
| Client storage (`furumi_library::podcasts::PodcastStore`, web PostgreSQL) | persistence and migrations implementing the rules below (SQLite reference: see "SQLite reference store") |
|
||||
| Client runtime | HTTP fetching, conditional requests, scheduling, audio buffering and decoding, UI |
|
||||
|
||||
Audio is always fetched from the publisher. It is never imported into the
|
||||
@@ -511,6 +511,148 @@ extracts `itunes:season`, `itunes:episode`, `itunes:episodeType`,
|
||||
Links are kept verbatim (trimmed only) because they are identity material.
|
||||
Text cleanup is linear in the input size. It performs no I/O.
|
||||
|
||||
## SQLite reference store
|
||||
|
||||
`furumi_library::podcasts::PodcastStore` (`furumi-library` 0.3) implements
|
||||
this contract for SQLite clients. Other stores (the web client's PostgreSQL)
|
||||
must keep the same invariants.
|
||||
|
||||
### File and migrations
|
||||
|
||||
* Own file, conventionally `podcasts.sqlite3` next to `library.db`; the caller
|
||||
passes the path. WAL, foreign keys, 5 s busy timeout.
|
||||
* `PRAGMA user_version` drives migrations. Each runs in a `BEGIN IMMEDIATE`
|
||||
transaction that re-reads the version, so concurrent openers and
|
||||
interrupted upgrades are safe and reopening is idempotent.
|
||||
|
||||
| Version | Content |
|
||||
| --- | --- |
|
||||
| 0 | empty file |
|
||||
| 1 | prototype schema of the original podcast PR (`podcasts`, `podcast_episodes`), verbatim |
|
||||
| 2 | the model below; the prototype tables are renamed and imported |
|
||||
|
||||
* Before writing anything (including the WAL switch), the store refuses a
|
||||
file with a `user_version` above its own, and a file whose tables are not
|
||||
the expected ones for its version (for example `library.db` passed by
|
||||
mistake). The file is left byte-for-byte unchanged.
|
||||
* Prototype import (migration 2). Tables are renamed `legacy_podcasts` and
|
||||
`legacy_podcast_episodes`, never dropped.
|
||||
* Each feed becomes a podcast with `podcast_id` from the canonical feed URL
|
||||
(`FeedUrl::parse`, then `FeedUrl::from_user_input`) and a subscription
|
||||
register `subscribed = true`, generation 0, `hlc_ms` = `added_at`,
|
||||
`op_id` = `legacy-import:<podcast_id>`, not shared. Every contract write
|
||||
orders after it; other devices learn it through the snapshot sections.
|
||||
Rows with the same canonical URL merge (first row wins).
|
||||
* URLs that cannot be canonicalized, episodes with a non-`http(s)` audio URL,
|
||||
invalid metadata or no identity are listed in `legacy_quarantine`; the
|
||||
rows stay in the `legacy_*` tables.
|
||||
* Episode ids come from `resolve_episode_ids` over the stored items. The
|
||||
prototype stored the enclosure URL as `guid` when an item had none: such
|
||||
rows are imported as GUID-less items and keep the value in
|
||||
`episodes.legacy_guid`. The prototype kept one row per enclosure URL, so
|
||||
rows with the same publication second, normalized title and `<link>`
|
||||
(or none) are one item whose audio URL moved: the newest row
|
||||
(`first_seen_at`, then `id`) is imported, the others are quarantined as
|
||||
duplicates.
|
||||
* Imported GUID-less episodes are **provisional**: stale rows can still
|
||||
change which key is unique among the stored items, so their ids may
|
||||
differ from what a device without the prototype derives. On a refresh,
|
||||
item ids are resolved against the stored episodes except provisional
|
||||
ones without progress or sessions; each such provisional episode is
|
||||
matched to its item (enclosure, then link, then date and title, GUIDs
|
||||
ignored) and re-keyed to that item's id. When another episode that
|
||||
stays already holds that id (for example one a remote operation created
|
||||
before this device's first refresh), it is the item's episode and the
|
||||
provisional one is a duplicate: it is deleted rather than kept as a
|
||||
second copy. The same applies when `legacy_guid` turns out to be the
|
||||
item's real GUID. Listing an episode clears `legacy_guid`, which makes
|
||||
its id final. Provisional episodes carry no personal state, so nothing
|
||||
personal moves or is lost; an imported episode played before its first
|
||||
refresh is not provisional and keeps its id (the feed item then becomes
|
||||
a separate episode if the ids differ).
|
||||
* ETag, Last-Modified, check time, last error and the cover cache move to the
|
||||
fetch state. The prototype stores no progress or history, so none is
|
||||
created.
|
||||
|
||||
### Tables (version 2)
|
||||
|
||||
| Table | Key | Content | Synced |
|
||||
| --- | --- | --- | --- |
|
||||
| `podcasts` | `podcast_id` | local channel metadata, `feed_url` = URL this device fetches, `meta_from_feed` | catalog, no |
|
||||
| `podcast_feed_aliases` | `(podcast_id, feed_url)` | bounded grow-only alias set (`merge_feed_aliases`) | yes, inside subscription records and `podcast_feed_aliases_added` |
|
||||
| `episodes` | `episode_id` | `EpisodeMeta` columns (season, number, type, published, duration, audio URL, MIME, length, artwork, link), `source` (`feed`/`sync`), `in_feed`, `legacy_guid` | catalog, no |
|
||||
| `podcast_seasons` | `(podcast_id, season)` | derived from `episodes`: episode count, first/last publication; `season` NULL groups episodes without a season | no |
|
||||
| `podcast_subscriptions` | `podcast_id` | subscription register: `subscribed`, `PodcastMeta` JSON, `generation`, `hlc_ms`, `op_id` | yes |
|
||||
| `podcast_shares` | `podcast_id` | share register | yes |
|
||||
| `episode_progress` | `episode_id` | progress register (`ProgressRecord`) | yes |
|
||||
| `podcast_listen_sessions` | `session_id` | grow-only set of `ListenSession` | yes |
|
||||
| `podcast_fetch_state` | `podcast_id` | ETag, Last-Modified, checked / last success / next check times, last error, consecutive failures, cover cache | **never** |
|
||||
| `legacy_quarantine` | `(legacy_table, legacy_id)` | prototype rows that could not be imported | no |
|
||||
|
||||
### Invariants
|
||||
|
||||
* Registers and sessions have no foreign keys to the catalog: operations for
|
||||
podcasts or episodes the device has not fetched are stored as they arrive,
|
||||
in any order. An operation's `PodcastEpisodeRef` adds a `source = 'sync'`
|
||||
catalog row only when the episode is unknown; feed metadata always wins.
|
||||
* One apply path. Local actions validate the payload, draw a stamp from the
|
||||
client's sync log, build the `SyncOpWire`, and apply it like a received
|
||||
operation: `SyncOpWire::podcast_*` materializers, then `merge_*`. The
|
||||
returned operation must be appended with exactly that stamp. Snapshot
|
||||
records go through the same functions after validation. The result is
|
||||
therefore commutative, associative and idempotent, and a snapshot never
|
||||
replaces newer local state.
|
||||
* Invalid operations (`validate_podcast`, a generation above `i64::MAX`, a
|
||||
session whose identity differs from the stored one) are skipped and
|
||||
reported as rejected, so the client can still acknowledge them.
|
||||
* Only `subscribe`/`unsubscribe` write the subscription register and only
|
||||
`set_shared`/`unsubscribe` the share register, with
|
||||
`next_podcast_generation`. No-op actions draw no stamp and emit nothing.
|
||||
`unsubscribe` emits `subscribed = false` and `shared = false` and keeps
|
||||
progress, sessions and the catalog.
|
||||
* Explicit progress actions use `explicit_generation(max(local, carried))`;
|
||||
checkpoints keep the session generation and are refused (`Superseded`)
|
||||
once the register holds a greater one, or after the session completed.
|
||||
* RSS refreshes update the catalog (channel metadata, episode metadata and
|
||||
audio URLs, `in_feed`) and the alias set only; episodes missing from the
|
||||
feed are kept. Permanent redirects and `itunes:new-feed-url` become the
|
||||
fetch URL and aliases, and emit `podcast_feed_aliases_added` only when the
|
||||
stored alias set changes.
|
||||
* Snapshot export reads each section newest first up to its count, then
|
||||
applies `bound_podcast_state_to(budget)`. Fetch state and the catalog
|
||||
(descriptions, episodes) are never exported.
|
||||
* Feed URLs resolve through `resolve_podcast_id` over the alias set, the
|
||||
catalog fetch URL and the register's `podcast.feed_url`.
|
||||
* Fetch-state writes (`record_fetch_*`, `set_cover_cache`) read and write
|
||||
the row in one immediate transaction, so concurrent writers on one or
|
||||
several connections never lose a column or a failure increment.
|
||||
* `TrackItem::podcast_episode` builds queue items whose identity is the
|
||||
explicit source `TrackItem.podcast` alone. Their `id`, `release_id` and
|
||||
artist id are the fixed sentinel `TrackItem::NOT_A_LIBRARY_ROW` (-1),
|
||||
shared by every episode, the same "not a library row" convention as
|
||||
pending federated tracks. The sentinel carries no identity and is never
|
||||
decoded; it only keeps music-only code that predates podcasts
|
||||
(`id >= 0`, `release_id < 0` checks) from taking an episode for library
|
||||
track, release or artist 0. There is no reserved range of negative ids.
|
||||
* `TrackItem::key` (a typed `TrackItemKey`; string form `lib:<id>`,
|
||||
`fed:<owner>:<item_id>`, `ep:<episode id>`) is the source identity of a
|
||||
queue item; on the wire it is `PlaybackSource`. An episode is identified
|
||||
by `TrackItem.podcast` / `ep:` only and must never be keyed by its numeric
|
||||
id: keying by id (for example a `local:{id}` string) would merge all
|
||||
episodes into one. `TrackItemKey` has no content-id equivalence (a pending
|
||||
federated item and its imported library copy differ), so clients that
|
||||
match music content-first keep doing that and add the podcast branch
|
||||
before any fallback to the id.
|
||||
* Wire ids of podcast items and of Jam private placeholders are `0`
|
||||
(`PlaybackTrack::podcast_episode`, `PlaybackTrack::private_placeholder`)
|
||||
and must not be copied into a `TrackItem`.
|
||||
`TrackItem::from_playback_track` classifies a received item as
|
||||
`PlaybackItem::Music` (the client's music path), `PodcastEpisode` (built
|
||||
with `TrackItem::podcast_episode`, sentinel ids) or `Private` (not
|
||||
playable, no `TrackItem`). Besides the ids, `TrackItem::podcast_episode`
|
||||
also differs from the wire form in `release_year` (publication year) and
|
||||
`audio_format` (`None`; the MIME type stays in the reference).
|
||||
|
||||
## Versioning
|
||||
|
||||
* `music-dht` 0.6.0 and `furumi-library` 0.3.0: new public types, enum
|
||||
|
||||
@@ -16,8 +16,11 @@ anyhow = { workspace = true }
|
||||
blake3 = { workspace = true }
|
||||
directories = { workspace = true }
|
||||
lofty = { workspace = true }
|
||||
music-dht = { workspace = true }
|
||||
music-dht = { workspace = true, features = ["podcast-feed"] }
|
||||
rusqlite = { workspace = true, features = ["functions"] }
|
||||
serde = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
tracing = { workspace = true }
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = { workspace = true }
|
||||
@@ -21,5 +21,63 @@ library used by every client. Connections use WAL, foreign keys, and a busy
|
||||
timeout so independently running clients can safely share the database.
|
||||
|
||||
Version 0.3 follows `music-dht` 0.6, whose device-sync and podcast contract
|
||||
types it re-exposes. Podcast persistence is not part of this release; it will
|
||||
implement the rules of the workspace `PODCASTS.md`.
|
||||
types it re-exposes.
|
||||
|
||||
## Podcasts
|
||||
|
||||
`furumi_library::podcasts::PodcastStore` is the SQLite reference store of the
|
||||
podcast contract (workspace `PODCASTS.md`, section "SQLite reference store").
|
||||
It lives in its own file, conventionally `podcasts.sqlite3`
|
||||
(`PODCAST_DB_FILE`) next to `library.db`; the caller passes the path.
|
||||
|
||||
- **Migrations** run on `PRAGMA user_version`, each in its own transaction.
|
||||
A database of the original podcast prototype (`user_version` 1) is imported:
|
||||
its tables are renamed `legacy_podcasts` / `legacy_podcast_episodes` (never
|
||||
dropped), feeds become subscriptions with contract ids, rows that cannot be
|
||||
imported are listed in `legacy_quarantine`. A database from a newer Furumi,
|
||||
or a file that is not a podcast database, is refused without being touched.
|
||||
- **Local actions** (`subscribe`, `unsubscribe`, `set_shared`,
|
||||
`start_session`, `checkpoint`, `seek`, `restart`, `complete_session`,
|
||||
`mark_played`, `mark_unplayed`, `record_listen_session`) draw stamps from a
|
||||
`StampSource` (usually the client's sync log), apply the operation and
|
||||
return `PodcastLocalOp`s to append to the log with exactly those stamps.
|
||||
- **Sync**: `apply_remote_op` applies the five `podcast_*` operations,
|
||||
`fill_snapshot` / `export_snapshot` produce the bounded snapshot sections
|
||||
and `apply_snapshot` merges them. Everything goes through the `music-dht`
|
||||
materializers and merge functions, so the result does not depend on
|
||||
delivery order and never loses newer local state.
|
||||
- **RSS**: `apply_feed` ingests a document parsed by
|
||||
`music_dht::podcast::feed` (the crate enables the `podcast-feed` feature):
|
||||
channel metadata, episodes with stable ids and current audio URLs, feed
|
||||
aliases from permanent redirects and `itunes:new-feed-url`. It never
|
||||
touches subscriptions, progress or history. HTTP, conditional requests and
|
||||
scheduling stay in the client; `record_fetch_*`, `fetch_state` and
|
||||
`podcasts_due_for_refresh` keep the local-only fetch state, which is never
|
||||
synced.
|
||||
- **Queries**: `list_podcasts`, `seasons`, `episodes` (season, played-state
|
||||
and text filters, date or episode-number order), `resume_position_ms`,
|
||||
`episode_ref` / `episode_track` for queueing, `listen_history`.
|
||||
|
||||
The store is synchronous: methods take `&self` and lock one connection, so
|
||||
`Arc<PodcastStore>` can be shared; run calls on a blocking pool.
|
||||
|
||||
`TrackItem.podcast` marks a queue item as a podcast episode
|
||||
(`TrackItem::podcast_episode`). Music-only code (likes, playlists, history,
|
||||
content ids, federation downloads) must check `TrackItem::is_podcast`; the
|
||||
music library never stores episodes.
|
||||
|
||||
`TrackItem::key` returns the typed source identity of a queue item
|
||||
(`TrackItemKey`: `lib:<id>`, `fed:<owner>:<item_id>` or `ep:<episode id>` as
|
||||
a string). A negative `TrackItem::id` only means "not a library row". Every
|
||||
podcast item has the same sentinel `id`, `release_id` and artist id
|
||||
(`TrackItem::NOT_A_LIBRARY_ROW` = -1). It carries no identity and must not
|
||||
be decoded; it only keeps music-only code that predates podcasts from taking
|
||||
an episode for library track, release or artist 0. The episode's identity is
|
||||
`TrackItem.podcast` alone, so queue, current-track and position matching
|
||||
must check it before any fallback to the id. `TrackItemKey` does not equate
|
||||
a pending federated item with its library copy; clients that match music by
|
||||
content id keep doing so.
|
||||
|
||||
Convert received `PlaybackTrack`s with `TrackItem::from_playback_track`
|
||||
(`PlaybackItem::Music`, `PodcastEpisode` or `Private`): podcast items and
|
||||
Jam private placeholders carry id 0 on the wire, which must never be copied.
|
||||
@@ -6,9 +6,13 @@
|
||||
//!
|
||||
//! All methods take `&self` and lock the connection internally, so an
|
||||
//! `Arc<Library>` can be shared across blocking tasks.
|
||||
//!
|
||||
//! Podcasts live in a separate database, [`podcasts::PodcastStore`]
|
||||
//! (`podcasts.sqlite3`); the music library never stores episodes.
|
||||
|
||||
pub mod import;
|
||||
pub mod models;
|
||||
pub mod podcasts;
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::io::Write as _;
|
||||
@@ -22,8 +26,8 @@ use rusqlite::{Connection, OptionalExtension as _, params};
|
||||
|
||||
pub use models::{
|
||||
ArtistCard, ArtistDetail, ArtistRef, ArtistsPage, Availability, FederatedTrack, LibraryFilters,
|
||||
LibrarySourceMode, PlaylistCard, PlaylistDetail, ReleaseCard, ReleaseDetail, ReleaseEdit,
|
||||
SearchResults, TrackEdit, TrackItem,
|
||||
LibrarySourceMode, PlaybackItem, PlaylistCard, PlaylistDetail, ReleaseCard, ReleaseDetail,
|
||||
ReleaseEdit, SearchResults, TrackEdit, TrackItem, TrackItemKey,
|
||||
};
|
||||
|
||||
/// The virtual "Liked tracks" playlist id, kept from the server API.
|
||||
@@ -3283,6 +3287,7 @@ fn query_tracks(
|
||||
artists: Vec::new(),
|
||||
featured_artists: Vec::new(),
|
||||
fed: None,
|
||||
podcast: None,
|
||||
})
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
@@ -3348,6 +3353,7 @@ fn pending_track(fed: &FederatedTrack) -> TrackItem {
|
||||
file_size_bytes: None,
|
||||
play_count: 0,
|
||||
fed: Some(fed.clone()),
|
||||
podcast: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
//! Data shapes the views render. They mirror what the furumusic API used to
|
||||
//! return, but every field is now filled from the local SQLite library.
|
||||
|
||||
use music_dht::device_sync::{PlaybackSource, PlaybackTrack};
|
||||
use music_dht::podcast::{EpisodeId, PodcastEpisodeRef};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Which catalog sources a client wants library queries to include.
|
||||
@@ -120,6 +122,56 @@ pub struct ArtistRef {
|
||||
pub name: String,
|
||||
}
|
||||
|
||||
/// A playable queue item: a library track, a pending federated track or a
|
||||
/// podcast episode.
|
||||
///
|
||||
/// # Identity
|
||||
///
|
||||
/// What an item *is* comes from its explicit source fields, never from the
|
||||
/// numeric `id` alone; [`TrackItem::key`] returns it as a typed
|
||||
/// [`TrackItemKey`]:
|
||||
///
|
||||
/// * a library track is a library row: `id >= 0`, `fed` and `podcast` are
|
||||
/// `None`;
|
||||
/// * a federated track is identified by [`TrackItem::fed`] (owner and item
|
||||
/// id). While it is not in the library its `id` is a negative,
|
||||
/// process-local counter value (pending federated tracks count down from
|
||||
/// -1) and `release_id` / artist ids are -1;
|
||||
/// * a podcast episode is identified **only** by [`TrackItem::podcast`] (its
|
||||
/// [`EpisodeId`]). Its `id`, `release_id` and artist id are the fixed
|
||||
/// sentinel [`TrackItem::NOT_A_LIBRARY_ROW`] (-1), shared by every episode.
|
||||
///
|
||||
/// A negative `id` therefore means "not a library row" and nothing else: it
|
||||
/// carries no identity and must never be decoded, compared across items or
|
||||
/// used as a key. The -1 sentinel only keeps music-only code that predates
|
||||
/// podcasts (`id >= 0` / `release_id < 0` checks) from treating an episode
|
||||
/// as library track, release or artist 0.
|
||||
///
|
||||
/// [`TrackItem::key`] is the *source* identity. It does not merge a pending
|
||||
/// federated item with its imported library copy, which share a content id
|
||||
/// but have different keys (`fed:` and `lib:`). Clients that key playback
|
||||
/// items content-first (content id, then federated source, then id) keep
|
||||
/// doing so for music; what they must add is a podcast branch
|
||||
/// ([`TrackItem::podcast`], or [`TrackItemKey::PodcastEpisode`]) checked
|
||||
/// before any fallback to `id`. An episode must never be keyed by `id`: a
|
||||
/// `local:{id}` key would merge all episodes into one. On the wire the item
|
||||
/// kind is `music_dht::device_sync::PlaybackSource`; convert received items
|
||||
/// with [`TrackItem::from_playback_track`], never by copying wire ids.
|
||||
///
|
||||
/// # Podcast episodes
|
||||
///
|
||||
/// Podcast episodes carry their [`PodcastEpisodeRef`] in
|
||||
/// [`TrackItem::podcast`]: `file_path` is empty, `content_id`, `fed` and
|
||||
/// `cover_path` are `None`, and the display fields mirror the reference
|
||||
/// (episode title, podcast title as artist and release). The audio URL is
|
||||
/// only in the reference: it is never a local path, never content-addressed
|
||||
/// and never offered for P2P transfer.
|
||||
///
|
||||
/// **Music-only code paths must check [`TrackItem::is_podcast`]** (or
|
||||
/// `podcast.is_none()`) before treating an item as music: likes, playlists,
|
||||
/// listening history, content-id lookups, federation downloads, similarity
|
||||
/// and file operations do not apply to episodes. Library queries never
|
||||
/// return podcast items; the library database stores none.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct TrackItem {
|
||||
pub id: i64,
|
||||
@@ -149,9 +201,187 @@ pub struct TrackItem {
|
||||
/// carries everything needed to download them from the owning peer.
|
||||
/// With an empty `file_path` the player resolves the track on demand.
|
||||
pub fed: Option<FederatedTrack>,
|
||||
/// Set for podcast episodes (see the type documentation); `None` for music.
|
||||
pub podcast: Option<PodcastEpisodeRef>,
|
||||
}
|
||||
|
||||
/// Typed source identity of a [`TrackItem`], from [`TrackItem::key`].
|
||||
///
|
||||
/// Two items with equal keys are the same track or episode; the same track
|
||||
/// may still appear more than once in a queue. The key has no content-id
|
||||
/// equivalence: a pending federated item and its library copy differ. Use it
|
||||
/// where the source matters (and always for podcast episodes, which have no
|
||||
/// other identity); code that matches music by content id keeps doing that
|
||||
/// and checks the podcast variant first (see [`TrackItem`]).
|
||||
///
|
||||
/// The [`Display`](std::fmt::Display) form is stable and suitable as a map
|
||||
/// key or in logs:
|
||||
///
|
||||
/// | Variant | String form |
|
||||
/// | --- | --- |
|
||||
/// | `Library(123)` | `lib:123` |
|
||||
/// | `Federated { owner, item_id }` | `fed:<owner>:<item_id>` |
|
||||
/// | `PodcastEpisode(id)` | `ep:<episode id>`, e.g. `ep:ep1:...` |
|
||||
/// | `Transient(-5)` | `tmp:-5` |
|
||||
///
|
||||
/// `owner` is a peer endpoint id and contains no `:`, so the federated form
|
||||
/// is unambiguous. Keys are not persisted by this crate; a `Transient` key is
|
||||
/// only meaningful inside the process that built the item.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
|
||||
pub enum TrackItemKey {
|
||||
/// A row of the local music library (`TrackItem::id >= 0`).
|
||||
Library(i64),
|
||||
/// A federated track that is not a library row, identified by its owner
|
||||
/// peer and DHT item id ([`FederatedTrack::owner`] /
|
||||
/// [`FederatedTrack::item_id`]), not by its process-local negative `id`.
|
||||
Federated { owner: String, item_id: String },
|
||||
/// A podcast episode, identified by its [`EpisodeId`] only.
|
||||
PodcastEpisode(EpisodeId),
|
||||
/// A non-library item with no explicit source (negative `id`, no `fed`,
|
||||
/// no `podcast`). furumi-library never builds one; clients that do must
|
||||
/// keep its `id` unique within the process.
|
||||
Transient(i64),
|
||||
}
|
||||
|
||||
/// What a received `music_dht::device_sync::PlaybackTrack` is locally, from
|
||||
/// [`TrackItem::from_playback_track`].
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum PlaybackItem {
|
||||
/// A music track. Convert it with the client's music path (content id,
|
||||
/// then federated source); its wire ids are the sender's.
|
||||
Music,
|
||||
/// A podcast episode, built with [`TrackItem::podcast_episode`].
|
||||
PodcastEpisode(Box<TrackItem>),
|
||||
/// A Jam private placeholder: not playable and without identity. Show it
|
||||
/// as `music_dht::device_sync::PRIVATE_ITEM_TITLE` where the queue is
|
||||
/// displayed; never insert it as a library or queue track, never key it.
|
||||
Private,
|
||||
}
|
||||
|
||||
impl std::fmt::Display for TrackItemKey {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::Library(id) => write!(f, "lib:{id}"),
|
||||
Self::Federated { owner, item_id } => write!(f, "fed:{owner}:{item_id}"),
|
||||
Self::PodcastEpisode(episode_id) => write!(f, "ep:{}", episode_id.as_str()),
|
||||
Self::Transient(id) => write!(f, "tmp:{id}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl TrackItem {
|
||||
/// The `id`, `release_id` and artist id of items that are not library
|
||||
/// rows and have no numeric identity: every podcast episode uses it.
|
||||
///
|
||||
/// It is a sentinel, not an identifier: it is shared by all episodes
|
||||
/// (and can coincide with the first pending federated track's id), so it
|
||||
/// must never be decoded or used as a key. See [`TrackItem::key`].
|
||||
pub const NOT_A_LIBRARY_ROW: i64 = -1;
|
||||
|
||||
/// The typed identity of this item; see [`TrackItemKey`].
|
||||
///
|
||||
/// Precedence: [`TrackItem::podcast`] makes it a podcast episode; else a
|
||||
/// non-negative `id` is a library row; else [`TrackItem::fed`] makes it
|
||||
/// a federated track; anything else is [`TrackItemKey::Transient`].
|
||||
pub fn key(&self) -> TrackItemKey {
|
||||
if let Some(episode) = &self.podcast {
|
||||
return TrackItemKey::PodcastEpisode(episode.episode_id.clone());
|
||||
}
|
||||
if self.id >= 0 {
|
||||
return TrackItemKey::Library(self.id);
|
||||
}
|
||||
match &self.fed {
|
||||
Some(fed) => TrackItemKey::Federated {
|
||||
owner: fed.owner.clone(),
|
||||
item_id: fed.item_id.clone(),
|
||||
},
|
||||
None => TrackItemKey::Transient(self.id),
|
||||
}
|
||||
}
|
||||
|
||||
/// Queue item for a podcast episode streamed from the publisher.
|
||||
///
|
||||
/// Title, artist and release follow `PlaybackTrack::podcast_episode` of
|
||||
/// `music_dht::device_sync` (the episode title, the podcast title as
|
||||
/// artist and release, the announced duration; no track or disc number).
|
||||
/// The other fields differ from that wire form on purpose:
|
||||
///
|
||||
/// * `id`, `release_id` and the artist id are the sentinel
|
||||
/// [`TrackItem::NOT_A_LIBRARY_ROW`] (-1) for every episode, where the
|
||||
/// wire form carries `0`; the item's identity is [`TrackItem::podcast`]
|
||||
/// alone ([`TrackItem::key`]);
|
||||
/// * `release_year` is the publication year (`None` on the wire);
|
||||
/// * `audio_format` is `None` (the wire form carries the MIME type, which
|
||||
/// stays in the reference).
|
||||
///
|
||||
/// Season and episode number stay in the reference.
|
||||
pub fn podcast_episode(episode: PodcastEpisodeRef) -> Self {
|
||||
Self {
|
||||
id: Self::NOT_A_LIBRARY_ROW,
|
||||
title: episode.episode_title.clone(),
|
||||
track_number: None,
|
||||
disc_number: None,
|
||||
duration_seconds: episode
|
||||
.duration_ms
|
||||
.map_or(0.0, |ms| ms.max(0) as f64 / 1_000.0),
|
||||
artists: vec![ArtistRef {
|
||||
id: Self::NOT_A_LIBRARY_ROW,
|
||||
name: episode.podcast_title.clone(),
|
||||
}],
|
||||
featured_artists: Vec::new(),
|
||||
release_id: Self::NOT_A_LIBRARY_ROW,
|
||||
release_title: episode.podcast_title.clone(),
|
||||
release_year: episode.published_at_ms.map(year_of_unix_ms),
|
||||
file_path: String::new(),
|
||||
content_id: None,
|
||||
cover_path: None,
|
||||
audio_format: None,
|
||||
audio_bitrate: None,
|
||||
audio_sample_rate: None,
|
||||
audio_bit_depth: None,
|
||||
file_size_bytes: None,
|
||||
play_count: 0,
|
||||
fed: None,
|
||||
podcast: Some(episode),
|
||||
}
|
||||
}
|
||||
|
||||
/// Classifies a received `music_dht::device_sync::PlaybackTrack` and
|
||||
/// builds the local queue item for non-music sources.
|
||||
///
|
||||
/// The wire `id` and `release_id` of podcast episodes and of Jam private
|
||||
/// placeholders are `0`, and those of music items are the sender's local
|
||||
/// ids. None of them identifies anything on this device, so they are
|
||||
/// never copied:
|
||||
///
|
||||
/// * a podcast episode becomes [`TrackItem::podcast_episode`] of its
|
||||
/// [`PlaybackSource::PodcastEpisode`] reference (sentinel ids, key
|
||||
/// `ep:`), whatever the legacy display fields say;
|
||||
/// * a private placeholder ([`PlaybackSource::Private`]) is not playable
|
||||
/// and has no identity, so no `TrackItem` exists for it;
|
||||
/// * music (no source, or [`PlaybackSource::Music`]) is left to the
|
||||
/// client's music path, which resolves it by content id or federated
|
||||
/// source.
|
||||
pub fn from_playback_track(track: &PlaybackTrack) -> PlaybackItem {
|
||||
match &track.source {
|
||||
None | Some(PlaybackSource::Music) => PlaybackItem::Music,
|
||||
Some(PlaybackSource::PodcastEpisode(episode)) => PlaybackItem::PodcastEpisode(
|
||||
Box::new(Self::podcast_episode(episode.as_ref().clone())),
|
||||
),
|
||||
Some(PlaybackSource::Private) => PlaybackItem::Private,
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether this item is a podcast episode (not music).
|
||||
pub fn is_podcast(&self) -> bool {
|
||||
self.podcast.is_some()
|
||||
}
|
||||
|
||||
/// The episode reference of a podcast item.
|
||||
pub fn podcast_episode_ref(&self) -> Option<&PodcastEpisodeRef> {
|
||||
self.podcast.as_ref()
|
||||
}
|
||||
|
||||
/// A federated track that still needs downloading before playback.
|
||||
pub fn is_fed_pending(&self) -> bool {
|
||||
self.fed.is_some() && self.file_path.is_empty()
|
||||
@@ -202,6 +432,21 @@ impl TrackItem {
|
||||
}
|
||||
}
|
||||
|
||||
/// Proleptic Gregorian year of a unix timestamp in milliseconds (UTC).
|
||||
fn year_of_unix_ms(ms: i64) -> i32 {
|
||||
// Howard Hinnant's civil_from_days.
|
||||
let days = ms.div_euclid(86_400_000);
|
||||
let z = days + 719_468;
|
||||
let era = z.div_euclid(146_097);
|
||||
let doe = z - era * 146_097;
|
||||
let yoe = (doe - doe / 1_460 + doe / 36_524 - doe / 146_096) / 365;
|
||||
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
|
||||
let mp = (5 * doy + 2) / 153;
|
||||
let month = if mp < 10 { mp + 3 } else { mp - 9 };
|
||||
let year = yoe + era * 400 + i64::from(month <= 2);
|
||||
i32::try_from(year).unwrap_or(if year < 0 { i32::MIN } else { i32::MAX })
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ReleaseCard {
|
||||
pub id: i64,
|
||||
@@ -340,8 +585,228 @@ mod tests {
|
||||
file_size_bytes: None,
|
||||
play_count: 0,
|
||||
fed: None,
|
||||
podcast: None,
|
||||
};
|
||||
|
||||
assert_eq!(track.artist_line(), "Main feat. Guest, Other");
|
||||
assert!(!track.is_podcast());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn year_of_unix_ms_handles_epoch_and_leap_years() {
|
||||
assert_eq!(year_of_unix_ms(0), 1970);
|
||||
assert_eq!(year_of_unix_ms(-1), 1969);
|
||||
// 2024-02-29T12:00:00Z and 2024-12-31T23:59:59Z.
|
||||
assert_eq!(year_of_unix_ms(1_709_208_000_000), 2024);
|
||||
assert_eq!(year_of_unix_ms(1_735_689_599_000), 2024);
|
||||
assert_eq!(year_of_unix_ms(1_735_689_600_000), 2025);
|
||||
}
|
||||
|
||||
fn episode_ref(guid: &str) -> PodcastEpisodeRef {
|
||||
use music_dht::podcast::{FeedUrl, PodcastId};
|
||||
let feed_url = FeedUrl::parse("https://example.test/feed.xml").unwrap();
|
||||
let podcast_id = PodcastId::from_feed_url(&feed_url);
|
||||
PodcastEpisodeRef {
|
||||
episode_id: EpisodeId::from_guid(&podcast_id, guid).unwrap(),
|
||||
podcast_id,
|
||||
feed_url,
|
||||
podcast_title: "Show".into(),
|
||||
episode_title: "Pilot".into(),
|
||||
audio_url: "https://cdn.example.test/1.mp3".into(),
|
||||
mime: Some("audio/mpeg".into()),
|
||||
duration_ms: Some(90_500),
|
||||
published_at_ms: Some(1_709_208_000_000),
|
||||
season: Some(2),
|
||||
episode_number: Some(7),
|
||||
image_url: None,
|
||||
progress_generation: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn library_track(id: i64) -> TrackItem {
|
||||
TrackItem {
|
||||
id,
|
||||
title: "Track".into(),
|
||||
track_number: None,
|
||||
disc_number: None,
|
||||
duration_seconds: 1.0,
|
||||
artists: vec![artist("Main")],
|
||||
featured_artists: Vec::new(),
|
||||
release_id: 1,
|
||||
release_title: "Release".into(),
|
||||
release_year: None,
|
||||
file_path: "/tmp/track.mp3".into(),
|
||||
content_id: None,
|
||||
cover_path: None,
|
||||
audio_format: None,
|
||||
audio_bitrate: None,
|
||||
audio_sample_rate: None,
|
||||
audio_bit_depth: None,
|
||||
file_size_bytes: None,
|
||||
play_count: 0,
|
||||
fed: None,
|
||||
podcast: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn federated(owner: &str, item_id: &str) -> FederatedTrack {
|
||||
FederatedTrack {
|
||||
item_id: item_id.into(),
|
||||
owner: owner.into(),
|
||||
own: false,
|
||||
title: "Remote".into(),
|
||||
artist_names: vec!["Someone".into()],
|
||||
featured_artist_names: Vec::new(),
|
||||
year: None,
|
||||
duration_seconds: None,
|
||||
content_id: None,
|
||||
release_title: None,
|
||||
track_number: None,
|
||||
disc_number: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn podcast_track_mirrors_the_episode_reference() {
|
||||
let episode = episode_ref("e1");
|
||||
let track = TrackItem::podcast_episode(episode.clone());
|
||||
assert!(track.is_podcast());
|
||||
assert_eq!(track.podcast_episode_ref(), Some(&episode));
|
||||
// The sentinel marks "not a library row, release or artist" and
|
||||
// carries no identity.
|
||||
assert_eq!(TrackItem::NOT_A_LIBRARY_ROW, -1);
|
||||
assert_eq!(track.id, -1);
|
||||
assert_eq!(track.release_id, -1);
|
||||
assert!(track.artists.iter().all(|artist| artist.id == -1));
|
||||
assert!(track.featured_artists.is_empty());
|
||||
assert!(track.file_path.is_empty());
|
||||
assert!(track.content_id.is_none() && track.fed.is_none());
|
||||
assert!(!track.is_fed_pending());
|
||||
assert_eq!(track.title, "Pilot");
|
||||
assert_eq!(track.artist_line(), "Show");
|
||||
assert_eq!(track.release_title, "Show");
|
||||
assert_eq!(track.release_year, Some(2024));
|
||||
assert_eq!(track.duration_label(), "1:31");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn key_is_typed_by_source() {
|
||||
let library = library_track(123);
|
||||
assert_eq!(library.key(), TrackItemKey::Library(123));
|
||||
assert_eq!(library.key().to_string(), "lib:123");
|
||||
assert_eq!(library_track(0).key(), TrackItemKey::Library(0));
|
||||
|
||||
let mut pending = library_track(-5);
|
||||
pending.release_id = -1;
|
||||
pending.file_path.clear();
|
||||
pending.fed = Some(federated("peer1", "item-9"));
|
||||
let fed_key = TrackItemKey::Federated {
|
||||
owner: "peer1".into(),
|
||||
item_id: "item-9".into(),
|
||||
};
|
||||
assert_eq!(pending.key(), fed_key);
|
||||
assert_eq!(pending.key().to_string(), "fed:peer1:item-9");
|
||||
// The process-local negative id is not part of a federated identity.
|
||||
let mut renumbered = pending.clone();
|
||||
renumbered.id = -6;
|
||||
assert_eq!(renumbered.key(), pending.key());
|
||||
|
||||
let episode = episode_ref("e1");
|
||||
let track = TrackItem::podcast_episode(episode.clone());
|
||||
let ep_key = TrackItemKey::PodcastEpisode(episode.episode_id.clone());
|
||||
assert_eq!(track.key(), ep_key);
|
||||
assert_eq!(
|
||||
track.key().to_string(),
|
||||
format!("ep:{}", episode.episode_id.as_str())
|
||||
);
|
||||
assert!(track.key().to_string().starts_with("ep:ep1:"));
|
||||
|
||||
let transient = library_track(-7);
|
||||
assert_eq!(transient.key(), TrackItemKey::Transient(-7));
|
||||
assert_eq!(transient.key().to_string(), "tmp:-7");
|
||||
|
||||
// The source wins over the numeric id: a podcast item is never keyed
|
||||
// as a library row, a federated item never by its counter value.
|
||||
let mut odd = TrackItem::podcast_episode(episode);
|
||||
odd.id = 42;
|
||||
assert_eq!(odd.key(), ep_key);
|
||||
let mut pending_minus_one = pending.clone();
|
||||
pending_minus_one.id = TrackItem::NOT_A_LIBRARY_ROW;
|
||||
assert_eq!(pending_minus_one.key(), fed_key);
|
||||
assert_ne!(pending_minus_one.key(), track.key());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn received_playback_tracks_never_copy_wire_ids() {
|
||||
// F1 wire form of an episode: id 0, release_id 0.
|
||||
let episode = episode_ref("e1");
|
||||
let wire = PlaybackTrack::podcast_episode(episode.clone());
|
||||
assert_eq!((wire.id, wire.release_id), (0, 0));
|
||||
let PlaybackItem::PodcastEpisode(item) = TrackItem::from_playback_track(&wire) else {
|
||||
panic!("an episode is not music");
|
||||
};
|
||||
assert_eq!(item.id, TrackItem::NOT_A_LIBRARY_ROW);
|
||||
assert_eq!(item.release_id, TrackItem::NOT_A_LIBRARY_ROW);
|
||||
assert!(item.artists.iter().all(|artist| artist.id < 0));
|
||||
assert_eq!(
|
||||
item.key(),
|
||||
TrackItemKey::PodcastEpisode(episode.episode_id.clone())
|
||||
);
|
||||
assert_eq!(item.podcast_episode_ref(), Some(&episode));
|
||||
// Legacy display fields an older sender filled in do not matter.
|
||||
let mut odd = wire.clone();
|
||||
odd.id = 42;
|
||||
odd.release_id = 7;
|
||||
odd.title = "Something else".into();
|
||||
let PlaybackItem::PodcastEpisode(odd) = TrackItem::from_playback_track(&odd) else {
|
||||
panic!("the source decides");
|
||||
};
|
||||
assert_eq!(odd.key(), item.key());
|
||||
assert_eq!(odd.id, TrackItem::NOT_A_LIBRARY_ROW);
|
||||
|
||||
// A private placeholder (also id 0) never becomes library row 0.
|
||||
let private = PlaybackTrack::private_placeholder();
|
||||
assert_eq!(private.id, 0);
|
||||
assert!(matches!(
|
||||
TrackItem::from_playback_track(&private),
|
||||
PlaybackItem::Private
|
||||
));
|
||||
|
||||
// Music is left to the client's music path.
|
||||
let mut music = wire;
|
||||
music.source = None;
|
||||
assert!(matches!(
|
||||
TrackItem::from_playback_track(&music),
|
||||
PlaybackItem::Music
|
||||
));
|
||||
music.source = Some(PlaybackSource::Music);
|
||||
assert!(matches!(
|
||||
TrackItem::from_playback_track(&music),
|
||||
PlaybackItem::Music
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn episodes_share_the_sentinel_id_but_not_the_key() {
|
||||
let first = TrackItem::podcast_episode(episode_ref("e1"));
|
||||
let second = TrackItem::podcast_episode(episode_ref("e2"));
|
||||
assert_eq!(first.id, TrackItem::NOT_A_LIBRARY_ROW);
|
||||
assert_eq!(second.id, TrackItem::NOT_A_LIBRARY_ROW);
|
||||
assert_eq!(first.id, second.id);
|
||||
assert_eq!(first.release_id, second.release_id);
|
||||
assert_ne!(first.key(), second.key());
|
||||
assert_ne!(first.key().to_string(), second.key().to_string());
|
||||
assert_eq!(
|
||||
TrackItem::podcast_episode(episode_ref("e1")).key(),
|
||||
first.key()
|
||||
);
|
||||
let keys: std::collections::HashSet<TrackItemKey> =
|
||||
[first.key(), second.key(), first.key()]
|
||||
.into_iter()
|
||||
.collect();
|
||||
assert_eq!(keys.len(), 2);
|
||||
let mut ordered = [second.key(), first.key(), TrackItemKey::Library(1)];
|
||||
ordered.sort();
|
||||
assert_eq!(ordered[0], TrackItemKey::Library(1));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,544 @@
|
||||
//! Local user actions: they build the device-sync operation, apply it through
|
||||
//! the shared code path and return it for the client's sync log.
|
||||
|
||||
use anyhow::Result;
|
||||
use music_dht::device_sync::SyncOpPayload;
|
||||
use music_dht::podcast::{
|
||||
EpisodeId, FeedUrl, ListenSession, PodcastEpisodeRef, PodcastId, PodcastMeta, ProgressRecord,
|
||||
ProgressStatus, effective_share, merge_feed_aliases, merge_listen_session,
|
||||
next_podcast_generation,
|
||||
};
|
||||
use rusqlite::{Connection, TransactionBehavior};
|
||||
|
||||
use super::apply::{self, ApplyOutcome};
|
||||
use super::{OpStamp, PodcastLocalOp, PodcastStore, PodcastStoreError, StampSource};
|
||||
|
||||
/// Result of [`PodcastStore::subscribe`].
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct SubscribeOutcome {
|
||||
/// Podcast the URL resolved to (an existing one when the URL is a known
|
||||
/// feed URL or alias).
|
||||
pub podcast_id: PodcastId,
|
||||
/// The podcast was already subscribed; no operation was emitted.
|
||||
pub already_subscribed: bool,
|
||||
/// Operations to append to the sync log.
|
||||
pub ops: Vec<PodcastLocalOp>,
|
||||
}
|
||||
|
||||
/// A listening session of this device as seen by the progress register.
|
||||
///
|
||||
/// Returned by explicit actions; periodic checkpoints of the session use its
|
||||
/// `generation` ([`PodcastStore::checkpoint`]).
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ProgressSession {
|
||||
/// Episode being played.
|
||||
pub episode: PodcastEpisodeRef,
|
||||
/// Session id (also the `ListenSession::session_id`).
|
||||
pub session_id: String,
|
||||
/// Generation of the session's latest explicit action.
|
||||
pub generation: u64,
|
||||
}
|
||||
|
||||
/// A progress register written by a local action.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct ProgressWrite {
|
||||
/// The register as written.
|
||||
pub record: ProgressRecord,
|
||||
/// The session after the action (its generation for later checkpoints).
|
||||
pub session: ProgressSession,
|
||||
/// Operation to append to the sync log.
|
||||
pub op: PodcastLocalOp,
|
||||
}
|
||||
|
||||
/// Result of [`PodcastStore::checkpoint`].
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum CheckpointOutcome {
|
||||
/// The checkpoint was written.
|
||||
Written(Box<ProgressWrite>),
|
||||
/// The register holds a greater generation (another device or an
|
||||
/// explicit action took over). Nothing was written: stop writing at the
|
||||
/// old generation, or continue locally with an explicit action
|
||||
/// ([`PodcastStore::start_session`] / [`PodcastStore::seek`]).
|
||||
Superseded {
|
||||
/// Current register.
|
||||
current: ProgressRecord,
|
||||
},
|
||||
/// This session already completed the episode at its generation; a
|
||||
/// checkpoint would turn it back into "in progress". Nothing was written.
|
||||
AlreadyCompleted {
|
||||
/// Current register.
|
||||
current: ProgressRecord,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
enum ExplicitAction {
|
||||
Start(i64),
|
||||
Seek(i64),
|
||||
Restart,
|
||||
Complete(i64),
|
||||
MarkPlayed,
|
||||
MarkUnplayed,
|
||||
}
|
||||
|
||||
fn invalid(reason: impl ToString) -> anyhow::Error {
|
||||
PodcastStoreError::Invalid(reason.to_string()).into()
|
||||
}
|
||||
|
||||
/// Validates the payload (with a provisional stamp), draws the real stamp,
|
||||
/// and applies the operation through [`apply::apply_op`].
|
||||
pub(crate) fn emit(
|
||||
conn: &Connection,
|
||||
stamps: &mut dyn StampSource,
|
||||
make: impl Fn(&OpStamp) -> SyncOpPayload,
|
||||
) -> Result<PodcastLocalOp> {
|
||||
let device_id = stamps.device_id().to_string();
|
||||
let provisional = OpStamp::new("provisional", device_id.clone(), 0);
|
||||
provisional
|
||||
.wire(make(&provisional))
|
||||
.validate_podcast()
|
||||
.map_err(invalid)?;
|
||||
let stamp = stamps.next_stamp()?;
|
||||
if stamp.origin_device_id != device_id {
|
||||
return Err(invalid("stamp origin differs from the local device id"));
|
||||
}
|
||||
let payload = make(&stamp);
|
||||
let op = stamp.wire(payload.clone());
|
||||
match apply::apply_op(conn, &op)? {
|
||||
ApplyOutcome::Rejected(reason) => Err(invalid(reason)),
|
||||
_ => Ok(PodcastLocalOp { stamp, payload }),
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolves a feed URL against every known URL of every podcast.
|
||||
pub(crate) fn resolve_podcast(conn: &Connection, url: &FeedUrl) -> Result<PodcastId> {
|
||||
let mut statement = conn.prepare_cached(
|
||||
"SELECT podcast_id FROM podcast_feed_aliases WHERE feed_url = ?1
|
||||
UNION SELECT podcast_id FROM podcasts WHERE feed_url = ?1
|
||||
UNION SELECT podcast_id FROM podcast_subscriptions
|
||||
WHERE json_extract(podcast_json, '$.feed_url') = ?1",
|
||||
)?;
|
||||
let ids = statement
|
||||
.query_map([url.as_str()], |row| super::podcast_id_col(row.get(0)?))?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
let pairs: Vec<(PodcastId, FeedUrl)> = ids.into_iter().map(|id| (id, url.clone())).collect();
|
||||
Ok(music_dht::podcast::resolve_podcast_id(
|
||||
url,
|
||||
pairs.iter().map(|(id, url)| (id, url)),
|
||||
))
|
||||
}
|
||||
|
||||
fn catalog_meta(conn: &Connection, podcast_id: &PodcastId) -> Result<Option<PodcastMeta>> {
|
||||
use rusqlite::OptionalExtension as _;
|
||||
Ok(conn
|
||||
.query_row(
|
||||
"SELECT feed_url, title, author, description, link, image_url
|
||||
FROM podcasts WHERE podcast_id = ?1",
|
||||
[podcast_id.as_str()],
|
||||
|row| {
|
||||
Ok(PodcastMeta {
|
||||
feed_url: super::feed_url_col(row.get(0)?)?,
|
||||
title: row.get(1)?,
|
||||
author: row.get(2)?,
|
||||
description: row.get(3)?,
|
||||
link: row.get(4)?,
|
||||
image_url: row.get(5)?,
|
||||
})
|
||||
},
|
||||
)
|
||||
.optional()?)
|
||||
}
|
||||
|
||||
fn share_payload(conn: &Connection, podcast_id: &PodcastId, shared: bool) -> Result<SyncOpPayload> {
|
||||
let subscription = apply::load_subscription_register(conn, podcast_id)?;
|
||||
let share = apply::load_share(conn, podcast_id)?;
|
||||
Ok(SyncOpPayload::PodcastShareSet {
|
||||
podcast_id: podcast_id.clone(),
|
||||
shared,
|
||||
generation: next_podcast_generation(subscription.as_ref(), share.as_ref()),
|
||||
})
|
||||
}
|
||||
|
||||
/// The reference carried by progress operations: identity and display only.
|
||||
fn op_ref(episode: &PodcastEpisodeRef) -> PodcastEpisodeRef {
|
||||
let mut episode = episode.clone();
|
||||
episode.progress_generation = None;
|
||||
episode
|
||||
}
|
||||
|
||||
impl PodcastStore {
|
||||
/// Subscribes to the podcast of `podcast.feed_url` (explicit user action).
|
||||
///
|
||||
/// The URL is resolved against known feed URLs and aliases first, so
|
||||
/// subscribing again, through a redirect target or from someone else's
|
||||
/// card reuses the existing podcast with its progress and history. When
|
||||
/// the podcast is already subscribed nothing is emitted. Otherwise one
|
||||
/// `podcast_subscription_set` with generation
|
||||
/// `next_podcast_generation(..)` and the known aliases is applied and
|
||||
/// returned. The new subscription is private until
|
||||
/// [`PodcastStore::set_shared`].
|
||||
///
|
||||
/// Pass the channel metadata of the fetched feed
|
||||
/// (`ParsedFeed::to_podcast_meta(url)`), then ingest the episodes with
|
||||
/// [`PodcastStore::apply_feed`].
|
||||
pub fn subscribe(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
podcast: &PodcastMeta,
|
||||
) -> Result<SubscribeOutcome> {
|
||||
let meta = podcast.bounded();
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
let podcast_id = resolve_podcast(&tx, &meta.feed_url)?;
|
||||
let current = apply::load_subscription_register(&tx, &podcast_id)?;
|
||||
if current.as_ref().is_some_and(|record| record.subscribed) {
|
||||
return Ok(SubscribeOutcome {
|
||||
podcast_id,
|
||||
already_subscribed: true,
|
||||
ops: Vec::new(),
|
||||
});
|
||||
}
|
||||
let share = apply::load_share(&tx, &podcast_id)?;
|
||||
let generation = next_podcast_generation(current.as_ref(), share.as_ref());
|
||||
let aliases = merge_feed_aliases(
|
||||
&podcast_id,
|
||||
&apply::load_aliases(&tx, &podcast_id)?,
|
||||
std::slice::from_ref(&meta.feed_url),
|
||||
);
|
||||
let op = emit(&tx, stamps, |_| SyncOpPayload::PodcastSubscriptionSet {
|
||||
podcast_id: podcast_id.clone(),
|
||||
subscribed: true,
|
||||
podcast: meta.clone(),
|
||||
feed_aliases: aliases.clone(),
|
||||
generation,
|
||||
})?;
|
||||
tx.commit()?;
|
||||
Ok(SubscribeOutcome {
|
||||
podcast_id,
|
||||
already_subscribed: false,
|
||||
ops: vec![op],
|
||||
})
|
||||
}
|
||||
|
||||
/// Unsubscribes (explicit user action).
|
||||
///
|
||||
/// Emits `podcast_subscription_set { subscribed: false }` followed by
|
||||
/// `podcast_share_set { shared: false }`, each with the next podcast
|
||||
/// generation. Progress, sessions and the catalog are kept, so
|
||||
/// subscribing again continues where the user stopped. Does nothing when
|
||||
/// the podcast is not subscribed.
|
||||
pub fn unsubscribe(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
podcast_id: &PodcastId,
|
||||
) -> Result<Vec<PodcastLocalOp>> {
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
let Some(current) = apply::load_subscription_register(&tx, podcast_id)? else {
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
if !current.subscribed {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let share = apply::load_share(&tx, podcast_id)?;
|
||||
let generation = next_podcast_generation(Some(¤t), share.as_ref());
|
||||
let mut meta = catalog_meta(&tx, podcast_id)?
|
||||
.map(|meta| meta.bounded())
|
||||
.unwrap_or_else(|| current.podcast.clone());
|
||||
meta.feed_url = current.podcast.feed_url.clone();
|
||||
let aliases = current.feed_aliases.clone();
|
||||
let unsubscribe = emit(&tx, stamps, |_| SyncOpPayload::PodcastSubscriptionSet {
|
||||
podcast_id: podcast_id.clone(),
|
||||
subscribed: false,
|
||||
podcast: meta.clone(),
|
||||
feed_aliases: aliases.clone(),
|
||||
generation,
|
||||
})?;
|
||||
let revoke_payload = share_payload(&tx, podcast_id, false)?;
|
||||
let revoke = emit(&tx, stamps, |_| revoke_payload.clone())?;
|
||||
tx.commit()?;
|
||||
Ok(vec![unsubscribe, revoke])
|
||||
}
|
||||
|
||||
/// Sets the decision to publish the podcast to the federation.
|
||||
///
|
||||
/// Sharing needs an active subscription and is a no-op when the podcast
|
||||
/// is already effectively shared (`effective_share`). Revoking is a no-op
|
||||
/// when no share decision says `true`. The write uses the next podcast
|
||||
/// generation, so it orders after the subscription it observed.
|
||||
pub fn set_shared(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
podcast_id: &PodcastId,
|
||||
shared: bool,
|
||||
) -> Result<Vec<PodcastLocalOp>> {
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
let subscription = apply::load_subscription_register(&tx, podcast_id)?;
|
||||
let share = apply::load_share(&tx, podcast_id)?;
|
||||
if shared {
|
||||
if !subscription
|
||||
.as_ref()
|
||||
.is_some_and(|record| record.subscribed)
|
||||
{
|
||||
return Err(PodcastStoreError::NotSubscribed(podcast_id.clone()).into());
|
||||
}
|
||||
if effective_share(subscription.as_ref(), share.as_ref()) {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
} else if !share.as_ref().is_some_and(|record| record.shared) {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let payload = share_payload(&tx, podcast_id, shared)?;
|
||||
let op = emit(&tx, stamps, |_| payload.clone())?;
|
||||
tx.commit()?;
|
||||
Ok(vec![op])
|
||||
}
|
||||
|
||||
fn explicit_progress(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
episode: &PodcastEpisodeRef,
|
||||
session_id: Option<&str>,
|
||||
carried_generation: Option<u64>,
|
||||
action: ExplicitAction,
|
||||
) -> Result<ProgressWrite> {
|
||||
episode.validate().map_err(invalid)?;
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
let current = apply::load_progress(&tx, &episode.episode_id)?;
|
||||
let max_seen = current
|
||||
.as_ref()
|
||||
.map(|record| record.generation)
|
||||
.into_iter()
|
||||
.chain(carried_generation)
|
||||
.max();
|
||||
let generation = ProgressRecord::explicit_generation(max_seen);
|
||||
let current_position = current.as_ref().map_or(0, |record| record.position_ms);
|
||||
let (position_ms, status) = match action {
|
||||
ExplicitAction::Start(position) | ExplicitAction::Seek(position) => {
|
||||
(position, ProgressStatus::InProgress)
|
||||
}
|
||||
ExplicitAction::Restart => (0, ProgressStatus::InProgress),
|
||||
ExplicitAction::Complete(position) => (position, ProgressStatus::Completed),
|
||||
ExplicitAction::MarkPlayed => (current_position, ProgressStatus::Completed),
|
||||
ExplicitAction::MarkUnplayed => (0, ProgressStatus::NotStarted),
|
||||
};
|
||||
let reference = op_ref(episode);
|
||||
let op = emit(&tx, stamps, |stamp| SyncOpPayload::PodcastProgressSet {
|
||||
episode_id: episode.episode_id.clone(),
|
||||
podcast_id: episode.podcast_id.clone(),
|
||||
position_ms,
|
||||
status,
|
||||
generation,
|
||||
session_id: session_id.unwrap_or(&stamp.op_id).to_string(),
|
||||
episode: Some(reference.clone()),
|
||||
})?;
|
||||
tx.commit()?;
|
||||
let wire = op.stamp.wire(op.payload.clone());
|
||||
let record = wire.podcast_progress().expect("progress payload");
|
||||
Ok(ProgressWrite {
|
||||
session: ProgressSession {
|
||||
episode: episode.clone(),
|
||||
session_id: record.session_id.clone(),
|
||||
generation,
|
||||
},
|
||||
record,
|
||||
op,
|
||||
})
|
||||
}
|
||||
|
||||
/// Starts a listening session at `position_ms` (explicit action).
|
||||
///
|
||||
/// The generation is `explicit_generation(max(local, carried_generation))`;
|
||||
/// pass `PodcastEpisodeRef::progress_generation` of a queue item received
|
||||
/// through a handoff as `carried_generation`. Ordinary Play resumes from
|
||||
/// [`PodcastStore::resume_position_ms`].
|
||||
pub fn start_session(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
episode: &PodcastEpisodeRef,
|
||||
session_id: &str,
|
||||
position_ms: i64,
|
||||
carried_generation: Option<u64>,
|
||||
) -> Result<ProgressWrite> {
|
||||
self.explicit_progress(
|
||||
stamps,
|
||||
episode,
|
||||
Some(session_id),
|
||||
carried_generation,
|
||||
ExplicitAction::Start(position_ms),
|
||||
)
|
||||
}
|
||||
|
||||
/// Periodic checkpoint (or pause/stop save) of a running session.
|
||||
///
|
||||
/// Keeps the session generation. When the register already holds a
|
||||
/// greater generation nothing is written ([`CheckpointOutcome::Superseded`]).
|
||||
pub fn checkpoint(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
session: &ProgressSession,
|
||||
position_ms: i64,
|
||||
) -> Result<CheckpointOutcome> {
|
||||
let episode = &session.episode;
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
if let Some(current) = apply::load_progress(&tx, &episode.episode_id)? {
|
||||
if current.generation > session.generation {
|
||||
return Ok(CheckpointOutcome::Superseded { current });
|
||||
}
|
||||
if current.generation == session.generation
|
||||
&& current.session_id == session.session_id
|
||||
&& current.status == ProgressStatus::Completed
|
||||
{
|
||||
return Ok(CheckpointOutcome::AlreadyCompleted { current });
|
||||
}
|
||||
}
|
||||
let reference = op_ref(episode);
|
||||
let op = emit(&tx, stamps, |_| SyncOpPayload::PodcastProgressSet {
|
||||
episode_id: episode.episode_id.clone(),
|
||||
podcast_id: episode.podcast_id.clone(),
|
||||
position_ms,
|
||||
status: ProgressStatus::InProgress,
|
||||
generation: session.generation,
|
||||
session_id: session.session_id.clone(),
|
||||
episode: Some(reference.clone()),
|
||||
})?;
|
||||
tx.commit()?;
|
||||
let record = op
|
||||
.stamp
|
||||
.wire(op.payload.clone())
|
||||
.podcast_progress()
|
||||
.expect("progress payload");
|
||||
Ok(CheckpointOutcome::Written(Box::new(ProgressWrite {
|
||||
record,
|
||||
session: session.clone(),
|
||||
op,
|
||||
})))
|
||||
}
|
||||
|
||||
/// User seek during a session (explicit action; the returned session
|
||||
/// carries the new generation for later checkpoints).
|
||||
pub fn seek(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
session: &ProgressSession,
|
||||
position_ms: i64,
|
||||
) -> Result<ProgressWrite> {
|
||||
self.explicit_progress(
|
||||
stamps,
|
||||
&session.episode,
|
||||
Some(&session.session_id),
|
||||
Some(session.generation),
|
||||
ExplicitAction::Seek(position_ms),
|
||||
)
|
||||
}
|
||||
|
||||
/// The session reached the end (`reaches_completion` or end of stream):
|
||||
/// writes `completed` with an explicit generation. Record the session
|
||||
/// itself with [`PodcastStore::record_listen_session`].
|
||||
pub fn complete_session(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
session: &ProgressSession,
|
||||
position_ms: i64,
|
||||
) -> Result<ProgressWrite> {
|
||||
self.explicit_progress(
|
||||
stamps,
|
||||
&session.episode,
|
||||
Some(&session.session_id),
|
||||
Some(session.generation),
|
||||
ExplicitAction::Complete(position_ms),
|
||||
)
|
||||
}
|
||||
|
||||
/// "Start over": position 0, in progress, explicit generation.
|
||||
pub fn restart(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
episode: &PodcastEpisodeRef,
|
||||
session_id: &str,
|
||||
) -> Result<ProgressWrite> {
|
||||
self.explicit_progress(
|
||||
stamps,
|
||||
episode,
|
||||
Some(session_id),
|
||||
None,
|
||||
ExplicitAction::Restart,
|
||||
)
|
||||
}
|
||||
|
||||
/// "Mark played": status completed, explicit generation. Creates no
|
||||
/// listening session and invents no listening time.
|
||||
pub fn mark_played(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
episode: &PodcastEpisodeRef,
|
||||
) -> Result<ProgressWrite> {
|
||||
self.explicit_progress(stamps, episode, None, None, ExplicitAction::MarkPlayed)
|
||||
}
|
||||
|
||||
/// "Mark unplayed": status not started at position 0, explicit generation.
|
||||
pub fn mark_unplayed(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
episode: &PodcastEpisodeRef,
|
||||
) -> Result<ProgressWrite> {
|
||||
self.explicit_progress(stamps, episode, None, None, ExplicitAction::MarkUnplayed)
|
||||
}
|
||||
|
||||
/// Records (or grows) a listening session of this device.
|
||||
///
|
||||
/// Sessions below the history threshold (`ListenSession::should_record`)
|
||||
/// and versions that do not grow the stored session emit nothing.
|
||||
/// `session.device_id` must be the local device id. The episode reference
|
||||
/// defaults to the catalog's.
|
||||
pub fn record_listen_session(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
session: &ListenSession,
|
||||
episode: Option<&PodcastEpisodeRef>,
|
||||
) -> Result<Option<PodcastLocalOp>> {
|
||||
if session.device_id != stamps.device_id() {
|
||||
return Err(invalid(
|
||||
"session device_id differs from the local device id",
|
||||
));
|
||||
}
|
||||
session.validate().map_err(invalid)?;
|
||||
if !session.should_record() {
|
||||
return Ok(None);
|
||||
}
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
if let Some(current) = apply::load_session(&tx, &session.session_id)? {
|
||||
let merged = merge_listen_session(¤t, session).map_err(invalid)?;
|
||||
if merged == current {
|
||||
return Ok(None);
|
||||
}
|
||||
}
|
||||
let reference = match episode {
|
||||
Some(episode) => Some(op_ref(episode)),
|
||||
None => super::queries::episode_ref_in(&tx, &session.episode_id)?.map(|r| op_ref(&r)),
|
||||
};
|
||||
let op = emit(&tx, stamps, |_| SyncOpPayload::PodcastListenRecorded {
|
||||
session: session.clone(),
|
||||
episode: reference.clone(),
|
||||
})?;
|
||||
tx.commit()?;
|
||||
Ok(Some(op))
|
||||
}
|
||||
|
||||
/// Resolves a feed URL to a podcast id: the existing podcast when the
|
||||
/// URL is a known feed URL or alias (`resolve_podcast_id`), otherwise
|
||||
/// the id a first subscription through this URL would derive.
|
||||
pub fn resolve_podcast_id(&self, url: &FeedUrl) -> Result<PodcastId> {
|
||||
resolve_podcast(&self.lock(), url)
|
||||
}
|
||||
|
||||
/// The episode reference for queueing or progress actions, or `None`
|
||||
/// when the episode is unknown.
|
||||
pub fn episode_ref(&self, episode_id: &EpisodeId) -> Result<Option<PodcastEpisodeRef>> {
|
||||
super::queries::episode_ref_in(&self.lock(), episode_id)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,742 @@
|
||||
//! The single code path that applies podcast records to the store.
|
||||
//!
|
||||
//! Remote operations, local actions and snapshot sections all end here.
|
||||
//! Records come from the `SyncOpWire` materializers of
|
||||
//! `music_dht::device_sync` and are merged with the functions of
|
||||
//! `music_dht::podcast`, so applying is idempotent and order-independent.
|
||||
|
||||
use anyhow::Result;
|
||||
use music_dht::device_sync::{SyncOpPayload, SyncOpWire, SyncSnapshot};
|
||||
use music_dht::podcast::{
|
||||
EpisodeIdentityHints, EpisodeMeta, FeedUrl, KnownEpisode, ListenSession, PodcastEpisodeRef,
|
||||
PodcastError, PodcastId, ProgressRecord, ShareRecord, SubscriptionRecord, merge_feed_aliases,
|
||||
merge_listen_session,
|
||||
};
|
||||
use rusqlite::{Connection, OptionalExtension as _, TransactionBehavior, params};
|
||||
|
||||
use super::{
|
||||
PROGRESS_COLUMNS, PodcastStore, PodcastStoreError, SESSION_COLUMNS, episode_id_col,
|
||||
feed_url_col, generation_to_sql, now_ms, podcast_meta_col, progress_from_row, session_from_row,
|
||||
status_to_sql,
|
||||
};
|
||||
|
||||
/// Result of applying one operation.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
#[non_exhaustive]
|
||||
pub enum ApplyOutcome {
|
||||
/// Stored state changed.
|
||||
Changed,
|
||||
/// The operation was already reflected (duplicate, or older than the
|
||||
/// stored register).
|
||||
Unchanged,
|
||||
/// Not a podcast operation; nothing to do here.
|
||||
Ignored,
|
||||
/// The operation violates the contract and was skipped. Acknowledge it
|
||||
/// anyway so the log keeps moving (`PODCASTS.md`).
|
||||
Rejected(String),
|
||||
}
|
||||
|
||||
/// Counters returned by [`PodcastStore::apply_snapshot`].
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub struct SnapshotApplyStats {
|
||||
/// Records that changed stored state.
|
||||
pub changed: usize,
|
||||
/// Valid records already reflected locally.
|
||||
pub unchanged: usize,
|
||||
/// Records that failed validation and were skipped.
|
||||
pub rejected: usize,
|
||||
/// Whether the podcast sections were ignored for exceeding the counts.
|
||||
pub ignored_over_limits: bool,
|
||||
}
|
||||
|
||||
/// Where an episode row came from.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub(crate) enum EpisodeSource {
|
||||
/// Parsed from the podcast's feed on this device.
|
||||
Feed,
|
||||
/// Learned from a `PodcastEpisodeRef` (sync operation or queue item).
|
||||
Sync,
|
||||
}
|
||||
|
||||
impl EpisodeSource {
|
||||
fn as_sql(self) -> &'static str {
|
||||
match self {
|
||||
Self::Feed => "feed",
|
||||
Self::Sync => "sync",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn invalid(error: impl ToString) -> anyhow::Error {
|
||||
PodcastStoreError::Invalid(error.to_string()).into()
|
||||
}
|
||||
|
||||
pub(crate) fn load_subscription_register(
|
||||
conn: &Connection,
|
||||
podcast_id: &PodcastId,
|
||||
) -> Result<Option<SubscriptionRecord>> {
|
||||
let row = conn
|
||||
.query_row(
|
||||
"SELECT subscribed, podcast_json, generation, hlc_ms, op_id
|
||||
FROM podcast_subscriptions WHERE podcast_id = ?1",
|
||||
[podcast_id.as_str()],
|
||||
|row| {
|
||||
let generation: i64 = row.get(2)?;
|
||||
Ok(SubscriptionRecord {
|
||||
podcast_id: podcast_id.clone(),
|
||||
subscribed: row.get(0)?,
|
||||
podcast: podcast_meta_col(&row.get::<_, String>(1)?)?,
|
||||
feed_aliases: Vec::new(),
|
||||
generation: u64::try_from(generation).unwrap_or(0),
|
||||
hlc_ms: row.get(3)?,
|
||||
op_id: row.get(4)?,
|
||||
})
|
||||
},
|
||||
)
|
||||
.optional()?;
|
||||
let Some(mut record) = row else {
|
||||
return Ok(None);
|
||||
};
|
||||
record.feed_aliases = load_aliases(conn, podcast_id)?;
|
||||
Ok(Some(record))
|
||||
}
|
||||
|
||||
pub(crate) fn load_aliases(conn: &Connection, podcast_id: &PodcastId) -> Result<Vec<FeedUrl>> {
|
||||
let mut statement = conn.prepare_cached(
|
||||
"SELECT feed_url FROM podcast_feed_aliases WHERE podcast_id = ?1 ORDER BY feed_url",
|
||||
)?;
|
||||
let urls = statement
|
||||
.query_map([podcast_id.as_str()], |row| feed_url_col(row.get(0)?))?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
Ok(urls)
|
||||
}
|
||||
|
||||
pub(crate) fn load_share(conn: &Connection, podcast_id: &PodcastId) -> Result<Option<ShareRecord>> {
|
||||
Ok(conn
|
||||
.query_row(
|
||||
"SELECT shared, generation, hlc_ms, op_id FROM podcast_shares WHERE podcast_id = ?1",
|
||||
[podcast_id.as_str()],
|
||||
|row| {
|
||||
let generation: i64 = row.get(1)?;
|
||||
Ok(ShareRecord {
|
||||
podcast_id: podcast_id.clone(),
|
||||
shared: row.get(0)?,
|
||||
generation: u64::try_from(generation).unwrap_or(0),
|
||||
hlc_ms: row.get(2)?,
|
||||
op_id: row.get(3)?,
|
||||
})
|
||||
},
|
||||
)
|
||||
.optional()?)
|
||||
}
|
||||
|
||||
pub(crate) fn load_progress(
|
||||
conn: &Connection,
|
||||
episode_id: &music_dht::podcast::EpisodeId,
|
||||
) -> Result<Option<ProgressRecord>> {
|
||||
Ok(conn
|
||||
.query_row(
|
||||
&format!("SELECT {PROGRESS_COLUMNS} FROM episode_progress p WHERE p.episode_id = ?1"),
|
||||
[episode_id.as_str()],
|
||||
|row| progress_from_row(row, 0),
|
||||
)
|
||||
.optional()?)
|
||||
}
|
||||
|
||||
pub(crate) fn load_session(conn: &Connection, session_id: &str) -> Result<Option<ListenSession>> {
|
||||
Ok(conn
|
||||
.query_row(
|
||||
&format!(
|
||||
"SELECT {SESSION_COLUMNS} FROM podcast_listen_sessions s WHERE s.session_id = ?1"
|
||||
),
|
||||
[session_id],
|
||||
|row| session_from_row(row, 0),
|
||||
)
|
||||
.optional()?)
|
||||
}
|
||||
|
||||
/// Replaces the alias set of a podcast with `merged` when it differs.
|
||||
fn store_aliases(conn: &Connection, podcast_id: &PodcastId, merged: &[FeedUrl]) -> Result<bool> {
|
||||
let current = load_aliases(conn, podcast_id)?;
|
||||
if current == merged {
|
||||
return Ok(false);
|
||||
}
|
||||
conn.execute(
|
||||
"DELETE FROM podcast_feed_aliases WHERE podcast_id = ?1",
|
||||
[podcast_id.as_str()],
|
||||
)?;
|
||||
let mut insert = conn.prepare_cached(
|
||||
"INSERT INTO podcast_feed_aliases (podcast_id, feed_url) VALUES (?1, ?2)",
|
||||
)?;
|
||||
for url in merged {
|
||||
insert.execute(params![podcast_id.as_str(), url.as_str()])?;
|
||||
}
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// Merges feed URLs into the bounded grow-only alias set of a podcast.
|
||||
///
|
||||
/// Returns whether the stored set changed.
|
||||
pub(crate) fn merge_aliases(
|
||||
conn: &Connection,
|
||||
podcast_id: &PodcastId,
|
||||
urls: &[FeedUrl],
|
||||
) -> Result<bool> {
|
||||
let current = load_aliases(conn, podcast_id)?;
|
||||
let merged = merge_feed_aliases(podcast_id, ¤t, urls);
|
||||
store_aliases(conn, podcast_id, &merged)
|
||||
}
|
||||
|
||||
/// Inserts the catalog row of a podcast if it does not exist yet.
|
||||
fn ensure_podcast_row(
|
||||
conn: &Connection,
|
||||
podcast_id: &PodcastId,
|
||||
feed_url: &FeedUrl,
|
||||
title: &str,
|
||||
now: i64,
|
||||
) -> Result<()> {
|
||||
conn.execute(
|
||||
"INSERT OR IGNORE INTO podcasts
|
||||
(podcast_id, feed_url, title, meta_from_feed, created_at_ms, updated_at_ms)
|
||||
VALUES (?1, ?2, ?3, 0, ?4, ?4)",
|
||||
params![podcast_id.as_str(), feed_url.as_str(), title, now],
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Applies a subscription register record (operation or snapshot).
|
||||
///
|
||||
/// The register is last-writer-wins ([`SubscriptionRecord::supersedes`]);
|
||||
/// the alias set is a bounded grow-only set merged independently, so the
|
||||
/// stored aliases are the bounded union of every URL seen (equal to what
|
||||
/// `merge_subscription` computes). The catalog row is created from the
|
||||
/// register metadata and follows the winning register until this device has
|
||||
/// refreshed the feed itself.
|
||||
pub(crate) fn apply_subscription_record(
|
||||
conn: &Connection,
|
||||
record: &SubscriptionRecord,
|
||||
now: i64,
|
||||
) -> Result<bool> {
|
||||
generation_to_sql(record.generation)?;
|
||||
let podcast_id = &record.podcast_id;
|
||||
let current = load_subscription_register(conn, podcast_id)?;
|
||||
let wins = current
|
||||
.as_ref()
|
||||
.is_none_or(|current| record.supersedes(current));
|
||||
let mut changed = false;
|
||||
if wins {
|
||||
conn.execute(
|
||||
"INSERT INTO podcast_subscriptions
|
||||
(podcast_id, subscribed, podcast_json, generation, hlc_ms, op_id)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6)
|
||||
ON CONFLICT(podcast_id) DO UPDATE SET
|
||||
subscribed = excluded.subscribed, podcast_json = excluded.podcast_json,
|
||||
generation = excluded.generation, hlc_ms = excluded.hlc_ms,
|
||||
op_id = excluded.op_id",
|
||||
params![
|
||||
podcast_id.as_str(),
|
||||
record.subscribed,
|
||||
serde_json::to_string(&record.podcast)?,
|
||||
generation_to_sql(record.generation)?,
|
||||
record.hlc_ms,
|
||||
record.op_id,
|
||||
],
|
||||
)?;
|
||||
changed = true;
|
||||
}
|
||||
let mut urls = record.feed_aliases.clone();
|
||||
urls.push(record.podcast.feed_url.clone());
|
||||
changed |= merge_aliases(conn, podcast_id, &urls)?;
|
||||
|
||||
ensure_podcast_row(
|
||||
conn,
|
||||
podcast_id,
|
||||
&record.podcast.feed_url,
|
||||
&record.podcast.title,
|
||||
now,
|
||||
)?;
|
||||
if wins {
|
||||
let meta = &record.podcast;
|
||||
conn.execute(
|
||||
"UPDATE podcasts
|
||||
SET feed_url = ?2, title = ?3, author = ?4,
|
||||
description = COALESCE(?5, description), link = ?6, image_url = ?7,
|
||||
updated_at_ms = ?8
|
||||
WHERE podcast_id = ?1 AND meta_from_feed = 0",
|
||||
params![
|
||||
podcast_id.as_str(),
|
||||
meta.feed_url.as_str(),
|
||||
meta.title,
|
||||
meta.author,
|
||||
meta.description,
|
||||
meta.link,
|
||||
meta.image_url,
|
||||
now
|
||||
],
|
||||
)?;
|
||||
}
|
||||
Ok(changed)
|
||||
}
|
||||
|
||||
/// Applies a share register record.
|
||||
pub(crate) fn apply_share_record(conn: &Connection, record: &ShareRecord) -> Result<bool> {
|
||||
let generation = generation_to_sql(record.generation)?;
|
||||
let current = load_share(conn, &record.podcast_id)?;
|
||||
if current
|
||||
.as_ref()
|
||||
.is_some_and(|current| !record.supersedes(current))
|
||||
{
|
||||
return Ok(false);
|
||||
}
|
||||
conn.execute(
|
||||
"INSERT INTO podcast_shares (podcast_id, shared, generation, hlc_ms, op_id)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5)
|
||||
ON CONFLICT(podcast_id) DO UPDATE SET
|
||||
shared = excluded.shared, generation = excluded.generation,
|
||||
hlc_ms = excluded.hlc_ms, op_id = excluded.op_id",
|
||||
params![
|
||||
record.podcast_id.as_str(),
|
||||
record.shared,
|
||||
generation,
|
||||
record.hlc_ms,
|
||||
record.op_id
|
||||
],
|
||||
)?;
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// Applies a progress register record. Never touches the catalog.
|
||||
pub(crate) fn apply_progress_record(conn: &Connection, record: &ProgressRecord) -> Result<bool> {
|
||||
let generation = generation_to_sql(record.generation)?;
|
||||
let current = load_progress(conn, &record.episode_id)?;
|
||||
if current
|
||||
.as_ref()
|
||||
.is_some_and(|current| !record.supersedes(current))
|
||||
{
|
||||
return Ok(false);
|
||||
}
|
||||
conn.execute(
|
||||
"INSERT INTO episode_progress
|
||||
(episode_id, podcast_id, position_ms, status, generation, session_id, hlc_ms,
|
||||
device_id, op_id)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)
|
||||
ON CONFLICT(episode_id) DO UPDATE SET
|
||||
podcast_id = excluded.podcast_id, position_ms = excluded.position_ms,
|
||||
status = excluded.status, generation = excluded.generation,
|
||||
session_id = excluded.session_id, hlc_ms = excluded.hlc_ms,
|
||||
device_id = excluded.device_id, op_id = excluded.op_id",
|
||||
params![
|
||||
record.episode_id.as_str(),
|
||||
record.podcast_id.as_str(),
|
||||
record.position_ms,
|
||||
status_to_sql(record.status),
|
||||
generation,
|
||||
record.session_id,
|
||||
record.hlc_ms,
|
||||
record.device_id,
|
||||
record.op_id,
|
||||
],
|
||||
)?;
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// Adds or grows a listening session (grow-only set keyed by `session_id`).
|
||||
pub(crate) fn apply_listen_session(conn: &Connection, session: &ListenSession) -> Result<bool> {
|
||||
let current = load_session(conn, &session.session_id)?;
|
||||
let merged = match ¤t {
|
||||
Some(current) => merge_listen_session(current, session).map_err(invalid)?,
|
||||
None => session.clone(),
|
||||
};
|
||||
if current.as_ref() == Some(&merged) {
|
||||
return Ok(false);
|
||||
}
|
||||
conn.execute(
|
||||
"INSERT INTO podcast_listen_sessions
|
||||
(session_id, episode_id, podcast_id, device_id, started_at_ms, ended_at_ms,
|
||||
start_position_ms, end_position_ms, listened_ms, completed)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10)
|
||||
ON CONFLICT(session_id) DO UPDATE SET
|
||||
started_at_ms = excluded.started_at_ms, ended_at_ms = excluded.ended_at_ms,
|
||||
start_position_ms = excluded.start_position_ms,
|
||||
end_position_ms = excluded.end_position_ms,
|
||||
listened_ms = excluded.listened_ms, completed = excluded.completed",
|
||||
params![
|
||||
merged.session_id,
|
||||
merged.episode_id.as_str(),
|
||||
merged.podcast_id.as_str(),
|
||||
merged.device_id,
|
||||
merged.started_at_ms,
|
||||
merged.ended_at_ms,
|
||||
merged.start_position_ms,
|
||||
merged.end_position_ms,
|
||||
merged.listened_ms,
|
||||
merged.completed,
|
||||
],
|
||||
)?;
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// Inserts an episode row.
|
||||
pub(crate) fn insert_episode(
|
||||
conn: &Connection,
|
||||
meta: &EpisodeMeta,
|
||||
source: EpisodeSource,
|
||||
in_feed: bool,
|
||||
legacy_guid: Option<&str>,
|
||||
first_seen_at_ms: i64,
|
||||
now: i64,
|
||||
) -> Result<()> {
|
||||
conn.execute(
|
||||
"INSERT INTO episodes
|
||||
(episode_id, podcast_id, guid, title, description, season, episode_number,
|
||||
episode_type, published_at_ms, duration_ms, audio_url, mime, byte_len,
|
||||
image_url, link, source, in_feed, legacy_guid, first_seen_at_ms, updated_at_ms)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17,
|
||||
?18, ?19, ?20)",
|
||||
params![
|
||||
meta.episode_id.as_str(),
|
||||
meta.podcast_id.as_str(),
|
||||
meta.guid,
|
||||
meta.title,
|
||||
meta.description,
|
||||
meta.season,
|
||||
meta.episode_number,
|
||||
episode_type_to_sql(meta.episode_type),
|
||||
meta.published_at_ms,
|
||||
meta.duration_ms,
|
||||
meta.audio_url,
|
||||
meta.mime,
|
||||
meta.byte_len.and_then(|len| i64::try_from(len).ok()),
|
||||
meta.image_url,
|
||||
meta.link,
|
||||
source.as_sql(),
|
||||
in_feed,
|
||||
legacy_guid,
|
||||
first_seen_at_ms,
|
||||
now,
|
||||
],
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) fn episode_type_to_sql(kind: music_dht::podcast::EpisodeType) -> &'static str {
|
||||
match kind {
|
||||
music_dht::podcast::EpisodeType::Full => "full",
|
||||
music_dht::podcast::EpisodeType::Trailer => "trailer",
|
||||
music_dht::podcast::EpisodeType::Bonus => "bonus",
|
||||
}
|
||||
}
|
||||
|
||||
/// Records an episode learned through sync when the catalog does not know it.
|
||||
///
|
||||
/// Feed metadata always wins: an existing row is left untouched. The
|
||||
/// podcast catalog row is created from the reference when missing (no
|
||||
/// register is written).
|
||||
pub(crate) fn ensure_episode_from_ref(
|
||||
conn: &Connection,
|
||||
episode: &PodcastEpisodeRef,
|
||||
) -> Result<()> {
|
||||
let exists = conn
|
||||
.query_row(
|
||||
"SELECT 1 FROM episodes WHERE episode_id = ?1",
|
||||
[episode.episode_id.as_str()],
|
||||
|_| Ok(()),
|
||||
)
|
||||
.optional()?
|
||||
.is_some();
|
||||
if exists {
|
||||
return Ok(());
|
||||
}
|
||||
let now = now_ms();
|
||||
ensure_podcast_row(
|
||||
conn,
|
||||
&episode.podcast_id,
|
||||
&episode.feed_url,
|
||||
&episode.podcast_title,
|
||||
now,
|
||||
)?;
|
||||
let meta = EpisodeMeta {
|
||||
episode_id: episode.episode_id.clone(),
|
||||
podcast_id: episode.podcast_id.clone(),
|
||||
guid: None,
|
||||
title: episode.episode_title.clone(),
|
||||
description: None,
|
||||
season: episode.season,
|
||||
episode_number: episode.episode_number,
|
||||
episode_type: Default::default(),
|
||||
published_at_ms: episode.published_at_ms,
|
||||
duration_ms: episode.duration_ms,
|
||||
audio_url: episode.audio_url.clone(),
|
||||
mime: episode.mime.clone(),
|
||||
byte_len: None,
|
||||
image_url: episode.image_url.clone(),
|
||||
link: None,
|
||||
};
|
||||
insert_episode(conn, &meta, EpisodeSource::Sync, false, None, now, now)?;
|
||||
refresh_seasons(conn, &episode.podcast_id)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Recomputes the derived season rows of a podcast from its episodes.
|
||||
pub(crate) fn refresh_seasons(conn: &Connection, podcast_id: &PodcastId) -> Result<()> {
|
||||
conn.execute(
|
||||
"DELETE FROM podcast_seasons WHERE podcast_id = ?1",
|
||||
[podcast_id.as_str()],
|
||||
)?;
|
||||
conn.execute(
|
||||
"INSERT INTO podcast_seasons
|
||||
(podcast_id, season, episode_count, first_published_at_ms, last_published_at_ms)
|
||||
SELECT podcast_id, season, COUNT(*), MIN(published_at_ms), MAX(published_at_ms)
|
||||
FROM episodes WHERE podcast_id = ?1 GROUP BY season",
|
||||
[podcast_id.as_str()],
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Known episodes of a podcast with the identity hints they were last seen with.
|
||||
pub(crate) fn known_episodes(
|
||||
conn: &Connection,
|
||||
podcast_id: &PodcastId,
|
||||
) -> Result<Vec<KnownEpisode>> {
|
||||
let mut statement = conn.prepare_cached(
|
||||
"SELECT episode_id, guid, audio_url, link, published_at_ms, title
|
||||
FROM episodes WHERE podcast_id = ?1 ORDER BY episode_id",
|
||||
)?;
|
||||
let known = statement
|
||||
.query_map([podcast_id.as_str()], |row| {
|
||||
Ok(KnownEpisode {
|
||||
episode_id: episode_id_col(row.get(0)?)?,
|
||||
hints: EpisodeIdentityHints {
|
||||
guid: row.get(1)?,
|
||||
enclosure_url: Some(row.get(2)?),
|
||||
link: row.get(3)?,
|
||||
published_at_ms: row.get(4)?,
|
||||
title: Some(row.get(5)?),
|
||||
},
|
||||
})
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
Ok(known)
|
||||
}
|
||||
|
||||
/// Applies one podcast operation inside the caller's transaction.
|
||||
pub(crate) fn apply_op(conn: &Connection, op: &SyncOpWire) -> Result<ApplyOutcome> {
|
||||
if !op.payload.is_podcast() {
|
||||
return Ok(ApplyOutcome::Ignored);
|
||||
}
|
||||
if let Err(error) = op.validate_podcast() {
|
||||
return Ok(ApplyOutcome::Rejected(error.to_string()));
|
||||
}
|
||||
let result = match &op.payload {
|
||||
SyncOpPayload::PodcastSubscriptionSet { .. } => {
|
||||
let record = op.podcast_subscription().expect("subscription payload");
|
||||
apply_subscription_record(conn, &record, now_ms())
|
||||
}
|
||||
SyncOpPayload::PodcastShareSet { .. } => {
|
||||
apply_share_record(conn, &op.podcast_share().expect("share payload"))
|
||||
}
|
||||
SyncOpPayload::PodcastFeedAliasesAdded { .. } => {
|
||||
let (podcast_id, urls) = op.podcast_feed_aliases().expect("alias payload");
|
||||
merge_aliases(conn, podcast_id, urls)
|
||||
}
|
||||
SyncOpPayload::PodcastProgressSet { episode, .. } => {
|
||||
let record = op.podcast_progress().expect("progress payload");
|
||||
if let Err(error) = generation_to_sql(record.generation) {
|
||||
return Ok(ApplyOutcome::Rejected(error.to_string()));
|
||||
}
|
||||
if let Some(episode) = episode {
|
||||
ensure_episode_from_ref(conn, episode)?;
|
||||
}
|
||||
apply_progress_record(conn, &record)
|
||||
}
|
||||
SyncOpPayload::PodcastListenRecorded { session, episode } => {
|
||||
if let Some(current) = load_session(conn, &session.session_id)?
|
||||
&& let Err(error) = merge_listen_session(¤t, session)
|
||||
{
|
||||
return Ok(ApplyOutcome::Rejected(error.to_string()));
|
||||
}
|
||||
if let Some(episode) = episode {
|
||||
ensure_episode_from_ref(conn, episode)?;
|
||||
}
|
||||
apply_listen_session(conn, session)
|
||||
}
|
||||
_ => return Ok(ApplyOutcome::Ignored),
|
||||
};
|
||||
match result {
|
||||
Ok(true) => Ok(ApplyOutcome::Changed),
|
||||
Ok(false) => Ok(ApplyOutcome::Unchanged),
|
||||
Err(error) => match error.downcast_ref::<PodcastStoreError>() {
|
||||
Some(PodcastStoreError::Invalid(reason)) => Ok(ApplyOutcome::Rejected(reason.clone())),
|
||||
_ => Err(error),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn record_outcome(stats: &mut SnapshotApplyStats, result: Result<bool>) -> Result<()> {
|
||||
match result {
|
||||
Ok(true) => stats.changed += 1,
|
||||
Ok(false) => stats.unchanged += 1,
|
||||
Err(error) => match error.downcast_ref::<PodcastStoreError>() {
|
||||
Some(PodcastStoreError::Invalid(_)) => stats.rejected += 1,
|
||||
_ => return Err(error),
|
||||
},
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn check(result: Result<(), PodcastError>) -> Result<()> {
|
||||
result.map_err(invalid)
|
||||
}
|
||||
|
||||
impl PodcastStore {
|
||||
/// Applies a device-sync operation received from a trusted device.
|
||||
///
|
||||
/// Non-podcast operations return [`ApplyOutcome::Ignored`]. Podcast
|
||||
/// operations are validated (`SyncOpWire::validate_podcast`) and merged
|
||||
/// with the contract functions in one transaction; applying the same
|
||||
/// operations in any order, with duplicates, gives the same state.
|
||||
pub fn apply_remote_op(&self, op: &SyncOpWire) -> Result<ApplyOutcome> {
|
||||
if !op.payload.is_podcast() {
|
||||
return Ok(ApplyOutcome::Ignored);
|
||||
}
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
let outcome = apply_op(&tx, op)?;
|
||||
tx.commit()?;
|
||||
Ok(outcome)
|
||||
}
|
||||
|
||||
/// Merges the podcast sections of a sync snapshot.
|
||||
///
|
||||
/// Sections exceeding the contract counts are ignored
|
||||
/// (`SyncSnapshot::podcast_state_within_limits`); every record is
|
||||
/// validated and merged with the same functions as operations, so a
|
||||
/// snapshot never replaces newer local state. One transaction.
|
||||
pub fn apply_snapshot(&self, snapshot: &SyncSnapshot) -> Result<SnapshotApplyStats> {
|
||||
let mut stats = SnapshotApplyStats::default();
|
||||
if !snapshot.podcast_state_within_limits() {
|
||||
stats.ignored_over_limits = true;
|
||||
return Ok(stats);
|
||||
}
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
let now = now_ms();
|
||||
for record in &snapshot.podcast_subscriptions {
|
||||
let mut record = record.clone();
|
||||
record.normalize();
|
||||
let result = check(record.validate())
|
||||
.and_then(|()| apply_subscription_record(&tx, &record, now));
|
||||
record_outcome(&mut stats, result)?;
|
||||
}
|
||||
for record in &snapshot.podcast_shares {
|
||||
let result = check(record.validate()).and_then(|()| apply_share_record(&tx, record));
|
||||
record_outcome(&mut stats, result)?;
|
||||
}
|
||||
for record in &snapshot.podcast_progress {
|
||||
let result = check(record.validate()).and_then(|()| apply_progress_record(&tx, record));
|
||||
record_outcome(&mut stats, result)?;
|
||||
}
|
||||
for session in &snapshot.podcast_listen_sessions {
|
||||
let result =
|
||||
check(session.validate()).and_then(|()| apply_listen_session(&tx, session));
|
||||
record_outcome(&mut stats, result)?;
|
||||
}
|
||||
tx.commit()?;
|
||||
Ok(stats)
|
||||
}
|
||||
|
||||
/// Fills the podcast sections of `snapshot` with this store's registers
|
||||
/// and sessions, bounded by `SyncSnapshot::bound_podcast_state_to`.
|
||||
///
|
||||
/// `budget_bytes` is usually `MAX_PODCAST_SNAPSHOT_BYTES` or
|
||||
/// `device_sync::podcast_snapshot_budget(other_bytes)`. Records are read
|
||||
/// newest first and capped by the per-section counts before the byte
|
||||
/// budget is applied. Local fetch state and the catalog are never part
|
||||
/// of the snapshot. Send it only to v4 peers (`SyncSnapshot::for_peer`).
|
||||
pub fn fill_snapshot(&self, snapshot: &mut SyncSnapshot, budget_bytes: usize) -> Result<()> {
|
||||
use music_dht::podcast::{
|
||||
MAX_SNAPSHOT_LISTEN_SESSIONS, MAX_SNAPSHOT_PROGRESS, MAX_SNAPSHOT_SHARES,
|
||||
MAX_SNAPSHOT_SUBSCRIPTIONS,
|
||||
};
|
||||
let conn = self.lock();
|
||||
let ids: Vec<PodcastId> = {
|
||||
let mut statement = conn.prepare(
|
||||
"SELECT podcast_id FROM podcast_subscriptions
|
||||
ORDER BY generation DESC, hlc_ms DESC, op_id DESC, podcast_id DESC
|
||||
LIMIT ?1",
|
||||
)?;
|
||||
statement
|
||||
.query_map([MAX_SNAPSHOT_SUBSCRIPTIONS as i64], |row| {
|
||||
super::podcast_id_col(row.get(0)?)
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?
|
||||
};
|
||||
let mut subscriptions = Vec::with_capacity(ids.len());
|
||||
for id in &ids {
|
||||
if let Some(record) = load_subscription_register(&conn, id)? {
|
||||
subscriptions.push(record);
|
||||
}
|
||||
}
|
||||
let shares = {
|
||||
let mut statement = conn.prepare(
|
||||
"SELECT podcast_id, shared, generation, hlc_ms, op_id FROM podcast_shares
|
||||
ORDER BY generation DESC, hlc_ms DESC, op_id DESC, podcast_id DESC
|
||||
LIMIT ?1",
|
||||
)?;
|
||||
statement
|
||||
.query_map([MAX_SNAPSHOT_SHARES as i64], |row| {
|
||||
let generation: i64 = row.get(2)?;
|
||||
Ok(ShareRecord {
|
||||
podcast_id: super::podcast_id_col(row.get(0)?)?,
|
||||
shared: row.get(1)?,
|
||||
generation: u64::try_from(generation).unwrap_or(0),
|
||||
hlc_ms: row.get(3)?,
|
||||
op_id: row.get(4)?,
|
||||
})
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?
|
||||
};
|
||||
let progress = {
|
||||
let mut statement = conn.prepare(&format!(
|
||||
"SELECT {PROGRESS_COLUMNS} FROM episode_progress p
|
||||
ORDER BY p.generation DESC, p.hlc_ms DESC, p.op_id DESC, p.episode_id DESC
|
||||
LIMIT ?1"
|
||||
))?;
|
||||
statement
|
||||
.query_map([MAX_SNAPSHOT_PROGRESS as i64], |row| {
|
||||
progress_from_row(row, 0)
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?
|
||||
};
|
||||
let sessions = {
|
||||
let mut statement = conn.prepare(&format!(
|
||||
"SELECT {SESSION_COLUMNS} FROM podcast_listen_sessions s
|
||||
ORDER BY s.ended_at_ms DESC, s.session_id DESC
|
||||
LIMIT ?1"
|
||||
))?;
|
||||
statement
|
||||
.query_map([MAX_SNAPSHOT_LISTEN_SESSIONS as i64], |row| {
|
||||
session_from_row(row, 0)
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?
|
||||
};
|
||||
drop(conn);
|
||||
snapshot.podcast_subscriptions = subscriptions;
|
||||
snapshot.podcast_shares = shares;
|
||||
snapshot.podcast_progress = progress;
|
||||
snapshot.podcast_listen_sessions = sessions;
|
||||
snapshot.bound_podcast_state_to(budget_bytes);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A snapshot holding only this store's podcast sections, bounded with
|
||||
/// the default budget (`MAX_PODCAST_SNAPSHOT_BYTES`).
|
||||
pub fn export_snapshot(&self) -> Result<SyncSnapshot> {
|
||||
let mut snapshot = SyncSnapshot::default();
|
||||
self.fill_snapshot(
|
||||
&mut snapshot,
|
||||
music_dht::podcast::MAX_PODCAST_SNAPSHOT_BYTES,
|
||||
)?;
|
||||
Ok(snapshot)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,451 @@
|
||||
//! SQLite reference store for podcasts (`podcasts.sqlite3`).
|
||||
//!
|
||||
//! [`PodcastStore`] persists the podcast contract of `music_dht::podcast`
|
||||
//! for SQLite clients: the local episode catalog refreshed from RSS, the
|
||||
//! synchronized personal registers (subscription, share, progress), the
|
||||
//! grow-only listening sessions and the local-only fetch state. It owns no
|
||||
//! networking, HTTP, scheduling or audio: clients fetch feeds, parse them
|
||||
//! with `music_dht::podcast::feed`, append sync operations to their own log
|
||||
//! and exchange snapshots. The normative rules live in the workspace
|
||||
//! `PODCASTS.md` (section "SQLite reference store" describes this schema).
|
||||
//!
|
||||
//! # Files and migrations
|
||||
//!
|
||||
//! The store lives in its own file, conventionally [`PODCAST_DB_FILE`] next
|
||||
//! to the library database; the caller passes the path. Connections use
|
||||
//! WAL, foreign keys and a busy timeout. `PRAGMA user_version` drives the
|
||||
//! migrations, each applied in its own `BEGIN IMMEDIATE` transaction that
|
||||
//! re-checks the version, so concurrent openers and interrupted upgrades are
|
||||
//! safe:
|
||||
//!
|
||||
//! | `user_version` | Schema |
|
||||
//! | --- | --- |
|
||||
//! | 0 | empty file |
|
||||
//! | 1 | prototype schema of the original podcast PR (`podcasts`, `podcast_episodes`) |
|
||||
//! | 2 | contract model; the prototype tables are renamed `legacy_*` and imported |
|
||||
//!
|
||||
//! A database written by a newer Furumi ([`PODCAST_SCHEMA_VERSION`] lower
|
||||
//! than its `user_version`) or a file that is not a podcast database is
|
||||
//! refused before anything is written ([`PodcastStoreError`]).
|
||||
//!
|
||||
//! # Personal state and sync
|
||||
//!
|
||||
//! Local actions ([`PodcastStore::subscribe`], [`PodcastStore::seek`], ...)
|
||||
//! draw operation stamps from a [`StampSource`], build the device-sync
|
||||
//! operation, apply it through the same code path as
|
||||
//! [`PodcastStore::apply_remote_op`] (the `SyncOpWire` materializers of
|
||||
//! `music_dht::device_sync` and the merge functions of `music_dht::podcast`)
|
||||
//! and return the [`PodcastLocalOp`]s. The caller appends them to its sync
|
||||
//! log **with exactly these stamps**; applying them again is a no-op.
|
||||
//! Applying operations and snapshot sections is idempotent and independent
|
||||
//! of delivery order.
|
||||
//!
|
||||
//! RSS refreshes ([`PodcastStore::apply_feed`]) only touch the catalog and
|
||||
//! feed aliases: they never write a register, progress or a session.
|
||||
//!
|
||||
//! ```
|
||||
//! use furumi_library::podcasts::{ApplyOutcome, OpStamp, PodcastStore, StampSource};
|
||||
//! use music_dht::podcast::{FeedUrl, PodcastMeta};
|
||||
//!
|
||||
//! /// Stand-in for the client's sync log, which reserves seq and hlc.
|
||||
//! struct Log {
|
||||
//! device: String,
|
||||
//! seq: i64,
|
||||
//! }
|
||||
//!
|
||||
//! impl StampSource for Log {
|
||||
//! fn device_id(&self) -> &str {
|
||||
//! &self.device
|
||||
//! }
|
||||
//! fn next_stamp(&mut self) -> anyhow::Result<OpStamp> {
|
||||
//! self.seq += 1;
|
||||
//! let op_id = format!("{}:{}", self.device, self.seq);
|
||||
//! Ok(OpStamp::new(op_id, self.device.clone(), 1_700_000_000_000 + self.seq))
|
||||
//! }
|
||||
//! }
|
||||
//!
|
||||
//! let store = PodcastStore::open_in_memory()?;
|
||||
//! let mut log = Log { device: "device-a".into(), seq: 0 };
|
||||
//! let meta = PodcastMeta {
|
||||
//! feed_url: FeedUrl::parse("https://example.test/feed.xml")?,
|
||||
//! title: "Show".into(),
|
||||
//! author: None,
|
||||
//! description: None,
|
||||
//! link: None,
|
||||
//! image_url: None,
|
||||
//! };
|
||||
//! let outcome = store.subscribe(&mut log, &meta)?;
|
||||
//! assert_eq!(outcome.ops.len(), 1);
|
||||
//! for op in outcome.ops {
|
||||
//! // Append to the sync log with this stamp; re-applying is a no-op.
|
||||
//! let op = op.into_wire(log.seq);
|
||||
//! assert_eq!(store.apply_remote_op(&op)?, ApplyOutcome::Unchanged);
|
||||
//! }
|
||||
//! # Ok::<(), anyhow::Error>(())
|
||||
//! ```
|
||||
//!
|
||||
//! # Thread safety
|
||||
//!
|
||||
//! All methods take `&self` and serialize on an internal mutex around one
|
||||
//! synchronous `rusqlite` connection, so an `Arc<PodcastStore>` can be shared
|
||||
//! between threads. Calls block on SQLite I/O: run them on a blocking pool
|
||||
//! (for example `tokio::task::spawn_blocking`), never on an async reactor
|
||||
//! thread. Several processes may open the same file; SQLite serializes the
|
||||
//! writers.
|
||||
|
||||
mod actions;
|
||||
mod apply;
|
||||
mod queries;
|
||||
mod refresh;
|
||||
mod schema;
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
|
||||
use std::fmt;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Mutex, MutexGuard, PoisonError};
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::{Context as _, Result};
|
||||
use music_dht::device_sync::{SyncOpPayload, SyncOpWire};
|
||||
use music_dht::podcast::{
|
||||
EpisodeId, FeedUrl, ListenSession, PodcastError, PodcastId, PodcastMeta, ProgressRecord,
|
||||
ProgressStatus,
|
||||
};
|
||||
use rusqlite::Connection;
|
||||
|
||||
pub use actions::{CheckpointOutcome, ProgressSession, ProgressWrite, SubscribeOutcome};
|
||||
pub use apply::{ApplyOutcome, SnapshotApplyStats};
|
||||
pub use queries::{
|
||||
EpisodeListItem, EpisodeQuery, EpisodeSort, LegacyQuarantineEntry, PlayedFilter,
|
||||
PodcastListFilter, PodcastListenEntry, PodcastSummary, SeasonFilter, SeasonSummary,
|
||||
};
|
||||
pub use refresh::{FeedRefresh, FetchState, RefreshOutcome};
|
||||
pub use schema::{LEGACY_PROTOTYPE_SCHEMA_VERSION, PODCAST_SCHEMA_VERSION};
|
||||
|
||||
/// Conventional file name of the podcast store, next to `library.db`.
|
||||
pub const PODCAST_DB_FILE: &str = "podcasts.sqlite3";
|
||||
|
||||
/// `op_id` prefix of registers imported from the prototype database.
|
||||
///
|
||||
/// Imported subscriptions use generation 0, `hlc_ms` = the prototype
|
||||
/// `added_at` and `op_id` = this prefix followed by the podcast id, so every
|
||||
/// write made through the contract orders after them.
|
||||
pub const LEGACY_IMPORT_OP_PREFIX: &str = "legacy-import:";
|
||||
|
||||
/// Typed errors of [`PodcastStore`]; other failures are SQLite or I/O errors.
|
||||
///
|
||||
/// They are returned inside `anyhow::Error`; use `downcast_ref` to match.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
#[non_exhaustive]
|
||||
pub enum PodcastStoreError {
|
||||
/// The database was written by a newer Furumi. It was not modified.
|
||||
NewerSchema {
|
||||
/// `user_version` found in the file.
|
||||
found: i64,
|
||||
/// Newest version this build understands.
|
||||
supported: i64,
|
||||
},
|
||||
/// The file is not a podcast database (or has an unexpected shape). It
|
||||
/// was not modified.
|
||||
UnrecognizedDatabase(String),
|
||||
/// The podcast is unknown to this store.
|
||||
UnknownPodcast(PodcastId),
|
||||
/// The action needs an active subscription.
|
||||
NotSubscribed(PodcastId),
|
||||
/// Input violates the podcast contract.
|
||||
Invalid(String),
|
||||
}
|
||||
|
||||
impl fmt::Display for PodcastStoreError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
Self::NewerSchema { found, supported } => write!(
|
||||
f,
|
||||
"podcast database schema v{found} is newer than this Furumi supports (v{supported}); \
|
||||
update Furumi, the database was left unchanged"
|
||||
),
|
||||
Self::UnrecognizedDatabase(reason) => {
|
||||
write!(f, "not a Furumi podcast database: {reason}")
|
||||
}
|
||||
Self::UnknownPodcast(id) => write!(f, "unknown podcast {id}"),
|
||||
Self::NotSubscribed(id) => write!(f, "not subscribed to podcast {id}"),
|
||||
Self::Invalid(reason) => write!(f, "invalid podcast data: {reason}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for PodcastStoreError {}
|
||||
|
||||
impl From<PodcastError> for PodcastStoreError {
|
||||
fn from(error: PodcastError) -> Self {
|
||||
Self::Invalid(error.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
/// Stamp of a device-sync operation: the fields the `SyncOpWire`
|
||||
/// materializers turn into register stamps.
|
||||
///
|
||||
/// `seq` is not part of it: the client's sync log assigns it when it appends
|
||||
/// the operation ([`PodcastLocalOp::into_wire`]).
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct OpStamp {
|
||||
/// Unique operation id, usually `<origin_device_id>:<seq>`.
|
||||
pub op_id: String,
|
||||
/// Origin device id (the progress register `device_id`).
|
||||
pub origin_device_id: String,
|
||||
/// Operation timestamp (register `hlc_ms`).
|
||||
pub hlc_ms: i64,
|
||||
}
|
||||
|
||||
impl OpStamp {
|
||||
/// Builds a stamp.
|
||||
pub fn new(op_id: impl Into<String>, origin_device_id: impl Into<String>, hlc_ms: i64) -> Self {
|
||||
Self {
|
||||
op_id: op_id.into(),
|
||||
origin_device_id: origin_device_id.into(),
|
||||
hlc_ms,
|
||||
}
|
||||
}
|
||||
|
||||
/// The stamp of an existing operation.
|
||||
pub fn of(op: &SyncOpWire) -> Self {
|
||||
Self::new(op.op_id.clone(), op.origin_device_id.clone(), op.hlc_ms)
|
||||
}
|
||||
|
||||
pub(crate) fn wire(&self, payload: SyncOpPayload) -> SyncOpWire {
|
||||
SyncOpWire {
|
||||
op_id: self.op_id.clone(),
|
||||
origin_device_id: self.origin_device_id.clone(),
|
||||
seq: 0,
|
||||
hlc_ms: self.hlc_ms,
|
||||
payload,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Source of operation stamps for local actions, usually the client's sync
|
||||
/// log (which reserves the next sequence number and timestamp).
|
||||
///
|
||||
/// A stamp is drawn only when an action actually emits an operation.
|
||||
pub trait StampSource {
|
||||
/// The local device id; every stamp must use it as origin.
|
||||
fn device_id(&self) -> &str;
|
||||
/// Reserves the stamp of the next local operation.
|
||||
fn next_stamp(&mut self) -> Result<OpStamp>;
|
||||
}
|
||||
|
||||
/// A local operation produced by a [`PodcastStore`] action.
|
||||
///
|
||||
/// The store has already applied it. Append it to the sync log with exactly
|
||||
/// this stamp: a different `op_id` or `hlc_ms` would make other devices
|
||||
/// materialize a different register than this one.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct PodcastLocalOp {
|
||||
/// Stamp drawn from the [`StampSource`].
|
||||
pub stamp: OpStamp,
|
||||
/// Operation payload (one of the `podcast_*` kinds).
|
||||
pub payload: SyncOpPayload,
|
||||
}
|
||||
|
||||
impl PodcastLocalOp {
|
||||
/// The operation with the sequence number assigned by the sync log.
|
||||
pub fn into_wire(self, seq: i64) -> SyncOpWire {
|
||||
let mut op = self.stamp.wire(self.payload);
|
||||
op.seq = seq;
|
||||
op
|
||||
}
|
||||
}
|
||||
|
||||
/// SQLite store of podcast catalog, personal state and fetch state.
|
||||
///
|
||||
/// See the [module documentation](self) for the schema, the sync model and
|
||||
/// thread safety.
|
||||
pub struct PodcastStore {
|
||||
conn: Mutex<Connection>,
|
||||
path: Option<PathBuf>,
|
||||
}
|
||||
|
||||
impl fmt::Debug for PodcastStore {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.debug_struct("PodcastStore")
|
||||
.field("path", &self.path)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
|
||||
impl PodcastStore {
|
||||
/// Opens (creating and migrating if needed) the podcast store at `path`.
|
||||
///
|
||||
/// A prototype database (`user_version` 1) is imported; see the module
|
||||
/// documentation. Fails without modifying the file when it was written
|
||||
/// by a newer Furumi or is not a podcast database.
|
||||
pub fn open(path: &Path) -> Result<Self> {
|
||||
if let Some(parent) = path.parent()
|
||||
&& !parent.as_os_str().is_empty()
|
||||
{
|
||||
std::fs::create_dir_all(parent)
|
||||
.with_context(|| format!("creating {}", parent.display()))?;
|
||||
}
|
||||
let conn = Connection::open(path)
|
||||
.with_context(|| format!("opening podcast database {}", path.display()))?;
|
||||
Self::init(conn, Some(path.to_path_buf()))
|
||||
}
|
||||
|
||||
/// Opens a private in-memory store (tests and previews).
|
||||
pub fn open_in_memory() -> Result<Self> {
|
||||
Self::init(Connection::open_in_memory()?, None)
|
||||
}
|
||||
|
||||
fn init(mut conn: Connection, path: Option<PathBuf>) -> Result<Self> {
|
||||
conn.busy_timeout(Duration::from_secs(5))?;
|
||||
// Refuse newer or foreign files before anything (WAL included) is written.
|
||||
schema::check_openable(&conn)?;
|
||||
if path.is_some() {
|
||||
let _: String = conn.query_row("PRAGMA journal_mode=WAL", [], |row| row.get(0))?;
|
||||
}
|
||||
conn.pragma_update(None, "foreign_keys", "ON")?;
|
||||
register_norm_function(&conn)?;
|
||||
schema::migrate(&mut conn)?;
|
||||
Ok(Self {
|
||||
conn: Mutex::new(conn),
|
||||
path,
|
||||
})
|
||||
}
|
||||
|
||||
/// Path of the database file (`None` for in-memory stores).
|
||||
pub fn path(&self) -> Option<&Path> {
|
||||
self.path.as_deref()
|
||||
}
|
||||
|
||||
/// Current `PRAGMA user_version` (always [`PODCAST_SCHEMA_VERSION`]
|
||||
/// after a successful open).
|
||||
pub fn schema_version(&self) -> Result<i64> {
|
||||
Ok(self
|
||||
.lock()
|
||||
.query_row("PRAGMA user_version", [], |row| row.get(0))?)
|
||||
}
|
||||
|
||||
pub(crate) fn lock(&self) -> MutexGuard<'_, Connection> {
|
||||
self.conn.lock().unwrap_or_else(PoisonError::into_inner)
|
||||
}
|
||||
}
|
||||
|
||||
fn register_norm_function(conn: &Connection) -> Result<()> {
|
||||
use rusqlite::functions::FunctionFlags;
|
||||
conn.create_scalar_function(
|
||||
"norm",
|
||||
1,
|
||||
FunctionFlags::SQLITE_UTF8 | FunctionFlags::SQLITE_DETERMINISTIC,
|
||||
|ctx| {
|
||||
let value: Option<String> = ctx.get(0)?;
|
||||
Ok(music_dht::normalize_name(
|
||||
value.as_deref().unwrap_or_default(),
|
||||
))
|
||||
},
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) fn now_ms() -> i64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|duration| i64::try_from(duration.as_millis()).unwrap_or(i64::MAX))
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
/// Converts a contract generation to the SQLite column value.
|
||||
///
|
||||
/// Generations above `i64::MAX` cannot be stored; records carrying one are
|
||||
/// rejected (no conforming client produces them).
|
||||
pub(crate) fn generation_to_sql(generation: u64) -> Result<i64, PodcastStoreError> {
|
||||
i64::try_from(generation)
|
||||
.map_err(|_| PodcastStoreError::Invalid("generation exceeds i64::MAX".to_string()))
|
||||
}
|
||||
|
||||
pub(crate) fn status_to_sql(status: ProgressStatus) -> &'static str {
|
||||
match status {
|
||||
ProgressStatus::NotStarted => "not_started",
|
||||
ProgressStatus::InProgress => "in_progress",
|
||||
ProgressStatus::Completed => "completed",
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn status_from_sql(value: &str) -> ProgressStatus {
|
||||
match value {
|
||||
"in_progress" => ProgressStatus::InProgress,
|
||||
"completed" => ProgressStatus::Completed,
|
||||
_ => ProgressStatus::NotStarted,
|
||||
}
|
||||
}
|
||||
|
||||
/// Parses a stored id; stored ids are validated on write.
|
||||
pub(crate) fn podcast_id_col(value: String) -> rusqlite::Result<PodcastId> {
|
||||
PodcastId::try_from(value).map_err(|error| conversion_error(error.to_string()))
|
||||
}
|
||||
|
||||
pub(crate) fn episode_id_col(value: String) -> rusqlite::Result<EpisodeId> {
|
||||
EpisodeId::try_from(value).map_err(|error| conversion_error(error.to_string()))
|
||||
}
|
||||
|
||||
pub(crate) fn feed_url_col(value: String) -> rusqlite::Result<FeedUrl> {
|
||||
FeedUrl::try_from(value).map_err(|error| conversion_error(error.to_string()))
|
||||
}
|
||||
|
||||
pub(crate) fn podcast_meta_col(value: &str) -> rusqlite::Result<PodcastMeta> {
|
||||
serde_json::from_str(value).map_err(|error| conversion_error(error.to_string()))
|
||||
}
|
||||
|
||||
pub(crate) fn conversion_error(message: String) -> rusqlite::Error {
|
||||
rusqlite::Error::FromSqlConversionFailure(
|
||||
0,
|
||||
rusqlite::types::Type::Text,
|
||||
Box::new(PodcastStoreError::Invalid(message)),
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn progress_from_row(
|
||||
row: &rusqlite::Row<'_>,
|
||||
at: usize,
|
||||
) -> rusqlite::Result<ProgressRecord> {
|
||||
let generation: i64 = row.get(at + 4)?;
|
||||
Ok(ProgressRecord {
|
||||
episode_id: episode_id_col(row.get(at)?)?,
|
||||
podcast_id: podcast_id_col(row.get(at + 1)?)?,
|
||||
position_ms: row.get(at + 2)?,
|
||||
status: status_from_sql(&row.get::<_, String>(at + 3)?),
|
||||
generation: u64::try_from(generation).unwrap_or(0),
|
||||
session_id: row.get(at + 5)?,
|
||||
hlc_ms: row.get(at + 6)?,
|
||||
device_id: row.get(at + 7)?,
|
||||
op_id: row.get(at + 8)?,
|
||||
})
|
||||
}
|
||||
|
||||
/// Column list matching [`progress_from_row`], for table alias `p`.
|
||||
pub(crate) const PROGRESS_COLUMNS: &str = "p.episode_id, p.podcast_id, p.position_ms, p.status, \
|
||||
p.generation, p.session_id, p.hlc_ms, p.device_id, p.op_id";
|
||||
|
||||
pub(crate) fn session_from_row(
|
||||
row: &rusqlite::Row<'_>,
|
||||
at: usize,
|
||||
) -> rusqlite::Result<ListenSession> {
|
||||
Ok(ListenSession {
|
||||
session_id: row.get(at)?,
|
||||
episode_id: episode_id_col(row.get(at + 1)?)?,
|
||||
podcast_id: podcast_id_col(row.get(at + 2)?)?,
|
||||
device_id: row.get(at + 3)?,
|
||||
started_at_ms: row.get(at + 4)?,
|
||||
ended_at_ms: row.get(at + 5)?,
|
||||
start_position_ms: row.get(at + 6)?,
|
||||
end_position_ms: row.get(at + 7)?,
|
||||
listened_ms: row.get(at + 8)?,
|
||||
completed: row.get(at + 9)?,
|
||||
})
|
||||
}
|
||||
|
||||
/// Column list matching [`session_from_row`], for table alias `s`.
|
||||
pub(crate) const SESSION_COLUMNS: &str = "s.session_id, s.episode_id, s.podcast_id, s.device_id, \
|
||||
s.started_at_ms, s.ended_at_ms, s.start_position_ms, s.end_position_ms, s.listened_ms, \
|
||||
s.completed";
|
||||
@@ -0,0 +1,605 @@
|
||||
//! Read-only queries for clients' podcast UI.
|
||||
|
||||
use anyhow::Result;
|
||||
use music_dht::podcast::{
|
||||
EpisodeId, EpisodeMeta, EpisodeType, FeedUrl, ListenSession, PodcastEpisodeRef, PodcastId,
|
||||
PodcastMeta, ProgressRecord, ProgressStatus, ShareRecord, SubscriptionRecord, effective_share,
|
||||
};
|
||||
use rusqlite::types::Value;
|
||||
use rusqlite::{Connection, OptionalExtension as _, params, params_from_iter};
|
||||
|
||||
use super::apply;
|
||||
use super::{
|
||||
PROGRESS_COLUMNS, PodcastStore, SESSION_COLUMNS, episode_id_col, feed_url_col, podcast_id_col,
|
||||
progress_from_row, session_from_row,
|
||||
};
|
||||
use crate::TrackItem;
|
||||
|
||||
/// Which podcasts [`PodcastStore::list_podcasts`] returns.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub enum PodcastListFilter {
|
||||
/// Podcasts with an active subscription.
|
||||
#[default]
|
||||
Subscribed,
|
||||
/// Every podcast in the catalog, including unsubscribed ones and
|
||||
/// podcasts only known from synced progress or queue items.
|
||||
All,
|
||||
}
|
||||
|
||||
/// A podcast with its personal state and episode counters.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct PodcastSummary {
|
||||
/// Podcast id.
|
||||
pub podcast_id: PodcastId,
|
||||
/// Local channel metadata; `feed_url` is the URL this device fetches.
|
||||
pub meta: PodcastMeta,
|
||||
/// Whether the subscription register says subscribed.
|
||||
pub subscribed: bool,
|
||||
/// Whether the podcast must currently be published (`effective_share`).
|
||||
pub shared: bool,
|
||||
/// Episodes in the catalog.
|
||||
pub episode_count: u64,
|
||||
/// Episodes not completed (no progress, not started or in progress).
|
||||
pub unplayed_count: u64,
|
||||
/// Episodes in progress.
|
||||
pub in_progress_count: u64,
|
||||
/// Newest publication time.
|
||||
pub latest_published_at_ms: Option<i64>,
|
||||
/// Numbered seasons (episodes without a season are not counted).
|
||||
pub season_count: u64,
|
||||
}
|
||||
|
||||
/// One season group of a podcast. `season == None` groups the episodes
|
||||
/// without `itunes:season`.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct SeasonSummary {
|
||||
/// Season number; `None` for episodes without a season.
|
||||
pub season: Option<u32>,
|
||||
/// Episodes in the group.
|
||||
pub episode_count: u64,
|
||||
/// Episodes not completed.
|
||||
pub unplayed_count: u64,
|
||||
/// Oldest publication time.
|
||||
pub first_published_at_ms: Option<i64>,
|
||||
/// Newest publication time.
|
||||
pub last_published_at_ms: Option<i64>,
|
||||
}
|
||||
|
||||
/// Season filter of an [`EpisodeQuery`].
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub enum SeasonFilter {
|
||||
/// All episodes.
|
||||
#[default]
|
||||
Any,
|
||||
/// Episodes of one season.
|
||||
Season(u32),
|
||||
/// Episodes without a season.
|
||||
NoSeason,
|
||||
}
|
||||
|
||||
/// Listening-state filter of an [`EpisodeQuery`].
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub enum PlayedFilter {
|
||||
/// All episodes.
|
||||
#[default]
|
||||
Any,
|
||||
/// Not completed (not started or in progress).
|
||||
Unplayed,
|
||||
/// Never started or marked unplayed.
|
||||
NotStarted,
|
||||
/// In progress.
|
||||
InProgress,
|
||||
/// Completed.
|
||||
Played,
|
||||
}
|
||||
|
||||
/// Episode ordering of an [`EpisodeQuery`].
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub enum EpisodeSort {
|
||||
/// Newest publication first; undated episodes last.
|
||||
#[default]
|
||||
NewestFirst,
|
||||
/// Oldest publication first; undated episodes last.
|
||||
OldestFirst,
|
||||
/// By season, then episode number, ascending; unnumbered last.
|
||||
EpisodeNumber,
|
||||
/// By season, then episode number, descending; unnumbered last.
|
||||
EpisodeNumberDesc,
|
||||
}
|
||||
|
||||
/// Parameters of [`PodcastStore::episodes`].
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct EpisodeQuery {
|
||||
/// Podcast.
|
||||
pub podcast_id: PodcastId,
|
||||
/// Season filter.
|
||||
pub season: SeasonFilter,
|
||||
/// Listening-state filter.
|
||||
pub played: PlayedFilter,
|
||||
/// Text searched in titles and descriptions (Unicode-normalized,
|
||||
/// case-insensitive substring match). Blank means no filter.
|
||||
pub search: Option<String>,
|
||||
/// Ordering.
|
||||
pub sort: EpisodeSort,
|
||||
/// Maximum rows (`None`: all).
|
||||
pub limit: Option<usize>,
|
||||
/// Rows to skip.
|
||||
pub offset: usize,
|
||||
}
|
||||
|
||||
impl EpisodeQuery {
|
||||
/// All episodes of a podcast, newest first.
|
||||
pub fn new(podcast_id: PodcastId) -> Self {
|
||||
Self {
|
||||
podcast_id,
|
||||
season: SeasonFilter::Any,
|
||||
played: PlayedFilter::Any,
|
||||
search: None,
|
||||
sort: EpisodeSort::NewestFirst,
|
||||
limit: None,
|
||||
offset: 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// An episode with its personal state.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct EpisodeListItem {
|
||||
/// Episode metadata (feed or synced reference).
|
||||
pub episode: EpisodeMeta,
|
||||
/// Progress register, if any.
|
||||
pub progress: Option<ProgressRecord>,
|
||||
/// Distinct completed listening sessions (`completed_listen_count`).
|
||||
pub completed_count: u64,
|
||||
/// Whether the latest refresh of the feed listed the episode.
|
||||
pub in_feed: bool,
|
||||
}
|
||||
|
||||
impl EpisodeListItem {
|
||||
/// Listening status (`NotStarted` without a register).
|
||||
pub fn status(&self) -> ProgressStatus {
|
||||
self.progress
|
||||
.as_ref()
|
||||
.map_or(ProgressStatus::NotStarted, |progress| progress.status)
|
||||
}
|
||||
|
||||
/// Position ordinary Play resumes from.
|
||||
pub fn resume_position_ms(&self) -> i64 {
|
||||
self.progress
|
||||
.as_ref()
|
||||
.map_or(0, ProgressRecord::resume_position_ms)
|
||||
}
|
||||
}
|
||||
|
||||
/// A listening session in history, with display titles when known.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct PodcastListenEntry {
|
||||
/// The session.
|
||||
pub session: ListenSession,
|
||||
/// Episode title from the catalog.
|
||||
pub episode_title: Option<String>,
|
||||
/// Podcast title from the catalog.
|
||||
pub podcast_title: Option<String>,
|
||||
}
|
||||
|
||||
/// A prototype row that could not be imported (kept in its `legacy_*` table).
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct LegacyQuarantineEntry {
|
||||
/// `legacy_podcasts` or `legacy_podcast_episodes`.
|
||||
pub legacy_table: String,
|
||||
/// Row id in that table.
|
||||
pub legacy_id: i64,
|
||||
/// Short reason.
|
||||
pub reason: String,
|
||||
/// Details (the offending value or error).
|
||||
pub detail: Option<String>,
|
||||
}
|
||||
|
||||
const EPISODE_COLUMNS: &str = "e.episode_id, e.podcast_id, e.guid, e.title, e.description, \
|
||||
e.season, e.episode_number, e.episode_type, e.published_at_ms, e.duration_ms, \
|
||||
e.audio_url, e.mime, e.byte_len, e.image_url, e.link";
|
||||
const EPISODE_COLUMN_COUNT: usize = 15;
|
||||
|
||||
fn episode_from_row(row: &rusqlite::Row<'_>) -> rusqlite::Result<EpisodeMeta> {
|
||||
Ok(EpisodeMeta {
|
||||
episode_id: episode_id_col(row.get(0)?)?,
|
||||
podcast_id: podcast_id_col(row.get(1)?)?,
|
||||
guid: row.get(2)?,
|
||||
title: row.get(3)?,
|
||||
description: row.get(4)?,
|
||||
season: row.get(5)?,
|
||||
episode_number: row.get(6)?,
|
||||
episode_type: EpisodeType::parse(&row.get::<_, String>(7)?),
|
||||
published_at_ms: row.get(8)?,
|
||||
duration_ms: row.get(9)?,
|
||||
audio_url: row.get(10)?,
|
||||
mime: row.get(11)?,
|
||||
byte_len: row
|
||||
.get::<_, Option<i64>>(12)?
|
||||
.and_then(|len| u64::try_from(len).ok()),
|
||||
image_url: row.get(13)?,
|
||||
link: row.get(14)?,
|
||||
})
|
||||
}
|
||||
|
||||
fn optional_progress(
|
||||
row: &rusqlite::Row<'_>,
|
||||
at: usize,
|
||||
) -> rusqlite::Result<Option<ProgressRecord>> {
|
||||
if row.get::<_, Option<String>>(at)?.is_none() {
|
||||
return Ok(None);
|
||||
}
|
||||
progress_from_row(row, at).map(Some)
|
||||
}
|
||||
|
||||
pub(crate) fn episode_meta_in(
|
||||
conn: &Connection,
|
||||
episode_id: &EpisodeId,
|
||||
) -> Result<Option<EpisodeMeta>> {
|
||||
Ok(conn
|
||||
.query_row(
|
||||
&format!("SELECT {EPISODE_COLUMNS} FROM episodes e WHERE e.episode_id = ?1"),
|
||||
[episode_id.as_str()],
|
||||
episode_from_row,
|
||||
)
|
||||
.optional()?)
|
||||
}
|
||||
|
||||
pub(crate) fn podcast_meta_in(
|
||||
conn: &Connection,
|
||||
podcast_id: &PodcastId,
|
||||
) -> Result<Option<PodcastMeta>> {
|
||||
Ok(conn
|
||||
.query_row(
|
||||
"SELECT feed_url, title, author, description, link, image_url
|
||||
FROM podcasts WHERE podcast_id = ?1",
|
||||
[podcast_id.as_str()],
|
||||
|row| {
|
||||
Ok(PodcastMeta {
|
||||
feed_url: feed_url_col(row.get(0)?)?,
|
||||
title: row.get(1)?,
|
||||
author: row.get(2)?,
|
||||
description: row.get(3)?,
|
||||
link: row.get(4)?,
|
||||
image_url: row.get(5)?,
|
||||
})
|
||||
},
|
||||
)
|
||||
.optional()?)
|
||||
}
|
||||
|
||||
/// Queue reference of a catalog episode, with `progress_generation` set to
|
||||
/// the generation of its progress register (handoff hint).
|
||||
pub(crate) fn episode_ref_in(
|
||||
conn: &Connection,
|
||||
episode_id: &EpisodeId,
|
||||
) -> Result<Option<PodcastEpisodeRef>> {
|
||||
let Some(episode) = episode_meta_in(conn, episode_id)? else {
|
||||
return Ok(None);
|
||||
};
|
||||
let Some(podcast) = podcast_meta_in(conn, &episode.podcast_id)? else {
|
||||
return Ok(None);
|
||||
};
|
||||
let mut reference = episode.to_ref(&podcast);
|
||||
reference.progress_generation =
|
||||
apply::load_progress(conn, episode_id)?.map(|progress| progress.generation);
|
||||
Ok(Some(reference))
|
||||
}
|
||||
|
||||
const NOT_COMPLETED: &str = "IFNULL(g.status, 'not_started') <> 'completed'";
|
||||
|
||||
impl PodcastStore {
|
||||
/// Podcasts with subscription state and counters, sorted by title.
|
||||
pub fn list_podcasts(&self, filter: PodcastListFilter) -> Result<Vec<PodcastSummary>> {
|
||||
let conn = self.lock();
|
||||
let join = match filter {
|
||||
PodcastListFilter::Subscribed => {
|
||||
"JOIN podcast_subscriptions r ON r.podcast_id = p.podcast_id AND r.subscribed = 1"
|
||||
}
|
||||
PodcastListFilter::All => "",
|
||||
};
|
||||
let ids: Vec<PodcastId> = {
|
||||
let mut statement = conn.prepare(&format!(
|
||||
"SELECT p.podcast_id FROM podcasts p {join}
|
||||
ORDER BY p.title COLLATE NOCASE, p.podcast_id"
|
||||
))?;
|
||||
statement
|
||||
.query_map([], |row| podcast_id_col(row.get(0)?))?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?
|
||||
};
|
||||
let mut out = Vec::with_capacity(ids.len());
|
||||
for id in ids {
|
||||
if let Some(summary) = summary_in(&conn, &id)? {
|
||||
out.push(summary);
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// One podcast with subscription state and counters.
|
||||
pub fn podcast(&self, podcast_id: &PodcastId) -> Result<Option<PodcastSummary>> {
|
||||
summary_in(&self.lock(), podcast_id)
|
||||
}
|
||||
|
||||
/// The subscription register (with the stored alias set), if any.
|
||||
pub fn subscription(&self, podcast_id: &PodcastId) -> Result<Option<SubscriptionRecord>> {
|
||||
apply::load_subscription_register(&self.lock(), podcast_id)
|
||||
}
|
||||
|
||||
/// The share register, if any.
|
||||
pub fn share(&self, podcast_id: &PodcastId) -> Result<Option<ShareRecord>> {
|
||||
apply::load_share(&self.lock(), podcast_id)
|
||||
}
|
||||
|
||||
/// The bounded, synced feed alias set of a podcast.
|
||||
pub fn feed_aliases(&self, podcast_id: &PodcastId) -> Result<Vec<FeedUrl>> {
|
||||
apply::load_aliases(&self.lock(), podcast_id)
|
||||
}
|
||||
|
||||
/// Season groups of a podcast: numbered seasons ascending, then the
|
||||
/// group of episodes without a season.
|
||||
pub fn seasons(&self, podcast_id: &PodcastId) -> Result<Vec<SeasonSummary>> {
|
||||
let conn = self.lock();
|
||||
let mut statement = conn.prepare(&format!(
|
||||
"SELECT s.season, s.episode_count, s.first_published_at_ms, s.last_published_at_ms,
|
||||
(SELECT COUNT(*) FROM episodes e
|
||||
LEFT JOIN episode_progress g ON g.episode_id = e.episode_id
|
||||
WHERE e.podcast_id = s.podcast_id AND e.season IS s.season
|
||||
AND {NOT_COMPLETED})
|
||||
FROM podcast_seasons s WHERE s.podcast_id = ?1
|
||||
ORDER BY s.season IS NULL, s.season"
|
||||
))?;
|
||||
let seasons = statement
|
||||
.query_map([podcast_id.as_str()], |row| {
|
||||
Ok(SeasonSummary {
|
||||
season: row.get(0)?,
|
||||
episode_count: count(row.get(1)?),
|
||||
first_published_at_ms: row.get(2)?,
|
||||
last_published_at_ms: row.get(3)?,
|
||||
unplayed_count: count(row.get(4)?),
|
||||
})
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
Ok(seasons)
|
||||
}
|
||||
|
||||
/// Episodes of a podcast or season with their progress.
|
||||
pub fn episodes(&self, query: &EpisodeQuery) -> Result<Vec<EpisodeListItem>> {
|
||||
let mut conditions = vec!["e.podcast_id = ?".to_string()];
|
||||
let mut values = vec![Value::Text(query.podcast_id.as_str().to_string())];
|
||||
match query.season {
|
||||
SeasonFilter::Any => {}
|
||||
SeasonFilter::Season(season) => {
|
||||
conditions.push("e.season = ?".to_string());
|
||||
values.push(Value::Integer(i64::from(season)));
|
||||
}
|
||||
SeasonFilter::NoSeason => conditions.push("e.season IS NULL".to_string()),
|
||||
}
|
||||
match query.played {
|
||||
PlayedFilter::Any => {}
|
||||
PlayedFilter::Unplayed => conditions.push(NOT_COMPLETED.to_string()),
|
||||
PlayedFilter::NotStarted => {
|
||||
conditions.push("IFNULL(g.status, 'not_started') = 'not_started'".to_string());
|
||||
}
|
||||
PlayedFilter::InProgress => conditions.push("g.status = 'in_progress'".to_string()),
|
||||
PlayedFilter::Played => conditions.push("g.status = 'completed'".to_string()),
|
||||
}
|
||||
if let Some(search) = query.search.as_deref() {
|
||||
let needle = music_dht::normalize_name(search);
|
||||
if !needle.is_empty() {
|
||||
conditions.push(
|
||||
"(instr(norm(e.title), ?) > 0 OR instr(norm(e.description), ?) > 0)"
|
||||
.to_string(),
|
||||
);
|
||||
values.push(Value::Text(needle.clone()));
|
||||
values.push(Value::Text(needle));
|
||||
}
|
||||
}
|
||||
let order = match query.sort {
|
||||
EpisodeSort::NewestFirst => {
|
||||
"e.published_at_ms IS NULL, e.published_at_ms DESC, e.episode_id"
|
||||
}
|
||||
EpisodeSort::OldestFirst => {
|
||||
"e.published_at_ms IS NULL, e.published_at_ms, e.episode_id"
|
||||
}
|
||||
EpisodeSort::EpisodeNumber => {
|
||||
"e.season IS NULL, e.season, e.episode_number IS NULL, e.episode_number, \
|
||||
e.published_at_ms IS NULL, e.published_at_ms, e.episode_id"
|
||||
}
|
||||
EpisodeSort::EpisodeNumberDesc => {
|
||||
"e.season IS NULL, e.season DESC, e.episode_number IS NULL, \
|
||||
e.episode_number DESC, e.published_at_ms IS NULL, e.published_at_ms DESC, \
|
||||
e.episode_id"
|
||||
}
|
||||
};
|
||||
values.push(Value::Integer(
|
||||
query
|
||||
.limit
|
||||
.map_or(-1, |limit| i64::try_from(limit).unwrap_or(i64::MAX)),
|
||||
));
|
||||
values.push(Value::Integer(
|
||||
i64::try_from(query.offset).unwrap_or(i64::MAX),
|
||||
));
|
||||
let progress_columns = PROGRESS_COLUMNS.replace("p.", "g.");
|
||||
let sql = format!(
|
||||
"SELECT {EPISODE_COLUMNS}, e.in_feed, {progress_columns},
|
||||
(SELECT COUNT(*) FROM podcast_listen_sessions s
|
||||
WHERE s.episode_id = e.episode_id AND s.completed = 1)
|
||||
FROM episodes e
|
||||
LEFT JOIN episode_progress g ON g.episode_id = e.episode_id
|
||||
WHERE {}
|
||||
ORDER BY {order}
|
||||
LIMIT ? OFFSET ?",
|
||||
conditions.join(" AND ")
|
||||
);
|
||||
let conn = self.lock();
|
||||
let mut statement = conn.prepare(&sql)?;
|
||||
let items = statement
|
||||
.query_map(params_from_iter(values), list_item_from_row)?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
Ok(items)
|
||||
}
|
||||
|
||||
/// One episode with its progress.
|
||||
pub fn episode(&self, episode_id: &EpisodeId) -> Result<Option<EpisodeListItem>> {
|
||||
let progress_columns = PROGRESS_COLUMNS.replace("p.", "g.");
|
||||
Ok(self
|
||||
.lock()
|
||||
.query_row(
|
||||
&format!(
|
||||
"SELECT {EPISODE_COLUMNS}, e.in_feed, {progress_columns},
|
||||
(SELECT COUNT(*) FROM podcast_listen_sessions s
|
||||
WHERE s.episode_id = e.episode_id AND s.completed = 1)
|
||||
FROM episodes e
|
||||
LEFT JOIN episode_progress g ON g.episode_id = e.episode_id
|
||||
WHERE e.episode_id = ?1"
|
||||
),
|
||||
[episode_id.as_str()],
|
||||
list_item_from_row,
|
||||
)
|
||||
.optional()?)
|
||||
}
|
||||
|
||||
/// Progress register of an episode (also for episodes the catalog does
|
||||
/// not know yet).
|
||||
pub fn progress(&self, episode_id: &EpisodeId) -> Result<Option<ProgressRecord>> {
|
||||
apply::load_progress(&self.lock(), episode_id)
|
||||
}
|
||||
|
||||
/// Position ordinary Play resumes from (`ProgressRecord::resume_position_ms`, 0 without progress).
|
||||
pub fn resume_position_ms(&self, episode_id: &EpisodeId) -> Result<i64> {
|
||||
Ok(self
|
||||
.progress(episode_id)?
|
||||
.map_or(0, |progress| progress.resume_position_ms()))
|
||||
}
|
||||
|
||||
/// A queue item for an episode (see [`TrackItem::podcast_episode`]).
|
||||
pub fn episode_track(&self, episode_id: &EpisodeId) -> Result<Option<TrackItem>> {
|
||||
Ok(self
|
||||
.episode_ref(episode_id)?
|
||||
.map(TrackItem::podcast_episode))
|
||||
}
|
||||
|
||||
/// Listening history (sessions that `should_record`), newest first.
|
||||
pub fn listen_history(
|
||||
&self,
|
||||
podcast_id: Option<&PodcastId>,
|
||||
limit: usize,
|
||||
) -> Result<Vec<PodcastListenEntry>> {
|
||||
let conn = self.lock();
|
||||
let mut statement = conn.prepare(&format!(
|
||||
"SELECT {SESSION_COLUMNS}, e.title, p.title
|
||||
FROM podcast_listen_sessions s
|
||||
LEFT JOIN episodes e ON e.episode_id = s.episode_id
|
||||
LEFT JOIN podcasts p ON p.podcast_id = s.podcast_id
|
||||
WHERE (s.completed = 1 OR s.listened_ms >= ?1)
|
||||
AND (?2 IS NULL OR s.podcast_id = ?2)
|
||||
ORDER BY s.ended_at_ms DESC, s.session_id DESC
|
||||
LIMIT ?3"
|
||||
))?;
|
||||
let entries = statement
|
||||
.query_map(
|
||||
params![
|
||||
music_dht::podcast::MIN_RECORDED_EPISODE_LISTEN_MS,
|
||||
podcast_id.map(PodcastId::as_str),
|
||||
i64::try_from(limit).unwrap_or(i64::MAX)
|
||||
],
|
||||
|row| {
|
||||
Ok(PodcastListenEntry {
|
||||
session: session_from_row(row, 0)?,
|
||||
episode_title: row.get(10)?,
|
||||
podcast_title: row.get(11)?,
|
||||
})
|
||||
},
|
||||
)?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
Ok(entries)
|
||||
}
|
||||
|
||||
/// Number of distinct completed sessions of an episode.
|
||||
pub fn completed_listen_count(&self, episode_id: &EpisodeId) -> Result<u64> {
|
||||
let total: i64 = self.lock().query_row(
|
||||
"SELECT COUNT(*) FROM podcast_listen_sessions WHERE episode_id = ?1 AND completed = 1",
|
||||
[episode_id.as_str()],
|
||||
|row| row.get(0),
|
||||
)?;
|
||||
Ok(count(total))
|
||||
}
|
||||
|
||||
/// Prototype rows that could not be imported.
|
||||
pub fn legacy_quarantine(&self) -> Result<Vec<LegacyQuarantineEntry>> {
|
||||
let conn = self.lock();
|
||||
let mut statement = conn.prepare(
|
||||
"SELECT legacy_table, legacy_id, reason, detail FROM legacy_quarantine
|
||||
ORDER BY legacy_table, legacy_id",
|
||||
)?;
|
||||
let entries = statement
|
||||
.query_map([], |row| {
|
||||
Ok(LegacyQuarantineEntry {
|
||||
legacy_table: row.get(0)?,
|
||||
legacy_id: row.get(1)?,
|
||||
reason: row.get(2)?,
|
||||
detail: row.get(3)?,
|
||||
})
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
Ok(entries)
|
||||
}
|
||||
}
|
||||
|
||||
fn count(value: i64) -> u64 {
|
||||
u64::try_from(value).unwrap_or(0)
|
||||
}
|
||||
|
||||
fn list_item_from_row(row: &rusqlite::Row<'_>) -> rusqlite::Result<EpisodeListItem> {
|
||||
Ok(EpisodeListItem {
|
||||
episode: episode_from_row(row)?,
|
||||
in_feed: row.get(EPISODE_COLUMN_COUNT)?,
|
||||
progress: optional_progress(row, EPISODE_COLUMN_COUNT + 1)?,
|
||||
completed_count: count(row.get(EPISODE_COLUMN_COUNT + 10)?),
|
||||
})
|
||||
}
|
||||
|
||||
fn summary_in(conn: &Connection, podcast_id: &PodcastId) -> Result<Option<PodcastSummary>> {
|
||||
let Some(meta) = podcast_meta_in(conn, podcast_id)? else {
|
||||
return Ok(None);
|
||||
};
|
||||
let (episode_count, unplayed_count, in_progress_count, latest, season_count) = conn.query_row(
|
||||
&format!(
|
||||
"SELECT COUNT(*),
|
||||
COUNT(*) FILTER (WHERE {NOT_COMPLETED}),
|
||||
COUNT(*) FILTER (WHERE g.status = 'in_progress'),
|
||||
MAX(e.published_at_ms),
|
||||
(SELECT COUNT(*) FROM podcast_seasons s
|
||||
WHERE s.podcast_id = ?1 AND s.season IS NOT NULL)
|
||||
FROM episodes e
|
||||
LEFT JOIN episode_progress g ON g.episode_id = e.episode_id
|
||||
WHERE e.podcast_id = ?1"
|
||||
),
|
||||
[podcast_id.as_str()],
|
||||
|row| {
|
||||
Ok((
|
||||
row.get::<_, i64>(0)?,
|
||||
row.get::<_, i64>(1)?,
|
||||
row.get::<_, i64>(2)?,
|
||||
row.get::<_, Option<i64>>(3)?,
|
||||
row.get::<_, i64>(4)?,
|
||||
))
|
||||
},
|
||||
)?;
|
||||
let subscription = apply::load_subscription_register(conn, podcast_id)?;
|
||||
let share = apply::load_share(conn, podcast_id)?;
|
||||
Ok(Some(PodcastSummary {
|
||||
podcast_id: podcast_id.clone(),
|
||||
meta,
|
||||
subscribed: subscription
|
||||
.as_ref()
|
||||
.is_some_and(|record| record.subscribed),
|
||||
shared: effective_share(subscription.as_ref(), share.as_ref()),
|
||||
episode_count: count(episode_count),
|
||||
unplayed_count: count(unplayed_count),
|
||||
in_progress_count: count(in_progress_count),
|
||||
latest_published_at_ms: latest,
|
||||
season_count: count(season_count),
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,618 @@
|
||||
//! Feed refresh ingestion and the local-only fetch state.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
|
||||
use anyhow::Result;
|
||||
use music_dht::device_sync::SyncOpPayload;
|
||||
use music_dht::podcast::feed::ParsedFeed;
|
||||
use music_dht::podcast::{
|
||||
EpisodeId, EpisodeIdentityHints, EpisodeMeta, FeedUrl, KnownEpisode, PodcastId,
|
||||
merge_feed_aliases, resolve_episode_ids,
|
||||
};
|
||||
use rusqlite::{Connection, OptionalExtension as _, TransactionBehavior, params};
|
||||
|
||||
use super::apply::{self, EpisodeSource};
|
||||
use super::{PodcastLocalOp, PodcastStore, PodcastStoreError, StampSource, now_ms};
|
||||
|
||||
/// A fetched and parsed feed document for [`PodcastStore::apply_feed`].
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct FeedRefresh<'a> {
|
||||
/// Podcast the feed belongs to.
|
||||
pub podcast_id: &'a PodcastId,
|
||||
/// The parsed document (`music_dht::podcast::feed::parse_feed_with_base`
|
||||
/// with `fetched_from` as base).
|
||||
pub feed: &'a ParsedFeed,
|
||||
/// URL the document was finally served from, after redirects.
|
||||
pub fetched_from: &'a FeedUrl,
|
||||
/// Whether every redirect leading to `fetched_from` was permanent
|
||||
/// (301/308). Temporary redirects do not move the feed.
|
||||
pub moved_permanently: bool,
|
||||
}
|
||||
|
||||
/// Result of [`PodcastStore::apply_feed`].
|
||||
#[derive(Debug, Clone, Default, PartialEq)]
|
||||
pub struct RefreshOutcome {
|
||||
/// Episodes seen for the first time.
|
||||
pub new_episodes: usize,
|
||||
/// Known episodes whose metadata changed (title, audio URL, ...).
|
||||
pub updated_episodes: usize,
|
||||
/// Known episodes without metadata changes.
|
||||
pub unchanged_episodes: usize,
|
||||
/// Items without identity or with invalid metadata.
|
||||
pub skipped_items: usize,
|
||||
/// Feed URLs learned from a permanent redirect or `itunes:new-feed-url`.
|
||||
pub learned_aliases: Vec<FeedUrl>,
|
||||
/// URL the podcast is fetched from after this refresh.
|
||||
pub feed_url: Option<FeedUrl>,
|
||||
/// `podcast_feed_aliases_added`, only when the learned URLs changed the
|
||||
/// stored alias set. Append it to the sync log.
|
||||
pub ops: Vec<PodcastLocalOp>,
|
||||
}
|
||||
|
||||
/// Local-only fetch state of a podcast. Never part of sync output.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct FetchState {
|
||||
/// HTTP `ETag` validator of the last successful fetch.
|
||||
pub etag: Option<String>,
|
||||
/// HTTP `Last-Modified` validator of the last successful fetch.
|
||||
pub last_modified: Option<String>,
|
||||
/// Last fetch attempt, unix milliseconds.
|
||||
pub checked_at_ms: Option<i64>,
|
||||
/// Last successful fetch (including "not modified").
|
||||
pub last_success_at_ms: Option<i64>,
|
||||
/// When the scheduler should check again (`None`: due now).
|
||||
pub next_check_at_ms: Option<i64>,
|
||||
/// Error of the last attempt; `None` after a success.
|
||||
pub last_error: Option<String>,
|
||||
/// Failed attempts since the last success.
|
||||
pub consecutive_failures: u32,
|
||||
/// Image URL the cached cover was downloaded from.
|
||||
pub cover_url: Option<String>,
|
||||
/// Local path of the cached cover.
|
||||
pub cover_path: Option<String>,
|
||||
}
|
||||
|
||||
fn podcast_fetch_url(conn: &Connection, podcast_id: &PodcastId) -> Result<Option<FeedUrl>> {
|
||||
Ok(conn
|
||||
.query_row(
|
||||
"SELECT feed_url FROM podcasts WHERE podcast_id = ?1",
|
||||
[podcast_id.as_str()],
|
||||
|row| super::feed_url_col(row.get(0)?),
|
||||
)
|
||||
.optional()?)
|
||||
}
|
||||
|
||||
fn fetch_state_in(conn: &Connection, podcast_id: &PodcastId) -> Result<FetchState> {
|
||||
Ok(conn
|
||||
.query_row(
|
||||
"SELECT etag, last_modified, checked_at_ms, last_success_at_ms,
|
||||
next_check_at_ms, last_error, consecutive_failures, cover_url,
|
||||
cover_path
|
||||
FROM podcast_fetch_state WHERE podcast_id = ?1",
|
||||
[podcast_id.as_str()],
|
||||
|row| {
|
||||
Ok(FetchState {
|
||||
etag: row.get(0)?,
|
||||
last_modified: row.get(1)?,
|
||||
checked_at_ms: row.get(2)?,
|
||||
last_success_at_ms: row.get(3)?,
|
||||
next_check_at_ms: row.get(4)?,
|
||||
last_error: row.get(5)?,
|
||||
consecutive_failures: u32::try_from(row.get::<_, i64>(6)?).unwrap_or(u32::MAX),
|
||||
cover_url: row.get(7)?,
|
||||
cover_path: row.get(8)?,
|
||||
})
|
||||
},
|
||||
)
|
||||
.optional()?
|
||||
.unwrap_or_default())
|
||||
}
|
||||
|
||||
fn has_personal_state(conn: &Connection, episode_id: &EpisodeId) -> Result<bool> {
|
||||
Ok(conn.query_row(
|
||||
"SELECT EXISTS (SELECT 1 FROM episode_progress WHERE episode_id = ?1)
|
||||
OR EXISTS (SELECT 1 FROM podcast_listen_sessions WHERE episode_id = ?1)",
|
||||
[episode_id.as_str()],
|
||||
|row| row.get(0),
|
||||
)?)
|
||||
}
|
||||
|
||||
/// Re-keys episodes imported from the prototype whose stored `guid` was the
|
||||
/// enclosure URL (`legacy_guid`) when the feed shows it is a real GUID.
|
||||
///
|
||||
/// Only episodes without progress or sessions move, so no personal state is
|
||||
/// touched; otherwise the GUID item becomes a separate episode. When another
|
||||
/// episode already holds the GUID id (for example one a remote operation
|
||||
/// created), the imported episode is a duplicate of it and is deleted.
|
||||
fn rekey_legacy_guids(conn: &Connection, podcast_id: &PodcastId, feed: &ParsedFeed) -> Result<()> {
|
||||
for item in &feed.episodes {
|
||||
let Some(guid) = item.guid.as_deref() else {
|
||||
continue;
|
||||
};
|
||||
let Some(guid_id) = EpisodeId::from_guid(podcast_id, guid) else {
|
||||
continue;
|
||||
};
|
||||
let taken = conn
|
||||
.query_row(
|
||||
"SELECT 1 FROM episodes WHERE episode_id = ?1",
|
||||
[guid_id.as_str()],
|
||||
|_| Ok(()),
|
||||
)
|
||||
.optional()?
|
||||
.is_some();
|
||||
let legacy: Option<String> = conn
|
||||
.query_row(
|
||||
"SELECT episode_id FROM episodes
|
||||
WHERE podcast_id = ?1 AND legacy_guid = ?2 ORDER BY episode_id LIMIT 1",
|
||||
params![podcast_id.as_str(), guid],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.optional()?;
|
||||
let Some(legacy) = legacy else {
|
||||
continue;
|
||||
};
|
||||
let legacy = super::episode_id_col(legacy)?;
|
||||
if legacy == guid_id || has_personal_state(conn, &legacy)? {
|
||||
continue;
|
||||
}
|
||||
if taken {
|
||||
conn.execute(
|
||||
"DELETE FROM episodes WHERE episode_id = ?1",
|
||||
[legacy.as_str()],
|
||||
)?;
|
||||
continue;
|
||||
}
|
||||
conn.execute(
|
||||
"UPDATE episodes SET episode_id = ?2, guid = ?3, legacy_guid = NULL
|
||||
WHERE episode_id = ?1",
|
||||
params![legacy.as_str(), guid_id.as_str(), guid],
|
||||
)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Episodes imported from the prototype without a GUID (`legacy_guid` set)
|
||||
/// that carry no progress and no session.
|
||||
///
|
||||
/// Their ids were resolved over every stored prototype row at once, stale
|
||||
/// rows included, so they may differ from what a fresh device derives from
|
||||
/// the current feed. They are provisional until a refresh confirms them.
|
||||
fn provisional_episode_ids(
|
||||
conn: &Connection,
|
||||
podcast_id: &PodcastId,
|
||||
) -> Result<BTreeSet<EpisodeId>> {
|
||||
let mut statement = conn.prepare(
|
||||
"SELECT e.episode_id FROM episodes e
|
||||
WHERE e.podcast_id = ?1 AND e.legacy_guid IS NOT NULL
|
||||
AND NOT EXISTS (SELECT 1 FROM episode_progress p WHERE p.episode_id = e.episode_id)
|
||||
AND NOT EXISTS (SELECT 1 FROM podcast_listen_sessions s
|
||||
WHERE s.episode_id = e.episode_id)",
|
||||
)?;
|
||||
let ids = statement
|
||||
.query_map([podcast_id.as_str()], |row| {
|
||||
super::episode_id_col(row.get(0)?)
|
||||
})?
|
||||
.collect::<rusqlite::Result<BTreeSet<_>>>()?;
|
||||
Ok(ids)
|
||||
}
|
||||
|
||||
/// Resolves the ids of the feed items as a device without prototype history
|
||||
/// would, and moves provisional imported episodes onto them.
|
||||
///
|
||||
/// * The ids come from the contract identity rule against every stored
|
||||
/// episode except the provisional ones (episodes from earlier refreshes,
|
||||
/// from sync, and imported ones that already carry personal state).
|
||||
/// * Each provisional episode is matched to the item it was imported from
|
||||
/// with the same rule (enclosure, then link, then date and title; GUIDs
|
||||
/// ignored, since the prototype may have stored the enclosure URL as GUID)
|
||||
/// and re-keyed to that item's id. When another stored episode that stays
|
||||
/// already holds that id, it is the item's episode and the provisional one
|
||||
/// is a duplicate: it is deleted. Unmatched provisional episodes stay as
|
||||
/// they are (not in the feed); the next refresh that lists them re-keys
|
||||
/// them.
|
||||
///
|
||||
/// Provisional episodes have no progress and no session, so no personal
|
||||
/// state moves. The refresh then clears `legacy_guid` on every episode it
|
||||
/// lists, which makes their ids final.
|
||||
fn resolve_and_rekey_provisional(
|
||||
conn: &Connection,
|
||||
podcast_id: &PodcastId,
|
||||
feed: &ParsedFeed,
|
||||
) -> Result<Vec<Option<EpisodeId>>> {
|
||||
let known = apply::known_episodes(conn, podcast_id)?;
|
||||
let provisional = provisional_episode_ids(conn, podcast_id)?;
|
||||
if provisional.is_empty() {
|
||||
return Ok(feed.resolve_episode_ids(podcast_id, &known));
|
||||
}
|
||||
let (provisional_known, stable_known): (Vec<KnownEpisode>, Vec<KnownEpisode>) = known
|
||||
.into_iter()
|
||||
.partition(|episode| provisional.contains(&episode.episode_id));
|
||||
let ids = feed.resolve_episode_ids(podcast_id, &stable_known);
|
||||
|
||||
let guidless: Vec<EpisodeIdentityHints> = feed
|
||||
.identity_hints()
|
||||
.into_iter()
|
||||
.map(|hints| EpisodeIdentityHints {
|
||||
guid: None,
|
||||
..hints
|
||||
})
|
||||
.collect();
|
||||
let matches = resolve_episode_ids(
|
||||
podcast_id,
|
||||
feed.link.as_deref(),
|
||||
&guidless,
|
||||
&provisional_known,
|
||||
);
|
||||
let mut moves: Vec<(EpisodeId, EpisodeId)> = matches
|
||||
.into_iter()
|
||||
.zip(&ids)
|
||||
.filter_map(|(matched, target)| {
|
||||
let matched = matched.filter(|id| provisional.contains(id))?;
|
||||
let target = target.clone()?;
|
||||
(matched != target).then_some((matched, target))
|
||||
})
|
||||
.collect();
|
||||
// A target held by an episode that stays (for example a row a remote
|
||||
// operation created before this first refresh) is the item's episode
|
||||
// already: the provisional episode is a duplicate of it. It has no
|
||||
// progress and no session, and its metadata would be overwritten by the
|
||||
// item anyway, so it is removed instead of kept as a second copy. A
|
||||
// removed duplicate frees its own id for other moves.
|
||||
let mut occupied: BTreeSet<EpisodeId> = stable_known
|
||||
.iter()
|
||||
.chain(&provisional_known)
|
||||
.map(|episode| episode.episode_id.clone())
|
||||
.collect();
|
||||
let mut duplicates: Vec<EpisodeId> = Vec::new();
|
||||
loop {
|
||||
let leaving: BTreeSet<&EpisodeId> = moves.iter().map(|(from, _)| from).collect();
|
||||
let blocked: BTreeSet<EpisodeId> = moves
|
||||
.iter()
|
||||
.filter(|(_, to)| occupied.contains(to) && !leaving.contains(to))
|
||||
.map(|(from, _)| from.clone())
|
||||
.collect();
|
||||
if blocked.is_empty() {
|
||||
break;
|
||||
}
|
||||
moves.retain(|(from, _)| !blocked.contains(from));
|
||||
for from in blocked {
|
||||
occupied.remove(&from);
|
||||
duplicates.push(from);
|
||||
}
|
||||
}
|
||||
for duplicate in &duplicates {
|
||||
// Guarded again: only a provisional episode without personal state.
|
||||
conn.execute(
|
||||
"DELETE FROM episodes
|
||||
WHERE episode_id = ?1 AND legacy_guid IS NOT NULL
|
||||
AND NOT EXISTS (SELECT 1 FROM episode_progress p
|
||||
WHERE p.episode_id = episodes.episode_id)
|
||||
AND NOT EXISTS (SELECT 1 FROM podcast_listen_sessions s
|
||||
WHERE s.episode_id = episodes.episode_id)",
|
||||
[duplicate.as_str()],
|
||||
)?;
|
||||
}
|
||||
// Two phases, so a move may take an id another move vacates.
|
||||
for (index, (from, _)) in moves.iter().enumerate() {
|
||||
conn.execute(
|
||||
"UPDATE episodes SET episode_id = ?2 WHERE episode_id = ?1",
|
||||
params![from.as_str(), format!("rekey-pending:{index}")],
|
||||
)?;
|
||||
}
|
||||
for (index, (_, to)) in moves.iter().enumerate() {
|
||||
conn.execute(
|
||||
"UPDATE episodes SET episode_id = ?2 WHERE episode_id = ?1",
|
||||
params![format!("rekey-pending:{index}"), to.as_str()],
|
||||
)?;
|
||||
}
|
||||
Ok(ids)
|
||||
}
|
||||
|
||||
fn upsert_feed_episode(conn: &Connection, meta: &EpisodeMeta, now: i64) -> Result<Option<bool>> {
|
||||
let Some(stored) = super::queries::episode_meta_in(conn, &meta.episode_id)? else {
|
||||
apply::insert_episode(conn, meta, EpisodeSource::Feed, true, None, now, now)?;
|
||||
return Ok(None);
|
||||
};
|
||||
let changed = stored != *meta;
|
||||
conn.execute(
|
||||
"UPDATE episodes
|
||||
SET podcast_id = ?2, guid = ?3, title = ?4, description = ?5, season = ?6,
|
||||
episode_number = ?7, episode_type = ?8, published_at_ms = ?9,
|
||||
duration_ms = ?10, audio_url = ?11, mime = ?12, byte_len = ?13,
|
||||
image_url = ?14, link = ?15, source = 'feed', in_feed = 1,
|
||||
legacy_guid = NULL,
|
||||
updated_at_ms = CASE WHEN ?16 THEN ?17 ELSE updated_at_ms END
|
||||
WHERE episode_id = ?1",
|
||||
params![
|
||||
meta.episode_id.as_str(),
|
||||
meta.podcast_id.as_str(),
|
||||
meta.guid,
|
||||
meta.title,
|
||||
meta.description,
|
||||
meta.season,
|
||||
meta.episode_number,
|
||||
apply::episode_type_to_sql(meta.episode_type),
|
||||
meta.published_at_ms,
|
||||
meta.duration_ms,
|
||||
meta.audio_url,
|
||||
meta.mime,
|
||||
meta.byte_len.and_then(|len| i64::try_from(len).ok()),
|
||||
meta.image_url,
|
||||
meta.link,
|
||||
changed,
|
||||
now,
|
||||
],
|
||||
)?;
|
||||
Ok(Some(changed))
|
||||
}
|
||||
|
||||
impl PodcastStore {
|
||||
/// Ingests a fetched feed document for a podcast.
|
||||
///
|
||||
/// * Channel metadata of the catalog row is replaced by the feed's.
|
||||
/// * Episode ids are resolved with the contract identity rule
|
||||
/// (`ParsedFeed::resolve_episode_ids`) against every stored episode of
|
||||
/// the podcast, including episodes learned through sync, so a changed
|
||||
/// title, audio URL or a GUID-less item keeps its id. Known episodes
|
||||
/// get the new metadata (including the audio URL); new ones are added;
|
||||
/// episodes missing from the document are kept (`in_feed = false`).
|
||||
/// * A permanent redirect target and `itunes:new-feed-url` become feed
|
||||
/// aliases and the new fetch URL. When they change the stored alias
|
||||
/// set, one `podcast_feed_aliases_added` operation is emitted.
|
||||
/// * Subscription, share and progress registers and listening sessions
|
||||
/// are never touched, so a refresh cannot resurrect an unsubscribed
|
||||
/// podcast or move anyone's position.
|
||||
///
|
||||
/// Record the HTTP outcome separately with
|
||||
/// [`PodcastStore::record_fetch_success`] and friends.
|
||||
pub fn apply_feed(
|
||||
&self,
|
||||
stamps: &mut dyn StampSource,
|
||||
refresh: &FeedRefresh<'_>,
|
||||
) -> Result<RefreshOutcome> {
|
||||
let podcast_id = refresh.podcast_id;
|
||||
let feed = refresh.feed;
|
||||
let now = now_ms();
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
let current_url = match podcast_fetch_url(&tx, podcast_id)? {
|
||||
Some(url) => url,
|
||||
None => {
|
||||
let Some(register) = apply::load_subscription_register(&tx, podcast_id)? else {
|
||||
return Err(PodcastStoreError::UnknownPodcast(podcast_id.clone()).into());
|
||||
};
|
||||
apply::apply_subscription_record(&tx, ®ister, now)?;
|
||||
register.podcast.feed_url
|
||||
}
|
||||
};
|
||||
|
||||
rekey_legacy_guids(&tx, podcast_id, feed)?;
|
||||
let ids = resolve_and_rekey_provisional(&tx, podcast_id, feed)?;
|
||||
tx.execute(
|
||||
"UPDATE episodes SET in_feed = 0 WHERE podcast_id = ?1",
|
||||
[podcast_id.as_str()],
|
||||
)?;
|
||||
let mut outcome = RefreshOutcome::default();
|
||||
for (item, id) in feed.episodes.iter().zip(ids) {
|
||||
let Some(id) = id else {
|
||||
outcome.skipped_items += 1;
|
||||
continue;
|
||||
};
|
||||
let meta = item.to_meta(podcast_id, id);
|
||||
if meta.validate().is_err() {
|
||||
outcome.skipped_items += 1;
|
||||
continue;
|
||||
}
|
||||
match upsert_feed_episode(&tx, &meta, now)? {
|
||||
None => outcome.new_episodes += 1,
|
||||
Some(true) => outcome.updated_episodes += 1,
|
||||
Some(false) => outcome.unchanged_episodes += 1,
|
||||
}
|
||||
}
|
||||
|
||||
let mut fetch_url = current_url.clone();
|
||||
let mut learned = BTreeSet::new();
|
||||
if refresh.moved_permanently && *refresh.fetched_from != fetch_url {
|
||||
learned.insert(fetch_url.clone());
|
||||
learned.insert(refresh.fetched_from.clone());
|
||||
fetch_url = refresh.fetched_from.clone();
|
||||
}
|
||||
if let Some(new_url) = feed
|
||||
.new_feed_url
|
||||
.as_deref()
|
||||
.and_then(|url| FeedUrl::parse(url).ok())
|
||||
&& new_url != fetch_url
|
||||
{
|
||||
learned.insert(fetch_url.clone());
|
||||
learned.insert(new_url.clone());
|
||||
fetch_url = new_url;
|
||||
}
|
||||
let meta = feed.to_podcast_meta(fetch_url.clone());
|
||||
tx.execute(
|
||||
"UPDATE podcasts
|
||||
SET feed_url = ?2, title = ?3, author = ?4, description = ?5, link = ?6,
|
||||
image_url = ?7, meta_from_feed = 1, updated_at_ms = ?8
|
||||
WHERE podcast_id = ?1",
|
||||
params![
|
||||
podcast_id.as_str(),
|
||||
meta.feed_url.as_str(),
|
||||
meta.title,
|
||||
meta.author,
|
||||
meta.description,
|
||||
meta.link,
|
||||
meta.image_url,
|
||||
now
|
||||
],
|
||||
)?;
|
||||
if !learned.is_empty() {
|
||||
let learned: Vec<FeedUrl> = learned.into_iter().collect();
|
||||
let current = apply::load_aliases(&tx, podcast_id)?;
|
||||
if merge_feed_aliases(podcast_id, ¤t, &learned) != current {
|
||||
let op = super::actions::emit(&tx, stamps, |_| {
|
||||
SyncOpPayload::PodcastFeedAliasesAdded {
|
||||
podcast_id: podcast_id.clone(),
|
||||
feed_aliases: learned.clone(),
|
||||
}
|
||||
})?;
|
||||
outcome.ops.push(op);
|
||||
}
|
||||
outcome.learned_aliases = learned;
|
||||
}
|
||||
apply::refresh_seasons(&tx, podcast_id)?;
|
||||
tx.commit()?;
|
||||
outcome.feed_url = Some(fetch_url);
|
||||
Ok(outcome)
|
||||
}
|
||||
|
||||
/// Local fetch state of a podcast (defaults when never fetched).
|
||||
pub fn fetch_state(&self, podcast_id: &PodcastId) -> Result<FetchState> {
|
||||
fetch_state_in(&self.lock(), podcast_id)
|
||||
}
|
||||
|
||||
/// Read-modify-write of the fetch state under one lock and one immediate
|
||||
/// transaction, so concurrent `record_*` / `set_cover_cache` calls (from
|
||||
/// this or another process) never lose each other's columns or failure
|
||||
/// increments.
|
||||
fn update_fetch_state(
|
||||
&self,
|
||||
podcast_id: &PodcastId,
|
||||
update: impl FnOnce(&mut FetchState),
|
||||
) -> Result<FetchState> {
|
||||
let mut conn = self.lock();
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
if podcast_fetch_url(&tx, podcast_id)?.is_none() {
|
||||
return Err(PodcastStoreError::UnknownPodcast(podcast_id.clone()).into());
|
||||
}
|
||||
let mut state = fetch_state_in(&tx, podcast_id)?;
|
||||
update(&mut state);
|
||||
tx.execute(
|
||||
"INSERT INTO podcast_fetch_state
|
||||
(podcast_id, etag, last_modified, checked_at_ms, last_success_at_ms,
|
||||
next_check_at_ms, last_error, consecutive_failures, cover_url, cover_path)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10)
|
||||
ON CONFLICT(podcast_id) DO UPDATE SET
|
||||
etag = excluded.etag, last_modified = excluded.last_modified,
|
||||
checked_at_ms = excluded.checked_at_ms,
|
||||
last_success_at_ms = excluded.last_success_at_ms,
|
||||
next_check_at_ms = excluded.next_check_at_ms,
|
||||
last_error = excluded.last_error,
|
||||
consecutive_failures = excluded.consecutive_failures,
|
||||
cover_url = excluded.cover_url, cover_path = excluded.cover_path",
|
||||
params![
|
||||
podcast_id.as_str(),
|
||||
state.etag,
|
||||
state.last_modified,
|
||||
state.checked_at_ms,
|
||||
state.last_success_at_ms,
|
||||
state.next_check_at_ms,
|
||||
state.last_error,
|
||||
i64::from(state.consecutive_failures),
|
||||
state.cover_url,
|
||||
state.cover_path,
|
||||
],
|
||||
)?;
|
||||
tx.commit()?;
|
||||
Ok(state)
|
||||
}
|
||||
|
||||
/// Records a successful fetch (HTTP 200) with its new validators.
|
||||
pub fn record_fetch_success(
|
||||
&self,
|
||||
podcast_id: &PodcastId,
|
||||
now_ms: i64,
|
||||
etag: Option<&str>,
|
||||
last_modified: Option<&str>,
|
||||
next_check_at_ms: Option<i64>,
|
||||
) -> Result<()> {
|
||||
self.update_fetch_state(podcast_id, |state| {
|
||||
state.etag = etag.map(str::to_string);
|
||||
state.last_modified = last_modified.map(str::to_string);
|
||||
state.checked_at_ms = Some(now_ms);
|
||||
state.last_success_at_ms = Some(now_ms);
|
||||
state.next_check_at_ms = next_check_at_ms;
|
||||
state.last_error = None;
|
||||
state.consecutive_failures = 0;
|
||||
})
|
||||
.map(|_| ())
|
||||
}
|
||||
|
||||
/// Records a conditional fetch answered with "304 Not Modified".
|
||||
pub fn record_fetch_not_modified(
|
||||
&self,
|
||||
podcast_id: &PodcastId,
|
||||
now_ms: i64,
|
||||
next_check_at_ms: Option<i64>,
|
||||
) -> Result<()> {
|
||||
self.update_fetch_state(podcast_id, |state| {
|
||||
state.checked_at_ms = Some(now_ms);
|
||||
state.last_success_at_ms = Some(now_ms);
|
||||
state.next_check_at_ms = next_check_at_ms;
|
||||
state.last_error = None;
|
||||
state.consecutive_failures = 0;
|
||||
})
|
||||
.map(|_| ())
|
||||
}
|
||||
|
||||
/// Records a failed fetch; returns the consecutive failure count. The
|
||||
/// stored catalog is kept.
|
||||
pub fn record_fetch_failure(
|
||||
&self,
|
||||
podcast_id: &PodcastId,
|
||||
now_ms: i64,
|
||||
error: &str,
|
||||
next_check_at_ms: Option<i64>,
|
||||
) -> Result<u32> {
|
||||
self.update_fetch_state(podcast_id, |state| {
|
||||
state.checked_at_ms = Some(now_ms);
|
||||
state.next_check_at_ms = next_check_at_ms;
|
||||
state.last_error = Some(error.to_string());
|
||||
state.consecutive_failures = state.consecutive_failures.saturating_add(1);
|
||||
})
|
||||
.map(|state| state.consecutive_failures)
|
||||
}
|
||||
|
||||
/// Records the locally cached cover image of a podcast.
|
||||
pub fn set_cover_cache(
|
||||
&self,
|
||||
podcast_id: &PodcastId,
|
||||
cover_url: Option<&str>,
|
||||
cover_path: Option<&str>,
|
||||
) -> Result<()> {
|
||||
self.update_fetch_state(podcast_id, |state| {
|
||||
state.cover_url = cover_url.map(str::to_string);
|
||||
state.cover_path = cover_path.map(str::to_string);
|
||||
})
|
||||
.map(|_| ())
|
||||
}
|
||||
|
||||
/// Subscribed podcasts due for a refresh at `now_ms` (never checked or
|
||||
/// `next_check_at_ms <= now_ms`), most overdue first, with their fetch URL.
|
||||
pub fn podcasts_due_for_refresh(
|
||||
&self,
|
||||
now_ms: i64,
|
||||
limit: usize,
|
||||
) -> Result<Vec<(PodcastId, FeedUrl)>> {
|
||||
let conn = self.lock();
|
||||
let mut statement = conn.prepare(
|
||||
"SELECT p.podcast_id, p.feed_url
|
||||
FROM podcasts p
|
||||
JOIN podcast_subscriptions s ON s.podcast_id = p.podcast_id AND s.subscribed = 1
|
||||
LEFT JOIN podcast_fetch_state f ON f.podcast_id = p.podcast_id
|
||||
WHERE f.next_check_at_ms IS NULL OR f.next_check_at_ms <= ?1
|
||||
ORDER BY IFNULL(f.next_check_at_ms, 0), p.podcast_id
|
||||
LIMIT ?2",
|
||||
)?;
|
||||
let due = statement
|
||||
.query_map(
|
||||
params![now_ms, i64::try_from(limit).unwrap_or(i64::MAX)],
|
||||
|row| {
|
||||
Ok((
|
||||
super::podcast_id_col(row.get(0)?)?,
|
||||
super::feed_url_col(row.get(1)?)?,
|
||||
))
|
||||
},
|
||||
)?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?;
|
||||
Ok(due)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,796 @@
|
||||
//! Schema, migrations and the import of the prototype database.
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use anyhow::{Context as _, Result};
|
||||
use music_dht::podcast::{
|
||||
EpisodeId, EpisodeIdentityHints, EpisodeMeta, EpisodeType, FeedUrl, KnownEpisode,
|
||||
MAX_DESCRIPTION_CHARS, MAX_TITLE_CHARS, MAX_URL_BYTES, PodcastId, PodcastMeta,
|
||||
SubscriptionRecord, bounded_text, is_http_url, normalize_episode_title, resolve_episode_ids,
|
||||
resolve_podcast_id,
|
||||
};
|
||||
use rusqlite::{Connection, OptionalExtension as _, Transaction, TransactionBehavior, params};
|
||||
|
||||
use super::apply::{self, EpisodeSource};
|
||||
use super::{LEGACY_IMPORT_OP_PREFIX, PodcastStoreError, now_ms};
|
||||
|
||||
/// Newest schema version (`PRAGMA user_version`) this build writes.
|
||||
pub const PODCAST_SCHEMA_VERSION: i64 = 2;
|
||||
|
||||
/// `user_version` of the prototype database of the original podcast PR.
|
||||
pub const LEGACY_PROTOTYPE_SCHEMA_VERSION: i64 = 1;
|
||||
|
||||
/// Migration 1: the prototype schema of the original podcast PR, verbatim
|
||||
/// (`furumi-tui` `src/podcasts/store.rs`, `MIGRATIONS[0]`). It versioned
|
||||
/// itself with `PRAGMA user_version = 1` in the same transaction.
|
||||
const PROTOTYPE_SCHEMA_V1: &str = "
|
||||
CREATE TABLE IF NOT EXISTS podcasts (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
feed_url TEXT NOT NULL UNIQUE,
|
||||
title TEXT NOT NULL,
|
||||
author TEXT,
|
||||
description TEXT,
|
||||
link TEXT,
|
||||
image_url TEXT,
|
||||
cover_url TEXT,
|
||||
cover_path TEXT,
|
||||
etag TEXT,
|
||||
last_modified TEXT,
|
||||
added_at INTEGER NOT NULL,
|
||||
last_checked_at INTEGER,
|
||||
last_error TEXT
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS podcast_episodes (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
podcast_id INTEGER NOT NULL REFERENCES podcasts(id) ON DELETE CASCADE,
|
||||
guid TEXT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
description TEXT,
|
||||
published_at INTEGER,
|
||||
duration_seconds REAL,
|
||||
audio_url TEXT NOT NULL,
|
||||
mime_type TEXT,
|
||||
byte_len INTEGER,
|
||||
link TEXT,
|
||||
image_url TEXT,
|
||||
first_seen_at INTEGER NOT NULL,
|
||||
UNIQUE (podcast_id, guid)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS podcast_episodes_by_date
|
||||
ON podcast_episodes (podcast_id, published_at DESC, id DESC);
|
||||
";
|
||||
|
||||
#[cfg(test)]
|
||||
pub(super) const PROTOTYPE_SCHEMA_V1_FOR_TESTS: &str = PROTOTYPE_SCHEMA_V1;
|
||||
|
||||
/// Columns every prototype `podcasts` table has.
|
||||
const PROTOTYPE_PODCAST_COLUMNS: &[&str] = &[
|
||||
"id",
|
||||
"feed_url",
|
||||
"title",
|
||||
"author",
|
||||
"description",
|
||||
"link",
|
||||
"image_url",
|
||||
"cover_url",
|
||||
"cover_path",
|
||||
"etag",
|
||||
"last_modified",
|
||||
"added_at",
|
||||
"last_checked_at",
|
||||
"last_error",
|
||||
];
|
||||
|
||||
/// Columns every prototype `podcast_episodes` table has.
|
||||
const PROTOTYPE_EPISODE_COLUMNS: &[&str] = &[
|
||||
"id",
|
||||
"podcast_id",
|
||||
"guid",
|
||||
"title",
|
||||
"description",
|
||||
"published_at",
|
||||
"duration_seconds",
|
||||
"audio_url",
|
||||
"mime_type",
|
||||
"byte_len",
|
||||
"link",
|
||||
"image_url",
|
||||
"first_seen_at",
|
||||
];
|
||||
|
||||
/// Migration 2: the contract model. See `PODCASTS.md`, "SQLite reference store".
|
||||
const SCHEMA_V2: &str = "
|
||||
CREATE TABLE IF NOT EXISTS podcasts (
|
||||
podcast_id TEXT PRIMARY KEY,
|
||||
feed_url TEXT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
author TEXT,
|
||||
description TEXT,
|
||||
link TEXT,
|
||||
image_url TEXT,
|
||||
meta_from_feed INTEGER NOT NULL DEFAULT 0,
|
||||
created_at_ms INTEGER NOT NULL,
|
||||
updated_at_ms INTEGER NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS podcast_feed_aliases (
|
||||
podcast_id TEXT NOT NULL,
|
||||
feed_url TEXT NOT NULL,
|
||||
PRIMARY KEY (podcast_id, feed_url)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS podcast_feed_aliases_by_url
|
||||
ON podcast_feed_aliases (feed_url);
|
||||
CREATE TABLE IF NOT EXISTS episodes (
|
||||
episode_id TEXT PRIMARY KEY,
|
||||
podcast_id TEXT NOT NULL REFERENCES podcasts(podcast_id),
|
||||
guid TEXT,
|
||||
title TEXT NOT NULL,
|
||||
description TEXT,
|
||||
season INTEGER,
|
||||
episode_number INTEGER,
|
||||
episode_type TEXT NOT NULL DEFAULT 'full',
|
||||
published_at_ms INTEGER,
|
||||
duration_ms INTEGER,
|
||||
audio_url TEXT NOT NULL,
|
||||
mime TEXT,
|
||||
byte_len INTEGER,
|
||||
image_url TEXT,
|
||||
link TEXT,
|
||||
source TEXT NOT NULL CHECK (source IN ('feed', 'sync')),
|
||||
in_feed INTEGER NOT NULL DEFAULT 0,
|
||||
legacy_guid TEXT,
|
||||
first_seen_at_ms INTEGER NOT NULL,
|
||||
updated_at_ms INTEGER NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS episodes_by_podcast_date
|
||||
ON episodes (podcast_id, published_at_ms DESC);
|
||||
CREATE INDEX IF NOT EXISTS episodes_by_podcast_season
|
||||
ON episodes (podcast_id, season, episode_number);
|
||||
CREATE TABLE IF NOT EXISTS podcast_seasons (
|
||||
podcast_id TEXT NOT NULL REFERENCES podcasts(podcast_id),
|
||||
season INTEGER,
|
||||
episode_count INTEGER NOT NULL,
|
||||
first_published_at_ms INTEGER,
|
||||
last_published_at_ms INTEGER
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS podcast_seasons_key
|
||||
ON podcast_seasons (podcast_id, IFNULL(season, -1));
|
||||
CREATE TABLE IF NOT EXISTS podcast_subscriptions (
|
||||
podcast_id TEXT PRIMARY KEY,
|
||||
subscribed INTEGER NOT NULL,
|
||||
podcast_json TEXT NOT NULL,
|
||||
generation INTEGER NOT NULL CHECK (generation >= 0),
|
||||
hlc_ms INTEGER NOT NULL,
|
||||
op_id TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS podcast_shares (
|
||||
podcast_id TEXT PRIMARY KEY,
|
||||
shared INTEGER NOT NULL,
|
||||
generation INTEGER NOT NULL CHECK (generation >= 0),
|
||||
hlc_ms INTEGER NOT NULL,
|
||||
op_id TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS episode_progress (
|
||||
episode_id TEXT PRIMARY KEY,
|
||||
podcast_id TEXT NOT NULL,
|
||||
position_ms INTEGER NOT NULL,
|
||||
status TEXT NOT NULL
|
||||
CHECK (status IN ('not_started', 'in_progress', 'completed')),
|
||||
generation INTEGER NOT NULL CHECK (generation >= 0),
|
||||
session_id TEXT NOT NULL,
|
||||
hlc_ms INTEGER NOT NULL,
|
||||
device_id TEXT NOT NULL,
|
||||
op_id TEXT NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS episode_progress_by_podcast
|
||||
ON episode_progress (podcast_id);
|
||||
CREATE TABLE IF NOT EXISTS podcast_listen_sessions (
|
||||
session_id TEXT PRIMARY KEY,
|
||||
episode_id TEXT NOT NULL,
|
||||
podcast_id TEXT NOT NULL,
|
||||
device_id TEXT NOT NULL,
|
||||
started_at_ms INTEGER NOT NULL,
|
||||
ended_at_ms INTEGER NOT NULL,
|
||||
start_position_ms INTEGER NOT NULL,
|
||||
end_position_ms INTEGER NOT NULL,
|
||||
listened_ms INTEGER NOT NULL,
|
||||
completed INTEGER NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS podcast_listen_sessions_by_episode
|
||||
ON podcast_listen_sessions (episode_id);
|
||||
CREATE INDEX IF NOT EXISTS podcast_listen_sessions_by_end
|
||||
ON podcast_listen_sessions (ended_at_ms DESC);
|
||||
CREATE TABLE IF NOT EXISTS podcast_fetch_state (
|
||||
podcast_id TEXT PRIMARY KEY REFERENCES podcasts(podcast_id),
|
||||
etag TEXT,
|
||||
last_modified TEXT,
|
||||
checked_at_ms INTEGER,
|
||||
last_success_at_ms INTEGER,
|
||||
next_check_at_ms INTEGER,
|
||||
last_error TEXT,
|
||||
consecutive_failures INTEGER NOT NULL DEFAULT 0,
|
||||
cover_url TEXT,
|
||||
cover_path TEXT
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS legacy_quarantine (
|
||||
legacy_table TEXT NOT NULL,
|
||||
legacy_id INTEGER NOT NULL,
|
||||
reason TEXT NOT NULL,
|
||||
detail TEXT,
|
||||
PRIMARY KEY (legacy_table, legacy_id)
|
||||
);
|
||||
";
|
||||
|
||||
/// Tables a version-2 store owns (the `legacy_*` tables included).
|
||||
const V2_TABLES: &[&str] = &[
|
||||
"podcasts",
|
||||
"podcast_feed_aliases",
|
||||
"episodes",
|
||||
"podcast_seasons",
|
||||
"podcast_subscriptions",
|
||||
"podcast_shares",
|
||||
"episode_progress",
|
||||
"podcast_listen_sessions",
|
||||
"podcast_fetch_state",
|
||||
"legacy_quarantine",
|
||||
"legacy_podcasts",
|
||||
"legacy_podcast_episodes",
|
||||
];
|
||||
|
||||
fn user_version(conn: &Connection) -> Result<i64> {
|
||||
Ok(conn.query_row("PRAGMA user_version", [], |row| row.get(0))?)
|
||||
}
|
||||
|
||||
fn user_tables(conn: &Connection) -> Result<BTreeSet<String>> {
|
||||
let mut statement = conn.prepare(
|
||||
"SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'",
|
||||
)?;
|
||||
let names = statement
|
||||
.query_map([], |row| row.get::<_, String>(0))?
|
||||
.collect::<rusqlite::Result<BTreeSet<_>>>()?;
|
||||
Ok(names)
|
||||
}
|
||||
|
||||
pub(crate) fn table_columns(conn: &Connection, table: &str) -> Result<BTreeSet<String>> {
|
||||
let mut statement = conn.prepare("SELECT name FROM pragma_table_info(?1)")?;
|
||||
let names = statement
|
||||
.query_map([table], |row| row.get::<_, String>(0))?
|
||||
.collect::<rusqlite::Result<BTreeSet<_>>>()?;
|
||||
Ok(names)
|
||||
}
|
||||
|
||||
fn unrecognized(reason: impl Into<String>) -> anyhow::Error {
|
||||
PodcastStoreError::UnrecognizedDatabase(reason.into()).into()
|
||||
}
|
||||
|
||||
fn has_columns(conn: &Connection, table: &str, required: &[&str]) -> Result<bool> {
|
||||
let columns = table_columns(conn, table)?;
|
||||
Ok(required.iter().all(|column| columns.contains(*column)))
|
||||
}
|
||||
|
||||
/// Checks the prototype tables that exist have the prototype shape.
|
||||
fn check_prototype_shape(conn: &Connection, tables: &BTreeSet<String>) -> Result<()> {
|
||||
if tables.contains("podcasts") && !has_columns(conn, "podcasts", PROTOTYPE_PODCAST_COLUMNS)? {
|
||||
return Err(unrecognized(
|
||||
"table `podcasts` does not have the prototype columns",
|
||||
));
|
||||
}
|
||||
if tables.contains("podcast_episodes")
|
||||
&& !has_columns(conn, "podcast_episodes", PROTOTYPE_EPISODE_COLUMNS)?
|
||||
{
|
||||
return Err(unrecognized(
|
||||
"table `podcast_episodes` does not have the prototype columns",
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Read-only check run before anything is written to the file.
|
||||
///
|
||||
/// * `user_version` above [`PODCAST_SCHEMA_VERSION`]: written by a newer
|
||||
/// Furumi, refused.
|
||||
/// * 0: must be empty, or hold only the prototype tables (the prototype
|
||||
/// sets its version in the same transaction, so this is defensive).
|
||||
/// * 1: the prototype database; its tables must have the prototype shape.
|
||||
/// * 2: the current schema.
|
||||
///
|
||||
/// Any other table means the path points at a different database (for
|
||||
/// example `library.db`), which is refused instead of being modified.
|
||||
pub(super) fn check_openable(conn: &Connection) -> Result<()> {
|
||||
let version = user_version(conn)?;
|
||||
if version > PODCAST_SCHEMA_VERSION {
|
||||
return Err(PodcastStoreError::NewerSchema {
|
||||
found: version,
|
||||
supported: PODCAST_SCHEMA_VERSION,
|
||||
}
|
||||
.into());
|
||||
}
|
||||
if version < 0 {
|
||||
return Err(unrecognized(format!("negative user_version {version}")));
|
||||
}
|
||||
let tables = user_tables(conn)?;
|
||||
let allowed: &[&str] = if version < PODCAST_SCHEMA_VERSION {
|
||||
&["podcasts", "podcast_episodes"]
|
||||
} else {
|
||||
V2_TABLES
|
||||
};
|
||||
if let Some(table) = tables
|
||||
.iter()
|
||||
.find(|table| !allowed.contains(&table.as_str()))
|
||||
{
|
||||
return Err(unrecognized(format!(
|
||||
"unexpected table `{table}` at schema version {version}"
|
||||
)));
|
||||
}
|
||||
if version < PODCAST_SCHEMA_VERSION {
|
||||
check_prototype_shape(conn, &tables)?;
|
||||
} else if tables.contains("podcasts")
|
||||
&& !has_columns(conn, "podcasts", &["podcast_id", "feed_url"])?
|
||||
{
|
||||
return Err(unrecognized("table `podcasts` is not the version 2 table"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Applies every pending migration, each in its own immediate transaction
|
||||
/// that re-reads the version (another process may have migrated meanwhile).
|
||||
pub(super) fn migrate(conn: &mut Connection) -> Result<()> {
|
||||
for target in 1..=PODCAST_SCHEMA_VERSION {
|
||||
let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
|
||||
let version = user_version(&tx)?;
|
||||
if version > PODCAST_SCHEMA_VERSION {
|
||||
return Err(PodcastStoreError::NewerSchema {
|
||||
found: version,
|
||||
supported: PODCAST_SCHEMA_VERSION,
|
||||
}
|
||||
.into());
|
||||
}
|
||||
if version >= target {
|
||||
continue;
|
||||
}
|
||||
match target {
|
||||
1 => tx
|
||||
.execute_batch(PROTOTYPE_SCHEMA_V1)
|
||||
.context("creating the prototype podcast schema (v1)")?,
|
||||
2 => migrate_to_v2(&tx).context("migrating the podcast database to v2")?,
|
||||
_ => unreachable!("no migration for version {target}"),
|
||||
}
|
||||
tx.pragma_update(None, "user_version", target)?;
|
||||
tx.commit()
|
||||
.with_context(|| format!("committing podcast schema v{target}"))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn migrate_to_v2(tx: &Transaction<'_>) -> Result<()> {
|
||||
let tables = user_tables(tx)?;
|
||||
// Rename, never drop: the prototype data stays available as `legacy_*`.
|
||||
// Renaming `podcasts` also rewrites the foreign key of the episode table.
|
||||
if tables.contains("podcasts") && !tables.contains("legacy_podcasts") {
|
||||
tx.execute_batch("ALTER TABLE podcasts RENAME TO legacy_podcasts")?;
|
||||
}
|
||||
if tables.contains("podcast_episodes") && !tables.contains("legacy_podcast_episodes") {
|
||||
tx.execute_batch("ALTER TABLE podcast_episodes RENAME TO legacy_podcast_episodes")?;
|
||||
}
|
||||
tx.execute_batch(SCHEMA_V2)?;
|
||||
let tables = user_tables(tx)?;
|
||||
if tables.contains("legacy_podcasts") {
|
||||
import_prototype(tx, tables.contains("legacy_podcast_episodes"))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
struct LegacyPodcast {
|
||||
id: i64,
|
||||
feed_url: String,
|
||||
title: String,
|
||||
author: Option<String>,
|
||||
description: Option<String>,
|
||||
link: Option<String>,
|
||||
image_url: Option<String>,
|
||||
cover_url: Option<String>,
|
||||
cover_path: Option<String>,
|
||||
etag: Option<String>,
|
||||
last_modified: Option<String>,
|
||||
added_at: i64,
|
||||
last_checked_at: Option<i64>,
|
||||
last_error: Option<String>,
|
||||
}
|
||||
|
||||
struct LegacyEpisode {
|
||||
id: i64,
|
||||
guid: String,
|
||||
title: String,
|
||||
description: Option<String>,
|
||||
published_at: Option<i64>,
|
||||
duration_seconds: Option<f64>,
|
||||
audio_url: String,
|
||||
mime_type: Option<String>,
|
||||
byte_len: Option<i64>,
|
||||
link: Option<String>,
|
||||
image_url: Option<String>,
|
||||
first_seen_at: i64,
|
||||
}
|
||||
|
||||
fn quarantine(
|
||||
tx: &Transaction<'_>,
|
||||
table: &str,
|
||||
id: i64,
|
||||
reason: &str,
|
||||
detail: &str,
|
||||
) -> Result<()> {
|
||||
tx.execute(
|
||||
"INSERT OR IGNORE INTO legacy_quarantine (legacy_table, legacy_id, reason, detail)
|
||||
VALUES (?1, ?2, ?3, ?4)",
|
||||
params![table, id, reason, detail],
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn secs_to_ms(secs: i64) -> i64 {
|
||||
secs.saturating_mul(1_000)
|
||||
}
|
||||
|
||||
/// Imports the prototype tables into the contract model.
|
||||
///
|
||||
/// * Every prototype row is an active subscription: it becomes a
|
||||
/// subscription register with `subscribed = true`, generation 0,
|
||||
/// `hlc_ms` = `added_at` and `op_id` = [`LEGACY_IMPORT_OP_PREFIX`] + id,
|
||||
/// so any later contract write orders after it. Other devices learn it
|
||||
/// through the snapshot sections. Nothing is shared (private by default).
|
||||
/// * The podcast id is derived from the canonical feed URL
|
||||
/// ([`FeedUrl::parse`], falling back to [`FeedUrl::from_user_input`]); rows
|
||||
/// whose URL cannot be canonicalized are recorded in `legacy_quarantine`
|
||||
/// and stay in `legacy_podcasts` with their episodes. Rows with the same
|
||||
/// canonical URL merge into one podcast (the first row's subscription and
|
||||
/// fetch state win; the episodes of all of them are imported).
|
||||
/// * Episodes get ids from the contract identity rule over the stored rows.
|
||||
/// The prototype stored the enclosure URL as `guid` when the item had
|
||||
/// none: such rows are imported as GUID-less items and keep the stored
|
||||
/// value in `legacy_guid`. Rows of one GUID-less item seen under several
|
||||
/// enclosure URLs are collapsed into the newest (the others are
|
||||
/// quarantined as duplicates). Because stale rows can still change which
|
||||
/// key is unique, these ids are provisional: the first refresh that lists
|
||||
/// the item re-keys an episode without progress or sessions to the id the
|
||||
/// current feed gives (see `refresh.rs`), including its GUID id when the
|
||||
/// stored value turns out to be a real GUID.
|
||||
/// * ETag, Last-Modified, check time, last error and the cover cache move to
|
||||
/// the local fetch state. The prototype has no progress or history, so no
|
||||
/// progress register or session is created.
|
||||
fn import_prototype(tx: &Transaction<'_>, with_episodes: bool) -> Result<()> {
|
||||
let now = now_ms();
|
||||
let legacy: Vec<LegacyPodcast> = {
|
||||
let mut statement = tx.prepare(
|
||||
"SELECT id, feed_url, title, author, description, link, image_url, cover_url,
|
||||
cover_path, etag, last_modified, added_at, last_checked_at, last_error
|
||||
FROM legacy_podcasts ORDER BY id",
|
||||
)?;
|
||||
statement
|
||||
.query_map([], |row| {
|
||||
Ok(LegacyPodcast {
|
||||
id: row.get(0)?,
|
||||
feed_url: row.get(1)?,
|
||||
title: row.get(2)?,
|
||||
author: row.get(3)?,
|
||||
description: row.get(4)?,
|
||||
link: row.get(5)?,
|
||||
image_url: row.get(6)?,
|
||||
cover_url: row.get(7)?,
|
||||
cover_path: row.get(8)?,
|
||||
etag: row.get(9)?,
|
||||
last_modified: row.get(10)?,
|
||||
added_at: row.get(11)?,
|
||||
last_checked_at: row.get(12)?,
|
||||
last_error: row.get(13)?,
|
||||
})
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?
|
||||
};
|
||||
let mut known_urls: Vec<(PodcastId, FeedUrl)> = Vec::new();
|
||||
for podcast in legacy {
|
||||
let url = match FeedUrl::parse(&podcast.feed_url)
|
||||
.or_else(|_| FeedUrl::from_user_input(&podcast.feed_url))
|
||||
{
|
||||
Ok(url) => url,
|
||||
Err(error) => {
|
||||
quarantine(
|
||||
tx,
|
||||
"legacy_podcasts",
|
||||
podcast.id,
|
||||
"invalid feed URL",
|
||||
&error.to_string(),
|
||||
)?;
|
||||
continue;
|
||||
}
|
||||
};
|
||||
let podcast_id = resolve_podcast_id(&url, known_urls.iter().map(|(id, url)| (id, url)));
|
||||
let duplicate = known_urls.iter().any(|(id, _)| *id == podcast_id);
|
||||
known_urls.push((podcast_id.clone(), url.clone()));
|
||||
if duplicate {
|
||||
// Same canonical feed as an earlier row: its episodes join that
|
||||
// podcast; the first row's subscription and fetch state are kept.
|
||||
if with_episodes {
|
||||
import_prototype_episodes(tx, &podcast, &podcast_id, now)?;
|
||||
}
|
||||
apply::refresh_seasons(tx, &podcast_id)?;
|
||||
continue;
|
||||
}
|
||||
let meta = PodcastMeta {
|
||||
feed_url: url.clone(),
|
||||
title: podcast.title.clone(),
|
||||
author: podcast.author.clone(),
|
||||
description: podcast.description.clone(),
|
||||
link: podcast.link.clone(),
|
||||
image_url: podcast.image_url.clone(),
|
||||
}
|
||||
.bounded();
|
||||
let added_ms = secs_to_ms(podcast.added_at);
|
||||
let record = SubscriptionRecord {
|
||||
podcast_id: podcast_id.clone(),
|
||||
subscribed: true,
|
||||
podcast: meta.clone(),
|
||||
feed_aliases: vec![url.clone()],
|
||||
generation: 0,
|
||||
hlc_ms: added_ms,
|
||||
op_id: format!("{LEGACY_IMPORT_OP_PREFIX}{podcast_id}"),
|
||||
};
|
||||
apply::apply_subscription_record(tx, &record, now)?;
|
||||
// The prototype metadata came from the feed itself.
|
||||
tx.execute(
|
||||
"UPDATE podcasts
|
||||
SET title = ?2, author = ?3, description = ?4, link = ?5, image_url = ?6,
|
||||
meta_from_feed = 1, created_at_ms = MIN(created_at_ms, ?7)
|
||||
WHERE podcast_id = ?1 AND meta_from_feed = 0",
|
||||
params![
|
||||
podcast_id.as_str(),
|
||||
meta.title,
|
||||
meta.author,
|
||||
meta.description,
|
||||
meta.link,
|
||||
meta.image_url,
|
||||
added_ms
|
||||
],
|
||||
)?;
|
||||
tx.execute(
|
||||
"INSERT OR IGNORE INTO podcast_fetch_state
|
||||
(podcast_id, etag, last_modified, checked_at_ms, last_success_at_ms,
|
||||
next_check_at_ms, last_error, consecutive_failures, cover_url, cover_path)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, NULL, ?6, ?7, ?8, ?9)",
|
||||
params![
|
||||
podcast_id.as_str(),
|
||||
podcast.etag,
|
||||
podcast.last_modified,
|
||||
podcast.last_checked_at.map(secs_to_ms),
|
||||
podcast
|
||||
.last_error
|
||||
.is_none()
|
||||
.then(|| podcast.last_checked_at.map(secs_to_ms))
|
||||
.flatten(),
|
||||
podcast.last_error,
|
||||
i64::from(podcast.last_error.is_some()),
|
||||
podcast.cover_url,
|
||||
podcast.cover_path,
|
||||
],
|
||||
)?;
|
||||
if with_episodes {
|
||||
import_prototype_episodes(tx, &podcast, &podcast_id, now)?;
|
||||
}
|
||||
apply::refresh_seasons(tx, &podcast_id)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn import_prototype_episodes(
|
||||
tx: &Transaction<'_>,
|
||||
podcast: &LegacyPodcast,
|
||||
podcast_id: &PodcastId,
|
||||
now: i64,
|
||||
) -> Result<()> {
|
||||
let rows: Vec<LegacyEpisode> = {
|
||||
let mut statement = tx.prepare(
|
||||
"SELECT id, guid, title, description, published_at, duration_seconds, audio_url,
|
||||
mime_type, byte_len, link, image_url, first_seen_at
|
||||
FROM legacy_podcast_episodes WHERE podcast_id = ?1 ORDER BY id",
|
||||
)?;
|
||||
statement
|
||||
.query_map([podcast.id], |row| {
|
||||
Ok(LegacyEpisode {
|
||||
id: row.get(0)?,
|
||||
guid: row.get(1)?,
|
||||
title: row.get(2)?,
|
||||
description: row.get(3)?,
|
||||
published_at: row.get(4)?,
|
||||
duration_seconds: row.get(5)?,
|
||||
audio_url: row.get(6)?,
|
||||
mime_type: row.get(7)?,
|
||||
byte_len: row.get(8)?,
|
||||
link: row.get(9)?,
|
||||
image_url: row.get(10)?,
|
||||
first_seen_at: row.get(11)?,
|
||||
})
|
||||
})?
|
||||
.collect::<rusqlite::Result<Vec<_>>>()?
|
||||
};
|
||||
let mut usable = Vec::new();
|
||||
for row in rows {
|
||||
if !is_http_url(&row.audio_url) {
|
||||
quarantine(
|
||||
tx,
|
||||
"legacy_podcast_episodes",
|
||||
row.id,
|
||||
"invalid audio URL",
|
||||
&row.audio_url,
|
||||
)?;
|
||||
continue;
|
||||
}
|
||||
usable.push(row);
|
||||
}
|
||||
// The prototype used the enclosure URL as GUID when the item had none.
|
||||
let real_guid = |row: &LegacyEpisode| {
|
||||
let guid = row.guid.trim();
|
||||
(!guid.is_empty() && guid != row.audio_url.trim()).then(|| guid.to_string())
|
||||
};
|
||||
let usable = collapse_moved_guidless_rows(tx, usable, real_guid)?;
|
||||
let hints: Vec<EpisodeIdentityHints> = usable
|
||||
.iter()
|
||||
.map(|row| EpisodeIdentityHints {
|
||||
guid: real_guid(row),
|
||||
enclosure_url: Some(row.audio_url.clone()),
|
||||
link: row.link.clone(),
|
||||
published_at_ms: row.published_at.map(secs_to_ms),
|
||||
title: Some(row.title.clone()),
|
||||
})
|
||||
.collect();
|
||||
let known: Vec<KnownEpisode> = apply::known_episodes(tx, podcast_id)?;
|
||||
let ids = resolve_episode_ids(podcast_id, podcast.link.as_deref(), &hints, &known);
|
||||
let mut seen: BTreeMap<EpisodeId, i64> = BTreeMap::new();
|
||||
for ((row, hint), id) in usable.iter().zip(&hints).zip(ids) {
|
||||
let Some(episode_id) = id else {
|
||||
quarantine(
|
||||
tx,
|
||||
"legacy_podcast_episodes",
|
||||
row.id,
|
||||
"episode cannot be identified",
|
||||
"no usable identity key or duplicate item",
|
||||
)?;
|
||||
continue;
|
||||
};
|
||||
if let Some(first) = seen.insert(episode_id.clone(), row.id) {
|
||||
quarantine(
|
||||
tx,
|
||||
"legacy_podcast_episodes",
|
||||
row.id,
|
||||
"duplicate episode",
|
||||
&format!("same identity as legacy episode {first}"),
|
||||
)?;
|
||||
continue;
|
||||
}
|
||||
let meta = EpisodeMeta {
|
||||
episode_id,
|
||||
podcast_id: podcast_id.clone(),
|
||||
guid: hint.guid.clone(),
|
||||
title: bounded_title(&row.title),
|
||||
description: row
|
||||
.description
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|text| !text.is_empty())
|
||||
.map(|text| bounded_text(text, MAX_DESCRIPTION_CHARS)),
|
||||
season: None,
|
||||
episode_number: None,
|
||||
episode_type: EpisodeType::Full,
|
||||
published_at_ms: row.published_at.map(secs_to_ms),
|
||||
duration_ms: row
|
||||
.duration_seconds
|
||||
.filter(|secs| secs.is_finite() && *secs > 0.0)
|
||||
.map(|secs| (secs * 1_000.0).round() as i64),
|
||||
audio_url: row.audio_url.clone(),
|
||||
mime: row.mime_type.clone().filter(|mime| {
|
||||
!mime.is_empty() && mime.len() <= 128 && mime.bytes().all(|b| b.is_ascii_graphic())
|
||||
}),
|
||||
byte_len: row.byte_len.and_then(|len| u64::try_from(len).ok()),
|
||||
image_url: row.image_url.clone().filter(|url| is_http_url(url)),
|
||||
link: row.link.clone().filter(|link| link.len() <= MAX_URL_BYTES),
|
||||
};
|
||||
if let Err(error) = meta.validate() {
|
||||
quarantine(
|
||||
tx,
|
||||
"legacy_podcast_episodes",
|
||||
row.id,
|
||||
"invalid episode metadata",
|
||||
&error.to_string(),
|
||||
)?;
|
||||
continue;
|
||||
}
|
||||
let legacy_guid = hint.guid.is_none().then(|| row.guid.clone());
|
||||
let exists: bool = tx
|
||||
.query_row(
|
||||
"SELECT 1 FROM episodes WHERE episode_id = ?1",
|
||||
[meta.episode_id.as_str()],
|
||||
|_| Ok(true),
|
||||
)
|
||||
.optional()?
|
||||
.unwrap_or(false);
|
||||
if exists {
|
||||
continue;
|
||||
}
|
||||
apply::insert_episode(
|
||||
tx,
|
||||
&meta,
|
||||
EpisodeSource::Feed,
|
||||
true,
|
||||
legacy_guid.as_deref(),
|
||||
secs_to_ms(row.first_seen_at),
|
||||
now,
|
||||
)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Collapses prototype rows that are one GUID-less item seen under several
|
||||
/// enclosure URLs.
|
||||
///
|
||||
/// The prototype keyed GUID-less items by their enclosure URL, so an item
|
||||
/// whose audio URL changed (new CDN, tracking prefix) left one row per URL.
|
||||
/// Those stale rows are not part of the current feed; resolved together with
|
||||
/// it they would make the item's `<link>` and `(date, title)` look
|
||||
/// non-unique and give it an id a fresh device never derives. Rows with the
|
||||
/// same publication second, normalized title and `<link>` (or none) are the
|
||||
/// same item: the newest (`first_seen_at`, then `id`) is imported, the others
|
||||
/// are quarantined as duplicates. The link must match too, so distinct
|
||||
/// episodes that only share a title and second are never merged.
|
||||
fn collapse_moved_guidless_rows(
|
||||
tx: &Transaction<'_>,
|
||||
rows: Vec<LegacyEpisode>,
|
||||
real_guid: impl Fn(&LegacyEpisode) -> Option<String>,
|
||||
) -> Result<Vec<LegacyEpisode>> {
|
||||
type ItemKey = (i64, String, Option<String>);
|
||||
let key = |row: &LegacyEpisode| -> Option<ItemKey> {
|
||||
if real_guid(row).is_some() {
|
||||
return None;
|
||||
}
|
||||
let title = normalize_episode_title(&row.title);
|
||||
let link = row
|
||||
.link
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|link| !link.is_empty())
|
||||
.map(str::to_string);
|
||||
(!title.is_empty()).then_some((row.published_at?, title, link))
|
||||
};
|
||||
let mut newest: BTreeMap<ItemKey, (i64, i64)> = BTreeMap::new();
|
||||
for row in &rows {
|
||||
if let Some(key) = key(row) {
|
||||
let candidate = (row.first_seen_at, row.id);
|
||||
newest
|
||||
.entry(key)
|
||||
.and_modify(|best| *best = (*best).max(candidate))
|
||||
.or_insert(candidate);
|
||||
}
|
||||
}
|
||||
let mut kept = Vec::with_capacity(rows.len());
|
||||
for row in rows {
|
||||
let keeper = key(&row).and_then(|key| newest.get(&key).map(|(_, id)| *id));
|
||||
match keeper {
|
||||
Some(keeper) if keeper != row.id => quarantine(
|
||||
tx,
|
||||
"legacy_podcast_episodes",
|
||||
row.id,
|
||||
"duplicate episode",
|
||||
&format!(
|
||||
"older enclosure URL of the GUID-less item imported from legacy episode {keeper}"
|
||||
),
|
||||
)?,
|
||||
_ => kept.push(row),
|
||||
}
|
||||
}
|
||||
Ok(kept)
|
||||
}
|
||||
|
||||
fn bounded_title(title: &str) -> String {
|
||||
let title = title.trim();
|
||||
if title.is_empty() {
|
||||
"Untitled episode".to_string()
|
||||
} else {
|
||||
bounded_text(title, MAX_TITLE_CHARS)
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
Reference in new issue
Block a user