Re-key imported GUID-less episodes and make fetch-state writes atomic

The prototype kept one row per enclosure URL for GUID-less items, so stale
rows could change which identity key is unique and give imported episodes
ids a fresh device never derives. The import now collapses rows of one item
seen under several enclosure URLs, and the first refresh re-keys imported
GUID-less episodes without progress or sessions to the ids the current feed
gives.

Fetch-state writes read and write the row under one lock and one immediate
transaction, so concurrent writers no longer lose columns or failure counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
sanapachandClaude Opus 5.5 committed 2026-11-19 17:02:31 +02:00
1 parent e627be51d4
commit 9709f5e2b5
5 files changed
+586 -43

No files matched your search

+10 -1
View File
@@ -31,7 +31,16 @@
- `furumi-library 0.3.0` (breaking): `TrackItem.podcast: - `furumi-library 0.3.0` (breaking): `TrackItem.podcast:
Option<PodcastEpisodeRef>` and `TrackItem::podcast_episode`/`is_podcast`. Option<PodcastEpisodeRef>` and `TrackItem::podcast_episode`/`is_podcast`.
Exhaustive `TrackItem` literals in clients need the new field; music-only Exhaustive `TrackItem` literals in clients need the new field; music-only
code must skip podcast items. The library database is unchanged. code must skip podcast items. Podcast items use a negative per-episode id
(`TrackItem::podcast_track_id`, at most `PODCAST_TRACK_ID_MAX`) and
`release_id`/artist id -1, so existing `id < 0` / `release_id < 0` checks
already treat them as non-library items. The library database is
unchanged.
- `furumi-library 0.3.0`: prototype import collapses rows of one GUID-less
item under several enclosure URLs, and the first refresh re-keys imported
GUID-less episodes without personal state to the ids the current feed
gives, so they match devices that never had the prototype. Fetch-state
writes are atomic across threads and processes.
- See `PODCASTS.md` for the normative contract. - See `PODCASTS.md` for the normative contract.
## v0.4.1 — 2026-09-02 ## v0.4.1 — 2026-09-02
+25 -3
View File
@@ -549,9 +549,23 @@ must keep the same invariants.
* Episode ids come from `resolve_episode_ids` over the stored items. The * Episode ids come from `resolve_episode_ids` over the stored items. The
prototype stored the enclosure URL as `guid` when an item had none: such prototype stored the enclosure URL as `guid` when an item had none: such
rows are imported as GUID-less items and keep the value in rows are imported as GUID-less items and keep the value in
`episodes.legacy_guid`. When a later refresh shows that value as a real `episodes.legacy_guid`. The prototype kept one row per enclosure URL, so
GUID, the episode is re-keyed to its GUID id if it has no progress and no rows with the same publication second, normalized title and `<link>`
session (otherwise the GUID item becomes a new episode). (or none) are one item whose audio URL moved: the newest row
(`first_seen_at`, then `id`) is imported, the others are quarantined as
duplicates.
* Imported GUID-less episodes are **provisional**: stale rows can still
change which key is unique among the stored items, so their ids may
differ from what a device without the prototype derives. On a refresh,
item ids are resolved against the stored episodes except provisional
ones without progress or sessions; each such provisional episode is
matched to its item (enclosure, then link, then date and title, GUIDs
ignored) and re-keyed to that item's id unless another episode holds
it. The same applies when `legacy_guid` turns out to be the item's real
GUID. Listing an episode clears `legacy_guid`, which makes its id final.
Provisional episodes carry no personal state, so nothing personal moves;
an imported episode played before its first refresh keeps its id (the
feed item then becomes a separate episode if the ids differ).
* ETag, Last-Modified, check time, last error and the cover cache move to the * ETag, Last-Modified, check time, last error and the cover cache move to the
fetch state. The prototype stores no progress or history, so none is fetch state. The prototype stores no progress or history, so none is
created. created.
@@ -605,6 +619,14 @@ must keep the same invariants.
(descriptions, episodes) are never exported. (descriptions, episodes) are never exported.
* Feed URLs resolve through `resolve_podcast_id` over the alias set, the * Feed URLs resolve through `resolve_podcast_id` over the alias set, the
catalog fetch URL and the register's `podcast.feed_url`. catalog fetch URL and the register's `podcast.feed_url`.
* Fetch-state writes (`record_fetch_*`, `set_cover_cache`) read and write
the row in one immediate transaction, so concurrent writers on one or
several connections never lose a column or a failure increment.
* `TrackItem::podcast_episode` builds queue items with a negative,
per-episode `id` (`TrackItem::podcast_track_id`, at most
`PODCAST_TRACK_ID_MAX`, below the ids of pending federated tracks) and
`release_id` / artist id -1, so code written for library tracks
(`id >= 0`) skips them even before it checks `is_podcast()`.
## Versioning ## Versioning
+156 -33
View File
@@ -5,7 +5,10 @@ use std::collections::BTreeSet;
use anyhow::Result; use anyhow::Result;
use music_dht::device_sync::SyncOpPayload; use music_dht::device_sync::SyncOpPayload;
use music_dht::podcast::feed::ParsedFeed; use music_dht::podcast::feed::ParsedFeed;
use music_dht::podcast::{EpisodeId, EpisodeMeta, FeedUrl, PodcastId, merge_feed_aliases}; use music_dht::podcast::{
EpisodeId, EpisodeIdentityHints, EpisodeMeta, FeedUrl, KnownEpisode, PodcastId,
merge_feed_aliases, resolve_episode_ids,
};
use rusqlite::{Connection, OptionalExtension as _, TransactionBehavior, params}; use rusqlite::{Connection, OptionalExtension as _, TransactionBehavior, params};
use super::apply::{self, EpisodeSource}; use super::apply::{self, EpisodeSource};
@@ -79,6 +82,32 @@ fn podcast_fetch_url(conn: &Connection, podcast_id: &PodcastId) -> Result<Option
.optional()?) .optional()?)
} }
fn fetch_state_in(conn: &Connection, podcast_id: &PodcastId) -> Result<FetchState> {
Ok(conn
.query_row(
"SELECT etag, last_modified, checked_at_ms, last_success_at_ms,
next_check_at_ms, last_error, consecutive_failures, cover_url,
cover_path
FROM podcast_fetch_state WHERE podcast_id = ?1",
[podcast_id.as_str()],
|row| {
Ok(FetchState {
etag: row.get(0)?,
last_modified: row.get(1)?,
checked_at_ms: row.get(2)?,
last_success_at_ms: row.get(3)?,
next_check_at_ms: row.get(4)?,
last_error: row.get(5)?,
consecutive_failures: u32::try_from(row.get::<_, i64>(6)?).unwrap_or(u32::MAX),
cover_url: row.get(7)?,
cover_path: row.get(8)?,
})
},
)
.optional()?
.unwrap_or_default())
}
fn has_personal_state(conn: &Connection, episode_id: &EpisodeId) -> Result<bool> { fn has_personal_state(conn: &Connection, episode_id: &EpisodeId) -> Result<bool> {
Ok(conn.query_row( Ok(conn.query_row(
"SELECT EXISTS (SELECT 1 FROM episode_progress WHERE episode_id = ?1) "SELECT EXISTS (SELECT 1 FROM episode_progress WHERE episode_id = ?1)
@@ -136,6 +165,119 @@ fn rekey_legacy_guids(conn: &Connection, podcast_id: &PodcastId, feed: &ParsedFe
Ok(()) Ok(())
} }
/// Episodes imported from the prototype without a GUID (`legacy_guid` set)
/// that carry no progress and no session.
///
/// Their ids were resolved over every stored prototype row at once, stale
/// rows included, so they may differ from what a fresh device derives from
/// the current feed. They are provisional until a refresh confirms them.
fn provisional_episode_ids(
conn: &Connection,
podcast_id: &PodcastId,
) -> Result<BTreeSet<EpisodeId>> {
let mut statement = conn.prepare(
"SELECT e.episode_id FROM episodes e
WHERE e.podcast_id = ?1 AND e.legacy_guid IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM episode_progress p WHERE p.episode_id = e.episode_id)
AND NOT EXISTS (SELECT 1 FROM podcast_listen_sessions s
WHERE s.episode_id = e.episode_id)",
)?;
let ids = statement
.query_map([podcast_id.as_str()], |row| {
super::episode_id_col(row.get(0)?)
})?
.collect::<rusqlite::Result<BTreeSet<_>>>()?;
Ok(ids)
}
/// Resolves the ids of the feed items as a device without prototype history
/// would, and moves provisional imported episodes onto them.
///
/// * The ids come from the contract identity rule against every stored
/// episode except the provisional ones (episodes from earlier refreshes,
/// from sync, and imported ones that already carry personal state).
/// * Each provisional episode is matched to the item it was imported from
/// with the same rule (enclosure, then link, then date and title; GUIDs
/// ignored, since the prototype may have stored the enclosure URL as GUID)
/// and re-keyed to that item's id unless another stored episode already
/// holds it. Unmatched provisional episodes stay as they are (not in the
/// feed); the next refresh that lists them re-keys them.
///
/// Provisional episodes have no progress and no session, so no personal
/// state moves. The refresh then clears `legacy_guid` on every episode it
/// lists, which makes their ids final.
fn resolve_and_rekey_provisional(
conn: &Connection,
podcast_id: &PodcastId,
feed: &ParsedFeed,
) -> Result<Vec<Option<EpisodeId>>> {
let known = apply::known_episodes(conn, podcast_id)?;
let provisional = provisional_episode_ids(conn, podcast_id)?;
if provisional.is_empty() {
return Ok(feed.resolve_episode_ids(podcast_id, &known));
}
let (provisional_known, stable_known): (Vec<KnownEpisode>, Vec<KnownEpisode>) = known
.into_iter()
.partition(|episode| provisional.contains(&episode.episode_id));
let ids = feed.resolve_episode_ids(podcast_id, &stable_known);
let guidless: Vec<EpisodeIdentityHints> = feed
.identity_hints()
.into_iter()
.map(|hints| EpisodeIdentityHints {
guid: None,
..hints
})
.collect();
let matches = resolve_episode_ids(
podcast_id,
feed.link.as_deref(),
&guidless,
&provisional_known,
);
let mut moves: Vec<(EpisodeId, EpisodeId)> = matches
.into_iter()
.zip(&ids)
.filter_map(|(matched, target)| {
let matched = matched.filter(|id| provisional.contains(id))?;
let target = target.clone()?;
(matched != target).then_some((matched, target))
})
.collect();
// A target held by an episode that does not move away blocks the move.
let occupied: BTreeSet<EpisodeId> = stable_known
.iter()
.chain(&provisional_known)
.map(|episode| episode.episode_id.clone())
.collect();
loop {
let leaving: BTreeSet<&EpisodeId> = moves.iter().map(|(from, _)| from).collect();
let blocked: BTreeSet<EpisodeId> = moves
.iter()
.filter(|(_, to)| occupied.contains(to) && !leaving.contains(to))
.map(|(from, _)| from.clone())
.collect();
if blocked.is_empty() {
break;
}
moves.retain(|(from, _)| !blocked.contains(from));
}
// Two phases, so a move may take an id another move vacates.
for (index, (from, _)) in moves.iter().enumerate() {
conn.execute(
"UPDATE episodes SET episode_id = ?2 WHERE episode_id = ?1",
params![from.as_str(), format!("rekey-pending:{index}")],
)?;
}
for (index, (_, to)) in moves.iter().enumerate() {
conn.execute(
"UPDATE episodes SET episode_id = ?2 WHERE episode_id = ?1",
params![format!("rekey-pending:{index}"), to.as_str()],
)?;
}
Ok(ids)
}
fn upsert_feed_episode(conn: &Connection, meta: &EpisodeMeta, now: i64) -> Result<Option<bool>> { fn upsert_feed_episode(conn: &Connection, meta: &EpisodeMeta, now: i64) -> Result<Option<bool>> {
let Some(stored) = super::queries::episode_meta_in(conn, &meta.episode_id)? else { let Some(stored) = super::queries::episode_meta_in(conn, &meta.episode_id)? else {
apply::insert_episode(conn, meta, EpisodeSource::Feed, true, None, now, now)?; apply::insert_episode(conn, meta, EpisodeSource::Feed, true, None, now, now)?;
@@ -215,8 +357,7 @@ impl PodcastStore {
}; };
rekey_legacy_guids(&tx, podcast_id, feed)?; rekey_legacy_guids(&tx, podcast_id, feed)?;
let known = apply::known_episodes(&tx, podcast_id)?; let ids = resolve_and_rekey_provisional(&tx, podcast_id, feed)?;
let ids = feed.resolve_episode_ids(podcast_id, &known);
tx.execute( tx.execute(
"UPDATE episodes SET in_feed = 0 WHERE podcast_id = ?1", "UPDATE episodes SET in_feed = 0 WHERE podcast_id = ?1",
[podcast_id.as_str()], [podcast_id.as_str()],
@@ -295,45 +436,26 @@ impl PodcastStore {
/// Local fetch state of a podcast (defaults when never fetched). /// Local fetch state of a podcast (defaults when never fetched).
pub fn fetch_state(&self, podcast_id: &PodcastId) -> Result<FetchState> { pub fn fetch_state(&self, podcast_id: &PodcastId) -> Result<FetchState> {
Ok(self fetch_state_in(&self.lock(), podcast_id)
.lock()
.query_row(
"SELECT etag, last_modified, checked_at_ms, last_success_at_ms,
next_check_at_ms, last_error, consecutive_failures, cover_url,
cover_path
FROM podcast_fetch_state WHERE podcast_id = ?1",
[podcast_id.as_str()],
|row| {
Ok(FetchState {
etag: row.get(0)?,
last_modified: row.get(1)?,
checked_at_ms: row.get(2)?,
last_success_at_ms: row.get(3)?,
next_check_at_ms: row.get(4)?,
last_error: row.get(5)?,
consecutive_failures: u32::try_from(row.get::<_, i64>(6)?)
.unwrap_or(u32::MAX),
cover_url: row.get(7)?,
cover_path: row.get(8)?,
})
},
)
.optional()?
.unwrap_or_default())
} }
/// Read-modify-write of the fetch state under one lock and one immediate
/// transaction, so concurrent `record_*` / `set_cover_cache` calls (from
/// this or another process) never lose each other's columns or failure
/// increments.
fn update_fetch_state( fn update_fetch_state(
&self, &self,
podcast_id: &PodcastId, podcast_id: &PodcastId,
update: impl FnOnce(&mut FetchState), update: impl FnOnce(&mut FetchState),
) -> Result<FetchState> { ) -> Result<FetchState> {
let mut state = self.fetch_state(podcast_id)?; let mut conn = self.lock();
update(&mut state); let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?;
let conn = self.lock(); if podcast_fetch_url(&tx, podcast_id)?.is_none() {
if podcast_fetch_url(&conn, podcast_id)?.is_none() {
return Err(PodcastStoreError::UnknownPodcast(podcast_id.clone()).into()); return Err(PodcastStoreError::UnknownPodcast(podcast_id.clone()).into());
} }
conn.execute( let mut state = fetch_state_in(&tx, podcast_id)?;
update(&mut state);
tx.execute(
"INSERT INTO podcast_fetch_state "INSERT INTO podcast_fetch_state
(podcast_id, etag, last_modified, checked_at_ms, last_success_at_ms, (podcast_id, etag, last_modified, checked_at_ms, last_success_at_ms,
next_check_at_ms, last_error, consecutive_failures, cover_url, cover_path) next_check_at_ms, last_error, consecutive_failures, cover_url, cover_path)
@@ -359,6 +481,7 @@ impl PodcastStore {
state.cover_path, state.cover_path,
], ],
)?; )?;
tx.commit()?;
Ok(state) Ok(state)
} }
+73 -6
View File
@@ -6,7 +6,8 @@ use anyhow::{Context as _, Result};
use music_dht::podcast::{ use music_dht::podcast::{
EpisodeId, EpisodeIdentityHints, EpisodeMeta, EpisodeType, FeedUrl, KnownEpisode, EpisodeId, EpisodeIdentityHints, EpisodeMeta, EpisodeType, FeedUrl, KnownEpisode,
MAX_DESCRIPTION_CHARS, MAX_TITLE_CHARS, MAX_URL_BYTES, PodcastId, PodcastMeta, MAX_DESCRIPTION_CHARS, MAX_TITLE_CHARS, MAX_URL_BYTES, PodcastId, PodcastMeta,
SubscriptionRecord, bounded_text, is_http_url, resolve_episode_ids, resolve_podcast_id, SubscriptionRecord, bounded_text, is_http_url, normalize_episode_title, resolve_episode_ids,
resolve_podcast_id,
}; };
use rusqlite::{Connection, OptionalExtension as _, Transaction, TransactionBehavior, params}; use rusqlite::{Connection, OptionalExtension as _, Transaction, TransactionBehavior, params};
@@ -442,11 +443,16 @@ fn secs_to_ms(secs: i64) -> i64 {
/// and stay in `legacy_podcasts` with their episodes. Rows with the same /// and stay in `legacy_podcasts` with their episodes. Rows with the same
/// canonical URL merge into one podcast (the first row's subscription and /// canonical URL merge into one podcast (the first row's subscription and
/// fetch state win; the episodes of all of them are imported). /// fetch state win; the episodes of all of them are imported).
/// * Episodes get ids from the contract identity rule over the whole stored /// * Episodes get ids from the contract identity rule over the stored rows.
/// document. The prototype stored the enclosure URL as `guid` when the /// The prototype stored the enclosure URL as `guid` when the item had
/// item had none: such rows are imported as GUID-less items and keep the /// none: such rows are imported as GUID-less items and keep the stored
/// stored value in `legacy_guid`, which a later refresh uses to re-key the /// value in `legacy_guid`. Rows of one GUID-less item seen under several
/// episode to its GUID id when the feed does have that GUID. /// enclosure URLs are collapsed into the newest (the others are
/// quarantined as duplicates). Because stale rows can still change which
/// key is unique, these ids are provisional: the first refresh that lists
/// the item re-keys an episode without progress or sessions to the id the
/// current feed gives (see `refresh.rs`), including its GUID id when the
/// stored value turns out to be a real GUID.
/// * ETag, Last-Modified, check time, last error and the cover cache move to /// * ETag, Last-Modified, check time, last error and the cover cache move to
/// the local fetch state. The prototype has no progress or history, so no /// the local fetch state. The prototype has no progress or history, so no
/// progress register or session is created. /// progress register or session is created.
@@ -623,6 +629,7 @@ fn import_prototype_episodes(
let guid = row.guid.trim(); let guid = row.guid.trim();
(!guid.is_empty() && guid != row.audio_url.trim()).then(|| guid.to_string()) (!guid.is_empty() && guid != row.audio_url.trim()).then(|| guid.to_string())
}; };
let usable = collapse_moved_guidless_rows(tx, usable, real_guid)?;
let hints: Vec<EpisodeIdentityHints> = usable let hints: Vec<EpisodeIdentityHints> = usable
.iter() .iter()
.map(|row| EpisodeIdentityHints { .map(|row| EpisodeIdentityHints {
@@ -719,6 +726,66 @@ fn import_prototype_episodes(
Ok(()) Ok(())
} }
/// Collapses prototype rows that are one GUID-less item seen under several
/// enclosure URLs.
///
/// The prototype keyed GUID-less items by their enclosure URL, so an item
/// whose audio URL changed (new CDN, tracking prefix) left one row per URL.
/// Those stale rows are not part of the current feed; resolved together with
/// it they would make the item's `<link>` and `(date, title)` look
/// non-unique and give it an id a fresh device never derives. Rows with the
/// same publication second, normalized title and `<link>` (or none) are the
/// same item: the newest (`first_seen_at`, then `id`) is imported, the others
/// are quarantined as duplicates. The link must match too, so distinct
/// episodes that only share a title and second are never merged.
fn collapse_moved_guidless_rows(
tx: &Transaction<'_>,
rows: Vec<LegacyEpisode>,
real_guid: impl Fn(&LegacyEpisode) -> Option<String>,
) -> Result<Vec<LegacyEpisode>> {
type ItemKey = (i64, String, Option<String>);
let key = |row: &LegacyEpisode| -> Option<ItemKey> {
if real_guid(row).is_some() {
return None;
}
let title = normalize_episode_title(&row.title);
let link = row
.link
.as_deref()
.map(str::trim)
.filter(|link| !link.is_empty())
.map(str::to_string);
(!title.is_empty()).then_some((row.published_at?, title, link))
};
let mut newest: BTreeMap<ItemKey, (i64, i64)> = BTreeMap::new();
for row in &rows {
if let Some(key) = key(row) {
let candidate = (row.first_seen_at, row.id);
newest
.entry(key)
.and_modify(|best| *best = (*best).max(candidate))
.or_insert(candidate);
}
}
let mut kept = Vec::with_capacity(rows.len());
for row in rows {
let keeper = key(&row).and_then(|key| newest.get(&key).map(|(_, id)| *id));
match keeper {
Some(keeper) if keeper != row.id => quarantine(
tx,
"legacy_podcast_episodes",
row.id,
"duplicate episode",
&format!(
"older enclosure URL of the GUID-less item imported from legacy episode {keeper}"
),
)?,
_ => kept.push(row),
}
}
Ok(kept)
}
fn bounded_title(title: &str) -> String { fn bounded_title(title: &str) -> String {
let title = title.trim(); let title = title.trim();
if title.is_empty() { if title.is_empty() {
+322
View File
@@ -1758,3 +1758,325 @@ fn concurrent_openers_migrate_once() {
); );
} }
} }
// ---------------------------------------------------------------------------
// Review regressions
/// A prototype `podcast_episodes` row: guid, title, published_at (seconds),
/// audio URL, link.
type PrototypeRow<'a> = (&'a str, &'a str, Option<i64>, &'a str, Option<&'a str>);
/// A prototype database with one podcast at [`FEED`] and the given rows.
fn prototype_with_rows(path: &Path, rows: &[PrototypeRow<'_>]) -> Vec<i64> {
let conn = fabricate_prototype(path);
conn.execute(
"INSERT INTO podcasts (feed_url, title, link, added_at) VALUES (?1, 'Show', ?2, 1700000000)",
params![FEED, "https://example.test/"],
)
.unwrap();
let show = conn.last_insert_rowid();
rows.iter()
.map(|(guid, title, published, audio, link)| {
prototype_episode(&conn, show, guid, title, *published, audio, *link)
})
.collect()
}
fn in_feed_ids(store: &PodcastStore, podcast_id: &PodcastId) -> Vec<(EpisodeId, bool)> {
store
.episodes(&EpisodeQuery::new(podcast_id.clone()))
.unwrap()
.into_iter()
.map(|item| (item.episode.episode_id, item.in_feed))
.collect()
}
/// 2023-11-14T22:13:20Z.
const PILOT_DATE: &str = "Tue, 14 Nov 2023 22:13:20 GMT";
const PILOT_SECS: i64 = 1_700_000_000;
const OLD_AUDIO: &str = "https://cdn.example.test/old/ep1.mp3";
const NEW_AUDIO: &str = "https://cdn.example.test/new/ep1.mp3";
const PILOT_LINK: &str = "https://example.test/ep1";
fn current_pilot_feed() -> ParsedFeed {
parse(
&rss(
"Show",
None,
&[Item {
link: Some(PILOT_LINK),
date: Some(PILOT_DATE),
..Item::new(None, "Pilot", NEW_AUDIO)
}],
),
&url(FEED),
)
}
#[test]
fn prototype_rows_of_a_moved_guidless_item_import_as_the_fresh_id() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join(PODCAST_DB_FILE);
// The prototype keyed GUID-less items by enclosure URL: after the CDN
// move the same item has two rows.
let rows = prototype_with_rows(
&path,
&[
(
OLD_AUDIO,
"Pilot",
Some(PILOT_SECS),
OLD_AUDIO,
Some(PILOT_LINK),
),
(
NEW_AUDIO,
"Pilot",
Some(PILOT_SECS),
NEW_AUDIO,
Some(PILOT_LINK),
),
],
);
let store_a = PodcastStore::open(&path).unwrap();
let podcast_id = PodcastId::from_feed_url(&url(FEED));
let feed = current_pilot_feed();
let fresh = feed.resolve_episode_ids(&podcast_id, &[])[0]
.clone()
.unwrap();
// One episode, with the id a fresh device derives; the stale row is a
// quarantined duplicate that stays in the legacy table.
assert_eq!(
in_feed_ids(&store_a, &podcast_id),
vec![(fresh.clone(), true)]
);
let quarantine = store_a.legacy_quarantine().unwrap();
assert_eq!(quarantine.len(), 1);
assert_eq!(quarantine[0].legacy_id, rows[0]);
assert_eq!(quarantine[0].reason, "duplicate episode");
assert_eq!(
store_a.episode(&fresh).unwrap().unwrap().episode.audio_url,
NEW_AUDIO
);
// Device A refreshes and plays; fresh device B subscribed, refreshed
// and played too. A's operation lands on B's own episode.
let mut stamps_a = TestStamps::new("device-a", 1_800_000_000_000);
refresh(
&store_a,
&mut stamps_a,
&podcast_id,
&feed,
&url(FEED),
false,
);
assert_eq!(
in_feed_ids(&store_a, &podcast_id),
vec![(fresh.clone(), true)]
);
let store_b = PodcastStore::open_in_memory().unwrap();
let mut stamps_b = TestStamps::new("device-b", 1_900_000_000_000);
store_b
.subscribe(&mut stamps_b, &meta(FEED, "Show"))
.unwrap();
refresh(
&store_b,
&mut stamps_b,
&podcast_id,
&feed,
&url(FEED),
false,
);
let reference_b = store_b.episode_ref(&fresh).unwrap().unwrap();
store_b
.start_session(&mut stamps_b, &reference_b, "b-1", 600_000, None)
.unwrap();
let reference_a = store_a.episode_ref(&fresh).unwrap().unwrap();
let write = store_a
.start_session(&mut stamps_a, &reference_a, "a-1", 5_000, None)
.unwrap();
store_b.apply_remote_op(&write.op.into_wire(1)).unwrap();
refresh(
&store_b,
&mut stamps_b,
&podcast_id,
&feed,
&url(FEED),
false,
);
assert_eq!(
in_feed_ids(&store_b, &podcast_id),
vec![(fresh.clone(), true)]
);
assert!(store_b.progress(&fresh).unwrap().is_some());
}
#[test]
fn first_refresh_rekeys_provisional_imported_guidless_episodes() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join(PODCAST_DB_FILE);
// The stale row was also retitled, so the import cannot tell it is the
// same item: the shared link is not identifying among the stored rows
// and both rows get (date, title) ids.
prototype_with_rows(
&path,
&[
(
OLD_AUDIO,
"Pilot (draft)",
Some(PILOT_SECS),
OLD_AUDIO,
Some(PILOT_LINK),
),
(
NEW_AUDIO,
"Pilot",
Some(PILOT_SECS),
NEW_AUDIO,
Some(PILOT_LINK),
),
],
);
let store = PodcastStore::open(&path).unwrap();
let podcast_id = PodcastId::from_feed_url(&url(FEED));
let feed = current_pilot_feed();
let fresh = feed.resolve_episode_ids(&podcast_id, &[])[0]
.clone()
.unwrap();
let imported = in_feed_ids(&store, &podcast_id);
assert_eq!(imported.len(), 2);
assert!(
imported.iter().all(|(id, _)| *id != fresh),
"the import alone cannot know the current feed"
);
let mut stamps = TestStamps::new("device-a", 1_800_000_000_000);
let outcome = refresh(&store, &mut stamps, &podcast_id, &feed, &url(FEED), false);
assert_eq!(
outcome.new_episodes, 0,
"the imported row moved, not duplicated"
);
assert_eq!(outcome.updated_episodes + outcome.unchanged_episodes, 1);
let after = in_feed_ids(&store, &podcast_id);
assert_eq!(after.len(), 2);
assert!(after.contains(&(fresh.clone(), true)));
assert_eq!(after.iter().filter(|(_, in_feed)| *in_feed).count(), 1);
assert_eq!(
store.episode(&fresh).unwrap().unwrap().episode.audio_url,
NEW_AUDIO
);
let legacy_guid: Option<String> = store
.lock()
.query_row(
"SELECT legacy_guid FROM episodes WHERE episode_id = ?1",
[fresh.as_str()],
|row| row.get(0),
)
.unwrap();
assert_eq!(legacy_guid, None, "the refresh confirmed the id");
// Ids are final now: later refreshes keep them.
refresh(&store, &mut stamps, &podcast_id, &feed, &url(FEED), false);
assert_eq!(in_feed_ids(&store, &podcast_id), after);
}
#[test]
fn imported_episodes_with_personal_state_are_not_rekeyed() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join(PODCAST_DB_FILE);
prototype_with_rows(
&path,
&[
(
OLD_AUDIO,
"Pilot (draft)",
Some(PILOT_SECS),
OLD_AUDIO,
Some(PILOT_LINK),
),
(
NEW_AUDIO,
"Pilot",
Some(PILOT_SECS),
NEW_AUDIO,
Some(PILOT_LINK),
),
],
);
let store = PodcastStore::open(&path).unwrap();
let podcast_id = PodcastId::from_feed_url(&url(FEED));
let played = store
.episodes(&EpisodeQuery::new(podcast_id.clone()))
.unwrap()
.into_iter()
.find(|item| item.episode.audio_url == NEW_AUDIO)
.unwrap()
.episode
.episode_id;
let mut stamps = TestStamps::new("device-a", 1_800_000_000_000);
let reference = store.episode_ref(&played).unwrap().unwrap();
store
.start_session(&mut stamps, &reference, "s-1", 42_000, None)
.unwrap();
refresh(
&store,
&mut stamps,
&podcast_id,
&current_pilot_feed(),
&url(FEED),
false,
);
// The played episode keeps its id and its position and is the feed item.
assert!(in_feed_ids(&store, &podcast_id).contains(&(played.clone(), true)));
assert_eq!(store.resume_position_ms(&played).unwrap(), 42_000);
}
#[test]
fn concurrent_fetch_state_writers_lose_no_update() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join(PODCAST_DB_FILE);
let first = std::sync::Arc::new(PodcastStore::open(&path).unwrap());
let second = std::sync::Arc::new(PodcastStore::open(&path).unwrap());
let mut stamps = TestStamps::new("device-a", 1_000);
let podcast_id = first
.subscribe(&mut stamps, &meta(FEED, "Show"))
.unwrap()
.podcast_id;
const THREADS: usize = 4;
const FAILURES: usize = 25;
let workers: Vec<_> = (0..THREADS)
.map(|thread| {
// Two connections (as two processes would have) and two threads
// per connection.
let store = if thread % 2 == 0 {
first.clone()
} else {
second.clone()
};
let podcast_id = podcast_id.clone();
std::thread::spawn(move || {
for attempt in 0..FAILURES {
store
.record_fetch_failure(&podcast_id, attempt as i64, "timeout", None)
.unwrap();
if thread == 0 && attempt == FAILURES / 2 {
store
.set_cover_cache(
&podcast_id,
Some("https://example.test/c.jpg"),
Some("/covers/c.jpg"),
)
.unwrap();
}
}
})
})
.collect();
for worker in workers {
worker.join().unwrap();
}
let state = second.fetch_state(&podcast_id).unwrap();
assert_eq!(state.consecutive_failures as usize, THREADS * FAILURES);
assert_eq!(state.cover_path.as_deref(), Some("/covers/c.jpg"));
}