Remove imported podcast episodes that duplicate a held id

A provisional imported GUID-less episode whose feed item id is already
held by an episode that stays (typically one a remote operation created
before this device's first refresh) is now deleted instead of kept as a
second, never re-keyed copy. The same applies when its legacy GUID turns
out to be the item's real GUID. Such episodes carry no personal state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
sanapachandClaude Opus 5.5 committed 2026-11-19 18:29:19 +02:00
1 parent 06ac3691d8
commit b8d8b532f2
4 files changed
+218 -19

No files matched your search

+4 -1
View File
@@ -41,7 +41,10 @@
- `furumi-library 0.3.0`: prototype import collapses rows of one GUID-less
item under several enclosure URLs, and the first refresh re-keys imported
GUID-less episodes without personal state to the ids the current feed
gives, so they match devices that never had the prototype. Fetch-state
gives, so they match devices that never had the prototype; when another
episode (for example one created by a remote operation) already holds
that id, the imported copy is deleted instead of kept as a duplicate.
Fetch-state
writes are atomic across threads and processes.
- See `PODCASTS.md` for the normative contract.
+10 -6
View File
@@ -560,12 +560,16 @@ must keep the same invariants.
item ids are resolved against the stored episodes except provisional
ones without progress or sessions; each such provisional episode is
matched to its item (enclosure, then link, then date and title, GUIDs
ignored) and re-keyed to that item's id unless another episode holds
it. The same applies when `legacy_guid` turns out to be the item's real
GUID. Listing an episode clears `legacy_guid`, which makes its id final.
Provisional episodes carry no personal state, so nothing personal moves;
an imported episode played before its first refresh keeps its id (the
feed item then becomes a separate episode if the ids differ).
ignored) and re-keyed to that item's id. When another episode that
stays already holds that id (for example one a remote operation created
before this device's first refresh), it is the item's episode and the
provisional one is a duplicate: it is deleted rather than kept as a
second copy. The same applies when `legacy_guid` turns out to be the
item's real GUID. Listing an episode clears `legacy_guid`, which makes
its id final. Provisional episodes carry no personal state, so nothing
personal moves or is lost; an imported episode played before its first
refresh is not provisional and keeps its id (the feed item then becomes
a separate episode if the ids differ).
* ETag, Last-Modified, check time, last error and the cover cache move to the
fetch state. The prototype stores no progress or history, so none is
created.
+40 -10
View File
@@ -121,7 +121,9 @@ fn has_personal_state(conn: &Connection, episode_id: &EpisodeId) -> Result<bool>
/// enclosure URL (`legacy_guid`) when the feed shows it is a real GUID.
///
/// Only episodes without progress or sessions move, so no personal state is
/// touched; otherwise the GUID item becomes a separate episode.
/// touched; otherwise the GUID item becomes a separate episode. When another
/// episode already holds the GUID id (for example one a remote operation
/// created), the imported episode is a duplicate of it and is deleted.
fn rekey_legacy_guids(conn: &Connection, podcast_id: &PodcastId, feed: &ParsedFeed) -> Result<()> {
for item in &feed.episodes {
let Some(guid) = item.guid.as_deref() else {
@@ -138,9 +140,6 @@ fn rekey_legacy_guids(conn: &Connection, podcast_id: &PodcastId, feed: &ParsedFe
)
.optional()?
.is_some();
if taken {
continue;
}
let legacy: Option<String> = conn
.query_row(
"SELECT episode_id FROM episodes
@@ -153,7 +152,14 @@ fn rekey_legacy_guids(conn: &Connection, podcast_id: &PodcastId, feed: &ParsedFe
continue;
};
let legacy = super::episode_id_col(legacy)?;
if has_personal_state(conn, &legacy)? {
if legacy == guid_id || has_personal_state(conn, &legacy)? {
continue;
}
if taken {
conn.execute(
"DELETE FROM episodes WHERE episode_id = ?1",
[legacy.as_str()],
)?;
continue;
}
conn.execute(
@@ -199,9 +205,11 @@ fn provisional_episode_ids(
/// * Each provisional episode is matched to the item it was imported from
/// with the same rule (enclosure, then link, then date and title; GUIDs
/// ignored, since the prototype may have stored the enclosure URL as GUID)
/// and re-keyed to that item's id unless another stored episode already
/// holds it. Unmatched provisional episodes stay as they are (not in the
/// feed); the next refresh that lists them re-keys them.
/// and re-keyed to that item's id. When another stored episode that stays
/// already holds that id, it is the item's episode and the provisional one
/// is a duplicate: it is deleted. Unmatched provisional episodes stay as
/// they are (not in the feed); the next refresh that lists them re-keys
/// them.
///
/// Provisional episodes have no progress and no session, so no personal
/// state moves. The refresh then clears `legacy_guid` on every episode it
@@ -244,12 +252,18 @@ fn resolve_and_rekey_provisional(
(matched != target).then_some((matched, target))
})
.collect();
// A target held by an episode that does not move away blocks the move.
let occupied: BTreeSet<EpisodeId> = stable_known
// A target held by an episode that stays (for example a row a remote
// operation created before this first refresh) is the item's episode
// already: the provisional episode is a duplicate of it. It has no
// progress and no session, and its metadata would be overwritten by the
// item anyway, so it is removed instead of kept as a second copy. A
// removed duplicate frees its own id for other moves.
let mut occupied: BTreeSet<EpisodeId> = stable_known
.iter()
.chain(&provisional_known)
.map(|episode| episode.episode_id.clone())
.collect();
let mut duplicates: Vec<EpisodeId> = Vec::new();
loop {
let leaving: BTreeSet<&EpisodeId> = moves.iter().map(|(from, _)| from).collect();
let blocked: BTreeSet<EpisodeId> = moves
@@ -261,6 +275,22 @@ fn resolve_and_rekey_provisional(
break;
}
moves.retain(|(from, _)| !blocked.contains(from));
for from in blocked {
occupied.remove(&from);
duplicates.push(from);
}
}
for duplicate in &duplicates {
// Guarded again: only a provisional episode without personal state.
conn.execute(
"DELETE FROM episodes
WHERE episode_id = ?1 AND legacy_guid IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM episode_progress p
WHERE p.episode_id = episodes.episode_id)
AND NOT EXISTS (SELECT 1 FROM podcast_listen_sessions s
WHERE s.episode_id = episodes.episode_id)",
[duplicate.as_str()],
)?;
}
// Two phases, so a move may take an id another move vacates.
for (index, (from, _)) in moves.iter().enumerate() {
+164 -2
View File
@@ -1976,9 +1976,17 @@ fn first_refresh_rekeys_provisional_imported_guidless_episodes() {
.unwrap();
assert_eq!(legacy_guid, None, "the refresh confirmed the id");
// Ids are final now: later refreshes keep them.
// Ids are final now: later refreshes keep them. The retitled stale row
// is the only provisional episode left; its link now identifies the
// item, whose id the confirmed episode holds, so it is removed as a
// duplicate.
refresh(&store, &mut stamps, &podcast_id, &feed, &url(FEED), false);
assert_eq!(in_feed_ids(&store, &podcast_id), after);
assert_eq!(
in_feed_ids(&store, &podcast_id),
vec![(fresh.clone(), true)]
);
refresh(&store, &mut stamps, &podcast_id, &feed, &url(FEED), false);
assert_eq!(in_feed_ids(&store, &podcast_id), vec![(fresh, true)]);
}
#[test]
@@ -2032,6 +2040,160 @@ fn imported_episodes_with_personal_state_are_not_rekeyed() {
assert_eq!(store.resume_position_ms(&played).unwrap(), 42_000);
}
#[test]
fn remote_progress_before_the_first_refresh_leaves_no_duplicate_episode() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join(PODCAST_DB_FILE);
prototype_with_rows(
&path,
&[
(
OLD_AUDIO,
"Pilot (draft)",
Some(PILOT_SECS),
OLD_AUDIO,
Some(PILOT_LINK),
),
(
NEW_AUDIO,
"Pilot",
Some(PILOT_SECS),
NEW_AUDIO,
Some(PILOT_LINK),
),
],
);
let store_a = PodcastStore::open(&path).unwrap();
let podcast_id = PodcastId::from_feed_url(&url(FEED));
let feed = current_pilot_feed();
let fresh = feed.resolve_episode_ids(&podcast_id, &[])[0]
.clone()
.unwrap();
// A fresh device B plays the episode, and A applies B's operation before
// its own first refresh: A now holds the fresh id as a synced episode.
let store_b = PodcastStore::open_in_memory().unwrap();
let mut stamps_b = TestStamps::new("device-b", 1_900_000_000_000);
store_b
.subscribe(&mut stamps_b, &meta(FEED, "Show"))
.unwrap();
refresh(
&store_b,
&mut stamps_b,
&podcast_id,
&feed,
&url(FEED),
false,
);
let reference = store_b.episode_ref(&fresh).unwrap().unwrap();
let write = store_b
.start_session(&mut stamps_b, &reference, "b-1", 600_000, None)
.unwrap();
assert_eq!(
store_a.apply_remote_op(&write.op.into_wire(1)).unwrap(),
ApplyOutcome::Changed
);
let mut stamps_a = TestStamps::new("device-a", 1_800_000_000_000);
// First refresh: the provisional "Pilot" row matches the item by its
// enclosure, but the synced episode holds the item's id, so the
// provisional row is a duplicate and is removed. Second refresh: the
// retitled stale row is now the only provisional episode, its link
// identifies the same item, and it goes too.
for expected_rows in [2, 1] {
refresh(
&store_a,
&mut stamps_a,
&podcast_id,
&feed,
&url(FEED),
false,
);
let items = store_a
.episodes(&EpisodeQuery::new(podcast_id.clone()))
.unwrap();
let pilots: Vec<_> = items
.iter()
.filter(|item| item.episode.title == "Pilot")
.collect();
assert_eq!(pilots.len(), 1, "one copy of the feed item");
assert_eq!(pilots[0].episode.episode_id, fresh);
assert!(pilots[0].in_feed);
assert_eq!(store_a.resume_position_ms(&fresh).unwrap(), 600_000);
assert_eq!(items.len(), expected_rows);
assert!(
items
.iter()
.filter(|item| item.episode.episode_id != fresh)
.all(|item| !item.in_feed && item.progress.is_none())
);
let summary = store_a.podcast(&podcast_id).unwrap().unwrap();
assert_eq!(summary.episode_count, expected_rows as u64);
assert_eq!(summary.in_progress_count, 1);
}
}
#[test]
fn remote_progress_on_the_guid_id_removes_the_imported_copy() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join(PODCAST_DB_FILE);
// The feed's GUID is the old enclosure URL, so the prototype row looks
// GUID-less; title and enclosure changed since, and there is no link or
// date to match on.
prototype_with_rows(
&path,
&[(OLD_AUDIO, "Pilot (draft)", None, OLD_AUDIO, None)],
);
let store_a = PodcastStore::open(&path).unwrap();
let podcast_id = PodcastId::from_feed_url(&url(FEED));
let feed = parse(
&rss(
"Show",
None,
&[Item::new(Some(OLD_AUDIO), "Pilot", NEW_AUDIO)],
),
&url(FEED),
);
let fresh = feed.resolve_episode_ids(&podcast_id, &[])[0]
.clone()
.unwrap();
assert!(in_feed_ids(&store_a, &podcast_id)[0].0 != fresh);
let store_b = PodcastStore::open_in_memory().unwrap();
let mut stamps_b = TestStamps::new("device-b", 1_900_000_000_000);
store_b
.subscribe(&mut stamps_b, &meta(FEED, "Show"))
.unwrap();
refresh(
&store_b,
&mut stamps_b,
&podcast_id,
&feed,
&url(FEED),
false,
);
let reference = store_b.episode_ref(&fresh).unwrap().unwrap();
let write = store_b
.start_session(&mut stamps_b, &reference, "b-1", 5_000, None)
.unwrap();
store_a.apply_remote_op(&write.op.into_wire(1)).unwrap();
let mut stamps_a = TestStamps::new("device-a", 1_800_000_000_000);
refresh(
&store_a,
&mut stamps_a,
&podcast_id,
&feed,
&url(FEED),
false,
);
assert_eq!(
in_feed_ids(&store_a, &podcast_id),
vec![(fresh.clone(), true)]
);
assert_eq!(store_a.resume_position_ms(&fresh).unwrap(), 5_000);
}
#[test]
fn concurrent_fetch_state_writers_lose_no_update() {
let dir = tempfile::tempdir().unwrap();