Never ingest another feed's document into a followed podcast
Subscribing from a URL that maps onto an existing podcast only through an alias or the feed's itunes:new-feed-url claim no longer writes that document into the podcast: its episodes, metadata and validators stay, and an unsubscribed podcast is subscribed again with its stored metadata. The document is ingested only for a new podcast or when it came from the podcast's own fetch URL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
bbdac75967
commit
c5f3b3cfa4
4 files changed
+204
-24
No files matched your search
@@ -203,7 +203,11 @@ from). If you follow the feed only under the newer address of a card's
|
||||
moved feed, the result list cannot tell yet; opening the card reads the
|
||||
feed, sees the move and shows **Already subscribed** with **Open**, and
|
||||
neither **Subscribe** nor **Add RSS** with the old address creates a
|
||||
second subscription.
|
||||
second subscription. A podcast you follow is changed only by its own
|
||||
address: when a feed found under another address only claims to have
|
||||
moved to yours (or is an old copy of it), nothing of that feed is written
|
||||
into your podcast; if you had unsubscribed, it is subscribed again with
|
||||
its own data and refreshed from its own address.
|
||||
|
||||
- Opening a card shows its title, author, description, artwork and the feed
|
||||
URL, then loads the seasons and episodes from the feed itself through the
|
||||
|
||||
+9
-4
@@ -304,10 +304,15 @@ Search and subscription from a card (`src/podcasts/search.rs`):
|
||||
newest 50 episodes of the season, without storing anything; subscribe
|
||||
fetches the origin URL, then `runner::subscribe_fetched` writes a
|
||||
private subscription with the card's id, or the account's existing
|
||||
podcast for the first known URL of `runner::document_urls` (the origin,
|
||||
the end of a permanent redirect chain, the feed's `itunes:new-feed-url`;
|
||||
also used by "add RSS"). Nothing is written when the account is already
|
||||
subscribed, by id or known URL, before or after the fetch. Both share
|
||||
podcast for a known URL of `runner::document_urls` (the origin, the end
|
||||
of a permanent redirect chain, the feed's `itunes:new-feed-url`;
|
||||
`search::known_podcast`; also used by "add RSS"). The document is
|
||||
ingested only into a new podcast or when it came from the existing
|
||||
podcast's own fetch URL (requested there, or a permanent redirect ended
|
||||
there); otherwise the podcast is subscribed again with its stored
|
||||
metadata and nothing of the document is written. Nothing is written when
|
||||
the account is already subscribed, by id or known URL, before or after
|
||||
the fetch. Both share
|
||||
the "add RSS" request limits; the browser aborts a preview it no longer
|
||||
shows (`AbortController`), so a closed card frees its slot once the
|
||||
server sees the connection close.
|
||||
|
||||
+46
-19
@@ -480,7 +480,13 @@ pub(crate) async fn fetch_to_subscribe(
|
||||
/// The URLs a document fetched from `requested` shows for the same
|
||||
/// podcast: `requested`, the end of a chain of permanent redirects and the
|
||||
/// feed's own `itunes:new-feed-url` (the same URLs a refresh learns as
|
||||
/// aliases), without duplicates.
|
||||
/// aliases and moves to), without duplicates.
|
||||
///
|
||||
/// Only the first two are where the document came from. The
|
||||
/// `itunes:new-feed-url` is a claim of the document: following the feed
|
||||
/// moves to it anyway (its first refresh fetches from there), so it may
|
||||
/// name the account's podcast for the feed, but the document is never that
|
||||
/// podcast's content ([`subscribe_fetched`] does not ingest it there).
|
||||
pub(crate) fn document_urls(requested: &FeedUrl, document: &FetchedFeed) -> Vec<FeedUrl> {
|
||||
let mut urls = vec![requested.clone()];
|
||||
if document.moved_permanently {
|
||||
@@ -499,16 +505,32 @@ pub(crate) fn document_urls(requested: &FeedUrl, document: &FetchedFeed) -> Vec<
|
||||
urls
|
||||
}
|
||||
|
||||
/// Writes the subscription for a document fetched from `url` and ingests
|
||||
/// it like a scheduled refresh (which also stores the validators and the
|
||||
/// next check).
|
||||
/// Whether a document fetched from `requested` is the content of a podcast
|
||||
/// fetched from `feed_url`: the request went to `feed_url` itself (what a
|
||||
/// refresh does), or a permanent redirect chain ended there.
|
||||
fn fetched_from_feed(requested: &FeedUrl, document: &FetchedFeed, feed_url: &FeedUrl) -> bool {
|
||||
requested == feed_url || (document.moved_permanently && document.fetched_from == *feed_url)
|
||||
}
|
||||
|
||||
/// Writes the subscription for a document fetched from `url`.
|
||||
///
|
||||
/// * The podcast is the account's podcast for the feed
|
||||
/// ([`super::search::known_podcast`] over [`document_urls`]: a subscribed
|
||||
/// one first), so a feed the account already follows under another
|
||||
/// address is not subscribed a second time; otherwise a new podcast with
|
||||
/// the id of `url`.
|
||||
/// * The document is ingested like a scheduled refresh (which also stores
|
||||
/// the validators and the next check) only for a new podcast, or when it
|
||||
/// came from the existing podcast's own fetch URL
|
||||
/// ([`fetched_from_feed`]). Otherwise (the match came through another
|
||||
/// alias or the feed's `itunes:new-feed-url` claim) it may be a stale
|
||||
/// copy or another feed altogether: the existing podcast is subscribed
|
||||
/// again with its stored channel metadata, like "subscribe again", and
|
||||
/// the refresh job fetches its own URL. An active subscription is never
|
||||
/// written.
|
||||
///
|
||||
/// The podcast is the account's podcast for the first of
|
||||
/// [`document_urls`] it knows (its id, a known feed URL or alias), so a
|
||||
/// feed the account already follows under its newer address is not
|
||||
/// subscribed a second time; otherwise a new podcast with the id of `url`.
|
||||
/// [`PodcastStore::subscribe`] reuses the existing podcast with its
|
||||
/// progress and history, and writes nothing for an active subscription.
|
||||
/// progress and history.
|
||||
pub(crate) async fn subscribe_fetched(
|
||||
store: &PodcastStore,
|
||||
user_id: i64,
|
||||
@@ -516,17 +538,22 @@ pub(crate) async fn subscribe_fetched(
|
||||
document: &FetchedFeed,
|
||||
now_ms: i64,
|
||||
) -> Result<UrlSubscription> {
|
||||
let mut subscribe_as = url.clone();
|
||||
for candidate in document_urls(url, document) {
|
||||
let podcast_id = store.resolve_podcast_id(user_id, &candidate).await?;
|
||||
if store.podcast(user_id, &podcast_id).await?.is_some() {
|
||||
subscribe_as = candidate;
|
||||
break;
|
||||
let requested_id = store.resolve_podcast_id(user_id, url).await?;
|
||||
let urls = document_urls(url, document);
|
||||
let existing = super::search::known_podcast(store, user_id, &requested_id, &urls).await?;
|
||||
let meta = match &existing {
|
||||
Some(known) if !fetched_from_feed(url, document, &known.meta.feed_url) => {
|
||||
let subscription = store.subscribe(user_id, &known.meta).await?;
|
||||
return Ok(UrlSubscription {
|
||||
podcast_id: subscription.podcast_id,
|
||||
already_subscribed: subscription.already_subscribed,
|
||||
refresh: RefreshOutcome::default(),
|
||||
});
|
||||
}
|
||||
}
|
||||
let subscription = store
|
||||
.subscribe(user_id, &document.feed.to_podcast_meta(subscribe_as))
|
||||
.await?;
|
||||
Some(known) => document.feed.to_podcast_meta(known.meta.feed_url.clone()),
|
||||
None => document.feed.to_podcast_meta(url.clone()),
|
||||
};
|
||||
let subscription = store.subscribe(user_id, &meta).await?;
|
||||
let target = RefreshTarget {
|
||||
user_id,
|
||||
podcast_id: subscription.podcast_id.clone(),
|
||||
|
||||
@@ -580,3 +580,147 @@ async fn postgres_subscribing_from_a_card_keeps_the_card_id() {
|
||||
|
||||
cleanup(&pool, [user, other, third]).await;
|
||||
}
|
||||
|
||||
/// The catalog row and the episodes of one podcast of the account:
|
||||
/// `(title, description, meta_from_feed, [(episode title, in_feed)])`.
|
||||
async fn podcast_snapshot(
|
||||
pool: &sqlx::PgPool,
|
||||
user: i64,
|
||||
podcast_id: &PodcastId,
|
||||
) -> (String, Option<String>, bool, Vec<(String, bool)>) {
|
||||
let (title, description, meta_from_feed): (String, Option<String>, bool) = sqlx::query_as(
|
||||
"SELECT title, description, meta_from_feed FROM furumusic__podcast
|
||||
WHERE user_id = $1 AND podcast_id = $2",
|
||||
)
|
||||
.bind(user)
|
||||
.bind(podcast_id.as_str())
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let episodes: Vec<(String, bool)> = sqlx::query_as(
|
||||
"SELECT title, in_feed FROM furumusic__podcast_episode
|
||||
WHERE user_id = $1 ORDER BY title",
|
||||
)
|
||||
.bind(user)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.unwrap();
|
||||
(title, description, meta_from_feed, episodes)
|
||||
}
|
||||
|
||||
async fn active_subscriptions(pool: &sqlx::PgPool, user: i64) -> i64 {
|
||||
sqlx::query_scalar(
|
||||
"SELECT COUNT(*) FROM furumusic__podcast_subscription
|
||||
WHERE user_id = $1 AND subscribed",
|
||||
)
|
||||
.bind(user)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "requires a disposable database in FURU_DEVICE_SYNC_TEST_DATABASE_URL"]
|
||||
async fn postgres_a_feed_claiming_a_followed_address_never_rewrites_that_podcast() {
|
||||
let pool = test_pool().await;
|
||||
let store = PodcastStore::new(pool.clone());
|
||||
let user = random_user();
|
||||
cleanup(&pool, [user]).await;
|
||||
let server = TestServer::start();
|
||||
let real = server.url("/real.xml");
|
||||
server.set(
|
||||
"/real.xml",
|
||||
Route::new(rss("Real Show", ITEMS), "application/rss+xml"),
|
||||
);
|
||||
// The old address serves a frozen copy that names the new address; a
|
||||
// feed of somebody else claims to have moved there too.
|
||||
let claim = |title: &str, guid: &str| {
|
||||
rss(
|
||||
title,
|
||||
&format!(
|
||||
r#"<itunes:new-feed-url>{real}</itunes:new-feed-url>
|
||||
<item><title>{title} item</title><guid>{guid}</guid>
|
||||
<enclosure url="/audio/{guid}.mp3" type="audio/mpeg" length="1000"/></item>"#
|
||||
),
|
||||
)
|
||||
};
|
||||
server.set(
|
||||
"/stub.xml",
|
||||
Route::new(claim("Stale Copy", "stale"), "application/rss+xml"),
|
||||
);
|
||||
server.set(
|
||||
"/evil.xml",
|
||||
Route::new(claim("Injected", "evil"), "application/rss+xml"),
|
||||
);
|
||||
let fetcher = local_fetcher();
|
||||
let now = super::now_ms();
|
||||
let real_id = PodcastId::from_feed_url(&url(&real));
|
||||
let followed = runner::subscribe_url(&store, &fetcher, user, &url(&real), now)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(followed.podcast_id, real_id);
|
||||
let before = podcast_snapshot(&pool, user, &real_id).await;
|
||||
assert_eq!(before.0, "Real Show");
|
||||
assert_eq!(before.3.len(), 3);
|
||||
assert!(before.3.iter().all(|(_, in_feed)| *in_feed));
|
||||
let register_before = store.subscription(user, &real_id).await.unwrap().unwrap();
|
||||
let log = log_len(&pool, user).await;
|
||||
|
||||
// "Add RSS" of the old address and "Subscribe" on a card of the other
|
||||
// feed: the account already follows the feed they name, nothing of
|
||||
// theirs is written into it.
|
||||
for added in [
|
||||
runner::subscribe_url(&store, &fetcher, user, &url(&server.url("/stub.xml")), now)
|
||||
.await
|
||||
.unwrap(),
|
||||
runner::subscribe_url(&store, &fetcher, user, &url(&server.url("/evil.xml")), now)
|
||||
.await
|
||||
.unwrap(),
|
||||
] {
|
||||
assert!(added.already_subscribed);
|
||||
assert_eq!(added.podcast_id, real_id);
|
||||
assert_eq!(added.refresh.new_episodes, 0);
|
||||
}
|
||||
let evil = hit(&server.url("/evil.xml"), "Injected");
|
||||
let body =
|
||||
json!({ "podcast_id": evil.podcast_id.to_string(), "feed_url": evil.feed_url.to_string() });
|
||||
let found = api::subscribe_found(&store, &fetcher, user, body.clone(), now)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(found.already_subscribed);
|
||||
assert_eq!(found.podcast.podcast_id, real_id.to_string());
|
||||
assert_eq!(podcast_snapshot(&pool, user, &real_id).await, before);
|
||||
assert_eq!(log_len(&pool, user).await, log);
|
||||
assert_eq!(active_subscriptions(&pool, user).await, 1);
|
||||
|
||||
// Unsubscribed, the same claim subscribes the account's podcast again
|
||||
// with its own stored metadata; the claiming document is not ingested
|
||||
// (the refresh job fetches the podcast's own address).
|
||||
store.unsubscribe(user, &real_id).await.unwrap();
|
||||
let again = runner::subscribe_url(&store, &fetcher, user, &url(&server.url("/evil.xml")), now)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(!again.already_subscribed);
|
||||
assert_eq!(again.podcast_id, real_id);
|
||||
assert_eq!(again.refresh.new_episodes, 0);
|
||||
assert_eq!(podcast_snapshot(&pool, user, &real_id).await, before);
|
||||
let register = store.subscription(user, &real_id).await.unwrap().unwrap();
|
||||
assert!(register.subscribed);
|
||||
assert_eq!(register.podcast, register_before.podcast);
|
||||
assert!(
|
||||
store
|
||||
.podcast(user, &evil.podcast_id)
|
||||
.await
|
||||
.unwrap()
|
||||
.is_none()
|
||||
);
|
||||
|
||||
// Its own address is still ingested as before (a refresh).
|
||||
let own = runner::subscribe_url(&store, &fetcher, user, &url(&real), now)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(own.already_subscribed);
|
||||
assert_eq!(own.refresh.unchanged_episodes, 3);
|
||||
|
||||
cleanup(&pool, [user]).await;
|
||||
}
|
||||
Reference in new issue
Block a user