Keep a published card's representative across restarts
The account that provides a published card stays its representative ahead of server-fetched metadata, and after a restart it is recognized by the text of the active card. Server-fetched metadata only counts as trusted when the catalog URL is the podcast's origin, since an account can point its synced catalog URL anywhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
767134d1dd
commit
bbdac75967
3 files changed
+140
-26
No files matched your search
@@ -144,10 +144,14 @@ your progress, history, listening counts, device names or your account.
|
||||
node key that publishes its music library), not with an identity of your
|
||||
account. When several accounts of this server share the same podcast,
|
||||
the server publishes one card for it, built from one of these accounts'
|
||||
copies of the feed (preferably one the server downloaded itself, and the
|
||||
same account as long as it keeps sharing, so another account sharing the
|
||||
podcast later does not replace the card's text), and keeps it while any
|
||||
of them still shares it. Other members see that this server shares the
|
||||
copies of the feed, and keeps it while any of them still shares it. The
|
||||
account whose copy the published card shows keeps providing it while it
|
||||
shares (also after a server restart, recognised by the card's text), so
|
||||
another account sharing the podcast later does not replace the card's
|
||||
text; a new card preferably uses a copy the server downloaded itself
|
||||
from the podcast's original address. If that account's copy changes
|
||||
while the server restarts, the card may be rebuilt from another sharing
|
||||
account's copy. Other members see that this server shares the
|
||||
podcast, not who. Podcasts shared by a deactivated account are no longer
|
||||
published (within five minutes).
|
||||
- The podcast page says what this server does with the card: published,
|
||||
|
||||
+7
-3
@@ -240,9 +240,13 @@ only while the overlay runs.
|
||||
- `aggregate_share_states` keeps the accounts for which `effective_share`
|
||||
holds and whose card has no privacy warnings or a matching confirmation,
|
||||
and builds one `PodcastShareState` per podcast with
|
||||
`privacy_confirmed = true`. Representative: no warnings, then a catalog
|
||||
from the server's own fetch (`meta_from_feed`), then the previous
|
||||
round's representative (kept in memory), then the smallest account id.
|
||||
`privacy_confirmed = true`. Representative: no warnings, then the
|
||||
incumbent (the previous round's representative, kept in memory, or after
|
||||
a restart an account whose card text equals the active card's text from
|
||||
`PodcastDht::published`), then `AccountShare::trusted_meta`
|
||||
(`meta_from_feed` and the catalog URL is the origin; an account can point
|
||||
its synced catalog URL anywhere before the server fetches it), then the
|
||||
smallest account id.
|
||||
- At most 192 podcasts (replicas keep 256 records per publisher, tombstones
|
||||
included), allocated round-robin over the sharing accounts (`allocate`):
|
||||
each account's queue holds its podcasts the node already publishes
|
||||
|
||||
+125
-19
@@ -66,8 +66,8 @@ use std::time::Duration;
|
||||
use anyhow::Result;
|
||||
use music_dht::podcast::{FeedUrl, PodcastId, PodcastMeta, ShareRecord, SubscriptionRecord};
|
||||
use music_dht::podcast_dht::{
|
||||
FeedUrlPrivacyWarning, MAX_PODCAST_RECORDS_PER_PUBLISHER, PodcastCardSpec, PodcastDht,
|
||||
PodcastDhtSyncStats, PodcastShareState, PublishedPodcast,
|
||||
FeedUrlPrivacyWarning, MAX_PODCAST_RECORDS_PER_PUBLISHER, PodcastCard, PodcastCardSpec,
|
||||
PodcastDht, PodcastDhtSyncStats, PodcastShareState, PublishedPodcast,
|
||||
};
|
||||
use sqlx::{PgConnection, PgPool, Row};
|
||||
use tokio::sync::Notify;
|
||||
@@ -201,6 +201,19 @@ impl AccountShare {
|
||||
&& !self.privacy_confirmed()
|
||||
}
|
||||
|
||||
/// Whether the catalog text comes from the server's own fetch of the
|
||||
/// podcast's origin URL (the URL its id is derived from): an account
|
||||
/// can point its synced catalog at any URL before the server fetches
|
||||
/// it, so `meta_from_feed` alone says nothing about where the text
|
||||
/// comes from.
|
||||
pub(crate) fn trusted_meta(&self) -> bool {
|
||||
self.meta_from_feed
|
||||
&& self.card.as_ref().is_ok_and(|card| {
|
||||
card.id_feed_url.is_none()
|
||||
&& PodcastId::from_feed_url(&card.feed_url) == self.subscription.podcast_id
|
||||
})
|
||||
}
|
||||
|
||||
/// Whether the account counts as sharing the podcast publicly.
|
||||
pub(crate) fn publishable(&self) -> bool {
|
||||
self.is_shared()
|
||||
@@ -440,13 +453,16 @@ pub(crate) struct Aggregation {
|
||||
pub(crate) struct Incumbents {
|
||||
/// Podcasts with an active card of this node (`PodcastDht::published`).
|
||||
pub(crate) published: BTreeSet<PodcastId>,
|
||||
/// Public text of every active card, so its representative is
|
||||
/// recognized after a restart too ([`CardText`]).
|
||||
pub(crate) cards: BTreeMap<PodcastId, CardText>,
|
||||
/// Podcasts of which replicas still hold a record of this node: active
|
||||
/// cards and tombstones that have not expired. Each one occupies a
|
||||
/// `(podcast_id, publisher)` key in the replicas; republishing it
|
||||
/// reuses the key.
|
||||
pub(crate) live_keys: BTreeSet<PodcastId>,
|
||||
/// Representative account of each card in the previous round (kept in
|
||||
/// memory; empty after a restart).
|
||||
/// memory; empty after a restart, then [`Self::cards`] decides).
|
||||
pub(crate) representatives: BTreeMap<PodcastId, i64>,
|
||||
}
|
||||
|
||||
@@ -457,6 +473,11 @@ impl Incumbents {
|
||||
for publication in publications {
|
||||
if publication.active {
|
||||
incumbents.published.insert(publication.podcast_id.clone());
|
||||
if let Some(card) = publication.record.card() {
|
||||
incumbents
|
||||
.cards
|
||||
.insert(publication.podcast_id.clone(), CardText::of_card(card));
|
||||
}
|
||||
}
|
||||
if publication.active || publication.record.expires_at_ms() > now_ms {
|
||||
incumbents.live_keys.insert(publication.podcast_id.clone());
|
||||
@@ -482,6 +503,44 @@ impl Incumbents {
|
||||
}
|
||||
}
|
||||
|
||||
/// The public text of a card: what a representative decides.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub(crate) struct CardText {
|
||||
feed_url: FeedUrl,
|
||||
id_feed_url: Option<FeedUrl>,
|
||||
title: String,
|
||||
author: Option<String>,
|
||||
description: Option<String>,
|
||||
image_url: Option<String>,
|
||||
link: Option<String>,
|
||||
}
|
||||
|
||||
impl CardText {
|
||||
fn of_spec(spec: &PodcastCardSpec) -> Self {
|
||||
Self {
|
||||
feed_url: spec.feed_url.clone(),
|
||||
id_feed_url: spec.id_feed_url.clone(),
|
||||
title: spec.title.clone(),
|
||||
author: spec.author.clone(),
|
||||
description: spec.description.clone(),
|
||||
image_url: spec.image_url.clone(),
|
||||
link: spec.link.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
fn of_card(card: &PodcastCard) -> Self {
|
||||
Self {
|
||||
feed_url: card.feed_url.clone(),
|
||||
id_feed_url: card.id_feed_url.clone(),
|
||||
title: card.title.clone(),
|
||||
author: card.author.clone(),
|
||||
description: card.description.clone(),
|
||||
image_url: card.image_url.clone(),
|
||||
link: card.link.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Computes what the server node publishes from the accounts' views.
|
||||
///
|
||||
/// * Only accounts for which [`AccountShare::publishable`] holds count.
|
||||
@@ -490,11 +549,12 @@ impl Incumbents {
|
||||
/// warnings, or the account confirmed exactly these URLs for its current
|
||||
/// share decision), so `sync_shares` never holds anything back on its
|
||||
/// own. The representative is, in this order: an account whose card has
|
||||
/// no privacy warnings; an account whose catalog comes from the server's
|
||||
/// own fetch of the feed (`meta_from_feed`), not from metadata a client
|
||||
/// synced; the previous round's representative, so another account
|
||||
/// sharing the same podcast does not take over a published card; the
|
||||
/// smallest account id.
|
||||
/// no privacy warnings; the *incumbent* (the previous round's
|
||||
/// representative, or after a restart an account whose card has the
|
||||
/// exact text of the published card), so another account sharing the
|
||||
/// same podcast never takes over a published card; an account whose
|
||||
/// catalog the server fetched itself from the podcast's origin URL
|
||||
/// ([`AccountShare::trusted_meta`]); the smallest account id.
|
||||
/// * At most [`MAX_SERVER_SHARED_PODCASTS`] podcasts, shared fairly between
|
||||
/// the accounts ([`allocate`]): an account whose podcasts fit in its fair
|
||||
/// part of the budget never loses a published card because of other
|
||||
@@ -521,15 +581,23 @@ pub(crate) fn aggregate_share_states(
|
||||
let mut queues: BTreeMap<i64, Vec<QueuedPodcast>> = BTreeMap::new();
|
||||
for (podcast_id, accounts) in by_podcast {
|
||||
let previous = incumbents.representatives.get(&podcast_id).copied();
|
||||
let published_text = incumbents.cards.get(&podcast_id);
|
||||
let publishable: Vec<&AccountShare> = accounts
|
||||
.iter()
|
||||
.filter(|share| share.publishable())
|
||||
.collect();
|
||||
let representative = publishable.iter().min_by_key(|share| {
|
||||
let same_text = published_text.is_some_and(|published| {
|
||||
share
|
||||
.card
|
||||
.as_ref()
|
||||
.is_ok_and(|card| CardText::of_spec(card) == *published)
|
||||
});
|
||||
(
|
||||
!share.warnings().is_empty(),
|
||||
!share.meta_from_feed,
|
||||
previous != Some(share.user_id),
|
||||
!same_text,
|
||||
!share.trusted_meta(),
|
||||
share.user_id,
|
||||
)
|
||||
});
|
||||
@@ -1191,12 +1259,13 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_published_card_keeps_its_representative_and_server_fetched_metadata() {
|
||||
fn a_published_card_keeps_its_representative() {
|
||||
// Account 9 provides the published card; account 3 starts sharing
|
||||
// the same podcast with its own (synced) metadata.
|
||||
// the same podcast with other text.
|
||||
let provider = account(9, "a.xml", true, 10);
|
||||
let mut newcomer = account(3, "a.xml", true, 20);
|
||||
newcomer.card.as_mut().unwrap().title = "Rewritten".into();
|
||||
let published_card = CardText::of_spec(provider.card.as_ref().unwrap());
|
||||
let incumbents = Incumbents {
|
||||
published: BTreeSet::from([id("a.xml")]),
|
||||
representatives: BTreeMap::from([(id("a.xml"), 9)]),
|
||||
@@ -1210,16 +1279,53 @@ mod tests {
|
||||
"Show a.xml"
|
||||
);
|
||||
|
||||
// Metadata only a client synced loses to the server's own fetch,
|
||||
// even for a smaller account id or the previous representative.
|
||||
newcomer.meta_from_feed = false;
|
||||
let aggregation = aggregate(vec![provider.clone(), newcomer.clone()]);
|
||||
// After a restart (no memory) the published card's text names it.
|
||||
let restarted = Incumbents {
|
||||
published: BTreeSet::from([id("a.xml")]),
|
||||
cards: BTreeMap::from([(id("a.xml"), published_card)]),
|
||||
..Incumbents::default()
|
||||
};
|
||||
let aggregation =
|
||||
aggregate_share_states(vec![provider.clone(), newcomer.clone()], &restarted);
|
||||
assert_eq!(aggregation.representatives[&id("a.xml")], 9);
|
||||
let mut synced_provider = provider;
|
||||
|
||||
// The incumbent also outranks server-fetched metadata of another
|
||||
// account: whoever shares first keeps the card's text.
|
||||
let mut synced_provider = provider.clone();
|
||||
synced_provider.meta_from_feed = false;
|
||||
newcomer.meta_from_feed = true;
|
||||
let aggregation = aggregate_share_states(vec![synced_provider, newcomer], &incumbents);
|
||||
assert_eq!(aggregation.representatives[&id("a.xml")], 3);
|
||||
let aggregation =
|
||||
aggregate_share_states(vec![synced_provider, newcomer.clone()], &incumbents);
|
||||
assert_eq!(aggregation.representatives[&id("a.xml")], 9);
|
||||
|
||||
// Without an incumbent, metadata the server fetched from the origin
|
||||
// URL wins over metadata a client synced, even for a smaller id.
|
||||
newcomer.meta_from_feed = false;
|
||||
let aggregation = aggregate(vec![provider, newcomer]);
|
||||
assert_eq!(aggregation.representatives[&id("a.xml")], 9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn metadata_fetched_from_another_url_is_not_trusted() {
|
||||
// Account 3 synced a catalog URL of its choice (the origin stays
|
||||
// in its aliases); the server fetched that URL, so `meta_from_feed`
|
||||
// holds, but the text does not come from the podcast's origin.
|
||||
let mut pointed = account(3, "a.xml", true, 10);
|
||||
pointed.card = build_card(
|
||||
&pointed.subscription,
|
||||
Some(&PodcastMeta {
|
||||
feed_url: url("elsewhere.xml"),
|
||||
title: "Elsewhere".into(),
|
||||
..pointed.subscription.podcast.clone()
|
||||
}),
|
||||
None,
|
||||
None,
|
||||
);
|
||||
assert!(pointed.meta_from_feed);
|
||||
assert!(!pointed.trusted_meta());
|
||||
let origin = account(9, "a.xml", true, 20);
|
||||
assert!(origin.trusted_meta());
|
||||
let aggregation = aggregate(vec![pointed, origin]);
|
||||
assert_eq!(aggregation.representatives[&id("a.xml")], 9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
Reference in new issue
Block a user