Merge pull request 'Подкасты в FRID: общий контракт music_dht::podcast, RSS-парсер и device-sync v4 (MR-F1)' (#1) from sanapach/frid:podcasts/contract into main
CI / check (push) Successful in 1m8s

This commit was merged in pull request #1.
This commit is contained in:
ab committed 2026-11-20 13:51:49 +00:00
commit ddea053e9d
23 files changed
+8470 -16

No files matched your search

+35 -1
View File
@@ -177,13 +177,42 @@ their local output is playing. The web gateway uses a passive server profile;
browser startup and explicit selection are separate inputs.
This register provides eventual exclusivity after a partition heals, not a
quorum lock during a partition. Device-sync profiles now advertise version 3;
quorum lock during a partition. Device-sync profiles now advertise version 4
(version 3 introduced coordination, version 4 adds podcast queue items);
the additive coordination envelope is independently versioned and transported
over the existing JSON `furumi/sync/2` stream. Legacy library records remain
readable, but legacy unfenced personal playback messages cannot control an
upgraded output. See [PLAYBACK_PROTOCOL.md](PLAYBACK_PROTOCOL.md) for the
contract, adapter obligations, configuration and rollout rules.
Podcasts are a separate content type with a pure shared contract in
`music_dht::podcast` (normative description: [PODCASTS.md](PODCASTS.md)). It
owns canonical feed URLs, domain-separated BLAKE3 podcast and episode ids,
feed aliases, the no-GUID episode rule, bounded metadata DTOs, the
self-contained `PodcastEpisodeRef` queue item, and convergent merge functions
for subscriptions, share decisions, progress registers and listening
sessions. The tolerant RSS/Atom parser is shared behind the `podcast-feed`
feature because episode identity depends on it. HTTP fetching, scheduling,
buffering and decoding remain client runtime work, and persistence lives in
client storage that implements the same rules. Podcast audio is fetched from
the publisher: it is never imported, content-addressed or offered for P2P
transfer.
Device-sync protocol v4 carries podcast state between trusted devices only:
five `podcast_*` operations, bounded snapshot sections (the recovery path,
because sync vectors never re-request operations hidden from an older peer;
past the bounds an upgraded peer recovers only the newest records), and an
explicit `PlaybackTrack.source`. The operation enum is closed and v3 clients
ignore `source`, so senders gate on the version each peer reported itself
(relayed profiles only raise it; pairing requests use the version in the
invite); podcast state is never sent to peers below v4, handoff to them is
refused, and their playback commands are ignored while the queue contains
episodes. Jam snapshots replace episodes with opaque placeholders. Progress,
subscription and share registers order by a Lamport generation before
wall-clock `hlc_ms`, so a device with a fast clock cannot undo an observed
seek, restart, unsubscribe or share decision. Public podcast discovery is a future
separate overlay; the music DHT schema and `federation-net` are unchanged.
Music discovery identifies an owner and content; it does not force all domain
traffic through DHT messages. `music-dht` exposes the underlying authenticated
stream capability and defines shared wire models for Furumi catalog and device
@@ -283,3 +312,8 @@ architectural constraint:
- Local durable state survives ordinary peer and network failure.
- Additional stream protocols reuse authenticated connectivity without
coupling bulk data to the typed control channel.
- Private personal state (podcast subscriptions, progress, history) stays in
the trusted-device group; Jam and public overlays only see explicit public
projections or opaque placeholders.
- A device's protocol version is what that device reported; relays never
substitute their own.
+20
View File
@@ -1,5 +1,25 @@
# Changelog
## Unreleased — podcast contract
- `music-dht 0.6.0` (breaking): new `music_dht::podcast` module with the shared
podcast contract: canonical `FeedUrl`, `PodcastId`/`EpisodeId` derivation
(`furumi-podcast:*:v1`), feed aliases, the no-GUID episode identity rule,
`PodcastMeta`/`EpisodeMeta`/`PodcastEpisodeRef`, and convergent merge rules
for subscriptions, shares, progress registers and listening sessions.
- `music-dht 0.6.0`: optional `podcast-feed` feature with the shared tolerant
RSS/Atom parser (`quick-xml`, `encoding_rs`, `url`).
- `music-dht 0.6.0`: device-sync protocol version 4 (ALPN unchanged): five
`podcast_*` operation kinds (subscription, share, feed aliases, progress,
listening sessions), bounded podcast snapshot sections,
`PlaybackTrack.source`, reserved `PlaybackStateWire.playback_rate`,
`DeviceProfileSet.protocol_version`, optional `InviteWire` protocol version
(`p`), self-reported vs relayed version helpers, version-gating helpers and
Jam redaction. New struct fields and enum variants break exhaustive
construction and matching in clients.
- `furumi-library 0.3.0`: follows `music-dht 0.6.0`, whose types it re-exposes.
- See `PODCASTS.md` for the normative contract.
## v0.4.1 — 2026-09-02
- `federation-net 0.3.1`: recover timed-out or failed Mainline-DHT rendezvous clients and expose network health to applications.
Generated
+31 -2
View File
@@ -389,6 +389,12 @@ version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "core_detect"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f8f80099a98041a3d1622845c271458a2d73e688351bf3cb999266764b81d48"
[[package]]
name = "cpufeatures"
version = "0.2.17"
@@ -790,6 +796,20 @@ version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "edd0f118536f44f5ccd48bcb8b111bdc3de888b58c74639dfb034a357d0f206d"
[[package]]
name = "encoding_rs"
version = "0.8.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "89253175e704ab869b04f9db224d38630e0211a53841a79181deae0889e169f5"
dependencies = [
"cfg-if",
"core_detect",
"multiversion_no_op",
"rustversion",
"scopeguard",
"simdutf8",
]
[[package]]
name = "enum-assoc"
version = "1.3.0"
@@ -912,7 +932,7 @@ dependencies = [
[[package]]
name = "furumi-library"
version = "0.2.0"
version = "0.3.0"
dependencies = [
"anyhow",
"blake3",
@@ -2071,18 +2091,26 @@ dependencies = [
"uuid",
]
[[package]]
name = "multiversion_no_op"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "743fb55ba31b18fb1ecef6bdc9aa2743314978ac084044301a7eee33fb99a20d"
[[package]]
name = "music-dht"
version = "0.5.0"
version = "0.6.0"
dependencies = [
"anyhow",
"async-trait",
"blake3",
"data-encoding",
"encoding_rs",
"federation-net",
"futures",
"iroh",
"postcard",
"quick-xml",
"rand 0.9.5",
"rusqlite",
"serde",
@@ -2092,6 +2120,7 @@ dependencies = [
"tokio",
"tracing",
"unicode-normalization",
"url",
]
[[package]]
+4 -1
View File
@@ -14,7 +14,7 @@ repository = "https://gt.hexor.cy/ab/frid"
[workspace.dependencies]
federation-net = { path = "crates/federation-net", version = "0.3.1" }
music-dht = { path = "crates/music-dht", version = "0.5.0" }
music-dht = { path = "crates/music-dht", version = "0.6.0" }
iroh = "1"
iroh-base = "1"
iroh-tickets = "1"
@@ -38,3 +38,6 @@ lofty = "0.22"
data-encoding = "2"
rand = "0.9"
tempfile = "3"
quick-xml = "0.41"
encoding_rs = "0.8.35"
url = "2.5"
+55 -2
View File
@@ -97,8 +97,9 @@ is a runtime input, not a persisted setting.
## Versioning and rollout
The library-sync JSON ALPN remains `furumi/sync/2`; catalog, pairing and library
data do not require a new transport. Device profiles advertise protocol version
3. Coordination envelopes independently carry version 1. Snapshot coordination
data do not require a new transport. Device profiles advertised protocol
version 3 when coordination was introduced (version 4 adds podcast queue items,
see below). Coordination envelopes independently carry version 1. Snapshot coordination
and operation authority are additive optional JSON fields so library-sync
records from older clients still deserialize. Upgraded clients ignore legacy
personal playback snapshots/commands without the coordination envelope.
@@ -109,6 +110,58 @@ applications while preventing divergent control message definitions. These
envelopes use JSON, not a backwards-compatible promise for postcard layouts.
Do not change Jam's independent authority by interpreting personal stamps there.
## Protocol v4: podcast queue items
Device profiles advertise protocol version 4. The JSON ALPN stays
`furumi/sync/2` and the coordination envelope stays version 1. Version 4 adds
to the portable queue DTO:
- `PlaybackTrack.source`: `music`, `podcast_episode` with a self-contained
`PodcastEpisodeRef`, or `private` (Jam placeholder). It is omitted for
music, so music-only state serializes exactly as in v3. Podcast episodes are
never encoded as negative ids or URLs in `file_path`. Queue items carry
`PodcastEpisodeRef.progress_generation`, the greatest progress generation
the sender has seen for the episode.
- `PlaybackStateWire.playback_rate`: reserved, omitted when unset. Treat a
missing or unsupported value as 1.0.
A v3 client does not know `source` and ignores it: it would treat an episode
as a broken music item (id 0, no content id) and forward the queue that way,
losing the episode for every device. `PlaybackSource` is also a closed enum
for v4 clients, so a future variant needs a new protocol version. Adapters
therefore apply the shared helpers from `music_dht::device_sync`:
1. Send a playback snapshot to a peer only when
`playback_allowed_for_peer(state, peer_version)` holds; otherwise send
`None`.
2. Refuse an explicit transfer to a device below v4 while the queue contains
podcast items and show that the target needs an update. The decision uses
the version that device reported itself (`profile_protocol_version`;
profiles relayed by other devices only raise it, `relayed_protocol_version`),
never the local constant. Do not strip episodes from the queue.
3. Ignore `SetState`/`ActiveChanged` from peers below v4 while the local
queue contains podcast items (`playback_command_allowed_from_peer`).
4. A stamped `playback_command` operation with podcast items is itself gated
by `op_allowed_for_peer`.
5. A `PairRequest` is sent before the invite consumer hears from the inviter:
gate its playback snapshot by `InviteWire::inviter_protocol_version()`
(version 1 for invites without one).
A handoff of a podcast episode records the position and the play/pause intent
first, transfers the queue with its sources, and lets the receiver open the
publisher's HTTP source and seek. Ownership still changes only through the
claim, so a late response cannot make the previous owner resume audio. The
durable episode progress is a separate register; see `PODCASTS.md`. Only the
receiver writes the handoff's explicit progress action, with
`ProgressRecord::explicit_generation(max(local, progress_generation))`; the
sender writes a final checkpoint at its session generation.
Jam hosts send `music_dht::jam::redact_jam_snapshot`, which replaces podcast
items with opaque `private` placeholders and drops the personal coordination
envelope. They reject participant commands with real episodes
(`jam_command_allowed`) and map placeholders back with
`restore_jam_private_items` before applying a participant's state.
For local integration checks, patch the three frid crates with Cargo's external
`--config` mechanism. Keep released dependency declarations and lockfiles in
the applications; publish frid and then bump client dependencies for rollout.
+529
View File
@@ -0,0 +1,529 @@
# Podcast contract
Status: normative for `music-dht` 0.6 and device-sync protocol v4.
Podcasts are a separate content type shared by every Furumi client. This
document defines what all clients must agree on: identity, normalization,
private state and its merge rules, trusted-device sync operations, snapshot
recovery, queue items, version gating and Jam privacy. The Rust definitions
live in `music_dht::podcast` (pure types and functions) and
`music_dht::device_sync` (wire DTOs and gating helpers). Clients must use
them instead of local copies.
Public discovery (signed podcast cards in a separate DHT overlay) is not part
of this version of the contract; it will be specified with the overlay.
## Responsibilities
| Layer | Owns |
| --- | --- |
| `music_dht::podcast` | ids, URL canonicalization, no-GUID rule, metadata DTOs and bounds, merge functions, RSS/Atom parser (feature `podcast-feed`) |
| `music_dht::device_sync` | sync operation kinds, snapshot sections, `PlaybackSource`, version gating, Jam redaction |
| Client storage (`furumi-library`, web PostgreSQL) | persistence and migrations implementing the rules below |
| Client runtime | HTTP fetching, conditional requests, scheduling, audio buffering and decoding, UI |
Audio is always fetched from the publisher. It is never imported into the
music library, never content-addressed with a BLAKE3 content id and never
offered for P2P transfer. RSS refreshes fetch metadata only.
## Data classes
| Data | Trusted devices | Public federation | Local only |
| --- | --- | --- | --- |
| Subscription state, channel metadata | `podcast_subscription_set` + snapshot | never | |
| Feed aliases | `podcast_feed_aliases_added`, `podcast_subscription_set` + snapshot | never | |
| Share decision | `podcast_share_set` + snapshot | only the published card (future overlay) | |
| Episode progress | `podcast_progress_set` + snapshot | never | |
| Listening sessions, play counts | `podcast_listen_recorded` + snapshot | never | |
| Episode catalog | each device parses RSS itself; `PodcastEpisodeRef` travels with progress and queue items | never | |
| ETag/Last-Modified, last check time, fetch errors, artwork cache, audio buffer | | | yes |
Synchronizing to a trusted device is not permission to publish. Private
podcasts, their URLs, descriptions and episodes must not reach DHT records,
public catalogs, public search results or Jam participants.
## Identity
### Feed URL canonicalization
`FeedUrl::parse` produces the canonical form. Every client must use it (or
an implementation that passes the same tests) before deriving ids or
comparing URLs.
1. Only `http://` and `https://` are accepted (case-insensitive) and the
scheme is lowercased.
2. Userinfo (`user@`, `user:pass@`), whitespace, control characters and `\`
are rejected.
3. The host is lowercased. Non-ASCII hosts are rejected; pass the IDNA
(`xn--`) form as produced by WHATWG URL parsers. Bracketed IPv6 literals
are accepted.
4. An empty port and the default port (80 for http, 443 for https) are
removed; other ports lose leading zeros.
5. The fragment is removed.
6. An empty path becomes `/`. The rest of the path and the **entire** query
string are kept byte-for-byte: parameter order, case, percent-encoding and
dot segments are preserved. Query parameters can be part of the source
identity (for example a per-user token) and must never be dropped. Only
non-ASCII characters are percent-encoded (UTF-8, uppercase hex).
7. The result is at most 2048 bytes. Canonicalization is idempotent;
deserialization accepts only canonical strings.
| Input | Canonical |
| --- | --- |
| `HTTPS://Example.COM:443/Feed.xml?b=2&a=1#top` | `https://example.com/Feed.xml?b=2&a=1` |
| `http://example.com:80` | `http://example.com/` |
| `https://example.com/rss?token=XyZ&x=%2f` | unchanged |
| `https://user@example.com/rss` | rejected |
`FeedUrl::from_user_input` is for typed or pasted input only: it trims, maps
`feed:`, `podcast:`, `pcast:`, `itpc:` to `https://` and adds `https://` when
the scheme is missing, then canonicalizes.
### Podcast id
```text
podcast_id = "pod1:" + base32_lower_nopad(BLAKE3("furumi-podcast:podcast:v1" || 0x00 || canonical_feed_url))
```
The id is derived **once**, from the canonical URL of the first subscription.
It never changes afterwards.
### Feed aliases
Redirect targets (permanent redirects), `itunes:new-feed-url` and URLs used to
subscribe again are feed aliases of an existing podcast:
* aliases form a grow-only set per podcast, merged independently of the
subscription register (`merge_feed_aliases`). The URL that derives the
podcast id is always kept, plus the 8 lexicographically smallest other
URLs; this bounded union is commutative, associative and idempotent;
* aliases travel in their own operation, `podcast_feed_aliases_added`. A
background refresh that learns a URL emits it, but only when merging the
URL changes the stored set (a URL outside the bound is not sent again on
every refresh). The operation never touches the subscription or share
register, so it cannot resurrect a subscription or reset a share decision.
Subscribe and unsubscribe operations also carry the aliases known at that
time;
* `podcast.feed_url` of the subscription register is the URL used at the last
explicit subscribe or unsubscribe. It can fall outside the bounded alias
set, so the URLs that identify a podcast are
`SubscriptionRecord::feed_urls()`: the aliases plus `podcast.feed_url`;
* before deriving a new id for a URL, clients call `resolve_podcast_id` with
every known `(podcast_id, url)` pair, including `feed_urls()` and the URL
each device currently fetches. A match reuses the existing podcast, its
progress and its history. With several matches the smallest id wins.
Limitations: two devices that subscribe offline, through different URLs of
the same feed, before either learns the alias, create two podcast ids; the
contract does not merge podcast ids automatically in this version. A podcast
with more than 8 aliases can lose a larger URL from the synced set; a device
that only knows that URL derives a new id.
### Episode id
```text
episode_id = "ep1:" + base32_lower_nopad(BLAKE3("furumi-podcast:episode:v1" || 0x00 || podcast_id || key))
```
| Key | Material |
| --- | --- |
| GUID | `guid:` + RSS `<guid>` / Atom `<id>` trimmed, case-sensitive |
| link | `link:` + item `<link>` text (XML-unescaped once) or Atom `href`, trimmed of surrounding whitespace; no other cleanup |
| date + title | `date-title:` + publication unix seconds + `\0` + normalized title |
| enclosure | `enclosure:` + enclosure URL trimmed |
The normalized title is `music_dht::normalize_name` (NFKC, lowercase,
non-alphanumeric runs collapsed). GUIDs longer than 2048 bytes are treated as
absent. Changing the title, artwork, description or audio URL of an episode
never changes its id and never resets its progress.
### No-GUID rule
`resolve_episode_ids(podcast_id, channel_link, items, known)` is normative.
For one parsed feed document:
1. Items with a GUID get the GUID id. A GUID never matches another known
episode, so the result does not depend on local history.
2. Items without GUID then search the known episodes of the podcast in three
stages, each run over the **whole document** before the next one starts:
same enclosure URL, then same `<link>`, then same publication second and
normalized title. Among candidates the smallest id is reused. A known
episode can be claimed by only one item. Because a weak match of an
earlier item cannot take a known episode that a later item matches by
enclosure, the result does not depend on the order of items.
3. Otherwise a new id is derived from the first available key: `<link>`,
then date + title, then enclosure URL.
A `<link>` is identifying only if it is unique in the document and differs
from the channel `<link>` (many feeds repeat the homepage in every item).
Enclosure URLs and date + title pairs shared by several GUID-less items are
not identifying either. Items without any usable key, and duplicates of an
earlier item of the same document, get no id and are skipped.
The identity hints (GUID, enclosure, link, publication time, title) are the
ones `music_dht::podcast::feed` produces: which enclosure or `media:content`
is chosen, how relative URLs are resolved, which date field is used and how
it is parsed, how the title is cleaned. They are part of the rule; clients
must use that parser (or an implementation that passes its tests). Relative
media URLs are resolved against the URL the document was fetched from.
"Known episodes" include episodes from earlier refreshes **and** episodes
learned through sync (`PodcastEpisodeRef::identity_hints`). Feed GUID-less
episodes into it so a device that sees an episode first through sync keeps
the id used by the other devices.
Limitations, GUID-less episodes only:
* The id can depend on what a device saw first. If the publisher changes the
field the id was derived from (the `<link>`, or the title or date for a
date + title id), a device that saw the old version keeps the old id
through the enclosure match, while a device that never saw it derives a
new id. One changed field is enough.
* Feeds with relative enclosure URLs fetched through different aliases give
different enclosure URLs, so the enclosure match and enclosure-derived ids
can differ between such devices.
## Metadata and bounds
| Field | Bound |
| --- | --- |
| titles, author | 1000 characters, non-empty titles |
| descriptions | 4000 characters (plain text) |
| URLs | 2048 bytes, audio and artwork URLs absolute `http(s)` |
| GUID | 2048 bytes |
| MIME | 128 printable ASCII bytes |
| device, session, operation ids | 1..256 bytes, no control characters |
| feed aliases | 8 plus the origin URL |
`PodcastMeta` (channel), `EpisodeMeta` (episode, including season, episode
number, `itunes:episodeType`, publication time and duration in milliseconds,
audio URL, MIME, byte length, artwork and link) and `PodcastEpisodeRef`
(queue item) validate these bounds. `PodcastMeta::bounded` and
`EpisodeMeta::to_ref` truncate publisher data to fit. Receivers drop records
that fail validation.
Episodes without a season are shown in their own group. Unknown
`itunes:episodeType` values are `full`.
## Personal state and merge rules
All merge functions are commutative, associative and idempotent per entity;
ties are broken by a total order over all fields. The same set of operations
therefore yields the same state in any delivery order, with duplicates.
`hlc_ms` is the sync operation timestamp; clients advance it only past their
own operations, so it is not a hybrid logical clock and a device with a fast
clock stamps every write "later". Every register therefore orders by a
Lamport `generation` first and uses `hlc_ms` only between writes that did not
observe each other.
### Subscription
* Register: `subscribed` + `podcast` metadata, last-writer-wins by
`(generation, hlc_ms, op_id)`.
* `generation` is a per-podcast Lamport counter shared with the share
register: every subscribe, unsubscribe and share write uses
`next_podcast_generation(subscription, share)` = one more than the greatest
generation of either register (1 when there is none). An action taken after
observing another device's write orders after it whatever the clocks say.
* Unsubscribing writes `subscribed = false`. It is state, not a tombstone,
and is never compacted, so it beats a stale subscription from a device that
returns after being offline.
* Unsubscribing stops monitoring but keeps progress and history.
Subscribing again continues where the user stopped. Erasing history is a
separate explicit action (not part of this version).
* Only explicit subscribe and unsubscribe actions write the register. RSS
refreshes keep their metadata locally; a background refresh that wrote the
register could resurrect an unsubscribed podcast. Aliases learned by a
refresh use `podcast_feed_aliases_added`.
* Subscribing to a podcast already subscribed locally must not emit a new
operation.
| Events | Result |
| --- | --- |
| A (clock 10 min ahead) subscribes at g1; B sees it and unsubscribes at g2 | unsubscribed on every device |
| A and B, both offline after g1, unsubscribe and subscribe at g2 | the later `hlc_ms` (then `op_id`) wins |
### Share
* `ShareRecord`, last-writer-wins by `(generation, hlc_ms, op_id)`, with the
generation from `next_podcast_generation`.
* `shared = false` revokes the publication. Like an unsubscribe it is
privacy-protecting state, **not** a tombstone: it is never compacted and
its row is never deleted. Otherwise a stale `shared = true` still in some
device's log, or delivered again to a new device, would republish the
podcast. The register is one record per podcast, so keeping it is cheap.
* Unsubscribing also writes `shared = false`.
* `effective_share(subscription, share)` decides whether to publish: the user
is subscribed, chose to share, **and** the share decision orders after the
latest subscription write by `(generation, hlc_ms, op_id)`. Every new
subscription, including subscribing again or subscribing from someone
else's public card, is private until the user shares it again; a stale
share decision cannot republish; a share decision taken after observing the
subscription always applies.
### Progress
`ProgressRecord { episode_id, podcast_id, position_ms, status, generation,
session_id, hlc_ms, device_id, op_id }`, with `status` one of `not_started`,
`in_progress`, `completed`. The winner is the greatest
`(generation, hlc_ms, device_id, op_id)`. Progress is never the maximum
position: seeking back and "start over" are real changes.
* Explicit actions use `generation = max_seen + 1`
(`ProgressRecord::explicit_generation`): starting a listening session,
seeking, "start over", "mark played", "mark unplayed", taking over playback
through a handoff. `max_seen` is the generation of the current register,
including received records. An explicit action during a session becomes
the generation of that session: its later checkpoints use it.
* Periodic checkpoints and pause/stop saves within a session keep the
generation of that session's latest explicit action.
* Handoff: queue items carry `PodcastEpisodeRef.progress_generation`, the
greatest generation the sender has seen for the episode, because the
playback state can arrive before the sender's progress operations. The
sender writes a final checkpoint at its session generation and does not
bump. Only the receiver writes the explicit action, with
`explicit_generation(max(local, progress_generation))`, so its writes order
after the sender's whatever the clocks say.
* A device that observes a greater generation for the episode it is playing
stops writing at its old generation: either playback moved away (the
coordination claim says so), or the user keeps listening locally and the
device writes its next record as an explicit action
(`explicit_generation(max_seen)`).
* Write progress periodically (every 15–30 s) and on pause, seek, stop,
episode change and handoff, in the background. The outgoing log may drop a
pending progress operation only when another pending operation of the same
episode `supersedes` it (greater `(generation, hlc_ms, device_id, op_id)`).
Never coalesce by sequence order: a checkpoint of an older session queued
after "mark played" must not replace it.
| Events | Result |
| --- | --- |
| A checkpoints session s1 at g3; B seeks back at g4 with a slower clock; A's late g3 checkpoint arrives | B's seek (g4) wins |
| A restarts (g3, position 0); a queued g2 checkpoint of the old session arrives later | restart wins |
| A and B both saw g5 and listen offline; both start sessions at g6 | the later `hlc_ms` (then `device_id`, `op_id`) wins on every device; both sessions are kept |
| A plays at g5 with a fast clock and hands off to B, which has only seen g2 | B starts at g6 (`max(2, 5) + 1`) and its checkpoints win |
| A marks played (g5), then a queued g3 checkpoint of its session is written | the outgoing log keeps g5; g3 is dominated |
Ordinary Play resumes from `resume_position_ms()` (the position when in
progress, otherwise 0). "Start over" is a separate explicit action. Publisher
metadata changes never touch progress; if the duration or audio changed, the
client keeps the stored position and lets the user decide.
### Listening sessions and counts
`ListenSession { session_id, episode_id, podcast_id, device_id,
started_at_ms, ended_at_ms, start_position_ms, end_position_ms, listened_ms,
completed }` forms a grow-only set keyed by `session_id` (generated at
playback start). A device may resend its session with more listening time;
versions merge to the greatest `(completed, listened_ms, ended_at_ms, ...)`.
* A session is recorded in history when `completed` or
`listened_ms >= 30 000` (`should_record`).
* `completed` is set when the session reaches the end of the stream or
`reaches_completion(position_ms, duration_ms)`: within the smaller of 30 s
and 5 % of the duration from the end. Completing also writes progress with
status `completed` (explicit generation).
* The play count of an episode is the number of distinct completed sessions
(`completed_listen_count`). Each session counts at most once, so
re-delivery never double counts, and listening again is a new session.
* "Mark played" only writes progress; it does not create a session or invent
listening time.
* Concurrent offline sessions are all kept; there is no promise of a single
global session during a partition.
## Trusted-device sync (protocol v4)
The ALPN remains `furumi/sync/2`; device profiles advertise
`protocol_version = 4`. New operation kinds (none of them is a tombstone):
| Kind | Payload | Materialization |
| --- | --- | --- |
| `podcast_subscription_set` | `podcast_id`, `subscribed`, `podcast`, `feed_aliases`, `generation` | `SyncOpWire::podcast_subscription` |
| `podcast_share_set` | `podcast_id`, `shared`, `generation` | `SyncOpWire::podcast_share` |
| `podcast_feed_aliases_added` | `podcast_id`, `feed_aliases` (1..9 URLs) | `SyncOpWire::podcast_feed_aliases` |
| `podcast_progress_set` | `episode_id`, `podcast_id`, `position_ms`, `status`, `generation`, `session_id`, optional `episode` ref | `SyncOpWire::podcast_progress` |
| `podcast_listen_recorded` | `session`, optional `episode` ref | `SyncOpWire::podcast_listen_session` |
Register stamps come from the operation: `hlc_ms`, `op_id`, and for progress
`device_id = origin_device_id`. A session's `device_id` must equal the
operation origin. `SyncOpWire::validate_podcast` checks this and the bounds;
receivers skip operations that fail it but still acknowledge them, so the
log keeps moving.
Ids (`PodcastId`, `EpisodeId`) and feed URLs (`FeedUrl`) are validated while
the message is deserialized: a malformed id or a non-canonical feed URL is a
schema violation, like a wrong JSON type, and the **whole message** fails to
parse. Conforming clients cannot produce one, because these types serialize
only canonical values, and their rules are frozen with the `v1` derivation
domains (changing them needs a new protocol version, see Versioning).
### Peer versions
Each device reports its own version: in the `profile` of its `Hello`,
`PairRequest` and `PairResponse`, in its own entry of the `devices` list (a
`SyncResponse` has no `profile`, so an initiator learns the responder's
version from the entry whose `device_id` is the responder), and in
`DeviceProfileSet.protocol_version` of operations it originates. A client
emits a `DeviceProfileSet` with `protocol_version` on the first start of a
new protocol version.
* A self-report replaces the stored version:
`profile_protocol_version(reported, stored)` (`None` keeps the stored
value).
* Profiles of other devices relayed in `devices` only raise it:
`relayed_protocol_version(relayed, stored)` = `max(stored, relayed)`, or
`relayed` when nothing is stored. Deployed v1–v3 clients relay every
device with their own constant, so a relayed 3 must not downgrade a v4
peer.
* Relays forward the stored version of other devices in
`DeviceProfileWire.protocol_version` and must never substitute their own
constant: version gating would then send podcast state to an old client.
* A device that never reported a version is treated as version 1.
Downgrading a device below v4 is not supported while relays still hold
v4 for it.
### Pairing
The invite consumer sends its `PairRequest` (operations, snapshot, playback)
before it hears from the inviter. Invites created by v4 clients carry the
inviter's version in `InviteWire.protocol_version` (JSON key `p`, omitted by
older clients). The consumer gates the whole request by
`InviteWire::inviter_protocol_version()`, which is version 1 for invites
without it: no podcast operations, `snapshot.for_peer(version)`, and no
playback snapshot with podcast items. Podcast state then catches up through
the snapshot of the next regular sync. The inviter gates its `PairResponse`
by the version in the request's `profile`.
### Snapshot recovery
Sync vectors only grow, so operations hidden from a peer while it ran an
older version are never re-requested after it upgrades. Podcast state is
therefore part of `SyncSnapshot`:
| Section | Content | Limit | Budget share |
| --- | --- | --- | --- |
| `podcast_subscriptions` | subscription registers, including unsubscribed, with feed aliases; `podcast.description` is omitted | 1000 | 25 % |
| `podcast_shares` | share registers, including `shared = false` | 1000 | 5 % |
| `podcast_progress` | progress registers | 5000 | 45 % |
| `podcast_listen_sessions` | most recent sessions | 2000 | 25 % |
`SyncSnapshot::bound_podcast_state_to(budget)` orders each section newest
first (registers by `(generation, hlc_ms)`, sessions by `ended_at_ms`), caps
it by count and by its share of the JSON byte budget; unused budget passes on
to the following sections in table order, so large subscription records
cannot crowd out progress. `bound_podcast_state()` uses the default 4 MiB
budget. Senders whose other message parts are large pass
`podcast_snapshot_budget(other_bytes)`, which keeps the whole line under the
8 MiB limit. Channel descriptions are not snapshotted: every device refreshes
channel metadata from the feed.
Receivers ignore podcast sections that exceed the counts
(`podcast_state_within_limits`), validate each record and merge it with the
same functions as operations.
Limitations: records cut by the bounds reach a peer only through operations
it has not acknowledged yet. A peer that upgrades from v3, or that received
operations through a v3 relay, already skipped the podcast operations: it
recovers only the newest records that fit, so older progress registers and
listening sessions are missing on that device and its play counts
(`completed_listen_count`) can differ from other devices.
Empty sections are not serialized, so snapshots without podcast state are
byte-identical to v3.
## Queue and playback
`PlaybackTrack.source: Option<PlaybackSource>` makes the item source
explicit: absent or `music` for music, `podcast_episode` with a
`PodcastEpisodeRef`, or `private` for a Jam placeholder. Negative ids,
placeholder releases and URLs in `file_path` are not part of the contract.
`PlaybackTrack::podcast_episode` fills the legacy display fields (episode
title, podcast title as artist and release, `id = 0`, empty `file_path`, no
`content_id`); receivers read `source`.
`PodcastEpisodeRef` is self-contained: podcast and episode ids, canonical
feed URL, titles, audio URL, MIME, duration, publication time, season,
episode number, artwork. A device without the subscription can display,
resume and play the item; if the audio URL is stale it refreshes the feed and
looks the episode up by id. Mixed queues keep order, position and identity of
every item.
Handoff follows `PLAYBACK_PROTOCOL.md`: record position and play/pause
intent, transfer the queue with its sources, the receiver opens the HTTP
source and seeks, and ownership moves only through the coordination claim.
`PlaybackStateWire.playback_rate: Option<f32>` is reserved. No client
implements variable speed yet; receivers treat a missing or unsupported value
as 1.0. When implemented, position extrapolation and handoff must use it.
## Version gating for peers below v4
`SyncOpPayload` is a closed serde enum. A v3 client that receives an unknown
operation `kind` fails to parse the **whole** message, not one operation,
which would stall synchronization. A v3 client does not know
`PlaybackTrack.source` and ignores it: it would treat an episode as a broken
music item (id 0, no content id) and forward the queue that way, losing the
episode for every device. Additive `serde(default)` fields alone therefore do
not make podcasts compatible. The peer version comes from "Peer versions"
above, and from the invite while pairing. Therefore:
1. Never send podcast operations to a peer below v4
(`op_allowed_for_peer`).
2. Never send podcast snapshot sections (`SyncSnapshot::for_peer`) or a
playback snapshot whose queue contains podcast items
(`playback_allowed_for_peer`) to a peer below v4.
3. Refuse a handoff to a device below v4 while the queue contains podcast
items and tell the user that the target must be updated. Never strip the
episodes to make the queue fit.
4. Reject `SetState`/`ActiveChanged` from a peer below v4 while the local
queue contains podcast items (`playback_command_allowed_from_peer`): its
state would silently drop them.
5. Music operations and music-only playback keep working with v1–v3 peers.
`PlaybackSource` is closed for v4 clients as well: a v4 client fails to parse
a message with an unknown source `kind`.
## Jam privacy
Jam participants are outside the trusted-device group.
* Hosts send `jam::redact_jam_snapshot` (or `redact_for_jam` on the state):
every podcast item becomes a `private` placeholder titled "Private item"
without URL, feed, ids, titles or duration; music items, order and position
are kept.
* Commands containing real podcast episodes are rejected
(`jam_command_allowed`): participants cannot add episodes.
* Before applying a participant's state, hosts call
`restore_jam_private_items`: the k-th placeholder maps to the host's k-th
private item. States that drop, duplicate or inject private items are
rejected. Participants may still pause, seek and reorder music.
## Feed parser
With the `podcast-feed` feature, `music_dht::podcast::feed` parses RSS 2.0,
RSS 1.0 and Atom tolerantly: BOM and declared single-byte encodings, HTML
entities, CDATA, mismatched end tags, undeclared prefixes, depth and size
limits, lenient dates and durations, relative media URLs resolved against
the fetch URL. Items without an absolute `http(s)` audio URL are dropped. It
extracts `itunes:season`, `itunes:episode`, `itunes:episodeType`,
`guid isPermaLink` and `itunes:new-feed-url`, and produces identity hints for
`resolve_episode_ids`; those hints are normative (see the no-GUID rule).
Links are kept verbatim (trimmed only) because they are identity material.
Text cleanup is linear in the input size. It performs no I/O.
## Versioning
* `music-dht` 0.6.0 and `furumi-library` 0.3.0: new public types, enum
variants and struct fields are breaking for 0.x.
* Device-sync protocol 4, ALPN unchanged. Podcast payloads are JSON; no
postcard layout changes.
* Id domains `furumi-podcast:podcast:v1` and `furumi-podcast:episode:v1` and
the `pod1:`/`ep1:` prefixes are frozen. A different derivation needs new
domains, new prefixes, a migration and a new protocol version.
* Adding an operation kind or a `PlaybackSource` variant requires a new
protocol version and gating as above.
* Feed URL canonicalization, the identity bounds (`MAX_URL_BYTES`,
`MAX_GUID_BYTES`) and the no-GUID rule are frozen with the `v1` domains:
receivers reject non-canonical ids and URLs while parsing.
* Public podcast cards will use a separate signed overlay with its own ALPN
and version; the music DHT schema and `federation-net` do not change.
+1 -1
View File
@@ -13,7 +13,7 @@ The workspace has three layers:
- [`music-dht`](crates/music-dht) builds a Kademlia-style distributed music
directory on top of that transport, including catalog discovery, content-id
lookup, signed similarity-LSH routing, direct byte streams, and shared Furumi
wire types.
wire types, including the shared podcast contract ([PODCASTS.md](PODCASTS.md)).
- [`furumi-library`](crates/furumi-library) owns the shared local `library.db`
contract, migrations, catalog operations, metadata import, artwork,
playlists, likes, history, and similarity persistence used by Furumi clients.
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "furumi-library"
version = "0.2.0"
version = "0.3.0"
description = "Client-independent SQLite music library for Furumi players"
readme = "README.md"
documentation = "https://docs.rs/furumi-library"
+4
View File
@@ -19,3 +19,7 @@ publication, clients can depend on the released crate version instead.
`Library::open(default_db_path()?)` opens the same platform-native Furumi
library used by every client. Connections use WAL, foreign keys, and a busy
timeout so independently running clients can safely share the database.
Version 0.3 follows `music-dht` 0.6, whose device-sync and podcast contract
types it re-exposes. Podcast persistence is not part of this release; it will
implement the rules of the workspace `PODCASTS.md`.
+8 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "music-dht"
version = "0.5.0"
version = "0.6.0"
description = "Distributed music library search: a Kademlia-style DHT on top of federation-net"
readme = "README.md"
documentation = "https://docs.rs/music-dht"
@@ -26,6 +26,13 @@ thiserror = { workspace = true }
tracing = { workspace = true }
rand = { workspace = true }
data-encoding = { workspace = true }
quick-xml = { workspace = true, optional = true }
encoding_rs = { workspace = true, optional = true }
url = { workspace = true, optional = true }
[features]
# RSS 2.0 / RSS 1.0 / Atom podcast feed parser (`music_dht::podcast::feed`).
podcast-feed = ["dep:quick-xml", "dep:encoding_rs", "dep:url"]
[dev-dependencies]
anyhow = { workspace = true }
+59 -1
View File
@@ -185,7 +185,9 @@ new clients should accept both ALPNs and only send `ListenRecorded` when
`supports_listen_history(peer.protocol_version)` returns true. Likes,
playlists and membership remain compatible with v1. Profile version 3 adds
versioned personal playback coordination over the existing JSON sync stream;
legacy unfenced playback messages are ignored by upgraded clients.
legacy unfenced playback messages are ignored by upgraded clients. Profile
version 4 adds podcasts (see below); `op_allowed_for_peer` applies all of
these rules to one operation.
`music_dht::playback` provides the shared pure ownership engine and generic
snapshot/command envelopes. Configure startup and failover with `Config`, feed
@@ -193,12 +195,68 @@ actual output reports, and persist `Checkpoint` before applying decisions.
`Config::passive()` is intended for always-running gateways. See the workspace
`PLAYBACK_PROTOCOL.md` for invariants, configuration and migration details.
## Podcasts
`music_dht::podcast` is the shared, pure podcast contract (no SQLite, HTTP or
clock). Clients use it for identity, merge rules and wire DTOs; see the
workspace `PODCASTS.md` for the normative rules.
```rust
use music_dht::podcast::{
EpisodeIdentityHints, FeedUrl, PodcastId, ProgressRecord, resolve_episode_ids,
};
let url = FeedUrl::from_user_input("feed://example.com/rss?token=abc")?;
let podcast = PodcastId::from_feed_url(&url); // "pod1:..."
let hints = [EpisodeIdentityHints {
guid: Some("episode-1".into()),
..Default::default()
}];
let ids = resolve_episode_ids(&podcast, None, &hints, &[]); // "ep1:..."
// Explicit actions (seek, start over, new session) bump the generation.
let generation = ProgressRecord::explicit_generation(Some(4)); // 5
```
* `FeedUrl` canonicalizes feed URLs and keeps every query parameter.
* `PodcastId`, `EpisodeId`, `EpisodeKey`, `resolve_podcast_id`,
`merge_feed_aliases`, `resolve_episode_ids`: identity, aliases and the
no-GUID rule.
* `PodcastMeta`, `EpisodeMeta`, `PodcastEpisodeRef`: bounded metadata and the
self-contained queue item.
* `SubscriptionRecord`, `ShareRecord`, `ProgressRecord`, `ListenSession` with
`merge_subscription`, `merge_share`, `merge_progress`,
`merge_listen_session`, `next_podcast_generation`, `effective_share`,
`reaches_completion` and `completed_listen_count`.
Enable the `podcast-feed` feature for `music_dht::podcast::feed`, the shared
tolerant RSS 2.0 / RSS 1.0 / Atom parser:
```toml
music-dht = { version = "0.6", features = ["podcast-feed"] }
```
Device-sync protocol v4 adds `podcast_subscription_set`, `podcast_share_set`,
`podcast_feed_aliases_added`, `podcast_progress_set` and
`podcast_listen_recorded`, podcast snapshot sections, `PlaybackTrack.source`
and a reserved `playback_rate`. Old clients cannot parse the new operation
kinds and ignore `source`, so always filter with `op_allowed_for_peer`,
`playback_allowed_for_peer`, `SyncSnapshot::for_peer` and
`playback_command_allowed_from_peer`, using the protocol version each device
reported itself (`profile_protocol_version`; relayed profiles go through
`relayed_protocol_version`, pairing requests use
`InviteWire::inviter_protocol_version`). Bound snapshots with
`SyncSnapshot::bound_podcast_state_to(podcast_snapshot_budget(..))`. Jam hosts use
`jam::redact_jam_snapshot`, `jam_command_allowed` and
`restore_jam_private_items`.
## Verification
```bash
cargo fmt --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test -p music-dht
cargo test -p music-dht --features podcast-feed
```
The integration test starts three real nodes in one process (they use Iroh's
File diff suppressed because it is too large. Load diff
+65
View File
@@ -3,6 +3,12 @@
//! Jam is separate from personal-device sync. Possession of a [`JamInvite`]
//! capability authorizes playback control for one host process, but never
//! grants likes, playlists, history, or trusted-device membership.
//!
//! Podcast episodes are private. Hosts send [`redact_jam_snapshot`] to
//! participants, reject participant commands for which
//! [`crate::device_sync::jam_command_allowed`] is false, and map placeholders
//! back with [`crate::device_sync::restore_jam_private_items`] before applying
//! a participant's state.
use serde::{Deserialize, Serialize};
use tokio::io::{AsyncRead, AsyncReadExt};
@@ -140,6 +146,17 @@ pub enum JamWireMessage {
},
}
/// Returns a playback snapshot safe to send to Jam participants.
///
/// Podcast episodes become opaque placeholders (see
/// [`crate::device_sync::redact_for_jam`]) and the personal coordination
/// envelope is removed, since it belongs to the trusted-device scope.
pub fn redact_jam_snapshot(mut snapshot: PlaybackSnapshot) -> PlaybackSnapshot {
snapshot.state = crate::device_sync::redact_for_jam(snapshot.state);
snapshot.coordination = None;
snapshot
}
/// Reads a bounded Jam message from an async reader.
pub async fn read_message_from<R: AsyncRead + Unpin>(reader: &mut R) -> Result<JamWireMessage> {
let mut line = Vec::new();
@@ -277,6 +294,54 @@ mod tests {
assert!(JamInvite::from_uri("frid://i/abcd").is_err());
}
#[test]
fn jam_snapshots_hide_podcast_episodes() {
use crate::device_sync::{PlaybackRepeat, PlaybackStateWire, PlaybackTrack};
use crate::podcast::{EpisodeId, FeedUrl, PodcastEpisodeRef, PodcastId};
let feed = FeedUrl::parse("https://private.test/feed").unwrap();
let podcast_id = PodcastId::from_feed_url(&feed);
let episode = PodcastEpisodeRef {
episode_id: EpisodeId::from_guid(&podcast_id, "e").unwrap(),
podcast_id,
feed_url: feed,
podcast_title: "Hidden show".to_string(),
episode_title: "Hidden episode".to_string(),
audio_url: "https://private.test/e.mp3".to_string(),
mime: None,
duration_ms: None,
published_at_ms: None,
season: None,
episode_number: None,
image_url: None,
progress_generation: Some(7),
};
let snapshot = PlaybackSnapshot {
device_id: "host".to_string(),
device_name: "Host".to_string(),
active: true,
updated_at_ms: 1,
state: PlaybackStateWire {
queue: vec![PlaybackTrack::podcast_episode(episode)],
queue_pos: 0,
playing: true,
paused: false,
idle_since_ms: None,
position_secs: 1.0,
volume: 50,
shuffle: false,
repeat: PlaybackRepeat::Off,
playback_rate: None,
},
coordination: None,
};
let redacted = redact_jam_snapshot(snapshot);
let json = serde_json::to_string(&redacted).unwrap();
assert!(!json.contains("Hidden"));
assert!(!json.contains("private.test"));
assert!(redacted.state.queue[0].is_private_placeholder());
}
#[test]
fn weak_capability_is_rejected() {
let mut invite = invite();
+5
View File
@@ -23,6 +23,10 @@
//! * [`similarity_lsh`] turns those embeddings into deterministic compact
//! routing summaries, while [`similarity_dht`] stores and discovers signed
//! peer summaries without owning model inference or an embedding database.
//! * [`podcast`] is the pure, shared podcast contract (canonical feed URLs,
//! podcast/episode ids, metadata DTOs and convergent merge rules), with an
//! optional RSS/Atom parser behind the `podcast-feed` feature;
//! [`device_sync`] protocol v4 carries it between trusted devices.
//! * Records are replicated to the `K` nodes whose ids are XOR-closest to
//! each key. Publishers pick the targets from their routing table and send
//! batched store requests (one pipeline per peer), so even a large library
@@ -92,6 +96,7 @@ pub mod jam;
mod message;
mod node;
mod normalization;
pub mod podcast;
mod record;
mod request;
mod routing;
File diff suppressed because it is too large. Load diff
+731
View File
@@ -0,0 +1,731 @@
use super::*;
const ITUNES_NS: &str = "http://www.itunes.com/dtds/podcast-1.0.dtd";
fn rss(channel_extra: &str, items: &str) -> String {
format!(
r#"<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:itunes="{ITUNES_NS}" xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
<title>Test Cast</title>
<link>https://example.test/cast</link>
{channel_extra}
{items}
</channel>
</rss>"#
)
}
fn item(guid: &str, title: &str, date: &str) -> String {
format!(
r#"<item>
<title>{title}</title>
<guid isPermaLink="false">{guid}</guid>
<pubDate>{date}</pubDate>
<enclosure url="https://cdn.example.test/{guid}.mp3" length="1234" type="audio/mpeg"/>
</item>"#
)
}
#[test]
fn rss_with_itunes_tags_cdata_and_fallbacks() {
let xml = rss(
r#"<description><![CDATA[<p>Weekly <b>talk</b> show.</p><script>evil()</script>]]></description>
<itunes:author>Jane Host</itunes:author>
<itunes:image href="https://example.test/cover.jpg"/>
<image><url>https://example.test/small.png</url><title>ignored</title></image>"#,
r#"<item>
<title> Episode one </title>
<itunes:title>One</itunes:title>
<description>&lt;p&gt;First &amp;amp; best&lt;/p&gt;</description>
<guid>ep-1</guid>
<pubDate>Tue, 02 Jan 2024 10:00:00 GMT</pubDate>
<itunes:duration>1:02:03</itunes:duration>
<itunes:image href="https://example.test/ep1.jpg"/>
<link>https://example.test/ep1</link>
<enclosure url="https://cdn.example.test/1.mp3" length="5000" type="audio/mpeg"/>
</item>
<item>
<title></title>
<itunes:title>Only itunes title</itunes:title>
<content:encoded><![CDATA[<div>Long <i>show</i> notes</div>]]></content:encoded>
<guid>ep-2</guid>
<itunes:duration>754</itunes:duration>
<enclosure url="https://cdn.example.test/2.m4a" type="audio/x-m4a"/>
</item>
<item>
<itunes:summary>Summary only</itunes:summary>
<guid>ep-3</guid>
<enclosure url="https://cdn.example.test/3.mp3"/>
</item>"#,
);
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(feed.title, "Test Cast");
assert_eq!(feed.author.as_deref(), Some("Jane Host"));
assert_eq!(feed.description.as_deref(), Some("Weekly talk show."));
assert_eq!(feed.link.as_deref(), Some("https://example.test/cast"));
assert_eq!(
feed.image_url.as_deref(),
Some("https://example.test/cover.jpg")
);
assert_eq!(feed.episodes.len(), 3);
let first = &feed.episodes[0];
assert_eq!(first.guid.as_deref(), Some("ep-1"));
assert_eq!(first.title, "Episode one");
assert_eq!(first.description.as_deref(), Some("First & best"));
assert_eq!(first.published_at_ms, Some(1_704_189_600_000));
assert_eq!(first.duration_ms, Some(3_723_000));
assert_eq!(first.audio_url, "https://cdn.example.test/1.mp3");
assert_eq!(first.mime_type.as_deref(), Some("audio/mpeg"));
assert_eq!(first.byte_len, Some(5000));
assert_eq!(first.link.as_deref(), Some("https://example.test/ep1"));
assert_eq!(
first.image_url.as_deref(),
Some("https://example.test/ep1.jpg")
);
let second = &feed.episodes[1];
assert_eq!(second.title, "Only itunes title");
assert_eq!(second.description.as_deref(), Some("Long show notes"));
assert_eq!(second.duration_ms, Some(754_000));
assert_eq!(second.byte_len, None);
assert_eq!(second.published_at_ms, None);
let third = &feed.episodes[2];
assert_eq!(third.title, "Untitled episode");
assert_eq!(third.description.as_deref(), Some("Summary only"));
assert_eq!(third.mime_type, None);
}
#[test]
fn entities_numeric_refs_and_unknown_html_entities() {
let xml = rss(
"",
r#"<item>
<title>A&nbsp;&amp;&nbsp;B &#8212; &#x41;&#1055; &mdash; &bogus; R&amp;D &copy;</title>
<description>x &lt; y &amp;&amp; z</description>
<guid>e</guid>
<enclosure url="https://cdn.example.test/a.mp3?x=1&amp;y=2" type="audio/mpeg"/>
</item>"#,
);
let feed = parse_feed(xml.as_bytes()).unwrap();
let episode = &feed.episodes[0];
assert_eq!(episode.title, "A & B — AП — &bogus; R&D ©");
assert_eq!(episode.description.as_deref(), Some("x < y && z"));
assert_eq!(episode.audio_url, "https://cdn.example.test/a.mp3?x=1&y=2");
}
#[test]
fn atom_feed() {
let xml = r#"<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title type="text">Atom Cast</title>
<subtitle>About &lt;b&gt;things&lt;/b&gt;</subtitle>
<link rel="self" href="https://example.test/atom.xml"/>
<link rel="alternate" href="https://example.test/"/>
<logo>https://example.test/logo.png</logo>
<author><name>Atom Author</name></author>
<entry>
<title>Entry one</title>
<id>urn:uuid:1</id>
<published>2024-03-01T12:30:00+02:00</published>
<updated>2024-03-05T00:00:00Z</updated>
<summary type="html">&lt;p&gt;Entry summary&lt;/p&gt;</summary>
<link rel="alternate" href="https://example.test/e1"/>
<link rel="enclosure" href="https://cdn.example.test/e1.ogg" type="audio/ogg" length="42"/>
</entry>
<entry>
<title>Entry two</title>
<id>urn:uuid:2</id>
<updated>2024-03-06T00:00:00Z</updated>
<content type="xhtml"><div xmlns="http://www.w3.org/1999/xhtml"><p>Inline</p><p>xhtml</p></div></content>
<link rel="enclosure" href="https://cdn.example.test/e2.mp3"/>
</entry>
<entry>
<title>No audio</title>
<id>urn:uuid:3</id>
<link rel="alternate" href="https://example.test/e3"/>
</entry>
</feed>"#;
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(feed.title, "Atom Cast");
assert_eq!(feed.description.as_deref(), Some("About things"));
assert_eq!(feed.link.as_deref(), Some("https://example.test/"));
assert_eq!(
feed.image_url.as_deref(),
Some("https://example.test/logo.png")
);
assert_eq!(feed.author.as_deref(), Some("Atom Author"));
assert_eq!(feed.episodes.len(), 2);
let first = &feed.episodes[0];
assert_eq!(first.guid.as_deref(), Some("urn:uuid:1"));
assert_eq!(first.published_at_ms, Some(1_709_289_000_000));
assert_eq!(first.description.as_deref(), Some("Entry summary"));
assert_eq!(first.link.as_deref(), Some("https://example.test/e1"));
assert_eq!(first.audio_url, "https://cdn.example.test/e1.ogg");
assert_eq!(first.byte_len, Some(42));
let second = &feed.episodes[1];
assert_eq!(second.published_at_ms, Some(1_709_683_200_000));
assert_eq!(second.description.as_deref(), Some("Inline xhtml"));
}
#[test]
fn items_without_audio_are_skipped_and_media_content_is_a_fallback() {
let xml = r#"<rss xmlns:media="http://search.yahoo.com/mrss/"><channel><title>T</title>
<item><title>No enclosure</title><guid>a</guid></item>
<item><title>Image enclosure</title><guid>b</guid><enclosure url="https://x.test/b.jpg" type="image/jpeg"/></item>
<item><title>Empty url</title><guid>c</guid><enclosure url="" type="audio/mpeg"/></item>
<item><title>Media</title><guid>d</guid>
<media:content url="https://x.test/d.png" medium="image"/>
<media:group><media:content url="https://x.test/d.mp3" type="audio/mpeg" fileSize="77"/></media:group>
<media:thumbnail url="https://x.test/d-thumb.jpg"/>
</item>
<item><title>Video</title><guid>e</guid><enclosure url="https://x.test/e.mp4" type="video/mp4"/></item>
</channel></rss>"#;
let feed = parse_feed(xml.as_bytes()).unwrap();
let titles: Vec<_> = feed.episodes.iter().map(|e| e.title.as_str()).collect();
assert_eq!(titles, ["Media", "Video"]);
assert_eq!(feed.episodes[0].audio_url, "https://x.test/d.mp3");
assert_eq!(feed.episodes[0].byte_len, Some(77));
assert_eq!(
feed.episodes[0].image_url.as_deref(),
Some("https://x.test/d-thumb.jpg")
);
}
#[test]
fn duplicate_guids_keep_the_first_and_missing_guids_stay_absent() {
let items = [
item("dup", "First", "Mon, 01 Jan 2024 00:00:00 +0000"),
item("dup", "Second", "Tue, 02 Jan 2024 00:00:00 +0000"),
r#"<item><title>No guid</title><enclosure url="https://cdn.example.test/noguid.mp3" type="audio/mpeg"/></item>"#.to_string(),
r#"<item><title>Blank guid</title><guid> </guid><enclosure url="https://cdn.example.test/blank.mp3" type="audio/mpeg"/></item>"#.to_string(),
]
.concat();
let feed = parse_feed(rss("", &items).as_bytes()).unwrap();
let pairs: Vec<_> = feed
.episodes
.iter()
.map(|e| (e.guid.as_deref(), e.title.as_str(), e.audio_url.as_str()))
.collect();
assert_eq!(
pairs,
[
(Some("dup"), "First", "https://cdn.example.test/dup.mp3"),
(None, "No guid", "https://cdn.example.test/noguid.mp3"),
(None, "Blank guid", "https://cdn.example.test/blank.mp3"),
]
);
}
#[test]
fn durations() {
let cases = [
("01:02:03", Some(3723.0)),
("1:02:03", Some(3723.0)),
("62:03", Some(3723.0)),
("3723", Some(3723.0)),
("3723.5", Some(3723.5)),
("00:00:01.25", Some(1.25)),
(" 45:00 ", Some(2700.0)),
("0", None),
("00:00:00", None),
("", None),
("abc", None),
("1:2:3:4", None),
("-5", None),
("1e3", None),
("NaN", None),
("12:", None),
];
for (input, expected) in cases {
assert_eq!(parse_duration(input), expected, "{input:?}");
}
}
#[test]
fn dates() {
let reference = Some(1_704_189_600); // 2024-01-02 10:00:00 UTC
let cases = [
("Tue, 02 Jan 2024 10:00:00 GMT", reference),
("Tue, 02 Jan 2024 10:00:00 +0000", reference),
("02 Jan 2024 10:00:00 UT", reference),
("Tue, 2 Jan 2024 13:00:00 +0300", reference),
("Tue, 02 Jan 2024 05:00:00 EST", reference),
("Tue, 02 Jan 2024 02:00:00 PST", reference),
("Tuesday, 02 January 2024 10:00 GMT", reference),
("Tue, 02 Jan 24 10:00:00 GMT", reference),
("Tue, 02 Jan 2024 13:00:00 +03:00", reference),
("Tue, 02 Jan 2024 10:00:00", reference),
("Вт, 02 янв 2024 13:00:00 +0300", reference),
("2024-01-02T10:00:00Z", reference),
("2024-01-02T13:00:00+03:00", reference),
("2024-01-02T10:00:00.123Z", reference),
("2024-01-02 10:00:00", reference),
("2024-01-02", Some(1_704_153_600)),
("not a date", None),
("Tue, 31 Feb 2024 10:00:00 GMT", None),
("Tue, 02 Foo 2024 10:00:00 GMT", None),
("2024-13-02T10:00:00Z", None),
("", None),
];
for (input, expected) in cases {
assert_eq!(parse_date(input), expected, "{input:?}");
}
}
#[test]
fn non_ascii_zones_do_not_panic() {
for input in [
"Mon, 07 Nov 2022 10:00:00 +й0",
"Mon, 07 Nov 2022 10:00:00 -йй",
"Mon, 07 Nov 2022 10:00+й0",
"Mon, 07 Nov 2022 10:00:00 +0й00",
"2022-11-07T10:00:00+0й0",
"2022-11-07T10:00:00+й",
"2022-11-07T10:00:00-€:00",
"2022-11-07T10:00:00+日本",
] {
assert_eq!(parse_date(input), None, "{input:?}");
}
// Exhaustive-ish: every short suffix of mixed ASCII / multi-byte
// characters after a sign.
let alphabet = ['0', '9', ':', 'й', '€', '日', '+', '-'];
for a in alphabet {
for b in alphabet {
for c in alphabet {
for prefix in ["Mon, 07 Nov 2022 10:00:00 ", "2022-11-07T10:00:00"] {
for sign in ['+', '-'] {
let _ = parse_date(&format!("{prefix}{sign}{a}{b}{c}"));
}
}
}
}
}
// In a feed, the bad date only loses the date.
let xml = rss("", &item("z", "Zone", "Mon, 07 Nov 2022 10:00:00 +й0"));
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(feed.episodes.len(), 1);
assert_eq!(feed.episodes[0].published_at_ms, None);
}
#[test]
fn media_urls_are_resolved_and_must_be_http() {
let xml = rss(
r#"<itunes:image href="/img/cover.jpg"/>"#,
r#"<item><guid>rel</guid><title>Relative</title>
<itunes:image href="ep.png"/>
<enclosure url="/audio/1.mp3" type="audio/mpeg"/></item>
<item><guid>proto</guid><title>Protocol relative</title>
<enclosure url="//cdn.example.test/2.mp3" type="audio/mpeg"/></item>
<item><guid>file</guid><title>Local file</title>
<enclosure url="file:///etc/passwd" type="audio/mpeg"/></item>
<item><guid>ftp</guid><title>FTP</title>
<enclosure url="ftp://example.test/3.mp3" type="audio/mpeg"/></item>
<item><guid>fallback</guid><title>Media fallback</title>
<enclosure url="file:///tmp/x.mp3" type="audio/mpeg"/>
<media:content xmlns:media="http://search.yahoo.com/mrss/" url="https://cdn.example.test/4.mp3" type="audio/mpeg"/></item>
<item><guid>abs</guid><title>Absolute</title>
<enclosure url="HTTPS://cdn.example.test/5.mp3" type="audio/mpeg"/></item>"#,
);
let feed =
parse_feed_with_base(xml.as_bytes(), Some("https://host.test/feeds/rss.xml")).unwrap();
let urls: Vec<(&str, &str)> = feed
.episodes
.iter()
.map(|e| (e.guid.as_deref().unwrap_or_default(), e.audio_url.as_str()))
.collect();
assert_eq!(
urls,
[
("rel", "https://host.test/audio/1.mp3"),
("proto", "https://cdn.example.test/2.mp3"),
("fallback", "https://cdn.example.test/4.mp3"),
("abs", "HTTPS://cdn.example.test/5.mp3"),
]
);
assert_eq!(
feed.image_url.as_deref(),
Some("https://host.test/img/cover.jpg")
);
assert_eq!(
feed.episodes[0].image_url.as_deref(),
Some("https://host.test/feeds/ep.png")
);
for episode in &feed.episodes {
assert!(is_remote_url(&episode.audio_url), "{}", episode.audio_url);
}
// Without a base, relative media cannot be resolved and is dropped;
// protocol-relative URLs default to https.
let feed = parse_feed(xml.as_bytes()).unwrap();
let guids: Vec<&str> = feed
.episodes
.iter()
.map(|e| e.guid.as_deref().unwrap_or_default())
.collect();
assert_eq!(guids, ["proto", "fallback", "abs"]);
assert_eq!(feed.image_url, None);
}
#[test]
fn rss_with_an_unrelated_default_namespace() {
for ns in ["http://example.test/x", "http://www.w3.org/2005/Atom"] {
let xml = format!(
r#"<rss version="2.0" xmlns="{ns}" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>T</title>
<atom:link href="https://a.test/feed" rel="self"/>
<item><guid>1</guid><title>One</title><enclosure url="https://a.test/1.mp3" type="audio/mpeg"/></item>
</channel></rss>"#
);
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(feed.title, "T", "{ns}");
assert_eq!(feed.episodes.len(), 1, "{ns}");
assert_eq!(feed.episodes[0].title, "One");
}
}
#[test]
fn html_pages_and_garbage_are_rejected() {
let html = br#"<!DOCTYPE html><html><head><title>Vivat</title></head><body><p>Hi &nbsp; there</p></body></html>"#;
let err = parse_feed(html).unwrap_err().to_string();
assert!(err.contains("not an RSS or Atom feed"), "{err}");
for garbage in [
&b""[..],
b"hello world",
b"\x00\xff\xfe garbage",
b"{\"json\": true}",
b"<",
b"<<<>>>",
] {
let err = parse_feed(garbage).unwrap_err().to_string();
assert!(
err.contains("not an RSS or Atom feed"),
"{garbage:?}: {err}"
);
}
}
#[test]
fn truncated_and_malformed_xml_never_panics() {
let items: String = (0..5)
.map(|i| item(&format!("g{i}"), &format!("T {i} &amp; &nbsp;"), "bad date"))
.collect();
let xml = rss(
"<description><![CDATA[cdata]]></description><itunes:image href=\"x\"/>",
&items,
);
let bytes = xml.as_bytes();
for len in 0..=bytes.len() {
let _ = parse_feed(&bytes[..len]);
}
// Mismatched end tags are tolerated.
let sloppy = r#"<rss><channel><title>S</title><item><title>A</b></title><guid>1</guid><enclosure url="https://x.test/1.mp3"/></item></channel></rss>"#;
let feed = parse_feed(sloppy.as_bytes()).unwrap();
assert_eq!(feed.episodes.len(), 1);
assert_eq!(feed.episodes[0].title, "A");
// Truncation after complete items keeps them.
let third = xml.find("<title>T 3").unwrap();
let cut = &bytes[..xml[..third].rfind("<item>").unwrap()];
let feed = parse_feed(cut).unwrap();
assert_eq!(feed.episodes.len(), 3);
}
#[test]
fn deep_nesting_does_not_overflow() {
let depth = 100_000;
let mut xml = String::from("<rss><channel><title>Deep</title>");
xml.push_str(&"<x>".repeat(depth));
xml.push_str(&"</x>".repeat(depth));
xml.push_str(&item("after", "After", ""));
xml.push_str("</channel></rss>");
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(feed.episodes.len(), 1);
}
#[test]
fn cyrillic_titles_in_utf8_and_windows_1251() {
let xml = rss(
"<itunes:author>Иван Петров</itunes:author>",
&item(
"1",
"Выпуск №1: «Привет»",
"Mon, 01 Jan 2024 00:00:00 +0300",
),
)
.replace("Test Cast", "Подкаст");
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(feed.title, "Подкаст");
assert_eq!(feed.author.as_deref(), Some("Иван Петров"));
assert_eq!(feed.episodes[0].title, "Выпуск №1: «Привет»");
let cp1251 = xml.replace("encoding=\"UTF-8\"", "encoding=\"windows-1251\"");
let (encoded, _, unmappable) = encoding_rs::WINDOWS_1251.encode(&cp1251);
assert!(!unmappable);
let feed = parse_feed(&encoded).unwrap();
assert_eq!(feed.title, "Подкаст");
assert_eq!(feed.episodes[0].title, "Выпуск №1: «Привет»");
// Invalid UTF-8 without a declaration is decoded lossily.
let mut broken = xml.into_bytes();
let at = broken
.windows(7)
.position(|w| w == b"Test Ca")
.unwrap_or(10);
broken.insert(at, 0xff);
assert!(parse_feed(&broken).is_ok());
}
#[test]
fn namespace_prefixes_are_matched_tolerantly() {
// Undeclared `itunes:` prefix and a custom prefix bound to the
// iTunes namespace.
let xml = format!(
r#"<rss><channel><title>P</title><itunes:author>Undeclared</itunes:author>
<item xmlns:it="{ITUNES_NS}"><guid>1</guid><it:title>Custom prefix</it:title><it:duration>90</it:duration>
<enclosure url="https://x.test/1.mp3" type="audio/mpeg"/></item></channel></rss>"#
);
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(feed.author.as_deref(), Some("Undeclared"));
assert_eq!(feed.episodes[0].title, "Custom prefix");
assert_eq!(feed.episodes[0].duration_ms, Some(90_000));
}
#[test]
fn long_descriptions_are_truncated() {
let long = "word ".repeat(2000);
let text = html_to_text(&long, 4000);
assert!(text.chars().count() <= 4000);
assert!(text.ends_with('…'));
let xml = rss(&format!("<description>{long}</description>"), "");
let feed = parse_feed(xml.as_bytes()).unwrap();
assert!(feed.description.unwrap().chars().count() <= 4000);
}
#[test]
fn itunes_season_episode_and_type() {
let xml = rss(
"",
r#"<item><guid>a</guid><title>A</title>
<itunes:season>2</itunes:season><itunes:episode> 7 </itunes:episode>
<itunes:episodeType>Trailer</itunes:episodeType>
<enclosure url="https://cdn.example.test/a.mp3" type="audio/mpeg"/></item>
<item><guid>b</guid><title>B</title>
<itunes:season>two</itunes:season><itunes:episode>-1</itunes:episode>
<itunes:episodeType>mystery</itunes:episodeType>
<enclosure url="https://cdn.example.test/b.mp3" type="audio/mpeg"/></item>
<item><guid>c</guid><title>C</title><itunes:episodeType>bonus</itunes:episodeType>
<itunes:episode>99999999999</itunes:episode>
<enclosure url="https://cdn.example.test/c.mp3" type="audio/mpeg"/></item>"#,
);
let feed = parse_feed(xml.as_bytes()).unwrap();
let a = &feed.episodes[0];
assert_eq!((a.season, a.episode_number), (Some(2), Some(7)));
assert_eq!(a.episode_type, EpisodeType::Trailer);
let b = &feed.episodes[1];
assert_eq!((b.season, b.episode_number), (None, None));
assert_eq!(b.episode_type, EpisodeType::Full);
let c = &feed.episodes[2];
assert_eq!((c.season, c.episode_number), (None, None));
assert_eq!(c.episode_type, EpisodeType::Bonus);
}
#[test]
fn guid_permalink_flag() {
let xml = rss(
"",
r#"<item><guid isPermaLink="false">x</guid><enclosure url="https://c.test/x.mp3"/></item>
<item><guid>https://e.test/y</guid><enclosure url="https://c.test/y.mp3"/></item>
<item><guid isPermaLink=" FALSE ">z</guid><enclosure url="https://c.test/z.mp3"/></item>
<item><enclosure url="https://c.test/n.mp3"/></item>"#,
);
let feed = parse_feed(xml.as_bytes()).unwrap();
let flags: Vec<Option<bool>> = feed.episodes.iter().map(|e| e.guid_is_permalink).collect();
assert_eq!(flags, [Some(false), Some(true), Some(false), None]);
}
#[test]
fn new_feed_url_is_reported() {
let xml = rss(
"<itunes:new-feed-url> https://new.example.test/feed?x=1 </itunes:new-feed-url>",
"",
);
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(
feed.new_feed_url.as_deref(),
Some("https://new.example.test/feed?x=1")
);
let alias = FeedUrl::parse(feed.new_feed_url.as_deref().unwrap()).unwrap();
assert_eq!(alias.as_str(), "https://new.example.test/feed?x=1");
}
#[test]
fn oversized_guid_is_treated_as_absent() {
let long = "g".repeat(MAX_GUID_BYTES + 1);
let xml = rss(
"",
&format!(r#"<item><guid>{long}</guid><enclosure url="https://c.test/1.mp3"/></item>"#),
);
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(feed.episodes[0].guid, None);
}
#[test]
fn media_urls_with_spaces_are_percent_encoded() {
let xml = rss(
"",
r#"<item><guid>s</guid><enclosure url="https://c.test/my file.mp3" type="audio/mpeg"/></item>"#,
);
let feed = parse_feed(xml.as_bytes()).unwrap();
assert_eq!(feed.episodes[0].audio_url, "https://c.test/my%20file.mp3");
}
#[test]
fn parsed_feed_maps_to_valid_contract_metadata() {
let base = FeedUrl::parse("https://example.test/feed.xml").unwrap();
let items = [
item("g1", "One", "Tue, 02 Jan 2024 10:00:00 GMT"),
r#"<item><title>No guid</title><link>https://example.test/no-guid</link>
<pubDate>Wed, 03 Jan 2024 10:00:00 GMT</pubDate>
<enclosure url="https://cdn.example.test/ng.mp3" type="audio/mpeg"/></item>"#
.to_string(),
// Same page as the channel link: not identifying.
r#"<item><title>Home link</title><link>https://example.test/cast</link>
<enclosure url="https://cdn.example.test/home.mp3" type="audio/mpeg"/></item>"#
.to_string(),
]
.concat();
let feed = parse_feed_with_base(rss("", &items).as_bytes(), Some(base.as_str())).unwrap();
let podcast_id = PodcastId::from_feed_url(&base);
let ids = feed.resolve_episode_ids(&podcast_id, &[]);
assert_eq!(ids[0], EpisodeId::from_guid(&podcast_id, "g1"));
assert_eq!(
ids[1],
Some(EpisodeId::derive(
&podcast_id,
&crate::podcast::EpisodeKey::link("https://example.test/no-guid").unwrap()
))
);
assert_eq!(
ids[2],
Some(EpisodeId::derive(
&podcast_id,
&crate::podcast::EpisodeKey::enclosure("https://cdn.example.test/home.mp3").unwrap()
))
);
let podcast = feed.to_podcast_meta(base.clone());
podcast.validate().unwrap();
for (episode, id) in feed.episodes.iter().zip(ids) {
let meta = episode.to_meta(&podcast_id, id.unwrap());
meta.validate().unwrap();
meta.to_ref(&podcast).validate().unwrap();
}
// A later refresh with a changed title and audio URL keeps every id,
// using the stored hints of the previous refresh.
let known: Vec<KnownEpisode> = feed
.episodes
.iter()
.zip(feed.resolve_episode_ids(&podcast_id, &[]))
.map(|(episode, id)| KnownEpisode {
episode_id: id.unwrap(),
hints: episode.identity_hints(),
})
.collect();
let refreshed = items
.replace("Home link", "Home link (edited)")
.replace("cdn.example.test/ng.mp3", "cdn2.example.test/ng.mp3");
let feed2 = parse_feed_with_base(rss("", &refreshed).as_bytes(), Some(base.as_str())).unwrap();
let ids2 = feed2.resolve_episode_ids(&podcast_id, &known);
let expected: Vec<Option<EpisodeId>> =
known.iter().map(|k| Some(k.episode_id.clone())).collect();
assert_eq!(ids2, expected);
}
#[test]
fn hostile_sizes_stay_bounded() {
let huge_title = "t".repeat(MAX_CAPTURE_BYTES * 2);
let xml = rss(
"",
&format!(
r#"<item><guid>h</guid><title>{huge_title}</title><enclosure url="https://c.test/h.mp3"/></item>"#
),
);
let feed = parse_feed(xml.as_bytes()).unwrap();
assert!(feed.episodes[0].title.chars().count() <= MAX_TITLE_CHARS);
let duration = rss(
"",
r#"<item><guid>d</guid><itunes:duration>99999999999999999999</itunes:duration><enclosure url="https://c.test/d.mp3"/></item>"#,
);
assert_eq!(
parse_feed(duration.as_bytes()).unwrap().episodes[0].duration_ms,
None
);
}
#[test]
fn text_full_of_unclosed_angle_brackets_is_linear() {
// '<a' repeated up to the capture cap, with no '>' anywhere: the old
// scanner searched for '>' to the end of the text at every '<'.
let body = "&lt;a".repeat(MAX_CAPTURE_BYTES / 2);
let xml = rss(
"",
&format!(
r#"<item><guid>lt</guid><title>{body}</title><description>{body}</description><enclosure url="https://c.test/lt.mp3"/></item>"#
),
);
let started = std::time::Instant::now();
let feed = parse_feed(xml.as_bytes()).unwrap();
assert!(
started.elapsed() < std::time::Duration::from_secs(2),
"took {:?}",
started.elapsed()
);
let episode = &feed.episodes[0];
assert!(episode.title.starts_with("<a<a"));
assert!(episode.title.chars().count() <= MAX_TITLE_CHARS);
assert_eq!(
html_to_text("a < b and <b>bold</b> <c", 100),
"a < b and bold <c"
);
}
#[test]
fn links_are_identity_material_and_kept_verbatim() {
// Entities are decoded once by the XML reader and never again; tag-like
// text, inner whitespace and long links are not rewritten by the display
// cleaner.
let long_path = "p".repeat(1_500);
let items = format!(
r#"<item><title>A</title><link> https://e.test/a?x=1&amp;amp;y=&lt;b&gt; </link><enclosure url="https://c.test/a.mp3"/></item>
<item><title>B</title><link>https://e.test/{long_path}</link><enclosure url="https://c.test/b.mp3"/></item>"#
);
let feed = parse_feed(rss("", &items).as_bytes()).unwrap();
assert_eq!(
feed.episodes[0].link.as_deref(),
Some("https://e.test/a?x=1&amp;y=<b>")
);
let long = format!("https://e.test/{long_path}");
assert_eq!(feed.episodes[1].link.as_deref(), Some(long.as_str()));
let podcast_id =
PodcastId::from_feed_url(&FeedUrl::parse("https://example.test/feed").unwrap());
let ids = feed.resolve_episode_ids(&podcast_id, &[]);
assert_eq!(
ids[1],
Some(EpisodeId::derive(
&podcast_id,
&crate::podcast::EpisodeKey::link(&long).unwrap()
))
);
let atom = r#"<?xml version="1.0"?><feed xmlns="http://www.w3.org/2005/Atom"><title>T</title>
<link href=" https://e.test/home?a=1&amp;b=2 "/>
<entry><title>E</title><link rel="alternate" href="https://e.test/e?q=&lt;x&gt;"/>
<link rel="enclosure" href="https://c.test/e.mp3" type="audio/mpeg"/></entry></feed>"#;
let feed = parse_feed(atom.as_bytes()).unwrap();
assert_eq!(feed.link.as_deref(), Some("https://e.test/home?a=1&b=2"));
assert_eq!(
feed.episodes[0].link.as_deref(),
Some("https://e.test/e?q=<x>")
);
}
+530
View File
@@ -0,0 +1,530 @@
//! Feed URL canonicalization and feed aliases.
use std::collections::BTreeSet;
use std::fmt;
use serde::{Deserialize, Serialize};
use super::{MAX_FEED_ALIASES, MAX_URL_BYTES, PodcastError, PodcastId, PodcastResult};
/// Canonical podcast feed URL.
///
/// Canonicalization ([`FeedUrl::parse`]) is deliberately small so that every
/// client, whatever its URL library, produces the same bytes:
///
/// 1. Only `http://` and `https://` (any case) are accepted; the scheme is
/// lowercased.
/// 2. Userinfo (`user:pass@`), whitespace, control characters and `\` are
/// rejected.
/// 3. The host is lowercased. Non-ASCII hosts are rejected: pass the IDNA
/// (`xn--`) form, as produced by WHATWG URL parsers. IPv6 literals are
/// accepted in brackets.
/// 4. An empty port and the scheme's default port (80, 443) are dropped;
/// other ports lose leading zeros.
/// 5. The fragment is dropped.
/// 6. An empty path becomes `/`. The path and the **whole** query string are
/// otherwise kept byte-for-byte, including parameter order, case and
/// percent-encoding; only non-ASCII bytes are percent-encoded (`%XX`,
/// uppercase hex) so the result is ASCII.
///
/// The canonical form is at most [`MAX_URL_BYTES`] bytes and canonicalizing it
/// again is the identity. Deserialization accepts only canonical strings.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
#[serde(try_from = "String", into = "String")]
pub struct FeedUrl(String);
impl FeedUrl {
/// Canonicalizes an absolute `http(s)` feed URL.
pub fn parse(input: &str) -> PodcastResult<Self> {
canonicalize(input).map(Self)
}
/// Canonicalizes a URL typed or pasted by a user.
///
/// Surrounding whitespace is trimmed, `feed:`, `podcast:`, `pcast:` and
/// `itpc:` prefixes map to `https://` (or to the embedded `http(s)` URL),
/// and input without a scheme defaults to `https://`. `http:` or `https:`
/// not followed by `//` (for example `https:/host/feed`) is rejected.
pub fn from_user_input(input: &str) -> PodcastResult<Self> {
let trimmed = input.trim();
if trimmed.is_empty() {
return Err(invalid("feed URL is empty"));
}
let lower = trimmed.to_ascii_lowercase();
if is_http_scheme_typo(&lower) {
return Err(invalid("feed URL must start with http:// or https://"));
}
let mut candidate = trimmed.to_string();
for scheme in ["feed:", "podcast:", "pcast:", "itpc:"] {
if lower.starts_with(scheme) {
let rest = &trimmed[scheme.len()..];
let rest_lower = rest.to_ascii_lowercase();
if is_http_scheme_typo(&rest_lower) {
return Err(invalid("feed URL must start with http:// or https://"));
}
candidate =
if rest_lower.starts_with("http://") || rest_lower.starts_with("https://") {
rest.to_string()
} else {
format!("https://{}", rest.trim_start_matches('/'))
};
break;
}
}
if !has_scheme(&candidate) {
candidate = format!("https://{candidate}");
}
Self::parse(&candidate)
}
/// Canonical string form.
pub fn as_str(&self) -> &str {
&self.0
}
}
impl fmt::Display for FeedUrl {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.0)
}
}
impl TryFrom<String> for FeedUrl {
type Error = PodcastError;
fn try_from(value: String) -> PodcastResult<Self> {
if canonicalize(&value)? == value {
Ok(Self(value))
} else {
Err(invalid("feed URL is not in canonical form"))
}
}
}
impl From<FeedUrl> for String {
fn from(value: FeedUrl) -> String {
value.0
}
}
impl AsRef<str> for FeedUrl {
fn as_ref(&self) -> &str {
&self.0
}
}
fn invalid(reason: &str) -> PodcastError {
PodcastError::InvalidFeedUrl(reason.to_string())
}
/// `http:` or `https:` not followed by `//`, such as `https:/host/feed`.
fn is_http_scheme_typo(lower: &str) -> bool {
["http:", "https:"]
.iter()
.any(|scheme| lower.starts_with(scheme) && !lower[scheme.len()..].starts_with("//"))
}
fn has_scheme(text: &str) -> bool {
let Some((scheme, rest)) = text.split_once(':') else {
return false;
};
let mut chars = scheme.chars();
chars.next().is_some_and(|c| c.is_ascii_alphabetic())
&& chars.all(|c| c.is_ascii_alphanumeric() || matches!(c, '+' | '-' | '.'))
&& rest.starts_with("//")
}
fn canonicalize(input: &str) -> PodcastResult<String> {
if input.len() > MAX_URL_BYTES * 3 {
return Err(invalid("feed URL is too long"));
}
if input.chars().any(|c| c.is_whitespace() || c.is_control()) {
return Err(invalid("feed URL must not contain whitespace"));
}
if input.contains('\\') {
return Err(invalid("feed URL must not contain '\\'"));
}
let (scheme, rest) = input
.split_once("://")
.ok_or_else(|| invalid("feed URL must start with http:// or https://"))?;
let (scheme, default_port) = if scheme.eq_ignore_ascii_case("http") {
("http", 80)
} else if scheme.eq_ignore_ascii_case("https") {
("https", 443)
} else {
return Err(invalid("only http and https feed URLs are supported"));
};
let authority_end = rest.find(['/', '?', '#']).unwrap_or(rest.len());
let (authority, tail) = rest.split_at(authority_end);
if authority.contains('@') {
return Err(invalid("feed URL must not contain credentials"));
}
let (host, port) = split_host_port(authority)?;
let host = canonical_host(host)?;
let port = match port {
None | Some("") => None,
Some(port) => {
if port.len() > 5 || !port.bytes().all(|b| b.is_ascii_digit()) {
return Err(invalid("invalid feed URL port"));
}
match port.parse::<u32>() {
Ok(value) if (1..=65_535).contains(&value) => {
(value != default_port).then_some(value)
}
_ => return Err(invalid("invalid feed URL port")),
}
}
};
let tail = tail.split('#').next().unwrap_or_default();
let (path, query) = match tail.find('?') {
Some(at) => tail.split_at(at),
None => (tail, ""),
};
let mut out = String::with_capacity(input.len() + 8);
out.push_str(scheme);
out.push_str("://");
out.push_str(&host);
if let Some(port) = port {
out.push(':');
out.push_str(&port.to_string());
}
if path.is_empty() {
out.push('/');
} else {
push_ascii(&mut out, path);
}
push_ascii(&mut out, query);
if out.len() > MAX_URL_BYTES {
return Err(invalid("feed URL is too long"));
}
Ok(out)
}
fn split_host_port(authority: &str) -> PodcastResult<(&str, Option<&str>)> {
if authority.starts_with('[') {
let close = authority
.find(']')
.ok_or_else(|| invalid("unterminated IPv6 host"))?;
let (host, after) = authority.split_at(close + 1);
return match after {
"" => Ok((host, None)),
_ => after
.strip_prefix(':')
.map(|port| (host, Some(port)))
.ok_or_else(|| invalid("invalid feed URL host")),
};
}
Ok(match authority.rsplit_once(':') {
Some((host, port)) => (host, Some(port)),
None => (authority, None),
})
}
fn canonical_host(host: &str) -> PodcastResult<String> {
if host.is_empty() {
return Err(invalid("feed URL has no host"));
}
if !host.is_ascii() {
return Err(invalid(
"feed URL host must be ASCII (use the IDNA xn-- form)",
));
}
if let Some(inner) = host.strip_prefix('[').and_then(|h| h.strip_suffix(']')) {
if inner.is_empty()
|| !inner
.bytes()
.all(|b| b.is_ascii_hexdigit() || matches!(b, b':' | b'.'))
{
return Err(invalid("invalid IPv6 host"));
}
return Ok(host.to_ascii_lowercase());
}
if !host
.bytes()
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'-' | b'.' | b'_'))
|| !host.bytes().any(|b| b.is_ascii_alphanumeric())
{
return Err(invalid("invalid feed URL host"));
}
Ok(host.to_ascii_lowercase())
}
fn push_ascii(out: &mut String, value: &str) {
for ch in value.chars() {
if ch.is_ascii() {
out.push(ch);
} else {
let mut buf = [0u8; 4];
for byte in ch.encode_utf8(&mut buf).bytes() {
out.push_str(&format!("%{byte:02X}"));
}
}
}
}
/// Resolves the podcast a canonical feed URL belongs to.
///
/// `known` yields `(podcast_id, url)` pairs of every URL already associated
/// with a podcast: the URL of its first subscription, its current fetch URL
/// and its feed aliases ([`super::SubscriptionRecord::feed_urls`]; the bounded
/// alias set alone can miss the current fetch URL). When `url` matches, the existing id is returned (the
/// smallest id if, after a concurrent offline subscription, several podcasts
/// share the URL). Otherwise a new id is derived from `url`.
///
/// Re-subscribing through a redirect target or an `itunes:new-feed-url`
/// therefore reuses the original podcast, its progress and its history.
pub fn resolve_podcast_id<'a>(
url: &FeedUrl,
known: impl IntoIterator<Item = (&'a PodcastId, &'a FeedUrl)>,
) -> PodcastId {
known
.into_iter()
.filter(|(_, alias)| *alias == url)
.map(|(id, _)| id)
.min()
.cloned()
.unwrap_or_else(|| PodcastId::from_feed_url(url))
}
/// Merges two feed alias sets of `podcast` as a bounded grow-only set.
///
/// The result is sorted and duplicate-free. The URL from which `podcast` was
/// derived is always kept; of the other URLs the [`MAX_FEED_ALIASES`]
/// lexicographically smallest are kept. Taking the smallest `k` elements of a
/// union is commutative, associative and idempotent, so devices converge
/// regardless of delivery order. Other URLs, including a podcast's current
/// fetch URL, can fall outside the bound; the materialized subscription keeps
/// its `podcast.feed_url` separately.
pub fn merge_feed_aliases(podcast: &PodcastId, a: &[FeedUrl], b: &[FeedUrl]) -> Vec<FeedUrl> {
let mut origin = None;
let mut rest = BTreeSet::new();
for url in a.iter().chain(b) {
if PodcastId::from_feed_url(url) == *podcast {
origin = Some(url.clone());
} else {
rest.insert(url.clone());
}
}
let mut out: Vec<FeedUrl> = rest.into_iter().take(MAX_FEED_ALIASES).collect();
if let Some(origin) = origin {
out.push(origin);
out.sort();
}
out
}
#[cfg(test)]
mod tests {
use super::*;
fn canon(input: &str) -> String {
FeedUrl::parse(input).unwrap().as_str().to_string()
}
#[test]
fn canonicalization_rules() {
let cases = [
(
"https://example.com/feed.xml",
"https://example.com/feed.xml",
),
(
"HTTPS://Example.COM/Feed.XML",
"https://example.com/Feed.XML",
),
("http://example.com:80/rss", "http://example.com/rss"),
("https://example.com:443/rss", "https://example.com/rss"),
("http://example.com:443/rss", "http://example.com:443/rss"),
(
"https://example.com:08080/rss",
"https://example.com:8080/rss",
),
("https://example.com:/rss", "https://example.com/rss"),
("https://example.com", "https://example.com/"),
("https://example.com?x=1", "https://example.com/?x=1"),
("https://example.com/rss#latest", "https://example.com/rss"),
("https://example.com/rss?#x", "https://example.com/rss?"),
(
"https://example.com/a/../b/./c?b=2&a=1&A=%2f&token=XyZ",
"https://example.com/a/../b/./c?b=2&a=1&A=%2f&token=XyZ",
),
(
"https://example.com/rss?url=https://other.test/x?y#frag",
"https://example.com/rss?url=https://other.test/x?y",
),
(
"https://example.com/подкаст?q=ё",
"https://example.com/%D0%BF%D0%BE%D0%B4%D0%BA%D0%B0%D1%81%D1%82?q=%D1%91",
),
(
"https://[2001:DB8::1]:443/feed",
"https://[2001:db8::1]/feed",
),
("https://[::1]:8443/feed", "https://[::1]:8443/feed"),
(
"https://xn--d1acufc.xn--p1ai/rss",
"https://xn--d1acufc.xn--p1ai/rss",
),
(
"https://my_host.example./rss",
"https://my_host.example./rss",
),
];
for (input, expected) in cases {
assert_eq!(canon(input), expected, "{input:?}");
// Idempotent.
assert_eq!(canon(expected), expected, "{expected:?}");
}
}
#[test]
fn rejected_urls() {
for bad in [
"",
"example.com/feed",
"ftp://example.com/feed",
"file:///etc/passwd",
"javascript:alert(1)",
"https://",
"https:///path",
"https://user:pass@example.com/feed",
"https://token@example.com/feed",
"https://example.com/my feed",
" https://example.com/feed",
"https://example.com/feed\n",
"https://example.com\\feed",
"https://пример.рф/rss",
"https://example.com:0/rss",
"https://example.com:65536/rss",
"https://example.com:12a/rss",
"https://[::1/feed",
"https://[zz]/feed",
"https://exa%6dple.com/feed",
"https://.../feed",
] {
assert!(FeedUrl::parse(bad).is_err(), "{bad:?}");
}
}
#[test]
fn length_limit_is_inclusive() {
let prefix = "https://a.test/";
let at = format!("{prefix}{}", "x".repeat(MAX_URL_BYTES - prefix.len()));
assert_eq!(canon(&at).len(), MAX_URL_BYTES);
let over = format!("{at}x");
assert!(FeedUrl::parse(&over).is_err());
}
#[test]
fn user_input_forms() {
let ok = [
(
"https://podster.test/rss.xml?pid=778",
"https://podster.test/rss.xml?pid=778",
),
(" http://a.test/feed ", "http://a.test/feed"),
("a.test/feed.xml", "https://a.test/feed.xml"),
("feed://a.test/rss", "https://a.test/rss"),
("feed:https://a.test/rss", "https://a.test/rss"),
("podcast://a.test/rss", "https://a.test/rss"),
("itpc://a.test/rss", "https://a.test/rss"),
("HTTPS://A.TEST/Feed", "https://a.test/Feed"),
("a.test:8080/rss", "https://a.test:8080/rss"),
(
"example.test/rss?src=https://other.test",
"https://example.test/rss?src=https://other.test",
),
(
"localhost:8080/feed?u=http://x.test/",
"https://localhost:8080/feed?u=http://x.test/",
),
];
for (input, expected) in ok {
assert_eq!(
FeedUrl::from_user_input(input).unwrap().as_str(),
expected,
"{input:?}"
);
}
for bad in [
"",
" ",
"ftp://a.test/feed",
"https://a.test/my feed",
"https://",
"file:///etc/passwd",
"javascript:alert(1)",
// One-slash typos must not become a host named "https".
"https:/a.test/feed",
"http:/a.test/feed",
"HTTPS:a.test/feed",
"https:///a.test/feed",
"feed:https:/a.test/feed",
] {
assert!(FeedUrl::from_user_input(bad).is_err(), "{bad:?}");
}
}
#[test]
fn deserialization_requires_canonical_form() {
let url = FeedUrl::parse("https://example.com/rss").unwrap();
let json = serde_json::to_string(&url).unwrap();
assert_eq!(serde_json::from_str::<FeedUrl>(&json).unwrap(), url);
assert!(serde_json::from_str::<FeedUrl>("\"HTTPS://example.com/rss\"").is_err());
assert!(serde_json::from_str::<FeedUrl>("\"https://example.com/rss#x\"").is_err());
assert!(serde_json::from_str::<FeedUrl>("\"ftp://example.com/rss\"").is_err());
}
#[test]
fn query_parameters_change_identity() {
let a = PodcastId::from_feed_url(&FeedUrl::parse("https://a.test/rss?id=1").unwrap());
let b = PodcastId::from_feed_url(&FeedUrl::parse("https://a.test/rss?id=2").unwrap());
let c = PodcastId::from_feed_url(&FeedUrl::parse("HTTPS://A.test:443/rss?id=1#x").unwrap());
assert_ne!(a, b);
assert_eq!(a, c);
}
#[test]
fn aliases_resolve_to_the_original_podcast() {
let original = FeedUrl::parse("http://old.test/feed").unwrap();
let moved = FeedUrl::parse("https://new.test/feed").unwrap();
let id = PodcastId::from_feed_url(&original);
let known = [(&id, &original), (&id, &moved)];
assert_eq!(resolve_podcast_id(&moved, known), id);
assert_eq!(resolve_podcast_id(&original, known), id);
let unrelated = FeedUrl::parse("https://other.test/feed").unwrap();
assert_eq!(
resolve_podcast_id(&unrelated, known),
PodcastId::from_feed_url(&unrelated)
);
// Ambiguity after concurrent offline subscriptions resolves to the
// smallest id, independent of iteration order.
let other = PodcastId::from_feed_url(&moved);
let ambiguous = [(&id, &moved), (&other, &moved)];
let reversed = [(&other, &moved), (&id, &moved)];
let expected = (&id).min(&other).clone();
assert_eq!(resolve_podcast_id(&moved, ambiguous), expected);
assert_eq!(resolve_podcast_id(&moved, reversed), expected);
}
#[test]
fn alias_merge_is_a_bounded_semilattice() {
let origin = FeedUrl::parse("https://zzz.test/origin").unwrap();
let id = PodcastId::from_feed_url(&origin);
let urls: Vec<FeedUrl> = (0..(MAX_FEED_ALIASES + 4))
.map(|i| FeedUrl::parse(&format!("https://a.test/{i:02}")).unwrap())
.collect();
let a: Vec<FeedUrl> = urls[..5].to_vec();
let b: Vec<FeedUrl> = urls[3..].iter().cloned().chain([origin.clone()]).collect();
let c: Vec<FeedUrl> = urls[7..].to_vec();
let ab = merge_feed_aliases(&id, &a, &b);
assert_eq!(ab, merge_feed_aliases(&id, &b, &a));
assert_eq!(ab, merge_feed_aliases(&id, &ab, &ab));
let left = merge_feed_aliases(&id, &ab, &c);
let right = merge_feed_aliases(&id, &a, &merge_feed_aliases(&id, &b, &c));
assert_eq!(left, right);
assert_eq!(left.len(), MAX_FEED_ALIASES + 1);
assert!(left.contains(&origin));
assert!(left.windows(2).all(|w| w[0] < w[1]));
}
}
+343
View File
@@ -0,0 +1,343 @@
//! Cross-device podcast and episode identifiers.
use std::fmt;
use std::str::FromStr;
use serde::{Deserialize, Serialize};
use super::{FeedUrl, MAX_GUID_BYTES, PodcastError, PodcastResult};
/// BLAKE3 derivation domain of [`PodcastId`]. Changing it changes every id.
pub const PODCAST_ID_DOMAIN: &str = "furumi-podcast:podcast:v1";
/// BLAKE3 derivation domain of [`EpisodeId`]. Changing it changes every id.
pub const EPISODE_ID_DOMAIN: &str = "furumi-podcast:episode:v1";
/// String prefix of a [`PodcastId`].
pub const PODCAST_ID_PREFIX: &str = "pod1:";
/// String prefix of an [`EpisodeId`].
pub const EPISODE_ID_PREFIX: &str = "ep1:";
/// Length of the base32 body: 32 bytes, RFC 4648 lowercase, no padding.
const DIGEST_CHARS: usize = 52;
fn digest(domain: &str, parts: &[&[u8]]) -> [u8; 32] {
let mut hasher = blake3::Hasher::new();
hasher.update(domain.as_bytes());
hasher.update(&[0]);
for part in parts {
hasher.update(part);
}
*hasher.finalize().as_bytes()
}
fn encode(prefix: &str, digest: &[u8; 32]) -> String {
let mut body = data_encoding::BASE32_NOPAD.encode(digest);
body.make_ascii_lowercase();
format!("{prefix}{body}")
}
fn decode(prefix: &str, value: &str) -> Option<[u8; 32]> {
let body = value.strip_prefix(prefix)?;
if body.len() != DIGEST_CHARS
|| !body
.bytes()
.all(|b| b.is_ascii_lowercase() || (b'2'..=b'7').contains(&b))
{
return None;
}
let bytes = data_encoding::BASE32_NOPAD
.decode(body.to_ascii_uppercase().as_bytes())
.ok()?;
let digest: [u8; 32] = bytes.try_into().ok()?;
// Reject non-canonical trailing bits so every id has one spelling.
(encode(prefix, &digest) == value).then_some(digest)
}
macro_rules! string_id {
($name:ident, $prefix:expr, $kind:literal) => {
impl $name {
/// Parses and validates the canonical string form.
pub fn parse(value: &str) -> PodcastResult<Self> {
decode($prefix, value)
.map(|_| Self(value.to_string()))
.ok_or(PodcastError::InvalidId { kind: $kind })
}
/// Canonical string form.
pub fn as_str(&self) -> &str {
&self.0
}
/// The 32-byte BLAKE3 digest encoded in the id.
pub fn digest(&self) -> [u8; 32] {
decode($prefix, &self.0).expect("validated on construction")
}
}
impl fmt::Display for $name {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.0)
}
}
impl FromStr for $name {
type Err = PodcastError;
fn from_str(value: &str) -> PodcastResult<Self> {
Self::parse(value)
}
}
impl TryFrom<String> for $name {
type Error = PodcastError;
fn try_from(value: String) -> PodcastResult<Self> {
decode($prefix, &value)
.map(|_| Self(value))
.ok_or(PodcastError::InvalidId { kind: $kind })
}
}
impl From<$name> for String {
fn from(value: $name) -> String {
value.0
}
}
impl AsRef<str> for $name {
fn as_ref(&self) -> &str {
&self.0
}
}
};
}
/// Stable cross-device podcast identity: `pod1:<52 lowercase base32 chars>`.
///
/// Derived as `BLAKE3("furumi-podcast:podcast:v1" || 0x00 || canonical_feed_url)`
/// from the URL used by the **first** subscription. The id never changes when
/// the feed moves; new URLs are recorded as aliases (see
/// [`super::resolve_podcast_id`]). Deserialization validates the format.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
#[serde(try_from = "String", into = "String")]
pub struct PodcastId(String);
string_id!(PodcastId, PODCAST_ID_PREFIX, "podcast");
impl PodcastId {
/// Derives the id of a podcast first subscribed through `url`.
pub fn from_feed_url(url: &FeedUrl) -> Self {
Self(encode(
PODCAST_ID_PREFIX,
&digest(PODCAST_ID_DOMAIN, &[url.as_str().as_bytes()]),
))
}
}
/// Stable cross-device episode identity: `ep1:<52 lowercase base32 chars>`.
///
/// Derived as `BLAKE3("furumi-podcast:episode:v1" || 0x00 || podcast_id || key)`
/// where `key` is the material of an [`EpisodeKey`]. Title, artwork or audio
/// URL changes never change the id of an existing episode.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
#[serde(try_from = "String", into = "String")]
pub struct EpisodeId(String);
string_id!(EpisodeId, EPISODE_ID_PREFIX, "episode");
impl EpisodeId {
/// Derives an episode id from a key inside `podcast`.
pub fn derive(podcast: &PodcastId, key: &EpisodeKey) -> Self {
let material = key.material();
Self(encode(
EPISODE_ID_PREFIX,
&digest(
EPISODE_ID_DOMAIN,
&[podcast.as_str().as_bytes(), material.as_bytes()],
),
))
}
/// Derives the id of an episode with an RSS GUID (or Atom `<id>`).
///
/// Returns `None` when the GUID is blank or longer than [`MAX_GUID_BYTES`].
pub fn from_guid(podcast: &PodcastId, guid: &str) -> Option<Self> {
EpisodeKey::guid(guid).map(|key| Self::derive(podcast, &key))
}
}
/// Key material from which an [`EpisodeId`] is derived.
///
/// | Variant | Material |
/// | --- | --- |
/// | `Guid` | `guid:` + GUID trimmed of surrounding whitespace |
/// | `Link` | `link:` + `<link>` trimmed |
/// | `DateTitle` | `date-title:` + unix seconds + `\0` + [`super::normalize_episode_title`] |
/// | `Enclosure` | `enclosure:` + enclosure URL trimmed |
///
/// GUIDs and URLs are otherwise byte-for-byte: GUIDs are case-sensitive.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum EpisodeKey {
/// RSS `<guid>` or Atom `<id>`.
Guid(String),
/// Item `<link>`.
Link(String),
/// Publication time (whole unix seconds) and normalized title.
DateTitle {
/// Publication time in unix seconds.
published_at_secs: i64,
/// Title normalized by [`super::normalize_episode_title`].
normalized_title: String,
},
/// Enclosure (audio) URL.
Enclosure(String),
}
impl EpisodeKey {
/// GUID key; `None` for a blank or oversized GUID.
pub fn guid(guid: &str) -> Option<Self> {
let guid = guid.trim();
(!guid.is_empty() && guid.len() <= MAX_GUID_BYTES).then(|| Self::Guid(guid.to_string()))
}
/// Link key; `None` for a blank or oversized link.
pub fn link(link: &str) -> Option<Self> {
let link = link.trim();
(!link.is_empty() && link.len() <= super::MAX_URL_BYTES)
.then(|| Self::Link(link.to_string()))
}
/// Date + title key; `None` when the title normalizes to nothing.
pub fn date_title(published_at_ms: i64, title: &str) -> Option<Self> {
let normalized_title = super::normalize_episode_title(title);
(!normalized_title.is_empty()).then(|| Self::DateTitle {
published_at_secs: published_at_ms.div_euclid(1_000),
normalized_title,
})
}
/// Enclosure key; `None` for a blank or oversized URL.
pub fn enclosure(url: &str) -> Option<Self> {
let url = url.trim();
(!url.is_empty() && url.len() <= super::MAX_URL_BYTES)
.then(|| Self::Enclosure(url.to_string()))
}
/// Exact bytes hashed after the podcast id.
pub fn material(&self) -> String {
match self {
Self::Guid(guid) => format!("guid:{guid}"),
Self::Link(link) => format!("link:{link}"),
Self::DateTitle {
published_at_secs,
normalized_title,
} => format!("date-title:{published_at_secs}\0{normalized_title}"),
Self::Enclosure(url) => format!("enclosure:{url}"),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn podcast() -> PodcastId {
PodcastId::from_feed_url(&FeedUrl::parse("https://example.test/feed.xml").unwrap())
}
#[test]
fn podcast_id_is_domain_separated_blake3() {
let url = FeedUrl::parse("https://example.test/feed.xml").unwrap();
let mut input = b"furumi-podcast:podcast:v1\0".to_vec();
input.extend_from_slice(url.as_str().as_bytes());
let expected = blake3::hash(&input);
let id = PodcastId::from_feed_url(&url);
assert_eq!(&id.digest(), expected.as_bytes());
assert_eq!(id.as_str().len(), PODCAST_ID_PREFIX.len() + DIGEST_CHARS);
assert!(id.as_str()[5..].bytes().all(|b| !b.is_ascii_uppercase()));
}
#[test]
fn podcast_id_golden_value() {
// Golden vector: changing it breaks every stored and synced id.
let id = podcast();
let mut input = b"furumi-podcast:podcast:v1\0".to_vec();
input.extend_from_slice(b"https://example.test/feed.xml");
let mut body = data_encoding::BASE32_NOPAD.encode(blake3::hash(&input).as_bytes());
body.make_ascii_lowercase();
assert_eq!(id.as_str(), format!("pod1:{body}"));
}
#[test]
fn episode_id_follows_the_plan_formula() {
let podcast = podcast();
let id = EpisodeId::from_guid(&podcast, " abc-123 ").unwrap();
let mut input = b"furumi-podcast:episode:v1\0".to_vec();
input.extend_from_slice(podcast.as_str().as_bytes());
input.extend_from_slice(b"guid:abc-123");
assert_eq!(&id.digest(), blake3::hash(&input).as_bytes());
// GUIDs are case-sensitive and podcast-scoped.
assert_ne!(EpisodeId::from_guid(&podcast, "ABC-123"), Some(id.clone()));
let other = PodcastId::from_feed_url(&FeedUrl::parse("https://other.test/").unwrap());
assert_ne!(EpisodeId::from_guid(&other, "abc-123"), Some(id));
}
#[test]
fn keys_are_distinct_per_kind() {
let podcast = podcast();
let value = "https://example.test/e1";
let link = EpisodeId::derive(&podcast, &EpisodeKey::link(value).unwrap());
let guid = EpisodeId::derive(&podcast, &EpisodeKey::guid(value).unwrap());
let enclosure = EpisodeId::derive(&podcast, &EpisodeKey::enclosure(value).unwrap());
assert_ne!(link, guid);
assert_ne!(link, enclosure);
assert_ne!(guid, enclosure);
}
#[test]
fn blank_and_oversized_keys_are_rejected() {
assert!(EpisodeKey::guid(" ").is_none());
assert!(EpisodeKey::guid(&"g".repeat(MAX_GUID_BYTES)).is_some());
assert!(EpisodeKey::guid(&"g".repeat(MAX_GUID_BYTES + 1)).is_none());
assert!(EpisodeKey::date_title(0, " -- ").is_none());
assert!(EpisodeKey::link("").is_none());
}
#[test]
fn date_title_uses_whole_seconds_and_normalized_title() {
let a = EpisodeKey::date_title(1_700_000_000_123, "Hello, World!").unwrap();
let b = EpisodeKey::date_title(1_700_000_000_999, "hello world").unwrap();
assert_eq!(a, b);
let c = EpisodeKey::date_title(-1, "x").unwrap();
assert_eq!(
c,
EpisodeKey::DateTitle {
published_at_secs: -1,
normalized_title: "x".to_string()
}
);
}
#[test]
fn parsing_validates_and_round_trips() {
let id = podcast();
assert_eq!(PodcastId::parse(id.as_str()).unwrap(), id);
let json = serde_json::to_string(&id).unwrap();
assert_eq!(serde_json::from_str::<PodcastId>(&json).unwrap(), id);
let upper = id.as_str().to_ascii_uppercase().replace("POD1:", "pod1:");
assert!(PodcastId::parse(&upper).is_err());
assert!(PodcastId::parse(&id.as_str()[..id.as_str().len() - 1]).is_err());
assert!(PodcastId::parse(&format!("{id}a")).is_err());
assert!(PodcastId::parse("pod1:").is_err());
assert!(PodcastId::parse("").is_err());
assert!(EpisodeId::parse(id.as_str()).is_err());
assert!(serde_json::from_str::<PodcastId>("\"pod1:xyz\"").is_err());
// Non-canonical trailing bits: last char encodes 1 data bit + 4 zero bits.
let mut chars: Vec<char> = id.as_str().chars().collect();
let last = chars.len() - 1;
chars[last] = if chars[last] == 'b' { 'c' } else { 'b' };
let tampered: String = chars.into_iter().collect();
assert!(PodcastId::parse(&tampered).is_err());
}
}
+604
View File
@@ -0,0 +1,604 @@
//! Episode identity resolution, including the documented no-GUID rule.
use std::collections::{BTreeMap, BTreeSet};
use serde::{Deserialize, Serialize};
use super::{EpisodeId, EpisodeKey, PodcastId};
/// Normalizes an episode title for the `(date, title)` identity key.
///
/// This is [`crate::normalize_name`]: NFKC, lowercase, non-alphanumeric runs
/// collapsed to one space, trimmed. It is part of the episode-id derivation.
pub fn normalize_episode_title(title: &str) -> String {
crate::normalize_name(title)
}
/// Fields of a feed item (or of a stored episode) that identify it.
///
/// The parser in `podcast::feed` produces these hints; stores keep them for
/// known episodes so that a later refresh can recognize items without GUID.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct EpisodeIdentityHints {
/// RSS `<guid>` / Atom `<id>`, when present.
#[serde(default)]
pub guid: Option<String>,
/// Enclosure (audio) URL.
#[serde(default)]
pub enclosure_url: Option<String>,
/// Item `<link>`.
#[serde(default)]
pub link: Option<String>,
/// Publication time in unix milliseconds.
#[serde(default)]
pub published_at_ms: Option<i64>,
/// Item title.
#[serde(default)]
pub title: Option<String>,
}
impl EpisodeIdentityHints {
fn guid_key(&self) -> Option<EpisodeKey> {
self.guid.as_deref().and_then(EpisodeKey::guid)
}
fn enclosure(&self) -> Option<&str> {
trimmed(self.enclosure_url.as_deref())
}
fn link(&self) -> Option<&str> {
trimmed(self.link.as_deref())
}
fn date_title(&self) -> Option<(i64, String)> {
let published = self.published_at_ms?.div_euclid(1_000);
let title = normalize_episode_title(self.title.as_deref()?);
(!title.is_empty()).then_some((published, title))
}
}
/// One matching stage of the no-GUID rule: does `known` match `item`?
type StageMatcher<'f> = &'f dyn Fn(&Usable<'_>, &EpisodeIdentityHints) -> bool;
struct Usable<'a> {
link: Option<&'a str>,
enclosure: Option<&'a str>,
date_title: Option<(i64, String)>,
}
fn trimmed(value: Option<&str>) -> Option<&str> {
value.map(str::trim).filter(|value| !value.is_empty())
}
/// An episode already stored for the podcast, with the hints it was last seen with.
///
/// Include episodes learned through sync (progress operations, queue items)
/// as well as episodes from earlier RSS refreshes.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct KnownEpisode {
/// Existing stable id.
pub episode_id: EpisodeId,
/// Identity hints recorded for it.
pub hints: EpisodeIdentityHints,
}
/// Resolves the id of a single feed item. See [`resolve_episode_ids`].
///
/// A single item has no feed context, so its `<link>` is only used when it
/// differs from `channel_link`. Prefer [`resolve_episode_ids`] for a whole
/// feed refresh.
pub fn resolve_episode_id(
podcast: &PodcastId,
channel_link: Option<&str>,
item: &EpisodeIdentityHints,
known: &[KnownEpisode],
) -> Option<EpisodeId> {
resolve_episode_ids(podcast, channel_link, std::slice::from_ref(item), known)
.pop()
.flatten()
}
/// Resolves stable ids for the items of one feed document.
///
/// Returns one entry per item, in order; `None` marks an item that cannot be
/// identified (no usable key) or that duplicates an earlier item of the same
/// document. The rule (normative, see `PODCASTS.md`):
///
/// 1. An item with a GUID gets `EpisodeId::from_guid`. A GUID never matches
/// a different known episode, so the id is independent of local history.
/// 2. Items without a GUID then look for known episodes in three stages run
/// over the whole document: same enclosure URL, then same `<link>`, then
/// same publication second and [`normalize_episode_title`]. A stage
/// assigns matches for all remaining items before the next stage starts;
/// among candidates the smallest id is reused. A known episode already
/// claimed (by a GUID item or an earlier stage) is not a candidate, so the
/// result does not depend on item order.
/// 3. Otherwise a new id is derived from the first available key: `<link>`,
/// then `(date, title)`, then enclosure URL.
///
/// A `<link>` is only identifying when it is unique within the document and
/// differs from the channel link; many feeds repeat the show homepage in every
/// item. The same applies to enclosure URLs and `(date, title)` pairs shared
/// by several GUID-less items.
pub fn resolve_episode_ids(
podcast: &PodcastId,
channel_link: Option<&str>,
items: &[EpisodeIdentityHints],
known: &[KnownEpisode],
) -> Vec<Option<EpisodeId>> {
let channel_link = trimmed(channel_link);
let mut link_counts: BTreeMap<&str, usize> = BTreeMap::new();
let mut enclosure_counts: BTreeMap<&str, usize> = BTreeMap::new();
let mut date_title_counts: BTreeMap<(i64, String), usize> = BTreeMap::new();
for item in items {
if let Some(link) = item.link() {
*link_counts.entry(link).or_default() += 1;
}
if item.guid_key().is_none() {
if let Some(url) = item.enclosure() {
*enclosure_counts.entry(url).or_default() += 1;
}
if let Some(pair) = item.date_title() {
*date_title_counts.entry(pair).or_default() += 1;
}
}
}
let usable: Vec<Usable<'_>> = items
.iter()
.map(|item| Usable {
link: item
.link()
.filter(|link| Some(*link) != channel_link && link_counts.get(link) == Some(&1)),
enclosure: item
.enclosure()
.filter(|url| enclosure_counts.get(url) == Some(&1)),
date_title: item
.date_title()
.filter(|pair| date_title_counts.get(pair) == Some(&1)),
})
.collect();
let mut claimed: BTreeSet<EpisodeId> = BTreeSet::new();
let mut out: Vec<Option<EpisodeId>> = vec![None; items.len()];
// Pass 1: GUID items claim their derived ids first.
for (index, item) in items.iter().enumerate() {
if let Some(key) = item.guid_key() {
let id = EpisodeId::derive(podcast, &key);
if claimed.insert(id.clone()) {
out[index] = Some(id);
}
}
}
// Pass 2: GUID-less items reuse a matching known episode. Stages run
// across the whole document: every enclosure match is assigned before any
// link match, and every link match before any (date, title) match, so a
// weak match of an earlier item cannot steal the known episode that a
// later item matches strongly. Within one stage an item's key is unique in
// the document, so two items never compete for the same known episode
// and the result does not depend on item order.
let mut pending: Vec<usize> = (0..items.len())
.filter(|index| items[*index].guid_key().is_none())
.collect();
let stages: [StageMatcher<'_>; 3] = [
&|own, candidate| {
own.enclosure
.is_some_and(|url| candidate.enclosure() == Some(url))
},
&|own, candidate| own.link.is_some_and(|link| candidate.link() == Some(link)),
&|own, candidate| {
own.date_title
.as_ref()
.is_some_and(|pair| candidate.date_title().as_ref() == Some(pair))
},
];
for matches in stages {
let mut unmatched = Vec::with_capacity(pending.len());
for index in pending {
let own = &usable[index];
let reused = known
.iter()
.filter(|known| !claimed.contains(&known.episode_id) && matches(own, &known.hints))
.map(|known| &known.episode_id)
.min()
.cloned();
match reused {
Some(id) => {
claimed.insert(id.clone());
out[index] = Some(id);
}
None => unmatched.push(index),
}
}
pending = unmatched;
}
// Pass 3: derive new ids from the first available fallback key.
for index in pending {
let own = &usable[index];
let key = own
.link
.and_then(EpisodeKey::link)
.or_else(|| {
own.date_title
.as_ref()
.map(|(secs, title)| EpisodeKey::DateTitle {
published_at_secs: *secs,
normalized_title: title.clone(),
})
})
.or_else(|| own.enclosure.and_then(EpisodeKey::enclosure));
if let Some(key) = key {
let id = EpisodeId::derive(podcast, &key);
if claimed.insert(id.clone()) {
out[index] = Some(id);
}
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use crate::podcast::FeedUrl;
fn podcast() -> PodcastId {
PodcastId::from_feed_url(&FeedUrl::parse("https://example.test/feed").unwrap())
}
fn hints(
guid: Option<&str>,
enclosure: Option<&str>,
link: Option<&str>,
date: Option<i64>,
title: Option<&str>,
) -> EpisodeIdentityHints {
EpisodeIdentityHints {
guid: guid.map(str::to_string),
enclosure_url: enclosure.map(str::to_string),
link: link.map(str::to_string),
published_at_ms: date,
title: title.map(str::to_string),
}
}
fn derived(key: EpisodeKey) -> Option<EpisodeId> {
Some(EpisodeId::derive(&podcast(), &key))
}
#[test]
fn guid_wins_and_ignores_history() {
let item = hints(Some("g1"), Some("https://cdn/1.mp3"), None, None, Some("T"));
let known = [KnownEpisode {
episode_id: EpisodeId::from_guid(&podcast(), "other").unwrap(),
hints: hints(None, Some("https://cdn/1.mp3"), None, None, None),
}];
assert_eq!(
resolve_episode_id(&podcast(), None, &item, &known),
derived(EpisodeKey::guid("g1").unwrap())
);
}
#[test]
fn title_cover_and_audio_url_changes_keep_the_guid_id() {
let before = hints(
Some("g1"),
Some("https://cdn/a.mp3"),
None,
Some(1),
Some("Old"),
);
let after = hints(
Some("g1"),
Some("https://cdn/b.mp3"),
None,
Some(2),
Some("New"),
);
assert_eq!(
resolve_episode_id(&podcast(), None, &before, &[]),
resolve_episode_id(&podcast(), None, &after, &[])
);
}
#[test]
fn fallback_key_order_for_new_items() {
let p = podcast();
let with_link = hints(
None,
Some("https://cdn/1.mp3"),
Some("https://e/1"),
Some(5_000),
Some("A"),
);
assert_eq!(
resolve_episode_id(&p, None, &with_link, &[]),
derived(EpisodeKey::link("https://e/1").unwrap())
);
let date_title = hints(
None,
Some("https://cdn/1.mp3"),
None,
Some(5_000),
Some("A"),
);
assert_eq!(
resolve_episode_id(&p, None, &date_title, &[]),
derived(EpisodeKey::date_title(5_000, "A").unwrap())
);
let enclosure_only = hints(None, Some("https://cdn/1.mp3"), None, None, Some("A"));
assert_eq!(
resolve_episode_id(&p, None, &enclosure_only, &[]),
derived(EpisodeKey::enclosure("https://cdn/1.mp3").unwrap())
);
assert_eq!(
resolve_episode_id(&p, None, &EpisodeIdentityHints::default(), &[]),
None
);
}
#[test]
fn guidless_episode_keeps_its_id_after_title_and_url_changes() {
let p = podcast();
// First seen: no link, so the id comes from (date, title).
let first = hints(
None,
Some("https://cdn/a.mp3"),
None,
Some(9_000),
Some("Pilot"),
);
let id = resolve_episode_id(&p, None, &first, &[]).unwrap();
let known = [KnownEpisode {
episode_id: id.clone(),
hints: first.clone(),
}];
// Title changed: matched by enclosure.
let retitled = hints(
None,
Some("https://cdn/a.mp3"),
None,
Some(9_000),
Some("Pilot (remastered)"),
);
assert_eq!(
resolve_episode_id(&p, None, &retitled, &known),
Some(id.clone())
);
// Audio URL changed: matched by (date, title).
let moved = hints(
None,
Some("https://cdn2/a.mp3"),
None,
Some(9_999),
Some("PILOT!"),
);
assert_eq!(
resolve_episode_id(&p, None, &moved, &known),
Some(id.clone())
);
// Link appeared later: the enclosure still matches first.
let linked = hints(
None,
Some("https://cdn/a.mp3"),
Some("https://e/pilot"),
None,
None,
);
assert_eq!(resolve_episode_id(&p, None, &linked, &known), Some(id));
}
#[test]
fn shared_channel_links_are_not_identifying() {
let p = podcast();
let home = "https://show.test/";
let items = [
hints(None, Some("https://cdn/1.mp3"), Some(home), None, None),
hints(None, Some("https://cdn/2.mp3"), Some(home), None, None),
];
let ids = resolve_episode_ids(&p, Some(home), &items, &[]);
assert_eq!(
ids[0],
derived(EpisodeKey::enclosure("https://cdn/1.mp3").unwrap())
);
assert_eq!(
ids[1],
derived(EpisodeKey::enclosure("https://cdn/2.mp3").unwrap())
);
// The same holds without the channel link: the link repeats.
assert_eq!(resolve_episode_ids(&p, None, &items, &[]), ids);
// A single item whose link equals the channel link.
assert_eq!(resolve_episode_id(&p, Some(home), &items[0], &[]), ids[0]);
}
#[test]
fn duplicates_in_one_document_are_dropped() {
let p = podcast();
let items = [
hints(Some("g"), Some("https://cdn/1.mp3"), None, None, None),
hints(Some("g"), Some("https://cdn/2.mp3"), None, None, None),
hints(None, Some("https://cdn/3.mp3"), None, None, None),
hints(None, Some("https://cdn/3.mp3"), None, None, None),
];
let ids = resolve_episode_ids(&p, None, &items, &[]);
assert!(ids[0].is_some());
assert_eq!(ids[1], None);
// The shared enclosure is not identifying: both GUID-less items have
// no usable key.
assert_eq!(ids[2], None);
assert_eq!(ids[3], None);
}
#[test]
fn a_known_episode_is_claimed_once() {
let p = podcast();
let known_id = derived(EpisodeKey::enclosure("https://cdn/old.mp3").unwrap()).unwrap();
let known = [KnownEpisode {
episode_id: known_id.clone(),
hints: hints(
None,
Some("https://cdn/old.mp3"),
None,
Some(1_000),
Some("Same"),
),
}];
// Two new items both match by (date, title) only if unique; here
// they share it, so neither matches; the enclosure one matches.
let items = [
hints(
None,
Some("https://cdn/x.mp3"),
None,
Some(1_000),
Some("Same"),
),
hints(
None,
Some("https://cdn/old.mp3"),
None,
Some(1_000),
Some("Same"),
),
];
let ids = resolve_episode_ids(&p, None, &items, &known);
assert_eq!(ids[1], Some(known_id.clone()));
assert_eq!(
ids[0],
derived(EpisodeKey::enclosure("https://cdn/x.mp3").unwrap())
);
assert_ne!(ids[0], ids[1]);
}
#[test]
fn ambiguous_known_matches_pick_the_smallest_id_in_any_order() {
let p = podcast();
let a = EpisodeId::from_guid(&p, "a").unwrap();
let b = EpisodeId::from_guid(&p, "b").unwrap();
let same = hints(None, Some("https://cdn/1.mp3"), None, None, None);
let k1 = KnownEpisode {
episode_id: a.clone(),
hints: same.clone(),
};
let k2 = KnownEpisode {
episode_id: b.clone(),
hints: same.clone(),
};
let expected = Some(a.clone().min(b.clone()));
assert_eq!(
resolve_episode_id(&p, None, &same, &[k1.clone(), k2.clone()]),
expected
);
assert_eq!(resolve_episode_id(&p, None, &same, &[k2, k1]), expected);
}
#[test]
fn result_is_independent_of_known_order() {
let p = podcast();
let items: Vec<EpisodeIdentityHints> = (0..6)
.map(|i| {
hints(
None,
Some(&format!("https://cdn/{i}.mp3")),
Some(&format!("https://e/{i}")),
Some(i * 1_000),
Some(&format!("Ep {i}")),
)
})
.collect();
let mut known: Vec<KnownEpisode> = items
.iter()
.enumerate()
.map(|(i, h)| KnownEpisode {
episode_id: EpisodeId::from_guid(&p, &format!("legacy-{i}")).unwrap(),
hints: h.clone(),
})
.collect();
let forward = resolve_episode_ids(&p, None, &items, &known);
known.reverse();
assert_eq!(resolve_episode_ids(&p, None, &items, &known), forward);
assert!(forward.iter().all(Option::is_some));
}
#[test]
fn strong_matches_beat_earlier_weak_matches_in_any_item_order() {
let p = podcast();
let k1 = EpisodeId::from_guid(&p, "legacy-k1").unwrap();
let known = [KnownEpisode {
episode_id: k1.clone(),
hints: hints(
None,
Some("https://cdn/e1.mp3"),
None,
Some(5_000),
Some("Same"),
),
}];
// A new item that only matches K1 by (date, title), and the real K1
// (same enclosure) renamed and re-dated by the publisher.
let weak = hints(
None,
Some("https://cdn/new.mp3"),
None,
Some(5_000),
Some("Same"),
);
let strong = hints(
None,
Some("https://cdn/e1.mp3"),
None,
Some(9_000),
Some("Renamed"),
);
let forward = resolve_episode_ids(&p, None, &[weak.clone(), strong.clone()], &known);
assert_eq!(forward[1].as_ref(), Some(&k1), "enclosure match keeps K1");
assert_ne!(forward[0].as_ref(), Some(&k1));
let reverse = resolve_episode_ids(&p, None, &[strong, weak], &known);
assert_eq!(reverse, vec![forward[1].clone(), forward[0].clone()]);
}
#[test]
fn result_is_independent_of_item_order() {
let p = podcast();
let items: Vec<EpisodeIdentityHints> = (0..5)
.map(|i| {
hints(
None,
Some(&format!("https://cdn/{i}.mp3")),
(i % 2 == 0).then(|| format!("https://e/{i}")).as_deref(),
Some((i % 3) * 1_000),
Some(&format!("Ep {}", i % 2)),
)
})
.collect();
// Known episodes whose hints overlap different items at different
// stages.
let known: Vec<KnownEpisode> = (0..5)
.map(|i| KnownEpisode {
episode_id: EpisodeId::from_guid(&p, &format!("legacy-{i}")).unwrap(),
hints: hints(
None,
Some(&format!("https://cdn/{}.mp3", (i + 2) % 7)),
Some(&format!("https://e/{}", (i + 1) % 5)),
Some((i % 3) * 1_000),
Some(&format!("Ep {}", i % 2)),
),
})
.collect();
let reference = resolve_episode_ids(&p, None, &items, &known);
let mut order: Vec<usize> = (0..items.len()).collect();
for _ in 0..20 {
order.rotate_left(1);
let middle = order.len() / 2;
order.swap(0, middle);
let shuffled: Vec<EpisodeIdentityHints> =
order.iter().map(|i| items[*i].clone()).collect();
let result = resolve_episode_ids(&p, None, &shuffled, &known);
for (position, index) in order.iter().enumerate() {
assert_eq!(result[position], reference[*index], "order {order:?}");
}
}
}
}
+493
View File
@@ -0,0 +1,493 @@
//! Podcast and episode metadata DTOs.
use serde::{Deserialize, Serialize};
use super::{
EpisodeId, EpisodeIdentityHints, FeedUrl, MAX_DESCRIPTION_CHARS, MAX_GUID_BYTES,
MAX_MIME_BYTES, MAX_TITLE_CHARS, MAX_URL_BYTES, PodcastError, PodcastId, PodcastResult,
bounded_text, check_opt_text, check_opt_url, check_text, check_url,
};
/// `itunes:episodeType`.
///
/// Unknown values deserialize (and parse) as [`EpisodeType::Full`], the
/// iTunes default, so publishers cannot break the wire format.
#[derive(
Debug, Clone, Copy, Default, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize,
)]
#[serde(rename_all = "snake_case", from = "String")]
pub enum EpisodeType {
/// Regular episode.
#[default]
Full,
/// Trailer or teaser.
Trailer,
/// Bonus content.
Bonus,
}
impl EpisodeType {
/// Parses an `itunes:episodeType` value, case-insensitively.
pub fn parse(value: &str) -> Self {
match value.trim().to_ascii_lowercase().as_str() {
"trailer" => Self::Trailer,
"bonus" => Self::Bonus,
_ => Self::Full,
}
}
}
impl From<String> for EpisodeType {
fn from(value: String) -> Self {
Self::parse(&value)
}
}
/// Channel metadata of a podcast, as last fetched from its feed.
///
/// Synced inside subscription operations so that another trusted device can
/// show and refresh the podcast without the user typing the URL again. It is
/// private user data: never publish it outside the trusted-device group.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
pub struct PodcastMeta {
/// Canonical URL the feed is currently fetched from.
pub feed_url: FeedUrl,
/// Podcast title.
pub title: String,
/// Author.
#[serde(default)]
pub author: Option<String>,
/// Plain-text description, at most [`MAX_DESCRIPTION_CHARS`].
#[serde(default)]
pub description: Option<String>,
/// Publisher web page.
#[serde(default)]
pub link: Option<String>,
/// Artwork URL.
#[serde(default)]
pub image_url: Option<String>,
}
impl PodcastMeta {
/// Validates contract bounds.
pub fn validate(&self) -> PodcastResult<()> {
check_text("podcast title", &self.title, MAX_TITLE_CHARS, true)?;
check_opt_text("podcast author", self.author.as_deref(), MAX_TITLE_CHARS)?;
check_opt_text(
"podcast description",
self.description.as_deref(),
MAX_DESCRIPTION_CHARS,
)?;
if self
.link
.as_ref()
.is_some_and(|link| link.len() > MAX_URL_BYTES)
{
return Err(PodcastError::InvalidField {
field: "podcast link",
reason: "is too long",
});
}
check_opt_url("podcast image_url", self.image_url.as_deref())
}
/// Returns a copy that fits the contract bounds.
///
/// Text is truncated with [`bounded_text`]; invalid optional URLs are
/// dropped; an empty title becomes `"Untitled podcast"`.
pub fn bounded(&self) -> Self {
Self {
feed_url: self.feed_url.clone(),
title: bounded_title(&self.title, "Untitled podcast"),
author: bounded_opt(self.author.as_deref(), MAX_TITLE_CHARS),
description: bounded_opt(self.description.as_deref(), MAX_DESCRIPTION_CHARS),
link: self
.link
.clone()
.filter(|link| link.len() <= MAX_URL_BYTES && !link.trim().is_empty()),
image_url: valid_url(self.image_url.as_deref()),
}
}
}
/// Full episode metadata, as stored by clients and refreshed from RSS.
///
/// Publisher metadata never touches personal progress or history: those are
/// separate records keyed by [`EpisodeId`].
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct EpisodeMeta {
/// Stable episode id.
pub episode_id: EpisodeId,
/// Owning podcast.
pub podcast_id: PodcastId,
/// RSS GUID / Atom id, when present.
#[serde(default)]
pub guid: Option<String>,
/// Episode title.
pub title: String,
/// Plain-text description, at most [`MAX_DESCRIPTION_CHARS`].
#[serde(default)]
pub description: Option<String>,
/// `itunes:season`. Episodes without a season form their own group.
#[serde(default)]
pub season: Option<u32>,
/// `itunes:episode`.
#[serde(default)]
pub episode_number: Option<u32>,
/// `itunes:episodeType`.
#[serde(default)]
pub episode_type: EpisodeType,
/// Publication time in unix milliseconds.
#[serde(default)]
pub published_at_ms: Option<i64>,
/// Duration announced by the publisher, in milliseconds.
#[serde(default)]
pub duration_ms: Option<i64>,
/// Absolute http(s) audio URL at the publisher.
pub audio_url: String,
/// Enclosure MIME type.
#[serde(default)]
pub mime: Option<String>,
/// Enclosure length in bytes.
#[serde(default)]
pub byte_len: Option<u64>,
/// Episode artwork URL.
#[serde(default)]
pub image_url: Option<String>,
/// Episode web page.
#[serde(default)]
pub link: Option<String>,
}
impl EpisodeMeta {
/// Validates contract bounds.
pub fn validate(&self) -> PodcastResult<()> {
if self
.guid
.as_ref()
.is_some_and(|guid| guid.len() > MAX_GUID_BYTES)
{
return Err(PodcastError::InvalidField {
field: "episode guid",
reason: "is too long",
});
}
check_text("episode title", &self.title, MAX_TITLE_CHARS, true)?;
check_opt_text(
"episode description",
self.description.as_deref(),
MAX_DESCRIPTION_CHARS,
)?;
check_duration(self.duration_ms)?;
check_url("episode audio_url", &self.audio_url)?;
check_mime(self.mime.as_deref())?;
check_opt_url("episode image_url", self.image_url.as_deref())?;
if self
.link
.as_ref()
.is_some_and(|link| link.len() > MAX_URL_BYTES)
{
return Err(PodcastError::InvalidField {
field: "episode link",
reason: "is too long",
});
}
Ok(())
}
/// Identity hints to keep for the no-GUID matching rule.
pub fn identity_hints(&self) -> EpisodeIdentityHints {
EpisodeIdentityHints {
guid: self.guid.clone(),
enclosure_url: Some(self.audio_url.clone()),
link: self.link.clone(),
published_at_ms: self.published_at_ms,
title: Some(self.title.clone()),
}
}
/// Builds the self-contained queue reference for this episode.
///
/// The episode artwork falls back to the podcast artwork.
pub fn to_ref(&self, podcast: &PodcastMeta) -> PodcastEpisodeRef {
PodcastEpisodeRef {
podcast_id: self.podcast_id.clone(),
episode_id: self.episode_id.clone(),
feed_url: podcast.feed_url.clone(),
podcast_title: bounded_title(&podcast.title, "Untitled podcast"),
episode_title: bounded_title(&self.title, "Untitled episode"),
audio_url: self.audio_url.clone(),
mime: self
.mime
.clone()
.filter(|mime| check_mime(Some(mime)).is_ok()),
duration_ms: self.duration_ms.filter(|ms| *ms > 0),
published_at_ms: self.published_at_ms,
season: self.season,
episode_number: self.episode_number,
image_url: valid_url(self.image_url.as_deref())
.or_else(|| valid_url(podcast.image_url.as_deref())),
progress_generation: None,
}
}
}
/// Self-contained, bounded reference to a podcast episode.
///
/// Carried by playback queues (`PlaybackSource::PodcastEpisode`) and by
/// progress operations, so a receiving trusted device can display, resume and
/// play the episode without being subscribed and without its own copy of the
/// feed. Apart from the optional handoff hint
/// [`progress_generation`](Self::progress_generation) it contains no personal
/// state. It is private: Jam snapshots replace it with an opaque placeholder.
///
/// If the audio URL went stale, refresh the feed at `feed_url` and look the
/// episode up by `episode_id`.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct PodcastEpisodeRef {
/// Owning podcast.
pub podcast_id: PodcastId,
/// Episode.
pub episode_id: EpisodeId,
/// Canonical feed URL used to refresh the episode.
pub feed_url: FeedUrl,
/// Podcast title for display.
pub podcast_title: String,
/// Episode title for display.
pub episode_title: String,
/// Absolute http(s) audio URL at the publisher.
pub audio_url: String,
/// Enclosure MIME type.
#[serde(default)]
pub mime: Option<String>,
/// Announced duration in milliseconds.
#[serde(default)]
pub duration_ms: Option<i64>,
/// Publication time in unix milliseconds.
#[serde(default)]
pub published_at_ms: Option<i64>,
/// `itunes:season`.
#[serde(default)]
pub season: Option<u32>,
/// `itunes:episode`.
#[serde(default)]
pub episode_number: Option<u32>,
/// Episode or podcast artwork URL.
#[serde(default)]
pub image_url: Option<String>,
/// Greatest progress generation the sender has seen for this episode.
///
/// Set on queue items of a playback state (in particular on handoff),
/// because the playback state can arrive before the sender's progress
/// operations. A device that starts playing the item writes its explicit
/// action with
/// [`ProgressRecord::explicit_generation`](super::ProgressRecord::explicit_generation)`(max(local, progress_generation))`.
/// Ignored in progress and listening operations, whose own generation is
/// authoritative. Not part of the episode identity.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub progress_generation: Option<u64>,
}
impl PodcastEpisodeRef {
/// Validates contract bounds. Receivers must drop invalid references.
pub fn validate(&self) -> PodcastResult<()> {
check_text("podcast title", &self.podcast_title, MAX_TITLE_CHARS, true)?;
check_text("episode title", &self.episode_title, MAX_TITLE_CHARS, true)?;
check_url("episode audio_url", &self.audio_url)?;
check_mime(self.mime.as_deref())?;
check_duration(self.duration_ms)?;
check_opt_url("episode image_url", self.image_url.as_deref())
}
/// Identity hints known from this reference.
///
/// Add them to the known-episode set before resolving ids from a feed so
/// a GUID-less episode learned through sync keeps its id.
pub fn identity_hints(&self) -> EpisodeIdentityHints {
EpisodeIdentityHints {
guid: None,
enclosure_url: Some(self.audio_url.clone()),
link: None,
published_at_ms: self.published_at_ms,
title: Some(self.episode_title.clone()),
}
}
}
fn check_duration(duration_ms: Option<i64>) -> PodcastResult<()> {
if duration_ms.is_some_and(|ms| ms < 0) {
return Err(PodcastError::InvalidField {
field: "duration_ms",
reason: "must not be negative",
});
}
Ok(())
}
fn check_mime(mime: Option<&str>) -> PodcastResult<()> {
let Some(mime) = mime else { return Ok(()) };
if mime.is_empty() || mime.len() > MAX_MIME_BYTES || !mime.bytes().all(|b| b.is_ascii_graphic())
{
return Err(PodcastError::InvalidField {
field: "mime",
reason: "must be a short printable ASCII token",
});
}
Ok(())
}
fn bounded_title(title: &str, fallback: &str) -> String {
let title = title.trim();
if title.is_empty() {
fallback.to_string()
} else {
bounded_text(title, MAX_TITLE_CHARS)
}
}
fn bounded_opt(value: Option<&str>, max_chars: usize) -> Option<String> {
value
.map(str::trim)
.filter(|value| !value.is_empty())
.map(|value| bounded_text(value, max_chars))
}
fn valid_url(value: Option<&str>) -> Option<String> {
value
.filter(|value| super::is_http_url(value))
.map(str::to_string)
}
#[cfg(test)]
mod tests {
use super::*;
fn podcast_meta() -> PodcastMeta {
PodcastMeta {
feed_url: FeedUrl::parse("https://example.test/feed").unwrap(),
title: "Show".to_string(),
author: Some("Host".to_string()),
description: None,
link: Some("https://example.test/".to_string()),
image_url: Some("https://example.test/cover.jpg".to_string()),
}
}
fn episode() -> EpisodeMeta {
let podcast_id = PodcastId::from_feed_url(&podcast_meta().feed_url);
EpisodeMeta {
episode_id: EpisodeId::from_guid(&podcast_id, "g1").unwrap(),
podcast_id,
guid: Some("g1".to_string()),
title: "Pilot".to_string(),
description: Some("Notes".to_string()),
season: Some(2),
episode_number: Some(7),
episode_type: EpisodeType::Bonus,
published_at_ms: Some(1_700_000_000_000),
duration_ms: Some(3_600_000),
audio_url: "https://cdn.example.test/1.mp3?sig=abc".to_string(),
mime: Some("audio/mpeg".to_string()),
byte_len: Some(1234),
image_url: None,
link: Some("https://example.test/1".to_string()),
}
}
#[test]
fn episode_type_is_tolerant() {
assert_eq!(EpisodeType::parse(" Trailer "), EpisodeType::Trailer);
assert_eq!(EpisodeType::parse("BONUS"), EpisodeType::Bonus);
assert_eq!(EpisodeType::parse("full"), EpisodeType::Full);
assert_eq!(EpisodeType::parse("teaser"), EpisodeType::Full);
assert_eq!(
serde_json::to_string(&EpisodeType::Trailer).unwrap(),
"\"trailer\""
);
assert_eq!(
serde_json::from_str::<EpisodeType>("\"something-new\"").unwrap(),
EpisodeType::Full
);
}
#[test]
fn metadata_round_trips_and_validates() {
let episode = episode();
episode.validate().unwrap();
let json = serde_json::to_string(&episode).unwrap();
assert_eq!(serde_json::from_str::<EpisodeMeta>(&json).unwrap(), episode);
podcast_meta().validate().unwrap();
let mut bad = episode.clone();
bad.audio_url = "file:///tmp/x.mp3".to_string();
assert!(bad.validate().is_err());
let mut bad = episode.clone();
bad.description = Some("x".repeat(MAX_DESCRIPTION_CHARS + 1));
assert!(bad.validate().is_err());
let mut bad = episode;
bad.mime = Some("audio/mpeg; codecs=\"mp3\"".to_string());
assert!(bad.validate().is_err());
}
#[test]
fn minimal_legacy_json_uses_defaults() {
let episode = episode();
let json = format!(
r#"{{"episode_id":"{}","podcast_id":"{}","title":"T","audio_url":"https://a.test/1.mp3"}}"#,
episode.episode_id, episode.podcast_id
);
let parsed: EpisodeMeta = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.episode_type, EpisodeType::Full);
assert_eq!(parsed.season, None);
}
#[test]
fn episode_ref_is_self_contained_and_bounded() {
let podcast = podcast_meta();
let mut episode = episode();
episode.title = "x".repeat(MAX_TITLE_CHARS + 10);
let reference = episode.to_ref(&podcast);
reference.validate().unwrap();
assert_eq!(reference.episode_title.chars().count(), MAX_TITLE_CHARS);
assert_eq!(reference.podcast_title, "Show");
assert_eq!(
reference.image_url.as_deref(),
Some("https://example.test/cover.jpg")
);
assert_eq!(reference.feed_url, podcast.feed_url);
let json = serde_json::to_string(&reference).unwrap();
assert!(!json.contains("description"));
assert_eq!(
serde_json::from_str::<PodcastEpisodeRef>(&json).unwrap(),
reference
);
let mut bad = reference.clone();
bad.episode_title = " ".to_string();
assert!(bad.validate().is_err());
let mut bad = reference.clone();
bad.audio_url = "javascript:alert(1)".to_string();
assert!(bad.validate().is_err());
let mut bad = reference;
bad.duration_ms = Some(-1);
assert!(bad.validate().is_err());
}
#[test]
fn refs_reject_invalid_ids_on_deserialization() {
let json = r#"{"podcast_id":"pod1:bad","episode_id":"ep1:bad","feed_url":"https://a.test/","podcast_title":"P","episode_title":"E","audio_url":"https://a.test/1.mp3"}"#;
assert!(serde_json::from_str::<PodcastEpisodeRef>(json).is_err());
}
#[test]
fn bounded_podcast_meta_fits_the_contract() {
let mut meta = podcast_meta();
meta.title = " ".to_string();
meta.description = Some("d".repeat(MAX_DESCRIPTION_CHARS * 2));
meta.image_url = Some("ftp://x".to_string());
let bounded = meta.bounded();
bounded.validate().unwrap();
assert_eq!(bounded.title, "Untitled podcast");
assert_eq!(bounded.image_url, None);
}
}
+310
View File
@@ -0,0 +1,310 @@
//! Shared podcast contract for Furumi clients.
//!
//! This module is pure: it has no SQLite, HTTP, clock, audio or UI
//! dependency. Every Furumi client (TUI, web, desktop) uses it so that
//! podcast identity, normalization, merge rules and wire DTOs are defined
//! exactly once. The normative description lives in the workspace
//! `PODCASTS.md`; this documentation summarizes the API.
//!
//! # Identity
//!
//! * [`FeedUrl`] is a canonical feed URL. [`FeedUrl::parse`] lowercases the
//! scheme and host, drops a default port and the fragment, and keeps the
//! path and **every** query parameter byte-for-byte (only non-ASCII bytes
//! are percent-encoded). Non-`http(s)` URLs, userinfo and whitespace are
//! rejected. [`FeedUrl::from_user_input`] additionally accepts the
//! `feed:`/`podcast:`/`pcast:`/`itpc:` forms and a missing scheme.
//! * [`PodcastId`] is `pod1:` followed by the lowercase base32 BLAKE3 hash of
//! `"furumi-podcast:podcast:v1\0"` and the canonical feed URL. It is
//! derived once, from the URL of the first subscription; later URLs
//! (redirects, `itunes:new-feed-url`) become feed aliases resolved by
//! [`resolve_podcast_id`].
//! * [`EpisodeId`] is `ep1:` followed by the lowercase base32 BLAKE3 hash of
//! `"furumi-podcast:episode:v1\0"`, the podcast id and a key. The key is
//! the RSS GUID when present. Without a GUID, [`resolve_episode_ids`] first
//! reuses the id of a known episode that matches by enclosure URL, `<link>`
//! or (date, normalized title) and only then derives a new id from the
//! first available fallback key.
//!
//! # Personal state
//!
//! * [`SubscriptionRecord`]: last-writer-wins by `(generation, hlc_ms, op_id)`
//! where `generation` is a per-podcast Lamport counter
//! ([`next_podcast_generation`]); unsubscribing is state, not a tombstone.
//! Feed aliases merge as a bounded grow-only set.
//! * [`ShareRecord`]: last-writer-wins with the same ordering; `shared = false`
//! is register state that is never compacted. See [`effective_share`] for
//! the rule that a (re)subscription is private.
//! * [`ProgressRecord`]: register ordered by
//! `(generation, hlc_ms, device_id, op_id)`. Explicit user actions bump the
//! generation to `max_seen + 1`; periodic checkpoints keep the generation of
//! their session, so a late checkpoint of an old session always loses.
//! * [`ListenSession`]: grow-only set keyed by `session_id`; one completion per
//! session, so re-delivery never double counts.
//!
//! # Queue items
//!
//! [`PodcastEpisodeRef`] is the bounded, self-contained reference carried by
//! playback queues and progress operations. A receiving device can show and
//! play the episode without being subscribed to the podcast.
//!
//! # Feed parser
//!
//! With the `podcast-feed` cargo feature, `feed` provides the shared tolerant
//! RSS 2.0 / RSS 1.0 / Atom parser that produces the identity hints consumed
//! by [`resolve_episode_ids`].
//!
//! ```
//! use music_dht::podcast::{EpisodeId, FeedUrl, PodcastId};
//!
//! let url = FeedUrl::parse("HTTPS://Example.COM:443/feed.xml?token=Ab#top").unwrap();
//! assert_eq!(url.as_str(), "https://example.com/feed.xml?token=Ab");
//! let podcast = PodcastId::from_feed_url(&url);
//! assert!(podcast.as_str().starts_with("pod1:"));
//! let episode = EpisodeId::from_guid(&podcast, "episode-42").unwrap();
//! assert!(episode.as_str().starts_with("ep1:"));
//! ```
mod feed_url;
mod id;
mod identity;
mod meta;
mod state;
#[cfg(feature = "podcast-feed")]
pub mod feed;
pub use feed_url::{FeedUrl, merge_feed_aliases, resolve_podcast_id};
pub use id::{
EPISODE_ID_DOMAIN, EPISODE_ID_PREFIX, EpisodeId, EpisodeKey, PODCAST_ID_DOMAIN,
PODCAST_ID_PREFIX, PodcastId,
};
pub use identity::{
EpisodeIdentityHints, KnownEpisode, normalize_episode_title, resolve_episode_id,
resolve_episode_ids,
};
pub use meta::{EpisodeMeta, EpisodeType, PodcastEpisodeRef, PodcastMeta};
pub use state::{
COMPLETION_TAIL_MAX_MS, ListenSession, MIN_RECORDED_EPISODE_LISTEN_MS, ProgressRecord,
ProgressStatus, ShareRecord, SubscriptionRecord, completed_listen_count, effective_share,
merge_listen_session, merge_progress, merge_share, merge_subscription, next_podcast_generation,
reaches_completion,
};
/// Maximum characters of a title (podcast, episode, author).
pub const MAX_TITLE_CHARS: usize = 1_000;
/// Maximum characters of a podcast or episode description.
pub const MAX_DESCRIPTION_CHARS: usize = 4_000;
/// Maximum bytes of any URL carried by the contract.
pub const MAX_URL_BYTES: usize = 2_048;
/// Maximum bytes of an RSS GUID. Longer GUIDs are treated as absent.
pub const MAX_GUID_BYTES: usize = 2_048;
/// Maximum bytes of a MIME type.
pub const MAX_MIME_BYTES: usize = 128;
/// Maximum bytes of a device, session or operation id.
pub const MAX_TOKEN_BYTES: usize = 256;
/// Maximum feed aliases kept per podcast, besides the URL the id was derived from.
pub const MAX_FEED_ALIASES: usize = 8;
/// Maximum subscription records in one sync snapshot.
pub const MAX_SNAPSHOT_SUBSCRIPTIONS: usize = 1_000;
/// Maximum share records in one sync snapshot.
pub const MAX_SNAPSHOT_SHARES: usize = 1_000;
/// Maximum progress registers in one sync snapshot.
pub const MAX_SNAPSHOT_PROGRESS: usize = 5_000;
/// Maximum listening sessions in one sync snapshot.
pub const MAX_SNAPSHOT_LISTEN_SESSIONS: usize = 2_000;
/// Serialized JSON budget shared by all podcast snapshot sections.
///
/// It leaves room for music state inside the 8 MiB sync line limit.
pub const MAX_PODCAST_SNAPSHOT_BYTES: usize = 4 * 1024 * 1024;
/// Errors produced by podcast contract validation and parsing.
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
#[non_exhaustive]
pub enum PodcastError {
/// A feed URL cannot be canonicalized.
#[error("invalid feed URL: {0}")]
InvalidFeedUrl(String),
/// A string is not a valid podcast or episode id.
#[error("invalid {kind} id")]
InvalidId {
/// `podcast` or `episode`.
kind: &'static str,
},
/// A field violates the contract (missing, too long, malformed).
#[error("invalid {field}: {reason}")]
InvalidField {
/// Field name.
field: &'static str,
/// Human-readable reason.
reason: &'static str,
},
/// Two records that must describe the same entity do not.
#[error("cannot merge records of different {0}")]
Mismatch(&'static str),
/// The document is not an RSS or Atom feed.
#[error("not an RSS or Atom feed{0}")]
NotAFeed(String),
/// The feed is malformed and nothing usable could be parsed.
#[error("malformed feed: {0}")]
MalformedFeed(String),
}
/// Result alias for this module.
pub type PodcastResult<T> = std::result::Result<T, PodcastError>;
/// Truncates `text` to at most `max_chars` characters, ending with `…` when cut.
///
/// Clients use it to fit publisher metadata into the contract bounds.
pub fn bounded_text(text: &str, max_chars: usize) -> String {
if text.chars().count() <= max_chars {
return text.to_string();
}
if max_chars == 0 {
return String::new();
}
let mut out: String = text.chars().take(max_chars - 1).collect();
out.push('…');
out
}
pub(crate) fn check_text(
field: &'static str,
value: &str,
max_chars: usize,
required: bool,
) -> PodcastResult<()> {
if required && value.trim().is_empty() {
return Err(PodcastError::InvalidField {
field,
reason: "must not be empty",
});
}
if value.len() > max_chars * 4 || value.chars().count() > max_chars {
return Err(PodcastError::InvalidField {
field,
reason: "is too long",
});
}
Ok(())
}
pub(crate) fn check_opt_text(
field: &'static str,
value: Option<&str>,
max_chars: usize,
) -> PodcastResult<()> {
value.map_or(Ok(()), |value| check_text(field, value, max_chars, false))
}
pub(crate) fn check_token(field: &'static str, value: &str) -> PodcastResult<()> {
if value.is_empty() {
return Err(PodcastError::InvalidField {
field,
reason: "must not be empty",
});
}
if value.len() > MAX_TOKEN_BYTES {
return Err(PodcastError::InvalidField {
field,
reason: "is too long",
});
}
if value.chars().any(|c| c.is_control()) {
return Err(PodcastError::InvalidField {
field,
reason: "contains control characters",
});
}
Ok(())
}
/// Returns whether `value` is an absolute `http`/`https` URL with a host,
/// without whitespace or control characters and within [`MAX_URL_BYTES`].
///
/// Media URLs keep the publisher's spelling; only feed URLs are canonicalized.
pub fn is_http_url(value: &str) -> bool {
if value.is_empty()
|| value.len() > MAX_URL_BYTES
|| value.chars().any(|c| c.is_whitespace() || c.is_control())
{
return false;
}
let lower_prefix = |prefix: &str| {
value.len() >= prefix.len() && value[..prefix.len()].eq_ignore_ascii_case(prefix)
};
let rest = if lower_prefix("http://") {
&value[7..]
} else if lower_prefix("https://") {
&value[8..]
} else {
return false;
};
let authority = rest.split(['/', '?', '#']).next().unwrap_or_default();
let host = authority.rsplit('@').next().unwrap_or_default();
!host.is_empty() && !host.starts_with(':')
}
pub(crate) fn check_url(field: &'static str, value: &str) -> PodcastResult<()> {
if is_http_url(value) {
Ok(())
} else {
Err(PodcastError::InvalidField {
field,
reason: "must be an absolute http(s) URL",
})
}
}
pub(crate) fn check_opt_url(field: &'static str, value: Option<&str>) -> PodcastResult<()> {
value.map_or(Ok(()), |value| check_url(field, value))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn bounded_text_cuts_on_char_boundaries() {
assert_eq!(bounded_text("hello", 5), "hello");
assert_eq!(bounded_text("hello!", 5), "hell…");
assert_eq!(bounded_text("привет", 3), "пр…");
assert_eq!(bounded_text("x", 0), "");
}
#[test]
fn text_limits_accept_the_limit_and_reject_one_more() {
let at = "я".repeat(MAX_TITLE_CHARS);
let over = "я".repeat(MAX_TITLE_CHARS + 1);
assert!(check_text("title", &at, MAX_TITLE_CHARS, true).is_ok());
assert!(check_text("title", &over, MAX_TITLE_CHARS, true).is_err());
assert!(check_text("title", " ", MAX_TITLE_CHARS, true).is_err());
assert!(check_text("title", "", MAX_TITLE_CHARS, false).is_ok());
}
#[test]
fn token_limits() {
assert!(check_token("op_id", &"a".repeat(MAX_TOKEN_BYTES)).is_ok());
assert!(check_token("op_id", &"a".repeat(MAX_TOKEN_BYTES + 1)).is_err());
assert!(check_token("op_id", "").is_err());
assert!(check_token("op_id", "a\nb").is_err());
}
#[test]
fn http_urls() {
assert!(is_http_url("https://cdn.example.test/a.mp3?x=1"));
assert!(is_http_url("HTTP://cdn.example.test"));
assert!(!is_http_url("ftp://cdn.example.test/a.mp3"));
assert!(!is_http_url("file:///etc/passwd"));
assert!(!is_http_url("https://"));
assert!(!is_http_url("https:///path"));
assert!(!is_http_url("https://a.test/my file.mp3"));
assert!(!is_http_url(""));
let long = format!("https://a.test/{}", "x".repeat(MAX_URL_BYTES));
assert!(!is_http_url(&long));
let at = format!("https://a.test/{}", "x".repeat(MAX_URL_BYTES - 15));
assert_eq!(at.len(), MAX_URL_BYTES);
assert!(is_http_url(&at));
}
}
+526
View File
@@ -0,0 +1,526 @@
//! Personal podcast state and its convergent merge rules.
//!
//! Every merge function here is commutative, associative and idempotent over
//! records of the same entity, so trusted devices that receive the same set of
//! operations converge to the same state regardless of delivery order and
//! duplication. All ties are broken by a total order over every field.
use std::cmp::Ordering;
use std::collections::BTreeSet;
use serde::{Deserialize, Serialize};
use super::{
EpisodeId, FeedUrl, MAX_FEED_ALIASES, PodcastError, PodcastId, PodcastMeta, PodcastResult,
check_token, merge_feed_aliases,
};
/// Largest tail of an episode that still counts as finished, in milliseconds.
///
/// See [`reaches_completion`].
pub const COMPLETION_TAIL_MAX_MS: i64 = 30_000;
/// Minimum actual listening time for a non-completed session to enter history.
pub const MIN_RECORDED_EPISODE_LISTEN_MS: i64 = 30_000;
/// Shared automatic completion criterion for every client.
///
/// A position completes an episode when it is within the smaller of 30 s and
/// 5 % of the duration from the end. Natural end of stream always completes,
/// whatever the announced duration. Without a known positive duration only the
/// end of stream completes.
pub fn reaches_completion(position_ms: i64, duration_ms: Option<i64>) -> bool {
let Some(duration) = duration_ms.filter(|duration| *duration > 0) else {
return false;
};
let tail = COMPLETION_TAIL_MAX_MS.min(duration / 20);
position_ms >= duration - tail
}
/// Listening status of one episode.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ProgressStatus {
/// Never started, or explicitly marked unplayed.
NotStarted,
/// Started and not finished.
InProgress,
/// Finished, automatically or by an explicit "mark played".
Completed,
}
/// Durable resume register of one episode.
///
/// The winner among records of the same episode is the greatest
/// `(generation, hlc_ms, device_id, op_id)`; remaining fields only break ties
/// between malformed duplicates. Progress is **never** the maximum position:
/// seeking backwards and "start over" are real changes.
///
/// * Explicit user actions (start of a new listening session, seek, "start
/// over", "mark played", "mark unplayed", taking over playback through a
/// handoff) use [`ProgressRecord::explicit_generation`]: one more than the
/// greatest generation the device has seen for the episode. This is a
/// Lamport clock, so wall-clock skew cannot undo an explicit action that was
/// observed. An explicit action during a session becomes the generation of
/// that session.
/// * Periodic checkpoints keep the generation of the latest explicit action of
/// their session. A late checkpoint of an older session therefore loses
/// against any newer explicit action, whatever its `hlc_ms`.
/// * Concurrent explicit actions on offline devices receive the same
/// generation and are ordered by `hlc_ms`, then `device_id`, then `op_id`.
/// * Handoff: only the receiving device writes the explicit action, with
/// `explicit_generation(max(local, carried))` where `carried` is
/// [`PodcastEpisodeRef::progress_generation`](super::PodcastEpisodeRef::progress_generation)
/// of the queue item. The sender writes a final checkpoint at its session
/// generation.
/// * A device that observes a greater generation for the episode it is
/// playing stops writing at its old generation: either playback moved away,
/// or the user keeps listening locally and the device writes its next
/// record as an explicit action.
/// * Outgoing-log coalescing may drop a pending progress operation only when
/// another pending operation of the same episode [supersedes](Self::supersedes)
/// it; dropping by sequence order could discard an explicit action.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ProgressRecord {
/// Episode the register belongs to.
pub episode_id: EpisodeId,
/// Owning podcast.
pub podcast_id: PodcastId,
/// Resume position in milliseconds.
pub position_ms: i64,
/// Listening status.
pub status: ProgressStatus,
/// Lamport generation of the latest explicit action (see type docs).
pub generation: u64,
/// Listening session (or explicit action) that wrote the record.
pub session_id: String,
/// Logical write time, normally the sync operation `hlc_ms`.
pub hlc_ms: i64,
/// Writing device, normally the sync operation origin.
pub device_id: String,
/// Writing operation id.
pub op_id: String,
}
impl ProgressRecord {
/// Generation for an explicit user action, given the greatest generation
/// already seen for the episode (`None` when none was seen).
pub fn explicit_generation(max_seen: Option<u64>) -> u64 {
max_seen.map_or(1, |generation| generation.saturating_add(1))
}
/// Primary ordering key.
pub fn order_key(&self) -> (u64, i64, &str, &str) {
(
self.generation,
self.hlc_ms,
self.device_id.as_str(),
self.op_id.as_str(),
)
}
fn total_cmp(&self, other: &Self) -> Ordering {
self.order_key()
.cmp(&other.order_key())
.then_with(|| self.position_ms.cmp(&other.position_ms))
.then_with(|| self.status.cmp(&other.status))
.then_with(|| self.session_id.cmp(&other.session_id))
.then_with(|| self.podcast_id.cmp(&other.podcast_id))
}
/// Returns whether this record wins over `other` (same episode).
pub fn supersedes(&self, other: &Self) -> bool {
self.total_cmp(other) == Ordering::Greater
}
/// Position from which ordinary Play resumes.
///
/// Completed and not-started episodes start from the beginning; use the
/// explicit "start over" action to restart an in-progress episode.
pub fn resume_position_ms(&self) -> i64 {
match self.status {
ProgressStatus::InProgress => self.position_ms,
ProgressStatus::NotStarted | ProgressStatus::Completed => 0,
}
}
/// Validates contract bounds.
pub fn validate(&self) -> PodcastResult<()> {
if self.position_ms < 0 {
return Err(PodcastError::InvalidField {
field: "position_ms",
reason: "must not be negative",
});
}
check_token("session_id", &self.session_id)?;
check_token("device_id", &self.device_id)?;
check_token("op_id", &self.op_id)
}
}
/// Merges two progress registers of the same episode.
pub fn merge_progress(
current: &ProgressRecord,
incoming: &ProgressRecord,
) -> PodcastResult<ProgressRecord> {
if current.episode_id != incoming.episode_id {
return Err(PodcastError::Mismatch("episodes"));
}
Ok(if incoming.supersedes(current) {
incoming.clone()
} else {
current.clone()
})
}
/// Personal subscription state of one podcast.
///
/// `subscribed` and `podcast` form one last-writer-wins register ordered by
/// `(generation, hlc_ms, op_id)`. `generation` is a Lamport counter shared by
/// the subscription and share registers of the podcast (see
/// [`next_podcast_generation`]), so an action taken after observing another
/// device's write wins whatever the device clocks say. Unsubscribing writes
/// `subscribed = false`: it is state, not a tombstone, is never compacted, and
/// never deletes progress or history, so subscribing again continues where
/// the user stopped.
///
/// Only explicit user actions (subscribe, unsubscribe) write this register.
/// Background RSS refreshes keep their metadata locally and must not emit a
/// subscription operation; otherwise a stale device could resurrect an
/// unsubscribed podcast. Feed aliases learned by refreshes travel in their
/// own grow-only operation.
///
/// `feed_aliases` is a bounded grow-only set merged independently of the
/// register (see [`merge_feed_aliases`]): the URL the id was derived from
/// plus the [`MAX_FEED_ALIASES`] smallest other URLs. The current
/// `podcast.feed_url` may fall outside that bound, so look podcasts up by
/// [`SubscriptionRecord::feed_urls`].
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct SubscriptionRecord {
/// Podcast.
pub podcast_id: PodcastId,
/// Whether the user is subscribed.
pub subscribed: bool,
/// Channel metadata at the time of the write.
pub podcast: PodcastMeta,
/// Known feed URLs of the podcast.
#[serde(default)]
pub feed_aliases: Vec<FeedUrl>,
/// Lamport generation shared with the share register (see type docs).
#[serde(default)]
pub generation: u64,
/// Logical write time.
pub hlc_ms: i64,
/// Writing operation id.
pub op_id: String,
}
impl SubscriptionRecord {
/// Register ordering key: `(generation, hlc_ms, op_id)`.
pub fn order_key(&self) -> (u64, i64, &str) {
(self.generation, self.hlc_ms, self.op_id.as_str())
}
fn total_cmp(&self, other: &Self) -> Ordering {
self.order_key()
.cmp(&other.order_key())
.then_with(|| self.subscribed.cmp(&other.subscribed))
.then_with(|| self.podcast.cmp(&other.podcast))
}
/// Returns whether this record's register wins over `other`'s.
pub fn supersedes(&self, other: &Self) -> bool {
self.total_cmp(other) == Ordering::Greater
}
/// Merges `podcast.feed_url` into `feed_aliases` and bounds the set.
///
/// The result is sorted and bounded by [`merge_feed_aliases`]; it may not
/// contain `podcast.feed_url` when many smaller aliases are known.
pub fn normalize(&mut self) {
self.feed_aliases = merge_feed_aliases(
&self.podcast_id,
&self.feed_aliases,
std::slice::from_ref(&self.podcast.feed_url),
);
}
/// Adds aliases learned through a `podcast_feed_aliases_added` operation.
///
/// Only the grow-only alias set changes; the register is untouched.
pub fn add_feed_aliases(&mut self, aliases: &[FeedUrl]) {
self.feed_aliases = merge_feed_aliases(&self.podcast_id, &self.feed_aliases, aliases);
}
/// Every URL that identifies this podcast: the aliases and the current
/// `podcast.feed_url`, sorted and without duplicates.
///
/// Stores pass these pairs to [`super::resolve_podcast_id`].
pub fn feed_urls(&self) -> Vec<FeedUrl> {
let mut urls: BTreeSet<FeedUrl> = self.feed_aliases.iter().cloned().collect();
urls.insert(self.podcast.feed_url.clone());
urls.into_iter().collect()
}
/// Validates contract bounds.
pub fn validate(&self) -> PodcastResult<()> {
self.podcast.validate()?;
if self.feed_aliases.len() > MAX_FEED_ALIASES + 1 {
return Err(PodcastError::InvalidField {
field: "feed_aliases",
reason: "has too many entries",
});
}
check_token("op_id", &self.op_id)
}
}
/// Merges two subscription records of the same podcast.
///
/// The result is normalized (see [`SubscriptionRecord::normalize`]).
pub fn merge_subscription(
current: &SubscriptionRecord,
incoming: &SubscriptionRecord,
) -> PodcastResult<SubscriptionRecord> {
if current.podcast_id != incoming.podcast_id {
return Err(PodcastError::Mismatch("podcasts"));
}
let mut merged = if incoming.supersedes(current) {
incoming.clone()
} else {
current.clone()
};
let mut left = current.feed_aliases.clone();
left.push(current.podcast.feed_url.clone());
let mut right = incoming.feed_aliases.clone();
right.push(incoming.podcast.feed_url.clone());
merged.feed_aliases = merge_feed_aliases(&merged.podcast_id, &left, &right);
Ok(merged)
}
/// Decision to publish a podcast card to the federation.
///
/// Last-writer-wins by `(generation, hlc_ms, op_id)`, with `generation` the
/// per-podcast Lamport counter shared with [`SubscriptionRecord`] (see
/// [`next_podcast_generation`]). `shared = false` revokes the publication.
/// It is privacy-protecting register state, **not** a tombstone: like an
/// unsubscribe it is never compacted, otherwise a stale `shared = true`
/// delivered again would republish the podcast. Unsubscribing must also write
/// `shared = false`.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ShareRecord {
/// Podcast.
pub podcast_id: PodcastId,
/// Whether the user chose to share the podcast.
pub shared: bool,
/// Lamport generation shared with the subscription register.
#[serde(default)]
pub generation: u64,
/// Logical write time.
pub hlc_ms: i64,
/// Writing operation id.
pub op_id: String,
}
impl ShareRecord {
/// Register ordering key: `(generation, hlc_ms, op_id)`.
pub fn order_key(&self) -> (u64, i64, &str) {
(self.generation, self.hlc_ms, self.op_id.as_str())
}
fn total_cmp(&self, other: &Self) -> Ordering {
self.order_key()
.cmp(&other.order_key())
.then_with(|| self.shared.cmp(&other.shared))
}
/// Returns whether this record wins over `other`.
pub fn supersedes(&self, other: &Self) -> bool {
self.total_cmp(other) == Ordering::Greater
}
/// Validates contract bounds.
pub fn validate(&self) -> PodcastResult<()> {
check_token("op_id", &self.op_id)
}
}
/// Merges two share records of the same podcast.
pub fn merge_share(current: &ShareRecord, incoming: &ShareRecord) -> PodcastResult<ShareRecord> {
if current.podcast_id != incoming.podcast_id {
return Err(PodcastError::Mismatch("podcasts"));
}
Ok(if incoming.supersedes(current) {
incoming.clone()
} else {
current.clone()
})
}
/// Generation for a new write of the subscription or share register.
///
/// One more than the greatest generation of either register of the podcast
/// (`None` when it has none), so a write made after observing another
/// device's subscription, unsubscription or share decision orders after it
/// even when that device's clock runs ahead. Concurrent writes on devices
/// that did not see each other receive the same generation and are ordered
/// by `hlc_ms`, then `op_id`.
pub fn next_podcast_generation(
subscription: Option<&SubscriptionRecord>,
share: Option<&ShareRecord>,
) -> u64 {
subscription
.map(|record| record.generation)
.into_iter()
.chain(share.map(|record| record.generation))
.max()
.map_or(1, |generation| generation.saturating_add(1))
}
/// Whether a podcast must currently be published.
///
/// True only when the user is subscribed, chose to share, and the share
/// decision orders after the latest subscription write (by
/// `(generation, hlc_ms, op_id)`). A new subscription (including
/// re-subscribing, or subscribing from someone else's public card) is
/// therefore always private until the user shares it again, and a stale
/// device cannot republish a podcast through an old share decision. A share
/// decision taken after observing the subscription always applies, because it
/// carries a greater generation.
pub fn effective_share(
subscription: Option<&SubscriptionRecord>,
share: Option<&ShareRecord>,
) -> bool {
match (subscription, share) {
(Some(subscription), Some(share)) => {
subscription.subscribed
&& share.shared
&& subscription.podcast_id == share.podcast_id
&& share.order_key() > subscription.order_key()
}
_ => false,
}
}
/// One listening session of an episode on one device.
///
/// Sessions form a grow-only set keyed by `session_id`, generated when
/// playback of the episode starts on a device. A device may re-send its
/// session with more listening time (for example after a crash); versions of
/// one session merge by [`merge_listen_session`].
///
/// * A session enters history when it completed or lasted at least
/// [`MIN_RECORDED_EPISODE_LISTEN_MS`] ([`ListenSession::should_record`]).
/// * `completed` is set when the session reached [`reaches_completion`] or the
/// natural end of the stream. Each session counts at most once, so the play
/// count of an episode is the number of distinct completed sessions
/// ([`completed_listen_count`]); duplicate delivery never double counts.
/// * "Mark played" changes [`ProgressRecord::status`] only. It does not create
/// a session and does not invent listening time.
/// * Sessions listened concurrently on offline devices are all kept.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ListenSession {
/// Unique session id, normally a UUID.
pub session_id: String,
/// Episode.
pub episode_id: EpisodeId,
/// Owning podcast.
pub podcast_id: PodcastId,
/// Device that played the session.
pub device_id: String,
/// Session start in unix milliseconds.
pub started_at_ms: i64,
/// Latest update of the session in unix milliseconds.
pub ended_at_ms: i64,
/// Position when the session started.
pub start_position_ms: i64,
/// Position when the session ended (or was last reported).
pub end_position_ms: i64,
/// Actual accumulated listening time, excluding pauses and seeks.
pub listened_ms: i64,
/// Whether the session completed the episode.
pub completed: bool,
}
impl ListenSession {
fn total_cmp(&self, other: &Self) -> Ordering {
(
self.completed,
self.listened_ms,
self.ended_at_ms,
self.end_position_ms,
self.started_at_ms,
self.start_position_ms,
)
.cmp(&(
other.completed,
other.listened_ms,
other.ended_at_ms,
other.end_position_ms,
other.started_at_ms,
other.start_position_ms,
))
}
/// Whether the session belongs in listening history.
pub fn should_record(&self) -> bool {
self.completed || self.listened_ms >= MIN_RECORDED_EPISODE_LISTEN_MS
}
/// Validates contract bounds.
pub fn validate(&self) -> PodcastResult<()> {
check_token("session_id", &self.session_id)?;
check_token("device_id", &self.device_id)?;
if self.started_at_ms < 0 || self.ended_at_ms < self.started_at_ms {
return Err(PodcastError::InvalidField {
field: "ended_at_ms",
reason: "must not precede started_at_ms",
});
}
if self.start_position_ms < 0 || self.end_position_ms < 0 || self.listened_ms < 0 {
return Err(PodcastError::InvalidField {
field: "listened_ms",
reason: "positions and listening time must not be negative",
});
}
Ok(())
}
}
/// Merges two versions of the same listening session.
///
/// The version with the greatest `(completed, listened_ms, ended_at_ms, ...)`
/// wins, so a session can only grow.
pub fn merge_listen_session(
current: &ListenSession,
incoming: &ListenSession,
) -> PodcastResult<ListenSession> {
if current.session_id != incoming.session_id {
return Err(PodcastError::Mismatch("sessions"));
}
if current.episode_id != incoming.episode_id
|| current.podcast_id != incoming.podcast_id
|| current.device_id != incoming.device_id
{
return Err(PodcastError::Mismatch("session identities"));
}
Ok(if incoming.total_cmp(current) == Ordering::Greater {
incoming.clone()
} else {
current.clone()
})
}
/// Number of distinct sessions that completed an episode.
pub fn completed_listen_count<'a>(sessions: impl IntoIterator<Item = &'a ListenSession>) -> usize {
sessions
.into_iter()
.filter(|session| session.completed)
.map(|session| session.session_id.as_str())
.collect::<BTreeSet<_>>()
.len()
}
#[cfg(test)]
#[path = "state_tests.rs"]
mod tests;
+699
View File
@@ -0,0 +1,699 @@
use std::collections::BTreeMap;
use super::*;
use crate::podcast::{EpisodeIdentityHints, EpisodeMeta, EpisodeType, resolve_episode_ids};
fn permutations<T: Clone>(items: &[T]) -> Vec<Vec<T>> {
if items.len() <= 1 {
return vec![items.to_vec()];
}
let mut out = Vec::new();
for index in 0..items.len() {
let mut rest = items.to_vec();
let head = rest.remove(index);
for mut tail in permutations(&rest) {
tail.insert(0, head.clone());
out.push(tail);
}
}
out
}
fn feed_url(path: &str) -> FeedUrl {
FeedUrl::parse(&format!("https://example.test/{path}")).unwrap()
}
fn podcast_id() -> PodcastId {
PodcastId::from_feed_url(&feed_url("feed"))
}
fn episode_id(guid: &str) -> EpisodeId {
EpisodeId::from_guid(&podcast_id(), guid).unwrap()
}
fn progress(
generation: u64,
hlc_ms: i64,
device: &str,
op: &str,
position_ms: i64,
status: ProgressStatus,
session: &str,
) -> ProgressRecord {
ProgressRecord {
episode_id: episode_id("e1"),
podcast_id: podcast_id(),
position_ms,
status,
generation,
session_id: session.to_string(),
hlc_ms,
device_id: device.to_string(),
op_id: op.to_string(),
}
}
fn fold<T: Clone>(items: &[T], merge: impl Fn(&T, &T) -> PodcastResult<T>) -> T {
let mut iter = items.iter();
let first = iter.next().expect("non-empty").clone();
iter.fold(first, |acc, item| merge(&acc, item).unwrap())
}
fn assert_converges<T: Clone + PartialEq + std::fmt::Debug>(
items: &[T],
merge: impl Fn(&T, &T) -> PodcastResult<T> + Copy,
) -> T {
let reference = fold(items, merge);
for order in permutations(items) {
assert_eq!(fold(&order, merge), reference, "order: {order:?}");
}
// Merge laws on every pair and triple.
for a in items {
assert_eq!(
merge(&merge(a, a).unwrap(), a).unwrap(),
merge(a, a).unwrap()
);
for b in items {
assert_eq!(merge(a, b).unwrap(), merge(b, a).unwrap());
for c in items {
assert_eq!(
merge(&merge(a, b).unwrap(), c).unwrap(),
merge(a, &merge(b, c).unwrap()).unwrap()
);
}
}
}
reference
}
#[test]
fn progress_converges_for_every_permutation() {
use ProgressStatus::*;
let ops = [
progress(1, 100, "dev-a", "dev-a:1", 10_000, InProgress, "s1"),
progress(1, 160, "dev-a", "dev-a:2", 70_000, InProgress, "s1"),
progress(2, 150, "dev-b", "dev-b:1", 0, InProgress, "s2"),
progress(2, 150, "dev-c", "dev-c:1", 5_000, InProgress, "s3"),
progress(2, 140, "dev-b", "dev-b:2", 0, Completed, "s2"),
// Duplicate delivery of the same operation.
progress(1, 160, "dev-a", "dev-a:2", 70_000, InProgress, "s1"),
];
let winner = assert_converges(&ops, merge_progress);
assert_eq!(winner.op_id, "dev-c:1");
}
#[test]
fn late_old_generation_checkpoint_loses() {
use ProgressStatus::*;
// Device A keeps checkpointing session s1 (generation 3) with a clock far
// ahead; device B explicitly seeks back (generation 4) with a slow clock.
let seek_back = progress(4, 1_000, "dev-b", "dev-b:9", 60_000, InProgress, "s2");
let late = progress(
3, 9_000_000, "dev-a", "dev-a:50", 3_000_000, InProgress, "s1",
);
for order in [[&seek_back, &late], [&late, &seek_back]] {
let merged = merge_progress(order[0], order[1]).unwrap();
assert_eq!(merged, seek_back);
}
assert!(seek_back.supersedes(&late));
assert!(!late.supersedes(&seek_back));
}
#[test]
fn restart_beats_later_checkpoint_of_the_old_session() {
use ProgressStatus::*;
let checkpoint = progress(2, 100, "dev-a", "dev-a:1", 2_400_000, InProgress, "s1");
let generation = ProgressRecord::explicit_generation(Some(checkpoint.generation));
assert_eq!(generation, 3);
let restart = progress(generation, 110, "dev-a", "dev-a:2", 0, InProgress, "s2");
// A checkpoint of s1 that was queued before the restart and delivered
// later, even stamped after the restart.
let late = progress(2, 120, "dev-a", "dev-a:3", 2_460_000, InProgress, "s1");
let merged = assert_converges(&[checkpoint, restart.clone(), late], merge_progress);
assert_eq!(merged, restart);
assert_eq!(merged.resume_position_ms(), 0);
}
#[test]
fn progress_is_not_the_maximum_position() {
use ProgressStatus::*;
let ahead = progress(1, 100, "dev-a", "dev-a:1", 3_000_000, InProgress, "s1");
let back = progress(2, 101, "dev-a", "dev-a:2", 60_000, InProgress, "s1");
assert_eq!(merge_progress(&ahead, &back).unwrap().position_ms, 60_000);
}
#[test]
fn concurrent_offline_sessions_converge_and_keep_both_sessions() {
use ProgressStatus::*;
// Both devices last saw generation 5 and start listening offline.
let generation = ProgressRecord::explicit_generation(Some(5));
let a_start = progress(
generation, 200, "dev-a", "dev-a:1", 300_000, InProgress, "sa",
);
let a_cp = progress(
generation, 300, "dev-a", "dev-a:2", 900_000, InProgress, "sa",
);
let b_start = progress(
generation, 210, "dev-b", "dev-b:1", 300_000, InProgress, "sb",
);
let b_cp = progress(
generation, 280, "dev-b", "dev-b:2", 1_500_000, InProgress, "sb",
);
let merged = assert_converges(&[a_start, a_cp.clone(), b_start, b_cp], merge_progress);
// Deterministic: the latest write of the shared generation wins.
assert_eq!(merged, a_cp);
let session = |id: &str, device: &str, listened: i64, completed: bool| ListenSession {
session_id: id.to_string(),
episode_id: episode_id("e1"),
podcast_id: podcast_id(),
device_id: device.to_string(),
started_at_ms: 200,
ended_at_ms: 300,
start_position_ms: 300_000,
end_position_ms: 900_000,
listened_ms: listened,
completed,
};
let sessions = [
session("sa", "dev-a", 600_000, false),
session("sb", "dev-b", 1_200_000, true),
session("sb", "dev-b", 1_200_000, true),
];
let mut set: BTreeMap<String, ListenSession> = BTreeMap::new();
for order in permutations(&sessions) {
set.clear();
for item in order {
let merged = match set.get(&item.session_id) {
Some(current) => merge_listen_session(current, &item).unwrap(),
None => item.clone(),
};
set.insert(item.session_id.clone(), merged);
}
assert_eq!(set.len(), 2);
assert_eq!(completed_listen_count(set.values()), 1);
}
}
#[test]
fn listen_session_versions_only_grow() {
let base = ListenSession {
session_id: "s".to_string(),
episode_id: episode_id("e1"),
podcast_id: podcast_id(),
device_id: "dev-a".to_string(),
started_at_ms: 1_000,
ended_at_ms: 2_000,
start_position_ms: 0,
end_position_ms: 60_000,
listened_ms: 60_000,
completed: false,
};
let longer = ListenSession {
ended_at_ms: 3_000,
end_position_ms: 120_000,
listened_ms: 120_000,
..base.clone()
};
let finished = ListenSession {
ended_at_ms: 3_500,
end_position_ms: 3_600_000,
listened_ms: 125_000,
completed: true,
..base.clone()
};
let merged = assert_converges(
&[base.clone(), longer, finished.clone(), base.clone()],
merge_listen_session,
);
assert_eq!(merged, finished);
assert!(merged.should_record());
assert_eq!(completed_listen_count([&merged, &merged, &finished]), 1);
let mut other_episode = base.clone();
other_episode.episode_id = episode_id("e2");
assert!(merge_listen_session(&base, &other_episode).is_err());
let short = ListenSession {
listened_ms: MIN_RECORDED_EPISODE_LISTEN_MS - 1,
..base.clone()
};
assert!(!short.should_record());
assert!(
ListenSession {
listened_ms: MIN_RECORDED_EPISODE_LISTEN_MS,
..base
}
.should_record()
);
}
#[test]
fn completion_criterion() {
// 60 min episode: tail is 30 s.
assert!(!reaches_completion(3_569_999, Some(3_600_000)));
assert!(reaches_completion(3_570_000, Some(3_600_000)));
// 5 min episode: tail is 5 % = 15 s.
assert!(!reaches_completion(284_999, Some(300_000)));
assert!(reaches_completion(285_000, Some(300_000)));
assert!(!reaches_completion(1_000_000, None));
assert!(!reaches_completion(1_000_000, Some(0)));
}
fn podcast_meta(title: &str, url: &str) -> PodcastMeta {
PodcastMeta {
feed_url: feed_url(url),
title: title.to_string(),
author: None,
description: None,
link: None,
image_url: None,
}
}
fn subscription(
subscribed: bool,
hlc_ms: i64,
op: &str,
title: &str,
url: &str,
) -> SubscriptionRecord {
let mut record = SubscriptionRecord {
podcast_id: podcast_id(),
subscribed,
podcast: podcast_meta(title, url),
feed_aliases: Vec::new(),
generation: 1,
hlc_ms,
op_id: op.to_string(),
};
record.normalize();
record
}
fn with_generation<T: Clone>(record: &T, set: impl Fn(&mut T)) -> T {
let mut record = record.clone();
set(&mut record);
record
}
#[test]
fn subscriptions_converge_for_every_permutation() {
let ops = [
// Same podcast added on two devices.
subscription(true, 100, "dev-a:1", "Show", "feed"),
subscription(true, 101, "dev-b:1", "Show (B)", "feed"),
// Unsubscribed on A while B was offline.
subscription(false, 200, "dev-a:2", "Show", "feed"),
// B comes back and re-delivers its old state after the feed moved.
subscription(true, 150, "dev-b:2", "Show", "moved"),
subscription(true, 101, "dev-b:1", "Show (B)", "feed"),
];
let merged = assert_converges(&ops, merge_subscription);
assert!(
!merged.subscribed,
"unsubscribe survives the offline device"
);
assert_eq!(merged.op_id, "dev-a:2");
assert!(merged.feed_aliases.contains(&feed_url("moved")));
assert!(merged.feed_aliases.contains(&feed_url("feed")));
merged.validate().unwrap();
}
#[test]
fn resubscribing_keeps_progress_and_history() {
use ProgressStatus::*;
let mut progress_register = BTreeMap::new();
let saved = progress(4, 500, "dev-a", "dev-a:5", 1_234_000, InProgress, "s1");
progress_register.insert(saved.episode_id.clone(), saved.clone());
let subscribed = subscription(true, 100, "dev-a:1", "Show", "feed");
let unsubscribed = subscription(false, 600, "dev-a:6", "Show", "feed");
let again = subscription(true, 700, "dev-b:1", "Show", "feed");
let merged = fold(&[subscribed, unsubscribed, again], merge_subscription);
assert!(merged.subscribed);
assert_eq!(progress_register.get(&saved.episode_id), Some(&saved));
}
fn share(shared: bool, hlc_ms: i64, op: &str) -> ShareRecord {
ShareRecord {
podcast_id: podcast_id(),
shared,
generation: 1,
hlc_ms,
op_id: op.to_string(),
}
}
#[test]
fn shares_converge_and_false_is_never_beaten_by_a_stale_true() {
let ops = [
share(true, 100, "dev-a:1"),
share(false, 200, "dev-a:2"),
share(true, 150, "dev-b:1"),
share(false, 200, "dev-a:2"),
];
let merged = assert_converges(&ops, merge_share);
assert!(!merged.shared, "late stale share cannot republish");
}
#[test]
fn effective_share_rules() {
let subscribed = subscription(true, 100, "dev-a:1", "Show", "feed");
let shared = share(true, 150, "dev-a:2");
assert!(effective_share(Some(&subscribed), Some(&shared)));
assert!(!effective_share(Some(&subscribed), None));
assert!(!effective_share(None, Some(&shared)));
// New subscriptions are private: a share decision older than the
// latest subscription write does not apply.
let resubscribed = subscription(true, 300, "dev-a:4", "Show", "feed");
assert!(!effective_share(Some(&resubscribed), Some(&shared)));
// Unsubscribing revokes publication even if a stale share wins.
let unsubscribed = subscription(false, 200, "dev-a:3", "Show", "feed");
let stale = share(true, 250, "dev-b:1");
assert!(!effective_share(Some(&unsubscribed), Some(&stale)));
}
#[test]
fn clock_skew_cannot_undo_an_observed_unsubscribe() {
// Device A's clock runs ten minutes ahead. B sees A's subscription, then
// the user unsubscribes on B.
let sub_a = subscription(true, 1_000_600_000, "dev-a:1", "Show", "feed");
assert_eq!(next_podcast_generation(None, None), 1);
let generation = next_podcast_generation(Some(&sub_a), None);
assert_eq!(generation, 2);
let unsub_b = with_generation(
&subscription(false, 1_000_060_000, "dev-b:1", "Show", "feed"),
|r| r.generation = generation,
);
let merged = assert_converges(&[sub_a.clone(), unsub_b.clone()], merge_subscription);
assert!(!merged.subscribed, "the later unsubscribe wins");
// Concurrent writes (same generation) still fall back to the clock.
let concurrent = with_generation(&unsub_b, |r| r.generation = 1);
assert!(merge_subscription(&sub_a, &concurrent).unwrap().subscribed);
}
#[test]
fn share_after_subscribe_applies_despite_clock_skew() {
let sub_a = subscription(true, 1_000_600_000, "dev-a:1", "Show", "feed");
let share_b = with_generation(&share(true, 1_000_060_000, "dev-b:1"), |r| {
r.generation = next_podcast_generation(Some(&sub_a), None)
});
assert!(effective_share(Some(&sub_a), Some(&share_b)));
// Unsubscribe written after the share (higher generation) revokes it,
// and so does a later re-subscription until the user shares again.
let unsub = with_generation(&subscription(false, 1, "dev-a:2", "Show", "feed"), |r| {
r.generation = next_podcast_generation(Some(&sub_a), Some(&share_b))
});
assert!(!effective_share(Some(&unsub), Some(&share_b)));
let resub = with_generation(&subscription(true, 2, "dev-a:3", "Show", "feed"), |r| {
r.generation = next_podcast_generation(Some(&unsub), Some(&share_b))
});
assert!(!effective_share(Some(&resub), Some(&share_b)));
let reshare = with_generation(&share(true, 3, "dev-a:4"), |r| {
r.generation = next_podcast_generation(Some(&resub), Some(&share_b))
});
assert!(effective_share(Some(&resub), Some(&reshare)));
// Revoking (shared = false) with a higher generation wins over every
// stale true, in any order.
let revoke = with_generation(&share(false, 0, "dev-b:2"), |r| {
r.generation = next_podcast_generation(Some(&resub), Some(&reshare))
});
let merged = assert_converges(&[share_b, reshare, revoke.clone()], merge_share);
assert_eq!(merged, revoke);
}
#[test]
fn current_feed_url_stays_resolvable_beyond_the_alias_bound() {
let mut record = subscription(true, 1, "dev-a:1", "Show", "feed");
record.add_feed_aliases(
&(0..=crate::podcast::MAX_FEED_ALIASES)
.map(|i| feed_url(&format!("a{i}")))
.collect::<Vec<_>>(),
);
record.podcast.feed_url = FeedUrl::parse("https://z.test/current").unwrap();
record.normalize();
assert!(!record.feed_aliases.contains(&record.podcast.feed_url));
let urls = record.feed_urls();
assert!(urls.contains(&record.podcast.feed_url));
assert!(urls.contains(&feed_url("feed")), "origin is kept");
let known: Vec<(&PodcastId, &FeedUrl)> =
urls.iter().map(|url| (&record.podcast_id, url)).collect();
assert_eq!(
crate::podcast::resolve_podcast_id(&record.podcast.feed_url, known),
podcast_id()
);
}
#[test]
fn handoff_receiver_continues_with_the_carried_generation() {
use ProgressStatus::*;
// Sender A plays session s1 at generation 5 with a fast clock; its
// progress operations have not reached B when the handoff arrives.
let a_checkpoint = progress(5, 9_000_000, "dev-a", "dev-a:9", 600_000, InProgress, "s1");
let carried = Some(a_checkpoint.generation);
let b_local = Some(2);
let generation = ProgressRecord::explicit_generation(b_local.max(carried));
assert_eq!(generation, 6);
let b_start = progress(
generation, 1_000, "dev-b", "dev-b:1", 600_000, InProgress, "s2",
);
let b_pause = progress(
generation, 2_000, "dev-b", "dev-b:2", 660_000, InProgress, "s2",
);
// Without the carried generation B would write generation 3 and lose
// every write to A's older checkpoint.
let uninformed = progress(
ProgressRecord::explicit_generation(b_local),
2_000,
"dev-b",
"dev-b:2",
660_000,
InProgress,
"s2",
);
assert!(a_checkpoint.supersedes(&uninformed));
let merged = assert_converges(&[a_checkpoint, b_start, b_pause.clone()], merge_progress);
assert_eq!(merged, b_pause);
assert_eq!(merged.resume_position_ms(), 660_000);
}
#[test]
fn coalescing_keeps_the_superseding_record_not_the_last_queued() {
use ProgressStatus::*;
// "Mark played" (explicit, generation 5) during a session, followed in
// the outgoing log by a checkpoint of the old session (generation 3).
let mark_played = progress(5, 100, "dev-a", "dev-a:7", 0, Completed, "mark");
let late_checkpoint = progress(3, 101, "dev-a", "dev-a:8", 30_000, InProgress, "s1");
let pending = [mark_played.clone(), late_checkpoint.clone()];
// Keep only records that no other pending record supersedes.
let kept: Vec<&ProgressRecord> = pending
.iter()
.filter(|record| !pending.iter().any(|other| other.supersedes(record)))
.collect();
assert_eq!(kept, vec![&mark_played]);
// Dropping by sequence order would have published the wrong state.
assert_ne!(pending.last(), Some(&mark_played));
assert_eq!(fold(&pending, merge_progress), mark_played);
}
/// Minimal client store used to show that whole operation sets converge and
/// that publisher metadata never reaches personal progress.
#[derive(Debug, Default, Clone, PartialEq)]
struct Store {
subscriptions: BTreeMap<PodcastId, SubscriptionRecord>,
shares: BTreeMap<PodcastId, ShareRecord>,
progress: BTreeMap<EpisodeId, ProgressRecord>,
sessions: BTreeMap<String, ListenSession>,
catalog: BTreeMap<EpisodeId, EpisodeMeta>,
}
#[derive(Debug, Clone)]
enum Op {
Subscription(SubscriptionRecord),
Share(ShareRecord),
Progress(ProgressRecord),
Session(ListenSession),
/// Local RSS refresh: title and audio URL changed at the publisher.
Rss {
title: &'static str,
audio_url: &'static str,
},
}
fn upsert<K: Ord + Clone, V: Clone>(
map: &mut BTreeMap<K, V>,
key: K,
value: &V,
merge: impl Fn(&V, &V) -> PodcastResult<V>,
) {
let merged = match map.get(&key) {
Some(current) => merge(current, value).unwrap(),
None => value.clone(),
};
map.insert(key, merged);
}
impl Store {
fn apply(&mut self, op: &Op) {
match op {
Op::Subscription(record) => upsert(
&mut self.subscriptions,
record.podcast_id.clone(),
record,
merge_subscription,
),
Op::Share(record) => upsert(
&mut self.shares,
record.podcast_id.clone(),
record,
merge_share,
),
Op::Progress(record) => upsert(
&mut self.progress,
record.episode_id.clone(),
record,
merge_progress,
),
Op::Session(session) => upsert(
&mut self.sessions,
session.session_id.clone(),
session,
merge_listen_session,
),
Op::Rss { title, audio_url } => {
let hints = EpisodeIdentityHints {
guid: Some("e1".to_string()),
enclosure_url: Some(audio_url.to_string()),
link: None,
published_at_ms: Some(1_000),
title: Some(title.to_string()),
};
let id = resolve_episode_ids(&podcast_id(), None, &[hints], &[])[0]
.clone()
.unwrap();
// Metadata only: last refresh on this device wins locally.
let meta = EpisodeMeta {
episode_id: id.clone(),
podcast_id: podcast_id(),
guid: Some("e1".to_string()),
title: title.to_string(),
description: None,
season: None,
episode_number: None,
episode_type: EpisodeType::Full,
published_at_ms: Some(1_000),
duration_ms: None,
audio_url: audio_url.to_string(),
mime: None,
byte_len: None,
image_url: None,
link: None,
};
self.catalog.insert(id, meta);
}
}
}
fn personal(&self) -> Store {
Store {
catalog: BTreeMap::new(),
..self.clone()
}
}
}
#[test]
fn whole_operation_sets_converge_and_rss_never_touches_progress() {
use ProgressStatus::*;
let session = ListenSession {
session_id: "s2".to_string(),
episode_id: episode_id("e1"),
podcast_id: podcast_id(),
device_id: "dev-b".to_string(),
started_at_ms: 10,
ended_at_ms: 20,
start_position_ms: 0,
end_position_ms: 400_000,
listened_ms: 400_000,
completed: false,
};
let ops = [
Op::Subscription(subscription(true, 100, "dev-a:1", "Show", "feed")),
Op::Share(share(true, 120, "dev-a:2")),
Op::Progress(progress(
1, 130, "dev-a", "dev-a:3", 900_000, InProgress, "s1",
)),
Op::Progress(progress(
2, 125, "dev-b", "dev-b:1", 400_000, InProgress, "s2",
)),
Op::Session(session),
Op::Rss {
title: "Old title",
audio_url: "https://cdn.test/old.mp3",
},
Op::Rss {
title: "New title",
audio_url: "https://cdn.test/new.mp3",
},
];
let mut reference: Option<Store> = None;
for order in permutations(&ops) {
let mut store = Store::default();
for op in &order {
store.apply(op);
}
// One episode id across both metadata versions.
assert_eq!(store.catalog.len(), 1);
let progress = store.progress.values().next().unwrap();
assert_eq!(progress.op_id, "dev-b:1");
assert_eq!(progress.position_ms, 400_000);
let personal = store.personal();
match &reference {
Some(reference) => assert_eq!(&personal, reference),
None => reference = Some(personal),
}
}
}
#[test]
fn validation_bounds() {
use ProgressStatus::*;
let ok = progress(1, 1, "dev-a", "dev-a:1", 0, NotStarted, "s");
ok.validate().unwrap();
let mut bad = ok.clone();
bad.position_ms = -1;
assert!(bad.validate().is_err());
let mut bad = ok.clone();
bad.session_id = String::new();
assert!(bad.validate().is_err());
let mut bad = ok;
bad.device_id = "d".repeat(crate::podcast::MAX_TOKEN_BYTES + 1);
assert!(bad.validate().is_err());
let mut record = subscription(true, 1, "dev-a:1", "Show", "feed");
record.feed_aliases = (0..=crate::podcast::MAX_FEED_ALIASES + 1)
.map(|i| feed_url(&format!("alias-{i}")))
.collect();
assert!(record.validate().is_err());
record.normalize();
record.validate().unwrap();
}
#[test]
fn progress_json_shape() {
let record = progress(
3,
42,
"dev-a",
"dev-a:7",
1_000,
ProgressStatus::InProgress,
"s",
);
let json = serde_json::to_value(&record).unwrap();
assert_eq!(json["status"], "in_progress");
assert_eq!(json["generation"], 3);
assert_eq!(
serde_json::from_value::<ProgressRecord>(json).unwrap(),
record
);
}