Files
furumi-web/docs/development.md
T
sanapachandClaude Opus 5.5 9b43a25d8f Resume episodes from the progress register and keep saving after a takeover
A reload or a Play after a pause continues from the register instead of
this page's saved position, a seek during preparation is kept, a player
that keeps listening after being superseded writes again as a new
session, a player that loses the playback role records its listening,
marking the loaded episode closes its session, and Prev no longer
resets an episode to 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 15:12:38 +02:00

15 KiB
Raw Blame History

Development

Project overview · Installation · Admin UI conventions

Architecture

Furumusic is written in Rust on the Cot web framework. PostgreSQL stores the catalog, accounts, playlists, configuration, and background-job state. Audio inspection uses Symphonia, torrent downloads use librqbit, and the shared music-dht/Frid protocol stack provides decentralized catalog search, media transfer, and connected-device synchronization.

The browser interface and JSON API are served by the same application. Import, artwork, metadata enrichment, similarity indexing, and maintenance run as durable background jobs rather than blocking playback requests.

Build profiles

Development and test builds use basic optimization (opt-level = 1) for dependencies to keep macOS unwind tables below the compact-unwind size limit. Application code stays unoptimized; debug symbols, assertions and panic unwinding remain enabled. The first build compiles dependencies with these settings, then reuses them from Cargo's cache. Dependency stepping in a debugger can be less exact because of optimization. Release settings are independent.

See the Cargo profile override documentation for dependency-specific settings.

Container builds and CI

CI uses Rust 1.97.0 and cargo-chef 0.1.74 to cache compiled dependencies separately from application sources. Buildx imports and exports intermediate layers through ultradesu/furumusic:buildcache (mode=max). After the first build, changes to source code, templates or prompts reuse the dependency layer when its recipe and toolchain are unchanged. Both dependency and application builds use --locked. Update the pinned build tools in Dockerfile; changing them rebuilds the cache.

Connected devices and podcast queue items

Trusted-device sync uses the shared music_dht::device_sync types (no local copies) and announces protocol version 4. Each account is one virtual device:

  • podcast operations stay in the operation log and are relayed to v4 devices only. They are also applied to the account's podcast store (src/podcasts/, PodcastStore), which also provides the podcast snapshot sections;
  • queue items that are not library tracks become external browser items (source plus the verbatim wire_track), so podcast episodes and Jam placeholders survive peer → server → browser → server → peer unchanged;
  • furumusic__playback_state.queue_items_json stores the typed queue next to queue_json. queue_json (library ids) is still written for rollback; the typed value is used only while the legacy columns it recorded (queue_json, current_track_id, queue_position) match the row. Jam placeholders are never stored in a user's own queue.
  • a browser that selects an item it cannot play (a Jam placeholder), or an episode whose resume position the server has not sent yet, reports position_unknown, and the server sends the resume position of the episode's progress register instead of 0. Outgoing playback states raise each episode's progress_generation to the register generation, so the device that takes over writes above it. Jam participant commands carry jam_participant; the host browser checks them against its own queue.

Rolling back to 0.12 needs the podcast operations removed from the operation log first; see "Rolling back to 0.12" in administration.md.

Podcast store

PodcastStore implements the podcast contract of music_dht::podcast (Frid PODCASTS.md) on PostgreSQL, with the invariants of the SQLite reference store in furumi-library; every table has a user_id key.

Table Content Synced
furumusic__podcast channel metadata, fetch URL no (catalog)
furumusic__podcast_feed_alias bounded grow-only alias set in subscription records and podcast_feed_aliases_added
furumusic__podcast_episode EpisodeMeta, source feed/sync, in_feed, cached search text no (catalog)
furumusic__podcast_season view over the episodes no
furumusic__podcast_subscription, _share, _progress registers yes
furumusic__podcast_listen_session grow-only set of sessions yes
furumusic__podcast_fetch_state ETag, Last-Modified, check times, errors, cover cache never
  • One apply path: received operations, snapshot records and local actions go through the SyncOpWire materializers and the merge_* functions of music_dht::podcast. Each write transaction holds a per-account advisory lock.
  • Local actions (subscribe, unsubscribe, set_shared, start_session, checkpoint, seek, complete_if_current, restart, mark_played, mark_unplayed, record_listen_session) append their operation to the account's sync log with the next stamp (devices::append_local_op: seq, hlc_ms, op_id = <device>:<seq>) and apply exactly that operation in the same transaction.
  • apply_feed resolves episode ids with the shared no-GUID rule against the stored episodes, including those only known from synced references. It updates the catalog and the alias set, never a register, progress or a session.
  • Snapshots export each section newest first up to its count and bound it with bound_podcast_state_to. The catalog and the fetch state are never exported.

Migrations m_0056_create_podcast_store and m_0057_import_podcast_relay create the tables and import what 0.13.0 kept in furumusic__fed_podcast_relay (registers as contract JSON, already merged) plus the catalog rows of the episode references in the operation log. The relay table is no longer written or read; it stays until a later cleanup release.

Feed refresh and audio delivery

The runtime around the store lives next to it in src/podcasts/:

  • net.rs — SafeHttp, the only HTTP client for podcast feeds and audio. It accepts http/https URLs without userinfo, resolves names with GuardedResolver and refuses the whole answer when any address is not public (loopback, RFC 1918, CGNAT, link-local, cloud metadata, unique local, multicast, unspecified, documentation, reserved; IPv4-mapped, NAT64 and 6to4 addresses are judged by their IPv4 part). The connector connects to exactly the checked addresses, so a second DNS answer cannot be swapped in. IP literals are checked before connecting, redirects are followed manually (each hop checked, at most 5), and system proxies, cookies and Referer are disabled.
  • fetch.rs — FeedFetcher: conditional GET (If-None-Match, If-Modified-Since), Accept-Encoding: identity, 32 MiB and 60 s limits, Retry-After on 429/503, the shared music_dht parser. Only a chain of permanent redirects (301/308) moves a feed.
  • runner.rs — scheduling: podcasts_due_for_refresh, one request per feed URL for all accounts subscribed to it (conditional only when they share the same validators), apply_feed, then the fetch state. Healthy feeds are checked hourly plus a stable spread of up to 10 minutes; failures back off 15 min × 2ⁿ⁻¹ up to a day (a longer Retry-After wins). refresh_now is the manual refresh: subscribed podcasts only, at most every 30 s. jobs/podcast_refresh.rs runs refresh_due every five minutes (200 podcasts, 4 requests in flight, one run at a time via an advisory lock).
  • proxy.rs and http.rs — the optional audio proxy and the endpoints under /api/podcasts. The proxy takes an episode id, loads the URL from the account's catalog (PodcastStore::audio_source), forwards a single sanitized byte Range, passes 200/206/416 through, accepts audio content types only (a generic binary type is replaced by the stored enclosure type or the URL extension), and stops at 1 GiB, 20 s without headers, 30 s without data or 4 h per response. It is off unless podcast_audio_proxy_enabled is set; by default the browser streams from the publisher URL.

Podcast UI and playback

api.rs holds what the endpoints do, independent of HTTP (http.rs only authenticates and maps statuses), so the Postgres tests drive it directly:

Endpoint (/api/podcasts) Does
GET /feeds[?all=1], POST /feeds {url} list; subscribe after fetching the feed (runner::subscribe_url)
GET /feeds/{id}, GET /feeds/{id}/episodes?season=&played=&q=&sort=&limit=&offset= podcast, seasons, history; episode pages with the browser queue item
POST /feeds/{id}/subscribe, /unsubscribe, POST /{id}/refresh subscription and manual refresh
POST /episodes/{id}/restart, /played, /unplayed explicit progress actions
POST /episodes/{id}/playback source, resume position; with start a new session (explicit, above carried_generation); records an unknown handed-off reference first
POST /episodes/{id}/progress checkpoint/pause/stop/handoff at the session generation, seek explicit, completion once (ended or reaches_completion), listening session record

Request bodies must be application/json (415 otherwise, like cot's Json extractor): other types need no CORS preflight and would let a same-site page post with the session cookie.

In the browser (templates/player/scripts.html), _playEpisodeLocal asks for the playback, loads the publisher URL (or the proxy: mixed content, or a failed direct load) and seeks before playing; when the source cannot seek to the saved position it shows that instead of playing from elsewhere. Only a handoff or the user's own seek sends a position (explicitPosition); an ordinary Play, including after a reload, resumes from the register, and Play after a pause first checks that the register is still this session's (register_session_id). A seek made while the source is being prepared is applied when it arrives. Writes of one session run in order (_reportEpisode), read the session generation at send time and carry the position of their own episode. After superseded, checkpoints stop; if the user keeps listening here, the next write (one checkpoint interval later, or at once on pause, stop or the end of the stream) records the old session and starts a new explicit one (_takeOverEpisode). A browser that loses the active role sends one final stop at its session generation (it records the listening; the new device's start keeps the register), unless it handed off itself. Marking the loaded episode played or unplayed in the same tab saves and closes its session and pauses it. Prev on an episode goes back 15 s locally instead of resetting it. Jam guests never play or save episodes (they only ever see placeholders). The podcasts store backs the Podcasts views in shell.html.

Tests never reach the internet: test_server.rs serves on 127.0.0.1 and provides a scripted resolver. Loopback is reachable only through AddressPolicy::test_allow_loopback, which exists only under #[cfg(test)].

Everything sent to a device is gated by the version that device reported itself (profile_protocol_version; relayed profiles only raise it). See the podcast contract (PODCASTS.md) and PLAYBACK_PROTOCOL.md in the Frid repository.

cargo test devices:: queue_items podcasts:: queue_persistence
node --test tests/player_queue_sources.test.cjs tests/player_podcasts.test.cjs
# Disposable PostgreSQL; the tests run the production migrations on it, and
# the migration test creates and drops a database next to it:
FURU_DEVICE_SYNC_TEST_DATABASE_URL=postgresql://... \
  cargo test postgres_tests -- --ignored
FURU_METADATA_FIX_TEST_DATABASE_URL=postgresql://... \
  cargo test undo_accepts_playback_archives_written_before_the_typed_queue -- --ignored

wire_compat_tests compares the shared types with the private copies of 0.12.x byte for byte; fixtures/v3_hello.golden.json is the v3 shape. Rewrite it only for a deliberate change, with FURUMUSIC_BLESS_GOLDEN=1.

Contributing

Bug reports, design discussions, and patches are welcome. Before submitting a change, run:

cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo test --all-targets

Last.fm database tests use temporary tables on a disposable PostgreSQL database:

FURUMUSIC_LASTFM_TEST_DATABASE_URL=postgres://user:pass@localhost/test_db \
  cargo test lastfm_popularity -- --include-ignored

Access and sharing tests

The access regression suite uses a disposable migrated database and a local OIDC provider with signed tokens and PKCE:

node --test tests/*.test.cjs
FURU_ACCESS_TEST_DATABASE_URL=postgresql://... cargo test postgres_approval_survives_sso_changes_without_implicit_grants -- --ignored
# Fresh disposable instance on :18048, media root /tmp/furumi-access-media,
# PostgreSQL container furumi-access-test-pg, database furumi_access_test:
node tests/browser-access-sharing.cjs

The browser test requires Playwright (PLAYWRIGHT_MODULE) and Chrome (CHROME_PATH), and uses local port 18049 for its OIDC provider. It checks approval, SSO precedence and group removal, guest playback and downloads, unauthorized track substitution, and desktop/mobile player behavior.

Statistics tests

node --test tests/*.test.cjs
cargo test admin:: -- --skip postgres_statistics_drilldown_and_pagination
# Use a disposable PostgreSQL; this test creates connection-local temporary tables.
FURU_STATISTICS_TEST_DATABASE_URL=postgresql://... cargo test postgres_statistics_drilldown_and_pagination -- --ignored

tests/browser-statistics.cjs additionally exercises a fresh disposable instance at FURU_TEST_URL (default http://127.0.0.1:3018) and seeds the disposable Docker container furumi-statistics-test-pg. It requires Playwright (or PLAYWRIGHT_MODULE) and Chrome (or CHROME_PATH). Never point this test at a live service.

Metadata Repair tests

node --test tests/admin_metadata_repair.test.cjs
cargo test admin::metadata_fix::tests

tests/browser-metadata-repair.cjs checks the chat and plan updates, a four-release merge, deletion and restoration of tracks and artist credits, playlists, playback queues, and undo conflicts. Run it only against a disposable instance and a PostgreSQL container named furumi-repair-test-pg, with database furumi_repair_test. Set FURU_TEST_URL, FURU_REPAIR_TEST_PG, CHROME_PATH, and PLAYWRIGHT_MODULE as needed. It seeds tests/metadata_repair_fixture.sql after creating the test administrator. Set FURU_REPAIR_SKIP_SEED=1 for a seeded database. With FURU_REPAIR_TEST_MODEL=1, it additionally checks the reported release-only selection and full ordered tracklist against the configured model, applies and undoes that plan, and verifies individual track exclusions.

Import destination tests

node --test tests/import_playlists.test.cjs
# Disposable PostgreSQL; both tests use temporary tables.
FURU_IMPORT_TEST_DATABASE_URL=postgresql://... cargo test import_destinations -- --ignored
FURU_IMPORT_TEST_DATABASE_URL=postgresql://... cargo test torrent_playlist -- --ignored

tests/browser-import-playlists.cjs checks the file upload flow, database reconciliation and the torrent picker on desktop and mobile. It requires a fresh migrated instance with the import agent disabled at FURU_TEST_URL (default http://127.0.0.1:3019), the disposable PostgreSQL container furumi-import-playlist-test-pg with database furumi_import_test, and Playwright and Chrome configured as above. It uses a local torrent preview fixture and simulated imported tracks, without downloading external torrents.