2026-11-18 15:17:50 +00:00
|
|
|
# Development
|
|
|
|
|
|
|
|
|
|
[Project overview](../README.md) · [Installation](setup.md) · [Admin UI conventions](admin-ui.md)
|
|
|
|
|
|
|
|
|
|
## Architecture
|
|
|
|
|
|
|
|
|
|
Furumusic is written in Rust on the
|
|
|
|
|
[Cot](https://cot.rs) web framework. PostgreSQL stores the catalog, accounts,
|
|
|
|
|
playlists, configuration, and background-job state. Audio inspection uses
|
|
|
|
|
Symphonia, torrent downloads use librqbit, and the shared `music-dht`/Frid
|
|
|
|
|
protocol stack provides decentralized catalog search, media transfer, and
|
|
|
|
|
connected-device synchronization.
|
|
|
|
|
|
|
|
|
|
The browser interface and JSON API are served by the same application. Import,
|
|
|
|
|
artwork, metadata enrichment, similarity indexing, and maintenance run as
|
|
|
|
|
durable background jobs rather than blocking playback requests.
|
|
|
|
|
|
|
|
|
|
## Build profiles
|
|
|
|
|
|
|
|
|
|
Development and test builds use basic optimization (`opt-level = 1`) for
|
|
|
|
|
dependencies to keep macOS unwind tables below the compact-unwind size limit.
|
|
|
|
|
Application code stays unoptimized; debug symbols, assertions and panic
|
|
|
|
|
unwinding remain enabled. The first build compiles dependencies with these
|
|
|
|
|
settings, then reuses them from Cargo's cache. Dependency stepping in a debugger
|
|
|
|
|
can be less exact because of optimization. Release settings are independent.
|
|
|
|
|
|
|
|
|
|
See the [Cargo profile override documentation](https://doc.rust-lang.org/cargo/reference/profiles.html#overrides) for dependency-specific settings.
|
|
|
|
|
|
|
|
|
|
## Container builds and CI
|
|
|
|
|
|
|
|
|
|
CI uses Rust 1.97.0 and cargo-chef 0.1.74 to cache compiled dependencies separately
|
|
|
|
|
from application sources. Buildx imports and exports intermediate layers through
|
|
|
|
|
`ultradesu/furumusic:buildcache` (`mode=max`). After the first build, changes to
|
|
|
|
|
source code, templates or prompts reuse the dependency layer when its recipe and
|
|
|
|
|
toolchain are unchanged. Both dependency and application builds use `--locked`.
|
|
|
|
|
Update the pinned build tools in `Dockerfile`; changing them rebuilds the cache.
|
|
|
|
|
|
2026-11-19 16:58:32 +02:00
|
|
|
## Connected devices and podcast queue items
|
|
|
|
|
|
|
|
|
|
Trusted-device sync uses the shared `music_dht::device_sync` types (no local
|
2026-11-19 19:55:46 +02:00
|
|
|
copies) and announces protocol version 4. Each account is one virtual device:
|
2026-11-19 16:58:32 +02:00
|
|
|
|
|
|
|
|
- podcast operations stay in the operation log and are relayed to v4 devices
|
2026-11-19 19:55:46 +02:00
|
|
|
only. They are also applied to the account's podcast store
|
|
|
|
|
(`src/podcasts/`, `PodcastStore`), which also provides the podcast snapshot
|
|
|
|
|
sections;
|
2026-11-19 16:58:32 +02:00
|
|
|
- queue items that are not library tracks become external browser items
|
|
|
|
|
(`source` plus the verbatim `wire_track`), so podcast episodes and Jam
|
|
|
|
|
placeholders survive peer → server → browser → server → peer unchanged;
|
|
|
|
|
- `furumusic__playback_state.queue_items_json` stores the typed queue next to
|
|
|
|
|
`queue_json`. `queue_json` (library ids) is still written for rollback; the
|
2026-11-19 19:22:26 +02:00
|
|
|
typed value is used only while the legacy columns it recorded
|
|
|
|
|
(`queue_json`, `current_track_id`, `queue_position`) match the row. Jam
|
|
|
|
|
placeholders are never stored in a user's own queue.
|
|
|
|
|
- a browser that selects a podcast episode it cannot play does not know the
|
|
|
|
|
episode's position. It reports `position_unknown`, and the server sends the
|
2026-11-19 19:55:46 +02:00
|
|
|
resume position of the episode's progress register instead of 0. Jam
|
2026-11-19 19:22:26 +02:00
|
|
|
participant commands carry `jam_participant`; the host browser checks them
|
|
|
|
|
against its own queue.
|
|
|
|
|
|
|
|
|
|
Rolling back to 0.12 needs the podcast operations removed from the operation
|
|
|
|
|
log first; see "Rolling back to 0.12" in `administration.md`.
|
2026-11-19 16:58:32 +02:00
|
|
|
|
2026-11-19 19:55:46 +02:00
|
|
|
### Podcast store
|
|
|
|
|
|
|
|
|
|
`PodcastStore` implements the podcast contract of `music_dht::podcast`
|
|
|
|
|
(Frid `PODCASTS.md`) on PostgreSQL, with the invariants of the SQLite
|
|
|
|
|
reference store in `furumi-library`; every table has a `user_id` key.
|
|
|
|
|
|
|
|
|
|
| Table | Content | Synced |
|
|
|
|
|
| --- | --- | --- |
|
|
|
|
|
| `furumusic__podcast` | channel metadata, fetch URL | no (catalog) |
|
|
|
|
|
| `furumusic__podcast_feed_alias` | bounded grow-only alias set | in subscription records and `podcast_feed_aliases_added` |
|
|
|
|
|
| `furumusic__podcast_episode` | `EpisodeMeta`, `source` `feed`/`sync`, `in_feed`, cached search text | no (catalog) |
|
|
|
|
|
| `furumusic__podcast_season` | view over the episodes | no |
|
|
|
|
|
| `furumusic__podcast_subscription`, `_share`, `_progress` | registers | yes |
|
|
|
|
|
| `furumusic__podcast_listen_session` | grow-only set of sessions | yes |
|
|
|
|
|
| `furumusic__podcast_fetch_state` | ETag, Last-Modified, check times, errors, cover cache | never |
|
|
|
|
|
|
|
|
|
|
- One apply path: received operations, snapshot records and local actions go
|
|
|
|
|
through the `SyncOpWire` materializers and the `merge_*` functions of
|
|
|
|
|
`music_dht::podcast`. Each write transaction holds a per-account advisory
|
|
|
|
|
lock.
|
|
|
|
|
- Local actions (`subscribe`, `unsubscribe`, `set_shared`, `start_session`,
|
|
|
|
|
`checkpoint`, `seek`, `complete_session`, `restart`, `mark_played`,
|
|
|
|
|
`mark_unplayed`, `record_listen_session`) append their operation to the
|
|
|
|
|
account's sync log with the next stamp (`devices::append_local_op`: `seq`,
|
|
|
|
|
`hlc_ms`, `op_id = <device>:<seq>`) and apply exactly that operation in
|
|
|
|
|
the same transaction.
|
|
|
|
|
- `apply_feed` resolves episode ids with the shared no-GUID rule against the
|
|
|
|
|
stored episodes, including those only known from synced references. It
|
|
|
|
|
updates the catalog and the alias set, never a register, progress or a
|
|
|
|
|
session.
|
|
|
|
|
- Snapshots export each section newest first up to its count and bound it
|
|
|
|
|
with `bound_podcast_state_to`. The catalog and the fetch state are never
|
|
|
|
|
exported.
|
|
|
|
|
|
|
|
|
|
Migrations `m_0056_create_podcast_store` and `m_0057_import_podcast_relay`
|
|
|
|
|
create the tables and import what 0.13.0 kept in `furumusic__fed_podcast_relay`
|
|
|
|
|
(registers as contract JSON, already merged) plus the catalog rows of the
|
|
|
|
|
episode references in the operation log. The relay table is no longer written
|
|
|
|
|
or read; it stays until a later cleanup release.
|
|
|
|
|
|
2026-11-19 16:58:32 +02:00
|
|
|
Everything sent to a device is gated by the version that device reported itself
|
|
|
|
|
(`profile_protocol_version`; relayed profiles only raise it). See the podcast
|
|
|
|
|
contract (`PODCASTS.md`) and `PLAYBACK_PROTOCOL.md` in the Frid repository.
|
|
|
|
|
|
|
|
|
|
```bash
|
2026-11-19 19:55:46 +02:00
|
|
|
cargo test devices:: queue_items podcasts:: queue_persistence
|
2026-11-19 16:58:32 +02:00
|
|
|
node --test tests/player_queue_sources.test.cjs
|
2026-11-19 19:55:46 +02:00
|
|
|
# Disposable PostgreSQL; the tests run the production migrations on it, and
|
|
|
|
|
# the migration test creates and drops a database next to it:
|
2026-11-19 16:58:32 +02:00
|
|
|
FURU_DEVICE_SYNC_TEST_DATABASE_URL=postgresql://... \
|
2026-11-19 19:55:46 +02:00
|
|
|
cargo test postgres_tests -- --ignored
|
2026-11-19 18:35:49 +02:00
|
|
|
FURU_METADATA_FIX_TEST_DATABASE_URL=postgresql://... \
|
|
|
|
|
cargo test undo_accepts_playback_archives_written_before_the_typed_queue -- --ignored
|
2026-11-19 16:58:32 +02:00
|
|
|
```
|
|
|
|
|
|
|
|
|
|
`wire_compat_tests` compares the shared types with the private copies of
|
|
|
|
|
0.12.x byte for byte; `fixtures/v3_hello.golden.json` is the v3 shape. Rewrite
|
|
|
|
|
it only for a deliberate change, with `FURUMUSIC_BLESS_GOLDEN=1`.
|
|
|
|
|
|
2026-11-18 15:17:50 +00:00
|
|
|
## Contributing
|
|
|
|
|
|
|
|
|
|
Bug reports, design discussions, and patches are welcome. Before submitting a
|
|
|
|
|
change, run:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
cargo fmt --all -- --check
|
|
|
|
|
cargo clippy --all-targets -- -D warnings
|
|
|
|
|
cargo test --all-targets
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Last.fm database tests use temporary tables on a disposable PostgreSQL database:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
FURUMUSIC_LASTFM_TEST_DATABASE_URL=postgres://user:pass@localhost/test_db \
|
|
|
|
|
cargo test lastfm_popularity -- --include-ignored
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Access and sharing tests
|
|
|
|
|
|
|
|
|
|
The access regression suite uses a disposable migrated database and a local OIDC
|
|
|
|
|
provider with signed tokens and PKCE:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
node --test tests/*.test.cjs
|
|
|
|
|
FURU_ACCESS_TEST_DATABASE_URL=postgresql://... cargo test postgres_approval_survives_sso_changes_without_implicit_grants -- --ignored
|
|
|
|
|
# Fresh disposable instance on :18048, media root /tmp/furumi-access-media,
|
|
|
|
|
# PostgreSQL container furumi-access-test-pg, database furumi_access_test:
|
|
|
|
|
node tests/browser-access-sharing.cjs
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The browser test requires Playwright (`PLAYWRIGHT_MODULE`) and Chrome
|
|
|
|
|
(`CHROME_PATH`), and uses local port 18049 for its OIDC provider. It checks
|
|
|
|
|
approval, SSO precedence and group removal, guest playback and downloads,
|
|
|
|
|
unauthorized track substitution, and desktop/mobile player behavior.
|
|
|
|
|
|
|
|
|
|
## Statistics tests
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
node --test tests/*.test.cjs
|
|
|
|
|
cargo test admin:: -- --skip postgres_statistics_drilldown_and_pagination
|
|
|
|
|
# Use a disposable PostgreSQL; this test creates connection-local temporary tables.
|
|
|
|
|
FURU_STATISTICS_TEST_DATABASE_URL=postgresql://... cargo test postgres_statistics_drilldown_and_pagination -- --ignored
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
`tests/browser-statistics.cjs` additionally exercises a fresh disposable instance
|
|
|
|
|
at `FURU_TEST_URL` (default `http://127.0.0.1:3018`) and seeds the disposable Docker
|
|
|
|
|
container `furumi-statistics-test-pg`. It requires Playwright (or
|
|
|
|
|
`PLAYWRIGHT_MODULE`) and Chrome (or `CHROME_PATH`). Never point this test at a
|
|
|
|
|
live service.
|
|
|
|
|
|
|
|
|
|
## Metadata Repair tests
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
node --test tests/admin_metadata_repair.test.cjs
|
|
|
|
|
cargo test admin::metadata_fix::tests
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
`tests/browser-metadata-repair.cjs` checks the chat and plan updates, a four-release
|
|
|
|
|
merge, deletion and restoration of tracks and artist credits, playlists, playback
|
|
|
|
|
queues, and undo conflicts. Run it only against a disposable instance and a
|
|
|
|
|
PostgreSQL container named `furumi-repair-test-pg`, with database
|
|
|
|
|
`furumi_repair_test`. Set `FURU_TEST_URL`, `FURU_REPAIR_TEST_PG`, `CHROME_PATH`, and
|
|
|
|
|
`PLAYWRIGHT_MODULE` as needed. It seeds `tests/metadata_repair_fixture.sql` after
|
|
|
|
|
creating the test administrator. Set `FURU_REPAIR_SKIP_SEED=1` for a seeded database.
|
|
|
|
|
With `FURU_REPAIR_TEST_MODEL=1`, it additionally checks the reported release-only
|
|
|
|
|
selection and full ordered tracklist against the configured model, applies and
|
|
|
|
|
undoes that plan, and verifies individual track exclusions.
|
|
|
|
|
|
|
|
|
|
## Import destination tests
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
node --test tests/import_playlists.test.cjs
|
|
|
|
|
# Disposable PostgreSQL; both tests use temporary tables.
|
|
|
|
|
FURU_IMPORT_TEST_DATABASE_URL=postgresql://... cargo test import_destinations -- --ignored
|
|
|
|
|
FURU_IMPORT_TEST_DATABASE_URL=postgresql://... cargo test torrent_playlist -- --ignored
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
`tests/browser-import-playlists.cjs` checks the file upload flow, database
|
|
|
|
|
reconciliation and the torrent picker on desktop and mobile. It requires a fresh
|
|
|
|
|
migrated instance with the import agent disabled at `FURU_TEST_URL` (default
|
|
|
|
|
`http://127.0.0.1:3019`), the disposable PostgreSQL container
|
|
|
|
|
`furumi-import-playlist-test-pg` with database `furumi_import_test`, and Playwright
|
|
|
|
|
and Chrome configured as above. It uses a local torrent preview fixture and
|
|
|
|
|
simulated imported tracks, without downloading external torrents.
|