Cache every played or saved video on the server via background jobs with a validation gate and a Broken re-download button
This commit is contained in:
35
CLAUDE.md
35
CLAUDE.md
@@ -15,15 +15,44 @@ The PWA is what runs in production; `legacy/` holds the old native-only docs.
|
||||
| `frontend/fingerprint.js` | Browser fingerprint used as the sync key |
|
||||
| `frontend/async-guard.js` | Stale-async-response guard (unit-tested) |
|
||||
| `server/server.js` | Bun + Hono backend — endpoint list is in its header comment |
|
||||
| `server/db.js` | libsql schema/queries (`users`, `profiles`, `video_history`…) |
|
||||
| `server/db.js` | libsql schema/queries (`users`, `profiles`, `video_history`, `media_cache`…) |
|
||||
| `server/media-cache.js` | Server-side video cache: background fetch jobs, validation gate, x264 lane, LRU |
|
||||
| `bin/yt-dlp` | Downloaded by `npm run setup`, gitignored |
|
||||
| `scripts/` | icon generation, yt-dlp setup, push helper |
|
||||
| `tests/` | Playwright e2e specs · unit tests live next to sources in `frontend/*.test.js` |
|
||||
| `legacy/` | Old native-shell docs — do not treat as current |
|
||||
|
||||
API endpoints: `GET /api/search|channel|streams|download/:id|version|user/data|profile/load`,
|
||||
`GET /api/media/:id|media/:id/status|media/stats`, `POST /api/media/:id/redownload`,
|
||||
`POST /api/user/sync|profile/create|profile/save`, `GET /sw.js` (BUILD_TAG-injected), `GET /*` static.
|
||||
JSON shapes mirror the Tauri Rust bridge exactly — don't change one side alone.
|
||||
JSON shapes mirror the Tauri Rust bridge exactly — don't change one side alone
|
||||
(`/api/streams`' `data.serverCached` is additive and web-only).
|
||||
|
||||
## Server media cache (`server/media-cache.js`)
|
||||
- Every played (`/api/streams`, LOW priority, ≤ `MEDIA_AUTO_MAX_SECONDS`) or saved
|
||||
(`/api/download`, HIGH, ≤ 3 h) video gets ONE copy: `$MEDIA_DIR/<id>.<gen>.mp4`
|
||||
(≤720p H.264 8-bit + AAC, faststart) + `<id>.<gen>.m4a` audio sidecar for audio-only
|
||||
mode. `MEDIA_DIR` defaults to `./data/media` (the `ytplayer-data` volume in prod).
|
||||
- Jobs are server-owned — **never pass a request AbortSignal into them**; a client
|
||||
closing its tab must not kill a fetch. Downloads still run inside `withSaveSlot`.
|
||||
- **Validation gate** (`validateMedia`): h264 yuv420p + AAC, container/stream durations
|
||||
cover the source, a full `-c copy` demux pass with zero stderr, head + tail decode.
|
||||
The demux pass is what catches a faststart file whose index is intact but whose
|
||||
data is cut. Nothing reaches the cache dir without passing it; failures back off
|
||||
(15 min · 2^n, max 24 h) and saves fall back to the legacy per-request path.
|
||||
- `gen` is in the filename and URL (`/api/media/<id>?g=<gen>`): a URL never serves
|
||||
different bytes. A replaced gen is kept 30 min so in-flight Range playback survives.
|
||||
- **Compression lane**: niced x264 `-preset slow -crf 28 -fpsmax 30`, audio copied,
|
||||
kept only if ≥15% smaller. Measured: YouTube's 720p avc1 is already ~0.6–0.7 Mbps, so
|
||||
CRF 24 made files *bigger*; CRF 28 ≈ −20% video at SSIM 0.994. Lyric videos usually
|
||||
keep the original. HEVC/AV1/10-bit are smaller but don't play in iOS `<video>`.
|
||||
- **Budget**: `MEDIA_CACHE_MAX_BYTES` (10 GiB) LRU by `last_access` (anything played in
|
||||
the last 10 min is protected) + `MEDIA_MIN_FREE_BYTES` (5 GiB) disk guard → skip, stream.
|
||||
- **Broken**: `POST /api/media/:id/redownload` (now-playing ⚠ Broken button). When a
|
||||
client falls back from a cached copy (`/api/streams?nocache=1`) the server also
|
||||
re-validates its copy and refetches it if it fails — never trusts one device's error.
|
||||
- Ops: `curl https://worship.hesed.sbs/api/media/stats`; files under
|
||||
`/app/data/media` in the container.
|
||||
|
||||
## Local dev
|
||||
|
||||
@@ -129,6 +158,8 @@ Local DB file: `server/data/ytplayer.db` (gitignored). `BUILD_TAG` is computed f
|
||||
|
||||
## Testing
|
||||
- Unit: **`node --test frontend/*.test.js`** (sw, sw-update, async-guard, video-edit).
|
||||
Server: **`cd server && bun test`** (media-cache: real ffmpeg fixtures + a temp libsql DB).
|
||||
Local e2e needs a CURRENT yt-dlp — a 2-month-old one 403s on every download.
|
||||
The directory form `node --test frontend/` fails on Node 22 with
|
||||
`Cannot find module .../frontend` — it resolves the dir as a module, not a test
|
||||
glob. That's the harness, not the tests.
|
||||
|
||||
Reference in New Issue
Block a user