From 5fb863ba257331b228b9c19a8e7da825fd3951a1 Mon Sep 17 00:00:00 2001 From: Jonathan Sykes Date: Sun, 13 Sep 2026 09:45:01 +0800 Subject: [PATCH] Cache every played or saved video on the server via background jobs with a validation gate and a Broken re-download button --- CLAUDE.md | 35 ++- docker-compose.yml | 9 + frontend/app.js | 78 +++++- frontend/index.html | 1 + server/db.js | 102 +++++++ server/media-cache.js | 556 +++++++++++++++++++++++++++++++++++++ server/media-cache.test.js | 249 +++++++++++++++++ server/server.js | 230 ++++++++++++++- 8 files changed, 1240 insertions(+), 20 deletions(-) create mode 100644 server/media-cache.js create mode 100644 server/media-cache.test.js diff --git a/CLAUDE.md b/CLAUDE.md index dae73e8..431c919 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,15 +15,44 @@ The PWA is what runs in production; `legacy/` holds the old native-only docs. | `frontend/fingerprint.js` | Browser fingerprint used as the sync key | | `frontend/async-guard.js` | Stale-async-response guard (unit-tested) | | `server/server.js` | Bun + Hono backend — endpoint list is in its header comment | -| `server/db.js` | libsql schema/queries (`users`, `profiles`, `video_history`…) | +| `server/db.js` | libsql schema/queries (`users`, `profiles`, `video_history`, `media_cache`…) | +| `server/media-cache.js` | Server-side video cache: background fetch jobs, validation gate, x264 lane, LRU | | `bin/yt-dlp` | Downloaded by `npm run setup`, gitignored | | `scripts/` | icon generation, yt-dlp setup, push helper | | `tests/` | Playwright e2e specs · unit tests live next to sources in `frontend/*.test.js` | | `legacy/` | Old native-shell docs — do not treat as current | API endpoints: `GET /api/search|channel|streams|download/:id|version|user/data|profile/load`, +`GET /api/media/:id|media/:id/status|media/stats`, `POST /api/media/:id/redownload`, `POST /api/user/sync|profile/create|profile/save`, `GET /sw.js` (BUILD_TAG-injected), `GET /*` static. -JSON shapes mirror the Tauri Rust bridge exactly — don't change one side alone. +JSON shapes mirror the Tauri Rust bridge exactly — don't change one side alone +(`/api/streams`' `data.serverCached` is additive and web-only). + +## Server media cache (`server/media-cache.js`) +- Every played (`/api/streams`, LOW priority, ≤ `MEDIA_AUTO_MAX_SECONDS`) or saved + (`/api/download`, HIGH, ≤ 3 h) video gets ONE copy: `$MEDIA_DIR/..mp4` + (≤720p H.264 8-bit + AAC, faststart) + `..m4a` audio sidecar for audio-only + mode. `MEDIA_DIR` defaults to `./data/media` (the `ytplayer-data` volume in prod). +- Jobs are server-owned — **never pass a request AbortSignal into them**; a client + closing its tab must not kill a fetch. Downloads still run inside `withSaveSlot`. +- **Validation gate** (`validateMedia`): h264 yuv420p + AAC, container/stream durations + cover the source, a full `-c copy` demux pass with zero stderr, head + tail decode. + The demux pass is what catches a faststart file whose index is intact but whose + data is cut. Nothing reaches the cache dir without passing it; failures back off + (15 min · 2^n, max 24 h) and saves fall back to the legacy per-request path. +- `gen` is in the filename and URL (`/api/media/?g=`): a URL never serves + different bytes. A replaced gen is kept 30 min so in-flight Range playback survives. +- **Compression lane**: niced x264 `-preset slow -crf 28 -fpsmax 30`, audio copied, + kept only if ≥15% smaller. Measured: YouTube's 720p avc1 is already ~0.6–0.7 Mbps, so + CRF 24 made files *bigger*; CRF 28 ≈ −20% video at SSIM 0.994. Lyric videos usually + keep the original. HEVC/AV1/10-bit are smaller but don't play in iOS `