ytplayer — project facts for agents

What this is

Ad-free YouTube player. Three shells share frontend/: web PWA (Bun + Hono in server/, the production deployment), Tauri (Windows), zero-native (Linux/macOS). The PWA is what runs in production; legacy/ holds the old native-only docs.

Repo map

Path	What lives there
`frontend/app.js`	The entire UI (~3.6k lines, no framework): player, playlists, search, settings, sync
`frontend/sw.js` + `frontend/sw-update.js`	Service worker + update flow (FRAGILE — see below)
`frontend/opfs.js` / `opfs-worker.js`	OPFS offline audio cache
`frontend/fingerprint.js`	Browser fingerprint used as the sync key
`frontend/async-guard.js`	Stale-async-response guard (unit-tested)
`server/server.js`	Bun + Hono backend — endpoint list is in its header comment
`server/db.js`	libsql schema/queries (`users`, `profiles`, `video_history`, `media_cache`…)
`server/media-cache.js`	Server-side video cache: background fetch jobs, validation gate, x264 lane, LRU
`bin/yt-dlp`	Downloaded by `npm run setup`, gitignored
`scripts/`	icon generation, yt-dlp setup, push helper
`tests/`	Playwright e2e specs · unit tests live next to sources in `frontend/*.test.js`
`legacy/`	Old native-shell docs — do not treat as current

API endpoints (notes/admin routes are listed in the header of server/notes.js): GET /api/search|channel|streams|download/:id|version|user/data|profile/load, GET /api/media/:id|media/:id/status|media/stats, POST /api/media/:id/redownload, POST /api/user/sync|profile/create|profile/save, GET /sw.js (BUILD_TAG-injected), GET /* static. JSON shapes mirror the Tauri Rust bridge exactly — don't change one side alone (/api/streams' data.serverCached is additive and web-only).

Server media cache (`server/media-cache.js`)

Every played (/api/streams, LOW priority, ≤ MEDIA_AUTO_MAX_SECONDS) or saved (/api/download, HIGH, ≤ 3 h) video gets ONE copy: $MEDIA_DIR/<id>.<gen>.mp4 (≤720p H.264 8-bit + AAC, faststart) + <id>.<gen>.m4a audio sidecar for audio-only mode. MEDIA_DIR defaults to ./data/media (the ytplayer-data volume in prod).
Jobs are server-owned — never pass a request AbortSignal into them; a client closing its tab must not kill a fetch. Downloads still run inside withSaveSlot.
Validation gate (validateMedia): h264 yuv420p + AAC, container/stream durations cover the source, a full -c copy demux pass with zero stderr, head + tail decode. The demux pass is what catches a faststart file whose index is intact but whose data is cut. Nothing reaches the cache dir without passing it; failures back off (15 min · 2^n, max 24 h) and saves fall back to the legacy per-request path.
gen is in the filename and URL (/api/media/<id>?g=<gen>): a URL never serves different bytes. A replaced gen is kept 30 min so in-flight Range playback survives.
Compression lane (rev OPT_REV = 2): niced x265 8-bit -preset medium -crf 28, hvc1 tag, anime-style tuning (X265_TUNING), -fpsmax 30, audio copied — kept only if ≥15% smaller, and the copy's vcodec becomes hevc. Sample-first: videos ≥60 s get two 10-s windows encoded first and compared with the source's own packet bytes over the same timestamps; the full encode only runs if the prediction clears the 15% bar (a skip costs ~20 s of video; a full encode ~2× realtime on the homelab). Why: re-encoding YouTube's already-compressed H.264 usually needs MORE bits for LOWER quality (generation loss) — on prod 0 of 5 videos shrank at CRF 28, one grew 25% (VMAF 93.6 at 126% of source; capping to 60–70% drops VMAF to ~84–87). Only videos where YouTube over-spent bits win (a live worship video: −34%). AnimeOut-style sizes come from pristine Blu-ray sources, which YouTube never gives us. Bumping OPT_REV re-queues every cached copy once at boot.
HEVC gating: an HEVC copy is served only to clients that send ?hevc=1 on /api/streams and /api/download (app.js hevcCapable() = canPlayType hvc1, minus a per-device ytpNoHevc flag set when an HEVC copy fails to play). Everyone else gets the pre-cache behaviour (YouTube proxy / legacy H.264 save). Verified: iPhone XR, Galaxy S10, Windows Chrome play HEVC; AV1 fails on the XR (no decoder), so it isn't used. The validation gate requires hvc1 (Safari refuses hev1) and 8-bit 4:2:0.
Budget: MEDIA_CACHE_MAX_BYTES (10 GiB) LRU by last_access (anything played in the last 10 min is protected) + MEDIA_MIN_FREE_BYTES (5 GiB) disk guard → skip, stream.
Broken: POST /api/media/:id/redownload (now-playing ⚠ Broken button). When a client falls back from a cached copy (/api/streams?nocache=1) the server also re-validates its copy and refetches it if it fails — never trusts one device's error.
Ops: curl https://worship.hesed.sbs/api/media/stats; files under /app/data/media in the container.

Lyrics · chapters · bookmarks (`server/notes.js`, `frontend/lyrics-core.js`)

Shared per video (every user sees the same copy): video_notes holds the live lyrics / chapters doc per (video_id, kind); every save also lands in video_note_revs as a full snapshot (history + undo). A daily JSON dump of all live notes goes to <DB dir>/backups/notes-YYYY-MM-DD.json (30 kept).
Who writes: a user linked to an online profile (body profile must exist — same trust level as the profile API), a script with an API token (Authorization: Bearer ytp_…, only a SHA-256 is stored), or the admin (signed httpOnly cookie). Saves are optimistic: client sends baseRev, gets 409 + current if someone saved in between (UI asks "Keep mine / Load theirs").
Personal, not shared: bookmarks (data.bookmarks[videoId]) and the per-song sync offset (data.lyricOffsets[videoId]) — both ride the profile sync. Line shows at t + doc.offset + personal offset (positive = later).
Editor text format = LRC superset: [1:23.45] line, # Section, ! band cue, @ Key G, 70 BPM tags, multi-stamp LRC lines and [offset:±ms] accepted.
Captions: GET /api/notes/:id/captions (preview) and token/admin-only POST /api/notes/:id/lyrics/auto pick human subs first, else the auto track in the spoken language (*-orig), never a machine translation. Auto captions of music are mostly junk ("oh", "n to") — expect to hand-fix or paste lyrics.
Admin: /admin (frontend/admin.html, never cached by the SW) — tokens, edit feed with view/restore, caption injection, JSON export, API examples. Disabled unless ADMIN_PASSWORD is set (Dokploy Environment tab; compose passes ${ADMIN_PASSWORD:-}). Login is rate-limited per IP.
Offline: the last-seen notes per video are cached in localStorage (ytpNotesCache, 200 videos), so saved songs keep their lyrics offline.
Toast cap bug (fixed with this feature): toast() used while (children.length > 3) with a deferred remove — the 4th toast spun forever and Chrome killed the tab ("Target crashed" in Playwright, no JS error). Never cap with a loop whose exit depends on a deferred removal.

Local dev

npm run setup            # download bin/yt-dlp (once)
cd server && bun install
ln -s ../frontend public # once — the server serves ONLY ./public (Docker copies frontend/ there)
bun --hot server.js      # http://localhost:3000
node --test frontend/    # unit tests (run from repo root)
npx playwright test      # e2e (see Testing below)

Local DB file: server/data/ytplayer.db (gitignored). BUILD_TAG is computed from ./public contents.

Production deployment (web PWA)

URL: https://worship.hesed.sbs (Traefik label in docker-compose.yml)
Runs on the homelab Dokploy remote node; control plane is Dokploy on the VPS (193.160.119.172, API key in ~/development/.secrets/dokploy-api.env).
Compose ID: wprYCM8T51f7JtSHb983p (project ytplayer, env production).

Pushing to git does NOT deploy. Trigger explicitly (build ≈ 5–6 min):

ssh -i ~/.ssh/tmp_vps/dokploy_session root@193.160.119.172 \
  "curl -s -X POST -H 'x-api-key: $KEY' -H 'Content-Type: application/json' \
   -d '{\"composeId\":\"wprYCM8T51f7JtSHb983p\"}' http://localhost:3000/api/compose.deploy"
# poll composeStatus via /api/compose.one?composeId=... until done|error

Confirm the deploy landed: curl https://worship.hesed.sbs/api/version — the buildTag (content hash of every file under ./public) must change.
Homelab node is NOT always reachable on LAN; SSH via the VPS hop: ssh root@193.160.119.172 → ssh root@10.8.0.2 (WireGuard). Container name: ytplayer-main-1dihzn-ytplayer-1. DB: libsql file /app/data/ytplayer.db (query with docker exec <c> bun -e '...' using @libsql/client).

Update-flow architecture (fragile — read before touching)

GET /sw.js is served by the server with the real BUILD_TAG injected by regex over the fallback expression in frontend/sw.js. Never switch back to an exact-string replace: when the fallback literal was bumped (v1.0.3→v1.0.4) the exact match silently failed, the SW version froze, and no client ever received another update while /api/version kept announcing one — the "Update available keeps showing" bug.
BUILD_TAG hashes every file under ./public recursively. Don't reduce it to a file subset; a change to an unlisted shell file would stop busting caches.
The banner has ONE rule (maybeShowUpdateBanner in app.js): show it only when the build this page runs differs from /api/version. The server stamps the running build into index.html (<meta name="ytp-build">, at request time like sw.js's BUILD_TAG); the SW caches that index.html with the rest of the shell, so the meta always describes the code in the tab. The poll, a waiting worker and the SW_UPDATE_AVAILABLE broadcast are only prompts to re-check. This loop was "fixed" three times by chasing individual triggers — don't add a trigger that calls showUpdateBanner() directly.
Refresh UI (frontend/sw-update.js): refreshShellInPlace() re-downloads every file the shell caches hold — cache-busted (?__ytpfresh=) so even an OLD worker's cache-first handler can't answer from its cache, 3 tries per file, all-or-nothing — and writes them into every versioned shell cache; then it activates a waiting worker (SKIP_WAITING) if any and reloads once. A failed download shows an error toast and re-offers later; it never reloads into the old shell. checkUpdateOutcome() verifies after the reload (sessionStorage, max 3 attempts) instead of looping.
Why it kept looping on prod: the VPS→homelab link is slow and drops requests. The SW install (cache.addAll, all-or-nothing) failed, Refresh UI reloaded into the old cached shell, and the partial ytplayer-<tag> cache left by the failed install later made activate broadcast "update available" to pages that were already current. sw.js now precaches with cache: 'reload', retries each file, and deletes its partial cache on failure.
Reproduce with the throttling/stalling proxy approach: WebKit (Playwright on the Windows side) or Chrome over CDP against a scratch copy of the server, stalling every Nth shell request after "deploying" v2. A fast local link never shows the bug.

Cache layout & offline thumbnails

Three cache families, and the split matters on activate: the versioned shell cache ytplayer-<BUILD_TAG> is evicted on every deploy, while the utility caches ytplayer-thumbs and ytplayer-fonts are listed in UTILITY_CACHES and deliberately survive it. Adding a new utility cache means adding it there too, or it gets wiped on the next deploy.
Thumbnails are cache-first, not stale-while-revalidate — a given thumbnail URL is immutable, so revalidating just burns a round trip per image per launch.
The opaque-response trap (this silently emptied the thumb cache for months). An <img> to another origin is a no-cors request, so fetch(request) in the SW resolves to an opaque response with status === 0 — not 200. The old guard was if (r.status === 200) cache.put(...), which rejected every single thumbnail, so ytplayer-thumbs was permanently empty and offline showed broken images (measured on prod: 0 entries after browsing pages full of visible thumbs). Fix in thumbnail(): re-issue the request in cors mode — ytimg/ggpht all send Access-Control-Allow-Origin: * — and cache that readable response; an opaque one is accepted only as a last resort. Never reintroduce a bare status === 200 check on a cross-origin subresource.
Thumbnail hosts live in THUMB_HOSTS. An unlisted host doesn't error — it just bypasses the cache and breaks offline, so add mirrors/avatar hosts there.
The thumb cache is capped at THUMB_CACHE_MAX (800, oldest-first via the insertion-ordered cache.keys()). Keep a cap: CacheStorage and the OPFS offline audio share one origin quota, and opaque entries are padded to ~7 MB each for quota accounting, so an unbounded thumb cache can evict saved audio.
App side (app.js): warmThumb() pulls artwork through the SW when a video is saved offline, and warmOfflineThumbs() runs a bounded launch backfill (THUMB_WARM_MAX = 400, 4 at a time) over cached ids + playlist videos so libraries saved before this fix repair themselves. Thumbnails only cache when something requests them — a device needs one online launch to get offline art.
staleWhileRevalidate() (fonts) must not return cached || networkFetch bare: the fetch resolves to null offline and respondWith(null) throws. It ends with || Response.error().

Data model quirks

Client state persists in localStorage key _ytpdata and syncs (debounced 400 ms) to POST /api/user/sync, keyed by a browser fingerprint.
A-B loop markers are per-song-per-playlist: stored on the playlist's own copy of the video (entry.ab = {a, b}) when playback source is that playlist; data.abMarkers[videoId] is only the fallback for non-playlist playback.
Online profiles (profiles table, /api/profile/*): named cross-device sync where the lowercase profile NAME is the only credential (passkey-style, by design). Client stores data.profile = {name, syncedAt}; sync is last-write-wins — push debounced on every persist(), pull on app launch when the server's updated_at is newer than the local syncedAt.
Profile share links: ?profile=<name> is consumed by adoptProfileFromUrl() at boot, before any rendering. It strips the param via replaceState (so a reload can't re-fire it) and confirms first when the device already has playlists/history or another profile — adopting replaces the synced slice. The name comes off the URL untrusted, hence escapeHtml() on it.
Empty-home playlist grid: renderHomePlaylists() swaps the branding hero in #playerPlaceholder for the user's playlists (hero is the no-playlists fallback). It is driven from the tail of renderSidebar() — not from render() — so every playlist mutation refreshes both in one place.

Testing

Unit: node --test frontend/*.test.js (sw, sw-update, async-guard, video-edit). Server: cd server && bun run test — runs each file in its own process (db.js is a singleton, so two test files in one bun test run share one temp DB; bare bun test also drags in the frontend node tests via the public symlink). media-cache needs a 60 s hook timeout for its ffmpeg fixtures. Local e2e needs a CURRENT yt-dlp — a 2-month-old one 403s on every download. The directory form node --test frontend/ fails on Node 22 with Cannot find module .../frontend — it resolves the dir as a module, not a test glob. That's the harness, not the tests.
E2E: npx playwright test — WebKit iPhone-12 profile against a static serve of frontend/ (needs npx playwright install webkit). A spurious update banner will make the settings-panel specs fail with #modal intercepts pointer events — that failure mode is a real app bug, not test flake. On the WSL laptop the WebKit install fails host validation (missing libgtk-4, libgstreamer*, …, needs sudo npx playwright install-deps); chromium is already downloaded, so ad-hoc rendering/offline checks can drive it directly instead.
Service-worker behaviour (thumb caching, offline) is only provable in a real browser: serve frontend/ statically, let the SW take control (needs a second reload), then context.setOffline(true) and assert img.naturalWidth > 0 plus the ytplayer-thumbs entry count. Asserting against the browser's own HTTP cache proves nothing — check the Cache API entry count.
Test records on prod use Probe */Recon * names; clean via the container DB, children (video_history) first.

Harness

Skills live in .agents/skills/ (symlinked into .claude/skills/):

deploy-prod — the Dokploy deploy + buildTag verification flow (manual-only; use for any "deploy"/"release" request).
mobile-app-ui-design — UI/UX design work on the PWA screens.

Commit rules

One changeset = one commit, single-line imperative message, no AI attribution of any kind (global rule). git push origin main pushes to both remotes.

17 KiB Raw Blame History Unescape Escape