40 KiB
ytplayer — project facts for agents
What this is
Ad-free YouTube player. Three shells share frontend/: web PWA (Bun + Hono in
server/, the production deployment), Tauri (Windows), zero-native (Linux/macOS).
The PWA is what runs in production; legacy/ holds the old native-only docs.
Repo map
| Path | What lives there |
|---|---|
frontend/app.js |
The entire UI (~3.6k lines, no framework): player, playlists, search, settings, sync |
frontend/sw.js + frontend/sw-update.js |
Service worker + update flow (FRAGILE — see below) |
frontend/opfs.js / opfs-worker.js |
OPFS offline audio cache |
frontend/fingerprint.js |
Browser fingerprint used as the sync key |
frontend/async-guard.js |
Stale-async-response guard (unit-tested) |
server/server.js |
Bun + Hono backend — endpoint list is in its header comment |
server/db.js |
libsql schema/queries (users, profiles, video_history, media_cache…) |
server/media-cache.js |
Server-side video cache: background fetch jobs, validation gate, x264 lane, LRU |
bin/yt-dlp |
Downloaded by npm run setup, gitignored |
scripts/ |
icon generation, yt-dlp setup, push helper |
tests/ |
Playwright e2e specs · unit tests live next to sources in frontend/*.test.js |
legacy/ |
Old native-shell docs — do not treat as current |
API endpoints (notes/admin routes are listed in the header of server/notes.js): GET /api/search|channel|streams|download/:id|version|user/data|profile/load,
GET /api/media/:id|media/:id/status|media/stats, POST /api/media/:id/redownload,
POST /api/user/sync|profile/create|profile/save, GET /sw.js (BUILD_TAG-injected), GET /* static.
JSON shapes mirror the Tauri Rust bridge exactly — don't change one side alone
(/api/streams' data.serverCached is additive and web-only).
Long videos (1 h+)
- Bun's default request-body cap is 128 MB —
Bun.servesetsmaxRequestBodySizeto 5 GiB; routes enforce their own limits (uploads 4 GB, intakeP2P_INTAKE_MAX_BYTES3 GiB). Without it every large upload/intake died with 413. - Admin uploads use
PUT /api/admin/uploads/stream?name=…(raw body → disk in chunks, browser sends theFileitself); cover images go toPUT /api/admin/uploads/:id/art. The multipartPOST /api/admin/uploadsparses the whole body in memory — scripts with small files only. - Videos over
transcode.maxSecondsare never re-encoded (lane would be tied up for hours). - Deferred ideas (Cloudflare Worker fetch, Android device download, ffmpeg.wasm client editing):
docs/deferred-ideas.md.
Save to device + "On other devices"
exportToDevice(video)(app.js, helpers infrontend/export.js): puts the file in Photos / Files / Downloads. Source order: copy saved in OPFS → server copy (GET /api/export/:id?name=, attachment + Range; direct download on desktop, via OPFS + share sheet on phones) → online P2P holder → normal save via server. Buttons appear only once the file is ready:navigator.shareneeds a fresh tap.GET /api/p2p/available+ view{type:'p2p'}= the "📡 On other devices" list (sidebar, web only): verified files with ≥1 sharing holder, online first, stale flagged. Rows play, export, or go to a playlist.
Layout themes + Glass Stage
- Settings → Layout theme sets
html[data-layout]; rules live instyles.css(older ones) andfrontend/theme-glass.css(glass-stage: frosted cards, coral glow). Glass Stage also changes layout: overlay on the video (quality chip, Video / Audio-only, −10 / play / +10, shown when paused or after a tap), deck row prev · −10 · play · +10 · next, volume pod + labelled mode pill, quick-setting chips (#deckChips, refreshed every 1.5 s by theglassStageIIFE), four stacked icon tiles and secondary actions as a pill row, floating mini player. Elements that only this theme shows (.stg-overlay,.skip10,.deck-chips,.vol-podextras,.mode-name) are in the shared markup and hidden by default. - Now-playing tiles are
[svg.ti][span.tl]; change state text withsetTile(btn, label), nevertextContent.
Saved searches + caching notes
SearchLibrary(app.js) keeps every search: metadata index in localStorageytpSearchIndex(sync, ~250 KB), result lists in IndexedDBytpSearchLibrary(≤1000 queries × 100 results incl. thumbnail URLs;get()is async). A saved copy is trusted 15 days (FRESH_MS): a fresh one is shown with NO network call, an older one is shown then refreshed. The "Update search" button (#searchMeta) callsAPI.search(q,{refresh:true})→/api/search?refresh=1which skips every server cache. The search box's recent list and its "Clear this list" only touchytpRecentSearches; results are deleted ONLY in Settings → Saved searches. Thumbnails are warmed into the SWytplayer-thumbscache (THUMB_CACHE_MAX25000, trimmed every 100th put —cache.keys()is costly at that size). Browse view:{type:'savedSearches'}. Web Workers were NOT added: IndexedDB/fetch are already async off-thread I/O and postMessage would clone the same data; DOM rendering can't move to a worker.- Server
/api/search(server/search-cache.js, tablessearch_cache+video_meta): memory (200, 10 min fresh) → libsql (up to 200 YouTube cards per query viainnertube.searchDeepcontinuation paging; ≤250,000 queries ANDSEARCH_CACHE_MAX_BYTES2 GiB, LRU) → YouTube. Stale copies are served for 15 days while refreshing in the background. Every card seen is also stored invideo_meta(≤500k);GET /api/search/local?q=matches known videos by title/channel (all words) + uploads in ms, and the app paints them while the real search loads. yt-dlp fallback still returns 25. - Static files: index.html is stamped with
?v=<BUILD_TAG>on local css/js (indexHtml()); a request whosevequals the running build isimmutable, max-age=1y, anything elseno-cache. The SWcacheFirstmatches?v=URLs withignoreSearchagainst the plain-URL precache. A new build = new URLs, so old ones simply get evicted./fontsand/iconsare cached 30 days.
SHA-256 in WebAssembly (frontend/sha256.js)
Sha256.create()runs its block function as WebAssembly (frontend/wasm/sha256.c→scripts/build-sha256-wasm.sh→ generatedfrontend/sha256-wasm.js, base64, ~2 KB) and falls back to pure JS if WebAssembly is missing, the CSP refuses it, or the load-time self-test ("abc") fails. ~150 MB/s vs ~65 MB/s in JS (Node/Chromium; phones are slower but the ratio holds). It is synchronous on purpose (callers aren't async) — fine because the module is under 4 KB.- Needs
'wasm-unsafe-eval'in index.html's CSPscript-src. Workers pull the blob in withimportScripts('/sha256-wasm.js')from inside sha256.js; the file is in the SWSHELLlist. After editing the C file, re-run the build script and commit both.Sha256.engine()says which one is active;create({js:true})forces JS (tests).
Server media cache (server/media-cache.js)
- Every played (
/api/streams, LOW priority, ≤MEDIA_AUTO_MAX_SECONDS, default 3 h) or saved (/api/download, HIGH, ≤ 3 h) video gets ONE copy:$MEDIA_DIR/<id>.<gen>.mp4(≤720p H.264 8-bit + AAC, faststart) +<id>.<gen>.m4aaudio sidecar for audio-only mode.MEDIA_DIRdefaults to./data/media(theytplayer-datavolume in prod). - Jobs are server-owned — never pass a request AbortSignal into them; a client
closing its tab must not kill a fetch. Downloads still run inside
withSaveSlot. - Validation gate (
validateMedia): h264 yuv420p + AAC, container/stream durations cover the source, a full-c copydemux pass with zero stderr, head + tail decode. The demux pass is what catches a faststart file whose index is intact but whose data is cut. Nothing reaches the cache dir without passing it; failures back off (15 min · 2^n, max 24 h) and saves fall back to the legacy per-request path. genis in the filename and URL (/api/media/<id>?g=<gen>): a URL never serves different bytes. A replaced gen is kept 30 min so in-flight Range playback survives.- Compression lane (rev
OPT_REV= 2): niced x265 8-bit-preset medium -crf 28,hvc1tag, anime-style tuning (X265_TUNING),-fpsmax 30, audio copied — kept only if ≥15% smaller, and the copy'svcodecbecomeshevc. Sample-first: videos ≥60 s get two 10-s windows encoded first and compared with the source's own packet bytes over the same timestamps; the full encode only runs if the prediction clears the 15% bar (a skip costs ~20 s of video; a full encode ~2× realtime on the homelab). Why: re-encoding YouTube's already-compressed H.264 usually needs MORE bits for LOWER quality (generation loss) — on prod 0 of 5 videos shrank at CRF 28, one grew 25% (VMAF 93.6 at 126% of source; capping to 60–70% drops VMAF to ~84–87). Only videos where YouTube over-spent bits win (a live worship video: −34%). AnimeOut-style sizes come from pristine Blu-ray sources, which YouTube never gives us. BumpingOPT_REVre-queues every cached copy once at boot. - HEVC gating: an HEVC copy is served only to clients that send
?hevc=1on/api/streamsand/api/download(app.jshevcCapable()= canPlayType hvc1, minus a per-deviceytpNoHevcflag set when an HEVC copy fails to play). Everyone else gets the pre-cache behaviour (YouTube proxy / legacy H.264 save). Verified: iPhone XR, Galaxy S10, Windows Chrome play HEVC; AV1 fails on the XR (no decoder), so it isn't used. The validation gate requireshvc1(Safari refuseshev1) and 8-bit 4:2:0. - Budget:
MEDIA_CACHE_MAX_BYTES(10 GiB) LRU bylast_access(anything played in the last 10 min is protected) +MEDIA_MIN_FREE_BYTES(5 GiB) disk guard → skip, stream. - Broken:
POST /api/media/:id/redownload(now-playing ⚠ Broken button). When a client falls back from a cached copy (/api/streams?nocache=1) the server also re-validates its copy and refetches it if it fails — never trusts one device's error. - Ops:
curl https://worship.hesed.sbs/api/media/stats; files under/app/data/mediain the container.
Lyrics · chapters · bookmarks (server/notes.js, frontend/lyrics-core.js)
- Shared per video (every user sees the same copy):
video_notesholds the live lyrics / chapters doc per(video_id, kind); every save also lands invideo_note_revsas a full snapshot (history + undo). A daily JSON dump of all live notes goes to<DB dir>/backups/notes-YYYY-MM-DD.json(30 kept). - Who writes: a user linked to an online profile (body
profilemust exist — same trust level as the profile API), a script with an API token (Authorization: Bearer ytp_…, only a SHA-256 is stored), or the admin (signed httpOnly cookie). Saves are optimistic: client sendsbaseRev, gets 409 +currentif someone saved in between (UI asks "Keep mine / Load theirs"). - Personal, not shared: bookmarks (
data.bookmarks[videoId]) and the per-song sync offset (data.lyricOffsets[videoId]) — both ride the profile sync. Line shows att + doc.offset + personal offset(positive = later). - Editor text format = LRC superset:
[1:23.45] line,# Section,! band cue,@ Key G, 70 BPMtags, multi-stamp LRC lines and[offset:±ms]accepted. - Captions:
GET /api/notes/:id/captions(preview) and token/admin-onlyPOST /api/notes/:id/lyrics/autopick human subs first, else the auto track in the spoken language (*-orig), never a machine translation. Auto captions of music are mostly junk ("oh", "n to") — expect to hand-fix or paste lyrics. - Admin:
/admin(frontend/admin.html, never cached by the SW) — tokens, edit feed with view/restore, caption injection, JSON export, API examples. Disabled unlessADMIN_PASSWORDis set (Dokploy Environment tab; compose passes${ADMIN_PASSWORD:-}). Login is rate-limited per IP. - Service mode → 🎵 Lyrics (
data.settings.serviceLyrics,serviceLyricsAutoscroll): lyrics-only view; the sung line is centred, highlighted and sized to the largest font that fits ONE row. Sizing is measured with a hidden real.sl-linein TWO passes — Bricolage Grotesque has an optical-size axis, so glyphs get relatively wider at small sizes and a single 100px measurement over-fits on phones. Re-fit (in place, never rebuilding rows — that would move the scroll position) on resize,document.fontsload, and +900 ms. Lyrics-only and the 📺 video panel are mutually exclusive. - Free batch transcription:
scripts/lyrics/auto_lyrics.py(faster-whisperlarge-v3-turboint8 on CPU, no API key/credits; venv at~/.local/share/lyrics-asr/.venv).--missingusesGET /api/admin/media(admin or token). Measured on the 16-core laptop: ~0.3–0.65× real time; openai-whisper medium was ~2.5×.vad_filtermust stay False — VAD calls sung music non-speech and returns an empty transcript. Karaoke/minus-one tracks have no vocals (reported "instrumental?"); their lyrics were read from the on-screen text (frame sampling + OCR), not audio. ElevenLabs Scribe also works but spends the TTS credit balance (free tier = 10k/month). - Offline: the last-seen notes per video are cached in localStorage
(
ytpNotesCache, 200 videos), so saved songs keep their lyrics offline. - Toast cap bug (fixed with this feature):
toast()usedwhile (children.length > 3)with a deferred remove — the 4th toast spun forever and Chrome killed the tab ("Target crashed" in Playwright, no JS error). Never cap with a loop whose exit depends on a deferred removal.
Playlist transitions + waveform (Transition / Wave in app.js)
data.settings.transition=off|gapless|crossfade(crossfadeSec, default 6); the ↦/⇥/⤨ button cycles it with a 6-s toast. Applies only to playlist/queue playback with a next song, not with loop-one or an A-B loop.- A hidden bridge
<audio>preloads the next song's audio ~25 s early (offline copy, else/api/streams). Crossfade starts itcrossfadeSecearly and ramps both volumes; gapless starts it when the song ends. Then the queue advances normally withPlayer._handoffset:_startPlayback()seeks the new track to the bridge's position, plays, and releases the bridge onplaying(failsafe 5 s / 20 s). afterLoad skips resume/A-marker seeks during a handoff;onTrackEndedignoresendedwhile one runs. - iOS ignores
element.volume, so crossfade degrades to gapless there. - End-of-song race (fixed): the song's last
timeupdatearrives with the element alreadypaused(it ended), andendedcan beat the fade timer. Both used to look like "user paused" → the crossfade was cancelled and the next song restarted from 0.tick()andonTrackEnded(Transition.onEnded()) now finish the handoff when a fade is running at the end. - Waveform:
GET /api/media/:id/peaks= 400 RMS buckets (0..100, ^0.7 so quiet verses stay visible) from ffmpeg over the server-cached copy, memoised per file path (gen). Audio-only mode draws it on#waveCanvasBEHIND the range input (made transparent, still the control), + hover time tooltip and "⚡ Jump to the loudest part" (loudest ~5 s window). Not cached on the server → plain seek bar. Envelopes are kept in localStorage (ytpPeaks, 80 songs). - Tested with real audio: a local server whose media_cache holds 20-s clips
(upsertMedia rows +
<id>.1.m4a/.mp4files) — Playwright's Chromium plays AAC; launch with--autoplay-policy=no-user-gesture-required.
Uploads + web lyrics
- Uploads (
server/uploads.js,UPLOAD_DIR, tableuploads): an admin posts a file to/api/admin/uploads(multipart, ≤ 4 GB, admin cookie or API token). ffprobe runs ONCE and gives everything: title/artist/album tags, duration, whether there is a REAL video stream (a cover picture also appears as a video stream — checkdisposition.attached_pic), the cover itself (extracted to<id>.art.jpg; an uploaded image wins) and embedded lyrics tags (LYRICS/lyrics-eng/UNSYNCEDLYRICS/©lyr) which are parsed withparseLrcand saved as that song's shared lyrics (synced when in LRC form). Ids areupl_<12 hex>and are accepted anywhere a video id is:/api/streams(audio →audioUrl+ art, video → one "Original" quality),/api/download, peaks/gif/clip (audioSourcePath/videoSourcePath), notes./api/searchputs the library first and still answers when yt-dlp fails. The app: 📁 Library view,isUploadId/isMediaIdhelpers, audio uploads force audio mode so the cover art shows where the video would be. - Web lyrics:
POST /api/notes/:id/lyrics/webasks LRCLIB (free, key-less, often SYNCED) using a cleaned title/artist + duration (cleanTitle/cleanArtist/pickLrclib, ± 6 s). Profile users, tokens and admins may call it; the lyrics worker tries it BEFORE transcribing.scripts/lyrics/web_lyrics.py --missing --agyadds an agy web-search fallback (plain text → untimed lines, to be timed with Tap-sync). Gotcha fixed here: a/gregex reused for.test()skips every other call (lastIndex) —cleanTitlekeeps a stateless copy.
Watch party, sharing, playback extras (all in app.js unless noted)
- Lyrics instead of the video (
Notes.stageLyrics,#stageLyricsinside.player-stage): a per-device view for watch parties and sing-alongs —data.settings.stageLyrics/stageLyricsAutoscroll. It reuses the service-mode machinery (fitLyricLines,LyricsCore.activeIndex, the same.sl-lineclasses) and is driven from the normal playback tick. The<video>is only hidden (visibility), never unloaded, so playback, audio and party sync carry on untouched — that is why it is safe for a guest to switch views mid-party. Toggle from the watch-party panel or the bar inside the stage (hover on desktop, always visible on touch). - Watch party (
server/party.js,Party):/ws/party(Bun has ONE websocket handler — sockets are taggedws.data.hub = 'party'and routed in server.js). Host drives; guests extrapolate the host'sstatefrom the server-stampedts(clock offset fromhello.now), seek when > 1.5 s off, mirror play/pause. Guest actions within the 1.5 s "quiet" window after a sync are ignored; later ones → acmd(if the host allows control) or "off sync" (Re-sync button).playNext/playPrev→Party.intercept, and guests never auto-advance or crossfade. Chat is relayed (≤ 500 chars, 400 ms flood guard, last 60 kept). Voice = WebRTC mesh, the smaller pid offers, STUN only (no TURN — strict NATs won't connect). Host resumes after a reload with thesecretfromhello(party survives 5 min without its host). - Timestamp sharing (
Share):/?v=<id>&t=<sec>deep links (handled at boot viaplayVideoAt), YouTubeyoutu.be/<id>?t=, GIFs fromGET /api/media/:id/gif(palettegen, ≤ 6 s, server-cached videos only) and soundbites fromGET /api/media/:id/clip?fmt=mp3|m4r(m4r ≤ 40 s, written to a temp file so it's a normal faststart MP4 — iPhones reject fragmented ones). - External players (
External): hands out/api/media/<id>(the single-file server copy) via Androidintent:(VLC / MX / chooser), iOSvlc-x-callback, desktopvlc://, or an.m3u(this video or the queue's cached songs). - Notes: bookmarks with
kind:'note'hold multi-line text (≤ 2000) and carryvt/vc(video title/channel); the 📝 Notes view lists them all, exports Markdown. Transcript search = 🔎 tab over the lyrics or the YouTube captions (/api/notes/:id/captions, cached per video). - PiP mirrors the PiP window's play/pause onto the whole player (dual mode's
sound is on
<audio>). EQ: 5 BiquadFilters; media elements are wired in only once a non-flat curve is picked (irreversible), disabled on iOS (Web Audio suspends on lock → would kill background audio). Sleep timer: fade oversleepFades, then pause / "close" (window.close, falls back) / black curtain; "end of this song" also blocks Transition. Gestures: double-tap thirds ±10 s / play, pinch 1–3× (CSS transform), vertical swipes for volume / brightness (CSS filter) only in fullscreen orbody.landscape-fs(inline the page keepstouch-action: pan-yso it still scrolls). - Local media tests: seed
media_cacherows +<id>.1.m4a/.mp4files; the Playwrightchannel: 'chromium'build plays H.264/AAC;--use-fake-device-for-media-streamgives WebRTC a fake mic.
Presenter view, stats, lyrics worker
- Presenter (
Presenterin app.js,/?present=<code>): a lyrics-only projector/TV screen. Pairs through the phone-remote relay like a phone (listen-only); the host'sstatecarriesv.id,cur,rate,pausedandoff(the host's personal lyric offset), and the presenter extrapolates the position between the ~1/s updates. Host modal → "Open presenter window here" opens a popup to drag onto a second display;/api/remote/qr/<code>?kind=presentis its QR. Keys: F / double-click fullscreen, B blank screen. Forced dark tokens so it stays readable if the device is in light theme. fitLyricLines(list)is the shared one-row font fitter for service-mode lyrics and the presenter — re-run it on font load, resize and +900 ms (first fit on a fresh page happens before Bricolage loads).- Stats (
frontend/stats-core.js, pure + tested;StatsTrack+renderStats):data.stats.days[YYYY-MM-DD] = {s, p, songs:{id:plays}}(local days, 400 kept) synced with the profile. Seconds = media-time deltas while playing (seeks/pauses excluded, divided by playback rate); a play = 30 s heard in one load; commits every 60 s / on play / on page hide to avoid profile-push spam. Streak = consecutive days ≥ 5 min (today may still be pending). - lyrics-worker (compose service,
scripts/lyrics/Dockerfile): runsauto_lyrics.py --missing --watch 300forever — one song per cycle, nice 10,cpus: 2,mem_limit: 3g, audio fetched fromhttp://ytplayer:3000over the privatelyricsnetwork (never the WAN). Auth = envLYRICS_WORKER_TOKEN(≥ 24 chars) accepted by notes.js asapi:lyrics-workerwithout a DB row; unset → the worker idles./data/state.jsonremembers instrumentals (never retried) and failures (backoff 15 min·2^n). Model (~1.6 GB) downloads to thelyrics-modelsvolume on first start.
Phone remote (server/remote.js, Remote in app.js)
- Desktop tab playing to a TV = host; phone = remote. Both are browser
tabs of this app, so the server relays over
/ws/remote(Bunserver.upgradein theBun.servefetch wrapper, before Hono). Mockup:docs/mockups/mock-06-remote-control.html(it assumed a LAN host + mDNS; the web app can't do that, so it's a server relay + a 6-digit code instead). - Host secret lives in the desktop's localStorage (
ytpRemoteHost); room id = hash(secret). Pairing (POST /api/remote/pair) is a one-time 6-digit code (10 min) → phone getstoken = HMAC(secret, remoteId)(ytpRemotePair). Nothing about phones is stored server-side: tokens are re-checked against the connected host's secret, so server restarts / desktop reloads keep phones paired, and "Unpair all" (new secret) revokes every phone at once. - The host pushes
state(≤1/s, deduped) andqueue({items, idx}of the livequeue/queueIndex); commands are whitelisted server-side (REMOTE_COMMANDS) and run byrunCommand()on the host with the same functions the UI uses. REMOTE_SAME_NETWORK=1= pairing requires the phone and desktop to share a public IP (firstX-Forwarded-Forhop). Off by default — verify that the VPS→homelab Traefik chain forwards the real client IP before turning it on.- QR:
GET /api/remote/qr/:code(server-side SVG viaqrcode) encodes<origin>/?pair=<code>; the app consumes and strips?pair=at boot.
Local dev
npm run setup # download bin/yt-dlp (once)
cd server && bun install
ln -s ../frontend public # once — the server serves ONLY ./public (Docker copies frontend/ there)
bun --hot server.js # http://localhost:3000
node --test frontend/ # unit tests (run from repo root)
npx playwright test # e2e (see Testing below)
Local DB file: server/data/ytplayer.db (gitignored). BUILD_TAG is computed from ./public contents.
Production deployment (web PWA)
- URL: https://worship.hesed.sbs (Traefik label in
docker-compose.yml) - Runs on the homelab Dokploy remote node; control plane is Dokploy on the VPS
(
193.160.119.172, API key in~/development/.secrets/dokploy-api.env). - Compose ID:
wprYCM8T51f7JtSHb983p(projectytplayer, envproduction). - Pushing to git does NOT deploy. Trigger explicitly (build ≈ 5–6 min):
ssh -i ~/.ssh/tmp_vps/dokploy_session root@193.160.119.172 \ "curl -s -X POST -H 'x-api-key: $KEY' -H 'Content-Type: application/json' \ -d '{\"composeId\":\"wprYCM8T51f7JtSHb983p\"}' http://localhost:3000/api/compose.deploy" # poll composeStatus via /api/compose.one?composeId=... until done|error - Confirm the deploy landed:
curl https://worship.hesed.sbs/api/version— thebuildTag(content hash of every file under./public) must change. - Homelab node is NOT always reachable on LAN; SSH via the VPS hop:
ssh root@193.160.119.172→ssh root@10.8.0.2(WireGuard). Container name:ytplayer-main-1dihzn-ytplayer-1. DB: libsql file/app/data/ytplayer.db(query withdocker exec <c> bun -e '...'using@libsql/client).
Update-flow architecture (fragile — read before touching)
GET /sw.jsis served by the server with the realBUILD_TAGinjected by regex over the fallback expression infrontend/sw.js. Never switch back to an exact-string replace: when the fallback literal was bumped (v1.0.3→v1.0.4) the exact match silently failed, the SW version froze, and no client ever received another update while/api/versionkept announcing one — the "Update available keeps showing" bug.BUILD_TAGhashes every file under./publicrecursively. Don't reduce it to a file subset; a change to an unlisted shell file would stop busting caches.- The banner has ONE rule (
maybeShowUpdateBannerin app.js): show it only when the build this page runs differs from/api/version. The server stamps the running build into index.html (<meta name="ytp-build">, at request time like sw.js's BUILD_TAG); the SW caches that index.html with the rest of the shell, so the meta always describes the code in the tab. The poll, a waiting worker and the SW_UPDATE_AVAILABLE broadcast are only prompts to re-check. This loop was "fixed" three times by chasing individual triggers — don't add a trigger that callsshowUpdateBanner()directly. - Background download + instant swap (added 2026-09-22). A new worker
precaches the whole shell during
install, so by the time the user presses "Refresh UI" the build is usually already on disk.applyUpdateasks the waiting workerCACHE_STATUSover aMessageChannel; when it answersready: true(it re-checks every SHELL url, since a cache can be evicted) the download is skipped entirely and the swap is immediate. Two rules keep this safe: the probe is opt-in (askStatus) — never implicit, because a probe awaiting a reply hangs forever if the caller injects asetTimeoutthat never fires (the existing tests do exactly that) — and any doubt (no reply, timeout, thrown error,ready: false) falls back to the all-or-nothing download below, which is still what guarantees correctness.maybeShowUpdateBanneralso callsprefetchUpdate()→registration.update()so the download starts the moment a new build is seen rather than when the user clicks. - Refresh UI (
frontend/sw-update.js):refreshShellInPlace()re-downloads every file the shell caches hold — cache-busted (?__ytpfresh=) so even an OLD worker's cache-first handler can't answer from its cache, 3 tries per file, all-or-nothing — and writes them into every versioned shell cache; then it activates a waiting worker (SKIP_WAITING) if any and reloads once. A failed download shows an error toast and re-offers later; it never reloads into the old shell.checkUpdateOutcome()verifies after the reload (sessionStorage, max 3 attempts) instead of looping. - Why it kept looping on prod: the VPS→homelab link is slow and drops
requests. The SW install (
cache.addAll, all-or-nothing) failed, Refresh UI reloaded into the old cached shell, and the partialytplayer-<tag>cache left by the failed install later made activate broadcast "update available" to pages that were already current. sw.js now precaches withcache: 'reload', retries each file, and deletes its partial cache on failure. - Reproduce with the throttling/stalling proxy approach: WebKit (Playwright on the Windows side) or Chrome over CDP against a scratch copy of the server, stalling every Nth shell request after "deploying" v2. A fast local link never shows the bug.
Cache layout & offline thumbnails
- Three cache families, and the split matters on activate: the versioned shell
cache
ytplayer-<BUILD_TAG>is evicted on every deploy, while the utility cachesytplayer-thumbsandytplayer-fontsare listed inUTILITY_CACHESand deliberately survive it. Adding a new utility cache means adding it there too, or it gets wiped on the next deploy. - Thumbnails are cache-first, not stale-while-revalidate — a given thumbnail URL is immutable, so revalidating just burns a round trip per image per launch.
- The opaque-response trap (this silently emptied the thumb cache for months).
An
<img>to another origin is a no-cors request, sofetch(request)in the SW resolves to an opaque response withstatus === 0— not 200. The old guard wasif (r.status === 200) cache.put(...), which rejected every single thumbnail, soytplayer-thumbswas permanently empty and offline showed broken images (measured on prod: 0 entries after browsing pages full of visible thumbs). Fix inthumbnail(): re-issue the request incorsmode — ytimg/ggpht all sendAccess-Control-Allow-Origin: *— and cache that readable response; an opaque one is accepted only as a last resort. Never reintroduce a barestatus === 200check on a cross-origin subresource. - Thumbnail hosts live in
THUMB_HOSTS. An unlisted host doesn't error — it just bypasses the cache and breaks offline, so add mirrors/avatar hosts there. - The thumb cache is capped at
THUMB_CACHE_MAX(800, oldest-first via the insertion-orderedcache.keys()). Keep a cap: CacheStorage and the OPFS offline audio share one origin quota, and opaque entries are padded to ~7 MB each for quota accounting, so an unbounded thumb cache can evict saved audio. - App side (
app.js):warmThumb()pulls artwork through the SW when a video is saved offline, andwarmOfflineThumbs()runs a bounded launch backfill (THUMB_WARM_MAX= 400, 4 at a time) over cached ids + playlist videos so libraries saved before this fix repair themselves. Thumbnails only cache when something requests them — a device needs one online launch to get offline art. staleWhileRevalidate()(fonts) must notreturn cached || networkFetchbare: the fetch resolves tonulloffline andrespondWith(null)throws. It ends with|| Response.error().
Section rail + foldable cards (phones)
SectionRail in app.js draws a floating right-hand rail on phones with one
icon per visible card (player · now playing · lyrics · related · list). It
follows the scroll, a tap jumps to that card, and a tap on the card you are
already on folds/unfolds it (data.settings.foldedCards, per device).
Folding hides a card's body but never its header, so it stays a landmark you
can scroll to and reopen; the Related card's own +/− button routes through
SectionRail.setFolded so the two can never disagree.
Two things about this page that break naive implementations:
- The document does not scroll.
.player-paneis the scroll container in the browser, but the installed PWA in portrait scrolls.bodyinstead (.player-panebecomesoverflow-y: visiblethere).scrollerOf()walks up to whichever ancestor actually scrolls, andviewBox()measures against it. - Scroll events do not bubble, so a
windowscroll listener never fires for an element scroller — the listener is ondocumentwithcapture: true. Cards also appear outsiderender()(Related arrives with the video's related list), soonScrollre-runsbuild(), which is a no-op unless the visible set changed; without that the rail could highlight nothing at all.
First paint vs. the network (launch)
boot() used to await the profile pull and the shared-playlist inbox before
the first render(), so on a slow link the sidebar stayed empty for as long as
the network took. Everything the device knows is already in localStorage, so it
now paints immediately and reconciles afterwards:
?list=/?profile=share links still run before the first paint — they replace the synced slice, so painting first would flash the old playlists and swap them out. They are skipped entirely when the parameter is absent.syncOnLaunch()then runs the network pass with a spinner (#syncSpinner, beside the Playlists header,setSyncing()is counted so the last finisher clears it) and re-renders only ifplaylistFingerprint()changed — a needless render would drop the sidebar's scroll position.
Data model quirks
- Client state persists in localStorage key
_ytpdataand syncs (debounced 400 ms) toPOST /api/user/sync, keyed by a browser fingerprint. - A-B loop markers are per-song-per-playlist: stored on the playlist's own
copy of the video (
entry.ab = {a, b}) when playback source is that playlist;data.abMarkers[videoId]is only the fallback for non-playlist playback. - Online profiles (
profilestable,/api/profile/*): named cross-device sync where the lowercase profile NAME is the only credential (passkey-style, by design). Client storesdata.profile = {name, syncedAt}; sync is last-write-wins — push debounced on every persist(), pull on app launch when the server'supdated_atis newer than the localsyncedAt. - Profile share links:
?profile=<name>is consumed byadoptProfileFromUrl()at boot, before any rendering. It strips the param viareplaceState(so a reload can't re-fire it) and confirms first when the device already has playlists/history or another profile — adopting replaces the synced slice. The name comes off the URL untrusted, henceescapeHtml()on it. - Empty-home playlist grid:
renderHomePlaylists()swaps the branding hero in#playerPlaceholderfor the user's playlists (hero is the no-playlists fallback). It is driven from the tail ofrenderSidebar()— not fromrender()— so every playlist mutation refreshes both in one place.
Testing
- Unit:
node --test frontend/*.test.js(sw, sw-update, async-guard, video-edit, lyrics-core, stats-core, sha256). Server:cd server && bun run test— runs each file in its own process (db.js is a singleton, so two test files in onebun testrun share one temp DB; barebun testalso drags in the frontend node tests via thepublicsymlink). media-cache needs a 60 s hook timeout for its ffmpeg fixtures. Local e2e needs a CURRENT yt-dlp — a 2-month-old one 403s on every download. The directory formnode --test frontend/fails on Node 22 withCannot find module .../frontend— it resolves the dir as a module, not a test glob. That's the harness, not the tests. - E2E:
npx playwright test— WebKit iPhone-12 profile against a static serve offrontend/(needsnpx playwright install webkit). A spurious update banner will make the settings-panel specs fail with#modal intercepts pointer events— that failure mode is a real app bug, not test flake. On the WSL laptop the WebKit install fails host validation (missinglibgtk-4,libgstreamer*, …, needssudo npx playwright install-deps); chromium is already downloaded, so ad-hoc rendering/offline checks can drive it directly instead. - Service-worker behaviour (thumb caching, offline) is only provable in a real
browser: serve
frontend/statically, let the SW take control (needs a second reload), thencontext.setOffline(true)and assertimg.naturalWidth > 0plus theytplayer-thumbsentry count. Asserting against the browser's own HTTP cache proves nothing — check the Cache API entry count. - Test records on prod use
Probe */Recon *names; clean via the container DB, children (video_history) first.
Harness
Skills live in .agents/skills/ (symlinked into .claude/skills/):
- deploy-prod — the Dokploy deploy + buildTag verification flow (manual-only; use for any "deploy"/"release" request).
- mobile-app-ui-design — UI/UX design work on the PWA screens.
- lyrics-lookup — fill in songs that have NO lyrics (LRCLIB → agy web search → local faster-whisper).
- lyrics-regenerate — back up, then replace wrong/mistimed lyrics from LRCLIB (
scripts/lyrics/lrclib_regen.py). - lyrics-agy — last resort when LRCLIB has nothing: agy (flat-rate) searches the web and often returns
LRC-timed lyrics (
scripts/lyrics/agy_lyrics.py). Also the--from-filepath for pasting lyrics you already have. agy is NOT deterministic — the script asks up to--triestimes and keeps the best (timed > untimed > longer). Its answer arrives on stderr, long answers are truncated behind akeep_id, and a quota-failed instance returns error prose inside aSTATUS: okenvelope that will parse as lyrics if you let it.
Matching a song on LRCLIB (learned the hard way)
/api/streams gives the YouTube channel, not the artist, so a title+artist
query finds nothing for a lyric-video channel ("Integrity Worship", "Christian
Lyrics") — lrclib_lookup falls back to a title-only search and then ranks
candidates whose artist verifies 50 points above those that don't. Two guards
sit on the result, and both exist because they caught a real wrong song:
artist_ok()ignoresGENERICwords. Channel "Christian Lyrics" once vouched for a track featuring Christian Burns (Nicky Romero, "Still the Same Man"). Never verify an artist on a word that says nothing about who recorded the song.title_run()compares whole words, never substrings — "Still" is a substring of "(You Can Still) Rock in America", and a one-word title is rejected outright. It gates theLENGTHrule (same title, within 3 s of the same length ⇒ accept even when the artist can't be checked). An unverified match is printed asUNSUREand skipped;--looseoverrides that and is almost always the wrong answer.
Admin lyric editor (frontend/admin.html)
/admin?v=<id> (or the Edit/Lyrics buttons) opens a song with its audio from
/api/streams, the waveform from /api/media/:id/peaks, and one row per line:
Set stamps the playhead, ±0.2 s nudges, Tap mode stamps the highlighted
line on Space and walks down, plus shift-all, line/section/cue kinds, a ✎ Text
LRC view and a ⤓ LRCLIB pull. Saves are optimistic (baseRev, 409 → keep
mine / load theirs). Rows are built once per load and then only their classes
and values change — re-rendering on every tick would steal focus from the
field being typed in and reset the scroll position.
Commit rules
One changeset = one commit, single-line imperative message, no AI attribution
of any kind (global rule). git push origin main pushes to both remotes.