ytplayer/docs/improvement-plan.md

ytplayer — Speed & Features Plan (2026-09-29)

Scope: make the app faster (startup, search, pressing play, moving around) and add new features, led by peer-to-peer video sharing. Hardening/refactor work is intentionally out of scope.

Integrated roadmap (what is queued and executable today)

Everything below is broken into step-by-step plans a smaller model can execute without exploring the code — see plans/INDEX.md. All 19 were dry-run end to end.

Order	Plans	What it delivers	Source
1	001	Timing marks (ytp:boot, ytp:search, ytp:tap-to-play) + yt-dlp duration logs	A "Measuring"
2	002–003	Compressed + ETagged shell (app.js 426 → 94 KB), self-hosted fonts	A1.1–A1.3
3	004–005	One yt-dlp resolve per video at a time; resolve likely next plays before the tap	A3.1–A3.2
4	006–007	Search via InnerTube (~0.8 s, yt-dlp fallback); long-lived yt-dlp workers	A2.1, A3.3
5	008–019	Peer-to-peer sharing — design and rules in docs/p2p-architecture.md	Part B0

Not queued yet (write plans for them with /plan-queue when reached): A1.4 minify, A1.5 deferred boot work, A1.6 lazy modules, A2.2 suggestions, A2.3/A2.4, A3.4/A3.5, A4 (VPS media cache), A5 polish, and the Part B feature list.

Where the time goes today (measured against prod)

Measured with curl from a cloud container on 2026-09-29, so absolute numbers include this client's network; the differences are what matter.

What	Measured	Of which is the app itself
Trivial request (/api/version)	~1.25 s (≈0.9 s TLS setup, ≈0.35 s request)	~0.35 s round trip through VPS → WireGuard → homelab
Download app.js	2.8 s at ~150 KB/s	426 KB sent uncompressed — no content-encoding, no ETag
/api/search (new query)	4.1–5.0 s	~3.5 s is one yt-dlp spawn (already --flat-playlist)
/api/search (repeat, 3-min cache)	1.2 s	cache hit = baseline
/api/streams (first play)	7.5 s	~6 s is yt-dlp -J
/api/streams (second play)	1.2 s	streamCache hit

Shell sizes: app.js 426 KB → 118 KB gzip; styles.css 161 KB → 32 KB; index.html 32 KB → 8 KB. Total first load 618 KB → 158 KB just by compressing. Plus a render-blocking Google Fonts stylesheet (3 families, 10 weights) before first paint.

The three biggest wins, in order: (1) compress + ETag the shell, (2) stop paying a cold yt-dlp spawn on every search and first play, (3) start that work before the user taps.

---

Part A — Speed

A1. App startup (first paint, cold and warm)

#	Change	Expected gain	Effort
A1.1	Compress static + JSON. At boot (and when BUILD_TAG changes) pre-compress every shell file with Bun.gzipSync / brotli, keep them in memory, serve by Accept-Encoding with Vary: Accept-Encoding. JSON API responses via hono/compress. Never compress /api/play, /api/media/* (Range media).	Shell 618 → ~158 KB; ~2 s off a cold load at the measured rate	S
A1.2	ETags on the shell. Everything is Cache-Control: no-cache, which is right (see CLAUDE.md update flow), but without an ETag every revalidation re-downloads the full body. Use the content hash already computed for BUILD_TAG per file → 304 Not Modified. Keeps the update-flow rules intact.	Non-SW loads and SW revalidations drop to a few hundred bytes	S
A1.3	Self-host the fonts as subset woff2 (Latin only, only the weights used: --ui, --display, --mono), <link rel="preload"> the display face, font-display: swap. Removes 2 third-party origins (DNS + TLS each) from the critical path and makes fonts work offline via the existing ytplayer-fonts cache. fitLyricLines already re-fits on document.fonts load, so late swaps are safe.	0.3–1 s off first paint on mobile	S
A1.4	defer the scripts and minify at Docker build time with esbuild --minify-whitespace --minify-syntax (no identifier mangling — the files share top-level globals as classic scripts). Keep sources unminified in git; BUILD_TAG still hashes ./public.	app.js gzip ~118 → ~80 KB, parse time down	S
A1.5	Defer non-critical boot work. boot() wires everything synchronously. Move warmOfflineThumbs, preloadPlaylist sweeps, stats commits, remote/party sockets and the inbox poll behind requestIdleCallback (fallback setTimeout 1500). Measure with a performance.mark around boot first.	Faster time-to-interactive on older phones	M
A1.6	Lazy-load rare features (Party, Presenter, Remote, EQ, Share/GIF, the video editor) with import() on first use. Needs those sections pulled out of app.js into their own files — do only the ones that are large and self-contained. Every new file must be added to the SW SHELL list (or offline breaks).	20–35 % less JS parsed at start	M–L

A2. Search (4–5 s → target < 1.5 s)

#	Change	Effort
A2.1	Direct InnerTube search. Call https://www.youtube.com/youtubei/v1/search from Bun with fetch (WEB client context), map videoRenderer items to the existing slim card shape. One HTTPS call instead of starting Python. Keep yt-dlp as the automatic fallback on any parse error / non-200, so a YouTube change degrades to today's speed, not to broken. Add a contract test on a saved response fixture.	M
A2.2	Suggestions as you type from suggestqueries.google.com/complete/search?client=youtube&ds=yt (proxied + cached server-side, debounced 150 ms client-side), mixed with the existing on-device RecentSearches.	S
A2.3	Start the search on Enter-intent: fire the request on the debounced input once the query is stable for ~600 ms, so pressing Enter usually hits the 3-min server cache (the ⚡ indicator already exists).	S
A2.4	Stream results in: render the library/uploads hits (local DB, instant) immediately, then the YouTube results when they land.	S

A3. Pressing play (7.5 s cold → target < 2 s)

#	Change	Effort
A3.1	Coalesce concurrent resolves. resolveStreams() checks streamCache but has no in-flight map, so a warm-up + the real request (or two devices) spawn two yt-dlp processes for one id. Add inflightStreams: Map<id, Promise>. Prerequisite for A3.2.	S
A3.2	Warm on intent. New GET /api/streams/warm?v= (204, fire-and-forget, low priority, capped concurrency) that fills streamCache. Client calls it: for the top 3 results when a search renders; on pointerdown/hover of a card (≥150 ms hover on desktop); for the next 2 queue items when a song starts. Cold plays then hit the 1.2 s cached path.	S–M
A3.3	Persistent extractor process. Replace per-call spawn(yt-dlp) with a small long-lived Python worker (scripts/ytdlp-worker.py) that imports yt_dlp once and answers JSON requests over stdin/stdout (or a unix socket). Saves Python start-up, extractor import and — with a persistent --cache-dir on the data volume — repeated player-JS / n-challenge work. Keep the spawn path as fallback; restart the worker if it dies or after N requests. Applies to search/channel/expand too.	M
A3.4	Instant UI on tap. Show title, thumbnail (already cached by the SW), lyrics and related immediately from the card data while /api/streams is pending, with a thin progress bar on the player — no blank player.	S
A3.5	Fast first frame: start playback on the lowest adaptive height that looks acceptable on the device (e.g. 360p on phones), then switch up once buffered, instead of waiting on the highest quality's first bytes through the proxy.	M

A4. Playback throughput & seeking

• The VPS → homelab WireGuard link carries every media byte (/api/play proxies googlevideo; the server cache serves from the homelab). Measure it (iperf3 over the tunnel) before optimising anything else here.
• If it's the bottleneck: a small Traefik/nginx cache on the VPS for /api/media/<id>?g=<gen> — those URLs are immutable by design (the gen is in the URL), so they can be cached aggressively with Range support (slice module). Hot songs then come straight from the VPS.
• Mark /api/media/<id>?g= responses Cache-Control: public, max-age=31536000, immutable so repeat plays on the same device hit the browser cache.
• Waveform peaks: ytpPeaks is already local; also send peaks with a long max-age since they're per-gen.

A5. Feels-faster polish

• View Transitions API between Home ↔ playlist ↔ search (progressive; no-op where unsupported).
• Optimistic playlist edits (add/remove/reorder paint first, sync after — the sync is already debounced).
• content-visibility: auto on long playlist/history lists; virtualise lists over ~300 rows.
• Image decoding: loading="lazy" decoding="async" + fixed aspect-ratio boxes on every thumbnail (no layout shift while scrolling results).

Measuring (do this first, keep it running)

• Add performance.marks: boot-start, first-render, search-submit → results-rendered, play-tap → playing. Report the medians to a tiny POST /api/perf (sampled, no PII) and show p50/p90 in /admin.
• Server-side: log yt-dlp duration per call type, and cache hit rates for search/streams/media.
• Targets: cold load < 2 s on 4G, repeat load < 0.5 s (SW), search < 1.5 s, tap-to-sound < 2 s cold / < 1 s warmed.

---

Part B — New features

B0. Peer-to-peer video sharing (queued: plans 008–019)

The server keeps the user list, the video list and metadata; every validated copy gets a content id = SHA-256 of its bytes; devices that save a video become persistent holders; when YouTube and the server copy are gone, devices serve each other over WebRTC and can restore the server's copy. Rules the owner set (2026-09-29): P2P is on by default, the server malware scan is off by default (hashing + media validation always run), and holder records are persistent — the UI shows when each was last verified and marks old ones stale instead of dropping them. Full design, data model, flows and security rules: docs/p2p-architecture.md. The earlier phase 02–06 draft is superseded; its differences are listed at the end of that document.

Other features (not queued yet)

Ranked by fit with how the app is actually used (worship sets, sing-alongs, offline playback). Each is sized; most reuse machinery that already exists.

B1. Worship & service

Feature	What it is	Reuses	Effort
Service plans	Date-stamped set lists: songs in order + key, notes, who leads; a read-only share link for the band; one tap opens service mode on it.	playlists, per-song notes, share codes, service mode	M
Chord charts	ChordPro import/paste, chords shown above lyric lines, transpose ± semitones using the existing @ Key G tag, capo helper.	lyrics editor + lyrics-core.js parser	M–L
Confidence monitor	Presenter variant for the stage: current line large, next line below, song section and a clock.	Presenter + fitLyricLines	S
Presenter themes	Background colour/image/looping video, font choice, safe-area margins, lower-third mode for livestreams (transparent background for OBS).	Presenter	M
Pitch shift	Play a song in the band's key: pitch-preserving key change via a Web Audio worklet (desktop/Android; iOS keeps original because of the Web Audio lock-screen issue already documented).	EQ wiring, rate control	M
Count-in & click	Optional metronome click + 4-beat count-in from the @ 70 BPM tag for practice.	lyrics tags, Web Audio	S
Section loops	Loop a chorus/bridge by tapping a # Section in the lyrics (sets A-B from the section's stamps).	A-B loop, lyric sections	S

B2. Listening & discovery

Feature	What it is	Effort
Radio / autoplay mix	When the queue ends, keep going with related videos weighted by your play counts and skips.	M
Smart playlists v2	Rules like "played ≥ 5× in 30 days", "saved offline", "has synced lyrics", "under 5 min". SMART_PLAYLISTS already exists — make them user-defined.	S–M
Stats wrap-up	Monthly/yearly recap card (top songs, minutes, streak) from data.stats, shareable as an image.	S
Sleep/wake alarm	Start a playlist at a set time (while the app is open / PWA foreground).	S
Lyrics search	Search across all saved songs' lyrics ("which song has 'goodness of God'"). Server-side over video_notes.	S

B3. Offline & library

Feature	What it is	Effort
Download manager	One screen: queued/active/done saves with progress, retry, pause-all, storage used per playlist.	M
Auto-offline favourites	Keep the N most-played songs saved automatically, evict the least-played.	S
Uploads for everyone	Let profile users (not just admin) upload audio to their own library, with a per-profile quota.	M

B4. Together

Feature	What it is	Effort
Collaborative playlists	A playlist several profiles can edit (last-write-wins per entry, change feed).	M–L
Party queue voting	Guests suggest songs; host approves or upvotes decide order.	M
Reactions in party	Lightweight emoji reactions over the video/lyrics.	S

---

Suggested order

1. Run the queue (/run-queue): plans 001–007 (speed), then 008–019 (peer-to-peer). Deploy after 007 and measure with the ytp:* marks and the [ytdlp] log lines before starting P2P.
2. Next speed plans to write: A3.4 instant UI on tap, A2.2 suggestions, A1.5 deferred boot work, then measure the WireGuard link (A4) and decide on the VPS media cache.
3. Then features, starting with the small high-fit ones: section loops, confidence monitor, count-in, lyrics search, stats wrap-up — then service plans and chord charts.

Each item is sized to be one commit (CLAUDE.md commit rules); queue new ones with the plan-queue skill.