josh/ytplayer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
459a27b6eb0a271c1ba586a1bf66e889ba58a349
ytplayer/plans/jamesdsp-integration-plan.md
Jonathan Sykes 933e920701 Add JamesDSP integration plan
2026-10-09 18:30:26 +08:00

7.2 KiB
Raw Blame History

JamesDSP in ytplayer — integration plan (2026-10-09)

What exists

  • Standalone PWA/port: ~/development/personal/jamesdsp-web (GPL-3 port of JamesDSPManager's portable core to WASM). build.sh -> web/dsp/jamesdsp.{js,wasm} (1.2 MB wasm), ES modules web/jamesdsp-node.js, params.js, worklet.js. Docs: INTEGRATION.md, PLAN.md, VERIFICATION.md (Chromium-only verified: 44.1/48 kHz, 15-band EQ, compander, bass, convolver, stereo, reverb, crossfeed, tube, liveprog/DDC/arbitrary EQ; fixed 128 MiB WASM heap per node; no real-phone/Safari testing).
  • Prototype in ytplayer: branch codex/jamesdsp (3 commits on old base 93d3b05): vendor/jamesdsp/{controller,jamesdsp-node,params,worklet}.js + wasm, JamesDSP IIFE in app.js, Settings group (enable, 5 presets, 3 EQ sliders, bass, crossfeed, limiter), sw.js precache, beta compose stack (docker-compose.beta.yml, beta.worship.hesed.sbs, already a deploy target "codex/jamesdsp"), tests/jamesdsp.spec.js.

Problems the prototype does not solve (why it can't just be merged)

  1. Second AudioContext + second MediaElementSource: app.js already owns EQ (app.js ~5500: 10-band biquad EQ, Level/limiter, VocalReducer, widener) which calls createMediaElementSource on els.video/els.audio. An element can have only ONE source, ever. The two systems fight; whichever attaches first wins and the other throws/silences.
  2. iOS background playback: Safari suspends Web Audio when the screen locks; the existing EQ is disabled on iOS unless liveAudioIOS is on, and iOS saved songs get EQ rendered into a copy (EqRender, eqAudioUrl). The prototype ignores this -> likely silence on lock screen.
  3. Dual elements / audio-continuity (audio-continuity.js, master/secondary, Player.soundEl) and crossfade/preload swaps.
  4. Assets: now manifest-driven, lazy assets.json groups with contract, N-1 retention, offline completion job, per-file hashed URLs. The wasm/worklet must be a lazy group (feature:jamesdsp), in the manifest, cached for offline, never in first paint/install-critical set, and fetched only after opt-in (or idle-prefetch only on Wi-Fi). The worklet module URL must be a hashed immutable URL resolvable offline; wasm bytes are fetched on the main thread and passed via processorOptions (already designed so).
  5. CORS/proxy: streaming <audio> must be same-origin (/api/... proxy) or have crossOrigin='anonymous' set before src; OPFS blob URLs are fine. Verify every playback path (stream, saved/OPFS, P2P, rendered eqAudioUrl).
  6. Memory/CPU: 128 MiB heap per node; one node only, created lazily, destroyed on disable; low-end phone CPU for FIR EQ/reverb/convolver; settings changes suspend audio briefly.
  7. Settings persistence: new settings.jamesDsp must survive updates/profile sync (see settings-persist harness perf/settings-persist.mjs), default OFF.
  8. Licence: DSP port is GPL-3.0-or-later; ytplayer's licence/distribution must be checked and frontend/vendor/jamesdsp/LICENSE.txt + corresponding-source pointer + attribution in Settings/About. Owner decision flagged below.

Design decision (Claude, final)

One audio graph, one context. Do not add a second pipeline. Refactor the existing EQ graph into a small shared "AudioGraph" owner (single AudioContext, one MediaElementSource per element, retained) with an ordered insert slot: source -> [JamesDSP node OR legacy EQ/Level/Vocal/Widener chain] -> destination. Modes (Settings -> Playback & sound -> "Sound engine"): Standard (today's EQ, default) | JamesDSP (replaces the 10-band EQ/vocal/widener while on; Level/limiter remain via JamesDSP's own limiter+gain, loudness gain preserved). Never both EQ chains active -> no double processing. Bypass = exact (setBypass) and, when engine = Standard, JamesDSP is not loaded at all. iOS: JamesDSP live only when liveAudioIOS (same opt-in as today's EQ) because of lock-screen suspension; for iOS saved songs offer the existing render-into-copy approach only if feasible via OfflineAudioContext + the same worklet (investigate; otherwise document unsupported and show the warning). Android/desktop: live. UI: Settings group (keep prototype controls but extended: preset list, 15-band EQ graph/sliders, bass, stereo, crossfeed, reverb preset, tube, limiter, output gain, compressor on/off, A/B bypass button, import preset JSON, IR upload, Reset), themed per layout, works in all 4 layouts + dark/light/contrast. No Liveprog/DDC text effects in v1 (trusted-code risk) — hidden behind an "Advanced" flag off by default.

Phases (single Codex thread, sequential, one commit per step, branch codex/dsp-integrate from main; port the useful prototype code, do not merge the old branch wholesale)

  1. Audit + design note: plans/jamesdsp-audit.md — map every createMediaElementSource/AudioContext/liveAudioIOS/EqRender/audio-continuity/Level/VocalReducer use; confirm single-source constraint; decide the AudioGraph seam; list iOS facts.
  2. Refactor: extract AudioGraph (single ctx/source owner) with the Standard chain unchanged; all existing EQ tests + new unit tests pass; behaviour identical (EQ harness, level tests).
  3. Assets: vendor files from the standalone repo (pin the commit hash in frontend/vendor/jamesdsp/VERSION), add assets.json group feature:jamesdsp (+contract), manifest entries, sw precache removal in favour of the manifest sync, offline-complete + lazy + migration harnesses green, first-paint bytes unchanged (assert perf cold-start numbers do not regress).
  4. Engine: JamesDSP controller on AudioGraph, lazy create, one node, error -> auto fall back to Standard + toast (never silence), 44.1/48 rate check (create the AudioContext at 48000 where allowed; otherwise fall back), suspend/resume handling, destroy on disable, pause-safe config changes (never while Player._wantsPlaying glitch-check: apply with short mute ramp).
  5. UI: Settings group + themed controls + preset library (Flat, Bass boost, Vocal, Loudness, Headphone crossfeed, Bright, Warm), A/B bypass in the Now Playing "more" menu; persisted settings.jamesDsp; included in profile sync + import/export; settings-persist harness extended.
  6. Verification: Playwright (Chromium; WebKit where Web Audio worklets work) — measured RMS change per effect with a real clip, bypass exact, switching tracks/elements/crossfade keeps processing, saved/OPFS + streamed + P2P sources all processed, offline reload with engine on, error injection (corrupt wasm) falls back dry, memory (one 128 MiB heap only), CPU timing for worst-case preset, node --test, manifest/hash, migration, ui-geometry/icons all green. Write plans/jamesdsp-integration-report.md with limits (iOS lock-screen, real-ear tuning).
  7. Beta: push branch only; deploy to beta.worship.hesed.sbs via app-deploy (branch codex/dsp-integrate) for owner listening tests BEFORE merging to main. Main merge only after owner OKs on beta.

Owner decisions needed (defaults chosen)

  • Licence: GPL-3 inclusion in a (possibly public) repo — default: keep ytplayer repo as is, vendor under its own LICENSE, add attribution; confirm you are OK shipping GPL code in this app.
  • iOS: default off unless liveAudioIOS opt-in (matches today's EQ).
  • Default engine stays Standard until you approve on beta.