# Performance Baseline Harness (`perf/`) Automated performance baseline and regression harness for the **ytplayer PWA**. The harness drives the real Bun backend server (`server/server.js`) serving static Brotli-compressed assets, combined with Playwright browser automation on Chromium and WebKit. --- ## 1. Architecture ### Backend Server (`server/server.js`) - Runs as an isolated child process using **Bun** on an ephemeral port. - Configured with dedicated scratch directories under `perf/.tmp/` (`DATA_DIR`, `DB_PATH`, `MEDIA_DIR`) so no production or developer data is ever touched. - Employs the real static asset delivery stack (`Brotli` encoding, `ETag` generation, `Cache-Control` immutable caching for version-stamped assets, and `computeBuildTag()`). ### Browser Automation & Network Throttling - **Chromium**: - Network emulation via Chrome DevTools Protocol (`Network.emulateNetworkConditions`). - CPU throttling at **4× slowdown** (`Emulation.setCPUThrottlingRate`). - Unrestricted audio/video playback via `--autoplay-policy=no-user-gesture-required`. - **WebKit**: - WebKit on Linux lacks CDP network emulation capabilities. - Throttling and stalling are handled by a dedicated local proxy (`perf/proxy.mjs`) sitting between WebKit and the Bun server. - The proxy shapes traffic by pacing chunk delivery through backpressured streams and injecting round-trip latency and stalls. - **Network Profiles**: - **`lte`**: 1.6 Mbps (200 KB/s) download, 750 kbps upload, 150 ms round-trip latency. - **`lossy`**: 1.0 Mbps (125 KB/s) download, 250 ms latency, with intermittent connection stalls (every 7th request delayed by 1,500 ms). ### Shell Fixture Extractor (`perf/make-shell-fixture.sh`) - Archives the frontend from git commits (`git archive frontend`) into `perf/fixtures/shell-/`. - Computes the build tag using the exact sha256 walk algorithm from `server/server.js computeBuildTag()`. - Validates the computed tag against live production (`https://worship.hesed.sbs/api/version`). --- ## 2. Test Scenarios 1. **`cold` (Cold First Load)**: - Fresh browser context with empty cache, no service worker, and clean storage. - Measures initial page fetch, FCP, LCP, DOMContentLoaded, boot milestone (`#cards` render), media playback readiness, service worker precaching duration, and origin storage quota. 2. **`warm` (Warm Reload under SW)**: - Re-navigates the page with active Service Worker and primed HTTP cache. - Measures cached asset retrieval efficiency and second-load boot performance. 3. **`offline` (Offline Reload)**: - Sets offline state and reloads the application. - Verifies the app boots entirely from `CacheStorage` precached shell assets without network connectivity. 4. **`update-js` (Update N → N+1 with 1-line JS comment)**: - Boots client against Tree N (base frontend), establishing an active Service Worker. - Swaps server to Tree N+1 (one comment line appended to `app.js`, altering `BUILD_TAG`). - Triggers `registration.update()` and measures total wire bytes and time until the new Service Worker reaches `waiting` state (`installed`). 5. **`update-css` (Update N → N+1 with 1-line CSS rule)**: - Identical to `update-js`, but Tree N+1 has one CSS rule appended to `theme-glass.css`. - Measures the full shell re-download penalty caused by any stylesheet modification. --- ## 3. Metrics Collected Each scenario records the following metrics across **5 independent runs** (reporting min, max, and median): - **Request Count**: Total HTTP requests made across the wire. - **Wire Bytes**: Total compressed bytes transferred over the network. - **Wire Bytes by Type**: Categorized byte breakdown (`html`, `js`, `css`, `fonts`, `images`, `media`, `api`, `other`). - **FCP (First Contentful Paint)**: Time from navigation start to first rendered content (ms). - **LCP (Largest Contentful Paint)**: Time to largest contentful paint via `PerformanceObserver` (ms). - **DOMContentLoaded**: Time to DOMContentLoaded event end (ms). - **Boot Done**: Time from navigation start until the first `#cards` view rendering is completed (ms). - **Long Tasks Total**: Cumulative execution time of main thread tasks exceeding 50 ms (ms, Chromium). - **SW Install Duration**: Time from Service Worker registration until precaching completes (ms). - **SW Install Wire Bytes**: Network bytes transmitted during Service Worker installation. - **Storage Estimate**: Quota and usage reported by `navigator.storage.estimate()` (bytes). - **Media Ready**: Elapsed time for deterministic media fixture (`benchmedia01`) to reach `readyState >= 3` / playing state. Notes if autoplay policy blocked playback. --- ## 4. How to Run ### Prerequisites Symlink dependencies and ensure server dependencies are installed: ```bash # Worktree root ln -s ~/development/personal/ytplayer/node_modules node_modules cd server && bun install && cd .. ``` ### Basic Commands Run the complete baseline suite (5 runs, all browsers, LTE profile): ```bash node perf/baseline.mjs ``` Run a specific browser or profile: ```bash # Chromium on LTE profile node perf/baseline.mjs --browser chromium --profile lte # WebKit on LTE profile node perf/baseline.mjs --browser webkit --profile lte # Both browsers on lossy profile node perf/baseline.mjs --browser all --profile lossy ``` Run a single scenario: ```bash # Cold load only node perf/baseline.mjs --scenario cold # Update scenario with 1-line JS change node perf/baseline.mjs --scenario update-js # Update scenario with 1-line CSS change node perf/baseline.mjs --scenario update-css ``` Configure run count: ```bash # Quick sanity check (1 iteration) node perf/baseline.mjs --runs 1 # High precision benchmark (7 iterations) node perf/baseline.mjs --runs 7 ``` --- ## 5. Phase 5 Comparison Mode (`--compare`) To verify performance gains and detect regressions in later phases: ```bash node perf/baseline.mjs --compare perf/results/baseline-2026-10-07.json ``` Prints an ASCII/Markdown delta comparison table highlighting percentage changes across wire bytes, boot duration, and update overhead. --- ## 6. Hidden Timers Verification (`perf/hidden-timers.mjs`) Verifies that non-playback UI intervals (`chips`, `reanchor`, `rotateChips`, `flush`, `checkBuildTag`) suspend while the document is hidden and restore upon visibility, while essential background intervals (`maybeCheck`, 120s resume check) remain active: ```bash node perf/hidden-timers.mjs ``` Exits with code 0 on success, or code 1 if any interval fails to pause, fails to resume, or if page errors occur. ## Phase 1 production-client migration Build the production fixture locally without contacting the unavailable production site (COMMON.md takes precedence over the older live-tag instructions above): ```bash perf/make-shell-fixture.sh b77938a 1175f1a1d2c1 node perf/migration.mjs # Chromium and WebKit node perf/migration.mjs --browser webkit # one browser ``` The harness archives the actual `b77938a` server and uses the reconstructed old frontend, isolated database/media/upload paths, and an ephemeral origin behind the `lossy` throttle/stall proxy. It restarts onto the current server and frontend at that same origin, then invokes the OLD application's update check and clicks its actual Refresh UI button. Assertions cover one banner, exactly one reload, cleared update-attempt state, all new SHELL entries with correct hashes, no script errors, and offline boot. Plain, stale `?v=` and `?__ytpfresh=` asset responses are also compared byte-for-byte with the current files. Results are stored in `perf/results/phase1-migration-2026-10-07.json`; scratch trees are removed on exit. No service-worker source is patched or simulated. Offline checks stop Bun and block the proxy; WebKit avoids its broken native setOffline API. Known WebKit transport errors for disconnected version/recommendation requests are verified separately from script errors. The current server snapshots static bytes at boot; restart it after frontend changes. `ASSET_HASHING=0` restores the previous single-tag stamp/cache behavior. The manifest's tag hashes sorted canonical metadata: CSS/JS URLs are stamped before hashing the index, with its derived build-meta placeholder intact. The published index/SW hashes describe their final served bytes after tag injection; these derived hashes are not fed recursively into their own build identity. JS-created workers/importScripts/dynamic imports retain their existing plain URLs in Phase 1. Adding a separate loader would require changing the intentionally unchanged SHELL; worker import URL propagation belongs with the client migration. WASM binaries are embedded in the existing `sha256-wasm.js`/`loudness-wasm.js`, which receive hashed URLs as HTML scripts. Font preloads retain plain URLs to match `fonts.css`; the webmanifest and its icon URLs also stay plain and precached. `piano-engine.mjs` is manifested but is historically absent from SHELL; this existing offline piano limitation is deferred to the loader phase. ## Phase 2 incremental-sync verification The current migration harness verifies the production page's own ready fast path into `ytplayer-assets`: no `__ytpfresh` downloads, no reload before Refresh UI, one banner/reload, an open N-1 tab, playback guard, incremental refresh after an eviction, and offline launch. It preserves the legacy user-triggered reload exception. Async CacheStorage/registration conditions are polled with awaited `page.evaluate`; Playwright's `waitForFunction` does not poll async predicates. ```bash node perf/migration.mjs --browser all --profile lossy node perf/migration.mjs --browser all --profile unthrottled node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario all \ --out perf/results/phase2-final-2026-10-07.json \ --compare perf/results/after-phase0-2026-10-07.json ASSET_SYNC=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \ --scenario offline --out perf/results/phase2-rollback-2026-10-07.json ASSET_HASHING=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \ --scenario offline --out perf/results/phase2-hashing-rollback-2026-10-07.json ``` Results now include response paths/statuses and compressed response-body bytes for cold/update runs. The total includes service-worker script checks/imports; inspect individual responses to distinguish those from app asset downloads. Migration results are `phase2-migration--2026-10-07.json`. `ASSET_SYNC=0` selects the legacy path while preserving the persistent utility cache. `ASSET_HASHING=0` also selects legacy sync because single-tag URLs cannot pass per-file hash checks. In Phase 2 all assets.json groups blocked; Phase 3 selects core plus the active layout and warms background groups at idle. Incremental mode now caches the manifested `piano-engine.mjs` even though it remains absent from the legacy SHELL. ## Phase 3 lazy loading and staged apply ```bash node perf/lazy.mjs --browser all node perf/migration.mjs --browser all --profile lossy node perf/migration.mjs --browser all --profile unthrottled node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario all \ --out perf/results/phase3-final-2026-10-08.json \ --compare perf/results/after-phase0-2026-10-07.json node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario cold \ --frontend-commit 7bc5b4f8b50d78ece3cca5100d54a97fece2748a \ --out perf/results/phase3-phase0-initial-2026-10-08.json ASSET_SYNC=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \ --scenario offline --out perf/results/phase3-sync-rollback-2026-10-08.json ASSET_HASHING=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \ --scenario offline --out perf/results/phase3-hashing-rollback-2026-10-08.json ``` `initialJsCss` sums compressed Resource Timing bodies for page JS/CSS initiated before DOMContentLoaded, excluding SW precache and later idle feature parsing. The saved Phase 0 results lack that metric: `--frontend-commit` reconstructs its frontend in scratch using the current delivery backend for a matched initial-byte comparison. Historical boot/FCP comparisons still use `after-phase0`. Reduced single-run measurements are directional; Phase 5 supplies repeated medians. Update scenarios now wait for every build-N group to finish idle caching before starting the counter, excluding unfinished initial downloads. Waiting-worker bytes represent blocking readiness; an inactive layout's changed CSS downloads later at idle after activation. The browser harness checks pre-body layout selection, immediate offline reload, four distinct Car mode controls, optional Settings search, all extension panels, zero network for warmed first use/layout switches, contract guards and pinned instances. Known WebKit disconnected API transport errors are recorded separately; script errors still fail. Save-Data prevents automatic warming. The index contains inert JSON and an external head bootstrap under unchanged CSP. Source JSON mirrors assets.json for native/static shells; the server replaces it with hashed build-local URLs. No boot-time manifest network request is added. Shared layout visibility defaults remain eager in layout-base.css. Pure EQ settings metadata and DeviceDB remain eager to preserve synchronous player defaults and offline file bookkeeping. P2P/direct parse on actual use or the existing eight-second sharing startup. Searching Settings explicitly loads optional panels so their original labels remain searchable. | JS-requested assets | Phase 3 URL treatment | |---|---| | opfs-worker.js, hash-worker.js, direct-recv-worker.js, p2p-recv-worker.js | Lazy.worker uses embedded per-file URL | | Workers' sha256.js, resume-core.js, sha256-wasm.js | First postMessage carries originating build URLs; importScripts uses them | | piano-engine.mjs | Dynamic import uses Lazy.url; incremental cache warms it | | vendor/framework7-swipe.min.js | Ordered layout group and fallback use Lazy.url | | loudness-wasm.js | Remains an eager stamped script; embedded binary needs no separate URL | | fonts/fonts.css and font preloads; webmanifest/icons | Plain URLs stay aligned with CSS/manifest references; all belong to manifested cache groups | Legacy pages cannot report their selected layout. Their one-time migration blocks all layouts to preserve immediate offline selection; modern pages block only the reported layout and its shared controls. Executing groups never re-evaluate live. See plans/phase3-staged-apply-decision.md: Phase 4 must add disposal/state handoff. ## Phase 4 seams and verification The dependency inventory in `plans/phase4-seams.md` records Phase 3 declarations, resolved outside references and byte/coupling rankings. Reproduce its source using `npm install --prefix perf/.tmp/analysis --no-audit --no-fund acorn@8 eslint-scope@8` and `node perf/analyze-app.cjs cd4a25a` (development scratch dependencies only). Shell/shared view/Settings/section-rail definitions remain eager before app.js. Presenter, Remote client, Watch party, Share/External and Stats view register classic singletons on `window.YT`; synchronous playback hooks remain facades. Executing chunks remain pinned: disposal/state handoff is still required before any live replacement can be enabled. ```bash node --test frontend/*.test.js (cd server && bun install && bun run test) node perf/hidden-timers.mjs node perf/seams.mjs --browser all node perf/lazy.mjs --browser all node perf/migration.mjs --browser all --profile lossy \ --out perf/results/phase4-migration-lossy-2026-10-08.json node perf/migration.mjs --browser all --profile unthrottled \ --out perf/results/phase4-migration-unthrottled-2026-10-08.json node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario all \ --out perf/results/phase4-final-2026-10-08.json \ --compare perf/results/after-phase0-2026-10-07.json node perf/baseline.mjs --runs 3 --browser chromium --profile lte --scenario cold \ --out perf/results/phase4-cold-isolated-2026-10-08.json ``` `update-feature` appends one comment to presenter.js in scratch, installs the worker, explicitly activates, then waits for the changed background chunk to be verified in CacheStorage. It asserts that no other application JS/CSS/font/icon payload transfers; derived index/manifest and worker script checks are included in totals. `all` includes this scenario when the source contains presenter.js. Cold records now include individual long-task timing without changing the aggregate metric, plus the source frontend build tag. WebKit does not expose the long-task API. Migration supports `--out` to preserve earlier phase evidence. Existing lazy harness results are copied to `phase4-lazy-2026-10-08.json` after each Phase 4 run and the Phase 3 result is restored. No production requests or deploys are needed. ## Phase 5 complete measurements ```bash node --test perf/measurement.test.mjs node perf/baseline.mjs --runs 5 --browser all --profile all --scenario all \ --out perf/results/final-2026-10-08.json \ --compare perf/results/baseline-2026-10-07.json node perf/lazy.mjs --browser all --out perf/results/phase5-lazy-2026-10-08.json ``` Warm reloads now wait for verified idle asset caching before counters reset and assert that no application asset body transfers. CSS update totals now include activation and the changed inactive stylesheet's verified background cache fill, matching the feature-update boundary. `readyWireBytes`/`readyDuration` retain the blocking readiness snapshot separately. Core updates stop at blocking readiness because app.js is blocking. Every update asserts that the only application asset payload is the edited file; worker script checks/imports and index/manifest count as overhead in the total. Historical Phase 3/4 CSS numbers were readiness only. No product behavior changes are involved in these measurement corrections. Keep browser/build/test processes idle during full timing measurements. WebKit's zero long-task field means unavailable, not zero work. Autoplay-blocked/media-ready null results do not establish playback performance or iPhone audio continuity.