# Performance Baseline Harness (`perf/`) Automated performance baseline and regression harness for the **ytplayer PWA**. The harness drives the real Bun backend server (`server/server.js`) serving static Brotli-compressed assets, combined with Playwright browser automation on Chromium and WebKit. --- ## 1. Architecture ### Backend Server (`server/server.js`) - Runs as an isolated child process using **Bun** on an ephemeral port. - Configured with dedicated scratch directories under `perf/.tmp/` (`DATA_DIR`, `DB_PATH`, `MEDIA_DIR`) so no production or developer data is ever touched. - Employs the real static asset delivery stack (`Brotli` encoding, `ETag` generation, `Cache-Control` immutable caching for version-stamped assets, and `computeBuildTag()`). ### Browser Automation & Network Throttling - **Chromium**: - Network emulation via Chrome DevTools Protocol (`Network.emulateNetworkConditions`). - CPU throttling at **4× slowdown** (`Emulation.setCPUThrottlingRate`). - Unrestricted audio/video playback via `--autoplay-policy=no-user-gesture-required`. - **WebKit**: - WebKit on Linux lacks CDP network emulation capabilities. - Throttling and stalling are handled by a dedicated local proxy (`perf/proxy.mjs`) sitting between WebKit and the Bun server. - The proxy shapes traffic by pacing chunk delivery through backpressured streams and injecting round-trip latency and stalls. - **Network Profiles**: - **`lte`**: 1.6 Mbps (200 KB/s) download, 750 kbps upload, 150 ms round-trip latency. - **`lossy`**: 1.0 Mbps (125 KB/s) download, 250 ms latency, with intermittent connection stalls (every 7th request delayed by 1,500 ms). ### Shell Fixture Extractor (`perf/make-shell-fixture.sh`) - Archives the frontend from git commits (`git archive frontend`) into `perf/fixtures/shell-/`. - Computes the build tag using the exact sha256 walk algorithm from `server/server.js computeBuildTag()`. - Validates the computed tag against live production (`https://worship.hesed.sbs/api/version`). --- ## 2. Test Scenarios 1. **`cold` (Cold First Load)**: - Fresh browser context with empty cache, no service worker, and clean storage. - Measures initial page fetch, FCP, LCP, DOMContentLoaded, boot milestone (`#cards` render), media playback readiness, service worker precaching duration, and origin storage quota. 2. **`warm` (Warm Reload under SW)**: - Re-navigates the page with active Service Worker and primed HTTP cache. - Measures cached asset retrieval efficiency and second-load boot performance. 3. **`offline` (Offline Reload)**: - Sets offline state and reloads the application. - Verifies the app boots entirely from `CacheStorage` precached shell assets without network connectivity. 4. **`update-js` (Update N → N+1 with 1-line JS comment)**: - Boots client against Tree N (base frontend), establishing an active Service Worker. - Swaps server to Tree N+1 (one comment line appended to `app.js`, altering `BUILD_TAG`). - Triggers `registration.update()` and measures total wire bytes and time until the new Service Worker reaches `waiting` state (`installed`). 5. **`update-css` (Update N → N+1 with 1-line CSS rule)**: - Identical to `update-js`, but Tree N+1 has one CSS rule appended to `theme-glass.css`. - Measures the full shell re-download penalty caused by any stylesheet modification. --- ## 3. Metrics Collected Each scenario records the following metrics across **5 independent runs** (reporting min, max, and median): - **Request Count**: Total HTTP requests made across the wire. - **Wire Bytes**: Total compressed bytes transferred over the network. - **Wire Bytes by Type**: Categorized byte breakdown (`html`, `js`, `css`, `fonts`, `images`, `media`, `api`, `other`). - **FCP (First Contentful Paint)**: Time from navigation start to first rendered content (ms). - **LCP (Largest Contentful Paint)**: Time to largest contentful paint via `PerformanceObserver` (ms). - **DOMContentLoaded**: Time to DOMContentLoaded event end (ms). - **Boot Done**: Time from navigation start until the first `#cards` view rendering is completed (ms). - **Long Tasks Total**: Cumulative execution time of main thread tasks exceeding 50 ms (ms, Chromium). - **SW Install Duration**: Time from Service Worker registration until precaching completes (ms). - **SW Install Wire Bytes**: Network bytes transmitted during Service Worker installation. - **Storage Estimate**: Quota and usage reported by `navigator.storage.estimate()` (bytes). - **Media Ready**: Elapsed time for deterministic media fixture (`benchmedia01`) to reach `readyState >= 3` / playing state. Notes if autoplay policy blocked playback. --- ## 4. How to Run ### Prerequisites Symlink dependencies and ensure server dependencies are installed: ```bash # Worktree root ln -s ~/development/personal/ytplayer/node_modules node_modules cd server && bun install && cd .. ``` ### Basic Commands Run the complete baseline suite (5 runs, all browsers, LTE profile): ```bash node perf/baseline.mjs ``` Run a specific browser or profile: ```bash # Chromium on LTE profile node perf/baseline.mjs --browser chromium --profile lte # WebKit on LTE profile node perf/baseline.mjs --browser webkit --profile lte # Both browsers on lossy profile node perf/baseline.mjs --browser all --profile lossy ``` Run a single scenario: ```bash # Cold load only node perf/baseline.mjs --scenario cold # Update scenario with 1-line JS change node perf/baseline.mjs --scenario update-js # Update scenario with 1-line CSS change node perf/baseline.mjs --scenario update-css ``` Configure run count: ```bash # Quick sanity check (1 iteration) node perf/baseline.mjs --runs 1 # High precision benchmark (7 iterations) node perf/baseline.mjs --runs 7 ``` --- ## 5. Phase 5 Comparison Mode (`--compare`) To verify performance gains and detect regressions in later phases: ```bash node perf/baseline.mjs --compare perf/results/baseline-2026-10-07.json ``` Prints an ASCII/Markdown delta comparison table highlighting percentage changes across wire bytes, boot duration, and update overhead. --- ## 6. Hidden Timers Verification (`perf/hidden-timers.mjs`) Verifies that non-playback UI intervals (`chips`, `reanchor`, `rotateChips`, `flush`, `checkBuildTag`) suspend while the document is hidden and restore upon visibility, while essential background intervals (`maybeCheck`, 120s resume check) remain active: ```bash node perf/hidden-timers.mjs ``` Exits with code 0 on success, or code 1 if any interval fails to pause, fails to resume, or if page errors occur. ## Phase 1 production-client migration Build the production fixture locally without contacting the unavailable production site (COMMON.md takes precedence over the older live-tag instructions above): ```bash perf/make-shell-fixture.sh b77938a 1175f1a1d2c1 node perf/migration.mjs # Chromium and WebKit node perf/migration.mjs --browser webkit # one browser ``` The harness archives the actual `b77938a` server and uses the reconstructed old frontend, isolated database/media/upload paths, and an ephemeral origin behind the `lossy` throttle/stall proxy. It restarts onto the current server and frontend at that same origin, then invokes the OLD application's update check and clicks its actual Refresh UI button. Assertions cover one banner, exactly one reload, cleared update-attempt state, all new SHELL entries with correct hashes, no script errors, and offline boot. Plain, stale `?v=` and `?__ytpfresh=` asset responses are also compared byte-for-byte with the current files. Results are stored in `perf/results/phase1-migration-2026-10-07.json`; scratch trees are removed on exit. No service-worker source is patched or simulated. Offline checks stop Bun and block the proxy; WebKit avoids its broken native setOffline API. Known WebKit transport errors for disconnected version/recommendation requests are verified separately from script errors. The current server snapshots static bytes at boot; restart it after frontend changes. `ASSET_HASHING=0` restores the previous single-tag stamp/cache behavior. The manifest's tag hashes sorted canonical metadata: CSS/JS URLs are stamped before hashing the index, with its derived build-meta placeholder intact. The published index/SW hashes describe their final served bytes after tag injection; these derived hashes are not fed recursively into their own build identity. JS-created workers/importScripts/dynamic imports retain their existing plain URLs in Phase 1. Adding a separate loader would require changing the intentionally unchanged SHELL; worker import URL propagation belongs with the client migration. WASM binaries are embedded in the existing `sha256-wasm.js`/`loudness-wasm.js`, which receive hashed URLs as HTML scripts. Font preloads retain plain URLs to match `fonts.css`; the webmanifest and its icon URLs also stay plain and precached. `piano-engine.mjs` is manifested but is historically absent from SHELL; this existing offline piano limitation is deferred to the loader phase. ## Phase 2 incremental-sync verification The current migration harness verifies the production page's own ready fast path into `ytplayer-assets`: no `__ytpfresh` downloads, no reload before Refresh UI, one banner/reload, an open N-1 tab, playback guard, incremental refresh after an eviction, and offline launch. It preserves the legacy user-triggered reload exception. Async CacheStorage/registration conditions are polled with awaited `page.evaluate`; Playwright's `waitForFunction` does not poll async predicates. ```bash node perf/migration.mjs --browser all --profile lossy node perf/migration.mjs --browser all --profile unthrottled node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario all \ --out perf/results/phase2-final-2026-10-07.json \ --compare perf/results/after-phase0-2026-10-07.json ASSET_SYNC=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \ --scenario offline --out perf/results/phase2-rollback-2026-10-07.json ASSET_HASHING=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \ --scenario offline --out perf/results/phase2-hashing-rollback-2026-10-07.json ``` Results now include response paths/statuses and compressed response-body bytes for cold/update runs. The total includes service-worker script checks/imports; inspect individual responses to distinguish those from app asset downloads. Migration results are `phase2-migration--2026-10-07.json`. `ASSET_SYNC=0` selects the legacy path while preserving the persistent utility cache. `ASSET_HASHING=0` also selects legacy sync because single-tag URLs cannot pass per-file hash checks. In Phase 2 all assets.json groups blocked; Phase 3 selects core plus the active layout and warms background groups at idle. Incremental mode now caches the manifested `piano-engine.mjs` even though it remains absent from the legacy SHELL. ## Phase 3 lazy loading and staged apply ```bash node perf/lazy.mjs --browser all node perf/migration.mjs --browser all --profile lossy node perf/migration.mjs --browser all --profile unthrottled node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario all \ --out perf/results/phase3-final-2026-10-08.json \ --compare perf/results/after-phase0-2026-10-07.json node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario cold \ --frontend-commit 7bc5b4f8b50d78ece3cca5100d54a97fece2748a \ --out perf/results/phase3-phase0-initial-2026-10-08.json ASSET_SYNC=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \ --scenario offline --out perf/results/phase3-sync-rollback-2026-10-08.json ASSET_HASHING=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \ --scenario offline --out perf/results/phase3-hashing-rollback-2026-10-08.json ``` `initialJsCss` sums compressed Resource Timing bodies for page JS/CSS initiated before DOMContentLoaded, excluding SW precache and later idle feature parsing. The saved Phase 0 results lack that metric: `--frontend-commit` reconstructs its frontend in scratch using the current delivery backend for a matched initial-byte comparison. Historical boot/FCP comparisons still use `after-phase0`. Reduced single-run measurements are directional; Phase 5 supplies repeated medians. Update scenarios now wait for every build-N group to finish idle caching before starting the counter, excluding unfinished initial downloads. Waiting-worker bytes represent blocking readiness; an inactive layout's changed CSS downloads later at idle after activation. The browser harness checks pre-body layout selection, immediate offline reload, four distinct Car mode controls, optional Settings search, all extension panels, zero network for warmed first use/layout switches, contract guards and pinned instances. Known WebKit disconnected API transport errors are recorded separately; script errors still fail. Save-Data prevents automatic warming. The index contains inert JSON and an external head bootstrap under unchanged CSP. Source JSON mirrors assets.json for native/static shells; the server replaces it with hashed build-local URLs. No boot-time manifest network request is added. Shared layout visibility defaults remain eager in layout-base.css. Pure EQ settings metadata and DeviceDB remain eager to preserve synchronous player defaults and offline file bookkeeping. P2P/direct parse on actual use or the existing eight-second sharing startup. Searching Settings explicitly loads optional panels so their original labels remain searchable. | JS-requested assets | Phase 3 URL treatment | |---|---| | opfs-worker.js, hash-worker.js, direct-recv-worker.js, p2p-recv-worker.js | Lazy.worker uses embedded per-file URL | | Workers' sha256.js, resume-core.js, sha256-wasm.js | First postMessage carries originating build URLs; importScripts uses them | | piano-engine.mjs | Dynamic import uses Lazy.url; incremental cache warms it | | vendor/framework7-swipe.min.js | Ordered layout group and fallback use Lazy.url | | loudness-wasm.js | Remains an eager stamped script; embedded binary needs no separate URL | | fonts/fonts.css and font preloads; webmanifest/icons | Plain URLs stay aligned with CSS/manifest references; all belong to manifested cache groups | Legacy pages cannot report their selected layout. Their one-time migration blocks all layouts to preserve immediate offline selection; modern pages block only the reported layout and its shared controls. Executing groups never re-evaluate live. See plans/phase3-staged-apply-decision.md: Phase 4 must add disposal/state handoff. ## Phase 4 seams and verification The dependency inventory in `plans/phase4-seams.md` records Phase 3 declarations, resolved outside references and byte/coupling rankings. Reproduce its source using `npm install --prefix perf/.tmp/analysis --no-audit --no-fund acorn@8 eslint-scope@8` and `node perf/analyze-app.cjs cd4a25a` (development scratch dependencies only). Shell/shared view/Settings/section-rail definitions remain eager before app.js. Presenter, Remote client, Watch party, Share/External and Stats view register classic singletons on `window.YT`; synchronous playback hooks remain facades. Executing chunks remain pinned: disposal/state handoff is still required before any live replacement can be enabled. ```bash node --test frontend/*.test.js (cd server && bun install && bun run test) node perf/hidden-timers.mjs node perf/seams.mjs --browser all node perf/lazy.mjs --browser all node perf/migration.mjs --browser all --profile lossy \ --out perf/results/phase4-migration-lossy-2026-10-08.json node perf/migration.mjs --browser all --profile unthrottled \ --out perf/results/phase4-migration-unthrottled-2026-10-08.json node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario all \ --out perf/results/phase4-final-2026-10-08.json \ --compare perf/results/after-phase0-2026-10-07.json node perf/baseline.mjs --runs 3 --browser chromium --profile lte --scenario cold \ --out perf/results/phase4-cold-isolated-2026-10-08.json ``` `update-feature` appends one comment to presenter.js in scratch, installs the worker, explicitly activates, then waits for the changed background chunk to be verified in CacheStorage. It asserts that no other application JS/CSS/font/icon payload transfers; derived index/manifest and worker script checks are included in totals. `all` includes this scenario when the source contains presenter.js. Cold records now include individual long-task timing without changing the aggregate metric, plus the source frontend build tag. WebKit does not expose the long-task API. Migration supports `--out` to preserve earlier phase evidence. Existing lazy harness results are copied to `phase4-lazy-2026-10-08.json` after each Phase 4 run and the Phase 3 result is restored. No production requests or deploys are needed. ## Phase 5 complete measurements ```bash node --test perf/measurement.test.mjs node perf/baseline.mjs --runs 5 --browser all --profile all --scenario all \ --out perf/results/final-2026-10-08.json \ --compare perf/results/baseline-2026-10-07.json node perf/lazy.mjs --browser all --out perf/results/phase5-lazy-2026-10-08.json ``` Warm reloads now wait for verified idle asset caching before counters reset and assert that no application asset body transfers. CSS update totals now include activation and the changed inactive stylesheet's verified background cache fill, matching the feature-update boundary. `readyWireBytes`/`readyDuration` retain the blocking readiness snapshot separately. Core updates stop at blocking readiness because app.js is blocking. Every update asserts that the only application asset payload is the edited file; worker script checks/imports and index/manifest count as overhead in the total. Historical Phase 3/4 CSS numbers were readiness only. No product behavior changes are involved in these measurement corrections. Keep browser/build/test processes idle during full timing measurements. WebKit's zero long-task field means unavailable, not zero work. Autoplay-blocked/media-ready null results do not establish playback performance or iPhone audio continuity. ## UI geometry and media icons ```sh node perf/ui-geometry.mjs node perf/ui-icons.mjs node --test frontend/media-icons.test.js ``` Both harnesses run Chromium and WebKit at mobile dimensions with stubbed SpeechRecognition and local Saved/Queue metadata. They serve only the frontend, block service workers and external requests, and never download media. Geometry tests cold and switched layouts, six widths (320–430), mic visible/hidden and update icon visible/hidden: center spread ≤1px, touch targets ≥40px, Saved image/text gap ≥8px, no horizontal overflow or clipped car control. `--quick` limits widths and speech cases; `--browser` selects an engine; `--frontend` selects historical fixtures. `--measure-only --label before` records known broken geometry without exiting nonzero. Icon checks switch the same page through four layouts × dark/light/contrast, assert SVG definitions, inherited colour, stroke family and save/progress/offline state, and save Queue/Now Playing/navigation shots. Glass Stage intentionally hides bottom navigation, so its full page is recorded instead. Any assertion failure exits nonzero. Shots and raw measurements live in git-ignored plans/ui-shots/. Full production inventory: plans/ui-icon-inventory.md. Native speech recognition, installed-PWA safe-area insets and iPhone rendering still need device verification. ### Cold waterfall diagnosis `--source-root perf/.tmp/` runs that worktree's matching frontend **and server**; `--frontend-commit` instead combines archived frontend with the current backend. Run historical comparisons sequentially to avoid CPU contention. The result records `sourceCommit`, `sourceRoot` and the served build. Every cold sample has `waterfalls`: proxy request order/start/headers/end times and page Resource Timing discovery/start/end times. Proxy times start when tracking begins; page times start at navigation. Compare within each clock, then use the page FCP to locate the rendering dependency. Pass `--assert-cold` on new builds to require exactly one positive-byte app.js response in every cold sample, including worker installation. Omit this assertion when diagnosing historical regressions. Phase 6 counts its fetch-owned app.js response in initial JS/CSS bytes. Since the bootstrap consumes that response asynchronously, DOMContentLoaded no longer means app execution is complete; compare the existing actual boot-done landmark. `--scenario warm,offline,update-js,update-css,update-feature` runs a regression sweep without repeating an already-completed cold acceptance run. ## Runtime manifest hygiene `server/asset-manifest.js` excludes unit tests (`*.test.js`, `*.test.mjs`), Markdown, text licences/data (`*.txt`), C build sources, `*.entry.js` build entries, `/admin.html`, and files below any `perf/` directory from the runtime manifest and both build-tag modes. They remain directly servable with the usual hash and cache headers. If a matching file is genuinely needed at runtime, add its exact root-relative path to the optional `include` array in `frontend/assets.json`; group declarations referencing excluded files fail at boot unless explicitly included. The offline completeness harness compares cache contents with the runtime manifest dynamically; it must not hard-code the old 151-file inventory. See `plans/manifest-hygiene-report.md` and `perf/results/hygiene-*.json` for the before/after measurements and compatibility checks.