josh/ytplayer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4853c2c97727019b305ce3c3487013e760fc312c
ytplayer/perf
History

README.md

Performance Baseline Harness (perf/)

Automated performance baseline and regression harness for the ytplayer PWA.

The harness drives the real Bun backend server (server/server.js) serving static Brotli-compressed assets, combined with Playwright browser automation on Chromium and WebKit.

1. Architecture

Backend Server (server/server.js)

  • Runs as an isolated child process using Bun on an ephemeral port.
  • Configured with dedicated scratch directories under perf/.tmp/ (DATA_DIR, DB_PATH, MEDIA_DIR) so no production or developer data is ever touched.
  • Employs the real static asset delivery stack (Brotli encoding, ETag generation, Cache-Control immutable caching for version-stamped assets, and computeBuildTag()).

Browser Automation & Network Throttling

  • Chromium:
    • Network emulation via Chrome DevTools Protocol (Network.emulateNetworkConditions).
    • CPU throttling at 4× slowdown (Emulation.setCPUThrottlingRate).
    • Unrestricted audio/video playback via --autoplay-policy=no-user-gesture-required.
  • WebKit:
    • WebKit on Linux lacks CDP network emulation capabilities.
    • Throttling and stalling are handled by a dedicated local proxy (perf/proxy.mjs) sitting between WebKit and the Bun server.
    • The proxy shapes traffic by pacing chunk delivery through backpressured streams and injecting round-trip latency and stalls.
  • Network Profiles:
    • lte: 1.6 Mbps (200 KB/s) download, 750 kbps upload, 150 ms round-trip latency.
    • lossy: 1.0 Mbps (125 KB/s) download, 250 ms latency, with intermittent connection stalls (every 7th request delayed by 1,500 ms).

Shell Fixture Extractor (perf/make-shell-fixture.sh)

  • Archives the frontend from git commits (git archive <commit> frontend) into perf/fixtures/shell-<tag>/.
  • Computes the build tag using the exact sha256 walk algorithm from server/server.js computeBuildTag().
  • Validates the computed tag against live production (https://worship.hesed.sbs/api/version).

2. Test Scenarios

  1. cold (Cold First Load):
    • Fresh browser context with empty cache, no service worker, and clean storage.
    • Measures initial page fetch, FCP, LCP, DOMContentLoaded, boot milestone (#cards render), media playback readiness, service worker precaching duration, and origin storage quota.
  2. warm (Warm Reload under SW):
    • Re-navigates the page with active Service Worker and primed HTTP cache.
    • Measures cached asset retrieval efficiency and second-load boot performance.
  3. offline (Offline Reload):
    • Sets offline state and reloads the application.
    • Verifies the app boots entirely from CacheStorage precached shell assets without network connectivity.
  4. update-js (Update N → N+1 with 1-line JS comment):
    • Boots client against Tree N (base frontend), establishing an active Service Worker.
    • Swaps server to Tree N+1 (one comment line appended to app.js, altering BUILD_TAG).
    • Triggers registration.update() and measures total wire bytes and time until the new Service Worker reaches waiting state (installed).
  5. update-css (Update N → N+1 with 1-line CSS rule):
    • Identical to update-js, but Tree N+1 has one CSS rule appended to theme-glass.css.
    • Measures the full shell re-download penalty caused by any stylesheet modification.

3. Metrics Collected

Each scenario records the following metrics across 5 independent runs (reporting min, max, and median):

  • Request Count: Total HTTP requests made across the wire.
  • Wire Bytes: Total compressed bytes transferred over the network.
  • Wire Bytes by Type: Categorized byte breakdown (html, js, css, fonts, images, media, api, other).
  • FCP (First Contentful Paint): Time from navigation start to first rendered content (ms).
  • LCP (Largest Contentful Paint): Time to largest contentful paint via PerformanceObserver (ms).
  • DOMContentLoaded: Time to DOMContentLoaded event end (ms).
  • Boot Done: Time from navigation start until the first #cards view rendering is completed (ms).
  • Long Tasks Total: Cumulative execution time of main thread tasks exceeding 50 ms (ms, Chromium).
  • SW Install Duration: Time from Service Worker registration until precaching completes (ms).
  • SW Install Wire Bytes: Network bytes transmitted during Service Worker installation.
  • Storage Estimate: Quota and usage reported by navigator.storage.estimate() (bytes).
  • Media Ready: Elapsed time for deterministic media fixture (benchmedia01) to reach readyState >= 3 / playing state. Notes if autoplay policy blocked playback.

4. How to Run

Prerequisites

Symlink dependencies and ensure server dependencies are installed:

# Worktree root
ln -s ~/development/personal/ytplayer/node_modules node_modules
cd server && bun install && cd ..

Basic Commands

Run the complete baseline suite (5 runs, all browsers, LTE profile):

node perf/baseline.mjs

Run a specific browser or profile:

# Chromium on LTE profile
node perf/baseline.mjs --browser chromium --profile lte

# WebKit on LTE profile
node perf/baseline.mjs --browser webkit --profile lte

# Both browsers on lossy profile
node perf/baseline.mjs --browser all --profile lossy

Run a single scenario:

# Cold load only
node perf/baseline.mjs --scenario cold

# Update scenario with 1-line JS change
node perf/baseline.mjs --scenario update-js

# Update scenario with 1-line CSS change
node perf/baseline.mjs --scenario update-css

Configure run count:

# Quick sanity check (1 iteration)
node perf/baseline.mjs --runs 1

# High precision benchmark (7 iterations)
node perf/baseline.mjs --runs 7

5. Phase 5 Comparison Mode (--compare)

To verify performance gains and detect regressions in later phases:

node perf/baseline.mjs --compare perf/results/baseline-2026-10-07.json

Prints an ASCII/Markdown delta comparison table highlighting percentage changes across wire bytes, boot duration, and update overhead.

6. Hidden Timers Verification (perf/hidden-timers.mjs)

Verifies that non-playback UI intervals (chips, reanchor, rotateChips, flush, checkBuildTag) suspend while the document is hidden and restore upon visibility, while essential background intervals (maybeCheck, 120s resume check) remain active:

node perf/hidden-timers.mjs

Exits with code 0 on success, or code 1 if any interval fails to pause, fails to resume, or if page errors occur.

Phase 1 production-client migration

Build the production fixture locally without contacting the unavailable production site (COMMON.md takes precedence over the older live-tag instructions above):

perf/make-shell-fixture.sh b77938a 1175f1a1d2c1
node perf/migration.mjs                    # Chromium and WebKit
node perf/migration.mjs --browser webkit  # one browser

The harness archives the actual b77938a server and uses the reconstructed old frontend, isolated database/media/upload paths, and an ephemeral origin behind the lossy throttle/stall proxy. It restarts onto the current server and frontend at that same origin, then invokes the OLD application's update check and clicks its actual Refresh UI button. Assertions cover one banner, exactly one reload, cleared update-attempt state, all new SHELL entries with correct hashes, no script errors, and offline boot. Plain, stale ?v=<oldTag> and ?__ytpfresh= asset responses are also compared byte-for-byte with the current files. Results are stored in perf/results/phase1-migration-2026-10-07.json; scratch trees are removed on exit. No service-worker source is patched or simulated. Offline checks stop Bun and block the proxy; WebKit avoids its broken native setOffline API. Known WebKit transport errors for disconnected version/recommendation requests are verified separately from script errors.

The current server snapshots static bytes at boot; restart it after frontend changes. ASSET_HASHING=0 restores the previous single-tag stamp/cache behavior. The manifest's tag hashes sorted canonical metadata: CSS/JS URLs are stamped before hashing the index, with its derived build-meta placeholder intact. The published index/SW hashes describe their final served bytes after tag injection; these derived hashes are not fed recursively into their own build identity.

JS-created workers/importScripts/dynamic imports retain their existing plain URLs in Phase 1. Adding a separate loader would require changing the intentionally unchanged SHELL; worker import URL propagation belongs with the client migration. WASM binaries are embedded in the existing sha256-wasm.js/loudness-wasm.js, which receive hashed URLs as HTML scripts. Font preloads retain plain URLs to match fonts.css; the webmanifest and its icon URLs also stay plain and precached. piano-engine.mjs is manifested but is historically absent from SHELL; this existing offline piano limitation is deferred to the loader phase.

Phase 2 incremental-sync verification

The current migration harness verifies the production page's own ready fast path into ytplayer-assets: no __ytpfresh downloads, no reload before Refresh UI, one banner/reload, an open N-1 tab, playback guard, incremental refresh after an eviction, and offline launch. It preserves the legacy user-triggered reload exception. Async CacheStorage/registration conditions are polled with awaited page.evaluate; Playwright's waitForFunction does not poll async predicates.

node perf/migration.mjs --browser all --profile lossy
node perf/migration.mjs --browser all --profile unthrottled
node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario all \
  --out perf/results/phase2-final-2026-10-07.json \
  --compare perf/results/after-phase0-2026-10-07.json
ASSET_SYNC=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \
  --scenario offline --out perf/results/phase2-rollback-2026-10-07.json
ASSET_HASHING=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \
  --scenario offline --out perf/results/phase2-hashing-rollback-2026-10-07.json

Results now include response paths/statuses and compressed response-body bytes for cold/update runs. The total includes service-worker script checks/imports; inspect individual responses to distinguish those from app asset downloads. Migration results are phase2-migration-<profile>-2026-10-07.json. ASSET_SYNC=0 selects the legacy path while preserving the persistent utility cache. ASSET_HASHING=0 also selects legacy sync because single-tag URLs cannot pass per-file hash checks. Current assets.json groups all block; idle loading and lazy parsing await Phase 3. Incremental mode now caches the manifested piano-engine.mjs even though it remains absent from the legacy SHELL.