josh/ytplayer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
cd4a25ae23f0dd8034b012e4606dec8bf5337de5
ytplayer/perf/README.md

14 KiB
Raw Blame History

Performance Baseline Harness (perf/)

Automated performance baseline and regression harness for the ytplayer PWA.

The harness drives the real Bun backend server (server/server.js) serving static Brotli-compressed assets, combined with Playwright browser automation on Chromium and WebKit.

1. Architecture

Backend Server (server/server.js)

  • Runs as an isolated child process using Bun on an ephemeral port.
  • Configured with dedicated scratch directories under perf/.tmp/ (DATA_DIR, DB_PATH, MEDIA_DIR) so no production or developer data is ever touched.
  • Employs the real static asset delivery stack (Brotli encoding, ETag generation, Cache-Control immutable caching for version-stamped assets, and computeBuildTag()).

Browser Automation & Network Throttling

  • Chromium:
    • Network emulation via Chrome DevTools Protocol (Network.emulateNetworkConditions).
    • CPU throttling at 4× slowdown (Emulation.setCPUThrottlingRate).
    • Unrestricted audio/video playback via --autoplay-policy=no-user-gesture-required.
  • WebKit:
    • WebKit on Linux lacks CDP network emulation capabilities.
    • Throttling and stalling are handled by a dedicated local proxy (perf/proxy.mjs) sitting between WebKit and the Bun server.
    • The proxy shapes traffic by pacing chunk delivery through backpressured streams and injecting round-trip latency and stalls.
  • Network Profiles:
    • lte: 1.6 Mbps (200 KB/s) download, 750 kbps upload, 150 ms round-trip latency.
    • lossy: 1.0 Mbps (125 KB/s) download, 250 ms latency, with intermittent connection stalls (every 7th request delayed by 1,500 ms).

Shell Fixture Extractor (perf/make-shell-fixture.sh)

  • Archives the frontend from git commits (git archive <commit> frontend) into perf/fixtures/shell-<tag>/.
  • Computes the build tag using the exact sha256 walk algorithm from server/server.js computeBuildTag().
  • Validates the computed tag against live production (https://worship.hesed.sbs/api/version).

2. Test Scenarios

  1. cold (Cold First Load):
    • Fresh browser context with empty cache, no service worker, and clean storage.
    • Measures initial page fetch, FCP, LCP, DOMContentLoaded, boot milestone (#cards render), media playback readiness, service worker precaching duration, and origin storage quota.
  2. warm (Warm Reload under SW):
    • Re-navigates the page with active Service Worker and primed HTTP cache.
    • Measures cached asset retrieval efficiency and second-load boot performance.
  3. offline (Offline Reload):
    • Sets offline state and reloads the application.
    • Verifies the app boots entirely from CacheStorage precached shell assets without network connectivity.
  4. update-js (Update N → N+1 with 1-line JS comment):
    • Boots client against Tree N (base frontend), establishing an active Service Worker.
    • Swaps server to Tree N+1 (one comment line appended to app.js, altering BUILD_TAG).
    • Triggers registration.update() and measures total wire bytes and time until the new Service Worker reaches waiting state (installed).
  5. update-css (Update N → N+1 with 1-line CSS rule):
    • Identical to update-js, but Tree N+1 has one CSS rule appended to theme-glass.css.
    • Measures the full shell re-download penalty caused by any stylesheet modification.

3. Metrics Collected

Each scenario records the following metrics across 5 independent runs (reporting min, max, and median):

  • Request Count: Total HTTP requests made across the wire.
  • Wire Bytes: Total compressed bytes transferred over the network.
  • Wire Bytes by Type: Categorized byte breakdown (html, js, css, fonts, images, media, api, other).
  • FCP (First Contentful Paint): Time from navigation start to first rendered content (ms).
  • LCP (Largest Contentful Paint): Time to largest contentful paint via PerformanceObserver (ms).
  • DOMContentLoaded: Time to DOMContentLoaded event end (ms).
  • Boot Done: Time from navigation start until the first #cards view rendering is completed (ms).
  • Long Tasks Total: Cumulative execution time of main thread tasks exceeding 50 ms (ms, Chromium).
  • SW Install Duration: Time from Service Worker registration until precaching completes (ms).
  • SW Install Wire Bytes: Network bytes transmitted during Service Worker installation.
  • Storage Estimate: Quota and usage reported by navigator.storage.estimate() (bytes).
  • Media Ready: Elapsed time for deterministic media fixture (benchmedia01) to reach readyState >= 3 / playing state. Notes if autoplay policy blocked playback.

4. How to Run

Prerequisites

Symlink dependencies and ensure server dependencies are installed:

# Worktree root
ln -s ~/development/personal/ytplayer/node_modules node_modules
cd server && bun install && cd ..

Basic Commands

Run the complete baseline suite (5 runs, all browsers, LTE profile):

node perf/baseline.mjs

Run a specific browser or profile:

# Chromium on LTE profile
node perf/baseline.mjs --browser chromium --profile lte

# WebKit on LTE profile
node perf/baseline.mjs --browser webkit --profile lte

# Both browsers on lossy profile
node perf/baseline.mjs --browser all --profile lossy

Run a single scenario:

# Cold load only
node perf/baseline.mjs --scenario cold

# Update scenario with 1-line JS change
node perf/baseline.mjs --scenario update-js

# Update scenario with 1-line CSS change
node perf/baseline.mjs --scenario update-css

Configure run count:

# Quick sanity check (1 iteration)
node perf/baseline.mjs --runs 1

# High precision benchmark (7 iterations)
node perf/baseline.mjs --runs 7

5. Phase 5 Comparison Mode (--compare)

To verify performance gains and detect regressions in later phases:

node perf/baseline.mjs --compare perf/results/baseline-2026-10-07.json

Prints an ASCII/Markdown delta comparison table highlighting percentage changes across wire bytes, boot duration, and update overhead.

6. Hidden Timers Verification (perf/hidden-timers.mjs)

Verifies that non-playback UI intervals (chips, reanchor, rotateChips, flush, checkBuildTag) suspend while the document is hidden and restore upon visibility, while essential background intervals (maybeCheck, 120s resume check) remain active:

node perf/hidden-timers.mjs

Exits with code 0 on success, or code 1 if any interval fails to pause, fails to resume, or if page errors occur.

Phase 1 production-client migration

Build the production fixture locally without contacting the unavailable production site (COMMON.md takes precedence over the older live-tag instructions above):

perf/make-shell-fixture.sh b77938a 1175f1a1d2c1
node perf/migration.mjs                    # Chromium and WebKit
node perf/migration.mjs --browser webkit  # one browser

The harness archives the actual b77938a server and uses the reconstructed old frontend, isolated database/media/upload paths, and an ephemeral origin behind the lossy throttle/stall proxy. It restarts onto the current server and frontend at that same origin, then invokes the OLD application's update check and clicks its actual Refresh UI button. Assertions cover one banner, exactly one reload, cleared update-attempt state, all new SHELL entries with correct hashes, no script errors, and offline boot. Plain, stale ?v=<oldTag> and ?__ytpfresh= asset responses are also compared byte-for-byte with the current files. Results are stored in perf/results/phase1-migration-2026-10-07.json; scratch trees are removed on exit. No service-worker source is patched or simulated. Offline checks stop Bun and block the proxy; WebKit avoids its broken native setOffline API. Known WebKit transport errors for disconnected version/recommendation requests are verified separately from script errors.

The current server snapshots static bytes at boot; restart it after frontend changes. ASSET_HASHING=0 restores the previous single-tag stamp/cache behavior. The manifest's tag hashes sorted canonical metadata: CSS/JS URLs are stamped before hashing the index, with its derived build-meta placeholder intact. The published index/SW hashes describe their final served bytes after tag injection; these derived hashes are not fed recursively into their own build identity.

JS-created workers/importScripts/dynamic imports retain their existing plain URLs in Phase 1. Adding a separate loader would require changing the intentionally unchanged SHELL; worker import URL propagation belongs with the client migration. WASM binaries are embedded in the existing sha256-wasm.js/loudness-wasm.js, which receive hashed URLs as HTML scripts. Font preloads retain plain URLs to match fonts.css; the webmanifest and its icon URLs also stay plain and precached. piano-engine.mjs is manifested but is historically absent from SHELL; this existing offline piano limitation is deferred to the loader phase.

Phase 2 incremental-sync verification

The current migration harness verifies the production page's own ready fast path into ytplayer-assets: no __ytpfresh downloads, no reload before Refresh UI, one banner/reload, an open N-1 tab, playback guard, incremental refresh after an eviction, and offline launch. It preserves the legacy user-triggered reload exception. Async CacheStorage/registration conditions are polled with awaited page.evaluate; Playwright's waitForFunction does not poll async predicates.

node perf/migration.mjs --browser all --profile lossy
node perf/migration.mjs --browser all --profile unthrottled
node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario all \
  --out perf/results/phase2-final-2026-10-07.json \
  --compare perf/results/after-phase0-2026-10-07.json
ASSET_SYNC=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \
  --scenario offline --out perf/results/phase2-rollback-2026-10-07.json
ASSET_HASHING=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \
  --scenario offline --out perf/results/phase2-hashing-rollback-2026-10-07.json

Results now include response paths/statuses and compressed response-body bytes for cold/update runs. The total includes service-worker script checks/imports; inspect individual responses to distinguish those from app asset downloads. Migration results are phase2-migration-<profile>-2026-10-07.json. ASSET_SYNC=0 selects the legacy path while preserving the persistent utility cache. ASSET_HASHING=0 also selects legacy sync because single-tag URLs cannot pass per-file hash checks. In Phase 2 all assets.json groups blocked; Phase 3 selects core plus the active layout and warms background groups at idle. Incremental mode now caches the manifested piano-engine.mjs even though it remains absent from the legacy SHELL.

Phase 3 lazy loading and staged apply

node perf/lazy.mjs --browser all
node perf/migration.mjs --browser all --profile lossy
node perf/migration.mjs --browser all --profile unthrottled
node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario all \
  --out perf/results/phase3-final-2026-10-08.json \
  --compare perf/results/after-phase0-2026-10-07.json
node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario cold \
  --frontend-commit 7bc5b4f8b50d78ece3cca5100d54a97fece2748a \
  --out perf/results/phase3-phase0-initial-2026-10-08.json
ASSET_SYNC=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \
  --scenario offline --out perf/results/phase3-sync-rollback-2026-10-08.json
ASSET_HASHING=0 node perf/baseline.mjs --runs 1 --browser all --profile lte \
  --scenario offline --out perf/results/phase3-hashing-rollback-2026-10-08.json

initialJsCss sums compressed Resource Timing bodies for page JS/CSS initiated before DOMContentLoaded, excluding SW precache and later idle feature parsing. The saved Phase 0 results lack that metric: --frontend-commit reconstructs its frontend in scratch using the current delivery backend for a matched initial-byte comparison. Historical boot/FCP comparisons still use after-phase0. Reduced single-run measurements are directional; Phase 5 supplies repeated medians. Update scenarios now wait for every build-N group to finish idle caching before starting the counter, excluding unfinished initial downloads. Waiting-worker bytes represent blocking readiness; an inactive layout's changed CSS downloads later at idle after activation.

The browser harness checks pre-body layout selection, immediate offline reload, four distinct Car mode controls, optional Settings search, all extension panels, zero network for warmed first use/layout switches, contract guards and pinned instances. Known WebKit disconnected API transport errors are recorded separately; script errors still fail. Save-Data prevents automatic warming.

The index contains inert JSON and an external head bootstrap under unchanged CSP. Source JSON mirrors assets.json for native/static shells; the server replaces it with hashed build-local URLs. No boot-time manifest network request is added. Shared layout visibility defaults remain eager in layout-base.css. Pure EQ settings metadata and DeviceDB remain eager to preserve synchronous player defaults and offline file bookkeeping. P2P/direct parse on actual use or the existing eight-second sharing startup. Searching Settings explicitly loads optional panels so their original labels remain searchable.

JS-requested assets Phase 3 URL treatment
opfs-worker.js, hash-worker.js, direct-recv-worker.js, p2p-recv-worker.js Lazy.worker uses embedded per-file URL
Workers' sha256.js, resume-core.js, sha256-wasm.js First postMessage carries originating build URLs; importScripts uses them
piano-engine.mjs Dynamic import uses Lazy.url; incremental cache warms it
vendor/framework7-swipe.min.js Ordered layout group and fallback use Lazy.url
loudness-wasm.js Remains an eager stamped script; embedded binary needs no separate URL
fonts/fonts.css and font preloads; webmanifest/icons Plain URLs stay aligned with CSS/manifest references; all belong to manifested cache groups

Legacy pages cannot report their selected layout. Their one-time migration blocks all layouts to preserve immediate offline selection; modern pages block only the reported layout and its shared controls. Executing groups never re-evaluate live. See plans/phase3-staged-apply-decision.md: Phase 4 must add disposal/state handoff.