josh/ytplayer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4bc15a55e39b4f7fec223982928ed55eff2202ae
ytplayer/perf/README.md

6.0 KiB
Raw Blame History

Performance Baseline Harness (perf/)

Automated performance baseline and regression harness for the ytplayer PWA.

The harness drives the real Bun backend server (server/server.js) serving static Brotli-compressed assets, combined with Playwright browser automation on Chromium and WebKit.

1. Architecture

Backend Server (server/server.js)

  • Runs as an isolated child process using Bun on an ephemeral port.
  • Configured with dedicated scratch directories under perf/.tmp/ (DATA_DIR, DB_PATH, MEDIA_DIR) so no production or developer data is ever touched.
  • Employs the real static asset delivery stack (Brotli encoding, ETag generation, Cache-Control immutable caching for version-stamped assets, and computeBuildTag()).

Browser Automation & Network Throttling

  • Chromium:
    • Network emulation via Chrome DevTools Protocol (Network.emulateNetworkConditions).
    • CPU throttling at 4× slowdown (Emulation.setCPUThrottlingRate).
    • Unrestricted audio/video playback via --autoplay-policy=no-user-gesture-required.
  • WebKit:
    • WebKit on Linux lacks CDP network emulation capabilities.
    • Throttling and stalling are handled by a dedicated local proxy (perf/proxy.mjs) sitting between WebKit and the Bun server.
    • The proxy shapes traffic by pacing chunk delivery through backpressured streams and injecting round-trip latency and stalls.
  • Network Profiles:
    • lte: 1.6 Mbps (200 KB/s) download, 750 kbps upload, 150 ms round-trip latency.
    • lossy: 1.0 Mbps (125 KB/s) download, 250 ms latency, with intermittent connection stalls (every 7th request delayed by 1,500 ms).

Shell Fixture Extractor (perf/make-shell-fixture.sh)

  • Archives the frontend from git commits (git archive <commit> frontend) into perf/fixtures/shell-<tag>/.
  • Computes the build tag using the exact sha256 walk algorithm from server/server.js computeBuildTag().
  • Validates the computed tag against live production (https://worship.hesed.sbs/api/version).

2. Test Scenarios

  1. cold (Cold First Load):
    • Fresh browser context with empty cache, no service worker, and clean storage.
    • Measures initial page fetch, FCP, LCP, DOMContentLoaded, boot milestone (#cards render), media playback readiness, service worker precaching duration, and origin storage quota.
  2. warm (Warm Reload under SW):
    • Re-navigates the page with active Service Worker and primed HTTP cache.
    • Measures cached asset retrieval efficiency and second-load boot performance.
  3. offline (Offline Reload):
    • Sets offline state and reloads the application.
    • Verifies the app boots entirely from CacheStorage precached shell assets without network connectivity.
  4. update-js (Update N → N+1 with 1-line JS comment):
    • Boots client against Tree N (base frontend), establishing an active Service Worker.
    • Swaps server to Tree N+1 (one comment line appended to app.js, altering BUILD_TAG).
    • Triggers registration.update() and measures total wire bytes and time until the new Service Worker reaches waiting state (installed).
  5. update-css (Update N → N+1 with 1-line CSS rule):
    • Identical to update-js, but Tree N+1 has one CSS rule appended to theme-glass.css.
    • Measures the full shell re-download penalty caused by any stylesheet modification.

3. Metrics Collected

Each scenario records the following metrics across 5 independent runs (reporting min, max, and median):

  • Request Count: Total HTTP requests made across the wire.
  • Wire Bytes: Total compressed bytes transferred over the network.
  • Wire Bytes by Type: Categorized byte breakdown (html, js, css, fonts, images, media, api, other).
  • FCP (First Contentful Paint): Time from navigation start to first rendered content (ms).
  • LCP (Largest Contentful Paint): Time to largest contentful paint via PerformanceObserver (ms).
  • DOMContentLoaded: Time to DOMContentLoaded event end (ms).
  • Boot Done: Time from navigation start until the first #cards view rendering is completed (ms).
  • Long Tasks Total: Cumulative execution time of main thread tasks exceeding 50 ms (ms, Chromium).
  • SW Install Duration: Time from Service Worker registration until precaching completes (ms).
  • SW Install Wire Bytes: Network bytes transmitted during Service Worker installation.
  • Storage Estimate: Quota and usage reported by navigator.storage.estimate() (bytes).
  • Media Ready: Elapsed time for deterministic media fixture (benchmedia01) to reach readyState >= 3 / playing state. Notes if autoplay policy blocked playback.

4. How to Run

Prerequisites

Symlink dependencies and ensure server dependencies are installed:

# Worktree root
ln -s ~/development/personal/ytplayer/node_modules node_modules
cd server && bun install && cd ..

Basic Commands

Run the complete baseline suite (5 runs, all browsers, LTE profile):

node perf/baseline.mjs

Run a specific browser or profile:

# Chromium on LTE profile
node perf/baseline.mjs --browser chromium --profile lte

# WebKit on LTE profile
node perf/baseline.mjs --browser webkit --profile lte

# Both browsers on lossy profile
node perf/baseline.mjs --browser all --profile lossy

Run a single scenario:

# Cold load only
node perf/baseline.mjs --scenario cold

# Update scenario with 1-line JS change
node perf/baseline.mjs --scenario update-js

# Update scenario with 1-line CSS change
node perf/baseline.mjs --scenario update-css

Configure run count:

# Quick sanity check (1 iteration)
node perf/baseline.mjs --runs 1

# High precision benchmark (7 iterations)
node perf/baseline.mjs --runs 7

5. Phase 5 Comparison Mode (--compare)

To verify performance gains and detect regressions in later phases:

node perf/baseline.mjs --compare perf/results/baseline-2026-10-07.json

Prints an ASCII/Markdown delta comparison table highlighting percentage changes across wire bytes, boot duration, and update overhead.