josh/ytplayer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
7bc5b4f8b50d78ece3cca5100d54a97fece2748a
ytplayer/plans/perf-baseline.md
Jonathan Sykes 4bc15a55e3 Add baseline benchmark results and report
2026-10-07 04:25:14 +08:00

9.4 KiB
Raw Blame History

Performance Baseline Report (Phase 0 / UNCHANGED code)

Date: 2026-10-07
Commit: b77938a
Dataset: perf/results/baseline-2026-10-07.json

1. Executive Summary & Headline Baseline Numbers

This baseline captures the performance metrics of the unoptimized ytplayer codebase (commit b77938a, matching production tag 1175f1a1d2c1) across 5 scenarios on both Chromium and WebKit under simulated LTE and Lossy network profiles (median of 5 runs).

Headline Baseline Numbers

  • Cold First-Load Wire Transfer:

    • Chromium: 164 requests, 1,015.0 KB compressed wire bytes (boot-done median: 10,163 ms under 4× CPU throttle + LTE).
    • WebKit: 162 requests, 1,152.0 KB (1.12 MB) wire bytes (boot-done median: 3,201 ms under proxy LTE).
    • Breakdown: ~477 KB JavaScript, ~397 KB Web Fonts, ~131 KB CSS, ~27 KB HTML.

  • Warm Reload (under Service Worker):

    • Chromium: 4 requests, 5.2 KB wire bytes (periodic API version poll), boot-done median: 675 ms (LTE) / 651 ms (Lossy).
    • WebKit: 0 requests, 0 KB wire bytes, boot-done median: 131 ms (LTE) / 131 ms (Lossy).

  • Offline Reload (under Service Worker):

    • Chromium: 0 requests, 0 B wire bytes, reload median: 240 ms, boot-done median: 219 ms.
    • WebKit: 0 requests, 0 B wire bytes, reload median: 135 ms, boot-done median: 128 ms.

  • Update N -> N+1 Wire Cost (The Invalidation Penalty):

    • 1-line app.js change:
      • Chromium: 88 requests, 596.6 KB wire bytes (~611 KB uncompressed/brotli), duration: 2,013 ms (LTE) / 2,031 ms (Lossy).
      • WebKit: 86 requests, 664.3 KB wire bytes (~680 KB wire), duration: 4,457 ms (LTE) / 8,135 ms (Lossy).
    • 1-line theme-glass.css change:
      • Chromium: 88 requests, 596.7 KB wire bytes, duration: 1,970 ms (LTE) / 2,004 ms (Lossy).
      • WebKit: 86-87 requests, 664.3 KB wire bytes, duration: 4,510 ms (LTE) / 8,269 ms (Lossy).

Core Invalidation Finding: A single 1-line change to any static file (app.js or theme-glass.css) mutates BUILD_TAG calculated by computeBuildTag(). This causes sw.js to re-fetch and re-cache all 86-88 shell assets (~600-664 KB wire payload) rather than just the mutated resource.

2. Benchmark Harness Execution

The benchmark harness is located in perf/baseline.mjs and driven by Playwright with an isolated ephemeral Bun server and a custom pacing proxy (perf/proxy.mjs).

Quick Start / Exact Commands

# Ensure node_modules is symlinked to root
ln -s ~/development/personal/ytplayer/node_modules node_modules

# Run full baseline suite (all scenarios, both browsers, both profiles, 5 runs)
node perf/baseline.mjs --runs 5 --browser all --profile all --scenario all --out perf/results/baseline-2026-10-07.json

# Run individual browser or scenario
node perf/baseline.mjs --runs 5 --browser chromium --profile lte --scenario cold
node perf/baseline.mjs --runs 5 --browser webkit --profile lte --scenario update-js

# Run comparison against baseline after optimizations
node perf/baseline.mjs --runs 5 --browser all --profile all --scenario all --compare perf/results/baseline-2026-10-07.json

3. Results by Browser and Profile (Median of 5 Runs)

Chromium — LTE (1.6 Mbps down, 750 kbps up, 150 ms RTT, 4× CPU throttle)

Scenario Reqs (med) Wire Bytes (med) FCP (ms) LCP (ms) Boot Done (ms) SW Ready / Reload (ms) Long Task Total (ms)
Cold First Load 164 1,015.0 KB 1,540 1,896 10,163 38,446 397
Warm Reload (SW) 4 5.2 KB 180 684 675 - 156
Offline Reload 0 0.0 KB 72 108 219 240 -
Update N->N+1 (JS) 88 596.6 KB - - - 2,013 -
Update N->N+1 (CSS) 88 596.7 KB - - - 1,970 -
  • Storage Estimate: Quota = 4,298,447,325 B (~4.0 GB), Usage = 3,480,029 B (~3.31 MB).
  • Cold Wire Bytes by Type: JS 477.2 KB, Fonts 397.1 KB, CSS 130.7 KB, HTML 27.1 KB, Images 6.3 KB, API/Other 1.0 KB.

Chromium — Lossy (1.0 Mbps down, 500 kbps up, 250 ms RTT, 4× CPU throttle)

Scenario Reqs (med) Wire Bytes (med) FCP (ms) LCP (ms) Boot Done (ms) SW Ready / Reload (ms) Long Task Total (ms)
Cold First Load 164 1,015.0 KB 2,344 2,912 16,015 42,518 432
Warm Reload (SW) 4 5.2 KB 184 644 651 - 155
Offline Reload 0 0.0 KB 80 92 237 253 -
Update N->N+1 (JS) 88 596.6 KB - - - 2,031 -
Update N->N+1 (CSS) 88 596.7 KB - - - 2,004 -

WebKit — LTE (1.6 Mbps down, 150 ms RTT via proxy)

Scenario Reqs (med) Wire Bytes (med) FCP (ms) LCP (ms) Boot Done (ms) SW Ready / Reload (ms) Long Task Total (ms)
Cold First Load 162 1.12 MB 1,088 1,287 3,201 30,508 0*
Warm Reload (SW) 0 0.0 KB 110 110 131 - 0*
Offline Reload 0 0.0 KB 107 107 128 135 -
Update N->N+1 (JS) 86 664.3 KB - - - 4,457 -
Update N->N+1 (CSS) 86 664.3 KB - - - 4,510 -

WebKit — Lossy (1.0 Mbps down, 250 ms RTT + 1,500 ms stalls via proxy)

Scenario Reqs (med) Wire Bytes (med) FCP (ms) LCP (ms) Boot Done (ms) SW Ready / Reload (ms) Long Task Total (ms)
Cold First Load 162 1.12 MB 3,557 3,735 7,303 35,449 0*
Warm Reload (SW) 0 0.0 KB 111 112 131 - 0*
Offline Reload 0 0.0 KB 109 119 129 141 -
Update N->N+1 (JS) 86 664.3 KB - - - 8,135 -
Update N->N+1 (CSS) 87 664.4 KB - - - 8,269 -

*Note: WebKit does not expose PerformanceLongTaskTiming (longtask entries), so long task total evaluates to 0 ms.

4. Variance & Noise Observed

Over the 5 runs per configuration, measurements demonstrated high stability:

  • Wire Bytes & Request Counts: Deterministic (0% variance across identical runs).
  • Cold Boot-Done:
    • Chromium LTE: Min 10,129 ms, Max 10,798 ms (<6% variance).
    • Chromium Lossy: Min 15,871 ms, Max 16,031 ms (<1% variance).
    • WebKit LTE: Min 3,138 ms, Max 3,245 ms (<3% variance).
    • WebKit Lossy: Min 6,963 ms, Max 8,710 ms (~20% variance due to proxy random stalls).
  • Warm & Offline Reload:
    • Chromium Offline Reload: Min 238 ms, Max 257 ms (<8% variance).
    • WebKit Offline Reload: Min 131 ms, Max 141 ms (<7% variance).
  • Update Duration:
    • Chromium: Runs cluster around ~2,000 ms (2,013 ms median). Occasional outlier runs (~45–200 ms) occur if the browser resolves reg.update() immediately before network completion reporting.
    • WebKit: Steady scaling between LTE (~4.5 s) and Lossy (~8.2 s) pacing.

5. Limitations & Caveats

  1. WebKit Network Throttling: WebKit lacks Chrome DevTools Protocol (CDP) support for hardware network emulation and CPU throttling. We paced throughput (1.6 Mbps) and injected latency/stalls using a custom local reverse proxy (perf/proxy.mjs). This throttles HTTP response chunks truthfully but cannot simulate OS-level socket buffer delays or 4× CPU throttling.
  2. Autoplay Policy in WebKit: Playwright WebKit on Linux rejects --autoplay-policy=no-user-gesture-required and enforces Safari-style gesture requirements on unmuted audio elements. The media playback benchmark detects this and correctly reports autoplayBlocked: true without failing the test run.
  3. WebKit SetOffline Incompatibility: Calling Playwright's context.setOffline(true) on Linux WebKit causes an unrecoverable engine crash (WebKit encountered an internal error). Offline mode for WebKit is simulated by terminating the proxy upstream connection (proxy.setOffline(true)).

6. Manual iPhone Testing Protocol (Phase 0 / iOS Safari PWA)

Because automated headless WebKit cannot fully replicate iOS Safari's mobile background constraints and battery telemetry, the following manual protocol is required:

Protocol Steps

  1. Install PWA:

    • Open Safari on iPhone and navigate to https://worship.hesed.sbs.
    • Tap the Share button -> "Add to Home Screen".
    • Launch ytplayer directly from the Home Screen icon.

  2. Verify Version & Cache:

    • Verify the footer / version string matches 1175f1a1d2c1 (or current production build tag).
    • Turn on Airplane Mode and force-reload to confirm offline readiness. Turn Airplane Mode back off.

  3. 30-Minute Screen-Off Battery Test:

    • Note baseline battery percentage (e.g., 85%).
    • Select a pre-cached playlist or track and tap Play. Ensure continuous audio is heard.
    • Lock the screen (display off) and let playback run uninterrupted for 30 minutes.
    • Note battery percentage at 30 minutes (calculate drain %/hr).
    • Listen for any audio dropouts, stuttering, or premature process suspension.

  4. Interruption & Resumption Test:

    • Trigger an interruption while locked (e.g., initiate an incoming phone call or start Siri).
    • End the interruption and observe if playback automatically resumes via MediaSession handlers.
    • Test Lock Screen Controls (Play/Pause, Next Track, scrubber).

  5. Log Findings:

    • Record iPhone Model, iOS Version, Battery Drain (% per 30m), Dropouts Count (0 = pass), and Resumption Status (Pass/Fail).