Add piano transcription, interactive key visualizer, and practice queue features

Add admin analytics, metadata collection, and grouped lyric cues
Queue reviewable Whisper drafts from the song list and lyrics editor. Preserve line breaks within one timed cue across editing, saving, reporting, and service views. Add storage and listening analytics with a durable metadata collector, related-search depth, video limits, thumbnail storage, and a browsable metadata library.
2026-10-03 07:55:33 +08:00 · 2026-10-03 07:55:33 +08:00 · 2026-10-03 07:55:33 +08:00
24 changed files with 1485 additions and 48 deletions
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -79,7 +79,7 @@ services:
      - "traefik.http.routers.ytplayer-https.tls.certresolver=letsencrypt"
      - "traefik.http.services.ytplayer-svc.loadbalancer.server.port=3000"
-  # Transcribes saved songs that have no lyrics yet, one at a time, with
+  # Handles requested Whisper drafts and transcribes saved songs without lyrics, with
  # faster-whisper on CPU (no API keys, no credits). Separate container so it
  # never competes with playback/downloads: capped CPU + memory, low priority,
  # and it fetches audio from the ytplayer service over the private network.
--- a/docs/admin-lyrics-and-analytics.md
+++ b/docs/admin-lyrics-and-analytics.md
@@ -0,0 +1,122 @@
 # Lyrics and server analytics
 ## Whisper drafts
 In `/admin#editor`, every song has a Whisper button alongside its Open button.
 The loaded song also has **Transcribe with Whisper**. Both enqueue the same
 server job; duplicate active requests reuse that job. The existing CPU
 faster-whisper container polls explicit requests every ten seconds and returns
 a timed draft even when that song already has lyrics. Manual requests always
 use Whisper, bypassing LRCLIB.
 **Use transcript in editor** loads the draft for review. **Save lyrics** publishes
 it through the existing revision history and conflict checks. Until saved,
 shared lyrics remain unchanged. Applying a draft can be undone, including the
 original sync offset. Queued/running jobs survive restarts; worker leases expire
 after two minutes and heartbeat every 25 seconds. After three failed worker
 attempts the job reports a failure. Completed/failed drafts are retained for
 30 days; cleanup runs when workers report progress.
 A YouTube video must already have a ready server media copy; uploads are also
 supported. The existing maximum of one hour and 40 MiB audio input applies.
 Songs without enough recognized vocals fail visibly. Transcription accuracy
 and word timing depend on the singing and audio mix; review before saving.
 Deploy the updated server and rebuild the `lyrics-worker` image. Both need the
 same `LYRICS_WORKER_TOKEN` (at least 24 characters; compose passes it to the
 worker as `YTP_TOKEN`). A configured token permits queuing; the status endpoint
 also reports whether a worker has recently polled. No paid transcription API
 is involved.
 Admin endpoints (admin cookie or API token):
 - `POST /api/admin/transcriptions/:video` queues a draft.
 - `GET /api/admin/transcriptions/:video` returns the latest draft/progress.
 Worker-only endpoints require the worker token:
 - `POST /api/lyrics-worker/claim` claims one job.
 - `POST /api/lyrics-worker/jobs/:job` reports progress/results with its lease.
 ## Grouped lyric lines
 One lyric cue can contain multiple visual lines with one timestamp. In the
 admin editor, **Shift+Enter** inserts a tight line break. On phones, select the
 cue and use **Line break** in its options. Ordinary Enter or **Below** adds a
 separate cue with its own timing. Both visual lines of a grouped cue highlight
 and seek together in service mode, with a smaller internal gap than the space
 between cues. The player panel, stage and presenter views also preserve breaks.
 Text editing/export uses a continuation prefix to preserve the grouping:
 ```text
 [0:12.00] Because You are God
 | You can do anything
 [0:18.00] Another separate cue
 ```
 The continuation belongs to the preceding cue. JSON stores its text as
 `"Because You are God\nYou can do anything"`. Saving, revisions and text
 round trips retain the single timestamp. Reporting a wrong lyric also preserves the grouped text. Existing single-line lyrics work
 unchanged.
 ## Server analytics and metadata collector
 Open **Stats** (`/admin#analytics`). It shows cached video and upload totals,
 recorded plays, discovery sources, metadata/thumbnail payload sizes, and
 available filesystem space for media, uploads and the database. Paths on the
 same device share their free space; their capacities must not be added.
 Media totals come from library records and exclude temporary files and
 filesystem overhead. Metadata bytes exclude indexes and SQLite overhead.
 The collector accepts a search, a total unique-video limit (1–500), and depth:
 - **0:** only the specified search.
 - **1:** also search channels and tags found in its results.
 - **2/3:** follow up to two/three related rounds.
 The total limit applies across every round. The collector reserves result
 budget for deeper rounds, deduplicates video IDs and related queries, and stops
 after at most 24 searches. Duplicate, unavailable or empty results can produce
 fewer videos than requested. Up to three collections may be queued/running;
 one executes at a time per server. Atomic ownership leases prevent two
 servers from claiming the same job. Progress is saved after each discovery
 batch and video. An expired interrupted job resumes its pending items.
 **Stop collection** cancels future work; an extraction already in progress
 can finish its network request before the cancellation is observed.
 Discovery cards enter the same catalog used for recommendations. Each video
 is then enriched through yt-dlp without downloading media. The server stores
 full descriptive fields (including descriptions, dates, engagement counts,
 language, tags, chapters, thumbnail variants and format specifications) in
 `video_details`. Expiring media URLs and request headers are excluded. Details
 are capped at 500 KB per video and follow catalog eviction. Thumbnail image
 bytes use the existing durable thumbnail queue, host restrictions and budget
 (default 512 MiB). Failures stay visible in job history; successful cards remain
 stored even when enrichment fails. Refresh analytics to update the aggregate
 numbers after collection; job progress polls automatically while this tab is
 open.
 The metadata library supports title/channel/tag filtering, 50-row pages and a
 raw descriptive metadata view. Admin cookie or API token is required for:
 - `GET /api/admin/analytics`
 - `GET /api/admin/metadata?q=...&offset=...`
 - `GET /api/admin/metadata/:id`
 - `GET /api/admin/collections`
 - `POST /api/admin/collections` with `{query,maxVideos,depth}`
 - `POST /api/admin/collections/:id/cancel`
 ## Verification
 Run `bun run test` in `server`, `npm test` at the repo root,
 `python3 -m unittest discover -s scripts/lyrics -p test_auto_lyrics.py`, and
 `npx playwright test -c playwright.admin.config.js`. Admin browser fixtures
 cover Whisper review/save/undo, unsaved edits, failures, mobile actions,
 grouped cues and service highlighting, and analytics collector controls.
 Design review fixed the primary action’s white-on-gradient contrast by using a
 solid purple fill, and fixed tablet navigation overflow. No new findings were
 suppressed. Existing admin play/tap gradient contrast and decorative glow
 findings remain outside these controls. Real viewport captures passed the
 mobile dashboard/collector and desktop visual checks.
--- a/docs/feature-ideas.md
+++ b/docs/feature-ideas.md
@@ -0,0 +1,304 @@
 # ytplayer — Feature Ideas & Enhancement Roadmap
 > **Status:** Research & Proposed Roadmap  
 > **Target:** Web PWA, Tauri (Windows), zero-native (macOS / Linux)  
 > **Date:** October 2026  
 ---
 ## 1. Context & Architecture Baseline
 `ytplayer` is an ad-free, privacy-preserving media engine with a dual identity:
 1. **Ad-Free Media Player & Offline PWA**: Built with vanilla JS on the frontend and Bun + Hono on the backend (libsql SQLite database). It uses `yt-dlp` and InnerTube for direct stream extraction without Google accounts, and caches media locally (OPFS in browser, single-file H.264/HEVC faststart on server, WebRTC P2P mesh between devices).
 2. **Live Worship & Stage Performance Tool**: Equipped with **Service Mode** (wake-locked setlist view, auto-scaling lyric fonts, per-song arrangement notes), synchronized LRC lyrics with section/cue tags, and a **Presenter screen** (`/?present=<code>`) with QR pairing.
 This document synthesizes high-value feature opportunities gathered from research into modern YouTube frontends (FreeTube, ReVanced, SmartTube, Namida), modern music players (Plexamp, Spotube), and live stage tools (Planning Center Music Stand, OnSong, MultiTracks Playback).
 ---
 ## 2. Feature Proposals by Domain
 ### Domain 1: Smart YouTube & Playback Experience
 #### 1.1 SponsorBlock & Non-Music Segment Skipping
 * **The Concept**: Automatically skip or visually mark sponsored segments, self-promos, interaction reminders, and non-music sections.
 * **Why it matters**: Music videos frequently feature skits, prolonged silent pauses, or spoken intros. Skipping `music_offtopic` ensures the music starts immediately.
 * **Technical Path**:
  * Free, unauthenticated REST API: `GET https://sponsor.ajay.app/api/skipSegments?videoID=<id>&categories=["sponsor","intro","outro","music_offtopic","selfpromo"]`.
  * Highlight segments as color-coded spans over the seek bar (`#timeline` / Glass Stage scrubber).
  * During the playback tick in `frontend/app.js`, if `currentTime` enters a skippable segment and auto-skip is enabled in Settings, seek to the segment end and show an "Unskip" toast.
  * Server media cache can optionally apply `--sponsorblock-mark` during ffmpeg intake.
 #### 1.2 Automatic YouTube Chapter Extraction & Interactive Scrubber
 * **The Concept**: Automatically parse video chapters from YouTube metadata and display chapter splits on the progress bar.
 * **Why it matters**: Currently, chapters are only created manually. Full albums, sermons, conference talks, and live sets already have timestamps in video descriptions.
 * **Technical Path**:
  * In `server/server.js` (`/api/streams`), expose `chapters: [{ start_time, end_time, title }]` directly from InnerTube or `yt-dlp`.
  * Render tick marks / gaps on the progress scrubber for each chapter boundary.
  * Display the current chapter title next to the timestamp.
  * Add keyboard shortcuts: `[` (previous chapter) and `]` (next chapter).
 #### 1.3 Return YouTube Dislike (RYD) & Quality Rating
 * **The Concept**: Show community like/dislike ratios and ratings for videos.
 * **Why it matters**: Helps identify misleading arrangements, clickbait, poor audio rips, or broken videos before playback.
 * **Technical Path**:
  * Query `https://returnyoutubedislikeapi.com/votes?videoId=<id>` (public, CORS-enabled).
  * Display a discreet like ratio bar or percentage badge on the video card and now-playing details.
 #### 1.4 On-Screen Subtitles & Closed Captions (CC) Overlay
 * **The Concept**: Render synchronized captions directly over the video playback stage.
 * **Why it matters**: `ytplayer` currently only uses captions for transcript search or importing into lyrics. Spoken sermons, tutorials, foreign tracks, and non-musical videos need readable on-screen subtitles.
 * **Technical Path**:
  * Add a `CC` button on the player controls.
  * Fetch timed lines from `/api/notes/:id/captions`.
  * Render an HTML overlay synchronized to `Player.master.currentTime` with settings for font size, background opacity, and positioning.
 #### 1.5 Silence Skipping (Talk & Sermon Mode)
 * **The Concept**: Automatically speed up or skip past silent gaps in spoken-word media.
 * **Why it matters**: Long sermons, podcasts, and talks often have pauses; skipping silence saves 10–20% of listening time without altering speech tone.
 * **Technical Path**:
  * Connect an `AnalyserNode` or `AudioWorklet` in the Web Audio graph.
  * If audio energy (RMS) stays below -45 dB for >800 ms, ramp `playbackRate` to 2.5× until speech resumes.
 ---
 ### Domain 2: Audiophile & Music Player Polish
 #### 2.1 EBU R128 / LUFS Loudness Normalization
 * **The Concept**: Eliminate jarring volume jumps between tracks in playlists and queues.
 * **Why it matters**: YouTube master volumes vary widely (-24 LUFS to -8 LUFS). Normalization ensures consistent listening levels.
 * **Technical Path**:
  * When generating waveform peaks in `server/media-cache.js`, run ffmpeg's `ebur128` filter once to measure integrated loudness (LUFS).
  * Store `lufs` in the database and pass it via `/api/streams` and `/api/media/:id`.
  * In `frontend/app.js`, apply a Web Audio `GainNode` to normalize tracks to a standard -14 LUFS target.
 #### 2.2 Real-Time Pitch Shifting & Independent Tempo Adjustment
 * **The Concept**: Transpose playback key (±6 semitones) without altering tempo, or slow down tempo without changing pitch.
 * **Why it matters**: Singers and worship teams often need to adapt songs to a band's vocal key. Instrumentalists need to slow down fast sections to learn them.
 * **Technical Path**:
  * Use an AudioWorklet running a Phase Vocoder or `SoundTouchJS` DSP processor.
  * Provide stepper controls for semitone transposition and tempo percentage (0.5×–1.5×).
 #### 2.3 Ambient Glow / Reactive Lighting Mode
 * **The Concept**: A soft dynamic glow behind the video player that mirrors current video colors (similar to YouTube Ambient Mode or Apple Music dynamic art).
 * **Technical Path**:
  * Render video frames onto an offscreen 16×16 `<canvas>` at ~10 FPS.
  * Display a blurred canvas (`filter: blur(50px)`) directly behind the player container with smooth CSS opacity.
  * Auto-disable when `settings.perfMode` is active or under `prefers-reduced-motion`.
 #### 2.4 Smart Autoplay Radio / Endless Mix
 * **The Concept**: Keep playing related songs when the active playlist or queue concludes.
 * **Technical Path**:
  * Fetch candidate related tracks via InnerTube `next` continuity tokens.
  * Re-rank candidates using the client's local `StatsTrack` (`playCount`, listening duration, and skips) to favor familiar favorites while avoiding immediate repeats.
 #### 2.5 Audio Stem Separation & Vocal Attenuator (Karaoke / Rehearsal Mode)
 * **The Concept**: Mute or attenuate lead vocals in real time (`🎤 Vocal: 100% ────● 0%`), or isolate the vocal track for harmony practice.
 * **Why it matters**: Gives singers clean backing tracks for rehearsal and live events even when official minus-one recordings do not exist.
 * **Technical Path**:
  * Client-side: Run `demucs-onnx` with WebGPU acceleration to process saved OPFS tracks offline.
  * Server-side: Provide `GET /api/media/:id/stems` (caching `.vocals.m4a` and `.instrumental.m4a` alongside the existing `.m4a` audio sidecar in `server/media-cache.js`).
  * Frontend: Two synchronized audio elements or Web Audio `GainNode` faders for seamless real-time blending.
 #### 2.6 10-Band Graphic Equalizer & Spatial Hall Simulation
 * **The Concept**: Upgrade the existing 5-band filter in `EQ` (`frontend/app.js`) to a full 10-band graphic equalizer with acoustic presets and spatial audio widening.
 * **Technical Path**:
  * 10 `BiquadFilterNode` bands (31Hz, 62Hz, 125Hz, 250Hz, 500Hz, 1kHz, 2kHz, 4kHz, 8kHz, 16kHz).
  * Presets: *Vocal Clarity (speech presence boost), Acoustic Warmth, Worship Stage, Bass Punch, and Flat*.
  * Spatial stereo widener using a Haas-effect micro-delay and `StereoPannerNode` for an immersive headphone soundstage.
 ---
 ### Domain 3: Live Worship, Band & Stage Performance Engine
 #### 3.1 Interactive ChordPro Sheets & Live Transposition
 * **The Concept**: Display chords inline or above lyrics with 1-tap live transposition and capo calculation.
 * **Why it matters**: Replaces the need for external music stand apps (Planning Center Music Stand, OnSong) on stage.
 * **Technical Path**:
  * Extend `frontend/lyrics-core.js` to parse ChordPro directives (`[G]`, `[D/F#]`, `{key: G}`).
  * Add a Transpose controller (+1 / -1 semitone, Capo selector) that dynamically recalculates chord symbols in real time.
  * Add a view toggle: "Lyrics Only" vs. "Chords & Lyrics".
 #### 3.2 Bluetooth Foot Pedal / Page-Turner Support
 * **The Concept**: Hands-free navigation for musicians on stage using standard Bluetooth foot pedals (AirTurn, PageFlip, Donner, Stomp).
 * **Why it matters**: Musicians cannot remove their hands from instruments while playing.
 * **Technical Path**:
  * Standard foot pedals map to keyboard keys: `PageDown`, `PageUp`, `ArrowUp`, `ArrowDown`.
  * Extend the global `keydown` listener in `frontend/app.js` to handle pedal events in Service Mode:
    * `PageDown` / `PageUp`: Scroll down / up by one lyric section.
    * Configurable actions: Next/Previous track, trigger A-B loop, or jump to Chorus.
 #### 3.3 Confidence Monitor & Stage Clock (Presenter Mode)
 * **The Concept**: A high-visibility stage display mode for the Presenter screen (`/?present=<code>`).
 * **Why it matters**: Singers and worship leaders need glanceable cues for upcoming lyrics, section names, and remaining time.
 * **Technical Path**:
  * In `Presenter` (`frontend/app.js`), provide a **Confidence Monitor** layout:
    * **Current Line**: Extra-large, high-contrast text.
    * **Next Line Preview**: Smaller, dimmed text directly underneath.
    * **Section Badge**: Prominent label (e.g. `# CHORUS 2`).
    * **Stage Clock & Timer**: Current wall clock plus elapsed/remaining song timer.
 #### 3.4 Broadcast Lower-Third Overlay for OBS & Live Streams
 * **The Concept**: A dedicated live stream output URL (`/?present=<code>&mode=lowerthird`) designed to be added as a transparent Browser Source in OBS, vMix, or ATEM.
 * **Why it matters**: Enables churches and streamers to overlay synchronized lower-third lyrics onto their video feed without separate presentation software.
 * **Technical Path**:
  * Transparent background (`background: transparent;`).
  * Auto-updating 2-line broadcast graphic with smooth slide/fade transitions and drop-shadows.
  * Driven in real-time by the host or phone remote via the existing WebSocket relay in `server/remote.js`.
 #### 3.5 Section Quick-Jump Pad
 * **The Concept**: A grid of buttons representing song sections (`[Intro]`, `[Verse 1]`, `[Chorus]`, `[Bridge]`, `[Outro]`).
 * **Why it matters**: Worship teams often repeat choruses or spontaneously transition between sections during live services.
 * **Technical Path**:
  * Parse `# Section` headers from synced lyrics in `frontend/lyrics-core.js`.
  * Render an arrangement button strip in Service Mode.
  * Tapping a section immediately seeks playback to that section's start timestamp.
 #### 3.6 Hardware MIDI Controller Integration (Web MIDI API)
 * **The Concept**: Direct plug-and-play support for hardware USB/Bluetooth MIDI pedalboards (Morningstar MC3/MC6, Looptimus, Behringer FCB1010) without native drivers.
 * **Technical Path**:
  * Use the browser `navigator.requestMIDIAccess()` API.
  * Map MIDI Note-On and CC messages to player actions (Play/Pause, Next Track, Chorus Repeat, Tap Tempo, Mute).
 #### 3.7 Planning Center Services (PCO) & SongSelect Setlist Importer
 * **The Concept**: One-click setlist and chord chart importing from Planning Center Services and CCLI SongSelect.
 * **Technical Path**:
  * Import setlist text or CSV files from PCO.
  * Auto-match titles with YouTube media, attaching keys (`@ Key E`) and chord sheets directly to playlist entries.
 #### 3.8 Bilingual & Phonetic Lyrics (Romanization / Transliteration)
 * **The Concept**: Simultaneous dual-language or phonetic pronunciation lyrics (e.g. Mandarin with Pinyin, Spanish/English).
 * **Technical Path**:
  * Extend `frontend/lyrics-core.js` to parse secondary translation lines.
  * Display translations or romanized cues in smaller muted text under the primary lyric line.
 #### 3.9 Visual Metronome & Count-In Click
 * **The Concept**: Visual flash or gentle audio click count-in before track playback begins.
 * **Why it matters**: Enables the band to start playing on beat before the recorded track introduces audio.
 * **Technical Path**:
  * When `@ 70 BPM` metadata is present in song notes, offer an optional 4-beat visual flash on the stage frame or Web Audio click before playback starts.
 ---
 ### Domain 4: Interactive Piano Transcription, Sheet Music & Practice Engine
 #### 4.1 Automatic Audio-to-Piano Transcription (Polyphonic Audio to Score)
 * **The Concept**: Transcribe piano melodies and chords from recorded audio into polyphonic note events (MIDI / MusicXML).
 * **Why it matters**: Allows musicians, vocalists, and students to learn the exact piano parts played in any YouTube song or uploaded track.
 * **Technical Path**:
  * **Option A (Client-Side WASM / WebGPU)**: Run Spotify's **Basic Pitch** model (`@spotify/basic-pitch`) in the browser via TensorFlow.js or ONNX Runtime Web. It performs polyphonic pitch prediction directly on decoded audio frames without server compute.
  * **Option B (Server-Side Worker, like `lyrics-worker`)**: Run ByteDance's `piano_transcription_inference` (High-Resolution Piano Transcription on PyTorch/ONNX) inside a container worker. Saves `<id>.notes.json` / `<id>.mid` to the cache directory.
  * Output data shape: Array of note events `[{ pitch: 60, start: 1.25, end: 1.85, velocity: 84 }]`.
 #### 4.2 Sheet Music Generation & Real-Time Scrolling Score
 * **The Concept**: Convert transcribed note events into standard grand staff sheet music (Treble and Bass clefs) rendered on screen.
 * **Technical Path**:
  * Render sheet notation on an HTML5 `<canvas>` or `<svg>` using **VexFlow** or **OpenSheetMusicDisplay (OSMD)**.
  * Display a vertical playhead cursor that scrolls synchronously with `Player.master.currentTime`.
  * Support instant transposition: adjusting the song key shifts the musical notation and key signature on the fly.
 #### 4.3 Interactive Virtual Piano & Falling Notes (Synthesia Style)
 * **The Concept**: A responsive 88-key or 61-key virtual piano keyboard that visualizes which keys to press in sync with playback.
 * **Visual Modes**:
  * **Falling Notes Mode**: Vertical roll above the keyboard where color-coded note bars fall downward toward the keys (Treble notes in Amber/Coral, Bass notes in Cyan/Blue). When a bar hits the key line, the corresponding piano key illuminates and depresses.
  * **Sheet Music + Lit Keys Mode**: Traditional score at the top, illuminated keyboard at the bottom highlighting active notes in real time.
 * **Technical Path**:
  * Canvas-rendered piano keyboard with responsive key sizing that fits desktop, tablet, and mobile orientations.
  * Note rendering loop tied to `requestAnimationFrame` and driven by the current playback timestamp.
 #### 4.4 Queued Song & Section Learning Mode
 * **The Concept**: A dedicated learning queue and practice loop system that guides users step-by-step through mastering a song.
 * **Features**:
  * **Queue Practice Sessions**: Queue an entire song, specific sections (e.g. `[Intro]`, `[Verse 1]`, `[Chorus]`), or arbitrary A-B loop bars into the practice queue.
  * **Tempo Slowdown (Pitch-Preserved)**: Slow down playback to 50%, 65%, 75%, or 90% speed without altering key.
  * **Speed Ramp Auto-Trainer**: Start a looped section at 60% speed; each time the user completes the loop, speed automatically increases by +5% until 100% full tempo is achieved.
  * **Loop Pause Interval**: A configurable 1–2 beat pause between repetitions to allow players to reset their hand position.
 #### 4.5 Interactive "Wait-for-Note" Practice via Hardware MIDI Input
 * **The Concept**: Connect any USB or Bluetooth digital piano / MIDI keyboard to the computer or tablet. The player pauses and waits for the user to press the correct key(s) before advancing!
 * **Technical Path**:
  * Listen to incoming MIDI `noteon` events via the Web MIDI API (`navigator.requestMIDIAccess()`).
  * Compare the user's played MIDI note number against active notes in the transcription dataset.
  * If the note is correct, turn the key green and resume playback to the next note event. If incorrect, turn the key red and provide visual guidance on the virtual keyboard.
 #### 4.6 Technical Feasibility Analysis
 | Component | Technical Feasibility | Resource Cost & Constraints | Recommendation |
 |:---|:---|:---|:---|
 | **Audio-to-MIDI Transcription** | **High (Proven)** | Basic Pitch: ~15 MB model file, runs on client WebGPU/CPU in ~0.3×–0.8× real time. ByteDance: higher accuracy, requires ~1.5 GB PyTorch model on server. | Use **Basic Pitch (WASM/ONNX)** on client for instant preview; offload to homelab server worker for full-song high-fidelity scores. |
 | **Piano Roll / Virtual Keyboard** | **High (Straightforward)** | Pure HTML5 Canvas rendering. Minimal CPU overhead (~60 FPS with <5% CPU usage). | Built natively into frontend without heavy external dependencies. |
 | **Sheet Music Engraving** | **High (Proven)** | **VexFlow** (~180 KB minified) renders standard grand staff notation cleanly from JSON notes. | Dynamic import (`import()`) of VexFlow only when the user opens Sheet View. |
 | **Interactive MIDI Feedback** | **High (Zero-latency)** | Web MIDI API is supported natively in Chrome, Edge, and Opera (and via polyfill in Firefox/Safari). Latency is sub-5ms. | Native browser event listeners listening to MIDI input. |
 ---
 ### Domain 5: Collaboration, Windowing & Mobile Experience
 #### 5.1 Document Picture-in-Picture (Always-On-Top Lyrics & Chords)
 * **The Concept**: An always-on-top floating desktop window containing interactive synchronized lyrics, chords, and transport controls.
 * **Technical Path**:
  * Use the Chrome/Edge `documentPictureInPicture.requestWindow({ width: 420, height: 500 })` API.
  * Moves the active `#stageLyrics` or ChordPro view into the floating window with full DOM interactivity.
 #### 5.2 Voice Search & Driving Car Mode
 * **The Concept**: A hands-free, distraction-free interface for driving or kitchen listening.
 * **Features**:
  * **Voice Search**: Microphone button on the search bar powered by `webkitSpeechRecognition` for real-time speech-to-query search.
  * **Car Mode**: One-tap toggle for a simplified layout with giant 100px touch buttons (Play/Pause, Next, Previous, 15s Rewind) and disabled scroll lists.
 #### 5.3 Collaborative Setlists & Playlists
 * **The Concept**: Multiple devices editing a single playlist simultaneously with live sync.
 * **Technical Path**:
  * Extend playlist sync in `server/db.js` with shared write keys.
  * Broadcast item reordering and song additions over the existing WebSocket hub in `server/party.js`.
 #### 5.4 Watch Party DJ Mode & Upvote Queue
 * **The Concept**: Allow guests in a watch party (`/ws/party`) to submit song requests that the group can upvote.
 * **Technical Path**:
  * Add a "Suggested Queue" tab in the watch party drawer.
  * Upvotes dynamically sort songs; when current playback finishes, the highest-voted song plays next.
 ---
 ### Domain 6: Library, Offline & Sync Innovations
 #### 6.1 Visual Download Manager & Storage Dashboard
 * **The Concept**: A unified management dashboard in Settings for offline media.
 * **Technical Path**:
  * Show active download tasks with real-time progress bars.
  * List storage usage grouped by playlist, audio vs. video format, and OPFS vs. server cache.
  * Add controls: "Pause All", "Resume All", and "Retry Failed".
 #### 6.2 Smart Auto-Offline Favorites
 * **The Concept**: Keep a user's most essential songs available offline automatically.
 * **Technical Path**:
  * Setting: *"Keep Top 25 Most-Played Songs Offline"* or *"Keep Active Setlist Offline"*.
  * Automated background routine checks local `playCount` and triggers cache downloads for missing items during idle time.
 #### 6.3 Self-Contained Offline Playlist Bundling (Zip Export)
 * **The Concept**: Export a full offline bundle containing playlist JSON plus the saved audio/video files for complete air-gapped device migration.
 ---
 ## 3. Implementation Matrix & Recommended Phases
 | Phase | Feature | Effort | Prerequisites & Reused Modules |
 |:---|:---|:---:|:---|
 | **Phase 1: Quick Wins** | **SponsorBlock & Non-Music Skip** | Low | Free SponsorBlock API, playback tick in `frontend/app.js` |
 | | **Automatic YouTube Chapter Ticks** | Low | InnerTube metadata in `server/server.js`, timeline scrubber |
 | | **Bluetooth Foot Pedal Shortcuts** | Low | Keydown event listener in `frontend/app.js` |
 | | **Section Quick-Jump Pad** | Low | Lyric section parser in `frontend/lyrics-core.js` |
 | | **Voice Search (SpeechRecognition)** | Low | Web Speech API, search input handler |
 | **Phase 2: Stage & Audio** | **EBU R128 LUFS Normalization** | Medium | ffmpeg `ebur128` in `server/media-cache.js`, Web Audio GainNode |
 | | **ChordPro Display & Transpose** | Medium | `frontend/lyrics-core.js`, chord parser |
 | | **Confidence Monitor Stage View** | Medium | `Presenter` & `fitLyricLines` in `frontend/app.js` |
 | | **OBS Lower-Third Overlay** | Low-Med | `server/remote.js` WebSocket relay, transparent route |
 | | **On-Screen Subtitles (CC)** | Medium | `/api/notes/:id/captions`, overlay renderer |
 | **Phase 3: Piano & Interactive Learning** | **Interactive Piano Roll & Key Visualizer** | Medium | HTML5 Canvas keyboard renderer, note event synchronizer |
 | | **Audio-to-MIDI Transcription** | Medium-High | Spotify Basic Pitch (WASM) / ByteDance transcription worker |
 | | **Sheet Music View (VexFlow)** | Medium | VexFlow dynamic import, MusicXML / MIDI generator |
 | | **Wait-for-Note Practice (Web MIDI)**| Medium | Web MIDI API (`navigator.requestMIDIAccess`), practice queue |
 | **Phase 4: Advanced Engine** | **Audio Stem Separation (Karaoke)** | High | ONNX Runtime Web / server Demucs |
 | | **Pitch Shift & Tempo Worklet** | High | AudioWorklet / `SoundTouchJS` DSP |
 | | **Document Picture-in-Picture** | Low-Med | Chrome Document PiP API |
 | | **Collaborative Setlists** | High | WebSocket hub in `server/party.js` |
--- a/frontend/admin.html
+++ b/frontend/admin.html
@@ -92,7 +92,7 @@
    }
    .btn:hover { background: rgba(255,255,255,.1); }
    .btn:active { transform: scale(.97); }
-    .btn.pri { border: 0; background: var(--grad); color: #fff; box-shadow: 0 10px 26px -10px var(--a1); }
+    .btn.pri { border: 0; background: #6143cf; color: #fff; box-shadow: none; }
    .btn.pri:hover { filter: brightness(1.1); }
    .btn.ok { border: 0; background: var(--ok); color: #06210f; }
    .btn.dng { color: var(--bad); border-color: rgba(255,107,107,.35); background: rgba(255,107,107,.07); }
@@ -164,6 +164,13 @@
    .song-row small { color: var(--text-dim); font-size: 12px; }
    .song-row .st { flex: none; font: 700 11px var(--ui); padding: 3px 9px; border-radius: 99px; border: 1px solid var(--line-2); color: var(--text-dim); }
    .song-row .st.has { color: var(--ok); border-color: rgba(74,222,128,.4); }
    .song-open { display: flex; align-items: center; gap: 10px; min-width: 0; flex: 1; padding: 0; border: 0; background: none; color: inherit; text-align: left; }
    .whisper-request { min-height: 44px; flex: none; }
    .whisper-draft { margin-block: 14px; }
    .whisper-draft pre { white-space: pre-wrap; overflow-wrap: anywhere; max-height: 180px; overflow: auto; font: 400 14px/1.55 var(--ui); color: var(--text); }
    #whisperStatus { color: var(--text-dim); font-size: 14px; margin-block: 10px; }
    .song { flex-wrap: wrap; }
    @media (max-width: 480px) { .song-row { flex-wrap: wrap; } .song-open { flex-basis: 100%; min-height: 44px; } }
    .chips { display: flex; flex-wrap: wrap; gap: 8px; margin-top: 12px; }
    .chip { height: 34px; padding: 0 13px; border-radius: 99px; border: 1px solid var(--line-2); background: rgba(255,255,255,.04); font-size: 13px; font-weight: 600; color: var(--text-2); max-width: 100%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
    .chip:hover { color: var(--text); border-color: var(--a1); }
@@ -211,7 +218,7 @@
    }
    .tc:hover { border-color: var(--a2); color: var(--text); }
    .ln.untimed .tc { color: var(--warn); border-style: dashed; border-color: rgba(255,180,84,.55); }
-    .ln-text { flex: 1 1 auto; min-width: 0; height: 38px; padding: 0 8px; border: 0; border-radius: 9px; background: transparent; font-size: 16px; }
+    .ln-text { flex: 1 1 auto; min-width: 0; min-height: 38px; height: auto; padding: 7px 8px; resize: vertical; overflow: hidden; line-height: 1.3; border: 0; border-radius: 9px; background: transparent; font-size: 16px; }
    .ln.section .ln-text { font-weight: 800; text-transform: uppercase; letter-spacing: .06em; font-size: 13px; color: var(--a2); }
    .ln.cue .ln-text { font-style: italic; color: var(--warn); }
    .ln.now { background: rgba(124,92,255,.16); border-color: rgba(124,92,255,.55); box-shadow: 0 0 0 1px rgba(124,92,255,.25), 0 10px 30px -16px var(--a1); }
@@ -257,6 +264,26 @@
    .tap small { display: block; font-weight: 600; opacity: .8; font-size: 11px; letter-spacing: 0; max-width: 84px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
    .tap:active { transform: scale(.93); }
    .analytics-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); gap: 12px; margin: 20px 0; }
    .metric { padding: 18px; background: var(--bg-2); border: 1px solid var(--line-2); border-radius: 14px; }
    .metric b { display: block; font-size: 26px; margin: 4px 0; }
    .metric small, .metric span { color: var(--text-2); }
    .analytics-form { display: flex; flex-wrap: wrap; align-items: end; gap: 12px; }
    .analytics-form label { display: grid; gap: 5px; min-width: 0; }
    .analytics-form .query { flex: 1 1 250px; }
    .analytics-form .field { width: 100%; min-width: 0; }
    .analytics-form button { min-height: 44px; }
    .analytics-table { overflow-x: auto; margin-top: 12px; }
    .analytics-table table { width: 100%; }
    .collection-job { padding: 14px 0; border-bottom: 1px solid var(--line-2); overflow-wrap: anywhere; }
    .collection-job p { margin: 4px 0; color: var(--text-2); }
    #metadataDetail { white-space: pre-wrap; overflow-wrap: anywhere; max-height: 400px; overflow: auto; font-size: 13px; }
    @media (max-width: 480px) { .analytics-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } .metric { padding: 12px; } .metric b { font-size: 22px; } }
    @media (min-width: 761px) and (max-width: 1100px) {
      .top { flex-wrap: wrap; }
      .tabs { order: 2; width: 100%; margin-left: 0; overflow-x: auto; }
      .tabs .tab { flex: none; }
    }
    .nav { display: none; }
    @media (max-width: 760px) {
      .tabs, .top .ghost.desk { display: none; }
@@ -347,6 +374,7 @@
          <div class="ring" id="ring"><span id="ringTxt">0/0</span></div>
          <div style="min-width:0"><h1 id="songTitle"></h1><small id="songMeta"></small></div>
          <span class="grow"></span>
          <button class="btn sm whisper-request" id="whisperBtn" type="button">Transcribe with Whisper</button>
          <button class="ghost" id="closeSong" type="button">Change song</button>
        </div>
@@ -386,8 +414,15 @@
        </div>
        <div id="msg" class="msg"></div>
        <p id="whisperStatus" role="status" aria-live="polite"></p>
        <div id="whisperResult" class="card whisper-draft hidden">
          <h2 style="font-size:18px;margin:0 0 8px">Whisper transcript ready</h2>
          <p class="muted">Saved lyrics stay unchanged until you review and save this draft.</p>
          <pre id="whisperPreview"></pre>
          <button id="whisperUse" class="btn" type="button">Use transcript in editor</button>
        </div>
        <div id="lines" class="lines"></div>
-        <textarea id="raw" class="hidden" spellcheck="false"></textarea>
+        <textarea id="raw" aria-label="Lyrics as text; prefix continuation lines with vertical bar and space" class="hidden" spellcheck="false"></textarea>
        <div class="row" style="margin-top:14px">
          <button class="btn sm" id="addLine" type="button">＋ Add line</button>
          <span class="muted" style="font-size:12.5px">Shortcuts: <code>Space</code> stamp (Tap-sync) · <code>S</code> stamp selected · <code>K</code> play · <code>↑↓</code> select · <code>⌘/Ctrl S</code> save</span>
@@ -480,6 +515,35 @@
    </section>
    <!-- ================= ACCESS ================= -->
    <section id="p-analytics" class="panel">
      <h1 class="page">Server analytics</h1>
      <p class="lead">Video storage, listening activity and the metadata library.</p>
      <button class="btn" type="button" id="analyticsRefresh">Refresh analytics</button>
      <p id="analyticsStatus" role="status" aria-live="polite"></p>
      <div class="analytics-grid" id="analyticsMetrics"></div>
      <div class="card"><h2>Remaining disk space</h2><div id="analyticsVolumes"></div><p class="muted">Volumes on the same device share the reported free space. Stored media totals exclude temporary files and filesystem overhead.</p></div>
      <div class="card"><h2>Metadata collector</h2>
        <p class="muted">Search and save descriptive video metadata and thumbnails. Video and audio files are not downloaded.</p>
        <form id="collectorForm" class="analytics-form">
          <label class="query">Search<input id="collectorQuery" class="field" type="search" maxlength="200" required placeholder="Cathedral of Praise Worship" /></label>
          <label>Total video limit<input id="collectorLimit" class="field" type="number" min="1" max="500" value="30" required /></label>
          <label>Related-search depth<select id="collectorDepth" class="field"><option value="0">0 · Search only</option><option value="1">1 · Channels and topics</option><option value="2">2 · Two related rounds</option><option value="3">3 · Three related rounds</option></select></label>
          <button class="btn pri" type="submit" id="collectorStart">Collect metadata</button>
        </form>
        <p class="muted">The limit applies across every round. Depth follows channels and tags found in video metadata, with at most 24 searches. A collection can return fewer videos when results repeat or sources are unavailable. Thumbnails follow the server’s storage budget.</p>
        <p id="collectorStatus" role="status" aria-live="polite"></p><div id="collectionJobs"></div>
      </div>
      <div class="card"><h2>Metadata library</h2>
        <form id="metadataFilterForm" class="analytics-form"><label class="query">Filter saved metadata<input id="metadataFilter" class="field" type="search" maxlength="200" placeholder="Title, channel or tag" /></label><button class="btn" type="submit">Filter</button></form>
        <p id="metadataCount" class="muted"></p>
        <div class="analytics-table"><table id="metadataTable"></table></div>
        <div class="row"><button id="metadataPrev" class="btn" type="button">Previous</button><button id="metadataNext" class="btn" type="button">Next</button></div>
        <details id="metadataDetails" class="hidden"><summary>Video metadata</summary><pre id="metadataDetail"></pre></details>
      </div>
      <div class="card"><h2>Discovery sources</h2><div class="analytics-table"><table id="analyticsSources"></table></div></div>
      <div class="card"><h2>Most played</h2><div id="analyticsTop"></div></div>
    </section>
    <section id="p-access" class="panel">
      <h1 class="page">API access</h1>
      <p class="lead">Tokens let scripts write shared lyrics and chapters. A token is shown once — store it in your secrets vault.</p>
@@ -505,6 +569,7 @@
  <nav class="nav" id="tabsBottom" role="tablist"></nav>
 </div>
 <script src="lyrics-core.js"></script>
 <script>
 (function () {
  'use strict';
@@ -537,9 +602,10 @@
    library: '<path d="M9 18V6l10-2v12"/><circle cx="6.5" cy="18" r="2.5"/><circle cx="16.5" cy="16" r="2.5"/>',
    peers: '<circle cx="6" cy="12" r="2.5"/><circle cx="18" cy="6" r="2.5"/><circle cx="18" cy="18" r="2.5"/><path d="M8.3 10.9l7.4-3.8M8.3 13.1l7.4 3.8"/>',
    history: '<path d="M3 12a9 9 0 109-9 9 9 0 00-6.4 2.6L3 8"/><path d="M3 3v5h5M12 7v5l3 2"/>',
    analytics: '<path d="M4 20h16M7 16V9M12 16V4M17 16v-5"/>',
    access: '<circle cx="8" cy="15" r="4"/><path d="M11 12l9-9M16 7l3 3M14 9l2 2"/>',
  };
-  const TABS = [['editor', 'Editor'], ['reports', 'Reports'], ['library', 'Library'], ['peers', 'P2P'], ['history', 'History'], ['access', 'API']];
+  const TABS = [['editor', 'Editor'], ['reports', 'Reports'], ['library', 'Library'], ['peers', 'P2P'], ['history', 'History'], ['analytics', 'Stats'], ['access', 'API']];
  let tab = 'editor';
  function buildTabs() {
    const html = TABS.map(([k, l]) => `<button class="tab" role="tab" data-tab="${k}" aria-selected="false"><svg viewBox="0 0 24 24">${ICONS[k]}</svg><span>${l}</span>${k === 'reports' ? '<span class="badge hidden" data-badge>0</span>' : ''}</button>`).join('');
@@ -558,6 +624,7 @@
    if (name === 'peers') loadP2p();
    if (name === 'library') loadUploads();
    if (name === 'access') loadTokens();
    if (name === 'analytics') loadAnalytics();
    window.scrollTo({ top: 0 });
  }
  function setBadge(n) {
@@ -596,6 +663,67 @@
  });
  $('logoutBtn').addEventListener('click', async () => { await api('/api/admin/logout', { method: 'POST' }); showLogin(); });
  /* ---------------- server analytics / metadata collection ---------------- */
  let analyticsTimer = null, metadataOffset = 0;
  const bytes = value => { let size = Number(value) || 0, unit = 0; const units = ['B','KB','MB','GB','TB']; while (size >= 1024 && unit < 4) { size /= 1024; unit++; } return `${size.toFixed(unit ? 1 : 0)} ${units[unit]}`; };
  async function loadAnalytics() {
    $('analyticsStatus').textContent = 'Loading server analytics…';
    try {
      const j = await api('/api/admin/analytics');
      if (!j.ok) throw Error(j.error || 'Analytics unavailable.');
      const ready = j.media.find(row => row.status === 'ready') || {}, uploadBytes = j.uploads.reduce((total, row) => total + Number(row.bytes), 0);
      const metrics = [
        ['Stored media', bytes(Number(ready.bytes || 0) + uploadBytes), `${ready.count || 0} cached videos · ${j.uploads.reduce((sum, row) => sum + Number(row.count), 0)} uploads`],
        ['Metadata videos', Number(j.catalog.videos).toLocaleString(), `${j.details.videos} with detailed extractor metadata`],
        ['Saved thumbnails', Number(j.thumbnails.saved || 0).toLocaleString(), `${bytes(j.thumbnails.bytes)} · ${j.thumbnails.pending || 0} awaiting download`],
        ['Recorded plays', Number(j.listening.plays).toLocaleString(), `${j.listening.videos} different videos`],
        ['Cached video size', bytes(ready.bytes), `${Math.round(Number(ready.seconds || 0) / 3600)} hours of audio/video`],
        ['Upload size', bytes(uploadBytes), 'Server media library'],
        ['Metadata size', bytes(Number(j.catalog.bytes) + Number(j.details.bytes)), 'JSON payloads; database indexes excluded'],
        ['Cache queue', j.media.filter(row => row.status !== 'ready').reduce((sum, row) => sum + Number(row.count), 0), j.media.filter(row => row.status !== 'ready').map(row => `${row.status}: ${row.count}`).join(' · ') || 'No pending downloads'],
      ];
      $('analyticsMetrics').innerHTML = metrics.map(([label, value, detail]) => `<div class="metric"><span>${esc(label)}</span><b>${esc(value)}</b><small>${esc(detail)}</small></div>`).join('');
      $('analyticsVolumes').innerHTML = j.volumes.map(v => `<p><b>${esc(v.label)}</b> · ${v.unavailable ? 'Storage unavailable' : `${bytes(v.free)} available of ${bytes(v.total)} · device ${esc(v.device)}`}</p>`).join('');
      $('analyticsSources').innerHTML = '<tr><th>Source</th><th>Videos</th><th>Discoveries</th></tr>' + j.sources.map(row => `<tr><td>${esc(row.source)}</td><td>${esc(row.videos)}</td><td>${esc(row.discoveries)}</td></tr>`).join('');
      $('analyticsTop').innerHTML = j.topPlayed.length ? j.topPlayed.map(row => `<p>${esc(row.title)} · <b>${esc(row.plays)} plays</b></p>`).join('') : '<p class="muted">No listening activity recorded yet.</p>';
      $('analyticsStatus').textContent = 'Updated ' + new Date().toLocaleTimeString();
      await Promise.all([loadCollections(), loadMetadata()]);
    } catch (error) { $('analyticsStatus').textContent = error.message || 'Could not load server analytics.'; }
  }
  async function loadCollections() {
    try {
      const j = await api('/api/admin/collections'); if (!j.ok) return;
      $('collectionJobs').innerHTML = j.jobs.length ? j.jobs.map(job => `<div class="collection-job"><b>${esc(job.query)}</b> · ${esc(job.status)}<p>${job.collected}/${job.maxVideos} videos collected · ${job.enriched} enriched · ${job.failed} failed · ${job.searches} searches · depth ${job.depth}</p>${job.current ? `<p>Working on ${esc(job.current)}</p>` : ''}${job.error ? `<p>${esc(job.error)}</p>` : ''}${job.errors?.length ? `<details><summary>Extraction errors</summary>${job.errors.map(e => `<p>${esc(e.id)}: ${esc(e.error)}</p>`).join('')}</details>` : ''}${['queued','running'].includes(job.status) ? `<button class="btn" type="button" data-cancel-collection="${esc(job.id)}">Stop collection</button>` : ''}</div>`).join('') : '<p class="muted">No collections yet.</p>';
      clearTimeout(analyticsTimer); analyticsTimer = null;
      if (tab === 'analytics' && j.jobs.some(job => ['queued','running'].includes(job.status))) analyticsTimer = setTimeout(loadCollections, 4000);
    } catch { $('collectorStatus').textContent = 'Connection interrupted. Collection continues on the server.'; }
  }
  async function loadMetadata() {
    const j = await api(`/api/admin/metadata?q=${encodeURIComponent($('metadataFilter').value)}&offset=${metadataOffset}`); if (!j.ok) return;
    $('metadataCount').textContent = `${j.total} saved videos · ${j.total ? metadataOffset + 1 : 0}–${metadataOffset + j.videos.length}`;
    $('metadataTable').innerHTML = '<tr><th>Video</th><th>Metadata</th><th>Thumbnail</th></tr>' + j.videos.map(v => `<tr><td><button class="btn sm" type="button" data-metadata-id="${esc(v.id)}">${esc(v.title)}</button><br><small>${esc(v.channel)} · ${Number(v.duration) > 0 ? Math.round(Number(v.duration)) + 's' : 'Duration unavailable'}</small></td><td>${v.enriched ? 'Detailed' : 'Search card'}</td><td>${v.thumbnailBytes ? bytes(v.thumbnailBytes) : 'Pending / unavailable'}</td></tr>`).join('');
    $('metadataPrev').disabled = metadataOffset === 0; $('metadataNext').disabled = metadataOffset + 50 >= j.total;
  }
  $('analyticsRefresh').addEventListener('click', loadAnalytics);
  $('collectorForm').addEventListener('submit', async e => {
    e.preventDefault(); $('collectorStart').disabled = true; $('collectorStatus').textContent = 'Queuing collection…';
    try {
      const j = await api('/api/admin/collections', { method: 'POST', body: { query: $('collectorQuery').value, maxVideos: Number($('collectorLimit').value), depth: Number($('collectorDepth').value) } });
      $('collectorStatus').textContent = j.ok ? 'Collection queued. You can leave this page while it runs.' : j.error || 'Could not start collection.';
      if (j.ok) await loadCollections();
    } catch { $('collectorStatus').textContent = 'Could not reach the server.'; }
    finally { $('collectorStart').disabled = false; }
  });
  $('collectionJobs').addEventListener('click', async e => { const btn = e.target.closest('[data-cancel-collection]'); if (!btn) return; await api('/api/admin/collections/' + btn.dataset.cancelCollection + '/cancel', { method: 'POST', body: {} }); await loadCollections(); });
  $('metadataFilterForm').addEventListener('submit', e => { e.preventDefault(); metadataOffset = 0; loadMetadata(); });
  $('metadataPrev').addEventListener('click', () => { metadataOffset = Math.max(0, metadataOffset - 50); loadMetadata(); });
  $('metadataNext').addEventListener('click', () => { metadataOffset += 50; loadMetadata(); });
  $('metadataTable').addEventListener('click', async e => {
    const btn = e.target.closest('[data-metadata-id]'); if (!btn) return;
    const j = await api('/api/admin/metadata/' + btn.dataset.metadataId); if (!j.ok) return;
    $('metadataDetail').textContent = JSON.stringify(j.metadata, null, 2); $('metadataDetails').classList.remove('hidden'); $('metadataDetails').open = true;
  });
  /* ---------------- tokens / inject ---------------- */
  async function loadTokens() {
    const j = await api('/api/admin/tokens');
@@ -776,13 +904,18 @@
  function paintSongs() {
    const q = $('songFilter').value.trim().toLowerCase();
    const rows = SONGS.filter((r) => !q || (r.title + ' ' + (r.sub || '')).toLowerCase().includes(q));
-    $('songList').innerHTML = rows.length ? rows.map((r) => `<button type="button" class="song-row" data-open="${esc(r.id)}">
+    $('songList').innerHTML = rows.length ? rows.map((r) => `<div class="song-row"><button type="button" class="song-open" data-open="${esc(r.id)}">
      <span style="min-width:0;flex:1"><b>${esc(r.title)}</b><small>${esc(r.sub || '')}</small></span>
-      <span class="st ${r.lines > 0 ? 'has' : ''}">${r.lines > 0 ? `${r.lines} lines` : r.lines === 0 ? 'no lyrics' : 'open'}</span></button>`).join('')
+      <span class="st ${r.lines > 0 ? 'has' : ''}">${r.lines > 0 ? `${r.lines} lines` : r.lines === 0 ? 'no lyrics' : 'open'}</span></button>
      <button type="button" class="btn sm whisper-request" data-whisper="${esc(r.id)}" aria-label="Transcribe ${esc(r.title)} with Whisper">Whisper</button></div>`).join('')
      : '<div class="muted" style="padding:10px">Nothing matches — paste a link above to open any YouTube song.</div>';
    paintWhisper();
  }
  $('songFilter').addEventListener('input', paintSongs);
-  $('songList').addEventListener('click', (e) => { const b = e.target.closest('[data-open]'); if (b) leOpen(b.dataset.open); });
+  $('songList').addEventListener('click', (e) => {
    const whisper = e.target.closest('[data-whisper]'); if (whisper) { startWhisper(whisper.dataset.whisper); return; }
    const b = e.target.closest('[data-open]'); if (b) leOpen(b.dataset.open);
  });
  $('recentChips').addEventListener('click', (e) => { const b = e.target.closest('[data-open]'); if (b) leOpen(b.dataset.open); });
  /* ---------------- p2p ---------------- */
@@ -907,6 +1040,67 @@
    $('save').disabled = !on;
    $('discard').disabled = !on;
  }
  const whisperJobs = new Map(), whisperStarting = new Set();
  let whisperTimer = null;
  const activeWhisper = job => job && (job.status === 'queued' || job.status === 'running');
  function paintWhisper() {
    document.querySelectorAll('[data-whisper]').forEach(btn => {
      const job = whisperJobs.get(btn.dataset.whisper);
      btn.disabled = whisperStarting.has(btn.dataset.whisper) || activeWhisper(job);
      btn.textContent = job?.status === 'running' ? 'Transcribing…' : job?.status === 'queued' ? 'Queued' : 'Whisper';
    });
    const job = whisperJobs.get(LE.id);
    $('whisperBtn').disabled = whisperStarting.has(LE.id) || activeWhisper(job);
    $('whisperBtn').textContent = activeWhisper(job) ? (job.status === 'queued' ? 'Whisper queued' : 'Whisper transcribing…') : 'Transcribe with Whisper';
    const labels = { 'loading-model': 'Whisper is loading its model…', 'downloading-audio': 'Whisper is reading the song’s audio…', transcribing: 'Whisper is transcribing the vocals. This can take several minutes.' };
    $('whisperStatus').textContent = job?.status === 'queued' ? 'Queued for Whisper. Waiting for the lyrics worker.'
      : job?.status === 'running' ? labels[job.stage] || 'Whisper is working…'
      : job?.status === 'failed' ? 'Whisper could not finish: ' + (job.error || 'Try again.') : '';
    const ready = job?.status === 'complete' && job.result?.lines?.length;
    $('whisperResult').classList.toggle('hidden', !ready);
    $('whisperPreview').textContent = ready ? job.result.lines.map(l => l.text).join('\n') : '';
  }
  async function refreshWhisper(id) {
    try {
      const result = await api('/api/admin/transcriptions/' + encodeURIComponent(id));
      if (result.ok) whisperJobs.set(id, result.job);
      paintWhisper(); scheduleWhisper();
    } catch { /* a reconnect or another poll can recover the durable job */ }
  }
  function scheduleWhisper() {
    if (whisperTimer || ![...whisperJobs.values()].some(activeWhisper)) return;
    whisperTimer = setTimeout(async () => {
      whisperTimer = null;
      for (const [id, job] of whisperJobs) if (activeWhisper(job)) await refreshWhisper(id);
      scheduleWhisper();
    }, 5000);
  }
  async function startWhisper(id) {
    if (!id || whisperStarting.has(id) || activeWhisper(whisperJobs.get(id))) return;
    whisperStarting.add(id); paintWhisper();
    try {
      if (LE.id !== id || !LE.loaded) { const opened = await leOpen(id); if (opened === false) return; }
      const result = await api('/api/admin/transcriptions/' + encodeURIComponent(id), { method: 'POST', body: {} });
      if (!result.ok) { say(result.error || 'Whisper could not start.', 'err'); return; }
      whisperJobs.set(id, result.job); paintWhisper(); scheduleWhisper();
    } catch { say('Could not reach Whisper. Try again when connected.', 'err'); }
    finally { whisperStarting.delete(id); paintWhisper(); }
  }
  $('whisperBtn').addEventListener('click', () => startWhisper(LE.id));
  $('whisperUse').addEventListener('click', () => {
    const job = whisperJobs.get(LE.id); if (job?.status !== 'complete' || !job.result?.lines?.length) return;
    if (LE.dirty && !confirm('Replace your unsaved editor text with the Whisper draft? Undo can restore it.')) return;
    edit(() => {
      LE.lines = job.result.lines.map(l => ({ ...l })); LE.offset = 0;
      LE.tags = [...LE.tags.filter(t => !/^(from |auto-transcribed)/i.test(t)), ...job.result.tags];
      LE.raw = false; $('raw').classList.add('hidden'); $('lines').classList.remove('hidden');
      $('rawBtn').textContent = '✎ Edit as text'; $('tags').value = LE.tags.join(', ');
    });
    render(); draw(); syncBars();
    $('whisperResult').classList.add('hidden');
    say('Whisper draft loaded. Check the words and timing, then Save to publish; Undo restores the previous text.', 'ok');
  });
  function progress() {
    const timed = LE.lines.filter((l) => l.kind === 'line' && l.t != null).length;
    const all = LE.lines.filter((l) => l.kind === 'line').length;
@@ -919,10 +1113,10 @@
  }
  /* undo / redo — whole-document snapshots, typing is coalesced */
-  const snap = () => JSON.stringify({ l: LE.lines, t: LE.tags });
+  const snap = () => JSON.stringify({ l: LE.lines, t: LE.tags, o: LE.offset });
  function pushUndo() { LE.undo.push(snap()); if (LE.undo.length > 100) LE.undo.shift(); LE.redo = []; undoBtns(); }
  function undoBtns() { $('bUndo').disabled = !LE.undo.length; $('bRedo').disabled = !LE.redo.length; }
-  function restore(s) { const o = JSON.parse(s); LE.lines = o.l; LE.tags = o.t; $('tags').value = LE.tags.join(', '); render(); setDirty(true); undoBtns(); }
+  function restore(s) { const o = JSON.parse(s); LE.lines = o.l; LE.tags = o.t; LE.offset = o.o ?? LE.offset; $('tags').value = LE.tags.join(', '); render(); setDirty(true); undoBtns(); }
  function doUndo() { if (!LE.undo.length) return; LE.redo.push(snap()); restore(LE.undo.pop()); say('Undone'); }
  function doRedo() { if (!LE.redo.length) return; LE.undo.push(snap()); restore(LE.redo.pop()); say('Redone'); }
  let typing = null;
@@ -957,6 +1151,7 @@
    $('raw').classList.add('hidden'); $('lines').classList.remove('hidden'); $('rawBtn').textContent = '✎ Edit as text';
    await loadFlags();
    render(); setDirty(false); undoBtns(); syncBars();
    paintWhisper(); refreshWhisper(id);
    say('');
    if (!keepAudio) initAudio(id, streams);
    draw();
@@ -993,7 +1188,7 @@
    row.innerHTML = `
      <div class="ln-main">
        <button class="tc" data-act="chip" type="button"></button>
-        <input class="ln-text" value="${esc(line.text)}" placeholder="Type the line…" aria-label="Line text" />
+        <textarea class="ln-text" rows="1" placeholder="Type the line… (Shift+Enter for a line break)" aria-label="Line text">${esc(line.text)}</textarea>
        <span class="flagchip hidden" data-flag><svg viewBox="0 0 24 24"><path d="M5 21V4M5 4h11l-2 4 2 4H5"/></svg><b></b></span>
        <button class="more" data-act="more" type="button" aria-label="Line options">⋯</button>
      </div>
@@ -1005,9 +1200,12 @@
        <button class="btn" data-act="nudge" data-d="-0.2" type="button">−0.2</button>
        <button class="btn" data-act="nudge" data-d="0.2" type="button">+0.2</button>
        <select class="mini" data-kind aria-label="Line type"><option value="line">Line</option><option value="section">Section</option><option value="cue">Cue</option></select>
        <button class="btn" data-act="break" type="button" title="Same timestamp, both lines highlight together">↵ Line break</button>
        <button class="btn" data-act="add" type="button">＋ Below</button>
        <button class="btn dng" data-act="del" type="button">Delete</button>
      </div>`;
    const text = row.querySelector('.ln-text');
    text.rows = Math.max(1, line.text.split('\n').length);
    row.querySelector('[data-kind]').value = line.kind;
    return row;
  }
@@ -1036,6 +1234,9 @@
    LE.rows = LE.lines.map((l, i) => { const r = rowEl(l, i); box.appendChild(r); return r; });
    if (!LE.lines.length) box.innerHTML = '<div class="empty">No lines yet — press “＋ Add line”, paste text with ✎ Edit as text, or pull them from LRCLIB (⋯ tools).</div>';
    LE.lines.forEach((_, i) => paintRow(i));
    const fields = LE.rows.map(row => row.querySelector('.ln-text'));
    const heights = fields.map(field => field.scrollHeight);
    fields.forEach((field, i) => { field.style.height = heights[i] + 'px'; });
    paintFlags(); progress(); lastIdx = -2; tick(); tapLabel();
    $('songMeta').textContent = $('songMeta').textContent.replace(/\d+ lines/, `${LE.lines.length} lines`);
  }
@@ -1109,6 +1310,13 @@
    else if (act === 'set') stamp(i);
    else if (act === 'seek') { seek((LE.lines[i].t || 0) + LE.offset); if (LE.audio && LE.audio.paused) LE.audio.play().catch(() => {}); }
    else if (act === 'nudge') { if (LE.lines[i].t == null) return; edit(() => { LE.lines[i].t = r2(Math.max(0, LE.lines[i].t + Number(btn.dataset.d))); }); paintRow(i); draw(); }
    else if (act === 'break') {
      const text = LE.rows[i].querySelector('.ln-text');
      const pos = text.selectionStart;
      edit(() => { LE.lines[i].text = text.value.slice(0, pos) + '\n' + text.value.slice(text.selectionEnd); });
      render(); select(i);
      const target = LE.rows[i].querySelector('.ln-text'); target.focus(); target.setSelectionRange(pos + 1, pos + 1);
    }
    else if (act === 'add') { edit(() => LE.lines.splice(i + 1, 0, { t: null, text: '', kind: 'line' })); render(); select(i + 1); LE.rows[i + 1].querySelector('.ln-text').focus(); }
    else if (act === 'del') { edit(() => LE.lines.splice(i, 1)); LE.sel = -1; render(); draw(); }
    else if (act === 'resolve') {
@@ -1118,7 +1326,7 @@
  });
  $('lines').addEventListener('input', (e) => {
    const row = e.target.closest('.ln'); if (!row) return; const i = Number(row.dataset.i);
-    if (e.target.classList.contains('ln-text')) { typed(); LE.lines[i].text = e.target.value; setDirty(true); tapLabel(); }
+    if (e.target.classList.contains('ln-text')) { typed(); LE.lines[i].text = e.target.value; e.target.rows = Math.max(1, e.target.value.split('\n').length); e.target.style.height = 'auto'; e.target.style.height = e.target.scrollHeight + 'px'; setDirty(true); tapLabel(); }
  });
  $('lines').addEventListener('change', (e) => {
    const row = e.target.closest('.ln'); if (!row) return; const i = Number(row.dataset.i);
@@ -1128,6 +1336,7 @@
  $('lines').addEventListener('focusin', (e) => { const row = e.target.closest('.ln'); if (row && Number(row.dataset.i) !== LE.sel) select(Number(row.dataset.i)); });
  $('lines').addEventListener('keydown', (e) => {
    if (e.key !== 'Enter' || !e.target.classList.contains('ln-text')) return;
    if (e.shiftKey) return; // Native textarea break stays in the current timed cue.
    e.preventDefault();
    const i = Number(e.target.closest('.ln').dataset.i);
    edit(() => LE.lines.splice(i + 1, 0, { t: null, text: '', kind: LE.lines[i].kind === 'section' ? 'line' : LE.lines[i].kind }));
@@ -1199,17 +1408,10 @@
    $('raw').classList.toggle('hidden', !LE.raw); $('lines').classList.toggle('hidden', LE.raw);
    $('rawBtn').textContent = LE.raw ? '≡ Back to lines' : '✎ Edit as text';
    if (LE.raw) {
-      $('raw').value = LE.lines.map((l) => (l.t == null ? '' : `[${fmtT(l.t)}] `) + (l.kind === 'section' ? '# ' : l.kind === 'cue' ? '! ' : '') + l.text).join('\n');
+      $('raw').value = LyricsCore.toText({ lines: LE.lines });
    } else {
      pushUndo();
-      LE.lines = $('raw').value.split('\n').map((raw) => {
+      LE.lines = LyricsCore.fromText($('raw').value).lines;
        let rest = raw.trim(), t = null;
        const m = rest.match(/^\[(\d{1,3}):(\d{1,2}(?:[.:]\d{1,3})?)\]\s*/);
        if (m) { t = r2(Number(m[1]) * 60 + Number(m[2].replace(':', '.'))); rest = rest.slice(m[0].length); }
        let kind = 'line';
        if (rest.startsWith('# ')) { kind = 'section'; rest = rest.slice(2); } else if (rest.startsWith('! ')) { kind = 'cue'; rest = rest.slice(2); }
        return { t, text: rest, kind };
      }).filter((l) => l.text);
      setDirty(true); render(); draw();
    }
    syncBars();
@@ -1242,12 +1444,12 @@
    leOpen(id);
  });
  function leOpen(id, focusText) {
-    if (LE.dirty && id !== LE.id && !confirm('Discard the unsaved changes to the song you have open?')) return;
+    if (LE.dirty && id !== LE.id && !confirm('Discard the unsaved changes to the song you have open?')) return false;
    showTab('editor');
    if (id === LE.id && LE.loaded && focusText) {
      const i = LE.lines.findIndex((l) => l.text === focusText); if (i >= 0) select(i, true, true); return;
    }
-    leLoad(id, false, focusText);
+    return leLoad(id, false, focusText);
  }
  function renderDocs() {
--- a/frontend/app.js
+++ b/frontend/app.js
@@ -3247,7 +3247,12 @@ function fitLyricLines(list, { maxShare = 0.17, maxPx = 160, minPx = 18, uniform
  // re-laid-out every line (O(n²) — seconds on a phone for a long sermon).
  const fits = [];
  for (const el of rows) {
-    widthAt.text = el.textContent;
+    // Size a grouped cue by its widest visual line, preserving tight breaks.
    const parts = el.textContent.split('\n');
    widthAt.text = parts.reduce((widest, part) => {
      widthAt.text = part; const width = widthAt(100);
      return width > widest.width ? { text: part, width } : widest;
    }, { text: '', width: -1 }).text;
    const w100 = widthAt(100);
    let px = cap;
    if (w100) {
@@ -3650,16 +3655,19 @@ const Notes = (() => {
    body.innerHTML = `
      <p class="lyr-editor-help">One lyric line per line. Optional: <code>[1:23.45]</code> times a line
        (or time them later with Tap-sync), <code># Chorus</code> is a section, <code>! Drums in</code> a band cue,
-        <code>@ Key G, Capo 2, 70 BPM</code> adds tags. Pasted .lrc files work as-is.</p>
+        <code>@ Key G, Capo 2, 70 BPM</code> adds tags. Use Shift+Enter or Line break to group words under the same timestamp (<code>| continuation</code> in text). Pasted .lrc files work as-is.</p>
      ${note ? `<p class="lyr-editor-note">${escapeHtml(note)}</p>` : ''}
      <textarea class="lyr-editor-text" spellcheck="false" placeholder="Paste or type the lyrics…"></textarea>
      <div class="lyr-editor-tools">
        <button type="button" class="np-btn" data-act="captions">⤓ Import YouTube captions</button>
        <button type="button" class="np-btn" data-act="break">↵ Line break (same time)</button>
        <button type="button" class="np-btn" data-act="stamp">⏱ Stamp current time on this line</button>
      </div>`;
    const ta = body.querySelector('textarea');
    ta.value = prefill != null ? prefill : (st.lyrics ? LC.toText(st.lyrics.data) : '');
-    ta.addEventListener('keydown', (e) => e.stopPropagation());
+    const insertBreak = () => { ta.setRangeText('\n| ', ta.selectionStart, ta.selectionEnd, 'end'); ta.focus(); };
    body.querySelector('[data-act="break"]').addEventListener('click', insertBreak);
    ta.addEventListener('keydown', (e) => { e.stopPropagation(); if (e.key === 'Enter' && e.shiftKey) { e.preventDefault(); insertBreak(); } });
    const runCaptions = async () => {
      const text = await captionsText();
      if (text != null) ta.value = text;
@@ -3667,7 +3675,8 @@ const Notes = (() => {
    body.querySelector('[data-act="captions"]').addEventListener('click', runCaptions);
    body.querySelector('[data-act="stamp"]').addEventListener('click', () => {
      const pos = ta.selectionStart;
-      const start = ta.value.lastIndexOf('\n', pos - 1) + 1;
+      let start = ta.value.lastIndexOf('\n', pos - 1) + 1;
      while (start > 0 && /^\s*\| /.test(ta.value.slice(start).split('\n')[0])) start = ta.value.lastIndexOf('\n', start - 2) + 1;
      const endNl = ta.value.indexOf('\n', start);
      const end = endNl < 0 ? ta.value.length : endNl;
      const lineText = ta.value.slice(start, end).replace(/^(\[[^\]]*\]\s*)+/, '');
@@ -3675,7 +3684,11 @@ const Notes = (() => {
      ta.value = ta.value.slice(0, start) + stamp + lineText + ta.value.slice(end);
      // Move to the next line so repeated taps walk down the sheet.
      const next = ta.value.indexOf('\n', start);
-      const caret = next < 0 ? ta.value.length : next + 1;
+      let caret = next < 0 ? ta.value.length : next + 1;
      while (/^\s*\| /.test(ta.value.slice(caret).split('\n')[0])) {
        const following = ta.value.indexOf('\n', caret);
        caret = following < 0 ? ta.value.length : following + 1;
      }
      ta.focus();
      ta.setSelectionRange(caret, caret);
    });
--- a/frontend/lyrics-core.js
+++ b/frontend/lyrics-core.js
@@ -11,6 +11,7 @@
 *   @ Key G, Capo 2 (Play F), 70 BPM                    tags
 *   [00:12.00][01:30.00] repeated line                  LRC multi-stamp
 *   [offset:+500]                                       LRC offset (ms, + = sooner)
 *   | You can do anything                              continuation, same cue
 * Other LRC headers ([ar:], [ti:], …) are ignored.
 *
 * Sync offset: a line is shown at (t + offset) seconds of playback, so a
@@ -60,9 +61,15 @@
    const lines = [];
    const tags = [];
    let offset = null;
    let group = [];
    for (const rawLine of String(text || '').replace(/\r/g, '').split('\n')) {
      let rest = rawLine.trim();
      if (!rest) continue;
      if (rest.startsWith('| ') && group.length) {
        for (const line of group) line.text += '\n' + rest.slice(2).trim();
        continue;
      }
      group = [];
      const header = rest.match(HEADER_RE);
      if (header && !/^\d/.test(header[1])) {
        if (header[1].toLowerCase() === 'offset') {
@@ -83,8 +90,9 @@
      if (rest.startsWith('#')) { kind = 'section'; rest = rest.replace(/^#+/, '').trim(); }
      else if (rest.startsWith('!')) { kind = 'cue'; rest = rest.slice(1).trim(); }
      if (!rest) continue;
-      if (!stamps.length) lines.push({ t: null, text: rest, kind });
+      group = stamps.length ? stamps.map(t => ({ t: Math.round(t * 100) / 100, text: rest, kind }))
-      for (const t of stamps) lines.push({ t: Math.round(t * 100) / 100, text: rest, kind });
+        : [{ t: null, text: rest, kind }];
      lines.push(...group);
    }
    // A fully timed sheet (typical .lrc with repeated-chorus stamps) is put in
    // time order; a partly timed one keeps the order it was written in.
@@ -100,7 +108,8 @@
    for (const l of (doc && doc.lines) || []) {
      const stamp = l.t === null || l.t === undefined ? '' : `[${fmtStamp(l.t)}] `;
      const mark = l.kind === 'section' ? '# ' : l.kind === 'cue' ? '! ' : '';
-      out.push(stamp + mark + l.text);
+      const parts = String(l.text || '').split('\n');
      out.push(stamp + mark + parts[0], ...parts.slice(1).map(part => '| ' + part));
    }
    return out.join('\n');
  }
--- a/frontend/lyrics-core.test.js
+++ b/frontend/lyrics-core.test.js
@@ -78,3 +78,16 @@ test('YouTube export adds a 0:00 entry only when needed', () => {
  assert.strictEqual(LC.activeMarker(items, 60).title, 'Intro');
  assert.strictEqual(LC.activeMarker(items, 10), null);
 });
 test('multiline cues round-trip without adding timestamps or active rows', () => {
  const doc = { lines: [{ t: 12, text: 'Because You are God\nYou can do anything', kind: 'line' }, { t: 18, text: 'Next lyric', kind: 'line' }], tags: [] };
  const copy = LC.fromText(LC.toText(doc));
  assert.deepStrictEqual(copy.lines, doc.lines);
  assert.strictEqual(LC.activeIndex(copy.lines, 15, 0), 0);
  assert.strictEqual(LC.activeIndex(copy.lines, 18, 0), 1);
  assert.strictEqual(LC.timedCount(copy.lines), 2);
  const repeated = LC.fromText('[0:12][0:30] Because You are God\n| You can do anything');
  assert.strictEqual(repeated.lines.length, 2);
  assert.strictEqual(repeated.lines[1].text, doc.lines[0].text);
 });
--- a/frontend/styles.css
+++ b/frontend/styles.css
@@ -3522,7 +3522,7 @@ html[data-layout="bento-hub"] .mini-bar-inner {
 .lyr-line:hover { background: var(--bg-3); }
 .lyr-line.untimed { cursor: default; }
 .lyr-time { flex: none; width: 38px; font-family: var(--mono); font-size: 10px; color: var(--text-dim); }
-.lyr-text { font-size: 15px; line-height: 1.45; }
+.lyr-text { white-space: pre-line; font-size: 15px; line-height: 1.45; }
 .lyr-line.active { background: var(--bg-3); color: var(--text); }
 .lyr-line.active .lyr-text { font-weight: 700; font-size: 17px; }
 .lyr-line.active .lyr-time { color: var(--accent-bright); }
@@ -3779,6 +3779,8 @@ body.remote-open { overflow: hidden; }
  font-weight: 800;
  line-height: 1.15;
  letter-spacing: -0.01em;
  white-space: pre-line;
  flex-shrink: 0;
  /* --fit = the size at which THIS line fills one row; others are fractions. */
  font-size: calc(var(--fit, 40px) * 0.48);
  color: var(--text-2);
--- a/package.json
+++ b/package.json
@@ -11,7 +11,7 @@
    "tauri": "tauri",
    "tauri:dev": "tauri dev",
    "tauri:build": "tauri build",
-    "test": "node --test frontend/"
+    "test": "node --test frontend/*.test.js"
  },
  "author": "",
  "license": "MIT",
--- a/playwright.admin.config.js
+++ b/playwright.admin.config.js
@@ -0,0 +1,3 @@
 const { defineConfig } = require('@playwright/test');
 const classic = require('./playwright.classic.config');
 module.exports = defineConfig({ ...classic, testMatch: /admin-lyrics\.spec\.js/ });
--- a/scripts/lyrics/auto_lyrics.py
+++ b/scripts/lyrics/auto_lyrics.py
@@ -29,6 +29,7 @@ import urllib.error
 import urllib.request
 import glob
 import http.cookiejar
 import threading
 FILLER = re.compile(r"^(?:(?:oh|ooh|ohh|oh-oh|ah|ahh|hey|yeah|mm|mm-mm|mm-mm-mm|hmm|whoa|woah|la|na|uh|come on)[\s,.!?-]*)+$", re.I)
 KEEP_CAP = {'I', "I'm", "I'll", "I've", "I'd", 'You', 'Your', "You're", 'Yours', 'Lord', 'God', 'Jesus', 'Christ',
@@ -101,7 +102,7 @@ class Api:
                                     data=json.dumps(body).encode() if body is not None else None)
        try:
            with self.opener.open(req, timeout=600) as r:
-                data = r.read()
+                data = r.read(MAX_AUDIO_BYTES + 1) if raw else r.read()
                return r.status, data if raw else json.loads(data or b'{}')
        except urllib.error.HTTPError as e:
            data = e.read()
@@ -193,9 +194,29 @@ def watch(args):
                          'error': 'worker died while transcribing (likely out of memory)'}
            print(f'lyrics worker: {vid} crashed the previous run — backing off', flush=True)
    save_state(args.state, state)
    next_auto = 0
    def get_model():
        nonlocal model
        if model is None:
            from faster_whisper import WhisperModel
            print(f'lyrics worker: loading {args.model}', flush=True)
            model = WhisperModel(args.model, device='cpu', compute_type='int8', cpu_threads=args.threads)
        return model
    while True:
        try:
            api = Api(args.base, token, password)
            # Explicit admin requests take priority and always use Whisper.
            # Poll every ten seconds, independently of the automatic interval.
            if token:
                st, request = api.call('POST', '/api/lyrics-worker/claim', {})
                if st == 200 and request.get('job'):
                    process_requested(args, api, request['job'], get_model)
                    continue
            if time.time() < next_auto:
                time.sleep(min(args.watch, 10))
                continue
            st, r = api.call('GET', '/api/admin/media')
            if st != 200:
                raise RuntimeError(f'listing failed ({st}): {r.get("error")}')
@@ -205,10 +226,7 @@ def watch(args):
                    and state.get(m['id'], {}).get('status') != 'instrumental'
                    and state.get(m['id'], {}).get('retry_at', 0) <= now]
            if todo:
-                if model is None:
+                get_model()
                    from faster_whisper import WhisperModel
                    print(f'lyrics worker: loading {args.model}', flush=True)
                    model = WhisperModel(args.model, device='cpu', compute_type='int8', cpu_threads=args.threads)
                m = todo[0]  # one song per cycle keeps the worker's footprint small
                entry = state.get(m['id'], {})
                state[m['id']] = {**entry, 'status': 'in_progress'}
@@ -224,9 +242,58 @@ def watch(args):
                save_state(args.state, state)
                print(f'lyrics worker: {m["id"]} ({m.get("title", "")[:60]}): {result}', flush=True)
                continue  # straight on to the next song
            next_auto = time.time() + args.watch
        except Exception as e:  # never die: the next cycle retries
            print(f'lyrics worker: {e}', flush=True)
-        time.sleep(args.watch)
+            next_auto = time.time() + args.watch
        time.sleep(min(args.watch, 10))
 def process_requested(args, api, job, get_model):
    """Lease a manual request, heartbeat through model loading, return a draft."""
    stopped = threading.Event()
    stage = ['loading-model']
    path = f'/api/lyrics-worker/jobs/{job["id"]}'
    def report(extra=None):
        return api.call('POST', path, {'lease': job['lease'], 'stage': stage[0], **(extra or {})})
    def heartbeat():
        while not stopped.wait(25):
            try:
                st, _ = report()
                if st == 409:
                    stopped.set()
            except Exception:
                pass  # transient connection failure; the durable lease handles recovery
    thread = threading.Thread(target=heartbeat, daemon=True)
    thread.start()
    try:
        model = get_model()
        def progress(value):
            stage[0] = value
            st, _ = report()
            if st == 409:
                raise RuntimeError('The transcription lease expired.')
        result = transcribe_one(args, api, model, job['videoId'], draft_job=job, on_stage=progress)
        stopped.set()
        thread.join(timeout=2)
        payload = {'status': 'complete', 'result': result} if isinstance(result, dict) else {'status': 'failed', 'error': result}
        st, response = report(payload)
        if st == 400 and payload['status'] == 'complete':
            report({'status': 'failed', 'error': response.get('error', 'The transcript was rejected.')[:300]})
        if st != 200:
            print(f'lyrics worker: request {job["id"]} could not finish ({st}): {response.get("error", "failed")}', flush=True)
    except Exception as e:
        stopped.set()
        try:
            report({'status': 'failed', 'error': str(e)[:300]})
        except Exception:
            pass  # the next worker can reclaim an expired request
    finally:
        stopped.set()
        thread.join(timeout=2)
 def run_once(args, api):
@@ -247,20 +314,22 @@ def run_once(args, api):
    print('\n'.join(f'{v}  {s}' for v, s in summary))
-def transcribe_one(args, api, model, vid):
+def transcribe_one(args, api, model, vid, draft_job=None, on_stage=None):
    """Give one saved song lyrics. Published (often synced) lyrics from LRCLIB
    beat a machine transcript, so that is tried first; transcription is the
    fallback. Returns a one-line result."""
    st, cur = api.call('GET', f'/api/notes/{vid}')
    live = (cur or {}).get('lyrics') if st == 200 else None
-    if live and live['data']['lines'] and not args.overwrite:
+    if not draft_job and live and live['data']['lines'] and not args.overwrite:
        return 'skip: has lyrics'
-    if not getattr(args, 'no_web', False):
+    if not draft_job and not getattr(args, 'no_web', False):
        st, r = api.call('POST', f'/api/notes/{vid}/lyrics/web', {'overwrite': bool(args.overwrite)})
        if st == 200:
            m = r.get('match') or {}
            return f"saved rev {r.get('rev')} — LRCLIB {'synced' if r.get('synced') else 'plain'}: {m.get('artist', '')} – {m.get('track', '')}"
-    st, audio = api.call('GET', f'/api/media/{vid}?a=1', raw=True)
+    if on_stage:
        on_stage('downloading-audio')
    st, audio = api.call('GET', draft_job['audioPath'] if draft_job else f'/api/media/{vid}?a=1', raw=True)
    if st != 200:
        return f'no cached audio ({st})'
    if len(audio) > MAX_AUDIO_BYTES:
@@ -269,6 +338,8 @@ def transcribe_one(args, api, model, vid):
        f.write(audio)
        f.flush()
        t0 = time.time()
        if on_stage:
            on_stage('transcribing')
        # vad_filter must stay OFF: it classifies sung music as non-speech
        # and silently drops the whole song.
        segs, info = model.transcribe(f.name, language=args.language, word_timestamps=True, vad_filter=False,
@@ -282,6 +353,8 @@ def transcribe_one(args, api, model, vid):
    doc = {'lines': lines, 'tags': ['auto-transcribed (whisper)'], 'offset': 0}
    head = ' / '.join(l['text'] for l in lines[:3])
    print(f'{vid}: {len(lines)} lines, lang={info.language}, {took:.0f}s for {info.duration:.0f}s audio | {head[:100]}', flush=True)
    if draft_job:
        return doc
    if args.dry_run:
        print(json.dumps(doc, ensure_ascii=False)[:2000])
        return f'dry-run {len(lines)} lines'
--- a/scripts/lyrics/test_auto_lyrics.py
+++ b/scripts/lyrics/test_auto_lyrics.py
@@ -0,0 +1,66 @@
 import importlib.util
 import pathlib
 import types
 import unittest
 spec = importlib.util.spec_from_file_location('auto_lyrics', pathlib.Path(__file__).with_name('auto_lyrics.py'))
 worker = importlib.util.module_from_spec(spec)
 spec.loader.exec_module(worker)
 class FakeApi:
    def __init__(self):
        self.calls = []
    def call(self, method, path, body=None, raw=False):
        self.calls.append((method, path, body))
        if raw:
            return 200, b'fixture audio'
        if path.startswith('/api/notes/'):
            return 200, {'lyrics': {'rev': 5, 'data': {'lines': [{'t': 0, 'text': 'Keep the original'}]}}}
        return 200, {'ok': True}
 class FakeModel:
    def transcribe(self, path, **options):
        assert options['vad_filter'] is False
        words = [types.SimpleNamespace(word=word, start=i, end=i + .8) for i, word in enumerate('Because You are God You can do anything'.split())]
        return [types.SimpleNamespace(words=words)], types.SimpleNamespace(language='en', duration=10)
 class WhisperRequests(unittest.TestCase):
    def setUp(self):
        self.args = types.SimpleNamespace(overwrite=False, no_web=False, language=None, min_words=1, dry_run=False)
        self.api = FakeApi()
        self.job = {'id': 'fixture-job', 'videoId': '0gfX0dFLaBc', 'lease': 'fixture-lease', 'audioPath': '/api/media/0gfX0dFLaBc?a=1'}
    def test_manual_request_uses_whisper_despite_existing_lyrics_and_never_publishes(self):
        result = worker.transcribe_one(self.args, self.api, FakeModel(), self.job['videoId'], draft_job=self.job)
        self.assertIsInstance(result, dict)
        self.assertTrue(result['lines'])
        self.assertFalse(any(method == 'PUT' or path.endswith('/lyrics/web') for method, path, body in self.api.calls))
    def test_requested_job_reports_stages_and_completes_a_draft(self):
        worker.process_requested(self.args, self.api, self.job, lambda: FakeModel())
        posts = [body for method, path, body in self.api.calls if path.startswith('/api/lyrics-worker/jobs/')]
        self.assertEqual(posts[0]['stage'], 'downloading-audio')
        self.assertEqual(posts[1]['stage'], 'transcribing')
        self.assertEqual(posts[-1]['status'], 'complete')
        self.assertEqual(posts[-1]['lease'], 'fixture-lease')
    def test_rejected_draft_is_reported_failed_instead_of_retrying_forever(self):
        original = self.api.call
        def reject(method, path, body=None, raw=False):
            if body and body.get('status') == 'complete':
                return 400, {'error': 'too many lines'}
            return original(method, path, body, raw)
        self.api.call = reject
        worker.process_requested(self.args, self.api, self.job, lambda: FakeModel())
        self.assertEqual(self.api.calls[-1][2]['status'], 'failed')
        self.assertEqual(self.api.calls[-1][2]['error'], 'too many lines')
    def test_no_vocals_fails_without_touching_saved_lyrics(self):
        self.args.min_words = 25
        worker.process_requested(self.args, self.api, self.job, lambda: FakeModel())
        last = self.api.calls[-1][2]
        self.assertEqual(last['status'], 'failed')
        self.assertFalse(any(method == 'PUT' for method, path, body in self.api.calls))
 if __name__ == '__main__':
    unittest.main()
--- a/server/admin-analytics.js
+++ b/server/admin-analytics.js
@@ -0,0 +1,149 @@
 import { randomUUID } from 'node:crypto';
 import { statfs, stat } from 'node:fs/promises';
 import { db } from './db.js';
 import { ingest } from './video-catalog.js';
 const LEASE = 120000;
 const n = v => Number(v) || 0;
 const clean = value => String(value || '').trim().slice(0, 200);
 const parse = value => { try { return JSON.parse(value) || {}; } catch { return {}; } };
 function publicJob(row) {
  const state = parse(row.state);
  return { id: row.id, query: row.query, maxVideos: n(row.max_videos), depth: n(row.depth), status: row.status,
    collected: state.ids?.length || 0, enriched: state.enriched || 0, failed: state.failed || 0,
    searches: state.searches || 0, current: state.current || '', error: row.error, errors: state.errors || [],
    createdAt: n(row.created_at), updatedAt: n(row.updated_at) };
 }
 export function descriptiveMetadata(raw) {
  const out = {};
  for (const key of ['id','title','fulltitle','description','channel','channel_id','channel_url','uploader','uploader_id','uploader_url','upload_date','release_date','timestamp','duration','view_count','like_count','comment_count','tags','categories','language','live_status','availability','age_limit','license','chapters','thumbnails','thumbnail','webpage_url','artist','artists','album','track','release_year']) {
    if (raw[key] != null) out[key] = raw[key];
  }
  // Retain format specs, excluding expiring media URLs and request headers.
  if (Array.isArray(raw.formats)) out.formats = raw.formats.map(f => Object.fromEntries(['format_id','format_note','ext','width','height','fps','vcodec','acodec','abr','tbr','filesize','filesize_approx','audio_channels','asr'].filter(k => f[k] != null).map(k => [k, f[k]])));
  if (JSON.stringify(out).length > 500000) throw Error('Extracted metadata exceeds 500 KB.');
  return out;
 }
 function card(raw) {
  return { ...raw, channel: raw.channel || raw.uploader || '', channelId: raw.channel_id,
    channelUrl: raw.channel_url, thumbnail: raw.thumbnail || raw.thumbnails?.at(-1)?.url };
 }
 export async function storageAnalytics(paths = []) {
  const queries = [
    "SELECT status,COUNT(*) AS count,COALESCE(SUM(size),0) AS bytes,COALESCE(SUM(duration),0) AS seconds FROM media_cache GROUP BY status",
    "SELECT kind,COUNT(*) AS count,COALESCE(SUM(size),0) AS bytes FROM uploads GROUP BY kind",
  ];
  // Keep each aggregate on its own table, so thumbnail bytes and plays never multiply.
  const [media, uploads, catalog, thumbs, details, sources, plays, top] = await Promise.all([
    db.execute(queries[0]), db.execute(queries[1]),
    db.execute('SELECT COUNT(*) AS videos,COALESCE(SUM(length(CAST(card AS BLOB))),0) AS bytes,MAX(updated_at) AS lastSeen FROM video_meta'),
    db.execute('SELECT COUNT(*) AS total,SUM(CASE WHEN data IS NOT NULL THEN 1 ELSE 0 END) AS saved,COALESCE(SUM(size),0) AS bytes,SUM(CASE WHEN data IS NULL AND retry_at<9007199254740991 THEN 1 ELSE 0 END) AS pending FROM video_thumbnails'),
    db.execute('SELECT COUNT(*) AS videos,COALESCE(SUM(length(CAST(metadata AS BLOB))),0) AS bytes FROM video_details'),
    db.execute('SELECT source,COUNT(*) AS videos,SUM(discoveries) AS discoveries,MAX(last_seen) AS lastSeen FROM video_meta_sources GROUP BY source ORDER BY videos DESC'),
    db.execute('SELECT COALESCE(SUM(plays),0) AS plays,COUNT(DISTINCT video_id) AS videos FROM listening_daily'),
    db.execute('SELECT l.video_id,SUM(l.plays) AS plays,m.card FROM listening_daily l LEFT JOIN video_meta m ON m.id=l.video_id GROUP BY l.video_id ORDER BY plays DESC LIMIT 10'),
  ]);
  const volumes = await Promise.all(paths.map(async ({ label, path }) => {
    try { const [fs, file] = await Promise.all([statfs(path), stat(path)]); return { label, device: String(file.dev), total: n(fs.blocks) * n(fs.bsize), free: n(fs.bavail) * n(fs.bsize) }; }
    catch { return { label, unavailable: true }; }
  }));
  return { media: media.rows, uploads: uploads.rows, catalog: catalog.rows[0], thumbnails: thumbs.rows[0], details: details.rows[0], sources: sources.rows, listening: plays.rows[0], topPlayed: top.rows.map(r => ({ id: r.video_id, plays: n(r.plays), title: parse(r.card).title || r.video_id })), volumes };
 }
 export function registerAnalyticsRoutes(app, { adminAuth, runYtdlp, paths = [], autoStart = true }) {
  let running = null;
  const owner = randomUUID();
  async function runNext() {
    if (running) return running;
    running = execute().finally(() => { running = null; });
    return running;
  }
  async function execute() {
    const time = Date.now();
    const job = (await db.execute({ sql: `UPDATE metadata_collections SET status='running',owner=?,lease_until=?,updated_at=?
      WHERE id=(SELECT id FROM metadata_collections WHERE status='queued' OR (status='running' AND lease_until<?) ORDER BY created_at LIMIT 1) RETURNING *`, args: [owner, time + LEASE, time, time] })).rows[0];
    if (!job) return false;
    const state = { ids: [], queries: [{ q: job.query, level: 0 }], pending: [], enriched: 0, failed: 0, searches: 0, errors: [], ...parse(job.state) };
    const seen = new Set(state.ids);
    async function checkpoint(status = 'running', error = null) {
      const result = await db.execute({ sql: "UPDATE metadata_collections SET state=?,status=?,error=?,lease_until=?,updated_at=? WHERE id=? AND owner=? AND status='running'", args: [JSON.stringify(state), status, error, Date.now() + LEASE, Date.now(), job.id, owner] });
      if (!result.rowsAffected) { const error = Error('Collection cancelled or reassigned.'); error.name = 'CollectionStopped'; throw error; }
    }
    try {
      while (state.pending.length || (state.queries.length && seen.size < job.max_videos && state.searches < 24)) {
        if (!state.pending.length) {
          const { q, level } = state.queries[0]; state.current = q; await checkpoint();
          const remaining = job.max_videos - seen.size;
          const count = Math.max(1, Math.ceil(remaining / (job.depth - level + 1)));
          const output = await runYtdlp([`ytsearch${count}:${q}`, '--dump-json', '--flat-playlist', '--skip-download', '--no-warnings', '--ignore-errors'], { signal: AbortSignal.timeout(60000) });
          await checkpoint(); // Recheck cancellation/ownership before storing discoveries.
          const unique = new Map();
          for (const raw of output.split('\n').filter(Boolean).map(parse)) {
            if (/^[\w-]{11}$/.test(raw.id) && !seen.has(raw.id)) unique.set(raw.id, raw);
          }
          const results = [...unique.values()].slice(0, remaining);
          await ingest(results.map(card), 'collector');
          for (const raw of results) { if (seen.has(raw.id)) continue; seen.add(raw.id); state.ids.push(raw.id); state.pending.push({ id: raw.id, level }); }
          state.queries.shift(); state.searches++; await checkpoint();
          continue;
        }
        const item = state.pending[0]; state.current = item.id; await checkpoint();
        try {
          const raw = JSON.parse(await runYtdlp(['-J', '--skip-download', '--no-playlist', '--no-warnings', `https://www.youtube.com/watch?v=${item.id}`], { signal: AbortSignal.timeout(60000) }));
          if (raw.id !== item.id) throw Error('Extractor returned a different video.');
          await checkpoint();
          const metadata = descriptiveMetadata(raw);
          await ingest([card(raw)], 'collector');
          await db.execute({ sql: 'INSERT INTO video_details (video_id,metadata,updated_at) VALUES (?,?,?) ON CONFLICT(video_id) DO UPDATE SET metadata=excluded.metadata,updated_at=excluded.updated_at', args: [item.id, JSON.stringify(metadata), Date.now()] });
          state.enriched++;
          if (item.level < job.depth && state.queries.length < 24) {
            for (const q of [raw.channel || raw.uploader, ...(raw.tags || []).slice(0, 2)]) {
              if (state.queries.length >= 24) break;
              const term = clean(q); if (!term) continue;
              state.visited ??= [job.query.toLowerCase()];
              if (state.visited.includes(term.toLowerCase())) continue;
              state.visited.push(term.toLowerCase()); state.queries.push({ q: term, level: item.level + 1 });
            }
          }
        } catch (error) { if (error.name === 'CollectionStopped') throw error; state.failed++; if (state.errors.length < 20) state.errors.push({ id: item.id, error: String(error.message).slice(0, 200) }); }
        state.pending.shift(); await checkpoint();
      }
      state.current = ''; await checkpoint('complete');
    } catch (error) {
      if (error.name === 'CollectionStopped') return true;
      await db.execute({ sql: "UPDATE metadata_collections SET status='failed',state=?,error=?,updated_at=? WHERE id=? AND owner=? AND status='running'", args: [JSON.stringify(state), String(error.message).slice(0, 300), Date.now(), job.id, owner] });
    }
    return true;
  }
  const kick = () => runNext().catch(error => console.error('[metadata collector]', error.message));
  if (autoStart) { const timer = setInterval(kick, 10000); timer.unref?.(); kick(); }
  app.get('/api/admin/analytics', adminAuth, async c => c.json({ ok: true, ...await storageAnalytics(paths) }, 200, { 'Cache-Control': 'no-store' }));
  app.get('/api/admin/metadata', adminAuth, async c => {
    const query = clean(c.req.query('q')), offset = Math.max(0, Math.min(500000, parseInt(c.req.query('offset')) || 0));
    const where = query ? 'WHERE m.hay LIKE ?' : '', args = query ? ['%' + query.replace(/[\\%_]/g, '\\$&') + '%'] : [];
    const filter = where ? where + " ESCAPE '\\'" : '';
    const rows = (await db.execute({ sql: `SELECT m.id,m.card,m.updated_at,d.updated_at AS enriched,t.size AS thumbnailBytes FROM video_meta m LEFT JOIN video_details d ON d.video_id=m.id LEFT JOIN video_thumbnails t ON t.video_id=m.id ${filter} ORDER BY m.updated_at DESC LIMIT 50 OFFSET ?`, args: [...args, offset] })).rows;
    const total = n((await db.execute({ sql: `SELECT COUNT(*) AS n FROM video_meta m ${filter}`, args })).rows[0].n);
    return c.json({ ok: true, total, offset, videos: rows.map(r => ({ ...parse(r.card), updatedAt: n(r.updated_at), enriched: !!r.enriched, thumbnailBytes: n(r.thumbnailBytes) })) });
  });
  app.get('/api/admin/metadata/:id', adminAuth, async c => {
    const row = (await db.execute({ sql: 'SELECT m.card,d.metadata FROM video_meta m LEFT JOIN video_details d ON d.video_id=m.id WHERE m.id=?', args: [c.req.param('id')] })).rows[0];
    return row ? c.json({ ok: true, metadata: row.metadata ? parse(row.metadata) : parse(row.card) }) : c.json({ ok: false, error: 'Not found.' }, 404);
  });
  app.get('/api/admin/collections', adminAuth, async c => c.json({ ok: true, jobs: (await db.execute('SELECT * FROM metadata_collections ORDER BY created_at DESC LIMIT 30')).rows.map(publicJob) }));
  app.post('/api/admin/collections', adminAuth, async c => {
    let body; try { body = await c.req.json(); } catch { return c.json({ ok: false, error: 'Invalid JSON.' }, 400); }
    const query = clean(body?.query), limit = Number(body?.maxVideos), depth = Number(body?.depth);
    if (!query || !Number.isInteger(limit) || limit < 1 || limit > 500 || !Number.isInteger(depth) || depth < 0 || depth > 3) return c.json({ ok: false, error: 'Enter a search, 1–500 videos and depth 0–3.' }, 400);
    const time = Date.now(), id = randomUUID();
    const result = await db.execute({ sql: "INSERT INTO metadata_collections (id,query,max_videos,depth,created_at,updated_at) SELECT ?,?,?,?,?,? WHERE (SELECT COUNT(*) FROM metadata_collections WHERE status IN ('queued','running'))<3", args: [id, query, limit, depth, time, time] });
    if (!result.rowsAffected) return c.json({ ok: false, error: 'Three collections are already active.' }, 429);
    if (autoStart) kick();
    return c.json({ ok: true, id }, 202);
  });
  app.post('/api/admin/collections/:id/cancel', adminAuth, async c => {
    const result = await db.execute({ sql: "UPDATE metadata_collections SET status='cancelled',updated_at=? WHERE id=? AND status IN ('queued','running')", args: [Date.now(), c.req.param('id')] });
    return c.json({ ok: !!result.rowsAffected });
  });
  return { runNext };
 }
--- a/server/admin-analytics.test.js
+++ b/server/admin-analytics.test.js
@@ -0,0 +1,83 @@
 import { test, expect, beforeAll, afterAll } from 'bun:test';
 import { mkdtempSync, rmSync } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import { Hono } from 'hono';
 const root = mkdtempSync(join(tmpdir(), 'ytp-analytics-'));
 process.env.DB_PATH = join(root, 'test.db');
 const { db, initDb, upsertMedia } = await import('./db.js');
 const { registerAnalyticsRoutes, storageAnalytics } = await import('./admin-analytics.js');
 const catalog = await import('./video-catalog.js');
 const originalFetch = globalThis.fetch;
 globalThis.fetch = async () => new Response(new Uint8Array([1,2,3]), { headers: { 'Content-Type': 'image/jpeg' } });
 let app, runner, calls = [], serial = 0;
 const adminAuth = async (c, next) => c.req.header('x-test-admin') === 'yes' ? next() : c.json({ ok: false }, 401);
 const get = path => app.request(path, { headers: { 'x-test-admin': 'yes' } });
 const post = (path, body = {}) => app.request(path, { method: 'POST', headers: { 'x-test-admin': 'yes', 'Content-Type': 'application/json' }, body: JSON.stringify(body) });
 const fakeExtractor = async args => {
  calls.push(args);
  if (args[0].startsWith('ytsearch')) return Array.from({ length: Number(args[0].match(/^ytsearch(\d+)/)[1]) }, () => ({ id: String(++serial).padStart(11, '0'), title: 'Worship ' + serial, channel: 'Channel ' + serial })).map(JSON.stringify).join('\n');
  const id = args.at(-1).split('v=')[1];
  return JSON.stringify({ id, title: 'Enriched ' + id, channel: 'Related ' + id, tags: ['Topic ' + id], duration: 60, description: 'Full description '.repeat(200), view_count: 1234, formats: [{ format_id: '140', acodec: 'aac', url: 'https://temporary.example', http_headers: { Cookie: 'omitted' } }] });
 };
 beforeAll(async () => { await initDb(); app = new Hono(); runner = registerAnalyticsRoutes(app, { adminAuth, runYtdlp: fakeExtractor, autoStart: false, paths: [{ label: 'DB', path: root }] }); });
 afterAll(async () => { await catalog.drainThumbnails(); globalThis.fetch = originalFetch; db.close(); rmSync(root, { recursive: true, force: true }); });
 test('admin routes reject unauthenticated requests and validate collector limits', async () => {
  for (const path of ['/api/admin/analytics','/api/admin/metadata','/api/admin/metadata/00000000001','/api/admin/collections']) expect((await app.request(path)).status).toBe(401);
  expect((await app.request('/api/admin/collections', { method: 'POST' })).status).toBe(401);
  for (const body of [{ query: '', maxVideos: 5, depth: 0 }, { query: 'worship', maxVideos: 501, depth: 0 }, { query: 'worship', maxVideos: 5, depth: 4 }]) expect((await post('/api/admin/collections', body)).status).toBe(400);
  for (let i = 0; i < 3; i++) expect((await post('/api/admin/collections', { query: 'queue ' + i, maxVideos: 1, depth: 0 })).status).toBe(202);
  expect((await post('/api/admin/collections', { query: 'queue 4', maxVideos: 1, depth: 0 })).status).toBe(429);
  for (const job of (await (await get('/api/admin/collections')).json()).jobs) await post('/api/admin/collections/' + job.id + '/cancel');
 });
 test('depth zero stores full descriptive metadata and actual thumbnail bytes', async () => {
  calls = []; await post('/api/admin/collections', { query: 'root worship', maxVideos: 4, depth: 0 }); await runner.runNext(); await catalog.drainThumbnails();
  const job = (await (await get('/api/admin/collections')).json()).jobs.find(j => j.query === 'root worship');
  expect(job.status).toBe('complete'); expect(job.collected).toBe(4); expect(job.enriched).toBe(4); expect(calls.filter(a => a[0].startsWith('ytsearch'))).toHaveLength(1);
  const videos = (await (await get('/api/admin/metadata')).json()).videos;
  expect(videos).toHaveLength(4); expect(videos.every(v => v.enriched && v.thumbnailBytes === 3)).toBe(true);
  const detail = (await (await get('/api/admin/metadata/' + videos[0].id)).json()).metadata;
  expect(detail.description.length).toBeGreaterThan(1200); expect(detail.view_count).toBe(1234); expect(detail.formats[0].url).toBeUndefined(); expect(detail.formats[0].http_headers).toBeUndefined();
 });
 test('related depth follows channels/topics but caps unique videos across all searches', async () => {
  calls = []; await post('/api/admin/collections', { query: 'deep worship', maxVideos: 10, depth: 2 }); await runner.runNext();
  const job = (await (await get('/api/admin/collections')).json()).jobs.find(j => j.query === 'deep worship');
  expect(job.status).toBe('complete'); expect(job.collected).toBe(10); expect(job.enriched).toBe(10);
  const searches = calls.filter(a => a[0].startsWith('ytsearch')); expect(searches.length).toBeGreaterThan(1); expect(searches[1][0]).toContain('Related'); expect(searches.length).toBeLessThanOrEqual(24);
 });
 test('expired jobs resume pending videos without repeating their search', async () => {
  await db.execute({ sql: "INSERT INTO metadata_collections (id,query,max_videos,depth,status,state,lease_until,created_at,updated_at) VALUES ('restart','resume',1,0,'running',?,0,0,0)", args: [JSON.stringify({ ids: ['99999999999'], queries: [], pending: [{ id: '99999999999', level: 0 }] })] });
  calls = []; await runner.runNext(); const job = (await (await get('/api/admin/collections')).json()).jobs.find(j => j.id === 'restart');
  expect(job.status).toBe('complete'); expect(job.collected).toBe(1); expect(job.enriched).toBe(1); expect(calls).toHaveLength(1);
 });
 test('storage totals count table aggregates once and use actual disk capacity', async () => {
  await upsertMedia('aaaaaaaaaaa', { status: 'ready', size: 2048, duration: 120 });
  await db.execute("INSERT INTO listening_daily (fingerprint,day,video_id,plays) VALUES ('listener','2026-10-03','aaaaaaaaaaa',3)");
  const result = await storageAnalytics([{ label: 'DB', path: root }, { label: 'Absent', path: root + '/missing' }]);
  expect(Number(result.media.find(r => r.status === 'ready').bytes)).toBe(2048); expect(Number(result.listening.plays)).toBe(3); expect(result.volumes[0].free).toBeGreaterThan(0); expect(result.volumes[1].unavailable).toBe(true);
  expect(result.sources.find(r => r.source === 'collector')).toBeDefined(); expect((await get('/api/admin/metadata?q=Related&offset=0')).status).toBe(200); expect((await get('/api/admin/metadata?q=%25')).status).toBe(200);
 });
 test('extraction errors remain visible while discovered cards stay stored', async () => {
  const failureApp = new Hono(); const failed = registerAnalyticsRoutes(failureApp, { adminAuth, autoStart: false, runYtdlp: async a => a[0].startsWith('ytsearch') ? JSON.stringify({ id: 'failure0001', title: 'Discovered before failure' }) : Promise.reject(Error('Unavailable video')) });
  await post('/api/admin/collections', { query: 'failure', maxVideos: 1, depth: 0 }); await failed.runNext();
  const job = (await (await get('/api/admin/collections')).json()).jobs.find(j => j.query === 'failure');
  expect(job.status).toBe('complete'); expect(job.failed).toBe(1); expect(job.collected).toBe(1); expect(job.errors[0].error).toBe('Unavailable video');
 });
 test('duplicate search cards do not consume slots before unique results', async () => {
  const a = { id: 'unique00001', title: 'Unique A' }, b = { id: 'unique00002', title: 'Unique B' };
  const worker = registerAnalyticsRoutes(new Hono(), { adminAuth, autoStart: false, runYtdlp: async args => args[0].startsWith('ytsearch') ? [a,a,b].map(JSON.stringify).join('\n') : JSON.stringify(args.at(-1).endsWith(a.id) ? a : b) });
  await post('/api/admin/collections', { query: 'duplicates', maxVideos: 2, depth: 0 }); await worker.runNext();
  const job = (await (await get('/api/admin/collections')).json()).jobs.find(j => j.query === 'duplicates');
  expect(job.collected).toBe(2); expect(job.enriched).toBe(2);
 });
 test('cancelling an in-flight search prevents storing its returned cards', async () => {
  let started, release;
  const began = new Promise(resolve => { started = resolve; }), pause = new Promise(resolve => { release = resolve; });
  const worker = registerAnalyticsRoutes(new Hono(), { adminAuth, autoStart: false, runYtdlp: async () => { started(); await pause; return JSON.stringify({ id: 'cancel00001', title: 'Should not save' }); } });
  const queued = await (await post('/api/admin/collections', { query: 'cancel while searching', maxVideos: 1, depth: 0 })).json();
  const work = worker.runNext(); await began; await post('/api/admin/collections/' + queued.id + '/cancel'); release(); await work;
  const job = (await (await get('/api/admin/collections')).json()).jobs.find(j => j.id === queued.id);
  expect(job.status).toBe('cancelled'); expect(job.failed).toBe(0);
  expect((await db.execute("SELECT * FROM video_meta WHERE id='cancel00001'")).rows).toHaveLength(0);
 });
--- a/server/db.js
+++ b/server/db.js
@@ -157,6 +157,28 @@ export async function initDb() {
    );
    CREATE INDEX IF NOT EXISTS idx_video_channel ON video_channels (channel,updated_at DESC,video_id);
    CREATE TABLE IF NOT EXISTS video_details (
      video_id TEXT PRIMARY KEY, metadata TEXT NOT NULL, updated_at INTEGER NOT NULL
    );
    CREATE TABLE IF NOT EXISTS metadata_collections (
      id TEXT PRIMARY KEY, query TEXT NOT NULL, max_videos INTEGER NOT NULL, depth INTEGER NOT NULL,
      status TEXT NOT NULL DEFAULT 'queued', state TEXT NOT NULL DEFAULT '{}', error TEXT,
      lease_until INTEGER NOT NULL DEFAULT 0, owner TEXT,
      created_at INTEGER NOT NULL, updated_at INTEGER NOT NULL
    );
    CREATE INDEX IF NOT EXISTS idx_collections_queue ON metadata_collections (status,created_at);
    CREATE TABLE IF NOT EXISTS lyric_transcriptions (
      id TEXT PRIMARY KEY, video_id TEXT NOT NULL, status TEXT NOT NULL DEFAULT 'queued',
      base_rev INTEGER NOT NULL DEFAULT 0, result TEXT, error TEXT, stage TEXT NOT NULL DEFAULT 'queued',
      attempts INTEGER NOT NULL DEFAULT 0, lease_token TEXT, lease_until INTEGER NOT NULL DEFAULT 0,
      created_at INTEGER NOT NULL, updated_at INTEGER NOT NULL
    );
    CREATE UNIQUE INDEX IF NOT EXISTS idx_transcription_active ON lyric_transcriptions (video_id)
      WHERE status IN ('queued','running');
    CREATE INDEX IF NOT EXISTS idx_transcription_queue ON lyric_transcriptions (status,created_at);
    CREATE TABLE IF NOT EXISTS lyrics_worker_state (id INTEGER PRIMARY KEY CHECK(id=1), last_seen INTEGER NOT NULL);
    -- Shared per-video documents (kind = lyrics | chapters), visible to every
    -- user. The live copy is here; every save also lands in video_note_revs
    -- as a full snapshot, which is the server-side backup and undo history.
--- a/server/notes.js
+++ b/server/notes.js
@@ -42,6 +42,8 @@ import { createHash, createHmac, randomBytes, timingSafeEqual } from 'node:crypt
 import { mkdirSync, readdirSync, unlinkSync, writeFileSync, readFileSync } from 'node:fs';
 import { join } from 'node:path';
 import { getCookie, setCookie, deleteCookie } from 'hono/cookie';
 import { registerAnalyticsRoutes } from './admin-analytics.js';
 import { registerTranscriptionRoutes } from './transcriptions.js';
 export const NOTE_KINDS = new Set(['lyrics', 'chapters']);
 // A YouTube id or one of the server's own uploads (see uploads.js).
@@ -59,6 +61,12 @@ function cleanText(v, max) {
  return String(v == null ? '' : v).replace(/[\u0000-\u001f\u007f]+/g, ' ').trim().slice(0, max);
 }
 function cleanLyricText(value) {
  return String(value ?? '').replace(/\r\n?/g, '\n')
      .replace(/[\u0000-\u0009\u000b-\u001f\u007f]+/g, ' ')
      .split('\n').map(part => part.trim()).filter(Boolean).join('\n').trim().slice(0, MAX_LINE_CHARS);
 }
 function cleanTime(v) {
  if (v === null || v === undefined || v === '') return null;
  const n = Number(v);
@@ -76,7 +84,7 @@ export function sanitizeLyrics(input) {
  const lines = [];
  for (const l of rawLines) {
    if (!l || typeof l !== 'object') continue;
-    const text = cleanText(l.text, MAX_LINE_CHARS);
+    const text = cleanLyricText(l.text);
    if (!text) continue;
    const kind = l.kind === 'section' || l.kind === 'cue' ? l.kind : 'line';
    lines.push({ t: cleanTime(l.t), text, kind });
@@ -426,7 +434,7 @@ export function registerNoteRoutes(app, deps) {
    if (!who) return c.json({ ok: false, error: 'link an online profile to report a lyric line' }, 401);
    if (who.invalid) return c.json({ ok: false, error: 'invalid API token' }, 401);
    if (flagOverBudget(who.by)) return c.json({ ok: false, error: 'too many reports — wait a few minutes' }, 429);
-    const text = cleanText(body.text, MAX_LINE_CHARS);
+    const text = cleanLyricText(body.text);
    if (!text) return c.json({ ok: false, error: 'which line? (text is missing)' }, 400);
    try {
      const lyr = (await db.getNotes(id)).lyrics;
@@ -662,6 +670,18 @@ export function registerNoteRoutes(app, deps) {
    await next();
  };
  if (deps.analytics) registerAnalyticsRoutes(app, { adminAuth: requireAdminOrToken, runYtdlp, ...deps.analytics });
  registerTranscriptionRoutes(app, {
    db, adminAuth: requireAdminOrToken, sanitizeLyrics,
    workerEnabled: !!workerToken && workerToken.length >= 24,
    workerAuth: async (c, next) => {
      const token = (c.req.header('authorization') || '').match(/^Bearer\s+(\S+)$/i)?.[1];
      if (!workerToken || workerToken.length < 24 || !token || !safeEqual(token, workerToken)) return c.json({ ok: false, error: 'lyrics worker token required' }, 401);
      await next();
    },
  });
  // Saved (server-cached) videos with whether each already has lyrics — the
  // work list for batch lyric injection.
  app.get('/api/admin/media', requireAdminOrToken, async (c) => {
--- a/server/notes.test.js
+++ b/server/notes.test.js
@@ -84,6 +84,12 @@ describe('pure helpers', () => {
    expect(d.offset).toBe(30);
  });
  test('lyrics retain tight line breaks while other metadata strips controls', () => {
    const result = N.sanitizeLyrics({ lines: [{ t: 12, text: ' Because You are God\r\n You can do anything\u0007 ', kind: 'line' }], tags: ['Key\nG'] });
    expect(result.lines).toEqual([{ t: 12, text: 'Because You are God\nYou can do anything', kind: 'line' }]);
    expect(result.tags).toEqual(['Key G']);
  });
  test('sanitizeChapters requires time + title and sorts', () => {
    const d = N.sanitizeChapters({ items: [{ t: 30, title: 'B' }, { t: 5, title: 'A', note: 'n' }, { t: null, title: 'no time' }] });
    expect(d.items).toEqual([{ t: 5, title: 'A', note: 'n' }, { t: 30, title: 'B', note: '' }]);
@@ -245,3 +251,19 @@ describe('routes', () => {
    expect(await res.text()).toContain('admin');
  });
 });
 test('saving a grouped cue preserves one timestamp in current and revision documents', async () => {
  const id = 'groupedCue1';
  const lines = [{ t: 12, text: 'Because You are God\nYou can do anything', kind: 'line' }];
  const response = await app.request(`/api/notes/${id}/lyrics`, json('PUT', { data: { lines }, baseRev: 0, profile: 'josh' }));
  expect(response.status).toBe(200);
  const current = await (await app.request(`/api/notes/${id}`)).json();
  expect(current.lyrics.data.lines).toEqual(lines);
  const revision = await (await app.request(`/api/notes/${id}/lyrics/revs/1`)).json();
  expect(revision.data.lines).toEqual(lines);
  const flag = await app.request(`/api/notes/${id}/flags`, json('POST', { text: lines[0].text, profile: 'josh', reason: 'words' }));
  expect(flag.status).toBe(200);
  const flags = await (await app.request(`/api/notes/${id}/flags`)).json();
  expect(flags.flags[0].text).toBe(lines[0].text);
 });
--- a/server/package.json
+++ b/server/package.json
@@ -6,7 +6,7 @@
  "scripts": {
    "start": "bun server.js",
    "dev": "bun --hot server.js",
-    "test": "bun test ./recommendations.test.js && bun test --timeout 60000 ./media-cache.test.js && bun test ./notes.test.js && bun test ./remote.test.js && bun test ./party.test.js && bun test ./uploads.test.js && bun test ./innertube.test.js && bun test ./ytdlp-pool.test.js && bun test ./p2p-db.test.js && bun test ./p2p-admit.test.js && bun test ./p2p-retention.test.js && bun test ./p2p-routes.test.js && bun test ./p2p-hub.test.js && bun test --timeout 60000 ./p2p-intake.test.js && bun test ./flags.test.js"
+    "test": "bun test ./recommendations.test.js && bun test --timeout 60000 ./media-cache.test.js && bun test ./notes.test.js && bun test ./transcriptions.test.js && bun test ./admin-analytics.test.js && bun test ./remote.test.js && bun test ./party.test.js && bun test ./uploads.test.js && bun test ./innertube.test.js && bun test ./ytdlp-pool.test.js && bun test ./p2p-db.test.js && bun test ./p2p-admit.test.js && bun test ./p2p-retention.test.js && bun test ./p2p-routes.test.js && bun test ./p2p-hub.test.js && bun test --timeout 60000 ./p2p-intake.test.js && bun test ./flags.test.js"
  },
  "dependencies": {
    "@hono/node-server": "^1.14.0",
--- a/server/recommendations.js
+++ b/server/recommendations.js
@@ -84,7 +84,7 @@ export function registerCatalogRoutes(app, { resolveListener = async (c, name, f
      ? c.req.raw.clone() : null;
    await next();
    if (!c.res.ok) return;
-    const discovery = c.req.method === 'GET' && !path.startsWith('/api/catalog/') && path !== '/api/recommendations';
+    const discovery = c.req.method === 'GET' && !/^\/api\/admin\/(analytics|metadata|collections)(\/|$)/.test(path) && !path.startsWith('/api/catalog/') && path !== '/api/recommendations';
    try {
      if (discovery && c.res.headers.get('content-type')?.includes('application/json')) {
        const result = await c.res.clone().json();
--- a/server/server.js
+++ b/server/server.js
@@ -2328,6 +2328,11 @@ const notes = registerNoteRoutes(app, {
  backupDir: pathJoin(dirname(process.env.DB_PATH || './data/ytplayer.db'), 'backups'),
  adminHtmlPath: './public/admin.html',
  workerToken: process.env.LYRICS_WORKER_TOKEN || '',
  analytics: { paths: [
    { label: 'Video cache', path: MEDIA_DIR },
    { label: 'Uploads', path: process.env.UPLOAD_DIR || pathJoin(dirname(process.env.DB_PATH || './data/ytplayer.db'), 'uploads') },
    { label: 'Database', path: dirname(process.env.DB_PATH || './data/ytplayer.db') },
  ] },
 });
 // ============================================================================
--- a/server/transcriptions.js
+++ b/server/transcriptions.js
@@ -0,0 +1,79 @@
 // Explicit Whisper requests produce reviewable drafts, never published notes.
 import { randomUUID } from 'node:crypto';
 import { db as sql } from './db.js';
 const ID = /^([\w-]{11}|upl_[a-f0-9]{12})$/;
 const LEASE_MS = 120000;
 function publicJob(row) {
  if (!row) return null;
  return { id: row.id, videoId: row.video_id, status: row.status, stage: row.stage,
    baseRev: Number(row.base_rev), result: row.result ? JSON.parse(row.result) : null,
    error: row.error, createdAt: Number(row.created_at), updatedAt: Number(row.updated_at) };
 }
 export function registerTranscriptionRoutes(app, { db, adminAuth, workerAuth, workerEnabled, sanitizeLyrics, now = Date.now }) {
  const seen = () => sql.execute({ sql: 'INSERT INTO lyrics_worker_state (id,last_seen) VALUES (1,?) ON CONFLICT(id) DO UPDATE SET last_seen=excluded.last_seen', args: [now()] });
  async function source(id) {
    if (id.startsWith('upl_')) return db.getUpload(id);
    const row = await db.getMedia(id); return row?.status === 'ready' ? row : null;
  }
  app.post('/api/admin/transcriptions/:video', adminAuth, async c => {
    const id = c.req.param('video');
    if (!ID.test(id)) return c.json({ ok: false, error: 'invalid video id' }, 400);
    if (!workerEnabled) return c.json({ ok: false, error: 'Whisper is unavailable: the lyrics worker is not configured.' }, 503);
    const audio = await source(id);
    if (!audio) return c.json({ ok: false, error: 'Save this video on the server before transcribing its audio.' }, 409);
    if (Number(audio.duration) > 3600) return c.json({ ok: false, error: 'Whisper requests are limited to one hour of audio.' }, 422);
    const notes = await db.getNotes(id);
    // One active job per song and a bounded queue, including across restarts.
    const existing = (await sql.execute({ sql: "SELECT * FROM lyric_transcriptions WHERE video_id=? AND status IN ('queued','running')", args: [id] })).rows[0];
    if (existing) return c.json({ ok: true, job: publicJob(existing) });
    const active = Number((await sql.execute("SELECT COUNT(*) AS n FROM lyric_transcriptions WHERE status IN ('queued','running')")).rows[0].n);
    if (active >= 25) return c.json({ ok: false, error: 'The transcription queue is full. Try again after a job finishes.' }, 429);
    const time = now();
    await sql.execute({ sql: `INSERT OR IGNORE INTO lyric_transcriptions (id,video_id,base_rev,created_at,updated_at)
      SELECT ?,?,?,?,? WHERE (SELECT COUNT(*) FROM lyric_transcriptions WHERE status IN ('queued','running')) < 25`,
      args: [randomUUID(), id, notes.lyrics?.rev || 0, time, time] });
    const row = (await sql.execute({ sql: "SELECT * FROM lyric_transcriptions WHERE video_id=? AND status IN ('queued','running')", args: [id] })).rows[0];
    return row ? c.json({ ok: true, job: publicJob(row) }, 202) : c.json({ ok: false, error: 'The transcription queue is full.' }, 429);
  });
  app.get('/api/admin/transcriptions/:video', adminAuth, async c => {
    const id = c.req.param('video');
    if (!ID.test(id)) return c.json({ ok: false, error: 'invalid video id' }, 400);
    const row = (await sql.execute({ sql: 'SELECT * FROM lyric_transcriptions WHERE video_id=? ORDER BY created_at DESC,rowid DESC LIMIT 1', args: [id] })).rows[0];
    const worker = (await sql.execute('SELECT last_seen FROM lyrics_worker_state WHERE id=1')).rows[0];
    return c.json({ ok: true, job: publicJob(row), workerOnline: !!worker && now() - Number(worker.last_seen) < 90000, enabled: !!workerEnabled }, 200, { 'Cache-Control': 'no-store' });
  });
  app.post('/api/lyrics-worker/claim', workerAuth, async c => {
    await seen(); const time = now();
    await sql.execute({ sql: "UPDATE lyric_transcriptions SET status='failed',stage='failed',error='The worker stopped repeatedly. Please retry.',updated_at=? WHERE status='running' AND lease_until<? AND attempts>=3", args: [time, time] });
    // UPDATE RETURNING makes claiming atomic for multiple worker processes.
    const row = (await sql.execute({ sql: `UPDATE lyric_transcriptions SET status='running',stage='loading-model',attempts=attempts+1,
      lease_token=?,lease_until=?,updated_at=? WHERE id=(SELECT id FROM lyric_transcriptions
      WHERE (status='queued' OR (status='running' AND lease_until<?)) AND attempts<3 ORDER BY created_at LIMIT 1) RETURNING *`,
      args: [randomUUID(), time + LEASE_MS, time, time] })).rows[0];
    return c.json({ ok: true, job: row ? { ...publicJob(row), lease: row.lease_token,
      audioPath: row.video_id.startsWith('upl_') ? `/api/uploads/${row.video_id}` : `/api/media/${row.video_id}?a=1` } : null });
  });
  app.post('/api/lyrics-worker/jobs/:job', workerAuth, async c => {
    let body;
    try { const raw = await c.req.text(); if (raw.length > 400000) return c.json({ ok: false, error: 'transcript too large' }, 413); body = JSON.parse(raw); }
    catch { return c.json({ ok: false, error: 'invalid JSON' }, 400); }
    if (!body || typeof body.lease !== 'string') return c.json({ ok: false, error: 'missing lease' }, 400);
    const time = now();
    let result = null, status = 'running', error = null;
    let stage = ['loading-model', 'downloading-audio', 'transcribing'].includes(body.stage) ? body.stage : 'transcribing';
    if (body.status === 'complete') {
      try { result = sanitizeLyrics(body.result); if (!result.lines.length) throw Error('No sung lyrics were found.'); }
      catch (e) { return c.json({ ok: false, error: e.message }, 400); }
      result.tags = ['auto-transcribed (whisper)']; status = 'complete'; stage = 'complete';
    } else if (body.status === 'failed') { status = 'failed'; stage = 'failed'; error = String(body.error || 'Transcription failed.').slice(0, 300); }
    const changed = await sql.execute({ sql: `UPDATE lyric_transcriptions SET status=?,stage=?,result=?,error=?,lease_until=?,updated_at=?
      WHERE id=? AND status='running' AND lease_token=? AND lease_until>=?`,
      args: [status, stage, result ? JSON.stringify(result) : null, error, time + LEASE_MS, time, c.req.param('job'), body.lease, time] });
    if (!changed.rowsAffected) return c.json({ ok: false, error: 'This job lease has expired or finished.' }, 409);
    await seen();
    // Bound completed draft retention; active jobs are never deleted.
    await sql.execute({ sql: "DELETE FROM lyric_transcriptions WHERE status IN ('complete','failed') AND updated_at<?", args: [time - 30 * 86400000] });
    return c.json({ ok: true });
  });
 }
--- a/server/transcriptions.test.js
+++ b/server/transcriptions.test.js
@@ -0,0 +1,97 @@
 import { test, expect, beforeAll, afterAll } from 'bun:test';
 import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import { Hono } from 'hono';
 const root = mkdtempSync(join(tmpdir(), 'ytp-transcriptions-'));
 process.env.DB_PATH = join(root, 'test.db');
 const db = await import('./db.js');
 const { registerNoteRoutes } = await import('./notes.js');
 const VID = '0gfX0dFLaBc';
 const TOKEN = 'worker-test-token-'.repeat(3);
 const worker = { Authorization: 'Bearer ' + TOKEN };
 const json = (body = {}, headers = {}) => ({ method: 'POST', headers: { 'Content-Type': 'application/json', ...headers }, body: JSON.stringify(body) });
 let app, cookie, job, lease;
 beforeAll(async () => {
  await db.initDb(); await db.upsertMedia(VID, { status: 'ready', duration: 240 });
  await db.saveNote({ videoId: VID, kind: 'lyrics', baseRev: 0, source: 'user', updatedBy: 'fixture', data: { lines: [{ t: 1, text: 'Original saved lyrics', kind: 'line' }], tags: [], offset: 0 } });
  writeFileSync(join(root, 'admin.html'), '<title>admin</title>');
  app = new Hono();
  registerNoteRoutes(app, { db, getProfile: async () => null, profileNameRe: /^[\w-]{3,40}$/, runYtdlp: async () => '{}', adminPassword: 'test-password', workerToken: TOKEN, backupDir: join(root, 'backups'), adminHtmlPath: join(root, 'admin.html') });
  const login = await app.request('/api/admin/login', json({ password: 'test-password' }));
  cookie = login.headers.get('set-cookie').split(';')[0];
 });
 afterAll(() => { db.db.close(); rmSync(root, { recursive: true, force: true }); });
 const admin = () => ({ Cookie: cookie });
 const draft = { lines: [{ t: 2, text: 'Because You are God', kind: 'line' }, { t: 6, text: 'You can do anything', kind: 'line' }], tags: ['wrong source'], offset: 0 };
 test('admin and worker authentication are enforced separately', async () => {
  expect((await app.request('/api/admin/transcriptions/' + VID, json())).status).toBe(401);
  expect((await app.request('/api/admin/transcriptions/' + VID)).status).toBe(401);
  expect((await app.request('/api/lyrics-worker/claim', json({}, admin()))).status).toBe(401);
 });
 test('two button requests deduplicate and preserve published lyrics', async () => {
  const first = await app.request('/api/admin/transcriptions/' + VID, json({}, admin()));
  expect(first.status).toBe(202); job = (await first.json()).job;
  const second = await (await app.request('/api/admin/transcriptions/' + VID, json({}, admin()))).json();
  expect(second.job.id).toBe(job.id); expect(job.baseRev).toBe(1);
  expect((await db.getNotes(VID)).lyrics.data.lines[0].text).toBe('Original saved lyrics');
  await db.initDb();
  expect((await (await app.request('/api/admin/transcriptions/' + VID, { headers: admin() })).json()).job.id).toBe(job.id);
 });
 test('claiming is atomic and does not expose leases to the admin UI', async () => {
  const responses = await Promise.all([app.request('/api/lyrics-worker/claim', json({}, worker)), app.request('/api/lyrics-worker/claim', json({}, worker))]);
  const jobs = await Promise.all(responses.map(r => r.json()));
  expect(jobs.filter(r => r.job)).toHaveLength(1); lease = jobs.find(r => r.job).job.lease;
  expect(jobs.find(r => r.job).job.audioPath).toBe('/api/media/' + VID + '?a=1');
  const view = await (await app.request('/api/admin/transcriptions/' + VID, { headers: admin() })).json();
  expect(view.job.lease).toBeUndefined(); expect(view.workerOnline).toBe(true);
 });
 test('heartbeat advances stages and invalid leases cannot finish jobs', async () => {
  expect((await app.request('/api/lyrics-worker/jobs/' + job.id, json({ lease: 'wrong', status: 'complete', result: draft }, worker))).status).toBe(409);
  expect((await app.request('/api/lyrics-worker/jobs/' + job.id, json({ lease, stage: 'transcribing' }, worker))).status).toBe(200);
  const view = await (await app.request('/api/admin/transcriptions/' + VID, { headers: admin() })).json();
  expect(view.job.stage).toBe('transcribing');
 });
 test('completion returns a sanitized review draft without overwriting saved lyrics', async () => {
  expect((await app.request('/api/lyrics-worker/jobs/' + job.id, json({ lease, status: 'complete', result: draft }, worker))).status).toBe(200);
  const view = await (await app.request('/api/admin/transcriptions/' + VID, { headers: admin() })).json();
  expect(view.job.status).toBe('complete'); expect(view.job.result.lines).toHaveLength(2);
  expect(view.job.result.tags).toEqual(['auto-transcribed (whisper)']);
  expect((await db.getNotes(VID)).lyrics.rev).toBe(1);
  expect((await app.request('/api/lyrics-worker/jobs/' + job.id, json({ lease, status: 'failed' }, worker))).status).toBe(409);
 });
 test('expired jobs are reclaimed with a new lease; stale results are rejected', async () => {
  const queued = await (await app.request('/api/admin/transcriptions/' + VID, json({}, admin()))).json();
  expect(queued.job.id).not.toBe(job.id); job = queued.job;
  const first = (await (await app.request('/api/lyrics-worker/claim', json({}, worker))).json()).job;
  await db.db.execute({ sql: 'UPDATE lyric_transcriptions SET lease_until=0 WHERE id=?', args: [job.id] });
  const reclaimed = (await (await app.request('/api/lyrics-worker/claim', json({}, worker))).json()).job;
  expect(reclaimed.lease).not.toBe(first.lease);
  expect((await app.request('/api/lyrics-worker/jobs/' + job.id, json({ lease: first.lease, status: 'complete', result: draft }, worker))).status).toBe(409);
  lease = reclaimed.lease;
 });
 test('empty transcripts and oversized drafts are rejected, and failures retain saved lyrics', async () => {
  expect((await app.request('/api/lyrics-worker/jobs/' + job.id, json({ lease, status: 'complete', result: { lines: [] } }, worker))).status).toBe(400);
  expect((await app.request('/api/lyrics-worker/jobs/' + job.id, json({ lease, result: 'x'.repeat(400001) }, worker))).status).toBe(413);
  expect((await app.request('/api/lyrics-worker/jobs/' + job.id, json({ lease, status: 'failed', error: 'No sung vocals were detected.' }, worker))).status).toBe(200);
  const view = await (await app.request('/api/admin/transcriptions/' + VID, { headers: admin() })).json();
  expect(view.job.error).toBe('No sung vocals were detected.'); expect((await db.getNotes(VID)).lyrics.rev).toBe(1);
 });
 test('uncached songs, invalid IDs, and overlong audio give usable errors', async () => {
  expect((await app.request('/api/admin/transcriptions/nope', json({}, admin()))).status).toBe(400);
  expect((await app.request('/api/admin/transcriptions/aaaaaaaaaaa', json({}, admin()))).status).toBe(409);
  await db.upsertMedia('bbbbbbbbbbb', { status: 'ready', duration: 4000 });
  expect((await app.request('/api/admin/transcriptions/bbbbbbbbbbb', json({}, admin()))).status).toBe(422);
 });
 test('uploads use the upload media endpoint and repeated worker crashes fail visibly', async () => {
  const id = 'upl_0123456789ab';
  await db.createUpload({ id, kind: 'audio', title: 'Fixture upload', duration: 30, ext: 'm4a', mime: 'audio/mp4', size: 100 });
  await app.request('/api/admin/transcriptions/' + id, json({}, admin()));
  const claim = (await (await app.request('/api/lyrics-worker/claim', json({}, worker))).json()).job;
  expect(claim.audioPath).toBe('/api/uploads/' + id);
  await db.db.execute({ sql: 'UPDATE lyric_transcriptions SET lease_until=0,attempts=3 WHERE id=?', args: [claim.id] });
  await app.request('/api/lyrics-worker/claim', json({}, worker));
  const view = await (await app.request('/api/admin/transcriptions/' + id, { headers: admin() })).json();
  expect(view.job.status).toBe('failed');
 });
--- a/server/video-catalog.js
+++ b/server/video-catalog.js
@@ -2,7 +2,7 @@
 import { db } from './db.js';
 const VIDEO_ID = /^[\w-]{11}$/;
-const SOURCES = new Set(['search', 'search-cache', 'client-search', 'channel', 'streams', 'playlist', 'profile', 'sync', 'related', 'backfill']);
+const SOURCES = new Set(['search', 'search-cache', 'client-search', 'channel', 'streams', 'playlist', 'profile', 'sync', 'related', 'backfill', 'collector']);
 const text = (v, n = 300) => typeof v === 'string' ? v.trim().slice(0, n) : '';
 const positive = v => Number.isFinite(Number(v)) && Number(v) > 0 ? Number(v) : 0;
 const canonicalThumb = id => `https://i.ytimg.com/vi/${id}/hqdefault.jpg`;
@@ -90,6 +90,7 @@ export async function trimCatalog(max = Number(process.env.VIDEO_META_MAX) || 50
    'DELETE FROM video_meta_sources WHERE video_id NOT IN (SELECT id FROM video_meta)',
    'DELETE FROM video_thumbnails WHERE video_id NOT IN (SELECT id FROM video_meta)',
    'DELETE FROM video_channels WHERE video_id NOT IN (SELECT id FROM video_meta)',
    'DELETE FROM video_details WHERE video_id NOT IN (SELECT id FROM video_meta)',
  ], 'write');
  return count - max;
 }
--- a/tests/admin-lyrics.spec.js
+++ b/tests/admin-lyrics.spec.js
@@ -0,0 +1,152 @@
 const { test, expect } = require('@playwright/test');
 const id = '0gfX0dFLaBc';
 async function fixture(page) {
  const state = { job: null, starts: 0, saves: [] };
  await page.route('**/api/**', async route => {
    const req = route.request(), url = new URL(req.url()); let body = { ok: true };
    if (url.pathname === '/api/admin/me') body = { enabled: true, admin: true };
    else if (url.pathname === '/api/admin/media') body = { ok: true, media: [{ id, title: 'Because You are God', channel: 'Cathedral of Praise Worship', lyricsLines: 47 }] };
    else if (url.pathname === '/api/uploads') body.uploads = [];
    else if (url.pathname === '/api/admin/flags') body = { ok: true, flags: [], open: 0 };
    else if (url.pathname === '/api/notes/' + id) body = { lyrics: { rev: 8, data: { lines: Array.from({ length: 47 }, (_, i) => ({ t: i * 5, text: 'Original cue ' + i, kind: 'line' })), tags: ['Key G'], offset: 0 } } };
    else if (url.pathname === '/api/streams') body = { data: { meta: { title: 'Because You are God', duration: 240 }, qualities: [] } };
    else if (url.pathname === '/api/admin/transcriptions/' + id) {
      if (req.method() === 'POST') { state.starts++; state.job = { id: 'job', videoId: id, status: 'queued', baseRev: 8 }; }
      body = { ok: true, job: state.job };
    } else if (url.pathname === '/api/notes/' + id + '/lyrics' && req.method() === 'PUT') { state.saves.push(req.postDataJSON()); body.rev = 9; }
    return route.fulfill({ contentType: 'application/json', body: JSON.stringify(body) });
  });
  await page.goto('/admin.html#editor');
  await expect(page.locator('[data-open="' + id + '"]').first()).toBeVisible();
  return state;
 }
 function complete(state) {
  state.job = { ...state.job, status: 'complete', result: { lines: [{ t: 1, text: 'Because You are God', kind: 'line' }, { t: 4, text: 'You can do anything', kind: 'line' }], tags: ['auto-transcribed (whisper)'], offset: 0 } };
 }
 test('row and editor buttons queue one job and require review before saving', async ({ page }) => {
  const state = await fixture(page);
  await page.locator('[data-whisper]').click();
  await expect(page.locator('#whisperBtn')).toBeDisabled();
  expect(state.starts).toBe(1);
  await expect(page.locator('.ln')).toHaveCount(47);
  complete(state);
  await expect(page.locator('#whisperUse')).toBeVisible({ timeout: 10000 });
  await expect(page.locator('.ln')).toHaveCount(47); expect(state.saves).toHaveLength(0);
  await page.locator('#whisperUse').click();
  await expect(page.locator('.ln')).toHaveCount(2); expect(state.saves).toHaveLength(0);
  await page.locator('#bUndo').click(); await expect(page.locator('.ln')).toHaveCount(47);
  await page.locator('#bRedo').click(); await expect(page.locator('.ln')).toHaveCount(2);
  await page.locator('#save').click();
  await expect.poll(() => state.saves.length).toBe(1);
  expect(state.saves[0].baseRev).toBe(8); expect(state.saves[0].data.lines[0].text).toBe('Because You are God');
 });
 test('editor button keeps unsaved words until the draft is explicitly applied', async ({ page }) => {
  const state = await fixture(page);
  await page.locator('[data-open="' + id + '"]').first().click();
  await page.locator('.ln-text').first().fill('My correction');
  await page.locator('#whisperBtn').click(); complete(state);
  await expect(page.locator('#whisperUse')).toBeVisible({ timeout: 10000 });
  await expect(page.locator('.ln-text').first()).toHaveValue('My correction');
  page.once('dialog', dialog => dialog.dismiss()); await page.locator('#whisperUse').click();
  await expect(page.locator('.ln-text').first()).toHaveValue('My correction');
  expect(state.saves).toHaveLength(0);
 });
 test('mobile buttons remain reachable and failed requests allow retry', async ({ page }) => {
  await page.setViewportSize({ width: 390, height: 844 });
  await fixture(page);
  const button = page.locator('[data-whisper]'); await expect(button).toBeVisible();
  const box = await button.boundingBox(); expect(box.height).toBeGreaterThanOrEqual(44); expect(box.x + box.width).toBeLessThanOrEqual(390);
  await page.route('**/api/admin/transcriptions/' + id, route => route.fulfill({ status: 503, contentType: 'application/json', body: JSON.stringify({ ok: false, error: 'Lyrics worker is not configured.' }) }));
  await button.click(); await expect(page.locator('#msg')).toContainText('not configured');
  await expect(page.locator('#whisperBtn')).toBeEnabled();
 });
 for (const width of [390, 1440]) {
  test(`grouped cue keeps one time through text editing and save at ${width}px`, async ({ page }) => {
    await page.setViewportSize({ width, height: 900 });
    const state = await fixture(page);
    await page.locator('[data-open="' + id + '"]').first().click();
    const input = page.locator('.ln-text').first();
    await input.fill('Because You are God'); await input.press('End');
    if (width === 390) await page.locator('.ln').first().locator('[data-act="break"]').click();
    else await input.press('Shift+Enter');
    await input.press('ControlOrMeta+End'); await input.pressSequentially('You can do anything');
    await expect(input).toHaveValue('Because You are God\nYou can do anything');
    await expect(page.locator('.ln')).toHaveCount(47);
    await page.locator('#bTools').click(); await page.locator('#rawBtn').click();
    await expect(page.locator('#raw')).toContainText('');
    expect(await page.locator('#raw').inputValue()).toContain('| You can do anything');
    await page.locator('#rawBtn').click();
    await expect(page.locator('.ln-text').first()).toHaveValue('Because You are God\nYou can do anything');
    await page.locator('#save').click(); await expect.poll(() => state.saves.length).toBe(1);
    expect(state.saves[0].data.lines[0]).toEqual({ t: 0, text: 'Because You are God\nYou can do anything', kind: 'line' });
  });
 }
 const { openClassic } = require('./helpers/classic-fixture');
 for (const width of [390, 1440]) {
  test(`service mode highlights the whole grouped cue with a tight internal gap at ${width}px`, async ({ page }) => {
    await page.setViewportSize({ width, height: 900 });
    await openClassic(page);
    await page.route('**/api/notes/' + id, route => route.fulfill({ contentType: 'application/json', body: JSON.stringify({ ok: true, lyrics: { rev: 1, data: { lines: [{ t: 0, text: 'Because You are God\nYou can do anything', kind: 'line' }, { t: 30, text: 'Next lyric', kind: 'line' }], tags: [], offset: 0 } } }) }));
    await page.evaluate(async id => {
      current = { meta: { id, title: 'Because You are God' }, qualities: [], audioUrl: '' };
      await Notes.load(current.meta); enterServiceMode(); setServiceView('lyrics'); Notes.tick(0, true);
    }, id);
    const cue = page.locator('#serviceLyricsList .sl-line').first();
    await expect(cue).toHaveClass(/active/); await expect(cue).toHaveText('Because You are God\nYou can do anything');
    await expect(page.locator('#serviceLyricsList .sl-line')).toHaveCount(2);
    await page.waitForTimeout(450);
    const metrics = await cue.evaluate(el => {
      const style = getComputedStyle(el), next = el.nextElementSibling;
      return { space: style.whiteSpace, leading: parseFloat(style.lineHeight) - parseFloat(style.fontSize), gap: next.getBoundingClientRect().top - el.getBoundingClientRect().bottom, height: el.clientHeight, lineHeight: parseFloat(style.lineHeight) };
    });
    expect(metrics.space).toBe('pre-line'); expect(metrics.gap).toBeGreaterThan(metrics.leading);
    expect(metrics.height).toBeGreaterThan(metrics.lineHeight * 1.8);
  });
 }
 for (const width of [320, 390, 820, 1440]) {
  test(`admin analytics exposes storage and collector controls at ${width}px`, async ({ page }) => {
    await page.setViewportSize({ width, height: 900 }); await fixture(page);
    let request = null, cancelled = false;
    await page.route('**/api/admin/analytics', route => route.fulfill({ contentType: 'application/json', body: JSON.stringify({ ok: true, media: [{ status: 'ready', count: 2, bytes: 1048576, seconds: 3600 }], uploads: [{ count: 1, bytes: 1024 }], catalog: { videos: 15, bytes: 3000 }, details: { videos: 4, bytes: 10000 }, thumbnails: { saved: 12, pending: 3, bytes: 1000 }, listening: { plays: 47, videos: 2 }, topPlayed: [{ title: 'Because You are God', plays: 47 }], sources: [{ source: 'search', videos: 15, discoveries: 23 }], volumes: [{ label: 'Video cache', device: '1', total: 1000000000, free: 500000000 }] }) }));
    await page.route('**/api/admin/metadata?**', route => route.fulfill({ contentType: 'application/json', body: JSON.stringify({ ok: true, total: 1, videos: [{ id, title: 'Because You are God', channel: 'Cathedral of Praise Worship', enriched: true, thumbnailBytes: 1024, duration: 240 }] }) }));
    await page.route('**/api/admin/metadata/' + id, route => route.fulfill({ contentType: 'application/json', body: JSON.stringify({ ok: true, metadata: { id, description: 'Stored descriptive metadata', tags: ['worship'] } }) }));
    await page.route('**/api/admin/collections', route => {
      if (route.request().method() === 'POST') request = route.request().postDataJSON();
      return route.fulfill({ contentType: 'application/json', body: JSON.stringify({ ok: true, id: 'job', jobs: request ? [{ id: 'job', query: request.query, maxVideos: request.maxVideos, depth: request.depth, status: cancelled ? 'cancelled' : 'queued', collected: 0, enriched: 0, failed: 0, searches: 0 }] : [] }) });
    });
    await page.route('**/api/admin/collections/job/cancel', route => { cancelled = true; return route.fulfill({ contentType: 'application/json', body: '{"ok":true}' }); });
    await page.locator('[data-tab="analytics"]:visible').click();
    await expect(page.locator('#analyticsMetrics .metric')).toHaveCount(8);
    await expect(page.locator('#analyticsVolumes')).toContainText('available');
    await page.locator('#collectorQuery').fill('Cathedral worship'); await page.locator('#collectorLimit').fill('45'); await page.locator('#collectorDepth').selectOption('2');
    await page.locator('#collectorStart').click();
    await expect(page.locator('#collectorStatus')).toContainText('queued'); expect(request).toEqual({ query: 'Cathedral worship', maxVideos: 45, depth: 2 });
    await page.getByRole('button', { name: 'Stop collection' }).click(); await expect(page.locator('#collectionJobs')).toContainText('cancelled');
    await page.locator('[data-metadata-id]').click(); await expect(page.locator('#metadataDetail')).toContainText('Stored descriptive metadata');
    const overflow = await page.evaluate(() => document.documentElement.scrollWidth > innerWidth + 1); expect(overflow).toBe(false);
    await page.locator('#analyticsRefresh').scrollIntoViewIfNeeded();
    await page.screenshot({ path: `/tmp/ytplayer-admin-analytics-${width}.png`, fullPage: width >= 1000 });
    if (width < 1000) {
      await page.locator('#collectorForm').evaluate(el => el.scrollIntoView({ block: 'center' }));
      await page.screenshot({ path: `/tmp/ytplayer-admin-collector-${width}.png` });
      const reached = await page.locator('#collectorStart').evaluate(el => { const r = el.getBoundingClientRect(); return el.contains(document.elementFromPoint(r.x + r.width / 2, r.y + r.height / 2)); });
      expect(reached).toBe(true);
    }
  });
 }
 test('player lyric editor groups Shift+Enter words and stamps their shared cue', async ({ page }) => {
  await openClassic(page);
  await page.route('**/api/notes/' + id, route => route.fulfill({ contentType: 'application/json', body: JSON.stringify({ ok: true, lyrics: { rev: 1, data: { lines: [{ t: 12, text: 'Because You are God', kind: 'line' }, { t: 30, text: 'Next lyric', kind: 'line' }], tags: [], offset: 0 } } }) }));
  await page.evaluate(async id => { data.profile = { name: 'fixture' }; current = { meta: { id, title: 'Song' } }; await Notes.load(current.meta); document.getElementById('lyricsEditBtn').click(); }, id);
  const text = page.locator('.lyr-editor-text'); await expect(text).toBeVisible();
  await text.evaluate(el => { el.focus(); const pos = el.value.indexOf('\n'); el.setSelectionRange(pos, pos); });
  await text.press('Shift+Enter'); await text.pressSequentially('You can do anything');
  expect(await text.inputValue()).toContain('Because You are God\n| You can do anything');
  await page.locator('.lyr-editor [data-act="stamp"]').click();
  const document = await text.inputValue(); expect(document).toContain('[0:00.00] Because You are God\n| You can do anything');
  expect(document).not.toContain('[0:00.00] |');
 });
Author	SHA1	Message	Date
Jonathan Sykes	258aad83dd	Add piano transcription, interactive key visualizer, and practice queue features	2026-10-03 07:55:33 +08:00
Jonathan Sykes	2b38717c05	Add admin analytics, metadata collection, and grouped lyric cues Queue reviewable Whisper drafts from the song list and lyrics editor. Preserve line breaks within one timed cue across editing, saving, reporting, and service views. Add storage and listening analytics with a durable metadata collector, related-search depth, video limits, thumbnail storage, and a browsable metadata library.	2026-10-03 07:55:33 +08:00
Jonathan Sykes	716b61ffee	Document feature ideas and enhancement roadmap across playback, audio, worship and library	2026-10-03 07:55:33 +08:00