2b38717c05
Queue reviewable Whisper drafts from the song list and lyrics editor. Preserve line breaks within one timed cue across editing, saving, reporting, and service views. Add storage and listening analytics with a durable metadata collector, related-search depth, video limits, thumbnail storage, and a browsable metadata library.
123 lines
6.1 KiB
Markdown
123 lines
6.1 KiB
Markdown
# Lyrics and server analytics
|
||
|
||
## Whisper drafts
|
||
|
||
In `/admin#editor`, every song has a Whisper button alongside its Open button.
|
||
The loaded song also has **Transcribe with Whisper**. Both enqueue the same
|
||
server job; duplicate active requests reuse that job. The existing CPU
|
||
faster-whisper container polls explicit requests every ten seconds and returns
|
||
a timed draft even when that song already has lyrics. Manual requests always
|
||
use Whisper, bypassing LRCLIB.
|
||
|
||
**Use transcript in editor** loads the draft for review. **Save lyrics** publishes
|
||
it through the existing revision history and conflict checks. Until saved,
|
||
shared lyrics remain unchanged. Applying a draft can be undone, including the
|
||
original sync offset. Queued/running jobs survive restarts; worker leases expire
|
||
after two minutes and heartbeat every 25 seconds. After three failed worker
|
||
attempts the job reports a failure. Completed/failed drafts are retained for
|
||
30 days; cleanup runs when workers report progress.
|
||
|
||
A YouTube video must already have a ready server media copy; uploads are also
|
||
supported. The existing maximum of one hour and 40 MiB audio input applies.
|
||
Songs without enough recognized vocals fail visibly. Transcription accuracy
|
||
and word timing depend on the singing and audio mix; review before saving.
|
||
|
||
Deploy the updated server and rebuild the `lyrics-worker` image. Both need the
|
||
same `LYRICS_WORKER_TOKEN` (at least 24 characters; compose passes it to the
|
||
worker as `YTP_TOKEN`). A configured token permits queuing; the status endpoint
|
||
also reports whether a worker has recently polled. No paid transcription API
|
||
is involved.
|
||
|
||
Admin endpoints (admin cookie or API token):
|
||
|
||
- `POST /api/admin/transcriptions/:video` queues a draft.
|
||
- `GET /api/admin/transcriptions/:video` returns the latest draft/progress.
|
||
|
||
Worker-only endpoints require the worker token:
|
||
|
||
- `POST /api/lyrics-worker/claim` claims one job.
|
||
- `POST /api/lyrics-worker/jobs/:job` reports progress/results with its lease.
|
||
|
||
## Grouped lyric lines
|
||
|
||
One lyric cue can contain multiple visual lines with one timestamp. In the
|
||
admin editor, **Shift+Enter** inserts a tight line break. On phones, select the
|
||
cue and use **Line break** in its options. Ordinary Enter or **Below** adds a
|
||
separate cue with its own timing. Both visual lines of a grouped cue highlight
|
||
and seek together in service mode, with a smaller internal gap than the space
|
||
between cues. The player panel, stage and presenter views also preserve breaks.
|
||
|
||
Text editing/export uses a continuation prefix to preserve the grouping:
|
||
|
||
```text
|
||
[0:12.00] Because You are God
|
||
| You can do anything
|
||
[0:18.00] Another separate cue
|
||
```
|
||
|
||
The continuation belongs to the preceding cue. JSON stores its text as
|
||
`"Because You are God\nYou can do anything"`. Saving, revisions and text
|
||
round trips retain the single timestamp. Reporting a wrong lyric also preserves the grouped text. Existing single-line lyrics work
|
||
unchanged.
|
||
|
||
## Server analytics and metadata collector
|
||
|
||
Open **Stats** (`/admin#analytics`). It shows cached video and upload totals,
|
||
recorded plays, discovery sources, metadata/thumbnail payload sizes, and
|
||
available filesystem space for media, uploads and the database. Paths on the
|
||
same device share their free space; their capacities must not be added.
|
||
Media totals come from library records and exclude temporary files and
|
||
filesystem overhead. Metadata bytes exclude indexes and SQLite overhead.
|
||
|
||
The collector accepts a search, a total unique-video limit (1–500), and depth:
|
||
|
||
- **0:** only the specified search.
|
||
- **1:** also search channels and tags found in its results.
|
||
- **2/3:** follow up to two/three related rounds.
|
||
|
||
The total limit applies across every round. The collector reserves result
|
||
budget for deeper rounds, deduplicates video IDs and related queries, and stops
|
||
after at most 24 searches. Duplicate, unavailable or empty results can produce
|
||
fewer videos than requested. Up to three collections may be queued/running;
|
||
one executes at a time per server. Atomic ownership leases prevent two
|
||
servers from claiming the same job. Progress is saved after each discovery
|
||
batch and video. An expired interrupted job resumes its pending items.
|
||
**Stop collection** cancels future work; an extraction already in progress
|
||
can finish its network request before the cancellation is observed.
|
||
|
||
Discovery cards enter the same catalog used for recommendations. Each video
|
||
is then enriched through yt-dlp without downloading media. The server stores
|
||
full descriptive fields (including descriptions, dates, engagement counts,
|
||
language, tags, chapters, thumbnail variants and format specifications) in
|
||
`video_details`. Expiring media URLs and request headers are excluded. Details
|
||
are capped at 500 KB per video and follow catalog eviction. Thumbnail image
|
||
bytes use the existing durable thumbnail queue, host restrictions and budget
|
||
(default 512 MiB). Failures stay visible in job history; successful cards remain
|
||
stored even when enrichment fails. Refresh analytics to update the aggregate
|
||
numbers after collection; job progress polls automatically while this tab is
|
||
open.
|
||
|
||
The metadata library supports title/channel/tag filtering, 50-row pages and a
|
||
raw descriptive metadata view. Admin cookie or API token is required for:
|
||
|
||
- `GET /api/admin/analytics`
|
||
- `GET /api/admin/metadata?q=...&offset=...`
|
||
- `GET /api/admin/metadata/:id`
|
||
- `GET /api/admin/collections`
|
||
- `POST /api/admin/collections` with `{query,maxVideos,depth}`
|
||
- `POST /api/admin/collections/:id/cancel`
|
||
|
||
## Verification
|
||
|
||
Run `bun run test` in `server`, `npm test` at the repo root,
|
||
`python3 -m unittest discover -s scripts/lyrics -p test_auto_lyrics.py`, and
|
||
`npx playwright test -c playwright.admin.config.js`. Admin browser fixtures
|
||
cover Whisper review/save/undo, unsaved edits, failures, mobile actions,
|
||
grouped cues and service highlighting, and analytics collector controls.
|
||
|
||
Design review fixed the primary action’s white-on-gradient contrast by using a
|
||
solid purple fill, and fixed tablet navigation overflow. No new findings were
|
||
suppressed. Existing admin play/tap gradient contrast and decorative glow
|
||
findings remain outside these controls. Real viewport captures passed the
|
||
mobile dashboard/collector and desktop visual checks.
|