diff --git a/docker-compose.yml b/docker-compose.yml index 6089f9b..d723f5c 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -79,7 +79,7 @@ services: - "traefik.http.routers.ytplayer-https.tls.certresolver=letsencrypt" - "traefik.http.services.ytplayer-svc.loadbalancer.server.port=3000" - # Transcribes saved songs that have no lyrics yet, one at a time, with + # Handles requested Whisper drafts and transcribes saved songs without lyrics, with # faster-whisper on CPU (no API keys, no credits). Separate container so it # never competes with playback/downloads: capped CPU + memory, low priority, # and it fetches audio from the ytplayer service over the private network. diff --git a/docs/admin-lyrics-and-analytics.md b/docs/admin-lyrics-and-analytics.md new file mode 100644 index 0000000..913cc04 --- /dev/null +++ b/docs/admin-lyrics-and-analytics.md @@ -0,0 +1,122 @@ +# Lyrics and server analytics + +## Whisper drafts + +In `/admin#editor`, every song has a Whisper button alongside its Open button. +The loaded song also has **Transcribe with Whisper**. Both enqueue the same +server job; duplicate active requests reuse that job. The existing CPU +faster-whisper container polls explicit requests every ten seconds and returns +a timed draft even when that song already has lyrics. Manual requests always +use Whisper, bypassing LRCLIB. + +**Use transcript in editor** loads the draft for review. **Save lyrics** publishes +it through the existing revision history and conflict checks. Until saved, +shared lyrics remain unchanged. Applying a draft can be undone, including the +original sync offset. Queued/running jobs survive restarts; worker leases expire +after two minutes and heartbeat every 25 seconds. After three failed worker +attempts the job reports a failure. Completed/failed drafts are retained for +30 days; cleanup runs when workers report progress. + +A YouTube video must already have a ready server media copy; uploads are also +supported. The existing maximum of one hour and 40 MiB audio input applies. +Songs without enough recognized vocals fail visibly. Transcription accuracy +and word timing depend on the singing and audio mix; review before saving. + +Deploy the updated server and rebuild the `lyrics-worker` image. Both need the +same `LYRICS_WORKER_TOKEN` (at least 24 characters; compose passes it to the +worker as `YTP_TOKEN`). A configured token permits queuing; the status endpoint +also reports whether a worker has recently polled. No paid transcription API +is involved. + +Admin endpoints (admin cookie or API token): + +- `POST /api/admin/transcriptions/:video` queues a draft. +- `GET /api/admin/transcriptions/:video` returns the latest draft/progress. + +Worker-only endpoints require the worker token: + +- `POST /api/lyrics-worker/claim` claims one job. +- `POST /api/lyrics-worker/jobs/:job` reports progress/results with its lease. + +## Grouped lyric lines + +One lyric cue can contain multiple visual lines with one timestamp. In the +admin editor, **Shift+Enter** inserts a tight line break. On phones, select the +cue and use **Line break** in its options. Ordinary Enter or **Below** adds a +separate cue with its own timing. Both visual lines of a grouped cue highlight +and seek together in service mode, with a smaller internal gap than the space +between cues. The player panel, stage and presenter views also preserve breaks. + +Text editing/export uses a continuation prefix to preserve the grouping: + +```text +[0:12.00] Because You are God +| You can do anything +[0:18.00] Another separate cue +``` + +The continuation belongs to the preceding cue. JSON stores its text as +`"Because You are God\nYou can do anything"`. Saving, revisions and text +round trips retain the single timestamp. Reporting a wrong lyric also preserves the grouped text. Existing single-line lyrics work +unchanged. + +## Server analytics and metadata collector + +Open **Stats** (`/admin#analytics`). It shows cached video and upload totals, +recorded plays, discovery sources, metadata/thumbnail payload sizes, and +available filesystem space for media, uploads and the database. Paths on the +same device share their free space; their capacities must not be added. +Media totals come from library records and exclude temporary files and +filesystem overhead. Metadata bytes exclude indexes and SQLite overhead. + +The collector accepts a search, a total unique-video limit (1–500), and depth: + +- **0:** only the specified search. +- **1:** also search channels and tags found in its results. +- **2/3:** follow up to two/three related rounds. + +The total limit applies across every round. The collector reserves result +budget for deeper rounds, deduplicates video IDs and related queries, and stops +after at most 24 searches. Duplicate, unavailable or empty results can produce +fewer videos than requested. Up to three collections may be queued/running; +one executes at a time per server. Atomic ownership leases prevent two +servers from claiming the same job. Progress is saved after each discovery +batch and video. An expired interrupted job resumes its pending items. +**Stop collection** cancels future work; an extraction already in progress +can finish its network request before the cancellation is observed. + +Discovery cards enter the same catalog used for recommendations. Each video +is then enriched through yt-dlp without downloading media. The server stores +full descriptive fields (including descriptions, dates, engagement counts, +language, tags, chapters, thumbnail variants and format specifications) in +`video_details`. Expiring media URLs and request headers are excluded. Details +are capped at 500 KB per video and follow catalog eviction. Thumbnail image +bytes use the existing durable thumbnail queue, host restrictions and budget +(default 512 MiB). Failures stay visible in job history; successful cards remain +stored even when enrichment fails. Refresh analytics to update the aggregate +numbers after collection; job progress polls automatically while this tab is +open. + +The metadata library supports title/channel/tag filtering, 50-row pages and a +raw descriptive metadata view. Admin cookie or API token is required for: + +- `GET /api/admin/analytics` +- `GET /api/admin/metadata?q=...&offset=...` +- `GET /api/admin/metadata/:id` +- `GET /api/admin/collections` +- `POST /api/admin/collections` with `{query,maxVideos,depth}` +- `POST /api/admin/collections/:id/cancel` + +## Verification + +Run `bun run test` in `server`, `npm test` at the repo root, +`python3 -m unittest discover -s scripts/lyrics -p test_auto_lyrics.py`, and +`npx playwright test -c playwright.admin.config.js`. Admin browser fixtures +cover Whisper review/save/undo, unsaved edits, failures, mobile actions, +grouped cues and service highlighting, and analytics collector controls. + +Design review fixed the primary action’s white-on-gradient contrast by using a +solid purple fill, and fixed tablet navigation overflow. No new findings were +suppressed. Existing admin play/tap gradient contrast and decorative glow +findings remain outside these controls. Real viewport captures passed the +mobile dashboard/collector and desktop visual checks. diff --git a/frontend/admin.html b/frontend/admin.html index 3b29b92..5c7eba0 100644 --- a/frontend/admin.html +++ b/frontend/admin.html @@ -92,7 +92,7 @@ } .btn:hover { background: rgba(255,255,255,.1); } .btn:active { transform: scale(.97); } - .btn.pri { border: 0; background: var(--grad); color: #fff; box-shadow: 0 10px 26px -10px var(--a1); } + .btn.pri { border: 0; background: #6143cf; color: #fff; box-shadow: none; } .btn.pri:hover { filter: brightness(1.1); } .btn.ok { border: 0; background: var(--ok); color: #06210f; } .btn.dng { color: var(--bad); border-color: rgba(255,107,107,.35); background: rgba(255,107,107,.07); } @@ -164,6 +164,13 @@ .song-row small { color: var(--text-dim); font-size: 12px; } .song-row .st { flex: none; font: 700 11px var(--ui); padding: 3px 9px; border-radius: 99px; border: 1px solid var(--line-2); color: var(--text-dim); } .song-row .st.has { color: var(--ok); border-color: rgba(74,222,128,.4); } + .song-open { display: flex; align-items: center; gap: 10px; min-width: 0; flex: 1; padding: 0; border: 0; background: none; color: inherit; text-align: left; } + .whisper-request { min-height: 44px; flex: none; } + .whisper-draft { margin-block: 14px; } + .whisper-draft pre { white-space: pre-wrap; overflow-wrap: anywhere; max-height: 180px; overflow: auto; font: 400 14px/1.55 var(--ui); color: var(--text); } + #whisperStatus { color: var(--text-dim); font-size: 14px; margin-block: 10px; } + .song { flex-wrap: wrap; } + @media (max-width: 480px) { .song-row { flex-wrap: wrap; } .song-open { flex-basis: 100%; min-height: 44px; } } .chips { display: flex; flex-wrap: wrap; gap: 8px; margin-top: 12px; } .chip { height: 34px; padding: 0 13px; border-radius: 99px; border: 1px solid var(--line-2); background: rgba(255,255,255,.04); font-size: 13px; font-weight: 600; color: var(--text-2); max-width: 100%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } .chip:hover { color: var(--text); border-color: var(--a1); } @@ -211,7 +218,7 @@ } .tc:hover { border-color: var(--a2); color: var(--text); } .ln.untimed .tc { color: var(--warn); border-style: dashed; border-color: rgba(255,180,84,.55); } - .ln-text { flex: 1 1 auto; min-width: 0; height: 38px; padding: 0 8px; border: 0; border-radius: 9px; background: transparent; font-size: 16px; } + .ln-text { flex: 1 1 auto; min-width: 0; min-height: 38px; height: auto; padding: 7px 8px; resize: vertical; overflow: hidden; line-height: 1.3; border: 0; border-radius: 9px; background: transparent; font-size: 16px; } .ln.section .ln-text { font-weight: 800; text-transform: uppercase; letter-spacing: .06em; font-size: 13px; color: var(--a2); } .ln.cue .ln-text { font-style: italic; color: var(--warn); } .ln.now { background: rgba(124,92,255,.16); border-color: rgba(124,92,255,.55); box-shadow: 0 0 0 1px rgba(124,92,255,.25), 0 10px 30px -16px var(--a1); } @@ -257,6 +264,26 @@ .tap small { display: block; font-weight: 600; opacity: .8; font-size: 11px; letter-spacing: 0; max-width: 84px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } .tap:active { transform: scale(.93); } + .analytics-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); gap: 12px; margin: 20px 0; } + .metric { padding: 18px; background: var(--bg-2); border: 1px solid var(--line-2); border-radius: 14px; } + .metric b { display: block; font-size: 26px; margin: 4px 0; } + .metric small, .metric span { color: var(--text-2); } + .analytics-form { display: flex; flex-wrap: wrap; align-items: end; gap: 12px; } + .analytics-form label { display: grid; gap: 5px; min-width: 0; } + .analytics-form .query { flex: 1 1 250px; } + .analytics-form .field { width: 100%; min-width: 0; } + .analytics-form button { min-height: 44px; } + .analytics-table { overflow-x: auto; margin-top: 12px; } + .analytics-table table { width: 100%; } + .collection-job { padding: 14px 0; border-bottom: 1px solid var(--line-2); overflow-wrap: anywhere; } + .collection-job p { margin: 4px 0; color: var(--text-2); } + #metadataDetail { white-space: pre-wrap; overflow-wrap: anywhere; max-height: 400px; overflow: auto; font-size: 13px; } + @media (max-width: 480px) { .analytics-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } .metric { padding: 12px; } .metric b { font-size: 22px; } } + @media (min-width: 761px) and (max-width: 1100px) { + .top { flex-wrap: wrap; } + .tabs { order: 2; width: 100%; margin-left: 0; overflow-x: auto; } + .tabs .tab { flex: none; } + } .nav { display: none; } @media (max-width: 760px) { .tabs, .top .ghost.desk { display: none; } @@ -347,6 +374,7 @@
Space stamp (Tap-sync) · S stamp selected · K play · ↑↓ select · ⌘/Ctrl S save
@@ -480,6 +515,35 @@
+ Video storage, listening activity and the metadata library.
+ + + +Volumes on the same device share the reported free space. Stored media totals exclude temporary files and filesystem overhead.
Search and save descriptive video metadata and thumbnails. Video and audio files are not downloaded.
+ +The limit applies across every round. Depth follows channels and tags found in video metadata, with at most 24 searches. A collection can return fewer videos when results repeat or sources are unavailable. Thumbnails follow the server’s storage budget.
+ +Tokens let scripts write shared lyrics and chapters. A token is shown once — store it in your secrets vault.
@@ -505,6 +569,7 @@