Plan: phase 12 — permanent overscan band with a machine-readable server clock; drift-aware decode

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-10-31 16:00:03 -05:00
parent f1bd28f8d1
commit 89841c3d2a

View File

@@ -287,32 +287,37 @@ Discord roles); the publish path must start with the user's own name.
- [ ] dock + README; remove streamers from configure.py; delete `streamers.json` - [ ] dock + README; remove streamers from configure.py; delete `streamers.json`
- [ ] test: OBS with an app password; Go Live with no key; a non-streamer gets denied - [ ] test: OBS with an app password; Go Live with no key; a non-streamer gets denied
### Phase 12 — frame-accurate timecode for multicam, invisible in the footage (planned) ### Phase 12 — frame-accurate timecode for multicam via an overscan band (planned)
Target: clips from several streamers align to the frame (16.7 ms @60) on a multicam timeline, with Target: clips from several streamers align to the frame (16.7 ms @60) on a multicam timeline; the game
**nothing visible in the finals**. Reference clock = the server; streamer PC clocks are never trusted. image is untouched and finals carry nothing. Reference clock = the server; streamer PC clocks are never trusted.
Key fact: a streamer's capture→server offset is steady for a whole stream, so it only has to be measured once.
- **Sync burst**: for the first ~10 s of every stream, a machine-readable strip (black/white cells: server Facts that shape it: within a recording, frame timing is the encoder's capture clock (network lag only shifts
time in ms + checksum + framing, ~400×16 px on a 1440p canvas, sized to the OBS canvas) is shown in a *arrival*, which is measured once); but a PC clock drifts vs the server (10 ppm ≈ 4 frames over 2 h), so a
corner, then hidden. OBS: the dock creates a `browser_source` → `/clock/` via obs-websocket and toggles it permanent reference is needed for long sessions, not just a start burst.
on StreamStateChanged (on for 10 s, off). Go Live: painted into the canvas composite for the same window.
The `/clock/` page measures its offset to the server over HTTPS (±few ms) — the strip shows *server* time. - **Overscan band**: the published canvas is the game image plus 16 extra rows at the bottom. The band holds a
- **Sweep**: when publishing a stream, extract frames from the first 10 s, locate + decode the strip in several machine-readable strip (black/white cells: server time in ms + checksum + framing, cell size scaled to the
frames, median(decoded − pts) = server-clock capture time of frame 0 (frame-accurate). Write it as a QuickTime canvas) and a small human-readable server clock, **continuously**.
`tmcd` track (time of day, Eastern, file fps) and `creation_time` (UTC) inside the existing lossless remux. - Go Live: canvas composite = video (W×H) + band → W×(H+16); the `/clock/` logic runs in-page.
No strip found → fall back to server arrival time (coarse) and log it. Never re-encode, never crop. - OBS: the dock (obs-websocket) offers "add overscan band": SetVideoSettings canvas/output to H+16 while not
- MediaMTX `recordPath` with `%f` (microseconds) for the fallback; plugin `ItemTimestamp` accepts it and prefers streaming, CreateInput `browser_source` → `/clock/` positioned in the band, toggle/remove. The browser
`creation_time`. source shares the dock's cookies (Authentik). Opt-out = coarse timecode only.
- Editors: multicam → sync by timecode (now frame-accurate when the burst was present), shared voice audio as - `/clock/` measures its offset to the server clock over HTTPS (±few ms) and shows **server** time.
the cross-check. Finals carry no overlay. - **Sweep**: when publishing a stream: extract frames at the start, middle and end; locate + decode the strip;
- Streamer opt-out in the dock / Go Live (coarse timecode only). frame-0 capture time = median(decoded − pts) at the start (frame-accurate) → QuickTime `tmcd` track
- [ ] `/clock/` page (server-offset measurement, strip encoder, optional human-readable readout) (time of day, Eastern, file fps) + `creation_time` (UTC) in the existing lossless remux; a `.tc.json`
- [ ] dock: create/position/toggle the overlay source via obs-websocket; 10 s burst on stream start sidecar with the samples records drift over the file. No strip → server arrival time (coarse), logged.
- [ ] portal Go Live: canvas composite with the burst for 10 s (toggle) Never re-encode, never crop.
- [ ] sweep: `bin/timecode-decode.py` (ffmpeg frames → decode → offset), tmcd + creation_time in the remux - **Viewing**: live portal crops the band with CSS; Jellyfin shows a thin strip at the bottom edge (VHS-style).
- [ ] MediaMTX `%f`; plugin timestamp update; README editor workflow Editors export at the game resolution (2560×1440), which drops the band.
- rejected: LTC audio (needs SRT multitrack + per-PC generator); permanent visible clock (in the finals); - MediaMTX `recordPath` with `%f` (microseconds) for the fallback; plugin `ItemTimestamp` prefers `creation_time`.
cropping at remux (would force a re-encode) - [ ] `/clock/` page: offset measurement, strip encoder, readable clock, band layout
- [ ] dock: overscan band setup + browser source via obs-websocket; opt-out
- [ ] portal Go Live: band composite (toggle); portal viewer: CSS crop of the band
- [ ] sweep: `bin/timecode-decode.py` (ffmpeg frames → decode → offset + drift), tmcd + creation_time, `.tc.json`
- [ ] MediaMTX `%f`; plugin timestamp update; README editor workflow (multicam: timecode → audio cross-check;
export at 2560×1440)
- rejected: LTC audio (SRT multitrack + per-PC generator); clock inside the picture; cropping at remux (re-encode)
### Option — more clip sources (Medal, Twitch/YouTube/Kick clips, Discord uploads) ### Option — more clip sources (Medal, Twitch/YouTube/Kick clips, Discord uploads)