From 89841c3d2af230a256bac354632ec1a31bab6730 Mon Sep 17 00:00:00 2001 From: Brandyn / Techy Date: Sat, 31 Oct 2026 16:00:03 -0500 Subject: [PATCH] =?UTF-8?q?Plan:=20phase=2012=20=E2=80=94=20permanent=20ov?= =?UTF-8?q?erscan=20band=20with=20a=20machine-readable=20server=20clock;?= =?UTF-8?q?=20drift-aware=20decode?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5.1 --- CommunityDiscordMedia.md | 55 ++++++++++++++++++++++------------------ 1 file changed, 30 insertions(+), 25 deletions(-) diff --git a/CommunityDiscordMedia.md b/CommunityDiscordMedia.md index 35dce06..0749e48 100644 --- a/CommunityDiscordMedia.md +++ b/CommunityDiscordMedia.md @@ -287,32 +287,37 @@ Discord roles); the publish path must start with the user's own name. - [ ] dock + README; remove streamers from configure.py; delete `streamers.json` - [ ] test: OBS with an app password; Go Live with no key; a non-streamer gets denied -### Phase 12 — frame-accurate timecode for multicam, invisible in the footage (planned) -Target: clips from several streamers align to the frame (16.7 ms @60) on a multicam timeline, with -**nothing visible in the finals**. Reference clock = the server; streamer PC clocks are never trusted. -Key fact: a streamer's capture→server offset is steady for a whole stream, so it only has to be measured once. +### Phase 12 — frame-accurate timecode for multicam via an overscan band (planned) +Target: clips from several streamers align to the frame (16.7 ms @60) on a multicam timeline; the game +image is untouched and finals carry nothing. Reference clock = the server; streamer PC clocks are never trusted. -- **Sync burst**: for the first ~10 s of every stream, a machine-readable strip (black/white cells: server - time in ms + checksum + framing, ~400×16 px on a 1440p canvas, sized to the OBS canvas) is shown in a - corner, then hidden. OBS: the dock creates a `browser_source` → `/clock/` via obs-websocket and toggles it - on StreamStateChanged (on for 10 s, off). Go Live: painted into the canvas composite for the same window. - The `/clock/` page measures its offset to the server over HTTPS (±few ms) — the strip shows *server* time. -- **Sweep**: when publishing a stream, extract frames from the first 10 s, locate + decode the strip in several - frames, median(decoded − pts) = server-clock capture time of frame 0 (frame-accurate). Write it as a QuickTime - `tmcd` track (time of day, Eastern, file fps) and `creation_time` (UTC) inside the existing lossless remux. - No strip found → fall back to server arrival time (coarse) and log it. Never re-encode, never crop. -- MediaMTX `recordPath` with `%f` (microseconds) for the fallback; plugin `ItemTimestamp` accepts it and prefers - `creation_time`. -- Editors: multicam → sync by timecode (now frame-accurate when the burst was present), shared voice audio as - the cross-check. Finals carry no overlay. -- Streamer opt-out in the dock / Go Live (coarse timecode only). -- [ ] `/clock/` page (server-offset measurement, strip encoder, optional human-readable readout) -- [ ] dock: create/position/toggle the overlay source via obs-websocket; 10 s burst on stream start -- [ ] portal Go Live: canvas composite with the burst for 10 s (toggle) -- [ ] sweep: `bin/timecode-decode.py` (ffmpeg frames → decode → offset), tmcd + creation_time in the remux -- [ ] MediaMTX `%f`; plugin timestamp update; README editor workflow -- rejected: LTC audio (needs SRT multitrack + per-PC generator); permanent visible clock (in the finals); - cropping at remux (would force a re-encode) +Facts that shape it: within a recording, frame timing is the encoder's capture clock (network lag only shifts +*arrival*, which is measured once); but a PC clock drifts vs the server (10 ppm ≈ 4 frames over 2 h), so a +permanent reference is needed for long sessions, not just a start burst. + +- **Overscan band**: the published canvas is the game image plus 16 extra rows at the bottom. The band holds a + machine-readable strip (black/white cells: server time in ms + checksum + framing, cell size scaled to the + canvas) and a small human-readable server clock, **continuously**. + - Go Live: canvas composite = video (W×H) + band → W×(H+16); the `/clock/` logic runs in-page. + - OBS: the dock (obs-websocket) offers "add overscan band": SetVideoSettings canvas/output to H+16 while not + streaming, CreateInput `browser_source` → `/clock/` positioned in the band, toggle/remove. The browser + source shares the dock's cookies (Authentik). Opt-out = coarse timecode only. + - `/clock/` measures its offset to the server clock over HTTPS (±few ms) and shows **server** time. +- **Sweep**: when publishing a stream: extract frames at the start, middle and end; locate + decode the strip; + frame-0 capture time = median(decoded − pts) at the start (frame-accurate) → QuickTime `tmcd` track + (time of day, Eastern, file fps) + `creation_time` (UTC) in the existing lossless remux; a `.tc.json` + sidecar with the samples records drift over the file. No strip → server arrival time (coarse), logged. + Never re-encode, never crop. +- **Viewing**: live portal crops the band with CSS; Jellyfin shows a thin strip at the bottom edge (VHS-style). + Editors export at the game resolution (2560×1440), which drops the band. +- MediaMTX `recordPath` with `%f` (microseconds) for the fallback; plugin `ItemTimestamp` prefers `creation_time`. +- [ ] `/clock/` page: offset measurement, strip encoder, readable clock, band layout +- [ ] dock: overscan band setup + browser source via obs-websocket; opt-out +- [ ] portal Go Live: band composite (toggle); portal viewer: CSS crop of the band +- [ ] sweep: `bin/timecode-decode.py` (ffmpeg frames → decode → offset + drift), tmcd + creation_time, `.tc.json` +- [ ] MediaMTX `%f`; plugin timestamp update; README editor workflow (multicam: timecode → audio cross-check; + export at 2560×1440) +- rejected: LTC audio (SRT multitrack + per-PC generator); clock inside the picture; cropping at remux (re-encode) ### Option — more clip sources (Medal, Twitch/YouTube/Kick clips, Discord uploads)