Plan: phase 12 redesigned as a 10 s machine-readable sync burst decoded by the sweep into the timecode track; nothing visible in finals

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-10-31 15:58:20 -05:00
parent 4419cc9d67
commit f1bd28f8d1

View File

@@ -287,27 +287,32 @@ Discord roles); the publish path must start with the user's own name.
- [ ] dock + README; remove streamers from configure.py; delete `streamers.json`
- [ ] test: OBS with an app password; Go Live with no key; a non-streamer gets denied
### Phase 12 — timecode for multicam editing (frame accurate) (planned)
Target: clips from several streamers align to the frame (16.7 ms @60) on a multicam timeline.
**Reference clock = the server** (NTP-disciplined). Recording start, the `tmcd` track and the burned-in clock
all derive from it; streamer PC clocks are never trusted.
Server-side stamping alone cannot do that (encoder + network delay differs per streamer by 0.1–0.3 s),
so: a real timecode track for coarse placement + a capture-side reference for the frame.
- [ ] MediaMTX `recordPath` with `%f` (microseconds) — paths section, hot-reloaded
- [ ] sweep: on publishing a stream, write a QuickTime `tmcd` timecode track (start = Eastern time of day of
the first frame, file fps) and `creation_time` (UTC) metadata, inside the existing lossless remux
- [ ] plugin `ItemTimestamp`: accept the microsecond filename; prefer `creation_time` when present
- [ ] `/clock/` page on the main host (behind Authentik): HTTPS offset measurement to the server clock
(±few ms), 60 fps time-of-day readout with ms + once-per-second flash — add as an OBS browser source
for burned-in timecode (BITC) when shared audio isn't available; optional per streamer
- [ ] dock: manage the OBS clock overlay via obs-websocket — CreateInput (browser_source → `/clock/`, sized for
the corner), SetSceneItemTransform for the chosen corner, SetSceneItemEnabled toggle, "add to this scene";
the browser source shares the dock's cookies so the Authentik login carries over
- [ ] live portal Go Live: composite the captured window through a canvas and paint the server-synced clock
into a corner on every frame before publishing (toggle, default on) — burned-in clock on the WebRTC path
- [ ] README: editor workflow — multicam: sync by timecode (coarse), then audio waveform (sample-accurate,
everyone is in the same voice chat) or the burned-in clock
- rejected: LTC audio timecode per PC (needs SRT multitrack + a generator app on every machine)
### Phase 12 — frame-accurate timecode for multicam, invisible in the footage (planned)
Target: clips from several streamers align to the frame (16.7 ms @60) on a multicam timeline, with
**nothing visible in the finals**. Reference clock = the server; streamer PC clocks are never trusted.
Key fact: a streamer's capture→server offset is steady for a whole stream, so it only has to be measured once.
- **Sync burst**: for the first ~10 s of every stream, a machine-readable strip (black/white cells: server
time in ms + checksum + framing, ~400×16 px on a 1440p canvas, sized to the OBS canvas) is shown in a
corner, then hidden. OBS: the dock creates a `browser_source` → `/clock/` via obs-websocket and toggles it
on StreamStateChanged (on for 10 s, off). Go Live: painted into the canvas composite for the same window.
The `/clock/` page measures its offset to the server over HTTPS (±few ms) — the strip shows *server* time.
- **Sweep**: when publishing a stream, extract frames from the first 10 s, locate + decode the strip in several
frames, median(decoded − pts) = server-clock capture time of frame 0 (frame-accurate). Write it as a QuickTime
`tmcd` track (time of day, Eastern, file fps) and `creation_time` (UTC) inside the existing lossless remux.
No strip found → fall back to server arrival time (coarse) and log it. Never re-encode, never crop.
- MediaMTX `recordPath` with `%f` (microseconds) for the fallback; plugin `ItemTimestamp` accepts it and prefers
`creation_time`.
- Editors: multicam → sync by timecode (now frame-accurate when the burst was present), shared voice audio as
the cross-check. Finals carry no overlay.
- Streamer opt-out in the dock / Go Live (coarse timecode only).
- [ ] `/clock/` page (server-offset measurement, strip encoder, optional human-readable readout)
- [ ] dock: create/position/toggle the overlay source via obs-websocket; 10 s burst on stream start
- [ ] portal Go Live: canvas composite with the burst for 10 s (toggle)
- [ ] sweep: `bin/timecode-decode.py` (ffmpeg frames → decode → offset), tmcd + creation_time in the remux
- [ ] MediaMTX `%f`; plugin timestamp update; README editor workflow
- rejected: LTC audio (needs SRT multitrack + per-PC generator); permanent visible clock (in the finals);
cropping at remux (would force a re-encode)
### Option — more clip sources (Medal, Twitch/YouTube/Kick clips, Discord uploads)