Media-Server-Setup/STATUS.md

Vanguard stack — what runs, what people do, what's unfinished

Living document. Updated whenever something is deployed or changes behaviour. Last update: 2026-11-01. Box: OVH SYS-1, Debian 13, ssh ovh1.dumpstervigilantes.rip. Web hosts (one Discord login): media. gateway at /, files. copyparty, clips., sync., live. (portal + /dock/, /dump, /time), watch., git., bot., auth. — all under dumpstervigilantes.rip. Restructure deployed 2026-11-01 (ovh1 is ssh-only now). Bundle: ~/vanguard-stream on the box. Source of truth: this repo, techy/Media-Server-Setup on the stack's own Gitea (ssh://git@git.dumpstervigilantes.rip:2222/techy/Media-Server-Setup.git).

Runs on its own

Cadence	What happens	Done by
Continuous	OBS streams record into a hidden streams/<user>/<Title>/.live/ folder, invisible to Jellyfin and copyparty until finished	MediaMTX
Continuous	Gamers' clip folders mirror onto the server (video files only, their structure untouched, receive-only)	Syncthing
Continuous	New recordings and clips are announced in a Discord channel with the streamer's name and a frame grab	Jellyfin plugin (needs bot token + channel in its config)
Every 5 min	Finished files are checked. Streams idle ≥3 min: timecode band decoded (server clock) or arrival time used, then published out of <Title>/.live/ to <user>/<Title>__<ts>.mp4 with a frame-accurate tmcd track + sidecar — band present: re-encode with the band cropped off (hours of CPU per night, niced); no band: lossless remux — one file per recording (this is when they appear in Jellyfin/copyparty and the Discord notice fires). Non-compliant clips: converted to the editing standard, raw removed, one copy per clip	normalize sweep (systemd timer)
Every 5 min	Discord scheduled events snapshotted into a history so recordings can be matched to their session	Jellyfin plugin
Every 30 min + startup	Every non-admin Jellyfin user set to direct-play-only (no hardware encoder on this box)	Jellyfin plugin
Daily 04:10 ET	Recordings roll between weekday → week → month → year seasons as the calendar moves; no files change	Jellyfin plugin
Every 5 min	Each user's clips folder is measured against their quota (100 GB default, per-user overrides); over quota → their Syncthing folder is paused, under 90 % → resumed; the clips page shows usage	dumpd quota loop — pending deploy
Daily 05:00 ET	Dumped nights older than 3 days deleted for good	trash purge (systemd timer)
On boot	Everything restarts itself after the media mount is up; interrupted recordings stay playable	systemd + Docker
Every 5 min	If a deploy deferred a MediaMTX restart, retry when no stream is live	deploy-pending.timer — pending first apply
On push to main	(OFF: DEPLOY_AUTO=0) Gitea webhook → dumpd → vanguard-deploy pull applies only what changed	built, disabled
Continuous	Active streams are watchable live (WebRTC, HLS fallback) by Discord-server members; optional "is live" Discord post when a stream starts	MediaMTX + live portal (page deployed; MediaMTX port fix pending apply)
Not yet running	Pull of files older than 15 days to the home HDD	home/rclone-archive.sh, needs the home box
Not yet running	Nightly backup of the Authentik database (Postgres)	planned

What people do

Who	Activity	Where	Login
Streamer	One-time: dock → Timecode band → "Add band + clock source" (OBS) or leave the Go Live checkbox on — gives editors frame-accurate timecode	OBS dock / Go Live	live
Streamer	Type a title (live preview + naming rules), Go Live, Stop. "Dump tonight" if it was junk (3-day undo). Manual-setup accordion with endpoints and a hidden token	OBS browser dock https://live.…/dock/	Discord login in the dock; dock mints a 12 h token per Go Live — no passwords, verified live
Gamer	Self-serve pairing (paste Syncthing Device ID → folder + share created); then nothing, clips save where they always did	https://clips.…/ + Syncthing	Discord (any server member)
Viewer	Live portal lays streams out as an even grid automatically when one joins or leaves ("Arrange" button redoes it after manual moves); audio follows the focused window; double-click a tab to rename a stream (kept in that browser), double-click the slot at the left of a tab bar to name the group (session); once the viewer has moved/resized/renamed anything the layout is left alone: new streams chime and wait as draggable chips in a tray, "Arrange" re-grids and returns to auto	https://live.…/	Discord
Everyone	Landing page with role-aware links and instructions	https://media.…/	Discord
Editor	Multicam: drop clips on a timeline, sync by the embedded timecode track (frame-accurate when the band was on), cross-check with voice audio; files are already at game resolution (band removed by the sweep)	Resolve / Premiere	—
Editor	Browse by streamer/title, scrub in browser, download originals (zip/tar for folders), upload finals	copyparty https://files.…/	Discord (Authentik); rights by group editors / media-admins
Anyone in the Discord server	Watch recordings and clips by streamer and season	Jellyfin https://watch.…/	Discord (Authentik)
Anyone in the Discord server	Watch live: all active streams in dockable windows, audio follows focus, per-window mute/volume	Live portal https://live.…/	Discord (Authentik, domain-level) — live, shows a real stream
Streamer (Windows)	Go Live from the browser: pick the WarDogs window (+audio on Windows Chrome 141+; Linux/Mac video only), streams as <user>/HelmetCam-<date>; no OBS needed	Live portal → Go Live	Discord; key fetched for the logged-in user — verified end to end (publish → .live → sweep → title folder)
Devs	Host code, review, CI	Gitea https://git.…/, ssh 2222	Discord (Authentik), group git-users; git-admins = site admin
Bot dev	Push code, restart container, read logs; web UI at https://bot.…/	Gitea + compose; bot/AGENT.md	web UI behind Discord (Authentik), once its provider exists
Admin	Hostnames, bot token/guild, webhook; streamers/editors are Discord roles → Authentik groups	./configure.py	ssh
Admin	Watch pairings / folder states (gamers self-serve the setup)	Syncthing UI https://sync.…/	Discord + group media-admins (Syncthing's own login disabled)
Admin	Groups, who may log in, app access	Authentik https://auth.…/	Discord or akadmin
Admin	Install/upgrade the Jellyfin plugin	./installJellyfinPlugin.sh	ssh
Admin	Register the CI runner	./registerGiteaRunner.sh	ssh

Services and their state

Service	Role	State
MediaMTX	RTMP/SRT ingest, records fMP4; publish auth delegated to dumpd (tokens via /mtx-auth); on-ready hook posts "is live" and runs the WebRTC audio mirror (<path>.webrtc, audio → Opus, never recorded)	live, token auth verified
copyparty	browse/scrub/download/finals upload	behind Authentik, group permissions (editors, media-admins), Discord-only, no WebDAV
dumpd + dock	dump, stream tokens, MediaMTX auth hook, /time, /me identity (on live.), /me/clips Syncthing self-service (on clips.); OBS dock with timecode-band setup	identity from Authentik headers
media. gateway, clips. self-service	static pages behind the gate	—
Timecode	bin/vgstrip.py decoder, tmcd + sidecar in the sweep, /dock/clock/ strip page, Go Live band composite, portal crop	server clock is the reference
Syncthing	clip intake, UI at sync.	UI behind Authentik + media-admins, own GUI auth off
normalize sweep	editing standard, one copy per clip	live
nftables	host firewall (22, 80/443, 1935, 2222, 3306, 8890/udp, 22000)	live
Caddy	TLS + routing for all hostnames; JSON access logs per site (identity headers kept, cookies/tokens dropped)	live (access logs: deploy pending)
Metrics taps	node_exporter (host), cAdvisor (containers), MediaMTX and Caddy metrics — Prometheus format on loopback, no store yet	committed 2026-11-01, deploy pending
Logging framework	rotated Docker logs + vanguard.service/category labels on every container; dumpd/sweep/vgdeband/deploy emit logfmt events (LOGGING.md); store not chosen (Loki vs VictoriaLogs vs Netdata)	committed 2026-11-01, deploy pending (recreates every container)
Authentik (+Postgres, Redis)	Discord login, groups, guild-membership policy	live
Jellyfin	watch page, SSO via Authentik, direct play only	live
Vanguard Metadata plugin	seasons by tag, Discord names/avatars/events, feed, playback policy	0.1.3 built (flat <Title>__<ts> names, policy task, feed wording) — install pending; server has 0.1.0 or 0.1.2
MariaDB	databases for the bot and Gitea; public :3306, root local-only	live
Gitea	git hosting on MariaDB, Actions enabled; holds this repo	live; Discord login via Authentik OIDC verified; access = git-users/git-admins groups; local admin as fallback
act_runner	CI jobs in Docker	not registered
bot. host + Discord bot	the bot's API and its web pages on one port (WEB_SERVE_UI), behind Authentik forward-auth; Caddy requires bot-users, the bot re-checks bot-users,admins per request	a push to main deploys itself: CI builds a release bundle (app + vendor/ dependencies) and posts it signed to dumpd POST /deploy-bot (bridge-only port, no Docker socket in CI) together with the bot's own secrets, which are managed in Gitea; dumpd verifies everything, writes the keys BOT_ENV_KEYS allows into .env and swaps the bundle into /srv/bot/app → /app; image installs no bot dependencies
Live portal (live/, live. host)	React + dockview multi-stream viewer + browser Go Live (WHIP); MediaMTX HLS/WebRTC/API on loopback; dumpd /me/stream-key	live; Go Live verified

Unfinished, in rough priority

0a. WebRTC audio (found 2026-11-01 with the 5-stream test): AAC cannot cross WebRTC, so the portal was silent unless it fell back to HLS. Fix committed: audio mirror in bin/mtx-on-ready.sh + portal mapping. Deploy pending (MediaMTX recreate between streams; dumpd restart; portal push). Streamers must set B-frames 0 for WebRTC playback. 0a. Discord bot: live on MariaDB since 2026-11-03 (139 members tracked, 36 commands synced, web UI on the bot. host). Old SQLite history (1030 rows: Cool Points, training, invites, growth) not yet merged — bin/vgbot-merge-sqlite.py, gateway stopped, report then --apply. The bot's own importer cannot be used: it requires an empty destination. 0. Band removal: deployed 2026-11-01; vgdeband.sh catch-up ran on the Vanguard-Training recordings (check journalctl -u vgdeband for the done lines). Live portal: viewer now clips the band with a measured game-aspect frame (refits on every dockview resize/move) — committed 2026-11-01, deploy with ./deployLive.sh.

1. Roles→groups policy: bound and updated (revokes only its own grants); will be proven by the next natural Discord logins (/whoami should show bot-users). Nested groups verified 2026-11-01: git-admins now reaches the header via the Vanguard profile (nested groups) scope mapping (README 5e). akadmin remains the recovery login if a policy misfires.
2. Confirm plugin 0.1.3 under Jellyfin → Plugins and the Gitea runner registered.
3. Portal viewer during a real stream: corner chip webrtc; Windows Go Live with WarDogs audio (first Windows streamer).
4. Housekeeping: pin AUTHENTIK_TAG, rotate the Discord client secret seen in a screenshot, nightly Postgres dump, Caddyfile formatting warning.
5. Medal filename pattern in the plugin (need a sample filename).
6. Home archive puller on the home box.
7. Plugin repository served by Caddy so Jellyfin updates from its catalog.
8. Discord bot: first bundle install (./deployBotBundle.sh), Authentik application for bot., first real MariaDB run (the bot's own README flags it as never tested end to end), then clip submissions. Review notes: its pages are served from the bot itself now; an Authentik username is matched to one Discord member by name.
9. Authentik blueprint from .env (Discord source, groups, policies, provider) for reproducible installs.
10. SYS-3 when transcoding or sweep speed hurts (2×6 TB option currently unavailable; watching for it). Interim option documented: off-site gap filling on the home NVENC box.

Verified behaviours worth remembering

• Never scp generated configs (mediamtx.yml, copyparty.conf, Caddyfile, dump-users.txt, .env): a template copy wiped all streamers once. They are gitignored now; *.example.* are reference only.
• Caddy runs in a container: anything it serves from disk must be a mount (/srv/live was missing → 404).
• Go Live quality depends on the browser's encoder: Windows Chrome uses the GPU (fine); Linux Chromium on NVIDIA is software H.264 Baseline (soft at any bitrate, CPU-limited at 1440p60) and cannot be fixed by flags — Linux streamers use OBS. The stats strip shows encoder + profile.
• Several videos inside one non-season folder = ONE Jellyfin episode with "versions". Streams are therefore published flat as <Title>__<ts>.mp4 in the streamer's folder (plugin 0.1.3 parses the title from the name). Four or five helmet cams a night each get their own episode.
• Streams are not re-encoded for VFR/gaps by default (dropped frames are legitimate; a 3 h re-encode is hours of CPU); players hold the last frame across a gap. The timecode sidecar lists gaps. Gap filling is available (FILL_GAPS_MS in normalize-sweep.service, README "Timecode") — turn on when the box has a hardware encoder. Clips always get CFR.
• The sweep's VFR check used a gawk-only function; Debian's mawk made it pass everything silently. Portable now.
• A deploy block that is only half-run leaves old binaries in /usr/local/bin; grep -c the installed script before blaming logic.
• MediaMTX does not reload mediamtx.yml live here (bind-mounted file): any change to it needs docker compose restart mediamtx between streams. Nothing day-to-day touches that file.
• Timecode verified live (2026-10-31): OBS + NVENC with the band → source: strip, drift −3 ms over 80 s, 0 gaps, 1920×1092 (band present); arrival time was 2.3 s late vs the first frame, which the strip corrected. Go Live from Linux Chromium (software encode) → strip decoded fine but 239 gaps and −85 ms: frame dropping, as expected on that path.
• MediaMTX stamps recording filenames in its container clock: without TZ it is UTC and the plugin reads names as Eastern (4 h off). TZ=America/New_York is set on the container.
• MediaMTX 1.21 refuses to start without webrtcAdditionalHosts or webrtcIPsFromInterfaces; configure.py writes the public host in.
• Jellyfin honours provider-supplied season numbers on a Shows library (day seasons appeared). Next check: Monday 04:10 ET should fold them into a week season.
• A stream indexed by Jellyfin while still recording has no duration and skips to the next episode on seek — fixed by recording into .live/ so nothing sees a file until it is finished and flat.
• Sweep leaves compliant Syncthing files untouched and keeps exactly one copy of converted ones (tested).
• Firewall: Docker-published ports bypass the input chain; MariaDB uses host networking precisely so the rules apply.
• Never systemctl stop|restart nftables (flushes Docker's NAT). Reload only.