vanguard-stream — SYS-1 (Xeon E-2136, 32 GB ECC, 2×4 TB SATA soft RAID)

FRESH-INSTALL.md = the ordered walkthrough from an empty box to a working stack and pipeline, manual steps included. HOSTNAME-CHANGE.md = every place a hostname lives and how to rename a subdomain or move to a new base domain safely. LOGGING.md = log framework: journald + rotated Docker logs, container labels, logfmt events, categories; store (Loki/VictoriaLogs/Netdata) chosen later. UPCOMING.md = what is still missing for a mil-sim community (gap list, techy's take, agreed direction: metadata DB, markers, watch app).

Hostnames (all CNAMEs to the main host; one Discord login covers all of them):

Host	What	Who
media.BASE (main)	gateway page at / (role-aware links + how-tos), /whoami, /logout	everyone
files.BASE	copyparty: browse, scrub, download, upload finals	editors / media-admins
clips.BASE	self-service clip intake (pair your Syncthing)	any member
sync.BASE	Syncthing admin UI	media-admins
live.BASE	live portal, Go Live, OBS dock /dock/, clock source /dock/clock/, /dump, /me, /time	everyone (LIVE_REQUIRE_GROUP)
watch.BASE	Jellyfin	Discord via OIDC
git.BASE, bot.BASE, auth.BASE	Gitea, bot web UI (bot-users), Authentik	—

/time (the server clock the timecode strip syncs to) lives only on live because browsers fetch it same-origin: both pages that use it, the clock source and Go Live, are on that host.

STATUS.md = what runs, who does what, what's unfinished (kept current). CommunityDiscordMedia.md = design + phases.

Ingest from OBS, browse/scrub/download for editors, automatic normalization to one editing standard, dump button, 15-day archive pull to a home HDD. Everything is plain files under /srv/media.

1. Reinstall from the OVH manager

Server → Reinstall → Debian 13 (Trixie). Tick custom partitioning and use distribution kernel. Layout (RAID 1 across both HDDs):

mount	size	fs
/boot	1 GB	ext4
/	250 GB	ext4
swap	8 GB per disk	swap
/srv	rest (~3.4 TiB)	xfs

Add your SSH key. OVH's template creates a debian user with sudo; root SSH is disabled. Wait for the mail.

2. Install

scp -r vanguard-stream debian@SERVER:~/
ssh debian@SERVER
cd ~/vanguard-stream && sudo bash install.sh

install.sh installs docker, ffmpeg, rclone; creates /srv/media/{streams,clips,finals,trash,.markers} owned by uid 1000 (the debian user); copies the systemd units; adds debian to the docker group; starts the compose stack. It prints what you still have to edit.

The folder stays where it is: compose bind-mounts the configs and the dock page from it. After install, log out and back in once so the docker group applies, then plain docker compose works from ~/vanguard-stream without sudo.

3. Configure (domain, admin, streamers, editors)

mediamtx.yml, copyparty.conf, Caddyfile, dump-users.txt and .env are generated on the server by configure.py and gitignored. The *.example.* files in the repo are reference only. Never copy them to the server: doing so once replaced the live streamer passwords with CHANGEME and the next apply wiped the streamers. To update the server, copy scripts (configure.py, bin/, docker-compose.yml, …) and re-run ./configure.py.

cd ~/vanguard-stream
./configure.py --full      # first time: asks everything, then applies
./configure.py             # later: add/remove/reset streamers and editors, then apply
./configure.py --show      # print current users and the OBS settings to hand out

Stateless: it reads the current values out of Caddyfile, mediamtx.yml and .env, regenerates Caddyfile, mediamtx.yml, copyparty.conf and streamers.json (→ /etc/vanguard), creates streams/<name> and clips/<name> (with .stignore) for each streamer, restarts dumpd and the containers. Editors and admins are Authentik groups (editors, media-admins, admins), not local accounts. Removing a streamer revokes publishing only; their files stay on disk. Passwords may not contain : & ? # / \ quotes or spaces (they ride inside the stream key).

4. Firewall (host nftables; OVH edge firewall is optional on top)

Applied automatically at the end of install.sh. To re-apply by hand:

cd ~/vanguard-stream && sudo bash firewall/firewall.sh

Installs firewall/nftables.conf as /etc/nftables.conf, syntax-checks, applies, enables at boot. It uses its own table (inet vanguard) so applying or reloading never disturbs Docker's rules, and filters input only so container networking keeps working. Keep your SSH session open and test a new one before closing it.

port	proto	who
22	tcp	anywhere, rate limited (edit ssh_allow in /etc/nftables.conf to restrict to your IPs)
80, 443	tcp	world (Caddy)
443	udp	world (Caddy HTTP/3)
1935	tcp	world (RTMP ingest)
8890	udp	world (SRT ingest)
22000	tcp+udp	world (Syncthing)
3306	tcp	world, rate limited (MariaDB for the bot dev; restrict via db_allow in /etc/nftables.conf)
2222	tcp	world (Gitea ssh)
8189	tcp+udp	world (WebRTC media for the live portal)

Everything else is dropped and a trickle is logged (journalctl -k | grep vg-drop). Authentik (9000), Jellyfin (8096), copyparty, the Syncthing UI and dumpd bind to 127.0.0.1 only and are reached via Caddy or an ssh tunnel, so they are deliberately not opened. MariaDB runs on the host network (not a Docker published port) precisely so the firewall can govern it. Syncthing's admin UI is proxied at https://sync.BASE/ behind the Authentik gate + group media-admins; its own GUI login is disabled (no user/password, insecureAdminAccess=true) so Discord is the only login:

docker exec syncthing syncthing cli --home /var/syncthing/config config gui insecure-admin-access set true
docker exec syncthing syncthing cli --home /var/syncthing/config config gui user set ""
docker exec syncthing syncthing cli --home /var/syncthing/config config gui password set ""

The ssh tunnel to 8384 still works as an admin fallback (no login at all — ssh is the gate there).

To change rules: edit /etc/nftables.conf, then sudo nft -f /etc/nftables.conf or sudo systemctl reload nftables. Never systemctl stop|restart nftables — Debian's unit flushes the whole ruleset on stop, which also wipes Docker's NAT rules and takes the containers offline until Docker is restarted.

5. Streamer setup (once)

Being a streamer = being in the Authentik streamers group (a Discord role via the roles→groups policy). There are no stream passwords or keys to hand out: the dock mints a 12-hour publish token for the logged-in Discord user each time Go Live is pressed and sets it as the OBS stream key itself.

OBS → Settings → Output → Encoder: B-frames = 0 (NVENC "Max B-frames: 0"; x264 add bframes=0). MediaMTX refuses WebRTC playback of H.264 with B-frames ("WebRTC doesn't support H264 streams with B-frames"), so the live portal would fall back to HLS with seconds of delay. Keyframe interval 2 s. Recording quality is unaffected. OBS → Tools → WebSocket Server Settings → enable, set a password. OBS → Docks → Custom Browser Docks → https://live.BASE/dock/ → log in with Discord once. In the dock's Settings: ingest host (prefilled), OBS WebSocket password/port, optional title prefix/suffix ({date} → YYYY-MM-DD, e.g. prefix HelmetCam- or suffix -{date}). Per stream: type a title, click Go Live. The dock shows the final stream name (<you>/<Title>) as you type and enforces the naming rules the metadata relies on: letters, digits, _ . - only, no double underscore (it separates title and timestamp in filenames), no leading dot, max 64 chars. After a bad night: Dump tonight (dumps the night of the account you are logged in as; admins/media-admins may dump anyone's).

Manual OBS/other software: dock → Manual OBS setup shows the RTMP server, the stream key and the SRT URL with the token hidden (Show / Copy / Fresh token). Tokens expire after 12 h; a running stream is not cut when its token expires, but a reconnect needs a fresh one.

Browser streaming without OBS: the live portal's Go Live (Windows Chrome 141+ for window audio + GPU encode).

5a. Onboarding a gamer's clips (Syncthing) — self-service

Gamers do it themselves at https://clips.BASE/ (Discord login): the page shows our Syncthing Device ID and asks for theirs; on submit, dumpd creates clips/<username> with the video-only .stignore, adds their device to our Syncthing (REST API, key read from /srv/media/.syncthing/config.xml) and creates the receive-only folder clips-<username> shared with them (min free 300 GB). They accept the device + folder on their PC, point it at their clip directory, Send Only. The page shows pairing/connection/sync state. Idempotent. Per-user quota (soft): dumpd measures every clips/<user> every 5 min and pauses that user's Syncthing folder when it exceeds the quota (default CLIPS_QUOTA_GB=100 in dumpd.service), resuming automatically under 90 %. The clips page shows usage and the paused state. Overrides per user in /etc/vanguard/clip-quotas.json, e.g. {"techy2493": 250, "newguy": 50} (GB) — read live, no restart. A gamer frees space by deleting clips in their own folder (Syncthing mirrors the deletion). Hard byte-exact quotas would be XFS project quotas on /srv (needs a remount with prjquota); not set up. Manual fallback (admin, same result for the server folder):

./addClipUser.sh <username>        # creates /srv/media/clips/<username> with the video-only .stignore

Their side (Windows: Syncthing installer or SyncTrayzor; Mac/Linux: Syncthing): open the UI, Actions → Show ID, send you the Device ID. In their UI: Add Remote Device with the server's ID (ours: sync.BASE → Actions → Show ID), then Add Folder on their clip directory (ShadowPlay/Medal/Steam saves), Sharing tab → tick the server, Folder Type Send Only, Save.

Our side at https://sync.BASE/: Add Remote Device → paste their ID, name it after them, Save. When their folder offer pops up: Add → Folder Path /var/syncthing/clips/<username>, Folder Type Receive Only, Advanced → Minimum Free Disk Space 300 GB, Save. Clips start arriving; the sweep normalizes them; Jellyfin shows the folder as that person's show. Their own folder structure is mirrored; only video files are pulled.

5a². Gateway page — https://media.BASE/ (the main host root; /about/ redirects here)

Role-aware landing page (Discord login): everyone sees Watch / Live / Send clips / Sign out with short how-tos; streamers get the dock + OBS setup steps; editors the browser + multicam/timecode notes; media-admins trash/Syncthing/whoami; git-users Gitea; bot-users the bot; admins/authentik Admins the admin consoles and where things live. Identity comes from GET /me (dumpd). Point people here first.

5b. Syncthing hardening (do once, in the server's Syncthing UI)

Syncthing receives straight into clips/<user> (Syncthing UI path /var/syncthing/clips/<user>), receive-only, with a .stignore (from syncthing/stignore) that pulls video files only. The gamer's folder structure is mirrored as-is. One copy per clip on the server: compliant files are left alone; a non-compliant file (HEVC/MKV/VFR) is converted to name.mp4 (name.h264.mp4 if the raw was already .mp4), the raw's name is appended to .stlocal (an #included ignore list, so Syncthing drops it without flagging or re-pulling), and the raw is deleted locally — it still exists on the gamer's PC. configure.py creates the folder + ignore file for each streamer.

In the UI (https://sync.BASE/):

• each clips folder → path /var/syncthing/clips/<user>, Folder Type: Receive Only, Edit → Advanced → Minimum Free Disk Space: 300 GB (so clips can never fill the mirror and stall the recordings).
• Actions → Settings → Connections → Incoming Rate Limit: 12500 KiB/s (~100 Mbps) so a gamer sharing a year of old clips cannot starve the ingest during stream hours.

5c. Authentik (Discord login) and Jellyfin

DNS: auth, watch (and files, clips, sync, live, git, bot) as CNAMEs to the main host. Then run ./addCommunityMediaServer.sh — it asks for every hostname, checks DNS, generates .env (Authentik secret + DB password; never commit it), adds the Caddy site blocks, starts the containers, waits for them and prints the steps below with your hostnames filled in. (./configure.py --full asks for the same hostnames if you prefer.) Then:

Authentik first run — https://auth.HOST/if/flow/initial-setup/ → set the akadmin password.

1. Discord developer portal → New Application → OAuth2: add redirect https://auth.HOST/source/oauth/callback/discord/ (last segment = the source slug below), copy Client ID + Secret.
2. Authentik → Directory → Federation & Social login → Create → Discord OAuth Source: slug discord, consumer key/secret = Client ID/Secret, user matching "link on unique identifier", scopes empty.
3. Flows & Stages → Stages → default-authentication-identification → Sources: select Discord, tick "Show sources' labels". Without this the Discord button never appears on the login page.
4. Flows & Stages → Stages → default-source-enrollment-write → User type: Internal. Otherwise Discord users are External and get "Interface can only be accessed by internal users".
5. Directory → Groups: create streamers, editors, admins. Add people after their first login.
6. (optional) Restrict login to members of your Discord server: add scope guilds to the source; Customization → Policies → Create Expression Policy from authentik/discord-guild-policy.py (fill in your server ID); bind it to both default-source-enrollment and default-source-authentication (flow → Policy / Group / User Bindings) and set each flow's Policy engine mode to all. Server ID: Discord Developer Mode → right-click server → Copy ID.

Jellyfin first run — https://watch.HOST/ → wizard: admin user, add libraries: /media/streams and /media/clips as Shows with only "Vanguard Metadata" as metadata downloader (Series/Season/Episode) and Screen Grabber + Vanguard Metadata as image fetchers; /media/finals as Home Videos with fetchers off. The Vanguard Metadata plugin (jellyfin-plugin-vanguard/, built with its package.sh) is installed by addCommunityMediaServer.sh / installJellyfinPlugin.sh from jellyfin-plugin-vanguard/dist/.

• Dashboard → Playback → Transcoding: disable hardware acceleration. Per-user "allow transcoding" is switched off automatically for everyone (SSO users included) by the Vanguard plugin's "enforce playback policy" task; turn that toggle off in the plugin config once the server has Quick Sync (SYS-3).
• Dashboard → Plugins → Repositories → add https://raw.githubusercontent.com/9p4/jellyfin-plugin-sso/manifest-release/manifest.json, then Catalog → SSO Authentication → install → restart Jellyfin.
• Authentik → Applications → Create with provider → OAuth2/OpenID: redirect https://watch.HOST/sso/OID/redirect/authentik, note client ID/secret.
• Jellyfin → Plugins → SSO-Auth → add OID provider "authentik": endpoint https://auth.HOST/application/o/<slug>/, client ID/secret, enable, roles claim groups, admin roles admins, enabled folders = the three libraries, Scheme Override https, Port Override empty (a value appends :443 and Authentik rejects the redirect). Save.
• The plugin adds no button. Test with https://watch.HOST/sso/OID/start/authentik, then Dashboard → General → Branding: Login disclaimer = jellyfin/branding-disclaimer.html (set the hostname), Custom CSS = jellyfin/branding.css. The page then shows only "Sign in with Discord"; a small "Admin login" link reveals the local username/password form. Jellyfin creates the account on first SSO login.

Phase 4 is the Vanguard Metadata Jellyfin plugin; Phase 5 puts copyparty, /sync and /dump behind Authentik forward-auth.

5d. MariaDB + Discord bot (for the bot dev)

mariadb (public on 3306, host network, root local-only, 32-char generated passwords) with a database/user for the bot, and discord-bot, a Python 3.12 container that runs /app/main.py and idles until a release is installed. Credentials live in .env (generated by configure.py; bot token + guild ID are asked in --full or the [d] menu item; ./configure.py --show prints them).

Releases are self-contained bundles, and a push to main deploys itself. The bot repo's Gitea Actions workflow runs its tests and then builds one tar.gz holding main.py, app/, website/, BUILD.json and vendor/ — its own dependencies, as prebuilt wheels for this image's Python. PYTHONPATH=/app/vendor is how the code finds them, so a new dependency never needs an image rebuild and the server never contacts a package index.

The workflow's last job posts that bundle to dumpd at POST /deploy-bot on 8081 — the Docker-bridge-only port the firewall already restricts, so a CI job never needs the Docker socket or an ssh key. dumpd checks an HMAC-SHA256 signature over the body, then runs /usr/local/bin/vanguard-deploy-bot (this repo's deployBotBundle.sh), which verifies every file against BUILD.json, swaps the bundle into /srv/bot/app (bind-mounted at /app, previous release kept beside it) and restarts the container. The install log goes back in the HTTP response, so it appears in the CI job.

The bot's own secrets live in Gitea, not here. The deploy job also sends the bot's settings, signed with the bundle, and dumpd writes them into this stack's .env — but only the keys BOT_ENV_KEYS lists. Anything else comes back to the CI job as settings_refused and is not written, so the database password and the Authentik keys this repo generates cannot be overwritten from CI. Rotating the Discord token is therefore a Gitea edit plus a re-run, with no server access.

In Gitea, on the bot repo under Settings → Actions:

Secret BOT_DEPLOY_SECRET	must equal the server's; ./configure.py --show prints it
Secret DISCORD_BOT_TOKEN	the bot token
Variable DISCORD_GUILD_ID	the server id
Variable VANGUARD_DEPLOY_HOST	only if the job cannot detect the Docker gateway

On the server, generated into .env and copied to /etc/vanguard/deploy.env by configure.py:

BOT_DEPLOY_SECRET	64 hex chars, the shared signing key
BOT_DEPLOY_AUTO	1 (default) accepts CI installs; 0 makes dumpd refuse them with 403
BOT_ENV_KEYS	the only .env keys CI may set, DISCORD_BOT_TOKEN,DISCORD_GUILD_ID by default. Extend it when the bot needs a new setting.

By hand, any time:

./deployBotBundle.sh vanguard-bot-bundle.zip      # install a release (the artifact as Gitea hands it over, zip or tar.gz)
./deployBotBundle.sh --latest                     # fetch the newest main build first (needs BOT_REPO_TOKEN in .env)
./deployBotBundle.sh --status                     # version, commit, package list, container state
./deployBotBundle.sh --rollback                   # back to the previous bundle
docker compose logs -f discord-bot                # watch it

The bot serves both its API and its web pages on 127.0.0.1:8000, published as https://bot.HOST/ behind Authentik forward-auth: Caddy asks Authentik, then injects X-Authentik-Username/-Groups/-Email, and the bot authorises per request from those headers (WEB_ALLOWED_GROUPS=bot-users,admins; WEB_SERVE_UI=true turns the pages on). Create the matching Proxy Provider in Authentik (forward auth, single application, external host https://bot.HOST), add it to the embedded outpost, and bind a group policy. Until then the site returns an error, never an open page. Caddy's own gate is BOT_REQUIRE_GROUP (bot-users by default), so admins need the bot-users role too — or set BOT_REQUIRE_GROUP= and let the bot's group list decide.

The build brief for whoever writes the bot is bot/AGENT.md; bot/README.md is the ops cheat-sheet.

5e. Gitea + Actions runner

gitea (MariaDB-backed, git.HOST via Caddy, git-over-ssh on 2222, registration closed) and act_runner (jobs run as Docker containers; labels ubuntu-latest, ubuntu-22.04, dotnet, python). The gitea db/user are created by mariadb/init/10-gitea.sh on MariaDB's first start; for a MariaDB that already existed run ./createGiteaDb.sh once.

1. ./configure.py --full asks the Gitea hostname (CNAME git); apply starts both containers.
2. https://git.HOST/ → install page (db pre-filled) → create the admin account → Install.
3. ./registerGiteaRunner.sh → generates a registration token, stores it in .env, starts the runner.
4. Remotes: ssh://git@git.HOST:2222/<org>/<repo>.git or https://git.HOST/<org>/<repo>.git.

Runner caveat: it holds the Docker socket, so anyone who can push a workflow can run as root on the box. Keep repo write access to trusted people.

Login via Authentik (Discord) — same pattern as Jellyfin, done once:

1. Authentik → Applications → Create with provider → name Gitea, slug gitea; OAuth2/OpenID Provider, authorization flow implicit-consent, client type Confidential, redirect URI (strict) https://git.HOST/user/oauth2/authentik/callback, signing key = the default certificate, scopes openid email profile (the default profile mapping carries preferred_username and groups). Copy client ID + secret.

2. Gitea → Site Administration → Identity & Access → Authentication Sources → Add: type OAuth2, name authentik (must match the callback path), provider OpenID Connect, client ID/secret, OpenID Connect Auto Discovery URL https://auth.HOST/application/o/gitea/.well-known/openid-configuration, tick "Skip local 2FA", claim name for groups groups, administrator group claim value git-admins. Save. Authentik group nesting makes the admins inherit it: authentik Admins → parent admins → parent git-admins (members of a child group count as members of the parent). Inherited groups are not in the groups claim by default: the managed profile mapping uses ak_groups (direct memberships only), and proxy providers re-add that mapping on every save, so it cannot be swapped out. Add a second scope mapping, Customization → Property Mappings → Create → Scope Mapping, name Vanguard profile (nested groups), scope profile, expression:

   direct = set(request.user.ak_groups.all())
   return {"groups": sorted(g.name for g in request.user.all_groups() if g not in direct)}
   

   and tick it (next to the default one) on the Vanguard sites, Jellyfin and Gitea providers → Advanced protocol settings → Scopes. Claims from several mappings merge, so the header becomes direct + inherited groups without duplicates. Takes effect on the next login.

3. compose (already set): registration only via external sources, auto-registration + account linking on, username from preferred_username. A first Discord login creates the Gitea user with no form.

4. Access = Authentik group git-users: Directory → Groups → create it (add a Discord role to it in the roles→groups table); Applications → Gitea → Policy / Group / User Bindings → bind git-users and git-admins. Non-members are refused by Authentik before Gitea sees them.

5. Test in a private window: https://git.HOST/user/login → "Sign in with authentik" → Discord → logged in. Members of admins become Gitea site admins on login. The local admin account stays as a fallback. Optional later: hide the password form (custom template) so Discord is the only visible way in.

5f. Live portal (live.HOST)

React + dockview page (live/) behind Authentik forward-auth: every active stream is a dockable window, audio follows the focused window, per-window mute + volume. WebRTC (WHEP) with HLS fallback, served by MediaMTX (HLS 8888, WebRTC 8889, API 9997 on loopback; WebRTC media on 8189 udp/tcp, firewall opened). MediaMTX runs the -ffmpeg image for the on-ready hook bin/mtx-on-ready.sh (bind-mounted into the container): the optional "is live" Discord post (LIVE_WEBHOOK_URL in .env, asked by configure.py --full) and the WebRTC audio mirror. Browsers cannot receive AAC over WebRTC (MediaMTX forwards Opus/G711/G722/LPCM only) and OBS sends AAC, so for every stream whose audio is not already Opus the hook runs one ffmpeg that copies the video and transcodes the audio to Opus into the sibling path <user>/<Title>.webrtc (via MediaMTX's RTSP on the container loopback). The portal plays the mirror over WebRTC and hides it as a panel; recordings, HLS and the Discord post stay on the original. Mirrors are never recorded and can only be published from inside the container (dumpd checks the source IP). Browser Go Live already sends Opus and gets no mirror. Cost: a few % of a core per stream. Viewer requirement on the streamer side: B-frames 0 (section 5), or WebRTC is refused and the portal falls back to HLS.

1. CNAME live → main host; ./configure.py --full asks the hostname (also --hosts … LIVE).
2. Authentik: Proxy Provider (forward auth, single application, external host https://live.HOST) → embedded outpost → group policy (e.g. everyone in the server, or streamers+editors+admins).
3. Dev box: ./deployLive.sh builds live/ and copies dist/ to /srv/live (Caddy serves it directly).

Go Live button (window audio: Windows + Chrome/Edge 141+; Linux/Mac get video only): broadcast a window from the browser (no OBS) as <user>/HelmetCam-<date> — a fresh publish token comes from /me/stream-token for the logged-in Discord user (member of streamers).

Bandwidth: no transcoding, every viewer gets the source bitrate (~15 Mbps) → ~30 concurrent viewers on the 500 Mbps line. For a crowd, restream to Twitch from OBS.

5g. Authentik forward-auth for the Caddy-gated hosts (media., files., clips., sync., live., bot.)

Design: Authentik authenticates once for the whole domain; Caddy authorizes per host from the X-Authentik-Groups header the outpost returns; Discord roles fill the Authentik groups at login.

1. Applications → Outposts → edit "authentik Embedded Outpost" → config authentik_host: https://auth.HOST.
2. Applications → Create with provider → Application Vanguard sites; Proxy Provider, authorization flow implicit-consent, mode Forward auth (domain level), authentication URL https://auth.HOST, cookie domain = the parent domain (e.g. dumpstervigilantes.rip). Edit the Embedded Outpost → Applications → add it → Update. One login now covers every gated host.
3. Per-host group requirement = .env: BOT_REQUIRE_GROUP=bot-users (default), LIVE_REQUIRE_GROUP= (empty = any logged-in member), SYNC_REQUIRE_GROUP=media-admins (the Syncthing UI on sync.). On files. copyparty itself authorizes by group (editors, media-admins); the dump endpoint lets admins/media-admins dump anyone's night. Leave "Intercept header authentication" off: login is Discord only, no app passwords, no WebDAV. ./configure.py apply regenerates the Caddy blocks; a user outside the group gets a 403 from Caddy before the app sees the request.
4. Discord roles → groups: Directory → Groups: create bot-users (and keep streamers, editors, admins). Discord source → Scopes: guilds guilds.members.read. Customization → Policies → Expression Policy discord-roles-to-groups from authentik/discord-roles-to-groups.py (fill in the server ID and the role-id → group table). Bind it to default-source-enrollment and default-source-authentication (engine mode all, after discord-guild-member). From then on giving someone a Discord role gives them the matching access on next login; removing it removes it. Memberships added by hand in Authentik are left alone: the policy only revokes what it granted itself (tracked in the user attribute discord_role_groups). copyparty additionally requires a header with a secret name that only Caddy adds (idp-h-key, COPYPARTY_IDP_KEY in .env), so identity headers cannot be forged even if copyparty were reached directly. Logout: copyparty's logout link, the live portal's "Sign out" and the dock's "sign out" go through /outpost.goauthentik.io/sign_out; https://<any gated host>/logout does the same from anywhere (e.g. from the Syncthing UI, which has no logout of its own). Set the provider's Invalidation flow to default-provider-invalidation-flow so that also ends the Authentik login: every logout then forces a full Discord re-login on the next visit (and picks up group changes). Discord itself may approve silently if you are still logged into Discord in that browser.
5. Debug: https://HOST/whoami prints the username/groups headers exactly as the apps receive them. Group changes in Authentik apply on the next login — log out at https://auth.HOST/ and back in.
6. Test in a private window: https://live.HOST/ → Discord login → portal; https://bot.HOST/ → 403 unless you hold the bot role; https://live.HOST/me/stream-key shows your username.

Jellyfin and Gitea use OIDC providers instead, not this.

6. Editors

Log in with Discord at https://files.BASE/ (copyparty behind Caddy + Authentik). Access is by Authentik group: editors read streams and clips and write /finals; media-admins have full rights everywhere including /trash. Scrub any .mp4 in the browser, download what you want.

No WebDAV: login is Discord only (no passwords of any kind), and a mounted drive cannot do a browser login. Download from the browser: single files, or select files/folders and use copyparty's zip/tar download.

Available but not set up. WebDAV mounts can be enabled later without code changes if we decide we want them: in Authentik, edit the Vanguard sites proxy provider and tick Intercept header authentication; each editor then creates an app password under their Authentik user settings and mounts the same URL with their Authentik username + that app password. Caddy and copyparty already pass the identity through, so nothing else changes. It is off because it introduces a password-like credential alongside Discord login.

Every .mp4 an editor sees is the same kind of file (see "Editing standard"). Raw clipper output that needed converting is kept in a hidden .src/ folder next to it (media-admins can show dotfiles in copyparty).

7. Home archive box

Copy home/rclone-archive.sh to the home machine, set the vars, run from cron nightly. It connects over SFTP as debian (all media files are owned by uid 1000 so it can delete them). Pulls everything older than 15 days off the server, including the hidden .src/ raw files.

Publish authentication (no passwords)

MediaMTX asks dumpd (POST /mtx-auth on 127.0.0.1:8081, reachable only from Docker's bridge) for every connection. Reads/playback are allowed (viewers are gated by Caddy + Authentik). Publishing needs a token minted by GET /me/stream-token behind Authentik for a member of streamers/admins/media-admins; dumpd checks the HMAC signature (/etc/vanguard/stream-token.secret, generated into .env as STREAM_TOKEN_SECRET), the expiry, and that the path starts with that user's name. MediaMTX does not pick up mediamtx.yml changes live on this setup: after any change to it, docker compose restart mediamtx between streams. Day-to-day operations (streamers, tokens, titles) never touch that file. configure.py reloads Caddy in place and restarts only copyparty.

Timecode for multicam editing (frame accurate)

Every published stream carries a QuickTime timecode track (tmcd, time of day in Eastern, at the file's frame rate) and a UTC creation_time, written by the sweep when it publishes the file, plus a sidecar <Title>__<ts>.json with the exact capture time (the Jellyfin plugin reads it). Resolve and Premiere read the track natively: drop clips from several streamers on a multicam timeline, sync by timecode.

Where the time comes from, best first:

1. Overscan band — a thin strip of extra rows below the picture carrying the server clock, encoded (black/white cells, format in bin/vgstrip.py) plus a small readable clock. The server decodes it (bin/vgstrip.py) at the start, middle and end of each recording → the timecode is frame-accurate at the start and the sidecar records drift over the file. Then the band is cropped off: the sweep re-encodes the stream (libx264 CRF 18, frame timing preserved, audio copied) so the published file is the game image + audio + the tmcd track and nothing else; the raw with the band is deleted (KEEP_RAW_STREAMS=1 keeps it in .src/). Streams without the band are remuxed losslessly. Cost: every re-encode (streams and clips) runs x264 on ENCODE_THREADS threads (2, set in systemd/normalize-sweep.service) so Jellyfin and live viewers stay responsive; a 3 h 1440p60 night therefore takes most of the next day to publish. OBS: dock → Timecode band → "Add band + clock source" (canvas grows by W/160 rows, a locked browser source Vanguard Clock → https://live.BASE/dock/clock/ sits in the band; only while not streaming). Go Live: the "Timecode band" checkbox (default on).
2. Server arrival time (no band) — microseconds in the recording filename. Within ~0.1–0.3 s per streamer (encoder + network delay); coarse but always available.

Viewing: the live portal crops the band on the live picture (the only place it still exists); recordings in Jellyfin, copyparty and the editors' timelines are already at game resolution. Recordings published before the crop existed: sudo vgdeband.sh [FILE…] re-encodes them the same way and marks the sidecar. Cross-check when it matters: everyone is in the same voice chat, so audio-waveform sync is sample-accurate. Why continuous and not a start burst: a PC clock drifts vs the server (10 ppm ≈ 4 frames over 2 h).

Dropped frames / gaps: OBS keeps stamping frames from its clock, so after a drop the file has a hole and every frame that exists is still at the right time — the timecode stays correct, and players (Jellyfin, the portal) hold the last frame across the hole with audio in sync. By default the sweep does not re-encode a stream to fill holes (hours of CPU on this box); the sidecar lists them (gaps.count, total_missing_ms, first 50 positions). Premiere preserves timing on such files; in Resolve check the clip's frame-rate interpretation. Gap filling (available, off by default). The sweep can re-encode a stream whose holes total at least FILL_GAPS_MS milliseconds to constant frame rate with duplicated frames (raw kept in .src/, timecode carried over). It costs ~2× the stream length in CPU on the E-2136, so it is off until the box has a hardware encoder. Turn it on (threshold 1 s) / off, on the server:

sudo sed -i 's|^#* *Environment=FILL_GAPS_MS=.*|Environment=FILL_GAPS_MS=1000|' /etc/systemd/system/normalize-sweep.service && sudo systemctl daemon-reload
sudo sed -i 's|^Environment=FILL_GAPS_MS=.*|# Environment=FILL_GAPS_MS=1000|' /etc/systemd/system/normalize-sweep.service && sudo systemctl daemon-reload

Check: systemctl show normalize-sweep -p Environment. Takes effect on the next sweep run; already-published files are not revisited. Clips are always converted to CFR regardless. Planned alternative: re-encode flagged streams off-site on a machine with a hardware encoder (home box NVENC or a Spot GPU) and push the result back — see CommunityDiscordMedia.md → Later.

Deployment (push to deploy — built, OFF by default)

The server folder is a git clone of this repo (./switchToClone.sh key → add the printed deploy key in Gitea → ./switchToClone.sh clone). Generated files (.env, Caddyfile, mediamtx.yml, copyparty.conf, the token secret) are gitignored and survive pulls; static pages are bind-mounted from the checkout.

sudo vanguard-deploy pull fetches main and applies only what changed, after check.sh:

changed	action	interruption
configure.py, dock/, clips/, about/, stignore	configure.py --apply → Caddy reload, copyparty restart	none / a second
bin/*, systemd/*	install; dumpd restart when it changed; timers pick up next run	a second
firewall/nftables.conf	nft -f	none
docker-compose.yml, bot/Dockerfile, bot/entrypoint.sh	compose up -d --build for changed services	that service
deployBotBundle.sh	reinstalled as /usr/local/bin/vanguard-deploy-bot (what dumpd runs for the bot's CI)	none
anything that changes mediamtx.yml or the mediamtx service	restart only when no stream is live; otherwise deferred, deploy-pending.timer retries every 5 min	none while live
jellyfin-plugin-vanguard/dist/*.zip (committed)	installJellyfinPlugin.sh	Jellyfin restart
live/	npm run build on the server → /srv/live	none

sudo vanguard-deploy rollback checks out the previously deployed revision and applies the reverse diff. sudo vanguard-deploy status shows revisions, auto mode, pending restarts, live streams.

One-time rollout (the last scp-style deploy; afterwards the server pulls). From the dev box:

cd ~/claudevs/vanguard-stream && git push
scp bin/dumpd.py debian@HOST:~/vanguard-stream/bin/
scp systemd/deploy-pending.service systemd/deploy-pending.timer debian@HOST:~/vanguard-stream/systemd/
scp docker-compose.yml configure.py deploy.sh switchToClone.sh check.sh install.sh README.md STATUS.md CommunityDiscordMedia.md debian@HOST:~/vanguard-stream/
ssh -t debian@HOST 'cd ~/vanguard-stream && sudo apt-get install -y -q git nodejs npm rsync python3-yaml >/dev/null && sudo install -m 0755 bin/dumpd.py /usr/local/bin/ && ./configure.py --apply && sudo systemctl restart dumpd && sudo vanguard-deploy status'

The apply prints the post-start fixups (Syncthing, Gitea admin user, Jellyfin plugin) and ends with a status block: DEPLOY_AUTO=0, no pending restart, and "not a git checkout" until the next step.

Then switch the server folder to a clone (two steps):

ssh -t debian@HOST 'cd ~/vanguard-stream && ./switchToClone.sh key'       # prints a public key
# Gitea → repo → Settings → Deploy Keys → add it, read-only
ssh -t debian@HOST 'cd ~/vanguard-stream && ./switchToClone.sh clone && cd ~/vanguard-stream && ./configure.py --apply >/dev/null && sudo vanguard-deploy status'

switchToClone.sh clone keeps .env, the generated configs, the token secret and plugin zips, and leaves the old copy at ~/vanguard-stream.old until you delete it. From here on a manual deploy is sudo vanguard-deploy pull on the server.

Automatic: Gitea → repo → Settings → Webhooks → Gitea, URL http://<docker gateway>:8081/deploy (docker network inspect vanguard-stream_default -f '{{(index .IPAM.Config 0).Gateway}}', usually 172.18.0.1), secret = DEPLOY_WEBHOOK_SECRET from .env, push events, branch main. dumpd verifies the HMAC and runs vanguard-deploy pull only when DEPLOY_AUTO=1 in .env (then ./configure.py --apply to push the setting to /etc/vanguard/deploy.env). Until then every push is logged (journalctl -u dumpd | grep deploy) and ignored; Gitea's "test delivery" should get a 202 "auto-deploy disabled" back. Optional DEPLOY_NOTIFY_WEBHOOK_URL posts deploy results to Discord. Turning it on: DEPLOY_AUTO=1 in .env, ./configure.py --apply, and git mv .gitea/draft-workflows/ci.yml .gitea/workflows/ci.yml for CI.

CI: .gitea/draft-workflows/ci.yml (inert until moved to .gitea/workflows/) runs check.sh and the plugin tests/build on the Actions runner. The runner never deploys — it must not touch the host.

Rebuild from scratch (what is and isn't automated)

Server side, scripted:

git clone ssh://git@git.HOST:2222/techy/Media-Server-Setup.git ~/vanguard-stream   # or scp the folder
cd ~/vanguard-stream && sudo bash install.sh        # packages, docker, folders, host scripts, timers, firewall, portal build
./configure.py --full                               # hostnames, secrets, configs, containers, post-start fixups

Restore first if rebuilding a known box: .env (all secrets — without it every password is regenerated and the databases won't match), /srv/appdata (Authentik DB, Jellyfin, Gitea, MariaDB, Syncthing config), /srv/media (the footage), /etc/vanguard/clip-quotas.json if used. With those restored, nothing below is needed except DNS.

Post-start fixups configure.py performs on every apply (idempotent): Syncthing GUI auth off, Gitea admin user created from .env (GITEA_ADMIN_USER / GITEA_ADMIN_PASSWORD; the install page is locked), Jellyfin plugin installed from the committed zip. Authentik's akadmin password is set on first start from AUTHENTIK_BOOTSTRAP_PASSWORD in .env (no initial-setup page).

Still manual on a truly fresh box (not in the repo; documented in §5c–5g):

• Authentik: Discord source, identification-stage source, enrollment user type, groups, the two policies, the domain-level proxy provider + outpost host, the Jellyfin and Gitea OIDC providers, application bindings
• Jellyfin: first-run wizard, libraries (Shows + Vanguard provider), SSO plugin + provider, branding
• Gitea: authentication source (authentik), deploy key, webhook; ./registerGiteaRunner.sh
• Discord bot: its repo lives outside this one, so nothing here restores it. Put BOT_DEPLOY_SECRET (./configure.py --show), DISCORD_BOT_TOKEN and DISCORD_GUILD_ID into that repo's Gitea Actions secrets/variables, then push to its main — or install a bundle by hand with ./deployBotBundle.sh. /srv/bot/app is created empty by install.sh; the container idles until a release lands there.
• Discord developer portal: application, redirect URIs, bot; DNS records A .env-driven Authentik blueprint would remove the first line (planned).

Layout on the server

/srv/media/streams/<user>/<Title>/.live/<ts>.mp4             recording in progress (hidden from Jellyfin/copyparty)
/srv/media/streams/<user>/<Title>__<ts>.json                 capture time (UTC ms), timecode, source (strip|arrival), drift
/srv/media/streams/<user>/<Title>__<ts>.mp4                  finished stream, one flat MP4 per recording, directly in
                                                             the streamer's folder (several files inside one title
                                                             folder would be ONE Jellyfin episode with "versions")
/srv/media/clips/<user>/clip.mp4                             clip (synced as-is, or converted; raw removed)
/srv/media/clips/<user>/.stlocal                             Syncthing ignore list maintained by the sweep
/srv/media/clips/<user>/.src/clip.mkv                        raw source, only for clips that did NOT arrive via Syncthing
/srv/media/clips/<user>/clip.normalize-failed.log            only if normalization failed; delete to retry
/srv/bot/app/                                                the Discord bot's installed release (bind-mounted at /app)
/srv/bot/previous/                                           the release before it (deployBotBundle.sh --rollback)
/srv/media/finals/                                           editors upload here
/srv/media/trash/<night>/...                                 dumped, purged after 3 days
/srv/media/.markers/                                         dump markers
/srv/media/.syncthing/                                       Syncthing config

Editing standard (what normalize-sweep enforces)

MP4 · H.264 High, 8-bit 4:2:0 · constant frame rate · AAC audio (all tracks kept) · index at the front (instant seek in browser and NLE) · at most 2560x1440 and 60 fps (downscaled to fit, never upscaled; aspect kept).

input	action
already compliant	nothing
MediaMTX recording in <Title>/.live/ (fragmented MP4)	once idle 3 min: decode the timecode band (or use arrival time), then publish to <user>/<Title>__<ts>.mp4 with a tmcd track + creation_time — band present: re-encode with the band cropped off; no band: lossless remux — write the .json sidecar, remove .live and the empty title folder
in a Syncthing folder (.stfolder marker)	compliant / index-only: untouched; otherwise convert → name.mp4, add raw to .stlocal, delete raw (one copy)
HEVC / 10-bit / VFR / MKV / >1440p / >60fps / non-AAC	full-res re-encode (x264 veryfast CRF 18) → name.mp4; source kept in .src/
ffmpeg fails or duration mismatch	name.normalize-failed.log written, file skipped until that log is deleted

Runs every 5 min on files idle ≥3 min and not open. Software x264 on the E-2136 does ~0.5x realtime at 1440p60, so a 2 h clip batch takes ~4 h; streams only remux (minutes). All at idle priority.

Reboot behaviour

Everything comes back on its own, in this order:

1. mdadm assembles the mirror, systemd mounts /srv (local-fs). A degraded mirror still boots.
2. docker.service has RequiresMountsFor=/srv/media (drop-in installed by install.sh), so it never starts with the media tree missing. All containers are restart: unless-stopped.
3. dumpd.service starts; normalize-sweep.timer fires 2 min after boot and every 5 min after; trash-purge.timer is Persistent=true so a missed 05:00 run fires at boot.
4. A stream cut by a reboot is playable up to the last second (fMP4 parts are flushed every 1 s); OBS reconnects and MediaMTX opens a new file in the same title folder.
5. Half-written *.part files are overwritten by the next sweep. Dumps are idempotent.

Time: install.sh sets the server clock to America/New_York so journal, timers, file timestamps and the dump window all read in Eastern. Dump "night" = 17:00 → 02:00 Eastern (pinned in dumpd via DUMP_TZ, independent of the host zone). Between 02:00 and 17:00 the default night is the previous evening; after 17:00 it is tonight. The 05:00 trash purge is 05:00 Eastern.

Timers

unit	when	does
normalize-sweep.timer	every 5 min	remux/re-encode finished files to the editing standard
trash-purge.timer	daily 05:00 Eastern	deletes trash older than 3 days
dumpd.service	always	the dump endpoint on 127.0.0.1:8080, behind Caddy on live. at /dump, /me, /time and on clips. at /me/clips