Media-Server-Setup/FRESH-INSTALL.md

Fresh install — from an empty OVH box to a working stack with the deploy pipeline

Everything in order. Scripted steps are commands; manual steps are click lists. HOST = your main hostname (ours: media.dumpstervigilantes.rip), BASE = the parent domain. Detailed background for each part is in README.md (section numbers given).

0. Before touching the server

• DNS: A record for HOST → the server IP; CNAMEs files, clips, sync, live, auth, watch, git, bot → HOST. Optional: the bare domain A → server IP (it redirects to https://HOST/).
• Discord developer portal → New Application (one app serves Authentik login, the Jellyfin plugin's bot and the Discord bot):
  • OAuth2 → Redirects: https://auth.BASE/source/oauth/callback/discord/ (the last segment is the Authentik source slug — keep them identical). Copy Client ID + Client Secret.
  • Bot → Reset Token (copy it); enable Server Members intent; OAuth2 URL generator: scope bot, permissions View Channels + Send Messages + Embed Links + Attach Files → invite it to the server.
  • Discord → User Settings → Advanced → Developer Mode on; right-click the server → Copy Server ID; Server Settings → Roles → Copy Role ID for each role you will map to a group.
• If this is a rebuild, have the backups ready: .env, /srv/appdata, /srv/media, /etc/vanguard/clip-quotas.json. Restoring .env + /srv/appdata makes sections 5–7 unnecessary (that state lives there).

1. OS (README §1)

OVH manager → Reinstall → Debian 13, custom partitioning, distribution kernel. RAID 1: /boot 1 GB ext4 · / 250 GB ext4 · swap 8 GB per disk · /srv rest xfs. Add your SSH key. You get the debian user with sudo.

2. Code onto the box

Fresh box has no deploy key yet, so copy the repo in:

scp -r vanguard-stream debian@HOST:~/        # from a machine that has the repo (or git clone over https with a token)

3. Host install (README §2, §4)

ssh debian@HOST
cd ~/vanguard-stream && sudo bash install.sh

Packages, Docker, timezone (Eastern), uid 1000, /srv folders, host scripts and timers, sysctl, live-portal build, firewall (keep this session open; test a new ssh in a second terminal). Log out and back in once (docker group).

4. Configure + start (README §3)

cd ~/vanguard-stream && ./configure.py --full

Asks: main hostname (the gateway page); files/clips/sync/auth/watch/git/bot/live hostnames (defaults); bare domain (optional) and where it redirects; Discord bot token + server ID (optional now, needed for the plugin feed); "is live" webhook (optional). Generates .env with every secret, the configs, starts all containers, waits, then runs the post-start fixups (Syncthing GUI auth off, Gitea admin user, Jellyfin plugin). ./configure.py --show prints every credential you will need below (akadmin password, Gitea admin, MariaDB, bot DB).

Check: docker compose ps → all Up; https://auth.BASE/, https://watch.BASE/, https://git.BASE/, https://HOST/ answer with certificates (Caddy fetches them itself, give it a minute).

5. Authentik (README §5c, §5g) — manual, ~20 minutes

Log in at https://auth.BASE/ as akadmin with AUTHENTIK_BOOTSTRAP_PASSWORD from .env (change it: user menu → settings).

1. Directory → Federation & Social login → Create → Discord OAuth Source: name Discord, slug discord, consumer key/secret = the Discord app's Client ID/Secret, scopes guilds guilds.members.read, user matching "link on unique identifier", flows default-source-authentication / default-source-enrollment.
2. Flows & Stages → Stages → default-authentication-identification → Sources: add Discord, tick "Show sources' labels". (No Discord button appears until this is done.)
3. Flows & Stages → Stages → default-source-enrollment-write → User type Internal, user path users.
4. Directory → Groups: streamers, editors, media-admins, admins, bot-users, git-users, git-admins. Nesting: admins parent authentik Admins; authentik Admins parent git-admins (admins inherit everything).
5. Customization → Policies → Create Expression Policy ×2 from the repo: discord-guild-member (authentik/discord-guild-policy.py, set your server ID) and discord-roles-to-groups (authentik/discord-roles-to-groups.py, set server ID + role-id→group table). Tick "Execution logging" on both.
6. Flows & Stages → Flows → default-source-enrollment and default-source-authentication: bind both policies (guild first), Policy engine mode: all.
7. Applications → Outposts → edit "authentik Embedded Outpost" → config authentik_host: https://auth.BASE.
8. Applications → Create with provider → Vanguard sites: Proxy Provider, implicit consent, Forward auth (domain level), authentication URL https://auth.BASE, cookie domain BASE, invalidation flow default-provider-invalidation-flow. Edit the Embedded Outpost → add the application.
9. Applications → Create with provider → Jellyfin: OAuth2/OpenID, implicit consent, Confidential, redirect https://watch.BASE/sso/OID/redirect/authentik, default signing cert, scopes openid email profile. Keep ID + secret.
10. Applications → Create with provider → Gitea (slug gitea): OAuth2/OpenID, same settings, redirect https://git.BASE/user/oauth2/authentik/callback. Bind groups git-users + git-admins on the application.
11. Test in a private window: https://auth.BASE/ → Discord → your user exists and is Internal. Add yourself to admins (and streamers, media-admins as needed) or give yourself the Discord roles; sign out and back in. https://HOST/whoami shows your groups.

6. Jellyfin (README §5c) — manual, ~10 minutes

https://watch.BASE/ → wizard: admin user; libraries /media/streams and /media/clips as Shows with metadata downloaders = only "Vanguard Metadata" (Series/Season/Episode), image fetchers Vanguard Metadata (Series) + Screen Grabber (Episode), real-time monitoring on; /media/finals as Home Videos, fetchers off. Dashboard → Playback → Transcoding: hardware acceleration none. Dashboard → Plugins: Vanguard Metadata is installed → its page: Discord bot token + server ID, feed channel ID, public URL https://watch.BASE. Plugins → Repositories → add https://raw.githubusercontent.com/9p4/jellyfin-plugin-sso/manifest-release/manifest.json → Catalog → SSO Authentication → install → docker compose restart jellyfin. Plugins → SSO-Auth → add OID provider authentik: endpoint https://auth.BASE/application/o/jellyfin/, client ID/secret, roles claim groups, admin roles admins, enabled folders = the 3 libraries, Scheme Override https, Port Override EMPTY, Save. Dashboard → General → Branding: paste jellyfin/branding-disclaimer.html (set the hostname) and jellyfin/branding.css. Test: https://watch.BASE/sso/OID/start/authentik. Scheduled Tasks → run "Vanguard: snapshot Discord events", "roll seasons", "enforce playback policy" once.

7. Gitea (README §5e) — manual, ~10 minutes

Log in at https://git.BASE/ as GITEA_ADMIN_USER / GITEA_ADMIN_PASSWORD from .env.

1. Site Administration → Identity & Access → Authentication Sources → Add: OAuth2, name authentik, OpenID Connect, client ID/secret from step 5.10, discovery URL https://auth.BASE/application/o/gitea/.well-known/openid-configuration, Skip local 2FA, group claim groups, administrator group git-admins.
2. Create the repo techy/Media-Server-Setup (empty) and push this repo to it from your dev box: git remote add origin ssh://git@git.BASE:2222/techy/Media-Server-Setup.git && git push -u origin main (your SSH key under Settings → SSH keys).
3. Runner: ./registerGiteaRunner.sh on the server → Site Administration → Actions → Runners shows it idle.

8. Pipeline (README "Deployment")

1. On the server: ./switchToClone.sh key → add the printed key as a read-only Deploy Key on the repo → ./switchToClone.sh clone (keeps .env, generated configs, plugin zips) → ./configure.py --apply.
2. Gitea → repo → Settings → Webhooks → Add → Gitea: URL http://<docker gateway>:8081/deploy (docker network inspect vanguard-stream_default -f '{{(index .IPAM.Config 0).Gateway}}'), secret = DEPLOY_WEBHOOK_SECRET from .env, push events, branch main. Test delivery → 202 "auto-deploy disabled".
3. Manual deploys from now on: push, then sudo vanguard-deploy pull on the server. When trusted: DEPLOY_AUTO=1 in .env → ./configure.py --apply; CI: git mv .gitea/draft-workflows/ci.yml .gitea/workflows/ci.yml.

9. Streamers, gamers, editors (README §5, §5a, §6)

• Streamers: OBS → WebSocket server on; Docks → Custom Browser Dock https://live.BASE/dock/ → Discord login → Settings: OBS WebSocket password; Timecode band → "Add band + clock source" (not streaming). Then title → Go Live.
• Gamers: https://clips.BASE/ → paste their Syncthing Device ID → accept device + folder on their PC (Send Only). Syncthing UI (https://sync.BASE/, media-admins): Actions → Settings → Connections → incoming rate limit 12500 KiB/s.
• Everyone: https://HOST/ (the gateway page; editors browse at https://files.BASE/).

10. Afterwards

• Pin AUTHENTIK_TAG in .env to the running version. Rotate any secret that was ever pasted into chat.
• Back up .env, /srv/appdata, /srv/media (home archive puller: README §7).
• STATUS.md = what runs and what is unfinished; check.sh before any commit.

---

Appendix — the runbook: every action, in order, with the exact commands

Conventions: commands prefixed dev$ run on your workstation (it has the repo at ~/claudevs/vanguard-stream and your SSH key); srv$ run on the server over ssh as debian. Replace HOST / BASE with your names (ours: media.dumpstervigilantes.rip / dumpstervigilantes.rip). Click paths are written App → Menu → Item. Every step ends with how you know it worked.

A. DNS and Discord (no server yet)

A1. DNS at your registrar: A HOST → CNAME auth.BASE → HOST CNAME watch.BASE → HOST CNAME git.BASE → HOST CNAME bot.BASE → HOST CNAME live.BASE → HOST (optional) A BASE → Check from dev: dev$ for h in HOST auth.BASE watch.BASE git.BASE bot.BASE live.BASE; do printf '%-28s ' $h; getent hosts $h | awk '{print $1}'; done → every line shows the server IP.

A2. Discord Developer Portal → New Application "Vanguard". OAuth2 → Redirects → add https://auth.BASE/source/oauth/callback/discord/ → Save. Copy Client ID and (Reset) Client Secret. Bot → Reset Token → copy. Privileged Gateway Intents → Server Members Intent ON → Save. OAuth2 → URL Generator → scopes: bot; permissions: View Channels, Send Messages, Embed Links, Attach Files → open the URL → invite the bot to your server. → the bot appears in your server's member list.

A3. Discord IDs: User Settings → Advanced → Developer Mode ON. Right-click your server → Copy Server ID. Server Settings → Roles → right-click each role you'll map → Copy Role ID. Keep a note like: guild=123… streamers=… editors=… media-admins=… admins=… bot-users=… git-users=… git-admins=…

B. Operating system

B1. OVH manager → the server → Reinstall → Debian 13 → custom partitioning, "use distribution kernel": /boot 1 GiB ext4 (RAID1) · / 250 GiB ext4 (RAID1) · swap 8 GiB (per disk) · /srv remaining xfs (RAID1). Add your SSH public key. Wait for the mail. B2. dev$ ssh debian@HOST 'lsblk -o NAME,SIZE,FSTYPE,MOUNTPOINT; df -h /srv' → /srv is xfs, ~3.4 TiB, mounted.

C. Code onto the box

C1. dev$ cd /claudevs && scp -r vanguard-stream debian@HOST:/ C2. dev$ ssh debian@HOST 'ls ~/vanguard-stream | head; test -f ~/vanguard-stream/install.sh && echo ok' → "ok".

D. Host install

D1. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && sudo bash install.sh' (~5 min: packages, docker, timezone, folders, scripts, timers, portal build, firewall.) KEEP THIS TERMINAL OPEN. D2. In a SECOND terminal: dev$ ssh debian@HOST 'echo still in' → "still in" (firewall did not lock you out). D3. dev$ ssh debian@HOST 'timedatectl show -p Timezone --value; systemctl is-active docker dumpd normalize-sweep.timer trash-purge.timer nftables; docker --version; node --version; ls /srv/live | head -3' → America/New_York; five "active"; versions; /srv/live has index.html. D4. Log out/in once for the docker group: dev$ ssh debian@HOST 'docker ps >/dev/null && echo docker-ok' → docker-ok.

E. Configure and start the stack

E1. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./configure.py --full' Answers: HOST; Enter ×8 for the sub-hostnames; bare domain (Enter = BASE, or blank to skip) and its redirect; bot token (from A2) ; server ID (A3) ; "is live" webhook (optional); y to apply. (Pulls ~2 GB of images, starts 13 containers, waits, runs post-start fixups.) E2. dev$ ssh debian@HOST 'cd ~/vanguard-stream && docker compose ps --format "{{.Name}} {{.Status}}"' → every container "Up" (authentik-server/worker take ~1 min; act_runner restarts until step G3 — fine). E3. dev$ for h in HOST auth.BASE watch.BASE git.BASE live.BASE bot.BASE; do printf '%-26s ' $h; curl -s -o /dev/null -m 10 -w '%{http_code}\n' https://$h/; done → 200/302 for all (302 = redirected to login). If 000, wait a minute for certificates and retry. E4. dev$ ssh debian@HOST 'cd ~/vanguard-stream && ./configure.py --show' → copy out: akadmin password (AUTHENTIK_BOOTSTRAP_PASSWORD), Gitea admin user/password, MariaDB root + bot + gitea passwords. E5. dev$ ssh debian@HOST 'journalctl -u dumpd -n 2 --no-pager | tail -1; grep -c "authMethod: http" ~/vanguard-stream/mediamtx.yml; sudo nft list table inet vanguard | grep -c dport' → dumpd line ending "token-secret=ok"; 1; a dozen rules.

F. Authentik (browser, logged in as akadmin at https://auth.BASE/ with the E4 password)

F1. Change akadmin's password: top-right user → Settings → Change password. F2. Directory → Federation & Social login → Create → "Discord OAuth Source": Name Discord · Slug discord · Enabled · User matching "Link users on unique identifier" · Consumer key = Client ID · Consumer secret = Client Secret · Scopes: guilds guilds.members.read · (Flow settings if shown: authentication default-source-authentication, enrollment default-source-enrollment) → Finish. F3. Flows & Stages → Stages → default-authentication-identification → Edit → Sources: select Discord; tick "Show sources' labels" → Update. → the login page (private window) now shows a Discord button. F4. Flows & Stages → Stages → default-source-enrollment-write → Edit → User type: Internal · User path: users → Update. F5. Directory → Groups → Create, one each: streamers, editors, media-admins, admins, bot-users, git-users, git-admins. Then nesting: Groups → admins → Edit → Parent: authentik Admins → Update. Groups → authentik Admins → Edit → Parent: git-admins → Update. F6. Customization → Policies → Create → Expression Policy: name discord-guild-member · expression = contents of authentik/discord-guild-policy.py with ACCEPTED_GUILD_ID + GUILD_NAME filled in · tick Execution logging → Create. Again: name discord-roles-to-groups · expression = authentik/discord-roles-to-groups.py with GUILD_ID + ROLE_TO_GROUP filled from A3 · Execution logging → Create. F7. Flows & Stages → Flows → default-source-enrollment → Policy / Group / User Bindings → Bind existing policy → discord-guild-member (order 0) → Create; again → discord-roles-to-groups (order 10) → Create. Then Edit the flow → Policy engine mode: all → Update. Repeat all of F7 for default-source-authentication. F8. Applications → Outposts → "authentik Embedded Outpost" → Edit → in the configuration box set authentik_host: https://auth.BASE (leave authentik_host_browser empty) → Update. F9. Applications → Create with provider: Application name "Vanguard sites" · Provider type Proxy Provider · Authorization flow default-provider-authorization-implicit-consent · Mode "Forward auth (domain level)" · Authentication URL https://auth.BASE · Cookie domain BASE · Invalidation flow default-provider-invalidation-flow → Finish. Applications → Outposts → Embedded Outpost → Edit → Applications: add "Vanguard sites" → Update. F10. Applications → Create with provider: "Jellyfin" (slug jellyfin) · OAuth2/OpenID Provider · implicit-consent · Client type Confidential · Redirect URI (strict) https://watch.BASE/sso/OID/redirect/authentik · Signing key: the default certificate · Advanced → Scopes: openid, email, profile → Finish. Note Client ID + Secret. F11. Applications → Create with provider: "Gitea" (slug gitea) · same provider settings · Redirect URI https://git.BASE/user/oauth2/authentik/callback → Finish. Note Client ID + Secret. Applications → Gitea → Policy / Group / User Bindings → Bind existing group → git-users → Create; again → git-admins. F12. Private window → https://auth.BASE/ → Discord → log in. Admin: Directory → Users → your user exists, type Internal. Directory → Groups → admins → Users tab → Add existing user → you (and media-admins, streamers as you like) — or give yourself the Discord roles. F13. Private window → https://HOST/whoami → "user: " and "groups: …" listing what you added. (If groups are missing: sign out at https://auth.BASE/if/flow/default-invalidation-flow/ and log in again.) Events → Logs → filter policy_execution → entries for both policies with result pass.

G. Gitea (browser at https://git.BASE/, log in with GITEA_ADMIN_USER / GITEA_ADMIN_PASSWORD from E4)

G1. Site Administration → Identity & Access → Authentication Sources → Add Authentication Source: Type OAuth2 · Name authentik · Provider OpenID Connect · Client ID / Secret from F11 · OpenID Connect Auto Discovery URL https://auth.BASE/application/o/gitea/.well-known/openid-configuration · Skip local 2FA ✓ · Additional scopes (blank) · Claim name providing group names: groups · Group claim value for administrator users: git-admins → Add. G2. New repository (top-right +) → owner techy → name Media-Server-Setup → NOT initialized (no README) → Create. Your SSH key: Settings → SSH / GPG Keys → Add Key (paste ~/.ssh/id_ed25519.pub from dev). G3. dev$ cd ~/claudevs/vanguard-stream && git remote add origin ssh://git@git.BASE:2222/techy/Media-Server-Setup.git && git push -u origin main (accept the host key for port 2222) → "branch 'main' set up to track". G4. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./registerGiteaRunner.sh' → log lines "runner: connect…" ; Site Administration → Actions → Runners lists vanguard-runner, Idle. G5. Private window → https://git.BASE/user/login → "Sign in with authentik" → Discord → logged in as your Discord username. (Requires git-users or git-admins membership; admins inherit git-admins via F5.)

H. Jellyfin (browser at https://watch.BASE/)

H1. Wizard: language → admin user + password → Add Media Library: Content type Shows · Display name Streams · Folder /media/streams · Metadata downloaders (Series, Season, Episode): tick ONLY "Vanguard Metadata" · Image fetchers: Series: Vanguard Metadata; Episode: Screen Grabber · Enable real time monitoring ✓ → OK. Add Media Library: Shows · Clips · /media/clips · same provider settings → OK. Add Media Library: Home Videos and Photos · Finals · /media/finals · all fetchers unticked → OK. Finish the wizard (metadata language, remote access defaults). H2. Dashboard → Playback → Transcoding → Hardware acceleration: None → Save. H3. Dashboard → Plugins → My Plugins → Vanguard Metadata → settings: Discord bot token (A2), Server ID (A3), feed channel ID (right-click the Discord channel → Copy Channel ID), Public URL https://watch.BASE, Authentik URL https://auth.BASE (token optional) → Save. H4. Dashboard → Plugins → Repositories → + → Name SSO-Auth · URL https://raw.githubusercontent.com/9p4/jellyfin-plugin-sso/manifest-release/manifest.json → Save. Catalog → SSO Authentication → Install. Then: dev$ ssh debian@HOST 'cd ~/vanguard-stream && docker compose restart jellyfin' H5. Dashboard → Plugins → SSO-Auth → OpenID: Add provider: name authentik · OID Endpoint https://auth.BASE/application/o/jellyfin/ · Client ID / Secret from F10 · Enabled ✓ · Enable Authorization by Plugin ✓ · Roles claim: groups · Admin roles: admins · Enabled folders: Streams, Clips, Finals · Scheme Override: https · Port Override: EMPTY · (default: disable transcoding ✓) → Save (bottom of page). H6. Private window → https://watch.BASE/sso/OID/start/authentik → Discord → lands in Jellyfin logged in. H7. Dashboard → General → Branding → Login disclaimer: paste jellyfin/branding-disclaimer.html with watch.HOST replaced by watch.BASE; Custom CSS: paste jellyfin/branding.css → Save. Login page now shows "Sign in with Discord" + a small "Admin login" link. H8. Dashboard → Scheduled Tasks → Vanguard: run "snapshot Discord events", then "roll seasons", then "enforce playback policy" (▶ each). Dashboard → Users → your SSO user → Playback → "Allow video playback that requires transcoding" is unticked.

I. Pipeline

I1. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./switchToClone.sh key' → prints a public key. Gitea → repo → Settings → Deploy Keys → Add Deploy Key → paste → read-only → Add. I2. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./switchToClone.sh clone && cd ~/vanguard-stream && ./configure.py --apply && sudo vanguard-deploy status' → "cloned. kept: …" ; status shows "main @ ", DEPLOY_AUTO=0. I3. dev$ ssh debian@HOST 'cd ~/vanguard-stream && docker network inspect vanguard-stream_default -f "{{(index .IPAM.Config 0).Gateway}}"; grep ^DEPLOY_WEBHOOK_SECRET= .env' → gateway IP (e.g. 172.18.0.1) and the secret. I4. Gitea → repo → Settings → Webhooks → Add Webhook → Gitea: Target URL http://:8081/deploy · HTTP POST · Content type JSON · Secret = the value from I3 · Trigger on Push events · Branch filter main · Active → Add Webhook. Open it → Test Delivery. dev$ ssh debian@HOST 'journalctl -u dumpd --since -5m --no-pager | grep "deploy webhook"' → "… auto-deploy disabled (DEPLOY_AUTO=0)". I5. From now on, after a push: dev$ ssh -t debian@HOST 'sudo vanguard-deploy pull' (shows the changed files and every action). I6. (later, when trusted) dev$ ssh debian@HOST 'cd ~/vanguard-stream && sed -i "s/^DEPLOY_AUTO=.*/DEPLOY_AUTO=1/" .env && ./configure.py --apply >/dev/null && sudo vanguard-deploy status' and dev$ cd ~/claudevs/vanguard-stream && git mv .gitea/draft-workflows/ci.yml .gitea/workflows/ci.yml && git commit -m "enable CI" && git push

J. People

J1. Streamer (OBS): Tools → WebSocket Server Settings → Enable, set a password → OK. Docks → Custom Browser Docks → name Vanguard, URL https://live.BASE/dock/ → Apply → log in with Discord inside the dock. Dock → Settings: OBS WebSocket password → Save. Dock → Timecode band → "Add band + clock source" (while not streaming). Dock: title → Go Live. Check: dev$ ssh debian@HOST 'journalctl -u dumpd --since -5m --no-pager | grep mtx-auth | tail -1' → "allow publish user=…". J2. Gamer: https://clips.BASE/ → follow the three steps on the page (install Syncthing, paste Device ID, accept device + folder as Send Only). Admin check: https://sync.BASE/ shows their device connected. Once: Syncthing → Actions → Settings → Connections → Incoming Rate Limit 12500 KiB/s. J3. Editor: https://files.BASE/ (copyparty) → browse, download; https://files.BASE/finals/ to upload. Needs the editors role. J4. Everyone: https://HOST/ (gateway page).

K. Verify the whole pipeline once

K1. Stream 1 minute from OBS with the band, stop, wait ~8 min. K2. dev$ ssh debian@HOST 'f=$(ls -t /srv/media/streams//.mp4 | head -1); echo "$f"; ffprobe -v error -show_entries stream_tags=timecode -of compact=nk=0:p=0 "$f" | head -1; python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print({k:d.get(k) for k in ("source","timecode","drift_ms")}, "gaps:", d.get("gaps",{}).get("count"))" "${f%.*}.json"' → a flat file named