FRESH-INSTALL: appendix runbook with every action and command, A–L

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

FRESH-INSTALL.md

@@ -121,3 +121,185 @@ Log in at `https://git.BASE/` as `GITEA_ADMIN_USER` / `GITEA_ADMIN_PASSWORD` fro
 - Pin `AUTHENTIK_TAG` in `.env` to the running version. Rotate any secret that was ever pasted into chat.
 - Back up `.env`, `/srv/appdata`, `/srv/media` (home archive puller: README §7).
 - `STATUS.md` = what runs and what is unfinished; `check.sh` before any commit.
+
+---
+
+# Appendix — the runbook: every action, in order, with the exact commands
+
+Conventions: commands prefixed `dev$` run on your workstation (it has the repo at `~/claudevs/vanguard-stream`
+and your SSH key); `srv$` run on the server over ssh as `debian`. Replace `HOST` / `BASE` with your names
+(ours: `ovh1.dumpstervigilantes.rip` / `dumpstervigilantes.rip`). Click paths are written `App → Menu → Item`.
+Every step ends with how you know it worked.
+
+## A. DNS and Discord (no server yet)
+
+A1. DNS at your registrar:
+    A      HOST            → <server IP>
+    CNAME  auth.BASE       → HOST
+    CNAME  watch.BASE      → HOST
+    CNAME  git.BASE        → HOST
+    CNAME  bot.BASE        → HOST
+    CNAME  live.BASE       → HOST
+    (optional) A  BASE     → <server IP>
+    Check from dev:  dev$ for h in HOST auth.BASE watch.BASE git.BASE bot.BASE live.BASE; do printf '%-28s ' $h; getent hosts $h | awk '{print $1}'; done
+    → every line shows the server IP.
+
+A2. Discord Developer Portal → New Application "Vanguard".
+    OAuth2 → Redirects → add  https://auth.BASE/source/oauth/callback/discord/  → Save. Copy Client ID and (Reset) Client Secret.
+    Bot → Reset Token → copy. Privileged Gateway Intents → Server Members Intent ON → Save.
+    OAuth2 → URL Generator → scopes: bot; permissions: View Channels, Send Messages, Embed Links, Attach Files → open the URL → invite the bot to your server.
+    → the bot appears in your server's member list.
+
+A3. Discord IDs: User Settings → Advanced → Developer Mode ON. Right-click your server → Copy Server ID.
+    Server Settings → Roles → right-click each role you'll map → Copy Role ID. Keep a note like:
+    guild=123…  streamers=…  editors=…  media-admins=…  admins=…  bot-users=…  git-users=…  git-admins=…
+
+## B. Operating system
+
+B1. OVH manager → the server → Reinstall → Debian 13 → custom partitioning, "use distribution kernel":
+    /boot 1 GiB ext4 (RAID1) · / 250 GiB ext4 (RAID1) · swap 8 GiB (per disk) · /srv remaining xfs (RAID1).
+    Add your SSH public key. Wait for the mail.
+B2. dev$ ssh debian@HOST 'lsblk -o NAME,SIZE,FSTYPE,MOUNTPOINT; df -h /srv'
+    → /srv is xfs, ~3.4 TiB, mounted.
+
+## C. Code onto the box
+
+C1. dev$ cd ~/claudevs && scp -r vanguard-stream debian@HOST:~/
+C2. dev$ ssh debian@HOST 'ls ~/vanguard-stream | head; test -f ~/vanguard-stream/install.sh && echo ok'
+    → "ok".
+
+## D. Host install
+
+D1. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && sudo bash install.sh'
+    (~5 min: packages, docker, timezone, folders, scripts, timers, portal build, firewall.) KEEP THIS TERMINAL OPEN.
+D2. In a SECOND terminal:  dev$ ssh debian@HOST 'echo still in'  → "still in" (firewall did not lock you out).
+D3. dev$ ssh debian@HOST 'timedatectl show -p Timezone --value; systemctl is-active docker dumpd normalize-sweep.timer trash-purge.timer nftables; docker --version; node --version; ls /srv/live | head -3'
+    → America/New_York; five "active"; versions; /srv/live has index.html.
+D4. Log out/in once for the docker group:  dev$ ssh debian@HOST 'docker ps >/dev/null && echo docker-ok'  → docker-ok.
+
+## E. Configure and start the stack
+
+E1. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./configure.py --full'
+    Answers: HOST; Enter ×5 for the sub-hostnames; bare domain (Enter = BASE, or blank to skip) and its redirect;
+    bot token (from A2) ; server ID (A3) ; "is live" webhook (optional); y to apply.
+    (Pulls ~2 GB of images, starts 13 containers, waits, runs post-start fixups.)
+E2. dev$ ssh debian@HOST 'cd ~/vanguard-stream && docker compose ps --format "{{.Name}} {{.Status}}"'
+    → every container "Up" (authentik-server/worker take ~1 min; act_runner restarts until step G3 — fine).
+E3. dev$ for h in HOST auth.BASE watch.BASE git.BASE live.BASE bot.BASE; do printf '%-26s ' $h; curl -s -o /dev/null -m 10 -w '%{http_code}\n' https://$h/; done
+    → 200/302 for all (302 = redirected to login). If 000, wait a minute for certificates and retry.
+E4. dev$ ssh debian@HOST 'cd ~/vanguard-stream && ./configure.py --show'
+    → copy out: akadmin password (AUTHENTIK_BOOTSTRAP_PASSWORD), Gitea admin user/password, MariaDB root + bot + gitea passwords.
+E5. dev$ ssh debian@HOST 'journalctl -u dumpd -n 2 --no-pager | tail -1; grep -c "authMethod: http" ~/vanguard-stream/mediamtx.yml; sudo nft list table inet vanguard | grep -c dport'
+    → dumpd line ending "token-secret=ok"; 1; a dozen rules.
+
+## F. Authentik (browser, logged in as akadmin at https://auth.BASE/ with the E4 password)
+
+F1. Change akadmin's password: top-right user → Settings → Change password.
+F2. Directory → Federation & Social login → Create → "Discord OAuth Source":
+    Name Discord · Slug discord · Enabled · User matching "Link users on unique identifier" ·
+    Consumer key = Client ID · Consumer secret = Client Secret · Scopes: guilds guilds.members.read ·
+    (Flow settings if shown: authentication default-source-authentication, enrollment default-source-enrollment) → Finish.
+F3. Flows & Stages → Stages → default-authentication-identification → Edit → Sources: select Discord; tick "Show sources' labels" → Update.
+    → the login page (private window) now shows a Discord button.
+F4. Flows & Stages → Stages → default-source-enrollment-write → Edit → User type: Internal · User path: users → Update.
+F5. Directory → Groups → Create, one each: streamers, editors, media-admins, admins, bot-users, git-users, git-admins.
+    Then nesting: Groups → admins → Edit → Parent: authentik Admins → Update. Groups → authentik Admins → Edit → Parent: git-admins → Update.
+F6. Customization → Policies → Create → Expression Policy:
+    name discord-guild-member · expression = contents of authentik/discord-guild-policy.py with ACCEPTED_GUILD_ID + GUILD_NAME filled in · tick Execution logging → Create.
+    Again: name discord-roles-to-groups · expression = authentik/discord-roles-to-groups.py with GUILD_ID + ROLE_TO_GROUP filled from A3 · Execution logging → Create.
+F7. Flows & Stages → Flows → default-source-enrollment → Policy / Group / User Bindings → Bind existing policy → discord-guild-member (order 0) → Create;
+    again → discord-roles-to-groups (order 10) → Create. Then Edit the flow → Policy engine mode: all → Update.
+    Repeat all of F7 for default-source-authentication.
+F8. Applications → Outposts → "authentik Embedded Outpost" → Edit → in the configuration box set
+    authentik_host: https://auth.BASE  (leave authentik_host_browser empty) → Update.
+F9. Applications → Create with provider:
+    Application name "Vanguard sites" · Provider type Proxy Provider · Authorization flow default-provider-authorization-implicit-consent ·
+    Mode "Forward auth (domain level)" · Authentication URL https://auth.BASE · Cookie domain BASE ·
+    Invalidation flow default-provider-invalidation-flow → Finish.
+    Applications → Outposts → Embedded Outpost → Edit → Applications: add "Vanguard sites" → Update.
+F10. Applications → Create with provider: "Jellyfin" (slug jellyfin) · OAuth2/OpenID Provider · implicit-consent ·
+     Client type Confidential · Redirect URI (strict) https://watch.BASE/sso/OID/redirect/authentik ·
+     Signing key: the default certificate · Advanced → Scopes: openid, email, profile → Finish. Note Client ID + Secret.
+F11. Applications → Create with provider: "Gitea" (slug gitea) · same provider settings · Redirect URI https://git.BASE/user/oauth2/authentik/callback → Finish.
+     Note Client ID + Secret. Applications → Gitea → Policy / Group / User Bindings → Bind existing group → git-users → Create; again → git-admins.
+F12. Private window → https://auth.BASE/ → Discord → log in. Admin: Directory → Users → your user exists, type Internal.
+     Directory → Groups → admins → Users tab → Add existing user → you (and media-admins, streamers as you like) — or give yourself the Discord roles.
+F13. Private window → https://HOST/whoami → "user: <you>" and "groups: …" listing what you added. (If groups are missing: sign out at https://auth.BASE/if/flow/default-invalidation-flow/ and log in again.)
+     Events → Logs → filter policy_execution → entries for both policies with result pass.
+
+## G. Gitea (browser at https://git.BASE/, log in with GITEA_ADMIN_USER / GITEA_ADMIN_PASSWORD from E4)
+
+G1. Site Administration → Identity & Access → Authentication Sources → Add Authentication Source:
+    Type OAuth2 · Name authentik · Provider OpenID Connect · Client ID / Secret from F11 ·
+    OpenID Connect Auto Discovery URL https://auth.BASE/application/o/gitea/.well-known/openid-configuration ·
+    Skip local 2FA ✓ · Additional scopes (blank) · Claim name providing group names: groups · Group claim value for administrator users: git-admins → Add.
+G2. New repository (top-right +) → owner techy → name Media-Server-Setup → NOT initialized (no README) → Create.
+    Your SSH key: Settings → SSH / GPG Keys → Add Key (paste ~/.ssh/id_ed25519.pub from dev).
+G3. dev$ cd ~/claudevs/vanguard-stream && git remote add origin ssh://git@git.BASE:2222/techy/Media-Server-Setup.git && git push -u origin main
+    (accept the host key for port 2222) → "branch 'main' set up to track".
+G4. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./registerGiteaRunner.sh'
+    → log lines "runner: connect…" ; Site Administration → Actions → Runners lists vanguard-runner, Idle.
+G5. Private window → https://git.BASE/user/login → "Sign in with authentik" → Discord → logged in as your Discord username.
+    (Requires git-users or git-admins membership; admins inherit git-admins via F5.)
+
+## H. Jellyfin (browser at https://watch.BASE/)
+
+H1. Wizard: language → admin user + password → Add Media Library:
+    Content type Shows · Display name Streams · Folder /media/streams · Metadata downloaders (Series, Season, Episode): tick ONLY "Vanguard Metadata" ·
+    Image fetchers: Series: Vanguard Metadata; Episode: Screen Grabber · Enable real time monitoring ✓ → OK.
+    Add Media Library: Shows · Clips · /media/clips · same provider settings → OK.
+    Add Media Library: Home Videos and Photos · Finals · /media/finals · all fetchers unticked → OK.
+    Finish the wizard (metadata language, remote access defaults).
+H2. Dashboard → Playback → Transcoding → Hardware acceleration: None → Save.
+H3. Dashboard → Plugins → My Plugins → Vanguard Metadata → settings: Discord bot token (A2), Server ID (A3), feed channel ID
+    (right-click the Discord channel → Copy Channel ID), Public URL https://watch.BASE, Authentik URL https://auth.BASE (token optional) → Save.
+H4. Dashboard → Plugins → Repositories → + → Name SSO-Auth · URL https://raw.githubusercontent.com/9p4/jellyfin-plugin-sso/manifest-release/manifest.json → Save.
+    Catalog → SSO Authentication → Install. Then:  dev$ ssh debian@HOST 'cd ~/vanguard-stream && docker compose restart jellyfin'
+H5. Dashboard → Plugins → SSO-Auth → OpenID: Add provider: name authentik · OID Endpoint https://auth.BASE/application/o/jellyfin/ ·
+    Client ID / Secret from F10 · Enabled ✓ · Enable Authorization by Plugin ✓ · Roles claim: groups · Admin roles: admins ·
+    Enabled folders: Streams, Clips, Finals · Scheme Override: https · Port Override: EMPTY · (default: disable transcoding ✓) → Save (bottom of page).
+H6. Private window → https://watch.BASE/sso/OID/start/authentik → Discord → lands in Jellyfin logged in.
+H7. Dashboard → General → Branding → Login disclaimer: paste jellyfin/branding-disclaimer.html with watch.HOST replaced by watch.BASE;
+    Custom CSS: paste jellyfin/branding.css → Save. Login page now shows "Sign in with Discord" + a small "Admin login" link.
+H8. Dashboard → Scheduled Tasks → Vanguard: run "snapshot Discord events", then "roll seasons", then "enforce playback policy" (▶ each).
+    Dashboard → Users → your SSO user → Playback → "Allow video playback that requires transcoding" is unticked.
+
+## I. Pipeline
+
+I1. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./switchToClone.sh key'
+    → prints a public key. Gitea → repo → Settings → Deploy Keys → Add Deploy Key → paste → read-only → Add.
+I2. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./switchToClone.sh clone && cd ~/vanguard-stream && ./configure.py --apply && sudo vanguard-deploy status'
+    → "cloned. kept: …" ; status shows "main @ <commit>", DEPLOY_AUTO=0.
+I3. dev$ ssh debian@HOST 'cd ~/vanguard-stream && docker network inspect vanguard-stream_default -f "{{(index .IPAM.Config 0).Gateway}}"; grep ^DEPLOY_WEBHOOK_SECRET= .env'
+    → gateway IP (e.g. 172.18.0.1) and the secret.
+I4. Gitea → repo → Settings → Webhooks → Add Webhook → Gitea: Target URL http://<gateway>:8081/deploy · HTTP POST · Content type JSON ·
+    Secret = the value from I3 · Trigger on Push events · Branch filter main · Active → Add Webhook. Open it → Test Delivery.
+    dev$ ssh debian@HOST 'journalctl -u dumpd --since -5m --no-pager | grep "deploy webhook"'  → "… auto-deploy disabled (DEPLOY_AUTO=0)".
+I5. From now on, after a push:  dev$ ssh -t debian@HOST 'sudo vanguard-deploy pull'  (shows the changed files and every action).
+I6. (later, when trusted) dev$ ssh debian@HOST 'cd ~/vanguard-stream && sed -i "s/^DEPLOY_AUTO=.*/DEPLOY_AUTO=1/" .env && ./configure.py --apply >/dev/null && sudo vanguard-deploy status'
+    and  dev$ cd ~/claudevs/vanguard-stream && git mv .gitea/draft-workflows/ci.yml .gitea/workflows/ci.yml && git commit -m "enable CI" && git push
+
+## J. People
+
+J1. Streamer (OBS): Tools → WebSocket Server Settings → Enable, set a password → OK.
+    Docks → Custom Browser Docks → name Vanguard, URL https://HOST/dock/ → Apply → log in with Discord inside the dock.
+    Dock → Settings: OBS WebSocket password → Save. Dock → Timecode band → "Add band + clock source" (while not streaming).
+    Dock: title → Go Live.  Check:  dev$ ssh debian@HOST 'journalctl -u dumpd --since -5m --no-pager | grep mtx-auth | tail -1'  → "allow publish user=…".
+J2. Gamer: https://HOST/clips/ → follow the three steps on the page (install Syncthing, paste Device ID, accept device + folder as Send Only).
+    Admin check: https://HOST/sync/ shows their device connected. Once: Syncthing → Actions → Settings → Connections → Incoming Rate Limit 12500 KiB/s.
+J3. Editor: https://HOST/ (copyparty) → browse, download; https://HOST/finals/ to upload. Needs the editors role.
+J4. Everyone: https://HOST/about/.
+
+## K. Verify the whole pipeline once
+
+K1. Stream 1 minute from OBS with the band, stop, wait ~8 min.
+K2. dev$ ssh debian@HOST 'f=$(ls -t /srv/media/streams/*/*.mp4 | head -1); echo "$f"; ffprobe -v error -show_entries stream_tags=timecode -of compact=nk=0:p=0 "$f" | head -1; python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print({k:d.get(k) for k in (\"source\",\"timecode\",\"drift_ms\")}, \"gaps:\", d.get(\"gaps\",{}).get(\"count\"))" "${f%.*}.json"'
+    → a flat file named <Title>__<timestamp>.mp4, tag:timecode set, source "strip", gaps 0.
+K3. https://watch.BASE/ → the recording is an episode under this weekday's season; Discord feed channel got a post with a frame grab.
+K4. https://live.BASE/ while someone streams → a window with the corner chip "webrtc".
+
+## L. Afterwards
+
+L1. dev$ ssh debian@HOST 'docker inspect authentik-server --format "{{.Config.Image}}"'  → put that tag in .env as AUTHENTIK_TAG, then ./configure.py --apply.
+L2. Rotate anything pasted into a chat or screenshot (Discord client secret: portal → Reset → Authentik source → update).
+L3. Backups: .env, /srv/appdata, /srv/media, /etc/vanguard/clip-quotas.json. Home archive puller: README §7.