diff --git a/FRESH-INSTALL.md b/FRESH-INSTALL.md index 474abbe..6aa383e 100644 --- a/FRESH-INSTALL.md +++ b/FRESH-INSTALL.md @@ -121,3 +121,185 @@ Log in at `https://git.BASE/` as `GITEA_ADMIN_USER` / `GITEA_ADMIN_PASSWORD` fro - Pin `AUTHENTIK_TAG` in `.env` to the running version. Rotate any secret that was ever pasted into chat. - Back up `.env`, `/srv/appdata`, `/srv/media` (home archive puller: README §7). - `STATUS.md` = what runs and what is unfinished; `check.sh` before any commit. + +--- + +# Appendix — the runbook: every action, in order, with the exact commands + +Conventions: commands prefixed `dev$` run on your workstation (it has the repo at `~/claudevs/vanguard-stream` +and your SSH key); `srv$` run on the server over ssh as `debian`. Replace `HOST` / `BASE` with your names +(ours: `ovh1.dumpstervigilantes.rip` / `dumpstervigilantes.rip`). Click paths are written `App → Menu → Item`. +Every step ends with how you know it worked. + +## A. DNS and Discord (no server yet) + +A1. DNS at your registrar: + A HOST → + CNAME auth.BASE → HOST + CNAME watch.BASE → HOST + CNAME git.BASE → HOST + CNAME bot.BASE → HOST + CNAME live.BASE → HOST + (optional) A BASE → + Check from dev: dev$ for h in HOST auth.BASE watch.BASE git.BASE bot.BASE live.BASE; do printf '%-28s ' $h; getent hosts $h | awk '{print $1}'; done + → every line shows the server IP. + +A2. Discord Developer Portal → New Application "Vanguard". + OAuth2 → Redirects → add https://auth.BASE/source/oauth/callback/discord/ → Save. Copy Client ID and (Reset) Client Secret. + Bot → Reset Token → copy. Privileged Gateway Intents → Server Members Intent ON → Save. + OAuth2 → URL Generator → scopes: bot; permissions: View Channels, Send Messages, Embed Links, Attach Files → open the URL → invite the bot to your server. + → the bot appears in your server's member list. + +A3. Discord IDs: User Settings → Advanced → Developer Mode ON. Right-click your server → Copy Server ID. + Server Settings → Roles → right-click each role you'll map → Copy Role ID. Keep a note like: + guild=123… streamers=… editors=… media-admins=… admins=… bot-users=… git-users=… git-admins=… + +## B. Operating system + +B1. OVH manager → the server → Reinstall → Debian 13 → custom partitioning, "use distribution kernel": + /boot 1 GiB ext4 (RAID1) · / 250 GiB ext4 (RAID1) · swap 8 GiB (per disk) · /srv remaining xfs (RAID1). + Add your SSH public key. Wait for the mail. +B2. dev$ ssh debian@HOST 'lsblk -o NAME,SIZE,FSTYPE,MOUNTPOINT; df -h /srv' + → /srv is xfs, ~3.4 TiB, mounted. + +## C. Code onto the box + +C1. dev$ cd ~/claudevs && scp -r vanguard-stream debian@HOST:~/ +C2. dev$ ssh debian@HOST 'ls ~/vanguard-stream | head; test -f ~/vanguard-stream/install.sh && echo ok' + → "ok". + +## D. Host install + +D1. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && sudo bash install.sh' + (~5 min: packages, docker, timezone, folders, scripts, timers, portal build, firewall.) KEEP THIS TERMINAL OPEN. +D2. In a SECOND terminal: dev$ ssh debian@HOST 'echo still in' → "still in" (firewall did not lock you out). +D3. dev$ ssh debian@HOST 'timedatectl show -p Timezone --value; systemctl is-active docker dumpd normalize-sweep.timer trash-purge.timer nftables; docker --version; node --version; ls /srv/live | head -3' + → America/New_York; five "active"; versions; /srv/live has index.html. +D4. Log out/in once for the docker group: dev$ ssh debian@HOST 'docker ps >/dev/null && echo docker-ok' → docker-ok. + +## E. Configure and start the stack + +E1. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./configure.py --full' + Answers: HOST; Enter ×5 for the sub-hostnames; bare domain (Enter = BASE, or blank to skip) and its redirect; + bot token (from A2) ; server ID (A3) ; "is live" webhook (optional); y to apply. + (Pulls ~2 GB of images, starts 13 containers, waits, runs post-start fixups.) +E2. dev$ ssh debian@HOST 'cd ~/vanguard-stream && docker compose ps --format "{{.Name}} {{.Status}}"' + → every container "Up" (authentik-server/worker take ~1 min; act_runner restarts until step G3 — fine). +E3. dev$ for h in HOST auth.BASE watch.BASE git.BASE live.BASE bot.BASE; do printf '%-26s ' $h; curl -s -o /dev/null -m 10 -w '%{http_code}\n' https://$h/; done + → 200/302 for all (302 = redirected to login). If 000, wait a minute for certificates and retry. +E4. dev$ ssh debian@HOST 'cd ~/vanguard-stream && ./configure.py --show' + → copy out: akadmin password (AUTHENTIK_BOOTSTRAP_PASSWORD), Gitea admin user/password, MariaDB root + bot + gitea passwords. +E5. dev$ ssh debian@HOST 'journalctl -u dumpd -n 2 --no-pager | tail -1; grep -c "authMethod: http" ~/vanguard-stream/mediamtx.yml; sudo nft list table inet vanguard | grep -c dport' + → dumpd line ending "token-secret=ok"; 1; a dozen rules. + +## F. Authentik (browser, logged in as akadmin at https://auth.BASE/ with the E4 password) + +F1. Change akadmin's password: top-right user → Settings → Change password. +F2. Directory → Federation & Social login → Create → "Discord OAuth Source": + Name Discord · Slug discord · Enabled · User matching "Link users on unique identifier" · + Consumer key = Client ID · Consumer secret = Client Secret · Scopes: guilds guilds.members.read · + (Flow settings if shown: authentication default-source-authentication, enrollment default-source-enrollment) → Finish. +F3. Flows & Stages → Stages → default-authentication-identification → Edit → Sources: select Discord; tick "Show sources' labels" → Update. + → the login page (private window) now shows a Discord button. +F4. Flows & Stages → Stages → default-source-enrollment-write → Edit → User type: Internal · User path: users → Update. +F5. Directory → Groups → Create, one each: streamers, editors, media-admins, admins, bot-users, git-users, git-admins. + Then nesting: Groups → admins → Edit → Parent: authentik Admins → Update. Groups → authentik Admins → Edit → Parent: git-admins → Update. +F6. Customization → Policies → Create → Expression Policy: + name discord-guild-member · expression = contents of authentik/discord-guild-policy.py with ACCEPTED_GUILD_ID + GUILD_NAME filled in · tick Execution logging → Create. + Again: name discord-roles-to-groups · expression = authentik/discord-roles-to-groups.py with GUILD_ID + ROLE_TO_GROUP filled from A3 · Execution logging → Create. +F7. Flows & Stages → Flows → default-source-enrollment → Policy / Group / User Bindings → Bind existing policy → discord-guild-member (order 0) → Create; + again → discord-roles-to-groups (order 10) → Create. Then Edit the flow → Policy engine mode: all → Update. + Repeat all of F7 for default-source-authentication. +F8. Applications → Outposts → "authentik Embedded Outpost" → Edit → in the configuration box set + authentik_host: https://auth.BASE (leave authentik_host_browser empty) → Update. +F9. Applications → Create with provider: + Application name "Vanguard sites" · Provider type Proxy Provider · Authorization flow default-provider-authorization-implicit-consent · + Mode "Forward auth (domain level)" · Authentication URL https://auth.BASE · Cookie domain BASE · + Invalidation flow default-provider-invalidation-flow → Finish. + Applications → Outposts → Embedded Outpost → Edit → Applications: add "Vanguard sites" → Update. +F10. Applications → Create with provider: "Jellyfin" (slug jellyfin) · OAuth2/OpenID Provider · implicit-consent · + Client type Confidential · Redirect URI (strict) https://watch.BASE/sso/OID/redirect/authentik · + Signing key: the default certificate · Advanced → Scopes: openid, email, profile → Finish. Note Client ID + Secret. +F11. Applications → Create with provider: "Gitea" (slug gitea) · same provider settings · Redirect URI https://git.BASE/user/oauth2/authentik/callback → Finish. + Note Client ID + Secret. Applications → Gitea → Policy / Group / User Bindings → Bind existing group → git-users → Create; again → git-admins. +F12. Private window → https://auth.BASE/ → Discord → log in. Admin: Directory → Users → your user exists, type Internal. + Directory → Groups → admins → Users tab → Add existing user → you (and media-admins, streamers as you like) — or give yourself the Discord roles. +F13. Private window → https://HOST/whoami → "user: " and "groups: …" listing what you added. (If groups are missing: sign out at https://auth.BASE/if/flow/default-invalidation-flow/ and log in again.) + Events → Logs → filter policy_execution → entries for both policies with result pass. + +## G. Gitea (browser at https://git.BASE/, log in with GITEA_ADMIN_USER / GITEA_ADMIN_PASSWORD from E4) + +G1. Site Administration → Identity & Access → Authentication Sources → Add Authentication Source: + Type OAuth2 · Name authentik · Provider OpenID Connect · Client ID / Secret from F11 · + OpenID Connect Auto Discovery URL https://auth.BASE/application/o/gitea/.well-known/openid-configuration · + Skip local 2FA ✓ · Additional scopes (blank) · Claim name providing group names: groups · Group claim value for administrator users: git-admins → Add. +G2. New repository (top-right +) → owner techy → name Media-Server-Setup → NOT initialized (no README) → Create. + Your SSH key: Settings → SSH / GPG Keys → Add Key (paste ~/.ssh/id_ed25519.pub from dev). +G3. dev$ cd ~/claudevs/vanguard-stream && git remote add origin ssh://git@git.BASE:2222/techy/Media-Server-Setup.git && git push -u origin main + (accept the host key for port 2222) → "branch 'main' set up to track". +G4. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./registerGiteaRunner.sh' + → log lines "runner: connect…" ; Site Administration → Actions → Runners lists vanguard-runner, Idle. +G5. Private window → https://git.BASE/user/login → "Sign in with authentik" → Discord → logged in as your Discord username. + (Requires git-users or git-admins membership; admins inherit git-admins via F5.) + +## H. Jellyfin (browser at https://watch.BASE/) + +H1. Wizard: language → admin user + password → Add Media Library: + Content type Shows · Display name Streams · Folder /media/streams · Metadata downloaders (Series, Season, Episode): tick ONLY "Vanguard Metadata" · + Image fetchers: Series: Vanguard Metadata; Episode: Screen Grabber · Enable real time monitoring ✓ → OK. + Add Media Library: Shows · Clips · /media/clips · same provider settings → OK. + Add Media Library: Home Videos and Photos · Finals · /media/finals · all fetchers unticked → OK. + Finish the wizard (metadata language, remote access defaults). +H2. Dashboard → Playback → Transcoding → Hardware acceleration: None → Save. +H3. Dashboard → Plugins → My Plugins → Vanguard Metadata → settings: Discord bot token (A2), Server ID (A3), feed channel ID + (right-click the Discord channel → Copy Channel ID), Public URL https://watch.BASE, Authentik URL https://auth.BASE (token optional) → Save. +H4. Dashboard → Plugins → Repositories → + → Name SSO-Auth · URL https://raw.githubusercontent.com/9p4/jellyfin-plugin-sso/manifest-release/manifest.json → Save. + Catalog → SSO Authentication → Install. Then: dev$ ssh debian@HOST 'cd ~/vanguard-stream && docker compose restart jellyfin' +H5. Dashboard → Plugins → SSO-Auth → OpenID: Add provider: name authentik · OID Endpoint https://auth.BASE/application/o/jellyfin/ · + Client ID / Secret from F10 · Enabled ✓ · Enable Authorization by Plugin ✓ · Roles claim: groups · Admin roles: admins · + Enabled folders: Streams, Clips, Finals · Scheme Override: https · Port Override: EMPTY · (default: disable transcoding ✓) → Save (bottom of page). +H6. Private window → https://watch.BASE/sso/OID/start/authentik → Discord → lands in Jellyfin logged in. +H7. Dashboard → General → Branding → Login disclaimer: paste jellyfin/branding-disclaimer.html with watch.HOST replaced by watch.BASE; + Custom CSS: paste jellyfin/branding.css → Save. Login page now shows "Sign in with Discord" + a small "Admin login" link. +H8. Dashboard → Scheduled Tasks → Vanguard: run "snapshot Discord events", then "roll seasons", then "enforce playback policy" (▶ each). + Dashboard → Users → your SSO user → Playback → "Allow video playback that requires transcoding" is unticked. + +## I. Pipeline + +I1. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./switchToClone.sh key' + → prints a public key. Gitea → repo → Settings → Deploy Keys → Add Deploy Key → paste → read-only → Add. +I2. dev$ ssh -t debian@HOST 'cd ~/vanguard-stream && ./switchToClone.sh clone && cd ~/vanguard-stream && ./configure.py --apply && sudo vanguard-deploy status' + → "cloned. kept: …" ; status shows "main @ ", DEPLOY_AUTO=0. +I3. dev$ ssh debian@HOST 'cd ~/vanguard-stream && docker network inspect vanguard-stream_default -f "{{(index .IPAM.Config 0).Gateway}}"; grep ^DEPLOY_WEBHOOK_SECRET= .env' + → gateway IP (e.g. 172.18.0.1) and the secret. +I4. Gitea → repo → Settings → Webhooks → Add Webhook → Gitea: Target URL http://:8081/deploy · HTTP POST · Content type JSON · + Secret = the value from I3 · Trigger on Push events · Branch filter main · Active → Add Webhook. Open it → Test Delivery. + dev$ ssh debian@HOST 'journalctl -u dumpd --since -5m --no-pager | grep "deploy webhook"' → "… auto-deploy disabled (DEPLOY_AUTO=0)". +I5. From now on, after a push: dev$ ssh -t debian@HOST 'sudo vanguard-deploy pull' (shows the changed files and every action). +I6. (later, when trusted) dev$ ssh debian@HOST 'cd ~/vanguard-stream && sed -i "s/^DEPLOY_AUTO=.*/DEPLOY_AUTO=1/" .env && ./configure.py --apply >/dev/null && sudo vanguard-deploy status' + and dev$ cd ~/claudevs/vanguard-stream && git mv .gitea/draft-workflows/ci.yml .gitea/workflows/ci.yml && git commit -m "enable CI" && git push + +## J. People + +J1. Streamer (OBS): Tools → WebSocket Server Settings → Enable, set a password → OK. + Docks → Custom Browser Docks → name Vanguard, URL https://HOST/dock/ → Apply → log in with Discord inside the dock. + Dock → Settings: OBS WebSocket password → Save. Dock → Timecode band → "Add band + clock source" (while not streaming). + Dock: title → Go Live. Check: dev$ ssh debian@HOST 'journalctl -u dumpd --since -5m --no-pager | grep mtx-auth | tail -1' → "allow publish user=…". +J2. Gamer: https://HOST/clips/ → follow the three steps on the page (install Syncthing, paste Device ID, accept device + folder as Send Only). + Admin check: https://HOST/sync/ shows their device connected. Once: Syncthing → Actions → Settings → Connections → Incoming Rate Limit 12500 KiB/s. +J3. Editor: https://HOST/ (copyparty) → browse, download; https://HOST/finals/ to upload. Needs the editors role. +J4. Everyone: https://HOST/about/. + +## K. Verify the whole pipeline once + +K1. Stream 1 minute from OBS with the band, stop, wait ~8 min. +K2. dev$ ssh debian@HOST 'f=$(ls -t /srv/media/streams/*/*.mp4 | head -1); echo "$f"; ffprobe -v error -show_entries stream_tags=timecode -of compact=nk=0:p=0 "$f" | head -1; python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print({k:d.get(k) for k in (\"source\",\"timecode\",\"drift_ms\")}, \"gaps:\", d.get(\"gaps\",{}).get(\"count\"))" "${f%.*}.json"' + → a flat file named __<timestamp>.mp4, tag:timecode set, source "strip", gaps 0. +K3. https://watch.BASE/ → the recording is an episode under this weekday's season; Discord feed channel got a post with a frame grab. +K4. https://live.BASE/ while someone streams → a window with the corner chip "webrtc". + +## L. Afterwards + +L1. dev$ ssh debian@HOST 'docker inspect authentik-server --format "{{.Config.Image}}"' → put that tag in .env as AUTHENTIK_TAG, then ./configure.py --apply. +L2. Rotate anything pasted into a chat or screenshot (Discord client secret: portal → Reset → Authentik source → update). +L3. Backups: .env, /srv/appdata, /srv/media, /etc/vanguard/clip-quotas.json. Home archive puller: README §7.