Vanguard media stack: ingest, copyparty, Syncthing, normalize sweep, Authentik, Jellyfin + Vanguard Metadata plugin, MariaDB, Gitea, bot runtime

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-10-30 15:43:30 -05:00
commit c602a6ff52
66 changed files with 4229 additions and 0 deletions

78
STATUS.md Normal file
View File

@@ -0,0 +1,78 @@
# Vanguard stack — what runs, what people do, what's unfinished
Living document. Updated whenever something is deployed or changes behaviour. Last update: 2026-10-30.
Host: `ovh1.dumpstervigilantes.rip` (OVH SYS-1, Debian 13). Bundle: `~/vanguard-stream` on the box, `vanguard-stream/` in this repo.
## Runs on its own
| Cadence | What happens | Done by |
|---|---|---|
| Continuous | OBS streams land as files the moment Go Live is pressed, one per stream under `streams/<user>/<Title>/` | MediaMTX |
| Continuous | Gamers' clip folders mirror onto the server (video files only, their structure untouched, receive-only) | Syncthing |
| Continuous | New recordings and clips are announced in a Discord channel with the streamer's name and a frame grab | Jellyfin plugin (needs bot token + channel in its config) |
| Every 5 min | Finished files are checked. Streams: lossless remux so they seek instantly. Non-compliant clips: converted to the editing standard, raw removed, one copy per clip | normalize sweep (systemd timer) |
| Every 5 min | Discord scheduled events snapshotted into a history so recordings can be matched to their session | Jellyfin plugin |
| Every 30 min + startup | Every non-admin Jellyfin user set to direct-play-only (no hardware encoder on this box) | Jellyfin plugin |
| Daily 04:10 ET | Recordings roll between weekday → week → month → year seasons as the calendar moves; no files change | Jellyfin plugin |
| Daily 05:00 ET | Dumped nights older than 3 days deleted for good | trash purge (systemd timer) |
| On boot | Everything restarts itself after the media mount is up; interrupted recordings stay playable | systemd + Docker |
| **Not yet running** | Pull of files older than 15 days to the home HDD | `home/rclone-archive.sh`, needs the home box |
| **Not yet running** | Nightly backup of the Authentik database (Postgres) | planned |
## What people do
| Who | Activity | Where | Login |
|---|---|---|---|
| Streamer | Type a title, Go Live, Stop. "Dump tonight" if it was junk (3-day undo) | OBS browser dock `https://ovh1…/dock/` | stream password in dock settings |
| Gamer | Nothing; clips save where they always did | their clipper + Syncthing, paired once | — |
| Editor | Browse by streamer/title, scrub in browser, download originals, upload finals | copyparty `https://ovh1…/` | **local account** (to move to Discord) |
| Editor | Mount the library as a drive | WebDAV, same URL | same |
| Anyone in the Discord server | Watch recordings and clips by streamer and season | Jellyfin `https://watch.…/` | Discord (Authentik) |
| Devs | Host code, review, CI | Gitea `https://git.…/`, ssh 2222 | **local account** (Discord later) |
| Bot dev | Push code, restart container, read logs; web UI at `https://bot.…/` | Gitea + compose; `bot/AGENT.md` | web UI behind Discord (Authentik), once its provider exists |
| Admin | Add/remove streamers & editors, hostnames, bot token | `./configure.py` | ssh |
| Admin | Pair a gamer's device, approve their folder | Syncthing UI `https://ovh1…/sync/` | Syncthing GUI password |
| Admin | Groups, who may log in, app access | Authentik `https://auth.…/` | Discord or akadmin |
| Admin | Install/upgrade the Jellyfin plugin | `./installJellyfinPlugin.sh` | ssh |
| Admin | Register the CI runner | `./registerGiteaRunner.sh` | ssh |
## Services and their state
| Service | Role | State |
|---|---|---|
| MediaMTX | RTMP/SRT ingest, records fMP4 | live |
| copyparty | browse/scrub/download/WebDAV/finals upload | live, local accounts |
| dumpd + dock | dump-tonight endpoint, OBS dock page | live, own password file |
| Syncthing | clip intake, UI at `/sync/` | live, Syncthing GUI login |
| normalize sweep | editing standard, one copy per clip | live |
| nftables | host firewall (22, 80/443, 1935, 2222, 3306, 8890/udp, 22000) | live |
| Caddy | TLS + routing for all hostnames | live |
| Authentik (+Postgres, Redis) | Discord login, groups, guild-membership policy | live |
| Jellyfin | watch page, SSO via Authentik, direct play only | live |
| Vanguard Metadata plugin | seasons by tag, Discord names/avatars/events, feed, playback policy | **0.1.0 live, 0.1.1 built** (policy task) — install pending |
| MariaDB | databases for the bot and Gitea; public :3306, root local-only | live |
| Gitea | git hosting on MariaDB, Actions enabled | live, local admin |
| act_runner | CI jobs in Docker | **not registered** |
| discord-bot | Python runtime, idles until `bot/app/main.py` exists | live, no code yet |
| `bot.` host | bot web UI behind Authentik forward-auth | Caddy block ready; **CNAME + Authentik proxy provider pending** |
## Unfinished, in rough priority
1. **Install plugin 0.1.1** (`./installJellyfinPlugin.sh`), run "enforce playback policy", check a user.
2. **Register the runner** (`./registerGiteaRunner.sh`).
3. **`bot` CNAME + Authentik proxy provider** — first use of forward-auth; proves the pattern for copyparty.
4. **copyparty behind Authentik** (group permissions, WebDAV via app passwords, then drop local accounts). The one change that can lock editors out; keep a local admin until a Discord editor has used the new path.
5. **Syncthing UI + dump behind Authentik**; dump reads the identity header; dock fetches the caller's own stream key.
6. **Gitea login via Authentik** (OIDC source, auto-register, `admins` → Gitea admin).
7. **Housekeeping**: pin `AUTHENTIK_TAG` in `.env`, rotate the Discord client secret that was in a screenshot, nightly Postgres dump, Jellyfin login branding (Discord button first), check a non-admin Discord user's Jellyfin view.
8. **Medal filename pattern** in the plugin's timestamp parser — need one sample filename.
9. **Home archive puller** on the home box.
10. **Plugin repository** served by Caddy so Jellyfin updates the plugin from its catalog.
11. **Clip submissions** (Medal/Twitch/YouTube links, Discord uploads) — belongs to the bot.
12. **SYS-3** when transcoding or sweep speed hurts: Quick Sync, flip the plugin's policy toggle off.
## Verified behaviours worth remembering
- Jellyfin honours provider-supplied season numbers on a Shows library (day seasons appeared). Next check: Monday 04:10 ET should fold them into a week season.
- Sweep leaves compliant Syncthing files untouched and keeps exactly one copy of converted ones (tested).
- Firewall: Docker-published ports bypass the input chain; MariaDB uses host networking precisely so the rules apply.
- Never `systemctl stop|restart nftables` (flushes Docker's NAT). Reload only.