Commit Graph

55 Commits

Author SHA1 Message Date
25e9078d9b Drop the last references to the bot's code living in this repo
switchToClone no longer tries to carry bot/app across a clone, and the compose comment, STATUS row,
README and the plan doc all describe the installed-release model instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-02 21:58:34 -06:00
1ee8befdee install.sh, deploy.sh and the runbooks cover the bot bundle path
install.sh installs deployBotBundle.sh as vanguard-deploy-bot and creates /srv/bot/app; the pipeline
reinstalls it when it changes. FRESH-INSTALL gains section 7a and appendix L (the first release, the
python312 runner label, the Gitea secrets, the Authentik app), with the appendix relettered so it stays in
order. README's rebuild list says the bot's repo is outside this one and what to restore where.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-02 21:57:52 -06:00
1e4c0e88dc The bot's secrets live in Gitea: signed settings travel with the release bundle
POST /deploy-bot may now carry X-Vanguard-Env, base64 JSON of the bot's own settings, and the HMAC covers
that header together with the body so the pair cannot be separated or swapped. dumpd merges only the keys
BOT_ENV_KEYS allows (DISCORD_BOT_TOKEN, DISCORD_GUILD_ID by default) into the stack .env, atomically and
preserving mode and owner, and reports the rest to the CI job as settings_refused. The server keeps what it
generates: a CI job cannot touch the database password or the Authentik keys.

Verified: allowed keys applied, server-owned keys refused while the rest still applied, a settings swap
after signing rejected with 401, and an empty settings header leaving .env untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-02 21:37:35 -06:00
b05f129bed Bot releases install themselves: signed bundle upload to dumpd, no Docker socket in CI
dumpd gains POST /deploy-bot on the bridge-only auth port: HMAC-SHA256 over the body with BOT_DEPLOY_SECRET,
a 300 MB cap, one install at a time, and the installer's output returned in the response so the CI job shows
it. BOT_DEPLOY_AUTO=0 makes the server refuse. configure.py generates the secret, writes both settings into
/etc/vanguard/deploy.env, installs deployBotBundle.sh as /usr/local/bin/vanguard-deploy-bot and creates
/srv/bot/app; --show prints what to paste into Gitea.

Chose this over mounting the Docker socket into job containers, which would give every CI job root on the
host. Also moved threading/subprocess to dumpd's module imports, which the new module-level lock needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-02 21:32:34 -06:00
22dcf893d5 deployBotBundle.sh: accept the artifact as Gitea serves it (a zip holding the tar.gz)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-02 21:20:04 -06:00
223458550a Bot hosting: releases are self-contained CI bundles, and the bot serves its own web UI
The image no longer installs the bot's dependencies: its CI builds one tar.gz holding the app plus a
vendor/ directory of prebuilt wheels, and PYTHONPATH=/app/vendor makes the code import them. A new
dependency therefore needs no image rebuild and the server contacts no package index.

- deployBotBundle.sh: install / --latest / --status / --rollback, verifies every file against BUILD.json
  and keeps the previous release beside the live one.
- compose: mount ${BOT_APP_DIR:-/srv/bot/app} at /app, set DB_BACKEND, DATA_DIR, WEB_ALLOWED_GROUPS,
  WEB_SERVE_UI and PYTHONPATH. bot/app placeholder untracked; install.sh creates /srv/bot/app.
- The placeholder requirements would have crashed the bot on import (no sqlglot), which is why the
  image now ships none of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-02 21:18:15 -06:00
704bc8c4d3 Logging framework: logfmt events in our scripts, container labels + rotated json-file logs, Caddy JSON access logs; LOGGING.md
Store-agnostic: journald + Docker logs are the only taps any collector (Alloy/Loki, VictoriaLogs, Netdata) needs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 19:53:05 -06:00
88bfda4c9d Sweep/vgdeband: cap x264 at ENCODE_THREADS (2) so conversions stop starving Jellyfin and the live portal
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 19:30:29 -06:00
2078a53419 UPCOMING.md: gap list, techy's take, agreed direction (MariaDB metadata, markers, future watch app)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 18:21:38 -06:00
9bdf9f1947 WebRTC audio mirror: AAC cannot cross WebRTC, so transcode audio to Opus into <path>.webrtc
bin/mtx-on-ready.sh (runOnReady inside the mediamtx container) posts the is-live notice and, for sources whose
audio is not Opus, runs ffmpeg: video copy, audio -> Opus, published over MediaMTX's loopback RTSP to
<user>/<Title>.webrtc (record: no). dumpd accepts .webrtc publishes only from 127.0.0.1. The portal hides
mirror paths, plays the mirror over WHEP (HLS stays on the source), holds an AAC source for 15 s while its
mirror comes up, and corrects the viewer count. Verified headless with a mocked path list.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 13:58:15 -06:00
f060277167 Streamer guidance: B-frames 0 (MediaMTX refuses WebRTC playback of H.264 with B-frames)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 13:29:34 -06:00
2cd936beaf Sweep: crop the timecode band off published streams (re-encode after decode); vgdeband.sh catch-up
The published recording is the game picture + audio + tmcd track, nothing else. Streams without the
band are still remuxed losslessly. vgstrip plan exports TC_BAND and marks the sidecar band_removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 13:05:35 -06:00
6128a4ef3a README: nested groups need a second scope mapping (managed profile mapping is direct-only and sticky on proxy providers)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 01:26:14 -06:00
18f4e07de8 Split hosts: media (gateway at /), files, clips, sync; live gains /dock, /dock/clock, /dump, /me, /time
configure.py: three new host roles (files/sync/clips) in --full, --hosts, .env, Caddy generator;
main host serves about/ at the root (/about* redirects); /time only where it is fetched same-origin.
about/index.html links across hosts; addCommunityMediaServer.sh asks for the new names; Caddyfile.example
regenerated; README/FRESH-INSTALL/HOSTNAME-CHANGE (procedure C)/STATUS updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 01:38:35 -05:00
c1a46d66dc HOSTNAME-CHANGE.md: hostname inventory, subdomain rename and base-domain move procedures; linked from README
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 01:23:33 -05:00
861a88fc7e FRESH-INSTALL.md: ordered walkthrough from empty box to working pipeline, manual steps included; linked from README
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 01:19:06 -05:00
49d33e8210 README: rollout block matches the current one (compose included, visible apply output)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 01:16:42 -05:00
abb7186bd0 Rebuild gaps closed: Authentik bootstrap creds, Gitea install lock + admin user from .env, Syncthing GUI-auth-off fixup, plugin ensured on apply, portal built by install.sh; README rebuild section with the honest manual remainder
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 01:14:46 -05:00
33482bfff3 README: push-to-deploy rollout commands, clone switch, webhook test, enabling steps
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 01:09:42 -05:00
66f05cf9a7 Push-to-deploy (OFF by default): vanguard-deploy applies only changed parts with a live-stream guard for MediaMTX, rollback, pending timer; dumpd /deploy webhook gated by DEPLOY_AUTO; configure --apply; switchToClone.sh; draft CI workflow; committed plugin zip
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 01:08:43 -05:00
07ffae0746 Soft per-user clip quotas: dumpd pauses/resumes Syncthing folders by folder size (CLIPS_QUOTA_GB, per-user overrides file); clips page shows usage
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 18:30:22 -05:00
48409b5dbf Self-service clip intake (/clips + dumpd /me/clips via Syncthing REST) and role-aware gateway page (/about + dumpd /me)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 18:00:01 -05:00
367d41fe18 addClipUser.sh + README: onboarding a gamer's clips via Syncthing
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 17:54:49 -05:00
7213460c10 Caddy: /logout on every gated host redirects to Authentik sign-out
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 17:53:02 -05:00
68c2866a28 Syncthing: GUI auth disabled behind the Authentik gate; docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 17:49:17 -05:00
52c065b753 roles policy: only revoke memberships it granted itself (manual adds survive); track in user attributes
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 17:34:43 -05:00
2ab0a4e837 Docs: git-admins claim with Authentik group nesting
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 17:27:51 -05:00
7e935c31d1 Gitea access gated by Authentik group git-users; roles table example
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 17:10:09 -05:00
b661ee795b Phase 10: Gitea OIDC login prerequisites in compose (external-only registration, auto-register, account linking); README steps
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 17:09:33 -05:00
6db786e00e Phase 12 verified live; MediaMTX config changes documented as restart-only on this setup
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:54:08 -05:00
f1d19ba32d Plan: off-site gap filling option (home NVENC / Spot GPU) and SYS-3 availability note
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:26:44 -05:00
9dc81ae5aa README/STATUS: gap filling availability and the on/off commands
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:25:08 -05:00
473247e710 Sweep: opt-in gap filling for streams (FILL_GAPS_MS) — CFR re-encode with duplicated frames, raw kept in .src/, timecode carried; default off
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:19:25 -05:00
f36487d4b5 Timecode: nominal fps + proper drop-frame detection, shell-safe plan output; streams never VFR-re-encoded; gap list in the sidecar; docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:17:26 -05:00
3fcb8fbe99 Phase 12: frame-accurate timecode via an overscan band — vgstrip encoder/decoder, sweep writes tmcd + creation_time + sidecar, dumpd /time, Caddy /time routes, MediaMTX microsecond names, /dock/clock/ page, dock band setup via obs-websocket, Go Live band composite + toggle, portal crops the band
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:07:56 -05:00
5085f8ab55 Docs/apply note: MediaMTX does not hot-reload auth settings; restart between streams after auth changes
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 15:46:25 -05:00
65aa8b0894 Docs: provider invalidation flow so every logout forces a full Discord re-login
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 15:38:01 -05:00
dcc347e1ea Phase 7+11: token-based publishing — MediaMTX authMethod http -> dumpd /mtx-auth (bridge-only port 8081); dumpd mints 12 h HMAC tokens for Discord-authenticated streamers; dock rewritten (no passwords, naming rules with live preview, prefix/suffix, manual-setup accordion with hidden token); portal uses tokens; configure.py drops streamers; MediaMTX hot-reload, Caddy reload — no stream interruption
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 15:22:15 -05:00
2a74cb7b40 Logout ends the Authentik session: copyparty idp-logout -> outpost sign_out; live portal Sign out link
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 15:11:53 -05:00
268a6a2f81 copyparty idp-h-key: Caddy injects a header with a secret name (COPYPARTY_IDP_KEY); IdP headers ignored without it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 15:09:13 -05:00
b89afb50d2 Caddy: /whoami on the main host echoes the Authentik identity headers for debugging
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 15:08:16 -05:00
c4a79c3316 README/plan: note WebDAV is available via app passwords but deliberately not set up
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 15:01:14 -05:00
15402eeb82 Discord-only login: drop WebDAV/app-password path from docs and generated config comments
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 15:00:48 -05:00
50c037e514 Phase 5+6: main host behind Authentik — copyparty trusts X-Authentik-* headers with group permissions (editors, media-admins), dumpd authorizes from headers (admins/media-admins), /sync gated to media-admins, dock drops its dump password; configure.py loses editors/admin
WebDAV continues via Authentik app passwords (provider: Intercept header authentication).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 14:55:48 -05:00
e02accb855 Streams published flat as <user>/<Title>__<ts>.mp4 so each recording is its own Jellyfin episode; plugin 0.1.3 parses the title from the name
Jellyfin merges several videos in one non-season folder into a single episode with versions,
which collapsed a night of helmet cams into one entry. Recording still lands in <Title>/.live/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 03:42:53 -05:00
f737fc6289 Go Live: state the Windows-only window-audio limitation in the UI and docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 01:47:45 -05:00
b634413773 Go Live: request window-scoped audio (windowAudio: window, Chrome 141+); keep the picker window-only
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 01:44:45 -05:00
b2869c3740 Generated configs out of the repo: *.example.* templates only; configure.py refuses to apply over CHANGEME placeholders
A deploy copied mediamtx.yml (template) over the server's generated file and the next apply
wiped all streamers. Generated files are now gitignored so scp -r cannot clobber them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 00:32:30 -05:00
384a7ed552 Per-host group gating in Caddy from X-Authentik-Groups; Discord roles -> Authentik groups policy; domain-level forward auth documented
BOT_REQUIRE_GROUP / LIVE_REQUIRE_GROUP in .env drive a header_regexp matcher after forward_auth
(403 before the app). authentik/discord-roles-to-groups.py syncs Discord roles into groups at
every login so access follows Discord roles without manual edits.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-30 23:22:20 -05:00
bee53f2718 README: domain-level forward auth as the recommended Authentik setup
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-30 23:19:45 -05:00