Commit Graph

22 Commits

Author SHA1 Message Date
5c91fa7e99 Add vgbot-merge-sqlite.py: merge the bot's old SQLite history into the live MariaDB
The bot's own importer requires an empty destination, but the bot rebuilds part of its state from Discord on
every start, so after the backend switch neither side is a superset. This merges them: log tables append on a
natural key, and the profile/state tables resolve column by column with one explicit rule per column, so the
original joined_at, first_seen_at, rank started_at and tracking start survive a rebuild that stamped today,
while current rank, presence and latest activity stay fresh.

One transaction, idempotent, and it stops rather than guessing when a column has no rule or a table is absent
from the destination. Tested against a fixture built from the real 1041-row file plus a simulated rebuild:
history restored, fresh values kept, 30 membership events (29 historical + tonight's), re-run a no-op, and
both guards verified. A KeyError on insert-only tables during the second run was found and fixed that way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-03 00:01:19 -06:00
1e4c0e88dc The bot's secrets live in Gitea: signed settings travel with the release bundle
POST /deploy-bot may now carry X-Vanguard-Env, base64 JSON of the bot's own settings, and the HMAC covers
that header together with the body so the pair cannot be separated or swapped. dumpd merges only the keys
BOT_ENV_KEYS allows (DISCORD_BOT_TOKEN, DISCORD_GUILD_ID by default) into the stack .env, atomically and
preserving mode and owner, and reports the rest to the CI job as settings_refused. The server keeps what it
generates: a CI job cannot touch the database password or the Authentik keys.

Verified: allowed keys applied, server-owned keys refused while the rest still applied, a settings swap
after signing rejected with 401, and an empty settings header leaving .env untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-02 21:37:35 -06:00
b05f129bed Bot releases install themselves: signed bundle upload to dumpd, no Docker socket in CI
dumpd gains POST /deploy-bot on the bridge-only auth port: HMAC-SHA256 over the body with BOT_DEPLOY_SECRET,
a 300 MB cap, one install at a time, and the installer's output returned in the response so the CI job shows
it. BOT_DEPLOY_AUTO=0 makes the server refuse. configure.py generates the secret, writes both settings into
/etc/vanguard/deploy.env, installs deployBotBundle.sh as /usr/local/bin/vanguard-deploy-bot and creates
/srv/bot/app; --show prints what to paste into Gitea.

Chose this over mounting the Docker socket into job containers, which would give every CI job root on the
host. Also moved threading/subprocess to dumpd's module imports, which the new module-level lock needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-02 21:32:34 -06:00
704bc8c4d3 Logging framework: logfmt events in our scripts, container labels + rotated json-file logs, Caddy JSON access logs; LOGGING.md
Store-agnostic: journald + Docker logs are the only taps any collector (Alloy/Loki, VictoriaLogs, Netdata) needs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 19:53:05 -06:00
88bfda4c9d Sweep/vgdeband: cap x264 at ENCODE_THREADS (2) so conversions stop starving Jellyfin and the live portal
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 19:30:29 -06:00
f32730ba94 Clips folders: create an empty .stlocal with the folder (Syncthing refuses to scan while the #include target is missing)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 19:08:58 -06:00
9bdf9f1947 WebRTC audio mirror: AAC cannot cross WebRTC, so transcode audio to Opus into <path>.webrtc
bin/mtx-on-ready.sh (runOnReady inside the mediamtx container) posts the is-live notice and, for sources whose
audio is not Opus, runs ffmpeg: video copy, audio -> Opus, published over MediaMTX's loopback RTSP to
<user>/<Title>.webrtc (record: no). dumpd accepts .webrtc publishes only from 127.0.0.1. The portal hides
mirror paths, plays the mirror over WHEP (HLS stays on the source), holds an AAC source for 15 s while its
mirror comes up, and corrects the viewer count. Verified headless with a mocked path list.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 13:58:15 -06:00
2cd936beaf Sweep: crop the timecode band off published streams (re-encode after decode); vgdeband.sh catch-up
The published recording is the game picture + audio + tmcd track, nothing else. Streams without the
band are still remuxed losslessly. vgstrip plan exports TC_BAND and marks the sidecar band_removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 13:05:35 -06:00
66f05cf9a7 Push-to-deploy (OFF by default): vanguard-deploy applies only changed parts with a live-stream guard for MediaMTX, rollback, pending timer; dumpd /deploy webhook gated by DEPLOY_AUTO; configure --apply; switchToClone.sh; draft CI workflow; committed plugin zip
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 01:08:43 -05:00
07ffae0746 Soft per-user clip quotas: dumpd pauses/resumes Syncthing folders by folder size (CLIPS_QUOTA_GB, per-user overrides file); clips page shows usage
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 18:30:22 -05:00
48409b5dbf Self-service clip intake (/clips + dumpd /me/clips via Syncthing REST) and role-aware gateway page (/about + dumpd /me)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 18:00:01 -05:00
4101498ad7 Sweep: fix gap-total parsing so FILL_GAPS_MS actually triggers
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:19:58 -05:00
473247e710 Sweep: opt-in gap filling for streams (FILL_GAPS_MS) — CFR re-encode with duplicated frames, raw kept in .src/, timecode carried; default off
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:19:25 -05:00
f36487d4b5 Timecode: nominal fps + proper drop-frame detection, shell-safe plan output; streams never VFR-re-encoded; gap list in the sidecar; docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:17:26 -05:00
3fcb8fbe99 Phase 12: frame-accurate timecode via an overscan band — vgstrip encoder/decoder, sweep writes tmcd + creation_time + sidecar, dumpd /time, Caddy /time routes, MediaMTX microsecond names, /dock/clock/ page, dock band setup via obs-websocket, Go Live band composite + toggle, portal crops the band
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 16:07:56 -05:00
dcc347e1ea Phase 7+11: token-based publishing — MediaMTX authMethod http -> dumpd /mtx-auth (bridge-only port 8081); dumpd mints 12 h HMAC tokens for Discord-authenticated streamers; dock rewritten (no passwords, naming rules with live preview, prefix/suffix, manual-setup accordion with hidden token); portal uses tokens; configure.py drops streamers; MediaMTX hot-reload, Caddy reload — no stream interruption
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 15:22:15 -05:00
50c037e514 Phase 5+6: main host behind Authentik — copyparty trusts X-Authentik-* headers with group permissions (editors, media-admins), dumpd authorizes from headers (admins/media-admins), /sync gated to media-admins, dock drops its dump password; configure.py loses editors/admin
WebDAV continues via Authentik app passwords (provider: Intercept header authentication).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 14:55:48 -05:00
e02accb855 Streams published flat as <user>/<Title>__<ts>.mp4 so each recording is its own Jellyfin episode; plugin 0.1.3 parses the title from the name
Jellyfin merges several videos in one non-season folder into a single episode with versions,
which collapsed a night of helmet cams into one entry. Recording still lands in <Title>/.live/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 03:42:53 -05:00
2510cfaceb sweep: portable VFR detection (mawk has no asort; the check silently passed every file on Debian)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 01:20:32 -05:00
82df0b1583 Live portal: Go Live — browser window capture published over WHIP as <user>/HelmetCam-<date>; dumpd /me/stream-key
Go Live closes viewer windows, fetches the Authentik-identified user's MediaMTX key from dumpd
(/etc/vanguard/streamers.json written by configure.py), opens Chrome's window-only picker with
audio, prefers H.264 so MediaMTX can record, caps at 12 Mbps / 1440p60, shows live stats, Stop.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-30 23:04:33 -05:00
3d42271ae2 Record streams into hidden .live/ and publish finished files via the sweep; plugin 0.1.2 feed wording
Jellyfin indexed in-progress fragmented recordings (no duration -> seek jumped to next episode).
MediaMTX now writes under <Title>/.live/; the sweep remuxes finished files into the title folder
and removes the source. Feed says 'New recording' since it fires after publish.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-30 22:45:30 -05:00
c602a6ff52 Vanguard media stack: ingest, copyparty, Syncthing, normalize sweep, Authentik, Jellyfin + Vanguard Metadata plugin, MariaDB, Gitea, bot runtime
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-30 15:43:30 -05:00