From b05f129bed0dcb24a31cc38fdc94f0b088a10d63 Mon Sep 17 00:00:00 2001 From: Brandyn / Techy Date: Mon, 2 Nov 2026 21:32:34 -0600 Subject: [PATCH] Bot releases install themselves: signed bundle upload to dumpd, no Docker socket in CI dumpd gains POST /deploy-bot on the bridge-only auth port: HMAC-SHA256 over the body with BOT_DEPLOY_SECRET, a 300 MB cap, one install at a time, and the installer's output returned in the response so the CI job shows it. BOT_DEPLOY_AUTO=0 makes the server refuse. configure.py generates the secret, writes both settings into /etc/vanguard/deploy.env, installs deployBotBundle.sh as /usr/local/bin/vanguard-deploy-bot and creates /srv/bot/app; --show prints what to paste into Gitea. Chose this over mounting the Docker socket into job containers, which would give every CI job root on the host. Also moved threading/subprocess to dumpd's module imports, which the new module-level lock needs. Co-Authored-By: Claude Opus 5 (1M context) --- LOGGING.md | 4 ++-- README.md | 27 +++++++++++++++++----- STATUS.md | 2 +- bin/dumpd.py | 57 +++++++++++++++++++++++++++++++++++++++++++--- bot/AGENT.md | 17 +++++++++----- bot/README.md | 11 +++++---- configure.py | 8 ++++++- deployBotBundle.sh | 9 +++++++- 8 files changed, 111 insertions(+), 24 deletions(-) diff --git a/LOGGING.md b/LOGGING.md index 44d86a6..4eb2b36 100644 --- a/LOGGING.md +++ b/LOGGING.md @@ -38,7 +38,7 @@ into stream labels without parsing a single line. | `media` | sweep verdicts / timecode / deband / failed, vgdeband, jellyfin, copyparty | `file` | | `intake` | syncthing (`folder.id`), dumpd `clips_setup`, `quota_*` | `folder` = `clips-` | | `infra` | mariadb, caddy TLS/ACME, container restarts, disk | host only | -| `deploy` | deploy.sh, dumpd `deploy_*`, act_runner | `rev` | +| `deploy` | deploy.sh, dumpd `deploy_*` and `bot_deploy_*` (the bot's CI posting a release bundle), act_runner | `rev` | | `code`, `bot` | gitea, discord-bot | — | ## Labels vs fields @@ -55,7 +55,7 @@ into stream labels without parsing a single line. - Same key means the same thing everywhere: `stream` (user/Title), `user`, `file`, `folder`, `rev`, `ip`, `reason`. - Implemented by: `log()` in `bin/dumpd.py`, `vglog` in `bin/normalize-sweep.sh` and `bin/vgdeband.sh`, `log()` in `deploy.sh` (`msg=` only), `bin/mtx-on-ready.sh` (MediaMTX prefixes its own path/time). -Events today: dumpd `publish_allow publish_deny dump clips_setup quota_paused quota_resumed quota_* deploy_*`; +Events today: dumpd `publish_allow publish_deny dump clips_setup quota_paused quota_resumed quota_* deploy_* bot_deploy_start bot_deploy_exit bot_deploy_bad_signature bot_deploy_refused`; sweep `verdict timecode deband fill_gaps published done failed skip`; vgdeband `deband done failed skip`; hook `mirror`. A future alert script watches `level=error`, `event=failed`, `event=publish_deny` bursts, `event=quota_paused`, plus disk. diff --git a/README.md b/README.md index c255294..f7ea4b4 100644 --- a/README.md +++ b/README.md @@ -240,12 +240,27 @@ for the bot, and `discord-bot`, a Python 3.12 container that runs `/app/main.py` installed. Credentials live in `.env` (generated by `configure.py`; bot token + guild ID are asked in `--full` or the `[d]` menu item; `./configure.py --show` prints them). -**Releases are self-contained bundles, not loose files.** The bot repo's Gitea Actions workflow runs its tests -and then builds one tar.gz holding `main.py`, `app/`, `website/`, `BUILD.json` **and `vendor/`** — its own -dependencies, as prebuilt wheels for this image's Python. `./deployBotBundle.sh ` verifies it -against `BUILD.json`, swaps it into `/srv/bot/app` (bind-mounted at `/app`, previous release kept beside it) -and restarts the container. `PYTHONPATH=/app/vendor` is how the code finds its dependencies, so **a new -dependency never needs an image rebuild** and the server never contacts a package index. +**Releases are self-contained bundles, and a push to `main` deploys itself.** The bot repo's Gitea Actions +workflow runs its tests and then builds one tar.gz holding `main.py`, `app/`, `website/`, `BUILD.json` **and +`vendor/`** — its own dependencies, as prebuilt wheels for this image's Python. `PYTHONPATH=/app/vendor` is how +the code finds them, so **a new dependency never needs an image rebuild** and the server never contacts a +package index. + +The workflow's last job posts that bundle to **dumpd** at `POST /deploy-bot` on 8081 — the Docker-bridge-only +port the firewall already restricts, so a CI job never needs the Docker socket or an ssh key. dumpd checks an +HMAC-SHA256 signature over the body, then runs `/usr/local/bin/vanguard-deploy-bot` (this repo's +`deployBotBundle.sh`), which verifies every file against `BUILD.json`, swaps the bundle into `/srv/bot/app` +(bind-mounted at `/app`, previous release kept beside it) and restarts the container. The install log goes back +in the HTTP response, so it appears in the CI job. + +Two settings, both generated into `.env` and copied to `/etc/vanguard/deploy.env` by `configure.py`: + +| | | +|---|---| +| `BOT_DEPLOY_SECRET` | 64 hex chars. Paste it into Gitea → the bot repo → Settings → Actions → Secrets as **`BOT_DEPLOY_SECRET`**. `./configure.py --show` prints it. | +| `BOT_DEPLOY_AUTO` | `1` (default) accepts CI installs; `0` makes dumpd refuse them with 403 and you install by hand. | + +By hand, any time: ./deployBotBundle.sh vanguard-bot-bundle.zip # install a release (the artifact as Gitea hands it over, zip or tar.gz) ./deployBotBundle.sh --latest # fetch the newest main build first (needs BOT_REPO_TOKEN in .env) diff --git a/STATUS.md b/STATUS.md index 91f528b..efdd990 100644 --- a/STATUS.md +++ b/STATUS.md @@ -68,7 +68,7 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea | Gitea | git hosting on MariaDB, Actions enabled; holds this repo | live; Discord login via Authentik OIDC **verified**; access = `git-users`/`git-admins` groups; local admin as fallback | | act_runner | CI jobs in Docker | **not registered** | | discord-bot | Python runtime, idles until `bot/app/main.py` exists | live, no code yet | -| `bot.` host + Discord bot | the bot's API **and** its web pages on one port (`WEB_SERVE_UI`), behind Authentik forward-auth; Caddy requires `bot-users`, the bot re-checks `bot-users,admins` per request | code arrives as a CI **release bundle** (app + `vendor/` dependencies) installed by `./deployBotBundle.sh` into `/srv/bot/app` → `/app`; image installs no bot dependencies | **reviewed + wired 2026-11-02, first install pending** (needs the Authentik app for `bot.`, the bot token in `.env`, and a bundle from the ci workflow) | +| `bot.` host + Discord bot | the bot's API **and** its web pages on one port (`WEB_SERVE_UI`), behind Authentik forward-auth; Caddy requires `bot-users`, the bot re-checks `bot-users,admins` per request | a push to `main` deploys itself: CI builds a **release bundle** (app + `vendor/` dependencies) and posts it signed to dumpd `POST /deploy-bot` (bridge-only port, no Docker socket in CI), which verifies it and swaps it into `/srv/bot/app` → `/app`; image installs no bot dependencies | **reviewed + wired 2026-11-02, first install pending** (needs the Authentik app for `bot.`, the bot token in `.env`, and `BOT_DEPLOY_SECRET` as a Gitea repo secret) | | Live portal (`live/`, `live.` host) | React + dockview multi-stream viewer + browser Go Live (WHIP); MediaMTX HLS/WebRTC/API on loopback; dumpd `/me/stream-key` | live; Go Live verified | ## Unfinished, in rough priority diff --git a/bin/dumpd.py b/bin/dumpd.py index 5ba8a44..ef82392 100755 --- a/bin/dumpd.py +++ b/bin/dumpd.py @@ -12,6 +12,8 @@ POST /me/clips -> {"device_id": "..."}: create clips/ (+ .stignore), add POST /deploy -> Gitea push webhook (bridge port only, HMAC-verified with DEPLOY_WEBHOOK_SECRET from /etc/vanguard/deploy.env). Runs /usr/local/bin/vanguard-deploy pull ONLY when DEPLOY_AUTO=1; otherwise logs the push and answers 202 "auto-deploy disabled". +POST /deploy-bot -> Gitea Actions (job container -> host gateway) hands over a Discord-bot release bundle, + HMAC-SHA256 signed with BOT_DEPLOY_SECRET; installs it with vanguard-deploy-bot. POST /mtx-auth -> MediaMTX authHTTP hook: allows read/playback; allows publish iff user/pass carries a valid token whose user owns the path prefix. Tokens are HMAC-signed with /etc/vanguard/stream-token.secret. @@ -19,7 +21,7 @@ POST /mtx-auth -> MediaMTX authHTTP hook: allows read/playback; allows publish i Default night: today if it is 17:00 or later, else yesterday (so 00:00-02:00 still counts as last night). Users may dump only their own prefix; members of the admins / media-admins groups may pass "user" to dump anyone, or "all". """ -import base64, hashlib, hmac, json, os, re, shutil, sys +import base64, hashlib, hmac, json, os, re, shutil, subprocess, sys, threading import time as _time from datetime import datetime, timedelta, time from zoneinfo import ZoneInfo @@ -260,9 +262,39 @@ def deploy_env(): pass return d +_bot_deploy_lock = threading.Lock() +BOT_DEPLOY_MAX_BYTES = 300 * 1024 * 1024 +BOT_DEPLOY_INSTALLER = os.environ.get("BOT_DEPLOY_INSTALLER", "/usr/local/bin/vanguard-deploy-bot") + +def install_bot_bundle(raw, reason): + """Write the uploaded bundle to a temp file and let vanguard-deploy-bot verify, swap and restart it.""" + import tempfile + if not _bot_deploy_lock.acquire(blocking=False): + return 409, {"error": "a bot deploy is already running"} + tmp = None + try: + with tempfile.NamedTemporaryFile(prefix="vanguard-bot-bundle.", suffix=".bin", delete=False) as handle: + handle.write(raw); tmp = handle.name + log("bot_deploy_start", bytes=len(raw), reason=reason) + done = subprocess.run([BOT_DEPLOY_INSTALLER, tmp], capture_output=True, text=True, timeout=600) + output = (done.stdout + done.stderr).strip().splitlines() + log("bot_deploy_exit", level=("info" if done.returncode == 0 else "error"), rc=done.returncode, reason=reason) + return (200 if done.returncode == 0 else 500), {"installed": done.returncode == 0, "rc": done.returncode, + "output": output[-40:]} + except subprocess.TimeoutExpired: + log("bot_deploy_timeout", level="error", reason=reason) + return 504, {"error": "the installer did not finish within 10 minutes"} + except OSError as error: + log("bot_deploy_error", level="error", error=error) + return 500, {"error": str(error)} + finally: + if tmp: + try: os.unlink(tmp) + except OSError: pass + _bot_deploy_lock.release() + _deploy_lock = {"running": False} def run_deploy(reason): - import subprocess, threading if _deploy_lock["running"]: return False _deploy_lock["running"] = True def go(): @@ -396,6 +428,26 @@ class H(BaseHTTPRequestHandler): "files": [str(f.relative_to(ROOT)) for f in files], "gb": round(sum(f.stat().st_size for f in files) / 1e9, 2)}) def do_POST(self): + if self.server.server_port == AUTH_LISTEN[1] and self.path.startswith("/deploy-bot"): + # The bot's CI posts its release bundle here (bridge-only port; the firewall allows docker0/br-* alone). + # The body is the artifact; the signature proves it came from a runner holding BOT_DEPLOY_SECRET. + env = deploy_env() + if env.get("BOT_DEPLOY_AUTO", "1") != "1": + log("bot_deploy_refused", level="warn", reason="BOT_DEPLOY_AUTO=0") + return self._json(403, {"error": "automatic bot deploys are turned off (BOT_DEPLOY_AUTO=0)"}) + secret = env.get("BOT_DEPLOY_SECRET", "") + size = int(self.headers.get("Content-Length") or 0) + if not secret: + return self._json(503, {"error": "BOT_DEPLOY_SECRET is not set on the server"}) + if not 0 < size <= BOT_DEPLOY_MAX_BYTES: + return self._json(413, {"error": f"body must be 1..{BOT_DEPLOY_MAX_BYTES} bytes, got {size}"}) + raw = self.rfile.read(size) + sig = (self.headers.get("X-Vanguard-Signature") or "").replace("sha256=", "").strip() + if not sig or not hmac.compare_digest(hmac.new(secret.encode(), raw, hashlib.sha256).hexdigest(), sig): + log("bot_deploy_bad_signature", level="warn", ip=self.client_address[0], bytes=len(raw)) + return self._json(401, {"error": "bad signature"}) + code, body = install_bot_bundle(raw, self.headers.get("X-Vanguard-Reason") or "ci") + return self._json(code, body) if self.server.server_port == AUTH_LISTEN[1] and self.path.startswith("/deploy"): # Gitea (container) -> host gateway -> here. Verify HMAC-SHA256 of the raw body with the webhook secret. n = int(self.headers.get("Content-Length") or 0); raw = self.rfile.read(n) @@ -481,7 +533,6 @@ class H(BaseHTTPRequestHandler): if __name__ == "__main__": TRASH.mkdir(exist_ok=True); MARK.mkdir(exist_ok=True) - import threading print(f"dumpd listening on {LISTEN[0]}:{LISTEN[1]} (dump, me) and {AUTH_LISTEN[0]}:{AUTH_LISTEN[1]} (mtx-auth) tz={TZ.key} night={NIGHT_START:%H:%M}-{NIGHT_END:%H:%M} token-secret={'ok' if token_secret() else 'MISSING'}", flush=True) auth_srv = ThreadingHTTPServer(AUTH_LISTEN, H) threading.Thread(target=auth_srv.serve_forever, daemon=True).start() diff --git a/bot/AGENT.md b/bot/AGENT.md index 91bfc11..059be85 100644 --- a/bot/AGENT.md +++ b/bot/AGENT.md @@ -43,9 +43,14 @@ Your CI (`.gitea/workflows/ci.yml`, job `bundle`) runs after the tests and produ `vendor/` is `pip install --only-binary=:all: --target vendor -r requirements.txt`, built on `python:3.12` so the wheels match the runtime image's ABI. `BUILD.json` records the version, commit, build time, the -package list and a sha256 of every file. The admin installs it with `./deployBotBundle.sh `, -which checks the bundle against `BUILD.json`, keeps the previous release for `--rollback`, and restarts the -container. Rules that follow from this: +package list and a sha256 of every file. + +**A push to `main` deploys itself.** The workflow's last job posts the bundle to the server's deploy service, +signed with the repository secret `BOT_DEPLOY_SECRET`; the server verifies the signature and every checksum, +keeps the previous release for rollback, and restarts the container. The install log comes back in your CI +job, so a failed deploy fails the run. Branches and pull requests test and build, and deploy nothing. + +Rules that follow from this: - Keep `requirements.txt` resolvable with **prebuilt wheels only**; a dependency that must compile will fail the build. Ask the admin if you truly need one. @@ -123,9 +128,9 @@ message content; Message Content is a privileged intent, enable it only if you n ## 6. Working with the admin -- **Source**: push to the community Gitea. CI runs your tests on the `python` runner label, then builds the - release bundle (§1a); the admin installs it with `deployBotBundle.sh`. Pushes to `main` can install it - automatically once the repository variable `BOT_AUTO_DEPLOY` is set to `1`; it is unset by default. +- **Source**: push to the community Gitea. CI runs your tests on the `python` runner label, builds the release + bundle (§1a), and on `main` installs it on the server. Keep `main` deployable: a red test stops the deploy, + and a bundle that fails its checksum check is refused. - **Need infra?** Open a Gitea issue titled `infra: …` stating the exact change: a new env var and why, a mount, a Caddy path that should skip login (e.g. a public webhook endpoint), a new database. Don't work around the proxy or the firewall. diff --git a/bot/README.md b/bot/README.md index 4e69031..fa5dce8 100644 --- a/bot/README.md +++ b/bot/README.md @@ -5,9 +5,12 @@ Start with **AGENT.md** in this folder: the stack, the rules, the layout, the we ## How code reaches the server Your repo's Gitea Actions workflow builds a **release bundle**: one tar.gz holding `main.py`, `app/`, -`website/`, `version.json`, `BUILD.json` and `vendor/` (your dependencies as prebuilt wheels). The admin -installs it with `./deployBotBundle.sh ` from the stack root; it lands in `/srv/bot/app`, -which is bind-mounted into the container at `/app`, and the previous release is kept for `--rollback`. +`website/`, `version.json`, `BUILD.json` and `vendor/` (your dependencies as prebuilt wheels). **A push to +`main` installs itself**: the workflow's last job posts the bundle to the server's deploy service, which +checks its signature and every checksum, swaps it into `/srv/bot/app` (bind-mounted at `/app`, previous +release kept for `--rollback`) and restarts the container. The install log comes back in the CI job. + +A branch or pull request runs the tests and builds the artifact, and deploys nothing. The image contains **no** bot dependencies: the container imports yours from `/app/vendor` (`PYTHONPATH=/app/vendor`). Adding a dependency means editing `app/requirements.txt` and pushing — no image @@ -15,7 +18,7 @@ rebuild, no package index on the server. The image only changes if you need a ne curl are already there); ask the admin for that. ## Day to day (on the server, in ~/vanguard-stream) - ./deployBotBundle.sh vanguard-bot-bundle.tar.gz # install a release + ./deployBotBundle.sh vanguard-bot-bundle.zip # install a release by hand (zip or tar.gz) ./deployBotBundle.sh --status # what is installed; container state ./deployBotBundle.sh --rollback # previous release docker compose logs -f discord-bot # live log diff --git a/configure.py b/configure.py index aacb627..0936b98 100755 --- a/configure.py +++ b/configure.py @@ -370,6 +370,8 @@ def write_env(watch_host, discord_token=None, discord_guild=None, git_host="", b env.setdefault("BOT_DB_NAME", "vanguardbot") env.setdefault("BOT_DB_USER", "bot") env.setdefault("BOT_DB_PASSWORD", "".join(c for c in secrets.token_urlsafe(40) if c.isalnum())[:32]) + env.setdefault("BOT_DEPLOY_SECRET", secrets.token_hex(32)) # Gitea repo secret BOT_DEPLOY_SECRET: signs the bot's release bundle + env.setdefault("BOT_DEPLOY_AUTO", "1") # 0 = refuse bundles posted by CI; install by hand instead env.setdefault("DISCORD_BOT_TOKEN", "") # fill in: Discord dev portal -> the Authentik app -> Bot env.setdefault("DISCORD_GUILD_ID", "") env["WATCH_HOST"] = watch_host or "watch.example.com" @@ -535,7 +537,7 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis run(["sudo", "install", "-m", "0600", "stream-token.secret", "/etc/vanguard/stream-token.secret"]) run(["sudo", "install", "-m", "0644", "syncthing/stignore", "/etc/vanguard/stignore"]) # dumpd copies it into new clips folders e = read_env() - Path("deploy.env").write_text("".join(f"{k}={e.get(k, '')}\n" for k in ("DEPLOY_AUTO", "DEPLOY_WEBHOOK_SECRET", "DEPLOY_BRANCH", "DEPLOY_NOTIFY_WEBHOOK_URL")) + f"DEPLOY_REPO_DIR={Path.cwd()}\n"); os.chmod("deploy.env", 0o600) + Path("deploy.env").write_text("".join(f"{k}={e.get(k, '')}\n" for k in ("DEPLOY_AUTO", "DEPLOY_WEBHOOK_SECRET", "DEPLOY_BRANCH", "DEPLOY_NOTIFY_WEBHOOK_URL", "BOT_DEPLOY_AUTO", "BOT_DEPLOY_SECRET")) + f"DEPLOY_REPO_DIR={Path.cwd()}\n"); os.chmod("deploy.env", 0o600) run(["sudo", "install", "-m", "0600", "deploy.env", "/etc/vanguard/deploy.env"]) # read by dumpd (/deploy webhook) and deploy.sh if all(Path(f).exists() for f in ("deploy.sh", "systemd/deploy-pending.service", "systemd/deploy-pending.timer")): run(["sudo", "install", "-m", "0755", "deploy.sh", "/usr/local/bin/vanguard-deploy"]) @@ -543,6 +545,8 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis run(["sudo", "systemctl", "daemon-reload"]); run(["sudo", "systemctl", "enable", "--now", "deploy-pending.timer"]) else: print(" (deploy pipeline files missing — deploy.sh / systemd/deploy-pending.* not uploaded; skipping vanguard-deploy install)") + run(["sudo", "install", "-m", "0755", "deployBotBundle.sh", "/usr/local/bin/vanguard-deploy-bot"]) # dumpd runs this for POST /deploy-bot + run(["sudo", "install", "-d", "-o", "1000", "-g", "1000", "/srv/bot", "/srv/bot/app"]) # the bot's released bundle -> /app run(["sudo", "systemctl", "restart", "dumpd"]) print("== containers") dc = ["docker", "compose"] if (DRY or subprocess.call(["docker", "info"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) == 0) else ["sudo", "docker", "compose"] @@ -608,6 +612,8 @@ def show(domain, admin_pw, streamers, editors, auth_host="", watch_host="", git_ tok = env.get("DISCORD_BOT_TOKEN", "") print(f"== discord : guild {env.get('DISCORD_GUILD_ID') or '(not set)'}; bot token {'set (' + tok[:6] + '…)' if tok else '(not set)'}") print(f"== mariadb : {domain or 'HOST'}:3306 (public, root local-only) root pass {env.get('MARIADB_ROOT_PASSWORD','(run apply)')}") + print(f" bot ci : Gitea -> VanguardBot -> Settings -> Actions -> Secrets: BOT_DEPLOY_SECRET={env.get('BOT_DEPLOY_SECRET','(run apply)')}") + print(f" pushes to main install themselves (BOT_DEPLOY_AUTO={env.get('BOT_DEPLOY_AUTO','1')}); by hand: ./deployBotBundle.sh ") print(f" bot : db {env.get('BOT_DB_NAME','vanguardbot')} user {env.get('BOT_DB_USER','bot')} pass {env.get('BOT_DB_PASSWORD','(run apply)')}") print(f" gitea : db gitea user gitea pass {env.get('GITEA_DB_PASSWORD','(run apply)')} (created on MariaDB first init, else ./createGiteaDb.sh)") print("== access : by Authentik group — editors (read streams/clips, write finals), media-admins (everything), admins (dump anyone)") diff --git a/deployBotBundle.sh b/deployBotBundle.sh index 57e9f3a..f75bca0 100755 --- a/deployBotBundle.sh +++ b/deployBotBundle.sh @@ -13,7 +13,14 @@ # --latest needs a Gitea token with read access to the repo, in .env as BOT_REPO_TOKEN (or $BOT_REPO_TOKEN). # Layout: $BOT_DIR/app = live (mounted at /app) $BOT_DIR/previous = the one before $BOT_DIR/.incoming = staging set -euo pipefail -cd "$(dirname "$0")" +# Runs from the bundle root or as /usr/local/bin/vanguard-deploy-bot (dumpd calls it that way): find the stack. +STACK="${VANGUARD_DIR:-}" +if [[ -z $STACK && -f /etc/vanguard/deploy.env ]]; then + STACK=$(grep -E '^DEPLOY_REPO_DIR=' /etc/vanguard/deploy.env | head -1 | cut -d= -f2- || true) +fi +if [[ -z $STACK || ! -f $STACK/docker-compose.yml ]]; then STACK="$(cd "$(dirname "$0")" && pwd)"; fi +[[ -f $STACK/docker-compose.yml ]] || { echo "error: cannot find the stack (set VANGUARD_DIR)" >&2; exit 1; } +cd "$STACK" BOT_DIR="${BOT_DIR:-/srv/bot}" REPO="${BOT_REPO:-VanguardOrg/VanguardBot}" BRANCH="${BOT_BRANCH:-main}"