Bot releases install themselves: signed bundle upload to dumpd, no Docker socket in CI

dumpd gains POST /deploy-bot on the bridge-only auth port: HMAC-SHA256 over the body with BOT_DEPLOY_SECRET,
a 300 MB cap, one install at a time, and the installer's output returned in the response so the CI job shows
it. BOT_DEPLOY_AUTO=0 makes the server refuse. configure.py generates the secret, writes both settings into
/etc/vanguard/deploy.env, installs deployBotBundle.sh as /usr/local/bin/vanguard-deploy-bot and creates
/srv/bot/app; --show prints what to paste into Gitea.

Chose this over mounting the Docker socket into job containers, which would give every CI job root on the
host. Also moved threading/subprocess to dumpd's module imports, which the new module-level lock needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

LOGGING.md

@@ -38,7 +38,7 @@ into stream labels without parsing a single line.
 | `media` | sweep verdicts / timecode / deband / failed, vgdeband, jellyfin, copyparty | `file` |
 | `intake` | syncthing (`folder.id`), dumpd `clips_setup`, `quota_*` | `folder` = `clips-<user>` |
 | `infra` | mariadb, caddy TLS/ACME, container restarts, disk | host only |
-| `deploy` | deploy.sh, dumpd `deploy_*`, act_runner | `rev` |
+| `deploy` | deploy.sh, dumpd `deploy_*` and `bot_deploy_*` (the bot's CI posting a release bundle), act_runner | `rev` |
 | `code`, `bot` | gitea, discord-bot | — |
 
 ## Labels vs fields
@@ -55,7 +55,7 @@ into stream labels without parsing a single line.
 - Same key means the same thing everywhere: `stream` (user/Title), `user`, `file`, `folder`, `rev`, `ip`, `reason`.
 - Implemented by: `log()` in `bin/dumpd.py`, `vglog` in `bin/normalize-sweep.sh` and `bin/vgdeband.sh`, `log()` in `deploy.sh` (`msg=` only), `bin/mtx-on-ready.sh` (MediaMTX prefixes its own path/time).
 
-Events today: dumpd `publish_allow publish_deny dump clips_setup quota_paused quota_resumed quota_* deploy_*`;
+Events today: dumpd `publish_allow publish_deny dump clips_setup quota_paused quota_resumed quota_* deploy_* bot_deploy_start bot_deploy_exit bot_deploy_bad_signature bot_deploy_refused`;
 sweep `verdict timecode deband fill_gaps published done failed skip`; vgdeband `deband done failed skip`; hook `mirror`.
 A future alert script watches `level=error`, `event=failed`, `event=publish_deny` bursts, `event=quota_paused`, plus disk.
 

README.md

@@ -240,12 +240,27 @@ for the bot, and `discord-bot`, a Python 3.12 container that runs `/app/main.py`
 installed. Credentials live in `.env` (generated by `configure.py`; bot token + guild ID are asked in `--full`
 or the `[d]` menu item; `./configure.py --show` prints them).
 
-**Releases are self-contained bundles, not loose files.** The bot repo's Gitea Actions workflow runs its tests
-and then builds one tar.gz holding `main.py`, `app/`, `website/`, `BUILD.json` **and `vendor/`** — its own
-dependencies, as prebuilt wheels for this image's Python. `./deployBotBundle.sh <bundle.tar.gz>` verifies it
-against `BUILD.json`, swaps it into `/srv/bot/app` (bind-mounted at `/app`, previous release kept beside it)
-and restarts the container. `PYTHONPATH=/app/vendor` is how the code finds its dependencies, so **a new
-dependency never needs an image rebuild** and the server never contacts a package index.
+**Releases are self-contained bundles, and a push to `main` deploys itself.** The bot repo's Gitea Actions
+workflow runs its tests and then builds one tar.gz holding `main.py`, `app/`, `website/`, `BUILD.json` **and
+`vendor/`** — its own dependencies, as prebuilt wheels for this image's Python. `PYTHONPATH=/app/vendor` is how
+the code finds them, so **a new dependency never needs an image rebuild** and the server never contacts a
+package index.
+
+The workflow's last job posts that bundle to **dumpd** at `POST /deploy-bot` on 8081 — the Docker-bridge-only
+port the firewall already restricts, so a CI job never needs the Docker socket or an ssh key. dumpd checks an
+HMAC-SHA256 signature over the body, then runs `/usr/local/bin/vanguard-deploy-bot` (this repo's
+`deployBotBundle.sh`), which verifies every file against `BUILD.json`, swaps the bundle into `/srv/bot/app`
+(bind-mounted at `/app`, previous release kept beside it) and restarts the container. The install log goes back
+in the HTTP response, so it appears in the CI job.
+
+Two settings, both generated into `.env` and copied to `/etc/vanguard/deploy.env` by `configure.py`:
+
+| | |
+|---|---|
+| `BOT_DEPLOY_SECRET` | 64 hex chars. Paste it into Gitea → the bot repo → Settings → Actions → Secrets as **`BOT_DEPLOY_SECRET`**. `./configure.py --show` prints it. |
+| `BOT_DEPLOY_AUTO` | `1` (default) accepts CI installs; `0` makes dumpd refuse them with 403 and you install by hand. |
+
+By hand, any time:
 
     ./deployBotBundle.sh vanguard-bot-bundle.zip      # install a release (the artifact as Gitea hands it over, zip or tar.gz)
     ./deployBotBundle.sh --latest                     # fetch the newest main build first (needs BOT_REPO_TOKEN in .env)

STATUS.md

@@ -68,7 +68,7 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea
 | Gitea | git hosting on MariaDB, Actions enabled; holds this repo | live; Discord login via Authentik OIDC **verified**; access = `git-users`/`git-admins` groups; local admin as fallback |
 | act_runner | CI jobs in Docker | **not registered** |
 | discord-bot | Python runtime, idles until `bot/app/main.py` exists | live, no code yet |
-| `bot.` host + Discord bot | the bot's API **and** its web pages on one port (`WEB_SERVE_UI`), behind Authentik forward-auth; Caddy requires `bot-users`, the bot re-checks `bot-users,admins` per request | code arrives as a CI **release bundle** (app + `vendor/` dependencies) installed by `./deployBotBundle.sh` into `/srv/bot/app` → `/app`; image installs no bot dependencies | **reviewed + wired 2026-11-02, first install pending** (needs the Authentik app for `bot.`, the bot token in `.env`, and a bundle from the ci workflow) |
+| `bot.` host + Discord bot | the bot's API **and** its web pages on one port (`WEB_SERVE_UI`), behind Authentik forward-auth; Caddy requires `bot-users`, the bot re-checks `bot-users,admins` per request | a push to `main` deploys itself: CI builds a **release bundle** (app + `vendor/` dependencies) and posts it signed to dumpd `POST /deploy-bot` (bridge-only port, no Docker socket in CI), which verifies it and swaps it into `/srv/bot/app` → `/app`; image installs no bot dependencies | **reviewed + wired 2026-11-02, first install pending** (needs the Authentik app for `bot.`, the bot token in `.env`, and `BOT_DEPLOY_SECRET` as a Gitea repo secret) |
 | Live portal (`live/`, `live.` host) | React + dockview multi-stream viewer + browser Go Live (WHIP); MediaMTX HLS/WebRTC/API on loopback; dumpd `/me/stream-key` | live; Go Live verified |
 
 ## Unfinished, in rough priority

bin/dumpd.py

@@ -12,6 +12,8 @@ POST /me/clips  -> {"device_id": "..."}: create clips/<user> (+ .stignore), add
 POST /deploy    -> Gitea push webhook (bridge port only, HMAC-verified with DEPLOY_WEBHOOK_SECRET from
              /etc/vanguard/deploy.env). Runs /usr/local/bin/vanguard-deploy pull ONLY when DEPLOY_AUTO=1;
              otherwise logs the push and answers 202 "auto-deploy disabled".
+POST /deploy-bot -> Gitea Actions (job container -> host gateway) hands over a Discord-bot release bundle,
+                   HMAC-SHA256 signed with BOT_DEPLOY_SECRET; installs it with vanguard-deploy-bot.
 POST /mtx-auth -> MediaMTX authHTTP hook: allows read/playback; allows publish iff user/pass carries a valid
              token whose user owns the path prefix. Tokens are HMAC-signed with /etc/vanguard/stream-token.secret.
 
@@ -19,7 +21,7 @@ POST /mtx-auth -> MediaMTX authHTTP hook: allows read/playback; allows publish i
 Default night: today if it is 17:00 or later, else yesterday (so 00:00-02:00 still counts as last night).
 Users may dump only their own prefix; members of the admins / media-admins groups may pass "user" to dump anyone, or "all".
 """
-import base64, hashlib, hmac, json, os, re, shutil, sys
+import base64, hashlib, hmac, json, os, re, shutil, subprocess, sys, threading
 import time as _time
 from datetime import datetime, timedelta, time
 from zoneinfo import ZoneInfo
@@ -260,9 +262,39 @@ def deploy_env():
         pass
     return d
 
+_bot_deploy_lock = threading.Lock()
+BOT_DEPLOY_MAX_BYTES = 300 * 1024 * 1024
+BOT_DEPLOY_INSTALLER = os.environ.get("BOT_DEPLOY_INSTALLER", "/usr/local/bin/vanguard-deploy-bot")
+
+def install_bot_bundle(raw, reason):
+    """Write the uploaded bundle to a temp file and let vanguard-deploy-bot verify, swap and restart it."""
+    import tempfile
+    if not _bot_deploy_lock.acquire(blocking=False):
+        return 409, {"error": "a bot deploy is already running"}
+    tmp = None
+    try:
+        with tempfile.NamedTemporaryFile(prefix="vanguard-bot-bundle.", suffix=".bin", delete=False) as handle:
+            handle.write(raw); tmp = handle.name
+        log("bot_deploy_start", bytes=len(raw), reason=reason)
+        done = subprocess.run([BOT_DEPLOY_INSTALLER, tmp], capture_output=True, text=True, timeout=600)
+        output = (done.stdout + done.stderr).strip().splitlines()
+        log("bot_deploy_exit", level=("info" if done.returncode == 0 else "error"), rc=done.returncode, reason=reason)
+        return (200 if done.returncode == 0 else 500), {"installed": done.returncode == 0, "rc": done.returncode,
+                                                        "output": output[-40:]}
+    except subprocess.TimeoutExpired:
+        log("bot_deploy_timeout", level="error", reason=reason)
+        return 504, {"error": "the installer did not finish within 10 minutes"}
+    except OSError as error:
+        log("bot_deploy_error", level="error", error=error)
+        return 500, {"error": str(error)}
+    finally:
+        if tmp:
+            try: os.unlink(tmp)
+            except OSError: pass
+        _bot_deploy_lock.release()
+
 _deploy_lock = {"running": False}
 def run_deploy(reason):
-    import subprocess, threading
     if _deploy_lock["running"]: return False
     _deploy_lock["running"] = True
     def go():
@@ -396,6 +428,26 @@ class H(BaseHTTPRequestHandler):
                          "files": [str(f.relative_to(ROOT)) for f in files],
                          "gb": round(sum(f.stat().st_size for f in files) / 1e9, 2)})
     def do_POST(self):
+        if self.server.server_port == AUTH_LISTEN[1] and self.path.startswith("/deploy-bot"):
+            # The bot's CI posts its release bundle here (bridge-only port; the firewall allows docker0/br-* alone).
+            # The body is the artifact; the signature proves it came from a runner holding BOT_DEPLOY_SECRET.
+            env = deploy_env()
+            if env.get("BOT_DEPLOY_AUTO", "1") != "1":
+                log("bot_deploy_refused", level="warn", reason="BOT_DEPLOY_AUTO=0")
+                return self._json(403, {"error": "automatic bot deploys are turned off (BOT_DEPLOY_AUTO=0)"})
+            secret = env.get("BOT_DEPLOY_SECRET", "")
+            size = int(self.headers.get("Content-Length") or 0)
+            if not secret:
+                return self._json(503, {"error": "BOT_DEPLOY_SECRET is not set on the server"})
+            if not 0 < size <= BOT_DEPLOY_MAX_BYTES:
+                return self._json(413, {"error": f"body must be 1..{BOT_DEPLOY_MAX_BYTES} bytes, got {size}"})
+            raw = self.rfile.read(size)
+            sig = (self.headers.get("X-Vanguard-Signature") or "").replace("sha256=", "").strip()
+            if not sig or not hmac.compare_digest(hmac.new(secret.encode(), raw, hashlib.sha256).hexdigest(), sig):
+                log("bot_deploy_bad_signature", level="warn", ip=self.client_address[0], bytes=len(raw))
+                return self._json(401, {"error": "bad signature"})
+            code, body = install_bot_bundle(raw, self.headers.get("X-Vanguard-Reason") or "ci")
+            return self._json(code, body)
         if self.server.server_port == AUTH_LISTEN[1] and self.path.startswith("/deploy"):
             # Gitea (container) -> host gateway -> here. Verify HMAC-SHA256 of the raw body with the webhook secret.
             n = int(self.headers.get("Content-Length") or 0); raw = self.rfile.read(n)
@@ -481,7 +533,6 @@ class H(BaseHTTPRequestHandler):
 
 if __name__ == "__main__":
     TRASH.mkdir(exist_ok=True); MARK.mkdir(exist_ok=True)
-    import threading
     print(f"dumpd listening on {LISTEN[0]}:{LISTEN[1]} (dump, me) and {AUTH_LISTEN[0]}:{AUTH_LISTEN[1]} (mtx-auth)  tz={TZ.key}  night={NIGHT_START:%H:%M}-{NIGHT_END:%H:%M}  token-secret={'ok' if token_secret() else 'MISSING'}", flush=True)
     auth_srv = ThreadingHTTPServer(AUTH_LISTEN, H)
     threading.Thread(target=auth_srv.serve_forever, daemon=True).start()

bot/AGENT.md

@@ -43,9 +43,14 @@ Your CI (`.gitea/workflows/ci.yml`, job `bundle`) runs after the tests and produ
 
 `vendor/` is `pip install --only-binary=:all: --target vendor -r requirements.txt`, built on `python:3.12`
 so the wheels match the runtime image's ABI. `BUILD.json` records the version, commit, build time, the
-package list and a sha256 of every file. The admin installs it with `./deployBotBundle.sh <bundle.tar.gz>`,
-which checks the bundle against `BUILD.json`, keeps the previous release for `--rollback`, and restarts the
-container. Rules that follow from this:
+package list and a sha256 of every file.
+
+**A push to `main` deploys itself.** The workflow's last job posts the bundle to the server's deploy service,
+signed with the repository secret `BOT_DEPLOY_SECRET`; the server verifies the signature and every checksum,
+keeps the previous release for rollback, and restarts the container. The install log comes back in your CI
+job, so a failed deploy fails the run. Branches and pull requests test and build, and deploy nothing.
+
+Rules that follow from this:
 
 - Keep `requirements.txt` resolvable with **prebuilt wheels only**; a dependency that must compile will fail
   the build. Ask the admin if you truly need one.
@@ -123,9 +128,9 @@ message content; Message Content is a privileged intent, enable it only if you n
 
 ## 6. Working with the admin
 
-- **Source**: push to the community Gitea. CI runs your tests on the `python` runner label, then builds the
-  release bundle (§1a); the admin installs it with `deployBotBundle.sh`. Pushes to `main` can install it
-  automatically once the repository variable `BOT_AUTO_DEPLOY` is set to `1`; it is unset by default.
+- **Source**: push to the community Gitea. CI runs your tests on the `python` runner label, builds the release
+  bundle (§1a), and on `main` installs it on the server. Keep `main` deployable: a red test stops the deploy,
+  and a bundle that fails its checksum check is refused.
 - **Need infra?** Open a Gitea issue titled `infra: …` stating the exact change: a new env var and why,
   a mount, a Caddy path that should skip login (e.g. a public webhook endpoint), a new database.
   Don't work around the proxy or the firewall.

bot/README.md

@@ -5,9 +5,12 @@ Start with **AGENT.md** in this folder: the stack, the rules, the layout, the we
 ## How code reaches the server
 
 Your repo's Gitea Actions workflow builds a **release bundle**: one tar.gz holding `main.py`, `app/`,
-`website/`, `version.json`, `BUILD.json` and `vendor/` (your dependencies as prebuilt wheels). The admin
-installs it with `./deployBotBundle.sh <bundle.tar.gz>` from the stack root; it lands in `/srv/bot/app`,
-which is bind-mounted into the container at `/app`, and the previous release is kept for `--rollback`.
+`website/`, `version.json`, `BUILD.json` and `vendor/` (your dependencies as prebuilt wheels). **A push to
+`main` installs itself**: the workflow's last job posts the bundle to the server's deploy service, which
+checks its signature and every checksum, swaps it into `/srv/bot/app` (bind-mounted at `/app`, previous
+release kept for `--rollback`) and restarts the container. The install log comes back in the CI job.
+
+A branch or pull request runs the tests and builds the artifact, and deploys nothing.
 
 The image contains **no** bot dependencies: the container imports yours from `/app/vendor`
 (`PYTHONPATH=/app/vendor`). Adding a dependency means editing `app/requirements.txt` and pushing — no image
@@ -15,7 +18,7 @@ rebuild, no package index on the server. The image only changes if you need a ne
 curl are already there); ask the admin for that.
 
 ## Day to day (on the server, in ~/vanguard-stream)
-    ./deployBotBundle.sh vanguard-bot-bundle.tar.gz   # install a release
+    ./deployBotBundle.sh vanguard-bot-bundle.zip      # install a release by hand (zip or tar.gz)
     ./deployBotBundle.sh --status                     # what is installed; container state
     ./deployBotBundle.sh --rollback                   # previous release
     docker compose logs -f discord-bot                # live log

configure.py

@@ -370,6 +370,8 @@ def write_env(watch_host, discord_token=None, discord_guild=None, git_host="", b
     env.setdefault("BOT_DB_NAME", "vanguardbot")
     env.setdefault("BOT_DB_USER", "bot")
     env.setdefault("BOT_DB_PASSWORD", "".join(c for c in secrets.token_urlsafe(40) if c.isalnum())[:32])
+    env.setdefault("BOT_DEPLOY_SECRET", secrets.token_hex(32))   # Gitea repo secret BOT_DEPLOY_SECRET: signs the bot's release bundle
+    env.setdefault("BOT_DEPLOY_AUTO", "1")                       # 0 = refuse bundles posted by CI; install by hand instead
     env.setdefault("DISCORD_BOT_TOKEN", "")          # fill in: Discord dev portal -> the Authentik app -> Bot
     env.setdefault("DISCORD_GUILD_ID", "")
     env["WATCH_HOST"] = watch_host or "watch.example.com"
@@ -535,7 +537,7 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis
     run(["sudo", "install", "-m", "0600", "stream-token.secret", "/etc/vanguard/stream-token.secret"])
     run(["sudo", "install", "-m", "0644", "syncthing/stignore", "/etc/vanguard/stignore"])   # dumpd copies it into new clips folders
     e = read_env()
-    Path("deploy.env").write_text("".join(f"{k}={e.get(k, '')}\n" for k in ("DEPLOY_AUTO", "DEPLOY_WEBHOOK_SECRET", "DEPLOY_BRANCH", "DEPLOY_NOTIFY_WEBHOOK_URL")) + f"DEPLOY_REPO_DIR={Path.cwd()}\n"); os.chmod("deploy.env", 0o600)
+    Path("deploy.env").write_text("".join(f"{k}={e.get(k, '')}\n" for k in ("DEPLOY_AUTO", "DEPLOY_WEBHOOK_SECRET", "DEPLOY_BRANCH", "DEPLOY_NOTIFY_WEBHOOK_URL", "BOT_DEPLOY_AUTO", "BOT_DEPLOY_SECRET")) + f"DEPLOY_REPO_DIR={Path.cwd()}\n"); os.chmod("deploy.env", 0o600)
     run(["sudo", "install", "-m", "0600", "deploy.env", "/etc/vanguard/deploy.env"])           # read by dumpd (/deploy webhook) and deploy.sh
     if all(Path(f).exists() for f in ("deploy.sh", "systemd/deploy-pending.service", "systemd/deploy-pending.timer")):
         run(["sudo", "install", "-m", "0755", "deploy.sh", "/usr/local/bin/vanguard-deploy"])
@@ -543,6 +545,8 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis
         run(["sudo", "systemctl", "daemon-reload"]); run(["sudo", "systemctl", "enable", "--now", "deploy-pending.timer"])
     else:
         print("   (deploy pipeline files missing — deploy.sh / systemd/deploy-pending.* not uploaded; skipping vanguard-deploy install)")
+    run(["sudo", "install", "-m", "0755", "deployBotBundle.sh", "/usr/local/bin/vanguard-deploy-bot"])   # dumpd runs this for POST /deploy-bot
+    run(["sudo", "install", "-d", "-o", "1000", "-g", "1000", "/srv/bot", "/srv/bot/app"])               # the bot's released bundle -> /app
     run(["sudo", "systemctl", "restart", "dumpd"])
     print("== containers")
     dc = ["docker", "compose"] if (DRY or subprocess.call(["docker", "info"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) == 0) else ["sudo", "docker", "compose"]
@@ -608,6 +612,8 @@ def show(domain, admin_pw, streamers, editors, auth_host="", watch_host="", git_
     tok = env.get("DISCORD_BOT_TOKEN", "")
     print(f"== discord  : guild {env.get('DISCORD_GUILD_ID') or '(not set)'}; bot token {'set (' + tok[:6] + '…)' if tok else '(not set)'}")
     print(f"== mariadb  : {domain or 'HOST'}:3306 (public, root local-only)  root pass {env.get('MARIADB_ROOT_PASSWORD','(run apply)')}")
+    print(f"     bot ci : Gitea -> VanguardBot -> Settings -> Actions -> Secrets: BOT_DEPLOY_SECRET={env.get('BOT_DEPLOY_SECRET','(run apply)')}")
+    print(f"              pushes to main install themselves (BOT_DEPLOY_AUTO={env.get('BOT_DEPLOY_AUTO','1')}); by hand: ./deployBotBundle.sh <bundle>")
     print(f"     bot    : db {env.get('BOT_DB_NAME','vanguardbot')}  user {env.get('BOT_DB_USER','bot')}  pass {env.get('BOT_DB_PASSWORD','(run apply)')}")
     print(f"     gitea  : db gitea  user gitea  pass {env.get('GITEA_DB_PASSWORD','(run apply)')}   (created on MariaDB first init, else ./createGiteaDb.sh)")
     print("== access   : by Authentik group — editors (read streams/clips, write finals), media-admins (everything), admins (dump anyone)")

deployBotBundle.sh

@@ -13,7 +13,14 @@
 # --latest needs a Gitea token with read access to the repo, in .env as BOT_REPO_TOKEN (or $BOT_REPO_TOKEN).
 # Layout:  $BOT_DIR/app = live (mounted at /app)   $BOT_DIR/previous = the one before   $BOT_DIR/.incoming = staging
 set -euo pipefail
-cd "$(dirname "$0")"
+# Runs from the bundle root or as /usr/local/bin/vanguard-deploy-bot (dumpd calls it that way): find the stack.
+STACK="${VANGUARD_DIR:-}"
+if [[ -z $STACK && -f /etc/vanguard/deploy.env ]]; then
+  STACK=$(grep -E '^DEPLOY_REPO_DIR=' /etc/vanguard/deploy.env | head -1 | cut -d= -f2- || true)
+fi
+if [[ -z $STACK || ! -f $STACK/docker-compose.yml ]]; then STACK="$(cd "$(dirname "$0")" && pwd)"; fi
+[[ -f $STACK/docker-compose.yml ]] || { echo "error: cannot find the stack (set VANGUARD_DIR)" >&2; exit 1; }
+cd "$STACK"
 BOT_DIR="${BOT_DIR:-/srv/bot}"
 REPO="${BOT_REPO:-VanguardOrg/VanguardBot}"
 BRANCH="${BOT_BRANCH:-main}"