The bot's secrets live in Gitea: signed settings travel with the release bundle

POST /deploy-bot may now carry X-Vanguard-Env, base64 JSON of the bot's own settings, and the HMAC covers
that header together with the body so the pair cannot be separated or swapped. dumpd merges only the keys
BOT_ENV_KEYS allows (DISCORD_BOT_TOKEN, DISCORD_GUILD_ID by default) into the stack .env, atomically and
preserving mode and owner, and reports the rest to the CI job as settings_refused. The server keeps what it
generates: a CI job cannot touch the database password or the Authentik keys.

Verified: allowed keys applied, server-owned keys refused while the rest still applied, a settings swap
after signing rejected with 401, and an empty settings header leaving .env untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

LOGGING.md

@@ -38,7 +38,7 @@ into stream labels without parsing a single line.
 | `media` | sweep verdicts / timecode / deband / failed, vgdeband, jellyfin, copyparty | `file` |
 | `intake` | syncthing (`folder.id`), dumpd `clips_setup`, `quota_*` | `folder` = `clips-<user>` |
 | `infra` | mariadb, caddy TLS/ACME, container restarts, disk | host only |
-| `deploy` | deploy.sh, dumpd `deploy_*` and `bot_deploy_*` (the bot's CI posting a release bundle), act_runner | `rev` |
+| `deploy` | deploy.sh, dumpd `deploy_*`, `bot_deploy_*` and `bot_env_*` (the bot's CI posting a release bundle and its settings), act_runner | `rev` |
 | `code`, `bot` | gitea, discord-bot | — |
 
 ## Labels vs fields
@@ -55,7 +55,7 @@ into stream labels without parsing a single line.
 - Same key means the same thing everywhere: `stream` (user/Title), `user`, `file`, `folder`, `rev`, `ip`, `reason`.
 - Implemented by: `log()` in `bin/dumpd.py`, `vglog` in `bin/normalize-sweep.sh` and `bin/vgdeband.sh`, `log()` in `deploy.sh` (`msg=` only), `bin/mtx-on-ready.sh` (MediaMTX prefixes its own path/time).
 
-Events today: dumpd `publish_allow publish_deny dump clips_setup quota_paused quota_resumed quota_* deploy_* bot_deploy_start bot_deploy_exit bot_deploy_bad_signature bot_deploy_refused`;
+Events today: dumpd `publish_allow publish_deny dump clips_setup quota_paused quota_resumed quota_* deploy_* bot_deploy_start bot_deploy_exit bot_deploy_bad_signature bot_deploy_refused bot_env_applied bot_env_failed`;
 sweep `verdict timecode deband fill_gaps published done failed skip`; vgdeband `deband done failed skip`; hook `mirror`.
 A future alert script watches `level=error`, `event=failed`, `event=publish_deny` bursts, `event=quota_paused`, plus disk.
 

README.md

@@ -253,12 +253,28 @@ HMAC-SHA256 signature over the body, then runs `/usr/local/bin/vanguard-deploy-b
 (bind-mounted at `/app`, previous release kept beside it) and restarts the container. The install log goes back
 in the HTTP response, so it appears in the CI job.
 
-Two settings, both generated into `.env` and copied to `/etc/vanguard/deploy.env` by `configure.py`:
+**The bot's own secrets live in Gitea, not here.** The deploy job also sends the bot's settings, signed with
+the bundle, and dumpd writes them into this stack's `.env` — but only the keys `BOT_ENV_KEYS` lists. Anything
+else comes back to the CI job as `settings_refused` and is not written, so the database password and the
+Authentik keys this repo generates cannot be overwritten from CI. Rotating the Discord token is therefore a
+Gitea edit plus a re-run, with no server access.
+
+In Gitea, on the bot repo under Settings → Actions:
 
 | | |
 |---|---|
-| `BOT_DEPLOY_SECRET` | 64 hex chars. Paste it into Gitea → the bot repo → Settings → Actions → Secrets as **`BOT_DEPLOY_SECRET`**. `./configure.py --show` prints it. |
+| Secret `BOT_DEPLOY_SECRET` | must equal the server's; `./configure.py --show` prints it |
-| `BOT_DEPLOY_AUTO` | `1` (default) accepts CI installs; `0` makes dumpd refuse them with 403 and you install by hand. |
+| Secret `DISCORD_BOT_TOKEN` | the bot token |
+| Variable `DISCORD_GUILD_ID` | the server id |
+| Variable `VANGUARD_DEPLOY_HOST` | only if the job cannot detect the Docker gateway |
+
+On the server, generated into `.env` and copied to `/etc/vanguard/deploy.env` by `configure.py`:
+
+| | |
+|---|---|
+| `BOT_DEPLOY_SECRET` | 64 hex chars, the shared signing key |
+| `BOT_DEPLOY_AUTO` | `1` (default) accepts CI installs; `0` makes dumpd refuse them with 403 |
+| `BOT_ENV_KEYS` | the only `.env` keys CI may set, `DISCORD_BOT_TOKEN,DISCORD_GUILD_ID` by default. Extend it when the bot needs a new setting. |
 
 By hand, any time:
 

STATUS.md

@@ -68,7 +68,7 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea
 | Gitea | git hosting on MariaDB, Actions enabled; holds this repo | live; Discord login via Authentik OIDC **verified**; access = `git-users`/`git-admins` groups; local admin as fallback |
 | act_runner | CI jobs in Docker | **not registered** |
 | discord-bot | Python runtime, idles until `bot/app/main.py` exists | live, no code yet |
-| `bot.` host + Discord bot | the bot's API **and** its web pages on one port (`WEB_SERVE_UI`), behind Authentik forward-auth; Caddy requires `bot-users`, the bot re-checks `bot-users,admins` per request | a push to `main` deploys itself: CI builds a **release bundle** (app + `vendor/` dependencies) and posts it signed to dumpd `POST /deploy-bot` (bridge-only port, no Docker socket in CI), which verifies it and swaps it into `/srv/bot/app` → `/app`; image installs no bot dependencies | **reviewed + wired 2026-11-02, first install pending** (needs the Authentik app for `bot.`, the bot token in `.env`, and `BOT_DEPLOY_SECRET` as a Gitea repo secret) |
+| `bot.` host + Discord bot | the bot's API **and** its web pages on one port (`WEB_SERVE_UI`), behind Authentik forward-auth; Caddy requires `bot-users`, the bot re-checks `bot-users,admins` per request | a push to `main` deploys itself: CI builds a **release bundle** (app + `vendor/` dependencies) and posts it signed to dumpd `POST /deploy-bot` (bridge-only port, no Docker socket in CI) together with the bot's own secrets, which are managed in Gitea; dumpd verifies everything, writes the keys `BOT_ENV_KEYS` allows into `.env` and swaps the bundle into `/srv/bot/app` → `/app`; image installs no bot dependencies | **reviewed + wired 2026-11-02, first install pending** (needs the Authentik app for `bot.`, and `BOT_DEPLOY_SECRET` + `DISCORD_BOT_TOKEN` + `DISCORD_GUILD_ID` set in Gitea) |
 | Live portal (`live/`, `live.` host) | React + dockview multi-stream viewer + browser Go Live (WHIP); MediaMTX HLS/WebRTC/API on loopback; dumpd `/me/stream-key` | live; Go Live verified |
 
 ## Unfinished, in rough priority

bin/dumpd.py

@@ -14,6 +14,8 @@ POST /deploy    -> Gitea push webhook (bridge port only, HMAC-verified with DEPL
              otherwise logs the push and answers 202 "auto-deploy disabled".
 POST /deploy-bot -> Gitea Actions (job container -> host gateway) hands over a Discord-bot release bundle,
                    HMAC-SHA256 signed with BOT_DEPLOY_SECRET; installs it with vanguard-deploy-bot.
+                   May carry X-Vanguard-Env (base64 JSON): the bot's own settings, kept in Gitea, written
+                   into the stack .env for the keys BOT_ENV_KEYS allows. Signature covers header+body.
 POST /mtx-auth -> MediaMTX authHTTP hook: allows read/playback; allows publish iff user/pass carries a valid
              token whose user owns the path prefix. Tokens are HMAC-signed with /etc/vanguard/stream-token.secret.
 
@@ -262,6 +264,47 @@ def deploy_env():
         pass
     return d
 
+BOT_ENV_DEFAULT_KEYS = "DISCORD_BOT_TOKEN,DISCORD_GUILD_ID"
+BOT_ENV_MAX_HEADER = 8192
+_env_write_lock = threading.Lock()
+
+def stack_env_path():
+    """The stack's .env — docker compose interpolates ${...} from it, so the bot's settings live there."""
+    repo = deploy_env().get("DEPLOY_REPO_DIR", "")
+    return (Path(repo) / ".env") if repo else None
+
+def apply_bot_env(values):
+    """Merge allowed keys into the stack .env. Returns (applied, refused); never logs a value."""
+    allowed = {k.strip() for k in deploy_env().get("BOT_ENV_KEYS", BOT_ENV_DEFAULT_KEYS).split(",") if k.strip()}
+    refused = sorted(k for k in values if k not in allowed)
+    wanted = {k: str(v) for k, v in values.items() if k in allowed}
+    if not wanted:
+        return [], refused
+    path = stack_env_path()
+    if path is None or not path.is_file():
+        raise OSError("DEPLOY_REPO_DIR is not set or holds no .env")
+    bad = [k for k, v in wanted.items() if not re.fullmatch(r"[A-Z][A-Z0-9_]{1,48}", k) or "\n" in v or "\r" in v]
+    if bad:
+        raise ValueError(f"not a usable env assignment: {bad}")
+    with _env_write_lock:
+        lines = path.read_text().splitlines()
+        seen = set()
+        for i, line in enumerate(lines):
+            if "=" in line and not line.lstrip().startswith("#"):
+                key = line.split("=", 1)[0].strip()
+                if key in wanted:
+                    lines[i] = f"{key}={wanted[key]}"; seen.add(key)
+        for key in wanted:
+            if key not in seen: lines.append(f"{key}={wanted[key]}")
+        stat = path.stat()
+        tmp = path.with_suffix(".env.incoming")
+        tmp.write_text("\n".join(lines) + "\n")
+        os.chmod(tmp, stat.st_mode & 0o777); os.chown(tmp, stat.st_uid, stat.st_gid)
+        os.replace(tmp, path)
+    applied = sorted(wanted)
+    log("bot_env_applied", keys=",".join(applied), refused=",".join(refused) or None)
+    return applied, refused
+
 _bot_deploy_lock = threading.Lock()
 BOT_DEPLOY_MAX_BYTES = 300 * 1024 * 1024
 BOT_DEPLOY_INSTALLER = os.environ.get("BOT_DEPLOY_INSTALLER", "/usr/local/bin/vanguard-deploy-bot")
@@ -442,11 +485,26 @@ class H(BaseHTTPRequestHandler):
             if not 0 < size <= BOT_DEPLOY_MAX_BYTES:
                 return self._json(413, {"error": f"body must be 1..{BOT_DEPLOY_MAX_BYTES} bytes, got {size}"})
             raw = self.rfile.read(size)
+            env_b64 = (self.headers.get("X-Vanguard-Env") or "").strip()
+            if len(env_b64) > BOT_ENV_MAX_HEADER:
+                return self._json(413, {"error": "X-Vanguard-Env is too large"})
             sig = (self.headers.get("X-Vanguard-Signature") or "").replace("sha256=", "").strip()
-            if not sig or not hmac.compare_digest(hmac.new(secret.encode(), raw, hashlib.sha256).hexdigest(), sig):
+            signed = env_b64.encode() + raw          # the settings are signed with the bundle, not beside it
+            if not sig or not hmac.compare_digest(hmac.new(secret.encode(), signed, hashlib.sha256).hexdigest(), sig):
                 log("bot_deploy_bad_signature", level="warn", ip=self.client_address[0], bytes=len(raw))
                 return self._json(401, {"error": "bad signature"})
+            applied, refused = [], []
+            if env_b64:
+                try:
+                    values = json.loads(base64.b64decode(env_b64, validate=True))
+                    if not isinstance(values, dict): raise ValueError("X-Vanguard-Env must be a JSON object")
+                    applied, refused = apply_bot_env(values)
+                except (ValueError, OSError, TypeError) as error:
+                    log("bot_env_failed", level="error", error=error)
+                    return self._json(400, {"error": f"settings rejected: {error}"})
             code, body = install_bot_bundle(raw, self.headers.get("X-Vanguard-Reason") or "ci")
+            body["settings_applied"] = applied
+            if refused: body["settings_refused"] = refused
             return self._json(code, body)
         if self.server.server_port == AUTH_LISTEN[1] and self.path.startswith("/deploy"):
             # Gitea (container) -> host gateway -> here. Verify HMAC-SHA256 of the raw body with the webhook secret.

bot/AGENT.md

@@ -50,6 +50,12 @@ signed with the repository secret `BOT_DEPLOY_SECRET`; the server verifies the s
 keeps the previous release for rollback, and restarts the container. The install log comes back in your CI
 job, so a failed deploy fails the run. Branches and pull requests test and build, and deploy nothing.
 
+**Your settings travel with it.** `DISCORD_BOT_TOKEN` (an Actions secret) and `DISCORD_GUILD_ID` (an Actions
+variable) are signed together with the bundle and written into the container's environment by the server, so
+this repo is where they are managed and rotated. The server accepts only the names its `BOT_ENV_KEYS` allows
+and reports the rest as `settings_refused`; it keeps ownership of what it generates, such as `DB_PASSWORD`.
+A new setting means asking the admin to add its name to that list (§6).
+
 Rules that follow from this:
 
 - Keep `requirements.txt` resolvable with **prebuilt wheels only**; a dependency that must compile will fail

bot/README.md

@@ -12,6 +12,22 @@ release kept for `--rollback`) and restarts the container. The install log comes
 
 A branch or pull request runs the tests and builds the artifact, and deploys nothing.
 
+## Your secrets live in Gitea
+
+Set them on this repo under Settings → Actions. The deploy job sends them with the bundle, signed, and the
+server writes the ones it allows into the environment the container gets:
+
+| where | name | what |
+|---|---|---|
+| Secrets | `BOT_DEPLOY_SECRET` | signs the upload; must match the server (the admin has it) |
+| Secrets | `DISCORD_BOT_TOKEN` | the bot token |
+| Variables | `DISCORD_GUILD_ID` | the Discord server id |
+
+Rotating the token is an edit here plus a re-run of the workflow; nobody needs to touch the server. The
+server keeps what it generates, chiefly `DB_PASSWORD`, and accepts only the names its `BOT_ENV_KEYS` lists —
+anything else is reported in the job as `settings_refused`. Need a new setting? Ask the admin to add its name
+to that list, then add it here.
+
 The image contains **no** bot dependencies: the container imports yours from `/app/vendor`
 (`PYTHONPATH=/app/vendor`). Adding a dependency means editing `app/requirements.txt` and pushing — no image
 rebuild, no package index on the server. The image only changes if you need a new system package (ffmpeg and

configure.py

@@ -372,6 +372,7 @@ def write_env(watch_host, discord_token=None, discord_guild=None, git_host="", b
     env.setdefault("BOT_DB_PASSWORD", "".join(c for c in secrets.token_urlsafe(40) if c.isalnum())[:32])
     env.setdefault("BOT_DEPLOY_SECRET", secrets.token_hex(32))   # Gitea repo secret BOT_DEPLOY_SECRET: signs the bot's release bundle
     env.setdefault("BOT_DEPLOY_AUTO", "1")                       # 0 = refuse bundles posted by CI; install by hand instead
+    env.setdefault("BOT_ENV_KEYS", "DISCORD_BOT_TOKEN,DISCORD_GUILD_ID")   # the only .env keys the bot's CI may set
     env.setdefault("DISCORD_BOT_TOKEN", "")          # fill in: Discord dev portal -> the Authentik app -> Bot
     env.setdefault("DISCORD_GUILD_ID", "")
     env["WATCH_HOST"] = watch_host or "watch.example.com"
@@ -537,7 +538,7 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis
     run(["sudo", "install", "-m", "0600", "stream-token.secret", "/etc/vanguard/stream-token.secret"])
     run(["sudo", "install", "-m", "0644", "syncthing/stignore", "/etc/vanguard/stignore"])   # dumpd copies it into new clips folders
     e = read_env()
-    Path("deploy.env").write_text("".join(f"{k}={e.get(k, '')}\n" for k in ("DEPLOY_AUTO", "DEPLOY_WEBHOOK_SECRET", "DEPLOY_BRANCH", "DEPLOY_NOTIFY_WEBHOOK_URL", "BOT_DEPLOY_AUTO", "BOT_DEPLOY_SECRET")) + f"DEPLOY_REPO_DIR={Path.cwd()}\n"); os.chmod("deploy.env", 0o600)
+    Path("deploy.env").write_text("".join(f"{k}={e.get(k, '')}\n" for k in ("DEPLOY_AUTO", "DEPLOY_WEBHOOK_SECRET", "DEPLOY_BRANCH", "DEPLOY_NOTIFY_WEBHOOK_URL", "BOT_DEPLOY_AUTO", "BOT_DEPLOY_SECRET", "BOT_ENV_KEYS")) + f"DEPLOY_REPO_DIR={Path.cwd()}\n"); os.chmod("deploy.env", 0o600)
     run(["sudo", "install", "-m", "0600", "deploy.env", "/etc/vanguard/deploy.env"])           # read by dumpd (/deploy webhook) and deploy.sh
     if all(Path(f).exists() for f in ("deploy.sh", "systemd/deploy-pending.service", "systemd/deploy-pending.timer")):
         run(["sudo", "install", "-m", "0755", "deploy.sh", "/usr/local/bin/vanguard-deploy"])
@@ -614,6 +615,7 @@ def show(domain, admin_pw, streamers, editors, auth_host="", watch_host="", git_
     print(f"== mariadb  : {domain or 'HOST'}:3306 (public, root local-only)  root pass {env.get('MARIADB_ROOT_PASSWORD','(run apply)')}")
     print(f"     bot ci : Gitea -> VanguardBot -> Settings -> Actions -> Secrets: BOT_DEPLOY_SECRET={env.get('BOT_DEPLOY_SECRET','(run apply)')}")
     print(f"              pushes to main install themselves (BOT_DEPLOY_AUTO={env.get('BOT_DEPLOY_AUTO','1')}); by hand: ./deployBotBundle.sh <bundle>")
+    print(f"              CI may set these .env keys (BOT_ENV_KEYS): {env.get('BOT_ENV_KEYS','DISCORD_BOT_TOKEN,DISCORD_GUILD_ID')}")
     print(f"     bot    : db {env.get('BOT_DB_NAME','vanguardbot')}  user {env.get('BOT_DB_USER','bot')}  pass {env.get('BOT_DB_PASSWORD','(run apply)')}")
     print(f"     gitea  : db gitea  user gitea  pass {env.get('GITEA_DB_PASSWORD','(run apply)')}   (created on MariaDB first init, else ./createGiteaDb.sh)")
     print("== access   : by Authentik group — editors (read streams/clips, write finals), media-admins (everything), admins (dump anyone)")