diff --git a/.gitignore b/.gitignore index e854d15..abf5d25 100644 --- a/.gitignore +++ b/.gitignore @@ -10,3 +10,4 @@ copyparty.conf Caddyfile stream-token.secret deploy.env +bot/app/ diff --git a/README.md b/README.md index 2765d8b..bf4e728 100644 --- a/README.md +++ b/README.md @@ -236,21 +236,33 @@ Phase 4 is the Vanguard Metadata Jellyfin plugin; Phase 5 puts copyparty, /sync ## 5d. MariaDB + Discord bot (for the bot dev) `mariadb` (public on 3306, host network, root local-only, 32-char generated passwords) with a database/user -for the bot, and `discord-bot`, a Python 3.12 container that runs `bot/app/main.py` and idles until that file -exists. Credentials live in `.env` (generated by `configure.py`; bot token + guild ID are asked in `--full` +for the bot, and `discord-bot`, a Python 3.12 container that runs `/app/main.py` and idles until a release is +installed. Credentials live in `.env` (generated by `configure.py`; bot token + guild ID are asked in `--full` or the `[d]` menu item; `./configure.py --show` prints them). -The bot also serves a web UI on 127.0.0.1:8000, published as `https://bot.HOST/` behind **Authentik -forward-auth**: Caddy asks Authentik, then injects `X-Authentik-Username/-Groups/-Email`. Create the matching -Proxy Provider in Authentik (forward auth, single application, external host `https://bot.HOST`), add it to -the embedded outpost, and bind a group policy. Until then the site returns an error, never an open page. +**Releases are self-contained bundles, not loose files.** The bot repo's Gitea Actions workflow runs its tests +and then builds one tar.gz holding `main.py`, `app/`, `website/`, `BUILD.json` **and `vendor/`** — its own +dependencies, as prebuilt wheels for this image's Python. `./deployBotBundle.sh ` verifies it +against `BUILD.json`, swaps it into `/srv/bot/app` (bind-mounted at `/app`, previous release kept beside it) +and restarts the container. `PYTHONPATH=/app/vendor` is how the code finds its dependencies, so **a new +dependency never needs an image rebuild** and the server never contacts a package index. + + ./deployBotBundle.sh vanguard-bot-bundle.tar.gz # install a release (downloaded from the ci workflow) + ./deployBotBundle.sh --latest # fetch the newest main build first (needs BOT_REPO_TOKEN in .env) + ./deployBotBundle.sh --status # version, commit, package list, container state + ./deployBotBundle.sh --rollback # back to the previous bundle + docker compose logs -f discord-bot # watch it + +The bot serves **both** its API and its web pages on 127.0.0.1:8000, published as `https://bot.HOST/` behind +**Authentik forward-auth**: Caddy asks Authentik, then injects `X-Authentik-Username/-Groups/-Email`, and the +bot authorises per request from those headers (`WEB_ALLOWED_GROUPS=bot-users,admins`; `WEB_SERVE_UI=true` +turns the pages on). Create the matching Proxy Provider in Authentik (forward auth, single application, +external host `https://bot.HOST`), add it to the embedded outpost, and bind a group policy. Until then the +site returns an error, never an open page. Caddy's own gate is `BOT_REQUIRE_GROUP` (`bot-users` by default), +so **admins need the `bot-users` role too** — or set `BOT_REQUIRE_GROUP=` and let the bot's group list decide. The build brief for whoever writes the bot is `bot/AGENT.md`; `bot/README.md` is the ops cheat-sheet. - docker compose logs -f discord-bot # watch it - docker compose restart discord-bot # after code changes - docker compose build discord-bot && docker compose up -d discord-bot # after requirements.txt changes - ## 5e. Gitea + Actions runner `gitea` (MariaDB-backed, `git.HOST` via Caddy, git-over-ssh on **2222**, registration closed) and diff --git a/STATUS.md b/STATUS.md index 66cd581..91f528b 100644 --- a/STATUS.md +++ b/STATUS.md @@ -68,7 +68,7 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea | Gitea | git hosting on MariaDB, Actions enabled; holds this repo | live; Discord login via Authentik OIDC **verified**; access = `git-users`/`git-admins` groups; local admin as fallback | | act_runner | CI jobs in Docker | **not registered** | | discord-bot | Python runtime, idles until `bot/app/main.py` exists | live, no code yet | -| `bot.` host | bot web UI behind Authentik forward-auth; Caddy requires group `bot-users` (from a Discord role) | Caddy block live; **`bot` CNAME still points at the wrong IP; roles→groups policy pending** | +| `bot.` host + Discord bot | the bot's API **and** its web pages on one port (`WEB_SERVE_UI`), behind Authentik forward-auth; Caddy requires `bot-users`, the bot re-checks `bot-users,admins` per request | code arrives as a CI **release bundle** (app + `vendor/` dependencies) installed by `./deployBotBundle.sh` into `/srv/bot/app` → `/app`; image installs no bot dependencies | **reviewed + wired 2026-11-02, first install pending** (needs the Authentik app for `bot.`, the bot token in `.env`, and a bundle from the ci workflow) | | Live portal (`live/`, `live.` host) | React + dockview multi-stream viewer + browser Go Live (WHIP); MediaMTX HLS/WebRTC/API on loopback; dumpd `/me/stream-key` | live; Go Live verified | ## Unfinished, in rough priority @@ -83,7 +83,9 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea 6. Medal filename pattern in the plugin (need a sample filename). 7. Home archive puller on the home box. 8. Plugin repository served by Caddy so Jellyfin updates from its catalog. -9. Clip submissions (belongs to the bot). Bot code from the other dev. +9. Discord bot: first bundle install (`./deployBotBundle.sh`), Authentik application for `bot.`, first real MariaDB run + (the bot's own README flags it as never tested end to end), then clip submissions. Review notes: its pages are served + from the bot itself now; an Authentik username is matched to one Discord member by name. 10. Authentik blueprint from `.env` (Discord source, groups, policies, provider) for reproducible installs. 11. SYS-3 when transcoding or sweep speed hurts (2×6 TB option currently unavailable; watching for it). Interim option documented: off-site gap filling on the home NVENC box. diff --git a/bot/.dockerignore b/bot/.dockerignore index 5812ae6..d928727 100644 --- a/bot/.dockerignore +++ b/bot/.dockerignore @@ -1,2 +1,4 @@ -app/__pycache__/ -app/*.pyc +# only entrypoint.sh is COPYed; keep everything else out of the build context +app/ +AGENT.md +README.md diff --git a/bot/AGENT.md b/bot/AGENT.md index 1f8823d..91bfc11 100644 --- a/bot/AGENT.md +++ b/bot/AGENT.md @@ -15,23 +15,43 @@ What you are given: | | | |---|---| -| **Runtime** | `python:3.12-slim` + ffmpeg + curl. Dependencies from `app/requirements.txt`, installed at image build (`docker compose build discord-bot`). | -| **Your code** | bind-mounted at `/app` from `bot/app/`. Not baked into the image. Edit, then `docker compose restart discord-bot`. | -| **Entry point** | the container runs `python -u /app/main.py` and restarts it if it exits. With no `main.py` it idles. | +| **Runtime** | `python:3.12-slim` + ffmpeg + curl, and nothing else. **Your dependencies ship with your code** (see §1a), imported from `/app/vendor` via `PYTHONPATH`. Adding one needs no image rebuild. | +| **Your code** | a release bundle from your CI, installed at `/srv/bot/app` and bind-mounted at `/app` (§1a). Treat `/app` as read-only; write to `/data`. | +| **Entry point** | the container runs `python -u /app/main.py` and restarts it if it exits. With no bundle installed it idles and logs a hint. | | **User** | uid 1000 | | **Database** | MariaDB, database `vanguardbot`, user `bot`, reachable at `DB_HOST:DB_PORT`. Yours alone. Other databases on the same server belong to other services; never touch them. | | **Persistent disk** | `/data`, yours | -| **Web UI** | bind `WEB_HOST:WEB_PORT` (0.0.0.0:8000). The proxy publishes it as `https://bot./` behind login (§3). | +| **Web UI** | bind `WEB_HOST:WEB_PORT` (0.0.0.0:8000) — one port for your API *and* your pages. The proxy publishes it as `https://bot./` behind login (§3). | | **Discord** | `DISCORD_BOT_TOKEN`, `DISCORD_GUILD_ID`. The bot user belongs to the community's existing Discord application. | | **Clock** | `TZ=America/New_York` | ### Environment variables -`DISCORD_BOT_TOKEN`, `DISCORD_GUILD_ID`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD`, -`WEB_HOST`, `WEB_PORT`, `WEB_PUBLIC_URL`, `AUTH_HEADER_USER`, `AUTH_HEADER_GROUPS`, `AUTH_HEADER_EMAIL`, -`MEDIA_CLIPS`, `TZ`. Secrets arrive only through env. Never write them to the repo, `/data`, or logs. +`DISCORD_BOT_TOKEN`, `DISCORD_GUILD_ID`, `DB_BACKEND=mariadb`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, +`DB_PASSWORD`, `DATA_DIR=/data`, `WEB_HOST`, `WEB_PORT`, `WEB_PUBLIC_URL`, `WEB_SERVE_UI=true`, +`WEB_ALLOWED_GROUPS=bot-users,admins`, `AUTH_HEADER_USER`, `AUTH_HEADER_GROUPS`, `AUTH_HEADER_EMAIL`, +`MEDIA_CLIPS`, `PYTHONPATH=/app/vendor`, `TZ`. Secrets arrive only through env. Never write them to the repo, +`/data`, or logs. SQLAlchemy URL: `mysql+pymysql://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}` (async: `mysql+aiomysql`). +## 1a. Releases: one bundle, dependencies included + +Your CI (`.gitea/workflows/ci.yml`, job `bundle`) runs after the tests and produces the artifact +`vanguard-bot-bundle` — a single tar.gz whose contents **are** `/app`: + + main.py app/ website/ version.json BUILD.json vendor/ + +`vendor/` is `pip install --only-binary=:all: --target vendor -r requirements.txt`, built on `python:3.12` +so the wheels match the runtime image's ABI. `BUILD.json` records the version, commit, build time, the +package list and a sha256 of every file. The admin installs it with `./deployBotBundle.sh `, +which checks the bundle against `BUILD.json`, keeps the previous release for `--rollback`, and restarts the +container. Rules that follow from this: + +- Keep `requirements.txt` resolvable with **prebuilt wheels only**; a dependency that must compile will fail + the build. Ask the admin if you truly need one. +- Don't import anything that is not in `requirements.txt` or the standard library. +- Nothing is installed on the server at deploy time, and the server reaches no package index. + ## 2. Process model - One process, one asyncio loop: the Discord gateway client and the web server run together @@ -52,6 +72,10 @@ Every request to `https://bot./` passes through Caddy, which sends the u | `X-Authentik-Groups` | group names, separated by `\|` or `,` — the community uses `admins`, `editors`, `streamers`; others may be added | | `X-Authentik-Email`, `X-Authentik-Name`, `X-Authentik-Uid` | extras | +`WEB_SERVE_UI=true` is set on the server, so this process serves your HTML pages at `/` and their assets +at `/assets/` as well as `/api/…` — same origin, one port, no second service and no proxy of your own. +Locally the separate website launcher still works; it is off unless `WEB_SERVE_UI` says otherwise. + Rules: 1. **Implement no login page, no sessions, no passwords.** Identity is the headers, per request. 2. Port 8000 is loopback-only on the host, so the headers can be trusted. Still fail closed: a request @@ -99,10 +123,12 @@ message content; Message Content is a privileged intent, enable it only if you n ## 6. Working with the admin -- **Source**: push to the community Gitea, `ssh://git@git.:2222/vanguard/discord-bot.git`. - The admin deploys by pulling into `bot/app` and restarting. CI on the `python` runner label runs your tests. +- **Source**: push to the community Gitea. CI runs your tests on the `python` runner label, then builds the + release bundle (§1a); the admin installs it with `deployBotBundle.sh`. Pushes to `main` can install it + automatically once the repository variable `BOT_AUTO_DEPLOY` is set to `1`; it is unset by default. - **Need infra?** Open a Gitea issue titled `infra: …` stating the exact change: a new env var and why, a mount, a Caddy path that should skip login (e.g. a public webhook endpoint), a new database. Don't work around the proxy or the firewall. -- **Dependencies changed?** Say so in the PR; the image must be rebuilt. +- **Dependencies changed?** Just push: they travel inside the bundle. Only a new *system* package (beyond + ffmpeg and curl) needs the admin to rebuild the image. - Keep `README.md` updated with every command, env var, and table you add. diff --git a/bot/Dockerfile b/bot/Dockerfile index 155b7a8..22f195d 100644 --- a/bot/Dockerfile +++ b/bot/Dockerfile @@ -1,12 +1,11 @@ -# Python runtime for the Discord bot. Source is bind-mounted at /app from ./bot/app (not baked in), -# so the dev edits code and restarts the container. Dependencies come from app/requirements.txt and -# are installed at BUILD time: after changing it run docker compose build discord-bot && docker compose up -d discord-bot +# Runtime for the Discord bot. The bot's CI builds a self-contained bundle (code + vendor/ dependencies) +# and deployBotBundle.sh installs it at BOT_APP_DIR, bind-mounted here at /app. Dependencies therefore live +# WITH the code, not in this image: a new dependency needs no rebuild and no package index on the server. +# Only the interpreter and the tools AGENT.md promises are baked in. FROM python:3.12-slim -ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1 +ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1 PYTHONDONTWRITEBYTECODE=1 RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg ca-certificates curl \ && rm -rf /var/lib/apt/lists/* -COPY app/requirements.txt /tmp/requirements.txt -RUN pip install -r /tmp/requirements.txt COPY entrypoint.sh /entrypoint.sh RUN chmod +x /entrypoint.sh WORKDIR /app diff --git a/bot/README.md b/bot/README.md index 3bd38ab..4e69031 100644 --- a/bot/README.md +++ b/bot/README.md @@ -2,29 +2,42 @@ Start with **AGENT.md** in this folder: the stack, the rules, the layout, the web-UI auth contract. -Your code goes in this folder (`bot/app/`). The container runs `python -u /app/main.py` and restarts -it if it exits. No `main.py` yet → the container idles and logs a hint. +## How code reaches the server + +Your repo's Gitea Actions workflow builds a **release bundle**: one tar.gz holding `main.py`, `app/`, +`website/`, `version.json`, `BUILD.json` and `vendor/` (your dependencies as prebuilt wheels). The admin +installs it with `./deployBotBundle.sh ` from the stack root; it lands in `/srv/bot/app`, +which is bind-mounted into the container at `/app`, and the previous release is kept for `--rollback`. + +The image contains **no** bot dependencies: the container imports yours from `/app/vendor` +(`PYTHONPATH=/app/vendor`). Adding a dependency means editing `app/requirements.txt` and pushing — no image +rebuild, no package index on the server. The image only changes if you need a new system package (ffmpeg and +curl are already there); ask the admin for that. ## Day to day (on the server, in ~/vanguard-stream) - docker compose logs -f discord-bot # live log - docker compose restart discord-bot # after editing code - docker compose build discord-bot && docker compose up -d discord-bot # after editing requirements.txt - docker compose exec discord-bot python # a REPL inside the container - -Edit files directly in `~/vanguard-stream/bot/app/` (they are bind-mounted) or scp them up. + ./deployBotBundle.sh vanguard-bot-bundle.tar.gz # install a release + ./deployBotBundle.sh --status # what is installed; container state + ./deployBotBundle.sh --rollback # previous release + docker compose logs -f discord-bot # live log + docker compose restart discord-bot # restart without changing code + docker compose exec discord-bot python # a REPL inside the container (your vendor/ is on sys.path) ## Environment the container gives you | var | value | |-----|-------| -| `DISCORD_BOT_TOKEN` | from `.env` (bundle root) — the Bot added to the existing Authentik Discord application | +| `DISCORD_BOT_TOKEN` | from `.env` (stack root) — the Bot added to the existing Authentik Discord application | | `DISCORD_GUILD_ID` | the server | +| `DB_BACKEND` | `mariadb` | | `DB_HOST` / `DB_PORT` | `host.docker.internal` / `3306` (MariaDB runs on the host network) | | `DB_NAME` / `DB_USER` / `DB_PASSWORD` | your own database + user (`vanguardbot` / `bot` / generated) | +| `DATA_DIR` | `/data` — persistent, owned by you | | `MEDIA_CLIPS` | `/media/clips` — rw mount of `/srv/media/clips` (folder per user, for clip submissions) | -| `/data` | persistent scratch dir owned by the bot | -| `WEB_HOST` / `WEB_PORT` | `0.0.0.0` / `8000` — your web UI; public as `https://bot./` via Caddy + Authentik | +| `WEB_HOST` / `WEB_PORT` | `0.0.0.0` / `8000` — published as `https://bot./` via Caddy + Authentik | | `WEB_PUBLIC_URL` | `https://bot.` for absolute links | +| `WEB_SERVE_UI` | `true` — serve your pages at `/` on the same port as `/api` | +| `WEB_ALLOWED_GROUPS` | `bot-users,admins` — the groups your own code must accept | | `AUTH_HEADER_USER` / `_GROUPS` / `_EMAIL` | `X-Authentik-Username` / `X-Authentik-Groups` / `X-Authentik-Email` — identity injected by Caddy, trust these, implement no login | +| `PYTHONPATH` | `/app/vendor` | | `TZ` | America/New_York | SQLAlchemy URL: `mysql+pymysql://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}` @@ -37,7 +50,8 @@ rate limited at the firewall; if yours get dropped, ask the admin to add your IP ## Conventions - Run as uid 1000 (files you write under `/media/clips` are owned like everything else there). +- `/app` holds a released bundle; treat it as read-only and write to `/data` instead. - If you write clips: `/media/clips//.mp4` plus `.json` with `{"timestamp": "", "source": "medal", "source_url": "...", "uploader": "...", "title": "..."}` — the Jellyfin plugin reads that sidecar for the exact recording time. -- Log to stdout; the container captures it. +- Log to stdout; the container captures it. One event per line, `key=value`, keeps it searchable (`LOGGING.md`). diff --git a/bot/app/requirements.txt b/bot/app/requirements.txt deleted file mode 100644 index d14ef59..0000000 --- a/bot/app/requirements.txt +++ /dev/null @@ -1,8 +0,0 @@ -# Baseline for a discord.py bot with MariaDB. Add what you need; then: -# docker compose build discord-bot && docker compose up -d discord-bot -discord.py>=2.4 -aiomysql>=0.2 -PyMySQL>=1.1 -SQLAlchemy>=2.0 -python-dotenv>=1.0 -yt-dlp diff --git a/bot/entrypoint.sh b/bot/entrypoint.sh index 7becf9a..5c4291e 100644 --- a/bot/entrypoint.sh +++ b/bot/entrypoint.sh @@ -1,7 +1,13 @@ #!/bin/sh -# Run the bot if there is one; otherwise idle so the container stays up until code arrives. +# /app is a bundle built by the bot's CI: main.py + app/ + website/ + vendor/ (its dependencies). +# PYTHONPATH=/app/vendor is set by compose, so nothing is installed at run time. if [ -f /app/main.py ]; then + if [ ! -d /app/vendor ]; then + echo "discord-bot: /app has code but no vendor/ — this image installs no bot dependencies." + echo " Install a CI bundle: ./deployBotBundle.sh " + fi exec python -u /app/main.py fi -echo "discord-bot: no /app/main.py yet — drop the bot code into ./bot/app and 'docker compose restart discord-bot'" +echo "discord-bot: /app is empty — install a bundle from the bot's CI: ./deployBotBundle.sh " +echo " (artifact 'vanguard-bot-bundle' on the VanguardBot repo's ci workflow)" exec sleep infinity diff --git a/check.sh b/check.sh index 3414a47..9ef5f92 100755 --- a/check.sh +++ b/check.sh @@ -5,7 +5,7 @@ cd "$(dirname "$0")" fail=0 say() { printf '%-28s %s\n' "$1" "$2"; } -for f in install.sh addCommunityMediaServer.sh installJellyfinPlugin.sh registerGiteaRunner.sh createGiteaDb.sh deployLive.sh addClipUser.sh deploy.sh switchToClone.sh firewall/firewall.sh bin/*.sh home/*.sh bot/entrypoint.sh mariadb/init/*.sh; do +for f in install.sh addCommunityMediaServer.sh installJellyfinPlugin.sh registerGiteaRunner.sh createGiteaDb.sh deployLive.sh addClipUser.sh deploy.sh deployBotBundle.sh switchToClone.sh firewall/firewall.sh bin/*.sh home/*.sh bot/entrypoint.sh mariadb/init/*.sh; do [[ -f $f ]] || continue bash -n "$f" && say "$f" ok || { say "$f" "BASH SYNTAX"; fail=1; } done diff --git a/deploy.sh b/deploy.sh index a587282..754523c 100755 --- a/deploy.sh +++ b/deploy.sh @@ -61,7 +61,7 @@ apply_changes() { if has '^bin/dumpd\.py$|^systemd/dumpd\.service$'; then run systemctl restart dumpd; fi # 3. compose: recreate changed services, but never MediaMTX while live - if has '^docker-compose\.yml$|^bot/Dockerfile$|^bot/app/requirements\.txt$'; then + if has '^docker-compose\.yml$|^bot/Dockerfile$|^bot/entrypoint\.sh$'; then local mtx_changed=0 git diff "$old" "$new" -- docker-compose.yml | grep -qE '^[+-]' && git diff -U0 "$old" "$new" -- docker-compose.yml | grep -qiE '^[+-].*(mediamtx|8189|8890|1935)' && mtx_changed=1 if [[ $mtx_changed == 1 && $(live_streams) != 0 ]]; then diff --git a/deployBotBundle.sh b/deployBotBundle.sh new file mode 100755 index 0000000..5c36712 --- /dev/null +++ b/deployBotBundle.sh @@ -0,0 +1,111 @@ +#!/bin/bash +# Install a Vanguard Bot release bundle into the place the discord-bot container expects it. +# +# The bundle comes from the bot repo's CI (workflow "ci", job "bundle", artifact "vanguard-bot-bundle"): +# one tar.gz holding main.py, app/, website/, version.json, BUILD.json and vendor/ — the bot's own +# dependencies. Nothing is installed here and the server needs no package index. +# +# ./deployBotBundle.sh vanguard-bot-bundle.tar.gz install that file +# ./deployBotBundle.sh --latest download the newest main build from Gitea first +# ./deployBotBundle.sh --rollback put the previous bundle back +# ./deployBotBundle.sh --status what is installed right now +# +# --latest needs a Gitea token with read access to the repo, in .env as BOT_REPO_TOKEN (or $BOT_REPO_TOKEN). +# Layout: $BOT_DIR/app = live (mounted at /app) $BOT_DIR/previous = the one before $BOT_DIR/.incoming = staging +set -euo pipefail +cd "$(dirname "$0")" +BOT_DIR="${BOT_DIR:-/srv/bot}" +REPO="${BOT_REPO:-VanguardOrg/VanguardBot}" +BRANCH="${BOT_BRANCH:-main}" +fromenv() { [[ -f .env ]] && grep -E "^$1=" .env | head -1 | cut -d= -f2- || true; } +GIT_HOST="$(fromenv GIT_HOST)" +BOT_HOST="$(fromenv BOT_HOST)" +BOT_REPO_TOKEN="${BOT_REPO_TOKEN:-$(fromenv BOT_REPO_TOKEN)}" + +say() { printf '\n== %s\n' "$*"; } +die() { echo "error: $*" >&2; exit 1; } + +summary() { # summary + python3 - "$1" <<'PY' +import json,sys,pathlib +root=pathlib.Path(sys.argv[1]); build=root/'BUILD.json' +if not build.is_file(): print(' (no BUILD.json — not a CI bundle)'); sys.exit(0) +d=json.loads(build.read_text()) +print(f" version {d.get('version')} commit {(d.get('commit') or '?')[:8]} built {d.get('built_at')} {d.get('python')} ({d.get('abi')})") +print(f" {len(d.get('files',{}))} files, {len(d.get('packages',[]))} packages: {', '.join(d.get('packages',[])[:6])}…") +PY +} + +case "${1:-}" in + --status) + say "installed at $BOT_DIR/app"; summary "$BOT_DIR/app" + if [[ -d $BOT_DIR/previous ]]; then say "previous"; summary "$BOT_DIR/previous"; fi + say "container"; docker ps --filter name=discord-bot --format ' {{.Names}} {{.Status}}' + exit 0 ;; + --rollback) + [[ -d $BOT_DIR/previous ]] || die "no previous bundle in $BOT_DIR/previous" + say "rolling back"; summary "$BOT_DIR/previous" + docker stop discord-bot >/dev/null + rm -rf "$BOT_DIR/.rollback" && mv "$BOT_DIR/app" "$BOT_DIR/.rollback" && mv "$BOT_DIR/previous" "$BOT_DIR/app" && mv "$BOT_DIR/.rollback" "$BOT_DIR/previous" + docker start discord-bot >/dev/null; sleep 8; docker logs --tail 12 discord-bot; exit 0 ;; + --latest) + [[ -n ${BOT_REPO_TOKEN:-} ]] || die "set BOT_REPO_TOKEN in .env (Gitea token with read access to $REPO)" + [[ -n $GIT_HOST ]] || die "GIT_HOST not found in .env" + TARBALL=$(mktemp /tmp/vanguard-bot-bundle.XXXXXX.tar.gz) + say "newest '$BRANCH' bundle from https://$GIT_HOST/$REPO" + URL=$(curl -fsS -H "Authorization: token $BOT_REPO_TOKEN" \ + "https://$GIT_HOST/api/v1/repos/$REPO/actions/artifacts?name=vanguard-bot-bundle" | + python3 -c " +import json,sys +items=json.load(sys.stdin).get('artifacts') or [] +ok=[a for a in items if not a.get('expired')] +if not ok: sys.exit('no unexpired vanguard-bot-bundle artifact — check the ci workflow ran') +print(sorted(ok,key=lambda a:a.get('created_at',''))[-1]['archive_download_url'])") + curl -fsSL -H "Authorization: token $BOT_REPO_TOKEN" -o "$TARBALL.zip" "$URL" + # Gitea serves artifacts as a zip holding the tar.gz + if python3 -c "import zipfile,sys; sys.exit(0 if zipfile.is_zipfile(sys.argv[1]) else 1)" "$TARBALL.zip"; then + python3 - "$TARBALL.zip" "$TARBALL" <<'PY' +import sys,zipfile +with zipfile.ZipFile(sys.argv[1]) as z: + name=[n for n in z.namelist() if n.endswith('.tar.gz')] or z.namelist() + open(sys.argv[2],'wb').write(z.read(name[0])) +PY + else mv "$TARBALL.zip" "$TARBALL"; fi + rm -f "$TARBALL.zip" ;; + "" | -h | --help) sed -n '2,20p' "$0"; exit 2 ;; + *) TARBALL="$1"; [[ -f $TARBALL ]] || die "no such file: $TARBALL" ;; +esac + +say "staging $TARBALL" +rm -rf "$BOT_DIR/.incoming"; mkdir -p "$BOT_DIR/.incoming" +tar -xzf "$TARBALL" -C "$BOT_DIR/.incoming" +for required in main.py app/main.py app/config.py website/static/home.html vendor; do + [[ -e "$BOT_DIR/.incoming/$required" ]] || die "bundle is missing $required — is this the CI artifact?" +done +summary "$BOT_DIR/.incoming" +python3 - "$BOT_DIR/.incoming" <<'PY' +import hashlib,json,pathlib,sys +root=pathlib.Path(sys.argv[1]); d=json.loads((root/'BUILD.json').read_text()) +bad=[name for name,digest in d.get('files',{}).items() + if name!='BUILD.json' and hashlib.sha256((root/name).read_bytes()).hexdigest()!=digest] +sys.exit(f" {len(bad)} file(s) do not match BUILD.json: {bad[:3]}" if bad else 0) +PY +echo " checksums match BUILD.json" + +say "swapping in and restarting discord-bot" +docker stop discord-bot >/dev/null 2>&1 || true +rm -rf "$BOT_DIR/previous" +if [[ -d $BOT_DIR/app ]]; then mv "$BOT_DIR/app" "$BOT_DIR/previous"; fi +mv "$BOT_DIR/.incoming" "$BOT_DIR/app" +chown -R 1000:1000 "$BOT_DIR/app" 2>/dev/null || sudo chown -R 1000:1000 "$BOT_DIR/app" +docker compose up -d discord-bot >/dev/null +sleep 10 + +say "result" +docker ps --filter name=discord-bot --format ' {{.Names}} {{.Status}}' +code=$(curl -s -o /dev/null -w '%{http_code}' -H 'X-Authentik-Username: deploy-check' -H 'X-Authentik-Groups: admins' http://127.0.0.1:8000/health || true) +echo " /health -> ${code:-no answer} (200 = serving; 401/403 = running but the identity headers were refused)" +docker logs --tail 15 discord-bot 2>&1 | sed 's/^/ /' +echo +echo " web UI: https://${BOT_HOST:-BOT_HOST-not-set}/ (Caddy + Authentik; needs the bot-users group)" +echo " rollback: ./deployBotBundle.sh --rollback" diff --git a/docker-compose.yml b/docker-compose.yml index 233b245..7dc9d56 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -214,6 +214,11 @@ services: DB_NAME: ${BOT_DB_NAME:-vanguardbot} DB_USER: ${BOT_DB_USER:-bot} DB_PASSWORD: ${BOT_DB_PASSWORD} + DB_BACKEND: mariadb + DATA_DIR: /data + WEB_ALLOWED_GROUPS: ${BOT_WEB_ALLOWED_GROUPS:-bot-users,admins} # the bot's own gate; Caddy already requires BOT_REQUIRE_GROUP + WEB_SERVE_UI: "true" # serve the bot's own pages at / on this port (same origin as /api) + PYTHONPATH: /app/vendor # the bundle's dependencies; the image installs none MEDIA_CLIPS: /media/clips WEB_HOST: 0.0.0.0 WEB_PORT: "8000" @@ -226,7 +231,7 @@ services: ports: - "127.0.0.1:8000:8000" # web UI; public only via Caddy + Authentik forward-auth at https://BOT_HOST/ volumes: - - ./bot/app:/app # the bot's source; edit and `docker compose restart discord-bot` + - ${BOT_APP_DIR:-/srv/bot/app}:/app # CI bundle installed by ./deployBotBundle.sh (code + vendor/) - /srv/media/clips:/media/clips # rw: for a clip-submissions feature (writes into clips//) - /srv/appdata/discord-bot:/data # persistent scratch (sqlite, caches) if the bot wants files extra_hosts: diff --git a/install.sh b/install.sh index 5d989af..af9e8e1 100755 --- a/install.sh +++ b/install.sh @@ -41,6 +41,7 @@ install -d -o 1000 -g 1000 /srv/media/{streams,clips,finals,trash,.markers,.sync install -d -o 1000 -g 1000 /srv/appdata/jellyfin/{config,cache} /srv/appdata/authentik/{media,templates,certs} install -d /srv/appdata/authentik/{db,redis} /srv/appdata/mariadb install -d -o 1000 -g 1000 /srv/appdata/discord-bot /srv/appdata/gitea /srv/appdata/act_runner +install -d -o 1000 -g 1000 /srv/bot /srv/bot/app # the Discord bot's deployed CI bundle (bind-mounted at /app) install -d -o "$OWNER" /srv/live # live portal static build (deployLive.sh) install -d -o 1000 -g 1000 /srv/media/streams/{techy,diceofire} /srv/media/clips/{techy,diceofire} for d in /srv/media/clips/*/; do