Bot hosting: releases are self-contained CI bundles, and the bot serves its own web UI
The image no longer installs the bot's dependencies: its CI builds one tar.gz holding the app plus a
vendor/ directory of prebuilt wheels, and PYTHONPATH=/app/vendor makes the code import them. A new
dependency therefore needs no image rebuild and the server contacts no package index.
- deployBotBundle.sh: install / --latest / --status / --rollback, verifies every file against BUILD.json
and keeps the previous release beside the live one.
- compose: mount ${BOT_APP_DIR:-/srv/bot/app} at /app, set DB_BACKEND, DATA_DIR, WEB_ALLOWED_GROUPS,
WEB_SERVE_UI and PYTHONPATH. bot/app placeholder untracked; install.sh creates /srv/bot/app.
- The placeholder requirements would have crashed the bot on import (no sqlglot), which is why the
image now ships none of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,2 +1,4 @@
|
||||
app/__pycache__/
|
||||
app/*.pyc
|
||||
# only entrypoint.sh is COPYed; keep everything else out of the build context
|
||||
app/
|
||||
AGENT.md
|
||||
README.md
|
||||
|
||||
46
bot/AGENT.md
46
bot/AGENT.md
@@ -15,23 +15,43 @@ What you are given:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Runtime** | `python:3.12-slim` + ffmpeg + curl. Dependencies from `app/requirements.txt`, installed at image build (`docker compose build discord-bot`). |
|
||||
| **Your code** | bind-mounted at `/app` from `bot/app/`. Not baked into the image. Edit, then `docker compose restart discord-bot`. |
|
||||
| **Entry point** | the container runs `python -u /app/main.py` and restarts it if it exits. With no `main.py` it idles. |
|
||||
| **Runtime** | `python:3.12-slim` + ffmpeg + curl, and nothing else. **Your dependencies ship with your code** (see §1a), imported from `/app/vendor` via `PYTHONPATH`. Adding one needs no image rebuild. |
|
||||
| **Your code** | a release bundle from your CI, installed at `/srv/bot/app` and bind-mounted at `/app` (§1a). Treat `/app` as read-only; write to `/data`. |
|
||||
| **Entry point** | the container runs `python -u /app/main.py` and restarts it if it exits. With no bundle installed it idles and logs a hint. |
|
||||
| **User** | uid 1000 |
|
||||
| **Database** | MariaDB, database `vanguardbot`, user `bot`, reachable at `DB_HOST:DB_PORT`. Yours alone. Other databases on the same server belong to other services; never touch them. |
|
||||
| **Persistent disk** | `/data`, yours |
|
||||
| **Web UI** | bind `WEB_HOST:WEB_PORT` (0.0.0.0:8000). The proxy publishes it as `https://bot.<domain>/` behind login (§3). |
|
||||
| **Web UI** | bind `WEB_HOST:WEB_PORT` (0.0.0.0:8000) — one port for your API *and* your pages. The proxy publishes it as `https://bot.<domain>/` behind login (§3). |
|
||||
| **Discord** | `DISCORD_BOT_TOKEN`, `DISCORD_GUILD_ID`. The bot user belongs to the community's existing Discord application. |
|
||||
| **Clock** | `TZ=America/New_York` |
|
||||
|
||||
### Environment variables
|
||||
`DISCORD_BOT_TOKEN`, `DISCORD_GUILD_ID`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD`,
|
||||
`WEB_HOST`, `WEB_PORT`, `WEB_PUBLIC_URL`, `AUTH_HEADER_USER`, `AUTH_HEADER_GROUPS`, `AUTH_HEADER_EMAIL`,
|
||||
`MEDIA_CLIPS`, `TZ`. Secrets arrive only through env. Never write them to the repo, `/data`, or logs.
|
||||
`DISCORD_BOT_TOKEN`, `DISCORD_GUILD_ID`, `DB_BACKEND=mariadb`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`,
|
||||
`DB_PASSWORD`, `DATA_DIR=/data`, `WEB_HOST`, `WEB_PORT`, `WEB_PUBLIC_URL`, `WEB_SERVE_UI=true`,
|
||||
`WEB_ALLOWED_GROUPS=bot-users,admins`, `AUTH_HEADER_USER`, `AUTH_HEADER_GROUPS`, `AUTH_HEADER_EMAIL`,
|
||||
`MEDIA_CLIPS`, `PYTHONPATH=/app/vendor`, `TZ`. Secrets arrive only through env. Never write them to the repo,
|
||||
`/data`, or logs.
|
||||
|
||||
SQLAlchemy URL: `mysql+pymysql://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}` (async: `mysql+aiomysql`).
|
||||
|
||||
## 1a. Releases: one bundle, dependencies included
|
||||
|
||||
Your CI (`.gitea/workflows/ci.yml`, job `bundle`) runs after the tests and produces the artifact
|
||||
`vanguard-bot-bundle` — a single tar.gz whose contents **are** `/app`:
|
||||
|
||||
main.py app/ website/ version.json BUILD.json vendor/
|
||||
|
||||
`vendor/` is `pip install --only-binary=:all: --target vendor -r requirements.txt`, built on `python:3.12`
|
||||
so the wheels match the runtime image's ABI. `BUILD.json` records the version, commit, build time, the
|
||||
package list and a sha256 of every file. The admin installs it with `./deployBotBundle.sh <bundle.tar.gz>`,
|
||||
which checks the bundle against `BUILD.json`, keeps the previous release for `--rollback`, and restarts the
|
||||
container. Rules that follow from this:
|
||||
|
||||
- Keep `requirements.txt` resolvable with **prebuilt wheels only**; a dependency that must compile will fail
|
||||
the build. Ask the admin if you truly need one.
|
||||
- Don't import anything that is not in `requirements.txt` or the standard library.
|
||||
- Nothing is installed on the server at deploy time, and the server reaches no package index.
|
||||
|
||||
## 2. Process model
|
||||
|
||||
- One process, one asyncio loop: the Discord gateway client and the web server run together
|
||||
@@ -52,6 +72,10 @@ Every request to `https://bot.<domain>/` passes through Caddy, which sends the u
|
||||
| `X-Authentik-Groups` | group names, separated by `\|` or `,` — the community uses `admins`, `editors`, `streamers`; others may be added |
|
||||
| `X-Authentik-Email`, `X-Authentik-Name`, `X-Authentik-Uid` | extras |
|
||||
|
||||
`WEB_SERVE_UI=true` is set on the server, so this process serves your HTML pages at `/` and their assets
|
||||
at `/assets/` as well as `/api/…` — same origin, one port, no second service and no proxy of your own.
|
||||
Locally the separate website launcher still works; it is off unless `WEB_SERVE_UI` says otherwise.
|
||||
|
||||
Rules:
|
||||
1. **Implement no login page, no sessions, no passwords.** Identity is the headers, per request.
|
||||
2. Port 8000 is loopback-only on the host, so the headers can be trusted. Still fail closed: a request
|
||||
@@ -99,10 +123,12 @@ message content; Message Content is a privileged intent, enable it only if you n
|
||||
|
||||
## 6. Working with the admin
|
||||
|
||||
- **Source**: push to the community Gitea, `ssh://git@git.<domain>:2222/vanguard/discord-bot.git`.
|
||||
The admin deploys by pulling into `bot/app` and restarting. CI on the `python` runner label runs your tests.
|
||||
- **Source**: push to the community Gitea. CI runs your tests on the `python` runner label, then builds the
|
||||
release bundle (§1a); the admin installs it with `deployBotBundle.sh`. Pushes to `main` can install it
|
||||
automatically once the repository variable `BOT_AUTO_DEPLOY` is set to `1`; it is unset by default.
|
||||
- **Need infra?** Open a Gitea issue titled `infra: …` stating the exact change: a new env var and why,
|
||||
a mount, a Caddy path that should skip login (e.g. a public webhook endpoint), a new database.
|
||||
Don't work around the proxy or the firewall.
|
||||
- **Dependencies changed?** Say so in the PR; the image must be rebuilt.
|
||||
- **Dependencies changed?** Just push: they travel inside the bundle. Only a new *system* package (beyond
|
||||
ffmpeg and curl) needs the admin to rebuild the image.
|
||||
- Keep `README.md` updated with every command, env var, and table you add.
|
||||
|
||||
@@ -1,12 +1,11 @@
|
||||
# Python runtime for the Discord bot. Source is bind-mounted at /app from ./bot/app (not baked in),
|
||||
# so the dev edits code and restarts the container. Dependencies come from app/requirements.txt and
|
||||
# are installed at BUILD time: after changing it run docker compose build discord-bot && docker compose up -d discord-bot
|
||||
# Runtime for the Discord bot. The bot's CI builds a self-contained bundle (code + vendor/ dependencies)
|
||||
# and deployBotBundle.sh installs it at BOT_APP_DIR, bind-mounted here at /app. Dependencies therefore live
|
||||
# WITH the code, not in this image: a new dependency needs no rebuild and no package index on the server.
|
||||
# Only the interpreter and the tools AGENT.md promises are baked in.
|
||||
FROM python:3.12-slim
|
||||
ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1
|
||||
ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1 PYTHONDONTWRITEBYTECODE=1
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg ca-certificates curl \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY app/requirements.txt /tmp/requirements.txt
|
||||
RUN pip install -r /tmp/requirements.txt
|
||||
COPY entrypoint.sh /entrypoint.sh
|
||||
RUN chmod +x /entrypoint.sh
|
||||
WORKDIR /app
|
||||
|
||||
@@ -2,29 +2,42 @@
|
||||
|
||||
Start with **AGENT.md** in this folder: the stack, the rules, the layout, the web-UI auth contract.
|
||||
|
||||
Your code goes in this folder (`bot/app/`). The container runs `python -u /app/main.py` and restarts
|
||||
it if it exits. No `main.py` yet → the container idles and logs a hint.
|
||||
## How code reaches the server
|
||||
|
||||
Your repo's Gitea Actions workflow builds a **release bundle**: one tar.gz holding `main.py`, `app/`,
|
||||
`website/`, `version.json`, `BUILD.json` and `vendor/` (your dependencies as prebuilt wheels). The admin
|
||||
installs it with `./deployBotBundle.sh <bundle.tar.gz>` from the stack root; it lands in `/srv/bot/app`,
|
||||
which is bind-mounted into the container at `/app`, and the previous release is kept for `--rollback`.
|
||||
|
||||
The image contains **no** bot dependencies: the container imports yours from `/app/vendor`
|
||||
(`PYTHONPATH=/app/vendor`). Adding a dependency means editing `app/requirements.txt` and pushing — no image
|
||||
rebuild, no package index on the server. The image only changes if you need a new system package (ffmpeg and
|
||||
curl are already there); ask the admin for that.
|
||||
|
||||
## Day to day (on the server, in ~/vanguard-stream)
|
||||
docker compose logs -f discord-bot # live log
|
||||
docker compose restart discord-bot # after editing code
|
||||
docker compose build discord-bot && docker compose up -d discord-bot # after editing requirements.txt
|
||||
docker compose exec discord-bot python # a REPL inside the container
|
||||
|
||||
Edit files directly in `~/vanguard-stream/bot/app/` (they are bind-mounted) or scp them up.
|
||||
./deployBotBundle.sh vanguard-bot-bundle.tar.gz # install a release
|
||||
./deployBotBundle.sh --status # what is installed; container state
|
||||
./deployBotBundle.sh --rollback # previous release
|
||||
docker compose logs -f discord-bot # live log
|
||||
docker compose restart discord-bot # restart without changing code
|
||||
docker compose exec discord-bot python # a REPL inside the container (your vendor/ is on sys.path)
|
||||
|
||||
## Environment the container gives you
|
||||
| var | value |
|
||||
|-----|-------|
|
||||
| `DISCORD_BOT_TOKEN` | from `.env` (bundle root) — the Bot added to the existing Authentik Discord application |
|
||||
| `DISCORD_BOT_TOKEN` | from `.env` (stack root) — the Bot added to the existing Authentik Discord application |
|
||||
| `DISCORD_GUILD_ID` | the server |
|
||||
| `DB_BACKEND` | `mariadb` |
|
||||
| `DB_HOST` / `DB_PORT` | `host.docker.internal` / `3306` (MariaDB runs on the host network) |
|
||||
| `DB_NAME` / `DB_USER` / `DB_PASSWORD` | your own database + user (`vanguardbot` / `bot` / generated) |
|
||||
| `DATA_DIR` | `/data` — persistent, owned by you |
|
||||
| `MEDIA_CLIPS` | `/media/clips` — rw mount of `/srv/media/clips` (folder per user, for clip submissions) |
|
||||
| `/data` | persistent scratch dir owned by the bot |
|
||||
| `WEB_HOST` / `WEB_PORT` | `0.0.0.0` / `8000` — your web UI; public as `https://bot.<domain>/` via Caddy + Authentik |
|
||||
| `WEB_HOST` / `WEB_PORT` | `0.0.0.0` / `8000` — published as `https://bot.<domain>/` via Caddy + Authentik |
|
||||
| `WEB_PUBLIC_URL` | `https://bot.<domain>` for absolute links |
|
||||
| `WEB_SERVE_UI` | `true` — serve your pages at `/` on the same port as `/api` |
|
||||
| `WEB_ALLOWED_GROUPS` | `bot-users,admins` — the groups your own code must accept |
|
||||
| `AUTH_HEADER_USER` / `_GROUPS` / `_EMAIL` | `X-Authentik-Username` / `X-Authentik-Groups` / `X-Authentik-Email` — identity injected by Caddy, trust these, implement no login |
|
||||
| `PYTHONPATH` | `/app/vendor` |
|
||||
| `TZ` | America/New_York |
|
||||
|
||||
SQLAlchemy URL: `mysql+pymysql://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}`
|
||||
@@ -37,7 +50,8 @@ rate limited at the firewall; if yours get dropped, ask the admin to add your IP
|
||||
|
||||
## Conventions
|
||||
- Run as uid 1000 (files you write under `/media/clips` are owned like everything else there).
|
||||
- `/app` holds a released bundle; treat it as read-only and write to `/data` instead.
|
||||
- If you write clips: `/media/clips/<discord-username>/<name>.mp4` plus `<name>.json` with
|
||||
`{"timestamp": "<UTC ISO-8601>", "source": "medal", "source_url": "...", "uploader": "...", "title": "..."}`
|
||||
— the Jellyfin plugin reads that sidecar for the exact recording time.
|
||||
- Log to stdout; the container captures it.
|
||||
- Log to stdout; the container captures it. One event per line, `key=value`, keeps it searchable (`LOGGING.md`).
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
# Baseline for a discord.py bot with MariaDB. Add what you need; then:
|
||||
# docker compose build discord-bot && docker compose up -d discord-bot
|
||||
discord.py>=2.4
|
||||
aiomysql>=0.2
|
||||
PyMySQL>=1.1
|
||||
SQLAlchemy>=2.0
|
||||
python-dotenv>=1.0
|
||||
yt-dlp
|
||||
@@ -1,7 +1,13 @@
|
||||
#!/bin/sh
|
||||
# Run the bot if there is one; otherwise idle so the container stays up until code arrives.
|
||||
# /app is a bundle built by the bot's CI: main.py + app/ + website/ + vendor/ (its dependencies).
|
||||
# PYTHONPATH=/app/vendor is set by compose, so nothing is installed at run time.
|
||||
if [ -f /app/main.py ]; then
|
||||
if [ ! -d /app/vendor ]; then
|
||||
echo "discord-bot: /app has code but no vendor/ — this image installs no bot dependencies."
|
||||
echo " Install a CI bundle: ./deployBotBundle.sh <vanguard-bot-bundle.tar.gz>"
|
||||
fi
|
||||
exec python -u /app/main.py
|
||||
fi
|
||||
echo "discord-bot: no /app/main.py yet — drop the bot code into ./bot/app and 'docker compose restart discord-bot'"
|
||||
echo "discord-bot: /app is empty — install a bundle from the bot's CI: ./deployBotBundle.sh <bundle.tar.gz>"
|
||||
echo " (artifact 'vanguard-bot-bundle' on the VanguardBot repo's ci workflow)"
|
||||
exec sleep infinity
|
||||
|
||||
Reference in New Issue
Block a user