Vanguard media stack: ingest, copyparty, Syncthing, normalize sweep, Authentik, Jellyfin + Vanguard Metadata plugin, MariaDB, Gitea, bot runtime
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2
bot/.dockerignore
Normal file
2
bot/.dockerignore
Normal file
@@ -0,0 +1,2 @@
|
||||
app/__pycache__/
|
||||
app/*.pyc
|
||||
108
bot/AGENT.md
Normal file
108
bot/AGENT.md
Normal file
@@ -0,0 +1,108 @@
|
||||
# Vanguard Discord bot — brief for the agent building it
|
||||
|
||||
You are building the Discord bot for Vanguard. What the bot *does* is up to the community and you.
|
||||
This file only covers where it runs, what it is given, and the few rules that keep it from breaking
|
||||
the box it shares. Nothing here obliges you to integrate with the other services on that box.
|
||||
|
||||
## 1. Hosting
|
||||
|
||||
One Debian server runs a set of Docker Compose services from `~/vanguard-stream/`. You own exactly
|
||||
one of them: the `discord-bot` container. You do not own the compose file, the reverse proxy (Caddy),
|
||||
the firewall, the identity provider (Authentik), or the database server config. If you need any of
|
||||
those changed, ask the admin with the exact change (see §6).
|
||||
|
||||
What you are given:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Runtime** | `python:3.12-slim` + ffmpeg + curl. Dependencies from `app/requirements.txt`, installed at image build (`docker compose build discord-bot`). |
|
||||
| **Your code** | bind-mounted at `/app` from `bot/app/`. Not baked into the image. Edit, then `docker compose restart discord-bot`. |
|
||||
| **Entry point** | the container runs `python -u /app/main.py` and restarts it if it exits. With no `main.py` it idles. |
|
||||
| **User** | uid 1000 |
|
||||
| **Database** | MariaDB, database `vanguardbot`, user `bot`, reachable at `DB_HOST:DB_PORT`. Yours alone. Other databases on the same server belong to other services; never touch them. |
|
||||
| **Persistent disk** | `/data`, yours |
|
||||
| **Web UI** | bind `WEB_HOST:WEB_PORT` (0.0.0.0:8000). The proxy publishes it as `https://bot.<domain>/` behind login (§3). |
|
||||
| **Discord** | `DISCORD_BOT_TOKEN`, `DISCORD_GUILD_ID`. The bot user belongs to the community's existing Discord application. |
|
||||
| **Clock** | `TZ=America/New_York` |
|
||||
|
||||
### Environment variables
|
||||
`DISCORD_BOT_TOKEN`, `DISCORD_GUILD_ID`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD`,
|
||||
`WEB_HOST`, `WEB_PORT`, `WEB_PUBLIC_URL`, `AUTH_HEADER_USER`, `AUTH_HEADER_GROUPS`, `AUTH_HEADER_EMAIL`,
|
||||
`MEDIA_CLIPS`, `TZ`. Secrets arrive only through env. Never write them to the repo, `/data`, or logs.
|
||||
|
||||
SQLAlchemy URL: `mysql+pymysql://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}` (async: `mysql+aiomysql`).
|
||||
|
||||
## 2. Process model
|
||||
|
||||
- One process, one asyncio loop: the Discord gateway client and the web server run together
|
||||
(`discord.py` plus `aiohttp` or FastAPI via uvicorn's programmatic API). Do not spawn a second container.
|
||||
- Log to stdout. The admin reads `docker compose logs -f discord-bot`.
|
||||
- On fatal misconfiguration (no token, DB unreachable after retries) log the reason and exit non-zero;
|
||||
the restart policy retries.
|
||||
- Handle SIGTERM: close the gateway and the web server cleanly.
|
||||
|
||||
## 3. Web UI: login is handled outside your code
|
||||
|
||||
Every request to `https://bot.<domain>/` passes through Caddy, which sends the user to Authentik
|
||||
(Discord login) and, once authenticated, forwards the request to you with identity headers:
|
||||
|
||||
| header | content |
|
||||
|---|---|
|
||||
| `X-Authentik-Username` | the user's account name |
|
||||
| `X-Authentik-Groups` | group names, separated by `\|` or `,` — the community uses `admins`, `editors`, `streamers`; others may be added |
|
||||
| `X-Authentik-Email`, `X-Authentik-Name`, `X-Authentik-Uid` | extras |
|
||||
|
||||
Rules:
|
||||
1. **Implement no login page, no sessions, no passwords.** Identity is the headers, per request.
|
||||
2. Port 8000 is loopback-only on the host, so the headers can be trusted. Still fail closed: a request
|
||||
without the username header gets 401.
|
||||
3. Authorize by group, read from the header each time. Don't cache roles in your DB.
|
||||
4. Serve under `/`; the proxy does no path rewriting for this host.
|
||||
5. Build absolute links from `WEB_PUBLIC_URL`.
|
||||
6. The Authentik username may differ from the Discord username. Use the header value as the user key;
|
||||
if you need the Discord snowflake, resolve it through your gateway client and cache it in your DB.
|
||||
|
||||
Until the admin creates the matching Authentik application, the hostname returns an error, never an
|
||||
open page. Test against `http://127.0.0.1:8000/` on the server with the headers set by hand.
|
||||
|
||||
## 4. Repository layout (recommended, not enforced)
|
||||
|
||||
```
|
||||
bot/
|
||||
AGENT.md, README.md this brief; the ops cheat-sheet (keep README current)
|
||||
Dockerfile, entrypoint.sh admin-owned; ask before changing
|
||||
app/
|
||||
main.py config → migrations → gateway + web in one loop → clean shutdown
|
||||
requirements.txt pin versions once past the prototype
|
||||
config.py the only module that reads os.environ
|
||||
bot/ discord.py client, cogs/ (one feature per cog), permission checks
|
||||
web/ routes/, templates/, static/, auth.py (header → user/groups)
|
||||
db/ SQLAlchemy models, session, alembic/ migrations (run at startup)
|
||||
services/ the actual logic, shared by cogs and web; no Discord or HTTP types
|
||||
tests/ pytest; must pass offline with no token
|
||||
```
|
||||
Slash commands scoped to the guild so they appear instantly. Prefer slash commands over reading
|
||||
message content; Message Content is a privileged intent, enable it only if you need it.
|
||||
|
||||
## 5. Shared ground on the box
|
||||
|
||||
- `/media/clips` is mounted read-write. It is the community's clip storage, organized as one folder per
|
||||
user. You may **add** files there; never modify, move or delete files you did not create (parts of it
|
||||
are mirrored from members' machines and will fight back). If you write a video, write
|
||||
`<name>.part` then rename to `<name>.mp4`, and put a `<name>.json` beside it with at least
|
||||
`{"timestamp": "<UTC ISO-8601 creation time>", "source": "...", "uploader": "<discord username>"}`.
|
||||
Other services on the box index that folder and read the sidecar. If you never touch this mount,
|
||||
none of this applies.
|
||||
- The same bot user is used by another service for REST calls (posting notices, reading scheduled
|
||||
events). Sharing a token is fine. Don't rename the bot or change its avatar; pick your own channels.
|
||||
- Don't bind other ports, don't start other containers, don't write outside `/app`, `/data`, `/media/clips`.
|
||||
|
||||
## 6. Working with the admin
|
||||
|
||||
- **Source**: push to the community Gitea, `ssh://git@git.<domain>:2222/vanguard/discord-bot.git`.
|
||||
The admin deploys by pulling into `bot/app` and restarting. CI on the `python` runner label runs your tests.
|
||||
- **Need infra?** Open a Gitea issue titled `infra: …` stating the exact change: a new env var and why,
|
||||
a mount, a Caddy path that should skip login (e.g. a public webhook endpoint), a new database.
|
||||
Don't work around the proxy or the firewall.
|
||||
- **Dependencies changed?** Say so in the PR; the image must be rebuilt.
|
||||
- Keep `README.md` updated with every command, env var, and table you add.
|
||||
13
bot/Dockerfile
Normal file
13
bot/Dockerfile
Normal file
@@ -0,0 +1,13 @@
|
||||
# Python runtime for the Discord bot. Source is bind-mounted at /app from ./bot/app (not baked in),
|
||||
# so the dev edits code and restarts the container. Dependencies come from app/requirements.txt and
|
||||
# are installed at BUILD time: after changing it run docker compose build discord-bot && docker compose up -d discord-bot
|
||||
FROM python:3.12-slim
|
||||
ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg ca-certificates curl \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY app/requirements.txt /tmp/requirements.txt
|
||||
RUN pip install -r /tmp/requirements.txt
|
||||
COPY entrypoint.sh /entrypoint.sh
|
||||
RUN chmod +x /entrypoint.sh
|
||||
WORKDIR /app
|
||||
ENTRYPOINT ["/entrypoint.sh"]
|
||||
43
bot/README.md
Normal file
43
bot/README.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# discord-bot — for the bot dev
|
||||
|
||||
Start with **AGENT.md** in this folder: the stack, the rules, the layout, the web-UI auth contract.
|
||||
|
||||
Your code goes in this folder (`bot/app/`). The container runs `python -u /app/main.py` and restarts
|
||||
it if it exits. No `main.py` yet → the container idles and logs a hint.
|
||||
|
||||
## Day to day (on the server, in ~/vanguard-stream)
|
||||
docker compose logs -f discord-bot # live log
|
||||
docker compose restart discord-bot # after editing code
|
||||
docker compose build discord-bot && docker compose up -d discord-bot # after editing requirements.txt
|
||||
docker compose exec discord-bot python # a REPL inside the container
|
||||
|
||||
Edit files directly in `~/vanguard-stream/bot/app/` (they are bind-mounted) or scp them up.
|
||||
|
||||
## Environment the container gives you
|
||||
| var | value |
|
||||
|-----|-------|
|
||||
| `DISCORD_BOT_TOKEN` | from `.env` (bundle root) — the Bot added to the existing Authentik Discord application |
|
||||
| `DISCORD_GUILD_ID` | the server |
|
||||
| `DB_HOST` / `DB_PORT` | `host.docker.internal` / `3306` (MariaDB runs on the host network) |
|
||||
| `DB_NAME` / `DB_USER` / `DB_PASSWORD` | your own database + user (`vanguardbot` / `bot` / generated) |
|
||||
| `MEDIA_CLIPS` | `/media/clips` — rw mount of `/srv/media/clips` (folder per user, for clip submissions) |
|
||||
| `/data` | persistent scratch dir owned by the bot |
|
||||
| `WEB_HOST` / `WEB_PORT` | `0.0.0.0` / `8000` — your web UI; public as `https://bot.<domain>/` via Caddy + Authentik |
|
||||
| `WEB_PUBLIC_URL` | `https://bot.<domain>` for absolute links |
|
||||
| `AUTH_HEADER_USER` / `_GROUPS` / `_EMAIL` | `X-Authentik-Username` / `X-Authentik-Groups` / `X-Authentik-Email` — identity injected by Caddy, trust these, implement no login |
|
||||
| `TZ` | America/New_York |
|
||||
|
||||
SQLAlchemy URL: `mysql+pymysql://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}`
|
||||
async: `mysql+aiomysql://...`
|
||||
|
||||
## Database from your own machine
|
||||
MariaDB is reachable directly: `ovh1.dumpstervigilantes.rip:3306`, user `bot`, database `vanguardbot`,
|
||||
password from the admin (`./configure.py --show` on the server). Root is local-only. New connections are
|
||||
rate limited at the firewall; if yours get dropped, ask the admin to add your IP to `db_allow`.
|
||||
|
||||
## Conventions
|
||||
- Run as uid 1000 (files you write under `/media/clips` are owned like everything else there).
|
||||
- If you write clips: `/media/clips/<discord-username>/<name>.mp4` plus `<name>.json` with
|
||||
`{"timestamp": "<UTC ISO-8601>", "source": "medal", "source_url": "...", "uploader": "...", "title": "..."}`
|
||||
— the Jellyfin plugin reads that sidecar for the exact recording time.
|
||||
- Log to stdout; the container captures it.
|
||||
8
bot/app/requirements.txt
Normal file
8
bot/app/requirements.txt
Normal file
@@ -0,0 +1,8 @@
|
||||
# Baseline for a discord.py bot with MariaDB. Add what you need; then:
|
||||
# docker compose build discord-bot && docker compose up -d discord-bot
|
||||
discord.py>=2.4
|
||||
aiomysql>=0.2
|
||||
PyMySQL>=1.1
|
||||
SQLAlchemy>=2.0
|
||||
python-dotenv>=1.0
|
||||
yt-dlp
|
||||
7
bot/entrypoint.sh
Normal file
7
bot/entrypoint.sh
Normal file
@@ -0,0 +1,7 @@
|
||||
#!/bin/sh
|
||||
# Run the bot if there is one; otherwise idle so the container stays up until code arrives.
|
||||
if [ -f /app/main.py ]; then
|
||||
exec python -u /app/main.py
|
||||
fi
|
||||
echo "discord-bot: no /app/main.py yet — drop the bot code into ./bot/app and 'docker compose restart discord-bot'"
|
||||
exec sleep infinity
|
||||
Reference in New Issue
Block a user