bunch of shit
This commit is contained in:
@@ -5,9 +5,27 @@ domain. Both are mostly `configure.py --full` plus a known list of places outsid
|
||||
Do it between streams: the main host is MediaMTX's WebRTC advertised host, so a main-host change needs a
|
||||
MediaMTX restart.
|
||||
|
||||
Roles: `main` (gateway page at `/`), `files` (copyparty), `clips`, `sync`, `live` (portal + `/dock/`, `/dock/clock/`,
|
||||
`/dump`, `/me`, `/time`), `auth`, `watch`, `git`, `bot`. Ours: `media.`, `files.`, `clips.`, `sync.`, `live.`, … under
|
||||
`dumpstervigilantes.rip`.
|
||||
Roles: `main` (gateway page at `/`, `/onboarding/`), `files` (copyparty), `clips`, `sync`, `live` (portal + Go Live,
|
||||
`/dock/`, `/dock/clock/`, `/dump`, `/me`, `/time`, `/whoami`), `auth`, `watch`, `git`, `bot`, `diag` (Kibana at `/`,
|
||||
`/streaming`, `/disk`, dumpd's `/streaming/api/*`), and optionally `apex` (the bare domain, a redirect to
|
||||
`APEX_REDIRECT`; it needs its own A record at the box or Caddy cannot get its certificate — as of 2026-11-06 the bare
|
||||
`dumpstervigilantes.rip` resolves elsewhere, so leave `APEX_HOST` empty or fix the record first). Ours: `media.`,
|
||||
`files.`, `clips.`, `sync.`, `live.`, `diag.`, … under `dumpstervigilantes.rip`. `ovh1.BASE` is ssh only.
|
||||
|
||||
## The one rule: apps before Authentik
|
||||
|
||||
Change the app, then Authentik, never the other way round. An app that already lives on its new name but whose
|
||||
Authentik provider still says the old one is simply unreachable until Authentik is updated; a provider pointed at
|
||||
a name the app does not answer on yet locks you out of the app, and if that app is the gate in front of the admin
|
||||
UI, out of everything. So in every procedure below Authentik comes after the servers, the containers, Jellyfin and
|
||||
Gitea — and the Discord developer portal after Authentik. The safety net at every step is akadmin's password
|
||||
card, `https://AUTH/if/flow/default-authentication-flow/?admin=1`, which bypasses the Discord forward.
|
||||
|
||||
Note that `configure.py --apply` does touch Authentik on its own: the login redirect stage's target (the Discord
|
||||
source's login URL on the auth host) and the brand's default application. Both are Authentik-internal, neither
|
||||
decides whether an app is reachable, and the admin card ignores the redirect stage — so they cannot lock you out.
|
||||
The things that can — outpost host, proxy provider external host and cookie domain, Jellyfin/Gitea redirect
|
||||
URIs — are never changed by any script here.
|
||||
|
||||
## Where hostnames live
|
||||
|
||||
@@ -15,16 +33,19 @@ Roles: `main` (gateway page at `/`), `files` (copyparty), `clips`, `sync`, `live
|
||||
| Place | Holds |
|
||||
|---|---|
|
||||
| `Caddyfile` | every site block — the source of truth `configure.py` reads back (`# vanguard:<role>` markers) |
|
||||
| `.env` | `MAIN_HOST`, `FILES_HOST`, `CLIPS_HOST`, `SYNC_HOST`, `DIAG_HOST`, `WATCH_HOST`, `GIT_HOST`, `BOT_HOST`, `LIVE_HOST`, `APEX_HOST`, `APEX_REDIRECT` |
|
||||
| `.env` | `MAIN_HOST`, `FILES_HOST`, `CLIPS_HOST`, `SYNC_HOST`, `DIAG_HOST`, `WATCH_HOST`, `GIT_HOST`, `BOT_HOST`, `LIVE_HOST`, `APEX_HOST`, `APEX_REDIRECT`; during a move `ALT_BASE` (every site also answers under it) and `REDIRECT_BASE` (old names 308 to the new) |
|
||||
| `mediamtx.yml` | `webrtcAdditionalHosts: [MAIN_HOST]` (needs a MediaMTX restart to take effect) |
|
||||
| `docker-compose.yml` → container env | Jellyfin `PublishedServerUrl` (WATCH), Gitea `DOMAIN/ROOT_URL/SSH_DOMAIN` (GIT), bot `WEB_PUBLIC_URL` (BOT), MediaMTX `LIVE_HOST` (text of the "is live" post), act_runner `GITEA_INSTANCE_URL` (GIT). `compose up -d` in the apply recreates the affected containers (seconds each) |
|
||||
| `docker-compose.yml` → container env | Jellyfin `PublishedServerUrl` (WATCH), Gitea `DOMAIN/ROOT_URL/SSH_DOMAIN` (GIT), bot `WEB_PUBLIC_URL` (BOT), MediaMTX `LIVE_HOST` (text of the "is live" post), act_runner `GITEA_INSTANCE_URL` (GIT), Kibana `server.publicBaseUrl` via `DIAG_HOST` (DIAG). `compose up -d` in the apply recreates the affected containers (seconds each). MediaMTX is on the host network and is deliberately **not** recreated by the apply |
|
||||
| Authentik, by `configure.py --apply` (`--ak-branding` alone) | the login **redirect stage** target `https://AUTH/source/oauth/login/<discord-slug>/` (members never see an Authentik page), the brand's default application (the proxy provider whose external host is MAIN), logo/favicon/background paths (relative, no host). Re-run after an **auth** or **main** change |
|
||||
| Kibana landing dashboard, by `configure.py --apply` (`KEY=… kibana-setup.py landing` alone) | the Vanguard tile's links: LIVE, FILES, WATCH, MAIN, GIT, BOT, SYNC from `.env`, AUTH derived as `auth.<domain of MAIN>`; `/streaming`, `/disk` relative. Re-run after any change |
|
||||
| Caddy TLS | certificates for new names are fetched automatically; old ones linger harmlessly |
|
||||
|
||||
### Pages in the repo that derive sibling hosts from the browser's location (label names hard-coded)
|
||||
| File | Assumes |
|
||||
|---|---|
|
||||
| `about/index.html` | `files.`, `clips.`, `sync.`, `live.`, `auth.`, `watch.`, `git.`, `bot.` + the parent of the current host |
|
||||
| `dock/index.html` | `auth.` (sign-out), ingest host prefilled from the current host (`live.`; any name of the box works for RTMP/SRT) |
|
||||
| `about/index.html` | `files.`, `clips.`, `sync.`, `live.`, `auth.`, `watch.`, `git.`, `bot.`, `diag.` + the parent of the current host (`sub('<label>')`); `/onboarding/` relative |
|
||||
| `dock/index.html` | `auth.` (sign-out), ingest host prefilled from the current host (`live.`; any name of the box works for RTMP/SRT); `/dump`, `/me/*`, `/time` relative |
|
||||
| `live/` portal, `streaming/index.html`, `streaming/disk.html`, `about/onboarding/` | nothing: every call is same-origin (`/webrtc/`, `/hls/`, `/me/*`, `/streaming/api/*`) |
|
||||
| `switchToClone.sh`, `deployLive.sh` | defaults for `git.dumpstervigilantes.rip` / `media.dumpstervigilantes.rip` (override by env/arg, or edit) |
|
||||
| `README.md`, `FRESH-INSTALL.md` | examples only |
|
||||
Renaming a *label* (e.g. `watch` → `video`) means editing the first two; changing only the domain does not.
|
||||
@@ -49,7 +70,9 @@ Renaming a *label* (e.g. `watch` → `video`) means editing the first two; chang
|
||||
| Streamers' OBS | dock URL (Custom Browser Docks), dock Settings → ingest host (re-save), clock browser source URL (dock → Timecode band → "Add band + clock source" again), any manual RTMP/SRT configs (copy from the dock) | live |
|
||||
| Jellyfin apps on phones/TVs | server address | watch |
|
||||
| Bookmarks, Discord pinned links | main (gateway), clips, files, watch, live | all |
|
||||
Not affected: Syncthing pairings (device IDs), MariaDB, stream tokens, the Gitea webhook (gateway IP), Authentik users/groups/policies.
|
||||
Not affected: Syncthing pairings (device IDs), MariaDB, stream tokens, the Gitea webhook (gateway IP), Authentik
|
||||
users/groups/policies, the alert pipeline (Discord webhook + loopback Elasticsearch), the deadman, the publish block
|
||||
list and the WebRTC session history (`.markers/`), MediaMTX's host-network ports.
|
||||
|
||||
## A. Renaming one subdomain (example: `watch.BASE` → `video.BASE`)
|
||||
|
||||
@@ -58,50 +81,80 @@ Not affected: Syncthing pairings (device IDs), MariaDB, stream tokens, the Gitea
|
||||
"Jellyfin hostname" prompt; `y`. Caddy reloads (new block, old gone), Jellyfin is recreated with the new
|
||||
published URL. `curl -sI https://video.BASE/ | head -1` → 200/302 once the certificate is issued.
|
||||
(Only the *main* host touches `mediamtx.yml`; other hosts need no MediaMTX restart.)
|
||||
3. Authentik: Applications → Providers → `Jellyfin` → Edit → redirect URI → the new host → Update.
|
||||
4. Jellyfin: Dashboard → General → Branding → disclaimer form action → new host → Save.
|
||||
Dashboard → Plugins → Vanguard Metadata → Public URL → new host → Save.
|
||||
3. Jellyfin first: Dashboard → General → Branding → disclaimer form action → new host → Save.
|
||||
Dashboard → Plugins → Vanguard Metadata → Public URL → new host → Save. (`https://video.BASE/` now loads and
|
||||
fails only at the SSO step: expected until step 4.)
|
||||
4. Authentik last: Applications → Providers → `Jellyfin` → Edit → redirect URI → the new host → Update.
|
||||
5. If the *label* changed (`watch`→`video`), edit `about/index.html` (`sub('watch')` → `sub('video')`) — commit, push,
|
||||
`sudo vanguard-deploy pull`. (`.env APEX_REDIRECT` too if the bare domain pointed at it: `./configure.py --full`.)
|
||||
6. Private window: `https://video.BASE/` → Discord → in. The gateway `https://HOST/` links to the new name.
|
||||
7. Remove the old DNS record. Tell members with Jellyfin apps to update the server address.
|
||||
|
||||
Per-host extras when it is a different host:
|
||||
- **auth**: Discord portal redirect; Authentik outpost `authentik_host` and the proxy provider's authentication URL;
|
||||
Jellyfin SSO plugin OID endpoint; Vanguard plugin Authentik URL; Gitea auth source discovery URL;
|
||||
`dock/index.html` + `about/index.html` if the label changed. Everyone re-logs in.
|
||||
- **auth**, in this order: Jellyfin SSO plugin OID endpoint and Vanguard plugin Authentik URL; Gitea auth source
|
||||
discovery URL; `dock/index.html` + `about/index.html` if the label changed; THEN Authentik's outpost
|
||||
`authentik_host` and the proxy provider's authentication URL (log in there with the admin card); THEN the
|
||||
Discord portal redirect. Everyone re-logs in.
|
||||
- **git**: Authentik Gitea provider redirect; `./registerGiteaRunner.sh`; server clone remote + `~/.ssh/config.vanguard`;
|
||||
dev remote; `switchToClone.sh` default.
|
||||
- **live**, **bot**: only the compose env (automatic) and `about/index.html` if the label changed.
|
||||
- **live**, **bot**: only the compose env (automatic) and `about/index.html` if the label changed. Streamers who use
|
||||
Go Live need nothing (same-origin); OBS users re-add the dock URL (the dock lives on `live`).
|
||||
- **diag**: compose env → Kibana's `publicBaseUrl` (automatic, Kibana recreated), the Kibana landing tile
|
||||
(automatic in the apply), `DIAG_REQUIRE_GROUP` in `.env` if the gate group should differ, `about/index.html`
|
||||
if the label changed. Alarm rules and the Discord alert bus carry no hostname.
|
||||
- **auth**, in addition to the list above: `configure.py --ak-branding` (the apply does it) so the Discord-forward
|
||||
redirect stage points at the new auth host — until then every login lands on the old name; the admin card
|
||||
moves to `https://NEW-AUTH/if/flow/default-authentication-flow/?admin=1` (the Kibana tile link follows).
|
||||
- **main (HOST)**: `mediamtx.yml` changes → `docker compose restart mediamtx` between streams; every streamer
|
||||
re-adds the dock URL, re-saves Settings (ingest host), and re-runs "Add band + clock source"; manual RTMP/SRT
|
||||
configs are re-copied from the dock; `deployLive.sh` default host.
|
||||
|
||||
## B. Changing the base domain (everything at once)
|
||||
|
||||
Plan a maintenance window: Caddy, Jellyfin, Gitea, the bot and MediaMTX all restart, and every member logs in again
|
||||
(the Authentik session cookie is bound to the base domain).
|
||||
No maintenance window: every site answers on **both** bases first (`--alias`), so the old names keep working for
|
||||
members while the apps and Authentik are moved one by one; the flip (`--base`) is the last server step and turns the
|
||||
old names into redirects. Order follows the rule above: servers, then the apps, then Authentik, then Discord, then
|
||||
the flip. `OLD` = the current base, `NEW` = the new one. Members' sessions survive until step 6; from step 6 on
|
||||
they log in once on the new names.
|
||||
|
||||
1. DNS on the new domain: `A HOST2 → IP`, CNAMEs `files`, `clips`, `sync`, `live`, `auth`, `watch`, `git`, `bot` → `HOST2` (optional bare domain A).
|
||||
Verify they resolve before continuing. Leave the old domain's records in place for now.
|
||||
2. Discord developer portal → OAuth2 → add redirect `https://auth.NEW/source/oauth/callback/discord/` (keep the old one until step 9).
|
||||
3. Server, between streams: `./configure.py --full` → new main hostname; the eight sub-hostnames default to the new
|
||||
base (Enter); bare domain; `y`. Then `docker compose restart mediamtx` (webrtcAdditionalHosts changed).
|
||||
All sites answer on the new names within a minute (certificates).
|
||||
4. Authentik (log in at the NEW auth host with akadmin — Discord login is broken until step 5 is done):
|
||||
- Outposts → Embedded Outpost → `authentik_host: https://auth.NEW`
|
||||
- Providers → `Vanguard sites` → authentication URL `https://auth.NEW`, **cookie domain `NEW`**
|
||||
- Providers → `Jellyfin` → redirect `https://watch.NEW/sso/OID/redirect/authentik`
|
||||
- Providers → `Gitea` → redirect `https://git.NEW/user/oauth2/authentik/callback`
|
||||
5. Private window: `https://auth.NEW/` → Discord login works (the redirect from step 2 is used).
|
||||
6. Jellyfin: SSO plugin OID endpoint `https://auth.NEW/application/o/jellyfin/`; Branding disclaimer form action
|
||||
`https://watch.NEW/sso/OID/start/authentik`; Vanguard plugin Public URL + Authentik URL. Test the SSO start URL.
|
||||
7. Gitea: auth source discovery URL `https://auth.NEW/application/o/gitea/.well-known/openid-configuration`;
|
||||
`./registerGiteaRunner.sh`; server: `cd ~/vanguard-stream && git remote set-url origin ssh://git@git.NEW:2222/techy/Media-Server-Setup.git`
|
||||
and edit `~/.ssh/config.vanguard` (Host line); dev box: `git remote set-url origin …`. Test `sudo vanguard-deploy pull`.
|
||||
8. Repo defaults: `switchToClone.sh` (REPO_SSH), `deployLive.sh` (HOST) — commit, push, pull.
|
||||
9. Streamers: re-add the dock at `https://live.NEW/dock/`, re-save Settings, "Add band + clock source" again.
|
||||
Members: `https://HOST2/`; Jellyfin apps → new server address. Remove the old Discord redirect and the old DNS.
|
||||
1. **DNS on NEW** (nothing else until this resolves): `A media.NEW` or CNAMEs for `media`, `files`, `clips`, `sync`,
|
||||
`live`, `diag`, `auth`, `watch`, `git`, `bot` → the box. Leave every OLD record in place. Verify:
|
||||
`for h in media files clips sync live diag auth watch git bot; do printf '%-6s ' $h; getent hosts $h.NEW | awk '{print $1}' | tail -1; done`
|
||||
— ten identical addresses. No bare-domain record unless it really points at the box.
|
||||
2. **Both bases live:** `./configure.py --alias NEW`. Only the Caddyfile changes (`media.OLD, media.NEW {` …) and
|
||||
Caddy reloads; certificates for the new names follow within a minute. Nothing else moves. Check:
|
||||
`curl -sI https://auth.NEW/ | head -1` → 302, and the old names unchanged. `sudo vghosts-check.py` now also
|
||||
probes every alias.
|
||||
3. **Discord developer portal** → OAuth2 → Redirects → **add** `https://auth.NEW/source/oauth/callback/discord/`
|
||||
beside the old one.
|
||||
4. **Jellyfin** (any name, SSO still works): SSO-Auth `authentik` provider OID endpoint
|
||||
`https://auth.NEW/application/o/jellyfin/` — works at once, Authentik answers on both names; Branding login
|
||||
disclaimer `https://watch.NEW/sso/OID/start/authentik`; Vanguard Metadata Public URL `https://watch.NEW`,
|
||||
Authentik URL `https://auth.NEW`.
|
||||
5. **Gitea**: Site administration → Authentication sources → `authentik` → discovery URL
|
||||
`https://auth.NEW/application/o/gitea/.well-known/openid-configuration` (works at once, same reason).
|
||||
Server clone remote and `~/.ssh/config.vanguard` → `git.NEW` (the host answers already). Dev box remote likewise.
|
||||
6. **Authentik, then the flip, back to back, between streams.** This is the only moment anything is unreachable,
|
||||
and it is seconds:
|
||||
`./configure.py --ak-hosts OLD NEW --dry-run` (read it) → `./configure.py --ak-hosts OLD NEW` → immediately
|
||||
`./configure.py --base NEW` → `docker compose restart mediamtx` → `./configure.py --ak-branding` →
|
||||
`./registerGiteaRunner.sh`.
|
||||
`--ak-hosts` moves the outpost host, the proxy provider (external host + cookie domain `NEW`) and the
|
||||
Jellyfin/Gitea redirect URIs: from this instant a login only sticks on the NEW names. `--base` makes NEW the
|
||||
primary everywhere (`.env`, `mediamtx.yml`, Jellyfin/Gitea/runner/bot/Kibana recreated, Kibana tile), drops
|
||||
the alias, and gives every OLD name a block that 308s to its NEW name, so bookmarks and pinned links keep
|
||||
working. Members on an old name get redirected and log in once.
|
||||
7. **Test, private window:** `https://auth.NEW/` → Discord → in. `https://media.OLD/` → 308 → `https://media.NEW/`
|
||||
→ gate → in. `https://watch.NEW/` SSO round trip. `https://git.NEW/user/login` authentik sign-in.
|
||||
`sudo vanguard-deploy pull`. `sudo vghosts-check.py` → 0 failed (it also checks every OLD name 308s).
|
||||
8. **Repo defaults:** `deployLive.sh` (HOST), `switchToClone.sh` (REPO_SSH) → NEW; commit, push, pull.
|
||||
9. **People:** OBS streamers re-add the dock at `https://live.NEW/dock/` (the old URL redirects, but OBS docks
|
||||
want the real one), re-save Settings, "Add band + clock source" again; Go Live users just reload. Jellyfin apps
|
||||
→ new server address. Pins can stay, they redirect; re-pin at leisure. Remove the OLD Discord redirect entry
|
||||
once everyone has logged in. The OLD DNS records stay as long as the NEW names are CNAMEs onto them.
|
||||
|
||||
Rollback before step 6: `./configure.py --alias ""` and revert the two Jellyfin/Gitea screens. Rollback after step 6:
|
||||
`./configure.py --ak-hosts NEW OLD` then `./configure.py --base OLD` (which makes the NEW names the redirects).
|
||||
|
||||
## C. Splitting the single main host into media / files / clips / sync (done 2026-11-01)
|
||||
|
||||
@@ -144,8 +197,40 @@ After that, `--apply` maintains it like any other host, because the marker is no
|
||||
4. Verify the marker landed: `grep -c "vanguard:<role>" Caddyfile` → 1. Then `curl -sI https://NEW/` → 302.
|
||||
5. Add it to `addCommunityMediaServer.sh` so a rebuild asks for it too.
|
||||
|
||||
## Checklist after any change
|
||||
dev$ for h in HOST files.BASE clips.BASE sync.BASE live.BASE diag.BASE auth.BASE watch.BASE git.BASE bot.BASE; do printf '%-26s ' $h; curl -s -o /dev/null -m 10 -w '%{http_code}\n' https://$h/; done
|
||||
## E. Validating a change: `sudo vghosts-check.py`
|
||||
|
||||
One command on the server checks every place in the tables above against the Caddyfile, the same source of
|
||||
truth `configure.py` reads back, and prints `ok` / `FAIL` / `warn` per item plus a `todo` list for the things no
|
||||
script can read (the Discord developer portal, streamers' OBS). Exit code 1 if anything failed.
|
||||
|
||||
server$ cd ~/vanguard-stream && sudo vghosts-check.py # everything
|
||||
dev$ ./bin/vghosts-check.py --repo # repo defaults, about-page labels, docs only
|
||||
server$ sudo vghosts-check.py --json # for a script
|
||||
|
||||
**The order is the order of the change, and it matters.** The sections run top-down because each one depends
|
||||
on the one above it, so a failure early on is usually the *cause* of the failures below it; fix the first
|
||||
failure and re-run rather than chasing the later ones:
|
||||
|
||||
| # | Section | Depends on | Why this position |
|
||||
|---|---|---|---|
|
||||
| 1 | DNS | — | nothing gets a certificate or a visitor until the name resolves to the box |
|
||||
| 2 | Caddyfile | 1 | the roles and names `--full`/`--hosts` wrote; Caddy fetches certificates from here |
|
||||
| 3 | `.env` | 2 | the same names as compose variables; a mismatch is a half-finished `--full` |
|
||||
| 4 | `mediamtx.yml` | 3 | `webrtcAdditionalHosts` = main host; only a MediaMTX restart (between streams) applies it |
|
||||
| 5 | containers | 3 | the env each container was *created* with: Jellyfin, Gitea, runner, bot, MediaMTX, Kibana |
|
||||
| 6 | HTTPS | 1, 2 | every name answers with a valid certificate; gated names bounce to the auth host |
|
||||
| 7 | Jellyfin | 5, 6 | SSO plugin OID endpoint, branding disclaimer, Vanguard plugin URLs; live SSO start |
|
||||
| 8 | Gitea | 5, 6 | `app.ini`, the auth source (live redirect), the runner's registration, the server clone's remote |
|
||||
| 9 | Authentik | 7, 8 | **last of the services**: outpost host, proxy provider external host + cookie domain, Jellyfin/Gitea redirect URIs — changing these before the apps is the one way to lock yourself out |
|
||||
| 10 | Discord portal | 9 | the OAuth2 redirect is printed exactly; Discord login is broken until it matches; akadmin's card still works |
|
||||
| 11 | repo | 2 | `deployLive.sh`/`switchToClone.sh` defaults, about-page labels = Caddy roles, no browser links to `ovh1` in the docs |
|
||||
| 12 | streamers | 2 | OBS dock URL, clock source, Jellyfin apps, Discord pins: listed, never checked |
|
||||
|
||||
What it knows that a human forgets: a container keeps the environment it was created with, so step 5 catches a
|
||||
`.env` that changed without `docker compose up -d <service>`; Authentik's cookie domain decides whether one
|
||||
login spans every host; and `ovh1.BASE` must resolve (ssh) but must *not* answer https.
|
||||
|
||||
By hand, for the round trips the script only probes:
|
||||
https://HOST/whoami → your identity (gate works on the new name)
|
||||
https://watch.BASE/sso/OID/start/authentik → Jellyfin SSO round trip
|
||||
https://git.BASE/user/login → "Sign in with authentik" round trip
|
||||
|
||||
Reference in New Issue
Block a user