bunch of shit

HOSTNAME-CHANGE.md

@@ -5,9 +5,27 @@ domain. Both are mostly `configure.py --full` plus a known list of places outsid
 Do it between streams: the main host is MediaMTX's WebRTC advertised host, so a main-host change needs a
 MediaMTX restart.
 
-Roles: `main` (gateway page at `/`), `files` (copyparty), `clips`, `sync`, `live` (portal + `/dock/`, `/dock/clock/`,
+Roles: `main` (gateway page at `/`, `/onboarding/`), `files` (copyparty), `clips`, `sync`, `live` (portal + Go Live,
-`/dump`, `/me`, `/time`), `auth`, `watch`, `git`, `bot`. Ours: `media.`, `files.`, `clips.`, `sync.`, `live.`, … under
+`/dock/`, `/dock/clock/`, `/dump`, `/me`, `/time`, `/whoami`), `auth`, `watch`, `git`, `bot`, `diag` (Kibana at `/`,
-`dumpstervigilantes.rip`.
+`/streaming`, `/disk`, dumpd's `/streaming/api/*`), and optionally `apex` (the bare domain, a redirect to
+`APEX_REDIRECT`; it needs its own A record at the box or Caddy cannot get its certificate — as of 2026-11-06 the bare
+`dumpstervigilantes.rip` resolves elsewhere, so leave `APEX_HOST` empty or fix the record first). Ours: `media.`,
+`files.`, `clips.`, `sync.`, `live.`, `diag.`, … under `dumpstervigilantes.rip`. `ovh1.BASE` is ssh only.
+
+## The one rule: apps before Authentik
+
+Change the app, then Authentik, never the other way round. An app that already lives on its new name but whose
+Authentik provider still says the old one is simply unreachable until Authentik is updated; a provider pointed at
+a name the app does not answer on yet locks you out of the app, and if that app is the gate in front of the admin
+UI, out of everything. So in every procedure below Authentik comes after the servers, the containers, Jellyfin and
+Gitea — and the Discord developer portal after Authentik. The safety net at every step is akadmin's password
+card, `https://AUTH/if/flow/default-authentication-flow/?admin=1`, which bypasses the Discord forward.
+
+Note that `configure.py --apply` does touch Authentik on its own: the login redirect stage's target (the Discord
+source's login URL on the auth host) and the brand's default application. Both are Authentik-internal, neither
+decides whether an app is reachable, and the admin card ignores the redirect stage — so they cannot lock you out.
+The things that can — outpost host, proxy provider external host and cookie domain, Jellyfin/Gitea redirect
+URIs — are never changed by any script here.
 
 ## Where hostnames live
 
@@ -15,16 +33,19 @@ Roles: `main` (gateway page at `/`), `files` (copyparty), `clips`, `sync`, `live
 | Place | Holds |
 |---|---|
 | `Caddyfile` | every site block — the source of truth `configure.py` reads back (`# vanguard:<role>` markers) |
-| `.env` | `MAIN_HOST`, `FILES_HOST`, `CLIPS_HOST`, `SYNC_HOST`, `DIAG_HOST`, `WATCH_HOST`, `GIT_HOST`, `BOT_HOST`, `LIVE_HOST`, `APEX_HOST`, `APEX_REDIRECT` |
+| `.env` | `MAIN_HOST`, `FILES_HOST`, `CLIPS_HOST`, `SYNC_HOST`, `DIAG_HOST`, `WATCH_HOST`, `GIT_HOST`, `BOT_HOST`, `LIVE_HOST`, `APEX_HOST`, `APEX_REDIRECT`; during a move `ALT_BASE` (every site also answers under it) and `REDIRECT_BASE` (old names 308 to the new) |
 | `mediamtx.yml` | `webrtcAdditionalHosts: [MAIN_HOST]` (needs a MediaMTX restart to take effect) |
-| `docker-compose.yml` → container env | Jellyfin `PublishedServerUrl` (WATCH), Gitea `DOMAIN/ROOT_URL/SSH_DOMAIN` (GIT), bot `WEB_PUBLIC_URL` (BOT), MediaMTX `LIVE_HOST` (text of the "is live" post), act_runner `GITEA_INSTANCE_URL` (GIT). `compose up -d` in the apply recreates the affected containers (seconds each) |
+| `docker-compose.yml` → container env | Jellyfin `PublishedServerUrl` (WATCH), Gitea `DOMAIN/ROOT_URL/SSH_DOMAIN` (GIT), bot `WEB_PUBLIC_URL` (BOT), MediaMTX `LIVE_HOST` (text of the "is live" post), act_runner `GITEA_INSTANCE_URL` (GIT), Kibana `server.publicBaseUrl` via `DIAG_HOST` (DIAG). `compose up -d` in the apply recreates the affected containers (seconds each). MediaMTX is on the host network and is deliberately **not** recreated by the apply |
+| Authentik, by `configure.py --apply` (`--ak-branding` alone) | the login **redirect stage** target `https://AUTH/source/oauth/login/<discord-slug>/` (members never see an Authentik page), the brand's default application (the proxy provider whose external host is MAIN), logo/favicon/background paths (relative, no host). Re-run after an **auth** or **main** change |
+| Kibana landing dashboard, by `configure.py --apply` (`KEY=… kibana-setup.py landing` alone) | the Vanguard tile's links: LIVE, FILES, WATCH, MAIN, GIT, BOT, SYNC from `.env`, AUTH derived as `auth.<domain of MAIN>`; `/streaming`, `/disk` relative. Re-run after any change |
 | Caddy TLS | certificates for new names are fetched automatically; old ones linger harmlessly |
 
 ### Pages in the repo that derive sibling hosts from the browser's location (label names hard-coded)
 | File | Assumes |
 |---|---|
-| `about/index.html` | `files.`, `clips.`, `sync.`, `live.`, `auth.`, `watch.`, `git.`, `bot.` + the parent of the current host |
+| `about/index.html` | `files.`, `clips.`, `sync.`, `live.`, `auth.`, `watch.`, `git.`, `bot.`, `diag.` + the parent of the current host (`sub('<label>')`); `/onboarding/` relative |
-| `dock/index.html` | `auth.` (sign-out), ingest host prefilled from the current host (`live.`; any name of the box works for RTMP/SRT) |
+| `dock/index.html` | `auth.` (sign-out), ingest host prefilled from the current host (`live.`; any name of the box works for RTMP/SRT); `/dump`, `/me/*`, `/time` relative |
+| `live/` portal, `streaming/index.html`, `streaming/disk.html`, `about/onboarding/` | nothing: every call is same-origin (`/webrtc/`, `/hls/`, `/me/*`, `/streaming/api/*`) |
 | `switchToClone.sh`, `deployLive.sh` | defaults for `git.dumpstervigilantes.rip` / `media.dumpstervigilantes.rip` (override by env/arg, or edit) |
 | `README.md`, `FRESH-INSTALL.md` | examples only |
 Renaming a *label* (e.g. `watch` → `video`) means editing the first two; changing only the domain does not.
@@ -49,7 +70,9 @@ Renaming a *label* (e.g. `watch` → `video`) means editing the first two; chang
 | Streamers' OBS | dock URL (Custom Browser Docks), dock Settings → ingest host (re-save), clock browser source URL (dock → Timecode band → "Add band + clock source" again), any manual RTMP/SRT configs (copy from the dock) | live |
 | Jellyfin apps on phones/TVs | server address | watch |
 | Bookmarks, Discord pinned links | main (gateway), clips, files, watch, live | all |
-Not affected: Syncthing pairings (device IDs), MariaDB, stream tokens, the Gitea webhook (gateway IP), Authentik users/groups/policies.
+Not affected: Syncthing pairings (device IDs), MariaDB, stream tokens, the Gitea webhook (gateway IP), Authentik
+users/groups/policies, the alert pipeline (Discord webhook + loopback Elasticsearch), the deadman, the publish block
+list and the WebRTC session history (`.markers/`), MediaMTX's host-network ports.
 
 ## A. Renaming one subdomain (example: `watch.BASE` → `video.BASE`)
 
@@ -58,50 +81,80 @@ Not affected: Syncthing pairings (device IDs), MariaDB, stream tokens, the Gitea
    "Jellyfin hostname" prompt; `y`. Caddy reloads (new block, old gone), Jellyfin is recreated with the new
    published URL. `curl -sI https://video.BASE/ | head -1` → 200/302 once the certificate is issued.
    (Only the *main* host touches `mediamtx.yml`; other hosts need no MediaMTX restart.)
-3. Authentik: Applications → Providers → `Jellyfin` → Edit → redirect URI → the new host → Update.
+3. Jellyfin first: Dashboard → General → Branding → disclaimer form action → new host → Save.
-4. Jellyfin: Dashboard → General → Branding → disclaimer form action → new host → Save.
+   Dashboard → Plugins → Vanguard Metadata → Public URL → new host → Save. (`https://video.BASE/` now loads and
-   Dashboard → Plugins → Vanguard Metadata → Public URL → new host → Save.
+   fails only at the SSO step: expected until step 4.)
+4. Authentik last: Applications → Providers → `Jellyfin` → Edit → redirect URI → the new host → Update.
 5. If the *label* changed (`watch`→`video`), edit `about/index.html` (`sub('watch')` → `sub('video')`) — commit, push,
    `sudo vanguard-deploy pull`. (`.env APEX_REDIRECT` too if the bare domain pointed at it: `./configure.py --full`.)
 6. Private window: `https://video.BASE/` → Discord → in. The gateway `https://HOST/` links to the new name.
 7. Remove the old DNS record. Tell members with Jellyfin apps to update the server address.
 
 Per-host extras when it is a different host:
-- **auth**: Discord portal redirect; Authentik outpost `authentik_host` and the proxy provider's authentication URL;
+- **auth**, in this order: Jellyfin SSO plugin OID endpoint and Vanguard plugin Authentik URL; Gitea auth source
-  Jellyfin SSO plugin OID endpoint; Vanguard plugin Authentik URL; Gitea auth source discovery URL;
+  discovery URL; `dock/index.html` + `about/index.html` if the label changed; THEN Authentik's outpost
-  `dock/index.html` + `about/index.html` if the label changed. Everyone re-logs in.
+  `authentik_host` and the proxy provider's authentication URL (log in there with the admin card); THEN the
+  Discord portal redirect. Everyone re-logs in.
 - **git**: Authentik Gitea provider redirect; `./registerGiteaRunner.sh`; server clone remote + `~/.ssh/config.vanguard`;
   dev remote; `switchToClone.sh` default.
-- **live**, **bot**: only the compose env (automatic) and `about/index.html` if the label changed.
+- **live**, **bot**: only the compose env (automatic) and `about/index.html` if the label changed. Streamers who use
+  Go Live need nothing (same-origin); OBS users re-add the dock URL (the dock lives on `live`).
+- **diag**: compose env → Kibana's `publicBaseUrl` (automatic, Kibana recreated), the Kibana landing tile
+  (automatic in the apply), `DIAG_REQUIRE_GROUP` in `.env` if the gate group should differ, `about/index.html`
+  if the label changed. Alarm rules and the Discord alert bus carry no hostname.
+- **auth**, in addition to the list above: `configure.py --ak-branding` (the apply does it) so the Discord-forward
+  redirect stage points at the new auth host — until then every login lands on the old name; the admin card
+  moves to `https://NEW-AUTH/if/flow/default-authentication-flow/?admin=1` (the Kibana tile link follows).
 - **main (HOST)**: `mediamtx.yml` changes → `docker compose restart mediamtx` between streams; every streamer
   re-adds the dock URL, re-saves Settings (ingest host), and re-runs "Add band + clock source"; manual RTMP/SRT
   configs are re-copied from the dock; `deployLive.sh` default host.
 
 ## B. Changing the base domain (everything at once)
 
-Plan a maintenance window: Caddy, Jellyfin, Gitea, the bot and MediaMTX all restart, and every member logs in again
+No maintenance window: every site answers on **both** bases first (`--alias`), so the old names keep working for
-(the Authentik session cookie is bound to the base domain).
+members while the apps and Authentik are moved one by one; the flip (`--base`) is the last server step and turns the
+old names into redirects. Order follows the rule above: servers, then the apps, then Authentik, then Discord, then
+the flip. `OLD` = the current base, `NEW` = the new one. Members' sessions survive until step 6; from step 6 on
+they log in once on the new names.
 
-1. DNS on the new domain: `A HOST2 → IP`, CNAMEs `files`, `clips`, `sync`, `live`, `auth`, `watch`, `git`, `bot` → `HOST2` (optional bare domain A).
+1. **DNS on NEW** (nothing else until this resolves): `A media.NEW` or CNAMEs for `media`, `files`, `clips`, `sync`,
-   Verify they resolve before continuing. Leave the old domain's records in place for now.
+   `live`, `diag`, `auth`, `watch`, `git`, `bot` → the box. Leave every OLD record in place. Verify:
-2. Discord developer portal → OAuth2 → add redirect `https://auth.NEW/source/oauth/callback/discord/` (keep the old one until step 9).
+   `for h in media files clips sync live diag auth watch git bot; do printf '%-6s ' $h; getent hosts $h.NEW | awk '{print $1}' | tail -1; done`
-3. Server, between streams: `./configure.py --full` → new main hostname; the eight sub-hostnames default to the new
+   — ten identical addresses. No bare-domain record unless it really points at the box.
-   base (Enter); bare domain; `y`. Then `docker compose restart mediamtx` (webrtcAdditionalHosts changed).
+2. **Both bases live:** `./configure.py --alias NEW`. Only the Caddyfile changes (`media.OLD, media.NEW {` …) and
-   All sites answer on the new names within a minute (certificates).
+   Caddy reloads; certificates for the new names follow within a minute. Nothing else moves. Check:
-4. Authentik (log in at the NEW auth host with akadmin — Discord login is broken until step 5 is done):
+   `curl -sI https://auth.NEW/ | head -1` → 302, and the old names unchanged. `sudo vghosts-check.py` now also
-   - Outposts → Embedded Outpost → `authentik_host: https://auth.NEW`
+   probes every alias.
-   - Providers → `Vanguard sites` → authentication URL `https://auth.NEW`, **cookie domain `NEW`**
+3. **Discord developer portal** → OAuth2 → Redirects → **add** `https://auth.NEW/source/oauth/callback/discord/`
-   - Providers → `Jellyfin` → redirect `https://watch.NEW/sso/OID/redirect/authentik`
+   beside the old one.
-   - Providers → `Gitea` → redirect `https://git.NEW/user/oauth2/authentik/callback`
+4. **Jellyfin** (any name, SSO still works): SSO-Auth `authentik` provider OID endpoint
-5. Private window: `https://auth.NEW/` → Discord login works (the redirect from step 2 is used).
+   `https://auth.NEW/application/o/jellyfin/` — works at once, Authentik answers on both names; Branding login
-6. Jellyfin: SSO plugin OID endpoint `https://auth.NEW/application/o/jellyfin/`; Branding disclaimer form action
+   disclaimer `https://watch.NEW/sso/OID/start/authentik`; Vanguard Metadata Public URL `https://watch.NEW`,
-   `https://watch.NEW/sso/OID/start/authentik`; Vanguard plugin Public URL + Authentik URL. Test the SSO start URL.
+   Authentik URL `https://auth.NEW`.
-7. Gitea: auth source discovery URL `https://auth.NEW/application/o/gitea/.well-known/openid-configuration`;
+5. **Gitea**: Site administration → Authentication sources → `authentik` → discovery URL
-   `./registerGiteaRunner.sh`; server: `cd ~/vanguard-stream && git remote set-url origin ssh://git@git.NEW:2222/techy/Media-Server-Setup.git`
+   `https://auth.NEW/application/o/gitea/.well-known/openid-configuration` (works at once, same reason).
-   and edit `~/.ssh/config.vanguard` (Host line); dev box: `git remote set-url origin …`. Test `sudo vanguard-deploy pull`.
+   Server clone remote and `~/.ssh/config.vanguard` → `git.NEW` (the host answers already). Dev box remote likewise.
-8. Repo defaults: `switchToClone.sh` (REPO_SSH), `deployLive.sh` (HOST) — commit, push, pull.
+6. **Authentik, then the flip, back to back, between streams.** This is the only moment anything is unreachable,
-9. Streamers: re-add the dock at `https://live.NEW/dock/`, re-save Settings, "Add band + clock source" again.
+   and it is seconds:
-   Members: `https://HOST2/`; Jellyfin apps → new server address. Remove the old Discord redirect and the old DNS.
+   `./configure.py --ak-hosts OLD NEW --dry-run` (read it) → `./configure.py --ak-hosts OLD NEW` → immediately
+   `./configure.py --base NEW` → `docker compose restart mediamtx` → `./configure.py --ak-branding` →
+   `./registerGiteaRunner.sh`.
+   `--ak-hosts` moves the outpost host, the proxy provider (external host + cookie domain `NEW`) and the
+   Jellyfin/Gitea redirect URIs: from this instant a login only sticks on the NEW names. `--base` makes NEW the
+   primary everywhere (`.env`, `mediamtx.yml`, Jellyfin/Gitea/runner/bot/Kibana recreated, Kibana tile), drops
+   the alias, and gives every OLD name a block that 308s to its NEW name, so bookmarks and pinned links keep
+   working. Members on an old name get redirected and log in once.
+7. **Test, private window:** `https://auth.NEW/` → Discord → in. `https://media.OLD/` → 308 → `https://media.NEW/`
+   → gate → in. `https://watch.NEW/` SSO round trip. `https://git.NEW/user/login` authentik sign-in.
+   `sudo vanguard-deploy pull`. `sudo vghosts-check.py` → 0 failed (it also checks every OLD name 308s).
+8. **Repo defaults:** `deployLive.sh` (HOST), `switchToClone.sh` (REPO_SSH) → NEW; commit, push, pull.
+9. **People:** OBS streamers re-add the dock at `https://live.NEW/dock/` (the old URL redirects, but OBS docks
+   want the real one), re-save Settings, "Add band + clock source" again; Go Live users just reload. Jellyfin apps
+   → new server address. Pins can stay, they redirect; re-pin at leisure. Remove the OLD Discord redirect entry
+   once everyone has logged in. The OLD DNS records stay as long as the NEW names are CNAMEs onto them.
+
+Rollback before step 6: `./configure.py --alias ""` and revert the two Jellyfin/Gitea screens. Rollback after step 6:
+`./configure.py --ak-hosts NEW OLD` then `./configure.py --base OLD` (which makes the NEW names the redirects).
 
 ## C. Splitting the single main host into media / files / clips / sync (done 2026-11-01)
 
@@ -144,8 +197,40 @@ After that, `--apply` maintains it like any other host, because the marker is no
 4. Verify the marker landed: `grep -c "vanguard:<role>" Caddyfile` → 1. Then `curl -sI https://NEW/` → 302.
 5. Add it to `addCommunityMediaServer.sh` so a rebuild asks for it too.
 
-## Checklist after any change
+## E. Validating a change: `sudo vghosts-check.py`
-    dev$ for h in HOST files.BASE clips.BASE sync.BASE live.BASE diag.BASE auth.BASE watch.BASE git.BASE bot.BASE; do printf '%-26s ' $h; curl -s -o /dev/null -m 10 -w '%{http_code}\n' https://$h/; done
+
+One command on the server checks every place in the tables above against the Caddyfile, the same source of
+truth `configure.py` reads back, and prints `ok` / `FAIL` / `warn` per item plus a `todo` list for the things no
+script can read (the Discord developer portal, streamers' OBS). Exit code 1 if anything failed.
+
+    server$ cd ~/vanguard-stream && sudo vghosts-check.py          # everything
+    dev$    ./bin/vghosts-check.py --repo                           # repo defaults, about-page labels, docs only
+    server$ sudo vghosts-check.py --json                            # for a script
+
+**The order is the order of the change, and it matters.** The sections run top-down because each one depends
+on the one above it, so a failure early on is usually the *cause* of the failures below it; fix the first
+failure and re-run rather than chasing the later ones:
+
+| # | Section | Depends on | Why this position |
+|---|---|---|---|
+| 1 | DNS | — | nothing gets a certificate or a visitor until the name resolves to the box |
+| 2 | Caddyfile | 1 | the roles and names `--full`/`--hosts` wrote; Caddy fetches certificates from here |
+| 3 | `.env` | 2 | the same names as compose variables; a mismatch is a half-finished `--full` |
+| 4 | `mediamtx.yml` | 3 | `webrtcAdditionalHosts` = main host; only a MediaMTX restart (between streams) applies it |
+| 5 | containers | 3 | the env each container was *created* with: Jellyfin, Gitea, runner, bot, MediaMTX, Kibana |
+| 6 | HTTPS | 1, 2 | every name answers with a valid certificate; gated names bounce to the auth host |
+| 7 | Jellyfin | 5, 6 | SSO plugin OID endpoint, branding disclaimer, Vanguard plugin URLs; live SSO start |
+| 8 | Gitea | 5, 6 | `app.ini`, the auth source (live redirect), the runner's registration, the server clone's remote |
+| 9 | Authentik | 7, 8 | **last of the services**: outpost host, proxy provider external host + cookie domain, Jellyfin/Gitea redirect URIs — changing these before the apps is the one way to lock yourself out |
+| 10 | Discord portal | 9 | the OAuth2 redirect is printed exactly; Discord login is broken until it matches; akadmin's card still works |
+| 11 | repo | 2 | `deployLive.sh`/`switchToClone.sh` defaults, about-page labels = Caddy roles, no browser links to `ovh1` in the docs |
+| 12 | streamers | 2 | OBS dock URL, clock source, Jellyfin apps, Discord pins: listed, never checked |
+
+What it knows that a human forgets: a container keeps the environment it was created with, so step 5 catches a
+`.env` that changed without `docker compose up -d <service>`; Authentik's cookie domain decides whether one
+login spans every host; and `ovh1.BASE` must resolve (ssh) but must *not* answer https.
+
+By hand, for the round trips the script only probes:
     https://HOST/whoami            → your identity (gate works on the new name)
     https://watch.BASE/sso/OID/start/authentik   → Jellyfin SSO round trip
     https://git.BASE/user/login    → "Sign in with authentik" round trip

LOGGING.md

@@ -20,7 +20,7 @@ journald input for our units. Metrics are the same story: see the taps table bel
 
 | endpoint | what | from |
 |---|---|---|
-| `http://127.0.0.1:9998/metrics` | MediaMTX: per path bytes/s, readers, publishers, WebRTC/HLS/RTMP/SRT sessions, dropped packets | MediaMTX `metrics: yes` |
+| `http://127.0.0.1:9998/metrics` | MediaMTX: per path bytes/s, readers, publishers, WebRTC/HLS/RTMP/SRT sessions, dropped packets. `remoteAddr` labels are real client addresses now that MediaMTX runs on the host network; before, every publisher showed as the bridge gateway `172.18.0.1` | MediaMTX `metrics: yes` |
 | `http://127.0.0.1:2019/metrics` | Caddy: requests by host/status, durations, TLS | Caddy admin API (`servers { metrics }`) |
 | `http://127.0.0.1:8080/metrics` | ours: the normalise queue, encode time, live stream count | `bin/dumpd.py` |
 
@@ -36,6 +36,15 @@ Three layers on purpose, because each answers a different question and none of t
 - **How bad is it, and is it getting worse** — the gauges: `vanguard_media_pending`,
   `vanguard_media_pending_oldest_seconds`, `vanguard_media_failed_files`. A Lens dashboard over
   `metricbeat-*` shows the trend across a night, which is the thing a list never can.
+- **Stitching** — `event=stitched stream= parts= joins= missing_ms= seconds= out= trash=` when a stream's reconnect
+  fragments were joined; `event=stitch_wait` (stream still going), `event=stitch_skipped reason=too_many`
+  (500+ files: use vgstuck.sh), `event=stitch_break reason=gap|params_changed` (chain split there),
+  `event=stitch_failed` (fragments left alone, published one by one), `event=defer` (a fragment left alone
+  because a sibling is still being written).
+- **Stop stream** — dumpd `stream_stop stream= kind= by= minutes= result=kicked|kick failed…`, then `publish_block user= minutes= by=`
+  / `publish_unblock`, and every refused reconnect as `publish_deny reason=blocked by=`.
+- **Wipe** — `component=vgstuck event=wipe_done recordings= stale_files= trash=` when an admin pressed Bin all
+  unprocessed on the Dump tab (dumpd logs `stuck_wipe by=`), with one `event=trashed file=` line per file before it.
 - **Which recordings** — the sweep logs `event=parked file= stream= log=` once per run for every file a
   failure log is holding back, plus `event=failed` with ffmpeg's own stderr when it first breaks. Both land
   in `filebeat-*`, so a Discover search or a dashboard table names them and stays current. Names deliberately

README.md

@@ -128,22 +128,34 @@ logged-in Discord user each time Go Live is pressed and sets it as the OBS strea
 
 OBS → Settings → Output → Encoder: **B-frames = 0** (NVENC "Max B-frames: 0"; x264 add `bframes=0`). MediaMTX
 refuses WebRTC playback of H.264 with B-frames ("WebRTC doesn't support H264 streams with B-frames"), so the live
+page falls back to HLS. The dock checks the actual recording a few seconds after Go Live and shows a red banner
+if B-frames are present (and in Simple output mode with x264 it sets `bf=0` for you); the Live tab on `diag.`
+shows the same per publisher. Nothing can set it for you in Advanced output mode, so the live
 portal would fall back to HLS with seconds of delay. Keyframe interval 2 s. Recording quality is unaffected.
 OBS → Tools → WebSocket Server Settings → enable, set a password.
 OBS → Docks → Custom Browser Docks → `https://live.BASE/dock/` → log in with Discord once.
 In the dock's Settings: ingest host (prefilled), OBS WebSocket password/port, optional title prefix/suffix
 (`{date}` → YYYY-MM-DD, e.g. prefix `HelmetCam-` or suffix `-{date}`).
 Per stream: type a title, click **Go Live**. The dock shows the final stream name (`<you>/<Title>`) as you type
-and enforces the naming rules the metadata relies on: letters, digits, `_ . -` only, no double underscore
+and enforces the naming rules the metadata relies on: letters, digits, `_ -` only (no dots: `<title>.webrtc` is reserved for the audio mirror), no double underscore
 (it separates title and timestamp in filenames), no leading dot, max 64 chars.
-After a bad night: **Dump tonight** (dumps the night of the account you are logged in as; `admins`/`media-admins`
+After a bad night: **Dump tonight** in the dock dumps the night of the account you are logged in as, and only
-may dump anyone's).
+that, admin or not. Dumping someone else's night is triage: **Dump a night**, a tab of the streaming page on
+`diag.BASE/streaming#dump` (media-admins): the night's recordings and clips per streamer, a date picker, Dump per
+streamer or Dump everyone, two confirmations. Same endpoint, same 3-day undo; dumpd honours a `user` only on that
+route. The night is 17:00–02:00 Eastern.
 
 Manual OBS/other software: dock → **Manual OBS setup** shows the RTMP server, the stream key and the SRT URL
 with the token hidden (Show / Copy / Fresh token). Tokens expire after 12 h; a running stream is not cut when
 its token expires, but a reconnect needs a fresh one.
 
 Browser streaming without OBS: the live portal's **Go Live** (Windows Chrome 141+ for window audio + GPU encode).
+On a lossy uplink Go Live switches itself to TCP: when the server reports more than 0.3% packet loss over 20 s it
+re-publishes the same capture with only MediaMTX's TCP candidate, which costs latency and nothing else. The stats
+line shows loss, retransmit requests (nack) and `via TCP` once it has happened; the recording picks up one extra
+file at the switch, which the sweep stitches. The switch is remembered in that browser, so the next stream starts
+on TCP; the "Lossy uplink" checkbox in Go Live's setup shows and controls it. Starlink users: tick it before the
+first stream, the dish drops a burst on every satellite handoff (~15 s cycle) and that is what was splitting files.
 
 ## 5a. Onboarding a gamer's clips (Syncthing) — self-service
 
@@ -198,6 +210,15 @@ In the UI (https://sync.BASE/):
 
 ## 5c. Authentik (Discord login) and Jellyfin
 
+**Login page branding and the Discord forward are not clicked in the UI.** `./configure.py --apply` (or
+`./configure.py --ak-branding` on its own) sets them from the ORM every time: brand title, logo, favicon and
+background from `authentik/brand/*.svg`, the login-card CSS, the media gateway as the brand's default application,
+flow titles, and a redirect stage at the top of `default-authentication-flow` that sends members straight to
+Discord. Nobody sees an Authentik page unless they open the admin card at
+`https://auth.BASE/if/flow/default-authentication-flow/?admin=1` (akadmin + password; the link is on the diag
+landing dashboard). Change the look by editing the SVGs or the `CSS` list in `configure.py`, then re-run.
+A UI edit to those objects is overwritten by the next apply, on purpose.
+
 DNS: `auth`, `watch` (and `files`, `clips`, `sync`, `live`, `git`, `bot`) as CNAMEs to the main host. Then run `./addCommunityMediaServer.sh` — it asks
 for every hostname, checks DNS, generates `.env` (Authentik secret + DB password; never commit it),
 adds the Caddy site blocks, starts the containers, waits for them and prints the steps below with your
@@ -447,8 +468,8 @@ Two paths on that host are ours rather than Kibana's:
 
 | path | what |
 |---|---|
-| `/stuck` | **stuck recordings**: everything that finished but never published, why each one is stuck, and Retry / Bin buttons. The list is readable by anyone the gate allows; the actions are enforced in dumpd for the admin groups, not hidden in the page. Binning moves the file to `trash/<today>/`, the same 3-day undo as a dumped night |
+| `/streaming` (Stuck tab) | **stuck recordings**: everything that finished but never published, why each one is stuck, and Retry / Bin buttons. The list is readable by anyone the gate allows; the actions are enforced in dumpd for the admin groups, not hidden in the page. Binning moves the file to `trash/<today>/`, the same 3-day undo as a dumped night |
-| `/stuck/api` | dumpd serves it. Both the page and `vgstuck.sh` on the host run the same script, so they cannot disagree about why something is stuck |
+| `/streaming/api/stuck` | dumpd serves it. Both the page and `vgstuck.sh` on the host run the same script, so they cannot disagree about why something is stuck |
 
 Kibana deliberately does **not** live under `/kibana`. That would need `server.basePath` plus
 `rewriteBasePath`, a Caddy handle that does not strip the prefix, and the prefix appended in both Beats'
@@ -456,8 +477,11 @@ Kibana deliberately does **not** live under `/kibana`. That would need `server.b
 one crash-loops Metricbeat, which exits when Kibana is unreachable. `defaultRoute` buys the landing page
 without any of it.
 
+After any hostname change, `sudo vghosts-check.py` validates every place a name lives, in the order the change
+must be made (HOSTNAME-CHANGE.md §E).
 `observability/kibana-setup.py` creates the first-run objects and takes a step name, so you can redo one:
-`connector`, `rule`, `dataview`, `landing`, or all of them when omitted. `observability/kibana-alarms.py`
+`connector`, `rule`, `dataview`, `landing`, or all of them when omitted. Disk use by streamer, clipper, month, logs
+and system is its own page, `https://diag.BASE/disk`. `observability/kibana-alarms.py`
 creates the six alarm rules.
 
 | container | job |
@@ -685,6 +709,11 @@ A `.env`-driven Authentik blueprint would remove the first line (planned).
 
 ## Layout on the server
 
+MediaMTX runs with `network_mode: host`, as its documentation recommends for WebRTC: the bridge put Docker's
+userland UDP proxy between browsers and port 8189 and it lost packets, which corrupted recordings and split
+them into fragments. Its ingest ports (1935, 8890/udp, 8189 tcp+udp) are accepted by the host firewall; its
+API, HLS, WHIP/WHEP signalling and metrics bind 127.0.0.1 and are reached only through Caddy and Metricbeat.
+
     /srv/media/streams/<user>/<Title>/.live/<ts>.mp4             recording in progress (hidden from Jellyfin/copyparty)
     /srv/media/streams/<user>/<Title>__<ts>.json                 capture time (UTC ms), timecode, source (strip|arrival), drift
     /srv/media/streams/<user>/<Title>__<ts>.mp4                  finished stream, one flat MP4 per recording, directly in
@@ -744,4 +773,4 @@ evening; after 17:00 it is tonight. The 05:00 trash purge is 05:00 Eastern.
 | trash-purge.timer | daily 05:00 Eastern | deletes trash older than 3 days |
 | vanguard-alerts.timer | every 1 min | delivers Kibana's alert documents to Discord, with a 15-min per-rule repeat guard |
 | vanguard-deadman.timer | every 5 min | watches the alerting stack from outside it, and the disks |
-| dumpd.service     | always | the dump endpoint on 127.0.0.1:8080, behind Caddy on `live.` at /dump, /me, /time, on `clips.` at /me/clips and on `diag.` at /stuck/api |
+| dumpd.service     | always | the dump endpoint on 127.0.0.1:8080, behind Caddy on `live.` at /dump, /me, /time, on `clips.` at /me/clips and on `diag.` at /streaming/api/stuck |

STATUS.md

@@ -50,7 +50,7 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea
 
 | Service | Role | State |
 |---|---|---|
-| MediaMTX | RTMP/SRT ingest, records fMP4; publish auth delegated to dumpd (tokens via `/mtx-auth`); on-ready hook posts "is live" and runs the WebRTC audio mirror (`<path>.webrtc`, audio → Opus, never recorded) | live, token auth verified |
+| MediaMTX | **Host network since 2026-11-05** (MediaMTX's recommended deployment for WebRTC). On the bridge, every WebRTC media packet on 8189/udp was relayed by Docker's userland proxy, which lost ~0.4% of them: each loss corrupted an access unit, the recorder reopened its file, and one evening produced 471 fragments; every publisher also appeared as 172.18.0.1. Ingest (1935, 8890/udp, 8189) now arrives through the nftables INPUT chain; API, HLS, WHIP/WHEP signalling and metrics bind 127.0.0.1 and are reached only through Caddy and Metricbeat. RTMP/SRT ingest, records fMP4; publish auth delegated to dumpd (tokens via `/mtx-auth`); on-ready hook posts "is live" and runs the WebRTC audio mirror (`<path>.webrtc`, audio → Opus, never recorded) | live, token auth verified |
 | copyparty | browse/scrub/download/finals upload | behind Authentik, group permissions (`editors`, `media-admins`), Discord-only, no WebDAV | live |
 | dumpd + dock | dump, stream tokens, MediaMTX auth hook, `/time`, `/me` identity (on `live.`), `/me/clips` Syncthing self-service (on `clips.`); OBS dock with timecode-band setup | identity from Authentik headers | live |
 | `media.` gateway, `clips.` self-service | static pages behind the gate | — | live; **first real pairing 2026-11-01 (thrahg)** — exposed a missing `.stlocal` on fresh folders, fixed |
@@ -64,7 +64,22 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea
 | Alerting | **Kibana evaluates, we deliver.** A basic licence has no webhook/Slack/email connector (403, Gold+), so rules write documents into the `vanguard-alerts` index through the free `.index` connector and `vanguard-alerts.timer` takes them to Discord every minute, with 15-minute per-rule repeat suppression. Alerts are therefore searchable in Discover beside the logs that caused them. Rules are still authored in Kibana's UI; `observability/kibana-setup.py` creates the connector and the heartbeat rule | **added 2026-11-03** |
 | Alarm rules | Six, created by `observability/kibana-alarms.py`: disk and memory below 15% free, bot not running, bot unhealthy, recordings stuck over an hour, normalise failures parked. Each repeats every 12 h while unresolved and stops when it clears | **added 2026-11-03**; needs the bot release that fixes `/health` before the bot-unhealthy rule means anything |
 | Deadman | `vanguard-deadman.timer` on the host, every 5 min, curl only: Elasticsearch answering and not red, no index read-only from the watermark (never self-clears), Kibana available, **each Beat actually writing** (running ≠ shipping), the Kibana heartbeat fresh, and the disks. Reports on change; journal-only and non-zero exit when no webhook is set | **live 2026-11-03** — and it earned itself on the first run by catching Filebeat up-but-not-shipping, which every process check called healthy |
-| Stuck recordings | A page at `https://diag.…/stuck` for anyone past the gate, on the triage host (already `media-admins` only), linked from the gateway's Media admin card and from the diag landing dashboard: every unpublished recording, why it is stuck, and Retry / Bin buttons. dumpd serves `/stuck/api` and allows the actions only for the admin groups; both the page and the CLI run `bin/vgstuck.sh`, so they can never disagree. `bin/vgstuck.sh` triages anything that never published: it says why each one is stuck (`parked` by a failure log with ffmpeg's reason, `published` collision, `waiting`, or still `writing`), then `retry <match>` clears the blocker and `drop <match>` moves it to trash with the usual 3-day undo. `drop --all` is refused. The sweep also logs `event=parked` once per run so Kibana can name them, not just count them | **added 2026-11-04** |
+| Fragment stitching | A stream that dropped and came back is one recording in several `.live/` files. `bin/vgstitch.py` (run first by every sweep) joins them losslessly with the concat demuxer when there are fewer than `STITCH_MAX_PARTS` (500; was 20 until Kyle's lossy night showed a recorder restart every ~10 s is normal for 0.5% loss) and the gaps are under `STITCH_MAX_GAP_S` (10 min); the arrival-time file names place every piece. Joins are collapsed and recorded in the sidecar (`stitched.parts/joins`, `missing_ms`). Nothing in a stream's `.live/` is touched until no file there is open or fresh (`event=defer` / `event=stitch_wait`). A resolution/codec change breaks the chain (copy-concat is impossible across it), so Go Live now holds resolution and sheds frame rate instead. The fragments go to `trash/<night>/` for the 3-day undo; a failed stitch leaves them alone and the old per-file path publishes them | **added 2026-11-05** |
+| Reconnect fragments | Every publisher drop-and-reconnect opened a new MediaMTX recording; one bad night left **471** one- and two-second files in `.live/`, mostly from the 6 Mbps Go Live floor (fixed). Cleanup: `vgstuck.sh drop --under 5` or the page's bulk button, admin-only, confirm shows the count. Prevention: fragments are stitched first (row above); a **lone** stream recording under `MIN_LIVE_SECONDS` (10) goes to trash instead of being published. Clips are never subject to either: they are a few seconds long on purpose and never pass through `.live/` | **added 2026-11-04** |
+| Login branding + Discord forward | Done from the ORM by `configure.py` on every apply (`--ak-branding` runs just this): brand title/logo/favicon/background from `authentik/brand/*.svg` (installed into Authentik's own `/media/public/brand/`, the only unauthenticated place that can serve them), login-card CSS in the site palette with Discord blurple, brand default application = the media gateway (so `auth.` root never shows Authentik's app library), flow titles renamed, Discord attached to the identification stage. Then the forward: a **redirect stage first in `default-authentication-flow`** whose static target is the Discord source's login URL, bound to an expression policy that skips it when the flow URL carries `?admin=1` (the executor keeps the flow query in the session key `authentik/flows/get`, not `request.GET`). Members never see an Authentik page: protected link → Discord consent → back where they were (the outpost's `next` rides the same session key). akadmin's password card is `https://auth.BASE/if/flow/default-authentication-flow/?admin=1`, linked only from the diag landing dashboard | **added 2026-11-06** |
+| Frozen Go Live picture | Breezy: one frame, then one more per alt-tab. Window capture on Windows comes from the compositor and an **exclusive full-screen** game bypasses it, so Chrome only gets a frame when Windows composites one (alt-tab). Go Live now has a capture choice, window (default) or **whole screen** (`displaySurface: monitor`, desktop duplication: sees exclusive full-screen, carries system audio), remembered per browser, plus a hint naming the symptom and the better fix: the game in borderless/windowed full-screen keeps window audio and needs no change | **added 2026-11-06** |
+| Base-domain move, no outage | `configure.py --alias NEW` makes every Caddy site answer on both bases (`media.old, media.new {`), certificates issued, nothing else touched; apps and Authentik are then moved one by one while the old names keep working; `--ak-hosts OLD NEW` (dry-run first) moves Authentik last; `--base NEW` flips the primary, recreates the host-bearing containers, and turns the old names into 308 redirects (`REDIRECT_BASE`). `read_hosts` understands both Caddyfile forms; the sweep probes aliases and old-name redirects. Rule documented at the top of HOSTNAME-CHANGE.md: apps before Authentik, never the reverse | **added 2026-11-06** |
+| Hostname sweep | `bin/vghosts-check.py` (installed; `sudo vghosts-check.py` on the box, `--repo` on a dev box, `--json`): validates every place HOSTNAME-CHANGE.md lists against the Caddyfile — DNS, `.env`, `mediamtx.yml`, the env each running container was created with, HTTPS + gate redirects per host, Authentik (outpost host, proxy external host + cookie domain, Jellyfin/Gitea redirect URIs, the exact Discord callback to check by eye), Jellyfin config XML + live SSO start, Gitea `app.ini` + live auth-source redirect + runner + clone remote, repo defaults, about-page labels, docs with browser links to `ovh1`. Sections run in the order a change must be made (documented in HOSTNAME-CHANGE.md §E) because each depends on the previous; exit 1 on any failure | **added 2026-11-06** |
+| Onboarding checklist | `media.BASE/onboarding/` (`about/onboarding/index.html`, served by the gateway's file server, so behind the same Discord gate). For the person **doing** the onboarding: eight topics to tick as they are covered — who we are, about/verify, community, who Vanguard is, who Spartan are, comms, training, push to mute/talk — and a Clear button for the next recruit. Ticks are per browser. The gateway's `try_files` gained `{path}/index.html` so folder pages under `about/` resolve without a trailing slash | **added 2026-11-06** |
+| Disk page | `diag.BASE/disk`, its own page (`streaming/disk.html`, served from the same mount): filesystem bars, then recordings per **streamer by month** (month from the recording's own `__YYYY-MM-DD` stamp; raw `.src` and unpublished `.live` called out), **clips per clipper by month**, trash by night, **logs** (Elasticsearch indices by the date in their names, the ES data dir as the on-disk truth, journald, Docker json logs) and **system** (each `appdata/<app>`, live portal, bot, Docker's own image/volume figures). dumpd walks the tree with allocated-block sizes, caches ten minutes, `?refresh=1` / "Rewalk now" forces | **added 2026-11-06** |
+| Stop a stream (Live tab) | Each publisher row has **Stop stream** with a block length (kick only / 30 min / 2 h / 8 h, default 30 min). dumpd kicks the session through MediaMTX's API (`/v3/<webrtcsessions|rtmpconns|srtconns>/kick/<id>`) **and** writes the account into `.markers/publish-blocks.json`; `/mtx-auth` denies a blocked account on every publish, so OBS's auto-reconnect and a Go Live restart both get "publishing is paused for this account by an admin" until it expires or an admin presses Allow again in the new "Blocked from publishing" section. The block is written before the kick, so there is no reconnect window. Admin groups only; the recording so far is kept and the sweep publishes it as usual | **added 2026-11-06** |
+| Transport on the Live tab | MediaMTX's session API labels the selected ICE pair `host/udp/…` or `host/tcp/…`, so the publishers table now shows `webrtc udp` / `webrtc tcp` and the session start time, and a new "Publish sessions — last 24 h" table lists every WebRTC publish session with transport, start, end, duration, received/lost, and a "switched from udp" marker on the session that began the TCP leg: that is the when and the before/after loss of a lossy-uplink switch. dumpd keeps the history in `.markers/webrtc-history.json`, updated on every Live-tab poll and every metrics scrape, so it fills in while nobody is watching | **added 2026-11-06** |
+| Lossy uplinks (Go Live) | Kyle's night: one publish session, 16 recorder restarts, 165 processing errors, 0.5% loss. A loss burst that eats a keyframe's slices leaves MediaMTX's recorder an access unit with only SPS/PPS, it refuses it (`access unit doesn't contain an IDR or non-IDR NALU`) and the path restarts the recorder 2 s later = a new file every ~10 s. Three levers pulled: (1) retransmits — MediaMTX's WebRTC code calls pion `ConfigureNack` for every peer connection, so NACK is on; the Go Live stats line now shows `loss x% (n nack, m resent)` so a stream where NACKs stay at 0 under loss is visible at a glance; (2) the stitcher cap went 20 → 500 so such a night is one recording; (3) **automatic TCP**: Go Live samples the server's receiver reports every second and when loss over the last 20 s exceeds 0.3% (after at least 1500 packets) it ends the WHIP session and re-publishes the same tracks with only MediaMTX's TCP candidates (`webrtcLocalTCPAddress :8189`) left in the answer. One extra recording file (stitched), ~2 s of picture lost in the swap, then no loss can reach the recorder; latency is what TCP costs instead. It never flips back during a stream; the stats line says `via TCP` and why. Kyle is on **Starlink**, whose scheduler reshuffles every ~15 s and drops a burst at each handoff — that is the every-10-to-15-s restart cadence in his log, and no NACK can cover a burst that long. So the switch is also **remembered per browser** (`vg.golive.tcp`): once it has fired, the next stream starts on TCP, no 20 s of corrupted recording and no extra file; a "Lossy uplink" checkbox in Go Live's setup shows and controls it. First TCP night: stable, one file, zero recorder restarts — but soft: with no loss to react to, Chrome's estimator only sees the dish's delay swings and parks near the 1.2 Mbps floor. TCP publishes now use a **4 Mbps floor** (`TCP_MIN_KBPS`): safe there because the UDP failure mode of a floor (sending into loss) cannot happen on TCP, the worst case is added latency | **added 2026-11-06** |
+| B-frame enforcement | obs-websocket cannot set encoder options in Advanced output mode, but it can switch **profiles**, and encoder settings live in a profile. The dock has an "OBS profile to stream from" setting (default `Vanguard`) and switches OBS to it before every Go Live, plus a checkbox to do so when the dock loads, i.e. on OBS startup. A streamer sets the profile up once with B-frames 0; the post-Go-Live probe verifies the result. Missing profile = a red status line naming what to create, never a silent fallback | **added 2026-11-05** |
+| B-frame detection | MediaMTX refuses H.264 with B-frames over WebRTC, and nothing on the publish path can see it: obs-websocket cannot set encoder options in OBS's Advanced output mode, and MediaMTX only notices when a viewer connects. dumpd now reads the B-frame depth off the first fragments of the live recording (`ffprobe`, small probesize, ~50 ms, works on a growing file, cached 30 s). The **dock** polls `/me/stream-health` after Go Live and shows a red banner with per-encoder instructions; in Simple output mode with x264 it writes `bf=0` into the options itself first. The **Live tab** shows an encode column per publisher: "no B-frames" or "N B-frames · WebRTC refused" | **added 2026-11-05** |
+| Streaming page | `https://diag.…/streaming`, two tabs. **Live**: every publisher with its real address, packets received and lost on the way in, the last ten minutes of publish sessions / recorder restarts / processing errors, and a one-line verdict (loss, reconnects, or parameter changes) with the recorder log tail. **Stuck**: the triage below. Served by dumpd `/streaming/api/live` and `/streaming/api/stuck`; `/stuck` redirects | **added 2026-11-05** |
+| Dump a night (web) | Third tab of the streaming page on the triage host: the night's recordings and clips grouped by streamer with sizes, a date picker (defaults to dumpd's "tonight, else last night"), a Dump button per streamer and a Dump everyone button for admins. Two confirmations, then a POST to `/streaming/api/dump` on this host, so one cookie covers it. The dock's own Dump tonight button stays for a streamer's own night and is personal for everyone, admins included: dumpd honours a `user`/`all` only when the request came in on `/streaming/api/dump`, so the only place another streamer's night can be dumped is this tab. GET is a dry run; only the POST moves files, to `trash/<night>/` with the 3-day undo. Below the night table: **Bin all unprocessed**, which runs `vgstuck.sh wipe` — stops the sweep (killing any conversion in flight; the timer restarts it to an empty queue), then every `.live/` file from any night, parked or fresh, plus the failure logs and `.part` leftovers, then the emptied folders. The only thing it leaves is a file still held open after the sweep is down, i.e. a stream MediaMTX is recording this minute. `drop --all` stays refused so a pattern can never do this by accident | **added 2026-11-05** |
+| Stuck recordings | The Stuck tab of the page above for anyone past the gate, on the triage host (already `media-admins` only), linked from the gateway's Media admin card and from the diag landing dashboard: every unpublished recording, why it is stuck, and Retry / Bin buttons. dumpd serves `/streaming/api/stuck` and allows the actions only for the admin groups; both the page and the CLI run `bin/vgstuck.sh`, so they can never disagree. `bin/vgstuck.sh` triages anything that never published: it says why each one is stuck (`parked` by a failure log with ffmpeg's reason, `published` collision, `waiting`, or still `writing`), then `retry <match>` clears the blocker and `drop <match>` moves it to trash with the usual 3-day undo. `drop --all` is refused. The sweep also logs `event=parked` once per run so Kibana can name them, not just count them | **added 2026-11-04** |
 | Our own metrics | `bin/vgmetrics.sh` counters from the sweep and vgdeband + live gauges from dumpd `GET /metrics` (loopback, unrouted by Caddy), scraped by Metricbeat. Answers what MediaMTX and the system module cannot: normalise queue depth and its oldest item, encodes in flight, ffmpeg seconds by verdict and by stream/clip, failures parked, live stream count with the `.webrtc` mirrors separated. `vanguard_media_pending` trending up across a night is the measured signal that CPU-only encoding is not keeping pace | **added 2026-11-03** |
 | Logging framework | rotated Docker logs + `vanguard.service/category` labels on every container; dumpd/sweep/vgdeband/deploy emit logfmt events (`LOGGING.md`). Consumed by Filebeat into Elasticsearch | live (labels + logfmt); Elastic deploy pending |
 | Authentik (+Postgres, Redis) | Discord login, groups, guild-membership policy | live |
@@ -119,6 +134,10 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea
 - Sweep leaves compliant Syncthing files untouched and keeps exactly one copy of converted ones (tested).
 - Firewall: Docker-published ports bypass the input chain; MariaDB uses host networking precisely so the rules apply.
 - Never `systemctl stop|restart nftables` (flushes Docker's NAT). Reload only.
+- **"Enforce B-frames 0 through obs-websocket" is only possible in Simple output mode with x264.** Advanced-mode encoder settings live in `streamEncoder.json`, which the protocol does not expose. The enforceable thing is detection on the server (the recording reveals the reorder depth within seconds) with a loud dock banner, which covers every encoder. Found 2026-11-05 when an OBS stream was unwatchable with `has_b_frames=1`.
+- **A published UDP port on the Docker bridge goes through `docker-proxy`, a userland relay, and it drops packets under load.** The sign is every client appearing as the bridge gateway (`172.18.0.1`) in MediaMTX's metrics. For WebRTC media at ~1000 packets/s this produced steady ~0.4% loss that no browser-side setting could fix, because the loss was on our box. Anything doing real-time UDP belongs on `network_mode: host`, which is also what MediaMTX documents. Found 2026-11-05 after chasing it as a client bitrate problem first.
+- **Two MediaMTX path regexes that can both match a name is a coin toss, not "first rule wins".** The mirror rule (`…\.webrtc$`) and the generic rule both matched `Techy_multi_3.webrtc` because titles were allowed dots, and the generic one won: a **36 GB** audio mirror was recorded as a full duplicate of its source. Both rules now have a dot-free title segment, so a name satisfies exactly one, and titles no longer accept dots anywhere they are typed. Needs a MediaMTX recreate between streams. Found 2026-11-04.
+- **Never `ffprobe` fragmented MP4 in a request path.** `.live/` recordings have no duration in their header, so ffprobe reads the whole file to answer. The stuck-recordings survey did that per file, took over three minutes for 471 of them, dumpd's 180 s request ceiling killed the response, Caddy returned an empty 502 and the page said "unexpected end of JSON". Size and age come from `stat`; duration is opt-in (`vgstuck.sh list --durations`). Found 2026-11-04.
 - **A high `x-google-min-bitrate` disables WebRTC congestion control.** Go Live sent `min-bitrate=6000`, so Chrome never encoded below 6 Mbps; on an uplink that could not carry it, the excess was dropped, MediaMTX logged `invalid FU-A packet (non-starting)` and every viewer saw blocky corruption. The loss was on the PUBLISH session (4761 packets for one streamer) while the viewer sessions lost nothing, which is the fingerprint: the server relays damage it received. Floor lowered to 1200 and `scaleResolutionDownBy` unpinned so the encoder can shed resolution. Found 2026-11-04.
 - `tracks=['Opus','H264']` on a path means a browser (Go Live); `MPEG-4 Audio` means OBS. Quick way to tell who is publishing how.
 - **`handle_path` strips the prefix, so an upstream's own redirects lose it.** MediaMTX redirects every HLS playlist request once (to `?cookieCheck=1`) with a root-relative `Location`, so the browser followed `/<user>/<Title>/index.m3u8` into the portal's SPA catch-all, got `index.html`, and hls.js said `manifestParsingError` — a web page where a manifest should be. Fixed with `header_down Location ^/ /hls/`. `curl` against MediaMTX directly hides this completely, because there is no prefix to lose. Found 2026-11-04.

about/index.html

@@ -32,6 +32,7 @@
     <div class="links">
       <a class="card" id="l-watch" href="#"><b>Recordings &amp; clips</b><span>Jellyfin — by streamer, by day/week/month</span></a>
       <a class="card" id="l-live" href="#"><b>Live now</b><span>every active stream in one page, audio follows the window you click</span></a>
+      <a class="card" href="/onboarding/"><b>Onboarding checklist</b><span>for whoever is onboarding someone: tick the topics as you cover them</span></a>
       <a class="card" href="/logout"><b>Sign out</b><span>ends the Discord session for all Vanguard sites</span></a>
     </div>
     <details><summary>How watching works</summary>
@@ -65,7 +66,7 @@
       <ol>
         <li>OBS → Tools → WebSocket Server Settings → enable, set a password.</li>
         <li>OBS → Docks → Custom Browser Docks → add <code id="dockurl">…/dock/</code>; log in with Discord inside the dock.</li>
-        <li>OBS → Settings → Output → encoder <b>B-frames: 0</b> (NVENC "Max B-frames"). With B-frames the live page cannot play you over WebRTC and falls back to a delayed HLS view.</li>
+        <li>Make an OBS <b>profile named Vanguard</b> once (Profile → New) and in it set Output → Advanced → your encoder with <b>B-frames 0</b> (NVENC "Max B-frames", x264 <code>bf=0</code>) and keyframe interval 2 s. The dock switches OBS to that profile on every Go Live, and on startup if you tick the box in its settings; after Go Live it checks what the server actually received and turns red if B-frames got through. With B-frames the live page cannot play you over WebRTC and falls back to a delayed HLS view.</li>
         <li>Dock → Settings: OBS WebSocket password (the rest is prefilled). Optional title prefix/suffix (<code>{date}</code> = today).</li>
         <li>Dock → Timecode band → <b>Add band + clock source</b> while not streaming (gives editors a frame-accurate timecode).</li>
         <li>Per stream: type a title, <b>Go Live</b>. The dock mints a 12-hour token and sets your stream key itself.</li>
@@ -74,7 +75,7 @@
     <details><summary>Rules worth knowing</summary>
       <ul>
         <li>Everything streamed is recorded and visible to editors and members. <b>Dump tonight</b> throws a night away (3-day undo by an admin).</li>
-        <li>Titles: letters, digits, <code>_ . -</code>; no double underscore; the dock shows the final name as you type.</li>
+        <li>Titles: letters, digits, <code>_ -</code>; no dots (the server reserves <code>.webrtc</code>); no double underscore; the dock shows the final name as you type.</li>
         <li>Browser Go Live needs Windows + Chrome/Edge 141+ for window audio and the GPU encoder; on Linux use OBS.</li>
       </ul>
     </details>
@@ -120,7 +121,7 @@
     <h2>Media admin</h2>
     <p class="lead" data-group="media-admins" data-yes="You are in <code>media-admins</code>: full rights on every folder, trash, Syncthing, dump anyone's night." data-no="Admin view — you are <b>not</b> in <code>media-admins</code>: copyparty will show you nothing and <code>/sync/</code> will 403 until you get the role."></p>
     <div class="links">
-      <a class="card" id="l-stuck" href="#"><b>Stuck recordings</b><span>never published — see why, retry or bin</span></a>
+      <a class="card" id="l-stuck" href="#"><b>Streaming</b><span>who is live and losing packets; recordings that never published</span></a>
       <a class="card" id="l-trash" href="#"><b>Trash</b><span>dumped nights, purged after 3 days</span></a>
       <a class="card" id="l-sync" href="#"><b>Syncthing</b><span>device pairing, folder states</span></a>
       <a class="card" href="/whoami"><b>Who am I</b><span>identity headers as the apps see them</span></a>
@@ -181,8 +182,8 @@ $('l-git').href = sub('git'); $('l-bot').href = sub('bot'); $('l-auth').href = s
 $('l-jfadmin').href = sub('watch') + '/web/#/dashboard'; $('l-gitadmin').href = sub('git') + '/-/admin';
 $('l-dock').href = sub('live') + '/dock/'; $('dockurl').textContent = sub('live') + '/dock/';
 $('l-files').href = sub('files') + '/'; $('l-finals').href = sub('files') + '/finals/'; $('l-trash').href = sub('files') + '/trash/'; $('l-sync').href = sub('sync') + '/';
-$('l-stuck').href = sub('diag') + '/stuck';
+$('l-stuck').href = sub('diag') + '/streaming#stuck';
-document.querySelectorAll('a.l-stuck-note').forEach(a => a.href = sub('diag') + '/stuck');
+document.querySelectorAll('a.l-stuck-note').forEach(a => a.href = sub('diag') + '/streaming#stuck');
 $('l-clips').href = sub('clips') + '/'; document.querySelectorAll('a.l-clips').forEach(a => a.href = sub('clips') + '/'); $('gitssh').textContent = `ssh://git@${'git.' + base}:2222/org/repo.git`;
 (async () => {
   try {

about/onboarding/index.html

@@ -0,0 +1,58 @@
+<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<title>Onboarding — Vanguard</title>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<style>
+  :root{--bg:#141414;--fg:#e6e6e6;--mut:#9a9a9a;--acc:#5865F2;--ok:#2ecc71;--box:#1f1f1f;--line:#2e2e2e}
+  html,body{margin:0;background:var(--bg);color:var(--fg);font:17px/1.5 system-ui,sans-serif}
+  main{max-width:560px;margin:0 auto;padding:36px 16px 48px}
+  header{display:flex;align-items:center;gap:14px;margin-bottom:18px}
+  h1{font-size:26px;margin:0}
+  .right{margin-left:auto;display:flex;align-items:center;gap:12px;color:var(--mut);font-size:14px}
+  button{font:inherit;font-size:14px;border-radius:8px;border:1px solid #3a3a3a;background:#262626;color:var(--fg);padding:7px 14px;cursor:pointer}
+  button:hover{border-color:var(--acc)}
+  section{background:var(--box);border:1px solid var(--line);border-radius:14px;padding:8px 10px}
+  label{display:flex;gap:14px;align-items:center;padding:12px 10px;border-radius:10px;cursor:pointer;font-size:19px}
+  label:hover{background:#262626}
+  label input{width:22px;height:22px;margin:0;accent-color:var(--acc);flex:none}
+  label input:checked + span{color:var(--mut);text-decoration:line-through}
+  .done{display:none;color:var(--ok);margin-top:14px;font-size:15px}
+</style>
+</head>
+<body>
+<main>
+  <header>
+    <h1>Onboarding</h1>
+    <div class="right"><span id="count"></span><button id="clear">Clear</button></div>
+  </header>
+  <section>
+    <label><input type="checkbox" data-id="who"><span>Who we are</span></label>
+    <label><input type="checkbox" data-id="about"><span>About / verify</span></label>
+    <label><input type="checkbox" data-id="community"><span>Community</span></label>
+    <label><input type="checkbox" data-id="vanguard"><span>Who Vanguard is</span></label>
+    <label><input type="checkbox" data-id="spartan"><span>Who Spartan are</span></label>
+    <label><input type="checkbox" data-id="comms"><span>Comms</span></label>
+    <label><input type="checkbox" data-id="training"><span>Training</span></label>
+    <label><input type="checkbox" data-id="ptt"><span>Push to mute / talk</span></label>
+  </section>
+  <div class="done" id="done">All covered. Clear for the next one.</div>
+</main>
+<script>
+const KEY = 'vg.onboarding';
+const boxes = [...document.querySelectorAll('input[type=checkbox]')];
+const load = () => { try { return JSON.parse(localStorage.getItem(KEY) || '{}'); } catch { return {}; } };
+const save = s => { try { localStorage.setItem(KEY, JSON.stringify(s)); } catch {} };
+function render() {
+  const s = load(); let n = 0;
+  for (const b of boxes) { b.checked = !!s[b.dataset.id]; if (b.checked) n++; }
+  document.getElementById('count').textContent = `${n} / ${boxes.length}`;
+  document.getElementById('done').style.display = n === boxes.length ? 'block' : 'none';
+}
+for (const b of boxes) b.addEventListener('change', () => { const s = load(); s[b.dataset.id] = b.checked; save(s); render(); });
+document.getElementById('clear').onclick = () => { save({}); render(); };
+render();
+</script>
+</body>
+</html>

authentik/brand/favicon.svg

@@ -0,0 +1,4 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="64" height="64" viewBox="0 0 64 64">
+  <rect width="64" height="64" rx="14" fill="#5865F2"/>
+  <path d="M14 16h13l5 24 5-24h13L40 52H24z" fill="#141414"/>
+</svg>

authentik/brand/login-bg.svg

@@ -0,0 +1,16 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="1920" height="1080" viewBox="0 0 1920 1080" preserveAspectRatio="xMidYMid slice">
+  <defs>
+    <radialGradient id="g" cx="30%" cy="20%" r="90%">
+      <stop offset="0" stop-color="#23264a"/>
+      <stop offset="0.55" stop-color="#141414"/>
+      <stop offset="1" stop-color="#0c0c0c"/>
+    </radialGradient>
+    <pattern id="grid" width="48" height="48" patternUnits="userSpaceOnUse">
+      <path d="M48 0H0v48" fill="none" stroke="#5865F2" stroke-opacity="0.08" stroke-width="1"/>
+    </pattern>
+  </defs>
+  <rect width="1920" height="1080" fill="url(#g)"/>
+  <rect width="1920" height="1080" fill="url(#grid)"/>
+  <circle cx="1500" cy="860" r="420" fill="#5865F2" fill-opacity="0.07"/>
+  <circle cx="300" cy="150" r="260" fill="#3d8bfd" fill-opacity="0.06"/>
+</svg>

authentik/brand/logo.svg

@@ -0,0 +1,5 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="400" height="96" viewBox="0 0 400 96" role="img" aria-label="Vanguard">
+  <rect x="4" y="8" width="80" height="80" rx="18" fill="#5865F2"/>
+  <path d="M22 26h16l6 30 6-30h16L54 72H34z" fill="#141414"/>
+  <text x="100" y="66" font-family="Inter, Segoe UI, Helvetica, Arial, sans-serif" font-weight="800" font-size="44" letter-spacing="3" fill="#e6e6e6">VANGUARD</text>
+</svg>

bin/dumpd.py

@@ -3,7 +3,10 @@
 
 POST /dump   identity from X-Authentik-Username / X-Authentik-Groups (set by Caddy after Authentik)
              body (optional JSON): {"night":"YYYY-MM-DD","user":"name"}
-GET  /dump   -> lists what a dump would move (dry run)
+GET  /dump   -> lists what a dump would move (dry run); ?user=all&night=YYYY-MM-DD select whose and when
+             Two doors, different reach: /dump on live. (the dock's Dump tonight) is ALWAYS the caller's own night,
+             admin or not — "user" is ignored there. /streaming/api/dump on diag. (the streaming page's Dump tab) is
+             where admins pick a streamer or "all"; non-admins still get only their own there.
 GET  /me/stream-token -> short-lived publish token for the caller (member of streamers/admins); the dock
              and the live portal set it as the OBS/WHIP credential. No stored passwords anywhere.
 GET  /me/clips  -> Syncthing self-service status for the caller: our device id, their folder/device state
@@ -21,10 +24,18 @@ POST /deploy-bot -> Gitea Actions (job container -> host gateway) hands over a D
                    HMAC-SHA256 signed with BOT_DEPLOY_SECRET; installs it with vanguard-deploy-bot.
                    May carry X-Vanguard-Env (base64 JSON): the bot's own settings, kept in Gitea, written
                    into the stack .env for the keys BOT_ENV_KEYS allows. Signature covers header+body.
-GET  /stuck/api  -> every unpublished recording and why it is stuck (any gated user)
+GET  /me/stream-health -> the caller's live streams with B-frame depth: the dock warns when WebRTC will refuse
-POST /stuck/api  -> {"action":"retry"|"drop","src":...}: clear the blocker, or move it to trash with the
+GET  /streaming/api/live  -> what is publishing, from where, loss on the way in, recorder restarts, transport history, blocks
-             usual 3-day undo. Admin groups only; both actions run bin/vgstuck.sh so the web page and the
+POST /streaming/api/live  -> {"action":"stop","path","kind","session","minutes"}: kick that publish session via MediaMTX's
-             CLI share one implementation.
+             API and block the account's publishing for N minutes (0 = kick only); {"action":"unblock","user"}. Admins only.
+             /mtx-auth denies a blocked account, so neither OBS's auto-reconnect nor a Go Live restart gets around it.
+GET  /streaming/api/stuck -> every unpublished recording and why it is stuck (any gated user)
+GET  /streaming/api/disk  -> disk use by streamer / clipper / trash by month, system dirs, Docker, and logs (ES indices
+             by month, journald, Docker json logs). Cached 10 min; ?refresh=1 rewalks.
+POST /streaming/api/stuck -> {"action":"retry"|"drop","src":...}: clear the blocker, or move it to trash with the
+             usual 3-day undo; {"action":"drop_under","mb":N} bins every stream fragment under N MB; {"action":"wipe"}
+             bins everything unprocessed (not being written). Admin groups only; every action runs bin/vgstuck.sh
+             so the web page and the CLI share one implementation.
 GET  /metrics -> Prometheus text for Metricbeat: the media scripts' counters (bin/vgmetrics.sh) plus live
              gauges for the normalise queue and MediaMTX. Loopback listener only; Caddy routes nothing here.
 POST /mtx-auth -> MediaMTX authHTTP hook: allows read/playback; allows publish iff user/pass carries a valid
@@ -32,10 +43,12 @@ POST /mtx-auth -> MediaMTX authHTTP hook: allows read/playback; allows publish i
 
 "Night" = everything the user recorded from 17:00 Eastern on <night> until 02:00 Eastern the next day.
 Default night: today if it is 17:00 or later, else yesterday (so 00:00-02:00 still counts as last night).
-Users may dump only their own prefix; members of the admins / media-admins groups may pass "user" to dump anyone, or "all".
+Users may dump only their own prefix; members of the admins / media-admins groups may pass "user" to dump anyone, or "all",
+but only through /streaming/api/dump (the diag triage page). On live.'s /dump everyone dumps their own night only.
 """
 import base64, hashlib, hmac, json, os, re, shutil, subprocess, sys, threading
 import time as _time
+import urllib.parse
 from datetime import datetime, timedelta, time
 from zoneinfo import ZoneInfo
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
@@ -51,7 +64,7 @@ PUBLISH_GROUPS = set(g.strip() for g in os.environ.get("PUBLISH_GROUPS", "stream
 AUTH_USER_HEADER = os.environ.get("AUTH_USER_HEADER", "X-Authentik-Username")
 AUTH_GROUPS_HEADER = os.environ.get("AUTH_GROUPS_HEADER", "X-Authentik-Groups")
 ADMIN_GROUPS = set(g.strip() for g in os.environ.get("DUMP_ADMIN_GROUPS", "admins,media-admins").split(",") if g.strip())
-LISTEN = ("127.0.0.1", 8080)                     # /dump, /me/* — only via Caddy (identity headers)
+LISTEN = ("127.0.0.1", 8080)                     # /dump, /streaming/api/*, /me/* — only via Caddy (identity headers)
 AUTH_LISTEN = (os.environ.get("MTX_AUTH_BIND", "0.0.0.0"), int(os.environ.get("MTX_AUTH_PORT", "8081")))   # /mtx-auth only — reachable from Docker's bridge (firewall), used by MediaMTX
 TZ = ZoneInfo(os.environ.get("DUMP_TZ", "America/New_York"))
 NIGHT_START = time(17, 0)   # 5 pm
@@ -429,20 +442,441 @@ def verify_token(token):
         return None
     return user.lower()
 
+
+
+# ---- stream health: B-frames ------------------------------------------------------------------------
+# MediaMTX refuses to serve H.264 with B-frames over WebRTC ("WebRTC doesn't support H264 streams with
+# B-frames"), so an OBS encoder left on its default of 2 B-frames is unwatchable on the portal. Nothing on the
+# publish path can see this: obs-websocket cannot set encoder settings in OBS's Advanced output mode, and
+# MediaMTX only notices when a viewer tries. The recording can: ffprobe reads the reorder depth off the first
+# fragments of the growing .live file in ~50 ms with a small probesize. Cached per path for 30 s.
+_health_cache = {}
+
+def stream_health(path):
+    """{'bframes', 'profile', 'res', 'fps', 'file'} for a live path, or None if nothing is on disk yet."""
+    now = _time.time()
+    hit = _health_cache.get(path)
+    if hit and now - hit["at"] < 30:
+        return hit["data"]
+    live_dir = ROOT / "streams" / path / ".live"
+    try:
+        newest = max((f for f in live_dir.glob("*.mp4")), key=lambda f: f.stat().st_mtime, default=None)
+    except OSError:
+        newest = None
+    data = None
+    if newest is not None:
+        # One probe, fields by name: csv column order follows ffprobe's internal field order, not the list.
+        done = subprocess.run(["ffprobe", "-v", "error", "-probesize", "2M", "-analyzeduration", "2M",
+                               "-select_streams", "v:0",
+                               "-show_entries", "stream=has_b_frames,profile,width,height,r_frame_rate",
+                               "-of", "default=noprint_wrappers=1", str(newest)],
+                              capture_output=True, text=True, timeout=10)
+        fields = dict(line.split("=", 1) for line in done.stdout.splitlines() if "=" in line)
+        if fields:
+            try:
+                num, den = (fields.get("r_frame_rate", "0/1").split("/") + ["1"])[:2]
+                fps = round(float(num) / float(den), 2) if float(den) else None
+            except (ValueError, ZeroDivisionError):
+                fps = None
+            data = {"bframes": int(fields.get("has_b_frames") or 0), "profile": fields.get("profile", ""),
+                    "res": f"{fields.get('width', '?')}x{fields.get('height', '?')}", "fps": fps, "file": newest.name}
+    _health_cache[path] = {"at": now, "data": data}
+    return data
+
+# ---- live streaming status ------------------------------------------------------------------------
+# GET /streaming/api/live: what is publishing, from where, how much it has lost on the way in, and why
+# MediaMTX has been opening recording files. Cached 5 s: the Live tab polls every 10 s and several admins
+# may have it open. Reads MediaMTX's API and metrics on loopback, and its container log for the counters.
+_live_cache = {"at": 0.0, "data": None}
+MTX_METRICS = os.environ.get("MTX_METRICS_URL", "http://127.0.0.1:9998/metrics")
+
+# WebRTC publish sessions and their transport. MediaMTX's API labels the selected ICE pair as
+# "<type>/<protocol>/<addr>/<port>" (host/udp/… or host/tcp/…), so the protocol a publisher is really on is
+# right there — that is how the Live tab tells a Go Live on TCP (the lossy-uplink switch) from one on UDP.
+# A session history is kept in MARK/webrtc-history.json so "when did he switch" survives the page being closed:
+# it is updated on every Live-tab poll AND every metrics scrape (Metricbeat, every few seconds), and a session
+# that is no longer listed is stamped ended. 24 h, 50 sessions per path.
+HISTORY_FILE = MARK / "webrtc-history.json"
+_history_lock = threading.Lock()
+
+def webrtc_sessions():
+    with urllib.request.urlopen(f"{MTX_API}/v3/webrtcsessions/list?itemsPerPage=1000", timeout=3) as response:
+        return json.load(response).get("items") or []
+
+def session_transport(item):
+    label = item.get("localCandidate") or item.get("remoteCandidate") or ""
+    parts = label.split("/")
+    return parts[1].lower() if len(parts) >= 2 and parts[1] else ""
+
+def track_sessions(items):
+    """Fold the current publish sessions into the history file; returns the history (path -> sessions)."""
+    now = datetime.now(TZ)
+    with _history_lock:
+        try:
+            hist = json.loads(HISTORY_FILE.read_text())
+        except (OSError, ValueError):
+            hist = {}
+        live_ids = set()
+        for it in items:
+            if str(it.get("state", "")).lower() != "publish":
+                continue
+            sid = it.get("id", ""); live_ids.add(sid)
+            rows = hist.setdefault(it.get("path", "?"), [])
+            row = next((r for r in rows if r["id"] == sid), None)
+            if row is None:
+                row = {"id": sid, "created": it.get("created"), "remote": it.get("remoteAddr"), "transport": "", "ended": None}
+                rows.append(row)
+            if session_transport(it): row["transport"] = session_transport(it)
+            row["received"] = it.get("rtpPacketsReceived", 0); row["lost"] = it.get("rtpPacketsLost", 0)
+            row["bytes"] = it.get("bytesReceived", 0); row["last_seen"] = now.isoformat(timespec="seconds")
+        cutoff = now - timedelta(hours=24)
+        for path in list(hist):
+            kept = []
+            for r in hist[path]:
+                if r["id"] not in live_ids and not r.get("ended"):
+                    r["ended"] = r.get("last_seen") or now.isoformat(timespec="seconds")
+                try:
+                    when = datetime.fromisoformat((r.get("ended") or r.get("last_seen") or "").replace("Z", "+00:00"))
+                    if when.tzinfo is None: when = when.replace(tzinfo=TZ)
+                except ValueError:
+                    when = now
+                if when >= cutoff: kept.append(r)
+            kept.sort(key=lambda r: r.get("created") or "")
+            hist[path] = kept[-50:]
+            if not hist[path]: del hist[path]
+        try:
+            MARK.mkdir(exist_ok=True)
+            tmp = HISTORY_FILE.with_suffix(".tmp"); tmp.write_text(json.dumps(hist)); os.replace(tmp, HISTORY_FILE)
+        except OSError as error:
+            log("history_write_failed", level="warn", error=str(error))
+        return hist
+
+def session_history():
+    """History as the page wants it: one list, newest first, with switch markers."""
+    try:
+        hist = json.loads(HISTORY_FILE.read_text())
+    except (OSError, ValueError):
+        return []
+    out = []
+    for path, rows in hist.items():
+        prev = None
+        for r in rows:
+            entry = {"path": path, **{k: r.get(k) for k in ("id", "created", "ended", "remote", "transport", "received", "lost", "bytes")}}
+            entry["id"] = (entry["id"] or "")[:8]
+            if prev is not None and prev.get("transport") and r.get("transport") and prev["transport"] != r["transport"]:
+                entry["switched_from"] = prev["transport"]
+            prev = r
+            out.append(entry)
+    out.sort(key=lambda e: e.get("created") or "", reverse=True)
+    return out
+
+# Turning a stream off from the Live tab. Two parts, because a kick alone is a hiccup: OBS reconnects on its
+# own within seconds and Go Live can be restarted with one click. So "stop" = kick the publish session through
+# MediaMTX's API AND put the streamer on a block list that /mtx-auth consults for every publish, for a chosen
+# number of minutes (0 = kick only). Blocks live in MARK/publish-blocks.json, survive a dumpd restart, expire on
+# their own, and the Live tab lists them with an unblock button.
+BLOCKS_FILE = MARK / "publish-blocks.json"
+_blocks_lock = threading.Lock()
+KICK_ENDPOINT = {"webrtc": "webrtcsessions", "rtmp": "rtmpconns", "srt": "srtconns", "rtsp": "rtspsessions"}
+
+def read_blocks():
+    try:
+        blocks = json.loads(BLOCKS_FILE.read_text())
+    except (OSError, ValueError):
+        blocks = {}
+    now = _time.time()
+    live = {u: b for u, b in blocks.items() if b.get("until", 0) > now}
+    if len(live) != len(blocks):
+        write_blocks(live)
+    return live
+
+def write_blocks(blocks):
+    try:
+        MARK.mkdir(exist_ok=True)
+        tmp = BLOCKS_FILE.with_suffix(".tmp"); tmp.write_text(json.dumps(blocks)); os.replace(tmp, BLOCKS_FILE)
+    except OSError as error:
+        log("blocks_write_failed", level="error", error=str(error))
+
+def block_user(user, minutes, by, reason=""):
+    with _blocks_lock:
+        blocks = read_blocks()
+        blocks[user] = {"until": _time.time() + minutes * 60, "by": by, "reason": reason,
+                        "since": datetime.now(TZ).isoformat(timespec="seconds")}
+        write_blocks(blocks)
+    log("publish_block", user=user, minutes=minutes, by=by)
+
+def unblock_user(user, by):
+    with _blocks_lock:
+        blocks = read_blocks()
+        if blocks.pop(user, None) is not None:
+            write_blocks(blocks); log("publish_unblock", user=user, by=by); return True
+    return False
+
+def blocked(user):
+    return read_blocks().get(user)
+
+def kick_publisher(kind, session_id):
+    endpoint = KICK_ENDPOINT.get(kind)
+    if not endpoint:
+        raise ValueError(f"cannot kick a {kind} publisher")
+    request = urllib.request.Request(f"{MTX_API}/v3/{endpoint}/kick/{urllib.parse.quote(session_id, safe='')}", method="POST")
+    with urllib.request.urlopen(request, timeout=5) as response:
+        return response.status
+
+def stop_stream(path, kind, session_id, minutes, who):
+    user = path.split("/", 1)[0].lower()
+    out = {"path": path, "user": user, "kicked": False, "blocked_minutes": minutes}
+    if minutes > 0:
+        block_user(user, minutes, who, reason=f"stopped from the Live tab ({path})")   # block first: no reconnect window
+    try:
+        kick_publisher(kind, session_id); out["kicked"] = True
+    except Exception as error:
+        out["error"] = f"kick failed: {error}"
+    # the Opus mirror of an OBS stream dies with its source (runOnReady exits), so nothing else to stop
+    _live_cache["data"] = None
+    log("stream_stop", level="info" if out["kicked"] else "error", stream=path, kind=kind, by=who, minutes=minutes,
+        result=out.get("error", "kicked"))
+    return out
+
+def live_status():
+    now = _time.time()
+    if _live_cache["data"] is not None and now - _live_cache["at"] < 5:
+        return _live_cache["data"]
+    out = {"paths": [], "publishers": [], "counts": {}, "log": [], "errors": [], "history": []}
+    sessions = {}
+    try:
+        items = webrtc_sessions()
+        track_sessions(items)
+        for it in items:
+            sessions[it.get("id", "")] = {"transport": session_transport(it), "since": it.get("created"),
+                                          "established": it.get("peerConnectionEstablished")}
+        out["history"] = session_history()
+    except Exception as error:
+        out["errors"].append(f"MediaMTX WebRTC sessions: {error}")
+    try:
+        with urllib.request.urlopen(f"{MTX_API}/v3/paths/list?itemsPerPage=1000", timeout=3) as response:
+            for item in json.load(response).get("items") or []:
+                if item.get("ready"):
+                    row = {"name": item["name"], "tracks": item.get("tracks") or [],
+                           "readers": len(item.get("readers") or []),
+                           "since": item.get("readyTime"), "bytes": item.get("bytesReceived", 0),
+                           "mirror": str(item["name"]).endswith(".webrtc")}
+                    if not row["mirror"]:
+                        try:
+                            row["health"] = stream_health(item["name"])
+                        except Exception as error:
+                            row["health_error"] = str(error)
+                    out["paths"].append(row)
+    except Exception as error:
+        out["errors"].append(f"MediaMTX API: {error}")
+    try:
+        with urllib.request.urlopen(MTX_METRICS, timeout=3) as response:
+            text = response.read().decode(errors="replace")
+        rows = {}
+        for match in re.finditer(r"^(webrtc_sessions|rtmp_conns|srt_conns)(_rtp_packets_received|_rtp_packets_lost|_bytes_received)?\{([^}]*)\} (\d+)$", text, re.M):
+            kind, field, labels_text, value = match.groups()
+            labels = dict(re.findall(r'(\w+)="([^"]*)"', labels_text))
+            if labels.get("state") != "publish":
+                continue
+            key = (labels.get("path", "?"), labels.get("id", ""), labels.get("remoteAddr", "?"))
+            row = rows.setdefault(key, {"kind": kind.split("_")[0]})
+            if field == "_rtp_packets_received": row["received"] = int(value)
+            elif field == "_rtp_packets_lost": row["lost"] = int(value)
+            elif field == "_bytes_received": row["bytes"] = int(value)
+        for (path, sid, addr), row in rows.items():
+            out["publishers"].append({"path": path, "id": sid[:8], "session": sid, "from": addr, **row, **sessions.get(sid, {})})
+        out["blocks"] = [{"user": u, **b, "until_iso": datetime.fromtimestamp(b["until"], TZ).isoformat(timespec="seconds")}
+                         for u, b in sorted(read_blocks().items())]
+    except Exception as error:
+        out["errors"].append(f"MediaMTX metrics: {error}")
+    try:
+        done = subprocess.run(["docker", "logs", "mediamtx", "--since", "10m"],
+                              capture_output=True, text=True, timeout=15)
+        log_text = done.stdout + done.stderr
+        out["counts"] = {
+            "publish_sessions": log_text.count("is publishing"),
+            "recorder_restarts": len(re.findall(r"recording \d+ tracks", log_text)),
+            "processing_errors": log_text.count("processing errors"),
+            "access_unit_errors": log_text.count("recorder] access unit"),
+            "window": "10m",
+        }
+        out["log"] = [l for l in log_text.splitlines() if re.search(r"recorder|is publishing|closed|ERR", l)][-14:]
+    except Exception as error:
+        out["errors"].append(f"MediaMTX log: {error}")
+    _live_cache.update(at=now, data=out)
+    return out
+
+# ---- disk report ------------------------------------------------------------------------------------
+# GET /streaming/api/disk: where the bytes are. Streamers (streams/<user>, month from the recording's own
+# "__YYYY-MM-DD" stamp, else mtime), clippers (clips/<user>, mtime), trash (its night folders), system (appdata
+# per app, /srv/live, /srv/bot, Docker's own accounting) and logs (Elasticsearch indices by the date in their
+# name, the ES data dir as the on-disk truth, journald, Docker's json logs). Walking a few TB of files takes
+# seconds, so the report is cached for ten minutes and ?refresh=1 forces it. Sizes are allocated blocks, i.e.
+# what the disk actually loses, not apparent length.
+_disk_cache = {"at": 0.0, "data": None}
+_disk_lock = threading.Lock()
+DISK_CACHE_S = 600
+APPDATA = Path("/srv/appdata")
+ES_NETRC = Path(os.environ.get("ES_NETRC_FILE", "/etc/vanguard/es-netrc"))
+ES_URL = os.environ.get("ES_URL", "http://127.0.0.1:9200")
+MONTH_IN_NAME = re.compile(r"__(\d{4})-(\d{2})-\d{2}_")
+MONTH_IN_INDEX = re.compile(r"(\d{4})\.(\d{2})\.\d{2}")
+
+def _blocks(st):
+    return st.st_blocks * 512 if hasattr(st, "st_blocks") else st.st_size
+
+def walk_sizes(root, month_of):
+    """{month: bytes} and total for everything under root; month_of(path, stat) -> 'YYYY-MM'."""
+    months, total = {}, 0
+    stack = [root]
+    while stack:
+        current = stack.pop()
+        try:
+            with os.scandir(current) as entries:
+                for entry in entries:
+                    try:
+                        if entry.is_symlink(): continue
+                        if entry.is_dir(follow_symlinks=False): stack.append(entry.path); continue
+                        st = entry.stat(follow_symlinks=False)
+                    except OSError:
+                        continue
+                    size = _blocks(st); total += size
+                    month = month_of(entry.path, st)
+                    months[month] = months.get(month, 0) + size
+        except OSError:
+            continue
+    return {"total": total, "months": months}
+
+def month_from_mtime(_path, st):
+    return datetime.fromtimestamp(st.st_mtime, TZ).strftime("%Y-%m")
+
+def month_from_name(path, st):
+    m = MONTH_IN_NAME.search(os.path.basename(path))
+    return f"{m.group(1)}-{m.group(2)}" if m else month_from_mtime(path, st)
+
+def dir_total(path):
+    return walk_sizes(path, lambda p, s: "all")["total"] if Path(path).is_dir() else 0
+
+def es_indices():
+    """[{index, bytes, month}] from _cat/indices, with the netrc the deadman also uses."""
+    import netrc as _netrc
+    request = urllib.request.Request(f"{ES_URL}/_cat/indices?bytes=b&format=json&h=index,store.size,creation.date.string&expand_wildcards=all")
+    try:
+        creds = _netrc.netrc(str(ES_NETRC)).authenticators(urllib.parse.urlparse(ES_URL).hostname or "127.0.0.1")
+    except (OSError, _netrc.NetrcParseError):
+        creds = None
+    if creds:
+        request.add_header("Authorization", "Basic " + base64.b64encode(f"{creds[0]}:{creds[2]}".encode()).decode())
+    with urllib.request.urlopen(request, timeout=10) as response:
+        rows = json.load(response)
+    out = []
+    for row in rows:
+        name = row.get("index", "")
+        m = MONTH_IN_INDEX.search(name)
+        month = f"{m.group(1)}-{m.group(2)}" if m else (row.get("creation.date.string") or "")[:7] or "other"
+        out.append({"index": name, "bytes": int(row.get("store.size") or 0), "month": month})
+    return out
+
+def docker_df():
+    done = subprocess.run(["docker", "system", "df", "--format", "{{json .}}"], capture_output=True, text=True, timeout=60)
+    rows = []
+    for line in done.stdout.splitlines():
+        try: rows.append(json.loads(line))
+        except ValueError: pass
+    return rows
+
+def journal_bytes():
+    done = subprocess.run(["journalctl", "--disk-usage"], capture_output=True, text=True, timeout=20)
+    m = re.search(r"([\d.]+)\s*([KMGT]?)B?", done.stdout.replace(",", ""))
+    if not m: return 0
+    return int(float(m.group(1)) * {"": 1, "K": 1e3, "M": 1e6, "G": 1e9, "T": 1e12}[m.group(2)])
+
+def disk_report(force=False):
+    now = _time.time()
+    with _disk_lock:
+        if not force and _disk_cache["data"] is not None and now - _disk_cache["at"] < DISK_CACHE_S:
+            return _disk_cache["data"]
+        t0 = _time.time()
+        out = {"generated": datetime.now(TZ).isoformat(timespec="seconds"), "fs": {}, "streamers": {}, "clippers": {},
+               "trash": {}, "system": {}, "logs": {}, "errors": []}
+        for label, path in (("media", ROOT), ("root", Path("/"))):
+            try:
+                u = shutil.disk_usage(path); out["fs"][label] = {"path": str(path), "total": u.total, "used": u.used, "free": u.free}
+            except OSError as error:
+                out["errors"].append(f"{path}: {error}")
+        same_fs = False
+        try: same_fs = os.stat(ROOT).st_dev == os.stat("/").st_dev
+        except OSError: pass
+        out["fs"]["same_filesystem"] = same_fs
+        for user_dir in sorted(p for p in (ROOT / "streams").glob("*") if p.is_dir()):
+            r = walk_sizes(user_dir, month_from_name)
+            r["raw"] = dir_total(user_dir / ".src"); r["unpublished"] = sum(dir_total(d) for d in user_dir.glob("*/.live"))
+            out["streamers"][user_dir.name] = r
+        for user_dir in sorted(p for p in (ROOT / "clips").glob("*") if p.is_dir()):
+            out["clippers"][user_dir.name] = walk_sizes(user_dir, month_from_mtime)
+        out["trash"] = walk_sizes(ROOT / "trash", lambda p, s: (re.search(r"/trash/(\d{4}-\d{2})-\d{2}/", p) or [None, month_from_mtime(p, s)])[1]) if (ROOT / "trash").is_dir() else {"total": 0, "months": {}}
+        other_media = 0
+        for entry in ROOT.iterdir():
+            if entry.name not in ("streams", "clips", "trash") and entry.is_dir() and not entry.is_symlink():
+                other_media += dir_total(entry)
+        out["system"]["media_other"] = other_media                       # .markers, .syncthing, .stversions, …
+        for app_dir in sorted(p for p in APPDATA.glob("*") if p.is_dir()) if APPDATA.is_dir() else []:
+            size = dir_total(app_dir)
+            if app_dir.name == "elasticsearch": out["logs"]["es_data_dir"] = size
+            else: out["system"][f"appdata/{app_dir.name}"] = size
+        for label, path in (("live portal", "/srv/live"), ("bot", "/srv/bot"), ("etc/vanguard", "/etc/vanguard")):
+            out["system"][label] = dir_total(path)
+        try:
+            for row in docker_df():
+                out["system"][f"docker {row.get('Type', '?').lower()}"] = row.get("Size", "?")   # docker's own figures (human strings)
+        except Exception as error:
+            out["errors"].append(f"docker system df: {error}")
+        try:
+            out["logs"]["indices"] = es_indices()
+            out["logs"]["es_total"] = sum(i["bytes"] for i in out["logs"]["indices"])
+            by_month = {}
+            for i in out["logs"]["indices"]: by_month[i["month"]] = by_month.get(i["month"], 0) + i["bytes"]
+            out["logs"]["months"] = by_month
+        except Exception as error:
+            out["errors"].append(f"Elasticsearch: {error}")
+        try: out["logs"]["journald"] = journal_bytes()
+        except Exception as error: out["errors"].append(f"journalctl: {error}")
+        out["logs"]["docker_json_logs"] = sum(_blocks(p.stat()) for p in Path("/var/lib/docker/containers").glob("*/*-json.log*")) if Path("/var/lib/docker/containers").is_dir() else 0
+        out["seconds"] = round(_time.time() - t0, 1)
+        _disk_cache.update(at=_time.time(), data=out)
+        return out
+
 # ---- stuck recordings -----------------------------------------------------------------------------
-# GET  /stuck/api   every unpublished recording and why, for anyone the gate let through
+# GET  /streaming/api/stuck   every unpublished recording and why, for anyone the gate let through
-# POST /stuck/api   {"action":"retry"|"drop","src":"<path>"} — admins only
+# POST /streaming/api/stuck   {"action":"retry"|"drop","src":"<path>"} — admins only
 # The classification and the file moves both live in bin/vgstuck.sh, so the web page and the CLI can never
 # disagree about why something is stuck, and "delete" means the same three-day trash in both.
 STUCK_TOOL = os.environ.get("VGSTUCK_TOOL", "/usr/local/bin/vgstuck.sh")
 
+_stuck_cache = {"at": 0.0, "items": None}
+STUCK_CACHE_S = 20          # the page refreshes every 30 s and several admins may have it open
+
 def stuck_list():
-    done = subprocess.run([STUCK_TOOL, "json"], capture_output=True, text=True, timeout=180)
+    """The survey, cached briefly. It runs ffprobe on every stuck file, so with many of them it is slow, and
+    without a cache every open tab re-ran it every 30 s. The cache is also invalidated by any action."""
+    now = _time.time()
+    if _stuck_cache["items"] is not None and now - _stuck_cache["at"] < STUCK_CACHE_S:
+        return _stuck_cache["items"]
     try:
-        return json.loads(done.stdout or "[]")
+        done = subprocess.run([STUCK_TOOL, "json"], capture_output=True, text=True, timeout=120)
-    except ValueError:
+    except subprocess.TimeoutExpired:
-        log("stuck_list_failed", level="error", rc=done.returncode, err=(done.stderr or "")[:200])
+        log("stuck_list_timeout", level="error", seconds=120)
-        return []
+        raise RuntimeError("the survey took over 120 s; too many files, or ffprobe is stuck") from None
+    if done.returncode != 0:
+        log("stuck_list_failed", level="error", rc=done.returncode, err=(done.stderr or "")[:300])
+        raise RuntimeError(f"vgstuck.sh exited {done.returncode}: {(done.stderr or '').strip()[-200:]}")
+    try:
+        items = json.loads(done.stdout or "[]")
+    except ValueError as error:
+        log("stuck_list_badjson", level="error", err=str(error), head=(done.stdout or "")[:120])
+        raise RuntimeError(f"vgstuck.sh did not return JSON: {error}") from None
+    _stuck_cache.update(at=now, items=items)
+    return items
 
 def stuck_target(src):
     """Resolve what the browser asked us to act on, or raise. This is the only thing between a request body
@@ -461,6 +895,7 @@ def stuck_target(src):
 def stuck_action(action, src, who):
     path = stuck_target(src)
     done = subprocess.run([STUCK_TOOL, action, str(path)], capture_output=True, text=True, timeout=600)
+    _stuck_cache["items"] = None                    # the list just changed; the next read must re-survey
     lines = [l for l in (done.stdout + done.stderr).splitlines() if l.strip()]
     log(f"stuck_{action}", level="info" if done.returncode == 0 else "error",
         file=str(path), by=who, rc=done.returncode)
@@ -579,6 +1014,10 @@ def metrics_text():
     # 4. MediaMTX. The Prometheus scrape of :9998 already has per-path detail; this is the count the portal
     #    and the dock actually mean by "live", with the derived .webrtc audio mirrors separated out.
     paths = mtx_paths()
+    try:
+        track_sessions(webrtc_sessions())          # keeps the transport history current between page views
+    except Exception:
+        pass
     add("vanguard_mediamtx_up", "gauge", "did the MediaMTX API answer this scrape", {}, 0 if paths is None else 1)
     if paths is not None:
         ready = [p for p in paths if p.get("ready")]
@@ -621,6 +1060,11 @@ class H(BaseHTTPRequestHandler):
     def _parse(self):
         n = int(self.headers.get("Content-Length") or 0)
         body = json.loads(self.rfile.read(n) or b"{}") if n else {}
+        # ?user=all&night=YYYY-MM-DD also work (the Dump tab's dry run). GET never moves anything: a link
+        # unfurled by Discord must not be able to dump a night.
+        query = urllib.parse.parse_qs(urllib.parse.urlparse(self.path).query)
+        for key in ("night", "user"):
+            if key not in body and query.get(key): body[key] = query[key][0]
         night = body.get("night")
         if night:
             datetime.strptime(night, "%Y-%m-%d")  # validate
@@ -637,8 +1081,15 @@ class H(BaseHTTPRequestHandler):
             if pf:
                 return prev, pf
         return n, files
+    def _is_dump(self):
+        path = self.path.split("?", 1)[0]
+        return path in ("/dump", "/streaming/api/dump")       # live. dock button / diag. Dump tab
+    def _admin_door(self):
+        return self.path.split("?", 1)[0] == "/streaming/api/dump"
     def _target(self, auth_user, is_admin, req_user):
-        if is_admin:
+        # Dumping someone else's night is triage, so it is only offered where triage lives: the diag host's
+        # streaming page. The dock's Dump tonight (live. /dump) is personal for everyone, admins included.
+        if is_admin and self._admin_door():
             return req_user or auth_user          # admins default to their own night; pass "all" or a name for others
         return auth_user
     def do_GET(self):
@@ -660,10 +1111,51 @@ class H(BaseHTTPRequestHandler):
             return self._json(200, {"user": who, "groups": groups, "admin": is_admin,
                                     "name": self.headers.get("X-Authentik-Name") or who, "email": self.headers.get("X-Authentik-Email") or "",
                                     "discord_id": proxy_claim(self.headers, "discord_id")})
-        if self.path == "/stuck/api":
+        if self.path == "/me/stream-health":
+            # The dock polls this after Go Live. Only the caller's own paths, and only what is live now.
+            who, _ = identity(self.headers)
+            if not who: return self._json(401, {"error": "not authenticated"})
+            streams = []
+            try:
+                with urllib.request.urlopen(f"{MTX_API}/v3/paths/list?itemsPerPage=1000", timeout=3) as response:
+                    for item in json.load(response).get("items") or []:
+                        name = str(item.get("name", ""))
+                        if item.get("ready") and name.lower().startswith(who.lower() + "/") and not name.endswith(".webrtc"):
+                            health = stream_health(name)
+                            streams.append({"path": name, "tracks": item.get("tracks") or [], **(health or {}),
+                                            "webrtc_ok": bool(health) and health["bframes"] == 0})
+            except Exception as error:
+                return self._json(502, {"error": f"MediaMTX: {error}"})
+            # Go Live polls this while live: an empty list means the server no longer has the stream (kicked, or
+            # the session died), and "blocked" says why it must not simply reconnect.
+            hold = blocked(who.lower())
+            return self._json(200, {"user": who, "streams": streams,
+                                    "blocked": ({"until": datetime.fromtimestamp(hold["until"], TZ).isoformat(timespec="seconds"),
+                                                 "by": hold.get("by")} if hold else None)})
+        if self.path.split("?", 1)[0] == "/streaming/api/disk":
             who, is_admin = identity(self.headers)
             if not who: return self._json(401, {"error": "not authenticated"})
-            return self._json(200, {"admin": is_admin, "user": who, "items": stuck_list()})
+            force = "refresh=1" in (self.path.split("?", 1) + [""])[1]
+            try:
+                return self._json(200, {"admin": is_admin, "user": who, **disk_report(force)})
+            except Exception as error:
+                return self._json(500, {"error": str(error)})
+        if self.path == "/streaming/api/live":
+            who, is_admin = identity(self.headers)
+            if not who: return self._json(401, {"error": "not authenticated"})
+            try:
+                return self._json(200, {"admin": is_admin, "user": who, **live_status()})
+            except Exception as error:
+                return self._json(500, {"error": str(error)})
+        if self.path == "/streaming/api/stuck":
+            who, is_admin = identity(self.headers)
+            if not who: return self._json(401, {"error": "not authenticated"})
+            try:
+                return self._json(200, {"admin": is_admin, "user": who, "items": stuck_list()})
+            except Exception as error:
+                # Always answer. A handler that raises sends nothing, Caddy turns that into an empty 502,
+                # and the page can only say "unexpected end of JSON" — which is exactly what happened.
+                return self._json(500, {"error": str(error)})
         if self.path.startswith("/me/clips"):
             who, _ = identity(self.headers)
             if not who: return self._json(401, {"error": "not authenticated"})
@@ -682,17 +1174,24 @@ class H(BaseHTTPRequestHandler):
                 return self._json(500, {"error": str(e)})
             # "pass" kept for the dock/portal: the token IS the password for MediaMTX
             return self._json(200, {"user": who, "pass": tok, "token": tok, "expires": exp, "ttl_hours": TOKEN_TTL_H})
-        if not self.path.startswith("/dump"):
+        if not self._is_dump():
             return self._json(404, {"error": "not found"})
         u, is_admin = identity(self.headers)
         if not u:
             return self._deny()
-        night, req_user = self._parse()
+        try:
+            night, req_user = self._parse()
+        except Exception as e:
+            return self._json(400, {"error": str(e)})
         tgt = self._target(u, is_admin, req_user)
         night, files = self._resolve(night, tgt)
-        self._json(200, {"night": night, "user": tgt, "dry_run": True,
+        start, end = night_window(night)
-                         "files": [str(f.relative_to(ROOT)) for f in files],
+        entries = [{"path": str(f.relative_to(ROOT)), "bytes": f.stat().st_size} for f in files]
-                         "gb": round(sum(f.stat().st_size for f in files) / 1e9, 2)})
+        self._json(200, {"night": night, "user": tgt, "by": u, "admin": is_admin, "dry_run": True,
+                         "window": [start.isoformat(timespec="minutes"), end.isoformat(timespec="minutes")],
+                         "default_night": default_night(),
+                         "files": [e["path"] for e in entries], "entries": entries,
+                         "gb": round(sum(e["bytes"] for e in entries) / 1e9, 2)})
     def do_POST(self):
         if self.server.server_port == AUTH_LISTEN[1] and self.path.startswith("/heartbeat"):
             # Alertmanager posts here every minute because the Watchdog alert always fires. Nothing is read from
@@ -776,7 +1275,7 @@ class H(BaseHTTPRequestHandler):
             path = (req.get("path") or "").lower()
             if path.endswith(".webrtc"):
                 # audio mirrors (<user>/<Title>.webrtc) are published by the hook inside the mediamtx container only
-                if req.get("ip") in ("127.0.0.1", "::1") and re.fullmatch(r"[a-z0-9_.-]+/[a-z0-9_.-]+\.webrtc", path):
+                if req.get("ip") in ("127.0.0.1", "::1") and re.fullmatch(r"[a-z0-9_.-]+/[a-z0-9_-]+\.webrtc", path):
                     return self._json(200, {"ok": True})
                 log("publish_deny", level="warn", stream=path, ip=req.get("ip"), reason="mirror paths are internal")
                 return self._json(401, {"error": ".webrtc paths are reserved"})
@@ -787,12 +1286,43 @@ class H(BaseHTTPRequestHandler):
             if not user:
                 log("publish_deny", level="warn", stream=path, ip=req.get("ip"), reason="bad or expired token")
                 return self._json(401, {"error": "invalid or expired stream token"})
-            if not re.fullmatch(r"[a-z0-9_.-]+/[a-z0-9_.-]+", path) or not path.startswith(user + "/"):
+            # title segment is dot-free (mirrors are <title>.webrtc and must never collide with a title)
+            if not re.fullmatch(r"[a-z0-9_.-]+/[a-z0-9_-]+", path) or not path.startswith(user + "/"):
                 log("publish_deny", level="warn", user=user, stream=path, reason="not their prefix")
                 return self._json(401, {"error": "path must be <your-username>/<Title>"})
+            hold = blocked(user.lower())
+            if hold:                                             # stopped from the Live tab: no reconnecting around it
+                log("publish_deny", level="warn", user=user, stream=path, ip=req.get("ip"), reason="blocked", by=hold.get("by"))
+                return self._json(401, {"error": "publishing is paused for this account by an admin"})
             log("publish_allow", user=user, stream=path, ip=req.get("ip"))
             return self._json(200, {"ok": True})
-        if self.path == "/stuck/api":
+        if self.path == "/streaming/api/live":
+            # {"action":"stop","path":..,"kind":..,"session":..,"minutes":N} or {"action":"unblock","user":..} — admins only
+            who, is_admin = identity(self.headers)
+            if not who: return self._deny()
+            if not is_admin:
+                return self._json(403, {"error": f"stopping streams is for {', '.join(sorted(ADMIN_GROUPS))}"})
+            try:
+                body = json.loads(self.rfile.read(int(self.headers.get("Content-Length") or 0)) or b"{}")
+            except ValueError as error:
+                return self._json(400, {"error": str(error)})
+            action = body.get("action")
+            if action == "unblock":
+                user = str(body.get("user", "")).lower()
+                if not re.fullmatch(r"[a-z0-9_.-]+", user): return self._json(400, {"error": "bad user"})
+                _live_cache["data"] = None
+                return self._json(200, {"ok": unblock_user(user, who), "user": user})
+            if action == "stop":
+                path, kind, session = str(body.get("path", "")), str(body.get("kind", "")), str(body.get("session", ""))
+                if not re.fullmatch(r"[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+", path) or kind not in KICK_ENDPOINT or not re.fullmatch(r"[A-Za-z0-9-]{8,64}", session):
+                    return self._json(400, {"error": "need path, kind and session as the live list reports them"})
+                try:
+                    minutes = max(0, min(int(body.get("minutes", 30)), 24 * 60))
+                except (TypeError, ValueError):
+                    minutes = 30
+                return self._json(200, stop_stream(path, kind, session, minutes, who))
+            return self._json(400, {"error": "action must be stop or unblock"})
+        if self.path == "/streaming/api/stuck":
             who, is_admin = identity(self.headers)
             if not who: return self._deny()
             if not is_admin:
@@ -802,8 +1332,33 @@ class H(BaseHTTPRequestHandler):
             except ValueError as error:
                 return self._json(400, {"error": str(error)})
             action = body.get("action")
+            if action == "wipe":
+                # Everything unprocessed, deliberately (the Dump tab's button): stops the sweep, then every .live/
+                # file except one MediaMTX still has open, plus stale failure logs and .part leftovers. vgstuck.sh
+                # does the moving, into trash.
+                done = subprocess.run([STUCK_TOOL, "wipe"], capture_output=True, text=True, timeout=900)
+                _stuck_cache["items"] = None
+                lines = [l for l in (done.stdout + done.stderr).splitlines() if l.strip()]
+                log("stuck_wipe", level="info" if done.returncode == 0 else "error", by=who, rc=done.returncode,
+                    result=(lines[-1] if lines else ""))
+                return self._json(200, {"ok": done.returncode == 0, "rc": done.returncode, "output": lines[-6:]})
+            if action == "drop_under":
+                # Bulk: every unpublished STREAM recording smaller than N MB — reconnect stubs. Bounded so a
+                # typo cannot bin real recordings; clips never appear here because they never pass through
+                # .live/. Still-recording files are skipped by the tool itself.
+                try:
+                    mb = int(body.get("mb", 0))
+                except (TypeError, ValueError):
+                    mb = 0
+                if not 1 <= mb <= 50:
+                    return self._json(400, {"error": "mb must be between 1 and 50"})
+                done = subprocess.run([STUCK_TOOL, "drop", "--under", str(mb)], capture_output=True, text=True, timeout=900)
+                _stuck_cache["items"] = None
+                lines = [l for l in (done.stdout + done.stderr).splitlines() if l.strip()]
+                log("stuck_drop_under", level="info" if done.returncode == 0 else "error", mb=mb, by=who, rc=done.returncode)
+                return self._json(200, {"ok": done.returncode == 0, "rc": done.returncode, "output": lines[-6:]})
             if action not in ("retry", "drop"):
-                return self._json(400, {"error": "action must be retry or drop"})
+                return self._json(400, {"error": "action must be retry, drop, drop_under or wipe"})
             try:
                 return self._json(200, stuck_action(action, str(body.get("src", "")), who))
             except ValueError as error:
@@ -821,7 +1376,7 @@ class H(BaseHTTPRequestHandler):
                 return self._json(400, {"error": str(e)})
             except Exception as e:
                 return self._json(502, {"error": f"syncthing: {e}"})
-        if not self.path.startswith("/dump"):
+        if not self._is_dump():
             return self._json(404, {"error": "not found"})
         u, is_admin = identity(self.headers)
         if not u:

bin/install.list

@@ -7,6 +7,8 @@
 0755 bin/trash-purge.sh
 0755 bin/dumpd.py
 0755 bin/vgstrip.py
+0755 bin/vgstitch.py
+0755 bin/vghosts-check.py
 0755 bin/vgdeband.sh
 0755 bin/vgstuck.sh
 0755 bin/vanguard-deadman.sh

bin/normalize-sweep.sh

@@ -16,6 +16,12 @@
 #   alone (zero copies); a non-compliant file is converted to name.mp4 (name.h264.mp4 if the raw is
 #   already .mp4), the raw's name is appended to .stlocal (an #include'd ignore list) so Syncthing
 #   drops it without flagging or re-pulling, then the raw is deleted. One copy on the server.
+# - reconnect fragments of ONE stream (same user, same title, fewer than STITCH_MAX_PARTS (500) files, gaps under
+#   STITCH_MAX_GAP_S) are first stitched back into one file by vgstitch.py (lossless concat; the arrival-time
+#   file names say where every piece sits), then published as one recording. Joins are recorded in the sidecar.
+#   Nothing in a stream's .live/ is touched until the stream is over (no open or recently written file).
+# - a LONE stream recording under MIN_LIVE_SECONDS (10) is a reconnect stub with nothing to join to: moved to
+#   trash, never published. Only .live/ files — clips are frequently a few seconds long and are never subject to this.
 # - failure: name.normalize-failed.log is written and the file is skipped until that log is deleted
 # Idempotent; safe every few minutes.
 set -uo pipefail
@@ -32,6 +38,18 @@ KEEP_RAW_STREAMS="${KEEP_RAW_STREAMS:-0}"
 # Encoder threads for every re-encode (clips and streams). 2 leaves the box responsive for Jellyfin/live viewers;
 # conversions take longer. systemd Environment=ENCODE_THREADS=... to change.
 ENCODE_THREADS="${ENCODE_THREADS:-2}"
+# STREAM recordings shorter than this are reconnect stubs, not recordings: every time a publisher drops and
+# comes back, MediaMTX opens a new file, and one bad night left 471 one- and two-second files in .live/.
+# They go to trash (3-day undo) instead of being published. Applies ONLY to .live/ — clips are often a few
+# seconds long on purpose and are never touched by this. 0 disables. systemd Environment=MIN_LIVE_SECONDS=…
+MIN_LIVE_SECONDS="${MIN_LIVE_SECONDS:-10}"
+# Stitching (vgstitch.py): fewer than this many fragments in one stream's .live/ are joined into one recording.
+# 500, not 20: on a lossy uplink MediaMTX restarts the recorder every ~10 s (a keyframe whose slices were lost
+# leaves an access unit with only SPS/PPS, which the recorder refuses), so a half-hour stream is ~180 same-
+# parameter files two seconds apart — exactly the case for stitching. The gap rule (STITCH_MAX_GAP_S seconds
+# between fragments = separate recordings) is the real guard. 0 parts disables stitching. systemd Environment=…
+STITCH_MAX_PARTS="${STITCH_MAX_PARTS:-500}"
+STITCH_MAX_GAP_S="${STITCH_MAX_GAP_S:-600}"
 MAX_W=2560; MAX_H=1440; MAX_FPS=60
 exec 9>"${LOCK_FILE:-/run/lock/normalize-sweep.lock}"
 flock -n 9 || exit 0
@@ -50,6 +68,11 @@ vglog() { local lvl="$1"; shift; printf 'ts=%s level=%s component=sweep %s\n' "$
 log() { vglog info "msg=\"$*\""; }
 dur() { ffprobe -v error -show_entries format=duration -of csv=p=0 "$1" 2>/dev/null | cut -d. -f1; }
 
+# A stream's fragments are one recording: join them before anything below looks at them one by one.
+if (( STITCH_MAX_PARTS > 0 )) && [[ -r "$(dirname "$0")/vgstitch.py" ]]; then        # installed side by side
+  python3 "$(dirname "$0")/vgstitch.py" "$ROOT" --max-parts "$STITCH_MAX_PARTS" --max-gap-s "$STITCH_MAX_GAP_S" --idle-min "$MINS_IDLE"
+fi
+
 # --- probe: prints "verdict reason" where verdict ∈ ok|remux|encode|skip ----------------------
 probe() {
   local f="$1" ext="${1##*.}"
@@ -123,6 +146,31 @@ while IFS= read -r -d '' src; do
   fi
   fuser -s "$src" 2>/dev/null && continue            # still being written
   [[ -e "$src" ]] || continue                          # dumped meanwhile
+  if (( live )); then
+    # The stream is still going while any sibling fragment is open or fresh: publishing this piece now would
+    # make it a separate recording that the stitcher can never join. Wait for the stream to end.
+    going=0
+    for sib in "$(dirname "$src")"/*.mp4; do
+      [[ -e $sib && $sib != "$src" ]] || continue
+      if fuser -s "$sib" 2>/dev/null || [[ -n $(find "$sib" -mmin -"$MINS_IDLE" 2>/dev/null) ]]; then going=1; break; fi
+    done
+    if (( going )); then vglog info "event=defer reason=\"stream still going\" file=\"$src\" stream=\"$stream\""; continue; fi
+  fi
+
+  if (( live && MIN_LIVE_SECONDS > 0 )); then
+    secs=$(dur "$src"); secs=${secs:-0}
+    if (( secs < MIN_LIVE_SECONDS )); then
+      night="$(date +%F)"; rel="${src#"$ROOT"/}"; dst="$ROOT/trash/$night/$rel"
+      mkdir -p "$(dirname "$dst")"
+      mv -f "$src" "$dst"
+      [[ -e "${src%.*}.json" ]] && mv -f "${src%.*}.json" "$(dirname "$dst")/"
+      rm -f "${src%.*}.stitch.json"
+      rmdir "$(dirname "$src")" "$titledir" 2>/dev/null || true
+      vgmet_add "jobs:discard:done:stream"
+      vglog info "event=discarded reason=too_short seconds=$secs min=$MIN_LIVE_SECONDS file=\"$src\" stream=\"$stream\" trash=\"$dst\""
+      continue
+    fi
+  fi
 
   read -r verdict reason <<<"$(probe "$src")"
   # Keep what the PROBE said: the deband branch below overwrites $reason with its own text, which silently
@@ -220,8 +268,19 @@ PYEOF
     (( deband )) && vgmet_add deband 1
     if (( live )); then
       mv -f "$tmp" "$out"; chown 1000:1000 "$out" 2>/dev/null
-      if [[ -n $tcjson ]]; then                        # sidecar: exact capture time (read by the Jellyfin plugin) + drift samples
+      stitchjson="${src%.*}.stitch.json"               # vgstitch.py: parts, offsets and the time missing at each join
-        printf '%s\n' "$tcjson" > "${out%.*}.json"; chown 1000:1000 "${out%.*}.json" 2>/dev/null
+      if [[ -n $tcjson || -e $stitchjson ]]; then      # sidecar: exact capture time (read by the Jellyfin plugin) + drift samples
+        python3 - "$tcjson" "$stitchjson" > "${out%.*}.json" <<'PYEOF'
+import json, sys, os
+tc, sj = sys.argv[1], sys.argv[2]
+d = json.loads(tc) if tc else {}
+if os.path.exists(sj):
+    d["stitched"] = json.load(open(sj))
+    # the strip decode's drift figure spans the joins too, so it is not clock drift on a stitched file
+    if "drift_ms" in d: d["drift_ms_note"] = "includes collapsed joins; see stitched.joins"
+print(json.dumps(d))
+PYEOF
+        chown 1000:1000 "${out%.*}.json" 2>/dev/null; rm -f "$stitchjson"
       fi
       if [[ $verdict == encode ]] && (( ! deband || KEEP_RAW_STREAMS )); then   # gap-filled re-encode (or asked to): keep the raw in the hidden .src/
         srcdir="$(dirname "$out")/.src"; mkdir -p "$srcdir"; chown 1000:1000 "$srcdir" 2>/dev/null

bin/vghosts-check.py

@@ -0,0 +1,407 @@
+#!/usr/bin/env python3
+"""Validate a hostname change end to end: what is on disk, what is running, what the UIs were told, what DNS says.
+
+    sudo vghosts-check.py            on the server: everything
+    vghosts-check.py --repo          on a dev box: only the checks that need the repo (defaults, docs, pages)
+    sudo vghosts-check.py --json     machine-readable
+
+Source of truth is the Caddyfile (`# vanguard:<role>` markers), the same place configure.py reads the hostnames
+back from. Every other place a hostname lives is checked AGAINST it: a host that is not one of Caddy's is stale.
+"ovh1.<base>" is the box's ssh name and must appear nowhere a browser is sent.
+
+ORDER MATTERS, and the sections below run in the order a change must be made, because each depends on the one
+before it (HOSTNAME-CHANGE.md, section E):
+   1 DNS           nothing can be issued a certificate or reached until the name resolves to the box
+   2 Caddyfile     the roles and their names: what configure.py --full / --hosts wrote; certificates follow
+   3 .env          the same names as compose variables; a mismatch here means a half-done --full
+   4 mediamtx.yml  webrtcAdditionalHosts = main host (WebRTC ICE); needs a MediaMTX restart to take effect
+   5 containers    the env each running container was CREATED with (a .env edit without a recreate shows here)
+   6 HTTPS         every host answers with a valid certificate; gated hosts send browsers to the auth host
+   7 Jellyfin      SSO plugin OID endpoint, branding disclaimer, Vanguard plugin URLs (on disk) + the SSO start
+   8 Gitea         app.ini, the auth source (live redirect), the runner, the server clone's remote
+   9 Authentik     outpost host, proxy provider external host + cookie domain, Jellyfin/Gitea redirect URIs.
+                   LAST among the services, on purpose: an app changed before Authentik is merely unreachable on
+                   its new name until Authentik catches up; Authentik changed before the app locks you out of it
+  10 Discord       the OAuth2 redirect in the developer portal — cannot be read; the exact value is printed
+  11 repo          deployLive.sh / switchToClone.sh defaults, about page labels = Caddy roles, docs
+  12 streamers     OBS dock URL and clock source: cannot be read; listed so nobody forgets
+A failure in an early section usually explains failures in later ones: fix top-down, re-run. The sweep runs the
+Jellyfin and Gitea checks before Authentik, and the report lists them that way, because that is the order to make
+the change in.
+"""
+import argparse, json, os, re, socket, ssl, subprocess, sys, urllib.request, urllib.error
+from pathlib import Path
+
+ROLES = ("main", "auth", "watch", "git", "bot", "live", "files", "sync", "clips", "diag")
+ENV_KEY = {"main": "MAIN_HOST", "watch": "WATCH_HOST", "git": "GIT_HOST", "bot": "BOT_HOST", "live": "LIVE_HOST",
+           "files": "FILES_HOST", "sync": "SYNC_HOST", "clips": "CLIPS_HOST", "diag": "DIAG_HOST"}
+GATED = ("main", "files", "sync", "clips", "live", "bot", "diag")            # forward-auth: anonymous -> auth host
+REPO = Path(os.environ.get("VANGUARD_REPO", str(Path.home() / "vanguard-stream") if Path.home().joinpath("vanguard-stream").is_dir() else "."))
+APPDATA = Path("/srv/appdata")
+
+results = []            # (section, status, text)
+def rec(section, status, text): results.append((section, status, text))
+def ok(section, text): rec(section, "PASS", text)
+def bad(section, text): rec(section, "FAIL", text)
+def warn(section, text): rec(section, "WARN", text)
+def manual(section, text): rec(section, "MANUAL", text)
+
+def sh(cmd, timeout=30, **kw):
+    try:
+        return subprocess.run(cmd, capture_output=True, text=True, timeout=timeout, **kw)
+    except (OSError, subprocess.TimeoutExpired) as error:
+        class R: returncode, stdout, stderr = 1, "", str(error)
+        return R()
+
+def read_hosts(caddyfile):
+    hosts, role = {}, None
+    for line in caddyfile.read_text().splitlines():
+        if line[:1].isspace(): continue
+        t = line.strip()
+        m = re.match(r"#\s*vanguard:(\w+)", t)
+        if m: role = m.group(1); continue
+        if t and not t.startswith(("#", "}", "{")) and t.endswith("{") and " " in t:
+            host = t.split()[0].rstrip(",")          # "a, b {": an alias; the first name is the role's
+            if "example.com" in host or role == "redirect": role = None; continue
+            hosts[role or ("main" if "main" not in hosts else host)] = host
+            role = None
+    return hosts
+
+def read_env(path):
+    env = {}
+    try:
+        for line in path.read_text().splitlines():
+            if "=" in line and not line.startswith("#"):
+                k, v = line.split("=", 1); env[k.strip()] = v.strip()
+    except OSError:
+        pass
+    return env
+
+URL_RE = re.compile(r"https?://([A-Za-z0-9.-]+\.[A-Za-z]{2,})(?::\d+)?")
+def hosts_in(text):
+    return sorted(set(h.lower() for h in URL_RE.findall(text)))
+
+def classify(section, where, found, allowed, expect=None, base=""):
+    """found: hostnames seen in `where`. allowed: every current host. expect: the one role's host, if specific."""
+    if not found:
+        warn(section, f"{where}: no hostname found"); return
+    for h in found:
+        if expect and h == expect: ok(section, f"{where}: {h}")
+        elif expect and h != expect and (h.endswith("." + base) or h.startswith("ovh1.")):
+            bad(section, f"{where}: {h} — expected {expect}")
+        elif h in allowed: ok(section, f"{where}: {h}")
+        elif base and (h.endswith("." + base) or h == base or h.startswith("ovh1.")):
+            bad(section, f"{where}: {h} — not a current host (stale)")
+        else:
+            pass    # third-party hostnames (discord.com, github.com, …) are not ours to judge
+
+class _NoRedirect(urllib.request.HTTPRedirectHandler):
+    def redirect_request(self, *a, **k): return None
+_opener = urllib.request.build_opener(_NoRedirect)
+
+def curl_head(url, timeout=12):
+    """(status, location, error) for a browser-like GET, without following redirects. GET, not HEAD: the
+    forward-auth gate answers HEAD with 200 and the SSO start endpoints are GET-only."""
+    req = urllib.request.Request(url, method="GET", headers={"User-Agent": "Mozilla/5.0 vghosts-check", "Accept": "text/html,*/*"})
+    try:
+        with _opener.open(req, timeout=timeout) as r:
+            r.read(2048)
+            return r.status, r.headers.get("Location", ""), ""
+    except urllib.error.HTTPError as e:
+        return e.code, e.headers.get("Location", ""), ""
+    except Exception as e:
+        return 0, "", str(e)
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--repo", action="store_true", help="repo-only checks (dev box)")
+    ap.add_argument("--json", action="store_true")
+    ap.add_argument("--caddyfile", default=None, help="path to the generated Caddyfile (default: <repo>/Caddyfile)")
+    args = ap.parse_args()
+    repo = REPO
+    caddyfile = Path(args.caddyfile) if args.caddyfile else repo / "Caddyfile"
+
+    # ---- 2 Caddyfile: the truth -------------------------------------------------------------------
+    S = "2 Caddyfile"
+    if not caddyfile.is_file():
+        if args.repo:
+            hosts = {}
+            warn(S, f"no Caddyfile at {caddyfile} (dev box): repo checks use the names in deployLive.sh / switchToClone.sh")
+        else:
+            bad(S, f"no Caddyfile at {caddyfile}: nothing to validate against — run configure.py --full first"); return report(args)
+    else:
+        hosts = read_hosts(caddyfile)
+    main_host = hosts.get("main", "")
+    base = main_host.split(".", 1)[1] if "." in main_host else ""
+    allowed = set(hosts.values())
+    for role in (ROLES if hosts else ()):
+        if role in hosts: ok(S, f"{role:6} {hosts[role]}")
+        else: warn(S, f"{role:6} no site block (configure.py --hosts to add one)")
+    if base and f"ovh1.{base}" in allowed: bad(S, f"ovh1.{base} still has a site block")
+
+    # ---- 1 DNS -------------------------------------------------------------------------------------
+    S = "1 DNS"
+    if hosts and not args.repo:
+        ips = {}
+        for role, h in hosts.items():
+            try: ips[h] = sorted(set(a[4][0] for a in socket.getaddrinfo(h, 443, proto=socket.IPPROTO_TCP)))
+            except OSError as e: bad(S, f"{h}: does not resolve ({e})")
+        role_ips = {h: v for r, h in hosts.items() for hh, v in ips.items() if hh == h and r in ROLES}
+        distinct = set(tuple(v) for v in role_ips.values())
+        for h, v in ips.items():
+            if h in role_ips: (ok if len(distinct) == 1 else bad)(S, f"{h} -> {', '.join(v)}")
+            else: warn(S, f"{h} -> {', '.join(v)} (apex/other block: a redirect may live anywhere, but Caddy must hold its certificate to answer)")
+        if len(distinct) > 1: bad(S, "the role hosts resolve to different addresses — a CNAME points elsewhere")
+        if base:
+            try:
+                old = sorted(set(a[4][0] for a in socket.getaddrinfo(f"ovh1.{base}", 22, proto=socket.IPPROTO_TCP)))
+                ok(S, f"ovh1.{base} -> {', '.join(old)} (ssh only; fine to keep)")
+            except OSError: warn(S, f"ovh1.{base} no longer resolves — ssh by IP or another name")
+
+    # ---- 3 .env --------------------------------------------------------------------------------------
+    S = "3 .env"
+    env = read_env(repo / ".env")
+    if not args.repo:
+        if not env: bad(S, f"{repo / '.env'} unreadable (run with sudo, from the checkout)")
+        for role, key in ENV_KEY.items():
+            if role not in hosts: continue
+            v = env.get(key, "")
+            if v == hosts[role]: ok(S, f"{key}={v}")
+            elif not v: bad(S, f"{key} unset — expected {hosts[role]}")
+            else: bad(S, f"{key}={v} — Caddyfile says {hosts[role]}")
+        apex, apex_to = env.get("APEX_HOST", ""), env.get("APEX_REDIRECT", "")
+        if apex:
+            found = hosts_in(apex_to); classify(S, f"APEX_REDIRECT ({apex} ->)", found, allowed, base=base)
+        else: ok(S, "APEX_HOST empty (bare domain not served)")
+
+    # ---- 4 mediamtx.yml ------------------------------------------------------------------------------
+    S = "4 mediamtx.yml"
+    if not args.repo:
+        mtx = repo / "mediamtx.yml"
+        if mtx.is_file():
+            m = re.search(r"^webrtcAdditionalHosts:\s*\[([^\]]*)\]", mtx.read_text(), re.M)
+            got = [x.strip().strip("'\"") for x in (m.group(1).split(",") if m else []) if x.strip()]
+            if main_host in got: ok(S, f"webrtcAdditionalHosts contains {main_host}")
+            else: bad(S, f"webrtcAdditionalHosts = {got or 'missing'} — expected {main_host} (then: docker compose restart mediamtx, between streams)")
+        else: bad(S, "mediamtx.yml missing from the checkout")
+
+    # ---- 5 running containers ------------------------------------------------------------------------
+    S = "5 containers"
+    if not args.repo:
+        want = {"jellyfin": ("JELLYFIN_PublishedServerUrl", "watch"), "gitea": ("GITEA__server__ROOT_URL", "git"),
+                "act_runner": ("GITEA_INSTANCE_URL", "git"), "discord-bot": ("WEB_PUBLIC_URL", "bot"),
+                "mediamtx": ("LIVE_HOST", "live"), "kibana": ("DIAG_HOST", "diag")}
+        for container, (var, role) in want.items():
+            if role not in hosts: continue
+            r = sh(["docker", "inspect", "-f", "{{range .Config.Env}}{{println .}}{{end}}", container])
+            if r.returncode != 0: warn(S, f"{container}: not running or no such container"); continue
+            val = next((l.split("=", 1)[1] for l in r.stdout.splitlines() if l.startswith(var + "=")), None)
+            if val is None: warn(S, f"{container}: {var} not in its environment"); continue
+            if hosts[role] in val: ok(S, f"{container}: {var}={val}")
+            else: bad(S, f"{container}: {var}={val} — expected {hosts[role]}; the container was created before .env changed: docker compose up -d {container}")
+        gd = sh(["docker", "inspect", "-f", "{{range .Config.Env}}{{println .}}{{end}}", "gitea"])
+        for var in ("GITEA__server__DOMAIN", "GITEA__server__SSH_DOMAIN"):
+            val = next((l.split("=", 1)[1] for l in gd.stdout.splitlines() if l.startswith(var + "=")), None)
+            if val and "git" in hosts: (ok if val == hosts["git"] else bad)(S, f"gitea: {var}={val}" + ("" if val == hosts["git"] else f" — expected {hosts['git']}"))
+
+    # ---- 6 HTTPS -------------------------------------------------------------------------------------
+    S = "6 HTTPS"
+    if hosts and not args.repo:
+        auth = hosts.get("auth", "")
+        for role, h in hosts.items():
+            status, loc, err = curl_head(f"https://{h}/")
+            if err:
+                if role in ROLES: bad(S, f"{h}: {err[:90]} (no certificate yet? Caddy block missing? DNS?)")
+                else: warn(S, f"{h}: {err[:90]} — the bare domain's A record does not reach this box, so its Caddy block cannot get a certificate; fix the record or clear APEX_HOST")
+                continue
+            loc_host = (hosts_in(loc) or [""])[0]
+            if role in GATED:
+                if status in (301, 302, 303, 307, 308) and loc_host == auth: ok(S, f"{h}: {status} -> {auth} (gate)")
+                elif status in (301, 302, 303, 307, 308): bad(S, f"{h}: {status} -> {loc} — gate sends to the wrong host")
+                else: warn(S, f"{h}: {status} without a redirect to {auth} — is the forward-auth gate on?")
+            else:
+                (ok if status < 400 else bad)(S, f"{h}: {status}{(' -> ' + loc) if loc else ''}")
+        # a move in progress: aliases must answer like the primary names; old names must redirect to the new ones
+        alt, red = env.get("ALT_BASE", "").strip().lower(), env.get("REDIRECT_BASE", "").strip().lower()
+        for role, h in hosts.items():
+            if role not in ROLES or not h.endswith("." + base): continue
+            if alt and alt != base:
+                a = h[: -len(base)] + alt
+                status, loc, err = curl_head(f"https://{a}/")
+                (bad if err else ok)(S, f"alias {a}: {err[:80] if err else status}")
+            if red and red != base:
+                o = h[: -len(base)] + red
+                status, loc, err = curl_head(f"https://{o}/")
+                if err: warn(S, f"old name {o}: {err[:80]} (its certificate may still be issuing)")
+                elif status == 308 and (hosts_in(loc) or [""])[0] == h: ok(S, f"old name {o}: 308 -> {h}")
+                else: bad(S, f"old name {o}: {status} -> {loc or '(no Location)'} — expected 308 to {h}")
+        if base:
+            status, loc, err = curl_head(f"https://ovh1.{base}/", timeout=8)
+            if err: ok(S, f"ovh1.{base}: no https (expected: ssh only)")
+            else: bad(S, f"ovh1.{base} still answers https ({status}) — a Caddy block or another server is serving the old name")
+
+    # ---- 7 Authentik ----------------------------------------------------------------------------------
+    S = "9 Authentik"
+    if not args.repo and hosts:
+        AK = r'''
+import json
+from urllib.parse import urlparse
+from authentik.outposts.models import Outpost
+from authentik.providers.proxy.models import ProxyProvider
+from authentik.providers.oauth2.models import OAuth2Provider
+from authentik.sources.oauth.models import OAuthSource
+from authentik.brands.models import Brand
+out = {"outposts": [], "proxy": [], "oauth2": [], "sources": [], "brands": []}
+for o in Outpost.objects.all():
+    out["outposts"].append({"name": o.name, "authentik_host": (o.config or {}).get("authentik_host", "") if isinstance(o.config, dict) else getattr(o.config, "authentik_host", "")})
+for p in ProxyProvider.objects.all():
+    out["proxy"].append({"name": p.name, "external_host": p.external_host, "cookie_domain": p.cookie_domain, "mode": str(p.mode)})
+for p in OAuth2Provider.objects.all():
+    uris = []
+    try:
+        uris = [u.url for u in p.redirect_uris]
+    except Exception:
+        try: uris = [u.get("url", "") for u in (p._redirect_uris or [])]
+        except Exception: uris = [str(getattr(p, "redirect_uris", ""))]
+    out["oauth2"].append({"name": p.name, "redirect_uris": uris})
+for s in OAuthSource.objects.all():
+    out["sources"].append({"name": s.name, "slug": s.slug, "provider_type": s.provider_type})
+for b in Brand.objects.all():
+    out["brands"].append({"domain": b.domain, "default": b.default, "logo": str(b.branding_logo), "default_app": str(b.default_application) if b.default_application_id else ""})
+print("VGHOSTS " + json.dumps(out))
+'''
+        r = sh(["docker", "exec", "-i", "authentik-server", "ak", "shell", "-c", "import sys; exec(sys.stdin.read())"], input=AK, timeout=120)
+        line = next((l for l in (r.stdout + r.stderr).splitlines() if l.startswith("VGHOSTS ")), None)
+        if not line:
+            bad(S, "could not read Authentik (" + (r.stderr.strip().splitlines() or ["no output"])[-1][:120] + ")")
+        else:
+            ak = json.loads(line[8:])
+            auth = hosts.get("auth", "")
+            for o in ak["outposts"]:
+                h = hosts_in(o["authentik_host"])
+                if h == [auth]: ok(S, f"outpost {o['name']!r}: authentik_host {o['authentik_host']}")
+                elif not o["authentik_host"]: warn(S, f"outpost {o['name']!r}: authentik_host unset (the embedded outpost then uses its own address; set it to https://{auth})")
+                else: bad(S, f"outpost {o['name']!r}: authentik_host {o['authentik_host']} — expected https://{auth}")
+            for p in ak["proxy"]:
+                h = hosts_in(p["external_host"])
+                classify(S, f"proxy provider {p['name']!r} external_host", h, allowed, base=base)
+                cd = p["cookie_domain"]
+                if "domain" in p["mode"].lower():
+                    (ok if cd == base else bad)(S, f"proxy provider {p['name']!r} cookie_domain={cd or '(unset)'}" + ("" if cd == base else f" — expected {base}; sessions will not span the hosts"))
+            for p in ak["oauth2"]:
+                expect = hosts.get("watch") if "jellyfin" in p["name"].lower() else hosts.get("git") if "gitea" in p["name"].lower() else None
+                for u in p["redirect_uris"]:
+                    classify(S, f"OAuth2 provider {p['name']!r} redirect", hosts_in(u), allowed, expect=expect, base=base)
+                if not p["redirect_uris"]: warn(S, f"OAuth2 provider {p['name']!r}: no redirect URIs")
+            for b in ak["brands"]:
+                if b["domain"] in ("", "authentik-default"): ok(S, f"brand domain {b['domain']!r} (default)")
+                elif b["domain"] == base or b["domain"].endswith(base): ok(S, f"brand domain {b['domain']}")
+                else: warn(S, f"brand domain {b['domain']} — not under {base}")
+            # ---- 8 Discord --------------------------------------------------------------------------
+            for s in ak["sources"]:
+                if s["provider_type"] == "discord":
+                    manual("10 Discord", f"developer portal → OAuth2 → Redirects must contain exactly: https://{auth}/source/oauth/callback/{s['slug']}/")
+            if not any(s["provider_type"] == "discord" for s in ak["sources"]):
+                bad("10 Discord", "no Discord OAuth source in Authentik")
+
+    # ---- 9 Jellyfin -----------------------------------------------------------------------------------
+    S = "7 Jellyfin"
+    if not args.repo and hosts:
+        jf = APPDATA / "jellyfin" / "config"
+        if jf.is_dir():
+            for xml in sorted(jf.rglob("*.xml")):
+                try: text = xml.read_text(errors="replace")
+                except OSError: continue
+                found = [h for h in hosts_in(text) if base and (h.endswith("." + base) or h == base or h.startswith("ovh1."))]
+                if not found: continue
+                name = xml.name.lower()
+                expect = hosts.get("auth") if "sso" in name else None
+                classify(S, f"{xml.relative_to(jf)}", found, allowed, expect=expect, base=base)
+        else: warn(S, f"{jf} not found")
+        if "watch" in hosts:
+            status, loc, err = curl_head(f"https://{hosts['watch']}/sso/OID/start/authentik")
+            lh = (hosts_in(loc) or [""])[0]
+            if err: bad(S, f"SSO start: {err[:80]}")
+            elif status in (301, 302, 303, 307) and lh == hosts.get("auth"): ok(S, f"SSO start -> {hosts['auth']} (plugin OID endpoint is right)")
+            else: bad(S, f"SSO start answered {status} -> {loc or '(no Location)'} — SSO-Auth plugin OID endpoint should be https://{hosts.get('auth')}/application/o/jellyfin/")
+
+    # ---- 10 Gitea ---------------------------------------------------------------------------------------
+    S = "8 Gitea"
+    if not args.repo and hosts and "git" in hosts:
+        ini = APPDATA / "gitea" / "gitea" / "conf" / "app.ini"
+        if ini.is_file():
+            text = ini.read_text(errors="replace")
+            for key in ("ROOT_URL", "DOMAIN", "SSH_DOMAIN"):
+                m = re.search(rf"^{key}\s*=\s*(\S+)", text, re.M)
+                v = m.group(1) if m else ""
+                (ok if hosts["git"] in v else bad)(S, f"app.ini {key}={v or '(missing)'}" + ("" if hosts["git"] in v else f" — expected {hosts['git']} (compose env sets it; recreate gitea)"))
+        else: warn(S, f"{ini} not found")
+        status, loc, err = curl_head(f"https://{hosts['git']}/user/oauth2/authentik")
+        lh = (hosts_in(loc) or [""])[0]
+        if err: bad(S, f"auth source: {err[:80]}")
+        elif lh == hosts.get("auth"): ok(S, f"auth source 'authentik' -> {hosts['auth']} (discovery URL is right)")
+        else: bad(S, f"auth source: {status} -> {loc or '(no Location)'} — Site administration → Authentication sources → authentik → discovery URL https://{hosts.get('auth')}/application/o/gitea/.well-known/openid-configuration")
+        runner = Path("/srv/appdata/act_runner/.runner")
+        if runner.is_file():
+            classify(S, "runner registration (.runner)", hosts_in(runner.read_text(errors="replace")), allowed, expect=hosts["git"], base=base)
+        else: warn(S, "runner: /srv/appdata/act_runner/.runner not found — ./registerGiteaRunner.sh")
+        r = sh(["git", "-C", str(repo), "remote", "get-url", "origin"])
+        if r.returncode == 0:
+            url = r.stdout.strip(); m = re.search(r"@([A-Za-z0-9.-]+)", url)
+            h = (m.group(1).lower() if m else "")
+            (ok if h == hosts["git"] else bad)(S, f"server clone remote: {url}" + ("" if h == hosts["git"] else f" — expected {hosts['git']}"))
+        else: warn(S, f"{repo}: not a git clone (deploy.sh pull needs one)")
+        for cfg in (Path.home() / ".ssh" / "config.vanguard", Path("/home/debian/.ssh/config.vanguard")):
+            if cfg.is_file():
+                classify(S, f"{cfg}", [h for h in re.findall(r"(?im)^\s*host(?:name)?\s+(\S+)", cfg.read_text()) if "." in h], allowed, expect=hosts["git"], base=base); break
+
+    # ---- 11 repo ----------------------------------------------------------------------------------------
+    S = "11 repo"
+    for script, var in (("deployLive.sh", "HOST"), ("switchToClone.sh", "REPO_SSH")):
+        p = repo / script
+        if not p.is_file(): warn(S, f"{script} missing"); continue
+        m = re.search(rf"^{var}=.*", p.read_text(), re.M)
+        found = [h for h in re.findall(r"@([A-Za-z0-9.-]+\.[a-z]{2,})", m.group(0))] if m else []
+        if hosts: classify(S, f"{script} {var}", [h.lower() for h in found], allowed, base=base)
+        else: ok(S, f"{script} {var}: {', '.join(found) or '(none)'}")
+    about = repo / "about" / "index.html"
+    if about.is_file():
+        labels = sorted(set(re.findall(r"sub\('([a-z]+)'\)", about.read_text())))
+        if hosts:
+            for label in labels:
+                (ok if label in hosts else bad)(S, f"about page label '{label}'" + ("" if label in hosts else " — no such Caddy role; the gateway link will 404/TLS-fail"))
+        else: ok(S, f"about page labels: {', '.join(labels)}")
+    for doc in ("README.md", "FRESH-INSTALL.md", "STATUS.md", "LOGGING.md"):
+        p = repo / doc
+        if not p.is_file(): continue
+        stale = [(n, l.strip()) for n, l in enumerate(p.read_text().splitlines(), 1)
+                 if re.search(r"https?://ovh1\.|ovh1\.[a-z.]+/", l) and not re.search(r"ssh|scp|ssh-only|ssh only", l)]
+        if stale: bad(S, f"{doc}: {len(stale)} line(s) send a browser to ovh1 — first: {stale[0][0]}: {stale[0][1][:80]}")
+        else: ok(S, f"{doc}: no browser links to ovh1")
+    plan = repo / "CommunityDiscordMedia.md"
+    if plan.is_file() and re.search(r"https?://ovh1\.", plan.read_text()):
+        warn(S, "CommunityDiscordMedia.md still describes ovh1 as a web host (historical plan; mark the lines as superseded or leave)")
+
+    # ---- 12 streamers -------------------------------------------------------------------------------
+    if hosts.get("live"):
+        manual("12 streamers", f"OBS → Docks → Custom Browser Docks: https://{hosts['live']}/dock/ ; dock Settings re-saved; Timecode band → \"Add band + clock source\" again; manual RTMP/SRT configs re-copied from the dock")
+        manual("12 streamers", f"Jellyfin apps on phones/TVs: server https://{hosts.get('watch','')} ; Discord pins: https://{main_host}/ (members), https://{hosts.get('clips','')}/ (gamers), https://{hosts.get('files','')}/ (editors)")
+    return report(args)
+
+def report(args):
+    order = lambda s: int(s.split()[0])
+    sections = sorted(set(s for s, _, _ in results), key=order)
+    if args.json:
+        print(json.dumps([{"section": s, "status": st, "text": t} for s, st, t in results], indent=1)); return 0
+    mark = {"PASS": "  ok  ", "FAIL": " FAIL ", "WARN": " warn ", "MANUAL": " todo "}
+    for s in sections:
+        print(f"\n== {s}")
+        for sec, st, t in results:
+            if sec == s: print(f" [{mark[st]}] {t}")
+    fails = sum(1 for _, st, _ in results if st == "FAIL"); warns = sum(1 for _, st, _ in results if st == "WARN")
+    todo = sum(1 for _, st, _ in results if st == "MANUAL")
+    print(f"\n{fails} failed, {warns} warnings, {todo} to check by hand. Fix top-down: an early failure explains the later ones.")
+    return 1 if fails else 0
+
+if __name__ == "__main__":
+    sys.exit(main())

bin/vgstitch.py

@@ -0,0 +1,179 @@
+#!/usr/bin/env python3
+"""Stitch reconnect fragments of one stream back into one recording (run by normalize-sweep.sh).
+
+Every time a publisher drops and comes back, MediaMTX opens a new file under streams/<user>/<Title>/.live/,
+so a night with a flaky uplink leaves one stream as a row of fragments: 20 min, 2 s, 14 min, 1 s, ... The
+fragments are the SAME stream (same user, same title) and the file names carry the server arrival time to
+the microsecond, so we know exactly where each piece sits on the clock. This joins them losslessly:
+
+  * only when the stream is over: nothing in .live/ open (fuser) or modified in the last IDLE_MIN minutes
+  * only fewer than MAX_PARTS fragments (default 500 — a lossy uplink splits a half-hour stream into ~180
+    files, see normalize-sweep.sh): beyond that is a broken night, left for vgstuck.sh / the streaming page
+  * a chain breaks where the gap between fragments is longer than MAX_GAP_S (default 10 min: a reconnect
+    is seconds, a second session the same evening is not), or where the video/audio parameters change
+    (resolution, codec, rate): those cannot be copy-concatenated, so each side is stitched on its own
+  * ffmpeg concat demuxer with -c copy — no re-encode, the sweep's own pass (timecode, band removal,
+    Opus->AAC) runs on the result as on any other recording
+  * the gaps between fragments are COLLAPSED (the picture jumps), and recorded in <stem>.stitch.json which
+    the sweep merges into the published sidecar: part files, each part's arrival time, its offset in the
+    stitched file, and the wall-clock time missing at every join
+  * the stitched file takes the FIRST fragment's name (so the arrival-time fallback for the timecode still
+    points at the real start), the fragments move to trash/<night>/ for the usual 3-day undo
+  * a fragment ffprobe cannot read is binned (reason=unreadable) rather than blocking its stream forever
+
+Prints logfmt lines (component=sweep) for Kibana. Exit 0 unless the arguments are wrong.
+    vgstitch.py <media root> [--max-parts N] [--max-gap-s S] [--idle-min M] [--dry-run]
+"""
+import argparse, json, os, re, subprocess, sys, time
+from datetime import datetime
+from pathlib import Path
+from zoneinfo import ZoneInfo
+
+TZ = ZoneInfo("America/New_York")
+NAME_RE = re.compile(r"(\d{4})-(\d{2})-(\d{2})_(\d{2})-(\d{2})-(\d{2})(?:-(\d{1,6}))?")
+
+def vglog(level, **kv):
+    parts = [f"ts={datetime.now().astimezone().isoformat(timespec='seconds')}", f"level={level}", "component=sweep"]
+    for k, v in kv.items():
+        v = str(v)
+        parts.append(f'{k}="{v}"' if (" " in v or "/" in v or v == "") else f"{k}={v}")
+    print(" ".join(parts), flush=True)
+
+def arrival_ms(stem):
+    m = NAME_RE.search(stem)
+    if not m: return None
+    y, mo, d, h, mi, s, us = m.groups()
+    us = int((us or "0").ljust(6, "0"))
+    return int(datetime(int(y), int(mo), int(d), int(h), int(mi), int(s), us, tzinfo=TZ).timestamp() * 1000)
+
+def ffprobe(path):
+    """(params tuple, duration_s) or None when unreadable."""
+    try:
+        out = subprocess.run(["ffprobe", "-v", "error", "-show_entries",
+                              "stream=codec_type,codec_name,width,height,pix_fmt,sample_rate,channels:format=duration",
+                              "-of", "json", str(path)], capture_output=True, text=True, timeout=120, check=True).stdout
+        j = json.loads(out)
+        v = [s for s in j["streams"] if s["codec_type"] == "video"]
+        a = [s for s in j["streams"] if s["codec_type"] == "audio"]
+        if not v: return None
+        dur = float(j["format"].get("duration") or 0)
+        if dur <= 0: return None
+        params = (v[0].get("codec_name"), v[0].get("width"), v[0].get("height"), v[0].get("pix_fmt"),
+                  tuple((s.get("codec_name"), s.get("sample_rate"), s.get("channels")) for s in a))
+        return params, dur
+    except Exception:
+        return None
+
+def busy(path):
+    return subprocess.run(["fuser", "-s", str(path)], capture_output=True).returncode == 0
+
+def to_trash(root, path, night):
+    rel = path.relative_to(root)
+    dst = root / "trash" / night / rel
+    dst.parent.mkdir(parents=True, exist_ok=True)
+    os.replace(path, dst)
+    return dst
+
+def stitch_dir(root, live, args):
+    titledir = live.parent
+    stream = f"{titledir.parent.name}/{titledir.name}"
+    files = sorted(p for p in live.iterdir() if p.is_file() and p.suffix.lower() == ".mp4" and not p.name.endswith(".part"))
+    if len(files) < 2: return
+    now = time.time()
+    for p in files:                                   # still streaming? leave the whole set alone
+        if busy(p) or now - p.stat().st_mtime < args.idle_min * 60:
+            vglog("info", event="stitch_wait", stream=stream, parts=len(files), reason="stream still going")
+            return
+    if len(files) >= args.max_parts:
+        vglog("warn", event="stitch_skipped", stream=stream, parts=len(files), reason="too_many", max=args.max_parts)
+        return
+    night = datetime.now().strftime("%Y-%m-%d")
+    parts = []
+    for p in files:
+        t = arrival_ms(p.stem)
+        info = ffprobe(p)
+        if t is None:
+            vglog("warn", event="stitch_skipped", stream=stream, file=p, reason="name has no timestamp"); return
+        if info is None:
+            if not args.dry_run: dst = to_trash(root, p, night)
+            else: dst = "(dry run)"
+            vglog("warn", event="discarded", reason="unreadable", file=p, stream=stream, trash=dst)
+            continue
+        parts.append({"file": p, "arrival_ms": t, "params": info[0], "duration_ms": int(info[1] * 1000)})
+    parts.sort(key=lambda x: x["arrival_ms"])
+    # chains: consecutive, same parameters, gap under the limit
+    chains, cur = [], []
+    for part in parts:
+        if cur:
+            prev = cur[-1]
+            gap = part["arrival_ms"] - (prev["arrival_ms"] + prev["duration_ms"])
+            if part["params"] != prev["params"]:
+                vglog("info", event="stitch_break", stream=stream, reason="params_changed", file=part["file"],
+                      before=f"{prev['params'][1]}x{prev['params'][2]}", after=f"{part['params'][1]}x{part['params'][2]}")
+                chains.append(cur); cur = []
+            elif gap > args.max_gap_s * 1000:
+                vglog("info", event="stitch_break", stream=stream, reason="gap", gap_s=gap // 1000, file=part["file"])
+                chains.append(cur); cur = []
+        cur.append(part)
+    if cur: chains.append(cur)
+    for chain in chains:
+        if len(chain) < 2: continue
+        first = chain[0]["file"]
+        offset, joins, plist = 0, [], []
+        for i, part in enumerate(chain):
+            if i:
+                prev = chain[i - 1]
+                missing = part["arrival_ms"] - (prev["arrival_ms"] + prev["duration_ms"])
+                joins.append({"at_s": round(offset / 1000, 3), "missing_ms": max(0, missing)})
+            plist.append({"file": part["file"].name, "arrival_ms": part["arrival_ms"],
+                          "duration_ms": part["duration_ms"], "offset_ms": offset})
+            offset += part["duration_ms"]
+        if args.dry_run:
+            vglog("info", event="stitched", stream=stream, parts=len(chain), joins=len(joins), dry_run=1,
+                  missing_ms=sum(j["missing_ms"] for j in joins), out=first); continue
+        listing = live / f"{first.stem}.concat.txt"
+        tmp = live / f"{first.stem}.stitch.part"
+        listing.write_text("".join("file '%s'\n" % str(p["file"]).replace("'", r"'\''") for p in chain))
+        run = subprocess.run(["nice", "-n", "15", "ionice", "-c", "3", "ffmpeg", "-nostdin", "-hide_banner", "-loglevel", "error", "-y",
+                              "-f", "concat", "-safe", "0", "-i", str(listing), "-map", "0", "-c", "copy",
+                              "-movflags", "+faststart", "-f", "mp4", str(tmp)], capture_output=True, text=True)
+        listing.unlink(missing_ok=True)
+        got = ffprobe(tmp) if run.returncode == 0 else None
+        want = offset / 1000
+        if got is None or abs(got[1] - want) > 2 + len(chain):
+            tmp.unlink(missing_ok=True)
+            why = (run.stderr.strip().splitlines() or ["no stderr"])[-1][:200]
+            vglog("error", event="stitch_failed", stream=stream, parts=len(chain), rc=run.returncode,
+                  want_s=int(want), got_s=int(got[1]) if got else "?", reason=why)
+            continue                                    # fragments untouched: the sweep publishes them one by one
+        last_mtime = max(p["file"].stat().st_mtime for p in chain)
+        for p in chain: to_trash(root, p["file"], night)
+        final = live / f"{first.stem}.mp4"
+        os.replace(tmp, final)
+        os.utime(final, (last_mtime, last_mtime))        # keep it "idle" so this very sweep publishes it
+        meta = {"parts": plist, "joins": joins, "missing_ms": sum(j["missing_ms"] for j in joins),
+                "trash": str(root / "trash" / night / live.relative_to(root)), "stitched_at": datetime.now(TZ).isoformat(timespec="seconds")}
+        (live / f"{first.stem}.stitch.json").write_text(json.dumps(meta))
+        vglog("info", event="stitched", stream=stream, parts=len(chain), joins=len(joins),
+              missing_ms=meta["missing_ms"], seconds=int(want), out=final, trash=meta["trash"])
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("root")
+    ap.add_argument("--max-parts", type=int, default=500)
+    ap.add_argument("--max-gap-s", type=int, default=600)
+    ap.add_argument("--idle-min", type=int, default=3)
+    ap.add_argument("--dry-run", action="store_true")
+    args = ap.parse_args()
+    root = Path(args.root)
+    streams = root / "streams"
+    if not streams.is_dir(): return 0
+    for live in sorted(streams.glob("*/*/.live")):
+        if live.is_dir():
+            try: stitch_dir(root, live, args)
+            except Exception as error:
+                vglog("error", event="stitch_failed", dir=live, reason=str(error)[:200])
+    return 0
+
+if __name__ == "__main__":
+    sys.exit(main())

bin/vgstuck.sh

@@ -2,15 +2,23 @@
 # Triage recordings that never published: see why each one is stuck, then retry it or bin it.
 #
 #   vgstuck.sh                     list everything stuck, with the reason        (read-only, the default)
+#   vgstuck.sh list --durations    ...and measure each one's length (slow: fragmented MP4 is read end to end)
 #   vgstuck.sh retry <match>       clear what is blocking it so the sweep tries again
 #   vgstuck.sh retry --all         ...for everything that is parked
 #   vgstuck.sh drop  <match>       move the recording to trash: 3-day undo, same as "dump tonight"
+#   vgstuck.sh list  --under 5     only recordings smaller than 5 MB (reconnect fragments)
+#   vgstuck.sh drop  --under 5     bin all of those; still-recording files are never touched
+#   vgstuck.sh wipe                bin EVERYTHING unprocessed: stops the sweep (killing any conversion in flight),
+#                                  then moves every .live/ file — parked, waiting, published-collision, fresh — with
+#                                  its sidecars, failure logs and .part leftovers to trash, then drops the emptied
+#                                  folders. The ONLY thing left alone is a file MediaMTX has open, i.e. a stream
+#                                  being recorded this minute. The Dump tab's "Bin all unprocessed" button runs this.
 #   vgstuck.sh sweep               kick the sweep once, without waiting on it
 #   vgstuck.sh json                the same survey as JSON (what dumpd serves to the web page)
 #
 # <match> is any substring of the path (a streamer name, a title, a timestamp). It must match at least one
-# recording, and `drop` refuses --all: binning every stuck recording in one keystroke is never a thing
+# recording, and `drop` refuses --all so a stray pattern cannot bin everything by accident: the deliberate
-# somebody meant to do.
+# version is spelled out as its own word, `wipe`.
 #
 # WHY A RECORDING STAYS IN .live/
 #   parked     a <Title>__<ts>.normalize-failed.log sits next to the intended output. The sweep skips the
@@ -57,66 +65,118 @@ survey() {
       reason="nothing blocking it; the sweep has not got to it"
     fi
     [[ -e "$out.part" ]] && reason="$reason [leftover .part]"
-    dur=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$src" 2>/dev/null | cut -d. -f1)
+    # Duration is OPT-IN. Everything in .live/ is fragmented MP4 with no duration in its header, so ffprobe
+    # has to read the whole file to answer — gigabytes per recording. With many stuck files the survey took
+    # over three minutes, dumpd's request timed out, and the web page broke. Size and age come from stat.
+    dur=""
+    if [[ ${VGSTUCK_DURATIONS:-0} == 1 ]]; then
+      dur=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$src" 2>/dev/null | cut -d. -f1)
+    fi
     printf '%s|%s|%s|%s|%s|%s|%s\n' "$src" "$out" "$state" \
-      "$(( $(date +%s) - $(stat -c %Y "$src") ))" "$(stat -c %s "$src")" "${dur:-0}" "$reason"
+      "$(( $(date +%s) - $(stat -c %Y "$src") ))" "$(stat -c %s "$src")" "${dur:--}" "$reason"
   done < <(find "$ROOT/streams" -path '*/.live/*' -name '*.mp4' -print0 2>/dev/null)
 }
 
 show() {
-  local any=0 src out state age bytes dur reason
+  local any=0 total=0 src out state age bytes dur reason
   printf '\n  %-9s %-9s %9s %8s  %s\n' state waiting length size recording
   while IFS='|' read -r src out state age bytes dur reason; do
+    total=$(( total + bytes ))
     any=1
+    if [[ $dur == - || -z $dur ]]; then length="-"; else
+      length=$(printf '%d:%02d:%02d' $(( dur / 3600 )) $(( (dur % 3600) / 60 )) $(( dur % 60 ))); fi
     printf '  %-9s %-9s %9s %7.0fM  %s\n' "$state" \
       "$(( age / 3600 ))h$(printf '%02d' $(( (age % 3600) / 60 )))m" \
-      "$(printf '%d:%02d:%02d' $(( dur / 3600 )) $(( (dur % 3600) / 60 )) $(( dur % 60 )))" \
+      "$length" \
       "$(awk -v b="$bytes" 'BEGIN{print b/1000000}')" "${src#"$ROOT"/streams/}"
     printf '            %s\n' "$reason"
-  done < <(survey | sort -t'|' -k3,3 -k4,4nr)
+  done < <(selected "${1:---all}" "${2:-}" | sort -t'|' -k3,3 -k4,4nr)
-  (( any )) || printf '  nothing is stuck.\n'
+  (( any )) || printf '  nothing matches.\n'
-  printf '\n  retry one:  vgstuck.sh retry <substring>      bin one:  vgstuck.sh drop <substring>\n\n'
+  (( any )) && printf '  %s item(s), %s GB\n' "$(selected "${1:---all}" "${2:-}" | wc -l)" "$(awk -v b="$total" 'BEGIN{printf "%.1f", b/1e9}')"
+  [[ ${VGSTUCK_DURATIONS:-0} == 1 ]] || printf '\n  lengths not measured (slow on fragmented MP4): vgstuck.sh list --durations\n'
+  printf '  retry one:  vgstuck.sh retry <substring>      bin one:  vgstuck.sh drop <substring>\n\n'
 }
 
-# rows matching the pattern, or everything when --all
+# rows matching the pattern, everything when --all, or everything smaller than N MB with --under N.
+# --under exists for reconnect fragments: every time a publisher drops and comes back MediaMTX opens a new
+# recording, so a bad night leaves hundreds of one- and two-second files. Still-recording files never match.
 selected() {
-  local pattern="$1"
+  local pattern="$1" limit="${2:-}"
-  if [[ $pattern == --all ]]; then survey; else survey | grep -F -- "$pattern" || true; fi
+  if [[ $pattern == --under ]]; then
+    [[ $limit =~ ^[0-9]+$ ]] || { echo "usage: --under <megabytes>" >&2; return 2; }
+    survey | awk -F'|' -v max="$(( limit * 1000000 ))" '$3 != "writing" && $5 < max'
+  elif [[ $pattern == --all ]]; then survey
+  else survey | grep -F -- "$pattern" || true
+  fi
 }
 
 do_retry() {
-  local pattern="$1" n=0 src out state age bytes dur reason
+  local pattern="$1" limit="${2:-}" n=0 src out state age bytes dur reason
   while IFS='|' read -r src out state age bytes dur reason; do
     [[ $state == writing ]] && { vglog info "event=skip reason=still_recording file=\"$src\""; continue; }
     rm -f "$out.part"                                  # leftover partial output: harmless, but no reason to keep
     rm -f "${out%.mp4}.normalize-failed.log"           # THIS is what makes the sweep skip the file forever
     n=$((n + 1))
     vglog info "event=retry file=\"$src\" was=$state"
-  done < <(selected "$pattern")
+  done < <(selected "$pattern" "$limit")
   (( n )) || { vglog warn "event=nothing_matched pattern=\"$pattern\""; return 1; }
   vglog info "event=retry_done count=$n msg=\"run vgstuck.sh sweep, or wait for the timer\""
 }
 
-do_drop() {
+# one row per line on stdin (survey format) -> trash. Prints the count it moved.
-  local pattern="$1" night n=0 src out state age bytes dur reason rel dst
+drop_rows() {
-  [[ $pattern == --all ]] && { vglog error 'event=refused msg="drop --all is not allowed; name them"' >&2; return 2; }
+  local night="$1" n=0 src out state age bytes dur reason rel dst victim
-  night="$(date +%F)"
   while IFS='|' read -r src out state age bytes dur reason; do
-    [[ $state == writing ]] && { vglog info "event=skip reason=still_recording file=\"$src\""; continue; }
+    # The survey labels a file by its blocker first (a parked file stays "parked" even while being written), so the
-    for victim in "$src" "${src%.*}.json" "${out%.mp4}.normalize-failed.log" "$out.part"; do
+    # safety check here does not trust the label: open by any process, or touched inside MINS_IDLE, means leave it.
+    # In wipe mode (WIPE=1) only "open" counts — the sweep has been stopped, so an open file is MediaMTX recording.
+    if fuser -s "$src" 2>/dev/null || { [[ ${WIPE:-0} != 1 ]] && { [[ $state == writing ]] || [[ -n $(find "$src" -mmin "-$MINS_IDLE" -print -quit 2>/dev/null) ]]; }; }; then
+      vglog info "event=skip reason=still_recording file=\"$src\"" >&2; continue
+    fi
+    for victim in "$src" "${src%.*}.json" "${src%.*}.stitch.json" "${out%.mp4}.normalize-failed.log" "$out.part"; do
       [[ -e $victim ]] || continue
       rel="${victim#"$ROOT"/}"
       dst="$TRASH/$night/$rel"
       mkdir -p "$(dirname "$dst")"
-      mv -f "$victim" "$dst" && vglog info "event=trashed file=\"$rel\" undo_days=3"
+      mv -f "$victim" "$dst" && vglog info "event=trashed file=\"$rel\" undo_days=3" >&2
     done
     rmdir "$(dirname "$src")" "$(dirname "$(dirname "$src")")" 2>/dev/null || true
     n=$((n + 1))
-  done < <(selected "$pattern")
+  done
+  echo "$n"
+}
+
+do_drop() {
+  local pattern="$1" limit="${2:-}" night n
+  [[ $pattern == --all ]] && { vglog error 'event=refused msg="drop --all is not allowed; name them, use --under <MB>, or wipe"' >&2; return 2; }
+  night="$(date +%F)"
+  n=$(selected "$pattern" "$limit" | drop_rows "$night")          # per-file lines go to stderr, the count to stdout
   (( n )) || { vglog warn "event=nothing_matched pattern=\"$pattern\""; return 1; }
   vglog info "event=drop_done count=$n trash=\"$TRASH/$night\""
 }
 
+# Everything unprocessed, deliberately. First the sweep is stopped, which kills any ffmpeg conversion it has in
+# flight (the timer starts it again within five minutes, to an empty queue). Then every .live/ file goes, parked
+# or fresh, and so do failure logs whose recording is already gone (they would keep the Stuck tab listing ghosts)
+# and leftover .part outputs. The one exception: a file some process still holds open after the sweep is down,
+# which is MediaMTX recording a live stream — pulling that from under it would lose the stream in progress.
+do_wipe() {
+  local night n stale=0 log rel dst
+  night="$(date +%F)"
+  if systemctl is-active --quiet normalize-sweep.service; then
+    systemctl stop normalize-sweep.service && vglog info "event=cancelled unit=normalize-sweep.service msg=\"conversion in flight killed\""
+  fi
+  systemctl is-active --quiet vgdeband.service && systemctl stop vgdeband.service && vglog info "event=cancelled unit=vgdeband.service"
+  n=$(survey | WIPE=1 drop_rows "$night")
+  while IFS= read -r -d '' log; do
+    rel="${log#"$ROOT"/}"; dst="$TRASH/$night/$rel"; mkdir -p "$(dirname "$dst")"
+    mv -f "$log" "$dst" && stale=$((stale + 1))
+  done < <(find "$ROOT/streams" -maxdepth 2 \( -name '*.normalize-failed.log' -o -name '*.mp4.part' \) -print0 2>/dev/null)
+  find "$ROOT/streams" -mindepth 2 -type d -name .live -empty -delete 2>/dev/null
+  find "$ROOT/streams" -mindepth 2 -maxdepth 2 -type d -empty -delete 2>/dev/null
+  vglog info "event=wipe_done recordings=${n:-0} stale_files=$stale trash=\"$TRASH/$night\" undo_days=3"
+}
+
 # Same survey, as JSON. bin/dumpd.py serves this to the web page, so the classification lives in exactly
 # one place and the browser and the CLI can never disagree about why something is stuck.
 emit_json() {
@@ -129,15 +189,19 @@ for line in sys.stdin:
         continue
     src, out, state, age, size, dur, reason = parts
     rows.append({"src": src, "out": out, "state": state, "age_s": int(age or 0),
-                 "bytes": int(size or 0), "duration_s": int(dur or 0), "reason": reason})
+                 "bytes": int(size or 0),
+                 "duration_s": int(dur) if dur not in ("", "-") else None, "reason": reason})
 print(json.dumps(rows))'
 }
 
+for a in "$@"; do [[ $a == --durations ]] && export VGSTUCK_DURATIONS=1; done
+
 case "${1:-list}" in
-  list|"")  show ;;
+  list|"")  show "${2:---all}" "${3:-}" ;;
   json)     emit_json ;;
-  retry)    [[ -n ${2:-} ]] || { echo "usage: vgstuck.sh retry <substring>|--all" >&2; exit 1; }; do_retry "$2" ;;
+  retry)    [[ -n ${2:-} ]] || { echo "usage: vgstuck.sh retry <substring>|--all|--under <MB>" >&2; exit 1; }; do_retry "$2" "${3:-}" ;;
-  drop)     [[ -n ${2:-} ]] || { echo "usage: vgstuck.sh drop <substring>" >&2; exit 1; }; do_drop "$2" ;;
+  drop)     [[ -n ${2:-} ]] || { echo "usage: vgstuck.sh drop <substring>|--under <MB>" >&2; exit 1; }; do_drop "$2" "${3:-}" ;;
+  wipe)     do_wipe ;;
   sweep)    systemctl start --no-block normalize-sweep.service && vglog info "event=sweep_started" ;;
   *)        sed -n '2,30p' "$0" | sed 's/^# \{0,1\}//'; exit 1 ;;
 esac

check.sh

@@ -9,7 +9,7 @@ for f in install.sh addCommunityMediaServer.sh installJellyfinPlugin.sh register
   [[ -f $f ]] || continue
   bash -n "$f" && say "$f" ok || { say "$f" "BASH SYNTAX"; fail=1; }
 done
-for f in configure.py bin/dumpd.py bin/vgstrip.py observability/kibana-setup.py observability/kibana-alarms.py; do
+for f in configure.py bin/dumpd.py bin/vgstrip.py bin/vgstitch.py bin/vghosts-check.py observability/kibana-setup.py observability/kibana-alarms.py; do
   python3 -m py_compile "$f" && say "$f" ok || { say "$f" "PYTHON SYNTAX"; fail=1; }
 done
 rm -rf __pycache__ bin/__pycache__
@@ -39,7 +39,7 @@ if ! command -v node >/dev/null; then
 else
   node -e "
 const fs=require('fs');
-for (const f of ['dock/index.html','dock/clock/index.html','clips/index.html','about/index.html','stuck/index.html']) { if (!fs.existsSync(f)) continue; const html=fs.readFileSync(f,'utf8'); for (const m of html.matchAll(/<script(?:[^>]*)>([\s\S]*?)<\/script>/g)) if (m[1].trim()) new Function(m[1]); }
+for (const f of ['dock/index.html','dock/clock/index.html','clips/index.html','about/index.html','streaming/index.html']) { if (!fs.existsSync(f)) continue; const html=fs.readFileSync(f,'utf8'); for (const m of html.matchAll(/<script(?:[^>]*)>([\s\S]*?)<\/script>/g)) if (m[1].trim()) new Function(m[1]); }
 if (fs.existsSync('dock/clock/vgclock.js')) new Function(fs.readFileSync('dock/clock/vgclock.js','utf8'));" \
     && say "page JS" ok || { say "page JS" "SYNTAX ERROR"; fail=1; }
 fi

configure.py

@@ -16,7 +16,7 @@
                             NEW host needs --full or this.
 
 Hosts: main = the gateway page at /, files = copyparty, clips = clip intake, sync = Syncthing UI,
-       live = portal + OBS dock (/dock/, /dock/clock/, /dump, /me, /time), diag = logs/metrics/alerts,
+       live = portal + OBS dock (/dock/, /dock/clock/, /dump, /me, /time), diag = logs/metrics/alerts + streaming triage (live, stuck, dump),
        auth, watch, git, bot.
 
 Reads  : Caddyfile (hostnames), mediamtx.yml (streamers), .env
@@ -27,7 +27,7 @@ Writes : Caddyfile, mediamtx.yml, copyparty.conf, stream-token.secret -> /etc/va
          Discord bot token/guild), folders, restarts
 Set VG_DRY=1 to write files only (no sudo / docker / folders).
 """
-import base64, getpass, hashlib, json, os, re, secrets, shutil, subprocess, sys, time
+import base64, getpass, hashlib, json, os, re, secrets, shutil, socket, subprocess, sys, time
 import urllib.error, urllib.request
 from pathlib import Path
 
@@ -48,8 +48,8 @@ def read_hosts():
             m = re.match(r"#\s*vanguard:(\w+)", t)
             if m: role = m.group(1); continue
             if t and not t.startswith(("#", "}", "{")) and t.endswith("{") and " " in t:
-                host = t.split()[0]
+                host = t.split()[0].rstrip(",")          # "a, b {" = an alias (decorate_caddy); the first name is the role's
-                if "example.com" in host: role = None; continue
+                if "example.com" in host or role == "redirect": role = None; continue   # redirect blocks are not roles
                 hosts[role or ("main" if "main" not in hosts else host)] = host
                 role = None
     except FileNotFoundError:
@@ -348,13 +348,19 @@ def write_caddy(domain, auth_host="", watch_host="", git_host="", bot_host="", l
 {gate(env.get("DIAG_REQUIRE_GROUP", "media-admins"))}	# Triage lives on this host, alongside the dashboards it belongs with. Both handles come BEFORE the
 	# catch-all below, and the Kibana proxy is wrapped in its own handle so the three are evaluated in the
 	# order written rather than by Caddy's directive ordering.
-	handle /stuck/api {{
+	handle /streaming/api/* {{
-		reverse_proxy 127.0.0.1:8080          # dumpd: the list, and retry/bin for the admin groups
+		reverse_proxy 127.0.0.1:8080          # dumpd: live publishers + loss, and the stuck list / retry / bin
 	}}
-	handle_path /stuck* {{
+	handle_path /streaming* {{
-		root * /srv/stuck
+		root * /srv/streaming
 		try_files {{path}} /index.html
-		file_server                           # stuck-recording triage; this host is already media-admins only
+		file_server                           # Live and Stuck tabs; this host is already media-admins only
+	}}
+	redir /stuck* /streaming#stuck 302        # it used to live here
+	handle_path /disk* {{
+		root * /srv/streaming
+		rewrite * /disk.html                  # the Disk page: same mount as the streaming page, its own address
+		file_server
 	}}
 	# Kibana IS the rest of this host: logs (Discover), dashboards and alerting in one app, served at the
 	# root so there is no path prefix to get wrong. Elasticsearch itself is never published.
@@ -383,7 +389,7 @@ groups: {{http.request.header.X-Authentik-Groups}}"
 		reverse_proxy 127.0.0.1:8080          # server clock for the timecode strip (clock page + Go Live)
 	}}
 	handle /dump* {{
-		reverse_proxy 127.0.0.1:8080          # dump tonight (dock)
+		reverse_proxy 127.0.0.1:8080          # dump tonight (dock): the signed-in streamer's own night
 	}}
 	handle_path /dock* {{
 		root * /srv/dock
@@ -475,7 +481,9 @@ name:   {{http.request.header.X-Authentik-Name}}"
 	redir /about* / 308                       # old bookmarks: the gateway page used to live at /about/
 	handle {{
 		root * /srv/about
-		try_files {{path}} /index.html
+		# {{path}}/index.html: Caddy's file matcher takes a slash-less path as "a file", so /onboarding would skip
+		# the onboarding/ folder and fall through to the gateway page. Sub-pages live in folders under about/.
+		try_files {{path}} {{path}}/index.html /index.html
 		file_server                           # the gateway page is the root of the main host
 	}}
 }}
@@ -548,19 +556,7 @@ def write_env(watch_host, discord_token=None, discord_guild=None, git_host="", b
     Path(".env").write_text(render_env(env))
     os.chmod(".env", 0o600)
 
-def docker_gateway():
-    """Host IP as seen from the compose network: MediaMTX (bridge) calls dumpd's /mtx-auth there."""
-    if DRY: return "172.18.0.1"
-    for net in ("vanguard-stream_default", "vanguard_default"):
-        try:
-            out = subprocess.run(["docker", "network", "inspect", net, "-f", "{{(index .IPAM.Config 0).Gateway}}"], capture_output=True, text=True, timeout=10)
-            if out.returncode == 0 and out.stdout.strip(): return out.stdout.strip()
-        except Exception:
-            pass
-    return "172.18.0.1"
-
 def write_mediamtx(public_host=""):
-    auth_gw = docker_gateway()
     Path("mediamtx.yml").write_text(f"""# MediaMTX — ingest only. No playback endpoints exposed.  (generated by configure.py)
 logLevel: info
 
@@ -572,18 +568,18 @@ rtspTransports: [tcp]
 rtspEncryption: "no"
 moq: no
 metrics: yes                     # Prometheus text format, scraped by Metricbeat (LOGGING.md); loopback only
-metricsAddress: :9998
+metricsAddress: 127.0.0.1:9998   # host network: loopback only; Metricbeat scrapes it there
 pprof: no
 
 # live portal (served to the browser only via Caddy + Authentik on live.HOST; these ports are published on 127.0.0.1)
 api: yes
-apiAddress: :9997
+apiAddress: 127.0.0.1:9997       # host network: loopback only; Caddy proxies live.HOST/api/*
 hls: yes
-hlsAddress: :8888
+hlsAddress: 127.0.0.1:8888       # host network: loopback only; Caddy proxies live.HOST/hls/*
 hlsVariant: lowLatency
 hlsAlwaysRemux: no
 webrtc: yes
-webrtcAddress: :8889
+webrtcAddress: 127.0.0.1:8889    # WHIP/WHEP signalling: loopback only, via Caddy. Media is 8189 below
 webrtcLocalUDPAddress: :8189
 webrtcLocalTCPAddress: :8189
 webrtcIPsFromInterfaces: no
@@ -601,7 +597,7 @@ srtAddress: :8890
 # NOTE: on this setup (bind-mounted file) MediaMTX does not reload this file live: restart after any change here.
 # Day-to-day operations (streamers, tokens, titles) never touch this file.
 authMethod: http
-authHTTPAddress: http://{auth_gw}:8081/mtx-auth
+authHTTPAddress: http://127.0.0.1:8081/mtx-auth   # dumpd, on the same host network
 authHTTPExclude:
   - action: api
   - action: metrics
@@ -621,14 +617,51 @@ pathDefaults:
   runOnReadyRestart: no
 
 paths:
-  # audio mirrors (first rule wins): never recorded, no hook; only publishable from inside the container (dumpd)
+  # Audio mirrors: never recorded, no hook; only publishable from inside the container (dumpd).
-  ~^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+\\.webrtc$:
+  # The title segment of BOTH rules is dot-free, so a name matches exactly one of them and the order MediaMTX
+  # evaluates them in is irrelevant. They used to overlap (titles allowed dots) and whichever won was luck:
+  # a 36 GB mirror got recorded as a full duplicate of its source. Usernames keep dots: Discord allows them.
+  ~^[A-Za-z0-9_.-]+/[A-Za-z0-9_-]+\\.webrtc$:
     record: no
     runOnReady: ""
   # <username>/<Title>; dumpd decides who may publish where (token user == prefix)
-  ~^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$: {{}}
+  ~^[A-Za-z0-9_.-]+/[A-Za-z0-9_-]+$: {{}}
 """)
 
+# Two .env keys for a base-domain move with no outage (HOSTNAME-CHANGE.md §B):
+#   ALT_BASE=new.base       every role site ALSO answers on its name under that base ("media.old, media.new {"),
+#                           so certificates for the new names are issued and both sets work while nothing else
+#                           has moved yet. Set with --alias NEW; cleared by --base.
+#   REDIRECT_BASE=old.base  after the move: the old names get their own blocks that 308 to the primary names, so
+#                           bookmarks and pinned links keep working instead of failing TLS. Set by --base.
+# Both are applied by rewriting the Caddyfile that write_caddy just produced; read_hosts understands both forms.
+def decorate_caddy():
+    env = read_env()
+    alt, red = env.get("ALT_BASE", "").strip().lower(), env.get("REDIRECT_BASE", "").strip().lower()
+    if not alt and not red:
+        return
+    hosts = read_hosts()
+    main = hosts.get("main", "")
+    base = main.split(".", 1)[1] if "." in main else ""
+    if not base:
+        return
+    role_hosts = {h for r, h in hosts.items() if r != "redirect"}
+    out = []
+    for line in Path("Caddyfile").read_text().splitlines():
+        m = re.match(r"^([A-Za-z0-9.-]+) \{$", line)
+        if m and alt and alt != base and m.group(1) in role_hosts and m.group(1).endswith("." + base):
+            line = f"{m.group(1)}, {m.group(1)[: -len(base)]}{alt} {{"
+        out.append(line)
+    text = "\n".join(out) + "\n"
+    if red and red != base:
+        for role, h in hosts.items():
+            if role != "redirect" and h.endswith("." + base):
+                old = h[: -len(base)] + red
+                text += f"\n# vanguard:redirect\n{old} {{\n\tredir https://{h}{{uri}} 308         # old base {red}: bookmarks follow\n}}\n"
+    Path("Caddyfile").write_text(text)
+    if alt and alt != base: print(f"   Caddyfile: every site also answers under {alt}")
+    if red and red != base: print(f"   Caddyfile: names under {red} redirect (308) to the new ones")
+
 def write_copyparty():
     """copyparty behind Caddy + Authentik: identity and groups arrive as headers; permissions are per group."""
     key = read_env().get("COPYPARTY_IDP_KEY", "key")
@@ -689,12 +722,13 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis
         if not yes("   continue anyway?", False): print("aborted; add streamers first ([s] in the menu)"); sys.exit(1)
     print("\n== writing configs")
     write_env(watch_host, discord_token, discord_guild, git_host, bot_host, live_host, domain, live_webhook, files_host, sync_host, clips_host, diag_host)
-    write_caddy(domain, auth_host, watch_host, git_host, bot_host, live_host, files_host, sync_host, clips_host, diag_host); write_mediamtx(domain); write_copyparty()
+    write_caddy(domain, auth_host, watch_host, git_host, bot_host, live_host, files_host, sync_host, clips_host, diag_host); decorate_caddy(); write_mediamtx(domain); write_copyparty()
     Path("stream-token.secret").write_text(read_env()["STREAM_TOKEN_SECRET"] + "\n"); os.chmod("stream-token.secret", 0o600)
     run(["sudo", "install", "-d", "-o", os.environ.get("SUDO_USER") or os.environ.get("USER") or "debian", "/srv/live"])
     print("== app data folders")
     run(["sudo", "install", "-d", "-o", "1000", "-g", "1000", "/srv/appdata/jellyfin/config", "/srv/appdata/jellyfin/cache",
          "/srv/appdata/authentik/media", "/srv/appdata/authentik/templates", "/srv/appdata/authentik/certs"])
+    install_brand_assets()       # authentik/brand/*.svg -> the login page's logo, favicon and background
     run(["sudo", "install", "-d", "/srv/appdata/authentik/db", "/srv/appdata/authentik/redis", "/srv/appdata/mariadb"])
     run(["sudo", "install", "-d", "-o", "1000", "-g", "1000", "/srv/appdata/discord-bot", "/srv/appdata/gitea", "/srv/appdata/act_runner"])
     print("== folders for legacy streamer entries (MediaMTX creates folders on first publish anyway)")
@@ -946,6 +980,198 @@ else:
             print(f"authentik: {provider.name!r} logout flow {before} -> {target.slug}")
 """
 
+# Login branding and "Discord forward". Everything the Authentik admin UI would want clicked for, done from the
+# ORM on every apply so it is reproducible and a UI save cannot quietly undo it:
+#   brand      title, logo, favicon, flow background, login-card CSS, root -> the media gateway (default application)
+#   identification stage   Discord source attached with its label (what akadmin sees behind ?admin=1)
+#   flow titles            the headings on the sign-in / signed-out / first-visit cards
+#   redirect stage         FIRST stage of default-authentication-flow, static target = the Discord source's login
+#                          URL, so a member never sees an Authentik page: protected link -> Discord consent -> back
+#   expression policy      on that binding: skip the redirect when the flow URL carries ?admin=1 — the executor
+#                          keeps the flow's query in the session under "authentik/flows/get", not in request.GET —
+#                          so https://auth.HOST/if/flow/default-authentication-flow/?admin=1 is the password card
+#                          for akadmin (linked from the diag landing dashboard, nowhere public)
+# The outpost's ?next= survives the detour: the source flow manager reads it back from the same session key.
+AK_BRANDING = r"""
+from urllib.parse import urlparse
+from authentik.brands.models import Brand
+from authentik.core.models import Application
+from authentik.flows.models import Flow, FlowStageBinding
+from authentik.policies.models import PolicyBinding
+from authentik.policies.expression.models import ExpressionPolicy
+from authentik.providers.proxy.models import ProxyProvider
+from authentik.sources.oauth.models import OAuthSource
+from authentik.stages.identification.models import IdentificationStage
+AUTH = "__AUTH_HOST__"; MAIN = "__MAIN_HOST__"; TITLE = "__TITLE__"
+ASSETS = "/media/public/brand"
+CSS = "\n".join([
+    ":root { --ak-accent: #5865F2; --pf-global--primary-color--100: #5865F2; --pf-global--primary-color--200: #4752c4; --pf-global--link--Color: #8ea1ff; --pf-global--link--Color--hover: #b4c0ff; }",
+    ".pf-c-login__main { background: rgba(20,20,20,.92); border: 1px solid #2e2e2e; border-radius: 14px; box-shadow: 0 20px 60px rgba(0,0,0,.6); }",
+    ".pf-c-login__main-header, .pf-c-login__main-body, .pf-c-login__main-footer, .pf-c-login__main-header .pf-c-title { color: #e6e6e6; }",
+    ".pf-c-button.pf-m-secondary, .pf-c-button.pf-m-primary { background: #5865F2; color: #fff; border: 0; border-radius: 10px; font-size: 1.05rem; padding: .85rem 1.2rem; }",
+    ".pf-c-button.pf-m-secondary:hover, .pf-c-button.pf-m-primary:hover { background: #4752c4; }",
+    ".pf-c-login__main-footer-band, .pf-c-login__footer { display: none; }",
+    ".pf-c-form-control { background: #1f1f1f; color: #e6e6e6; border-color: #2e2e2e; }",
+])
+
+def setattrs(obj, label, **kv):
+    changed = []
+    for key, value in kv.items():
+        if not hasattr(obj, key):
+            continue                                   # field absent in this authentik version: leave it
+        if getattr(obj, key) != value:
+            setattr(obj, key, value); changed.append(key)
+    if changed:
+        obj.save(); print(f"authentik: {label}: set {', '.join(changed)}")
+    else:
+        print(f"authentik: {label}: ok")
+
+# 1. brand
+brand = Brand.objects.filter(default=True).first() or Brand.objects.order_by("domain").first()
+if brand is None:
+    print("authentik: no brand found; branding skipped")
+else:
+    gateway = None
+    for pp in ProxyProvider.objects.all():
+        if urlparse(pp.external_host or "").hostname == MAIN:
+            gateway = Application.objects.filter(provider=pp).first()
+    extra = {"default_application": gateway} if gateway else {}
+    setattrs(brand, "brand", branding_title=TITLE, branding_logo=f"{ASSETS}/logo.svg", branding_favicon=f"{ASSETS}/favicon.svg",
+             branding_default_flow_background=f"{ASSETS}/login-bg.svg", flow_background=f"{ASSETS}/login-bg.svg",
+             branding_custom_css=CSS, **extra)
+    if gateway is None:
+        print(f"authentik: no proxy provider has external host {MAIN}; brand default application left alone")
+
+# 2. flow titles (and the per-flow background on versions that still keep it there)
+titles = {"default-authentication-flow": f"{TITLE} — sign in", "default-invalidation-flow": f"Signed out of {TITLE}",
+          "default-source-enrollment": f"Welcome to {TITLE}", "default-source-authentication": f"{TITLE} — signing you in",
+          "default-provider-invalidation-flow": f"Signed out of this app — still signed in to {TITLE}"}
+for slug, title in titles.items():
+    flow = Flow.objects.filter(slug=slug).first()
+    if flow is not None:
+        setattrs(flow, f"flow {slug}", title=title, background=f"{ASSETS}/login-bg.svg")
+
+# 3. Discord on the identification stage (the admin card still offers Discord too)
+discord = OAuthSource.objects.filter(provider_type="discord").first()
+if discord is None:
+    print("authentik: no Discord OAuth source; Discord forward skipped")
+else:
+    for stage in IdentificationStage.objects.filter(name="default-authentication-identification"):
+        if not stage.sources.filter(pk=discord.pk).exists():
+            stage.sources.add(discord); print("authentik: identification stage: Discord source attached")
+        setattrs(stage, "identification stage", show_source_labels=True)
+
+    # 4. the forward: redirect stage first in the authentication flow, skipped for ?admin=1
+    try:
+        from authentik.stages.redirect.models import RedirectStage
+    except ImportError:
+        RedirectStage = None
+        print("authentik: this version has no redirect stage; Discord forward skipped (branding applied)")
+    flow = Flow.objects.filter(slug="default-authentication-flow").first()
+    if RedirectStage is not None and flow is not None:
+        target = f"https://{AUTH}/source/oauth/login/{discord.slug}/"
+        stage, made = RedirectStage.objects.get_or_create(name="vanguard-discord-forward", defaults={"mode": "static", "target_static": target, "keep_context": True})
+        setattrs(stage, "redirect stage" + (" (created)" if made else ""), mode="static", target_static=target, keep_context=True)
+        expr = "\n".join([
+            "# Vanguard: members go straight to Discord. ?admin=1 on the flow URL shows the password card for akadmin.",
+            "# The executor stores the flow URL's query in the session, not in request.GET, so look in both.",
+            "http = request.http_request",
+            "if http is None:",
+            "    return True",
+            "if http.GET.get('admin') is not None:",
+            "    return False",
+            "flow_get = http.session.get('authentik/flows/get') or {}",
+            "return flow_get.get('admin') is None",
+            ""])
+        policy, made = ExpressionPolicy.objects.get_or_create(name="vanguard-discord-forward-unless-admin", defaults={"expression": expr})
+        setattrs(policy, "forward policy" + (" (created)" if made else ""), expression=expr, execution_logging=False)
+        others = list(FlowStageBinding.objects.filter(target=flow).exclude(stage=stage).values_list("order", flat=True))
+        order = min(others + [10]) - 10
+        binding, made = FlowStageBinding.objects.get_or_create(target=flow, stage=stage, defaults={"order": order})
+        setattrs(binding, "redirect binding" + (" (created)" if made else ""), order=order, evaluate_on_plan=True, re_evaluate_policies=False)
+        pb, made = PolicyBinding.objects.get_or_create(target=binding, policy=policy, defaults={"order": 0, "enabled": True})
+        setattrs(pb, "policy binding" + (" (created)" if made else ""), enabled=True, negate=False)
+        print(f"authentik: members -> {target} ; admins -> https://{AUTH}/if/flow/default-authentication-flow/?admin=1")
+"""
+
+def ak_branding(dc):
+    env, hosts = read_env(), read_hosts()            # the auth host lives only in the Caddyfile's vanguard:auth block
+    auth_host, main_host = hosts.get("auth", ""), hosts.get("main", "") or env.get("MAIN_HOST", "")
+    if not auth_host or not main_host:
+        print("   authentik: auth/main host not in the Caddyfile yet; branding skipped"); return
+    script = (AK_BRANDING.replace("__AUTH_HOST__", auth_host).replace("__MAIN_HOST__", main_host)
+                         .replace("__TITLE__", env.get("BRAND_TITLE", "Vanguard")))
+    ak_shell(dc, script, "login branding + Discord forward")
+
+def install_brand_assets():
+    # Brand assets for the Authentik login (authentik/brand/ in the repo). They go into Authentik's own media
+    # dir, served at https://auth.HOST/media/public/... with NO login — the login page is where they are shown,
+    # and every other host is behind the gate, so nothing else could serve them to a signed-out visitor.
+    run(["sudo", "install", "-d", "-o", "1000", "-g", "1000", "/srv/appdata/authentik/media/public/brand"])
+    for asset in sorted(Path("authentik/brand").glob("*.svg")):
+        run(["sudo", "install", "-m", "0644", "-o", "1000", "-g", "1000", str(asset), "/srv/appdata/authentik/media/public/brand/"])
+
+
+# Moving Authentik to a new base domain. NOT part of --apply, on purpose (HOSTNAME-CHANGE.md, "apps before
+# Authentik"): this is the step that decides whether the gated hosts are reachable, so it runs only when asked, after
+# Jellyfin and Gitea already answer on the new names. Every field that carries OLD is rewritten to NEW:
+#   outposts        config.authentik_host            (the forward-auth gate's login address)
+#   proxy providers external_host, cookie_domain     (domain-level: external_host IS the auth URL; cookie_domain = base)
+#   oauth2 providers redirect URIs                   (Jellyfin, Gitea callbacks)
+#   brand           domain, if it was the old base
+# Prints every change; a second run prints only "ok". --ak-hosts OLD NEW --dry-run shows without writing.
+AK_HOSTS = r"""
+import json, re
+OLD = "__OLD__"; NEW = "__NEW__"; DRY = __DRY__
+from authentik.outposts.models import Outpost
+from authentik.providers.proxy.models import ProxyProvider
+from authentik.providers.oauth2.models import OAuth2Provider
+from authentik.brands.models import Brand
+pat = re.compile(r"(?<![A-Za-z0-9-])" + re.escape(OLD) + r"(?![A-Za-z0-9-])")
+def swap(s): return pat.sub(NEW, s) if isinstance(s, str) else s
+def report(label, before, after):
+    if before == after: print(f"authentik: {label}: ok ({after})")
+    else: print(f"authentik: {label}: {before!r} -> {after!r}" + ("  [dry run]" if DRY else ""))
+for o in Outpost.objects.all():
+    cfg = o.config
+    before = getattr(cfg, "authentik_host", None)
+    if before is None:
+        continue
+    after = swap(before); report(f"outpost {o.name!r} authentik_host", before, after)
+    if after != before and not DRY:
+        cfg.authentik_host = after; o.config = cfg; o.save()
+for p in ProxyProvider.objects.all():
+    eh, cd = p.external_host, p.cookie_domain
+    eh2, cd2 = swap(eh), (NEW if cd == OLD else swap(cd))
+    report(f"proxy provider {p.name!r} external_host", eh, eh2); report(f"proxy provider {p.name!r} cookie_domain", cd, cd2)
+    if (eh2, cd2) != (eh, cd) and not DRY:
+        p.external_host, p.cookie_domain = eh2, cd2; p.save(update_fields=["external_host", "cookie_domain"])
+for p in OAuth2Provider.objects.all():
+    raw = list(p._redirect_uris or [])
+    new = [dict(u, url=swap(u.get("url", ""))) if isinstance(u, dict) else u for u in raw]
+    report(f"oauth2 provider {p.name!r} redirect_uris", [u.get("url") for u in raw if isinstance(u, dict)], [u.get("url") for u in new if isinstance(u, dict)])
+    if new != raw and not DRY:
+        p._redirect_uris = new; p.save(update_fields=["_redirect_uris"])
+for b in Brand.objects.all():
+    d2 = swap(b.domain); report(f"brand domain", b.domain, d2)
+    if d2 != b.domain and not DRY:
+        b.domain = d2; b.save(update_fields=["domain"])
+print("authentik: outposts re-read their config within a minute; the embedded outpost needs no restart")
+"""
+
+def ak_hosts(dc, old, new, dry=False):
+    script = AK_HOSTS.replace("__OLD__", old).replace("__NEW__", new).replace("__DRY__", "True" if dry else "False")
+    ak_shell(dc, script, f"authentik hosts {old} -> {new}" + (" (dry run)" if dry else ""))
+
+def rebase_hosts(hosts, old, new):
+    """Every role host with its label kept and the base swapped: media.old -> media.new."""
+    out = {}
+    for role, host in hosts.items():
+        if host == old: out[role] = new
+        elif host.endswith("." + old): out[role] = host[: -len(old)] + new
+        else: out[role] = host                       # a host on some other domain is left alone
+    return out
+
 def post_start_fixups(dc):
     """Idempotent first-run state that otherwise needs clicking: Syncthing GUI auth off, Gitea admin user, Jellyfin plugin."""
     env = read_env()
@@ -954,6 +1180,7 @@ def post_start_fixups(dc):
     # /logout must end the whole Authentik session, not just the application's. The provider-scoped flow
     # ("Logged out of application") leaves the SSO session alive, so logout looks like a page refresh.
     ak_shell(dc, AK_LOGOUT_FIXUP, "proxy provider logout flow")
+    ak_branding(dc)              # login page on brand, members forwarded straight to Discord, ?admin=1 for akadmin
     # Syncthing: no GUI login (the Authentik gate is the login); needs insecureAdminAccess since Caddy is not "localhost" to it
     cfg = Path("/srv/media/.syncthing/config.xml")
     if not DRY and cfg.exists():
@@ -1165,6 +1392,78 @@ def main():
             print('\nexample: ./configure.py --set ALERT_WEBHOOK_URL=https://discord.com/api/webhooks/…')
             sys.exit(1)
         set_env(dict(p.split("=", 1) for p in pairs)); return
+    if mode == "--alias":
+        # Step one of a no-outage base move: every site answers on BOTH bases. Only the Caddyfile changes
+        # (plus one .env key) and Caddy reloads; certificates for the new names are issued in the background.
+        # --alias "" takes the aliases away again. Needs the new names to resolve (certificates).
+        if len(sys.argv) != 3:
+            print('usage: configure.py --alias NEW-DOMAIN     (e.g. --alias vng.one ; --alias "" to remove)'); sys.exit(2)
+        if not domain: print("domain not set yet — run ./configure.py --full first"); sys.exit(1)
+        alt = sys.argv[2].strip().lower()
+        if alt:
+            for r, h in rebase_hosts(hosts, domain.split(".", 1)[1], alt).items():
+                if r in ("main", "auth", "watch", "git", "bot", "live", "files", "sync", "clips", "diag"):
+                    try: socket.getaddrinfo(h, 443)
+                    except OSError: print(f"   !! {h} does not resolve yet — add the DNS first"); sys.exit(1)
+        env0 = read_env(); env0["ALT_BASE"] = alt
+        Path(".env").write_text(render_env(env0)); os.chmod(".env", 0o600)
+        write_caddy(domain, auth_host, watch_host, git_host, bot_host, live_host, files_host, sync_host, clips_host, diag_host); decorate_caddy()
+        dc = compose_cmd()
+        if not DRY: run(dc + ["exec", "caddy", "caddy", "reload", "--config", "/etc/caddy/Caddyfile"])
+        print(f"   ALT_BASE={alt or '(cleared)'}; Caddy reloaded. Nothing else changed." + (f" Check: curl -sI https://auth.{alt}/ | head -1" if alt else ""))
+        return
+    if mode == "--base":
+        # Move every host to a new base domain in ONE apply, labels kept: --base vng.one turns media.old into
+        # media.vng.one, files.old into files.vng.one, … Needs the DNS to exist already (certificates). Does not
+        # touch the things that must come later (HOSTNAME-CHANGE.md §B): Jellyfin/Gitea UI settings, then
+        # --ak-hosts OLD NEW for Authentik, then the Discord portal.
+        if len(sys.argv) != 3 or "." not in sys.argv[2]:
+            print("usage: configure.py --base NEW-DOMAIN     (e.g. --base vng.one)"); sys.exit(2)
+        if not domain: print("domain not set yet — run ./configure.py --full first"); sys.exit(1)
+        old_base = domain.split(".", 1)[1] if "." in domain else domain
+        new_base = sys.argv[2].strip().lower()
+        moved = rebase_hosts(hosts, old_base, new_base)
+        unchanged = [f"{r}={h}" for r, h in hosts.items() if moved[r] == h]
+        print(f"== base domain {old_base} -> {new_base}")
+        for r in ("main", "auth", "watch", "git", "bot", "live", "files", "sync", "clips", "diag"):
+            if r in hosts: print(f"   {r:6} {hosts[r]:34} -> {moved[r]}")
+        if unchanged: print("   left alone (not under the old base):", ", ".join(unchanged))
+        env0 = read_env()
+        # the alias did its job; from now on the OLD names are redirects to the new ones (bookmarks keep working)
+        if env0.get("ALT_BASE", "").strip().lower() in (new_base, old_base): env0["ALT_BASE"] = ""
+        env0["REDIRECT_BASE"] = old_base
+        Path(".env").write_text(render_env(env0)); os.chmod(".env", 0o600)
+        print(f"   old names under {old_base} will 308 to the new ones (REDIRECT_BASE); --alias \"\" later removes nothing, edit REDIRECT_BASE= in .env to drop them")
+        apex = env0.get("APEX_HOST", "")
+        if apex and (apex == old_base or apex.endswith("." + old_base)):
+            # The bare old domain cannot stay: its Caddy block would keep asking for a certificate for a name that
+            # is no longer ours to serve. The bare NEW domain is not assumed either (it needs its own A record at
+            # the box, and today's does not point here): add it later with --full once that record exists.
+            print(f"   APEX_HOST={apex} -> cleared (bare domain of the old base; re-add the new one with --full once an A record points here)")
+            env0["APEX_HOST"] = ""; env0["APEX_REDIRECT"] = ""        # not a SETTABLE key: written directly, the apply below regenerates Caddy
+            Path(".env").write_text(render_env(env0)); os.chmod(".env", 0o600)
+        for r, h in moved.items():
+            if r in ("main", "auth", "watch", "git", "bot", "live", "files", "sync", "clips", "diag"):
+                try: socket.getaddrinfo(h, 443)
+                except OSError: print(f"   !! {h} does not resolve yet — Caddy cannot get its certificate; add the DNS first"); sys.exit(1)
+        if not yes(f"\napply now? Caddy, Jellyfin, Gitea, the runner, the bot and Kibana move to {new_base}; gated hosts stay unreachable until --ak-hosts"):
+            print("nothing written"); return
+        apply(moved["main"], admin_pw, streamers, editors, moved.get("auth", ""), moved.get("watch", ""), git_host=moved.get("git", ""), bot_host=moved.get("bot", ""),
+              live_host=moved.get("live", ""), files_host=moved.get("files", ""), sync_host=moved.get("sync", ""), clips_host=moved.get("clips", ""), diag_host=moved.get("diag", ""))
+        show(moved["main"], admin_pw, streamers, editors, moved.get("auth", ""), moved.get("watch", ""), moved.get("git", ""), moved.get("bot", ""), moved.get("live", ""), moved.get("files", ""), moved.get("sync", ""), moved.get("clips", ""), moved.get("diag", ""))
+        print(f"\nNEXT, in this order (HOSTNAME-CHANGE.md §B): docker compose restart mediamtx (between streams) → Jellyfin UI → Gitea UI + runner + clone remote → ./configure.py --ak-hosts {old_base} {new_base} → ./configure.py --ak-branding (brand default app follows the provider) → Discord portal → sudo vghosts-check.py")
+        return
+    if mode == "--ak-hosts":
+        # Authentik's hostnames, OLD -> NEW. Separate from --base and from --apply by design: it is the last
+        # service step of a domain move (apps before Authentik). --dry-run prints what would change.
+        rest = [a for a in sys.argv[2:] if not a.startswith("--")]
+        if len(rest) != 2:
+            print("usage: configure.py --ak-hosts OLD-DOMAIN NEW-DOMAIN [--dry-run]     (e.g. --ak-hosts dumpstervigilantes.rip vng.one)"); sys.exit(2)
+        ak_hosts(compose_cmd(), rest[0].strip().lower(), rest[1].strip().lower(), dry="--dry-run" in sys.argv); return
+    if mode == "--ak-branding":
+        # Just the Authentik login branding + Discord forward (also part of every --apply). On its own so a
+        # change to authentik/brand/*.svg or the CSS can be pushed without touching anything else.
+        install_brand_assets(); ak_branding(compose_cmd()); return
     if mode == "--es-security":
         # Just the Elasticsearch user bootstrap. Worth having on its own: if it is skipped or fails during an
         # --apply, Kibana reports "unavailable", Metricbeat exits on Kibana's 503 and Filebeat sits up but

deployLive.sh

@@ -2,7 +2,7 @@
 # Build the live portal here and push the static files to /srv/live on the server. Caddy serves them as-is.
 set -euo pipefail
 cd "$(dirname "$0")/live"
-HOST="${1:-debian@media.dumpstervigilantes.rip}"   # any name of the box works for ssh
+HOST="${1:-debian@media.vng.one}"   # any name of the box works for ssh
 npm install --no-audit --no-fund --loglevel=error
 npm run build
 ssh "$HOST" 'sudo install -d -o "$(id -u)" -g "$(id -g)" /srv/live && rm -rf /srv/live/*'

dock/index.html

@@ -30,6 +30,12 @@
   .kv .k{color:var(--mut)}
   .tok{letter-spacing:1px}
   .who{color:var(--mut);font-size:12px}
+  /* post-Go-Live health check: what the server is actually receiving */
+  #health{margin-top:10px;padding:8px;border-radius:6px;font-size:13px;white-space:pre-wrap;word-break:break-word}
+  #health.hidden{display:none}
+  #health.bad{background:#43242a;color:#ffd0d0;border:1px solid var(--bad)}
+  #health.bad code{background:#1a1a1a;padding:1px 5px;border-radius:4px;color:var(--fg)}
+  #health.ok{color:var(--ok)}
 </style>
 </head>
 <body>
@@ -53,6 +59,11 @@
     <label>Ingest host</label><input id="host" placeholder="media.example.com">
     <label>OBS WebSocket password</label><input id="obspw" type="password">
     <label>OBS WebSocket port</label><input id="obsport" value="4455">
+    <label>OBS profile to stream from</label><input id="profile" value="Vanguard">
+    <label style="display:flex;gap:8px;align-items:center;margin-top:8px"><input type="checkbox" id="enforce" style="width:auto;margin:0"> switch OBS to this profile when the dock loads</label>
+    <div class="who">Encoder settings (B-frames, keyframe interval) live in an OBS profile and obs-websocket cannot change them,
+    but it can switch profiles. Create one named <b>Vanguard</b> once: Profile → New, then Output → Advanced → your encoder
+    with <b>B-frames 0</b> and keyframe interval 2 s. Go Live always switches to it; this box also does it on startup.</div>
     <label>Title prefix (optional, e.g. <code>HelmetCam-</code>)</label><input id="prefix" placeholder="">
     <label>Title suffix (optional; <code>{date}</code> becomes YYYY-MM-DD)</label><input id="suffix" placeholder="-{date}">
     <button id="save">Save settings</button>
@@ -69,6 +80,7 @@
     <button class="small" id="tc_remove">Remove band</button>
   </details>
 
+  <div id="health" class="hidden"></div>
   <details id="manual">
     <summary>Manual OBS setup (endpoints + token)</summary>
     <div class="kv"><span class="k">RTMP server  </span><span id="m_rtmp">—</span></div>
@@ -85,11 +97,12 @@
 
 <script>
 // ---- settings --------------------------------------------------------------
-const KEYS = ["host","obspw","obsport","prefix","suffix"];
+const KEYS = ["host","obspw","obsport","prefix","suffix","profile"];
+const BOOLS = ["enforce"];
 const $ = id => document.getElementById(id);
 const status = (msg, cls) => { $("status").textContent = msg; $("status").style.color = cls==="bad"?"#e74c3c":cls==="ok"?"#2ecc71":"#9a9a9a"; };
-function load(){ try{ KEYS.forEach(k => { const v = localStorage.getItem("vg."+k); if (v!==null) $(k).value = v; }); }catch(e){} if(!$("host").value) $("host").value = location.hostname; }
+function load(){ try{ KEYS.forEach(k => { const v = localStorage.getItem("vg."+k); if (v!==null) $(k).value = v; }); BOOLS.forEach(k => { $(k).checked = localStorage.getItem("vg."+k) === "1"; }); }catch(e){} if(!$("host").value) $("host").value = location.hostname; }
-function save(){ try{ KEYS.forEach(k => localStorage.setItem("vg."+k, $(k).value.trim())); status("settings saved"); }catch(e){ status("could not save settings (storage blocked)", "bad"); } renderManual(); }
+function save(){ try{ KEYS.forEach(k => localStorage.setItem("vg."+k, $(k).value.trim())); BOOLS.forEach(k => localStorage.setItem("vg."+k, $(k).checked ? "1" : "0")); status("settings saved"); }catch(e){ status("could not save settings (storage blocked)", "bad"); } renderManual(); }
 load(); $("save").onclick = save;
 const cfg = () => Object.fromEntries(KEYS.map(k => [k, $(k).value.trim()]));
 
@@ -99,7 +112,7 @@ function buildTitle(raw){
   const c = cfg();
   let t = (c.prefix||"") + raw.trim() + (c.suffix||"").replace("{date}", today());
   t = t.replace(/\s+/g, "-")              // spaces -> dashes
-       .replace(/[^A-Za-z0-9_.-]/g, "")   // allowed characters only
+       .replace(/[^A-Za-z0-9_-]/g, "")    // allowed characters only; no dots: <title>.webrtc is the audio mirror's name
        .replace(/_{2,}/g, "_")            // "__" is reserved: it separates title from timestamp in filenames
        .replace(/^[.-]+/, "")             // no leading dot (hidden) or dash
        .slice(0, 64);
@@ -108,7 +121,7 @@ function buildTitle(raw){
 function ruleText(raw){
   const r = [];
   if (/__/.test(raw)) r.push("double underscore is reserved (becomes single)");
-  if (/[^A-Za-z0-9_.\- ]/.test(raw)) r.push("only letters, digits, space, _ . - are kept");
+  if (/[^A-Za-z0-9_\- ]/.test(raw)) r.push("only letters, digits, space, _ - are kept");
   if (raw.trim().startsWith(".")) r.push("cannot start with a dot");
   if ((cfg().prefix + raw + cfg().suffix).length > 64) r.push("longer than 64 characters: trimmed");
   return r.join(" · ");
@@ -177,6 +190,28 @@ function obs(c){
   });
 }
 
+// ---- profile enforcement ----------------------------------------------------------------------------
+// Encoder settings (B-frames above all: MediaMTX refuses them over WebRTC) cannot be set through
+// obs-websocket in Advanced output mode — they live in the profile's streamEncoder.json, which the protocol
+// does not expose. Profiles can be switched, though, so the setting is enforced by making sure OBS is on a
+// profile that already has it. The post-Go-Live health check then verifies the actual encode.
+// Returns true when OBS is on the wanted profile, false (with a status line saying why) otherwise.
+async function ensureProfile(o, c) {
+  if (!c.profile) return true;
+  const { profiles, currentProfileName } = await o.call("GetProfileList");
+  if (!profiles.includes(c.profile)) {
+    status(`OBS has no profile named "${c.profile}". Create it once (Profile → New) with B-frames 0, or clear the name in settings. Streaming on "${currentProfileName}".`, "bad");
+    return false;
+  }
+  if (currentProfileName === c.profile) return true;
+  const st = await o.call("GetStreamStatus");
+  if (st.outputActive) { status(`OBS is on "${currentProfileName}" but streaming; not switching mid-stream`, "bad"); return false; }
+  await o.call("SetCurrentProfile", { profileName: c.profile });
+  await new Promise(r => setTimeout(r, 800));            // OBS reloads its outputs on a profile switch
+  status(`OBS switched to profile "${c.profile}"`, "ok");
+  return true;
+}
+
 // ---- timecode band (obs-websocket) -------------------------------------------------------------------
 const CLOCK_INPUT = "Vanguard Clock";
 const bandFor = w => { let b = Math.round(w / 160); return b + (b % 2); };
@@ -260,15 +295,66 @@ $("golive").onclick = async () => {
     const t = await mint();                                   // fresh token every Go Live; nothing stored
     status("connecting to OBS…");
     const o = await obs(c);
+    await ensureProfile(o, c);                                // the Vanguard profile carries B-frames 0; switch first
     await o.call("SetStreamServiceSettings", { streamServiceType: "rtmp_custom",
       streamServiceSettings: { server: `rtmp://${c.host}:1935/`, key: `${t.user}/${title}?user=${encodeURIComponent(t.user)}&pass=${encodeURIComponent(t.token)}`, use_auth: false } });
+    // B-frames make the stream unwatchable on the portal (MediaMTX refuses them over WebRTC). obs-websocket
+    // can only set encoder options in Simple output mode with x264, so do that much here; everything else is
+    // caught after start by checkHealth(), which reads the actual recording on the server.
+    try {
+      const mode = (await o.call("GetProfileParameter", { parameterCategory: "Output", parameterName: "Mode" })).parameterValue;
+      if (mode === "Simple") {
+        const enc = (await o.call("GetProfileParameter", { parameterCategory: "SimpleOutput", parameterName: "StreamEncoder" })).parameterValue || "";
+        if (/x264/i.test(enc)) {
+          const cur = (await o.call("GetProfileParameter", { parameterCategory: "SimpleOutput", parameterName: "x264Settings" })).parameterValue || "";
+          if (!/\b(bf|bframes)=0\b/.test(cur)) {
+            await o.call("SetProfileParameter", { parameterCategory: "SimpleOutput", parameterName: "x264Settings", parameterValue: (cur + " bf=0").trim() });
+            status("set x264 bf=0 in OBS (Simple output) so viewers can watch over WebRTC");
+          }
+        }
+      }
+    } catch (e) { /* older obs-websocket, or a locked profile: the health check will say if it matters */ }
     const st = await o.call("GetStreamStatus");
     if (st.outputActive) { await o.call("StopStream"); await new Promise(r => setTimeout(r, 1500)); }
     await o.call("StartStream"); o.close();
     status(`LIVE  →  ${t.user}/${title}`, "ok");
+    checkHealth(90);
   } catch (e) { status(e.message, "bad"); }
   $("golive").disabled = false;
 };
+// After Go Live, ask the server what it is actually receiving. The recording's first fragments reveal the
+// B-frame depth within seconds, and B-frames mean every WebRTC viewer is refused. Polls until the stream
+// shows up and is judged, or for `seconds`, whichever comes first.
+let healthTimer = null;
+async function checkHealth(seconds) {
+  clearInterval(healthTimer);
+  const until = Date.now() + seconds * 1000, box = $("health");
+  box.className = "who"; box.textContent = "checking what the server is receiving…";
+  healthTimer = setInterval(async () => {
+    try {
+      const r = await fetch("/me/stream-health", { credentials: "same-origin" });
+      if (!r.ok) throw new Error("HTTP " + r.status);
+      const d = await r.json();
+      const judged = (d.streams || []).filter(s => s.bframes != null);
+      if (judged.length) {
+        clearInterval(healthTimer);
+        const bad = judged.filter(s => !s.webrtc_ok);
+        if (bad.length) {
+          box.className = "bad";
+          box.innerHTML = `<b>Viewers cannot watch this stream over WebRTC.</b> Your encoder is sending B-frames `
+            + `(${bad.map(s => `${s.profile || "H.264"} ${s.res || ""}, B-frame depth ${s.bframes}`).join("; ")}). `
+            + `In OBS: Output → Advanced → <b>x264</b>: add <code>bf=0</code> to the options; <b>NVENC</b>: Max B-frames = 0; `
+            + `<b>AMF/QSV</b>: B-frames = 0. Then stop and start the stream. The recording itself is fine.`;
+        } else {
+          box.className = "ok";
+          box.textContent = `server receiving ${judged.map(s => `${s.res || ""} ${s.fps ? s.fps + " fps" : ""}`.trim()).join(", ")} with no B-frames — WebRTC viewers are good`;
+        }
+        return;
+      }
+    } catch (e) { /* not up yet, or the dock is not behind the proxy; keep trying until the deadline */ }
+    if (Date.now() > until) { clearInterval(healthTimer); box.className = "who"; box.textContent = "could not confirm what the server is receiving (is the stream up?)"; }
+  }, 6000);
+}
 $("stop").onclick = async () => { try { const o = await obs(cfg()); await o.call("StopStream"); o.close(); status("stream stopped"); } catch (e) { status(e.message, "bad"); } };
 
 let armed = 0;
@@ -290,6 +376,7 @@ $("dump").onclick = async () => {
   } catch (e) { status("dump failed: " + e.message, "bad"); }
 };
 
+
 $("signout").onclick = async e => {
   e.preventDefault(); status("signing out…");
   // Authentik's sign-out redirects to the site root (copyparty); do it in the background, then reload the dock,
@@ -304,6 +391,11 @@ $("signout").onclick = async e => {
   await whoami();
   try { await mint(); status("ready"); }
   catch (e) { status(e.message, "bad"); $("m_tok").textContent = "— " + e.message; }
+  // "switch OBS to this profile when the dock loads": inside OBS the dock loads with OBS, so this is startup
+  if ($("enforce").checked) {
+    try { const o = await obs(cfg()); await ensureProfile(o, cfg()); o.close(); }
+    catch (e) { status(`profile check on startup: ${e.message}`, "bad"); }
+  }
 })();
 renderPreview();
 </script>

docker-compose.yml

@@ -15,15 +15,13 @@ services:
       TZ: America/New_York          # recording filenames are stamped in the container's clock; must match the host
       LIVE_WEBHOOK_URL: ${LIVE_WEBHOOK_URL:-}
       LIVE_HOST: ${LIVE_HOST:-live.example.com}
-    ports:
+    # Host network, as MediaMTX recommends for WebRTC. On the bridge, every WebRTC media packet on 8189/udp
-      - "1935:1935"              # RTMP ingest
+    # was relayed by Docker's userland proxy (docker-proxy), which lost ~0.4% of them at a thousand packets a
-      - "8890:8890/udp"          # SRT ingest
+    # second; each loss corrupted an access unit, the recorder reopened its file, and one evening produced 471
-      - "8189:8189/udp"          # WebRTC media (viewers + Go Live)
+    # fragments. Publishers also all appeared as the bridge gateway (172.18.0.1). Now MediaMTX binds the host
-      - "8189:8189/tcp"          # WebRTC ICE-TCP fallback
+    # directly: ingest ports (1935, 8890/udp, 8189) are open in the nftables INPUT chain, and everything Caddy
-      - "127.0.0.1:8888:8888"    # HLS      -> Caddy live.HOST/hls/*
+    # or Metricbeat talks to is bound to 127.0.0.1 in mediamtx.yml. No ports: block, nothing to publish.
-      - "127.0.0.1:8889:8889"    # WebRTC   -> Caddy live.HOST/webrtc/*
+    network_mode: host
-      - "127.0.0.1:9997:9997"    # API      -> Caddy live.HOST/api/*
-      - "127.0.0.1:9998:9998"    # Prometheus metrics (per-stream bitrate, readers, drops) -> scraper on the host
     volumes:
       - ./mediamtx.yml:/mediamtx.yml:ro
       - ./bin/mtx-on-ready.sh:/vanguard/on-ready.sh:ro   # is-live notice + WebRTC audio mirror (runOnReady)
@@ -72,7 +70,7 @@ services:
       - ./dock:/srv/dock:ro
       - ./clips:/srv/clips:ro               # clip-intake self-service page
       - ./about:/srv/about:ro               # gateway page (/about)
-      - ./stuck:/srv/stuck:ro               # stuck-recording triage page (main host /stuck)
+      - ./streaming:/srv/streaming:ro       # Live + Stuck tabs (diag host /streaming)
       - /srv/live:/srv/live:ro              # live portal static build (deployLive.sh)
       - caddy_data:/data
       - caddy_config:/config

firewall/nftables.conf

@@ -1,6 +1,7 @@
 #!/usr/sbin/nft -f
 # vanguard-stream host firewall. Own table so (re)applying never disturbs Docker's rules.
-# Input only: Docker-published ports (1935, 8890, 22000) arrive via Docker's forward chain and are
+# Input only. MediaMTX runs on the host network, so 1935, 8890/udp and 8189 arrive through THIS chain (the
+# accepts below). Docker-published ports (22000) arrive via Docker's forward chain and are
 # listed here for the host-network case (Caddy) and clarity. Nothing else should listen publicly.
 
 destroy table inet vanguard

live/src/GoLive.jsx

@@ -1,5 +1,5 @@
 import React, { useEffect, useRef, useState } from 'react'
-import { captureWindow, publish, fetchStreamKey, defaultTitle, cleanTitle, withTimecodeBand, MAX_KBPS } from './broadcast.js'
+import { captureWindow, publish, fetchStreamKey, defaultTitle, cleanTitle, withTimecodeBand, MAX_KBPS, windowLoss, shouldSwitchToTcp } from './broadcast.js'
 
 // Broadcaster view: replaces the viewer grid while you're live.
 export default function GoLive({ onExit }) {
@@ -16,6 +16,81 @@ export default function GoLive({ onExit }) {
   const [hasAudio, setHasAudio] = useState(true)
   const [tcBand, setTcBand] = useState(() => { try { return localStorage.getItem('vg.tcband') !== '0' } catch { return true } })
   const bandRef = useRef(null)
+  const streamRef = useRef(null)                      // the tracks being published (survive a transport switch)
+  const samplesRef = useRef([])                       // recent {t, lost, sent} for the loss window
+  const switchingRef = useRef(false)
+  const [transport, setTransport] = useState('udp')
+  const [switched, setSwitched] = useState('')        // "switched to TCP after 0.6% loss" once it happens
+  // Remembered per browser. Starlink drops packets in bursts on every satellite handoff (its scheduler works in
+  // ~15 s cycles), so a dish user hits the auto-switch on every single stream; once it has fired, the next stream
+  // starts on TCP and skips the 20 s of corrupted recording and the extra file. Untick in setup to go back.
+  const [preferTcp, setPreferTcp] = useState(() => { try { return localStorage.getItem('vg.golive.tcp') === '1' } catch { return false } })
+  const [ended, setEnded] = useState('')               // why the server ended the stream, shown back on the setup card
+  const [surface, setSurface] = useState(() => { try { return localStorage.getItem('vg.golive.surface') === 'screen' ? 'screen' : 'window' } catch { return 'window' } })
+  const pickSurface = v => { setSurface(v); try { localStorage.setItem('vg.golive.surface', v) } catch {} }
+  const missingRef = useRef(0)                          // consecutive health polls that did not list our path
+  const disconnectRef = useRef(null)
+
+  // A kick is invisible to WebRTC for a long time: the browser sits in "disconnected" until ICE consent times
+  // out (~30 s) and only then goes "failed". So: treat failed/closed as the end immediately, give
+  // "disconnected" 8 s to recover, and ask the server every 5 s whether our path is still live — the kick
+  // path answers that in seconds and tells us whether we are blocked.
+  const endedByServer = async (why) => {
+    if (!sessionRef.current) return
+    const s = sessionRef.current; sessionRef.current = null
+    try { await s.stop() } catch {}
+    streamRef.current?.getTracks().forEach(t => t.stop()); streamRef.current = null
+    bandRef.current?.stop(); bandRef.current = null
+    if (videoRef.current) videoRef.current.srcObject = null
+    setPhase('setup'); setStarted(null); setStats(null); setConn(''); setTransport('udp'); setSwitched('')
+    setEnded(why)
+  }
+  const onConn = (state) => {
+    setConn(state)
+    if (switchingRef.current) return
+    if (state === 'failed' || state === 'closed') endedByServer('The server ended the stream.')
+    if (state === 'disconnected' && !disconnectRef.current) {
+      disconnectRef.current = setTimeout(() => { disconnectRef.current = null; if (sessionRef.current?.pc?.connectionState !== 'connected') endedByServer('Connection to the server was lost.') }, 8000)
+    }
+    if (state === 'connected' && disconnectRef.current) { clearTimeout(disconnectRef.current); disconnectRef.current = null }
+  }
+  useEffect(() => {
+    if (phase !== 'live') return
+    missingRef.current = 0
+    const t = setInterval(async () => {
+      const s = sessionRef.current; if (!s || switchingRef.current) return
+      try {
+        const r = await fetch('/me/stream-health', { credentials: 'same-origin' }); if (!r.ok) return
+        const d = await r.json()
+        const mine = (d.streams || []).some(x => x.path.toLowerCase() === s.path.toLowerCase())
+        missingRef.current = mine ? 0 : missingRef.current + 1
+        if (missingRef.current >= 2) {
+          endedByServer(d.blocked ? `An admin stopped your stream. Publishing is paused for your account until ${new Date(d.blocked.until).toLocaleTimeString()}.`
+                                  : 'The server ended the stream.')
+        }
+      } catch {}
+    }, 5000)
+    return () => clearInterval(t)
+  }, [phase])
+  const rememberTcp = on => { setPreferTcp(on); try { localStorage.setItem('vg.golive.tcp', on ? '1' : '0') } catch {} }
+
+  // Lossy uplink: re-publish the same tracks over TCP. One new recording file (the sweep stitches it), ~2 s
+  // of picture lost during the swap, and from then on no packet loss can reach the recorder. Never flips back:
+  // the latency cost of TCP is a nuisance, a corrupted recording is a loss.
+  const switchToTcp = async (loss) => {
+    const old = sessionRef.current; if (!old || switchingRef.current || old.transport === 'tcp') return
+    switchingRef.current = true
+    setConn(`loss ${loss.pct.toFixed(1)}% — switching to TCP…`)
+    try {
+      await old.release()
+      const next = await publish({ stream: streamRef.current, path: old.path, user: old.user, pass: old.pass, onState: onConn, transport: 'tcp' })
+      sessionRef.current = next; samplesRef.current = []
+      setTransport(next.transport)
+      if (next.transport === 'tcp') { rememberTcp(true); setSwitched(`switched to TCP after ${loss.pct.toFixed(1)}% loss — your next streams start on TCP`) }
+      else setSwitched('server offered no TCP candidate; still on UDP')
+    } catch (e) { setErr(`re-publish over TCP failed: ${e.message}`); setPhase('error') }
+    finally { switchingRef.current = false }
+  }
 
   useEffect(() => { fetchStreamKey().then(setKey).catch(e => { setErr(e.message); setPhase('error') }) }, [])
   useEffect(() => { if (!started) return; const t = setInterval(() => setElapsed(Math.floor((Date.now() - started) / 1000)), 1000); return () => clearInterval(t) }, [started])
@@ -25,11 +100,21 @@ export default function GoLive({ onExit }) {
     const t = setInterval(async () => {
       const pc = sessionRef.current?.pc; if (!pc) return
       const report = await pc.getStats()
-      let codec = ''
+      let codec = '', lost = 0, sent = 0
+      for (const s of report) if (s.type === 'remote-inbound-rtp' && s.kind === 'video') lost = s.packetsLost || 0   // what the server reports back
       for (const s of report) if (s.type === 'outbound-rtp' && s.kind === 'video') {
+        sent = s.packetsSent || 0
         const now = Date.now(); const kbps = Math.round((s.bytesSent - last.bytes) * 8 / ((now - last.t) || 1)); last = { bytes: s.bytesSent, t: now }
         const c = s.codecId && report.get(s.codecId); codec = c ? `${c.mimeType.replace('video/', '')}${/profile-level-id=64/i.test(c.sdpFmtpLine || '') ? ' High' : /profile-level-id=4d/i.test(c.sdpFmtpLine || '') ? ' Main' : /profile-level-id=42/i.test(c.sdpFmtpLine || '') ? ' Baseline' : ''}` : ''
-        setStats({ kbps, fps: Math.round(s.framesPerSecond || 0), w: s.frameWidth || 0, h: s.frameHeight || 0, q: s.qualityLimitationReason, enc: s.encoderImplementation || '', codec })
+        // nack = retransmit requests the server sent us; rtx = packets we re-sent. Both 0 under loss means the
+        // server is not asking for retransmits, and every lost packet is a corrupted frame in the recording.
+        setStats({ kbps, fps: Math.round(s.framesPerSecond || 0), w: s.frameWidth || 0, h: s.frameHeight || 0, q: s.qualityLimitationReason, enc: s.encoderImplementation || '', codec,
+                   lost, sent, nack: s.nackCount || 0, rtx: s.retransmittedPacketsSent || 0 })
+      }
+      if (sent && !switchingRef.current) {
+        const samples = samplesRef.current; samples.push({ t: Date.now(), lost, sent }); if (samples.length > 60) samples.shift()
+        const l = windowLoss(samples)
+        if (sessionRef.current?.transport === 'udp' && shouldSwitchToTcp(l)) switchToTcp(l)
       }
     }, 1000)
     return () => clearInterval(t)
@@ -41,19 +126,25 @@ export default function GoLive({ onExit }) {
     const t = cleanTitle(title); if (!t) return setErr('give it a title')
     if (!key) return setErr('no stream key')
     try {
+      // blocked by an admin? MediaMTX would answer the WHIP with a bare 401, which reads like a bad key — ask first
+      try { const h = await (await fetch('/me/stream-health', { credentials: 'same-origin' })).json()
+            if (h.blocked) return setErr(`Publishing is paused for your account by an admin until ${new Date(h.blocked.until).toLocaleTimeString()}.`) } catch {}
       setPhase('capturing')
-      const captured = await captureWindow()
+      const captured = await captureWindow(surface)
       setHasAudio(captured.getAudioTracks().length > 0)
       captured.getVideoTracks()[0].onended = () => stop()            // user clicked Chrome's "Stop sharing"
       let stream = captured
       if (tcBand) { bandRef.current = withTimecodeBand(captured); stream = bandRef.current.stream }
       if (videoRef.current) { videoRef.current.srcObject = stream; videoRef.current.muted = true; videoRef.current.play().catch(() => {}) }
-      const session = await publish({ stream, path: `${key.user}/${t}`, user: key.user, pass: key.pass, onState: setConn })
+      streamRef.current = stream; samplesRef.current = []; setSwitched('')
-      sessionRef.current = session
+      setEnded('')
+      const session = await publish({ stream, path: `${key.user}/${t}`, user: key.user, pass: key.pass, onState: onConn, transport: preferTcp ? 'tcp' : 'udp' })
+      sessionRef.current = session; setTransport(session.transport)
+      if (preferTcp && session.transport !== 'tcp') setSwitched('server offered no TCP candidate; on UDP')
       setStarted(Date.now()); setPhase('live')
     } catch (e) { setErr(e.message); setPhase(e.message === 'capture cancelled' ? 'setup' : 'error') }
   }
-  const stop = async () => { await sessionRef.current?.stop(); sessionRef.current = null; bandRef.current?.stop(); bandRef.current = null; if (videoRef.current) videoRef.current.srcObject = null; setPhase('setup'); setStarted(null); setStats(null); setConn('') }
+  const stop = async () => { await sessionRef.current?.stop(); sessionRef.current = null; streamRef.current?.getTracks().forEach(t => t.stop()); streamRef.current = null; bandRef.current?.stop(); bandRef.current = null; if (videoRef.current) videoRef.current.srcObject = null; setPhase('setup'); setStarted(null); setStats(null); setConn(''); setTransport('udp'); setSwitched('') }
 
   const mmss = s => `${String(Math.floor(s / 3600)).padStart(2, '0')}:${String(Math.floor(s / 60) % 60).padStart(2, '0')}:${String(s % 60).padStart(2, '0')}`
   return (
@@ -69,7 +160,7 @@ export default function GoLive({ onExit }) {
         <video ref={videoRef} autoPlay playsInline muted className={phase === 'live' || phase === 'capturing' ? 'show' : ''} />
         {phase !== 'live' && (
           <div className="golive-form">
-            <label>Stream title <small>(becomes the Jellyfin title; letters, digits, _ . - ; no double underscore)</small></label>
+            <label>Stream title <small>(becomes the Jellyfin title; letters, digits, _ - ; no double underscore)</small></label>
             <input value={title} onChange={e => setTitle(e.target.value)} disabled={phase === 'capturing'} />
             <div className="hint" style={{marginTop: '-4px'}}>stream name: <b>{key ? key.user : '<you>'}/{cleanTitle(title) || '—'}</b></div>
             <div className="hint">
@@ -78,11 +169,28 @@ export default function GoLive({ onExit }) {
               encoder. On Linux and Mac Chrome has no window audio and (on NVIDIA) no hardware encoder, so the stream is
               silent and visibly soft — use OBS with the dock there. Capped at {MAX_KBPS / 1000} Mbps, up to 1440p60, H.264.
             </div>
+            <div className="hint" style={{display:'flex',gap:16,alignItems:'center',flexWrap:'wrap'}}>
+              <span>Capture:</span>
+              <label style={{display:'flex',gap:6,alignItems:'center'}}><input type="radio" name="surface" checked={surface === 'window'} onChange={() => pickSurface('window')} /> the game's window</label>
+              <label style={{display:'flex',gap:6,alignItems:'center'}}><input type="radio" name="surface" checked={surface === 'screen'} onChange={() => pickSurface('screen')} /> the whole screen</label>
+            </div>
+            <div className="hint">
+              <b>Frozen picture that only updates when you alt-tab?</b> The game is in <i>exclusive</i> full-screen, which
+              window capture cannot see. Either set the game to <b>borderless / windowed full-screen</b> (best: keeps
+              window audio), or pick <b>the whole screen</b> above (always works; carries system audio, and shows
+              whatever else is on that monitor).
+            </div>
             <label className="hint" style={{display:'flex',gap:8,alignItems:'center'}}>
               <input type="checkbox" checked={tcBand} onChange={e => { setTcBand(e.target.checked); try { localStorage.setItem('vg.tcband', e.target.checked ? '1' : '0') } catch {} }} />
               <span>Timecode band: add a thin strip below the picture carrying the server clock, so editors get a
               frame-accurate timecode track. Viewers here never see it; Jellyfin shows a thin line at the bottom.</span>
             </label>
+            <label className="hint" style={{display:'flex',gap:8,alignItems:'center'}}>
+              <input type="checkbox" checked={preferTcp} onChange={e => rememberTcp(e.target.checked)} />
+              <span>Lossy uplink (Starlink, cellular, bad wifi): send over TCP from the start. Nothing gets lost on the
+              way to the recording; the cost is a second or two of extra delay for viewers. Ticks itself the first
+              time a stream has to switch.</span>
+            </label>
             <div className="warn-box">
               <b>⚠ This stream is recorded.</b> Everything the window shows and every sound it makes is saved to the
               server, visible to editors and members in Jellyfin, and kept until it is dumped or archived. Close
@@ -90,7 +198,8 @@ export default function GoLive({ onExit }) {
               should be thrown away (3-day undo).
             </div>
             {err && <div className="err">{err}</div>}
-            <button className="primary" disabled={!key || phase === 'capturing'} onClick={start}>{phase === 'capturing' ? 'Waiting for the window picker…' : 'Start'}</button>
+            {ended && <div className="warn-box"><b>Stream ended.</b> {ended} {/paused/.test(ended) ? 'Starting again before then will be refused.' : 'You can start again.'}</div>}
+            <button className="primary" disabled={!key || phase === 'capturing'} onClick={start}>{phase === 'capturing' ? 'Waiting for the picker…' : 'Start'}</button>
           </div>
         )}
         {phase === 'live' && (
@@ -99,8 +208,9 @@ export default function GoLive({ onExit }) {
             <span className="rec" title="this stream is being recorded to the server">● REC</span>
             <span>{mmss(elapsed)}</span>
             <span>{key.user}/{cleanTitle(title)}</span>
-            <span>{conn}</span>
+            <span>{conn}{transport === 'tcp' ? ' · via TCP' : ''}</span>
-            {stats && <span>{stats.w}×{stats.h} · {stats.fps} fps · {(stats.kbps / 1000).toFixed(1)} Mbps · {stats.codec}{stats.enc ? ` (${/openh264|libvpx|software|SimulcastEncoderAdapter/i.test(stats.enc) ? 'software' : 'hardware'}: ${stats.enc})` : ''}{stats.q && stats.q !== 'none' ? ` · limited by ${stats.q}` : ''}</span>}
+            {switched && <span className="warn" title="packet loss on your uplink was corrupting the recording; TCP retransmits everything, at the cost of some latency">{switched}</span>}
+            {stats && <span>{stats.w}×{stats.h} · {stats.fps} fps · {(stats.kbps / 1000).toFixed(1)} Mbps · {stats.codec}{stats.enc ? ` (${/openh264|libvpx|software|SimulcastEncoderAdapter/i.test(stats.enc) ? 'software' : 'hardware'}: ${stats.enc})` : ''}{stats.q && stats.q !== 'none' ? ` · limited by ${stats.q}` : ''}{stats.sent ? ` · loss ${(100 * stats.lost / stats.sent).toFixed(2)}% (${stats.nack} nack, ${stats.rtx} resent)` : ''}</span>}
             {!hasAudio && <span className="warn">no audio captured — restart and tick "Share audio"</span>}
           </div>
         )}

live/src/broadcast.js

@@ -10,18 +10,25 @@ export function defaultTitle(now = new Date()) {
   return `HelmetCam-${d}`
 }
 // Must match the dock and the server: the sweep publishes <Title>__<timestamp>.mp4, so "__" is reserved.
-export const cleanTitle = t => t.trim().replace(/\s+/g, '-').replace(/[^A-Za-z0-9_.-]/g, '').replace(/_{2,}/g, '_').replace(/^[.-]+/, '').slice(0, 64)
+// No dots: the server names the WebRTC audio mirror <title>.webrtc, so a dot in a title would let a title
+// collide with a mirror name and pick up the wrong MediaMTX rule (one such collision recorded 36 GB twice).
+export const cleanTitle = t => t.trim().replace(/\s+/g, '-').replace(/[^A-Za-z0-9_-]/g, '').replace(/_{2,}/g, '_').replace(/^[.-]+/, '').slice(0, 64)
 
-export async function captureWindow() {
+// surface: 'window' (default) or 'screen'. Window capture on Windows goes through the compositor, and a game in
-  // Window-only picker with that window's own audio. Chrome/Edge 141+ honour windowAudio: "window"
+// EXCLUSIVE full-screen bypasses the compositor: Chrome then gets a frame only when Windows composites one, i.e.
-  // (only the shared window's sound); older Chrome offers no audio for a window at all.
+// on alt-tab — Breezy's "one frame, then another when I alt-tab". Two ways out: the game in borderless/windowed
+// full-screen (compositor sees every frame), or capture the whole screen, which Chrome reads straight from the
+// GPU output (desktop duplication) and which exclusive full-screen cannot hide from. Screen capture carries
+// system audio instead of one window's audio.
+export async function captureWindow(surface = 'window') {
+  const screen = surface === 'screen'
   const constraints = {
-    video: { displaySurface: 'window', frameRate: { ideal: 60, max: 60 }, width: { max: 2560 }, height: { max: 1440 } },
+    video: { displaySurface: screen ? 'monitor' : 'window', frameRate: { ideal: 60, max: 60 }, width: { max: 2560 }, height: { max: 1440 } },
     audio: { echoCancellation: false, noiseSuppression: false, autoGainControl: false, sampleRate: 48000, channelCount: 2 },
     windowAudio: 'window',            // capture the picked window's audio (Chrome 141+)
-    systemAudio: 'exclude',
+    systemAudio: screen ? 'include' : 'exclude',
     selfBrowserSurface: 'exclude',
-    monitorTypeSurfaces: 'exclude',   // hide "Entire screen": this is a single-window helmet cam
+    monitorTypeSurfaces: screen ? 'include' : 'exclude',   // "Entire screen" only when asked for: a helmet cam is one window
     surfaceSwitching: 'include',
     preferCurrentTab: false,
   }
@@ -40,6 +47,12 @@ export async function captureWindow() {
 // skips the painfully slow cold ramp without disabling congestion control.
 export const MIN_KBPS = 1200
 export const START_KBPS = 9000
+// On TCP nothing is lost, so the only thing Chrome's estimator sees is delay — and a Starlink dish swings its
+// delay on every handoff, which reads as congestion and parks the bitrate near the floor: stable but soft.
+// A higher floor is safe HERE because the failure mode a floor causes on UDP (sending into loss) cannot happen
+// on TCP; if the uplink truly cannot carry it, latency grows instead of frames corrupting, which is the trade
+// the TCP switch already made. Still well under a dish's typical 10-20 Mbps up.
+export const TCP_MIN_KBPS = 4000
 
 function preferCodecs(pc, kind = 'video') {
   const caps = RTCRtpSender.getCapabilities?.(kind)
@@ -60,10 +73,10 @@ function preferCodecs(pc, kind = 'video') {
 
 // Chrome honours x-google-* fmtp hints. start-bitrate skips the slow cold ramp, max-bitrate is the ceiling,
 // and min-bitrate is a floor that must stay LOW: a high floor defeats congestion control entirely.
-function mungeBitrates(sdp) {
+function mungeBitrates(sdp, minKbps = MIN_KBPS) {
   return sdp.replace(/^a=fmtp:(\d+) (.*)$/gm, (line, pt, params) =>
     /packetization-mode|profile-id|VP9/i.test(params) || /H264|VP9/i.test(sdp.match(new RegExp(`a=rtpmap:${pt} .*`))?.[0] || '')
-      ? `a=fmtp:${pt} ${params};x-google-start-bitrate=${START_KBPS};x-google-min-bitrate=${MIN_KBPS};x-google-max-bitrate=${MAX_KBPS}`
+      ? `a=fmtp:${pt} ${params};x-google-start-bitrate=${START_KBPS};x-google-min-bitrate=${minKbps};x-google-max-bitrate=${MAX_KBPS}`
       : line)
 }
 
@@ -90,7 +103,23 @@ export function withTimecodeBand(stream) {
   return { stream: out, stop: () => { running = false; clearInterval(t); video.srcObject = null } }
 }
 
-export async function publish({ stream, path, user, pass, onState }) {
+// Transport. 'udp' is WebRTC as normal. 'tcp' keeps only MediaMTX's TCP candidates (webrtcLocalTCPAddress) in
+// the answer, so ICE can pair with nothing else and the media rides a TCP connection: no packet loss at all,
+// at the cost of latency spikes when the uplink hiccups. The recording cares about the former, not the latter.
+// MediaMTX requests retransmits (NACK, pion's ConfigureNack) either way, but a wifi burst that drops more
+// packets than its reorder window holds still reaches the recorder as a broken frame; TCP cannot.
+const LOSS_SWITCH_PCT = 0.3          // sustained loss above this (over LOSS_WINDOW_S) flips a publisher to TCP
+const LOSS_WINDOW_S = 20
+const LOSS_MIN_PACKETS = 1500        // don't judge a 2-second-old session
+
+function tcpOnly(sdp) {
+  const lines = sdp.split('\r\n')
+  const out = lines.filter(l => !(l.startsWith('a=candidate:') && / udp /i.test(l)))
+  if (!out.some(l => l.startsWith('a=candidate:'))) return null       // no TCP candidate offered: cannot do it
+  return out.join('\r\n')
+}
+
+export async function publish({ stream, path, user, pass, onState, transport = 'udp' }) {
   const pc = new RTCPeerConnection({ iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] })
   for (const track of stream.getTracks()) {
     if (track.kind === 'video') { try { track.contentHint = 'motion' } catch {} }   // game footage: favour frame rate + temporal quality
@@ -102,25 +131,34 @@ export async function publish({ stream, path, user, pass, onState }) {
     const p = vs.getParameters(); p.encodings = p.encodings?.length ? p.encodings : [{}]
     p.encodings[0].maxBitrate = MAX_KBPS * 1000       // a ceiling; congestion control may go far below it
     p.encodings[0].maxFramerate = 60
-    // No scaleResolutionDownBy: pinning it to 1 forbids the encoder from shedding resolution under
+    // Under pressure Chrome must shed FRAME RATE, never resolution. A resolution change is a new SPS, and
-    // pressure, which leaves it nothing to give but packets it cannot deliver.
+    // MediaMTX opens a new recording file on every parameter change — on a lossy uplink that turned one
-    p.degradationPreference = 'balanced'
+    // stream into a row of files the sweep cannot copy-join (different sizes). Frame-rate drops cost
+    // nothing on the recording side. (No scaleResolutionDownBy pin: that would forbid shedding entirely.)
+    p.degradationPreference = 'maintain-resolution'
     await vs.setParameters(p).catch(() => {})
   }
   pc.onconnectionstatechange = () => onState?.(pc.connectionState)
   const offer = await pc.createOffer()
-  await pc.setLocalDescription({ type: 'offer', sdp: mungeBitrates(offer.sdp) })
+  await pc.setLocalDescription({ type: 'offer', sdp: mungeBitrates(offer.sdp, transport === 'tcp' ? TCP_MIN_KBPS : MIN_KBPS) })
   await new Promise(res => { if (pc.iceGatheringState === 'complete') return res(); const t = setTimeout(res, 2000); pc.onicegatheringstatechange = () => { if (pc.iceGatheringState === 'complete') { clearTimeout(t); res() } } })
   // MediaMTX takes WebRTC credentials only in an Authorization header ("Bearer user:pass"). We send them as
   // X-Stream-Auth and Caddy rewrites the header on the MediaMTX hop, so Authentik's gate never sees it.
   const auth = { 'X-Stream-Auth': `Bearer ${user}:${pass}` }
   const r = await fetch(`/webrtc/${path}/whip`, { method: 'POST', headers: { 'Content-Type': 'application/sdp', ...auth }, body: pc.localDescription.sdp, credentials: 'same-origin' })
   if (!r.ok) { pc.close(); throw new Error(r.status === 401 || r.status === 403 ? 'stream key rejected' : r.status === 400 && /busy|already/i.test(await r.text().catch(() => '')) ? 'that stream name is already live (OBS running?)' : `publish failed (${r.status})`) }
-  await pc.setRemoteDescription({ type: 'answer', sdp: await r.text() })
+  let answer = await r.text()
   let session = r.headers.get('Location')                       // MediaMTX answers relative to its own root
+  if (transport === 'tcp') {
+    const only = tcpOnly(answer)
+    if (!only) { transport = 'udp' } else answer = only          // server offered no TCP: stay on UDP, say so in stats
+  }
+  await pc.setRemoteDescription({ type: 'answer', sdp: answer })
   if (session && session.startsWith('/') && !session.startsWith('/webrtc/')) session = '/webrtc' + session
   return {
     pc,
+    transport,
+    path, user, pass,
     stats: async () => {
       let kbps = 0, fps = 0, w = 0, h = 0
       for (const s of await pc.getStats()) {
@@ -128,6 +166,11 @@ export async function publish({ stream, path, user, pass, onState }) {
       }
       return { kbps, fps, w, h }
     },
+    // release: end this WHIP session but keep capturing — used to re-publish the same tracks over TCP
+    release: async () => {
+      try { if (session) await fetch(session.startsWith('http') ? '/webrtc' + new URL(session).pathname : session, { method: 'DELETE', headers: auth, credentials: 'same-origin' }) } catch {}
+      pc.close()
+    },
     stop: async () => {
       try { if (session) await fetch(session.startsWith('http') ? '/webrtc' + new URL(session).pathname : session, { method: 'DELETE', headers: auth, credentials: 'same-origin' }) } catch {}
       pc.close()
@@ -136,6 +179,18 @@ export async function publish({ stream, path, user, pass, onState }) {
   }
 }
 
+// Loss over a sliding window from the server's receiver reports. samples: [{t, lost, sent}] newest last.
+export function windowLoss(samples) {
+  const now = Date.now()
+  const win = samples.filter(s => now - s.t <= LOSS_WINDOW_S * 1000)
+  if (win.length < 2) return null
+  const a = win[0], b = win[win.length - 1]
+  const sent = b.sent - a.sent, lost = b.lost - a.lost
+  if (sent < LOSS_MIN_PACKETS) return null
+  return { pct: 100 * Math.max(0, lost) / sent, sent, lost }
+}
+export const shouldSwitchToTcp = l => !!l && l.pct >= LOSS_SWITCH_PCT
+
 export async function fetchStreamKey() {
   // a fresh 12 h publish token for the logged-in Discord user (member of streamers); nothing is stored
   const r = await fetch('/me/stream-token', { credentials: 'same-origin' })

observability/kibana-setup.py

@@ -203,6 +203,9 @@ def env_hosts():
                     key, value = line.split("=", 1)
                     values[key.strip()] = value.strip()
             break
+    # .env carries no AUTH_HOST (it lives in the Caddyfile); every host here is <role>.<domain>, so derive it
+    if "AUTH_HOST" not in values and values.get("MAIN_HOST", "").count(".") >= 2:
+        values["AUTH_HOST"] = "auth." + values["MAIN_HOST"].split(".", 1)[1]
     return values
 
 def landing_markdown(hosts):
@@ -210,7 +213,10 @@ def landing_markdown(hosts):
         host = hosts.get(key)
         return f"https://{host}{path}" if host else None
     rows = [
-        ("Stuck recordings", "/stuck", "finished but never published — why, and retry or bin"),
+        ("Live streams", "/streaming#live", "who is publishing, loss on the way in, why files are splitting"),
+        ("Stuck recordings", "/streaming#stuck", "finished but never published — why, and retry or bin"),
+        ("Dump a night", "/streaming#dump", "a streamer's or everyone's night to trash; bin all unprocessed"),
+        ("Disk", "/disk", "space by streamer and clipper per month, trash, logs, system"),
         ("Live portal", url("LIVE_HOST"), "watch every active stream"),
         ("OBS dock", url("LIVE_HOST", "/dock/"), "go live, titles, dump tonight"),
         ("Files", url("FILES_HOST"), "browse, download, upload finals"),
@@ -220,6 +226,7 @@ def landing_markdown(hosts):
         ("Bot", url("BOT_HOST"), "the Discord bot's own pages"),
         ("Syncthing", url("SYNC_HOST"), "clip intake, device pairing"),
         ("Authentik", url("AUTH_HOST", "/if/admin/"), "users, groups, policies"),
+        ("Admin sign-in", url("AUTH_HOST", "/if/flow/default-authentication-flow/?admin=1"), "password card for akadmin — members are forwarded straight to Discord"),
     ]
     lines = ["### Vanguard", "", "| | |", "|---|---|"]
     for name, link, why in rows:

streaming/disk.html

@@ -0,0 +1,132 @@
+<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>Disk — Vanguard</title>
+<style>
+  :root{--bg:#141414;--fg:#e6e6e6;--mut:#9a9a9a;--acc:#3d8bfd;--ok:#2ecc71;--warn:#e3a008;--bad:#e05260;--box:#1f1f1f;--line:#2e2e2e}
+  html,body{margin:0;background:var(--bg);color:var(--fg);font:15px/1.5 system-ui,sans-serif}
+  main{max-width:1100px;margin:0 auto;padding:24px 16px 48px}
+  header{display:flex;align-items:baseline;gap:14px;flex-wrap:wrap;margin-bottom:10px}
+  h1{font-size:24px;margin:0}.who{color:var(--mut)}.who b{color:var(--fg)}
+  nav{display:flex;gap:6px;margin:6px 0 18px;border-bottom:1px solid var(--line)}
+  nav a{padding:8px 14px;color:var(--mut);text-decoration:none;border-bottom:2px solid transparent}
+  nav a.on{color:var(--fg);border-color:var(--acc)}
+  section{background:var(--box);border:1px solid var(--line);border-radius:14px;padding:14px 18px;margin-bottom:12px}
+  section h2{margin:0 0 8px;font-size:15px;color:var(--mut);font-weight:600;text-transform:uppercase;letter-spacing:.3px}
+  table{width:100%;border-collapse:collapse;font-size:14px}th,td{text-align:left;padding:6px 8px;border-top:1px solid var(--line);vertical-align:top}
+  th{color:var(--mut);font-weight:600;border-top:0}td.n{text-align:right;font-variant-numeric:tabular-nums;white-space:nowrap}
+  .mono{font-family:ui-monospace,monospace;font-size:13px;word-break:break-all}
+  .pill{display:inline-block;font-size:12px;border-radius:999px;padding:1px 9px;white-space:nowrap}
+  .ok{background:#1f3b2a;color:var(--ok)}.warn{background:#3a3220;color:var(--warn)}.bad{background:#43242a;color:var(--bad)}.info{background:#1e3147;color:var(--acc)}
+  .state{font-size:12px;border-radius:999px;padding:2px 10px;text-align:center;white-space:nowrap}
+  .parked{background:#43242a;color:var(--bad)}.published{background:#3a3220;color:var(--warn)}.waiting{background:#1e3147;color:var(--acc)}.writing{background:#1f3b2a;color:var(--ok)}
+  .row{display:grid;grid-template-columns:96px 1fr auto;gap:12px;align-items:start}
+  .name{font-family:ui-monospace,monospace;font-size:13px;word-break:break-all}
+  .meta{color:var(--mut);font-size:13px;margin-top:2px}
+  .why{color:var(--mut);font-size:13px;margin-top:6px;border-left:2px solid var(--line);padding-left:10px;white-space:pre-wrap}
+  .acts{display:flex;gap:8px;align-items:center}
+  button{font:inherit;font-size:13px;border-radius:8px;border:1px solid #3a3a3a;background:#262626;color:var(--fg);padding:6px 12px;cursor:pointer}
+  select.stopmins{font:inherit;font-size:12px;border-radius:8px;border:1px solid #3a3a3a;background:#262626;color:var(--fg);padding:5px 6px}
+  button:hover:not(:disabled){border-color:var(--acc)}button:disabled{opacity:.45;cursor:default}
+  button.danger:hover:not(:disabled){border-color:var(--bad);color:var(--bad)}
+  input[type=number]{font:inherit;background:#121212;color:var(--fg);border:1px solid #3a3a3a;border-radius:6px;padding:4px 6px}
+  .note{color:var(--mut);font-size:13px}.err{color:var(--bad)}.hidden{display:none}
+  pre{background:#121212;border-radius:8px;padding:10px;overflow:auto;font-size:12px;margin:8px 0 0;max-height:280px}
+  .verdict{margin-top:8px;font-size:14px}
+  footer{color:var(--mut);font-size:13px;margin-top:20px}
+</style>
+</head>
+<body>
+<main>
+  <header><h1>Disk</h1><div class="who" id="who"></div></header>
+  <nav><a href="/disk" class="on">Disk</a><a href="/" style="margin-left:auto">Dashboards ↗</a></nav>
+  <div id="pane-disk">
+    <section>
+      <div class="row" style="grid-template-columns:1fr auto;align-items:center">
+        <div id="fs" class="note">Loading…</div>
+        <div class="acts"><span class="note" id="diskgen"></span><button id="diskrefresh">Rewalk now</button></div>
+      </div>
+    </section>
+    <section><h2>Streamers — recordings by month</h2><div id="dstreamers" class="note">Loading…</div></section>
+    <section><h2>Clippers — clips by month</h2><div id="dclippers" class="note">Loading…</div></section>
+    <section><h2>Logs</h2><div id="dlogs" class="note">Loading…</div></section>
+    <section><h2>System</h2><div id="dsystem" class="note">Loading…</div></section>
+    <footer>Sizes are allocated blocks, what the disk actually loses. A recording's month is the date in its own name; clips and
+    everything else go by modification time. Cached for ten minutes; Rewalk now re-reads every file.</footer>
+  </div>
+</main>
+<script>
+const API = '/streaming/api';
+let admin = false;
+const esc = s => String(s ?? '').replace(/[&<>"]/g, c => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;'}[c]));
+const hms = s => { s = Math.max(0, s|0); const h = s/3600|0, m = (s%3600)/60|0; return h ? `${h}h ${m}m` : `${m}m`; };
+const dur = s => { s = Math.max(0, s|0); return `${s/3600|0}:${String((s%3600)/60|0).padStart(2,'0')}:${String(s%60).padStart(2,'0')}`; };
+const short = p => p.replace(/^.*\/streams\//, '').replace('/.live/', ' · ');
+const mb = b => (b/1e6).toFixed(b < 10e6 ? 1 : 0) + ' MB';
+
+async function getJSON(path) {
+  const r = await fetch(path, { credentials: 'same-origin' });
+  if (r.status === 401) throw new Error('Not signed in. Reload the page.');
+  const text = await r.text();
+  let data; try { data = JSON.parse(text); }
+  catch { throw new Error(`Server answered HTTP ${r.status} with no usable body${text ? ': ' + text.slice(0, 160) : ' (an empty 502 means dumpd did not reply)'}`); }
+  if (data.error) throw new Error(`HTTP ${r.status}: ${data.error}`);
+  return data;
+}
+function whoami(d) { admin = !!d.admin; document.getElementById('who').innerHTML = `signed in as <b>${esc(d.user || '?')}</b>${admin ? '' : ' · read only'}`; }
+
+const gb = b => b >= 1e12 ? (b / 1e12).toFixed(2) + ' TB' : b >= 1e9 ? (b / 1e9).toFixed(1) + ' GB' : b >= 1e6 ? (b / 1e6).toFixed(0) + ' MB' : b ? (b / 1e3).toFixed(0) + ' kB' : '';
+const monthName = m => { const [y, mo] = m.split('-'); return new Date(+y, +mo - 1, 1).toLocaleString(undefined, { month: 'short', year: '2-digit' }); };
+function monthTable(rows, months, extra) {
+  // rows: [{name, total, months:{}, note}] ; months: newest first, older ones folded into "older"
+  const shown = months.slice(0, 6), older = months.slice(6);
+  const cell = (r, m) => { const v = r.months[m] || 0; return `<td class="n">${v ? gb(v) : '<span class="note">·</span>'}</td>`; };
+  const oldSum = r => older.reduce((a, m) => a + (r.months[m] || 0), 0);
+  const tot = key => rows.reduce((a, r) => a + (key === 'total' ? r.total : key === 'older' ? oldSum(r) : (r.months[key] || 0)), 0);
+  return `<table>
+    <tr><th></th>${shown.map(m => `<th class="n">${monthName(m)}</th>`).join('')}${older.length ? '<th class="n">older</th>' : ''}<th class="n">total</th>${extra ? '<th></th>' : ''}</tr>
+    ${rows.map(r => `<tr><td><b>${esc(r.name)}</b></td>${shown.map(m => cell(r, m)).join('')}${older.length ? `<td class="n">${gb(oldSum(r)) || '<span class="note">·</span>'}</td>` : ''}<td class="n"><b>${gb(r.total)}</b></td>${extra ? `<td class="note">${r.note || ''}</td>` : ''}</tr>`).join('')}
+    ${rows.length > 1 ? `<tr><td class="note">all</td>${shown.map(m => `<td class="n">${gb(tot(m))}</td>`).join('')}${older.length ? `<td class="n">${gb(tot('older'))}</td>` : ''}<td class="n"><b>${gb(tot('total'))}</b></td>${extra ? '<td></td>' : ''}</tr>` : ''}
+  </table>`;
+}
+async function loadDisk(force) {
+  let d;
+  document.getElementById('diskgen').textContent = force ? 'walking every file…' : '';
+  try { d = await getJSON(API + '/disk' + (force ? '?refresh=1' : '')); }
+  catch (e) { document.getElementById('fs').innerHTML = `<span class="err">${esc(e.message)}</span>`; return; }
+  whoami(d);
+  document.getElementById('diskgen').textContent = `walked ${new Date(d.generated).toLocaleTimeString()} in ${d.seconds}s`;
+  const bar = (label, f) => { const pct = f.total ? 100 * f.used / f.total : 0; const cls = pct > 90 ? 'bad' : pct > 75 ? 'warn' : 'ok';
+    return `<div style="margin:4px 0"><b>${esc(label)}</b> <span class="mono note">${esc(f.path)}</span> — ${gb(f.used)} of ${gb(f.total)} used, <span class="pill ${cls}">${gb(f.free)} free</span>
+      <div style="height:8px;background:#262626;border-radius:4px;margin-top:4px"><div style="width:${pct.toFixed(1)}%;height:8px;border-radius:4px;background:var(--${cls === 'ok' ? 'ok' : cls === 'warn' ? 'warn' : 'bad'})"></div></div></div>`; };
+  const fs = d.fs || {};
+  document.getElementById('fs').innerHTML = (fs.media ? bar('media', fs.media) : '') + (fs.root && !fs.same_filesystem ? bar('system disk', fs.root) : '') + (fs.same_filesystem ? '<div class="note">media and the system share one filesystem</div>' : '');
+  const monthsOf = objs => [...new Set(objs.flatMap(o => Object.keys(o.months || {})))].sort().reverse();
+  const toRows = obj => Object.entries(obj).map(([name, v]) => ({ name, total: v.total, months: v.months || {},
+    note: [v.raw ? `${gb(v.raw)} raw in .src` : '', v.unpublished ? `${gb(v.unpublished)} unpublished in .live` : ''].filter(Boolean).join(' · ') })).sort((a, b) => b.total - a.total);
+  const sRows = toRows(d.streamers || {}); const trash = d.trash || { total: 0, months: {} };
+  if (trash.total) sRows.push({ name: 'trash (all users, 3-day undo)', total: trash.total, months: trash.months, note: 'dumped nights and binned recordings, purged after 3 days' });
+  document.getElementById('dstreamers').innerHTML = sRows.length ? monthTable(sRows, monthsOf(sRows), true) : '<span class="note">no recordings</span>';
+  const cRows = toRows(d.clippers || {});
+  document.getElementById('dclippers').innerHTML = cRows.length ? monthTable(cRows, monthsOf(cRows), false) : '<span class="note">no clips</span>';
+  const L = d.logs || {};
+  const lRows = [];
+  if (L.months) lRows.push({ name: 'Elasticsearch indices (logs + metrics)', total: L.es_total || 0, months: L.months, note: L.es_data_dir ? `${gb(L.es_data_dir)} on disk in appdata/elasticsearch` : '' });
+  document.getElementById('dlogs').innerHTML = (lRows.length ? monthTable(lRows, monthsOf(lRows), true) : '') + `<table style="margin-top:8px">
+    ${L.es_data_dir != null ? `<tr><td>Elasticsearch data dir</td><td class="n">${gb(L.es_data_dir)}</td><td class="note">the on-disk truth for the indices above</td></tr>` : ''}
+    <tr><td>journald</td><td class="n">${gb(L.journald || 0)}</td><td class="note">systemd journal (dumpd, sweep, timers)</td></tr>
+    <tr><td>Docker json logs</td><td class="n">${gb(L.docker_json_logs || 0)}</td><td class="note">per-container log files Filebeat reads</td></tr>
+  </table>` + (L.indices ? `<details style="margin-top:8px"><summary class="note">${L.indices.length} indices</summary><table>${L.indices.sort((a, b) => b.bytes - a.bytes).map(i => `<tr><td class="mono">${esc(i.index)}</td><td class="n">${gb(i.bytes)}</td><td class="note">${esc(i.month)}</td></tr>`).join('')}</table></details>` : '');
+  const S = d.system || {};
+  document.getElementById('dsystem').innerHTML = `<table>${Object.entries(S).sort((a, b) => (typeof b[1] === 'number' ? b[1] : 0) - (typeof a[1] === 'number' ? a[1] : 0))
+    .map(([k, v]) => `<tr><td>${esc(k)}</td><td class="n">${typeof v === 'number' ? gb(v) : esc(v)}</td><td class="note">${k.startsWith('docker') ? "Docker's own figure" : k === 'media_other' ? 'markers, Syncthing state, versions under /srv/media' : ''}</td></tr>`).join('')}</table>`
+    + (d.errors && d.errors.length ? `<div class="err" style="margin-top:8px">${d.errors.map(esc).join('<br>')}</div>` : '');
+}
+document.getElementById('diskrefresh').onclick = () => loadDisk(true);
+
+loadDisk(false);
+</script>
+</body>
+</html>

streaming/index.html

@@ -0,0 +1,416 @@
+<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<title>Streaming — Vanguard</title>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<style>
+  :root{--bg:#141414;--fg:#e6e6e6;--mut:#9a9a9a;--acc:#3d8bfd;--ok:#2ecc71;--warn:#e3a008;--bad:#e05260;--box:#1f1f1f;--line:#2e2e2e}
+  html,body{margin:0;background:var(--bg);color:var(--fg);font:15px/1.5 system-ui,sans-serif}
+  main{max-width:1100px;margin:0 auto;padding:24px 16px 48px}
+  header{display:flex;align-items:baseline;gap:14px;flex-wrap:wrap;margin-bottom:10px}
+  h1{font-size:24px;margin:0}.who{color:var(--mut)}.who b{color:var(--fg)}
+  nav{display:flex;gap:6px;margin:6px 0 18px;border-bottom:1px solid var(--line)}
+  nav a{padding:8px 14px;color:var(--mut);text-decoration:none;border-bottom:2px solid transparent}
+  nav a.on{color:var(--fg);border-color:var(--acc)}
+  section{background:var(--box);border:1px solid var(--line);border-radius:14px;padding:14px 18px;margin-bottom:12px}
+  section h2{margin:0 0 8px;font-size:15px;color:var(--mut);font-weight:600;text-transform:uppercase;letter-spacing:.3px}
+  table{width:100%;border-collapse:collapse;font-size:14px}th,td{text-align:left;padding:6px 8px;border-top:1px solid var(--line);vertical-align:top}
+  th{color:var(--mut);font-weight:600;border-top:0}td.n{text-align:right;font-variant-numeric:tabular-nums;white-space:nowrap}
+  .mono{font-family:ui-monospace,monospace;font-size:13px;word-break:break-all}
+  .pill{display:inline-block;font-size:12px;border-radius:999px;padding:1px 9px;white-space:nowrap}
+  .ok{background:#1f3b2a;color:var(--ok)}.warn{background:#3a3220;color:var(--warn)}.bad{background:#43242a;color:var(--bad)}.info{background:#1e3147;color:var(--acc)}
+  .state{font-size:12px;border-radius:999px;padding:2px 10px;text-align:center;white-space:nowrap}
+  .parked{background:#43242a;color:var(--bad)}.published{background:#3a3220;color:var(--warn)}.waiting{background:#1e3147;color:var(--acc)}.writing{background:#1f3b2a;color:var(--ok)}
+  .row{display:grid;grid-template-columns:96px 1fr auto;gap:12px;align-items:start}
+  .name{font-family:ui-monospace,monospace;font-size:13px;word-break:break-all}
+  .meta{color:var(--mut);font-size:13px;margin-top:2px}
+  .why{color:var(--mut);font-size:13px;margin-top:6px;border-left:2px solid var(--line);padding-left:10px;white-space:pre-wrap}
+  .acts{display:flex;gap:8px;align-items:center}
+  button{font:inherit;font-size:13px;border-radius:8px;border:1px solid #3a3a3a;background:#262626;color:var(--fg);padding:6px 12px;cursor:pointer}
+  select.stopmins{font:inherit;font-size:12px;border-radius:8px;border:1px solid #3a3a3a;background:#262626;color:var(--fg);padding:5px 6px}
+  button:hover:not(:disabled){border-color:var(--acc)}button:disabled{opacity:.45;cursor:default}
+  button.danger:hover:not(:disabled){border-color:var(--bad);color:var(--bad)}
+  input[type=number]{font:inherit;background:#121212;color:var(--fg);border:1px solid #3a3a3a;border-radius:6px;padding:4px 6px}
+  .note{color:var(--mut);font-size:13px}.err{color:var(--bad)}.hidden{display:none}
+  pre{background:#121212;border-radius:8px;padding:10px;overflow:auto;font-size:12px;margin:8px 0 0;max-height:280px}
+  .verdict{margin-top:8px;font-size:14px}
+  footer{color:var(--mut);font-size:13px;margin-top:20px}
+</style>
+</head>
+<body>
+<main>
+  <header><h1>Streaming</h1><div class="who" id="who"></div></header>
+  <nav><a href="#live" id="tab-live">Live</a><a href="#stuck" id="tab-stuck">Stuck recordings</a><a href="#dump" id="tab-dump">Dump a night</a><a href="/disk" style="margin-left:auto">Disk ↗</a></nav>
+
+  <!-- ===================== LIVE ===================== -->
+  <div id="pane-live">
+    <section>
+      <h2>Publishing now</h2>
+      <div id="pubs" class="note">Loading…</div>
+    </section>
+    <section>
+      <h2>Blocked from publishing</h2>
+      <div id="blocks" class="note">Loading…</div>
+    </section>
+    <section>
+      <h2>Why MediaMTX opened recording files — last <span id="win">10m</span></h2>
+      <div id="counts" class="note">Loading…</div>
+      <div id="verdict" class="verdict"></div>
+    </section>
+    <section>
+      <h2>Live paths</h2>
+      <div id="paths" class="note">Loading…</div>
+    </section>
+    <section>
+      <h2>Publish sessions — last 24 h</h2>
+      <div id="hist" class="note">Loading…</div>
+    </section>
+    <section>
+      <h2>Recorder log</h2>
+      <pre id="log">Loading…</pre>
+    </section>
+    <footer>Refreshes every 10 seconds. Loss is counted on the <em>publish</em> session, i.e. on the way in: that is what corrupts recordings. Viewers losing packets only affects what they see.</footer>
+  </div>
+
+  <!-- ===================== STUCK ===================== -->
+  <div id="pane-stuck" class="hidden">
+    <p class="note">Recordings that finished but never reached Jellyfin or copyparty. Retry puts one back in the
+    queue; bin moves it to the trash, recoverable for three days just like a dumped night.</p>
+    <section id="bulk" class="hidden">
+      <div class="row" style="grid-template-columns:1fr auto">
+        <div>
+          <b>Reconnect fragments</b>
+          <div class="meta">Every time a publisher drops and comes back, MediaMTX opens a new recording. Stream
+          recordings this small are stubs, not recordings. Clips are not affected: they never pass through <code>.live/</code>.</div>
+        </div>
+        <div class="acts">
+          <label class="meta">under <input id="mb" type="number" min="1" max="50" value="5" style="width:4em"> MB</label>
+          <button id="bulkbtn" class="danger">Bin <span id="bulkcount">0</span> fragments</button>
+        </div>
+      </div>
+    </section>
+    <div id="list"><section class="note">Loading…</section></div>
+    <footer>Refreshes every 30 seconds. Trends and history live on the dashboards.</footer>
+  </div>
+
+  <!-- ===================== DUMP ===================== -->
+  <div id="pane-dump" class="hidden">
+    <p class="note">A bad night goes to the trash in one move: every stream recording and clip a streamer saved
+    between <b>17:00 and 02:00 Eastern</b> on the chosen night. Recoverable for three days, like the Stuck tab's
+    bin. Admins can dump anyone or everyone; everyone else sees and dumps only their own.</p>
+    <section>
+      <div class="row" style="grid-template-columns:auto 1fr auto;align-items:center">
+        <label class="meta">night <input id="night" type="date" style="font:inherit;background:#262626;color:var(--fg);border:1px solid #3a3a3a;border-radius:8px;padding:4px 8px"></label>
+        <div class="meta" id="dumpwin"></div>
+        <div class="acts"><button id="dumpall" class="danger hidden">Dump everyone</button></div>
+      </div>
+    </section>
+    <div id="dumplist"><section class="note">Loading…</section></div>
+    <section id="wipe" class="hidden">
+      <div class="row" style="grid-template-columns:1fr auto">
+        <div>
+          <b>Unprocessed and stuck recordings</b>
+          <div class="meta" id="wipemeta"></div>
+          <div class="meta">Everything still sitting in a <code>.live/</code> folder, whatever night it is from: stuck, parked, waiting, fresh, reconnect fragments, and the failure logs that go with them. Any conversion the sweep has in flight is cancelled first. The only thing left alone is a stream MediaMTX is recording this minute.</div>
+        </div>
+        <div class="acts"><button id="wipebtn" class="danger">Bin all unprocessed</button></div>
+      </div>
+      <pre id="wipeout" class="hidden"></pre>
+    </section>
+    <footer>Nothing here moves until you confirm twice. Streamers can also dump their own night from the OBS dock.</footer>
+  </div>
+
+</main>
+<script>
+const API = '/streaming/api';
+let admin = false;
+const esc = s => String(s ?? '').replace(/[&<>"]/g, c => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;'}[c]));
+const hms = s => { s = Math.max(0, s|0); const h = s/3600|0, m = (s%3600)/60|0; return h ? `${h}h ${m}m` : `${m}m`; };
+const dur = s => { s = Math.max(0, s|0); return `${s/3600|0}:${String((s%3600)/60|0).padStart(2,'0')}:${String(s%60).padStart(2,'0')}`; };
+const short = p => p.replace(/^.*\/streams\//, '').replace('/.live/', ' · ');
+const mb = b => (b/1e6).toFixed(b < 10e6 ? 1 : 0) + ' MB';
+
+async function getJSON(path) {
+  const r = await fetch(path, { credentials: 'same-origin' });
+  if (r.status === 401) throw new Error('Not signed in. Reload the page.');
+  const text = await r.text();
+  let data; try { data = JSON.parse(text); }
+  catch { throw new Error(`Server answered HTTP ${r.status} with no usable body${text ? ': ' + text.slice(0, 160) : ' (an empty 502 means dumpd did not reply)'}`); }
+  if (data.error) throw new Error(`HTTP ${r.status}: ${data.error}`);
+  return data;
+}
+function whoami(d) { admin = !!d.admin; document.getElementById('who').innerHTML = `signed in as <b>${esc(d.user || '?')}</b>${admin ? '' : ' · read only'}`; }
+
+// ---------------- Live ----------------
+async function loadLive() {
+  let d;
+  try { d = await getJSON(API + '/live'); }
+  catch (e) { document.getElementById('pubs').innerHTML = `<span class="err">${esc(e.message)}</span>`; return; }
+  whoami(d);
+
+  const pubs = d.publishers || [];
+  document.getElementById('pubs').innerHTML = pubs.length ? `<table>
+    <tr><th>path</th><th>via</th><th>from</th><th>since</th><th class="n">received</th><th class="n">lost</th><th>loss</th><th></th></tr>
+    ${pubs.map(p => {
+      const pct = p.received ? (100 * (p.lost || 0) / p.received) : 0;
+      const cls = p.kind !== 'webrtc' ? 'info' : pct === 0 ? 'ok' : pct < 0.05 ? 'warn' : 'bad';
+      const txt = p.kind !== 'webrtc' ? 'tcp' : `${pct.toFixed(3)}%`;
+      // webrtc: the selected ICE pair's protocol. tcp = the Go Live lossy-uplink switch (or the box ticked)
+      const via = p.kind === 'webrtc' ? `webrtc <span class="pill ${p.transport === 'tcp' ? 'info' : p.transport === 'udp' ? 'ok' : 'warn'}">${esc(p.transport || 'connecting')}</span>` : esc(p.kind);
+      return `<tr><td class="mono">${esc(p.path)}</td><td>${via}</td><td class="mono">${esc(p.from)}</td><td class="note">${p.since ? new Date(p.since).toLocaleTimeString() : ''}</td>
+        <td class="n">${p.received != null ? p.received.toLocaleString() : (p.bytes != null ? mb(p.bytes) : '')}</td>
+        <td class="n">${p.lost != null ? p.lost.toLocaleString() : ''}</td><td><span class="pill ${cls}">${txt}</span></td>
+        <td class="acts">${admin && p.session ? `<select class="stopmins" data-session="${esc(p.session)}" title="how long the account stays unable to publish after the kick">
+            <option value="0">kick only</option><option value="30" selected>block 30 min</option><option value="120">block 2 h</option><option value="480">block 8 h</option></select>
+          <button class="danger stopbtn" data-path="${esc(p.path)}" data-kind="${esc(p.kind)}" data-session="${esc(p.session)}">Stop stream</button>` : ''}</td></tr>`;
+    }).join('')}</table>` : '<span class="note">nobody is publishing</span>';
+  for (const b of document.querySelectorAll('#pubs .stopbtn')) b.onclick = () => stopStream(b);
+
+  // accounts currently blocked from publishing (set by Stop stream), with an unblock
+  const blocks = d.blocks || [];
+  document.getElementById('blocks').innerHTML = blocks.length ? `<table>
+    <tr><th>account</th><th>blocked</th><th>until</th><th>by</th><th>why</th><th></th></tr>
+    ${blocks.map(b => `<tr><td class="mono">${esc(b.user)}</td><td class="note">${b.since ? new Date(b.since).toLocaleTimeString() : ''}</td>
+      <td class="note">${new Date(b.until_iso).toLocaleTimeString()}</td><td>${esc(b.by || '')}</td><td class="note">${esc(b.reason || '')}</td>
+      <td class="acts">${admin ? `<button class="unblock" data-user="${esc(b.user)}">Allow again</button>` : ''}</td></tr>`).join('')}</table>`
+    : '<span class="note">nobody is blocked from publishing</span>';
+  for (const b of document.querySelectorAll('#blocks .unblock')) b.onclick = () => unblock(b.dataset.user);
+
+  const c = d.counts || {};
+  document.getElementById('win').textContent = c.window || '10m';
+  document.getElementById('counts').innerHTML = `<table>
+    <tr><td>new publish sessions</td><td class="n">${c.publish_sessions ?? '?'}</td><td class="note">a reconnect each</td></tr>
+    <tr><td>recorder (re)starts</td><td class="n">${c.recorder_restarts ?? '?'}</td><td class="note">a new .live file each</td></tr>
+    <tr><td>processing errors</td><td class="n">${c.processing_errors ?? '?'}</td><td class="note">lost packets corrupting frames</td></tr>
+    <tr><td>recorder access-unit errors</td><td class="n">${c.access_unit_errors ?? '?'}</td><td class="note">frames the recorder refused</td></tr>
+  </table>`;
+  let v = '';
+  if ((c.recorder_restarts|0) > 1) {
+    if ((c.processing_errors|0) > 0) v = '<span class="pill bad">loss</span> files are splitting because packets are being lost on the way in: check the publisher\'s uplink, and that MediaMTX is on the host network.';
+    else if ((c.publish_sessions|0) >= (c.recorder_restarts|0) - 1) v = '<span class="pill warn">reconnects</span> the publisher\'s session keeps ending: look at their browser console and the "closed" reasons below.';
+    else v = '<span class="pill warn">parameter changes</span> one stable session, no loss, yet files keep opening: the encoder is changing resolution (new SPS). Pin resolution in the publisher.';
+  } else if (pubs.length) v = '<span class="pill ok">steady</span> one file per stream, nothing lost.';
+  document.getElementById('verdict').innerHTML = v;
+
+  const paths = d.paths || [];
+  document.getElementById('paths').innerHTML = paths.length ? `<table>
+    <tr><th>path</th><th>tracks</th><th>encode</th><th class="n">viewers</th><th class="n">received</th><th>since</th></tr>
+    ${paths.map(p => {
+      const h = p.health;
+      // MediaMTX refuses WebRTC for H.264 with B-frames: that is the single most common "I can't watch it".
+      const enc = p.mirror ? '' : !h ? '<span class="note">probing…</span>'
+        : h.bframes > 0 ? `<span class="pill bad">B-frames (depth ${h.bframes}) · WebRTC refused</span> <span class="note">${esc(h.res || '')} ${h.fps ? h.fps + ' fps' : ''}</span>`
+        : `<span class="pill ok">no B-frames</span> <span class="note">${esc(h.res || '')} ${h.fps ? h.fps + ' fps' : ''}</span>`;
+      return `<tr><td class="mono">${esc(p.name)}${p.mirror ? ' <span class="pill info">audio mirror</span>' : ''}</td>
+      <td>${esc((p.tracks || []).join(', '))}</td><td>${enc}</td><td class="n">${p.readers}</td><td class="n">${mb(p.bytes || 0)}</td>
+      <td class="note">${p.since ? new Date(p.since).toLocaleTimeString() : ''}</td></tr>`;
+    }).join('')}</table>` : '<span class="note">nothing live</span>';
+
+  // Every WebRTC publish session MediaMTX has had in 24 h, with the transport it ran on. A session that starts
+  // on a different transport than the previous one of the same path is a switch: that is "when he swapped".
+  const hist = d.history || [];
+  const span = (a, b) => { if (!a) return ''; const s = Math.max(0, ((b ? new Date(b) : new Date()) - new Date(a)) / 1000 | 0); return hms(s) || '<1m'; };
+  document.getElementById('hist').innerHTML = hist.length ? `<table>
+    <tr><th>path</th><th>transport</th><th>started</th><th>ended</th><th>ran</th><th class="n">received</th><th class="n">lost</th><th>loss</th><th></th></tr>
+    ${hist.map(h => {
+      const pct = h.received ? (100 * (h.lost || 0) / h.received) : 0;
+      const cls = pct === 0 ? 'ok' : pct < 0.05 ? 'warn' : 'bad';
+      const sw = h.switched_from ? `<span class="pill info">switched from ${esc(h.switched_from)}</span>` : '';
+      return `<tr><td class="mono">${esc(h.path)}</td><td><span class="pill ${h.transport === 'tcp' ? 'info' : h.transport ? 'ok' : 'warn'}">${esc(h.transport || '?')}</span></td>
+        <td class="note">${h.created ? new Date(h.created).toLocaleTimeString() : ''}</td><td class="note">${h.ended ? new Date(h.ended).toLocaleTimeString() : '<b>live</b>'}</td>
+        <td class="note">${span(h.created, h.ended)}</td><td class="n">${(h.received || 0).toLocaleString()}</td><td class="n">${(h.lost || 0).toLocaleString()}</td>
+        <td><span class="pill ${cls}">${pct.toFixed(3)}%</span></td><td>${sw}</td></tr>`;
+    }).join('')}</table>` : '<span class="note">no WebRTC publish sessions in the last 24 h</span>';
+
+  document.getElementById('log').textContent = (d.log || []).join('\n') || '(quiet)';
+  if (d.errors && d.errors.length) document.getElementById('log').textContent += '\n\n' + d.errors.join('\n');
+}
+
+// Stop stream: MediaMTX kicks the publish session, and dumpd blocks the account's publishing for the chosen
+// time so OBS's auto-reconnect or a Go Live restart cannot bring it straight back. Kick only = a hiccup.
+async function stopStream(btn) {
+  const sel = document.querySelector(`#pubs .stopmins[data-session="${btn.dataset.session}"]`);
+  const minutes = parseInt(sel?.value || '30', 10);
+  const what = minutes ? `and block ${esc(btn.dataset.path.split('/')[0])} from publishing for ${minutes >= 60 ? (minutes / 60) + ' h' : minutes + ' min'}` : 'only (they can reconnect at once)';
+  if (!confirm(`Stop ${btn.dataset.path} now, ${what}?\n\nThe recording so far is kept.`)) return;
+  btn.disabled = true; btn.textContent = 'Stopping…';
+  try {
+    const r = await fetch(API + '/live', { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({ action: 'stop', path: btn.dataset.path, kind: btn.dataset.kind, session: btn.dataset.session, minutes }) });
+    const body = await r.json();
+    btn.textContent = body.error ? `Failed: ${body.error}` : (body.kicked ? 'Stopped' : 'Not stopped');
+  } catch (e) { btn.textContent = `Failed: ${e.message}`; }
+  setTimeout(loadLive, 1500);
+}
+async function unblock(user) {
+  if (!confirm(`Let ${user} publish again now?`)) return;
+  try {
+    await fetch(API + '/live', { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ action: 'unblock', user }) });
+  } catch {}
+  loadLive();
+}
+
+// ---------------- Stuck ----------------
+async function loadStuck() {
+  const list = document.getElementById('list');
+  let data;
+  try { data = await getJSON(API + '/stuck'); }
+  catch (e) { list.innerHTML = `<section class="note err">${esc(e.message)}</section>`; return; }
+  whoami(data);
+  const items = (data.items || []).sort((a, b) => (a.state === b.state ? b.age_s - a.age_s : a.state.localeCompare(b.state)));
+  const bulk = document.getElementById('bulk');
+  bulk.classList.toggle('hidden', !admin || !items.length);
+  window.__items = items; updateBulkCount();
+  if (!items.length) { list.innerHTML = '<section class="note">Nothing is stuck. Everything recorded has published.</section>'; return; }
+  list.innerHTML = '';
+  for (const it of items) {
+    const el = document.createElement('section');
+    const actionable = admin && it.state !== 'writing';
+    el.innerHTML = `
+      <div class="row">
+        <div class="state ${esc(it.state)}">${esc(it.state)}</div>
+        <div>
+          <div class="name">${esc(short(it.src))}</div>
+          <div class="meta">waiting ${hms(it.age_s)}${it.duration_s == null ? '' : ' · ' + dur(it.duration_s) + ' long'} · ${mb(it.bytes)}</div>
+          <div class="why">${esc(it.reason || '')}</div>
+        </div>
+        <div class="acts">
+          <button data-act="retry" ${actionable ? '' : 'disabled'}>Retry</button>
+          <button data-act="drop" class="danger" ${actionable ? '' : 'disabled'}>Bin</button>
+        </div>
+      </div>`;
+    for (const b of el.querySelectorAll('button')) b.onclick = () => act(it, b.dataset.act, el);
+    list.appendChild(el);
+  }
+}
+async function act(item, action, el) {
+  if (action === 'drop' && !confirm(`Move this recording to the trash?\n\n${short(item.src)}\n\nRecoverable for three days.`)) return;
+  for (const b of el.querySelectorAll('button')) b.disabled = true;
+  const out = document.createElement('pre'); out.textContent = action === 'retry' ? 'Clearing the blocker…' : 'Moving to trash…'; el.appendChild(out);
+  try {
+    const r = await fetch(API + '/stuck', { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ action, src: item.src }) });
+    const body = await r.json();
+    out.textContent = body.error ? `Failed: ${body.error}` : (body.output || []).join('\n') || 'Done.';
+    if (!body.error) setTimeout(loadStuck, 1200);
+  } catch (e) { out.textContent = `Failed: ${e.message}`; }
+}
+function updateBulkCount() {
+  const mbv = Math.max(1, Math.min(50, parseInt(document.getElementById('mb').value || '5', 10)));
+  const n = (window.__items || []).filter(i => i.state !== 'writing' && i.bytes < mbv * 1e6).length;
+  document.getElementById('bulkcount').textContent = n;
+  document.getElementById('bulkbtn').disabled = n === 0;
+}
+document.getElementById('mb').addEventListener('input', updateBulkCount);
+document.getElementById('bulkbtn').onclick = async () => {
+  const mbv = parseInt(document.getElementById('mb').value || '5', 10);
+  const n = document.getElementById('bulkcount').textContent;
+  if (!confirm(`Move ${n} stream recordings smaller than ${mbv} MB to the trash?\n\nRecoverable for three days. Clips are not affected.`)) return;
+  const btn = document.getElementById('bulkbtn'); btn.disabled = true; btn.textContent = 'Binning…';
+  try {
+    const r = await fetch(API + '/stuck', { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ action: 'drop_under', mb: mbv }) });
+    const body = await r.json();
+    btn.textContent = body.error ? `Failed: ${body.error}` : 'Done';
+  } catch (e) { btn.textContent = `Failed: ${e.message}`; }
+  setTimeout(loadStuck, 1500);
+};
+
+// ---------------- Dump ----------------
+// Same endpoint as the dock's "Dump tonight" button (dumpd), reached here as /streaming/api/dump so the
+// Authentik cookie for this host is enough. GET is always a dry run; only the POST moves anything.
+const nightEl = document.getElementById('night');
+let dumpNightTouched = false;
+nightEl.addEventListener('change', () => { dumpNightTouched = true; loadDump(); });
+async function loadDump() {
+  const list = document.getElementById('dumplist');
+  let d;
+  try {
+    const q = new URLSearchParams({ user: 'all' });                       // non-admins get their own back regardless
+    if (dumpNightTouched && nightEl.value) q.set('night', nightEl.value);
+    d = await getJSON(API + '/dump?' + q);
+  } catch (e) { list.innerHTML = `<section class="note err">${esc(e.message)}</section>`; return; }
+  whoami(d);
+  if (!dumpNightTouched) nightEl.value = d.night;
+  document.getElementById('dumpwin').textContent = d.window ? `${d.window[0].replace('T', ' ').slice(0, 16)} → ${d.window[1].replace('T', ' ').slice(0, 16)}` : '';
+  const byUser = {};
+  for (const e of d.entries || []) {
+    const m = e.path.match(/^(streams|clips)\/([^/]+)\//);
+    const u = m ? m[2] : '(other)'; const kind = m ? m[1] : 'other';
+    const row = byUser[u] ||= { user: u, streams: 0, clips: 0, bytes: 0 };
+    row[kind === 'streams' ? 'streams' : 'clips']++; row.bytes += e.bytes;
+  }
+  const rows = Object.values(byUser).sort((a, b) => b.bytes - a.bytes);
+  document.getElementById('dumpall').classList.toggle('hidden', !(admin && rows.length > 1));
+  if (!rows.length) { list.innerHTML = `<section class="note">Nothing recorded on the night of ${esc(d.night)}${admin ? '' : ' under your name'}.</section>`; return; }
+  list.innerHTML = `<section><table>
+    <tr><th>streamer</th><th class="n">stream recordings</th><th class="n">clips</th><th class="n">size</th><th></th></tr>
+    ${rows.map(r => `<tr data-user="${esc(r.user)}"><td><b>${esc(r.user)}</b></td><td class="n">${r.streams}</td><td class="n">${r.clips}</td><td class="n">${(r.bytes / 1e9).toFixed(2)} GB</td>
+      <td class="acts"><button class="danger" data-user="${esc(r.user)}">Dump ${esc(r.user)}'s night</button></td></tr>`).join('')}
+    <tr><td>total</td><td class="n">${rows.reduce((a, r) => a + r.streams, 0)}</td><td class="n">${rows.reduce((a, r) => a + r.clips, 0)}</td><td class="n"><b>${(d.gb || 0).toFixed(2)} GB</b></td><td></td></tr>
+  </table><pre id="dumpout" class="hidden"></pre></section>`;
+  for (const b of list.querySelectorAll('button[data-user]')) b.onclick = () => dumpNight(b.dataset.user, b);
+}
+async function dumpNight(user, btn) {
+  const night = nightEl.value;
+  const who = user === 'all' ? 'EVERYONE' : user;
+  if (!confirm(`Move ${who}'s recordings and clips from the night of ${night} to the trash?\n\nRecoverable for three days.`)) return;
+  if (btn.dataset.armed !== night) {                                   // second click, same night: the dock's two-step
+    btn.dataset.armed = night; const was = btn.textContent; btn.textContent = 'Click again to confirm';
+    setTimeout(() => { if (btn.isConnected) { btn.textContent = was; delete btn.dataset.armed; } }, 10000);
+    return;
+  }
+  btn.disabled = true; btn.textContent = 'Dumping…';
+  const out = document.getElementById('dumpout'); out.classList.remove('hidden'); out.textContent = `Moving ${who}'s night of ${night}…`;
+  try {
+    const r = await fetch(API + '/dump', { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ user, night }) });
+    const body = await r.json();
+    out.textContent = body.error ? `Failed: ${body.error}` : `Dumped ${body.moved} file(s), ${body.gb} GB from ${body.user}'s night of ${body.night}. Recoverable until ${body.undo_until}.`;
+  } catch (e) { out.textContent = `Failed: ${e.message}`; }
+  setTimeout(loadDump, 1200);
+}
+document.getElementById('dumpall').onclick = () => dumpNight('all', document.getElementById('dumpall'));
+
+// "Bin all unprocessed": the Stuck tab's survey gives the count and size; vgstuck.sh wipe does the moving.
+async function loadWipe() {
+  const box = document.getElementById('wipe');
+  let d;
+  try { d = await getJSON(API + '/stuck'); } catch (e) { box.classList.add('hidden'); return; }
+  const items = d.items || [];                         // "writing" is only a guess from mtime; the tool checks what is really open
+  box.classList.toggle('hidden', !admin || !items.length);
+  const gb = items.reduce((a, i) => a + i.bytes, 0) / 1e9;
+  document.getElementById('wipemeta').textContent = `${items.length} recording(s), ${gb.toFixed(2)} GB in .live/`;
+  const b = document.getElementById('wipebtn'); b.disabled = false; b.textContent = `Bin all ${items.length} unprocessed`; delete b.dataset.armed;
+}
+document.getElementById('wipebtn').onclick = async () => {
+  const btn = document.getElementById('wipebtn');
+  const meta = document.getElementById('wipemeta').textContent;
+  if (!confirm(`Move every unprocessed recording to the trash and cancel any conversion in progress?\n\n${meta}\n\nRecoverable for three days. Only a stream being recorded this minute is left alone.`)) return;
+  if (!btn.dataset.armed) { btn.dataset.armed = '1'; const was = btn.textContent; btn.textContent = 'Click again to confirm'; setTimeout(() => { if (btn.dataset.armed) { btn.textContent = was; delete btn.dataset.armed; } }, 10000); return; }
+  delete btn.dataset.armed; btn.disabled = true; btn.textContent = 'Binning…';
+  const out = document.getElementById('wipeout'); out.classList.remove('hidden'); out.textContent = 'Moving to trash…';
+  try {
+    const r = await fetch(API + '/stuck', { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ action: 'wipe' }) });
+    const body = await r.json();
+    out.textContent = body.error ? `Failed: ${body.error}` : (body.output || []).join('\n') || 'Done.';
+  } catch (e) { out.textContent = `Failed: ${e.message}`; }
+  setTimeout(loadWipe, 1500);
+};
+
+// ---------------- tabs ----------------
+let timer = null;
+function show(tab) {
+  if (!['live', 'stuck', 'dump'].includes(tab)) tab = 'live';
+  for (const t of ['live', 'stuck', 'dump']) {
+    document.getElementById('pane-' + t).classList.toggle('hidden', t !== tab);
+    document.getElementById('tab-' + t).classList.toggle('on', t === tab);
+  }
+  clearInterval(timer);
+  if (tab === 'live') { loadLive(); timer = setInterval(loadLive, 10000); }
+  else if (tab === 'stuck') { loadStuck(); timer = setInterval(loadStuck, 30000); }
+  else { dumpNightTouched = false; loadDump(); loadWipe(); }
+}
+window.addEventListener('hashchange', () => show(location.hash.replace('#', '')));
+show(location.hash.replace('#', '') || 'live');
+</script>
+</body>
+</html>

stuck/index.html

@@ -1,113 +0,0 @@
-<!doctype html>
-<html lang="en">
-<head>
-<meta charset="utf-8">
-<title>Stuck recordings — Vanguard</title>
-<meta name="viewport" content="width=device-width, initial-scale=1">
-<style>
-  :root{--bg:#141414;--fg:#e6e6e6;--mut:#9a9a9a;--acc:#3d8bfd;--ok:#2ecc71;--warn:#e3a008;--bad:#e05260;--box:#1f1f1f;--line:#2e2e2e}
-  html,body{margin:0;background:var(--bg);color:var(--fg);font:15px/1.5 system-ui,sans-serif}
-  main{max-width:1040px;margin:0 auto;padding:28px 16px 48px}
-  header{display:flex;align-items:baseline;gap:14px;flex-wrap:wrap;margin-bottom:6px}
-  h1{font-size:26px;margin:0}.who{color:var(--mut)}.who b{color:var(--fg)}
-  p.lead{color:var(--mut);margin:4px 0 20px}
-  section{background:var(--box);border:1px solid var(--line);border-radius:14px;padding:16px 18px;margin-bottom:14px}
-  .row{display:grid;grid-template-columns:96px 1fr auto;gap:12px;align-items:start}
-  .state{font-size:12px;border-radius:999px;padding:2px 10px;text-align:center;align-self:start;white-space:nowrap}
-  .parked{background:#43242a;color:var(--bad)}.published{background:#3a3220;color:var(--warn)}
-  .waiting{background:#1e3147;color:var(--acc)}.writing{background:#1f3b2a;color:var(--ok)}
-  .name{font-family:ui-monospace,monospace;font-size:13px;word-break:break-all}
-  .meta{color:var(--mut);font-size:13px;margin-top:2px}
-  .why{color:var(--mut);font-size:13px;margin-top:6px;border-left:2px solid var(--line);padding-left:10px;white-space:pre-wrap}
-  .acts{display:flex;gap:8px}
-  button{font:inherit;font-size:13px;border-radius:8px;border:1px solid #3a3a3a;background:#262626;color:var(--fg);padding:6px 12px;cursor:pointer}
-  button:hover:not(:disabled){border-color:var(--acc)}button:disabled{opacity:.45;cursor:default}
-  button.danger:hover:not(:disabled){border-color:var(--bad);color:var(--bad)}
-  .note{color:var(--mut);font-size:13px}.err{color:var(--bad)}
-  pre{background:#121212;border-radius:8px;padding:10px;overflow:auto;font-size:12px;margin:8px 0 0}
-  footer{color:var(--mut);font-size:13px;margin-top:20px}
-</style>
-</head>
-<body>
-<main>
-  <header><h1>Stuck recordings</h1><div class="who" id="who"></div></header>
-  <p class="lead">Recordings that finished but never reached Jellyfin or copyparty. Retry puts one back in the
-  queue; bin moves it to the trash, recoverable for three days just like a dumped night.</p>
-  <div id="list"><section class="note">Loading…</section></div>
-  <footer>Refreshes every 30&nbsp;seconds. Trends and history live on the diagnostics dashboard.</footer>
-</main>
-<script>
-const API = '/stuck/api';
-let admin = false;
-
-const hms = s => { s = Math.max(0, s|0); const h = s/3600|0, m = (s%3600)/60|0; return h ? `${h}h ${m}m` : `${m}m`; };
-const dur = s => { s = Math.max(0, s|0); return `${s/3600|0}:${String((s%3600)/60|0).padStart(2,'0')}:${String(s%60).padStart(2,'0')}`; };
-const short = p => p.replace(/^.*\/streams\//, '').replace('/.live/', ' · ');
-
-async function load() {
-  const list = document.getElementById('list');
-  let data;
-  try {
-    const r = await fetch(API, { credentials: 'same-origin' });
-    if (r.status === 401) { list.innerHTML = '<section class="note err">Not signed in. Reload the page.</section>'; return; }
-    data = await r.json();
-  } catch (e) {
-    list.innerHTML = `<section class="note err">Could not reach the server: ${e.message}</section>`; return;
-  }
-  admin = !!data.admin;
-  document.getElementById('who').innerHTML = `signed in as <b>${data.user || '?'}</b>${admin ? '' : ' · read only'}`;
-
-  const items = (data.items || []).sort((a, b) =>
-    (a.state === b.state ? b.age_s - a.age_s : a.state.localeCompare(b.state)));
-  if (!items.length) { list.innerHTML = '<section class="note">Nothing is stuck. Everything recorded has published.</section>'; return; }
-
-  list.innerHTML = '';
-  for (const it of items) {
-    const el = document.createElement('section');
-    // "writing" means it is still being recorded: acting on it would cut a live recording, so no buttons.
-    const actionable = admin && it.state !== 'writing';
-    el.innerHTML = `
-      <div class="row">
-        <div class="state ${it.state}">${it.state}</div>
-        <div>
-          <div class="name">${short(it.src)}</div>
-          <div class="meta">waiting ${hms(it.age_s)} · ${dur(it.duration_s)} long · ${(it.bytes/1e6).toFixed(0)} MB</div>
-          <div class="why">${it.reason || ''}</div>
-        </div>
-        <div class="acts">
-          <button data-act="retry" ${actionable ? '' : 'disabled'}>Retry</button>
-          <button data-act="drop" class="danger" ${actionable ? '' : 'disabled'}>Bin</button>
-        </div>
-      </div>`;
-    for (const b of el.querySelectorAll('button')) {
-      b.onclick = () => act(it, b.dataset.act, el);
-    }
-    list.appendChild(el);
-  }
-}
-
-async function act(item, action, el) {
-  if (action === 'drop' && !confirm(`Move this recording to the trash?\n\n${short(item.src)}\n\nRecoverable for three days.`)) return;
-  for (const b of el.querySelectorAll('button')) b.disabled = true;
-  const out = document.createElement('pre');
-  out.textContent = action === 'retry' ? 'Clearing the blocker…' : 'Moving to trash…';
-  el.appendChild(out);
-  try {
-    const r = await fetch(API, {
-      method: 'POST', credentials: 'same-origin',
-      headers: { 'Content-Type': 'application/json' },
-      body: JSON.stringify({ action, src: item.src }),
-    });
-    const body = await r.json();
-    out.textContent = body.error ? `Failed: ${body.error}` : (body.output || []).join('\n') || 'Done.';
-    if (!body.error) setTimeout(load, 1200);
-  } catch (e) {
-    out.textContent = `Failed: ${e.message}`;
-  }
-}
-
-load();
-setInterval(load, 30000);
-</script>
-</body>
-</html>

switchToClone.sh

@@ -3,13 +3,13 @@
 #   ./switchToClone.sh key     -> creates a deploy key and prints the public half (add it in Gitea: repo → Settings → Deploy keys, read-only)
 #   ./switchToClone.sh clone   -> clones over ssh (port 2222), moves .env + generated files + plugin zips into it
 set -euo pipefail
-REPO_SSH="${REPO_SSH:-ssh://git@git.dumpstervigilantes.rip:2222/techy/Media-Server-Setup.git}"
+REPO_SSH="${REPO_SSH:-ssh://git@git.vng.one:2222/techy/Media-Server-Setup.git}"
 DIR="$HOME/vanguard-stream"; KEY="$HOME/.ssh/vanguard-deploy"
 case "${1:-}" in
   key)
     [[ -f $KEY ]] || ssh-keygen -t ed25519 -N "" -C "vanguard-deploy@$(hostname)" -f "$KEY" >/dev/null
     cat > "$HOME/.ssh/config.vanguard" <<CFG
-Host git.dumpstervigilantes.rip
+Host git.vng.one
   Port 2222
   User git
   IdentityFile $KEY
@@ -19,7 +19,7 @@ CFG
     echo "Add this as a READ-ONLY deploy key on the repo (Gitea → repo → Settings → Deploy Keys):"; echo; cat "$KEY.pub" ;;
   clone)
     [[ -d $DIR/.git ]] && { echo "$DIR is already a clone"; exit 0; }
-    ssh -o StrictHostKeyChecking=accept-new -p 2222 git@git.dumpstervigilantes.rip 2>&1 | grep -qi "successfully authenticated\|Hi there" || { echo "deploy key not accepted yet — run './switchToClone.sh key' and add it in Gitea"; exit 1; }
+    ssh -o StrictHostKeyChecking=accept-new -p 2222 git@git.vng.one 2>&1 | grep -qi "successfully authenticated\|Hi there" || { echo "deploy key not accepted yet — run './switchToClone.sh key' and add it in Gitea"; exit 1; }
     mv "$DIR" "$DIR.old"
     git clone --quiet "$REPO_SSH" "$DIR"
     for f in .env Caddyfile mediamtx.yml copyparty.conf stream-token.secret deploy.env; do [[ -f $DIR.old/$f ]] && cp -a "$DIR.old/$f" "$DIR/$f"; done

systemd/normalize-sweep.service

@@ -9,4 +9,5 @@ ExecStart=/usr/local/bin/normalize-sweep.sh
 Nice=15
 IOSchedulingClass=idle
 Environment=ENCODE_THREADS=2   # x264 threads per conversion; keeps Jellyfin and the live portal responsive (default 2)
+# Environment=STITCH_MAX_PARTS=500  # join a stream's fragments when fewer than this (0 = never); STITCH_MAX_GAP_S=600 splits sessions
 # Environment=FILL_GAPS_MS=1000   # opt-in: CFR re-encode streams whose dropped-frame holes total >= 1 s (hours of CPU on this box)