Stuck-recording triage, a diag landing page, and the fixes behind Tuesday's symptoms

Triage
- bin/vgstuck.sh classifies every unpublished recording (parked by a failure log, published
  collision, waiting, still writing), with ffmpeg's own reason for parked ones. retry clears the
  blocker; drop moves to trash/<today>/ with the same 3-day undo as a dumped night; drop --all is
  refused. A leftover .mp4.part blocks nothing (ffmpeg overwrites with -y) and is only cleaned up.
- The same tool backs a page at diag.HOST/stuck. dumpd serves /stuck/api, lets anyone past the gate
  read, and allows retry/bin only for the admin groups; the path validation is the single thing
  between a request body and a file move, so it accepts only an existing .mp4 under .live/ inside
  the streams tree.
- The sweep logs event=parked once per run so Kibana can name stuck files rather than only count
  them.

Landing
- diag.HOST opens on a "Vanguard ops" dashboard via Kibana's defaultRoute: a markdown panel of links
  to every host and the triage page, plus Discover, Dashboards, Stack Rules and Index Management.
  Kibana stays at the root; moving it under /kibana would have meant the prefix in both Beats'
  setup.kibana, two scripts and the deadman, with Metricbeat crash-looping on whichever was missed.
- /api/kibana/settings is an internal API behind server.restrictInternalApis, which Kibana 9 turns
  on by default, so the script sends x-elastic-internal-origin. kibana-setup.py takes a step name and
  verifies each step by reading it back.

Installers
- bin/install.list is the one list of what lands in /usr/local/bin, read by install.sh,
  configure.py and deploy.sh. deploy.sh's own copy had drifted to five of ten files. check.sh fails
  on drift. deploy.sh also recreates a Beat or Kibana when its bind-mounted config changes, which
  nothing did before.
- check.sh distinguishes absent from broken: the server checkout is partial, and a yaml file or node
  binary that is not there is a skip, not a failure.

Fixes
- Go Live pixelation: x-google-min-bitrate=6000 forbade Chrome from encoding below 6 Mbps, so on a
  weak uplink the excess was dropped and MediaMTX logged invalid FU-A packets. Loss was on the
  publish sessions only. Floor is 1200 and scaleResolutionDownBy is unpinned.
- HLS manifestParsingError: handle_path strips /hls, so MediaMTX's cookieCheck redirect came back
  root-relative and landed on the SPA catch-all. header_down rewrites Location.
- Sweep audio: the band-removal branch overwrote $reason and lost the probe's audio=opus marker, so a
  Go Live recording with the band shipped Opus inside an MP4. probe_reason is kept separately.
- Deadman false alarm: alerts.state was only written when something changed, so quiet runs let it go
  stale. Every run stamps last_run now, and the deadman also checks is-failed on the unit.
- Both timer scripts load /etc/vanguard/alerts.env themselves, so a run by hand matches the timer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

LOGGING.md

@@ -29,6 +29,26 @@ and `docker` modules, not from an exporter — node_exporter and cAdvisor were r
 went in. None of these endpoints is reachable from outside: each binds loopback, the firewall opens none of
 them, and Caddy routes no host to `/metrics` (every dumpd route there is an explicit path matcher).
 
+### Stuck recordings: counts in Kibana, names in the logs, decisions on the host
+
+Three layers on purpose, because each answers a different question and none of them needs a new web page.
+
+- **How bad is it, and is it getting worse** — the gauges: `vanguard_media_pending`,
+  `vanguard_media_pending_oldest_seconds`, `vanguard_media_failed_files`. A Lens dashboard over
+  `metricbeat-*` shows the trend across a night, which is the thing a list never can.
+- **Which recordings** — the sweep logs `event=parked file= stream= log=` once per run for every file a
+  failure log is holding back, plus `event=failed` with ffmpeg's own stderr when it first breaks. Both land
+  in `filebeat-*`, so a Discover search or a dashboard table names them and stays current. Names deliberately
+  do NOT come from the metrics: a label per filename would wreck index cardinality for no gain.
+- **What to do about it** — `bin/vgstuck.sh` on the host. It classifies every unpublished recording as
+  `parked` (a failure log is blocking it, with the reason), `published` (an output already exists),
+  `waiting` (nothing blocking it yet) or `writing` (still recording), then `retry <match>` clears the
+  blocker so the sweep tries again and `drop <match>` moves it to `trash/<today>/` for the same three-day
+  undo as a dumped night. `drop --all` is refused.
+
+A leftover `<output>.mp4.part` does **not** block anything — ffmpeg overwrites it with `-y`. It only wastes
+disk, and `retry` clears it.
+
 ### Our own metrics, and how to add one
 
 MediaMTX answers "who is live" and Metricbeat answers "is the box busy". Neither can answer "is normalising
@@ -135,8 +155,15 @@ to Discord, and the pipeline is split:
 
     Kibana rule  --.index connector-->  vanguard-alerts  --vanguard-alerts.timer-->  Discord webhook
 
-- **`observability/kibana-setup.py`** creates the connector ("Vanguard alert index") and the heartbeat
-  rule. Run it once, *after* Elasticsearch security is on.
+- **`observability/kibana-setup.py`** creates the connector ("Vanguard alert index"), the heartbeat rule,
+  the data view, and the "Vanguard ops" landing dashboard plus the `defaultRoute` that makes `diag.BASE`
+  open on it. Run it once, *after* Elasticsearch security is on. Name a step to run only that one:
+  `kibana-setup.py landing`.
+  Kibana deliberately stays at the **root** of that host. Moving it under `/kibana` would mean
+  `server.basePath` plus `rewriteBasePath`, a Caddy handle that does not strip the prefix, and appending the
+  prefix in both Beats' `setup.kibana`, this script, the alarm script and the deadman's status check — and a
+  miss in the Beats one crash-loops Metricbeat, because it exits when Kibana is unreachable. `defaultRoute`
+  buys the landing page without any of that.
 - **`vanguard-alerts` index** is the alert bus, pre-created by `configure.py` with `rule`, `level` and `kind`
   as keywords. Alerts are ordinary documents, so they are searchable in Discover next to the logs that caused
   them, and replayable after the fact.