techy/Media-Server-Setup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

diag host: real path prefixes for every UI, and an index page served as a file

Browse Source 
Each service now runs behind the prefix it is published under (-http.pathPrefix, or route-prefix plus
external-url for Alertmanager). Without that, three of the four UIs would have loaded at their sub-path and
then fetched their assets from the domain root, so the pages would have come up broken. The prefixes cascade,
so the scrape paths, vmalert's notifier and remote-write URLs, and the deadman's health probes all follow.

The index is diag/index.html mounted into Caddy, not an inline respond: Caddy reads {...} as placeholders and
the page's CSS is full of braces, so inline would have been mangled. It also matches how the gateway, dock and
clips pages are already served.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Brandyn / Techy
2026-11-03 16:17:47 -06:00
parent d3ff452546
commit da9a1f4fa6
5 changed files with 66 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
bin/vanguard-deadman.sh
View File

@@ -35,11 +35,13 @@ alive() { # alive <label> <url>
[[ $code == 2* || $code == 3* ]] || note "$1 is not answering (HTTP ${code:-no response})"
}
alive "VictoriaLogs" "http://127.0.0.1:9428/logs/health"
alive "VictoriaMetrics" "http://127.0.0.1:8428/health"
alive "vmalert (metrics)" "http://127.0.0.1:8880/health"
alive "vmalert (logs)" "http://127.0.0.1:8881/health"
alive "Alertmanager" "http://127.0.0.1:9093/-/healthy"
# Each service runs behind the path prefix it is published under on diag.HOST, so its health endpoint
# lives under that prefix too.
alive "VictoriaLogs" "http://127.0.0.1:9428/logs/health"
alive "VictoriaMetrics" "http://127.0.0.1:8428/metrics-db/health"
alive "vmalert (metrics)" "http://127.0.0.1:8880/rules-metrics/health"
alive "vmalert (logs)" "http://127.0.0.1:8881/rules-logs/health"
alive "Alertmanager" "http://127.0.0.1:9093/alerts/-/healthy"
# 2. the heartbeat. A missing file right after a restart is not yet a fault, so only age counts.
if [[ -f $HEARTBEAT ]]; then

12
configure.py
View File

@@ -249,15 +249,9 @@ def write_caddy(domain, auth_host="", watch_host="", git_host="", bot_host="", l
reverse_proxy 127.0.0.1:8881 # vmalert over VictoriaLogs
}}
handle {{
respond "Vanguard diagnostics
/logs logs (VictoriaLogs)
/metrics-db metrics (VictoriaMetrics)
/alerts what is firing (Alertmanager)
/rules-metrics metric rules (vmalert)
/rules-logs log rules (vmalert)
Alerts go to Discord. The deadman runs on the host and watches this stack." 200
root * /srv/diag
try_files {{path}} /index.html
file_server # diag/index.html: what each path above is
}}
}}
"""

37
diag/index.html Normal file
View File

@@ -0,0 +1,37 @@
<!doctype html>
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>Vanguard diagnostics</title>
<style>
:root{--bg:#121417;--panel:#1c2026;--line:#2b313a;--fg:#f3f1ea;--mut:#a3acb8;--acc:#e5484d}
*{box-sizing:border-box}
body{background:var(--bg);color:var(--fg);font:16px/1.55 system-ui,-apple-system,"Segoe UI",sans-serif;margin:0;padding:48px 20px}
main{max-width:680px;margin:0 auto}
h1{font-size:30px;margin:0 0 6px;letter-spacing:.3px}
.lead{color:var(--mut);margin:0 0 30px}
a.card{display:block;padding:15px 18px;margin:0 0 10px;background:var(--panel);border:1px solid var(--line);
border-radius:12px;color:var(--fg);text-decoration:none}
a.card:hover{border-color:var(--acc)}
a.card b{display:block;font-size:17px}
a.card span{color:var(--mut);font-size:14px}
.note{margin-top:30px;padding:16px 18px;background:var(--panel);border:1px solid var(--line);border-radius:12px;
color:var(--mut);font-size:14px}
.note b{color:var(--fg)}
footer{margin-top:26px;color:#6b7280;font-size:13px}
footer a{color:#6b7280}
</style></head>
<body><main>
<h1>Vanguard diagnostics</h1>
<p class="lead">Logs, metrics and alerts for this server. Visible to <code>media-admins</code>.</p>
<a class="card" href="/logs"><b>Logs</b><span>every container and our own timers, last 30 days · VictoriaLogs</span></a>
<a class="card" href="/metrics-db"><b>Metrics</b><span>host, containers, MediaMTX, Caddy, last 90 days · VictoriaMetrics</span></a>
<a class="card" href="/alerts"><b>Alerts</b><span>what is firing now, and silences · Alertmanager</span></a>
<a class="card" href="/rules-metrics"><b>Metric rules</b><span>disk, load, memory, RAID, container restart loops</span></a>
<a class="card" href="/rules-logs"><b>Log rules</b><span>sweep failures, publish denials, failed bot releases</span></a>
<div class="note"><b>A deadman runs on the host, outside all of this.</b> Every five minutes it checks that each
service here still answers, that the alerting path is still delivering, and that the disks are not filling, and
it reports to Discord itself. Nothing on this page can tell you it has died, which is the point.</div>
<footer>Vanguard Media · <a href="/logout">sign out</a></footer>
</main></body></html>

16
docker-compose.yml
View File

@@ -72,6 +72,7 @@ services:
- ./dock:/srv/dock:ro
- ./clips:/srv/clips:ro # clip-intake self-service page
- ./about:/srv/about:ro # gateway page (/about)
- ./diag:/srv/diag:ro # diagnostics index (diag.HOST)
- /srv/live:/srv/live:ro # live portal static build (deployLive.sh)
- caddy_data:/data
- caddy_config:/config
@@ -395,6 +396,7 @@ services:
- -storageDataPath=/data
- -retentionPeriod=90d
- -httpListenAddr=:8428
- -http.pathPrefix=/metrics-db # served at diag.HOST/metrics-db, so its own UI must emit that prefix
- -promscrape.config=/etc/vm/scrape.yml
- -promscrape.config.strictParse=false
ports:
@@ -412,11 +414,12 @@ services:
image: docker.io/victoriametrics/vmalert:latest
container_name: vmalert-metrics
command:
- -datasource.url=http://victoriametrics:8428
- -notifier.url=http://alertmanager:9093
- -remoteWrite.url=http://victoriametrics:8428
- -datasource.url=http://victoriametrics:8428/metrics-db
- -notifier.url=http://alertmanager:9093/alerts
- -remoteWrite.url=http://victoriametrics:8428/metrics-db
- -rule=/etc/alerts/alerts-metrics.yml
- -httpListenAddr=:8880
- -http.pathPrefix=/rules-metrics
- -evaluationInterval=1m
ports:
- "127.0.0.1:8880:8880"
@@ -436,10 +439,11 @@ services:
# LogsQL stats endpoints, which answer in the Prometheus format vmalert expects.
command:
- -datasource.url=http://victorialogs:9428/logs
- -notifier.url=http://alertmanager:9093
- -remoteWrite.url=http://victoriametrics:8428
- -notifier.url=http://alertmanager:9093/alerts
- -remoteWrite.url=http://victoriametrics:8428/metrics-db
- -rule=/etc/alerts/alerts-logs.yml
- -httpListenAddr=:8881
- -http.pathPrefix=/rules-logs
- -evaluationInterval=2m
ports:
- "127.0.0.1:8881:8881"
@@ -459,6 +463,8 @@ services:
- --config.file=/etc/alertmanager/alertmanager.yml
- --storage.path=/data
- --web.listen-address=:9093
- --web.route-prefix=/alerts
- --web.external-url=https://${DIAG_HOST:-diag.example.com}/alerts
ports:
- "127.0.0.1:9093:9093"
volumes:

8
observability/scrape.yml
View File

@@ -18,5 +18,13 @@ scrape_configs:
metrics_path: /logs/metrics # it runs behind -http.pathPrefix=/logs
- job_name: victoriametrics
static_configs: [{targets: ["127.0.0.1:8428"]}]
metrics_path: /metrics-db/metrics
- job_name: alertmanager
static_configs: [{targets: ["alertmanager:9093"]}]
metrics_path: /alerts/metrics
- job_name: vmalert
static_configs: [{targets: ["vmalert-metrics:8880"], labels: {rules: metrics}}]
metrics_path: /rules-metrics/metrics
- job_name: vmalert-logs
static_configs: [{targets: ["vmalert-logs:8881"], labels: {rules: logs}}]
metrics_path: /rules-logs/metrics