Files
Media-Server-Setup/Caddyfile.example
Brandyn / Techy 8ae43905dc Phase 13 build: logs, metrics, alerting and a deadman on a diag. host
Six containers, all loopback-published, reachable only through the gated diag. host:
  victorialogs      30d retention, -http.pathPrefix=/logs so Caddy passes the prefix through unchanged
  vector            docker_logs + journald -> VictoriaLogs (elasticsearch bulk, the documented path)
  victoriametrics   90d, scrapes the four taps that were already live plus the new services
  vmalert-metrics   11 metric rules: disk, load, memory, RAID, restart loops, a core container gone
  vmalert-logs      5 LogsQL rules: sweep failures, publish-deny bursts, failed bot deploys, cert trouble
  alertmanager      groups and inhibits, posts to Discord, webhook read from a 0600 file not the config

The deadman (bin/vanguard-deadman.sh, host timer, 5 min) is the part that does not trust any of the above:
curl only, it checks each component answers, that the always-firing Watchdog alert is still arriving through
vmalert -> Alertmanager -> dumpd /heartbeat, and the disks. It reports on change and announces recovery once.
Verified here: failures listed, silence when nothing changes, disk threshold, recovery fired once.

Collection uses the Docker socket (Phase 13 option A, chosen explicitly). That is root-equivalent on this host
and is recorded as such in vector.yaml with the two upgrade paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-11-03 15:45:58 -06:00

366 lines
13 KiB
Caddyfile

{
servers {
metrics # Prometheus metrics at http://127.0.0.1:2019/metrics (LOGGING.md)
}
}
# vanguard:main
media.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
# /logout on every gated host = sign out of the Discord login (domain-wide session)
redir /logout /outpost.goauthentik.io/sign_out 302
# Authentik forward-auth (embedded outpost). Needs a Proxy Provider in Authentik — domain-level for the
# whole domain (recommended) or single-application for this host — assigned to the embedded outpost.
# Until then: 4xx, never open. Per-host group requirements are enforced below from X-Authentik-Groups.
reverse_proxy /outpost.goauthentik.io/* 127.0.0.1:9000
forward_auth 127.0.0.1:9000 {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Entitlements X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies private_ranges
}
# who does Authentik say I am? (debug: shows exactly the identity headers the apps receive)
handle /whoami {
header Content-Type text/plain
respond "user: {http.request.header.X-Authentik-Username}
groups: {http.request.header.X-Authentik-Groups}
email: {http.request.header.X-Authentik-Email}
name: {http.request.header.X-Authentik-Name}"
}
@me path /me /me/*
handle @me {
reverse_proxy 127.0.0.1:8080 # dumpd: identity for the gateway page
}
redir /about* / 308 # old bookmarks: the gateway page used to live at /about/
handle {
root * /srv/about
try_files {path} /index.html
file_server # the gateway page is the root of the main host
}
}
# vanguard:files
files.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
# /logout on every gated host = sign out of the Discord login (domain-wide session)
redir /logout /outpost.goauthentik.io/sign_out 302
# Authentik forward-auth (embedded outpost). Needs a Proxy Provider in Authentik — domain-level for the
# whole domain (recommended) or single-application for this host — assigned to the embedded outpost.
# Until then: 4xx, never open. Per-host group requirements are enforced below from X-Authentik-Groups.
reverse_proxy /outpost.goauthentik.io/* 127.0.0.1:9000
forward_auth 127.0.0.1:9000 {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Entitlements X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies private_ranges
}
# copyparty at the root. It honours X-Authentik-Username / -Groups only when the request also carries a header
# whose secret NAME only this proxy knows (copyparty idp-h-key). Clients cannot forge identity.
reverse_proxy 127.0.0.1:3923 {
header_up X-Vanguard-k 1
flush_interval -1
}
}
# vanguard:sync
sync.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
# /logout on every gated host = sign out of the Discord login (domain-wide session)
redir /logout /outpost.goauthentik.io/sign_out 302
# Authentik forward-auth (embedded outpost). Needs a Proxy Provider in Authentik — domain-level for the
# whole domain (recommended) or single-application for this host — assigned to the embedded outpost.
# Until then: 4xx, never open. Per-host group requirements are enforced below from X-Authentik-Groups.
reverse_proxy /outpost.goauthentik.io/* 127.0.0.1:9000
forward_auth 127.0.0.1:9000 {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Entitlements X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies private_ranges
}
@lacks_group not header_regexp X-Authentik-Groups (^|\|)media\-admins(\||$)
handle @lacks_group {
respond "This page is for members of the 'media-admins' group." 403
}
# Syncthing admin UI (its own GUI login is disabled; the gate + group is the login)
reverse_proxy 127.0.0.1:8384 {
header_up Host {upstream_hostport}
}
}
# vanguard:clips
clips.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
# /logout on every gated host = sign out of the Discord login (domain-wide session)
redir /logout /outpost.goauthentik.io/sign_out 302
# Authentik forward-auth (embedded outpost). Needs a Proxy Provider in Authentik — domain-level for the
# whole domain (recommended) or single-application for this host — assigned to the embedded outpost.
# Until then: 4xx, never open. Per-host group requirements are enforced below from X-Authentik-Groups.
reverse_proxy /outpost.goauthentik.io/* 127.0.0.1:9000
forward_auth 127.0.0.1:9000 {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Entitlements X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies private_ranges
}
@me path /me /me/*
handle @me {
reverse_proxy 127.0.0.1:8080 # dumpd: identity + /me/clips self-service
}
handle {
root * /srv/clips
try_files {path} /index.html
file_server
}
}
# vanguard:diag
diag.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
# /logout on every gated host = sign out of the Discord login (domain-wide session)
redir /logout /outpost.goauthentik.io/sign_out 302
# Authentik forward-auth (embedded outpost). Needs a Proxy Provider in Authentik — domain-level for the
# whole domain (recommended) or single-application for this host — assigned to the embedded outpost.
# Until then: 4xx, never open. Per-host group requirements are enforced below from X-Authentik-Groups.
reverse_proxy /outpost.goauthentik.io/* 127.0.0.1:9000
forward_auth 127.0.0.1:9000 {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Entitlements X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies private_ranges
}
@lacks_group not header_regexp X-Authentik-Groups (^|\|)media\-admins(\||$)
handle @lacks_group {
respond "This page is for members of the 'media-admins' group." 403
}
# VictoriaLogs runs with -http.pathPrefix=/logs and emits /logs/... URLs itself, so the prefix is
# passed through unchanged. Stripping it would break the UI's own asset paths.
redir /logs /logs/select/vmui/ 302
redir /logs/ /logs/select/vmui/ 302
handle /logs* {
reverse_proxy 127.0.0.1:9428
}
handle /metrics-db* {
reverse_proxy 127.0.0.1:8428 # VictoriaMetrics UI + query API
}
handle /alerts* {
reverse_proxy 127.0.0.1:9093 # Alertmanager: what is firing, silences
}
handle /rules-metrics* {
reverse_proxy 127.0.0.1:8880 # vmalert over VictoriaMetrics
}
handle /rules-logs* {
reverse_proxy 127.0.0.1:8881 # vmalert over VictoriaLogs
}
handle {
respond "Vanguard diagnostics
/logs logs (VictoriaLogs)
/metrics-db metrics (VictoriaMetrics)
/alerts what is firing (Alertmanager)
/rules-metrics metric rules (vmalert)
/rules-logs log rules (vmalert)
Alerts go to Discord. The deadman runs on the host and watches this stack." 200
}
}
# vanguard:live
live.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
# /logout on every gated host = sign out of the Discord login (domain-wide session)
redir /logout /outpost.goauthentik.io/sign_out 302
# Authentik forward-auth (embedded outpost). Needs a Proxy Provider in Authentik — domain-level for the
# whole domain (recommended) or single-application for this host — assigned to the embedded outpost.
# Until then: 4xx, never open. Per-host group requirements are enforced below from X-Authentik-Groups.
reverse_proxy /outpost.goauthentik.io/* 127.0.0.1:9000
forward_auth 127.0.0.1:9000 {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Entitlements X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies private_ranges
}
handle /whoami {
header Content-Type text/plain
respond "user: {http.request.header.X-Authentik-Username}
groups: {http.request.header.X-Authentik-Groups}"
}
@me path /me /me/*
handle @me {
reverse_proxy 127.0.0.1:8080 # dumpd: identity, stream tokens (Go Live + dock)
}
handle /time {
reverse_proxy 127.0.0.1:8080 # server clock for the timecode strip (clock page + Go Live)
}
handle /dump* {
reverse_proxy 127.0.0.1:8080 # dump tonight (dock)
}
handle_path /dock* {
root * /srv/dock
file_server # OBS browser dock + /dock/clock/
}
handle_path /api/* {
reverse_proxy 127.0.0.1:9997
}
handle_path /webrtc/* {
# Go Live sends MediaMTX credentials in X-Stream-Auth so Authentik's forward-auth never sees an
# Authorization header; Caddy turns it into Authorization only on the hop to MediaMTX.
@creds header X-Stream-Auth *
reverse_proxy @creds 127.0.0.1:8889 {
header_up Authorization {http.request.header.X-Stream-Auth}
header_up -X-Stream-Auth
}
reverse_proxy 127.0.0.1:8889
}
handle_path /hls/* {
reverse_proxy 127.0.0.1:8888
}
handle {
root * /srv/live
try_files {path} /index.html
file_server
}
}
# vanguard:bot
bot.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
# /logout on every gated host = sign out of the Discord login (domain-wide session)
redir /logout /outpost.goauthentik.io/sign_out 302
# Authentik forward-auth (embedded outpost). Needs a Proxy Provider in Authentik — domain-level for the
# whole domain (recommended) or single-application for this host — assigned to the embedded outpost.
# Until then: 4xx, never open. Per-host group requirements are enforced below from X-Authentik-Groups.
reverse_proxy /outpost.goauthentik.io/* 127.0.0.1:9000
forward_auth 127.0.0.1:9000 {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Entitlements X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies private_ranges
}
@lacks_group not header_regexp X-Authentik-Groups (^|\|)bot\-users(\||$)
handle @lacks_group {
respond "This page is for members of the 'bot-users' group." 403
}
reverse_proxy 127.0.0.1:8000
}
# vanguard:git
git.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
reverse_proxy 127.0.0.1:3000
}
# vanguard:auth
auth.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
reverse_proxy 127.0.0.1:9000
}
# vanguard:watch
watch.example.com {
encode zstd gzip
log {
output stderr
format filter {
wrap json
fields {
request>headers>Cookie delete
request>headers>Authorization delete
request>headers>X-Stream-Auth delete
}
}
}
reverse_proxy 127.0.0.1:8096
}