Phase 13 integration: install.sh, configure.py, and a deadman that degrades cleanly

install.sh: the three data dirs with correct owners (uid 1000 for the VictoriaMetrics pair, 65534 for
alertmanager), /var/lib/vanguard, the deadman script and timer, and /etc/vanguard/alert-webhook as a FILE —
compose bind-mounts it, so without it Docker creates a directory and alertmanager will not start.

configure.py: --full now asks for the alert webhook and stores it; --show says plainly when none is set; the
webhook lands at 0640 root:65534 rather than world-readable, since it is a credential for that channel.

The deadman no longer assumes a webhook. It trims and validates the file (the previous -s test passed on the
newline configure.py writes), skips delivery when there is none, still logs every finding, and exits non-zero
only when it found something it could not report — so the unit state is the signal. Verified for a missing
file, a newline-only file, a junk value, and the healthy case.

check.sh validates all five observability configs; confirmed it catches a deliberate break.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-11-03 15:56:49 -06:00
parent 8ae43905dc
commit cea02d2ee8
8 changed files with 147 additions and 59 deletions

View File

@@ -42,6 +42,9 @@ install -d -o 1000 -g 1000 /srv/appdata/jellyfin/{config,cache} /srv/appdata/aut
install -d /srv/appdata/authentik/{db,redis} /srv/appdata/mariadb
install -d -o 1000 -g 1000 /srv/appdata/discord-bot /srv/appdata/gitea /srv/appdata/act_runner
install -d -o 1000 -g 1000 /srv/bot /srv/bot/app # the Discord bot's released CI bundle (bind-mounted at /app; see bot/README.md)
install -d -o 1000 -g 1000 /srv/appdata/victorialogs /srv/appdata/victoriametrics # run as uid 1000
install -d -o 65534 -g 65534 /srv/appdata/alertmanager # prom/alertmanager runs as nobody
install -d /var/lib/vanguard # deadman state
install -d -o "$OWNER" /srv/live # live portal static build (deployLive.sh)
install -d -o 1000 -g 1000 /srv/media/streams/{techy,diceofire} /srv/media/clips/{techy,diceofire}
for d in /srv/media/clips/*/; do
@@ -52,11 +55,18 @@ done
echo "== host scripts + units"
install -m 0755 bin/normalize-sweep.sh bin/trash-purge.sh bin/dumpd.py bin/vgstrip.py bin/vgdeband.sh /usr/local/bin/
install -m 0755 deployBotBundle.sh /usr/local/bin/vanguard-deploy-bot # dumpd runs this for the bot CI's POST /deploy-bot
install -m 0755 bin/vanguard-deadman.sh /usr/local/bin/ # watches the observability stack from outside it
# Compose bind-mounts this as a FILE; without it Docker would create a directory and alertmanager would not
# start. configure.py rewrites it from ALERT_WEBHOOK_URL on every apply; an empty one means "log only".
if [[ ! -f /etc/vanguard/alert-webhook ]]; then
install -d -m 0750 /etc/vanguard
install -m 0640 -o root -g 65534 /dev/null /etc/vanguard/alert-webhook
fi
install -d -m 0750 /etc/vanguard
install -m 0640 dump-users.txt /etc/vanguard/dump-users.txt
install -m 0644 systemd/*.service systemd/*.timer /etc/systemd/system/
systemctl daemon-reload
systemctl enable --now normalize-sweep.timer trash-purge.timer dumpd.service
systemctl enable --now normalize-sweep.timer trash-purge.timer dumpd.service vanguard-deadman.timer
echo "== sysctl"
cat > /etc/sysctl.d/90-vanguard.conf <<'SYS'