Generate alertmanager.yml so no-webhook is a valid state, not a broken one

Deploying before the webhook exists is deliberate, so it must not depend on Alertmanager tolerating an empty
webhook_url_file — which I could not verify. configure.py now generates the config and omits the Discord
notifier entirely when ALERT_WEBHOOK_URL is unset: the receiver exists, drops notifications, and alerts still
evaluate, group and show at /alerts. Setting the variable and re-applying turns delivery on. The URL stays in
/etc/vanguard/alert-webhook rather than the config, because that file can be 0640 root:65534 for the
container's user while the repo copy cannot.

alertmanager.example.yml is the reference copy and is gitignored in generated form, like the other configs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-11-03 16:01:49 -06:00
parent cea02d2ee8
commit d3ff452546
6 changed files with 68 additions and 5 deletions

View File

@@ -439,6 +439,10 @@ alert by itself. Metric rules (`observability/alerts-metrics.yml`) cover disk, l
loops and a missing core container; log rules (`observability/alerts-logs.yml`, LogsQL) cover sweep failures,
publish-deny bursts, failed bot releases and certificate trouble. Both notify one Discord webhook,
`ALERT_WEBHOOK_URL` in `.env`, written to `/etc/vanguard/alert-webhook` so it never sits in a config file.
**Running without a webhook is a supported state**, not a half-configured one: `configure.py` generates
`observability/alertmanager.yml` with the Discord notifier left out entirely, so alerts still evaluate, group
and appear at `/alerts`, they are simply not delivered. The deadman behaves the same way, logging to the
journal and exiting non-zero when it finds something it could not report. Set the variable, re-apply, done.
**The deadman is the part that matters most.** `vanguard-deadman.timer` runs on the host every 5 minutes with
nothing but curl: are all six answering, is the always-firing `Watchdog` alert still coming through the whole