Phase 14: diag. host with VictoriaLogs at /logs, ops controls later
Records the store decision (VictoriaLogs over ELK and over Grafana+Loki, with the reasons), the hostname and path, the collector choice, and the one real decision left for build time: how Vector reads container logs without handing it root through the Docker socket. Also a design note for the later ops controls, so restart buttons end up as allowlisted dumpd endpoints rather than a web page with Docker access. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
17
LOGGING.md
17
LOGGING.md
@@ -1,7 +1,8 @@
|
||||
# Logging framework — store-agnostic
|
||||
# Logging framework
|
||||
|
||||
Decided 2026-11-01. The log *store* (Grafana + Loki, VictoriaLogs, Netdata, …) is not chosen yet. Everything
|
||||
upstream of it is fixed now, so the choice later is a collector config, not a code change.
|
||||
Decided 2026-11-01, and the store chosen on 2026-11-03: **VictoriaLogs**, on the `diag.BASE` host with its UI
|
||||
at `/logs` (Phase 14 in `CommunityDiscordMedia.md`). Everything upstream of the store was fixed first, so
|
||||
adopting it is a collector config rather than a code change.
|
||||
|
||||
## Where the lines come from (the only two taps a collector needs)
|
||||
|
||||
@@ -76,17 +77,19 @@ On for every site block (`CADDY_LOG` in `configure.py`): JSON to stderr; `Cookie
|
||||
`X-Stream-Auth` request headers are deleted before logging, the `X-Authentik-*` identity headers are kept — that is the
|
||||
"who downloaded what" audit trail for the files host.
|
||||
|
||||
## Retention (when a store exists)
|
||||
## Retention
|
||||
|
||||
14 days for everything; 90 for `auth` and `deploy`. Both Loki and VictoriaLogs do per-stream retention.
|
||||
One global retention period in VictoriaLogs (`-retentionPeriod`); start at 30 days and watch the disk. Keeping
|
||||
`auth` and `deploy` longer than the rest would need separate tenants, which is not worth it unless an audit does.
|
||||
|
||||
## Wiring sketches (not deployed; pick one later)
|
||||
|
||||
- **Grafana + Loki**: `alloy` container with `loki.source.journal` + `loki.source.docker` (labels from the container
|
||||
labels above) → `loki` on loopback → `grafana` behind Authentik OIDC (third provider, like Jellyfin/Gitea).
|
||||
Metrics: Alloy scrapes the four taps above → Prometheus or VictoriaMetrics, same Grafana.
|
||||
- **VictoriaLogs**: single container, journald + docker via its agent or Vector; built-in UI behind the Caddy
|
||||
forward-auth gate (`media-admins`). VictoriaMetrics beside it for metrics.
|
||||
- **VictoriaLogs** (chosen): single container on `127.0.0.1:9428`, run with `-http.pathPrefix=/logs`; Vector
|
||||
ships journald and the Docker logs. Its built-in UI sits behind the Caddy forward-auth gate on `diag.BASE`
|
||||
(`media-admins`). VictoriaMetrics beside it would consume the four metric taps.
|
||||
- **Netdata**: single container, host + container metrics and alerts (Discord) out of the box, journald viewer;
|
||||
behind the Caddy gate. Weakest at log search.
|
||||
- **ELK**: rejected — JVM heap and index disk on a box doing x264; our data is structured records (→ MariaDB) and a
|
||||
|
||||
Reference in New Issue
Block a user