metricbeat sets the same correlation fields as filebeat, so one dashboard spans logs and metrics

Both Beats already wrote to the same Elasticsearch, but a metric event carried container.name while a log
event carried service.name and vanguard.category, so nothing joined them and a dashboard could only show the
two side by side. Metricbeat now runs add_docker_metadata and the same copy_fields, giving both sides
@timestamp, host.name, container.name, service.name and vanguard.category.

LOGGING.md documents the join keys and the one Kibana step that makes it work: a single data view over
filebeat-*,metricbeat-* so one filter applies to metric panels and log panels together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

LOGGING.md

@@ -86,6 +86,38 @@ One global retention period in VictoriaLogs (`-retentionPeriod`); start at 30 da
 
 ## As built (Phase 13)
 
+| container | port | job |
+|---|---|---|
+| `elasticsearch` | 9200 | one store for **both** logs and metrics. 2 GB heap, single node, security off (loopback only) |
+| `kibana` | 5601 | the single pane: Discover for logs, dashboards, alerting. Served at the root of `diag.BASE` |
+| `filebeat` | — | container logs (autodiscover) + our journal units |
+| `metricbeat` | — | host + Docker metrics (retires node_exporter and cAdvisor) + MediaMTX and Caddy Prometheus endpoints |
+
+**Logs and metrics correlate because both Beats write the same fields into the same store.** That is what makes
+one dashboard able to show a CPU spike next to the errors that caused it, rather than two dashboards side by
+side:
+
+| field | set by | use |
+|---|---|---|
+| `@timestamp` | both (ECS) | every panel shares the dashboard's time range |
+| `host.name` | both | one box today, but the field is there |
+| `container.name` | both | per-service metrics joined to that service's log lines |
+| `service.name` | both, from the `vanguard.service` label | the name you actually think in |
+| `vanguard.category` | both, from the label | filter a whole area at once: stream, media, auth, intake, deploy |
+
+To get a dashboard that filters across both, make **one data view** spanning `filebeat-*,metricbeat-*` in
+Kibana (Stack Management → Data Views). A single filter such as `service.name: mediamtx` then applies to the
+metric panels and the log panel together. Metricbeat also ships its own system and Docker dashboards, which are
+a good starting point to copy panels from.
+
+Retention is one edit in Kibana rather than config: Stack Management → Index Lifecycle Policies → the beats
+policy → add a delete phase. Doing it there rather than in a file keeps it adjustable without a redeploy.
+
+### Old notes
+
+
+(superseded 2026-11-03 — the Victoria/vmalert/Alertmanager shape, kept for the reasoning)
+
 | container | port | job |
 |---|---|---|
 | `victorialogs` | 9428 | store, 30 d, `-http.pathPrefix=/logs` so Caddy passes the prefix through unchanged |

observability/metricbeat.yml

@@ -29,6 +29,17 @@ metricbeat.modules:
 
 processors:
   - add_host_metadata: ~
+  # The same correlation fields Filebeat sets, so ONE dashboard filter spans logs and metrics. Without these a
+  # metric event carries container.name while a log event carries service.name, and nothing joins them.
+  - add_docker_metadata: ~
+  - copy_fields:
+      fields:
+        - from: container.labels.vanguard_service
+          to: service.name
+        - from: container.labels.vanguard_category
+          to: vanguard.category
+      fail_on_error: false
+      ignore_missing: true
 
 output.elasticsearch:
   hosts: ["http://127.0.0.1:9200"]