techy/Media-Server-Setup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Alarm mentions, targeted --set, and a logout that actually logs out

Browse Source 
Mentions
- bin/vgnotify.sh is shared by the alert timer and the deadman. A webhook will not ping a role
  from <@&id> in the content alone: the request has to whitelist it, so every post now sends an
  explicit allowed_mentions. parse is always [] as well, so an @everyone appearing inside a log
  line or filename in an alert text can never ping the server.
- ALERT_MENTION is written as role:<id>/user:<id> rather than raw Discord syntax, and
  ALERT_MENTION_LEVELS keeps pings to errors by default so warnings arrive quietly. The deadman
  pings too, because its alarm means nothing else can get through, but its recovery is info level.
- Both scripts now read /etc/vanguard/alerts.env themselves instead of relying on systemd to
  inject it. Running one by hand previously behaved differently from the timer, which is exactly
  how "the mention is configured but never appears" happened.

configure.py --set
- Change one setting and apply only what it needs. --full re-prompts for every hostname and
  secret, which is far too blunt for pointing the alarms at a new channel, and mistyping a
  hostname there has broken TLS before.
- Every Discord credential and webhook is now one commented block at the top of .env, so there is
  one place to look, and --show prints the same five together with the secrets truncated.
- Each key applies only its own side effect: the webhook rewrites its root-only file, the deploy
  webhook rewrites deploy.env, the bot token recreates the bot container, log retention re-injects
  the ILM delete phase. LIVE_WEBHOOK_URL deliberately stops short, because MediaMTX only reads it
  at start and a recreate cuts a live recording.

Logout
- The proxy provider pointed at default-provider-invalidation-flow ("Logged out of application"),
  which ends only that application's session and leaves the SSO session alive, so /logout dropped
  the proxy cookie and forward-auth signed the user straight back in. It looked like a refresh.
  default-invalidation-flow ("Logout") is what Authentik's own button uses. --apply now enforces

  it had explicitly recommended the wrong flow.

Sweep
- The band-removal branch overwrote $reasocodec> marker, which is the
  only signal telling the encode to convert audio. A Go Live stream with both the band and Opus
  audio therefore got -c:a copy and shippeachable once Windows Go
  Live audio started working.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Brandyn / Techy
2026-11-03 21:07:17 -06:00
parent b38d40ab0c
commit a9de09fa18
11 changed files with 371 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files

194
configure.py
View File

@@ -4,6 +4,9 @@
./configure.py --full first-time setup: asks every question (current values as defaults)
./configure.py manage: skips what is already set; add/remove/reset streamers & editors
./configure.py --show print current streamers, editors, domain and the OBS settings
./configure.py --set K=V change ONE setting and apply only what it needs (Discord tokens and webhooks,
retention, alert tuning). Run it with no arguments to list what is settable.
Everything else regenerates a config or a container, so it needs --apply.
./configure.py --apply regenerate everything from the values already on disk and apply (non-interactive;
used by deploy.sh). Fails if the main hostname was never set.
./configure.py --hosts AUTH WATCH [GIT [BOT [LIVE [FILES [SYNC [CLIPS [DIAG]]]]]]]
@@ -55,6 +58,115 @@ def read_hosts():
def read_domain(): return read_hosts().get("main", "")
# Every Discord credential and webhook, in one block, in this order, with these comments written into .env.
# They used to be scattered through the file in the order they were added, so "where does the alarm webhook
# go" had no obvious answer. read_env ignores any line starting with #, so the comments are free.
DISCORD_KEYS = (
("DISCORD_BOT_TOKEN", "the bot's own token (Discord dev portal -> Bot). The bot's CI also sets this from Gitea"),
("DISCORD_GUILD_ID", "the server the bot and the Authentik membership policies work against"),
("ALERT_WEBHOOK_URL", "webhook: alarms (vanguard-alerts.timer) and the deadman"),
("LIVE_WEBHOOK_URL", "webhook: 'a stream went live' (MediaMTX runOnReady)"),
("DEPLOY_NOTIFY_WEBHOOK_URL", "webhook: deploy results (deploy.sh)"),
)
DEPLOY_ENV_KEYS = ("DEPLOY_AUTO", "DEPLOY_WEBHOOK_SECRET", "DEPLOY_BRANCH", "DEPLOY_NOTIFY_WEBHOOK_URL",
"BOT_DEPLOY_AUTO", "BOT_DEPLOY_SECRET", "BOT_ENV_KEYS")
def render_env(env):
""".env text with the Discord block first and commented, everything else after it."""
named = dict(DISCORD_KEYS)
out = ["# ---- Discord: tokens and webhooks ----\n",
"# change any of these with: ./configure.py --set KEY=VALUE\n"]
for key, why in DISCORD_KEYS:
out.append(f"# {why}\n{key}={env.get(key, '')}\n")
out.append("\n# ---- everything else ----\n")
out += [f"{k}={v}\n" for k, v in env.items() if k not in named]
return "".join(out)
def write_deploy_env(env=None):
"""/etc/vanguard/deploy.env — read by dumpd's /deploy webhook and by deploy.sh."""
env = env or read_env()
Path("deploy.env").write_text("".join(f"{k}={env.get(k, '')}\n" for k in DEPLOY_ENV_KEYS)
+ f"DEPLOY_REPO_DIR={Path.cwd()}\n")
os.chmod("deploy.env", 0o600)
run(["sudo", "install", "-m", "0600", "deploy.env", "/etc/vanguard/deploy.env"])
Path("deploy.env").unlink(missing_ok=True)
# Settings that can be changed on their own, with the one side effect each needs. Everything else has to go
# through --apply or --full because it regenerates a config file or a container: DIAG_REQUIRE_GROUP needs the
# Caddyfile rewritten and reloaded, DIAG_KIBANA_ROLES needs the Elasticsearch user changed and Kibana
# restarted, hostnames need the whole pass. Keeping this list short is the point.
SETTABLE = {
"DISCORD_BOT_TOKEN": "the bot's token (restarts the bot container)",
"DISCORD_GUILD_ID": "the Discord server id (restarts the bot container)",
"ALERT_WEBHOOK_URL": "Discord webhook for alarms and the deadman (blank = journal only)",
"LIVE_WEBHOOK_URL": "Discord webhook for 'went live' (needs a MediaMTX recreate — NOT done for you)",
"DEPLOY_NOTIFY_WEBHOOK_URL": "Discord webhook for deploy results",
"ALERT_RETENTION_DAYS": "how long delivered alerts stay queryable in vanguard-alerts",
"ALERT_SUPPRESS_S": "per-rule repeat suppression in the delivery timer, seconds",
"ALERT_MENTION": "who an alarm pings, e.g. role:123,role:456 or user:789 (blank = nobody)",
"ALERT_MENTION_LEVELS": "which levels ping, comma separated (default error; add warn to widen)",
"HEARTBEAT_MAX_S": "how stale the alerting heartbeat may get before the deadman alarms, seconds",
"LOG_RETENTION_DAYS": "ILM delete age for filebeat-*/metricbeat-* (re-applied immediately)",
}
def set_env(updates):
"""Change individual .env keys in place and apply only what each one needs.
--full re-prompts for every hostname and secret, which is far too blunt for "point the alarms at a new
Discord channel" — mistyping a hostname there has broken TLS before. This writes the keys, rewrites the
files that derive from them, and touches nothing else.
"""
env = read_env()
if not env:
print("no .env here — run ./configure.py --full first"); sys.exit(1)
for key, value in updates.items():
if key not in SETTABLE:
print(f"{key} is not settable on its own. Settable keys:")
for name, why in SETTABLE.items():
print(f" {name:22} {why}")
print("Anything else regenerates a config or a container: use --apply (or --full for hostnames).")
sys.exit(1)
env[key] = value
Path(".env").write_text(render_env(env))
os.chmod(".env", 0o600)
for key, value in updates.items():
print(f" {key}={value if key != 'ALERT_WEBHOOK_URL' else (value[:40] + '…' if value else '(cleared)')}")
# The webhook is a credential for that channel, so it stays in its own root-only file.
if "ALERT_WEBHOOK_URL" in updates:
Path("alert-webhook").write_text((env.get("ALERT_WEBHOOK_URL") or "") + "\n"); os.chmod("alert-webhook", 0o600)
run(["sudo", "install", "-m", "0600", "-o", "root", "-g", "root", "alert-webhook", "/etc/vanguard/alert-webhook"])
Path("alert-webhook").unlink(missing_ok=True)
print(" wrote /etc/vanguard/alert-webhook")
# Always, not only when one of its keys changed: it is two file writes, it is idempotent, and making it
# conditional means the file is missing on any system where the first --set happened to be the webhook.
# A systemd timer never reads the repo's .env, so this file is the only way those knobs reach the scripts.
write_alerts_env(env)
if "DEPLOY_NOTIFY_WEBHOOK_URL" in updates:
write_deploy_env(env); print(" wrote /etc/vanguard/deploy.env")
if "LOG_RETENTION_DAYS" in updates:
es_security(compose_cmd()) # re-injects the ILM delete phase at the new age
# The bot reads these from its container environment, so they only take effect on recreate. Its own CI
# does the same thing when it installs a bundle, so a restart here is nothing unusual for the bot.
if any(k in updates for k in ("DISCORD_BOT_TOKEN", "DISCORD_GUILD_ID")):
run(compose_cmd() + ["up", "-d", "discord-bot"]); print(" discord-bot recreated with the new settings")
# MediaMTX reads LIVE_WEBHOOK_URL from ITS environment too, but recreating MediaMTX cuts whatever is
# recording right now. That is never worth doing automatically, so it is left to a human and a quiet moment.
if "LIVE_WEBHOOK_URL" in updates:
print(" NOTE: MediaMTX still has the old value. It only reads this on start, and a restart cuts a")
print(" live recording, so run this yourself between streams: docker compose up -d mediamtx")
def write_alerts_env(env=None):
"""/etc/vanguard/alerts.env — read by vanguard-alerts.service and vanguard-deadman.service."""
env = env or read_env()
keys = ("ALERT_INDEX", "ALERT_RETENTION_DAYS", "ALERT_SUPPRESS_S", "HEARTBEAT_MAX_S",
"ALERT_MENTION", "ALERT_MENTION_LEVELS") # what the two scripts read
Path("alerts.env").write_text("".join(f"{k}={env[k]}\n" for k in keys if env.get(k)))
os.chmod("alerts.env", 0o644)
run(["sudo", "install", "-m", "0644", "-o", "root", "-g", "root", "alerts.env", "/etc/vanguard/alerts.env"])
Path("alerts.env").unlink(missing_ok=True)
print(" wrote /etc/vanguard/alerts.env")
def read_env():
env = {}
try:
@@ -408,10 +520,12 @@ def write_env(watch_host, discord_token=None, discord_guild=None, git_host="", b
env.setdefault("ALERT_INDEX", "vanguard-alerts") # the alert bus: Kibana writes, vanguard-alerts.sh delivers
env.setdefault("LOG_RETENTION_DAYS", "30") # ILM delete phase for filebeat-*/metricbeat-*
env.setdefault("ALERT_RETENTION_DAYS", "90") # how long delivered alerts stay queryable
env.setdefault("ALERT_MENTION", "") # role:<id>/user:<id> list an alarm pings
env.setdefault("ALERT_MENTION_LEVELS", "error") # only errors ping; warnings arrive quietly
env.setdefault("DISCORD_BOT_TOKEN", "") # fill in: Discord dev portal -> the Authentik app -> Bot
env.setdefault("DISCORD_GUILD_ID", "")
env["WATCH_HOST"] = watch_host or "watch.example.com"
Path(".env").write_text("".join(f"{k}={v}\n" for k, v in env.items()))
Path(".env").write_text(render_env(env))
os.chmod(".env", 0o600)
def docker_gateway():
@@ -573,13 +687,13 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis
run(["sudo", "install", "-m", "0600", "stream-token.secret", "/etc/vanguard/stream-token.secret"])
run(["sudo", "install", "-m", "0644", "syncthing/stignore", "/etc/vanguard/stignore"]) # dumpd copies it into new clips folders
e = read_env()
Path("deploy.env").write_text("".join(f"{k}={e.get(k, '')}\n" for k in ("DEPLOY_AUTO", "DEPLOY_WEBHOOK_SECRET", "DEPLOY_BRANCH", "DEPLOY_NOTIFY_WEBHOOK_URL", "BOT_DEPLOY_AUTO", "BOT_DEPLOY_SECRET", "BOT_ENV_KEYS")) + f"DEPLOY_REPO_DIR={Path.cwd()}\n"); os.chmod("deploy.env", 0o600)
write_deploy_env(e) # one copy of the key list, shared with --set
Path("alert-webhook").write_text((e.get("ALERT_WEBHOOK_URL") or "") + "\n"); os.chmod("alert-webhook", 0o600)
# URL is a credential for that channel, so it is not world-readable.
# 0600 root: the only readers are root timers (vanguard-deadman.sh, vanguard-alerts.sh). It used to be
# group-readable by nobody(65534) for the Alertmanager container, which no longer exists.
run(["sudo", "install", "-m", "0600", "-o", "root", "-g", "root", "alert-webhook", "/etc/vanguard/alert-webhook"])
run(["sudo", "install", "-m", "0600", "deploy.env", "/etc/vanguard/deploy.env"]) # read by dumpd (/deploy webhook) and deploy.sh
write_alerts_env(e) # the timers' own knobs; a systemd timer never reads .env
if all(Path(f).exists() for f in ("deploy.sh", "systemd/deploy-pending.service", "systemd/deploy-pending.timer")):
run(["sudo", "install", "-m", "0755", "deploy.sh", "/usr/local/bin/vanguard-deploy"])
run(["sudo", "install", "-m", "0644", "systemd/deploy-pending.service", "systemd/deploy-pending.timer", "/etc/systemd/system/"])
@@ -603,6 +717,8 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis
run(["sudo", "install", "-m", "0755"] + media + ["/usr/local/bin/"])
if Path("bin/vgmetrics.sh").exists():
run(["sudo", "install", "-m", "0644", "bin/vgmetrics.sh", "/usr/local/bin/"]) # sourced by the sweep and vgdeband
if Path("bin/vgnotify.sh").exists():
run(["sudo", "install", "-m", "0644", "bin/vgnotify.sh", "/usr/local/bin/"]) # sourced by the alerts timer and the deadman
if all(Path(f).exists() for f in ("bin/vanguard-deadman.sh", "systemd/vanguard-deadman.timer")):
run(["sudo", "install", "-m", "0755", "bin/vanguard-deadman.sh", "/usr/local/bin/"])
run(["sudo", "install", "-m", "0644", "systemd/vanguard-deadman.service", "systemd/vanguard-deadman.timer", "/etc/systemd/system/"])
@@ -752,11 +868,60 @@ def es_security(dc):
run(dc + ["restart", "kibana", "filebeat", "metricbeat"])
print(" elasticsearch: netrc written for the deadman; kibana + beats restarted")
def ak_shell(dc, script, label):
"""Run a short script inside authentik-server's Django shell and print only its own output.
`ak shell` is an interactive console: it reads stdin line by line and rejects indented blocks, so the
script goes in as a file and is executed by one unindented statement. Authentik also prints a wall of
JSON boot logs to the same stream, which is why the output is filtered.
"""
if DRY:
print(f" (dry) authentik: {label}"); return
path = Path("ak-fixup.py")
path.write_text(script)
try:
if subprocess.call(dc + ["cp", str(path), "authentik-server:/tmp/ak-fixup.py"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) != 0:
print(f" authentik: cannot reach the container — {label} skipped"); return
result = subprocess.run(dc + ["exec", "-T", "authentik-server", "ak", "shell",
"-c", "exec(open('/tmp/ak-fixup.py').read())"],
capture_output=True, text=True)
for line in (result.stdout + result.stderr).splitlines():
if line.startswith(("{", "#", ">>>", "...")) or not line.strip():
continue
print(" ", line.rstrip())
except Exception as error:
print(f" authentik: {label} skipped ({error})")
finally:
path.unlink(missing_ok=True)
# Proxy-provider state the Authentik UI can quietly put back. Enforced on every apply so a save in the UI
# cannot leave logout half-working for weeks without anyone noticing.
AK_LOGOUT_FIXUP = """
from authentik.providers.proxy.models import ProxyProvider
from authentik.flows.models import Flow
target = Flow.objects.filter(slug="default-invalidation-flow").first()
if target is None:
print("authentik: no default-invalidation-flow found; logout flow left alone")
else:
for provider in ProxyProvider.objects.all():
before = getattr(provider.invalidation_flow, "slug", None)
if before == target.slug:
print(f"authentik: {provider.name!r} logout flow ok ({before})")
else:
provider.invalidation_flow = target
provider.save(update_fields=["invalidation_flow"])
print(f"authentik: {provider.name!r} logout flow {before} -> {target.slug}")
"""
def post_start_fixups(dc):
"""Idempotent first-run state that otherwise needs clicking: Syncthing GUI auth off, Gitea admin user, Jellyfin plugin."""
env = read_env()
print("== post-start fixups (idempotent)")
es_security(dc) # before anything else: Kibana and the Beats cannot authenticate until this runs
# /logout must end the whole Authentik session, not just the application's. The provider-scoped flow
# ("Logged out of application") leaves the SSO session alive, so logout looks like a page refresh.
ak_shell(dc, AK_LOGOUT_FIXUP, "proxy provider logout flow")
# Syncthing: no GUI login (the Authentik gate is the login); needs insecureAdminAccess since Caddy is not "localhost" to it
cfg = Path("/srv/media/.syncthing/config.xml")
if not DRY and cfg.exists():
@@ -835,8 +1000,16 @@ def show(domain, admin_pw, streamers, editors, auth_host="", watch_host="", git_
print(f"== authentik: https://{auth_host}/ akadmin password = AUTHENTIK_BOOTSTRAP_PASSWORD in .env (set on first start; change it in the UI)" if auth_host else "== authentik: (not set — run --full)")
print(f"== jellyfin : https://{watch_host}/" if watch_host else "== jellyfin : (not set — run --full)")
env = read_env()
tok = env.get("DISCORD_BOT_TOKEN", "")
print(f"== discord : guild {env.get('DISCORD_GUILD_ID') or '(not set)'}; bot token {'set (' + tok[:6] + '…)' if tok else '(not set)'}")
print("== discord : (./configure.py --set KEY=VALUE)")
for key, why in DISCORD_KEYS:
value = env.get(key, "")
if not value:
shown = "(not set)"
elif key == "DISCORD_GUILD_ID":
shown = value
else: # tokens and webhook URLs are credentials: never print them whole
shown = value[:34] + "…" if len(value) > 34 else value[:8] + "…"
print(f" {key:26} {shown:40} {why}")
print(f"== mariadb : {domain or 'HOST'}:3306 (public, root local-only) root pass {env.get('MARIADB_ROOT_PASSWORD','(run apply)')}")
print(f" bot ci : Gitea -> VanguardBot -> Settings -> Actions -> Secrets: BOT_DEPLOY_SECRET={env.get('BOT_DEPLOY_SECRET','(run apply)')}")
print(f" pushes to main install themselves (BOT_DEPLOY_AUTO={env.get('BOT_DEPLOY_AUTO','1')}); by hand: ./deployBotBundle.sh <bundle>")
@@ -918,7 +1091,7 @@ def full(domain, admin_pw, streamers, editors, auth_host="", watch_host="", git_
Path(".env").write_text(f"APEX_HOST={apex}\nAPEX_REDIRECT={apex_to}\n"); os.chmod(".env", 0o600)
if alert_webhook != env0.get("ALERT_WEBHOOK_URL", ""):
env = read_env(); env["ALERT_WEBHOOK_URL"] = alert_webhook
Path(".env").write_text("".join(f"{k}={v}\n" for k, v in env.items())); os.chmod(".env", 0o600)
Path(".env").write_text(render_env(env)); os.chmod(".env", 0o600)
return domain, admin_pw, streamers, editors, auth_host, watch_host, discord_token, discord_guild, git_host, bot_host, live_host, live_webhook, files_host, sync_host, clips_host, diag_host
def ask_discord():
@@ -951,6 +1124,15 @@ def main():
streamers, accounts = read_streamers(), read_accounts()
admin_pw = accounts.pop("admin", "")
editors = accounts
if mode == "--set":
pairs = [a for a in sys.argv[2:] if "=" in a]
if not pairs:
print("usage: configure.py --set KEY=VALUE [KEY=VALUE ...]\n\nSettable on their own:")
for name, why in SETTABLE.items():
print(f" {name:22} {why}")
print('\nexample: ./configure.py --set ALERT_WEBHOOK_URL=https://discord.com/api/webhooks/…')
sys.exit(1)
set_env(dict(p.split("=", 1) for p in pairs)); return
if mode == "--es-security":
# Just the Elasticsearch user bootstrap. Worth having on its own: if it is skipped or fails during an
# --apply, Kibana reports "unavailable", Metricbeat exits on Kibana's 503 and Filebeat sits up but