Per-host group gating in Caddy from X-Authentik-Groups; Discord roles -> Authentik groups policy; domain-level forward auth documented

BOT_REQUIRE_GROUP / LIVE_REQUIRE_GROUP in .env drive a header_regexp matcher after forward_auth
(403 before the app). authentik/discord-roles-to-groups.py syncs Discord roles into groups at
every login so access follows Discord roles without manual edits.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

.env.example

@@ -16,3 +16,5 @@ BOT_HOST=bot.example.com
 LIVE_HOST=live.example.com
 MAIN_HOST=media.example.com
 LIVE_WEBHOOK_URL=
+BOT_REQUIRE_GROUP=bot-users
+LIVE_REQUIRE_GROUP=

CommunityDiscordMedia.md

@@ -228,7 +228,8 @@ Checklist:
 - `bot/AGENT.md` is the build brief for the agent writing the bot; `bot/README.md` the ops cheat-sheet. The clip-submissions feature (Option below) is a natural home
   for this bot: write `clips/<user>/<name>.mp4` + `<name>.json` sidecar and the Jellyfin plugin picks it up.
 - [ ] CNAME `bot` → main host; `configure.py --full` (asks the hostname, bot token + guild) → `docker compose up -d --build`
-- [ ] Authentik: Proxy Provider (forward auth, single app, external host `https://bot.HOST`) → embedded outpost → group policy
+- [ ] Authentik: domain-level Proxy Provider (covers live/bot/main); Caddy requires group `bot-users` on the bot host (`BOT_REQUIRE_GROUP`)
+- [ ] Discord roles → Authentik groups: `authentik/discord-roles-to-groups.py` on both source flows (scope `guilds.members.read`)
 - [ ] dev drops `main.py` into `bot/app/` → `docker compose restart discord-bot`
 
 ### Phase 9 — Gitea + Actions runner

README.md

@@ -208,22 +208,25 @@ the 500 Mbps line. For a crowd, restream to Twitch from OBS.
 
 ## 5g. Authentik forward-auth for the Caddy-gated hosts (`live.`, `bot.`; later the main host)
 
-Caddy's `forward_auth` asks Authentik's **embedded outpost** (127.0.0.1:9000) on every request. The
+Design: **Authentik authenticates once for the whole domain; Caddy authorizes per host** from the
-generated Caddy blocks work with either Authentik mode below; nothing in Caddy changes.
+`X-Authentik-Groups` header the outpost returns; **Discord roles fill the Authentik groups** at login.
 
-**Recommended: one domain-level provider for all hosts** (one login, one cookie, new hosts need no Authentik work):
 1. Applications → Outposts → edit "authentik Embedded Outpost" → config `authentik_host: https://auth.HOST`.
 2. Applications → Create with provider → Application `Vanguard sites`; **Proxy Provider**, authorization flow
    implicit-consent, mode **Forward auth (domain level)**, authentication URL `https://auth.HOST`,
-   cookie domain `HOST's parent domain` (e.g. `dumpstervigilantes.rip`).
+   cookie domain = the parent domain (e.g. `dumpstervigilantes.rip`). Edit the Embedded Outpost →
-3. Edit the Embedded Outpost → Applications → add `Vanguard sites` → Update.
+   Applications → add it → Update. One login now covers every gated host.
-4. Test in a private window: `https://live.HOST/` → Discord login → portal; `https://bot.HOST/` opens with no
+3. Per-host group requirement = `.env`: `BOT_REQUIRE_GROUP=bot-users` (default), `LIVE_REQUIRE_GROUP=`
-   second login; `https://live.HOST/me/stream-key` shows your username.
+   (empty = any logged-in member). `./configure.py` apply regenerates the Caddy blocks; a user outside the
-
+   group gets a 403 from Caddy before the app sees the request.
-Trade-off: one access policy for every gated host (the guild policy at login already limits it to server
+4. Discord roles → groups: Directory → Groups: create `bot-users` (and keep `streamers`, `editors`, `admins`).
-members). A host that must be restricted to a group gets its own **single-application** Proxy Provider
+   Discord source → Scopes: `guilds guilds.members.read`. Customization → Policies → Expression Policy
-(external host `https://x.HOST`, added to the outpost) and enforces nothing else; the bot web UI also
+   `discord-roles-to-groups` from `authentik/discord-roles-to-groups.py` (fill in the server ID and the
-authorizes by the `X-Authentik-Groups` header itself.
+   role-id → group table). Bind it to `default-source-enrollment` **and** `default-source-authentication`
+   (engine mode `all`, after `discord-guild-member`). From then on giving someone a Discord role gives them
+   the matching access on next login; removing it removes it.
+5. Test in a private window: `https://live.HOST/` → Discord login → portal; `https://bot.HOST/` → 403 unless
+   you hold the bot role; `https://live.HOST/me/stream-key` shows your username.
 
 Jellyfin and Gitea use OIDC providers instead, not this.
 

STATUS.md

@@ -58,7 +58,7 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea
 | Gitea | git hosting on MariaDB, Actions enabled; holds this repo | live, local admin |
 | act_runner | CI jobs in Docker | **not registered** |
 | discord-bot | Python runtime, idles until `bot/app/main.py` exists | live, no code yet |
-| `bot.` host | bot web UI behind Authentik forward-auth | Caddy block ready; **CNAME + Authentik proxy provider pending** |
+| `bot.` host | bot web UI behind Authentik forward-auth; Caddy requires group `bot-users` (from a Discord role) | Caddy block ready; **CNAME + Authentik domain-level provider + roles→groups policy pending** |
 | Live portal (`live/`, `live.` host) | React + dockview multi-stream viewer + browser Go Live (WHIP); MediaMTX HLS/WebRTC/API re-enabled on loopback; dumpd `/me/stream-key` | **built; CNAME, Authentik proxy provider, `./deployLive.sh`, configure apply pending** |
 
 ## Unfinished, in rough priority

authentik/discord-roles-to-groups.py

@@ -0,0 +1,54 @@
+# Authentik expression policy: sync a member's Discord ROLES into Authentik GROUPS at every login.
+# Paste into: Customization -> Policies -> Create -> Expression Policy  (name: discord-roles-to-groups)
+# Bind to BOTH flows (after discord-guild-member): default-source-enrollment and default-source-authentication.
+# Requires the "guilds.members.read" scope on the Discord source (source -> Protocol settings -> Scopes:
+#   guilds guilds.members.read ). Existing users re-consent once at Discord.
+# The Authentik groups must already exist (Directory -> Groups). Role IDs: Discord Developer Mode ->
+#   Server Settings -> Roles -> right-click a role -> Copy Role ID.
+
+GUILD_ID = "000000000000000000"                         # <- your Discord server ID
+ROLE_TO_GROUP = {                                       # <- Discord role id : Authentik group name
+    "111111111111111111": "bot-users",
+    "222222222222222222": "streamers",
+    "333333333333333333": "editors",
+    "444444444444444444": "admins",
+}
+
+from authentik.core.models import Group
+
+source = context.get("source")
+if not source or getattr(source, "provider_type", "") != "discord":
+    return True                                          # not a Discord login (e.g. akadmin): nothing to sync
+
+connection = context.get("goauthentik.io/sources/connection")
+if not connection:
+    return True
+
+resp = requests.get(
+    f"https://discord.com/api/users/@me/guilds/{GUILD_ID}/member",
+    headers={"Authorization": f"Bearer {connection.access_token}"},
+    timeout=10,
+)
+if resp.status_code != 200:
+    ak_message(f"Could not read your Discord roles ({resp.status_code}); groups unchanged.")
+    return True                                          # don't block login on a Discord hiccup
+
+roles = set(resp.json().get("roles", []))
+wanted = {g for r, g in ROLE_TO_GROUP.items() if r in roles}
+managed = set(ROLE_TO_GROUP.values())
+
+# Authentication flow: the user exists -> edit memberships directly.
+user = context.get("pending_user") or (request.user if request.user.is_authenticated else None)
+if user and user.pk:
+    for group in Group.objects.filter(name__in=managed):
+        if group.name in wanted:
+            group.users.add(user)
+        else:
+            group.users.remove(user)
+    return True
+
+# Enrollment flow: the user is created later by the user-write stage; hand it the groups to attach.
+plan = context.get("flow_plan")
+if plan is not None:
+    plan.context["groups"] = list(Group.objects.filter(name__in=wanted))
+return True

configure.py

@@ -124,8 +124,9 @@ def ask_name(kind, taken):
         return n
 
 # ------------------------------------------------------------------ writers
-AUTHENTIK_FORWARD_AUTH = """	# Authentik forward-auth (embedded outpost). Needs a Proxy Provider (forward auth, single application)
+AUTHENTIK_FORWARD_AUTH = """	# Authentik forward-auth (embedded outpost). Needs a Proxy Provider in Authentik — domain-level for the
-	# for this external host in Authentik, assigned to the embedded outpost. Until then: 4xx, never open.
+	# whole domain (recommended) or single-application for this host — assigned to the embedded outpost.
+	# Until then: 4xx, never open. Per-host group requirements are enforced below from X-Authentik-Groups.
 	reverse_proxy /outpost.goauthentik.io/* 127.0.0.1:9000
 	forward_auth 127.0.0.1:9000 {
 		uri /outpost.goauthentik.io/auth/caddy
@@ -134,14 +135,27 @@ AUTHENTIK_FORWARD_AUTH = """	# Authentik forward-auth (embedded outpost). Needs
 	}
 """
 
+def gate(required_group=""):
+    """Caddy lines: Authentik forward-auth, then deny unless X-Authentik-Groups contains required_group."""
+    block = AUTHENTIK_FORWARD_AUTH
+    if required_group:
+        g = re.escape(required_group)
+        block += f"""	@lacks_group not header_regexp X-Authentik-Groups (^|\\|){g}(\\||$)
+	handle @lacks_group {{
+		respond "This page is for members of the '{required_group}' group." 403
+	}}
+"""
+    return block
+
 def write_caddy(domain, auth_host="", watch_host="", git_host="", bot_host="", live_host=""):
+    env = read_env()
     extra = ""
     if live_host:
         extra += f"""
 # vanguard:live
 {live_host} {{
 	encode zstd gzip
-{AUTHENTIK_FORWARD_AUTH}	handle /me/* {{
+{gate(env.get("LIVE_REQUIRE_GROUP", ""))}	handle /me/* {{
 		reverse_proxy 127.0.0.1:8080          # dumpd: /me/stream-key for the logged-in streamer (Go Live)
 	}}
 	handle_path /api/* {{
@@ -165,7 +179,7 @@ def write_caddy(domain, auth_host="", watch_host="", git_host="", bot_host="", l
 # vanguard:bot
 {bot_host} {{
 	encode zstd gzip
-{AUTHENTIK_FORWARD_AUTH}	reverse_proxy 127.0.0.1:8000
+{gate(env.get("BOT_REQUIRE_GROUP", "bot-users"))}	reverse_proxy 127.0.0.1:8000
 }}
 """
     if git_host:
@@ -230,6 +244,8 @@ def write_env(watch_host, discord_token=None, discord_guild=None, git_host="", b
     if main_host: env["MAIN_HOST"] = main_host
     if live_webhook is not None: env["LIVE_WEBHOOK_URL"] = live_webhook
     env.setdefault("LIVE_WEBHOOK_URL", "")
+    env.setdefault("BOT_REQUIRE_GROUP", "bot-users")   # Caddy 403s anyone not in this Authentik group; empty = any logged-in member
+    env.setdefault("LIVE_REQUIRE_GROUP", "")
     env["GIT_HOST"] = git_host or env.get("GIT_HOST", "git.example.com")
     env["BOT_HOST"] = bot_host or env.get("BOT_HOST", "bot.example.com")
     env.setdefault("GITEA_DB_PASSWORD", "".join(c for c in secrets.token_urlsafe(40) if c.isalnum())[:32])
@@ -383,8 +399,8 @@ def run(cmd, **kw):
 
 def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", discord_token=None, discord_guild=None, git_host="", bot_host="", live_host="", live_webhook=None):
     print("\n== writing configs")
-    write_caddy(domain, auth_host, watch_host, git_host, bot_host, live_host); write_mediamtx(streamers); write_copyparty(admin_pw, editors); write_dump_users(admin_pw, streamers); write_streamers_json(streamers)
     write_env(watch_host, discord_token, discord_guild, git_host, bot_host, live_host, domain, live_webhook)
+    write_caddy(domain, auth_host, watch_host, git_host, bot_host, live_host); write_mediamtx(streamers); write_copyparty(admin_pw, editors); write_dump_users(admin_pw, streamers); write_streamers_json(streamers)
     run(["sudo", "install", "-d", "-o", os.environ.get("SUDO_USER") or os.environ.get("USER") or "debian", "/srv/live"])
     print("== app data folders")
     run(["sudo", "install", "-d", "-o", "1000", "-g", "1000", "/srv/appdata/jellyfin/config", "/srv/appdata/jellyfin/cache",
@@ -409,7 +425,7 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis
 def show(domain, admin_pw, streamers, editors, auth_host="", watch_host="", git_host="", bot_host="", live_host=""):
     print(f"\n== domain   : {domain or '(not set)'}")
     print(f"== live     : https://{live_host}/  (behind Authentik forward-auth; deploy live/dist -> /srv/live; WebRTC needs 8189 open)" if live_host else "== live     : (not set — run --full)")
-    print(f"== bot web  : https://{bot_host}/  (behind Authentik forward-auth; create the Proxy Provider for it)" if bot_host else "== bot web  : (not set — run --full)")
+    print(f"== bot web  : https://{bot_host}/  (Authentik forward-auth; Caddy requires group '{read_env().get('BOT_REQUIRE_GROUP','bot-users') or 'any'}')" if bot_host else "== bot web  : (not set — run --full)")
     print(f"== gitea    : https://{git_host}/  ssh port 2222  (runner: ./registerGiteaRunner.sh)" if git_host else "== gitea    : (not set — run --full)")
     print(f"== authentik: https://{auth_host}/   (first run: https://{auth_host}/if/flow/initial-setup/)" if auth_host else "== authentik: (not set — run --full)")
     print(f"== jellyfin : https://{watch_host}/" if watch_host else "== jellyfin : (not set — run --full)")