Push-to-deploy (OFF by default): vanguard-deploy applies only changed parts with a live-stream guard for MediaMTX, rollback, pending timer; dumpd /deploy webhook gated by DEPLOY_AUTO; configure --apply; switchToClone.sh; draft CI workflow; committed plugin zip

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-11-01 01:08:43 -05:00
parent 07ffae0746
commit 66f05cf9a7
15 changed files with 295 additions and 3 deletions

View File

@@ -0,0 +1,18 @@
# DRAFT — inert while it lives in .gitea/draft-workflows/. Activate with: git mv .gitea/draft-workflows/ci.yml .gitea/workflows/ci.yml
# Checks every push; builds + tests the Jellyfin plugin. Deploying is NOT done here (the runner must not touch
# the host): the Gitea push webhook -> dumpd /deploy -> vanguard-deploy handles that, when DEPLOY_AUTO=1.
name: ci
on: [push, pull_request]
jobs:
checks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: sudo apt-get update -q && sudo apt-get install -y -q python3 python3-yaml nodejs npm ffmpeg jq
- run: bash check.sh
plugin:
runs-on: dotnet
steps:
- uses: actions/checkout@v4
- run: cd jellyfin-plugin-vanguard && dotnet test Jellyfin.Plugin.Vanguard.Tests --nologo -v q
- run: cd jellyfin-plugin-vanguard && dotnet build Jellyfin.Plugin.Vanguard -c Release --nologo -v q

1
.gitignore vendored
View File

@@ -9,3 +9,4 @@ mediamtx.yml
copyparty.conf copyparty.conf
Caddyfile Caddyfile
stream-token.secret stream-token.secret
deploy.env

View File

@@ -318,6 +318,17 @@ permanent reference is needed for long sessions, not just a start burst.
export at 2560×1440) export at 2560×1440)
- rejected: LTC audio (SRT multitrack + per-PC generator); clock inside the picture; cropping at remux (re-encode) - rejected: LTC audio (SRT multitrack + per-PC generator); clock inside the picture; cropping at remux (re-encode)
### Phase 13 — push-to-deploy (built; OFF — `DEPLOY_AUTO=0`, CI workflow in draft-workflows/)
- server folder = git clone (`switchToClone.sh`); generated files gitignored and preserved
- `vanguard-deploy pull|apply|rollback|pending|status` applies only the changed parts; `check.sh` gates it;
MediaMTX restarts are deferred while anything is live (`deploy-pending.timer`)
- Gitea push webhook → dumpd `/deploy` (bridge port, HMAC) → `vanguard-deploy pull` when `DEPLOY_AUTO=1`
- CI (draft): `check.sh` + plugin tests/build on the runner; the runner never touches the host
- [ ] run `switchToClone.sh key` / add deploy key / `switchToClone.sh clone` on the server
- [ ] `./configure.py --apply` once (installs vanguard-deploy, deploy.env, the pending timer)
- [ ] add the Gitea webhook (secret from `.env`); watch `journalctl -u dumpd | grep deploy` for a few pushes
- [ ] when trusted: `DEPLOY_AUTO=1` in `.env` + `./configure.py --apply`; move `ci.yml` into `.gitea/workflows/`
### Option — more clip sources (Medal, Twitch/YouTube/Kick clips, Discord uploads) ### Option — more clip sources (Medal, Twitch/YouTube/Kick clips, Discord uploads)
Today the only clip intake is a local clipper folder synced by Syncthing. Members without that Today the only clip intake is a local clipper folder synced by Syncthing. Members without that

View File

@@ -396,6 +396,37 @@ files are not revisited. Clips are always converted to CFR regardless.
Planned alternative: re-encode flagged streams off-site on a machine with a hardware encoder (home box NVENC or a Planned alternative: re-encode flagged streams off-site on a machine with a hardware encoder (home box NVENC or a
Spot GPU) and push the result back — see CommunityDiscordMedia.md → Later. Spot GPU) and push the result back — see CommunityDiscordMedia.md → Later.
## Deployment (push to deploy — built, OFF by default)
The server folder is a git clone of this repo (`./switchToClone.sh key` → add the printed deploy key in Gitea →
`./switchToClone.sh clone`). Generated files (`.env`, `Caddyfile`, `mediamtx.yml`, `copyparty.conf`, the token
secret) are gitignored and survive pulls; static pages are bind-mounted from the checkout.
`sudo vanguard-deploy pull` fetches `main` and applies **only what changed**, after `check.sh`:
| changed | action | interruption |
|---|---|---|
| `configure.py`, `dock/`, `clips/`, `about/`, stignore | `configure.py --apply` → Caddy reload, copyparty restart | none / a second |
| `bin/*`, `systemd/*` | install; dumpd restart when it changed; timers pick up next run | a second |
| `firewall/nftables.conf` | `nft -f` | none |
| `docker-compose.yml`, bot image | `compose up -d --build` for changed services | that service |
| anything that changes `mediamtx.yml` or the mediamtx service | **restart only when no stream is live**; otherwise deferred, `deploy-pending.timer` retries every 5 min | none while live |
| `jellyfin-plugin-vanguard/dist/*.zip` (committed) | `installJellyfinPlugin.sh` | Jellyfin restart |
| `live/` | `npm run build` on the server → `/srv/live` | none |
`sudo vanguard-deploy rollback` checks out the previously deployed revision and applies the reverse diff.
`sudo vanguard-deploy status` shows revisions, auto mode, pending restarts, live streams.
**Automatic**: Gitea → repo → Settings → Webhooks → Gitea, URL `http://<docker gateway>:8081/deploy`
(`docker network inspect vanguard-stream_default -f '{{(index .IPAM.Config 0).Gateway}}'`, usually `172.18.0.1`),
secret = `DEPLOY_WEBHOOK_SECRET` from `.env`, push events, branch `main`. dumpd verifies the HMAC and runs
`vanguard-deploy pull` **only when `DEPLOY_AUTO=1`** in `.env` (then `./configure.py --apply` to push the setting
to `/etc/vanguard/deploy.env`). Until then every push is logged (`journalctl -u dumpd | grep deploy`) and ignored.
Optional `DEPLOY_NOTIFY_WEBHOOK_URL` posts deploy results to Discord.
CI: `.gitea/draft-workflows/ci.yml` (inert until moved to `.gitea/workflows/`) runs `check.sh` and the plugin
tests/build on the Actions runner. The runner never deploys — it must not touch the host.
## Layout on the server ## Layout on the server
/srv/media/streams/<user>/<Title>/.live/<ts>.mp4 recording in progress (hidden from Jellyfin/copyparty) /srv/media/streams/<user>/<Title>/.live/<ts>.mp4 recording in progress (hidden from Jellyfin/copyparty)

View File

@@ -18,6 +18,8 @@ Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea
| Every 5 min | Each user's clips folder is measured against their quota (100 GB default, per-user overrides); over quota → their Syncthing folder is paused, under 90 % → resumed; the clips page shows usage | dumpd quota loop — **pending deploy** | | Every 5 min | Each user's clips folder is measured against their quota (100 GB default, per-user overrides); over quota → their Syncthing folder is paused, under 90 % → resumed; the clips page shows usage | dumpd quota loop — **pending deploy** |
| Daily 05:00 ET | Dumped nights older than 3 days deleted for good | trash purge (systemd timer) | | Daily 05:00 ET | Dumped nights older than 3 days deleted for good | trash purge (systemd timer) |
| On boot | Everything restarts itself after the media mount is up; interrupted recordings stay playable | systemd + Docker | | On boot | Everything restarts itself after the media mount is up; interrupted recordings stay playable | systemd + Docker |
| Every 5 min | If a deploy deferred a MediaMTX restart, retry when no stream is live | `deploy-pending.timer` — **pending first apply** |
| On push to `main` | (OFF: `DEPLOY_AUTO=0`) Gitea webhook → dumpd → `vanguard-deploy pull` applies only what changed | **built, disabled** |
| Continuous | Active streams are watchable live (WebRTC, HLS fallback) by Discord-server members; optional "is live" Discord post when a stream starts | MediaMTX + live portal (page deployed; MediaMTX port fix pending apply) | | Continuous | Active streams are watchable live (WebRTC, HLS fallback) by Discord-server members; optional "is live" Discord post when a stream starts | MediaMTX + live portal (page deployed; MediaMTX port fix pending apply) |
| **Not yet running** | Pull of files older than 15 days to the home HDD | `home/rclone-archive.sh`, needs the home box | | **Not yet running** | Pull of files older than 15 days to the home HDD | `home/rclone-archive.sh`, needs the home box |
| **Not yet running** | Nightly backup of the Authentik database (Postgres) | planned | | **Not yet running** | Nightly backup of the Authentik database (Postgres) | planned |

View File

@@ -9,6 +9,9 @@ GET /me/stream-token -> short-lived publish token for the caller (member of str
GET /me/clips -> Syncthing self-service status for the caller: our device id, their folder/device state GET /me/clips -> Syncthing self-service status for the caller: our device id, their folder/device state
POST /me/clips -> {"device_id": "..."}: create clips/<user> (+ .stignore), add their device to our Syncthing, POST /me/clips -> {"device_id": "..."}: create clips/<user> (+ .stignore), add their device to our Syncthing,
create the receive-only folder clips-<user> shared with them (min free 300 GB). Idempotent. create the receive-only folder clips-<user> shared with them (min free 300 GB). Idempotent.
POST /deploy -> Gitea push webhook (bridge port only, HMAC-verified with DEPLOY_WEBHOOK_SECRET from
/etc/vanguard/deploy.env). Runs /usr/local/bin/vanguard-deploy pull ONLY when DEPLOY_AUTO=1;
otherwise logs the push and answers 202 "auto-deploy disabled".
POST /mtx-auth -> MediaMTX authHTTP hook: allows read/playback; allows publish iff user/pass carries a valid POST /mtx-auth -> MediaMTX authHTTP hook: allows read/playback; allows publish iff user/pass carries a valid
token whose user owns the path prefix. Tokens are HMAC-signed with /etc/vanguard/stream-token.secret. token whose user owns the path prefix. Tokens are HMAC-signed with /etc/vanguard/stream-token.secret.
@@ -244,6 +247,34 @@ def clips_setup(user, device_id, device_name=None):
print(f"{datetime.now(TZ):%F %T} clips setup user={user} device={device_id[:7]}… folder={fid}", flush=True) print(f"{datetime.now(TZ):%F %T} clips setup user={user} device={device_id[:7]}… folder={fid}", flush=True)
return clips_status(user) return clips_status(user)
DEPLOY_ENV = Path(os.environ.get("DEPLOY_ENV_FILE", "/etc/vanguard/deploy.env"))
def deploy_env():
d = {}
try:
for line in DEPLOY_ENV.read_text().splitlines():
if "=" in line and not line.startswith("#"): k, v = line.split("=", 1); d[k.strip()] = v.strip()
except OSError:
pass
return d
_deploy_lock = {"running": False}
def run_deploy(reason):
import subprocess, threading
if _deploy_lock["running"]: return False
_deploy_lock["running"] = True
def go():
try:
print(f"{datetime.now(TZ):%F %T} deploy: start ({reason})", flush=True)
r = subprocess.run(["/usr/local/bin/vanguard-deploy", "pull"], capture_output=True, text=True, timeout=1800)
for line in (r.stdout + r.stderr).splitlines()[-40:]: print(f"deploy| {line}", flush=True)
print(f"{datetime.now(TZ):%F %T} deploy: exit {r.returncode}", flush=True)
except Exception as e:
print(f"{datetime.now(TZ):%F %T} deploy: error {e}", flush=True)
finally:
_deploy_lock["running"] = False
threading.Thread(target=go, daemon=True).start()
return True
def token_secret(): def token_secret():
try: try:
sec = TOKEN_SECRET_FILE.read_bytes().strip() sec = TOKEN_SECRET_FILE.read_bytes().strip()
@@ -354,6 +385,25 @@ class H(BaseHTTPRequestHandler):
"files": [str(f.relative_to(ROOT)) for f in files], "files": [str(f.relative_to(ROOT)) for f in files],
"gb": round(sum(f.stat().st_size for f in files) / 1e9, 2)}) "gb": round(sum(f.stat().st_size for f in files) / 1e9, 2)})
def do_POST(self): def do_POST(self):
if self.server.server_port == AUTH_LISTEN[1] and self.path.startswith("/deploy"):
# Gitea (container) -> host gateway -> here. Verify HMAC-SHA256 of the raw body with the webhook secret.
n = int(self.headers.get("Content-Length") or 0); raw = self.rfile.read(n)
env = deploy_env(); secret = env.get("DEPLOY_WEBHOOK_SECRET", "")
sig = self.headers.get("X-Gitea-Signature") or self.headers.get("X-Hub-Signature-256", "").replace("sha256=", "")
if not secret or not sig or not hmac.compare_digest(hmac.new(secret.encode(), raw, hashlib.sha256).hexdigest(), sig):
print(f"{datetime.now(TZ):%F %T} deploy webhook: bad signature from {self.client_address[0]}", flush=True)
return self._json(401, {"error": "bad signature"})
try: payload = json.loads(raw or b"{}")
except json.JSONDecodeError: return self._json(400, {"error": "bad json"})
ref = payload.get("ref", ""); branch = env.get("DEPLOY_BRANCH", "main")
after = (payload.get("after") or "")[:10]; pusher = (payload.get("pusher") or {}).get("username", "?")
if ref != f"refs/heads/{branch}":
return self._json(200, {"ignored": f"ref {ref} is not {branch}"})
if env.get("DEPLOY_AUTO", "0") != "1":
print(f"{datetime.now(TZ):%F %T} deploy webhook: push {after} by {pusher} on {branch} — auto-deploy disabled (DEPLOY_AUTO=0)", flush=True)
return self._json(202, {"queued": False, "note": "auto-deploy disabled (DEPLOY_AUTO=0); run vanguard-deploy pull by hand"})
started = run_deploy(f"push {after} by {pusher}")
return self._json(202, {"queued": started, "note": "deploy started" if started else "a deploy is already running"})
if self.server.server_port == AUTH_LISTEN[1] and not self.path.startswith("/mtx-auth"): if self.server.server_port == AUTH_LISTEN[1] and not self.path.startswith("/mtx-auth"):
return self._json(404, {"error": "not found"}) return self._json(404, {"error": "not found"})
if self.path.startswith("/mtx-auth"): if self.path.startswith("/mtx-auth"):

View File

@@ -5,7 +5,7 @@ cd "$(dirname "$0")"
fail=0 fail=0
say() { printf '%-28s %s\n' "$1" "$2"; } say() { printf '%-28s %s\n' "$1" "$2"; }
for f in install.sh addCommunityMediaServer.sh installJellyfinPlugin.sh registerGiteaRunner.sh createGiteaDb.sh deployLive.sh addClipUser.sh firewall/firewall.sh bin/*.sh home/*.sh bot/entrypoint.sh mariadb/init/*.sh; do for f in install.sh addCommunityMediaServer.sh installJellyfinPlugin.sh registerGiteaRunner.sh createGiteaDb.sh deployLive.sh addClipUser.sh deploy.sh switchToClone.sh firewall/firewall.sh bin/*.sh home/*.sh bot/entrypoint.sh mariadb/init/*.sh; do
[[ -f $f ]] || continue [[ -f $f ]] || continue
bash -n "$f" && say "$f" ok || { say "$f" "BASH SYNTAX"; fail=1; } bash -n "$f" && say "$f" ok || { say "$f" "BASH SYNTAX"; fail=1; }
done done

View File

@@ -4,6 +4,8 @@
./configure.py --full first-time setup: asks every question (current values as defaults) ./configure.py --full first-time setup: asks every question (current values as defaults)
./configure.py manage: skips what is already set; add/remove/reset streamers & editors ./configure.py manage: skips what is already set; add/remove/reset streamers & editors
./configure.py --show print current streamers, editors, domain and the OBS settings ./configure.py --show print current streamers, editors, domain and the OBS settings
./configure.py --apply regenerate everything from the values already on disk and apply (non-interactive;
used by deploy.sh). Fails if the main hostname was never set.
./configure.py --hosts AUTH WATCH [GIT [BOT [LIVE]]] set the Authentik/Jellyfin/Gitea/bot/live hostnames and apply (non-interactive; ./configure.py --hosts AUTH WATCH [GIT [BOT [LIVE]]] set the Authentik/Jellyfin/Gitea/bot/live hostnames and apply (non-interactive;
used by addCommunityMediaServer.sh) used by addCommunityMediaServer.sh)
@@ -316,6 +318,10 @@ def write_env(watch_host, discord_token=None, discord_guild=None, git_host="", b
env.setdefault("APEX_REDIRECT", "") # where the bare domain sends people, e.g. https://watch.HOST/ env.setdefault("APEX_REDIRECT", "") # where the bare domain sends people, e.g. https://watch.HOST/
env.setdefault("SYNC_REQUIRE_GROUP", "media-admins") # Caddy gate for /sync on the main host env.setdefault("SYNC_REQUIRE_GROUP", "media-admins") # Caddy gate for /sync on the main host
env.setdefault("STREAM_TOKEN_SECRET", secrets.token_urlsafe(48)) # signs publish tokens (dumpd) env.setdefault("STREAM_TOKEN_SECRET", secrets.token_urlsafe(48)) # signs publish tokens (dumpd)
env.setdefault("DEPLOY_AUTO", "0") # 1 = Gitea push webhook runs deploy.sh (off until trusted)
env.setdefault("DEPLOY_WEBHOOK_SECRET", secrets.token_urlsafe(32)) # Gitea webhook secret (HMAC)
env.setdefault("DEPLOY_BRANCH", "main")
env.setdefault("DEPLOY_NOTIFY_WEBHOOK_URL", "") # optional Discord webhook for deploy notices
env.setdefault("COPYPARTY_IDP_KEY", "".join(c for c in secrets.token_urlsafe(40) if c.isalnum())[:32]) # Caddy->copyparty idp-h-key env.setdefault("COPYPARTY_IDP_KEY", "".join(c for c in secrets.token_urlsafe(40) if c.isalnum())[:32]) # Caddy->copyparty idp-h-key
env.setdefault("BOT_REQUIRE_GROUP", "bot-users") # Caddy 403s anyone not in this Authentik group; empty = any logged-in member env.setdefault("BOT_REQUIRE_GROUP", "bot-users") # Caddy 403s anyone not in this Authentik group; empty = any logged-in member
env.setdefault("LIVE_REQUIRE_GROUP", "") env.setdefault("LIVE_REQUIRE_GROUP", "")
@@ -489,6 +495,12 @@ def apply(domain, admin_pw, streamers, editors, auth_host="", watch_host="", dis
run(["sudo", "install", "-d", "-m", "0750", "/etc/vanguard"]) run(["sudo", "install", "-d", "-m", "0750", "/etc/vanguard"])
run(["sudo", "install", "-m", "0600", "stream-token.secret", "/etc/vanguard/stream-token.secret"]) run(["sudo", "install", "-m", "0600", "stream-token.secret", "/etc/vanguard/stream-token.secret"])
run(["sudo", "install", "-m", "0644", "syncthing/stignore", "/etc/vanguard/stignore"]) # dumpd copies it into new clips folders run(["sudo", "install", "-m", "0644", "syncthing/stignore", "/etc/vanguard/stignore"]) # dumpd copies it into new clips folders
e = read_env()
Path("deploy.env").write_text("".join(f"{k}={e.get(k, '')}\n" for k in ("DEPLOY_AUTO", "DEPLOY_WEBHOOK_SECRET", "DEPLOY_BRANCH", "DEPLOY_NOTIFY_WEBHOOK_URL")) + f"DEPLOY_REPO_DIR={Path.cwd()}\n"); os.chmod("deploy.env", 0o600)
run(["sudo", "install", "-m", "0600", "deploy.env", "/etc/vanguard/deploy.env"]) # read by dumpd (/deploy webhook) and deploy.sh
run(["sudo", "install", "-m", "0755", "deploy.sh", "/usr/local/bin/vanguard-deploy"])
run(["sudo", "install", "-m", "0644", "systemd/deploy-pending.service", "systemd/deploy-pending.timer", "/etc/systemd/system/"])
run(["sudo", "systemctl", "daemon-reload"]); run(["sudo", "systemctl", "enable", "--now", "deploy-pending.timer"])
run(["sudo", "systemctl", "restart", "dumpd"]) run(["sudo", "systemctl", "restart", "dumpd"])
print("== containers") print("== containers")
dc = ["docker", "compose"] if (DRY or subprocess.call(["docker", "info"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) == 0) else ["sudo", "docker", "compose"] dc = ["docker", "compose"] if (DRY or subprocess.call(["docker", "info"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) == 0) else ["sudo", "docker", "compose"]
@@ -613,6 +625,10 @@ def main():
editors = accounts editors = accounts
if mode == "--show": if mode == "--show":
show(domain, admin_pw, streamers, editors, auth_host, watch_host, git_host, bot_host, live_host); return show(domain, admin_pw, streamers, editors, auth_host, watch_host, git_host, bot_host, live_host); return
if mode == "--apply":
if not domain: print("domain not set yet — run ./configure.py --full first"); sys.exit(1)
apply(domain, admin_pw, streamers, editors, auth_host, watch_host, git_host=git_host, bot_host=bot_host, live_host=live_host)
show(domain, admin_pw, streamers, editors, auth_host, watch_host, git_host, bot_host, live_host); return
if mode == "--hosts": if mode == "--hosts":
if len(sys.argv) not in (4, 5, 6, 7): print("usage: configure.py --hosts AUTH_HOST WATCH_HOST [GIT_HOST [BOT_HOST [LIVE_HOST]]]"); sys.exit(2) if len(sys.argv) not in (4, 5, 6, 7): print("usage: configure.py --hosts AUTH_HOST WATCH_HOST [GIT_HOST [BOT_HOST [LIVE_HOST]]]"); sys.exit(2)
if not domain: print("domain not set yet — run ./configure.py --full first"); sys.exit(1) if not domain: print("domain not set yet — run ./configure.py --full first"); sys.exit(1)

113
deploy.sh Executable file
View File

@@ -0,0 +1,113 @@
#!/bin/bash
# Minimal-impact deployer for the server checkout. Run as root (sudo) — installed as /usr/local/bin/vanguard-deploy.
# vanguard-deploy pull fetch + fast-forward DEPLOY_BRANCH, then apply only what changed
# vanguard-deploy apply [OLDREV] apply changes between OLDREV (default: last deployed rev) and HEAD
# vanguard-deploy rollback check out the previously deployed rev and apply the reverse diff
# vanguard-deploy pending restart MediaMTX if a restart is pending and no stream is live (timer runs this)
# vanguard-deploy status
# Env (from /etc/vanguard/deploy.env): DEPLOY_REPO_DIR, DEPLOY_BRANCH, DEPLOY_NOTIFY_WEBHOOK_URL. VG_DRY=1 prints only.
set -uo pipefail
ENVF=/etc/vanguard/deploy.env; [[ -f $ENVF ]] && set -a && . "$ENVF" && set +a
REPO="${DEPLOY_REPO_DIR:-/home/debian/vanguard-stream}"; BRANCH="${DEPLOY_BRANCH:-main}"
STATE=/var/lib/vanguard; mkdir -p "$STATE" 2>/dev/null || true
LASTF="$STATE/deployed-rev"; PREVF="$STATE/previous-rev"; PENDING="$STATE/mediamtx-restart-pending"
DRY="${VG_DRY:-0}"; OWNER=$(stat -c %U "$REPO" 2>/dev/null || echo debian)
log() { echo "$(date '+%F %T') deploy: $*"; }
run() { log "\$ $*"; [[ $DRY == 1 ]] || "$@"; }
asowner() { log "\$ (as $OWNER) $*"; [[ $DRY == 1 ]] || sudo -u "$OWNER" -H "$@"; }
notify() {
[[ -n ${DEPLOY_NOTIFY_WEBHOOK_URL:-} && $DRY != 1 ]] || return 0
curl -s -m 10 -H 'Content-Type: application/json' -d "$(python3 -c 'import json,sys; print(json.dumps({"content": sys.argv[1][:1900]}))' "$1")" "$DEPLOY_NOTIFY_WEBHOOK_URL" >/dev/null || true
}
cd "$REPO" || { log "repo dir $REPO missing"; exit 1; }
git rev-parse --is-inside-work-tree >/dev/null 2>&1 || { log "$REPO is not a git checkout (run switchToClone.sh once)"; exit 1; }
live_streams() { curl -s -m 3 'http://127.0.0.1:9997/v3/paths/list' 2>/dev/null | python3 -c 'import json,sys; d=json.load(sys.stdin); print(sum(1 for p in d.get("items",[]) if p.get("ready")))' 2>/dev/null || echo 0; }
restart_mediamtx_or_defer() {
local n; n=$(live_streams)
if [[ $n == 0 ]]; then
run docker compose restart mediamtx; rm -f "$PENDING"; log "mediamtx restarted"
else
[[ $DRY == 1 ]] || touch "$PENDING"; log "mediamtx restart DEFERRED: $n live stream(s); deploy-pending.timer retries every 5 min"
notify "⏳ deploy: MediaMTX restart deferred ($n live stream(s)); will restart when idle"
fi
}
apply_changes() {
local old="$1" new="$2"
local changed; changed=$(git diff --name-only "$old" "$new" 2>/dev/null || git ls-files)
[[ -n $changed ]] || { log "nothing changed between $old and $new"; return 0; }
log "changed files:"; echo "$changed" | sed 's/^/ /'
has() { echo "$changed" | grep -qE "$1"; }
# 0. static checks on the new tree
if [[ -x check.sh ]]; then asowner bash check.sh >/tmp/vg-check.log 2>&1 || { log "check.sh FAILED — aborting"; tail -20 /tmp/vg-check.log; notify "❌ deploy $new: check.sh failed"; return 1; }; fi
# 1. host scripts + units
if has '^bin/'; then run install -m 0755 bin/normalize-sweep.sh bin/trash-purge.sh bin/dumpd.py bin/vgstrip.py /usr/local/bin/; fi
if has '^systemd/'; then run install -m 0644 systemd/*.service systemd/*.timer /etc/systemd/system/; run systemctl daemon-reload; fi
if has '^deploy\.sh$'; then run install -m 0755 deploy.sh /usr/local/bin/vanguard-deploy; fi
if has '^firewall/nftables\.conf$'; then run install -m 0644 firewall/nftables.conf /etc/nftables.conf; run nft -f /etc/nftables.conf; fi
# 2. regenerate configs (Caddy reload, copyparty restart) — configure.py decides; MediaMTX is handled below
if has '^configure\.py$|^syncthing/stignore$|^dock/|^clips/|^about/'; then
local before_mtx; before_mtx=$(md5sum mediamtx.yml 2>/dev/null | cut -d' ' -f1)
if [[ $DRY == 1 ]]; then log "\$ (as $OWNER) VG_DRY=1 python3 configure.py --apply"; else sudo -u "$OWNER" -H python3 configure.py --apply >/tmp/vg-configure.log 2>&1 || { log "configure --apply FAILED"; tail -20 /tmp/vg-configure.log; return 1; }; fi
local after_mtx; after_mtx=$(md5sum mediamtx.yml 2>/dev/null | cut -d' ' -f1)
[[ $before_mtx == "$after_mtx" ]] || { log "mediamtx.yml changed"; restart_mediamtx_or_defer; }
fi
if has '^bin/dumpd\.py$|^systemd/dumpd\.service$'; then run systemctl restart dumpd; fi
# 3. compose: recreate changed services, but never MediaMTX while live
if has '^docker-compose\.yml$|^bot/Dockerfile$|^bot/app/requirements\.txt$'; then
local mtx_changed=0
git diff "$old" "$new" -- docker-compose.yml | grep -qE '^[+-]' && git diff -U0 "$old" "$new" -- docker-compose.yml | grep -qiE '^[+-].*(mediamtx|8189|8890|1935)' && mtx_changed=1
if [[ $mtx_changed == 1 && $(live_streams) != 0 ]]; then
log "compose touches mediamtx and streams are live: recreating everything except mediamtx"
run docker compose up -d --build --no-deps $(docker compose config --services | grep -v '^mediamtx$' | tr '\n' ' ')
[[ $DRY == 1 ]] || touch "$PENDING"; notify "⏳ deploy: MediaMTX recreate deferred (live streams)"
else
run docker compose up -d --build
fi
fi
# 4. Jellyfin plugin (committed zip)
if has '^jellyfin-plugin-vanguard/dist/.*\.zip$'; then run bash installJellyfinPlugin.sh; fi
# 5. live portal
if has '^live/'; then
if command -v npm >/dev/null; then
asowner bash -c 'cd live && npm install --no-audit --no-fund --loglevel=error && npm run build' && { run install -d -o "$OWNER" /srv/live; run rsync -a --delete live/dist/ /srv/live/; }
else log "npm not installed on the server: live/ changed but not built (apt-get install nodejs npm)"; fi
fi
log "applied $old..$new"
}
case "${1:-}" in
pull)
old=$(git rev-parse HEAD)
asowner git fetch --quiet origin "$BRANCH" || { log "fetch failed"; exit 1; }
asowner git merge --ff-only "origin/$BRANCH" || { log "fast-forward failed (local changes?)"; exit 1; }
new=$(git rev-parse HEAD)
[[ $old == "$new" ]] && { log "already at $new"; exit 0; }
if apply_changes "$old" "$new"; then
[[ $DRY == 1 ]] || { cp -f "$LASTF" "$PREVF" 2>/dev/null; echo "$new" > "$LASTF"; }
notify "✅ deploy $(git log -1 --format='%h %s' "$new")"
else notify "❌ deploy $new failed — see journalctl -u dumpd"; exit 1; fi ;;
apply)
old="${2:-$(cat "$LASTF" 2>/dev/null || git rev-parse HEAD~1)}"; new=$(git rev-parse HEAD)
apply_changes "$old" "$new" && { [[ $DRY == 1 ]] || { cp -f "$LASTF" "$PREVF" 2>/dev/null; echo "$new" > "$LASTF"; }; } ;;
rollback)
prev=$(cat "$PREVF" 2>/dev/null) || { log "no previous rev recorded"; exit 1; }
cur=$(git rev-parse HEAD); log "rolling back $cur -> $prev"
asowner git checkout --quiet "$prev" && apply_changes "$cur" "$prev" && { [[ $DRY == 1 ]] || { echo "$cur" > "$PREVF"; echo "$prev" > "$LASTF"; }; notify "↩️ rolled back to $prev"; } ;;
pending)
[[ -f $PENDING ]] || exit 0
restart_mediamtx_or_defer ;;
status)
echo "repo: $REPO ($(git rev-parse --abbrev-ref HEAD) @ $(git rev-parse --short HEAD))"
echo "deployed: $(cat "$LASTF" 2>/dev/null || echo '-') previous: $(cat "$PREVF" 2>/dev/null || echo '-')"
echo "auto: DEPLOY_AUTO=${DEPLOY_AUTO:-0} branch: $BRANCH pending mediamtx restart: $([[ -f $PENDING ]] && echo yes || echo no) live streams: $(live_streams)" ;;
*) sed -n '2,9p' "$0"; exit 2 ;;
esac

View File

@@ -7,7 +7,7 @@ OWNER="${SUDO_USER:-debian}" # the login user; owns /srv/media and may run
echo "== packages" echo "== packages"
apt-get update -q apt-get update -q
apt-get install -y -q ca-certificates curl gnupg ffmpeg jq psmisc rclone python3 xfsprogs apt-get install -y -q ca-certificates curl gnupg ffmpeg jq psmisc rclone python3 python3-yaml xfsprogs git nodejs npm rsync
if ! command -v docker >/dev/null; then if ! command -v docker >/dev/null; then
echo "== docker" echo "== docker"

View File

@@ -1,4 +1,5 @@
bin/ bin/
obj/ obj/
*.user *.user
dist/ dist/*
!dist/VanguardMetadata_*.zip

Binary file not shown.

33
switchToClone.sh Executable file
View File

@@ -0,0 +1,33 @@
#!/bin/bash
# One-time: turn ~/vanguard-stream (an scp copy) into a git clone of the Gitea repo, keeping the generated files.
# ./switchToClone.sh key -> creates a deploy key and prints the public half (add it in Gitea: repo → Settings → Deploy keys, read-only)
# ./switchToClone.sh clone -> clones over ssh (port 2222), moves .env + generated files + plugin zips into it
set -euo pipefail
REPO_SSH="${REPO_SSH:-ssh://git@git.dumpstervigilantes.rip:2222/techy/Media-Server-Setup.git}"
DIR="$HOME/vanguard-stream"; KEY="$HOME/.ssh/vanguard-deploy"
case "${1:-}" in
key)
[[ -f $KEY ]] || ssh-keygen -t ed25519 -N "" -C "vanguard-deploy@$(hostname)" -f "$KEY" >/dev/null
cat > "$HOME/.ssh/config.vanguard" <<CFG
Host git.dumpstervigilantes.rip
Port 2222
User git
IdentityFile $KEY
IdentitiesOnly yes
CFG
grep -q 'config.vanguard' "$HOME/.ssh/config" 2>/dev/null || { echo "Include config.vanguard" | cat - "$HOME/.ssh/config" 2>/dev/null > "$HOME/.ssh/config.new" || echo "Include config.vanguard" > "$HOME/.ssh/config.new"; mv "$HOME/.ssh/config.new" "$HOME/.ssh/config"; chmod 600 "$HOME/.ssh/config"; }
echo "Add this as a READ-ONLY deploy key on the repo (Gitea → repo → Settings → Deploy Keys):"; echo; cat "$KEY.pub" ;;
clone)
[[ -d $DIR/.git ]] && { echo "$DIR is already a clone"; exit 0; }
ssh -o StrictHostKeyChecking=accept-new -p 2222 git@git.dumpstervigilantes.rip 2>&1 | grep -qi "successfully authenticated\|Hi there" || { echo "deploy key not accepted yet — run './switchToClone.sh key' and add it in Gitea"; exit 1; }
mv "$DIR" "$DIR.old"
git clone --quiet "$REPO_SSH" "$DIR"
for f in .env Caddyfile mediamtx.yml copyparty.conf stream-token.secret deploy.env; do [[ -f $DIR.old/$f ]] && cp -a "$DIR.old/$f" "$DIR/$f"; done
mkdir -p "$DIR/jellyfin-plugin-vanguard/dist"; cp -a "$DIR.old"/jellyfin-plugin-vanguard/dist/*.zip "$DIR/jellyfin-plugin-vanguard/dist/" 2>/dev/null || true
[[ -d $DIR.old/bot/app ]] && cp -a "$DIR.old/bot/app/." "$DIR/bot/app/" 2>/dev/null || true
( cd "$DIR" && sudo -n true 2>/dev/null && sudo install -m 0600 -o root -g root .env /dev/null >/dev/null 2>&1 || true )
echo "cloned. kept: .env, generated configs, plugin zips, bot/app. Old copy at $DIR.old (delete when happy)."
echo "next: cd $DIR && ./configure.py --apply (records nothing new; just confirms generation works from the clone)"
echo " then: sudo vanguard-deploy status" ;;
*) sed -n '2,5p' "$0"; exit 2 ;;
esac

View File

@@ -0,0 +1,7 @@
[Unit]
Description=Restart MediaMTX if a deploy deferred it and no stream is live
After=docker.service
[Service]
Type=oneshot
ExecStart=/usr/local/bin/vanguard-deploy pending

View File

@@ -0,0 +1,9 @@
[Unit]
Description=Retry deferred MediaMTX restart every 5 minutes
[Timer]
OnBootSec=3min
OnUnitActiveSec=5min
[Install]
WantedBy=timers.target