Files
Media-Server-Setup/about/index.html
Brandyn / Techy ca01516d42 Stuck-recording triage, a diag landing page, and the fixes behind Tuesday's symptoms
Triage
- bin/vgstuck.sh classifies every unpublished recording (parked by a failure log, published
  collision, waiting, still writing), with ffmpeg's own reason for parked ones. retry clears the
  blocker; drop moves to trash/<today>/ with the same 3-day undo as a dumped night; drop --all is
  refused. A leftover .mp4.part blocks nothing (ffmpeg overwrites with -y) and is only cleaned up.
- The same tool backs a page at diag.HOST/stuck. dumpd serves /stuck/api, lets anyone past the gate
  read, and allows retry/bin only for the admin groups; the path validation is the single thing
  between a request body and a file move, so it accepts only an existing .mp4 under .live/ inside
  the streams tree.
- The sweep logs event=parked once per run so Kibana can name stuck files rather than only count
  them.

Landing
- diag.HOST opens on a "Vanguard ops" dashboard via Kibana's defaultRoute: a markdown panel of links
  to every host and the triage page, plus Discover, Dashboards, Stack Rules and Index Management.
  Kibana stays at the root; moving it under /kibana would have meant the prefix in both Beats'
  setup.kibana, two scripts and the deadman, with Metricbeat crash-looping on whichever was missed.
- /api/kibana/settings is an internal API behind server.restrictInternalApis, which Kibana 9 turns
  on by default, so the script sends x-elastic-internal-origin. kibana-setup.py takes a step name and
  verifies each step by reading it back.

Installers
- bin/install.list is the one list of what lands in /usr/local/bin, read by install.sh,
  configure.py and deploy.sh. deploy.sh's own copy had drifted to five of ten files. check.sh fails
  on drift. deploy.sh also recreates a Beat or Kibana when its bind-mounted config changes, which
  nothing did before.
- check.sh distinguishes absent from broken: the server checkout is partial, and a yaml file or node
  binary that is not there is a skip, not a failure.

Fixes
- Go Live pixelation: x-google-min-bitrate=6000 forbade Chrome from encoding below 6 Mbps, so on a
  weak uplink the excess was dropped and MediaMTX logged invalid FU-A packets. Loss was on the
  publish sessions only. Floor is 1200 and scaleResolutionDownBy is unpinned.
- HLS manifestParsingError: handle_path strips /hls, so MediaMTX's cookieCheck redirect came back
  root-relative and landed on the SPA catch-all. header_down rewrites Location.
- Sweep audio: the band-removal branch overwrote $reason and lost the probe's audio=opus marker, so a
  Go Live recording with the band shipped Opus inside an MP4. probe_reason is kept separately.
- Deadman false alarm: alerts.state was only written when something changed, so quiet runs let it go
  stale. Every run stamps last_run now, and the deadman also checks is-failed on the unit.
- Both timer scripts load /etc/vanguard/alerts.env themselves, so a run by hand matches the timer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 21:25:43 -06:00

209 lines
16 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Vanguard Media</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<style>
:root{--bg:#141414;--fg:#e6e6e6;--mut:#9a9a9a;--acc:#3d8bfd;--ok:#2ecc71;--box:#1f1f1f;--line:#2e2e2e}
html,body{margin:0;background:var(--bg);color:var(--fg);font:15px/1.5 system-ui,sans-serif}
main{max-width:860px;margin:0 auto;padding:28px 16px 48px}
header{display:flex;align-items:baseline;gap:14px;flex-wrap:wrap;margin-bottom:6px}
h1{font-size:26px;margin:0}.who{color:var(--mut)}.who b{color:var(--fg)}
.pills{margin:6px 0 22px}.pill{display:inline-block;background:#2a2a2a;border-radius:999px;padding:2px 10px;font-size:12px;margin:0 6px 6px 0;color:#bbb}
section{background:var(--box);border:1px solid var(--line);border-radius:14px;padding:18px 20px;margin-bottom:16px}
section h2{margin:0 0 4px;font-size:17px}section .lead{color:var(--mut);margin:0 0 12px}
.links{display:grid;grid-template-columns:repeat(auto-fill,minmax(240px,1fr));gap:10px;margin:10px 0}
a.card{display:block;background:#262626;border:1px solid #333;border-radius:10px;padding:12px 14px;color:var(--fg);text-decoration:none}
a.card:hover{border-color:var(--acc)}a.card b{display:block;margin-bottom:2px}a.card span{color:var(--mut);font-size:13px}
details{margin-top:8px}summary{cursor:pointer;color:var(--mut);font-size:13px}
ol,ul{margin:8px 0 0;padding-left:20px}li{margin:4px 0}code{font-family:ui-monospace,monospace;background:#121212;padding:1px 6px;border-radius:5px}
.hidden{display:none}footer{color:var(--mut);font-size:13px;margin-top:20px}footer a{color:var(--mut)}
</style>
</head>
<body>
<main>
<header><h1>Vanguard Media</h1><div class="who">signed in as <b id="who">…</b></div></header>
<div class="pills" id="pills"></div>
<section id="s-everyone">
<h2>Watch</h2>
<p class="lead">Everything here uses your Discord login. One sign-in covers all of it.</p>
<div class="links">
<a class="card" id="l-watch" href="#"><b>Recordings &amp; clips</b><span>Jellyfin — by streamer, by day/week/month</span></a>
<a class="card" id="l-live" href="#"><b>Live now</b><span>every active stream in one page, audio follows the window you click</span></a>
<a class="card" href="/logout"><b>Sign out</b><span>ends the Discord session for all Vanguard sites</span></a>
</div>
<details><summary>How watching works</summary>
<ul>
<li>Jellyfin: each streamer is a show. This week's recordings sit under weekday seasons, older ones roll into week, month and year. Nothing moves; it is only re-labelled.</li>
<li>Live: windows open as people go live and close when they stop. Click a window to hear it; the speaker button pins audio on; the slider is per window.</li>
</ul>
</details>
<details><summary>Live page: windows, groups, names</summary>
<ul>
<li><b>Layout.</b> Until you touch it, windows arrange themselves into an even grid whenever someone goes live or stops. Drag a tab to move a window, drag the bars between windows to resize, drop a tab onto another window to stack them as tabs.</li>
<li><b>Your layout stays yours.</b> The moment you move, resize or rename anything, the page stops rearranging. Streams that end just close. New streams <b>do not</b> barge in: a chime plays and a blinking chip appears in a thin tray under the top bar. Drag the chip onto any window edge, onto a tab bar to stack it, or to the page edge; or click it to add it as a new column on the right. <b>Arrange</b> (top bar) places everything in an even grid and switches automatic layout back on.</li>
<li><b>Audio.</b> Sound follows the window you click. The speaker button pins a window's audio on even when you click elsewhere; <b>Mute all</b> silences everything until you click a window; <b>Follow focus</b> clears pins.</li>
<li><b>Rename a stream.</b> Double-click its tab, type a name, Enter. Escape cancels; an empty name goes back to <code>user · title</code>. Names stay in your browser and come back the next time that streamer uses the same title.</li>
<li><b>Name a group.</b> Every tab bar starts with a <code>+ name</code> slot. Double-click it, type, Enter. Group names last until you reload the page.</li>
<li><b>One window big.</b> The ⛶ button at the right of a tab bar maximises that window; click it again to restore. Arranging (a stream joining or leaving, or the Arrange button) restores it too.</li>
<li><b>Timecode band.</b> Streams with the timecode band on show it cropped off automatically; recordings never contain it.</li>
</ul>
</details>
</section>
<section id="s-streamers" class="hidden">
<h2>Contribute: stream</h2>
<p class="lead" data-group="streamers" data-yes="You are in <code>streamers</code>: you can publish under your own name. No passwords, no stream keys to keep." data-no="Admin view — you are <b>not</b> in <code>streamers</code>, so Go Live will refuse a token for you until you get the role."></p>
<div class="links">
<a class="card" id="l-dock" href="#"><b>OBS dock</b><span>title → Go Live; Dump tonight; manual setup</span></a>
<a class="card" id="l-golive" href="#"><b>Go Live from the browser</b><span>Windows Chrome: share the game window, no OBS</span></a>
</div>
<details><summary>OBS setup (once)</summary>
<ol>
<li>OBS → Tools → WebSocket Server Settings → enable, set a password.</li>
<li>OBS → Docks → Custom Browser Docks → add <code id="dockurl">…/dock/</code>; log in with Discord inside the dock.</li>
<li>OBS → Settings → Output → encoder <b>B-frames: 0</b> (NVENC "Max B-frames"). With B-frames the live page cannot play you over WebRTC and falls back to a delayed HLS view.</li>
<li>Dock → Settings: OBS WebSocket password (the rest is prefilled). Optional title prefix/suffix (<code>{date}</code> = today).</li>
<li>Dock → Timecode band → <b>Add band + clock source</b> while not streaming (gives editors a frame-accurate timecode).</li>
<li>Per stream: type a title, <b>Go Live</b>. The dock mints a 12-hour token and sets your stream key itself.</li>
</ol>
</details>
<details><summary>Rules worth knowing</summary>
<ul>
<li>Everything streamed is recorded and visible to editors and members. <b>Dump tonight</b> throws a night away (3-day undo by an admin).</li>
<li>Titles: letters, digits, <code>_ . -</code>; no double underscore; the dock shows the final name as you type.</li>
<li>Browser Go Live needs Windows + Chrome/Edge 141+ for window audio and the GPU encoder; on Linux use OBS.</li>
</ul>
</details>
</section>
<section id="s-clips">
<h2>Contribute: your game clips</h2>
<p class="lead">Anyone in the server. Pair your gaming PC once; after that your clips upload themselves and appear in Jellyfin under your name.</p>
<div class="links">
<a class="card" id="l-clips" href="#"><b>Clip setup &amp; status</b><span>pair your PC, see if it is connected and syncing</span></a>
</div>
<details open><summary>Three steps</summary>
<ol>
<li>Install <a href="https://syncthing.net/downloads/" target="_blank" rel="noopener">Syncthing</a> on the PC your clips save to (Windows: Syncthing Tray works well). Open it → Actions → Show ID.</li>
<li>Paste that Device ID on the <a class="l-clips" href="#">clips page</a> and press Connect.</li>
<li>Back in your Syncthing: accept the new device <b>Vanguard</b>, then accept the folder it offers, point it at your clip directory (ShadowPlay / Medal / Steam), Folder Type <b>Send Only</b>.</li>
</ol>
<ul>
<li>Only video files are pulled; your folder layout is mirrored; odd formats are converted on the server and one copy is kept per clip.</li>
<li>Nothing is ever written back to your PC.</li>
</ul>
</details>
</section>
<section id="s-editors" class="hidden">
<h2>Edit</h2>
<p class="lead" data-group="editors" data-yes="You are in <code>editors</code>: read streams and clips, write finals." data-no="Admin view — you are not in <code>editors</code>; your copyparty rights come from <code>media-admins</code> instead."></p>
<div class="links">
<a class="card" id="l-files" href="#"><b>Browse &amp; download</b><span>copyparty — scrub in the browser, download originals, zip folders</span></a>
<a class="card" id="l-finals" href="#"><b>Upload finals</b><span>the only folder editors can write to</span></a>
</div>
<details><summary>Multicam &amp; timecode</summary>
<ul>
<li>Every stream file has a timecode track (time of day, Eastern) and a <code>.json</code> sidecar with the exact capture time, source (<code>strip</code> = frame-accurate from the overscan band, <code>arrival</code> = coarse), drift and gaps.</li>
<li>Multicam: sync by timecode, cross-check with voice audio. Export at the game resolution (2560×1440 / 1920×1080) — that drops the thin band at the bottom.</li>
<li>Files are H.264 MP4, constant frame rate, index up front. Streams are published a few minutes after they end.</li>
<li>No WebDAV (Discord-only login). Select files/folders and use copyparty's zip/tar download.</li>
</ul>
</details>
</section>
<section id="s-mediaadmins" class="hidden">
<h2>Media admin</h2>
<p class="lead" data-group="media-admins" data-yes="You are in <code>media-admins</code>: full rights on every folder, trash, Syncthing, dump anyone's night." data-no="Admin view — you are <b>not</b> in <code>media-admins</code>: copyparty will show you nothing and <code>/sync/</code> will 403 until you get the role."></p>
<div class="links">
<a class="card" id="l-stuck" href="#"><b>Stuck recordings</b><span>never published — see why, retry or bin</span></a>
<a class="card" id="l-trash" href="#"><b>Trash</b><span>dumped nights, purged after 3 days</span></a>
<a class="card" id="l-sync" href="#"><b>Syncthing</b><span>device pairing, folder states</span></a>
<a class="card" href="/whoami"><b>Who am I</b><span>identity headers as the apps see them</span></a>
</div>
<details><summary>Admin notes</summary>
<ul>
<li>Group changes in Authentik apply on the next login: <a href="/logout">sign out</a> and back in.</li>
<li>Dump anyone's night: the dock's Dump works on your own; for others use the API with <code>{"user":"name"}</code> or ask the owner.</li>
<li>New gamer: they self-serve on the <a class="l-clips" href="#">clips page</a>; you only need to watch Syncthing accept the device.</li>
<li><a class="l-stuck-note" href="#">Stuck recordings</a> (on the diagnostics host, with the dashboards) lists anything that finished but never published, with the reason. Retry puts it back in the queue; binning keeps the same 3-day undo as a dumped night.</li>
</ul>
</details>
</section>
<section id="s-git" class="hidden">
<h2>Code</h2>
<p class="lead" data-group="git-users" data-alt="git-admins" data-yes="You are in <code>git-users</code>." data-no="Admin view — you are <b>not</b> in <code>git-users</code>/<code>git-admins</code>: Authentik will refuse the Gitea login until you get the role."></p>
<div class="links">
<a class="card" id="l-git" href="#"><b>Gitea</b><span>repos, issues, CI — sign in with authentik</span></a>
</div>
<details><summary>Notes</summary>
<ul><li>SSH remotes use port 2222: <code id="gitssh">ssh://git@…:2222/org/repo.git</code>. Add your key under Settings → SSH keys.</li>
<li>The stack's own repo is <code>techy/Media-Server-Setup</code>.</li></ul>
</details>
</section>
<section id="s-bot" class="hidden">
<h2>Bot</h2>
<p class="lead" data-group="bot-users" data-yes="You are in <code>bot-users</code>." data-no="Admin view — you are <b>not</b> in <code>bot-users</code>: the bot host will 403 you until you get the role."></p>
<div class="links"><a class="card" id="l-bot" href="#"><b>Bot web UI</b><span>Discord bot control panel</span></a></div>
</section>
<section id="s-admins" class="hidden">
<h2>Administration</h2>
<p class="lead">You are in <code>admins</code> or <code>authentik Admins</code>.</p>
<div class="links">
<a class="card" id="l-auth" href="#"><b>Authentik admin</b><span>users, groups, Discord source, policies</span></a>
<a class="card" id="l-jfadmin" href="#"><b>Jellyfin dashboard</b><span>libraries, plugin, scheduled tasks</span></a>
<a class="card" id="l-gitadmin" href="#"><b>Gitea site admin</b><span>auth sources, runners</span></a>
</div>
<details><summary>Where things live</summary>
<ul>
<li>Server: <code>~/vanguard-stream</code> on the box; <code>STATUS.md</code> = what runs / who does what / what's unfinished; <code>README.md</code> = how.</li>
<li>Change hostnames, bot token: <code>./configure.py</code>. Plugin updates: <code>./installJellyfinPlugin.sh</code>. Portal: <code>./deployLive.sh</code> (dev box).</li>
<li>Streamers = Authentik group <code>streamers</code>; editors = <code>editors</code>; everything follows Discord roles via the roles policy.</li>
</ul>
</details>
</section>
<footer>Vanguard Media · <a href="/">home</a> · <a href="/logout">sign out</a></footer>
</main>
<script>
const $ = id => document.getElementById(id);
const host = location.hostname, base = host.replace(/^[^.]+\./, '');
const sub = s => `https://${s}.${base}`;
$('l-watch').href = sub('watch'); $('l-live').href = sub('live'); $('l-golive').href = sub('live') + '/#golive';
$('l-git').href = sub('git'); $('l-bot').href = sub('bot'); $('l-auth').href = sub('auth') + '/if/admin/';
$('l-jfadmin').href = sub('watch') + '/web/#/dashboard'; $('l-gitadmin').href = sub('git') + '/-/admin';
$('l-dock').href = sub('live') + '/dock/'; $('dockurl').textContent = sub('live') + '/dock/';
$('l-files').href = sub('files') + '/'; $('l-finals').href = sub('files') + '/finals/'; $('l-trash').href = sub('files') + '/trash/'; $('l-sync').href = sub('sync') + '/';
$('l-stuck').href = sub('diag') + '/stuck';
document.querySelectorAll('a.l-stuck-note').forEach(a => a.href = sub('diag') + '/stuck');
$('l-clips').href = sub('clips') + '/'; document.querySelectorAll('a.l-clips').forEach(a => a.href = sub('clips') + '/'); $('gitssh').textContent = `ssh://git@${'git.' + base}:2222/org/repo.git`;
(async () => {
try {
const r = await fetch('/me', {credentials:'same-origin'}); const d = await r.json(); if (!r.ok) throw new Error(d.error || r.status);
$('who').textContent = d.name && d.name !== d.user ? `${d.name} (${d.user})` : d.user;
$('pills').innerHTML = d.groups.map(g => `<span class="pill">${g}</span>`).join('') || '<span class="pill">no groups yet — ask an admin for a role</span>';
if (d.discord_id) $('who').title = 'Discord ID ' + d.discord_id; // apps identify you by this, not by your name
const has = g => d.groups.includes(g);
const show = (id, on) => $(id).classList.toggle('hidden', !on);
// sections appear only for what this account can actually do (the gates check these same groups)
const isAdmin = has('admins') || has('authentik Admins');
show('s-streamers', has('streamers'));
show('s-editors', has('editors') || has('media-admins'));
show('s-mediaadmins', has('media-admins'));
show('s-git', has('git-users') || has('git-admins'));
show('s-bot', has('bot-users'));
show('s-admins', isAdmin);
document.querySelectorAll('p.lead[data-group]').forEach(p => { p.innerHTML = p.dataset.yes; });
} catch (e) { $('who').textContent = 'unknown (' + e.message + ')'; }
})();
</script>
</body>
</html>