Record streams into hidden .live/ and publish finished files via the sweep; plugin 0.1.2 feed wording

Jellyfin indexed in-progress fragmented recordings (no duration -> seek jumped to next episode). MediaMTX now writes under <Title>/.live/; the sweep remuxes finished files into the title folder and removes the source. Feed says 'New recording' since it fires after publish. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
STATUS: repo now hosted on the stack's Gitea
2026-10-30 22:45:30 -05:00 · 2026-10-30 15:56:44 -05:00
8 changed files with 40 additions and 18 deletions
--- a/README.md
+++ b/README.md
@@ -201,7 +201,8 @@ Pulls everything older than 15 days off the server, including the hidden `.src/`

 ## Layout on the server

-    /srv/media/streams/<user>/<Title>/2025-10-10_21-03-11.mp4   stream, remuxed flat (lossless) by the sweep
+    /srv/media/streams/<user>/<Title>/.live/<ts>.mp4             recording in progress (hidden from Jellyfin/copyparty)
+    /srv/media/streams/<user>/<Title>/2025-10-10_21-03-11.mp4   finished stream, flat MP4 published by the sweep
    /srv/media/clips/<user>/clip.mp4                             clip (synced as-is, or converted; raw removed)
    /srv/media/clips/<user>/.stlocal                             Syncthing ignore list maintained by the sweep
    /srv/media/clips/<user>/.src/clip.mkv                        raw source, only for clips that did NOT arrive via Syncthing
@@ -220,7 +221,7 @@ upscaled; aspect kept).
 | input | action |
 |-------|--------|
 | already compliant | nothing |
-| MediaMTX recording (fragmented MP4, good codecs) | lossless remux to flat MP4, replaced in place |
+| MediaMTX recording in `.live/` (fragmented MP4) | once idle 3 min: lossless remux to a flat MP4 in the title folder, `.live` source removed |
 | in a Syncthing folder (`.stfolder` marker) | compliant / index-only: untouched; otherwise convert → `name.mp4`, add raw to `.stlocal`, delete raw (one copy) |
 | HEVC / 10-bit / VFR / MKV / >1440p / >60fps / non-AAC | full-res re-encode (x264 veryfast CRF 18) → `name.mp4`; source kept in `.src/` |
 | ffmpeg fails or duration mismatch | `name.normalize-failed.log` written, file skipped until that log is deleted |
--- a/STATUS.md
+++ b/STATUS.md
@@ -1,16 +1,17 @@
 # Vanguard stack — what runs, what people do, what's unfinished

-Living document. Updated whenever something is deployed or changes behaviour. Last update: 2026-10-30.
-Host: `ovh1.dumpstervigilantes.rip` (OVH SYS-1, Debian 13). Bundle: `~/vanguard-stream` on the box, `vanguard-stream/` in this repo.
+Living document. Updated whenever something is deployed or changes behaviour. Last update: 2026-10-30 (evening).
+Host: `ovh1.dumpstervigilantes.rip` (OVH SYS-1, Debian 13). Bundle: `~/vanguard-stream` on the box.
+Source of truth: this repo, `techy/Media-Server-Setup` on the stack's own Gitea (`ssh://git@git.dumpstervigilantes.rip:2222/techy/Media-Server-Setup.git`).

 ## Runs on its own

 | Cadence | What happens | Done by |
 |---|---|---|
-| Continuous | OBS streams land as files the moment Go Live is pressed, one per stream under `streams/<user>/<Title>/` | MediaMTX |
+| Continuous | OBS streams record into a hidden `streams/<user>/<Title>/.live/` folder, invisible to Jellyfin and copyparty until finished | MediaMTX |
 | Continuous | Gamers' clip folders mirror onto the server (video files only, their structure untouched, receive-only) | Syncthing |
 | Continuous | New recordings and clips are announced in a Discord channel with the streamer's name and a frame grab | Jellyfin plugin (needs bot token + channel in its config) |
-| Every 5 min | Finished files are checked. Streams: lossless remux so they seek instantly. Non-compliant clips: converted to the editing standard, raw removed, one copy per clip | normalize sweep (systemd timer) |
+| Every 5 min | Finished files are checked. Streams idle ≥3 min: lossless remux out of `.live/` into the title folder as a flat, seekable MP4 (this is when they appear in Jellyfin/copyparty and the Discord notice fires). Non-compliant clips: converted to the editing standard, raw removed, one copy per clip | normalize sweep (systemd timer) |
 | Every 5 min | Discord scheduled events snapshotted into a history so recordings can be matched to their session | Jellyfin plugin |
 | Every 30 min + startup | Every non-admin Jellyfin user set to direct-play-only (no hardware encoder on this box) | Jellyfin plugin |
 | Daily 04:10 ET | Recordings roll between weekday → week → month → year seasons as the calendar moves; no files change | Jellyfin plugin |
@@ -49,16 +50,16 @@ Host: `ovh1.dumpstervigilantes.rip` (OVH SYS-1, Debian 13). Bundle: `~/vanguard-
 | Caddy | TLS + routing for all hostnames | live |
 | Authentik (+Postgres, Redis) | Discord login, groups, guild-membership policy | live |
 | Jellyfin | watch page, SSO via Authentik, direct play only | live |
-| Vanguard Metadata plugin | seasons by tag, Discord names/avatars/events, feed, playback policy | **0.1.0 live, 0.1.1 built** (policy task) — install pending |
+| Vanguard Metadata plugin | seasons by tag, Discord names/avatars/events, feed, playback policy | **0.1.0 live, 0.1.2 built** (policy task, "new recording" feed wording) — install pending |
 | MariaDB | databases for the bot and Gitea; public :3306, root local-only | live |
-| Gitea | git hosting on MariaDB, Actions enabled | live, local admin |
+| Gitea | git hosting on MariaDB, Actions enabled; holds this repo | live, local admin |
 | act_runner | CI jobs in Docker | **not registered** |
 | discord-bot | Python runtime, idles until `bot/app/main.py` exists | live, no code yet |
 | `bot.` host | bot web UI behind Authentik forward-auth | Caddy block ready; **CNAME + Authentik proxy provider pending** |

 ## Unfinished, in rough priority

-1. **Install plugin 0.1.1** (`./installJellyfinPlugin.sh`), run "enforce playback policy", check a user.
+1. **Deploy the `.live` recording change** (mediamtx.yml + sweep) and **plugin 0.1.2**; refresh the one stream Jellyfin indexed while it was still recording.
 2. **Register the runner** (`./registerGiteaRunner.sh`).
 3. **`bot` CNAME + Authentik proxy provider** — first use of forward-auth; proves the pattern for copyparty.
 4. **copyparty behind Authentik** (group permissions, WebDAV via app passwords, then drop local accounts). The one change that can lock editors out; keep a local admin until a Discord editor has used the new path.
@@ -73,6 +74,7 @@ Host: `ovh1.dumpstervigilantes.rip` (OVH SYS-1, Debian 13). Bundle: `~/vanguard-

 ## Verified behaviours worth remembering
 - Jellyfin honours provider-supplied season numbers on a Shows library (day seasons appeared). Next check: Monday 04:10 ET should fold them into a week season.
+- A stream indexed by Jellyfin while still recording has no duration and skips to the next episode on seek — fixed by recording into `.live/` so nothing sees a file until it is finished and flat.
 - Sweep leaves compliant Syncthing files untouched and keeps exactly one copy of converted ones (tested).
 - Firewall: Docker-published ports bypass the input chain; MariaDB uses host networking precisely so the rules apply.
 - Never `systemctl stop|restart nftables` (flushes Docker's NAT). Reload only.
--- a/bin/normalize-sweep.sh
+++ b/bin/normalize-sweep.sh
@@ -5,6 +5,9 @@
 # - fragmented/unindexed MP4 (MediaMTX output) with good codecs: lossless remux, replaced in place
 # - anything else (HEVC, 10-bit, VFR, MKV, >1440p, >60fps): full-res re-encode -> name.mp4,
 #   raw source kept in a hidden .src/ subfolder next to it (Jellyfin/copyparty hide dot-dirs)
+# - streams/<user>/<Title>/.live/<ts>.mp4 (MediaMTX writes here, hidden from Jellyfin/copyparty while live):
+#   when finished -> flat MP4 written to the parent folder, the .live source deleted. One copy, and nothing
+#   indexes a recording in progress.
 # - Syncthing folders (marker dir .stfolder in the folder root, receive-only): compliant files are left
 #   alone (zero copies); a non-compliant file is converted to name.mp4 (name.h264.mp4 if the raw is
 #   already .mp4), the raw's name is appended to .stlocal (an #include'd ignore list) so Syncthing
@@ -66,6 +69,11 @@ find "$ROOT/streams" "$ROOT/clips" -type f \
     ! -path '*/.src/*' ! -path '*/.stfolder/*' ! -path '*/.stversions/*' ! -name '*.part' -mmin +"$MINS_IDLE" -print0 2>/dev/null |
 while IFS= read -r -d '' src; do
  base="${src%.*}"; ext="${src##*.}"
+  live=0
+  if [[ $(basename "$(dirname "$src")") == .live ]]; then
+    live=1
+    base="$(dirname "$(dirname "$src")")/$(basename "${src%.*}")"
+  fi
  synced=0; stroot=""
  d="$(dirname "$src")"
  while [[ $d == "$ROOT"/* ]]; do                          # find the Syncthing folder root, if any
@@ -77,8 +85,14 @@ while IFS= read -r -d '' src; do
  [[ -e "$src" ]] || continue                          # dumped meanwhile

  read -r verdict reason <<<"$(probe "$src")"
-  case "$verdict" in ok|skip) continue ;; esac
-  out="$base.mp4"
+  if (( live )); then
+    [[ $verdict == skip ]] && continue
+    [[ $verdict == ok ]] && verdict=remux          # must leave .live/ either way
+    out="$base.mp4"; [[ -e "$out" ]] && { log "exists  $out (leaving $src)"; continue; }
+  else
+    case "$verdict" in ok|skip) continue ;; esac
+    out="$base.mp4"
+  fi
  if (( synced )); then
    [[ $verdict == remux ]] && continue                    # synced + only an index issue: not worth touching
    [[ ${ext,,} == mp4 ]] && out="$base.h264.mp4"          # raw is already name.mp4: don't collide
@@ -104,7 +118,11 @@ while IFS= read -r -d '' src; do
  rc=$?
  d1=$(dur "$src"); d2=$(dur "$tmp")
  if (( rc == 0 )) && [[ -n $d1 && -n $d2 ]] && (( d2 >= d1 - 2 && d2 <= d1 + 2 )); then
-    if (( synced )); then
+    if (( live )); then
+      mv -f "$tmp" "$out"; chown 1000:1000 "$out" 2>/dev/null
+      rm -f "$src"; rmdir "$(dirname "$src")" 2>/dev/null || true
+      log "done   $out"; continue
+    elif (( synced )); then
      rel_src="${src#"$stroot"/}"; rel_out="${out#"$stroot"/}"
      { echo "/$rel_out"; echo "/$rel_src"; } >> "$stroot/.stlocal"   # ignore our output and the raw
      chown 1000:1000 "$stroot/.stlocal" 2>/dev/null
--- a/configure.py
+++ b/configure.py
@@ -255,7 +255,7 @@ authInternalUsers:
 # ---- recording --------------------------------------------------------
 pathDefaults:
  record: yes
-  recordPath: /recordings/%path/%Y-%m-%d_%H-%M-%S
+  recordPath: /recordings/%path/.live/%Y-%m-%d_%H-%M-%S   # hidden while recording; the sweep publishes the finished file one level up
  recordFormat: fmp4
  recordPartDuration: 1s
  recordSegmentDuration: 12h      # one file per stream unless it runs >12h
--- a/jellyfin-plugin-vanguard/Jellyfin.Plugin.Vanguard/Jellyfin.Plugin.Vanguard.csproj
+++ b/jellyfin-plugin-vanguard/Jellyfin.Plugin.Vanguard/Jellyfin.Plugin.Vanguard.csproj
@@ -2,8 +2,8 @@
  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <RootNamespace>Jellyfin.Plugin.Vanguard</RootNamespace>
-    <AssemblyVersion>0.1.1.0</AssemblyVersion>
-    <FileVersion>0.1.1.0</FileVersion>
+    <AssemblyVersion>0.1.2.0</AssemblyVersion>
+    <FileVersion>0.1.2.0</FileVersion>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <GenerateDocumentationFile>false</GenerateDocumentationFile>
--- a/jellyfin-plugin-vanguard/Jellyfin.Plugin.Vanguard/Services/FeedService.cs
+++ b/jellyfin-plugin-vanguard/Jellyfin.Plugin.Vanguard/Services/FeedService.cs
@@ -54,7 +54,7 @@ public sealed class FeedService : IHostedService
            var link = string.IsNullOrWhiteSpace(cfg.PublicUrl) ? null : $"{cfg.PublicUrl.TrimEnd('/')}/web/#/details?id={ep.Id:N}";
            var name = who?.DisplayName ?? parts?.SeriesFolder ?? "someone";

-            var heading = kind == Kind.Stream ? $"🔴 {name} is recording: {title}" : $"🎬 New clip from {name}: {title}";
+            var heading = kind == Kind.Stream ? $"📼 New recording from {name}: {title}" : $"🎬 New clip from {name}: {title}";
            var fields = new List<(string, string)> { ("When", when.ToString("ddd MMM d, h:mm tt", CultureInfo.InvariantCulture)) };
            if (parts?.Title is not null && kind == Kind.Clip) fields.Add(("Folder", parts.Title));

--- a/jellyfin-plugin-vanguard/build.yaml
+++ b/jellyfin-plugin-vanguard/build.yaml
@@ -1,6 +1,6 @@
 name: "Vanguard Metadata"
 guid: "d00ebd97-f987-4678-8089-c8f0f8a6ab3e"
-version: "0.1.1.0"
+version: "0.1.2.0"
 targetAbi: "12.4.0.0"
 framework: "net10.0"
 overview: "Season/episode metadata for gameplay recordings from timestamps, Discord and Authentik."
@@ -14,5 +14,6 @@ artifacts:
 - "Jellyfin.Plugin.Vanguard.dll"
 - "Jellyfin.Plugin.Vanguard.Core.dll"
 changelog: >
+  0.1.2.0 feed says "New recording" (streams are published only after they finish)
  0.1.1.0 playback-policy enforcement task (direct-play-only for all users)
  0.1.0.0 initial skeleton
--- a/mediamtx.yml
+++ b/mediamtx.yml
@@ -34,7 +34,7 @@ authInternalUsers:
 # ---- recording --------------------------------------------------------
 pathDefaults:
  record: yes
-  recordPath: /recordings/%path/%Y-%m-%d_%H-%M-%S
+  recordPath: /recordings/%path/.live/%Y-%m-%d_%H-%M-%S   # hidden while recording; the sweep publishes the finished file one level up
  recordFormat: fmp4
  recordPartDuration: 1s
  recordSegmentDuration: 12h      # one file per stream unless it runs >12h