Cut your gameplay highlights while you sleep

The idea

A desktop app that sits on the same machine you stream or record from. When a session ends, it scans the raw capture, scores every second for "something happened here", and cuts the top moments into standalone clips with a few seconds of lead-in and tail. You wake up to a folder of twenty 15-second files, each named for when it happened and why it scored, plus a contact sheet so you can reject the bad ones in a minute instead of scrubbing a four-hour timeline.

Why build this

The bottleneck for anyone recording gameplay is not capture, it is triage. A three-hour session produces maybe eight usable moments, and finding them means scrubbing footage you already lived through once. The people who clip in real time use hotkeys and still miss things, because the interesting part is usually obvious only ten seconds later.

Everything needed to automate the triage now runs locally at reasonable speed. obs-websocket exposes the replay buffer and recording state. ffmpeg's ebur128 filter gives you a loudness curve for a long file in a fraction of realtime. whisper.cpp transcribes a mic track on a laptop CPU faster than the recording took to make. The signals are cheap; nobody has bothered to wire them together into something that just runs unattended.

Stack sketch

• Shell: Tauri — it is a file-watcher and a results grid, and shipping a 6 MB binary beats an Electron install for something that runs on a gaming PC.
• Capture integration: obs-websocket v5 to read the recording directory, session boundaries, and scene names; scene name is a useful label for the clip.
• Audio signals: ffmpeg ebur128 for a per-second loudness curve on the game track, and silencedetect on the mic track to find where you actually spoke.
• Speech: whisper.cpp with a small English model over the mic track only. You are not transcribing for accuracy, you are finding reaction spikes — laughter, swearing, "did you see that".
• Input signals: rdev to log keyboard and mouse events during the session at 100 ms resolution; bursts of input correlate with fights better than any pixel heuristic.
• Scoring: a weighted sum over the three curves, smoothed, with non-maximum suppression so one fight yields one clip instead of nine.
• Cutting: ffmpeg stream copy with keyframe-aligned seeks; re-encode only when the cut point has to be exact.
• Index: SQLite with the score curve, chosen segments, and your accept/reject decisions.

Scope for v1

In: - Watch one folder, process new recordings when the machine is idle. - The three signals above, fixed weights, a slider for "how many clips per hour". - A results grid with thumbnail, score reason, and keep/delete. - Export as-is, plus one vertical 9:16 crop preset.

Out: - Any game-specific detection — no kill-feed OCR, no per-title profiles. - Uploading anywhere. The output is files in a folder. - Cloud processing, accounts, or a hosted component of any kind. - Editing beyond trim handles. It is not a timeline.

Where it could go

The accept/reject decisions are training data sitting right there. After a few hundred clips you can fit a small per-user model over the same feature curves and replace the fixed weights, so the tool learns that this particular person clips quiet moments of concentration and not shouting. That is a genuine improvement over a generic highlight detector, and it stays local.

The second path is the other long-recording workflows that have the same shape and no tooling: tabletop sessions recorded over a webcam, a music practice rig, a coding stream where the interesting part is the moment something finally compiles. The signals change but the pipeline does not. A third path is pairing it with the replay buffer so scoring runs live and a hotkey saves the last thirty seconds and the reason it was already scoring high.

Watch out for

Non-maximum suppression is where this succeeds or fails — get the window wrong and one fight becomes nine near-identical clips, which is worse than no clips at all. Also budget for the fact that game audio and mic audio are often already mixed into one track in the user's OBS profile; detect that and say so up front rather than scoring garbage.