riffrec is a React-first package for capturing golden feedback: high-signal product sessions with screen video, microphone narration, clicks, navigation, network outcomes, and console errors. Installing the React package in your app is the recommended way to use Riffrec today.
There are already great tools for analytics and passive session replay. Riffrec is for the moments you intentionally turn on recording and capture the gold: the bug reproduction, the confused reaction, the broken flow, the product insight. In the age of AI slop, Riffrec gives agents concrete evidence instead of vague prompts.
Riffrec does not analyze sessions itself. Classic recording calls no LLM and writes local session files that agents, teammates, or Compound Engineering can inspect afterwards. Live mode, enabled per provider, streams the session to an endpoint you configure as it happens and runs a voice interviewer that connects to OpenAI Realtime from the browser with an ephemeral secret that endpoint mints; no provider option accepts an OpenAI key. See Live Mode.
AI is most useful when it has evidence. Product teams already find the evidence while using the app; Riffrec packages those moments into files an agent can inspect.
Use it when you want to turn product usage into:
- bug reports with reproduction context
- UI and UX improvement notes
- implementation tasks tied to exact DOM and network events
- sessions teammates or agents can review without asking the user to re-explain everything
Riffrec is designed to pair well with the Compound Engineering plugin. Record a session with Riffrec, then hand the session files to Compound Engineering so the agent can turn concrete product evidence into a sharper plan or implementation.
For a developer who can integrate Riffrec in a React app, the package can additionally identify React component context during feedback.
npm install riffrecimport { RiffrecProvider, RiffrecRecorder } from "riffrec";
export function App() {
return (
<RiffrecProvider>
<RiffrecRecorder />
{/* your app */}
</RiffrecProvider>
);
}RiffrecRecorder renders a start/stop button, consent dialog, capture checklist, and recording indicator. start() must still be called from a user gesture such as a button click because browsers require that for screen recording.
type RiffrecDisplayMediaVideo = MediaTrackConstraints;
type RiffrecDisplayMediaOptions = DisplayMediaStreamOptions & {
preferCurrentTab?: boolean;
selfBrowserSurface?: "include" | "exclude";
monitorTypeSurfaces?: "include" | "exclude";
surfaceSwitching?: "include" | "exclude";
systemAudio?: "include" | "exclude";
};
interface RiffrecConfig {
displayMedia?: Partial<RiffrecDisplayMediaOptions>;
displayMediaVideo?: Partial<RiffrecDisplayMediaVideo>;
downloadNoticeTitle?: string;
downloadNoticeMessage?: string;
forceEnable?: boolean;
forceEnableParam?: boolean | string;
onError?: (err: Error) => void;
sanitizeError?: (msg: string, stack: string | null) => string;
live?: RiffrecLiveConfig; // see Live Mode
}RiffrecProvider is disabled in production by default. In production builds it emits a single warning and start() is a no-op unless forceEnable={true} is passed explicitly.
Screen capture merges displayMedia and displayMediaVideo with built-in defaults. Riffrec asks Chromium to make the current tab prominent (preferCurrentTab: true, selfBrowserSurface: "include", monitorTypeSurfaces: "exclude", surfaceSwitching: "exclude", systemAudio: "exclude"), and records browser-tab video at frameRate: 5 by default. Override only what you need; browsers still require a user confirmation for screen capture.
For production debugging links, the host app can also opt into URL-param activation:
<RiffrecProvider forceEnableParam>
<App />
</RiffrecProvider>Then production recording is enabled when the page URL includes ?riffrec=1, ?riffrec=true, ?riffrec=on, or ?riffrec=yes. A custom param name is also supported:
<RiffrecProvider forceEnableParam="recordingDebug" />That enables recording for URLs such as ?recordingDebug=1. The param only bypasses the production guard; start() still requires a user gesture.
Use the hook when you want to build your own recording controls:
const { start, stop, status, live } = useRiffrec();status is one of "idle", "recording", "live", "stopping", "disabled", or "error". live is the live-mode control slice described under Live Mode; it reads "disabled" for hosts that do not enable live mode. While recording, RiffrecProvider renders a fixed stop control above the host app so the user always has a clear "Stop and save" action. After the download starts, it shows a confirmation telling the user to share the zip for feedback. Host apps can customize that confirmation with downloadNoticeTitle and downloadNoticeMessage. stop() downloads a zip file and returns:
{
sessionPath: string | null;
method: "zip";
filesPresent: string[];
sessionId: string;
filename: string;
archive: Blob;
}Hosts that upload or otherwise manage the archive can disable the automatic download per recording. The completion callback is captured when recording starts, so it also runs when the provider-level stop control is used:
<RiffrecRecorder
download={false}
onSessionComplete={async ({ archive, filename, sessionId }) => {
await uploadFeedback({ archive, filename, sessionId });
}}
/>Use the exported downloadSessionArchive(filename, archive) helper when a host-managed flow needs to offer a later local-download fallback.
Live mode turns a session into a stream instead of a zip you hand over afterwards. While the person riffs, a voice interviewer listens, is told what they click, draw on, and pin, can look at the screen, extracts one unit per requested change, and asks a clarifying question the moment something is ambiguous; a drawing layer lets them circle and pin elements; a board shows every unit and its status; and every event streams to an endpoint you configure, where a consumer such as ce-polish wakes a coding agent at each checkpoint. Live mode still records the screen and microphone locally; the zip is the fallback for a session the endpoint never received.
Enable it with the live prop:
<RiffrecProvider forceEnable live={{}}>
<App />
</RiffrecProvider>A runnable example lives in examples/live-demo: a small admin dashboard with exactly this mount, linked to the package in this checkout, with a README that walks through a live polish session locally and over HTTPS tunnels.
interface RiffrecLiveConfig {
endpoint?: string; // fallback endpoint origin when the URL fragment carries none
profile?: EvidenceProfileName | Partial<EvidenceProfile>; // what a unit carries on the wire; default "default"
autoStart?: boolean; // default: true only when the page carries #riffrec_live= credentials
drawShortcut?: string | null; // drawing-layer shortcut; default "Alt+Shift+D", null disables
endpointOwner?: string; // who runs the endpoint, named in the consent copy
download?: boolean; // default for the session's download option; see Ending
}Lazy loading. The live subtree — session, stream client, Realtime client, overlay, drawing layer, evidence capture — is a separate chunk loaded with React.lazy only when live is set and the production guard allows. A host that never sets live ships none of it and makes no new network calls. The production guard applies unchanged: without forceEnable (or the URL param) live mode renders nothing in production.
Bootstrap. The consumer hands the page its session credentials in the URL fragment: #riffrec_live=<page token>&endpoint=<origin>. Riffrec reads both on load, strips them from the URL before any history entry exists, and keeps them in sessionStorage for the session. live.endpoint is a fallback for hosts that run a fixed endpoint. Every request to the endpoint carries Authorization: Bearer <page token> and X-Riffrec-Session; the token never travels in a URL. Requests to the endpoint origin and to api.openai.com are excluded from network capture, and riffrec_live/endpoint fragment keys are stripped from every captured URL.
Starting. start() opens a consent step that names what will be streamed and to whom, derived from the active evidence profile: microphone audio, the session brief, what the person clicks, draws on, and pins, and screenshots of the page when they point at something or ask the interviewer to look, to OpenAI Realtime; transcript, units, strokes, frames, events, and any profile-enabled audio clips or telemetry to the named endpoint. Accepting acquires one microphone stream that is shared by the interviewer, the local voice.webm recording, and the utterance audio clips, then asks to share the screen. A denied microphone or a declined screen share keeps the session going with drawing and board only. With no endpoint configured the interviewer does not run (nothing can mint its secret) and the session records locally with annotations.
Voice. The interviewer connects to OpenAI Realtime over WebRTC from the browser using an ephemeral client secret the endpoint mints (POST /mint); no provider option accepts an OpenAI key. When the endpoint has no key, or OpenAI rejects its key, the live panel offers a field to paste one: it is kept in that browser's localStorage (riffrec:openai_key) and sent to the endpoint on each mint in the X-Riffrec-OpenAI-Key header, which the endpoint may use in place of its own. The endpoint's mint, stream, and wake contract is documented in docs/live-stream-contract.md, and the types, validateEnvelope, tool definitions, default persona, and fixtures are exported from the package root and the Node entry for endpoint authors.
What the interviewer knows about the screen. Every click the person makes reaches the interviewer as it happens, as a system note of the shape [PAGE] The riffer clicked <accessible name> with text "<visible text>" in component <Component> (selector <css>, route </path>) (anchor id: anchor_0007).; drawings and pins arrive the same way, and a burst of clicks on one element is one note. "This", "here", and "that" resolve to the most recent anchor id. The interviewer can also see: it has a look_at_screen tool that makes the page attach a screenshot of the current view as an image item, and the page attaches one on its own after a click or a drawing and when the person's words point at something visual, at most one every few seconds, downscaled to 1280 px wide. Frames the interviewer sees are also sent to the endpoint. Under an evidence profile with frames: "none" nothing visual leaves the page, and the tool says so. Clicks on riffrec's own panel are neither announced nor recorded.
Endpoint-owned persona. The endpoint's mint still owns the interviewer's instructions (persona and session brief) and may ship its own copy of the tools. After connecting, riffrec reads the session the endpoint minted and patches only what it must be able to answer: tools it handles that the mint lacked are added (the endpoint's copies win), a persona without the [SCREEN CONTEXT] section gets it appended, and a bare session with no riffrec tools gets the default persona and tool set. An endpoint that copies DEFAULT_INTERVIEWER_INSTRUCTIONS and LIVE_TOOLS verbatim is left untouched.
Controls. useRiffrec().live exposes:
interface RiffrecLiveControls {
status: LiveSessionStatus | "disabled"; // idle | consenting | connecting | live | live_novoice | buffering | reconnecting | incompatible | ended | error
mode: "instant" | "smart" | "collect"; // execution mode; each change streams a "mode" event
setMode: (mode) => void; // leaving Collect makes the endpoint wake the agent (mode_change)
muted: boolean;
setMuted: (muted: boolean) => void; // mutes the interviewer, voice.webm, and clips together
send: () => Promise<boolean>; // emits a "send" checkpoint
stop: () => Promise<SessionResult | null>; // ends the session and assembles the archive
}The overlay's own controls cover the same ground: a live indicator that distinguishes streaming, buffering, muted, and paused; the Instant / Smart / Collect switch; Send (a send checkpoint) and Done (the confirmation pass, then the final checkpoint, which always wakes the agent with the remaining backlog); withdraw and typed replies on the board; and a pause that stops frames and the stream while the local screen recording continues.
Reloads. A live session survives a page reload, a crash, and the provider unmounting: its state is persisted to sessionStorage and rehydrated on the next mount, delivery resumes with sequence numbers intact, the interviewer reconnects and re-seeds from the transcript, and the overlay asks once to share the screen again. Only stop(), the Done control, or the endpoint ending the session end it, and each assembles the archive.
Another session on the same link. The link also stays in localStorage after its session ends, because the fragment is gone once it has been read. While no session runs, riffrec asks the endpoint GET /session about every 20 seconds with that page token. If the endpoint can take a new session, the ended card shows Start another session; after it is dismissed, or after a reload, a small Endpoint ready pill takes its place. While the agent is still working through the last session's feedback, the button stays disabled. Nothing shows when this browser never opened a link, when the endpoint is unreachable, or when another tab holds the session. A token the endpoint rejects is forgotten.
Ending. Done emits the final checkpoint, waits for its acknowledgment, and posts the full-evidence archive to /session/end. When the endpoint confirms, the stream was the delivery: the archive is still assembled for onSessionComplete, the ended card replaces the download notice, and no zip is downloaded unless the host opted in with live.download: true or start({ download: true }). When the page ends the session on its own — no endpoint, a lost endpoint, an explicit stop(), or a Done the endpoint never confirmed — the zip is the delivery and downloads unless download is false; after a failed Done the notice names the reason (for example POST /session/end returned 502) and the host's onError receives it.
Archive additions. A live session's zip adds to the classic files:
transcript.json # when the interviewer ran
units.json # units with status and the end-of-session confirmations
annotations.json # strokes and pins with their anchors
frames/<id>.jpg # gesture, periodic, and composited frames
clips/<id>.webm # utterance audio clips, when the profile enables them
recording.webm, recording-002.webm, ... # one segment per screen share, split at reloads
events.json is unchanged, voice.webm stays, and session.json.files_present lists every file.
RiffrecRecorder is the quickest way to make recording understandable to the person using the app:
<RiffrecProvider forceEnableParam>
<RiffrecRecorder
startLabel="Record this issue"
stopLabel="Stop and save"
onSessionComplete={(result) => {
console.log(result?.filesPresent);
}}
/>
</RiffrecProvider>Before recording starts, the component explains that Riffrec records screen video, microphone audio, clicks, navigation, network URLs/statuses, and console errors. The user must check a consent box before browser capture prompts open. While recording, the provider-level stop control stays fixed above the page and saves the session zip when clicked.
For custom consent copy, pass consentTitle, consentDescription, or consentLabel.
Sessions are named riffrec-{YYYY-MM-DD}-{HHMM}-{shortid}. A React package zip contains:
session.json
events.json
recording.webm
voice.webm # when microphone narration was captured
Live sessions add transcript.json, units.json, annotations.json, frames/, clips/, and further recording-NNN.webm segments; see Live Mode.
The experimental desktop preview uses the same core session format and may additionally include context.json and notes.md.
session.json records URL, React version, browser, start/end timestamps, duration, and files_present. Consumers should use files_present, and for experimental desktop sessions context.json.capture_outcomes, rather than assuming optional media or text files exist.
events.json has schema_version: "1.0.0" and event records for clicks, network requests, console errors, and navigation. Click events include production-safe DOM context such as readable element names, selectors, class names, accessibility labels, nearby text, sibling context, bounding boxes, and a small computed-style snapshot. Credential-like query parameters such as token, api_key, and client_secret are redacted. Request and response bodies are not captured.
Experimental desktop zips keep the same events.json schema. context.json records desktop capture options and outcomes, app/browser versions, initial/final page information, captured viewport dimensions, marker timestamps, and unavailable signal disclosures.
| Feature | Chrome/Arc/Brave | Firefox | Safari |
|---|---|---|---|
| Screen recording | Yes | Yes | Partial |
| Microphone recording | Yes | Yes | Yes |
| Automatic zip download | Yes | Yes | Yes |
Riffrec downloads a zip automatically through the browser download flow instead of asking the user to choose a folder. Large recording.webm files over 50MB are excluded from the zip; session.json and events.json are still included.
Caution
Riffrec Desktop is a rough, highly experimental preview. It may break, change without notice, or disappear. It is not the recommended way to use Riffrec; use the React package above for the primary integration path.
The desktop preview is a standalone macOS feedback browser for trying Riffrec on a website that has not installed the React package. It trades the React integration's deeper in-app context for a contained experimental browser that can export a compatible session zip.
Experimental download for macOS (Apple Silicon): Riffrec Desktop Preview .dmg
The preview is development-signed and unnotarized. On first launch, right-click Riffrec.app and choose Open. Do not rely on it for production or durable workflows.
cd desktop
npm install
npm run package
open out/Riffrec-darwin-$(test "$(uname -m)" = arm64 && echo arm64 || echo x64).dmgThe app packages for the current Mac architecture. It requires macOS Screen Recording permission to record its browser window, and Microphone permission only when narration is enabled.
npm run package produces local .dmg and .zip development builds. To sign the app and disk image with a certificate installed through Xcode or Keychain Access, set RIFFREC_CODESIGN_IDENTITY:
RIFFREC_CODESIGN_IDENTITY="Apple Development: Your Name (TEAMID)" npm run package- Enter an
https://website URL in the address bar. Localhttp://localhostURLs are supported for development feedback. - Sign in or navigate inside Riffrec's isolated browser profile if required.
- Choose microphone and click capture, add optional reviewer notes, acknowledge the recording disclosure, and press Start recording.
- Reproduce the issue, add moment markers where useful, then press Stop and save session.
- Share the saved zip with an agent or teammate.
Desktop sessions capture the webpage loaded inside Riffrec: screen video, optional microphone audio, DOM click element details, top-level navigation, network URLs/methods/statuses/durations, console errors, notes, and capture context. They do not capture activity in an existing Safari/Chrome/Arc tab, request or response bodies, typed values, or reliable internal React component names on third-party sites.
Riffrec stores website cookies and local storage only in its dedicated local browser profile so authenticated reproductions work. It stages in-progress and unsaved recording media locally for crash recovery; interrupted or damaged drafts remain on this Mac until saved or deleted with the recovery-data action in the app. Use Clear website sign-in data after recording on sensitive sites.
Riffrec is development tooling. It can record anything visible on screen and anything spoken into the microphone. The React integration excludes password and hidden input values. In live mode, microphone audio and the consumer's session brief go to OpenAI Realtime, and transcript, units, strokes, frames, and events go to the configured endpoint as they happen; the consent step names both destinations, screenshots and frames exclude nothing automatically, and the riffer can mute the microphone or pause frames and the stream at any time. The experimental desktop preview excludes text inside form fields and editable controls from DOM click evidence. Screen video and microphone audio can still contain sensitive content.
Uninstrumented production sessions still include rich DOM context. Production React component names are only reliable when elements include data-component. React Fiber names are useful in development but often minified in production. A future riffrec-babel-plugin package can automate production component attributes.
The experimental desktop preview loads remote pages in an Electron browser surface with Node integration disabled, context isolation and sandboxing enabled, and unnecessary website permission requests and downloads denied. Credential-like URL query or fragment parameters are redacted from captured evidence. Recordings and recovery drafts remain local until the person recording exports or deletes them; only an exported zip is intended for sharing.
React and React DOM are peer dependencies and are externalized from the package bundle. fflate powers zip downloads. perfect-freehand (MIT, ~4.5 KB minified) draws the live drawing layer and lives only in the lazily loaded live chunk; hosts that never set live do not download it.