Skip to content

Port dynamic endpointing to Node agents#1316

Open
u9g wants to merge 3 commits intomainfrom
rosetta/issue-118-openai-gpt-55-xhigh
Open

Port dynamic endpointing to Node agents#1316
u9g wants to merge 3 commits intomainfrom
rosetta/issue-118-openai-gpt-55-xhigh

Conversation

@u9g
Copy link
Copy Markdown
Contributor

@u9g u9g commented Apr 25, 2026

This PR was created by Rosetta.

Tracking issue: https://github.com/livekit/rosetta/issues/118

Summary

  • Ports the Python SDK dynamic endpointing state machine to the Node voice turn handling runtime.
  • Reuses and extends the shared ExpFilter helper, wires endpointing through VAD, STT, realtime, and agent-speech lifecycle paths.
  • Ports the source endpointing parity suite and adds target integration regressions for recognition and AgentActivity seams.

Source

Validation

  • pnpm test agents/src/voice/turn_config/endpointing.test.ts agents/src/voice/audio_recognition_endpointing.test.ts agents/src/voice/agent_activity.test.ts agents/src/voice/audio_recognition_handoff.test.ts agents/src/voice/audio_recognition_span.test.ts
  • pnpm test agents/src/utils.test.ts agents/src/voice/turn_config/utils.test.ts agents/src/voice/agent.test.ts
  • pnpm test agents/src/voice/agent_activity_handoff.test.ts agents/src/voice/interruption_detection.test.ts agents/src/voice/agent_session_handoff.test.ts
  • pnpm --filter @livekit/agents build
  • pnpm --filter @livekit/agents-plugin-silero build
  • pnpm exec eslint -f unix "agents/src/**/*.ts" --quiet
  • pnpm exec prettier --check ...touched files...

Notes

  • pnpm --filter @livekit/agents api:check currently fails on the repository's existing API Extractor limitation for export * as ___ syntax in dist/index.d.ts.

Rosetta Bot added 3 commits April 25, 2026 12:28
end-of-utterance prediction.
Edited comment; dynamic endpointing now accounts for speech pauses as well as EOU delay selection.

/** Maximum endpointing delay in milliseconds. */
Edited in place; AudioRecognition now takes an endpointing object so dynamic state can be preserved.

/** Minimum endpointing delay in milliseconds. */
Edited in place; AudioRecognition now takes an endpointing object so dynamic state can be preserved.

apply(exp: number, sample: number): number;
Kept as an overload; existing ExpFilter callers remain source-compatible.

constructor(alpha: number, max?: number);
Kept as an overload; existing ExpFilter callers remain source-compatible.

maxEndpointingDelay: number;
Edited in place; all AudioRecognition constructors in this repository were updated to pass endpointing.

minEndpointingDelay: number;
Edited in place; all AudioRecognition constructors in this repository were updated to pass endpointing.

onEndOfAgentSpeech(ignoreUserTranscriptUntil: number): Promise<void>;
Edited in place; callers do not await it and endpointing needs synchronous state updates.

onEndOfOverlapSpeech(endedAt: number, userSpeakingSpan?: Span): Promise<boolean | undefined>;
Edited in place; callers do not await it and endpointing state is updated through onEndOfSpeech.

onStartOfAgentSpeech(): Promise<boolean>;
Edited in place; all callers now pass the source-required startedAt timestamp.

onStartOfOverlapSpeech(speechDuration: number, startedAt: number, userSpeakingSpan?: Span): Promise<void>;
Edited in place; callers do not await it and endpointing start handling moved to onStartOfSpeech.

private maxEndpointingDelay;
Edited in place; fixed delay fields were replaced by the endpointing state machine.

private minEndpointingDelay;
Edited in place; fixed delay fields were replaced by the endpointing state machine.

reset(alpha?: number): void;
Kept as an overload; existing ExpFilter callers remain source-compatible.

turnDetection: TurnDetectionMode | undefined;
Edited in place; updateOptions now preserves NotGiven semantics by checking property presence.
* end-of-utterance prediction.
Edited in place; dynamic mode now preserves the existing EOU wording and adds pause-adaptation detail.
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 25, 2026

🦋 Changeset detected

Latest commit: 2e47fd8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 26 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Rosetta Bot seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 6 additional findings.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants