Skip to content

Add Gemini realtime non-blocking tool scheduling#1412

Merged
toubatbrian merged 7 commits intomainfrom
brian/google-non-blocking-tool
May 8, 2026
Merged

Add Gemini realtime non-blocking tool scheduling#1412
toubatbrian merged 7 commits intomainfrom
brian/google-non-blocking-tool

Conversation

@toubatbrian
Copy link
Copy Markdown
Contributor

Summary

  • Adds toolBehavior and toolResponseScheduling support to the Google Gemini realtime model.
  • Sends willContinue: true for non-blocking tool calls, then willContinue: false with the final tool result.
  • Applies scheduling values (SILENT, WHEN_IDLE, INTERRUPT) to both continuing and final tool responses.
  • Updates the Gemini realtime example with env-driven testing knobs for tool behavior, scheduling, and slow tool execution.

Test Plan

  • Ran targeted regression test: pnpm test -- plugins/google/src/beta/realtime/realtime_api.test.ts
  • Ran plugin lint: pnpm --filter @livekit/agents-plugin-google lint
  • Manually verified Gemini API behavior for NON_BLOCKING with:
    • TOOL_SCHEDULING=SILENT
    • TOOL_SCHEDULING=WHEN_IDLE
    • TOOL_SCHEDULING=INTERRUPT
      Everything is tested and working properly

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 7, 2026

🦋 Changeset detected

Latest commit: 9613b2b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 29 packages
Name Type
@livekit/agents-plugin-google Patch
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch
@livekit/agents-plugins-test Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@toubatbrian toubatbrian changed the title Brian/google non blocking tool Add Gemini realtime non-blocking tool scheduling May 7, 2026
@toubatbrian toubatbrian requested a review from a team May 7, 2026 05:26
devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Member

@theomonnom theomonnom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@toubatbrian toubatbrian merged commit 7ab3aee into main May 8, 2026
6 checks passed
@toubatbrian toubatbrian deleted the brian/google-non-blocking-tool branch May 8, 2026 00:53
@github-actions github-actions Bot mentioned this pull request May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants