feat(google): add serviceTier option to LLM#1428
Conversation
Ports livekit/agents#5680 to add a `serviceTier` option (e.g. `ServiceTier.PRIORITY`) that is passed through to the Gemini GenerateContentConfig.
🦋 Changeset detectedLatest commit: 79da356 The changes in this PR will be included in the next version bump. This PR includes changesets to release 31 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79da356374
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| geminiTools?: LLMTools; | ||
| httpOptions?: types.HttpOptions; | ||
| seed?: number; | ||
| serviceTier?: types.ServiceTier; |
There was a problem hiding this comment.
This option depends on SDK support that was only added in @google/genai 1.47.0 (“Add service tier to GenerateContent”), but plugins/google/package.json still allows ^1.44.0. In projects that already resolve or override @google/genai to 1.44–1.46, this exported type references types.ServiceTier that does not exist, and the older SDK will not know how to forward the new config field. Please bump the dependency floor alongside adding this option so the advertised API is actually available wherever the plugin installs.
Useful? React with 👍 / 👎.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
4 participants
Summary
Ports livekit/agents#5680 from the Python
livekit-agentsrepo intoagents-js.This adds a new
serviceTieroption to the Google GeminiLLM, allowing users to opt into the Gemini API's service tiers (e.g.ServiceTier.PRIORITYfor priority inference).What changed
serviceTier?: types.ServiceTierfield onLLMOptions(plugins/google/src/llm.ts).#opts).GenerateContentConfigpassed toclient.models.generateContentStream({ config })viaextras.serviceTier.Implementation notes / parity with Python
Closely mirrors the Python implementation in
livekit-plugins-google/llm.py:service_tier: NotGivenOr[types.ServiceTier]on_LLMOptionsserviceTier?: types.ServiceTieronLLMOptionsif is_given(self._opts.service_tier): extra["service_tier"] = self._opts.service_tierif (this.#opts.serviceTier !== undefined) { extras.serviceTier = this.#opts.serviceTier; }Naming differences vs. Python (camelCase vs. snake_case) follow the existing JS plugin conventions:
serviceTierto match other agents-js plugins (openai,baseten,openai/ws,openai/responses) which already exposeserviceTier?: string.GenerateContentConfigisserviceTier(camelCase) in@google/genai≥ 1.50, while the Pythongoogle-genaiSDK acceptsservice_tier. The locked version inpnpm-lock.yaml(@google/genai@1.50.1) introduces this field, and the package range (^1.44.0) resolves to a compatible version.types.ServiceTierfrom@google/genai(rather than the loosestringtyping used by other plugins) since the enum is exported by the SDK, matching how the Python upstream usestypes.ServiceTier.No behavioural changes when
serviceTieris not provided.Changeset
Added
.changeset/google-llm-service-tier.mdas apatchchange to@livekit/agents-plugin-google.Test plan
pnpm build:agentsandpnpm --filter @livekit/agents-plugin-google buildsucceed (the plugin's own source compiles cleanly; remaining errors in the build output are in unrelatedbeta/realtime/realtime_api.tsand unbuilt test-only deps, not introduced by this PR).pnpm prettier --checkpasses for the changed files.pnpm --filter @livekit/agents-plugin-google lintproduces no new warnings (3 pre-existing warnings inbeta/realtime/realtime_api.tsonly).This PR was generated by an automated Claude Code Routine created by @toubatbrian, currently in experimentation. cc @toubatbrian @livekit/agent-devs for review.
Closes parity gap with livekit/agents#5680.
Generated by Claude Code