Feature Request: Add Explicit Context Caching Support for Gemini Models

[hsjun99]

Problem

Current implicit caching for Gemini models doesn't guarantee caching, even when exceeding minimum token thresholds (1,024 tokens for 2.5 Flash, 2,048 tokens for 2.5 Pro). This is the observed behavior after thorough testing.

Solution

Add explicit context caching support using Gemini API's caching feature:

• Cache Creation: Create caches with configurable TTL
• Guaranteed Savings: Predictable caching mechanism

[Mohd162004]

Hey, just wanted to ask, how did you do implicit caching for the gemini model? and was it the live api model? or was it the STT - LLM - TSS pipeline that you used?

[exponentiallyio]

Any update on this please? Gemini Implicit Caching is still broken - I tested through the gemini API