feat: add ReasoningToken as LLMRequestCostType#1913
feat: add ReasoningToken as LLMRequestCostType#1913mtparet wants to merge 3 commits intoenvoyproxy:mainfrom
Conversation
dosubot
bot
added
the
size:L
aabchoo
left a comment
aabchoo
left a comment
There was a problem hiding this comment.
could you please add tests for translator?
|
I just added the tests @aabchoo |
Codecov Report❌ Patch coverage is ❌ Your patch status has failed because the patch coverage (67.50%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #1913 +/- ##
==========================================
- Coverage 84.33% 84.29% -0.04%
==========================================
Files 130 130
Lines 17987 18024 +37
==========================================
+ Hits 15170 15194 +24
- Misses 1873 1884 +11
- Partials 944 946 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
aabchoo
left a comment
aabchoo
left a comment
There was a problem hiding this comment.
Could you please address the failing tests?
manifests/charts/ai-gateway-crds-helm/templates/aigateway.envoyproxy.io_aigatewayroutes.yaml
Outdated
Show resolved
Hide resolved
Add ReasoningToken as a 7th cost type to enable cost-based routing and rate limiting for reasoning tokens (used by OpenAI o1/o3, Gemini with thoughts, Claude with extended thinking). Thread reasoning tokens through all layers: CRD API types, TokenUsage struct, OTel metrics, filter config, CEL evaluation, cost calculation, and translators (OpenAI chat/completions/responses, GCP Vertex AI). Signed-off-by: Matthieu Paret <mp@blackfuel.ai>
mtparet
force-pushed
the
add-reasoning-token-cost-type
branch
2 times, most recently
from
5a04beb to
cf58222
Compare
Signed-off-by: Matthieu Paret <mp@blackfuel.ai>
mtparet
force-pushed
the
add-reasoning-token-cost-type
branch
from
cf58222 to
175afcd
Compare
aabchoo
left a comment
aabchoo
left a comment
There was a problem hiding this comment.
In the future, could you please avoid force-pushing your changes? It is hard to track what has changed when the commit history is forcibly changed.
Thank you for the contribution!
|
Yes of course, thanks @aabchoo |
|
Could you please run |
3 participants
Description
Add ReasoningToken as a 7th cost type to enable cost-based routing and rate limiting for reasoning tokens (used by OpenAI o1/o3, Gemini with thoughts, Claude with extended thinking).