Describe the Problem
We currently do not have access for getting token estimates of prompts (including functions) making RAG / histories hard to fit in context windows or brittle due to unfit estimates.
Propose a Solution
Base implementation of sdk chat model on @langchain/oneai models.
As they will handle this by default.
Describe Alternatives
Duplicating token counting code and making sure current community chains function correctly by extending /redoing them if they depend on calling these token functions.
oneai or library (Junjie: link to main branch might be changed, here is the actual link)
Affected Development Phase
Development
Impact
Impaired
Timeline
Next months
Additional Context
No response
Describe the Problem
We currently do not have access for getting token estimates of prompts (including functions) making RAG / histories hard to fit in context windows or brittle due to unfit estimates.
Propose a Solution
Base implementation of sdk chat model on
@langchain/oneaimodels.As they will handle this by default.
Describe Alternatives
Duplicating token counting code and making sure current community chains function correctly by extending /redoing them if they depend on calling these token functions.
oneai or library (Junjie: link to main branch might be changed, here is the actual link)
Affected Development Phase
Development
Impact
Impaired
Timeline
Next months
Additional Context
No response