-
Notifications
You must be signed in to change notification settings - Fork 12
Added offline token cost estimation with hard-isolated execution path #72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Added offline token cost estimation with hard-isolated execution path #72
Conversation
|
Thanks for the suggestion! litellm is great but it's a pretty heavy dependency just for pricing lookups - it pulls in a lot of LLM client code we don't need. Found a lighter option: https://github.com/pydantic/genai-prices Or we could just vendor their slim JSON directly. Either way, we should add a disclaimer to the CLI output warning users that cost estimates are approximate and may not reflect current pricing. What do you think? |
|
I see, that makes sense. genai-prices looks like a good fit and avoids pulling unnecessary client code I kept pricing static initially to stay strictly within the original issue scope. |
Sounds good, I’m aligned. Thanks for the thoughtful work on this. |
|
Updated the PR to use genai-prices for dynamic pricing and added a clear disclaimer that estimates are approximate |
A bit late but will do! |
|
|
ff774ae to
87fa619
Compare
Implements
--estimate-costto provide an offline, pre-flight estimate of token usage and cost.Closes #57