Skip to content

Conversation

@saksham-jain177
Copy link

Implements --estimate-cost to provide an offline, pre-flight estimate of token usage and cost.

  • Estimation runs in a fully isolated path before any provider initialization or network activity
  • Token counting is approximate and provider-agnostic, as specified in the issue
  • Cost lookup is best-effort and defaults to unavailable for unknown models
  • Includes unit tests for the estimator

Closes #57

@brammittendorff
Copy link
Contributor

brammittendorff commented Jan 12, 2026

Thanks for the suggestion! litellm is great but it's a pretty heavy dependency just for pricing lookups - it pulls in a lot of LLM client code we don't need.

Found a lighter option: genai-prices from Pydantic. It's specifically built for pricing data only. Covers 1000+ models across 29 providers, handles tiered pricing (Gemini), off-peak rates (DeepSeek), etc. No client bloat.

https://github.com/pydantic/genai-prices

Or we could just vendor their slim JSON directly.

Either way, we should add a disclaimer to the CLI output warning users that cost estimates are approximate and may not reflect current pricing.

What do you think?

@saksham-jain177
Copy link
Author

I see, that makes sense. genai-prices looks like a good fit and avoids pulling unnecessary client code
if you’re aligned, I can update the PR to use genai-prices (or the JSON) while adding a clear disclaimer that estimates are approximate

I kept pricing static initially to stay strictly within the original issue scope.

@brammittendorff
Copy link
Contributor

I see, that makes sense. genai-prices looks like a good fit and avoids pulling unnecessary client code if you’re aligned, I can update the PR to use genai-prices (or the JSON) while adding a clear disclaimer that estimates are approximate

I kept pricing static initially to stay strictly within the original issue scope.

Sounds good, I’m aligned.
Let’s use genai-prices directly, it’s small and keeps pricing updates centralized.
A clear disclaimer that prices are estimates is required.

Thanks for the thoughtful work on this.

@saksham-jain177
Copy link
Author

Updated the PR to use genai-prices for dynamic pricing and added a clear disclaimer that estimates are approximate
All changes stay strictly within the original scope and CI is clean could you give it a review when you get a chance ?

@brammittendorff
Copy link
Contributor

Updated the PR to use genai-prices for dynamic pricing and added a clear disclaimer that estimates are approximate All changes stay strictly within the original scope and CI is clean could you give it a review when you get a chance ?

A bit late but will do!

@brammittendorff
Copy link
Contributor

brammittendorff commented Jan 21, 2026

from put them in the top of the file "if you can" or else isort has no use in our pipelines.

@saksham-jain177 saksham-jain177 force-pushed the enhancement/estimate-cost branch from ff774ae to 87fa619 Compare January 21, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add flag to show estimated AI token usage before run

2 participants