Popular repositories Loading

1. token-aware-rate-limiter token-aware-rate-limiter Public

   Reference implementation of token-aware rate limiting for multi-tenant LLM APIs. Atomic Redis Lua token buckets, Jain's fairness index, real-time dashboard.

   Python

2. litellm litellm Public

   Forked from BerriAI/litellm

   Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthr…

   Python

3. Awesome-LLMOps Awesome-LLMOps Public

   Forked from tensorchord/Awesome-LLMOps

   An awesome & curated list of best LLMOps tools for developers

   Shell

4. llm-judge-calibrator llm-judge-calibrator Public

   Detect position bias, verbosity bias, and self-preference in LLM judges. Position-swap evaluation + Cohen's Kappa + calibration report.

   Python

5. FastChat FastChat Public

   Forked from lm-sys/FastChat

   An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

   Python

6. agent-eval agent-eval Public

   Compare coding agents head-to-head. Pass rate, cost, time, consistency — one command. No Docker required.

   Python