Pinned Loading

1. triton-inference-server triton-inference-server Public

   Forked from triton-inference-server/server

   The Triton Inference Server provides a cloud inferencing solution optimized for NVIDIA GPUs.

   C++

2. ai-dynamo/aiperf ai-dynamo/aiperf Public

   AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.

   Python 294 81

3. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 80k 16.8k

4. ai-dynamo/dynamo ai-dynamo/dynamo Public

   A Datacenter Scale Distributed Inference Serving Framework

   Rust 6.8k 1.1k