Skip to content

Add example: multi-provider comparison (same task, different LLMs) #9

@rajkumar42

Description

@rajkumar42

Summary

Create an example that runs the same agent task across multiple LLM providers (Claude, OpenAI, Gemini) and compares the outputs side-by-side. Demonstrates provider-agnostic design and helps users evaluate which provider works best for their use case.

What needs to happen

  • Create multi_provider_comparison/ directory
  • Define a single task (e.g., "summarize this article" or "extract key entities")
  • Run the task against Claude, OpenAI, and Gemini
  • Collect and display results side-by-side (output, tokens, latency, cost)
  • Add README explaining setup (API keys) and how to run

Example output

Task: "Summarize the key points of this earnings report"

| Provider | Tokens (in/out) | Latency | Output preview           |
|----------|----------------|---------|--------------------------|
| Claude   | 1200/350       | 2.1s    | "Revenue grew 15%..."    |
| OpenAI   | 1180/420       | 1.8s    | "The company reported..." |
| Gemini   | 1250/380       | 2.4s    | "Key highlights..."      |

Acceptance criteria

  • Runs against at least 2 providers (3 if all API keys available)
  • Output comparison is clear and useful
  • Gracefully handles missing API keys (skips that provider with a message)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions