Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 46 additions & 9 deletions python/tracing/openai-agents-sdk/README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,56 @@
# OpenAI Agents SDK Respan Examples
# OpenAI Agents SDK tracing examples

These examples use the current `respan-instrumentation-openai-agents` package. Older files still import `respan_exporter_openai_agents.RespanTraceProcessor`; the local compatibility bridge maps that import to the active instrumentation processor and initializes the unified Respan OTEL exporter.
These examples exercise `respan-instrumentation-openai-agents` against OpenAI Agents SDK `0.20.x`. The suite includes deterministic current-framework traces for success, failure, function tools, handoffs, guardrails, and streaming, plus the upstream-style live examples.

## Setup
## Install

From this directory:

```bash
python -m pip install -r requirements.txt
```

For repository development, install the local packages after the registry dependencies so validation uses the current checkout:

```bash
cd python/tracing/openai-agents-sdk
pip install openai-agents respan-ai respan-instrumentation-openai-agents python-dotenv pytest pytest-asyncio
python -m pip install -e ../../../../respan/python-sdks/respan-sdk
python -m pip install -e ../../../../respan/python-sdks/respan-tracing
python -m pip install -e ../../../../respan/python-sdks/respan
python -m pip install -e ../../../../respan/python-sdks/instrumentations/respan-instrumentation-openai-agents
```

Use the repository root `.env` values. OpenAI Agents 0.17 uses the Responses API by default. Because the current Respan OpenAI gateway covers chat-compatible routes, the local bridge forces `chat_completions` and routes model calls to `RESPAN_GATEWAY_BASE_URL` when `RESPAN_GATEWAY_API_KEY` is present. Set `RESPAN_OPENAI_AGENTS_USE_OPENAI=1` to use a direct `OPENAI_API_KEY`/Responses run instead. Traces always go to `RESPAN_BASE_URL` with `RESPAN_API_KEY`.
The suite reads the repository `.env` without overriding variables supplied by the shell. Set `RESPAN_API_KEY` and `RESPAN_BASE_URL` for trace export. Gateway-compatible live examples use that Respan credential and Chat Completions route.

The compatibility bridge disables Respan's direct OpenAI auto-instrumentation because the explicit Agents trace processor owns these provider calls; this keeps each model call to one canonical chat span even when both instrumentation packages are installed.

Run a focused example:
Hosted tools are intentionally not converted to Chat Completions. Web Search and the research bot require `RESPAN_OPENAI_AGENTS_USE_OPENAI=1` plus a direct `OPENAI_API_KEY`. File Search additionally requires `OPENAI_VECTOR_STORE_ID`. Computer Use additionally requires `RESPAN_OPENAI_AGENTS_ENABLE_COMPUTER=1` and an installed Playwright Chromium runtime:

```bash
pytest basic/hello_world_test.py -q
pytest tools/functions_test.py -q
python -m playwright install chromium
```

When those settings are absent, only the affected hosted examples are reported as explicit skips.

## Run

Run every collected example and the three legacy direct-run demos under one exact marker:

```bash
RESPAN_EXAMPLE_RUN_ID=otel2-openai-agents-check python run_all.py
```

Run only deterministic structural coverage:

```bash
RESPAN_EXAMPLE_RUN_ID=otel2-openai-agents-contract \
python -m pytest contract_scenarios_test.py -q -s
```

Nested handoff files are directly executable without a manual `PYTHONPATH` adjustment:

```bash
python handoffs/message_filter_test.py
python handoffs/message_filter_streaming_test.py
```

Every pytest case flushes after completion, every process performs an explicit final Respan shutdown, and the same `RESPAN_EXAMPLE_RUN_ID` is attached to every emitted record.
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
from dotenv import load_dotenv

load_dotenv(override=True)
load_dotenv(override=False)

import asyncio
import os

import pytest
from agents import Agent, ItemHelpers, MessageOutputItem, Runner, trace
from agents.tracing import set_trace_processors

from respan_exporter_openai_agents import RespanTraceProcessor

set_trace_processors(
Expand Down Expand Up @@ -69,6 +71,7 @@
instructions="You inspect translations, correct them if needed, and produce a final concatenated response.",
)


@pytest.mark.asyncio
async def test_main():
msg = "english to spanish"
Expand All @@ -90,7 +93,6 @@ async def test_main():
print(f"\n\nFinal response:\n{synthesizer_result.final_output}")



if __name__ == "__main__":
# For interactive use
asyncio.run(test_main())
asyncio.run(test_main())
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
from dotenv import load_dotenv
load_dotenv(override=True)

load_dotenv(override=False)

# ==========copy past below==========
import asyncio
import os
from pydantic import BaseModel
import sys

import pytest
from agents import Agent, Runner, trace
from agents.tracing import set_trace_processors
from pydantic import BaseModel

from respan_exporter_openai_agents import RespanTraceProcessor

set_trace_processors(
Expand Down Expand Up @@ -75,13 +79,15 @@ async def test_main():
assert isinstance(outline_checker_result.final_output, OutlineCheckerOutput)
if not outline_checker_result.final_output.good_quality:
print("Outline is not good quality, so we stop here.")
exit(0)
sys.exit(0)

if not outline_checker_result.final_output.is_scifi:
print("Outline is not a scifi story, so we stop here.")
exit(0)
sys.exit(0)

print("Outline is good quality and a scifi story, so we continue to write the story.")
print(
"Outline is good quality and a scifi story, so we continue to write the story."
)

# 4. Write the story
story_result = await Runner.run(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
from __future__ import annotations

from dotenv import load_dotenv
load_dotenv(override=True)

load_dotenv(override=False)

import asyncio
import os
import pytest
from typing import Union
from pydantic import BaseModel

import pytest
from agents import (
Agent,
GuardrailFunctionOutput,
Expand All @@ -17,8 +17,11 @@
TResponseInputItem,
input_guardrail,
)
from respan_exporter_openai_agents import RespanTraceProcessor
from agents.tracing import set_trace_processors, trace
from pydantic import BaseModel

from respan_exporter_openai_agents import RespanTraceProcessor

set_trace_processors(
[
RespanTraceProcessor(
Expand Down Expand Up @@ -57,7 +60,9 @@ class MathHomeworkOutput(BaseModel):

@input_guardrail
async def math_guardrail(
context: RunContextWrapper[None], agent: Agent, input: Union[str, list[TResponseInputItem]]
context: RunContextWrapper[None],
agent: Agent,
input: str | list[TResponseInputItem],
) -> GuardrailFunctionOutput:
"""This is an input guardrail function, which happens to call an agent to check if the input
is a math homework question.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
from __future__ import annotations

from dotenv import load_dotenv

load_dotenv(override=True)
load_dotenv(override=False)

import asyncio
import os
from typing import Literal, Union
import pytest
from pydantic import BaseModel
from typing import Literal

import pytest
from agents import Agent, ItemHelpers, Runner, TResponseInputItem, trace
from respan_exporter_openai_agents import RespanTraceProcessor
from agents.tracing import set_trace_processors
from pydantic import BaseModel

from respan_exporter_openai_agents import RespanTraceProcessor

set_trace_processors(
[
Expand Down Expand Up @@ -64,11 +65,11 @@ async def test_main() -> StoryEvaluationResult:
msg = "Sci fi"
input_items: list[TResponseInputItem] = [{"content": msg, "role": "user"}]

latest_outline: Union[str, None] = None
latest_outline: str | None = None
iterations = 0
max_iterations = 2
final_score = ""

# We'll run the entire workflow in a single trace
with trace("LLM as a judge"):
while True:
Expand All @@ -82,7 +83,9 @@ async def test_main() -> StoryEvaluationResult:
)

input_items = story_outline_result.to_input_list()
latest_outline = ItemHelpers.text_message_outputs(story_outline_result.new_items)
latest_outline = ItemHelpers.text_message_outputs(
story_outline_result.new_items
)

evaluator_result = await Runner.run(evaluator, input_items)
result: EvaluationFeedback = evaluator_result.final_output
Expand All @@ -91,15 +94,20 @@ async def test_main() -> StoryEvaluationResult:
if result.score == "pass":
break

input_items.append({"content": f"Feedback: {result.feedback}", "role": "user"})
input_items.append(
{"content": f"Feedback: {result.feedback}", "role": "user"}
)

return StoryEvaluationResult(
final_outline=latest_outline or "",
iterations=iterations,
final_score=final_score
final_score=final_score,
)


if __name__ == "__main__":
result = asyncio.run(test_main())
print(f"Final story outline after {result.iterations} iterations (score: {result.final_score}):")
print(
f"Final story outline after {result.iterations} iterations (score: {result.final_score}):"
)
print(result.final_outline)
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
from __future__ import annotations

from dotenv import load_dotenv

load_dotenv(override=True)
import pytest
load_dotenv(override=False)
import asyncio
import json
import os

from pydantic import BaseModel, Field

import pytest
from agents import (
Agent,
GuardrailFunctionOutput,
Expand All @@ -16,12 +16,12 @@
Runner,
output_guardrail,
)
from agents.tracing import set_trace_processors, trace
from pydantic import BaseModel, Field

from respan_exporter_openai_agents import (
RespanTraceProcessor,
)
from typing import Union
from agents.tracing import set_trace_processors, trace
import os

set_trace_processors(
[
Expand Down Expand Up @@ -50,7 +50,7 @@ class MessageOutput(BaseModel):
description="Thoughts on how to respond to the user's message"
)
response: str = Field(description="The response to the user's message")
user_name: Union[str, None] = Field(
user_name: str | None = Field(
description="The name of the user who sent the message, if known"
)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
import asyncio

from dotenv import load_dotenv

load_dotenv(override=True)
load_dotenv(override=False)
import os

import pytest
from agents import Agent, ItemHelpers, Runner, trace
from agents.tracing import set_trace_processors

from respan_exporter_openai_agents import (
RespanTraceProcessor,
)
from agents.tracing import set_trace_processors
import os

set_trace_processors(
[
Expand Down
22 changes: 15 additions & 7 deletions python/tracing/openai-agents-sdk/agent_patterns/routing_test.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
from dotenv import load_dotenv

load_dotenv(override=True)
import pytest
load_dotenv(override=False)
import asyncio
import os
import uuid

import pytest
from agents import Agent, RawResponsesStreamEvent, Runner, TResponseInputItem, trace
from agents.tracing import set_trace_processors
from openai.types.responses import ResponseContentPartDoneEvent, ResponseTextDeltaEvent

from agents import Agent, RawResponsesStreamEvent, Runner, TResponseInputItem, trace
from respan_exporter_openai_agents import (
RespanTraceProcessor,
)
from agents.tracing import set_trace_processors
import os

set_trace_processors(
[
Expand Down Expand Up @@ -55,8 +55,15 @@ async def test_main():
conversation_id = str(uuid.uuid4().hex[:16])

agent = triage_agent
inputs: list[TResponseInputItem] = [{"content": "Can you help me with my math homework?", "role": "user"}]
questions = ["Can you help me with my math homework?", "Yeah, how to solve for x: 2x + 5 = 11?", "What's the capital of France?", ""]
inputs: list[TResponseInputItem] = [
{"content": "Can you help me with my math homework?", "role": "user"}
]
questions = [
"Can you help me with my math homework?",
"Yeah, how to solve for x: 2x + 5 = 11?",
"What's the capital of France?",
"",
]

with trace("Routing example", group_id=conversation_id):
for question in questions:
Expand All @@ -83,5 +90,6 @@ async def test_main():
inputs.append({"content": question, "role": "user"})
agent = result.current_agent


if __name__ == "__main__":
asyncio.run(test_main())
Original file line number Diff line number Diff line change
@@ -1,20 +1,26 @@
from dotenv import load_dotenv

load_dotenv(override=True)
import pytest
import os
load_dotenv(override=False)
import asyncio
import os
import random
from typing import Literal

import pytest
from agents import Agent, RunContextWrapper, Runner
from agents.tracing import set_trace_processors

from respan_exporter_openai_agents import (
RespanTraceProcessor,
)
from agents.tracing import set_trace_processors

set_trace_processors(
[RespanTraceProcessor(os.getenv("RESPAN_API_KEY"), endpoint=os.getenv("RESPAN_OAIA_TRACING_ENDPOINT"))]
[
RespanTraceProcessor(
os.getenv("RESPAN_API_KEY"),
endpoint=os.getenv("RESPAN_OAIA_TRACING_ENDPOINT"),
)
]
)


Expand Down
Loading