LLM Tracer

An observability platform for LangChain and LangGraph applications. Capture traces, token usage, costs, and performance metrics from your LLM workflows.

Quick Start

Option 1: Docker Compose (Recommended)

# Clone with submodules
git clone --recurse-submodules git@gitlab.aitester.lab:llm-tracer/llm-tracer-main.git
cd llm-tracer-main

# Copy environment config
cp .env.example .env

# Start services
docker-compose up -d

Access:

• Web UI: http://localhost:3000
• API: http://localhost:8080
• API Docs: http://localhost:8080/docs

Option 2: Manual Setup

1. Start the API Server

cd api-server
pip install -e ".[dev]"
uvicorn api.main:app --host 0.0.0.0 --port 8080

2. Start the Web Frontend

cd web-frontend
npm install
npm run dev

3. Create an Application and API Key

Via API docs at http://localhost:8080/docs:

1. POST /api/applications - Create an application
2. POST /api/applications/{app_id}/api-keys - Create an API key
3. Save the key (it's only shown once)

4. Instrument Your Application

pip install -e client-sdk/

from langchain_openai import ChatOpenAI
from llm_tracer import LLMTraceCallback

callback = LLMTraceCallback(
    endpoint="http://localhost:8080",
    api_key="pt_your_api_key",
    component="MyAgent"
)

llm = ChatOpenAI(model="gpt-4")
response = llm.invoke("Hello!", config={"callbacks": [callback]})

callback.flush()

Project Structure

llm-tracer-main/
├── api-server/      # FastAPI backend service
├── client-sdk/      # Python SDK for instrumenting applications
├── web-frontend/    # React web interface
└── docs/            # User documentation

Components

Component	Description	Port
api-server	REST API for trace ingestion and queries	8080
client-sdk	LangChain/LangGraph callback handler	-
web-frontend	Browser-based trace viewer	3000/5173
docs	Installation and usage guides	-

Documentation

• Getting Started
• Client SDK User Guide - Complete SDK reference
• API Server User Guide - API endpoints
• Web Frontend User Guide - Using the UI
• Developer Notes - Architecture details

Client SDK Quick Reference

Environment Variables

export LLM_TRACER_API_KEY=pt_your_api_key
export LLM_TRACER_ENDPOINT=http://localhost:8080
export LLM_TRACER_COMPONENT=my-agent

Basic Usage

from llm_tracer import LLMTraceCallback

# Context manager (recommended)
with LLMTraceCallback(component="MyAgent") as callback:
    response = llm.invoke("Hello", config={"callbacks": [callback]})

# Manual usage
callback = LLMTraceCallback(component="MyAgent")
response = llm.invoke("Hello", config={"callbacks": [callback]})
callback.flush()
callback.stop()

Configuration Options

Parameter	Default	Description
api_key	Required	API key for authentication
endpoint	http://localhost:8080	API server URL
component	None	Component/agent name
batch_size	100	Events per batch
flush_interval	5.0	Seconds between flushes

See Client SDK User Guide for complete reference.

Features

• Zero Latency Impact: Background threading ensures tracing doesn't slow your app
• Automatic Hierarchy: Captures parent-child relationships between runs
• Token & Cost Tracking: Automatic extraction of token usage and cost calculation
• Application/Component Organization: Two-level hierarchy for organizing traces
• API Key Authentication: Secure trace ingestion with per-application keys
• Flexible Filtering: Filter traces by application, component, status, date range

Submodule Management

Pull latest changes:

git submodule update --remote --merge

Ensure submodules are on main branch:

git submodule foreach 'git checkout main'

If cloned without submodules:

git submodule update --init --recursive