Simulation and Evaluation Framework for LLM-Powered Autonomous Trading Agents in a Synthetic Market Environment
A sophisticated multi-agent trading system that leverages Large Language Models (LLMs) to simulate autonomous trading agents in a realistic synthetic market. The system demonstrates advanced agent coordination, risk management, and decision-making capabilities powered by state-of-the-art AI models.
- Multi-Agent System: 5+ specialized agents working in coordination (Traders, Research Analysts, Risk Manager, Market Monitor, Coordinator)
- 5-Stage Decision Pipeline: Information Gathering → Analysis → Strategy Formation → Risk Assessment → Execution
- Real-time Communication: Async message passing between agents via CommunicationHub
- Intelligent Coordination: Coordinator agent orchestrates multi-agent decision-making
- Multiple Provider Support: OpenAI, Anthropic, and OpenRouter (100+ models)
- Flexible Configuration: Easy switching between providers and models
- Mock Mode: Test and develop without API costs
- Structured Output: JSON-based responses for reliable agent decisions
- Realistic Market Dynamics: Stochastic price movements with configurable volatility
- Correlated Assets: Sector-based correlation matrices for realistic inter-asset relationships
- Market Events: Random and scheduled events affecting prices and volatility
- Intraday Patterns: Opening/closing volatility spikes and midday lulls
- Volume Modeling: Liquidity-based volume generation
- Real-time P&L Tracking: Position-level and portfolio-level profit/loss monitoring
- Risk Constraints: Configurable position limits, stop-losses, and risk parameters
- Multi-asset Support: Trade multiple correlated assets simultaneously
- Performance Analytics: Sharpe ratio, drawdown, win rate calculations
- Type Safety: Full type hints with mypy compatibility
- Async/Await: Non-blocking concurrent operations
- Clean Configuration: TOML for settings, .env for secrets only
- Rich Terminal UI: Beautiful, real-time display of simulation progress
- Comprehensive Testing: pytest test suite with async support
llm_trading_agents/
├── src/ # Core application code
│ ├── agents/ # Agent implementations
│ │ ├── base_agent.py # Base agent class with decision pipeline
│ │ ├── trader_agent.py # Trading execution agent
│ │ ├── research_analyst.py # Market research and analysis
│ │ ├── risk_manager.py # Risk assessment and limits
│ │ ├── market_monitor.py # Market condition monitoring
│ │ └── coordinator.py # Multi-agent coordination
│ ├── agent_manager.py # Agent lifecycle management
│ ├── market_simulator.py # Synthetic market with GBM dynamics
│ ├── data_services.py # Market data management
│ ├── portfolio_manager.py # Portfolio tracking and P&L
│ ├── communication_hub.py # Inter-agent messaging
│ ├── llm_client.py # LLM provider abstraction
│ ├── config.py # Configuration management
│ └── terminal_display.py # Rich-based UI
├── tests/ # Unit tests
├── config/ # Configuration files
│ ├── settings.toml # Non-secret configuration
│ └── settings.local.toml # Local overrides (gitignored)
├── docs/ # Documentation
│ ├── IMPLEMENTATION.md # Implementation details
│ ├── OPEN_ROUTER_GUIDE.md # OpenRouter setup guide
│ └── markdown/ # Additional documentation
├── examples/ # Example scripts
│ └── llm_example.py # LLM integration examples
├── main.py # Main simulation runner
├── run.py # Quick start script
├── test_llm.py # LLM configuration test
├── check_models.py # Available models checker
└── pyproject.toml # Project dependencies
- Python 3.10 or higher
- uv (recommended) or pip
Using uv (recommended - fast and modern):
# Windows PowerShell
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
# macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install dependencies
uv sync
# With all extras (development tools + LLM SDKs)
uv sync --all-extrasUsing pip:
pip install -r requirements.txt- Copy the environment template:
# Windows
Copy-Item .env.example .env
# macOS/Linux
cp .env.example .env- Add your API key to
.env:
# For OpenRouter (recommended - access to 100+ models)
LLM__OPENROUTER_API_KEY=sk-or-v1-your_key_here
# OR for OpenAI
LLM__OPENAI_API_KEY=sk-your_key_here
# OR for Anthropic
LLM__ANTHROPIC_API_KEY=sk-ant-your_key_here- Configure non-secret settings in
config/settings.toml:
[llm]
provider = "openrouter" # mock | openrouter | openai | anthropic
openrouter_model = "google/gemini-2.0-flash-exp:free"
timeout = 60
[simulation]
rounds = 10
trading_symbols = ["AAPL", "GOOGL", "MSFT", "TSLA", "AMZN"]
[portfolio]
initial_cash = 1000000.0
max_position_size = 0.2 # 20% of portfolio# Quick start with run.py
uv run python run.py
# Or run main.py directly
uv run python main.py
# Test your LLM configuration
uv run python test_llm.py
# Check available OpenRouter models
uv run python check_models.pyOpenRouter (Recommended)
- Access to 100+ models from various providers
- Free tier available with some models
- Get API key: https://openrouter.ai/keys
OpenAI
- GPT-4, GPT-3.5-turbo models
- Get API key: https://platform.openai.com/api-keys
Anthropic
- Claude 3 (Opus, Sonnet, Haiku)
- Get API key: https://console.anthropic.com/
Mock (No API Key)
- Use for testing and development
- Set
provider = "mock"in settings.toml
| File | Purpose | Version Control |
|---|---|---|
config/settings.toml |
Non-secret settings (models, simulation params) | ✅ Committed |
config/settings.local.toml |
Local overrides | ❌ Gitignored |
.env |
API keys and secrets only | ❌ Gitignored |
.env.example |
Template for .env | ✅ Committed |
# Run all tests
uv run pytest
# With coverage report
uv run pytest --cov=src --cov-report=html
# Run specific test file
uv run pytest tests/test_agents.py -v
# Run tests matching a pattern
uv run pytest -k "trader" -v# Format code
uv run black src/ tests/
# Sort imports
uv run isort src/ tests/
# Lint
uv run ruff check src/ tests/
# Type check
uv run mypy src/# Install with dev dependencies
uv sync --extra dev
# Install all extras
uv sync --all-extrasThe market simulator uses a stochastic per-minute model with the following components:
- Base Returns: Independent normal samples with mean 0 and std 0.003 per minute
- Correlation: Sector-based correlation matrix applied via Cholesky decomposition
- Intraday Patterns:
- Opening/closing: 1.5× volatility
- Midday: 0.7× volatility
- Market Events: Random events with severity and duration affecting prices
- Price Updates: Arithmetic updates with prices floored at $0.01
- Volume: Generated from liquidity, return magnitude, and random noise
- Market Regime: Evolving sentiment and volatility via random walk
Migration to GBM/log-return formulation is planned for more realistic price dynamics:
S_t = S_{t-1} * exp((μ - 0.5σ²)Δt + σ√Δt·Z)
where Z ~ N(0,1)
Benefits:
- Enforces non-negative prices naturally
- Multiplicative returns easier to calibrate
- Better composition over time
- Jump-diffusion support for events
Each agent follows a 5-stage decision pipeline:
- Information Gathering: Collect relevant market data and context
- Analysis: Process information and identify patterns/signals
- Strategy Formation: Develop trading strategy based on analysis
- Risk Assessment: Evaluate risks and apply constraints
- Execution: Make final trading decision
Stages can be run sequentially or in parallel depending on agent configuration.
┌────────────────────────────────────────────────────────────────┐
│ LLM Trading Agents Simulation │
└────────────────────────────────────────────────────────────────┘
Round 1/10 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:05
┌──────────────────────── Portfolio Summary ────────────────────────┐
│ Cash: $1,000,000.00 │ Total Value: $1,000,000.00 │
│ Positions: 0 │ P&L: $0.00 (0.00%) │
└───────────────────────────────────────────────────────────────────┘
┌──────────────────────── Agent Decisions ───────────────────────────┐
│ Trader_1 │ BUY AAPL │ 100 shares @ $180.50 │ Conf: 85% │
│ Trader_2 │ HOLD │ - │ Conf: 60% │
│ Risk Manager │ APPROVED │ Position within limits│ Conf: 95% │
└────────────────────────────────────────────────────────────────────┘
- Multi-agent architecture with specialized roles
- LLM integration (OpenAI, Anthropic, OpenRouter)
- Modern configuration with pydantic-settings v2
- Async agent communication
- Rich terminal UI
- Comprehensive test suite
- Portfolio and risk management
- Coordinator agent for multi-agent orchestration
- Live market data integration (Alpha Vantage, Yahoo Finance)
- Advanced visualization dashboard (Plotly, Streamlit)
- GBM-based market dynamics
- Machine learning market prediction
- Multi-market support (crypto, forex)
- Real-time risk monitoring dashboard
- Performance backtesting engine
- Database integration (PostgreSQL, TimescaleDB)
- REST API for external integration
- WebSocket real-time updates
- Docker containerization
- Kubernetes deployment
Contributions are welcome! Please feel free to submit a Pull Request.
- Fork the repository
- Create your feature branch (
git checkout -b feature/AmazingFeature) - Commit your changes (
git commit -m 'Add some AmazingFeature') - Push to the branch (
git push origin feature/AmazingFeature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.
- Built with modern Python best practices
- Powered by state-of-the-art LLM providers
- Inspired by autonomous trading research
- Rich terminal UI by Textualize
For questions, issues, or suggestions:
- Open an issue on GitHub
- Check the documentation
- Review example scripts in examples/
Note: This is a simulation framework for research and educational purposes. Not intended for live trading with real money.
Steps:
- Copy and edit secrets
- Windows PowerShell: Copy-Item .env.example .env
- macOS/Linux: cp .env.example .env
Then open .env and set only your secret(s):
# If using OpenRouter (recommended default)
LLM__OPENROUTER_API_KEY=sk-or-v1-...
# If using OpenAI / Anthropic instead
# LLM__OPENAI_API_KEY=sk-...
# LLM__ANTHROPIC_API_KEY=sk-ant-...
- Choose provider and adjust non-secrets
Edit config/settings.toml (or create config/settings.local.toml) and set:
[llm]
provider = "openrouter" # mock | openrouter | openai | anthropic
openrouter_model = "google/gemini-2.5-flash-lite-preview-09-2025"
openrouter_site_url = "https://example.com"
openrouter_app_name = "LLM Trading Agents"
[simulation]
rounds = 5
trading_symbols = ["AAPL", "GOOGL", "MSFT", "TSLA", "AMZN"]
Notes:
- Provider is configured in TOML, not in .env.
- Only API keys belong in .env (nested names like LLM__OPENROUTER_API_KEY).
- Local overrides file config/settings.local.toml is ignored by git and merged at load time.
Typical entrypoint is main.py, which will:
- Load config from TOML + .env via AppConfig
- Initialize the LLM client (falls back to mock if a key is missing)
- Spin up the market simulator, agents, and run N rounds
Minimal code examples:
from src.config import get_config
from src.llm_client import create_llm_client
config = get_config()
# Create an LLM client based on provider in config; secrets come from .env
llm = create_llm_client(config)
# Access other settings
print(config.simulation.rounds)
print(config.llm.provider)What’s done (Oct 2025):
- Centralized configuration with pydantic-settings v2
- Secrets in .env only; non-secrets in TOML (config/settings.toml)
- Local overrides via config/settings.local.toml (gitignored)
- OpenRouter integration with HTTP-Referer and X-Title headers
- Rich-based terminal UI dependency declared
- main.py reads rounds/symbols from config
- Safe fallback to MockLLM if provider key is missing
Recommended next steps:
- Add Dockerfile and compose with secrets mounts
- Wire CI (lint, typecheck, tests)
- Optional FastAPI endpoint to run simulations headlessly
- Config schema docs and validation tests
### Run Tests
```bash
# Run all tests
uv run pytest
# Run with coverage
uv run pytest --cov=src --cov-report=html
# Run specific test file
uv run pytest tests/test_agents.py -v
# Run tests matching a pattern
uv run pytest -k "trader" -v
# Format code with Black
uv run black src/ tests/
# Sort imports with isort
uv run isort src/ tests/
# Lint with Ruff (modern, fast linter)
uv run ruff check src/ tests/
# Type check with mypy
uv run mypy src/The project follows clean code principles with:
- Type hints throughout
- Async/await for concurrent operations
- Comprehensive logging
- Unit test coverage
- Black code formatting
- Import sorting with isort
- LLM integration (OpenAI, Anthropic) - Completed Oct 2025
- Modern configuration management - Completed Oct 2025
- Pydantic v2 compatibility - Completed Oct 2025
- Live market data feeds (Alpha Vantage, Yahoo Finance)
- Advanced visualization dashboard (Plotly, Streamlit)
- LangChain integration for advanced prompting
- Machine learning-based market prediction
- Multi-market support (stocks, crypto, forex)
- Real-time risk monitoring dashboard
- Performance analytics and backtesting engine
- Database integration (PostgreSQL, TimescaleDB)
- REST API for external integration
- WebSocket support for real-time updates
- Docker containerization
- Kubernetes deployment configurations
MIT License - see LICENSE file for details.