Making Simulation SimIIR: A Dashboard for User Simulation

This repository contains the source code of SimIIR API wrapper for the paper Making Simulation SimIIR: A Dashboard for User Simulation.

---

Abstract

User simulation is a valuable methodology for evaluation in Information Retrieval. However, the steep learning curve and high setup time of existing frameworks often limit their use and make reproducing studies difficult. This paper introduces SimIIR Studio, a web-based workbench created to address these issues. The platform offers a visual, low-code interface for the SimIIR simulation engine, abstracting its underlying complexity. It allows researchers to replicate existing models, construct new simulations by combining components, and conduct comparative analyses within a unified environment. By reducing the implementation overhead, SimIIR Studio makes advanced user simulation more accessible and reproducible for the research community.

Keywords: User simulation, Simulation platforms, Reproducibility

---

SimIIR API - Quick Start Guide

Get started with the SimIIR API in 5 minutes!

Prerequisites

• Python 3.9+
• Poetry (for dependency management)
• Git

Installation

Step 1: Clone SimIIR Framework

First, clone the simIIR framework at the root level (same level as simiir-api):

cd /simiir-studio  # Or your project root
git clone https://github.com/simint-ai/simiir-3.git simiir

Your directory structure should look like:

simiir-studio/
├── simiir/          ← SimIIR framework (just cloned)
└── simiir-api/      ← This API wrapper

Step 2: Setup SimIIR Dependencies

SimIIR uses pip and requirements.txt:

cd simiir
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
deactivate
cd ../simiir-api

Note: The SimIIR API will automatically use the system Python to run simIIR since it doesn't have poetry.

Step 3: Install SimIIR API

Option A: Using the start script (Recommended)

./start.sh

This will automatically:

1. Install poetry (if needed)
2. Install dependencies
3. Create directories
4. Set up environment
5. Initialize database
6. Start the server

Option B: Manual setup

# Install dependencies
poetry install

# Set up environment
cp .env.example .env

# Edit .env to set SIMIIR_REPO_PATH if not using default location
nano .env

# Create directories
mkdir -p outputs logs

# Start server
poetry run simiir-api

First Simulation

1. Start the API

poetry run simiir-api

The API will be available at: http://localhost:8000

2. View API Documentation

Open your browser to: http://localhost:8000/docs

3. Create a Simulation

Option A: Using the test script with real simIIR config (Recommended)

poetry run python examples/test_with_real_config.py

This will:

• Read a real simIIR configuration from ../simiir/example_sims/trec_bm25_simulation.xml
• Create a simulation via the API
• Show you commands to start and monitor it

Option B: Using the Python example script

poetry run python examples/api_usage.py

Option C: Using curl

# Read example config
CONFIG=$(cat examples/example_simulation_config.xml)

# Create simulation
curl -X POST http://localhost:8000/simulations/ \
  -H "Content-Type: application/json" \
  -d "{
    \"name\": \"My First Simulation\",
    \"description\": \"Testing the API\",
    \"config_content\": \"$CONFIG\",
    \"users\": [\"user1\", \"user2\"],
    \"topics\": [\"303\", \"307\"]
  }"

Response:

{
  "id": "sim_abc123def456",
  "name": "My First Simulation",
  "status": "pending",
  ...
}

4. Start the Simulation

# Replace sim_abc123 with your simulation ID
curl -X POST http://localhost:8000/simulations/sim_abc123/control \
  -H "Content-Type: application/json" \
  -d '{"action": "start"}'

5. Monitor Progress

# Check status
curl http://localhost:8000/simulations/sim_abc123

# View logs (last 50 lines)
curl http://localhost:8000/simulations/sim_abc123/logs?tail=50

6. Get Results

# Once completed, get results
curl http://localhost:8000/simulations/sim_abc123/results

Using Python

import requests

BASE_URL = "http://localhost:8000"

# Create simulation
response = requests.post(
    f"{BASE_URL}/simulations/",
    json={
        "name": "My Simulation",
        "config_content": open("examples/example_simulation_config.xml").read(),
        "users": ["user1", "user2"],
        "topics": ["303", "307"]
    }
)
sim_id = response.json()['id']

# Start simulation
requests.post(
    f"{BASE_URL}/simulations/{sim_id}/control",
    json={"action": "start"}
)

# Check status
status = requests.get(f"{BASE_URL}/simulations/{sim_id}").json()
print(f"Status: {status['status']}")

# Get results
results = requests.get(f"{BASE_URL}/simulations/{sim_id}/results").json()
print(f"Queries: {len(results['queries'])}")

Common Operations

List all simulations

curl http://localhost:8000/simulations/

Filter by status

curl "http://localhost:8000/simulations/?status=running"

Pause simulation

curl -X POST http://localhost:8000/simulations/sim_abc123/control \
  -H "Content-Type: application/json" \
  -d '{"action": "pause", "checkpoint_before_action": true}'

Resume simulation

curl -X POST http://localhost:8000/simulations/sim_abc123/control \
  -H "Content-Type: application/json" \
  -d '{"action": "resume"}'

Stop simulation

curl -X POST http://localhost:8000/simulations/sim_abc123/control \
  -H "Content-Type: application/json" \
  -d '{"action": "stop"}'

List checkpoints

curl http://localhost:8000/simulations/sim_abc123/checkpoints

Configuration

Edit .env file to configure:

# Server settings
HOST=0.0.0.0
PORT=8000

# SimIIR paths
# Default expects simIIR at same level as simiir-api:
SIMIIR_REPO_PATH=../simiir
SIMIIR_DATA_PATH=../simiir/example_data

# Or use absolute paths:
# SIMIIR_REPO_PATH=/absolute/path/to/simiir
# SIMIIR_DATA_PATH=/absolute/path/to/simiir/example_data

# Simulation settings
MAX_CONCURRENT_SIMULATIONS=3
CHECKPOINT_INTERVAL=100

Important: If you cloned simIIR to a different location, update SIMIIR_REPO_PATH in .env

Docker

Build and run with Docker:

docker-compose up -d

View logs:

docker-compose logs -f

Stop:

docker-compose down

Quick Test with SimIIR Config

After starting the server, test with a simIIR configuration:

# Create simulation from simIIR example
poetry run python examples/test_with_real_config.py

# Output will show simulation ID, then:
# Start it
curl -X POST http://localhost:8000/simulations/{sim_id}/control \
  -H "Content-Type: application/json" \
  -d '{"action": "start"}'

# Monitor progress
curl http://localhost:8000/simulations/{sim_id}

# Watch logs in real-time
watch -n 2 "curl -s http://localhost:8000/simulations/{sim_id}/logs?tail=20"

# Get results when complete
curl http://localhost:8000/simulations/{sim_id}/results

Troubleshooting

SimIIR not cloned

cd ..  # Go to project root
git clone https://github.com/simint-ai/simiir-3.git simiir
cd simiir
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
deactivate

Port already in use

Change the port in .env:

PORT=8001

SimIIR not found

Verify the path in .env matches where you cloned simIIR:

ls $SIMIIR_REPO_PATH/simiir/run_simiir.py

Update .env if needed:

SIMIIR_REPO_PATH=/absolute/path/to/simiir

Module not found errors

Make sure simIIR dependencies are installed:

cd ../simiir
source venv/bin/activate  # Activate the virtual environment
pip install -r requirements.txt

Or ensure the simIIR venv Python is in your PATH.

Permission denied on start.sh

chmod +x start.sh

Next Steps

1. API Reference: Check API_REFERENCE.md for endpoint documentation
2. Architecture: See ARCHITECTURE.md for technical details
3. Commands: Browse COMMANDS.md for command cheat sheet
4. Examples: Explore examples in the examples/ directory
5. Interactive Docs: Visit http://localhost:8000/docs for Swagger UI

Support

For issues and questions:

1. Check the logs in logs/simiir_api.log
2. Review simulation logs in outputs/{simulation_id}/simulation.log
3. Refer to the main simIIR documentation

Happy simulating!