Skip to content

Lab Experiment Specification

Jump to bottom
benwu edited this page May 20, 2026 · 3 revisions

Lab Experiment Specification

Related deployment handover notes: Deployment CICD

Overview

LOBX supports two authentication modes for experiments:

Mode Use case Link format
Lab In-person experiments (e.g., Udine) london-trader.web.app?LAB=T1_P1
Prolific Online recruitment london-trader.web.app?PROLIFIC_PID=xxx&STUDY_ID=yyy&SESSION_ID=zzz&TREATMENT=0

Basic Commands for Lab Experiments

To run a lab participant session, use:

https://london-trader.web.app/?LAB=T1_P1

Change the treatment and participant digits as needed, for example T1_P2.

To log out:

https://london-trader.web.app/logout

To reset an experiment and set the market count in a session back to zero, use Reset Experiment in the admin dashboard.

Lab Session Flow

sequenceDiagram
    participant E as Experimenter
    participant P1 as Participant 1
    participant P2 as Participant 2
    participant Platform as LOBX Platform
    participant Log as Log Files

    Note over E: Before the session
    E->>Platform: Configure treatments.yaml + admin settings

    Note over E: In the lab room
    E->>P1: "Open london-trader.web.app?LAB=T1_P1"
    E->>P2: "Open london-trader.web.app?LAB=T1_P2"

    P1->>Platform: GET ?LAB=T1_P1
    Platform->>Platform: Parse T1 = treatment 0, P1 = participant 1
    Platform->>P1: Login OK, join waiting room T0_M0

    P2->>Platform: GET ?LAB=T1_P2
    Platform->>Platform: Parse T1 = treatment 0, P2 = participant 2
    Platform->>P2: Login OK, join waiting room T0_M0

    Note over Platform: All participants ready
    Platform->>Platform: Start market
    Platform->>Log: T0_M0_1775749397.log
    Platform->>Log: parameter_history.json (participants, treatment, params)

    Note over E: Next session or next day
    E->>Platform: Reset Experiment
    E->>P1: Same link: ?LAB=T1_P1
    Platform->>Log: T0_M0_1775753000.log (different timestamp)
Loading

The experimenter controls assignment. The platform only parses the link and records data. No token generation, no link management, no state to get out of sync.

Link Format

Lab links

?LAB=T{treatment}_P{participant}
  • T1 = first treatment, T2 = second treatment (1-indexed)
  • P1 = first participant in that treatment group, P2 = second, etc.
  • Links are reusable entry points, not unique identifiers. The same link can be used across experiment sessions (e.g., morning and afternoon runs).
  • Uniqueness is guaranteed server-side via timestamps in log files and parameter_history.json records.

Example: 4 participants across 2 treatments

T1_P1, T1_P2   (treatment 1)
T2_P1, T2_P2   (treatment 2)

Links are constructed directly by the experimenter. No platform-side generation is needed. In a lab setting, the experimenter distributes links to participants in person (e.g., printed on paper, written on a whiteboard, or shared verbally). The format T{n}_P{n} is parsed server-side on login.

Prolific links

?PROLIFIC_PID=xxx&STUDY_ID=yyy&SESSION_ID=zzz&TREATMENT=0
  • PROLIFIC_PID is assigned by Prolific and serves as the unique participant identifier.
  • TREATMENT (optional) assigns a treatment group (0-indexed).

Treatment Configuration

Treatments are defined in back/config/treatments.yaml:

treatments:
- name: "1"
  informed_trade_intensity: 0.36
- name: "2"
  informed_trade_intensity: 0.69

Mapping: T1 in links to index 0 to treatment name "1", T2 to index 1 to treatment name "2".

Link prefix uses position, not name

The T{n} in ?LAB=T{n}_P{m} is the 1-based position of the treatment in treatments.yaml (validated as digits in back/api/lab_auth.py). The name field is only a display label in the admin dashboard and in parameter_history.json. Changing name: "1" to name: "Low" does not change the link; it is still ?LAB=T1_P1.

Base Parameters vs Treatment Overrides

  • Admin dashboard Config tab sets the base parameters applied to every market, such as trading duration, number of markets, and lot sizes.
  • treatments.yaml defines per-treatment overrides. When a participant joins via ?LAB=TN_PM, the treatment at position N is merged on top of the base settings at market start.

Editing Treatments from the Admin Panel

The admin dashboard Config tab includes a Treatment Sequence section (front/src/components/market/admin/ConfigTab.vue) that reads and writes back/config/treatments.yaml directly:

  • Load Treatments fetches the current file.
  • Save Treatments writes edits back.

The YAML file and the UI editor share the same source of truth; either can be used.

Log File Naming

Lab mode

T{treatment_group}_M{market_index}_{unix_timestamp}.log
Component Description Example
T{n} Treatment group (0-indexed) T0 = first treatment
M{n} Market index within sequence M0 = first market
{unix_timestamp} Epoch seconds at market start 1775749397

Example: T0_M0_1775749397.log. first treatment, first market, started at 2026-04-09 16:43:17.

Prolific mode

SESSION_{timestamp}_{uuid}_PROLIFIC_{pid}_MARKET_{index}.log

Example: SESSION_1775749181_7a4952bc_PROLIFIC_test123_MARKET_0.log

Design note

The timestamp in the log file name is the primary uniqueness mechanism. Two experiment runs on different days will produce different timestamps even if the same lab links (T1_P1) are reused. This is intentional. links are stable entry points; logs are unique records.

Data Traceability

parameter_history.json

Located at back/logs/parameters/parameter_history.json. Every market start is recorded:

{
  "2026-04-09T16:43:17.084684": {
    "source": "market_start",
    "market_id": "T0_M0_1775749397",
    "session_id": "T0_M0",
    "participants": ["LAB_T1_P1"],
    "treatment_name": "1",
    "treatment_index": 0,
    "parameters": { ... }
  }
}
Field Description
market_id Matches log file name (without .log)
session_id Groups markets in the same sequence
participants List of usernames in this market
treatment_name Human-readable treatment name from treatments.yaml
treatment_index 0-based treatment index actually applied to the market
parameters Full parameter snapshot used for this market

Traceability chain

Link (?LAB=T1_P1)
  to username: LAB_T1_P1
    to log file: T0_M0_{timestamp}.log
      to parameter_history.json: market_id, participants, treatment_index, treatment_name

To reconstruct which participant was in which market with which treatment:

# Find all markets for a participant
python3 -c "
import json
with open('back/logs/parameters/parameter_history.json') as f:
    data = json.load(f)
for ts, e in data.items():
    if e.get('source') == 'market_start' and 'LAB_T1_P1' in e.get('participants', []):
        print(f'{e[\"market_id\"]}  treatment={e.get(\"treatment_name\")}')
"

Admin Workflow

Admin dashboard: https://london-trader.web.app/admin (log in with the admin password). If you are currently signed in as a participant, go to /logout first.

Lab mode

  1. Configure treatments in back/config/treatments.yaml or via the Treatment Sequence section in the admin Config tab
  2. Set base parameters via the admin panel Config tab (trading duration, number of markets, etc.)
  3. Experimenter assigns links to participants in person. the platform does not manage link distribution
  4. Participants open london-trader.web.app?LAB=T{treatment}_P{participant} to consent to onboarding to trading
  5. After experiment: download logs + parameter_history.json via admin panel

Prolific mode

  1. Same setup as above (steps 1-2)
  2. Configure study on Prolific with the base URL + query parameters
  3. Prolific handles participant assignment and distribution

Experiment Reset

Between experiment runs (e.g., morning vs afternoon session), use the admin panel "Reset Experiment" button. This clears:

  • Active markets and waiting rooms
  • User session state and market counts

It does not clear:

  • Log files (they accumulate with unique timestamps)
  • parameter_history.json (append-only)

PnL Calculation

The PnL calculation function is in back/utils/logfiles_analysis.py.

For raw log analysis, it computes realized PnL from closed trades as:

sum(prices_sell) - sum(prices_buy)

Then it marks any unmatched position to the latest market with a liquidity adjustment:

  • num_buy == num_sell: fully flat, no extra adjustment
  • num_buy > num_sell: net long leftover units, assumes closing them by selling at the last best bid minus 2
  • num_sell > num_buy: net short leftover units, assumes closing them by buying at the last best ask plus 2

The hard-coded 2 can be changed to another value.

For participants with a non-zero goal, the final goal-adjusted metrics also use calculate_vwap_reward in the same file. That calculation uses the participant's completed trades, VWAP, latest mid price, target prices 110/90, and incomplete-goal penalty multipliers 1.5 for buys and 0.5 for sells. In other words, raw PnL and goal-adjusted reward/PnL are related but not identical.

Parameter Defaults

Parameter default values as they appear in the admin dashboard are defined in back/core/data_models.py.

The parameters used in a specific experiment are saved under the experiment logs parameters directory. Historical notes mention data/experiment_logs_20260401/parameters/base_settings.json; confirm the exact run directory for each experiment before analysis.

Platform Instructions

To change the main instructions on the welcome page, edit front/public/instructions/instructions.md.

Changes to the structure of the instruction pages require frontend route/session updates in files such as:

  • UserLanding.vue
  • navigation.js
  • index.js
  • guards.js
  • session.js

Given the current structure, content for instruction pages other than the main welcome instructions is edited in the relevant *.vue files under front/src/components/pages/.

Clone this wiki locally