Treatment Design: Group-bound vs Sequential

[dthinkr]

The current treatment system has two limitations we should align on before the next experiment:

1. Treatment = Group (between-subject only)

Lab links like T1_P1 bind the user to a fixed treatment group. The same treatment is applied every market, regardless of how many markets the user has played.

• T1_P1 → always treatment 0
• T2_P1 → always treatment 1

This works for between-subject designs (group A vs group B). But it does not support within-subject designs where the same person experiences different configurations across sequential markets (e.g. market 1 = config A, market 2 = config B).

Do we need within-subject treatment sequencing?

2. Treatments cannot override goals

Goals are assigned when a user joins the waiting room (based on predefined_goals in base params). Treatments are merged later, at market start. So writing predefined_goals in a treatment has no effect, the goal is already locked in.

Treatments currently only affect parameters consumed at market creation time: bot counts, duration, informed trader settings, etc.

Do we need treatments to control goals? My inclination is no. This would require a significant refactor (deferring goal assignment from join time to market start, reworking waiting room logic and pre-market role display), and I am not sure the added complexity is worth it. But want to check if anyone has a use case that needs this.

[dthinkr]

@asancetta @marjonuzaj

[asancetta]

We need two separate things, where one is urgent and the other is not.

1 (Urgent). We need to be able to carry out lab experiments where we have either one human participant per session or multiple ones.

2 (Not urgent and can skip it if problematic). Within the same session we want different type of treatments.

N.B. A session is a collection of markets (the exact number is given by max_markets_per_human) where the same participants interact.
Usually, we had only one participant per session, but we may want more.
This was discussed and addressed months ago, but now sure how to make it work after the latest changes.

If Point 1 is addressed, please specify how we select how many participants are in the same session. Sometimes, the number is not exactly divisible.

Please confirm that if we set treatments: [] in the dashboard, and use the link T1_P1 the treatment is as specified in the dashboard. We seemed had an instance where this was not true

[dthinkr]

1. Treatment = Group (between-subject only)

This is working as designed. Each lab link binds to a fixed group, and predefined_goals controls how many participants per session:

• 0 → 1 participant
• 0,0 → 2 participants
• 0,10,-10 → 3 participants (with different goals)

If the total number of lab participants isn't exactly divisible, the last incomplete room waits and won't start until it fills up.

Re "months ago": Multi-participant was first implemented in Oct 2025 (fbe5dc4, 4d881eb). The April rewrite (4938e9b) temporarily simplified to single-user sessions to reduce complexity during refactoring (852 → 230 LOC), and multi-participant was re-added (94f320c) with a cleaner design using T{treatment}_P{participant} lab links. You can verify on the dev server or review the commits directly.

On treatments: [] with T1_P1: this uses the dashboard base params. When the treatment list is empty, get_treatment_for_market(0) returns None, so base params apply unchanged. If there was an instance where this didn't hold, could you share the logs and a screenshot in an open issue?

2. Treatments cannot override goals

As noted above, my inclination is still no. We can revisit if there's a concrete experiment that needs it.

@asancetta

[asancetta]

predefined_goals did not work: it only allowed to insert one number no commas. I cannot double check because unable to connect to server.

Unfortunately, we need to cater for the possibility of non divisible groups.

[dthinkr]

The server connection issue is fixed: #33. That was a missing import from the dependency cleanup. Production is back online. I created that issue myself since we're just getting started with this workflow, but going forward please open issues directly for any bugs or feature requests. This way everything is tracked and categorized in one place.

On predefined_goals not accepting commas: I just verified on both dev and prod, it works. Entering 0,10,-10 saves correctly as [0, 10, -10]. If you're seeing different behavior, please open an issue with a screenshot so I can track it down.

Same for non-divisible groups: could you open an issue (I recommend "feature", not "bug") describing the specific scenario you need? (e.g. 5 participants, 2 per session, what should happen to the 5th?) That way we have a clear spec to work against.

One note on testing: it's best to test on local dev (docker compose up) rather than prod. We don't have a staging environment, so anything merged to main goes straight to production via CI/CD.

@asancetta

[asancetta]

I am unable to enter 0,10,-10 or any other sequence of numbers. I can only enter digits, e.g. 0 or 12343 etc. No point in taking screenshot because it just does not allow me to enter something different to show the behaviour.