[BUG] Random seed has no effect

[hsbhc]

Describe the bug

when I run
python mava/systems/q_learning/anakin/rec_qmix.py env=smax env.scenario.task_name=2s3z system.seed=1 system.total_timesteps=2000000
multiple times, the results are not the same each time.

However, with a fixed random seed, the results are expected to be identical across runs. How can I ensure the same results?

[hsbhc]

import os
os.environ["XLA_FLAGS"] = "--xla_gpu_deterministic_ops=true"

This can solve the problem. But It seem down the speed.

[sash-a]

Thanks for this @hsbhc that's interesting. Can you double check on LBF (it's our fastest first party env) I think that it should work. If it doesn't then we know it's an issue in the algorithm and not the env 🙏

[hsbhc]

@sash-a
python mava/systems/q_learning/anakin/rec_qmix.py env=lbf env/scenario=2s-8x8-2p-2f-coop system.total_timesteps=2000000
first run print:

ABSOLUTE - Steps per second: 4874.107 | Episode length mean: 98.513 | Episode length std: 10.910 | Episode length min: 12 | Episode length max: 100 | Episode return mean: 0.233 | Episode return std: 0.268 | Episode return min: 0.000 | Episode return max: 1.000

second run print:

ABSOLUTE - Steps per second: 4934.911 | Episode length mean: 92.209 | Episode length std: 22.931 | Episode length min: 7 | Episode length max: 100 | Episode return mean: 0.231 | Episode return std: 0.342 | Episode return min: 0.000 | Episode return max: 1.000

They are different. This issue becomes more pronounced in the SMAX environment and appears at earlier training steps.

Previously, I added the following debug code:

q_loss = jnp.mean((q_online - target) ** 2)
# Debug output using jax.debug
# jax.debug.print("q_online shape: {s}", s=q_online.shape)
# jax.debug.print("target shape: {s}", s=target.shape)
# jax.debug.print("q_online sample: {s}", s=q_online.flatten()[:5])
# jax.debug.print("target sample: {s}", s=target.flatten()[:5])

and executed

python mava/systems/q_learning/anakin/rec_qmix.py env=smax env.scenario.task_name=2s3z system.total_timesteps=2000000 > run1_debug.log 2>&1
python mava/systems/q_learning/anakin/rec_qmix.py env=smax env.scenario.task_name=2s3z system.total_timesteps=2000000 > run2_debug.log 2>&1

The results showed slight numerical differences between the two runs, and the divergence increased as training progressed.
To address this, I added:

# import os
# os.environ["XLA_FLAGS"] = "--xla_gpu_deterministic_ops=true"

This resolved the issue, but it led to slower execution. Do you have a better solution?