CCS_MARL

Code and experiment notebooks for cooperative, competitive, and mixed multi-agent reinforcement learning (MARL) studies for multi-operator geological carbon storage (GCS) projects.

Repository Contents

• MADDPG/: multi-agent deep deterministic policy gradient implementation.
• Enviroment/: CCS reservoir-control environment used by the MARL experiments.
• MSE2C.py, MSE2C_layers.py, MSloss.py, ROMWithMSE2C.py: reduced-order model and E2C/MSE2C components.
• data_preprocessing.py: utilities for loading and preprocessing MATLAB data files.
• MARL_train_*.ipynb: training and evaluation notebooks for fully collaborative, fully competitive, and mixed-agent settings.
• MOO_fullycomp*.ipynb: multi-objective optimization notebooks.
• saved_models/: pretrained E2C model components used by the notebooks.
• results/: selected result figures and small outputs.

Data

The large dataset is available on OSF:

https://osf.io/upqan/files/osfstorage

Download the required data files from OSF and place them in the local data/ directory before running the notebooks. In particular, data/states_norm_slt.mat is several GB in size and is intentionally excluded from Git tracking.

The repository currently includes only small data files that are practical to version with the code.

Setup

Create a Python environment and install the required packages:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

On Windows PowerShell, activate the environment with:

.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

The notebooks use PyTorch. Install the CUDA-enabled PyTorch build appropriate for your system if GPU acceleration is needed.

Running Experiments

1. Download the data from OSF into data/.
2. Install the dependencies from requirements.txt.
3. Open the relevant notebook in JupyterLab.
4. Run one of the MARL notebooks, depending on the scenario:
   • MARL_train_fullycollab.ipynb
   • MARL_train_fullycomp.ipynb
   • MARL_train_mixed_A&B_C.ipynb
   • MARL_train_mixed_A&C_B.ipynb
   • MARL_train_mixed_B&C_A.ipynb
5. Use the MOO_fullycomp*.ipynb notebooks for the multi-objective optimization experiments.

Citation

If you use this code or data, please cite:

Chen, J., & Hosseini, S. A. (2026). Optimal CO2 storage management considering safety constraints in multi-stakeholder multi-site GCS projects: A Markov game perspective. International Journal of Greenhouse Gas Control, 154, 104682. https://doi.org/10.1016/j.ijggc.2026.104682