Skip to content

Fluidflower notebook #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jwboth wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Fluidflower notebook #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c6b315f
add data I/O utils for FluidFlower data
jwboth Apr 22, 2026
2500125
init fluidflower notebook
jwboth Apr 22, 2026
c09dbac
add init file
jwboth Apr 22, 2026
67f28cb
clear all outputs
jwboth Apr 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
270 changes: 270 additions & 0 deletions notebooks/fluidflower_task.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,270 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "01fecc9d",
"metadata": {},
"source": [
"# FluidFlower Task 1: Full-feature temporal surrogate\n",
"\n",
"This notebook loads in the data for two laboratory CO2 storage experiments and metadata. The experiments have been performed in [FluidFlower rigs](https://fluidflower.w.uib.no/) containing several sand layers.\n",
"\n",
"The model learns how to move a full spatial state forward in time, first with one-step prediction and then with autoregressive rollout.\n",
"\n",
"## Benchmark goal\n",
"\n",
"Given an observed state at time $t$, predict the future state at a chosen horizon $T$.\n",
"In practice, we start from one snapshot, predict the next snapshot, and repeat the same learned map until we reach the target time.\n",
"\n",
"## Dataset snapshot\n",
"\n",
"The data contains dense interpretations of saturation and total mass of CO2, provided as snapshots in time, using a fixed Cartesian grid of f 368 x 220 cells (uniform cell-size of 0.0025 cm).\n",
"- Each snapshot is stored as a CSV file with time indicated in the file name in hours_minute format.\n",
"- Between 0 and 2.5 hours snapshots are provided every 2nd minute.\n",
"- Between 2.5 and 12 hours snapshots are provided every 10th minute.\n",
"- Between 12 and 72 hours snapshots are provided every 30th minute.\n",
"- Every row contains one grid location and the associated physical fields.\n",
"- The grid coordinates stay fixed through time, so the task is temporal prediction on a fixed spatial mesh.\n",
"\n",
"## Features (CSV dataformat)\n",
"\n",
"Each row contains two coordinate columns and two state variables:\n",
"\n",
"- `x [m]`, `y [m]`: fixed spatial coordinates\n",
"- `saturation [-]`\n",
"- `tmCO2 [kg]`\n",
"\n",
"The main and target feature is `tmCO2` describing the total mass of CO2 in the cell with cell center (`x`,`y`) given in Cartesian Coordinates.\n",
"\n",
"## Experiments AC14 and BC02\n",
"\n",
"The two datasets are reanalyzed photographic data from [Haugen et al.](https://www.bing.com/search?pglt=673&q=haugen+et+al+variability&PC=U531&cvid=7db76131cbe0473ea292246f6f2788e9&FORM=ANNAB1) using the Darcy-scale image analysis toolbox [DarSIA](https://link.springer.com/article/10.1007/s11242-023-02000-9). \n",
"\n",
"As in the reference article, the experiments are referred to as AC14 and BC02. The two geometries are conceptually similar and are on the same meter scale and their geometries consist of the same sands yet differently layered, the same fluids have been used, and the laboratory conditions are comparable.\n",
"\n",
"## Metadata AC14\n",
"\n",
"The metadata is defines below in the code and briefly described here.\n",
"\n",
"The geometry is structured in facies:\n",
"\n",
"![](../doc/img/image2.png)\n",
"\n",
"The geometry has the following dimensions:\n",
"- width: 0.90 m\n",
"- height: 0.49 m\n",
"- depth: 0.01 m\n",
"\n",
"The laboratory conditions have been:\n",
"- atmospheric pressure: 1.013 bar\n",
"- temperature: 23 deg C\n",
"\n",
"The injection of CO2 is performed in two locations port 1 at coordinates (0.45, 0.03) and port 2 at coordinates (0.72, 0.18). The injection rate is varying slightly over time using a ramp up and down due to experimental limitations. The injection protocol is part of the dataset and provided as csv file. Here for convenience:\n",
"\n",
"|port|duration_s|rate_kg_per_s|mass_kg|\n",
"|----|----|----|----|\n",
"|1|60|3.12e-09|1.872e-07|\n",
"|1|60|6.24e-09|3.744e-07|\n",
"|1|60|1.56e-08|9.36e-07|\n",
"|1|60|3.12e-08|1.872e-06|\n",
"|1|60|4.68e-08|2.808e-06|\n",
"|1|2700|6.24e-08|1.6848e-04|\n",
"|1|60|4.68e-08|2.808e-06|\n",
"|1|60|3.12e-08|1.872e-06|----|\n",
"|1|60|1.56e-08|9.36e-07|\n",
"|1|60|6.24e-09|3.744e-07|\n",
"|1|60|3.12e-09|1.872e-07|\n",
"|2|60|3.12e-09|1.872e-07|\n",
"|2|60|6.24e-09|3.744e-07|\n",
"|2|60|1.56e-08|9.36e-07|\n",
"|2|60|3.12e-08|1.872e-06|\n",
"|2|60|4.68e-08|2.808e-06|\n",
"|2|4500|6.24e-08|2.808e-04|\n",
"|2|60|4.68e-08|2.808e-06|\n",
"|2|60|3.12e-08|1.872e-06|\n",
"|2|60|1.56e-08|9.36e-07|\n",
"|2|60|6.24e-09|3.744e-07|\n",
"|2|60|3.12e-09|1.872e-07|\n",
"\n",
"\n",
"## Metadata BC02\n",
"\n",
"The metadata is defines below in the code and briefly described here.\n",
"\n",
"The geometry is structured in facies:\n",
"\n",
"![](../doc/img/image3.png)\n",
"\n",
"The geometry has the following dimensions:\n",
"- width: 0.92 m\n",
"- height: 0.55 m\n",
"- depth: 0.01 m\n",
"\n",
"The laboratory conditions have been:\n",
"- atmospheric pressure: 0.995 bar\n",
"- temperature: 23 deg C\n",
"\n",
"The injection of CO2 is performed in two locations port 1 at coordinates (0.74, 0.07) and port 2 at coordinates (0.23, 0.24). The injection rate is varying slightly over time using a ramp up and down due to experimental limitations. The injection protocol is part of the dataset and provided as csv file. Here for convenience:\n",
"\n",
"|port|duration_s|rate_kg_per_s|mass_kg|\n",
"|----|----|----|----|\n",
"|1|60|3.12e-08|1.872e-06|\n",
"|1|60|4.68e-08|2.808e-06|\n",
"|1|3793|6.24e-08|2.366832e-04|\n",
"|1|60|4.68e-08|2.808e-06|\n",
"|1|60|3.12e-08|1.872e-06|\n",
"|1|60|1.56e-08|9.36e-07|\n",
"|2|60|3.12e-08|1.872e-06|\n",
"|2|60|4.68e-08|2.808e-06|\n",
"|2|4372|6.24e-08|2.728128e-04|\n",
"|2|60|4.68e-08|2.808e-06|\n",
"|2|60|3.12e-08|1.872e-06|\n",
"|2|60|1.56e-08|9.36e-07|\n",
"\n",
"## Hydraulic properties\n",
"\n",
"The geometries of both experiments are based on the same four sand types, and a water layer on top, with the following parameters:\n",
"\n",
"|id|name|porosity|permeability|swi CO2|krel_gas|1-Sg|krel_water|pc [mbar]|\n",
"|----|----|----|----|----|----|----|----|----|\n",
"|0|water|Nan|Nan|Nan|Nan|Nan|Nan|Nan|\n",
"|1|ESF|0.44|4.40E+01|0.32|0.09|0.86|0.71|15|\n",
"|2|C|0.44|473|0.14|0.05|0.9|0.93|3.3|\n",
"|4|E|0.45|2005|0.12|0.1|0.94|0.93|0.26|\n",
"|5|F|0.44|4295|0.12|0.11|0.87|0.72|0.1|\n",
"\n",
"The spatial distribution of the different facies is provided as part of the dataset.\n",
"\n",
"## Notation\n",
"\n",
"Let $s_t$ denote the full spatial state at time $t$. In this notebook, one step of the surrogate is written as\n",
"\n",
"$$\n",
"s_{t+\\Delta t} = f_{\\theta}(s_t, \\Delta t)\n",
"$$\n",
"\n",
"where $\\Delta t$ is the time gap between two snapshots and $f_{\\theta}$ is a learned regression map.\n",
"When we apply the same map repeatedly, we get an autoregressive rollout:\n",
"\n",
"$$\n",
"s_{t_{k+1}} = f_{\\theta}(s_{t_k}, t_{k+1} - t_k)\n",
"$$\n",
"\n",
"\n",
"## What this notebook shows\n",
"\n",
"- how to load the csv files\n",
"- how to export predictions into the benchmark CSV format\n",
"- a simple DMD model for replaying the datasets\n"
]
},
{
"cell_type": "markdown",
"id": "b35e5e9e",
"metadata": {},
"source": [
"## Loading metadata for AC14\n",
"The metadata is store in a json file (dimensions, pressure, temperature) and csv file (injection protocol)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fac20102",
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"from pathlib import Path\n",
"\n",
"# Folder to data of AC14\n",
"ac14_folder = Path(\"/home/jovyan/shared/folder/data/ac14\")\n",
"\n",
"metadata_ac14 = json.load(open(ac14_folder / \"metadata.json\", \"r\"))\n",
"width_ac14 = metadata_ac14[\"width\"]\n",
"height_ac14 = metadata_ac14[\"height\"]\n",
"depth_ac14 = metadata_ac14[\"depth\"]\n",
"pressure_ac14 = metadata_ac14[\"pressure\"]\n",
"temperature_ac14 = metadata_ac14[\"temperature\"]\n"
]
},
{
"cell_type": "markdown",
"id": "48a8a725",
"metadata": {},
"source": [
"## Loading the AC14 data\n",
"As an example we load the csv file corresponding to the approx. the end of the injection phase of AC14, after 2:30 hours. We extract the mass and convert the data to a displayabe image."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2128566a",
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"from ml4gcs.fluidflower_utils.read_csv import load, save, to_2d_array, to_1d_array, TIMESTAMPS, SELECTED_TIMESTAMPS\n",
"\n",
"# Temporary folder for plotting\n",
"plt_folder = Path(\"plots_fludflower_ac14\")\n",
"plt_folder.mkdir(exist_ok=True)\n",
"\n",
"# Read facies file\n",
"facies = load(ac14_folder / \"facies.csv\", \"facies\")\n",
"facies_2d = to_2d_array(*facies)\n",
"\n",
"plt.figure(figsize=(6, 3))\n",
"plt.imshow(facies_2d, origin='lower', extent=[0, width_ac14, 0, height_ac14], cmap='tab20')\n",
"plt.colorbar(label='Facies')\n",
"plt.title('FluidFlower AC14 - Facies')\n",
"plt.xlabel('Width (m)')\n",
"plt.ylabel('Height (m)')\n",
"plt.grid(False)\n",
"plt.tight_layout()\n",
"plt.savefig(plt_folder / \"fluidflower_ac14_facies.png\")\n",
"plt.show()\n",
"display(Image(filename=str(plt_folder / \"fluidflower_ac14_facies.png\")))\n",
"\n",
"# Read snapshots and plot\n",
"selected_snapshots = [load(ac14_folder / f\"ml4gcs_spatial_map_{timestamp}.csv\") for timestamp in SELECTED_TIMESTAMPS]\n",
"selected_snapshots_2d = [to_2d_array(*snapshot) for snapshot in selected_snapshots]\n",
"\n",
"for timestamp, snapshot in zip(SELECTED_TIMESTAMPS, selected_snapshots_2d):\n",
" plt.figure(figsize=(6, 3))\n",
" plt.imshow(snapshot, origin='lower', extent=[0, width_ac14, 0, height_ac14], cmap='viridis')\n",
" plt.colorbar(label='Saturation')\n",
" plt.title(f'FluidFlower AC14 - Saturation at {timestamp.replace(\"_\", \":\")}')\n",
" plt.xlabel('Width (m)')\n",
" plt.ylabel('Height (m)')\n",
" plt.grid(False)\n",
" plt.tight_layout()\n",
" plt.savefig(plt_folder / f\"fluidflower_ac14_saturation_{timestamp}.png\")\n",
" plt.show()\n",
" display(Image(filename=str(plt_folder / f\"fluidflower_ac14_saturation_{timestamp}.png\")))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
23 changes: 23 additions & 0 deletions src/ml4gcs/fluidflower_utils/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
"""FluidFlower utilities for data loading and processing."""

from .data import (
ALL_TIMESTAMPS,
TIMESTAMPS,
SELECTED_TIMESTAMPS,
FluidFlowerData,
load,
to_2d_array,
to_1d_array,
save,
)

__all__ = [
"ALL_TIMESTAMPS",
"TIMESTAMPS",
"SELECTED_TIMESTAMPS",
"FluidFlowerData",
"load",
"to_2d_array",
"to_1d_array",
"save",
]
Loading