Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# macOS
.DS_Store

# Python
__pycache__/
*.py[cod]
.venv/
venv/
env/

# Jupyter
.ipynb_checkpoints/

# Raw data generated by Notebook 01
example_data/raw/*.csv
!example_data/raw/.gitkeep

# Generated figure outputs
results/output_pdfs/*.pdf

# Local environment files
.env
93 changes: 88 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,92 @@
# CAMEL
# Open Metrics and Visual Diagnostics for Solar Wind Model Validation

The CAMEL framework is an integrated and flexible framework allowing users to seamlessly compare space weather and space science model outputs with observational data sets. CAMEL was developed at [Community Coordinated Modeling Center](https://ccmc.gsfc.nasa.gov) at NASA GSFC.
This repository provides metrics and visual diagnostics for assessing the capabilities of solar wind models. It contains widely used procedures for comparing solar wind model outputs against near-Earth in situ measurements. The repository complements the [CAMEL Ambient Solar Wind Validation Dashboard](https://ccmc.gsfc.nasa.gov/camel/AmbientSolarWind/home), developed and maintained at the [Community Coordinated Modeling Center (CCMC)](https://ccmc.gsfc.nasa.gov), by enabling users to reproduce assessment results and modify procedures.

The CAMEL web application can be viewed at: https://ccmc.gsfc.nasa.gov/tools/CAMEL/
The validation procedures include point-to-point metrics, threshold-based event validation, and peak-detection-based event validation. This effort is part of ongoing community work through the [ISWAT H1-01 Team](https://iswat-cospar.org/h1-01) to support unified capability assessment for ambient solar wind models across the heliophysics and space weather community.

This repository currently only includes skill score part of CAMEL and does not include the web application and user interface.
## Overview

Example python notebooks will soon be available.
The repository is organized around a set of Python modules and a Jupyter notebook workflow. The modules include data preparation, metrics, and visual diagnostics, while the notebooks provide end-to-end examples of how to use the modules on solar wind model outputs included in the CAMEL Dashboard.

Core functionality includes:
- preprocessing and alignment of modeled and observed solar wind time series
- point-to-point metrics
- threshold-based event validation with user-selected thresholds
- peak-detection-based event validation focused on timing and amplitude errors
- multi-model comparison against a reference baseline model
- visual diagnostics and PDF figure export

## Python Modules

The main code is organized into three Python modules:

- `data_processing.py` contains functions for cleaning, preprocessing, and interpolating modeled and observed solar wind time series.
- `skill_scores.py` includes the metrics used in the CAMEL Dashboard and throughout the notebooks.
- `visualizations.py` contains the Plotly routines used to reproduce the CAMEL Dashboard visualizations.

## Jupyter Notebooks

The notebook workflow provides a step-by-step guide for reproducing the assessment results in the CAMEL Ambient Solar Wind Dashboard:

1. `01_api_and_preprocessing.ipynb` uses an API to retrieve modeled and observed solar wind time series and saves the example datasets.
2. `02_point_to_point_metrics.ipynb` computes point-to-point metrics and plots visual diagnostics.
3. `03_binary_event_metrics.ipynb` computes threshold-based event metrics and plots visual diagnostics.
4. `04_peak_analysis.ipynb` computes peak-detection-based event metrics and plots visual diagnostics.
5. `05_multi_model_comparison.ipynb` compares multiple model outputs side by side against in situ measurements.

The notebooks save generated PDF figures under `results/output_pdfs/`.

## Getting Started

### 1. Clone the repository

```bash
git clone <repository-url>
cd <repository-name>
```

### 2. Install dependencies

Create and activate a Python environment, then install the required packages:

```bash
pip install -r requirements.txt
```

The `requirements.txt` file includes `kaleido`, which is required for Plotly figure export.

### 3. Run the notebooks

Start Jupyter and open the notebook series:

```bash
jupyter notebook notebooks/01_api_and_preprocessing.ipynb
```

Start with `01_api_and_preprocessing.ipynb` to create the processed example data before computing metrics and generating visualizations in the later notebooks.

## Citation

A publication describing the current implementation is in preparation. Until that publication is available, please cite the following prior work:

Reiss, M. A., et al. (2022), *Unifying the validation of ambient solar wind models*, **Advances in Space Research**. [https://doi.org/10.1016/j.asr.2022.05.026](https://doi.org/10.1016/j.asr.2022.05.026)

## Acknowledgements

This effort was developed in collaboration with the [ISWAT H1-01 Team](https://iswat-cospar.org/h1-01) and builds on the CAMEL validation framework developed at the [CCMC](https://ccmc.gsfc.nasa.gov/).

We are grateful for the feedback and discussions gathered through COSPAR ISWAT meetings.

## Contributors

- Martin A. Reiss
- Maya Levisohn
- Rick Mullinix

## License

This project is released under the NASA Open Source Agreement, Version 1.3 (NOSA 1.3). See `LICENSE` for the full license text.

## Contact

Martin A. Reiss ([email](mailto:martin.a.reiss@outlook.com))
73 changes: 73 additions & 0 deletions assets/fig_layout.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import plotly.graph_objects as go

# FIG LAYOUT
font_style = {
'color' : '#2b2323'
}

margin_style = {
'b': 10,
'l': 50,
'r': 8,
't': 50,
'pad': 0
}

xaxis_style = {
'linewidth' : 1,
'linecolor' : 'rgba(0, 0, 0, 0.35%)',
'showgrid' : False,
'zeroline' : False
}

yaxis_style = {
'linewidth' : 1,
'linecolor' : 'rgba(0, 0, 0, 0.35%)',
'showgrid' : True,
'gridwidth' : 1,
'gridcolor' : 'rgba(0, 0, 0, 0.11%)',
'zeroline' : False
}

my_figlayout = go.Layout(
paper_bgcolor='rgba(0,0,0,0)',
plot_bgcolor='rgba(0,0,0,0)',
font = font_style,
margin = margin_style,
xaxis = xaxis_style,
yaxis = yaxis_style,
height = 300
)

# TRACES LAYOUT
obs_linelayout = {
'width' : 2,
'color' : 'gray'
}

my_tablelayout = {
"style_cell": {
"textAlign": "left",
"padding": "10px",
"backgroundColor": "rgba(0,0,0,0)", # transparent
"color": font_style,
"border": "none",
"fontSize": "16px",
},
"style_header": {
"backgroundColor": "rgba(0,0,0,0.3)",
"color": font_style,
"border": "none",
"borderBottom": "2px solid #d1d5db", # header separator
},
"style_data": {
"borderBottom": "1px solid #e5e7eb", # row lines
},
"style_data_conditional": [
{
"if": {"row_index": "odd"},
"backgroundColor": "rgba(0,0,0,0.1)",
}
],
"style_table": {"overflowX": "auto"},
}
Loading