From a70e2ec2c28e0dbc36b98f1e9c3e63f41be80bf8 Mon Sep 17 00:00:00 2001 From: Damien Delforge Date: Tue, 28 Apr 2026 11:48:00 +0200 Subject: [PATCH 1/6] Migrate documentation files to a unified `instructions/` directory and update references throughout the project. --- .junie/instructions/data.md | 20 +------- .junie/instructions/domain.md | 23 +-------- .junie/instructions/validation.md | 24 +-------- AGENT.md | 21 -------- AGENTS.md | 81 +++++++++++++++++++++++++++++++ instructions/data.md | 19 ++++++++ instructions/domain.md | 22 +++++++++ instructions/validation.md | 27 +++++++++++ 8 files changed, 152 insertions(+), 85 deletions(-) delete mode 100644 AGENT.md create mode 100644 AGENTS.md create mode 100644 instructions/data.md create mode 100644 instructions/domain.md create mode 100644 instructions/validation.md diff --git a/.junie/instructions/data.md b/.junie/instructions/data.md index c7898b5..793ba8a 100644 --- a/.junie/instructions/data.md +++ b/.junie/instructions/data.md @@ -1,19 +1 @@ -# Reference Data & Metadata Management - -EM-TEST relies on reference data to validate EM-DAT content. - -## 1. Reference Files -Reference data is stored in `emtest\validation_data\`. -- **`classification_tree.toml`**: The source of truth for disaster types and subgroups. -- **`UNSD_M49_standards.csv`**: Reference for country and region names. -- **`gaul_adm1_code.txt`**: GAUL administrative level 1 codes. - -## 2. Loading Data -Reference data is loaded by `emtest\validation_data\data_loader.py`. -When adding new reference files, update the `data_loader.py` and ensure they are included in `pyproject.toml` under `tool.setuptools.package-data`. - -## 3. Metadata and Citations -- **`CITATION.cff`**: Project citation info for scientific researchers. -- **`LICENSE`**: MIT License. -- **Versioning**: Follow semantic versioning, which is reflected in `pyproject.toml`. -- **README Updates**: If you add new validation checks, update the corresponding tables in `README.md`. +Moved to [`instructions/data.md`](../../instructions/data.md). diff --git a/.junie/instructions/domain.md b/.junie/instructions/domain.md index 999bacc..339f3f0 100644 --- a/.junie/instructions/domain.md +++ b/.junie/instructions/domain.md @@ -1,22 +1 @@ -# EM-DAT Domain Knowledge - -EM-TEST validates data related to international disasters. - -## 1. Geospatial Consistency -- **ISO3 Codes**: Country codes should be valid ISO-3166-1 alpha-3 codes. -- **Geography**: Valid latitude (-90 to 90) and longitude (-180 to 180). -- **Standards**: The project follows UNSD M49 standards for country and area codes. - -## 2. Temporal Logic -Disaster numbers (`DisNo.`) contain the year the disaster was recorded. -- **Consistency**: `Start Year` should generally match the year in `DisNo.`. -- **Duration**: `Start Year/Month/Day` must be before or equal to `End Year/Month/Day`. - -## 3. Classification -Disasters are categorized hierarchically: -- **Disaster Group** (e.g., Natural) -- **Disaster Subgroup** (e.g., Meteorological) -- **Disaster Type** (e.g., Storm) -- **Disaster Subtype** (e.g., Tropical cyclone) - -Validation must ensure that combinations of these fields follow the official `classification_tree.toml`. +Moved to [`instructions/domain.md`](../../instructions/domain.md). diff --git a/.junie/instructions/validation.md b/.junie/instructions/validation.md index f45e0f2..cb43756 100644 --- a/.junie/instructions/validation.md +++ b/.junie/instructions/validation.md @@ -1,23 +1 @@ -# Data Validation with Pandera and Pandas - -The core of EM-TEST is built on `pandera` and `pandas`. - -## 1. Schema Definition -Schemas are defined in `emtest\validation_schemas.py`. -- **`DataFrameSchema`**: Use this to define per-column constraints. -- **Nullability**: In Pandas, the standard `int` type is not nullable. Use `float` for columns that contain integers but may have null values (e.g., `Start Month`, `Start Day`). - -## 2. Custom Checks -Complex validation logic should be implemented in `emtest\custom_checks.py`. -- **Single-Column Checks**: Implement logic for a specific field (e.g., `check_disno`, `check_month`). -- **Multi-Column ("Wide") Checks**: Use these for consistency between fields (e.g., `check_start_end_year_consistency`). - -## 3. Regular Expression Engineering -Identifiers are often validated via Regex: -- **DisNo format**: `^\d{4}-\d{4}-[A-Z]{3}$` (Year-Sequence-CountryISO). -- **External IDs**: GLIDE numbers and other identifiers rely on specific patterns. - -## 4. Testing Pattern -Since there are no traditional automated tests, verify changes using the example script: -- `python examples\validation_script.py` -This script uses the schema to validate a sample dataset. +Moved to [`instructions/validation.md`](../../instructions/validation.md). diff --git a/AGENT.md b/AGENT.md deleted file mode 100644 index cbfa344..0000000 --- a/AGENT.md +++ /dev/null @@ -1,21 +0,0 @@ -# EM-TEST Agent Instructions - -EM-TEST is a Python-based testing framework for [EM-DAT](https://www.emdat.be/) disaster datasets using `pandas` and `pandera`. - -## Quick Start -- **Environment**: Python 3.11+, managed with `uv`. -- **Install**: `uv sync` -- **Core Logic**: `emtest/` module. -- **Validation**: `emtest/validation_schemas.py` and `emtest/custom_checks.py`. - -## Progressive Disclosure Instructions -For specific tasks, refer to the following guides: - -- [**Data Validation**](.junie/instructions/validation.md): Writing schemas, custom checks, and handling nullable types. -- [**EM-DAT Domain**](.junie/instructions/domain.md): Geospatial, temporal, and classification logic for disaster data. -- [**Reference Data & Metadata**](.junie/instructions/data.md): Managing reference files and project metadata (`CITATION.cff`). - -## General Principles -- **Transparency**: Every check must be well-documented. -- **Reproducibility**: Use `uv` for consistent environments. -- **Consistency**: Maintain the existing `pandera` schema structure. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..1d27351 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,81 @@ +# EM-TEST Agent Instructions + +EM-TEST is a Python data validation framework for [EM-DAT](https://www.emdat.be/) disaster datasets. It validates disaster event DataFrames against a comprehensive schema of type constraints, business rules, and multi-column consistency checks using `pandera` and `pandas`. + +Versioning follows CalVer: `YYYY.MM.N` (e.g., `2026.04.0`). Version is defined in `emtest/__init__.py` and must also be updated in `CITATION.cff` on release. + +## Commands + +```bash +# Install with all dev dependencies (preferred) +uv sync --all-extras --dev + +# Run all tests +uv run pytest + +# Run a single test file +uv run pytest tests/test_custom_checks.py + +# Run a specific test +uv run pytest tests/test_custom_checks.py::test_check_disno +``` + +No Makefile or linting config exists — the project relies on `pytest` for correctness checks and manual review. + +## Architecture + +``` +emtest/ +├── __init__.py # Public API: exports emdat_schema and __version__ +├── validation_schemas.py # DataFrameSchema: 50+ columns + multi-column checks +├── custom_checks.py # Check functions used inside validation_schemas.py +├── utils.py # get_validation_report(), update_column_checks(), etc. +└── validation_data/ # Reference data package + ├── data_loader.py # Loads TOML/CSV/TXT files via Path(__file__).parent + ├── classification.py # Disaster classification tree (loaded from .toml) + ├── areas.py # ISO3 codes, country names, regions, GAUL codes + └── magnitude.py # Magnitude scale units per disaster type +``` + +**Data flow**: user loads an Excel/CSV file → passes a DataFrame with `DisNo.` as index to `emdat_schema.validate(df, lazy=True)` → pandera raises `SchemaErrors` → `get_validation_report()` converts errors to a flat DataFrame for export. + +## Domain Knowledge + +See [`instructions/domain.md`](instructions/domain.md) for the full reference. Key points: + +- **ISO3 / geography**: country codes follow ISO-3166-1 alpha-3; latitude −90–90, longitude −180–180; regions follow UNSD M49. +- **DisNo. year**: `Start Year` should generally match the four-digit year embedded in `DisNo.` (a warning, not an error). +- **Date chronology**: `Start Year/Month/Day` ≤ `End Year/Month/Day` at every available resolution. +- **Classification**: Group → Subgroup → Type → Subtype combinations are validated against `classification_tree.toml`. + +## Non-Obvious Design Decisions + +**Nullable integer columns use `float`**: Fields like `Start Month`, `Start Day`, `End Month`, `End Day` can be null. Pandas `int` dtype is non-nullable, so these are defined as `float` with `NaN` for missing values throughout the schema and checks. + +**Wide (multi-column) checks and deduplication**: Multi-column check functions in pandera fail every row they touch, causing duplicated error rows in reports. `deduplicate_errors()` in `utils.py` filters these so each logical failure appears once. When adding new wide checks, define which column should "own" the error in `WIDE_CHECKS_TO_KEEP`. + +**Index access inside column checks**: The `check_disno_vs_start_year` check compares the year embedded in `DisNo.` (the DataFrame index) against the `Start Year` column. Since the index isn't a regular column in pandera, this check is attached to the `Start Year` column and accesses `series.index` internally. + +**Parameterized date consistency via `partial()`**: `check_start_end_consistency()` in `custom_checks.py` is called three times (year/month/day resolution) using `functools.partial()`. The helper `_convert_to_date` assembles a date string and calls `pd.to_datetime(..., errors='coerce')`. Two subtleties: (1) month/day NaN values map to `'00'`, producing an invalid date string that coerces to `NaT` — the `pd.isna()` guard in the caller then skips the comparison, which is the intended behaviour for incomplete dates; (2) year columns must be converted via `int()` before `str()` because when any row has a null `End Year`, pandas reads the entire column as `float64`, making `.astype(str)` produce `'2024.0'` instead of `'2024'`. + +**GAUL code validation is expensive**: `has_valid_GAUL_codes()` parses JSON strings in the `Admin Units` column, then validates codes against two large reference lists (ADM1: ~1k codes, ADM2: ~50k codes). It is only applied to non-null values. If performance is an issue on large datasets, this check is the likely bottleneck. + +**Disaster type-specific magnitude rules**: Magnitude validation branches on `Classification Key` (a semicolon-delimited hierarchy like `nat;geo;earthquake`). Earthquake expects 3–10, cold wave ≤10°C, heat wave ≥25°C, everything else >0. The logic lives in `custom_checks.py` functions like `check_earthquake_magnitude`. + +**Warnings vs errors**: Some checks (ISO code, country name, start year vs DisNo. year, CPI range) are configured as `raise_warning=True` in `validation_schemas.py`. `get_validation_report(add_warnings=True)` includes them; `set_warnings_to_errors()` in `utils.py` promotes all warnings to hard errors. + +## Tests + +`tests/conftest.py` provides two fixtures reused across test files: +- `valid_emdat_row` — dict with all 50 columns set to valid values +- `valid_df` — single-row DataFrame with `DisNo.` as index + +The typical test pattern mutates one field of `valid_emdat_row`, constructs a DataFrame, and asserts that `emdat_schema.validate()` raises `SchemaError`. + +## Detailed Instructions + +For specific tasks, refer to the following guides in `instructions/`: + +- [**Data Validation**](instructions/validation.md): Writing schemas, custom checks, and handling nullable types. +- [**EM-DAT Domain**](instructions/domain.md): Geospatial, temporal, and classification logic for disaster data. +- [**Reference Data & Metadata**](instructions/data.md): Managing reference files and project metadata (`CITATION.cff`). diff --git a/instructions/data.md b/instructions/data.md new file mode 100644 index 0000000..e1317ed --- /dev/null +++ b/instructions/data.md @@ -0,0 +1,19 @@ +# Reference Data & Metadata Management + +EM-TEST relies on reference data to validate EM-DAT content. + +## 1. Reference Files +Reference data is stored in `emtest/validation_data/`. +- **`classification_tree.toml`**: The source of truth for disaster types and subgroups. +- **`UNSD_M49_standards.csv`**: Reference for country and region names. +- **`gaul_adm1_code.txt`**: GAUL administrative level 1 codes. + +## 2. Loading Data +Reference data is loaded by `emtest/validation_data/data_loader.py`. +When adding new reference files, update `data_loader.py` and ensure they are included in `pyproject.toml` under `tool.setuptools.package-data`. + +## 3. Metadata and Citations +- **`CITATION.cff`**: Project citation info for scientific researchers. +- **`LICENSE`**: MIT License. +- **Versioning**: Follow CalVer (`YYYY.MM.N`), reflected in `emtest/__init__.py`, `pyproject.toml`, and `CITATION.cff`. +- **README Updates**: If you add new validation checks, update the corresponding tables in `README.md`. diff --git a/instructions/domain.md b/instructions/domain.md new file mode 100644 index 0000000..999bacc --- /dev/null +++ b/instructions/domain.md @@ -0,0 +1,22 @@ +# EM-DAT Domain Knowledge + +EM-TEST validates data related to international disasters. + +## 1. Geospatial Consistency +- **ISO3 Codes**: Country codes should be valid ISO-3166-1 alpha-3 codes. +- **Geography**: Valid latitude (-90 to 90) and longitude (-180 to 180). +- **Standards**: The project follows UNSD M49 standards for country and area codes. + +## 2. Temporal Logic +Disaster numbers (`DisNo.`) contain the year the disaster was recorded. +- **Consistency**: `Start Year` should generally match the year in `DisNo.`. +- **Duration**: `Start Year/Month/Day` must be before or equal to `End Year/Month/Day`. + +## 3. Classification +Disasters are categorized hierarchically: +- **Disaster Group** (e.g., Natural) +- **Disaster Subgroup** (e.g., Meteorological) +- **Disaster Type** (e.g., Storm) +- **Disaster Subtype** (e.g., Tropical cyclone) + +Validation must ensure that combinations of these fields follow the official `classification_tree.toml`. diff --git a/instructions/validation.md b/instructions/validation.md new file mode 100644 index 0000000..4c37e4d --- /dev/null +++ b/instructions/validation.md @@ -0,0 +1,27 @@ +# Data Validation with Pandera and Pandas + +The core of EM-TEST is built on `pandera` and `pandas`. + +## 1. Schema Definition +Schemas are defined in `emtest/validation_schemas.py`. +- **`DataFrameSchema`**: Use this to define per-column constraints. +- **Nullability**: In Pandas, the standard `int` type is not nullable. Use `float` for columns that contain integers but may have null values (e.g., `Start Month`, `Start Day`). + +## 2. Custom Checks +Complex validation logic should be implemented in `emtest/custom_checks.py`. +- **Single-Column Checks**: Implement logic for a specific field (e.g., `check_disno`, `check_month`). +- **Multi-Column ("Wide") Checks**: Use these for consistency between fields (e.g., `check_start_end_year_consistency`). + +## 3. Regular Expression Engineering +Identifiers are often validated via Regex: +- **DisNo format**: `^\d{4}-\d{4}-[A-Z]{3}$` (Year-Sequence-CountryISO). +- **External IDs**: GLIDE numbers and other identifiers rely on specific patterns. + +## 4. Testing Pattern +Changes must be verified with the pytest test suite: +- `uv run pytest` — runs all tests in `tests/` +- `uv run pytest tests/test_validation.py::test_date_consistency` — run a single test + +The suite tests both individual check functions (`tests/test_custom_checks.py`) and the full schema (`tests/test_validation.py`). The fixture in `tests/conftest.py` provides a valid single-row DataFrame; tests mutate one field at a time and assert the expected `SchemaError`. + +The example scripts in `examples/` validate real datasets and can be used for end-to-end verification after schema changes. From 556f8e78e91d818edd8da756570115d79dd044bb Mon Sep 17 00:00:00 2001 From: Damien Delforge Date: Tue, 28 Apr 2026 11:48:31 +0200 Subject: [PATCH 2/6] Fix and refine date consistency validation logic and tests; remove unused checks from utilities. --- emtest/custom_checks.py | 10 +++++--- emtest/utils.py | 2 -- tests/test_validation.py | 51 ++++++++++++++++++++++++++++++++++------ 3 files changed, 51 insertions(+), 12 deletions(-) diff --git a/emtest/custom_checks.py b/emtest/custom_checks.py index 3c55197..d0a8145 100644 --- a/emtest/custom_checks.py +++ b/emtest/custom_checks.py @@ -242,9 +242,13 @@ def float_to_string(x): else: return str(int(x)).zfill(2) + year = df[f'{start_or_end} Year'].apply( + lambda x: None if pd.isna(x) else str(int(x)) + ) + if resolution == 'day': return pd.to_datetime( - df[f'{start_or_end} Year'].astype(str) + + year + df[f'{start_or_end} Month'].apply(float_to_string) + df[f'{start_or_end} Day'].apply(float_to_string), format='%Y%m%d', @@ -252,13 +256,13 @@ def float_to_string(x): ) elif resolution == 'month': return pd.to_datetime( - df[f'{start_or_end} Year'].astype(str) + + year + df[f'{start_or_end} Month'].apply(float_to_string), format='%Y%m', errors='coerce' ) elif resolution == 'year': - return pd.to_datetime(df[f'{start_or_end} Year'], format='%Y') + return pd.to_datetime(year, format='%Y', errors='coerce') def _extract_GAUL_code(d: dict) -> tuple[int, int]: diff --git a/emtest/utils.py b/emtest/utils.py index c52311a..9a4a12a 100644 --- a/emtest/utils.py +++ b/emtest/utils.py @@ -7,7 +7,6 @@ WIDE_CHECKS_TO_KEEP: dict[str, list[str]] = { 'Missing latitude or longitude coordinates': ['Latitude', 'Longitude'], - 'Start date and end date inconsistency': ['Start Year'], 'Invalid coldwave magnitude': ['Magnitude'], 'Invalid earthquake magnitude': ['Magnitude'], 'Invalid heatwave magnitude': ['Magnitude'], @@ -20,7 +19,6 @@ } - def get_validation_report( df: pd.DataFrame, schema: DataFrameSchema, diff --git a/tests/test_validation.py b/tests/test_validation.py index 388c199..d5db239 100644 --- a/tests/test_validation.py +++ b/tests/test_validation.py @@ -20,12 +20,8 @@ def test_invalid_classification_key(valid_df): emdat_schema.validate(valid_df) def test_invalid_iso_code(valid_df): - """Test that an invalid 'ISO' code fails (regex check).""" - valid_df.loc[valid_df.index[0], "ISO"] = "BXX" # Invalid 3-letter code if we strictly check against reference, but let's see if regex fails it first - # The schema has two checks on ISO: validate_iso3_code (regex) and check_iso3_code (list check, warning only) - # Let's try one that fails regex: + """Test that a non-alphabetic ISO code fails the regex check and raises a warning.""" valid_df.loc[valid_df.index[0], "ISO"] = "123" - # We expect a SchemaError for regex AND a SchemaWarning for the list check with pytest.warns(pa.errors.SchemaWarning, match="ISO3 code not in reference list"): with pytest.raises(pa.errors.SchemaError, match="Invalid ISO3 code"): emdat_schema.validate(valid_df) @@ -34,7 +30,6 @@ def test_invalid_disno_index(valid_df): """Test that an invalid 'DisNo.' index pattern fails.""" valid_df.index = ["INVALID-DISNO"] valid_df.index.name = "DisNo." - # We expect a SchemaError for the index pattern AND a SchemaWarning for the Start Year vs DisNo. check with pytest.warns(pa.errors.SchemaWarning, match="Start year differs from DisNo year"): with pytest.raises(pa.errors.SchemaError, match="Invalid DisNo. Pattern"): emdat_schema.validate(valid_df) @@ -48,7 +43,7 @@ def test_magnitude_range_checks(valid_df): emdat_schema.validate(valid_df) def test_date_consistency(valid_df): - """Test that start date after end date fails.""" + """Test that start month after end month within the same year fails.""" valid_df.loc[valid_df.index[0], "Start Year"] = 2024 valid_df.loc[valid_df.index[0], "Start Month"] = 12 valid_df.loc[valid_df.index[0], "End Year"] = 2024 @@ -56,6 +51,48 @@ def test_date_consistency(valid_df): with pytest.raises(pa.errors.SchemaError, match="Start date inconsistency at the month resolution"): emdat_schema.validate(valid_df) +def test_start_year_after_end_year(valid_df): + """Test that start year after end year fails.""" + valid_df.index = ["2025-0001-BEL"] + valid_df.index.name = "DisNo." + valid_df.loc[valid_df.index[0], "Start Year"] = 2025 + valid_df.loc[valid_df.index[0], "End Year"] = 2024 + with pytest.raises(pa.errors.SchemaError, match="Start date inconsistency at the year resolution"): + emdat_schema.validate(valid_df) + +def test_start_day_after_end_day(valid_df): + """Test that start day after end day within the same year and month fails.""" + valid_df.loc[valid_df.index[0], "Start Month"] = 3.0 + valid_df.loc[valid_df.index[0], "Start Day"] = 20.0 + valid_df.loc[valid_df.index[0], "End Month"] = 3.0 + valid_df.loc[valid_df.index[0], "End Day"] = 10.0 + with pytest.raises(pa.errors.SchemaError, match="Start date inconsistency at the day resolution"): + emdat_schema.validate(valid_df) + +def test_date_consistency_skips_month_when_absent(valid_df): + """Test that missing months do not cause a false positive on the month check.""" + valid_df.loc[valid_df.index[0], "Start Month"] = None + valid_df.loc[valid_df.index[0], "Start Day"] = None + valid_df.loc[valid_df.index[0], "End Month"] = None + valid_df.loc[valid_df.index[0], "End Day"] = None + emdat_schema.validate(valid_df) + +def test_date_consistency_skips_day_when_absent(valid_df): + """Test that missing days do not cause a false positive on the day check.""" + valid_df.loc[valid_df.index[0], "Start Day"] = None + valid_df.loc[valid_df.index[0], "End Day"] = None + emdat_schema.validate(valid_df) + +def test_date_consistency_cross_year_no_false_positive(valid_df): + """Test that a cross-year event passes even when start month > end month.""" + valid_df.index = ["2025-0001-BEL"] + valid_df.index.name = "DisNo." + valid_df.loc[valid_df.index[0], "Start Year"] = 2025 + valid_df.loc[valid_df.index[0], "Start Month"] = 6.0 + valid_df.loc[valid_df.index[0], "End Year"] = 2026 + valid_df.loc[valid_df.index[0], "End Month"] = 3.0 + emdat_schema.validate(valid_df) + def test_lat_lon_consistency(valid_df): """Test that having only one of Latitude/Longitude fails.""" valid_df.loc[valid_df.index[0], "Latitude"] = 50.0 From bf49693626d31ccd713efdc9a36ee940c29ee08b Mon Sep 17 00:00:00 2001 From: Damien Delforge Date: Tue, 28 Apr 2026 11:48:39 +0200 Subject: [PATCH 3/6] Add has_valid_GAUL_codes validation check to README table --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 418babe..22cdb3a 100644 --- a/README.md +++ b/README.md @@ -245,6 +245,7 @@ See [EM-DAT Documentation](https://doc.emdat.be/docs/protocols/economic-adjustme | Total Damage, Adjusted ('000 US$) | greater_than(0.) | Test whether value is greater than 0 | Error | | CPI | in_range(0., 110.) | Test whether value is within range 0-110. | Warning | | Admin Units | is_valid_json | Test whether value is a json string | Error | +| | has_valid_GAUL_codes | Test whether value contains valid GAUL codes | Error | | GADM Admin Units | is_valid_json | Test whether value is a json string | Error | | Entry Date | in_range(1988/1/1, CURRENT_DATE) | Test whether value is within valid date range | Error | | Last Update | in_range(1988/1/1, CURRENT_DATE) | Test whether value is within valid date range | Error | From d8e8270c52562936f55d31de46c3bdf29727c445 Mon Sep 17 00:00:00 2001 From: Damien Delforge Date: Thu, 30 Apr 2026 15:25:42 +0200 Subject: [PATCH 4/6] Bump version to 2026.05.0 and update references in metadata and documentation --- CITATION.cff | 4 ++-- README.md | 8 ++++---- emtest/__init__.py | 2 +- pyproject.toml | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/CITATION.cff b/CITATION.cff index 703d191..0536a84 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -19,6 +19,6 @@ keywords: - testing-framework - python license: MIT -version: 2026.04.0 -date-released: '2026-04-08' +version: 2026.05.0 +date-released: '2026-04-30' diff --git a/README.md b/README.md index 22cdb3a..c856c2e 100644 --- a/README.md +++ b/README.md @@ -3,14 +3,14 @@ [![Tests](https://github.com/em-dat/em-test/actions/workflows/tests.yml/badge.svg)](https://github.com/em-dat/em-test/actions/workflows/tests.yml) ![Python Versions](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12%20%7C%203.13%20%7C%203.14-blue) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) -[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.14275790.svg)](https://doi.org/10.5281/zenodo.14275790) +[![DOI](https://img.shields.io/badge/DOI-10.5281%2Fzenodo.14275790-blue)](https://doi.org/10.5281/zenodo.14275790) EM-TEST is a testing framework for [EM-DAT](https://www.emdat.be/) public data, built on the [`pandas`](https://pandas.pydata.org/) and [`pandera`](https://pandera.readthedocs.io/en/stable/) Python packages. > [!IMPORTANT] -> This version of EM-TEST has been built for EM-DAT public data on 2026/04/08. +> This version of EM-TEST has been built for EM-DAT public data on 2026/04/30. > Some tests might fail for versions prior to this date. EM-TEST is not > suitable for EM-DAT versions prior to September 26, 2023. @@ -294,7 +294,7 @@ citation below or the citation metadata file `CITATION.cff`. month = dec, year = 2026, publisher = {Zenodo}, - version = {2026.04.0}, + version = {2026.05.0}, doi = {10.5281/zenodo.14275790}, url = {https://doi.org/10.5281/zenodo.14275790} } @@ -302,7 +302,7 @@ citation below or the citation metadata file `CITATION.cff`. Or >Delforge, D., & Wathelet, V. (2026). EM-TEST: A Testing Framework for the -> EM-DAT Data (2026.04.0). Zenodo. https://doi.org/10.5281/zenodo.14275790 +> EM-DAT Data (2026.05.0). Zenodo. https://doi.org/10.5281/zenodo.14275790 ## Useful Links diff --git a/emtest/__init__.py b/emtest/__init__.py index 88fbadb..7df56da 100644 --- a/emtest/__init__.py +++ b/emtest/__init__.py @@ -1,4 +1,4 @@ -__version__ = '2026.04.0' +__version__ = '2026.05.0' __author__ = 'Damien Delforge, Valentin Wathelet' from emtest.validation_schemas import ( diff --git a/pyproject.toml b/pyproject.toml index 3186466..fb14069 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta" [project] name = "emtest" -version = "2026.04.0" +version = "2026.05.0" description = "Data validation for EM-DAT datasets." readme = { file = "README.md", content-type = "text/markdown" } requires-python = ">=3.9,<4" From 28490f4ef0199ddbb44a5f4f28e2d3e50208c4b6 Mon Sep 17 00:00:00 2001 From: Damien Delforge Date: Thu, 30 Apr 2026 15:32:16 +0200 Subject: [PATCH 5/6] Expand CI workflow to include version branch testing for `dev/v2026.05`. --- .github/workflows/tests.yml | 4 ++-- checklist.md | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 4bc49bd..dc98ed5 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -2,9 +2,9 @@ name: Tests on: push: - branches: [ main, master, dev/v2026.01 ] + branches: [ main, master, dev/v2026.01, dev/v2026.05 ] pull_request: - branches: [ main, master, dev/v2026.01 ] + branches: [ main, master, dev/v2026.01, dev/v2026.05 ] jobs: test: diff --git a/checklist.md b/checklist.md index c5ad397..f59d402 100644 --- a/checklist.md +++ b/checklist.md @@ -28,6 +28,7 @@ This checklist outlines the items to check, modify, and do before finalizing the ## 5. Testing and CI/CD - [ ] Run the full test suite using `pytest`. +- [ ] Expand CI workflow to include version branch testing. - [ ] Ensure all GitHub Actions workflows are passing. - [ ] Add regression tests for any bugs fixed in this release cycle. - [ ] Verify that the package can be built successfully (`python -m build`). From f04f4acf4bffa63041f88cd4eef53550610a4565 Mon Sep 17 00:00:00 2001 From: Damien Delforge Date: Thu, 30 Apr 2026 15:36:27 +0200 Subject: [PATCH 6/6] Update CI workflow: bump `actions/checkout` to v5 and `setup-python` to v6 --- .github/workflows/tests.yml | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index dc98ed5..1090fb8 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -9,18 +9,16 @@ on: jobs: test: runs-on: ubuntu-latest - env: - FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true strategy: fail-fast: false matrix: python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"] steps: - - uses: actions/checkout@v4 - + - uses: actions/checkout@v5 + - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v5 + uses: actions/setup-python@v6 with: python-version: ${{ matrix.python-version }} allow-prereleases: true