Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,5 @@ data/*.csv linguist-generated=true
data/*.json linguist-generated=true
data/*.xlsx binary linguist-generated=true
data/*.sqlite binary linguist-generated=true
legacy/** linguist-vendored=true
baselines/** linguist-vendored=true
sources/snapshots/** binary linguist-vendored=true
62 changes: 52 additions & 10 deletions .github/workflows/data-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ jobs:
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
Expand All @@ -28,24 +30,50 @@ jobs:
cache-dependency-path: |
requirements.txt
requirements-dev.txt
requirements-geography.txt

- name: Install runtime and quality dependencies
run: python -m pip install -r requirements.txt -r requirements-dev.txt
run: >-
python -m pip install
-r requirements.txt
-r requirements-dev.txt
-r requirements-geography.txt

- name: Lint Python sources and tests
run: ruff check scripts tests

- name: Type-check all maintained Python scripts
run: mypy

- name: Require immutable GitHub Action references
run: python scripts/check_workflow_pins.py

- name: Validate the schema 4 typed preview
run: python scripts/validate_typed_contract.py

- name: Audit pinned Python dependencies
run: >-
pip-audit
-r requirements.txt
-r requirements-dev.txt
-r requirements-geography.txt

- name: Enforce maintained-script test coverage
run: |
coverage run -m unittest discover -s tests
coverage run --append scripts/build_dataset.py
coverage run --append scripts/check_determinism.py
coverage run --append scripts/validate_dataset.py
coverage run --append scripts/build_pages.py
coverage run --append scripts/build_release.py
coverage run --append scripts/validate_release.py
coverage run --append scripts/build_geography.py --check
coverage run --append scripts/build_geography.py --check --require-rebuild
coverage run --append scripts/validate_typed_contract.py
coverage run --append scripts/export_typed_json.py \
--output "${RUNNER_TEMP}/italian_locations.typed.json" \
--sqlite-output "${RUNNER_TEMP}/italian_locations.typed.sqlite"
coverage run --append scripts/reconciliation_backlog.py
coverage run --append scripts/check_workflow_pins.py
coverage report
coverage xml

Expand All @@ -56,19 +84,25 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.11", "3.12", "3.13"]
python-version: ["3.11", "3.12", "3.13", "3.14"]
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
cache: pip

- name: Install pipeline dependency
run: python -m pip install -r requirements.txt
- name: Install pipeline and test dependencies
run: >-
python -m pip install
-r requirements.txt
-r requirements-dev.txt
-r requirements-geography.txt

- name: Validate dataset
run: python scripts/validate_dataset.py
Expand All @@ -83,15 +117,21 @@ jobs:
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
cache: pip

- name: Install pipeline dependency
run: python -m pip install -r requirements.txt
- name: Install pipeline and geographic dependencies
run: >-
python -m pip install
-r requirements.txt
-r requirements-dev.txt
-r requirements-geography.txt

- name: Validate committed data and quality contracts
run: python scripts/validate_dataset.py
Expand All @@ -115,8 +155,11 @@ jobs:
- name: Test Pages search and map logic
run: node --test tests/pages_core.test.mjs

- name: Verify committed geographic base
run: python scripts/build_geography.py --check
- name: Rebuild and verify committed geographic base
run: python scripts/build_geography.py --check --require-rebuild

- name: Validate the schema 4 typed preview
run: python scripts/validate_typed_contract.py

- name: Build deterministic GitHub Pages site
run: python scripts/build_pages.py
Expand Down Expand Up @@ -146,7 +189,6 @@ jobs:
":(exclude)data/italian_locations.sqlite"
reports/release-diff.json
reports/build-metadata.json
reports/legacy-comparison.json
reports/reconciliation-backlog.json
reports/export-manifest.json
reports/determinism.json
Expand Down
29 changes: 22 additions & 7 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,15 @@ on:
- "assets/social-preview.jpg"
- "data/italian_locations.csv"
- "project.json"
- "requirements-geography.txt"
- "scripts/build_pages.py"
- "scripts/build_geography.py"
- "scripts/pages_model.py"
- "scripts/project_metadata.py"
- "scripts/validate_site.py"
- "site/**"
- "sources/cache/Limiti01012026_g.zip"
- "sources/manifest.json"
- "tests/pages_core.test.mjs"
- "tests/test_pages.py"
push:
Expand All @@ -21,18 +26,21 @@ on:
- "assets/social-preview.jpg"
- "data/italian_locations.csv"
- "project.json"
- "requirements-geography.txt"
- "scripts/build_pages.py"
- "scripts/build_geography.py"
- "scripts/pages_model.py"
- "scripts/project_metadata.py"
- "scripts/validate_site.py"
- "site/**"
- "sources/cache/Limiti01012026_g.zip"
- "sources/manifest.json"
- "tests/pages_core.test.mjs"
- "tests/test_pages.py"
workflow_dispatch:

permissions:
contents: read
pages: write
id-token: write

concurrency:
group: >-
Expand All @@ -48,33 +56,37 @@ jobs:
name: Build and test site
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"

- name: Install geographic build dependencies
run: python -m pip install -r requirements-geography.txt

- name: Test search and map logic
run: node --test tests/pages_core.test.mjs

- name: Test deterministic Pages build
run: python -m unittest discover -s tests -p "test_pages.py" -v

- name: Verify committed geographic base
run: python scripts/build_geography.py --check
- name: Rebuild and verify committed geographic base
run: python scripts/build_geography.py --check --require-rebuild

- name: Build GitHub Pages artifact
run: python scripts/build_pages.py --output _site

- name: Validate accessibility, local links and performance budgets
run: python scripts/validate_site.py _site

- name: Configure GitHub Pages
uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0

- name: Upload GitHub Pages artifact
uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
with:
Expand All @@ -89,6 +101,9 @@ jobs:
runs-on: ubuntu-latest
needs: build
timeout-minutes: 10
permissions:
id-token: write
pages: write
steps:
- name: Deploy GitHub Pages
id: deployment
Expand Down
14 changes: 12 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,12 @@ jobs:
python-version: "3.12"
cache: pip

- name: Install pipeline dependency
run: python -m pip install -r requirements.txt
- name: Install pipeline and validation dependencies
run: >-
python -m pip install
-r requirements.txt
-r requirements-dev.txt
-r requirements-geography.txt

- name: Require tag to match project metadata
run: >-
Expand All @@ -59,6 +63,12 @@ jobs:
- name: Run tests
run: python -m unittest discover -s tests -v

- name: Rebuild and verify committed geographic base
run: python scripts/build_geography.py --check --require-rebuild

- name: Validate the schema 4 typed preview
run: python scripts/validate_typed_contract.py

- name: Build release directory
run: >-
python scripts/build_release.py
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
sources/cache/*
!sources/cache/.gitkeep
!sources/cache/Limiti01012026_g.zip
.DS_Store
.venv/
__pycache__/
Expand Down
16 changes: 10 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Changelog

## v2.0.0 — prerelease candidate, not yet published
## v2.0.0 — clean-room prerelease

### Breaking

Expand All @@ -17,17 +17,16 @@
without a GeoNames postal-code match.
- Many-to-many postal relations, explicit reconciliation outcomes, source
record identities, source dates, confidence and GeoNames `accuracy`.
- Immutable GeoNames snapshot, checksum gate, attribution and
`reports/legacy-comparison.json`.
- Immutable GeoNames snapshot, checksum gate and attribution.
- Separate `structural_quality=passed` and
`operational_data_readiness=experimental_non_official`.
- Clean-room warnings and GeoNames attribution in JSON, XLSX, SQLite, Pages
and release notes.

### Changed

- Legacy data is historical comparison material only and no longer
contributes to current outputs or Pages.
- Pre-clean-room material with unresolved redistribution rights was withdrawn
from the current tree, public history and affected release assets.
- Coordinates are labelled `geonames_place_match`,
`geonames_estimated` or `missing`; none are official.
- Pages uses absolute counts rather than a potentially misleading coverage
Expand All @@ -40,10 +39,13 @@
- 10,320 GeoNames records are not reconciled with a municipality.
- GeoNames still contains 76 records using historical province code `SU`;
they remain unparented instead of being remapped automatically.
- The `v2.0.0` tag and GitHub release await maintainer confirmation.
- Release assets remain experimental and non-official.

## v1.1.0 — pre-release

> Withdrawn on 30 July 2026: downloadable data assets were removed because
> provenance and redistribution rights could not be established.

### Added

- `project.json` come unica fonte per versione dataset, schema, quality gate,
Expand Down Expand Up @@ -88,6 +90,8 @@

## v1.0.0 — baseline storica

> Withdrawn on 30 July 2026 for the same provenance and rights reason.

- Prima distribuzione riproducibile con CSV, JSON, XLSX, SQLite, SQL e
checksum.
- Pipeline, quality gate, GitHub Pages, issue form e documentazione iniziali.
Expand Down
7 changes: 4 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ Non modificare manualmente file sotto `data/`, SQL di release o report
generati. Ogni correzione parte da una fonte dichiarata o da una regola
riproducibile.

Non modificare o eliminare `legacy/`, i tag storici o le baseline di release.
Non reintrodurre materiale storico ritirato, copie dei relativi asset o
baseline anteriori alla ricostruzione clean-room.

## Correzione dati

Expand Down Expand Up @@ -59,8 +60,8 @@ Verificare che:
- `structural_quality` sia `passed`;
- `operational_data_readiness` resti `experimental_non_official`;
- due build siano identiche;
- nessun `source_ids` canonico contenga `legacy_csv`;
- il report legacy sia limitato, deterministico e non affermi provenienza;
- nessun `source_ids` canonico contenga identificativi di fonti ritirate;
- nessun input, report o asset ripristini materiale storico ritirato;
- la Pages mostri warning e attribuzione GeoNames.

## Pull request
Expand Down
Loading