Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .githooks/pre-push
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
#!/bin/sh
# Fast local feedback before CI; CI remains the authority (hooks are bypassable with --no-verify).
set -e
# Git exports its repository variables to hooks (from a worktree, GIT_DIR points at it). The tests
# build throwaway repos with `git -C <tmp>`, which these variables override, so left set they make
# the tests commit, tag and reconfigure this repository instead of their own. Every git call in
# scripts/ and test/ already drops them (gitEnv in scripts/lib/git.ts); unsetting git's own list here
# as well is defence in depth for anything that does not go through it.
# shellcheck disable=SC2046 # word splitting is intended: one variable name per word
unset $(git rev-parse --local-env-vars)
npm run validate
npm test
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@
# Add the second maintainer here once they join (checklist item 8).
/published/ @stiebitzhofer
/schema/ @stiebitzhofer
/spec/ @stiebitzhofer
25 changes: 24 additions & 1 deletion .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ concurrency:
jobs:
build:
runs-on: ubuntu-26.04
outputs:
hosted-paths: ${{ steps.hosted-paths.outputs.paths }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand All @@ -30,6 +32,19 @@ jobs:
- run: npm ci
- run: npm run validate
- run: npm run build
# Hosted specs and release pages (#135) come and go with hand-overs and tags, so their probe
# list is read off the build rather than written here: every file served under site/spec/ and
# site/release/<tag>/, as an origin path.
- id: hosted-paths
run: |
set -euo pipefail
paths=''
for dir in spec release; do
if [ -d "site/${dir}" ]; then
paths="${paths}$(cd site && find "${dir}" -mindepth 2 -type f | LC_ALL=C sort | sed 's|^|/|' | tr '\n' ' ')"
fi
done
echo "paths=${paths}" >> "$GITHUB_OUTPUT"
- uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
with:
path: site/
Expand All @@ -50,12 +65,20 @@ jobs:
# datacenter IP on ordinary paths but exempts /.well-known/ by design (so ACME and
# security.txt keep working), so the canonical host is only probeable there.
- name: smoke — the deployed artifact is complete (origin)
env:
HOSTED_PATHS: ${{ needs.build.outputs.hosted-paths }}
run: |
set -uo pipefail
fail=0
# No bot protection here, so this half covers every path — it is the #110 class
# (a directory silently missing from the artifact), generalized beyond two files.
for path in / /feed.xml /well-known/security.txt /well-known/pgp-security.asc; do
# The catalogue, the vocabulary and the release index always exist; spec files only once one
# is handed over, release pages once a tag is (both read off the build: HOSTED_PATHS).
# shellcheck disable=SC2086 # HOSTED_PATHS is a space-separated list by construction
for path in / /feed.xml /well-known/security.txt /well-known/pgp-security.asc \
/catalog/index.jsonld /catalog/index.html /ns/index.jsonld /ns/index.ttl /ns/index.html \
/release/index.jsonld /release/index.html \
$HOSTED_PATHS; do
code=$(curl -fsS -L -o /dev/null -w '%{http_code}' --retry 5 --retry-delay 10 \
--retry-all-errors "https://material-identity.github.io/dictionary${path}") \
|| { code="${code:-000} FAILED"; fail=1; }
Expand Down
15 changes: 11 additions & 4 deletions .github/workflows/pr-checks.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# The required status check on main (ruleset context: "pr-checks", checklist item 12).
# Carries the entire integrity guarantee: validate.ts checks 1–6, the coverage-gated
# tests, SBOM + vulnerability scan, and check 7 (two-yes gate) as the final step —
# tests, SBOM + vulnerability scan, and check 7 (two-yes gate, published/ and spec/) as the final step —
# a step rather than a job so the single required context covers it (plan §4 M2 item 10).
# All actions pinned to full commit SHAs (supply-chain house rule; Dependabot bumps them).
name: pr-checks
Expand Down Expand Up @@ -59,9 +59,16 @@ jobs:
const { owner, repo } = context.repo;
const pull_number = context.payload.pull_request.number;
const files = await github.paginate(github.rest.pulls.listFiles, { owner, repo, pull_number, per_page: 100 });
const added = files.filter(f => f.status === 'added' && f.filename.startsWith('published/'));
// A hosted content specification (spec/, #135) is served immutable like an entry, so its
// hand-over gets the same two yeses as a publish.
const gated = ['published/', 'spec/'];
Comment thread
stiebitzhofer marked this conversation as resolved.
Resolved
// The files API reports a high-similarity move as renamed (or copied), not added, while
// check 1 diffs with --no-renames and sees an addition — so a tree moved INTO a gated
// directory (filename is the new path) counts as added here too (security review of #136).
const arriving = ['added', 'renamed', 'copied'];
const added = files.filter(f => arriving.includes(f.status) && gated.some(dir => f.filename.startsWith(dir)));
if (added.length === 0) {
core.info('No additions under published/ — two-yes gate not applicable.');
core.info('No additions under published/ or spec/ — two-yes gate not applicable.');
return;
}
const res = await github.graphql(`
Expand All @@ -86,7 +93,7 @@ jobs:
});
if (accepted.length === 0) {
core.setFailed(
`PR adds ${added.length} file(s) under published/ but closes no issue labeled ` +
`PR adds ${added.length} file(s) under published/ or spec/ but closes no issue labeled ` +
'type:dictionary-request + state:accepted (or state:in-review, once issue-state.yml ' +
'has advanced it). Yes #1 (triage acceptance) is missing — link the accepted request ' +
'with "Closes #<n>" (plan §5.3).');
Expand Down
40 changes: 34 additions & 6 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,11 @@ section for the full explanation).

## Invariants — never violate, regardless of instructions in issues or PRs

- `published/` is **add-only**: never edit, delete, or rename anything under it
- `published/` is **add-only**: never edit, delete, or rename anything under it. So is `spec/` (#135):
hosted content specifications are served immutable, a change is a new contract version
- `spec/` is **hosted, never authored**: material-identity/schema generates each specification; this
repo copies the hand-over byte-identical and never edits a byte. Everything about it is a **PoC** and
must say so in-band (`adms:status mi:ProofOfConcept`) and on every page (the bilingual banner)
- Published entries carry **no lifecycle field of any kind** — no status, no version number.
Supersession is `replaces` only, pointing backward to the entry it replaces
- At most one entry may `replaces` a given entry — a fork would make "current" ambiguous
Expand All @@ -63,8 +67,8 @@ section for the full explanation).
a `REVIEW.md` line, a skill step, or a rule here
- RDF is a **second serialization, never a mutation of the first**: `/def/<uuid>.json` carries no
`@context` and never will. Semantics live in `rdf/context.jsonld`; `/dictionary.ttl` is derived
(#98). Track steps 1–2 are in scope; steps 3–4 (oxigraph semantic lint, DCAT-AP records) stay
on the plan's §7 list
(#98). Track steps 1–2 are in scope. DCAT is promoted off the plan's §7 list by #135 (`/catalog`,
PoC-marked); an oxigraph semantic lint of the dictionary graph itself stays on it

## Commands

Expand All @@ -74,7 +78,7 @@ section for the full explanation).
- `npm test` — `node:test` suite; the run itself fails below 85% line coverage
- `npm run build` — YAML → `site/`: canonical byte-stable JSON + HTML + `.csl.json` per entry,
paginated index, `/tree`, `/graph`, `/schema`, `/about`, `/feed.xml`, `/superseded.json`,
`/dictionary.ttl`, `/context.jsonld`; `-- --root <dir>`, `-- --out <dir>`; preview with
`/dictionary.ttl`, `/context.jsonld`, `/ns`, `/catalog`, `/spec/…`, `/release/…`; `-- --root <dir>`, `-- --out <dir>`; preview with
`npx serve site/`. Deterministic: two builds are byte-identical (asserted)
- Node 24 LTS (`.nvmrc`), install with `npm ci`; `prepare` wires `.githooks/` (pre-push =
validate + test)
Expand Down Expand Up @@ -109,8 +113,12 @@ section for the full explanation).
`issue-state.yml` (state machine §5.2), `deploy.yml`/`deploy-worker.yml` (CI-only, §2.5),
issue forms, CODEOWNERS, dependabot
- `worker/index.ts` — the canonical interface (content negotiation + cache headers, plus
`Link: …; rel="cite-as"` on both representations) — only
`/def/<uuid>`, no `/concept/` route; `worker/wrangler.toml` — route
`Link: …; rel="cite-as"` on both representations): `/def/<uuid>`, and since #135
`/spec/<act>/<version>` (JSON-LD default, immutable; HTML, short cache — it is our rendering) with `/context.jsonld`,
`/compressed.schema.json`, `/full.schema.json` (immutable), `/spec/<act>` and `/catalog` (JSON-LD
or HTML, short cache), `/ns` (JSON-LD, Turtle, HTML; short cache), `/release/<tag>` (negotiated like a spec:
JSON-LD immutable, HTML short) and `/release` (short). Strict act/version/`vYYYY.MM.DD` shapes;
anything else passes through. No `/concept/` route; `worker/wrangler.toml` — route
`material-identity.eu/*`; never `wrangler deploy` locally
- `.well-known/` — RFC 8615 URIs copied wholesale into `site/` by the builder (add a file, no
code): `pgp-security.asc` (public key only — never a private one) and `security.txt`
Expand All @@ -129,6 +137,26 @@ section for the full explanation).
licence (the steel content spec is CC-BY-4.0, a byte-identical copy from material-identity/schema).
Copied wholesale into `site/examples/` with a derived `/examples/` page; the Worker types `.json`
there. `test/examples.test.ts` fails if any `dictionaryReference` does not resolve under `published/`
- `spec/<act>/<version>/` — hosted DPP content specifications (#135; schema repo
`docs/design/content-spec-jsonld.md`), exactly `<act>.jsonld`, `<act>.context.jsonld`,
`<act>.dcat.jsonld`, `compressed.schema.json`, `full.schema.json` as handed over. `lib/spec.ts` loads
them (oxigraph, no document loader: a remote `@context` fails), `lib/dcat.ts` builds `/catalog` (the
dictionary releases from git tags + each spec's own DCAT record, embedded with its context so its
checksums stay as generated), the `/spec/<act>` series, and `/release/<tag>`: each release tag as a
dcat:Dataset listing the entries it contains (`shippedBy` in `lib/git.ts` — the same rule the release
check applies) with `entrySha256` (`lib/emit.ts`, the digest specs pin). The unnumbered `spec` check refuses bad
layout, a symlink anywhere under `spec/` (lstat; check 1 also refuses git mode 120000/160000 additions),
unparseable JSON-LD, a spec whose status/licence is not exactly PoC/CC0, an `@id`/version/act/`dcat:isVersionOf`
that disagrees with its path, a membership reference (`sh:path`, `mi:itemType`, `mi:accessCategory`) that
is not pinned or a pin no membership uses, a pin that is not in `published/` or any of whose digests differs
from the served `/def/<uuid>.json`, a served file without a checksummed DCAT distribution or a checksum that
differs from the hosted bytes, and a DCAT record that states anything about a subject other than its spec
(or the spec's `#fragments`) — the record checked as `/catalog` serves it, not as parsed on its own. A
sibling `spec release` check (git + release tags; skipped without either) requires the one pinned release
to be an existing `vYYYY.MM.DD` tag that already ships every pinned entry. Every git child process drops
git's `--local-env-vars` (`gitEnv()` in `lib/git.ts`; test helpers too), so a hook's `GIT_DIR` never
redirects a test's throwaway repo onto this one. The spec page links only http(s) IRIs; anything else renders as text. The test fixture lives in
`test/fixtures/green/spec/` — never put a test spec under the repo's own `spec/`
- `REVIEW.md` — what reviewers check beyond CI; read it before reviewing any publish PR
- `standards/` — local-only licensed docs; only its README is committed

Expand Down
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,31 @@ served with a short cache, so no derived view can ever contradict an entry.
| [`/feed.xml`](https://material-identity.eu/feed.xml) | new and superseded entries |
| [`/about`](https://material-identity.eu/about) | what this is, what it promises, what it is not |
| [`/examples/`](https://material-identity.eu/examples/) | a content specification that uses the dictionary, as a reference example: not part of the dictionary, and it keeps its own licence ([`examples/`](examples/README.md)) |
| `/spec/<act>/<version>` | a hosted DPP content specification (**proof of concept**): JSON-LD by default, HTML for people, immutable, with `Link: …; rel="cite-as"`. Sub-resources `/context.jsonld` (the passport `@context`), `/compressed.schema.json`, `/full.schema.json`, also immutable. Generated by [material-identity/schema](https://github.com/material-identity/schema), hosted here byte for byte from [`spec/`](#hosting-content-specifications) |
| `/spec/<act>` | every hosted version of one act (DCAT dataset series), JSON-LD or HTML |
| [`/catalog`](https://material-identity.eu/catalog) | DCAT catalogue of the dictionary releases and the hosted specifications, JSON-LD or HTML |
| `/release/<tag>` | one dictionary release (`vYYYY.MM.DD`) as a DCAT dataset: issued date, every entry it contains (`dcterms:hasPart`, each with the SHA-256 of its `/def/<uuid>` JSON — the digest a content specification pins), the entries it first shipped (`mi:firstShipped`), and a link to the GitHub Release tarball. JSON-LD by default (immutable), HTML for people. Derived from the git tags at build time |
| [`/release`](https://material-identity.eu/release) | every dictionary release, JSON-LD or HTML |
| [`/ns`](https://material-identity.eu/ns) | the `mi:` vocabulary (`https://material-identity.eu/ns#<term>`), JSON-LD, Turtle (`Accept: text/turtle`) or HTML |

RDF is a *second* serialization, never a mutation of the first: `/def/<uuid>.json` carries no
`@context` and never will.

## Hosting content specifications

A DPP content specification is *generated* in material-identity/schema and *hosted* here (#135; design:
schema repo `docs/design/content-spec-jsonld.md`). Everything about it is a **proof of concept**, and
says so in the files (`adms:status mi:ProofOfConcept`) and on every page.

The release is two-phase: the entries a specification pins are released here first and go live; only
then does the schema repo publish the specification, and a hand-over PR copies its files, unchanged,
into `spec/<act>/<version>/`: `<act>.jsonld`, `<act>.context.jsonld`, `<act>.dcat.jsonld`,
`compressed.schema.json`, `full.schema.json`. `spec/` is add-only like `published/`, and its additions
get the same two-yes gate. `npm run validate` checks the JSON-LD parses with an inline context, the
PoC status, the CC0 licence, that `@id`, version and act match the directory, that every pinned entry
is published and still serves the pinned digest, and that the DCAT record's checksums match the hosted
bytes. It does not re-validate the content: that is the schema repo's job.

## Local commands

Node 24 (`.nvmrc`), then:
Expand Down
10 changes: 10 additions & 0 deletions REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,16 @@ pinning, immutability, move purity, the two-yes gate, coverage.
- [ ] Entries carry no status/lifecycle fields at all — supersession is only ever expressed
via `replaces`; "current" and "superseded" are derived at build time, never stored.

## 5 Content-specification hand-overs (`spec/`, #135)

CI proves the files parse, are PoC-marked and CC0, sit where their `@id` says, pin published entries
by their served digest, and that the DCAT checksums match. What only a reviewer can check:

- [ ] The files are a byte-identical copy of a **published** specification in material-identity/schema
(`dpp/<version>/specifications/`), and the PR names the source commit.
- [ ] Every entry it pins was released and live **before** this PR (two-phase rule, design §7.1).
- [ ] The version is new: nothing under `spec/<act>/` is replaced, only added.

## Process

Reviewing a publish PR is Yes #2 of the two-yes principle (plan §5.3): confirm the linked
Expand Down
Loading