Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: Validation

on:
pull_request:
branches:
- main
push:
branches:
- main
workflow_dispatch:

permissions:
contents: read

jobs:
validate:
name: Python ${{ matrix.python-version }}
runs-on: ubuntu-latest
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
python-version:
- "3.11"
- "3.12"

steps:
- name: Check out the tested revision
uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.sha || github.sha }}

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Confirm the tested SHA
env:
EXPECTED_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
run: |
actual_sha=$(git rev-parse HEAD)
test "$actual_sha" = "$EXPECTED_SHA"
printf 'Tested SHA: %s\n' "$actual_sha"

- name: Validate skill packages
run: ./scripts/check_all_skills.sh

- name: Run deterministic angze-code-style fixtures
run: ./scripts/run_fixture_checks.sh angze-code-style

- name: Verify v0.4.0 release integrity
run: >-
python3 scripts/check_release_integrity.py
docs/evaluations/angze-code-style-v0.4.0.manifest.json
--expected-runtime-sha
7f79b22c4694717963966132967b1a03b4cf5790529e6c229ef2ef31920fad84
--expected-skill angze-code-style
--expected-version 0.4.0
--forbid-invocation '$scientific-code-documenter'

- name: Check changed whitespace
env:
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
PUSH_BEFORE_SHA: ${{ github.event.before }}
run: |
if [ "$GITHUB_EVENT_NAME" = "pull_request" ]; then
git diff --check "$PR_BASE_SHA...HEAD"
elif [ "$GITHUB_EVENT_NAME" = "push" ] &&
[ -n "$PUSH_BEFORE_SHA" ] &&
[ "$PUSH_BEFORE_SHA" != "0000000000000000000000000000000000000000" ]; then
git diff --check "$PUSH_BEFORE_SHA..HEAD"
else
git diff-tree --check --no-commit-id --root -r HEAD
fi

- name: Confirm checks did not modify tracked files
run: git diff --exit-code
35 changes: 35 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,41 @@ All notable changes to this collection are documented here. No remote release ha

No changes yet.

## [0.4.0] - 2026-08-28

### Added

- Scientific and artifact contract inventory with `evidence-backed`, `observable-only`, and `unknown` classifications.
- Tier 0–3 risk grading and standardized review findings.
- First-class conservative Jupyter Notebook policy, role classification, and progressive references.
- Dependency-free notebook structural/locality validator, deterministic synthetic probe, four notebook fixtures, and hard-gate tests.
- Comment-pruning rules that consolidate real data contracts instead of adding narration or erasing useful context.
- Fresh-pass idempotence checks and dirty-target preservation.

### Changed

- Renamed the active skill from `scientific-code-documenter` to `angze-code-style` as a clean break; the old invocation is not retained as an alias.
- Runtime behavior now covers scientific contracts, notebook artifacts, pruning, and explicit preservation boundaries.
- Controlled A/B evaluation now includes notebooks plus Python, Shell, LaTeX, and HTML regressions.
- Historical v0.1–v0.3 results remain predecessor evidence rather than acceptance of the changed runtime.

### Validated

- Six editing fixtures averaged 17.5/18, with six skill wins and behavior preservation at 3/3 throughout.
- Editable, review-only, pruning, and generated notebook hard gates, including exact probe output and byte-level locality.
- Six accepted editing candidates remained byte-identical on fresh second passes.
- Positive implicit triggering at 10/10 and false triggering at 1/10.
- Full post-rename trigger rerun at 10/10 positives and 0/10 negatives, plus one renamed editable-notebook validation and byte-identical fresh second pass.
- Authorized immutable real tutorial-notebook acceptance: zero-edit review, controlled Markdown-only A/B, strict notebook validation, unchanged executable sources, and byte-identical fresh-session idempotence.

### Known limitations

- One notebook feature-development prompt falsely triggered the skill in the original matrix; the post-rename rerun did not reproduce it.
- Manual A/B scoring was not independently blind.
- ShellCheck and rendered LaTeX compilation were unavailable in the original local acceptance environment. Deterministic Shell contracts and static LaTeX contracts passed. ShellCheck subsequently ran and passed under the existing policy in remote CI on Ubuntu for Python 3.11 and 3.12; rendered LaTeX compilation remains a disclosed, non-blocking confidence-depth limitation.
- Deterministic fixtures and bounded Markdown-only real-notebook evidence do not prove general notebook equivalence or scientific correctness.
- Human scientific review remains required; the release remains experimental.

## [0.3.0] - 2026-07-29

### Added
Expand Down
23 changes: 16 additions & 7 deletions GETTING_STARTED.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Getting started

This guide shows how to install and use the `scientific-code-documenter` skill in Codex. The skill helps review and improve scientific or technical code documentation and readability without intentionally changing behavior.
This guide shows how to install and use the `angze-code-style` skill in Codex. The skill helps review, document, and prune scientific code or Jupyter notebooks while preserving explicit scientific, data, interface, execution-state, and artifact contracts.

## What you need

Expand All @@ -15,7 +15,7 @@ Open a new Codex task and paste this prompt:

```text
$skill-installer
Install https://github.com/angzeli/codex-skills/tree/main/skills/scientific-code-documenter
Install https://github.com/angzeli/codex-skills/tree/main/skills/angze-code-style
```

The skill should be available on your next turn. If it does not appear, start a new task or restart Codex.
Expand All @@ -25,20 +25,29 @@ The skill should be available on your next turn. If it does not appear, start a
For a review that does not modify files:

```text
$scientific-code-documenter
$angze-code-style
Review this file for readability and documentation quality. Do not modify it.
```

For focused documentation and readability improvements:

```text
$scientific-code-documenter
$angze-code-style
Apply only high-confidence documentation and readability improvements to this file. Preserve calculations, public interfaces, file formats, and numerical behavior.
```

Attach the file, open its repository in Codex, or name the file path in the prompt. Codex may also select the skill automatically when a request clearly matches its purpose.

Always review scientific edits before accepting them. The skill preserves behavior by design, but it cannot independently confirm undocumented scientific meaning or every environment-specific interaction.
For a conservative notebook review:

```text
$angze-code-style
Review this notebook's role, scientific and data contracts, execution state, outputs, and documentation risks. Do not modify or execute it.
```

For an authorized notebook documentation edit, name the permitted cells or source fields and the protected state. The skill does not execute notebooks, clear outputs, add or reorder cells, or normalize notebook JSON by default.

Always review scientific edits before accepting them. The skill preserves behavior by design, but it cannot independently confirm undocumented scientific meaning, general notebook equivalence, stored-output validity, or every environment-specific interaction.

## Manual installation from a clone

Expand All @@ -47,7 +56,7 @@ This alternative is useful for contributors or anyone who wants updates in a loc
```sh
git clone https://github.com/angzeli/codex-skills.git
cd codex-skills
./scripts/install_skill.sh scientific-code-documenter
./scripts/install_skill.sh angze-code-style
```

The script validates the skill and links it into `$HOME/.agents/skills`. Start a new Codex task after the first installation. To update later, pull the latest repository changes; the link will continue to use the checked-out skill.
Expand All @@ -56,7 +65,7 @@ The script validates the skill and links it into `$HOME/.agents/skills`. Start a

- **The skill is not listed:** Start a new task or restart Codex after installation.
- **GitHub access fails:** Confirm that you can open or clone the repository, especially if it is private.
- **The manual installer will not run:** From the repository root, try `bash ./scripts/install_skill.sh scientific-code-documenter`.
- **The manual installer will not run:** From the repository root, try `bash ./scripts/install_skill.sh angze-code-style`.
- **You only want feedback:** Include `Do not modify files` in the prompt.

For maintainer commands, validation details, and evaluation status, see the main [README](README.md).
39 changes: 24 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This repository is the editable source of truth for a growing collection of custom Codex skills. Each direct child of `skills/` is an independent runtime package; collection documentation, evaluation suites, and shared tooling stay outside those packages.

New to Codex skills? Follow [Getting started](GETTING_STARTED.md) to install and use `scientific-code-documenter`.
New to Codex skills? Follow [Getting started](GETTING_STARTED.md) to install and use `angze-code-style`.

## Source and runtime installations

Expand All @@ -12,13 +12,16 @@ Edit skills in this repository. Runtime discovery uses one symbolic link per ski
codex_skills/
├── skills/
│ ├── README.md
│ └── scientific-code-documenter/
│ └── angze-code-style/
│ ├── SKILL.md
│ ├── agents/openai.yaml
│ └── references/comment-examples.md
│ └── references/
│ ├── comment-examples.md
│ ├── jupyter-notebooks.md
│ └── scientific-contracts.md
├── evals/
│ ├── README.md
│ └── scientific-code-documenter/
│ └── angze-code-style/
│ ├── fixtures/
│ ├── results/
│ ├── checks.sh
Expand All @@ -37,20 +40,20 @@ codex_skills/

| Skill | Purpose | Status |
| --- | --- | --- |
| `scientific-code-documenter` | Improve scientific and technical code documentation and readability while preserving behavior | Experimental v0.3.0; validated with scientific Python and ORCA Shell real-repository acceptance plus controlled Python, Shell, LaTeX, HTML/CSS/JavaScript, trigger, and boundary tests |
| `angze-code-style` | Document and prune scientific code and Jupyter notebooks using explicit contract inventory, evidence classes, and risk grading | Experimental v0.4.0 release candidate; synthetic and authorized real-notebook acceptance plus controlled Python, Shell, LaTeX, HTML, trigger, restraint, and idempotence gates pass |

## Validate and install

The collection-level commands work from any checkout path:

```sh
./scripts/list_skills.sh
./scripts/check_skill.sh scientific-code-documenter
./scripts/check_skill.sh angze-code-style
./scripts/check_all_skills.sh
./scripts/run_fixture_checks.sh scientific-code-documenter
./scripts/run_fixture_checks.sh angze-code-style

./scripts/install_skill.sh --dry-run scientific-code-documenter
./scripts/install_skill.sh scientific-code-documenter
./scripts/install_skill.sh --dry-run angze-code-style
./scripts/install_skill.sh angze-code-style
./scripts/install_all_skills.sh --dry-run
./scripts/install_all_skills.sh
```
Expand All @@ -67,10 +70,10 @@ Installers derive the repository root from their own location and create individ

## Invoke a skill

Use `scientific-code-documenter` for review-only documentation assessments, focused docstring and comment work, or behavior-preserving readability improvements. Invoke it explicitly for a review-only request:
Use `angze-code-style` for review-only documentation assessments, focused docstring and comment work, or behavior-preserving readability improvements. Invoke it explicitly for a review-only request:

```text
$scientific-code-documenter
$angze-code-style
Review this file for readability and documentation quality. Do not modify it.
```

Expand All @@ -81,15 +84,19 @@ Codex may also select it implicitly for requests such as:
- “Clarify this Shell workflow's aggregate-error behavior without changing its exit codes.”
- “Organize these LaTeX macros while preserving labels, rendered content, and the template workaround.”
- “Make this scientific dashboard easier to maintain without changing its DOM or CSV contracts.”
- “Document this notebook's cell inputs, missing-value rules, ordering, and stored-output state without executing it.”
- “Prune syntax-narrating notebook comments while preserving outputs, metadata, execution counts, and cell identity.”

For an editing-mode request:

```text
$scientific-code-documenter
$angze-code-style
Apply only high-confidence documentation and readability improvements to this file. Preserve calculations, public interfaces, file formats, and numerical behaviour.
```

Units, shapes, conventions, physical roles, and numerical context should be documented only when supported by code, tests, configuration, repository documentation, or user-provided evidence. Uncertainty should be stated rather than converted into scientific fact. A review-only or “do not edit” instruction takes precedence over opportunities to fix stale comments or improve code.
Units, shapes, conventions, physical roles, and numerical context should be documented precisely only when supported by code, tests, configuration, repository documentation, or user-provided evidence. Directly observable operations may be described neutrally; unknown scientific meaning stays unresolved. A review-only or “do not edit” instruction takes precedence over opportunities to fix stale comments or improve code.

Jupyter notebooks are treated as structured artifacts. The skill classifies notebook role and risk before editing, preserves cell identity, metadata, execution counts, attachments, and stored outputs by default, and does not execute notebooks merely to refresh state. Generated, unknown-role, and protected analysis notebooks normally remain review-only.

Ordinary debugging, dependency upgrades, test execution, and unrelated feature work should not trigger the skill merely because code is present.

Expand All @@ -109,9 +116,11 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for the complete workflow.

Structural validation proves that a skill can be discovered and parsed; it does not prove usefulness. Evaluation therefore combines controlled A/B runs, trigger and scope-boundary measurements, independent behavior checks, adversarial fixtures, and authorized real-repository acceptance.

`scientific-code-documenter` v0.3.0 retains the accepted v0.2.0 runtime unchanged. Its real-repository ORCA Shell acceptance reviewed nine production-style scripts (4,319 lines), made zero review-only edits, and tested three controlled A/B pairs. The skill scored 89/90 versus 80/90 baseline, won all three scripts, and preserved generated ORCA inputs, command order, CLI and environment contracts, failures, restarts, cleanup, logs, outputs, and exit codes under independent mocks and dry runs. No expensive scientific calculation was launched.
`angze-code-style` v0.4.0 changes runtime behavior and completes a clean-break rename from `scientific-code-documenter`. It adds scientific/artifact contract inventory, `evidence-backed` / `observable-only` / `unknown` classification, Tier 0–3 risk grading, structured review findings, conservative notebook policy, comment pruning, and repeated-pass stability.

Controlled synthetic evaluation covered an editable notebook, review-only stale claim, over-commented notebook, generated notebook, and representative Python, Shell, LaTeX, and HTML fixtures. The accepted skill candidates averaged 17.5/18, won all six editing comparisons, passed every behavior and notebook hard gate, and were byte-stable on fresh second passes. The initial trigger matrix scored 10/10 positives with one false trigger; the full post-rename rerun scored 10/10 positives and 0/10 negatives. A separately authorized real tutorial-notebook snapshot passed zero-edit review, controlled A/B editing, strict preservation validation, behavior-preservation, and fresh-session idempotence. Scoring was manual rather than independently blind. ShellCheck and `pdflatex` were unavailable in the original local acceptance environment, so Shell syntax/dynamic contracts and static LaTeX contracts were used. ShellCheck subsequently ran and passed under the existing policy in remote CI on Ubuntu for Python 3.11 and 3.12; rendered `pdflatex` compilation remains unavailable. See the [sanitized v0.4.0 evaluation summary](docs/evaluations/angze-code-style-v0.4.0.md), [real-notebook acceptance record](docs/evaluations/angze-code-style-v0.4.0-real-notebook.md), and [integrity manifest](docs/evaluations/angze-code-style-v0.4.0.manifest.json).

The prior scientific Python real-repository acceptance and controlled Shell, LaTeX, HTML, CSS, and JavaScript evidence remain applicable because the runtime hash did not change. Mock and dry-run equivalence cannot prove every scheduler, cluster, MPI, or operating-system interaction. See the [sanitized v0.3.0 evaluation summary](docs/evaluations/scientific-code-documenter-v0.3.0.md) for exact results and limitations. The skill remains experimental and scientific edits still require human review.
Historical v0.1–v0.3 scientific Python and ORCA Shell evidence remains useful predecessor evidence, but it is not accepted as proof for the changed v0.4 runtime. The real-notebook acceptance established bounded Markdown-only preservation, not general notebook equivalence or scientific correctness. The skill remains experimental, and human scientific review remains mandatory.

## Privacy and publication

Expand Down
Loading
Loading