Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,13 +89,11 @@ jobs:
shell: bash
run: |
set -euo pipefail
FERN_VERSION="$(node -p "require('./fern/fern.config.json').version")"
if [[ ! "$FERN_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "::error file=fern/fern.config.json::Invalid Fern CLI version: $FERN_VERSION"
exit 1
fi
npm install --global "fern-api@$FERN_VERSION"
fern check
# npm ci reproduces fern/package-lock.json exactly. tests/test_ci_workflows.py
# asserts fern/package.json and fern/fern.config.json name the same version.
npm ci --prefix fern --ignore-scripts --omit=optional
FERN_VERSION="$(node -p "require('./fern/package.json').dependencies['fern-api']")"
./fern/node_modules/.bin/fern check
{
echo "### Documentation-only CI"
echo
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/dco.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ jobs:
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.sha }}
persist-credentials: false

- name: Verify DCO sign-off on every commit
env:
Expand Down
21 changes: 17 additions & 4 deletions .github/workflows/publish-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,31 @@ on:
branches:
- main

permissions:
contents: read

jobs:
run:
runs-on: ubuntu-latest
if: ${{ github.event_name == 'push' && contains(github.ref, 'refs/heads/main') && github.run_number > 1 }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false

- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"

- name: Install Fern
run: npm install -g fern-api
# npm ci reproduces fern/package-lock.json exactly -- every version and
# integrity hash -- so nothing resolved at run time reaches the step below,
# which holds FERN_TOKEN.
- name: Install the pinned Fern CLI
run: npm ci --prefix fern --ignore-scripts --omit=optional

- name: Publish Docs
env:
FERN_TOKEN: ${{ secrets.FERN_TOKEN }}
run: fern generate --docs
run: ./fern/node_modules/.bin/fern generate --docs
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,6 @@ src/skillevaluator/tier3/reference_skills/*/evals/results/
skillevaluator-output*
skillevaluator-quality*
skillevaluator-rubric-eval*

# Node modules for the pinned Fern documentation CLI (see fern/package.json)
node_modules/
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,47 @@ All notable changes to SkillEvaluator are documented in this file.
directions at large case counts, and documents exact-rational omission
markers.

### Security

- The docs-publishing workflow, the only one that puts a secret in a job
environment, now pins `actions/checkout` to a commit SHA, declares
`permissions: contents: read`, and sets `persist-credentials: false`.
`dco.yml` no longer persists checkout credentials either.
- Docs are now published with the same Fern CLI version they are validated
with. `publish-docs.yml` derives it from `fern/fern.config.json`, as
`ci.yml` already did, instead of installing whatever `fern-api` resolved to
at run time.
- Workflow hardening guards now glob `.github/workflows/` instead of a
hardcoded two-file list, so every workflow — including any added later — must
pin each action to a commit, avoid persisting checkout credentials, and
declare a `permissions:` block. The guards also cover job-level
`permissions:` overrides and job-level reusable-workflow `uses:` references,
neither of which the step-level checks reached.
- The Fern CLI is now installed from a committed lockfile, `fern/package-lock.json`,
with `npm ci --prefix fern --ignore-scripts --omit=optional`, and both workflows
invoke `./fern/node_modules/.bin/fern` rather than a binary on `PATH`. Previously
`publish-docs.yml` ran `npm install`, which re-resolves every transitive
dependency from semver ranges on each run: pinning `fern-api`'s own version
pinned nothing beneath it, and `@scarf/scarf` — reached through `fern-api`'s
optional dependency on `@boundaryml/baml` — declares a `postinstall` script that
executed in the job holding `FERN_TOKEN`. The lockfile pins every package to an
exact version and integrity hash, `--ignore-scripts` stops lifecycle code
running, and `--omit=optional` (which npm honours for this local install, but
silently ignores for a `--global` one) leaves `fern-api` alone in the tree.
Guards now assert that no workflow resolves a Node dependency tree from the
registry, that every npm command carries `--ignore-scripts`, and that
`fern/package.json`, `fern/package-lock.json` and `fern/fern.config.json` all
name the same CLI version. The developer-facing install instructions
(`docs/README.md`, `docs/AGENTS.md`, `docs/developer-guide.mdx`) use the same
lockfile install, so a contributor or agent following them gets the tree CI runs.
- The action-pinning guard now accepts a same-repo composite action or
reusable workflow (`uses: ./...`) without requiring a commit SHA, since
GitHub always resolves a local reference from the caller's own commit and
nothing about it can float. A local reference that escapes the repository
(e.g. `./../outside`) or carries an `@ref` (a syntax local references don't
have) is still rejected, and a `docker://` reference is unaffected by the
exemption.

## 0.2.1 - 2026-08-24

### Added
Expand Down
7 changes: 4 additions & 3 deletions docs/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,12 @@ Navigation order, page titles, and slugs are defined in

## Verify before committing

Requires **Node.js 22+** and the Fern CLI (`npm install -g fern-api`).
Requires **Node.js 22+**. Install the pinned CLI from the committed lockfile with
`npm ci --prefix fern --ignore-scripts --omit=optional`.

```bash
fern check # validate docs.yml config and all links — must pass
fern docs dev # optional live preview at http://localhost:3000
./fern/node_modules/.bin/fern check # validate docs.yml config and all links — must pass
./fern/node_modules/.bin/fern docs dev # optional live preview at http://localhost:3000
```

`fern check` must pass; it is the same gate the site build relies on. Publishing
Expand Down
8 changes: 4 additions & 4 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,14 @@ published automatically from `main` via the Fern GitHub integration.
Prerequisites: Node.js 22+ and npm 10+ (the versions the Fern CLI requires).

```bash
# Install the Fern CLI
npm install -g fern-api
# Install the pinned Fern CLI from the committed lockfile
npm ci --prefix fern --ignore-scripts --omit=optional

# From the repo root, preview the site with live reload
fern docs dev
./fern/node_modules/.bin/fern docs dev

# Validate the docs configuration and links
fern check
./fern/node_modules/.bin/fern check
```

`fern docs dev` serves the site at <http://localhost:3000> and reloads on changes
Expand Down
12 changes: 7 additions & 5 deletions docs/developer-guide.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -83,24 +83,26 @@ The documentation site is built with [Fern](https://buildwithfern.com/) from the
<Steps>
### Install the Fern CLI

Requires Node.js 22+ and npm 10+.
Requires Node.js 22+ and npm 10+. `fern/package-lock.json` pins the CLI and every package
below it; `npm ci` reproduces that tree exactly, and the flags keep npm from running any
install-time scripts. This is the same install CI performs.

```bash title="Install fern-api"
npm install -g fern-api
```bash title="Install the pinned fern-api"
npm ci --prefix fern --ignore-scripts --omit=optional
```

### Preview with live reload

```bash title="Local docs preview"
fern docs dev
./fern/node_modules/.bin/fern docs dev
```

The site serves at `http://localhost:3000` and reloads on changes to `.mdx` files or `fern/docs.yml`.

### Validate before pushing

```bash title="Check config and links"
fern check
./fern/node_modules/.bin/fern check
```

`fern check` must pass — it is the same gate the site build relies on.
Expand Down
202 changes: 202 additions & 0 deletions fern/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions fern/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"name": "skillevaluator-docs",
"version": "0.0.0",
"private": true,
"description": "Pins the Fern CLI that validates and publishes the documentation site. Keep the fern-api version in step with fern.config.json; tests/test_ci_workflows.py asserts they agree.",
"license": "Apache-2.0",
"dependencies": {
"fern-api": "5.66.1"
}
}
Loading