Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
7505383
chore(plan): complete planning for OKF-11
eltmon Nov 15, 2026
d5b45d5
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
df3f24d
test: add pytest harness and run it in CI after the selftest
eltmon Nov 15, 2026
02a24e3
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
c60ebc4
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
94c8202
feat(publish): add the escaped stdlib Markdown renderer
eltmon Nov 15, 2026
a8e498f
chore(publish): record remaining markdown-renderer acceptance criteria
eltmon Nov 15, 2026
f028731
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
c037b7c
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
48a292a
feat(publish): add redaction profile loader, built-in rules, redact a…
eltmon Nov 15, 2026
79fd40f
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
ff9245e
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
649906f
feat(publish): add the publish model and the fixture bundle
eltmon Nov 15, 2026
fbea564
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
25864bb
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
72f9d1d
feat(publish): render concept and index pages with page-relative links
eltmon Nov 15, 2026
55fd6a7
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
c961300
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
221d946
feat(publish): add the client-side search index, page and script
eltmon Nov 15, 2026
edc4422
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
ca6371a
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
533c86e
feat(publish): add publish.py --out with redaction and safe clean reb…
eltmon Nov 15, 2026
b6e7046
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
1750d03
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
46c6da9
feat(publish): add the publish.py --check redaction gate
eltmon Nov 15, 2026
cf6aa8d
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
074e341
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
a5fc9fc
docs(skill): add the /okf publish command row, SKILL section and self…
eltmon Nov 15, 2026
c9bd5f3
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
a40b7f3
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
053cac2
docs(publish): document publishing in README, references/publish.md a…
eltmon Nov 15, 2026
0ada5de
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
a02e284
fix(publish): URL-quote search result links like every other page link
eltmon Nov 15, 2026
a4de29d
fix(publish): match private strings across line breaks
eltmon Nov 15, 2026
baa6863
chore(plan): record OKF-11 spec lifecycle status
eltmon Nov 15, 2026
e2ec643
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
94d64e8
chore(workspace): plan artifacts for OKF-11
eltmon Nov 15, 2026
2aa7417
chore(plan): record OKF-11 verification feedback in the continue file
eltmon Nov 15, 2026
7480eee
fix(publish): fail closed on index listings and close review leaks
eltmon Nov 15, 2026
85a0643
chore(plan): record OKF-11 review feedback in the continue file
eltmon Nov 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,6 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- run: python -m pip install pyyaml
- run: python -m pip install pyyaml pytest
- run: bash scripts/selftest.sh
- run: python -m pytest -q tests
283 changes: 283 additions & 0 deletions .pan/continues/OKF-11.xbrief.json

Large diffs are not rendered by default.

642 changes: 642 additions & 0 deletions .pan/drafts/OKF-11.md

Large diffs are not rendered by default.

Large diffs are not rendered by default.

71 changes: 71 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ The skill remains portable. Its core scripts do not import Overdeck and require
| `/okf validate [--strict]` | Run the deterministic conformance gate. |
| `/okf lint` | Run advisory semantic patrol. |
| `/okf embed [--profile <name>]` | Update embedding shards and rebuild the local index. |
| `/okf publish --out <dir> [--check]` | Build a public static site from public concepts, with redaction. |

## The Three Loops

Expand All @@ -98,6 +99,76 @@ Run `/okf sync [--topic "<focus>"]` when code or docs change. The pass maps diff

Run `/okf retro` after implementation. The pass asks what knowledge would have made the work easier, then files those answers as concepts through the knowledge PR flow.

## Publishing a bundle

`/okf publish` builds a read-only static site from the bundle's public concepts. The site has one page per public concept, an index page per published directory, and a client-side search page. It is plain HTML, CSS, one script and a JSON index, with no server code and no CDN, so any static host works. Deploying the site is outside the skill.

```bash
python3 <okf-skill-dir>/scripts/publish.py --bundle . --check
python3 <okf-skill-dir>/scripts/publish.py --bundle . --out _site
```

**Visibility.** A concept is published only when its frontmatter says `x_visibility: public`, or when it has no `x_visibility` and the root `schema.md` sets `x_visibility_default: public`. Without `schema.md` or that key, every concept is private. Any value other than `public` or `private` stops the run with an error.

```yaml
# schema.md frontmatter
x_visibility_default: public
# a concept that must stay off the site
x_visibility: private
```

A private concept has no page, no index line, no search record and no backlink. A link to it renders as plain text, and the link text stays, so do not put secrets in link text. `log.md`, non-Markdown files and dot-directories are never published.

**Redaction profile.** Before rendering, `publish.py` rewrites every published string with regex rules. Built-in rules always apply unless you disable them: `home-path-unix`, `home-path-windows`, `ipv4`, `github-token`, `openai-style-key`, `aws-access-key`, `slack-token` and `private-key-block`. Add your own rules in `okf-publish.yaml` at the bundle root (start from `templates/okf-publish.yaml`):

```yaml
okf_publish_version: "0.1"
disable_default_rules: [] # for example [ipv4] when version numbers trip it
rules:
- id: stream-box-host
pattern: '\bstream-box\.lan\b'
replace: '[internal-host]'
private_strings: # literal, case-insensitive, matches across line breaks
- Alice Example
```

Use `publish.py --bundle . --check` as the CI gate. It exits 2 when any rule matches text in a public concept or a published index page. It names the rule and the file line, never the matched text, because CI logs of public repos are public. Fix the source text, then rerun. A rule that matches a public concept ID or directory name is always an error, because URLs cannot be redacted.

**Sample Cloudflare Pages setup.** This workflow checks the bundle and deploys `_site` on every push to `main`:

```yaml
name: Publish knowledge site
on:
push:
branches: [main]
jobs:
publish:
runs-on: ubuntu-latest
permissions:
contents: read
deployments: write
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v4
with:
repository: eltmon/okf
ref: vX.Y.Z # a release that includes scripts/publish.py
path: .okf-skill # dot-directory: the publisher skips it
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: python3 -m pip install PyYAML
- run: python3 .okf-skill/scripts/publish.py --bundle . --check
- run: python3 .okf-skill/scripts/publish.py --bundle . --out _site
- uses: cloudflare/wrangler-action@v3
with:
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
command: pages deploy _site --project-name=<your-pages-project>
```

For GitHub Pages, upload `_site` with `actions/upload-pages-artifact` and deploy it with `actions/deploy-pages`; every link is page-relative, so the site works under a project subpath. Search fetches `search-index.json`, so preview the site with `python3 -m http.server --directory _site` rather than opening the files directly. See [references/publish.md](references/publish.md) for the full rules.

## Reading Rules

When using a bundle for answers or prompt context:
Expand Down
12 changes: 12 additions & 0 deletions SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ The bundle is the source of truth. Keep changes small, cited, and PR-gated. Dete
| `/okf validate [--strict]` | Run the deterministic conformance gate. | `references/conformance.md` |
| `/okf lint` | Run advisory semantic patrol for stale or weak knowledge. | `references/lint-prompt.md`, `references/conformance.md` |
| `/okf embed [--profile <name>]` | Update shareable embedding shards and rebuild the local index. | `references/workflow.md` |
| `/okf publish --out <dir> [--check]` | Build a public static site from public concepts, with redaction. | `references/publish.md` |

## Guardrails

Expand Down Expand Up @@ -184,3 +185,14 @@ Refresh embeddings.
- Re-embed only concepts whose normalized content hash changed.
- Write sorted JSONL shards under `embeddings/`.
- Rebuild the local `.okf-index/` cache, which is derived and gitignored.

## `/okf publish --out <dir> [--check]`

Build a public static site from the bundle.

- Run `python3 <okf-skill-dir>/scripts/publish.py --bundle <bundle> --check` first; it exits 2 when a redaction rule matches text in a public concept and names the rule and line, never the matched text.
- Then run `python3 <okf-skill-dir>/scripts/publish.py --bundle <bundle> --out <dir>`.
- Only concepts with `x_visibility: public`, or covered by `x_visibility_default: public` in root `schema.md`, are published. The default is private.
- Links to private or missing concepts render as plain text. `log.md` and non-Markdown files are never published.
- Redaction rules come from built-in defaults plus `okf-publish.yaml`; see `references/publish.md`.
- Never mark a concept public or edit a redaction profile without the user's request. Deploying the site is outside this skill.
14 changes: 14 additions & 0 deletions docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,20 @@ Refresh embeddings and the local search index.
- Writes sorted shard lines to `embeddings/<profile>.okfe.jsonl`.
- Rebuilds `.okf-index/`, which is derived and gitignored.

## `/okf publish --out <dir> [--check]`

Build a public static site from the bundle's public concepts.

```bash
python3 <okf-skill-dir>/scripts/publish.py --bundle . --check
python3 <okf-skill-dir>/scripts/publish.py --bundle . --out _site
```

- Run `--check` first. It exits 2 when a redaction rule matches text in a public concept, and names the rule and line but never the matched text.
- Only concepts with `x_visibility: public`, or covered by `x_visibility_default: public` in root `schema.md`, are published. The default is private.
- Links to private or missing concepts render as plain text. `log.md` and non-Markdown files are never published.
- Redaction rules come from built-in defaults plus `okf-publish.yaml`; see `references/publish.md`. Deploying the site is outside the skill.

## Model Ladder

For `author`, `sync`, `study`, and `retro`, `--model <model>` is a hard request:
Expand Down
154 changes: 154 additions & 0 deletions references/publish.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# OKF Publish

`scripts/publish.py` builds a read-only static site from the public concepts of a bundle. It uses only the Python standard library and PyYAML. Deploying the site is outside the skill.

## Synopsis

```text
python3 scripts/publish.py [--bundle B] (--out DIR | --check) [--profile FILE]
```

- `--bundle B`: the bundle root. The default is the current directory.
- `--out DIR`: write the site to `DIR`.
- `--check`: scan published text for redaction matches. Write nothing.
- `--profile FILE`: use this redaction profile instead of `<bundle>/okf-publish.yaml`.

`--out` and `--check` are required and mutually exclusive. Run `--check` in CI before `--out`.

## Visibility

Each concept resolves to `public` or `private`. Only public concepts are published.

1. The bundle default comes from `x_visibility_default` in the frontmatter of root `schema.md`.
- No `schema.md`: the default is `private`.
- `schema.md` without `x_visibility_default`: the default is `private`.
- `public` or `private`: that value.
- Any other value: finding `P_VISIBILITY_INVALID`, and the default is `private`.
2. A concept's own `x_visibility` overrides the default.
- Absent: the bundle default applies.
- `public` or `private`: that value.
- Any other value, including `Public`, `true` and `yes`: finding `P_VISIBILITY_INVALID`, and the concept is private.
3. `schema.md` is an ordinary concept. Its own `x_visibility`, or the default, decides whether it gets a page.

No input resolves to public by accident. When no concept is public, `publish.py` prints why the default is private.

A private concept appears nowhere on the site. It has no page, its index listing line is dropped, it has no search record and it appears in no "Linked from" list.

Index listings fail closed. Inside the generated `<!-- OKF:INDEX:BEGIN -->` … `<!-- OKF:INDEX:END -->` block of an `index.md`, a line is kept only when it is a listing line (`* ` or `- `) whose first link resolves to a public concept or a published directory. Every other line in the block is dropped: private entries, continuation lines of a multi-line description, and lines whose link cannot be parsed, such as a file name with a space. Outside the block, a hand-written listing line is dropped when its first link points at a private, missing or unpublished target; all other hand-written lines are kept, with links to unpublished targets rendered as text.

## Links

A link in published Markdown resolves this way:

| Target | Result |
| --- | --- |
| A public concept, with or without `#fragment` | A page-relative link to its page, keeping the fragment. |
| A published directory, or its `index.md` | A page-relative link to that directory's `index.html`. |
| `http:`, `https:` or `mailto:` | Kept as written. |
| `#fragment` only | Kept as written. |
| Anything else | Plain text: the link text stays and the link goes. |

"Anything else" covers private concepts, missing concepts, unpublished directories, `log.md`, non-Markdown files and every other scheme (`javascript:`, `data:`, `repo:` and so on). A private target and a missing target render identically, so the site does not reveal which private concepts exist. The link text itself is still published, so do not put secrets in link text.

A published directory is a directory that holds a public concept itself or in a subdirectory. The root is always published. Every href is page-relative (for example `../schema.html`), so the site works under a subpath such as a GitHub Pages project site.

## Concept pages

Each concept page shows the concept ID, title, type, status, trust tier, tags, description, rendered body, a "Sources" list and a "Linked from" list. "Linked from" lists public concepts only. Status is frontmatter `status`, or `stable` when absent.

The trust tier comes from `verified` frontmatter:

- `human-reviewed`: any `verified` entry has a `by` value that starts with `human:`.
- `machine-confirmed`: there is at least one `verified` entry, but none by `human:`.
- `unverified`: there are no `verified` entries.

A single mapping counts as a one-element list.

## Redaction profile

`okf-publish.yaml` at the bundle root adds rules. Start from `templates/okf-publish.yaml`.

```yaml
okf_publish_version: "0.1" # required, must be the string "0.1"
disable_default_rules: [] # optional; ids from the built-in rules
rules: # optional; applied after built-in rules
- id: stream-box-host # required, unique, matches ^[a-z0-9][a-z0-9-]*$
pattern: '\bstream-box\.lan\b' # required, Python re syntax; inline flags such as (?i) allowed
replace: '[internal-host]' # optional, default "[redacted]"
private_strings: # optional; literal strings, matched case-insensitively
- Alice Example
```

Rules apply in this order: enabled built-in rules, then `rules`, then `private_strings`. Private string *n* becomes rule `private-string-<n>` with replacement `[redacted]`. It matches case-insensitively, and any run of whitespace, including a line break, matches the spaces between its words, so a name wrapped across two lines is still caught. A replacement is literal text, so `\1` inserts a backslash and a one. Rules apply to every published string before rendering, including fenced code and link targets. A rule that rewrites a link target can turn that link into plain text.

An invalid profile stops the run with `P_PROFILE_INVALID`. That covers unreadable YAML, a top level that is not a mapping, an unknown key, a wrong `okf_publish_version`, an unknown id in `disable_default_rules`, a rule without `id` or `pattern`, an invalid or duplicate id, a pattern that does not compile, an empty private string and a non-string value.

### Built-in rules

Built-in rules apply even with no profile file.

| Id | Pattern | Replacement |
| --- | --- | --- |
| `home-path-unix` | `` /(?:home\|Users)/[^/\s'"`)\]>]+ `` | `~` |
| `home-path-windows` | `` [A-Za-z]:\\Users\\[^\\\s'"`)\]>]+ `` | `~` |
| `ipv4` | four dot-separated numbers from 0 to 255 | `[ip]` |
| `github-token` | `\bgh[pousr]_[A-Za-z0-9]{36,}\b` | `[token]` |
| `openai-style-key` | `\bsk-[A-Za-z0-9_-]{20,}` | `[token]` |
| `aws-access-key` | `\bAKIA[0-9A-Z]{16}\b` | `[token]` |
| `slack-token` | `\bxox[abprs]-[A-Za-z0-9-]{10,}` | `[token]` |
| `private-key-block` | `-----BEGIN [A-Z ]*PRIVATE KEY-----` | `[private-key]` |

`scripts/publish_redact.py` holds the exact patterns in `DEFAULT_RULES`.

`ipv4` also matches `127.0.0.1` and four-part version numbers such as `1.2.3.4`. When that happens, rewrite the text or add `ipv4` to `disable_default_rules`. There is no per-match allow list.

## Output layout

`--out DIR` writes exactly these files, and nothing else:

- `<concept-id>.html` for each public concept, for example `rules/no-spoilers.html`.
- `index.html` for each published directory, the root included. A directory's `index.md` supplies the page, with private listing lines dropped. A directory without `index.md` gets a generated listing.
- `search.html`, `search-index.json`, `assets/style.css` and `assets/search.js`.
- `.okf-publish`, the output marker.

The site title is the first `# ` heading of the root `index.md`, or `Knowledge`. Two runs on the same bundle produce byte-identical files. Search runs in the browser and fetches `search-index.json`, so serve the site over HTTP (`python3 -m http.server --directory DIR`) rather than opening files directly.

## `--out` safety and rebuilds

`publish.py` refuses `--out` with `P_OUT_UNSAFE` when it:

- resolves to the bundle root or an ancestor of it;
- exists and is not a directory; or
- is a non-empty directory without the `.okf-publish` marker.

When the marker is present, every run deletes all entries in `--out` and rebuilds. So a page for a concept that has turned private never survives. An `--out` inside the bundle is allowed and is skipped when walking the bundle.

## Findings and exit codes

Each finding is one line: `ERROR <CODE> <location>: <message>`. The location is a file, a file and line (`rules/a.md:14`), or a directory. Findings never echo matched text, because CI logs of public repos are public.

| Code | Mode | Meaning |
| --- | --- | --- |
| `P_BUNDLE_MISSING` | both | The `--bundle` directory does not exist. |
| `P_PROFILE_INVALID` | both | The redaction profile is invalid. |
| `P_FRONTMATTER_PARSE` | both | A concept's frontmatter does not parse. |
| `P_VISIBILITY_INVALID` | both | `x_visibility` or `x_visibility_default` is not `public` or `private`. |
| `P_ID_REDACTION` | both | A rule matches a public concept ID or a published directory path. URLs cannot be redacted, so rename it or make it private. |
| `P_REDACTION` | `--check` | A rule matches published text. The message names the rule id and the field; body and index findings carry the file line. |
| `P_OUT_UNSAFE` | `--out` | The `--out` directory is unsafe to write (see above). |

| Exit | Meaning |
| --- | --- |
| 0 | `--out`: the site was written. `--check`: no rule matches published text. |
| 2 | Any finding above, or a usage error. With `--out`, nothing is written and the existing `--out` contents stay untouched. |

`--check` scans the source text of public concepts and published index pages. It also scans each body and index line as a reader sees it, with link syntax, backslash escapes and emphasis markers removed, so `**Alice** Example` or `/home\/alice` still fails the check. It passes only after the author removes the matching text. It never passes just because `--out` would redact the text.

## What is never published

- `log.md` files.
- Private concepts, and their titles in index listings, search and backlinks.
- Non-Markdown files such as images and PDFs. An image renders as its alt text.
- Anything under a path component that starts with `.` (for example `.git`, `.github`, `.okf-index`), or under `node_modules`.
- Frontmatter keys other than title, description, type, status, tags and sources. The `verified` entries show only as the derived trust tier.
- Raw HTML in Markdown. It renders as escaped text.
Loading
Loading