From e39f1faed490904cd074aa461371558589a140fe Mon Sep 17 00:00:00 2001 From: Rob Konsdorf Date: Tue, 18 Aug 2026 00:45:59 -0400 Subject: [PATCH 1/4] docs: write each product name the way the product spells it Five sentences named a product with a bare lowercase word: the WharfKit client twice, an AtomicTools action, the AtomicMarket API, and the AtomicMarket indexer handler. This corpus already writes the contract account, the repository, and the package with a separator that keeps each of them lowercase, so a bare lowercase token in prose reads as one of those and sends a reader after an identifier that does not exist. --- guides/links.md | 2 +- guides/querying-the-api.md | 2 +- reference/atomicassets-api.md | 2 +- reference/sdk/atomicassets.md | 2 +- reference/sdk/atomicmarket.md | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/guides/links.md b/guides/links.md index ec4a9d3..dc8e6ee 100644 --- a/guides/links.md +++ b/guides/links.md @@ -64,7 +64,7 @@ Source: `src/link.cpp:21-98` ## Step 3: fund the link -Funding is a normal AtomicAssets `transfer` to the `atomictoolsx` account with the memo `"link"`; there is no atomictools action for it. The asset ids must be the same set announced in step 2 (order does not matter). +Funding is a normal AtomicAssets `transfer` to the `atomictoolsx` account with the memo `"link"`; there is no AtomicTools action for it. The asset ids must be the same set announced in step 2 (order does not matter). ```json { diff --git a/guides/querying-the-api.md b/guides/querying-the-api.md index a32597b..0854e80 100644 --- a/guides/querying-the-api.md +++ b/guides/querying-the-api.md @@ -43,7 +43,7 @@ Do not rely on socket notifications for lifecycle tracking: only new offers are When reading contract tables directly over `/v1/chain/get_table_rows`, three behaviors matter: - **Numeric keys need `key_type: 'i64'`.** @wharfkit/antelope's typed `client.v1.chain.get_table_rows` infers `key_type` only from typed bound instances; plain string or number bounds fall back to `key_type: 'name'` and are silently misread as account names, returning wrong ranges. Pass `key_type: 'i64'` explicitly (or typed bounds) for tables keyed by numeric ids. See [@wharfkit/antelope client behavior](../reference/wharfkit.md#typed-get_table_rows-is-not-a-drop-in-for-dynamic-reads) ("Typed get_table_rows is not a drop-in for dynamic reads"). -- **`show_payer` shapes differ by transport.** The raw endpoint wraps each row as `{ data, payer }`; the typed wharfkit client unwraps rows and moves payers into an index-aligned `ram_payers` array on the response. Read `response.ram_payers[i]` with the typed client, or call the raw endpoint with `json: true, show_payer: true` to keep the envelope. See [@wharfkit/antelope client behavior](../reference/wharfkit.md#show_payer-rows-are-unwrapped-into-ram_payers) ("show_payer rows are unwrapped into ram_payers"). +- **`show_payer` shapes differ by transport.** The raw endpoint wraps each row as `{ data, payer }`; the typed WharfKit client unwraps rows and moves payers into an index-aligned `ram_payers` array on the response. Read `response.ram_payers[i]` with the typed client, or call the raw endpoint with `json: true, show_payer: true` to keep the envelope. See [@wharfkit/antelope client behavior](../reference/wharfkit.md#show_payer-rows-are-unwrapped-into-ram_payers) ("show_payer rows are unwrapped into ram_payers"). - **Large uint64 values arrive as strings.** nodeos serializes uint64 values above 2^32 as JSON strings and smaller values as JSON numbers; current `sale_id`/`auction_id`/`offer_id` values arrive as numbers, but asset ids (around 2^40) arrive as strings. Parse id fields defensively rather than assuming one shape. See [AtomicMarket V2 changes](../reference/atomicmarket/v2-changes.md#large-integers-serialize-as-strings) ("Large integers serialize as strings"). - **The `assets` table is scoped by owner, with no collection or template index.** Enumerating every asset in a collection or template is an API-only capability: `/atomicassets/v1/assets?collection_name=...` joins across owners, but `get_table_rows` on `assets` takes the owner account as `scope`, so there is no chain-side path from a collection to its asset list without already knowing the owners. To read one specific asset over the chain, use its current owner as `scope` and its `asset_id` as an `i64`-typed bound. diff --git a/reference/atomicassets-api.md b/reference/atomicassets-api.md index 6fd7eb9..4c8bcd9 100644 --- a/reference/atomicassets-api.md +++ b/reference/atomicassets-api.md @@ -15,7 +15,7 @@ Source: `atomicassets-api package.json` (`name`, `repository.url`), `atomicasset ## The delphioracle handler is required everywhere -The AtomicMarket handler's schema setup declares a foreign key from `atomicmarket_symbol_pairs` to `delphioracle_pairs`. The `delphioracle_pairs` table is created only by the delphioracle handler's own bootstrap, so a filler configuration that omits the delphioracle handler fails during AtomicMarket table creation with a missing-relation error. This means the delphioracle handler must be registered in every chain's filler configuration, ordered before the atomicmarket handler, even on chains where no delphioracle contract is deployed; there it simply observes a non-existent account and stays idle. The failure surfaces as a schema/migration error on a fresh chain rather than anything pointing at handler configuration, which makes it a common trap when onboarding a new chain. +The AtomicMarket handler's schema setup declares a foreign key from `atomicmarket_symbol_pairs` to `delphioracle_pairs`. The `delphioracle_pairs` table is created only by the delphioracle handler's own bootstrap, so a filler configuration that omits the delphioracle handler fails during AtomicMarket table creation with a missing-relation error. This means the delphioracle handler must be registered in every chain's filler configuration, ordered before the AtomicMarket handler, even on chains where no delphioracle contract is deployed; there it simply observes a non-existent account and stays idle. The failure surfaces as a schema/migration error on a fresh chain rather than anything pointing at handler configuration, which makes it a common trap when onboarding a new chain. ## Reader-priority drain gate and queue dedup diff --git a/reference/sdk/atomicassets.md b/reference/sdk/atomicassets.md index 3232154..2c63d3d 100644 --- a/reference/sdk/atomicassets.md +++ b/reference/sdk/atomicassets.md @@ -123,6 +123,6 @@ Three read paths cover different needs, consistent with `guides/querying-the-api - **`ExplorerApi` (this SDK)** for typed indexer reads from JS/TS: filtered lists, search, sort, counts, stats, and cross-owner enumeration, with params and response objects already typed and errors raised as `ApiError`. This is the default for application and bot code. - **Raw HTTP against the atomicassets-api** when you are not in a JS runtime, or want full control over paging and caching without the wrapper. The endpoints, the limit cap, and the lifecycle-state enums are in [atomicassets-api HTTP API](../api.md). -- **Chain table reads** (`RpcApi` here, or `@wharfkit/antelope`'s `get_table_rows`) when you need unindexed chain truth without indexer lag, or a node is your only backend. Note the numeric-key and `show_payer` pitfalls of the typed wharfkit client in [@wharfkit/antelope client behavior](../wharfkit.md); `RpcApi` reads through its own queue and cache instead. +- **Chain table reads** (`RpcApi` here, or `@wharfkit/antelope`'s `get_table_rows`) when you need unindexed chain truth without indexer lag, or a node is your only backend. Note the numeric-key and `show_payer` pitfalls of the typed WharfKit client in [@wharfkit/antelope client behavior](../wharfkit.md); `RpcApi` reads through its own queue and cache instead. List endpoints reached through `ExplorerApi` inherit the deployment's `limit` cap of 100; see [atomicassets-api HTTP API](../api.md#list-endpoints-cap-limit-at-100) ("List endpoints cap limit at 100"). diff --git a/reference/sdk/atomicmarket.md b/reference/sdk/atomicmarket.md index 56d8f56..2b33fa2 100644 --- a/reference/sdk/atomicmarket.md +++ b/reference/sdk/atomicmarket.md @@ -111,7 +111,7 @@ Source: atomicmarket-sdk (main, 278bdfa) src/Networks.ts (`marketApiForNetwork`, The read-path choice mirrors the AtomicAssets SDK, consistent with `guides/querying-the-api.md`: - **`AtomicMarketApi` (this SDK)** for typed indexer reads of sales, auctions, buyoffers, marketplaces, and the royalty read layer from JS/TS, with params and response objects typed and failures raised as `ApiError`. -- **Raw HTTP against the atomicmarket API** outside a JS runtime, or when you want direct control over paging; the endpoints, limit cap, and per-endpoint `state` enums are in [atomicassets-api HTTP API](../api.md). +- **Raw HTTP against the AtomicMarket API** outside a JS runtime, or when you want direct control over paging; the endpoints, limit cap, and per-endpoint `state` enums are in [atomicassets-api HTTP API](../api.md). - **Chain table reads** (`@wharfkit/antelope` `get_table_rows`) for unindexed marketplace state without indexer lag; mind the numeric-key and `show_payer` behaviors in [@wharfkit/antelope client behavior](../wharfkit.md), and that large ids can arrive as strings. List endpoints reached through `AtomicMarketApi` inherit the deployment's `limit` cap of 100; see [atomicassets-api HTTP API](../api.md#list-endpoints-cap-limit-at-100) ("List endpoints cap limit at 100"). From 03acbbd4f49751e5a5032de9fa39b1cf243d9c37 Mon Sep 17 00:00:00 2001 From: Rob Konsdorf Date: Tue, 18 Aug 2026 00:46:04 -0400 Subject: [PATCH 2/4] docs: list the client library among the baselines a page is read against The ledger's row for the WharfKit page names the library and the version it was read at, while the pinned-baselines list above it carried no entry for the library at all. The list is what a reader takes as the full set of sources behind this corpus, and a Release body copies it verbatim, so a page resting on a source the list omits reads as a page resting on nothing. --- validation-log.md | 1 + 1 file changed, 1 insertion(+) diff --git a/validation-log.md b/validation-log.md index c41d2eb..35c7213 100644 --- a/validation-log.md +++ b/validation-log.md @@ -15,6 +15,7 @@ This log traces how every fact in `reference/` and `guides/` was checked before - `atomicassets-api` at its current `main` branch state (no release tag; indexer behavior and API surface are read from the running source tree; streaming and rate-limit pages pin `f6419858`) - `atomictools-contract` at commit `d89ce79e4` (the upstream repository has no release tag; the deployed `atomictoolsx` ABI on WAX matches this commit exactly) - `atomicassets-sdk` at main `80580c5` and `atomicmarket-sdk` at main `278bdfa` (both version 2.0.0) +- `@wharfkit/antelope` at `1.1.1` - `@atomichub/vert` at `2.2.0`, commit `a8a4160` A page's `key-modules` frontmatter names the specific baseline(s) it draws from; entries below carry the same pin unless noted otherwise. From 09204637c9aae0e980bfa96c88b3891e446c1f63 Mon Sep 17 00:00:00 2001 From: Rob Konsdorf Date: Tue, 18 Aug 2026 00:46:12 -0400 Subject: [PATCH 3/4] ci: gate a merge on the checks a reviewer cannot run by eye Nothing checked a page before it merged, and the defects that reach a reader are exactly the ones a reviewer does not see: a fragment with no matching heading, a description outside the band the site renders it into, a ledger row for a page that no longer exists. Each of those surfaces in the docs-site repository at pin-bump time instead, where the person who caused it is not looking. Each job id is its check name, so branch protection requires ten names and a red pull request has already said which rule failed. The four prose rules read a copy of each page with its code blanked to spaces of the same shape, because this corpus writes account names and ABI type names in code, and a rule that cannot tell code from prose either fails on a sample or gets switched off. External URL checking and starter signing are the two arms that need the network, and both run on the weekly schedule so a third-party outage never blocks a merge. --- .github/banned-terms.txt | 31 ++ .github/scripts/check-frontmatter.mjs | 95 +++++ .../scripts/check-validation-consistency.mjs | 115 ++++++ .github/scripts/install-ripgrep.sh | 25 ++ .github/scripts/lib/pages.mjs | 147 ++++++++ .github/scripts/strip-code.mjs | 129 +++++++ .github/workflows/checks.yml | 354 ++++++++++++++++++ .markdownlint-cli2.jsonc | 32 ++ _typos.toml | 36 ++ 9 files changed, 964 insertions(+) create mode 100644 .github/banned-terms.txt create mode 100644 .github/scripts/check-frontmatter.mjs create mode 100644 .github/scripts/check-validation-consistency.mjs create mode 100755 .github/scripts/install-ripgrep.sh create mode 100644 .github/scripts/lib/pages.mjs create mode 100644 .github/scripts/strip-code.mjs create mode 100644 .github/workflows/checks.yml create mode 100644 .markdownlint-cli2.jsonc create mode 100644 _typos.toml diff --git a/.github/banned-terms.txt b/.github/banned-terms.txt new file mode 100644 index 0000000..ad758b1 --- /dev/null +++ b/.github/banned-terms.txt @@ -0,0 +1,31 @@ +# Terms this corpus does not use, one ripgrep pattern per line. Run with +# --pcre2 over the stripped copy of every page, so a real action name, a real +# payload, and a real account name inside code are already gone before a line +# reaches this file. A finding here is prose. + +# The category head term. Mirrors the site's own vocabulary source, which the +# built-output gate compiles from the same string, so the two cannot drift: +# packages/atomic-brand/src/vocabulary.ts. The protocol's own nouns +# (collection, schema, template, asset, offer, sale) carry the meaning. +\bnfts?\b|non[-_\s]?fungible + +# Era branding rather than a protocol fact. Name the chain, the contract, or +# the API instead. +(?i)\bweb ?3(\.0)?\b + +# An asset is an asset. A template is a template. "Token instance" is neither +# contract vocabulary nor API vocabulary. +(?i)\btoken instances?\b + +# Wallet where the referent is an account. On Antelope the holder of an asset +# or a token balance is an account, and a wallet is the signing software in +# front of it, so the word points a reader at the wrong layer. +# +# Three senses are sanctioned and none of them takes these constructions: +# Cloud Wallet is a product name, "wallet" naming signing software is the word +# used correctly, and a wallet address is an address. Sample account names +# built on the word sit in code and the stripping step has already removed +# them. +(?i)\b(?:\w+'s|their|its|your|our|his|her|my)\s+wallets?\b(?!\s+addresses?\b) +(?i)\b(?:in|into|inside|from|to|between|out\s+of)\s+(?:the\s+|an?\s+)?wallets?\b(?!\s+addresses?\b) +(?i)\bwallets?\s+(?:holds?|contains?|owns?|receives?|stores?|keeps?)\b diff --git a/.github/scripts/check-frontmatter.mjs b/.github/scripts/check-frontmatter.mjs new file mode 100644 index 0000000..164b1ce --- /dev/null +++ b/.github/scripts/check-frontmatter.mjs @@ -0,0 +1,95 @@ +#!/usr/bin/env node +/** + * The frontmatter arms that a JSON Schema cannot express, plus the extraction + * ajv validates. + * + * Each arm mirrors a failure that would otherwise surface in the docs-site + * repository at pin-bump time, where the person who caused it is not looking: + * the site takes a page title from the leading H1 and a meta description from + * `scope`, and it throws on a page whose body does not open with an H1. + * + * Usage: node .github/scripts/check-frontmatter.mjs [root] + */ +import { access, mkdir, writeFile } from 'node:fs/promises'; +import { join, resolve } from 'node:path'; +import { leadingHeading, pagesUnder, readPage, FrontmatterError } from './lib/pages.mjs'; + +/** Every tree the schema binds. `skills/` carries a skill's own frontmatter and README.md carries none. */ +const TREES = ['reference', 'guides', 'tutorials', 'concepts', 'learning']; +const ROOT_PAGES = ['AGENTS.md', 'CLAUDE.md', 'validation-log.md']; + +/** + * The trees the docs site renders as routes. The band below is a meta + * description budget, so it binds a page that becomes one and says nothing + * about a page the site excludes from rendering. + */ +const RENDERED = ['reference/', 'guides/', 'tutorials/', 'concepts/']; + +/** The site composes a title from the H1 and fails its own build over the budget. */ +const MAX_HEADING = 40; + +/** The meta description band the site's SEO gate holds a rendered page to. */ +const SCOPE_BAND = { min: 140, max: 160 }; + +const [outdir, root = process.cwd()] = process.argv.slice(2); + +if (outdir === undefined) { + console.error('usage: node .github/scripts/check-frontmatter.mjs [root]'); + process.exit(2); +} + +const from = resolve(root); +const to = resolve(outdir); +const findings = []; + +async function exists(path) { + try { + await access(join(from, path)); + + return true; + } catch { + return false; + } +} + +const pages = []; +for (const tree of TREES) pages.push(...(await pagesUnder(from, tree))); +for (const page of ROOT_PAGES) if (await exists(page)) pages.push(page); + +await mkdir(to, { recursive: true }); + +for (const page of pages) { + let read; + try { + read = await readPage(from, page); + } catch (error) { + if (!(error instanceof FrontmatterError)) throw error; + findings.push(error.message); + continue; + } + + await writeFile(join(to, `${page.replaceAll('/', '__').replace(/\.md$/, '')}.yml`), `${read.block}\n`); + + const heading = leadingHeading(read.body); + if (heading === null) findings.push(`${page} body does not open with an H1`); + else if (heading.length > MAX_HEADING) { + findings.push(`${page} H1 is ${heading.length} characters, over ${MAX_HEADING}: ${heading}`); + } + + const scope = read.values.get('scope'); + if (typeof scope === 'string' && RENDERED.some((tree) => page.startsWith(tree))) { + if (scope.length < SCOPE_BAND.min || scope.length > SCOPE_BAND.max) { + findings.push(`${page} scope is ${scope.length} characters, outside ${SCOPE_BAND.min} to ${SCOPE_BAND.max}`); + } + } + + for (const entry of read.values.get('depends-on') ?? []) { + if (!(await exists(entry))) findings.push(`${page} depends-on names a page that does not exist: ${entry}`); + } +} + +console.log(`frontmatter: read ${pages.length} pages, wrote ${pages.length} frontmatter blocks for ajv`); + +for (const finding of findings) console.error(`error: ${finding}`); + +process.exit(findings.length === 0 ? 0 : 1); diff --git a/.github/scripts/check-validation-consistency.mjs b/.github/scripts/check-validation-consistency.mjs new file mode 100644 index 0000000..d4b4fde --- /dev/null +++ b/.github/scripts/check-validation-consistency.mjs @@ -0,0 +1,115 @@ +#!/usr/bin/env node +/** + * Holds the provenance ledger and the pages to each other. This is the check + * nobody downstream can run: a consumer of this corpus sees a page and a tier, + * and cannot tell that the tier belongs to a page that no longer exists or that + * a page was never graded at all. + * + * Three arms: + * - a reference or guides page with no row in the ledger, + * - a ledger row naming a page that does not exist, + * - a page whose `key-modules` names a baseline the ledger does not pin. + * + * Usage: node .github/scripts/check-validation-consistency.mjs [root] + */ +import { readFile } from 'node:fs/promises'; +import { join, resolve } from 'node:path'; +import { pagesUnder, readPage } from './lib/pages.mjs'; + +/** The trees the ledger grades. Tutorials and concepts carry no tier by design. */ +const GRADED = ['reference', 'guides']; + +/** Every tree that carries `key-modules`, so a new one is covered when it lands. */ +const PINNED = ['reference', 'guides', 'tutorials', 'concepts']; + +/** The ledger's own path. U8 moves it into the rendered tree; both spellings resolve. */ +const LEDGER = ['validation-log.md', 'reference/validation.md']; + +const root = resolve(process.argv[2] ?? process.cwd()); + +/** The body of one `## ` section, by its exact heading text. */ +function section(source, heading) { + const pattern = new RegExp(String.raw`^## ${heading}\s*$([\s\S]*?)(?=^## |\Z)`, 'm'); + const found = pattern.exec(source); + + return found === null ? null : found[1]; +} + +async function readLedger() { + for (const path of LEDGER) { + try { + return { path, source: await readFile(join(root, path), 'utf8') }; + } catch (error) { + if (error.code !== 'ENOENT') throw error; + } + } + + return null; +} + +const ledger = await readLedger(); +const findings = []; + +if (ledger === null) { + console.error(`error: no provenance ledger at ${LEDGER.join(' or ')}`); + process.exit(1); +} + +const baselinesSection = section(ledger.source, 'Pinned baselines'); +const pagesSection = section(ledger.source, 'Pages'); + +if (baselinesSection === null) findings.push(`${ledger.path} has no "## Pinned baselines" section`); +if (pagesSection === null) findings.push(`${ledger.path} has no "## Pages" section`); + +/** + * A pin rather than a name: a version, a commit, or a branch. One bullet can + * pin two baselines at once, so every backticked token on a bullet line is a + * candidate and the pins are what gets dropped. + */ +function isPin(token) { + return /^v?\d/.test(token) || /^[0-9a-f]{7,40}$/.test(token) || token === 'main'; +} + +const baselines = [...(baselinesSection ?? '').matchAll(/^- .*$/gm)] + .flatMap((line) => [...line[0].matchAll(/`([^`]+)`/g)].map((found) => found[1])) + .filter((token) => !isPin(token)); + +/** The first cell of each table row is the page the row grades. */ +const rows = new Map(); +for (const found of (pagesSection ?? '').matchAll(/^\| *`([^`]+)` *\|/gm)) { + rows.set(found[1], (rows.get(found[1]) ?? 0) + 1); +} + +const graded = []; +for (const tree of GRADED) graded.push(...(await pagesUnder(root, tree))); + +for (const page of graded) { + if (page === ledger.path) continue; + if (!rows.has(page)) findings.push(`${page} has no row in ${ledger.path}`); +} + +for (const [page, count] of rows) { + if (!graded.includes(page)) findings.push(`${ledger.path} grades a page that does not exist: ${page}`); + if (count > 1) findings.push(`${ledger.path} grades ${page} in ${count} rows`); +} + +const pinned = []; +for (const tree of PINNED) pinned.push(...(await pagesUnder(root, tree))); + +for (const page of pinned) { + if (page === ledger.path) continue; + const { values } = await readPage(root, page); + + for (const entry of values.get('key-modules') ?? []) { + if (baselines.some((baseline) => entry.includes(baseline))) continue; + findings.push(`${page} key-modules names a baseline ${ledger.path} does not pin: ${entry}`); + } +} + +console.log( + `validation-consistency: ${graded.length} graded pages, ${rows.size} ledger rows, ${baselines.length} pinned baselines`, +); + +for (const finding of findings) console.error(`error: ${finding}`); + +process.exit(findings.length === 0 ? 0 : 1); diff --git a/.github/scripts/install-ripgrep.sh b/.github/scripts/install-ripgrep.sh new file mode 100755 index 0000000..1d1cb37 --- /dev/null +++ b/.github/scripts/install-ripgrep.sh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash +# Installs ripgrep into a job-owned directory and puts it on the job's PATH. +# +# The runner image does not carry ripgrep, and three checks are ripgrep rules. +# The version and its checksum are pinned here rather than taken from apt, +# because a rule that silently changes engine between runs is a rule nobody can +# reason about: the wallet arm and the casing arm both need PCRE2 lookarounds, +# which this build carries and a distribution build need not. +set -euo pipefail + +VERSION='15.2.0' +SHA256='33e15bcf1624b25cdd2a55813a47a2f95dbe126268203e76aa6a585d1e7b149c' +TARGET="ripgrep-${VERSION}-x86_64-unknown-linux-musl" + +archive="${RUNNER_TEMP}/${TARGET}.tar.gz" + +curl --fail --silent --show-error --location --output "${archive}" \ + "https://github.com/BurntSushi/ripgrep/releases/download/${VERSION}/${TARGET}.tar.gz" + +echo "${SHA256} ${archive}" | sha256sum --check --status + +tar --extract --gzip --file "${archive}" --directory "${RUNNER_TEMP}" + +echo "${RUNNER_TEMP}/${TARGET}" >> "${GITHUB_PATH}" +"${RUNNER_TEMP}/${TARGET}/rg" --version | head -1 diff --git a/.github/scripts/lib/pages.mjs b/.github/scripts/lib/pages.mjs new file mode 100644 index 0000000..e06a789 --- /dev/null +++ b/.github/scripts/lib/pages.mjs @@ -0,0 +1,147 @@ +/** + * Reads the frontmatter block every page in this corpus carries, and walks the + * page trees the checks run over. + * + * The reader covers the shapes `.github/frontmatter.schema.json` admits and + * nothing else: a scalar, a flow sequence, and a block sequence, each of + * strings. Anything else throws by name and line rather than parsing to + * something plausible, because a check that silently reads a shape it does not + * understand reports green on a page it never looked at. The schema itself is + * validated by ajv, which is a real YAML and JSON Schema implementation; this + * reader serves the arms ajv cannot express. + */ +import { readdir, readFile } from 'node:fs/promises'; +import { join, relative } from 'node:path'; + +const KEY = /^([A-Za-z][\w-]*):[ \t]*(.*)$/; +const ITEM = /^[ \t]+-[ \t]+(.*)$/; + +/** Strips one layer of matching quotes, which is all YAML needs here. */ +function unquote(value) { + const trimmed = value.trim(); + const quote = trimmed.slice(0, 1); + + if ((quote === '"' || quote === "'") && trimmed.length > 1 && trimmed.endsWith(quote)) { + return trimmed.slice(1, -1); + } + + return trimmed; +} + +/** Splits a flow sequence on the commas that sit outside a quoted item. */ +function splitFlow(body) { + const items = []; + let item = ''; + let quote = null; + + for (const character of body) { + if (quote !== null) { + if (character === quote) quote = null; + item += character; + continue; + } + if (character === '"' || character === "'") { + quote = character; + item += character; + continue; + } + if (character === ',') { + items.push(item); + item = ''; + continue; + } + item += character; + } + items.push(item); + + return items.map(unquote).filter((entry) => entry !== ''); +} + +export class FrontmatterError extends Error {} + +/** + * Returns the frontmatter as a map, the raw block for ajv, and the body that + * follows it. A page with no block throws: the site reads `scope` off every + * page it renders, so an absent block is a failed sync rather than a default. + */ +export function readFrontmatter(source, file) { + const lines = source.split('\n'); + + if (lines[0] !== '---') throw new FrontmatterError(`${file}:1 no frontmatter block`); + + const close = lines.indexOf('---', 1); + if (close === -1) throw new FrontmatterError(`${file}:1 frontmatter block is never closed`); + + const block = lines.slice(1, close); + const values = new Map(); + let current = null; + + for (const [offset, line] of block.entries()) { + const at = `${file}:${offset + 2}`; + if (line.trim() === '') continue; + + const item = ITEM.exec(line); + if (item !== null) { + if (current === null) throw new FrontmatterError(`${at} list item before any key`); + const list = values.get(current); + if (!Array.isArray(list)) throw new FrontmatterError(`${at} list item under a scalar key`); + list.push(unquote(item[1])); + continue; + } + + const key = KEY.exec(line); + if (key === null) throw new FrontmatterError(`${at} unreadable frontmatter line: ${line}`); + + const [, name, rest] = key; + current = name; + + if (rest === '') { + values.set(name, []); + continue; + } + if (rest.startsWith('[')) { + if (!rest.endsWith(']')) throw new FrontmatterError(`${at} flow sequence is never closed`); + values.set(name, splitFlow(rest.slice(1, -1))); + continue; + } + values.set(name, unquote(rest)); + } + + return { values, block: block.join('\n'), body: lines.slice(close + 1), bodyOffset: close + 2 }; +} + +/** The leading H1 of a body, or null when the body does not open with one. */ +export function leadingHeading(body) { + for (const line of body) { + if (line.trim() === '') continue; + const heading = /^# +(.*?)\s*$/.exec(line); + + return heading === null ? null : heading[1]; + } + + return null; +} + +/** Every markdown page under `directory`, repository-relative, sorted. */ +export async function pagesUnder(root, directory, found = []) { + let entries; + try { + entries = await readdir(join(root, directory), { withFileTypes: true }); + } catch (error) { + if (error.code === 'ENOENT') return found; + throw error; + } + + for (const entry of entries.sort((a, b) => a.name.localeCompare(b.name))) { + const path = join(directory, entry.name); + if (entry.isDirectory()) await pagesUnder(root, path, found); + else if (entry.isFile() && entry.name.endsWith('.md')) found.push(relative('.', path)); + } + + return found; +} + +/** The frontmatter of one page, with the page path carried for the message. */ +export async function readPage(root, page) { + return readFrontmatter(await readFile(join(root, page), 'utf8'), page); +} diff --git a/.github/scripts/strip-code.mjs b/.github/scripts/strip-code.mjs new file mode 100644 index 0000000..ebc769c --- /dev/null +++ b/.github/scripts/strip-code.mjs @@ -0,0 +1,129 @@ +#!/usr/bin/env node +/** + * Writes a copy of every markdown page with its code removed, so a prose rule + * never reads a code sample. + * + * Fenced blocks and inline code spans become blank space of the same shape: + * every character goes to a space and every newline survives, so a finding in + * the stripped copy carries the line and column of the source line it came + * from. The corpus writes account names, ABI type names, and shell snippets in + * code, and a prose rule that cannot tell code from prose either fails on a + * sample or gets turned off. + * + * Usage: node .github/scripts/strip-code.mjs [root] + */ +import { mkdir, readdir, readFile, writeFile } from 'node:fs/promises'; +import { dirname, join, relative, resolve } from 'node:path'; + +const SKIP = new Set(['.git', 'node_modules', '.github']); + +/** Every markdown page under `root`, repository-relative, sorted. */ +async function markdownFiles(root, directory = root, found = []) { + const entries = await readdir(directory, { withFileTypes: true }); + + for (const entry of entries.sort((a, b) => a.name.localeCompare(b.name))) { + if (entry.isDirectory()) { + if (SKIP.has(entry.name)) continue; + await markdownFiles(root, join(directory, entry.name), found); + continue; + } + if (entry.isFile() && entry.name.endsWith('.md')) { + found.push(relative(root, join(directory, entry.name))); + } + } + + return found; +} + +/** Same width, same newlines, no content. */ +function blank(text) { + return text.replaceAll(/[^\n]/g, ' '); +} + +/** + * Blanks fenced blocks first, because a fence is one opaque region: a backtick + * inside a shell sample is not a code-span delimiter, and pairing it with a + * later one would swallow the prose between them. + */ +function stripFences(source) { + const lines = source.split('\n'); + let fence = null; + + return lines + .map((line) => { + const opener = /^ {0,3}(`{3,}|~{3,})/.exec(line); + + if (fence === null) { + if (opener === null) return line; + fence = opener[1][0].repeat(opener[1].length); + return blank(line); + } + + const closer = new RegExp(String.raw`^ {0,3}${fence[0]}{${fence.length},}\s*$`).exec(line); + if (closer !== null) fence = null; + + return blank(line); + }) + .join('\n'); +} + +/** + * A code span opens on a backtick run and closes on the next run of the same + * length, which is the CommonMark rule. A run with no partner is a literal + * backtick and stays. + */ +function stripCodeSpans(source) { + const characters = [...source]; + const runs = []; + + for (let at = 0; at < characters.length; ) { + if (characters[at] !== '`') { + at += 1; + continue; + } + let end = at; + while (end < characters.length && characters[end] === '`') end += 1; + runs.push({ start: at, length: end - at }); + at = end; + } + + const open = new Map(); + const spans = []; + + for (const run of runs) { + const partner = open.get(run.length); + if (partner === undefined) { + open.set(run.length, run); + continue; + } + spans.push([partner.start, run.start + run.length]); + open.clear(); + } + + let stripped = source; + for (const [start, end] of spans.reverse()) { + stripped = stripped.slice(0, start) + blank(stripped.slice(start, end)) + stripped.slice(end); + } + + return stripped; +} + +export function strip(source) { + return stripCodeSpans(stripFences(source)); +} + +const [outdir, root = process.cwd()] = process.argv.slice(2); + +if (outdir === undefined) { + console.error('usage: node .github/scripts/strip-code.mjs [root]'); + process.exit(2); +} + +const from = resolve(root); +const to = resolve(outdir); + +for (const page of await markdownFiles(from)) { + const target = join(to, page); + await mkdir(dirname(target), { recursive: true }); + await writeFile(target, strip(await readFile(join(from, page), 'utf8'))); +} diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml new file mode 100644 index 0000000..575adad --- /dev/null +++ b/.github/workflows/checks.yml @@ -0,0 +1,354 @@ +# Every check this repository has. No job carries a `name`, so the check a +# reader sees and the check branch protection requires are both the job id. +# +# Every job can fail a pull request, because there is no other gate here. The +# two network-dependent arms are the exception and both sit off the +# pull-request path: external URL checking and starter signing run on the weekly +# schedule, so a third-party host outage never blocks a merge. +# +# Actions are pinned by commit sha with the version in the trailing comment. A +# tag is a moving target, and this workflow is what stands between the corpus +# and a bad merge. +name: checks + +on: + pull_request: + push: + branches: [main] + schedule: + # Weekly. The online link arm and the signing starters run here. + - cron: '17 6 * * 1' + +# Every job reads the tree and writes nothing back. +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + # A superseded pull-request run has nothing left to say. A push to the + # default branch is the record of what landed, so it runs to the end. + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + # A relative markdown link whose target does not exist. + links: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - name: Relative links resolve + if: github.event_name != 'schedule' + uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2.9.0 + with: + lycheeVersion: v0.24.2 + args: --offline --no-progress --exclude-path .git './**/*.md' + fail: true + + - name: External hosts answer + if: github.event_name == 'schedule' + uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2.9.0 + with: + lycheeVersion: v0.24.2 + args: --no-progress --max-concurrency 4 --timeout 20 --max-retries 2 --exclude-path .git './**/*.md' + fail: true + + # A `#fragment` with no matching heading id. This mirrors the docs site's own + # link gate and moves the failure to the pull request that caused it. + anchors: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + # A fragment resolves inside the checkout, so this arm is offline on + # every event. It reports a missing file as well as a missing + # fragment, because lychee has no fragments-only mode. + - name: Heading fragments resolve + uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2.9.0 + with: + lycheeVersion: v0.24.2 + args: --offline --include-fragments --no-progress --exclude-path .git './**/*.md' + fail: true + + # The frontmatter block every page carries, against the committed schema, + # plus the three arms a JSON Schema cannot express. + frontmatter: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: '24' + + # Also extracts each block as YAML for ajv below. `skills/` carries a + # skill's own frontmatter and README.md carries none, so the walk + # covers neither. + - name: H1 length, scope band, and depends-on targets + run: node .github/scripts/check-frontmatter.mjs "${RUNNER_TEMP}/frontmatter" + + - name: Frontmatter matches the schema + run: | + npx --yes ajv-cli@5.0.0 validate \ + --spec=draft2020 \ + -s .github/frontmatter.schema.json \ + -d "${RUNNER_TEMP}/frontmatter/*.yml" \ + --all-errors + + # Terms this corpus does not use, read on the stripped copy. + banned-terms: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: '24' + + - name: Install ripgrep + run: .github/scripts/install-ripgrep.sh + + - name: Strip code from every page + run: node .github/scripts/strip-code.mjs "${RUNNER_TEMP}/stripped" + + - name: No banned term in prose + run: | + # The pattern file carries comments and blank lines for the + # reader. A blank line is an empty regex that matches every + # line, so both go before ripgrep sees the file. + grep -vE '^[[:space:]]*(#|$)' .github/banned-terms.txt > "${RUNNER_TEMP}/banned-terms.rg" + + cd "${RUNNER_TEMP}/stripped" + rg --pcre2 --line-number --no-heading --color never -f "${RUNNER_TEMP}/banned-terms.rg" . && found=0 || found=$? + + case "${found}" in + 0) echo "::error::a banned term reached prose, see the matches above"; exit 1 ;; + 1) echo "no banned term in prose" ;; + *) echo "::error::ripgrep failed with ${found}"; exit "${found}" ;; + esac + + # An em-dash and a curly quote fail. A bold-header bullet fails on the two + # trees a reader arrives at and warns elsewhere. The AI-frequent vocabulary + # warns, because one use in a paragraph is permitted prose. + prose-bans: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: '24' + + - name: Install ripgrep + run: .github/scripts/install-ripgrep.sh + + - name: Strip code from every page + run: node .github/scripts/strip-code.mjs "${RUNNER_TEMP}/stripped" + + - name: No AI tell in prose + run: | + cd "${RUNNER_TEMP}/stripped" + failed=0 + + # One helper for both severities, so a rule cannot drift into + # reporting the opposite of what it means. ripgrep answers 0 + # for a match, 1 for none, and anything else is a tool failure + # that must not read as a pass. + scan() { + local severity="$1" message="$2" pattern="$3" + shift 3 + rg --pcre2 --line-number --no-heading --color never -e "${pattern}" "$@" && found=0 || found=$? + + case "${found}" in + 0) + echo "::${severity}::${message}" + if [ "${severity}" = error ]; then failed=1; fi + ;; + 1) ;; + *) + echo "::error::ripgrep failed with ${found}" + exit "${found}" + ;; + esac + } + + # The character classes are written as escapes so this file + # does not have to carry the characters it bans. + scan error 'em-dash: use a hyphen, a comma, a period, or parentheses' '\x{2014}' . + scan error 'curly quote: this corpus writes straight quotes' '[\x{2018}\x{2019}\x{201C}\x{201D}]' . + scan error 'bold-header bullet: write the sentence instead of a label and a colon' \ + '^\s*[-*+]\s+\*\*[^*]*[A-Za-z][^*]*(:\s*\*\*|\*\*\s*:)' reference guides + scan warning 'bold-header bullet outside the reference and guides trees' \ + '^\s*[-*+]\s+\*\*[^*]*[A-Za-z][^*]*(:\s*\*\*|\*\*\s*:)' --glob '!reference/**' --glob '!guides/**' . + # "key" and "landscape" are on the standard's list and off this + # one: this corpus writes about primary keys and key modules, + # and those false positives would bury the real findings. + scan warning 'AI-frequent vocabulary, rewrite a paragraph that carries three' \ + '(?i)\b(additionally|comprehensive|crucial|delve|deep dive|dive into|empower|enhance|facilitate|foster|garner|intricate|leverage|pivotal|showcase|streamline|tapestry|testament|underscore|utilize|vital|seamless|robust|boasts)\b' . + + exit "${failed}" + + # A misspelling in prose. Stripping is what lets a real account name pass as + # code and fail as prose. + spelling: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: '24' + + - name: Strip code from every page + run: node .github/scripts/strip-code.mjs "${RUNNER_TEMP}/stripped" + + - name: No misspelling in prose + uses: crate-ci/typos@8a48f81b6c64dcfea44b3633223084c4be58ac5f # v1.49.0 + with: + files: ${{ runner.temp }}/stripped + config: ./_typos.toml + + # A bare product token that is not title-cased. The corpus writes the account + # name and the product name in one sentence, and stripping is what separates + # them, which is why this is not folded into spelling. + casing: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: '24' + + - name: Install ripgrep + run: .github/scripts/install-ripgrep.sh + + - name: Strip code from every page + run: node .github/scripts/strip-code.mjs "${RUNNER_TEMP}/stripped" + + - name: Product names keep their casing + run: | + cd "${RUNNER_TEMP}/stripped" + + # A bare token only. A repository name, a package name, a + # hostname, and a path each carry a separator on one side, and + # each of those is a lowercase identifier a reader has to be + # able to copy. + pattern='(? Date: Tue, 18 Aug 2026 00:48:58 -0400 Subject: [PATCH 4/4] ci: let dependabot move the action pins the workflow introduces --- .github/dependabot.yml | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 .github/dependabot.yml diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..1bf4f49 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,11 @@ +# The only dependencies this repository declares are the actions its workflow +# pins by commit. Dependabot keeps those pins moving; the corpus itself has no +# package manifest to watch. +version: 2 +updates: + - package-ecosystem: github-actions + directory: / + schedule: + interval: weekly + commit-message: + prefix: ci