Skip to content

Latest commit

 

History

History
401 lines (333 loc) · 24.1 KB

authoring.md

File metadata and controls

401 lines (333 loc) · 24.1 KB

Framework-guided policy authoring

Integration update — October 6, 2026: the applicable workspace, MCP, assessment and review implementations are now on main. See the merge reconciliation for exact commits and qualification scopes. Earlier draft states and failed receipts below retain their dated meanings; human and release gates remain separate.

forge author converts an exact, reviewed applicability/gap baseline and explicit human assignments into a drafting plan and Markdown skeletons. Default Phase 1 builds write no substantive text beyond supplied human clause files and never interpolate answer values. Opt-in Phase 2 components substitute only explicitly bound public/internal values into Markdown; reports never copy raw values.

This is a technical foundation. The dated owner gate register records readiness and contract/content-boundary dispositions. Those assertions do not authenticate rights or independent review. Actual design-partner exercises, measured pilot outcomes and release approval remain open. Authoring states do not change PRD-058 lifecycle state.

Rust library migration remains a release gate: this feature extends the public exhaustive ForgeError and CLI command enums, so downstream matches may need new arms. The recorded release target is 2.0.0; see the API migration guidance. Publication still requires the recorded release approval and downstream review. This technical tranche does not claim source compatibility for those exhaustive matches.

The Authoring review exchange candidate adds a separate local /3 review of one complete current saved Phase 1 plan in the integrated open draft. Its guide separates earlier TEMP test, lint, compiled-CLI, Rustdoc and physical LLVM evidence from the later integration/hosted readback. Published v1.1.0 does not include this exchange; remaining platform, human, pilot and release gates stay open. Asserted review policy does not update the plan, answers, clauses, authoring state or Lifecycle record.

Commands

forge author plan --manifest project.json --format text
forge author plan --manifest project.json --format json
forge author plan --manifest project.json --output-dir planning
forge author build --manifest project.json --output-dir drafts --format json

Both commands print the requested plan format to stdout; diagnostics use stderr. The synthetic example provides a complete pinned input set that can be copied to a new directory and built offline. plan optionally writes plan.json and plan.txt. build requires a new output directory and writes those reports, provenance.json, and one policies/<policy-key>.md file per explicitly selected policy.

All input paths and output-directory paths resolve from the author manifest's directory. Phase 1 requires project_root to be exactly ".". Supply canonical relative descendant input paths: no absolute paths, drive prefixes, backslashes, symlinks, hard links, or aliases. A nested PRD-056 manifest may use leading ../ segments only when the dependency remains inside the author root. Internal canceled descents such as sub/../framework.json, ./, repeated separators, and trailing separators are rejected before capture; discarded directory components cannot hide aliases. The manifest's directory ancestry must also be free of symlinks. Use the physical project path (for example, /private/tmp rather than macOS's /tmp alias). Remote links embedded in framework resources are never fetched.

Exit statuses:

Code Meaning
0 Valid plan or build; every applicable gap is assigned or deferred and required context is available
1 Valid plan or build with unresolved gaps or sections blocked by required context
2 Invalid input, mismatched fingerprint, unsupported contract, unsafe path, or output failure

Exit 0 describes authoring bookkeeping only. A skeleton without human clauses can still be skeleton-ready; it is not a finished policy.

Scaffolding an empty pack (S-1)

forge author scaffold --manifest project.json

scaffold writes an empty forge.authoring-pack/1 template to the path pinned by authoring_pack in the manifest. It captures and exactly regenerates the pinned framework inventory, copies the project's exact baseline fingerprints, creates no topics, questions, policy families, control assignments or family assignments, and leaves the required reviewers collection and content_rights reviewer fields empty. A reviewer must supply that provenance — and update the authoring_pack byte pin — before plan or build accepts the pack; the template is intentionally not yet a usable pack (the forge mapping init fail-until-attested convention). The command never replaces an existing file, never follows a symlinked destination, and creates no assignment, approval evidence or lifecycle state. The destination's parent directory must already exist, mirroring the authoring output boundary. It fails closed on platforms without atomic no-replace publication. The command adds a public AuthorCommand variant, so it is included in the pending public Rust API/semver and migration gate.

Local input contracts

The Phase 1 closed JSON schemas are authoring-pack and author-project. Runtime validation also enforces cross-record references, exact hashes, logical uniqueness, value constraints, and relationships that JSON Schema alone cannot express.

An authoring pack (forge.authoring-pack/1) contains:

  • A stable key, version, exact baseline binding, and declared content-rights evidence.
  • Framework control IDs, human-authored topic titles, policy-family keys, and questions.
  • Explicit control-to-topic and topic-to-family assignments, each with reviewer, review time, and rationale.

An author project (forge.author-project/1) contains:

  • A stable project key, explicit as_of, baseline review, and local reviewer registry.
  • Byte-pinned applicability manifest, gap report, and authoring pack.
  • Explicit policies, typed question answers, gap deferrals, and human clause references.

Reviewer names, keys, times, rights statements, and rationales are asserted provenance. FORGE does not authenticate reviewers, prove independent review, grant rights to framework content, or decide whether an assignment is appropriate. Portable framework relationships use control IDs. Repository fixtures contain synthetic framework content only.

Exact baseline binding

PRD-056 emits forge.applicability-report/1 without a report hash or gap ID. Authoring pins SHA-256 of the exact supplied report bytes, including whitespace. The framework and optional resolved Catalog companion also use exact raw-file SHA-256. Pack and project baseline bindings must agree.

The loader securely captures the applicability manifest, framework, companion, and Mapping Collections, then recomputes the complete unfiltered PRD-056 analysis in a private local snapshot. The supplied report must equal that analysis after strict JSON decoding. Filtering a report, deleting controls, changing totals, adding unsupported fields, or pinning a fabricated report is rejected.

Applicable authoring gaps are exactly the PRD-056 categories applicable-unmapped and applicable-reviewed-no-relationship. Mapped controls, scope deferrals, exclusions, and unfinished applicability decisions are not silently added to this set. Every gap is assigned to at least one explicit policy/topic, explicitly deferred, or visibly unresolved. Multiple destinations count once in total = assigned + deferred + unresolved.

Gap IDs hash a length-prefixed, versioned tuple of report SHA and control ID. They change with an exact report revision; the original control ID remains visible. Phase 1 rejects stale inputs but does not compare old and new baselines.

Questions and clauses

Question types are string, integer, boolean, and string-list. Definitions include required/optional state, owner, sensitivity, source label, optional maximum age, and bounded validation constraints. No substantive defaults exist.

Each answer is explicitly provided or no-answer. Its question hash and pack hash bind the answer to the reviewed definition. Evaluation uses only the manifest's explicit as_of, never the current clock. Missing, stale, expired, or invalid answers are visible and block only dependent sections. Optional unavailable answers remain visible without blocking unrelated content. Reports include answer-record hashes and metadata, not values.

Human clauses name a policy, topic, exact gap IDs, byte-pinned Markdown source, and any required answer records by key and canonical SHA-256. Editing an answer record requires an explicit update to dependent clause pins. A stale clause pin or changed source file fails the invocation, even if that clause would otherwise be blocked. Human clauses are validated before either planning or building.

Each policy has one generated title and ordered topic sections. Human clauses are preserved verbatim; unsupported Markdown structure is rejected rather than silently rewritten. Missing content is a visible unresolved marker. Authoring states are planned, blocked-context, skeleton-ready, and human-draft-present. None is a governance or control-outcome judgment.

Provenance and deterministic output

forge.authoring-provenance/1 relates exact output spans to policy/topic keys, gap IDs, control IDs, reviewed assignment records, answer hashes, human clauses, and all captured input hashes. Spans are zero-based UTF-8 byte offsets with an exclusive end. They partition the Markdown bytes, including generated metadata and separators. Human clause spans identify the exact source bytes they preserve.

Plans, skeletons, reports, and provenance exclude absolute processing paths, temporary-directory names, wall-clock timestamps, locale, and environment data. The same exact input bytes produce identical artifact bytes in different directories and across repeated builds. Build to distinct output-directory names to compare generations; destination names do not enter artifact identity.

Sensitive answers remain in the local input file and are hashed in reports. Hashing is not encryption: low-entropy values can be guessed. Human clause text is intentionally emitted verbatim and must be reviewed for its intended audience.

Atomic publication and limits

The complete generation is staged under an existing, confined output parent and published with a single atomic no-replace directory rename. The destination must not already exist, even as an empty directory. There is no overwrite flag and no implicit parent-directory creation. Source files are rechecked immediately before publication. Keep concurrent edits externally serialized; input hashes identify the captured generation, not an authenticated or continuously locked workspace.

Directory publication supports Linux and macOS. On other platforms it fails closed before staging; read-only planning remains available. This limitation is explicit because the existing per-file rollback writer cannot guarantee an all-or-nothing result after interruption.

A process interruption before publication can leave a private hidden staging directory, while the destination remains absent. After the atomic rename, the destination is complete. A later directory-sync failure may report an error even though a complete generation has been published; it never exposes a partial generation. Existing output generations remain unchanged on validation failures.

Limits for the two authoring contracts include 2 MiB per manifest, 16 KiB per string, depth 32, 256 reviewers, 1,000 topics/families/policies, 4,096 questions/answers, 10,000 assignments/deferrals/clauses, 128 references per record, 1 MiB per human clause, and 50 MiB total captured source bytes and rendered artifacts. The schema and runtime validators specify narrower type-specific bounds where needed.

Imported PRD-056 manifests, reports, framework resources, and Mapping Collections retain their upstream semantic limits within the same 50 MiB total captured-input budget. They do not inherit the narrower authoring-contract string or manifest limits. Each confined read is bounded by the remaining aggregate budget before allocation.

Windows file publication

Single-file outputs share one no-replace file publisher. These include review queues, responses, merges and notification exports; POA&M scaffold manifests, authored native builds and reports, baseline, evidence-inspection, portfolio and outbound reports; reviewed-risk exports; assessment-result epoch pairs; evidence overlays; and authoring-pack scaffolds. It supports Linux, macOS and Windows. Directory generation publication (forge author build, suggestion and reuse generations) still supports only Linux and macOS, because the standard library offers no atomic no-replace directory rename on Windows. Other platforms refuse both kinds of publication before staging.

On Windows the publisher uses only the standard library, with no new dependency and no unsafe code. Each root and parent directory is opened without following reparse points (symlinks and junctions) and held without delete sharing. A reparse point anywhere on the path is refused. The file is created as new in a private staging directory next to the destination. It is held so that no other process can write, rename or delete it, written completely and flushed. One hard-link call then creates the destination name. That call fails if any entry already exists at the name, so an existing file, link or reparse point is never replaced, and a concurrent winner keeps its bytes. The destination appears only with the complete, flushed contents. The staging name is removed afterwards.

The Windows guarantees differ from the Unix publisher:

  • Names are resolved by path, not by descriptor-relative calls. Containment depends on the held, no-delete-share handles and on the reparse refusal.
  • No directory flush is issued. Durability of the new directory entry after a power loss depends on NTFS metadata journaling, not on an explicit sync.
  • The staging directory and the published file inherit the parent directory's access control list. There is no private 0700/0600 mode, so the published file is as readable as its parent allows.
  • Between the hard link and the removal of the staging name, the file briefly has two names. If that removal fails, the command reports an error after the complete destination exists, and a hard-link alias stays in the private staging directory. Inspect and remove it before retrying. Commands that refuse hard-linked inputs will refuse the published file until then.
  • Volumes without hard-link support, such as FAT or exFAT, fail closed without creating the destination.

These properties are covered by tests on the hosted windows-latest CI job. They are not an installed-platform or owner qualification of Windows support. The owner disposition that tracks this shared Windows path is recorded in docs/plans/2026-10-25-owner-dispositions.md (Cross-cutting, Windows publication).

Reuse candidates (PRD-066 Phase 0)

forge author reuse --manifest project.json --corpus corpus.json --format text
forge author reuse --manifest project.json --corpus corpus.json \
  --format json --output-dir reuse --html

reuse ranks verbatim blocks of operator-supplied Markdown against the sections of the drafting plan that are not human-draft-present. It is retrieval only: there is no model, no network, no credentials, and no prompt assembly, so the report can only point at bytes the operator supplied. It never writes into an authoring pack, project, plan, or pinned input.

The corpus (forge.reuse-corpus/1, schema) names each document by a portable descendant path relative to the corpus manifest's directory, an exact SHA-256, an operator status (approved or draft), and asserted rights and source labels, with optional topic/control hints. The corpus manifest itself is read through the confined, no-follow path and must be a regular file of at most 2 MiB: symlinks, hard links, devices, FIFOs and directories are rejected, and the manifest and every document are revalidated immediately before any generation is published. Documents are captured through the same confined, bounded path as every other authoring input: no symlinks, hard links, aliases, escaping paths, or stale pins. draft documents are excluded unless --include-draft is passed.

The report (forge.authoring-reuse/1, schema) carries section metadata, every captured input with its role, path and SHA-256, and per candidate source_key, source_path, source_sha256, a zero-based half-open span, a six-decimal score, machine-readable reasons (control-match, topic-terms, question-terms, same-family), and low_confidence. Rankings are deterministic: BM25 (k1 = 1.2, b = 0.75) over lowercased alphanumeric tokens plus a +2.0 exact control-ID boost and a +0.5 shared scope-title token boost, ordered by score, then source path, span start and source hash. Scores are quantised to six decimals, so repeated runs and separate directories produce byte-identical reports and published trees.

--max-candidates defaults to 5 and accepts at most 100; --min-score drops candidates below a value and defaults to 0, while candidates scoring below 1.0 are still emitted and flagged low_confidence rather than hidden. Only approved documents compete by default, at most 100 blocks per document are ranked, and at most 100 candidates are kept per section. Output artifacts (≤ 50 MiB total, ≤ 2048 files) are published with the same single-generation atomic no-replace rename as plan/build; --html adds an inert, escaped reuse.html view and requires --output-dir.

Code Meaning
0 Valid report; every reported section has at least one candidate
1 Valid report; some section has no candidate or only low-confidence candidates
2 Invalid input, mismatched pin, unsafe path, or output failure

A candidate is a pointer, not a recommendation: nothing in the report asserts that a candidate is correct, current, applicable, or approved. Ranking is a lexical heuristic and its score is not a quality claim. Adopting a candidate remains a manual act — a human writes the bytes into a clause file the way they do today. The retrieval-first tranche is described in the MVP reuse plan; generation, critique and provider phases remain in PRD-066.

Local suggestions (PRD-066, offline)

forge suggest runs a five-step, local-only pipeline for quarantined model suggestions. Nothing leaves the machine: there is no provider, no [ai] network configuration and no outbound request. This tranche imports operator-recorded responses. Process execution is disabled until FORGE can enforce OS filesystem and network confinement, execute verified bytes, and own the full process tree. prepare --adapter fingerprints the prospective adapter for consent and provenance; it does not run that executable. A recorded receipt identifies the import mode, not a model execution.

Step Command What it does
Prepare forge suggest prepare --manifest project.json --output-dir prepared --adapter ./local-model --model-id my-model [--corpus corpus.json --include-document KEY] [--answer KEY] [--redact-rules rules.txt] [--consent --operator-key KEY] Assembles the exact payload, the closed forge.suggest-request/1 allowlist, the byte-exact preview, and — only with --consent — a token bound to the payload and adapter digests
Run forge suggest run --request prepared/request.json --consent prepared/consent.json --output-dir run-1 --recorded-response recorded.json Imports a bounded recorded response without starting an adapter and publishes response.raw with a forge.suggest-run/1 receipt
Validate forge suggest validate --request prepared/request.json --run prepared/run-1/run.json --output-dir bundle-1 [--retain-raw] Decodes the response against its closed task schema, resolves every citation against the allowlist and the payload, and publishes the forge.suggestions/1 quarantine bundle
Review forge suggest review --bundle prepared/bundle-1/suggestions.json --decisions decisions.json --output-dir review-1 Binds your forge.suggest-dispositions/1 document to exactly that bundle and publishes it unchanged
Promote forge suggest promote --bundle … --request prepared/request.json --dispositions …/dispositions.json --destination project.json --output-dir promotion-1 Proposes a forge.author-project/1 patch that the destination's own validator accepts, and records it as proposed-unapproved

Exit codes: 0 complete, 1 the step succeeded but a human decision is still outstanding (no unresolved section to prepare, an empty response, an undecided suggestion, process execution disabled or an adapter the environment could not read), 2 a refusal — invalid or unsafe input, a consent that does not authorise the payload, a citation that does not resolve, or an existing destination. A refusal publishes nothing.

Guarantees worth knowing before you rely on it:

  • Every published generation is a new, atomically created directory; existing destinations are refused, and no command writes into your pack, project, plan or lifecycle artifacts.
  • Model text is untrusted: it is decoded against a closed schema, must cite allowlisted units, and a quoted citation must match the payload bytes exactly.
  • Evidence ratings (high/medium/low) are computed from citations, never from a model's self-reported confidence, and no output claims approval.
  • Redaction covers section titles, prompts and labels, matches original text, and bounds memory before appending output. Rules are recorded by digest only.
  • Secret-shaped input or decoded model text (including JSON escapes and notes) is refused before the request or bundle is published. These pattern checks do not certify that arbitrary text is free of sensitive information.
  • Promotion validates a private snapshot of the destination's captured inputs, including answer pins and current pack/report relationships. Validation never stages files inside the destination. Review it, apply the clause files by hand, then use the ordinary forge author plan/build path.

Contracts and gates: suggestions pipeline plan, gate register.

Phase 2 extensions and remaining scope

Explicit component instances, M-13 impact comparisons, offline HTML, and draft-only lifecycle handoff are described in Phase 2 usage. These opt-in contracts preserve the Phase 1 inputs and default command behavior above. Phase 1 stale-input rejection alone did not implement M-13; its technical evidence belongs to the Phase 2 comparison matrix. Phase 3 design-partner packs, measured authoring exercises, legal/content acceptance, human readiness, pilot metrics and release approval remain pending. AI, web editing, remote registries, downloads, connectors, collaboration and hosted services remain outside this tranche.

See the implementation plan for interface ownership, requirement disposition, and the verification/delivery workflow. The review dispositions record confirmed fixes and source-backed explanations for the initial exact-commit OCR findings.

The optional Phase 2 component extension, impact request and handoff request have separate closed schemas. Runtime checks additionally enforce byte limits (JSON Schema string lengths count characters), exact pins, cross-record relationships and valid dates; see Phase 2 contracts.

The Phase 2 interface matrix and Phase 2 review dispositions retain implementation, review and compatibility evidence.