Closure Engineering

Research Preview — hypothesis under active evaluation

Closure Engineering investigates whether coding agents need a separate project-completeness layer: one that searches for necessary capabilities, states, invariants, constraints, and failure paths that are absent from the visible task graph.

Required Space
- Observed and Independently Verified Space
= Closure Gaps

This repository is an early research preview, not a validated agent framework, benchmark release, or effectiveness claim.

The distinction

Graph Engineering organizes and executes known work.

Closure Engineering asks whether the known graph is complete enough for the required outcome.

Goal
→ Closure discovers missing nodes and constraints
→ Graph organizes dependencies and execution paths
→ Coding Agent implements
→ Tests and evaluators produce evidence
→ Closure reopens unsupported or missing requirements

Graph without Closure can execute an incomplete plan precisely. Closure without Graph can degrade into an unbounded checklist. The intended architecture is Closure over Graph.

Falsifiable research question

Under the same model, executor, budget, and task set:

Does a Closure Harness improve critical-gap recall while controlling false discoveries, token use, elapsed time, cost, and rework?

The planned controls separate:

1. an ordinary coding-agent workflow;
2. Graph-only planning over known requirements;
3. Graph + Closure, which may propose missing capabilities and failure paths.

Current evidence

The evidence is mixed and does not yet establish a causal benefit:

• one controller-held development screen showed a positive discovery signal;
• a fresh validation replication failed its frozen effectiveness and cost criteria;
• a delivery-first variant restored completion but remained rejected by its frozen discovery and precision rule;
• the first blinded 30-finding adjudication calibration failed: raw agreement was 80% and Cohen's kappa was 0.655, below required minima of 85% and 0.70;
• all six calibration disagreements were resolved by a blinded third vote, but tie-breaking could not repair the failed primary thresholds;
• a hierarchical Adjudication v2 is now a draft and requires a non-scored practice set plus a completely fresh calibration sample.

Failed and negative results are retained because the project is designed to be falsifiable.

See RESEARCH_STATUS.md for claim-level status.

What this preview contains

• the current research framing and architecture;
• a transparent summary of positive and negative internal evidence;
• the draft hierarchical Novel Finding Adjudication v2 protocol;
• a blinded reviewer guide and machine-readable response schema;
• a standard-library semantic validator and tests.

Run the public checks:

python3 -m unittest discover -s tests -v
python3 -m json.tool schemas/calibration-reviewer-response-v2.schema.json
python3 -m json.tool \
  schemas/examples/calibration-reviewer-response-v2.example.json

What this preview excludes

It intentionally excludes:

• hidden tests and canonical gap answers;
• task-to-arm and source-run mappings;
• raw reviewer responses and reviewer identities;
• evaluator-custody reports;
• provider credentials and local configuration;
• historical run logs and tested-task snapshots;
• any claim that Closure Engineering is already effective.

These boundaries are part of the experiment, not missing documentation.

Next gates

1. Build a non-scored v2 reviewer practice set.
2. Freeze v2 protocol, schema, reviewers, sampling, and analysis by hash.
3. Run a fresh blinded calibration with no v1 finding reuse.
4. Proceed only if raw agreement is at least 85%, kappa is at least 0.70, and unresolved adjudication errors equal zero.
5. Run fresh Graph-only versus Graph + Closure experiments under independent hidden evaluation.

See docs/roadmap.md.

Feedback

The most useful feedback concerns:

• whether the Required/Observed Space distinction is operationally clear;
• whether Graph-only is a sufficient planning control;
• whether public entailment and latent necessity are separable in the v2 adjudication protocol;
• what evidence would falsify the Closure hypothesis.

Please do not submit hidden-test content, credentials, private task data, or model-provider logs.

License status

No open-source license has been selected for this research preview. Do not assume that public visibility is an open-source license. A license will be chosen before a benchmark or reusable software release.