From 06882f94b721159eea3f92b2f00da4c27441d0e6 Mon Sep 17 00:00:00 2001 From: Daniel Meyer Date: Fri, 11 Sep 2026 23:43:35 +0000 Subject: [PATCH] docs: Spec 004 M2 doctrine + hunt plan + pin hygiene Amend dataset doctrine so wrap/seed are lawful INFERRED gold while AMC/Foundations stay preferred OBSERVED. Add hunt plan, refresh handoff pin to main tip, stamp STATUS, and document pre-rebalance zip quarantine. --- README.md | 2 + STATUS.md | 3 ++ docs/aaron-train-handoff.md | 9 ++++- docs/amc-foundations-hunt-plan.md | 42 +++++++++++++++++++++ specs/004-encoder-gate/dataset.md | 61 +++++++++++++++++++++++-------- 5 files changed, 101 insertions(+), 16 deletions(-) create mode 100644 docs/amc-foundations-hunt-plan.md diff --git a/README.md b/README.md index 18512b2..260382b 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,8 @@ The Validate badge may stay red when Actions billing blocks the workflow even if | Spec 000–003 | Specified + code | | Spec 004 encoder gate | Specified · `name_gate` **false** · dataset note landed | | Local gold SoT | `~/.semion/corpus/dataset.jsonl` — **482** gold (**not in git**) | +| OBSERVED share | **14 / 482** (~3%) — wrap/seed lawful INFERRED; AMC/Foundations preferred | +| Hunt plan | [`docs/amc-foundations-hunt-plan.md`](docs/amc-foundations-hunt-plan.md) | | Splits | train **330** · val **50** · test **102** | | keep_weak | **150** (demoted wrap-symbol; **not** gold) | | Spec 004 class floors | **PASS** (icon/index/symbol ≥15%; mixed/NC ≥8%) | diff --git a/STATUS.md b/STATUS.md index 85ddedd..76759f5 100644 --- a/STATUS.md +++ b/STATUS.md @@ -21,3 +21,6 @@ | Boof ALLOW_TRAIN | **false** — Aaron trains on Spark | | Handoff doc | [`docs/aaron-train-handoff.md`](docs/aaron-train-handoff.md) | | README | operator front door expanded (stack parity with Athanor) | +| Spec 004 doctrine | M2 amend: wrap/seed lawful INFERRED gold; OBSERVED AMC/Foundations preferred | +| Pre-rebalance zip | quarantined under `/workspace/semion-gold/quarantine/` (404-row pack) | +| OBSERVED share | **14 / 482** (~3%) — hunt plan: `docs/amc-foundations-hunt-plan.md` | diff --git a/docs/aaron-train-handoff.md b/docs/aaron-train-handoff.md index 28572de..0665f5e 100644 --- a/docs/aaron-train-handoff.md +++ b/docs/aaron-train-handoff.md @@ -2,7 +2,7 @@ **Audience:** Aaron Godbout (Spark) **Date:** 2026-09-11 PT -**Repo pin:** `scrimshawlife-ctrl/Semion@9e29976` (docs tip of this branch may be newer) +**Repo pin:** `scrimshawlife-ctrl/Semion@ae47d9f` (docs tip may advance with this branch) ## Pack @@ -32,8 +32,15 @@ Class floors PASS (icon/index/symbol ≥15%; mixed/NC ≥8%). Train symbol share 5. Encoder gate: `specs/004-encoder-gate/`. `name_gate` stays **false** until your eval is green. 6. HF Hub skipped (Danny). Boof `ALLOW_TRAIN` is false — run train on Spark. +## Doctrine (post-audit) + +- Wrap/seed INFERRED rows are **lawful M2 gold** under Spec 004 amend (`specs/004-encoder-gate/dataset.md`). +- Prefer `gold_observed.jsonl` for T0/T1 paths; grow OBSERVED via [`amc-foundations-hunt-plan.md`](amc-foundations-hunt-plan.md). +- Pre-rebalance zip is **quarantined** on the operator box (`/workspace/semion-gold/quarantine/`) — do not train from it. + ## Do not - Mix `keep_weak` into gold without a weak-lane flag - Treat Notion/email as a green name-gate - Publish to Hugging Face without Danny `ALLOW_HUB` +- Train from the quarantined pre-rebalance (404-row) zip diff --git a/docs/amc-foundations-hunt-plan.md b/docs/amc-foundations-hunt-plan.md new file mode 100644 index 0000000..db98d75 --- /dev/null +++ b/docs/amc-foundations-hunt-plan.md @@ -0,0 +1,42 @@ +# AMC / Foundations hunt plan — Semion OBSERVED growth + +**Status:** PLAN (no auto-harvest) +**Date:** 2026-09-11 PT +**Why:** M2 gold floors PASS, but Spec 004 preferred lanes are thin — **14 OBSERVED / 482** (~3%). Wrap+seed are lawful INFERRED mass; OBSERVED still preferred before `name_gate`. + +## Goal + +Raise OBSERVED AMC + Foundations gold without inventing packets. + +| Milestone | Target | Gate | +|-----------|-------:|------| +| M2 floors (done) | 482 gold | Spec 004 floors PASS | +| OBSERVED share | ≥ **15%** of gold (~72+ of 482, or grow pool) | operator stamp | +| Size target | train **600** / test **150** | Spec 004 targets | +| Refuse placement | new NC/refuse → **test/val only** | Spec 004 split law | + +## Sources (OBSERVED only when labeled) + +1. **AMC LEARN** — LEARN-0002 primary; hunt LEARN-0001 / 0003+ only if filled packets exist (prior hunt: **0** new). +2. **Foundations** — VisualSemioticParse / SemiosisFrame inventories with a **single non-empty** bucket. +3. **Refuse** — dual-use / NOT_COMPUTABLE probes; append to **test/val**, not train. + +## Anti-goals + +- Do not re-flood wrap-symbol into gold (use `keep_weak`). +- Do not treat TEACH abstracts as gold. +- Do not invent AMC rows when inventories are empty. +- Do not flip `name_gate` or Hub from volume alone. +- Do not train from quarantined pre-rebalance zip (`/workspace/semion-gold/quarantine/`). + +## Operator sequence + +1. Inventory AMC dumps / Notion LEARN packets for unfilled → filled deltas. +2. Inventory Foundations JSON for new non-empty buckets. +3. If **0** new packets: **STOP** and ask Danny for source drop (no synthetic fill). +4. If packets exist: feed SEMION gold bot → append OBSERVED → rebuild pack → refresh Notion SHA. +5. Move future refuse harvests to test/val; leave existing M2 train NC as known debt unless Danny yes to reshuffle. + +## Current block (OBSERVED 2026-09-11) + +Expand AMC/Foundations hunt returned **0 new packets**. Next growth needs operator-supplied filled inventories or new LEARN packs. diff --git a/specs/004-encoder-gate/dataset.md b/specs/004-encoder-gate/dataset.md index b4912bf..64f1fa6 100644 --- a/specs/004-encoder-gate/dataset.md +++ b/specs/004-encoder-gate/dataset.md @@ -2,6 +2,7 @@ **Status**: SHADOW note. Not a harvest. name_gate still false. **Date**: 2026-09-11 +**Amend**: M2 rebalance doctrine (2026-09-11 PT) — wrap/seed as lawful INFERRED gold; OBSERVED AMC/Foundations still preferred. ## Verdict @@ -10,12 +11,22 @@ Train a **classifier**, not a speaker. | Axis | Choice | |---|---| | Type | Supervised `sign_class` (+ optional `is_sign`). No chat SFT. No DPO. No interpretant-as-prose | -| Source | AMC LEARN gold first. Foundations inventory second. TEACH abstracts are provenance only | +| Source | **Preferred:** AMC LEARN + Foundations inventories as **OBSERVED**. **Lawful M2 mass:** `wrapped_sign_atom` + Peirce/`seed_fixture` as **INFERRED** gold when floors pass. TEACH abstracts = provenance only | | Shape | jsonl `semion.dataset.row.v0` | -| Size | Floor **300 / 100** train/holdout. Target **600 / 150**. Dual-use refuse **30** held out of train | +| Size | Floor **300 / 50 / 100** train/val/test. Target **600 / 80 / 150**. Dual-use refuse floor **20** / target **30**, preferred **held out of train** | T0 already reads AMC labels. An encoder only earns a name if it beats those rules on a frozen holdout that includes mixed class and missing-object rows. +## Epistemic tiers (M2) + +| Tier | Meaning | May count toward Spec 004 gold floors? | +|---|---|---| +| `OBSERVED` | Operator / AMC LEARN / Foundations inventory gold | **Yes** (preferred) | +| `INFERRED` | Lawful wrap (`wrapped_sign_atom`) or Peirce pedagogy (`seed_fixture`) after SEMION label | **Yes** for M2 floors / Aaron pack | +| `keep_weak` | Demoted wrap-symbol (or similar) | **No** — weak lane only | + +Do **not** treat Notion delivery, email, or pack existence as a green `name_gate`. + ## Type Allowed heads: @@ -36,18 +47,33 @@ If a later cycle wants a retrieval encoder, that is a different artifact name an ## Source -| Lane | Role | Gold? | -|---|---|---| -| AMC LEARN-0002 packets | primary | yes, when `classification` is set | -| Foundations VisualSemioticParse / SemiosisFrame inventories | visual + frame | yes when a single inventory bucket is non-empty | -| `fixtures/seed/atoms.jsonl` | eval smoke | fixture, not SoT | -| TEACH arXiv abstract chunks | provenance / teaching | no | -| Orchestra `peircean-signs` | structure hint | no | -| Hyperlex slang harvest | form only | no unless wrapped as `sign_atom` with object or label | -| Athanor tradition atoms | structure only | no unless wrapped as `sign_atom` | -| HollerSports / settled forecast rows | Brier | never | - -Local SoT when harvest opens: `~/.semion/corpus/dataset.jsonl`. Not in git. +| Lane | Role | Gold? | Epistemic | +|---|---|---|---| +| AMC LEARN-0002 packets | **preferred primary** | yes, when `classification` is set | OBSERVED | +| Foundations VisualSemioticParse / SemiosisFrame inventories | **preferred visual + frame** | yes when a single inventory bucket is non-empty | OBSERVED | +| `wrapped_sign_atom` (Hyperlex/Athanor wrap) | **lawful M2 mass** | yes after SEMION label + Spec 004 floors | INFERRED | +| Peirce / pedagogy `seed_fixture` harvest | **lawful M2 mass** | yes for icon/index/mixed/NC probes | INFERRED | +| Demoted wrap-symbol | weak lane | **no** (`keep_weak.jsonl`) | — | +| `fixtures/seed/atoms.jsonl` | eval smoke | fixture, not SoT | — | +| TEACH arXiv abstract chunks | provenance / teaching | no | — | +| Orchestra `peircean-signs` | structure hint | no (until minted as seed_fixture) | — | +| Hyperlex slang harvest (raw) | form only | no unless wrapped as `sign_atom` with object or label | — | +| Athanor tradition atoms (raw) | structure only | no unless wrapped as `sign_atom` | — | +| HollerSports / settled forecast rows | Brier | never | — | + +Local SoT: `~/.semion/corpus/dataset.jsonl` (+ `keep_weak.jsonl`). Not in git. + +### M2 sealed snapshot (OBSERVED 2026-09-11 PT) + +| Slice | n | Notes | +|---|---:|---| +| gold pool | 482 | train 330 / val 50 / test 102 — floors **PASS** | +| OBSERVED | 14 | amc_learn 12 · foundations 2 | +| INFERRED wrap | 240 | `wrapped_sign_atom` | +| INFERRED seed | 228 | `seed_fixture` | +| keep_weak | 150 | demoted wrap-symbol — not gold | + +**Honesty:** floors pass; preferred OBSERVED share is still thin (~3%). Grow OBSERVED via the hunt plan before raising `name_gate`. Targets 600/150 are **not** hit. ## Shape @@ -72,7 +98,7 @@ Minimum live fields: | train | 300 | 600 | | val | 50 | 80 | | test / holdout frozen | 100 | 150 | -| dual-use refuse (not train) | 20 | 30 | +| dual-use refuse (preferred **not train**) | 20 | 30 | Class floor on the gold pool (train+val+test): icon, index, symbol each ≥ 15%. mixed and NOT_COMPUTABLE each ≥ 8%. @@ -83,7 +109,12 @@ This is probe-scale. It is not a 7B SFT corpus. Do not pad with synthetic mind-t - Freeze `test` hashes before any encoder run. - No source_run_id in both train and test. - Dual-use refuse rows never appear as positive `is_sign` gold. +- **Preferred:** `NOT_COMPUTABLE` / dual-use refuse live in **val/test** (held out of train). M2 currently has most NC in train (`seed_fixture`) — known debt; next harvest should move new refuse to test/val only. ## E-S3 bind N in the gate is the **floor** table above until an operator raises it in this file. + +## Hunt plan pointer + +See [`docs/amc-foundations-hunt-plan.md`](../../docs/amc-foundations-hunt-plan.md).