Repository navigation
Academy/memory v03 - #23
Merged
Merged
Conversation
- memory.py: contradiction-aware recycling (dead-antecedent set), ratio-gated contradiction transitions, evidence-window status counts (all off by default; v0.2 behavior preserved) - metargon_agent.py: adaptive probing (composite targets, support- weighted singles) and episode-local protection/probing gates - tools: memory_diagnosis.py (failure-mode separation), memory_experiment.py (matched-budget variant comparison) - tests: test_memory_v03.py (14 tests) - diagnosis: true clauses ARE generated (early, high support) but all hypotheses flip contradicted via cross-task nonevents; cap not binding; probes rarely target true antecedents
- memory_v03_runs.jsonl: append-only partial results (original/recycle/ window recorded; status in NIGHTLY_NOTES.md) - memory_merge.py: compose final summary from JSONL + paired CIs - docs/MEMORY_V03.md: report draft (numbers pending merge) - x/run_chunks.py: per-variant subprocess runner (robust to kills) - NIGHTLY_NOTES.md: continuation checklist for the morning
All 8 variants trained on the same 160-episode stream (seed 20261107, PYTHONHASHSEED=0) and evaluated on frozen grids (16 classic + 12 OOD configs) with paired bootstrap CIs. Measured: - probe / probe_local: train +0.144 / +0.106, classic transfer bit- identical (+0.0000) — positive training-side result, no transfer gain - recycle: neutral; protect_local: neutral in classic, +0.006 train - ratio / window: retain 57 verified hyps but classic numerically lower (0.344 / 0.312 vs 0.562), CIs include 0 — directionally negative - combined: strictly worse (train -0.175, classic 0.375) - no variant lifts held-out transfer above original on this grid Includes: memory_merge.py (jsonl->summary), review_runs.py, diagnosis_memory.json, policy states, report docs/MEMORY_V03.md.
Untrained student control on the same grids (classic 0.695 vs 0.562 trained; ood 0.417 vs 0.333) quantifies the negative-transfer cost of training on unseen configs — all trained variants fall below the fresh baseline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.