Implement Stage 17 hidden layer growth

ARCHITECTURE.md

@@ -356,13 +356,16 @@ teach("paris is the capital of france")
 
 ### When a Layer Grows
 
-Real new-depth layer growth is a later milestone. Stage 7 widens the current
-hidden layer and keeps the engine in its proven two-layer shape.
+Stage 17 adds real new-depth layer growth. Hidden layers are all layers before
+the final output layer. The output layer reads a flattened vector of every
+hidden activation, so adding a new hidden layer appends new output columns
+without rewriting older bound answer columns.
 
 A new layer is added when:
-- All neurons in every layer are `Frozen` or `Guarded`
-- Loss is still above threshold after MAX_LAYER_ATTEMPTS
-- Network depth is below MAX_LAYERS
+- Existing hidden memory is saturated (`Guarded` or `Frozen`) or the deepest
+  hidden layer has reached `MAX_NEURONS_PER_LAYER`
+- Loss is still above threshold after update attempts
+- Network depth is below `MAX_LAYERS`
 
 ### Growth is Bounded
 

BENCHMARKS.md

@@ -6,12 +6,12 @@ Mode: `full`
 
 | ID | Benchmark | Result | Unit | Detail |
 |---|---|---:|---|---|
-| B1 | single teach | 0.0384 | ms/op | 25 iterations |
-| B2 | single ask | 0.0190 | ms/op | 200 iterations |
-| B3 | .manas save | 0.4677 | ms/op | 25 iterations |
-| B4 | .manas load | 0.1514 | ms/op | 50 iterations |
-| B5 | tokenizer 1000 words | 0.3946 | ms/op | 100 iterations |
-| B6 | anti-forgetting proof | 0.3630 | s | single fixed seed |
+| B1 | single teach | 0.0446 | ms/op | 25 iterations |
+| B2 | single ask | 0.0187 | ms/op | 200 iterations |
+| B3 | .manas save | 0.1718 | ms/op | 25 iterations |
+| B4 | .manas load | 0.1518 | ms/op | 50 iterations |
+| B5 | tokenizer 1000 words | 0.3843 | ms/op | 100 iterations |
+| B6 | anti-forgetting proof | 0.4043 | s | single fixed seed |
 | B7 | 1000-neuron footprint | 419.6875 | KiB | estimated heap footprint, total=1000 |
 | B8 | brain growth per fact | 768.8750 | bytes/fact | n=32, min=646, max=4534 |
 

CHANGELOG.md

@@ -28,6 +28,7 @@ Manas uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - Stage 14 — Inspect, neurons, and debug commands.
 - Stage 15 — Compression and forget command.
 - Stage 16 — Benchmarks and test suite.
+- Stage 17 — Layer growth.
 
 ### Added
 
@@ -114,10 +115,18 @@ Manas uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
   persistence, growth, protection, compression, freshness, and ingestion.
 - Added CI benchmark smoke coverage through
   `cargo bench -p manas-benches -- --quick`.
+- Added real hidden-layer growth in `manas-core`; `grow_layer` now inserts a new
+  hidden layer before the output layer instead of widening layer 0.
+- Generalized forward caches, backpropagation, trainer growth, bound readout,
+  diagnostics, and `.manas` validation for multi-hidden-layer networks.
+- Preserved older bound-memory answers by keeping output weights connected to a
+  flattened hidden-activation readout.
+- Added Stage 17 tests for layer growth, max-layer bounds, deep gradients,
+  trainer-triggered layer insertion, deep persistence, and CLI inspect/query.
 
 ### Next
 
-- Stage 17 — Layer growth.
+- Stage 18 — Internet agent.
 
 ---
 

README.md

@@ -134,7 +134,8 @@ Manas v2 is in active development. The roadmap follows a strict rule:
 | Stage 14 | Inspect, neurons, and debug commands | Complete |
 | Stage 15 | Compression and forget command | Complete |
 | Stage 16 | Benchmarks and test suite | Complete |
-| Stage 17+ | Layer growth, future agents | Planned |
+| Stage 17 | Layer growth | Complete |
+| Stage 18+ | Internet agent, language generation | Planned |
 
 Stages 1 and 2 are preserved as a standalone proof in
 `manas-core/src/experiment.rs`. Stage 3 promotes the proven engine into
@@ -168,6 +169,9 @@ Stage 16 adds a dedicated benchmark crate, committed benchmark report, CI
 benchmark smoke test, and an eight-test integration gate covering the demo,
 anti-forgetting, persistence, growth, protection, compression, freshness, and
 ingestion.
+Stage 17 adds real hidden-layer growth: when existing hidden memory is saturated,
+Manas can insert a new hidden layer before the output layer while keeping older
+bound answer columns intact.
 
 Run the proof:
 
@@ -247,6 +251,13 @@ Run the Stage 16 integration proof:
 cargo test -p manas-cli stage16
 ```
 
+Run the Stage 17 layer-growth proof:
+
+```bash
+cargo test layer_growth
+cargo test stage17
+```
+
 Run the benchmarks:
 
 ```bash

ROADMAP.md

@@ -85,7 +85,7 @@ from Stage 2 onward.**
 | Stage 14 | Inspect, neurons, and debug commands | Complete |
 | Stage 15 | Compression and forget command | Complete |
 | Stage 16 | Benchmarks and test suite | Complete |
-| Stage 17 | Layer growth | Planned |
+| Stage 17 | Layer growth | Complete |
 | Stage 18 | Internet agent (future) | Planned |
 | Stage 19 | Language generation (future) | Planned |
 
@@ -2054,11 +2054,26 @@ fn deeper_network_can_represent_more_facts() {
 
 ### Done When
 
-- [ ] All layer growth tests pass
-- [ ] Anti-forgetting test passes with 2-layer networks
-- [ ] Layer count never exceeds `MAX_LAYERS`
-- [ ] `manas inspect` shows correct layer count and per-layer stats
-- [ ] `cargo test -p manas-core layer_growth` passes clean
+- [x] All layer growth tests pass
+- [x] Anti-forgetting test passes with 2-layer networks
+- [x] Layer count never exceeds `MAX_LAYERS`
+- [x] `manas inspect` shows correct layer count and per-layer stats
+- [x] `cargo test -p manas-core layer_growth` passes clean
+
+### Stage 17 Implementation Notes
+
+- Generalized `Network` to support multiple hidden layers plus a final output
+  layer.
+- Kept old bound answer columns stable by making the output layer read a
+  flattened vector of all hidden activations.
+- Changed `grow_layer` from Stage 7 compatibility widening into real hidden
+  layer insertion before the output layer.
+- Generalized forward caches, backpropagation, bound readout, trainer growth,
+  diagnostics, and `.manas` validation for deep networks.
+- Kept compression conservative: hidden compaction still only runs on the
+  original two-layer topology.
+- Added Stage 17 coverage for core growth, deep gradients, trainer layer growth,
+  deep persistence, and CLI inspect/query.
 
 ---
 

manas-cli/tests/stage17_layer_growth.rs

@@ -0,0 +1,123 @@
+use std::fs;
+use std::path::{Path, PathBuf};
+use std::process::{Command, Output};
+use std::time::{SystemTime, UNIX_EPOCH};
+
+use manas_core::Network;
+use manas_learn::Trainer;
+use manas_store::{BrainState, ManasBrain, VocabEntry};
+
+const EMBED_DIM: usize = 32;
+const LEARNING_RATE: f32 = 0.01;
+
+#[test]
+fn stage17_cli_inspects_and_queries_deep_brain() {
+    let dir = temp_dir("cli-deep");
+    let mut network = Network::new_empty(EMBED_DIM);
+    let mut trainer = Trainer::with_seed(42, EMBED_DIM, LEARNING_RATE);
+
+    trainer
+        .learn(&mut network, "cat", "small animal with fur")
+        .unwrap();
+    trainer
+        .learn(&mut network, "paris", "city in france")
+        .unwrap();
+    for neuron in &mut network.layers[0].neurons {
+        neuron.guard_all();
+    }
+    let report = trainer
+        .learn(&mut network, "rust", "systems programming language")
+        .unwrap();
+    assert_eq!(report.layers_grown, 1);
+    assert_eq!(network.layer_count(), 3);
+
+    ManasBrain::new(dir.join("brain.manas"))
+        .save_state(&BrainState::new(network, store_vocab(&trainer)))
+        .unwrap();
+
+    let inspect = run(&dir, &["inspect"]);
+    assert_success(&inspect);
+    let inspect_stdout = stdout(&inspect);
+    assert!(
+        inspect_stdout.contains("total layers          : 3"),
+        "{inspect_stdout}"
+    );
+    assert!(
+        inspect_stdout.contains("layers grown          : 1"),
+        "{inspect_stdout}"
+    );
+    assert!(
+        inspect_stdout.contains("layer 2 id=2 activation=linear"),
+        "{inspect_stdout}"
+    );
+
+    let ask = run(&dir, &["ask", "What is Rust?"]);
+    assert_success(&ask);
+    let ask_stdout = stdout(&ask);
+    assert!(
+        ask_stdout.contains("Answered from\n  neural weights"),
+        "{ask_stdout}"
+    );
+    assert!(
+        ask_stdout.contains("systems") || ask_stdout.contains("programming"),
+        "{ask_stdout}"
+    );
+
+    cleanup_dir(dir);
+}
+
+fn store_vocab(trainer: &Trainer) -> Vec<VocabEntry> {
+    trainer
+        .encoder
+        .export_vocab()
+        .into_iter()
+        .map(|entry| VocabEntry {
+            token: entry.token,
+            id: entry.id,
+            embedding: entry.embedding,
+        })
+        .collect()
+}
+
+fn run(dir: &Path, args: &[&str]) -> Output {
+    Command::new(env!("CARGO_BIN_EXE_manas"))
+        .args(args)
+        .current_dir(dir)
+        .output()
+        .unwrap()
+}
+
+fn assert_success(output: &Output) {
+    assert!(
+        output.status.success(),
+        "status: {:?}\nstdout:\n{}\nstderr:\n{}",
+        output.status.code(),
+        stdout(output),
+        stderr(output)
+    );
+}
+
+fn stdout(output: &Output) -> String {
+    String::from_utf8_lossy(&output.stdout).into_owned()
+}
+
+fn stderr(output: &Output) -> String {
+    String::from_utf8_lossy(&output.stderr).into_owned()
+}
+
+fn temp_dir(name: &str) -> PathBuf {
+    let nanos = SystemTime::now()
+        .duration_since(UNIX_EPOCH)
+        .unwrap()
+        .as_nanos();
+    let dir = std::env::temp_dir().join(format!(
+        "manas-stage17-{name}-{}-{nanos}",
+        std::process::id()
+    ));
+    fs::create_dir_all(&dir).unwrap();
+    dir
+}
+
+fn cleanup_dir(path: PathBuf) {
+    let _ = fs::remove_dir_all(path);
+}