Share request evidence and replan missing analysis prerequisites

.github/workflows/ci.yml

@@ -7,7 +7,7 @@ permissions:
 jobs:
   local:
     runs-on: ubuntu-latest
-    timeout-minutes: 30
+    timeout-minutes: 45
     env:
       FOREST_MODEL: ''
       FOREST_DATABASE_URL: sqlite:////tmp/forest-ci.db
@@ -54,6 +54,19 @@ jobs:
           "$install_dir/tectonic" --version
       - run: python -m pip install -r requirements.lock.txt
       - run: python -m pip install --no-deps -e .
+      - name: Install CPU training and native AIDE test dependencies
+        shell: bash
+        run: |
+          set -euo pipefail
+          python -m pip install torch==2.14.1 --index-url https://download.pytorch.org/whl/cpu
+          python -m pip install -r requirements-study-ci.txt
+          aide_root="$RUNNER_TEMP/aide-upstream"
+          git init "$aide_root"
+          git -C "$aide_root" remote add origin https://github.com/WecoAI/aideml.git
+          git -C "$aide_root" fetch --depth=1 origin 60b3978ddf65b71f86eb7c64506965048a1398cf
+          git -C "$aide_root" checkout --detach FETCH_HEAD
+          echo "FOREST_AIDE_UPSTREAM=$aide_root" >> "$GITHUB_ENV"
+          FOREST_AIDE_UPSTREAM="$aide_root" python -c 'import os; from pathlib import Path; from research.experiments.aide_runtime import load_upstream; load_upstream(Path(os.environ["FOREST_AIDE_UPSTREAM"]))'
       - run: forest --help
       - run: python scripts/export_api_contract.py --check
       - run: python -m pytest -q

README.md

@@ -114,6 +114,16 @@ After installing the Web workspace:
 ./scripts/test.sh
 ```
 
+FOREST approaches scientific research as a **continuous, evidence-driven process rather than a sequence of isolated experiments**. Its persistent Research Graph connects hypotheses, experimental results, and research decisions, enabling agents to iteratively explore, evaluate, and refine scientific solutions. We evaluated this approach using **GPT-6 Luna (Medium)** as the research model, demonstrating autonomous experimentation and measurable scientific improvements across multiple research domains.
+
+| Research Task | Dataset | Metric | Baseline | FOREST | Improvement |
+|---|---|---|---:|---:|---:|
+| PDE Gradient Optimization | Darcy64 | Relative Gradient L2 | 0.303645 | **0.256810** | **15.42% ** |
+| Time-Series Forecasting (Study 1) | ETTh1 | MAE | 2.914060 | **2.553352** | **12.38% ** |
+| PDE Solution Optimization | Darcy64 | Relative L2 | 0.108735 | **0.099608** | **8.39% ** |
+| Time-Series Forecasting (Study 2) | ETTh1 | MAE | 2.913240 | **2.728309** | **6.35% ** |
+| Medical Image Classification | BloodMNIST | Macro-F1 | 0.965410 | **0.969351** | **+0.394 pp** |
+
 ### Research Modes
 
 - **Manual** — edit the research graph and choose tasks to execute.

requirements-study-ci.txt

@@ -0,0 +1,47 @@
+annotated-types==0.8.0
+anthropic==1.13.0
+anyio==4.15.1
+attrs==26.1.0
+backoff==2.2.1
+black==26.10.1
+click==8.5.0
+coolname==5.0.0
+dataclasses-json==0.6.7
+docstring-parser==0.18.0
+funcy==2.1
+genson==1.4.0
+h11==0.16.0
+httpcore2==2.13.1
+httpx2==2.13.1
+humanize==4.16.0
+idna==3.20
+igraph==1.0.0
+jiter==0.17.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+loguru==0.7.3
+markdown-it-py==4.2.0
+marshmallow==3.26.2
+mdurl==0.1.2
+mypy-extensions==1.1.0
+omegaconf==2.4.0
+openai==3.29.0
+packaging==26.3
+pathspec==1.1.1
+platformdirs==4.13.0
+pydantic==2.13.5
+pydantic-core==2.46.5
+pygments==2.21.0
+pytokens==0.4.1
+pyyaml==6.0.3
+referencing==0.37.0
+rich==15.0.0
+rpds-py==2026.9.1
+shutup==0.3.0
+sniffio==1.3.1
+texttable==1.7.1
+tqdm==4.70.1
+truststore==0.10.4
+typing-extensions==4.16.0
+typing-inspect==0.9.0
+typing-inspection==0.4.4

research/experiments/STUDY_CONTEXT.md

@@ -18,6 +18,38 @@ training configurations and execution receipts can share identical payloads.
 Different revisions, seeds, scientific conditions and evidence identities keep
 their own original values. Nonidentical evidence is never merged.
 
+`intern_messages` uses one request-wide pool when it reduces the complete
+serialized request. Each original JSON block retains its message position and
+role through a fragment reference; the complete definitions appear once in the
+pool owner's document. Its root is the ordered array of source fragments.
+`source_fragments` maps each root element to its original message, role, block,
+offset and JSON spelling. A definition's storage position gives it no additional
+instruction authority. Literal prefixes, suffixes, message metadata and the
+response schema retain their original positions and values.
+
+`restore_messages` reconstructs exact message strings for supported JSON
+spellings. Unrecognized spelling, including custom numeric notation or duplicate
+object keys, stays literal and unchanged. Single-block requests retain the
+self-contained v1 document format. Older v1 documents without spelling metadata
+retain their original guarantee of exact ordered JSON values. Both formats are
+readable after restart; completed native requests can replay their original
+response only when the saved source messages, task context, settings and schema
+are unchanged. Representation changes never create another inference request.
+
+### Implementation references
+
+[Pydantic's JSON Schema documentation](https://docs.pydantic.dev/latest/concepts/json_schema/)
+shows the reusable definition/reference pattern with `$defs` and `$ref`.
+FOREST applies that pattern to complete data values through its own reversible
+codec; the data document is not a standard JSON Schema.
+[LangChain's ToolMessage source](https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/messages/tool.py)
+separates model-visible `content` from a complete `artifact` and retains
+`tool_call_id` identity. FOREST's existing `ContextStore`, immutable page receipts
+and `read_context_segment` provide the corresponding persistent-evidence and
+working-context separation. The request-wide pool changes representation only;
+cross-request originals remain stored and retrievable. Neither mechanism
+summarizes or truncates scientific source. These references add no dependency.
+
 ## Persistent memory and reads
 
 `StudyMemory` stores complete versioned public JSON through the existing

research/experiments/aide_runtime.py

@@ -23,6 +23,7 @@
 import time
 from typing import Any, Mapping
 import uuid
+import warnings
 
 
 SCHEMA = """
@@ -206,7 +207,7 @@ def _review_task_description(native, stdout):
 
 def _legacy_native_prompt_equivalent(saved, current):
     """Verify an old full-context prompt matches after reversible interning."""
-    from .study_context import restore_json
+    from .study_context import restore_messages
     original = current.get("original_task_messages")
     if not isinstance(original, list):
         return False
@@ -241,36 +242,38 @@ def replace_exact(messages, needles):
     try:
         # The outer task serialization is ordinary JSON; selected user message
         # bodies may contain their own reversible tagged documents.
-        if task_context.startswith("COMPLETE LOSSLESS JSON CONTEXT DOCUMENT follows."):
-            _, outer_document_text = task_context.split("\n", 1)
-            outer_document, _ = json.JSONDecoder().raw_decode(outer_document_text)
-            delivered_task = restore_json(outer_document)
-        else:
-            delivered_task = json.loads(task_context)
-        if not isinstance(delivered_task, list) or len(delivered_task) != len(original):
+        outer = restore_messages([{"role": "user", "content": task_context}])[0]["content"]
+        delivered_task = json.loads(outer)
+        if not isinstance(delivered_task, list) or restore_messages(delivered_task) != original:
             return False
-        for before, after in zip(original, delivered_task):
-            if not isinstance(before, Mapping) or not isinstance(after, Mapping) or before.get("role") != after.get("role"):
-                return False
-            before_content, after_content = before.get("content"), after.get("content")
-            if not isinstance(before_content, str) or not isinstance(after_content, str):
-                if before_content != after_content:
-                    return False
-                continue
-            if after_content.startswith("COMPLETE LOSSLESS JSON CONTEXT DOCUMENT follows."):
-                _, encoded = after_content.split("\n", 1)
-                document, _ = json.JSONDecoder().raw_decode(encoded)
-                if restore_json(document) != json.loads(before_content):
-                    return False
-            elif after_content != before_content:
-                return False
         old_normalized, old_count = replace_exact(saved_messages, old_needles)
         new_normalized, new_count = replace_exact(current_messages, new_needles)
     except (ValueError, TypeError, RecursionError):
         return False
     return old_count > 0 and new_count > 0 and old_normalized == new_normalized
 
 
+def _native_serialization_equivalent(saved, current):
+    """Replay an unchanged saved native source after a representation upgrade."""
+    if (not isinstance(saved.get("original_messages"), list)
+            or saved["original_messages"] != current.get("original_messages")):
+        return False
+    # These fields derive from the delivered representation or the current
+    # remaining allowance. Every original source, setting and schema must match.
+    presentation = {"messages", "context_encoding", "request_payload", "source_output_allocation",
+        "effective_timeout_seconds", "max_output_tokens", "transport_trace_path"}
+    if ({key: value for key, value in saved.items() if key not in presentation}
+            != {key: value for key, value in current.items() if key not in presentation}):
+        return False
+    before, after = deepcopy(saved.get("request_payload")), deepcopy(current.get("request_payload"))
+    if not isinstance(before, dict) or not isinstance(after, dict):
+        return before == after
+    for payload in (before, after):
+        payload.pop("messages", None)
+        payload.pop("max_output_tokens", None)
+    return before == after
+
+
 def _query_router(*args, **kwargs):
     backend = _BACKEND.get()
     if backend is None:
@@ -299,7 +302,14 @@ def load_upstream(root: Path):
             raise ValueError("One process cannot mix different upstream AIDE clones")
         if str(root) not in sys.path:
             sys.path.insert(0, str(root))
-        module = importlib.import_module("aide.agent")
+        # Native AIDE imports shutup, which replaces warnings.warn globally.
+        # Keep the caller's warning policy after importing the native modules.
+        original_warn = warnings.warn
+        try:
+            with warnings.catch_warnings():
+                module = importlib.import_module("aide.agent")
+        finally:
+            warnings.warn = original_warn
         if Path(module.__file__).resolve() != required[0]:
             raise ValueError("Imported AIDE Agent does not belong to the declared upstream clone")
         # Set one fail-closed router, not a per-worker global monkeypatch.
@@ -530,7 +540,7 @@ def query(self, system_message=None, user_message=None, model=None,
                     legacy_prompt_match = (legacy_prompt_match and
                         all(saved.get(name) == request.get(name) for name in stable_fields) and
                         saved_config == request_config)
-                if previous != comparison and not legacy_prompt_match:
+                if previous != comparison and not legacy_prompt_match and not _native_serialization_equivalent(saved, request):
                     raise ValueError("A native request identity was reused with different full messages/settings")
                 if row["status"] != "completed" or not row["response_json"]:
                     raise RuntimeError("Unsettled native model request requires reconciliation; it cannot be repeated")