perf(context): batch nested discovery and guide focused reads

docs/development/README.md

@@ -237,8 +237,12 @@ relevant changes and unresolved gaps. Keep it sufficient to act correctly. Prese
 complete source material durably without equating storage completeness with a
 requirement to load every historical record into every model input.
 
-Start discovery with the existing index, heading tree, bounded search or exact
-record selector. Then read the sections or records needed for the actual decision.
+Read known paths or record IDs directly, batching independent required reads
+within the combined output budget. When a location is unknown, use the existing
+index, heading tree or bounded search to discover the relevant parent, then read
+the sections or records needed for the actual decision. Keep dependent reads and
+mutations ordered; reuse an applicable captured response instead of rerunning its
+producer, and refresh authoritative facts after relevant state changes.
 A summary must not silently replace a requirement or conceal an exception. Retain
 source identity, revision and a reachable detail route; expose omissions and unknown
 coverage. Ownership alone does not establish irrelevance: shared constraints,

docs/reference/required-work-context.md

@@ -821,9 +821,28 @@ ordinary full reads retain their file-only path.
 
 `quota context-read --list` discovers immediate child pointers, JSON types,
 serialized value sizes and object/array counts without returning their bodies.
-Combine `--list` with `--select` to explore nested sections; omit the selectors
-for the root. Its `context_capture_index_v1` view shares the digest-bound
-pagination contract, but an index cursor cannot be used for a value selection.
+When pointers are known, skip discovery and batch the needed values with repeated
+`--select` in one call. When unknown, select the nearest known parent and add
+`--list --list-depth 2` to discover nested fields together. Depth defaults to one
+and accepts one through eight; select a deeper parent for further exploration.
+This bounds traversal depth, not the capture size or readable content. Prefer a
+focused subtree to recursively listing unrelated evidence. Omit selectors for the
+root. The generated capture guidance teaches this choice without making an index
+read mandatory or changing which requirements must be read before work.
+Normal capture replies, persistent native setup and post-compaction recovery
+carry the same selection advice. Recovery points directly to `--select`, so a
+known pointer does not require another index round trip. Batch within the combined
+output budget, avoid overlapping parent/child selections, and inspect a missing
+pointer's parent before retrying the rejected batch. These are caller guidelines;
+selection validation, required reads and authority checks remain unchanged.
+
+The `context_capture_index_v1` view shares the digest-bound pagination contract.
+An expanded index includes its `list_depth`; `next_command` preserves the depth,
+selectors and both hashes. Changing depth rejects the prior view's continuation.
+An index cursor cannot be used for a value selection. For a truncated or missing
+page, recover the affected range of the same capture with a sufficient output
+budget; keep the original Turn and effect identities rather than repeating
+admission. Indexing never claims that the listed bodies have been read.
 
 Selected/index EOF, including root selection, never proves full model retention.
 Native file comparison is independent of that claim. The model can inspect the
@@ -846,8 +865,20 @@ pointer 从原始摘要绑定的 capture 批量读取详情。提供已有 `--ca
 `--selection-sha256`。读取器校验两者，保留包括大整数在内的原 JSON 值，
 不准入、不结算、不查询实时 Goal。选择模式依赖已有 typed runtime；普通全量
 读取仍只走文件路径。`--list` 列出直接子字段的精确 pointer、类型、序列化字节数
-及对象/数组成员数，不返回正文。与 `--select` 组合可探索嵌套章节；省略路径从根开始。
-`context_capture_index_v1` 沿用双摘要分页，目录 cursor 不能用作正文选择 cursor。
+及对象/数组成员数，不返回正文。已知路径时跳过目录查询，一次重复 `--select` 批量
+读取所需字段；路径未知时选择最近的已知父节点，用 `--list --list-depth 2` 一次
+发现多层字段。深度默认为 1，可选 1–8；需要更深时选择新的父节点继续。这只限制
+单次遍历深度，不限制 capture 容量或可读正文。优先展开相关子树，省略路径才从根开始。
+生成的读取指引会提示上述选择，不把查目录变成前置步骤，也不改变工作前的必读要求。
+普通 capture 回复、原生持久配置与压缩后恢复携带同一份选择指引。恢复命令直接指向
+`--select`，已知路径无须多查一轮目录。按合计输出预算分批，避免同时选择父节点和
+子节点；路径缺失时先查其父节点，再重试被拒绝的批次。这些是调用建议，不改变
+选择校验、必读要求或权限检查。
+
+`context_capture_index_v1` 沿用双摘要分页；展开的目录包含 `list_depth`，
+`next_command` 保留深度、路径和两个摘要。更改深度会拒绝旧视图的续读，目录 cursor
+也不能用作正文 cursor。遇到输出截断或缺页，用足够的输出预算恢复同一 capture
+的受影响范围，保留原 Turn/操作身份，不重新准入。列出字段不表示读过其正文。
 包括根选择在内的目录/详情 EOF 不证明全量模型保留；原生文件比较与此独立。
 模型可直接查当前完整文件，也可查基线和差分，无须先通读。
 

loopx/cli_commands/quota_capture.py

@@ -40,7 +40,8 @@ def _capture_page(path: Path, raw: bytes, digest: str, *, offset: int, size: int
                   next_page_output_bytes: int | None = None,
                   select: list[str] | None = None,
                   selection_sha256: str | None = None,
-                  list_fields: bool = False) -> dict[str, Any]:
+                  list_fields: bool = False,
+                  list_depth: int | None = None) -> dict[str, Any]:
     end = min(offset + size, len(raw))
     while end < len(raw) and raw[end] & 0xC0 == 0x80:
         end -= 1
@@ -63,7 +64,7 @@ def page(text: str) -> dict[str, Any]:
         if cursor < len(raw):
             result["next_command"] = context_capture_read_command(path, digest, offset=cursor, page_bytes=next_page_bytes or size,
                 max_page_output_bytes=next_page_output_bytes or output_limit,
-                select=select, selection_sha256=selection_sha256, list_fields=list_fields)
+                select=select, selection_sha256=selection_sha256, list_fields=list_fields, list_depth=list_depth)
         return result
 
     def fits(result: dict[str, Any]) -> bool:
@@ -138,6 +139,9 @@ def read_context_capture(args: argparse.Namespace, *, presentation: dict[str, An
                 "larger pages require a sufficient max-page-output-bytes within the capture limit")
         path, raw, payload = _read_capture(value, digest)
         list_fields = getattr(args, "list_fields", False)
+        list_depth = getattr(args, "list_depth", None)
+        if list_depth is not None and (not list_fields or type(list_depth) is not int or not 1 <= list_depth <= 8):
+            raise ValueError("list-depth requires --list and an integer from 1 to 8")
         selectors = getattr(args, "select", None)
         if list_fields and selectors is None:
             selectors = [""]
@@ -151,7 +155,8 @@ def read_context_capture(args: argparse.Namespace, *, presentation: dict[str, An
                 raise ValueError("selected continuation requires selection-sha256; follow next_command")
             from ..control_plane.quota.turn_envelope import _turn_envelope_result
 
-            plan = _turn_envelope_result("context_select", context=payload, select=selectors, list_fields=list_fields)
+            depth_options = {"list_depth": list_depth} if list_depth is not None else {}
+            plan = _turn_envelope_result("context_select", context=payload, select=selectors, list_fields=list_fields, **depth_options)
             views: list[dict[str, Any]] = []
             for item in plan["selections"]:
                 selected: Any = payload
@@ -174,7 +179,8 @@ def read_context_capture(args: argparse.Namespace, *, presentation: dict[str, An
                     views.append({"pointer": item["pointer"], "fields": fields})
                 else:
                     views.append({"pointer": item["pointer"], "value": selected})
-            raw = json.dumps({"schema_version": "context_capture_index_v1" if list_fields else "context_capture_selection_v1", "selections": views},
+            view_options = {"list_depth": list_depth} if list_depth is not None and list_depth != 1 else {}
+            raw = json.dumps({"schema_version": "context_capture_index_v1" if list_fields else "context_capture_selection_v1", "selections": views, **view_options},
                 ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8") + b"\n"
             expected = hashlib.sha256(raw).hexdigest()
             if selection_digest is not None and selection_digest != expected:
@@ -186,7 +192,8 @@ def read_context_capture(args: argparse.Namespace, *, presentation: dict[str, An
         return _capture_page(path, raw, digest, offset=offset, size=size,
             output_limit=output_limit, instruction=bool(context_dir),
             presentation=presentation, next_page_bytes=next_page_bytes,
-            next_page_output_bytes=next_page_output_bytes, select=selectors, selection_sha256=selection_digest, list_fields=list_fields)
+            next_page_output_bytes=next_page_output_bytes, select=selectors, selection_sha256=selection_digest,
+            list_fields=list_fields, list_depth=list_depth)
     except (OSError, ValueError, RecursionError, EffectRuntimeRemoteError, EffectRuntimeStartupError) as exc:
         # Recovery is presentation-only: keep failures separate from quota's
         # collection error path and retain the existing receipt/capture.
@@ -596,6 +603,11 @@ def handoff_decision(directory: Path | None, socket_path: str | None,
 def handle_native_context_command(args: argparse.Namespace, *, print_payload: Callable[..., None],
                                   registry_path: Path | None = None, runtime_root_arg: str | None = None) -> int | None:
     """Dispatch native presentation/lifecycle IO before any quota authority read."""
+    list_depth = getattr(args, "list_depth", None)
+    if list_depth is not None and (args.quota_command != "context-read" or not getattr(args, "list_fields", False)
+            or type(list_depth) is not int or not 1 <= list_depth <= 8):
+        print_payload({"ok": False, "reason": "--list-depth requires context-read --list and an integer from 1 to 8"}, args.format, str)
+        return 1
     if getattr(args, "list_fields", False) and args.quota_command != "context-read":
         print_payload({"ok": False, "reason": "--list is only supported by context-read"}, args.format, str)
         return 1

loopx/cli_commands/quota_registration.py

@@ -234,6 +234,9 @@ def register_quota_command(
     quota_parser.add_argument("--list", dest="list_fields", action="store_true",
         help="For context-read: list immediate fields at the selected pointers (root by default), "
              "with exact pointers, types and sizes. Does not read bodies or establish a delta baseline.")
+    quota_parser.add_argument("--list-depth", type=int,
+        help="For context-read --list: expand 1 to 8 levels in one index (default: 1). "
+             "Select the nearest useful parent to avoid unrelated fields; known pointers can be read directly with repeated --select.")
     quota_parser.add_argument("--selection-sha256",
         help="For selected context-read continuations: exact derived-view digest from the preceding page.")
     quota_parser.add_argument("--offset", type=int, default=0, help="For context-read: byte offset returned by the previous page.")

loopx/control_plane/quota/context_selection.ts

@@ -8,6 +8,10 @@ export function capturedContextSelection(request: JsonObject): JsonObject {
   const selectors = request.select;
   if (!Array.isArray(selectors) || !selectors.length || selectors.some(value => typeof value !== "string"))
     throw new EffectRuntimeRequestError("context selection requires a nonempty array of JSON pointers");
+  const depth = request.list_depth === undefined ? 1 : request.list_depth;
+  if (typeof depth !== "number" || !Number.isInteger(depth) || depth < 1 || depth > 8
+      || request.list_depth !== undefined && request.list_fields !== true)
+    throw new EffectRuntimeRequestError("list-depth requires --list and an integer from 1 to 8");
   const pointers = [...new Set(selectors as string[])].sort();
   const selections = pointers.map(pointer => {
     if (pointer !== "" && !pointer.startsWith("/") || /~(?:[^01]|$)/.test(pointer))
@@ -29,10 +33,17 @@ export function capturedContextSelection(request: JsonObject): JsonObject {
     // Return the verified address, not a JS-number-coerced copy of the value.
     // The file adapter retains original JSON values and owns byte serialization.
     if (request.list_fields === true) {
-      const children = value !== null && typeof value === "object" ? Object.keys(value).map(key => ({
-        pointer: pointer + "/" + key.replaceAll("~", "~0").replaceAll("/", "~1"),
-        path: [...path, Array.isArray(value) ? Number(key) : key],
-      })) : [];
+      const children: {pointer: string; path: (string | number)[]}[] = [];
+      const visit = (parent: unknown, address: string, parentPath: (string | number)[], remaining: number): void => {
+        if (remaining === 0 || parent === null || typeof parent !== "object") return;
+        for (const key of Object.keys(parent)) {
+          const child = {pointer: address + "/" + key.replaceAll("~", "~0").replaceAll("/", "~1"),
+            path: [...parentPath, Array.isArray(parent) ? Number(key) : key]};
+          children.push(child);
+          visit((parent as JsonObject)[key], child.pointer, child.path, remaining - 1);
+        }
+      };
+      visit(value, pointer, path, depth);
       return {pointer, path, children};
     }
     return {pointer, path};

loopx/control_plane/turn_driver/codex_context.py

@@ -19,6 +19,7 @@
 from .decision_capture import (
     write_decision_capture, capture_context_projection, project_captured_context, CONTEXT_CAPTURE_PAGE_GUIDANCE,
     capture_page_range, read_context_capture_file, CONTEXT_CAPTURE_NAVIGATION_GUIDANCE,
+    CONTEXT_CAPTURE_SELECTION_GUIDANCE,
     context_capture_read_command, NATIVE_CONTEXT_PAGE_BYTES, NATIVE_CONTEXT_PAGE_OUTPUT_BYTES,
     NATIVE_CONTEXT_LARGE_OUTPUT_BYTES, NATIVE_CONTEXT_HISTORY_TOKENS, NATIVE_CONTEXT_OUTPUT_ENV,
     native_context_page_budget,
@@ -217,8 +218,8 @@ def _context_continuation_instructions(command: str) -> str:
         "Compaction or a failed hook starts a new baseline epoch for the next work Turn; it does not invalidate the existing Turn or its saved recovery files. "
         "Continue from its working brief, preserving Goal, requirements/stops and original task/capture/settlement identity. "
         "Do not replay read context or rerun admission solely because the baseline was invalidated. "
-        "To find missing necessary facts, discover this same capture with: " + command + " --list. "
-        "Then use --select for the relevant sections; full-file traversal is not a prerequisite. "
+        "Read missing necessary facts from this same capture with: " + command + " --select '<pointer>'. "
+        "Replace the placeholder with a required pointer; full-file traversal is not a prerequisite. "
         "Finish an interrupted selected section only if the next decision needs it; do not claim unread content was read. "
         "Never repeat completed effects or settlement. If the original reference is lost, repair it first; "
         "independently authorized work can continue. A brief or recovery grants no new authority/freshness, "
@@ -355,6 +356,7 @@ def codex_context_hook(directory: str | None, event: dict[str, Any]) -> dict[str
         return {"hookSpecificOutput": {"hookEventName": name, "additionalContext":
             "LoopX model-context baseline invalidated. Saved files remain available. "
             + _context_continuation_instructions(recovery["command"])
+            + CONTEXT_CAPTURE_SELECTION_GUIDANCE
             + _NATIVE_CONTEXT_OUTPUT_GUIDANCE
             + "If you choose to read an entire file: " + CONTEXT_CAPTURE_PAGE_GUIDANCE}}
     return {}

loopx/control_plane/turn_driver/decision_capture.py

@@ -150,11 +150,19 @@ def validate_handoff_socket(value: str) -> Path:
     "of this same capture with fewer pages or a smaller --page-bytes; do not trust a preview or EOF alone. "
     "Do not rerun admission or ask the owner. File existence and page EOF are not retention proof."
 )
+CONTEXT_CAPTURE_SELECTION_GUIDANCE = (
+    "For known required pointers, repeat --select in one call without listing again. "
+    "For unknown pointers, use --list on the nearest known parent; add --list-depth 2 "
+    "when nested discovery helps, expanding only needed branches. "
+    "Size batches against the combined output budget; avoid selecting both a parent and its children. "
+    "A missing pointer rejects the batch: inspect its parent in this capture before retrying. "
+)
+
 # A durable capture is a source library, not an instruction to import it all.
 CONTEXT_CAPTURE_NAVIGATION_GUIDANCE = (
     "The complete decision is saved locally. Choose the sections needed for the current decision; "
     "you do not need to read the whole file or drain all pages before working. "
-    "Use index_command to discover exact pointers, then repeat --select to batch relevant fields. "
+    + CONTEXT_CAPTURE_SELECTION_GUIDANCE +
     "Read applicable goals, acceptance criteria, constraints and stop conditions before dependent work; "
     "read the current operation/settlement contract before effects, and supporting evidence when verification needs it. "
     "An index is navigation, not evidence of reading or fresh authority. "
@@ -202,7 +210,8 @@ def context_capture_read_command(path: Path, digest: str, *, offset: int = 0,
                                  max_page_output_bytes: int | None = None,
                                  select: list[str] | None = None,
                                  selection_sha256: str | None = None,
-                                 list_fields: bool = False) -> str:
+                                 list_fields: bool = False,
+                                 list_depth: int | None = None) -> str:
     from ...command_invocation import loopx_cli_argv
 
     if page_bytes is None:
@@ -213,6 +222,7 @@ def context_capture_read_command(path: Path, digest: str, *, offset: int = 0,
         "--capture-file", str(path), "--capture-sha256", digest,
         "--offset", str(offset), "--page-bytes", str(page_bytes),
         *(["--list"] if list_fields else []),
+        *(["--list-depth", str(list_depth)] if list_depth is not None else []),
         *(argument for pointer in select or [] for argument in ("--select", pointer)),
         *(["--selection-sha256", selection_sha256] if selection_sha256 is not None else []),
         *(["--max-page-output-bytes", str(max_page_output_bytes)] if max_page_output_bytes is not None else [])])

skills/loopx-project/SKILL.md

@@ -149,6 +149,15 @@ Quota, context, tools and installed skills do not grant write or spawn authority
 A raw/legacy attention queue is not owner/gate/stop authority without a
 project-asset-backed source.
 
+Use the narrowest existing read that answers the next decision. Known paths or
+record IDs go directly to batched reads; discover only missing locations. Reuse
+the same captured response while applicable, and size independent read batches
+against their combined output budget. Keep dependent operations and mutations
+ordered; refresh authority after relevant state changes. For missing paths,
+truncation or uncertain effects, follow
+[targeted diagnostics](../loopx-self-repair/references/targeted-diagnostics.md),
+not a repeated directory/status/full-packet preflight.
+
 For first-time or recovered recurring hosts, read
 [heartbeat hosts](references/heartbeat-hosts.md) and install the returned stable
 bootstrap loader. Each wake reads the full fresh result and follows its current