feat: scope model trials by function and shared request limit

.gitignore

@@ -1,4 +1,5 @@
 .vscode/
 target/
 *.txt
+!/examples/bytes-smoke-*.txt
 api.json

docs/minimal-pipeline.md

@@ -68,3 +68,9 @@ The same check covers direct and input/prefix/oracle integration generation, rem
 ## Model Request Regression
 
 Local HTTP tests exercise the shared generation/repair request path, including response usage omissions, empty choices or content, invalid JSON, permanent 4xx errors, transient 429/5xx errors, dropped connections, and delayed response bodies. They verify actual attempt counts and bounded retry exhaustion without calling a real service. Existing task checks also verify request summaries and source/backup handling after a final request failure. `--request-timeout` defaults to 180 seconds per attempt and rejects zero before configuration loading. Request formatting and compilation repair retries remain business-level operations, not additional transport retry layers.
+
+## Function Selection and Request Budgets
+
+`scripts/check_tasks.py` generates and repairs just two selected functions while other saved candidates remain in the same directories. It checks actual request counts, selected-only statistics, untouched unselected candidate files, and invocation records. Duplicate names and blank lines are accepted. Separate CLI tests reject empty/unknown lists and a zero request limit before configuration or target changes.
+
+With four function workers and a two-attempt limit, both generation and repair must send exactly two requests, return failure when more work needs a request, restore sources, and skip statistics. Local HTTP unit tests additionally cover retries consuming the limit, concurrent clones sharing it, and success on the final allowed attempt. None of these checks uses real model credentials. The bytes analysis check validates the checked-in two- and eight-function lists against the complete analysis index for a later model trial.

docs/palm-rust-unit-test-generation.md

@@ -274,6 +274,8 @@ utgen fix --project-dir <PROJECT_DIR> [--work-dir <WORK_DIR>...] [--tasks <N>]
 | `-p, --project-dir` | 项目根目录。路径会相对于当前目录解析并 canonicalize。 |
 | `-w, --work-dir` | 工作目录，可重复传入或用逗号分隔，相对路径基于当前命令执行目录，默认等于 project dir。 |
 | `-t, --tasks` | 默认 `4`，必须大于 `0`。生成和修复均限制同时活跃的被测函数任务数；同一函数内的条件链、生成阶段或修复轮次依次执行。 |
+| `--functions-file` | 可选 UTF-8 函数清单，每行一个 `brinfo/name_map.json` 中的完整名称。生成、修复及其统计均按清单筛选；空行忽略、重复项去重，不支持通配符。 |
+| `--max-requests` | 可选的整次命令请求尝试次数上限，必须大于 `0`；默认不设总上限。所有并发任务和重试共享该额度。 |
 | `-i, --integration` | 生成集成测试，放到 `tests/` 目录，按分析结果的 `visible` 标志筛选函数并进行编译检查。 |
 | `-r, --requirement` | 在提示词中加入条件链约束。 |
 | `-c, --context` | 在提示词中加入 `focxt/<encoded>.rs` 上下文。 |
@@ -285,9 +287,11 @@ utgen fix --project-dir <PROJECT_DIR> [--work-dir <WORK_DIR>...] [--tasks <N>]
 
 任务异常会在等待所有任务结束后报告，并跳过后续覆盖率统计。修复成功时只删除本次创建的备份；任务失败时恢复源码并保留这些备份，已有备份不会被覆盖。候选本身编译不通过仍按测试结果处理。模型请求现在统一执行：`--request-timeout` 默认每次尝试 180 秒，包含响应体读取；网络错误、429（余额不足除外）和 5xx 最多尝试 3 次，其他 HTTP 错误或空回答直接报错。格式纠正和编译修复轮次单独计算，不再叠加网络重试。缺少 `usage` 的有效回答可以继续，但请求统计标记为不完整；生成和修复分别写入 `utgen/generation/gen-requests.json`、`fix-requests.json`。该期限不涵盖 Cargo 子进程，也不提供跨进程锁或强制终止恢复。
 
+小规模试跑可在 `gen`、`fix` 中分别传入同一份 `--functions-file`，不会自动沿用上次清单。空清单和未知名称在修改目标前报错；修复时缺少所选函数的候选结果也会报错。`--max-requests` 包含所有并发任务、生成阶段、格式纠正、修复轮次及网络重试的请求尝试，计数与额度检查使用同一个共享计数器。额度用完且仍需请求时，命令失败，等待已有请求收尾并恢复源码；恰好用完额度且工作全部完成则成功。请求报告记录模型、选项、去重后的函数清单、候选处理状态和用量；候选状态不等于测试通过或覆盖率统计完成。未选函数的已有文件保留，跨实验应使用新副本。准备好的 [2 函数和 8 函数清单及操作说明](../examples/README.md#prepare-a-small-model-trial)用于后续真实模型验收，目前不代表已取得真实模型结果。
+
 ### 8.2 LLM 配置
 
-`utgen/src/gene/llm.rs` 使用 `async-openai` 调用兼容 OpenAI Chat Completions 的接口。模型配置由 `utgen/src/config.rs` 在运行时加载；可以复制 `utgen/res/api.example.json` 并填写为：
+`utgen/src/gene/llm.rs` 保留 `async-openai` 的请求／响应类型，通过 `reqwest` 调用兼容 OpenAI Chat Completions 的接口，统一管理重试。模型配置由 `utgen/src/config.rs` 在运行时加载；可以复制 `utgen/res/api.example.json` 并填写为：
 
 ```text
 utgen/res/api.json

examples/README.md

@@ -44,6 +44,23 @@ utgen fix -p "$palm_example_dir"
 
 `--requirement` enables generation for representative condition chains; `--context` includes focal context in the prompt. Both flags are off by default. In unit mode, candidates are stored in JSON and temporarily inserted into the target source for checks and execution.
 
+### Prepare a Small Model Trial
+
+The checked-in [two-function list](bytes-smoke-2.txt) selects `Bytes::len` and `BytesMut::len`. The [eight-function list](bytes-smoke-8.txt) adds `is_empty`, `split_off`, and `truncate` for both types. They cover 2 and 16 representative condition chains respectively. The repeated namespace segments are exact compiler-index names. The analysis regression checks both lists without calling a model.
+
+For a future trial, agree on the model configuration and separate generation/repair attempt limits first. Set `palm_gen_limit` and `palm_fix_limit` to those positive integers, then use the prepared working copy:
+
+```sh
+utgen gen -p "$palm_example_dir" --requirement --context \
+  --functions-file "$palm_repo/examples/bytes-smoke-2.txt" \
+  --max-requests "$palm_gen_limit"
+utgen fix -p "$palm_example_dir" \
+  --functions-file "$palm_repo/examples/bytes-smoke-2.txt" \
+  --max-requests "$palm_fix_limit"
+```
+
+Both commands default to four active function tasks and a 180-second deadline per request attempt. Every retry consumes the command's attempt limit. Review the request reports, candidate compilation/pass counts, coverage, and restored sources before expanding to eight functions on another fresh copy. Keep configuration and generation mode fixed when comparing these runs. No real-model result is claimed by the deterministic checks; the lists and limits prepare that later validation.
+
 ### Integration Test Mode
 
 Use a fresh working copy and repeat extraction and preprocessing before choosing this mode, since existing generation results are reused independently of these flags:
@@ -64,6 +81,7 @@ Paths below are relative to the working copy:
 | `utgen/generation/prompt/` and `answer/` | Generation prompts and model responses. |
 | `utgen/generation/pre_fix/` | Candidate tests and compilation results before repair. |
 | `utgen/generation/llm_fix/` | Candidate tests and compilation results after repair. |
+| `utgen/generation/gen-requests.json` and `fix-requests.json` | Model, invocation selection/options, attempts, limits, token reporting, and candidate-stage status. |
 | `utgen/result/` and `utgen/fixed_result/` | Per-function coverage and execution statistics. |
 
 `coverage.xml` and `coverage.json` are intermediate files and may be removed after parsing. The current implementation writes JSON statistics; it does not generate the `result.html` report mentioned in earlier instructions. See [utgen](../utgen/README.md) for CLI limitations.

examples/bytes-smoke-2.txt

@@ -0,0 +1,2 @@
+bytes::bytes::bytes::Bytes::len
+bytes::bytes_mut::bytes_mut::BytesMut::len

examples/bytes-smoke-8.txt

@@ -0,0 +1,8 @@
+bytes::bytes::bytes::Bytes::len
+bytes::bytes::bytes::Bytes::is_empty
+bytes::bytes::bytes::Bytes::split_off
+bytes::bytes::bytes::Bytes::truncate
+bytes::bytes_mut::bytes_mut::BytesMut::len
+bytes::bytes_mut::bytes_mut::BytesMut::is_empty
+bytes::bytes_mut::bytes_mut::BytesMut::split_off
+bytes::bytes_mut::bytes_mut::BytesMut::truncate

scripts/check_bytes.py

@@ -119,6 +119,14 @@ def run(root, label, command):
         elapsed, output = run(root, 'analyze', ['utgen', 'analyze', '-p', str(root)])
         assert snapshot(root) == prepared, 'analysis modified prepared sources'
         names, infos = check_artifacts(root, 663)
+        # Keep the future model-smoke lists aligned with real compiler identities.
+        for count, chains in [(2, 2), (8, 16)]:
+            selected = (REPO / f'examples/bytes-smoke-{count}.txt').read_text().splitlines()
+            assert len(selected) == len(set(selected)) == count
+            assert set(selected) <= names.keys()
+            actual_chains = sum(json.loads((root / f'brinfo/brdata/{names[name]}.json').read_text())
+                                ['size']['min_set'] for name in selected)
+            assert actual_chains == chains, (count, actual_chains)
         # Verify representative default, generic-impl, and macro-generated methods.
         for name, info in infos.items():
             context = (root / f"focxt/{info['encoded_name']}.rs").read_text()

scripts/check_tasks.py

@@ -221,6 +221,9 @@ def check_queue():
             assert report['attempts'] == state['total'], report
             assert report['responses_without_usage'] == int(missing_usage), report
             assert report['usage_complete'] == (not missing_usage and not state['fault_sent']), report
+            assert report['model'] == 'fixture', report
+            assert report['invocation']['command'] == command[1], report
+            assert report['invocation']['candidate_status'] == ('failed' if fail else 'completed'), report
         return state.copy()
 
     def reset(seed=None):
@@ -232,9 +235,11 @@ def reset(seed=None):
     def command(mode, n, *extra):
         return ['utgen', mode, '-p', str(root), *([] if n == 4 else ['--tasks', str(n)]), *extra]
 
-    def results(directory, compiled):
+    def results(directory, compiled, names=None):
         data = [json.loads(p.read_text()) for p in (root / 'utgen' / directory).glob('*.json')]
-        assert len(data) == count, (directory, len(data))
+        assert len(data) == (len(names) if names is not None else count), (directory, len(data))
+        if names is not None:
+            assert {item['function_name'] for item in data} == set(names), data
         assert all(item['tests_compiled'] == compiled for item in data), data
         if compiled:
             assert all(item['tests_run'] == 1 and item['tests_passed'] == 1
@@ -261,6 +266,80 @@ def results(directory, compiled):
             assert measured['total'] == count and measured['peak'] == n, measured
             results('fixed_result', 1)
             assert not (root / 'src/lib.rs.bak').exists()
+
+        # Existing results for other functions must not enter this invocation.
+        nmap = json.loads((root / 'brinfo/name_map.json').read_text())
+        selected = sorted(nmap)[:2]
+        functions_file = work / 'functions.txt'
+        functions_file.write_text('\n' + '\n'.join(selected + [selected[0]]) + '\n')
+        selection_args = ['--functions-file', str(functions_file), '--max-requests', '2']
+        reset(seed)
+        pre_dir = root / 'utgen/generation/pre_fix'
+        for name in selected:
+            (pre_dir / (nmap[name] + '.json')).unlink()
+        untouched = {p: p.read_bytes() for p in pre_dir.glob('*.json')}
+        measured = run('gen-selected', command('gen', 2, *selection_args), limit=2)
+        assert measured['total'] == 2
+        results('result', 0, selected)
+        for path, content in untouched.items():
+            assert path.read_bytes() == content, path
+        report = json.loads((root / 'utgen/generation/gen-requests.json').read_text())
+        assert report['invocation']['functions'] == selected, report
+        assert report['max_requests'] == 2 and not report['budget_exhausted'], report
+
+        missing = pre_dir / (nmap[selected[0]] + '.json')
+        candidate = missing.read_bytes()
+        missing.unlink()
+        measured = run('fix-missing-selected', command('fix', 2, *selection_args), fail=True)
+        assert measured['total'] == 0
+        assert not (root / 'utgen/generation/llm_fix').exists()
+        assert not (root / 'src/lib.rs.bak').exists()
+        missing.write_bytes(candidate)
+
+        fixed_dir = root / 'utgen/generation/llm_fix'
+        fixed_dir.mkdir()
+        other_name = sorted(nmap)[2]
+        other = fixed_dir / (nmap[other_name] + '.json')
+        shutil.copyfile(pre_dir / other.name, other)
+        old_other = other.read_bytes()
+        before_fix = {p: p.read_bytes() for p in pre_dir.glob('*.json')}
+        measured = run('fix-selected', command('fix', 2, *selection_args), limit=2)
+        assert measured['total'] == 2
+        results('fixed_result', 1, selected)
+        assert len(list(fixed_dir.glob('*.json'))) == 3
+        assert other.read_bytes() == old_other
+        for path, content in before_fix.items():
+            assert path.read_bytes() == content, path
+        report = json.loads((root / 'utgen/generation/fix-requests.json').read_text())
+        assert report['invocation']['functions'] == selected, report
+        assert report['max_requests'] == 2 and not report['budget_exhausted'], report
+        measured = run('fix-selected-cached', command('fix', 2, *selection_args))
+        assert measured['total'] == 0
+        report = json.loads((root / 'utgen/generation/fix-requests.json').read_text())
+        assert report['attempts'] == 0 and report['invocation']['functions'] == selected, report
+        assert other.read_bytes() == old_other
+
+        reset(seed)
+        for name in selected:
+            (pre_dir / (nmap[name] + '.json')).unlink()
+        measured = run('integration-selected', command('gen', 2, '--integration', *selection_args),
+                       limit=2, integration=True)
+        assert measured['total'] == 2
+        results('result', 1, selected)
+
+        # More workers than requests: every clone shares the same hard cap.
+        for mode in ['gen', 'fix']:
+            reset(seed if mode == 'fix' else None)
+            measured = run(f'{mode}-budget', command(mode, 4, '--max-requests', '2'), limit=2, fail=True)
+            assert measured['total'] == 2, measured
+            report = json.loads((root / f'utgen/generation/{mode}-requests.json').read_text())
+            assert report['attempts'] == 2 and report['budget_exhausted'], report
+            assert not (root / 'utgen' / ('result' if mode == 'gen' else 'fixed_result')).exists()
+            if mode == 'fix':
+                backup = root / 'src/lib.rs.bak'
+                assert backup.read_text() == original
+                backup.unlink()
+
         for oracle in [False, True]:
             reset()
             preserved = {}