Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,5 @@ jobs:
run: python3 scripts/check_coverage.py
- name: Validate the minimal analysis and test pipeline
run: python3 scripts/check_minimal.py
- name: Check generation and repair task limits
run: python3 scripts/check_tasks.py
9 changes: 8 additions & 1 deletion docs/minimal-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ cargo build --workspace --locked
cargo +stable install cargo-llvm-cov --version 0.6.16 --locked
python3 scripts/check_coverage.py
python3 scripts/check_minimal.py
python3 scripts/check_tasks.py
```

No model credentials are required. The model endpoint is a temporary loopback HTTP server owned by the script; target commands receive only a dummy key. Dependency downloads may require network access.
Expand Down Expand Up @@ -47,7 +48,7 @@ The preprocessor parses `cfg` predicates conservatively, preserves `cfg(not(test

Recursive context lookup now resolves full function names through the compiler's encoded-name index and consumes the accumulated dependencies. External functions without local artifacts remain outside this source context.

The local macOS/Apple Silicon run passed using six generation responses and one repair response. Real-model behavior, bytes-scale execution, general multi-crate support, request concurrency limits, hard process timeouts, recovery after forced termination, and explicit run/resume behavior remain later work. Git history is retained.
The local macOS/Apple Silicon run passed using six generation responses and one repair response. Real-model behavior, bytes-scale execution, general multi-crate support, hard process timeouts, recovery after forced termination, and explicit run/resume behavior remain later work. Git history is retained.

The coverage entry temporarily annotates sources with `#[coverage(off)]` and enables the nightly feature at crate roots without adding newlines. It preserves compiler flags, restores original bytes on returned errors, and performs no hashing or source fingerprint checks. `cargo llvm-cov report --json` reuses the first run's data; the first XML-producing command retains the tool's normal profile cleanup between candidates. This does not provide crash recovery or general macro expansion. See the [coverage command reference](../utgen/README.md#coverage-of-existing-tests) for scope.

Expand All @@ -56,3 +57,9 @@ The coverage entry temporarily annotates sources with `#[coverage(off)]` and ena
`scripts/check_coverage.py` exercises 11 small standalone crates, without a model service or extra crate dependencies. It covers existing coverage attributes, `cfg_attr(test, ...)`, compound/nested conditions with Cargo features enabled and disabled, test-only methods and impls, file-level test modules, and an expression used by `include!`. Each case has an integration test so both normal-library and test-harness builds are checked. It compares execution outcomes, checks raw production-only coverage, verifies source restoration, and confirms preprocessing leaves no original tests while preserving byte offsets. A custom target entry containing an expression is rejected by both commands.

The script runs in Linux CI. Coverage and preprocessing share only source parsing and test predicates; they keep separate edit visitors. Coverage attributes and their feature gate are added under the complement of existing conditions, without a cfg evaluator or compiler-flag overrides. Explicit `coverage(on)` remains an intentional opt-in. Expression fragments are preserved, without macro expansion or automatic test exclusion inside those fragments.

## Task Scheduling Regression

`scripts/check_tasks.py` uses nine focal functions sharing one source file, a loopback model, and real Cargo commands. It measures generation and repair request peaks for N=1, N=2, and the default N=4, requires overlap for N>1, and checks that compiler invocations never overlap or observe changing source bytes. A paused compiler lets the N=1 generation queue fill: one candidate is being validated, one result is buffered, and one active job waits to send. Releasing compilation must drain every result.

The same check covers direct and input/prefix/oracle integration generation, removal of invalid imports from saved candidates, and preservation/removal of temporary compiler inputs. Integration coverage reports remain available until every function has been read, and all nine functions must have passing tests and nonzero covered lines. Missing usage in one fixture response exercises worker-panic propagation and slot release; a stale generation backup exercises queue draining after consumer failure, and blocking the diagnostic output file exercises failure while the repair lock is held. Sources must be restored, failed repair backups retained, unrelated backups untouched, existing recovery material rejected, and post-failure statistics skipped. These checks use byte comparisons without hashing and run on macOS/Linux. CLI tests separately verify the default and rejection of zero before configuration loading.
6 changes: 5 additions & 1 deletion docs/palm-rust-unit-test-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,14 +273,18 @@ utgen fix --project-dir <PROJECT_DIR> [--work-dir <WORK_DIR>...] [--tasks <N>]
| --- | --- |
| `-p, --project-dir` | 项目根目录。路径会相对于当前目录解析并 canonicalize。 |
| `-w, --work-dir` | 工作目录,可重复传入或用逗号分隔,相对路径基于当前命令执行目录,默认等于 project dir。 |
| `-t, --tasks` | 默认 `128`。生成阶段用于结果通道容量,未严格限制 LLM 请求并发;修复阶段虽接受该参数,但未将其传给调度器。 |
| `-t, --tasks` | 默认 `4`,必须大于 `0`。生成和修复均限制同时活跃的被测函数任务数;同一函数内的条件链、生成阶段或修复轮次依次执行。 |
| `-i, --integration` | 生成集成测试,放到 `tests/` 目录,按分析结果的 `visible` 标志筛选函数并进行编译检查。 |
| `-r, --requirement` | 在提示词中加入条件链约束。 |
| `-c, --context` | 在提示词中加入 `focxt/<encoded>.rs` 上下文。 |
| `-o, --oracle` | 开启三段式生成:输入范围、测试前缀、oracle 分别生成。该模式是可选实验模式,不是按路径任务分解本身。 |

`--requirement`、`--context` 和 `--oracle` 默认均关闭。即使不传 `--context`,当前生成器仍读取 focxt 索引和对应的上下文文件;该参数只控制是否把上下文加入提示词。

生成任务取得名额后才开始工作,将结果送入容量为 N 的队列后释放名额;串行消费者负责导入检查和候选编译。队列中的结果及正在验证的候选不占生成名额。修复任务则在该函数的修复结果保存后释放名额。源码插入、target 清理、编译或测试、诊断读取及源码恢复串行执行,模型请求可以重叠;该参数不改变 Cargo 内部的依赖编译并行度。同一工作副本不能供多个 PALM 进程同时使用,不同实验应使用独立副本和 target 目录。

任务异常会在等待所有任务结束后报告,并跳过后续覆盖率统计。修复成功时只删除本次创建的备份;任务失败时恢复源码并保留这些备份,已有备份不会被覆盖。候选本身编译不通过仍按测试结果处理。当前没有增加请求超时、重试策略、跨进程锁或强制终止后的恢复机制。

### 8.2 LLM 配置

`utgen/src/gene/llm.rs` 使用 `async-openai` 调用兼容 OpenAI Chat Completions 的接口。模型配置由 `utgen/src/config.rs` 在运行时加载;可以复制 `utgen/res/api.example.json` 并填写为:
Expand Down
Loading
Loading