diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 452fc50908..6e019d3e65 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -42,3 +42,5 @@ jobs: run: python3 scripts/check_coverage.py - name: Validate the minimal analysis and test pipeline run: python3 scripts/check_minimal.py + - name: Check generation and repair task limits + run: python3 scripts/check_tasks.py diff --git a/docs/minimal-pipeline.md b/docs/minimal-pipeline.md index b1ff37e9cf..a4fc4191f4 100644 --- a/docs/minimal-pipeline.md +++ b/docs/minimal-pipeline.md @@ -11,6 +11,7 @@ cargo build --workspace --locked cargo +stable install cargo-llvm-cov --version 0.6.16 --locked python3 scripts/check_coverage.py python3 scripts/check_minimal.py +python3 scripts/check_tasks.py ``` No model credentials are required. The model endpoint is a temporary loopback HTTP server owned by the script; target commands receive only a dummy key. Dependency downloads may require network access. @@ -47,7 +48,7 @@ The preprocessor parses `cfg` predicates conservatively, preserves `cfg(not(test Recursive context lookup now resolves full function names through the compiler's encoded-name index and consumes the accumulated dependencies. External functions without local artifacts remain outside this source context. -The local macOS/Apple Silicon run passed using six generation responses and one repair response. Real-model behavior, bytes-scale execution, general multi-crate support, request concurrency limits, hard process timeouts, recovery after forced termination, and explicit run/resume behavior remain later work. Git history is retained. +The local macOS/Apple Silicon run passed using six generation responses and one repair response. Real-model behavior, bytes-scale execution, general multi-crate support, hard process timeouts, recovery after forced termination, and explicit run/resume behavior remain later work. Git history is retained. The coverage entry temporarily annotates sources with `#[coverage(off)]` and enables the nightly feature at crate roots without adding newlines. It preserves compiler flags, restores original bytes on returned errors, and performs no hashing or source fingerprint checks. `cargo llvm-cov report --json` reuses the first run's data; the first XML-producing command retains the tool's normal profile cleanup between candidates. This does not provide crash recovery or general macro expansion. See the [coverage command reference](../utgen/README.md#coverage-of-existing-tests) for scope. @@ -56,3 +57,9 @@ The coverage entry temporarily annotates sources with `#[coverage(off)]` and ena `scripts/check_coverage.py` exercises 11 small standalone crates, without a model service or extra crate dependencies. It covers existing coverage attributes, `cfg_attr(test, ...)`, compound/nested conditions with Cargo features enabled and disabled, test-only methods and impls, file-level test modules, and an expression used by `include!`. Each case has an integration test so both normal-library and test-harness builds are checked. It compares execution outcomes, checks raw production-only coverage, verifies source restoration, and confirms preprocessing leaves no original tests while preserving byte offsets. A custom target entry containing an expression is rejected by both commands. The script runs in Linux CI. Coverage and preprocessing share only source parsing and test predicates; they keep separate edit visitors. Coverage attributes and their feature gate are added under the complement of existing conditions, without a cfg evaluator or compiler-flag overrides. Explicit `coverage(on)` remains an intentional opt-in. Expression fragments are preserved, without macro expansion or automatic test exclusion inside those fragments. + +## Task Scheduling Regression + +`scripts/check_tasks.py` uses nine focal functions sharing one source file, a loopback model, and real Cargo commands. It measures generation and repair request peaks for N=1, N=2, and the default N=4, requires overlap for N>1, and checks that compiler invocations never overlap or observe changing source bytes. A paused compiler lets the N=1 generation queue fill: one candidate is being validated, one result is buffered, and one active job waits to send. Releasing compilation must drain every result. + +The same check covers direct and input/prefix/oracle integration generation, removal of invalid imports from saved candidates, and preservation/removal of temporary compiler inputs. Integration coverage reports remain available until every function has been read, and all nine functions must have passing tests and nonzero covered lines. Missing usage in one fixture response exercises worker-panic propagation and slot release; a stale generation backup exercises queue draining after consumer failure, and blocking the diagnostic output file exercises failure while the repair lock is held. Sources must be restored, failed repair backups retained, unrelated backups untouched, existing recovery material rejected, and post-failure statistics skipped. These checks use byte comparisons without hashing and run on macOS/Linux. CLI tests separately verify the default and rejection of zero before configuration loading. diff --git a/docs/palm-rust-unit-test-generation.md b/docs/palm-rust-unit-test-generation.md index 0a3145f40b..d7514b2f5c 100644 --- a/docs/palm-rust-unit-test-generation.md +++ b/docs/palm-rust-unit-test-generation.md @@ -273,7 +273,7 @@ utgen fix --project-dir [--work-dir ...] [--tasks ] | --- | --- | | `-p, --project-dir` | 项目根目录。路径会相对于当前目录解析并 canonicalize。 | | `-w, --work-dir` | 工作目录,可重复传入或用逗号分隔,相对路径基于当前命令执行目录,默认等于 project dir。 | -| `-t, --tasks` | 默认 `128`。生成阶段用于结果通道容量,未严格限制 LLM 请求并发;修复阶段虽接受该参数,但未将其传给调度器。 | +| `-t, --tasks` | 默认 `4`,必须大于 `0`。生成和修复均限制同时活跃的被测函数任务数;同一函数内的条件链、生成阶段或修复轮次依次执行。 | | `-i, --integration` | 生成集成测试,放到 `tests/` 目录,按分析结果的 `visible` 标志筛选函数并进行编译检查。 | | `-r, --requirement` | 在提示词中加入条件链约束。 | | `-c, --context` | 在提示词中加入 `focxt/.rs` 上下文。 | @@ -281,6 +281,10 @@ utgen fix --project-dir [--work-dir ...] [--tasks ] `--requirement`、`--context` 和 `--oracle` 默认均关闭。即使不传 `--context`,当前生成器仍读取 focxt 索引和对应的上下文文件;该参数只控制是否把上下文加入提示词。 +生成任务取得名额后才开始工作,将结果送入容量为 N 的队列后释放名额;串行消费者负责导入检查和候选编译。队列中的结果及正在验证的候选不占生成名额。修复任务则在该函数的修复结果保存后释放名额。源码插入、target 清理、编译或测试、诊断读取及源码恢复串行执行,模型请求可以重叠;该参数不改变 Cargo 内部的依赖编译并行度。同一工作副本不能供多个 PALM 进程同时使用,不同实验应使用独立副本和 target 目录。 + +任务异常会在等待所有任务结束后报告,并跳过后续覆盖率统计。修复成功时只删除本次创建的备份;任务失败时恢复源码并保留这些备份,已有备份不会被覆盖。候选本身编译不通过仍按测试结果处理。当前没有增加请求超时、重试策略、跨进程锁或强制终止后的恢复机制。 + ### 8.2 LLM 配置 `utgen/src/gene/llm.rs` 使用 `async-openai` 调用兼容 OpenAI Chat Completions 的接口。模型配置由 `utgen/src/config.rs` 在运行时加载;可以复制 `utgen/res/api.example.json` 并填写为: diff --git a/scripts/check_tasks.py b/scripts/check_tasks.py new file mode 100644 index 0000000000..987f2e2e10 --- /dev/null +++ b/scripts/check_tasks.py @@ -0,0 +1,312 @@ +#!/usr/bin/env python3 +"""Check real gen/fix scheduling with a local model and real Cargo (macOS/Linux).""" +import argparse +import fcntl +import http.server +import json +import os +from pathlib import Path +import re +import shutil +import signal +import subprocess +import sys +import tempfile +import threading +import time + + +def sources(root): + return {str(p.relative_to(root)): p.read_bytes() + for folder in ['src', 'tests'] for p in (root / folder).rglob('*.rs')} + + +def cargo_wrapper(): + """Observe compilation without replacing Cargo or its diagnostics.""" + args = sys.argv[1:] + env = os.environ.copy() + command = [env['PALM_REAL_CARGO'], *args] + monitor = Path(env['PALM_TASKS_MONITOR']) + if not args or args[0] not in ['build', 'test'] or env.get('PALM_MONITOR_CHILD'): + os.execv(command[0], command) + env['PALM_MONITOR_CHILD'] = '1' + before = sources(Path.cwd()) + with (monitor / 'compiler.json').open('r+') as file: + fcntl.flock(file, fcntl.LOCK_EX) + state = json.load(file) + state['active'] += 1 + state['total'] += 1 + state['peak'] = max(state['peak'], state['active']) + first = state['total'] == 1 + file.seek(0) + json.dump(state, file) + file.truncate() + if first and env.get('PALM_HOLD_FIRST_BUILD'): + (monitor / 'build-started').touch() + deadline = time.monotonic() + 20 + while not (monitor / 'release-build').exists(): + if time.monotonic() > deadline: + raise RuntimeError('backpressure fixture did not release compilation') + time.sleep(0.02) + result = subprocess.run(command, env=env) + with (monitor / 'compiler.json').open('r+') as file: + fcntl.flock(file, fcntl.LOCK_EX) + state = json.load(file) + state['active'] -= 1 + if sources(Path.cwd()) != before: + state['source_changes'] += 1 + file.seek(0) + json.dump(state, file) + file.truncate() + sys.exit(result.returncode) + + +def main(): + repo = Path(__file__).resolve().parents[1] + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--bin-dir', type=Path, default=repo / 'target/debug') + args = parser.parse_args() + work = Path(tempfile.mkdtemp(prefix='palm-tasks-')).resolve() + root = work / 'crate' + (root / 'src').mkdir(parents=True) + for name in ['Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml']: + shutil.copyfile(repo / 'examples/minimal' / name, root / name) + count = 9 # More than 2N even for the default N=4. + original = ''.join(f'pub fn f{i}(x: i32) -> i32 {{ x + 1 }}\n' for i in range(count)) + (root / 'src/lib.rs').write_text(original) + (root / 'src/unrelated.rs').write_text('// unrelated source file\n') + unrelated = root / 'src/unrelated.rs.bak' + unrelated.write_bytes(b'previous recovery material\n') + wrapper = work / 'bin' + wrapper.mkdir() + shutil.copyfile(__file__, wrapper / 'cargo') + (wrapper / 'cargo').chmod(0o755) + env = os.environ.copy() + env.pop('CARGO_TARGET_DIR', None) + for name in ['PALM_CONFIG', 'PALM_API_BASE', 'PALM_API_KEY', 'PALM_MODEL']: + env.pop(name, None) + env['PALM_REAL_CARGO'] = shutil.which('cargo') + env['PATH'] = os.pathsep.join([str(wrapper), str(args.bin_dir.resolve()), env['PATH']]) + for name in ['NO_PROXY', 'no_proxy']: + env[name] = ','.join(filter(None, [env.get(name, ''), '127.0.0.1', 'localhost'])) + print(f'Validation artifacts: {work}', flush=True) + lock = threading.Lock() + ready = threading.Event() + state = {} + summaries = {} + + class Model(http.server.BaseHTTPRequestHandler): + def log_message(self, *_): + pass + + def do_POST(self): + try: + request = json.loads(self.rfile.read(int(self.headers['Content-Length']))) + system, user = request['messages'][0]['content'], request['messages'][-1]['content'] + with lock: + state['active'] += 1 + state['total'] += 1 + serial = state['total'] + state['peak'] = max(state['peak'], state['active']) + if state['active'] >= state['limit']: + ready.set() + # Hold the first batch to measure overlap, including slow repair builds. + assert ready.wait(20), 'model workers did not reach the requested concurrency' + time.sleep(0.05) + if 'The function to be tested is presented as follows:' in user: + focal = user.split('The function to be tested is presented as follows:', 1)[1] + function = re.search(r'\bfn (f\d+)\(', focal).group(1) + if 'infer the test input ranges' in system: + answer = 'x = 1' + elif 'generate accurate test oracles' in system: + answer = 'assert_eq!(actual, 2);' + else: + argument = '1' if state['integration'] else '"bad"' + body = ('let actual = ' + function + '(1);' if 'Omit test oracles' in system + else f'assert_eq!({function}({argument}), 2);') + imports = ('use palm_fixture::*;\nuse std::cmp::max;\nuse absent_crate::Missing;\n' + if state['integration'] else '') + answer = imports + '#[test]\nfn generated() {\n ' + body + '\n}\n' + else: + file = re.search(r'You can only modify lines \d+ to \d+ in file (.+?)\. For your answer', user).group(1) + match = re.search(r'^\[(\d+)\](.*assert_eq!\(f\d+\("bad"\), 2\);.*)$', user, re.M) + assert match, 'missing numbered repair statement' + number, old = match.groups() + new = old.replace('"bad"', '1') + answer = f'ChangeLog:1@{file}\nFixDescription: Use an integer.\nOriginalCode@{number}-{number}:\n[{number}]{old}\nFixedCode@{number}-{number}:\n[{number}]{new}\n' + response = {'id': 'fixture', 'object': 'chat.completion', 'created': 0, 'model': 'fixture', + 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': answer}, 'finish_reason': 'stop'}]} + # Exercise a real worker panic in the existing response parser. + if not (state['panic'] and serial == 1): + response['usage'] = {'prompt_tokens': 1, 'completion_tokens': 1, 'total_tokens': 2} + body = json.dumps(response).encode() + # Count requests awaiting a response, excluding HTTP-handler teardown. + with lock: + state['active'] -= 1 + self.send_response(200) + self.send_header('Content-Type', 'application/json') + self.send_header('Content-Length', str(len(body))) + self.end_headers() + self.wfile.write(body) + except Exception as error: + with lock: + state['errors'].append(str(error)) + self.send_error(400, str(error)) + + server = http.server.ThreadingHTTPServer(('127.0.0.1', 0), Model) + threading.Thread(target=server.serve_forever, daemon=True).start() + env.update(PALM_API_BASE=f'http://127.0.0.1:{server.server_port}/v1', PALM_API_KEY='fixture', PALM_MODEL='fixture') + + def run(label, command, limit=1, integration=False, panic=False, fail=False, backpressure=False): + monitor = work / label + monitor.mkdir() + (monitor / 'compiler.json').write_text(json.dumps(dict(active=0, peak=0, total=0, source_changes=0))) + env['PALM_TASKS_MONITOR'] = str(monitor) + env.pop('PALM_HOLD_FIRST_BUILD', None) + with lock: + state.clear() + state.update(active=0, peak=0, total=0, errors=[], limit=limit, integration=integration, panic=panic) + ready.clear() + watcher = None + if backpressure: + env['PALM_HOLD_FIRST_BUILD'] = '1' + + def check_queue(): + try: + deadline = time.monotonic() + 20 + while not (monitor / 'build-started').exists() or state['total'] < 3: + assert time.monotonic() < deadline, 'queue did not fill' + time.sleep(0.02) + time.sleep(0.3) + # One validating, one buffered, one blocked at send; no fourth job. + assert state['total'] == 3, state.copy() + except Exception as error: + state['errors'].append(str(error)) + finally: + (monitor / 'release-build').touch() + watcher = threading.Thread(target=check_queue) + watcher.start() + print(label, flush=True) + log = monitor / 'command.log' + with log.open('w') as output: + process = subprocess.Popen(command, cwd=root, env=env, stdout=output, stderr=subprocess.STDOUT, + start_new_session=True) + try: + status = process.wait(timeout=240) + except subprocess.TimeoutExpired: + os.killpg(process.pid, signal.SIGKILL) + process.wait() + raise AssertionError(f'{label}: timeout; see {log}') + if watcher: + watcher.join() + compiler = json.loads((monitor / 'compiler.json').read_text()) + summaries[label] = dict(model=state.copy(), compiler=compiler, exit=status) + (work / 'summary.json').write_text(json.dumps(summaries, indent=2) + '\n') + assert (status != 0) == fail, f'{label}: exit {status}; see {log}\n{log.read_text()[-5000:]}' + assert not state['errors'], state + assert state['peak'] <= limit, state + assert compiler['peak'] <= 1 and compiler['active'] == 0 and compiler['source_changes'] == 0, compiler + assert (root / 'src/lib.rs').read_text() == original, label + assert (root / 'src/unrelated.rs').read_text() == '// unrelated source file\n', label + assert unrelated.read_bytes() == b'previous recovery material\n', label + return state.copy() + + def reset(seed=None): + shutil.rmtree(root / 'utgen', ignore_errors=True) + shutil.rmtree(root / 'tests', ignore_errors=True) + if seed: + shutil.copytree(seed, root / 'utgen/generation/pre_fix') + + def command(mode, n, *extra): + return ['utgen', mode, '-p', str(root), *([] if n == 4 else ['--tasks', str(n)]), *extra] + + def results(directory, compiled): + data = [json.loads(p.read_text()) for p in (root / 'utgen' / directory).glob('*.json')] + assert len(data) == count, (directory, len(data)) + assert all(item['tests_compiled'] == compiled for item in data), data + if compiled: + assert all(item['tests_run'] == 1 and item['tests_passed'] == 1 + and item['lines_covered'] > 0 for item in data), data + + try: + run('analyze', ['utgen', 'analyze', '-p', str(root)]) + seed = work / 'broken-candidates' + for n in [1, 2, 4]: + reset() + measured = run(f'gen-{n}', command('gen', n), limit=n, backpressure=n == 1) + assert measured['total'] == count and measured['peak'] == n, measured + results('result', 0) + assert not (root / 'src/lib.rs.bak').exists() + if n == 1: + shutil.copytree(root / 'utgen/generation/pre_fix', seed) + for n in [1, 2, 4]: + reset(seed) + measured = run(f'fix-{n}', command('fix', n), limit=n) + assert measured['total'] == count and measured['peak'] == n, measured + results('fixed_result', 1) + assert not (root / 'src/lib.rs.bak').exists() + for oracle in [False, True]: + reset() + preserved = {} + if oracle: + (root / 'tests').mkdir() + for name in ['use_check.rs', 'test_check.rs']: + path = root / 'tests' / name + path.write_text('// existing compiler input\n') + preserved[path] = path.read_bytes() + measured = run(f'integration-{oracle}', command('gen', 2, '--integration', *(['--oracle'] if oracle else [])), + limit=2, integration=True) + assert measured['total'] == count * (3 if oracle else 1), measured + results('result', 1) + for file in (root / 'utgen/generation/pre_fix').glob('*.json'): + text = file.read_text() + assert 'absent_crate' not in text and 'std::cmp::max' in text, file + for name in ['use_check.rs', 'test_check.rs']: + path = root / 'tests' / name + if path in preserved: + assert path.read_bytes() == preserved[path], path + else: + assert not path.exists(), path + for mode in ['gen', 'fix']: + reset(seed if mode == 'fix' else None) + measured = run(f'{mode}-panic', command(mode, 1), panic=True, fail=True) + assert measured['total'] == count, measured # Remaining workers still finish. + assert not (root / 'utgen' / ('result' if mode == 'gen' else 'fixed_result')).exists() + if mode == 'fix': + backup = root / 'src/lib.rs.bak' + assert backup.read_text() == original + # A later invocation must not overwrite retained recovery material. + backup.write_bytes(b'old recovery material\n') + measured = run('fix-existing-backup', command('fix', 1), fail=True) + assert measured['total'] == 0 + assert backup.read_bytes() == b'old recovery material\n' + backup.unlink() + # A consumer failure must still drain the queue and let every sender finish. + reset() + backup = root / 'src/lib.rs.bak' + backup.write_bytes(b'old recovery material\n') + measured = run('gen-validation-failure', command('gen', 1), fail=True) + assert measured['total'] == count + assert backup.read_bytes() == b'old recovery material\n' + assert not (root / 'utgen/result').exists() + backup.unlink() + # Fail after source insertion while the repair compilation lock is held. + reset(seed) + diagnostics = root / 'error_output.json' + diagnostics.unlink(missing_ok=True) + diagnostics.mkdir() + measured = run('fix-io-failure', command('fix', 2), limit=2, fail=True) + assert measured['total'] == 0 + assert (root / 'src/lib.rs.bak').read_text() == original + assert not (root / 'utgen/fixed_result').exists() + print('Task limits, queue backpressure, serial compilation, imports and failure cleanup passed.', flush=True) + finally: + server.shutdown() + server.server_close() + + +if __name__ == '__main__': + if Path(sys.argv[0]).name == 'cargo': + cargo_wrapper() + else: + main() diff --git a/utgen/README.md b/utgen/README.md index 6914462360..3d25411edd 100644 --- a/utgen/README.md +++ b/utgen/README.md @@ -122,7 +122,7 @@ This command needs neither model configuration nor analysis artifacts. It writes The same coverage entry point is used by generation and repair. It temporarily adds `#[coverage(off)]` to test-only modules (including generated `llmtests`), standalone test functions, and `#[cfg(test)]` helper functions/methods. Methods inside test-only impls are excluded individually; files with `#![cfg(test)]` are excluded as a whole. Module exclusion also covers nested helpers. Production functions called by tests remain instrumented; mixed production/test files are never excluded wholesale. Integration-test files use cargo-llvm-cov's default directory exclusions. Doc tests are not run by `--tests`. -Existing coverage attributes and feature gates are reused. For conditional attributes, PALM adds a complementary `cfg_attr` only where the existing setting is absent; rustc evaluates the conditions for each build. Explicit `coverage(on)` settings are respected. The nightly feature gate is inserted temporarily on the same line at crate roots, leaving compiler flags and Cargo configuration untouched. Ordinary compilation and repair need no coverage attributes. Coverage annotations are removed by restoring the bytes saved at coverage entry, after success or a returned build/report error. Temporary insertions preserve line numbers, but columns and byte offsets can change while measuring; use working copies and do not edit them concurrently. Recovery for the broader generation/repair backup flow and forced termination is still pending. +Existing coverage attributes and feature gates are reused. For conditional attributes, PALM adds a complementary `cfg_attr` only where the existing setting is absent; rustc evaluates the conditions for each build. Explicit `coverage(on)` settings are respected. The nightly feature gate is inserted temporarily on the same line at crate roots, leaving compiler flags and Cargo configuration untouched. Ordinary compilation and repair need no coverage attributes. Coverage annotations are removed by restoring the bytes saved at coverage entry, after success or a returned build/report error. Temporary insertions preserve line numbers, but columns and byte offsets can change while measuring; use working copies and do not edit them concurrently. Recovery after forced process termination is still pending. Scope is a standalone crate with sources under `src/` and Cargo target entry files. Test identification reuses preprocessing's conservative `cfg` rules. Non-entry expression fragments are kept unchanged, including any code inside them; macros are not expanded, and unmarked helpers outside test-only modules cannot be inferred as test-only. The pinned nightly and cargo-llvm-cov version are required. @@ -138,10 +138,12 @@ utgen gen -p --requirement --context | `-c, --context` | Include the focal function's context in the prompt. Default: off. | | `-o, --oracle` | Use separate input-range, test-prefix, and oracle generation. Default: off; otherwise generate complete tests directly. | | `-i, --integration` | Generate integration tests under `tests/`, using the analysis visibility flag to select functions and compilation checks to filter candidates. Default: off. | -| `-t, --tasks` | Default: 128. Currently sizes the result channel and does not enforce a strict limit on concurrent LLM requests. | +| `-t, --tasks` | Default: 4. Maximum active focal-function generation jobs; must be positive. | Generation validates the branch index, context index, context files, and source paths before modifying the target. Missing or inconsistent artifacts are errors. A failed generation task is reported instead of being silently lost before statistics. `--tasks 0` is rejected. +Each generation job handles one focal function, including its condition chains and input/prefix/oracle stages sequentially. It holds a slot until its result enters the bounded queue, which also has capacity N. A single consumer validates candidates and their imports while other jobs can await model responses. Thus N limits active generation jobs, not the number of compiler processes or Cargo's internal build jobs; buffered results and the candidate currently being validated are separate. + Generation may append an `ntest` dependency to the target's Cargo.toml. Existing `utgen/generation/pre_fix/.json` results are skipped, so use a fresh target copy for a different model, prompt, or generation mode. ### Repair @@ -150,7 +152,11 @@ Generation may append an `ntest` dependency to the target's Cargo.toml. Existing utgen fix -p ``` -Repair reads the generated candidates and uses compiler diagnostics to revise those that fail compilation. It does not target runtime assertion failures. The command currently accepts `--tasks`, but that value is not passed to the repair scheduler. +Repair reads the generated candidates and uses compiler diagnostics to revise those that fail compilation. It does not target runtime assertion failures. `--tasks N` defaults to 4 and limits active focal-function repair jobs through result saving. Each job processes its candidates and repair rounds sequentially. + +Source insertion, target cleanup, compilation/test execution, diagnostic reading, and source restoration are serialized within each command. Model requests may overlap. Cargo retains its own dependency-build parallelism. Concurrent PALM commands or experiments require separate working copies and separate target directories. + +All workers are joined before cleanup. A task panic or infrastructure error returns a nonzero status and skips subsequent coverage statistics; candidate compilation errors remain ordinary repair outcomes. Repair restores its source backups before returning, deletes only backups created by that invocation on success, and retains them after a worker failure. Existing source backups are rejected without overwriting them. Temporary import/candidate files are restored or removed after validation, including unwinding. This is not recovery from forced process termination, and request/subprocess deadlines remain unchanged. There is no integration-mode option for repair. Candidates are inserted into source files as unit tests, and post-repair statistics use that same mode. Do not interpret `gen --integration` followed by `fix` as preserving an integration-only evaluation. diff --git a/utgen/src/gene/cot/prefix_gen.rs b/utgen/src/gene/cot/prefix_gen.rs index 9b74af0eb0..7db48e2ab4 100644 --- a/utgen/src/gene/cot/prefix_gen.rs +++ b/utgen/src/gene/cot/prefix_gen.rs @@ -2,7 +2,6 @@ use super::Prompt; use super::LLM; use super::{extract_test_functions, try_parse}; use crate::types::{ChainTestAnswer, TestInfo}; -use crate::utils::use_check; use log::{error, info}; use rand::Rng; use std::collections::HashSet; @@ -19,8 +18,6 @@ fn postprocess(inputs: &mut Vec) { pub async fn gen_prefix( llm: &LLM, - // project_dir: &Path, - work_dir: &Path, answer_dir: &Path, pt_info: &Prompt, id: usize, @@ -104,7 +101,6 @@ pub async fn gen_prefix( let mut use_set = prefixs.uses.into_iter().collect::>(); if integration { use_set.remove("use super::*;"); - use_check(&mut use_set, work_dir); } test_answer_list.push(ChainTestAnswer::new( use_set.into_iter().collect(), diff --git a/utgen/src/gene/cot/test_gen.rs b/utgen/src/gene/cot/test_gen.rs index 9cf0af3c2c..b8e75dc04b 100644 --- a/utgen/src/gene/cot/test_gen.rs +++ b/utgen/src/gene/cot/test_gen.rs @@ -2,7 +2,6 @@ use super::Prompt; use super::LLM; use super::{extract_test_functions, try_parse}; use crate::types::{ChainTestAnswer, TestInfo}; -use crate::utils::use_check; use log::{error, info}; use rand::Rng; use std::collections::HashSet; @@ -19,8 +18,6 @@ fn postprocess(inputs: &mut Vec) { pub async fn gen_test( llm: &LLM, - // project_dir: &Path, - work_dir: &Path, answer_dir: &Path, pt_info: &Prompt, id: usize, @@ -100,7 +97,6 @@ pub async fn gen_test( let mut use_set = codes.uses.into_iter().collect::>(); if integration { use_set.remove("use super::*;"); - use_check(&mut use_set, work_dir); } test_answer_list.push(ChainTestAnswer::new( use_set.into_iter().collect(), diff --git a/utgen/src/gene/generation.rs b/utgen/src/gene/generation.rs index aa421f983b..9926cf79f6 100644 --- a/utgen/src/gene/generation.rs +++ b/utgen/src/gene/generation.rs @@ -1,8 +1,10 @@ use super::{gen_input_range, gen_oracle, gen_prefix, gen_test, LLM}; use super::{inputprompts, oracleprompts, prefixprompts, testprompts}; -use crate::utils::target_clean; use crate::types::{BrData, ChainTestInfo, TestGenInfo}; -use crate::utils::{backup_file, cargo_check, delete_backup, insert_test, restore_file, use_check}; +use crate::utils::{ + RestoreOnDrop, TemporaryFile, backup_file, cargo_check, delete_backup, insert_test, + restore_file, target_clean, use_check, +}; use log::info; use std::collections::HashSet; use std::fs; @@ -162,8 +164,6 @@ async fn gen_tests_cot( // Generate test prefix for each condition chain let test_answer_list = gen_prefix( llm, - // project_dir, - work_dir, &answer_dir, &prefix_pt_info, id, @@ -229,8 +229,6 @@ async fn gen_tests_cot( // Generate test prefix for each condition chain let test_answer_list = gen_prefix( llm, - // project_dir, - work_dir, &answer_dir, &prefix_pt_info, id, @@ -358,8 +356,6 @@ async fn gen_full_tests( // Generate test functions for each condition chain let test_answer_list = gen_test( llm, - // project_dir, - work_dir, &answer_dir, &test_pt_info, id, @@ -382,8 +378,6 @@ async fn gen_full_tests( // Generate test functions for each condition chain let test_answer_list = gen_test( llm, - // project_dir, - work_dir, &answer_dir, &test_pt_info, id, @@ -422,6 +416,7 @@ pub fn check_unit(test_gen: &mut TestGenInfo, project_dir: &Path, work_dir: &Pat let code_template: Vec = serde_json::from_str(template).unwrap(); // Backup the file backup_file(&path); + let restore = RestoreOnDrop(&path); let mut id = 0; for chain_test in test_gen.get_tests_mut() { for test_answer in chain_test.get_answers_mut() { @@ -456,6 +451,7 @@ pub fn check_unit(test_gen: &mut TestGenInfo, project_dir: &Path, work_dir: &Pat } } } + drop(restore); delete_backup(&path); } @@ -463,10 +459,9 @@ pub fn check_integration(test_gen: &mut TestGenInfo, work_dir: &Path) { let file_path = test_gen.get_file(); let stem = Path::new(&file_path).file_stem().unwrap().to_str().unwrap(); let path = work_dir.join("tests/test_check.rs"); - if path.exists() { - fs::remove_file(&path).unwrap(); - } + let temporary = TemporaryFile::new(&path).unwrap(); fs::create_dir_all(&path.parent().unwrap()).unwrap(); + fs::write(&path, "").unwrap(); let name = test_gen.get_name().to_string(); let fn_name = name.split("::").last().unwrap(); @@ -478,6 +473,14 @@ pub fn check_integration(test_gen: &mut TestGenInfo, work_dir: &Path) { let mut id = 0; for chain_test in test_gen.get_tests_mut() { for test_answer in chain_test.get_answers_mut() { + // Model tasks only collect imports. All import compilation happens + // here, in the single validation consumer. + fs::write(&path, "").unwrap(); + let mut uses: HashSet<_> = test_answer.get_uses().iter().cloned().collect(); + use_check(&mut uses, work_dir); + let mut uses: Vec<_> = uses.into_iter().collect(); + uses.sort(); + test_answer.set_uses(uses); let mut answer_use_set = init_use_set.clone(); answer_use_set.extend(test_answer.get_uses().clone()); let common = test_answer.get_common().clone(); @@ -507,4 +510,5 @@ pub fn check_integration(test_gen: &mut TestGenInfo, work_dir: &Path) { } } } + temporary.finish().unwrap(); } diff --git a/utgen/src/gene/mod.rs b/utgen/src/gene/mod.rs index 735b509d75..9919cb8f0c 100644 --- a/utgen/src/gene/mod.rs +++ b/utgen/src/gene/mod.rs @@ -13,9 +13,10 @@ use log::{error, info, warn}; use prompt::{Prompt, inputprompts, oracleprompts, prefixprompts, testprompts}; use std::collections::HashMap; use std::path::Path; +use std::sync::Arc; use std::{fs, io}; use test_ext::{extract_test_functions, try_parse}; -use tokio::sync::mpsc; +use tokio::sync::{Semaphore, mpsc}; pub use llm::LLM; @@ -65,10 +66,12 @@ pub async fn gen_tests_project( context: bool, oracle: bool, ) -> io::Result<()> { - validate_analysis(project_dir, work_dir)?; - if tasks == 0 { - return Err(io::Error::other("--tasks must be greater than zero")); + if tasks == 0 || tasks > Semaphore::MAX_PERMITS { + return Err(io::Error::other( + "--tasks is outside the supported positive range", + )); } + validate_analysis(project_dir, work_dir)?; add_ntest_dependency(work_dir); let brdata_dir = work_dir.join("brinfo/brdata"); let map_path = work_dir.join("brinfo/name_map.json"); @@ -80,6 +83,7 @@ pub async fn gen_tests_project( serde_json::from_str(&fs::read_to_string(&focxt_name_informations_path).unwrap()) .unwrap(); let (tx, mut rx) = mpsc::channel(tasks); + let slots = Arc::new(Semaphore::new(tasks)); let mut handles = Vec::new(); for entry in fs::read_dir(brdata_dir).unwrap() { let entry = entry.unwrap(); @@ -118,7 +122,11 @@ pub async fn gen_tests_project( let project_dir = project_dir.to_path_buf(); let work_dir = work_dir.to_path_buf(); let llm = llm.clone(); + let slots = slots.clone(); handles.push(tokio::spawn(async move { + // Wait inside the task so the producer loop cannot block + // the result consumer. Hold the slot through result delivery. + let _permit = slots.acquire_owned().await.map_err(io::Error::other)?; generation_task( llm, brdata, @@ -137,24 +145,46 @@ pub async fn gen_tests_project( } } drop(tx); + let mut failure = None; while let Some(mut test_gen_info) = rx.recv().await { + if failure.is_some() { + continue; // Drain results so senders can finish and release slots. + } let fn_name = test_gen_info.get_name().to_string(); info!("Checking tests for {}", fn_name); - if integration { - check_integration(&mut test_gen_info, work_dir); - } else { - check_unit(&mut test_gen_info, project_dir, work_dir); - } let file_path = project_dir.join(format!( "utgen/generation/pre_fix/{}.json", nmap.get(&fn_name).unwrap() )); - test_gen_info.dump_json(&file_path); + let project_dir = project_dir.to_owned(); + let work_dir = work_dir.to_owned(); + // Await exactly one validation at a time, without blocking model tasks. + let checked = tokio::task::spawn_blocking(move || { + if integration { + check_integration(&mut test_gen_info, &work_dir); + } else { + check_unit(&mut test_gen_info, &project_dir, &work_dir); + } + test_gen_info.dump_json(&file_path); + }) + .await; + if let Err(error) = checked { + failure = Some(io::Error::other(format!( + "Validation failed for {fn_name}: {error}" + ))); + } } for handle in handles { - handle + let result = handle .await - .map_err(|e| io::Error::other(format!("Generation task failed: {e}")))??; + .map_err(|e| io::Error::other(format!("Generation task failed: {e}"))) + .and_then(|result| result); + if let Err(error) = result { + failure.get_or_insert(error); + } + } + if let Some(error) = failure { + return Err(error); } } else { return Err(io::Error::other(format!( diff --git a/utgen/src/main.rs b/utgen/src/main.rs index 97ba96cb83..46ffdfa02a 100644 --- a/utgen/src/main.rs +++ b/utgen/src/main.rs @@ -1,8 +1,8 @@ use clap::{Args, Parser, Subcommand}; use log::info; use simplelog::{ColorChoice, ConfigBuilder, LevelFilter, TermLogger, TerminalMode}; -use std::env; use std::path::PathBuf; +use std::{env, num::NonZeroUsize}; use utgen::{LLM, LlmConfig, analyze_project, collect_coverage, validate_repair}; use utgen::{comment_out_tests, gen_test_rate, gen_tests_project, llm_fix, rename_tests_to_bak}; @@ -38,9 +38,9 @@ enum Command { Gen { #[command(flatten)] options: Opts, - /// Number of parallel test generation tasks - #[arg(short, long, default_value = "128")] - tasks: usize, + /// Maximum active function-generation tasks; compilation stays serial + #[arg(short, long, default_value = "4")] + tasks: NonZeroUsize, /// Whether to generate integration tests #[arg(short, long)] integration: bool, @@ -58,9 +58,9 @@ enum Command { Fix { #[command(flatten)] options: Opts, - /// Number of parallel test fix tasks - #[arg(short, long, default_value = "128")] - tasks: usize, + /// Maximum active function-repair tasks; compilation stays serial + #[arg(short, long, default_value = "4")] + tasks: NonZeroUsize, }, } @@ -180,7 +180,7 @@ async fn main() -> Result<(), Box> { &llm, &project_dir, &work_dir, - tasks, + tasks.get(), integration, requirement, context, @@ -204,7 +204,7 @@ async fn main() -> Result<(), Box> { ); for work_dir in work_dirs.iter() { validate_repair(&project_dir, work_dir)?; - llm_fix(&llm, project_dir.clone(), work_dir.clone()).await; + llm_fix(&llm, project_dir.clone(), work_dir.clone(), tasks.get()).await?; } //gen_test_rate_aggregated(&project_dir, false); for work_dir in work_dirs.iter() { diff --git a/utgen/src/run/llm_fix.rs b/utgen/src/run/llm_fix.rs index a1fe4669b1..79eff03005 100644 --- a/utgen/src/run/llm_fix.rs +++ b/utgen/src/run/llm_fix.rs @@ -8,8 +8,7 @@ use crate::{ gene::LLM, types::{InsertKind, TestGenInfo}, utils::{ - backup_file, cargo_check, delete_all_backups, has_backup, insert_test, restore_file, - target_clean, + RestoreOnDrop, cargo_check, create_backup, insert_test, restore_file, target_clean, }, }; use log::{error, info, warn}; @@ -17,11 +16,11 @@ use rand::Rng; use serde::Deserialize; use std::{ cmp::min, - collections::HashMap, + collections::{BTreeSet, HashMap}, error::Error, fs::{self, create_dir_all, exists, read_to_string, File}, i32, - io::Write, + io::{self, Write}, path::{Path, PathBuf}, sync::{ atomic::{AtomicUsize, Ordering}, @@ -30,7 +29,7 @@ use std::{ }; use tokio::time::{sleep, Duration}; use tokio::{ - sync::{Mutex, OnceCell}, + sync::{Mutex, OnceCell, Semaphore}, time::Instant, }; @@ -262,6 +261,8 @@ async fn compilation_fix_assistant_for_an_error( mod_code.splice(pos..pos, insert_code); let guard = lock.lock().await; + restore_file(file_path); + let restore = RestoreOnDrop(file_path); insert_test(insert_kind, Path::new(&file_path), &mod_code); let _ = target_clean(&work_path); @@ -281,6 +282,7 @@ async fn compilation_fix_assistant_for_an_error( // if request_result.len() != 0 { new_test_code = TestCode::new(&file_path, &new_test_code.codes); restore_file(Path::new(&file_path)); + drop(restore); drop(guard); // let mut min_set_index = MAX; // let mut i = 0; @@ -336,11 +338,6 @@ async fn compilation_fix_assistant_for_one_fn( // Backup the file // info!("Fix for {}", fn_name); let lock = FIX_LOCK.get_or_init(|| async { Mutex::new(()) }).await; - let guard = lock.lock().await; - if !has_backup(Path::new(&file_path)) { - backup_file(Path::new(&file_path)); - } - drop(guard); let mut id = 0; for fn_test in test_gen_info.get_tests_mut().iter_mut() { @@ -371,6 +368,8 @@ async fn compilation_fix_assistant_for_one_fn( mod_code.splice(pos..pos, insert_code); let guard = lock.lock().await; + restore_file(&file_path); + let restore = RestoreOnDrop(&file_path); insert_test(insert_kind, Path::new(&file_path), &mod_code); let _ = target_clean(&work_path); @@ -389,6 +388,7 @@ async fn compilation_fix_assistant_for_one_fn( compile_error_set.push(compile_error); } restore_file(Path::new(&file_path)); + drop(restore); drop(guard); let mut initial_error_num = min(compile_error_set.len() + 2, 10); @@ -465,6 +465,8 @@ async fn compilation_fix_assistant_for_one_fn( mod_code.splice(pos..pos, insert_code); let guard = lock.lock().await; + restore_file(&file_path); + let restore = RestoreOnDrop(&file_path); insert_test(insert_kind, &file_path, &mod_code); *test_code = test_file_content.codes; chain_test.repaired[num] = true; @@ -473,6 +475,7 @@ async fn compilation_fix_assistant_for_one_fn( chain_test.can_compile[num] = cargo_check(&work_path); id += 1; restore_file(Path::new(&file_path)); + drop(restore); drop(guard); if start_time.elapsed() >= timeout { warn!("Fix time out for {}", fn_name); @@ -506,7 +509,17 @@ fn merge_common_in_code(test_gen_infos: &mut Vec) { } } -pub async fn llm_fix(llm: &LLM, project_path: PathBuf, work_path: PathBuf) { +pub async fn llm_fix( + llm: &LLM, + project_path: PathBuf, + work_path: PathBuf, + tasks: usize, +) -> io::Result<()> { + if tasks == 0 || tasks > Semaphore::MAX_PERMITS { + return Err(io::Error::other( + "--tasks is outside the supported positive range", + )); + } let mut test_gen_infos; let pre_dir = project_path.join("utgen/generation/pre_fix"); let parent_dir = project_path.join("utgen/generation/llm_fix"); @@ -559,42 +572,87 @@ pub async fn llm_fix(llm: &LLM, project_path: PathBuf, work_path: PathBuf) { } } + test_gen_infos.retain(|info| { + let relative = info.get_file(); + if project_path == work_path { + relative.starts_with("src") + } else { + project_path.join(relative).starts_with(&work_path) + } + }); + // Back up each source once before starting workers. Reject stale backups + // rather than reusing them, and only clean up files owned by this command. + let sources: BTreeSet<_> = test_gen_infos + .iter() + .map(|info| project_path.join(info.get_file())) + .collect(); + let mut backups = Vec::new(); + for source in sources { + match create_backup(&source) { + Ok(backup) => backups.push((source, backup)), + Err(error) => { + // No worker has started, so these newly created backups are unused. + for (_, backup) in &backups { + fs::remove_file(backup)?; + } + return Err(io::Error::other(format!( + "Cannot back up {}: {error}", + source.display() + ))); + } + } + } + let slots = Arc::new(Semaphore::new(tasks)); let counter = Arc::new(AtomicUsize::new(0)); let length = test_gen_infos.len(); let mut handles = Vec::new(); - for test_gen_info in test_gen_infos.into_iter() { + for test_gen_info in test_gen_infos { let project_path_clone = project_path.clone(); let work_path = work_path.clone(); - let file_rela = test_gen_info.get_file(); - let file_path = project_path.join(&file_rela); - if (project_path == work_path && !file_rela.starts_with("src")) - || (project_path != work_path && !file_path.starts_with(&work_path)) - { - continue; - } let parent_dir_clone = parent_dir.clone(); let encoded_name = nmap.get(test_gen_info.get_name()).unwrap().to_owned(); let counter_clone = Arc::clone(&counter); - + let slots = slots.clone(); let llm = llm.clone(); - let handle = tokio::spawn(async move { - let test_gen_info = - compilation_fix_assistant_for_one_fn(&llm, project_path_clone, work_path, test_gen_info) - .await; - let lock = FIX_LOCK.get_or_init(|| async { Mutex::new(()) }).await; - let guard = lock.lock().await; + handles.push(tokio::spawn(async move { + let _permit = slots.acquire_owned().await.map_err(io::Error::other)?; + let test_gen_info = compilation_fix_assistant_for_one_fn( + &llm, + project_path_clone, + work_path, + test_gen_info, + ) + .await; let json_path = parent_dir_clone.join(encoded_name + ".json"); test_gen_info.dump_json(&json_path); - drop(guard); - counter_clone.fetch_add(1, Ordering::Relaxed); - info!( - "Fix progress: {}/{}", - counter_clone.load(Ordering::Relaxed), - length - ); - }); - handles.push(handle); + let completed = counter_clone.fetch_add(1, Ordering::Relaxed) + 1; + info!("Fix progress: {completed}/{length}"); + Ok::<_, io::Error>(()) + })); + } + let mut failure = None; + // Await every worker before restoring or deleting any shared backup. + for handle in handles { + let result = handle + .await + .map_err(|error| io::Error::other(format!("Repair task failed: {error}"))) + .and_then(|result| result); + if let Err(error) = result { + failure.get_or_insert(error); + } + } + for (source, backup) in &backups { + if let Err(error) = fs::copy(backup, source) { + failure.get_or_insert_with(|| { + io::Error::other(format!("Cannot restore {}: {error}", source.display())) + }); + } + } + if let Some(error) = failure { + return Err(error); // Retain recovery material and skip coverage statistics. + } + for (_, backup) in backups { + fs::remove_file(backup)?; } - futures::future::join_all(handles).await; - delete_all_backups(&work_path); + Ok(()) } diff --git a/utgen/src/run/run.rs b/utgen/src/run/run.rs index 099513215a..221482adc9 100644 --- a/utgen/src/run/run.rs +++ b/utgen/src/run/run.rs @@ -381,6 +381,9 @@ pub fn gen_test_rate(project_dir: &Path, work_dir: &Path, integration: bool, is_ } } } + // Every integration function reads the same coverage run. + fs::remove_file(work_dir.join("coverage.xml")).unwrap(); + fs::remove_file(work_dir.join("coverage.json")).unwrap(); for integration_info in integration_infos.iter() { let mut tests = integration_info.test_function_names.len() as i32; let mut passed_tests = 0; @@ -571,7 +574,6 @@ fn gen_one_coverage_rate_lines( // test_rate_info.branches_coverage_rate = branches_coverage_rate; test_rate_info.codes_lines = codes_lines; // test_rate_info.codes_branches = codes_branches; - remove_file(&coverage_file_path).unwrap(); } } @@ -665,8 +667,6 @@ fn gen_one_coverage_rate_branches( test_rate_info.branches_covered = branches_covered; test_rate_info.branches_coverage_rate = branches_coverage_rate; test_rate_info.codes_branches = codes_branches; - - fs::remove_file(&coverage_file_path).unwrap(); } fn gen_one_pass_rate(test_output: &String) -> (i32, i32, f64) { diff --git a/utgen/src/types/testgen.rs b/utgen/src/types/testgen.rs index 051ee87dac..afc7a464e7 100644 --- a/utgen/src/types/testgen.rs +++ b/utgen/src/types/testgen.rs @@ -214,6 +214,10 @@ impl ChainTestAnswer { &self.uses } + pub fn set_uses(&mut self, uses: Vec) { + self.uses = uses; + } + pub fn has_test_mod(&self) -> bool { self.has_test_mod } diff --git a/utgen/src/utils/backup.rs b/utgen/src/utils/backup.rs index 43185cac9d..506b582b83 100644 --- a/utgen/src/utils/backup.rs +++ b/utgen/src/utils/backup.rs @@ -1,57 +1,108 @@ -use std::{fs, path::Path}; -use walkdir::WalkDir; +use std::{ + fs, + io::{self, Write}, + path::{Path, PathBuf}, +}; -pub fn has_backup(file_path: &Path) -> bool { - let backup_path = file_path.with_extension(format!( +fn backup_path(path: &Path) -> PathBuf { + path.with_extension(format!( "{}.bak", - file_path - .extension() - .unwrap_or_default() - .to_str() - .unwrap_or_default() - )); - return fs::exists(backup_path).unwrap(); + path.extension().unwrap_or_default().to_string_lossy() + )) } -pub fn delete_backup(file_path: &Path) { - let backup_path = file_path.with_extension(format!( - "{}.bak", - file_path - .extension() - .unwrap_or_default() - .to_str() - .unwrap_or_default() - )); - fs::remove_file(backup_path).unwrap(); +pub fn has_backup(path: &Path) -> bool { + backup_path(path).exists() +} + +pub fn delete_backup(path: &Path) { + fs::remove_file(backup_path(path)).unwrap(); +} + +/// Create only a new backup; never replace a previous run's recovery material. +pub(crate) fn create_backup(path: &Path) -> io::Result { + let original = fs::read(path)?; + let backup = backup_path(path); + let mut file = fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&backup)?; + file.write_all(&original)?; + file.set_permissions(fs::metadata(path)?.permissions())?; + Ok(backup) } pub fn backup_file(path: &Path) { - let backup_path = path.with_extension(format!( - "{}.bak", - path.extension() - .unwrap_or_default() - .to_str() - .unwrap_or_default() - )); - fs::copy(path, &backup_path).expect(format!("{:?}", path).as_str()); + create_backup(path).expect("cannot create source backup; check for an existing .bak file"); } pub fn restore_file(path: &Path) { - let backup_path = path.with_extension(format!( - "{}.bak", - path.extension() - .unwrap_or_default() - .to_str() - .unwrap_or_default() - )); - fs::copy(&backup_path, path).unwrap(); -} - -pub fn delete_all_backups(work_path: &Path) { - for entry in WalkDir::new(work_path).into_iter().filter_map(Result::ok) { - let path = entry.path(); - if path.is_file() && path.extension().unwrap_or_default() == "rs" && has_backup(path) { - delete_backup(path); + fs::copy(backup_path(path), path).unwrap(); +} + +/// Restore before a validation panic releases its compilation lock. Keep the +/// on-disk backup on failure; normal callers delete it after successful checks. +pub(crate) struct RestoreOnDrop<'a>(pub &'a Path); + +impl Drop for RestoreOnDrop<'_> { + fn drop(&mut self) { + if let Err(error) = fs::copy(backup_path(self.0), self.0) { + log::error!( + "Cannot restore {}: {error}; backup retained", + self.0.display() + ); + } + } +} + +/// Own a temporary compiler input, restoring any pre-existing content. +pub(crate) struct TemporaryFile { + path: PathBuf, + original: Option>, + finished: bool, +} + +impl TemporaryFile { + pub fn new(path: &Path) -> io::Result { + let original = match fs::read(path) { + Ok(bytes) => Some(bytes), + Err(error) if error.kind() == io::ErrorKind::NotFound => None, + Err(error) => return Err(error), + }; + Ok(Self { + path: path.to_owned(), + original, + finished: false, + }) + } + + fn restore(&self) -> io::Result<()> { + if let Some(bytes) = &self.original { + fs::write(&self.path, bytes) + } else { + match fs::remove_file(&self.path) { + Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(()), + result => result, + } + } + } + + pub fn finish(mut self) -> io::Result<()> { + self.restore()?; + self.finished = true; + Ok(()) + } +} + +impl Drop for TemporaryFile { + fn drop(&mut self) { + if !self.finished { + if let Err(error) = self.restore() { + log::error!( + "Cannot restore temporary file {}: {error}", + self.path.display() + ); + } } } } diff --git a/utgen/src/utils/check.rs b/utgen/src/utils/check.rs index 1ba5f0284c..bc88285d82 100644 --- a/utgen/src/utils/check.rs +++ b/utgen/src/utils/check.rs @@ -1,3 +1,4 @@ +use super::TemporaryFile; use std::collections::HashSet; use std::fs; use std::path::Path; @@ -23,9 +24,7 @@ pub fn cargo_check(work_dir: &Path) -> Result<(), String> { pub fn use_check(use_set: &mut HashSet, work_dir: &Path) { let path = work_dir.join("tests/use_check.rs"); - if path.exists() { - fs::remove_file(&path).unwrap(); - } + let temporary = TemporaryFile::new(&path).unwrap(); fs::create_dir_all(&path.parent().unwrap()).unwrap(); use_set.retain(|x| { @@ -33,4 +32,5 @@ pub fn use_check(use_set: &mut HashSet, work_dir: &Path) { fs::write(&path, code).unwrap(); cargo_check(&work_dir).is_ok() }); + temporary.finish().unwrap(); } diff --git a/utgen/tests/cli_config.rs b/utgen/tests/cli_config.rs index 9d73bf89ac..9b0995ba8d 100644 --- a/utgen/tests/cli_config.rs +++ b/utgen/tests/cli_config.rs @@ -66,6 +66,33 @@ fn help_and_non_model_commands_do_not_load_configuration() { } } +#[test] +fn tasks_default_to_four_and_reject_zero_before_loading_configuration() { + let fixture = Fixture::new(); + let before = fs::read(fixture.0.join("Cargo.toml")).unwrap(); + for command in ["gen", "fix"] { + let help = fixture + .command() + .args([command, "--help"]) + .output() + .unwrap(); + assert!(help.status.success()); + assert!(String::from_utf8_lossy(&help.stdout).contains("[default: 4]")); + let output = fixture + .command() + .args([command, "-p", ".", "--tasks", "0"]) + .env("PALM_CONFIG", "missing.json") + .output() + .unwrap(); + assert!(!output.status.success()); + let error = String::from_utf8_lossy(&output.stderr); + assert!(error.contains("--tasks"), "{error}"); + assert!(!error.contains("Cannot read"), "{error}"); + assert_eq!(before, fs::read(fixture.0.join("Cargo.toml")).unwrap()); + assert!(!fixture.0.join("utgen").exists()); + } +} + #[test] fn analyze_validates_the_crate_without_loading_model_configuration() { let fixture = Fixture::new();