Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,14 @@ name: Build and test

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions:
contents: read

Expand All @@ -18,15 +23,41 @@ jobs:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
with:
persist-credentials: false
fetch-depth: 0
- name: Check patch and determine validation scope
id: scope
env:
BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
run: |
if [[ "$GITHUB_EVENT_NAME" == "workflow_dispatch" ]] || ! git cat-file -e "$BASE_SHA^{commit}" 2>/dev/null; then
run_code=true
else
git diff --check "$BASE_SHA" HEAD
run_code=false
while IFS= read -r -d '' path; do
case "$path" in
README.md|*/README.md|docs/*.md|docs/assets/*.png|docs/ase2025/*.pdf) ;;
*) run_code=true; break ;;
esac
done < <(git diff --name-only --no-renames -z "$BASE_SHA" HEAD)
fi
echo "run_code=$run_code" >> "$GITHUB_OUTPUT"
if [[ "$run_code" == "false" ]]; then
echo "Documentation-only change: patch whitespace checked; Rust checks skipped." >> "$GITHUB_STEP_SUMMARY"
fi
- name: Install the pinned compiler and components
if: steps.scope.outputs.run_code == 'true'
run: rustup toolchain install nightly-2025-03-19 --profile minimal --component rust-src --component rustc-dev --component llvm-tools-preview
- name: Build without model configuration
if: steps.scope.outputs.run_code == 'true'
run: |
test ! -e utgen/res/api.json
cargo build --workspace --locked
- name: Run tests without a model service
if: steps.scope.outputs.run_code == 'true'
run: cargo test --workspace --locked
- name: Check the installer and CLI entry points
if: steps.scope.outputs.run_code == 'true'
run: |
bash -n install.sh
./install.sh --help
Expand All @@ -35,16 +66,23 @@ jobs:
./target/debug/focxt --help
./target/debug/utgen --help
- name: Check the standalone example workspace
if: steps.scope.outputs.run_code == 'true'
run: cargo metadata --manifest-path examples/bytes/Cargo.toml --no-deps --format-version 1 > /dev/null
- name: Install the validated coverage tool
if: steps.scope.outputs.run_code == 'true'
run: cargo +stable install cargo-llvm-cov --version 0.6.16 --locked
- name: Check coverage annotation compatibility
if: steps.scope.outputs.run_code == 'true'
run: python3 scripts/check_coverage.py
- name: Validate the minimal analysis and test pipeline
if: steps.scope.outputs.run_code == 'true'
run: python3 scripts/check_minimal.py
- name: Check generation and repair task limits
if: steps.scope.outputs.run_code == 'true'
run: python3 scripts/check_tasks.py
- name: Build release analysis tools
if: steps.scope.outputs.run_code == 'true'
run: cargo build --workspace --release --locked
- name: Validate bytes analysis and method identities
if: steps.scope.outputs.run_code == 'true'
run: python3 scripts/check_bytes.py
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,9 @@ cargo build --workspace --locked
cargo test --workspace --locked
```

Default tests use local model-response fixtures and do not contact a model service. The real-service test is opt-in; see [utgen testing](utgen/README.md#testing). Dependency downloads may still require network access. [Build validation](docs/build-validation.md) records the baseline and checks performed; the GitHub Actions workflow runs these checks and the minimal pipeline on Linux.
Default tests use local model-response fixtures and do not contact a model service. The real-service test is opt-in; see [utgen testing](utgen/README.md#testing). Dependency downloads may still require network access. [Build validation](docs/build-validation.md) records the baseline and checks performed.

The [GitHub Actions workflow](.github/workflows/ci.yml) runs on pull requests and pushes to main, canceling superseded runs for the same PR or branch. Changes limited to READMEs, Markdown guides, and documentation images/PDFs receive a lightweight patch whitespace check. Code changes and manual workflow runs execute the full offline suite on Linux.

## Documentation

Expand Down
4 changes: 2 additions & 2 deletions docs/minimal-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ The script runs in Linux CI. Coverage and preprocessing share only source parsin

## Task Scheduling Regression

`scripts/check_tasks.py` uses nine focal functions sharing one source file, a loopback model, and real Cargo commands. It measures generation and repair request peaks for N=1, N=2, and the default N=4, requires overlap for N>1, and checks that compiler invocations never overlap or observe changing source bytes. A paused compiler lets the N=1 generation queue fill: one candidate is being validated, one result is buffered, and one active job waits to send. Releasing compilation must drain every result.
`scripts/check_tasks.py` uses nine focal functions sharing one source file, a loopback model, and real Cargo commands. Nine-function runs exercise N=1 and the default N=4; N=2 uses five functions. Each scheduling case has more than 2N functions to exercise queued work. The check requires overlap for N>1 and verifies that compiler invocations never overlap or observe changing source bytes. Import-handling and model-request failure cases use the two-function selection. A paused compiler lets the N=1 generation queue fill: one candidate is being validated, one result is buffered, and one active job waits to send. Releasing compilation must drain every result.

The same check covers direct and input/prefix/oracle integration generation, removal of invalid imports from saved candidates, and preservation/removal of temporary compiler inputs. Integration coverage reports remain available until every function has been read, and all nine functions must have passing tests and nonzero covered lines. Valid answers without usage must complete successfully and mark request statistics incomplete. HTTP 401 during generation, oracle generation, and repair must fail without outer-layer retries. A blocked prompt directory exercises worker-panic propagation and slot release; a stale generation backup exercises queue draining after consumer failure, and blocking the diagnostic output file exercises failure while the repair lock is held. Sources must be restored, failed repair backups retained, unrelated backups untouched, existing recovery material rejected, and post-failure statistics skipped. These checks use byte comparisons without hashing and run on macOS/Linux. CLI tests separately verify the default and rejection of zero before configuration loading.
The same check covers direct and input/prefix/oracle integration generation, removal of invalid imports from saved candidates, and preservation/removal of temporary compiler inputs. Integration coverage reports remain available until every selected function has been read, and each must have passing tests and nonzero covered lines. Valid answers without usage must complete successfully and mark request statistics incomplete. HTTP 401 during generation, oracle generation, and repair must fail without outer-layer retries. A blocked prompt directory exercises worker-panic propagation and slot release; a stale generation backup exercises queue draining after consumer failure, and blocking the diagnostic output file exercises failure while the repair lock is held. Sources must be restored, failed repair backups retained, unrelated backups untouched, existing recovery material rejected, and post-failure statistics skipped. These checks use byte comparisons without hashing and run on macOS/Linux. CLI tests separately verify the default and rejection of zero before configuration loading.


## Model Request Regression
Expand Down
19 changes: 7 additions & 12 deletions focxt/src/collect_context/items_context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -961,20 +961,15 @@ impl ImplFnItem {
}
}

pub fn insert_item(&mut self, item: &ImplItemFn) {
pub fn insert_item(&mut self, item: &ImplItemFn, self_type: &syn::Type) {
self.item = Some(item.clone());
if let ReturnType::Type(_, r_type) = &item.sig.output {
match r_type {
_ => {
let r_type_string = r_type.to_token_stream().to_string();
if r_type_string.contains("Self")
|| r_type_string.contains(self.get_name().as_str())
{
self.is_constructor = true;
}
}
self.is_constructor = match &item.sig.output {
ReturnType::Type(_, return_type) => {
return_type.to_token_stream().to_string().contains("Self")
|| return_type.as_ref() == self_type
}
}
ReturnType::Default => false,
};
}

pub fn insert_fn_name(&mut self, fn_name: &String) {
Expand Down
4 changes: 2 additions & 2 deletions focxt/src/collect_context/syntax_context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1038,7 +1038,7 @@ impl SyntaxContext {
function
.insert_complete_name_in_file(&format!("{{impl#{}}}", item.get_impl_num()));
function.insert_visibility(parse_visibility(&parsed.vis));
function.insert_item(&parsed);
function.insert_item(&parsed, &item.get_item().self_ty);
function.bind_compiler_info(info);
item.insert_function(&function);
}
Expand Down Expand Up @@ -1279,7 +1279,7 @@ impl SyntaxContext {
let mut modified_item_fn = item_fn.clone();
modified_item_fn.attrs =
delete_doc_attributes(&modified_item_fn.attrs);
impl_fn_item.insert_item(&modified_item_fn);
impl_fn_item.insert_item(&modified_item_fn, &item_impl.self_ty);
let mut inside_items: Vec<Item> = Vec::new();
for stmt in item_fn.block.stmts.iter() {
if let Stmt::Item(stmt_item) = stmt {
Expand Down
41 changes: 41 additions & 0 deletions scripts/check_bytes.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,45 @@ def check_artifacts(root, expected):
return names, contexts


def check_constructor_context(root, run):
(root / 'src').mkdir(parents=True)
(root / 'Cargo.toml').write_text('[package]\nname="constructor_fixture"\nversion="0.1.0"\nedition="2021"\n[workspace]\n')
shutil.copyfile(REPO / 'rust-toolchain.toml', root / 'rust-toolchain.toml')
(root / 'src/lib.rs').write_text('''pub struct Subject(u32);
pub struct SubjectOther;
macro_rules! factory { () => { pub fn recovered(value: u32) -> Subject { Subject(value + 33) } }; }
impl Subject {
pub fn new(value: u32) -> Self { Self(value + 11) }
pub fn named(value: u32) -> Subject { Subject(value + 22) }
factory!();
pub fn ordinary(&self) -> u32 { self.0 + 44 }
pub fn unrelated() -> SubjectOther { SubjectOther }
pub fn value(&self) -> u32 { self.0 }
}
pub fn use_named(value: u32) -> Subject { Subject::named(value) }
pub struct Holder<T>(T);
impl<T> Holder<T> {
pub fn new(value: T) -> Self { Self(value) }
pub fn named(value: T) -> Holder<T> { Holder(value) }
pub fn view(&self) -> &T { &self.0 }
}
''')
before = snapshot(root)
run(root, 'analyze', ['utgen', 'analyze', '-p', str(root)])
assert snapshot(root) == before
_, infos = check_artifacts(root, 10)
contexts = {info['fn_name']: (root / f"focxt/{info['encoded_name']}.rs").read_text()
for info in infos.values() if info['fn_name'] in ['value', 'use_named', 'view']}
# Check the emitted impl, not the retained macro definition preceding it.
subject = contexts['value'].split('impl Subject {', 1)[1]
for name, result in [('new', 'Self'), ('named', 'Subject'), ('recovered', 'Subject')]:
assert re.search(r'fn ' + name + r'\(value: u32\) -> ' + result + r'\s*\{\s*' + result + r'\(value \+ \d+\)', subject), subject
assert re.search(r'fn ordinary\(&self\) -> u32 \{\}', subject), subject
assert re.search(r'fn unrelated\(\) -> SubjectOther \{\}', subject), subject
assert re.search(r'fn named\(value: T\) -> Holder<T>\s*\{\s*Holder\(value\)', contexts['view'])
assert re.search(r'fn named\(value: u32\) -> Subject\s*\{\s*Subject\(value \+ 22\)', contexts['use_named'])


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--bin-dir', type=Path, default=REPO / 'target/release')
Expand Down Expand Up @@ -109,6 +148,8 @@ def run(root, label, command):
seen.add('default method')
assert len(seen) == 4, seen

check_constructor_context(work / 'constructors', run)

original = snapshot(REPO / 'examples/bytes')
for number in range(1, args.runs + 1):
root = work / f'bytes-{number}'
Expand Down
45 changes: 26 additions & 19 deletions scripts/check_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -316,38 +316,45 @@ def results(directory, compiled, names=None):
else:
assert not (root / 'utgen/result').exists()
seed = work / 'broken-candidates'
for n in [1, 2, 4]:
two_task_names = sorted(nmap)[:5] # More than 2N for N=2.
two_task_file = work / 'two-task-functions.txt'
two_task_file.write_text('\n'.join(two_task_names) + '\n')
task_cases = [(1, None), (2, two_task_names), (4, None)]
for n, names in task_cases:
reset()
measured = run(f'gen-{n}', command('gen', n), limit=n, missing_usage=n == 1, backpressure=n == 1)
assert measured['total'] == count and measured['peak'] == n, measured
results('result', 0)
extra = ['--functions-file', str(two_task_file)] if names else []
measured = run(f'gen-{n}', command('gen', n, *extra), limit=n, missing_usage=n == 1, backpressure=n == 1)
assert measured['total'] == (len(names) if names else count) and measured['peak'] == n, measured
results('result', 0, names)
assert not (root / 'src/lib.rs.bak').exists()
if n == 1:
shutil.copytree(root / 'utgen/generation/pre_fix', seed)
measured = run('gen-cached', command('gen', 1))
assert measured['total'] == 0
report = json.loads((root / 'utgen/generation/gen-requests.json').read_text())
assert report['attempts'] == 0 and report['usage_complete'], report
for n in [1, 2, 4]:
for n, names in task_cases:
reset(seed)
measured = run(f'fix-{n}', command('fix', n), limit=n, missing_usage=n == 1)
assert measured['total'] == count and measured['peak'] == n, measured
results('fixed_result', 1)
extra = ['--functions-file', str(two_task_file)] if names else []
measured = run(f'fix-{n}', command('fix', n, *extra), limit=n, missing_usage=n == 1)
assert measured['total'] == (len(names) if names else count) and measured['peak'] == n, measured
results('fixed_result', 1, names)
assert not (root / 'src/lib.rs.bak').exists()

# Existing results for other functions must not enter this invocation.
nmap = json.loads((root / 'brinfo/name_map.json').read_text())
selected = sorted(nmap)[:2]
functions_file = work / 'functions.txt'
functions_file.write_text('\n' + '\n'.join(selected + [selected[0]]) + '\n')
selection_args = ['--functions-file', str(functions_file), '--max-requests', '2']
function_args = ['--functions-file', str(functions_file)]
selection_args = [*function_args, '--max-requests', '2']
reset(seed)
pre_dir = root / 'utgen/generation/pre_fix'
for name in selected:
(pre_dir / (nmap[name] + '.json')).unlink()
untouched = {p: p.read_bytes() for p in pre_dir.glob('*.json')}
measured = run('gen-selected', command('gen', 2, *selection_args), limit=2)
assert measured['total'] == 2
assert measured['total'] == 2 and measured['peak'] == 2, measured
results('result', 0, selected)
for path, content in untouched.items():
assert path.read_bytes() == content, path
Expand All @@ -372,7 +379,7 @@ def results(directory, compiled, names=None):
old_other = other.read_bytes()
before_fix = {p: p.read_bytes() for p in pre_dir.glob('*.json')}
measured = run('fix-selected', command('fix', 2, *selection_args), limit=2)
assert measured['total'] == 2
assert measured['total'] == 2 and measured['peak'] == 2, measured
results('fixed_result', 1, selected)
assert len(list(fixed_dir.glob('*.json'))) == 3
assert other.read_bytes() == old_other
Expand Down Expand Up @@ -417,10 +424,10 @@ def results(directory, compiled, names=None):
path = root / 'tests' / name
path.write_text('// existing compiler input\n')
preserved[path] = path.read_bytes()
measured = run(f'integration-{oracle}', command('gen', 2, '--integration', *(['--oracle'] if oracle else [])),
measured = run(f'integration-{oracle}', command('gen', 2, '--integration', *function_args, *(['--oracle'] if oracle else [])),
limit=2, integration=True)
assert measured['total'] == count * (3 if oracle else 1), measured
results('result', 1)
assert measured['total'] == len(selected) * (3 if oracle else 1), measured
results('result', 1, selected)
for file in (root / 'utgen/generation/pre_fix').glob('*.json'):
text = file.read_text()
assert 'absent_crate' not in text and 'std::cmp::max' in text, file
Expand All @@ -432,22 +439,22 @@ def results(directory, compiled, names=None):
assert not path.exists(), path
for mode in ['gen', 'fix']:
reset(seed if mode == 'fix' else None)
measured = run(f'{mode}-request-error', command(mode, 1), fault='request', fail=True)
assert measured['total'] == count, measured # Remaining workers still finish.
measured = run(f'{mode}-request-error', command(mode, 1, *function_args), fault='request', fail=True)
assert measured['total'] == len(selected), measured # The other selected function still finishes.
assert not (root / 'utgen' / ('result' if mode == 'gen' else 'fixed_result')).exists()
if mode == 'fix':
backup = root / 'src/lib.rs.bak'
assert backup.read_text() == original
# A later invocation must not overwrite retained recovery material.
backup.write_bytes(b'old recovery material\n')
measured = run('fix-existing-backup', command('fix', 1), fail=True)
measured = run('fix-existing-backup', command('fix', 1, *function_args), fail=True)
assert measured['total'] == 0
assert backup.read_bytes() == b'old recovery material\n'
backup.unlink()
reset()
measured = run('oracle-request-error', command('gen', 1, '--integration', '--oracle'),
measured = run('oracle-request-error', command('gen', 1, '--integration', '--oracle', *function_args),
integration=True, fault='oracle', fail=True)
assert measured['total'] == count * 3 and measured['fault_sent']
assert measured['total'] == len(selected) * 3 and measured['fault_sent']
assert not (root / 'utgen/result').exists()
reset()
encoded = next(iter(json.loads((root / 'brinfo/name_map.json').read_text()).values()))
Expand Down
Loading