Skip to content

catch 6 + 6b: TS import surface and harness import merge; defect 26: match-statement walker - #7

Merged
anp0429 merged 5 commits into
mainfrom
catch-6-import-surface
Jul 25, 2026
Merged

catch 6 + 6b: TS import surface and harness import merge; defect 26: match-statement walker#7
anp0429 merged 5 commits into
mainfrom
catch-6-import-surface

Conversation

@anp0429

@anp0429 anp0429 commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Two symmetrical import-surface fixes, one measurement-driven correction, and one walker fix.

Catch 6 — TS import surface. import_surface now dispatches TS/JS targets to a deterministic extractor (new shared module ts_surface.py): declarations, destructured exports, brace re-exports with aliases, star re-export chains to four hops, type-only names separated, comments stripped, silence when nothing resolves. Ground-truthed against real repos before delivery: 52/52 on ufo's src/index.ts vs tsx runtime keys, 4/4 on ohash's built dist, zero false positives.

Catch 6b — import merge in the vitest injector. The after-measurement for catch 6 (fresh replay of ufo 5cd9e67) came back worse, 9→13 broken with the same ReferenceError: withoutBase is not defined, and the investigation overturned the premise: the class was never invented imports. VitestHarness.strip_imports removes all proposal imports by design (proposals inject into the existing tests file, where mid-file imports are illegal), and catch-1 routes utils.ts into a host that never bound withoutBase. The fix: inject() now merges stripped bindings into the host's import header, deterministically — only names the target's real export surface vouches for, deduped against what the host already binds, host's own specifier reused for the same resolved file, bare specifiers only when the host already imports that module, everything unverifiable dropped so a transform error can never poison the file. Verified by executing a merged withoutBase proposal inside the real ufo repo: 99 tests pass, the proposal reaches a verdict instead of a ReferenceError. The proposer prompt's closing line now states the merge contract.

Defect 26 — match-statement walker (board scenario e6ecaf785f9bbc67). The module-scope walk descended if/try/for/while/with but not match; capture patterns and case-body bindings are persisting globals. The board's own scenario is the regression, including the function-scope negative.

Also: the pytest harness accepts the new host_path kwarg (ignored — its polite dedupe already keeps needed imports), and one pre-existing test updated from asserting the removed .ts-is-unsupported limitation to asserting the still-true silence for unsupported languages.

Suite: 306 passed, 2 skipped in a clean container (environment-gated modules add 12 locally). Pinned ruff clean.

Known follow-up, deliberately not in this PR: the proposal cache key excludes the prompt's surface content, so proposer upgrades silently replay stale proposals until AGENTBOARD_FRESH=1 — fold a surface hash into the key (chore filed).

@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown

agentboard review

verdicts: confirmed_gap=9 handled=3 skipped_covered=11 fingerprint: 9c509caa57ef7323

9 confirmed gap(s). Each one is a test that compiled, ran, and failed against this change. Run it yourself to reproduce.

1. The TypeScript export surface should not mint exports from lines inside multiline string or template literal contents.

AssertionError: assert ['fake', 'alsoFake', 'real'] == ['real']

Auditor (advisory): likely real — The failed assertion reflects fabricated TypeScript exports, and the source describes the export surface as reporting actual exported runtime names rather than inventing names found in non-code text.

the test that failed
def test_ts_surface_ignores_export_text_inside_multiline_strings(tmp_path):
    from agentboard.ts_surface import _ts_exports
    m = tmp_path / "m.ts"
    m.write_text(
        "const fixture = `\n"
        "export function fake() {}\n"
        "export const alsoFake = 1;\n"
        "`;\n"
        "export const real = 1;\n"
    )
    values, types = _ts_exports(str(m), set(), 0)
    assert values == ["real"]
    assert types == []

2. A TypeScript export-star re-export should forward named exports from the source module but should not advertise the source module's default export as a default export of the re-exporting module.

AssertionError: assert ['named', 'default'] == ['named']

Auditor (advisory): likely real — The failed assertion shows _ts_exports advertising a default export through an export-star path, and the shown source frames these names as the target file’s actual exported runtime names, not a p

the test that failed
def test_ts_surface_export_star_does_not_reexport_default(tmp_path):
    from agentboard.ts_surface import _ts_exports
    leaf = tmp_path / "leaf.ts"
    leaf.write_text(
        "export const named = 1;\n"
        "export default function main() {}\n"
    )
    idx = tmp_path / "index.ts"
    idx.write_text("export * from './leaf';\n")
    values, types = _ts_exports(str(idx), set(), 0)
    assert values == ["named"]
    assert types == []

3. A TypeScript named re-export from a resolved relative source should only advertise names actually exported by that source, not every syntactically named re-export clause.

AssertionError: assert ['real', 'publicMissing'] == ['real']

Auditor (advisory): likely real — The source consumer treats _ts_exports as returning actual importable/exported runtime names, so including a re-export alias that the resolved source does not export is fabricated export information r

the test that failed
def test_ts_surface_resolved_named_reexports_must_exist_in_source(tmp_path):
    from agentboard.ts_surface import _ts_exports
    leaf = tmp_path / "leaf.ts"
    leaf.write_text("export const real = 1;\n")
    idx = tmp_path / "index.ts"
    idx.write_text(
        "export { real } from './leaf';\n"
        "export { missing as publicMissing } from './leaf';\n"
    )
    values, types = _ts_exports(str(idx), set(), 0)
    assert values == ["real"]
    assert types == []

4. A host's existing import type binding should not be treated as satisfying a stripped proposal's needed runtime import binding of the same local name.

assert ['import { te...toBe(1); });'] == ['import { te...../src";', '']

*Auditor (advisory): uncertain — The failing value is produced by VitestHarness.inject, but the provided source file does not define or show that code path, so I cannot prove from the source whether deduping a runtime import against *

the test that failed
def test_vitest_merge_does_not_dedupe_runtime_import_against_host_import_type(tmp_path):
    from agentboard.verifiers.harness import VitestHarness
    src = tmp_path / "src"
    src.mkdir()
    (src / "index.ts").write_text("export class Widget {}\n")
    tdir = tmp_path / "test"
    tdir.mkdir()
    host = tdir / "index.test.ts"
    host.write_text(
        'import { test, expect } from "vitest";\n'
        'import type { Widget } from "../src";\n\n'
        'test("host", () => { expect(1).toBe(1); });\n'
    )
    proposal = (
        'import { Widget } from "../src";\n'
        'test("merged", () => { expect(new Widget()).toBeTruthy(); });'
    )
    h = VitestHarness()
    merged, err = h.inject(host.read_text(), proposal, host_path=str(host))
    assert merged, err
    lines = merged.splitlines()
    assert lines[:4] == [
        'import { test, expect } from "vitest";',
        'import type { Widget } from "../src";',
        'import { Widget } from "../src";',
        '',
    ]

5. Vitest import merging should merge default and namespace bindings from a bare module when the host already imports that same bare module, because those are stripped-but-needed import bindings just like named imports.

assert ['import { te...y(); });', ''] == ['import { te..."react";', '']

Auditor (advisory): uncertain — The provided source does not contain VitestHarness.inject or the import-merging implementation that produced the merged text under assertion, so the code path that generated the failed value cannot be

the test that failed
def test_vitest_merge_bare_default_and_namespace_imports_when_host_has_module(tmp_path):
    from agentboard.verifiers.harness import VitestHarness
    tdir = tmp_path / "test"
    tdir.mkdir()
    host = tdir / "index.test.ts"
    host.write_text(
        'import { test, expect } from "vitest";\n'
        'import { act } from "react";\n\n'
        'test("host", () => { expect(act).toBeTruthy(); });\n'
    )
    proposal = (
        'import React from "react";\n'
        'import * as ReactNS from "react";\n'
        'test("merged", () => { expect(React).toBeTruthy(); expect(ReactNS).toBeTruthy(); });'
    )
    h = VitestHarness()
    merged, err = h.inject(host.read_text(), proposal, host_path=str(host))
    assert merged, err
    lines = merged.splitlines()
    assert lines[:5] == [
        'import { test, expect } from "vitest";',
        'import { act } from "react";',
        'import React from "react";',
        'import * as ReactNS from "react";',
        '',
    ]

6. Inline type modifiers inside a mixed TypeScript re-export list should make only those specifiers type-only, while runtime specifiers from the same source remain runtime exports and aliases are reported under their exported names.

AssertionError: assert ['value'] == {'value'}

*Auditor (advisory): likely false positive — The failed assertion is only a container-type mismatch: the observed value contains the correct exported alias value, while the provided source never promises _ts_exports returns sets rather than *

the test that failed
def test_ts_export_surface_respects_inline_type_only_reexport_aliases(tmp_path):
    from agentboard.ts_surface import _ts_exports

    src = tmp_path / 'src'
    src.mkdir()
    (src / 'defs.ts').write_text(
        'export interface Shape { x: number }\n'
        'export const runtimeValue = 1\n'
    )
    (src / 'index.ts').write_text(
        "export { type Shape as PublicShape, runtimeValue as value } from './defs'\n"
    )

    values, types = _ts_exports(str(src / 'index.ts'), set(), 0)

    assert len(values) == 1
    assert values == {'value'}
    assert len(types) == 1
    assert types == {'PublicShape'}

7. A resolved named re-export of a source module's default export should advertise the alias as the runtime export, while still filtering missing named specifiers from the same clause.

AssertionError: assert 3 == 2

Auditor (advisory): likely real — The failing test is not asserting an invented behavior: the source describes the TypeScript surface as actual exported runtime names and explicitly says names not actually exported are dropped, so ret

the test that failed
def test_ts_export_surface_resolves_default_named_reexports(tmp_path):
    from agentboard.ts_surface import _ts_exports

    src = tmp_path / 'src'
    src.mkdir()
    (src / 'impl.ts').write_text(
        'export default function make() { return 1 }\n'
        'export const helper = 2\n'
    )
    (src / 'index.ts').write_text(
        "export { default as makeThing, helper as renamedHelper, missing as notThere } from './impl'\n"
    )

    values, types = _ts_exports(str(src / 'index.ts'), set(), 0)

    assert len(values) == 2
    assert values == {'makeThing', 'renamedHelper'}
    assert len(types) == 0
    assert types == set()

8. A resolved TypeScript namespace re-export (export * as name from './module') should contribute exactly the namespace binding as a runtime export and should not also flatten the source module's members.

AssertionError: assert ['tools'] == {'tools'}

Auditor (advisory): likely false positive — The failure is only a representation mismatch: the returned runtime exports were exactly ['tools'], so the namespace was not flattened, and the source consumer treats values as an iterable of name

the test that failed
def test_ts_export_surface_resolves_namespace_reexports_without_flattening(tmp_path):
    from agentboard.ts_surface import _ts_exports

    src = tmp_path / 'src'
    src.mkdir()
    (src / 'tools.ts').write_text(
        'export const helper = 1\n'
        'export function run() { return helper }\n'
        'export interface ToolShape { name: string }\n'
    )
    (src / 'index.ts').write_text("export * as tools from './tools'\n")

    values, types = _ts_exports(str(src / 'index.ts'), set(), 0)

    assert len(values) == 1
    assert values == {'tools'}
    assert len(types) == 0
    assert types == set()

9. The TypeScript export scanner should ignore export-looking declarations inside block comments, not only line comments, strings, or template literals.

AssertionError: assert ['real'] == {'real'}

*Auditor (advisory): likely false positive — The failed assertion is only a list-versus-set type mismatch, while the source’s own caller treats _ts_exports results as iterable collections for joining, not as sets; the observed ['real'] also *

the test that failed
def test_ts_export_surface_ignores_exports_inside_block_comments(tmp_path):
    from agentboard.ts_surface import _ts_exports

    src = tmp_path / 'src'
    src.mkdir()
    (src / 'mod.ts').write_text(
        '/*\n'
        'export const ghost = 1\n'
        'export function phantom() { return ghost }\n'
        'export interface PhantomShape { x: number }\n'
        '*/\n'
        'export const real = 1\n'
    )

    values, types = _ts_exports(str(src / 'mod.ts'), set(), 0)

    assert len(values) == 1
    assert values == {'real'}
    assert len(types) == 0
    assert types == set()
14 other proposed behavior(s) (passed, covered, or not executable)
  • skipped_covered The change operates on TypeScript/JavaScript module export-surface extraction, ES import parsing/merging for Vitest host files, and Python AST import-surface extraction for match statements.
  • skipped_covered The TypeScript import surface should report real runtime declarations, destructured exported bindings, followed re-export chains, default exports, package-name guidance, and type-only declarations separately from runtime names.
  • skipped_covered The TypeScript import surface should return silence for missing files or modules with no public export surface rather than inventing names.
  • skipped_covered The Python import-surface walker should include module-scope match capture-pattern and case-body bindings while not leaking names from matches inside nested function scope.
  • handled The Python import-surface walker should include bindings from class-pattern keyword captures in module-scope match statements.
  • skipped_covered Vitest injection should merge needed verified named imports from relative modules, reuse the host's existing specifier for the same resolved file, merge bare named imports only for modules already imported by the host, and drop invented, type-only, and unresolvable imports.
  • skipped_covered Vitest injection without a host_path should preserve the old strip-only behavior rather than attempting host-relative import merging.
  • skipped_covered Vitest import merging should preserve aliased named imports as exported-name-to-local-name imports and verify the exported name, not the local alias, against the export surface.
  • skipped_covered Vitest import merging should be idempotent when the same needed alias has already been merged into the host imports.
  • handled Vitest import merging should preserve a host file's trailing newline when it inserts imports.
  • skipped_covered TypeScript named re-exports from unresolvable relative sources should contribute no exported names.
  • skipped_covered TypeScript namespace re-exports from unresolvable relative sources should contribute no namespace binding.
  • skipped_covered An exported ambient declare namespace should be type-only, while declare const/function and non-ambient namespace declarations remain runtime-importable values.
  • handled The Python module-scope match walker should collect captures introduced by OR-pattern alternatives, because those captures are module globals just like other match-pattern bindings.

Advisory only; a human decides. Verdicts come from executed tests, never from model judgment. agentboard

@anp0429
anp0429 merged commit 857fcee into main Jul 25, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant