Skip to content

test/goroot: cover compile-only, errorcheck, and remaining runnable Go 1.26 cases #2062

Description

@cpunion

Status

Complete Go 1.26 GOROOT coverage is implemented on the current-main candidate and validated with Go 1.26.5 on both target platforms.

The runner discovers every active compile, errorcheck, errorcheckandrundir, run, runoutput, rundir, runindir, buildrun, and buildrundir case in the configured GOROOT test directories. Compile-only and diagnostic tests execute rather than being inferred or omitted.

Current-main Go 1.26.5 inventory and results

pass and fail describe the observed case result. A diagnostic test passes only when LLGo's diagnostics satisfy its // ERROR expectations. Xfails and failed flakes remain executed failures in this inventory.

The table starts from the complete current-main sweep and applies the individually verified former-skip delta: six cases moved from skip to pass, while the executed Darwin chanlinear.go result moved from skip to failed flake. It is an exact inventory/result reconciliation, not a second full sweep.

Directive Total Pass Fail Skip
buildrun 1 1 0 0
buildrundir 2 0 2 0
compile 510 499 11 0
errorcheck 650 472 178 0
errorcheckandrundir 5 1 4 0
run 1,030 982 48 0
rundir 116 110 6 0
runindir 9 6 3 0
runoutput 22 20 2 0
Total 2,345 2,091 254 0

Overall classification:

  • 2,087 other cases: passed
  • 246 xfails: failed as expected
  • 7 flakes: 4 passed and 3 failed
  • 5 resource-guard stops: bounded hard failures from the complete current-main sweep
  • 0 host skips for Go 1.26 on either target platform
  • 0 unexpected unclassified failures
  • 2,345 / 2,345 represented and executed (100%) across the complete sweep plus targeted former-skip verification

Failure conservation is 246 xfails + 3 failed flakes + 5 resource stops = 254 observed failures. Xfails remain part of coverage: each case is compiled, run, or diagnostic-matched before classification.

Final current-main flake outcomes on darwin/arm64:

  • Passed: rotate0.go, fixedbugs/issue79186.go, fixedbugs/issue43942.go, fixedbugs/issue48898.go
  • Failed: goprint.go, stack.go, chanlinear.go

Current main vs merged-PR comparison

The merged-PR comparison is preserved as data; it has not been rebased over the current-main candidate. Delta is merged PRs minus current main.

Directive Total Merged pass Main pass Delta pass Merged fail Main fail Delta fail Merged skip Main skip
buildrun 1 1 1 0 0 0 0 0 0
buildrundir 2 0 0 0 2 2 0 0 0
compile 510 499 499 0 11 11 0 0 0
errorcheck 650 470 472 -2 180 178 +2 0 0
errorcheckandrundir 5 1 1 0 4 4 0 0 0
run 1,030 984 982 +2 39 48 -9 7 0
rundir 116 112 110 +2 4 6 -2 0 0
runindir 9 8 6 +2 1 3 -2 0 0
runoutput 22 21 20 +1 1 2 -1 0 0
Total 2,345 2,096 2,091 +5 242 254 -12 7 0

The two extra merged-branch errorcheck failures are fixedbugs/bug299.go and fixedbugs/issue5089.go, both introduced by PR #2031. Detailed attribution and the complete failure-reason taxonomy are in this issue update.

Former host-skip validation

Commit d6cfed406 removes all Go 1.26 host skips while preserving the Go 1.24/1.25 protections. Unversioned Darwin rules were narrowed to Go 1.24/1.25 instead of being deleted globally.

Case darwin/arm64 linux/amd64
ddd.go pass pass
gcgort.go pass pass
fixedbugs/issue25897b.go pass pass
fixedbugs/issue27695.go pass pass
fixedbugs/issue39541.go pass pass
chan/goroutines.go pass pass
chanlinear.go executed; known timing flake pass

Darwin local validation selected all 7 cases and reported 7 passing subtests with no skips. Six passed cleanly; chanlinear.go reproduced the LLGo chanSelect linearity timing failure and is now an executed flake instead of a host skip. Peak process-group RSS was about 921 MiB under a 15 GiB hard limit.

Linux was validated on a native GitHub ubuntu-latest amd64 runner with Go 1.26.5. All 7 cases passed cleanly with no skips; peak RSS was about 982 MiB under a 4 GiB hard limit: Actions job. The optional hosted macOS duplicate was canceled after waiting 25 minutes for a runner; local darwin/arm64 validation is the authoritative Darwin result.

Go 1.26.5 delta

Go 1.26.0 contained 2,332 matching cases. Go 1.26.5 contains 2,345, adding 13:

Directive Go 1.26.0 Go 1.26.5 Delta
compile 504 510 +6
run 1,023 1,030 +7
Other directives 805 805 0
Total 2,332 2,345 +13

All 13 Go 1.26.5 additions passed LLGo validation.

Failure classification detail

All 254 current-main failures have a technical classification:

  • buildrundir: 2 Go assembler asmhdr/symabis and return-jump ABI gaps
  • compile: 10 unsupported gc/backend flags and 1 RSS resource stop
  • errorcheck: 177 diagnostic xfails and 1 RSS resource stop
  • errorcheckandrundir: 4 diagnostic xfails; the mandatory build/run stage is still executed
  • rundir / runindir: 9 expected directory-case failures
  • run: 44 xfails, 3 failed flakes, and 1 resource stop
  • runoutput: 2 resource stops

The detailed taxonomy separates gc-only compiler analysis/backend/runtime diagnostics from LLGo frontend wording/recovery/position differences and groups runnable failures by GC/liveness, panic/recover, stack metadata, compile/link, interface/reflection, and scheduler behavior. See the linked taxonomy comment above for counts and case boundaries.

Implemented coverage

  • Added coverage discovery mode with recursive nested-case discovery.
  • Added -directives, -list-cases, and -list-case-paths for reproducible inventory and sharding.
  • Added llgo tool compile with Go compiler-compatible flag names and value forms used by cmd/internal/testdir, including -c, -C, -e, -N, -l, -lang, -complete, -d, -p, -importcfg, -D, -I, and -o.
  • Propagated primary-package language versions and per-file versions through package loading.
  • Added compile-only handling for declarations without bodies.
  • Implemented compile and errorcheck execution with upstream-style compiler options and // ERROR matching.
  • Implemented errorcheckandrundir as diagnostic checking plus a mandatory build/run stage.
  • Implemented companion-directory staging for rundir, runindir, and buildrundir, including _test.go, relative imports, assembly placeholders, and nested module layouts.
  • Added recursive nested run cases and official run working-directory/environment behavior.
  • Preserved non-zero process exits and timeout/resource errors through xfail classification.

Resource controls

The complete current-main sweep used serial execution with GOMAXPROCS=2, go test -p=1, and a 1,536 MiB process-group RSS hard limit. It intentionally recorded five bounded resource stops rather than allowing xfail entries to hide them.

The former-skip validation used:

  • Local Darwin: GOMEMLIMIT=12GiB, 15 GiB RSS hard limit, 10 ms RSS polling, minimum 40% free memory, and minimum 1,536 MiB free swap.
  • Native Linux CI: GOMEMLIMIT=3GiB, 4 GiB RSS hard limit, 25 ms RSS polling, and GOMAXPROCS=2.

The runner:

  • checks physical-memory and swap thresholds before and during each command;
  • samples total RSS for the complete Unix process group;
  • kills the process group when the hard RSS limit is exceeded;
  • reports high-water RSS warnings;
  • treats resource-guard stops as hard failures that xfail entries cannot hide;
  • defaults the helper script to GOMAXPROCS=2 and go test -p=1.

These guards prevent recurrence of the unbounded LLGo coverage build that previously reached approximately 32.5 GiB RSS and exhausted swap.

Reproduction

Inventory only:

GOMAXPROCS=2 go test -p=1 ./test/goroot -run '^TestGoRootRunCases$' -count=1 -args \
  -goroot "$(go env GOROOT)" \
  -directive-mode coverage \
  -list-cases

Resource-constrained execution:

GOMAXPROCS=2 GOMEMLIMIT=12GiB go test -p=1 ./test/goroot -run '^TestGoRootRunCases$' -count=1 -timeout=180m -args \
  -goroot "$(go env GOROOT)" \
  -directive-mode coverage \
  -max-rss-mib 15360 \
  -rss-warn-mib 512 \
  -min-memory-free-percent 40 \
  -min-swap-free-mib 1536 \
  -rss-poll 10ms

Acceptance criteria

  • All 510 active Go 1.26.5 compile cases are discovered and classified.
  • All 650 errorcheck cases execute diagnostic matching and are classified.
  • All 5 errorcheckandrundir cases execute diagnostic and run stages.
  • All 1,185 runnable cases are discovered and classified.
  • All 2,345 active cases are represented in the inventory and result totals.
  • Every Go 1.26 case has been executed; no target-platform host skip remains.
  • Former host skips are validated on darwin/arm64 and linux/amd64 under bounded resources.
  • Compile-only success does not require a linked executable.
  • Resource use is bounded independently of xfail handling.

Pull request structure and CI status

The prerequisite implementation now uses two coherent review units:

The earlier fine-grained PRs #2069, #2070, and #2071 are closed as superseded by #2066. Their branches remain preserved. Merge #2068 and #2066 independently, then rebase draft #2065 onto main.

The previous CLI-only #2066 head is preserved as codex/go-tool-compile-cli-only; the consolidated implementation is mirrored at codex/go-tool-compile-action-cli-full.

Bounded local validation passes with Go 1.24.11 and Go 1.26.5 for both ordinary Go tests and real llgo test ./test/go runs. The #2068 regression fails on unmodified main with the expected unsafe.Offsetof panic. The #2066 compatibility test found and now covers multiline -C driver diagnostics. #2066 and #2068 each completed with 47 passing checks, 1 expected release skip, and 100.00% Codecov patch coverage.

Independent #2072 remains the select-output CI stability fix; its prior complete run had 47 passing checks, 1 expected release skip, and no product coverage change.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions