Skip to content

build: fit the OSS Scanner image into the build budget - #140

Merged
oschaaf merged 1 commit into
masterfrom
build/oss-scanner-build-budget
Oct 21, 2026
Merged

oschaaf merged 1 commit into
masterfrom
build/oss-scanner-build-budget

Conversation

@oschaaf

@oschaaf oschaaf commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Problem

The OSS Scanner builds .oss-scanner/Dockerfile with a 45-minute limit on a 16-core machine, and its enrolment check builds it on a 4-core runner with 40 minutes. On the 4-core runner the image reached 4,589 of 4,674 Bazel actions when the limit cut it off. The time went to three places: the exec configuration (protoc with every language generator, plus the in-repo code generators, all compiled at -O2), the libaom CMake build (serial at the time; #138 made it parallel), and compiling and running two unit-test suites inside the image.

#138 removed most of the in-repo code generator's dependencies from the exec configuration. This change takes the rest from the image's own settings.

Change

.oss-scanner/Dockerfile, in user.bazelrc and the build layer:

  • --host_compilation_mode=fastbuild: the exec configuration is built at -O0. The generators and protoc are built once and run a handful of times, so optimising them buys nothing.
  • --host_action_env=PATH=... next to the existing --action_env=PATH=...: the exec configuration's actions get the pinned PATH too, so the "pinned action PATH" comment is true for both configurations.
  • --copt=-gline-tables-only instead of --copt=-g: backtraces keep file and line; the module shrinks from 175 MB to 72 MB and links faster.
  • The image no longer builds or runs //test/pagespeed/kernel/base/... and //test/pagespeed/kernel/html/.... The engine objects stay in the cache, so any unit test still compiles and links offline inside the image.

.oss-scanner/threat_model.md: says which suites to run first (the first one also compiles googletest), that a different --output_base or a copy of the checkout cannot be used offline (the @cyclone and @aom_src git dependencies would be refetched), and how to add full debug info to the files under investigation with --per_file_copt rather than a global --copt=-g that would replace the cached build.

The libaom, curl, libevent and libmemcached CMake builds already run with --parallel 8 since #138, so the image sets nothing for them. An earlier revision of this branch set CMAKE_BUILD_PARALLEL_LEVEL=4 as well; an explicit --parallel overrides it, so it was inert and is gone.

Verification

  • tools/ci/check-public-hygiene.sh passes.
  • A 4-job, 4-CPU rebuild inside the previous image with these flags completed (libmod_pagespeed.so 72 MB, ngx_pagespeed_module.so 72 MB).
  • The scanner's own tools/check --no-shell against this branch, on a loaded many-core machine: the Bazel layer built 3,504 actions in 626 s (critical path 510 s), against 4,674 actions in 1,461 s for the previous image on the same machine. The whole tools/check run (clone, image build, scanner layer) passed in 1,758 s with the machine at load average 60 to 140 from other work; 836 s of that was Docker's layer export, which took 260 to 430 s in earlier runs on a quieter box. Verdict line from the scanner: "builds". The image's Bazel cache is 2.0 GB (was 2.6 GB).
  • An independent review of the branch found no build-breaking issue and asked for the wording fixes that are in the current revision.
  • The scanner's enrolment check (Enrol projects/mod_pagespeed anthropics/oss-scanner#957) re-runs from master once this is merged.

Checklist

  • Build and security-review configuration only; no product code changed.
  • Release-Note: none in the commit body.
  • Public hygiene gate passes.

The scanner builds .oss-scanner/Dockerfile on a 16-core machine with a
45-minute limit, and its pull-request check builds it on a 4-core runner with
40 minutes. On the 4-core runner the image reached 4,589 of 4,674 actions at
the cutoff. Most of the time went to Bazel's exec configuration (protoc and the
in-repo code generators, compiled at -O2) and to compiling and running two
unit-test suites inside the image.

The image now builds the exec configuration at -O0, compiles with line-table
debug info instead of full -g (the module shrinks from 175 MB to 72 MB and
links faster; backtraces keep file and line), pins PATH for the exec
configuration as well as the target one, and no longer builds or runs the
kernel/base and kernel/html test suites. The engine objects stay cached, so
any unit test still builds offline inside the image. The threat model says
which suites to start with, that a different output base or a copy of the
checkout cannot be used offline (the git dependencies would be refetched),
and how to add full debug info to the files under investigation without
replacing the cached build.

Release-Note: none - build and security-review configuration only, no product change.
@oschaaf
oschaaf force-pushed the build/oss-scanner-build-budget branch from 9b87328 to 392e46a Compare October 21, 2026 11:19
@oschaaf
oschaaf enabled auto-merge October 21, 2026 11:20
@oschaaf
oschaaf added this pull request to the merge queue Oct 21, 2026
Merged via the queue into master with commit 4acdc23 Oct 21, 2026
36 checks passed
@oschaaf
oschaaf deleted the build/oss-scanner-build-budget branch October 21, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant