Repository navigation
build: fit the OSS Scanner image into the build budget - #140
Merged
Merged
Conversation
The scanner builds .oss-scanner/Dockerfile on a 16-core machine with a 45-minute limit, and its pull-request check builds it on a 4-core runner with 40 minutes. On the 4-core runner the image reached 4,589 of 4,674 actions at the cutoff. Most of the time went to Bazel's exec configuration (protoc and the in-repo code generators, compiled at -O2) and to compiling and running two unit-test suites inside the image. The image now builds the exec configuration at -O0, compiles with line-table debug info instead of full -g (the module shrinks from 175 MB to 72 MB and links faster; backtraces keep file and line), pins PATH for the exec configuration as well as the target one, and no longer builds or runs the kernel/base and kernel/html test suites. The engine objects stay cached, so any unit test still builds offline inside the image. The threat model says which suites to start with, that a different output base or a copy of the checkout cannot be used offline (the git dependencies would be refetched), and how to add full debug info to the files under investigation without replacing the cached build. Release-Note: none - build and security-review configuration only, no product change.
oschaaf
force-pushed
the
build/oss-scanner-build-budget
branch
from
October 21, 2026 11:19
9b87328 to
392e46a
Compare
oschaaf
enabled auto-merge
October 21, 2026 11:20
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The OSS Scanner builds
.oss-scanner/Dockerfilewith a 45-minute limit on a 16-core machine, and its enrolment check builds it on a 4-core runner with 40 minutes. On the 4-core runner the image reached 4,589 of 4,674 Bazel actions when the limit cut it off. The time went to three places: the exec configuration (protoc with every language generator, plus the in-repo code generators, all compiled at-O2), the libaom CMake build (serial at the time; #138 made it parallel), and compiling and running two unit-test suites inside the image.#138 removed most of the in-repo code generator's dependencies from the exec configuration. This change takes the rest from the image's own settings.
Change
.oss-scanner/Dockerfile, inuser.bazelrcand the build layer:--host_compilation_mode=fastbuild: the exec configuration is built at-O0. The generators and protoc are built once and run a handful of times, so optimising them buys nothing.--host_action_env=PATH=...next to the existing--action_env=PATH=...: the exec configuration's actions get the pinned PATH too, so the "pinned action PATH" comment is true for both configurations.--copt=-gline-tables-onlyinstead of--copt=-g: backtraces keep file and line; the module shrinks from 175 MB to 72 MB and links faster.//test/pagespeed/kernel/base/...and//test/pagespeed/kernel/html/.... The engine objects stay in the cache, so any unit test still compiles and links offline inside the image..oss-scanner/threat_model.md: says which suites to run first (the first one also compiles googletest), that a different--output_baseor a copy of the checkout cannot be used offline (the@cycloneand@aom_srcgit dependencies would be refetched), and how to add full debug info to the files under investigation with--per_file_coptrather than a global--copt=-gthat would replace the cached build.The libaom, curl, libevent and libmemcached CMake builds already run with
--parallel 8since #138, so the image sets nothing for them. An earlier revision of this branch setCMAKE_BUILD_PARALLEL_LEVEL=4as well; an explicit--paralleloverrides it, so it was inert and is gone.Verification
tools/ci/check-public-hygiene.shpasses.libmod_pagespeed.so72 MB,ngx_pagespeed_module.so72 MB).tools/check --no-shellagainst this branch, on a loaded many-core machine: the Bazel layer built 3,504 actions in 626 s (critical path 510 s), against 4,674 actions in 1,461 s for the previous image on the same machine. The wholetools/checkrun (clone, image build, scanner layer) passed in 1,758 s with the machine at load average 60 to 140 from other work; 836 s of that was Docker's layer export, which took 260 to 430 s in earlier runs on a quieter box. Verdict line from the scanner: "builds". The image's Bazel cache is 2.0 GB (was 2.6 GB).masteronce this is merged.Checklist
Release-Note: nonein the commit body.