[wheel] Split up test image provisioning - #24925
Conversation
Split the provisioning of a test image into two stages, where we save everything not strictly Python-related until the latter stage. Under `--tag-stages` we track the tags used during the build; otherwise, we rely on Docker's caching of each stage. This prepares for a future commit to add testing for a single wheel on multiple Python versions by making said workflow more efficient.
tyler-yankee
added
the
release notes: none
|
@drake-jenkins-bot mac-arm-sequoia-clang-wheel-experimental-release please. |
tyler-yankee
left a comment
tyler-yankee
left a comment
There was a problem hiding this comment.
+a:@mwoehlke-kitware for feature review, please.
@tyler-yankee partially reviewed 4 files and all commit messages, and made 1 comment.
Reviewable status: LGTM missing from assignee mwoehlke-kitware, needs platform reviewer assigned, needs at least two assigned reviewers (waiting on mwoehlke-kitware).
mwoehlke-kitware
left a comment
mwoehlke-kitware
left a comment
There was a problem hiding this comment.
@mwoehlke-kitware partially reviewed 3 files and made 4 comments.
Reviewable status: 4 unresolved discussions, LGTM missing from assignee mwoehlke-kitware, needs platform reviewer assigned, needs at least two assigned reviewers (waiting on tyler-yankee).
tools/wheel/wheel_builder/linux.py line 30 at r1 (raw file):
_files_to_remove = set() _images_to_remove = set() _built_test_bases = set()
BTW, I rather think this isn't worth the added complexity. Sure, we'll "rebuild" some stages (which we do anyway when not tagging stages), but caching should make the difference minor, and TBH performance is not of utmost concern when tagging stages anyway, since that's a debugging mechanism.
tools/wheel/wheel_builder/linux.py line 259 at r1 (raw file):
""" platform = target.platform(TEST, test_index) manager = platform.python_manager.value
minor
Suggestion:
python_managertools/wheel/wheel_builder/linux.py line 262 at r1 (raw file):
if tag_prefix == "base": return f"{tag_base}:{tag_prefix}-{platform.alias}-{manager}" python_tag = target.python.tag
BTW, python_details here might be sensible for consistency.
tools/wheel/wheel_builder/linux.py line 390 at r1 (raw file):
provisioned_image = test_image _build_stage(provisioned_image, args, test_dir) _images_to_remove.add(provisioned_image)
Shouldn't this only happen when not tagging stages?
2 participants
Split the provisioning of a test image into two stages, where we save everything not strictly Python-related until the latter stage. Under
--tag-stageswe track the tags used during the build; otherwise, we rely on Docker's caching of each stage.Towards #24903, #24854.
This change is