Skip to content

[Build] Add Spark source artifact cache workflow - #35

Merged
murali-db merged 2 commits into
masterfrom
stack/spark-source-cache
May 29, 2026
Merged

murali-db merged 2 commits into
masterfrom
stack/spark-source-cache

Conversation

@murali-db

@murali-db murali-db commented May 29, 2026 •

Copy link
Copy Markdown
Owner

Stacked PR

Use this link to review all changes.

Stack:

Summary

Review feedback addressed from delta-io#6898

  • Uses spark-source-ref naming and accepts arbitrary Spark refs while resolving to a deterministic SHA.
  • Reads the base artifact version from Spark source when possible.
  • Uses shallow Spark fetches.

Test plan

  • bash -n project/scripts/build_spark_from_source.sh
  • Parsed .github/workflows/spark_commit_artifacts.yaml with PyYAML.
  • Verified changed files contain no internal Maven/proxy references.

murali-db added 2 commits May 29, 2026 09:48
Add a manual workflow and helper script that publish a selected Apache Spark source ref to the runner's local Maven cache under a deterministic commit-qualified version.
Keep the cache producer helper focused on the generic source-ref implementation so the workflow does not retain the old commit-only path.
@murali-db
murali-db merged commit 84ec2fc into master May 29, 2026
1 check passed
murali-db added a commit that referenced this pull request Jun 2, 2026
* [Build] Add Spark source artifact cache workflow

Add a manual workflow and helper script that publish a selected Apache Spark source ref to the runner's local Maven cache under a deterministic commit-qualified version.

* [Build] Remove duplicate Spark source helper body

Keep the cache producer helper focused on the generic source-ref implementation so the workflow does not retain the old commit-only path.
murali-db added a commit that referenced this pull request Jun 22, 2026
* [Build] Add Spark source artifact cache workflow

Add a manual workflow and helper script that publish a selected Apache Spark source ref to the runner's local Maven cache under a deterministic commit-qualified version.

* [Build] Remove duplicate Spark source helper body

Keep the cache producer helper focused on the generic source-ref implementation so the workflow does not retain the old commit-only path.
murali-db added a commit that referenced this pull request Jul 15, 2026
* [Build] Add Spark source artifact cache workflow

Add a manual workflow and helper script that publish a selected Apache Spark source ref to the runner's local Maven cache under a deterministic commit-qualified version.

* [Build] Remove duplicate Spark source helper body

Keep the cache producer helper focused on the generic source-ref implementation so the workflow does not retain the old commit-only path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant