Skip to content

Replace local repository metadata generation with SOW #769

Description

@Vonng

Background

Pigsty currently has more than one implementation for building the flat local/offline package repository:

  • the Ansible repo / cache roles use createrepo_c for RPM metadata and dpkg-scanpackages from dpkg-dev for DEB metadata;
  • pig repo create carries another repository-generation path and legacy fallback logic.

This duplicates format-sensitive logic and makes the bootstrap, Ansible, and PIG paths easy to drift apart. SOW 0.2.0 is now packaged in the Pigsty INFRA repository for both RPM and DEB platforms, so it can become the single repository builder.

Proposal

Use SOW as the only implementation for creating Pigsty local repositories:

sow create --pigsty --timeout 10m -- /www/pigsty

The migration should:

  • enable the infra upstream module for repository builders before installing SOW;
  • install sow during bootstrap/repository preparation;
  • make the Ansible repo_create and cache_create paths invoke SOW and fail if repository creation or the completion marker fails;
  • make pig repo create delegate directly to SOW;
  • remove the runtime dependencies on dpkg-dev, createrepo_c, and modulemd-tools;
  • retain dnf-utils / yum-utils, which are still needed for package downloading rather than metadata generation.

Compatibility decisions

SOW output must preserve the package-manager contract of the existing flat repository:

  • RPM: valid repodata/repomd.xml and referenced metadata, usable by DNF/YUM;
  • DEB: valid Packages and Packages.gz, usable by APT;
  • Pigsty: a repo_complete marker created only after successful metadata generation.

Additional decisions:

  • SOW writes SHA-256 entries to repo_complete; current Pigsty consumers only test marker existence, so this is compatible.
  • Do not synthesize ModuleMD for the local repository. Pigsty already uses module_hotfixes=1 for the local RPM repository, so package visibility does not depend on generated module metadata.
  • EL7 compatibility is out of scope for this migration.
  • Existing offline archives/local repositories built before SOW was bundled must be regenerated, or SOW must be installed before trying to rebuild them.

Prototype validation

The replacement path has been exercised on disposable Pigsty lab nodes:

  • Rocky Linux 9: Ansible repository creation succeeds; DNF can access and use the generated local repository without createrepo_c.
  • Rocky Linux 10: same validation, including current RPM metadata/compression handling.
  • Debian 13: Ansible repository creation succeeds; APT can access and use the generated local repository without dpkg-dev.
  • Ubuntu 22.04: same APT validation.
  • repo_create and cache_create check-mode guards pass.
  • PIG repository/ext unit tests and go vet pass.
  • SOW plain/v2 CLI tests and real RPM/DEB smoke tests pass.

Completion checklist

  • Merge Pigsty bootstrap, package-map, Ansible role, and documentation changes.
  • Merge the PIG delegation and catalog changes.
  • Rebuild and publish offline archives containing SOW.
  • Verify one released offline archive end-to-end after publication.

Metadata

Metadata

Assignees

Labels

INFRAInfrastructure Related

Projects

Status
Doing

Relationships

None yet

Development

No branches or pull requests

Issue actions