Replies: 2 comments 6 replies
-
|
It's a good idea, but the current approach of generating and storing binaries for every board is hugely wasteful. The firmware is the same on all boards, only the default config string is different. We should fix this first: generate a single / a few generic builds, then inject the appropriate config string on demand |
Beta Was this translation helpful? Give feedback.
6 replies
-
|
@romasku thoughts? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
First of all — a huge thank you to everyone who has contributed to this project. Being able to keep using existing wall switches to control smart lighting — without rewiring, without replacing hardware, and with full Z2M/ZHA integration — is genuinely impressive work and has added enormous value to a lot of home automation setups, including mine. I want to help make the project even better and easier to contribute to, which is the motivation behind this proposal.
Problem Statement
Currently, firmware binaries are committed and stored directly in the repository. This approach has several obvious drawbacks:
bin/in git history.Command used
bin/directory alone uses 144 MB of disk space; the.gitfolder is 137 MB.Commands used
Proposed Solution
Stop committing firmware binaries to the repository. Instead, use ORAS (OCI Registry As Storage) to push firmware as artifacts to GitHub Container Registry (
ghcr.io) on each tagged release. ORAS is a small CLI tool (also available as a GitHub Action) that lets you push and pull arbitrary files to/from an OCI registry, without needing Docker or containers.This is not a novel approach — it is exactly how Homebrew distributes its prebuilt packages: each formula's bottle is stored as an OCI artifact on
ghcr.io, with 9,000+ packages and no limits.Every build uploads a single zip of all firmware files (
.zigbee,.bin,.s37) as a GitHub Actions artifact (retained 90 days). This is intended for development and testing: contributors can download the zip, extract the file for their device, and flash it directly.On tagged release, the
.zigbeefiles are pushed as OCI artifacts toghcr.io. One package per board, one tag per role+variant. For example,MODULE_AVATTO_TS0011would get a package atghcr.io/OWNER/tuya-zigbee-switch/MODULE_AVATTO_TS0011with tags like:v1.2.3-router,v1.2.3-router-forced,v1.2.3-router-from_tuyav1.2.3-end_device,v1.2.3-end_device-forced,v1.2.3-end_device-from_tuyaAll tags are visible on the package page. Packages are public, free, and retained forever.
The OTA index is updated and committed to the repo. It contains direct HTTPS blob URLs pointing to the
.zigbeeartifacts onghcr.io, which Z2M/ZHA use to push firmware over-the-air. No authentication or special tooling required.Implementation Notes
Package and tag structure
Each board gets one OCI package (e.g.
MODULE_AVATTO_TS0011). Only.zigbeefiles are pushed — one tag per role+variant combination:Other file formats (
.bin,.s37) are only available via the GitHub Actions artifact zip, which is sufficient for the direct-flashing use case (development/testing). Restricting OCI to.zigbeekeeps the package clean and avoids UX confusion — users cannot easily identify or download individual layers from the GitHub packages UI.Pushing uses
GITHUB_TOKENin the CI workflow — no extra secrets needed.OTA index URL construction
oras pushoutputs the digest of each pushed layer, so the CI script captures the.zigbeedigest at push time and constructs the blob URL directly:The human-readable identity (board name, variant, version) is carried by the
fileNamefield in the OTA index alongside theurl— consumers like Z2M never need to parse the URL itself.Note: OCI blob URLs are always content-addressed (
blobs/sha256:DIGEST) — this is fundamental to the storage model and unaffected by package or tag naming. Tags appear on the package page for easy browsing but resolve to a manifest, not the file directly.Benefits
ghcr.ioare free and are not auto-deleted.Request for Feedback
Would maintainers and contributors support this change? Are there use cases that require binaries to be present in the repo? Any concerns about using
ghcr.ioas the firmware store?Alternatives Considered
GitHub Release Assets
Publishing firmware as GitHub Release assets was the first alternative considered. It requires no extra tooling and is easy to understand. However, GitHub limits the number of assets per release to 1,000, and we are already at ~660 firmware files per release (router + end_device variants for all boards). This will not scale as support for more devices is added.
Annex: What Is an OCI Registry?
An OCI Registry is a standardized server for storing and distributing software artifacts.
You may know it as the technology behind Docker images, but modern OCI registries support storing
any file — not just container images. Think of it as a content-addressable, versioned file store
with a well-known HTTP API.
GitHub Container Registry (
ghcr.io) is GitHub's OCI registry. It is:This is exactly how Homebrew stores its prebuilt binary packages ("bottles"):
each formula's bottle is stored as a separate OCI package on
ghcr.io(e.g.,
ghcr.io/homebrew/core/sqlite,ghcr.io/homebrew/core/openssl, etc.),with thousands of packages and no limits.
Beta Was this translation helpful? Give feedback.
All reactions