doc update

.github/RELEASE_ANNOUNCEMENT.md

@@ -2,7 +2,9 @@
 
 We're pleased to announce **ExZarr v1.2.0**: Zarr 3.1 conformance, standard
 `sharding_indexed` interoperability with zarr-python, optional byte-range cloud
-I/O, AzureSDK migration, and **precompiled Zig codec NIFs** for Hex/Livebook.
+I/O, AzureSDK migration, and compression codecs from
+[ExCodecs](https://hex.pm/packages/ex_codecs): precompiled pure-Rust NIFs, so
+ExZarr has no native code and needs no Zig, compiler or system libraries.
 
 ## Highlights
 
@@ -15,16 +17,21 @@ I/O, AzureSDK migration, and **precompiled Zig codec NIFs** for Hex/Livebook.
 
 ### Standard sharding + range reads
 
-- Spec-correct `sharding_indexed` v1.0 (dense uint64 index, empty sentinel)
+- Spec-correct `sharding_indexed` v1.0 (dense uint64 index, empty sentinel),
+  readable and writable by zarr-python
 - Optional `chunk_info` / `read_chunk_range` backend capabilities
 - Range-aware inner-chunk reads with full-shard fallback
 - S3, GCS, Azure, filesystem, and memory backends
 
-### Precompiled Zig codec NIFs
+### Codecs via ExCodecs
 
-- Hex/Livebook installs download platform NIFs (no Zig toolchain required)
-- Source builds via `EX_ZARR_BUILD=1` or when checksums are missing
-- See [docs/PRECOMPILATION.md](../docs/PRECOMPILATION.md)
+- zstd, lz4, snappy, blosc, bzip2 and crc32c from ExCodecs, precompiled for
+  macOS, Linux (glibc and musl) and Windows; zlib/gzip via Erlang `:zlib`
+- numcodecs-compatible byte formats: zarr-python reads ExZarr's lz4, bzip2
+  and blosc data, and ExZarr reads zarr-python's
+- New `compressor_config:` option (e.g. `[level: 9]`, or Blosc `cname`,
+  `shuffle`, `typesize`), stored in `.zarray` / `zarr.json`
+- `config :ex_zarr, default_zarr_version: 3` now applies to new arrays
 
 ### AzureSDK migration
 
@@ -58,10 +65,16 @@ end
 
 ## Upgrade notes
 
+- OTP 26 or later is required
+- Drop any Zig / system compression-library setup; it is no longer used
 - Azure users: add `azure_sdk`, remove `azurex`
 - Rewrite arrays written with ExZarr’s pre-1.2 private shard index
 - Cloud v3 arrays use `zarr.json` and `c/...` keys
 - GCS/Azure HTTP client: `{:req, "~> 0.6.1"}` (CVE-2026-49755)
+- New lz4/bzip2 data uses the numcodecs formats; 1.2 reads 1.1 data, but 1.1
+  cannot read lz4/bzip2 data written by 1.2
+- `ExZarr.create/1` returns errors for unknown compressors, invalid
+  `compressor_config` and unsupported Zarr versions
 
 ## Known limitations
 
@@ -79,4 +92,4 @@ end
 
 ## Full changelog
 
-See [CHANGELOG.md](../CHANGELOG.md#120---2026-10-03).
+See [CHANGELOG.md](../CHANGELOG.md#120---2026-10-04).

CHANGELOG.md

@@ -7,65 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-### Added
-- `compressor_config:` option for `ExZarr.create/1` (e.g. `[level: 9]`; Blosc
-  also `cname`, `shuffle`, `typesize`), validated per compressor by the new
-  `ExZarr.Codecs.CompressorConfig`. v2 arrays store the settings in `.zarray`
-  in numcodecs form and reuse them after reopening (including arrays created
-  by zarr-python); v3 arrays store them in the codec configuration. The
-  option was documented before but never implemented.
-- `compressor: :gzip` for Zarr v3 arrays.
-
-### Fixed
-- `config :ex_zarr, default_zarr_version` is now applied by `ExZarr.create/1`
-  and `ExZarr.Nx`. Unset, new arrays remain v2; `ExZarr.Version` previously
-  documented (and returned) 3 although nothing used it.
-- `ExZarr.create/1` rejects unknown compressors, invalid compressor settings
-  and Zarr versions other than 2 or 3 with an error tuple. Previously an
-  unknown compressor failed on the first write and an invalid version raised.
-- Zarr v3 arrays created with a compressor that has no v3 codec (`:snappy`,
-  custom codecs without `codecs:`) were silently stored uncompressed; they now
-  return `{:error, {:unsupported_codec_for_v3, compressor}}`.
-- `:zlib` honours a compression `level`.
-- v2 `.zarray` compressor levels now record the level ExZarr actually uses
-  (zlib 6, zstd 3) instead of a fixed 5.
-
-### Changed
-- Compression codecs (zstd, lz4, snappy, blosc, bzip2, crc32c) now come from
-  [ExCodecs](https://hex.pm/packages/ex_codecs) `~> 0.2.4`: pure-Rust NIFs
-  with precompiled binaries for macOS, Linux (glibc and musl) and Windows.
-  ExZarr no longer contains native code; Zig, zigler, zigler_precompiled and
-  system compression libraries are no longer needed.
-- `:lz4` writes the numcodecs `LZ4` format (4-byte little-endian size prefix)
-  and `:bzip2` writes plain bzip2 streams (numcodecs `BZ2`), so zarr-python can
-  read them. Data written by earlier ExZarr versions (8-byte size prefix) is
-  still read.
-- v2 `.zarray` compressor metadata uses numcodecs configurations: `lz4`
-  (`acceleration`), `blosc` (`cname`, `clevel`, `shuffle`, `blocksize`) and
-  `bz2` (previously `bzip2`). The old `bzip2` id is still read. zarr-python can
-  now open ExZarr v2 arrays compressed with lz4, blosc and bzip2.
-- The v3 `blosc` codec honours `cname`, `clevel`, `shuffle` and `typesize` from
-  its configuration; without them ExZarr keeps BloscLZ, byte shuffle,
-  typesize 1.
-- Decompressed chunks are capped at 256 MiB (`config :ex_zarr,
-  max_chunk_bytes: ...`) to guard against decompression bombs.
-- Minimum OTP is 26 (ExCodecs NIFs target NIF 2.17).
-- `ExZarr.Codecs.compress/3` accepts options as a map as well as a keyword list.
-
-### Removed
-- `ExZarr.Codecs.ZigCodecs`, `ExZarr.Codecs.CompressionConfig`, the
-  `fix_nif_rpaths` Mix task, the precompile workflow and
-  `docs/PRECOMPILATION.md` / `docs/COMPRESSION_SETUP.md`.
-- `COMPRESSION_LIB_DIRS`, `HOMEBREW_PREFIX`, `BZIP2_STATIC_LIB`, `ZIG_PATH` and
-  `EX_ZARR_BUILD` no longer have any effect.
-
-## [1.2.0] - 2026-10-03
+## [1.2.0] - 2026-10-04
 
 ### Zarr 3.1 Interoperability & Range-Aware Cloud I/O
 
 ### Added
-- Precompiled Zig NIFs for `ExZarr.Codecs.ZigCodecs` via ZiglerPrecompiled
-  (Linux/macOS x86_64 + aarch64); see `docs/PRECOMPILATION.md`
 - Optional storage backend callbacks: `chunk_info/2`, `read_chunk_range/4`,
   conditional `read_chunk_range/5` (`if_match:`), `capabilities/1`, `put_layout/2`
 - `ExZarr.Storage.ObjectKeys` for centralized v2/v3 object naming, including
@@ -82,6 +28,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   matrix (zarr-python 2.18.3 / 3.2.1 / 3.3.0 / 3.4.0)
 - Azurite-oriented Azure integration test (tagged `:azure`)
 - Showcase: `examples/range_aware_sharded_nx.exs` (run with `mix run`)
+- `compressor_config:` option for `ExZarr.create/1` (e.g. `[level: 9]`; Blosc
+  also `cname`, `shuffle`, `typesize`), validated per compressor by the new
+  `ExZarr.Codecs.CompressorConfig`. v2 arrays store the settings in `.zarray`
+  in numcodecs form and reuse them after reopening (including arrays created
+  by zarr-python); v3 arrays store them in the codec configuration. The
+  option was documented before but never implemented.
+- `compressor: :gzip` for Zarr v3 arrays.
 
 ### Fixed
 - `sharding_indexed` follows the v1.0 spec geometry (chunk grid = shard shape,
@@ -104,6 +57,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   files in the store
 - Scalar (`shape: {}`) and zero-length dimension support in MetadataV3 / Array
 - Dimension-name over-validation (spec strings/nil; uniqueness not required at metadata)
+- `config :ex_zarr, default_zarr_version` is now applied by `ExZarr.create/1`
+  and `ExZarr.Nx`. Unset, new arrays remain v2; `ExZarr.Version` previously
+  documented (and returned) 3 although nothing used it.
+- `ExZarr.create/1` rejects unknown compressors, invalid compressor settings
+  and Zarr versions other than 2 or 3 with an error tuple. Previously an
+  unknown compressor failed on the first write and an invalid version raised.
+- Zarr v3 arrays created with a compressor that has no v3 codec (`:snappy`,
+  custom codecs without `codecs:`) were silently stored uncompressed; they now
+  return `{:error, {:unsupported_codec_for_v3, compressor}}`.
+- `:zlib` honours a compression `level`.
+- v2 `.zarray` compressor levels now record the level ExZarr actually uses
+  (zlib 6, zstd 3) instead of a fixed 5.
 
 ### Changed
 - `create/1` with `shard_shape:` writes spec metadata: `shard_shape` is the
@@ -116,8 +81,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   in backend state
 - `Storage.chunk_info/2` returns `{:error, :unsupported}` for backends without
   `chunk_info/2` instead of downloading the object
-- Optional HTTP client is now `req ~> 0.6.1` (with `override: true` until
-  `azure_sdk` widens its `req ~> 0.5` constraint); `ex_aws` updated to 2.7.x
+- Optional HTTP client is now `req ~> 0.6.1` (`azure_sdk`'s `req ~> 0.5` already
+  allows 0.6); `ex_aws` updated to 2.7.x
+- Compression codecs (zstd, lz4, snappy, blosc, bzip2, crc32c) now come from
+  [ExCodecs](https://hex.pm/packages/ex_codecs) `~> 0.2.4`: pure-Rust NIFs
+  with precompiled binaries for macOS, Linux (glibc and musl) and Windows.
+  ExZarr no longer contains native code; Zig, zigler, zigler_precompiled and
+  system compression libraries are no longer needed.
+- `:lz4` writes the numcodecs `LZ4` format (4-byte little-endian size prefix)
+  and `:bzip2` writes plain bzip2 streams (numcodecs `BZ2`), so zarr-python can
+  read them. Data written by earlier ExZarr versions (8-byte size prefix) is
+  still read.
+- v2 `.zarray` compressor metadata uses numcodecs configurations: `lz4`
+  (`acceleration`), `blosc` (`cname`, `clevel`, `shuffle`, `blocksize`) and
+  `bz2` (previously `bzip2`). The old `bzip2` id is still read. zarr-python can
+  now open ExZarr v2 arrays compressed with lz4, blosc and bzip2.
+- The v3 `blosc` codec honours `cname`, `clevel`, `shuffle` and `typesize` from
+  its configuration; without them ExZarr keeps BloscLZ, byte shuffle,
+  typesize 1.
+- Decompressed chunks are capped at 256 MiB (`config :ex_zarr,
+  max_chunk_bytes: ...`) to guard against decompression bombs.
+- Minimum OTP is 26 (ExCodecs NIFs target NIF 2.17).
+- `ExZarr.Codecs.compress/3` accepts options as a map as well as a keyword list.
 
 ### Security
 - Bump `req` past CVE-2026-49755 (decompression-bomb DoS); GCS binary object
@@ -127,6 +112,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `azurex` dependency and Azurex-based adapter code
 - Unused optional `google_api_storage` (GCS uses Goth + Req directly; this also
   unblocked `mime` 2.x required by patched Req)
+- `ExZarr.Codecs.ZigCodecs`, `ExZarr.Codecs.CompressionConfig`, the
+  `fix_nif_rpaths` Mix task, the precompile workflow and
+  `docs/PRECOMPILATION.md` / `docs/COMPRESSION_SETUP.md`.
+- `COMPRESSION_LIB_DIRS`, `HOMEBREW_PREFIX`, `BZIP2_STATIC_LIB`, `ZIG_PATH` and
+  `EX_ZARR_BUILD` no longer have any effect.
 
 ### Known limitations
 - Storage transformers are not executed (metadata only)

README.md

@@ -872,9 +872,10 @@ Key modules:
 
 See [ROADMAP.md](docs/ROADMAP.md) for the full release plan.
 
-**v1.2.0 (released 2026-10-03)** - Zarr 3.1 interoperability & range-aware cloud I/O:
+**v1.2.0 (released 2026-10-04)** - Zarr 3.1 interoperability & range-aware cloud I/O:
 spec `sharding_indexed`, optional byte-range reads, version-aware cloud keys,
-AzureSDK migration, Python fixture/CI matrix.
+AzureSDK migration, codecs via ExCodecs (no native toolchain), `compressor_config`,
+Python fixture/CI matrix.
 
 **Upcoming** (high level):
 

docs/INTEROPERABILITY.md

@@ -392,7 +392,7 @@ then the same `mix test --include python …` commands.
 ### Notes
 
 - The `python` executable used by Mix tests must be on `PATH` (tests shell out to it).
-- Elixir/Zig/system libs are the same as a normal ExZarr test run (`mix deps.get`, Zig for NIFs).
+- Elixir setup is the same as a normal ExZarr test run (`mix deps.get`); no native toolchain is needed.
 - Without `--include python` / `--only python_fixtures`, these tests are skipped.
 - Fixture generator details: `test/support/python_fixtures/README.md`.
 

docs/ROADMAP.md

@@ -2,14 +2,16 @@
 
 ## v1.2.0 — Zarr 3.1 Interoperability & Range-Aware Cloud I/O
 
-Released 2026-10-03. Backward compatible with v1.1 (rewrite pre-1.2 private shards).
+Released 2026-10-04. Backward compatible with v1.1 (rewrite pre-1.2 private shards; OTP 26+).
 
 - [x] Zarr 3.1 metadata: scalars, zero-length dims, dimension names, extensions, `storage_transformers`
 - [x] Spec-correct `sharding_indexed` v1.0
 - [x] Optional byte-range storage (`chunk_info` / `read_chunk_range`)
 - [x] Range-aware sharded reads with full-shard fallback
 - [x] Version-aware cloud object keys (S3 / GCS / Azure)
 - [x] Azure Blob backend on `azure_sdk ~> 0.4.1` (azurex removed)
+- [x] Codecs via ExCodecs (precompiled pure-Rust NIFs; no Zig or system libraries)
+- [x] `compressor_config:` and `default_zarr_version` applied
 - [x] Python fixture generator + CI matrix job for zarr 2.x / 3.2 / 3.3 / 3.4
 
 ---

docs/release_notes_v1_2_0.md

@@ -4,8 +4,10 @@
 
 ExZarr v1.2.0 focuses on Zarr 3.1 conformance, standard `sharding_indexed`
 interoperability, optional byte-range cloud I/O, migrating Azure Blob
-storage to `azure_sdk`, and shipping **precompiled Zig codec NIFs** for
-Hex/Livebook installs (see `PRECOMPILATION.md`).
+storage to `azure_sdk`, and moving the compression codecs to
+[ExCodecs](https://hex.pm/packages/ex_codecs): precompiled pure-Rust NIFs, so
+ExZarr itself has no native code and needs no Zig, compiler or system
+libraries.
 
 ## Highlights
 
@@ -18,16 +20,36 @@ Hex/Livebook installs (see `PRECOMPILATION.md`).
 
 ### Standard sharding + range reads
 
-- `sharding_indexed` v1.0 dense uint64 index with empty sentinel
+- `sharding_indexed` v1.0 dense uint64 index with empty sentinel, readable and
+  writable by zarr-python
 - Optional `chunk_info` / `read_chunk_range` backend capabilities
 - Range-aware inner-chunk reads with full-shard fallback
 - S3, GCS, Azure, filesystem, and memory backends
 
-### Precompiled Zig codec NIFs
-
-- Hex/Livebook installs download platform NIFs (no Zig toolchain required)
-- Source builds via `EX_ZARR_BUILD=1` or missing checksum file
-- CI workflow: `.github/workflows/precompile.yml`
+### Codecs via ExCodecs
+
+- zstd, lz4, snappy, blosc, bzip2 and crc32c come from ExCodecs `~> 0.2.4`,
+  precompiled for macOS, Linux (glibc and musl) and Windows. zlib/gzip use
+  Erlang's `:zlib`.
+- Byte formats match numcodecs, so zarr-python reads ExZarr's lz4, bzip2 and
+  blosc data (v2 and v3), and ExZarr reads zarr-python's.
+- New `compressor_config:` option, for example:
+
+  ```elixir
+  ExZarr.create(
+    shape: {1000, 1000},
+    chunks: {100, 100},
+    compressor: :blosc,
+    compressor_config: [cname: :zstd, level: 5, shuffle: :bit, typesize: 8]
+  )
+  ```
+
+  v2 arrays keep the settings in `.zarray` (numcodecs form); v3 arrays keep
+  them in the codec configuration.
+- `config :ex_zarr, default_zarr_version: 3` now applies to new arrays
+  (unset: v2, as before).
+- Decompressed chunks are capped at 256 MiB by default
+  (`config :ex_zarr, max_chunk_bytes: ...`).
 
 ### AzureSDK migration
 
@@ -54,8 +76,8 @@ ExZarr.open(
 
 ### Python interoperability matrix
 
-Fixture generator under `test/support/python_fixtures/` and a CI job covering
-zarr-python 2.x / 3.2 / 3.3 / 3.4.
+Fixture generator under `test/support/python_fixtures/` and a blocking CI job
+covering zarr-python 2.18.3 / 3.2.1 / 3.3.0 / 3.4.0, in both directions.
 
 ### Security
 
@@ -64,15 +86,24 @@ zarr-python 2.x / 3.2 / 3.3 / 3.4.
 
 ## Upgrade
 
-- Azure users: add `{:azure_sdk, "~> 0.4.1", optional: true}` and remove `azurex`
-- Arrays written with ExZarr’s pre-1.2 private shard index must be rewritten
-- Cloud v3 arrays now use `zarr.json` and `c/...` keys
-- GCS/Azure users: use `{:req, "~> 0.6.1"}` (ExZarr overrides older azure_sdk caps)
+- **OTP 26 or later** is required (ExCodecs NIFs target NIF 2.17).
+- Remove any Zig, `COMPRESSION_LIB_DIRS`, `EX_ZARR_BUILD` or system
+  compression-library setup; it is no longer used.
+- Azure users: add `{:azure_sdk, "~> 0.4.1", optional: true}` and remove `azurex`.
+- GCS/Azure users: use `{:req, "~> 0.6.1"}`.
+- Arrays written with ExZarr's pre-1.2 private shard index must be rewritten.
+- Cloud v3 arrays now use `zarr.json` and `c/...` keys.
+- New lz4 and bzip2 data uses the numcodecs formats. ExZarr 1.2 still reads
+  data written by 1.1, but ExZarr 1.1 cannot read lz4/bzip2 data written by 1.2.
+- `ExZarr.create/1` now returns an error for unknown compressors, invalid
+  `compressor_config` and Zarr versions other than 2 or 3, and for v3
+  compressors without a v3 codec (such as `:snappy`), which were previously
+  stored uncompressed.
 
 ## Known limitations
 
 - No storage-transformer execution
 - No adaptive range coalescing
 - No partial writes inside cloud shards
 
-See [CHANGELOG.md](../CHANGELOG.md) and [ROADMAP.md](ROADMAP.md).
+See [CHANGELOG.md](https://github.com/AnimaLogica/ExZarr/blob/main/CHANGELOG.md#120---2026-10-04) and [ROADMAP.md](https://github.com/AnimaLogica/ExZarr/blob/main/docs/ROADMAP.md).