Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .formatter.exs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Used by "mix format"
[
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
inputs: ["{mix,.formatter}.exs", "{config,lib,test,native,bench}/**/*.{ex,exs}"]
]
2 changes: 1 addition & 1 deletion .github/workflows/elixir-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,4 @@ jobs:
run: mix compile --warnings-as-errors

- name: Run tests
run: mix test
run: mix test ${{ matrix.coverage && '--cover' || '' }}
32 changes: 29 additions & 3 deletions .github/workflows/precompiled-nifs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -224,10 +224,36 @@ jobs:
otp-version: "27"
elixir-version: "1.18.4"

- name: Install SuiteSparse
- name: Install build dependencies
run: |
sudo apt-get update
sudo apt-get install -y libgraphblas-dev pkg-config
sudo apt-get install -y pkg-config build-essential cmake

- name: Cache GraphBLAS v9.4.5
uses: actions/cache@v4
id: graphblas-cache-publish
with:
path: |
/usr/local/lib/libgraphblas.so*
/usr/local/include/suitesparse/GraphBLAS.h
/usr/local/lib/pkgconfig/GraphBLAS.pc
key: graphblas-v9.4.5-publish

- name: Build and install GraphBLAS v9.4.5
if: steps.graphblas-cache-publish.outputs.cache-hit != 'true'
run: |
git clone --depth 1 --branch v9.4.5 https://github.com/DrTimothyAldenDavis/GraphBLAS.git
cmake -S GraphBLAS -B GraphBLAS/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DGRAPHBLAS_BUILD_PKGCONFIG=ON
cmake --build GraphBLAS/build -j"$(nproc)"
sudo cmake --install GraphBLAS/build
sudo ldconfig
rm -rf GraphBLAS

- name: Configure library paths
run: |
echo "PKG_CONFIG_PATH=/usr/local/lib/pkgconfig" >> "$GITHUB_ENV"
echo "LD_LIBRARY_PATH=/usr/local/lib" >> "$GITHUB_ENV"
sudo ldconfig

- name: Restore Mix dependencies cache
uses: actions/cache@v4
Expand All @@ -249,7 +275,7 @@ jobs:
run: mix compile

- name: Generate checksum file from GitHub Release
run: mix zigler_precompiled.download GraphBLAS.Native.SuiteSparse --all --print 2>/dev/null
run: mix zigler_precompiled.download GraphBLAS.Native.SuiteSparse --all --print --ignore-unavailable

- name: Publish to Hex
run: mix hex.publish --yes
Expand Down
112 changes: 88 additions & 24 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,104 @@
# Changelog

## 0.2.0 (Unreleased)
## 0.2.0

First public release to Hex.pm.

### Highlights

- **Three-backend architecture**: Pure Elixir (reference), SuiteSparse:GraphBLAS (native
performance via Zigler NIFs), and ZigStub (CI verification without native dependencies).
- **Precompiled NIFs** for 6 platforms: aarch64/x86\_64 on Linux (glibc and musl) and macOS.
End users do not need Zig or SuiteSparse installed.
- **Backend behaviour** (`GraphBLAS.Backend`) with 30 callbacks. All three backends implement
the same contract, enforced by `@behaviour`.
- **7 graph algorithms** and a knowledge-graph query layer (`GraphBLAS.Relation`).
- **392 tests**, including property-based tests (StreamData), edge-case coverage, and
algebraic-property verification. 0 failures, 3 skipped (SuiteSparse-only).

### Added

- `backend` field in `%Matrix{}` and `%Vector{}` structs for robust backend dispatch.
- `GraphBLAS.Backend` behaviour with 30 `@callback` definitions.
- `GraphBLAS.Backend.Elixir` -- pure Elixir reference backend (COO maps).
- `GraphBLAS.Backend.SuiteSparse` -- native SuiteSparse:GraphBLAS backend via Zigler NIFs.
- `GraphBLAS.Backend.ZigStub` -- minimal Zig NIF backend for CI verification.
- `backend` field in `%Matrix{}` and `%Vector{}` structs for correct dispatch.
Inspection functions (`nvals`, `to_entries`, `to_coo`, `to_list`, `to_dense`) now
dispatch via the owning backend instead of always defaulting to Elixir.
dispatch via the owning backend.
- Conditional compilation: SuiteSparse modules live in `native/` and are only compiled
when `EX_GRAPHBLAS_COMPILE_NATIVE=1` (test/dev) or in `dev.exs`/`config.exs`.
- Precompiled NIF distribution via ZiglerPrecompiled and GitHub Releases for 6 targets:
`aarch64-linux-gnu`, `aarch64-linux-musl`, `aarch64-macos-none`, `x86_64-linux-gnu`,
`x86_64-linux-musl`, `x86_64-macos-none`.
- SuiteSparse include path configurable via `SUITESPARSE_INCLUDE_PATH` environment
variable or `:suitesparse_include_path` config key. Platform-specific defaults.
variable or `:suitesparse_include_path` config key.
- 7 graph algorithms: `bfs_reach`, `bfs_levels`, `sssp`, `triangle_count`,
`connected_components`, `degree`, `pagerank`.
- `GraphBLAS.Relation` module for knowledge-graph modelling with `traverse/4`,
`closure/4` (with `:max_iter` option), and `fixed_point/3`.
- 12 built-in semirings including `:min_plus`, `:min_plus_fp64`, `:max_plus`,
`:max_plus_fp64`, `:max_min`, `:max_min_fp64`.
- `sssp/3` accepts `:infinity` option (default: `1.0e18`).
- `closure/4` accepts `:max_iter` option (default: `100`).
- Core operations benchmark suite (`bench/core_ops_benchmarks.exs`).
- Benchmark runner script (`bench/run_all.exs`).
- Installation guide (`guides/installation_guide.md`).
- Edge-case test suite (22 tests): zero-dimension containers, negative dimensions,
out-of-bounds indices, all type variants, operations on empty containers.
- Algebraic property test suite (17 tests): commutativity, associativity, transpose
involution, dup independence, reduce correctness.
- ZigStub dimension validation guards matching the Elixir backend.
- `@doc` additions: `Raises KeyError` on `new/1` for BinaryOp, UnaryOp, Monoid,
Semiring; `Raises ArgumentError` on `fn_for/1` for BinaryOp, UnaryOp.
- `Logger.debug` in `application.ex` rescue blocks for SuiteSparse init/finalize
failures.
- Comprehensive guides: installation, architecture walkthrough, reference backend
walkthrough, native backend walkthrough, graph algorithms, masks and descriptors,
parity testing.
- Apache License 2.0.
- `coveralls.json` for coverage configuration.

### Fixed

- **Resource leaks**: Transposed matrices in `matrix_mxm`, `matrix_mxv`, `vector_vxm`
are now freed in `after` blocks via `maybe_free_transposed/2`.
- **Ignored Zig error codes**: Replaced 22 instances of `_ = GrB_*()` with
`try translate_info(GrB_*())` for dimension queries, type queries, descriptor set,
JIT disable, and finalize. Only `_free` calls remain unhandled (correct -- nothing
to do on free failure).
- **Silent type defaults**: `type_code_to_grb_type`, `semiring_from_code`,
`monoid_from_code` now return errors instead of silently defaulting to INT64.
- **Zig build buffer overread**: Added bounds checks (`nvals > *.len`) to all 6 build
functions in `graphblas.zig`.
- **Unsafe `hd()` call**: `algorithm.ex` uses pattern match `[{v, _} | _]` instead of
`hd(unvisited_entries)`.
- `Vector.nvals/1`, `Vector.to_entries/1`, `Vector.to_list/1` now correctly dispatch
to the backend that created the vector. Previously always used Elixir backend,
causing runtime errors on SuiteSparse vectors.
to the backend that created the vector.
- `Matrix.nvals/1`, `Matrix.to_coo/1`, `Matrix.to_dense/1` same dispatch fix.
- `Matrix.set/5`, `Matrix.extract/4`, `Vector.set/4`, `Vector.extract/3` now use
the container's backend instead of ignoring the backend parameter.
- `config/config.exs` updated to reflect SuiteSparse backend availability.
- Removed private `vector_to_entries/1` and `matrix_to_coo/1` helpers from
`algorithm.ex` — public API now handles backend dispatch correctly.
- Moved `bench_data.ex` from `lib/` to `bench/` (benchmark-only code).
- Fixed `Range.new` deprecation warning in `BenchData.undirected_random_graph/2`.
the container's backend.
- Precompiled NIF targets restricted to the 6 platforms we actually build (was
defaulting to 12, causing 404s on `arm-linux-gnueabihf` etc.).
- `publish_hex` CI job builds GraphBLAS v9.4.5 from source instead of using Ubuntu's
v7.4.0 `libgraphblas-dev` (which lacks `GrB_Descriptor_set_INT32`).
- Fixed `Range.new` deprecation warning.
- Fixed OTP 27 `+0.0` float literal warnings.

### Changed

- `README.md` overhauled to reflect Phases 1-6 completion.
- `mix.exs` updated with package metadata for Hex.pm.
- Credo clean (0 issues).
- `mix.exs`: added `source_url`, `source_ref`, explicit `files:` list. Elixir
requirement changed from `~> 1.17` to `~> 1.18`. Prod `elixirc_paths` is `["lib"]`
only (users don't need SuiteSparse headers to compile).
- `apply/3` calls replaced with dynamic module call syntax (`mod.func(args)`) to
satisfy Credo.
- Duplicate `max_int`/`min_int` private functions in `monoid.ex` replaced with
`@int64_max`/`@int64_min` module attributes.
- CI: `elixir-build.yml` caches only `deps/` (not `_build/`), preventing stale NIF
`.so` issues. Cache keys include source file hashes.
- CI: `precompiled-nifs.yml` uses `--ignore-unavailable` as a safety net during
checksum generation.
- Guides updated to reflect completed architecture (no more "coming soon" language).
- README overhauled with three-backend architecture, environment variables table,
and use-case examples.
- Credo strict: 0 issues. Dialyzer: 0 errors.

## 0.1.0 (Phase 1)

Expand All @@ -51,39 +116,38 @@

## Phase History

### Phase 2 — Pure Elixir Reference Backend
### Phase 2 -- Pure Elixir Reference Backend

- Correctness-first pure Elixir backend using `%{{row, col} => value}` maps.
- `mxm`, `mxv`, `vxm`, `ewise_add`, `ewise_mult`, `reduce`, `transpose`.
- Initial semiring set: `plus_times`, `lor_land`, `plus_min`.

### Phase 3 — SuiteSparse Native Backend Foundation
### Phase 3 -- SuiteSparse Native Backend Foundation

- SuiteSparse:GraphBLAS native backend via Zigler.
- NIF resource lifecycle with dirty CPU schedulers.
- Error mapping from GrB error codes to Elixir errors.

### Phase 4 — Core Sparse Operations Parity and Semantic Validation
### Phase 4 -- Core Sparse Operations Parity and Semantic Validation

- Full parity between Elixir and SuiteSparse backends.
- Shared test suites with property-based testing (StreamData).
- Extended semiring set to 10 built-in semirings.
- JIT disabled for reproducibility.

### Phase 5 — Masks, Descriptors, and API Honing
### Phase 5 -- Masks, Descriptors, and API Honing

- Structural and complement masks for all compute operations.
- Descriptor control: input transpose, output replacement.
- `set`, `extract`, `dup` for both Matrix and Vector.
- Masked mxm/vxm benchmarks.

### Phase 6 — Graph Algorithms, Knowledge Graphs, and Query Foundations
### Phase 6 -- Graph Algorithms, Knowledge Graphs, and Query Foundations

- 7 graph algorithms: bfs_reach, bfs_levels, sssp, triangle_count,
connected_components, degree, pagerank.
- `GraphBLAS.Relation` module for knowledge graph modeling.
- `GraphBLAS.Relation` module for knowledge graph modelling.
- `traverse/4` for multi-hop traversal across predicates.
- `closure/4` for transitive closure.
- `fixed_point/3` generic iteration primitive.
- Added `:min_plus` and `:min_plus_fp64` semirings.
- 470 tests passing, 88% coverage.
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ An Elixir library for sparse linear algebra and graph computation, inspired by t
[![HexDocs.pm](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/ex_graphblas)
[![Precompiled NIFs](https://github.com/thanos/ex_graphblas/actions/workflows/precompiled-nifs.yml/badge.svg)](https://github.com/thanos/ex_graphblas/actions/workflows/precompiled-nifs.yml)

GraphBLAS provides idiomatic Elixir data structures at the boundary while delegating computation to swappable backends. The same code runs on a pure Elixir reference backend for development and testing, and on SuiteSparse:GraphBLAS via Zigler NIFs for native performance in production.
GraphBLAS provides idiomatic Elixir data structures at the boundary while delegating computation to swappable backends. The same code runs on a pure Elixir reference backend for development and testing, on SuiteSparse:GraphBLAS via Zigler NIFs for native performance in production, and on a minimal ZigStub backend for CI verification without native dependencies.

## Contents

Expand Down Expand Up @@ -43,6 +43,7 @@ GraphBLAS.Matrix / GraphBLAS.Vector -- public API (backend-neutral)
GraphBLAS.Backend -- behaviour defining the computation contract
GraphBLAS.Backend.Elixir -- pure Elixir reference implementation
GraphBLAS.Backend.SuiteSparse -- SuiteSparse:GraphBLAS via Zigler NIFs
GraphBLAS.Backend.ZigStub -- minimal Zig NIF backend for CI verification
```

Each `%Matrix{}` and `%Vector{}` struct carries a `backend` field, so inspection and mutation operations dispatch to the correct backend automatically.
Expand Down Expand Up @@ -225,6 +226,14 @@ Or in your `config/config.exs`:

See `guides/installation_guide.md` for platform-specific instructions.

### Environment variables

| Variable | Purpose |
|----------|---------|
| `SUITESPARSE_INCLUDE_PATH` | Path to SuiteSparse headers (default: `/opt/homebrew/include/suitesparse`) |
| `EX_GRAPHBLAS_BUILD` | Set to `1` to force building NIFs from source instead of using precompiled binaries |
| `EX_GRAPHBLAS_COMPILE_NATIVE` | Set to `1` to include the SuiteSparse backend in test compilation |

## Guides

The guides in the `guides/` directory provide end-to-end walkthroughs:
Expand Down
11 changes: 9 additions & 2 deletions bench/core_ops_benchmarks.exs
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,15 @@ Benchee.run(

# --- mxv ---

{:ok, v100_elixir} = Vector.from_entries(100, Enum.map(0..99, fn i -> {i, :rand.uniform(10)} end), :int64, backend: RefBackend)
{:ok, v100_ss} = Vector.from_entries(100, Enum.map(0..99, fn i -> {i, :rand.uniform(10)} end), :int64, backend: SuiteSparse)
{:ok, v100_elixir} =
Vector.from_entries(100, Enum.map(0..99, fn i -> {i, :rand.uniform(10)} end), :int64,
backend: RefBackend
)

{:ok, v100_ss} =
Vector.from_entries(100, Enum.map(0..99, fn i -> {i, :rand.uniform(10)} end), :int64,
backend: SuiteSparse
)

IO.puts("\n=== mxv (100x100 * 100, :plus_times) ===")

Expand Down
18 changes: 15 additions & 3 deletions bench/phase5_benchmarks.exs
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,18 @@ ncols_a = 80
nrows_b = 50
ncols_b = 40

a_entries = for r <- 0..(nrows_a - 1), c <- 0..(ncols_a - 1), :rand.uniform() < 0.05, do: {r, c, :rand.uniform(50)}
b_entries = for r <- 0..(nrows_b - 1), c <- 0..(ncols_b - 1), :rand.uniform() < 0.05, do: {r, c, :rand.uniform(50)}
a_entries =
for r <- 0..(nrows_a - 1),
c <- 0..(ncols_a - 1),
:rand.uniform() < 0.05,
do: {r, c, :rand.uniform(50)}

b_entries =
for r <- 0..(nrows_b - 1),
c <- 0..(ncols_b - 1),
:rand.uniform() < 0.05,
do: {r, c, :rand.uniform(50)}

a_entries = if a_entries == [], do: [{0, 1, 1}], else: a_entries
b_entries = if b_entries == [], do: [{1, 0, 1}], else: b_entries

Expand Down Expand Up @@ -113,7 +123,9 @@ Benchee.run(
# 2. Descriptor inp0_transpose on mxm
# =============================================================================

IO.puts("\n=== Descriptor inp0_transpose on mxm (#{nrows_a}x#{ncols_a} * #{nrows_b}x#{ncols_b}) ===\n")
IO.puts(
"\n=== Descriptor inp0_transpose on mxm (#{nrows_a}x#{ncols_a} * #{nrows_b}x#{ncols_b}) ===\n"
)

desc = Descriptor.new(inp0_transpose: :transpose)

Expand Down
15 changes: 11 additions & 4 deletions bench/phase6_algorithms_benchmarks.exs
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,11 @@ Benchee.run(
size_levels = 200
rand_entries = BenchData.random_graph(size_levels, 0.05)

{:ok, ref_bfs_rand} = RefBackend.matrix_from_coo(size_levels, size_levels, rand_entries, :bool, [])
{:ok, ss_bfs_rand} = SuiteSparse.matrix_from_coo(size_levels, size_levels, rand_entries, :bool, [])
{:ok, ref_bfs_rand} =
RefBackend.matrix_from_coo(size_levels, size_levels, rand_entries, :bool, [])

{:ok, ss_bfs_rand} =
SuiteSparse.matrix_from_coo(size_levels, size_levels, rand_entries, :bool, [])

IO.puts("\n=== bfs_levels (random graph, #{size_levels} vertices, 5% density) ===\n")

Expand Down Expand Up @@ -113,7 +116,9 @@ cc_entries = BenchData.undirected_random_graph(size_cc, 0.05)
{:ok, ref_cc} = RefBackend.matrix_from_coo(size_cc, size_cc, cc_entries, :bool, [])
{:ok, ss_cc} = SuiteSparse.matrix_from_coo(size_cc, size_cc, cc_entries, :bool, [])

IO.puts("\n=== connected_components (undirected random graph, #{size_cc} vertices, 5% density) ===\n")
IO.puts(
"\n=== connected_components (undirected random graph, #{size_cc} vertices, 5% density) ===\n"
)

Benchee.run(
%{
Expand Down Expand Up @@ -293,4 +298,6 @@ SuiteSparse.matrix_free(ss_pr)
SuiteSparse.matrix_free(ss_fp)
GraphBLAS.Native.grb_finalize()

IO.puts("\n=====================================\nPhase 6 algorithm benchmarks complete.\n=====================================\n")
IO.puts(
"\n=====================================\nPhase 6 algorithm benchmarks complete.\n=====================================\n"
)
7 changes: 7 additions & 0 deletions config/prod.exs
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
import Config

# The default backend is pure Elixir -- no native dependencies required.
# To use the SuiteSparse native backend in production, set:
#
# config :ex_graphblas, default_backend: GraphBLAS.Backend.SuiteSparse
#
# and ensure SuiteSparse:GraphBLAS >= 9.4.5 is installed. See the
# installation guide for platform-specific instructions.
config :ex_graphblas,
default_backend: GraphBLAS.Backend.Elixir
Loading
Loading