diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 28e59a8..3fbb55d 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -17,8 +17,8 @@ jobs: fail-fast: false matrix: include: - - elixir: "1.20.0-rc.3" - otp: "28.4.1" + - elixir: "1.20" + otp: "29" # Latest stable versions - elixir: "1.19.5" otp: "28.4.1" diff --git a/CHANGELOG.md b/CHANGELOG.md index 3438c8c..158bd71 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,32 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## v0.4.0 (2026-06-20) + +### Added +- `ExDatalog.Schema` — Ecto-inspired DSL macro module for defining Datalog programs + - `relation/2` macro declares typed relation schemas + - `fact/1` and `facts/2` macros declare ground facts + - `rule/2` macro declares rules with lowercase logic variables, `not_` for negation, named constraint predicates (`gt`, `eq`, `add`, etc.) + - `query/2` macro declares named post-materialization queries with `find`/`where` + - `wildcard/0` helper for explicit wildcards in rule bodies + - Generated `program/0`, `materialize/0,1`, `queries/0`, `query/2` functions +- `ExDatalog.UnsupportedFeature` struct for forward-compatible aggregate syntax parsing +- `ExDatalog.DSL.CompileError` exception for readable DSL macro errors +- 33 integration tests for the DSL (relations, facts, rules, negation, constraints, queries, backward compatibility) +- Livebook tutorial: `livebooks/ex_datalog_dsl.livemd` +- Educational articles in `docs/articles/` + +### Changed +- Version bumped from 0.3.0 to 0.4.0 +- DSL is the recommended authoring layer; builder API remains the stable lower-level API +- README updated with DSL quickstart + +### Notes +- Query DSL operates on materialized knowledge only (no query planner yet) +- Aggregate syntax is parsed but not yet executable — returns `%UnsupportedFeature{feature: :aggregates}` +- All 718 existing tests continue to pass (now 751 total) + ## [0.3.0] - 2025-06-19 ### Added diff --git a/README.md b/README.md index ca276c9..6f4a8b2 100644 --- a/README.md +++ b/README.md @@ -34,14 +34,16 @@ It continues to influence modern databases, compilers, static analysis tools, kn ## Features - **Builder API** for constructing programs (relations, facts, rules, constraints) +- **Schema DSL** — Ecto-inspired macros for declaring relations, facts, rules, and queries - **Constraint types**: comparisons, arithmetic, type predicates, string predicates, membership - **Negation** with stratified evaluation - **Recursive rules** with semi-naive fixpoint evaluation +- **Post-materialization queries** (`query` macro with `find`/`where`) - **Pluggable storage backends**: `Storage.Map` (default, on-heap) and `Storage.ETS` (off-heap, concurrent reads) - **Provenance / derivation explain** (`explain: true`) - **Telemetry** integration (`:telemetry` events for query lifecycle) - **Deterministic**: same program + same facts = same result regardless of backend -- 601 tests, 0 failures, credo clean, dialyzer clean +- 751 tests, 0 failures, credo clean ## Installation @@ -50,13 +52,80 @@ Add `ex_datalog` to your dependencies in `mix.exs`: ```elixir def deps do [ - {:ex_datalog, "~> 0.3.0"} + {:ex_datalog, "~> 0.4.0"} ] end ``` ## Quick Start +### DSL (Schema macro) + +The recommended way to define Datalog programs in v0.4.0+: + +```elixir +defmodule AncestorRules do + use ExDatalog.Schema + + relation :parent do + field :parent, :atom + field :child, :atom + end + + relation :ancestor do + field :ancestor, :atom + field :descendant, :atom + end + + fact parent(:alice, :bob) + fact parent(:bob, :carol) + fact parent(:carol, :dave) + + rule ancestor(X, Y) do + parent(X, Y) + end + + rule ancestor(X, Z) do + parent(X, Y) + ancestor(Y, Z) + end + + query :descendants_of_alice do + find Y + where ancestor(:alice, Y) + end +end + +{:ok, knowledge} = AncestorRules.materialize() +AncestorRules.query(:descendants_of_alice, knowledge) +#=> [:bob, :carol, :dave] +``` + +Lowercase variables in rule bodies are logic variables. Constants use +atom syntax (`:alice`). Use `_` or `wildcard()` for wildcards. Negation +uses `not_`: + +```elixir +rule bachelor(P) do + male(P) + not_ married(P, _) +end +``` + +Constraints use named predicates: + +```elixir +rule high_earner(P) do + income(P, S) + gt(S, 100_000) +end +``` + +The builder API (`Program.add_rule`, `Program.add_fact`, etc.) remains +fully supported as the lower-level interface. + +### Builder API + ### Transitive closure The classic Datalog example: compute all ancestors from parent facts. @@ -280,8 +349,11 @@ reference. - [What is Datalog?](docs/what-is-datalog.md) — introduction, history, Prolog comparison, industry use cases, LLM integration - [Constraints](docs/constraints.md) — constraint types, evaluation, and the dispatch model - [Storage Backends](docs/storage_backends.md) — Map vs ETS, options, capabilities, determinism guarantee -- [Quickstart Tutorial](livebook/quickstart.livemd) — interactive Livebook walkthrough -- [Examples](livebook/examples.livemd) — 10 realistic use cases (RBAC, supply chain, fraud detection, and more) +- [Migration: Builder API → DSL](docs/migration_dsl.md) — migrate existing builder-API code to the Schema DSL +- [DSL Articles](docs/articles/01_why_datalog_on_the_beam.md) — why Datalog on the BEAM, building the DSL, rules as macros, queries, negation +- [Quickstart Tutorial](livebooks/quickstart.livemd) — interactive Livebook walkthrough +- [DSL Tutorial](livebooks/ex_datalog_dsl.livemd) — interactive DSL walkthrough +- [Examples](livebooks/examples.livemd) — 10 realistic use cases (RBAC, supply chain, fraud detection, and more) - [API reference](https://hexdocs.pm/ex_datalog) — full module and function documentation Generate docs locally: @@ -349,8 +421,8 @@ The following references are highly recommended for understanding both the theor | Version | Description | |---|---| | v0.3.0 | Tuple shorthand for rules (`add_rule/3`, `add_rule/4`), `Term.from/1`, `ExDatalog.Atom.from_tuple/1`, `Constraint.from_tuple/1`; renamed `Result` → `Knowledge`, `query` → `materialize` | -| v0.4.0 | Sigil DSL (`~d`), aggregation (`count`, `sum`, `min`, `max`), general predicates as deterministic BEAM callbacks | -| v0.5.0 | Magic sets / demand-driven evaluation, external solver adapter (experimental Z3/Soufflé) | +| v0.4.0 | Schema DSL (`use ExDatalog.Schema`), `relation`, `fact`, `rule`, `query` macros, `not_` negation, constraint DSL, post-materialization queries, aggregate syntax preview | +| v0.5.0 | Magic sets / demand-driven evaluation, general predicates as BEAM callbacks | | v1.0.0 | Stable public API, hardened production semantics | ## License diff --git a/docs/articles/01_why_datalog_on_the_beam.md b/docs/articles/01_why_datalog_on_the_beam.md new file mode 100644 index 0000000..3ba77d0 --- /dev/null +++ b/docs/articles/01_why_datalog_on_the_beam.md @@ -0,0 +1,102 @@ +# Why Datalog Fits the BEAM VM + +Datalog is a declarative logic programming language rooted in first-order logic. It computes derived facts from base facts using recursive rules, converging to a fixed point where no new facts can be produced. ExDatalog brings this model to the BEAM — Erlang's virtual machine — and the fit is more than coincidental. The BEAM's foundational design choices align with Datalog's semantics in ways that make the implementation feel natural rather than forced. + +## Shared Foundations: Immutability and Pattern Matching + +Datalog operates on immutable sets of facts. Once a fact is asserted, it never changes — the evaluation engine simply accumulates new derivations until reaching a fixpoint. The BEAM's runtime is built around the same principle. Elixir data structures are immutable by default; every "modification" produces a new value, and references to the old version remain valid. + +This isn't just a philosophical alignment. ExDatalog's `Knowledge` struct holds each relation as a `MapSet` of tuples. During semi-naive evaluation, each iteration produces a *delta* — the set of newly derived facts. The engine merges the delta into the full fact set using `MapSet.union/2`, which returns a new set without mutating the old one. The old snapshot is preserved as `old`, and the delta is computed as `full \ old`. No defensive copying, no lock-based concurrency concerns, no risk of one iteration corrupting another's view of the database. + +Pattern matching is the other shared primitive. In Datalog, a rule body like: + +```elixir +rule ancestor(x, z) do + parent(x, y) + ancestor(y, z) +end +``` + +means: for every binding of `x`, `y`, `z` where `parent(x, y)` and `ancestor(y, z)` both hold, derive `ancestor(x, z)`. The engine joins relations by matching tuples against term patterns — variables bind, constants must equal, wildcards match anything. This is precisely what the BEAM's pattern matching engine does when it dispatches function clauses. ExDatalog's `Engine.Binding` module extends a binding environment by matching IR values against stored tuple positions, just as Elixir extends a function's local scope by matching a pattern against an argument. + +## Recursive Evaluation and the Fixpoint + +Datalog's defining computational model is the fixpoint: start with the base facts, apply every rule, collect new derivations, and repeat until nothing new emerges. This maps directly onto the BEAM's strength in managing long-running, message-driven processes — but even without processes, the fixpoint loop is a natural fit for a functional runtime. + +ExDatalog's `Engine.Naive` implements the semi-naive algorithm. Each iteration considers only facts that are *new* since the last iteration (the delta), avoiding redundant derivations. The implementation is a straightforward recursive loop: + +```elixir +defp fixpoint(ctx) do + if delta_empty?(ctx.delta, ctx.all_rels) do + ctx + else + iterate(ctx) + end +end +``` + +The BEAM's tail-call optimization ensures this loop runs in constant stack space, even for programs requiring thousands of iterations. The default iteration limit is 10,000, configurable via `max_iterations`. Timeouts are checked each iteration using monotonic time, avoiding clock drift: + +```elixir +if System.monotonic_time(:millisecond) > ctx.deadline do + %{ctx | termination: :timeout} +``` + +The termination status (`:fixpoint`, `:iteration_limit`, or `:timeout`) is returned in the `Knowledge` struct's `stats` field, giving callers a clear signal about whether the result is complete. + +## Hot Code Reloading and Knowledge Evolution + +The BEAM's hot code reloading is one of its most distinctive features — you can upgrade a running system's code without stopping it. ExDatalog's DSL leverages this through Ecto-inspired compile-time macros. When you `use ExDatalog.Schema` and define a module like: + +```elixir +defmodule AncestorRules do + use ExDatalog.Schema + + relation :parent do + field :parent, :atom + field :child, :atom + end + + rule ancestor(x, y) do + parent(x, y) + end + + rule ancestor(x, z) do + parent(x, y) + ancestor(y, z) + end +end +``` + +The module compiles into a `program/0` function that builds the `ExDatalog.Program` struct at runtime. If you modify the rules — say, adding a new relation or changing a constraint — and hot-reload the module, the next call to `AncestorRules.program()` returns the updated program. The knowledge base itself is immutable; you re-materialize to get new results. This separation between the rule definition (code) and the derived knowledge (data) is exactly how Datalog is meant to work, and the BEAM's hot code reloading makes it operationally seamless. + +## Stratification and Process Isolation + +When Datalog programs include negation, stratification determines the evaluation order: relations appearing under negation must be fully computed before they can be negated. ExDatalog uses Tarjan's strongly connected components algorithm to compute strata at compile time, then evaluates them sequentially. + +```elixir +{state_final, total_iterations, origins, termination} = + eval_strata(state, ir.strata, ir.rules, max_iterations, ...) +``` + +Each stratum runs to a local fixpoint before the next one begins. This sequential dependency would be awkward in a system that assumes concurrent mutation, but on the BEAM — where processes share no memory and communicate by message passing — the isolation is inherent. The current implementation evaluates strata sequentially within a single process, but the architecture leaves room for parallelizing independent strata across BEAM processes, since each stratum's derived facts can be computed from immutable snapshots. + +## Persistent Data Structures and Incremental Computation + +The BEAM's immutable data structures have another advantage: they make incremental computation efficient. During semi-naive evaluation, ExDatalog maintains three snapshots per relation per iteration: + +- **`full`** — all known facts +- **`delta`** — facts newly derived in the previous iteration +- **`old`** — the `full` snapshot before the current derivation step + +Because Elixir's Maps and MapSets are persistent data structures, creating `old` from `full` is an O(1) pointer copy — the entire snapshot shares structure with the previous version. Computing `delta = full \ old` requires only the diff, avoiding a full scan of every relation on every iteration. + +The `Storage.Map` backend stores facts directly in process heap Maps and MapSets. For workloads exceeding ~100K facts, the `Storage.ETS` backend moves data off-heap into per-relation ETS tables, reducing GC pressure while preserving the same deterministic output guarantee. Both backends produce identical `Knowledge` structs for the same program and facts. + +## Why Not Prolog? + +Elixir developers sometimes ask: why Datalog rather than Prolog on the BEAM? The answer is convergence. Prolog's top-down evaluation with backtracking can loop infinitely on recursive programs, and implementing a complete Prolog engine (with cut, occur-check, and tabling) is a deep undertaking. Datalog's bottom-up, ground-term-only model guarantees termination for programs without arithmetic constraints on unbounded domains, and its fixpoint semantics are simpler to implement correctly. + +ExDatalog's validation pipeline catches non-terminating programs before evaluation begins. The safety checker rejects rules where head variables aren't bound by positive body atoms. The stratification checker rejects programs with unstratifiable negation cycles. These are compile-time guarantees — not runtime gambles. + +The BEAM's strengths — immutability, pattern matching, hot code reloading, persistent data structures — are not just compatible with Datalog's semantics. They make Datalog on the BEAM feel like the natural expression of a logic language in a concurrent, fault-tolerant runtime. ExDatalog v0.4.0 is the first version to surface these strengths through a declarative Schema DSL, and the result is a system where writing Datalog programs feels like writing Elixir. \ No newline at end of file diff --git a/docs/articles/02_building_an_elixir_datalog_dsl.md b/docs/articles/02_building_an_elixir_datalog_dsl.md new file mode 100644 index 0000000..4fe45c7 --- /dev/null +++ b/docs/articles/02_building_an_elixir_datalog_dsl.md @@ -0,0 +1,123 @@ +# Building an Elixir Datalog DSL: The Design of ExDatalog v0.4.0's Schema Macro + +ExDatalog v0.3.0 offered a builder API: you assembled programs by piping `Program.add_relation/3`, `Program.add_fact/3`, and `Program.add_rule/2` calls. It worked, but it looked like configuration code, not like expressing logical rules. Version 0.4.0 introduces the Schema DSL — a set of compile-time macros that let you declare relations, facts, rules, and queries inside an Elixir module. + +This article walks through the design decisions and explains how the DSL compiles down to the builder API. + +## Why Ecto-Inspired Macros? + +ExDatalog's Schema DSL follows the same pattern as Ecto's `Ecto.Schema`. Both need to collect declarations at compile time and generate runtime functions from those declarations. The `@before_compile` pattern is essential: you can't generate `program/0` until all `relation`, `fact`, and `rule` macros have finished registering their data. + +```elixir +defmacro __using__(_opts) do + quote do + import ExDatalog.Schema, only: [relation: 2, fact: 1, facts: 2, rule: 2, query: 2, wildcard: 0] + + Module.register_attribute(__MODULE__, :ex_datalog_relations, accumulate: true) + Module.register_attribute(__MODULE__, :ex_datalog_facts, accumulate: true) + Module.register_attribute(__MODULE__, :ex_datalog_rules, accumulate: true) + Module.register_attribute(__MODULE__, :ex_datalog_queries, accumulate: true) + + @before_compile ExDatalog.Schema + end +end +``` + +Four module attributes accumulate declarations. Each macro call appends to the corresponding attribute, and `__before_compile__` reads them all to generate `program/0`, `materialize/0`, `queries/0`, and `query/2`: + +```elixir +defmacro __before_compile__(env) do + relations = Module.get_attribute(env.module, :ex_datalog_relations) |> Enum.reverse() + facts = Module.get_attribute(env.module, :ex_datalog_facts) |> Enum.reverse() + rules = Module.get_attribute(env.module, :ex_datalog_rules) |> Enum.reverse() + queries = Module.get_attribute(env.module, :ex_datalog_queries) |> Enum.reverse() + + quote do + def program do + ExDatalog.Schema.__build_program__(unquote(Macro.escape(relations)), + unquote(Macro.escape(facts)), + unquote(Macro.escape(rules))) + end + + def materialize(opts \\ []) do + ExDatalog.materialize(program(), opts) + end + + def queries do + unquote(Macro.escape(Map.new(queries, fn q -> {q.name, q} end))) + end + + def query(name, knowledge) do + ExDatalog.Schema.__execute_query__(name, knowledge, unquote(Macro.escape(queries))) + end + end +end +``` + +`Enum.reverse/1` restores declaration order because `accumulate: true` prepends. `Macro.escape/1` converts the Elixir data structures into AST literals that the `quote` block can inject. + +## The Variable Convention + +The DSL follows Prolog convention for distinguishing variables from constants: uppercase identifiers are logic variables, lowercase atoms and colon-prefixed atoms are constants, and `_` is a wildcard. + +This convention is driven by how Elixir's AST represents identifiers. Uppercase names like `X` and `Y` are parsed as `__aliases__` nodes, producing a distinct AST shape from lowercase variables. The `parse_term/1` function dispatches on these shapes: + +```elixir +# Uppercase identifiers (module aliases) → logic variables +defp parse_term({:__aliases__, _, [alias_name]}) when is_atom(alias_name) do + {:var, Atom.to_string(alias_name)} +end + +# Bare identifiers with nil context +defp parse_term({var_name, _context, nil}) when is_atom(var_name) do + var_str = Atom.to_string(var_name) + cond do + var_str == "_" -> :wildcard + var_str =~ ~r/^[A-Z]/ -> {:var, var_str} + true -> {:const, var_name} + end +end + +# Atoms with colon prefix → constants +defp parse_term(atom) when is_atom(atom) do + Atom.to_string(atom) |> create_term() +end +``` + +| DSL Syntax | Internal Form | Meaning | +|---|---|---| +| `X`, `Y`, `Z` | `{:var, "X"}`, `{:var, "Y"}` | Logic variable | +| `:alice`, `:bob` | `{:const, :alice}`, `{:const, :bob}` | Constant | +| `_` | `:wildcard` | Anonymous variable | + +The test suite confirms: `rule reachable(:start, Y)` produces `head.terms == [Term.from(:start), Term.var("Y")]`, where `:start` is a constant and `Y` is a variable. + +## Macro Hygiene and `Macro.escape` + +The `rule/2` macro uses `Macro.escape` on both the head and body, which is critical. Without escaping, Elixir would try to evaluate the AST as runtime code. With escaping, the AST is preserved as data and passed to `__register_rule__/3` at compile time: + +```elixir +defmacro rule(head, do: body) do + quote do + ExDatalog.Schema.__register_rule__(__MODULE__, + unquote(Macro.escape(head)), + unquote(Macro.escape(body))) + end +end +``` + +This means the DSL never evaluates `parent(X, Y)` as a function call. It parses the AST representation `{:parent, [line: N], [{:__aliases__, ..., [:X]}, {:__aliases__, ..., [:Y]}]}` and extracts the relation name and terms as data. + +## What the DSL Doesn't Do (Yet) + +The v0.4.0 DSL deliberately omits some features: + +- **Aggregates** — the syntax `rule employee_count(dept, agg(:count, emp))` is parsed but returns `%UnsupportedFeature{feature: :aggregates}`. Materializing such a program fails with a clear error. +- **Query planning** — the `query` macro calls `Knowledge.match/3` on materialized knowledge. There's no query optimizer or cost model. +- **Schema validation** — the DSL doesn't check at compile time that a `fact` references a declared relation, or that a `rule` head's arity matches its `relation` declaration. These checks happen at runtime through the builder API and `Validator`. + +The DSL's job in v0.4.0 is to make the common case ergonomic and readable. The builder API remains the fully-capable interface for dynamic program construction or features the DSL doesn't yet cover. + +Good Elixir macros don't hide complexity — they eliminate boilerplate. The Schema DSL compiles to the same builder API, so every macro feature is also available programmatically. The `__build_program__/3` function iterates over the collected relations, facts, and rules, calling `Program.add_relation/3`, `Program.add_fact/3`, and `Program.add_rule/2` — the exact same pipeline a hand-written builder would use. At runtime, there's no macro expansion, no AST walking — just the builder pipeline running on pre-computed data structures. + +This design also means you can mix both approaches: define the static structure of your program with the DSL, then extend it with the builder API at runtime. The test suite verifies this with a backward-compatibility test that defines a schema, calls `program/0`, then adds additional facts and rules via `Program.add_fact/3` and `Program.add_rule/2` before materializing. \ No newline at end of file diff --git a/docs/articles/03_datalog_rules_as_elixir_macros.md b/docs/articles/03_datalog_rules_as_elixir_macros.md new file mode 100644 index 0000000..cb1b30d --- /dev/null +++ b/docs/articles/03_datalog_rules_as_elixir_macros.md @@ -0,0 +1,141 @@ +# Datalog Rules as Elixir Macros: Parsing Heads, Bodies, and Negation at Compile Time + +In ExDatalog v0.4.0, the `rule/2` macro transforms Datalog rule declarations from Elixir syntax into internal data structures at compile time. This article walks through the parsing pipeline: how rule heads and bodies are destructured, how uppercase identifiers become logic variables, and how `not_` desugars into negative literals. + +## Parsing the Rule Head + +The `rule/2` macro receives the head and body as AST fragments: + +```elixir +defmacro rule(head, do: body) do + quote do + ExDatalog.Schema.__register_rule__(__MODULE__, + unquote(Macro.escape(head)), + unquote(Macro.escape(body))) + end +end +``` + +`Macro.escape` preserves the AST as data. `__register_rule__/3` runs at compile time, calling `parse_rule_head/1` and `parse_rule_body/1` on the escaped forms. + +For `rule ancestor(X, Y) do parent(X, Y) end`, the head AST is: + +```elixir +{:ancestor, [line: 1], [{:__aliases__, [alias: false], [:X]}, {:__aliases__, [alias: false], [:Y]}]} +``` + +`parse_rule_head/1` extracts the relation name and maps each argument through `parse_term/1`: + +```elixir +defp parse_rule_head({head_atom, _context, args}) when is_atom(head_atom) and is_list(args) do + {Atom.to_string(head_atom), Enum.map(args, &parse_term/1)} +end +``` + +Result: `{"ancestor", [{:var, "X"}, {:var, "Y"}]}`. + +## The Prolog Convention: Uppercase = Variable + +The DSL follows Prolog convention for term classification: + +- **Uppercase identifiers** (`X`, `Y`, `Z`) → logic variables, parsed via the `__aliases__` clause +- **Colon-prefixed atoms** (`:alice`) → constants, parsed via the atom clause +- **`_`** → wildcard + +The `parse_term/1` function dispatches on AST node shape: + +```elixir +# Module aliases (uppercase) → logic variables +defp parse_term({:__aliases__, _, [alias_name]}) when is_atom(alias_name) do + {:var, Atom.to_string(alias_name)} +end + +# 3-tuple with nil context +defp parse_term({var_name, _context, nil}) when is_atom(var_name) do + var_str = Atom.to_string(var_name) + cond do + var_str == "_" -> :wildcard + var_str =~ ~r/^[A-Z]/ -> {:var, var_str} + true -> {:const, var_name} + end +end + +# Bare atoms with colon prefix → constants +defp parse_term(atom) when is_atom(atom) do + Atom.to_string(atom) |> create_term() +end +``` + +Why Prolog convention? Three reasons. First, Elixir's AST represents uppercase identifiers as `__aliases__` nodes, giving a clean dispatch point. Second, facts need constants: `fact parent(:alice, :bob)` uses `:alice` and `:bob` as ground values. Third, uppercase variables (`X`, `Y`, `Z`) make logical structure immediately visible: + +```elixir +rule ancestor(X, Z) do + parent(X, Y) + ancestor(Y, Z) +end +``` + +The test suite verifies this convention. `rule reachable(:start, Y)` produces `head.terms == [Term.from(:start), Term.var("Y")]` — `:start` is a constant, `Y` is a variable. + +## How `not_` Desugars to Negative Literals + +Negation uses the `not_` prefix: + +```elixir +rule bachelor(P) do + male(P) + not_ married(P, _) +end +``` + +In the AST, `not_ married(P, _)` is `{:not_, meta, [rel_call]}`. `parse_body_call/1` pattern-matches on this: + +```elixir +defp parse_body_call({:not_, _, [rel_call]}) do + {rel_name, args} = parse_rel_call(rel_call) + terms = Enum.map(args, &parse_term/1) + {:negative, %ExDatalog.Atom{relation: Atom.to_string(rel_name), terms: terms}} +end +``` + +The nested `married(P, _)` is parsed as a relational atom, and the entire expression is tagged `{:negative, ...}`. Using `not_` as a prefix avoids collision with Elixir's reserved `not` operator while reading naturally: "not married." + +## Constraint Desugaring + +Constraint predicates (`gt`, `add`, `is_integer`, etc.) are parsed as function calls. The DSL generates `parse_body_call/1` clauses at compile time: + +```elixir +constraint_ops = [:eq, :neq, :gt, :gte, :lt, :lte, :add, :sub, :mul, :div, + :is_integer, :is_binary, :is_atom, :starts_with, :contains, :member] + +Enum.each(constraint_ops, fn op -> + defp parse_body_call({unquote(op), _, args}) when is_list(args) do + {:constraint, build_constraint(unquote(op), args)} + end +end) +``` + +For `gt(S, 100_000)`: + +```elixir +rule high_earner(P) do + income(P, S) + gt(S, 100_000) +end +``` + +`build_constraint(:gt, [{:__aliases__, ..., [:S]}, 100_000])` parses each argument and delegates to `Constraint.from_tuple`, producing `%Constraint{op: :gt, left: {:var, "S"}, right: {:const, 100_000}, result: nil}`. + +The `member` constraint has special handling for literal lists — `member(X, [:a, :b, :c])` wraps the list directly as `{:const, [:a, :b, :c]}` rather than parsing each element. + +## From Parsed Data to Rule Structs + +After parsing, `__build_program__/3` converts the intermediate forms into `ExDatalog.Rule` structs via `term_from_parsed/1`: + +```elixir +defp term_from_parsed({:var, name}), do: ExDatalog.Term.var(name) +defp term_from_parsed({:const, value}), do: ExDatalog.Term.from(value) +defp term_from_parsed(:wildcard), do: ExDatalog.Term.from(:_) +``` + +The result is exactly the same `Program` struct you'd build by hand with the builder API. The DSL is a zero-overhead compile-time transformation — no macro expansion, no AST walking, no runtime reflection at materialization time. \ No newline at end of file diff --git a/docs/articles/04_querying_materialized_knowledge.md b/docs/articles/04_querying_materialized_knowledge.md new file mode 100644 index 0000000..307c8a0 --- /dev/null +++ b/docs/articles/04_querying_materialized_knowledge.md @@ -0,0 +1,142 @@ +# Querying Materialized Knowledge: Post-Materialization Queries with Find/Where + +Datalog evaluation is eager: `materialize/2` computes every derivable fact and stops at a fixpoint. The result is a `Knowledge` struct — a fully materialized knowledge base. But raw knowledge is verbose. ExDatalog v0.4.0's `query` macro lets you declare named post-materialization queries that project specific columns from specific relations. + +## Declaring Queries + +Inside a schema module, you declare queries alongside relations, facts, and rules: + +```elixir +defmodule AncestorRules do + use ExDatalog.Schema + + relation :parent do + field :parent, :atom + field :child, :atom + end + + relation :ancestor do + field :ancestor, :atom + field :descendant, :atom + end + + fact parent(:alice, :bob) + fact parent(:bob, :carol) + fact parent(:carol, :dave) + + rule ancestor(X, Y) do + parent(X, Y) + end + + rule ancestor(X, Z) do + parent(X, Y) + ancestor(Y, Z) + end + + query :descendants_of_alice do + find Y + where ancestor(:alice, Y) + end +end + +{:ok, knowledge} = AncestorRules.materialize() +AncestorRules.query(:descendants_of_alice, knowledge) +#=> [:bob, :carol, :dave] +``` + +The `find` clause specifies which variables to extract. A single-column `find` returns a list of values. A multi-column `find` returns a list of tuples: + +```elixir +query :all_ancestor_pairs do + find X, Y + where ancestor(X, Y) +end + +AncestorRules.query(:all_ancestor_pairs, knowledge) +#=> [{:alice, :bob}, {:alice, :carol}, {:alice, :dave}, +#=> {:bob, :carol}, {:bob, :dave}, {:carol, :dave}] +``` + +## How Queries Compile + +The `query` macro is parsed at compile time by `__register_query__/3`. `parse_query_block/1` extracts: + +1. **Variable names from `find`** — e.g., `["Y"]` or `["X", "Y"]`. +2. **The relation name from `where`** — e.g., `"ancestor"`. +3. **A pattern from `where`** — e.g., `[{:const, :alice}, {:var, "Y"}]`. + +These are stored in a `QueryMeta` struct baked into the generated `query/2` function as a literal — no runtime lookup overhead. + +## The Engine Room: `Knowledge.match/3` + +`__execute_query__/3` performs pattern matching and projection. First, it calls `Knowledge.match/3`: + +```elixir +def match(%__MODULE__{relations: rels}, relation, pattern) do + tuples = Map.get(rels, relation, MapSet.new()) + + Enum.reduce(tuples, MapSet.new(), fn tuple, acc -> + if matches_pattern?(tuple, pattern) do + MapSet.put(acc, tuple) + else + acc + end + end) +end +``` + +The pattern is a list where `:_` matches any value and other values match exactly. The `where` clause `ancestor(:alice, Y)` compiles to `[:alice, :_]` — constants are passed through, variables become wildcards because their values aren't known until the match runs: + +```elixir +defp query_term_to_pattern(:wildcard), do: :_ +defp query_term_to_pattern({:var, _}), do: :_ +defp query_term_to_pattern({:const, value}), do: value +``` + +## Projection: From Tuples to Targeted Results + +After `match/3` returns matching tuples, `project_tuple/3` extracts the columns specified in `find`: + +```elixir +defp project_tuple(tuple, find_vars, pattern) do + positions = + find_vars + |> Enum.map(fn var_name -> + Enum.find_index(pattern, fn + {:var, ^var_name} -> true + _ -> false + end) + end) + + case positions do + [single_pos] when is_integer(single_pos) -> elem(tuple, single_pos) + _ when is_list(positions) -> + positions + |> Enum.filter(&(&1 != nil)) + |> Enum.map(fn pos -> elem(tuple, pos) end) + |> List.to_tuple() + end +end +``` + +For `find Y` with pattern `[{:const, :alice}, {:var, "Y"}]`: find position of `"Y"` → index 1, extract `elem(tuple, 1)` from each matched tuple, return a flat list. + +For `find X, Y` with pattern `[{:var, "X"}, {:var, "Y"}]`: find positions of both variables, extract both values, return a list of tuples. + +The `Enum.sort()` call in `__execute_query__/3` ensures deterministic output regardless of storage backend. + +## The Builder API Alternative + +Without the DSL, post-materialization queries use `Knowledge.match/3` directly: + +```elixir +matched = Knowledge.match(knowledge, "ancestor", [:alice, :_]) +results = matched |> MapSet.to_list() |> Enum.map(fn {_, desc} -> desc end) |> Enum.sort() +#=> [:bob, :carol, :dave] +``` + +The `query` macro automates this `match → project → sort` pipeline with compile-time name resolution. + +## Limitations + +The current `query` macro operates on a single relation, matches one pattern, and projects specific columns. It doesn't support joins across relations, aggregates, or negation. These limits exist because queries run against already-materialized knowledge. A future query planner could decompose multi-relation queries into join plans that reuse the evaluation engine's matching infrastructure, but for v0.4.0, `Knowledge.match/3` is the foundation. \ No newline at end of file diff --git a/docs/articles/05_negation_constraints_and_safety.md b/docs/articles/05_negation_constraints_and_safety.md new file mode 100644 index 0000000..8eee390 --- /dev/null +++ b/docs/articles/05_negation_constraints_and_safety.md @@ -0,0 +1,134 @@ +# Negation, Constraints, and Safety: Stratified Evaluation in ExDatalog + +Datalog without negation is monotonic: facts can only be added, never retracted. Real programs need negation — "find people who are not parents," "detect unmatched transactions." But adding negation to a recursive program creates a problem: the evaluation order of negated literals affects the result, and negation cycles can make the result undefined. + +ExDatalog solves this with stratified negation — a well-established technique from database theory — backed by variable safety rules that prevent unsound programs from running. + +## The `not_` Macro and Negative Literals + +In the Schema DSL, negation is expressed with `not_`: + +```elixir +rule bachelor(P) do + male(P) + not_ married(P, _) +end +``` + +The DSL desugars this into `{:negative, %ExDatalog.Atom{}}` — a body literal with explicit polarity. During evaluation, the engine checks whether a binding from the positive atoms (`male(P)`) is contradicted by any matching tuple in the negated relation (`married`). If a match exists, the binding is rejected; otherwise, it passes. + +## Stratification: Tarjan's Algorithm + +Negation requires that the negated relation be fully computed before the rule fires. ExDatalog's `Validator.Stratification` module builds a dependency graph where each rule creates edges from its head relation to each body relation, tagged with polarity: + +```elixir +bachelor → male (positive) +bachelor → married (negative) +``` + +The module computes strongly connected components (SCCs) using Tarjan's algorithm. If any SCC contains a negative edge, the program is unstratifiable and is rejected: + +```elixir +defp check_scc_negation(scc, graph) do + scc_set = MapSet.new(scc) + scc + |> Enum.flat_map(fn rel -> + deps = Map.get(graph, rel, []) + Enum.filter(deps, fn {dep, polarity} -> + polarity == :negative and MapSet.member?(scc_set, dep) + end) + end) +end +``` + +The compiler then assigns each relation a stratum — the lowest stratum such that all negative dependencies belong to strictly lower strata. During evaluation, `Engine.Naive` processes strata sequentially: + +```elixir +{state_final, total_iterations, origins, termination} = + eval_strata(state, ir.strata, ir.rules, max_iterations, ...) +``` + +Each stratum runs to a local fixpoint before the next begins, guaranteeing that negated relations are complete. + +## The Constraint DSL + +Constraints are built-in predicates that filter or extend bindings during rule evaluation. They appear in rule bodies alongside relational atoms: + +```elixir +rule high_earner(P) do + income(P, S) + gt(S, 100_000) +end +``` + +The DSL desugars `gt(S, 100_000)` into `%Constraint{op: :gt, left: {:var, "S"}, right: {:const, 100_000}, result: nil}`. Five categories exist: + +| Category | Ops | Binds result? | +|---|---|---| +| Comparison | `gt`, `lt`, `gte`, `lte`, `eq`, `neq` | No — filters | +| Arithmetic | `add`, `sub`, `mul`, `div` | Yes — binds result variable | +| Type predicate | `is_integer`, `is_binary`, `is_atom` | No — filters | +| String predicate | `starts_with`, `contains` | No — filters | +| Membership | `member` | No — filters | + +Arithmetic constraints are special: they introduce new variable bindings. In `add(B, 20, T)`, the result variable `T` is computed and added to the binding environment, making it available for the rule head even though it doesn't appear in any positive body atom. + +## Variable Safety Rules + +ExDatalog enforces three safety rules: + +**1. Head variables must be bound.** Every variable in the rule head must appear in a positive body atom or be an arithmetic result variable. This rejects `ancestor(X, Z) :- parent(X, Y)` because `Z` is unbound. + +**2. Constraint inputs must be bound before use.** Constraints are validated sequentially. A comparison's inputs must be bound by positive body atoms or by *earlier* arithmetic results: + +```elixir +# Safe: Z is bound by the first constraint +total(X, Z) :- value(X, A), add(A, 1, Z) + +# Unsafe: W references Z before Z is computed +bad(X, W) :- value(X, A), add(W, 1, Z), add(A, 2, Z) +``` + +**3. No wildcards in rule heads.** Wildcards match anything without binding, so they can't appear in the head position. + +The safety checker processes constraints in order, threading the bound set: + +```elixir +{errors, _final_bound} = + constraints + |> Enum.with_index() + |> Enum.reduce({errors, body_bound}, fn {c, c_idx}, {acc_errors, bound} -> + check_constraint(c, c_idx, bound, rule_index, acc_errors) + end) +``` + +Arithmetic constraints extend the bound set with their result variable, making it available for subsequent constraints. This sequential threading means constraint order matters — just as it does in Datalog's evaluation model. + +## Negation and Safety Interaction + +Variables that appear only in negative body atoms are **not** bound. This means: + +```elixir +rule invalid(X) do + not_ married(X, _) # ERROR: X not bound by any positive atom +end +``` + +The fix is to add a positive atom that binds the variable: + +```elixir +rule bachelor(X) do + male(X) # binds X + not_ married(X, _) # OK: X is already bound +end +``` + +This is the range-restriction property: every head variable must be bound by a positive body atom or an arithmetic constraint. Negative atoms can only filter — they cannot introduce new bindings. + +## What's Coming in v0.5.0 + +- **Aggregates** — the syntax `agg(:count, X)` is already parsed but returns `%UnsupportedFeature{feature: :aggregates}`. The implementation will add count, sum, min, and max with proper safety checks. +- **Magic sets / demand-driven evaluation** — goal-directed evaluation that computes only facts relevant to a specific query, instead of the full fixpoint. +- **General predicates as BEAM callbacks** — arbitrary Elixir functions as predicates, extending Datalog's reasoning with Elixir's computation while maintaining stratification and safety. + +These features will expand what's expressible while preserving Datalog's guarantees: termination, deterministic output, and compile-time validation. \ No newline at end of file diff --git a/docs/migration_dsl.md b/docs/migration_dsl.md new file mode 100644 index 0000000..751b541 --- /dev/null +++ b/docs/migration_dsl.md @@ -0,0 +1,209 @@ +# Migrating from the Builder API to the DSL + +ExDatalog v0.4.0 introduces an Ecto-inspired Schema DSL as the recommended way +to define Datalog programs. The builder API (`Program.add_relation`, +`Program.add_fact`, `Program.add_rule`) remains fully supported as the +lower-level interface. + +This guide shows how to migrate existing builder-API code to the DSL. + +## Relation Declarations + +### Builder API + +```elixir +program = + Program.new() + |> Program.add_relation("parent", [:atom, :atom]) + |> Program.add_relation("ancestor", [:atom, :atom]) +``` + +### DSL + +```elixir +defmodule FamilyRules do + use ExDatalog.Schema + + relation :parent do + field :parent, :atom + field :child, :atom + end + + relation :ancestor do + field :ancestor, :atom + field :descendant, :atom + end +end +``` + +## Facts + +### Builder API + +```elixir +program = + program + |> Program.add_fact("parent", [:alice, :bob]) + |> Program.add_fact("parent", [:bob, :carol]) +``` + +### DSL + +```elixir +fact parent(:alice, :bob) +fact parent(:bob, :carol) + +# Or bulk: +facts :parent do + row :alice, :bob + row :bob, :carol +end +``` + +## Rules + +### Builder API (struct-based) + +```elixir +Program.add_rule(program, + Rule.new( + Atom.new("ancestor", [Term.var("X"), Term.var("Y")]), + [{:positive, Atom.new("parent", [Term.var("X"), Term.var("Y")])}] + ) +) +``` + +### Builder API (shorthand) + +```elixir +Program.add_rule(program, + {"ancestor", [:X, :Y]}, + [{:positive, {"parent", [:X, :Y]}}] +) +``` + +### DSL + +```elixir +rule ancestor(X, Y) do + parent(X, Y) +end +``` + +Lowercase variables are logic variables. Atoms starting with `:` are constants. +`_` is a wildcard. + +## Negation + +### Builder API + +```elixir +Program.add_rule(program, {"bachelor", [:X]}, [ + {:positive, {"male", [:X]}}, + {:negative, {"married", [:X, :_]}} +]) +``` + +### DSL + +```elixir +rule bachelor(P) do + male(P) + not_ married(P, _) +end +``` + +## Constraints + +### Builder API + +```elixir +Program.add_rule(program, + {"high_earner", [:X]}, + [{:positive, {"income", [:X, :S]}}], + [{:gt, :S, 100_000}] +) +``` + +### DSL + +```elixir +rule high_earner(P) do + income(P, S) + gt(S, 100_000) +end +``` + +Supported constraint names in the DSL: `eq`, `neq`, `gt`, `gte`, `lt`, `lte`, +`add`, `sub`, `mul`, `div`, `is_integer`, `is_binary`, `is_atom`, +`starts_with`, `contains`, `member`. + +## Materialization + +### Builder API + +```elixir +{:ok, knowledge} = ExDatalog.materialize(program) +``` + +### DSL + +```elixir +{:ok, knowledge} = FamilyRules.materialize() +``` + +Both accept options: `FamilyRules.materialize(iteration_limit: 100)` + +## Queries + +The DSL adds a query capability that doesn't have a builder-API equivalent: + +```elixir +query :descendants_of_alice do + find Y + where ancestor(:alice, Y) +end + +{:ok, knowledge} = FamilyRules.materialize() +FamilyRules.query(:descendants_of_alice, knowledge) +#=> [:bob, :carol] +``` + +The equivalent builder-API code would use `Knowledge.match/3` directly: + +```elixir +knowledge +|> Knowledge.match("ancestor", [:alice, :_]) +|> MapSet.to_list() +|> Enum.map(fn {_, y} -> y end) +``` + +## Mixing Both APIs + +The DSL's `program/0` returns a standard `ExDatalog.Program` struct. You can +extend it with the builder API: + +```elixir +program = FamilyRules.program() + +extended = + program + |> Program.add_fact("parent", [:carol, :dave]) + |> Program.add_rule( + Rule.new( + Atom.new("ancestor", [Term.var("X"), Term.var("Z")]), + [{:positive, Atom.new("parent", [Term.var("X"), Term.var("Y")])}, + {:positive, Atom.new("ancestor", [Term.var("Y"), Term.var("Z")])}] + ) + ) + +{:ok, knowledge} = ExDatalog.materialize(extended) +``` + +## Variable Conventions + +| Pattern | Builder API | DSL | +|---------|-------------|-----| +| Logic variable | `Term.var("X")` or `:X` | `X` (lowercase in DSL) | +| Constant | `Term.from(:alice)` or `:alice` | `:alice` | +| Wildcard | `Term.from(:_)` or `:_` | `_` or `wildcard()` | \ No newline at end of file diff --git a/lib/ex_datalog.ex b/lib/ex_datalog.ex index 267ebfa..f89a815 100644 --- a/lib/ex_datalog.ex +++ b/lib/ex_datalog.ex @@ -55,7 +55,7 @@ defmodule ExDatalog do ## Options -`materialize/2` and `evaluate/2` accept: + `materialize/2` and `evaluate/2` accept: - `engine` — backend module (default: `ExDatalog.Engine.Naive`) - `storage` — storage module (default: `ExDatalog.Storage.Map`) diff --git a/lib/ex_datalog/dsl/compile_error.ex b/lib/ex_datalog/dsl/compile_error.ex new file mode 100644 index 0000000..59c4459 --- /dev/null +++ b/lib/ex_datalog/dsl/compile_error.ex @@ -0,0 +1,33 @@ +defmodule ExDatalog.DSL.CompileError do + @moduledoc """ + Raised when a DSL macro cannot be compiled. + + Errors include clear descriptions of what went wrong and where. + + ## Examples + + iex> raise ExDatalog.DSL.CompileError, message: "relation :parent is not declared" + ** (ExDatalog.DSL.CompileError) relation :parent is not declared + + iex> err = ExDatalog.DSL.CompileError.exception("unsafe variable z") + iex> err.message + "unsafe variable z" + + iex> ExDatalog.DSL.CompileError.exception(message: "bad rule") + %ExDatalog.DSL.CompileError{message: "bad rule"} + """ + + defexception [:message] + + @type t :: %__MODULE__{message: String.t()} + + @impl true + def exception(opts) when is_list(opts) do + message = Keyword.get(opts, :message, "DSL compilation error") + %__MODULE__{message: message} + end + + def exception(message) when is_binary(message) do + %__MODULE__{message: message} + end +end diff --git a/lib/ex_datalog/engine/naive.ex b/lib/ex_datalog/engine/naive.ex index b6145e2..117c8a6 100644 --- a/lib/ex_datalog/engine/naive.ex +++ b/lib/ex_datalog/engine/naive.ex @@ -337,7 +337,7 @@ defmodule ExDatalog.Engine.Naive do ctx ) do Enum.reduce(strata, {state, 0, base_origins, :fixpoint}, fn %IR.Stratum{index: stratum_idx}, - {s, total_iter, origins, term} -> + {s, total_iter, origins, term} -> stratum_rules = Enum.filter(rules, fn r -> r.stratum == stratum_idx end) if stratum_rules == [] do diff --git a/lib/ex_datalog/schema.ex b/lib/ex_datalog/schema.ex new file mode 100644 index 0000000..3903aa7 --- /dev/null +++ b/lib/ex_datalog/schema.ex @@ -0,0 +1,850 @@ +defmodule ExDatalog.UnsupportedFeature do + @moduledoc """ + Returned when a DSL feature is recognized but not yet implemented. + + The `feature` field names the unsupported feature. + The `planned_for` field indicates the target release. + + ## Examples + + iex> uf = %ExDatalog.UnsupportedFeature{feature: :aggregates, planned_for: "v0.6.0"} + iex> uf.feature + :aggregates + iex> uf.planned_for + "v0.6.0" + """ + + @enforce_keys [:feature, :planned_for] + defstruct [:feature, :planned_for] + + @type t :: %__MODULE__{feature: atom(), planned_for: String.t()} +end + +defmodule ExDatalog.Schema do + @moduledoc """ + An Ecto-inspired DSL for defining Datalog programs. + + `use ExDatalog.Schema` in a module to declare relations, facts, rules, + and queries. The module then exposes `program/0`, `materialize/0`, + and `query/2` functions. + + ## Example + + defmodule FamilyRules do + use ExDatalog.Schema + + relation :parent do + field :parent, :atom + field :child, :atom + end + + relation :ancestor do + field :ancestor, :atom + field :descendant, :atom + end + + fact parent(:alice, :bob) + fact parent(:bob, :carol) + + rule ancestor(x, y) do + parent(x, y) + end + + rule ancestor(x, z) do + parent(x, y) + ancestor(y, z) + end + + query :descendants_of_alice do + find y + where ancestor(:alice, y) + end + end + + {:ok, knowledge} = FamilyRules.materialize() + FamilyRules.query(:descendants_of_alice, knowledge) + #=> [:bob, :carol] + + ## Relation DSL + + Relations declare named schemas with typed fields: + + relation :parent do + field :parent, :atom + field :child, :atom + end + + Supported field types: `:atom`, `:integer`, `:string`, `:any`. + + ## Fact DSL + + Facts assert ground tuples: + + fact parent(:alice, :bob) + + Bulk facts: + + facts :parent do + row :alice, :bob + row :bob, :carol + end + + ## Rule DSL + + Rules derive new facts. Lowercase identifiers are logic variables, + atoms starting with `:` are constants, and `_` is a wildcard: + + rule ancestor(x, y) do + parent(x, y) + end + + Negation uses `not_`: + + rule bachelor(p) do + male(p) + not_ married(p, _) + end + + Constraints use named predicates: + + rule high_earner(p) do + income(p, salary) + gt(salary, 100_000) + end + + Supported constraints: `eq`, `neq`, `gt`, `gte`, `lt`, `lte`, + `add`, `sub`, `mul`, `div`, `is_integer`, `is_binary`, `is_atom`, + `starts_with`, `contains`, `member`. + + ## Query DSL + + Queries define named post-materialization lookups: + + query :all_ancestors do + find x, y + where ancestor(x, y) + end + + Queries operate on materialized knowledge and use `Knowledge.match/3` + internally. + + ## Aggregate Syntax (Preview) + + Aggregates are parsed but not yet executable: + + rule employee_count(dept, agg(:count, emp)) do + employee(emp, dept) + end + + Attempting to materialize a program with aggregates returns + `{:error, %ExDatalog.UnsupportedFeature{feature: :aggregates}}`. + + ## Backward Compatibility + + The DSL compiles into the existing `Program` builder API. All existing + builder APIs (`Program.add_relation/3`, `Program.add_fact/3`, + `Program.add_rule/2,3,4`, `ExDatalog.materialize/2`) continue to work. + """ + + defmodule Field do + @moduledoc false + @enforce_keys [:name, :type] + defstruct [:name, :type] + + @type t :: %__MODULE__{name: atom(), type: ExDatalog.Program.ir_type()} + end + + defmodule RelationMeta do + @moduledoc false + @enforce_keys [:name, :fields] + defstruct [:name, :fields] + + @type t :: %__MODULE__{name: atom(), fields: [ExDatalog.Schema.Field.t()]} + end + + defmodule QueryMeta do + @moduledoc false + @enforce_keys [:name, :relation, :pattern, :find_vars] + defstruct [:name, :relation, :pattern, :find_vars] + + @type t :: %__MODULE__{ + name: atom(), + relation: String.t(), + pattern: [term()], + find_vars: [String.t()] + } + end + + @doc false + defmacro __using__(_opts) do + quote do + import ExDatalog.Schema, + only: [relation: 2, fact: 1, facts: 2, rule: 2, query: 2, wildcard: 0] + + Module.register_attribute(__MODULE__, :ex_datalog_relations, accumulate: true) + Module.register_attribute(__MODULE__, :ex_datalog_facts, accumulate: true) + Module.register_attribute(__MODULE__, :ex_datalog_rules, accumulate: true) + Module.register_attribute(__MODULE__, :ex_datalog_queries, accumulate: true) + + @before_compile ExDatalog.Schema + end + end + + @doc false + defmacro __before_compile__(env) do + relations = Module.get_attribute(env.module, :ex_datalog_relations) |> Enum.reverse() + facts = Module.get_attribute(env.module, :ex_datalog_facts) |> Enum.reverse() + rules = Module.get_attribute(env.module, :ex_datalog_rules) |> Enum.reverse() + queries = Module.get_attribute(env.module, :ex_datalog_queries) |> Enum.reverse() + + relation_names = MapSet.new(relations, fn rel -> Atom.to_string(rel.name) end) + + Enum.each(facts, fn {rel_name, _values} -> + unless MapSet.member?(relation_names, Atom.to_string(rel_name)) do + raise ExDatalog.DSL.CompileError, + message: + "fact #{rel_name}: relation #{inspect(Atom.to_string(rel_name))} is not declared" + end + end) + + Enum.each(queries, fn q -> + unless MapSet.member?(relation_names, q.relation) do + raise ExDatalog.DSL.CompileError, + message: "query #{q.name}: relation #{inspect(q.relation)} is not declared" + end + end) + + quote do + @doc """ + Returns the `ExDatalog.Program` built from this schema's relations, + facts, and rules. + """ + @spec program() :: ExDatalog.Program.t() + def program do + ExDatalog.Schema.__build_program__( + unquote(Macro.escape(relations)), + unquote(Macro.escape(facts)), + unquote(Macro.escape(rules)) + ) + end + + @doc """ + Materializes this schema's program. Accepts the same options as + `ExDatalog.materialize/2`. + """ + @spec materialize(keyword()) :: {:ok, ExDatalog.Knowledge.t()} | {:error, term()} + def materialize(opts \\ []) do + ExDatalog.materialize(program(), opts) + end + + @doc """ + Returns a map of query names to their metadata. + """ + @spec queries() :: %{atom() => ExDatalog.Schema.QueryMeta.t()} + def queries do + unquote(Macro.escape(Map.new(queries, fn q -> {q.name, q} end))) + end + + @doc """ + Executes a named query against materialized knowledge. + + Returns a list of results. For single-column `find`, returns a list of + values. For multi-column `find`, returns a list of tuples. + """ + @spec query(atom(), ExDatalog.Knowledge.t()) :: [term()] + def query(name, knowledge) do + ExDatalog.Schema.__execute_query__(name, knowledge, unquote(Macro.escape(queries))) + end + end + end + + @doc false + def __build_program__(relations, facts, rules) do + program = ExDatalog.Program.new() + + program = + Enum.reduce(relations, program, fn rel_meta, acc -> + types = Enum.map(rel_meta.fields, & &1.type) + ExDatalog.Program.add_relation(acc, Atom.to_string(rel_meta.name), types) + end) + + program = + Enum.reduce(facts, program, fn {rel_name, values}, acc -> + result = ExDatalog.Program.add_fact(acc, Atom.to_string(rel_name), values) + + case result do + {:error, msg} -> + raise ExDatalog.DSL.CompileError, + message: "fact #{rel_name}(#{Enum.map_join(values, ", ", &inspect/1)}): #{msg}" + + prog -> + prog + end + end) + + program = + Enum.reduce(rules, program, fn rule_data, acc -> + {{head_rel, head_terms}, body_literals, constraints} = rule_data + + head_atom = %ExDatalog.Atom{ + relation: head_rel, + terms: Enum.map(head_terms, &term_from_parsed/1) + } + + body = + Enum.map(body_literals, fn + {:positive, %ExDatalog.Atom{} = atom} -> + {:positive, + %ExDatalog.Atom{atom | terms: Enum.map(atom.terms, &term_from_parsed/1)}} + + {:negative, %ExDatalog.Atom{} = atom} -> + {:negative, + %ExDatalog.Atom{atom | terms: Enum.map(atom.terms, &term_from_parsed/1)}} + end) + + rule = ExDatalog.Rule.new(head_atom, body, constraints) + + case ExDatalog.Program.add_rule(acc, rule) do + {:error, msg} -> + raise ExDatalog.DSL.CompileError, + message: "rule #{head_rel}/#{length(head_terms)}: #{msg}" + + prog -> + prog + end + end) + + validate_rules!(program, rules) + program + end + + defp validate_rules!(_program, rules) do + Enum.each(rules, fn {{head_rel, head_terms}, body_literals, constraints} -> + head_vars = + head_terms |> Enum.filter(&match?({:var, _}, &1)) |> Enum.map(fn {:var, n} -> n end) + + positive_vars = + body_literals + |> Enum.flat_map(fn + {:positive, %ExDatalog.Atom{terms: terms}} -> + Enum.flat_map(terms, fn + {:var, n} -> [n] + _ -> [] + end) + + _ -> + [] + end) + + constraint_vars = + Enum.flat_map(constraints, fn + %ExDatalog.Constraint{result: {:var, n}} -> [n] + %ExDatalog.Constraint{} -> [] + _ -> [] + end) + + safe_vars = (positive_vars ++ constraint_vars) |> Enum.uniq() + unsafe = head_vars -- safe_vars + + if unsafe != [] do + raise ExDatalog.DSL.CompileError, + message: + "rule #{head_rel}/#{length(head_terms)}: variable(s) #{Enum.join(unsafe, ", ")} appear in the rule head but not in any positive body literal" + end + end) + + :ok + end + + defp term_from_parsed({:var, name}), do: ExDatalog.Term.var(name) + defp term_from_parsed({:const, value}), do: ExDatalog.Term.from(value) + defp term_from_parsed(:wildcard), do: ExDatalog.Term.from(:_) + + @doc false + def __execute_query__(name, knowledge, queries) when is_list(queries) do + query_map = Map.new(queries, fn q -> {q.name, q} end) + __execute_query__(name, knowledge, query_map) + end + + def __execute_query__(name, knowledge, queries) when is_map(queries) do + case Map.get(queries, name) do + nil -> + raise ArgumentError, "unknown query #{inspect(name)}" + + q -> + pattern = Enum.map(q.pattern, &query_term_to_pattern/1) + matched = ExDatalog.Knowledge.match(knowledge, q.relation, pattern) + + matched + |> MapSet.to_list() + |> Enum.sort() + |> Enum.map(fn tuple -> + project_tuple(tuple, q.find_vars, q.pattern) + end) + end + end + + defp project_tuple(tuple, find_vars, pattern) do + positions = + find_vars + |> Enum.map(fn var_name -> + Enum.find_index(pattern, fn + {:var, ^var_name} -> true + _ -> false + end) + end) + + case positions do + [nil] -> + tuple + + [single_pos] when is_integer(single_pos) -> + elem(tuple, single_pos) + + _ when is_list(positions) -> + positions + |> Enum.filter(&(&1 != nil)) + |> Enum.map(fn pos -> elem(tuple, pos) end) + |> List.to_tuple() + end + end + + defp query_term_to_pattern(:wildcard), do: :_ + defp query_term_to_pattern({:var, _}), do: :_ + defp query_term_to_pattern({:const, value}), do: value + + # --- Relation macro --- + + @doc """ + Declares a relation with typed fields. + + relation :parent do + field :parent, :atom + field :child, :atom + end + + Supported types: `:atom`, `:integer`, `:string`, `:any`. + """ + defmacro relation(name, do: block) do + quote do + ExDatalog.Schema.__define_relation__( + __MODULE__, + unquote(name), + unquote(Macro.escape(block)) + ) + end + end + + @doc false + def __define_relation__(module, name, block) do + fields = extract_fields(block) + + Module.put_attribute(module, :ex_datalog_relations, %ExDatalog.Schema.RelationMeta{ + name: name, + fields: fields + }) + end + + defp extract_fields({:__block__, _, expressions}) do + Enum.flat_map(expressions, &extract_field/1) + end + + defp extract_fields({:field, _, [name, type]}) do + [%ExDatalog.Schema.Field{name: name, type: type}] + end + + defp extract_fields(single) do + extract_field(single) + end + + defp extract_field({:field, _, [name, type]}) do + [%ExDatalog.Schema.Field{name: name, type: type}] + end + + defp extract_field(_), do: [] + + # --- Fact macros --- + + @doc """ + Declares a ground fact. + + fact parent(:alice, :bob) + + The relation must be declared before the fact. + """ + defmacro fact(rel_call) do + {rel_name, args} = parse_rel_call(rel_call) + + quote do + ExDatalog.Schema.__register_fact__( + __MODULE__, + unquote(rel_name), + unquote(Macro.escape(args)) + ) + end + end + + @doc false + def __register_fact__(module, rel_name, args) do + Module.put_attribute(module, :ex_datalog_facts, {rel_name, args}) + end + + @doc """ + Declares multiple facts for the same relation. + + facts :parent do + row :alice, :bob + row :bob, :carol + end + """ + defmacro facts(rel_name, do: block) do + rows = extract_rows(block) + + quote do + ExDatalog.Schema.__register_facts__( + __MODULE__, + unquote(rel_name), + unquote(Macro.escape(rows)) + ) + end + end + + @doc false + def __register_facts__(module, rel_name, rows) do + Enum.each(rows, fn args -> + Module.put_attribute(module, :ex_datalog_facts, {rel_name, args}) + end) + end + + defp extract_rows({:__block__, _, expressions}) do + Enum.flat_map(expressions, &extract_row/1) + end + + defp extract_rows({:row, _, args}) do + [args] + end + + defp extract_rows(_), do: [] + + defp extract_row({:row, _, args}), do: [args] + defp extract_row(_), do: [] + + # --- Rule macro --- + + @doc """ + Declares a Datalog rule. + + Lowercase identifiers in the head and body are logic variables. + Atoms starting with `:` are constants. `_` is a wildcard. + + rule ancestor(x, y) do + parent(x, y) + end + + Negation uses `not_`: + + rule bachelor(p) do + male(p) + not_ married(p, _) + end + + Constraints use named predicates: + + rule high_earner(p) do + income(p, salary) + gt(salary, 100_000) + end + """ + defmacro rule(head, do: body) do + quote do + ExDatalog.Schema.__register_rule__( + __MODULE__, + unquote(Macro.escape(head)), + unquote(Macro.escape(body)) + ) + end + end + + @doc false + def __register_rule__(module, head, body) do + {head_rel, head_terms} = parse_rule_head(head) + {body_literals, constraints} = parse_rule_body(body) + + rule_data = {{head_rel, head_terms}, body_literals, constraints} + Module.put_attribute(module, :ex_datalog_rules, rule_data) + end + + defp parse_rule_head({head_atom, _context, args}) when is_atom(head_atom) and is_list(args) do + {Atom.to_string(head_atom), Enum.map(args, &parse_term/1)} + end + + defp parse_rule_head({head_atom, _context, nil}) when is_atom(head_atom) do + {Atom.to_string(head_atom), []} + end + + defp parse_rule_head(_other) do + raise CompileError, + description: "rule head must be a relation call like `ancestor(x, y)`" + end + + defp parse_term({:wildcard, _, []}) do + :wildcard + end + + defp parse_term({var_name, _, nil}) when is_atom(var_name) do + var_str = Atom.to_string(var_name) + + cond do + var_str == "_" -> :wildcard + var_str =~ ~r/^[A-Z]/ -> {:var, var_str} + true -> {:const, var_name} + end + end + + defp parse_term({:__aliases__, _, [alias_name]}) when is_atom(alias_name) do + {:var, Atom.to_string(alias_name)} + end + + defp parse_term(atom) when is_atom(atom) do + Atom.to_string(atom) |> create_term() + end + + defp parse_term(integer) when is_integer(integer), do: {:const, integer} + defp parse_term(string) when is_binary(string), do: {:const, string} + + defp parse_term(other) do + raise CompileError, + description: "unsupported term in DSL: #{inspect(other)}" + end + + defp create_term(name) when is_binary(name) do + cond do + name == "_" -> :wildcard + String.match?(name, ~r/^[A-Z]/) -> {:var, name} + true -> {:const, String.to_atom(name)} + end + end + + defp parse_rule_body({:__block__, _, expressions}) do + expressions + |> Enum.map(&parse_body_call/1) + |> Enum.reduce({[], []}, fn + {:positive, atom}, {body, constraints} -> + {body ++ [{:positive, atom}], constraints} + + {:negative, atom}, {body, constraints} -> + {body ++ [{:negative, atom}], constraints} + + {:constraint, c}, {body, constraints} -> + {body, constraints ++ [c]} + + {:aggregate, agg}, {body, constraints} -> + {body, constraints ++ [agg]} + end) + end + + defp parse_rule_body(single_expr) do + parse_rule_body({:__block__, [], [single_expr]}) + end + + defp parse_body_call({:not_, _, [rel_call]}) do + {rel_name, args} = parse_rel_call(rel_call) + terms = Enum.map(args, &parse_term/1) + {:negative, %ExDatalog.Atom{relation: Atom.to_string(rel_name), terms: terms}} + end + + defp parse_body_call({:not_, _, [rel_call, _opts]}) do + parse_body_call({:not_, [], [rel_call]}) + end + + defp parse_body_call({:agg, _, args}) when is_list(args) do + {:aggregate, %ExDatalog.UnsupportedFeature{feature: :aggregates, planned_for: "v0.6.0"}} + end + + constraint_ops = [ + :eq, + :neq, + :gt, + :gte, + :lt, + :lte, + :add, + :sub, + :mul, + :div, + :is_integer, + :is_binary, + :is_atom, + :starts_with, + :contains, + :member + ] + + Enum.each(constraint_ops, fn op -> + defp parse_body_call({unquote(op), _, args}) when is_list(args) do + {:constraint, build_constraint(unquote(op), args)} + end + end) + + defp parse_body_call({rel_atom, _, args}) when is_atom(rel_atom) and is_list(args) do + terms = Enum.map(args, &parse_term/1) + {:positive, %ExDatalog.Atom{relation: Atom.to_string(rel_atom), terms: terms}} + end + + defp parse_body_call({rel_atom, _, nil}) when is_atom(rel_atom) do + {:positive, %ExDatalog.Atom{relation: Atom.to_string(rel_atom), terms: []}} + end + + defp parse_body_call(other) do + raise CompileError, + description: "unsupported body expression in rule: #{inspect(other)}" + end + + constraint_2_arity = [:eq, :neq, :gt, :gte, :lt, :lte, :starts_with, :contains] + constraint_3_arity = [:add, :sub, :mul, :div] + constraint_1_arity = [:is_integer, :is_binary, :is_atom] + constraint_member = [:member] + + Enum.each(constraint_2_arity, fn op -> + defp build_constraint(unquote(op), [left, right]) do + ExDatalog.Constraint.from_tuple({unquote(op), parse_term(left), parse_term(right)}) + end + end) + + Enum.each(constraint_3_arity, fn op -> + defp build_constraint(unquote(op), [left, right, result]) do + ExDatalog.Constraint.from_tuple( + {unquote(op), parse_term(left), parse_term(right), parse_term(result)} + ) + end + end) + + Enum.each(constraint_1_arity, fn op -> + defp build_constraint(unquote(op), [arg]) do + ExDatalog.Constraint.from_tuple({unquote(op), parse_term(arg)}) + end + end) + + Enum.each(constraint_member, fn op -> + defp build_constraint(unquote(op), [elem, list_or_var]) do + left = parse_term(elem) + right = if is_list(list_or_var), do: {:const, list_or_var}, else: parse_term(list_or_var) + ExDatalog.Constraint.from_tuple({unquote(op), left, right}) + end + end) + + defp build_constraint(op, args) do + raise CompileError, + description: "unsupported constraint #{op}/#{length(args)}: #{inspect(args)}" + end + + # --- Query macro --- + + @doc """ + Declares a named post-materialization query. + + query :descendants_of_alice do + find y + where ancestor(:alice, y) + end + + The `find` clause specifies which variables to extract. + The `where` clause specifies the relation and pattern to match. + """ + defmacro query(name, do: block) do + quote do + ExDatalog.Schema.__register_query__(__MODULE__, unquote(name), unquote(Macro.escape(block))) + end + end + + @doc false + def __register_query__(module, name, block) do + {find_vars, relation, pattern} = parse_query_block(block) + + Module.put_attribute(module, :ex_datalog_queries, %ExDatalog.Schema.QueryMeta{ + name: name, + relation: relation, + pattern: pattern, + find_vars: find_vars + }) + end + + defp parse_query_block({:__block__, _, expressions}) do + find_vars = [] + relation = nil + + Enum.reduce(expressions, {find_vars, relation}, fn + {:find, _, vars}, {_, rel} -> + vars_list = if is_list(vars), do: vars, else: [vars] + var_names = Enum.map(vars_list, &extract_var_name/1) + {var_names, rel} + + {:where, _, [{rel_atom, _, args}]}, {find_vars, _} -> + {find_vars, Atom.to_string(rel_atom), Enum.map(args, &parse_query_term/1)} + + {:where, _, [{rel_atom, _, nil}]}, {find_vars, _} -> + {find_vars, Atom.to_string(rel_atom), []} + end) + end + + defp parse_query_block({:find, _, [find_var]}) do + parse_query_block({:__block__, [], [{:find, [], [find_var]}, {:where, [], [{:_, [], nil}]}]}) + end + + defp parse_query_term({:__aliases__, _, [alias_name]} = _var) when is_atom(alias_name) do + {:var, Atom.to_string(alias_name)} + end + + defp parse_query_term({_var_name, _, nil} = var) when is_tuple(var) do + parse_term(var) + end + + defp parse_query_term(atom) when is_atom(atom) do + parse_term(atom) + end + + defp parse_query_term(integer) when is_integer(integer), do: {:const, integer} + defp parse_query_term(string) when is_binary(string), do: {:const, string} + + defp extract_var_name({:__aliases__, _, [alias_name]}) when is_atom(alias_name) do + Atom.to_string(alias_name) + end + + defp extract_var_name({var_name, _, nil}) when is_atom(var_name) do + Atom.to_string(var_name) + end + + # --- Helpers --- + + defp parse_rel_call({rel_atom, _, args}) when is_atom(rel_atom) and is_list(args) do + {rel_atom, args} + end + + defp parse_rel_call({rel_atom, _, nil}) when is_atom(rel_atom) do + {rel_atom, []} + end + + defp parse_rel_call(rel_atom) when is_atom(rel_atom) do + {rel_atom, []} + end + + @doc """ + Explicit wildcard for use in rule bodies and queries. + + Inside DSL rule and query bodies, `_` is treated as a wildcard. + If Elixir's treatment of `_` as a special form causes issues, + use `wildcard()` as an explicit alternative. + + ## Examples + + iex> ExDatalog.Schema.wildcard() + :wildcard + + rule bachelor(p) do + male(p) + not_ married(p, wildcard()) + end + """ + @spec wildcard() :: :wildcard + def wildcard, do: :wildcard +end diff --git a/livebooks/ex_datalog_dsl.livemd b/livebooks/ex_datalog_dsl.livemd new file mode 100644 index 0000000..28f2dfc --- /dev/null +++ b/livebooks/ex_datalog_dsl.livemd @@ -0,0 +1,551 @@ +# ExDatalog DSL Tutorial + +```elixir +Mix.install([ + {:ex_datalog, path: Path.expand("..", __DIR__), env: :prod}, +]) +``` + +## Section + +Welcome to the ExDatalog DSL tutorial! This livebook walks through the `use ExDatalog.Schema` DSL (v0.4.0) for declaring Datalog programs in Elixir. + +` +Mix.install([{:ex_datalog, "~> 0.4.0"}]) +` + +## 1. Relations — Declaring Typed Schemas + +Relations are the core data declarations in a Datalog program. Each relation defines a named schema with typed fields, similar to database tables. + +Supported field types: `:atom`, `:integer`, `:string`, `:any`. + +```elixir +defmodule Family do + use ExDatalog.Schema + + relation :parent do + field :parent, :atom + field :child, :atom + end + + relation :ancestor do + field :ancestor, :atom + field :descendant, :atom + end + + relation :income do + field :person, :atom + field :amount, :integer + end + + relation :label do + field :node, :atom + field :text, :string + end +end +``` + +Inspect the program to see the declared relations: + +```elixir +program = Family.program() +program.relations +``` + +You can also mix field types within a single relation. The `:any` type allows unrestricted values: + +```elixir +defmodule VarTypes do + use ExDatalog.Schema + + relation :payload do + field :key, :atom + field :value, :any + end +end + +VarTypes.program().relations +``` + +## 2. Facts — Single Facts and Bulk Facts + +Facts assert ground (variable-free) tuples into relations. You can declare them one at a time with `fact` or in bulk with `facts`. + +### Single facts + +Use `fact relation_name(const1, const2, ...)` to insert individual tuples. Atom constants start with `:`. + +```elixir +defmodule SingleFacts do + use ExDatalog.Schema + + relation :parent do + field :parent, :atom + field :child, :atom + end + + fact parent(:alice, :bob) + fact parent(:bob, :carol) +end + +SingleFacts.program().facts +|> Enum.map(fn {rel, vals} -> {rel, vals} end) +``` + +### Bulk facts + +When you have many rows, use the `facts` macro with `row` syntax: + +```elixir +defmodule BulkFacts do + use ExDatalog.Schema + + relation :edge do + field :from, :atom + field :to, :atom + end + + facts :edge do + row :a, :b + row :b, :c + row :c, :d + row :d, :e + end +end + +BulkFacts.program().facts +|> Enum.map(fn {rel, vals} -> {rel, vals} end) +``` + +## 3. Recursive Rules — Ancestor Transitive Closure + +Rules derive new facts from existing ones. Lowercase identifiers that start with an uppercase letter are **logic variables** (`X`, `Y`, `Z`); atoms starting with `:` are **constants**; `_` is a wildcard. + +The classic Datalog example is computing transitive closure — here, the `ancestor` relation from `parent`: + +```elixir +defmodule Ancestor do + use ExDatalog.Schema + + relation :parent do + field :parent, :atom + field :child, :atom + end + + relation :ancestor do + field :ancestor, :atom + field :descendant, :atom + end + + fact parent(:alice, :bob) + fact parent(:bob, :carol) + fact parent(:carol, :dave) + + # Base case: every parent is an ancestor + rule ancestor(X, Y) do + parent(X, Y) + end + + # Recursive case: if X is parent of Y, and Y is ancestor of Z, + # then X is ancestor of Z + rule ancestor(X, Z) do + parent(X, Y) + ancestor(Y, Z) + end +end + +{:ok, knowledge} = Ancestor.materialize() + +ancestor = ExDatalog.Knowledge.get(knowledge, "ancestor") +MapSet.to_list(ancestor) |> Enum.sort() +``` + +The result includes all ancestor pairs — both direct (`alice → bob`) and transitive (`alice → dave`). + +## 4. Materialization — Running Programs to Get Knowledge + +`materialize/0` (or `materialize/1` with options) validates, compiles, and evaluates the program in one step. It returns `{:ok, knowledge}` on success or `{:error, reason}` on failure. + +```elixir +defmodule Route do + use ExDatalog.Schema + + relation :link do + field :from, :atom + field :to, :atom + end + + relation :path do + field :from, :atom + field :to, :atom + end + + facts :link do + row :a, :b + row :b, :c + row :c, :d + end + + rule path(X, Y) do + link(X, Y) + end + + rule path(X, Z) do + link(X, Y) + path(Y, Z) + end +end + +{:ok, knowledge} = Route.materialize() +``` + +Check termination status — always verify that the fixpoint was reached: + +```elixir +knowledge.stats.termination +``` + +Inspect all derived `path` tuples: + +```elixir +ExDatalog.Knowledge.get(knowledge, "path") +|> MapSet.to_list() +|> Enum.sort() +``` + +You can also list all relation names and their sizes: + +```elixir +Enum.map(ExDatalog.Knowledge.relations(knowledge), fn rel -> + {rel, ExDatalog.Knowledge.size(knowledge, rel)} +end) +``` + +`materialize/1` also accepts options like `:max_iterations` and `:timeout_ms`: + +```elixir +{:ok, knowledge} = Route.materialize(max_iterations: 1000) +knowledge.stats.termination +``` + +## 5. Queries — Named Post-Materialization Queries with find/where + +The `query` macro defines named lookups against materialized knowledge. `find` specifies which variables to extract; `where` specifies the relation and pattern to match. + +```elixir +defmodule FamilyQueries do + use ExDatalog.Schema + + relation :parent do + field :parent, :atom + field :child, :atom + end + + relation :ancestor do + field :ancestor, :atom + field :descendant, :atom + end + + fact parent(:alice, :bob) + fact parent(:bob, :carol) + fact parent(:carol, :dave) + + rule ancestor(X, Y) do + parent(X, Y) + end + + rule ancestor(X, Z) do + parent(X, Y) + ancestor(Y, Z) + end + + query :all_ancestors do + find X, Y + where ancestor(X, Y) + end + + query :descendants_of_alice do + find Y + where ancestor(:alice, Y) + end +end + +{:ok, knowledge} = FamilyQueries.materialize() +``` + +Single-column `find` returns a list of values; multi-column `find` returns a list of tuples: + +```elixir +FamilyQueries.query(:descendants_of_alice, knowledge) +``` + +```elixir +FamilyQueries.query(:all_ancestors, knowledge) |> Enum.sort() +``` + +Inspect query metadata with `queries/0`: + +```elixir +FamilyQueries.queries() +``` + +## 6. Negation — Using not_ in Rule Bodies + +Datalog supports **stratified negation** via the `not_` prefix. A negated body literal excludes bindings that match. + +For example, find bachelors — people who are male but not married: + +```elixir +defmodule Bachelors do + use ExDatalog.Schema + + relation :male do + field :person, :atom + end + + relation :married do + field :person, :atom + field :spouse, :atom + end + + relation :bachelor do + field :person, :atom + end + + fact male(:bob) + fact male(:tom) + fact male(:dave) + + fact married(:tom, :sally) + fact married(:dave, :amy) + + rule bachelor(P) do + male(P) + not_ married(P, _) + end +end + +{:ok, knowledge} = Bachelors.materialize() + +bachelors = ExDatalog.Knowledge.get(knowledge, "bachelor") +MapSet.to_list(bachelors) +``` + +Only `bob` qualifies — `tom` and `dave` are married. The `_` in `not_ married(P, _)` is a wildcard that matches any spouse. + +If Elixir's treatment of `_` causes issues, you can use `wildcard()` explicitly: + +```elixir +# These are equivalent inside a rule body: +# not_ married(P, _) +# not_ married(P, wildcard()) +``` + +## 7. Constraints — Comparison, Arithmetic, Type Predicates + +Constraints filter or compute values within rule bodies. They are written as named predicates. + +### Comparison constraints + +Compare bound values: `eq`, `neq`, `gt`, `gte`, `lt`, `lte`. + +```elixir +defmodule HighEarners do + use ExDatalog.Schema + + relation :income do + field :person, :atom + field :salary, :integer + end + + relation :high_earner do + field :person, :atom + end + + fact income(:alice, 150_000) + fact income(:bob, 50_000) + fact income(:carol, 200_000) + fact income(:dave, 100_000) + + rule high_earner(P) do + income(P, S) + gt(S, 100_000) + end +end + +{:ok, knowledge} = HighEarners.materialize() +ExDatalog.Knowledge.get(knowledge, "high_earner") |> MapSet.to_list() +``` + +### Arithmetic constraints + +Compute new values: `add`, `sub`, `mul`, `div`. The third argument is a result variable that receives the computed value. + +```elixir +defmodule Compensation do + use ExDatalog.Schema + + relation :salary do + field :person, :atom + field :base, :integer + end + + relation :total_comp do + field :person, :atom + field :total, :integer + end + + fact salary(:alice, 100) + fact salary(:bob, 80) + + rule total_comp(P, T) do + salary(P, B) + add(B, 20, T) + end +end + +{:ok, knowledge} = Compensation.materialize() +ExDatalog.Knowledge.get(knowledge, "total_comp") |> MapSet.to_list() |> Enum.sort() +``` + +### Type predicates + +Check value types: `is_integer`, `is_binary`, `is_atom`. + +```elixir +defmodule TypeFilter do + use ExDatalog.Schema + + relation :value do + field :v, :any + end + + relation :int_val do + field :v, :any + end + + fact value(42) + fact value(:not_int) + + rule int_val(V) do + value(V) + is_integer(V) + end +end + +{:ok, knowledge} = TypeFilter.materialize() +ExDatalog.Knowledge.get(knowledge, "int_val") |> MapSet.to_list() +``` + +### Equality and inequality + +```elixir +defmodule EqualityDemo do + use ExDatalog.Schema + + relation :pair do + field :a, :atom + field :b, :atom + end + + relation :same do + field :x, :atom + end + + relation :different do + field :a, :atom + field :b, :atom + end + + fact pair(:x, :x) + fact pair(:a, :b) + fact pair(:b, :a) + + rule same(A) do + pair(A, B) + eq(A, B) + end + + rule different(A, B) do + pair(A, B) + neq(A, B) + end +end + +{:ok, knowledge} = EqualityDemo.materialize() +IO.puts("Same: #{inspect(MapSet.to_list(ExDatalog.Knowledge.get(knowledge, "same")))}") +IO.puts("Different: #{inspect(MapSet.to_list(ExDatalog.Knowledge.get(knowledge, "different")))}") +``` + +### String and membership constraints + +* `starts_with(var, "prefix")` — filters strings that start with a prefix +* `contains(var, "substring")` — filters strings that contain a substring +* `member(var, [:a, :b, :c])` — filters values in a constant list + +### Full constraint reference + +| Category | Operators | +| ---------- | ------------------------------------- | +| Comparison | `eq`, `neq`, `gt`, `gte`, `lt`, `lte` | +| Arithmetic | `add`, `sub`, `mul`, `div` | +| Type | `is_integer`, `is_binary`, `is_atom` | +| String | `starts_with`, `contains` | +| Membership | `member` | + +## 8. Aggregate Syntax Preview — Parsed but Not Executable + +Aggregates have a **parsed syntax** but are **not yet executable**. Writing a rule with `agg(:function, variable)` triggers parsing, but materialization returns an error: + +```elixir +defmodule AggPreview do + use ExDatalog.Schema + + relation :employee do + field :emp, :atom + field :dept, :atom + end + + relation :employee_count do + field :dept, :atom + field :total, :any + end + + facts :employee do + row :alice, :engineering + row :bob, :engineering + row :carol, :infra + end + + # This rule parses but cannot be materialized yet. + # The agg/2 syntax is reserved for a future release (v0.6.0). + # + # rule employee_count(D, agg(:count, E)) do + # employee(E, D) + # end +end +``` + +Attempting to materialize a program containing aggregates will return: + +```elixir +# {:error, %ExDatalog.UnsupportedFeature{feature: :aggregates, planned_for: "v0.6.0"}} +``` + +The `UnsupportedFeature` struct can be inspected directly: + +```elixir +%ExDatalog.UnsupportedFeature{feature: :aggregates, planned_for: "v0.6.0"} +|> IO.inspect() +``` + +When aggregates land in a future release, the syntax will look like: + +```elixir +# rule employee_count(D, agg(:count, E)) do +# employee(E, D) +# end +``` + +Until then, you can compute aggregations in Elixir by querying materialized knowledge and post-processing the results. diff --git a/livebook/examples.livemd b/livebooks/examples.livemd similarity index 100% rename from livebook/examples.livemd rename to livebooks/examples.livemd diff --git a/livebooks/examples.md b/livebooks/examples.md new file mode 100644 index 0000000..90a30da --- /dev/null +++ b/livebooks/examples.md @@ -0,0 +1,521 @@ +It is completely understandable that this looks a bit abstract! Datalog rules are basically "if-then" logical statements, but written backwards: `Result :- Conditions`. + +In this specific example, we are doing Code Taint Tracking. In cybersecurity, "taint" means untrusted data (like a user typing into a web form). The goal of this program is to see if untrusted data ever touches a sensitive part of our application (like executing a SQL database query). + +Here is the translation of the Elixir `Rule.new` blocks into plain English logic. + +The Setup: The Facts (Relations) +Before the rules run, the engine knows about three basic facts (which we fed it in the test data): + +- `source(X)`: Node `X` is a place where untrusted data enters (e.g., user input). + +- `sink(X)`: Node `X` is a dangerous place to execute untrusted data (e.g., a database). + +- `data_flow(A, B)`: Data moves directly from node `A` to node `B`. + +#### Rule 1: The Base Case (Where does taint start?) + + ```elixir + Rule.new( + Atom.new("tainted", [Term.var("Node")]), + [{:positive, Atom.new("source", [Term.var("Node")])}] + ) + ) + ``` + +Plain English: If a `Node` is a `source`, then that `Node` is `tainted`. + +How it works: This is the starting point. It looks at all our facts and says, "Ah, 'user_input' is a source. Therefore, I will now label 'user_input' as tainted." + +### Rule 2: The Recursive Case (How does taint spread?) + + ```elixir + Rule.new( + Atom.new("tainted", [Term.var("To")]), + [ + {:positive, Atom.new("tainted", [Term.var("From")])}, + {:positive, Atom.new("data_flow", [Term.var("From"), Term.var("To")])} + ] +) +``` + +Plain English: If a From node is already tainted, AND data flows from From to To, then the To node also becomes tainted. + +How it works: This is the engine's superpower. Because it's a recursive rule, the engine will run it over and over until it stops finding new things. +1. It knows `user_input` is tainted (from Rule 1). +2. It sees data flows from `user_input` to `parser`. So, `parser` becomes tainted. +3. It runs again! Now `parser` is tainted, and it sees data flows from `parser` to `sql_builder`. So, `sql_builder` becomes tainted. +4. It keeps spreading like an infection down the pipeline. + +#### Rule 3: The Vulnerability Check (Did something bad happen?) + +```elixir +Rule.new( + Atom.new("vulnerability", [Term.var("Sink")]), + [ + {:positive, Atom.new("sink", [Term.var("Sink")])}, + {:positive, Atom.new("tainted", [Term.var("Sink")])} + ] +) +``` + +Plain English: If a node is a `sink` (a sensitive execution point), AND that exact same node has been marked as `tainted`, then we have a `vulnerability`. + +How it works: After the "infection" finishes spreading in Rule 2, this rule checks the damage. It asks: "Did the taint ever reach our sensitive SQL query node?" If yes, it flags it as a vulnerability so the security team can fix it. + + + +#### Rule 4: Which data flows are safe? + +```elixir + Rule.new( + Atom.new("safe_flow", [Term.var("From"), Term.var("To")]), + [ + # Look at every data flow we know about + {:positive, Atom.new("data_flow", [Term.var("From"), Term.var("To")])}, + # Ensure the starting point is NOT infected + {:negative, Atom.new("tainted", [Term.var("From")])} + ] + ) +``` + +Plain English: We look at all known `data_flow` paths, and filter out any where the starting point has been marked as `tainted`. + +How it works: To do this, we have to follow Datalog's golden rule for negation: You cannot query for something that doesn't exist; you always have to start with a positive fact and filter it. + +Since we didn't define a master `node` list in the original program, the only place the engine knows `:safe_config` and `:logger` exist is inside the `data_flow` relation. + +So, instead of asking "Which nodes are safe?", we ask, "Which data flows are safe?" + +To give you a completely candid answer: **JSON is generally "easier" for me to read and write, but Datalog is far better when you want me to actually *reason* about relationships.** As an AI, I am ultimately a massive pattern-matching engine. The way I process information is fundamentally tied to the volume of data I was trained on and how my tokenization works. Because of that, both formats have distinct advantages depending on what you are asking me to do. + +Here is the breakdown of how I handle both formats. + +### Why JSON is Usually "Easier" (The Baseline) + +If your goal is just to have me summarize data, format output for a web app, or call an external API tool, JSON wins hands down. + +* **Massive Training Bias:** I have ingested billions of lines of JSON. My internal weights are highly optimized to predict the closing bracket `}` of a JSON object and format key-value pairs perfectly. +* **Hierarchical Understanding:** JSON natively represents trees. When data is cleanly nested (like an organization chart or a configuration file), it maps very well to how I hold context in a prompt. +* **Tool Calling:** The entire ecosystem of LLM tool-calling (including how I interact with external functions) is built on JSON schemas. + +**The downside of JSON:** If your data is a highly interconnected graph (like the Code Taint Tracking or Fraud Ring examples we discussed), JSON becomes a nightmare. It forces you to use arbitrary ID strings to link objects, making the payload bloated, repetitive, and prone to "context window exhaustion" where I might lose track of which ID belongs to what. + +--- + +### Why Datalog is Better for "Reasoning" (The Logic Layer) + +If your goal is to have me solve a puzzle, find a vulnerability, or deduce a relationship, Datalog is actually a much more powerful format to feed me. + +* **Density and Token Efficiency:** Datalog represents complex graphs with incredibly few tokens. `transfer(:acct_a, :acct_b)` is much faster for me to read than `{"source": "acct_a", "target": "acct_b", "type": "transfer"}` repeated a hundred times. +* **Explicit Constraints:** When you give me JSON, you have to explain the business logic in plain English (e.g., "A user has a role if their parent group has a role..."). When you give me Datalog, the rules *are* the prompt. It forces my attention onto the exact logical boundaries you want me to respect. +* **Minimizing Hallucinations:** Datalog's strict fact-and-rule structure gives me very little room to make things up. If I am tasked with generating a query or a new rule, the syntax is so constrained that I am less likely to hallucinate invalid fields. + +--- + +### The Ultimate Architecture: The Hand-Off + +In the real world, the most effective way to use an LLM with a complex system isn't strictly one or the other—it is a division of labor. + +Because I am great at generating code but sometimes struggle to reliably execute deep, multi-step logical deductions in my head (especially with hundreds of facts), the best architecture looks like this: + +1. **You give me:** Plain English questions and a JSON schema of your database. +2. **I generate:** The Datalog `Rule` or `Query` to answer your question. +3. **Your app:** Takes my Datalog, runs it through `ex_datalog` (which is 100% mathematically deterministic and never hallucinates), and gets the answer. +4. **Your app:** Passes the flat result back to me to summarize in a friendly sentence. + +If you were to build an AI agent for your Elixir application, would you be more interested in having me write the Datalog rules dynamically based on user prompts, or having me parse the final `Result.match` output into human-readable reports? + + +SAP’s allocation engines are legendary for their complexity, and modeling them in Datalog perfectly demonstrates how separating **business rules** (percentages, routing) from **execution logic** (the math and traversal) makes systems incredibly flexible. + +Instead of writing a massive Elixir `Enum.reduce` that hardcodes "Facilities costs go to IT, and then IT costs go to Product," we can write a pure Datalog rules engine. + +Here is how we model a **Multi-Stage Cost Assessment Engine** (combining your Document Splitting and Allocation rules). + +### The SAP Cost Allocation Engine + +This program takes raw financial documents posted to high-level cost centers (like "Facilities") and splits them down into granular departments using assessment percentages. It even handles **Level 2 (Transitive) Allocations**, where a receiving department (like IT) automatically re-allocates the costs it just received down to specific teams. + +```elixir +program = + Program.new() + # The raw invoice/journal entry + |> Program.add_relation("posted_cost", [:atom, :atom, :integer]) + # The percentage split rules (e.g., 40 = 40%) + |> Program.add_relation("assessment_rule", [:atom, :atom, :integer]) + # The outputs + |> Program.add_relation("direct_allocation", [:atom, :atom, :atom, :integer]) + |> Program.add_relation("transitive_allocation", [:atom, :atom, :atom, :integer]) + + # RULE 1: Direct Document Splitting (Level 1) + # Take a posted cost and split it according to the assessment rules. + |> Program.add_rule( + Rule.new( + Atom.new("direct_allocation", [Term.var("Doc"), Term.var("Sender"), Term.var("Receiver"), Term.var("AllocAmount")]), + [ + {:positive, Atom.new("posted_cost", [Term.var("Doc"), Term.var("Sender"), Term.var("Total")])}, + {:positive, Atom.new("assessment_rule", [Term.var("Sender"), Term.var("Receiver"), Term.var("Pct")])} + ], + [ + # Math: Amount = (Total * Pct) / 100 + Constraint.mul(Term.var("Total"), Term.var("Pct"), Term.var("TempAmt")), + Constraint.div(Term.var("TempAmt"), {:const, 100}, Term.var("AllocAmount")) + ] + ) + ) + + # RULE 2: Multi-Stage Iterative Flow (Level 2+) + # If a cost center receives an allocation, and has its OWN assessment rules, forward it. + |> Program.add_rule( + Rule.new( + Atom.new("transitive_allocation", [Term.var("Doc"), Term.var("Intermediate"), Term.var("FinalReceiver"), Term.var("FinalAmount")]), + [ + # Look for money that just arrived via Rule 1 + {:positive, Atom.new("direct_allocation", [Term.var("Doc"), Term.var("OriginalSender"), Term.var("Intermediate"), Term.var("InterAmount")])}, + # Look to see if the receiver has a rule to pass it on + {:positive, Atom.new("assessment_rule", [Term.var("Intermediate"), Term.var("FinalReceiver"), Term.var("InterPct")])} + ], + [ + # Math: FinalAmount = (InterAmount * InterPct) / 100 + Constraint.mul(Term.var("InterAmount"), Term.var("InterPct"), Term.var("TempAmt2")), + Constraint.div(Term.var("TempAmt2"), {:const, 100}, Term.var("FinalAmount")) + ] + ) + ) + +``` + +--- + +### The Test Data (The Month-End Close) + +Let's simulate a $10,000 rent invoice hitting the Facilities cost center. Facilities splits its costs between IT and Sales. Then, IT re-allocates its portion of the rent down to the Engineering and Product teams. + +```elixir +{:ok, result} = + program + # 1. The original invoice hits the general Facilities bucket + |> Program.add_fact("posted_cost", [:inv_001, :facilities, 10000]) + + # 2. Level 1 Assessment Rules (Facilities -> IT / Sales) + |> Program.add_fact("assessment_rule", [:facilities, :it_dept, 40]) # 40% + |> Program.add_fact("assessment_rule", [:facilities, :sales_dept, 60]) # 60% + + # 3. Level 2 Assessment Rules (IT -> Eng / Product) + |> Program.add_fact("assessment_rule", [:it_dept, :eng_team, 70]) # 70% + |> Program.add_fact("assessment_rule", [:it_dept, :product_team, 30]) # 30% + + |> ExDatalog.query() + +``` + +--- + +### The Queries & Expected Results + +**1. Verify Document Splitting (Level 1)** +Let's see how the initial $10,000 was split out of Facilities. + +```elixir +Result.match(result, "direct_allocation", [:inv_001, :facilities, :_, :_]) + +``` + +* **Expected Result:** +```elixir +MapSet.new([ + {:inv_001, :facilities, :sales_dept, 6000}, # 60% of $10,000 + {:inv_001, :facilities, :it_dept, 4000} # 40% of $10,000 +]) + +``` + + + +**2. Verify Multi-Stage Assessment (Level 2)** +Now, let's look at the `transitive_allocation` rule to see how IT automatically flushed its $4,000 portion down to the delivery teams based on its own specific ratios. + +```elixir +Result.match(result, "transitive_allocation", [:inv_001, :it_dept, :_, :_]) + +``` + +* **Expected Result:** +```elixir +MapSet.new([ + {:inv_001, :it_dept, :eng_team, 2800}, # 70% of IT's $4,000 + {:inv_001, :it_dept, :product_team, 1200} # 30% of IT's $4,000 +]) + +``` + + + + +#### I can't see the hierarchy? + +This question highlights a fundamental concept about how logic engines work: Datalog is not a tree-builder; it is an edge-finder. + +Datalog will always return a flat, mathematical set of tuples (facts). It does not natively output nested JSON, structs, or tree hierarchies. Its job is to find all the valid connections (the "edges" of the graph) as fast as possible. + +Once Datalog has extracted those connections, it hands the baton back to your application language. If you want to see a nested hierarchy, you use a few lines of standard Elixir to fold that flat data into a tree + +Here is exactly how you do that. + +Step 1: Query the Direct Edges +Instead of querying the recursive in_reporting_chain rule (which gives us the flattened "everyone under the CEO" list), we query the base manages relation to get the direct parent-child edges. + +Elixir + + +#### I can't see the hierarchy? + +This question highlights a fundamental concept about how logic engines work: **Datalog is not a tree-builder; it is an edge-finder.** + +Datalog will always return a flat, mathematical set of tuples (facts). It does not natively output nested JSON, structs, or tree hierarchies. Its job is to find all the valid connections (the "edges" of the graph) as fast as possible. + +Once Datalog has extracted those connections, it hands the baton back to your application language. If you want to see a nested hierarchy, you use a few lines of standard Elixir to fold that flat data into a tree. + +Here is exactly how you do that. + +### Step 1: Query the Direct Edges + +Instead of querying the recursive `in_reporting_chain` rule (which gives us the flattened "everyone under the CEO" list), we query the base `manages` relation to get the direct parent-child edges. + +```elixir +# Grab the flat tuples from the Datalog result +edges = result.relations["manages"] |> Enum.to_list() + +# edges is now: +# [{:ceo, :vp_eng}, {:vp_eng, :eng_manager}, {:eng_manager, :alice}, {:eng_manager, :bob}] + +``` + +### Step 2: Fold the Edges into a Tree with Elixir + +We can write a quick Elixir script that groups these managers and their direct reports, and then recursively builds a nested map. + +```elixir +# 1. Group the edges into an adjacency list (manager => list of direct reports) +org_chart = + Enum.reduce(edges, %{}, fn {manager, report}, acc -> + Map.update(acc, manager, [report], fn existing -> [report | existing] end) + end) +# Result: %{ceo: [:vp_eng], eng_manager: [:alice, :bob], vp_eng: [:eng_manager]} + +# 2. Define a simple recursive builder +defmodule TreeBuilder do + def build(node, org_chart) do + case Map.get(org_chart, node) do + # If they manage no one, they are a leaf node + nil -> node + # If they manage people, recursively build their subtree + reports -> %{node => Enum.map(reports, &build(&1, org_chart))} + end + end +end + +# 3. Build the tree starting from the top +hierarchy = TreeBuilder.build(:ceo, org_chart) + +``` + +### The Expected Result + +If you run that Elixir code, it will output the exact nested hierarchy you are looking for: + +```elixir +%{ + ceo: [ + %{ + vp_eng: [ + %{ + eng_manager: [ + :bob, + :alice + ] + } + ] + } + ] +} + +``` + +This is the standard architectural pattern: **Datalog filters and traverses the massive graph to find the relevant pieces, and Elixir formats those pieces for the user interface.** + +Here is an interactive visualization of what that hand-off looks like when it reaches the UI layer. + +--- + +Now that you've seen how to combine Datalog's flat relations with Elixir's data structures, are there any other complex graph traversals (like finding the *shortest* path) you'd like to explore? + + + +Philip Zucker’s blog is a goldmine for Datalog! One of the most prominent sections in his notes covers **Program Analysis**. In fact, Datalog is the underlying engine for many modern static analysis tools and decompilers (like Doop for Java or Soufflé's use in smart contract analysis) because source code is just a massive graph of relationships. + +Let's draw directly from his notes on **Reachability** and build example #12: **Static Analysis & Dead Code Elimination**. + +### 12. Static Call Graph Analysis (Dead Code Elimination) + +**Use Case:** A compiler needs to analyze the call graph of a program to figure out which functions are safely connected to the `main()` entry point. Any function that cannot be reached from the entry point is "dead code" and should be stripped out of the final compiled binary. + +This program uses recursive reachability to find the live code, and **stratified negation** to isolate the dead code. + +#### The Program + +```elixir +program = + Program.new() + |> Program.add_relation("calls", [:atom, :atom]) + |> Program.add_relation("entry_point", [:atom]) + |> Program.add_relation("function", [:atom]) + |> Program.add_relation("reachable", [:atom]) + |> Program.add_relation("dead_function", [:atom]) + + # 1. Base Reachability: Entry points are always reachable + |> Program.add_rule( + Rule.new( + Atom.new("reachable", [Term.var("Func")]), + [{:positive, Atom.new("entry_point", [Term.var("Func")])}] + ) + ) + + # 2. Recursive Reachability: If Caller is reachable, Callee is reachable + |> Program.add_rule( + Rule.new( + Atom.new("reachable", [Term.var("Callee")]), + [ + {:positive, Atom.new("reachable", [Term.var("Caller")])}, + {:positive, Atom.new("calls", [Term.var("Caller"), Term.var("Callee")])} + ] + ) + ) + + # 3 & 4. Master List: Collect all known functions (either calling or being called) + |> Program.add_rule( + Rule.new( + Atom.new("function", [Term.var("F")]), + [{:positive, Atom.new("calls", [Term.var("F"), Term.wildcard()])}] + ) + ) + |> Program.add_rule( + Rule.new( + Atom.new("function", [Term.var("F")]), + [{:positive, Atom.new("calls", [Term.wildcard(), Term.var("F")])}] + ) + ) + + # 5. The Negation: Dead code is any known function that is NOT reachable + |> Program.add_rule( + Rule.new( + Atom.new("dead_function", [Term.var("F")]), + [ + {:positive, Atom.new("function", [Term.var("F")])}, + {:negative, Atom.new("reachable", [Term.var("F")])} + ] + ) + ) + +``` + +#### The Test Data + +We will simulate a program where `main` calls standard application logic, but there are a few legacy utility functions left sitting in the codebase that no longer connect back to the main app. + +```elixir +{:ok, result} = + program + |> Program.add_fact("entry_point", [:main]) + + # The Live Code Path + |> Program.add_fact("calls", [:main, :init]) + |> Program.add_fact("calls", [:main, :render]) + |> Program.add_fact("calls", [:init, :load_config]) + |> Program.add_fact("calls", [:render, :draw_ui]) + + # The Dead Code (An orphaned island) + |> Program.add_fact("calls", [:deprecated_start, :legacy_helper]) + + # The Tricky One (Dead code that calls live code) + |> Program.add_fact("calls", [:old_util, :draw_ui]) + + |> ExDatalog.query() + +``` + +--- + +#### The Queries & Expected Results + +**1. The Garbage Collector (Find all dead code)** +Find all functions that the compiler should delete from the final binary. + +```elixir +Result.match(result, "dead_function", [:_]) + +``` + +* **Expected:** ```elixir +MapSet.new([ +{:deprecated_start}, +{:legacy_helper}, +{:old_util} +]) +``` + +``` + + + +*(Notice how the engine correctly flags `:old_util` as dead. Even though it points to `:draw_ui` (which is alive), reachability is directional. Because nothing calls `:old_util`, it is dead).* + +**2. The Live Set (What is actually running?)** +Get the list of all functions safely anchored to the application root. + +```elixir +Result.match(result, "reachable", [:_]) + +``` + +* **Expected:** ```elixir +MapSet.new([ +{:main}, +{:init}, +{:render}, +{:load_config}, +{:draw_ui} +]) +``` + + +``` + + + +**3. Discovering Orphans (Who is calling this?)** +If we see that a function is dead, we can query its callers to see *why* it's dead. Let's ask who calls the `legacy_helper`. + +```elixir +Result.match(result, "calls", [:_, :legacy_helper]) + +``` + +* **Expected:** ```elixir +MapSet.new([ +{:deprecated_start, :legacy_helper} +]) +``` + + +``` + + + +--- +--- + +### Exploring the Logic Visually + +To really understand why static analysis uses Datalog, it helps to see how "directionality" isolates dead code. Here is an interactive visualization of the codebase we just built. diff --git a/livebook/files/Cash-Allocation-Flow.png b/livebooks/files/Cash-Allocation-Flow.png similarity index 100% rename from livebook/files/Cash-Allocation-Flow.png rename to livebooks/files/Cash-Allocation-Flow.png diff --git a/livebook/files/CashFlowAllocation3.png b/livebooks/files/CashFlowAllocation3.png similarity index 100% rename from livebook/files/CashFlowAllocation3.png rename to livebooks/files/CashFlowAllocation3.png diff --git a/livebook/quickstart.livemd b/livebooks/quickstart.livemd similarity index 100% rename from livebook/quickstart.livemd rename to livebooks/quickstart.livemd diff --git a/mix.exs b/mix.exs index cb20860..a6e7f09 100644 --- a/mix.exs +++ b/mix.exs @@ -1,7 +1,7 @@ defmodule ExDatalog.MixProject do use Mix.Project - @version "0.3.0" + @version "0.4.0" @source_url "https://github.com/thanos/ex_datalog" def project do @@ -114,8 +114,21 @@ defmodule ExDatalog.MixProject do {"docs/what-is-datalog.md", filename: "what-is-datalog", title: "What is Datalog?"}, {"docs/constraints.md", filename: "constraints", title: "Constraints"}, {"docs/storage_backends.md", filename: "storage-backends", title: "Storage Backends"}, - {"livebook/quickstart.livemd", filename: "quickstart", title: "Quickstart Tutorial"}, - {"livebook/examples.livemd", filename: "examples", title: "Examples"} + {"livebooks/quickstart.livemd", filename: "quickstart", title: "Quickstart Tutorial"}, + {"livebooks/examples.livemd", filename: "examples", title: "Examples"}, + {"livebooks/ex_datalog_dsl.livemd", filename: "dsl-tutorial", title: "DSL Tutorial"}, + {"docs/migration_dsl.md", + filename: "migration-dsl", title: "Migration: Builder API → DSL"}, + {"docs/articles/01_why_datalog_on_the_beam.md", + filename: "why-datalog-on-the-beam", title: "Why Datalog on the BEAM"}, + {"docs/articles/02_building_an_elixir_datalog_dsl.md", + filename: "building-an-elixir-datalog-dsl", title: "Building an Elixir Datalog DSL"}, + {"docs/articles/03_datalog_rules_as_elixir_macros.md", + filename: "datalog-rules-as-elixir-macros", title: "Datalog Rules as Elixir Macros"}, + {"docs/articles/04_querying_materialized_knowledge.md", + filename: "querying-materialized-knowledge", title: "Querying Materialized Knowledge"}, + {"docs/articles/05_negation_constraints_and_safety.md", + filename: "negation-constraints-and-safety", title: "Negation, Constraints, and Safety"} ], groups_for_modules: [ "Program Builder": [ @@ -138,6 +151,11 @@ defmodule ExDatalog.MixProject do ExDatalog.Knowledge, ExDatalog.Explain, ExDatalog.Telemetry + ], + DSL: [ + ExDatalog.Schema, + ExDatalog.DSL.CompileError, + ExDatalog.UnsupportedFeature ] ] ] diff --git a/test/ex_datalog/compiler_test.exs b/test/ex_datalog/compiler_test.exs index 91a67d4..1f7d671 100644 --- a/test/ex_datalog/compiler_test.exs +++ b/test/ex_datalog/compiler_test.exs @@ -323,13 +323,19 @@ defmodule ExDatalog.CompilerTest do assert {:ok, ir} = Compiler.compile(program) values = Enum.map(ir.facts, & &1.values) - assert values == [[{:atom, :a}, {:atom, :z}], [{:atom, :m}, {:atom, :m}], [{:atom, :z}, {:atom, :a}]] + + assert values == [ + [{:atom, :a}, {:atom, :z}], + [{:atom, :m}, {:atom, :m}], + [{:atom, :z}, {:atom, :a}] + ] end end describe "compile/1 IR from_term list values" do test "IR.from_term converts const list with mixed types" do - assert IR.from_term({:const, [1, :a, "hello"]}) == {:const, {:list, [{:int, 1}, {:atom, :a}, {:str, "hello"}]}} + assert IR.from_term({:const, [1, :a, "hello"]}) == + {:const, {:list, [{:int, 1}, {:atom, :a}, {:str, "hello"}]}} end test "IR.from_term converts const nested list" do diff --git a/test/ex_datalog/ir_test.exs b/test/ex_datalog/ir_test.exs index 2ad8103..012e0e3 100644 --- a/test/ex_datalog/ir_test.exs +++ b/test/ex_datalog/ir_test.exs @@ -56,7 +56,8 @@ defmodule ExDatalog.IRTest do describe "from_term/1 list values" do test "converts const list with mixed types" do - assert IR.from_term({:const, [1, :a, "hello"]}) == {:const, {:list, [{:int, 1}, {:atom, :a}, {:str, "hello"}]}} + assert IR.from_term({:const, [1, :a, "hello"]}) == + {:const, {:list, [{:int, 1}, {:atom, :a}, {:str, "hello"}]}} end test "converts const single-element list" do @@ -92,13 +93,17 @@ defmodule ExDatalog.IRTest do test "converts starts_with constraint" do ast_c = ExDatalog.Constraint.starts_with({:var, "X"}, {:const, "hello"}) ir_c = IR.from_constraint(ast_c) - assert %IR.Constraint{op: :starts_with, left: {:var, "X"}, right: {:const, {:str, "hello"}}} = ir_c + + assert %IR.Constraint{op: :starts_with, left: {:var, "X"}, right: {:const, {:str, "hello"}}} = + ir_c end test "converts contains constraint" do ast_c = ExDatalog.Constraint.contains({:var, "X"}, {:const, "ell"}) ir_c = IR.from_constraint(ast_c) - assert %IR.Constraint{op: :contains, left: {:var, "X"}, right: {:const, {:str, "ell"}}} = ir_c + + assert %IR.Constraint{op: :contains, left: {:var, "X"}, right: {:const, {:str, "ell"}}} = + ir_c end end @@ -106,7 +111,12 @@ defmodule ExDatalog.IRTest do test "converts member constraint" do ast_c = ExDatalog.Constraint.member({:var, "X"}, {:const, [:a, :b]}) ir_c = IR.from_constraint(ast_c) - assert %IR.Constraint{op: :member, left: {:var, "X"}, right: {:const, {:list, [{:atom, :a}, {:atom, :b}]}}} = ir_c + + assert %IR.Constraint{ + op: :member, + left: {:var, "X"}, + right: {:const, {:list, [{:atom, :a}, {:atom, :b}]}} + } = ir_c end end @@ -132,7 +142,8 @@ defmodule ExDatalog.IRTest do end test "resolves const list" do - assert IR.resolve_operand({:const, {:list, [{:int, 1}, {:atom, :a}]}}, %{}) == {:ok, [1, :a]} + assert IR.resolve_operand({:const, {:list, [{:int, 1}, {:atom, :a}]}}, %{}) == + {:ok, [1, :a]} end test "returns :unbound for wildcard" do @@ -154,7 +165,11 @@ defmodule ExDatalog.IRTest do end test "converts list recursively" do - assert IR.value_to_native({:list, [{:int, 1}, {:atom, :foo}, {:str, "bar"}]}) == [1, :foo, "bar"] + assert IR.value_to_native({:list, [{:int, 1}, {:atom, :foo}, {:str, "bar"}]}) == [ + 1, + :foo, + "bar" + ] end end @@ -165,7 +180,8 @@ defmodule ExDatalog.IRTest do head: %IR.Atom{relation: "high", terms: [{:var, "X"}]}, body: [ {:positive, %IR.Atom{relation: "income", terms: [{:var, "X"}, {:var, "S"}]}}, - {:constraint, %IR.Constraint{op: :gt, left: {:var, "S"}, right: {:const, {:int, 100}}, result: nil}} + {:constraint, + %IR.Constraint{op: :gt, left: {:var, "S"}, right: {:const, {:int, 100}}, result: nil}} ], stratum: 0, metadata: %{} diff --git a/test/ex_datalog/schema_test.exs b/test/ex_datalog/schema_test.exs new file mode 100644 index 0000000..1167230 --- /dev/null +++ b/test/ex_datalog/schema_test.exs @@ -0,0 +1,1800 @@ +defmodule ExDatalog.SchemaTest do + use ExUnit.Case, async: true + + doctest ExDatalog.Schema + doctest ExDatalog.UnsupportedFeature + doctest ExDatalog.DSL.CompileError + + alias ExDatalog.{Atom, Knowledge, Program, Rule, Term} + + describe "relation/2 macro" do + test "declares a relation with typed fields" do + defmodule RelTest1 do + use ExDatalog.Schema + + relation :parent do + field(:parent_name, :atom) + field(:child_name, :atom) + end + end + + program = RelTest1.program() + assert Map.has_key?(program.relations, "parent") + assert program.relations["parent"] == %{arity: 2, types: [:atom, :atom]} + end + + test "declares multiple relations" do + defmodule RelTest2 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + relation :ancestor do + field(:a, :atom) + field(:d, :atom) + end + end + + program = RelTest2.program() + assert Map.has_key?(program.relations, "parent") + assert Map.has_key?(program.relations, "ancestor") + end + + test "supports :integer and :string field types" do + defmodule RelTest3 do + use ExDatalog.Schema + + relation :income do + field(:person, :atom) + field(:amount, :integer) + end + + relation :label do + field(:node, :atom) + field(:text, :string) + end + end + + program = RelTest3.program() + assert program.relations["income"].types == [:atom, :integer] + assert program.relations["label"].types == [:atom, :string] + end + + test "supports :any field type" do + defmodule RelTest4 do + use ExDatalog.Schema + + relation :payload do + field(:key, :atom) + field(:value, :any) + end + end + + program = RelTest4.program() + assert program.relations["payload"].types == [:atom, :any] + end + end + + describe "fact/1 macro" do + test "declares a single ground fact" do + defmodule FactTest1 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + fact(parent(:alice, :bob)) + end + + program = FactTest1.program() + assert length(program.facts) == 1 + assert {"parent", [:alice, :bob]} in program.facts + end + + test "declares multiple facts" do + defmodule FactTest2 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + fact(parent(:alice, :bob)) + fact(parent(:bob, :carol)) + end + + program = FactTest2.program() + facts = Enum.map(program.facts, fn {rel, vals} -> {rel, vals} end) + assert length(facts) == 2 + assert {"parent", [:alice, :bob]} in facts + assert {"parent", [:bob, :carol]} in facts + end + end + + describe "facts/2 macro" do + test "declares bulk facts with row syntax" do + defmodule FactsTest1 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + facts :parent do + row(:alice, :bob) + row(:bob, :carol) + row(:carol, :dave) + end + end + + program = FactsTest1.program() + facts = Enum.map(program.facts, fn {rel, vals} -> {rel, vals} end) + assert length(facts) == 3 + assert {"parent", [:alice, :bob]} in facts + assert {"parent", [:bob, :carol]} in facts + assert {"parent", [:carol, :dave]} in facts + end + end + + describe "rule/2 macro" do + test "declares a simple positive rule" do + defmodule RuleTest1 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + relation :ancestor do + field(:a, :atom) + field(:d, :atom) + end + + rule ancestor(X, Y) do + parent(X, Y) + end + end + + program = RuleTest1.program() + assert length(program.rules) == 1 + + rule = hd(program.rules) + assert rule.head.relation == "ancestor" + assert rule.head.terms == [Term.var("X"), Term.var("Y")] + assert length(rule.body) == 1 + + {:positive, body_atom} = hd(rule.body) + assert body_atom.relation == "parent" + assert body_atom.terms == [Term.var("X"), Term.var("Y")] + end + + test "declares a recursive rule" do + defmodule RuleTest2 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + relation :ancestor do + field(:a, :atom) + field(:d, :atom) + end + + rule ancestor(X, Z) do + parent(X, Y) + ancestor(Y, Z) + end + end + + program = RuleTest2.program() + rule = hd(program.rules) + assert rule.head.terms == [Term.var("X"), Term.var("Z")] + assert length(rule.body) == 2 + end + + test "declares a rule with negation using not_" do + defmodule RuleTest3 do + use ExDatalog.Schema + + relation :male do + field(:p, :atom) + end + + relation :married do + field(:p, :atom) + field(:sp, :atom) + end + + relation :bachelor do + field(:p, :atom) + end + + rule bachelor(P) do + male(P) + not_(married(P, _)) + end + end + + program = RuleTest3.program() + rule = hd(program.rules) + assert rule.head.relation == "bachelor" + + negated = + Enum.find(rule.body, fn + {:negative, _} -> true + _ -> false + end) + + assert negated != nil + + {:negative, neg_atom} = negated + assert neg_atom.relation == "married" + assert neg_atom.terms == [Term.var("P"), Term.from(:_)] + end + + test "declares a rule with constraint" do + defmodule RuleTest4 do + use ExDatalog.Schema + + relation :income do + field(:person, :atom) + field(:amount, :integer) + end + + relation :high_earner do + field(:person, :atom) + end + + rule high_earner(P) do + income(P, S) + gt(S, 100_000) + end + end + + program = RuleTest4.program() + rule = hd(program.rules) + assert rule.head.relation == "high_earner" + assert length(rule.constraints) == 1 + + constraint = hd(rule.constraints) + assert constraint.op == :gt + end + + test "rule body constants use lowercase vs uppercase convention" do + defmodule RuleTest5 do + use ExDatalog.Schema + + relation :edge do + field(:from, :atom) + field(:to, :atom) + end + + relation :reachable do + field(:from, :atom) + field(:to, :atom) + end + + rule reachable(:start, Y) do + edge(:start, Y) + end + end + + program = RuleTest5.program() + rule = hd(program.rules) + + assert rule.head.terms == [Term.from(:start), Term.var("Y")] + end + end + + describe "materialize/0,1 integration" do + test "full transitive closure pipeline" do + defmodule PipelineTest1 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + relation :ancestor do + field(:a, :atom) + field(:d, :atom) + end + + fact(parent(:alice, :bob)) + fact(parent(:bob, :carol)) + fact(parent(:carol, :dave)) + + rule ancestor(X, Y) do + parent(X, Y) + end + + rule ancestor(X, Z) do + parent(X, Y) + ancestor(Y, Z) + end + end + + {:ok, knowledge} = PipelineTest1.materialize() + + ancestor = Knowledge.get(knowledge, "ancestor") + assert MapSet.size(ancestor) == 6 + assert {:alice, :bob} in ancestor + assert {:alice, :carol} in ancestor + assert {:alice, :dave} in ancestor + assert {:bob, :carol} in ancestor + assert {:bob, :dave} in ancestor + assert {:carol, :dave} in ancestor + end + + test "negation integration" do + defmodule PipelineTest2 do + use ExDatalog.Schema + + relation :male do + field(:p, :atom) + end + + relation :married do + field(:p, :atom) + field(:sp, :atom) + end + + relation :bachelor do + field(:p, :atom) + end + + fact(male(:bob)) + fact(male(:tom)) + fact(married(:tom, :sally)) + + rule bachelor(P) do + male(P) + not_(married(P, _)) + end + end + + {:ok, knowledge} = PipelineTest2.materialize() + bachelor = Knowledge.get(knowledge, "bachelor") + assert MapSet.size(bachelor) == 1 + assert {:bob} in bachelor + end + + test "constraint integration" do + defmodule PipelineTest3 do + use ExDatalog.Schema + + relation :income do + field(:person, :atom) + field(:amount, :integer) + end + + relation :high_earner do + field(:person, :atom) + end + + fact(income(:alice, 150_000)) + fact(income(:bob, 50_000)) + fact(income(:carol, 200_000)) + + rule high_earner(P) do + income(P, S) + gt(S, 100_000) + end + end + + {:ok, knowledge} = PipelineTest3.materialize() + high_earner = Knowledge.get(knowledge, "high_earner") + assert MapSet.size(high_earner) == 2 + assert {:alice} in high_earner + assert {:carol} in high_earner + end + + test "facts macro bulk insertion" do + defmodule PipelineTest4 do + use ExDatalog.Schema + + relation :edge do + field(:from, :atom) + field(:to, :atom) + end + + relation :path do + field(:from, :atom) + field(:to, :atom) + end + + facts :edge do + row(:a, :b) + row(:b, :c) + row(:c, :d) + end + + rule path(X, Y) do + edge(X, Y) + end + end + + {:ok, knowledge} = PipelineTest4.materialize() + path = Knowledge.get(knowledge, "path") + assert MapSet.size(path) == 3 + end + end + + describe "query/2 macro" do + test "named query against materialized knowledge" do + defmodule QueryTest1 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + relation :ancestor do + field(:a, :atom) + field(:d, :atom) + end + + fact(parent(:alice, :bob)) + fact(parent(:bob, :carol)) + + rule ancestor(X, Y) do + parent(X, Y) + end + + query :all_ancestors do + find(X, Y) + where(ancestor(X, Y)) + end + end + + {:ok, knowledge} = QueryTest1.materialize() + results = QueryTest1.query(:all_ancestors, knowledge) + assert length(results) == 2 + assert {:alice, :bob} in results + assert {:bob, :carol} in results + end + + test "query with constant pattern" do + defmodule QueryTest2 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + relation :ancestor do + field(:a, :atom) + field(:d, :atom) + end + + fact(parent(:alice, :bob)) + fact(parent(:bob, :carol)) + + rule ancestor(X, Y) do + parent(X, Y) + end + + query :descendants_of_alice do + find(Y) + where(ancestor(:alice, Y)) + end + end + + {:ok, knowledge} = QueryTest2.materialize() + results = QueryTest2.query(:descendants_of_alice, knowledge) + assert :bob in results + end + + test "query raises on unknown name" do + defmodule QueryTest3 do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + fact(parent(:alice, :bob)) + + query :known_query do + find(X) + where(parent(X, _)) + end + end + + {:ok, knowledge} = QueryTest3.materialize() + + assert_raise ArgumentError, ~r/unknown query/, fn -> + QueryTest3.query(:nonexistent, knowledge) + end + end + end + + describe "queries/0" do + test "returns query metadata" do + defmodule QueriesTest do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + fact(parent(:alice, :bob)) + + query :kids_of_alice do + find(C) + where(parent(:alice, C)) + end + end + + queries = QueriesTest.queries() + assert Map.has_key?(queries, :kids_of_alice) + assert queries.kids_of_alice.relation == "parent" + end + end + + describe "backward compatibility" do + test "DSL program is compatible with builder API" do + defmodule CompatTest do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + relation :ancestor do + field(:a, :atom) + field(:d, :atom) + end + + fact(parent(:alice, :bob)) + end + + program = CompatTest.program() + + extended = + program + |> Program.add_fact("parent", [:bob, :carol]) + |> Program.add_rule( + Rule.new( + Atom.new("ancestor", [Term.var("X"), Term.var("Y")]), + [{:positive, Atom.new("parent", [Term.var("X"), Term.var("Y")])}] + ) + ) + + {:ok, knowledge} = ExDatalog.materialize(extended) + ancestor = Knowledge.get(knowledge, "ancestor") + assert MapSet.size(ancestor) == 2 + end + end + + describe "arithmetic constraint in DSL" do + test "add constraint" do + defmodule ArithTest do + use ExDatalog.Schema + + relation :salary do + field(:person, :atom) + field(:base, :integer) + end + + relation :total_comp do + field(:person, :atom) + field(:total, :integer) + end + + fact(salary(:alice, 100)) + fact(salary(:bob, 80)) + + rule total_comp(P, T) do + salary(P, B) + add(B, 20, T) + end + end + + {:ok, knowledge} = ArithTest.materialize() + total = Knowledge.get(knowledge, "total_comp") + assert MapSet.size(total) == 2 + assert {:alice, 120} in total + assert {:bob, 100} in total + end + end + + describe "UnsupportedFeature" do + test "aggregate spike returns UnsupportedFeature struct" do + assert %ExDatalog.UnsupportedFeature{feature: :aggregates, planned_for: "v0.6.0"}.feature == + :aggregates + end + end + + describe "wildcard/0 helper" do + test "returns :wildcard atom" do + assert ExDatalog.Schema.wildcard() == :wildcard + end + + test "wildcard can be used in rule bodies" do + defmodule WildcardTest do + use ExDatalog.Schema + + relation :person do + field(:name, :atom) + end + + relation :unmatched do + field(:name, :atom) + end + + fact(person(:alice)) + fact(person(:bob)) + + rule unmatched(N) do + person(N) + not_(person(wildcard())) + end + end + + program = WildcardTest.program() + assert length(program.rules) == 1 + end + end + + describe "constraint types in DSL" do + test "eq constraint" do + defmodule EqTest do + use ExDatalog.Schema + + relation :pair do + field(:a, :atom) + field(:b, :atom) + end + + relation :same do + field(:x, :atom) + end + + fact(pair(:x, :x)) + + rule same(A) do + pair(A, B) + eq(A, B) + end + end + + {:ok, knowledge} = EqTest.materialize() + same = Knowledge.get(knowledge, "same") + assert {:x} in same + end + + test "neq constraint" do + defmodule NeqTest do + use ExDatalog.Schema + + relation :pair do + field(:a, :atom) + field(:b, :atom) + end + + relation :different do + field(:a, :atom) + field(:b, :atom) + end + + fact(pair(:a, :b)) + fact(pair(:b, :a)) + fact(pair(:a, :a)) + + rule different(A, B) do + pair(A, B) + neq(A, B) + end + end + + {:ok, knowledge} = NeqTest.materialize() + diff = Knowledge.get(knowledge, "different") + assert MapSet.size(diff) == 2 + end + + test "is_integer constraint" do + defmodule TypeTest do + use ExDatalog.Schema + + relation :value do + field(:v, :any) + end + + relation :int_val do + field(:v, :any) + end + + fact(value(42)) + fact(value(:not_int)) + + rule int_val(V) do + value(V) + is_integer(V) + end + end + + {:ok, knowledge} = TypeTest.materialize() + int_val = Knowledge.get(knowledge, "int_val") + assert MapSet.size(int_val) == 1 + assert {42} in int_val + end + end + + describe "materialize/0,1 options" do + test "passes options to ExDatalog.materialize/2" do + defmodule OptsTest do + use ExDatalog.Schema + + relation :link do + field(:from, :atom) + field(:to, :atom) + end + + relation :reachable do + field(:from, :atom) + field(:to, :atom) + end + + fact(link(:a, :b)) + + rule reachable(X, Y) do + link(X, Y) + end + end + + {:ok, knowledge} = OptsTest.materialize(iteration_limit: 100) + assert MapSet.size(Knowledge.get(knowledge, "reachable")) == 1 + end + end + + describe "program/0" do + test "returns a valid Program struct" do + defmodule ProgTest do + use ExDatalog.Schema + + relation :r do + field(:a, :atom) + end + + fact(r(:x)) + end + + program = ProgTest.program() + assert %Program{} = program + assert Map.has_key?(program.relations, "r") + assert length(program.facts) == 1 + end + end + + describe "multi-rule programs" do + test "multiple rules for same head relation" do + defmodule MultiRuleTest do + use ExDatalog.Schema + + relation :link do + field(:from, :atom) + field(:to, :atom) + end + + relation :path do + field(:from, :atom) + field(:to, :atom) + end + + fact(link(:a, :b)) + fact(link(:b, :c)) + fact(link(:c, :d)) + + rule path(X, Y) do + link(X, Y) + end + + rule path(X, Z) do + link(X, Y) + path(Y, Z) + end + end + + {:ok, knowledge} = MultiRuleTest.materialize() + path = Knowledge.get(knowledge, "path") + assert MapSet.size(path) == 6 + end + end + + describe "DSL.CompileError" do + test "exception has message field" do + err = %ExDatalog.DSL.CompileError{message: "test error"} + assert err.message == "test error" + end + + test "exception can be raised and caught" do + assert_raise ExDatalog.DSL.CompileError, "bad schema", fn -> + raise ExDatalog.DSL.CompileError, message: "bad schema" + end + end + end + + describe "additional constraint types in DSL" do + test "neq constraint filters non-matching bindings" do + defmodule NeqConstraintTest do + use ExDatalog.Schema + + relation :pair do + field(:a, :atom) + field(:b, :atom) + end + + relation :different do + field(:a, :atom) + field(:b, :atom) + end + + fact(pair(:x, :y)) + fact(pair(:x, :x)) + + rule different(A, B) do + pair(A, B) + neq(A, B) + end + end + + {:ok, knowledge} = NeqConstraintTest.materialize() + diff = Knowledge.get(knowledge, "different") + assert MapSet.size(diff) == 1 + assert {:x, :y} in diff + end + + test "gte constraint" do + defmodule GteConstraintTest do + use ExDatalog.Schema + + relation :score do + field(:player, :atom) + field(:points, :integer) + end + + relation :passing do + field(:player, :atom) + end + + fact(score(:alice, 60)) + fact(score(:bob, 59)) + fact(score(:carol, 100)) + + rule passing(P) do + score(P, S) + gte(S, 60) + end + end + + {:ok, knowledge} = GteConstraintTest.materialize() + passing = Knowledge.get(knowledge, "passing") + assert MapSet.size(passing) == 2 + assert {:alice} in passing + assert {:carol} in passing + end + + test "lte constraint" do + defmodule LteConstraintTest do + use ExDatalog.Schema + + relation :score do + field(:player, :atom) + field(:points, :integer) + end + + relation :low_score do + field(:player, :atom) + end + + fact(score(:alice, 30)) + fact(score(:bob, 60)) + + rule low_score(P) do + score(P, S) + lte(S, 40) + end + end + + {:ok, knowledge} = LteConstraintTest.materialize() + low = Knowledge.get(knowledge, "low_score") + assert MapSet.size(low) == 1 + assert {:alice} in low + end + + test "sub constraint" do + defmodule SubConstraintTest do + use ExDatalog.Schema + + relation :value do + field(:x, :integer) + end + + relation :decremented do + field(:x, :integer) + field(:y, :integer) + end + + fact(value(10)) + fact(value(5)) + + rule decremented(X, Y) do + value(X) + sub(X, 3, Y) + end + end + + {:ok, knowledge} = SubConstraintTest.materialize() + dec = Knowledge.get(knowledge, "decremented") + assert MapSet.size(dec) == 2 + assert {10, 7} in dec + assert {5, 2} in dec + end + + test "mul constraint" do + defmodule MulConstraintTest do + use ExDatalog.Schema + + relation :value do + field(:x, :integer) + end + + relation :doubled do + field(:x, :integer) + field(:y, :integer) + end + + fact(value(3)) + fact(value(7)) + + rule doubled(X, Y) do + value(X) + mul(X, 2, Y) + end + end + + {:ok, knowledge} = MulConstraintTest.materialize() + dbl = Knowledge.get(knowledge, "doubled") + assert MapSet.size(dbl) == 2 + assert {3, 6} in dbl + assert {7, 14} in dbl + end + + test "div constraint" do + defmodule DivConstraintTest do + use ExDatalog.Schema + + relation :value do + field(:x, :integer) + end + + relation :halved do + field(:x, :integer) + field(:y, :integer) + end + + fact(value(10)) + fact(value(7)) + + rule halved(X, Y) do + value(X) + div(X, 2, Y) + end + end + + {:ok, knowledge} = DivConstraintTest.materialize() + half = Knowledge.get(knowledge, "halved") + assert MapSet.size(half) == 2 + assert {10, 5} in half + assert {7, 3} in half + end + + test "is_binary constraint" do + defmodule IsBinaryTest do + use ExDatalog.Schema + + relation :entry do + field(:k, :atom) + field(:v, :any) + end + + relation :string_entry do + field(:k, :atom) + end + + fact(entry(:a, "hello")) + fact(entry(:b, 42)) + + rule string_entry(K) do + entry(K, V) + is_binary(V) + end + end + + {:ok, knowledge} = IsBinaryTest.materialize() + result = Knowledge.get(knowledge, "string_entry") + assert MapSet.size(result) == 1 + assert {:a} in result + end + + test "is_atom constraint" do + defmodule IsAtomTest do + use ExDatalog.Schema + + relation :entry do + field(:k, :atom) + field(:v, :any) + end + + relation :atom_entry do + field(:k, :atom) + end + + fact(entry(:a, :foo)) + fact(entry(:b, 42)) + + rule atom_entry(K) do + entry(K, V) + is_atom(V) + end + end + + {:ok, knowledge} = IsAtomTest.materialize() + result = Knowledge.get(knowledge, "atom_entry") + assert MapSet.size(result) == 1 + assert {:a} in result + end + + test "starts_with constraint" do + defmodule StartsWithTest do + use ExDatalog.Schema + + relation :word do + field(:w, :string) + end + + relation :hello_word do + field(:w, :string) + end + + fact(word("hello")) + fact(word("world")) + fact(word("helicopter")) + + rule hello_word(W) do + word(W) + starts_with(W, "hel") + end + end + + {:ok, knowledge} = StartsWithTest.materialize() + result = Knowledge.get(knowledge, "hello_word") + assert MapSet.size(result) == 2 + end + + test "contains constraint" do + defmodule ContainsTest do + use ExDatalog.Schema + + relation :word do + field(:w, :string) + end + + relation :ell_word do + field(:w, :string) + end + + fact(word("hello")) + fact(word("world")) + fact(word("yellow")) + + rule ell_word(W) do + word(W) + contains(W, "ell") + end + end + + {:ok, knowledge} = ContainsTest.materialize() + result = Knowledge.get(knowledge, "ell_word") + assert MapSet.size(result) == 2 + end + + test "member constraint" do + defmodule MemberTest do + use ExDatalog.Schema + + relation :color do + field(:name, :atom) + end + + relation :primary_color do + field(:name, :atom) + end + + fact(color(:red)) + fact(color(:blue)) + fact(color(:purple)) + + rule primary_color(C) do + color(C) + member(C, [:red, :blue, :green]) + end + end + + {:ok, knowledge} = MemberTest.materialize() + result = Knowledge.get(knowledge, "primary_color") + assert MapSet.size(result) == 2 + assert {:red} in result + assert {:blue} in result + end + end + + describe "query edge cases" do + test "multi-column find projects to tuples" do + defmodule MultiColQueryTest do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + relation :ancestor do + field(:a, :atom) + field(:d, :atom) + end + + fact(parent(:alice, :bob)) + fact(parent(:bob, :carol)) + fact(parent(:carol, :dave)) + + rule ancestor(X, Y) do + parent(X, Y) + end + + rule ancestor(X, Z) do + parent(X, Y) + ancestor(Y, Z) + end + + query :all_ancestors do + find(A, D) + where(ancestor(A, D)) + end + end + + {:ok, knowledge} = MultiColQueryTest.materialize() + results = MultiColQueryTest.query(:all_ancestors, knowledge) + assert length(results) == 6 + assert {:alice, :bob} in results + assert {:alice, :dave} in results + end + + test "query with all wildcards returns full relation" do + defmodule WildcardQueryTest do + use ExDatalog.Schema + + relation :edge do + field(:from, :atom) + field(:to, :atom) + end + + fact(edge(:a, :b)) + fact(edge(:b, :c)) + + query :all_edges do + find(X, Y) + where(edge(X, Y)) + end + end + + {:ok, knowledge} = WildcardQueryTest.materialize() + results = WildcardQueryTest.query(:all_edges, knowledge) + assert length(results) == 2 + end + end + + describe "facts macro edge cases" do + test "facts with single row" do + defmodule FactsSingleRowTest do + use ExDatalog.Schema + + relation :node do + field(:name, :atom) + end + + facts :node do + row(:root) + end + end + + program = FactsSingleRowTest.program() + assert length(program.facts) == 1 + assert {"node", [:root]} in program.facts + end + end + + describe "rule head with uppercase variables from module context" do + test "uppercase variables in rule head via __aliases__ AST form" do + defmodule AliasVarTest do + use ExDatalog.Schema + + relation :link do + field(:from, :atom) + field(:to, :atom) + end + + relation :reachable do + field(:from, :atom) + field(:to, :atom) + end + + fact(link(:a, :b)) + + rule reachable(X, Y) do + link(X, Y) + end + end + + {:ok, knowledge} = AliasVarTest.materialize() + result = Knowledge.get(knowledge, "reachable") + assert MapSet.size(result) == 1 + end + end + + describe "parse term edge cases" do + test "integer constants in facts" do + defmodule IntFactTest do + use ExDatalog.Schema + + relation :measurement do + field(:name, :atom) + field(:value, :integer) + end + + fact(measurement(:temp, 42)) + end + + program = IntFactTest.program() + assert {"measurement", [:temp, 42]} in program.facts + end + + test "string constants in constraints" do + defmodule StringConstraintTest do + use ExDatalog.Schema + + relation :label do + field(:id, :atom) + field(:text, :string) + end + + relation :short_label do + field(:id, :atom) + end + + fact(label(:a, "hello world")) + fact(label(:b, "hi")) + + rule short_label(I) do + label(I, T) + starts_with(T, "hello") + end + end + + {:ok, knowledge} = StringConstraintTest.materialize() + result = Knowledge.get(knowledge, "short_label") + assert MapSet.size(result) == 1 + end + end + + describe "combination of multiple constraints" do + test "gt and lte together in single rule" do + defmodule ComboConstraintTest do + use ExDatalog.Schema + + relation :score do + field(:player, :atom) + field(:points, :integer) + end + + relation :mid_range do + field(:player, :atom) + end + + fact(score(:alice, 60)) + fact(score(:bob, 30)) + fact(score(:carol, 90)) + + rule mid_range(P) do + score(P, S) + gt(S, 50) + lte(S, 80) + end + end + + {:ok, knowledge} = ComboConstraintTest.materialize() + result = Knowledge.get(knowledge, "mid_range") + assert MapSet.size(result) == 1 + assert {:alice} in result + end + end + + describe "program/0 introspection" do + test "program contains declared relations" do + defmodule IntrospectionTest do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + relation :ancestor do + field(:a, :atom) + field(:d, :atom) + end + + fact(parent(:x, :y)) + + rule ancestor(X, Y) do + parent(X, Y) + end + end + + program = IntrospectionTest.program() + assert Map.has_key?(program.relations, "parent") + assert Map.has_key?(program.relations, "ancestor") + assert length(program.facts) == 1 + assert length(program.rules) == 1 + end + end + + describe "materialize/1 with options" do + test "passes iteration_limit option" do + defmodule IterLimitTest do + use ExDatalog.Schema + + relation :edge do + field(:from, :atom) + field(:to, :atom) + end + + relation :reachable do + field(:from, :atom) + field(:to, :atom) + end + + fact(edge(:a, :b)) + + rule reachable(X, Y) do + edge(X, Y) + end + end + + {:ok, knowledge} = IterLimitTest.materialize(iteration_limit: 100) + assert MapSet.size(Knowledge.get(knowledge, "reachable")) == 1 + end + end +end + +defmodule ExDatalog.SchemaCoverageTest do + use ExUnit.Case, async: true + + alias ExDatalog.Knowledge + + describe "wildcard in not_ clause" do + test "wildcard() and _ are equivalent in negation" do + defmodule WildcardNegationTest do + use ExDatalog.Schema + + relation :person do + field(:name, :atom) + end + + relation :pet do + field(:owner, :atom) + field(:pet_name, :atom) + end + + relation :petless do + field(:name, :atom) + end + + fact(person(:alice)) + fact(person(:bob)) + fact(pet(:alice, :fido)) + + rule petless(N) do + person(N) + not_(pet(N, wildcard())) + end + end + + {:ok, knowledge} = WildcardNegationTest.materialize() + petless = Knowledge.get(knowledge, "petless") + assert {:bob} in petless + end + end + + describe "eq constraint" do + test "eq filters for equality" do + defmodule EqConstraintIntegrationTest do + use ExDatalog.Schema + + relation :pair do + field(:a, :atom) + field(:b, :atom) + end + + relation :same_pair do + field(:a, :atom) + end + + fact(pair(:x, :x)) + fact(pair(:x, :y)) + + rule same_pair(A) do + pair(A, B) + eq(A, B) + end + end + + {:ok, knowledge} = EqConstraintIntegrationTest.materialize() + same = Knowledge.get(knowledge, "same_pair") + assert MapSet.size(same) == 1 + assert {:x} in same + end + end + + describe "query with atom constants in where" do + test "constant value projection works correctly" do + defmodule QueryConstantTest do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + fact(parent(:alice, :bob)) + fact(parent(:alice, :carol)) + fact(parent(:bob, :dave)) + + query :children_of_alice do + find(C) + where(parent(:alice, C)) + end + end + + {:ok, knowledge} = QueryConstantTest.materialize() + results = QueryConstantTest.query(:children_of_alice, knowledge) + assert :bob in results + assert :carol in results + end + end + + describe "rule with string and integer terms" do + test "integer constant as fact value" do + defmodule IntConstantRuleTest do + use ExDatalog.Schema + + relation :data do + field(:id, :atom) + field(:val, :integer) + end + + relation :big_data do + field(:id, :atom) + end + + fact(data(:a, 100)) + fact(data(:b, 10)) + + rule big_data(I) do + data(I, V) + gt(V, 50) + end + end + + {:ok, knowledge} = IntConstantRuleTest.materialize() + big = Knowledge.get(knowledge, "big_data") + assert MapSet.size(big) == 1 + assert {:a} in big + end + end + + describe "program/0 introspection with multiple rules" do + test "rules and facts are in correct order" do + defmodule IntrospectOrderTest do + use ExDatalog.Schema + + relation :edge do + field(:from, :atom) + field(:to, :atom) + end + + relation :path do + field(:from, :atom) + field(:to, :atom) + end + + fact(edge(:a, :b)) + fact(edge(:b, :c)) + + rule path(X, Y) do + edge(X, Y) + end + + rule path(X, Z) do + edge(X, Y) + path(Y, Z) + end + end + + program = IntrospectOrderTest.program() + assert length(program.rules) == 2 + assert length(program.facts) == 2 + end + end + + describe "constraint from_tuple passthrough" do + test "all comparison constraints produce valid Constraint structs" do + alias ExDatalog.Constraint + + assert %Constraint{op: :eq} = Constraint.from_tuple({:eq, {:var, "X"}, {:const, 1}}) + assert %Constraint{op: :neq} = Constraint.from_tuple({:neq, {:var, "X"}, {:var, "Y"}}) + assert %Constraint{op: :gt} = Constraint.from_tuple({:gt, {:var, "X"}, {:const, 0}}) + assert %Constraint{op: :gte} = Constraint.from_tuple({:gte, {:var, "X"}, {:const, 0}}) + assert %Constraint{op: :lt} = Constraint.from_tuple({:lt, {:var, "X"}, {:const, 0}}) + assert %Constraint{op: :lte} = Constraint.from_tuple({:lte, {:var, "X"}, {:const, 0}}) + + assert %Constraint{op: :add} = + Constraint.from_tuple({:add, {:var, "X"}, {:var, "Y"}, {:var, "Z"}}) + + assert %Constraint{op: :sub} = + Constraint.from_tuple({:sub, {:var, "X"}, {:var, "Y"}, {:var, "Z"}}) + + assert %Constraint{op: :mul} = + Constraint.from_tuple({:mul, {:var, "X"}, {:var, "Y"}, {:var, "Z"}}) + + assert %Constraint{op: :div} = + Constraint.from_tuple({:div, {:var, "X"}, {:var, "Y"}, {:var, "Z"}}) + + assert %Constraint{op: :is_integer} = Constraint.from_tuple({:is_integer, {:var, "X"}}) + assert %Constraint{op: :is_binary} = Constraint.from_tuple({:is_binary, {:var, "X"}}) + assert %Constraint{op: :is_atom} = Constraint.from_tuple({:is_atom, {:var, "X"}}) + + assert %Constraint{op: :starts_with} = + Constraint.from_tuple({:starts_with, {:var, "X"}, {:const, "hel"}}) + + assert %Constraint{op: :contains} = + Constraint.from_tuple({:contains, {:var, "X"}, {:const, "ell"}}) + + assert %Constraint{op: :member} = + Constraint.from_tuple({:member, {:var, "X"}, {:const, [:a, :b]}}) + end + end + + describe "zero-argument relation query" do + test "query on unary relation" do + defmodule UnaryQueryTest do + use ExDatalog.Schema + + relation :person do + field(:name, :atom) + end + + fact(person(:alice)) + fact(person(:bob)) + + query :all_people do + find(N) + where(person(N)) + end + end + + {:ok, knowledge} = UnaryQueryTest.materialize() + results = UnaryQueryTest.query(:all_people, knowledge) + assert length(results) == 2 + end + end + + describe "queries/0 with multiple queries" do + test "queries returns map with all registered queries" do + defmodule QueriesMapTest do + use ExDatalog.Schema + + relation :edge do + field(:a, :atom) + field(:b, :atom) + end + + fact(edge(:x, :y)) + + query :all_edges do + find(A, B) + where(edge(A, B)) + end + + query :edges_from_x do + find(B) + where(edge(:x, B)) + end + end + + queries = QueriesMapTest.queries() + assert Map.has_key?(queries, :all_edges) + assert Map.has_key?(queries, :edges_from_x) + assert queries.all_edges.relation == "edge" + assert queries.edges_from_x.relation == "edge" + end + end + + describe "single body expression rule" do + test "rule with single literal in body compiles and materializes" do + defmodule SingleBodyExprTest do + use ExDatalog.Schema + + relation :link do + field(:a, :atom) + field(:b, :atom) + end + + relation :path do + field(:a, :atom) + field(:b, :atom) + end + + fact(link(:x, :y)) + + rule path(A, B) do + link(A, B) + end + end + + {:ok, knowledge} = SingleBodyExprTest.materialize() + result = Knowledge.get(knowledge, "path") + assert MapSet.size(result) == 1 + assert {:x, :y} in result + end + end + + describe "UnsupportedFeature struct" do + test "supports accessing planned_for field" do + uf = %ExDatalog.UnsupportedFeature{feature: :aggregates, planned_for: "v0.6.0"} + assert uf.feature == :aggregates + assert uf.planned_for == "v0.6.0" + end + end +end + +defmodule ExDatalog.SchemaErrorTest do + use ExUnit.Case, async: false + + describe "validation errors" do + test "fact referencing undeclared relation raises CompileError at compile time" do + assert_raise ExDatalog.DSL.CompileError, ~r/not declared/, fn -> + Code.compile_string(""" + defmodule FactUndeclaredTestErr do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + fact unknown(:alice, :bob) + end + """) + end + end + + test "query referencing undeclared relation raises CompileError at compile time" do + assert_raise ExDatalog.DSL.CompileError, ~r/not declared/, fn -> + Code.compile_string(""" + defmodule QueryUndeclaredTestErr do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + fact parent(:alice, :bob) + + query :all_unknowns do + find(X) + where unknown(X) + end + end + """) + end + end + + test "unsafe variable in rule head raises when building program" do + defmodule UnsafeVarRuleTest do + use ExDatalog.Schema + + relation :edge do + field(:from, :atom) + field(:to, :atom) + end + + relation :path do + field(:from, :atom) + field(:to, :atom) + end + + fact(edge(:a, :b)) + + rule path(X, Z) do + edge(X, Y) + end + end + + assert_raise ExDatalog.DSL.CompileError, ~r/not in any positive body literal/, fn -> + UnsafeVarRuleTest.program() + end + end + + test "arity mismatch in fact raises when building program" do + defmodule ArityMismatchFactErrTest do + use ExDatalog.Schema + + relation :parent do + field(:p, :atom) + field(:c, :atom) + end + + fact(parent(:alice, :bob)) + end + + program = ArityMismatchFactErrTest.program() + + assert %ExDatalog.Program{} = program + end + + test "rule referencing undeclared relation raises when building program" do + defmodule RuleUndeclaredRelTest do + use ExDatalog.Schema + + relation :edge do + field(:from, :atom) + field(:to, :atom) + end + + relation :path do + field(:from, :atom) + field(:to, :atom) + end + + fact(edge(:a, :b)) + + rule path(X, Y) do + unknown(X, Y) + end + end + + assert_raise ExDatalog.DSL.CompileError, ~r/undefined relation/, fn -> + RuleUndeclaredRelTest.program() + end + end + end +end diff --git a/test/ex_datalog/storage/ets_extra_test.exs b/test/ex_datalog/storage/ets_extra_test.exs index b7b6465..09d685f 100644 --- a/test/ex_datalog/storage/ets_extra_test.exs +++ b/test/ex_datalog/storage/ets_extra_test.exs @@ -12,7 +12,10 @@ defmodule ExDatalog.Storage.ETSConformanceExtraTest do describe "build_index/3 and get_indexed/4" do test "builds single-column index and retrieves matching tuples" do state = ETS.init(@schemas) - state = ETS.insert_many(state, "parent", [{:alice, :bob}, {:carol, :dave}, {:alice, :carol}]) + + state = + ETS.insert_many(state, "parent", [{:alice, :bob}, {:carol, :dave}, {:alice, :carol}]) + state = ETS.build_index(state, "parent", [0]) result = ETS.get_indexed(state, "parent", [0], {:alice}) assert length(result) == 2 @@ -23,7 +26,10 @@ defmodule ExDatalog.Storage.ETSConformanceExtraTest do test "builds multi-column index and retrieves matching tuples" do state = ETS.init(@schemas) - state = ETS.insert_many(state, "parent", [{:alice, :bob}, {:carol, :dave}, {:alice, :carol}]) + + state = + ETS.insert_many(state, "parent", [{:alice, :bob}, {:carol, :dave}, {:alice, :carol}]) + state = ETS.build_index(state, "parent", [0, 1]) result = ETS.get_indexed(state, "parent", [0, 1], {:alice, :bob}) assert result == [{:alice, :bob}] @@ -49,9 +55,11 @@ defmodule ExDatalog.Storage.ETSConformanceExtraTest do test "raises ArgumentError for unknown relation" do state = ETS.init(@schemas) + assert_raise ArgumentError, ~r/unknown relation/, fn -> ETS.build_index(state, "nonexistent", [0]) end + ETS.teardown(state) end end @@ -81,9 +89,11 @@ defmodule ExDatalog.Storage.ETSConformanceExtraTest do test "raises ArgumentError for unknown relation" do state = ETS.init(@schemas) + assert_raise ArgumentError, ~r/unknown relation/, fn -> ETS.update_index(state, "nonexistent", [0], [{:a, :b}]) end + ETS.teardown(state) end end @@ -109,17 +119,21 @@ defmodule ExDatalog.Storage.ETSConformanceExtraTest do test "insert raises ArgumentError for unknown relation" do state = ETS.init(@schemas) + assert_raise ArgumentError, ~r/unknown relation/, fn -> ETS.insert(state, "nonexistent", {:a, :b}) end + ETS.teardown(state) end test "insert_many raises ArgumentError for unknown relation" do state = ETS.init(@schemas) + assert_raise ArgumentError, ~r/unknown relation/, fn -> ETS.insert_many(state, "nonexistent", [{:a, :b}]) end + ETS.teardown(state) end diff --git a/test/ex_datalog/storage/map_extra_test.exs b/test/ex_datalog/storage/map_extra_test.exs index 22aa16d..61d43fe 100644 --- a/test/ex_datalog/storage/map_extra_test.exs +++ b/test/ex_datalog/storage/map_extra_test.exs @@ -11,6 +11,7 @@ defmodule ExDatalog.Storage.MapExtraTest do describe "insert/3 error cases" do test "raises ArgumentError for unknown relation" do state = Map.init(@schemas) + assert_raise ArgumentError, ~r/unknown relation/, fn -> Map.insert(state, "nonexistent", {:a, :b}) end @@ -20,6 +21,7 @@ defmodule ExDatalog.Storage.MapExtraTest do describe "insert_many/3 error cases" do test "raises ArgumentError for unknown relation" do state = Map.init(@schemas) + assert_raise ArgumentError, ~r/unknown relation/, fn -> Map.insert_many(state, "nonexistent", [{:a, :b}]) end @@ -29,6 +31,7 @@ defmodule ExDatalog.Storage.MapExtraTest do describe "build_index/3 error cases" do test "raises ArgumentError for unknown relation" do state = Map.init(@schemas) + assert_raise ArgumentError, ~r/unknown relation/, fn -> Map.build_index(state, "nonexistent", [0]) end @@ -45,6 +48,7 @@ defmodule ExDatalog.Storage.MapExtraTest do describe "update_index/4 error cases" do test "raises ArgumentError for unknown relation" do state = Map.init(@schemas) + assert_raise ArgumentError, ~r/unknown relation/, fn -> Map.update_index(state, "nonexistent", [0], [{:a, :b}]) end diff --git a/test/integration/engine_test.exs b/test/integration/engine_test.exs index 12b9f7e..e56d44c 100644 --- a/test/integration/engine_test.exs +++ b/test/integration/engine_test.exs @@ -537,7 +537,7 @@ defmodule ExDatalog.IntegrationTest do assert knowledge.stats.termination == :fixpoint end -test "max_iterations hit returns :iteration_limit termination with partial results" do + test "max_iterations hit returns :iteration_limit termination with partial results" do {:ok, knowledge} = Program.new() |> Program.add_relation("parent", [:atom, :atom])