thanos · Jun 24, 2026 · Jun 24, 2026 · Jun 24, 2026 · Jun 24, 2026 · Jun 24, 2026
diff --git a/.sobelow-conf b/.sobelow-conf
@@ -1 +1 @@
-[exit: false, format: "txt", ignore_files: [], ignore: ["Config.HTTPS"], out: nil, private: false, router: nil, skip: false, threshold: :low, verbose: false, version: false]
+[exit: false, format: "txt", ignore_files: [], ignore: ["Config.HTTPS", "SQL.Query"], out: nil, private: false, router: nil, skip: false, threshold: :low, verbose: false, version: false]
diff --git a/AGENTS.md b/AGENTS.md
@@ -0,0 +1,134 @@
+# AGENTS.md
+
+Operating guide for AI agents (and humans) working on `terminusdb_ex`.
+Read this before making changes.
+
+## Project
+
+`terminusdb_ex` is an idiomatic Elixir client for TerminusDB (document graph DB
+with version control). Module namespace is `TerminusDB`; files live in
+`lib/terminus_db/*.ex`. App name: `terminusdb_client`.
+
+The full design lives in `ARCHITECTURE.md` and `docs/adr/`. The source spec is
+`baoulo/prompts/prompt-0.1.0.md` (do not edit it).
+
+## Commands
+
+Always run these from the project root. Run the full quality gate before declaring
+a task done:
+
+```bash
+mix deps.get                              # install deps
+mix compile --warnings-as-errors          # compile (fail on warnings)
+mix format                                # auto-format
+mix format --check-formatted              # verify formatting
+mix credo --strict                        # static analysis
+mix sobelow --exit Low                   # security scan (exits non-zero on findings)
+mix test                                  # unit + doctests (hermetic, no Docker)
+mix coveralls                             # coverage (enforced minimum 80% via coveralls.json)
+mix dialyzer                              # type checks (first run is slow; PLT cached)
+mix docs                                  # build docs
+mix quality                               # format+credo+sobelow+dialyzer alias
+mix verify                               # full quality gate + tests + docs
+```
+
+Integration tests (require a live TerminusDB):
+
+```bash
+docker compose up -d                      # start TerminusDB on :6363
+until curl -sf http://localhost:6363/api/ok >/dev/null 2>&1; do sleep 1; done
+mix test --only integration               # run integration suite
+docker compose down                       # teardown
+```
+
+### Verification checklist (run after every change)
+
+1. `mix format --check-formatted`
+2. `mix compile --warnings-as-errors`
+3. `mix credo --strict`
+4. `mix test` (must be 0 failures)
+5. `mix coveralls` (must be ≥ 80%)
+6. `mix dialyzer` (must be 0 errors)
+
+If you cannot find a command, ask the user and then add it here.
+
+## Conventions
+
+### Architecture (non-negotiable)
+
+- **`TerminusDB.Client` is the only module that touches the wire.** All API
+  modules (`Database`, `Document`, …) compose a request and call
+  `TerminusDB.Client.request/4`. Never call `Req` directly outside `Client`.
+- **Immutable config.** `TerminusDB.Config` is data. Scope with `with_database/2`,
+  `with_branch/2`, etc. — never mutate. No `Agent`/process state for connection.
+- **Tuple results.** Public functions return `{:ok, _} | {:error, %TerminusDB.Error{}}`.
+  Provide a `!/`-suffixed variant that raises. Predicates (`exists?/3`) return
+  booleans and may raise on unexpected errors.
+- **Telemetry on every operation.** Pass `area: :<area>` to `Client.request/4`;
+  `Client` emits `[:terminusdb, <area>, :start|:stop]`. Redact config via
+  `TerminusDB.Config.redact/1` before it enters metadata.
+- **String keys.** TerminusDB bodies use JSON keys like `@type`, `api:status`.
+  Jason defaults to `:strings` — do not decode to atoms (atom-exhaustion safety).
+  Match on string keys in error/response handling.
+
+### Style
+
+- No comments unless asked. Code should be self-documenting.
+- Line length 98 (enforced by formatter + credo).
+- Every public function gets `@moduledoc`/`@doc` + `@spec`.
+- Every public function with an example gets a doctest. Doctests must be hermetic
+  (use the Req fake `adapter:` — never hit the network in doctests/unit tests).
+- `@enforce_keys` for required struct fields. Validate options with NimbleOptions.
+- File naming: `TerminusDB.Foo` → `lib/terminus_db/foo.ex`.
+
+### Testing
+
+- **Unit tests** (`test/terminus_db/*_test.exs`): hermetic, fast, no network.
+  Stub HTTP with `Config.new(adapter: fn req -> {req, Req.Response.new(...)} end)`.
+  Assert both the outgoing request (method, URL, body, headers, params) and the
+  returned result/error.
+- **Doctests**: hermetic only.
+- **Property tests** (`test/terminus_db/property/*_test.exs`): use `StreamData`
+  for round-trips and invariants.
+- **Integration tests** (`test/integration/*_test.exs`): tag `@moduletag
+  :integration`; run only with `--only integration` against Dockerized TerminusDB.
+- **Telemetry-in-tests pitfall**: `:telemetry` handlers are global. When asserting
+  on events from concurrent tests, use a **unique request path** and filter
+  `assert_receive` by `meta.path` to avoid cross-test event leakage.
+- Retry is disabled in `Client` (`retry: false`) — keep it that way for
+  predictable tests and behavior.
+
+## Dependency policy
+
+Minimize deps. Runtime (v0.2): `req`, `jason`, `nimble_options`, `telemetry`.
+Ecto and ExDatalog will be **optional** dependencies, gated by compile flags,
+when their integrations land (v0.3/v0.4). Do not add a runtime dep without an
+ADR justifying it.
+
+## Milestone roadmap
+
+| Milestone | Scope | Status |
+| --- | --- | --- |
+| **v0.1 (foundation)** | Config, Error, Telemetry, Client (Req), Database API, unit tests, CI, ADRs, ARCHITECTURE, AGENTS | **Done** |
+| **v0.2 (APIs)** | Document, Schema, Branch, Streaming, Document.query/stream, guides, Livebook | **Done** |
+| v0.2.1+ | Commit, Diff, Merge, GraphQL, WOQL functional DSL, benchmarks | Pending |
+| v0.3 (Ecto) | `use TerminusDB.Schema` macro, schema generation, optional `:ecto` dep; research full `Ecto.Adapter` feasibility | Pending |
+| v0.4 (ExDatalog) | `TerminusDB.Datalog` bridge (`to_woql`, `to_jsonld`, `from_triples`), optional `:datalog` dep | Pending |
+| Release | CHANGELOG, release notes, migration guide, Hex.pm review, final review | Pending |
+
+## Key references
+
+- TerminusDB docs: https://terminusdb.org/docs/
+- TerminusDB OpenAPI: https://github.com/terminusdb/openapi-specs (raw `terminusdb.yaml`)
+- Python client (reference): https://github.com/terminusdb/terminusdb-client-python
+- Req: https://hexdocs.pm/req/Req.html
+- Design decisions: `docs/adr/` (Req, WOQL DSL, Ecto, ExDatalog, Telemetry, Testing, Streaming)
+
+## Do NOT
+
+- Do not commit secrets or `mix.lock` credentials.
+- Do not add `mix.lock` removal; keep `mix.lock` for reproducible CI.
+- Do not raise on expected API errors — return `{:error, %TerminusDB.Error{}}`.
+- Do not decode JSON to atoms.
+- Do not run integration tests in the default `mix test` run.
+- Do not commit changes unless the user explicitly asks.
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
@@ -136,10 +136,11 @@ the graph engine.
 
 **Adopt Option C as the target architecture, delivered incrementally.**
 
-- **v0.1 (this milestone):** Option A core - `Client`, `Config`, `Error`, `Database`,
-  telemetry, streaming, and the WOQL/raw-query execution primitives. This is the
-  verifiable foundation everything else builds on.
-- **v0.2:** Document + Schema + Branch + Commit + Diff + Merge APIs and the WOQL DSL.
+- **v0.1 (done):** Option A core - `Client`, `Config`, `Error`, `Database`,
+  telemetry, and the HTTP/wire primitives.
+- **v0.2 (done):** Document CRUD + query + streaming, Schema frame retrieval,
+  Branch management, and the concatenated-JSON streaming decoder.
+- **v0.2.1+:** Commit, Diff, Merge, GraphQL, and the WOQL functional DSL.
 - **v0.3:** Ecto integration (`TerminusDB.Schema`) - Option B.
 - **v0.4:** ExDatalog integration - Option C.
 - **Option D is rejected** for v0.x; a local engine is out of scope for a client.
@@ -156,19 +157,20 @@ TerminusDB
 ├── Application           OTP supervision tree
 ├── Config                immutable connection/context (NimbleOptions-validated)
 ├── Client                Req-based HTTP wrapper; the only module that touches the wire
+│   └── Params            internal query-param helpers (flag/bool)
 ├── Error                 typed error struct + exception
 ├── Database              database management API
-├── Document              document CRUD + query            (v0.2)
-├── Schema                schema frame API + Ecto macro    (v0.2/v0.3)
-├── Branch                branch API                       (v0.2)
-├── Commit                history / log                    (v0.2)
-├── Diff                  diff + patch                     (v0.2)
-├── Merge                 push / pull / rebase             (v0.2)
-├── WOQL                  functional DSL → JSON-LD         (v0.2)
-├── GraphQL               GraphQL execution                (v0.2)
-├── Datalog               ExDatalog integration            (v0.4)
-├── Telemetry             event definitions + helpers
-└── Streaming             document stream helpers
+├── Document              document CRUD + query + streaming   ✓ v0.2
+├── Schema                schema frame API                    ✓ v0.2
+├── Branch                branch API                          ✓ v0.2
+├── Streaming             concatenated-JSON stream decoder    ✓ v0.2
+├── Commit                history / log                       (v0.2.1+)
+├── Diff                  diff + patch                        (v0.2.1+)
+├── Merge                 push / pull / rebase                (v0.2.1+)
+├── WOQL                  functional DSL → JSON-LD            (v0.2.1+)
+├── GraphQL               GraphQL execution                   (v0.2.1+)
+├── Datalog               ExDatalog integration               (v0.4)
+└── Telemetry             event definitions + helpers
 ```
 
 ### 3.1 Principles

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -5,6 +5,63 @@ All notable changes to `terminusdb_ex` are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.0] — 2026-06-24
+
+### Added — document, schema, branch, and streaming APIs
+
+- **`TerminusDB.Document`**: document CRUD and query API — `insert/3`, `get/2`,
+  `query/3`, `replace/3`, `delete/2`, `stream/2` (plus `!/` variants). Supports
+  `graph_type` (instance/schema), `author`/`message` commit metadata,
+  `full_replace`, `raw_json`, `create`, `nuke`, pagination (`skip`/`count`),
+  `as_list`, `unfold`, `minimized`, `compress_ids`. The `stream/2` function
+  returns a lazy `Enumerable` of decoded documents with constant memory via
+  Req's `into: :self` and the concatenated-JSON splitter.
+- **`TerminusDB.Schema`**: schema frame API — `frame/3`, `all/2` (plus `!/`
+  variants). Supports `compress_ids` and `expand_abstract` params (including
+  explicit `false`).
+- **`TerminusDB.Branch`**: branch management API — `create/3`, `delete/3`,
+  `exists?/3` (plus `!/` variants). Supports `:from`, `:organization`, `:repo`
+  overrides in both the path and the origin body.
+- **`TerminusDB.Streaming`**: incremental concatenated-JSON decoder for streaming
+  document responses (ADR-0007). `split_concatenated/1` (bracket/depth-aware
+  splitter respecting string literals, escapes, and cross-chunk boundaries) and
+  `document_stream/2` (with a configurable receive timeout).
+- Internal shared helper for building query parameters (`Client.Params`),
+  parameters, distinguishing flag params (omit when false) from tri-state bool
+  params (send explicit false to override server defaults).
+- **`TerminusDB.Client.resource_path/2`**: now resolves org/db from config and
+  opts, used by all per-module path builders.
+- Integration tests for Document (insert/query/stream/delete), Schema (frame
+  retrieval), and Branch (create/exists?/delete) against a Dockerized TerminusDB.
+- 54 new unit tests covering Document, Schema, Branch, and Streaming.
+- Guides: `guides/introduction.md` (TerminusDB concepts), `guides/migrating-from-sql.md`
+  (SQL-to-TerminusDB migration by example), `guides/overview.md` (feature walkthrough),
+  `guides/terminusdb_ex_livebook.livemd` (full Livebook demo).
+- Hermetic doctest examples on every public function (27 doctests total).
+
+### Fixed
+
+- `Document.get/2`: `unfold`, `minimized`, `compress_ids`, and `as_list` set to
+  `false` are now sent to the server (previously silently dropped, so the
+  server's `true` defaults always won).
+- `Document.stream/2`: raises `TerminusDB.Error` on client errors instead of
+  `MatchError`.
+- `Branch.create/3`: `:organization` and `:repo` overrides are now reflected in
+  the origin body, not just the path.
+- `Streaming.document_stream/2`: no longer hangs if the server never sends
+  `:done` — a receive timeout halts the stream.
+- `Streaming.split_concatenated/1`: handles a lone trailing `\` at a chunk
+  boundary inside a string (retains it for the next chunk).
+- Flaky telemetry test fixed (unique-path filtered `refute_receive`).
+- All documentation examples corrected: `{:ok, _} = delete(...)` instead of
+  `:ok = delete(...)`.
+
+### Changed
+
+- `Client.resource_path/2` signature changed from `(org, db)` to `(config, opts)`.
+- Version bumped to 0.2.0; `source_ref` updated for HexDocs.
+- `.sobelow-conf` ignores `SQL.Query` false positive on `Document.query/3`.
+
 ## [0.1.0] — 2026-06-23
 
 ### Added — foundation (v0.1)
@@ -52,3 +109,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   manually via `mix test --only integration`.
 
 [0.1.0]: https://github.com/thanos/terminusdb-client-elixir/releases/tag/v0.1.0
+[0.2.0]: https://github.com/thanos/terminusdb-client-elixir/releases/tag/v0.2.0
diff --git a/README.md b/README.md
@@ -11,11 +11,11 @@ graph database with built-in version control. It is built on
 [Req](https://hexdocs.pm/req) and treats connection context as **immutable data**,
 making it safe for concurrent use.
 
-> **Status:** v0.1 (foundation). `Config`, `Error`, `Telemetry`, `Client`, and the
-> `Database` management API are implemented and tested. Document, schema, branch,
-> commit, diff, merge, WOQL, GraphQL, and streaming APIs, plus optional Ecto and
-> ExDatalog integrations, are planned for later milestones. See `ARCHITECTURE.md`
-> and `AGENTS.md` for the roadmap.
+> **Status:** v0.2.0. `Config`, `Error`, `Telemetry`, `Client`, `Database`,
+> `Document` (with streaming), `Schema`, and `Branch` are implemented and tested.
+> Commit, diff, merge, WOQL DSL, GraphQL, and optional Ecto/ExDatalog integrations
+> are planned for later milestones. See `ARCHITECTURE.md` and `AGENTS.md` for the
+> roadmap.
 
 ## Installation
 
@@ -24,7 +24,7 @@ Add `terminusdb_client` to your dependencies in `mix.exs`:
 ```elixir
 def deps do
   [
-    {:terminusdb_client, "~> 0.1.0"}
+    {:terminusdb_client, "~> 0.2.0"}
   ]
 end
 ```
@@ -37,23 +37,57 @@ Then run `mix deps.get`.
 # 1. Configure a connection (immutable context). Default auth is admin:root.
 config = TerminusDB.Config.new(endpoint: "http://localhost:6363")
 
-# 2. Create a database.
+# 2. Create a database with a schema graph.
 {:ok, _} =
   TerminusDB.Database.create(config, "mydb",
     label: "My Database",
     comment: "A demo database",
     schema: true
   )
 
-# 3. Inspect it.
-{:ok, details} = TerminusDB.Database.info(config, "mydb")
-true = TerminusDB.Database.exists?(config, "mydb")
-
-# 4. Scope the config to a database (for later document work).
+# 3. Scope the config to the database for document operations.
 config = TerminusDB.Config.with_database(config, "mydb")
 
-# 5. Clean up.
-:ok = TerminusDB.Database.delete(config, "mydb")
+# 4. Insert a schema (a Class document in the schema graph).
+{:ok, _} =
+  TerminusDB.Document.insert(config,
+    %{"@type" => "Class", "@id" => "Person", "name" => "xsd:string", "age" => "xsd:integer"},
+    author: "admin", message: "add Person schema",
+    graph_type: :schema
+  )
+
+# 5. Insert a document (an instance of Person).
+{:ok, _} =
+  TerminusDB.Document.insert(config,
+    %{"@type" => "Person", "name" => "Alice", "age" => 30},
+    author: "admin", message: "add Alice"
+  )
+
+# 6. Retrieve documents by type.
+{:ok, docs} = TerminusDB.Document.get(config, type: "Person", as_list: true)
+# => [%{"@id" => "Person/Alice", "name" => "Alice", "age" => 30}]
+
+# 7. Query by template (match all Person documents with age 30).
+{:ok, matches} =
+  TerminusDB.Document.query(config, %{"@type" => "Person", "age" => 30})
+
+# 8. Retrieve the schema frame for the Person class.
+{:ok, frame} = TerminusDB.Schema.frame(config, "Person")
+# => %{"@type" => "Class", "name" => "xsd:string", "age" => "xsd:integer"}
+
+# 9. Create a branch and work on it.
+{:ok, _} = TerminusDB.Branch.create(config, "feature")
+feature_config = TerminusDB.Config.with_branch(config, "feature")
+
+# 10. Stream large result sets without loading everything into memory.
+TerminusDB.Document.stream(config, type: "Person")
+|> Stream.each(&IO.inspect/1)
+|> Stream.run()
+
+# 11. Clean up.
+{:ok, _} = TerminusDB.Document.delete(config, id: "Person/Alice", author: "admin", message: "remove")
+{:ok, _} = TerminusDB.Branch.delete(config, "feature")
+{:ok, _} = TerminusDB.Database.delete(config, "mydb")
 ```
 
 All public functions return `{:ok, result}` or `{:error, %TerminusDB.Error{}}`. Each
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		[exit: false, format: "txt", ignore_files: [], ignore: ["Config.HTTPS"], out: nil, private: false, router: nil, skip: false, threshold: :low, verbose: false, version: false]
		[exit: false, format: "txt", ignore_files: [], ignore: ["Config.HTTPS", "SQL.Query"], out: nil, private: false, router: nil, skip: false, threshold: :low, verbose: false, version: false]