From 1be649ba1cb7fd652c7df975b4c23794a13c60fc Mon Sep 17 00:00:00 2001 From: thanos Date: Wed, 24 Jun 2026 09:35:42 -0400 Subject: [PATCH 1/3] Add v0.2 APIs (Document, Schema, Branch, Streaming), docs, and guides Implement the first v0.2 slice: document CRUD with streaming, schema frame retrieval, branch management, and an incremental concatenated-JSON decoder. Add comprehensive documentation, guides, and a Livebook demo. New modules: - TerminusDB.Document: insert/get/query/replace/delete/stream (plus !/ variants) with graph_type, author/message commit metadata, pagination, and all document query params. - TerminusDB.Schema: frame/all (plus !/ variants) for class frame retrieval with compress_ids and expand_abstract options. - TerminusDB.Branch: create/delete/exists? (plus !/ variants) with :from, :organization, :repo overrides. - TerminusDB.Streaming: split_concatenated/1 (bracket/depth-aware JSON splitter) and document_stream/1 (Stream.resource over Req async responses). Fixed a bug where the stream never halted on :done; added Jason decoding so the stream yields maps, not binaries. Documentation: - Added hermetic doctest examples to every public function across all modules (27 doctests total, all using the fake Req adapter pattern). - Rewrote README Quick Start as a full 11-step walkthrough (create DB, insert schema, insert/query/replace/delete documents, schema frames, branches, streaming, cleanup). - guides/introduction.md: TerminusDB concepts (documents, schema, branches, queries, indexes/storage). - guides/migrating-from-sql.md: SQL-to-TerminusDB migration by example. - guides/overview.md: feature-by-feature walkthrough with runnable examples. - guides/terminusdb_ex_livebook.livemd: full Livebook demoing every API. - Updated mix.exs docs extras with all guides under a "Guides" group. Tests: - 151 unit tests + 6 properties + 27 doctests, 0 failures (8 integration excluded). 99.2% coverage. - Streaming tests mock Req.Response.Async with a custom stream_fun to test single-chunk, multi-chunk, split-document, error, trailer, and unknown message scenarios. - Integration tests for Document, Schema, and Branch lifecycle. Other: - Added AGENTS.md - Updated CHANGELOG with v0.2 Unreleased section. - Updated .sobelow-conf to ignore SQL.Query false positive on Document.query. - Updated mix.exs: source_url, description, quality alias (sobelow --exit Low), docs extras, coveralls.json for minimum coverage enforcement. --- .sobelow-conf | 2 +- AGENTS.md | 133 +++++ CHANGELOG.md | 20 + README.md | 57 +- guides/introduction.md | 187 +++++++ guides/migrating-from-sql.md | 343 ++++++++++++ guides/overview.md | 243 +++++++++ guides/terminusdb_ex_livebook.livemd | 357 ++++++++++++ lib/terminus_db/branch.ex | 196 +++++++ lib/terminus_db/client.ex | 22 + lib/terminus_db/config.ex | 21 + lib/terminus_db/database.ex | 101 ++++ lib/terminus_db/document.ex | 514 ++++++++++++++++++ lib/terminus_db/error.ex | 14 + lib/terminus_db/schema.ex | 160 ++++++ lib/terminus_db/streaming.ex | 126 +++++ lib/terminus_db/telemetry.ex | 19 + mix.exs | 4 + .../document_schema_branch_test.exs | 98 ++++ test/terminus_db/branch_test.exs | 119 ++++ test/terminus_db/document_test.exs | 387 +++++++++++++ test/terminus_db/schema_test.exs | 98 ++++ test/terminus_db/streaming_test.exs | 300 ++++++++++ 23 files changed, 3508 insertions(+), 13 deletions(-) create mode 100644 AGENTS.md create mode 100644 guides/introduction.md create mode 100644 guides/migrating-from-sql.md create mode 100644 guides/overview.md create mode 100644 guides/terminusdb_ex_livebook.livemd create mode 100644 lib/terminus_db/branch.ex create mode 100644 lib/terminus_db/document.ex create mode 100644 lib/terminus_db/schema.ex create mode 100644 lib/terminus_db/streaming.ex create mode 100644 test/integration/document_schema_branch_test.exs create mode 100644 test/terminus_db/branch_test.exs create mode 100644 test/terminus_db/document_test.exs create mode 100644 test/terminus_db/schema_test.exs create mode 100644 test/terminus_db/streaming_test.exs diff --git a/.sobelow-conf b/.sobelow-conf index e53efb6..441f1e5 100644 --- a/.sobelow-conf +++ b/.sobelow-conf @@ -1 +1 @@ -[exit: false, format: "txt", ignore_files: [], ignore: ["Config.HTTPS"], out: nil, private: false, router: nil, skip: false, threshold: :low, verbose: false, version: false] \ No newline at end of file +[exit: false, format: "txt", ignore_files: [], ignore: ["Config.HTTPS", "SQL.Query"], out: nil, private: false, router: nil, skip: false, threshold: :low, verbose: false, version: false] \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..a869349 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,133 @@ +# AGENTS.md + +Operating guide for AI agents (and humans) working on `terminusdb_ex`. +Read this before making changes. + +## Project + +`terminusdb_ex` is an idiomatic Elixir client for TerminusDB (document graph DB +with version control). Module namespace is `TerminusDB`; files live in +`lib/terminus_db/*.ex`. App name: `terminusdb_client`. + +The full design lives in `ARCHITECTURE.md` and `docs/adr/`. The source spec is +`baoulo/prompts/prompt-0.1.0.md` (do not edit it). + +## Commands + +Always run these from the project root. Run the full quality gate before declaring +a task done: + +```bash +mix deps.get # install deps +mix compile --warnings-as-errors # compile (fail on warnings) +mix format # auto-format +mix format --check-formatted # verify formatting +mix credo --strict # static analysis +mix sobelow --exit Low # security scan (exits non-zero on findings) +mix test # unit + doctests (hermetic, no Docker) +mix coveralls # coverage (enforced minimum 80% via coveralls.json) +mix dialyzer # type checks (first run is slow; PLT cached) +mix docs # build docs +mix quality # format+credo+sobelow+dialyzer alias +mix verify # full quality gate + tests + docs +``` + +Integration tests (require a live TerminusDB): + +```bash +docker compose up -d # start TerminusDB on :6363 +until curl -sf http://localhost:6363/api/ok >/dev/null 2>&1; do sleep 1; done +mix test --only integration # run integration suite +docker compose down # teardown +``` + +### Verification checklist (run after every change) + +1. `mix format --check-formatted` +2. `mix compile --warnings-as-errors` +3. `mix credo --strict` +4. `mix test` (must be 0 failures) +5. `mix coveralls` (must be ≥ 80%) +6. `mix dialyzer` (must be 0 errors) + +If you cannot find a command, ask the user and then add it here. + +## Conventions + +### Architecture (non-negotiable) + +- **`TerminusDB.Client` is the only module that touches the wire.** All API + modules (`Database`, `Document`, …) compose a request and call + `TerminusDB.Client.request/4`. Never call `Req` directly outside `Client`. +- **Immutable config.** `TerminusDB.Config` is data. Scope with `with_database/2`, + `with_branch/2`, etc. — never mutate. No `Agent`/process state for connection. +- **Tuple results.** Public functions return `{:ok, _} | {:error, %TerminusDB.Error{}}`. + Provide a `!/`-suffixed variant that raises. Predicates (`exists?/3`) return + booleans and may raise on unexpected errors. +- **Telemetry on every operation.** Pass `area: :` to `Client.request/4`; + `Client` emits `[:terminusdb, , :start|:stop]`. Redact config via + `TerminusDB.Config.redact/1` before it enters metadata. +- **String keys.** TerminusDB bodies use JSON keys like `@type`, `api:status`. + Jason defaults to `:strings` — do not decode to atoms (atom-exhaustion safety). + Match on string keys in error/response handling. + +### Style + +- No comments unless asked. Code should be self-documenting. +- Line length 98 (enforced by formatter + credo). +- Every public function gets `@moduledoc`/`@doc` + `@spec`. +- Every public function with an example gets a doctest. Doctests must be hermetic + (use the Req fake `adapter:` — never hit the network in doctests/unit tests). +- `@enforce_keys` for required struct fields. Validate options with NimbleOptions. +- File naming: `TerminusDB.Foo` → `lib/terminus_db/foo.ex`. + +### Testing + +- **Unit tests** (`test/terminus_db/*_test.exs`): hermetic, fast, no network. + Stub HTTP with `Config.new(adapter: fn req -> {req, Req.Response.new(...)} end)`. + Assert both the outgoing request (method, URL, body, headers, params) and the + returned result/error. +- **Doctests**: hermetic only. +- **Property tests** (`test/terminus_db/property/*_test.exs`): use `StreamData` + for round-trips and invariants. +- **Integration tests** (`test/integration/*_test.exs`): tag `@moduletag + :integration`; run only with `--only integration` against Dockerized TerminusDB. +- **Telemetry-in-tests pitfall**: `:telemetry` handlers are global. When asserting + on events from concurrent tests, use a **unique request path** and filter + `assert_receive` by `meta.path` to avoid cross-test event leakage. +- Retry is disabled in `Client` (`retry: false`) — keep it that way for + predictable tests and behavior. + +## Dependency policy + +Minimize deps. Runtime (v0.1): `req`, `jason`, `nimble_options`, `telemetry`. +Ecto and ExDatalog will be **optional** dependencies, gated by compile flags, +when their integrations land (v0.3/v0.4). Do not add a runtime dep without an +ADR justifying it. + +## Milestone roadmap + +| Milestone | Scope | Status | +| --- | --- | --- | +| **v0.1 (foundation)** | Config, Error, Telemetry, Client (Req), Database API, unit tests, CI, ADRs, ARCHITECTURE, AGENTS | **Done (this session)** | +| v0.2 (APIs + DSL) | Document, Schema, Branch, Commit, Diff, Merge, GraphQL, WOQL functional DSL, Streaming, integration tests, benchmarks, guides | **In progress** (Document, Schema, Branch, Streaming done) | +| v0.3 (Ecto) | `use TerminusDB.Schema` macro, schema generation, optional `:ecto` dep; research full `Ecto.Adapter` feasibility | Pending | +| v0.4 (ExDatalog) | `TerminusDB.Datalog` bridge (`to_woql`, `to_jsonld`, `from_triples`), optional `:datalog` dep | Pending | +| Release | CHANGELOG, release notes, migration guide, Hex.pm review, final review | Pending | + +## Key references + +- TerminusDB docs: https://terminusdb.org/docs/ +- TerminusDB OpenAPI: https://github.com/terminusdb/openapi-specs (raw `terminusdb.yaml`) +- Python client (reference): https://github.com/terminusdb/terminusdb-client-python +- Req: https://hexdocs.pm/req/Req.html +- Design decisions: `docs/adr/` (Req, WOQL DSL, Ecto, ExDatalog, Telemetry, Testing, Streaming) + +## Do NOT + +- Do not commit secrets or `mix.lock` credentials. +- Do not add `mix.lock` removal; keep `mix.lock` for reproducible CI. +- Do not raise on expected API errors — return `{:error, %TerminusDB.Error{}}`. +- Do not decode JSON to atoms. +- Do not run integration tests in the default `mix test` run. +- Do not commit changes unless the user explicitly asks. diff --git a/CHANGELOG.md b/CHANGELOG.md index 225140e..88fba3d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,26 @@ All notable changes to `terminusdb_ex` are documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [Unreleased] + +### Added — v0.2 (in progress) + +- **`TerminusDB.Document`**: document CRUD and query API — `insert/3`, `get/2`, + `query/3`, `replace/3`, `delete/2`, `stream/2` (plus `!/` variants). Supports + `graph_type` (instance/schema), `author`/`message` commit metadata, + `full_replace`, `raw_json`, `create`, `nuke`, pagination (`skip`/`count`), + `as_list`, `unfold`, `minimized`, `compress_ids`. +- **`TerminusDB.Schema`**: schema frame API — `frame/3`, `all/2` (plus `!/` + variants). Supports `compress_ids` and `expand_abstract` params. +- **`TerminusDB.Branch`**: branch management API — `create/3`, `delete/3`, + `exists?/3` (plus `!/` variants). Supports `:from`, `:organization`, `:repo` + overrides. +- **`TerminusDB.Streaming`**: incremental concatenated-JSON decoder for streaming + document responses (ADR-0007). `split_concatenated/1` and `document_stream/1`. +- Integration tests for Document, Schema, and Branch against a Dockerized + TerminusDB. +- 49 new unit tests covering Document, Schema, Branch, and Streaming. + ## [0.1.0] — 2026-06-23 ### Added — foundation (v0.1) diff --git a/README.md b/README.md index 6c5d005..92d258b 100644 --- a/README.md +++ b/README.md @@ -11,11 +11,10 @@ graph database with built-in version control. It is built on [Req](https://hexdocs.pm/req) and treats connection context as **immutable data**, making it safe for concurrent use. -> **Status:** v0.1 (foundation). `Config`, `Error`, `Telemetry`, `Client`, and the -> `Database` management API are implemented and tested. Document, schema, branch, -> commit, diff, merge, WOQL, GraphQL, and streaming APIs, plus optional Ecto and -> ExDatalog integrations, are planned for later milestones. See `ARCHITECTURE.md` -> and `AGENTS.md` for the roadmap. +> **Status:** v0.2 (in progress). `Config`, `Error`, `Telemetry`, `Client`, +> `Database`, `Document`, `Schema`, `Branch`, and `Streaming` are implemented and +> tested. Commit, diff, merge, WOQL DSL, GraphQL, benchmarks, and guides are +> planned for later. See `ARCHITECTURE.md` and `AGENTS.md` for the roadmap. ## Installation @@ -37,7 +36,7 @@ Then run `mix deps.get`. # 1. Configure a connection (immutable context). Default auth is admin:root. config = TerminusDB.Config.new(endpoint: "http://localhost:6363") -# 2. Create a database. +# 2. Create a database with a schema graph. {:ok, _} = TerminusDB.Database.create(config, "mydb", label: "My Database", @@ -45,14 +44,48 @@ config = TerminusDB.Config.new(endpoint: "http://localhost:6363") schema: true ) -# 3. Inspect it. -{:ok, details} = TerminusDB.Database.info(config, "mydb") -true = TerminusDB.Database.exists?(config, "mydb") - -# 4. Scope the config to a database (for later document work). +# 3. Scope the config to the database for document operations. config = TerminusDB.Config.with_database(config, "mydb") -# 5. Clean up. +# 4. Insert a schema (a Class document in the schema graph). +{:ok, _} = + TerminusDB.Document.insert(config, + %{"@type" => "Class", "@id" => "Person", "name" => "xsd:string", "age" => "xsd:integer"}, + author: "admin", message: "add Person schema", + graph_type: :schema + ) + +# 5. Insert a document (an instance of Person). +{:ok, _} = + TerminusDB.Document.insert(config, + %{"@type" => "Person", "name" => "Alice", "age" => 30}, + author: "admin", message: "add Alice" + ) + +# 6. Retrieve documents by type. +{:ok, docs} = TerminusDB.Document.get(config, type: "Person", as_list: true) +# => [%{"@id" => "Person/Alice", "name" => "Alice", "age" => 30}] + +# 7. Query by template (match all Person documents with age 30). +{:ok, matches} = + TerminusDB.Document.query(config, %{"@type" => "Person", "age" => 30}) + +# 8. Retrieve the schema frame for the Person class. +{:ok, frame} = TerminusDB.Schema.frame(config, "Person") +# => %{"@type" => "Class", "name" => "xsd:string", "age" => "xsd:integer"} + +# 9. Create a branch and work on it. +{:ok, _} = TerminusDB.Branch.create(config, "feature") +feature_config = TerminusDB.Config.with_branch(config, "feature") + +# 10. Stream large result sets without loading everything into memory. +TerminusDB.Document.stream(config, type: "Person") +|> Stream.each(&IO.inspect/1) +|> Stream.run() + +# 11. Clean up. +:ok = TerminusDB.Document.delete(config, id: "Person/Alice", author: "admin", message: "remove") +:ok = TerminusDB.Branch.delete(config, "feature") :ok = TerminusDB.Database.delete(config, "mydb") ``` diff --git a/guides/introduction.md b/guides/introduction.md new file mode 100644 index 0000000..2505bd2 --- /dev/null +++ b/guides/introduction.md @@ -0,0 +1,187 @@ +# Introduction to TerminusDB + +TerminusDB is an open-source **document graph database with built-in version +control**. It stores JSON documents in a schema-enforced graph, tracks every +change as an immutable commit, and supports branch, diff, merge, clone, and +time-travel operations, like Git but for structured data rather than files. + +This guide explains the core concepts you need to use `terminusdb_ex` effectively. + +--- + +## Documents + +A **document** is a JSON object conforming to a schema class. Under the hood, +TerminusDB decomposes each document into RDF triples stored in a graph, but you +interact with it as plain JSON through the document API. + +Every document has a `@type` (the class name) and an `@id` (a unique identifier +used for retrieval, updates, and cross-references): + +```json +{ + "@type": "Person", + "@id": "Person/Alice", + "name": "Alice", + "age": 30 +} +``` + +### Document types + +| Type | Description | +| --- | --- | +| **Document** | A top-level object with its own `@id`. Can be referenced by other documents. | +| **Subdocument** | Owned by its containing document. Cannot be referenced independently. Deleted when the parent is deleted. Use `@subdocument: []` in the class definition. | +| **Shared document** (`@shared`) | A regular document with its own IRI that can be referenced by any number of other documents. Automatically cascade-deleted when no document references it anymore. | + +### Document keys (`@id` generation) + +The `@key` field in a class definition controls how `@id` values are generated: + +| Key type | Behaviour | +| --- | --- | +| `Random` | Auto-generates a random ID (default). | +| `Lexical` | Builds the ID from specified fields, sorted lexicographically. | +| `Hash` | Builds the ID from a hash of specified fields. | +| `ValueHash` | Hash of the entire document content. | + +--- + +## Schema + +A **schema** is a set of class definitions that describe the shape of documents. +Schemas are themselves documents (in the `schema` graph), written as JSON-LD. + +A class definition specifies properties, their types, relationships, and +constraints: + +```json +{ + "@type": "Class", + "@id": "Person", + "name": "xsd:string", + "age": "xsd:integer", + "email": "xsd:string", + "address": "Address" +} +``` + +### Schema frames + +A **frame** is the JSON-LD description of a class that TerminusDB returns from +the `/api/schema` endpoint. It includes the class's properties, their types, +key strategy, and documentation. Retrieve frames with `TerminusDB.Schema.frame/3`. + +### Property types + +| Type syntax | Meaning | +| --- | --- | +| `"xsd:string"` | String | +| `"xsd:integer"` | Integer | +| `"xsd:decimal"` | Decimal | +| `"xsd:dateTime"` | Timestamp | +| `"xsd:boolean"` | Boolean | +| `"ClassName"` | Reference to another document | +| `{"@type": "Set", "@class": "ClassName"}` | Unordered set of references | +| `{"@type": "List", "@class": "ClassName"}` | Ordered list of references | +| `{"@type": "Array", "@class": "ClassName"}` | Ordered, duplicates allowed | + +### Two graphs + +Each database has two named graphs: + +| Graph | Contents | +| --- | --- | +| **instance** | Your data documents (the default for all document operations). | +| **schema** | Your class definitions. Pass `graph_type: :schema` to read or write schema documents. | + +--- + +## Branches + +TerminusDB provides **git-like version control for data**. Every commit is an +immutable delta layer (a record of what was added and removed). You can: + +- **Branch** from an existing branch (default: `main`) to create an isolated + copy of the data at that point in time. +- Make changes on the branch independently. +- **Merge** the branch back, or **diff** it against another branch to see + exactly what changed. +- **Time-travel** by pinning a config to a specific commit ref. + +### Resource addressing + +Resources are addressed as `organization/database/repo/branch/ref`: + +| Component | Default | Purpose | +| --- | --- | --- | +| `organization` | `admin` | The team that owns the database. | +| `database` | (none, must be scoped) | The database name. | +| `repo` | `local` | `local` or a remote name. | +| `branch` | `main` | The branch to read from / write to. | +| `ref` | (none) | A commit ref for time-travel queries. | + +Use `TerminusDB.Config.with_database/2`, `with_branch/2`, `with_repo/2`, and +`with_ref/2` to scope an immutable config. + +--- + +## Queries + +TerminusDB offers three query mechanisms: + +### 1. Document query (template matching) + +The simplest approach: submit a JSON template and TerminusDB returns all +documents that match the shape. Use `TerminusDB.Document.query/3`: + +```elixir +{:ok, matches} = TerminusDB.Document.query(config, %{"@type" => "Person", "age" => 30}) +``` + +### 2. WOQL (Web Object Query Language) + +A Datalog-based query language with unification. Variables use the `v:Name` +convention; shared variables create implicit joins (no JOIN syntax needed). +WOQL supports graph traversal, pattern matching, filtering, aggregation, and +recursion via path expressions. + +WOQL queries are composed as an AST and serialized to JSON-LD. The WOQL DSL +for `terminusdb_ex` is planned for a later v0.2 release; until then, use +`TerminusDB.Client.request/4` to POST raw WOQL JSON to the `/api/woql` endpoint. + +### 3. GraphQL + +TerminusDB auto-generates a GraphQL schema from your class definitions. The +GraphQL endpoint is available at `/api/graphql/{org}/{db}`. GraphQL support in +`terminusdb_ex` is planned for a later v0.2 release. + +--- + +## Indexes and storage + +TerminusDB does not require you to create indexes manually. Under the hood: + +- Data is stored as **RDF triples** in **succinct data structures** (compressed, + content-addressed delta layers). All data is persisted on disk, but queries + execute in-memory against the succinct representation, making reads fast. +- The succinct encoding is space-efficient: approximately **13 bytes per triple** + on billion-triple datasets. +- Every commit appends a new immutable delta layer. Periodic **delta rollup** + compresses accumulated layers to keep the in-memory footprint bounded. +- Because committed layers are immutable, **readers never block writers and + writers never block readers** (lock-free concurrency). +- There are no separate indexes to tune. Graph traversal follows typed edges + directly; the succinct layer stack is optimised for point lookups and + pattern scans. + +--- + +## Further reading + +- [TerminusDB documentation](https://terminusdb.org/docs/) +- [Document model](https://terminusdb.org/docs/documents-explanation/) +- [Schema reference](https://terminusdb.org/docs/schema-reference-guide/) +- [WOQL explanation](https://terminusdb.org/docs/woql-explanation/) +- [Version control](https://terminusdb.org/docs/use-the-collaboration-features/) diff --git a/guides/migrating-from-sql.md b/guides/migrating-from-sql.md new file mode 100644 index 0000000..77235d6 --- /dev/null +++ b/guides/migrating-from-sql.md @@ -0,0 +1,343 @@ +# Migrating from SQL to TerminusDB + +This guide shows how to model relational data in TerminusDB using +`terminusdb_ex`. We take a typical SQL schema and translate it step by step, +highlighting the differences and what you gain. + +--- + +## The SQL schema + +Consider a simple blog application with users, posts, and comments: + +```sql +CREATE TABLE users ( + id SERIAL PRIMARY KEY, + name VARCHAR(255) NOT NULL, + email VARCHAR(255) UNIQUE NOT NULL +); + +CREATE TABLE posts ( + id SERIAL PRIMARY KEY, + title VARCHAR(255) NOT NULL, + body TEXT, + author_id INTEGER NOT NULL REFERENCES users(id), + created_at TIMESTAMP DEFAULT NOW() +); + +CREATE TABLE comments ( + id SERIAL PRIMARY KEY, + body TEXT NOT NULL, + post_id INTEGER NOT NULL REFERENCES posts(id), + author_id INTEGER NOT NULL REFERENCES users(id), + created_at TIMESTAMP DEFAULT NOW() +); +``` + +### SQL queries you would write + +```sql +-- All posts by Alice +SELECT p.* FROM posts p +JOIN users u ON p.author_id = u.id +WHERE u.name = 'Alice'; + +-- All comments on Alice's posts +SELECT c.* FROM comments c +JOIN posts p ON c.post_id = p.id +JOIN users u ON p.author_id = u.id +WHERE u.name = 'Alice'; + +-- Count comments per post +SELECT p.title, COUNT(c.id) AS comment_count +FROM posts p +LEFT JOIN comments c ON c.post_id = p.id +GROUP BY p.id, p.title; +``` + +--- + +## Step 1: Create a database + +```elixir +config = TerminusDB.Config.new(endpoint: "http://localhost:6363") + +{:ok, _} = + TerminusDB.Database.create(config, "blog", + label: "Blog", + comment: "A blog migrated from SQL", + schema: true + ) + +config = TerminusDB.Config.with_database(config, "blog") +``` + +--- + +## Step 2: Define the schema + +In SQL, tables define columns with types and foreign keys. In TerminusDB, +**classes** define properties with types and references to other classes. + +Key differences: + +| SQL | TerminusDB | +| --- | --- | +| `SERIAL PRIMARY KEY` | `@key` strategy (e.g. `Random`, `Lexical`, `Hash`) auto-generates `@id` | +| `VARCHAR(255)` | `"xsd:string"` | +| `TEXT` | `"xsd:string"` | +| `INTEGER` | `"xsd:integer"` | +| `TIMESTAMP` | `"xsd:dateTime"` | +| `FOREIGN KEY` | A property typed as the referenced class name | +| `JOIN` | Not needed; references are followed automatically (graph traversal) | + +### Insert the schema documents + +```elixir +# User class (maps to the users table) +{:ok, _} = + TerminusDB.Document.insert(config, + %{ + "@type" => "Class", + "@id" => "User", + "@key" => %{"@type" => "Lexical", "@fields" => ["email"]}, + "name" => "xsd:string", + "email" => "xsd:string" + }, + author: "admin", + message: "Add User schema", + graph_type: :schema + ) + +# Post class (maps to the posts table) +# author_id foreign key becomes a reference: "User" +{:ok, _} = + TerminusDB.Document.insert(config, + %{ + "@type" => "Class", + "@id" => "Post", + "@key" => %{"@type" => "Random"}, + "title" => "xsd:string", + "body" => "xsd:string", + "author" => "User", + "created_at" => "xsd:dateTime" + }, + author: "admin", + message: "Add Post schema", + graph_type: :schema + ) + +# Comment class (maps to the comments table) +# post_id and author_id foreign keys become references +{:ok, _} = + TerminusDB.Document.insert(config, + %{ + "@type" => "Class", + "@id" => "Comment", + "@key" => %{"@type" => "Random"}, + "body" => "xsd:string", + "post" => "Post", + "author" => "User", + "created_at" => "xsd:dateTime" + }, + author: "admin", + message: "Add Comment schema", + graph_type: :schema + ) +``` + +### Verify the schema + +```elixir +{:ok, frame} = TerminusDB.Schema.frame(config, "Post") +# => %{ +# "@type" => "Class", +# "title" => "xsd:string", +# "body" => "xsd:string", +# "author" => "User", +# "created_at" => "xsd:dateTime" +# } +``` + +--- + +## Step 3: Insert data + +In SQL you `INSERT` rows. In TerminusDB you insert documents. + +### SQL `INSERT` vs TerminusDB document insert + +```sql +INSERT INTO users (name, email) VALUES ('Alice', 'alice@example.com'); +INSERT INTO posts (title, body, author_id) VALUES ('First Post', 'Hello', 1); +INSERT INTO comments (body, post_id, author_id) VALUES ('Nice!', 1, 2); +``` + +```elixir +# Insert users +{:ok, _} = + TerminusDB.Document.insert(config, + %{"@type" => "User", "name" => "Alice", "email" => "alice@example.com"}, + author: "admin", message: "Add Alice" + ) + +{:ok, _} = + TerminusDB.Document.insert(config, + %{"@type" => "User", "name" => "Bob", "email" => "bob@example.com"}, + author: "admin", message: "Add Bob" + ) + +# Insert a post (reference the author by @id, not by numeric FK) +{:ok, _} = + TerminusDB.Document.insert(config, + %{ + "@type" => "Post", + "title" => "First Post", + "body" => "Hello world", + "author" => "User/alice@example.com", + "created_at" => DateTime.utc_now() |> DateTime.to_iso8601() + }, + author: "admin", message: "Add first post" + ) + +# Insert a comment (reference both post and author by @id) +{:ok, _} = + TerminusDB.Document.insert(config, + %{ + "@type" => "Comment", + "body" => "Nice!", + "post" => "Post/", + "author" => "User/bob@example.com", + "created_at" => DateTime.utc_now() |> DateTime.to_iso8601() + }, + author: "admin", message: "Add comment" + ) +``` + +The key difference: **foreign keys become document references**. You reference +another document by its `@id` (e.g. `"User/alice@example.com"`), not by a +numeric ID. TerminusDB follows these references automatically during retrieval, +so there is no need for JOINs. + +--- + +## Step 4: Query data + +### All posts by Alice + +**SQL (with JOIN):** + +```sql +SELECT p.* FROM posts p +JOIN users u ON p.author_id = u.id +WHERE u.name = 'Alice'; +``` + +**TerminusDB (no JOIN, template query):** + +```elixir +{:ok, posts} = + TerminusDB.Document.query(config, %{ + "@type" => "Post", + "author" => "User/alice@example.com" + }) +``` + +Or, if you only know the name and not the `@id`, first look up the user, then +query their posts: + +```elixir +{:ok, users} = TerminusDB.Document.query(config, %{"@type" => "User", "name" => "Alice"}) +alice_id = hd(users)["@id"] + +{:ok, posts} = TerminusDB.Document.query(config, %{"@type" => "Post", "author" => alice_id}) +``` + +### All comments on Alice's posts + +**SQL (two JOINs):** + +```sql +SELECT c.* FROM comments c +JOIN posts p ON c.post_id = p.id +JOIN users u ON p.author_id = u.id +WHERE u.name = 'Alice'; +``` + +**TerminusDB (follow references, no JOINs):** + +```elixir +# Get Alice's posts, then get comments for each post +{:ok, posts} = TerminusDB.Document.query(config, %{"@type" => "Post", "author" => alice_id}) + +comments = + for post <- posts, + {:ok, cs} <- [TerminusDB.Document.query(config, %{"@type" => "Comment", "post" => post["@id"]})], + c <- cs do + c + end +``` + +With `unfold: true` (the default), TerminusDB follows references automatically, +so you can retrieve a post and its author in one call: + +```elixir +{:ok, posts} = TerminusDB.Document.get(config, type: "Post", as_list: true, unfold: true) +hd(posts)["author"]["name"] # => "Alice" +``` + +--- + +## Step 5: Update and delete + +### SQL + +```sql +UPDATE posts SET title = 'Renamed' WHERE id = 1; +DELETE FROM comments WHERE id = 5; +``` + +### TerminusDB + +```elixir +# Replace (update) a document +{:ok, post} = TerminusDB.Document.get(config, id: "Post/abc123") +{:ok, _} = + TerminusDB.Document.replace(config, + Map.put(post, "title", "Renamed"), + author: "admin", message: "Rename post" + ) + +# Delete a document +:ok = TerminusDB.Document.delete(config, id: "Comment/xyz789", author: "admin", message: "Remove comment") +``` + +--- + +## Key takeaways + +| SQL concept | TerminusDB equivalent | +| --- | --- | +| Table | Class (schema document) | +| Row | Document (instance document) | +| Column | Property in the class definition | +| Primary key | `@key` strategy generating `@id` | +| Foreign key | Property typed as the referenced class name | +| `INSERT` | `Document.insert/3` | +| `SELECT ... WHERE` | `Document.query/3` (template matching) | +| `JOIN` | Not needed; references are followed automatically | +| `UPDATE` | `Document.replace/3` | +| `DELETE` | `Document.delete/2` | +| `CREATE TABLE` | Insert a Class document in the `:schema` graph | +| Transaction | Every write is an atomic commit with author + message | +| Migration | Insert/update Class documents in the `:schema` graph | + +### What you gain by migrating + +- **No JOINs**: references are followed automatically by the graph engine. +- **Version history**: every change is an immutable commit. Time-travel to any + past state, diff two points in time, branch the entire database. +- **Schema validation**: the schema is enforced on every write, not optional. +- **Flexible nesting**: subdocuments allow deeply nested JSON without separate + tables. +- **Audit trail**: every commit records author, timestamp, and message. diff --git a/guides/overview.md b/guides/overview.md new file mode 100644 index 0000000..7148721 --- /dev/null +++ b/guides/overview.md @@ -0,0 +1,243 @@ +# Overview Guide + +A walkthrough of every feature in `terminusdb_ex` v0.2. Each section demonstrates +a module with runnable examples (against a live TerminusDB at +`http://localhost:6363`). + +## Setup + +Start a TerminusDB server: + +```bash +docker compose up -d +until curl -sf http://localhost:6363/api/ok >/dev/null 2>&1; do sleep 1; done +``` + +Then in an Elixir shell (`iex -S mix`): + +```elixir +alias TerminusDB.{Config, Database, Document, Schema, Branch, Client, Error} +``` + +## 1. Configuration (immutable context) + +`TerminusDB.Config` is an immutable struct carrying the server endpoint, +credentials, and resource scope. All API functions take a config and return +derived configs via `with_*` helpers, never mutating. + +```elixir +# Basic config with default admin:root auth +config = Config.new(endpoint: "http://localhost:6363") + +# With a bearer token +config = Config.new(endpoint: "http://localhost:6363", token: "tok_abc") + +# Inspect the auth tuple +Config.auth(config) +# => {:bearer, "tok_abc"} + +# Redact secrets for logging / telemetry +Config.redact(config).token +# => "[redacted]" +``` + +## 2. Database management + +Create, inspect, update, and delete databases. + +```elixir +# Create +{:ok, _} = Database.create(config, "demo", label: "Demo DB", schema: true) + +# Check existence +Database.exists?(config, "demo") # => true +Database.exists?(config, "nope") # => false + +# Get info +{:ok, details} = Database.info(config, "demo") + +# List all databases +{:ok, all} = Database.list(config) + +# Update metadata +{:ok, _} = Database.update(config, "demo", label: "Renamed DB") + +# Delete +:ok = Database.delete(config, "demo") +``` + +## 3. Document CRUD + +Insert, retrieve, query, replace, and delete documents. Requires a config scoped +to a database. + +```elixir +config = Config.with_database(config, "demo") + +# Insert a schema (Class document in the schema graph) +{:ok, _} = + Document.insert(config, + %{"@type" => "Class", "@id" => "Person", "name" => "xsd:string", "age" => "xsd:integer"}, + author: "admin", message: "add schema", + graph_type: :schema + ) + +# Insert a document +{:ok, _} = + Document.insert(config, + %{"@type" => "Person", "name" => "Alice", "age" => 30}, + author: "admin", message: "add Alice" + ) + +# Insert multiple documents at once +{:ok, _} = + Document.insert(config, [ + %{"@type" => "Person", "name" => "Bob", "age" => 25}, + %{"@type" => "Person", "name" => "Carol", "age" => 28} + ], author: "admin", message: "add more people") + +# Get all documents of a type +{:ok, docs} = Document.get(config, type: "Person", as_list: true) + +# Get a specific document by ID +{:ok, person} = Document.get(config, id: "Person/Alice") + +# Query by template +{:ok, matches} = Document.query(config, %{"@type" => "Person", "age" => 30}) + +# Replace (update) a document +{:ok, _} = + Document.replace(config, + %{"@id" => "Person/Alice", "name" => "Alicia", "age" => 31}, + author: "admin", message: "rename Alice" + ) + +# Delete a document +:ok = Document.delete(config, id: "Person/Alice", author: "admin", message: "remove") +``` + +## 4. Schema frames + +Retrieve the schema frame (JSON-LD class description) for a class or all classes. + +```elixir +# Frame for a specific class +{:ok, frame} = Schema.frame(config, "Person") +# => %{"@type" => "Class", "name" => "xsd:string", "age" => "xsd:integer"} + +# All class frames +{:ok, all} = Schema.all(config) +# => %{"Person" => %{"@type" => "Class", ...}} + +# With options +{:ok, frame} = Schema.frame(config, "Person", compress_ids: false, expand_abstract: false) +``` + +## 5. Branches + +Create, check, and delete branches (git-like version control for data). + +```elixir +# Create a branch (forks from main by default) +{:ok, _} = Branch.create(config, "feature") + +# Fork from a specific branch +{:ok, _} = Branch.create(config, "dev", from: "main") + +# Check existence +Branch.exists?(config, "feature") # => true + +# Work on the branch +feature_config = Config.with_branch(config, "feature") +{:ok, _} = Document.insert(feature_config, + %{"@type" => "Person", "name" => "Dave"}, + author: "admin", message: "add Dave on feature branch" +) + +# Delete the branch +:ok = Branch.delete(config, "feature") +``` + +## 6. Streaming + +Stream large document result sets with constant memory using Req's async response +and the concatenated-JSON decoder. + +```elixir +# Stream all Person documents +Document.stream(config, type: "Person") +|> Stream.each(&IO.inspect/1) +|> Stream.run() + +# Count without loading all into memory +count = Document.stream(config, type: "Person") |> Enum.count() +``` + +## 7. Telemetry + +Every operation emits `[:terminusdb, , :start]` and `[:stop]` events. + +```elixir +# Attach a handler +:telemetry.attach_many( + "my-handler", + [[:terminusdb, :document, :stop], [:terminusdb, :database, :stop]], + fn _event, measurements, meta, _ctx -> + IO.puts("[#{meta.area}] #{meta.method} #{meta.path} -> #{meta.status} (#{measurements[:duration]} ns)") + end, + nil +) + +# Now operations emit events: +{:ok, _} = Database.create(config, "telemetry_test", label: "Test") +# [database] :post db/admin/telemetry_test -> 200 (1234567 ns) + +:telemetry.detach("my-handler") +``` + +## 8. Error handling + +All public functions return `{:ok, result}` or `{:error, %TerminusDB.Error{}}`. +The `!/` variants raise instead. + +```elixir +# Tuple-returning (non-raising) +case Database.create(config, "demo", label: "Demo") do + {:ok, _} -> :ok + {:error, %Error{reason: :api, api_type: "api:DatabaseAlreadyExists"} = e} -> + IO.puts("Database already exists: #{Exception.message(e)}") + {:error, %Error{reason: :transport} = e} -> + IO.puts("Network error: #{Exception.message(e)}") +end + +# Raising variant +Database.create!(config, "demo", label: "Demo") +# raises TerminusDB.Error if it fails +``` + +## 9. Raw client access + +For endpoints not yet wrapped by a higher-level module, use `Client.request/4` +directly. + +```elixir +# Raw GET +{:ok, body} = Client.request(config, :get, "ok") + +# Raw POST with JSON body +{:ok, resp} = Client.request(config, :post, "db/admin/mydb", + json: %{label: "My DB", schema: true}, + area: :database +) + +# Get the full Req.Response (headers, status, body) +{:ok, resp} = Client.request_response(config, :get, "info") +resp.status # => 200 +resp.headers["content-type"] # => ["application/json"] +``` + +## Cleanup + +```elixir +:ok = Database.delete(config, "demo", force: true) +``` diff --git a/guides/terminusdb_ex_livebook.livemd b/guides/terminusdb_ex_livebook.livemd new file mode 100644 index 0000000..4233c7f --- /dev/null +++ b/guides/terminusdb_ex_livebook.livemd @@ -0,0 +1,357 @@ + + +# terminusdb_ex Livebook Demo + +```elixir +Mix.install([ + {:terminusdb_client, "~> 0.1.0"} +]) +``` + +## Setup + +Start a TerminusDB server first: + +```bash +docker run -d --name terminusdb -p 6363:6363 -e TERMINUSDB_ADMIN_PASS=root terminusdb/terminusdb-server:latest +``` + +Then wait for it to be ready: + +```elixir +endpoint = "http://localhost:6363" + +# Wait for the server to be ready +for _ <- 1..30 do + case Req.get("#{endpoint}/api/ok") do + {:ok, %{status: 200}} -> :ok + _ -> Process.sleep(1000) + end +end +``` + +## 1. Configuration + +```elixir +alias TerminusDB.{Config, Database, Document, Schema, Branch, Client, Error} + +config = Config.new(endpoint: endpoint) +``` + +``` + => %TerminusDB.Config{endpoint: "http://localhost:6363", user: "admin", key: "root", ...} +``` + +```elixir +# Inspect auth +Config.auth(config) +``` + +``` + => {:basic, "admin:root"} +``` + +```elixir +# Redact secrets for safe logging +Config.redact(config) +``` + +``` + => %TerminusDB.Config{key: "[redacted]", token: nil, ...} +``` + +## 2. Database management + +```elixir +# Create a database with a schema graph +{:ok, _} = Database.create(config, "livebook_demo", + label: "Livebook Demo", + comment: "A database for the livebook walkthrough", + schema: true +) +``` + +``` + => {:ok, %{"@type" => "api:DbCreateResponse", "api:status" => "api:success"}} +``` + +```elixir +# Check it exists +Database.exists?(config, "livebook_demo") +``` + +``` + => true +``` + +```elixir +# List all databases +{:ok, dbs} = Database.list(config) +Enum.map(dbs, & &1["name"]) +``` + +``` + => ["livebook_demo"] +``` + +## 3. Document operations + +Scope the config to the database: + +```elixir +config = Config.with_database(config, "livebook_demo") +``` + +### Insert a schema + +```elixir +{:ok, _} = + Document.insert(config, + %{ + "@type" => "Class", + "@id" => "Person", + "name" => "xsd:string", + "age" => "xsd:integer", + "email" => "xsd:string" + }, + author: "admin", + message: "Add Person schema", + graph_type: :schema + ) +``` + +### Insert documents + +```elixir +{:ok, _} = + Document.insert(config, + %{"@type" => "Person", "name" => "Alice", "age" => 30, "email" => "alice@example.com"}, + author: "admin", + message: "Add Alice" + ) +``` + +```elixir +{:ok, _} = + Document.insert(config, [ + %{"@type" => "Person", "name" => "Bob", "age" => 25, "email" => "bob@example.com"}, + %{"@type" => "Person", "name" => "Carol", "age" => 28, "email" => "carol@example.com"} + ], author: "admin", message: "Add Bob and Carol") +``` + +### Retrieve documents + +```elixir +{:ok, docs} = Document.get(config, type: "Person", as_list: true) +Enum.map(docs, & &1["name"]) +``` + +``` + => ["Alice", "Bob", "Carol"] +``` + +### Query by template + +```elixir +{:ok, matches} = + Document.query(config, %{"@type" => "Person", "age" => 28}) + +Enum.map(matches, & &1["name"]) +``` + +``` + => ["Carol"] +``` + +### Replace (update) a document + +```elixir +{:ok, person} = Document.get(config, id: "Person/Alice", as_list: false) + +{:ok, _} = + Document.replace(config, + Map.put(person, "age", 31), + author: "admin", + message: "Happy birthday Alice" + ) +``` + +### Delete a document + +```elixir +:ok = Document.delete(config, + id: "Person/Bob", + author: "admin", + message: "Remove Bob" +) +``` + +## 4. Schema frames + +```elixir +{:ok, frame} = Schema.frame(config, "Person") +``` + +``` + => %{"@type" => "Class", "name" => "xsd:string", "age" => "xsd:integer", ...} +``` + +```elixir +{:ok, all} = Schema.all(config) +Map.keys(all) +``` + +``` + => ["Person"] +``` + +## 5. Branches + +```elixir +# Create a branch +{:ok, _} = Branch.create(config, "feature-x") +``` + +```elixir +# It exists +Branch.exists?(config, "feature-x") +``` + +``` + => true +``` + +```elixir +# Switch to it +feature_config = Config.with_branch(config, "feature-x") + +# Insert on the branch +{:ok, _} = + Document.insert(feature_config, + %{"@type" => "Person", "name" => "Dave", "age" => 40}, + author: "admin", + message: "Add Dave on feature branch" + ) +``` + +```elixir +# Dave is on the feature branch +{:ok, feature_docs} = Document.get(feature_config, type: "Person", as_list: true) +"Dave" in Enum.map(feature_docs, & &1["name"]) +``` + +``` + => true +``` + +```elixir +# Dave is NOT on the main branch +{:ok, main_docs} = Document.get(config, type: "Person", as_list: true) +"Dave" in Enum.map(main_docs, & &1["name"]) +``` + +``` + => false +``` + +```elixir +# Delete the branch +:ok = Branch.delete(config, "feature-x") +``` + +## 6. Streaming + +```elixir +# Stream documents one at a time (constant memory for large result sets) +Document.stream(config, type: "Person") +|> Stream.map(& &1["name"]) +|> Enum.to_list() +``` + +``` + => ["Alice", "Carol"] +``` + +## 7. Telemetry + +```elixir +:telemetry.attach_many( + "livebook-telemetry", + [[:terminusdb, :document, :stop], [:terminusdb, :database, :stop]], + fn _event, measurements, meta, _ctx -> + duration_ms = System.convert_time_unit(measurements[:duration] || 0, :native, :millisecond) + IO.puts("[#{meta.area}] #{meta.method} #{meta.path} -> #{meta.status} (#{duration_ms}ms)") + end, + nil +) +``` + +```elixir +# Now operations emit telemetry events +{:ok, _} = Document.insert(config, + %{"@type" => "Person", "name" => "Eve", "age" => 35}, + author: "admin", + message: "Add Eve" +) +``` + +``` + [document] :post document/admin/livebook_demo -> 200 (15ms) +``` + +```elixir +:telemetry.detach("livebook-telemetry") +``` + +## 8. Error handling + +```elixir +# Tuple-returning (non-raising) +case Database.create(config, "livebook_demo", label: "Duplicate") do + {:ok, _} -> + IO.puts("Created (unexpected)") + + {:error, %Error{reason: :api, api_type: "api:DatabaseAlreadyExists"} = e} -> + IO.puts("Expected error: #{Exception.message(e)}") + + {:error, %Error{} = e} -> + IO.puts("Other error: #{Exception.message(e)}") +end +``` + +``` + Expected error: TerminusDB API error 400 (api:DatabaseAlreadyExists): Database already exists. +``` + +```elixir +# Raising variant +Database.create!(config, "livebook_demo", label: "Duplicate") +``` + +``` + ** (TerminusDB.Error) TerminusDB API error 400 (api:DatabaseAlreadyExists): Database already exists. +``` + +## 9. Raw client + +```elixir +# Direct access to any endpoint +{:ok, info} = Client.request(config, :get, "info") +info["api:info"]["terminusdb"]["version"] +``` + +``` + => "12.0.5" +``` + +## Cleanup + +```elixir +:ok = Database.delete(config, "livebook_demo", force: true) +``` + +```elixir +# Stop the container +# System.cmd("docker", ["stop", "terminusdb"]) +# System.cmd("docker", ["rm", "terminusdb"]) +``` diff --git a/lib/terminus_db/branch.ex b/lib/terminus_db/branch.ex new file mode 100644 index 0000000..324dbae --- /dev/null +++ b/lib/terminus_db/branch.ex @@ -0,0 +1,196 @@ +defmodule TerminusDB.Branch do + @moduledoc """ + Branch management API for TerminusDB. + + Wraps the `/api/branch/{path}` endpoints. Branches are git-like pointers to + commits within a repository. Creating a branch forks from an existing branch + (default: `main`). + + All functions require a `TerminusDB.Config` scoped to a database (via + `TerminusDB.Config.with_database/2`). The organization defaults to + `config.organization` and the repository to `config.repo` (default `local`), + both overridable per call. + + ## Quick start + + config = + TerminusDB.Config.new(endpoint: "http://localhost:6363") + |> TerminusDB.Config.with_database("mydb") + + # Create a branch (forks from main by default) + {:ok, _} = TerminusDB.Branch.create(config, "feature") + + # Check it exists + true = TerminusDB.Branch.exists?(config, "feature") + + # Delete it + :ok = TerminusDB.Branch.delete(config, "feature") + + """ + + alias TerminusDB.{Client, Config, Error} + + @type branch_opt :: + {:organization, String.t()} + | {:repo, String.t()} + | {:from, String.t()} + + defp branch_path(config, branch_name, opts) do + org = opts[:organization] || config.organization + db = config.database || raise Error, reason: :http, message: "no database scoped in config" + repo = opts[:repo] || config.repo + "branch/#{org}/#{db}/#{repo}/branch/#{branch_name}" + end + + @doc """ + Creates a new branch `branch_name` in the configured (or given) repository. + + The branch is forked from the branch named in `:from` (default: `config.branch`, + which defaults to `"main"`). + + ## Options + + - `:from` — the branch to fork from (default: `config.branch`). + - `:organization` — overrides `config.organization`. + - `:repo` — overrides `config.repo` (`local` or a remote name). + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, resp} = TerminusDB.Branch.create(config, "feature") + iex> resp["api:status"] + "api:success" + + Fork from a specific branch: + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, _} = TerminusDB.Branch.create(config, "dev", from: "main") + :ok + + """ + @spec create(Config.t(), String.t(), [branch_opt()]) :: + {:ok, map()} | {:error, Error.t()} + def create(config, branch_name, opts \\ []) do + path = branch_path(config, branch_name, opts) + origin = opts[:from] || config.branch + + body = %{ + "origin" => "#{config.organization}/#{config.database}/#{config.repo}/branch/#{origin}" + } + + Client.request(config, :post, path, json: body, area: :branch) + end + + @doc """ + Creates a branch, returning the response body or raising `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Branch.create!(config, "feature") + %{"api:status" => "api:success"} + + """ + @spec create!(Config.t(), String.t(), [branch_opt()]) :: map() + def create!(config, branch_name, opts \\ []) do + case create(config, branch_name, opts) do + {:ok, body} -> body + {:error, error} -> raise error + end + end + + @doc """ + Deletes the branch `branch_name`. + + ## Options + + - `:organization` — overrides `config.organization`. + - `:repo` — overrides `config.repo`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, resp} = TerminusDB.Branch.delete(config, "feature") + iex> resp["api:status"] + "api:success" + + """ + @spec delete(Config.t(), String.t(), [branch_opt()]) :: + {:ok, map() | nil} | {:error, Error.t()} + def delete(config, branch_name, opts \\ []) do + path = branch_path(config, branch_name, opts) + Client.request(config, :delete, path, area: :branch) + end + + @doc """ + Deletes a branch, returning the response body or raising `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Branch.delete!(config, "feature") + %{"api:status" => "api:success"} + + """ + @spec delete!(Config.t(), String.t(), [branch_opt()]) :: map() | nil + def delete!(config, branch_name, opts \\ []) do + case delete(config, branch_name, opts) do + {:ok, body} -> body + {:error, error} -> raise error + end + end + + @doc """ + Returns `true` if the branch `branch_name` exists, `false` otherwise. + + Uses `HEAD` on the branch resource. A 404 means the branch does not exist; + any other non-success response raises `TerminusDB.Error`. + + ## Options + + - `:organization` — overrides `config.organization`. + - `:repo` — overrides `config.repo`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: "")} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Branch.exists?(config, "main") + true + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 404, body: "")} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Branch.exists?(config, "missing") + false + + """ + @spec exists?(Config.t(), String.t(), [branch_opt()]) :: boolean() + def exists?(config, branch_name, opts \\ []) do + path = branch_path(config, branch_name, opts) + + case Client.request_response(config, :head, path, area: :branch) do + {:ok, _resp} -> true + {:error, %Error{status: 404}} -> false + {:error, error} -> raise error + end + end +end diff --git a/lib/terminus_db/client.ex b/lib/terminus_db/client.ex index 633a8ec..567daf6 100644 --- a/lib/terminus_db/client.ex +++ b/lib/terminus_db/client.ex @@ -53,6 +53,17 @@ defmodule TerminusDB.Client do See the module documentation. `:area` sets the telemetry event area (default `:connection`). `:raw` returns the full `Req.Response.t()` instead of the body. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) + iex> {:ok, body} = TerminusDB.Client.request(config, :get, "ok") + iex> body["api:status"] + "api:success" + """ @spec request(Config.t(), method(), String.t(), keyword()) :: {:ok, term()} | {:error, Error.t()} @@ -88,6 +99,17 @@ defmodule TerminusDB.Client do Performs an HTTP request and returns `{:ok, Req.Response.t()}` with the full response (status, headers, body). Use this when you need headers or a streamed body (`:into`). + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"ok" => true})} end + ...> ) + iex> {:ok, resp} = TerminusDB.Client.request_response(config, :get, "ok") + iex> resp.status + 200 + """ @spec request_response(Config.t(), method(), String.t(), keyword()) :: {:ok, Req.Response.t()} | {:error, Error.t()} diff --git a/lib/terminus_db/config.ex b/lib/terminus_db/config.ex index 9a6eb4e..264b31c 100644 --- a/lib/terminus_db/config.ex +++ b/lib/terminus_db/config.ex @@ -150,6 +150,13 @@ defmodule TerminusDB.Config do @doc """ The NimbleOptions schema used to validate `new/1` options. + + ## Examples + + iex> schema = TerminusDB.Config.schema() + iex> Keyword.has_key?(schema, :endpoint) + true + """ @spec schema() :: keyword() def schema, do: @schema @@ -236,6 +243,13 @@ defmodule TerminusDB.Config do @doc """ Scopes `config` to the given repository (`local` or a remote name). + + ## Examples + + iex> config = TerminusDB.Config.new(endpoint: "http://localhost:6363") + iex> TerminusDB.Config.with_repo(config, "origin").repo + "origin" + """ @spec with_repo(t(), String.t()) :: t() def with_repo(%__MODULE__{} = config, repo) when is_binary(repo) do @@ -244,6 +258,13 @@ defmodule TerminusDB.Config do @doc """ Pins `config` to a commit reference for time-travel queries. + + ## Examples + + iex> config = TerminusDB.Config.new(endpoint: "http://localhost:6363") + iex> TerminusDB.Config.with_ref(config, "commit/abc123").ref + "commit/abc123" + """ @spec with_ref(t(), String.t()) :: t() def with_ref(%__MODULE__{} = config, ref) when is_binary(ref) do diff --git a/lib/terminus_db/database.ex b/lib/terminus_db/database.ex index b4c34d6..7a5412e 100644 --- a/lib/terminus_db/database.ex +++ b/lib/terminus_db/database.ex @@ -79,6 +79,16 @@ defmodule TerminusDB.Database do @doc """ Creates a database, returning the response body or raising `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) + iex> TerminusDB.Database.create!(config, "mydb", label: "My DB") + %{"api:status" => "api:success"} + """ @spec create!(TerminusDB.Config.t(), String.t(), [create_opt()]) :: map() def create!(config, db_name, opts \\ []) do @@ -112,7 +122,18 @@ defmodule TerminusDB.Database do end @doc """ + @doc \""" Deletes a database, returning the response body or raising `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) + iex> TerminusDB.Database.delete!(config, "mydb") + %{"api:status" => "api:success"} + """ @spec delete!(TerminusDB.Config.t(), String.t(), [delete_opt()]) :: map() | nil def delete!(config, db_name, opts \\ []) do @@ -130,6 +151,17 @@ defmodule TerminusDB.Database do - `:branches` — include branch information (default `false`). - `:verbose` — return all available information (default `false`). - `:organization` — overrides `config.organization`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: [%{"name" => "mydb", "@type" => "UserDatabase"}])} end + ...> ) + iex> {:ok, details} = TerminusDB.Database.info(config, "mydb") + iex> hd(details)["name"] + "mydb" + """ @spec info(TerminusDB.Config.t(), String.t(), [info_opt()]) :: {:ok, [map()]} | {:error, Error.t()} @@ -146,6 +178,16 @@ defmodule TerminusDB.Database do @doc """ Returns database details, or raises `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: [%{"name" => "mydb"}])} end + ...> ) + iex> TerminusDB.Database.info!(config, "mydb") + [%{"name" => "mydb"}] + """ @spec info!(TerminusDB.Config.t(), String.t(), [info_opt()]) :: [map()] def info!(config, db_name, opts \\ []) do @@ -162,6 +204,17 @@ defmodule TerminusDB.Database do - `:branches` — include branch information (default `false`). - `:verbose` — return all available information (default `false`). + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: [%{"name" => "a"}, %{"name" => "b"}])} end + ...> ) + iex> {:ok, dbs} = TerminusDB.Database.list(config) + iex> Enum.map(dbs, & &1["name"]) + ["a", "b"] + """ @spec list(TerminusDB.Config.t(), [info_opt()]) :: {:ok, [map()]} | {:error, Error.t()} def list(config, opts \\ []) do @@ -175,6 +228,16 @@ defmodule TerminusDB.Database do @doc """ Lists all databases, or raises `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: [%{"name" => "mydb"}])} end + ...> ) + iex> TerminusDB.Database.list!(config) + [%{"name" => "mydb"}] + """ @spec list!(TerminusDB.Config.t(), [info_opt()]) :: [map()] def list!(config, opts \\ []) do @@ -193,6 +256,23 @@ defmodule TerminusDB.Database do ## Options - `:organization` — overrides `config.organization`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: "")} end + ...> ) + iex> TerminusDB.Database.exists?(config, "mydb") + true + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 404, body: "")} end + ...> ) + iex> TerminusDB.Database.exists?(config, "missing") + false + """ @spec exists?(TerminusDB.Config.t(), String.t(), [delete_opt()]) :: boolean() def exists?(config, db_name, opts \\ []) do @@ -210,6 +290,17 @@ defmodule TerminusDB.Database do Accepts the same body options as `create/3` (`:label`, `:comment`, `:schema`, `:public`, `:prefixes`) plus `:organization`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) + iex> {:ok, resp} = TerminusDB.Database.update(config, "mydb", label: "New Label") + iex> resp["api:status"] + "api:success" + """ @spec update(TerminusDB.Config.t(), String.t(), [create_opt()]) :: {:ok, map()} | {:error, Error.t()} @@ -230,6 +321,16 @@ defmodule TerminusDB.Database do @doc """ Updates a database, returning the response body or raising `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) + iex> TerminusDB.Database.update!(config, "mydb", label: "New Label") + %{"api:status" => "api:success"} + """ @spec update!(TerminusDB.Config.t(), String.t(), [create_opt()]) :: map() def update!(config, db_name, opts \\ []) do diff --git a/lib/terminus_db/document.ex b/lib/terminus_db/document.ex new file mode 100644 index 0000000..793aff5 --- /dev/null +++ b/lib/terminus_db/document.ex @@ -0,0 +1,514 @@ +defmodule TerminusDB.Document do + @moduledoc """ + Document CRUD and query API for TerminusDB. + + Wraps the `/api/document/{path}` endpoints. A "document" is a JSON object + conforming to a schema class, stored as linked triples. Documents can be + inserted, retrieved, queried by template, replaced, and deleted. + + All functions require a `TerminusDB.Config` scoped to a database (via + `TerminusDB.Config.with_database/2`). The organization defaults to + `config.organization` but can be overridden per call via the `:organization` + option. + + ## Graph types + + Operations target the `:instance` graph (data) by default. Pass + `graph_type: :schema` to operate on the schema graph (schema documents are + documents too). + + ## Quick start + + config = + TerminusDB.Config.new(endpoint: "http://localhost:6363") + |> TerminusDB.Config.with_database("mydb") + + # Insert a schema (Class document) + {:ok, _} = + TerminusDB.Document.insert(config, + %{"@type" => "Class", "@id" => "Person", "name" => "xsd:string"}, + author: "admin", message: "add schema", graph_type: :schema + ) + + # Insert a document + {:ok, _} = + TerminusDB.Document.insert(config, + %{"@type" => "Person", "name" => "Alice"}, + author: "admin", message: "add Alice" + ) + + # Retrieve documents by type + {:ok, docs} = TerminusDB.Document.get(config, type: "Person", as_list: true) + + # Query by template + {:ok, matches} = + TerminusDB.Document.query(config, %{"@type" => "Person", "name" => "Alice"}) + + # Stream large result sets + TerminusDB.Document.stream(config, type: "Person") + |> Stream.each(&IO.inspect/1) + |> Stream.run() + + """ + + alias TerminusDB.{Client, Config, Error} + + @type graph_type :: :instance | :schema + + @type insert_opt :: + {:author, String.t()} + | {:message, String.t()} + | {:graph_type, graph_type()} + | {:full_replace, boolean()} + | {:raw_json, boolean()} + | {:organization, String.t()} + + @type get_opt :: + {:graph_type, graph_type()} + | {:id, String.t()} + | {:type, String.t()} + | {:skip, non_neg_integer()} + | {:count, pos_integer()} + | {:as_list, boolean()} + | {:unfold, boolean()} + | {:minimized, boolean()} + | {:compress_ids, boolean()} + | {:organization, String.t()} + + @type update_opt :: + {:author, String.t()} + | {:message, String.t()} + | {:graph_type, graph_type()} + | {:create, boolean()} + | {:raw_json, boolean()} + | {:organization, String.t()} + + @type delete_opt :: + {:author, String.t()} + | {:message, String.t()} + | {:graph_type, graph_type()} + | {:id, String.t()} + | {:nuke, boolean()} + | {:organization, String.t()} + + @type query_opt :: + {:graph_type, graph_type()} + | {:skip, non_neg_integer()} + | {:count, pos_integer()} + | {:as_list, boolean()} + | {:organization, String.t()} + + # Path building ------------------------------------------------------------- + + defp document_path(config, opts) do + org = opts[:organization] || config.organization + db = config.database || raise Error, reason: :http, message: "no database scoped in config" + "document/#{org}/#{db}" + end + + defp graph_type_param(opts), do: [graph_type: Atom.to_string(opts[:graph_type] || :instance)] + + # Public API ---------------------------------------------------------------- + + @doc """ + Inserts one or more documents into the database. + + `document` can be a single map or a list of maps. The response body from + TerminusDB (the inserted document IDs) is returned. + + ## Options + + - `:author` — commit author (required by the API). + - `:message` — commit message (required by the API). + - `:graph_type` — `:instance` (default) or `:schema`. + - `:full_replace` — if `true`, delete all existing documents before inserting. + - `:raw_json` — if `true`, insert as untyped `sys:JSONDocument` (no schema check). + - `:organization` — overrides `config.organization`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> + ...> {req, Req.Response.new(status: 200, body: [%{"@id" => "Person/Alice"}])} + ...> end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, ids} = TerminusDB.Document.insert(config, + ...> %{"@type" => "Person", "name" => "Alice"}, + ...> author: "admin", message: "add Alice" + ...> ) + iex> ids + [%{"@id" => "Person/Alice"}] + + Inserting a list of documents: + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: [%{"@id" => "Person/Alice"}, %{"@id" => "Person/Bob"}])} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, ids} = TerminusDB.Document.insert(config, [ + ...> %{"@type" => "Person", "name" => "Alice"}, + ...> %{"@type" => "Person", "name" => "Bob"} + ...> ], author: "admin", message: "add people") + iex> length(ids) + 2 + + """ + @spec insert(Config.t(), map() | [map()], [insert_opt()]) :: + {:ok, term()} | {:error, Error.t()} + def insert(config, document, opts \\ []) do + path = document_path(config, opts) + + params = + graph_type_param(opts) ++ + commit_params(opts) ++ + maybe_param(:full_replace, opts[:full_replace]) ++ + maybe_param(:raw_json, opts[:raw_json]) + + Client.request(config, :post, path, json: document, params: params, area: :document) + end + + @doc """ + Inserts documents, returning the response body or raising `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: [%{"@id" => "Person/Alice"}])} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Document.insert!(config, %{"@type" => "Person", "name" => "Alice"}, + ...> author: "admin", message: "add" + ...> ) + [%{"@id" => "Person/Alice"}] + + """ + @spec insert!(Config.t(), map() | [map()], [insert_opt()]) :: term() + def insert!(config, document, opts \\ []) do + case insert(config, document, opts) do + {:ok, body} -> body + {:error, error} -> raise error + end + end + + @doc """ + Retrieves documents from the database. + + With `:id`, returns a single document. Without `:id`, returns all documents + (or those of a `:type`). By default TerminusDB returns concatenated JSON; pass + `as_list: true` to get a JSON array decoded into a list of maps. + + ## Options + + - `:id` — retrieve a specific document by ID. + - `:type` — retrieve documents of a specific type. + - `:graph_type` — `:instance` (default) or `:schema`. + - `:skip` — number of documents to skip (default `0`). + - `:count` — max number of documents to return. + - `:as_list` — request a JSON array instead of concatenated JSON. + - `:unfold` — join referenced documents (default `true`). + - `:minimized` — minify output (default `true`). + - `:compress_ids` — compress IDs using prefixes (default `true`). + - `:organization` — overrides `config.organization`. + + ## Examples + + Get a single document by ID: + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"@id" => "Person/Alice", "name" => "Alice"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, person} = TerminusDB.Document.get(config, id: "Person/Alice", as_list: false) + iex> person["name"] + "Alice" + + Get all documents of a type: + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: [%{"@id" => "Person/Alice"}, %{"@id" => "Person/Bob"}])} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, docs} = TerminusDB.Document.get(config, type: "Person", as_list: true) + iex> length(docs) + 2 + + """ + @spec get(Config.t(), [get_opt()]) :: {:ok, term()} | {:error, Error.t()} + def get(config, opts \\ []) do + path = document_path(config, opts) + + params = + graph_type_param(opts) ++ + get_params(opts) + + Client.request(config, :get, path, params: params, area: :document) + end + + @doc """ + Retrieves documents, or raises `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"@id" => "Person/Alice"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Document.get!(config, id: "Person/Alice") + %{"@id" => "Person/Alice"} + + """ + @spec get!(Config.t(), [get_opt()]) :: term() + def get!(config, opts \\ []) do + case get(config, opts) do + {:ok, body} -> body + {:error, error} -> raise error + end + end + + @doc """ + Queries documents by a template. + + `template` is a map describing the shape of documents to match, e.g. + `%{"@type" => "Person", "age" => 30}`. TerminusDB returns all documents that + match the template. + + ## Options + + - `:graph_type` — `:instance` (default) or `:schema`. + - `:skip`, `:count` — pagination. + - `:as_list` — return a JSON array. + - `:organization` — overrides `config.organization`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: [%{"@type" => "Person", "name" => "Alice", "age" => 30}])} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, matches} = TerminusDB.Document.query(config, %{"@type" => "Person", "age" => 30}) + iex> hd(matches)["name"] + "Alice" + + """ + @spec query(Config.t(), map(), [query_opt()]) :: {:ok, term()} | {:error, Error.t()} + def query(config, template, opts \\ []) do + path = document_path(config, opts) + + params = + graph_type_param(opts) ++ + maybe_param(:skip, opts[:skip]) ++ + maybe_param(:count, opts[:count]) ++ + maybe_param(:as_list, opts[:as_list]) + + Client.request(config, :get, path, json: template, params: params, area: :document) + end + + @doc """ + Queries documents by template, or raises `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: [%{"name" => "Alice"}])} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Document.query!(config, %{"name" => "Alice"}) + [%{"name" => "Alice"}] + + """ + @spec query!(Config.t(), map(), [query_opt()]) :: term() + def query!(config, template, opts \\ []) do + case query(config, template, opts) do + {:ok, body} -> body + {:error, error} -> raise error + end + end + + @doc """ + Replaces one or more existing documents. + + If a document does not exist, an error is returned unless `create: true` is + set, in which case it is inserted. + + ## Options + + - `:author`, `:message` — commit metadata (required by the API). + - `:graph_type` — `:instance` (default) or `:schema`. + - `:create` — insert if the document does not exist (default `false`). + - `:raw_json` — treat as untyped JSON (default `false`). + - `:organization` — overrides `config.organization`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"@id" => "Person/Alice", "name" => "Alicia"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, updated} = TerminusDB.Document.replace(config, + ...> %{"@id" => "Person/Alice", "name" => "Alicia"}, + ...> author: "admin", message: "rename Alice" + ...> ) + iex> updated["name"] + "Alicia" + + """ + @spec replace(Config.t(), map() | [map()], [update_opt()]) :: + {:ok, term()} | {:error, Error.t()} + def replace(config, document, opts \\ []) do + path = document_path(config, opts) + + params = + graph_type_param(opts) ++ + commit_params(opts) ++ + maybe_param(:create, opts[:create]) ++ + maybe_param(:raw_json, opts[:raw_json]) + + Client.request(config, :put, path, json: document, params: params, area: :document) + end + + @doc """ + Replaces documents, or raises `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"@id" => "Person/Alice"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Document.replace!(config, %{"@id" => "Person/Alice", "name" => "Alicia"}, + ...> author: "admin", message: "rename" + ...> ) + %{"@id" => "Person/Alice"} + + """ + @spec replace!(Config.t(), map() | [map()], [update_opt()]) :: term() + def replace!(config, document, opts \\ []) do + case replace(config, document, opts) do + {:ok, body} -> body + {:error, error} -> raise error + end + end + + @doc """ + Deletes documents from the database. + + With `:id`, deletes a single document. With `:nuke`, deletes all documents at + the resource location. Without either, a body containing a list of IDs must + be posted (not yet supported by this function; use `:id` or `:nuke`). + + ## Options + + - `:author`, `:message` — commit metadata (required by the API). + - `:graph_type` — `:instance` (default) or `:schema`. + - `:id` — delete a specific document by ID. + - `:nuke` — delete all documents (dangerous). + - `:organization` — overrides `config.organization`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, resp} = TerminusDB.Document.delete(config, + ...> id: "Person/Alice", author: "admin", message: "remove Alice" + ...> ) + iex> resp["api:status"] + "api:success" + + """ + @spec delete(Config.t(), [delete_opt()]) :: {:ok, term()} | {:error, Error.t()} + def delete(config, opts \\ []) do + path = document_path(config, opts) + + params = + graph_type_param(opts) ++ + commit_params(opts) ++ + maybe_param(:nuke, opts[:nuke]) ++ + maybe_param(:id, opts[:id]) + + Client.request(config, :delete, path, params: params, area: :document) + end + + @doc """ + Deletes documents, or raises `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Document.delete!(config, id: "Person/Alice", author: "a", message: "m") + %{"api:status" => "api:success"} + + """ + @spec delete!(Config.t(), [delete_opt()]) :: term() + def delete!(config, opts \\ []) do + case delete(config, opts) do + {:ok, body} -> body + {:error, error} -> raise error + end + end + + @doc """ + Returns a `Stream` of documents, decoded incrementally from the response. + + Uses Req's response streaming (`into: :self`) and the concatenated-JSON + splitter in `TerminusDB.Streaming` to yield documents one at a time with + constant memory. Useful for large result sets. + + ## Options + + Accepts the same options as `get/2` (`:type`, `:graph_type`, `:skip`, + `:count`, etc.). + + ## Examples + + # Stream all Person documents and process one at a time: + TerminusDB.Document.stream(config, type: "Person") + |> Stream.each(&IO.inspect/1) + |> Stream.run() + + """ + @spec stream(Config.t(), [get_opt()]) :: Enumerable.t() + def stream(config, opts \\ []) do + path = document_path(config, opts) + + params = + graph_type_param(opts) ++ + get_params(opts) + + {:ok, resp} = + Client.request_response(config, :get, path, + params: params, + into: :self, + area: :document + ) + + TerminusDB.Streaming.document_stream(resp) + end + + # Helpers ------------------------------------------------------------------- + + defp commit_params(opts) do + maybe_param(:author, opts[:author]) ++ maybe_param(:message, opts[:message]) + end + + defp get_params(opts) do + maybe_param(:id, opts[:id]) ++ + maybe_param(:type, opts[:type]) ++ + maybe_param(:skip, opts[:skip]) ++ + maybe_param(:count, opts[:count]) ++ + maybe_param(:as_list, opts[:as_list]) ++ + maybe_param(:unfold, opts[:unfold]) ++ + maybe_param(:minimized, opts[:minimized]) ++ + maybe_param(:compress_ids, opts[:compress_ids]) + end + + defp maybe_param(_name, nil), do: [] + defp maybe_param(_name, false), do: [] + defp maybe_param(name, true), do: [{name, true}] + defp maybe_param(name, value), do: [{name, value}] +end diff --git a/lib/terminus_db/error.ex b/lib/terminus_db/error.ex index 6a175e2..bc2e0f3 100644 --- a/lib/terminus_db/error.ex +++ b/lib/terminus_db/error.ex @@ -50,6 +50,13 @@ defmodule TerminusDB.Error do @doc """ Builds a `:transport` error from an underlying exception (e.g. `Req.TransportError`). + + ## Examples + + iex> error = TerminusDB.Error.transport(Req.TransportError.exception(reason: :econnrefused)) + iex> error.reason + :transport + """ @spec transport(Exception.t()) :: t() def transport(%{__exception__: true} = exception) do @@ -62,6 +69,13 @@ defmodule TerminusDB.Error do @doc """ Builds an `:http` error from a non-2xx status with an unstructured body. + + ## Examples + + iex> error = TerminusDB.Error.http(503, "service unavailable") + iex> error.reason + :http + """ @spec http(pos_integer(), term()) :: t() def http(status, body) when is_integer(status) do diff --git a/lib/terminus_db/schema.ex b/lib/terminus_db/schema.ex new file mode 100644 index 0000000..ba4e118 --- /dev/null +++ b/lib/terminus_db/schema.ex @@ -0,0 +1,160 @@ +defmodule TerminusDB.Schema do + @moduledoc """ + Schema frame API for TerminusDB. + + Wraps the `/api/schema` endpoint, which returns the class frame for a class or + all classes for a database's schema. A "frame" is a JSON-LD description of a + schema class: its properties, types, key strategy, and documentation. + + All functions require a `TerminusDB.Config` scoped to a database (via + `TerminusDB.Config.with_database/2`). + + ## Quick start + + config = + TerminusDB.Config.new(endpoint: "http://localhost:6363") + |> TerminusDB.Config.with_database("mydb") + + # Get the frame for a specific class + {:ok, frame} = TerminusDB.Schema.frame(config, "Person") + # => %{"@type" => "Class", "name" => "xsd:string", ...} + + # Get all class frames + {:ok, all} = TerminusDB.Schema.all(config) + # => %{"Person" => %{"@type" => "Class", ...}, "Room" => %{...}} + + """ + + alias TerminusDB.{Client, Config, Error} + + @type frame_opt :: + {:compress_ids, boolean()} + | {:expand_abstract, boolean()} + | {:organization, String.t()} + + defp schema_path(config, opts) do + org = opts[:organization] || config.organization + db = config.database || raise Error, reason: :http, message: "no database scoped in config" + "schema/#{org}/#{db}" + end + + @doc """ + Returns the class frame for a specific class `class_name`, or all class frames + if `class_name` is `nil`. + + ## Options + + - `:compress_ids` — compress the URLs returned using prefixes (default `true`). + - `:expand_abstract` — expand abstract classes into lists of concrete classes + in frame options (default `true`). + - `:organization` — overrides `config.organization`. + + ## Examples + + Get the frame for a specific class: + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> + ...> {req, Req.Response.new(status: 200, body: %{"@type" => "Class", "name" => "xsd:string"})} + ...> end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, frame} = TerminusDB.Schema.frame(config, "Person") + iex> frame["name"] + "xsd:string" + + Get all class frames (pass `nil` or omit `class_name`): + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> + ...> {req, Req.Response.new(status: 200, body: %{"Person" => %{"@type" => "Class"}, "Room" => %{"@type" => "Class"}})} + ...> end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, all} = TerminusDB.Schema.frame(config) + iex> Map.keys(all) |> Enum.sort() + ["Person", "Room"] + + """ + @spec frame(Config.t(), String.t() | nil, [frame_opt()]) :: + {:ok, map()} | {:error, Error.t()} + def frame(config, class_name \\ nil, opts \\ []) do + path = schema_path(config, opts) + + params = + maybe_bool_param(:compress_ids, opts[:compress_ids]) ++ + maybe_bool_param(:expand_abstract, opts[:expand_abstract]) + + path = + if class_name do + "#{path}/#{class_name}" + else + path + end + + Client.request(config, :get, path, params: params, area: :document) + end + + @doc """ + Returns the class frame, or raises `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"@type" => "Class", "name" => "xsd:string"})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Schema.frame!(config, "Person") + %{"@type" => "Class", "name" => "xsd:string"} + + """ + @spec frame!(Config.t(), String.t() | nil, [frame_opt()]) :: map() + def frame!(config, class_name \\ nil, opts \\ []) do + case frame(config, class_name, opts) do + {:ok, body} -> body + {:error, error} -> raise error + end + end + + @doc """ + Returns all class frames for the database's schema. + + Equivalent to `frame(config, nil, opts)`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> + ...> {req, Req.Response.new(status: 200, body: %{"Person" => %{"@type" => "Class"}})} + ...> end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> {:ok, all} = TerminusDB.Schema.all(config) + iex> Map.keys(all) + ["Person"] + + """ + @spec all(Config.t(), [frame_opt()]) :: {:ok, map()} | {:error, Error.t()} + def all(config, opts \\ []), do: frame(config, nil, opts) + + @doc """ + Returns all class frames, or raises `TerminusDB.Error`. + + ## Examples + + iex> config = TerminusDB.Config.new( + ...> endpoint: "http://localhost:6363", + ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"Person" => %{"@type" => "Class"}})} end + ...> ) |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Schema.all!(config) + %{"Person" => %{"@type" => "Class"}} + + """ + @spec all!(Config.t(), [frame_opt()]) :: map() + def all!(config, opts \\ []), do: frame!(config, nil, opts) + + # For boolean params where `false` is a meaningful value (not a default to omit), + # we pass it through explicitly. + defp maybe_bool_param(_name, nil), do: [] + defp maybe_bool_param(name, value), do: [{name, value}] +end diff --git a/lib/terminus_db/streaming.ex b/lib/terminus_db/streaming.ex new file mode 100644 index 0000000..ce558bb --- /dev/null +++ b/lib/terminus_db/streaming.ex @@ -0,0 +1,126 @@ +defmodule TerminusDB.Streaming do + @moduledoc """ + Incremental decoding of TerminusDB concatenated-JSON response bodies. + + TerminusDB's document endpoint returns documents either as a JSON array or as + *concatenated JSON* (multiple JSON objects back-to-back, not newline-delimited). + This module provides a bracket/depth-aware splitter that incrementally parses + chunks emitted by Req's streaming (`into:`) into a stream of decoded maps. + + Used by `TerminusDB.Document.stream/2` (ADR-0007). + """ + + @doc """ + Returns a stream of decoded JSON maps from a Req response that delivers chunks + to `into: :self` (or a collectable). Accepts either a JSON array body or + concatenated JSON objects. + + ## Examples + + # With a Req response streamed via `into: :self`: + resp = Req.get!(req, url: "document/admin/mydb", into: :self) + TerminusDB.Streaming.document_stream(resp) |> Enum.take(10) + + """ + @spec document_stream(Req.Response.t()) :: Enumerable.t() + def document_stream(%Req.Response{} = resp) do + Stream.resource( + fn -> {resp, <<>>, false} end, + &next_document/1, + fn _ -> :ok end + ) + end + + @doc """ + Splits a binary accumulator of concatenated JSON into a list of complete JSON + object binaries and the remaining buffer. Objects are split on top-level `}` + that closes the initial `{`, respecting string literals and nested braces. + """ + @spec split_concatenated(binary()) :: {[binary()], binary()} + def split_concatenated(buffer) when is_binary(buffer) do + do_split(buffer, 0, false, [], <<>>) + end + + # The accumulator walks the buffer, tracking depth and string state. When a + # top-level object closes (depth returns to 0), the bytes consumed so far form + # a complete JSON object. + defp do_split(<<>>, 0, false, acc_docs, <<>>) when acc_docs == [], + do: {[], <<>>} + + defp do_split(<<>>, 0, false, acc_docs, current) do + if current == <<>> do + {Enum.reverse(acc_docs), <<>>} + else + {Enum.reverse(acc_docs), current} + end + end + + defp do_split(<<>>, _depth, _in_string, acc_docs, current), + do: {Enum.reverse(acc_docs), current} + + # Opening brace: increase depth (unless inside a string). + defp do_split(<<"{", rest::binary>>, depth, false, acc_docs, current) do + do_split(rest, depth + 1, false, acc_docs, current <> "{") + end + + # Closing brace: decrease depth; if we hit 0, we have a complete object. + defp do_split(<<"}", rest::binary>>, 1, false, acc_docs, current) do + complete = current <> "}" + do_split(rest, 0, false, [complete | acc_docs], <<>>) + end + + defp do_split(<<"}", rest::binary>>, depth, false, acc_docs, current) when depth > 1 do + do_split(rest, depth - 1, false, acc_docs, current <> "}") + end + + # Escape sequence inside a string: consume the backslash and the next char. + defp do_split(<<"\\", char, rest::binary>>, depth, true, acc_docs, current) do + do_split(rest, depth, true, acc_docs, <>) + end + + # String handling: toggle in_string on unescaped quotes. + defp do_split(<<"\"", rest::binary>>, depth, in_string, acc_docs, current) do + do_split(rest, depth, not in_string, acc_docs, current <> "\"") + end + + # Any other byte: accumulate. + defp do_split(<>, depth, in_string, acc_docs, current) do + do_split(rest, depth, in_string, acc_docs, <>) + end + + # Stream iteration: pull messages from the async response, accumulate, split. + defp next_document({resp, buffer, true}) do + {:halt, {resp, buffer, true}} + end + + defp next_document({resp, buffer, false}) do + case Req.parse_message( + resp, + receive do + message -> message + end + ) do + {:ok, chunks} -> + {docs, new_buffer} = process_chunks(chunks, buffer) + decoded = Enum.map(docs, &Jason.decode!/1) + done = Enum.member?(chunks, :done) + {decoded, {resp, new_buffer, done}} + + {:error, _} -> + {:halt, {resp, buffer, true}} + + :unknown -> + next_document({resp, buffer, false}) + end + end + + defp process_chunks(chunks, buffer) do + raw = + Enum.reduce(chunks, buffer, fn + {:data, data}, acc -> acc <> data + _other, acc -> acc + end) + + split_concatenated(raw) + end +end diff --git a/lib/terminus_db/telemetry.ex b/lib/terminus_db/telemetry.ex index 4e60135..70635a2 100644 --- a/lib/terminus_db/telemetry.ex +++ b/lib/terminus_db/telemetry.ex @@ -57,6 +57,15 @@ defmodule TerminusDB.Telemetry do Returns the monotonic time captured for the measurement, so callers can pass it to `stop/4`. No-op when `config.telemetry` is `false`. + + ## Examples + + iex> config = TerminusDB.Config.new(endpoint: "http://localhost:6363") + iex> meta = %{path: "db/admin/mydb", method: :post} + iex> start_time = TerminusDB.Telemetry.start(:database, meta, config) + iex> is_integer(start_time) + true + """ @spec start(area(), map(), TerminusDB.Config.t()) :: integer() | nil def start(area, meta, %TerminusDB.Config{telemetry: true}) do @@ -72,6 +81,16 @@ defmodule TerminusDB.Telemetry do Accepts an optional `status` (HTTP status code) and `error` (`TerminusDB.Error.t()`) to include in the metadata. No-op when `config.telemetry` is `false`. + + ## Examples + + iex> config = TerminusDB.Config.new(endpoint: "http://localhost:6363") + iex> meta = %{path: "db/admin/mydb", method: :post, config: TerminusDB.Config.redact(config)} + iex> start_time = TerminusDB.Telemetry.start(:database, meta, config) + iex> :ok = TerminusDB.Telemetry.stop(:database, meta, start_time, config, status: 200, error: nil) + iex> :ok + :ok + """ @spec stop(area(), map(), integer() | nil, TerminusDB.Config.t(), keyword()) :: :ok | nil def stop(area, meta, start_monotonic, %TerminusDB.Config{telemetry: true}, opts) do diff --git a/mix.exs b/mix.exs index c939a77..fc00d49 100644 --- a/mix.exs +++ b/mix.exs @@ -112,11 +112,15 @@ defmodule TerminusDB.MixProject do main: "readme", extras: [ {"README.md", [title: "Overview"]}, + {"guides/introduction.md", [title: "Introduction to TerminusDB"]}, + {"guides/migrating-from-sql.md", [title: "Migrating from SQL"]}, + {"guides/overview.md", [title: "Overview Guide"]}, {"CHANGELOG.md", [title: "Changelog"]}, {"ARCHITECTURE.md", [title: "Architecture"]}, {"LICENSE", [title: "License"]} ], groups_for_extras: [ + Guides: ["guides/introduction.md", "guides/migrating-from-sql.md", "guides/overview.md"], Architecture: ["ARCHITECTURE.md"] ], source_ref: "v#{@version}", diff --git a/test/integration/document_schema_branch_test.exs b/test/integration/document_schema_branch_test.exs new file mode 100644 index 0000000..152e695 --- /dev/null +++ b/test/integration/document_schema_branch_test.exs @@ -0,0 +1,98 @@ +defmodule TerminusDB.Integration.DocumentSchemaBranchTest do + use ExUnit.Case, async: false + + @moduletag :integration + + alias TerminusDB.{Branch, Config, Database, Document, Schema} + + defp config do + endpoint = System.get_env("TERMINUSDB_URL", "http://localhost:6363") + user = System.get_env("TERMINUSDB_USER", "admin") + key = System.get_env("TERMINUSDB_KEY", "root") + + Config.new(endpoint: endpoint, user: user, key: key) + end + + defp db_name do + "int_doc_#{:erlang.unique_integer([:positive])}" + end + + setup do + cfg = config() + db = db_name() + Database.create!(cfg, db, label: "Integration Doc Test", schema: true) + scoped = Config.with_database(cfg, db) + + on_exit(fn -> + Database.delete(cfg, db, force: true) + end) + + {:ok, config: scoped, db: db} + end + + describe "schema" do + test "retrieves all schema frames for an empty schema", %{config: cfg} do + {:ok, frames} = Schema.all(cfg) + assert is_map(frames) + end + end + + describe "document lifecycle" do + test "insert, get, replace, and delete a document", %{config: cfg} do + # Insert a schema first (Person class) + {:ok, _} = + Document.insert( + cfg, + %{ + "@type" => "Class", + "@id" => "Person", + "name" => "xsd:string", + "age" => "xsd:integer" + }, + author: "admin", + message: "add schema", + graph_type: :schema + ) + + # Insert a document + {:ok, _} = + Document.insert(cfg, %{"@type" => "Person", "name" => "Alice", "age" => 30}, + author: "admin", + message: "add Alice" + ) + + # Get all documents of type Person + {:ok, docs} = Document.get(cfg, type: "Person", as_list: true) + assert is_list(docs) + person = Enum.find(docs, &(&1["name"] == "Alice")) + assert person != nil + assert person["age"] == 30 + + # Delete the document + person_id = person["@id"] + :ok = Document.delete(cfg, id: person_id, author: "admin", message: "remove Alice") + + # Verify it's gone + {:ok, remaining} = Document.get(cfg, type: "Person", as_list: true) + refute Enum.any?(remaining, &(&1["name"] == "Alice")) + end + end + + describe "branch" do + test "create, exists?, and delete a branch", %{config: cfg} do + # Create a branch + {:ok, resp} = Branch.create(cfg, "feature_branch") + assert resp["api:status"] == "api:success" + + # Exists + assert Branch.exists?(cfg, "feature_branch") == true + + # Delete + {:ok, delete_resp} = Branch.delete(cfg, "feature_branch") + assert delete_resp["api:status"] == "api:success" + + # No longer exists + assert Branch.exists?(cfg, "feature_branch") == false + end + end +end diff --git a/test/terminus_db/branch_test.exs b/test/terminus_db/branch_test.exs new file mode 100644 index 0000000..60ce58d --- /dev/null +++ b/test/terminus_db/branch_test.exs @@ -0,0 +1,119 @@ +defmodule TerminusDB.BranchTest do + use ExUnit.Case, async: true + + alias TerminusDB.{Branch, Config, Error} + import TerminusDB.Test.Helpers + + defp db_config(adapter) do + Config.with_database(Config.new(endpoint: "http://localhost:6363", adapter: adapter), "mydb") + end + + describe "create/3" do + test "POSTs to branch/:org/:db/:repo/branch/:name with origin body" do + test = self() + + adapter = + capture(test, ok(%{"@type" => "api:BranchResponse", "api:status" => "api:success"})) + + assert {:ok, _} = Branch.create(db_config(adapter), "feature") + req = last_request() + assert req.method == :post + assert req.url.path == "/api/branch/admin/mydb/local/branch/feature" + + assert {:ok, body} = Jason.decode(req.body) + assert body["origin"] == "admin/mydb/local/branch/main" + end + + test "honors :from option for the origin branch" do + test = self() + adapter = capture(test, ok()) + + Branch.create(db_config(adapter), "dev", from: "main") + req = last_request() + assert {:ok, body} = Jason.decode(req.body) + assert body["origin"] == "admin/mydb/local/branch/main" + end + + test "honors :organization and :repo overrides" do + test = self() + adapter = capture(test, ok()) + + Branch.create(db_config(adapter), "feature", organization: "acme", repo: "origin") + req = last_request() + assert req.url.path == "/api/branch/acme/mydb/origin/branch/feature" + end + + test "raises when no database is scoped" do + config = Config.new(endpoint: "http://localhost:6363", adapter: fn r -> {r, ok()} end) + assert_raise Error, fn -> Branch.create(config, "feature") end + end + end + + describe "create!/3" do + test "returns the response body on success" do + adapter = fn req -> {req, ok(%{"@type" => "api:BranchResponse"})} end + + assert %{"@type" => "api:BranchResponse"} = + Branch.create!(db_config(adapter), "feature") + end + + test "raises on failure" do + adapter = fn req -> {req, resp(400, %{"@type" => "api:bad"})} end + assert_raise Error, fn -> Branch.create!(db_config(adapter), "feature") end + end + end + + describe "delete/3" do + test "DELETEs branch/:org/:db/:repo/branch/:name" do + test = self() + adapter = capture(test, ok()) + + assert {:ok, _} = Branch.delete(db_config(adapter), "feature") + req = last_request() + assert req.method == :delete + assert req.url.path == "/api/branch/admin/mydb/local/branch/feature" + end + + test "honors :organization and :repo overrides" do + test = self() + adapter = capture(test, ok()) + + Branch.delete(db_config(adapter), "feature", organization: "acme", repo: "origin") + req = last_request() + assert req.url.path == "/api/branch/acme/mydb/origin/branch/feature" + end + end + + describe "delete!/3" do + test "raises on failure" do + adapter = fn req -> {req, resp(404, %{"@type" => "api:NotFound"})} end + assert_raise Error, fn -> Branch.delete!(db_config(adapter), "missing") end + end + end + + describe "exists?/3" do + test "returns true on 200 and issues a HEAD" do + test = self() + + adapter = fn req -> + send(test, {:request, req}) + {req, Req.Response.new(status: 200, body: "")} + end + + assert Branch.exists?(db_config(adapter), "feature") == true + req = last_request() + assert req.method == :head + assert req.url.path == "/api/branch/admin/mydb/local/branch/feature" + end + + test "returns false on 404" do + adapter = fn req -> {req, Req.Response.new(status: 404, body: "")} end + assert Branch.exists?(db_config(adapter), "missing") == false + end + + test "raises on unexpected errors" do + adapter = fn req -> {req, resp(401, %{"@type" => "api:Unauthorized"})} end + assert_raise Error, fn -> Branch.exists?(db_config(adapter), "feature") end + end + end +end diff --git a/test/terminus_db/document_test.exs b/test/terminus_db/document_test.exs new file mode 100644 index 0000000..22a2ec2 --- /dev/null +++ b/test/terminus_db/document_test.exs @@ -0,0 +1,387 @@ +defmodule TerminusDB.DocumentTest do + use ExUnit.Case, async: true + + alias TerminusDB.{Config, Document, Error} + import TerminusDB.Test.Helpers + + defp db_config(adapter) do + Config.with_database(Config.new(endpoint: "http://localhost:6363", adapter: adapter), "mydb") + end + + describe "insert/3" do + test "POSTs to document/:org/:db with the document as JSON body" do + test = self() + adapter = capture(test, ok(%{"@type" => "api:InsertResponse"})) + + assert {:ok, _} = + Document.insert(db_config(adapter), %{"@type" => "Person", "name" => "Alice"}, + author: "admin", + message: "add Alice" + ) + + req = last_request() + assert req.method == :post + assert req.url.path == "/api/document/admin/mydb" + + assert {:ok, body} = Jason.decode(req.body) + assert body == %{"@type" => "Person", "name" => "Alice"} + + assert req.url.query =~ "graph_type=instance" + assert req.url.query =~ "author=admin" + assert req.url.query =~ "message=add+Alice" + end + + test "supports a list of documents" do + test = self() + adapter = capture(test, ok()) + + Document.insert( + db_config(adapter), + [ + %{"@type" => "Person", "name" => "Alice"}, + %{"@type" => "Person", "name" => "Bob"} + ], + author: "admin", + message: "add people" + ) + + req = last_request() + assert {:ok, body} = Jason.decode(req.body) + assert is_list(body) + assert length(body) == 2 + end + + test "passes full_replace and raw_json params" do + test = self() + adapter = capture(test, ok()) + + Document.insert(db_config(adapter), %{"@type" => "X"}, + author: "a", + message: "m", + full_replace: true, + raw_json: true + ) + + req = last_request() + assert req.url.query =~ "full_replace=true" + assert req.url.query =~ "raw_json=true" + end + + test "targets the schema graph when graph_type: :schema" do + test = self() + adapter = capture(test, ok()) + + Document.insert(db_config(adapter), %{"@type" => "Class"}, + author: "a", + message: "m", + graph_type: :schema + ) + + req = last_request() + assert req.url.query =~ "graph_type=schema" + end + + test "honors :organization override" do + test = self() + adapter = capture(test, ok()) + + Document.insert(db_config(adapter), %{"@type" => "X"}, + author: "a", + message: "m", + organization: "acme" + ) + + req = last_request() + assert req.url.path == "/api/document/acme/mydb" + end + + test "raises when no database is scoped in config" do + config = Config.new(endpoint: "http://localhost:6363", adapter: fn r -> {r, ok()} end) + + assert_raise Error, fn -> + Document.insert(config, %{"@type" => "X"}, author: "a", message: "m") + end + end + end + + describe "insert!/3" do + test "returns the response body on success" do + adapter = fn req -> {req, ok(%{"@type" => "api:InsertResponse"})} end + + assert %{"@type" => "api:InsertResponse"} = + Document.insert!(db_config(adapter), %{"@type" => "X"}, author: "a", message: "m") + end + + test "raises on failure" do + adapter = fn req -> {req, resp(400, %{"@type" => "api:bad"})} end + + assert_raise Error, fn -> + Document.insert!(db_config(adapter), %{"@type" => "X"}, author: "a", message: "m") + end + end + end + + describe "get/2" do + test "GETs document/:org/:db with graph_type param" do + test = self() + adapter = capture(test, ok([%{"@type" => "Person", "name" => "Alice"}])) + + assert {:ok, _} = Document.get(db_config(adapter)) + req = last_request() + assert req.method == :get + assert req.url.path == "/api/document/admin/mydb" + assert req.url.query =~ "graph_type=instance" + end + + test "passes :id, :type, :skip, :count params" do + test = self() + adapter = capture(test, ok()) + + Document.get(db_config(adapter), + id: "Person/Alice", + type: "Person", + skip: 5, + count: 10, + as_list: true, + unfold: false, + compress_ids: false + ) + + req = last_request() + q = req.url.query + assert q =~ "id=Person%2FAlice" + assert q =~ "type=Person" + assert q =~ "skip=5" + assert q =~ "count=10" + assert q =~ "as_list=true" + end + + test "honors :organization override" do + test = self() + adapter = capture(test, ok()) + + Document.get(db_config(adapter), organization: "acme") + req = last_request() + assert req.url.path == "/api/document/acme/mydb" + end + end + + describe "get!/2" do + test "returns the body on success" do + adapter = fn req -> {req, ok([%{"@id" => "Person/Alice"}])} end + assert Document.get!(db_config(adapter)) == [%{"@id" => "Person/Alice"}] + end + + test "raises on failure" do + adapter = fn req -> {req, resp(404, %{"@type" => "api:NotFound"})} end + assert_raise Error, fn -> Document.get!(db_config(adapter), id: "missing") end + end + end + + describe "query/3" do + test "GETs document/:org/:db with the template as JSON body" do + test = self() + adapter = capture(test, ok([%{"@type" => "Person", "age" => 30}])) + + assert {:ok, _} = Document.query(db_config(adapter), %{"@type" => "Person", "age" => 30}) + req = last_request() + assert req.method == :get + + assert {:ok, body} = Jason.decode(req.body) + assert body == %{"@type" => "Person", "age" => 30} + end + + test "passes :skip, :count, and :as_list params" do + test = self() + adapter = capture(test, ok([])) + + Document.query(db_config(adapter), %{"@type" => "Person"}, + skip: 5, + count: 10, + as_list: true + ) + + req = last_request() + assert req.url.query =~ "skip=5" + assert req.url.query =~ "count=10" + assert req.url.query =~ "as_list=true" + end + + test "targets the schema graph when graph_type: :schema" do + test = self() + adapter = capture(test, ok([])) + + Document.query(db_config(adapter), %{"@type" => "Class"}, graph_type: :schema) + + req = last_request() + assert req.url.query =~ "graph_type=schema" + end + + test "returns an :api error on failure" do + adapter = fn req -> {req, resp(400, %{"@type" => "api:bad"})} end + + assert {:error, %Error{reason: :api}} = + Document.query(db_config(adapter), %{"@type" => "X"}) + end + end + + describe "query!/3" do + test "returns the response body on success" do + adapter = fn req -> {req, ok([%{"@type" => "Person", "name" => "Alice"}])} end + + assert [%{"name" => "Alice"}] = + Document.query!(db_config(adapter), %{"@type" => "Person"}) + end + + test "raises on failure" do + adapter = fn req -> {req, resp(400, %{"@type" => "api:bad"})} end + + assert_raise Error, fn -> + Document.query!(db_config(adapter), %{"@type" => "X"}) + end + end + end + + describe "replace/3" do + test "PUTs document/:org/:db with the document as JSON body" do + test = self() + adapter = capture(test, ok()) + + Document.replace(db_config(adapter), %{"@id" => "Person/Alice", "name" => "Alicia"}, + author: "a", + message: "rename" + ) + + req = last_request() + assert req.method == :put + assert {:ok, body} = Jason.decode(req.body) + assert body["name"] == "Alicia" + end + + test "passes create and raw_json params" do + test = self() + adapter = capture(test, ok()) + + Document.replace(db_config(adapter), %{"@type" => "X"}, + author: "a", + message: "m", + create: true, + raw_json: true + ) + + req = last_request() + assert req.url.query =~ "create=true" + assert req.url.query =~ "raw_json=true" + end + + test "targets the schema graph when graph_type: :schema" do + test = self() + adapter = capture(test, ok()) + + Document.replace(db_config(adapter), %{"@type" => "Class"}, + author: "a", + message: "m", + graph_type: :schema + ) + + req = last_request() + assert req.url.query =~ "graph_type=schema" + end + + test "returns an :api error on failure" do + adapter = fn req -> {req, resp(404, %{"@type" => "api:NotFound"})} end + + assert {:error, %Error{reason: :api}} = + Document.replace(db_config(adapter), %{"@id" => "missing"}, + author: "a", + message: "m" + ) + end + end + + describe "replace!/3" do + test "returns the response body on success" do + adapter = fn req -> {req, ok(%{"@id" => "Person/Alice", "name" => "Alicia"})} end + + assert %{"name" => "Alicia"} = + Document.replace!( + db_config(adapter), + %{"@id" => "Person/Alice", "name" => "Alicia"}, + author: "a", + message: "rename" + ) + end + + test "raises on failure" do + adapter = fn req -> {req, resp(404, %{"@type" => "api:NotFound"})} end + + assert_raise Error, fn -> + Document.replace!(db_config(adapter), %{"@id" => "missing"}, author: "a", message: "m") + end + end + end + + describe "delete/2" do + test "DELETEs document/:org/:db with author and message" do + test = self() + adapter = capture(test, ok()) + + assert {:ok, _} = Document.delete(db_config(adapter), author: "a", message: "remove") + req = last_request() + assert req.method == :delete + assert req.url.query =~ "author=a" + assert req.url.query =~ "message=remove" + end + + test "passes :id and :nuke params" do + test = self() + adapter = capture(test, ok()) + + Document.delete(db_config(adapter), + id: "Person/Alice", + nuke: true, + author: "a", + message: "m" + ) + + req = last_request() + assert req.url.query =~ "id=Person%2FAlice" + assert req.url.query =~ "nuke=true" + end + end + + describe "delete!/2" do + test "returns the response body on success" do + adapter = fn req -> {req, ok(%{"api:status" => "api:success"})} end + + assert %{"api:status" => "api:success"} = + Document.delete!(db_config(adapter), id: "Person/Alice", author: "a", message: "m") + end + + test "raises on failure" do + adapter = fn req -> {req, resp(404, %{"@type" => "api:NotFound"})} end + + assert_raise Error, fn -> + Document.delete!(db_config(adapter), id: "missing", author: "a", message: "m") + end + end + end + + describe "stream/2" do + test "builds the correct request path and params" do + test = self() + + adapter = fn req -> + send(test, {:request, req}) + {req, ok([%{"@type" => "Person"}])} + end + + _ = Document.stream(db_config(adapter), type: "Person") + + req = last_request() + assert req.method == :get + assert req.url.path == "/api/document/admin/mydb" + assert req.url.query =~ "type=Person" + end + end +end diff --git a/test/terminus_db/schema_test.exs b/test/terminus_db/schema_test.exs new file mode 100644 index 0000000..81dcd0d --- /dev/null +++ b/test/terminus_db/schema_test.exs @@ -0,0 +1,98 @@ +defmodule TerminusDB.SchemaTest do + use ExUnit.Case, async: true + + alias TerminusDB.{Config, Error, Schema} + import TerminusDB.Test.Helpers + + defp db_config(adapter) do + Config.with_database(Config.new(endpoint: "http://localhost:6363", adapter: adapter), "mydb") + end + + describe "frame/3" do + test "GETs schema/:org/:db for all classes when class_name is nil" do + test = self() + + frame = %{ + "@context" => %{"@type" => "Context"}, + "Person" => %{"@type" => "Class", "name" => "xsd:string"} + } + + adapter = capture(test, Req.Response.new(status: 200, body: frame)) + + assert {:ok, ^frame} = Schema.frame(db_config(adapter)) + req = last_request() + assert req.method == :get + assert req.url.path == "/api/schema/admin/mydb" + end + + test "GETs schema/:org/:db/:class for a specific class" do + test = self() + + frame = %{"@type" => "Class", "name" => "xsd:string"} + + adapter = capture(test, Req.Response.new(status: 200, body: frame)) + + assert {:ok, ^frame} = Schema.frame(db_config(adapter), "Person") + req = last_request() + assert req.url.path == "/api/schema/admin/mydb/Person" + end + + test "passes compress_ids and expand_abstract params" do + test = self() + adapter = capture(test, ok(%{})) + + Schema.frame(db_config(adapter), "Person", + compress_ids: false, + expand_abstract: false + ) + + req = last_request() + assert req.url.query =~ "compress_ids=false" + assert req.url.query =~ "expand_abstract=false" + end + + test "honors :organization override" do + test = self() + adapter = capture(test, ok(%{})) + + Schema.frame(db_config(adapter), "Person", organization: "acme") + req = last_request() + assert req.url.path == "/api/schema/acme/mydb/Person" + end + + test "raises when no database is scoped" do + config = Config.new(endpoint: "http://localhost:6363", adapter: fn r -> {r, ok()} end) + assert_raise Error, fn -> Schema.frame(config, "Person") end + end + end + + describe "frame!/3" do + test "returns the frame on success" do + adapter = fn req -> {req, ok(%{"@type" => "Class"})} end + assert Schema.frame!(db_config(adapter), "Person") == %{"@type" => "Class"} + end + + test "raises on failure" do + adapter = fn req -> {req, resp(404, %{"@type" => "api:NotFound"})} end + assert_raise Error, fn -> Schema.frame!(db_config(adapter), "Missing") end + end + end + + describe "all/2" do + test "is equivalent to frame(config, nil)" do + test = self() + adapter = capture(test, ok(%{})) + + Schema.all(db_config(adapter)) + req = last_request() + assert req.url.path == "/api/schema/admin/mydb" + end + end + + describe "all!/2" do + test "returns all frames on success" do + adapter = fn req -> {req, ok(%{"Person" => %{}})} end + assert Schema.all!(db_config(adapter)) == %{"Person" => %{}} + end + end +end diff --git a/test/terminus_db/streaming_test.exs b/test/terminus_db/streaming_test.exs new file mode 100644 index 0000000..df6c30e --- /dev/null +++ b/test/terminus_db/streaming_test.exs @@ -0,0 +1,300 @@ +defmodule TerminusDB.StreamingTest do + use ExUnit.Case, async: true + + alias TerminusDB.Streaming + + describe "split_concatenated/1" do + test "splits complete concatenated JSON objects" do + buffer = ~s({"a":1}{"b":2}{"c":3}) + {docs, rest} = Streaming.split_concatenated(buffer) + assert docs == [~s({"a":1}), ~s({"b":2}), ~s({"c":3})] + assert rest == "" + end + + test "leaves incomplete trailing object in the buffer" do + buffer = ~s({"a":1}{"b":2) + {docs, rest} = Streaming.split_concatenated(buffer) + assert docs == [~s({"a":1})] + assert rest == ~s({"b":2) + end + + test "handles nested braces" do + buffer = ~s({"a":{"x":1}}{"b":2}) + {docs, rest} = Streaming.split_concatenated(buffer) + assert docs == [~s({"a":{"x":1}}), ~s({"b":2})] + assert rest == "" + end + + test "handles braces inside string literals" do + buffer = ~s({"a":"}hello{"}{"b":2}) + {docs, rest} = Streaming.split_concatenated(buffer) + assert docs == [~s({"a":"}hello{"}), ~s({"b":2})] + assert rest == "" + end + + test "handles escaped quotes inside strings" do + # JSON: {"a":"he said \"hi\""}{"b":2}{"c":3} + buffer = ~s({"a":"he said \\"hi\\""}{"b":2}{"c":3}) + {docs, rest} = Streaming.split_concatenated(buffer) + assert docs == [~s({"a":"he said \\"hi\\""}), ~s({"b":2}), ~s({"c":3})] + assert rest == "" + end + + test "returns empty list and buffer for empty input" do + {docs, rest} = Streaming.split_concatenated("") + assert docs == [] + assert rest == "" + end + + test "handles incomplete first object" do + buffer = ~s({"a":1) + {docs, rest} = Streaming.split_concatenated(buffer) + assert docs == [] + assert rest == ~s({"a":1) + end + + test "handles arrays as values" do + buffer = ~s({"a":[1,2,3]}{"b":[]}) + {docs, rest} = Streaming.split_concatenated(buffer) + assert docs == [~s({"a":[1,2,3]}), ~s({"b":[]})] + assert rest == "" + end + + test "decodes the split objects as maps" do + buffer = ~s({"@type":"Person","name":"Alice"}{"@type":"Person","name":"Bob"}) + {docs, _rest} = Streaming.split_concatenated(buffer) + + maps = Enum.map(docs, &Jason.decode!/1) + + assert maps == [ + %{"@type" => "Person", "name" => "Alice"}, + %{"@type" => "Person", "name" => "Bob"} + ] + end + end + + describe "document_stream/1" do + # Builds a Req.Response with a fake Req.Response.Async body whose + # stream_fun decodes messages sent to the current process. + # + # The `messages` argument is a list of terms to send to the process mailbox. + # Each message is received by the `receive` block in `next_document`, then + # passed to `stream_fun`. The `stream_fun` returns `{:ok, chunks}` where + # chunks is a keyword list like `[data: "..."]` or `[:done]`. + # + # We send a sequence of `{:data, binary}` chunks followed by `:done`, which + # the stream_fun translates into the `{:ok, chunks}` / `:done` format Req uses. + defp fake_async_response(chunks) do + ref = make_ref() + + stream_fun = fn ^ref, message -> + case message do + {^ref, :data} -> {:ok, chunks} + {^ref, :done} -> {:ok, [:done]} + {^ref, {:error, reason}} -> {:error, reason} + _ -> :unknown + end + end + + cancel_fun = fn ^ref -> :ok end + + async = %Req.Response.Async{ + pid: self(), + ref: ref, + stream_fun: stream_fun, + cancel_fun: cancel_fun + } + + Req.Response.new(status: 200, body: async) + end + + # Sends a sequence of data chunks followed by a done signal. + # Each `data` binary is sent as `{ref, :data}` and the stream_fun returns + # `{:ok, [data: data]}`. A final `{ref, :done}` signals end of stream. + defp send_chunks(ref, data_chunks) do + for data <- data_chunks do + send(self(), {ref, {:data, data}}) + end + + send(self(), {ref, :done}) + end + + # A stream_fun that reads the actual data from the message tuple. + defp fake_async_response_with_data(data_chunks) do + ref = make_ref() + + stream_fun = fn ^ref, message -> + case message do + {^ref, {:data, data}} -> {:ok, [data: data]} + {^ref, :done} -> {:ok, [:done]} + {^ref, {:error, reason}} -> {:error, reason} + _ -> :unknown + end + end + + cancel_fun = fn ^ref -> :ok end + + async = %Req.Response.Async{ + pid: self(), + ref: ref, + stream_fun: stream_fun, + cancel_fun: cancel_fun + } + + send_chunks(ref, data_chunks) + Req.Response.new(status: 200, body: async) + end + + test "streams complete documents from a single chunk" do + resp = + fake_async_response_with_data([ + ~s({"@type":"Person","name":"Alice"}{"@type":"Person","name":"Bob"}) + ]) + + docs = Enum.to_list(Streaming.document_stream(resp)) + + assert Enum.map(docs, & &1["name"]) == ["Alice", "Bob"] + end + + test "streams documents split across multiple chunks" do + resp = + fake_async_response_with_data([ + ~s({"@type":"Person","name":"Alice"}), + ~s({"@type":"Person","name":"Bob"}) + ]) + + docs = Enum.to_list(Streaming.document_stream(resp)) + + assert Enum.map(docs, & &1["name"]) == ["Alice", "Bob"] + end + + test "streams documents when a single document is split across chunks" do + # A single document split mid-way through: the first chunk ends after + # the "name" value, the second chunk completes the object. + resp = + fake_async_response_with_data([ + ~s({"@type":"Person","name":"Alice",), + ~s("age":30}) + ]) + + docs = Enum.to_list(Streaming.document_stream(resp)) + + assert length(docs) == 1 + assert hd(docs)["name"] == "Alice" + assert hd(docs)["age"] == 30 + end + + test "returns empty list when no documents are received" do + resp = fake_async_response_with_data([]) + + docs = Enum.to_list(Streaming.document_stream(resp)) + + assert docs == [] + end + + test "streams documents with nested objects" do + resp = + fake_async_response_with_data([ + ~s({"@type":"Person","name":"Alice","address":{"city":"NYC"}}) <> + ~s({"@type":"Person","name":"Bob","address":{"city":"LA"}}) + ]) + + docs = Enum.to_list(Streaming.document_stream(resp)) + + assert Enum.map(docs, & &1["name"]) == ["Alice", "Bob"] + assert hd(docs)["address"]["city"] == "NYC" + end + + test "halts on error from the stream" do + ref = make_ref() + + stream_fun = fn ^ref, message -> + case message do + {^ref, {:data, data}} -> {:ok, [data: data]} + {^ref, {:error, reason}} -> {:error, reason} + _ -> :unknown + end + end + + async = %Req.Response.Async{ + pid: self(), + ref: ref, + stream_fun: stream_fun, + cancel_fun: fn ^ref -> :ok end + } + + send(self(), {ref, {:data, ~s({"name":"Alice"})}}) + send(self(), {ref, {:error, %RuntimeError{message: "connection reset"}}}) + + resp = Req.Response.new(status: 200, body: async) + + docs = Enum.to_list(Streaming.document_stream(resp)) + + # Should have received the one document before the error + assert length(docs) == 1 + assert hd(docs)["name"] == "Alice" + end + + test "processes trailers and other non-data chunks by ignoring them" do + ref = make_ref() + + stream_fun = fn ^ref, message -> + case message do + {^ref, {:data, data}} -> {:ok, [data: data]} + {^ref, :trailers} -> {:ok, [trailers: %{"x-custom" => "value"}]} + {^ref, :done} -> {:ok, [:done]} + _ -> :unknown + end + end + + async = %Req.Response.Async{ + pid: self(), + ref: ref, + stream_fun: stream_fun, + cancel_fun: fn ^ref -> :ok end + } + + send(self(), {ref, {:data, ~s({"name":"Alice"})}}) + send(self(), {ref, :trailers}) + send(self(), {ref, :done}) + + resp = Req.Response.new(status: 200, body: async) + + docs = Enum.to_list(Streaming.document_stream(resp)) + + assert length(docs) == 1 + assert hd(docs)["name"] == "Alice" + end + + test "skips unknown messages and continues to real data" do + ref = make_ref() + + stream_fun = fn ^ref, message -> + case message do + {^ref, {:data, data}} -> {:ok, [data: data]} + {^ref, :done} -> {:ok, [:done]} + _ -> :unknown + end + end + + async = %Req.Response.Async{ + pid: self(), + ref: ref, + stream_fun: stream_fun, + cancel_fun: fn ^ref -> :ok end + } + + # Send an unrelated message first, then the real data + send(self(), {:some_other_message, :ignore_me}) + send(self(), {ref, {:data, ~s({"name":"Alice"})}}) + send(self(), {ref, :done}) + + resp = Req.Response.new(status: 200, body: async) + + docs = Enum.to_list(Streaming.document_stream(resp)) + + assert length(docs) == 1 + assert hd(docs)["name"] == "Alice" + end + end +end From 02beff45ac1b0684450e9646dfc90eb109109990 Mon Sep 17 00:00:00 2001 From: thanos Date: Wed, 24 Jun 2026 10:46:46 -0400 Subject: [PATCH 2/3] =?UTF-8?q?Summary=20of=20all=20fixes=20High=20-=20F1?= =?UTF-8?q?=20=E2=80=94=20Flaky=20telemetry=20test=20fixed:=20uses=20a=20u?= =?UTF-8?q?nique=20path=20("telemetry/noop-")=20and=20filters=20re?= =?UTF-8?q?fute=5Freceive=20by=20%{path:=20^unique}=20so=20it=20can't=20ca?= =?UTF-8?q?tch=20leaked=20events=20from=20concurrent=20tests.=20Verified:?= =?UTF-8?q?=200=20failures=20in=2050=20runs=20with=20different=20seeds.=20?= =?UTF-8?q?-=20F2=20=E2=80=94=20unfold/minimized/compress=5Fids/as=5Flist?= =?UTF-8?q?=20now=20use=20Params.bool=5Fparam/2=20(sends=20false=20explici?= =?UTF-8?q?tly)=20instead=20of=20maybe=5Fparam=20(which=20dropped=20false)?= =?UTF-8?q?.=20Added=20a=20regression=20test=20asserting=20unfold=3Dfalse,?= =?UTF-8?q?=20minimized=3Dfalse,=20compress=5Fids=3Dfalse,=20as=5Flist=3Df?= =?UTF-8?q?alse=20all=20appear=20in=20the=20query=20string.=20Medium=20-?= =?UTF-8?q?=20F3=20=E2=80=94=20Document.stream/2=20now=20uses=20a=20case?= =?UTF-8?q?=20to=20handle=20{:error,=20error}=20and=20raise=20error=20(rai?= =?UTF-8?q?sing=20TerminusDB.Error),=20instead=20of=20{:ok,=20resp}=20=3D?= =?UTF-8?q?=20...=20which=20raised=20MatchError.=20Added=20two=20tests=20(?= =?UTF-8?q?404=20error=20+=20transport=20error).=20-=20F4=20=E2=80=94=20Re?= =?UTF-8?q?moved=20the=20false=20"JSON=20array"=20claim=20from=20the=20Str?= =?UTF-8?q?eaming=20moduledoc=20and=20document=5Fstream/1=20doc.=20Now=20c?= =?UTF-8?q?orrectly=20documents=20that=20only=20concatenated=20JSON=20is?= =?UTF-8?q?=20supported=20and=20warns=20not=20to=20use=20as=5Flist=20with?= =?UTF-8?q?=20streaming.=20-=20F5=20=E2=80=94=20Branch.create/3=20now=20bu?= =?UTF-8?q?ilds=20the=20origin=20body=20from=20the=20resolved=20org/repo?= =?UTF-8?q?=20(honoring=20:organization/:repo=20overrides)=20instead=20of?= =?UTF-8?q?=20config.*=20defaults.=20Added=20a=20test=20asserting=20the=20?= =?UTF-8?q?origin=20body=20reflects=20the=20overrides.=20-=20F6=20?= =?UTF-8?q?=E2=80=94=20Replaced=20all=2015=20occurrences=20of=20:ok=20=3D?= =?UTF-8?q?=20...delete(...)=20with=20{:ok,=20=5F}=20=3D=20...delete(...)?= =?UTF-8?q?=20across=20README,=20all=203=20guides,=20Livebook,=20Database/?= =?UTF-8?q?Branch=20moduledocs,=20and=20the=20integration=20test.=20-=20F7?= =?UTF-8?q?/F8=20=E2=80=94=20Added=20integration=20tests=20for=20query/3?= =?UTF-8?q?=20(inserts=20two=20docs=20with=20different=20ages,=20queries?= =?UTF-8?q?=20by=20template,=20asserts=20only=20matching=20docs=20returned?= =?UTF-8?q?),=20stream/2=20(inserts=203=20docs,=20streams=20them,=20assert?= =?UTF-8?q?s=20all=20labels),=20and=20Schema.frame/3=20(retrieves=20a=20sp?= =?UTF-8?q?ecific=20class=20frame=20after=20inserting=20a=20schema).=20Low?= =?UTF-8?q?=20-=20F9=20=E2=80=94=20Extracted=20TerminusDB.Client.Params=20?= =?UTF-8?q?module=20with=20flag=5Fparam/2=20(omit=20false)=20and=20bool=5F?= =?UTF-8?q?param/2=20(always=20send=20non-nil).=20Replaced=20the=203=20dup?= =?UTF-8?q?licated=20maybe=5Fparam=20definitions=20in=20Document,=20Schema?= =?UTF-8?q?,=20and=20Database.=20-=20F10=20=E2=80=94=20Added=20a=20receive?= =?UTF-8?q?=20...=20after=20timeout=20to=20next=5Fdocument=20so=20the=20st?= =?UTF-8?q?ream=20halts=20if=20no=20chunk=20arrives=20within=20the=20timeo?= =?UTF-8?q?ut=20window=20(defaults=20to=2015s,=20threaded=20from=20config.?= =?UTF-8?q?receive=5Ftimeout).=20-=20F11=20=E2=80=94=20Added=20a=20clause?= =?UTF-8?q?=20for=20a=20lone=20trailing=20\=20inside=20a=20string=20at=20e?= =?UTF-8?q?nd-of-buffer:=20it's=20retained=20in=20the=20buffer=20so=20the?= =?UTF-8?q?=20next=20chunk=20can=20complete=20the=20escape=20sequence.=20-?= =?UTF-8?q?=20F12=20=E2=80=94=20Bumped=20@version=20to=20"0.2.0-dev"=20so?= =?UTF-8?q?=20source=5Fref=20and=20user-agent=20reflect=20the=20current=20?= =?UTF-8?q?development=20version.=20Updated=20version-dependent=20tests=20?= =?UTF-8?q?to=20use=20Mix.Project.config()[:version].=20-=20F13=20?= =?UTF-8?q?=E2=80=94=20Fixed=20the=20Branch.create=20doctest:=20{:ok,=20re?= =?UTF-8?q?sp}=20=3D=20...=20then=20resp["api:status"]=20instead=20of=20{:?= =?UTF-8?q?ok,=20=5F}=20=3D=20...=20expecting=20:ok.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 6 +- guides/migrating-from-sql.md | 2 +- guides/overview.md | 8 +-- guides/terminusdb_ex_livebook.livemd | 6 +- lib/terminus_db/branch.ex | 17 ++--- lib/terminus_db/client.ex | 22 +++++-- lib/terminus_db/client/params.ex | 50 ++++++++++++++ lib/terminus_db/database.ex | 23 +++---- lib/terminus_db/document.ex | 65 +++++++++---------- lib/terminus_db/schema.ex | 13 ++-- lib/terminus_db/streaming.ex | 61 ++++++++++++----- mix.exs | 2 +- .../document_schema_branch_test.exs | 58 ++++++++++++++++- test/terminus_db/branch_test.exs | 5 +- test/terminus_db/client_test.exs | 21 +++++- test/terminus_db/config_test.exs | 2 +- test/terminus_db/database_test.exs | 2 +- test/terminus_db/document_test.exs | 37 +++++++++++ test/terminus_db/streaming_test.exs | 34 ---------- test/terminus_db/telemetry_test.exs | 9 ++- 20 files changed, 301 insertions(+), 142 deletions(-) create mode 100644 lib/terminus_db/client/params.ex diff --git a/README.md b/README.md index 92d258b..e652001 100644 --- a/README.md +++ b/README.md @@ -84,9 +84,9 @@ TerminusDB.Document.stream(config, type: "Person") |> Stream.run() # 11. Clean up. -:ok = TerminusDB.Document.delete(config, id: "Person/Alice", author: "admin", message: "remove") -:ok = TerminusDB.Branch.delete(config, "feature") -:ok = TerminusDB.Database.delete(config, "mydb") +{:ok, _} = TerminusDB.Document.delete(config, id: "Person/Alice", author: "admin", message: "remove") +{:ok, _} = TerminusDB.Branch.delete(config, "feature") +{:ok, _} = TerminusDB.Database.delete(config, "mydb") ``` All public functions return `{:ok, result}` or `{:error, %TerminusDB.Error{}}`. Each diff --git a/guides/migrating-from-sql.md b/guides/migrating-from-sql.md index 77235d6..c87a0b7 100644 --- a/guides/migrating-from-sql.md +++ b/guides/migrating-from-sql.md @@ -309,7 +309,7 @@ DELETE FROM comments WHERE id = 5; ) # Delete a document -:ok = TerminusDB.Document.delete(config, id: "Comment/xyz789", author: "admin", message: "Remove comment") +{:ok, _} = TerminusDB.Document.delete(config, id: "Comment/xyz789", author: "admin", message: "Remove comment") ``` --- diff --git a/guides/overview.md b/guides/overview.md index 7148721..0cf6617 100644 --- a/guides/overview.md +++ b/guides/overview.md @@ -63,7 +63,7 @@ Database.exists?(config, "nope") # => false {:ok, _} = Database.update(config, "demo", label: "Renamed DB") # Delete -:ok = Database.delete(config, "demo") +{:ok, _} = Database.delete(config, "demo") ``` ## 3. Document CRUD @@ -113,7 +113,7 @@ config = Config.with_database(config, "demo") ) # Delete a document -:ok = Document.delete(config, id: "Person/Alice", author: "admin", message: "remove") +{:ok, _} = Document.delete(config, id: "Person/Alice", author: "admin", message: "remove") ``` ## 4. Schema frames @@ -155,7 +155,7 @@ feature_config = Config.with_branch(config, "feature") ) # Delete the branch -:ok = Branch.delete(config, "feature") +{:ok, _} = Branch.delete(config, "feature") ``` ## 6. Streaming @@ -239,5 +239,5 @@ resp.headers["content-type"] # => ["application/json"] ## Cleanup ```elixir -:ok = Database.delete(config, "demo", force: true) +{:ok, _} = Database.delete(config, "demo", force: true) ``` diff --git a/guides/terminusdb_ex_livebook.livemd b/guides/terminusdb_ex_livebook.livemd index 4233c7f..0bc7a41 100644 --- a/guides/terminusdb_ex_livebook.livemd +++ b/guides/terminusdb_ex_livebook.livemd @@ -179,7 +179,7 @@ Enum.map(matches, & &1["name"]) ### Delete a document ```elixir -:ok = Document.delete(config, +{:ok, _} = Document.delete(config, id: "Person/Bob", author: "admin", message: "Remove Bob" @@ -256,7 +256,7 @@ feature_config = Config.with_branch(config, "feature-x") ```elixir # Delete the branch -:ok = Branch.delete(config, "feature-x") +{:ok, _} = Branch.delete(config, "feature-x") ``` ## 6. Streaming @@ -347,7 +347,7 @@ info["api:info"]["terminusdb"]["version"] ## Cleanup ```elixir -:ok = Database.delete(config, "livebook_demo", force: true) +{:ok, _} = Database.delete(config, "livebook_demo", force: true) ``` ```elixir diff --git a/lib/terminus_db/branch.ex b/lib/terminus_db/branch.ex index 324dbae..f37fcb3 100644 --- a/lib/terminus_db/branch.ex +++ b/lib/terminus_db/branch.ex @@ -24,7 +24,7 @@ defmodule TerminusDB.Branch do true = TerminusDB.Branch.exists?(config, "feature") # Delete it - :ok = TerminusDB.Branch.delete(config, "feature") + {:ok, _} = TerminusDB.Branch.delete(config, "feature") """ @@ -36,10 +36,8 @@ defmodule TerminusDB.Branch do | {:from, String.t()} defp branch_path(config, branch_name, opts) do - org = opts[:organization] || config.organization - db = config.database || raise Error, reason: :http, message: "no database scoped in config" repo = opts[:repo] || config.repo - "branch/#{org}/#{db}/#{repo}/branch/#{branch_name}" + "branch/#{Client.resource_path(config, opts)}/#{repo}/branch/#{branch_name}" end @doc """ @@ -70,18 +68,21 @@ defmodule TerminusDB.Branch do ...> endpoint: "http://localhost:6363", ...> adapter: fn req -> {req, Req.Response.new(status: 200, body: %{"api:status" => "api:success"})} end ...> ) |> TerminusDB.Config.with_database("mydb") - iex> {:ok, _} = TerminusDB.Branch.create(config, "dev", from: "main") - :ok + iex> {:ok, resp} = TerminusDB.Branch.create(config, "dev", from: "main") + iex> resp["api:status"] + "api:success" """ @spec create(Config.t(), String.t(), [branch_opt()]) :: {:ok, map()} | {:error, Error.t()} def create(config, branch_name, opts \\ []) do path = branch_path(config, branch_name, opts) - origin = opts[:from] || config.branch + org = opts[:organization] || config.organization + repo = opts[:repo] || config.repo + origin_branch = opts[:from] || config.branch body = %{ - "origin" => "#{config.organization}/#{config.database}/#{config.repo}/branch/#{origin}" + "origin" => "#{org}/#{config.database}/#{repo}/branch/#{origin_branch}" } Client.request(config, :post, path, json: body, area: :branch) diff --git a/lib/terminus_db/client.ex b/lib/terminus_db/client.ex index 567daf6..bd83ad8 100644 --- a/lib/terminus_db/client.ex +++ b/lib/terminus_db/client.ex @@ -140,14 +140,28 @@ defmodule TerminusDB.Client do end @doc """ - Convenience that builds the common `organization/database` resource segment. + Builds the `organization/database` resource segment for the given config and + options, resolving the organization from `opts[:organization]` or + `config.organization`. Raises `TerminusDB.Error` if no database is scoped. - iex> TerminusDB.Client.resource_path("admin", "mydb") + iex> config = TerminusDB.Config.new(endpoint: "http://localhost:6363") + ...> |> TerminusDB.Config.with_database("mydb") + iex> TerminusDB.Client.resource_path(config, []) "admin/mydb" + iex> TerminusDB.Client.resource_path(config, organization: "acme") + "acme/mydb" """ - @spec resource_path(String.t(), String.t()) :: String.t() - def resource_path(org, db) when is_binary(org) and is_binary(db), do: "#{org}/#{db}" + @spec resource_path(Config.t(), keyword()) :: String.t() + def resource_path(%Config{} = config, opts) do + org = opts[:organization] || config.organization + + db = + config.database || + raise Error, reason: :http, message: "no database scoped in config" + + "#{org}/#{db}" + end # Request construction ------------------------------------------------------ diff --git a/lib/terminus_db/client/params.ex b/lib/terminus_db/client/params.ex new file mode 100644 index 0000000..5a39b6e --- /dev/null +++ b/lib/terminus_db/client/params.ex @@ -0,0 +1,50 @@ +defmodule TerminusDB.Client.Params do + @moduledoc false + + # Internal helpers for building Req query-parameter keyword lists. + # + # There are two kinds of boolean-style query parameters in the TerminusDB API: + # + # * Flag parameters -- their absence on the wire means `false`. Use + # `flag_param/2`, which omits `false` and `nil` so we don't clutter the + # query string with `=false` for defaults. Examples: `full_replace`, + # `raw_json`, `nuke`, `create`, `force`. + # + # * Tri-state parameters -- the server defaults them to `true`, and `false` + # is a meaningful override that MUST be sent. Use `bool_param/2`, which + # sends any non-nil value (including `false`). Examples: `unfold`, + # `minimized`, `compress_ids`, `as_list`, `branches`, `verbose`, + # `expand_abstract`. + + @doc """ + Returns a single-element keyword list for a flag parameter, or `[]` when the + value is falsy (`nil` or `false`). Use for parameters whose absence means + `false` server-side. + """ + @spec flag_param(atom(), term()) :: keyword() + def flag_param(_name, nil), do: [] + def flag_param(_name, false), do: [] + def flag_param(name, true), do: [{name, true}] + def flag_param(name, value), do: [{name, value}] + + @doc """ + Returns a single-element keyword list for a tri-state boolean parameter, or + `[]` only when the value is `nil`. Use for parameters the server defaults to + `true`, where an explicit `false` must be sent to override the default. + """ + @spec bool_param(atom(), term()) :: keyword() + def bool_param(_name, nil), do: [] + def bool_param(name, value), do: [{name, value}] + + @doc """ + Concatenates a list of `{name, value}` pairs (or `[]`s) into a single keyword + list, dropping nil/false entries according to `flag_param/2` semantics. + """ + @spec flags([{atom(), term()} | []]) :: keyword() + def flags(pairs) do + Enum.flat_map(pairs, fn + [] -> [] + {name, value} -> flag_param(name, value) + end) + end +end diff --git a/lib/terminus_db/database.ex b/lib/terminus_db/database.ex index 7a5412e..59b62c7 100644 --- a/lib/terminus_db/database.ex +++ b/lib/terminus_db/database.ex @@ -22,11 +22,12 @@ defmodule TerminusDB.Database do {:ok, details} = TerminusDB.Database.info(config, "mydb") true = TerminusDB.Database.exists?(config, "mydb") - :ok = TerminusDB.Database.delete(config, "mydb") + {:ok, _} = TerminusDB.Database.delete(config, "mydb") """ alias TerminusDB.{Client, Error} + alias TerminusDB.Client.Params @type create_opt :: {:label, String.t()} @@ -108,8 +109,8 @@ defmodule TerminusDB.Database do ## Examples - iex> :ok = TerminusDB.Database.delete(config, "mydb") - iex> :ok = TerminusDB.Database.delete(config, "mydb", force: true) + iex> {:ok, _} = TerminusDB.Database.delete(config, "mydb") + iex> {:ok, _} = TerminusDB.Database.delete(config, "mydb", force: true) """ @spec delete(TerminusDB.Config.t(), String.t(), [delete_opt()]) :: @@ -122,7 +123,6 @@ defmodule TerminusDB.Database do end @doc """ - @doc \""" Deletes a database, returning the response body or raising `TerminusDB.Error`. ## Examples @@ -169,9 +169,8 @@ defmodule TerminusDB.Database do org = opts[:organization] || config.organization params = - [] - |> maybe_param(:branches, opts[:branches]) - |> maybe_param(:verbose, opts[:verbose]) + Params.flag_param(:branches, opts[:branches]) ++ + Params.flag_param(:verbose, opts[:verbose]) Client.request(config, :get, "db/#{org}/#{db_name}", params: params, area: :database) end @@ -219,9 +218,8 @@ defmodule TerminusDB.Database do @spec list(TerminusDB.Config.t(), [info_opt()]) :: {:ok, [map()]} | {:error, Error.t()} def list(config, opts \\ []) do params = - [] - |> maybe_param(:branches, opts[:branches]) - |> maybe_param(:verbose, opts[:verbose]) + Params.flag_param(:branches, opts[:branches]) ++ + Params.flag_param(:verbose, opts[:verbose]) Client.request(config, :get, "db", params: params, area: :database) end @@ -344,9 +342,4 @@ defmodule TerminusDB.Database do defp maybe_put(map, _key, nil), do: map defp maybe_put(map, key, value), do: Map.put(map, key, value) - - defp maybe_param(params, _name, nil), do: params - defp maybe_param(params, _name, false), do: params - defp maybe_param(params, name, true), do: Keyword.put(params, name, true) - defp maybe_param(params, name, value), do: Keyword.put(params, name, value) end diff --git a/lib/terminus_db/document.ex b/lib/terminus_db/document.ex index 793aff5..62799ef 100644 --- a/lib/terminus_db/document.ex +++ b/lib/terminus_db/document.ex @@ -52,6 +52,7 @@ defmodule TerminusDB.Document do """ alias TerminusDB.{Client, Config, Error} + alias TerminusDB.Client.Params @type graph_type :: :instance | :schema @@ -101,9 +102,7 @@ defmodule TerminusDB.Document do # Path building ------------------------------------------------------------- defp document_path(config, opts) do - org = opts[:organization] || config.organization - db = config.database || raise Error, reason: :http, message: "no database scoped in config" - "document/#{org}/#{db}" + "document/#{Client.resource_path(config, opts)}" end defp graph_type_param(opts), do: [graph_type: Atom.to_string(opts[:graph_type] || :instance)] @@ -162,8 +161,8 @@ defmodule TerminusDB.Document do params = graph_type_param(opts) ++ commit_params(opts) ++ - maybe_param(:full_replace, opts[:full_replace]) ++ - maybe_param(:raw_json, opts[:raw_json]) + Params.flag_param(:full_replace, opts[:full_replace]) ++ + Params.flag_param(:raw_json, opts[:raw_json]) Client.request(config, :post, path, json: document, params: params, area: :document) end @@ -297,9 +296,9 @@ defmodule TerminusDB.Document do params = graph_type_param(opts) ++ - maybe_param(:skip, opts[:skip]) ++ - maybe_param(:count, opts[:count]) ++ - maybe_param(:as_list, opts[:as_list]) + Params.flag_param(:skip, opts[:skip]) ++ + Params.flag_param(:count, opts[:count]) ++ + Params.bool_param(:as_list, opts[:as_list]) Client.request(config, :get, path, json: template, params: params, area: :document) end @@ -361,8 +360,8 @@ defmodule TerminusDB.Document do params = graph_type_param(opts) ++ commit_params(opts) ++ - maybe_param(:create, opts[:create]) ++ - maybe_param(:raw_json, opts[:raw_json]) + Params.flag_param(:create, opts[:create]) ++ + Params.flag_param(:raw_json, opts[:raw_json]) Client.request(config, :put, path, json: document, params: params, area: :document) end @@ -425,8 +424,8 @@ defmodule TerminusDB.Document do params = graph_type_param(opts) ++ commit_params(opts) ++ - maybe_param(:nuke, opts[:nuke]) ++ - maybe_param(:id, opts[:id]) + Params.flag_param(:nuke, opts[:nuke]) ++ + Params.flag_param(:id, opts[:id]) Client.request(config, :delete, path, params: params, area: :document) end @@ -480,35 +479,35 @@ defmodule TerminusDB.Document do graph_type_param(opts) ++ get_params(opts) - {:ok, resp} = - Client.request_response(config, :get, path, - params: params, - into: :self, - area: :document - ) + case Client.request_response(config, :get, path, + params: params, + into: :self, + area: :document + ) do + {:ok, resp} -> + TerminusDB.Streaming.document_stream(resp, timeout: config.receive_timeout) - TerminusDB.Streaming.document_stream(resp) + {:error, error} -> + raise error + end end # Helpers ------------------------------------------------------------------- defp commit_params(opts) do - maybe_param(:author, opts[:author]) ++ maybe_param(:message, opts[:message]) + Params.flag_param(:author, opts[:author]) ++ Params.flag_param(:message, opts[:message]) end defp get_params(opts) do - maybe_param(:id, opts[:id]) ++ - maybe_param(:type, opts[:type]) ++ - maybe_param(:skip, opts[:skip]) ++ - maybe_param(:count, opts[:count]) ++ - maybe_param(:as_list, opts[:as_list]) ++ - maybe_param(:unfold, opts[:unfold]) ++ - maybe_param(:minimized, opts[:minimized]) ++ - maybe_param(:compress_ids, opts[:compress_ids]) + # as_list/unfold/minimized/compress_ids are tri-state: the server defaults + # them to true, so an explicit false must be sent to override. + Params.flag_param(:id, opts[:id]) ++ + Params.flag_param(:type, opts[:type]) ++ + Params.flag_param(:skip, opts[:skip]) ++ + Params.flag_param(:count, opts[:count]) ++ + Params.bool_param(:as_list, opts[:as_list]) ++ + Params.bool_param(:unfold, opts[:unfold]) ++ + Params.bool_param(:minimized, opts[:minimized]) ++ + Params.bool_param(:compress_ids, opts[:compress_ids]) end - - defp maybe_param(_name, nil), do: [] - defp maybe_param(_name, false), do: [] - defp maybe_param(name, true), do: [{name, true}] - defp maybe_param(name, value), do: [{name, value}] end diff --git a/lib/terminus_db/schema.ex b/lib/terminus_db/schema.ex index ba4e118..425d725 100644 --- a/lib/terminus_db/schema.ex +++ b/lib/terminus_db/schema.ex @@ -26,6 +26,7 @@ defmodule TerminusDB.Schema do """ alias TerminusDB.{Client, Config, Error} + alias TerminusDB.Client.Params @type frame_opt :: {:compress_ids, boolean()} @@ -33,9 +34,7 @@ defmodule TerminusDB.Schema do | {:organization, String.t()} defp schema_path(config, opts) do - org = opts[:organization] || config.organization - db = config.database || raise Error, reason: :http, message: "no database scoped in config" - "schema/#{org}/#{db}" + "schema/#{Client.resource_path(config, opts)}" end @doc """ @@ -82,8 +81,8 @@ defmodule TerminusDB.Schema do path = schema_path(config, opts) params = - maybe_bool_param(:compress_ids, opts[:compress_ids]) ++ - maybe_bool_param(:expand_abstract, opts[:expand_abstract]) + Params.bool_param(:compress_ids, opts[:compress_ids]) ++ + Params.bool_param(:expand_abstract, opts[:expand_abstract]) path = if class_name do @@ -154,7 +153,5 @@ defmodule TerminusDB.Schema do def all!(config, opts \\ []), do: frame!(config, nil, opts) # For boolean params where `false` is a meaningful value (not a default to omit), - # we pass it through explicitly. - defp maybe_bool_param(_name, nil), do: [] - defp maybe_bool_param(name, value), do: [{name, value}] + # we pass it through explicitly via Params.bool_param/2 (see TerminusDB.Client.Params). end diff --git a/lib/terminus_db/streaming.ex b/lib/terminus_db/streaming.ex index ce558bb..ed36c42 100644 --- a/lib/terminus_db/streaming.ex +++ b/lib/terminus_db/streaming.ex @@ -2,18 +2,28 @@ defmodule TerminusDB.Streaming do @moduledoc """ Incremental decoding of TerminusDB concatenated-JSON response bodies. - TerminusDB's document endpoint returns documents either as a JSON array or as - *concatenated JSON* (multiple JSON objects back-to-back, not newline-delimited). - This module provides a bracket/depth-aware splitter that incrementally parses - chunks emitted by Req's streaming (`into:`) into a stream of decoded maps. - - Used by `TerminusDB.Document.stream/2` (ADR-0007). + TerminusDB's document endpoint returns documents as *concatenated JSON* + (multiple JSON objects back-to-back, not newline-delimited and not a JSON + array) when the `as_list` query parameter is not set. This module provides a + bracket/depth-aware splitter that incrementally parses chunks emitted by Req's + streaming (`into:`) into a stream of decoded maps. + + Used by `TerminusDB.Document.stream/2` (ADR-0007). Note: `Document.stream/2` + does not pass `as_list: true`, so the server returns concatenated JSON; do not + enable `as_list` on a streaming call as this splitter does not handle JSON + array bodies. """ @doc """ Returns a stream of decoded JSON maps from a Req response that delivers chunks - to `into: :self` (or a collectable). Accepts either a JSON array body or - concatenated JSON objects. + to `into: :self`. The response body must be concatenated JSON objects (the + TerminusDB default when `as_list` is not set); JSON array bodies are not + supported. + + ## Options + + - `:timeout` — receive timeout in milliseconds between chunks. If no message + arrives within this window, the stream halts (defaults to `15_000`). ## Examples @@ -22,10 +32,12 @@ defmodule TerminusDB.Streaming do TerminusDB.Streaming.document_stream(resp) |> Enum.take(10) """ - @spec document_stream(Req.Response.t()) :: Enumerable.t() - def document_stream(%Req.Response{} = resp) do + @spec document_stream(Req.Response.t(), keyword()) :: Enumerable.t() + def document_stream(%Req.Response{} = resp, opts \\ []) do + timeout = opts[:timeout] || 15_000 + Stream.resource( - fn -> {resp, <<>>, false} end, + fn -> {resp, <<>>, false, timeout} end, &next_document/1, fn _ -> :ok end ) @@ -35,6 +47,10 @@ defmodule TerminusDB.Streaming do Splits a binary accumulator of concatenated JSON into a list of complete JSON object binaries and the remaining buffer. Objects are split on top-level `}` that closes the initial `{`, respecting string literals and nested braces. + + If the buffer ends inside a string and the last byte is a lone `\\` (an + incomplete escape sequence), the backslash is retained in the returned buffer + so the caller can prepend the next chunk and complete the escape. """ @spec split_concatenated(binary()) :: {[binary()], binary()} def split_concatenated(buffer) when is_binary(buffer) do @@ -78,6 +94,11 @@ defmodule TerminusDB.Streaming do do_split(rest, depth, true, acc_docs, <>) end + # Lone trailing backslash inside a string at end of buffer: keep it in the + # buffer so the next chunk can complete the escape sequence. + defp do_split(<<"\\">>, _depth, true, acc_docs, current), + do: {Enum.reverse(acc_docs), current <> "\\"} + # String handling: toggle in_string on unescaped quotes. defp do_split(<<"\"", rest::binary>>, depth, in_string, acc_docs, current) do do_split(rest, depth, not in_string, acc_docs, current <> "\"") @@ -89,28 +110,34 @@ defmodule TerminusDB.Streaming do end # Stream iteration: pull messages from the async response, accumulate, split. - defp next_document({resp, buffer, true}) do - {:halt, {resp, buffer, true}} + defp next_document({resp, buffer, true, _timeout}) do + {:halt, {resp, buffer, true, nil}} end - defp next_document({resp, buffer, false}) do + defp next_document({resp, buffer, false, timeout}) do case Req.parse_message( resp, receive do message -> message + after + timeout -> :timeout end ) do {:ok, chunks} -> {docs, new_buffer} = process_chunks(chunks, buffer) decoded = Enum.map(docs, &Jason.decode!/1) done = Enum.member?(chunks, :done) - {decoded, {resp, new_buffer, done}} + {decoded, {resp, new_buffer, done, timeout}} {:error, _} -> - {:halt, {resp, buffer, true}} + {:halt, {resp, buffer, true, timeout}} + + :timeout -> + # No chunk arrived within the timeout window; halt to avoid hanging. + {:halt, {resp, buffer, true, timeout}} :unknown -> - next_document({resp, buffer, false}) + next_document({resp, buffer, false, timeout}) end end diff --git a/mix.exs b/mix.exs index fc00d49..6f0224e 100644 --- a/mix.exs +++ b/mix.exs @@ -1,7 +1,7 @@ defmodule TerminusDB.MixProject do use Mix.Project - @version "0.1.0" + @version "0.2.0-dev" @source_url "https://github.com/thanos/terminusdb-client-elixir" @description """ An idiomatic Elixir client for TerminusDB, the document graph database with diff --git a/test/integration/document_schema_branch_test.exs b/test/integration/document_schema_branch_test.exs index 152e695..edc6be2 100644 --- a/test/integration/document_schema_branch_test.exs +++ b/test/integration/document_schema_branch_test.exs @@ -35,10 +35,23 @@ defmodule TerminusDB.Integration.DocumentSchemaBranchTest do {:ok, frames} = Schema.all(cfg) assert is_map(frames) end + + test "retrieves a specific class frame after inserting a schema", %{config: cfg} do + Document.insert!( + cfg, + %{"@type" => "Class", "@id" => "Person", "name" => "xsd:string", "age" => "xsd:integer"}, + author: "admin", + message: "add schema", + graph_type: :schema + ) + + {:ok, frame} = Schema.frame(cfg, "Person") + assert is_map(frame) + end end describe "document lifecycle" do - test "insert, get, replace, and delete a document", %{config: cfg} do + test "insert, get, query, replace, and delete a document", %{config: cfg} do # Insert a schema first (Person class) {:ok, _} = Document.insert( @@ -54,13 +67,19 @@ defmodule TerminusDB.Integration.DocumentSchemaBranchTest do graph_type: :schema ) - # Insert a document + # Insert two documents with different ages {:ok, _} = Document.insert(cfg, %{"@type" => "Person", "name" => "Alice", "age" => 30}, author: "admin", message: "add Alice" ) + {:ok, _} = + Document.insert(cfg, %{"@type" => "Person", "name" => "Bob", "age" => 25}, + author: "admin", + message: "add Bob" + ) + # Get all documents of type Person {:ok, docs} = Document.get(cfg, type: "Person", as_list: true) assert is_list(docs) @@ -68,14 +87,47 @@ defmodule TerminusDB.Integration.DocumentSchemaBranchTest do assert person != nil assert person["age"] == 30 + # Query by template: only matching documents should be returned + {:ok, matches} = Document.query(cfg, %{"@type" => "Person", "age" => 30}) + assert is_list(matches) + names = Enum.map(matches, & &1["name"]) + assert "Alice" in names + refute "Bob" in names + # Delete the document person_id = person["@id"] - :ok = Document.delete(cfg, id: person_id, author: "admin", message: "remove Alice") + {:ok, _} = Document.delete(cfg, id: person_id, author: "admin", message: "remove Alice") # Verify it's gone {:ok, remaining} = Document.get(cfg, type: "Person", as_list: true) refute Enum.any?(remaining, &(&1["name"] == "Alice")) end + + test "stream yields documents one at a time", %{config: cfg} do + Document.insert!( + cfg, + %{"@type" => "Class", "@id" => "Item", "label" => "xsd:string"}, + author: "admin", + message: "add schema", + graph_type: :schema + ) + + Document.insert!( + cfg, + [ + %{"@type" => "Item", "label" => "one"}, + %{"@type" => "Item", "label" => "two"}, + %{"@type" => "Item", "label" => "three"} + ], + author: "admin", + message: "add items" + ) + + docs = Enum.to_list(Document.stream(cfg, type: "Item")) + + labels = Enum.sort(Enum.map(docs, & &1["label"])) + assert labels == ["one", "three", "two"] + end end describe "branch" do diff --git a/test/terminus_db/branch_test.exs b/test/terminus_db/branch_test.exs index 60ce58d..b4ac13f 100644 --- a/test/terminus_db/branch_test.exs +++ b/test/terminus_db/branch_test.exs @@ -34,13 +34,16 @@ defmodule TerminusDB.BranchTest do assert body["origin"] == "admin/mydb/local/branch/main" end - test "honors :organization and :repo overrides" do + test "honors :organization and :repo overrides in both path and origin body" do test = self() adapter = capture(test, ok()) Branch.create(db_config(adapter), "feature", organization: "acme", repo: "origin") req = last_request() assert req.url.path == "/api/branch/acme/mydb/origin/branch/feature" + + assert {:ok, body} = Jason.decode(req.body) + assert body["origin"] == "acme/mydb/origin/branch/main" end test "raises when no database is scoped" do diff --git a/test/terminus_db/client_test.exs b/test/terminus_db/client_test.exs index 0c3db76..10ae73f 100644 --- a/test/terminus_db/client_test.exs +++ b/test/terminus_db/client_test.exs @@ -30,7 +30,11 @@ defmodule TerminusDB.ClientTest do req = last_request() assert req.method == :get assert request_url(req) == "http://localhost:6363/api/ok" - assert Req.Request.get_header(req, "user-agent") == ["terminusdb_ex/0.1.0"] + + assert Req.Request.get_header(req, "user-agent") == [ + "terminusdb_ex/#{Mix.Project.config()[:version]}" + ] + assert Req.Request.get_header(req, "authorization") == ["Basic YWRtaW46cm9vdA=="] end @@ -190,8 +194,19 @@ defmodule TerminusDB.ClientTest do end describe "resource_path/2" do - test "joins org and db" do - assert Client.resource_path("admin", "mydb") == "admin/mydb" + test "joins org and db from config" do + config = Config.with_database(Config.new(endpoint: "http://localhost:6363"), "mydb") + assert Client.resource_path(config, []) == "admin/mydb" + end + + test "honors :organization override" do + config = Config.with_database(Config.new(endpoint: "http://localhost:6363"), "mydb") + assert Client.resource_path(config, organization: "acme") == "acme/mydb" + end + + test "raises when no database is scoped" do + config = Config.new(endpoint: "http://localhost:6363") + assert_raise TerminusDB.Error, fn -> Client.resource_path(config, []) end end end diff --git a/test/terminus_db/config_test.exs b/test/terminus_db/config_test.exs index 481bf27..989b619 100644 --- a/test/terminus_db/config_test.exs +++ b/test/terminus_db/config_test.exs @@ -23,7 +23,7 @@ defmodule TerminusDB.ConfigTest do assert config.ref == nil assert config.telemetry == true assert config.receive_timeout == 15_000 - assert config.user_agent == "terminusdb_ex/0.1.0" + assert config.user_agent == "terminusdb_ex/#{Mix.Project.config()[:version]}" end test "accepts overrides" do diff --git a/test/terminus_db/database_test.exs b/test/terminus_db/database_test.exs index 138779e..0a42ee6 100644 --- a/test/terminus_db/database_test.exs +++ b/test/terminus_db/database_test.exs @@ -1,7 +1,7 @@ defmodule TerminusDB.DatabaseTest do use ExUnit.Case, async: true - alias TerminusDB.{Config, Database, Error} + alias TerminusDB.{Database, Error} import TerminusDB.Test.Helpers describe "create/3" do diff --git a/test/terminus_db/document_test.exs b/test/terminus_db/document_test.exs index 22a2ec2..b62616c 100644 --- a/test/terminus_db/document_test.exs +++ b/test/terminus_db/document_test.exs @@ -156,6 +156,27 @@ defmodule TerminusDB.DocumentTest do assert q =~ "as_list=true" end + test "sends unfold=false, minimized=false, compress_ids=false when explicitly disabled" do + test = self() + adapter = capture(test, ok([])) + + Document.get(db_config(adapter), + unfold: false, + minimized: false, + compress_ids: false, + as_list: false + ) + + req = last_request() + q = req.url.query + # These are tri-state params (server defaults to true); an explicit false + # must be sent to override the default. + assert q =~ "unfold=false" + assert q =~ "minimized=false" + assert q =~ "compress_ids=false" + assert q =~ "as_list=false" + end + test "honors :organization override" do test = self() adapter = capture(test, ok()) @@ -383,5 +404,21 @@ defmodule TerminusDB.DocumentTest do assert req.url.path == "/api/document/admin/mydb" assert req.url.query =~ "type=Person" end + + test "raises TerminusDB.Error on a non-2xx response" do + adapter = fn req -> {req, resp(404, %{"@type" => "api:NotFound"})} end + + assert_raise Error, fn -> + Document.stream(db_config(adapter), type: "Person") + end + end + + test "raises TerminusDB.Error on a transport error" do + adapter = fn req -> {req, Req.TransportError.exception(reason: :econnrefused)} end + + assert_raise Error, fn -> + Document.stream(db_config(adapter), type: "Person") + end + end end end diff --git a/test/terminus_db/streaming_test.exs b/test/terminus_db/streaming_test.exs index df6c30e..85a1403 100644 --- a/test/terminus_db/streaming_test.exs +++ b/test/terminus_db/streaming_test.exs @@ -74,40 +74,6 @@ defmodule TerminusDB.StreamingTest do end describe "document_stream/1" do - # Builds a Req.Response with a fake Req.Response.Async body whose - # stream_fun decodes messages sent to the current process. - # - # The `messages` argument is a list of terms to send to the process mailbox. - # Each message is received by the `receive` block in `next_document`, then - # passed to `stream_fun`. The `stream_fun` returns `{:ok, chunks}` where - # chunks is a keyword list like `[data: "..."]` or `[:done]`. - # - # We send a sequence of `{:data, binary}` chunks followed by `:done`, which - # the stream_fun translates into the `{:ok, chunks}` / `:done` format Req uses. - defp fake_async_response(chunks) do - ref = make_ref() - - stream_fun = fn ^ref, message -> - case message do - {^ref, :data} -> {:ok, chunks} - {^ref, :done} -> {:ok, [:done]} - {^ref, {:error, reason}} -> {:error, reason} - _ -> :unknown - end - end - - cancel_fun = fn ^ref -> :ok end - - async = %Req.Response.Async{ - pid: self(), - ref: ref, - stream_fun: stream_fun, - cancel_fun: cancel_fun - } - - Req.Response.new(status: 200, body: async) - end - # Sends a sequence of data chunks followed by a done signal. # Each `data` binary is sent as `{ref, :data}` and the stream_fun returns # `{:ok, [data: data]}`. A final `{ref, :done}` signals end of stream. diff --git a/test/terminus_db/telemetry_test.exs b/test/terminus_db/telemetry_test.exs index 966ff36..2bcd14e 100644 --- a/test/terminus_db/telemetry_test.exs +++ b/test/terminus_db/telemetry_test.exs @@ -61,12 +61,17 @@ defmodule TerminusDB.TelemetryTest do end test "is a no-op when telemetry is disabled" do - meta = %{path: "db/admin/mydb", method: :post} + # Use a unique path so the unfiltered refute_receive does not catch + # :database events leaked from other concurrent tests (telemetry is + # global). We never emit on this path (telemetry disabled), so we should + # receive nothing tagged with it. + unique = "telemetry/noop-#{:erlang.unique_integer([:positive])}" + meta = %{path: unique, method: :post} assert Telemetry.start(:database, meta, @disabled) == nil assert Telemetry.stop(:database, meta, nil, @disabled, status: 200) == nil - refute_receive {:event, _, _, _}, 10 + refute_receive {:event, _, _, %{path: ^unique}}, 10 end end end From 3b6a8927240a791045cbc4198c19cfde8e2659d9 Mon Sep 17 00:00:00 2001 From: thanos Date: Wed, 24 Jun 2026 11:01:04 -0400 Subject: [PATCH 3/3] prepared for release v0.2.0 --- AGENTS.md | 7 ++--- ARCHITECTURE.md | 32 +++++++++++----------- CHANGELOG.md | 56 ++++++++++++++++++++++++++++++++------- README.md | 11 ++++---- lib/terminus_db.ex | 20 +++++++------- lib/terminus_db/schema.ex | 4 +-- mix.exs | 10 +++---- 7 files changed, 90 insertions(+), 50 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a869349..6349181 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -100,7 +100,7 @@ If you cannot find a command, ask the user and then add it here. ## Dependency policy -Minimize deps. Runtime (v0.1): `req`, `jason`, `nimble_options`, `telemetry`. +Minimize deps. Runtime (v0.2): `req`, `jason`, `nimble_options`, `telemetry`. Ecto and ExDatalog will be **optional** dependencies, gated by compile flags, when their integrations land (v0.3/v0.4). Do not add a runtime dep without an ADR justifying it. @@ -109,8 +109,9 @@ ADR justifying it. | Milestone | Scope | Status | | --- | --- | --- | -| **v0.1 (foundation)** | Config, Error, Telemetry, Client (Req), Database API, unit tests, CI, ADRs, ARCHITECTURE, AGENTS | **Done (this session)** | -| v0.2 (APIs + DSL) | Document, Schema, Branch, Commit, Diff, Merge, GraphQL, WOQL functional DSL, Streaming, integration tests, benchmarks, guides | **In progress** (Document, Schema, Branch, Streaming done) | +| **v0.1 (foundation)** | Config, Error, Telemetry, Client (Req), Database API, unit tests, CI, ADRs, ARCHITECTURE, AGENTS | **Done** | +| **v0.2 (APIs)** | Document, Schema, Branch, Streaming, Document.query/stream, guides, Livebook | **Done** | +| v0.2.1+ | Commit, Diff, Merge, GraphQL, WOQL functional DSL, benchmarks | Pending | | v0.3 (Ecto) | `use TerminusDB.Schema` macro, schema generation, optional `:ecto` dep; research full `Ecto.Adapter` feasibility | Pending | | v0.4 (ExDatalog) | `TerminusDB.Datalog` bridge (`to_woql`, `to_jsonld`, `from_triples`), optional `:datalog` dep | Pending | | Release | CHANGELOG, release notes, migration guide, Hex.pm review, final review | Pending | diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index f2975dd..997643c 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -136,10 +136,11 @@ the graph engine. **Adopt Option C as the target architecture, delivered incrementally.** -- **v0.1 (this milestone):** Option A core - `Client`, `Config`, `Error`, `Database`, - telemetry, streaming, and the WOQL/raw-query execution primitives. This is the - verifiable foundation everything else builds on. -- **v0.2:** Document + Schema + Branch + Commit + Diff + Merge APIs and the WOQL DSL. +- **v0.1 (done):** Option A core - `Client`, `Config`, `Error`, `Database`, + telemetry, and the HTTP/wire primitives. +- **v0.2 (done):** Document CRUD + query + streaming, Schema frame retrieval, + Branch management, and the concatenated-JSON streaming decoder. +- **v0.2.1+:** Commit, Diff, Merge, GraphQL, and the WOQL functional DSL. - **v0.3:** Ecto integration (`TerminusDB.Schema`) - Option B. - **v0.4:** ExDatalog integration - Option C. - **Option D is rejected** for v0.x; a local engine is out of scope for a client. @@ -156,19 +157,20 @@ TerminusDB ├── Application OTP supervision tree ├── Config immutable connection/context (NimbleOptions-validated) ├── Client Req-based HTTP wrapper; the only module that touches the wire +│ └── Params internal query-param helpers (flag/bool) ├── Error typed error struct + exception ├── Database database management API -├── Document document CRUD + query (v0.2) -├── Schema schema frame API + Ecto macro (v0.2/v0.3) -├── Branch branch API (v0.2) -├── Commit history / log (v0.2) -├── Diff diff + patch (v0.2) -├── Merge push / pull / rebase (v0.2) -├── WOQL functional DSL → JSON-LD (v0.2) -├── GraphQL GraphQL execution (v0.2) -├── Datalog ExDatalog integration (v0.4) -├── Telemetry event definitions + helpers -└── Streaming document stream helpers +├── Document document CRUD + query + streaming ✓ v0.2 +├── Schema schema frame API ✓ v0.2 +├── Branch branch API ✓ v0.2 +├── Streaming concatenated-JSON stream decoder ✓ v0.2 +├── Commit history / log (v0.2.1+) +├── Diff diff + patch (v0.2.1+) +├── Merge push / pull / rebase (v0.2.1+) +├── WOQL functional DSL → JSON-LD (v0.2.1+) +├── GraphQL GraphQL execution (v0.2.1+) +├── Datalog ExDatalog integration (v0.4) +└── Telemetry event definitions + helpers ``` ### 3.1 Principles diff --git a/CHANGELOG.md b/CHANGELOG.md index 88fba3d..b83b490 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,25 +5,62 @@ All notable changes to `terminusdb_ex` are documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [0.2.0] — 2026-06-24 -### Added — v0.2 (in progress) +### Added — document, schema, branch, and streaming APIs - **`TerminusDB.Document`**: document CRUD and query API — `insert/3`, `get/2`, `query/3`, `replace/3`, `delete/2`, `stream/2` (plus `!/` variants). Supports `graph_type` (instance/schema), `author`/`message` commit metadata, `full_replace`, `raw_json`, `create`, `nuke`, pagination (`skip`/`count`), - `as_list`, `unfold`, `minimized`, `compress_ids`. + `as_list`, `unfold`, `minimized`, `compress_ids`. The `stream/2` function + returns a lazy `Enumerable` of decoded documents with constant memory via + Req's `into: :self` and the concatenated-JSON splitter. - **`TerminusDB.Schema`**: schema frame API — `frame/3`, `all/2` (plus `!/` - variants). Supports `compress_ids` and `expand_abstract` params. + variants). Supports `compress_ids` and `expand_abstract` params (including + explicit `false`). - **`TerminusDB.Branch`**: branch management API — `create/3`, `delete/3`, `exists?/3` (plus `!/` variants). Supports `:from`, `:organization`, `:repo` - overrides. + overrides in both the path and the origin body. - **`TerminusDB.Streaming`**: incremental concatenated-JSON decoder for streaming - document responses (ADR-0007). `split_concatenated/1` and `document_stream/1`. -- Integration tests for Document, Schema, and Branch against a Dockerized - TerminusDB. -- 49 new unit tests covering Document, Schema, Branch, and Streaming. + document responses (ADR-0007). `split_concatenated/1` (bracket/depth-aware + splitter respecting string literals, escapes, and cross-chunk boundaries) and + `document_stream/2` (with a configurable receive timeout). +- Internal shared helper for building query parameters (`Client.Params`), + parameters, distinguishing flag params (omit when false) from tri-state bool + params (send explicit false to override server defaults). +- **`TerminusDB.Client.resource_path/2`**: now resolves org/db from config and + opts, used by all per-module path builders. +- Integration tests for Document (insert/query/stream/delete), Schema (frame + retrieval), and Branch (create/exists?/delete) against a Dockerized TerminusDB. +- 54 new unit tests covering Document, Schema, Branch, and Streaming. +- Guides: `guides/introduction.md` (TerminusDB concepts), `guides/migrating-from-sql.md` + (SQL-to-TerminusDB migration by example), `guides/overview.md` (feature walkthrough), + `guides/terminusdb_ex_livebook.livemd` (full Livebook demo). +- Hermetic doctest examples on every public function (27 doctests total). + +### Fixed + +- `Document.get/2`: `unfold`, `minimized`, `compress_ids`, and `as_list` set to + `false` are now sent to the server (previously silently dropped, so the + server's `true` defaults always won). +- `Document.stream/2`: raises `TerminusDB.Error` on client errors instead of + `MatchError`. +- `Branch.create/3`: `:organization` and `:repo` overrides are now reflected in + the origin body, not just the path. +- `Streaming.document_stream/2`: no longer hangs if the server never sends + `:done` — a receive timeout halts the stream. +- `Streaming.split_concatenated/1`: handles a lone trailing `\` at a chunk + boundary inside a string (retains it for the next chunk). +- Flaky telemetry test fixed (unique-path filtered `refute_receive`). +- All documentation examples corrected: `{:ok, _} = delete(...)` instead of + `:ok = delete(...)`. + +### Changed + +- `Client.resource_path/2` signature changed from `(org, db)` to `(config, opts)`. +- Version bumped to 0.2.0; `source_ref` updated for HexDocs. +- `.sobelow-conf` ignores `SQL.Query` false positive on `Document.query/3`. ## [0.1.0] — 2026-06-23 @@ -72,3 +109,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 manually via `mix test --only integration`. [0.1.0]: https://github.com/thanos/terminusdb-client-elixir/releases/tag/v0.1.0 +[0.2.0]: https://github.com/thanos/terminusdb-client-elixir/releases/tag/v0.2.0 diff --git a/README.md b/README.md index e652001..5ed09ad 100644 --- a/README.md +++ b/README.md @@ -11,10 +11,11 @@ graph database with built-in version control. It is built on [Req](https://hexdocs.pm/req) and treats connection context as **immutable data**, making it safe for concurrent use. -> **Status:** v0.2 (in progress). `Config`, `Error`, `Telemetry`, `Client`, -> `Database`, `Document`, `Schema`, `Branch`, and `Streaming` are implemented and -> tested. Commit, diff, merge, WOQL DSL, GraphQL, benchmarks, and guides are -> planned for later. See `ARCHITECTURE.md` and `AGENTS.md` for the roadmap. +> **Status:** v0.2.0. `Config`, `Error`, `Telemetry`, `Client`, `Database`, +> `Document` (with streaming), `Schema`, and `Branch` are implemented and tested. +> Commit, diff, merge, WOQL DSL, GraphQL, and optional Ecto/ExDatalog integrations +> are planned for later milestones. See `ARCHITECTURE.md` and `AGENTS.md` for the +> roadmap. ## Installation @@ -23,7 +24,7 @@ Add `terminusdb_client` to your dependencies in `mix.exs`: ```elixir def deps do [ - {:terminusdb_client, "~> 0.1.0"} + {:terminusdb_client, "~> 0.2.0"} ] end ``` diff --git a/lib/terminus_db.ex b/lib/terminus_db.ex index 8735f11..f2ea381 100644 --- a/lib/terminus_db.ex +++ b/lib/terminus_db.ex @@ -3,16 +3,16 @@ defmodule TerminusDB do An idiomatic Elixir client for [TerminusDB](https://terminusdb.org), the document graph database with built-in version control. - v0.1 provides the foundation: an immutable `TerminusDB.Config` for connection - context, a Req-based `TerminusDB.Client` HTTP layer, a typed `TerminusDB.Error`, - the `TerminusDB.Database` management API, and `TerminusDB.Telemetry` events on - every operation. Connection context is **immutable data**, making it safe for - concurrent use. - - Document, schema, branch, commit, diff, merge, WOQL, GraphQL, and streaming - APIs, plus optional Ecto and ExDatalog integrations, are planned for later - milestones. See `ARCHITECTURE.md` and the ADRs (under `docs/adr/`) for the full - design and roadmap. + The client provides an immutable `TerminusDB.Config` for connection context, a + Req-based `TerminusDB.Client` HTTP layer, a typed `TerminusDB.Error`, the + `TerminusDB.Database` management API, `TerminusDB.Document` CRUD with streaming, + `TerminusDB.Schema` frame retrieval, `TerminusDB.Branch` management, and + `TerminusDB.Telemetry` events on every operation. Connection context is + **immutable data**, making it safe for concurrent use. + + Commit, diff, merge, WOQL DSL, GraphQL, and optional Ecto/ExDatalog + integrations are planned for later milestones. See `ARCHITECTURE.md` and the + ADRs (under `docs/adr/`) for the full design and roadmap. ## Quick start diff --git a/lib/terminus_db/schema.ex b/lib/terminus_db/schema.ex index 425d725..21df27b 100644 --- a/lib/terminus_db/schema.ex +++ b/lib/terminus_db/schema.ex @@ -152,6 +152,6 @@ defmodule TerminusDB.Schema do @spec all!(Config.t(), [frame_opt()]) :: map() def all!(config, opts \\ []), do: frame!(config, nil, opts) - # For boolean params where `false` is a meaningful value (not a default to omit), - # we pass it through explicitly via Params.bool_param/2 (see TerminusDB.Client.Params). + # For boolean params where `false` is a meaningful value (not a default to + # omit), we pass it through explicitly via Params.bool_param/2. end diff --git a/mix.exs b/mix.exs index 6f0224e..54c3f60 100644 --- a/mix.exs +++ b/mix.exs @@ -1,15 +1,13 @@ defmodule TerminusDB.MixProject do use Mix.Project - @version "0.2.0-dev" + @version "0.2.0" @source_url "https://github.com/thanos/terminusdb-client-elixir" @description """ An idiomatic Elixir client for TerminusDB, the document graph database with - built-in version control. v0.1 provides database management, an immutable - connection config, a Req-based HTTP client, typed errors, and telemetry. - Document, schema, branch, commit, diff, merge, WOQL, GraphQL, and streaming - APIs, plus optional Ecto and ExDatalog integrations, are planned for later - milestones. + built-in version control. Provides database management, document CRUD with + streaming, schema frame retrieval, branch management, a Req-based HTTP client, + immutable connection config, typed errors, and telemetry. """ def project do