From 8f39eb4d0d7f419da7e52ce90c3226e8294bb112 Mon Sep 17 00:00:00 2001 From: Remy Boutonnet Date: Mon, 9 Nov 2026 19:59:50 +0100 Subject: [PATCH 1/2] Update the docs for TuringDB 3.1 and add release notes Brings the docs in line with turingdb 3.1 (PyPI 3.1, npm 3.1.0). Every Cypher example in query/ (382 of them) was run against a 3.1 server, and the Python and JavaScript snippets against fresh 3.1 servers. Protocol and clients: - the server speaks only the binary protocol over HTTP/1.1, so the HTTP API section, USE_TURING_PROTO, TURINGDB_TYPE and type="json" are gone; commands.mdx gains a "Connecting clients" section; - TuringDB(host="localhost", port=6666) replaces the URL form, which 3.1 rejects, on every page; the Python SDK reference covers the two backends, the new connection errors, turingdb.Point and parameter types; - troubleshooting is rewritten around the errors 3.1 raises. New JavaScript SDK pages (jssdk/): get started for Node.js, TypeScript and the browser (same-origin proxy, Vite optimizeDeps.exclude, nginx), and a reference for options, columns, value and parameter types, versioning and errors. Query language: parameters for values, map projection and m['key'], SET n = map / SET n += map, points and spatial functions, exists() over a pattern, allReduce, IS label tests, elementId, isEmpty, coll.indexOf, split on a list, datetime of a map, sum/avg of durations, /* */ comments, keywords as names, UNION/NEXT/WHEN inside every subquery, COLLECT of maps and paths, paths as flat ID lists, writes from map rows without conversion, ID lookups with literal lists and parameters, and the 3.0 -> 3.1 behaviour changes. Release notes: a new "Release Notes" group with an overview of all releases and a TuringDB 3.1 page built around query parameters, the binary protocol over HTTP/1.1, the WebAssembly-based TypeScript client, allReduce, COLLECT and other subqueries, and points with haversine distance, plus six pixel icons for its highlight cards and a banner on the home page. --- benchmarks/technical-report.mdx | 6 +- claude-skill.mdx | 2 +- commands.mdx | 50 ++-- docs.json | 14 ++ graph_dev/create_graph.mdx | 5 +- graph_dev/graph_examples.mdx | 5 +- graph_dev/time_travelling.mdx | 22 +- import_data/gml.mdx | 2 +- import_data/jsonl.mdx | 9 +- import_data/neo4j.mdx | 2 +- import_data/parquet.mdx | 5 +- index.mdx | 31 ++- jssdk/get_started.mdx | 183 ++++++++++++++ jssdk/reference.mdx | 250 +++++++++++++++++++ public/images/icons/pixel-bolt.png | Bin 0 -> 281 bytes public/images/icons/pixel-braces.png | Bin 0 -> 294 bytes public/images/icons/pixel-dollar.png | Bin 0 -> 314 bytes public/images/icons/pixel-forall.png | Bin 0 -> 258 bytes public/images/icons/pixel-nested.png | Bin 0 -> 260 bytes public/images/icons/pixel-pin.png | Bin 0 -> 277 bytes pythonsdk/get_started.mdx | 29 ++- pythonsdk/reference.mdx | 148 ++++++----- query/aggregation.mdx | 35 ++- query/cheatsheet.mdx | 58 ++++- query/composition.mdx | 54 +++- query/cypher_subset.mdx | 83 +++++-- query/functions.mdx | 186 +++++++++++--- query/matching.mdx | 33 ++- query/paths.mdx | 40 ++- query/procedures.mdx | 7 +- query/subqueries.mdx | 23 +- query/writing.mdx | 73 ++++-- quickstart.mdx | 9 +- releases/overview.mdx | 36 +++ releases/v3-1.mdx | 354 +++++++++++++++++++++++++++ troubles/troubles.mdx | 78 ++++-- tutorials/create_graph.mdx | 2 +- vector-search.mdx | 2 +- 38 files changed, 1543 insertions(+), 293 deletions(-) create mode 100644 jssdk/get_started.mdx create mode 100644 jssdk/reference.mdx create mode 100644 public/images/icons/pixel-bolt.png create mode 100644 public/images/icons/pixel-braces.png create mode 100644 public/images/icons/pixel-dollar.png create mode 100644 public/images/icons/pixel-forall.png create mode 100644 public/images/icons/pixel-nested.png create mode 100644 public/images/icons/pixel-pin.png create mode 100644 releases/overview.mdx create mode 100644 releases/v3-1.mdx diff --git a/benchmarks/technical-report.mdx b/benchmarks/technical-report.mdx index 6a00660..43b66cf 100644 --- a/benchmarks/technical-report.mdx +++ b/benchmarks/technical-report.mdx @@ -28,7 +28,7 @@ Won 48/48 queries vs Neo4j Won 45/47 queries vs Memgraph And remember: -All of this is running over plain JSON/HTTP for now, while competitors used Bolt (binary protocol), which actually makes the benchmark conservative in our case. +These runs queried TuringDB through the Python client it shipped at the time, which exchanged JSON with the server, while competitors used Bolt (a binary protocol). That makes the benchmark conservative in our case. Since version 3.1, TuringDB's clients use its own binary protocol. We’re also running completely out-of-the-box with no manual tuning, no custom configuration, no indexes on our side. @@ -133,7 +133,7 @@ All databases were run locally on the same machine, one at a time, to avoid reso ### Client Protocol -TuringDB is queried over **HTTP** via its Python client, while Neo4j and Memgraph are queried over the **Bolt** binary protocol. Bolt is a more efficient wire protocol than HTTP for database communication, meaning TuringDB’s measured times include higher protocol overhead. This makes the benchmark conservative in TuringDB’s favor — with an equivalent binary protocol, TuringDB’s results would be even faster. +These results were measured with an earlier TuringDB Python client, which exchanged JSON with the server, while Neo4j and Memgraph are queried over the **Bolt** binary protocol. JSON costs more to encode and decode than a binary wire protocol, meaning TuringDB’s measured times include higher protocol overhead. This makes the benchmark conservative in TuringDB’s favor. Since version 3.1, TuringDB’s clients use its own binary protocol. --- @@ -598,7 +598,7 @@ The following **2 queries** (4% of benchmark) show a competitor outperforming Tu - **Single-machine benchmark.** All engines ran locally; distributed deployment characteristics are not measured. - **Cold-run only.** Results reflect first-execution performance. Workloads with repeated identical queries would benefit from Neo4j/Memgraph’s query caching. - **Import-provided configuration.** Neo4j and Memgraph use the indexes and constraints from the original dump, and Memgraph runs in in-memory analytical mode. No additional tuning was applied. Production deployments may include further optimizations. -- **Client-side timing.** Measurements include Python client overhead and network round-trip (localhost). TuringDB uses HTTP while Neo4j and Memgraph use the more efficient Bolt protocol, giving the competitors a wire-protocol advantage. +- **Client-side timing.** Measurements include Python client overhead and network round-trip (localhost). At the time of these runs TuringDB's client exchanged JSON with the server, while Neo4j and Memgraph use the more efficient Bolt protocol, giving the competitors a wire-protocol advantage. - **Two datasets.** Results may not generalize to all graph structures or query patterns. Additional datasets would strengthen conclusions. - **No concurrent load.** The benchmark runs queries sequentially with no simulated concurrent users. diff --git a/claude-skill.mdx b/claude-skill.mdx index 1d46609..9c03cf5 100644 --- a/claude-skill.mdx +++ b/claude-skill.mdx @@ -41,7 +41,7 @@ The skill is structured so Claude only loads the reference it needs: | Module | Covers | | --- | --- | -| `startup` | Installation, starting the server, connecting (HTTP, authentication, embedded), loading a graph | +| `startup` | Installation, starting the server, connecting (server, authentication, embedded), loading a graph | | `querying` | `MATCH`, `OPTIONAL MATCH`, label expressions, paths, `WITH`, `NEXT`, `WHEN`, aggregation, `UNWIND`, maps, subqueries, built-in functions, parameters, `LOAD CSV` | | `writing` | `CREATE`, `MERGE`, `SET`, `DELETE`, `FOREACH`, bulk edges by node ID, the change and commit workflow | | `importing` | CSV, Parquet, JSONL, GML and Neo4j imports | diff --git a/commands.mdx b/commands.mdx index c0551a5..f25ffc1 100644 --- a/commands.mdx +++ b/commands.mdx @@ -1,7 +1,7 @@ --- title: "CLI Commands" sidebarTitle: "Commands" -description: "Reference for TuringDB CLI commands (start, UI and stop), the interactive shell, environment variables and the HTTP API." +description: "Reference for TuringDB CLI commands (start, UI and stop), the interactive shell, environment variables and how clients connect." --- ## Start @@ -17,8 +17,8 @@ turingdb start [options] | Flag | Description | Default | |---|---|---| | `-demon` | Run TuringDB as a background daemon | off | -| `-p ` | HTTP server listen port | `6666` | -| `-i ` | HTTP server listen address | `127.0.0.1` | +| `-p ` | Server listen port | `6666` | +| `-i ` | Server listen address | `127.0.0.1` | | `-turing-dir ` | Root Turing home directory | `~/.turing` | | `-load ` | Load a graph at startup (can be repeated) | | | `-in-memory` | Run without writing graphs to disk | off | @@ -62,10 +62,10 @@ Require authentication: TURINGDB_AUTH_TOKEN=s3cret turingdb start -demon -auth-on ``` -Serve the binary protocol instead of JSON (see [Environment variables](#environment-variables)): +Serve a Node.js application, whose `fetch` refuses the default port (see [Connecting clients](#connecting-clients)): ```bash -USE_TURING_PROTO=1 turingdb start -demon +turingdb start -demon -p 6686 ``` @@ -74,15 +74,16 @@ USE_TURING_PROTO=1 turingdb start -demon ### Interactive shell -Without `-demon`, `turingdb start` serves HTTP **and** opens an interactive Cypher shell. Besides Cypher queries and [engine commands](/query/procedures#commands), the shell has a few local commands: +Without `-demon`, `turingdb start` serves clients **and** opens an interactive Cypher shell. Besides Cypher queries and [engine commands](/query/procedures#commands), the shell has a few local commands: | Command | Description | |---|---| | `cd ` | Switch the active graph | | `checkout change-` | Work on a change. `checkout ` opens a past commit, and `checkout` returns to the latest commit of main | | `read ` | Run the queries in a file | -| `:param => ` | Set a [query parameter](/query/cypher_subset#query-parameters), with the value written as JSON: `:param label => "Person"`. `:param clear` removes them all | -| `SHOW PARAMETERS` | List the parameters set in the shell | +| `:param => ` | Set a [query parameter](/query/cypher_subset#query-parameters), with the value written as a Cypher expression: `:param label => 'Person'`, `:param ids => [0, 1, 2]`, `:param since => datetime('2024-01-01T00:00:00Z')`. `:param clear` removes them all | +| `SHOW PARAMETERS` | List the parameters set in the shell, with their types | +| `connect ` | Send the next queries to another TuringDB server, for example `connect 127.0.0.1 6666`. `disconnect` returns to the local engine | | `help` | List the available commands | | `quit` / `exit` | Leave the shell | @@ -156,25 +157,24 @@ turingdb stop -turing-dir /data/myturing | Variable | Read by | Effect | |---|---|---| -| `USE_TURING_PROTO` | server | `1` makes the server speak TuringDB's **binary protocol** instead of JSON over HTTP. The server then serves the binary protocol only: connect with `TuringDB(type="native")`, and use a server in the default mode for `curl`, the JSON client and the visualizer. See [client backends](/pythonsdk/reference#client-backends) | -| `TURINGDB_AUTH_TOKEN` | server and Python SDK | With `-auth-on`, the token the server requires. The SDK sends it as a bearer token when set | -| `TURINGDB_TYPE` | Python SDK | Default client backend (`json`, `native` or `embedded`) when `type` is not passed | +| `TURINGDB_AUTH_TOKEN` | server and Python SDK | With `-auth-on`, the token the server requires. The Python SDK sends it as a bearer token when set. The JavaScript SDK takes the token as its `authToken` option | --- -## HTTP API +## Connecting clients -The Python SDK is a thin client over a single endpoint, so you can query a server from any language. With the default (JSON) server, send the query as a JSON object: +The server speaks TuringDB's binary protocol, on port 6666 unless you pass `-p`. Query it with one of these clients: -```bash -curl -s -X POST 'http://localhost:6666/query?graph=mygraph' \ - -d '{"query": "MATCH (n:$label) RETURN count(n)", "parameters": {"label": "Person"}}' -``` - -- The request body is a JSON object with a `query` string and an optional `parameters` object (see [query parameters](/query/cypher_subset#query-parameters)). -- The optional query-string parameters `change=` and `commit=` target a change or a past commit. The change ID is the decimal number that `CHANGE NEW` returns. -- With `-auth-on`, send `Authorization: Bearer `. A missing or incorrect token returns HTTP `401` with an empty body. -- A request, headers included, can be up to 1 MiB. A larger one returns HTTP `413` with the code `REQUEST_TOO_BIG`. -- A successful response is `{"header": {"column_names": [...], "column_types": [...]}, "data": [chunk, ...], "time": }`, where each chunk is a list of columns. Doubles are written with full precision. -- An error response is `{"error": "PARSE_ERROR", "error_details": "...", "time": }`. The code is one of `PARSE_ERROR`, `ANALYZE_ERROR`, `PLAN_ERROR`, `EXEC_ERROR`, `GRAPH_NOT_FOUND`, `CHANGE_NOT_FOUND`, `COMMIT_NOT_FOUND` or `REQUEST_TOO_BIG`, and the details name the graph, change or commit that was not found. -- Queries on a non-HEAD commit need `LOAD COMMIT ''` first when you talk to the API directly. The CLI and the SDK do this for you. +| Client | Install | Connect | +|---|---|---| +| [Python SDK](/pythonsdk/reference) | `pip install turingdb` | `TuringDB(host="localhost", port=6666)` | +| [JavaScript SDK](/jssdk/reference) | `npm install turingdb` | `new TuringClient({ url: "http://localhost:6686/query" })` | +| Interactive shell | ships with the server | `turingdb` without `-demon`, or `connect ` from another shell | +| [Visualizer](#ui) | ships with the server | `turingdb -ui` | + +- The protocol runs over HTTP/1.1: each query is a request to `/query`, and its result streams back as binary packets, one per HTTP chunk, on a connection kept open for the next query. Standard reverse proxies such as nginx forward it with a plain `proxy_pass`. +- Each query carries the graph, change and commit it runs on, so the server keeps no session for a client. +- A request can be up to 1 MiB, the query text and its parameters together. A larger one is refused with `The request is over the server limit of 1048576 bytes for the request line, headers and body together`. Split large batches, or load bulk data from a file with [`LOAD PARQUET`](/import_data/parquet) or [`LOAD JSONL`](/import_data/jsonl). +- With `-auth-on`, every request needs the token. A missing or wrong one is refused with `Unauthorized: missing or invalid authentication token`. +- Browsers and the `fetch` of Node.js refuse port 6666. For Node.js, start the server on another port, such as `-p 6686`. A web page reaches the server through a proxy on its own origin, see [In the browser](/jssdk/get_started#in-the-browser). +- Upgrade the clients and the server together: a 3.1 client speaks the protocol of a 3.1 server. diff --git a/docs.json b/docs.json index d797c5a..2c53b42 100644 --- a/docs.json +++ b/docs.json @@ -125,6 +125,20 @@ "pythonsdk/reference" ] }, + { + "group": "JavaScript SDK", + "pages": [ + "jssdk/get_started", + "jssdk/reference" + ] + }, + { + "group": "Release Notes", + "pages": [ + "releases/overview", + "releases/v3-1" + ] + }, { "group": "Troubleshooting", "pages": [ diff --git a/graph_dev/create_graph.mdx b/graph_dev/create_graph.mdx index 12a952d..ac5b250 100644 --- a/graph_dev/create_graph.mdx +++ b/graph_dev/create_graph.mdx @@ -16,9 +16,8 @@ description: "Create a TuringDB graph" from turingdb import TuringDB # Create TuringDB client - # set host parameter to the URL (as string) on which TuringDB is running, - # default "http://localhost:6666" - client = TuringDB(host="http://localhost:6666") + # host and port of the running TuringDB server (default localhost:6666) + client = TuringDB(host="localhost", port=6666) # Create a new graph called my_graph client.create_graph("mygraph") diff --git a/graph_dev/graph_examples.mdx b/graph_dev/graph_examples.mdx index 661bce7..4f1efb4 100644 --- a/graph_dev/graph_examples.mdx +++ b/graph_dev/graph_examples.mdx @@ -8,9 +8,8 @@ description: "Complete example on how to create and query a graph from scratch i from turingdb import TuringDB # Create TuringDB client -# set host parameter to the URL (as string) on which TuringDB is running, -# default "http://localhost:6666" -client = TuringDB(host="http://localhost:6666") +# host and port of the running TuringDB server (default localhost:6666) +client = TuringDB(host="localhost", port=6666) # Create a new graph called my_graph client.create_graph("mygraph") diff --git a/graph_dev/time_travelling.mdx b/graph_dev/time_travelling.mdx index 11a8e2b..38e8085 100644 --- a/graph_dev/time_travelling.mdx +++ b/graph_dev/time_travelling.mdx @@ -60,25 +60,27 @@ LOAD COMMIT 'c0a6ce3c0315be7a' **When do I need `LOAD COMMIT`?** - - **TuringDB CLI**: The `checkout` command automatically loads the commit for you — no need to call `LOAD COMMIT` manually. - - **Python SDK**: `client.checkout()` automatically loads the commit for you — no need to call `LOAD COMMIT` manually. - - **REST API (`/query` endpoint)**: If you send Cypher queries directly to the `/query` endpoint, you **must** call `LOAD COMMIT ''` explicitly before querying any commit that is not the HEAD commit at startup time. + - **TuringDB shell**: The `checkout` command automatically loads the commit for you — no need to call `LOAD COMMIT` manually. + - **Python SDK**: `client.checkout(commit=...)` automatically loads the commit for you. `client.set_commit(...)` does not, so call `LOAD COMMIT` first when you use it. + - **JavaScript SDK**: You **must** call `LOAD COMMIT ''` before querying a past commit with `setCommit()` or the `commit` option, unless the commit was created since the server started. -### Example: Querying a Past Commit via the REST API +### Example: Querying a past commit from the JavaScript SDK -```jsx +```js // Step 1: Load the commit into memory -LOAD COMMIT 'be9643a1c03f8e2b' +await client.query("LOAD COMMIT 'be9643a1c03f8e2b'"); // Step 2: Now you can query at that commit -MATCH (n:Person)-[e:KNOWS]->(m:Person) -RETURN n.name, m.name +const rows = await client.queryData( + "MATCH (n:Person)-[e:KNOWS]->(m:Person) RETURN n.name, m.name", + { commit: "be9643a1c03f8e2b" }, +); ``` - If you try to query a commit that has not been loaded, TuringDB will return an error: - `Commit not loaded to memory - use LOAD COMMIT '' to load a commit` + If you try to query a commit that has not been loaded, TuringDB will return a `COMMIT_NOT_LOADED` error: + `Commit 'be9643a1c03f8e2b' is not loaded to memory - use LOAD COMMIT 'be9643a1c03f8e2b' to load it` `LOAD COMMIT` is **idempotent** — calling it on an already-loaded commit is a no-op and returns immediately. \ No newline at end of file diff --git a/import_data/gml.mdx b/import_data/gml.mdx index ea2842e..8ec30ef 100644 --- a/import_data/gml.mdx +++ b/import_data/gml.mdx @@ -67,7 +67,7 @@ graph [ ```python from turingdb import TuringDB - client = TuringDB(host="http://localhost:6666") + client = TuringDB(host="localhost", port=6666) client.query("LOAD GML 'mygraph.gml' AS mygraph") ``` diff --git a/import_data/jsonl.mdx b/import_data/jsonl.mdx index 258d072..dc7e47f 100644 --- a/import_data/jsonl.mdx +++ b/import_data/jsonl.mdx @@ -45,12 +45,12 @@ Each line is a self-contained JSON object representing either a node or a relati - Scalars (strings, integers, doubles, booleans) become properties of the matching type, and `null` values are skipped. - Arrays load as **List** properties. -- Nested objects are stored as **JSON strings**. +- Nested objects are stored as **JSON strings**, unless you declare them as points (see below). - ISO date strings stay strings, and integer durations stay integers, unless you declare them (see below). Each property name has a single type across the whole graph, so use the same type for a property on every node. -## Loading date, duration and embedding properties +## Loading date, duration, point and embedding properties Typed options can be chained after the graph name: @@ -59,11 +59,14 @@ LOAD JSONL 'mydata.jsonl' AS mygraph WITH EMBEDDINGS [{"emb", 384}, {"summary_vec", 768}] WITH DATETIMES [born, updatedAt] WITH DURATIONS [tenure] + WITH POINTS [location] ``` - `WITH DATETIMES [prop, ...]` parses ISO 8601 strings into `DateTime` values. - `WITH DURATIONS [prop, ...]` reads integers as microseconds and stores them as `Duration` values. +- `WITH POINTS [prop, ...]` reads JSON objects as [points](/query/functions#points), with the keys `point()` takes: `{"longitude": 2.3522, "latitude": 48.8566}` is a WGS-84 point and `{"x": 1, "y": 2}` a cartesian one. A JSON `null` leaves the property unset. - `WITH EMBEDDINGS [{"prop", dim}, ...]` stores arrays as `Embedding` values, see below. +- A property can be declared with one type only. ### Loading embedding properties @@ -104,7 +107,7 @@ Each spec is `{"", }`. The dimension must be **greater ```python from turingdb import TuringDB - client = TuringDB(host="http://localhost:6666") + client = TuringDB(host="localhost", port=6666) client.query("LOAD JSONL 'mydata.jsonl' AS mygraph") ``` diff --git a/import_data/neo4j.mdx b/import_data/neo4j.mdx index baffbb1..106752a 100644 --- a/import_data/neo4j.mdx +++ b/import_data/neo4j.mdx @@ -61,7 +61,7 @@ Then start Neo4j 5.x and proceed with the export below. ```python from turingdb import TuringDB - client = TuringDB(host="http://localhost:6666") + client = TuringDB(host="localhost", port=6666) client.query("LOAD JSONL 'output.json' AS mygraph") ``` diff --git a/import_data/parquet.mdx b/import_data/parquet.mdx index 0974bc3..421b97f 100644 --- a/import_data/parquet.mdx +++ b/import_data/parquet.mdx @@ -46,10 +46,13 @@ Every other column becomes a property of the node or edge on that row. The colum | `BOOLEAN` | Boolean | | `BYTE_ARRAY` (string) | String | | timestamp | DateTime | +| WKB geometry of points | [Point](/query/functions#points) | | list of the types above | List | Use 64-bit columns: cast `INT32` and `FLOAT` columns to `int64` / `float64` first. To load an `INT64` column of microseconds as a `Duration`, name it in `WITH DURATIONS []`. +A geometry column is read as points with no extra clause: a column marked with Parquet's `GEOMETRY` or `GEOGRAPHY` logical type, or named in the `geo` metadata of a GeoParquet file, with the WKB encoding. Its values must be points, 2D or 3D. With the default CRS (`OGC:CRS84` or `EPSG:4326`) they load as WGS-84 points, and with `"crs": null` in the GeoParquet metadata as cartesian points. + ## From pandas To go from CSV, JSON, SQL extracts or API data, load it into pandas, map your natural keys to `INT64` ids, and write the two files: @@ -183,7 +186,7 @@ The example below uses the [OptimusKG](https://optimuskg.ai/) biomedical knowled ```python from turingdb import TuringDB - client = TuringDB(host="http://localhost:6666") + client = TuringDB(host="localhost", port=6666) client.load_graph("optimuskg") client.set_graph("optimuskg") diff --git a/index.mdx b/index.mdx index 40ca18e..f3ecedb 100644 --- a/index.mdx +++ b/index.mdx @@ -4,11 +4,15 @@ sidebarTitle: "Introduction" description: "TuringDB is the fastest graph database engine for analytical, AI-driven, and read-intensive workloads" --- + + **TuringDB 3.1 is out:** query parameters, the binary protocol over HTTP/1.1 by default, a WebAssembly-based TypeScript client on npm, `allReduce`, `COLLECT` and other subqueries, and points with haversine distances. Read the [release notes](/releases/v3-1). + + ## What is TuringDB? TuringDB is an in-memory column oriented graph database engine built in C++ for analytical, AI-driven, and read-intensive workloads. With version-controlled storage and zero-locking execution, it gives you **blazing-fast queries**, **snapshot isolation**, and **seamless integration** with modern AI pipelines. -The platform has three main components: the [TuringDB engine](/concepts/overview) (a columnar, version-controlled graph store), the [Python SDK](/pythonsdk/reference) for building applications and pipelines, and a [built-in visualizer](/quickstart) for exploring graphs in the browser. +The platform has three main components: the [TuringDB engine](/concepts/overview) (a columnar, version-controlled graph store), the [Python](/pythonsdk/reference) and [JavaScript](/jssdk/reference) SDKs for building applications and pipelines, and a [built-in visualizer](/quickstart) for exploring graphs in the browser. ## Guides @@ -59,11 +63,19 @@ pip install turingdb turingdb ``` +```bash npm install +# Install the TuringDB JavaScript SDK +npm install turingdb + +# Launch the server on a port Node.js accepts +turingdb -p 6686 +``` + ```python Python SDK from turingdb import TuringDB # Connect to a running TuringDB server -client = TuringDB(host="http://localhost:6666") +client = TuringDB(host="localhost", port=6666) # Create a graph and a versioned change client.create_graph("mygraph") @@ -81,6 +93,19 @@ client.checkout() client.query("MATCH (n) RETURN n") ``` +```js JavaScript SDK +import { TuringClient } from "turingdb"; + +// Connect to a running TuringDB server (started with -p 6686) +const client = new TuringClient({ url: "http://localhost:6686/query", graph: "mygraph" }); + +// Query the graph, with a parameter +const [[names]] = await client.queryData( + "MATCH (n:Person) WHERE n.name STARTS WITH $prefix RETURN n.name", + { parameters: { prefix: "J" } }, +); +``` + ```bash Docker # Run TuringDB in a container # (other install methods are preferred — Docker adds some performance loss) @@ -93,4 +118,4 @@ Once the server is running, open the built-in visualizer with `turingdb -ui` and ## SDKs -TuringDB ships a [Python SDK](/pythonsdk/reference) for creating graphs, managing changes, and running Cypher queries from your applications and notebooks. Using Claude Code? Install the [TuringDB skill](/claude-skill) and let Claude drive the database for you. +TuringDB ships a [Python SDK](/pythonsdk/reference) for creating graphs, managing changes, and running Cypher queries from your applications and notebooks, and a [JavaScript SDK](/jssdk/get_started), the `turingdb` npm package, for Node.js and web applications in JavaScript or TypeScript. Both talk to the server over TuringDB's binary protocol. Using Claude Code? Install the [TuringDB skill](/claude-skill) and let Claude drive the database for you. diff --git a/jssdk/get_started.mdx b/jssdk/get_started.mdx new file mode 100644 index 0000000..b72b593 --- /dev/null +++ b/jssdk/get_started.mdx @@ -0,0 +1,183 @@ +--- +title: "Get Started" +sidebarTitle: "Get Started" +description: "Query TuringDB from Node.js or the browser with the turingdb npm package, in JavaScript or TypeScript." +--- + +# QuickStart: TuringDB JavaScript SDK + +The `turingdb` npm package is the JavaScript client for TuringDB. It sends Cypher over TuringDB's binary protocol and decodes the results with a WebAssembly build of the engine's own decoder. It runs in Node.js 18 or later and in the browser, has no dependencies, and ships with TypeScript declarations. + +## 1. Install the package + +```bash +npm install turingdb +``` + +## 2. Start a server + +The client queries a running TuringDB server (see [CLI commands](/commands)). Node.js refuses to connect to port 6666, the server's default, because `fetch` blocks it as an unsafe port. For Node.js, start the server on another port: + +```bash +turingdb -demon -p 6686 +``` + +A web page reaches the server through its own origin instead, so in the browser the server keeps its default port. See [In the browser](#in-the-browser). + +## 3. Connect and create a graph + +```js +import { TuringClient } from "turingdb"; + +const client = new TuringClient({ url: "http://localhost:6686/query" }); + +await client.query("CREATE GRAPH people"); +client.setGraph("people"); +``` + +Save the examples on this page as `app.mjs` and run them with `node app.mjs`. Creating a client opens no connection, and each query is a separate request to `url`. + +## 4. Write in a change + +Writes run inside a change, as with the Python SDK (see [Writing data](/query/writing)). `CHANGE NEW` returns the ID of the new change: + +```js +const { chunks } = await client.query("CHANGE NEW"); +const change = chunks[0].columns[0].get(0); // the change ID, a bigint +client.setChange(change); + +await client.query("CREATE (:Person {name: $name, age: $age})", { + parameters: { name: "Marie", age: 33 }, +}); +await client.query("CREATE (:Person {name: 'Jane'})-[:KNOWS]->(:Person {name: 'John'})"); + +await client.query("CHANGE SUBMIT"); +client.setChange("main"); // back to main +``` + +`$name` and `$age` are [query parameters](/query/cypher_subset#query-parameters): the values travel next to the query, so they need no quoting or escaping. + +## 5. Query the graph + +`queryData` returns plain arrays, nested as chunks, then columns, then rows: + +```js +const [[names, ages]] = await client.queryData( + "MATCH (p:Person) RETURN p.name, p.age ORDER BY p.name", +); + +console.log(names); // [ 'Jane', 'John', 'Marie' ] +console.log(ages); // [ null, null, 33 ] +``` + +Results are column-oriented: `names` holds the first column for every row. A result streams in chunks of up to 65,536 rows, so loop over the chunks when a query can return more: + +```js +for (const [names, ages] of await client.queryData("MATCH (p:Person) RETURN p.name, p.age")) { + // one chunk: names[i] and ages[i] belong to the same row +} +``` + +`queryData` converts 64-bit integers to `number`. Use `query` to read typed columns, with 64-bit integers and node IDs as `bigint` (see the [Reference](/jssdk/reference#results)). + +## TypeScript + +The declarations ship with the package, so no `@types` package is needed: + +```ts +import { TuringClient, TuringQueryError } from "turingdb"; + +const client = new TuringClient({ url: "http://localhost:6686/query", graph: "people" }); + +async function namesOlderThan(age: number): Promise { + const chunks = await client.queryData( + "MATCH (p:Person) WHERE p.age > $age RETURN p.name ORDER BY p.name", + { parameters: { age } }, + ); + return chunks.flatMap(([names]) => names as string[]); +} + +try { + console.log(await namesOlderThan(30)); // [ 'Marie' ] +} catch (e) { + if (e instanceof TuringQueryError) { + console.error(e.status, e.message); // for example PARSE_ERROR and where the query failed + } else { + throw e; + } +} +``` + +## In the browser + +The same package works in a web page. Two things differ from Node.js: + +- **Same origin.** The server sends no CORS headers, so the browser refuses a request from a page on another origin (`TypeError: Failed to fetch`). Serve `/query` from the page's own origin and let it forward to the TuringDB server. The client's `url` defaults to `/query` for this. +- **Vite's dev server.** Exclude `turingdb` from dependency pre-bundling, so that the client still finds its `.wasm` decoder. + +With Vite, both fit in the config: + +```ts +// vite.config.ts +import { defineConfig } from "vite"; + +export default defineConfig({ + optimizeDeps: { + exclude: ["turingdb"], // keep the client next to its .wasm decoder + }, + server: { + proxy: { + "/query": "http://localhost:6666", // the TuringDB server + }, + }, +}); +``` + +```ts +// src/main.ts +import { TuringClient } from "turingdb"; + +const client = new TuringClient({ graph: "people" }); // url defaults to /query + +async function showPeople(): Promise { + const chunks = await client.queryData("MATCH (p:Person) RETURN p.name ORDER BY p.name"); + const names = chunks.flatMap(([column]) => column); + document.querySelector("#people")!.textContent = names.join(", "); +} + +showPeople(); +``` + +`vite build` copies the decoder into the build as an asset (`turing_wasm_decoder-.wasm`, about 440 kB, 140 kB compressed). Other bundlers that understand `new URL("…", import.meta.url)`, such as webpack 5, do the same. In production, serve the built app and forward `/query` from the same origin, for example with nginx: + +```nginx +server { + listen 80; + root /var/www/people-app; # the output of vite build + + location /query { + proxy_pass http://127.0.0.1:6666; + } +} +``` + +The browser only talks to the proxy, so the TuringDB server can stay on port 6666. + + + A token passed to `authToken` in a web page is visible to anyone who can open the page. For a server started with `-auth-on`, let the proxy add the token, and leave `authToken` unset in the browser: + + ```nginx + location /query { + proxy_pass http://127.0.0.1:6666; + proxy_set_header Authorization "Bearer "; + } + ``` + + The proxy then forwards every query the page sends, writes included, so put your application's own access control in front of it. + + +## What's next? + +- The [Reference](/jssdk/reference) covers every option, the result columns, value and parameter types, versioning and errors. +- Learn the [Query Language](/query/cypher_subset) and [query parameters](/query/cypher_subset#query-parameters). +- Explore [Versioning and changes](/concepts/versioning_system). diff --git a/jssdk/reference.mdx b/jssdk/reference.mdx new file mode 100644 index 0000000..cdf67fb --- /dev/null +++ b/jssdk/reference.mdx @@ -0,0 +1,250 @@ +--- +title: "Reference" +sidebarTitle: "Reference" +description: "Reference for the turingdb npm package: TuringClient options and methods, result columns, value and parameter types, versioning and errors." +--- + +# TuringDB JavaScript SDK + +The `turingdb` npm package runs in Node.js 18 or later and in the browser, and ships with TypeScript declarations. See [Get Started](/jssdk/get_started) for a first walkthrough, including the setup for a web page. + +```bash +npm install turingdb +``` + +Install the package version that matches your server: `turingdb@3.1` for a TuringDB 3.1 server. + +## TuringClient + +```js +import { TuringClient } from "turingdb"; + +const client = new TuringClient({ + url: "http://localhost:6686/query", + graph: "people", +}); +``` + +| Option | Description | Default | +| ------ | ----------- | ------- | +| `url` | The server's query endpoint, `http://:/query`, or a path on the page's own origin in a browser | `/query` | +| `graph` | The graph to query | `default` | +| `authToken` | The token of a server started with `-auth-on`. It is sent as `Authorization: Bearer ` | none | +| `bufferCapacity` | The starting size of the decoder buffer, in bytes. It grows to fit a larger result | 1 MiB | +| `fetch` | A `fetch` implementation to use instead of the global one | the global `fetch` | +| `wasmUrl` | Where to load the decoder, `turing_wasm_decoder.wasm`, from | the copy next to the client | + +Creating a client opens no connection: each query is a separate request to `url`. The decoder is compiled on the first query and shared by every client in the process or the page. + +### Methods + +| Method | Description | +| ------ | ----------- | +| `query(cypher, options?)` | Runs a query. Resolves to `{ chunks, execTimeMs }`, see [Results](#results) | +| `queryData(cypher, options?)` | Runs a query. Resolves to plain arrays, nested as chunks, then columns, then rows, with 64-bit integers as `number` | +| `setGraph(name)` / `getGraph()` | Set or read the graph | +| `setChange(change)` | Work on a change, by the ID `CHANGE NEW` returns, or `"main"` to return to main | +| `setCommit(commit)` | Read a past commit by its hash, or `"head"` for the latest one | +| `setAuthToken(token)` | Set the token, or `null` to stop sending one | + +Each query takes these options: + +| Option | Description | +| ------ | ----------- | +| `parameters` | Values for the query's `$name` [parameters](#parameters) | +| `graph`, `change`, `commit` | The graph, change or commit for this query only, in place of the client's | +| `signal` | An `AbortSignal` that cancels the query | + +```js +const controller = new AbortController(); +setTimeout(() => controller.abort(), 5000); + +// rejects with an AbortError if the query runs for more than 5 seconds +await client.query("MATCH (n) RETURN count(n)", { signal: controller.signal }); +``` + +## Results + +`query()` resolves to the result's chunks and the server-side execution time: + +```ts +{ + chunks: { names: string[]; columns: Column[] }[]; + execTimeMs: number | null; +} +``` + +A result streams in chunks of up to 65,536 rows, and each chunk holds every column of its rows. A result with no rows is one chunk of empty columns, so `RETURN a, b` matching nothing gives `[[[], []]]` from `queryData()`. + +```js +const { chunks } = await client.query("MATCH (p:Person) RETURN p.name AS name, p.age AS age"); + +for (const { names, columns } of chunks) { + const [name, age] = columns; + for (let row = 0; row < name.length; row++) { + console.log(name.get(row), age.get(row)); // 'Marie' 33n + } +} +``` + +### Column + +| Member | Description | +| ------ | ----------- | +| `get(row)` | One value, `undefined` past the last row | +| `toArray()` | Every row | +| `toNumberArray()` | Every row, with `bigint` values converted to `number`, inside lists, maps and paths too. Integers above 2^53 lose precision | +| `values` | A column of fixed-width values as a typed array (`BigInt64Array`, `Float64Array`, ...), one element per row, with no per-row work. `null` for strings, lists, maps, points and constant columns | +| `validity` | For a column with nulls, one bit per row, lowest bit first, set where the row holds a value. `null` when every row holds one | +| `name`, `length` | The column's name and number of rows | +| `typeCode`, `encoding`, `isConstant`, `isOptional` | How the column is stored. Compare `typeCode` with the exported `ColumnType` and `encoding` with `ColumnEncoding` | + +`values` reads a large numeric column without building a JavaScript value per row. A row that holds null reads as `0` there, so check `validity`: + +```js +const { chunks } = await client.query("MATCH (p:Person) RETURN p.age ORDER BY p.age"); +const ages = chunks[0].columns[0]; + +ages.values; // BigInt64Array [ 33n, 0n, 0n ] +ages.validity; // Uint8Array [ 1 ]: only row 0 holds a value +ages.toArray(); // [ 33n, null, null ] +``` + +### Value types + +| Value | `get()`, `toArray()` | `queryData()`, `toNumberArray()` | +| ----- | -------------------- | -------------------------------- | +| Integer, `count()`, node and edge IDs (`RETURN n`) | `bigint` | `number` | +| Double | `number` | `number` | +| String | `string` | `string` | +| Boolean | `boolean` | `boolean` | +| `null`, and an unmatched `OPTIONAL MATCH` | `null` | `null` | +| List | array | array | +| Map, `properties(n)`, a [map projection](/query/functions#map-projection) | plain object | plain object | +| Path (`RETURN p`) | array of IDs, node, edge, node, ... | the same, as numbers | +| Embedding | `Float32Array` | `Float32Array` | +| DateTime | `bigint`: microseconds since 1970-01-01T00:00:00Z | `number` | +| Duration | `bigint`: microseconds | `number` | +| Point | `Point`, with `x`, `y`, `z` (`undefined` in 2D), `srid` and `crs` | `Point` | + +To turn a DateTime into a JavaScript `Date`, which holds milliseconds: + +```js +const { chunks } = await client.query("RETURN datetime('2024-03-15T10:30:00Z') AS d"); +const micros = chunks[0].columns[0].get(0); // 1710498600000000n +new Date(Number(micros / 1000n)); // 2024-03-15T10:30:00.000Z +``` + +## Parameters + +`query(cypher, { parameters })` binds each `$name` in the query to `parameters.name`. A parameter can stand for a value anywhere an expression goes, or for a label or an edge type (see [Query parameters](/query/cypher_subset#query-parameters)): + +```js +await client.query("MATCH (p:Person) WHERE p.age >= $age RETURN p.name", { + parameters: { age: 30 }, +}); + +// a write, so inside a change (see Versioning below) +await client.query("UNWIND $rows AS row CREATE (:City {name: row.name, pop: row.pop})", { + parameters: { rows: [{ name: "Brest", pop: 139000 }, { name: "Rennes", pop: 220000 }] }, +}); +``` + +| JavaScript value | Sent as | +| ---------------- | ------- | +| A number with no fraction (`30`) | Integer | +| Any other number (`2.5`) | Double | +| `double(30)` | Double | +| `bigint` within the 64-bit range | Integer | +| `string`, `boolean`, `null` | String, Boolean, null | +| Array | List | +| Plain object | Map | + +`double()`, exported by the package, sends a whole number as a Double. It matters where the type changes the result, such as division: `RETURN $x / 4` is `2` for `{ x: 10 }` and `2.5` for `{ x: double(10) }`. + +Any other value, such as `undefined`, a `Date`, a `Map` or a class instance, is refused before the query is sent, with a `TuringQueryError` of status `PARAMETER_ERROR`: `Parameter $when is a Date, which a query parameter cannot hold`. Send a date as an ISO 8601 string and convert it in the query: + +```js +await client.query("RETURN datetime($when) AS d", { + parameters: { when: new Date().toISOString() }, +}); +``` + +Parameters count towards the 1 MiB limit of a request, together with the query text. + +## Versioning + +Writes run inside a change: + +```js +const { chunks } = await client.query("CHANGE NEW"); +client.setChange(chunks[0].columns[0].get(0)); + +await client.query("CREATE (:Person {name: 'Zoe'})"); +await client.query("CHANGE SUBMIT"); +client.setChange("main"); +``` + +To read a past commit, load it with `LOAD COMMIT` and then pass its hash: + +```js +const [[commits]] = await client.queryData("CALL db.history() YIELD `commit` RETURN `commit`"); +const previous = commits[1]; // newest first: commits[0] is the latest one, marked (HEAD) + +await client.query(`LOAD COMMIT '${previous}'`); +await client.queryData("MATCH (p:Person) RETURN count(p)", { commit: previous }); +``` + +- Unlike the Python SDK's `checkout()`, the client does not load a past commit for you. Without `LOAD COMMIT`, a query on a commit that is not in memory fails with `COMMIT_NOT_LOADED`. `LOAD COMMIT` on a loaded commit does nothing. +- `db.history()` marks the latest commit with a `(HEAD)` suffix. Strip it before passing that hash. +- `setChange("main")` and `setCommit("head")` return the client to the latest state of main. See [Time travel](/graph_dev/time_travelling) for the versioning model. + +## Errors + +A failed query rejects with a `TuringQueryError`, which carries a `status` and the server's `message`: + +```js +import { TuringQueryError } from "turingdb"; + +try { + await client.query("MATCH (n RETURN n"); +} catch (e) { + if (e instanceof TuringQueryError) { + console.log(e.status); // PARSE_ERROR + console.log(e.message); // the query, a marker under the error, and "syntax error, unexpected RETURN" + } +} +``` + +| `status` | Meaning | +| -------- | ------- | +| `PARSE_ERROR`, `ANALYZE_ERROR`, `PLAN_ERROR`, `EXEC_ERROR` | The query failed. The message shows where and why | +| `GRAPH_NOT_FOUND` | No graph has that name, or it is on disk but not loaded: run `LOAD GRAPH ` | +| `CHANGE_NOT_FOUND`, `COMMIT_NOT_FOUND` | No change has that ID, or no commit has that hash | +| `COMMIT_NOT_LOADED` | The commit is not in memory: run `LOAD COMMIT ''` first | +| `PARAMETER_ERROR` | A parameter value the client cannot send. Nothing was sent | +| `PROTOCOL_ERROR` | The server refused the request, with a message that says why: a missing or wrong token (`Unauthorized: missing or invalid authentication token`), or a request over 1 MiB. A server of another version also shows up here | +| `HTTP_ERROR` | An error status with no TuringDB message, typically from a proxy: `Server returned HTTP 502` when the TuringDB server behind it is down | +| `DECODE_ERROR` | The client could not decode the result | + +A failure to reach the server rejects with the `TypeError` that `fetch` throws instead: `fetch failed` in Node.js when no server is listening, and `Failed to fetch` in a browser, which also covers a request blocked because the page is on another origin. A cancelled query rejects with an `AbortError`. + +## The wasm decoder + +The client loads its decoder with `new URL("turing_wasm_decoder.wasm", import.meta.url)`, the file next to it in the package. Node.js, browsers and bundlers that understand that pattern find it with no configuration. With Vite's dev server, exclude the package from pre-bundling (`optimizeDeps: { exclude: ["turingdb"] }`, see [In the browser](/jssdk/get_started#in-the-browser)). + +To serve the file from somewhere else, such as a CDN, pass its address: + +```js +const client = new TuringClient({ + url: "/query", + wasmUrl: "https://cdn.example.com/turing_wasm_decoder.wasm", +}); +``` + +Serve the file as `application/wasm`, which most web servers already do. + +## Links + +- The package on npm: [npmjs.com/package/turingdb](https://www.npmjs.com/package/turingdb) +- Its source, in the TuringDB repository: [github.com/turing-db/turingdb/tree/main/wasm/npm](https://github.com/turing-db/turingdb/tree/main/wasm/npm) diff --git a/public/images/icons/pixel-bolt.png b/public/images/icons/pixel-bolt.png new file mode 100644 index 0000000000000000000000000000000000000000..9b0d403995124ad9e21fc777e7b3a803644e1524 GIT binary patch literal 281 zcmeAS@N?(olHy`uVBq!ia0vp^4j|0I1|(Ny7TyC=4?JBQLn>~)z3I(+$U(yO;j{!6 z-UUpoIvjox4_h{@US?D52&ntLnn z`_?^P^p%Yvfmy0Jj+udB!5g+IWgv#ad$udj_5f{jxW&8TZY3`RL&FW;jxsx|cPyF= z2U4c}fA;5T-Bq9~JSzAi?(!{^-VmSfJhwjQ+&^xho`b&|t~~G9c`h5M;K|*FTXU|1 zRPJWnYQ3J3fk9w5<9)07w+kVL{&-T8+Xqs3=5|B)^ZK(OVFtm8+bfygt?}EC>m^_a P`njxgN@xNAC4FhP literal 0 HcmV?d00001 diff --git a/public/images/icons/pixel-braces.png b/public/images/icons/pixel-braces.png new file mode 100644 index 0000000000000000000000000000000000000000..5a9351298f0fc19e01638afa3b9fd3addd1796e7 GIT binary patch literal 294 zcmeAS@N?(olHy`uVBq!ia0vp^4j|0I1|(Ny7TyC=Z#`WcLn>~)y?v0k$w9y^@%n`h z(+ts&2M2F-s#_b%tvR5zh0Xh*aohJX=c%OJ*z;eneRnq_c+SA_`zuWty%!g-%GO_noSaqO2)o!;sh~@CcHsO_}wfV)5oeYe!i{3+*MT^bsWaXvTxef#dMGw$CtT^bl{6{eq`XYl<$)5Tv( e4h#$?XQa>cg}Q5Jzi0+}oWax8&t;ucLK6V1CU?64 literal 0 HcmV?d00001 diff --git a/public/images/icons/pixel-dollar.png b/public/images/icons/pixel-dollar.png new file mode 100644 index 0000000000000000000000000000000000000000..26a4f68416d66339d58cbc260d058dee10617a7f GIT binary patch literal 314 zcmeAS@N?(olHy`uVBq!ia0vp^4j|0I1|(Ny7T#lEU}W@kaSW-r_4f8g-a`f=Er~sw zIE)X7_&7A)XzB75U==oCJ##=~;W5@}qQ?zCmB)%`cGm6P{bR31Vrb;^k9Pm8>N(p3 zK385_eU8=PJ%dwj|MY8D&Ll80q#UTP+*hKufq`Le_Ke$~w_kt!`p4egZJK)zKC6@p z1zN8&opEQE?fqL{BN!MY4c14!77YQ4PMOPoB>VH@vV6G%ObjJc7}ut|XbFQ~)y|Iz^kbwZ}1>;j5 zl5MFgZm29e!E&=nQbKjh38w~CYokxUOS^u&T=d(orK4v~$<6D3FEd{+@%vwMApf~_ zuN@kn`D%>co7#+{eCut&#)klf5lzysQ0zPP=PJ48JF%=@PEA*riDRp bVn;^>lmABNZK8pq>L6vFu6{1-oD!M~)y}gn5kbwZ}1(`)l zlNrkrB(%&HCJDP7WSPm7mcVGmW>BgzVZu|1H{VZ}RPM5l7fO|IS@QhI?^fqsOQz-j zkl20FR=;XJ&^REtP&{qt^B3!%-nMn)W-ws>^vUD=s`GrWBU(Z30D)I;7?+-Zd8xWh zj3I&T%->uM3!N}S=)qn#c`|*3<2eASN`QKV1h_A6jeLCtF9MiU^q~J sw&B`8^$Sc4XBslkZ4U%0RZzO*9_JM$-?aGEokJk?p00i_>zopr0N77p;s5{u literal 0 HcmV?d00001 diff --git a/public/images/icons/pixel-pin.png b/public/images/icons/pixel-pin.png new file mode 100644 index 0000000000000000000000000000000000000000..ec4b90db80f5c462a721056340565fa633304b5e GIT binary patch literal 277 zcmeAS@N?(olHy`uVBq!ia0vp^4j|0I1|(Ny7TyC=cRXDjLn>~)y|t0|kby``BKt-T z>kX{j0UMcw0}L6p6}ZG2G!xj|I89jtKFpi^;JuZ0P|n|7yCcfH6cbk$+`uyK# zSl{n1eK9M2tu+IKtih)<>E%rv3=5c-P5t@8Pk<3<2@ptK+w-mE_NHGy{sidBK!g|= z>UZ(4_&fJGj9an#V!Z literal 0 HcmV?d00001 diff --git a/pythonsdk/get_started.mdx b/pythonsdk/get_started.mdx index 2268be1..3880a21 100644 --- a/pythonsdk/get_started.mdx +++ b/pythonsdk/get_started.mdx @@ -27,7 +27,20 @@ source .venv/bin/activate pip install turingdb ``` -## 2. Create a Graph: +## 2. Connect + +Start a server with `turingdb -demon` (see [CLI commands](/commands)), then connect to it: + +```python +from turingdb import TuringDB + +# Connect to the server running on localhost:6666 +client = TuringDB(host="localhost", port=6666) +``` + +The client connects right away, and raises `Failed to connect to localhost:6666: Connection refused` if no server is running. To run the engine inside your Python process instead, with no server, use `TuringDB(type="embedded", data_dir="/home/me/.turing")` (see [client backends](/pythonsdk/reference#client-backends)). + +## 3. Create a Graph: Create a graph and set active graph: @@ -39,7 +52,7 @@ client.create_graph("my_first_graph") client.set_graph("my_first_graph") ``` -## 3. Create Nodes & Edges +## 4. Create Nodes & Edges First of all, create a new change on the graph @@ -59,6 +72,9 @@ client.query("CREATE (:Person {name: 'Marie', age: 33})") # Create two nodes and an edge client.query("CREATE (:Person {name: 'Jane'})-[:KNOWS]->(:Person {name: 'John'})") + +# Pass values as parameters: no quoting or escaping needed +client.query("CREATE (:Person {name: $name, age: $age})", {"name": "O'Brien", "age": 41}) ``` > ⚠️ All created entities must have at least one label (:Label) @@ -76,7 +92,7 @@ client.checkout() > You can create as many commits as you want before submitting a change -## 4. Query the Graph +## 5. Query the Graph Retrieve data using Cypher `MATCH` queries. @@ -89,7 +105,7 @@ df = client.query("MATCH (n:Person) RETURN n.name, n.age") print(df) ``` -## 5. Version Control & Snapshots +## 6. Version Control & Snapshots TuringDB supports versioned graphs and snapshot isolation. @@ -126,8 +142,8 @@ client.checkout() from turingdb import TuringDB # Create TuringDB client -# Connect to a locally running TuringDB server (default host shown): -client = TuringDB(host="http://localhost:6666") +# Connect to a locally running TuringDB server (default host and port shown): +client = TuringDB(host="localhost", port=6666) # Or run the engine in-process, with no server: # client = TuringDB(type="embedded", data_dir="/home/me/.turing") # use an absolute path @@ -158,5 +174,6 @@ print(df) - 📘 Learn the [Query Language](/query/cypher_subset) - 🔀 Explore [Versioning & Changes](/concepts/versioning_system) - 🔎 Try [Vector Search](/vector-search) +- 🌐 Query TuringDB from JavaScript or TypeScript with the [JavaScript SDK](/jssdk/get_started) Questions or feedback? → Join the [Discord](https://discord.gg/dMM48ns5VY) or [open an issue](https://github.com/turingdb) \ No newline at end of file diff --git a/pythonsdk/reference.mdx b/pythonsdk/reference.mdx index c56c13d..5feb445 100644 --- a/pythonsdk/reference.mdx +++ b/pythonsdk/reference.mdx @@ -22,88 +22,62 @@ or using `pip`: pip install turingdb ``` -The SDK targets TuringDB v3: pin `turingdb>=3.0.2` in your project (older projects may pin `1.x`, which uses the previous engine and a much smaller Cypher subset). Upgrade the SDK and the server together: from 3.0.2 the JSON client sends each query as a JSON body, which is what a 3.0.2 server expects. The wheel supports Python 3.10 to 3.14 on Linux (x86_64, aarch64) and macOS 15 or later (arm64). It also contains the `turingdb` command-line tool, the `turing-parquet` import tool, and the in-process engine. +The SDK targets TuringDB v3: pin `turingdb>=3.1` in your project (older projects may pin `1.x`, which uses the previous engine and a much smaller Cypher subset). Upgrade the SDK and the server together: a 3.1 SDK speaks the binary protocol of a 3.1 server. The wheel supports Python 3.10 to 3.14 on Linux (x86_64, aarch64) and macOS 15 or later (arm64). It also contains the `turingdb` command-line tool, the `turing-parquet` import tool, and the in-process engine. ## Getting Started ```python from turingdb import TuringDB -# Create TuringDB client -# set host parameter to the URL (as string) on which TuringDB is running, -# default "http://localhost:6666" -client = TuringDB(host="http://localhost:6666") +# Connect to a TuringDB server running on this machine (localhost:6666) +client = TuringDB() -# List available graphs +# or name the host and the port +client = TuringDB(host="localhost", port=6666) + +# List the graphs on disk print(client.list_available_graphs()) ``` ## Client backends -`TuringDB(...)` is a facade over three interchangeable backends, chosen with the `type` argument (or the `TURINGDB_TYPE` environment variable, default `"json"`): +`TuringDB(...)` is a facade over two interchangeable backends, chosen with the `type` argument: -| `type` | Encoding / transport | Constructor arguments | -| ------ | -------------------- | --------------------- | -| `"json"` *(default)* | JSON over HTTP against a running server | `host="http://localhost:6666"`, optional `token` | -| `"native"` | TuringDB's binary protocol against a running server. Faster for large results | `host="localhost"`, `port=6666`, optional `token` | -| `"embedded"` | In-process engine, no server or socket | `data_dir="/path/to/.turing"` | +| `type` | Runs against | Constructor arguments | +| ------ | ------------ | --------------------- | +| `"native"` *(default)* | A running TuringDB server, over TuringDB's binary protocol | `host="localhost"`, `port=6666`, optional `token` | +| `"embedded"` | An in-process engine, with no server or socket | `data_dir="/path/to/.turing"` | ```python -# Default HTTP/JSON client -client = TuringDB(host="http://localhost:6666") - -# Native binary protocol -client = TuringDB(type="native", host="localhost", port=6666) +# A running server (the default) +client = TuringDB(host="localhost", port=6666) # Embedded, in-process: no server needed client = TuringDB(type="embedded", data_dir="/home/me/.turing") ``` -Use the default `json` backend unless you have a reason not to. The other two are optimisations for specific situations: - -- **`native`** when you control the server and need more throughput on large results. -- **`embedded`** when you want a self-contained engine with no server to start, stop or connect to. - -### The native binary protocol - -The binary protocol is a **server-side mode**, selected once when the server starts with the environment variable `USE_TURING_PROTO=1`: - -```bash -USE_TURING_PROTO=1 turingdb start -demon -p 6666 -``` - -```python -client = TuringDB(type="native", host="localhost", port=6666) -``` - -A server speaks either JSON or the binary protocol on a given port: +Use a server unless you have a reason not to. Choose **`embedded`** when you want a self-contained engine with no server to start, stop or connect to. -| Server started | `type="json"` client | `type="native"` client | -| -------------- | -------------------- | ---------------------- | -| normally (JSON, the default) | works | returns `Proto header dataLen does not match chunk payload size` | -| with `USE_TURING_PROTO=1` | returns `Invalid response from the server` | works | +### Connecting to a server -A server in binary mode serves the binary protocol only, so `curl`, the JSON client and the browser visualizer (`-ui`) connect to a server running in the default mode. To use both, run two servers on different ports. - -- `host` accepts `"localhost"` together with `port=`, or a full URL such as `"http://localhost:6666"`. Use one of these two forms. -- The query API is identical, and results come back as the same DataFrames. Embeddings are returned as numpy `float32` arrays instead of Python lists. -- A [`shortestPath`](/shortest-path) `path` cannot be read over the native backend yet. Run those queries through the JSON backend. -- To list the graphs on disk, use the JSON client. On the native backend, `list_loaded_graphs()` lists the loaded graphs. -- `token`, `reconnect()` and the S3 helpers work as on the JSON backend. +- `host` is a host name or an IP address, and `port` a number (default `6666`). A URL such as `"http://localhost:6666"` is refused: `host must be a host name or IP address, not a URL`. +- The client connects as soon as it is created. If no server is listening it raises `TuringDBException: Failed to connect to localhost:6666: Connection refused`. +- The client sends the graph, change and commit it is set to with every query, so the server keeps no session for it. +- The query API is identical on both backends, and results come back as the same DataFrames. ### Authentication -A server started with `-auth-on` requires a bearer token. Pass it with `token=` or set the `TURINGDB_AUTH_TOKEN` environment variable, which the SDK reads automatically. This works on both the `json` and `native` backends: +A server started with `-auth-on` requires a bearer token. Pass it with `token=` or set the `TURINGDB_AUTH_TOKEN` environment variable, which the SDK reads automatically: ```bash TURINGDB_AUTH_TOKEN=s3cret turingdb start -demon -auth-on ``` ```python -client = TuringDB(host="http://localhost:6666", token="s3cret") +client = TuringDB(host="localhost", port=6666, token="s3cret") ``` -A missing or incorrect token returns HTTP `401`, which the JSON client raises as an `httpx.HTTPStatusError`. The embedded backend runs in-process and needs no authentication, so it does not take a `token`. +A missing or incorrect token raises `TuringDBException: Protocol error from server: Unauthorized: missing or invalid authentication token`. The embedded backend runs in-process and needs no authentication, so it does not take a `token`. ### The embedded backend @@ -116,17 +90,13 @@ client = TuringDB(type="embedded", data_dir=os.path.expanduser("~/.turing")) - `data_dir` is the TuringDB root directory (the one that holds `graphs/`, `data/` and `vector/`). Omit it to use `~/.turing`. - **Use an absolute path**, for example with `os.path.expanduser("~/...")`, since a leading `~` is not expanded. - Writes still persist to disk when you submit a change, so your work survives the process ending. -- The embedded backend is designed for a single process. For the browser visualizer, `list_available_graphs()`, concurrent access from several processes, the S3 helpers and `token=`, use a server. - - - `list_available_graphs()` is only supported on the default JSON backend. The S3 helpers (`s3_connect`, `transfer`) work on the JSON and native backends. - +- The embedded backend is designed for a single process. For the browser visualizer, concurrent access from several processes, the S3 helpers and `token=`, use a server. ## Core Methods ### `list_available_graphs() → list[str]` -Returns the graphs persisted on disk in the TuringDB data directory (JSON backend only). +Returns the graphs on disk in the TuringDB data directory, and the graphs in memory (`LIST AVAILABLE GRAPHS`). ```python client.list_available_graphs() @@ -148,7 +118,7 @@ Creates a new empty graph with the specified name. client.create_graph("mygraph") ``` -> Internally executes `CREATE GRAPH "graph_name"` +> Internally executes `CREATE GRAPH graph_name` ### `load_graph(graph_name: str, raise_if_loaded: bool = True)` @@ -166,14 +136,25 @@ Runs a Cypher query on the current graph. ```python df = client.query("MATCH (n:Person) RETURN n.name, n.age") +df = client.query("MATCH (n:Person) WHERE n.age >= $min RETURN n.name", {"min": 30}) df = client.query("MATCH (n:$label) RETURN n.name", {"label": "Person"}) ``` -Returns results as a `pandas.DataFrame`, with automatic typing and parsing. `query_parameters` fills `$name` [parameters](/query/cypher_subset#query-parameters), which stand for labels and edge types. +Returns results as a `pandas.DataFrame`, with automatic typing and parsing. `query_parameters` fills the query's `$name` [parameters](/query/cypher_subset#query-parameters), which stand for values, labels and edge types. Parameter values can be: + +| Python value | Cypher value | +| ------------ | ------------ | +| `str`, `int`, `float`, `bool`, `None` | String, Integer (64-bit), Double, Boolean, null | +| `list`, `tuple` | List | +| `dict` with string keys | Map | +| `datetime.datetime` with a time zone | DateTime | +| `datetime.timedelta` | Duration | + +Other values are refused before the query is sent, with a `TuringDBException` that names the type: a naive `datetime` (`A datetime parameter must carry a timezone: a naive datetime names no instant`), a `datetime.date`, `bytes`, and numpy scalars and arrays (`A parameter cannot hold a Python int64`). Convert numpy values with `.tolist()` or `int()` first. ### `query_raw(query: str, query_parameters: dict | None = None) → dict` -Like `query`, but returns the raw server response as a `dict` instead of a DataFrame. Useful for inspecting result metadata or column dtypes directly. +Like `query`, but returns the columns without building a DataFrame: `{"data": {column: values}, "dtypes": {column: type name}, "time": milliseconds}`. Each `values` is a numpy array, or a list for strings, nullable columns, lists and maps. ### `get_graph() → str` / `set_graph(graph_name: str)` @@ -205,7 +186,7 @@ Use the following helpers to navigate graph history or work in isolated changes: ### `checkout(change: int | str = "main", commit: str = "HEAD")` -Switches the working context to a specific change or commit. +Switches the working context to a specific change or commit. Checking out a past commit loads it into memory first (`LOAD COMMIT`). ```python client.checkout(change=2, commit="abc123") @@ -217,7 +198,7 @@ client.checkout("main") # Reset to default ### `set_commit(commit: str)` -Manually set the commit hash to use. +Manually set the commit hash to use. Unlike `checkout()`, it does not load the commit, so run `LOAD COMMIT ''` first if it is not in memory. ```python client.set_commit("haax42") @@ -245,11 +226,16 @@ You can also read the current context through the `current_graph`, `current_comm ### `reconnect()` -Resets the connection. A no-op for the HTTP and embedded backends. Reopens the socket for the native backend. +Closes the connection to the server and opens a new one. The client returns to the `default` graph, main and the latest commit, so set the graph again afterwards. A no-op for the embedded backend. -### `try_reach(timeout: int = 5)` / `warmup(timeout: int = 5)` +```python +client.reconnect() +client.set_graph("mygraph") +``` -Probe connectivity (`try_reach`) or warm the connection with a lightweight query (`warmup`). +### `try_reach(timeout: float = 5)` / `warmup(timeout: float = 5)` + +Probe connectivity (`try_reach`) or warm the connection with a lightweight query (`warmup`). Both raise `TuringDBException` when the server does not answer within `timeout` seconds. ### `get_query_exec_time() → float | None` / `get_total_exec_time() → float | None` @@ -257,7 +243,7 @@ Return the server-side execution time and the full round-trip time (in milliseco ## S3 / Data Transfer -These helpers move graph data between local paths, S3, and the TuringDB server. They are available on the JSON and native backends, but not on the embedded backend. +These helpers move graph data between local paths, S3, and the TuringDB server. They need a server, so they are not available on the embedded backend. ### `s3_connect(bucket_name, access_key=None, secret_key=None, region=None, use_scratch=True)` @@ -292,17 +278,20 @@ All queries return a `pandas.DataFrame`, typed according to the schema: | `DateTime` | `datetime64[us, UTC]` | | `Duration` | `timedelta64[us]` | | `List`, `Map` | object (Python list, dict) | -| `Embedding` | object (list over HTTP, numpy `float32` array on the native and embedded backends) | +| `Embedding` | object (numpy `float32` array) | +| `Point` | object (`turingdb.Point`) | + +Nodes and edges (`RETURN n`) come back as their internal integer IDs (`UInt64`), so project the properties you need, or return `properties(n)` as a dictionary. A named path comes back as a list of IDs, node and edge alternating: `[0, 0, 1]` is node 0, edge 0, node 1. A `DateTime` or `Duration` inside a list or a map comes back as a time-zone-aware `datetime.datetime` or a `datetime.timedelta`. Doubles keep their full precision. See [Functions and data types](/query/functions#how-results-come-back) for the full table. -Nodes and edges (`RETURN n`) come back as their internal integer IDs (`UInt64`), so project the properties you need, or return `properties(n)` as a dictionary. A named path comes back as a list of `{'type': 'node' | 'edge', 'id': ...}` dictionaries. Doubles keep their full precision on every backend. See [Functions and data types](/query/functions#how-results-come-back) for the full table. +`turingdb.Point` is a named tuple `(srid, x, y, z)`, with `z` set to `None` for a 2D point, and a `crs` property such as `'wgs-84'`. - Integer columns hold numpy integers, which an f-string prints as `np.int64(5)`. Convert IDs with `.tolist()` or `int()` before writing them into a query. + Integer columns hold numpy integers, which an f-string prints as `np.int64(5)`. Convert them with `.tolist()` or `int()` before writing them into a query or passing them as parameters. ## Error Handling -Query errors raise `TuringDBException`. The message starts with the error code (`PARSE_ERROR`, `ANALYZE_ERROR`, `PLAN_ERROR`, `EXEC_ERROR`, `GRAPH_NOT_FOUND`, `CHANGE_NOT_FOUND`, `COMMIT_NOT_FOUND`, `REQUEST_TOO_BIG`, ...) and points at the offending part of the query, or names the graph, change or commit that was not found. +Query errors raise `TuringDBException`. The message starts with the error code (`PARSE_ERROR`, `ANALYZE_ERROR`, `PLAN_ERROR`, `EXEC_ERROR`, `GRAPH_NOT_FOUND`, `CHANGE_NOT_FOUND`, `COMMIT_NOT_FOUND`, `COMMIT_NOT_LOADED`, ...) and points at the offending part of the query, or names the graph, change or commit that was not found. ```python from turingdb import TuringDB, TuringDBException @@ -313,17 +302,19 @@ except TuringDBException as e: print("Query failed:", e) ``` -On the JSON backend, transport failures (the server is down, or HTTP 401 for a bad or missing token) surface as `httpx` exceptions instead. HTTP errors that the server explains, such as `REQUEST_TOO_BIG`, still raise `TuringDBException`. +Connection problems raise `TuringDBException` too: + +- `Failed to connect to localhost:6666: Connection refused` when no server is listening. +- `Protocol error from server: ...` when the server refuses a request: a missing or wrong token, or a request over the 1 MiB limit. The server then closes the connection, so call `client.reconnect()` and set the graph again before the next query. +- `The query parameters exceed 1048576 bytes` when the parameters alone are over the limit. Nothing is sent, and the connection stays open. ## Example Workflow ```python from turingdb import TuringDB -# Create TuringDB client -# set host parameter to the URL (as string) on which TuringDB is running, -# default "http://localhost:6666" -client = TuringDB(host="http://localhost:6666") +# Connect to the server running on localhost:6666 +client = TuringDB(host="localhost", port=6666) # Create a new graph called mygraph client.create_graph("mygraph") @@ -355,17 +346,16 @@ client.query("CHANGE SUBMIT") # Checkout into main client.checkout() -# Query graph -df = client.query("MATCH (n:Gene {name: 'APOE'}) RETURN n.name, n.chromosome") +# Query graph, with the gene name as a parameter +df = client.query("MATCH (n:Gene {name: $name}) RETURN n.name, n.chromosome", {"name": "APOE"}) print(df) ``` ## Notes -- The JSON backend (`type="json"`) uses `httpx`; the `native` and `embedded` backends use a compiled extension -- Only the graph name, change ID, and commit hash are sent with HTTP requests -- The SDK supports the full TuringDB **Cypher dialect**, including matching by internal node ID (`WHERE n = 1234`) and versioned queries against past commits +- Both backends use a compiled extension that ships in the wheel. +- The SDK supports the full TuringDB **Cypher dialect**, including matching by internal node ID (`WHERE n = 1234`) and versioned queries against past commits. Need help writing queries? → See the [TuringDB Query Cheatsheet](/query/cheatsheet) -You can also find the TuringDB PythonSDK in [Github](https://github.com/turing-db/turingdb-sdk-python) \ No newline at end of file +The SDK's source is in the TuringDB repository on [GitHub](https://github.com/turing-db/turingdb/tree/main/python/turingdb). To query TuringDB from JavaScript or TypeScript, see the [JavaScript SDK](/jssdk/get_started). diff --git a/query/aggregation.mdx b/query/aggregation.mdx index f702ead..4dc79c1 100644 --- a/query/aggregation.mdx +++ b/query/aggregation.mdx @@ -48,7 +48,7 @@ ORDER BY n.age DESC SKIP 1 LIMIT 2 - You can sort by several keys, by aliases and by expressions. Ascending is the default, and `ASCENDING` / `DESCENDING` are synonyms of `ASC` / `DESC`. - `SKIP` and `LIMIT` combine for pagination: `SKIP 10 LIMIT 10` returns the 11th to 20th rows. - `SKIP` and `LIMIT` take constant expressions. -- `RETURN DISTINCT ... ORDER BY` can only sort by returned columns. +- `RETURN DISTINCT ... ORDER BY` sorts by expressions over the returned columns, such as a property of a returned node: `RETURN DISTINCT n ORDER BY n.age`. - Nulls sort last in ascending order and first in descending order. ## WITH @@ -95,7 +95,7 @@ MATCH (c:Company) RETURN c.name AS name ``` -`UNION` removes duplicates and `UNION ALL` keeps them. All branches must return the same column names, and each column must keep one value type across the branches. +`UNION` removes duplicates and `UNION ALL` keeps them. All branches must return the same column names. A column can hold values of different types from different branches, as in `RETURN 1 AS x UNION RETURN 'a' AS x`. To run several queries in one request, or choose which query runs, see [NEXT and WHEN](/query/composition). @@ -145,15 +145,18 @@ RETURN percentileCont(n.age, q) AS firstQuartile UNWIND [[1, 2], [1, 3], [0, 9]] AS x RETURN min(x) AS lo, max(x) AS hi + +MATCH (a:Person)-[:WORKS_AT]->(c:Company) +RETURN c.name, collect({name: a.name, age: a.age}) AS staff ``` | Aggregate | Notes | | --------- | ----- | | `count(x)`, `count(*)`, `count(DISTINCT x)` | `count(x)` ignores nulls | -| `collect(x)`, `collect(DISTINCT x)` | Nulls are dropped | -| `sum(x)` | | -| `avg(x)` | Always a Double | -| `min(x)`, `max(x)` | Numbers, strings, booleans, `DateTime`, `Duration`, lists and values of mixed types | +| `collect(x)`, `collect(DISTINCT x)` | Any value, including maps, nodes and paths. Nulls are dropped | +| `sum(x)` | Numbers and `Duration` values | +| `avg(x)` | A Double for numbers, a `Duration` for durations, and the centroid for [points](/query/functions#points) | +| `min(x)`, `max(x)` | Numbers, strings, booleans, `DateTime`, `Duration`, points, lists and values of mixed types | | `stDev(x)`, `stDevP(x)` | Sample and population standard deviation | | `percentileCont(x, p)`, `percentileDisc(x, p)` | Interpolated and nearest-value percentile; `p` is a fraction, such as `0.5` for the median | @@ -201,9 +204,11 @@ RETURN t, count(*) `range(start, end[, step])` includes both bounds. Unwound nodes stay nodes, so `p.name` works after `UNWIND ns AS p`. You can also `UNWIND` a list of maps and read their keys: `UNWIND [{a: 1}, {a: 2}] AS m RETURN m.a`. - - Write the list into the query text, for example `f"UNWIND {ids} AS x ..."` in Python. Value parameters such as `UNWIND $list` are not yet supported. - +To unwind a list from your application, pass it as a [parameter](/query/cypher_subset#query-parameters): + +```python +client.query("UNWIND $names AS nm MATCH (n:Person {name: nm}) RETURN nm, n.age", {"names": ["Alice", "Bob"]}) +``` ## Lists @@ -218,9 +223,10 @@ RETURN n.tags[0], size(n.tags) RETURN [1, 2] + [3], [1, 2] + 3, 0 + [1, 2], 'a' || 'b' ``` -- Indexing and slicing work on literals, list properties and function results (`labels(n)[0]`). Negative indexes count from the end, and an out-of-range index returns `null`. -- Lists and maps compare by value with `=`. -- `size(null)` and `head([])` return `null`. +- Indexing and slicing work on literals, list properties, parameters and function results (`labels(n)[0]`). Negative indexes count from the end, and an out-of-range index returns `null`. +- Lists and maps compare by value with `=`. A `null` inside makes the result `null` when it decides the comparison: `[1, null] = [1, 2]` is `null`. +- `size(null)` and `head([])` return `null`, and so does a list plus `null`. +- `isEmpty(list)` and `coll.indexOf(list, value)` are in [List functions](/query/functions#lists). ### List comprehensions @@ -235,12 +241,15 @@ WHERE any(t IN n.tags WHERE t = 'vip') RETURN n.name RETURN reduce(total = 0, x IN [1, 2, 3] | total + x) + +RETURN allReduce(total = 0, x IN [1, 2, 3] | total + x, total < 10) ``` - `[x IN list WHERE condition | expression]` filters and maps a list. -- A **pattern comprehension** `[(a)-[:KNOWS]->(b) | b.name]` collects a value for every match of a pattern. +- A **pattern comprehension** `[(a)-[:KNOWS]->(b) | b.name]` collects a value for every match of a pattern. Name its path to use it: `[p = (a)-[:KNOWS]->+() | length(p)]`. - `all`, `any`, `none` and `single` test a condition over a list: `any(t IN n.tags WHERE t = 'vip')`. - `reduce(acc = init, x IN list | expression)` folds a list into a single value. +- `allReduce(acc = init, x IN list | expression, predicate)` folds a list the same way and is `true` when the predicate holds after every step, see [Filtering paths with `allReduce`](/query/paths#filtering-paths-with-allreduce). ## CASE diff --git a/query/cheatsheet.mdx b/query/cheatsheet.mdx index 1ce141e..bbdbae3 100644 --- a/query/cheatsheet.mdx +++ b/query/cheatsheet.mdx @@ -47,8 +47,8 @@ RETURN n.name - Node labels use `:Label` and can be stacked: `(n:Person:Employee)`. - Edge types use `:EDGE_TYPE`: `-[:KNOWS]->`. An edge has exactly one type, and `[:A|B]` matches either. - Label expressions: `(n:A|B)`, `(n:!A)`, `(n:A&B)`, also on edge types and in `WHERE`. -- Labels and edge types can be [parameters](/query/cypher_subset#query-parameters): `client.query("MATCH (n:$label) RETURN n.name", {"label": "Person"})`. -- `RETURN n` gives the internal node ID. Return properties explicitly (`RETURN n.name`), or all of them with `properties(n)`. +- Values, labels and edge types can be [parameters](/query/cypher_subset#query-parameters), see [Parameters](#parameters) below. +- `RETURN n` gives the internal node ID. Return properties explicitly (`RETURN n.name`), as a map projection (`RETURN n {.name, .age}`), or all of them with `properties(n)`. - Unknown labels and properties are forgiving: they match nothing or read as `null`. ## OPTIONAL MATCH @@ -75,8 +75,7 @@ WHERE n.age >= 18 AND n.city = 'London' RETURN n.name MATCH (n:Person) -WHERE n.name STARTS -WITH 'A' OR n.name CONTAINS 'ro' +WHERE n.name STARTS WITH 'A' OR n.name CONTAINS 'ro' RETURN n.name MATCH (n:Person) @@ -98,11 +97,14 @@ RETURN n.name, m.name MATCH (p:Person) WHERE NOT (p)-[:KNOWS]->() RETURN p.name + +MATCH (p:Person) +RETURN p.name, exists((p)-[:LIKES]->()) AS likesSomething ``` -- Comparison: `= <> != < <= > >=`, null tests `IS NULL` / `IS NOT NULL`, boolean `AND OR XOR NOT`. +- Comparison: `= <> != < <= > >=`, null tests `IS NULL` / `IS NOT NULL`, label tests `n:Person` / `n IS Person`, boolean `AND OR XOR NOT`. - Strings and lists: `IN`, `STARTS WITH`, `ENDS WITH`, `CONTAINS`, concatenation with `+` or `||`. -- Use `IS NOT NULL` to test for a property (`=~` and `exists(n.prop)` are not yet supported). +- Use `IS NOT NULL` to test for a property, and `exists()` for a pattern. `=~` is not yet supported. - Integers and Doubles compare by value (`n.age = 30.0`). Compare two Doubles with a range (`n.score > 1.99 AND n.score < 2.01`). ## Lookup by native node ID @@ -135,8 +137,23 @@ WHERE n = 0 RETURN n.name ``` -- `n = 0` and `id(n) = 0` both look the node up directly. -- For batches, `UNWIND` a list of IDs, from the query text, a variable, `collect()` or `range()`. Prefer it to `WHERE n IN [...]`, which checks every node. +- `n = 0`, `id(n) = 0` and `n IN [0, 1, 2]` look the nodes up directly. +- With a parameter, compare the node itself: `WHERE n = $id`. For a list, `UNWIND $ids AS x MATCH (n) WHERE n = x`, since `WHERE n IN $ids` checks every node. + +## Parameters + +[Query parameters](/query/cypher_subset#query-parameters) + +```python +client.query("MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", {"minAge": 30}) +client.query("MATCH (n:Person {name: $name}) RETURN n.age", {"name": "O'Brien"}) +client.query("UNWIND $rows AS row CREATE (:City {name: row.name, pop: row.pop})", {"rows": [{"name": "Brest", "pop": 139000}]}) +client.query("MATCH (n:$label) RETURN n.name", {"label": "Person"}) +``` + +- A parameter holds a value anywhere an expression goes, or a label or an edge type. No quoting or escaping is needed. +- `SKIP`, `LIMIT` and the vector of a `VECTOR SEARCH` take literals. +- In the shell: `:param minAge => 30`. In JavaScript: `client.query(cypher, { parameters: { minAge: 30 } })`. ## Joins and Cartesian products @@ -287,8 +304,16 @@ RETURN reduce(total = 0, x IN [1, 2, 3] | total + x) MATCH (n:Person) RETURN n.name, CASE WHEN n.age >= 35 THEN 'senior' ELSE 'junior' END AS band + +MATCH (n:Person) +RETURN n {.name, .age, nextAge: n.age + 1} AS person + +WITH {a: {b: 2}} AS m +RETURN m.a.b, m['a'] ``` +- Map projection: `n {.name}` picks properties, `n {.*}` takes them all, `n {key: expr}` adds computed entries. + ## Subqueries [Subqueries](/query/subqueries) @@ -335,12 +360,14 @@ RETURN p.name, co | Category | Functions | | -------- | --------- | -| Entity | `id`, `labels`, `type`, `startNode`, `endNode`, `keys`, `properties` | -| Path and list | `nodes`, `relationships`, `length`, `size`, `head`, `last`, `tail`, `reverse`, `range`, `reduce` | +| Entity | `id`, `elementId`, `labels`, `type`, `startNode`, `endNode`, `keys`, `properties` | +| Path and list | `nodes`, `relationships`, `length`, `size`, `head`, `last`, `tail`, `reverse`, `isEmpty`, `range`, `reduce`, `allReduce`, `coll.indexOf` | +| Predicates | `all`, `any`, `none`, `single`, `exists(pattern)` | | Conversion | `toInteger`, `toFloat`, `toString`, `toBoolean`, their `...OrNull` forms, `coalesce` | | String | `toUpper`, `toLower`, `trim`, `ltrim`, `rtrim`, `substring`, `left`, `right`, `split`, `replace`, `reverse`, `char_length`, `normalize` | | Math | `abs`, `sign`, `ceil`, `floor`, `round`, `sqrt`, `exp`, `log`, `log10`, `pi`, `e`, `sin`, `cos`, `tan`, and more | | Temporal | `datetime`, `duration` | +| Spatial | `point`, `point.distance`, `point.withinBBox` | | Embedding | `cosine_similarity`, `euclidean_distance` | | Aggregate | `count`, `collect`, `sum`, `avg`, `min`, `max`, `stDev`, `stDevP`, `percentileCont`, `percentileDisc` | @@ -352,11 +379,13 @@ RETURN datetime('2024-03-15T10:30:00Z') + duration({days: 1}) MATCH (n:Document) RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) + +RETURN round(point.distance(point({latitude: 51.5074, longitude: -0.1278}), point({latitude: 48.8566, longitude: 2.3522}))) AS meters ``` - Function names are case-insensitive. - Use `type(e)` for an edge's type (it replaces `edgeType()`). -- Read map keys with `m.key` or `m[k]`, including maps stored as properties. +- Read map keys with `m.key`, `m['key']` or `m[k]`, including maps stored as properties. ## Data types @@ -370,6 +399,7 @@ RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) | Map | `{a: 1}` | | DateTime | `datetime('2024-03-15T10:30:00Z')` | | Duration | `duration({days: 2})` | +| Point | `point({x: 1, y: 2})`, `point({latitude: 51.5, longitude: -0.12})` | | Embedding | `(1.2, 2.0, 0.0)` (parentheses, not brackets) | ## Writing (inside a change) @@ -395,6 +425,9 @@ MERGE (n:Person {name: 'Alice'}) MATCH (n:Person {name: 'Alice'}) SET n.age = 31, n.position = 'Developer' +MATCH (n:Person {name: 'Alice'}) +SET n += {city: 'Berlin', level: 2} + MATCH (n:Person {name: 'Alice'}) REMOVE n.position @@ -417,6 +450,7 @@ CHANGE SUBMIT ``` - Every node needs at least one label and every edge exactly one type. Labels are set at creation. +- `SET n += {...}` adds and updates properties, `SET n = {...}` replaces them all. - Run `COMMIT` after creating entities that a later query must `MATCH`. `MERGE` sees them without it, and `CHANGE SUBMIT` commits for you. - In the Python SDK, call `client.checkout()` after `CHANGE SUBMIT`. - To build a new graph from more than a few thousand entities, use [`LOAD PARQUET`](/import_data/parquet). To add edges between existing nodes, `UNWIND` pairs of node IDs as above. @@ -463,7 +497,7 @@ See [Shortest path](/shortest-path) and [Vector search](/vector-search). | `LIST GRAPH` / `LIST AVAILABLE GRAPHS` | Loaded graphs / graphs on disk | | `CHANGE NEW` / `SUBMIT` / `DELETE` / `LIST` | Create, merge, discard and list changes | | `COMMIT` | Seal pending writes inside a change | -| `LOAD COMMIT ''` | Load a past commit (REST API only; the CLI and SDK do it for you) | +| `LOAD COMMIT ''` | Load a past commit before querying it (the shell's and the Python SDK's `checkout` do it for you) | | `MERGE_DATAPARTS` | Compact the graph's DataParts into one (renumbers nodes) | | `CREATE VECTOR INDEX WITH DIMENSION METRIC ` | Create a vector index | | `SHOW VECTOR INDEXES` / `DELETE VECTOR INDEX ` | List / delete vector indexes | diff --git a/query/composition.mdx b/query/composition.mdx index bf1a84a..e6193f4 100644 --- a/query/composition.mdx +++ b/query/composition.mdx @@ -75,7 +75,7 @@ UNION ### Inside subqueries -`NEXT` works inside [`CALL`, `EXISTS` and `COUNT`](/query/subqueries) subqueries, where the chain runs once for each outer row: +`NEXT` works inside [`CALL`, `EXISTS`, `COUNT` and `COLLECT`](/query/subqueries) subqueries, where the chain runs once for each outer row: ```cypher MATCH (c:Company) @@ -95,6 +95,15 @@ RETURN p.name, COUNT { MATCH (friend)-[:KNOWS]->(fof) RETURN fof } AS friendsOfFriends + +MATCH (p:Person {name: 'Alice'}) +RETURN COLLECT { + MATCH (p)-[:KNOWS]->(f) + RETURN f AS friend + NEXT + MATCH (friend)-[:KNOWS]->(fof) + RETURN fof.name +} AS namesTwoHopsAway ``` ### Writes @@ -113,7 +122,6 @@ RETURN count(x) AS tags - **Aliases.** Every expression in a `RETURN` followed by `NEXT` needs an alias. Without one: `Expression in a RETURN followed by NEXT must be aliased (use AS)`. - **Scope.** Only the returned columns carry over. `MATCH (n:Person) RETURN n.name AS nm NEXT RETURN n.age` reports `Variable 'n' not found`, because the first step did not return `n`. -- `NEXT` is not yet available inside a `COLLECT { }` body. ## WHEN @@ -163,6 +171,32 @@ CALL (n) { RETURN n.name, employer ``` +### Inside EXISTS, COUNT and COLLECT + +A conditional can also be the body of an [`EXISTS`, `COUNT` or `COLLECT`](/query/subqueries) subquery. For each row, the subquery tests, counts or collects what the chosen branch returns: + +```cypher +MATCH (p:Person) +WHERE EXISTS { + WHEN p.age > 30 THEN MATCH (p)-[:KNOWS]->(f) RETURN f +} +RETURN p.name + +MATCH (p:Person) +RETURN p.name, COUNT { + WHEN p.city = 'London' THEN MATCH (p)-[:KNOWS]->(f) RETURN f AS x + ELSE MATCH (p)-[:LIKES]->(x) RETURN x +} AS links + +MATCH (p:Person) +RETURN p.name, COLLECT { + WHEN p.age > 30 THEN MATCH (p)-[:KNOWS]->(f) RETURN f.name AS x + ELSE RETURN 'under 31' AS x +} AS friendsOrBand +``` + +The first query keeps Bob only: Dave is over 30 but knows nobody, and the others take no branch. + ### Without ELSE When no branch matches and there is no `ELSE`, the conditional returns no rows, so `WHEN false THEN RETURN 1 AS x` returns nothing. In a `CALL`, rows that match no branch are dropped. Use `OPTIONAL CALL` to keep them with `null` values: @@ -199,6 +233,17 @@ UNION { WHEN false THEN RETURN 'b' AS x ELSE RETURN 'c' AS x } ``` +A branch can hold a `UNION` too, in braces: + +```cypher +WHEN COUNT { MATCH (c:Company) } > 0 THEN { + MATCH (p:Person) RETURN p.name AS name + UNION + MATCH (c:Company) RETURN c.name AS name +} +ELSE RETURN 'no companies' AS name +``` + ### Writes Branches can write, inside a change. See [Conditional writes](/query/writing#conditional-writes): @@ -213,6 +258,5 @@ RETURN count(*) ### Rules -- **Same columns.** All branches return the same column names. Otherwise: `All branches of a conditional query must return the same column names`. -- **Where it goes.** A `WHEN` starts a query, follows `NEXT`, forms a `CALL` body, or is a braced side of a `UNION`. To branch on the results of a `MATCH`, return them and continue after `NEXT`, as above. -- **Not yet available:** `UNION` inside a branch, and `WHEN` inside an `EXISTS`, `COUNT` or `COLLECT` body. +- **Same columns.** All branches return the same column names. Otherwise: `All branches of a conditional query must return the same column names`. A column can hold values of different types from different branches, as in `WHEN true THEN RETURN 1 AS x ELSE RETURN 'a' AS x`. +- **Where it goes.** A `WHEN` starts a query, follows `NEXT`, forms the body of a `CALL`, `EXISTS`, `COUNT` or `COLLECT` subquery, or is a braced side of a `UNION`. To branch on the results of a `MATCH`, return them and continue after `NEXT`, as above. diff --git a/query/cypher_subset.mdx b/query/cypher_subset.mdx index 8786cdc..a3b313b 100644 --- a/query/cypher_subset.mdx +++ b/query/cypher_subset.mdx @@ -1,10 +1,10 @@ --- title: "Query Language in TuringDB" sidebarTitle: "Overview" -description: "TuringDB v3 runs Cypher on a new engine and supports most of openCypher: OPTIONAL MATCH, WITH, UNWIND, UNION, MERGE, variable-length paths, subqueries and more." +description: "TuringDB v3 runs Cypher on a new engine and supports most of openCypher: OPTIONAL MATCH, WITH, UNWIND, UNION, MERGE, variable-length paths, subqueries, parameters and more." --- -TuringDB v3 (`turingdb` 3.0.2 on PyPI) runs every query on a new MLIR-based engine and supports **most of openCypher**, extended with versioning, vector search and fast lookup by native node ID. +TuringDB v3 (`turingdb` 3.1 on PyPI and npm) runs every query on a new MLIR-based engine and supports **most of openCypher**, extended with versioning, vector search and fast lookup by native node ID. If you know Cypher from Neo4j, you can write the queries you are used to. This page covers the basics and the handful of places where TuringDB differs. The rest of the language is documented in the pages that follow. @@ -15,13 +15,13 @@ If you know Cypher from Neo4j, you can write the queries you are used to. This p | Reading | `MATCH`, `OPTIONAL MATCH`, `WHERE`, `WITH`, `RETURN [DISTINCT]`, `ORDER BY`, `SKIP`, `LIMIT`, `UNWIND`, `UNION [ALL]`, `LOAD CSV` | | Composing queries | [`NEXT`](/query/composition#next) to chain queries, [`WHEN ... THEN ... ELSE`](/query/composition#when) conditional queries | | Patterns | Multiple labels, label expressions (`:A\|B`, `:!A`, `:A&B`), edge type alternatives (`[:A\|B]`), inline `WHERE`, variable-length paths (`*1..3`, `->+`, `{2,4}`), quantified path patterns, named paths, pattern predicates | -| Expressions | Arithmetic, string and list concatenation, `CASE`, list and pattern comprehensions, `reduce`, `all` / `any` / `none` / `single`, list slicing, maps and map access | -| Functions | String (`substring`, `split`, `replace`, `left`, `right`, `normalize`, ...), math, conversion, `keys`, `properties` | +| Expressions | Arithmetic, string and list concatenation, `CASE`, list and pattern comprehensions, `reduce`, `allReduce`, `all` / `any` / `none` / `single`, `exists()` over a pattern, `IS` label tests, list slicing, maps, map access (`m.key`, `m['key']`) and map projection (`n {.name, .age}`) | +| Functions | String (`substring`, `split`, `replace`, `left`, `right`, `normalize`, ...), math, conversion, `keys`, `properties`, `elementId`, `isEmpty`, `coll.indexOf`, spatial (`point`, `point.distance`, `point.withinBBox`) | | Aggregation | `count`, `sum`, `avg`, `min`, `max`, `collect`, `stDev`, `stDevP`, `percentileCont`, `percentileDisc`, `DISTINCT`, implicit grouping, aggregates inside expressions | -| Subqueries | `EXISTS { }`, `COUNT { }`, `COLLECT { }`, `CALL { }`, `CALL (*) { }` and `OPTIONAL CALL { }` | -| Writing | `CREATE`, `MERGE` with `ON CREATE` / `ON MATCH`, `SET`, `REMOVE`, `DELETE`, `DETACH DELETE`, `FOREACH` | -| Parameters | Labels and edge types (`MATCH (n:$label)`) | -| Types | String, Integer, Double, Boolean, List, Map, `DateTime`, `Duration`, Embedding | +| Subqueries | `EXISTS { }`, `COUNT { }`, `COLLECT { }`, `CALL { }`, `CALL (*) { }` and `OPTIONAL CALL { }`, with `UNION`, `NEXT` and `WHEN` inside any of them | +| Writing | `CREATE`, `MERGE` with `ON CREATE` / `ON MATCH`, `SET` (including `SET n = {...}` and `SET n += {...}`), `REMOVE`, `DELETE`, `DETACH DELETE`, `FOREACH` | +| Parameters | Values anywhere an expression goes (`WHERE n.age > $min`, `UNWIND $rows`), labels and edge types (`MATCH (n:$label)`) | +| Types | String, Integer, Double, Boolean, List, Map, `DateTime`, `Duration`, Point, Embedding | | TuringDB extras | Changes, commits and time travel, `shortestPath`, `VECTOR SEARCH`, `gnn.graphSAGE`, lookup by native node ID | ## Basics @@ -41,35 +41,62 @@ RETURN a.name, e.since, b.name By convention, node labels are written in PascalCase (`Person`, `BankAccount`), edge types in upper case with underscores (`WORKS_AT`, `FRIENDS_WITH`), and strings use single quotes. -Keywords are case-insensitive (`MATCH` and `match` are the same). Labels, edge types and property names are case-sensitive. Only `//` starts a comment: +Keywords are case-insensitive (`MATCH` and `match` are the same), and a keyword can also be a name: `WITH n.name AS count, n.age AS order` is valid. Labels, edge types and property names are case-sensitive. `//` starts a line comment and `/* */` encloses a block comment: ```cypher // a line comment -MATCH (n:Person) RETURN n.name // also fine at the end of a line +MATCH (n:Person) /* a block comment */ +RETURN n.name // also fine at the end of a line ``` - Use `//` for comments. SQL-style `--` and `/* */` comments are not yet supported. + SQL-style `--` comments are not supported. Use `//` instead. ### Running a query -Queries run through the Python SDK, the interactive shell, or the HTTP endpoint. Read queries run directly against the current graph; [write queries](/query/writing) must run inside a change. +Queries run through the [Python SDK](/pythonsdk/reference), the [JavaScript SDK](/jssdk/reference), or the [interactive shell](/commands#interactive-shell). Read queries run directly against the current graph; [write queries](/query/writing) must run inside a change. -```python + + +```python Python from turingdb import TuringDB -client = TuringDB(host="http://localhost:6666") +client = TuringDB(host="localhost", port=6666) df = client.query("MATCH (n:Person) RETURN n.name, n.age") # a pandas DataFrame ``` +```js JavaScript +import { TuringClient } from "turingdb"; + +const client = new TuringClient({ url: "http://localhost:6686/query" }); +const chunks = await client.queryData("MATCH (n:Person) RETURN n.name, n.age"); // chunks of columns +``` + + + `EXPLAIN ` returns the compiled query plan without running the query, and `EXPLAIN (db) ` shows only the database plan. It is the easiest way to check how a lookup runs. ### Query parameters -`client.query()` takes an optional dictionary of parameters. A `$name` parameter can stand for a **label or an edge type**, anywhere a label or type is written: +`client.query()` takes an optional dictionary of parameters. A `$name` parameter stands for a **value** anywhere an expression goes, so values travel next to the query text and need no quoting or escaping: + +```python +client.query("MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", {"minAge": 30}) +client.query("MATCH (n:Person {name: $name}) RETURN n.age", {"name": "O'Brien"}) +client.query("MATCH (n:Person) WHERE n.name IN $names RETURN n.name", {"names": ["Alice", "Bob"]}) +client.query("UNWIND $rows AS row RETURN row.name, row.age", {"rows": [{"name": "Alice", "age": 30}]}) +client.query("WITH $filter AS f MATCH (n:Person) WHERE n.city = f.city RETURN n.name", {"filter": {"city": "London"}}) +client.query("RETURN $0 + 1 AS next", {"0": 41}) # numbered parameter + +# inside a change +client.query("CREATE (n:Person {name: $name, age: $age})", {"name": "Zoe", "age": 27}) +client.query("MATCH (n:Person {name: 'Alice'}) SET n += $props", {"props": {"city": "Berlin"}}) +``` + +A parameter can also stand for a **label or an edge type**, anywhere a label or type is written: ```python client.query("MATCH (n:$label) RETURN n.name", {"label": "Person"}) @@ -85,24 +112,28 @@ client.query("MERGE (t:$label {name: 'x'})", {"label": "Tag"}) client.query("MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Bob'}) MERGE (a)-[:$type]->(b)", {"type": "FOLLOWS"}) ``` -- A parameter holds a string, or a list of strings to require several labels. -- A parameter used in the query but missing from the dictionary is reported when the query runs: `Parameter $label is not defined`. -- Put values directly in the query text for now: value parameters, such as `WHERE n.name = $name`, are reported as `Not implemented: Parameters`. +- **Values.** From Python, a parameter can hold a string, a number, a boolean, `None`, a list, a dict, a time-zone-aware `datetime` (a DateTime) or a `timedelta` (a Duration). See [`query()`](/pythonsdk/reference#core-methods) for the full list, and [Parameters](/jssdk/reference#parameters) for JavaScript values. +- **Labels and edge types.** The parameter holds a string, or a list of strings to require several labels. +- **Missing parameters.** A parameter used in the query but missing from the dictionary is reported when the query runs: `Parameter $label is not defined`. +- **Where literals are still needed.** `SKIP` and `LIMIT` take a number written in the query: `LIMIT $n` is reported as `Unsupported expression in SKIP/LIMIT: PARAMETER`. The vector of a [`VECTOR SEARCH`](/vector-search) is written in the query too, and a list parameter is a List, not an Embedding. +- **Node IDs.** Compare the node itself with an ID parameter, `WHERE n = $id`, which looks the node up directly. For a list of IDs, see [lookup by native node ID](/query/matching#look-up-nodes-by-native-node-id). +- **Size.** Parameters count towards the 1 MiB limit of a request, but take far less room than the same values written into the query text. -In the [interactive shell](/commands#interactive-shell), set parameters with `:param`. Over [HTTP](/commands#http-api), send them in the `parameters` field of the request. +In the [interactive shell](/commands#interactive-shell), set parameters with `:param => `. In the [JavaScript SDK](/jssdk/reference#parameters), pass them as `client.query(cypher, { parameters })`. ## Coming from Neo4j TuringDB follows openCypher closely, so the queries you know carry over. These are the TuringDB specifics worth knowing: -- **`RETURN n` returns the node's internal ID**, an integer. Return the properties you need with `RETURN n.name, n.age`, or all of them with `RETURN properties(n)`. -- **Values go in the query text.** [Parameters](#query-parameters) name labels and edge types. Put literal values directly in the text and escape quotes in your application. +- **`RETURN n` returns the node's internal ID**, an integer. Return the properties you need with `RETURN n.name, n.age`, a [map projection](/query/functions#map-projection) such as `RETURN n {.name, .age}`, or all of them with `RETURN properties(n)`. A path comes back as the list of its node and edge IDs, and `elementId(n)` is the same integer as `id(n)`. +- **Parameters.** [`$name` parameters](#query-parameters) hold values, labels and edge types. `SKIP`, `LIMIT` and vector search take literals. - **One statement per request.** Send each statement as its own request (a single trailing `;` is fine), or chain steps in one request with [`NEXT`](/query/composition#next). - **Labels are set at creation.** Every node has at least one label and every edge exactly one edge type, chosen when it is created. - **Property types are global.** A property name has a single value type across the whole graph: if `age` is an Integer, it is an Integer on every node. - **Comparing Doubles.** Integers and Doubles compare by value (`n.age = 30.0` matches 30). Match one Double against another with a range (`n.score > 1.99 AND n.score < 2.01`). - **Forgiving names.** An unknown label matches nothing and a misspelled property reads as `null`. Check names with [`CALL db.labels()` and `CALL db.propertyTypes()`](/query/procedures). - **String matching.** Use `STARTS WITH`, `ENDS WITH`, `CONTAINS` and the [string functions](/query/functions). Regular expressions (`=~`) are not yet supported. +- **Testing for a property.** Write `n.prop IS NOT NULL`. As in Neo4j 5, `exists()` takes a pattern: `exists((p)-[:LIKES]->())`. - **Uniqueness.** Use `MERGE` to avoid duplicates. - **Shortest paths** use a TuringDB statement, see [Shortest path](/shortest-path). @@ -110,6 +141,15 @@ TuringDB follows openCypher closely, so the queries you know carry over. These a Coming from TuringDB 1.x? The v3 engine replaces the previous one, and many earlier limits are gone: `OPTIONAL MATCH`, `WITH`, `DISTINCT`, `UNION`, `MERGE`, `DELETE`, aggregates in any position, `UNWIND` over any list, `LOAD CSV` followed by `MATCH`, and `CREATE ... SET` all work now. One function changed name: `edgeType(e)` is now `type(e)`. + + Coming from TuringDB 3.0? Version 3.1 adds value parameters, map projection, `SET n = {...}` and `SET n += {...}`, points, `exists()`, `allReduce` and more. A few results change, to follow Neo4j: + + - A path comes back as a list of IDs, node and edge alternating (`[0, 0, 1]`), rather than a list of `{type, id}` dictionaries. + - `list + null` is `null`, and `=` between two maps is `null` when a `null` inside decides it, as between two lists: `{a: null} = {a: 1}` is `null`. + - `STARTS WITH`, `ENDS WITH` and `CONTAINS` bind more loosely than `+`: `'a' + 'bc' ENDS WITH 'bc'` is `true`. + - The Python SDK connects with `TuringDB(host="localhost", port=6666)`, see [client backends](/pythonsdk/reference#client-backends). + + ## Data types TuringDB offers these types for node and edge properties and for values in queries: @@ -124,6 +164,7 @@ TuringDB offers these types for node and edge properties and for values in queri | Map | `{a: 1, b: [1, 2]}` | See [Maps](/query/functions#maps) | | DateTime | `datetime('2024-03-15T10:30:00Z')` | Microsecond precision, always UTC | | Duration | `duration({days: 2})` | Microsecond precision, no month component | +| Point | `point({x: 1, y: 2})`, `point({latitude: 51.5, longitude: -0.12})` | 2D or 3D, cartesian or WGS-84. See [Points](/query/functions#points) | | Embedding | `(1.2, 2.0, 0.0)` | Parentheses, at least 2 numbers. See [Vector search](/vector-search) | diff --git a/query/functions.mdx b/query/functions.mdx index 7d5a6b6..8963029 100644 --- a/query/functions.mdx +++ b/query/functions.mdx @@ -1,23 +1,24 @@ --- title: "Functions and Data Types" sidebarTitle: "Functions and Types" -description: "Built-in functions, how values come back to the Python SDK, dates and durations, maps, and embedding comparison in TuringDB v3." +description: "Built-in functions, how values come back to the SDKs, dates and durations, maps and map projection, points, and embedding comparison in TuringDB v3." --- ## Built-in functions -Function names are case-insensitive, like keywords: `toInteger`, `TOINTEGER` and `ToInteger` are the same function. +Function names are case-insensitive, like keywords: `toInteger`, `TOINTEGER` and `ToInteger` are the same function. Functions in a namespace are written with a dot, such as `coll.indexOf` and `point.distance`. | Category | Functions | Notes | | -------- | --------- | ----- | -| Entity | `id(x)`, `labels(n)`, `type(r)`, `startNode(r)`, `endNode(r)`, `keys(x)`, `properties(x)` | `labels` and `keys` return **lists**. `properties` returns a map of a node's, an edge's or a map's properties. `type` returns a string. `startNode(r).name` works | +| Entity | `id(x)`, `elementId(x)`, `labels(n)`, `type(r)`, `startNode(r)`, `endNode(r)`, `keys(x)`, `properties(x)` | `labels` and `keys` return **lists**. `properties` returns a map of a node's, an edge's or a map's properties. `type` returns a string. `elementId` returns the same integer as `id`. `startNode(r).name` works | | Path | `nodes(p)`, `relationships(p)`, `length(p)` | Lists of IDs. Read properties through a [comprehension](/query/aggregation#list-comprehensions) | -| List | `size`, `length`, `head`, `last`, `tail`, `reverse`, `range(start, end[, step])`, `reduce` | `size`, `length` and `reverse` also work on strings. `range` bounds are inclusive | -| Predicates | `all`, `any`, `none`, `single` `(x IN list WHERE ...)` | | -| Conversion | `toInteger`, `toFloat`, `toString`, `toBoolean`, `toIntegerOrNull`, `toFloatOrNull`, `toStringOrNull`, `toBooleanOrNull`, `coalesce(a, b, ...)` | `toInteger` accepts strings, numbers and booleans, and `toFloat` strings and numbers. Both return `null` for a bad string. `toBoolean` accepts strings and integers. `toString` formats `DateTime` and `Duration` values. The `OrNull` forms return `null` for any value they cannot convert. `coalesce` arguments may mix types | -| String | `toUpper`, `toLower`, `trim`, `ltrim`, `rtrim`, `substring(s, start[, length])`, `left(s, n)`, `right(s, n)`, `split(s, delimiter)`, `replace(s, search, replacement)`, `reverse`, `char_length` (or `character_length`), `normalize(s[, NFC\|NFD\|NFKC\|NFKD])` | Positions start at 0. Plus the operators `STARTS WITH`, `ENDS WITH`, `CONTAINS`, `IS NORMALIZED` and `\|\|` | +| List | `size`, `length`, `head`, `last`, `tail`, `reverse`, `isEmpty`, `range(start, end[, step])`, `reduce`, `coll.indexOf(list, value)` | `size`, `length` and `reverse` also work on strings, and `isEmpty` on strings and maps. `range` bounds are inclusive. See [Lists](#lists) | +| Predicates | `all`, `any`, `none`, `single` `(x IN list WHERE ...)`, `allReduce`, `exists(pattern)` | See [Lists](#lists) and [pattern predicates](/query/matching#where-and-operators) | +| Conversion | `toInteger`, `toFloat`, `toString`, `toBoolean`, `toIntegerOrNull`, `toFloatOrNull`, `toStringOrNull`, `toBooleanOrNull`, `coalesce(a, b, ...)` | `toInteger` accepts strings, numbers and booleans, and `toFloat` strings and numbers. Both return `null` for a bad string. `toBoolean` accepts strings and integers. `toString` formats `DateTime`, `Duration` and Point values. The `OrNull` forms return `null` for any value they cannot convert. `coalesce` arguments may mix types | +| String | `toUpper`, `toLower`, `trim`, `ltrim`, `rtrim`, `substring(s, start[, length])`, `left(s, n)`, `right(s, n)`, `split(s, delimiter)`, `replace(s, search, replacement)`, `reverse`, `char_length` (or `character_length`), `normalize(s[, NFC\|NFD\|NFKC\|NFKD])` | Positions start at 0. `split` also takes a list of delimiters. Plus the operators `STARTS WITH`, `ENDS WITH`, `CONTAINS`, `IS NORMALIZED` and `\|\|` | | Math | `abs`, `sign`, `ceil`, `floor`, `round`, `sqrt`, `exp`, `log`, `log10`, `pi()`, `e()` | Trigonometry: `sin`, `cos`, `tan`, `cot`, `asin`, `acos`, `atan`, `degrees`, `radians`, `haversin` | -| Temporal | `datetime(isoString)`, `datetime(epochSeconds)`, `datetime()`, `duration({days: 2, hours: 3})`, `duration(micros)` | See [Dates and durations](#dates-and-durations) | +| Temporal | `datetime(isoString)`, `datetime(epochSeconds)`, `datetime({year: 2024, month: 3, day: 15})`, `datetime()`, `duration({days: 2, hours: 3})`, `duration(micros)` | See [Dates and durations](#dates-and-durations) | +| Spatial | `point({x, y})`, `point({latitude, longitude})`, `point.distance(a, b)`, `point.withinBBox(p, lowerLeft, upperRight)` | See [Points](#points) | | Embedding | `cosine_similarity(a, b)`, `euclidean_distance(a, b)` | Arguments must be Embeddings, not lists. See [Vector search](/vector-search) | | Aggregate | `count`, `collect`, `sum`, `avg`, `min`, `max`, `stDev`, `stDevP`, `percentileCont`, `percentileDisc` | See [Aggregation](/query/aggregation#aggregation) | @@ -49,7 +50,7 @@ WHERE toLower(n.name) STARTS WITH 'a' RETURN n.name MATCH (n:Person {name: 'Alice'}) -RETURN keys(n), properties(n) +RETURN keys(n), properties(n), elementId(n) RETURN toIntegerOrNull('x'), toString(datetime('2024-03-15T10:30:00Z')) ``` @@ -65,18 +66,38 @@ RETURN left('hello', 2) AS l, right('hello', 3) AS r RETURN split('a.b.c', '.') AS parts, replace('banana', 'a', 'o') AS swapped +RETURN split('a,b;c', [',', ';']) AS parts + RETURN char_length('héllo') AS chars, size('héllo') AS alsoChars MATCH (n:Person) RETURN toUpper(left(n.name, 1)) + toLower(substring(n.name, 1)) AS display ``` +`split` with a list of delimiters splits on each of them, so the fourth query returns `['a', 'b', 'c']`. + `normalize` converts a string to a Unicode normal form (`NFC` by default, or `NFD`, `NFKC`, `NFKD`), and `IS NORMALIZED` tests for one: ```cypher RETURN normalize('fi', NFKC) AS plain, 'café' IS NFC NORMALIZED AS composed ``` +### Lists + +```cypher +RETURN isEmpty([]) AS emptyList, isEmpty('') AS emptyString, isEmpty({}) AS emptyMap + +MATCH (n:Person {name: 'Carol'}) +RETURN coll.indexOf(n.tags, 'vip') AS position, coll.indexOf(n.tags, 'admin') AS missing + +RETURN allReduce(total = 0, x IN [1, 2, 3] | total + x, total < 10) AS staysUnder10, + allReduce(total = 0, x IN [1, 2, 3] | total + x, total < 5) AS staysUnder5 +``` + +- `coll.indexOf(list, value)` returns the position of the first element equal to `value`, or `-1`. The second query returns `1` and `-1`. +- `allReduce(acc = init, x IN list | step, predicate)` folds the list like `reduce`, and is `true` when the predicate holds for the accumulator after **every** step. The initial value is not tested, and an empty list gives `true`. It stops at the first step that fails, so later steps are not computed. Over a path, it [tests a running total at each hop](/query/paths#filtering-paths-with-allreduce). The third query returns `true` and `false`. +- `isEmpty`, `coll.indexOf` and `allReduce` return `null` for a `null` list. + ### Conversions The strict conversions return `null` for a string they cannot read, and reject other values they cannot convert. The `OrNull` forms return `null` instead of rejecting: @@ -95,7 +116,7 @@ RETURN toString(duration({days: 2, hours: 3})) AS iso In v3, `type(e)` returns an edge's type as a string. It replaces `edgeType()` from earlier versions. -The function library keeps growing. `rand()`, `date()`, `timestamp()`, `elementId()`, `atan2()`, `isNaN()` and the two-argument `round(x, digits)` are not yet available, so compute those in your application. +The function library keeps growing. `rand()`, `date()`, `timestamp()`, `atan2()`, `isNaN()` and the two-argument `round(x, digits)` are not yet available, so compute those in your application. ## Arithmetic in `RETURN` and `SET` @@ -113,7 +134,7 @@ WHERE r.since > 0 RETURN r.since / 10 * 100 ``` -`+ - * / % ^` are supported. Integer `/` truncates, and `^` returns a Double. `+` and `||` concatenate strings and lists, `'x' + 1` gives `'x1'`, and `[1, 2] + 3` gives `[1, 2, 3]`. +`+ - * / % ^` are supported. Integer `/` truncates, and `^` returns a Double. `+` and `||` concatenate strings and lists, `'x' + 1` gives `'x1'`, and `[1, 2] + 3` gives `[1, 2, 3]`. A list plus `null` is `null`. ## Dates and durations @@ -126,6 +147,8 @@ RETURN datetime('2024-03-15') AS midnight RETURN datetime(1700000000) AS fromEpoch +RETURN datetime({year: 2024, month: 3, day: 15, hour: 10, minute: 30}) AS fromParts + RETURN datetime('2024-03-15T10:30:00Z') + duration({days: 1, hours: 2}) AS later RETURN datetime('2024-03-16T00:00:00Z') - datetime('2024-03-15T00:00:00Z') AS elapsed @@ -135,7 +158,7 @@ RETURN datetime('2024-03-15T10:30:00Z').year, datetime('2024-03-15T10:30:00Z').m RETURN duration({days: 2, hours: 3}).hours ``` -Durations can be added, subtracted, scaled and compared with each other and with datetimes: +Durations can be added, subtracted, scaled, compared, summed and averaged: ```cypher RETURN duration({hours: 36}) + duration({hours: 12}) AS total @@ -147,12 +170,17 @@ RETURN duration({days: 2}) * 3 AS threeTimes RETURN duration({hours: 36}) / 2 AS half RETURN duration({hours: 1}) > duration({minutes: 30}) AS longer + +UNWIND [duration({hours: 1}), duration({hours: 2})] AS d +RETURN sum(d) AS total, avg(d) AS mean ``` - `datetime` values are stored as microseconds since the Unix epoch, **normalised to UTC**. A date-only string means midnight, and an offset (`+02:00`) is converted to the equivalent UTC instant. +- `datetime({...})` builds a datetime from its parts: `year`, then optionally `month`, `day`, `hour`, `minute`, `second`, `millisecond`, `microsecond` and `nanosecond`. Missing parts default to the start of the period, so `datetime({year: 2024})` is midnight on 1 January. A part needs the ones above it, as in `The datetime component month needs the component year`. A `timezone` key takes an offset, such as `'+02:00'` or `'Z'`, and `{epochSeconds: 1700000000}` also works. - `duration` is a signed count of microseconds. It has no month component, because a month has no fixed length. `duration()` reads the number as microseconds. - Components are available as properties: `d.year`, `d.month`, `d.day`, `d.hour`, and so on. On a duration, `dur.days`, `dur.hours`, ... give the **total** in that unit, so `duration({days: 2, hours: 3}).hours` is `51`. -- `<`, `>`, `min` and `max` work on temporal values. +- `<`, `>`, `min` and `max` work on temporal values, and `sum` and `avg` on durations. +- From Python, pass a time-zone-aware `datetime.datetime` as a parameter for a DateTime, and a `datetime.timedelta` for a Duration: `client.query("RETURN $since + duration({days: 1}) AS next", {"since": datetime(2024, 1, 1, tzinfo=timezone.utc)})`. - When you [import JSONL](/import_data/jsonl) you can declare which properties are `DateTime` or `Duration` with `WITH DATETIMES [...]` and `WITH DURATIONS [...]`. ## Maps @@ -163,7 +191,10 @@ Maps can be built, returned, compared, sorted and stored as node or edge propert RETURN {name: 'Alice', tags: ['a', 'b']} AS m WITH {a: 1, b: 'x'} AS m -RETURN m.a, m.b +RETURN m.a, m['b'] + +WITH {a: {b: 2}} AS m +RETURN m.a.b WITH {a: 1, b: 2} AS m UNWIND keys(m) AS k @@ -176,35 +207,118 @@ WITH [{a: 1}, {a: 2}] AS ms RETURN ms[1].a, [m IN ms | m.a] MATCH (n:Person {name: 'Alice'}) -RETURN properties(n).name, keys(n) +RETURN properties(n).name, keys(n), n['age'] + +MATCH (n:Person) +RETURN collect({name: n.name, city: n.city}) AS people +``` + +- `m.key` and `m['key']` read a key written in the query, and `m[k]` a key held in a variable. A chain such as `m.a.b` reads through nested maps, and a map stored as a property reads the same way: `n.meta.level`, `keys(n.meta)`. +- A map value can be a node, a datetime or any other value, and reads through it: `WITH {p: n} AS m RETURN m.p.name`, or `m.d.year` for a datetime under `d`. +- Keys can be any word, keywords included: `WITH {count: 3, order: 'x'} AS m RETURN m.count, m.order`. A key written twice keeps its last value: `{a: 1, a: 2}` is `{a: 2}`. +- `collect()` gathers maps, and `collect(DISTINCT ...)` compares them entry by entry. +- A value read from a map can be stored as a property directly, as in `CREATE (:City {name: row.name})`. The property takes the type of the value (see [Bulk writes](/query/writing#small-volumes)). + +### Map projection + +A map projection builds a map out of a node, an edge or a map, picking the entries you need: + +```cypher +MATCH (n:Person {name: 'Alice'}) +RETURN n {.name, .age} AS person + +MATCH (n:Person {name: 'Dave'}) +RETURN n {.*} AS everything + +MATCH (n:Person {name: 'Alice'}) +RETURN n {.name, nextAge: n.age + 1} AS m + +MATCH (n:Person)-[:WORKS_AT]->(c:Company) +RETURN c.name, collect(n {.name, .city}) AS staff + +MATCH (n:Person {name: 'Alice'}) +WITH n, 'x' AS extra +RETURN n {.name, extra} AS m + +WITH datetime('2024-03-15T10:30:00Z') AS d +RETURN d {.year, .month} AS parts +``` + +- `.key` holds the property `key`, or `null` when it is missing. `.*` holds every property, `key: expression` a computed value, and a bare variable `x` its value under the key `x`. +- On a datetime, a duration or a point, `.key` reads a component: the last query returns `{month: 3, year: 2024}`. +- An entry written explicitly wins over the one `.*` reads under the same key, wherever `.*` stands: `n {.*, age: n.age + 1}`. +- A `null` node, such as an unmatched `OPTIONAL MATCH`, projects to `null`. + +## Points + +A point is a location, 2D or 3D, either on a plane (cartesian) or on the Earth (WGS-84). `point()` builds one from a map: + +```cypher +RETURN point({x: 1, y: 2}) AS flat, point({x: 1, y: 2, z: 3}) AS space + +RETURN point({latitude: 51.5074, longitude: -0.1278}) AS london + +WITH point({latitude: 51.5074, longitude: -0.1278}) AS p +RETURN p.latitude, p.longitude, p.srid, p.crs + +RETURN point.distance(point({x: 0, y: 0}), point({x: 3, y: 4})) AS d + +RETURN round(point.distance(point({latitude: 51.5074, longitude: -0.1278}), point({latitude: 48.8566, longitude: 2.3522}))) AS londonToParis + +RETURN point.withinBBox(point({x: 1, y: 1}), point({x: 0, y: 0}), point({x: 2, y: 2})) AS inside +``` + +The keys of the map choose the coordinate reference system (CRS): + +| Keys | CRS | SRID | +| ---- | --- | ---- | +| `x`, `y` | `cartesian` | 7203 | +| `x`, `y`, `z` | `cartesian-3d` | 9157 | +| `longitude`, `latitude` | `wgs-84` | 4326 | +| `longitude`, `latitude`, `height` | `wgs-84-3d` | 4979 | + +- A `crs: 'wgs-84'` or `srid: 4326` key chooses the CRS explicitly, and `x` and `y` then hold the longitude and the latitude. +- **Components.** `p.x`, `p.y` and `p.z` read the coordinates, and `p.longitude`, `p.latitude` and `p.height` the same ones on a WGS-84 point. `p.crs` is the CRS name and `p.srid` its SRID. Reading `p.z` on a 2D point is an error. +- **Checks.** Coordinates must be numbers (`point() reads a number for 'x', not 'String'`), and a latitude must lie in [-90, 90]. A longitude outside [-180, 180] wraps around. A `null` map or a `null` coordinate gives `null`. +- **`point.distance(a, b)`** is the straight-line distance between two cartesian points. Between two WGS-84 points it is the great-circle distance in metres, on a sphere of the mean Earth radius (6,371,008.8 m), so the fifth query returns about 343,557 m. It is `null` for two points of different CRSs. +- **`point.withinBBox(p, lowerLeft, upperRight)`** tests whether `p` lies in the box, bounds included. On WGS-84, a `lowerLeft` east of `upperRight` makes a box that crosses the 180th meridian. +- **Comparing and aggregating.** Points work with `=`, `<`, `ORDER BY`, `DISTINCT`, grouping, `count`, `collect`, `min` and `max`, ordered by SRID, then x, y and z. `avg` of points of one CRS is their centroid. +- `toString(p)` writes `point({x: 2.3522, y: 48.8566, crs: 'wgs-84'})`. + +Store a point as a property like any other value, and filter on distance: + +```cypher +CREATE (:City {name: 'Paris', at: point({latitude: 48.8566, longitude: 2.3522})}), + (:City {name: 'London', at: point({latitude: 51.5074, longitude: -0.1278})}) + +MATCH (c:City) +WHERE point.distance(c.at, point({latitude: 51.5074, longitude: -0.1278})) < 400000 +RETURN c.name ``` -- `m.key` reads a key written in the query, and `m[k]` a key held in a variable. A map stored as a property reads the same way: `n.meta.level`, `keys(n.meta)`. -- Keys can be any word, keywords included: `WITH {count: 3, order: 'x'} AS m RETURN m.count, m.order`. -- Write a fixed key as `m.key`: the bracket form with a literal (`m['key']`) is reported with that hint. -- Map projection (`n {.name, .age}`) is not yet supported. Build the map you need (`{name: n.name, age: n.age}`) or use `properties(n)`. -- Values read from a map can hold any type. To store one as a property, convert it first, for example `toString(row.name)` or `toInteger(row.pop)` (see [Bulk writes](/query/writing#bulk-writes)). +Points can also be imported: from [JSONL](/import_data/jsonl#loading-date-duration-point-and-embedding-properties) with `WITH POINTS [...]`, and from the geometry columns of [GeoParquet](/import_data/parquet#property-columns) files. ## How results come back `client.query()` returns a pandas DataFrame. The column types follow the query's value types: -| Type | HTTP `column_types` | pandas dtype | -| ---- | ------------------- | ------------ | -| String | `String` | `string` | -| Integer | `Int64` | `Int64` | -| Unsigned integer (IDs, `count`, `length(p)`) | `UInt64` | `UInt64` | -| Double | `Double` | `float64` | -| Boolean | `Bool` | `boolean` | -| List | `List` | object (Python list) | -| Map | `Map` | object (dict) | -| Embedding | `Embedding` | object (list over HTTP, numpy `float32` array over the native and embedded backends) | -| DateTime | `DateTime` | `datetime64[us, UTC]` | -| Duration | `Duration` | `timedelta64[us]` | -| Node or edge (`RETURN n`) | `UInt64` | `UInt64` (internal ID, `null` when an `OPTIONAL MATCH` found nothing) | -| Named path (`RETURN p`) | `EntityList` | object (list of `{type, id}` dicts) | - -Doubles keep their full precision on every backend (`0.1 + 0.2` comes back as `0.30000000000000004`). `NaN` and infinities come back as the floats `nan` and `inf`, and `NaN` sorts after every number. +| Type | pandas dtype | +| ---- | ------------ | +| String | `string` | +| Integer | `Int64` | +| Unsigned integer (IDs, `count`, `length(p)`) | `UInt64` | +| Double | `float64` | +| Boolean | `boolean` | +| List | object (Python list) | +| Map | object (dict) | +| Embedding | object (numpy `float32` array) | +| DateTime | `datetime64[us, UTC]` | +| Duration | `timedelta64[us]` | +| Point | object (`turingdb.Point`, with `srid`, `x`, `y`, `z` and `crs`) | +| Node or edge (`RETURN n`) | `UInt64` (internal ID, `null` when an `OPTIONAL MATCH` found nothing) | +| Named path (`RETURN p`) | object (list of IDs, node and edge alternating: `[0, 0, 1]`) | + +A `DateTime` or a `Duration` inside a list or a map comes back as a time-zone-aware `datetime.datetime` or a `datetime.timedelta`. Doubles keep their full precision (`0.1 + 0.2` comes back as `0.30000000000000004`). `NaN` and infinities come back as the floats `nan` and `inf`, and `NaN` sorts after every number. The JavaScript SDK has its own mapping, see [Value types](/jssdk/reference#value-types). ## Embeddings diff --git a/query/matching.mdx b/query/matching.mdx index 6a70641..4741b59 100644 --- a/query/matching.mdx +++ b/query/matching.mdx @@ -161,15 +161,17 @@ WHERE n.age >= 18 AND n.city = 'London' RETURN n.name MATCH (n:Person) -WHERE n.name STARTS -WITH 'A' OR n.name ENDS -WITH 'e' OR n.name CONTAINS 'ro' +WHERE n.name STARTS WITH 'A' OR n.name ENDS WITH 'e' OR n.name CONTAINS 'ro' RETURN n.name MATCH (n:Person) WHERE n.name IN ['Alice', 'Bob'] RETURN n.name +MATCH (n) +WHERE n IS Company +RETURN n.name + MATCH (n:Person) WHERE 'admin' IN n.tags RETURN n.name @@ -193,15 +195,19 @@ RETURN p.name MATCH (p:Person) WHERE NOT (p)-[:KNOWS]->() RETURN p.name + +MATCH (p:Person) +RETURN p.name, exists((p)-[:WORKS_AT]->(:Company {name: 'Acme'})) AS atAcme ``` -- **Label and type tests:** `n:Person` tests a node label, `e:KNOWS` tests an edge type. Combine them with `AND`, `OR` and `NOT`, or write a [label expression](#label-expressions) such as `n:Person|Company`. -- **Pattern predicates:** `WHERE (p)-[:LIKES]->()` keeps the nodes for which the pattern exists. For richer conditions see [`EXISTS` subqueries](/query/subqueries). +- **Label and type tests:** `n:Person` tests a node label, `e:KNOWS` tests an edge type, and `n IS Person` is the same test written with `IS`. Combine them with `AND`, `OR` and `NOT`, or write a [label expression](#label-expressions) such as `n:Person|Company` or `n IS Person|Company`. Negate an `IS` test with `NOT n IS Person`. +- **Pattern predicates:** `WHERE (p)-[:LIKES]->()` keeps the nodes for which the pattern exists, and `exists((p)-[:LIKES]->())` is the same test as a value you can return. For richer conditions see [`EXISTS` subqueries](/query/subqueries). | Operators | | | --------- | - | | Comparison | `=`, `<>`, `!=`, `<`, `<=`, `>`, `>=`, and chained comparisons such as `1 < x < 3` | | Null tests | `IS NULL`, `IS NOT NULL` | +| Label tests | `n:Person`, `n IS Person`, label expressions such as `n:Person\|Company` | | Boolean | `AND`, `OR`, `XOR`, `NOT`, with three-valued logic for `null` as in Neo4j | | Membership and strings | `IN`, `STARTS WITH`, `ENDS WITH`, `CONTAINS`, `IS [NFC\|NFD\|NFKC\|NFKD] NORMALIZED` | | Arithmetic | `+`, `-`, `*`, `/`, `%`, `^`. Integer `/` truncates and `^` returns a Double | @@ -210,7 +216,7 @@ RETURN p.name Property constraints in a pattern, `MATCH (n {name: 'Matt', age: 20})`, are exactly equivalent to the same equality tests in `WHERE`. - Use `n.prop IS NOT NULL` to test for a property, and `STARTS WITH`, `ENDS WITH` and `CONTAINS` to match strings. Regular expressions (`=~`) and `exists(n.prop)` are not yet supported. + Use `n.prop IS NOT NULL` to test for a property: `exists()` takes a pattern, so `exists(n.prop)` is reported with that hint, as in Neo4j 5. Use `STARTS WITH`, `ENDS WITH` and `CONTAINS` to match strings. Regular expressions (`=~`) are not yet supported. ### Comparing numbers @@ -286,6 +292,10 @@ MATCH (n)-[:KNOWS]->(m) WHERE id(n) = 0 RETURN m.name +MATCH (n) +WHERE n IN [0, 1, 2] +RETURN n.name + UNWIND [0, 1, 2] AS x MATCH (n) WHERE n = x @@ -299,7 +309,7 @@ WHERE b = p[1] RETURN a.name, b.name ``` -`EXPLAIN` shows the lookup: the plan contains `const_scan_nodes` for IDs written in the query, or `fetch_nodes` for IDs read from a row, rather than `scan_nodes`: +`EXPLAIN` shows the lookup: the plan contains `const_scan_nodes` for IDs written in the query, or `fetch_nodes` for IDs read from a row or a parameter, rather than `scan_nodes`: ```cypher EXPLAIN MATCH (n) @@ -307,7 +317,14 @@ WHERE n = 0 RETURN n.name ``` -- **Batches.** `UNWIND` a list of IDs and compare each one: `UNWIND AS x MATCH (n) WHERE n = x`. The list can be written in the query (`f"UNWIND {ids} AS x ..."` in Python), held in a variable, or built with `collect()` or `range()`, and each ID costs a few microseconds. Use this form rather than `WHERE n IN [...]`, which checks every node. +**IDs in parameters.** With [query parameters](/query/cypher_subset#query-parameters), compare the node itself with the ID. `WHERE n = $id` looks the node up directly, while `WHERE id(n) = $id` and `WHERE n IN $ids` check every node. `UNWIND` a list of IDs: + +```python +client.query("MATCH (n) WHERE n = $id RETURN n.name", {"id": 0}) +client.query("UNWIND $ids AS x MATCH (n) WHERE n = x RETURN x, n.name", {"ids": [0, 1, 2]}) +``` + +- **Batches.** `UNWIND` a list of IDs and compare each one: `UNWIND $ids AS x MATCH (n) WHERE n = x`. The list can be a parameter, written in the query, held in a variable, or built with `collect()` or `range()`, and each ID costs a few microseconds. A list written in the query also works as `WHERE n IN [0, 1, 2]`. - **Several IDs per row.** Two IDs from the same row work too, which is the fast way to [create many edges between existing nodes](/query/writing#edges-between-existing-nodes). - **Fetch IDs once and reuse them.** Look nodes up with whatever filter you need (`MATCH (n:Person {name: 'Alice'}) RETURN id(n)`), keep the IDs in your application, and address the nodes by ID from then on. Looking each row's node up through a key property such as `n.uuid` reads every node with that label instead, and is about 100 times slower on a large graph. - **Vector search results are already nodes.** `VECTOR SEARCH ... YIELD ids` binds `ids` as nodes, so use `ids.title` or `MATCH (ids)-[...]->(...)` directly. See [Vector search](/vector-search). diff --git a/query/paths.mdx b/query/paths.mdx index 66c7670..6a3bf38 100644 --- a/query/paths.mdx +++ b/query/paths.mdx @@ -97,10 +97,48 @@ ORDER BY hops - `nodes(p)` and `relationships(p)` return **lists of IDs**. Read properties through a [list comprehension](/query/aggregation#list-comprehensions). - `length(p)` is the number of hops. -- A bare `RETURN p` comes back as a list of `{"type": "node" | "edge", "id": ...}` dictionaries. +- A bare `RETURN p` comes back as a list of IDs, node and edge alternating: `[0, 0, 1]` is node 0, edge 0 and node 1. - `count(p)` counts paths, and `RETURN DISTINCT p` groups on whole paths. - `OPTIONAL MATCH p = ... RETURN p IS NULL` works. +Paths are values like any other: `collect(p)` gathers them, lists and maps hold them, and a pattern comprehension can name its path: + +```cypher +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +WITH collect(p) AS routes +RETURN size(routes) AS routeCount, reduce(total = 0, q IN routes | total + length(q)) AS totalHops + +MATCH (a:Person {name: 'Alice'}) +RETURN [p = (a)-[:KNOWS]->+() | length(p)] AS lengths + +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->(b) +RETURN {route: p, hops: length(p)} AS summary +``` + +A property cannot hold a path: `A property value cannot hold a path, in a list or a map or on its own`. Store the IDs or the properties you need instead. + +## Filtering paths with `allReduce` + +`allReduce(acc = init, x IN list | step, predicate)` folds a list like `reduce` and is `true` when the predicate holds for the accumulator after every step. Over the nodes or edges of a path, it tests a running total at each hop: + +```cypher +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +WHERE allReduce(total = 0, n IN nodes(p) | total + n.age, total <= 100) +RETURN b.name, length(p) AS hops +``` + +The ages along `Alice → Bob → Carol` add up to 93, so Bob and Carol are kept. Adding Dave makes 134, so the path to Dave is dropped. + +A map accumulator carries more than one value from step to step. This query keeps the paths along which the `since` years of the edges increase, by comparing each edge with the one before: + +```cypher +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +WHERE allReduce(acc = {}, r IN relationships(p) | {prev: acc.cur, cur: r.since}, acc.prev IS NULL OR acc.prev < acc.cur) +RETURN b.name, length(p) AS hops +``` + +The initial value is never tested, and an empty list gives `true`. See [Lists](/query/functions#lists) for the details. + ## Finding routes For hop-count questions, undirected search, or when you need the properties along a route, combine variable-length paths with comprehensions: diff --git a/query/procedures.mdx b/query/procedures.mdx index f300ead..224f316 100644 --- a/query/procedures.mdx +++ b/query/procedures.mdx @@ -44,8 +44,7 @@ A procedure's output columns can be renamed, filtered and used by the rest of th CALL db.propertyTypes() YIELD * CALL db.labels() YIELD label AS l -WHERE l STARTS -WITH 'P' +WHERE l STARTS WITH 'P' RETURN l CALL db.labels() YIELD label @@ -57,7 +56,7 @@ CALL db.history() YIELD `commit`, nodeCount RETURN `commit`, nodeCount ``` -- **Constant arguments.** Procedure arguments are constants, so pass literal values (for example `db.getNodes([0, 1, 2])`) rather than values read from a row. +- **Constant arguments.** Procedure arguments are constants, so pass literal values (for example `db.getNodes([0, 1, 2])`) or [parameters](/query/cypher_subset#query-parameters) (`db.getNodes($ids)`) rather than values read from a row. `gnn.graphSAGE` is the exception: it reads its arguments from each row, as in `UNWIND $seedBatches AS seeds CALL gnn.graphSAGE(seeds, [10, 5, 2], 42) YIELD ...`. - **`commit` is a keyword.** Yield it with backticks: ``YIELD `commit` ``. - To filter on `valueType`, filter the resulting DataFrame in pandas. - `OPTIONAL CALL` is for subqueries. It is not yet available for procedures. @@ -134,7 +133,7 @@ Dropping and unloading graphs from a running server is not yet available. | `CHANGE DELETE` | Discard the current change | | `CHANGE LIST` | List the open changes on the current graph | | `COMMIT` | Seal the pending writes inside a change. Needed before a later query can `MATCH` newly created entities | -| `LOAD COMMIT ''` | Load a past commit for querying. Only needed when sending queries directly to the REST API. The CLI `checkout` and the Python SDK do it for you | +| `LOAD COMMIT ''` | Load a past commit for querying. The shell's `checkout` and the Python SDK's `checkout()` do it for you. Run it yourself before querying a past commit with the [JavaScript SDK](/jssdk/reference#versioning) or the Python SDK's `set_commit()` | | `MERGE_DATAPARTS` | Compact all [DataParts](/concepts/dataparts) of the graph into one. Run it on main with no change open on any graph | `MERGE_DATAPARTS` **renumbers** all nodes. Re-fetch any cached node IDs afterwards, and rebuild vector indexes keyed by node IDs. diff --git a/query/subqueries.mdx b/query/subqueries.mdx index e83a9fb..da3e66d 100644 --- a/query/subqueries.mdx +++ b/query/subqueries.mdx @@ -40,7 +40,9 @@ WHERE EXISTS { RETURN p.name ``` -The full form can hold a `UNION`, as in the last example, or a [`NEXT`](/query/composition#next) chain. +The full form can hold a `UNION`, as in the last example, a [`NEXT`](/query/composition#next) chain, or a [`WHEN` conditional](/query/composition#inside-exists-count-and-collect). + +For a single pattern, the function form `exists((p)-[:LIKES]->())` is equivalent to `EXISTS { (p)-[:LIKES]->() }`. `exists()` takes a pattern only: test a property with `p.name IS NOT NULL`. ## COUNT @@ -68,7 +70,7 @@ RETURN p.name, COUNT { } AS links ``` -Like `EXISTS`, the body of `COUNT` can hold a `UNION` or a [`NEXT`](/query/composition#next) chain. +Like `EXISTS`, the body of `COUNT` can hold a `UNION`, a [`NEXT`](/query/composition#next) chain or a [`WHEN` conditional](/query/composition#inside-exists-count-and-collect). ## COLLECT @@ -120,6 +122,16 @@ WITH COLLECT { MATCH (p)-[:KNOWS]->+(f) RETURN f } AS reachable RETURN [x IN reachable | x.name] AS names ``` +To collect several values per match, return a map, for example a [map projection](/query/functions#map-projection): + +```cypher +MATCH (p:Person) +RETURN p.name, COLLECT { + MATCH (p)-[:KNOWS]->(f) + RETURN f {.name, .age} +} AS friends +``` + A `COLLECT` is an ordinary list value, so it works anywhere a list does: in `WHERE`, `ORDER BY`, `UNWIND` or `SET`. ```cypher @@ -135,9 +147,8 @@ MATCH (p:Person {name: 'Alice'}) SET p.friendNames = COLLECT { MATCH (p)-[:KNOWS]->(f) RETURN f.name } ``` -- **One column.** The body returns exactly one column. Two columns are reported: `A COLLECT subquery must return exactly one column, not 2`. To collect several values per match, use a [pattern comprehension](/query/aggregation#list-comprehensions) or `collect()` in a `CALL` body. -- **A full query.** The body is `MATCH ... RETURN ...`, not a bare pattern. -- **Not yet available:** collecting map values, and a `NEXT` chain or `WHEN` inside the body. +- **One column.** The body returns exactly one column. Two columns are reported: `A COLLECT subquery must return exactly one column, not 2`. To collect several values per match, return a map as above. +- **A full query.** The body is `MATCH ... RETURN ...`, not a bare pattern. It can hold a `UNION`, a [`NEXT`](/query/composition#next) chain or a [`WHEN` conditional](/query/composition#inside-exists-count-and-collect). ## CALL subqueries @@ -210,5 +221,5 @@ RETURN t.name ``` - `CALL { } IN TRANSACTIONS`, `OPTIONAL CALL` on a procedure, and `WHEN` inside an `EXISTS`, `COUNT` or `COLLECT` body are not yet available. + `CALL { } IN TRANSACTIONS` and `OPTIONAL CALL` on a procedure are not yet available. diff --git a/query/writing.mdx b/query/writing.mdx index b973828..73c465a 100644 --- a/query/writing.mdx +++ b/query/writing.mdx @@ -201,12 +201,35 @@ REMOVE n.score, n.flag ``` - `SET` creates the property if it does not exist yet. Setting a property to `null` clears it, and `REMOVE n.prop` removes it. -- The value can be any expression, including a `CASE`, a function call or a value from another matched entity. -- Address nodes by [native node ID](/query/matching#look-up-nodes-by-native-node-id) for the fastest updates. Batch updates use the same literal `UNWIND` form as reads. +- The value can be any expression, including a `CASE`, a function call, a value from another matched entity or a [parameter](/query/cypher_subset#query-parameters). +- Address nodes by [native node ID](/query/matching#look-up-nodes-by-native-node-id) for the fastest updates. Batch updates use the same `UNWIND` form as reads. - For a one-off embedding, `SET n.emb = (...)` is fine. To write embeddings onto many existing nodes, use [`LOAD EMBEDDING`](/vector-search#embeddings-as-node-properties) instead of a `SET` loop. +### Setting properties from a map + +`SET n += map` adds and updates the properties in the map and keeps the others. `SET n = map` replaces all of them, so the properties missing from the map are removed: + +```cypher +MATCH (n:Person {name: 'Alice'}) +SET n += {city: 'Berlin', score: 1.5} + +MATCH (c:Company {name: 'Acme'}) +SET c = {name: 'Acme Corp', size: 10} + +MATCH (a:Company {name: 'Acme'}), (b:Company {name: 'Globex'}) +SET b += a {.founded} + +MATCH (:Person {name: 'Alice'})-[r:WORKS_AT]->() +SET r += {since: 2019} +``` + +- A key set to `null` in the map removes that property: `SET n += {city: null}`. +- `SET n = {}` and `SET n = null` remove every property of `n`, and `SET n += null` changes nothing. +- The map can come from a parameter, such as `SET n += $props`, from `properties(m)`, or from a [map projection](/query/functions#map-projection), as in the third query, which copies one property. +- Each value must fit the property's type, as with `SET n.prop = ...`. A new property takes the type of its value. + - To update several properties at once, list the assignments: `SET n.a = 1, n.b = 2`. Map forms (`SET n += {...}`, `SET n = {...}`), label changes (`SET n:Label`) and aggregates directly inside `SET` (`SET n.c = count(*)`) are not yet supported. Compute aggregates in a `WITH` first, as in the `Stats` example above. + Label changes (`SET n:Label`) and aggregates directly inside `SET` (`SET n.c = count(*)`) are not yet supported. Compute aggregates in a `WITH` first, as in the `Stats` example above. ## DELETE and DETACH DELETE @@ -256,12 +279,12 @@ FOREACH (i IN range(1, 3) | CREATE (a)-[:OWNS]->(:Item {n: i})) MATCH (n:Person) FOREACH (t IN n.tags | - MERGE (tag:TagNode {name: toString(t)}) + MERGE (tag:TagNode {name: t}) CREATE (n)-[:TAGGED]->(tag) ) ``` -Elements of a list property such as `n.tags` can hold any type, so convert them as you store them (`toString(t)`), as with [map and mixed rows](#small-volumes). +The elements of a list property such as `n.tags` are stored as they are, and a new property takes the type of the values written to it (see [map and mixed rows](#small-volumes)). **Writing only when a condition holds.** Loop over a list that is empty when the condition fails: @@ -289,12 +312,12 @@ RETURN count(*) **Building a new graph?** Beyond a few thousand entities, use [`LOAD PARQUET`](/import_data/parquet). It loads 100,000 nodes and 200,000 edges in about 0.2 seconds. - **Adding edges between nodes that already exist?** Send their [native node IDs](/query/matching#look-up-nodes-by-native-node-id) in pairs, as below: about 10,000 edges in 45 ms. + **Adding edges between nodes that already exist?** Send their [native node IDs](/query/matching#look-up-nodes-by-native-node-id) in pairs, as below: a batch of 10,000 edges takes a few milliseconds. ### Edges between existing nodes -Read each node's ID once together with your own key, map your edge list to ID pairs in your application, and create the edges in batches of about 10,000 pairs: +Read each node's ID once together with your own key, map your edge list to ID pairs in your application, and create the edges in batches of about 10,000 pairs, passed as a [parameter](/query/cypher_subset#query-parameters): ```python ids = client.query("MATCH (n:Person) RETURN n.pid AS pid, id(n) AS id") @@ -303,15 +326,17 @@ pairs = [[key_to_id[src], key_to_id[dst]] for src, dst in edge_list] client.new_change() for i in range(0, len(pairs), 10_000): - batch = pairs[i:i + 10_000] - client.query(f"UNWIND {batch} AS p " - "MATCH (a) WHERE a = p[0] MATCH (b) WHERE b = p[1] " - "CREATE (a)-[:KNOWS]->(b)") + client.query( + "UNWIND $pairs AS p " + "MATCH (a) WHERE a = p[0] MATCH (b) WHERE b = p[1] " + "CREATE (a)-[:KNOWS]->(b)", + {"pairs": pairs[i:i + 10_000]}, + ) client.query("CHANGE SUBMIT") client.checkout() ``` -Edge properties ride along in each row: `UNWIND [[12, 34, 2021], ...] AS p ... CREATE (a)-[:KNOWS {since: p[2]}]->(b)`. Use IDs rather than matching both endpoints by a key property (`MATCH (a:Person {pid: p[0]})`), which reads every `Person` node for each row. +In our tests a batch of 10,000 pairs takes about 6 ms as a parameter, against about 25 ms with the same list written into the query text. Edge properties ride along in each row: `UNWIND $rows AS p ... CREATE (a)-[:KNOWS {since: p[2]}]->(b)` with rows such as `[12, 34, 2021]`. Use IDs rather than matching both endpoints by a key property (`MATCH (a:Person {pid: p[0]})`), which reads every `Person` node for each row. ### Small volumes @@ -328,16 +353,19 @@ UNWIND [['Brest', 'FR-29'], ['Rennes', 'FR-35']] AS row CREATE (:City {name: row[0], code: row[1]}) ``` -Rows can also be maps, or lists of mixed types. Their values, like the elements of a list property, can hold any type, so convert each one as you store it: +Rows can also be maps, or lists of mixed types, and their values are stored as they are: ```cypher UNWIND [{name: 'Brest', pop: 139000}, {name: 'Rennes', pop: 220000}] AS row -CREATE (:City {name: toString(row.name), pop: toInteger(row.pop)}) +CREATE (:City {name: row.name, pop: row.pop}) UNWIND [['Brest', 139000], ['Rennes', 220000]] AS row -CREATE (:City {name: toString(row[0]), pop: toInteger(row[1])}) +CREATE (:City {name: row[0], pop: row[1]}) ``` +- A new property takes the type of the values written to it, here a String `name` and an Integer `pop`. One write cannot give it two types: `One write gives the new property 'v' two types: 'Integer' and 'String'`. +- An existing property keeps its type: `Cannot set a property of type 'Int64' to a value of type 'String'`. Convert such a value first, for example with `toInteger(row.pop)`. + `LOAD CSV` can drive writes too. Create the nodes, `COMMIT`, then create the edges (see [CSV import](/import_data/csv)). Alternatively `MERGE` both endpoints in a single pass, which deduplicates across the rows of one query so no `COMMIT` is needed: ```cypher @@ -349,18 +377,11 @@ LOAD CSV 'knows.csv' WITH HEADERS AS row Use `UNWIND` or [`FOREACH`](#foreach) to loop over a list. -**Building the query text in Python.** Values go in the query text, so escape quotes in your application: +**From Python, pass the rows as a parameter.** Values in a parameter need no quoting or escaping, and take less room than the same values written into the query text: ```python -def lit(v): - if isinstance(v, str): - return "'" + v.replace("\\", "\\\\").replace("'", "\\'") + "'" - if isinstance(v, bool): - return "true" if v else "false" - return repr(v) - -names = ["Alice", "O'Brien"] -client.query(f"UNWIND [{', '.join(lit(n) for n in names)}] AS nm MERGE (:Person {{name: nm}})") +rows = [{"name": "Brest", "pop": 139000}, {"name": "O'Brien Town", "pop": 1200}] +client.query("UNWIND $rows AS row MERGE (c:City {name: row.name}) SET c.pop = row.pop", {"rows": rows}) ``` -Keep each request under 1 MiB, the server's limit. A larger one is refused with `REQUEST_TOO_BIG`. +A request, the query text and its parameters together, can be up to 1 MiB. Split larger batches, as in the edge recipe above. diff --git a/quickstart.mdx b/quickstart.mdx index 92959cc..93e7535 100644 --- a/quickstart.mdx +++ b/quickstart.mdx @@ -81,9 +81,8 @@ The steps below walk you through setting up a Python project, installing the Tur from turingdb import TuringDB # Create TuringDB client - # set host parameter to the URL (as string) on which TuringDB is running, - # default "http://localhost:6666" - client = TuringDB(host="http://localhost:6666") + # host and port of the running TuringDB server (default localhost:6666) + client = TuringDB(host="localhost", port=6666) # Create a new graph called my_graph client.create_graph("mygraph") @@ -134,3 +133,7 @@ The steps below walk you through setting up a Python project, installing the Tur You are done! + + + Building a Node.js or web application? The `turingdb` npm package queries the same server from JavaScript or TypeScript: see the [JavaScript SDK](/jssdk/get_started). + diff --git a/releases/overview.mdx b/releases/overview.mdx new file mode 100644 index 0000000..691bcff --- /dev/null +++ b/releases/overview.mdx @@ -0,0 +1,36 @@ +--- +title: "Release Notes" +sidebarTitle: "All Releases" +description: "What changed in each TuringDB release, and how to upgrade." +--- + +TuringDB releases the server, the Python SDK (`turingdb` on PyPI) and the JavaScript SDK (`turingdb` on npm) together, under the same version number. Upgrade them together. + +## Latest release + + + + Query parameters, the binary protocol over HTTP/1.1 by default, a WebAssembly-based TypeScript client on npm, `allReduce`, `COLLECT` and other subqueries that hold any query, and a point type with haversine distances. + + + +## All releases + +| Version | Released | Highlights | +| ------- | -------- | ---------- | +| [3.1](/releases/v3-1) | November 9, 2026 | Query parameters, the binary protocol over HTTP/1.1 by default, a WebAssembly-based TypeScript client on npm, `allReduce`, `COLLECT` and other subqueries, points with haversine distances | +| [3.0.2](https://github.com/turing-db/turingdb/releases/tag/v3.0.2) | October 26, 2026 | Label expressions, `NEXT`, `COLLECT` subqueries, `FOREACH`, quantified path patterns with several hops, parameters for labels and edge types, `keys()` and `properties()`, `stDev` and percentiles | +| [3.0.1](https://github.com/turing-db/turingdb/releases/tag/v3.0.1) | October 8, 2026 | `WHEN ... THEN ... ELSE`, map access, `reduce`, relationships with both arrow heads, the common numeric and string functions | +| 3.0 | October 2, 2026 | TuringDB v3: every query runs on the new MLIR-based engine, which supports most of openCypher | +| [1.37](https://github.com/turing-db/turingdb/releases/tag/v1.37) and earlier | until August 24, 2026 | The previous engine | + +Each GitHub release lists every change that went into that version. + +## Staying up to date + +```bash +pip install -U turingdb # or: uv add 'turingdb>=3.1' +npm install turingdb@latest +``` + +Restart the server after upgrading, so that it runs the new `turingdb`. See the [Get Started guide](/quickstart) for a first install. diff --git a/releases/v3-1.mdx b/releases/v3-1.mdx new file mode 100644 index 0000000..659f5be --- /dev/null +++ b/releases/v3-1.mdx @@ -0,0 +1,354 @@ +--- +title: "TuringDB 3.1" +sidebarTitle: "TuringDB 3.1" +description: "Released November 9, 2026. Query parameters, the binary protocol over HTTP/1.1 by default, a WebAssembly-based TypeScript client on npm, allReduce, COLLECT and other subqueries that hold any query, and a point type with haversine distances." +--- + +TuringDB 3.1 brings query parameters, makes the binary protocol over HTTP/1.1 the way every client talks to the server, and adds a TypeScript client on npm, built on a WebAssembly build of the engine's decoder. Cypher gains `allReduce`, subqueries that hold any query, and a point type with great-circle distances. Most queries run unchanged. The main change for existing applications is how the Python SDK connects, see [Upgrading from 3.0](#upgrading-from-3-0). + + + +```bash pip +pip install -U turingdb +``` + +```bash uv +uv add 'turingdb>=3.1' +``` + +```bash npm +npm install turingdb +``` + + + +Released **November 9, 2026** · [Release on GitHub](https://github.com/turing-db/turingdb/releases/tag/v3.1) · [All changes since 3.0.2](https://github.com/turing-db/turingdb/compare/v3.0.2...v3.1) + +## Highlights + + + + Pass values next to the query, as in `WHERE n.age > $min` or `UNWIND $rows`: no quoting, no escaping, and faster bulk writes. + + + The default and only protocol: results stream as typed binary columns, through standard HTTP proxies. + + + `npm install turingdb` queries TuringDB from Node.js and the browser, decoding results in WebAssembly. + + + Fold a list and test a predicate after every step: a running total or an ordering along a path. + + + `UNION`, `NEXT` and `WHEN` inside `COLLECT`, `EXISTS`, `COUNT` and `CALL`, and `COLLECT` of maps and paths. + + + A point property type, cartesian or WGS-84, with great-circle distances in metres and bounding boxes. + + + +## Query parameters + +A `$name` parameter now stands for a value anywhere an expression goes, as well as for a label or an edge type. Values travel next to the query text, so they need no quoting or escaping: + + + +```python Python +client.query("MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", {"minAge": 30}) + +# inside a change +client.query( + "UNWIND $rows AS row CREATE (:City {name: row.name, pop: row.pop})", + {"rows": [{"name": "Brest", "pop": 139000}, {"name": "Rennes", "pop": 220000}]}, +) +``` + +```js JavaScript +await client.query("MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", { + parameters: { minAge: 30 }, +}); +``` + +```text Shell +:param minAge => 30 +MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name +``` + + + +- Parameters work in `WHERE`, `UNWIND`, property maps, `CREATE`, `MERGE`, `SET n += $props` and procedure arguments, and still stand for labels and edge types: `MATCH (n:$label)`. +- A parameter holds a string, a number, a boolean, `null`, a list or a map. From Python it can also hold a time-zone-aware `datetime` (a DateTime) or a `timedelta` (a Duration), and the shell's `:param` takes any Cypher expression, such as `datetime('2024-01-01T00:00:00Z')`. +- Parameters travel faster than text: creating 10,000 edges from a list of node-ID pairs takes about 6 ms with the list as a parameter, against about 25 ms with the same list written into the query. +- `SKIP`, `LIMIT` and the vector of a `VECTOR SEARCH` still take literals. + +See [Query parameters](/query/cypher_subset#query-parameters) for the details, and [IDs in parameters](/query/matching#look-up-nodes-by-native-node-id) for the fastest lookups by node ID. + +## Binary protocol by default + +TuringDB's binary protocol is now the default, and the only protocol the server speaks, on port 6666. It runs over HTTP/1.1: a query is one request to `/query`, and its result streams back as binary packets, one per HTTP chunk, on a connection that stays open for the next query: + +```text +POST /query?graph=people&commit=head&change=head HTTP/1.1 +Content-Type: application/turing-proto +Authorization: Bearer (with -auth-on) + + the query text, then its parameters, in binary + +HTTP/1.1 200 OK +Content-Type: application/turing-proto +Transfer-Encoding: chunked +Connection: Keep-Alive + + one binary packet per chunk: the column header, the column chunks, the end +``` + +- **Typed columns end to end.** A result streams as typed columns, in chunks of up to 65,536 rows, which each client turns straight into its own types: a pandas DataFrame in Python, typed arrays and `Column` objects in JavaScript. +- **HTTP infrastructure works as is.** Standard reverse proxies such as nginx forward the protocol with a plain `proxy_pass`, browsers reach it with `fetch`, and authentication uses the standard `Authorization: Bearer` header, which a proxy can also add. +- **Every client speaks it.** The Python SDK uses it by default, `TuringDB(host="localhost", port=6666)`, and so do the new TypeScript client, the shell's `connect ` and the visualizer. + +```mermaid +graph LR + py(["Python SDK
pip install turingdb"]) + js(["TypeScript client
npm install turingdb"]) + shell(["Remote shell
connect <addr> <port>"]) + ui(["Visualizer
turingdb -ui"]) + db[("TuringDB server
binary protocol over HTTP/1.1, port 6666")] + py --> db + js --> db + shell --> db + ui --> db +``` + +Use one of the clients rather than writing requests by hand. See [Connecting clients](/commands#connecting-clients). + +## A TypeScript client on npm, built on WebAssembly + +The `turingdb` npm package queries TuringDB from TypeScript and JavaScript. It decodes the streamed columns with a WebAssembly build of the engine's own decoder, so the browser, Node.js and the server run the same decoding code. It runs in Node.js 18 or later and in the browser, has no dependencies, and ships with TypeScript declarations. + +```ts +import { TuringClient, TuringQueryError } from "turingdb"; + +const client = new TuringClient({ url: "http://localhost:6686/query", graph: "people" }); + +try { + const [[names, ages]] = await client.queryData( + "MATCH (p:Person) WHERE p.age >= $age RETURN p.name, p.age ORDER BY p.name", + { parameters: { age: 30 } }, + ); + console.log(names, ages); // [ 'Marie' ] [ 33 ] +} catch (e) { + if (e instanceof TuringQueryError) console.error(e.status, e.message); +} +``` + +- **Typed columns.** `query()` returns `Column` objects: 64-bit integers and node IDs as `bigint`, numeric columns as typed arrays with no per-row work, embeddings as `Float32Array` and points as `Point` objects. `queryData()` returns plain arrays. +- **The whole workflow.** Parameters, changes (`CHANGE NEW`, `setChange()`, `CHANGE SUBMIT`), past commits through the `commit` option, cancellation with an `AbortSignal`, and `-auth-on` tokens. +- **Node.js and the browser.** Node.js refuses port 6666, so start the server on another one, such as `turingdb -p 6686`. A web page reaches the server through a proxy on its own origin, and Vite, webpack 5 and other bundlers ship the `.wasm` decoder with the build. + +Read [Get started](/jssdk/get_started) for a walkthrough, including a Vite and nginx setup for web applications, and the [Reference](/jssdk/reference) for every option and type. + +## `allReduce` + +`allReduce(acc = init, x IN list | step, predicate)` folds a list like `reduce`, and is `true` when the predicate holds for the accumulator after **every** step. It stops at the first step that fails. Over the nodes or edges of a path, it tests a condition at each hop: + +```cypher +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +WHERE allReduce(total = 0, n IN nodes(p) | total + n.age, total <= 100) +RETURN b.name, length(p) AS hops +``` + +The ages along `Alice → Bob → Carol` add up to 93, so the paths to Bob and Carol are kept, while adding Dave makes 134. A map accumulator carries more than one value from step to step, for example to keep the paths along which the `since` years of the edges increase: + +```cypher +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +WHERE allReduce(acc = {}, r IN relationships(p) | {prev: acc.cur, cur: r.since}, acc.prev IS NULL OR acc.prev < acc.cur) +RETURN b.name, length(p) AS hops +``` + +The initial value is never tested, an empty list gives `true`, and a `null` list gives `null`. Paths are values too: `collect(p)` gathers them, lists and maps hold them, and a pattern comprehension can name its path, as in `[p = (a)-[:KNOWS]->+() | length(p)]`. See [Filtering paths with `allReduce`](/query/paths#filtering-paths-with-allreduce). + +## COLLECT and other subqueries + +Every subquery now holds any query. `UNION`, `NEXT` chains and `WHEN` conditionals work inside `COLLECT`, `EXISTS`, `COUNT` and `CALL` bodies, and a `WHEN` branch can hold a `UNION`: + +```cypher +MATCH (p:Person {name: 'Alice'}) +RETURN COLLECT { + MATCH (p)-[:KNOWS]->(f) + RETURN f AS friend + NEXT + MATCH (friend)-[:KNOWS]->(fof) + RETURN fof {.name, .age} +} AS twoHopsAway + +MATCH (p:Person) +WHERE EXISTS { + WHEN p.age > 30 THEN MATCH (p)-[:KNOWS]->(f) RETURN f +} +RETURN p.name + +MATCH (p:Person) +RETURN p.name, COUNT { + WHEN p.city = 'London' THEN MATCH (p)-[:KNOWS]->(f) RETURN f AS x + ELSE MATCH (p)-[:LIKES]->(x) RETURN x +} AS links +``` + +- `COLLECT { }` and `collect()` gather maps and paths, so a subquery can collect several values per match: `COLLECT { MATCH (p)-[:KNOWS]->(f) RETURN f {.name, .age} }`. +- The branches of a `UNION` or a `WHEN` can return different types in one column. +- `exists()` takes a pattern as a function, as in Neo4j 5: `RETURN p.name, exists((p)-[:LIKES]->()) AS likes`. + +See [Subqueries](/query/subqueries) and [NEXT and WHEN](/query/composition#inside-exists-count-and-collect). + +## Points and haversine distance + +Points are a new property type, 2D or 3D, on a plane (cartesian) or on the Earth (WGS-84). `point.distance` gives the great-circle distance in metres between two WGS-84 points, by the haversine formula on a sphere of the mean Earth radius: + +```cypher +CREATE (:City {name: 'Paris', at: point({latitude: 48.8566, longitude: 2.3522})}), + (:City {name: 'London', at: point({latitude: 51.5074, longitude: -0.1278})}) + +MATCH (a:City {name: 'London'}), (b:City {name: 'Paris'}) +RETURN round(point.distance(a.at, b.at) / 1000) AS km + +MATCH (c:City) +WHERE point.withinBBox(c.at, point({latitude: 48, longitude: 1}), point({latitude: 50, longitude: 3})) +RETURN c.name +``` + +The second query returns `344`, and the third returns Paris. + +- **Building and reading.** `point({x, y})`, `point({x, y, z})`, `point({longitude, latitude})` and `point({longitude, latitude, height})`, read back through `p.x`, `p.latitude`, `p.crs`, `p.srid` and the other components. +- **Distances.** Straight-line between cartesian points. Great-circle on the Earth's mean radius, 6,371,008.8 m, between WGS-84 points, and combined with the height difference in 3D. Neo4j uses a radius of 6,378,140 m, so its distances are about 0.11% longer. +- **Bounding boxes.** `point.withinBBox(p, lowerLeft, upperRight)`, with boxes that can cross the 180th meridian. +- **Sorting and aggregating.** Points compare, sort, group and deduplicate, and `min`, `max` and `avg`, the centroid, work over them. +- **Importing.** `LOAD JSONL ... WITH POINTS [...]` reads JSON objects as points, and `LOAD PARQUET` reads the WKB point columns of GeoParquet files. +- **Clients.** Points come back as `turingdb.Point` in Python and as `Point` objects in TypeScript. + +See [Points](/query/functions#points). + +## Also in 3.1 + +### Map projection and map access + +```cypher +MATCH (n:Person)-[:WORKS_AT]->(c:Company) +RETURN c.name, collect(n {.name, .age, senior: n.age > 30}) AS staff + +WITH {a: {b: 2}, label: 'x'} AS m +RETURN m.a.b, m['label'] +``` + +`n {.*}` takes every property, and a map can hold nodes and datetimes and read through them: `m.p.name`, `m.d.year`. See [Map projection](/query/functions#map-projection). + +### Setting properties from a map + +```cypher +MATCH (n:Person {name: 'Alice'}) +SET n += {city: 'Berlin', score: 1.5} + +MATCH (c:Company {name: 'Acme'}) +SET c = {name: 'Acme Corp', size: 10} +``` + +`SET n += map` adds and updates properties, `SET n = map` replaces them all, and a key set to `null` removes its property. See [Setting properties from a map](/query/writing#setting-properties-from-a-map). + +### New functions and operators + +| Added | Example | +| ----- | ------- | +| `IS` label tests | `WHERE n IS Person\|Company` | +| `elementId(x)` | The same integer as `id(x)` | +| `isEmpty(x)` | `isEmpty([])`, `isEmpty('')` and `isEmpty({})` are `true` | +| `coll.indexOf(list, value)` | `coll.indexOf(['a', 'b'], 'b')` is `1` | +| `split` on several delimiters | `split('a,b;c', [',', ';'])` is `['a', 'b', 'c']` | +| `datetime` of a map | `datetime({year: 2024, month: 3, day: 15, hour: 10})` | +| `sum` and `avg` of durations | `sum(d)` over `Duration` values | + +### A new parser + +A hand-written parser replaces the generated one, and reads queries the way Neo4j does: + +- `/* */` block comments, and keywords as names: `RETURN n.name AS count, n.age AS order`. +- `STARTS WITH`, `ENDS WITH` and `CONTAINS` bind more loosely than `+`, so `'a' + 'bc' ENDS WITH 'bc'` is `true`. +- Dotted chains read through nested values: `m.a.b`, `m['key']`, `ms[1].a`. + +### More of what you write just works + +- Writes store values read from maps and lists directly: `CREATE (:City {name: row.name, pop: row.pop})` needs no `toString()` or `toInteger()`. A new property takes the type of its values. +- `ORDER BY` after `DISTINCT` or beside an aggregate sorts by a key or a property of a returned item: `RETURN DISTINCT {age: n.age} AS m ORDER BY m.age`. +- `WHERE n IN [0, 1, 2]` looks the nodes up by ID directly, like `UNWIND [0, 1, 2] AS x MATCH (n) WHERE n = x`. +- `range()` takes bounds read from maps and lists, and `sum()` of integers read from maps stays an Integer. +- A runtime type error names the type the row holds: `toUpper() reads a string, not 'Integer'`. +- `gnn.graphSAGE` reads its arguments from each row, and rejects a seed that names no node. +- The planner starts a pattern from the node expected to produce the fewest rows, with estimates built from the number of edges of each type, and sizes a hash join by its smaller side. + +### Python SDK + +- `TuringDB(host="localhost", port=6666)` connects over the binary protocol, as soon as it is created. `type` is `"native"`, the default, or `"embedded"`. +- A `DateTime` or a `Duration` inside a list or a map comes back as a time-zone-aware `datetime.datetime` or a `datetime.timedelta`, rather than a string. +- `list_available_graphs()` works on every backend, the embedded one included. +- `try_reach(timeout)` and `warmup(timeout)` give up after `timeout` seconds. +- The package no longer depends on `httpx` and `orjson`. + +See the [Python SDK reference](/pythonsdk/reference). + +## Upgrading from 3.0 + + + + A 3.1 client speaks the protocol of a 3.1 server, so upgrade `turingdb` on PyPI, `turingdb` on npm and the server at the same time, then restart the server: + + ```bash + pip install -U turingdb + turingdb stop && turingdb -demon + ``` + + A client and a server of different versions fail with a protocol error, see [Troubleshooting](/troubles/troubles#a-client-and-a-server-of-different-versions). + + + Pass the host and the port separately: + + ```python + from turingdb import TuringDB + + client = TuringDB(host="localhost", port=6666) # or TuringDB() + ``` + + A URL in `host` is refused with `host must be a host name or IP address, not a URL`. The client now connects as soon as it is created, so a server that is not running is reported right away. + + + `RETURN p` returns a path as a list of IDs, node and edge alternating: `[0, 0, 1]` is node 0, edge 0 and node 1. Code that read the `type` and `id` keys of each step should read the list instead. + + + A few results now follow Neo4j: + + | Expression | 3.0 | 3.1 | + | ---------- | --- | --- | + | `[1, 2] + null` | `[1, 2, null]` | `null` | + | `{a: null} = {a: 1}` | `false` | `null` | + | `'a' + 'bc' ENDS WITH 'bc'` | `'atrue'` | `true` | + | `duration({hours: null})` | `null` | an error naming the unit | + | `sum(row.v)` over integers read from maps | a Double | an Integer | + + + Optional, but recommended: replace the values your application writes into query text, and the code that escapes them, with [parameters](/query/cypher_subset#query-parameters). They are safer and faster. + + + +## Removed + +- **The JSON API.** The server answers the binary protocol only, so scripts that sent JSON queries to it, with `curl` for example, need one of the clients. +- **In the Python SDK:** `type="json"`, a URL in `host`, the `HTTPClient` class and the `TURINGDB_TYPE` environment variable. +- **On the server:** the `USE_TURING_PROTO` environment variable, since the binary protocol is always on, and the `REQUEST_TOO_BIG` error code. A request over 1 MiB is now refused with a protocol error that names the limit. + +## Known limitations + +These are not available yet, and are on the roadmap: + +- Parameters in `SKIP` and `LIMIT`, and as the vector of a `VECTOR SEARCH`. +- Regular expressions (`=~`) and `--` comments. +- `rand()`, `date()`, `timestamp()`, `atan2()`, `isNaN()` and `round(x, digits)`. +- `SET n:Label`, `CALL { } IN TRANSACTIONS` and `OPTIONAL CALL` on a procedure. + +Tell us which ones matter to you on [Discord](https://discord.gg/dMM48ns5VY) or by email at [team@turingdb.ai](mailto:team@turingdb.ai). For the full list of changes, see the [v3.1 release on GitHub](https://github.com/turing-db/turingdb/releases/tag/v3.1). diff --git a/troubles/troubles.mdx b/troubles/troubles.mdx index f60b16a..4fe35f6 100644 --- a/troubles/troubles.mdx +++ b/troubles/troubles.mdx @@ -27,37 +27,81 @@ turingdb -reset-default -turing-dir /path/to/your/turing/dir --- -## `Invalid response from the server` or `Proto header dataLen does not match chunk payload size` +## `host must be a host name or IP address, not a URL` -These two errors mean the client and the server disagree on the protocol. A TuringDB server speaks **either** JSON over HTTP **or** the binary protocol, chosen when it starts: +Pass the host and the port separately: -| Error | Cause | Fix | -|---|---|---| -| `Invalid response from the server: Expecting value...` from `TuringDB(host=...)` | The server was started with `USE_TURING_PROTO=1` (binary mode) but you used the default JSON client | Connect with `TuringDB(type="native", host="localhost", port=6666)`, or restart the server without `USE_TURING_PROTO` | -| `Proto header dataLen does not match chunk payload size` from `TuringDB(type="native", ...)` | The server is a normal JSON server | Use the default `TuringDB(host="http://localhost:6666")`, or restart the server with `USE_TURING_PROTO=1` | +```python +client = TuringDB(host="localhost", port=6666) +``` -See [client backends](/pythonsdk/reference#client-backends) for details. +`TuringDB()` alone connects to `localhost:6666`. Similarly, `Unknown type: ... (expected 'native' or 'embedded')` means the `type` argument names a backend that does not exist: leave `type` out to connect to a server. See [client backends](/pythonsdk/reference#client-backends). --- -## `The request body must be a JSON object` +## `Failed to connect to localhost:6666: Connection refused` + +No server is listening at that address. Start one with `turingdb -demon`, or check the port it was started with (`-p`). The Python client connects as soon as it is created, so this error comes from `TuringDB(...)` itself. + +--- + +## A client and a server of different versions + +A client speaks the protocol of the server version it was released with, so upgrade the SDKs and the server together. A mismatch shows up as one of these errors: + +| Error | Client | +|---|---| +| `httpx.HTTPStatusError: Client error '400 Bad Request' for url '.../query?graph=default'` | A Python SDK older than 3.1 against a 3.1 server | +| `Proto header dataLen does not match chunk payload size` | A 3.1 Python SDK against an older server | +| `PROTOCOL_ERROR`: `A packet payload of ... bytes is more than the 67108864 bytes the binary protocol allows` | A 3.1 JavaScript SDK against an older server | + +Upgrade with `pip install -U turingdb` (or `uv add 'turingdb>=3.1'`) and `npm install turingdb@latest`, then restart the server with the upgraded `turingdb`. + +--- + +## `The request is over the server limit of 1048576 bytes` + +A request can be up to 1 MiB, the query text and its parameters together. Split a large batch into several queries, for example 10,000 node-ID pairs per request, or load bulk data from a file with [`LOAD PARQUET`](/import_data/parquet) or [`LOAD JSONL`](/import_data/jsonl). Passing a list as a [parameter](/query/cypher_subset#query-parameters) also takes far less room than writing it into the query text. + +When the parameters alone are too large, the Python SDK refuses them before sending: `The query parameters exceed 1048576 bytes`. + +--- + +## `The client is not connected: call connect() before sending a query` + +The server closes the connection after refusing a request, for example one over 1 MiB or one with a wrong token, and the Python client stays disconnected. Reconnect, then set the graph again: + +```python +client.reconnect() +client.set_graph("mygraph") +``` + +--- + +## `Unauthorized: missing or invalid authentication token` + +The server was started with `-auth-on` and your client did not send a valid token. In Python, pass `token="..."` to `TuringDB(...)`, or set the `TURINGDB_AUTH_TOKEN` environment variable. In JavaScript, pass `authToken` to `TuringClient`. The Python SDK raises it as `TuringDBException: Protocol error from server: Unauthorized: ...`, and the JavaScript SDK as a `TuringQueryError` with status `PROTOCOL_ERROR`. + +--- -The server expects each query as a JSON body, `{"query": "...", "parameters": {...}}`. Two common causes: +## JavaScript: `fetch failed`, `bad port` or `Failed to fetch` -- **An older SDK against a 3.0.2 server.** Upgrade the Python SDK to the same version as the server: `pip install 'turingdb>=3.0.2'`. -- **A raw HTTP request with the query as plain text.** Send JSON instead, see the [HTTP API](/commands#http-api). +- **`fetch failed` with the cause `bad port` in Node.js.** `fetch` refuses port 6666, the server's default. Start the server on another port, such as `turingdb -p 6686`. +- **`fetch failed` with the cause `ECONNREFUSED`.** No server is listening at the client's `url`. +- **`Failed to fetch` in a browser.** The page and the server are on different origins, and the server sends no CORS headers. Forward `/query` from the page's own origin to the server, see [In the browser](/jssdk/get_started#in-the-browser). +- **`expected magic word 00 61 73 6d` with Vite's dev server.** Pre-bundling moved the client away from its `.wasm` decoder. Add `optimizeDeps: { exclude: ["turingdb"] }` to `vite.config.ts`. --- -## `REQUEST_TOO_BIG` +## `Graph 'mygraph' is on disk but not loaded` -A request can be up to 1 MiB. Split a large `UNWIND` into batches, for example 10,000 node-ID pairs per request, or load bulk data from a file with [`LOAD PARQUET`](/import_data/parquet) or [`LOAD JSONL`](/import_data/jsonl). +After a restart, the server loads only the `default` graph. Load the others with `LOAD GRAPH mygraph` or `client.load_graph("mygraph")`, or start the server with `-load mygraph`. --- -## `401 Unauthorized` +## `Commit '...' is not loaded to memory` -The server was started with `-auth-on` and your client did not send a valid token. Pass `token="..."` to `TuringDB(...)`, or set the `TURINGDB_AUTH_TOKEN` environment variable. The JSON client raises this as an `httpx.HTTPStatusError`. +A past commit has to be loaded before you query it. The Python SDK's `client.checkout(commit=...)` and the shell's `checkout` do it for you. With `set_commit()` in Python or the `commit` option of the JavaScript SDK, run `LOAD COMMIT ''` first. See [Time travel](/graph_dev/time_travelling#lazy-commit-loading). --- @@ -81,7 +125,7 @@ Entities created earlier in a change are visible to a later `MATCH` once you run ## `Function 'X' does not exist` -A few functions are not yet available in TuringDB, such as `rand()`, `date()`, `timestamp()` and `elementId()`. In v3, `type()` replaces `edgeType()`. See [Functions and data types](/query/functions) for the full list. +A few functions are not yet available in TuringDB, such as `rand()`, `date()` and `timestamp()`. In v3, `type()` replaces `edgeType()`. See [Functions and data types](/query/functions) for the full list. --- @@ -91,4 +135,4 @@ If you have any feedback, bugs to report, or features requests please send us an Or send us a message on [Discord](https://discord.gg/dMM48ns5VY) -Please do not hesitate to contact us for any requests or if you need help with anything as simple or complex as it may seem. \ No newline at end of file +Please do not hesitate to contact us for any requests or if you need help with anything as simple or complex as it may seem. diff --git a/tutorials/create_graph.mdx b/tutorials/create_graph.mdx index de9e6d2..dea3405 100644 --- a/tutorials/create_graph.mdx +++ b/tutorials/create_graph.mdx @@ -67,7 +67,7 @@ def build_create_query(graph): return query[:-2] # Initialize TuringDB client (connect to a locally running server) -client = TuringDB(host="http://localhost:6666") +client = TuringDB(host="localhost", port=6666) # Create and set up the graph client.create_graph("imported_gml_graph") diff --git a/vector-search.mdx b/vector-search.mdx index f2924ff..42f37fd 100644 --- a/vector-search.mdx +++ b/vector-search.mdx @@ -212,7 +212,7 @@ This removes the index and frees the associated resources. ```python from turingdb import TuringDB - client = TuringDB(host="http://localhost:6666") + client = TuringDB(host="localhost", port=6666) # 1. Create a vector index for product embeddings client.query("CREATE VECTOR INDEX product_vectors WITH DIMENSION 384 METRIC COSINE") From de366849aee872023aa9caefe0c8f5866cca5e3f Mon Sep 17 00:00:00 2001 From: Remy Boutonnet Date: Mon, 9 Nov 2026 21:57:25 +0100 Subject: [PATCH 2/2] Fit the code examples to the width of their blocks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rewraps the code lines that made a block scroll sideways, on every page, and makes the opening of the 3.1 release page gentler. Code layout (128 overflowing blocks before, 6 after): - Cypher breaks before relationships, after RETURN items, and around CASE, FOREACH, COLLECT bodies and point calls; VECTOR SEARCH and CALL put YIELD on its own line; - Python splits long query strings by implicit concatenation and moves trailing comments above their line; JavaScript does the same with +; - the six blocks left are JSONL records, a Jupyter log and CLI sessions, which must stay one line per record or command. Checked against the previous text: Python ASTs and JavaScript token streams are unchanged, and every changed Cypher query gives the same result on a 3.1 server. The few intended changes were run too: - the shortest-path recipe passes node IDs as a $ids parameter instead of an f-string (same stations); - the vector CSV writer, the release and JS reference parameter examples, and the JS get-started loop use a named variable (same output); - allReduce examples name the accumulator t, and query/functions names its columns under10 and under5 (same values); - the cosine/euclidean pair in vector-search is two queries. Release page: simpler intro paragraphs, and a first point in the parser section on its speed (23 µs instead of 54 µs for a 12-line query). --- benchmarks/technical-report.mdx | 67 ++++++++++++++-------- concepts/columnar_storage.mdx | 3 +- graph_dev/create_graph.mdx | 3 +- graph_dev/create_nodes_edges.mdx | 7 ++- graph_dev/graph_examples.mdx | 13 +++-- graph_dev/time_travelling.mdx | 3 +- graph_dev/update_properties.mdx | 20 +++++-- import_data/csv.mdx | 11 +++- import_data/gml.mdx | 3 +- import_data/jsonl.mdx | 3 +- import_data/neo4j.mdx | 3 +- import_data/parquet.mdx | 27 ++++++--- index.mdx | 17 ++++-- jssdk/get_started.mdx | 44 +++++++++++---- jssdk/reference.mdx | 54 ++++++++++++------ pythonsdk/get_started.mdx | 23 +++++--- pythonsdk/reference.mdx | 47 +++++++++++----- query/aggregation.mdx | 36 +++++++++--- query/cheatsheet.mdx | 57 ++++++++++++++----- query/composition.mdx | 18 ++++-- query/cypher_subset.mdx | 89 +++++++++++++++++++++++------- query/functions.mdx | 81 ++++++++++++++++++--------- query/matching.mdx | 27 ++++++--- query/paths.mdx | 40 ++++++++++---- query/procedures.mdx | 13 +++-- query/subqueries.mdx | 6 +- query/writing.mdx | 70 +++++++++++++++++------ quickstart.mdx | 11 ++-- releases/v3-1.mdx | 78 +++++++++++++++++++------- shortest-path.mdx | 32 ++++++++--- tutorials/example_notebooks.mdx | 11 ++-- vector-search.mdx | 95 ++++++++++++++++++++++---------- 32 files changed, 727 insertions(+), 285 deletions(-) diff --git a/benchmarks/technical-report.mdx b/benchmarks/technical-report.mdx index 43b66cf..d733bb5 100644 --- a/benchmarks/technical-report.mdx +++ b/benchmarks/technical-report.mdx @@ -654,10 +654,13 @@ The benchmark script generates a timestamped report and optionally updates the R ### Property-Based Filtering ``` - MATCH (p:Person {name: 'John'})-[:PARTY_TO]->(c:Crime) RETURN p, c + MATCH (p:Person {name: 'John'})-[:PARTY_TO]->(c:Crime) + RETURN p, c MATCH (p:Person {surname: 'Smith'})-[r]->(n) RETURN p MATCH (p:Person)-[r]->(n) WHERE p.surname = 'Smith' RETURN p - MATCH (p1:Person)-[:PARTY_TO]->(c:Crime)<-[:PARTY_TO]-(p2:Person) WHERE p1 <> p2 RETURN p1.name, p2.name, c.type + MATCH (p1:Person)-[:PARTY_TO]->(c:Crime) + <-[:PARTY_TO]-(p2:Person) + WHERE p1 <> p2 RETURN p1.name, p2.name, c.type ``` ### Aggregations @@ -671,8 +674,10 @@ The benchmark script generates a timestamped report and optionally updates the R ### Relationship Type Traversal ``` - MATCH (p:Person)-[:PARTY_TO]->(c:Crime) RETURN p.name, p.surname, c.type - MATCH (p1:Person)-[:KNOWS]->(p2:Person)-[:PARTY_TO]->(c:Crime) RETURN p1.name, p2.name + MATCH (p:Person)-[:PARTY_TO]->(c:Crime) + RETURN p.name, p.surname, c.type + MATCH (p1:Person)-[:KNOWS]->(p2:Person) + -[:PARTY_TO]->(c:Crime) RETURN p1.name, p2.name MATCH (c:Crime)-[:OCCURRED_AT]->(l:Location) RETURN l.postcode ``` @@ -689,10 +694,14 @@ The benchmark script generates a timestamped report and optionally updates the R ### Property-Based Filtering ``` - MATCH (n{displayName:"Autophagy"}) RETURN n - MATCH (n{displayName:"Autophagy"})-->(m) RETURN m - MATCH (n{displayName:"APOE-4 [extracellular region]"}) RETURN n - MATCH (n{displayName:"APOE-4 [extracellular region]"})-->(m) RETURN m + MATCH (n{displayName:"Autophagy"}) + RETURN n + MATCH (n{displayName:"Autophagy"}) + -->(m) RETURN m + MATCH (n{displayName:"APOE-4 [extracellular region]"}) + RETURN n + MATCH (n{displayName:"APOE-4 [extracellular region]"}) + -->(m) RETURN m MATCH (n:DatabaseObject{isChimeric:false}) RETURN n MATCH (n:DatabaseObject{isChimeric:true}) RETURN n ``` @@ -700,18 +709,30 @@ The benchmark script generates a timestamped report and optionally updates the R ### Multi-Hop Traversals ``` - MATCH (n{displayName:"Autophagy"})-->(m)-->(p) RETURN p - MATCH (n{displayName:"Autophagy"})-->(m)-->(p)-->(q) RETURN q - MATCH (n{displayName:"Autophagy"})-->(m)-->(p)-->(q)-->(r) RETURN r - MATCH (n{displayName:"Autophagy"})-->(m)-->(p)-->(q)-->(r)-->(s) RETURN s - MATCH (n{displayName:"Autophagy"})-->(m)-->(p)-->(q)-->(r)-->(s)-->(t) RETURN t - MATCH (n{displayName:"Autophagy"})-->(m)-->(p)-->(q)-->(r)-->(s)-->(t)-->(v) RETURN v - MATCH (n{displayName:"APOE-4 [extracellular region]"})-->(m)-->(p) RETURN p - MATCH (n{displayName:"APOE-4 [extracellular region]"})-->(m)-->(p)-->(q) RETURN q - MATCH (n{displayName:"APOE-4 [extracellular region]"})-->(m)-->(p)-->(q)-->(r) RETURN r - MATCH (n{displayName:"APOE-4 [extracellular region]"})-->(m)-->(p)-->(q)-->(r)-->(s) RETURN s - MATCH (n{displayName:"APOE-4 [extracellular region]"})-->(m)-->(p)-->(q)-->(r)-->(s)-->(t) RETURN t - MATCH (n{displayName:"APOE-4 [extracellular region]"})-->(m)-->(p)-->(q)-->(r)-->(s)-->(t)-->(v) RETURN v + MATCH (n{displayName:"Autophagy"}) + -->(m)-->(p) RETURN p + MATCH (n{displayName:"Autophagy"}) + -->(m)-->(p)-->(q) RETURN q + MATCH (n{displayName:"Autophagy"}) + -->(m)-->(p)-->(q)-->(r) RETURN r + MATCH (n{displayName:"Autophagy"}) + -->(m)-->(p)-->(q)-->(r)-->(s) RETURN s + MATCH (n{displayName:"Autophagy"}) + -->(m)-->(p)-->(q)-->(r)-->(s)-->(t) RETURN t + MATCH (n{displayName:"Autophagy"}) + -->(m)-->(p)-->(q)-->(r)-->(s)-->(t)-->(v) RETURN v + MATCH (n{displayName:"APOE-4 [extracellular region]"}) + -->(m)-->(p) RETURN p + MATCH (n{displayName:"APOE-4 [extracellular region]"}) + -->(m)-->(p)-->(q) RETURN q + MATCH (n{displayName:"APOE-4 [extracellular region]"}) + -->(m)-->(p)-->(q)-->(r) RETURN r + MATCH (n{displayName:"APOE-4 [extracellular region]"}) + -->(m)-->(p)-->(q)-->(r)-->(s) RETURN s + MATCH (n{displayName:"APOE-4 [extracellular region]"}) + -->(m)-->(p)-->(q)-->(r)-->(s)-->(t) RETURN t + MATCH (n{displayName:"APOE-4 [extracellular region]"}) + -->(m)-->(p)-->(q)-->(r)-->(s)-->(t)-->(v) RETURN v ``` ### Relationship Type Traversal @@ -721,8 +742,10 @@ The benchmark script generates a timestamped report and optionally updates the R MATCH (n)-[e:interactor]->(m) RETURN n,m MATCH (n)-[e:surroundedBy]->(m) RETURN n,m MATCH (n)-[:hasEvent]->(m) RETURN n,m - MATCH (n:Pathway)-[:hasEvent]->(m:ReactionLikeEvent) RETURN n,m - MATCH (r:ReactionLikeEvent)-[:output]->(s:PhysicalEntity) RETURN r,s + MATCH (n:Pathway)-[:hasEvent]->(m:ReactionLikeEvent) + RETURN n,m + MATCH (r:ReactionLikeEvent)-[:output]->(s:PhysicalEntity) + RETURN r,s ``` ### Complex Patterns diff --git a/concepts/columnar_storage.mdx b/concepts/columnar_storage.mdx index ab1a309..2405c5d 100644 --- a/concepts/columnar_storage.mdx +++ b/concepts/columnar_storage.mdx @@ -108,7 +108,8 @@ This model was designed for early relational systems and doesn't fit high-perfor ### TuringDB’s Columnar Streaming Model ``` -[Column of Nodes] → [Filter Column by Age] → [Filter Column by Location] → [Column of Edges] +[Column of Nodes] → [Filter Column by Age] + → [Filter Column by Location] → [Column of Edges] ``` - Works on **columns of nodes or edges** (stored in DataParts) diff --git a/graph_dev/create_graph.mdx b/graph_dev/create_graph.mdx index ac5b250..1bf71c7 100644 --- a/graph_dev/create_graph.mdx +++ b/graph_dev/create_graph.mdx @@ -15,8 +15,7 @@ description: "Create a TuringDB graph" ```python from turingdb import TuringDB - # Create TuringDB client - # host and port of the running TuringDB server (default localhost:6666) + # Connect to the TuringDB server (default: localhost:6666) client = TuringDB(host="localhost", port=6666) # Create a new graph called my_graph diff --git a/graph_dev/create_nodes_edges.mdx b/graph_dev/create_nodes_edges.mdx index 9cff65a..17515a8 100644 --- a/graph_dev/create_nodes_edges.mdx +++ b/graph_dev/create_nodes_edges.mdx @@ -24,7 +24,7 @@ description: "Once you have a valid change, you can start adding new nodes and e // You need to commit the changes after node creation COMMIT - // Match Jane and John nodes to create edge of label KNOWS between them + // Match Jane and John, then create a KNOWS edge between them MATCH (n:Person {name: 'Jane'}), (m:Person {name: 'John'}) CREATE (n)-[e:KNOWS]->(m) @@ -39,7 +39,10 @@ description: "Once you have a valid change, you can start adding new nodes and e ```python # Create a node Person (Jane) - Edge (knows) - node Person (John) - client.query("CREATE (:Person {name: 'Jane'})-[:KNOWS]->(:Person {name: 'John'})") + client.query( + "CREATE (:Person {name: 'Jane'})" + "-[:KNOWS]->(:Person {name: 'John'})" + ) # Commit the changes client.query("COMMIT") diff --git a/graph_dev/graph_examples.mdx b/graph_dev/graph_examples.mdx index 4f1efb4..ebc6ce0 100644 --- a/graph_dev/graph_examples.mdx +++ b/graph_dev/graph_examples.mdx @@ -7,8 +7,7 @@ description: "Complete example on how to create and query a graph from scratch i ```python from turingdb import TuringDB -# Create TuringDB client -# host and port of the running TuringDB server (default localhost:6666) +# Connect to the TuringDB server (default: localhost:6666) client = TuringDB(host="localhost", port=6666) # Create a new graph called my_graph @@ -24,7 +23,10 @@ change = client.new_change() client.checkout(change=change) # Create a node Person (Jane) - Edge (knows) - node (John) -client.query("CREATE (n:Person {name: 'Jane'})-[e:KNOWS]->(m:Person {name: 'John'})") +client.query( + "CREATE (n:Person {name: 'Jane'})" + "-[e:KNOWS]->(m:Person {name: 'John'})" +) # Commit the change client.query("COMMIT") @@ -34,6 +36,9 @@ client.query("CHANGE SUBMIT") client.checkout() # Query graph -df = client.query("MATCH (n:Person {name:'Jane'})-[e:KNOWS]-(m:Person {name:'John'}) RETURN n, e, m") +df = client.query( + "MATCH (n:Person {name:'Jane'})-[e:KNOWS]-" + "(m:Person {name:'John'}) RETURN n, e, m" +) print(df) ``` \ No newline at end of file diff --git a/graph_dev/time_travelling.mdx b/graph_dev/time_travelling.mdx index 38e8085..cce7cc7 100644 --- a/graph_dev/time_travelling.mdx +++ b/graph_dev/time_travelling.mdx @@ -73,7 +73,8 @@ await client.query("LOAD COMMIT 'be9643a1c03f8e2b'"); // Step 2: Now you can query at that commit const rows = await client.queryData( - "MATCH (n:Person)-[e:KNOWS]->(m:Person) RETURN n.name, m.name", + "MATCH (n:Person)-[e:KNOWS]->(m:Person) " + + "RETURN n.name, m.name", { commit: "be9643a1c03f8e2b" }, ); ``` diff --git a/graph_dev/update_properties.mdx b/graph_dev/update_properties.mdx index 038081b..581d538 100644 --- a/graph_dev/update_properties.mdx +++ b/graph_dev/update_properties.mdx @@ -60,7 +60,8 @@ The `SET` clause is used to update property values on existing nodes and edges. Edge properties can be updated using the same syntax. ```jsx - MATCH (n:Person {name: 'Alice'})-[e:KNOWS]->(m:Person {name: 'Bob'}) + MATCH (n:Person {name: 'Alice'}) + -[e:KNOWS]->(m:Person {name: 'Bob'}) SET e.since = 2020 ``` @@ -131,7 +132,10 @@ The `SET` clause is used to update property values on existing nodes and edges. Multiple properties can be updated in a single `SET` clause by separating assignments with commas. ```python - client.query("MATCH (n:Person {name: 'Alice'}) SET n.position = 'Developer', n.surname = 'Taylor'") + client.query( + "MATCH (n:Person {name: 'Alice'}) " + "SET n.position = 'Developer', n.surname = 'Taylor'" + ) ``` ## Setting a property from an expression @@ -139,7 +143,9 @@ The `SET` clause is used to update property values on existing nodes and edges. The value assigned can be any expression, including arithmetic operations on existing properties. ```python - client.query("MATCH (p:Product) SET p.discountPrice = p.price * (1 - 0.15)") + client.query( + "MATCH (p:Product) SET p.discountPrice = p.price * (1 - 0.15)" + ) ``` ## Setting a property from another variable @@ -159,7 +165,8 @@ The `SET` clause is used to update property values on existing nodes and edges. ```python client.query(""" - MATCH (n:Person {name: 'Alice'})-[e:KNOWS]->(m:Person {name: 'Bob'}) + MATCH (n:Person {name: 'Alice'}) + -[e:KNOWS]->(m:Person {name: 'Bob'}) SET e.since = 2020 """) ``` @@ -202,7 +209,10 @@ The `SET` clause is used to update property values on existing nodes and edges. ```python client.query("MATCH (n:Person {name: 'Alice'}) SET n.age = null") - client.query("MATCH (n:Person {name: 'Alice'}) REMOVE n.position, n.surname") + client.query( + "MATCH (n:Person {name: 'Alice'}) " + "REMOVE n.position, n.surname" + ) ``` diff --git a/import_data/csv.mdx b/import_data/csv.mdx index 0e263fc..1cb8a96 100644 --- a/import_data/csv.mdx +++ b/import_data/csv.mdx @@ -11,7 +11,8 @@ The file must live in the `data` subdirectory of the TuringDB working directory ## Syntax ``` -LOAD CSV "" [WITH HEADERS] [ON ERROR SKIP | ON ERROR FAIL] AS +LOAD CSV "" [WITH HEADERS] + [ON ERROR SKIP | ON ERROR FAIL] AS ``` | Clause | Description | @@ -60,8 +61,12 @@ Because `LOAD CSV` yields rows, you can drive `CREATE` from it inside a change: ```python client.new_change() -client.query('LOAD CSV "people.csv" WITH HEADERS AS row CREATE (:Person {name: row.name, age: toInteger(row.age)})') -client.query("COMMIT") # make the new nodes visible to the next query +client.query( + 'LOAD CSV "people.csv" WITH HEADERS AS row ' + 'CREATE (:Person {name: row.name, age: toInteger(row.age)})' +) +# make the new nodes visible to the next query +client.query("COMMIT") client.query("CHANGE SUBMIT") client.checkout() ``` diff --git a/import_data/gml.mdx b/import_data/gml.mdx index 8ec30ef..d14a35c 100644 --- a/import_data/gml.mdx +++ b/import_data/gml.mdx @@ -42,7 +42,8 @@ graph [ Property names also get a **type suffix**, for example `label (String)` and `weight (String)`. Quote them with backticks in queries, and run `CALL db.propertyTypes()` to see the exact names: ```jsx - MATCH (n:GMLNode) RETURN n.`label (String)`, toInteger(n.`age (String)`) + MATCH (n:GMLNode) + RETURN n.`label (String)`, toInteger(n.`age (String)`) ``` diff --git a/import_data/jsonl.mdx b/import_data/jsonl.mdx index dc7e47f..a02b27c 100644 --- a/import_data/jsonl.mdx +++ b/import_data/jsonl.mdx @@ -73,7 +73,8 @@ LOAD JSONL 'mydata.jsonl' AS mygraph An array-valued property is stored as a List by default. To load specific array properties as **embeddings** (type `Embedding`, usable with `cosine_similarity` / `euclidean_distance`), append a `WITH EMBEDDINGS` clause listing each embedding property and its dimension: ```jsx -LOAD JSONL 'mydata.jsonl' AS mygraph WITH EMBEDDINGS [{"embedding", 384}, {"summary_vec", 768}] +LOAD JSONL 'mydata.jsonl' AS mygraph + WITH EMBEDDINGS [{"embedding", 384}, {"summary_vec", 768}] ``` Each spec is `{"", }`. The dimension must be **greater than 1**, and every array under that property must have exactly that many numeric elements. This works for both node and relationship properties. Array properties not listed in `WITH EMBEDDINGS` are stored as Lists. diff --git a/import_data/neo4j.mdx b/import_data/neo4j.mdx index 106752a..00f7f5e 100644 --- a/import_data/neo4j.mdx +++ b/import_data/neo4j.mdx @@ -21,7 +21,8 @@ If your database was created with Neo4j 4.x, you need to migrate it to the 5.x f ```bash # Load your v4 dump -neo4j-admin database load --from-stdin neo4j --overwrite-destination=true < mydb.dump +neo4j-admin database load --from-stdin neo4j \ + --overwrite-destination=true < mydb.dump # Migrate to v5 format neo4j-admin database migrate neo4j --force-btree-indexes-to-range diff --git a/import_data/parquet.mdx b/import_data/parquet.mdx index 421b97f..d51db9b 100644 --- a/import_data/parquet.mdx +++ b/import_data/parquet.mdx @@ -13,7 +13,8 @@ To load Parquet files containing a JSON string column, which is then converted i `LOAD PARQUET` is the fastest way to get data into TuringDB, and the recommended path for anything beyond a few thousand entities. ``` -LOAD PARQUET '' [AS ] [WITH DURATIONS [, ...]] +LOAD PARQUET '' [AS ] + [WITH DURATIONS [, ...]] ``` The graph name defaults to the name of the directory. The directory is resolved inside the `data` subdirectory of the TuringDB working directory (`-turing-dir`) and must contain `nodes.parquet` and `edges.parquet`. @@ -62,15 +63,17 @@ import pandas as pd, pyarrow as pa, pyarrow.parquet as pq from pathlib import Path people = pd.read_csv("people.csv") # columns: email, name, age -knows = pd.read_csv("knows.csv") # columns: src, dst, since (src and dst are emails) +# columns: src, dst, since (src and dst are emails) +knows = pd.read_csv("knows.csv") -ids = {key: i for i, key in enumerate(people["email"])} # natural key -> int64 __id +# natural key -> int64 __id +ids = {key: i for i, key in enumerate(people["email"])} nodes = pd.DataFrame({ "__id": pd.Series(range(len(people)), dtype="int64"), "__labels": [["Person"]] * len(people), "email": people["email"], "name": people["name"], - "age": people["age"].astype("Int64"), # int64 + "age": people["age"].astype("Int64"), # int64 }) edges = pd.DataFrame({ "__source": knows["src"].map(ids).astype("int64"), @@ -79,10 +82,17 @@ edges = pd.DataFrame({ "since": knows["since"].astype("int64"), }) -out = Path.home() / ".turing" / "data" / "people_graph" # /data/ +# /data/ +out = Path.home() / ".turing" / "data" / "people_graph" out.mkdir(parents=True, exist_ok=True) -pq.write_table(pa.Table.from_pandas(nodes, preserve_index=False), out / "nodes.parquet") -pq.write_table(pa.Table.from_pandas(edges, preserve_index=False), out / "edges.parquet") +pq.write_table( + pa.Table.from_pandas(nodes, preserve_index=False), + out / "nodes.parquet", +) +pq.write_table( + pa.Table.from_pandas(edges, preserve_index=False), + out / "edges.parquet", +) client.query("LOAD PARQUET 'people_graph' AS people") client.set_graph("people") @@ -179,7 +189,8 @@ The example below uses the [OptimusKG](https://optimuskg.ai/) biomedical knowled cd optimuskg - MATCH (n:GEN) WHERE n.symbol = 'TSPAN6' RETURN n.id, n.symbol, n.biotype + MATCH (n:GEN) WHERE n.symbol = 'TSPAN6' + RETURN n.id, n.symbol, n.biotype ``` diff --git a/index.mdx b/index.mdx index f3ecedb..888c1da 100644 --- a/index.mdx +++ b/index.mdx @@ -84,7 +84,10 @@ change = client.new_change() client.checkout(change=change) # Write, commit, and submit -client.query("CREATE (n:Person {name: 'Jane'})-[e:KNOWS]->(m:Person {name: 'John'})") +client.query( + "CREATE (n:Person {name: 'Jane'})" + "-[e:KNOWS]->(m:Person {name: 'John'})" +) client.query("COMMIT") client.query("CHANGE SUBMIT") @@ -97,18 +100,24 @@ client.query("MATCH (n) RETURN n") import { TuringClient } from "turingdb"; // Connect to a running TuringDB server (started with -p 6686) -const client = new TuringClient({ url: "http://localhost:6686/query", graph: "mygraph" }); +const client = new TuringClient({ + url: "http://localhost:6686/query", + graph: "mygraph", +}); // Query the graph, with a parameter const [[names]] = await client.queryData( - "MATCH (n:Person) WHERE n.name STARTS WITH $prefix RETURN n.name", + `MATCH (n:Person) + WHERE n.name STARTS WITH $prefix + RETURN n.name`, { parameters: { prefix: "J" } }, ); ``` ```bash Docker # Run TuringDB in a container -# (other install methods are preferred — Docker adds some performance loss) +# (other install methods are preferred: +# Docker adds some performance loss) docker run -it turingdbai/turingdb:nightly turingdb ``` diff --git a/jssdk/get_started.mdx b/jssdk/get_started.mdx index b72b593..cdb4977 100644 --- a/jssdk/get_started.mdx +++ b/jssdk/get_started.mdx @@ -29,7 +29,9 @@ A web page reaches the server through its own origin instead, so in the browser ```js import { TuringClient } from "turingdb"; -const client = new TuringClient({ url: "http://localhost:6686/query" }); +const client = new TuringClient({ + url: "http://localhost:6686/query", +}); await client.query("CREATE GRAPH people"); client.setGraph("people"); @@ -43,13 +45,17 @@ Writes run inside a change, as with the Python SDK (see [Writing data](/query/wr ```js const { chunks } = await client.query("CHANGE NEW"); -const change = chunks[0].columns[0].get(0); // the change ID, a bigint +// the change ID, a bigint +const change = chunks[0].columns[0].get(0); client.setChange(change); await client.query("CREATE (:Person {name: $name, age: $age})", { parameters: { name: "Marie", age: 33 }, }); -await client.query("CREATE (:Person {name: 'Jane'})-[:KNOWS]->(:Person {name: 'John'})"); +await client.query( + "CREATE (:Person {name: 'Jane'})" + + "-[:KNOWS]->(:Person {name: 'John'})", +); await client.query("CHANGE SUBMIT"); client.setChange("main"); // back to main @@ -73,7 +79,10 @@ console.log(ages); // [ null, null, 33 ] Results are column-oriented: `names` holds the first column for every row. A result streams in chunks of up to 65,536 rows, so loop over the chunks when a query can return more: ```js -for (const [names, ages] of await client.queryData("MATCH (p:Person) RETURN p.name, p.age")) { +const result = await client.queryData( + "MATCH (p:Person) RETURN p.name, p.age", +); +for (const [names, ages] of result) { // one chunk: names[i] and ages[i] belong to the same row } ``` @@ -87,11 +96,15 @@ The declarations ship with the package, so no `@types` package is needed: ```ts import { TuringClient, TuringQueryError } from "turingdb"; -const client = new TuringClient({ url: "http://localhost:6686/query", graph: "people" }); +const client = new TuringClient({ + url: "http://localhost:6686/query", + graph: "people", +}); async function namesOlderThan(age: number): Promise { const chunks = await client.queryData( - "MATCH (p:Person) WHERE p.age > $age RETURN p.name ORDER BY p.name", + "MATCH (p:Person) WHERE p.age > $age " + + "RETURN p.name ORDER BY p.name", { parameters: { age } }, ); return chunks.flatMap(([names]) => names as string[]); @@ -101,7 +114,8 @@ try { console.log(await namesOlderThan(30)); // [ 'Marie' ] } catch (e) { if (e instanceof TuringQueryError) { - console.error(e.status, e.message); // for example PARSE_ERROR and where the query failed + // for example PARSE_ERROR, and where the query failed + console.error(e.status, e.message); } else { throw e; } @@ -123,11 +137,13 @@ import { defineConfig } from "vite"; export default defineConfig({ optimizeDeps: { - exclude: ["turingdb"], // keep the client next to its .wasm decoder + // keep the client next to its .wasm decoder + exclude: ["turingdb"], }, server: { proxy: { - "/query": "http://localhost:6666", // the TuringDB server + // the TuringDB server + "/query": "http://localhost:6666", }, }, }); @@ -137,12 +153,16 @@ export default defineConfig({ // src/main.ts import { TuringClient } from "turingdb"; -const client = new TuringClient({ graph: "people" }); // url defaults to /query +// url defaults to /query +const client = new TuringClient({ graph: "people" }); async function showPeople(): Promise { - const chunks = await client.queryData("MATCH (p:Person) RETURN p.name ORDER BY p.name"); + const chunks = await client.queryData( + "MATCH (p:Person) RETURN p.name ORDER BY p.name", + ); const names = chunks.flatMap(([column]) => column); - document.querySelector("#people")!.textContent = names.join(", "); + const list = document.querySelector("#people")!; + list.textContent = names.join(", "); } showPeople(); diff --git a/jssdk/reference.mdx b/jssdk/reference.mdx index cdf67fb..33d3ab0 100644 --- a/jssdk/reference.mdx +++ b/jssdk/reference.mdx @@ -59,8 +59,10 @@ Each query takes these options: const controller = new AbortController(); setTimeout(() => controller.abort(), 5000); -// rejects with an AbortError if the query runs for more than 5 seconds -await client.query("MATCH (n) RETURN count(n)", { signal: controller.signal }); +// rejects with an AbortError if it runs longer than 5 seconds +await client.query("MATCH (n) RETURN count(n)", { + signal: controller.signal, +}); ``` ## Results @@ -77,7 +79,9 @@ await client.query("MATCH (n) RETURN count(n)", { signal: controller.signal }); A result streams in chunks of up to 65,536 rows, and each chunk holds every column of its rows. A result with no rows is one chunk of empty columns, so `RETURN a, b` matching nothing gives `[[[], []]]` from `queryData()`. ```js -const { chunks } = await client.query("MATCH (p:Person) RETURN p.name AS name, p.age AS age"); +const { chunks } = await client.query( + "MATCH (p:Person) RETURN p.name AS name, p.age AS age", +); for (const { names, columns } of chunks) { const [name, age] = columns; @@ -102,7 +106,9 @@ for (const { names, columns } of chunks) { `values` reads a large numeric column without building a JavaScript value per row. A row that holds null reads as `0` there, so check `validity`: ```js -const { chunks } = await client.query("MATCH (p:Person) RETURN p.age ORDER BY p.age"); +const { chunks } = await client.query( + "MATCH (p:Person) RETURN p.age ORDER BY p.age", +); const ages = chunks[0].columns[0]; ages.values; // BigInt64Array [ 33n, 0n, 0n ] @@ -130,9 +136,11 @@ ages.toArray(); // [ 33n, null, null ] To turn a DateTime into a JavaScript `Date`, which holds milliseconds: ```js -const { chunks } = await client.query("RETURN datetime('2024-03-15T10:30:00Z') AS d"); +const { chunks } = await client.query( + "RETURN datetime('2024-03-15T10:30:00Z') AS d", +); const micros = chunks[0].columns[0].get(0); // 1710498600000000n -new Date(Number(micros / 1000n)); // 2024-03-15T10:30:00.000Z +new Date(Number(micros / 1000n)); // 2024-03-15T10:30:00.000Z ``` ## Parameters @@ -140,14 +148,21 @@ new Date(Number(micros / 1000n)); // 2024-03-15T10:30:00.000Z `query(cypher, { parameters })` binds each `$name` in the query to `parameters.name`. A parameter can stand for a value anywhere an expression goes, or for a label or an edge type (see [Query parameters](/query/cypher_subset#query-parameters)): ```js -await client.query("MATCH (p:Person) WHERE p.age >= $age RETURN p.name", { - parameters: { age: 30 }, -}); +await client.query( + "MATCH (p:Person) WHERE p.age >= $age RETURN p.name", + { parameters: { age: 30 } }, +); // a write, so inside a change (see Versioning below) -await client.query("UNWIND $rows AS row CREATE (:City {name: row.name, pop: row.pop})", { - parameters: { rows: [{ name: "Brest", pop: 139000 }, { name: "Rennes", pop: 220000 }] }, -}); +const rows = [ + { name: "Brest", pop: 139000 }, + { name: "Rennes", pop: 220000 }, +]; +await client.query( + "UNWIND $rows AS row " + + "CREATE (:City {name: row.name, pop: row.pop})", + { parameters: { rows } }, +); ``` | JavaScript value | Sent as | @@ -188,11 +203,16 @@ client.setChange("main"); To read a past commit, load it with `LOAD COMMIT` and then pass its hash: ```js -const [[commits]] = await client.queryData("CALL db.history() YIELD `commit` RETURN `commit`"); -const previous = commits[1]; // newest first: commits[0] is the latest one, marked (HEAD) +const [[commits]] = await client.queryData( + "CALL db.history() YIELD `commit` RETURN `commit`", +); +// newest first: commits[0] is the latest one, marked (HEAD) +const previous = commits[1]; await client.query(`LOAD COMMIT '${previous}'`); -await client.queryData("MATCH (p:Person) RETURN count(p)", { commit: previous }); +await client.queryData("MATCH (p:Person) RETURN count(p)", { + commit: previous, +}); ``` - Unlike the Python SDK's `checkout()`, the client does not load a past commit for you. Without `LOAD COMMIT`, a query on a commit that is not in memory fails with `COMMIT_NOT_LOADED`. `LOAD COMMIT` on a loaded commit does nothing. @@ -211,7 +231,9 @@ try { } catch (e) { if (e instanceof TuringQueryError) { console.log(e.status); // PARSE_ERROR - console.log(e.message); // the query, a marker under the error, and "syntax error, unexpected RETURN" + // the query, a marker under the error, + // and "syntax error, unexpected RETURN" + console.log(e.message); } } ``` diff --git a/pythonsdk/get_started.mdx b/pythonsdk/get_started.mdx index 3880a21..2b08043 100644 --- a/pythonsdk/get_started.mdx +++ b/pythonsdk/get_started.mdx @@ -71,10 +71,16 @@ Use Cypher `CREATE` queries to add data: client.query("CREATE (:Person {name: 'Marie', age: 33})") # Create two nodes and an edge -client.query("CREATE (:Person {name: 'Jane'})-[:KNOWS]->(:Person {name: 'John'})") +client.query( + "CREATE (:Person {name: 'Jane'})" + "-[:KNOWS]->(:Person {name: 'John'})" +) # Pass values as parameters: no quoting or escaping needed -client.query("CREATE (:Person {name: $name, age: $age})", {"name": "O'Brien", "age": 41}) +client.query( + "CREATE (:Person {name: $name, age: $age})", + {"name": "O'Brien", "age": 41}, +) ``` > ⚠️ All created entities must have at least one label (:Label) @@ -141,12 +147,12 @@ client.checkout() ```python from turingdb import TuringDB -# Create TuringDB client -# Connect to a locally running TuringDB server (default host and port shown): +# Connect to the TuringDB server (default: localhost:6666) client = TuringDB(host="localhost", port=6666) -# Or run the engine in-process, with no server: -# client = TuringDB(type="embedded", data_dir="/home/me/.turing") # use an absolute path +# Or run the engine in-process, with no server +# (use an absolute path for data_dir): +# client = TuringDB(type="embedded", data_dir="/home/me/.turing") # Create and set working graph client.create_graph("people") @@ -159,7 +165,10 @@ client.checkout(change=change) # Create nodes and edges client.query("CREATE (:Person {name: 'Marie', age:33})") -client.query("CREATE (:Person {name: 'Jane'})-[:FRIEND_OF]->(:Person {name: 'John'})") +client.query( + "CREATE (:Person {name: 'Jane'})" + "-[:FRIEND_OF]->(:Person {name: 'John'})" +) # Commit the change client.query("COMMIT") diff --git a/pythonsdk/reference.mdx b/pythonsdk/reference.mdx index 5feb445..97231c4 100644 --- a/pythonsdk/reference.mdx +++ b/pythonsdk/reference.mdx @@ -29,7 +29,7 @@ The SDK targets TuringDB v3: pin `turingdb>=3.1` in your project (older projects ```python from turingdb import TuringDB -# Connect to a TuringDB server running on this machine (localhost:6666) +# Connect to a server running on this machine (localhost:6666) client = TuringDB() # or name the host and the port @@ -84,7 +84,9 @@ A missing or incorrect token raises `TuringDBException: Protocol error from serv ```python import os -client = TuringDB(type="embedded", data_dir=os.path.expanduser("~/.turing")) +client = TuringDB( + type="embedded", data_dir=os.path.expanduser("~/.turing") +) ``` - `data_dir` is the TuringDB root directory (the one that holds `graphs/`, `data/` and `vector/`). Omit it to use `~/.turing`. @@ -136,8 +138,13 @@ Runs a Cypher query on the current graph. ```python df = client.query("MATCH (n:Person) RETURN n.name, n.age") -df = client.query("MATCH (n:Person) WHERE n.age >= $min RETURN n.name", {"min": 30}) -df = client.query("MATCH (n:$label) RETURN n.name", {"label": "Person"}) +df = client.query( + "MATCH (n:Person) WHERE n.age >= $min RETURN n.name", + {"min": 30}, +) +df = client.query( + "MATCH (n:$label) RETURN n.name", {"label": "Person"} +) ``` Returns results as a `pandas.DataFrame`, with automatic typing and parsing. `query_parameters` fills the query's `$name` [parameters](/query/cypher_subset#query-parameters), which stand for values, labels and edge types. Parameter values can be: @@ -258,8 +265,12 @@ client.s3_connect("my-bucket", region="eu-west-1") Transfer data between a local path, an `s3://…` URI, or a `turingdb://…` path. The direction and mechanism are inferred from the URI schemes. ```python -client.transfer("s3://graphs/data.jsonl", "turingdb://data.jsonl") # S3 → server -client.transfer("./local.gml", "turingdb://local.gml") # local → server (via scratch) +# S3 → server +client.transfer( + "s3://graphs/data.jsonl", "turingdb://data.jsonl" +) +# local → server (via scratch) +client.transfer("./local.gml", "turingdb://local.gml") ``` The equivalent engine commands (issued directly via `query`) are `S3 CONNECT "" "" ""`, `S3 PULL "" ""`, and `S3 PUSH "" ""`. @@ -330,13 +341,18 @@ client.checkout(change=change) # Multiple nodes and edges can be created at the same time # using the following syntax -client.query("""CREATE (apoe_gene:Gene {name: 'APOE', chromosome: '19', location: '19q13.32'}), -(apoe_protein:Protein {name: 'Apolipoprotein E', function: 'Lipid_transport', tissue: 'Brain_and_liver'}), -(alzheimers:Disease {name: 'Alzheimers_Disease', type: 'Neurodegenerative'}), -(cholesterol:Molecule {name: 'Cholesterol', type: 'Lipid'}), -(apoe_gene)-[:ENCODES]->(apoe_protein), -(apoe_protein)-[:BINDS]->(cholesterol), -(apoe_protein)-[:ASSOCIATED_WITH]->(alzheimers) +client.query(""" +CREATE + (apoe_gene:Gene {name: 'APOE', chromosome: '19', + location: '19q13.32'}), + (apoe_protein:Protein {name: 'Apolipoprotein E', + function: 'Lipid_transport', tissue: 'Brain_and_liver'}), + (alzheimers:Disease {name: 'Alzheimers_Disease', + type: 'Neurodegenerative'}), + (cholesterol:Molecule {name: 'Cholesterol', type: 'Lipid'}), + (apoe_gene)-[:ENCODES]->(apoe_protein), + (apoe_protein)-[:BINDS]->(cholesterol), + (apoe_protein)-[:ASSOCIATED_WITH]->(alzheimers) """) # Commit the change @@ -347,7 +363,10 @@ client.query("CHANGE SUBMIT") client.checkout() # Query graph, with the gene name as a parameter -df = client.query("MATCH (n:Gene {name: $name}) RETURN n.name, n.chromosome", {"name": "APOE"}) +df = client.query( + "MATCH (n:Gene {name: $name}) RETURN n.name, n.chromosome", + {"name": "APOE"}, +) print(df) ``` diff --git a/query/aggregation.mdx b/query/aggregation.mdx index 4dc79c1..b583b8d 100644 --- a/query/aggregation.mdx +++ b/query/aggregation.mdx @@ -105,13 +105,15 @@ Grouping is implicit, as in Neo4j: the non-aggregated items in the `RETURN` are ```cypher MATCH (a:Person)-[:WORKS_AT]->(c:Company) -RETURN c.name, count(a) AS employees, collect(a.name) AS who, avg(a.age) AS avgAge +RETURN c.name, count(a) AS employees, collect(a.name) AS who, + avg(a.age) AS avgAge ORDER BY employees DESC ``` ```cypher MATCH (n:Person) -RETURN count(*), count(n.age), sum(n.age), avg(n.age), min(n.age), max(n.age) +RETURN count(*), count(n.age), sum(n.age), + avg(n.age), min(n.age), max(n.age) MATCH (a)-[w:WORKS_AT]->(c) RETURN w.role, count(DISTINCT c) AS companies @@ -134,7 +136,9 @@ RETURN labels(n) AS l, count(*) AS c ORDER BY c DESC MATCH (n:Person) -RETURN stDev(n.age) AS sd, percentileCont(n.age, 0.5) AS median, percentileDisc(n.age, 0.9) AS p90 +RETURN stDev(n.age) AS sd, + percentileCont(n.age, 0.5) AS median, + percentileDisc(n.age, 0.9) AS p90 MATCH (n:Person) RETURN stDev(n.age) AS sample, stDevP(n.age) AS population @@ -207,15 +211,21 @@ RETURN t, count(*) To unwind a list from your application, pass it as a [parameter](/query/cypher_subset#query-parameters): ```python -client.query("UNWIND $names AS nm MATCH (n:Person {name: nm}) RETURN nm, n.age", {"names": ["Alice", "Bob"]}) +client.query( + "UNWIND $names AS nm MATCH (n:Person {name: nm}) " + "RETURN nm, n.age", + {"names": ["Alice", "Bob"]}, +) ``` ## Lists ```cypher -RETURN [1, 2, 3, 4, 5][0], [1, 2, 3, 4, 5][-1], [1, 2, 3, 4, 5][1..3], [1, 2, 3][10] +RETURN [1, 2, 3, 4, 5][0], [1, 2, 3, 4, 5][-1], + [1, 2, 3, 4, 5][1..3], [1, 2, 3][10] -RETURN size([1, 2]), head([1, 2]), last([1, 2]), tail([1, 2]), reverse([1, 2, 3]) +RETURN size([1, 2]), head([1, 2]), last([1, 2]), + tail([1, 2]), reverse([1, 2, 3]) MATCH (n:Person {name: 'Alice'}) RETURN n.tags[0], size(n.tags) @@ -242,7 +252,7 @@ RETURN n.name RETURN reduce(total = 0, x IN [1, 2, 3] | total + x) -RETURN allReduce(total = 0, x IN [1, 2, 3] | total + x, total < 10) +RETURN allReduce(t = 0, x IN [1, 2, 3] | t + x, t < 10) ``` - `[x IN list WHERE condition | expression]` filters and maps a list. @@ -256,8 +266,16 @@ RETURN allReduce(total = 0, x IN [1, 2, 3] | total + x, total < 10) ```cypher MATCH (n:Person) RETURN n.name, - CASE WHEN n.age >= 35 THEN 'senior' WHEN n.age IS NULL THEN 'unknown' ELSE 'junior' END AS band, - CASE n.city WHEN 'London' THEN 'UK' WHEN 'Paris' THEN 'FR' ELSE 'other' END AS country + CASE + WHEN n.age >= 35 THEN 'senior' + WHEN n.age IS NULL THEN 'unknown' + ELSE 'junior' + END AS band, + CASE n.city + WHEN 'London' THEN 'UK' + WHEN 'Paris' THEN 'FR' + ELSE 'other' + END AS country ``` The simple form (`CASE n.city WHEN 'London' THEN ...`) and the searched form (`CASE WHEN condition THEN ...`) both work. TuringDB also accepts comparison tests in the simple form, such as `CASE n.age WHEN > 30 THEN 'old' WHEN IS NULL THEN '?' END`. diff --git a/query/cheatsheet.mdx b/query/cheatsheet.mdx index bbdbae3..75b90a0 100644 --- a/query/cheatsheet.mdx +++ b/query/cheatsheet.mdx @@ -35,7 +35,8 @@ RETURN n.name MATCH (a:Person)-[e]-(b) RETURN a.name, b.name -MATCH (a:Person WHERE a.age > 30)-[r:KNOWS WHERE r.since > 2012]->(b) +MATCH (a:Person WHERE a.age > 30) + -[r:KNOWS WHERE r.since > 2012]->(b) RETURN b.name MATCH (n:Person) @@ -145,10 +146,22 @@ RETURN n.name [Query parameters](/query/cypher_subset#query-parameters) ```python -client.query("MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", {"minAge": 30}) -client.query("MATCH (n:Person {name: $name}) RETURN n.age", {"name": "O'Brien"}) -client.query("UNWIND $rows AS row CREATE (:City {name: row.name, pop: row.pop})", {"rows": [{"name": "Brest", "pop": 139000}]}) -client.query("MATCH (n:$label) RETURN n.name", {"label": "Person"}) +client.query( + "MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", + {"minAge": 30}, +) +client.query( + "MATCH (n:Person {name: $name}) RETURN n.age", + {"name": "O'Brien"}, +) +client.query( + "UNWIND $rows AS row " + "CREATE (:City {name: row.name, pop: row.pop})", + {"rows": [{"name": "Brest", "pop": 139000}]}, +) +client.query( + "MATCH (n:$label) RETURN n.name", {"label": "Person"} +) ``` - A parameter holds a value anywhere an expression goes, or a label or an edge type. No quoting or escaping is needed. @@ -192,14 +205,16 @@ RETURN DISTINCT b.name MATCH (a:Person {name: 'Alice'})-[:KNOWS*1..3]->(b) RETURN DISTINCT b.name -MATCH (a:Person {name: 'Alice'}) ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) +MATCH (a:Person {name: 'Alice'}) + ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) RETURN b.name MATCH (a:Person {name: 'Alice'})-[e:KNOWS]->+(b) RETURN b.name, [r IN e | r.since] MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) -RETURN [n IN nodes(p) | n.name], length(p), reduce(t = 0, r IN relationships(p) | t + r.since) +RETURN [n IN nodes(p) | n.name], length(p), + reduce(t = 0, r IN relationships(p) | t + r.since) ``` - Postfix `->*` is **0** or more hops. Bracket `-[:KNOWS*]->` is **1** or more. @@ -258,7 +273,8 @@ ELSE RETURN 'small' AS size ```cypher MATCH (n:Person) -RETURN count(*), count(n.age), sum(n.age), avg(n.age), min(n.age), max(n.age) +RETURN count(*), count(n.age), sum(n.age), + avg(n.age), min(n.age), max(n.age) MATCH (a:Person)-[:WORKS_AT]->(c:Company) RETURN c.name, count(a) AS employees, collect(a.name) AS who @@ -303,7 +319,8 @@ RETURN n.name RETURN reduce(total = 0, x IN [1, 2, 3] | total + x) MATCH (n:Person) -RETURN n.name, CASE WHEN n.age >= 35 THEN 'senior' ELSE 'junior' END AS band +RETURN n.name, + CASE WHEN n.age >= 35 THEN 'senior' ELSE 'junior' END AS band MATCH (n:Person) RETURN n {.name, .age, nextAge: n.age + 1} AS person @@ -327,10 +344,14 @@ MATCH (p:Person) RETURN p.name, COUNT { (p)-[:KNOWS]->() } AS outDegree MATCH (p:Person) -RETURN p.name, COLLECT { MATCH (p)-[:KNOWS]->(f) RETURN f.name } AS friends +RETURN p.name, + COLLECT { MATCH (p)-[:KNOWS]->(f) RETURN f.name } AS friends MATCH (c:Company) -RETURN c.name, COLLECT { MATCH (p:Person)-[:WORKS_AT]->(c) RETURN p.name ORDER BY p.age DESC LIMIT 1 } AS oldest +RETURN c.name, COLLECT { + MATCH (p:Person)-[:WORKS_AT]->(c) + RETURN p.name ORDER BY p.age DESC LIMIT 1 +} AS oldest MATCH (p:Person) CALL (*) { @@ -380,7 +401,10 @@ RETURN datetime('2024-03-15T10:30:00Z') + duration({days: 1}) MATCH (n:Document) RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) -RETURN round(point.distance(point({latitude: 51.5074, longitude: -0.1278}), point({latitude: 48.8566, longitude: 2.3522}))) AS meters +RETURN round(point.distance( + point({latitude: 51.5074, longitude: -0.1278}), + point({latitude: 48.8566, longitude: 2.3522}) +)) AS meters ``` - Function names are case-insensitive. @@ -411,7 +435,8 @@ CHANGE NEW CREATE (:Person {name: 'Alice', age: 30}) -CREATE (:Person {name: 'Mick'})-[:FRIEND_OF {since: 2020}]->(:Person {name: 'John'}) +CREATE (:Person {name: 'Mick'}) + -[:FRIEND_OF {since: 2020}]->(:Person {name: 'John'}) COMMIT @@ -474,11 +499,13 @@ Outputs can be chained: `CALL db.labels() YIELD label MATCH (n) WHERE label IN l ## Graph algorithms and search ```cypher -MATCH (a:Station {name: 'Ashchurch'}), (b:Station {name: 'Pershore'}) +MATCH (a:Station {name: 'Ashchurch'}), + (b:Station {name: 'Pershore'}) shortestPath(a, b, distance, dist, path) RETURN dist, path -VECTOR SEARCH IN my_index FOR 5 (1.2, 0.5, 3.0, 0.1) YIELD ids, score +VECTOR SEARCH IN my_index FOR 5 (1.2, 0.5, 3.0, 0.1) +YIELD ids, score RETURN ids, score ``` diff --git a/query/composition.mdx b/query/composition.mdx index e6193f4..b4326c1 100644 --- a/query/composition.mdx +++ b/query/composition.mdx @@ -68,9 +68,17 @@ RETURN count(*) AS total A whole chain in braces can also be one side of a `UNION`: ```cypher -{ MATCH (p:Person) RETURN p.age AS v NEXT RETURN max(v) AS v } +{ + MATCH (p:Person) RETURN p.age AS v + NEXT + RETURN max(v) AS v +} UNION -{ MATCH (c:Company) RETURN c.founded AS v NEXT RETURN max(v) AS v } +{ + MATCH (c:Company) RETURN c.founded AS v + NEXT + RETURN max(v) AS v +} ``` ### Inside subqueries @@ -136,7 +144,8 @@ WHEN true THEN RETURN 1 AS x ELSE RETURN 2 AS x Conditions can be subqueries over the graph: ```cypher -WHEN EXISTS { MATCH (:Company {name: 'Acme'}) } THEN RETURN 'has Acme' AS result +WHEN EXISTS { MATCH (:Company {name: 'Acme'}) } + THEN RETURN 'has Acme' AS result ELSE RETURN 'no Acme' AS result WHEN COUNT { MATCH (n:Person) } > 3 THEN RETURN 'big' AS size @@ -184,7 +193,8 @@ RETURN p.name MATCH (p:Person) RETURN p.name, COUNT { - WHEN p.city = 'London' THEN MATCH (p)-[:KNOWS]->(f) RETURN f AS x + WHEN p.city = 'London' + THEN MATCH (p)-[:KNOWS]->(f) RETURN f AS x ELSE MATCH (p)-[:LIKES]->(x) RETURN x } AS links diff --git a/query/cypher_subset.mdx b/query/cypher_subset.mdx index a3b313b..377d897 100644 --- a/query/cypher_subset.mdx +++ b/query/cypher_subset.mdx @@ -63,14 +63,20 @@ Queries run through the [Python SDK](/pythonsdk/reference), the [JavaScript SDK] from turingdb import TuringDB client = TuringDB(host="localhost", port=6666) -df = client.query("MATCH (n:Person) RETURN n.name, n.age") # a pandas DataFrame +# a pandas DataFrame +df = client.query("MATCH (n:Person) RETURN n.name, n.age") ``` ```js JavaScript import { TuringClient } from "turingdb"; -const client = new TuringClient({ url: "http://localhost:6686/query" }); -const chunks = await client.queryData("MATCH (n:Person) RETURN n.name, n.age"); // chunks of columns +const client = new TuringClient({ + url: "http://localhost:6686/query", +}); +// chunks of columns +const chunks = await client.queryData( + "MATCH (n:Person) RETURN n.name, n.age", +); ``` @@ -84,32 +90,77 @@ const chunks = await client.queryData("MATCH (n:Person) RETURN n.name, n.age"); `client.query()` takes an optional dictionary of parameters. A `$name` parameter stands for a **value** anywhere an expression goes, so values travel next to the query text and need no quoting or escaping: ```python -client.query("MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", {"minAge": 30}) -client.query("MATCH (n:Person {name: $name}) RETURN n.age", {"name": "O'Brien"}) -client.query("MATCH (n:Person) WHERE n.name IN $names RETURN n.name", {"names": ["Alice", "Bob"]}) -client.query("UNWIND $rows AS row RETURN row.name, row.age", {"rows": [{"name": "Alice", "age": 30}]}) -client.query("WITH $filter AS f MATCH (n:Person) WHERE n.city = f.city RETURN n.name", {"filter": {"city": "London"}}) -client.query("RETURN $0 + 1 AS next", {"0": 41}) # numbered parameter +client.query( + "MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", + {"minAge": 30}, +) +client.query( + "MATCH (n:Person {name: $name}) RETURN n.age", + {"name": "O'Brien"}, +) +client.query( + "MATCH (n:Person) WHERE n.name IN $names RETURN n.name", + {"names": ["Alice", "Bob"]}, +) +client.query( + "UNWIND $rows AS row RETURN row.name, row.age", + {"rows": [{"name": "Alice", "age": 30}]}, +) +client.query( + "WITH $filter AS f MATCH (n:Person) WHERE n.city = f.city " + "RETURN n.name", + {"filter": {"city": "London"}}, +) +# numbered parameter +client.query("RETURN $0 + 1 AS next", {"0": 41}) # inside a change -client.query("CREATE (n:Person {name: $name, age: $age})", {"name": "Zoe", "age": 27}) -client.query("MATCH (n:Person {name: 'Alice'}) SET n += $props", {"props": {"city": "Berlin"}}) +client.query( + "CREATE (n:Person {name: $name, age: $age})", + {"name": "Zoe", "age": 27}, +) +client.query( + "MATCH (n:Person {name: 'Alice'}) SET n += $props", + {"props": {"city": "Berlin"}}, +) ``` A parameter can also stand for a **label or an edge type**, anywhere a label or type is written: ```python -client.query("MATCH (n:$label) RETURN n.name", {"label": "Person"}) -client.query("MATCH (n:$labels) RETURN n.name", {"labels": ["Person", "Employee"]}) # every label in the list -client.query("MATCH (n:Person|$other) RETURN n.name", {"other": "Company"}) # inside a label expression -client.query("MATCH (n) WHERE n:$label RETURN n.name", {"label": "Company"}) # in a label test -client.query("MATCH (a)-[r:$type]->(b) RETURN a.name, b.name", {"type": "WORKS_AT"}) -client.query("MATCH (n:$0) RETURN n.name", {"0": "Company"}) # numbered parameter +client.query( + "MATCH (n:$label) RETURN n.name", {"label": "Person"} +) +# every label in the list +client.query( + "MATCH (n:$labels) RETURN n.name", + {"labels": ["Person", "Employee"]}, +) +# inside a label expression +client.query( + "MATCH (n:Person|$other) RETURN n.name", {"other": "Company"} +) +# in a label test +client.query( + "MATCH (n) WHERE n:$label RETURN n.name", {"label": "Company"} +) +client.query( + "MATCH (a)-[r:$type]->(b) RETURN a.name, b.name", + {"type": "WORKS_AT"}, +) +# numbered parameter +client.query("MATCH (n:$0) RETURN n.name", {"0": "Company"}) # inside a change -client.query("CREATE (n:$label {name: 'Zoe'})", {"label": "Person"}) +client.query( + "CREATE (n:$label {name: 'Zoe'})", {"label": "Person"} +) client.query("MERGE (t:$label {name: 'x'})", {"label": "Tag"}) -client.query("MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Bob'}) MERGE (a)-[:$type]->(b)", {"type": "FOLLOWS"}) +client.query( + "MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Bob'})" + " MERGE (a)-[:$type]->(b)", + {"type": "FOLLOWS"}, +) ``` - **Values.** From Python, a parameter can hold a string, a number, a boolean, `None`, a list, a dict, a time-zone-aware `datetime` (a DateTime) or a `timedelta` (a Duration). See [`query()`](/pythonsdk/reference#core-methods) for the full list, and [Parameters](/jssdk/reference#parameters) for JavaScript values. diff --git a/query/functions.mdx b/query/functions.mdx index 8963029..9922550 100644 --- a/query/functions.mdx +++ b/query/functions.mdx @@ -43,7 +43,8 @@ RETURN abs(-3), round(2.5), sqrt(16.0), pi() MATCH (n:Person) RETURN n.name, coalesce(n.nickname, n.name) -RETURN substring('hello', 1, 3), left('hello', 2), split('a,b,c', ','), replace('aXbX', 'X', '-') +RETURN substring('hello', 1, 3), left('hello', 2), + split('a,b,c', ','), replace('aXbX', 'X', '-') MATCH (n:Person) WHERE toLower(n.name) STARTS WITH 'a' @@ -52,7 +53,8 @@ RETURN n.name MATCH (n:Person {name: 'Alice'}) RETURN keys(n), properties(n), elementId(n) -RETURN toIntegerOrNull('x'), toString(datetime('2024-03-15T10:30:00Z')) +RETURN toIntegerOrNull('x'), + toString(datetime('2024-03-15T10:30:00Z')) ``` ### String functions @@ -60,18 +62,21 @@ RETURN toIntegerOrNull('x'), toString(datetime('2024-03-15T10:30:00Z')) Positions are 0-based, and `substring` without a length runs to the end of the string: ```cypher -RETURN substring('hello', 1, 3) AS s, substring('hello', 2) AS rest +RETURN substring('hello', 1, 3) AS s, + substring('hello', 2) AS rest RETURN left('hello', 2) AS l, right('hello', 3) AS r -RETURN split('a.b.c', '.') AS parts, replace('banana', 'a', 'o') AS swapped +RETURN split('a.b.c', '.') AS parts, + replace('banana', 'a', 'o') AS swapped RETURN split('a,b;c', [',', ';']) AS parts RETURN char_length('héllo') AS chars, size('héllo') AS alsoChars MATCH (n:Person) -RETURN toUpper(left(n.name, 1)) + toLower(substring(n.name, 1)) AS display +RETURN toUpper(left(n.name, 1)) + toLower(substring(n.name, 1)) + AS display ``` `split` with a list of delimiters splits on each of them, so the fourth query returns `['a', 'b', 'c']`. @@ -79,19 +84,23 @@ RETURN toUpper(left(n.name, 1)) + toLower(substring(n.name, 1)) AS display `normalize` converts a string to a Unicode normal form (`NFC` by default, or `NFD`, `NFKC`, `NFKD`), and `IS NORMALIZED` tests for one: ```cypher -RETURN normalize('fi', NFKC) AS plain, 'café' IS NFC NORMALIZED AS composed +RETURN normalize('fi', NFKC) AS plain, + 'café' IS NFC NORMALIZED AS composed ``` ### Lists ```cypher -RETURN isEmpty([]) AS emptyList, isEmpty('') AS emptyString, isEmpty({}) AS emptyMap +RETURN isEmpty([]) AS emptyList, isEmpty('') AS emptyString, + isEmpty({}) AS emptyMap MATCH (n:Person {name: 'Carol'}) -RETURN coll.indexOf(n.tags, 'vip') AS position, coll.indexOf(n.tags, 'admin') AS missing +RETURN coll.indexOf(n.tags, 'vip') AS position, + coll.indexOf(n.tags, 'admin') AS missing -RETURN allReduce(total = 0, x IN [1, 2, 3] | total + x, total < 10) AS staysUnder10, - allReduce(total = 0, x IN [1, 2, 3] | total + x, total < 5) AS staysUnder5 +RETURN + allReduce(t = 0, x IN [1, 2, 3] | t + x, t < 10) AS under10, + allReduce(t = 0, x IN [1, 2, 3] | t + x, t < 5) AS under5 ``` - `coll.indexOf(list, value)` returns the position of the first element equal to `value`, or `-1`. The second query returns `1` and `-1`. @@ -103,9 +112,11 @@ RETURN allReduce(total = 0, x IN [1, 2, 3] | total + x, total < 10) AS staysUnde The strict conversions return `null` for a string they cannot read, and reject other values they cannot convert. The `OrNull` forms return `null` instead of rejecting: ```cypher -RETURN toInteger('12') AS i, toInteger('x') AS bad, toInteger(true) AS fromBool, toBoolean(1) AS b +RETURN toInteger('12') AS i, toInteger('x') AS bad, + toInteger(true) AS fromBool, toBoolean(1) AS b -RETURN toIntegerOrNull('12') AS a, toFloatOrNull('2.5') AS c, toBooleanOrNull('yes') AS d, toStringOrNull(3.5) AS e +RETURN toIntegerOrNull('12') AS a, toFloatOrNull('2.5') AS c, + toBooleanOrNull('yes') AS d, toStringOrNull(3.5) AS e RETURN toString(duration({days: 2, hours: 3})) AS iso ``` @@ -147,13 +158,18 @@ RETURN datetime('2024-03-15') AS midnight RETURN datetime(1700000000) AS fromEpoch -RETURN datetime({year: 2024, month: 3, day: 15, hour: 10, minute: 30}) AS fromParts +RETURN datetime({ + year: 2024, month: 3, day: 15, hour: 10, minute: 30 +}) AS fromParts -RETURN datetime('2024-03-15T10:30:00Z') + duration({days: 1, hours: 2}) AS later +RETURN datetime('2024-03-15T10:30:00Z') + + duration({days: 1, hours: 2}) AS later -RETURN datetime('2024-03-16T00:00:00Z') - datetime('2024-03-15T00:00:00Z') AS elapsed +RETURN datetime('2024-03-16T00:00:00Z') + - datetime('2024-03-15T00:00:00Z') AS elapsed -RETURN datetime('2024-03-15T10:30:00Z').year, datetime('2024-03-15T10:30:00Z').month +RETURN datetime('2024-03-15T10:30:00Z').year, + datetime('2024-03-15T10:30:00Z').month RETURN duration({days: 2, hours: 3}).hours ``` @@ -163,7 +179,8 @@ Durations can be added, subtracted, scaled, compared, summed and averaged: ```cypher RETURN duration({hours: 36}) + duration({hours: 12}) AS total -RETURN duration({days: 2, hours: 12}) - duration({days: 1}) AS remaining +RETURN duration({days: 2, hours: 12}) - duration({days: 1}) + AS remaining RETURN duration({days: 2}) * 3 AS threeTimes @@ -254,18 +271,26 @@ RETURN d {.year, .month} AS parts A point is a location, 2D or 3D, either on a plane (cartesian) or on the Earth (WGS-84). `point()` builds one from a map: ```cypher -RETURN point({x: 1, y: 2}) AS flat, point({x: 1, y: 2, z: 3}) AS space +RETURN point({x: 1, y: 2}) AS flat, + point({x: 1, y: 2, z: 3}) AS space RETURN point({latitude: 51.5074, longitude: -0.1278}) AS london WITH point({latitude: 51.5074, longitude: -0.1278}) AS p RETURN p.latitude, p.longitude, p.srid, p.crs -RETURN point.distance(point({x: 0, y: 0}), point({x: 3, y: 4})) AS d +RETURN point.distance( + point({x: 0, y: 0}), point({x: 3, y: 4}) +) AS d -RETURN round(point.distance(point({latitude: 51.5074, longitude: -0.1278}), point({latitude: 48.8566, longitude: 2.3522}))) AS londonToParis +RETURN round(point.distance( + point({latitude: 51.5074, longitude: -0.1278}), + point({latitude: 48.8566, longitude: 2.3522}) +)) AS londonToParis -RETURN point.withinBBox(point({x: 1, y: 1}), point({x: 0, y: 0}), point({x: 2, y: 2})) AS inside +RETURN point.withinBBox( + point({x: 1, y: 1}), point({x: 0, y: 0}), point({x: 2, y: 2}) +) AS inside ``` The keys of the map choose the coordinate reference system (CRS): @@ -288,11 +313,16 @@ The keys of the map choose the coordinate reference system (CRS): Store a point as a property like any other value, and filter on distance: ```cypher -CREATE (:City {name: 'Paris', at: point({latitude: 48.8566, longitude: 2.3522})}), - (:City {name: 'London', at: point({latitude: 51.5074, longitude: -0.1278})}) +CREATE + (:City {name: 'Paris', + at: point({latitude: 48.8566, longitude: 2.3522})}), + (:City {name: 'London', + at: point({latitude: 51.5074, longitude: -0.1278})}) MATCH (c:City) -WHERE point.distance(c.at, point({latitude: 51.5074, longitude: -0.1278})) < 400000 +WHERE point.distance( + c.at, point({latitude: 51.5074, longitude: -0.1278}) +) < 400000 RETURN c.name ``` @@ -326,7 +356,8 @@ Embeddings are stored as ordinary properties (type `Embedding`) and compared wit ```cypher MATCH (n:Document) -RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) AS sim +RETURN n.title, + cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) AS sim ORDER BY sim DESC LIMIT 10 MATCH (n:Document) diff --git a/query/matching.mdx b/query/matching.mdx index 4741b59..49e53ba 100644 --- a/query/matching.mdx +++ b/query/matching.mdx @@ -10,10 +10,14 @@ The examples on this page use a small graph of people, companies and interests. ```cypher CREATE - (alice:Person {name: 'Alice', age: 30, city: 'London', active: true, tags: ['admin', 'vip']}), - (bob:Person {name: 'Bob', age: 35, city: 'London', active: true, tags: ['dev']}), - (carol:Person {name: 'Carol', age: 28, city: 'Paris', active: false, tags: ['dev', 'vip']}), - (dave:Person {name: 'Dave', age: 41, city: 'Paris', active: true}), + (alice:Person {name: 'Alice', age: 30, city: 'London', + active: true, tags: ['admin', 'vip']}), + (bob:Person {name: 'Bob', age: 35, city: 'London', + active: true, tags: ['dev']}), + (carol:Person {name: 'Carol', age: 28, city: 'Paris', + active: false, tags: ['dev', 'vip']}), + (dave:Person {name: 'Dave', age: 41, city: 'Paris', + active: true}), (acme:Company {name: 'Acme', founded: 1999}), (globex:Company {name: 'Globex', founded: 2010}), (music:Interest {name: 'Music'}), @@ -112,7 +116,8 @@ RETURN n.name A `WHERE` can sit inside a node or edge pattern. The variable must be named: ```cypher -MATCH (a:Person WHERE a.age > 30)-[r:KNOWS WHERE r.since > 2012]->(b) +MATCH (a:Person WHERE a.age > 30) + -[r:KNOWS WHERE r.since > 2012]->(b) RETURN b.name ``` @@ -161,7 +166,9 @@ WHERE n.age >= 18 AND n.city = 'London' RETURN n.name MATCH (n:Person) -WHERE n.name STARTS WITH 'A' OR n.name ENDS WITH 'e' OR n.name CONTAINS 'ro' +WHERE n.name STARTS WITH 'A' + OR n.name ENDS WITH 'e' + OR n.name CONTAINS 'ro' RETURN n.name MATCH (n:Person) @@ -197,7 +204,8 @@ WHERE NOT (p)-[:KNOWS]->() RETURN p.name MATCH (p:Person) -RETURN p.name, exists((p)-[:WORKS_AT]->(:Company {name: 'Acme'})) AS atAcme +RETURN p.name, + exists((p)-[:WORKS_AT]->(:Company {name: 'Acme'})) AS atAcme ``` - **Label and type tests:** `n:Person` tests a node label, `e:KNOWS` tests an edge type, and `n IS Person` is the same test written with `IS`. Combine them with `AND`, `OR` and `NOT`, or write a [label expression](#label-expressions) such as `n:Person|Company` or `n IS Person|Company`. Negate an `IS` test with `NOT n IS Person`. @@ -321,7 +329,10 @@ RETURN n.name ```python client.query("MATCH (n) WHERE n = $id RETURN n.name", {"id": 0}) -client.query("UNWIND $ids AS x MATCH (n) WHERE n = x RETURN x, n.name", {"ids": [0, 1, 2]}) +client.query( + "UNWIND $ids AS x MATCH (n) WHERE n = x RETURN x, n.name", + {"ids": [0, 1, 2]}, +) ``` - **Batches.** `UNWIND` a list of IDs and compare each one: `UNWIND $ids AS x MATCH (n) WHERE n = x`. The list can be a parameter, written in the query, held in a variable, or built with `collect()` or `range()`, and each ID costs a few microseconds. A list written in the query also works as `WHERE n IN [0, 1, 2]`. diff --git a/query/paths.mdx b/query/paths.mdx index 6a3bf38..31aa20b 100644 --- a/query/paths.mdx +++ b/query/paths.mdx @@ -56,27 +56,33 @@ RETURN b.name, [r IN e | r.since] AS sinces A quantified path pattern repeats a sub-pattern, with its own `WHERE`, a number of times: ```cypher -MATCH (a:Person {name: 'Alice'}) ((x)-[r:KNOWS]->(y) WHERE r.since > 2009){1,3} (b) +MATCH (a:Person {name: 'Alice'}) + ((x)-[r:KNOWS]->(y) WHERE r.since > 2009){1,3} (b) RETURN b.name -MATCH (a:Person {name: 'Alice'}) ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) +MATCH (a:Person {name: 'Alice'}) + ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) RETURN b.name -MATCH (a:Person {name: 'Alice'}) ((x)-[:KNOWS]->(y)-[:KNOWS]->(z)){1,2} (b) +MATCH (a:Person {name: 'Alice'}) + ((x)-[:KNOWS]->(y)-[:KNOWS]->(z)){1,2} (b) RETURN b.name -MATCH (a:Person {name: 'Alice'}) ((x)-[r:KNOWS]->(y)){1,3} (b) +MATCH (a:Person {name: 'Alice'}) + ((x)-[r:KNOWS]->(y)){1,3} (b) RETURN b.name, [n IN y | n.name] AS via, size(r) AS hops ``` The second query follows `KNOWS` edges only while ages strictly increase, and the third repeats a two-hop sub-pattern. The inner variables (`x`, `y`, `r`) are **group variables**: each becomes a list with one element per repetition, which you can use in comprehensions, `size()`, `UNWIND` and sort keys: ```cypher -MATCH (a:Person {name: 'Alice'}) ((x)-[r:KNOWS]->(y)){1,3} (b:Person {name: 'Dave'}) +MATCH (a:Person {name: 'Alice'}) + ((x)-[r:KNOWS]->(y)){1,3} (b:Person {name: 'Dave'}) UNWIND r AS e RETURN e.since -MATCH (a:Person {name: 'Alice'}) ((x)-[r:KNOWS]->(y)){1,3} (b) +MATCH (a:Person {name: 'Alice'}) + ((x)-[r:KNOWS]->(y)){1,3} (b) RETURN b.name ORDER BY size(r) DESC LIMIT 1 ``` @@ -106,7 +112,8 @@ Paths are values like any other: `collect(p)` gathers them, lists and maps hold ```cypher MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) WITH collect(p) AS routes -RETURN size(routes) AS routeCount, reduce(total = 0, q IN routes | total + length(q)) AS totalHops +RETURN size(routes) AS routeCount, + reduce(t = 0, q IN routes | t + length(q)) AS totalHops MATCH (a:Person {name: 'Alice'}) RETURN [p = (a)-[:KNOWS]->+() | length(p)] AS lengths @@ -123,7 +130,10 @@ A property cannot hold a path: `A property value cannot hold a path, in a list o ```cypher MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) -WHERE allReduce(total = 0, n IN nodes(p) | total + n.age, total <= 100) +WHERE allReduce( + total = 0, n IN nodes(p) | total + n.age, + total <= 100 +) RETURN b.name, length(p) AS hops ``` @@ -133,7 +143,11 @@ A map accumulator carries more than one value from step to step. This query keep ```cypher MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) -WHERE allReduce(acc = {}, r IN relationships(p) | {prev: acc.cur, cur: r.since}, acc.prev IS NULL OR acc.prev < acc.cur) +WHERE allReduce( + acc = {}, r IN relationships(p) | + {prev: acc.cur, cur: r.since}, + acc.prev IS NULL OR acc.prev < acc.cur +) RETURN b.name, length(p) AS hops ``` @@ -145,7 +159,8 @@ For hop-count questions, undirected search, or when you need the properties alon ```cypher // Fewest hops, returning the names along the route -MATCH p = (a:Person {name: 'Alice'})-[:KNOWS*1..6]->(b:Person {name: 'Dave'}) +MATCH p = (a:Person {name: 'Alice'}) + -[:KNOWS*1..6]->(b:Person {name: 'Dave'}) RETURN [n IN nodes(p) | n.name] AS route, length(p) AS hops ORDER BY hops LIMIT 1 @@ -160,9 +175,10 @@ RETURN count(DISTINCT b) To sum a property along a route, fold the path's edges with `reduce`: ```cypher -MATCH p = (a:Person {name: 'Alice'})-[:KNOWS*1..6]->(b:Person {name: 'Dave'}) +MATCH p = (a:Person {name: 'Alice'}) + -[:KNOWS*1..6]->(b:Person {name: 'Dave'}) RETURN [n IN nodes(p) | n.name] AS route, - reduce(total = 0, r IN relationships(p) | total + r.since) AS total + reduce(t = 0, r IN relationships(p) | t + r.since) AS total ORDER BY total LIMIT 1 ``` diff --git a/query/procedures.mdx b/query/procedures.mdx index 224f316..d78bbf3 100644 --- a/query/procedures.mdx +++ b/query/procedures.mdx @@ -74,10 +74,12 @@ LIMIT 5 The procedures `db.getNodes` and `db.listNodes` also return the properties, as a JSON string, together with edge counts: ```cypher -CALL db.getNodes([0, 1, 2]) YIELD id, labels, inEdgeCount, outEdgeCount, properties +CALL db.getNodes([0, 1, 2]) +YIELD id, labels, inEdgeCount, outEdgeCount, properties RETURN id, labels, inEdgeCount, outEdgeCount, properties -CALL db.listNodes(['Person'], {name: 'ali'}, 0, 10) YIELD id, labels, properties +CALL db.listNodes(['Person'], {name: 'ali'}, 0, 10) +YIELD id, labels, properties ``` `db.listNodes(labels, properties, skip, limit)` does a case-insensitive substring search on property values. @@ -85,7 +87,9 @@ CALL db.listNodes(['Person'], {name: 'ali'}, 0, 10) YIELD id, labels, properties ```python import json -df = client.query("CALL db.getNodes([0, 1, 2]) YIELD id, labels, properties") +df = client.query( + "CALL db.getNodes([0, 1, 2]) YIELD id, labels, properties" +) df["properties"] = df["properties"].map(json.loads) ``` @@ -97,7 +101,8 @@ RETURN labels(n) AS labels, count(*) AS n ORDER BY n DESC MATCH (a)-[r]->(b) -RETURN labels(a) AS src, type(r) AS rel, labels(b) AS dst, count(*) AS n +RETURN labels(a) AS src, type(r) AS rel, labels(b) AS dst, + count(*) AS n ORDER BY n DESC MATCH (n:Person) diff --git a/query/subqueries.mdx b/query/subqueries.mdx index da3e66d..e9a13fc 100644 --- a/query/subqueries.mdx +++ b/query/subqueries.mdx @@ -78,7 +78,8 @@ Like `EXISTS`, the body of `COUNT` can hold a `UNION`, a [`NEXT`](/query/composi ```cypher MATCH (p:Person) -RETURN p.name, COLLECT { MATCH (p)-[:KNOWS]->(f) RETURN f.name } AS friends +RETURN p.name, + COLLECT { MATCH (p)-[:KNOWS]->(f) RETURN f.name } AS friends ``` The body is a full query, so it can filter, deduplicate, sort and limit: @@ -144,7 +145,8 @@ UNWIND COLLECT { MATCH (p)-[:KNOWS]-(f) RETURN f.name } AS friend RETURN friend MATCH (p:Person {name: 'Alice'}) -SET p.friendNames = COLLECT { MATCH (p)-[:KNOWS]->(f) RETURN f.name } +SET p.friendNames = + COLLECT { MATCH (p)-[:KNOWS]->(f) RETURN f.name } ``` - **One column.** The body returns exactly one column. Two columns are reported: `A COLLECT subquery must return exactly one column, not 2`. To collect several values per match, return a map as above. diff --git a/query/writing.mdx b/query/writing.mdx index 73c465a..72289b7 100644 --- a/query/writing.mdx +++ b/query/writing.mdx @@ -9,9 +9,13 @@ All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`, `FOREACH`) must run ** ## The change workflow ```python -client.new_change() # CHANGE NEW: the client is now on this change +# CHANGE NEW: the client is now on this change +client.new_change() -client.query("CREATE (:Person {name: 'Alice'})-[:KNOWS]->(:Person {name: 'Bob'})") +client.query( + "CREATE (:Person {name: 'Alice'})" + "-[:KNOWS]->(:Person {name: 'Bob'})" +) # ... more writes ... client.query("CHANGE SUBMIT") # merge the change into main @@ -40,9 +44,15 @@ See [Create a change](/graph_dev/create_change), [Submit a change](/graph_dev/su ```python client.new_change() -client.query("CREATE (:Person {name: 'Alice'}), (:Person {name: 'Bob'})") -client.query("COMMIT") # make the new nodes visible to the next query -client.query("MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Bob'}) CREATE (a)-[:KNOWS]->(b)") +client.query( + "CREATE (:Person {name: 'Alice'}), (:Person {name: 'Bob'})" +) +# make the new nodes visible to the next query +client.query("COMMIT") +client.query( + "MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Bob'})" + " CREATE (a)-[:KNOWS]->(b)" +) client.query("CHANGE SUBMIT") client.checkout() ``` @@ -63,14 +73,19 @@ CREATE (:Person {name: 'Alice', age: 30}) CREATE (a:Person:Employee {name: 'Bob'}) RETURN a.name, id(a) -CREATE (:Person {name: 'Mick'})-[:FRIEND_OF {since: 2020}]->(:Person {name: 'John'}) +CREATE (:Person {name: 'Mick'}) + -[:FRIEND_OF {since: 2020}]->(:Person {name: 'John'}) CREATE (a:T {x: 1}), (b:T {x: 2}), (a)-[:R]->(b) -CREATE p = (:City {name: 'Paris'})-[:ROAD {km: 450}]->(:City {name: 'Lyon'}) +CREATE p = (:City {name: 'Paris'}) + -[:ROAD {km: 450}]->(:City {name: 'Lyon'}) RETURN p -CREATE (c:City {name: 'Li' + 'lle', pop: 2 * 1000, tags: ['a', 'b'], info: {zone: 1}, emb: (0.1, 0.2, 0.3)}) +CREATE (c:City { + name: 'Li' + 'lle', pop: 2 * 1000, tags: ['a', 'b'], + info: {zone: 1}, emb: (0.1, 0.2, 0.3) +}) CREATE (n:Item {sku: 'A1'}) SET n.price = 9.99 @@ -264,9 +279,13 @@ FOREACH (nm IN ['x', 'y', 'x'] | MERGE (:Tag {name: nm})) MATCH (n:Person) WITH collect(n) AS people -FOREACH (p IN people | SET p.reviewed = true, p.nextAge = p.age + 1) +FOREACH (p IN people | + SET p.reviewed = true, p.nextAge = p.age + 1 +) -FOREACH (i IN [1, 2] | FOREACH (j IN [10, 20] | CREATE (:Cell {v: i * j}))) +FOREACH (i IN [1, 2] | + FOREACH (j IN [10, 20] | CREATE (:Cell {v: i * j})) +) ``` The second example creates two `Tag` nodes, since `MERGE` deduplicates, and the last shows that `FOREACH` blocks nest. @@ -290,7 +309,9 @@ The elements of a list property such as `n.tags` are stored as they are, and a n ```cypher MATCH (n:Person) -FOREACH (x IN CASE WHEN n.age > 35 THEN [1] ELSE [] END | SET n.senior = true) +FOREACH (x IN CASE WHEN n.age > 35 THEN [1] ELSE [] END | + SET n.senior = true +) ``` A `RETURN` can follow, since `FOREACH` keeps the rows: `MATCH (n:Person) FOREACH (x IN [1] | SET n.seen = true) RETURN count(*)`. @@ -320,9 +341,14 @@ RETURN count(*) Read each node's ID once together with your own key, map your edge list to ID pairs in your application, and create the edges in batches of about 10,000 pairs, passed as a [parameter](/query/cypher_subset#query-parameters): ```python -ids = client.query("MATCH (n:Person) RETURN n.pid AS pid, id(n) AS id") -key_to_id = dict(zip(ids["pid"].tolist(), ids["id"].tolist())) # .tolist() gives plain ints -pairs = [[key_to_id[src], key_to_id[dst]] for src, dst in edge_list] +ids = client.query( + "MATCH (n:Person) RETURN n.pid AS pid, id(n) AS id" +) +# .tolist() gives plain ints +key_to_id = dict(zip(ids["pid"].tolist(), ids["id"].tolist())) +pairs = [ + [key_to_id[src], key_to_id[dst]] for src, dst in edge_list +] client.new_change() for i in range(0, len(pairs), 10_000): @@ -356,7 +382,10 @@ CREATE (:City {name: row[0], code: row[1]}) Rows can also be maps, or lists of mixed types, and their values are stored as they are: ```cypher -UNWIND [{name: 'Brest', pop: 139000}, {name: 'Rennes', pop: 220000}] AS row +UNWIND [ + {name: 'Brest', pop: 139000}, + {name: 'Rennes', pop: 220000} +] AS row CREATE (:City {name: row.name, pop: row.pop}) UNWIND [['Brest', 139000], ['Rennes', 220000]] AS row @@ -380,8 +409,15 @@ Use `UNWIND` or [`FOREACH`](#foreach) to loop over a list. **From Python, pass the rows as a parameter.** Values in a parameter need no quoting or escaping, and take less room than the same values written into the query text: ```python -rows = [{"name": "Brest", "pop": 139000}, {"name": "O'Brien Town", "pop": 1200}] -client.query("UNWIND $rows AS row MERGE (c:City {name: row.name}) SET c.pop = row.pop", {"rows": rows}) +rows = [ + {"name": "Brest", "pop": 139000}, + {"name": "O'Brien Town", "pop": 1200}, +] +client.query( + "UNWIND $rows AS row MERGE (c:City {name: row.name}) " + "SET c.pop = row.pop", + {"rows": rows}, +) ``` A request, the query text and its parameters together, can be up to 1 MiB. Split larger batches, as in the edge recipe above. diff --git a/quickstart.mdx b/quickstart.mdx index 93e7535..8d7b2c7 100644 --- a/quickstart.mdx +++ b/quickstart.mdx @@ -18,7 +18,8 @@ description: "Start using TuringDB Community Version with the TuringDB python SD **Using Claude Code?** Install the TuringDB skill once and let Claude handle the rest of this quickstart for you: ```bash - npx skills add https://github.com/turing-db/turingdb-skills + npx skills add \ + https://github.com/turing-db/turingdb-skills ``` Then in Claude Code run `/turingdb install turingdb and create a first graph`. See the [Claude Code Skill](/claude-skill) page for more. @@ -80,8 +81,7 @@ The steps below walk you through setting up a Python project, installing the Tur ```python from turingdb import TuringDB - # Create TuringDB client - # host and port of the running TuringDB server (default localhost:6666) + # Connect to the TuringDB server (default: localhost:6666) client = TuringDB(host="localhost", port=6666) # Create a new graph called my_graph @@ -97,7 +97,10 @@ The steps below walk you through setting up a Python project, installing the Tur client.checkout(change=change) # Create a node Person (Jane) - Edge (knows) - node (John) - client.query("CREATE (n:Person {name: 'Jane'})-[e:KNOWS]->(m:Person {name: 'John'})") + client.query( + "CREATE (n:Person {name: 'Jane'})" + "-[e:KNOWS]->(m:Person {name: 'John'})" + ) # Commit the change client.query("COMMIT") diff --git a/releases/v3-1.mdx b/releases/v3-1.mdx index 659f5be..c30b67a 100644 --- a/releases/v3-1.mdx +++ b/releases/v3-1.mdx @@ -1,10 +1,12 @@ --- title: "TuringDB 3.1" sidebarTitle: "TuringDB 3.1" -description: "Released November 9, 2026. Query parameters, the binary protocol over HTTP/1.1 by default, a WebAssembly-based TypeScript client on npm, allReduce, COLLECT and other subqueries that hold any query, and a point type with haversine distances." +description: "We released TuringDB 3.1 on November 9, 2026. With this version, you can send values with your queries as parameters and use TuringDB from TypeScript. It also adds new Cypher features." --- -TuringDB 3.1 brings query parameters, makes the binary protocol over HTTP/1.1 the way every client talks to the server, and adds a TypeScript client on npm, built on a WebAssembly build of the engine's decoder. Cypher gains `allReduce`, subqueries that hold any query, and a point type with great-circle distances. Most queries run unchanged. The main change for existing applications is how the Python SDK connects, see [Upgrading from 3.0](#upgrading-from-3-0). +TuringDB 3.1 adds query parameters. Now you can send values with a query, and you do not have to write them into the query text. All clients now use TuringDB's binary protocol, which runs over HTTP/1.1. A new TypeScript client is available on npm. This client uses WebAssembly to read the results. For Cypher, this version adds the `allReduce` function, and subqueries can now contain any query. It also adds a point type, which you can use to calculate distances on the Earth. + +Most of your queries work without changes. If you use the Python SDK, you must change how you connect to the server. Refer to [Upgrading from 3.0](#upgrading-from-3-0). @@ -54,19 +56,28 @@ A `$name` parameter now stands for a value anywhere an expression goes, as well ```python Python -client.query("MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", {"minAge": 30}) +client.query( + "MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", + {"minAge": 30}, +) # inside a change +rows = [ + {"name": "Brest", "pop": 139000}, + {"name": "Rennes", "pop": 220000}, +] client.query( - "UNWIND $rows AS row CREATE (:City {name: row.name, pop: row.pop})", - {"rows": [{"name": "Brest", "pop": 139000}, {"name": "Rennes", "pop": 220000}]}, + "UNWIND $rows AS row " + "CREATE (:City {name: row.name, pop: row.pop})", + {"rows": rows}, ) ``` ```js JavaScript -await client.query("MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", { - parameters: { minAge: 30 }, -}); +await client.query( + "MATCH (n:Person) WHERE n.age >= $minAge RETURN n.name", + { parameters: { minAge: 30 } }, +); ``` ```text Shell @@ -99,7 +110,8 @@ Content-Type: application/turing-proto Transfer-Encoding: chunked Connection: Keep-Alive - one binary packet per chunk: the column header, the column chunks, the end + one binary packet per chunk: + the column header, the column chunks, the end ``` - **Typed columns end to end.** A result streams as typed columns, in chunks of up to 65,536 rows, which each client turns straight into its own types: a pandas DataFrame in Python, typed arrays and `Column` objects in JavaScript. @@ -128,16 +140,22 @@ The `turingdb` npm package queries TuringDB from TypeScript and JavaScript. It d ```ts import { TuringClient, TuringQueryError } from "turingdb"; -const client = new TuringClient({ url: "http://localhost:6686/query", graph: "people" }); +const client = new TuringClient({ + url: "http://localhost:6686/query", + graph: "people", +}); try { const [[names, ages]] = await client.queryData( - "MATCH (p:Person) WHERE p.age >= $age RETURN p.name, p.age ORDER BY p.name", + "MATCH (p:Person) WHERE p.age >= $age " + + "RETURN p.name, p.age ORDER BY p.name", { parameters: { age: 30 } }, ); console.log(names, ages); // [ 'Marie' ] [ 33 ] } catch (e) { - if (e instanceof TuringQueryError) console.error(e.status, e.message); + if (e instanceof TuringQueryError) { + console.error(e.status, e.message); + } } ``` @@ -153,7 +171,10 @@ Read [Get started](/jssdk/get_started) for a walkthrough, including a Vite and n ```cypher MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) -WHERE allReduce(total = 0, n IN nodes(p) | total + n.age, total <= 100) +WHERE allReduce( + total = 0, n IN nodes(p) | total + n.age, + total <= 100 +) RETURN b.name, length(p) AS hops ``` @@ -161,7 +182,11 @@ The ages along `Alice → Bob → Carol` add up to 93, so the paths to Bob and C ```cypher MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) -WHERE allReduce(acc = {}, r IN relationships(p) | {prev: acc.cur, cur: r.since}, acc.prev IS NULL OR acc.prev < acc.cur) +WHERE allReduce( + acc = {}, r IN relationships(p) | + {prev: acc.cur, cur: r.since}, + acc.prev IS NULL OR acc.prev < acc.cur +) RETURN b.name, length(p) AS hops ``` @@ -189,7 +214,8 @@ RETURN p.name MATCH (p:Person) RETURN p.name, COUNT { - WHEN p.city = 'London' THEN MATCH (p)-[:KNOWS]->(f) RETURN f AS x + WHEN p.city = 'London' + THEN MATCH (p)-[:KNOWS]->(f) RETURN f AS x ELSE MATCH (p)-[:LIKES]->(x) RETURN x } AS links ``` @@ -205,14 +231,21 @@ See [Subqueries](/query/subqueries) and [NEXT and WHEN](/query/composition#insid Points are a new property type, 2D or 3D, on a plane (cartesian) or on the Earth (WGS-84). `point.distance` gives the great-circle distance in metres between two WGS-84 points, by the haversine formula on a sphere of the mean Earth radius: ```cypher -CREATE (:City {name: 'Paris', at: point({latitude: 48.8566, longitude: 2.3522})}), - (:City {name: 'London', at: point({latitude: 51.5074, longitude: -0.1278})}) +CREATE + (:City {name: 'Paris', + at: point({latitude: 48.8566, longitude: 2.3522})}), + (:City {name: 'London', + at: point({latitude: 51.5074, longitude: -0.1278})}) MATCH (a:City {name: 'London'}), (b:City {name: 'Paris'}) RETURN round(point.distance(a.at, b.at) / 1000) AS km MATCH (c:City) -WHERE point.withinBBox(c.at, point({latitude: 48, longitude: 1}), point({latitude: 50, longitude: 3})) +WHERE point.withinBBox( + c.at, + point({latitude: 48, longitude: 1}), + point({latitude: 50, longitude: 3}) +) RETURN c.name ``` @@ -233,7 +266,8 @@ See [Points](/query/functions#points). ```cypher MATCH (n:Person)-[:WORKS_AT]->(c:Company) -RETURN c.name, collect(n {.name, .age, senior: n.age > 30}) AS staff +RETURN c.name, + collect(n {.name, .age, senior: n.age > 30}) AS staff WITH {a: {b: 2}, label: 'x'} AS m RETURN m.a.b, m['label'] @@ -267,8 +301,9 @@ SET c = {name: 'Acme Corp', size: 10} ### A new parser -A hand-written parser replaces the generated one, and reads queries the way Neo4j does: +A hand-written recursive descent parser replaces the generated one. It is much faster, and reads queries the way Neo4j does: +- **Much faster.** In our measurements, it parses queries about twice as fast as the parser of 3.0: a 12-line query in 23 µs instead of 54 µs, and a 130 KB query in 6.6 ms instead of 13 ms. - `/* */` block comments, and keywords as names: `RETURN n.name AS count, n.age AS order`. - `STARTS WITH`, `ENDS WITH` and `CONTAINS` bind more loosely than `+`, so `'a' + 'bc' ENDS WITH 'bc'` is `true`. - Dotted chains read through nested values: `m.a.b`, `m['key']`, `ms[1].a`. @@ -312,7 +347,8 @@ See the [Python SDK reference](/pythonsdk/reference). ```python from turingdb import TuringDB - client = TuringDB(host="localhost", port=6666) # or TuringDB() + # or simply TuringDB() + client = TuringDB(host="localhost", port=6666) ``` A URL in `host` is refused with `host must be a host name or IP address, not a URL`. The client now connects as soon as it is created, so a server that is not running is reported right away. diff --git a/shortest-path.mdx b/shortest-path.mdx index d10db8e..386c9b4 100644 --- a/shortest-path.mdx +++ b/shortest-path.mdx @@ -41,7 +41,10 @@ CREATE (asch)-[:LINK {distance: 7.25}]->(cnm), ## Syntax ```jsx -shortestPath(sourceSet, targetSet, edgePropName, distanceReturnVar, pathReturnVar) +shortestPath( + sourceSet, targetSet, edgePropName, + distanceReturnVar, pathReturnVar +) ``` | Parameter | Description | @@ -57,7 +60,8 @@ shortestPath(sourceSet, targetSet, edgePropName, distanceReturnVar, pathReturnVa ```jsx - MATCH (n{name:"Ashchurch"}), (m{name:"Worcestershire Parkway"}) + MATCH (n{name:"Ashchurch"}), + (m{name:"Worcestershire Parkway"}) shortestPath(n, m, distance, dist, path) RETURN dist, path ``` @@ -65,7 +69,8 @@ shortestPath(sourceSet, targetSet, edgePropName, distanceReturnVar, pathReturnVa ```python df = client.query(""" - MATCH (n{name:"Ashchurch"}), (m{name:"Worcestershire Parkway"}) + MATCH (n{name:"Ashchurch"}), + (m{name:"Worcestershire Parkway"}) shortestPath(n, m, distance, dist, path) RETURN dist, path """) @@ -88,7 +93,8 @@ Both output variables are always computed — you can choose to return one or bo ```jsx - MATCH (n{name:"Ashchurch"}), (m{name:"Worcestershire Parkway"}) + MATCH (n{name:"Ashchurch"}), + (m{name:"Worcestershire Parkway"}) shortestPath(n, m, distance, dist, path) RETURN dist ``` @@ -96,7 +102,8 @@ Both output variables are always computed — you can choose to return one or bo ```python df = client.query(""" - MATCH (n{name:"Ashchurch"}), (m{name:"Worcestershire Parkway"}) + MATCH (n{name:"Ashchurch"}), + (m{name:"Worcestershire Parkway"}) shortestPath(n, m, distance, dist, path) RETURN dist """) @@ -149,7 +156,8 @@ If a set matches several source and target nodes, only the single best pair is r The variables in the pattern going into `shortestPath` are consumed by the statement, so `RETURN` takes `dist` and `path`. Asking for a pattern variable such as `n` is reported as an error: ```jsx - MATCH (n{name:"Ashchurch"}), (m{name:"Worcestershire Parkway"}) + MATCH (n{name:"Ashchurch"}), + (m{name:"Worcestershire Parkway"}) shortestPath(n, m, distance, dist, path) RETURN dist, path, n ``` @@ -170,13 +178,19 @@ To turn the IDs in `path` back into names, run a second query. The even position ```python row = client.query(""" -MATCH (a:Station {name: 'Bromsgrove'}), (b:Station {name: 'Worcester Shrub Hill'}) +MATCH (a:Station {name: 'Bromsgrove'}), + (b:Station {name: 'Worcester Shrub Hill'}) shortestPath(a, b, distance, dist, path) RETURN dist, path """).iloc[0] -node_ids = [int(i) for i in row["path"][0::2]] # the stations, from source to target -names = client.query(f"UNWIND {node_ids} AS x MATCH (n) WHERE n = x RETURN x AS id, n.name AS name") +# the stations, from source to target +node_ids = [int(i) for i in row["path"][0::2]] +names = client.query( + "UNWIND $ids AS x MATCH (n) WHERE n = x " + "RETURN x AS id, n.name AS name", + {"ids": node_ids}, +) ``` Each station is looked up directly by its ID (see [lookup by node ID](/query/matching#look-up-nodes-by-native-node-id)). diff --git a/tutorials/example_notebooks.mdx b/tutorials/example_notebooks.mdx index 6ab0399..67e0723 100644 --- a/tutorials/example_notebooks.mdx +++ b/tutorials/example_notebooks.mdx @@ -21,14 +21,16 @@ Query examples: ```jsx // Find all transactions between two accounts MATCH (a1:Account)-->(t:Transaction)-->(a2:Account) -RETURN a1.id, a1.type, t.amount, t.type, a2.id, a2.type, t.is_fraud +RETURN a1.id, a1.type, t.amount, t.type, + a2.id, a2.type, t.is_fraud // Find all 2-hops transaction chains MATCH (a1:Account)-[s1:SENT]->(t1:Transaction)-[r1:RECEIVED]-> (a2:Account)-[s2:SENT]->(t2:Transaction)-[r2:RECEIVED]-> (a3:Account) -RETURN a1.id, a2.id, a3.id, t1.id, t1.type, t1.amount, t1.step, t1.is_fraud , - t2.id, t2.type, t2.amount, t2.step, t2.is_fraud +RETURN a1.id, a2.id, a3.id, + t1.id, t1.type, t1.amount, t1.step, t1.is_fraud, + t2.id, t2.type, t2.amount, t2.step, t2.is_fraud ``` ### [Crypto Orbitaal Fraud Detection](https://github.com/turing-db/turingdb-examples/blob/main/examples/notebooks/public_version/crypto_orbitaal_fraud_detection.ipynb) @@ -95,7 +97,8 @@ Query examples: // Find all relationships involving FPGAs MATCH (n)-[e]->(m) WHERE n.description = 'FPGA' -RETURN n.displayName, n.description, e, m.displayName, m.description +RETURN n.displayName, n.description, e, + m.displayName, m.description ``` ## Domain: Healthcare & Life Sciences diff --git a/vector-search.mdx b/vector-search.mdx index 42f37fd..192bf13 100644 --- a/vector-search.mdx +++ b/vector-search.mdx @@ -22,7 +22,8 @@ A vector index is defined by a name, a dimension, a distance metric and, optiona **Syntax:** ``` -CREATE VECTOR INDEX WITH DIMENSION METRIC [TYPE ] +CREATE VECTOR INDEX + WITH DIMENSION METRIC [TYPE ] ``` | Parameter | Description | @@ -35,14 +36,22 @@ CREATE VECTOR INDEX WITH DIMENSION METRIC [TYPE ] ```jsx - CREATE VECTOR INDEX doc_embeddings WITH DIMENSION 768 METRIC COSINE - CREATE VECTOR INDEX doc_hnsw WITH DIMENSION 768 METRIC EUCLID TYPE HNSW + CREATE VECTOR INDEX doc_embeddings + WITH DIMENSION 768 METRIC COSINE + CREATE VECTOR INDEX doc_hnsw + WITH DIMENSION 768 METRIC EUCLID TYPE HNSW ``` ```python - client.query("CREATE VECTOR INDEX doc_embeddings WITH DIMENSION 768 METRIC COSINE") - client.query("CREATE VECTOR INDEX doc_hnsw WITH DIMENSION 768 METRIC EUCLID TYPE HNSW") + client.query( + "CREATE VECTOR INDEX doc_embeddings " + "WITH DIMENSION 768 METRIC COSINE" + ) + client.query( + "CREATE VECTOR INDEX doc_hnsw " + "WITH DIMENSION 768 METRIC EUCLID TYPE HNSW" + ) ``` @@ -81,7 +90,9 @@ LOAD VECTOR FROM "" IN ```python - client.query('LOAD VECTOR FROM "document_vectors.csv" IN doc_embeddings') + client.query( + 'LOAD VECTOR FROM "document_vectors.csv" IN doc_embeddings' + ) ``` @@ -89,12 +100,16 @@ LOAD VECTOR FROM "" IN To build that file from your graph, export `id(n)` next to the text you want to embed, compute the embeddings in Python, and write `id,v1,...,vd` lines: ```python -docs = client.query("MATCH (d:Document) RETURN id(d) AS id, d.text AS text") -vectors = model.encode(docs["text"].tolist()) # any embedding model +docs = client.query( + "MATCH (d:Document) RETURN id(d) AS id, d.text AS text" +) +# any embedding model +vectors = model.encode(docs["text"].tolist()) with open("/home/me/.turing/data/document_vectors.csv", "w") as f: for node_id, vec in zip(docs["id"], vectors): - f.write(f"{node_id}," + ",".join(f"{x:.6f}" for x in vec) + "\n") + values = ",".join(f"{x:.6f}" for x in vec) + f.write(f"{node_id},{values}\n") ``` ## Search @@ -103,7 +118,8 @@ with open("/home/me/.turing/data/document_vectors.csv", "w") as f: **Syntax:** ``` -VECTOR SEARCH IN FOR () YIELD ids [AS ] [, score [AS ]] +VECTOR SEARCH IN FOR () + YIELD ids [AS ] [, score [AS ]] ``` | Parameter | Description | @@ -119,14 +135,16 @@ VECTOR SEARCH IN FOR () YIELD ids [AS ] [, score ```jsx - VECTOR SEARCH IN doc_embeddings FOR 5 (0.12, 0.45, 0.78, 0.33) YIELD ids, score + VECTOR SEARCH IN doc_embeddings FOR 5 (0.12, 0.45, 0.78, 0.33) + YIELD ids, score RETURN ids, score ``` ```python df = client.query(""" - VECTOR SEARCH IN doc_embeddings FOR 5 (0.12, 0.45, 0.78, 0.33) YIELD ids, score + VECTOR SEARCH IN doc_embeddings FOR 5 (0.12, 0.45, 0.78, 0.33) + YIELD ids, score RETURN ids, score """) print(df) @@ -141,7 +159,8 @@ This is where it gets interesting. When the vector file is keyed by native node ```jsx - VECTOR SEARCH IN doc_embeddings FOR 10 (0.12, 0.45, 0.78, 0.33) YIELD ids AS doc, score + VECTOR SEARCH IN doc_embeddings FOR 10 (0.12, 0.45, 0.78, 0.33) + YIELD ids AS doc, score WHERE score > 0.8 MATCH (doc)-[:ABOUT]->(t:Topic) RETURN doc.title, t.name, score @@ -150,7 +169,8 @@ This is where it gets interesting. When the vector file is keyed by native node ```python df = client.query(""" - VECTOR SEARCH IN doc_embeddings FOR 10 (0.12, 0.45, 0.78, 0.33) YIELD ids AS doc, score + VECTOR SEARCH IN doc_embeddings FOR 10 (0.12, 0.45, 0.78, 0.33) + YIELD ids AS doc, score WHERE score > 0.8 MATCH (doc)-[:ABOUT]->(t:Topic) RETURN doc.title, t.name, score @@ -192,13 +212,15 @@ This removes the index and frees the associated resources. ```jsx // 1. Create a vector index for product embeddings - CREATE VECTOR INDEX product_vectors WITH DIMENSION 384 METRIC COSINE + CREATE VECTOR INDEX product_vectors + WITH DIMENSION 384 METRIC COSINE - // 2. Load embeddings generated by your model (keyed by native node ID) + // 2. Load your model's embeddings (keyed by native node ID) LOAD VECTOR FROM "product_embeddings.csv" IN product_vectors // 3. Find the 10 most similar products to a query embedding - VECTOR SEARCH IN product_vectors FOR 10 (0.15, 0.82, 0.44, 0.91) YIELD ids AS p, score + VECTOR SEARCH IN product_vectors FOR 10 (0.15, 0.82, 0.44, 0.91) + YIELD ids AS p, score RETURN p.name, p.price, p.category, score // 4. Inspect existing indexes @@ -215,14 +237,20 @@ This removes the index and frees the associated resources. client = TuringDB(host="localhost", port=6666) # 1. Create a vector index for product embeddings - client.query("CREATE VECTOR INDEX product_vectors WITH DIMENSION 384 METRIC COSINE") + client.query( + "CREATE VECTOR INDEX product_vectors " + "WITH DIMENSION 384 METRIC COSINE" + ) - # 2. Load embeddings generated by your model (keyed by native node ID) - client.query('LOAD VECTOR FROM "product_embeddings.csv" IN product_vectors') + # 2. Load your model's embeddings (keyed by native node ID) + client.query( + 'LOAD VECTOR FROM "product_embeddings.csv" IN product_vectors' + ) # 3. Find the 10 most similar products to a query embedding df = client.query(""" - VECTOR SEARCH IN product_vectors FOR 10 (0.15, 0.82, 0.44, 0.91) YIELD ids AS p, score + VECTOR SEARCH IN product_vectors FOR 10 (0.15, 0.82, 0.44, 0.91) + YIELD ids AS p, score RETURN p.name, p.price, p.category, score """) print(df) @@ -242,11 +270,15 @@ The vector index above is a standalone, root-level structure keyed by numeric ID ```jsx // Set an embedding property (inside a change) -MATCH (n:Document {name: 'intro'}) SET n.emb = (0.12, 0.45, 0.78, 0.33) +MATCH (n:Document {name: 'intro'}) +SET n.emb = (0.12, 0.45, 0.78, 0.33) // Compare embeddings inline -MATCH (n:Document) RETURN n.name, cosine_similarity(n.emb, (0.12, 0.45, 0.78, 0.33)) -MATCH (n:Document) RETURN n.name, euclidean_distance(n.emb, (0.12, 0.45, 0.78, 0.33)) +MATCH (n:Document) +RETURN n.name, cosine_similarity(n.emb, (0.12, 0.45, 0.78, 0.33)) + +MATCH (n:Document) +RETURN n.name, euclidean_distance(n.emb, (0.12, 0.45, 0.78, 0.33)) ``` `cosine_similarity` is true cosine similarity and `euclidean_distance` is true (not squared) L2 distance. Both accept Embeddings only, not lists, and work in `RETURN`, `WHERE` and `ORDER BY`. This is a brute-force scan, so for large collections use a vector index. @@ -271,20 +303,27 @@ The Parquet file (relative to the TuringDB `data` directory) must have exactly t import numpy as np, pyarrow as pa, pyarrow.parquet as pq from pathlib import Path -docs = client.query("MATCH (d:Document) RETURN id(d) AS node_id, d.text AS text") -vecs = model.encode(docs["text"].tolist()).astype("