squonk

A fast, compact, and standards-minded library for parsing, rendering, transforming, and analysing SQL.

Why another SQL parser?

The sqlparser crate, maintained in the datafusion-sqlparser-rs repository, is a mature and widely used SQL parser for Rust. squonk addresses the same problem with a different architecture: declarative dialects, a compact owned AST, and conformance testing against real database engines.

Dialect behaviour is defined through a trait with 184 overridable methods. As a result, each dialect becomes a patchwork of code overrides that is difficult to inspect, compare, and extend. Its allocation-heavy AST also keeps parsing among DataFusion’s identified performance opportunities.

squonk approaches the problem differently:

• Dialects are data, not subclasses.
• The AST is owned, compact, and designed for long-lived tooling.
• Engine-backed dialects are tested against the engines themselves, not just against our expectations

What makes it useful?

• Dialects are data. An attempt to create a common feature based spec has been made, users and contributors can inspect a dialect, compare two presets field by field, or define a custom dialect as a small delta instead of forking parser code.
• The AST is built for tooling. Every node is owned, 'static, and includes a byte span and stable node ID. Trees can be stored, sent across threads, and mapped back to the exact source bytes.
• Errors are structured, and recovery is built in. A recovering parse preserves valid statements while reporting invalid ones with locations and diagnostics. One malformed query does not get to ruin everyone else’s day.
• Rendering understands context. Produce canonical, fully parenthesized, or PII-redacted SQL; render for a target dialect; or transpile between dialects in one call.
• The default dependency tree is tiny. The Rust crate depends on one micro-dependency, thin-vec. Serde support is optional, keeping compile times and supply-chain surface pleasantly boring.
• Rust, Python, and TypeScript share one core. Every language surface parses SQL using the same implementation, avoiding subtle differences between bindings.

The full subsystem and statement inventory lives in docs/architecture.md. Design decisions are recorded in docs/adr/.

Quick start

Rust

Add squonk with Cargo:

cargo add squonk

use squonk::parse;

let parsed = parse("select 1 +  2").expect("well-formed SQL parses");

assert_eq!(parsed.to_string(), "SELECT 1 + 2");

The default build includes only the ANSI dialect. Other native dialects are enabled through Cargo features; see Feature flags.

Python

pip install squonk

import squonk as sql

doc = sql.parse(
    "select salary from employees",
    dialect="ansi",
)

assert doc.to_sql() == "SELECT salary FROM employees"

idents = [ident.text for ident in doc.find_all(sql.Ident)]

The Python API also supports recovering parses, structured diagnostics, tokenization with optional trivia, rendering, redaction, and transpilation.

See crates/squonk-python.

TypeScript

npm install @squonk-sql/postgres

import { parse } from "@squonk-sql/postgres";

const doc = parse("select id, name from users where id = $1");

console.log(doc.toSQL());

Node and Bun expose a synchronous API backed by a prebuilt Node-API engine, with automatic WASM fallback when addons are unavailable. Deno and Workers receive permissionless runtime-specific WASM entrypoints. Browsers use the explicit async factory from @squonk-sql/postgres/browser. Focused @squonk-sql/* packages retain dialect-gated APIs and dialect-sized WASM artifacts; the unscoped squonk package includes every built-in dialect.

The parser is pure Rust, so building it does not require a C toolchain.

See crates/squonk-wasm.

Correctness

Parsers should not grade their own homework.

Engine-backed dialects are tested against the engines they claim to support:

• PostgreSQL through libpg_query, running in process
• SQLite and DuckDB through their embedded engines
• MySQL through a running server

The test corpus includes regression suites vendored from those projects. CI pins exact conformance counts and follows one hard rule: squonk must not accept SQL that the reference engine rejects. Pins move only after fresh measurements.

Additional coverage includes:

• property-based tests
• cross-dialect fuzzing
• round-trip checks requiring parse, render, and reparse operations to produce the same structure

Dialects without a full engine oracle—such as Snowflake, Databricks, and Redshift—use deliberately conservative presets. QuiltDB is held to its frozen parser-verdict corpus and curated reject fixture. Every enabled feature must cite an engine, standard, or first-party contract rather than rely on optimistic guesswork.

Partial or comparison-based oracles are labelled clearly. ClickHouse is checked with clickhouse local, while BigQuery is cross-checked against sqlglot.

Each dialect’s support tier and source of truth are listed in docs/support-tiers.md.

Performance

The publication benchmark measures one operation: parsing the same frozen SQL into a complete AST through each tool's public API. Results are compared within an ecosystem so language-runtime costs are not mistaken for parser-core differences.

On the controlled Linux snapshot, Squonk delivered 2.33×, 43.43×, and 13.63× the full-AST throughput of its direct Rust, Python, and Node peers, respectively. The binding measurements exercise the public document APIs, including their native object construction; they are product measurements rather than Rust-core-only results. The chart uses actual MiB/s on its shared axis; multiplier annotations are comparisons only within each ecosystem pair.

Why this metric is useful, which comparisons are valid, cold-start and retained-memory results, uncertainty, limitations, and the raw observations are documented in docs/performance.md.

Feature flags

The default build includes only ANSI and depends on thin-vec. Everything else is opt-in:

• postgres, mysql, sqlite, duckdb, quiltdb — stable, conformance-gated dialects
• bigquery, clickhouse, databricks, hive, mssql, redshift, snowflake — documentation-backed or partially verified presets; see support tiers
• lenient — a permissive union dialect for parse-almost-anything workloads
• full — all dialect features listed above
• serde — AST serialization and deserialization
• serde-serialize — serialization only
• serde-deserialize — deserialization only

Platform support

The cargo add, pip install, and npm install examples are intentionally operating-system neutral.

Actual support levels—fully tested, compile-verified, or best effort—are defined in docs/platform-support.md. Every promised tier has a corresponding CI check.

The squonk may be elusive. Platform guarantees should not be.

Safety and MSRV

The AST crate, squonk-ast, declares:

#![forbid(unsafe_code)]

This is a compiler-enforced rule that cannot be overridden within the crate.

The parser crate and the rest of the workspace use:

#![deny(unsafe_code)]

The workspace contains no unsafe code today. Using deny outside the AST crate leaves room for a narrow, reviewed #[allow(unsafe_code)] block if a future performance-critical path genuinely requires one.

The default build’s sole dependency, thin-vec, is the only vetted crate containing internal unsafe.

The minimum supported Rust version is 1.85. It is declared through rust-version in the workspace manifest and verified in CI, preventing accidental version drift.

Contributing

We welcome focused issues and pull requests. Please read CONTRIBUTING.md before starting a substantial change, or contact us at opensource@moderately.ai.

Send security-related reports directly to security@moderately.ai.

All project interaction is governed by our Code of Conduct. Conduct concerns may be reported to opensource@moderately.ai.

Before requesting a new dialect or syntax feature, read the dialect and syntax request policy. It explains the initial support tier for new dialects and the engine or documentation evidence required for a request.

For everything else:

• SUPPORT.md explains where questions, bugs, and reports belong.
• SECURITY.md explains how to report vulnerabilities.

Sponsors

squonk is maintained and sponsored by Moderately AI Inc.

To discuss sponsorship, contact opensource@moderately.ai.

Acknowledgements

No parser appears from nowhere. This project builds on prior work and draws much of its test surface from the engines themselves:

• datafusion-sqlparser-rs — the primary prior art, a source of lessons, and the benchmark reference
• libpg_query, through pg_query.rs — PostgreSQL’s extracted parser, our PostgreSQL differential oracle, and our first-party performance comparison
• PostgreSQL, MySQL, SQLite, and DuckDB — their engines act as conformance oracles, and their regression suites provide the corpora we vendor and test against

License

Licensed under the MIT License.