Skip to content

architecture.md

Latest commit

 

History

History
52 lines (30 loc) · 4.64 KB

architecture.md

File metadata and controls

52 lines (30 loc) · 4.64 KB
Raw

Architecture — the C-ABI boundary

Deep reference for maintainers working on the forked wrapper. AGENTS.md carries the day-to-day onboarding and a one-paragraph summary; this file is the "why the internals look the way they do" companion.

Why we fork the wrapper

modernc.org/sqlite is two layers:

Layer Source Our relationship
Transpiled C (inside modernc.org/sqlite: lib/, vec/, vfs/c/) generated by ccgo, per-platform external dependency — never edit (distinct from this repo's own vec/ package)
Hand-written Go wrapper (sqlite.go, conn.go, driver.go, stmt.go, rows.go, tx.go, backup.go, blob.go, error.go, result.go, convert.go, vtab.go, pre_update_hook.go, fcntl.go, mutex.go) ~6500 lines of glue forked into this repo's root

We fork because *Conn / *Driver hold unexported db uintptr + tls *libc.TLS fields, so per-conn methods (RegisterFunc, RegisterAuthorizer, …) cannot be added from outside the package. Mattn-compat also needs &SQLiteDriver{Extensions, ConnectHook} struct-literal construction, which means we must own the Driver type. Forking is the only path.

Consequence: edits to root-level *.go often touch modernc-derived code. The patterns there (uintptr arithmetic, unsafe.Pointer casts, "empty critical sections" around mutex handshakes) look wrong but are correct — that is how the transpiled C is called. Don't rewrite them for style.

The function-pointer dance (internal/cabi)

The forked wrapper converts Go function values to uintptr slots the transpiled C can store and call back, and the inverse. go vet flags these as unsafeptr; we silence it with -unsafeptr=false (justfile / CI / .golangci.yml). Do not "fix" these casts — the pattern is the contract for talking to modernc.org/sqlite/lib.

Both sides live in internal/cabi:

  • FuncPointer[T] (producer: Go func → uintptr) and AsFunc[F] (consumer: stored uintptr → callable Go func), with the typed CallX* family in callx.go for io-methods slot dispatch.
  • Registry[T] — token→*T map with an atomic counter (FS-instance maps in every vfs sub-package; minted keys live in FpAppData or a per-file tail allocation).
  • PtrMap[T] — pointer→*T keyed by caller-allocated addresses (the fileMap shape in vfs/crypto + vfs/cksm).
  • UniqueName(prefix) — process-global unique VFS / module name suffixes.

The unstated assumption is the Go runtime's function-value memory layout (https://golang.org/s/go11func) — stable in practice, unstable in theory. internal/cabi is the single point of repair if it ever moves.

Struct-drift discipline (named-field literals, never memcpy)

vfs/crypto/, vfs/cksm/, and the root wrapper reach into lib/'s exported Tsqlite3_vfs / Tsqlite3_io_methods / sqlite3_blob_* structs via named-field struct literals, never memcpy. Be precise about what that buys:

  • A field the literal references that upstream renames or removes fails to compile — the real guard.
  • A field upstream adds or reorders does not fail — named fields bind by name, and an unlisted new field is left zero.

Safety against additions comes from hard-coding FiVersion (and the io-methods iVersion) below any field we don't forward, so SQLite never reads the zero field. blob.go follows the same convention for the sqlite3_blob_* family.

On every just bump-modernc, re-check the field lists by hand. The places that may need fixing: conn.go, vtab.go, hooks.go, pre_update_hook.go, blob.go, vfs/crypto/crypto.go, vfs/cksm/cksm.go.

cksm / crypto chaining

vfs/crypto/crypto.go::Options.WrapVFS + a per-package fileMap (a cabi.PtrMap[FS] in each vfs.go) maps pFile → owning *FS; per-FS ourIoMethods + wrappedSzOsFile. Both layers store state at their own offset and forward via the captured wrapped methods. Each layer owns its own PtrMap so chained inner/outer instances don't collide.

Compile-time drift guards

A handful of compile-time assertions exist so an upstream dep growing a new interface method, or changing a signature, fails the test build with a useful error rather than breaking silently at runtime:

  • gorm/interfaces_test.go asserts *Dialector / *Migrator satisfy gorm.Dialector, gorm.Migrator, gorm.ErrorTranslator, and gorm.SavePointerDialectorInterface. If gorm adds a method, go test ./gorm/... fails with "missing method X" — implement, stub, or drop the claim consciously.

When adding new contracts of this kind, prefer compile-time assertions over runtime panics: the point is that the compiler refuses to build the test binary.