Repository navigation
v2: uv workspace and async ODM core (mongospecs 0.2.0) - #6
Merged
Merged
Conversation
Replaces the 0.1.x layout with a uv workspace of two distributions (mongospecs 0.2.0, mongospecs-web 0.1.0) plus an unpublished bench package. 0.1.x continues on the 0.1.x branch. - remove the synchronous v1 sources, tests, poetry files and stale reference docs - add Docker compose files for a single-node replica set and a small sharded cluster; tests run against real MongoDB, not mongomock - add CI (Python 3.11-3.15, MongoDB 7.0/8.0, free-threaded as experimental), a tag-driven publish workflow and a uv Makefile - add the design and work graph (claudedocs/v2-plan.md) and the v1 test inventory mapping all 302 v1 test runs to their v2 fate Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rewrite of the ODM on pymongo's AsyncMongoClient. Async-only, Python 3.11+, not backwards compatible with 0.1.x. Models and codecs - one Codec seam with msgspec, pydantic and attrs implementations; SpecAdapter for existing classes - field markers in typing.Annotated (Policy, Deferred, Immutable, Alias, CreatedAt, UpdatedAt, DeletedAt, Revision, Backref, OnDelete) - Ref[T] references, OID, discriminators, lazy upcasting Reads and writes - immutable chainable queries, Q expressions with name checks and type casting, untrusted raw filters unless wrapped in trusted() - tracked / lean / raw read modes; snapshot-based change tracking so save() sends only the delta - atomic update operators, upsert, typed bulk_write, bulk_save, partial-failure insert_many, cursor and offset pagination Relations - include() by batched queries or $lookup with identical results, nested paths, reverse relations, polymorphic refs, cascade rules, $graphLookup and $unionWith Operations - explicit Registry lifecycle, callback transactions that restore in-memory document state on abort and retry - plugins (timestamps, soft delete, revisions), registry-wide plugins, document hooks, named scopes - index planner with gated destructive actions, $jsonSchema validators, change streams, multi-database managers, shard-key aware writes - command events with model and operation names; logging, OpenTelemetry and Sentry listeners - one error hierarchy that wraps pymongo errors 415 tests against a replica set and a sharded cluster, passing on Python 3.11, 3.13 and 3.15; ruff and mypy --strict clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Resource exposes a model as REST operations described as pure data; plugs translate requests and responses for a framework and share everything else. - Resource, Operation, Ctx, Request, Response with no framework imports; custom operations, before/after hooks - field Policy drives Create, Patch, Read and Filter views; input is filtered by policy and validated against the real model, and JSON Schema per view is a transform of the model's own schema - allowlisted query-string filters with a bounded operator set, sort, cursor and offset pagination, includes - operation guards plus a row scope that is ANDed into every query - ETag / If-Match from the revision field, RFC 9457 problem responses, OpenAPI 3.1 generation - plugs: pure ASGI, Starlette, FastAPI (schemas merged into the host app's OpenAPI for any model backend), Litestar, Quart - mongospecs_web.testing: an in-memory backend and a contract suite whose transcript every plug must reproduce Core changes this needed: Policy(private=True) is no longer writable unless insert/update are set explicitly; lax casting for values that arrive as text; msgspec missing-field errors name the field. 49 web tests; every plug x model backend matches the reference transcript. 464 tests in the workspace. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unpublished workspace package answering performance questions with data from real MongoDB (replica set and sharded cluster). - mongospecs-bench run: workloads B1-B8 (materialise, find, include strategies, writes, pagination, plugin overhead, HTTP per plug, cost of change tracking) - mongospecs-bench stress: scenarios X1-X7 (pool pressure, transaction contention, optimistic-lock storm, cancellation, 16MB results, change stream across a step-down, threads with own loops) - mongospecs-bench trace: Sentry SDK through mongospecs-web into a local rustrak server, asserting one trace per request with Model.op database spans - results are JSON under bench/results/<git sha>/ with server, topology and machine metadata; --quick only proves the harness runs - smoke tests keep every workload and scenario runnable Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Changes driven by the first full benchmark run and by booting a real application. Performance - tracked reads keep the raw BSON the driver received as the snapshot and build the comparable form only when a document is saved, instead of re-encoding every loaded document - cursors are drained with to_list() batches, not one await per document - remove the 'trusted' read flag: the pydantic fast path it enabled was slower than validating, and it did nothing on msgspec and attrs Fixes found by the end-to-end tests - aggregate, union, graph and backref-count paths sent max_time_ms in find() spelling and failed whenever a default time limit was set - web: unexpected errors are now logged server-side (the client still gets a bare 500) - ids given as strings to reference fields are coerced to the target's id type on backends that do not validate on construction - stored documents are read leniently on strict backends (an integer stored as a double), input stays strict - a document stored before a Revision field existed can be saved Reference application and tests - bench: 'storefront', a production-shaped app (auth, admin guards, row scope, transactional orders that cannot oversell, soft delete, ETags, reviews as a reverse relation, a resumable exactly-once change-stream projector, health check, graceful shutdown) - end-to-end tests boot it under uvicorn and drive it over HTTP on every plug and model backend: customer journey, concurrent buyers, all-or-nothing multi-item orders, projector resume across restarts, in-flight requests on shutdown, consistency across a primary step-down - HTTP load benchmark against the running server with a hand-written pymongo + Starlette baseline of the same endpoints - Resource(server_fields=...) for values the server sets on create - guide pages whose examples are executed by the test suite, migration guide, Mongoose parity page, Mongoose benchmark comparator Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nerator - Codec.to_json: msgspec.to_builtins / pydantic model_dump(mode='json') produce the Read view directly instead of encoding to BSON types and converting those; responses are dumped with msgspec's JSON encoder when available - list and get requests read lean documents: nothing is saved, so nothing is tracked - upserts give the inserted document the model's defaults, so it has the same shape as one created by insert() - load benchmark: the httpx-based generator capped out below the server's capacity, so its throughput numbers measured the client. Replaced with a minimal keep-alive client; the primary metric is now server CPU time per request, with latency taken on an unloaded server and throughput at saturation - CI job for the Sentry -> rustrak tracing check; plan amendments A14-A21 and nodes E4-E6 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- results for all workloads, the HTTP load test, the stress scenarios and the Mongoose comparator at 6dc837f (earlier B1-B6 results kept as the before picture for the tracking optimisation) - docs/benchmarks.md: what was measured, how to read it, and what follows from it - decisions recorded in the plan: include() defaults to batched queries; no Rust in 0.2, with a BSON-to-model decoder for bulk reads as the one candidate; tracked stays the default read mode - drop workload B7 in favour of LOAD: measuring plugs through an in-process client measured the client - ignore stale class copies in subclass discovery (found by the free-threaded CI job) with a regression test - CI builds the docs in strict mode Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fixture file written from a user's point of view; mypy --strict must reject exactly the lines marked expect-error (wrong model type, a possibly-None result used as a document, a count used as a document, a reference's target type) and accept the rest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A timestamp with a zero fractional second is printed without one by some serialisers, which made a lexical comparison fail intermittently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A timestamp with a zero fractional second is printed without one by some serialisers, which made a lexical comparison fail intermittently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 279bd6c)
Each has a regression test that failed before the fix. - cursor pagination with select() repeated documents and did not terminate: sort keys were left out of the projection, so the next cursor was built from defaults. Sort keys are now always fetched - bulk_save left memory out of sync after a partial failure. Writes in the batch are independent; the ones that succeeded are now brought in sync and the error lists the rest in .failed. Which updates applied is determined by reading the documents back, since the server only reports a total - bulk_write hard deletes skipped OnDelete cascade rules - a cycle of cascading references recursed forever; cascades now carry the set of documents already being deleted - cascades resolved ids with distinct() (16MB limit) and one unbounded $in; ids are now read with find and applied in chunks - saving an untracked instance that has an id (a copy, a deepcopy, a lean read) raised a bare duplicate-key error; it now says the instance is not tracked and what to do - reload() loaded Deferred fields - Query.update() returns an int that also carries .matched - the Location header of a create was a bare id; it is now a URL under the API prefix Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review of the web layer. - request bodies are read under the size limit on every plug (declared length checked first, then chunk by chunk); previously only the pure ASGI plug stopped at the limit and the others buffered the whole body - DELETE applies the row scope in the delete's own filter, not only in the lookup before it, and honours If-Match - deleting a loaded document checks its revision on hard deletes too - offset pagination refuses pages deeper than Pagination.max_offset (10,000 skipped documents by default) - totals are opt-in per resource: Pagination.total now defaults to False - reads made for a request carry a 10s time limit when neither the registry nor the model sets one - Api(openapi=False) keeps the API description private on every plug - HEAD and OPTIONS are answered by Api.handle and the pure ASGI plug - Resource(explicit_writes=True): a field is writable only when its Policy says so - mongospecs_web.testing.mount_on(plug, api) builds an app on any built-in plug BREAKING: ?total=true needs page=Cursor(total=True) / Offset(total=True). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- parity page: 28 of 35 Mongoose behaviours now confirmed against its documentation; three more deliberate differences recorded - plan amendments A25-A29: the computed decorator, Atlas Search and vector indexes declared out of scope for 0.2 (not testable here), revision-checked hard deletes, the changed web defaults - guides: limits and defaults of the web layer, untracked documents, partial bulk_save failures, registries and explicit managers - CI: MongoDB 7.0 on Python 3.11, 3.13 and 3.15 - publish workflow runs lint, types and the test suite first Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each has a regression test that failed before the change
(tests/test_review_round2.py in both packages).
core
- a field whose default is None is not stored while it is None, and
setting it to None unsets it; a unique sparse index no longer
collides on stored nulls
- field names that shadow the Document API raise ConfigurationError
- registry.transaction() inside a running transaction raises unless
independent=True (it would commit on its own)
- UpdateCount formats as its int value in str() and f-strings
- filter values are cast by one rule on every backend
- empty Or()/Nor(), non-text prefix()/contains(), sort keys that are
not field paths and server type mismatches raise QueryError
- include() checks its path and strategy when the query is built
- find().on_invalid('raise' | 'skip' | 'raw')
web
- integers beyond 64 bits are a 422 on create and patch, and an
EncodingError is no longer a 500
- Quart: init_app installs BodyLimit, which reads API bodies under the
limit before Quart queues them
- a guard runs once per request however many places list it
- OpenAPI publishes only referenced definitions and no private field
- HEAD on the ASGI plug reports the length of the GET body
BREAKING: optional fields that are None are omitted from stored
documents, so an exists filter now means 'has a value'.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Private fields of embedded (non-document) models were serialised, writable and described in OpenAPI. They are now handled like private fields of the document itself, at any depth and inside lists, dicts, optionals and unions: never in a response (including included documents), a 422 'unknown field' when a client supplies one on create or patch, and absent from every published schema. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Leaving None-valued optional fields out of stored documents stays the default. StoreNone on a field, or none='store' on a model, stores an explicit null instead (OmitNone opts a field back out). Encoding, change tracking, update(), upsert defaults and the bulk paths follow the field's mode; loading never produces a spurious change in either mode. A unique sparse index over a field that stores null is refused with a pointer to a partial index. Also applies the omit rule to embedded pydantic models, which were encoded with their None values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- web GET list/get read stored documents with a projection of the readable fields and serialise them directly when that is certain to equal the model path; anything else falls back to the model - DELETE is one conditional write unless the model needs the document (delete hooks, cascades) - independent includes are loaded concurrently (not inside a session) - per_parent_limit on a reverse relation is applied on the server - tracked documents use one weak reference instead of weakref.finalize - Query._clone copies slots instead of re-running __init__ - routes are looked up by resource before patterns are tried fix(core): cursor pagination no longer skips or stops at documents without a value for a sort key Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Requests beyond the limit are answered at once with 503 and Retry-After instead of queueing for database connections. test: correctness of the optimised paths (tracking, clone, includes, per-parent limits, routing) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…om the stored document Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- docs/guide/production.md: pool and timeout settings, backpressure, read and pagination choices, relations, writes, what needs a replica set; each item marked measured, tested or guidance - docs/benchmarks.md: hot paths before and after the review - bench/results: B9 before (49a32a9) and after (fb543de) - plan: decision per review finding, amendment A36 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An ObjectId, Binary, Timestamp, Regex or Int64 inside a field typed Any (or dict[str, Any], list[Any]) made the JSON conversion fail, and the web layer answered 500: on pydantic for any ObjectId, on msgspec and attrs for the BSON types their serialiser does not know. All three backends now give the same JSON for such values, at any depth. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A PATCH that gave an object for an embedded model replaced the stored object, writing defaults over everything the client did not send, including private, read-only and immutable values it is not allowed to set. Objects are now merged key by key (RFC 7396) in one atomic update; lists and dicts are still replaced whole. - nested keys are held to the embedded model's own Policy - an embedded object holding protected values cannot be set to null, and a list or dict of such objects cannot be replaced through PATCH - Policy(insert=False) is enforced on nested keys at create - the OpenAPI Patch schema describes embedded objects as partial BREAKING: clients that relied on PATCH replacing an embedded object must now send null for the keys they want cleared. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PATCH merging into embedded objects (A37) had no way out for resources
that need the embedded model validated whole, need to write over a
stored null, or use a backend without supports_merge.
- Resource(merge_patch=False): PATCH replaces an embedded object with
the one sent. Values the client cannot send go back to their
defaults; nested keys the policy closes are still rejected.
- Put: PUT /{id} replaces every field the client may update (the sent
value, or the default when left out) and leaves the rest as stored,
in one atomic update with the If-Match check. An embedded object
holding protected values is replaced key by key; one without is
replaced whole. It never creates, and is not in the default ops.
- OpenAPI gains <Model>Put schemas where a resource has the operation.
Recorded as A39 in the plan.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each was reproduced against the replica set before it was fixed; the tests are in test_review_round3.py of both packages. Core - A single-field update on a pydantic model failed a model-level validator that only saw the other fields' defaults. The field's own validators still run; the cross-field verdict is no longer taken. - insert_many and bulk_save run document hooks, as insert and save do. - page() starts its count after its arguments are validated, so a bad cursor no longer leaves an un-awaited coroutine. Web - Request bodies are strict JSON: NaN, Infinity and over-deep nesting are a 400 (NaN was stored; deep nesting was a logged 500). - If-Match on a model without a Revision is a 412, not an unconditional write. If-None-Match accepts lists, weak tags and *. - The ASGI plug drops a request whose client disconnected mid-body. - Api(openapi=False) is honoured by the FastAPI plug. - Guards and hooks must name an operation the resource has. - The concurrency limit is applied before authentication. - A filter or sort key given twice is a 400. Recorded as A40 in the plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Branch pushes and their pull requests both triggered CI, so every job ran twice. mongospecs[all] now includes the otel and sentry extras, and mongospecs-web declares msgspec as the optional encoder it already uses when present. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new user had reference-quality guides and no way into them. - examples/: a quickstart script, a to-do REST API with its tests, and a compose file for a single-node replica set. They are run by the suite, and the tutorial page and the README show them as they are. - docs/tutorial.md: from MongoDB to a model, an API and its tests. - New guides: choosing a backend, errors (what each means and the HTTP status it becomes), authentication and authorization, testing an API. The last two are executed like the other guide pages. - The replica-set requirement is stated in the README, the landing page and the tutorial instead of only in the production notes. - The package README is checked to be the root README. - The reference section is removed: it was generated from docstrings, and the source is the better reference. mkdocstrings goes with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- CHANGELOG.md, CONTRIBUTING.md and SECURITY.md. - A workflow deploys the documentation to GitHub Pages from main. - The publish workflow refuses a release tag that does not match the version in the package it names. - Package metadata links to the documentation and the changelog. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
msgspec and attrs do not check types when a model is constructed or assigned to, and pydantic does not check assignments. A wrongly typed value was therefore stored as it was, and the document could not be loaded afterwards. insert, save, insert_many and bulk_save now decode the document's own encoding with the backend's strict rules before sending anything, and raise ValidationError if it does not fit. save checks only when there is a change, and only the changed fields of a document loaded with a projection. The check can be skipped with validate=False on a call or SpecConfig(validate_writes=False) on a model. It cost about 3 µs (msgspec) and 5 µs (pydantic) per seven-field document when measured. Recorded as A41 in the plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move the site config to zensical.toml on the new default theme, swap the docs dependency, and deploy through the GitHub Pages actions since zensical has no gh-deploy. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
v2: evidence track — benchmarks, stress, end-to-end storefront, docs
v2: mongospecs-web 0.1.0, REST resources with five plugs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First of a stack of PRs implementing
claudedocs/v2-plan.md. This one replaces the 0.1.x code with a uv workspace and lands the core ODM.mongospecs-web, the benchmark track and the docs follow as separate PRs stacked on this branch.0.1.x is preserved on the
0.1.xbranch. The in-progressmixins-testwork was committed and pushed there as-is; none of it is carried over.Commits
build!– workspace restart: two distributions (mongospecs0.2.0,mongospecs-web0.1.0) plus an unpublishedbench/, Docker compose for a replica set and a sharded cluster, CI, the plan, and an inventory mapping all 302 v1 test runs to their v2 fate (docs/v1-test-inventory.md).feat!– the ODM core (about 5k lines of source, 3k of tests).What the core does
Codecseam;SpecAdapterfor existing classes.Qexpressions with field-name checks and type casting, raw filters untrusted unless wrapped intrusted(),tracked/lean()/raw()modes, cursor and offset pagination.save()sends only the delta), atomic operators, single-round-trip upsert, typedbulk_write,bulk_save, partial-failureinsert_many.include()served by batched queries or$lookupwith identical results; nested paths, reverse relations, polymorphic refs, cascade rules,$graphLookup,$unionWith.Registrylifecycle;registry.transaction(fn)retries and restores in-memory document state on abort; timestamps, soft delete and revisions as marker-activated plugins; registry-wide plugins; discriminators; index planner with gated destructive actions; change streams; multi-database managers; shard-key aware writes.Verification
ruffandmypy --strictclean; both wheels build.Not yet run: the CI workflow itself (this PR is its first run), MongoDB 7.0, and the free-threaded build.
Things to know when reviewing
Ref[T]is a wrapper object,__spec__must be assigned without an annotation, and query values are cast by type only while updates are fully validated.include(strategy="auto")currently means batched queries. The default is to be chosen from benchmark data in the bench PR.lean()document cannot be saved: untracked documents are inserted, so it fails on the existing id.🤖 Generated with Claude Code