feat(state): MongoDB state backend, CLI wiring, and backend tests#11
Merged
Conversation
Add a MongoDB storage backend for session state using three collections (sessions, items, batches) that mirror the JsonFileBackend directory layout. Introduce a URI-based build_backend() factory so future backends can be added with zero config changes — the URI scheme selects the implementation. - Add q2google/state/mongo.py with MongoBackend (pymongo optional extra) - Add build_backend(cfg) factory to q2google/state/__init__.py with lazy MongoBackend import and clear error when pymongo is not installed - Add Q2GOOGLE_STATE_URI setting to Q2GoogleSettings; state_dir unchanged - Refactor _runner.py to accept SyncStateBackend instead of state_dir Path - Update _app.py to use build_backend(); --state-dir flag still overrides - Add parametrized backend fixture in tests/conftest.py (json + mongo via mongomock); extract protocol-compliance tests to test_state_backends.py - Add docs/backends.md guide; update configuration.md, getting-started.md, ARCHITECTURE.md, api/state.md, and mkdocs.yml nav Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Use sync_settings for --state-dir overrides, honor Q2GOOGLE_LOG_LEVEL, and pass a single state backend through the runner. Add CI test job and docker-compose for local MongoDB. Co-authored-by: Cursor <cursoragent@cursor.com>
himewel
changed the title
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
MongoBackendinq2google/state/mongo.pywith three collections (sessions,items,batches) mirroring theJsonFileBackendlayout for per-document visibility and resume checkpoints.build_backend(cfg)inq2google/state/__init__.py—Q2GOOGLE_STATE_URIscheme selects the backend; filesystem fallback usesQ2GOOGLE_STATE_DIRwhen the URI is unset.syncCLI throughbuild_backendandsync_settings()so env/.envsettings (includingQ2GOOGLE_STATE_URIandQ2GOOGLE_LOG_LEVEL) apply consistently; pass one backend instance through the runner for pre/post-run inspection.pymongostays optional (pip install q2google[mongo]); lazy import raises a clear error only when amongodb://URI is requested without the extra installed.Rebased on latest
main(unified GoPro settings viaQ2GoogleSettings,gopro-api0.0.9).Changes
q2google/state/mongo.py—MongoBackendwithload/save, lazy indexes,session_idon item/batch documentsq2google/state/__init__.py—build_backend(cfg)factoryq2google/config.py—state_uri: str | None(Q2GOOGLE_STATE_URI)q2google/cli/_settings.py— merge CLI--state-dirinto settings beforebuild_backendq2google/cli/_app.py,q2google/cli/_runner.py— usebuild_backend(sync_cfg); runner acceptsstate_backendinstead of constructingJsonFileBackendlocallyq2google/cli/_logging.py— honorQ2GOOGLE_LOG_LEVELwhen--log-level/-vare omittedpyproject.toml—[mongo]optional extra;mongomock/pymongoin dev grouptests/conftest.py,tests/test_state_backends.py— parametrized json + mongo (mongomock) protocol tests (12 cases)tests/test_sync.py— removed; filesystem-specific coverage lives in backend parametrized tests.github/workflows/ci.yml— dedicatedtestjob runninguv run task testdocker-compose.yaml— local MongoDB 7 for manual testingdocs/backends.mdand related docs — backend guide, configuration, and architecture updatesTest plan
pip install q2google— no pymongo; filesystem backend unchanged whenQ2GOOGLE_STATE_URIis unsetpip install q2google[mongo]+Q2GOOGLE_STATE_URI=mongodb://localhost:27017/q2google— MongoBackend used by CLI and librarydocker compose up -dthen sync with Mongo URI — state persists across restartsuv run pytest tests/ -vv— 12 parametrized backend tests passValueErrorwith supported schemes listedmongodb://without pymongo →ImportErrorpointing topip install q2google[mongo]