feat: Add async feature store interface and in-memory implementation#457
Open
jsonbailey wants to merge 1 commit into
Open
feat: Add async feature store interface and in-memory implementation#457jsonbailey wants to merge 1 commit into
jsonbailey wants to merge 1 commit into
Conversation
Adds the async interfaces (AsyncFeatureStore, AsyncReadOnlyStore, AsyncDataSourceUpdateSink, AsyncBigSegmentStore, AsyncInitializer, AsyncSynchronizer), the AsyncInMemoryFeatureStore, and async test utilities. Stacked on the async support classes slice.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The async data-store foundation the async SDK builds on.
interfaces.py— async interfaces:AsyncFeatureStore,AsyncReadOnlyStore(asyncget/all; the async analog ofReadOnlyStore),AsyncDataSourceUpdateSink,AsyncBigSegmentStore,AsyncInitializer,AsyncSynchronizer. Pure additions alongside the existing sync interfaces.async_feature_store.py—AsyncInMemoryFeatureStore. Holds no lock: every method reads/mutates its dict without awaiting in between, so it runs to completion without the event loop switching coroutines (the sync twin needs aReadWriteLockbecause threads can preempt each other).testing/— async test utilities:mock_async_components.py,async_feature_store_test_base.py, andtest_async_in_memory_feature_store.py.Why
Foundation for the async Python SDK (epic SDK-60). Subsequent slices (async evaluator, big segments, data sources, data systems, client) depend on these interfaces and the in-memory store.
Validation
uv run pytest ldclient/testing/test_async_in_memory_feature_store.py— 17 passeduv run mypy ldclient— cleanEpic SDK-60.
Note
Low Risk
Additive, experimental APIs and an in-memory reference implementation with broad test coverage; no changes to existing sync evaluation or storage paths.
Overview
Adds the experimental async data-store layer for the upcoming async Python SDK: new protocols/ABCs in
interfaces.pyand a default in-memory store, plus shared tests and mocks.interfaces.pygains async counterparts to the sync data path—AsyncReadOnlyStore,AsyncFeatureStore(versionedget/all/init/upsert/delete,initialized, optionalclose),AsyncDataSourceUpdateSink,AsyncBigSegmentStore, and FDv2-styleAsyncInitializer/AsyncSynchronizer(asyncfetchand async-generatorsync). Existing sync types are unchanged.async_feature_store.pyintroducesAsyncInMemoryFeatureStore, mirroring sync in-memory versioning and tombstone semantics but without aReadWriteLock; methods do notawaitbetween dict reads/writes, so work completes on one event-loop turn.testing/addsAsyncFeatureStoreTestBase(parity with the sync store test suite),MockAsyncFeatureStore/MockAsyncEventProcessor/MockAsyncUpdateProcessor, andtest_async_in_memory_feature_store.pywiring the in-memory implementation through that base.Reviewed by Cursor Bugbot for commit bbe77d7. Bugbot is set up for automated code reviews on this repo. Configure here.