SK-3118: FlowVault Python SDK (skyflow-flowvault-python) - #277
Open
saileshwar-skyflow wants to merge 54 commits into
Open
saileshwar-skyflow wants to merge 54 commits into
saileshwar-skyflow wants to merge 54 commits into
Conversation
…lowdb (v3) insert support Restructures the repo into three build variants sharing a bundled common/ module (SK-2938 Option C): v2 (today's SDK, behavior-preserving) and a new v3 built on the flowservice/flowdb API, insert-only this round. common/ - Shared credential resolution, vault-URL resolution, and bearer-token fetch/cache/expiry logic (VaultController, BaseVaultClient), enums, errors, service_account, and generic validators. - VaultController declares insert/get/update/delete/query/detokenize as abstract methods (Java-interface-style); v2 and v3 each provide their own concrete/stub implementations. v2 - Relocated from the repo root via git mv; public API unchanged (same class names, signatures, import paths). Vault is now a backward-compatible alias for the internal PdbVaultController class. - Fixed a latent bug where v2's own Env enum failed cross-class comparisons against common's Env; both now share one definition. v3 (skyflow-flowvault, starting at 1.0.0) - New InsertRequest/InsertRecord/Upsert/InsertResponse types, FlowVaultController, and VaultClient targeting the flowservice REST API. - Insert validation ported from Java's v3 Validations.java: table/upsert must live in exactly one place (request-level or per-record, matching in both), 10k record cap, empty key/value checks. - InsertResponse mirrors Java's v3 shape (summary/success/errors) as plain dicts, each result tagged with its index in the original record list (stable across batch boundaries). - Structured per-record error parsing from the backend's actual error body, plus x-request-id propagation onto error entries. - Batching via INSERT_BATCH_SIZE (default 50, max 1000), sequential, isolate-and-continue on a failing batch. - Vault URL resolution uses v3's own skyvault.skyflowapis.* domain for all four envs (DEV/SANDBOX/STAGE/PROD), confirmed to differ from v2's vault.skyflowapis.* domain. CI/CD - shared-tests.yml and shared-build-and-deploy.yml now take a `variant` input and scope every step to v2/ or v3/ via working-directory. - main.yml, ci.yml, beta-release.yml, internal-release.yml, and release.yml matrix over both variants. v3 releases are distinguished from v2's via a flowvault- tag/branch prefix (flowvault-1.0.0, flowvault-release/*) so a release trigger is never ambiguous between the two independently-versioned packages; v2's existing bare-semver tags are untouched. - Fixed ruff.toml/.codespellrc still excluding a pre-split "skyflow/generated" path that no longer existed after the relocation. - Fixed a bump_version.sh sed collision with a comment that happened to contain the literal text "__version__ = ...". - Added a common/ test job to main.yml/ci.yml. Note: v3/samples/ and the root samples/ folder are deliberately excluded from this branch/commit -- local working copies there contain credentials used for live testing against a real vault and must not be pushed. Tests: common 36, v3 65, v2 426 (2 pre-existing unrelated fixture failures), tests/contract passing for both variants. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Distribution name was already skyflow-flowvault (setup.py) but the importable package stayed skyflow, colliding with v2's skyflow import name if both are ever installed in the same environment. Rename the v3 directory to flowvault and its package to skyflow_flowvault to match the distribution name and remove the collision. generated/ content is left untouched (Fern-owned).
…d base insert response Consolidates duplicated logic between v2 (PDB) and flowvault per architecture review: shared validation (vault config, credentials, log level), LogLevel/Logger, and insert field/table validation now live in common with per-variant message injection; adds BaseInsertResponse alongside BaseInsertRequest so each variant's InsertRequest/InsertResponse can extend a common base while keeping its own shape. Also fixes flowvault's insert() response shape (drop redundant 'data'/'table', flatten tokens, errors=None when empty) and a stale SDK_VERSION drift bug. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…insert() Splits BaseSkyflow, BaseVaultController, and BaseVaultClient into a pure interface (ISkyflow/IVaultController/IVaultClient, declaring the contract via ABC + abstractmethod) plus a base class implementing the shared logic, so future variant-specific overrides have a clear contract to satisfy. Also adds type hints to insert() at every layer (BaseInsertRequest/BaseInsertResponse in the abstract method, each variant's own InsertRequest/InsertResponse in their concrete override), and renames base_vault.py to base_vault_controller.py to match its class name. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…shape, add types BaseInsertRequest's shared field is now named values (matching PDB's terminology) instead of records, and table/values are required rather than defaulted; each variant's InsertRequest forwards them explicitly. Removes flowvault's Upsert class in favor of a plain dict (now typed via a TypedDict) to match the rest of flowvault's dict-based request shape, and adds return type hints to every method on v2's VaultController to match insert's. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
BaseSkyflow implements every ISkyflow abstract method, so ABC alone doesn't block instantiating it directly -- only make_skyflow_class-produced subclasses should be constructed. Raises SkyflowError with a new SkyflowMessages entry instead of a raw NotImplementedError, matching how every other SDK error is raised. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI workflows (ci.yml, main.yml, release.yml, beta-release.yml, internal-release.yml) still referenced the pre-rename v3/ directory, so every flowvault CI job failed outright. Threads a package-name (skyflow vs skyflow_flowvault) through shared-tests.yml/shared-build-and-deploy.yml/ bump_version.sh, since those hardcoded the skyflow package name too -- a real release would've bumped the wrong version file. Also fixes common/setup.py's missing python-dotenv dependency (test-common CI job installs only this file's declared deps), and removes the empty-value insert tests now that empty/null field values are explicitly allowed rather than rejected. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… noise
flowvault/requirements.txt was missing coverage, so the v3 test job's
`python -m coverage run` step failed outright with "No module named
coverage" once the workflow correctly pointed at flowvault. Also excludes
**/generated/** (Fern-owned) from semgrep, since the generated REST clients
trip its secret-detection heuristics on parameter names like `token`, and
fixes a real semgrep finding: shared-build-and-deploy.yml interpolated
${{ }} context values directly into a run: shell block instead of routing
them through env: first.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…e positives test-common never ran coverage or uploaded to Codecov, so common/ (which grew substantially this session) was invisible to Codecov's patch/project checks. Adds a coverage run + Codecov upload step matching v2/flowvault's pattern. Also fixes .semgreprules/customRule.yml's check-sensitive-info regex: an optional quote-capture group let its own backreference match empty string, so any `keyword: value` matched regardless of quoting -- tightened to require an actual quoted literal and exclude self-referential values (e.g. TOKEN = 'token'), which eliminates 28 false positives without any inline suppressions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tion/detect structurally absent when unsupported Renames ISkyflow -> BaseSkyflow (pure interface) and the old BaseSkyflow -> BaseSkyflowImpl (concrete). Moves connection/detect support into ConnectionCapable/DetectCapable interfaces + ConnectionMixin/DetectMixin in a new common/client/utils/_utils.py, conditionally composed into a variant's Skyflow class by make_skyflow_class() so unsupported variants (e.g. flowvault) genuinely lack .connection()/.detect() (AttributeError) instead of raising NotImplementedError from a present-but-guarded method. Also fixes two review-flagged bugs: adding a vault/connection config with a duplicate id to an already-built client now raises SkyflowError instead of silently overwriting the existing entry, and update_connection_config no longer risks a bare KeyError on a missing connection_id. Extracts the Builder's raw NotImplementedError string literals into named constants. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… common Main advanced with the 2.1.3 release and SK-3039 (read the token roles/context from credentials, not the vault config, and validate them). Those changes landed on the old root-level `skyflow/` tree, which this branch has split into `common/` + `skyvault/`, so they are re-applied where that logic now lives: - common/utils/validations: add validate_token_options (roles/context checks, broadened context types, role-element validation) and call it from validate_credentials. - common/vault/base_vault_client.get_bearer_token: validate token options and build role_ids/ctx from credentials instead of config. - common/service_account._validate_and_resolve_ctx: take a messages param so delegated skyvault errors keep skyvault's SDK version in their text. - skyvault validations delegate validate_token_options to common; skyvault bumped to 2.1.3 (setup.py + _version.py). - SK-3039 client tests ported to common/tests/vault/test_base_vault_client.py (patch targets -> common.vault.base_vault_client) and the skyvault end-to-end tests repointed accordingly. - shared-build-and-deploy.yml: keep module `ref` input alongside main's SK-2986 admin-token comment. - flowvault bulk response: extract retryable-HTTP-code literals into constants (clears ruff PLR2004 that was failing CI). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The shared per-module test loop aborted on the first module (common) and `bash -e` then skipped skyvault and flowvault, so their tests never ran on a PR. Two fixes: - Add the repo root to PYTHONPATH for the test run. common/ is imported as a namespace package (common.vault has no __init__), so its own wheel can't be imported as `common`; resolving from the source tree fixes discovery for all modules. - Run every module even if one fails and fail the job only at the end, so one module never hides the others' results. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Fern-generated REST clients are not hand-written and were dragging module
coverage down (failing Codecov) and producing 82 of 83 Semgrep code-scanning
alerts.
- codecov.yml: ignore **/generated/**.
- Add .coveragerc omit for */generated/* to common, skyvault and flowvault so
generated code is not measured (flowvault 96.6%, common 86.6% after).
- semgrep.yml: pass --exclude generated so the SARIF upload no longer flags
generated code.
- shared-build-and-deploy.yml: use the built-in $GITHUB_ACTOR env var instead
of interpolating ${{ github.actor }} in a run step (the remaining Semgrep
shell-injection finding).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…v targets
- shared-build-and-deploy.yml: route inputs/steps-outputs/github.ref_name
through an env: block so no untrusted ${{ }} is interpolated in the run:
script (clears the last Semgrep run-shell-injection alert).
- codecov.yml: the base's auto-target is 99.75% (mature v2 code), which a large
PR adding new flowvault/common code can't hit; allow a 5% project threshold
and an 85% patch target so codecov reflects real, healthy coverage
(project 95.4%, patch 90.6%) instead of blocking on the inherited target.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port the Java SDK's japicmp contract gate to Python using griffe (static API analysis), for both published packages: - ci-scripts/contract/griffe_contract.py: builds each package's public-API surface from an explicit module allowlist (mirrors Java's <includes>; excludes generated/ and internal utils helpers) and dumps/checks it against a committed baseline. Removed/changed entries are breaking, added entries are new surface; any drift fails. - Committed baselines skyvault/api-report/skyflow.api.json (358 members) and flowvault/api-report/skyflow_flowvault.api.json (153). - ci-scripts/contract-snapshot-update.sh: regenerate baselines after an intentional public API change. - .github/workflows/contract-tests.yml: per-module matrix gate (fail-fast false), a skyvault-only guard `griffe check skyflow -a skyflow==2.1.3` (no breaking changes vs the released skyflow), and a PR comment showing the baseline diff when it changes. - Add griffe[pypi] to each module's dev extras. Verified skyvault's public surface has no breaking changes vs released skyflow 2.1.3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
griffe 2.2.0 requires Python >= 3.10, so `pip install griffe==2.2.0` failed on the 3.9 runner. griffe analyses the SDK source statically, so the analyzer's Python version is independent of the SDK's own >= 3.9 support and can be 3.10. Also mark the `griffe` dev extra `python_version >= "3.10"` so a 3.9 dev install does not fail. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Java's contract test compares only against the committed baseline, never a published release, so remove the skyvault-only `griffe check -a skyflow==2.1.3` step and the SKYVAULT_RELEASE bump it needed. The committed baseline is the contract for both modules. Current skyvault was verified to have no breaking changes vs released skyflow 2.1.3. Drop the now-unneeded griffe pypi extra. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address the code-review P0/P1 findings for flowvault: - Cover the remaining error-handling/edge branches to reach 100% on flowvault product code: batching .env fallback + zero/negative concurrency, response parsing None/[None] cases, get_vault_url validation + get_metrics fallbacks (new test__utils.py), full validate_bulk_insert/detokenize_request coverage, update ApiError body variants, async bulk error paths, and the bulk response __str__/empty-retry helpers. - Exclude setup.py and the `if __name__` boilerplate from coverage via .coveragerc. - requirements.txt: bump pydantic floor 1.9.2 -> 2.0.0 (setup.py and the Fern client require pydantic v2). - Remove dead GetRecordRequest import and MAX_BULK_DATA_SIZE constant; extract the repeated 'additional_headers'/'Unknown error' literals into named constants; drop explanatory comments from _validations.py. - get_vault_url: reject an invalid env with `not isinstance(env, Env)` instead of `env not in Env`, which raised TypeError (and left the guard unreachable) for a non-member on Python < 3.12. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Change the published distribution name from skyflow-flowvault to skyflow-flowvault-python in flowvault/setup.py, and update the pip-install and prose references in the root README, flowvault README, and samples README. The importable package stays skyflow_flowvault. No workflow changes are needed -- the release workflows resolve the module from the branch/tag directory name and twine publishes whatever setup.py builds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…table_name Match Java's flowdb contract: rename BulkInsertRecord -> BulkInsertRequestRecord and add a `tokens` field (BYOT) on it; use `table_name` (not `table`) on BulkInsertRequest, GetRequest, GetRecordRequest, and DeleteRequest. Update the affected vault_api samples to the new field names. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Custom request headers for bulk_insert/bulk_detokenize (+ async): a BulkInsertOptions/BulkDetokenizeOptions interceptor runs once per batch via a RequestContext, mirroring Java's RequestInterceptor/CustomHeaderKey. Full parity with Java's VaultConfig HTTP settings (timeout, connect/read/write timeouts, max_retries, initial/max retry delay, vault_url), at both per-vault (config dict keys) and client-wide (builder methods) levels, resolved per-field per-vault -> client-wide -> default. Like Java (which injects a configured OkHttpClient), flowvault injects a custom httpx transport (RetryTransport) plus an httpx.Timeout into the generated SkyflowAuth client -- no generated-code edits. The client-wide builder methods are flowvault-only via a builder mixin and a private base hook, so skyvault's public surface stays identical to 2.1.3. Also wires in the renamed BulkInsertRequestRecord/table_name across the controller, validations, and exports, and regenerates the flowvault public-API contract snapshot (154 -> 175 members). Full unit coverage; skyvault contract unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port the Java flowvault samples to Python: multi-table bulk insert (sync/async), custom-header and timeout/retry config examples, and the service-account token samples (bearer, context-aware, threaded, scoped, signed). Bulk tokenize and bulk delete-tokens samples are intentionally omitted -- those operations are not part of the Python flowvault surface. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…port-python-sdk SK-3118: FlowDB support for Python SDK (skyflow-flowvault) + module segregation
…copy The version-bump commit step ran find "$MODULE" -name "_version.py" after the build step created "$MODULE/build/", so it could return the copy under the gitignored build/ dir and "git add" would abort with "paths are ignored" (order-dependent, so it surfaced on flowvault). Exclude build/ and dist/ from the find so only the tracked source _version.py is staged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… doc) Remove the internal request/response shapes reference from the branch; it stays as a local-only file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… refinements Rename the importable package skyflow_flowvault -> skyflow so Flow DB code reads identically to the skyvault (Privacy DB) SDK (`from skyflow import Skyflow`); the PyPI distribution name stays skyflow-flowvault-python. skyvault's package and its frozen 2.1.3 contract are untouched. Contract tooling (griffe config, snapshot script, workflow) is updated to the skyflow package name. Error handling now matches Java: unary ops raise SkyflowError with the parsed server details (message, http_code, grpc_code, http_status, details, request_id) on any API error, instead of returning a fake record with the raw exception dump; bulk ops keep resilient per-record inline reporting but now surface the clean server message. Drop the redundant `data` field from bulk insert response records. Add a typed TokenGroupRedactions(token_group_name, redaction) for detokenize requests and a typed Callable[[RequestContext], None] interceptor on BulkInsert/BulkDetokenizeOptions -- both matching Java. Regenerate the public-API contract snapshot (-> skyflow.api.json, 179 members) and update the vault_api / service-account samples. Full unit coverage; skyvault contract unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- update(): require non-empty dict 'data' per record and build wire records inside the try/except, so a missing/invalid 'data' raises SkyflowError instead of a raw pydantic.ValidationError escaping the SDK error contract. - update(): plumb BYOT 'tokens' through to the update wire record (previously accepted by validation then silently dropped); matches the API/Java contract. - VaultClient: close the previous sync/async httpx clients on reinitialization and add close()/aclose(), fixing the connection-pool leak on update_config(). - Add tests covering missing/invalid update data, BYOT tokens on update, and httpx client close-on-reinit / explicit close. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Upgrades the generated REST layer to the newer Fern Python generator, which memoizes resolved type hints and short-circuits convert_and_respect_annotation_metadata in core/serialization.py. This removes the per-request typing.get_type_hints recomputation that dominated serialize/parse CPU (~3x lower per-request CPU cost, 122 -> 381 req/s single-thread in a mock benchmark). Output is unchanged and the public skyflow API is unaffected; hand-written SDK code is untouched. Adds the generated http_sse core module. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… defaults Parity with the Java flowvault SDK: - update(): send updateType to the API (was silently dropped). Wired the regenerated update endpoint's updateType through the controller, converting the public UpsertType enum to its wire value and omitting it when unset. - Add typed DetokenizeResponseRecordMetadata for the metadata field on both unary and bulk detokenize responses (was an untyped dict). - Rename public ColumnRedaction -> ColumnRedactions (matches Java; pre-release, no alias). Generated wire type untouched. - Type update_type as UpsertType on UpdateRequest and UpsertOptions. Internal HTTP tuning (not exposed via VaultConfig): - httpx connection pool defaults: max_connections=100, max_keepalive_connections=100, keepalive_expiry=60s (was 20 / 5s) to reduce connection churn. - Raise bulk insert/detokenize MAX_CONCURRENCY cap from 10 to 100. Tests updated accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Typed response records matching Java: Insert/Get/Update/Delete/Detokenize/ Query response records, Query metadata, Bulk insert/detokenize records, Token. Controllers now return these typed objects instead of dicts. - update() returns a single unified records list (UpdateResponseRecord), errors folded inline; dropped UpdateResponse.errors (matches Java). - request_id carried on error records across unary + bulk. - Per-operation unary options (Insert/Get/Update/Delete/Query/Detokenize) with interceptor support, mirroring the bulk options. - UpdateRequestRecord type; UpdateRequest.update_type typed as UpsertType. - GetRequest/GetRequestRecord: ids -> skyflow_ids, fields -> columns. - Updated validations, tests, samples, README, and regenerated the public API contract baseline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ty mirror Remove the `query` operation from the Python flowvault SDK so its operation set mirrors Java flowvault (which has no query): - Drop VaultController.query and the query wire helper; remove QueryRequest/ QueryResponse/QueryResponseRecord/QueryResponseMetadata/QueryOptions data classes, validation, messages, the client query-api accessor, tests, and the query sample. - Relax the shared common IVaultController interface: query is no longer a required abstract op (skyvault keeps its own concrete query, unaffected), so flowvault's controller is concrete without it. Updated the common contract test. - Regenerated the flowvault public-API contract baseline (query classes removed). Rewrite flowvault/README.md as a full mirror of the Java flowvault README structure, adapted to Python: expanded Authenticate (bearer/context/scoped/signed token generation via skyflow.service_account), VaultConfig + builder reference tables, Schema-vs-schemaless, Unary-vs-bulk parity, the "SDK Guidelines: Unary vs Bulk Operations" section, per-operation sections with sample responses, Custom Request Headers, and a full Error Handling breakdown. Dropped the Java-only tokenize/delete-tokens ops and the removed query op; removed the "Privacy DB" install-vs-import note; corrected the bulk concurrency max to 100. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Java enforces MAX_BULK_DATA_SIZE=100000 only on bulk operations; unary insert has no SDK ceiling. Python diverged: it capped at 10000 and also capped unary insert. - Rename MAX_INSERT_RECORDS(10000) -> MAX_BULK_DATA_SIZE(100000) in _validations. - Remove the size check from validate_insert_request (unary insert), matching Java. - Bulk insert / bulk detokenize now cap at 100000; updated their messages. - Drop the now-unused TOO_MANY_RECORDS_IN_INSERT message. - Update tests (bulk boundary 100001; remove obsolete unary-insert ceiling tests). - README: bulk ceiling 10,000 -> 100,000; unary parity row now "not enforced by the SDK", matching Java. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ary ops - Unary insert/get/update/delete/detokenize now route a non-2xx response whose body carries a records array through the per-record formatter (matching Java and the existing bulk behavior) instead of raising, so per-record not-found/invalid rows surface with skyflowId/httpCode/error. - Make the shared record formatters dict/object-aware via __wire_record_value so they handle both success models and camelCase error-body dicts. - Treat ParsingError like ApiError in __to_skyflow_error for clean whole-call error shapes. - Regenerated types: skyflowID, detokenize value and tokenGroupName are now optional (spec nullability fix); fern SDK version bump 0.0.21 -> 0.0.23. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Insert: add a multi-record example (and a per-record-table/upsert variant),
since InsertRequest.records is a list but only a single-record sample existed.
- Retrying the failed records: add a worked unary retry pattern with the exact
500-599-excluding-529 predicate; flag insert-retry idempotency (use upsert on
a unique column so a lost-response resubmit updates instead of duplicating).
- Re-declare `vault = skyflow_client.vault('<VAULT_ID>')` at the top of every
operation snippet so TOC-jump readers don't hit an undefined name.
- Configuration: warn that this package and the main `skyflow` SDK share the
top-level `skyflow` import and cannot coexist in one environment.
- Batching and concurrency: explain the env-var-only model as an intentional,
cross-SDK (Java-parity) deployment-time tuning choice.
- Per-record table: mark `.error`/`.request_id` as always present, populated
only on failure (they show as null on success), not "failures only".
- Rename "Flow DB" -> "FlowVault" in README and samples README.
Docs only; no SDK behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The wire record response carries `data` (the record's stored column values), and the Java SDK maps it on both unary and bulk insert. Python was dropping it: unary insert built records with include_data=False, and the bulk insert batch formatter never set data, so callers saw data=None even though the API returned values. Map data on both paths, matching get/update and Java. Also update the README: drop the "insert omits .data" notes, add `.data` to the insert accessor lists, and show data in the insert sample responses. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Public API contract change (
|
Public API contract change (
|
- generated/rest/version.py looked up a non-existent distribution name
("skyflow.generated.rest"), so importing the generated package during
unittest discovery raised PackageNotFoundError and failed CI. Use the real
distribution name (skyflow-flowvault-python) and fall back to a default when
the package is not installed, so the import can never crash. __version__ is
not used at runtime (the client wrapper sends a hardcoded SDK version).
- Update the unary error tests: a non-2xx response carrying a records body now
returns per-record error rows (matching Java and bulk) instead of raising,
and insert responses now carry .data. Whole-call/flat-body errors still raise.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
yaswanth-pula-skyflow
previously approved these changes
Sep 15, 2026
skyflow-bharti
previously approved these changes
Sep 15, 2026
The custom build_py vendors the sibling common/ tree into the wheel, but the sdist never included it, so a source build (pip --no-binary, some mirrors) produced a package that fails to import common. Add a custom sdist command that vendors common/ into the tarball (MANIFEST grafts it) and clean it up after, and resolve COMMON_SRC from that vendored copy when the sibling checkout is absent (i.e. when building from the sdist). Exclude common from find_packages so the vendored copy is bundled only through the build_py mechanism. Wheel builds are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
saileshwar-skyflow
dismissed stale reviews from skyflow-bharti and yaswanth-pula-skyflow
via
September 15, 2026 13:42
876e4d4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the FlowVault Python SDK (
skyflow-flowvault-python, imports asskyflow) — a high-throughput SDK for FlowVault vaults, built to mirror the Java FlowVault SDK. It shares thecommonclient/credentials/config layer with skyvault and exposes a narrower surface: unary vault operations plus bulk (batched, concurrent) insert and detokenize.What's included
insert,get,update,delete,detokenize— typed request/response objects, per-recordhttp_code/error/request_id.bulk_insertandbulk_detokenize, each with sync and async variants, batching + bounded concurrency (env-var tuned), per-record results, andrecords_to_retry()/tokens_to_retry()helpers.build_pybundles the siblingcommon/into the wheel; installs under theskyflowimport name.Notable fixes on this branch
.dataon insert responses (unary + bulk) — the field the wire returns and Java maps was being dropped.skyflowID, detokenizevalue/tokenGroupName) so partial-failure responses parse.vaultre-declaration, import-name collision warning, batching-config rationale, accurate per-record field docs).Testing
sdk-integration-testsrepo) exercising bulk + unary operations against a real vault.🤖 Generated with Claude Code