Repository navigation
Conversation
The server-side API gained POST /v3/search/metadata/raw, which runs a caller-authored OpenSearch query body and overlays the tenant filter itself. This is the client for it. The body is passed through untouched and the community travels in the query string, which is why the method uses the inline-dict _single form rather than the generic builder: on a POST the builder puts every parameter in the JSON body, and the server reads community/communities out of that body when it resolves scope. A community folded into the body would therefore choose the scope via a key the server's own grammar then rejects. A POST is never paginated, so _single returns the one composite result - hits and aggregations together - rather than a generator. Tests are in the respx tier rather than the e2e one: what they pin is that the body arrives verbatim and the community is not in it, and neither is visible from the far side of a live call. The live lifecycle test follows separately.
A VCR lifecycle test per transport, each with its own cassette recorded against a freshly booted stack running the matching server branch. Both replay offline for the unit jobs and both pass with TESTS_VCR=off against the live stack, which is the invariant that keeps the cassettes an efficiency cache rather than the thing under test. The query aggregates on `extension` rather than a mimetype field: the metadata document has no artifact.mimetype at all (the mimetype it does carry is scan.mimetype), and a terms aggregation on an unmapped field is not an error - it returns an empty bucket list, so the assertion fails with nothing to say why. The respx example moved to scan.mimetype for the same reason, even though a mock would have accepted either. Recording these needed the stack's ai-es-dispatch-pending step: it is the only thing that turns a claimed es_metadata row into an Elasticsearch write, so without it the metadata index is never created and every metadata test polls an empty corpus until it gives up.
poll_equals absorbs NotFoundException as "the index has not caught up". That is right for a document and wrong for an endpoint: a search matching nothing is a 200 with total 0, so a 404 from this route means the server side is not deployed. Absorbed as lag it burned 90 polls and then failed as `assert None == 1`, naming nothing - and because the e2e job runs with TESTS_VCR=off against a live stack, that is what develop would do if this landed before the server release, blocking the :latest promotion every other repo's pipeline pulls. The reader now skips with the reason instead. It costs no extra request - the first poll is the probe - so the cassettes still replay. Drop it once the route ships. Also document two things the method's contract depends on and its docstring left out: omitting `size` returns no documents, because the server defaults it to 0 so an aggregation caller does not pay for hits they did not ask for; and `partial` is a list of reasons, whose absence - not an empty `hits` - is what says a count is whole.
The docstring told callers that partial's absence means a count is whole. It does not: the server suppresses the hit-total signal when the body asked for no hits, because a total capped at track_total_hits is what every large query returns and reporting it would mark all of them partial. Exactness of a count is hits.total.relation - eq or gte - which is the engine's own field and passes through untouched. partial speaks for the aggregations and the search as a whole.
…specs AGENTS.md step 5 asks for specs/03-endpoints.md in the same PR as a new resource, and the class tree in specs/02-resources.md lists every BaseJsonResource. Both omitted this one.
The e2e job runs TESTS_VCR=off against a live stack, so a skip let a renamed or withdrawn route report green and take the :latest promotion with it. AGENTS.md already orders the server endpoint first, so a 404 is a real failure rather than a merge-ordering inconvenience to absorb. The early exit stays: it still names the cause immediately instead of spending 90 polls to fail as `assert None == 1`. Renamed to _raw_hit_count, since it no longer skips.
The fixture mocked a dict, so the test demonstrated a shape the server never sends and the docstring does not promise.
…-unit AGENTS.md routes a new resource through a builder classmethod on the resource, and specs/04 invariant 7 puts request-shape assertions in the pure-unit tier. The inline request dict forced the opposite: body-vs-query routing could only be observed through respx, which that invariant reserves for what the e2e stack cannot produce. MetadataRawSearch.search returns the PolyswarmRequest, so the routing reads off request_parameters with no httpx and no transport parametrisation. A mutation folding the community into the body kills three of the new tests.
The docstring stated the exception as the rule: it said a capped total is not reported as partial, when the server reports exactly that whenever the body asked for hits, and omits it only when it asked for none. A caller following the old text would read a complete aggregation as truncated and re-run a narrowed query, paying for another metered search that can never clear the marker.
The two paragraphs contradicted each other: one said partial speaks for the search as a whole, the next that a large query normally carries one. Both are true of different reasons, which is the point - so the docstring now names the closed set of codes and says to branch on them. The reasons are prose no program can discriminate, which the server fixed by giving each a code. Also drops a stray blank line before _raw_hit_count.
AGENTS.md asks for the spec in the same PR as the code, and the Files list in specs/04-testing.md names every test module. specs/02 and 03 were updated for this endpoint; 04 was missed.
feat: add metadata_raw_search for the raw metadata search endpoint
A historical hunt can now be bounded to the last 30, 60, 90 or 180 days. The new optional keyword goes into the POST body as an int; None drops the key, so a bare call sends what it always did and the server applies its own default (the platform maximum). The SDK does not validate the value: the server owns the enum and answers anything else with a 400, which surfaces as the usual typed exception. The sync mirror is regenerated from the async source. Co-Authored-By: Alan Justino <alan.justino@yahoo.com.br>
The server now returns the hunt's time window on the single GET and on the list. It is an additive .get() like the provenance fields: an int for a hunt created with a window, None for one created without (it reaches the platform maximum) or for a server that predates the field. HistoricalHuntList inherits the parse, so both endpoints are covered. Co-Authored-By: Alan Justino <alan.justino@yahoo.com.br>
Pure-unit tests cover the request shape (an int in the body, the key absent for None, no client-side validation) and the parse (int, explicit null, missing key). One respx body per client pair checks what historical_create sends and what the create, get and list responses carry; VCR cannot, because it does not match bodies and the recorded hunts predate the field. The live round trip needs a fresh e2e stack, so it is recorded as an open question rather than faked into the existing cassettes. Co-Authored-By: Alan Justino <alan.justino@yahoo.com.br>
The CLI client adopts `historical_create(window_days=)` in the paired change set and expresses that as `polyswarm_api>=4.8.0`. develop already declares 4.7.0 for a surface that does not include this keyword, so the floor needs the next version up; a floor cannot name a version this repo has not declared, so the bump lands here, in the feature PR (AGENTS.md's standing exception). Minor: one new optional keyword whose default preserves today's request, plus an additive resource field, so no existing call changes behaviour. Bumped with bump-my-version; the emitted string is a clean `4.8.0`. The server already accepts and returns `window_days`, so this repo can release ahead of the CLI. Co-Authored-By: Alan Justino <alan.justino@yahoo.com.br>
A live pair, sync and async, creates one hunt with a 30-day window and one without, then reads both back on create, get and list: 30 and None. The cassettes were recorded against the e2e stack, and the pair also passes with VCR off. It replaces the respx stand-in that asserted fabricated envelopes, and closes the open question that tracked the gap. The builder, pass-through and parse tests stay: a cassette cannot see request bodies. Co-Authored-By: Alan Justino <alan.justino@yahoo.com.br>
Their recorded create, get and list responses predated window_days. Re-recorded by deleting each cassette and running its test against the e2e stack, so replay serves the shape the server returns today, the key included. Co-Authored-By: Alan Justino <alan.justino@yahoo.com.br>
Co-Authored-By: Alan Justino <alan.justino@yahoo.com.br>
feat: expose window_days on historical hunts
|
Clean against AGENTS.md and specs/: develop → master release with a clean |
sbneto
approved these changes
Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Includes: