Migrate the postcard flow to postcardCollect; remove the legacy sighting API - #41
Draft
parrot-tailor wants to merge 35 commits into
Draft
Migrate the postcard flow to postcardCollect; remove the legacy sighting API#41parrot-tailor wants to merge 35 commits into
parrot-tailor wants to merge 35 commits into
Conversation
Replace the drifted .gitignore with a byte-for-byte copy of upstream github/gitignore Python.gitignore, then append a project-specific section. Upstream adds .ruff_cache/ and comments out .python-version so the pinned interpreter can be checked in. Preserve the maintainer's explicit editor ignores (.idea/, *.iml, .vscode, *.whl) and add junit.xml, emitted by the pytest config.
Adopt PEP 621 metadata with the setuptools build backend. Scope packages to birdbuddy* so tests/ is no longer shipped in the wheel (find_packages() previously packaged it). Move dev/test/publish tooling into an optional [dev] extra: pytest stack, ruff, pyright, build, twine. Pin the dev/lint interpreter via .python-version (3.10.20, the support floor). Remove setup.py, requirements.txt and requirements.test.txt. Publishing is now: python -m build && twine upload dist/*
Configure the toolchain in pyproject.toml: - ruff: full google-style ruleset at line-length 79, target py310, with absolute-import enforcement (TID252 + ban-relative-imports = all) - pytest: asyncio_mode = auto and coverage/junit addopts - coverage: source = birdbuddy, omit tests Add pyrightconfig.json (py310, tests exec env), .markdownlint.yaml (matches the linters used in CI), and a Makefile with deps/format/lint/ test/check/clean plus build, publish (python -m build + twine) and schema targets. Lint is intentionally red until the cleanup commit; the source is not yet formatted or annotated.
Proposing to license this project under the MIT license. Fixes jhansche#32.
Bring the tree to a clean gate (make check: ruff, ruff format, markdownlint, pyright, pytest) without changing valid-input behavior: - Format to line-length 79 and sort imports; convert package-relative imports to absolute (TID252, ban-relative-imports = all). Symbol imports are kept (not module-qualified). - Rewrite docstrings to Google style: Args/Returns/Raises on functions with parameters, attribute-style one-liners on properties; convert the old reStructuredText :param blocks. - Add type annotations; refine loose return types (e.g. str | None, datetime | None), fix UserDict[str, any] (the builtin) to typing.Any, and correct GraphqlError.error_code to str | None. - Remove asserts from source (they are reserved for tests): precondition asserts become explicit raises, and the two assert-await-refresh cases become plain await calls so the request is not tied to an assertion. - Enable the S101 (assert) and FA (future-annotations) ruff rules; ignore E501 for GraphQL query strings; keep the N818 noqa on CompositeException (public name, no rename). - Add a PEP 561 py.typed marker (shipped in the wheel) so downstream consumers can use the annotations. - Reflow README prose to satisfy markdownlint. make check passes on 3.10.20; the frozen public API imports unchanged.
Move the schema artifacts out of the repo root: - scripts/dump_schema.py writes the fixture via a __file__-relative path (run with make schema) - tests/fixtures/schema.json, refreshed from the live introspection endpoint - remove the stale schema-20230225.txt Add tests/test_schema.py, which derives the GraphQL types the library's queries use (input types from birdbuddy.queries, object types from the SightingType enum) and asserts the committed schema still defines them. The guard immediately surfaces real drift: the refreshed schema no longer defines SightingCreateInput or SightingCreateCheckProgressInput, so make check fails here -- the sighting_create and sighting_create_check_progress queries are stale against the current API.
The schema drift guard flags that the API dropped SightingCreateInput and SightingCreateCheckProgressInput. Exempt them via a documented _KNOWN_MISSING allowlist: the generic sightingCreate flow was removed with no equivalent (the app now collects via postcardCollect), so sighting_create / sighting_create_check_progress are non-functional but are kept for backwards compatibility (see jhansche#29). Add scripts/dump_payloads.py: a maintainer tool that logs in via BB_EMAIL/BB_PASSWORD and dumps the postcard -> sighting flow (capturing server errors) to a git-ignored *.dump.json, for debugging and for building sanitized test fixtures.
reanalyze_postcard and sighting_from_postcard branched on str / FeedNode with no else, leaving postcard_id unbound (UnboundLocalError) for any other input type. Add an else that raises a clear TypeError. Valid str / FeedNode inputs behave identically. Add a guard test.
Merge the issue/feature-named test_issue_40.py and test_sighting_create.py into a source-module-named test_client.py. Parameterize the best-guess / anomaly-correction finish cases and the in-progress / completed check-progress cases; hoist the repeated UUIDs into named constants. Rename the issue_40 fixture and issue-40.yaml to postcard_sighting. Behavior and coverage are unchanged.
Add tests/test_sightings.py pinning the current behavior of the complex sighting-report logic the postcard finishing flow relies on: SightingType classification, token_json (plain / signed / malformed), the finishing strategies (recognized, recognized-species propagation, mystery fallback), and highest_confidence_matches. Gives a safety net for a later rewrite.
Rewrite .github/workflows/python-package.yml as a single check job that runs make deps then make check across a Python 3.10-3.14 matrix, so CI exercises the same ruff, ruff format, markdownlint, pyright and pytest gate as local development. Bump to actions/checkout@v7 and actions/setup-python@v6, add contents: read permissions, and drop the inherited weekly cron in favor of push, pull_request and workflow_dispatch triggers. Add .github/dependabot.yml to track pip and github-actions updates, grouped weekly.
Give the README a clear structure and fix its broken examples: - Add the project description and status shields (PyPI version, build, maintenance, GitHub release, license). - Split the content into Installation, Usage, Translations, Development and Releasing sections. - Fix the Translations example: import BirdBuddy from birdbuddy.client (the package root only exports LOGGER/VERBOSE) and instantiate it with credentials instead of referencing the bare class. - Document the development workflow: pyenv, make deps and the make targets (each self-activates the venv), plus pip install -e '.[dev]'. - Document the PyPI release process: bump the version, then make build (python -m build -> sdist + wheel) and make publish (twine check + upload), including how twine reads its credentials. The FeederForPrivate fragment is unchanged; it is still the type the library queries and is present in the refreshed schema.
Cover the model classes that had no direct tests, built from dict
fixtures with no network:
- Species and BirdBuddyUser property mapping.
- Feed / FeedNode / FeedNodeType: type resolution and the Unknown
fallback, datetime parsing, and Feed.filter / newest_edge.
- Media / Collection / is_media_expired: signed-URL expiry, image vs
video, and collection species/visits/cover media.
- Feeder enums (known values + UNKNOWN fallback), Signal, Battery,
Feeder, and FeederUpdateStatus.
Raises total coverage from 63% to 71%.
Pin one dormant quirk (and mark it with a TODO) rather than change
behavior here: Feeder.power_profile defaults to "STANDARD", which does
not match the PowerProfile enum value ("STANDARD_MODE"), so a Feeder
missing powerProfile resolves to UNKNOWN. This never surfaces in
practice -- only owner responses carry powerProfile and they always
include it -- so ha-birdbuddy is unaffected. It is an independent
bugfix, not part of the 0.1.0 breaking changes.
Convert postcard_sighting.yaml to JSON and load fixtures via json; drop the now-unused pyyaml dev dependency. The API, the payload dumper and schema.json are all JSON, so the fixtures follow suit. Sanitize postcard_sighting.json: remap every account identifier (feeder, sighting and media ids) to generated UUIDs and re-encode its JWT reportToken with scrubbed userId/feederId. The token's reportItems, which the finish-flow tests rely on, are preserved, and test_client.py's expected constants are remapped to match. Add tests/fixtures/api_payloads.json: a sanitized capture of live responses (2026-07). Identifying values (UUIDs, media URLs, feeder name, serial number, location, invite/report tokens) are replaced; structure, enum values, timestamps and the sighting_create error are preserved. Add tests/test_api_payloads.py, exercising Feeder, FeedNode, PostcardSighting/SightingReport and the recognized finish strategy against the real payload, and pinning the live sighting_create drift (Unknown type "SightingCreateInput", jhansche#29). Mark a second dormant model bug with a TODO: FeederForOwner nests location{city,country} while FeederForMember/Public use flat locationCity/locationCountry (neither deprecated), so Feeder.location returns (None, None) for owner feeders.
The next release carries breaking changes (the postcard flow is migrated to postcardCollect and the removed-upstream sighting_create methods are dropped), so bump the minor version.
Extend scripts/dump_payloads.py to embed the postcardCollect / reanalyze operation text directly so the script is self-contained, and gate the destructive postcardCollect behind BB_COLLECT_POSTCARD_ID so it only runs when a specific postcard is opted in. Reanalyze (AI inference) runs by default and is idempotent. Add tests/fixtures/collect_flow.json: a sanitized 2026-07 capture of reanalyze + postcardCollect against a real account. It pins the flow's confirmed behavior: reanalyze is synchronous and idempotent (MANUAL_COMPLETED / ALREADY_REANALYZED), postcardCollect succeeds for an auto-recognized postcard without mediaSpeciesItems, and FeedItemCollectedPostcard.species is a list. UUIDs are remapped, media URLs / feeder identity / tokens scrubbed; enums and public species names kept.
Add the POSTCARD_COLLECT query string (with a CollectedPostcardFields fragment) and a CollectedPostcard model wrapping FeedItemCollectedPostcard, with accessors for species, medias, the mystery/new-species flags, inference mode/type, and the species-name identification confidence. Postcard confidence is reported by the API only as enum buckets (inferenceConfidenceLevel HIGH_CONFIDENCE/LOW_CONFIDENCE at the new-postcard stage; mediaSpeciesNameIdentificationConfidenceLevel CANNOT_DECIDE/VERY_CONFIDENT on the postcard) -- the numeric InferenceMediaRecognitionSuggestion.confidence is not reachable through the collect flow -- so the model surfaces the enum, not a score.
Add BirdBuddy.collect_postcard, which reanalyzes a postcard first (idempotent, so it is safe whether or not inference has already run) and then collects it with the species the backend recognized, returning a CollectedPostcard. Add a shared _postcard_id resolver used by both reanalyze and collect. Reanalyzing before collecting is the fix for the postcard internal-server errors: converting or collecting a postcard whose inference had not run errored server-side, and nothing ran inference first. Fixes jhansche#29 Related to jhansche/ha-birdbuddy#98
Delete the report-based postcard flow the API and app have moved off of: the sighting_from_postcard, finish_postcard, sighting_choose_species and sighting_choose_mystery methods, the removed-upstream sighting_create and sighting_create_check_progress, their query strings, and the whole birdbuddy.sightings module (PostcardSighting, SightingReport, Sighting, SightingType, SightingFinishStrategy, SightingFinishMod, SightingCreateProgress) with its best-guess/anomaly logic. collect_postcard replaces it -- a breaking API change. Drop the now-obsolete tests and the postcard_sighting fixture, and simplify the schema-drift test since no query references a dropped input type. Related to jhansche/ha-birdbuddy#78 Related to jhansche/ha-birdbuddy#40
Feeder.power_profile passed a "STANDARD" default into PowerProfile when the field was absent, which does not match the enum value "STANDARD_MODE" and so resolved to UNKNOWN while logging a spurious "Unexpected power profile: STANDARD" warning. Return the reported value, or UNKNOWN when the feeder does not carry one (non-owner feeders) -- dropping the bogus default and the warning. A present value (owner responses) is unchanged.
FeederForOwner nests location as location{city,country} while
FeederForMember/Public use flat locationCity/locationCountry;
Feeder.location read only the flat keys, so owner feeders reported
(None, None). Read the nested shape when present, else the flat keys.
Tests cover both shapes and the absent case.
collection() returned only the first page, silently truncating larger collections. Follow the connection cursor until hasNextPage is false so the full set of media is returned. Add a page_size argument (default 50) that caps the per-request batch. The API rejects a page size above 100 -- it responds HTTP 200 with a GraphQL INTERNAL_SERVER_ERROR rather than a client error -- so guard the input to 1-100 and raise ValueError instead of round-tripping to fail. page_size only tunes how many media are fetched per request; the full collection is returned regardless. Paginate through a shared _iter_pages helper that advances by endCursor and stops defensively when the server claims another page but returns no advancing cursor (missing, null, or already seen), so a stuck cursor cannot spin the client in an infinite request loop. supersedes jhansche#38 Co-authored-by: tombeaulah <tom.beaulah@gmail.com>
The feed methods fetched only the first page. refresh_feed therefore under-reported when more than one page of items was new since the last refresh -- fewer results than the app showed -- and new_postcards and feed_nodes saw only the newest page. feed() also passed first straight through, so grabbing a large window with first above 100 errored server side (HTTP 200 with a GraphQL INTERNAL_SERVER_ERROR). Guard feed(first=) to 1-100 and raise ValueError otherwise. Page refresh_feed backward through the feed until it reaches items no newer than since, so every new item is returned rather than truncated; with no cutoff (no prior refresh) it still returns only the newest page to avoid replaying the whole history on startup. Page feed_nodes and new_postcards across every page. All of it reuses the defensive _iter_pages cursor walk, so a server that reports another page without an advancing cursor cannot spin the client in an infinite request loop. Refs jhansche#29
AnyFeedItem is a union whose members all implement the FeedItem interface (id, createdAt), but AnyFeedItemFields selected createdAt only inside inline fragments for the ten enumerated members. Any other member -- the union has seventeen, and more are added over time -- came back with just __typename and therefore no createdAt, which surfaced as sorting and filtering the feed on a null timestamp. Add an inline fragment on the FeedItem interface so every member, including those not individually enumerated and any added later, returns id and createdAt. The None-guards in Feed.newest_edge and Feed.filter stay as defense in depth. Fixes jhansche#24
createdAt (Media, FeedItem) and visitLastTime (collections) are DateTime! in the schema, and every query returning those objects selects them, so the values are never null. They were typed datetime | None only because parse_datetime accepted None and propagated it to every caller. Narrow parse_datetime to str -> datetime and keep optional handling in FeedNode.created_at alone -- the one caller whose value is genuinely optional, since feed items are AnyFeedItem union members and a member whose createdAt the query does not select carries no timestamp. Media.created_at and Collection.last_visit now return datetime. Feed.newest_edge filters undated edges with a walrus guard so the type checker can follow that the compared values are non-null.
Extend scripts/dump_payloads.py to capture the read-only ME profile and collections alongside the feed and reanalyze, and add a deterministic sanitizer: UUIDs are remapped with uuid5, and tokens, email, names, serials, URLs and location are replaced with stable fakes, while enums, numeric metrics, timestamps and public species names are preserved. The script writes both the raw dump and a sanitized copy (both git-ignored) so fixtures can be regenerated cleanly from real responses.
Rebuild tests/fixtures/api_payloads.json from a fresh sanitized capture: the ME profile (user + feeders), a slice of collections (both bird and mystery-visitor), and new postcards -- dropping the dead sighting-flow payloads left over from the removed legacy API. Update test_api_payloads to the me-wrapped structure and the fresh sanitized values.
Cover the previously untested client operations via graphql_mock and the regenerated fixtures: sign-in, token refresh, refresh, collection filtering, feeder options, media sharing, power profile, and the firmware check. The power-profile and firmware-check responses are real captures: both mutations resolve asynchronously (power profile reports the old value until a refresh; an up-to-date firmware check returns a succeeded result), so the tests exercise that observed behavior.
Add a "Breaking changes (v0.0.x -> v0.1.0)" section covering the move to collect_postcard, the removed sighting flow and dead sighting_create methods, collection and feed pagination, the power_profile and location fixes, and the tightened timestamp return types. Update the intro to describe collecting postcards rather than finishing sightings.
Starting a firmware update on a feeder already running the latest version returns an internal server error rather than a client error -- the same class of misbehavior as the collection and feed page-size limits. The feeder reports firmwareVersion and availableFirmwareVersion, so compare them after the progress check and raise NoFirmwareUpdateAvailableError instead of round-tripping into the server error.
latest_collections referenced queries.me.LATEST_MEDIA, which is not defined anywhere, so every call raised AttributeError. Remove the dead public method; refresh_collections covers fetching collections.
test_schema.py only checks that the input types a query references exist; it cannot catch a malformed field selection or a missing field argument. Add graphql-core to the dev extra and a test that parses every query string in birdbuddy.queries and validates it against the committed introspection schema, so an invalid selection fails make check.
Replace the raw ListFeederFields schema fragment with a runnable example that logs in and prints each feeder's state, battery, signal, and firmware version.
Restore the read/preview that the removed sighting_from_postcard gave: a postcard's recognized species, media, and feeder WITHOUT collecting. reanalyze_postcard (metadata only) becomes identify_postcard and returns a PostcardAnalysis. Extend POSTCARD_REANALYZE to select the sighting-report preview's confidently-recognized species, the postcard media (inline MediaImage/MediaVideo contentUrl), and the feeder -- omitting matchTokens/shareableMatchTokens/reportToken. Only confidently recognized species are exposed; the can't-decide "suggestions" are a manual identification affordance in the app, not an identification, so they are not surfaced. Have dump_payloads.py send the library's own POSTCARD_REANALYZE and POSTCARD_COLLECT queries so fixtures cannot drift from what the client actually sends -- the drift that previously hid query bugs from tests. Regenerate the collect_flow reanalyze fixture, add PostcardAnalysis tests, and document the flow in the README. Related to jhansche/ha-birdbuddy#78
- is_media_expired: return None when the URL has no Expires query param (as the docstring already promises) instead of raising KeyError. Real signed media URLs carry Expires, so this is defensive. - identify_postcard / collect_postcard: raise UnexpectedResponseError when the GraphQL response lacks the expected nested fields, instead of a bare KeyError/TypeError on result[...][...]. Both paths are consumed pervasively by downstream integrations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Depends on #40. This branch is cut from
modernize-tooling(#40) but targetsmain— a fork branch can't be a PR base, so both PRs base onmain. Until #40 merges, the diff and GitHub's commit list here include #40's commits; review/merge #40 first, then this rebases cleanly ontomain. The Commits section below lists only the v0.1.0 work (#40's commits are documented in #40).Migrates the postcard flow to
postcardCollect, removes the legacy sighting/report API outright, paginates the collection and feed endpoints, fixes thepower_profileandlocationmodel bugs, and bumps0.0.21→0.1.0. This is the follow-up PR I mentioned in #40. Unlike #40, it is breaking: the Bird Buddy app has moved entirely topostcardCollect. Thesighting_create/sighting_create_check_progressinput types are gone from the API; the postcard report-token flow (sightingCreateFromPostcard/sightingReportPostcardFinish) still exists upstream but is dropped here by choice — the app abandoned it, and itsreportToken/matchTokenswere the ha-birdbuddy #78 event bloat. Inference without collecting is preserved:identify_postcardreturns aPostcardAnalysis(recognized species + media, no collect).Every claim below was verified against the live API with a real account, the committed schema, and the app's JS bundle.
Commits
scripts/dump_payloads.pyandcollect_flow.json(a sanitized reanalyze +postcardCollectcapture) pinning the flow's real behavior.POSTCARD_COLLECT+ aCollectedPostcardmodel (species, medias, mystery/new-species flags, inference mode/type, confidence). Media selectscontentUrlvia inlineMediaImage/MediaVideofragments.postcardCollect; reanalyzing first is the fix for the postcard internal-server errors. Fixes BB API update / increased number of internal errors from server #29. Related to HA with Blueprint is no longer autocollecting Postcards ha-birdbuddy#98.sighting_from_postcard,finish_postcard,sighting_choose_*, the deadsighting_create/sighting_create_check_progress(their input types are gone upstream), thebirdbuddy.sightingsmodule, and the best-guess/anomaly logic. Breaking. Related to Bird Buddy Postcard Event Exceeds 32768 ha-birdbuddy#78 and Modernize packaging, tooling, and CI; refresh the GraphQL schema #40."STANDARD"default (never matchedSTANDARD_MODE); add the schema'sULTRA_FRENZY_MODE.location{city,country}; read it as well as the flat member/publiclocationCity/locationCountry._iter_pages(stops on a stuck cursor); add apage_sizeguard (1–100, the API's limit). supersedes Paginate collection() to return all media #38.refresh_feedpages to thesincecutoff;feed_nodes/new_postcardspage fully;feed(first=)guarded to 1–100. Refs BB API update / increased number of internal errors from server #29.AnyFeedItemis a union whose members all implementFeedItem; an interface inline fragment makes every member returncreatedAt. Fixes Some feed items do not have created_at #24.Media.created_at/Collection.last_visitare non-null in the schema, so they returndatetime;parse_datetimebecomes a plainstr -> datetimeparser andFeedNode.created_atstays optional (the union case).api_payloads.jsonfrom a fresh sanitized capture (profile, collections, postcards), dropping the dead sighting-flow payloads.NoFirmwareUpdateAvailableErrorinstead.AttributeErroron every call.test_schema.pyonly checks input types, so this catches malformed selections/args (it caught a malformedPOSTCARD_COLLECTduring development).sighting_from_postcardgave:identify_postcardreturns aPostcardAnalysis(recognized species + media + feeder, no collect); the extendedPOSTCARD_REANALYZEomitsreportToken/matchTokens. Only confidently-recognized species are exposed — the "can't decide" suggestions are a manual-identification affordance in the app, not an identification. The dumper now sends the library's own queries so the fixtures can't drift from what the client sends. Related to Bird Buddy Postcard Event Exceeds 32768 ha-birdbuddy#78.is_media_expiredreturnsNonefor a URL with noExpiresparam (as its docstring promises) instead ofKeyError;identify_postcard/collect_postcardraiseUnexpectedResponseErrorinstead of a bareKeyError/TypeErrorwhen the response lacks the expected fields. Both are consumed pervasively by ha-birdbuddy (sensor/image/visitors).Issue references
created_at('>' not supported between NoneType and datetime); the interface fragment makes the field always present, and a regression test pins the guards.collect_postcardreanalyzes before collecting, and the feed/collection page-size guards stop thefirst > 100internal errors.Co-authored-bytrailer and extended with thepage_sizeguard. A keyword can't auto-close a PR, so please close Paginate collection() to return all media #38 on merge.collect_postcard), #78 (postcard event > 32768 bytes —the bloat was thereportToken/matchTokens, which the new preview omits), #40 (per-photo anomaly correction — obsolete now that the backend aggregates media and reports confidence).Testing
make check(ruff, ruff format, markdownlint, pyright, pytest) passes locally on Python 3.10.20; CI runs the same gate across 3.10–3.14. Coverage: 71% → 85% (52 → 91 tests). The newtest_query_validation.pyvalidates every query against the schema. Beyond the mocked suite, the actual client methods were driven live against a real account:refresh,refresh_collections,collection(cursor pagination),new_postcards,refresh_feed,identify_postcard(a freshMANUAL_NOT_STARTEDpostcard →PostcardAnalysiswith species +media), and one opted-in
collect_postcard(returned aCollectedPostcard;contentUrlresolved through the fixed query).Follow-up (ha-birdbuddy)
ha-birdbuddy stays pinned
==0.0.20until a fast-follow PR adopts the new API:identify_postcard(aPostcardAnalysiswith species + media) to display the current visitor without collecting, andcollect_postcardfor the opt-in collect. That migration drops the postcard event bloat (jhansche/ha-birdbuddy#78) and the autocollect errors (#98). I'll prepare it against ha-birdbuddy shortly.