chore: release main - #1287
Merged
Merged
chore: release main#1287
Conversation
Contributor
|
Claude Code Review — skipped: PR author 'caura-deploy-bot[bot]' is not a public member of the 'caura-ai' org |
caura-deploy-bot
Bot
force-pushed
the
release-please--branches--main
branch
from
September 4, 2026 23:10
e7b4b6f to
08a3992
Compare
Contributor
|
Claude Code Review — skipped: PR author 'caura-deploy-bot[bot]' is not a public member of the 'caura-ai' org |
caura-deploy-bot
Bot
force-pushed
the
release-please--branches--main
branch
from
September 4, 2026 23:21
08a3992 to
ca5e86a
Compare
Contributor
|
Claude Code Review — skipped: PR author 'caura-deploy-bot[bot]' is not a public member of the 'caura-ai' org |
Signed-off-by: release-please[bot] <release-please[bot]@users.noreply.github.com>
caura-deploy-bot
Bot
force-pushed
the
release-please--branches--main
branch
from
September 4, 2026 23:29
ca5e86a to
763a6e1
Compare
Contributor
|
Claude Code Review — skipped: PR author 'caura-deploy-bot[bot]' is not a public member of the 'caura-ai' org |
Eldad-Caura
approved these changes
Sep 4, 2026
Contributor
Author
|
🤖 Created releases: 🌻 |
caura-deploy-bot
Bot
added
autorelease: tagged
Eldad-Caura
added a commit
to adity982/caura-memclaw
that referenced
this pull request
Sep 5, 2026
…aura-ai#949) Ports the fix already merged as [caura-enterprise caura-ai#1287](caura-ai/caura-enterprise#1287) to this repo's copy of `common/events/pubsub.py` — which turns out to be where nearly all of the volume actually comes from. ## Problem An idle Pub/Sub subscriber emits **one error span per empty pull window**, forever. `PubSubEventBus._pull_loop` calls `subscriber.pull(timeout=...)` and already treats `DEADLINE_EXCEEDED` as the non-event it is: ```python except gexc.DeadlineExceeded: # No messages in the pull window; loop back and try again. continue ``` ddtrace's gRPC integration marks the span failed on *any* non-OK status before that `except` ever runs — `contrib/internal/grpc/client_interceptor.py`, `if response_code != grpc.StatusCode.OK` → `span.error = 1` — with no per-status config to opt out. So the application handles the timeout correctly and APM records a failure anyway. **Measured over 24h**, `@error.type:StatusCode.DEADLINE_EXCEEDED` on `/google.pubsub.v1.Subscriber/Pull`: | service | env | spans | |---|---|---| | core-api | staging | 51,568 | | core-api | prod | 617 | | core-worker | prod / staging | 99 / 90 | An idle deployment is far noisier than a busy one — a pull that returns messages succeeds — which is why staging dwarfs prod. Even so, in prod this was the single largest source of error spans (1,223 of ~3,163), so error rate and any monitor on it were reporting mostly poll timeouts. ## Fix A trace processor that clears `span.error` for that exact resource + status, and nothing else. - **Fixes the metric, not just the display.** `SpanAggregator.on_span_finish` chains `user_processors` ahead of sampling and the writer, so the payload the Agent computes APM stats from already carries `error: 0`. - **The span is kept, not dropped.** A pull that fails for a real reason still arrives as an error, and poll timing stays visible. `error.*` tags are left in place so the timeouts stay queryable on purpose (`@error.type:StatusCode.DEADLINE_EXCEEDED`) rather than merely absent. - **`StatusCode.CANCELLED` is deliberately untouched** — a pull interrupted by shutdown, which I observed for real alongside the timeouts. It says something true about the process. - **Registration is additive.** `configure(trace_processors=...)` replaces the user-processor list rather than appending, so a bare call would silently evict any processor registered before us. `install()` reads what's registered and passes it back through. - **`install()` never raises.** Both the private-aggregator read and the `configure()` call are guarded — a signature change across a major ddtrace bump would otherwise propagate out of `PubSubEventBus.start()` and fail service boot. Correct error rates aren't worth trading a service that starts. - **Registered inside `start()`'s `topic_count > 0` branch** — the spans come from the pull loop, so a publisher-only bus can't produce one and shouldn't pay for registration (which recreates the trace writer). There is no gRPC status→error config knob in ddtrace 4.x (nothing like `DD_TRACE_HTTP_SERVER_ERROR_STATUSES` for gRPC; `config.grpc` carries no error-status field), so a user trace processor is the only supported altitude. ## Tests deliberately do not require ddtrace Worth calling out, because it differs from the enterprise version. CI installs only each service's `[dev]` extra, never `datadog` — so a module-level `pytest.importorskip("ddtrace")` would skip the whole file in the one environment that gates merges, shipping this with no coverage at all. The filter only touches `.error`, `.resource` and `.get_tag()`, so a stub span covers it. One additional test pins the stub to the real `Span` shape (and asserts `ddtrace.constants.ERROR_TYPE == "error.type"`) whenever the extra happens to be installed — skipped in CI by design, so stub drift gets caught locally rather than never. 16 tests plus that fidelity check: the clear; the trace returned rather than dropped; five must-stay-an-error cases (`PERMISSION_DENIED`, `NOT_FOUND`, `CANCELLED`, Publish timeout, `StreamingPull`); a missing `error.type` tag; `install()`'s idempotency guard; an already-registered processor surviving; a renamed private aggregator; an absorbed `configure()` failure; and a publisher-only bus not registering at all. **Each was confirmed to fail without its fix.** ## Verification - `16 passed, 1 skipped` for the new file, run in a venv **without** ddtrace — i.e. the CI configuration - `5075 passed, 4 skipped, 1 xfailed` for the full repo-root suite - `ruff check common/` clean; `ruff format --check` clean; `mypy --config-file common/mypy.ini common/` reports only two pre-existing `dateutil` stub errors in `common/enrichment/service.py`, none in the new code 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: eldad-caura <eldad@caura.ai> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 I have created a release beep boop
backend: 2.47.3
2.47.3 (2026-09-04)
Bug Fixes
Documentation
This PR was generated with Release Please. See documentation.