Skip to content

fix: eliminate redundant incident-candidate DB fetches in new-issue intake #SUPERLOG - #537

Open
superlog-app[bot] wants to merge 1 commit into
mainfrom
superlog/fix-intake-candidate-dedup
Open

superlog-app[bot] wants to merge 1 commit into
mainfrom
superlog/fix-intake-candidate-dedup

Conversation

@superlog-app

@superlog-app superlog-app Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

New-issue transitions were averaging ~95 s execution time (40% above the ~68 s baseline) for the full observed hour. The elevated duration is caused by two compounding defects in the new-issue intake path, both of which are addressed here.

Root Cause

1. Redundant DB round-trips in findMatchingIncident

For every new issue that falls through heuristic matching, findMatchingIncident was calling findOpenIncidentCandidates(filterService: false) and loadLinkedIncidentIssues twice — once inside findSameTraceMatchingIncident and again inside findLlmMatchingIncident. The comment in findSameTraceMatchingIncident even acknowledged this: "so no-trace intakes don't do them here only for the LLM path to immediately repeat them" — but only the no-trace early-exit was implemented; the duplicate fetch for trace-having issues was never eliminated.

Under the perProject keyed mutex, each additional second of per-transition latency serializes for every concurrent new issue in the same project, multiplying the wall-clock delay across any project experiencing an issue storm.

2. No per-incident row limit in loadLinkedIncidentIssues

loadLinkedIncidentIssues had no ORDER BY and no LIMIT. An incident with thousands of linked issues (observed at 14,000+ during a Railway backfill event) caused the query to return the entire result set on every call. Combined with the redundant calls above, that same enormous result set was fetched and transferred 2–3 times per transition, even though buildGroupingCandidate only uses the 5 most-recent issues per incident.

Remediation

apps/worker/src/incidents/intake.ts — findMatchingIncident now loads broad candidates + linked issues once and passes them into both the same-trace check (now inlined as a call to the pure findSameTraceIncidentMatch) and findLlmMatchingIncident. The findSameTraceMatchingIncident async helper is unchanged because it is still called from the recurrence path inside ensureIncidentForIssueAttempt.

apps/worker/src/issues/repository.ts — loadLinkedIncidentIssues now orders results by (incident_id ASC, last_seen DESC) and caps the returned rows to LINKED_ISSUES_PER_INCIDENT_CAP = 5 per incident in TypeScript. This matches the GROUPING_CANDIDATE_ISSUE_LIMIT = 5 in domain.ts and prevents a single high-issue-count incident from returning an unbounded result set to callers.

Expected Impact

  • Saves one findOpenIncidentCandidates + one loadLinkedIncidentIssues DB round-trip per new-issue transition that goes through the LLM path (the common case for new issues)
  • Prevents the pathological case where one incident with thousands of linked issues causes loadLinkedIncidentIssues to return 14,000+ rows on every call
  • Both changes compound: saves time per transition × number of concurrent new-issue transitions under the perProject mutex

Incident: c9757c47-db13-4139-9b81-674f321ed80d


Was this PR helpful? Leave feedback — goes straight to the Superlog team.

…ntake #SUPERLOG

Delivery-Id: 67d338ea005db7279e254af5fc2197d7009362a5322158c37de0ddda94b9d80e
Delivery-Base: main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant