Skip to content

test(devin): assert which error the refused send raises and what the caller sees #5170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lidge-jun merged 1 commit into from
Sep 19, 2026
+13 −1
Merged

test(devin): assert which error the refused send raises and what the caller sees #5170

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion tests/adapters/adapter-inner-send-budget-wiring.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { createDevinAdapter, DEVIN_API_SERVER } from "../../src/adapters/devin";
import { setCachedCatalogForTests } from "../../src/adapters/devin/cloud-direct/catalog";
import { resetKiroThrottleStateForTests } from "../../src/adapters/kiro-retry";
import type { AdapterFetchContext } from "../../src/adapters/base";
import { SendBudgetExhaustedError } from "../../src/lib/upstream-retry";
import { encodeMessage } from "../../src/lib/eventstream-decoder";
import { createRequestExecutionBudget, type RequestExecutionBudgetPolicy } from "../../src/lib/request-execution-budget";
import type { AdapterEvent, OcxParsedRequest, OcxProviderConfig } from "../../src/types";
Expand Down Expand Up @@ -206,6 +207,7 @@ describe("Devin runTurn execution wiring", () => {
adapter: "devin", baseUrl: DEVIN_API_SERVER, apiKey,
} as unknown as OcxProviderConfig);

let refusal: unknown;
try {
await adapter.runTurn?.(
{
Expand All @@ -223,7 +225,7 @@ describe("Devin runTurn execution wiring", () => {
onPhysicalSend: send => { observed.push(send); },
},
event => events.push(event),
).catch(() => { /* the refusal escapes the adapter for the caller to map */ });
).catch((error: unknown) => { refusal = error; });
} finally {
setCachedCatalogForTests(null);
if (previousHome === undefined) delete process.env.OPENCODEX_HOME;
Expand All @@ -233,6 +235,11 @@ describe("Devin runTurn execution wiring", () => {
removeTreeWithRetry(home);
}

// The refusal escapes the adapter for the caller to map, and WHICH error escapes is the
// contract: a bare catch here passed even when the adapter threw something else entirely,
// which would have left the turn reported as an ordinary upstream failure instead of a
// budget refusal.
expect(refusal).toBeInstanceOf(SendBudgetExhaustedError);
expect(urls.filter(url => url.includes("GetChatMessage"))).toHaveLength(0);
expect(observed).toEqual([]);
expect(budget.used).toBe(0);
Expand Down
5 changes: 5 additions & 0 deletions tests/responses/responses-send-budget-counts.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,12 +217,17 @@ describe("upstream sends per logical request", () => {
// without proving the refusal was recorded against the turn that was refused.
expect(attempts).toHaveLength(1);
expect({
// Pinned, not merely reported in the failure message. The refusal reaches the client as
// an error code on a buffered FAILED response, so the outer HTTP status stays 200; a
// reader who assumes 429 here would be describing a transport this path never uses.
status: response.status,
adapter: attempts[0]?.adapter,
sendCount: attempts[0]?.sendCount,
chatCalls: urls.filter(url => url.includes("GetChatMessage")).length,
totalSends: totalSends(logCtx),
refused: body.includes("request_send_budget_exhausted"),
}, `status ${response.status}: ${body.slice(0, 240)}`).toEqual({
status: 200,
adapter: "devin",
sendCount: 0,
chatCalls: 0,
Expand Down
Loading