Skip to content

fix(api): use Promise.allSettled for resilient multi-source fetching#826

Merged
koala73 merged 2 commits intokoala73:mainfrom
NewCoder3294:fix/promise-allsettled-resilience
Mar 5, 2026
Merged

fix(api): use Promise.allSettled for resilient multi-source fetching#826
koala73 merged 2 commits intokoala73:mainfrom
NewCoder3294:fix/promise-allsettled-resilience

Conversation

@NewCoder3294
Copy link
Contributor

Summary

Several API handlers used Promise.all() for concurrent data fetching from multiple upstream sources. If any single source failed (network timeout, upstream outage, etc.), Promise.all would reject immediately and all results from other successful sources would be discarded. This is especially problematic for multi-source aggregation endpoints where partial data is far more useful than no data.

This PR converts five Promise.all calls to Promise.allSettled, allowing each handler to return partial results when one source is unavailable:

  • unrest/v1/list-unrest-events.ts — ACLED + GDELT protest data. If one source fails, events from the other are still returned. Fallback: empty array per source.
  • economic/v1/get-fred-series.ts — FRED observations + metadata. If metadata fails, observations are still returned with default title/units/frequency. If observations fail, returns undefined (no usable data).
  • cyber/v1/list-cyber-threats.ts — Five independent threat intel sources (Feodo, URLhaus, C2Intel, OTX, AbuseIPDB). Any subset of sources succeeding yields partial results. Fallback: { ok: false, threats: [] } per source.
  • trade/v1/get-trade-flows.ts — WTO export + import data. If one direction fails, the other is still returned. Fallback: null per source.
  • trade/v1/get-trade-barriers.ts — Agricultural + non-agricultural tariff data from WTO. Either dataset can be returned independently. Fallback: null per source.

Intentionally NOT converted

  • military/v1/list-military-bases.ts — Two halves of an antimeridian bounding box split; both are required for correct spatial results.
  • military/v1/get-theater-posture.ts — Parallel cache writes, not data fetching.
  • supply-chain/v1/get-chokepoint-status.ts — Already uses .catch() handlers for resilience.

Behavior on rejection

Each rejected promise now triggers a console.warn identifying the failed source and its rejection reason, making upstream outages easy to diagnose in logs.

Test plan

  • Verify TypeScript compiles cleanly (tsc --noEmit passes)
  • Confirm unrest endpoint returns GDELT-only results when ACLED is down (and vice versa)
  • Confirm FRED series endpoint returns observations with default metadata fields when metadata fetch fails
  • Confirm cyber threats endpoint returns data from 4 sources when 1 source is down
  • Confirm trade flows endpoint returns import-only data when export fetch fails (and vice versa)
  • Confirm trade barriers endpoint returns non-agricultural data when agricultural tariff fetch fails (and vice versa)
  • Verify console.warn messages appear in server logs when a source is rejected

🤖 Generated with Claude Code

@vercel
Copy link

vercel bot commented Mar 2, 2026

@NewCoder3294 is attempting to deploy a commit to the Elie Team on Vercel.

A member of the Team first needs to authorize it.

@koala73 koala73 added Ready to Merge PR is mergeable, passes checks, and adds value High Value Meaningful contribution to the project labels Mar 3, 2026
Copy link
Owner

@koala73 koala73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution and the well-written PR description!

After tracing through the codebase, most of the changes here are no-ops — the source functions in the affected files already absorb their own errors internally and never reject. Here's the breakdown:

  • **** — all 5 source functions (, , etc.) have that returns . They never reject a promise. The new rejected branches + fallback variable are dead code.
  • **** — both fetchAcledProtests and fetchGdeltEvents wrap in try/catch returning []. Also dead code.
  • get-trade-flows.ts / get-trade-barriers.tswtoFetch has its own try/catch returning null. Also dead code.

The one genuinely useful fix is get-fred-series.ts — the two raw fetch() calls there are not individually wrapped, so a timeout on either propagates through Promise.all and discards the whole result. Promise.allSettled here correctly preserves observations even when metadata fails.

My suggestion: open a focused PR with just the get-fred-series.ts change. It's a clean, reviewable, impactful fix that stands on its own. The other 4 files could follow in a separate PR if/when those source functions are refactored to not catch internally — but right now they'd be adding complexity without changing behavior.

Smaller, focused PRs are much easier to review and merge quickly.

NewCoder3294 and others added 2 commits March 5, 2026 12:12
Replace Promise.all with Promise.allSettled in five API handlers so that
a single upstream failure no longer discards results from all other
sources. Each conversion falls back to an appropriate default (empty
array, null, or {ok:false}) and logs a console.warn identifying which
source failed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…estructuring

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@koala73 koala73 force-pushed the fix/promise-allsettled-resilience branch from 2c9e8ea to a2c243f Compare March 5, 2026 08:13
@koala73 koala73 merged commit 88ee556 into koala73:main Mar 5, 2026
1 check failed
aldoyh pushed a commit to aldoyh/worldmonitor that referenced this pull request Mar 6, 2026
…oala73#826)

* fix(api): use Promise.allSettled for resilient multi-source fetching

Replace Promise.all with Promise.allSettled in five API handlers so that
a single upstream failure no longer discards results from all other
sources. Each conversion falls back to an appropriate default (empty
array, null, or {ok:false}) and logs a console.warn identifying which
source failed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(cyber): resolve TS18048 possibly-undefined errors in allSettled destructuring

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

High Value Meaningful contribution to the project Ready to Merge PR is mergeable, passes checks, and adds value

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants