Skip to content

Bound external catalog cross-match with a hard timeout - #41

Merged
eagnespuerto merged 1 commit into
mainfrom
fix/crossmatch-timeout
Oct 10, 2026
Merged

eagnespuerto merged 1 commit into
mainfrom
fix/crossmatch-timeout

Conversation

@eagnespuerto

Copy link
Copy Markdown
Owner

Problem

Fetch & vet froze at "External catalog cross-match" and later ended with "stream timed out".

  • Gaia.launch_job_async polls the ESA job queue with no time limit.
  • The SIMBAD query in astroquery ≥0.4.8 runs through pyvo, which ignores TIMEOUT.
  • Cross-match sends no progress events while it runs, so the SSE stream's 10-minute no-event check gives up.

Fix (backend/app/pipeline.py, mirrored in pios/vetstar_pi/pipeline.py)

  • Gaia, SIMBAD and NASA Exoplanet Archive now run at the same time in a shared thread pool, with a 30 s total budget (CROSSMATCH_TIMEOUT_S). The exoplanets.org fallback gets the same budget.
  • A source that runs late becomes {"error": "timed out after 30s"}, and the verdict is left as it was.
  • Gaia now uses the synchronous launch_job. The query is a single-row cone search.
  • The Gaia and SIMBAD queries moved into their own functions: _query_gaia_dr3 and _query_simbad.

Testing

  • Stubbed a Gaia query that hangs for 60 s. With a 2 s budget, cross-match returned in 2.1 s with the SIMBAD result kept.

🤖 Generated with Claude Code

https://claude.ai/code/session_0117Lzq78TmMPd1FXyxq2SXF


Generated by Claude Code

Gaia launch_job_async polls the ESA job queue with no upper bound and the
pyvo-backed SIMBAD query ignores TIMEOUT, so a slow service froze the
job until the SSE stream's 10-minute safety valve fired. Run the catalog
queries concurrently under a 30 s wall-clock budget, switch Gaia to a
synchronous job, and report late sources as errors so the verdict stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117Lzq78TmMPd1FXyxq2SXF

Copy link
Copy Markdown
Owner Author

Backend tests: 83 passed, 1 failed. The failure, test_microlensing_coverage.py::test_all_known_windows_are_monotonic_in_start, also happens on main without this PR. It's in microlensing coverage code that this diff doesn't change, so I've left it out of this PR.


Generated by Claude Code

@eagnespuerto
eagnespuerto marked this pull request as ready for review October 10, 2026 10:05
@eagnespuerto
eagnespuerto merged commit e16c54d into main Oct 10, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants