Skip to content

fix(profile): bound pdftotext during boot to prevent startup deadlock#657

Open
RealDiligent wants to merge 4 commits into
GeniePod:mainfrom
RealDiligent:fix/critical-issue-profile-pdftotext-boot-timeout
Open

fix(profile): bound pdftotext during boot to prevent startup deadlock#657
RealDiligent wants to merge 4 commits into
GeniePod:mainfrom
RealDiligent:fix/critical-issue-profile-pdftotext-boot-timeout

Conversation

@RealDiligent

Copy link
Copy Markdown
Contributor

Closes #617

Summary

The highest-severity subprocess hang: synchronous unbounded pdftotext during load_profile ran inside with_shared_memory while holding the process-wide Memory mutex during boot — a pathological PDF could prevent genie-core from ever starting.

Root cause

profile/ingest.rs used blocking std::process::Command::output() with no deadline. main.rs invoked load_profile inside with_shared_memory, so the subprocess held the global memory lock for its entire (unbounded) duration before HTTP/voice started.

Fix

  • Add genie_common::subprocess::output_with_timeout (kill_on_drop + tokio::time::timeout).
  • Split PDF ingest: async extract_pdf_text (30s cap) runs without the memory lock; sync ingest_pdf_text writes facts under a short lock.
  • Make load_profile async and call it directly from main.rs.

Impact

Boot no longer wedges on a hung pdftotext. Other subprocess sites in #617 remain for follow-up PRs.

Risk / tradeoffs

  • Very large but valid PDFs that need >30s extraction are skipped with a warning instead of blocking boot.

Test plan

  • genie-common::subprocess unit tests (fast command succeeds; slow command times out).
  • extract_pdf_text returns None for missing files without panicking.
  • Full workspace cargo test — CI validation required (fork workflow approval).

Real Behavior Proof

Local: cargo test -p genie-common --lib subprocess:: — 2/2 passed. No Jetson hardware available.

@github-actions github-actions Bot added the bug Something isn't working label Jul 7, 2026

@matedev01 matedev01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — this is exactly the surgical split I asked for when closing #618 (closes #617). extract_pdf_text is now async so pdftotext runs outside the Memory mutex, and the genie_common::subprocess::run_with_timeout helper reaps the child via kill_on_drop on timeout. Scoped to the deadlock fix only (5 files). fmt clean; CI validates the subprocess tests + full suite.

@matedev01 matedev01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — the design is right (async extract_pdf_text so pdftotext runs outside the Memory mutex, with a hard-deadline helper). But CI fails to compile: output_with_timeout(cmd: Command, …) takes Command by value, and both call sites pass &mut Command.

  • crates/genie-core/src/profile/ingest.rs:27Command::new("pdftotext").args([…]) returns &mut Command.
  • crates/genie-common/src/subprocess.rs:72 (test) — Command::new("sleep").arg("60").

Bind first, then pass by value, e.g.:

let mut cmd = Command::new("pdftotext");
cmd.args(["-layout", &path.to_string_lossy(), "-"]);
let output = output_with_timeout(cmd, PDF_EXTRACT_TIMEOUT).await;

Same in the test. cargo clippy --workspace --all-targets -- -D warnings should pass before this can merge.

@RealDiligent RealDiligent force-pushed the fix/critical-issue-profile-pdftotext-boot-timeout branch from 6c9bd69 to 213be9f Compare July 12, 2026 18:16
RealDiligent and others added 4 commits July 13, 2026 02:20
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Unbounded subprocess calls across 12 files can hang forever — including one that deadlocks genie-core's entire boot sequence

2 participants