Skip to content

feat(launching): detect and surface post-spawn launch failures - #340

Open
bobtista wants to merge 7 commits into
developmentfrom
feat/post-spawn-failure-detection
Open

feat(launching): detect and surface post-spawn launch failures#340
bobtista wants to merge 7 commits into
developmentfrom
feat/post-spawn-failure-detection

Conversation

@bobtista

@bobtista bobtista commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Detect engine initialization failures both during startup and after a launch has been reported as running, preserving stderr evidence so users receive actionable errors.

Changes

  • Replace the fixed exit snapshot with bounded exit-or-settle detection and continuous stderr capture.
  • Recognize advisory [ggc] mount-failure sentinels and name affected archives.
  • Record late non-zero exits in the launch registry and surface them through the existing status, error, and notification channel.
  • Buffer unmatched exit events and synchronize buffering with real-PID registration to prevent the placeholder-registration race.
  • Apply exit events idempotently, including double delivery.
  • Mark requested termination before killing the process and classify it as a normal stop rather than a crash.

Testing

  • dotnet test GenHub/GenHub.sln -c Release — 1,461 tests passed.
  • Real native-client test collection passed.

Risks and rollback

The Windows system-modal crash dialog remains an engine-side limitation because the process does not exit while the dialog is open. A crash occurring immediately before a requested Stop may be classified as intentional termination; this tradeoff is documented. Reverting this PR restores the previous fixed-window behavior.

Related issues

Fixes #334

Greptile Summary

This PR expands launch-failure detection and reporting across the process manager, launch registry, and profile UI.

  • Replaces the fixed startup snapshot with a bounded exit-or-settle window and retained stderr capture.
  • Recognizes archive mount-failure sentinels and carries late non-zero exits into launch status and user notifications.
  • Buffers unmatched process exits across real-PID registration and applies duplicate events idempotently.
  • Distinguishes requested termination from unexpected process failure.
  • Adds unit and native-client coverage for startup, late-exit, buffering, and termination behavior.

Confidence Score: 4/5

The PR is not yet safe to merge because cancellation can orphan a spawned game, termination failures can suppress later crash reporting, and process-exit callbacks still mutate UI state off the Avalonia thread.

The current process-start path can return failure after spawning without tracking or terminating the process; the termination path leaves its classification marker behind when killing fails or is cancelled; and the exit callback updates observable UI state directly from the process event thread.

Files Needing Attention: GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs; GenHub/GenHub/Features/GameProfiles/ViewModels/GameProfileLauncherViewModel.cs

Important Files Changed

Filename Overview
GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs Adds bounded startup monitoring, continuous stderr retention, archive-sentinel extraction, and requested-termination classification.
GenHub/GenHub/Features/Launching/LaunchRegistry.cs Adds synchronized unmatched-exit buffering and idempotent application of process-exit state to registered launches.
GenHub/GenHub/Features/GameProfiles/ViewModels/GameProfileLauncherViewModel.cs Surfaces abnormal post-launch exits through profile state, status text, error text, and notifications.
GenHub/GenHub.Core/Models/Events/GameProcessExitedEventArgs.cs Extends process-exit events with stderr, archive, and requested-termination details plus centralized failure wording.
GenHub/GenHub.Core/Models/GameProfile/GameLaunchInfo.cs Records exit codes and retroactive failure reasons in launch-registry entries.

Sequence Diagram

sequenceDiagram
    participant UI as Profile Launcher UI
    participant Launcher as GameLauncher
    participant Registry as LaunchRegistry
    participant Manager as GameProcessManager
    participant Process as Game Process

    Launcher->>Registry: Register placeholder launch (PID -1)
    Launcher->>Manager: StartProcessAsync
    Manager->>Process: Spawn and capture stderr
    alt Process exits within detection window
        Process-->>Manager: Exit code and stderr
        Manager-->>Launcher: Failed startup result
    else Process survives detection window
        Manager-->>Launcher: Running process and real PID
        Launcher->>Registry: Register real PID
        alt Process exits after launch
            Process-->>Manager: Exit event
            Manager->>Registry: Record termination or late failure
            Manager->>UI: Clear running state and surface failure
        end
    end
Loading

Reviews (4): Last reviewed commit: "fix(launching): stop buffered exits reac..." | Re-trigger Greptile

Context used:

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@bobtista, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 51 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 181db8c3-6381-4e43-b17e-5ab910b5c0fc

📥 Commits

Reviewing files that changed from the base of the PR and between b3f5c4a and 31b2784.

📒 Files selected for processing (11)
  • GenHub/GenHub.Core/Constants/ProcessConstants.cs
  • GenHub/GenHub.Core/Constants/RetailArchiveConstants.cs
  • GenHub/GenHub.Core/Models/Events/GameProcessExitedEventArgs.cs
  • GenHub/GenHub.Core/Models/GameProfile/GameLaunchInfo.cs
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/GameProfiles/NativeClientLaunchIntegrationTests.cs
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/GameProfiles/PostSpawnFailureDetectionTests.cs
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/GameProfiles/ViewModels/GameProfileLauncherViewModelTests.cs
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/Launching/LaunchRegistryTests.cs
  • GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs
  • GenHub/GenHub/Features/GameProfiles/ViewModels/GameProfileLauncherViewModel.cs
  • GenHub/GenHub/Features/Launching/LaunchRegistry.cs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kilo-code-bot

kilo-code-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown

Code Review Summary

Status: 3 Issues Found | Recommendation: Address before merge

Incremental pass on commit 31b2784 (since f2eba92).
The LaunchRegistry change resolves the prior WARNING: retention is reduced from 30 s to 2 s, the exit-event lookup now matches only the live launch holding a PID, and a terminated-launch with that PID absorbs the event instead of re-buffering it. This stops a buffered exit from being drained against a later launch that recycled the same PID. The changed lines introduce no new issues.

Overview

Severity Count
CRITICAL 3
WARNING 0
SUGGESTION 0
Issue Details (click to expand)

CRITICAL

File Line Issue
GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs 521 Stale requested-termination marker — _requestedTerminations[processId] is set before process.Kill; the finally only releases the semaphore and never clears the marker, so a Kill failure or cancellation leaves it in place and later crashes are misclassified as intentional stops (verified still present on HEAD)
GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs 258 Cancellation orphans spawned process — token cancellation during the post-spawn wait leaves the already-started process running without management control (verified still present on HEAD)
GenHub/GenHub/Features/GameProfiles/ViewModels/GameProfileLauncherViewModel.cs 1610 Worker-thread UI state updates — UI-bound StatusMessage/ErrorMessage updated from the process-exit worker thread without dispatching to the Avalonia UI thread (verified still present on HEAD)
Files Reviewed (3 files)
  • GenHub/GenHub/Features/Launching/LaunchRegistry.cs — changed this pass; prior WARNING resolved, no new issues
  • GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs — 2 carried-forward CRITICAL (unchanged since prior pass)
  • GenHub/GenHub/Features/GameProfiles/ViewModels/GameProfileLauncherViewModel.cs — 1 carried-forward CRITICAL (unchanged since prior pass)

Fix these issues in Kilo Cloud

Previous Review Summaries (2 snapshots, latest commit f2eba92)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit f2eba92)

Status: 4 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 3
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

CRITICAL

File Line Issue
GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs 521 Stale requested-termination marker — when process.Kill fails or is cancelled after the marker is set, subsequent crashes are misclassified as intentional stops (reported by Greptile, re-verified on current HEAD)
GenHub/GenHub/Features/GameProfiles/ViewModels/GameProfileLauncherViewModel.cs 1610 Worker-thread UI state updates — UI-bound StatusMessage/ErrorMessage updated from the process-exit worker thread without dispatching to the Avalonia UI thread (reported by Greptile, re-verified on current HEAD)
GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs 258 Cancellation orphans spawned process — token cancellation during the post-spawn wait leaves the already-started process running without management control (reported by Greptile, re-verified on current HEAD)

WARNING

File Line Issue
GenHub/GenHub/Features/Launching/LaunchRegistry.cs 30 Pending-exit retention (30 s) is orders of magnitude larger than the millisecond race gap it buffers; a buffered exit for a recycled PID can be falsely applied to a later, unrelated launch, marking it terminated/failed
Files Reviewed (11 files)
  • GenHub/GenHub.Core/Constants/ProcessConstants.cs
  • GenHub/GenHub.Core/Constants/RetailArchiveConstants.cs
  • GenHub/GenHub.Core/Models/Events/GameProcessExitedEventArgs.cs
  • GenHub/GenHub.Core/Models/GameProfile/GameLaunchInfo.cs
  • GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs - 2 issues
  • GenHub/GenHub/Features/GameProfiles/ViewModels/GameProfileLauncherViewModel.cs - 1 issue
  • GenHub/GenHub/Features/Launching/LaunchRegistry.cs - 1 issue
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/GameProfiles/NativeClientLaunchIntegrationTests.cs
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/GameProfiles/PostSpawnFailureDetectionTests.cs
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/GameProfiles/ViewModels/GameProfileLauncherViewModelTests.cs
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/Launching/LaunchRegistryTests.cs

Note: The previous review SHA was rebased away, so this pass re-reviewed the full PR diff against current HEAD (f2eba92). The 3 CRITICAL findings were re-verified and remain open; the WARNING is newly reported. The PR's post-spawn failure detection, exit buffering, and requested-termination classification are otherwise well-structured and well-tested.

Fix these issues in Kilo Cloud

Previous review (commit 163343e)

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 3
WARNING 0
SUGGESTION 0
Issue Details (click to expand)

CRITICAL

File Line Issue
GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs 521 Stale requested-termination marker - when process.Kill fails or is cancelled after marker is set, subsequent crashes are misclassified as intentional stops
GenHub/GenHub/Features/GameProfiles/ViewModels/GameProfileLauncherViewModel.cs 1608-1610 Worker-thread UI state updates - updating UI-bound properties from process-exit worker thread without dispatching to Avalonia UI thread
GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs 258 Cancellation orphans spawned process - token cancellation during post-spawn wait leaves process running without management control
Files Reviewed (11 files)
  • GenHub/GenHub.Core/Constants/ProcessConstants.cs - Added detection window constant
  • GenHub/GenHub.Core/Constants/RetailArchiveConstants.cs - Added sentinel constants
  • GenHub/GenHub.Core/Models/Events/GameProcessExitedEventArgs.cs - Added failure detection support
  • GenHub/GenHub.Core/Models/GameProfile/GameLaunchInfo.cs - Added failure tracking properties
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/GameProfiles/PostSpawnFailureDetectionTests.cs - Added comprehensive tests
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/GameProfiles/ViewModels/GameProfileLauncherViewModelTests.cs - Added UI tests
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/Launching/LaunchRegistryTests.cs - Added race condition tests
  • GenHub/GenHub/Features/GameProfiles/Infrastructure/GameProcessManager.cs - 3 CRITICAL issues - Added failure detection logic
  • GenHub/GenHub/Features/GameProfiles/ViewModels/GameProfileLauncherViewModel.cs - 1 CRITICAL issue - Added UI failure handling
  • GenHub/GenHub/Features/Launching/LaunchRegistry.cs - Added exit buffering and failure recording
  • GenHub/GenHub.Tests/GenHub.Tests.Core/Features/GameProfiles/NativeClientLaunchIntegrationTests.cs - Updated test assertions

Note: All identified issues have already been reported by Greptile inline comments. No duplicate issues found. The PR makes significant improvements to post-spawn failure detection but has critical threading and error handling issues that need to be addressed.


Reviewed by glm-5.2 · Input: 27.8K · Output: 11.4K · Cached: 251.9K

@bobtista
bobtista force-pushed the feat/post-spawn-failure-detection branch from 163343e to f2eba92 Compare August 3, 2026 12:43
coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 3, 2026
@bobtista
bobtista changed the base branch from feat/native-launch to development August 3, 2026 12:44
@bobtista
bobtista dismissed coderabbitai[bot]’s stale review August 3, 2026 12:44

The base branch was changed.

@bobtista

bobtista commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto development and retargeted from feat/native-launch.

feat/native-launch was squash-merged as #332, so this PR was stacked on a branch that no longer exists in development's history. Retargeting alone would have produced a diff of 57 files, +1763/-3642 — one that deletes the work merged since #331 (#332, #337, #338, #339, #348, #349). That is the squash-orphaning failure #327 describes.

Instead the six commits unique to this branch were replayed onto development with git rebase --onto origin/development d1adeeb. No conflicts. The diff is now 11 files, +1297/-29 — this branch's own work and nothing else.

Verified after rebase: full core suite 1,553 passed, 0 failed.

No approvals existed, so nothing was dismissed by the force-push.

/// PIDs are recycled: an event held indefinitely could be applied to an unrelated
/// later launch that happened to receive the same PID.
/// </remarks>
private static readonly TimeSpan PendingExitRetention = TimeSpan.FromSeconds(30);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Pending-exit retention is far larger than the race it covers, so a buffered exit can be applied to a later, unrelated launch that recycles the PID.

PendingExitRetention is 30 s, but the gap it buffers (between StartProcessAsync returning and the launcher updating the placeholder entry with the real PID) is described above as milliseconds. Buffered exits are keyed by PID, and this PR explicitly targets Windows, where PIDs recycle. If a later launch is handed a PID that was recycled within the 30 s window, RegisterLaunchAsync drains the stale exit via TryRemove at line 118 and calls ApplyProcessExit, marking the unrelated (possibly still-running) launch as terminated/failed using the dead process exit code and stderr. A second consequence: a terminated launch is never removed from _activeLaunches, so its lingering PID makes OnProcessExited match the recycled-PID event against it (idempotency then drops it) instead of buffering it for the new launch. Tightening this to ~1-2 s still covers the millisecond race with a large margin while shrinking the recycling exposure by an order of magnitude.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant