chore(scale): add worker failure recovery and bounded retry semantics

[parthrohit22]

Parent: #210

Goal

Make worker death and retry behavior deterministic without orphaning analysis jobs.

Scope

• Detect worker loss or lease expiry.
• Restore jobs to a safe queued or terminal state.
• Bound retries and preserve the existing CANCELLED_CODE lifecycle.
• Record recovery events without logging source contents or secrets.

Acceptance criteria

• Worker failure does not leave an analysis job permanently stuck.
• Retry count and terminal failure are deterministic.
• Cancellation is not resurrected by recovery.
• Snapshots are not duplicated or partially committed.
• Tests cover crash, timeout, retry exhaustion, and cleanup.

[parthrohit22]

PARTHA is self-hosted only. On restart, jobs that were left stuck on running are already marked failed (#324).