Skip to content

fix: deduplicate jobs for same issue — prevent multiple pending jobs per (repo, issue_number) #149

Description

@telleroutlook

Problem

Multiple jobs can exist for the same (repo, issue_number) pair in non-terminal states. This happens when:

  1. A job is manually inserted with a different priority
  2. The scheduler re-enqueues a job that was blocked/failed but another still exists
  3. A discussion state job is promoted to pending while the original job also becomes pending

Observed today: job 46 and job 115 both targeted telleroutlook/claude-bot#27, causing one to be wasted.

Fix

1. Dedup on scheduler enqueue

Before inserting a new job, check if an active job already exists for the same (repo, issue_number):

existing = conn.execute("""
    SELECT id FROM jobs
    WHERE repo=? AND issue_number=?
      AND state NOT IN ('merged','blocked','max_retry_exceeded')
    LIMIT 1
""", (repo, issue_number)).fetchone()
if existing:
    log.debug("Skipping duplicate job for %s#%d (existing job %d)", repo, issue_number, existing['id'])
    return  # Don't insert duplicate

2. Dedup on manual insert

When claim_job() selects jobs, if multiple pending jobs exist for the same issue, block all but the lowest-id one:

# In claim_job(), after fetching the row:
dupes = conn.execute("""
    SELECT id FROM jobs
    WHERE repo=? AND issue_number=?
      AND state='pending'
      AND id != ?
""", (row['repo'], row['issue_number'], row['id'])).fetchall()
for dupe in dupes:
    conn.execute(
        "UPDATE jobs SET state='blocked', stage='duplicate_job' WHERE id=?",
        (dupe['id'],)
    )

3. Startup cleanup

On worker startup, run a dedup pass: for each (repo, issue_number) with multiple active jobs, keep the lowest-priority (most urgent) one and block the rest.

Acceptance criteria

  • Scheduler does not insert a new job if an active job already exists for the same issue
  • claim_job() auto-blocks duplicate pending jobs, keeping only the most urgent
  • Startup dedup pass in worker_loop() or open_db()
  • Blocked duplicates use stage duplicate_job for diagnosability
  • Test: inserting two jobs for the same issue results in only one being executed

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingclaude-hardHigh-capability (opus) model taskreliability

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions