IntegrityError Stacktrace encountered on busy systems

[philippfreyer]

I encountered this stacktrace on a busy system with multiple workers:

2026-02-11 06:03:30,123 - steady_queue - ERROR - error in AppExecutor: duplicate key value violates unique constraint "steady_queue_claimedexecution_job_id_key"
DETAIL:  Key (job_id)=(38887) already exists.
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/psycopg/cursor.py", line 117, in execute
    raise ex.with_traceback(None)
psycopg.errors.UniqueViolation: duplicate key value violates unique constraint "steady_queue_claimedexecution_job_id_key"
DETAIL:  Key (job_id)=(38887) already exists.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.14/site-packages/steady_queue/app_executor.py", line 33, in wrap_in_app_executor
    yield
  File "/usr/local/lib/python3.14/site-packages/steady_queue/processes/poller.py", line 45, in start_loop
    delay = self.poll()
  File "/usr/local/lib/python3.14/site-packages/steady_queue/processes/worker.py", line 32, in poll
    claimed_executions = self.claim_executions()
  File "/usr/local/lib/python3.14/site-packages/steady_queue/processes/worker.py", line 47, in claim_executions
    return ReadyExecution.objects.claim(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.queues, self.pool.idle_threads, self.process_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.14/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/steady_queue/models/ready_execution.py", line 26, in claim
    locked = relation.select_and_lock(process_id, limit)
  File "/usr/local/lib/python3.14/site-packages/steady_queue/models/ready_execution.py", line 38, in select_and_lock
    claimed = candidates.lock_candidates(process_id)
  File "/usr/local/lib/python3.14/site-packages/steady_queue/models/ready_execution.py", line 52, in lock_candidates
    ClaimedExecution.objects.claiming(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.values_list("job_id", flat=True), process_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.14/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/steady_queue/models/claimed_execution.py", line 21, in claiming
    self.bulk_create(claimed_executions)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/django/db/models/query.py", line 833, in bulk_create
    returned_columns = self._batched_insert(
        objs_without_pk,
    ...<4 lines>...
        unique_fields=unique_fields,
    )
  File "/usr/local/lib/python3.14/site-packages/django/db/models/query.py", line 1956, in _batched_insert
    self._insert(
    ~~~~~~~~~~~~^
        item,
        ^^^^^
    ...<5 lines>...
        returning_fields=returning_fields,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.14/site-packages/django/db/models/query.py", line 1918, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/django/db/models/sql/compiler.py", line 1925, in execute_sql
    cursor.execute(sql, params)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        sql, params, many=False, executor=self._execute
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.14/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.14/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/django/db/utils.py", line 94, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.14/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/psycopg/cursor.py", line 117, in execute
    raise ex.with_traceback(None)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "steady_queue_claimedexecution_job_id_key"
DETAIL:  Key (job_id)=(38887) already exists.

[knifecake]

Thanks for the report, @philippfreyer! I already have a working theory and am testing it in #28

Did you get this error by using the same database and queue configs as in #26?

[philippfreyer]

Hi @knifecake , yes, this is the same project and configuration- Thank you for your fixes!

[knifecake]

Hello! I had another look at this issue and added a couple of regression tests via #55. Still, i'm not 100% sure the fixes we introduced a while ago in #38 really did address this specific issue or not.

Feel free to reopen if it happens again and we can try to have another look! Logging's also better now :)