Skip to content

Fix job timeout - #351

Open
HeDo88TH wants to merge 3 commits into
masterfrom
fix-job-timeout
Open

Fix job timeout#351
HeDo88TH wants to merge 3 commits into
masterfrom
fix-job-timeout

Conversation

@HeDo88TH

@HeDo88TH HeDo88TH commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

As title says

@HeDo88TH HeDo88TH self-assigned this Aug 5, 2026
Copilot AI lite review requested due to automatic review settings August 5, 2026 01:31
@HeDo88TH HeDo88TH added the bug Something isn't working label Aug 5, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses long-running Hangfire jobs being re-dequeued (and effectively “timing out”) by making the MySQL storage invisibility timeout configurable, and adjusts DroneDB build wrappers to treat “build already in progress” as a benign condition.

Changes:

  • Add HangfireInvisibilityTimeoutHours to AppSettings + default configuration, and surface it in the configuration UI.
  • Apply the configured invisibility timeout to Hangfire MySQL storage options.
  • Change HangfireUtils behavior on DdbBuildInProgressException to log and skip instead of sleeping/retrying inline.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Registry.Web/Utilities/StartupExtenders.cs Configures Hangfire MySQL storage InvisibilityTimeout from settings.
Registry.Web/Utilities/HangfireUtils.cs Updates build wrappers’ handling of DdbBuildInProgressException.
Registry.Web/Services/Adapters/ConfigurationDataBuilder.cs Exposes the new invisibility-timeout setting in the Cron Jobs configuration section.
Registry.Web/Models/Configuration/AppSettings.cs Adds the new HangfireInvisibilityTimeoutHours setting with XML documentation and default.
Registry.Web/appsettings-default.json Adds the default HangfireInvisibilityTimeoutHours value under AppSettings.
Suppressed comments (1)

Registry.Web/Utilities/HangfireUtils.cs:92

  • Similar to BuildWrapper: when BuildPendingWrapper hits DdbBuildInProgressException it currently logs "skipping" but still continues to log "Done build pending", which can make the job look like it successfully processed pending builds even though it did no work.
        catch (DdbBuildInProgressException ex)
        {
            // Benign: the lock holder will process the pending builds. See BuildWrapper.
            writeLine($"Build lock currently held by another process ({ex.Message}); skipping");
        }

Comment thread Registry.Web/Utilities/HangfireUtils.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants