refactor: disable validation checks in main function and implement re…#4
Open
goUp9 wants to merge 4 commits into
Open
refactor: disable validation checks in main function and implement re…#4goUp9 wants to merge 4 commits into
goUp9 wants to merge 4 commits into
Conversation
added 4 commits
May 21, 2026 20:50
…-evaluation logic Commented out the validation checks in the main function of miner.py to streamline the execution flow. Introduced a new function, select_reevals, to handle the selection of re-evaluation candidates based on existing commitments. Updated the round processing logic to incorporate re-evaluations alongside challengers, ensuring a more efficient evaluation round. This change enhances the overall structure and clarity of the evaluation process.
Restored the validation checks in the main function of miner.py to ensure that all necessary conditions are met before proceeding. The checks now run the validation process and provide feedback on any failures, enhancing the robustness of the execution flow. This change improves the overall reliability of the mining process by enforcing validation before committing changes.
Added a new function `_merge_reeval_composite` to handle the weighted merging of re-evaluation composites into stored results, ensuring prompt-count-based averaging of scores. Updated the `process_round` function to incorporate this logic, allowing for more accurate composite score calculations during re-evaluations. Enhanced logging to provide insights into the merging process, including stored and fresh prompt counts. This change improves the evaluation accuracy and robustness of the system.
Added logic to track re-evaluation failures, introducing a maximum limit for failures before candidates are disqualified. Updated the `process_round` function to log re-evaluation load failures and modified the `select_reevals` function to skip candidates exceeding the failure threshold. Additionally, introduced a new file for persisting re-evaluation failure states. This enhancement improves the robustness of the evaluation process by managing re-evaluation attempts more effectively.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…-evaluation logic
Commented out the validation checks in the main function of miner.py to streamline the execution flow. Introduced a new function, select_reevals, to handle the selection of re-evaluation candidates based on existing commitments. Updated the round processing logic to incorporate re-evaluations alongside challengers, ensuring a more efficient evaluation round. This change enhances the overall structure and clarity of the evaluation process.