Refine prompts, schemas and expected outputs for figure checklist#10
Merged
Conversation
Bring HS_dev branch up to date with staging
Moved the _in_progress_ data checks to the checklist "data-checklist": - 'western-blot'matching-SD' - 'SD-mapping' - 'panel-data-replication-validation' Put the cache in gitignore (and removed some cached files from this commit...)
…ficance-level-defined' schema Added two full mansucripts from the workshop and generated the expected outputs and run the analysis after fixing the prompts with the fixes derrived from the workshop: - 10.1038_s44318-026-00735-x - 10.1038_s44318-026-00759-3 Overhauled the 'significance-level-defined' check by generating a prompt that does not include the boilerplate bla about panels but describes the background better. It still uses a .json schema but the schema has a new field now 'decision_and_explanation' which is added retrospectively to all the 'expected outputs' (empty, except for most recent run).
This commit includes all the expected output files after a full revision especially in -error-bars-defined -image-annotation-definition -replication reporting Some of these checks were modified so that there are new fields
…cklist Revised the prompts of all figure checklist checks, revamped the n-larger-two and replicates-defined into a single 'replication-reporting' check. Some have changes in their schema so the schema and the analysis is included as well.
…ist' Since some checks are revamped I retired the following checks: - micrograph-symbols-defined - n-larger-two - panel-label-detection - panelisation-and-classification - replicates-defined
- Generated evaluation manifests for the checks on the figure checklist - some more updates on prompts andexpected outputs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refined prompts for multiple figure checks.
Updated JSON schemas where needed
Curated expected outputs for benchmarking.
Regenerated evaluation/analysis results.
Retired a few checks and replaced them with better ones.
Merged the latest master into hs_dev to resolve conflicts.