Redesign config validation#412
Conversation
swpenninga
added
the
data format
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
| (consistent with :class:`zea.Parameters`). | ||
| """ | ||
|
|
||
| xlims: Any = None |
There was a problem hiding this comment.
ParametersConfig is now really defined in here, while we can also inherit that in some way from ProbeSpec + ConfigSpec. Kind of what is being done in the zea.Parameters class now already. That way there is a signal source of truth that defines which parameters are in the spec.
| import sys | ||
| from pathlib import Path | ||
|
|
||
| from schema import And, Optional, Or, Schema |
There was a problem hiding this comment.
TODO: we can probably remove the schema package from dep list as well in our toml
- Strip DataConfig to path/local/indices/user; remove dtype, dynamic_range, resolution, apodization, to_dtype, input_range, output_range - Consolidate dataset_folder + file_path into a single path field (supports hf://) - Rename frame_no → indices in DataConfig and all YAML configs - ParametersConfig becomes a pure open pass-through (no predefined fields) - Remove PlotConfig; keep DataConfig for path management in setup_zea - Delete zea/interface.py and tests/test_interface.py (Interface class was outdated) - Replace schema library dependency with inline _validate_convert_config() - Rewrite config.rst as a hand-authored paradigm doc with explicit key reference tables - Redesign parameters_doc.py (no RST generation, workspace path fix) - Fix docs build: remove interface refs from __init__.py, autosummary, cli.rst - Add Config() TypeError when a path string is passed instead of a dict
2 participants
Regarding #407 , #408 had broken through merge, new PR.
Redesign Config validation as dataclass-based Specs
Replaces the
schema-library validation layer with dataclass-based Specs, aligning Config validation with theProbeSpec/ScanSpecpattern used elsewhere in the codebase. Theparameterssection is now open/extensible by design rather than via the interimignore_extra_keys=Trueworkaround.What changed
zea/internal/config/validation.py— full rewrite:ConfigSpecbase class (dataclass +__post_init__validation +from_dict/to_dict+ introspection helpersfield_names/required_fields/all_field_paths), mirroring theSpecstyle but for config values rather than numpy arrays.DataConfig,PlotConfig,PipelineConfig,ParametersConfig, and top-levelConfigSchema, with every field/default/validator ported 1:1.enum,regex,any_of,optional, numeric/range checks) replacingOr/And/Regex.parameterssection and the top-level config are open by design (ALLOW_EXTRA = True) — arbitrary keys are stored and re-emitted unchanged, mirroringzea.Parameters._custom_params. This replaces theignore_extra_keys=Trueworkaround, which is gone.postprocess_schemaremoved (no consumers).validate_config(dict) -> dictentry point.zea/config.py—check_confignow callsvalidate_configinstead ofconfig_schema.validate.zea/internal/config/create.py— interactive creator rewritten to traverse the specs via introspection; input coerced withyaml.safe_load+ validators. Also fixed its already-brokencheck_configimport (now fromzea.config).docs/source/parameters_doc.py—flatten_schema_keysreplaced byConfigSchema.all_field_paths(); deprecatedscanalias tolerated via aDEPRECATED_KEYSset.Tests — new
tests/test_config_validation.py(defaults, required-field errors, enum/range/regex rejection, valid devices, parameters & top-level pass-through, idempotency,scan→parametersmigration, freeze behavior);tests/test_configs.pyupdated to drop theschemaimport.Note on the
schemadependencyschema >=0.7is left inpyproject.toml: it's still used byzea/data/convert/verasonics.py(unrelated to config), so removing it isn't safe. Nothing underzea/,docs/, ortests/importsschemafor config purposes anymore.