You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I think that I am getting more comfortable with finding some nits in the workflow that I think we can update and fix. I would like to have these ready for v1.0; I think they would GREATLY reduce friction for all users.
Case metadata:
I don't think event_type should be a requirement for our case metadata. It should be up to the user to identify what their event type is. For example, separating a bunch of cases into different YAMLs.
I also think we can remove case_id_number as a requirement.
We should create a method to allow users to include any arbitrary sets of metadata in the case metadata, so that (if they want) they can add event_type, case_id_number, or whatever else (e.g. was_i_located_here_at_the_time or which_mode_of_transportation_did_this_impact)
Overarching workflow
Importing all of the modules is a nuisance. Instead of from extremeweatherbench import defaults, cases, metrics, derived, evaluate, I think something like import extremeweatherbench as ewb and having each as a distinct submodule would reduce unneeded verbosity. An example here then would be ewb.metrics.MeanAbsoluteError() or ewb.inputs.ERA5(). This change would also reduce risk of users unintentionally overriding modules with variable calls for things we're commonly interacting with, like cases or metrics.
Also, to build on 1, we would bury the ExtremeWeatherBench class in evaluate.py away from the user and instead have it directly callable, e.g. ewb.evaluation(...)
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I think that I am getting more comfortable with finding some nits in the workflow that I think we can update and fix. I would like to have these ready for v1.0; I think they would GREATLY reduce friction for all users.
Case metadata:
I don't think
event_typeshould be a requirement for our case metadata. It should be up to the user to identify what their event type is. For example, separating a bunch of cases into different YAMLs.I also think we can remove
case_id_numberas a requirement.We should create a method to allow users to include any arbitrary sets of metadata in the case metadata, so that (if they want) they can add
event_type,case_id_number, or whatever else (e.g.was_i_located_here_at_the_timeorwhich_mode_of_transportation_did_this_impact)Overarching workflow
Importing all of the modules is a nuisance. Instead of
from extremeweatherbench import defaults, cases, metrics, derived, evaluate, I think something likeimport extremeweatherbench as ewband having each as a distinct submodule would reduce unneeded verbosity. An example here then would beewb.metrics.MeanAbsoluteError()orewb.inputs.ERA5(). This change would also reduce risk of users unintentionally overriding modules with variable calls for things we're commonly interacting with, like cases or metrics.Also, to build on 1, we would bury the
ExtremeWeatherBenchclass inevaluate.pyaway from the user and instead have it directly callable, e.g.ewb.evaluation(...)I have already identified it in No simple way to build a list of targets or forecasts without needing metrics #117 but there should be a way to easily grab the subset target or forecast data without setting up a full blown evaluation.
I have a branch set up for 2 which should handle it.
Beta Was this translation helpful? Give feedback.
All reactions