Add ability to cache command line arguments #478
Conversation
This adds a feature which allows complex command line arguments to be written to a cache file and then re-used. Where an argument is specified in both the cache and on the current command line, the command line version takes priority. Cache arguments themselves are not sticky, so if --save-cache and --use-cache are set on the command line, they will get written to the cache but ignored when the cache is next loaded. This prevents the cache from being inadvertently overwritten with new arguments.
A problem with TestArgumentCaching::test_disabled seems to be tripping up the CI testing on github even though it works interactively. Try to fix this by removing the cache file before running the test. Also take the opportunity to add some missing docstrings to the test cases.
|
It looks like the CI problems may be a consequence of the earlier versions of python - and pyfakefs? - because the 3.11 tests have run through cleanly. I've converted the PR into draft while I try to narrow down the problem. |
This replaces the pyfakefs calls with tmp_path in the cache tests to prevent failures with older versions of python from tripping up the CI tests on github.
|
I was able to duplicate the problem interactively with python 3.9. Replacing the pyfakefs fixtures with pytest tmp_path has resolved the problem. Marking the PR as ready to go. |
|
This should probably go on after #476. It's going to conflict, so I'll need to update the branch once the other change is on. |
|
This new feature should also be added to the Fab basceclass, so the user experience across the various CLI is consistent. So, it should go in after #414 (which after all is the only high priority item and a release blocker, ready to be reviewed and as such must have priority). |
Some of the caching features were defined in the wrong places; the argument group was added in FabArgumentParser and the post-parse_args setup was entirely contained in the _parser_wrapper decorator. This prevents the class from being self-contained and makes it harder to reuse. This moves the methods into the _FabArgumentCache class. This also creates a CachingArgumentParser which is a simple combination of _FabArgumentCache and ArgumentParser. This provides a self-contained parser for use by infrastructure that does not use the fab CUI.
Add the CachingArgumentParser class to FabBase as a drop-in replacement for argparse.ArgumentParser. Also add some tests confirming the core caching functionality works as expected with FabBase.
|
@hiker, that's a good suggestion. Now that the I've added this as a drop-in replacement for the parser in |
The _FabArgumentCache class is a mixin and only works when added to a class which also inherits from argparse.ArgumentParser. Ignore type complaints related to missing attributes caused by references to methods defined elsewhere.
Looks nearly perfect to me: the only suggestion I have is to keep the type declaration to be |
|
Thanks @hiker, I'm glad you think this is useful. I'm going to pick this up again shortly, integrate your suggestions, and get the PR moving again. |
2 participants
This adds a feature which allows complex command line arguments to be written to a cache file and then re-used. Where an argument is specified in both the cache and on the current command line, the command line version takes priority.
Cache arguments themselves are not sticky, so if --save-cache and --use-cache are set on the command line, they will get written to the cache but ignored when the cache is next loaded. This prevents the cache from being inadvertently overwritten with new arguments.