Currently, `ConfigGenerator` does the minimal setting with one IC config, (LoRA + FFT) finetuning and corrseponding evaluations. TODOs: - [ ] Extend this to different possible configurations For example: - IC experiments can be conducted on different prompt templates, consider using Dspy? - Zero-shot, few-shot - Fine tuning experiments can be extended to different PEFT techniques - For PEFT fine tuning, different hyperparams can be tried - [ ] How to extend the Train and Eval Configurations? Maybe use Hydra and hierarchical configurations for this?
Currently,
ConfigGeneratordoes the minimal setting with one IC config, (LoRA + FFT) finetuning and corrseponding evaluations.TODOs:
For example:
Maybe use Hydra and hierarchical configurations for this?