Summary
Consider adding more generation/sampling methods to LMOrchestrator, such as one that returns an iterator.
Motivation
Some ITS algorithms (e.g., Adaptive-Consistency, implemented in its_hub/core/algorithms/beta_self_consistency.py) dynamically choose the number of samples to generate from the model. Such algorithms could benefit from more flexible generation functions
Summary
Consider adding more generation/sampling methods to
LMOrchestrator, such as one that returns an iterator.Motivation
Some ITS algorithms (e.g., Adaptive-Consistency, implemented in
its_hub/core/algorithms/beta_self_consistency.py) dynamically choose the number of samples to generate from the model. Such algorithms could benefit from more flexible generation functions