Problem Statement
The global settings state is currently mutable and shared across the entire process. This can lead to configuration leakage in concurrent or multi-step workflows.
For example, temporarily modifying settings such as enable_cache, lm, or helper_lm for a specific task can unintentionally affect other tasks running at the same time. This becomes especially problematic in:
Parallel workloads
Test environments
Multi-step pipelines with heterogeneous model requirements
Proposed Solution
Introduce a scoped context manager API, e.g. lotus.settings.context(...), that:
Applies temporary overrides for settings (e.g., lm, rm, helper_lm, enable_cache)
Automatically restores the previous state after exiting the context
Safely supports nested contexts
Guarantees restoration even if exceptions occur
This provides a minimal, explicit, and Pythonic way to isolate configuration changes.
Use Cases
Running llm_as_judge evaluations without affecting global cache behavior
Temporarily switching to a cheaper or faster model for a specific operator in a pipeline
Isolating settings in tests to prevent cross-test contamination
Running parallel workloads where each worker requires different model or configuration settings
Alternative Solutions
Manual save/restore in user code
Error-prone and easy to forget, especially in the presence of exceptions
Thread-local settings
Helps in some concurrency scenarios but does not cover all execution models and is less explicit to users
Copying Settings objects per call
Requires larger API changes and introduces significant refactoring overhead
A context manager approach is the most lightweight, explicit, and backward-compatible solution.
Additional Context
This feature is fully additive and does not require breaking changes to existing settings.configure(...) usage.
It also aligns with the existing note that current settings are not thread-safe, and provides a clear migration path toward safer concurrent execution patterns.
Problem Statement
The global settings state is currently mutable and shared across the entire process. This can lead to configuration leakage in concurrent or multi-step workflows.
For example, temporarily modifying settings such as enable_cache, lm, or helper_lm for a specific task can unintentionally affect other tasks running at the same time. This becomes especially problematic in:
Parallel workloads
Test environments
Multi-step pipelines with heterogeneous model requirements
Proposed Solution
Introduce a scoped context manager API, e.g. lotus.settings.context(...), that:
Applies temporary overrides for settings (e.g., lm, rm, helper_lm, enable_cache)
Automatically restores the previous state after exiting the context
Safely supports nested contexts
Guarantees restoration even if exceptions occur
This provides a minimal, explicit, and Pythonic way to isolate configuration changes.
Use Cases
Running llm_as_judge evaluations without affecting global cache behavior
Temporarily switching to a cheaper or faster model for a specific operator in a pipeline
Isolating settings in tests to prevent cross-test contamination
Running parallel workloads where each worker requires different model or configuration settings
Alternative Solutions
Manual save/restore in user code
Error-prone and easy to forget, especially in the presence of exceptions
Thread-local settings
Helps in some concurrency scenarios but does not cover all execution models and is less explicit to users
Copying Settings objects per call
Requires larger API changes and introduces significant refactoring overhead
A context manager approach is the most lightweight, explicit, and backward-compatible solution.
Additional Context
This feature is fully additive and does not require breaking changes to existing settings.configure(...) usage.
It also aligns with the existing note that current settings are not thread-safe, and provides a clear migration path toward safer concurrent execution patterns.