Skip to content

fix: RITS/hosted_vllm pricing tolerance, AppWorld init lock, env-conf#231

Open
korenLazar wants to merge 1 commit into
mainfrom
fix/rits-compat-and-appworld-init-lock
Open

fix: RITS/hosted_vllm pricing tolerance, AppWorld init lock, env-conf#231
korenLazar wants to merge 1 commit into
mainfrom
fix/rits-compat-and-appworld-init-lock

Conversation

@korenLazar

Copy link
Copy Markdown
Collaborator

…igurable timeouts, child-process env forwarding

Small set of focused fixes uncovered while running CE-Manager + Mnemex sweeps on AppWorld with RITS-hosted models. Each fix is independently useful for any user running:

  • benchmarks with parallel sessions on AppWorld,
  • models that LiteLLM has no pricing for (e.g. hosted_vllm/, rits/),
  • subprocess (venv) runners with slow model-health endpoints,
  • subprocess runners that need PYTHONPATH or locale forwarded.

Bugfixes:

  • benchmarks/appworld/appworld_eval.py: serialise AppWorld(init) under a class-level threading.Lock. AppWorld's init clears a global cache dict that races under parallel workers, producing 'dictionary changed size during iteration'. New tests/benchmarks/test_appworld_session_init_lock.py.
  • integrations/litellm/trace_logger.py: guard against ctx is None or ctx.otel_context is None in start_span_context_from_kwargs. Avoids AttributeError when LiteLLM emits trace events outside an Exgentic context. Coverage in tests/observers/test_otel.py.
  • observers/handlers/results.py + utils/cost.py: tolerate models that litellm.cost_calculator has no pricing for (currently hosted_vllm/* and rits/*). Previously crashed the entire results pipeline with 'No pricing info found for model'; now returns zero-cost and logs a warning. Coverage in tests/utils/test_cost.py.

Configurability:

  • adapters/runners/venv.py: EXGENTIC_VENV_HEALTH_TIMEOUT and EXGENTIC_VENV_TRANSPORT_TIMEOUT override the previously-hardcoded defaults (360s / 600s), needed for slow inference endpoints.
  • agents/smolagents/base_instance.py: EXGENTIC_MODEL_HEALTH_TIMEOUT flows into check_model_accessible_sync.
  • adapters/runners/utils.py: forward PYTHONPATH, HOME, USER, LANG, LC_ALL, and CE_MANAGER* env vars to subprocess runners so optional packages and locale survive the venv hop. Coverage in tests/adapters/runners/test_utils.py.

…igurable timeouts, child-process env forwarding

Small set of focused fixes uncovered while running CE-Manager + Mnemex
sweeps on AppWorld with RITS-hosted models. Each fix is independently
useful for any user running:
- benchmarks with parallel sessions on AppWorld,
- models that LiteLLM has no pricing for (e.g. hosted_vllm/*, rits/*),
- subprocess (venv) runners with slow model-health endpoints,
- subprocess runners that need PYTHONPATH or locale forwarded.

Bugfixes:
- benchmarks/appworld/appworld_eval.py: serialise AppWorld(__init__)
  under a class-level threading.Lock. AppWorld's init clears a global
  cache dict that races under parallel workers, producing
  'dictionary changed size during iteration'. New
  tests/benchmarks/test_appworld_session_init_lock.py.
- integrations/litellm/trace_logger.py: guard against ctx is None or
  ctx.otel_context is None in start_span_context_from_kwargs. Avoids
  AttributeError when LiteLLM emits trace events outside an Exgentic
  context. Coverage in tests/observers/test_otel.py.
- observers/handlers/results.py + utils/cost.py: tolerate models that
  litellm.cost_calculator has no pricing for (currently hosted_vllm/*
  and rits/*). Previously crashed the entire results pipeline with
  'No pricing info found for model'; now returns zero-cost and logs
  a warning. Coverage in tests/utils/test_cost.py.

Configurability:
- adapters/runners/venv.py: EXGENTIC_VENV_HEALTH_TIMEOUT and
  EXGENTIC_VENV_TRANSPORT_TIMEOUT override the previously-hardcoded
  defaults (360s / 600s), needed for slow inference endpoints.
- agents/smolagents/base_instance.py: EXGENTIC_MODEL_HEALTH_TIMEOUT
  flows into check_model_accessible_sync.
- adapters/runners/_utils.py: forward PYTHONPATH, HOME, USER, LANG,
  LC_ALL, and CE_MANAGER_* env vars to subprocess runners so optional
  packages and locale survive the venv hop. Coverage in
  tests/adapters/runners/test_utils.py.
@korenLazar korenLazar requested a review from elronbandel June 2, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant