diff --git a/CONFIGURATION.md b/CONFIGURATION.md
index 2ca67350..2d9d4f62 100644
--- a/CONFIGURATION.md
+++ b/CONFIGURATION.md
@@ -257,6 +257,7 @@ The project contains predefined configs of application and predefined tools
| deployment | Yes | Object | The DIAL deployment configuration. See [Deployment configuration](#deployment-configuration) | - | - |
| system_prompt | Yes | Object | The configuration for the system prompt. See [System prompt configuration](#system-prompt-configuration) | - | - |
| max_iterations | No | Integer | The max count of orchestrator(agent) operations. -1 value for infinite | Integer | 15 |
+| tool_discovery | No | Object | `[Preview]` Dynamic tool discovery configuration. See [Tool discovery configuration](#tool-discovery-configuration) | - | `null` |
#### Deployment configuration
@@ -331,6 +332,39 @@ Custom system prompt:
+#### Tool discovery configuration
+
+`[Preview]` Requires `ENABLE_PREVIEW_FEATURES=true`. When enabled, toolsets withheld from the initial LLM payload
+(see the per-toolset `deferred` field in [Tool sets configuration](#tool-sets-configuration)) are surfaced on demand
+via the `internal_tool_search` meta-tool (referred to as "tool search" below), which routes the query to the matching
+tool schemas through an isolated LLM call.
+
+| Field | Required | Type | Description | Available Values | Default Value |
+|------------------------|----------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------|---------------|
+| enabled | No | Boolean | Enable dynamic tool discovery. When `true`, toolsets with `deferred: true` are withheld from the initial LLM payload and surfaced via the `internal_tool_search` meta-tool. | - | `false` |
+| service_model | No | String | DIAL deployment used for the anonymous routing call inside `internal_tool_search`. Falls back to the orchestrator's own deployment when omitted. | - | - |
+| min_tools_for_deferral | No | Integer | Minimum number of tools in a toolset for deferral to apply. Toolsets smaller than this threshold are promoted to eager loading even when `deferred: true`. Deployment-wide default set by `MIN_TOOLS_FOR_DEFERRAL`. | - | `10` |
+
+
+Tool discovery configuration JSON sample
+
+```json
+{
+ "orchestrator": {
+ "deployment": { "name": "gpt-4o" },
+ "tool_discovery": {
+ "enabled": true,
+ "service_model": "gpt-4o-mini",
+ "min_tools_for_deferral": 5
+ }
+ }
+}
+```
+
+
+
+See [Dynamic Tool Discovery design doc](docs/designs/dynamic_tool_discovery.md) for the full behavioral reference.
+
### Contexts configuration
| Field | Required | Type | Description | Available Values | Default Value |
@@ -471,6 +505,12 @@ SSRF envelope, deployment dispatch table, error messages and agent retry behavio
### Tool sets configuration
+Most toolset types also accept a `deferred` field (Boolean, default `true`): `[Preview]` when true or unset, and
+[Tool discovery configuration](#tool-discovery-configuration) is enabled, the toolset's tool schemas are withheld
+from the initial LLM payload and discovered on demand via the `internal_tool_search` meta-tool. Set to `false` to
+keep a specific toolset always eager. Currently honored by REST API, MCP, and Internal toolsets; DIAL deployment and
+DIAL app toolsets accept the field but do not yet act on it.
+
#### RestApiToolSet Configuration
| Field | Required | Type | Description | Default Value |
diff --git a/README.md b/README.md
index c267594c..9ade1bfb 100644
--- a/README.md
+++ b/README.md
@@ -91,6 +91,59 @@ Key fields:
See [Config-Driven Hooks design doc](docs/designs/config_driven_hooks.md) for the full reference.
+### Dynamic Tool Discovery `[Preview]`
+
+Dynamic tool discovery defers large toolsets from the initial LLM payload and surfaces them on demand via the `internal_tool_search` meta-tool (referred to as "tool search" below). The orchestrator calls `internal_tool_search` with a natural-language query when it needs a tool it hasn't seen yet; a lightweight anonymous LLM routing call selects the relevant tool schemas and injects them into the next iteration.
+
+Enable with `ENABLE_PREVIEW_FEATURES=true`, then add `orchestrator.tool_discovery` to the app manifest:
+
+```json
+{
+ "orchestrator": {
+ "deployment": { "deployment_id": "gpt-4o" },
+ "tool_discovery": {
+ "enabled": true,
+ "service_model": "gpt-4o-mini",
+ "min_tools_for_deferral": 5
+ }
+ },
+ "tool_sets": [
+ {
+ "name": "my-mcp-server",
+ "type": "mcp",
+ "mcp_server_info": {
+ "url": "http://localhost:8003/mcp",
+ "protocol": "streamable_http"
+ }
+ }
+ ]
+}
+```
+
+Toolsets are deferred by default — omitting `deferred` or setting it to `true` both defer the toolset. To keep a specific toolset always eager, set `"deferred": false` on that toolset:
+
+```json
+{
+ "name": "always-eager-toolset",
+ "type": "rest_api",
+ "deferred": false,
+ "open_api": { "url": "https://api.example.com/openapi.json" }
+}
+```
+
+Key fields:
+
+| Field | Default | Description |
+|---|---------|---|
+| `orchestrator.tool_discovery.enabled` | `false` | Activates dynamic discovery for this app. Must be `true` for deferral to take effect. |
+| `orchestrator.tool_discovery.service_model` | — | DIAL deployment used for the anonymous routing call inside `internal_tool_search`. Falls back to the orchestrator's own deployment when omitted. |
+| `orchestrator.tool_discovery.min_tools_for_deferral` | `10` | Minimum number of tools in a toolset for deferral to apply. Toolsets smaller than this threshold are promoted to eager loading even when `deferred: true`. |
+| `.deferred` | `true` | Per-toolset opt-out. Set to `false` to force a specific toolset into the initial payload regardless of `tool_discovery.enabled`. |
+
+The `MIN_TOOLS_FOR_DEFERRAL` environment variable sets the deployment-wide default for `min_tools_for_deferral`; individual apps can override it in their manifest.
+
+See [Tool discovery configuration](./CONFIGURATION.md#tool-discovery-configuration) for the full field reference and the [Dynamic Tool Discovery design doc](docs/designs/dynamic_tool_discovery.md) for the behavioral design.
+
### Forwarding headers
Incoming request headers whose names start with `X-` (case-insensitive) are automatically forwarded to all outbound
@@ -127,67 +180,69 @@ Controls which tool-execution stages are surfaced in the DIAL UI for each app. S
### Environment Variables
-| Variable | Default | Required | Description |
-|--------------------------------------------|----------------------------|----------|--------------------------------------------------------------------------------------------------------------|
-| **DIAL Core** | | | |
-| `DIAL_URL` | — | Yes | URL of the DIAL Core API |
-| `DIAL_API_VERSION` | `2025-01-01-preview` | No | API version for DIAL Core API |
+| Variable | Default | Required | Description |
+|--------------------------------------------|-----------------------------------------------------------------|----------|----------------------------------------------------------------------------------------------------------------|
+| **DIAL Core** | | | |
+| `DIAL_URL` | — | Yes | URL of the DIAL Core API |
+| `DIAL_API_VERSION` | `2025-01-01-preview` | No | API version for DIAL Core API |
| `APP_SCHEMA_ID` | `https://mydial.epam.com/custom_application_schemas/quickapps2` | No | Full application type schema `$id` emitted in the generated app schema. When unset, the built-in default is used. |
-| **Proxy** | | | |
-| `PROXY_LANGUAGE_HEADER` | `accept-language` | No | Name of the incoming HTTP request header that carries the locale for UI display (stage name localization). Override when a reverse proxy rewrites the standard `Accept-Language` header before forwarding the request. |
-| **Logging** | | | |
-| `DIAL_SDK_LOG_FORMAT` | `text` | No | Console log output format: `text` (human-readable) or `json` (escape-safe, one record per line). See [docs/logging.md](docs/logging.md). |
-| `DIAL_SDK_TEXT_LOG_FORMAT` | [see docs/logging.md](docs/logging.md) | No | Custom `%`-style format string for `text` output. Unset (default) keeps the built-in format with the conditional OTEL trace block. |
-| `DIAL_SDK_JSON_LOG_FORMAT` | [see docs/logging.md](docs/logging.md) | No | Custom template for `json` output — a JSON document whose string leaves are `%`-style format strings, values escaped via `json.dumps`. |
-| `LOG_LEVEL` | `INFO` | No | Root logger level (all loggers except quickapp) |
-| `QUICKAPP_LOG_LEVEL` | `INFO` | No | Log level for quickapp loggers |
-| `LOG_PAYLOADS` | `false` | No | Emit payload content (message bodies, tool-call arguments, tool/LLM response bodies) at DEBUG. When `false`, no payload content is logged at **any** level and the payload-capable third-party loggers (`openai`/`httpx`/`httpcore`) are capped at INFO. **Local development only** — see [Payload Logging](#payload-logging). |
-| `LOG_PAYLOADS_MAX_LENGTH` | `2000` | No | Per-field character cap applied to each payload value when `LOG_PAYLOADS=true`; longer values are truncated. Inert when `LOG_PAYLOADS=false`. |
-| **Agent** | | | |
-| `DEFAULT_AGENT_MAX_ITERATIONS` | `15` | No | Maximum number of orchestrator iterations (`-1` for infinite) |
-| `DEFAULT_ORCHESTRATOR_DEPLOYMENT_ID` | — | No | Default DIAL deployment id used as the orchestrator model when a QuickApp manifest omits `orchestrator.deployment`. Also surfaces as the JSON-schema `default` for that field so DIAL Core can pre-fill new manifests. Apps can override per-app. |
-| `SHOW_USAGE_STATISTICS` | `false` | No | Include usage statistics in chat completion stream |
-| `SHOW_EXECUTION_TIME_STAGE` | `false` | No | Show execution time stage in the UI |
-| **Python Interpreter** | | | |
-| `PY_INTERPRETER_LOCAL_RUN` | `false` | No | Run PyInterpreter locally instead of via DIAL Core API |
-| `PY_INTERPRETER_URL` | *(falls back to DIAL_URL)* | No | URL of the PyInterpreter service |
-| `PY_INTERPRETER_API_KEY` | — | No | API key for local-run PyInterpreter |
-| `PY_INTERPRETER_DEFAULT_SESSION_ID` | — | No | Default session ID for the PyInterpreter |
-| `PY_INTERPRETER_CLIENT_MAX_RETRIES` | `3` | No | Max retries for PyInterpreter client requests |
-| **Tool Timeouts** | | | |
-| `DEFAULT_TOOL_TIMEOUT_SECONDS` | `300.0` | No | Deployment-wide default timeout (seconds, `0 < x ≤ 3600`) applied to every tool call (deployment, REST API, MCP, Python interpreter). Apps can override per-app via `tool_defaults.timeout_seconds`. |
-| `DEFAULT_FILE_LOADING_SIZE_LIMIT` | `10485760` | No | Deployment-wide default maximum size (in bytes) for files the agent downloads. Apps can override per-app via `features.file_loading.size_limit`. |
-| **Stage Display** | | | |
-| `DEFAULT_STAGE_DISPLAY_LEVEL` | — | No | Deployment-wide override for stage visibility threshold (`none`, `error`, `info`, `debug`; case-insensitive). When set, wins over every app's `features.stage_display.level`. Unset (default) defers to the per-app config, which defaults to `info`. |
-| **DIAL Files — Tool-Response Offload** | | | |
-| `TOOL_CALL_RESULT_OFFLOAD__ENABLED_BY_DEFAULT` | `true` | No | Default value of the per-app `enabled` flag (`features.dial_files.tool_call_result_offload.enabled`). Apps override per-app; `enabled: false` disables offload for that app. |
-| `TOOL_CALL_RESULT_OFFLOAD__SIZE_THRESHOLD` | `40000` | No | Default byte threshold above which a tool-call response is offloaded to a DIAL file. Apps override per-app via `features.dial_files.tool_call_result_offload.size_threshold`. |
-| `TOOL_CALL_RESULT_OFFLOAD__EXCLUDED_TOOLS` | `[]` | No | Default JSON list of **additional** tool names exempt from offloading. The read-back tools (`internal_file_read_lines`, `internal_file_search`) are always excluded regardless of this value, so a large read-back slice is never re-offloaded. Apps add more per-app via `features.dial_files.tool_call_result_offload.excluded_tools`. |
-| **External URL Egress** | | | |
-| `EXTERNAL_URL_FETCH_ENABLED` | `false` | No | Admin cap on fetching external (non-DIAL) URLs. When `false` (default), no app may fetch external URLs regardless of its manifest; the deployment-handoff branch (deployments with `features.url_attachments`) is unaffected. Apps can opt out per-app via `features.external_url_fetch.enabled=false` even when the admin allows. |
-| `EXTERNAL_URL_FETCH_HOST_ALLOWLIST` | — | No | Comma-separated allowlist of host patterns for external URL fetches. Unset (default) means no admin-level host restriction. Patterns: exact host (`example.com`) or `*.example.com` for any subdomain. Re-checked on every redirect hop. Per-app `features.external_url_fetch.host_allowlist` narrows further (intersection) but never expands. |
-| `EXTERNAL_URL_FETCH_MAX_REDIRECTS` | `5` | No | Maximum HTTP redirects on external URL fetches. Each hop is SSRF-checked. Hard ceiling 10. |
-| `EXTERNAL_URL_FETCH_CONNECT_TIMEOUT_SECONDS` | `5.0` | No | TCP connect timeout (seconds) for external URL fetches. Read/write/pool timeouts use the resolved tool timeout. |
-| **Skills** | | | |
-| `DIAL_SKILLS_FILE_MAX_BYTES` | `262144` | No | Cap on a single file read from a DIAL skill resource, `SKILL.md` included. Must exceed the largest manifest you expect: an over-cap manifest drops the skill. See [docs/skills.md](docs/skills.md). |
-| `DIAL_SKILLS_MAX_FILES` | `200` | No | Maximum bundled files advertised to the agent per DIAL skill resource; beyond it the listing is truncated |
-| `DIAL_SKILLS_LISTING_MAX_PAGES` | `10` | No | Maximum file-listing pages followed per DIAL skill resource, bounding a server-supplied cursor |
-| `SKILL_INVOCATION_MAX_SKILLS` | `10` | No | Maximum distinct skills a user may have invoked from the messages of one conversation (`custom_content.skills`), counted newest first. Each one adds a `` block to the system prompt and one DIAL Core fetch per turn; beyond the cap the oldest picks stop being registered. Preview-gated. See [docs/skills.md](docs/skills.md). |
-| **Feature Gating** | | | |
-| `ENABLE_PREVIEW_FEATURES` | `false` | No | Enable preview features across the deployment (schema visibility + runtime activation) |
-| **Templates** | | | |
-| `PREDEFINED_EXTRA_PATHS` | — | No | JSON list of directories layered on top of built-in predefined content (later entries override earlier ones) |
-| `CONFIG_PROMPT_MAPPING` | *(built-in mapping)* | No | JSON mapping of predefined system prompts to DIAL Core deployments |
-| **Observability** | | | |
-| `OTEL_SERVICE_NAME` | `quickapps` | No | Service name stamped on all exported telemetry (traces, metrics, logs) |
-| `OTEL_TRACES_EXPORTER` | — | No | Set to `otlp` to enable tracing and export spans over OTLP/gRPC. Instruments the FastAPI server and outgoing HTTP clients (`httpx`, `requests`, `aiohttp`, `urllib`) and stamps trace context onto log records — see [docs/logging.md](docs/logging.md). |
-| `OTEL_METRICS_EXPORTER` | — | No | Comma-separated metric exporters: `otlp` (push over OTLP/gRPC) and/or `prometheus` (serve a scrape endpoint). Enables FastAPI and system/process metrics. |
-| `OTEL_LOGS_EXPORTER` | — | No | Set to `otlp` to export log records (INFO and above) over OTLP/gRPC alongside console output — see [docs/logging.md](docs/logging.md). |
-| `OTEL_EXPORTER_OTLP_ENDPOINT` | `http://localhost:4317` | No | OTLP/gRPC collector endpoint shared by trace, metric, and log export. One of the [standard OpenTelemetry SDK variables](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/), which the underlying exporters honor as usual (per-signal endpoints, headers, timeouts, resource attributes, …). |
-| `OTEL_EXPORTER_PROMETHEUS_PORT` | `9464` | No | Port of the Prometheus scrape endpoint (effective only with `prometheus` in `OTEL_METRICS_EXPORTER`) |
-| **Scripts & Tests** | | | |
-| `REMOTE_DIAL_URL` | — | No | URL of the remote DIAL Core, used only by `generate_dial_config` script and e2e/integration tests |
-| `REMOTE_DIAL_API_KEY` | — | No | API key of the remote DIAL Core, used only by `generate_dial_config` script and e2e/integration tests |
+| **Proxy** | | | |
+| `PROXY_LANGUAGE_HEADER` | `accept-language` | No | Name of the incoming HTTP request header that carries the locale for UI display (stage name localization). Override when a reverse proxy rewrites the standard `Accept-Language` header before forwarding the request. |
+| **Logging** | | | |
+| `DIAL_SDK_LOG_FORMAT` | `text` | No | Console log output format: `text` (human-readable) or `json` (escape-safe, one record per line). See [docs/logging.md](docs/logging.md). |
+| `DIAL_SDK_TEXT_LOG_FORMAT` | [see docs/logging.md](docs/logging.md) | No | Custom `%`-style format string for `text` output. Unset (default) keeps the built-in format with the conditional OTEL trace block. |
+| `DIAL_SDK_JSON_LOG_FORMAT` | [see docs/logging.md](docs/logging.md) | No | Custom template for `json` output — a JSON document whose string leaves are `%`-style format strings, values escaped via `json.dumps`. |
+| `LOG_LEVEL` | `INFO` | No | Root logger level (all loggers except quickapp) |
+| `QUICKAPP_LOG_LEVEL` | `INFO` | No | Log level for quickapp loggers |
+| `LOG_PAYLOADS` | `false` | No | Emit payload content (message bodies, tool-call arguments, tool/LLM response bodies) at DEBUG. When `false`, no payload content is logged at **any** level and the payload-capable third-party loggers (`openai`/`httpx`/`httpcore`) are capped at INFO. **Local development only** — see [Payload Logging](#payload-logging). |
+| `LOG_PAYLOADS_MAX_LENGTH` | `2000` | No | Per-field character cap applied to each payload value when `LOG_PAYLOADS=true`; longer values are truncated. Inert when `LOG_PAYLOADS=false`. |
+| **Agent** | | | |
+| `DEFAULT_AGENT_MAX_ITERATIONS` | `15` | No | Maximum number of orchestrator iterations (`-1` for infinite) |
+| `DEFAULT_ORCHESTRATOR_DEPLOYMENT_ID` | — | No | Default DIAL deployment id used as the orchestrator model when a QuickApp manifest omits `orchestrator.deployment`. Also surfaces as the JSON-schema `default` for that field so DIAL Core can pre-fill new manifests. Apps can override per-app. |
+| `SHOW_USAGE_STATISTICS` | `false` | No | Include usage statistics in chat completion stream |
+| `SHOW_EXECUTION_TIME_STAGE` | `false` | No | Show execution time stage in the UI |
+| **Python Interpreter** | | | |
+| `PY_INTERPRETER_LOCAL_RUN` | `false` | No | Run PyInterpreter locally instead of via DIAL Core API |
+| `PY_INTERPRETER_URL` | *(falls back to DIAL_URL)* | No | URL of the PyInterpreter service |
+| `PY_INTERPRETER_API_KEY` | — | No | API key for local-run PyInterpreter |
+| `PY_INTERPRETER_DEFAULT_SESSION_ID` | — | No | Default session ID for the PyInterpreter |
+| `PY_INTERPRETER_CLIENT_MAX_RETRIES` | `3` | No | Max retries for PyInterpreter client requests |
+| **Tool Timeouts** | | | |
+| `DEFAULT_TOOL_TIMEOUT_SECONDS` | `300.0` | No | Deployment-wide default timeout (seconds, `0 < x ≤ 3600`) applied to every tool call (deployment, REST API, MCP, Python interpreter). Apps can override per-app via `tool_defaults.timeout_seconds`. |
+| `DEFAULT_FILE_LOADING_SIZE_LIMIT` | `10485760` | No | Deployment-wide default maximum size (in bytes) for files the agent downloads. Apps can override per-app via `features.file_loading.size_limit`. |
+| **Stage Display** | | | |
+| `DEFAULT_STAGE_DISPLAY_LEVEL` | — | No | Deployment-wide override for stage visibility threshold (`none`, `error`, `info`, `debug`; case-insensitive). When set, wins over every app's `features.stage_display.level`. Unset (default) defers to the per-app config, which defaults to `info`. |
+| **DIAL Files — Tool-Response Offload** | | | |
+| `TOOL_CALL_RESULT_OFFLOAD__ENABLED_BY_DEFAULT` | `true` | No | Default value of the per-app `enabled` flag (`features.dial_files.tool_call_result_offload.enabled`). Apps override per-app; `enabled: false` disables offload for that app. |
+| `TOOL_CALL_RESULT_OFFLOAD__SIZE_THRESHOLD` | `40000` | No | Default byte threshold above which a tool-call response is offloaded to a DIAL file. Apps override per-app via `features.dial_files.tool_call_result_offload.size_threshold`. |
+| `TOOL_CALL_RESULT_OFFLOAD__EXCLUDED_TOOLS` | `[]` | No | Default JSON list of **additional** tool names exempt from offloading. The read-back tools (`internal_file_read_lines`, `internal_file_search`) are always excluded regardless of this value, so a large read-back slice is never re-offloaded. Apps add more per-app via `features.dial_files.tool_call_result_offload.excluded_tools`. |
+| **External URL Egress** | | | |
+| `EXTERNAL_URL_FETCH_ENABLED` | `false` | No | Admin cap on fetching external (non-DIAL) URLs. When `false` (default), no app may fetch external URLs regardless of its manifest; the deployment-handoff branch (deployments with `features.url_attachments`) is unaffected. Apps can opt out per-app via `features.external_url_fetch.enabled=false` even when the admin allows. |
+| `EXTERNAL_URL_FETCH_HOST_ALLOWLIST` | — | No | Comma-separated allowlist of host patterns for external URL fetches. Unset (default) means no admin-level host restriction. Patterns: exact host (`example.com`) or `*.example.com` for any subdomain. Re-checked on every redirect hop. Per-app `features.external_url_fetch.host_allowlist` narrows further (intersection) but never expands. |
+| `EXTERNAL_URL_FETCH_MAX_REDIRECTS` | `5` | No | Maximum HTTP redirects on external URL fetches. Each hop is SSRF-checked. Hard ceiling 10. |
+| `EXTERNAL_URL_FETCH_CONNECT_TIMEOUT_SECONDS` | `5.0` | No | TCP connect timeout (seconds) for external URL fetches. Read/write/pool timeouts use the resolved tool timeout. |
+| **Dynamic Tool Discovery** `[Preview]` | | | |
+| `MIN_TOOLS_FOR_DEFERRAL` | `10` | No | Deployment-wide minimum toolset size for deferral to apply. Toolsets with fewer tools than this threshold are promoted to eager loading even when `deferred=true`. Apps override per-app via `orchestrator.tool_discovery.min_tools_for_deferral`. Requires `ENABLE_PREVIEW_FEATURES=true`. |
+| **Skills** | | | |
+| `DIAL_SKILLS_FILE_MAX_BYTES` | `262144` | No | Cap on a single file read from a DIAL skill resource, `SKILL.md` included. Must exceed the largest manifest you expect: an over-cap manifest drops the skill. See [docs/skills.md](docs/skills.md). |
+| `DIAL_SKILLS_MAX_FILES` | `200` | No | Maximum bundled files advertised to the agent per DIAL skill resource; beyond it the listing is truncated |
+| `DIAL_SKILLS_LISTING_MAX_PAGES` | `10` | No | Maximum file-listing pages followed per DIAL skill resource, bounding a server-supplied cursor |
+| `SKILL_INVOCATION_MAX_SKILLS` | `10` | No | Maximum distinct skills a user may have invoked from the messages of one conversation (`custom_content.skills`), counted newest first. Each one adds a `` block to the system prompt and one DIAL Core fetch per turn; beyond the cap the oldest picks stop being registered. Preview-gated. See [docs/skills.md](docs/skills.md). |
+| **Feature Gating** | | | |
+| `ENABLE_PREVIEW_FEATURES` | `false` | No | Enable preview features across the deployment (schema visibility + runtime activation) |
+| **Templates** | | | |
+| `PREDEFINED_EXTRA_PATHS` | — | No | JSON list of directories layered on top of built-in predefined content (later entries override earlier ones) |
+| `CONFIG_PROMPT_MAPPING` | *(built-in mapping)* | No | JSON mapping of predefined system prompts to DIAL Core deployments |
+| **Observability** | | | |
+| `OTEL_SERVICE_NAME` | `quickapps` | No | Service name stamped on all exported telemetry (traces, metrics, logs) |
+| `OTEL_TRACES_EXPORTER` | — | No | Set to `otlp` to enable tracing and export spans over OTLP/gRPC. Instruments the FastAPI server and outgoing HTTP clients (`httpx`, `requests`, `aiohttp`, `urllib`) and stamps trace context onto log records — see [docs/logging.md](docs/logging.md). |
+| `OTEL_METRICS_EXPORTER` | — | No | Comma-separated metric exporters: `otlp` (push over OTLP/gRPC) and/or `prometheus` (serve a scrape endpoint). Enables FastAPI and system/process metrics. |
+| `OTEL_LOGS_EXPORTER` | — | No | Set to `otlp` to export log records (INFO and above) over OTLP/gRPC alongside console output — see [docs/logging.md](docs/logging.md). |
+| `OTEL_EXPORTER_OTLP_ENDPOINT` | `http://localhost:4317` | No | OTLP/gRPC collector endpoint shared by trace, metric, and log export. One of the [standard OpenTelemetry SDK variables](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/), which the underlying exporters honor as usual (per-signal endpoints, headers, timeouts, resource attributes, …). |
+| `OTEL_EXPORTER_PROMETHEUS_PORT` | `9464` | No | Port of the Prometheus scrape endpoint (effective only with `prometheus` in `OTEL_METRICS_EXPORTER`) |
+| **Scripts & Tests** | | | |
+| `REMOTE_DIAL_URL` | — | No | URL of the remote DIAL Core, used only by `generate_dial_config` script and e2e/integration tests |
+| `REMOTE_DIAL_API_KEY` | — | No | API key of the remote DIAL Core, used only by `generate_dial_config` script and e2e/integration tests |
#### Deprecated Environment Variables
diff --git a/config/predefined/toolset/weather.json b/config/predefined/toolset/weather.json
index 7519159e..7b6e5613 100644
--- a/config/predefined/toolset/weather.json
+++ b/config/predefined/toolset/weather.json
@@ -56,9 +56,7 @@
},
"required": [
"latitude",
- "longitude",
- "current",
- "format"
+ "longitude"
]
}
}
diff --git a/docs/designs/dynamic_tool_discovery.md b/docs/designs/dynamic_tool_discovery.md
new file mode 100644
index 00000000..dbc43bae
--- /dev/null
+++ b/docs/designs/dynamic_tool_discovery.md
@@ -0,0 +1,607 @@
+# Design: Dynamic Tool Discovery
+
+- **Status:** Implemented
+- **Issue:** [#430](https://github.com/epam/ai-dial-quickapps-backend/issues/430)
+- **Chosen approach:** Option #6
+
+## Problem Statement
+
+All tool definitions (REST API, MCP, DIAL deployment) are merged into one flat list and sent
+verbatim to the LLM on every request:
+
+```python
+# _chat_completion_config_builder.py
+payload["tools"] = self.__tools # every tool, fully expanded, every call
+```
+
+With five or more MCP servers or large REST APIs this can consume ~55 K tokens upfront per turn,
+regardless of which tools the model will actually use. This dilutes model attention and degrades
+tool-selection accuracy.
+
+**Root cause:** `AgentModule.provide_openai_tools` assembles a single flat `list[OpenAiToolConfigDict]`
+at DI-wiring time. There is no mechanism to defer, filter, or paginate that list at request time.
+
+---
+
+## Goals
+
+- Reduce upfront token cost for applications with many tools by deferring full definitions until
+ the model indicates it needs them.
+- Non-breaking: existing applications that do not opt in behave identically.
+- Shaped for future configurability (per-tool granularity, semantic search, result caching)
+ without implementing those knobs in MVP.
+
+---
+
+## Codebase Anchors
+
+| What | File |
+|---|---|
+| LLM payload assembly | `src/quickapp/core/agent/_chat_completion_config_builder.py` |
+| Tool list DI wiring | `src/quickapp/core/agent/agent_module.py` (`provide_openai_tools`) |
+| MCP schema loading | `src/quickapp/mcp_tooling/_mcp_tool_initializer.py` |
+| Existing lazy-injection precedent | `src/quickapp/orchestrator_attachment_strategies/lazy_on_demand/` |
+| Toolset config root | `src/quickapp/config/toolsets/toolset.py`, `BaseToolSet` |
+| Internal tool base | `src/quickapp/common/staged_base_tool.py` (`StagedBaseTool`) |
+| Stream parsing | `src/quickapp/common/chat_completion_stream/parse.py` |
+| Orchestrator loop | `src/quickapp/core/agent/orchestrator.py` |
+
+---
+
+## Decision Points
+
+Four orthogonal decisions drive the design space:
+
+| # | Decision | Choices |
+|---|---|---|
+| A | **Deferral granularity** | Per-toolset vs per-tool |
+| B | **Discovery surface** | Keyword search · Compact manifest · Semantic search · Subagent-based |
+| C | **How full definitions reach the LLM** | As tool-call result text · Injected into `payload["tools"]` · Server-side expansion (`tool_reference`) |
+| D | **Provider scope** | Model-agnostic (any DIAL deployment) vs Anthropic-native API feature |
+
+---
+
+## Options
+
+### Option 1 — Per-toolset deferred flag + custom keyword-search discovery tool
+
+**Mechanism:**
+Add `deferred: bool = False` to `BaseToolSet`. When `deferred=true`, the toolset does not
+contribute its tool schemas to `payload["tools"]`. Instead it contributes a single internal
+discovery tool:
+
+```
+{toolset_name}_discover(query: str) → list[{name, description, parameters}]
+```
+
+The tool performs a keyword match on tool name + description and returns full definitions as
+JSON text in the tool-call result. The LLM reads the definitions from context and then makes
+the actual tool call.
+
+**Round-trip cost:** +1 before first tool use (discover → read result → call tool).
+
+**MCP init:** schemas still fetched at startup (no change); they are just withheld from
+`payload["tools"]` until discovered.
+
+**Config example:**
+```json
+{
+ "name": "my-mcp-server",
+ "type": "mcp",
+ "deferred": true,
+ "server": { "url": "..." }
+}
+```
+
+**Pros:**
+- Follows `LazyOnDemandStrategyModule` pattern exactly — no orchestrator changes.
+- Model-agnostic: works with any DIAL deployment.
+- Non-breaking opt-in.
+- Token savings are immediate (entire toolset suppressed).
+
+**Cons:**
+- Model must understand and follow the discovery protocol → system-prompt engineering required.
+- Full definition returned as text; the model cannot use the schema for structured argument
+ generation on the same call (requires an extra round-trip to actually call the tool natively).
+- Keyword search quality may be insufficient for large or ambiguously-named tool catalogs.
+
+---
+
+### Option 2 — Compact manifest upfront + single `get_tool_definition` tool
+
+**Mechanism:**
+All tool names + one-line descriptions (no `parameters`) are sent upfront, either as a
+minimalist `tools` array (empty parameters) or as a structured section in the system prompt.
+A single internal tool `get_tool_definition(tool_name: str)` returns the full `OpenAiToolConfig`
+JSON for any tool on demand.
+
+**Round-trip cost:** +1 before first use of any previously-unseen tool.
+
+**Config:** opt-out flag per-toolset (`deferred: false` to disable for a specific toolset).
+
+**Pros:**
+- Model always has full name-space visibility (all names + descriptions visible).
+- Single shared discovery tool regardless of toolset count.
+- Model-agnostic.
+
+**Cons:**
+- Manifest can still be substantial for 100+ tools.
+- Full definition returned as text; same two-round-trip issue as Option 1.
+
+---
+
+### Option 3 — Orchestrator-level dynamic tool injection
+
+**Mechanism:**
+A discovery tool returns tool names. The orchestrator intercepts the result and **injects the
+corresponding full schemas into `payload["tools"]` on the next LLM call** — not into the
+message history. The model then uses the tool natively with proper schema-based argument
+generation.
+
+**Changes required:**
+- Orchestrator maintains `_discovered_tool_names: set[str]` state across iterations.
+- `_ChatCompletionConfigBuilder` accepts a per-request "additional tools" override.
+- Discovery tool result processed as a side-effect before the next iteration.
+- Discovered names must optionally be persisted in conversation state for multi-turn continuity.
+
+**Round-trip cost:** +1 (discover call → definitions in `tools` array → native call).
+
+**Pros:**
+- LLM interacts with discovered tools natively (proper schema, no prompt workarounds).
+- Cleanest UX: discovered tools behave identically to pre-loaded ones from the model's perspective.
+- Model-agnostic.
+
+**Cons:**
+- Significant orchestrator changes.
+- Discovered-tool state must survive across iterations and possibly across conversation turns.
+- More complex error paths.
+
+---
+
+### Option 4 — Anthropic native `defer_loading` + server-side Tool Search
+
+**Mechanism:**
+Anthropic's Messages API supports `defer_loading: true` on individual tool definitions and a
+built-in server-side search tool. The API runs the search on Anthropic's infrastructure and
+returns `tool_reference` blocks that it auto-expands into full definitions before the model sees
+them.
+
+**API contract:**
+```json
+{
+ "tools": [
+ { "type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25" },
+ {
+ "name": "my_tool",
+ "description": "...",
+ "input_schema": { ... },
+ "defer_loading": true
+ }
+ ]
+}
+```
+
+- `defer_loading: true` controls what enters the **model's context window**, not what is sent
+ over the wire. All definitions are still transmitted to the API on every request.
+- The API excludes deferred tools from the system-prompt prefix → **prompt cache is preserved**.
+- Two search variants: `tool_search_tool_regex_20251119` (Python regex patterns) and
+ `tool_search_tool_bm25_20251119` (BM25 natural-language queries).
+- Supports up to **10,000 deferred tools**.
+- The response contains `server_tool_use` and `tool_search_tool_result` blocks (server-executed;
+ no client `tool_result` reply needed for these) plus a `tool_reference` block that the API
+ expands automatically.
+
+**Model support:** Claude Haiku 4.5, Sonnet 4.5, Opus 4.5 and all newer models.
+Claude Opus 4.1 and earlier do not support this feature.
+
+**Custom client-side variant:** The `tool_reference` response format is also usable by a
+custom search tool (embedding-based, semantic, etc.): return `tool_reference` blocks in a
+standard `tool_result`, and the API expands them the same way.
+
+**Changes required in QuickApp:**
+1. `_ChatCompletionConfigBuilder` — optionally set `defer_loading: true` on tool dicts and
+ include the tool search tool entry.
+2. `parse.py` / stream handler — parse `server_tool_use` and `tool_search_tool_result` block
+ types (currently unknown to the parser).
+3. `orchestrator.py` message builder — preserve `server_tool_use` and `tool_search_tool_result`
+ blocks verbatim in the ASSISTANT message history (do not treat them as executable tool calls).
+4. Config — a new `OrchestratorConfig.tool_search` sub-config controlling search variant and
+ which toolsets/tools to defer.
+5. Capability detection — fall back gracefully when the orchestrator deployment is not an
+ Anthropic model that supports this feature.
+
+**Round-trip cost:** +1 search turn. But definitions are expanded server-side within that
+same turn, so the model can call the discovered tool in the very next turn with no extra
+client round-trip.
+
+**Pros:**
+- No custom search implementation to maintain — Anthropic handles indexing, matching, and
+ schema expansion.
+- Prompt cache preserved (deferred tools excluded from the stable prefix).
+- Native `tool_reference` expansion means the model always works with real tool schemas.
+- Supports per-tool granularity on the API side.
+
+**Cons:**
+- **Not model-agnostic**: only works when the orchestrator deployment exposes Anthropic's
+ Messages API (Claude Sonnet/Opus 4.5+). Other DIAL deployments (GPT-4, Gemini, etc.)
+ do not have this feature.
+- Full definitions still transmitted to the API on every request (wire cost unchanged, only
+ context-window cost reduced).
+- New block types (`server_tool_use`, `tool_search_tool_result`, `tool_reference`) require
+ changes to the stream parser and message history handling.
+- `defer_loading: true` and `cache_control` cannot be set on the same tool (API returns 400).
+
+---
+
+### Option 5 — MCP-protocol progressive discovery (three-layer)
+
+**Mechanism:**
+Follows the [MCP client best practices](https://modelcontextprotocol.io/docs/2026-07-28/develop/clients/client-best-practices)
+progressive discovery pattern. The host fetches all tool definitions via `tools/list` at startup
+but exposes them to the model through a three-layer surface:
+
+| Layer | Tool | What it returns |
+|---|---|---|
+| 1 — Catalog | `search_tools(query)` | `[{name, description}]` — names + one-liners only |
+| 2 — Inspect | `get_tool_details(name)` | Full `inputSchema` for one tool |
+| 3 — Execute | `{tool_name}(...)` | Normal tool execution |
+
+Full definitions enter the context only at Layer 2, after the model identifies the specific
+tool it wants. When implemented with custom client-side `tool_reference` blocks (see Option 4
+custom variant), the Layer 2 inspection call can be eliminated and the Layer 1 result can
+directly expand into full definitions.
+
+**Threshold recommendation (from MCP spec):** switch to progressive discovery once tool
+definitions exceed 1–5% of the model's context window.
+
+**Dynamic server management extension:** connect MCP servers lazily — maintain a server
+registry, connect only when the model requests a server's capabilities. Requires changes to
+`_MCPToolInitializer` to support deferred connection and `tools/list` fetching.
+
+**Round-trip cost:** +1 (search) or +2 (search + inspect) before first tool use.
+
+**Pros:**
+- Fully model-agnostic (any LLM; no Anthropic-specific API features).
+- Covers both tool-level and server-level deferral.
+- Aligns with the emerging MCP ecosystem standard, future-proofing integration as MCP
+ clients converge on this pattern.
+- Layer 2 (inspect) is optional: with Option 4's `tool_reference` extension it collapses to
+ a single extra round-trip.
+- `list_changed` notification support (already a concept in MCP) enables cache invalidation.
+
+**Cons:**
+- Two round-trips (catalog + inspect) in the full three-layer form.
+- Dynamic server management is a significant additional change (`_MCPToolInitializer`,
+ connection lifecycle, reconnect on demand).
+- Without the `tool_reference` trick (Option 4 custom variant), full definition is returned as
+ text — same schema-generation problem as Options 1–2.
+
+---
+
+### Option 6 — `DeferredToolsContext` + anonymous agent search + lazy schema injection (Recommended, chosen)
+
+**Mechanism:**
+At request time, toolset initializers (MCP, REST, and internal) split their output between the
+normal `list[StagedBaseTool]` (eager tools) and a shared `DeferredToolsContext` (deferred tools).
+A single `tool_search` meta-tool is injected; when triggered it fires an isolated "anonymous agent"
+chat completion that consults only the deferred catalog, then stores the matched tools' full
+schemas in a request-scoped `LazyLoadedToolsHolder`, which `_ChatCompletionConfigBuilder` merges
+into `payload["tools"]` on the next build. No Anthropic-specific API features are required, and
+**no orchestrator changes were needed** — the mechanism is entirely local to the `tool_search`
+tool's own execution and the request-scoped holder it writes to.
+
+> **As built:** the sections below describe the mechanism as originally proposed. Where the
+> landed implementation differs, an **As built** note calls it out. See the
+> [Changes required](#changes-required) table for the final shape.
+
+#### Step 1 — Request initialisation
+
+When a new chat completion request arrives, toolset initializers run as today and build full
+`OpenAiToolConfig` definitions. They then apply the deferral decision per toolset (the real
+predicate, `is_toolset_deferred` — note the tri-state `deferred` field, see
+[Deferral threshold](#deferral-threshold)):
+
+```
+if is_toolset_deferred(toolset, discovery_cfg, len(tools)):
+ → register {name, description} entries + full definitions with DeferredToolsContext
+ → the same tools are still appended to the module's own list[StagedBaseTool]
+ (they exist as StagedBaseTool instances throughout — deferral only withholds
+ their *schema* from the initial LLM payload, in AgentModule.provide_openai_tools)
+else:
+ → nothing extra happens; the tool is eager as today
+```
+
+**As built:** REST API and MCP toolsets support deferral (`rest_api_tooling_module.py`,
+`_mcp_tool_initializer.py`). Internal toolsets also support it. `dial-deployment` and `dial-app`
+toolsets do not yet call `is_toolset_deferred` — setting `deferred` on those has no effect today
+(tracked as a follow-up).
+
+`DeferredToolsContext` holds two structures across all deferred toolsets in the request (not
+one instance per toolset — a single request-scoped context aggregates all of them):
+- **catalog**: `list[{name, description}]` — compact, never forwarded to the main LLM
+- **definitions**: `dict[str, OpenAiToolConfigDict]` — full schemas (transformed the same way
+ as eager tools — const params stripped, `enrich_openai_tool_schema` applied), looked up by
+ `_ToolSearchTool` when the anonymous agent returns matches
+
+Toolsets with `deferred: false`, or that fall below the tool-count threshold, stay eager —
+`AgentModule.provide_openai_tools` includes them in `payload["tools"]` directly.
+
+#### Step 2 — Main orchestrator call
+
+`payload["tools"]` is built from the eager `list[StagedBaseTool]` plus the single `tool_search`
+meta-tool injected by `AgentModule`. Deferred tools are **absent**.
+
+```
+payload["tools"] = [tool_search, ...eager tools from RequestContext]
+payload["messages"] = full conversation history
+```
+
+The description of `tool_search` explicitly states that additional tools are available and can
+be discovered on demand, so the model knows to search before assuming a capability is missing.
+
+**As built:** the description also carries a dynamic, per-request section listing every currently
+deferred toolset by name, its tool count, and its own `description` (when set) — e.g. "Additional
+toolsets available for discovery: - salesforce. Available tools: 12. Query and update Salesforce
+records". This is built in `_ToolSearchTool.enrich_openai_tool_schema` from
+`DeferredToolsContext.toolset_summaries`, giving the model a hint about *what* (and how much) is
+hidden, not just that *something* is discoverable — without the per-tool schema cost that listing
+every tool upfront would incur.
+
+#### Step 3 — `tool_search` execution: anonymous agent
+
+When the orchestrator calls `tool_search(query)`, its handler delegates to a new
+**`AnonymousAgent`** module — a self-contained, isolated chat completion with no conversation
+history and no system prompt from the main request:
+
+```
+model: service_model (config: defaults to orchestrator deployment)
+system: "You are a tool routing assistant. Given a user query, return the names of
+ the tools from the following catalog that are most relevant.
+ Catalog: [{name, description}, ...]" ← injected from DeferredToolsContext
+messages:[{"role": "user", "content": }]
+tools: none
+```
+
+The anonymous agent returns a list of tool names. Because this call carries no conversation
+history and no application system prompt, its token cost is bounded by the catalog size alone.
+
+**Result returned to the main LLM:** `[{name, description}]` of matched tools, confirming what
+is now available to load.
+
+#### Step 4 — `_ToolSearchTool` builds OpenAI definitions
+
+Rather than a separate lazy-initializer module, `_ToolSearchTool` (a `StagedBaseTool`) itself
+looks up each matched name in `DeferredToolsContext.get_definition(...)` and passes the
+corresponding `OpenAiToolConfigDict` objects to `LazyLoadedToolsHolder.add(...)`. No LLM call is
+made at this step.
+
+#### Step 5 — Lazy schema injection (no orchestrator involvement)
+
+**As built, this differs from the original proposal:** there is no orchestrator-side interception
+or `_lazy_loaded_tools` state inside `orchestrator.py`. Instead:
+1. `LazyLoadedToolsHolder` is a request-scoped holder (`core/agent/lazy_loaded_tools_holder.py`)
+ that `_ToolSearchTool` writes into directly during its own tool-call execution.
+2. On every subsequent `_ChatCompletionConfigBuilder.build()` call within the same request,
+ the builder reads `lazy_loaded_tools_holder.get_all()` and merges those definitions into
+ `payload["tools"]`, de-duplicated by function name against the eager tools.
+3. The orchestrator loop is completely unaware of tool discovery — it just re-builds the payload
+ each iteration as it always did, and the holder's contents are picked up automatically.
+
+The main LLM now sees the discovered tools natively alongside `tool_search` and the eager tools,
+and calls them with proper schema-based argument generation.
+
+Cross-turn persistence (serialising discovered tool names into
+`custom_content.state["lazy_loaded_tools"]` so rediscovery is skipped on the next user message)
+was **not implemented** — see [Out of Scope](#out-of-scope-mvp).
+
+#### Flow diagram
+
+```
+New request arrives
+ │
+ ├─ MCP/REST/internal initializers: len(tools) >= threshold → DeferredToolsContext
+ │ (catalog + definitions)
+ │ len(tools) < threshold → eager list[StagedBaseTool]
+ │
+ ▼
+Orchestrator — iteration 1
+ payload["tools"] = [tool_search, ...eager tools]
+ payload["messages"] = full conversation history
+ │
+ └─ Main LLM calls tool_search("I need to query Salesforce contacts")
+ │
+ └─ _ToolSearchTool → _AnonymousAgent (isolated chat completion):
+ model = service_model
+ system = routing prompt + catalog from DeferredToolsContext
+ message = "I need to query Salesforce contacts"
+ → ["sf_query_contacts", "sf_list_contacts"]
+ _ToolSearchTool: names → full OpenAiToolConfigDicts, written into
+ LazyLoadedToolsHolder (request-scoped; orchestrator is not involved)
+ Result returned to main LLM: [{name, description}, ...]
+
+Orchestrator — iteration 2
+ _ChatCompletionConfigBuilder reads LazyLoadedToolsHolder.get_all() and merges:
+ payload["tools"] = [tool_search, ...eager tools,
+ sf_query_contacts ←injected,
+ sf_list_contacts ←injected]
+ │
+ └─ Main LLM calls sf_query_contacts(filter="LastName='Smith'") natively ✓
+```
+
+#### Deferral threshold
+
+A toolset is only placed into `DeferredToolsContext` if `is_toolset_deferred` returns true.
+`deferred` is a tri-state field (`bool | None`, default `None`/unset) — unset **or** `true`
+both defer; only an explicit `false` forces eager regardless of count:
+
+```python
+def is_toolset_deferred(toolset, discovery_cfg, tool_count) -> bool:
+ return (
+ toolset.deferred is not False
+ and discovery_cfg is not None
+ and discovery_cfg.enabled
+ and tool_count >= discovery_cfg.min_tools_for_deferral
+ )
+```
+
+Below the threshold — or when `tool_discovery` is disabled/unset entirely — a toolset is loaded
+eagerly, no discovery overhead.
+
+#### Configuration
+
+```json
+{
+ "orchestrator": {
+ "tool_discovery": {
+ "enabled": true,
+ "service_model": "claude-haiku-dial-deployment",
+ "min_tools_for_deferral": 5
+ }
+ },
+ "tool_sets": [
+ {
+ "name": "salesforce",
+ "type": "mcp",
+ "deferred": true,
+ "server": { "url": "..." }
+ },
+ {
+ "name": "internal-utils",
+ "type": "internal",
+ "deferred": false
+ }
+ ]
+}
+```
+
+- `deferred: true` (or unset/`null`) opts the toolset into `DeferredToolsContext`. Default: `true` (omitting the field defers by default).
+- `service_model` names the DIAL deployment used for the `AnonymousAgent` chat completion.
+ When omitted, falls back to the orchestrator's own deployment.
+- `min_tools_for_deferral` is the tool-count guard below which a deferred toolset is silently
+ promoted to eager. Default: `5`.
+- Non-deferred and below-threshold toolsets populate `RequestContext` immediately, as today.
+
+#### Changes required (as built)
+
+| Area | Change |
+|---|---|
+| `BaseToolSet` (`config/toolsets/base.py`) | Add `deferred: bool \| None` field (tri-state, default `None`/unset — unset behaves as deferred, see [Deferral threshold](#deferral-threshold)) |
+| `config/tool_discovery.py` | New `ToolDiscoveryConfig` (`enabled`, `service_model`, `min_tools_for_deferral`), referenced by `OrchestratorConfig.tool_discovery` |
+| `shared/deferred_tools/` (`DeferredToolsContext`, `is_toolset_deferred`) | Request-scoped shared object aggregating `catalog`/`definitions` across all deferred toolsets in the request; `is_toolset_deferred` is the pure threshold predicate. Bound via its own `DeferredToolsModule`, spliced into `shared_module` |
+| REST, MCP, internal toolset modules | After building each toolset's tools, evaluate `is_toolset_deferred`; register with `DeferredToolsContext` or leave in the eager `list[StagedBaseTool]` accordingly. **`dial-deployment`/`dial-app` toolsets do not yet do this** — follow-up |
+| `tool_discovery/_anonymous_agent.py` (`_AnonymousAgent`) | Fires a single isolated `chat.completions.create` call (no history, no app system prompt); takes the catalog and a user query; returns matched tool names |
+| `tool_discovery/_tool_search_tool.py` (`_ToolSearchTool`) | Internal `tool_search` (registered name: `internal_tool_search`) tool injected via `ToolDiscoveryModule`'s own `@multiprovider` (preview-gated); calls `_AnonymousAgent`, looks up matched names in `DeferredToolsContext`, writes results into `LazyLoadedToolsHolder`, returns `[{name, description}]` to the main LLM. Its own `enrich_openai_tool_schema` override appends a dynamic list of deferred toolset names/descriptions (`DeferredToolsContext.toolset_summaries`) to the static tool description |
+| `core/agent/lazy_loaded_tools_holder.py` (`LazyLoadedToolsHolder`) | Request-scoped holder of discovered `OpenAiToolConfigDict`s — replaces the originally-proposed lazy-initializer + orchestrator-side `_lazy_loaded_tools` state |
+| `tool_discovery/_tool_search_hint_prompt_provider.py` (`_ToolSearchHintPromptProvider`) | System-prompt-level reminder to call `internal_tool_search` before declaring a limitation, plus the same deferred-toolset summary list (name, tool count, description — via the shared `_toolset_summary_format.format_toolset_summaries`) that `_ToolSearchTool.enrich_openai_tool_schema` appends to the tool's own description. Contributed by `ToolDiscoveryModule`'s own `_provide_prompt_parts` (preview-gated, same condition as the tool itself). `ToolDiscoveryModule` is registered in `app_factory.py` **before** `SkillsModule` specifically so this hint lands immediately ahead of the `` block in the aggregated system prompt |
+| `_chat_completion_config_builder.py` | Reads `LazyLoadedToolsHolder.get_all()` on every build and merges into `payload["tools"]`, de-duplicated against eager tool names — **`orchestrator.py` itself was not changed** |
+| Cross-turn state persistence | **Not implemented** — see [Out of Scope](#out-of-scope-mvp) |
+
+**Round-trip cost:** +1 turn before first native tool use (search + inject → tool call).
+The anonymous agent call happens inside the `tool_search` tool execution, not as a separate
+orchestrator iteration. Subsequent calls to the same tool within a turn are free (already in
+`_lazy_loaded_tools`). With cross-turn state, rediscovery is skipped on later turns.
+
+**Token cost of `tool_search` (anonymous agent call):**
+`catalog_tokens + query_tokens` — independent of conversation length. For 200 deferred tools
+with ~20-token descriptions each, this is ~4 K tokens regardless of how long the conversation is.
+
+**Pros:**
+- Fully model-agnostic: works with any DIAL deployment as orchestrator.
+- `DeferredToolsContext` cleanly separates eager and deferred tool state at the DI layer —
+ no orchestrator logic needed to decide what to defer.
+- Anonymous agent isolates search cost from main conversation tokens; scales with catalog size,
+ not conversation length.
+- Native schema injection means the main LLM calls discovered tools with proper structured
+ arguments from the iteration after discovery.
+- Opt-out: `deferred: true` by default; threshold guard prevents regression for small toolsets.
+ Set `deferred: false` on a toolset to keep it always-eager.
+- `AnonymousAgent` is a reusable module independent of tool discovery.
+- Search strategy is swappable (keyword, embedding, different model) without touching the
+ orchestrator or injection logic.
+
+**Cons:**
+- +1 orchestrator iteration before first native use of a deferred tool.
+- `AnonymousAgent` introduces a new code path for isolated completions.
+- Discovered tools only live for the current turn (`LazyLoadedToolsHolder` is request-scoped);
+ cross-turn persistence would require state serialisation (not implemented, see Out of Scope).
+- Search quality depends on service model and catalog description quality.
+
+---
+
+## Comparison
+
+| | Option 1 | Option 2 | Option 3 | Option 4 | Option 5 | **Option 6** |
+|---|---|---|---|---|---|---|
+| Model-agnostic | Yes | Yes | Yes | No (Anthropic 4.5+) | Yes | **Yes** |
+| Orchestrator changes | None | None | Significant | Medium | Medium | **None** (as built) |
+| Native schema on discovered tool call | No | No | Yes | Yes | Depends | **Yes** |
+| Full name-space visible upfront | No | Yes | No | No | No | **No** |
+| Prompt cache preserved | — | — | — | Yes (by design) | — | **Partial** ¹ |
+| Per-tool granularity | Toolset | Toolset | Toolset | Per-tool | Per-tool | **Toolset** |
+| Search uses separate LLM call | No | No | No | No (server-side) | No | **Yes** |
+| Implementation complexity | Low | Low | High | Medium | Medium–High | **Medium** |
+| Follows existing lazy pattern | Yes | Partial | No | No | No | **Partial** |
+| Custom search logic possible | Yes | Yes | Yes | Yes | Yes | **Yes** |
+
+¹ Deferred tools are absent from `payload["tools"]` in main calls, so the stable prefix
+(eager tools + `tool_search`) is cacheable. Lazy-loaded tools appended after discovery break
+the cache for that iteration only.
+
+---
+
+## Recommendation
+
+**Option 6** is the recommended approach. It is the only option that is simultaneously
+model-agnostic, delivers native schema-based tool calling after discovery, and isolates the
+search token cost from the conversation context.
+
+Options 1–3 are useful reference points for the individual sub-problems Option 6 combines.
+Option 4 is the right choice if the team decides to target Anthropic deployments exclusively
+and wants to offload search infrastructure entirely. Option 5 (full MCP three-layer + dynamic
+server management) remains the long-term architecture for MCP toolsets and can be layered on
+top of Option 6 incrementally.
+
+---
+
+## Open Questions
+
+1. **Granularity:** `deferred` on `BaseToolSet` (per-toolset) or also settable per-tool within
+ a toolset? Per-toolset covers MCP servers and REST API groups cleanly; per-tool is needed
+ only for mixed toolsets where some tools are always-on. **Still open** — per-toolset is what
+ shipped; per-tool granularity remains a possible future extension.
+
+2. ~~**`service_model` default**~~ — **Resolved, as built:** falls back to the orchestrator's
+ own deployment when omitted (`_AnonymousAgent.route`).
+
+3. ~~**Search implementation in MVP**~~ — **Resolved, as built:** pure LLM routing via
+ `_AnonymousAgent`, no keyword-only fallback. Not currently planned.
+
+4. ~~**Multi-turn persistence**~~ — **Resolved, as built: not implemented.** Discovered tools
+ are always rediscovered each turn; see [Out of Scope](#out-of-scope-mvp).
+
+5. **Always-on threshold:** Should there be a heuristic that automatically promotes a
+ frequently-discovered tool to eager loading (e.g. seen in last N turns), or is that always
+ explicit config? **Still open** — not implemented; today it's always explicit config
+ (`deferred: false` per toolset).
+
+---
+
+## Out of Scope (MVP)
+
+| Item | Reason |
+|---|---|
+| Embedding-based search in `_AnonymousAgent` | Swap-in strategy; `_AnonymousAgent` interface is the extension point |
+| Dynamic MCP server connection/disconnection | Significant lifecycle change; Option 5 follow-on |
+| Per-tool granularity within a toolset | Per-toolset is sufficient for the initial use case |
+| Automatic threshold based on token count | Tool-count threshold is simpler and good enough for MVP |
+| Cross-turn discovery persistence (`custom_content.state["lazy_loaded_tools"]`) | Not implemented — `LazyLoadedToolsHolder` is request-scoped only; discovered tools are rediscovered every turn |
+| `dial-deployment` / `dial-app` toolset deferral | REST, MCP, and internal toolsets support `deferred`; deployment/app toolsets do not yet call `is_toolset_deferred` — tracked as a follow-up |
+
+---
+
+## References
+
+- [MCP Client Best Practices — Progressive Tool Discovery](https://modelcontextprotocol.io/docs/2026-07-28/develop/clients/client-best-practices)
+- [Anthropic Tool Search Tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool)
+- [Anthropic — Advanced Tool Use](https://www.anthropic.com/engineering/advanced-tool-use)
+- [Anthropic — Effective Context Engineering](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)
diff --git a/docs/generated-app-schema.json b/docs/generated-app-schema.json
index e52ecd60..5d7c12e4 100644
--- a/docs/generated-app-schema.json
+++ b/docs/generated-app-schema.json
@@ -684,6 +684,20 @@
"title": "Enabled",
"type": "boolean"
},
+ "deferred": {
+ "anyOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": true,
+ "description": "When true or unset, this toolset's tool schemas are withheld from the initial LLM payload. Requires orchestrator.tool_discovery.enabled=true. Tools are discovered on demand via the tool_search meta-tool. Set to false to keep this toolset always eager.",
+ "title": "Deferred",
+ "x-preview": true
+ },
"type": {
"const": "dial-deployment",
"default": "dial-deployment",
@@ -759,6 +773,20 @@
"title": "Enabled",
"type": "boolean"
},
+ "deferred": {
+ "anyOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": true,
+ "description": "When true or unset, this toolset's tool schemas are withheld from the initial LLM payload. Requires orchestrator.tool_discovery.enabled=true. Tools are discovered on demand via the tool_search meta-tool. Set to false to keep this toolset always eager.",
+ "title": "Deferred",
+ "x-preview": true
+ },
"type": {
"const": "dial-app",
"default": "dial-app",
@@ -1267,6 +1295,20 @@
"title": "Enabled",
"type": "boolean"
},
+ "deferred": {
+ "anyOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": true,
+ "description": "When true or unset, this toolset's tool schemas are withheld from the initial LLM payload. Requires orchestrator.tool_discovery.enabled=true. Tools are discovered on demand via the tool_search meta-tool. Set to false to keep this toolset always eager.",
+ "title": "Deferred",
+ "x-preview": true
+ },
"type": {
"const": "dial-mcp",
"default": "dial-mcp",
@@ -1872,6 +1914,20 @@
"title": "Enabled",
"type": "boolean"
},
+ "deferred": {
+ "anyOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": true,
+ "description": "When true or unset, this toolset's tool schemas are withheld from the initial LLM payload. Requires orchestrator.tool_discovery.enabled=true. Tools are discovered on demand via the tool_search meta-tool. Set to false to keep this toolset always eager.",
+ "title": "Deferred",
+ "x-preview": true
+ },
"type": {
"const": "internal",
"default": "internal",
@@ -2266,6 +2322,20 @@
"title": "Enabled",
"type": "boolean"
},
+ "deferred": {
+ "anyOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": true,
+ "description": "When true or unset, this toolset's tool schemas are withheld from the initial LLM payload. Requires orchestrator.tool_discovery.enabled=true. Tools are discovered on demand via the tool_search meta-tool. Set to false to keep this toolset always eager.",
+ "title": "Deferred",
+ "x-preview": true
+ },
"type": {
"const": "mcp",
"default": "mcp",
@@ -3114,6 +3184,20 @@
"title": "Enabled",
"type": "boolean"
},
+ "deferred": {
+ "anyOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": true,
+ "description": "When true or unset, this toolset's tool schemas are withheld from the initial LLM payload. Requires orchestrator.tool_discovery.enabled=true. Tools are discovered on demand via the tool_search meta-tool. Set to false to keep this toolset always eager.",
+ "title": "Deferred",
+ "x-preview": true
+ },
"type": {
"const": "rest-api",
"default": "rest-api",
@@ -3406,6 +3490,38 @@
"title": "ToolCallTimestampConfig",
"type": "object"
},
+ "ToolDiscoveryConfig": {
+ "properties": {
+ "enabled": {
+ "default": false,
+ "description": "Enable dynamic tool discovery. When true, toolsets with deferred=true are withheld from the initial LLM payload and surfaced via the internal_tool_search meta-tool.",
+ "title": "Enabled",
+ "type": "boolean"
+ },
+ "service_model": {
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "description": "DIAL deployment used for the anonymous routing call inside internal_tool_search. When omitted, falls back to the orchestrator's own deployment.",
+ "title": "Service Model"
+ },
+ "min_tools_for_deferral": {
+ "default": 10,
+ "description": "Minimum number of tools in a toolset for deferral to apply. Toolsets with fewer tools than this threshold are promoted to eager loading even when deferred=true, avoiding discovery overhead for small toolsets. Default: 10 (or the value of MIN_TOOLS_FOR_DEFERRAL env var)",
+ "minimum": 1,
+ "title": "Min Tools For Deferral",
+ "type": "integer"
+ }
+ },
+ "title": "ToolDiscoveryConfig",
+ "type": "object"
+ },
"ToolDisplayConfig": {
"properties": {
"stage": {
@@ -3662,6 +3778,19 @@
],
"default": null,
"description": "How the orchestrator receives request-scoped attachments. When unset, the orchestrator gets no admin/user attachments on the native path (legacy behaviour: USER `image/*` passes through, other MIMEs are surfaced as XML metadata only)."
+ },
+ "tool_discovery": {
+ "anyOf": [
+ {
+ "$ref": "#/$defs/ToolDiscoveryConfig"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "description": "Dynamic tool discovery configuration. When enabled, toolsets with deferred=true are withheld from the initial LLM payload and discovered on demand via the tool_search meta-tool.",
+ "x-preview": true
}
},
"required": [
diff --git a/docs/generated-internal-tools.json b/docs/generated-internal-tools.json
index 383c187c..2ee98442 100644
--- a/docs/generated-internal-tools.json
+++ b/docs/generated-internal-tools.json
@@ -385,6 +385,19 @@
}
}
},
+ {
+ "name": "internal_tool_search",
+ "description": "\nPurpose: Discover additional tools and capabilities.\n\nYou MUST call this tool before stating any limitation or saying you cannot fulfill a user request.\n\nIn addition, you MUST call this tool whenever:\n- The user’s request is open-ended or underspecified.\n- The request might involve capabilities beyond your currently listed tools.\n- The request could plausibly be served by a specialized tool or by returning a resource, even if you believe you can respond text-only.\n\nIf a user request might involve capabilities beyond your current listed tools, you are REQUIRED to:\n1) Call internal_tool_search with a brief description of the needed capability.\n2) Inspect any returned tools.\n\nYou may NOT:\n- Assume that the initially listed tools are exhaustive.\n- Say \"I can't\", \"I don't have access\", or express similar limitations until you have called internal_tool_search in this conversation turn.\n\nOnly if internal_tool_search returns no suitable tools, or all relevant tools fail, may you tell the user you cannot do it.\n\nThis tool dynamically discovers additional toolsets (including hidden or MCP tools) that may provide additional capabilities:\n",
+ "properties": {
+ "query": {
+ "type": "string",
+ "description": "A natural-language description of the capability you need."
+ }
+ },
+ "required": [
+ "query"
+ ]
+ },
{
"name": "internal_web_fetch",
"description": "Fetch a resource from an external http(s) URL (e.g. a README, a source file, a documentation page). Without save_path it returns the text inline in a single call — text only: binary content (images, PDFs, archives) is rejected, re-call with a save_path instead. Text larger than the inline cap is returned truncated to its head with a notice stating the total size; the head is often enough, otherwise re-call with a save_path. With save_path it persists the resource (any content type) at that workspace-relative path under the agent home and returns the saved path (+ a short preview for text), so other available tools can process the full content. DIAL file paths (files/...) are not fetched here.",
diff --git a/src/quickapp/app_factory.py b/src/quickapp/app_factory.py
index b3660057..905212ab 100644
--- a/src/quickapp/app_factory.py
+++ b/src/quickapp/app_factory.py
@@ -33,6 +33,7 @@
from quickapp.skills.skills_module import SkillsModule
from quickapp.starters.starters_module import StartersModule
from quickapp.timestamp_tooling.timestamp_module import TimestampModule
+from quickapp.tool_discovery.tool_discovery_module import ToolDiscoveryModule
from quickapp.web_tooling.web_tooling_module import WebToolingModule
@@ -58,6 +59,10 @@ def build_di_modules() -> list[Module]:
FileTransferModule(),
AttachmentProcessingModule(),
LazyOnDemandStrategyModule(),
+ # ToolDiscoveryModule is registered before SkillsModule so its tool_search prompt
+ # hint (when active) lands immediately ahead of the block in the
+ # aggregated system prompt (list[PromptPartProvider] preserves module registration order).
+ ToolDiscoveryModule(),
SkillsModule(),
DialPromptSkillsModule(),
DialSkillsModule(),
diff --git a/src/quickapp/common/deferred_tool_types.py b/src/quickapp/common/deferred_tool_types.py
new file mode 100644
index 00000000..d25fd2e0
--- /dev/null
+++ b/src/quickapp/common/deferred_tool_types.py
@@ -0,0 +1,14 @@
+from typing import Annotated
+
+from pydantic import BaseModel
+
+from quickapp.config.tools.base import OpenAiToolConfigDict
+
+DeferredToolName = Annotated[str, "DeferredToolName"]
+DeferredToolCatalogEntry = Annotated[dict[str, str], "DeferredToolCatalogEntry"]
+DeferredToolsetSummary = Annotated[dict[str, str | int | None], "DeferredToolsetSummary"]
+
+
+class DeferredToolDefinition(BaseModel):
+ name: str
+ definition: OpenAiToolConfigDict
diff --git a/src/quickapp/common/deferred_tools_accumulator.py b/src/quickapp/common/deferred_tools_accumulator.py
new file mode 100644
index 00000000..dab62e31
--- /dev/null
+++ b/src/quickapp/common/deferred_tools_accumulator.py
@@ -0,0 +1,84 @@
+from quickapp.common.localized_string import resolve_localized
+from quickapp.common.staged_base_tool import StagedBaseTool
+from quickapp.config.tool_discovery import ToolDiscoveryConfig
+from quickapp.config.tools.base import (
+ BaseOpenAITool,
+ OpenAiToolConfigDict,
+ remove_const_schema_params,
+)
+from quickapp.config.toolsets.base import BaseToolSet
+
+
+class DeferredToolsAccumulator:
+ """Base contract for a tooling module's own deferred-tool registry.
+
+ Concrete subclasses are bound request-scoped by their owning module (mirrors
+ `ToolingContextBase`), not by a Module of their own.
+ """
+
+ def __init__(self) -> None:
+ self._catalog: list[dict[str, str]] = []
+ self._definitions: dict[str, OpenAiToolConfigDict] = {}
+ self._toolset_summaries: list[dict[str, str | int | None]] = []
+
+ def register_deferred_tools(self, toolset: BaseToolSet, tools: list[StagedBaseTool]) -> None:
+ entries: list[tuple[StagedBaseTool, BaseOpenAITool, str]] = [
+ (t, t.tool_config, name)
+ for t in tools
+ if isinstance(t.tool_config, BaseOpenAITool)
+ and (name := t.tool_config.open_ai_tool.function.name)
+ ]
+ self._catalog.extend(
+ {
+ "name": name,
+ "description": tool_config.open_ai_tool.function.description or "",
+ }
+ for _, tool_config, name in entries
+ )
+ self._definitions.update(
+ {
+ # Apply the same transform pipeline as the eager path (AgentModule.provide_openai_tools)
+ # so a discovered tool's schema matches what it would have looked like loaded eagerly.
+ name: tool.enrich_openai_tool_schema(
+ remove_const_schema_params(tool_config.open_ai_tool)
+ ).model_dump(mode="json", exclude_none=True)
+ for tool, tool_config, name in entries
+ }
+ )
+ self._toolset_summaries.append(
+ {
+ "name": resolve_localized(toolset.name),
+ "description": (
+ resolve_localized(toolset.description) if toolset.description else None
+ ),
+ "tool_count": len(entries),
+ }
+ )
+
+ @property
+ def deferred_names(self) -> frozenset[str]:
+ return frozenset(self._definitions.keys())
+
+ @property
+ def catalog(self) -> list[dict[str, str]]:
+ return list(self._catalog)
+
+ @property
+ def toolset_summaries(self) -> list[dict[str, str | int | None]]:
+ return list(self._toolset_summaries)
+
+ def get_definition(self, name: str) -> OpenAiToolConfigDict | None:
+ return self._definitions.get(name)
+
+
+def is_toolset_deferred(
+ toolset: BaseToolSet,
+ discovery_cfg: ToolDiscoveryConfig | None,
+ tool_count: int,
+) -> bool:
+ return (
+ toolset.deferred is not False
+ and discovery_cfg is not None
+ and discovery_cfg.enabled
+ and tool_count >= discovery_cfg.min_tools_for_deferral
+ )
diff --git a/src/quickapp/common/tool_names.py b/src/quickapp/common/tool_names.py
index fcc3347d..8ffc88ea 100644
--- a/src/quickapp/common/tool_names.py
+++ b/src/quickapp/common/tool_names.py
@@ -15,6 +15,8 @@
# DIAL files tools — all share the ``internal_file_`` prefix.
INTERNAL_FILE_TOOL_NAME_PREFIX = "internal_file_"
+INTERNAL_TOOL_SEARCH_TOOL_NAME = "internal_tool_search"
+
INTERNAL_FILE_LIST_TOOL_NAME = f"{INTERNAL_FILE_TOOL_NAME_PREFIX}list"
INTERNAL_FILE_READ_LINES_TOOL_NAME = f"{INTERNAL_FILE_TOOL_NAME_PREFIX}read_lines"
INTERNAL_FILE_SEARCH_TOOL_NAME = f"{INTERNAL_FILE_TOOL_NAME_PREFIX}search"
diff --git a/src/quickapp/config/application.py b/src/quickapp/config/application.py
index 88f7efdc..bbddb83e 100644
--- a/src/quickapp/config/application.py
+++ b/src/quickapp/config/application.py
@@ -22,6 +22,7 @@
from quickapp.config.skill import SkillConfig
from quickapp.config.starters import ConversationStartersConfig
from quickapp.config.timestamp import TimestampConfig, ToolCallTimestampConfig
+from quickapp.config.tool_discovery import ToolDiscoveryConfig
from quickapp.config.toolsets.toolset import ToolSet
from quickapp.config.web_fetch import WebFetchConfig
@@ -95,6 +96,10 @@ class OrchestratorConfig(BaseModel):
"MIMEs are surfaced as XML metadata only)."
),
)
+ tool_discovery: ToolDiscoveryConfig | None = PreviewField( # type: ignore[assignment]
+ default=None,
+ description="Dynamic tool discovery configuration. When enabled, toolsets with deferred=true are withheld from the initial LLM payload and discovered on demand via the tool_search meta-tool.",
+ )
def nullify_preview_fields(model: BaseModel) -> None:
diff --git a/src/quickapp/config/tool_discovery.py b/src/quickapp/config/tool_discovery.py
new file mode 100644
index 00000000..ea26603f
--- /dev/null
+++ b/src/quickapp/config/tool_discovery.py
@@ -0,0 +1,46 @@
+from pydantic import BaseModel, ConfigDict, Field
+from pydantic.fields import FieldInfo
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+
+class ToolDiscoverySettings(BaseSettings):
+ model_config = SettingsConfigDict()
+
+ min_tools_for_deferral: int = Field(
+ default=10,
+ ge=1,
+ description="Minimum toolset size for deferral to apply deployment-wide.",
+ alias="MIN_TOOLS_FOR_DEFERRAL",
+ )
+
+
+def _min_tools_for_deferral_field() -> FieldInfo:
+ description = (
+ "Minimum number of tools in a toolset for deferral to apply. "
+ "Toolsets with fewer tools than this threshold are promoted to eager loading "
+ "even when deferred=true, avoiding discovery overhead for small toolsets. "
+ "Default: 10 (or the value of MIN_TOOLS_FOR_DEFERRAL env var)"
+ )
+ return Field( # type: ignore[return-value]
+ default_factory=lambda: ToolDiscoverySettings().min_tools_for_deferral,
+ json_schema_extra={"default": 10},
+ ge=1,
+ description=description,
+ )
+
+
+class ToolDiscoveryConfig(BaseModel):
+ model_config = ConfigDict(frozen=True)
+
+ enabled: bool = Field(
+ default=False,
+ description="Enable dynamic tool discovery. When true, toolsets with deferred=true are withheld from the initial LLM payload and surfaced via the internal_tool_search meta-tool.",
+ )
+ service_model: str | None = Field(
+ default=None,
+ description=(
+ "DIAL deployment used for the anonymous routing call inside internal_tool_search. "
+ "When omitted, falls back to the orchestrator's own deployment."
+ ),
+ )
+ min_tools_for_deferral: int = _min_tools_for_deferral_field() # type: ignore[assignment]
diff --git a/src/quickapp/config/tools/base.py b/src/quickapp/config/tools/base.py
index e1e46a08..a91fb199 100644
--- a/src/quickapp/config/tools/base.py
+++ b/src/quickapp/config/tools/base.py
@@ -1,5 +1,6 @@
+import copy
from enum import Enum
-from typing import Annotated, Any, Generic, Literal, TypeVar, Union
+from typing import Annotated, Any, Generic, Literal, TypeAlias, TypeVar, Union
from pydantic import BaseModel, Field, field_validator
@@ -196,6 +197,21 @@ class OpenAiToolConfig(
]
+OpenAiToolConfigDict: TypeAlias = dict[str, Any]
+
+
+def remove_const_schema_params(open_ai_tool: OpenAiToolConfig) -> OpenAiToolConfig:
+ """Strip const-valued parameters (fixed values hidden from the LLM) from a tool's JSON schema."""
+ tool_copy = copy.deepcopy(open_ai_tool)
+ props = tool_copy.function.parameters.properties
+
+ for prop_name in list(props.keys()):
+ if issubclass(type(props[prop_name]), JsonSchemaConst):
+ del props[prop_name]
+
+ return tool_copy
+
+
class BaseTool(BaseModel):
attachment: AttachmentConfig = Field(
default_factory=AttachmentConfig, description="Configuration for tool attachments."
diff --git a/src/quickapp/config/toolsets/base.py b/src/quickapp/config/toolsets/base.py
index 22c5a42c..9cbf74f5 100644
--- a/src/quickapp/config/toolsets/base.py
+++ b/src/quickapp/config/toolsets/base.py
@@ -1,5 +1,6 @@
from pydantic import BaseModel, Field
+from quickapp.common.base_config import PreviewField
from quickapp.common.localized_string import LocalizedString
@@ -17,3 +18,13 @@ class BaseToolSet(BaseModel):
default=None, description="The description of the tool set."
)
enabled: bool = Field(default=True, description="Whether the toolset is enabled.")
+ deferred: bool | None = PreviewField( # type: ignore[assignment]
+ default=None,
+ json_schema_extra={"default": True},
+ description=(
+ "When true or unset, this toolset's tool schemas are withheld from the initial LLM payload. "
+ "Requires orchestrator.tool_discovery.enabled=true. "
+ "Tools are discovered on demand via the tool_search meta-tool. "
+ "Set to false to keep this toolset always eager."
+ ),
+ )
diff --git a/src/quickapp/core/agent/_chat_completion_config_builder.py b/src/quickapp/core/agent/_chat_completion_config_builder.py
index 5fa32045..32208738 100644
--- a/src/quickapp/core/agent/_chat_completion_config_builder.py
+++ b/src/quickapp/core/agent/_chat_completion_config_builder.py
@@ -12,6 +12,7 @@
from quickapp.common.presentation_settings import PresentationSettings
from quickapp.config.application import ApplicationConfig
from quickapp.core.agent._tool_choice_holder import _ToolChoiceHolder
+from quickapp.core.agent.lazy_loaded_tools_holder import LazyLoadedToolsHolder
from quickapp.core.agent.models import STATE_KEY_ORCHESTRATOR, OpenAiToolConfigDict
logger = logging.getLogger(__name__)
@@ -28,6 +29,7 @@ def __init__(
pre_invocation_transformers: list[PreInvocationTransformer],
presentation_settings: PresentationSettings,
forwarded_headers: ForwardedHeaders,
+ lazy_loaded_tools_holder: LazyLoadedToolsHolder,
) -> None:
self.__config: ApplicationConfig = config
self.__tools: list[OpenAiToolConfigDict] = tools
@@ -36,34 +38,22 @@ def __init__(
self.__pre_invocation_transformers = pre_invocation_transformers
self.__presentation_settings = presentation_settings
self.__forwarded_headers = forwarded_headers
+ self.__lazy_loaded_tools_holder = lazy_loaded_tools_holder
def build(self, messages: list[Message]) -> dict[str, Any]:
chat_completion_config = self.__config.orchestrator.deployment.parameters.model_dump(
exclude_none=True
)
prepared_messages = self._prepare_messages(messages)
+ all_tools = self._merge_tools()
payload: dict[str, Any] = {
"messages": prepared_messages,
"stream": True,
"model": self.__config.orchestrator.deployment.deployment_id,
- "tools": self.__tools,
+ "tools": all_tools,
}
- if self.__response_format:
- logger.debug("Setting response format (type=%s)", type(self.__response_format).__name__)
- log_payload(logger, "Response format: %s", self.__response_format)
- if hasattr(self.__response_format, "model_dump"):
- payload["response_format"] = self.__response_format.model_dump(
- exclude_none=True, mode="json"
- )
- elif isinstance(self.__response_format, dict):
- payload["response_format"] = self.__response_format
- else:
- logger.error(
- "Unsupported response format type: %s. The response format will not be applied.",
- type(self.__response_format),
- )
-
+ self._apply_response_format(payload)
self._apply_tool_choice(payload)
if self.__presentation_settings.show_usage_statistics:
@@ -73,13 +63,50 @@ def build(self, messages: list[Message]) -> dict[str, Any]:
payload["extra_headers"] = self.__forwarded_headers
chat_completion_config.update(payload)
+ self._log_result(chat_completion_config, prepared_messages, all_tools)
+ return chat_completion_config
+
+ def _merge_tools(self) -> list[OpenAiToolConfigDict]:
+ eager_names: set[str] = {t.get("function", {}).get("name", "") for t in self.__tools}
+ lazy_tools = [
+ t
+ for t in self.__lazy_loaded_tools_holder.get_all()
+ if t.get("function", {}).get("name", "") not in eager_names
+ ]
+ return self.__tools + lazy_tools
+
+ def _apply_response_format(self, payload: dict[str, Any]) -> None:
+ if not self.__response_format:
+ return
+ logger.debug("Setting response format (type=%s)", type(self.__response_format).__name__)
+ log_payload(logger, "Response format: %s", self.__response_format)
+ if hasattr(self.__response_format, "model_dump"):
+ payload["response_format"] = self.__response_format.model_dump(
+ exclude_none=True, mode="json"
+ )
+ elif isinstance(self.__response_format, dict):
+ payload["response_format"] = self.__response_format
+ else:
+ logger.error(
+ "Unsupported response format type: %s. The response format will not be applied.",
+ type(self.__response_format),
+ )
+
+ def _log_result(
+ self,
+ chat_completion_config: dict[str, Any],
+ prepared_messages: list[dict[str, Any]],
+ all_tools: list[OpenAiToolConfigDict],
+ ) -> None:
if logger.isEnabledFor(logging.DEBUG):
logger.debug(
- "Chat completion config: messages=%d, roles=%s, tools=%d, response_format=%s, "
+ "Chat completion config: messages=%d, roles=%s, tools=%d (eager=%d, lazy=%d), response_format=%s, "
"model=%s, forwarded_headers=%s",
len(prepared_messages),
summarize_roles(prepared_messages),
+ len(all_tools),
len(self.__tools),
+ len(all_tools) - len(self.__tools),
"response_format" in chat_completion_config,
chat_completion_config.get("model"),
# Header NAMES only — forwarded X-* header values are never logged, even
@@ -94,7 +121,6 @@ def build(self, messages: list[Message]) -> dict[str, Any]:
log_payload(
logger, "Chat completion config: %s", json.dumps(loggable, ensure_ascii=False)
)
- return chat_completion_config
def _apply_tool_choice(self, payload: dict[str, Any]) -> None:
tool_choice = self.__tool_choice_holder.consume()
diff --git a/src/quickapp/core/agent/agent_module.py b/src/quickapp/core/agent/agent_module.py
index a73c209f..eb2c330e 100644
--- a/src/quickapp/core/agent/agent_module.py
+++ b/src/quickapp/core/agent/agent_module.py
@@ -1,5 +1,3 @@
-import copy
-
from aidial_sdk.chat_completion.request import StaticTool
from aidial_sdk.exceptions import InvalidRequestError
from fastapi_injector import request_scope
@@ -25,6 +23,7 @@
from quickapp.common.chat_completion_recovery import ChatCompletionRecoveryService
from quickapp.common.chat_completion_stream.chat_stream_sink_factory import ChatStreamSinkFactory
from quickapp.common.chat_completion_stream.handler import ChatCompletionStreamHandler
+from quickapp.common.deferred_tool_types import DeferredToolName
from quickapp.common.dial_settings import DialSettings
from quickapp.common.request_async_close_registry import RequestAsyncCloseRegistry
from quickapp.common.stage_close_registry import DeferredStageCloseRegistry
@@ -35,10 +34,10 @@
BaseOpenAITool,
ConfigurableSchemaArray,
ConfigurableSchemaSimpleType,
- JsonSchemaConst,
JsonSchemaSimpleType,
JsonTypeEnum,
OpenAiToolConfig,
+ remove_const_schema_params,
)
from quickapp.config.tools.deployment import DialDeploymentTool
from quickapp.config.tools.display.paramenter import (
@@ -55,6 +54,7 @@
from quickapp.core.agent._prompt_providers import ConfigBasedPromptProvider
from quickapp.core.agent._tool_choice_holder import _ToolChoiceHolder
from quickapp.core.agent.assistant_invoker import AssistantInvoker
+from quickapp.core.agent.lazy_loaded_tools_holder import LazyLoadedToolsHolder
from quickapp.core.agent.models import OpenAiToolConfigDict
from quickapp.core.agent.orchestrator import Orchestrator
from quickapp.core.agent.orchestrator_capabilities import OrchestratorCapabilities
@@ -103,6 +103,7 @@ def configure(self, binder: Binder) -> None:
)
binder.bind(AssistantInvoker, to=AssistantInvoker, scope=NoScope)
binder.bind(_ChatCompletionConfigBuilder, to=_ChatCompletionConfigBuilder, scope=NoScope)
+ binder.bind(LazyLoadedToolsHolder, to=LazyLoadedToolsHolder, scope=request_scope)
binder.bind(ChatStreamSinkFactory, to=ChatStreamSinkFactory, scope=NoScope)
binder.bind(ChatCompletionStreamHandler, to=ChatCompletionStreamHandler, scope=NoScope)
binder.bind(_AttachmentFilter, to=_AttachmentFilter, scope=request_scope)
@@ -164,13 +165,19 @@ def provide_openai_client(
@multiprovider
def provide_openai_tools(
- self, tools: list[StagedBaseTool], static_tools: list[StaticTool]
+ self,
+ tools: list[StagedBaseTool],
+ static_tools: list[StaticTool],
+ deferred_tool_names: list[DeferredToolName],
) -> list[OpenAiToolConfigDict]:
+ deferred_names = frozenset(deferred_tool_names)
openai_functions = []
for tool in tools:
if isinstance(tool.tool_config, BaseOpenAITool):
open_ai_tool: OpenAiToolConfig = tool.tool_config.open_ai_tool
- open_ai_tool = self._remove_const_params(open_ai_tool)
+ if open_ai_tool.function.name in deferred_names:
+ continue
+ open_ai_tool = remove_const_schema_params(open_ai_tool)
if isinstance(tool.tool_config, DialDeploymentTool):
open_ai_tool = self._append_default_props(open_ai_tool)
open_ai_tool = tool.enrich_openai_tool_schema(open_ai_tool)
@@ -218,17 +225,6 @@ def provide_tool_names(self, context: _RequestContext) -> EXTERNAL_TOOL_NAMES:
t.function.name for t in context.extra_tools if t.function and t.function.name
)
- @staticmethod
- def _remove_const_params(open_ai_tool):
- tool_copy = copy.deepcopy(open_ai_tool)
- props = tool_copy.function.parameters.properties
-
- for prop_name in list(props.keys()):
- if issubclass(type(props[prop_name]), JsonSchemaConst):
- del props[prop_name]
-
- return tool_copy
-
@staticmethod
def _append_default_props(converted_open_ai_tool: OpenAiToolConfig):
if "query" not in converted_open_ai_tool.function.parameters.properties:
diff --git a/src/quickapp/core/agent/lazy_loaded_tools_holder.py b/src/quickapp/core/agent/lazy_loaded_tools_holder.py
new file mode 100644
index 00000000..ab007bd0
--- /dev/null
+++ b/src/quickapp/core/agent/lazy_loaded_tools_holder.py
@@ -0,0 +1,25 @@
+from injector import inject
+
+from quickapp.core.agent.models import OpenAiToolConfigDict
+
+
+@inject
+class LazyLoadedToolsHolder:
+ """Request-scoped accumulator for tool schemas discovered via tool_search.
+
+ _ToolSearchTool writes to this holder during execution.
+ _ChatCompletionConfigBuilder reads from it on every build() call and merges
+ the accumulated schemas into payload["tools"].
+ """
+
+ def __init__(self) -> None:
+ self._tools: dict[str, OpenAiToolConfigDict] = {}
+
+ def add(self, tools: list[OpenAiToolConfigDict]) -> None:
+ for tool in tools:
+ name: str = tool.get("function", {}).get("name", "")
+ if name:
+ self._tools[name] = tool
+
+ def get_all(self) -> list[OpenAiToolConfigDict]:
+ return list(self._tools.values())
diff --git a/src/quickapp/core/agent/models.py b/src/quickapp/core/agent/models.py
index 843e7b92..547ea3b7 100644
--- a/src/quickapp/core/agent/models.py
+++ b/src/quickapp/core/agent/models.py
@@ -1,6 +1,6 @@
-from typing import Any, TypeAlias
+from quickapp.config.tools.base import OpenAiToolConfigDict
TOOL_EXECUTION_HISTORY: str = "tool_execution_history"
STATE_KEY_ORCHESTRATOR: str = "orchestrator_state"
-OpenAiToolConfigDict: TypeAlias = dict[str, Any]
+__all__ = ["TOOL_EXECUTION_HISTORY", "STATE_KEY_ORCHESTRATOR", "OpenAiToolConfigDict"]
diff --git a/src/quickapp/internal_tooling/_internal_deferred_tools_context.py b/src/quickapp/internal_tooling/_internal_deferred_tools_context.py
new file mode 100644
index 00000000..6ad3d066
--- /dev/null
+++ b/src/quickapp/internal_tooling/_internal_deferred_tools_context.py
@@ -0,0 +1,5 @@
+from quickapp.common.deferred_tools_accumulator import DeferredToolsAccumulator
+
+
+class _InternalDeferredToolsContext(DeferredToolsAccumulator):
+ """Request-scoped deferred-tool registry owned by internal tooling."""
diff --git a/src/quickapp/internal_tooling/internal_tooling_module.py b/src/quickapp/internal_tooling/internal_tooling_module.py
index 851ce572..7ef1152e 100644
--- a/src/quickapp/internal_tooling/internal_tooling_module.py
+++ b/src/quickapp/internal_tooling/internal_tooling_module.py
@@ -4,11 +4,20 @@
from injector import AssistedBuilder, Binder, Module, multiprovider, provider, singleton
from quickapp.common import DIAL_API_KEY, StagedBaseTool
+from quickapp.common.deferred_tool_types import (
+ DeferredToolCatalogEntry,
+ DeferredToolDefinition,
+ DeferredToolName,
+ DeferredToolsetSummary,
+)
+from quickapp.common.deferred_tools_accumulator import is_toolset_deferred
from quickapp.common.dial_settings import DialSettings
+from quickapp.common.localized_string import resolve_localized
from quickapp.common.tool_names import INTERNAL_CODE_EXECUTION_PYTHON_INTERPRETER_TOOL_NAME
from quickapp.config.application import ApplicationConfig
from quickapp.config.tools.predefined import PredefinedTool
from quickapp.config.toolsets.internal import InternalToolSet
+from quickapp.internal_tooling._internal_deferred_tools_context import _InternalDeferredToolsContext
from quickapp.internal_tooling.py_interpreter_tooling._py_interpreter_client import (
_PyInterpreterClient,
)
@@ -36,6 +45,9 @@ def configure(self, binder: Binder) -> None:
binder.bind(SessionManager, to=SessionManager, scope=request_scope)
binder.bind(_PyInterpreterTool, to=_PyInterpreterTool, scope=request_scope)
binder.bind(InputFileHandler, to=InputFileHandler, scope=request_scope)
+ binder.bind(
+ _InternalDeferredToolsContext, to=_InternalDeferredToolsContext, scope=request_scope
+ )
logger.debug("InternalTooling module configuration completed")
@multiprovider
@@ -43,11 +55,13 @@ def _provide_internal_tools(
self,
app_config: ApplicationConfig,
py_builder: AssistedBuilder[_PyInterpreterTool],
+ deferred_context: _InternalDeferredToolsContext,
) -> list[StagedBaseTool]:
tools: list[StagedBaseTool] = []
for tool_set in app_config.tool_sets:
if isinstance(tool_set, InternalToolSet):
+ toolset_tools: list[StagedBaseTool] = []
for tool_config in tool_set.tools:
if tool_config.enabled:
if isinstance(tool_config, PredefinedTool):
@@ -58,7 +72,7 @@ def _provide_internal_tools(
INTERNAL_CODE_EXECUTION_PYTHON_INTERPRETER_TOOL_NAME
):
# TODO: remove this filtering by name, the user may configure any name of the tool.
- tools.append(
+ toolset_tools.append(
py_builder.build(
tool_config=tool_config,
name=tool_config.open_ai_tool.function.name,
@@ -66,8 +80,46 @@ def _provide_internal_tools(
)
)
+ discovery_cfg = app_config.orchestrator.tool_discovery
+ if is_toolset_deferred(tool_set, discovery_cfg, len(toolset_tools)):
+ deferred_context.register_deferred_tools(tool_set, toolset_tools)
+ logger.debug(
+ "Deferred %d tools from internal toolset '%s' into the deferred tools registry",
+ len(toolset_tools),
+ resolve_localized(tool_set.name),
+ )
+ tools.extend(toolset_tools)
+
return tools
+ @multiprovider
+ def _provide_deferred_tool_names(
+ self, deferred_context: _InternalDeferredToolsContext
+ ) -> list[DeferredToolName]:
+ return list(deferred_context.deferred_names)
+
+ @multiprovider
+ def _provide_deferred_catalog_entries(
+ self, deferred_context: _InternalDeferredToolsContext
+ ) -> list[DeferredToolCatalogEntry]:
+ return deferred_context.catalog
+
+ @multiprovider
+ def _provide_deferred_tool_definitions(
+ self, deferred_context: _InternalDeferredToolsContext
+ ) -> list[DeferredToolDefinition]:
+ return [
+ DeferredToolDefinition(name=name, definition=definition)
+ for name in deferred_context.deferred_names
+ if (definition := deferred_context.get_definition(name)) is not None
+ ]
+
+ @multiprovider
+ def _provide_deferred_toolset_summaries(
+ self, deferred_context: _InternalDeferredToolsContext
+ ) -> list[DeferredToolsetSummary]:
+ return deferred_context.toolset_summaries
+
@singleton
@provider
def _provide_py_interpreter_settings(
diff --git a/src/quickapp/mcp_tooling/_mcp_tool_initializer.py b/src/quickapp/mcp_tooling/_mcp_tool_initializer.py
index b03c5596..d20192d3 100644
--- a/src/quickapp/mcp_tooling/_mcp_tool_initializer.py
+++ b/src/quickapp/mcp_tooling/_mcp_tool_initializer.py
@@ -11,11 +11,13 @@
from quickapp.common import ACCEPT_LANGUAGE, DIAL_API_KEY, StagedBaseTool
from quickapp.common.base_initializer import CompletionInitializer
+from quickapp.common.deferred_tools_accumulator import is_toolset_deferred
from quickapp.common.dial_settings import DialSettings
from quickapp.common.exceptions import ToolInitializationException
from quickapp.common.json_schema_converter import JsonSchemaConverter
from quickapp.common.localized_string import resolve_localized
from quickapp.common.utils import posix_path_last_segment, sanitize_toolname
+from quickapp.config.application import ApplicationConfig
from quickapp.config.tools.base import (
JsonTypeEnum,
OpenAiToolConfig,
@@ -131,6 +133,7 @@ def __init__(
tool_config_service: ToolConfigCoreService,
login_service: InteractiveLoginService,
accept_language: ACCEPT_LANGUAGE,
+ app_config: ApplicationConfig,
):
# Resolved lazily in initialize() because dial_app_tooling contributes
# to this multibinder only after _DialAppResolver runs.
@@ -146,6 +149,7 @@ def __init__(
self.__tool_config_service: ToolConfigCoreService = tool_config_service
self.__login_service: InteractiveLoginService = login_service
self.__accept_language: ACCEPT_LANGUAGE = accept_language
+ self.__app_config: ApplicationConfig = app_config
@staticmethod
# todo add Title to config so that we could use it in stage name
@@ -284,6 +288,14 @@ async def _load_tools(
)
created_tools.append(mcp_tool)
if created_tools:
+ discovery_cfg = self.__app_config.orchestrator.tool_discovery
+ if is_toolset_deferred(toolset_info, discovery_cfg, len(created_tools)):
+ self.__mcp_context.register_deferred_tools(toolset_info, created_tools)
+ logger.debug(
+ "Deferred %d tools from MCP toolset '%s' into the deferred tools registry",
+ len(created_tools),
+ resolve_localized(resolved_toolset.name),
+ )
self.__mcp_context.extend_tools(created_tools)
async def _load_resources(
diff --git a/src/quickapp/mcp_tooling/_mcp_tooling_context.py b/src/quickapp/mcp_tooling/_mcp_tooling_context.py
index 974ef571..ccba8221 100644
--- a/src/quickapp/mcp_tooling/_mcp_tooling_context.py
+++ b/src/quickapp/mcp_tooling/_mcp_tooling_context.py
@@ -1,3 +1,4 @@
+from quickapp.common.deferred_tools_accumulator import DeferredToolsAccumulator
from quickapp.common.tooling_context_base import ToolingContextBase
from quickapp.mcp_tooling._mcp_eager_resource import MCPEagerResource
from quickapp.mcp_tooling._mcp_resource_meta import MCPResourceMeta
@@ -5,9 +6,10 @@
from quickapp.mcp_tooling._mcp_toolset_client import _MCPToolsetClient
-class _MCPToolingContext(ToolingContextBase):
+class _MCPToolingContext(ToolingContextBase, DeferredToolsAccumulator):
def __init__(self) -> None:
- super().__init__()
+ ToolingContextBase.__init__(self)
+ DeferredToolsAccumulator.__init__(self)
self._resource_metas: list[MCPResourceMeta] = []
self._eager_resources: list[MCPEagerResource] = []
self._server_capabilities: list[MCPServerCapabilities] = []
diff --git a/src/quickapp/mcp_tooling/mcp_tooling_module.py b/src/quickapp/mcp_tooling/mcp_tooling_module.py
index dc4e9bed..c86473da 100644
--- a/src/quickapp/mcp_tooling/mcp_tooling_module.py
+++ b/src/quickapp/mcp_tooling/mcp_tooling_module.py
@@ -7,6 +7,12 @@
from quickapp.common.abstract.base_prompt_provider import PromptPartProvider
from quickapp.common.abstract.base_transformer import MessagesTransformer
from quickapp.common.base_initializer import CompletionInitializer
+from quickapp.common.deferred_tool_types import (
+ DeferredToolCatalogEntry,
+ DeferredToolDefinition,
+ DeferredToolName,
+ DeferredToolsetSummary,
+)
from quickapp.common.exceptions import InitializationException
from quickapp.common.tool_names import INTERNAL_MCP_READ_RESOURCE_TOOL_NAME
from quickapp.config.application import ApplicationConfig
@@ -74,6 +80,34 @@ def __provide_initializers(
def _provide_mcp_tools(self, mcp_context: _MCPToolingContext) -> list[StagedBaseTool]:
return mcp_context.tools
+ @multiprovider
+ def _provide_deferred_tool_names(
+ self, mcp_context: _MCPToolingContext
+ ) -> list[DeferredToolName]:
+ return list(mcp_context.deferred_names)
+
+ @multiprovider
+ def _provide_deferred_catalog_entries(
+ self, mcp_context: _MCPToolingContext
+ ) -> list[DeferredToolCatalogEntry]:
+ return mcp_context.catalog
+
+ @multiprovider
+ def _provide_deferred_tool_definitions(
+ self, mcp_context: _MCPToolingContext
+ ) -> list[DeferredToolDefinition]:
+ return [
+ DeferredToolDefinition(name=name, definition=definition)
+ for name in mcp_context.deferred_names
+ if (definition := mcp_context.get_definition(name)) is not None
+ ]
+
+ @multiprovider
+ def _provide_deferred_toolset_summaries(
+ self, mcp_context: _MCPToolingContext
+ ) -> list[DeferredToolsetSummary]:
+ return mcp_context.toolset_summaries
+
@multiprovider
def __provide_initialization_exceptions(
self, context: _MCPToolingContext
diff --git a/src/quickapp/rest_api_tooling/_rest_api_deferred_tools_context.py b/src/quickapp/rest_api_tooling/_rest_api_deferred_tools_context.py
new file mode 100644
index 00000000..cdff893f
--- /dev/null
+++ b/src/quickapp/rest_api_tooling/_rest_api_deferred_tools_context.py
@@ -0,0 +1,5 @@
+from quickapp.common.deferred_tools_accumulator import DeferredToolsAccumulator
+
+
+class _RestApiDeferredToolsContext(DeferredToolsAccumulator):
+ """Request-scoped deferred-tool registry owned by REST API tooling."""
diff --git a/src/quickapp/rest_api_tooling/rest_api_tooling_module.py b/src/quickapp/rest_api_tooling/rest_api_tooling_module.py
index 71d553b2..a84ac793 100644
--- a/src/quickapp/rest_api_tooling/rest_api_tooling_module.py
+++ b/src/quickapp/rest_api_tooling/rest_api_tooling_module.py
@@ -4,6 +4,13 @@
from injector import Binder, ClassAssistedBuilder, Module, multiprovider
from quickapp.common import ACCEPT_LANGUAGE, StagedBaseTool
+from quickapp.common.deferred_tool_types import (
+ DeferredToolCatalogEntry,
+ DeferredToolDefinition,
+ DeferredToolName,
+ DeferredToolsetSummary,
+)
+from quickapp.common.deferred_tools_accumulator import is_toolset_deferred
from quickapp.common.localized_string import resolve_localized
from quickapp.common.oauth_token_fetcher import OAuthTokenFetcher
from quickapp.common.utils import sanitize_toolname
@@ -12,6 +19,7 @@
from quickapp.config.toolsets.rest_api import RestApiToolSet
from ._request_detail_builder import _RequestDetailsBuilder
+from ._rest_api_deferred_tools_context import _RestApiDeferredToolsContext
from ._rest_api_stage_wrapper import _RestApiStageWrapper
from ._rest_api_tool import _RestApiTool
@@ -25,6 +33,9 @@ def configure(self, binder: Binder) -> None:
binder.bind(_RestApiTool, to=_RestApiTool, scope=request_scope)
binder.bind(_RequestDetailsBuilder, to=_RequestDetailsBuilder)
binder.bind(OAuthTokenFetcher, to=OAuthTokenFetcher)
+ binder.bind(
+ _RestApiDeferredToolsContext, to=_RestApiDeferredToolsContext, scope=request_scope
+ )
logger.debug("RestApiTooling module configuration completed")
@multiprovider
@@ -33,16 +44,52 @@ def __provide_rest_api_tools(
app_config: ApplicationConfig,
tool_builder: ClassAssistedBuilder[_RestApiTool],
accept_language: ACCEPT_LANGUAGE,
+ deferred_context: _RestApiDeferredToolsContext,
) -> list[StagedBaseTool]:
result: list[StagedBaseTool] = []
for toolset_info in app_config.tool_sets:
if isinstance(toolset_info, RestApiToolSet) and toolset_info.enabled:
toolset_stage_name = resolve_localized(toolset_info.name, accept_language)
- result.extend(
- self.__create_rest_api_tools(toolset_info, tool_builder, toolset_stage_name)
- )
+ tools = self.__create_rest_api_tools(toolset_info, tool_builder, toolset_stage_name)
+ discovery_cfg = app_config.orchestrator.tool_discovery
+ if is_toolset_deferred(toolset_info, discovery_cfg, len(tools)):
+ deferred_context.register_deferred_tools(toolset_info, tools)
+ logger.debug(
+ "Deferred %d tools from REST toolset '%s' into the deferred tools registry",
+ len(tools),
+ toolset_stage_name,
+ )
+ result.extend(tools)
return result
+ @multiprovider
+ def _provide_deferred_tool_names(
+ self, deferred_context: _RestApiDeferredToolsContext
+ ) -> list[DeferredToolName]:
+ return list(deferred_context.deferred_names)
+
+ @multiprovider
+ def _provide_deferred_catalog_entries(
+ self, deferred_context: _RestApiDeferredToolsContext
+ ) -> list[DeferredToolCatalogEntry]:
+ return deferred_context.catalog
+
+ @multiprovider
+ def _provide_deferred_tool_definitions(
+ self, deferred_context: _RestApiDeferredToolsContext
+ ) -> list[DeferredToolDefinition]:
+ return [
+ DeferredToolDefinition(name=name, definition=definition)
+ for name in deferred_context.deferred_names
+ if (definition := deferred_context.get_definition(name)) is not None
+ ]
+
+ @multiprovider
+ def _provide_deferred_toolset_summaries(
+ self, deferred_context: _RestApiDeferredToolsContext
+ ) -> list[DeferredToolsetSummary]:
+ return deferred_context.toolset_summaries
+
@staticmethod
def __create_rest_api_tools(
rest_api_toolset: RestApiToolSet,
diff --git a/src/quickapp/tool_discovery/__init__.py b/src/quickapp/tool_discovery/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/src/quickapp/tool_discovery/_anonymous_agent.py b/src/quickapp/tool_discovery/_anonymous_agent.py
new file mode 100644
index 00000000..adcc5c5c
--- /dev/null
+++ b/src/quickapp/tool_discovery/_anonymous_agent.py
@@ -0,0 +1,79 @@
+import json
+import logging
+
+import openai
+from injector import inject
+
+from quickapp.common import ORCHESTRATOR_AZURE_CLIENT
+from quickapp.config.application import ApplicationConfig
+
+logger = logging.getLogger(__name__)
+
+_ROUTING_SYSTEM_PROMPT = (
+ "You are a tool routing assistant. "
+ "Given a user query, return the names of the tools from the provided catalog "
+ "that are most relevant to fulfilling that query. "
+ "Respond with a JSON array of tool name strings only — no explanation, no markdown. "
+ "Example: [\"tool_a\", \"tool_b\"]\n\n"
+ "Catalog:\n{catalog}"
+)
+
+
+@inject
+class _AnonymousAgent:
+ """Fires an isolated, non-streaming LLM call to route a query against the deferred tool catalog.
+
+ No conversation history or application system prompt is included — token cost is bounded
+ by the catalog size alone.
+ """
+
+ def __init__(
+ self,
+ client: ORCHESTRATOR_AZURE_CLIENT,
+ config: ApplicationConfig,
+ ) -> None:
+ self.__client = client
+ self.__config = config
+
+ async def route(self, query: str, catalog: list[dict[str, str]]) -> list[str]:
+ """Return tool names from catalog that best match query."""
+ if not catalog:
+ return []
+
+ discovery = self.__config.orchestrator.tool_discovery
+ if discovery is None:
+ logger.warning(
+ "Anonymous agent routing call invoked while tool_discovery is disabled — "
+ "returning no matches"
+ )
+ return []
+ service_model = (
+ discovery.service_model or self.__config.orchestrator.deployment.deployment_id
+ )
+
+ catalog_text = "\n".join(
+ f"- {entry['name']}: {entry.get('description', '')}" for entry in catalog
+ )
+ system_content = _ROUTING_SYSTEM_PROMPT.format(catalog=catalog_text)
+
+ try:
+ response = await self.__client.chat.completions.create(
+ model=service_model,
+ messages=[
+ {"role": "system", "content": system_content},
+ {"role": "user", "content": query},
+ ],
+ stream=False,
+ )
+ except openai.OpenAIError:
+ logger.exception("Anonymous agent routing call failed")
+ return []
+
+ raw = (response.choices[0].message.content or "").strip()
+ try:
+ names = json.loads(raw)
+ if isinstance(names, list):
+ return [n for n in names if isinstance(n, str)]
+ except (json.JSONDecodeError, ValueError):
+ logger.warning("Anonymous agent returned non-JSON response (length=%d)", len(raw))
+ return []
diff --git a/src/quickapp/tool_discovery/_tool_configs.py b/src/quickapp/tool_discovery/_tool_configs.py
new file mode 100644
index 00000000..9cbcb3e7
--- /dev/null
+++ b/src/quickapp/tool_discovery/_tool_configs.py
@@ -0,0 +1,51 @@
+from quickapp.common.tool_names import INTERNAL_TOOL_SEARCH_TOOL_NAME
+from quickapp.config.tools.base import (
+ ConfigurableSchemaSimpleType,
+ JsonTypeEnum,
+ OpenAiToolConfig,
+ OpenAiToolFunction,
+ OpenAiToolFunctionParameters,
+)
+from quickapp.config.tools.display.tool import ToolDisplayConfig, ToolStageConfig
+from quickapp.config.tools.internal import InternalTool
+
+TOOL_SEARCH_TOOL_CONFIG = InternalTool(
+ open_ai_tool=OpenAiToolConfig(
+ function=OpenAiToolFunction(
+ name=INTERNAL_TOOL_SEARCH_TOOL_NAME,
+ description=("""
+Purpose: Discover additional tools and capabilities.
+
+You MUST call this tool before stating any limitation or saying you cannot fulfill a user request.
+
+In addition, you MUST call this tool whenever:
+- The user’s request is open-ended or underspecified.
+- The request might involve capabilities beyond your currently listed tools.
+- The request could plausibly be served by a specialized tool or by returning a resource, even if you believe you can respond text-only.
+
+If a user request might involve capabilities beyond your current listed tools, you are REQUIRED to:
+1) Call internal_tool_search with a brief description of the needed capability.
+2) Inspect any returned tools.
+
+You may NOT:
+- Assume that the initially listed tools are exhaustive.
+- Say "I can't", "I don't have access", or express similar limitations until you have called internal_tool_search in this conversation turn.
+
+Only if internal_tool_search returns no suitable tools, or all relevant tools fail, may you tell the user you cannot do it.
+
+This tool dynamically discovers additional toolsets (including hidden or MCP tools) that may provide additional capabilities:
+"""),
+ parameters=OpenAiToolFunctionParameters(
+ type=JsonTypeEnum.object,
+ properties={
+ "query": ConfigurableSchemaSimpleType(
+ type=JsonTypeEnum.string,
+ description="A natural-language description of the capability you need.",
+ )
+ },
+ required=["query"],
+ ),
+ )
+ ),
+ display=ToolDisplayConfig(stage=ToolStageConfig(name="Searching tools")),
+)
diff --git a/src/quickapp/tool_discovery/_tool_search_hint_prompt_provider.py b/src/quickapp/tool_discovery/_tool_search_hint_prompt_provider.py
new file mode 100644
index 00000000..3fdd12a4
--- /dev/null
+++ b/src/quickapp/tool_discovery/_tool_search_hint_prompt_provider.py
@@ -0,0 +1,34 @@
+from injector import inject
+
+from quickapp.common.abstract.base_prompt_provider import PromptPartProvider
+from quickapp.common.deferred_tool_types import DeferredToolsetSummary
+from quickapp.common.tool_names import INTERNAL_TOOL_SEARCH_TOOL_NAME
+from quickapp.tool_discovery._toolset_summary_format import format_toolset_summaries
+
+_TOOL_SEARCH_HINT = (
+ "If a user request might involve capabilities beyond your current listed tools, you are "
+ "REQUIRED to:\n"
+ f"1) Call `{INTERNAL_TOOL_SEARCH_TOOL_NAME}` with a brief description of the needed capability.\n"
+ "2) Inspect any returned tools."
+)
+
+
+@inject
+class _ToolSearchHintPromptProvider(PromptPartProvider):
+ """Reminds the orchestrator to try tool discovery before declaring a limitation, and lists
+ the toolsets currently withheld from the initial payload (name, tool count, description).
+ """
+
+ def __init__(self, toolset_summaries: list[DeferredToolsetSummary]) -> None:
+ self.__toolset_summaries = toolset_summaries
+
+ async def get_prompt_part(self) -> str:
+ summaries = self.__toolset_summaries
+ if not summaries:
+ return _TOOL_SEARCH_HINT
+
+ return (
+ _TOOL_SEARCH_HINT
+ + "\n\nAdditional toolsets available for discovery:\n"
+ + format_toolset_summaries(summaries)
+ )
diff --git a/src/quickapp/tool_discovery/_tool_search_stage_wrapper.py b/src/quickapp/tool_discovery/_tool_search_stage_wrapper.py
new file mode 100644
index 00000000..803079dc
--- /dev/null
+++ b/src/quickapp/tool_discovery/_tool_search_stage_wrapper.py
@@ -0,0 +1,19 @@
+from typing import Any
+
+from injector import inject
+
+from quickapp.common import TimedStageWrapper, ToolCallResult
+
+
+@inject
+class _ToolSearchStageWrapper(TimedStageWrapper):
+
+ def _get_formatted_parameters(self, parameters: dict[str, Any]) -> str:
+ query = parameters.get("query", "")
+ return f"> ##### Query:\n{query}\n" if query else ""
+
+ def _build_debug_info_from_exception(self, exception: Exception) -> str:
+ return f"> ##### Exception:\n{exception}\n"
+
+ def _build_debug_info_from_result(self, result: ToolCallResult) -> str:
+ return f"> ##### Discovered tools:\n{result.content}\n"
diff --git a/src/quickapp/tool_discovery/_tool_search_tool.py b/src/quickapp/tool_discovery/_tool_search_tool.py
new file mode 100644
index 00000000..0398745f
--- /dev/null
+++ b/src/quickapp/tool_discovery/_tool_search_tool.py
@@ -0,0 +1,118 @@
+import json
+import logging
+from typing import Any
+
+from injector import AssistedBuilder, inject
+
+from quickapp.common import StagedBaseTool, ToolCallResult
+from quickapp.common.abstract.base_tool_argument_transformer import ToolArgumentTransformer
+from quickapp.common.base_stage_wrapper import BaseStageWrapper
+from quickapp.common.deferred_tool_types import (
+ DeferredToolCatalogEntry,
+ DeferredToolDefinition,
+ DeferredToolsetSummary,
+)
+from quickapp.common.perf_timer.perf_timer import PerformanceTimer
+from quickapp.config.application import StageDisplayLevel
+from quickapp.config.tools.base import OpenAiToolConfig, OpenAiToolConfigDict
+from quickapp.config.tools.internal import InternalTool
+from quickapp.core.agent.lazy_loaded_tools_holder import LazyLoadedToolsHolder
+from quickapp.tool_discovery._anonymous_agent import _AnonymousAgent
+from quickapp.tool_discovery._tool_search_stage_wrapper import _ToolSearchStageWrapper
+from quickapp.tool_discovery._toolset_summary_format import format_toolset_summaries
+
+logger = logging.getLogger(__name__)
+
+
+@inject
+class _ToolSearchTool(StagedBaseTool):
+ """Meta-tool that discovers deferred tools on demand via an anonymous LLM routing call."""
+
+ def __init__(
+ self,
+ stage_wrapper_builder: AssistedBuilder[_ToolSearchStageWrapper],
+ tool_config: InternalTool,
+ perf_timer: PerformanceTimer,
+ catalog: list[DeferredToolCatalogEntry],
+ definitions: list[DeferredToolDefinition],
+ toolset_summaries: list[DeferredToolsetSummary],
+ lazy_holder: LazyLoadedToolsHolder,
+ anonymous_agent: _AnonymousAgent,
+ stage_display_level: StageDisplayLevel = StageDisplayLevel.INFO,
+ argument_transformers: list[ToolArgumentTransformer] | None = None,
+ **kwargs: Any,
+ ):
+ super().__init__(
+ stage_wrapper_builder=stage_wrapper_builder, # type: ignore[arg-type]
+ tool_config=tool_config,
+ perf_timer=perf_timer,
+ stage_display_level=stage_display_level,
+ argument_transformers=argument_transformers,
+ **kwargs,
+ )
+ self.__catalog = catalog
+ self.__toolset_summaries = toolset_summaries
+ self.__definitions_by_name: dict[str, OpenAiToolConfigDict] = {
+ definition.name: definition.definition for definition in definitions
+ }
+ self.__lazy_holder = lazy_holder
+ self.__anonymous_agent = anonymous_agent
+
+ def enrich_openai_tool_schema(self, open_ai_tool: OpenAiToolConfig) -> OpenAiToolConfig:
+ summaries = self.__toolset_summaries
+ if not summaries:
+ return open_ai_tool
+
+ open_ai_tool.function.description = (
+ open_ai_tool.function.description
+ + "\n\nAdditional toolsets available for discovery:\n"
+ + format_toolset_summaries(summaries)
+ )
+ return open_ai_tool
+
+ async def _run_in_stage_async(
+ self,
+ stage_wrapper: BaseStageWrapper | None = None,
+ tool_call_id: str | None = None,
+ *args: Any,
+ **kwargs: Any,
+ ) -> ToolCallResult:
+ query: str = kwargs.get("query", "")
+ catalog = self.__catalog
+
+ if not catalog:
+ result = ToolCallResult(
+ content="No additional tools are available for discovery.",
+ content_type="text/plain",
+ )
+ if stage_wrapper:
+ stage_wrapper.add_result(result)
+ return result
+
+ matched_names = await self.__anonymous_agent.route(query, catalog)
+
+ discovered: list[dict[str, str]] = []
+ new_definitions = []
+ for name in matched_names:
+ definition = self.__definitions_by_name.get(name)
+ if definition is None:
+ logger.warning("tool_search matched unknown tool name %r — skipping", name)
+ continue
+ description: str = definition.get("function", {}).get("description", "")
+ discovered.append({"name": name, "description": description})
+ new_definitions.append(definition)
+
+ if new_definitions:
+ self.__lazy_holder.add(new_definitions)
+
+ if discovered:
+ content = json.dumps(discovered, ensure_ascii=False)
+ content_type = "application/json"
+ else:
+ content = "No matching tools found for the given query."
+ content_type = "text/plain"
+
+ result = ToolCallResult(content=content, content_type=content_type)
+ if stage_wrapper:
+ stage_wrapper.add_result(result)
+ return result
diff --git a/src/quickapp/tool_discovery/_toolset_summary_format.py b/src/quickapp/tool_discovery/_toolset_summary_format.py
new file mode 100644
index 00000000..852fd4ff
--- /dev/null
+++ b/src/quickapp/tool_discovery/_toolset_summary_format.py
@@ -0,0 +1,11 @@
+def format_toolset_summaries(summaries: list[dict[str, str | int | None]]) -> str:
+ """Render deferred-toolset summaries as a bullet list: name, tool count, description."""
+ lines = [_format_one(summary) for summary in summaries]
+ return "\n".join(lines)
+
+
+def _format_one(summary: dict[str, str | int | None]) -> str:
+ line = f"- {summary['name']}. Available tools: {summary['tool_count']}."
+ if summary["description"]:
+ line += f" {summary['description']}"
+ return line
diff --git a/src/quickapp/tool_discovery/tool_discovery_module.py b/src/quickapp/tool_discovery/tool_discovery_module.py
new file mode 100644
index 00000000..16f8c461
--- /dev/null
+++ b/src/quickapp/tool_discovery/tool_discovery_module.py
@@ -0,0 +1,59 @@
+import logging
+
+from fastapi_injector import request_scope
+from injector import AssistedBuilder, Binder, Module, multiprovider
+
+from quickapp.common import StagedBaseTool
+from quickapp.common.abstract.base_prompt_provider import PromptPartProvider
+from quickapp.common.preview import preview_module
+from quickapp.config.application import ApplicationConfig
+from quickapp.tool_discovery._anonymous_agent import _AnonymousAgent
+from quickapp.tool_discovery._tool_configs import TOOL_SEARCH_TOOL_CONFIG
+from quickapp.tool_discovery._tool_search_hint_prompt_provider import _ToolSearchHintPromptProvider
+from quickapp.tool_discovery._tool_search_stage_wrapper import _ToolSearchStageWrapper
+from quickapp.tool_discovery._tool_search_tool import _ToolSearchTool
+
+logger = logging.getLogger(__name__)
+
+
+@preview_module
+class ToolDiscoveryModule(Module):
+
+ def configure(self, binder: Binder) -> None:
+ binder.bind(_AnonymousAgent, to=_AnonymousAgent, scope=request_scope)
+ binder.bind(_ToolSearchTool, to=_ToolSearchTool, scope=request_scope)
+ binder.bind(_ToolSearchStageWrapper, to=_ToolSearchStageWrapper)
+ binder.bind(
+ _ToolSearchHintPromptProvider, to=_ToolSearchHintPromptProvider, scope=request_scope
+ )
+
+ @staticmethod
+ def _is_enabled(config: ApplicationConfig) -> bool:
+ return bool(
+ config.orchestrator.tool_discovery and config.orchestrator.tool_discovery.enabled
+ )
+
+ @multiprovider
+ def _provide_tool_search_tools(
+ self,
+ config: ApplicationConfig,
+ tool_builder: AssistedBuilder[_ToolSearchTool],
+ ) -> list[StagedBaseTool]:
+ if not self._is_enabled(config):
+ return []
+
+ tool = tool_builder.build(
+ tool_config=TOOL_SEARCH_TOOL_CONFIG,
+ )
+ logger.debug("ToolDiscoveryModule: tool_search meta-tool registered")
+ return [tool]
+
+ @multiprovider
+ def _provide_prompt_parts(
+ self,
+ config: ApplicationConfig,
+ tool_search_hint: _ToolSearchHintPromptProvider,
+ ) -> list[PromptPartProvider]:
+ if not self._is_enabled(config):
+ return []
+ return [tool_search_hint]
diff --git a/src/scripts/dump_internal_tools.py b/src/scripts/dump_internal_tools.py
index 7e4a4b46..ae1d6860 100644
--- a/src/scripts/dump_internal_tools.py
+++ b/src/scripts/dump_internal_tools.py
@@ -43,6 +43,7 @@
from quickapp.config.orchestrator_attachment_strategy import LazyOnDemandAttachmentStrategy
from quickapp.config.prompt import CustomSystemPromptConfig
from quickapp.config.timestamp import ToolCallTimestampConfig
+from quickapp.config.tool_discovery import ToolDiscoveryConfig
from quickapp.config.tools.const import ALL_MIME_TYPES
from quickapp.config.web_fetch import WebFetchConfig
from quickapp.core.agent import OrchestratorCapabilities
@@ -71,6 +72,7 @@ def build_dump_application_config() -> ApplicationConfig:
variables={},
),
attachment_strategy=LazyOnDemandAttachmentStrategy(),
+ tool_discovery=ToolDiscoveryConfig(enabled=True),
),
contexts=[
FileContextConfig(
diff --git a/src/tests/integration_tests/test_runner/mcp_server/mcp_http_test_server.py b/src/tests/integration_tests/test_runner/mcp_server/mcp_http_test_server.py
index e3fdecdc..f08b3cdb 100644
--- a/src/tests/integration_tests/test_runner/mcp_server/mcp_http_test_server.py
+++ b/src/tests/integration_tests/test_runner/mcp_server/mcp_http_test_server.py
@@ -65,7 +65,7 @@ async def sum_integers(incoming: list[int]):
@mcp.tool(description="Returns a predefined small picture")
async def get_small_picture() -> Image:
- return Image(path="auto.jpg")
+ return Image(path="./auto.jpg")
def __get_file_data(file_path: str) -> str:
@@ -87,7 +87,7 @@ async def get_test_pdf() -> EmbeddedResource:
return EmbeddedResource(
type="resource",
resource=BlobResourceContents(
- blob=__get_file_data("mcp_pdf.pdf"),
+ blob=__get_file_data("./mcp_pdf.pdf"),
uri=AnyUrl("file://test/test.pdf"),
mimeType="application/pdf",
),
@@ -99,7 +99,7 @@ async def get_test_plotly() -> EmbeddedResource:
return EmbeddedResource(
type="resource",
resource=BlobResourceContents(
- blob=__get_file_data("plotly.json"),
+ blob=__get_file_data("./plotly.json"),
uri=AnyUrl("file://test/plotly.json"),
mimeType="application/vnd.plotly.v1+json",
),
@@ -124,4 +124,6 @@ def get_config() -> dict:
if __name__ == "__main__":
+ # result = asyncio.run(get_small_picture())
+ # print(result)
mcp.run(transport="streamable-http", host="0.0.0.0", port=8003, log_level="debug")
diff --git a/src/tests/unit_tests/agent_tests/test_assistant_invoker.py b/src/tests/unit_tests/agent_tests/test_assistant_invoker.py
index afe368af..c2e737ab 100644
--- a/src/tests/unit_tests/agent_tests/test_assistant_invoker.py
+++ b/src/tests/unit_tests/agent_tests/test_assistant_invoker.py
@@ -10,6 +10,7 @@
from quickapp.core.agent import AssistantInvoker
from quickapp.core.agent._chat_completion_config_builder import _ChatCompletionConfigBuilder
from quickapp.core.agent._tool_choice_holder import _ToolChoiceHolder
+from quickapp.core.agent.lazy_loaded_tools_holder import LazyLoadedToolsHolder
def _presentation_settings(show_usage: bool):
@@ -69,6 +70,7 @@ def _make_config_builder(
pre_invocation_transformers=[mock_filter],
presentation_settings=_presentation_settings(show_usage),
forwarded_headers=forwarded_headers,
+ lazy_loaded_tools_holder=LazyLoadedToolsHolder(),
)
diff --git a/src/tests/unit_tests/agent_tests/test_chat_completion_config_builder.py b/src/tests/unit_tests/agent_tests/test_chat_completion_config_builder.py
index 6c1ec7f0..f54bcee8 100644
--- a/src/tests/unit_tests/agent_tests/test_chat_completion_config_builder.py
+++ b/src/tests/unit_tests/agent_tests/test_chat_completion_config_builder.py
@@ -1,7 +1,71 @@
+from unittest.mock import MagicMock
+
+from aidial_sdk.chat_completion import Message, Role
+
from quickapp.core.agent._chat_completion_config_builder import _ChatCompletionConfigBuilder
+from quickapp.core.agent.lazy_loaded_tools_holder import LazyLoadedToolsHolder
from quickapp.core.agent.models import STATE_KEY_ORCHESTRATOR as ORCH
+def _make_tool_dict(name: str) -> dict:
+ return {"type": "function", "function": {"name": name, "description": "", "parameters": {}}}
+
+
+def _make_builder(
+ tools: list[dict], lazy_holder: LazyLoadedToolsHolder
+) -> _ChatCompletionConfigBuilder:
+ config = MagicMock()
+ config.orchestrator.deployment.parameters.model_dump.return_value = {}
+ config.orchestrator.deployment.deployment_id = "orchestrator-model"
+ tool_choice_holder = MagicMock()
+ tool_choice_holder.consume.return_value = None
+ presentation_settings = MagicMock()
+ presentation_settings.show_usage_statistics = False
+ return _ChatCompletionConfigBuilder(
+ config=config,
+ tools=tools,
+ response_format=None,
+ tool_choice_holder=tool_choice_holder,
+ pre_invocation_transformers=[],
+ presentation_settings=presentation_settings,
+ forwarded_headers=None,
+ lazy_loaded_tools_holder=lazy_holder,
+ )
+
+
+def test_build_merges_lazy_tools_after_eager_tools():
+ """Discovered (lazy) tools are appended to the eager tools in the outgoing payload."""
+ lazy_holder = LazyLoadedToolsHolder()
+ lazy_holder.add([_make_tool_dict("discovered_tool")])
+ builder = _make_builder([_make_tool_dict("eager_tool")], lazy_holder)
+
+ payload = builder.build([Message(role=Role.USER, content="hi")])
+
+ names = [t["function"]["name"] for t in payload["tools"]]
+ assert names == ["eager_tool", "discovered_tool"]
+
+
+def test_build_dedupes_lazy_tools_against_eager_names():
+ """A lazy tool whose name collides with an eager tool is dropped, not duplicated."""
+ lazy_holder = LazyLoadedToolsHolder()
+ lazy_holder.add([_make_tool_dict("shared_name"), _make_tool_dict("discovered_tool")])
+ builder = _make_builder([_make_tool_dict("shared_name")], lazy_holder)
+
+ payload = builder.build([Message(role=Role.USER, content="hi")])
+
+ names = [t["function"]["name"] for t in payload["tools"]]
+ assert names == ["shared_name", "discovered_tool"]
+
+
+def test_build_with_no_lazy_tools_returns_eager_tools_only():
+ lazy_holder = LazyLoadedToolsHolder()
+ builder = _make_builder([_make_tool_dict("eager_tool")], lazy_holder)
+
+ payload = builder.build([Message(role=Role.USER, content="hi")])
+
+ assert [t["function"]["name"] for t in payload["tools"]] == ["eager_tool"]
+
+
def test_promote_orchestrator_state_to_top_level():
"""Before the next orchestrator call, state.orchestrator (response state only) is promoted to top-level."""
msg = {
diff --git a/src/tests/unit_tests/agent_tests/test_tool_choice_config_builder.py b/src/tests/unit_tests/agent_tests/test_tool_choice_config_builder.py
index b15ca310..df462657 100644
--- a/src/tests/unit_tests/agent_tests/test_tool_choice_config_builder.py
+++ b/src/tests/unit_tests/agent_tests/test_tool_choice_config_builder.py
@@ -6,6 +6,7 @@
from quickapp.core.agent._chat_completion_config_builder import _ChatCompletionConfigBuilder
from quickapp.core.agent._tool_choice_holder import _ToolChoiceHolder
+from quickapp.core.agent.lazy_loaded_tools_holder import LazyLoadedToolsHolder
def _make_builder(
@@ -24,6 +25,7 @@ def _make_builder(
pre_invocation_transformers=[],
presentation_settings=MagicMock(show_usage_statistics=False),
forwarded_headers=None,
+ lazy_loaded_tools_holder=LazyLoadedToolsHolder(),
)
@@ -70,6 +72,7 @@ def test_tool_choice_consumed_only_on_first_build(self):
pre_invocation_transformers=[],
presentation_settings=MagicMock(show_usage_statistics=False),
forwarded_headers=None,
+ lazy_loaded_tools_holder=LazyLoadedToolsHolder(),
)
first = builder.build([])
assert first["tool_choice"] == "required"
diff --git a/src/tests/unit_tests/common/test_deferred_tools_accumulator.py b/src/tests/unit_tests/common/test_deferred_tools_accumulator.py
new file mode 100644
index 00000000..83313670
--- /dev/null
+++ b/src/tests/unit_tests/common/test_deferred_tools_accumulator.py
@@ -0,0 +1,230 @@
+from unittest.mock import MagicMock
+
+from quickapp.common.deferred_tools_accumulator import DeferredToolsAccumulator, is_toolset_deferred
+from quickapp.config.tool_discovery import ToolDiscoveryConfig
+from quickapp.config.tools.base import (
+ OpenAiToolConfig,
+ OpenAiToolFunction,
+ OpenAiToolFunctionParameters,
+)
+from quickapp.config.tools.rest_api import (
+ RestApiEndpointConstParam,
+ RestApiEndpointHeaderParamInfo,
+ RestApiEndpointMethodInfo,
+ RestApiEndpointSimpleTypeParam,
+ RestApiTool,
+ ToolEndpointInfoMethodType,
+ ToolEndpointParamType,
+)
+from quickapp.config.toolsets.rest_api import RestApiToolSet
+
+
+def _make_toolset(deferred: bool | None = None) -> RestApiToolSet:
+ return RestApiToolSet(name="my-toolset", deferred=deferred, tools=[])
+
+
+def _make_discovery_config(
+ enabled: bool = True, min_tools_for_deferral: int = 5
+) -> ToolDiscoveryConfig:
+ return ToolDiscoveryConfig(enabled=enabled, min_tools_for_deferral=min_tools_for_deferral)
+
+
+class TestIsToolsetDeferred:
+ def test_deferred_true_above_threshold_is_deferred(self):
+ assert (
+ is_toolset_deferred(_make_toolset(True), _make_discovery_config(), tool_count=5) is True
+ )
+
+ def test_unset_defaults_to_deferred(self):
+ """`deferred` unset (None) behaves the same as `deferred: true`."""
+ assert (
+ is_toolset_deferred(_make_toolset(None), _make_discovery_config(), tool_count=5) is True
+ )
+
+ def test_explicit_false_is_never_deferred_even_above_threshold(self):
+ assert (
+ is_toolset_deferred(_make_toolset(False), _make_discovery_config(), tool_count=50)
+ is False
+ )
+
+ def test_below_threshold_is_not_deferred(self):
+ assert (
+ is_toolset_deferred(_make_toolset(True), _make_discovery_config(), tool_count=4)
+ is False
+ )
+
+ def test_at_threshold_boundary_is_deferred(self):
+ cfg = _make_discovery_config(min_tools_for_deferral=5)
+ assert is_toolset_deferred(_make_toolset(True), cfg, tool_count=5) is True
+
+ def test_discovery_disabled_is_never_deferred(self):
+ cfg = _make_discovery_config(enabled=False)
+ assert is_toolset_deferred(_make_toolset(True), cfg, tool_count=50) is False
+
+ def test_discovery_config_none_is_never_deferred(self):
+ assert is_toolset_deferred(_make_toolset(True), None, tool_count=50) is False
+
+
+def _make_rest_api_tool_with_const_param(name: str = "test_tool") -> RestApiTool:
+ return RestApiTool(
+ rest_api_method_info=RestApiEndpointMethodInfo(
+ method_url="https://example.com", method_type=ToolEndpointInfoMethodType.get
+ ),
+ open_ai_tool=OpenAiToolConfig(
+ function=OpenAiToolFunction(
+ name=name,
+ description="A test tool",
+ parameters=OpenAiToolFunctionParameters(
+ type="object",
+ properties={
+ "query": RestApiEndpointSimpleTypeParam(
+ type="string",
+ description="Query param",
+ parameter_info=RestApiEndpointHeaderParamInfo(
+ type=ToolEndpointParamType.query, key="query"
+ ),
+ ),
+ "api_key": RestApiEndpointConstParam(
+ type=None,
+ const="secret-value",
+ parameter_info=RestApiEndpointHeaderParamInfo(
+ type=ToolEndpointParamType.header, key="X-Api-Key"
+ ),
+ ),
+ },
+ ),
+ )
+ ),
+ )
+
+
+def _make_staged_tool(tool_config, enrich_side_effect=None) -> MagicMock:
+ staged_tool = MagicMock()
+ staged_tool.tool_config = tool_config
+ staged_tool.enrich_openai_tool_schema.side_effect = enrich_side_effect or (lambda t: t)
+ return staged_tool
+
+
+class TestRegisterDeferredTools:
+ def test_catalog_contains_name_and_description(self):
+ context = DeferredToolsAccumulator()
+ tool_config = _make_rest_api_tool_with_const_param()
+ context.register_deferred_tools(_make_toolset(), [_make_staged_tool(tool_config)])
+
+ assert context.catalog == [{"name": "test_tool", "description": "A test tool"}]
+
+ def test_deferred_names_reflects_registered_tools(self):
+ context = DeferredToolsAccumulator()
+ context.register_deferred_tools(
+ _make_toolset(), [_make_staged_tool(_make_rest_api_tool_with_const_param("tool_a"))]
+ )
+
+ assert context.deferred_names == frozenset({"tool_a"})
+
+ def test_definition_strips_const_params_same_as_eager_path(self):
+ """Regression: a discovered tool's schema must match the eager path — const params
+ (fixed values hidden from the LLM) must not leak into the schema surfaced via tool_search.
+ """
+ context = DeferredToolsAccumulator()
+ tool_config = _make_rest_api_tool_with_const_param()
+ context.register_deferred_tools(_make_toolset(), [_make_staged_tool(tool_config)])
+
+ definition = context.get_definition("test_tool")
+
+ assert definition is not None
+ properties = definition["function"]["parameters"]["properties"]
+ assert "query" in properties
+ assert "api_key" not in properties
+
+ def test_definition_applies_enrich_openai_tool_schema(self):
+ """The per-tool enrich_openai_tool_schema hook runs for deferred tools too."""
+
+ def enrich(open_ai_tool: OpenAiToolConfig) -> OpenAiToolConfig:
+ enriched = open_ai_tool.model_copy(deep=True)
+ enriched.function.description = "enriched"
+ return enriched
+
+ context = DeferredToolsAccumulator()
+ tool_config = _make_rest_api_tool_with_const_param()
+ context.register_deferred_tools(
+ _make_toolset(), [_make_staged_tool(tool_config, enrich_side_effect=enrich)]
+ )
+
+ definition = context.get_definition("test_tool")
+ assert definition is not None
+ assert definition["function"]["description"] == "enriched"
+
+ def test_get_definition_returns_none_for_unknown_name(self):
+ context = DeferredToolsAccumulator()
+ assert context.get_definition("does_not_exist") is None
+
+ def test_ignores_tools_without_openai_tool_config(self):
+ context = DeferredToolsAccumulator()
+ non_openai_staged_tool = MagicMock()
+ non_openai_staged_tool.tool_config = MagicMock() # not a BaseOpenAITool instance
+
+ context.register_deferred_tools(_make_toolset(), [non_openai_staged_tool])
+
+ assert context.catalog == []
+ assert context.deferred_names == frozenset()
+
+
+class TestToolsetSummaries:
+ def test_summary_includes_name_description_and_tool_count(self):
+ context = DeferredToolsAccumulator()
+ toolset = RestApiToolSet(
+ name="salesforce", description="Query Salesforce records", tools=[]
+ )
+ context.register_deferred_tools(
+ toolset, [_make_staged_tool(_make_rest_api_tool_with_const_param())]
+ )
+
+ assert context.toolset_summaries == [
+ {"name": "salesforce", "description": "Query Salesforce records", "tool_count": 1}
+ ]
+
+ def test_summary_is_name_only_when_description_is_none(self):
+ context = DeferredToolsAccumulator()
+ toolset = RestApiToolSet(name="salesforce", description=None, tools=[])
+ context.register_deferred_tools(
+ toolset, [_make_staged_tool(_make_rest_api_tool_with_const_param())]
+ )
+
+ assert context.toolset_summaries == [
+ {"name": "salesforce", "description": None, "tool_count": 1}
+ ]
+
+ def test_tool_count_reflects_number_of_registered_tools(self):
+ context = DeferredToolsAccumulator()
+ toolset = RestApiToolSet(
+ name="salesforce", description="Query Salesforce records", tools=[]
+ )
+ context.register_deferred_tools(
+ toolset,
+ [
+ _make_staged_tool(_make_rest_api_tool_with_const_param("tool_a")),
+ _make_staged_tool(_make_rest_api_tool_with_const_param("tool_b")),
+ _make_staged_tool(_make_rest_api_tool_with_const_param("tool_c")),
+ ],
+ )
+
+ assert context.toolset_summaries[0]["tool_count"] == 3
+
+ def test_summaries_accumulate_across_multiple_toolsets(self):
+ context = DeferredToolsAccumulator()
+ context.register_deferred_tools(
+ RestApiToolSet(name="toolset_a", description="Does A", tools=[]),
+ [_make_staged_tool(_make_rest_api_tool_with_const_param("tool_a"))],
+ )
+ context.register_deferred_tools(
+ RestApiToolSet(name="toolset_b", description=None, tools=[]),
+ [
+ _make_staged_tool(_make_rest_api_tool_with_const_param("tool_b1")),
+ _make_staged_tool(_make_rest_api_tool_with_const_param("tool_b2")),
+ ],
+ )
+
+ assert context.toolset_summaries == [
+ {"name": "toolset_a", "description": "Does A", "tool_count": 1},
+ {"name": "toolset_b", "description": None, "tool_count": 2},
+ ]
diff --git a/src/tests/unit_tests/internal_tooling_tests/test_internal_tooling_module.py b/src/tests/unit_tests/internal_tooling_tests/test_internal_tooling_module.py
new file mode 100644
index 00000000..f767f03d
--- /dev/null
+++ b/src/tests/unit_tests/internal_tooling_tests/test_internal_tooling_module.py
@@ -0,0 +1,87 @@
+from unittest.mock import MagicMock
+
+from quickapp.common.tool_names import INTERNAL_CODE_EXECUTION_PYTHON_INTERPRETER_TOOL_NAME
+from quickapp.config.tool_discovery import ToolDiscoveryConfig
+from quickapp.config.tools.base import (
+ OpenAiToolConfig,
+ OpenAiToolFunction,
+ OpenAiToolFunctionParameters,
+)
+from quickapp.config.tools.internal import InternalTool
+from quickapp.config.toolsets.internal import InternalToolSet
+from quickapp.internal_tooling._internal_deferred_tools_context import _InternalDeferredToolsContext
+from quickapp.internal_tooling.internal_tooling_module import InternalToolModule
+
+
+def _make_internal_tool_config(
+ name: str = INTERNAL_CODE_EXECUTION_PYTHON_INTERPRETER_TOOL_NAME,
+) -> InternalTool:
+ return InternalTool(
+ open_ai_tool=OpenAiToolConfig(
+ function=OpenAiToolFunction(
+ name=name,
+ description="Runs python code",
+ parameters=OpenAiToolFunctionParameters(type="object", properties={}),
+ )
+ )
+ )
+
+
+def _make_app_config(
+ toolset: InternalToolSet, discovery_cfg: ToolDiscoveryConfig | None
+) -> MagicMock:
+ app_config = MagicMock()
+ app_config.tool_sets = [toolset]
+ app_config.orchestrator.tool_discovery = discovery_cfg
+ return app_config
+
+
+def _make_py_builder(tool_config: InternalTool) -> MagicMock:
+ staged_tool = MagicMock()
+ staged_tool.tool_config = tool_config
+ builder = MagicMock()
+ builder.build.return_value = staged_tool
+ return staged_tool, builder
+
+
+class TestProvideInternalTools:
+ def test_registers_deferred_toolset_with_deferred_context(self):
+ tool_config = _make_internal_tool_config()
+ toolset = InternalToolSet(name="internal", deferred=True, tools=[tool_config])
+ discovery_cfg = ToolDiscoveryConfig(enabled=True, min_tools_for_deferral=1)
+ app_config = _make_app_config(toolset, discovery_cfg)
+ staged_tool, py_builder = _make_py_builder(tool_config)
+ deferred_context = MagicMock(spec=_InternalDeferredToolsContext)
+
+ module = InternalToolModule()
+ result = module._provide_internal_tools(app_config, py_builder, deferred_context)
+
+ assert result == [staged_tool]
+ deferred_context.register_deferred_tools.assert_called_once_with(toolset, [staged_tool])
+
+ def test_does_not_defer_below_threshold(self):
+ tool_config = _make_internal_tool_config()
+ toolset = InternalToolSet(name="internal", deferred=True, tools=[tool_config])
+ discovery_cfg = ToolDiscoveryConfig(enabled=True, min_tools_for_deferral=5)
+ app_config = _make_app_config(toolset, discovery_cfg)
+ staged_tool, py_builder = _make_py_builder(tool_config)
+ deferred_context = MagicMock(spec=_InternalDeferredToolsContext)
+
+ module = InternalToolModule()
+ result = module._provide_internal_tools(app_config, py_builder, deferred_context)
+
+ assert result == [staged_tool]
+ deferred_context.register_deferred_tools.assert_not_called()
+
+ def test_does_not_defer_when_discovery_disabled(self):
+ tool_config = _make_internal_tool_config()
+ toolset = InternalToolSet(name="internal", deferred=True, tools=[tool_config])
+ app_config = _make_app_config(toolset, discovery_cfg=None)
+ staged_tool, py_builder = _make_py_builder(tool_config)
+ deferred_context = MagicMock(spec=_InternalDeferredToolsContext)
+
+ module = InternalToolModule()
+ result = module._provide_internal_tools(app_config, py_builder, deferred_context)
+
+ assert result == [staged_tool]
+ deferred_context.register_deferred_tools.assert_not_called()
diff --git a/src/tests/unit_tests/mcp_tool_tests/test_mcp_initializer_interactive_login.py b/src/tests/unit_tests/mcp_tool_tests/test_mcp_initializer_interactive_login.py
index 95d13b4e..4ad2059c 100644
--- a/src/tests/unit_tests/mcp_tool_tests/test_mcp_initializer_interactive_login.py
+++ b/src/tests/unit_tests/mcp_tool_tests/test_mcp_initializer_interactive_login.py
@@ -128,6 +128,7 @@ def _make_initializer(
tool_config_service=MagicMock(),
login_service=login_service,
accept_language=None,
+ app_config=MagicMock(),
)
return initializer, mcp_context, login_service
diff --git a/src/tests/unit_tests/mcp_tool_tests/test_mcp_tool_initializer.py b/src/tests/unit_tests/mcp_tool_tests/test_mcp_tool_initializer.py
index ce6302f9..d0a13e22 100644
--- a/src/tests/unit_tests/mcp_tool_tests/test_mcp_tool_initializer.py
+++ b/src/tests/unit_tests/mcp_tool_tests/test_mcp_tool_initializer.py
@@ -29,6 +29,13 @@
from tests.unit_tests.common.common import make_provider, noop_timeout_resolver
+def _make_app_config_mock() -> MagicMock:
+ """Return an app_config mock with tool discovery disabled."""
+ m = MagicMock()
+ m.orchestrator.tool_discovery.enabled = False
+ return m
+
+
def _setup_open_init_session(conn: MagicMock, supports_tools: bool = True) -> MagicMock:
"""Configure conn.open_init_session to yield (mock_session, mock_init_result)."""
session = MagicMock()
@@ -227,6 +234,7 @@ def _create(protocol: MCPProtocol, allowed_tools=None, name="test_toolset"):
MagicMock(), # tool_config_service
MagicMock(), # login_service
None, # accept_language
+ _make_app_config_mock(), # app_config
)
return initializer, mcp_context
@@ -314,6 +322,7 @@ async def test_initialize_multiple_toolsets(tool1, tool2, builder_mock):
MagicMock(), # tool_config_service
MagicMock(), # login_service
None, # accept_language
+ _make_app_config_mock(), # app_config
)
await initializer.initialize()
@@ -403,6 +412,7 @@ async def test_no_exception_if_toolset_list_is_empty():
MagicMock(), # tool_config_service
MagicMock(), # login_service
None, # accept_language
+ _make_app_config_mock(), # app_config
)
await initializer.initialize()
mcp_context.append_tool.assert_not_called()
@@ -594,6 +604,7 @@ async def test_initialize_surfaces_session_terminated_through_nested_exception_g
MagicMock(), # tool_config_service
MagicMock(), # login_service
None, # accept_language
+ _make_app_config_mock(), # app_config
)
await initializer.initialize()
diff --git a/src/tests/unit_tests/rest_api_tooling_tests/test_rest_api_tool.py b/src/tests/unit_tests/rest_api_tooling_tests/test_rest_api_tool.py
index 1e253fd1..11c79df7 100644
--- a/src/tests/unit_tests/rest_api_tooling_tests/test_rest_api_tool.py
+++ b/src/tests/unit_tests/rest_api_tooling_tests/test_rest_api_tool.py
@@ -6,7 +6,7 @@
from aidial_sdk.chat_completion import Attachment, Stage
from fastapi_injector import Injected
from httpx import QueryParams
-from injector import Binder, Injector, InstanceProvider
+from injector import Binder, InstanceProvider
from pydantic import SecretStr
from starlette.testclient import TestClient
@@ -434,12 +434,18 @@ def configure(binder: Binder):
binder.bind(ACCEPT_LANGUAGE, to=InstanceProvider(None))
binder.multibind(list[ToolArgumentTransformer], to=[])
- injector = Injector(modules=[RestApiToolingModule, configure])
- tools = injector.get(list[StagedBaseTool])
+ app = create_test_app([RestApiToolingModule, configure])
+
+ @app.get("/")
+ async def get_method(tools: list[StagedBaseTool] = Injected(list[StagedBaseTool])):
+ assert len(tools) == 1
+ tool_config: BaseOpenAITool = tools[0].tool_config
+ assert tool_config.open_ai_tool.function.name == f"{toolset_name}_{tool_name}"
+ return {}
- assert len(tools) == 1
- tool_config: BaseOpenAITool = tools[0].tool_config
- assert tool_config.open_ai_tool.function.name == f"{toolset_name}_{tool_name}"
+ client = TestClient(app)
+ response = client.get("/")
+ assert response.status_code == 200
@pytest.mark.asyncio
diff --git a/src/tests/unit_tests/tool_discovery_tests/__init__.py b/src/tests/unit_tests/tool_discovery_tests/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/src/tests/unit_tests/tool_discovery_tests/test_anonymous_agent.py b/src/tests/unit_tests/tool_discovery_tests/test_anonymous_agent.py
new file mode 100644
index 00000000..d2f1d6a1
--- /dev/null
+++ b/src/tests/unit_tests/tool_discovery_tests/test_anonymous_agent.py
@@ -0,0 +1,125 @@
+from unittest.mock import AsyncMock, MagicMock
+
+import openai
+import pytest
+
+from quickapp.tool_discovery._anonymous_agent import _AnonymousAgent
+
+
+def _make_config(
+ tool_discovery=None, service_model=None, orchestrator_deployment_id="orchestrator-model"
+):
+ config = MagicMock()
+ config.orchestrator.deployment.deployment_id = orchestrator_deployment_id
+ if tool_discovery is None:
+ config.orchestrator.tool_discovery = None
+ else:
+ discovery = MagicMock()
+ discovery.service_model = service_model
+ config.orchestrator.tool_discovery = discovery
+ return config
+
+
+def _make_response(content: str):
+ response = MagicMock()
+ response.choices = [MagicMock()]
+ response.choices[0].message.content = content
+ return response
+
+
+CATALOG = [{"name": "tool_a", "description": "Does A"}, {"name": "tool_b", "description": "Does B"}]
+
+
+@pytest.mark.asyncio
+async def test_route_returns_empty_list_for_empty_catalog():
+ client = AsyncMock()
+ agent = _AnonymousAgent(client=client, config=_make_config())
+
+ result = await agent.route("find a tool", [])
+
+ assert result == []
+ client.chat.completions.create.assert_not_called()
+
+
+@pytest.mark.asyncio
+async def test_route_fails_open_when_tool_discovery_is_none():
+ client = AsyncMock()
+ agent = _AnonymousAgent(client=client, config=_make_config(tool_discovery=None))
+
+ result = await agent.route("find a tool", CATALOG)
+
+ assert result == []
+ client.chat.completions.create.assert_not_called()
+
+
+@pytest.mark.asyncio
+async def test_route_returns_matched_names_on_valid_json_response():
+ client = AsyncMock()
+ client.chat.completions.create.return_value = _make_response('["tool_a", "tool_b"]')
+ agent = _AnonymousAgent(client=client, config=_make_config(tool_discovery=True))
+
+ result = await agent.route("find a tool", CATALOG)
+
+ assert result == ["tool_a", "tool_b"]
+
+
+@pytest.mark.asyncio
+async def test_route_uses_service_model_when_configured():
+ client = AsyncMock()
+ client.chat.completions.create.return_value = _make_response("[]")
+ agent = _AnonymousAgent(
+ client=client, config=_make_config(tool_discovery=True, service_model="cheap-router-model")
+ )
+
+ await agent.route("find a tool", CATALOG)
+
+ assert client.chat.completions.create.call_args.kwargs["model"] == "cheap-router-model"
+
+
+@pytest.mark.asyncio
+async def test_route_falls_back_to_orchestrator_deployment_when_service_model_unset():
+ client = AsyncMock()
+ client.chat.completions.create.return_value = _make_response("[]")
+ agent = _AnonymousAgent(
+ client=client,
+ config=_make_config(
+ tool_discovery=True, service_model=None, orchestrator_deployment_id="orchestrator-model"
+ ),
+ )
+
+ await agent.route("find a tool", CATALOG)
+
+ assert client.chat.completions.create.call_args.kwargs["model"] == "orchestrator-model"
+
+
+@pytest.mark.asyncio
+async def test_route_returns_empty_list_on_non_json_response():
+ client = AsyncMock()
+ client.chat.completions.create.return_value = _make_response("not json at all")
+ agent = _AnonymousAgent(client=client, config=_make_config(tool_discovery=True))
+
+ result = await agent.route("find a tool", CATALOG)
+
+ assert result == []
+
+
+@pytest.mark.asyncio
+async def test_route_filters_out_non_string_entries_from_response():
+ client = AsyncMock()
+ client.chat.completions.create.return_value = _make_response('["tool_a", 123, null]')
+ agent = _AnonymousAgent(client=client, config=_make_config(tool_discovery=True))
+
+ result = await agent.route("find a tool", CATALOG)
+
+ assert result == ["tool_a"]
+
+
+@pytest.mark.asyncio
+async def test_route_returns_empty_list_on_openai_error():
+ client = AsyncMock()
+ client.chat.completions.create.side_effect = openai.APIConnectionError(request=MagicMock())
+ agent = _AnonymousAgent(client=client, config=_make_config(tool_discovery=True))
+
+ result = await agent.route("find a tool", CATALOG)
+
+ assert result == []
diff --git a/src/tests/unit_tests/tool_discovery_tests/test_tool_discovery_module.py b/src/tests/unit_tests/tool_discovery_tests/test_tool_discovery_module.py
new file mode 100644
index 00000000..e11506bd
--- /dev/null
+++ b/src/tests/unit_tests/tool_discovery_tests/test_tool_discovery_module.py
@@ -0,0 +1,60 @@
+from unittest.mock import MagicMock
+
+from quickapp.tool_discovery._tool_search_hint_prompt_provider import _ToolSearchHintPromptProvider
+from quickapp.tool_discovery.tool_discovery_module import ToolDiscoveryModule
+
+
+def _make_config(enabled: bool | None) -> MagicMock:
+ config = MagicMock()
+ if enabled is None:
+ config.orchestrator.tool_discovery = None
+ else:
+ config.orchestrator.tool_discovery.enabled = enabled
+ return config
+
+
+class TestProvidePromptParts:
+ def test_includes_hint_when_discovery_enabled(self):
+ module = ToolDiscoveryModule()
+ tool_search_hint = MagicMock(spec=_ToolSearchHintPromptProvider)
+
+ result = module._provide_prompt_parts(_make_config(True), tool_search_hint)
+
+ assert result == [tool_search_hint]
+
+ def test_omits_hint_when_discovery_disabled(self):
+ module = ToolDiscoveryModule()
+ tool_search_hint = MagicMock(spec=_ToolSearchHintPromptProvider)
+
+ result = module._provide_prompt_parts(_make_config(False), tool_search_hint)
+
+ assert result == []
+
+ def test_omits_hint_when_discovery_unset(self):
+ module = ToolDiscoveryModule()
+ tool_search_hint = MagicMock(spec=_ToolSearchHintPromptProvider)
+
+ result = module._provide_prompt_parts(_make_config(None), tool_search_hint)
+
+ assert result == []
+
+
+class TestProvideToolSearchTools:
+ def test_returns_empty_list_when_discovery_disabled(self):
+ module = ToolDiscoveryModule()
+ tool_builder = MagicMock()
+
+ result = module._provide_tool_search_tools(_make_config(False), tool_builder)
+
+ assert result == []
+ tool_builder.build.assert_not_called()
+
+ def test_builds_tool_when_discovery_enabled(self):
+ module = ToolDiscoveryModule()
+ tool_builder = MagicMock()
+ built_tool = MagicMock()
+ tool_builder.build.return_value = built_tool
+
+ result = module._provide_tool_search_tools(_make_config(True), tool_builder)
+
+ assert result == [built_tool]
diff --git a/src/tests/unit_tests/tool_discovery_tests/test_tool_search_hint_prompt_provider.py b/src/tests/unit_tests/tool_discovery_tests/test_tool_search_hint_prompt_provider.py
new file mode 100644
index 00000000..65785f7c
--- /dev/null
+++ b/src/tests/unit_tests/tool_discovery_tests/test_tool_search_hint_prompt_provider.py
@@ -0,0 +1,43 @@
+import pytest
+
+from quickapp.tool_discovery._tool_search_hint_prompt_provider import _ToolSearchHintPromptProvider
+
+
+def _make_provider(
+ toolset_summaries: list[dict[str, str | int | None]],
+) -> _ToolSearchHintPromptProvider:
+ return _ToolSearchHintPromptProvider(toolset_summaries)
+
+
+@pytest.mark.asyncio
+async def test_get_prompt_part_mentions_internal_tool_search():
+ provider = _make_provider([])
+
+ part = await provider.get_prompt_part()
+
+ assert "internal_tool_search" in part
+
+
+@pytest.mark.asyncio
+async def test_get_prompt_part_omits_toolset_list_when_nothing_deferred():
+ provider = _make_provider([])
+
+ part = await provider.get_prompt_part()
+
+ assert "Additional toolsets available for discovery" not in part
+
+
+@pytest.mark.asyncio
+async def test_get_prompt_part_appends_deferred_toolset_summaries():
+ provider = _make_provider(
+ [
+ {"name": "salesforce", "description": "Query Salesforce records", "tool_count": 12},
+ {"name": "internal-utils", "description": None, "tool_count": 1},
+ ]
+ )
+
+ part = await provider.get_prompt_part()
+
+ assert "Additional toolsets available for discovery:" in part
+ assert "- salesforce. Available tools: 12. Query Salesforce records" in part
+ assert "- internal-utils. Available tools: 1." in part
diff --git a/src/tests/unit_tests/tool_discovery_tests/test_tool_search_tool.py b/src/tests/unit_tests/tool_discovery_tests/test_tool_search_tool.py
new file mode 100644
index 00000000..a7f48a2b
--- /dev/null
+++ b/src/tests/unit_tests/tool_discovery_tests/test_tool_search_tool.py
@@ -0,0 +1,87 @@
+from unittest.mock import MagicMock
+
+from quickapp.config.tools.base import (
+ OpenAiToolConfig,
+ OpenAiToolFunction,
+ OpenAiToolFunctionParameters,
+)
+from quickapp.tool_discovery._tool_search_tool import _ToolSearchTool
+
+
+def _make_open_ai_tool(description: str = "Search for additional tools.") -> OpenAiToolConfig:
+ return OpenAiToolConfig(
+ function=OpenAiToolFunction(
+ name="internal_tool_search",
+ description=description,
+ parameters=OpenAiToolFunctionParameters(type="object", properties={}),
+ )
+ )
+
+
+def _make_tool_search_tool(toolset_summaries: list[dict[str, str | int | None]]) -> _ToolSearchTool:
+ return _ToolSearchTool(
+ stage_wrapper_builder=MagicMock(),
+ tool_config=MagicMock(),
+ perf_timer=MagicMock(),
+ catalog=[],
+ definitions=[],
+ toolset_summaries=toolset_summaries,
+ lazy_holder=MagicMock(),
+ anonymous_agent=MagicMock(),
+ )
+
+
+class TestEnrichOpenAiToolSchema:
+ def test_no_op_when_no_deferred_toolsets(self):
+ tool = _make_tool_search_tool([])
+ open_ai_tool = _make_open_ai_tool()
+
+ result = tool.enrich_openai_tool_schema(open_ai_tool)
+
+ assert result.function.description == "Search for additional tools."
+
+ def test_appends_toolset_with_description_and_tool_count(self):
+ tool = _make_tool_search_tool(
+ [{"name": "salesforce", "description": "Query Salesforce records", "tool_count": 12}]
+ )
+ open_ai_tool = _make_open_ai_tool("Search for additional tools.")
+
+ result = tool.enrich_openai_tool_schema(open_ai_tool)
+
+ assert result.function.description == (
+ "Search for additional tools.\n\n"
+ "Additional toolsets available for discovery:\n"
+ "- salesforce. Available tools: 12. Query Salesforce records"
+ )
+
+ def test_appends_toolset_name_and_count_only_when_description_is_none(self):
+ tool = _make_tool_search_tool(
+ [{"name": "internal-utils", "description": None, "tool_count": 1}]
+ )
+ open_ai_tool = _make_open_ai_tool("Search for additional tools.")
+
+ result = tool.enrich_openai_tool_schema(open_ai_tool)
+
+ assert result.function.description == (
+ "Search for additional tools.\n\n"
+ "Additional toolsets available for discovery:\n"
+ "- internal-utils. Available tools: 1."
+ )
+
+ def test_appends_multiple_toolsets_in_order(self):
+ tool = _make_tool_search_tool(
+ [
+ {"name": "toolset_a", "description": "Does A", "tool_count": 3},
+ {"name": "toolset_b", "description": None, "tool_count": 7},
+ ]
+ )
+ open_ai_tool = _make_open_ai_tool("Search for additional tools.")
+
+ result = tool.enrich_openai_tool_schema(open_ai_tool)
+
+ assert result.function.description == (
+ "Search for additional tools.\n\n"
+ "Additional toolsets available for discovery:\n"
+ "- toolset_a. Available tools: 3. Does A\n"
+ "- toolset_b. Available tools: 7."
+ )
diff --git a/src/tests/unit_tests/tool_discovery_tests/test_toolset_summary_format.py b/src/tests/unit_tests/tool_discovery_tests/test_toolset_summary_format.py
new file mode 100644
index 00000000..1e6eff36
--- /dev/null
+++ b/src/tests/unit_tests/tool_discovery_tests/test_toolset_summary_format.py
@@ -0,0 +1,32 @@
+from quickapp.tool_discovery._toolset_summary_format import format_toolset_summaries
+
+
+def test_formats_summary_with_description():
+ result = format_toolset_summaries(
+ [{"name": "salesforce", "description": "Query Salesforce records", "tool_count": 12}]
+ )
+
+ assert result == "- salesforce. Available tools: 12. Query Salesforce records"
+
+
+def test_formats_summary_without_description():
+ result = format_toolset_summaries(
+ [{"name": "internal-utils", "description": None, "tool_count": 1}]
+ )
+
+ assert result == "- internal-utils. Available tools: 1."
+
+
+def test_formats_multiple_summaries_in_order():
+ result = format_toolset_summaries(
+ [
+ {"name": "toolset_a", "description": "Does A", "tool_count": 3},
+ {"name": "toolset_b", "description": None, "tool_count": 7},
+ ]
+ )
+
+ assert result == "- toolset_a. Available tools: 3. Does A\n- toolset_b. Available tools: 7."
+
+
+def test_formats_empty_list():
+ assert format_toolset_summaries([]) == ""