feat: add schema-driven reasoning and improve agent reliability

[trankien84]

Summary

This PR adds runtime schema-driven native reasoning support and improves reliability for long-running agent workflows.

The main changes are:

1. Native thinking and reasoning effort support based on each model's runtime schema.
2. Opt-in diagnostics for reasoning, payloads, streams, and chunk normalization.
3. Reliability improvements for agent tool execution, long streams, and profile region discovery.

---

1. Schema-driven native reasoning

Problem

Kiro exposes model-specific request capabilities through additionalModelRequestFieldsSchema, but the proxy did not preserve or use that schema when building requests.

As a result:

• native thinking and effort fields could not be forwarded reliably;
• supported effort values and field paths had to be assumed;
• Claude Messages, OpenAI Chat Completions, and Responses requests behaved differently;
• the legacy -thinking suffix relied mainly on prompt injection instead of model-native request fields.

Solution

The proxy now preserves additionalModelRequestFieldsSchema from ListAvailableModels and parses it at runtime.

The parsed capability includes:

• supported thinking types;
• supported thinking display values;
• support for budget_tokens;
• the native effort path:
  • output_config.effort, or
  • reasoning.effort;
• supported effort values such as low, medium, high, xhigh, and max.

Requested values are validated against the selected model's schema before the upstream request is sent.

API coverage

Native reasoning is supported across:

• Anthropic Messages API;
• OpenAI Chat Completions API;
• OpenAI Responses API;
• Claude token-count requests.

The generated fields are forwarded through
additionalModelRequestFields.

Backward compatibility

The configured thinking suffix, such as -thinking, remains supported.

When compatible native reasoning fields are available, they are forwarded directly and the legacy thinking prompt is not injected.

When native fields are unavailable, the existing legacy thinking prompt remains the fallback.

---

2. Environment-gated diagnostics

Problem

Investigating issues such as truncated payloads, incomplete streams, reasoning forwarding, and chunk normalization previously required adding temporary logs directly to the request path.

Solution

This PR adds opt-in diagnostic flags:

Variable	Description	Default
KIRO_DIAGNOSTICS	Enables stream, payload, and reasoning diagnostics	false
KIRO_DIAG_STREAM	Logs aggregate stream metrics	false
KIRO_DIAG_PAYLOAD	Logs payload size and truncation metrics	false
KIRO_DIAG_REASONING	Logs reasoning capabilities and forwarded fields	false
KIRO_DIAG_CHUNKS	Logs raw and normalized stream chunks; may contain sensitive content	false

Accepted enabled values are 1, true, yes, on, and enabled.

KIRO_DIAGNOSTICS intentionally does not enable
KIRO_DIAG_CHUNKS, because chunk logs may contain prompts, reasoning, source code, or tool output.

All diagnostics remain disabled by default.

---

3. Agent tool-execution reliability

Problem

During coding-agent workflows, a model may describe or plan a tool action without actually invoking the required tool, causing the turn to end before the requested work is complete.

Solution

Tool-enabled agent requests now include execution guidance that instructs the model to:

• invoke the appropriate tool instead of only describing the action;
• continue tool execution while required work remains;
• validate the result before returning the final response.

A shorter reminder is also added on continued tool turns when pending work still requires tool execution.

The guidance is not added when tool use is explicitly disabled through tool_choice.

---

4. Stream and region reliability

Longer streaming timeout

The Kiro streaming HTTP timeout is increased from 5 minutes to 15 minutes to support long-running coding-agent operations.

The separate REST client keeps its existing 30-second timeout.

IDC and external IdP region probing

For idc and external_idp accounts, the authentication region may differ from the Kiro profile region.

Profile discovery can now probe configured fallback regions for these account types when resolving the profile ARN.

Once a profile ARN is resolved, its region continues to determine the data-plane endpoint.

---

5. Payload and stream diagnostics

Payload diagnostics report:

• serialized byte size;
• estimated token usage;
• history size;
• tool and tool-result counts;
• current-message size;
• before/after truncation metrics.

The diagnostics wrap the existing byte-and-token-aware truncation logic without changing its preservation rules.

Stream diagnostics report aggregate information such as:

• assistant events and characters;
• reasoning events and characters;
• tool events;
• token usage;
• credits;
• completion reason and duration.

Raw chunk previews remain separately opt-in.

---

Changes

• Added a diagnostics package with startup-time environment flags.
• Preserved model-specific request schemas from ListAvailableModels.
• Added runtime reasoning-capability parsing and validation.
• Added native request-field generation for Claude and OpenAI-compatible request shapes.
• Added Responses API reasoning compatibility.
• Added outbound reasoning-field forwarding.
• Retained legacy thinking fallback behavior.
• Added agent tool-execution guidance.
• Increased the streaming timeout while preserving the short REST timeout.
• Added IDC and external IdP profile-region fallback probing.
• Documented the new diagnostic environment variables in README.md.

---

Tests

Added and updated tests covering:

• parsing output_config.effort;
• parsing reasoning.effort;
• thinking type discovery;
• thinking display discovery;
• supported effort normalization;
• native Claude request-field generation;
• rejection of unsupported thinking types;
• rejection of unsupported display values;
• rejection of unsupported effort values;
• separation of streaming and REST timeouts.

Verification performed:

• gofmt
• go test ./...
• go vet ./...
• git diff --check zsec/main --

---

Compatibility notes

• Existing requests without native reasoning fields keep their previous behavior.
• Existing -thinking model suffixes remain supported.
• Diagnostics are disabled unless explicitly enabled.
• Raw chunk logging requires a separate opt-in.
• The 30-second REST timeout is unchanged.

[zsecducna]

Merged into main as d635d44 (via a420b21). Thanks for this — the schema-driven approach of validating requested effort against Kiro's own additionalModelRequestFieldsSchema, rather than forwarding blind, is a genuine improvement.

Changes made while merging:

Dropped: the proxy/kiro_api.go hunk extending shouldProbeFallbackRegions() to match "idc". That function no longer exists — upstream 4192f10 replaced it with kiroRegionForProfile/kiroProfileRegionCandidates, which already probes fallback regions for every account type including idc.

Fixed on top:

• Unrecognised thinking.type values were dropped silently and treated as "no reasoning requested"; any explicit non-empty type now reaches selectThinkingType for validation.
• The OpenAI path never called normalizeClaudeThinkingType, so "enabled" wasn't mapped to its "adaptive" alias and got rejected against schemas exposing only adaptive/disabled.
• OpenAI reasoning_effort: "none" set the same Disabled flag as an explicit thinking.type: "disabled", which synthesizes a disable payload. "none" now simply leaves reasoning unrequested.
• reasoningCapabilityForModel is now a cache read only. It runs on every Claude, count_tokens and OpenAI request, and on a cache miss it called refreshModelsCache() inline — a live ListAvailableModels fan-out across every enabled account before the request was even translated. It now returns an empty capability on a cold cache and warms in the background, single-flighted and rate-limited, with panic recovery.
• Capability validation ran above the per-account dispatch loop, so a Kiro-derived schema could hard-400 a request destined for a Bedrock or custom_api account. It's now skipped when no Kiro-capable account is eligible.
• Broadening the thinking bool to include natively-requested reasoning changed client-visible response shape: a request sending only output_config.effort and no thinking object could receive thinking blocks in the operator's default format. Shape gating and format selection now use the same signal.

Gated behind a new default-off config flag (agentToolSteering): agentToolExecutionPrompt and agentToolTurnReminder. These appended agent-steering text to the system prompt and to the customer's own user message on every tool-bearing request. This proxy is used as a transparent Claude passthrough, so that can't be unconditional — but the behaviour is preserved and available for operators who want it. It now has test coverage in both states.

629 tests pass; go build, go vet and gofmt clean.