code clean up / refactor

[franciscojavierarceo]

Summary

Several Rust files are large enough to make review and maintenance difficult, consistent with feedback from maintainers. Track a focused cleanup into cohesive modules with clear ownership and smaller review surfaces.

Proposed file-size policy

• Prefer production files under 300 lines; 300–500 lines is reasonable for one clear responsibility.
• Require a split or a documented exception above 500 production lines.
• Prevent new oversized production files and growth of existing oversized files through a fast pre-commit/CI check with an explicit baseline.
• Count inline tests separately and exclude generated code. Choose a separate test-file threshold before enforcing one.
• Split by responsibility, not arbitrary line counts. Preserve public APIs, wire formats, behavior, and test coverage.

Production refactor checklist

Inventory at commit 6d038d12da167255861a71d22681930dd46f904b. Counts include comments and blank lines. Approximate production counts use the lines before the first top-level #[cfg(test)] mod tests section; these are triage measurements, not an AST-based enforcement implementation. Recount and confirm module boundaries when taking each item.

Entries are ordered by approximate production size; each should be refactored or given a documented cohesion-based exception.

• crates/agentic-server-core/src/tool/tool_search.rs — ~1,764 production lines; 2,253 total.
• crates/agentic-server-core/src/types/io/output.rs — ~1,110 production lines; 1,695 total.
• crates/agentic-server/src/handler/websocket/responses.rs — ~864 production lines; 1,072 total.
• crates/agentic-server-core/src/executor/engine.rs — ~825 production lines; 1,464 total.
• crates/agentic-server/src/auth.rs — ~807 production lines; 1,141 total.
• crates/agentic-server-core/src/executor/accumulator/mod.rs — ~716 production lines; 716 total.
• crates/agentic-server-core/src/executor/accumulator/slot.rs — ~679 production lines; 679 total.
• crates/agentic-server-core/src/executor/gateway.rs — ~678 production lines; 1,853 total.
• crates/agentic-server-core/src/types/io/input.rs — ~678 production lines; 1,169 total.
• crates/agentic-server-core/src/types/request_response.rs — ~582 production lines; 1,301 total.
• crates/agentic-server/src/agentic_process.rs — ~558 production lines; 1,297 total.
• crates/agentic-server-core/src/storage/schema.rs — ~552 production lines; 1,063 total.
• crates/agentic-server-core/src/tool/web_search/mod.rs — ~539 production lines; 869 total.
• crates/agentic-server-core/src/types/tools/params.rs — ~537 production lines; 959 total.
• crates/agentic-server-core/src/executor/compaction.rs — ~534 production lines; 1,308 total.
• crates/agentic-server-core/src/tool/registry.rs — ~533 production lines; 1,154 total.
• crates/agentic-server-core/src/executor/messages_stream.rs — ~520 production lines; 893 total.
• crates/agentic-server-core/src/executor/upstream.rs — ~513 production lines; 513 total.
• crates/agentic-server-core/src/executor/session.rs — ~511 production lines; 873 total.

Start with these boundaries

• tool/tool_search.rs: separate request validation, loaded-definition/catalog state, and tool-call translation.
• types/io/output.rs: group output-item types and conversions by item family, preserving the public exports and wire schema.
• handler/websocket/responses.rs: review request parsing, admission/multiplexing, connection lifecycle, and outbound delivery boundaries.
• executor/engine.rs: keep round orchestration, tool-loop control, and persistence ownership explicit; move responsibilities only to their architecture-defined owners.
• auth.rs: review configuration, provider/JWKS fetching and caching, token verification, and HTTP middleware boundaries.

These are starting points for review, not predetermined module designs.

Large test and benchmark files — separate cleanup

The following files exceed 800 total lines and deserve a scenario-based organization review. This is a triage cutoff, not a proposed test-file enforcement limit. Preserve coverage and share fixtures only where that improves readability.

• crates/agentic-server/tests/responses_websocket_test.rs — 3,476 lines.
• crates/agentic-server-core/tests/web_search_tool_test.rs — 2,542 lines.
• crates/agentic-server-core/src/executor/accumulator/tests.rs — 1,785 lines.
• crates/agentic-server-core/tests/tool_search_test.rs — 1,650 lines.
• crates/agentic-server/tests/responses_test.rs — 1,636 lines.
• crates/agentic-server-core/tests/accumulator_cassette_test.rs — 1,556 lines.
• crates/agentic-server-core/src/executor/translate/tests.rs — 1,547 lines.
• crates/agentic-server-core/tests/session_continuation_test.rs — 1,516 lines.
• crates/agentic-server-core/tests/tool_search_characterization_test.rs — 1,505 lines.
• crates/agentic-server-core/tests/stateful_responses_integration.rs — 1,493 lines.
• crates/agentic-server-core/tests/tool_search_state_test.rs — 1,338 lines.
• crates/agentic-server/tests/oidc_auth_test.rs — 1,238 lines.
• crates/agentic-server-core/tests/storage_integration.rs — 1,218 lines.
• crates/agentic-server/benches/client/response_provider/runner.rs — 1,134 lines.
• crates/agentic-llm-d/tests/split_execution_integration.rs — 908 lines.
• crates/agentic-server-core/tests/messages_loop_test.rs — 861 lines.
• crates/agentic-server-core/tests/event_normalizer_test.rs — 822 lines.

Completion criteria

• Work through the production checklist in small, behavior-preserving PRs; document justified exceptions.
• Organize oversized test suites by behavior/scenario without reducing coverage.
• Implement and document the agreed file-size check, counting test-only code separately and baselining existing oversized files.
• Preserve the ownership boundaries in ARCHITECTURE.md and vocabulary in TERMINOLOGY.md.
• Coordinate executor/streaming changes with [RFC]: one linear ingest pipeline for the executor streaming path #241, refactor: unify synchronous streaming ingestion and item lifecycle #243, refactor: extract StreamRelay and bound client delivery #244, and perf: benchmark inline versus worker placement for streaming ingestion #245. Keep a single ingestion path and typed extension points; do not introduce duplicate lifecycle validators, assemblers, or client-emission paths just to shorten files.
• For each refactor, run relevant tests, cargo fmt -- --check, cargo clippy --all-targets -- -D warnings, and required pre-commit hooks. Preserve lifecycle, cancellation, backpressure, and disconnect coverage for affected streaming boundaries.

[amarjaleelbanbhan]

Hi, I’d like to work on a small, behavior-preserving refactor of crates/agentic-server-core/src/executor/upstream.rs.

My plan is to first study its responsibilities and existing tests, then propose clear module boundaries before changing code. I’ll preserve public APIs and run the relevant tests, cargo fmt, and cargo clippy.

Would this be an appropriate first contribution, or would you recommend another smaller file from the checklist?

[maralbahari]

@franciscojavierarceo let's separate the unit tests from source files as well. in #274 the test file is separated for newly added files and modules.

[franciscojavierarceo]

@amarjaleelbanbhan sounds good