Skip to content

Restructure into reusable libraries + per-harness install extras #12

Description

@krmayankb

Motivation

SOCBench is organized around per-capability harnesses, but the repo isn't: the detection harness lives flat under src/socbench/, mixed with shared infrastructure. Users should pick their harness at install time:

uv pip install -e ".[dev,detection]"
uv pip install -e ".[dev,patchloop]"

Future harnesses (triage, hunting) land as sibling packages reusing the shared libraries.

Layout

Rule: top level of src/socbench/ = shared, harness-agnostic; subpackages = one harness each. Harnesses import from top level, never from each other.

src/socbench/
├── cli.py            # entry point, mounts harness commands (existing commands unchanged)
├── providers/        # shared: LLM adapters
├── pricing.py        # shared: extracted from config.py
├── hashing.py, logging_config.py
├── detection/        # agent, tools/, index, sampling, scoring, schema, prompts,
│                     #   aggregate, models, config, cli commands
└── patchloop/        # TS harness, untouched; Python side is a separate issue

agent.py and tools/ stay in detection/; extracting a harness-neutral loop waits for a second consumer.

Extras

  • Base deps shrink to shared needs (pydantic, pyyaml, click, rich, jsonschema, python-json-logger).
  • detection = polars, duckdb, pyarrow + provider SDKs (one-command runnable).
  • patchloop = [] placeholder (harness is Node 20+, which pip can't express).
  • Bare pip install socbench and socbench --help must still work.

Tasks

  • Move detection modules to src/socbench/detection/; extract pricing.py
  • Rewrite imports (all absolute, mechanical) in src + 16 test files
  • pyproject: extras, slimmed base deps, ruff path fixes, exclude TS artifacts from wheel
  • Update README / REPRODUCE.md; bump version (import paths change)

Acceptance

Both install commands succeed; bare install works without detection deps; tests/ruff/mypy green; wheel contains no TS/npm files; existing CLI commands unchanged.

Out of scope: PatchLoop Python wrapper/port (separate issue).

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions