Deepen coverage 4x: 15 new demos, 741 new tests, error-path hardening#6
Open
cognis-digital wants to merge 1 commit into
Open
Deepen coverage 4x: 15 new demos, 741 new tests, error-path hardening#6cognis-digital wants to merge 1 commit into
cognis-digital wants to merge 1 commit into
Conversation
Tests: 232 -> 973 (all green). New suites cover malformed/hostile observation records, unknown/absent C2 families, empty feeds and batches, JSON/JSONL parser robustness, rule-gen determinism (stable Sigma UUIDs + Suricata SIDs), the CLI exit-code contract and error paths, the stdlib MCP server (scan/correlate/list + JSON-RPC dispatch), the datafeeds cache and air-gap snapshot round-trip, the authorized active probe (injected connector: scope/rate-limit/error handling), the opt-in AI backend (disabled-by-default + JSON extraction + never-raises), per-family DB detection, a CLI integration matrix over every bundled fixture, and the public API surface. Demos: 5 -> 20 audience scenarios (SARIF/code-scanning, CI gate, HTML report, status badge, JSONL ingest, free-text triage, correlation graph, threshold tuning, signature inventory, self-check, layered feeds+signatures, beacon cadence, per-family Sigma, campaign gate, air-gap workflow). Each runs fully offline and exits 0; run_all.py, tests/test_demos.py and docs/DEMOS.md updated. Fixes (public API unchanged): - cli rules -o <bad-path>: raised an uncaught FileNotFoundError; now prints a clear error and returns exit 2, matching the scan/correlate error paths. - mcp_server scan/correlate: threshold=0 was silently coerced to 35 by an `x or 35` fallback (0 is falsy); added _coerce_threshold so an explicit 0 (report everything) is honored, matching the core/CLI contract. - .gitignore: the `!demos/**` negation was un-ignoring demos/__pycache__/*.pyc; re-excluded *.pyc and **/__pycache__/ so compiled artifacts stay out.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Roughly 4x the depth of the repo across tests, demos, and error handling — all real work, public API unchanged.
Tests: 232 → 973 (all green)
New edge-case / error-path suites:
test_core_edgecases— malformed/hostile observation records, unknown/absent families, empty feeds & batches, JSON/JSONL parser robustness, threshold behavior, determinism.test_rules_edgecases— Sigma UUID / Suricata SID determinism, families with no strong indicator, custom DBs, SID-band invariants.test_cli_errors+test_cli_integration— the exit-code contract (0/1/2), missing files, bad output paths, and a scan/correlate matrix over every bundled fixture in all output formats.test_mcp_server— the stdlib MCP server:scan/correlate/list_signatures+ JSON-RPCinitialize/tools/list/tools/calldispatch and error paths.test_datafeeds— cache freshness, offline-miss error path, format parsing, air-gap snapshot export/import round-trip.test_probe_integration— authorized active probe via an injected connector: authorization/scope refusal, CIDR & pinned-port scope, connection-error handling, rate limiting, sweep skipping, CS attribution.test_ai_backend— opt-in backend disabled-by-default, JSON-array extraction (fenced/prose/<think>), finding normalization, never-raises contract.test_db_coverage— parametrized per-family self-detection + rule validity across the whole DB.test_matching_engine,test_reporting_edgecases,test_correlate_edgecases,test_selfcheck_edgecases,test_connect_edgecases,test_public_api.Demos: 5 → 20 audience scenarios
SARIF/code-scanning, CI gate, HTML report, status badge, JSONL ingest, free-text triage, correlation graph, threshold tuning, signature inventory, self-check, layered feeds+signatures, beacon cadence, per-family Sigma, campaign gate, and a full air-gap workflow. Each runs fully offline and exits 0.
run_all.py,tests/test_demos.py, anddocs/DEMOS.mdupdated.Fixes (public API unchanged)
cli rules -o <bad-path>raised an uncaughtFileNotFoundError(ugly traceback); now prints a clear error and returns exit 2, matching thescan/correlateerror paths.mcp_serverscan/correlatesilently coercedthreshold=0to35via anx or 35fallback (0 is falsy); added_coerce_thresholdso an explicit0(report everything) is honored, matching the core/CLI contract..gitignore— the!demos/**negation was un-ignoringdemos/__pycache__/*.pyc; re-excluded compiled artifacts.Verification
python -m pytest -q→ 973 passedpython -m c2detect self-check→ HEALTHY (rc 0)PYTHONUTF8=1 python demos/run_all.py→ rc 0; all 20 demos also pass standalone