diff --git a/.env.example b/.env.example index 3d32bf9..6e5165c 100644 --- a/.env.example +++ b/.env.example @@ -39,6 +39,10 @@ LOCAL_MODEL=local-model OLLAMA_BASE_URL=http://localhost:11434 OLLAMA_MODEL=llama2 LLM_CONFIG_FILE=./llm_config.json +# Inference-time threat evidence verification (Design phase) +EVIDENCE_CRITIC_ENABLED=true +EVIDENCE_CRITIC_MAX_CANDIDATES=5 +EVIDENCE_CRITIC_MAX_THREATS=20 # Governance policy packs and optional infrastructure POLICY_PACK_ID=generic-ssdlc diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4118f80..2935764 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -41,6 +41,9 @@ jobs: - name: Run tests run: pytest --cov=app --cov-report=xml --cov-report=term + - name: Run evaluation harness tests + run: pytest evals/tests + - name: Verify generated contracts run: python scripts/export_contracts.py --check diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 375e9cd..b613caf 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -201,10 +201,15 @@ stateDiagram-v2 ### 2. SSDLC Orchestrator (LangGraph) | SSDLC 编排器 - Built on **LangChain + LangGraph**: stateful, graph-based agent workflow with conditional edges. -- **Graph Definition**: `StateGraph` with nodes for Router, 6 phase agents, and Reviewer. Graph nodes: Parser → SSDLC Router → Policy+History Agent ∥ Evidence Agent → Drafter Agent → Reviewer Agent. +- **Graph Definition**: `StateGraph` with nodes for Router, 6 phase agents, and Reviewer. Assessment graph nodes: Skill Loader → Document Context → Policy+History ∥ Evidence → Drafter → Reviewer → Report Parser → Threat Evidence Critic → Governance Persistence. - **State Schema**: `SSDLCState` TypedDict containing parsed documents, phase findings, threat models, cross-phase references, and metadata. - **Conditional Edges**: Route based on requested phase, project risk level, or full SSDLC mode. SSDLC Router node determines the lifecycle stage and injects stage-specific skill + checklist. - **Parallel Execution**: Policy and Evidence nodes run **in parallel** (LangGraph fan-out/fan-in). Within phases, sub-tasks (e.g. KB lookup + document parsing) run concurrently via `asyncio.gather`. +- **Threat Evidence Critic**: For Design reports, an independent inference-time + verifier checks every normalized STRIDE threat against stable, line-addressed + passages from the current uploaded documents. Unknown citations and verifier + failures become `insufficient_evidence`; policy and history chunks cannot prove + current-design facts. - **Checkpointing**: Persistent state via LangGraph `MemorySaver` or database-backed checkpointer. - Assessment submission is **non-blocking** — returns task_id immediately, processes in background. - Singleton `KnowledgeBaseService` and cached LLM client shared across requests. diff --git a/CHANGELOG.md b/CHANGELOG.md index 38880dd..babedb1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] — PallasGuard merge ### Added +- Inference-time Threat Evidence Critic for Design assessments, with + supported/contradicted/insufficient-evidence verdicts, stable current-document + line citations, safe abstention, grounding metrics, and a reviewer-facing + evidence panel. +- Public threat-model demo architecture document under `examples/`. - Governance domain model and Alembic migrations for projects, submissions, control instances, questionnaires, audit trails, prompt audit records, and sub-agent runs. @@ -36,6 +41,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). - v5 product trust RFC and architecture, evidence-model, and frontend-design ADRs. ### Changed +- Design Drafter and Reviewer prompts now preserve a normalized STRIDE/DREAD + threat model before the independent evidence-verification graph node runs. - Converged orchestration on LangGraph while keeping DocSentinel's assessment task lifecycle, report contracts, and existing API surface intact. - Converged LLM access on the DocSentinel provider abstraction with diff --git a/README.md b/README.md index 22294b1..e278165 100644 --- a/README.md +++ b/README.md @@ -249,6 +249,26 @@ project workflows: - **Human-in-the-loop**: Interrupt points for human review at phase boundaries - **Checkpointing**: Long-running assessments persist state and resume +### Threat Evidence Critic +Design-phase STRIDE threats are independently checked against the currently +uploaded architecture documents in one inference-time pass: + +- Every threat receives a `supported`, `contradicted`, or + `insufficient_evidence` verdict with a support score and reviewer-facing + rationale. +- Evidence references use content hashes plus stable line locators and include + the exact source excerpt shown in the Assessment Workbench. +- Only current-project document passages are accepted. Invented, missing, policy, + or historical references cannot make a threat appear supported. +- Verification failures safely abstain as `insufficient_evidence`; all verdicts + still require human review. +- No training or fine-tuning pipeline is required. The critic uses the configured + OpenAI, Anthropic, Qwen, DeepSeek, compatible, or local runtime model. + +For a public-demo input and walkthrough, see +[`examples/evidence-critic-architecture.md`](./examples/evidence-critic-architecture.md) +and [`examples/README.md`](./examples/README.md). + ### RAG-Powered Knowledge Base Upload your organization's security policies, standards, and past audits. Phase-specific collections ensure each agent retrieves the most relevant context: - Requirements: compliance frameworks, security policies @@ -279,6 +299,9 @@ implemented path covers OWASP Benchmark v1.2 SAST triage: semantics while staying self-contained. - **Hard-key scoring**: M1 scores CWE-based binary triage with accuracy, precision, recall, F1, and false-positive rate, with no LLM judge. +- **Threat grounding scoring**: the Evidence Critic scorer measures verdict + accuracy, supported precision/recall/F1, contradiction recall, abstention rate, + citation validity, and the full verdict confusion matrix. - **Scorecards**: every run writes machine-readable `scorecard.json` and a human-readable `scorecard.md` under `evals/reports//`. @@ -563,6 +586,9 @@ DocSentinel/ | `DEEPSEEK_API_KEY` / `DEEPSEEK_MODEL` | DeepSeek OpenAI-compatible API | -- / `deepseek-chat` | | `COMPAT_API_KEY` / `COMPAT_BASE_URL` / `COMPAT_MODEL` | Any OpenAI-compatible hosted API | -- | | `LOCAL_API_KEY` / `LOCAL_BASE_URL` / `LOCAL_MODEL` | Local OpenAI-compatible API | -- / `http://localhost:1234/v1` / `local-model` | +| `EVIDENCE_CRITIC_ENABLED` | Verify Design-phase threats against current document evidence | `true` | +| `EVIDENCE_CRITIC_MAX_CANDIDATES` | Candidate passages supplied per threat | `5` | +| `EVIDENCE_CRITIC_MAX_THREATS` | Maximum threats verified in one inference pass | `20` | | `CHROMA_PERSIST_DIR` | Vector DB path | `./data/chroma` | | `PARSER_ENGINE` | Parser: `auto`, `docling`, or `legacy` | `auto` | | `ENABLE_GRAPH_RAG` | Enable LightRAG graph retrieval | `true` | diff --git a/app/agent/graph/assessment_graph.py b/app/agent/graph/assessment_graph.py index 98345d9..9d834e8 100644 --- a/app/agent/graph/assessment_graph.py +++ b/app/agent/graph/assessment_graph.py @@ -221,6 +221,18 @@ async def _parse_report(state: AssessmentGraphState) -> AssessmentGraphState: return {"report": report} +async def _verify_threat_evidence( + state: AssessmentGraphState, +) -> AssessmentGraphState: + from app.services.evidence_critic import verify_threat_model_evidence + + report = await verify_threat_model_evidence( + state["report"], + state["parsed_documents"], + ) + return {"report": report} + + async def _persist_governance(state: AssessmentGraphState) -> AssessmentGraphState: try: count = persist_assessment_control_evidence( @@ -241,6 +253,7 @@ def compile_assessment_graph(): graph.add_node("draft_assessment", _draft) graph.add_node("review_assessment", _review) graph.add_node("parse_report", _parse_report) + graph.add_node("verify_threat_evidence", _verify_threat_evidence) graph.add_node("persist_gate3_control_evidence", _persist_governance) graph.add_edge(START, "load_skill") graph.add_edge("load_skill", "build_document_context") @@ -248,7 +261,8 @@ def compile_assessment_graph(): graph.add_edge("gather_policy_history_and_evidence", "draft_assessment") graph.add_edge("draft_assessment", "review_assessment") graph.add_edge("review_assessment", "parse_report") - graph.add_edge("parse_report", "persist_gate3_control_evidence") + graph.add_edge("parse_report", "verify_threat_evidence") + graph.add_edge("verify_threat_evidence", "persist_gate3_control_evidence") graph.add_edge("persist_gate3_control_evidence", END) return graph.compile() diff --git a/app/agent/orchestrator.py b/app/agent/orchestrator.py index 6ac0614..dec8b85 100644 --- a/app/agent/orchestrator.py +++ b/app/agent/orchestrator.py @@ -257,7 +257,8 @@ async def _drafter_agent( "You are DrafterAgent in a multi-agent security workflow. " f"{UNTRUSTED_CONTENT_INSTRUCTION} " "Create an assessment draft in JSON only with keys: summary, risk_items, " - "compliance_gaps, remediations." + "compliance_gaps, threat_model, remediations. For design reviews, " + "threat_model must use STRIDE categories and identify affected components." ) if skill: @@ -267,7 +268,12 @@ async def _drafter_agent( f"Focus areas: {', '.join(skill.risk_focus)}.\n" f"{UNTRUSTED_CONTENT_INSTRUCTION}\n" "Output strictly JSON with keys: summary, risk_items, " - "compliance_gaps, remediations." + "compliance_gaps, threat_model, remediations. For design reviews, " + "threat_model must contain methodology and a threats array. Every threat " + "must contain id, category, description, affected_component, and " + "mitigations. Category must be exactly one of: Spoofing, Tampering, " + "Repudiation, InformationDisclosure, DenialOfService, " + "ElevationOfPrivilege." ) user_prompt = ( @@ -318,8 +324,9 @@ async def _merge_drafts(drafts: list[str], skill: object | None = None) -> str: f"{UNTRUSTED_CONTENT_INSTRUCTION} " f"{skill_info}" "Deduplicate findings, keep the highest-severity version of duplicates, " - "and merge remediations. " - "Output JSON with keys: summary, risk_items, compliance_gaps, remediations." + "merge threat-model entries, and merge remediations. " + "Output JSON with keys: summary, risk_items, compliance_gaps, threat_model, " + "remediations." ) user_prompt = ( f"{_truncate_at_boundary(drafts_text, 14000)}\n\n" @@ -352,7 +359,8 @@ async def _reviewer_agent( "hallucination resistance. " f"{UNTRUSTED_CONTENT_INSTRUCTION} " "Output JSON only with keys: summary, confidence, risk_items, " - "compliance_gaps, remediations, sources. " + "compliance_gaps, threat_model, remediations, sources. Preserve and improve " + "the design threat model when one is present. " f"Available chunk IDs: {chunk_ids}. " "In the `sources` array each entry MUST have: " '`"chunk_id"` (one of the available IDs above) and ' @@ -367,7 +375,8 @@ async def _reviewer_agent( "Ensure findings match the persona's focus. " f"{UNTRUSTED_CONTENT_INSTRUCTION} " "Output JSON only with keys: summary, confidence, risk_items, " - "compliance_gaps, remediations, sources. " + "compliance_gaps, threat_model, remediations, sources. Preserve and " + "improve the design threat model when one is present. " f"Available chunk IDs: {chunk_ids}. " "In the `sources` array each entry MUST have: " '`"chunk_id"` (one of the available IDs above) and ' @@ -467,6 +476,7 @@ def _resolve_citations_from_llm( excerpt=src.get("quote", doc.page_content[:240]), evidence_link=evidence_link, score=float(metadata["score"]) if metadata.get("score") else None, + source_kind="history" if is_history else "policy", ) ) return citations @@ -494,6 +504,7 @@ def _derive_sources_from_chunks( excerpt=doc.page_content[:240], evidence_link=f"{file}#chunk={paragraph_id}" if paragraph_id else None, score=float(metadata.get("score")) if metadata.get("score") else None, + source_kind=origin, ) ) if origin == "history" and citations[-1].evidence_link: @@ -501,6 +512,93 @@ def _derive_sources_from_chunks( return citations +_THREAT_CATEGORY_ALIASES = { + "spoofing": "Spoofing", + "tampering": "Tampering", + "repudiation": "Repudiation", + "informationdisclosure": "InformationDisclosure", + "informationleakage": "InformationDisclosure", + "denialofservice": "DenialOfService", + "dos": "DenialOfService", + "elevationofprivilege": "ElevationOfPrivilege", + "privilegeescalation": "ElevationOfPrivilege", +} + + +def _canonical_threat_category(value: object) -> str | None: + normalized = "".join(character for character in str(value) if character.isalnum()) + return _THREAT_CATEGORY_ALIASES.get(normalized.casefold()) + + +def _normalized_dread_score(value: object) -> dict | None: + if not isinstance(value, dict): + return None + normalized: dict[str, int | float] = {} + for key in [ + "damage", + "reproducibility", + "exploitability", + "affected_users", + "discoverability", + ]: + try: + normalized[key] = min(10, max(1, int(value[key]))) + except (KeyError, TypeError, ValueError): + continue + try: + normalized["total"] = float(value["total"]) + except (KeyError, TypeError, ValueError): + pass + return normalized or None + + +def _normalize_threat_model(value: object) -> dict | None: + """Normalize common model variations into the strict report contract.""" + if not isinstance(value, dict): + return None + raw_threats = value.get("threats", []) + if not isinstance(raw_threats, list): + return None + + threats: list[dict] = [] + seen_ids: set[str] = set() + for index, raw in enumerate(raw_threats, start=1): + if not isinstance(raw, dict): + continue + category = _canonical_threat_category(raw.get("category")) + description = str(raw.get("description") or "").strip() + if not category or not description: + continue + threat_id = str(raw.get("id") or f"T{index}").strip() or f"T{index}" + if threat_id in seen_ids: + threat_id = f"{threat_id}-{index}" + seen_ids.add(threat_id) + mitigations = raw.get("mitigations", []) + if not isinstance(mitigations, list): + mitigations = [mitigations] if mitigations else [] + threats.append( + { + "id": threat_id, + "category": category, + "description": description, + "affected_component": raw.get("affected_component"), + "dread_score": _normalized_dread_score(raw.get("dread_score")), + "mitigations": [ + str(mitigation).strip() + for mitigation in mitigations + if str(mitigation).strip() + ], + } + ) + + if not threats: + return None + methodology = str(value.get("methodology") or "STRIDE_DREAD").upper() + if methodology not in {"STRIDE", "DREAD", "STRIDE_DREAD"}: + methodology = "STRIDE_DREAD" + return {"methodology": methodology, "threats": threats} + + def _parse_llm_output_to_report( raw: str, task_id: UUID, @@ -565,7 +663,7 @@ def _parse_llm_output_to_report( ) for gap in parsed.get("compliance_gaps", []) ], - threat_model=parsed.get("threat_model"), + threat_model=_normalize_threat_model(parsed.get("threat_model")), vulnerabilities=parsed.get("vulnerabilities", []), remediations=[ Remediation( diff --git a/app/core/config.py b/app/core/config.py index 99d0974..b5cbd23 100644 --- a/app/core/config.py +++ b/app/core/config.py @@ -73,6 +73,9 @@ class Settings(BaseSettings): "http://localhost:1234/v1,http://127.0.0.1:1234/v1," "http://[::1]:1234/v1" ) + EVIDENCE_CRITIC_ENABLED: bool = True + EVIDENCE_CRITIC_MAX_CANDIDATES: int = 5 + EVIDENCE_CRITIC_MAX_THREATS: int = 20 # Governance and optional infrastructure POLICY_PACK_ID: str = "generic-ssdlc" diff --git a/app/models/assessment.py b/app/models/assessment.py index 2a4416b..73ec42d 100644 --- a/app/models/assessment.py +++ b/app/models/assessment.py @@ -55,6 +55,15 @@ class DreadScore(BaseModel): total: float | None = None +class EvidenceVerification(BaseModel): + status: Literal["supported", "contradicted", "insufficient_evidence"] + support_score: float = Field(default=0.0, ge=0.0, le=1.0) + rationale: str + evidence_ids: list[str] = Field(default_factory=list) + counterevidence_ids: list[str] = Field(default_factory=list) + requires_human_review: Literal[True] = True + + class Threat(BaseModel): id: str category: Literal[ @@ -69,11 +78,24 @@ class Threat(BaseModel): affected_component: str | None = None dread_score: DreadScore | None = None mitigations: list[str] = Field(default_factory=list) + confidence: float | None = Field(default=None, ge=0.0, le=1.0) + citation_ids: list[str] = Field(default_factory=list) + verification: EvidenceVerification | None = None + + +class EvidenceCriticSummary(BaseModel): + status: Literal["completed", "fallback"] + verifier: str + supported: int = 0 + contradicted: int = 0 + insufficient_evidence: int = 0 + total: int = 0 class ThreatModel(BaseModel): methodology: Literal["STRIDE", "DREAD", "STRIDE_DREAD"] | None = None threats: list[Threat] = Field(default_factory=list) + verification_summary: EvidenceCriticSummary | None = None class Vulnerability(BaseModel): @@ -112,6 +134,13 @@ class SourceCitation(BaseModel): excerpt: str evidence_link: str | None = None score: float | None = None + document_hash: str | None = None + locator: str | None = None + source_kind: Literal[ + "current_document", + "policy", + "history", + ] = "policy" class ReportMetadata(BaseModel): diff --git a/app/services/evidence_critic.py b/app/services/evidence_critic.py new file mode 100644 index 0000000..11aa7b6 --- /dev/null +++ b/app/services/evidence_critic.py @@ -0,0 +1,532 @@ +"""Inference-time evidence verification for design-phase threat models.""" + +from __future__ import annotations + +import hashlib +import json +import logging +import math +import re +from dataclasses import dataclass, replace +from typing import Any + +from app.core.config import settings +from app.core.guardrails import UNTRUSTED_CONTENT_INSTRUCTION, wrap_untrusted_content +from app.llm.base import invoke_llm +from app.models.assessment import ( + AssessmentReport, + EvidenceCriticSummary, + EvidenceVerification, + SourceCitation, + Threat, +) +from app.models.parser import ParsedDocument + +logger = logging.getLogger(__name__) + +_TOKEN_RE = re.compile(r"[a-zA-Z0-9][a-zA-Z0-9_-]{1,}") +_STOPWORDS = { + "about", + "after", + "against", + "also", + "because", + "before", + "being", + "could", + "from", + "into", + "might", + "must", + "that", + "their", + "there", + "these", + "this", + "through", + "under", + "using", + "when", + "where", + "which", + "with", + "would", +} +_STRIDE_TERMS = { + "Spoofing": { + "authentication", + "credential", + "identity", + "impersonation", + "login", + "oidc", + "spoof", + "token", + }, + "Tampering": { + "change", + "integrity", + "modify", + "signature", + "tamper", + "validation", + "webhook", + }, + "Repudiation": { + "audit", + "evidence", + "log", + "nonrepudiation", + "repudiation", + "trace", + }, + "InformationDisclosure": { + "confidential", + "data", + "disclosure", + "encrypt", + "exposure", + "pii", + "secret", + "tls", + }, + "DenialOfService": { + "availability", + "capacity", + "denial", + "dos", + "limit", + "quota", + "rate", + "resource", + }, + "ElevationOfPrivilege": { + "admin", + "authorization", + "permission", + "privilege", + "role", + "service", + }, +} + + +@dataclass(frozen=True) +class EvidencePassage: + id: str + file: str + document_hash: str + locator: str + excerpt: str + start_line: int + end_line: int + score: float = 0.0 + + +def _document_hash(document: ParsedDocument) -> str: + if document.metadata.file_hash: + return document.metadata.file_hash + content = ( + document.content if isinstance(document.content, str) else str(document.content) + ) + return hashlib.sha256(content.encode("utf-8")).hexdigest() + + +def _flush_passage( + passages: list[EvidencePassage], + *, + document: ParsedDocument, + document_index: int, + document_hash: str, + buffer: list[str], + start_line: int, + end_line: int, +) -> None: + excerpt = "\n".join(buffer).strip() + if not excerpt: + return + short_hash = document_hash[:10] + passages.append( + EvidencePassage( + id=f"DOC-{document_index}-L{start_line}-L{end_line}-{short_hash}", + file=document.metadata.filename, + document_hash=document_hash, + locator=f"L{start_line}-L{end_line}", + excerpt=excerpt[:900], + start_line=start_line, + end_line=end_line, + ) + ) + + +def build_document_passages( + parsed_documents: list[ParsedDocument], +) -> list[EvidencePassage]: + """Create stable, line-addressable passages from current-project documents.""" + passages: list[EvidencePassage] = [] + for document_index, document in enumerate(parsed_documents, start=1): + content = ( + document.content + if isinstance(document.content, str) + else str(document.content) + ) + document_hash = _document_hash(document) + lines = content.splitlines() or [content] + buffer: list[str] = [] + start_line = 1 + + for line_number, line in enumerate(lines, start=1): + if not line.strip(): + if buffer: + _flush_passage( + passages, + document=document, + document_index=document_index, + document_hash=document_hash, + buffer=buffer, + start_line=start_line, + end_line=line_number - 1, + ) + buffer = [] + continue + + if not buffer: + start_line = line_number + buffer.append(line) + if len(buffer) >= 6 or sum(len(part) for part in buffer) >= 850: + _flush_passage( + passages, + document=document, + document_index=document_index, + document_hash=document_hash, + buffer=buffer, + start_line=start_line, + end_line=line_number, + ) + buffer = [] + + if buffer: + _flush_passage( + passages, + document=document, + document_index=document_index, + document_hash=document_hash, + buffer=buffer, + start_line=start_line, + end_line=len(lines), + ) + return passages + + +def _tokens(value: str) -> set[str]: + return { + token + for token in _TOKEN_RE.findall(value.casefold()) + if token not in _STOPWORDS + } + + +def _rank_passages( + threat: Threat, + passages: list[EvidencePassage], + *, + limit: int, +) -> list[EvidencePassage]: + query = " ".join( + part + for part in [ + threat.category, + threat.affected_component or "", + threat.description, + ] + if part + ) + query_tokens = _tokens(query) | _STRIDE_TERMS.get(threat.category, set()) + component = (threat.affected_component or "").casefold().strip() + ranked: list[EvidencePassage] = [] + + for passage in passages: + passage_tokens = _tokens(passage.excerpt) + overlap = len(query_tokens & passage_tokens) + denominator = math.sqrt(max(len(query_tokens), 1) * max(len(passage_tokens), 1)) + score = overlap / denominator + if component and component in passage.excerpt.casefold(): + score += 0.35 + if passage.excerpt.lstrip().startswith("#"): + score += 0.02 + ranked.append(replace(passage, score=round(score, 4))) + + ranked.sort( + key=lambda passage: ( + passage.score, + -passage.start_line, + passage.file, + ), + reverse=True, + ) + positive = [passage for passage in ranked if passage.score > 0] + if positive: + return positive[:limit] + return ranked[: min(2, limit)] + + +def _critic_prompt( + threats: list[Threat], + candidates: dict[str, list[EvidencePassage]], +) -> tuple[str, str]: + system_prompt = ( + "You are EvidenceCriticAgent, an independent threat-model evidence verifier. " + f"{UNTRUSTED_CONTENT_INSTRUCTION} " + "For each STRIDE threat, decide whether the current design documents contain " + "facts that support the architectural preconditions of the threat, explicitly " + "contradict it, or provide insufficient evidence. Do not treat the absence " + "of a mitigation as proof that a threat is supported. Do not use outside " + "knowledge " + "as evidence. Use only the candidate evidence IDs allowed for that threat. " + "Return JSON only with a `verdicts` array. Each verdict must contain: " + "`threat_id`, `status` (`supported`, `contradicted`, or " + "`insufficient_evidence`), `support_score` from 0 to 1, `evidence_ids`, " + "`counterevidence_ids`, and a concise `rationale`. A supported verdict " + "requires " + "at least one evidence ID. A contradicted verdict requires at least one " + "counterevidence ID. Prefer insufficient_evidence over guessing." + ) + + sections: list[str] = [] + for threat in threats: + allowed = candidates.get(threat.id, []) + threat_payload = { + "id": threat.id, + "category": threat.category, + "description": threat.description, + "affected_component": threat.affected_component, + "mitigations": threat.mitigations, + "allowed_evidence_ids": [passage.id for passage in allowed], + } + evidence = "\n".join( + f"[{passage.id}] {passage.file}#{passage.locator} " + f"(retrieval_score={passage.score})\n" + f"{wrap_untrusted_content(passage.id, passage.excerpt)}" + for passage in allowed + ) + sections.append( + f"## Threat\n{json.dumps(threat_payload, ensure_ascii=False)}\n" + f"## Candidate evidence\n{evidence or '(none)'}" + ) + return system_prompt, "\n\n---\n\n".join(sections) + + +def _json_object(raw: str) -> dict[str, Any]: + cleaned = raw.replace("```json", "").replace("```", "").strip() + start = cleaned.find("{") + end = cleaned.rfind("}") + if start == -1 or end < start: + raise ValueError("Evidence critic returned no JSON object") + value = json.loads(cleaned[start : end + 1]) + if not isinstance(value, dict): + raise ValueError("Evidence critic response must be a JSON object") + return value + + +def _score(value: Any) -> float: + try: + parsed = float(value) + if not math.isfinite(parsed): + return 0.0 + return min(1.0, max(0.0, parsed)) + except (TypeError, ValueError): + return 0.0 + + +def _fallback_verification(reason: str) -> EvidenceVerification: + return EvidenceVerification( + status="insufficient_evidence", + support_score=0.0, + rationale=reason, + requires_human_review=True, + ) + + +def _valid_ids(value: Any, allowed_ids: set[str]) -> list[str]: + if not isinstance(value, list): + return [] + valid: list[str] = [] + seen: set[str] = set() + for item in value: + evidence_id = str(item) + if evidence_id in allowed_ids and evidence_id not in seen: + valid.append(evidence_id) + seen.add(evidence_id) + return valid + + +def _apply_verdicts( + report: AssessmentReport, + candidates: dict[str, list[EvidencePassage]], + payload: dict[str, Any] | None, + *, + fallback_reason: str | None = None, +) -> AssessmentReport: + updated = report.model_copy(deep=True) + assert updated.threat_model is not None + + by_threat: dict[str, dict[str, Any]] = {} + if payload is not None: + verdicts = payload.get("verdicts", []) + if isinstance(verdicts, list): + for verdict in verdicts: + if isinstance(verdict, dict) and verdict.get("threat_id"): + by_threat.setdefault(str(verdict["threat_id"]), verdict) + + passages_by_id = { + passage.id: passage for allowed in candidates.values() for passage in allowed + } + referenced: set[str] = set() + counts = { + "supported": 0, + "contradicted": 0, + "insufficient_evidence": 0, + } + + for threat in updated.threat_model.threats: + allowed_ids = {passage.id for passage in candidates.get(threat.id, [])} + verdict = by_threat.get(threat.id) + if verdict is None: + verification = _fallback_verification( + fallback_reason or "The verifier returned no verdict for this threat." + ) + else: + status = str(verdict.get("status", "insufficient_evidence")) + if status not in counts: + status = "insufficient_evidence" + evidence_ids = _valid_ids(verdict.get("evidence_ids"), allowed_ids) + counterevidence_ids = _valid_ids( + verdict.get("counterevidence_ids"), + allowed_ids, + ) + rationale = str(verdict.get("rationale") or "").strip()[:700] + support_score = _score(verdict.get("support_score")) + + if status == "supported" and not evidence_ids: + status = "insufficient_evidence" + rationale = ( + "The verifier did not provide a valid current-document citation. " + + rationale + ).strip() + elif status == "contradicted" and not counterevidence_ids: + if evidence_ids: + counterevidence_ids = evidence_ids + evidence_ids = [] + else: + status = "insufficient_evidence" + rationale = ( + "The verifier did not provide valid counterevidence. " + + rationale + ).strip() + + if status == "insufficient_evidence": + evidence_ids = [] + counterevidence_ids = [] + verification = EvidenceVerification( + status=status, + support_score=support_score, + rationale=rationale or "No verifier rationale was provided.", + evidence_ids=evidence_ids, + counterevidence_ids=counterevidence_ids, + requires_human_review=True, + ) + + threat.verification = verification + threat.confidence = verification.support_score + threat.citation_ids = ( + verification.evidence_ids + verification.counterevidence_ids + ) + referenced.update(threat.citation_ids) + counts[verification.status] += 1 + + existing_ids = {source.id for source in updated.sources} + for evidence_id in sorted(referenced): + if evidence_id in existing_ids: + continue + passage = passages_by_id[evidence_id] + updated.sources.append( + SourceCitation( + id=passage.id, + file=passage.file, + paragraph_id=passage.locator, + excerpt=passage.excerpt, + evidence_link=(f"document://{passage.document_hash}#{passage.locator}"), + score=passage.score, + document_hash=passage.document_hash, + locator=passage.locator, + source_kind="current_document", + ) + ) + + status = "fallback" if fallback_reason else "completed" + updated.threat_model.verification_summary = EvidenceCriticSummary( + status=status, + verifier=( + "safe-fallback" if fallback_reason else f"{settings.LLM_PROVIDER}:inference" + ), + supported=counts["supported"], + contradicted=counts["contradicted"], + insufficient_evidence=counts["insufficient_evidence"], + total=len(updated.threat_model.threats), + ) + return updated + + +async def verify_threat_model_evidence( + report: AssessmentReport, + parsed_documents: list[ParsedDocument], +) -> AssessmentReport: + """Verify all threats in one inference-time pass against current documents.""" + design_selected = report.phase == "design" or ( + report.metadata is not None and report.metadata.skill_id == "ssdlc-design" + ) + if ( + not settings.EVIDENCE_CRITIC_ENABLED + or not design_selected + or report.threat_model is None + or not report.threat_model.threats + ): + return report + + max_threats = max(1, settings.EVIDENCE_CRITIC_MAX_THREATS) + max_candidates = max(1, settings.EVIDENCE_CRITIC_MAX_CANDIDATES) + threats = report.threat_model.threats[:max_threats] + passages = build_document_passages(parsed_documents) + candidates = { + threat.id: _rank_passages( + threat, + passages, + limit=max_candidates, + ) + for threat in threats + } + + if not passages: + return _apply_verdicts( + report, + candidates, + None, + fallback_reason="No current-document evidence passages were available.", + ) + + system_prompt, user_prompt = _critic_prompt(threats, candidates) + try: + raw = await invoke_llm(system_prompt, user_prompt) + payload = _json_object(raw) + except Exception as exc: + logger.warning("Evidence critic failed safely: %s", exc) + return _apply_verdicts( + report, + candidates, + None, + fallback_reason=( + "Evidence verification was unavailable; human review is required." + ), + ) + + return _apply_verdicts(report, candidates, payload) diff --git a/docs/03-assessment-report-and-skill-contract.md b/docs/03-assessment-report-and-skill-contract.md index e1e4136..88b6439 100644 --- a/docs/03-assessment-report-and-skill-contract.md +++ b/docs/03-assessment-report-and-skill-contract.md @@ -12,6 +12,12 @@ Agent outputs a **structured report** conforming to this schema. It is used for API responses, cross-phase traceability, sign-off workflows, and optional ServiceNow write-back. Each report is tagged with the SSDLC phase that generated it. +For Design assessments, every threat may also carry an inference-time evidence +verdict. `supported` means current-project document passages establish the +architectural preconditions of the threat; `contradicted` means current evidence +explicitly conflicts with it; `insufficient_evidence` is the mandatory abstention +state when support cannot be verified. All three remain drafts until human review. + ### 1.1 JSON Schema ```json @@ -125,6 +131,9 @@ Agent outputs a **structured report** conforming to this schema. It is used for "category": { "type": "string", "enum": ["Spoofing", "Tampering", "Repudiation", "InformationDisclosure", "DenialOfService", "ElevationOfPrivilege"] }, "description": { "type": "string" }, "affected_component": { "type": "string" }, + "confidence": { "type": "number", "minimum": 0, "maximum": 1 }, + "citation_ids": { "type": "array", "items": { "type": "string" } }, + "verification": { "$ref": "#/$defs/EvidenceVerification" }, "dread_score": { "type": "object", "properties": { @@ -139,7 +148,34 @@ Agent outputs a **structured report** conforming to this schema. It is used for "mitigations": { "type": "array", "items": { "type": "string" } } } } - } + }, + "verification_summary": { "$ref": "#/$defs/EvidenceCriticSummary" } + } + }, + "EvidenceVerification": { + "type": "object", + "required": ["status", "support_score", "rationale", "requires_human_review"], + "properties": { + "status": { + "type": "string", + "enum": ["supported", "contradicted", "insufficient_evidence"] + }, + "support_score": { "type": "number", "minimum": 0, "maximum": 1 }, + "rationale": { "type": "string" }, + "evidence_ids": { "type": "array", "items": { "type": "string" } }, + "counterevidence_ids": { "type": "array", "items": { "type": "string" } }, + "requires_human_review": { "type": "boolean", "const": true } + } + }, + "EvidenceCriticSummary": { + "type": "object", + "properties": { + "status": { "type": "string", "enum": ["completed", "fallback"] }, + "verifier": { "type": "string" }, + "supported": { "type": "integer" }, + "contradicted": { "type": "integer" }, + "insufficient_evidence": { "type": "integer" }, + "total": { "type": "integer" } } }, "Vulnerability": { @@ -195,7 +231,13 @@ Agent outputs a **structured report** conforming to this schema. It is used for "paragraph_id": { "type": "string" }, "excerpt": { "type": "string", "description": "Relevant excerpt from source" }, "evidence_link": { "type": "string" }, - "score": { "type": "number", "description": "Relevance score" } + "score": { "type": "number", "description": "Relevance score" }, + "document_hash": { "type": "string", "description": "Stable current-document content hash" }, + "locator": { "type": "string", "description": "Stable page, paragraph, table, or line locator" }, + "source_kind": { + "type": "string", + "enum": ["current_document", "policy", "history"] + } } } } diff --git a/docs/05-deployment-runbook.md b/docs/05-deployment-runbook.md index 47d289c..4d3f6a7 100644 --- a/docs/05-deployment-runbook.md +++ b/docs/05-deployment-runbook.md @@ -181,7 +181,20 @@ When running through Docker or a reverse proxy, configure | `LIGHTRAG_WORKING_DIR` | LightRAG data directory | `./data/lightrag` | | `GRAPH_RAG_QUERY_MODE` | Query mode: naive/local/global/hybrid | `hybrid` | -### 4.8 SSDLC Pipeline +### 4.8 Threat Evidence Critic + +| Variable | Description | Default | +| :--- | :--- | :--- | +| `EVIDENCE_CRITIC_ENABLED` | Run inference-time evidence verification for Design threats | `true` | +| `EVIDENCE_CRITIC_MAX_CANDIDATES` | Current-document passages supplied per threat | `5` | +| `EVIDENCE_CRITIC_MAX_THREATS` | Threats included in one verifier request | `20` | + +The critic never uses policy or assessment-history chunks as proof of a current +design fact. If inference is unavailable or a returned evidence ID does not +resolve to an allowed current-document passage, the affected threat is marked +`insufficient_evidence` and remains subject to human review. + +### 4.9 SSDLC Pipeline | Variable | Description | Default | | :-------------------- | :------------------------------------------------------------------- | :------ | diff --git a/docs/07-evaluation-plan.md b/docs/07-evaluation-plan.md index a61d43b..2fe3157 100644 --- a/docs/07-evaluation-plan.md +++ b/docs/07-evaluation-plan.md @@ -15,6 +15,12 @@ > read as a methodology reference for others building evals over > document-review / agentic-security tools. +**Implementation note:** M0 and M1 are implemented. A focused Design-phase +Threat Evidence Critic scorer is also available for verdict accuracy, +supported precision/recall/F1, contradiction recall, abstention, citation +validity, and confusion reporting. This vertical slice supports the public demo +but does not mark the broader M2 or M5 milestones complete. + --- ## 1. Goals and Non-Goals diff --git a/docs/openapi.json b/docs/openapi.json index d8f1214..fb168b6 100644 --- a/docs/openapi.json +++ b/docs/openapi.json @@ -717,6 +717,98 @@ "title": "DreadScore", "type": "object" }, + "EvidenceCriticSummary": { + "properties": { + "contradicted": { + "default": 0, + "title": "Contradicted", + "type": "integer" + }, + "insufficient_evidence": { + "default": 0, + "title": "Insufficient Evidence", + "type": "integer" + }, + "status": { + "enum": [ + "completed", + "fallback" + ], + "title": "Status", + "type": "string" + }, + "supported": { + "default": 0, + "title": "Supported", + "type": "integer" + }, + "total": { + "default": 0, + "title": "Total", + "type": "integer" + }, + "verifier": { + "title": "Verifier", + "type": "string" + } + }, + "required": [ + "status", + "verifier" + ], + "title": "EvidenceCriticSummary", + "type": "object" + }, + "EvidenceVerification": { + "properties": { + "counterevidence_ids": { + "items": { + "type": "string" + }, + "title": "Counterevidence Ids", + "type": "array" + }, + "evidence_ids": { + "items": { + "type": "string" + }, + "title": "Evidence Ids", + "type": "array" + }, + "rationale": { + "title": "Rationale", + "type": "string" + }, + "requires_human_review": { + "const": true, + "default": true, + "title": "Requires Human Review", + "type": "boolean" + }, + "status": { + "enum": [ + "supported", + "contradicted", + "insufficient_evidence" + ], + "title": "Status", + "type": "string" + }, + "support_score": { + "default": 0.0, + "maximum": 1.0, + "minimum": 0.0, + "title": "Support Score", + "type": "number" + } + }, + "required": [ + "status", + "rationale" + ], + "title": "EvidenceVerification", + "type": "object" + }, "GateReviewPayload": { "properties": { "reviewed_by_id": { @@ -2139,6 +2231,17 @@ }, "SourceCitation": { "properties": { + "document_hash": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Document Hash" + }, "evidence_link": { "anyOf": [ { @@ -2162,6 +2265,17 @@ "title": "Id", "type": "string" }, + "locator": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Locator" + }, "page": { "anyOf": [ { @@ -2194,6 +2308,16 @@ } ], "title": "Score" + }, + "source_kind": { + "default": "policy", + "enum": [ + "current_document", + "policy", + "history" + ], + "title": "Source Kind", + "type": "string" } }, "required": [ @@ -2292,6 +2416,26 @@ "title": "Category", "type": "string" }, + "citation_ids": { + "items": { + "type": "string" + }, + "title": "Citation Ids", + "type": "array" + }, + "confidence": { + "anyOf": [ + { + "maximum": 1.0, + "minimum": 0.0, + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Confidence" + }, "description": { "title": "Description", "type": "string" @@ -2316,6 +2460,16 @@ }, "title": "Mitigations", "type": "array" + }, + "verification": { + "anyOf": [ + { + "$ref": "#/components/schemas/EvidenceVerification" + }, + { + "type": "null" + } + ] } }, "required": [ @@ -2350,6 +2504,16 @@ }, "title": "Threats", "type": "array" + }, + "verification_summary": { + "anyOf": [ + { + "$ref": "#/components/schemas/EvidenceCriticSummary" + }, + { + "type": "null" + } + ] } }, "title": "ThreatModel", diff --git a/docs/schemas/assessment-report.json b/docs/schemas/assessment-report.json index 2e21e90..cc9230c 100644 --- a/docs/schemas/assessment-report.json +++ b/docs/schemas/assessment-report.json @@ -201,6 +201,98 @@ "title": "DreadScore", "type": "object" }, + "EvidenceCriticSummary": { + "properties": { + "contradicted": { + "default": 0, + "title": "Contradicted", + "type": "integer" + }, + "insufficient_evidence": { + "default": 0, + "title": "Insufficient Evidence", + "type": "integer" + }, + "status": { + "enum": [ + "completed", + "fallback" + ], + "title": "Status", + "type": "string" + }, + "supported": { + "default": 0, + "title": "Supported", + "type": "integer" + }, + "total": { + "default": 0, + "title": "Total", + "type": "integer" + }, + "verifier": { + "title": "Verifier", + "type": "string" + } + }, + "required": [ + "status", + "verifier" + ], + "title": "EvidenceCriticSummary", + "type": "object" + }, + "EvidenceVerification": { + "properties": { + "counterevidence_ids": { + "items": { + "type": "string" + }, + "title": "Counterevidence Ids", + "type": "array" + }, + "evidence_ids": { + "items": { + "type": "string" + }, + "title": "Evidence Ids", + "type": "array" + }, + "rationale": { + "title": "Rationale", + "type": "string" + }, + "requires_human_review": { + "const": true, + "default": true, + "title": "Requires Human Review", + "type": "boolean" + }, + "status": { + "enum": [ + "supported", + "contradicted", + "insufficient_evidence" + ], + "title": "Status", + "type": "string" + }, + "support_score": { + "default": 0.0, + "maximum": 1.0, + "minimum": 0.0, + "title": "Support Score", + "type": "number" + } + }, + "required": [ + "status", + "rationale" + ], + "title": "EvidenceVerification", + "type": "object" + }, "Remediation": { "properties": { "action": { @@ -480,6 +572,18 @@ }, "SourceCitation": { "properties": { + "document_hash": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Document Hash" + }, "evidence_link": { "anyOf": [ { @@ -504,6 +608,18 @@ "title": "Id", "type": "string" }, + "locator": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Locator" + }, "page": { "anyOf": [ { @@ -539,6 +655,16 @@ ], "default": null, "title": "Score" + }, + "source_kind": { + "default": "policy", + "enum": [ + "current_document", + "policy", + "history" + ], + "title": "Source Kind", + "type": "string" } }, "required": [ @@ -575,6 +701,27 @@ "title": "Category", "type": "string" }, + "citation_ids": { + "items": { + "type": "string" + }, + "title": "Citation Ids", + "type": "array" + }, + "confidence": { + "anyOf": [ + { + "maximum": 1.0, + "minimum": 0.0, + "type": "number" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Confidence" + }, "description": { "title": "Description", "type": "string" @@ -600,6 +747,17 @@ }, "title": "Mitigations", "type": "array" + }, + "verification": { + "anyOf": [ + { + "$ref": "#/$defs/EvidenceVerification" + }, + { + "type": "null" + } + ], + "default": null } }, "required": [ @@ -635,6 +793,17 @@ }, "title": "Threats", "type": "array" + }, + "verification_summary": { + "anyOf": [ + { + "$ref": "#/$defs/EvidenceCriticSummary" + }, + { + "type": "null" + } + ], + "default": null } }, "title": "ThreatModel", diff --git a/evals/README.md b/evals/README.md index ceb75cd..eed0ece 100644 --- a/evals/README.md +++ b/evals/README.md @@ -8,6 +8,11 @@ Current scope: - **M0**: evaluation layout, `EvalCase`, `RunConfig`, and dataset registry. - **M1**: OWASP Benchmark SAST triage adapter, direct `AssessmentService` runner, hard-key CWE triage scoring, and scorecard outputs. +- **Threat Evidence Critic vertical slice**: ID-aligned grounding scoring for + inference-time threat verdicts, including status accuracy, supported + precision/recall/F1, contradiction recall, abstention rate, citation validity, + and a verdict confusion matrix. This is a focused Design-phase addition; it + does not claim completion of the broader M2 or M5 milestones. Raw datasets are not committed. Use the dataset-specific fetch script to download public data into ignored local paths. @@ -27,4 +32,3 @@ The runner writes: - `evals/reports//scorecard.md` M1 uses deterministic hard-key CWE triage scoring and does not use an LLM judge. - diff --git a/evals/scoring/scorers/grounding.py b/evals/scoring/scorers/grounding.py index 30ed3fd..bc0d4da 100644 --- a/evals/scoring/scorers/grounding.py +++ b/evals/scoring/scorers/grounding.py @@ -1,2 +1,137 @@ -"""Future citation grounding and hallucination scorer.""" +"""Claim-level grounding metrics for threat evidence verification.""" +from __future__ import annotations + +from typing import Any, Literal + +from pydantic import BaseModel, Field + +from app.models.assessment import Threat + +VerificationStatus = Literal[ + "supported", + "contradicted", + "insufficient_evidence", +] + + +class GroundingScorecard(BaseModel): + total: int = 0 + correct: int = 0 + status_accuracy: float = 0.0 + supported_precision: float = 0.0 + supported_recall: float = 0.0 + supported_f1: float = 0.0 + contradiction_recall: float = 0.0 + abstention_rate: float = 0.0 + citation_validity: float = 0.0 + confusion: dict[str, dict[str, int]] = Field(default_factory=dict) + + +def _divide(numerator: int, denominator: int) -> float: + return numerator / denominator if denominator else 0.0 + + +def _threat_value(threat: Threat | dict[str, Any], key: str) -> Any: + if isinstance(threat, Threat): + return getattr(threat, key) + return threat.get(key) + + +def _predicted_status(threat: Threat | dict[str, Any]) -> str: + verification = _threat_value(threat, "verification") + if verification is None: + return "insufficient_evidence" + if hasattr(verification, "status"): + return str(verification.status) + if isinstance(verification, dict): + return str(verification.get("status") or "insufficient_evidence") + return "insufficient_evidence" + + +def _citation_ids(threat: Threat | dict[str, Any]) -> list[str]: + values = _threat_value(threat, "citation_ids") or [] + return [str(value) for value in values] + + +def score_threat_grounding( + predicted: list[Threat | dict[str, Any]], + expected: list[dict[str, Any]], + *, + valid_source_ids: set[str] | None = None, +) -> GroundingScorecard: + """Score verifier status and citation validity on ID-aligned threat cases.""" + predicted_by_id = { + str(_threat_value(threat, "id")): threat + for threat in predicted + if _threat_value(threat, "id") + } + confusion: dict[str, dict[str, int]] = {} + correct = 0 + supported_tp = 0 + supported_fp = 0 + supported_fn = 0 + contradicted_total = 0 + contradicted_correct = 0 + abstentions = 0 + citation_total = 0 + citation_valid = 0 + + for item in expected: + threat_id = str(item.get("id") or "") + expected_status = str( + item.get("verification_status") or "insufficient_evidence" + ) + predicted_threat = predicted_by_id.get(threat_id) + predicted_status = ( + _predicted_status(predicted_threat) + if predicted_threat is not None + else "insufficient_evidence" + ) + confusion.setdefault(expected_status, {}) + confusion[expected_status][predicted_status] = ( + confusion[expected_status].get(predicted_status, 0) + 1 + ) + + if predicted_status == expected_status: + correct += 1 + if expected_status == "supported": + if predicted_status == "supported": + supported_tp += 1 + else: + supported_fn += 1 + elif predicted_status == "supported": + supported_fp += 1 + if expected_status == "contradicted": + contradicted_total += 1 + if predicted_status == "contradicted": + contradicted_correct += 1 + if predicted_status == "insufficient_evidence": + abstentions += 1 + + if predicted_threat is not None: + ids = _citation_ids(predicted_threat) + citation_total += len(ids) + if valid_source_ids is None: + citation_valid += len(ids) + else: + citation_valid += sum( + citation_id in valid_source_ids for citation_id in ids + ) + + total = len(expected) + precision = _divide(supported_tp, supported_tp + supported_fp) + recall = _divide(supported_tp, supported_tp + supported_fn) + f1 = _divide(2 * precision * recall, precision + recall) + return GroundingScorecard( + total=total, + correct=correct, + status_accuracy=_divide(correct, total), + supported_precision=precision, + supported_recall=recall, + supported_f1=f1, + contradiction_recall=_divide(contradicted_correct, contradicted_total), + abstention_rate=_divide(abstentions, total), + citation_validity=_divide(citation_valid, citation_total), + confusion=confusion, + ) diff --git a/evals/tests/test_grounding_scorer.py b/evals/tests/test_grounding_scorer.py new file mode 100644 index 0000000..95dd28f --- /dev/null +++ b/evals/tests/test_grounding_scorer.py @@ -0,0 +1,49 @@ +from app.models.assessment import EvidenceVerification, Threat +from evals.scoring.scorers.grounding import score_threat_grounding + + +def _threat( + threat_id: str, + status: str, + citation_ids: list[str] | None = None, +) -> Threat: + return Threat( + id=threat_id, + category="Tampering", + description=f"Threat {threat_id}", + citation_ids=citation_ids or [], + verification=EvidenceVerification( + status=status, + support_score=0.8, + rationale="test verdict", + evidence_ids=citation_ids or [], + ), + ) + + +def test_grounding_scorer_reports_status_and_citation_quality(): + predicted = [ + _threat("T1", "supported", ["E1"]), + _threat("T2", "insufficient_evidence"), + _threat("T3", "supported", ["INVENTED"]), + ] + expected = [ + {"id": "T1", "verification_status": "supported"}, + {"id": "T2", "verification_status": "contradicted"}, + {"id": "T3", "verification_status": "insufficient_evidence"}, + ] + + score = score_threat_grounding( + predicted, + expected, + valid_source_ids={"E1"}, + ) + + assert score.total == 3 + assert score.correct == 1 + assert score.status_accuracy == 1 / 3 + assert score.supported_precision == 0.5 + assert score.supported_recall == 1.0 + assert score.contradiction_recall == 0.0 + assert score.abstention_rate == 1 / 3 + assert score.citation_validity == 0.5 diff --git a/examples/README.md b/examples/README.md index 715a7a2..74bad41 100644 --- a/examples/README.md +++ b/examples/README.md @@ -7,6 +7,20 @@ Sample documents for testing DocSentinel locally. |------|-----| | **sample.txt** | Minimal "security questionnaire" for testing the assessment API. Use with `curl -F "files=@sample.txt"` or via Swagger `/docs`. | | **sample-policy.txt** | Short policy excerpt for testing KB upload and RAG query (`/api/v1/kb/documents`, then `/api/v1/kb/query`). | +| **evidence-critic-architecture.md** | Public-demo design document with supported, contradicted, and insufficient-evidence STRIDE scenarios for the inference-time Evidence Critic. | + +## Threat Evidence Critic demo + +1. Start the API and React console, then open `/console/assessments`. +2. Upload `examples/evidence-critic-architecture.md`. +3. Select phase `design`, skill `SSDLC Design Agent`, and enable human review. +4. Open the completed assessment. The **Threat Evidence Critic** panel shows each + threat's verdict, support score, exact document line locator, source excerpt, + and human-review requirement. + +The critic uses the configured runtime model and the uploaded document only. It +does not require fine-tuning or a training dataset. If verification fails, every +unverified threat safely falls back to `insufficient_evidence`. ## Quick test (from repo root) diff --git a/examples/evidence-critic-architecture.md b/examples/evidence-critic-architecture.md new file mode 100644 index 0000000..768a383 --- /dev/null +++ b/examples/evidence-critic-architecture.md @@ -0,0 +1,44 @@ +# Checkout Service — Design Review Demo + +## Internet edge and identity + +The customer-facing single-page application sends HTTPS requests to a public API +Gateway. The gateway validates the signature, issuer, audience, and expiry of each +OIDC access token before forwarding an authenticated subject identifier to the +Checkout API. + +The Checkout API accepts an `account_id` path parameter. It verifies that the +authenticated subject owns the requested account before reading an order. + +## Payment webhook + +The Payment Webhook is reachable from the public internet and accepts JSON event +payloads from the payment provider. HMAC signature validation is planned for the +next release but is not implemented in the current design. + +Accepted webhook events are written directly to the order state machine. Duplicate +event detection is implemented using the provider event ID, but payload integrity +is not otherwise verified. + +## Administrative access + +The internal Admin Console calls the Checkout API through a private network route. +Administrators authenticate with individual SSO accounts and phishing-resistant +MFA. Every privileged action records the administrator subject, action, target, +timestamp, and result in an append-only audit stream. + +The report-export worker uses one shared service token with the `order-admin` role. +The token is stored in the deployment secret store and currently has no automatic +rotation schedule. + +## Data flows + +All service-to-service connections use TLS 1.3. The order database uses managed +encryption at rest. Payment card numbers are never stored; the system stores only +the payment provider token and the last four digits for display. + +## Availability + +The public API Gateway enforces 100 requests per minute per authenticated subject. +The Payment Webhook has no per-sender rate limit because provider IP ranges have not +yet been integrated into the gateway policy. diff --git a/frontend/package-lock.json b/frontend/package-lock.json index 29a8b9f..df4c4a9 100644 --- a/frontend/package-lock.json +++ b/frontend/package-lock.json @@ -16,7 +16,7 @@ "openapi-fetch": "^0.17.0", "react": "^18.3.1", "react-dom": "^18.3.1", - "react-router-dom": "^6.28.0" + "react-router-dom": "^7.18.1" }, "devDependencies": { "@testing-library/jest-dom": "^6.9.1", @@ -1680,15 +1680,6 @@ "js-yaml": "bin/js-yaml.js" } }, - "node_modules/@remix-run/router": { - "version": "1.23.3", - "resolved": "https://registry.npmjs.org/@remix-run/router/-/router-1.23.3.tgz", - "integrity": "sha512-4An71tdz9X8+3sI4Qqqd2LWd9vS39J7sqd9EU4Scw7TJE/qB10Flv/UuqbPVgfQV9XoK8Np6jNquZitnZq5i+Q==", - "license": "MIT", - "engines": { - "node": ">=14.0.0" - } - }, "node_modules/@rolldown/pluginutils": { "version": "1.0.0-beta.27", "resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-beta.27.tgz", @@ -2816,6 +2807,19 @@ "dev": true, "license": "MIT" }, + "node_modules/cookie": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/cookie/-/cookie-1.1.1.tgz", + "integrity": "sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==", + "license": "MIT", + "engines": { + "node": ">=18" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/express" + } + }, "node_modules/css-tree": { "version": "3.2.1", "resolved": "https://registry.npmjs.org/css-tree/-/css-tree-3.2.1.tgz", @@ -4066,35 +4070,41 @@ } }, "node_modules/react-router": { - "version": "6.30.4", - "resolved": "https://registry.npmjs.org/react-router/-/react-router-6.30.4.tgz", - "integrity": "sha512-SVUsDe+DybHM/WmYKIVYhZh1o5Dcuf16yM6WjG02Q9XVFMZIJyHYhwrr6bFBXZkVP6z69kNkMyBCujt8FaFLJA==", + "version": "7.18.1", + "resolved": "https://registry.npmjs.org/react-router/-/react-router-7.18.1.tgz", + "integrity": "sha512-GDLgg3i3uM0aeJO3Fm+TCS+sDQ7gu12T6x0qdTEzcwqEfleci7JwugVNIF3U//0FWKnJT7ptG+20B2jfDqnZAg==", "license": "MIT", "dependencies": { - "@remix-run/router": "1.23.3" + "cookie": "^1.0.1", + "set-cookie-parser": "^2.6.0" }, "engines": { - "node": ">=14.0.0" + "node": ">=20.0.0" }, "peerDependencies": { - "react": ">=16.8" + "react": ">=18", + "react-dom": ">=18" + }, + "peerDependenciesMeta": { + "react-dom": { + "optional": true + } } }, "node_modules/react-router-dom": { - "version": "6.30.4", - "resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-6.30.4.tgz", - "integrity": "sha512-q4HvNl+mmDdkS0g+MqiBZNteQJCuimWoOyHMy4T/RQLAn9Z29+E91QXRaxOujeMl2HTzRSS0KFPd7lxX3PjV0Q==", + "version": "7.18.1", + "resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-7.18.1.tgz", + "integrity": "sha512-KaZh+X/6UtEp28x51AUYZDMg9NGoz2ja3dNHa+ta/tk40vCzKhQ/RypCWBMLbmDr6//E24Vv5uPsrqXFozdkAg==", "license": "MIT", "dependencies": { - "@remix-run/router": "1.23.3", - "react-router": "6.30.4" + "react-router": "7.18.1" }, "engines": { - "node": ">=14.0.0" + "node": ">=20.0.0" }, "peerDependencies": { - "react": ">=16.8", - "react-dom": ">=16.8" + "react": ">=18", + "react-dom": ">=18" } }, "node_modules/react-style-singleton": { @@ -4300,6 +4310,12 @@ "semver": "bin/semver.js" } }, + "node_modules/set-cookie-parser": { + "version": "2.7.2", + "resolved": "https://registry.npmjs.org/set-cookie-parser/-/set-cookie-parser-2.7.2.tgz", + "integrity": "sha512-oeM1lpU/UvhTxw+g3cIfxXHyJRc/uidd3yK1P242gzHds0udQBYzs3y8j4gCCW+ZJ7ad0yctld8RYO+bdurlvw==", + "license": "MIT" + }, "node_modules/siginfo": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz", diff --git a/frontend/package.json b/frontend/package.json index b717f1c..bdcb475 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -21,7 +21,7 @@ "openapi-fetch": "^0.17.0", "react": "^18.3.1", "react-dom": "^18.3.1", - "react-router-dom": "^6.28.0" + "react-router-dom": "^7.18.1" }, "devDependencies": { "@testing-library/jest-dom": "^6.9.1", diff --git a/frontend/src/api/schema.d.ts b/frontend/src/api/schema.d.ts index 0da6bd6..b45aab4 100644 --- a/frontend/src/api/schema.d.ts +++ b/frontend/src/api/schema.d.ts @@ -1336,6 +1336,61 @@ export interface components { /** Total */ total?: number | null; }; + /** EvidenceCriticSummary */ + EvidenceCriticSummary: { + /** + * Contradicted + * @default 0 + */ + contradicted: number; + /** + * Insufficient Evidence + * @default 0 + */ + insufficient_evidence: number; + /** + * Status + * @enum {string} + */ + status: "completed" | "fallback"; + /** + * Supported + * @default 0 + */ + supported: number; + /** + * Total + * @default 0 + */ + total: number; + /** Verifier */ + verifier: string; + }; + /** EvidenceVerification */ + EvidenceVerification: { + /** Counterevidence Ids */ + counterevidence_ids?: string[]; + /** Evidence Ids */ + evidence_ids?: string[]; + /** Rationale */ + rationale: string; + /** + * Requires Human Review + * @default true + * @constant + */ + requires_human_review: true; + /** + * Status + * @enum {string} + */ + status: "supported" | "contradicted" | "insufficient_evidence"; + /** + * Support Score + * @default 0 + */ + support_score: number; + }; /** GateReviewPayload */ GateReviewPayload: { /** Reviewed By Id */ @@ -1734,6 +1789,8 @@ export interface components { }; /** SourceCitation */ SourceCitation: { + /** Document Hash */ + document_hash?: string | null; /** Evidence Link */ evidence_link?: string | null; /** Excerpt */ @@ -1742,12 +1799,20 @@ export interface components { file: string; /** Id */ id: string; + /** Locator */ + locator?: string | null; /** Page */ page?: number | null; /** Paragraph Id */ paragraph_id?: string | null; /** Score */ score?: number | null; + /** + * Source Kind + * @default policy + * @enum {string} + */ + source_kind: "current_document" | "policy" | "history"; }; /** * SubAgentStatus @@ -1783,6 +1848,10 @@ export interface components { * @enum {string} */ category: "Spoofing" | "Tampering" | "Repudiation" | "InformationDisclosure" | "DenialOfService" | "ElevationOfPrivilege"; + /** Citation Ids */ + citation_ids?: string[]; + /** Confidence */ + confidence?: number | null; /** Description */ description: string; dread_score?: components["schemas"]["DreadScore"] | null; @@ -1790,6 +1859,7 @@ export interface components { id: string; /** Mitigations */ mitigations?: string[]; + verification?: components["schemas"]["EvidenceVerification"] | null; }; /** ThreatModel */ ThreatModel: { @@ -1797,6 +1867,7 @@ export interface components { methodology?: ("STRIDE" | "DREAD" | "STRIDE_DREAD") | null; /** Threats */ threats?: components["schemas"]["Threat"][]; + verification_summary?: components["schemas"]["EvidenceCriticSummary"] | null; }; /** TokenRequest */ TokenRequest: { diff --git a/frontend/src/components/ThreatEvidencePanel.test.tsx b/frontend/src/components/ThreatEvidencePanel.test.tsx new file mode 100644 index 0000000..a207d72 --- /dev/null +++ b/frontend/src/components/ThreatEvidencePanel.test.tsx @@ -0,0 +1,85 @@ +import { render, screen } from "@testing-library/react"; +import { describe, expect, it } from "vitest"; + +import type { SourceCitation, ThreatModel } from "../types"; +import { ThreatEvidencePanel } from "./ThreatEvidencePanel"; + +const evidenceId = "DOC-1-L8-L10-abcdef"; + +const threatModel: ThreatModel = { + methodology: "STRIDE", + verification_summary: { + status: "completed", + verifier: "ollama:inference", + supported: 1, + contradicted: 0, + insufficient_evidence: 1, + total: 2 + }, + threats: [ + { + id: "T1", + category: "Tampering", + description: "Unsigned webhook payloads can be modified.", + affected_component: "Payment webhook", + mitigations: ["Implement HMAC verification."], + confidence: 0.94, + citation_ids: [evidenceId], + verification: { + status: "supported", + support_score: 0.94, + rationale: "The design explicitly says HMAC validation is not implemented.", + evidence_ids: [evidenceId], + counterevidence_ids: [], + requires_human_review: true + } + }, + { + id: "T2", + category: "Repudiation", + description: "Administrators may deny privileged actions.", + mitigations: [], + citation_ids: [], + verification: { + status: "insufficient_evidence", + support_score: 0.2, + rationale: "The design does not describe audit logging.", + evidence_ids: [], + counterevidence_ids: [], + requires_human_review: true + } + } + ] +}; + +const sources: SourceCitation[] = [ + { + id: evidenceId, + file: "checkout-architecture.md", + excerpt: "HMAC signature validation is planned but is not implemented.", + locator: "L8-L10", + document_hash: "abcdef", + source_kind: "current_document" + } +]; + +describe("ThreatEvidencePanel", () => { + it("shows critic status, exact evidence, and the human-review gate", () => { + render( + + ); + + expect( + screen.getByRole("heading", { name: "Threat Evidence Critic" }) + ).toBeInTheDocument(); + expect(screen.getByText("Verification complete")).toBeInTheDocument(); + expect(screen.getAllByText("Supported")).toHaveLength(2); + expect(screen.getByText("Insufficient evidence")).toBeInTheDocument(); + expect( + screen.getByText("checkout-architecture.md#L8-L10") + ).toBeInTheDocument(); + expect( + screen.getAllByText("Human review is required before approval.") + ).toHaveLength(2); + }); +}); diff --git a/frontend/src/components/ThreatEvidencePanel.tsx b/frontend/src/components/ThreatEvidencePanel.tsx new file mode 100644 index 0000000..69ee9aa --- /dev/null +++ b/frontend/src/components/ThreatEvidencePanel.tsx @@ -0,0 +1,223 @@ +import { + Bot, + CircleHelp, + Quote, + ShieldAlert, + ShieldCheck +} from "lucide-react"; + +import type { SourceCitation, ThreatModel } from "../types"; +import { Badge, Card, CardHeader, EmptyState } from "./ui"; + +type VerificationStatus = + | "supported" + | "contradicted" + | "insufficient_evidence"; + +const statusView: Record< + VerificationStatus, + { + label: string; + tone: "good" | "bad" | "warn"; + icon: typeof ShieldCheck; + } +> = { + supported: { + label: "Supported", + tone: "good", + icon: ShieldCheck + }, + contradicted: { + label: "Contradicted", + tone: "bad", + icon: ShieldAlert + }, + insufficient_evidence: { + label: "Insufficient evidence", + tone: "warn", + icon: CircleHelp + } +}; + +export function ThreatEvidencePanel({ + threatModel, + sources +}: { + threatModel: ThreatModel; + sources: SourceCitation[]; +}) { + const threats = threatModel.threats ?? []; + const summary = threatModel.verification_summary; + const sourcesById = new Map(sources.map((source) => [source.id, source])); + + return ( + + + {summary.status === "completed" ? "Verification complete" : "Safe fallback"} + + ) : null + } + /> + {summary ? ( +
+ + + + +
+ ) : null} + + {threats.length ? ( +
+ {threats.map((threat) => { + const verification = threat.verification; + const status = verification?.status; + const view = status ? statusView[status] : null; + const Icon = view?.icon ?? Bot; + const evidence = (verification?.evidence_ids ?? []) + .map((id) => sourcesById.get(id)) + .filter((source): source is SourceCitation => Boolean(source)); + const counterevidence = (verification?.counterevidence_ids ?? []) + .map((id) => sourcesById.get(id)) + .filter((source): source is SourceCitation => Boolean(source)); + + return ( +
+
+
+
+
+

+ {threat.description} +

+
+ {view ? ( + {view.label} + ) : ( + Not verified + )} +
+ + {verification ? ( +
+
+
+ Critic rationale +
+ + support {(verification.support_score * 100).toFixed(0)}% + +
+

+ {verification.rationale} +

+ {verification.requires_human_review ? ( +

+ Human review is required before approval. +

+ ) : null} +
+ ) : null} + + {evidence.map((source) => ( + + ))} + {counterevidence.map((source) => ( + + ))} +
+ ); + })} +
+ ) : ( + + )} + + ); +} + +function SummaryMetric({ + label, + value, + tone +}: { + label: string; + value: number; + tone: "good" | "bad" | "warn" | "neutral"; +}) { + const colors = { + good: "text-good", + bad: "text-bad", + warn: "text-warn", + neutral: "text-text" + }; + return ( +
+
{value}
+
{label}
+
+ ); +} + +function EvidenceQuote({ + source, + label, + tone +}: { + source: SourceCitation; + label: string; + tone: "good" | "bad"; +}) { + return ( +
+
+
+

+ {source.excerpt} +

+
+ ); +} diff --git a/frontend/src/pages/Assessments.tsx b/frontend/src/pages/Assessments.tsx index 2f35ccc..5be7bce 100644 --- a/frontend/src/pages/Assessments.tsx +++ b/frontend/src/pages/Assessments.tsx @@ -2,6 +2,7 @@ import { Check, FileUp, MessageSquare, RefreshCw, Send, Shield, X } from "lucide import { FormEvent, useEffect, useMemo, useState } from "react"; import { SeverityBadge, StatusBadge, taskTitle } from "../components/domain"; +import { ThreatEvidencePanel } from "../components/ThreatEvidencePanel"; import { Badge, Button, @@ -502,7 +503,12 @@ function ReportSections({ ) : } - {report.threat_model ? : null} + {report.threat_model ? ( + + ) : null} {report.vulnerabilities?.length ? : null} {report.cross_phase_refs?.length ? : null} diff --git a/frontend/src/types.ts b/frontend/src/types.ts index d125b9f..f246636 100644 --- a/frontend/src/types.ts +++ b/frontend/src/types.ts @@ -12,6 +12,7 @@ export type RiskItem = Schemas["RiskItem"]; export type ComplianceGap = Schemas["ComplianceGap"]; export type Remediation = Schemas["Remediation"]; export type SourceCitation = Schemas["SourceCitation"]; +export type Threat = Schemas["Threat"]; export type ThreatModel = Schemas["ThreatModel"]; export type Vulnerability = Schemas["Vulnerability"]; export type CrossPhaseRef = Schemas["CrossPhaseRef"]; diff --git a/tests/test_assessment_contract.py b/tests/test_assessment_contract.py index 6c4f024..d0dfc2e 100644 --- a/tests/test_assessment_contract.py +++ b/tests/test_assessment_contract.py @@ -1,4 +1,11 @@ -from app.models.assessment import AssessmentReport, Remediation, RiskItem +from app.models.assessment import ( + AssessmentReport, + EvidenceVerification, + Remediation, + RiskItem, + SourceCitation, + Threat, +) def test_report_contract_defaults_to_v2(): @@ -31,3 +38,33 @@ def test_contract_accepts_critical_remediation_and_finding_evidence(): assert remediation.priority == "critical" assert risk.citation_ids == ["E1"] + + +def test_contract_carries_threat_evidence_verdicts_and_stable_locators(): + verdict = EvidenceVerification( + status="supported", + support_score=0.93, + rationale="The design exposes a public unsigned webhook.", + evidence_ids=["DOC-1-L10-L12-abc"], + ) + threat = Threat( + id="T1", + category="Tampering", + description="Webhook payloads can be modified.", + confidence=0.93, + citation_ids=["DOC-1-L10-L12-abc"], + verification=verdict, + ) + source = SourceCitation( + id="DOC-1-L10-L12-abc", + file="architecture.md", + excerpt="HMAC validation is not implemented.", + document_hash="abcdef", + locator="L10-L12", + source_kind="current_document", + ) + + assert threat.verification is not None + assert threat.verification.status == "supported" + assert source.locator == "L10-L12" + assert source.source_kind == "current_document" diff --git a/tests/test_evidence_critic.py b/tests/test_evidence_critic.py new file mode 100644 index 0000000..86980d6 --- /dev/null +++ b/tests/test_evidence_critic.py @@ -0,0 +1,206 @@ +import json +from unittest.mock import AsyncMock, patch + +import pytest + +from app.models.assessment import AssessmentReport, Threat, ThreatModel +from app.models.parser import ParsedDocument, ParsedDocumentMetadata +from app.services.evidence_critic import ( + build_document_passages, + verify_threat_model_evidence, +) + + +def _document() -> ParsedDocument: + return ParsedDocument( + content=( + "# Checkout architecture\n" + "The public API gateway validates OIDC access tokens before forwarding " + "requests to the payment service.\n\n" + "# Webhooks\n" + "The public payment webhook accepts event payloads. HMAC signature " + "validation is planned but is not implemented.\n" + ), + metadata=ParsedDocumentMetadata( + filename="checkout.md", + type="md", + file_hash="abc123def456", + ), + ) + + +def _report(phase: str = "design") -> AssessmentReport: + return AssessmentReport( + task_id="00000000-0000-0000-0000-000000000001", + phase=phase, + status="completed", + summary="Design review", + threat_model=ThreatModel( + methodology="STRIDE", + threats=[ + Threat( + id="T1", + category="Spoofing", + description=( + "Attackers may bypass authentication at the API gateway." + ), + affected_component="API gateway", + ), + Threat( + id="T2", + category="Tampering", + description="Unsigned webhook payloads may be modified in transit.", + affected_component="payment webhook", + ), + ], + ), + ) + + +def test_build_document_passages_creates_stable_current_document_locators(): + first = build_document_passages([_document()]) + second = build_document_passages([_document()]) + + assert [passage.id for passage in first] == [passage.id for passage in second] + assert first[0].id.startswith("DOC-1-L1-L2-abc123def4") + assert first[0].locator == "L1-L2" + assert first[0].document_hash == "abc123def456" + + +@pytest.mark.asyncio +async def test_evidence_critic_verifies_threats_and_attaches_exact_sources(): + passages = build_document_passages([_document()]) + gateway_evidence = passages[0].id + webhook_evidence = passages[1].id + response = { + "verdicts": [ + { + "threat_id": "T1", + "status": "contradicted", + "support_score": 0.91, + "evidence_ids": [], + "counterevidence_ids": [gateway_evidence], + "rationale": "The gateway explicitly validates OIDC tokens.", + }, + { + "threat_id": "T2", + "status": "supported", + "support_score": 0.96, + "evidence_ids": [webhook_evidence], + "counterevidence_ids": [], + "rationale": "The public webhook has no implemented HMAC validation.", + }, + ] + } + + with patch( + "app.services.evidence_critic.invoke_llm", + new=AsyncMock(return_value=f"```json\n{json.dumps(response)}\n```"), + ) as verifier: + result = await verify_threat_model_evidence(_report(), [_document()]) + + verifier.assert_awaited_once() + assert result.threat_model is not None + assert result.threat_model.threats[0].verification is not None + assert result.threat_model.threats[0].verification.status == "contradicted" + assert result.threat_model.threats[1].verification is not None + assert result.threat_model.threats[1].verification.status == "supported" + assert result.threat_model.verification_summary is not None + assert result.threat_model.verification_summary.supported == 1 + assert result.threat_model.verification_summary.contradicted == 1 + assert {source.id for source in result.sources} == { + gateway_evidence, + webhook_evidence, + } + assert all(source.source_kind == "current_document" for source in result.sources) + assert all( + source.evidence_link.startswith("document://") for source in result.sources + ) + + +@pytest.mark.asyncio +async def test_evidence_critic_rejects_invented_citation_ids(): + response = { + "verdicts": [ + { + "threat_id": "T1", + "status": "supported", + "support_score": 0.99, + "evidence_ids": ["DOC-INVENTED"], + "counterevidence_ids": [], + "rationale": "Invented citation.", + } + ] + } + with patch( + "app.services.evidence_critic.invoke_llm", + new=AsyncMock(return_value=json.dumps(response)), + ): + result = await verify_threat_model_evidence(_report(), [_document()]) + + assert result.threat_model is not None + verdict = result.threat_model.threats[0].verification + assert verdict is not None + assert verdict.status == "insufficient_evidence" + assert verdict.evidence_ids == [] + assert "valid current-document citation" in verdict.rationale + assert result.sources == [] + + +@pytest.mark.asyncio +async def test_evidence_critic_handles_malformed_reference_arrays_safely(): + response = { + "verdicts": [ + { + "threat_id": "T1", + "status": "supported", + "support_score": 0.99, + "evidence_ids": "DOC-1-L1-L2", + "counterevidence_ids": None, + "rationale": "Malformed references.", + } + ] + } + with patch( + "app.services.evidence_critic.invoke_llm", + new=AsyncMock(return_value=json.dumps(response)), + ): + result = await verify_threat_model_evidence(_report(), [_document()]) + + assert result.threat_model is not None + verdict = result.threat_model.threats[0].verification + assert verdict is not None + assert verdict.status == "insufficient_evidence" + + +@pytest.mark.asyncio +async def test_evidence_critic_fails_safe_when_inference_is_unavailable(): + with patch( + "app.services.evidence_critic.invoke_llm", + new=AsyncMock(side_effect=RuntimeError("offline")), + ): + result = await verify_threat_model_evidence(_report(), [_document()]) + + assert result.threat_model is not None + assert result.threat_model.verification_summary is not None + assert result.threat_model.verification_summary.status == "fallback" + assert result.threat_model.verification_summary.insufficient_evidence == 2 + assert { + threat.verification.status + for threat in result.threat_model.threats + if threat.verification is not None + } == {"insufficient_evidence"} + + +@pytest.mark.asyncio +async def test_evidence_critic_only_runs_for_design_phase(): + verifier = AsyncMock() + with patch("app.services.evidence_critic.invoke_llm", new=verifier): + result = await verify_threat_model_evidence( + _report(phase="testing"), + [_document()], + ) + + verifier.assert_not_awaited() + assert result.threat_model is not None + assert result.threat_model.verification_summary is None diff --git a/tests/test_orchestrator.py b/tests/test_orchestrator.py index 60fa2a1..aa4f950 100644 --- a/tests/test_orchestrator.py +++ b/tests/test_orchestrator.py @@ -7,6 +7,7 @@ _build_chunk_lookup, _evidence_agent_keyword_fallback, _extract_query_seed, + _normalize_threat_model, _resolve_citations_from_llm, _split_text_with_overlap, run_assessment, @@ -268,9 +269,39 @@ def test_graph_registry_exports_assessment_mermaid(): compiled = GRAPH_REGISTRY["docsentinel_assessment"][1]() mermaid = compiled.get_graph(xray=True).draw_mermaid() assert "draft_assessment" in mermaid + assert "verify_threat_evidence" in mermaid assert "persist_gate3_control_evidence" in mermaid +def test_normalize_threat_model_accepts_common_stride_variations(): + normalized = _normalize_threat_model( + { + "methodology": "stride+dread", + "threats": [ + { + "category": "Information Disclosure", + "description": "Sensitive payloads may be exposed.", + "affected_component": "Event bus", + "dread_score": {"damage": 15, "exploitability": 0}, + }, + { + "id": "T2", + "category": "Privilege Escalation", + "description": "A shared service token grants admin access.", + }, + ], + } + ) + + assert normalized is not None + assert normalized["methodology"] == "STRIDE_DREAD" + assert normalized["threats"][0]["id"] == "T1" + assert normalized["threats"][0]["category"] == "InformationDisclosure" + assert normalized["threats"][0]["dread_score"]["damage"] == 10 + assert normalized["threats"][0]["dread_score"]["exploitability"] == 1 + assert normalized["threats"][1]["category"] == "ElevationOfPrivilege" + + @pytest.mark.asyncio async def test_citations_from_reviewer_output_used_over_fallback(): """Reviewer chunk_id sources become the report citations.""" @@ -322,6 +353,101 @@ async def mock_llm(system_prompt, user_prompt): assert report.sources[0].excerpt == "Access tokens must expire within 1 hour." +@pytest.mark.asyncio +async def test_design_assessment_runs_inference_time_threat_evidence_critic(): + import json + + from app.services.evidence_critic import build_document_passages + + skill = _make_skill( + id="ssdlc-design", + name="Design Agent", + risk_focus=["Threat Modeling"], + ) + document = _make_doc( + "# Webhook design\n" + "The public payment webhook accepts unsigned payloads. " + "HMAC validation is not implemented.", + filename="architecture.md", + ) + evidence_id = build_document_passages([document])[0].id + reviewed = json.dumps( + { + "summary": "Unsigned webhook threat", + "confidence": 0.8, + "risk_items": [], + "compliance_gaps": [], + "remediations": [], + "sources": [], + "threat_model": { + "methodology": "STRIDE", + "threats": [ + { + "id": "T1", + "category": "Tampering", + "description": "Unsigned webhook payloads may be modified.", + "affected_component": "payment webhook", + "mitigations": ["Implement HMAC verification."], + } + ], + }, + } + ) + critic = json.dumps( + { + "verdicts": [ + { + "threat_id": "T1", + "status": "supported", + "support_score": 0.94, + "evidence_ids": [evidence_id], + "counterevidence_ids": [], + "rationale": ( + "The current design explicitly accepts unsigned payloads." + ), + } + ] + } + ) + + async def assessment_llm(system_prompt, _user_prompt): + if "Reviewer" in system_prompt or "Validate" in system_prompt: + return reviewed + if "evidence extractor" in system_prompt: + return "architecture.md#L2: HMAC validation is not implemented." + return reviewed + + with patch("app.agent.orchestrator.get_skill_service") as mock_svc: + inst = MagicMock() + inst.get_skill.return_value = skill + inst.list_skills.return_value = [skill] + mock_svc.return_value = inst + with patch("app.agent.orchestrator.invoke_llm", side_effect=assessment_llm): + with patch("app.agent.orchestrator.get_kb_service") as mock_kb_svc: + mock_kb = MagicMock() + mock_kb.query = AsyncMock(return_value=[]) + mock_kb.query_history_responses.return_value = [] + mock_kb_svc.return_value = mock_kb + with patch( + "app.services.evidence_critic.invoke_llm", + new=AsyncMock(return_value=critic), + ): + report = await run_assessment( + uuid4(), + [document], + phase="design", + skill_id="ssdlc-design", + ) + + assert report.threat_model is not None + assert report.threat_model.verification_summary is not None + assert report.threat_model.verification_summary.supported == 1 + assert report.threat_model.threats[0].verification is not None + assert report.threat_model.threats[0].verification.status == "supported" + assert report.threat_model.threats[0].citation_ids == [evidence_id] + assert report.sources[0].source_kind == "current_document" + + @pytest.mark.asyncio async def test_langgraph_assessment_persists_gate3_control_evidence( monkeypatch,