Add grounded safety intelligence generation

.env.example

@@ -0,0 +1,5 @@
+# Safety-intelligence generation. Mock is deterministic and makes no network calls.
+LLM_PROVIDER=mock
+LLM_MODEL=
+GROQ_API_KEY=
+LLM_TIMEOUT_SECONDS=20

.gitignore

@@ -3,44 +3,50 @@ __pycache__/
 *.pyc
 *.pyo
 *.pyd
+.pytest_cache/
 *.sqlite3
 *.db
 
+# RAG indexes are generated locally from authorized documents and should not
+# be committed to the public repository.
+backend/data/knowledge/vector_store/*
+!backend/data/knowledge/vector_store/.gitkeep
+backend/data/knowledge/processed/*
+!backend/data/knowledge/processed/.gitkeep
+backend/data/knowledge/model_cache/
+.cache/
+.huggingface/
 # Virtual Environment
 .venv/
 venv/
-
 # Environment Variables
 .env
-
 # VS Code
 .vscode/
-
 # Logs
 *.log
-
 # Node
 node_modules/
-
+# Digital-twin generated datasets and scenario output.
+digital twin/output/
+digital twin/*.csv
+digital twin/*.jsonl
 # OS
 .DS_Store
 Thumbs.db
 # AI and ML Caches
-.pytest_cache/
 .mypy_cache/
 .ruff_cache/
 .coverage
 htmlcov/
 .ipynb_checkpoints/
-
 # Data and Models
 data/raw/
 data/processed/
 data/sim_output/
 *.parquet
 *.pkl
 *.joblib
-
 # Temporary Evaluation Outputs
 evaluation/canonical_scenario_timeline.csv
 evaluation/event_level_predictions.csv

AGENTS.md

@@ -0,0 +1,9 @@
+# Repository Notes
+
+- The FastAPI application and Pydantic contracts live under `backend/app`.
+- Install `requirements-dev.txt` and run backend tests with `.venv/bin/python -m pytest`.
+- Import public contracts from `app.schemas`; keep internal schema imports absolute.
+- Do not push directly to `main`.
+- Do not rename existing API fields or `maintainance.py` without checking every usage.
+- Keep safety recommendations advisory and human-approved by default.
+- Update schema tests, `docs/examples`, and `docs/SCHEMA_ARCHITECTURE.md` together.

README.md

@@ -1,12 +1,199 @@
 # SpecGuard
 
+SpecGuard is an AI-powered industrial safety intelligence platform for
+petroleum-refinery maintenance operations. It combines operational signals,
+compound-risk detection, safety-document retrieval, and human-reviewed
+interventions in one prototype platform.
 
-nahi voting chal raha hai: 
-1 -4 (decided)
-8-2 
-5-1 
-7-1 
-toh ab first wala karenge
+## Problem
 
-pakka first wala?
-ek baar research karte hai fir dedcide karte hai!
+Industrial facilities often already have SCADA, gas sensors, permits, work
+orders, shift logs, and cameras. These sources usually remain isolated,
+however, so weak signals are considered independently and are not converted
+into timely compound-risk decisions. SpecGuard connects those signals and
+returns evidence-backed context for a qualified human reviewer.
+
+## Core demo scenario
+
+The primary demonstration follows Pump P-101 during maintenance:
+
+- A hot-work permit is active.
+- Hydrocarbon gas concentration is rising.
+- Ventilation fails.
+- Workers remain in Zone B.
+- No individual signal initially exceeds its standalone alarm threshold.
+
+SpecGuard correlates the permit, atmosphere, ventilation, maintenance, and
+worker context to identify an emerging fire/explosion risk earlier than a
+single-sensor alarm workflow.
+
+## Architecture
+
+```text
+Digital twin
+    ↓
+SCADA / gas / permit / maintenance / worker / CCTV streams
+    ↓
+Compound-risk engine
+    ↓
+RiskEngineInput
+    ↓
+Safety-document and incident RAG
+    ↓
+SafetyIntelligenceResponse
+    ↓
+FastAPI
+    ↓
+Frontend command centre
+```
+
+The current RAG layer stops at evidence retrieval. Future response generation
+must remain advisory and human-approved; it must not directly control plant
+equipment.
+
+## Platform components
+
+- Synthetic petroleum-refinery digital twin with process, SCADA, gas, permit,
+  maintenance, shift-log, worker, and CCTV event streams.
+- Compound-risk contracts shared through Pydantic schemas.
+- Deterministic local safety metadata and persistent JSON vector retrieval.
+- Authorized local incident/regulatory documents and clearly labelled synthetic
+  SOPs.
+- FastAPI backend foundations and a frontend safety-command-centre scaffold.
+- Evidence-backed intervention workflow for the next intelligence layer.
+
+## Repository structure
+
+```text
+digital twin/                  # synthetic refinery simulator and scenarios
+backend/app/schemas/           # public Pydantic safety contracts
+backend/app/rag/               # loading, cleaning, chunking, embedding, retrieval
+backend/data/knowledge/        # manifest, source corpus, and generated local index
+backend/tests/                 # schema and RAG tests
+backend/examples/              # integration handoff examples
+docs/                          # schema and RAG architecture guides
+frontend/                      # frontend command-centre scaffold
+```
+
+## Current implementation status
+
+### Completed
+
+- Digital-twin simulator with compound refinery scenarios.
+- Shared safety schemas, including `RiskEngineInput` and
+  `SafetyIntelligenceResponse`.
+- PDF/Markdown/text RAG ingestion and conservative cleaning.
+- Deterministic offline embeddings and persistent local vector retrieval.
+- Optional Sentence Transformers support using MiniLM.
+- Synthetic SOP corpus and official/local incident corpus, subject to provenance
+  and licensing review.
+- Metadata filtering, deterministic safety-aware reranking, tests, and
+  subsystem documentation.
+
+### Next
+
+- Integrate the compound-risk model with live or simulated event streams.
+- Add an LLM-grounded intelligence response while preserving citations and
+  human approval.
+- Expose the intelligence workflow through a FastAPI endpoint.
+- Connect the frontend command centre.
+- Evaluate compound detection against a single-sensor baseline.
+
+## Quick start
+
+Create and activate a virtual environment, then install the core and test
+dependencies:
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+pip install -r requirements-dev.txt
+```
+
+The deterministic backend test command verified in this repository is:
+
+```bash
+PYTHONPATH=backend .venv/bin/python -m pytest backend/tests -v
+```
+
+To enable real local semantic embeddings, install the optional RAG dependency
+set. This downloads no model during tests; model weights are managed by the
+local Sentence Transformers environment:
+
+```bash
+pip install -r requirements-rag.txt
+```
+
+### Rebuild and query the knowledge base
+
+From `backend/`, the verified deterministic commands are:
+
+```bash
+cd backend
+PYTHONPATH=. ../.venv/bin/python -m app.rag.cli inspect
+PYTHONPATH=. ../.venv/bin/python -m app.rag.cli ingest --rebuild
+PYTHONPATH=. ../.venv/bin/python -m app.rag.cli query \
+  "Hydrocarbon gas is rising near Pump P-101 while hot work is active, ventilation has failed and workers are present" \
+  --mode all --top-k 6
+```
+
+Use optional MiniLM embeddings explicitly after installing
+`requirements-rag.txt`:
+
+```bash
+PYTHONPATH=. ../.venv/bin/python -m app.rag.cli ingest --rebuild \
+  --embedder sentence-transformers
+PYTHONPATH=. ../.venv/bin/python -m app.rag.cli query \
+  --embedder sentence-transformers \
+  "Hydrocarbon gas is rising while hot work is active and ventilation has failed"
+```
+
+The default is deterministic. It can also be selected through
+`RAG_EMBEDDER=deterministic` or `RAG_EMBEDDER=sentence-transformers`. The real
+model defaults to `sentence-transformers/all-MiniLM-L6-v2` and can be changed
+with `RAG_EMBEDDING_MODEL`.
+
+### RiskEngineInput handoff example
+
+After rebuilding the index, run:
+
+```bash
+PYTHONPATH=backend .venv/bin/python backend/examples/risk_to_retrieval.py
+```
+
+The example converts a validated `RiskEngineInput` into a deterministic query,
+searches operational SOP/regulatory evidence and historical incidents, and
+prints structured JSON with raw similarity, final reranking score, source
+metadata, synthetic status, and matched safety tags.
+
+### Run a digital-twin scenario
+
+The following short smoke command was verified without generating a large
+dataset:
+
+```bash
+cd "digital twin"
+../.venv/bin/python simulate.py --scenario explosion_risk --duration 10 \
+  --format json --output /tmp/specguard-digital-twin-smoke --quiet
+```
+
+For a full scenario, omit `--duration` or select another scenario such as
+`hot_work_gas_leak`, `ventilation_failure`, or `confined_space`. See
+`digital twin/README.md` for simulator details.
+
+## Documentation
+
+- [Schema architecture](docs/SCHEMA_ARCHITECTURE.md) explains public contracts,
+  validation, ownership, and integration boundaries.
+- [RAG knowledge base](docs/RAG_KNOWLEDGE_BASE.md) explains the corpus,
+  ingestion, embeddings, filters, reranking, and safety limitations.
+
+## Safety disclaimer
+
+SpecGuard is a hackathon prototype. Synthetic SOPs are explicitly **not for
+real industrial use**. Retrieved evidence and future recommendations require
+qualified human review and approval. The platform does not directly control
+industrial equipment, replace approved refinery procedures, establish universal
+alarm thresholds, or override a facility's permit, isolation, emergency, or
+engineering controls.

backend/app/intelligence/__init__.py

@@ -0,0 +1,5 @@
+"""Grounded, advisory safety-intelligence generation."""
+
+from app.intelligence.service import SafetyIntelligenceService, provider_from_environment
+
+__all__ = ["SafetyIntelligenceService", "provider_from_environment"]

backend/app/intelligence/confidence.py

@@ -0,0 +1,42 @@
+"""Transparent prototype confidence heuristic for intelligence responses."""
+
+from __future__ import annotations
+
+from app.intelligence.models import RetrievedEvidence
+from app.schemas import RiskEngineInput
+
+
+def calculate_confidence(
+    risk: RiskEngineInput,
+    sop_evidence: list[RetrievedEvidence],
+    incident_evidence: list[RetrievedEvidence],
+    *,
+    grounded_actions: int,
+    rejected_actions: int,
+    fallback_used: bool,
+    insufficient_evidence: bool,
+) -> float:
+    """Calculate an auditable heuristic score, not a calibrated probability."""
+
+    risk_confidence = (
+        risk.model_confidence if risk.model_confidence is not None else risk.risk_score
+    )
+    all_evidence = sop_evidence + incident_evidence
+    average_score = (
+        sum(item.reranked_score for item in all_evidence) / len(all_evidence)
+        if all_evidence
+        else 0.0
+    )
+    score = 0.30 * risk_confidence
+    score += 0.15 * min(len(all_evidence) / 5.0, 1.0)
+    score += 0.20 * average_score
+    score += 0.10 if sop_evidence else 0.0
+    score += 0.05 if incident_evidence else 0.0
+    score += 0.15 * min(grounded_actions / 2.0, 1.0)
+    if rejected_actions:
+        score -= 0.10 * min(rejected_actions / max(grounded_actions + rejected_actions, 1), 1.0)
+    if fallback_used:
+        score -= 0.25
+    if insufficient_evidence:
+        score -= 0.15
+    return round(max(0.05, min(0.95, score)), 3)