docs: add setup guides for engine, dashboard and update project documentation

Author: AsieduDevelopmentHub

README.md

@@ -1,220 +1,128 @@
 # SentraCore
 
-SentraCore is a local system behavior intelligence layer that continuously analyzes time-based system telemetry to understand performance behavior, estimate degradation risk, and provide explainable root-cause insights with safe optimization recommendations.
+SentraCore is a **Local System Behavior Intelligence Platform** for Windows. It continuously analyzes system telemetry to understand performance behavior, detect statistical anomalies, explain the root cause of slowdowns, and forecast resource exhaustion before it occurs.
 
-It is designed to go beyond traditional monitoring tools by interpreting system behavior over time rather than simply displaying real-time system metrics.
+Unlike traditional monitoring tools that display raw snapshots, SentraCore interprets system behavior over time — answering not just *what* is happening, but *why* it is happening and *when* it will become critical.
 
 ---
 
-## Overview
-
-Most system tools focus on snapshots of current resource usage. SentraCore focuses on behavior over time.
-
-It answers three key questions:
-
-- What is happening in the system?
-- Why is it happening?
-- What is likely to happen next in terms of performance impact?
-
-SentraCore does not replace tools like Task Manager. Instead, it enhances understanding by providing behavioral intelligence and explanation layers on top of raw system data.
-
----
-
-## Core Capabilities
-
-### System Telemetry Collection
-Collects real-time system data including:
-- CPU usage trends
-- Memory usage and paging behavior
-- Disk activity and latency signals
-- Process-level resource usage
-- System events and state changes
-
----
-
-### Behavioral Modeling
-Builds a baseline model of normal system behavior per machine using:
-- Moving averages
-- Trend analysis
-- Statistical deviation tracking
-- Time-windowed behavior patterns
-
----
-
-### Anomaly Detection (Probabilistic)
-Detects deviations from normal system behavior using a risk-based model rather than deterministic rules.
-
-Outputs a system degradation risk score based on:
-- Sustained resource pressure
-- Behavioral deviation from baseline
-- Rate of change in system metrics
-- Multi-resource saturation patterns
-
----
-
-### Correlation and Root Cause Analysis
-Identifies likely contributors to system performance degradation by analyzing:
-- Temporal relationships between system events and performance changes
-- Resource contention across CPU, memory, and disk
-- Process behavior patterns over time
-
-Outputs explainable insights rather than raw process rankings.
-
----
+## Current Status
 
-### Responsiveness Modeling
-Estimates user-perceived system performance impact by analyzing:
-- Disk latency patterns
-- CPU scheduling pressure
-- Memory paging frequency
-- Context switching behavior
+**All six phases are complete and production-ready.**
 
-This translates system-level metrics into real-world usability impact such as lag or slow response.
+| Phase | Name | Status |
+|---|---|---|
+| Phase 1 | Core Telemetry Engine | Complete |
+| Phase 2 | Behavioral Intelligence Layer | Complete |
+| Phase 3 | Correlation & Root Cause Engine | Complete |
+| Phase 4 | Prediction & Risk Engine | Complete |
+| Phase 5 | Flutter Dashboard System | Complete |
+| Phase 6 | Productization Layer | Complete |
 
 ---
 
-### Process Intelligence
-Processes are analyzed based on:
-- Sustained resource impact over time
-- Historical correlation with system slowdown events
-- Contribution to system-wide pressure
-
-Processes are ranked by system impact rather than instantaneous usage.
+## What SentraCore Does
 
----
-
-### Baseline Learning
-SentraCore adapts to each machine by learning normal behavior patterns over time, including:
-- Idle behavior
-- Active workload patterns
-- Time-of-day usage trends
-- System-specific performance characteristics
+### System Stability Index
+The primary dashboard metric is the **System Stability Index (1–100)**. Unlike a raw CPU percentage, this score synthesises instantaneous resource pressure, statistical anomaly deviation, and forward-looking predictive risk into a single, actionable number. A score of 100 is perfect health.
 
----
+### Root Cause Analysis
+When the system enters a high-stress state and an alert fires, SentraCore's Correlation Engine automatically generates a **Root Cause Analysis** — identifying the primary bottleneck (CPU, Memory, or Disk), the most likely offending process, and the system event that triggered the degradation.
 
-### Event Correlation
-Tracks system events such as:
-- Application launches and closures
-- Background service execution
-- System updates
-- Disk indexing and maintenance operations
+### Predictive Forecasting
+The Prediction Engine uses Exponential Moving Averages on trend slopes to generate **Time-to-Exhaustion (ETA)** countdowns. If memory is growing at a sustained rate, SentraCore will warn you that saturation is expected in X seconds, not just that usage is high.
 
-These events improve accuracy of behavioral and correlation analysis.
+### Adaptive Baseline Learning
+SentraCore does not use static thresholds. It learns what is *normal* for the specific machine it runs on, segmented by time-of-day. A CPU spike at 2 AM during a known backup job is treated differently from the same spike at 2 PM.
 
 ---
 
-### Safety-First Optimization Layer
-SentraCore provides system optimization suggestions with strict safety constraints.
+## Architecture
 
-Three levels of intervention:
+SentraCore is structured as two decoupled layers communicating over a local WebSocket and REST API:
 
-- Recommendation Only (default)
-- User-approved actions
-- Safe automated actions (restricted to non-critical processes)
-
-No forced termination of critical system processes is performed.
+```
+Flutter Dashboard (Windows Desktop)
+         ↕  WebSocket (ws://localhost:8000/ws/live)
+         ↕  REST API  (http://localhost:8000/api/v1/)
+Python Engine (Headless Background Process)
+    ├── SystemCollector      (psutil telemetry)
+    ├── Normalizer           (EMA smoothing)
+    ├── TimeSeriesBuffer     (ring buffers)
+    ├── BaselineModel        (adaptive learning)
+    ├── TrendAnalyzer        (linear regression)
+    ├── AnomalyDetector      (Z-score analysis)
+    ├── StressEngine         (multi-state composite)
+    ├── PredictionEngine     (ETA forecasting)
+    ├── StabilityCalculator  (global health index)
+    ├── CorrelationEngine    (root cause analysis)
+    └── AlertManager         (threshold evaluation)
+```
 
 ---
 
-## System Architecture
+## Technology Stack
 
-Flutter Dashboard ↓ (WebSocket / Local API) SentraCore Engine (Python Core) ├── Telemetry Collector ├── Time-Series Storage Layer ├── Behavior Modeling Engine ├── Anomaly Detection Engine ├── Correlation Engine ├── Decision & Safety Controller ↓ Operating System (CPU, RAM, Disk, Processes)
+| Layer | Technology |
+|---|---|
+| Monitoring Engine | Python 3.11, psutil |
+| API Server | FastAPI, uvicorn |
+| Real-time Communication | WebSockets |
+| Dashboard UI | Flutter (Dart), Windows Desktop |
+| Build & Packaging | PyInstaller, Inno Setup |
+| CI/CD | GitHub Actions (ruff, pytest, flutter analyze) |
+| Testing | pytest, flutter test |
 
 ---
 
-## Data Model (Conceptual)
+## Getting Started
 
-SentraCore operates on structured time-series data:
+### Run in Development
 
-- Timestamped CPU usage
-- Memory consumption trends
-- Disk activity metrics
-- Process-level statistics
-- System event logs
+See [docs/setup/development_setup.md](docs/setup/development_setup.md) for the full guide.
 
-This enables historical analysis rather than snapshot-based monitoring.
-
----
+```powershell
+# 1. Start the Python Engine
+.venv\Scripts\python -m engine.main
 
-## System Output Example
+# 2. In a second terminal, run the Flutter Dashboard
+cd dashboard
+flutter run -d windows
+```
 
-Instead of raw metrics, SentraCore produces structured insights:
+### Install from Installer
 
-System State: Degrading  
-Primary Pressure: Memory and Disk I/O  
-Risk Level: 72% (Moderate)  
-User Impact: Application lag likely during multitasking  
-Likely Contributors: Chrome processes, background update service, increased disk queue activity  
+Download the latest `SentraCore_Setup.exe` from the [Releases](../../releases) page and run it. The installer will:
+- Place both executables in `C:\Program Files\SentraCore\`.
+- Optionally configure the engine to start automatically on Windows login.
+- Create Desktop and Start Menu shortcuts for the dashboard.
 
 ---
 
-## Technology Stack
+## Documentation
 
-- Python (core engine)
-- psutil (system telemetry)
-- FastAPI or local API layer (communication)
-- WebSockets (real-time updates)
-- Flutter Desktop (dashboard UI)
-- SQLite or lightweight time-series storage (optional)
+| Document | Description |
+|---|---|
+| [Development Setup](docs/setup/development_setup.md) | Full local development environment guide |
+| [Engine Setup](docs/setup/engine_setup.md) | Engine-specific installation reference |
+| [Dashboard Setup](docs/setup/dashboard_setup.md) | Flutter-specific setup reference |
+| [Intelligence Pipeline](docs/architecture/intelligence_layer.md) | Deep dive into all 10 processing stages |
+| [Building SentraCore](docs/architecture/building.md) | How to produce production executables and the installer |
 
 ---
 
 ## Project Philosophy
 
 SentraCore is built around five principles:
 
-1. Observation — collect raw system data
-2. Behavioral Modeling — understand normal system behavior
-3. Anomaly Detection — detect deviation patterns
-4. Correlation Analysis — identify likely contributing factors
-5. User Impact Translation — convert system data into human-readable insights
-
----
-
-## Positioning
-
-SentraCore belongs to a new category of system software:
-
-System Behavior Intelligence Layer
-
-It differs from traditional tools:
-
-- Task Manager: reactive snapshots
-- Process Explorer: manual deep inspection
-- Monitoring tools: infrastructure-focused
-
-SentraCore focuses on:
-- time-based behavior understanding
-- probabilistic performance degradation detection
-- explainable system insights
-- user-centric performance interpretation
-
----
-
-## Current Status
-
-This project is in architecture and design phase, transitioning toward MVP implementation.
-
----
-
-## Future Roadmap
-
-- Core telemetry collector implementation
-- Baseline learning engine
-- Anomaly detection model
-- Process correlation system
-- Flutter dashboard integration
-- Optimization recommendation engine
-- Optional automation layer (user-controlled)
+1. **Observation** — Collect accurate, smoothed system telemetry.
+2. **Behavioral Modeling** — Understand what is normal for this machine.
+3. **Anomaly Detection** — Detect statistically significant deviations.
+4. **Correlation Analysis** — Explain *why* the system is degrading.
+5. **Prediction** — Forecast *when* critical thresholds will be breached.
 
 ---
 
 ## License
 
 To be defined.
-
----
-
-## Author
-
-SentraCore is designed and developed as a system intelligence research and engineering project focused on next-generation local system understanding.

docs/README.md

@@ -1,12 +1,37 @@
 # SentraCore Documentation
 
-Welcome to the SentraCore documentation directory. This folder contains all necessary guides and architectural overviews for the SentraCore Mono-Repo.
+This directory contains all technical documentation for the SentraCore project.
 
-## Table of Contents
+---
 
-### Setup Guides
-- [Engine Setup](setup/engine_setup.md) - How to install and run the Python backend.
-- [Dashboard Setup](setup/dashboard_setup.md) - How to compile and run the Flutter UI.
+## Setup Guides
 
-### Architecture
-- [Intelligence Layer (Phase 2)](architecture/intelligence_layer.md) - Deep dive into the behavioral anomaly detection and trend analysis engines.
+| Guide | Description |
+|---|---|
+| [Development Setup](setup/development_setup.md) | Complete local development environment guide for both the engine and dashboard |
+| [Engine Setup](setup/engine_setup.md) | Python engine installation reference |
+| [Dashboard Setup](setup/dashboard_setup.md) | Flutter dashboard setup reference |
+
+---
+
+## Architecture & Technical Reference
+
+| Document | Description |
+|---|---|
+| [Intelligence Pipeline](architecture/intelligence_layer.md) | Deep dive into all 10 processing stages from raw telemetry to Root Cause Analysis |
+| [Building SentraCore](architecture/building.md) | How to produce standalone executables and compile the Inno Setup installer |
+
+---
+
+## Project Status
+
+All six development phases are complete:
+
+| Phase | Scope |
+|---|---|
+| Phase 1 | Core Telemetry Engine — psutil collection, normalizer, buffers, alert system |
+| Phase 2 | Behavioral Intelligence — baseline learning, anomaly detection, trend analysis |
+| Phase 3 | Correlation & RCA Engine — root cause analysis, event correlation, alert enrichment |
+| Phase 4 | Prediction & Risk Engine — ETA forecasting, risk scoring, stability index |
+| Phase 5 | Flutter Dashboard — stability indicator, prediction panel, RCA panel, charts |
+| Phase 6 | Productization — PyInstaller packaging, Flutter build, Inno Setup installer |