diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 0000000..5c73c72 --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1,9 @@ +# Default ownership: the project maintainer reviews everything. +# Add collaborators as: /path/ @username (or @org/team) +* @abhinav-phi + +# Notebook build system โ€” changes here regenerate all four notebooks +/nids-backend/notebooks/colab/_build/ @abhinav-phi + +# Specification set โ€” keep docs in sync per CONTRIBUTING.md +/docs/ @abhinav-phi diff --git a/.github/CODE_OF_CONDUCT.md b/.github/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..1548cc3 --- /dev/null +++ b/.github/CODE_OF_CONDUCT.md @@ -0,0 +1,79 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our community include: + +* Demonstrating empathy and kindness toward other people +* Being respectful of differing opinions, viewpoints, and experiences +* Giving and gracefully accepting constructive feedback +* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience +* Focusing on what is best not just for us as individuals, but for the overall community + +Examples of unacceptable behavior include: + +* The use of sexualized language or imagery, and sexual attention or advances of any kind +* Trolling, insulting or derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or email address, without their explicit permission +* Other conduct which could reasonably be considered inappropriate in a professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at **[GitHub โ€” open a private contact via the repository owner's profile](https://github.com/abhinav-phi)**. All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact:** Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community. + +**Consequence:** A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested. + +### 2. Warning + +**Community Impact:** A violation through a single incident or series of actions. + +**Consequence:** A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban. + +### 3. Temporary Ban + +**Community Impact:** A serious violation of community standards, including sustained inappropriate behavior. + +**Consequence:** A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact:** Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals. + +**Consequence:** A permanent ban from any sort of public interaction within the community. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html. + +Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder](https://github.com/mozilla/inclusion). + +[homepage]: https://www.contributor-covenant.org + +For answers to common questions about this code of conduct, see https://www.contributor-covenant.org/faq diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 0000000..28620d3 --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,100 @@ +# Contributing to The Sentinel NIDS + +First off, thank you for considering a contribution! ๐Ÿ›ก๏ธ + +This project is a Colab-only, ML-powered Network Intrusion Detection System. The entire implementation โ€” EDA, T4-GPU training, inference, embedded FastAPI backend, and the Gradio dashboard โ€” lives inside **4 Colab notebooks** that are **generated** from readable Python sources. Contributions follow that build system. + +--- + +## The Golden Rule + +> **Never edit the `.ipynb` files directly.** They are build artifacts. +> +> Edit the percent-format sources in `nids-backend/notebooks/colab/_build/`, then rebuild. + +``` +_edit_ _build/02_Training_GPU.py โ†’ python _build/pack.py โ†’ notebooks regenerated +``` + +This keeps diffs reviewable, keeps the four notebooks in sync with shared fragments (`frag_*.py`), and lets CI validate everything. + +--- + +## Development Environment + +You only need local Python for **building and testing** โ€” the notebooks themselves run on Google Colab. + +| Requirement | Purpose | +|-------------|---------| +| Python 3.10+ | Running `pack.py` and `smoke_test.py` | +| `pip install "numpy<2" pandas scikit-learn shap fastapi "uvicorn[standard]" sqlalchemy requests joblib matplotlib nbformat` | Local validation toolchain | +| Google Drive | Only for actually running the notebooks (dataset + artifacts) | + +> โš ๏ธ `numpy<2` matters locally: older `shap` builds break on NumPy 2.x. Colab resolves this automatically via its own pinned stack. + +--- + +## Contribution Workflow + +1. **Fork & branch** โ€” `feat/`, `fix/`, or `docs/`. +2. **Make your change** in `nids-backend/notebooks/colab/_build/*.py` (and `docs/` if behavior or paths changed). +3. **Rebuild the notebooks:** + ```bash + cd nids-backend/notebooks/colab/_build + python pack.py # regenerates all 4 .ipynb files + nbformat/syntax validation + ``` +4. **Run the smoke test:** + ```bash + python smoke_test.py # must print: SMOKE TEST PASSED + ``` + This executes notebook 03's embedded inference + API stack and asserts the full production contract (predictions, persistence, every 400/422 validation guard, severity mapping). +5. **(Notebook-logic changes)** Run the affected notebook top-to-bottom on Colab with the T4 runtime and confirm the summary tables. +6. **Open a Pull Request** using the provided template. + +--- + +## What Changes Where + +| Area | File(s) | +|------|---------| +| EDA notebook | `_build/01_EDA.py` | +| Training pipeline (models, Optuna, ROC/AUC) | `_build/02_Training_GPU.py` | +| Inference, `FlowExtractor`, FastAPI, replay | `_build/03_Inference_API.py` + `frag_*.py` | +| Gradio dashboard | `_build/04_Dashboard.py` | +| Shared inference stack | `_build/frag_inference.py` (single source of the 52-feature contract & severity map) | +| Shared API app | `_build/frag_api.py` | +| Documentation | `docs/NIDS_*.md`, `README.md`, `nids-backend/notebooks/colab/README.md` | + +--- + +## Code Standards + +- **Determinism:** every random operation uses `random_state=42` / seeded generators. +- **Artifact contract is sacred:** `model.pkl` must stay a sklearn-compatible estimator with `n_features_in_ == 52`; `scaler.pkl` stays a StandardScaler. Notebooks 03/04 depend on it. +- **No silent coercion:** malformed feature vectors are rejected (422), never "fixed" โ€” mirror the validation semantics in `frag_api.py`. +- **Graceful GPU degradation:** CUDA cells must fall back to CPU with a clear warning, never crash. +- **No secrets:** API keys are entered via `getpass` per session. Never commit keys, tokens, or dataset files. +- **Style:** PEP 8, type hints on function signatures, docstrings for public helpers, section headers in notebook sources (`# %% [markdown]`). + +--- + +## Commit & PR Conventions + +- **Conventional commits:** `feat:`, `fix:`, `docs:`, `chore:`, `refactor:`, `test:` โ€” e.g. `feat(colab): add per-class ROC curves to training notebook`. +- **One logical change per PR.** The repo history is split into reviewable units on purpose. +- **Docs stay in sync:** any behavior/path change gets a matching update in the relevant `docs/NIDS_*.md` and the Tracker (see `docs/NIDS_Rules.md`). + +--- + +## Reporting Bugs & Suggesting Features + +- Bugs โ†’ [Issue: Bug report](https://github.com/abhinav-phi/nids/issues/new?template=1-bug-report.md) +- Features โ†’ [Issue: Feature request](https://github.com/abhinav-phi/nids/issues/new?template=4-feature-request.md) +- Questions โ†’ [GitHub Discussions](https://github.com/abhinav-phi/nids/discussions) +- **Security vulnerabilities โ†’ [Private disclosure](https://github.com/abhinav-phi/nids/security/advisories/new) โ€” never a public issue** (see [SECURITY.md](SECURITY.md)) + +--- + +## Licensing + +By contributing, you agree that your contributions will be licensed under the [MIT License](LICENSE) that covers the project. diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml new file mode 100644 index 0000000..80c9797 --- /dev/null +++ b/.github/FUNDING.yml @@ -0,0 +1,10 @@ +# Funding options for this repository. +# Uncomment a platform and add your handle to enable the "Sponsor" button. +# See https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/displaying-a-sponsor-button-in-your-repository + +# github: [abhinav-phi] +# ko-fi: [your_kofi_username] +# patreon: [your_patreon_username] +# open_collective: [your_collective_name] +# buy_me_a_coffee: [your_bmac_username] +# custom: ["https://your-link.example"] diff --git a/.github/ISSUE_TEMPLATE/1-bug-report.md b/.github/ISSUE_TEMPLATE/1-bug-report.md new file mode 100644 index 0000000..1383f3b --- /dev/null +++ b/.github/ISSUE_TEMPLATE/1-bug-report.md @@ -0,0 +1,49 @@ +--- +name: ๐Ÿ› Bug report +about: Something in a notebook, the embedded API, or the build system is broken +title: "[Bug]: " +labels: ["bug", "triage"] +assignees: ["abhinav-phi"] +--- + +## Affected Component + + + +- [ ] `01_EDA_Colab.ipynb` โ€” EDA +- [ ] `02_Training_GPU_Colab.ipynb` โ€” training / Optuna / evaluation +- [ ] `03_Inference_API_Colab.ipynb` โ€” inference, FlowExtractor, FastAPI +- [ ] `04_Dashboard_Colab.ipynb` โ€” Gradio dashboard +- [ ] `_build/` โ€” build system (`pack.py`, `smoke_test.py`) +- [ ] `docs/` โ€” documentation mismatch + +## Environment + +- **Colab runtime:** +- **Notebook cell number / step:** +- **Artifact state:** +- **Dataset location:** + +## What Happened + + + +## Steps to Reproduce + +1. +2. +3. + +**Expected behavior:** + +**Actual behavior / error output:** + +``` + +``` + +## Additional Context + + + +> โš ๏ธ **Security vulnerabilities are reported privately** โ€” see [SECURITY.md](../SECURITY.md), not this template. diff --git a/.github/ISSUE_TEMPLATE/4-feature-request.md b/.github/ISSUE_TEMPLATE/4-feature-request.md new file mode 100644 index 0000000..cb105cb --- /dev/null +++ b/.github/ISSUE_TEMPLATE/4-feature-request.md @@ -0,0 +1,29 @@ +--- +name: ๐Ÿš€ Feature request +about: Suggest a new capability for the NIDS notebooks or pipeline +title: "[Feature]: " +labels: ["enhancement", "triage"] +assignees: ["abhinav-phi"] +--- + +## Problem / Motivation + + + +## Proposed Solution + + + +- Target notebook: + +## Alternatives Considered + + + +## Additional Context + + + +## Contribution + +- [ ] I am willing to help implement this (see [CONTRIBUTING.md](../../.github/CONTRIBUTING.md) โ€” edit `_build/*.py`, rebuild with `pack.py`, pass `smoke_test.py`) diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..aba8a67 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,14 @@ +blank_issues_enabled: false + +contact_links: + - name: โ“ Question / Support + url: https://github.com/abhinav-phi/nids/discussions + about: Ask usage questions (dataset setup, artifact flow, Colab runtime) here โ€” see also SUPPORT.md. + + - name: ๐Ÿ”’ Security Vulnerability + url: https://github.com/abhinav-phi/nids/security/advisories/new + about: Report vulnerabilities PRIVATELY via GitHub Security Advisories โ€” never in a public issue. See SECURITY.md. + + - name: ๐Ÿ“– Documentation + url: https://github.com/abhinav-phi/nids/tree/main/docs + about: PRD, TechSpec, AppFlow, Design, Schema, ImplementationPlan, Tracker and Rules โ€” most questions are answered there. diff --git a/.github/SECURITY.md b/.github/SECURITY.md new file mode 100644 index 0000000..fce30da --- /dev/null +++ b/.github/SECURITY.md @@ -0,0 +1,77 @@ +# Security Policy + +**Project:** The Sentinel โ€” Network Intrusion Detection System (NIDS) + +The Sentinel is an ML-powered security tool โ€” which means we hold it to the same standard it applies to network traffic. Thank you for taking the time to report issues responsibly. + +--- + +## Supported Versions + +| Version | Supported | Notes | +|---------|-----------|-------| +| 1.0.x (Colab Edition) | โœ… | Current release line | +| < 1.0.0 (local backend / React frontend) | โŒ | Removed in the Colab-only cleanup; fixes land on the current edition only | + +--- + +## How to Report a Vulnerability + +**Please do NOT open a public GitHub issue for security vulnerabilities.** + +1. **Preferred:** use GitHub's **[Private Vulnerability Reporting](https://github.com/abhinav-phi/nids/security/advisories/new)** โ€” this keeps the report confidential end-to-end and lets us coordinate a fix and advisory. +2. **Alternative:** open a draft security advisory through the repository's *Security* tab. + +### What to include + +- Affected notebook / component (`01_EDA`, `02_Training_GPU`, `03_Inference_API`, `04_Dashboard`, `_build/*`) +- A minimal reproduction (cell, payload, or request body) +- Impact assessment โ€” what could an attacker do? +- Any known workarounds + +### What to expect + +| Step | Target | +|------|--------| +| Acknowledgement | within **48 hours** | +| Triage & severity assessment | within **7 days** | +| Fix or mitigation | severity-dependent, coordinated with you | +| Public disclosure | after the fix ships, credit to you unless you prefer otherwise | + +--- + +## Scope + +### โœ… In scope + +- The **embedded FastAPI API** in notebook 03 (input validation, metadata handling, WebSocket fan-out, response sanitization) +- **Artifact integrity** โ€” anything that could make `model.pkl` / `scaler.pkl` / `label_encoder.pkl` load or behave incorrectly (e.g., bypassing the `n_features_in_` parity guards) +- **Notebook code paths** that execute or persist attacker-controlled data (feature vectors, `_source_ip` metadata, replayed CSV content) +- **Supply-chain issues** in the pinned Colab dependencies (`_build` install cells) + +### โŒ Out of scope + +- **Google Colab platform security** โ€” report to [Google VRP](https://bughunters.google.com) instead +- **Denial-of-service against the demo API** โ€” the embedded server is intentionally demo-grade: optional API key (unset by default), no TLS, per-IP rate limit 120/min, WS client cap 20 +- **Missing authentication / TLS on the public tunnel URLs** โ€” documented limitation of the Colab edition; reports on this alone will be closed as *documented behavior* unless accompanied by a concrete hardening proposal +- **Attacks requiring write access to your Google Drive or Colab VM** โ€” that is platform compromise, not a project vulnerability +- **Findings from intentionally adversarial inputs in `_build/smoke_test.py`** โ€” those are the test suite's own fixtures + +--- + +## Known & Accepted Limitations (by design) + +These are documented in `README.md` / `docs/NIDS_PRD.md` and are **not** vulnerabilities: + +1. No user authentication / authorization โ€” the optional `X-API-Key` + WS token is defence-in-depth, not access control. +2. No TLS termination on the Cloudflare quick tunnel / Gradio share URL. +3. Per-process rate limiting and WS counters (single-process demo deployment). +4. Up to 8 of 52 missing features are zero-filled by design and surfaced via `missing_features`. + +## Safe Handling + +If you discover an exploitable issue, please do not run it against public deployments you do not own. For local reproduction, use the synthetic-flow fixtures in `_build/smoke_test.py` rather than live traffic. + +--- + +*This policy follows the standard responsible-disclosure practice. For general questions (not vulnerabilities), see [SUPPORT.md](SUPPORT.md).* diff --git a/.github/SUPPORT.md b/.github/SUPPORT.md new file mode 100644 index 0000000..84e119d --- /dev/null +++ b/.github/SUPPORT.md @@ -0,0 +1,47 @@ +# Support + +Thanks for using **The Sentinel NIDS**! Here's how to get help quickly. + +--- + +## ๐Ÿ“– Read the docs first (most questions are answered here) + +| Question | Where | +|----------|-------| +| How do I run the project on Colab? | [`nids-backend/notebooks/colab/README.md`](nids-backend/notebooks/colab/README.md) | +| What does each notebook do? | [README โ€” Notebooks table](README.md) | +| Dataset setup (where to put the CSV) | Colab README โ†’ *One-time setup* | +| How artifacts flow between notebooks | Colab README โ†’ *How artifacts flow* | +| Full product requirements / architecture | [`docs/NIDS_PRD.md`](docs/NIDS_PRD.md) ยท [`docs/NIDS_TechSpec.md`](docs/NIDS_TechSpec.md) | +| What deliberately doesn't run on Colab | Colab README โ†’ *What deliberately does NOT run on Colab* | +| Contributing / build system | [`CONTRIBUTING.md`](.github/CONTRIBUTING.md) | + +--- + +## ๐Ÿ†˜ Quick troubleshooting + +| Symptom | Fix | +|---------|-----| +| `Dataset not found!` assertion in notebook 01/02 | Upload `cicids2017_cleaned.csv` to **Google Drive โ†’ `MyDrive/nids_data/`**, then re-run | +| `Trained artifacts not found!` in notebook 03/04 | Run **`02_Training_GPU_Colab.ipynb`** first (it syncs to `MyDrive/nids_artifacts/`), or upload `nids_artifacts.zip` to `/content/` | +| `NO GPU โ€” go to Runtime โ†’ Change runtime type` | Select **T4 GPU** runtime and re-run the cell | +| Optuna / SHAP cell fails on NumPy | Colab handles versions automatically; locally install `numpy<2` (see CONTRIBUTING) | +| Dashboard shows "Backend offline" | Re-run the API/server cells of notebook 04 (or 03) | + +--- + +## ๐Ÿ—ฃ๏ธ Ask a question + +- **[GitHub Discussions](https://github.com/abhinav-phi/nids/discussions)** โ€” best place for usage questions, ideas and sharing your results. + +## ๐Ÿ› Found a bug? + +Open an issue with the **[Bug report template](https://github.com/abhinav-phi/nids/issues/new?template=1-bug-report.md)** โ€” include the notebook, cell number, runtime, and the full error output. + +## ๐Ÿ”’ Security issue? + +**Never** in a public issue โ€” use [private vulnerability reporting](https://github.com/abhinav-phi/nids/security/advisories/new) (see [SECURITY.md](.github/SECURITY.md)). + +## โฑ๏ธ Response expectations + +This is a solo-maintained project โ€” expect responses within a few days. CI runs on every push/PR, so `pack.py` + `smoke_test.py` failures are caught automatically. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..bbc5f96 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,50 @@ + + +## Description + + + +## Motivation & Context + + + +## Type of Change + + + +- [ ] ๐Ÿ› Bug fix (non-breaking change that fixes an issue) +- [ ] โœจ New feature (non-breaking change that adds functionality) +- [ ] ๐Ÿ’ฅ Breaking change (fix or feature that would cause existing notebooks/artifacts to fail) +- [ ] ๐Ÿ““ Notebook change (training / inference / EDA / dashboard logic) +- [ ] ๐Ÿ“ Documentation update +- [ ] ๐Ÿ”ง Build / CI / tooling +- [ ] โ™ป๏ธ Refactor (no behavior change) + +## Areas Touched + + + +- [ ] `_build/01_EDA.py` (EDA notebook) +- [ ] `_build/02_Training_GPU.py` (training pipeline) +- [ ] `_build/03_Inference_API.py` / `frag_inference.py` / `frag_api.py` (inference + API) +- [ ] `_build/04_Dashboard.py` (Gradio dashboard) +- [ ] `docs/` (specification set) +- [ ] `README.md` / `.github/` + +## Checklist + + + +- [ ] `python _build/pack.py` runs clean (notebooks regenerate, nbformat + syntax validation pass) +- [ ] `python _build/smoke_test.py` prints **SMOKE TEST PASSED** +- [ ] Notebook logic changes were executed top-to-bottom on Colab (T4) without errors +- [ ] Artifact contract preserved: `model.pkl` sklearn-compatible, `n_features_in_ == 52`, StandardScaler +- [ ] Determinism: all new randomness uses `random_state=42` / seeded generators +- [ ] GPU cells degrade gracefully to CPU +- [ ] No secrets, no dataset files, no generated artifacts committed +- [ ] Docs updated (`docs/NIDS_*.md` / README) if behavior or paths changed +- [ ] Conventional commit title (`feat:`, `fix:`, `docs:`, `chore:`, โ€ฆ) + +## Screenshots / Output + + diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..bb47ac4 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,66 @@ +# Changelog + +All notable changes to **The Sentinel โ€” Network Intrusion Detection System** are documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +--- + +## [Unreleased] + +### Planned +- NSL-KDD / UNSW-NB15 multi-dataset generalization experiments +- Live topology graph of aggregated flows (Gradio `graphviz` panel) +- Model registry + automated retraining experiments (MLflow on Colab) + +--- + +## [1.0.0] โ€” 2026-08-28 ยท Colab Edition + +The complete rewrite of the deployment model: the entire NIDS โ€” EDA, T4-GPU training, inference, embedded API, and dashboard โ€” now runs inside **4 Google Colab notebooks**, generated from versioned Python sources. + +### Added + +**Notebook suite** (all T4-GPU aware, artifact contract preserved) +- **`01_EDA_Colab.ipynb`** โ€” CICIDS2017 exploration: class distribution & imbalance, data-quality audit, feature distributions, correlation analysis, attack-vs-normal behaviour, key-numbers summary table. +- **`02_Training_GPU_Colab.ipynb`** โ€” the full training pipeline: chunked stratified sampling (~400k rows, float32), cleaning, label encoding, stratified split, dual scalers (Standard + Robust), **7-model arena** ranked by Macro F1, **XGBoost on T4 GPU** with a CPU-vs-GPU timing bake-off, **PyTorch MLP (256-128-64)** with class-weighted loss and early stopping, **Optuna Bayesian tuning** (20 trials), **per-class ROC curves + AUC**, **normalized confusion matrix**, per-class F1 + benign FPR, SHAP smoke test, GPU benchmark table, and a training-summary table. +- **`03_Inference_API_Colab.ipynb`** โ€” the embedded backend: production inference stack (severity map, cached SHAP `TreeExplainer`, artifact parity guards), the 52-feature `FlowExtractor`, synthetic attack flows (DDoS / port-scan / brute force / normal), **global + per-class SHAP deep-dive plots**, a **FastAPI server** (`/health`, `/api/predict`, `/api/alerts`, `/api/stats`, `/api/ip-leaderboard`, `/ws/live`) with the full validation-guard contract, balanced **CSV replay**, an **API contract summary**, a public **Cloudflare tunnel**, and an optional **Gemini mini-chatbot**. +- **`04_Dashboard_Colab.ipynb`** โ€” a **Gradio command center** (KPI strip, attack pie, severity bars, 3-second live alert feed, attacker leaderboard, attack timeline, per-alert SHAP explainer, traffic-injection lab) plus static dashboard PNGs for reports. + +**Build & verification system** (`nids-backend/notebooks/colab/_build/`) +- Percent-format notebook sources (`01_EDA.py`, `02_Training_GPU.py`, `03_Inference_API.py`, `04_Dashboard.py`) with shared fragments (`frag_discovery`, `frag_inference`, `frag_api`, `frag_server`). +- `pack.py` โ€” regenerates all four `.ipynb` files with T4-GPU metadata; runs syntax + nbformat validation on every rebuild. +- `smoke_test.py` โ€” executes notebook 03's embedded inference + API stack end-to-end and asserts the production contract (predictions, persistence, every 400/422 validation guard, severity mapping). **Status: PASSED.** + +**Project hygiene** +- `README.md` rewritten for the Colab-only workflow; `docs/` specification set (PRD, TechSpec, AppFlow, Design, Schema, ImplementationPlan, Tracker, Rules) fully synced. +- `.github/` community kit: `SECURITY.md` (private disclosure policy), `CONTRIBUTING.md` (build-system workflow), `CODE_OF_CONDUCT.md`, `SUPPORT.md`, `CODEOWNERS`, `FUNDING.yml`, PR template, structured issue templates, and a **GitHub Actions CI** workflow (`pack.py` + `smoke_test.py` on every push/PR). + +### Changed + +- **ISSUE-05 (fixed)** โ€” notebook label detection now resolves the CSV's `Attack Type` column directly (previously printed `Label column: None`). +- **ISSUE-06 (fixed)** โ€” benign FPR is computed with the benign index looked up from the label encoder (the old `evaluate.py` hardcoded class 0, which was `Bots`). +- Model suite refocused: Optuna-tuned XGBoost and a PyTorch MLP replace the SVM-RBF / Voting / Stacking entries of the local 9-model suite (ranking metric unchanged: Macro F1). +- Documentation legend extended with ๐ŸŸ  LOCAL-ONLY for components present only in git history. + +### Removed + +- **Local-only runtime** (kept in git history): `nids-backend/src/` (FastAPI backend, sniffer, extractor, simulators), `nids-frontend/` (React dashboard), `tests/` (78 pytest cases), `venv/` (~1.1 GB), `model.pkl`/scalers/`nids.db`, and the three original notebooks โ€” all superseded by the notebook-embedded implementations and the `_build/` smoke test. +- Root utility dumps and scripts (`combine_files.py`, `combine_nids_clean.py`, `remove_comments.py`, codebase txt/md), historical `NIDS_AuditReport.md`, `NIDS_Final_Report.pdf`, `VIVA_GUIDE.md` โ€” ~1.4 GB freed overall. + +--- + +## [0.x] โ€” Local Edition (pre-2026-08-28) + +The original local-machine implementation, preserved in git history (`main` before `a577e28`): + +- FastAPI backend (`src/api/`) with strict input validation, optional API-key auth, per-IP rate limiting, WebSocket alert streaming, SQLite/WAL persistence, and a hardened LangChain + Gemini chatbot. +- Scapy packet capture (`src/capture/sniffer.py`) with 5-tuple flow assembly, retry/backoff delivery, and counters. +- 9-model training pipeline (`src/model/train.py`), 52-feature `FlowExtractor` (`src/features/extractor.py`), SHAP-explained inference (`src/model/predict.py`). +- React 18 + TypeScript + Vite command-center dashboard with vitest suites (21 specs); 78 pytest backend tests. +- Final security-hardening pass (PH10): WS auth + client cap, body-size cap, metadata IP validation, event-loop offloading, timezone-aware UTC, SHAP JSON safety, corrupt-artifact degradation (113 tracked items | 104 done). + +--- + +[unreleased]: https://github.com/abhinav-phi/nids/compare/v1.0.0...HEAD +[1.0.0]: https://github.com/abhinav-phi/nids/commits/main