Added new Files

README.md

@@ -1,2 +1,55 @@
-# SentinelNet
-The goal of this project is to develop an AI-powered Network Intrusion Detection System (NIDS) capable of identifying malicious network trafic and cyber-attacks in real time. By leveraging machine learning techniques, the system will classify trafic as normal or suspicious based on historical data. 
+# SentinelNet – AI-Powered Network Intrusion Detection System (NIDS)
+
+## 📌 Project Overview
+The goal of this project is to develop an AI-powered Network Intrusion Detection System (NIDS) capable of identifying malicious network traffic and cyber-attacks in real time.  
+By leveraging machine learning techniques, the system classifies traffic as normal or suspicious based on historical data. It processes network traffic records, extracts relevant features, trains classification models, and generates alerts for detected anomalies.  
+
+---
+
+## 🎯 Objectives
+- Understand network traffic data and common types of cyberattacks.  
+- Apply machine learning algorithms to detect intrusions.  
+- Build and evaluate classification models (e.g., Decision Tree, Random Forest, SVM).  
+- Perform feature engineering and anomaly detection.  
+- Generate alerts or logs for detected threats.  
+- Prepare a detailed report and presentation showcasing the project.  
+
+---
+
+## 📊 Datasets
+The project uses well-known public network intrusion datasets for training and testing:  
+- **NSL-KDD Dataset:** https://www.unb.ca/cic/datasets/nsl.html
+- **CICIDS2017 Dataset:** https://www.unb.ca/cic/datasets/ids-2017.html  
+
+---
+
+## 🧩 Modules to be Implemented
+1. Dataset Acquisition and Exploration  
+2. Data Cleaning and Preprocessing  
+3. Feature Engineering and Selection  
+4. Model Building and Training  
+5. Evaluation and Performance Analysis  
+6. Alerts and Reporting  
+
+---
+
+## 🛠️ Tech Stack
+- **Language:** Python  
+- **Libraries:** Pandas, NumPy, Matplotlib, Scikit-learn, Seaborn  
+
+---
+
+## 🚀 Future Work
+- Extend system to handle real-time traffic streams.  
+- Explore deep learning models (e.g., CNN, LSTM) for improved detection.  
+- Integrate with SIEM tools for enterprise deployment.  
+
+---
+
+## 🤝 Contributing
+Contributions are welcome! Please feel free to submit issues and pull requests to improve the project.  
+
+---
+
+## 📄 License
+This project is licensed under the **MIT License** 

docs/data_overview.md

@@ -0,0 +1,43 @@
+# Network Intrusion Detection Datasets Overview
+
+## 1. NSL-KDD Dataset
+
+### Dataset Source
+- File: `KDDTrain+.txt`
+- Description: NSL-KDD is a refined version of the original KDD Cup 1999 dataset.
+
+### Dataset Shape
+- Number of rows: 125,973
+- Number of columns: 42
+- Label column: Last column
+
+### Labels
+- Unique labels: normal, neptune, smurf, satan, ipsweep, and others
+- Top 5 attack types:
+  1. neptune
+  2. smurf
+  3. normal
+  4. satan
+  5. ipsweep
+
+## 2. CICIDS2017 Dataset
+
+### Dataset Source
+- File: `DoS-Wednesday-no-metadata.parquet`
+- Description: CICIDS2017 captures realistic network traffic including normal and attack flows.
+
+### Dataset Shape
+- Rows and columns: 2,830,000+ × 80+ 
+- Label column: `Label`
+
+### Labels
+- Unique attack types: DoS Hulk, DoS GoldenEye, DoS slowloris, DDoS, PortScan, FTP-Patator, SSH-Patator, Normal
+- Top 5 frequent attack types:
+  1. DoS Hulk
+  2. DoS GoldenEye
+  3. DoS slowloris
+  4. DDoS
+  5. PortScan
+
+## Summary
+Both datasets provide labeled network traffic data for evaluating intrusion detection systems. NSL-KDD is smaller and classical, while CICIDS2017 is large-scale and more realistic.

documentation.txt

@@ -0,0 +1 @@
+https://docs.google.com/document/d/166nwy5J_9mbznkFrL2Cru06aG-sX15F1yFbCU8psI2A/edit?addon_store&tab=t.0

requirements.txt

@@ -0,0 +1,3 @@
+pandas==2.3.1
+numpy==2.3.2
+matplotlib==3.10.5

scripts/main.py

@@ -0,0 +1,100 @@
+import pandas as pd
+import matplotlib.pyplot as plt
+
+
+# NSL-KDD Dataset
+
+
+# Load dataset
+df = pd.read_csv("./data/KDDTrain+.txt", header=None)
+
+# Rename columns immediately
+columns = [
+    'duration','protocol_type','service','flag','src_bytes','dst_bytes','land','wrong_fragment','urgent','hot',
+    'num_failed_logins','logged_in','num_compromised','root_shell','su_attempted','num_root','num_file_creations',
+    'num_shells','num_access_files','num_outbound_cmds','is_host_login','is_guest_login','count','srv_count',
+    'serror_rate','srv_serror_rate','rerror_rate','srv_rerror_rate','same_srv_rate','diff_srv_rate',
+    'srv_diff_host_rate','dst_host_count','dst_host_srv_count','dst_host_same_srv_rate','dst_host_diff_srv_rate',
+    'dst_host_same_src_port_rate','dst_host_srv_diff_host_rate','dst_host_serror_rate','dst_host_srv_serror_rate',
+    'dst_host_rerror_rate','dst_host_srv_rerror_rate','outcome','level'
+]
+df.columns = columns
+
+print("Number of rows:", len(df))
+print("\nUnique labels in outcome:", df["outcome"].unique())
+
+# Top 5 frequent attack types
+print("\nTop 5 attack types:")
+print(df["outcome"].value_counts().head(5))
+
+# Plot attack distribution
+attack_counts = df["outcome"].value_counts().head(10)
+plt.figure(figsize=(10, 6))
+attack_counts.plot(kind="bar")
+plt.title("Top 10 Attack Types in NSL-KDD Dataset")
+plt.xlabel("Attack Type")
+plt.ylabel("Count")
+plt.xticks(rotation=90)
+plt.show()
+
+# Dataset overview
+print("\n--- Dataset Overview ---")
+print(df.head())
+print(df.info())
+print(df.describe())
+
+print("\nFeature Types:")
+print(df.dtypes)
+
+# Group attacks into broader categories
+attack_mapping = {
+    'neptune': 'DoS', 'smurf': 'DoS', 'back': 'DoS', 'teardrop': 'DoS', 'pod': 'DoS',
+    'satan': 'Probe', 'ipsweep': 'Probe', 'nmap': 'Probe', 'portsweep': 'Probe',
+    'guess_passwd': 'R2L', 'ftp_write': 'R2L', 'imap': 'R2L', 'phf': 'R2L',
+    'multihop': 'R2L', 'warezmaster': 'R2L', 'warezclient': 'R2L',
+    'buffer_overflow': 'U2R', 'loadmodule': 'U2R', 'rootkit': 'U2R', 'perl': 'U2R',
+    'normal': 'Normal'
+}
+df['category'] = df['outcome'].map(attack_mapping).fillna('Other')
+
+print("\nAttack categories distribution:")
+print(df['category'].value_counts())
+
+# Plot categories
+df['category'].value_counts().plot(kind='bar', figsize=(6,4))
+plt.title("Attack Categories in NSL-KDD")
+plt.xlabel("Category")
+plt.ylabel("Count")
+plt.show()
+
+# Class imbalance check
+print("\nClass Imbalance Check (%):")
+total = len(df)
+imbalance = (df['category'].value_counts() / total) * 100
+print(imbalance)
+
+
+
+# CICIDS2017 Dataset
+
+
+# Load dataset
+cicids = pd.read_parquet("./data/DoS-Wednesday-no-metadata.parquet")
+
+print("\nCICIDS2017 Dataset Loaded")
+print("Shape:", cicids.shape)
+
+if "Label" in cicids.columns:
+    print("\nUnique attack types:", cicids["Label"].nunique())
+    print("\nTop 5 frequent attack types:\n", cicids["Label"].value_counts().head())
+
+    cicids["Label"].value_counts().plot(kind="bar", figsize=(8,5))
+    plt.title("CICIDS2017 Attack Type Distribution")
+    plt.xlabel("Attack Type")
+    plt.ylabel("Count")
+    plt.show()
+
+print("\n--- CICIDS Dataset Overview ---")
+print(cicids.head())
+print(cicids.info())
+print(cicids.describe())