🛡️ Dark Web & Threat Intelligence Monitoring System

An enterprise-grade, full-stack cybersecurity threat intelligence platform designed to aggregate, analyze, visualize, and classify malicious feeds and dark web telemetry in real time.

Built with a Flask REST API, MongoDB Cloud Atlas, and an interactive React Dashboard, this platform ingests global open-source threat feeds (OSINT), provides AI-driven risk scoring, visualizes geographic attack distribution on a global map, and empowers security analysts with role-based monitoring capabilities.

---

✨ Key Features

• 🌐 Live Threat Feed Ingestion (OSINT)

  • Automated integration with OpenPhish, URLhaus, and AlienVault OTX (Open Threat Exchange).
  • Background scheduler (APScheduler) automatically fetches and deduplicates global threat feeds every hour.
  • Manual sync capability for immediate threat updates.

• 📊 Comprehensive Analytics & Visualizations

  • Severity Chart: High, Medium, Low, and Critical severity distribution.
  • Threat Types: Malware, Phishing, Botnet, Ransomware, Compromised Server tracking.
  • Source Breakdown: Intel distribution across feed providers and custom entries.
  • Timeline Chart: Historic threat ingestion trends.

• 🗺️ Interactive Global Threat Map

  • Live spatial mapping of threat origins powered by Leaflet and ip-api IP geolocation.
  • Interactive map markers with country coordinates and IP details.

• 🤖 AI Threat Classification & Risk Scoring

  • Rule-based & heuristic NLP classification model to evaluate risk level, category, and potential impact.
  • Generates actionable mitigation recommendations for SOC analysts.

• 🔐 Role-Based Access Control (RBAC)

  • Support for Admin and Analyst roles.
  • Analysts can view telemetry, inspect details, and run analytics.
  • Admins retain write privileges (adding/deleting threat IOCs).

• ⚡ High-Performance REST API

  • Automated indexing on MongoDB collections for instant regex search across URLs, IPs, and threat titles.
  • Embedded Flask-Limiter rate-limiting protection to safeguard API endpoints.

• 📄 Data Export & Reporting

  • Export live threat datasets into CSV and Excel (.xlsx) formats for reporting and offline analysis.

---

🏗️ Architecture & Project Structure

Dark-Web-Monitoring-System/
├── backend/
│   ├── app.py                 # Flask server entry point & APScheduler configuration
│   ├── config.py              # Application settings
│   ├── requirements.txt       # Python dependencies
│   ├── .env                   # Backend environment variables
│   ├── database/
│   │   └── mongodb.py         # MongoDB connection & index configuration
│   ├── models/
│   │   └── threat.py          # Threat data model factory
│   ├── routes/
│   │   ├── auth.py            # User authentication endpoints
│   │   ├── dashboard.py       # Metrics & telemetry aggregation endpoints
│   │   └── threats.py         # Threat CRUD, search, and feed sync endpoints
│   ├── services/
│   │   ├── classifier.py      # AI Threat classification & risk engine
│   │   ├── geo_service.py     # IP Geolocation service
│   │   ├── openphish_service.py # OpenPhish feed consumer
│   │   ├── otx_service.py       # AlienVault OTX consumer
│   │   └── urlhaus_service.py   # URLhaus feed consumer
│   └── utils/
├── frontend/
│   ├── package.json           # React dependencies & scripts
│   ├── public/
│   └── src/
│       ├── App.js             # Main routing setup
│       ├── components/        # Reusable UI charts, maps, tables, modals & cards
│       └── pages/             # Dashboard, Threats, Analytics, Global Map, Settings
└── docs/                      # Project documentation and presentation files

---

🛠️ Technology Stack

Domain	Technology / Library
Frontend Framework	React 18, React Router v6
Data Visualization	Chart.js, React-Chartjs-2
Geographic Mapping	Leaflet, React-Leaflet
HTTP Client	Axios
Export Engines	FileSaver, XLSX
Backend Framework	Python 3.10+, Flask
Background Jobs	APScheduler
Security & Rate Limits	Flask-Limiter, Flask-CORS
Database	MongoDB Atlas, PyMongo
OSINT Intelligence Feeds	AlienVault OTX, URLhaus, OpenPhish, AbuseIPDB

---

🚀 Quick Start Guide

Prerequisites

Ensure you have the following installed on your machine:

• Python 3.10+
• Node.js 18+ & npm
• MongoDB Atlas account (or a local MongoDB instance)

---

1️⃣ Backend Setup

1. Navigate to the backend directory:

   cd backend

2. Create a virtual environment (optional but recommended):

   python -m venv venv
   # On Windows:
   venv\Scripts\activate
   # On macOS/Linux:
   source venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt

4. Configure Environment Variables: Create a .env file in the backend/ directory with the following variables:

   MONGO_URI=mongodb+srv://<username>:<password>@cluster.mongodb.net/darkweb_db?retryWrites=true&w=majority
   SECRET_KEY=your_secret_key_here
   OTX_API_KEY=your_alienvault_otx_api_key
   ABUSEIPDB_API_KEY=your_abuseipdb_api_key

5. Run the Flask Development Server:

   python app.py

   The backend API will start at http://127.0.0.1:5000.

---

2️⃣ Frontend Setup

1. Open a new terminal and navigate to the frontend directory:

   cd frontend

2. Install Node modules:

   npm install

3. Start the React application:

   npm start

   The dashboard will automatically open at http://localhost:3000.

---

🛰️ API Endpoints Summary

Method	Endpoint	Description	Access
GET	/	System status and active scheduler health	Public
GET	/api/dashboard/stats	Aggregated threat counts, severity & country stats	All
GET	/api/threats	Retrieve all recorded threat intelligence IOCs	All
POST	/api/threats	Manually insert a new threat IOC	Admin
DELETE	/api/threats/<id>	Remove a specific threat IOC	Admin
GET	/api/search?q=<query>	Regex search across titles, URLs, IPs	All
POST	/api/sync-feeds	Force immediate synchronization with OSINT feeds	All

---

🤝 Contributing

Contributions are welcome! Follow these steps to contribute:

1. Fork the repository.
2. Create your feature branch (git checkout -b feature/AmazingFeature).
3. Commit your changes (git commit -m 'Add some AmazingFeature').
4. Push to the branch (git push origin feature/AmazingFeature).
5. Open a Pull Request.

---

📜 License

Distributed under the MIT License. See LICENSE for more information.