Skip to content
BranchesTags

Repository files navigation

HackerOne Intelligence Platform

AI-first enterprise vulnerability intelligence platform with natural language querying, intelligent insights, RESTful API, and interactive dashboards.

View Live Dashboard

Python FastAPI Streamlit DuckDB OpenAI

Overview

An AI-first, Intelligence platform that transforms 10,094 HackerOne vulnerability reports into actionable insights through:

  • AI Assistant - Natural language queries with intelligent SQL generation powered by OpenAI GPT-4o-mini
  • Data-Driven Insights - Automated pattern detection, concentration analysis, and vulnerability trend identification
  • Interactive Dashboard - Real-time analytics with 8 specialized pages and AI-powered exploration
  • Secure REST API - JWT authentication with role-based access control
  • High Performance - DuckDB-powered analytics with <100ms response times

Platform Highlights

  • 10,094 Reports analyzed from HackerOne's disclosed reports dataset
  • 344 Organizations tracked across the bug bounty ecosystem
  • 3,895 Security Researchers contributing to platform security
  • 151 Vulnerability Types identified and categorized
  • 53.3% Bounty Rate indicating strong report quality

Architecture

High-Level Design (HLD)

HLD Architecture

View Editable HLD Diagram (Open in draw.io)

Low-Level Design (LLD)

LLD Architecture

View Editable LLD Diagram (Open in draw.io)

The LLD diagram shows:

  • Database Schema: Star schema with fact and dimension tables
  • Business Views: Pre-aggregated metrics for performance
  • REST API Endpoints: All 15+ API routes with authentication
  • Component Interactions: ELT pipeline, API, Dashboard, AI Engine
  • Technology Stack: Complete tech stack with versions

Tech Stack: Python • FastAPI • Streamlit • DuckDB • OpenAI • Plotly • JWT

Quick Start

Prerequisites

  • Python 3.11+
  • OpenAI API key (optional, for AI features)

Installation

# 1. Install dependencies
pip install -r requirements.txt

# 2. Run ELT pipeline (downloads 10,094 reports from HuggingFace)
python run_pipeline.py
# This will:
# - Download data from Hacker0x01/disclosed_reports (train split)
# - Create DuckDB database with star schema
# - Build dimension and fact tables
# - Generate business views for analytics

# 3. Configure AI (optional - required for AI Assistant)
cp .env.example .env
# Add your OPENAI_API_KEY to .env

Running the Platform

Dashboard (Streamlit)

python run_dashboard.py
# → http://localhost:8501

API Server (FastAPI)

python run_api.py
# → http://localhost:8000
# → http://localhost:8000/docs (Swagger UI)

Features

Dashboard (8 Pages)

  • Executive Dashboard - Comprehensive metrics, KPIs, and recent activity
  • Threat Intelligence - Vulnerability analysis, attack patterns, and bounty economics
  • Program Benchmarks - Organization performance comparison and maturity analysis
  • Community Analytics - Researcher statistics, engagement metrics, and top performers
  • Market Evolution - Temporal trends, volume analysis, and market dynamics
  • Strategic Insights - Data-driven recommendations, concentration analysis, and ROI projections
  • Security Reference - Vulnerability taxonomy, CWE mappings, and knowledge base
  • Data Workbench - Advanced filtering, search, and CSV export capabilities
  • AI Assistant - Natural language query interface with intelligent SQL generation

REST API (15+ Endpoints)

  • Authentication - JWT-based login with role management
  • Vulnerabilities - CRUD operations with filtering
  • Organizations - Metrics and performance data
  • Reporters - Researcher statistics
  • Trends - Time-series and severity analysis
  • AI Queries - Natural language to SQL conversion
  • Admin - User management (admin-only)

API Usage

Postman Collection

Import the pre-configured Postman collection for instant API testing:

Collection Files:

Import Steps:

  1. Open Postman
  2. Click Import → Select both JSON files
  3. Select the "HackerOne API" environment
  4. Run the Login request to get your token (auto-saved to environment)
  5. Test any endpoint - authentication is handled automatically!

Authentication

# Login
curl -X POST "http://localhost:8000/api/v1/auth/login" \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "password": "admin123"}'

# Response: {"access_token": "eyJ...", "token_type": "bearer"}

Query Endpoints

# Get all vulnerabilities
curl "http://localhost:8000/api/v1/vulnerabilities" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Get specific vulnerability
curl "http://localhost:8000/api/v1/vulnerabilities/SQL%20Injection" \
  -H "Authorization: Bearer YOUR_TOKEN"

# AI-powered query
curl -X POST "http://localhost:8000/api/v1/query/nlp" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "Show me top 5 vulnerabilities"}'

Interactive Docs: http://localhost:8000/docs

Demo Accounts

Username Password Role Access
admin admin123 Admin Full platform access
mailru mailru123 Customer Mail.ru data only
shopify shopify123 Customer Shopify data only

AI Assistant

Intelligent Query Detection

The AI automatically distinguishes between data queries and conversational questions:

Data Queries (returns SQL + results)

"Show me vulnerabilities with high bounty rates"
"Which organizations have the best programs?"
"Top 5 critical vulnerabilities"

Conversational (returns explanations)

"What is this platform?"
"How does RBAC work?"
"Explain the data model"

Features

  • Smart Intent Detection - Automatically routes to appropriate handler
  • SQL Generation - Converts natural language to DuckDB queries
  • View-Aware - Uses optimized business views (vw_*)
  • Context Memory - Remembers last 5 interactions
  • RBAC Integration - Respects user permissions in queries

Example Response

{
  "query": "Show me top 5 vulnerabilities",
  "sql_generated": "SELECT * FROM vw_vulnerability_metrics ORDER BY total_reports DESC LIMIT 5",
  "results": [{"weakness_name": "XSS", "total_reports": 1234, ...}],
  "explanation": "Found 5 vulnerabilities ordered by report count"
}

Note: Requires OPENAI_API_KEY in .env

Project Structure

├── src/
│   ├── api/
│   │   ├── auth.py          # JWT authentication & RBAC
│   │   ├── routes.py        # API endpoints
│   │   ├── models.py        # Pydantic schemas
│   │   └── main.py          # FastAPI app
│   ├── ai/
│   │   └── nlp_query.py     # AI query engine
│   ├── dashboard/
│   │   └── app.py           # Streamlit dashboard
│   ├── elt/
│   │   ├── extract.py       # Data extraction
│   │   ├── load.py          # Data loading
│   │   └── transform.py     # View creation
│   └── database/
│       ├── schema.py        # Database schema
│       └── connection.py    # DuckDB connection
├── data/
│   ├── raw/                 # Source CSV files
│   └── hackerone.duckdb     # Analytics database
├── docs/
│   └── ARCHITECTURE.md      # Technical documentation
├── run_pipeline.py          # Data pipeline runner
├── run_api.py              # API server
└── run_dashboard.py        # Dashboard server

Security

  • JWT Authentication - Secure token-based auth with expiration
  • Role-Based Access Control - Admin vs. Customer permissions
  • Data Isolation - Organizations see only their data
  • SQL Injection Prevention - Parameterized queries and escaping
  • Secrets Management - Environment-based configuration

Performance

Metric Value
Dataset Size 10,094 reports
Organizations 344 programs
Researchers 3,895 contributors
Vulnerability Types 151 distinct
Database Size ~15MB (DuckDB)
API Response Time <100ms avg
Dashboard Load <2s
AI Query Time 1-3s

Data Insights

Verified Metrics (from 10,094 reports)

Top Vulnerabilities:

  1. Information Disclosure - 1,010 reports (10.0%)
  2. Cross-site Scripting (XSS) - Generic - 741 reports (7.3%)
  3. Violation of Secure Design Principles - 598 reports (5.9%)
  4. Improper Authentication - Generic - 528 reports (5.2%)
  5. Improper Access Control - Generic - 457 reports (4.5%)

Platform Statistics:

  • Market Concentration: Top 3 vulnerabilities account for 29.1% of all reports
  • Bounty Success Rate: 53.3% overall (5,383 bounties awarded)
  • Active Organizations: 344 bug bounty programs
  • Security Researchers: 3,895 contributors
  • Vulnerability Diversity: 151 distinct weakness types

Data Source: HuggingFace dataset Hacker0x01/disclosed_reports (train split)

Technology Stack

Backend

  • FastAPI 0.104+ (REST API)
  • DuckDB 0.9+ (Analytics Database)
  • Python 3.11+ (Core Language)

Frontend

  • Streamlit 1.29+ (Dashboard)
  • Plotly (Visualizations)

AI/ML

  • OpenAI GPT-4o-mini (NLP)
  • Custom query detection engine

Security

  • python-jose (JWT)
  • passlib (Password hashing)

Documentation

Contributing

Author: Hitesh Kumar
Date: March 2026

License

This project is built for educational and demonstration purposes.

Built with Python, FastAPI, Streamlit, DuckDB, and OpenAI

About

Scalable enterprise vulnerability intelligence platform with AI-driven analytics, customer data access, and agentic capabilities. Built for HackerOne's business reporting and customer insights - Assignment

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages