BBTrecon

BBTrecon is a modular reconnaissance framework built in Python 3.12+ for pentesting, bug bounty workflows, and HTB-style target discovery.

It focuses on a clean, production-style architecture with asynchronous networking, structured outputs, optional integration with external enumeration tools, and a simple CLI that supports running individual phases or the full pipeline.

Features

• Asynchronous recon pipeline powered by asyncio and aiohttp
• Certificate Transparency enumeration through crt.sh
• Wayback Machine CDX enumeration for passive host discovery
• Optional external tooling support for subfinder and amass
• HTTP and HTTPS resolution of discovered subdomains
• Common Crawl harvesting using the latest 3 indexes
• Structured output under output/<domain>/
• JSON reporting for downstream automation
• Rich terminal output and file-based logging
• Modular codebase with clear separation of concerns

Recon Workflow

Phase 1: Subdomain Enumeration

BBTrecon collects subdomains from:

• crt.sh
• Wayback Machine CDX API
• subfinder if installed
• amass if installed

All results are normalized, deduplicated, and saved to:

output/<domain>/subdomains.txt

Phase 2: HTTP Resolution

BBTrecon probes discovered subdomains over both https:// and http:// using asynchronous requests to identify live endpoints.

Results are saved to:

output/<domain>/alive.txt

Phase 3: Common Crawl URL Harvesting

BBTrecon queries the latest 3 Common Crawl indexes, collects archived URLs within the target scope, and filters results against currently alive hosts.

Results are saved to:

output/<domain>/urls.txt

Phase 4: Structured Reporting

Each run produces a JSON report containing metadata, configuration, and artifact locations:

output/<domain>/report.json

Project Structure

BBTrecon/
├── main.py
├── requirements.txt
├── README.md
└── bbtrecon/
    ├── core/
    │   └── orchestrator.py
    ├── modules/
    │   ├── subdomain.py
    │   ├── resolver.py
    │   └── commoncrawl.py
    └── utils/
        └── logger.py

Requirements

• Python 3.12+
• pip
• Optional:
  • subfinder
  • amass

Installation

Create and activate a virtual environment:

python3 -m venv .venv
source .venv/bin/activate

Install dependencies:

pip install -r requirements.txt

Usage

Show Help

python3 main.py -h

Full Recon

python3 main.py -d example.com --full

Full Recon Without External Tools

python3 main.py -d example.com --full --no-external

Run Individual Phases

Subdomain enumeration:

python3 main.py -d example.com --enum

HTTP resolution:

python3 main.py -d example.com --resolve

Common Crawl harvesting:

python3 main.py -d example.com --crawl

Tune Concurrency and Timeout

python3 main.py -d example.com --full --concurrency 20 --timeout 20

Enable Verbose Logging

python3 main.py -d example.com --full --verbose

CLI Options

-d, --domain         Target domain
--enum               Run subdomain enumeration
--resolve            Run HTTP resolution
--crawl              Run Common Crawl harvesting
--full               Run the full pipeline
--no-external        Disable subfinder/amass execution
--concurrency        Maximum concurrent network tasks
--timeout            HTTP timeout in seconds
--output-root        Base output directory
--verbose            Enable debug logging

Output Layout

Example output directory:

output/example.com/
├── alive.txt
├── bbtrecon.log
├── report.json
├── subdomains.txt
└── urls.txt

Example report.json

{
  "tool": "BBTrecon",
  "domain": "example.com",
  "configuration": {
    "run_enum": true,
    "run_resolve": true,
    "run_crawl": true
  },
  "results": {
    "subdomains": {
      "count": 42,
      "file": "output/example.com/subdomains.txt"
    },
    "alive_endpoints": {
      "count": 18,
      "file": "output/example.com/alive.txt"
    },
    "archived_urls": {
      "count": 137,
      "file": "output/example.com/urls.txt"
    }
  }
}

Notes

• --resolve expects subdomains.txt to already exist unless you also run --enum or --full.
• --crawl expects alive.txt to already exist unless you also run --resolve or --full.
• External tools are used only if they are installed and discoverable in PATH.
• Common Crawl coverage depends on whether archived content exists for the target scope.

Development

Basic syntax validation:

python3 -m compileall main.py bbtrecon

Legal and Ethical Use

Use BBTrecon only against systems you own or are explicitly authorized to assess. You are responsible for complying with applicable laws, platform rules, and program scopes.

License

Add your preferred license here before publishing, for example MIT.