Skip to content

Readme.md

Latest commit

 

History

History
151 lines (106 loc) · 7.68 KB

Readme.md

File metadata and controls

151 lines (106 loc) · 7.68 KB
Raw

Python Libraries

A reference catalog of the third-party Python libraries that power security automation and tooling — grouped by role, from HTTP clients and packet crafting to crypto, CLIs, and web frameworks.

Part of the [[Readme|Python for Security Professionals]] course.

Overview

This module documents the external (pip install-able) libraries you reach for when building real security tools in Python. Each note follows the same anatomy — overview, installation, runnable examples, security use cases, best practices, and references — so you can evaluate and adopt a library quickly. All examples target authorized, in-scope systems and are strictly for defensive/educational study.

Learning Objectives

  • Choose the right library for HTTP testing, packet work, remote access, crypto, output, and web serving.
  • Install each library and run its core examples from a clean environment.
  • Map every library to concrete penetration-testing and automation tasks.
  • Combine libraries (fetch + parse, scan + display, crack + progress) into complete tools.
  • Apply security best practices — authorization, safe crypto, no plaintext secrets, TLS verification.

Prerequisites

  • [[Managing-Virtual-Environments|Managing Virtual Environments]] — install these into a venv, never system-wide.
  • [[Pip-Package-Manager|Pip Package Manager]] — installing and pinning packages.
  • [[Popular-Security-Packages|Popular Security Packages]] — how to decide whether you need a third-party package at all.

Topics Covered

HTTP & Web

  • [[requests|requests]] — high-level HTTP client for web-app and API testing
  • [[BeautifulSoup|BeautifulSoup]] — HTML/XML parsing for OSINT scraping and surface mapping
  • [[Selenium|Selenium]] — real-browser automation for JavaScript-heavy targets

Networking & Packets

  • [[Scapy|Scapy]] — packet crafting, sniffing, and custom scanners
  • [[dnspython|dnspython]] — DNS enumeration, subdomain discovery, zone-transfer tests
  • [[pwntools|pwntools]] — exploit development and CTF binary interaction

Remote Access

  • [[Paramiko|Paramiko]] — SSH command execution and SFTP automation

Crypto

  • [[Cryptography|Cryptography]] — modern encryption, hashing, KDFs, and X.509 handling
  • [[pyOpenSSL|pyOpenSSL]] — live TLS connections and certificate operations

CLI & Output

  • [[Rich|Rich]] — tables, progress bars, and colored terminal reports
  • [[Click|Click]] — decorator-based CLI framework for packaging tools
  • [[Typer|Typer]] — type-hint-driven CLIs built on Click
  • [[colorama|colorama]] — cross-platform ANSI color for lightweight status output
  • [[tqdm|tqdm]] — progress bars for long scans and brute-force loops

Web Frameworks

  • [[Flask|Flask]] — micro-framework for listeners, test targets, and dashboards
  • [[FastAPI|FastAPI]] — async, validated APIs with automatic OpenAPI docs

System

  • [[psutil|psutil]] — process, connection, and resource inventory for host triage

Practical Exercises

  • Build a mini recon tool: fetch pages with [[requests]], extract links/forms with [[BeautifulSoup]], and print a [[Rich]] table of findings.
  • Write a [[Scapy]] SYN scanner and wrap it in a [[Typer]] CLI with a [[tqdm]] progress bar.
  • Enumerate a domain's records and subdomains with [[dnspython]], then probe live hosts with [[requests]].
  • Automate a host inventory over SSH: connect with [[Paramiko]], collect process/port data with [[psutil]], and encrypt the results with [[Cryptography]].
  • Stand up a [[Flask]] or [[FastAPI]] callback listener in a lab and drive check-ins with [[requests]].

Security Applications

Reach for a third-party package when it does something genuinely hard to write yourself; otherwise the standard library is more portable and one less dependency to install on a locked-down host.

Need Package Standard-library alternative
HTTP sessions, retries, auth [[requests]] urllib.request for one-off requests
HTML parsing [[BeautifulSoup]] none — never parse HTML with regex
JavaScript-rendered pages [[Selenium]] none
Packet crafting and sniffing [[Scapy]] socket for TCP/UDP only
Arbitrary DNS record types [[dnspython]] socket.getaddrinfo for A/AAAA
Programmatic SSH and SFTP [[Paramiko]] subprocess calling ssh
CTF exploit development [[pwntools]] socket plus struct
Modern crypto and X.509 [[Cryptography]] · [[pyOpenSSL]] hashlib, hmac, secrets, ssl
Command-line interfaces [[Click]] · [[Typer]] argparse
Terminal output [[Rich]] · [[colorama]] · [[tqdm]] print and f-strings
Callback listeners, mock targets [[Flask]] · [[FastAPI]] http.server for a throwaway
Process and network inventory [[psutil]] os and subprocess

[!warning] Authorized use only Scapy, pwntools, Paramiko, and Selenium all drive traffic at real systems. Use them only against hosts you own or are explicitly authorized to test.

Common Mistakes

  • Installing system-wide with sudo pip — breaks the distribution's Python and mixes engagement dependencies into the OS.
  • Unpinned dependencies, so a scan that worked last month behaves differently today.
  • Typosquatting — the real package is beautifulsoup4, not beautifulsoup; python-nmap and nmap are different projects.
  • Disabling TLS verification (verify=False) to work around a certificate error rather than fixing the trust store.
  • Assuming an API is stable across major versions — verify against the version you have installed.
  • Adding a dependency for something the standard library already does, then being unable to install it on the target host.
  • Using low-level cryptography primitives instead of its recipes layer.

Best Practices

  • Install into a per-project virtual environment and pin with requirements.txt or pyproject.toml.
  • Prefer the standard library for anything simple; document which third-party packages are genuinely required.
  • Keep requests, urllib3, and cryptography current — all ship security fixes.
  • Check the official project page before installing an unfamiliar package.
  • Verify version-specific behaviour with importlib.metadata.version() rather than assuming.
  • Set an explicit timeout on every network call, whichever library you use.

Review Questions

  1. When is requests worth the dependency over urllib.request?
  2. What does Scapy give you that the socket module cannot?
  3. What security issue can occur if TLS verification is disabled to make a script work?
  4. How would you troubleshoot a package that imports in one shell but not another?
  5. Why should cryptography's recipes layer be preferred over its primitives?
  6. Which packages here require root privileges, and why?

Commands

# Create an isolated environment for tooling
python3 -m venv .venv && source .venv/bin/activate

# Install the full toolkit covered in this module
pip install requests beautifulsoup4 lxml selenium scapy dnspython pwntools \
            paramiko cryptography pyOpenSSL rich click "typer[all]" \
            colorama tqdm psutil Flask "fastapi[standard]"

# Freeze exact versions for reproducible engagement tooling
pip freeze > requirements.txt

References

Related Notes

  • [[Readme|Python for Security Professionals]] — course home

Navigation

  • Home: [[Readme|Course Home]]
  • See also: [[Scripts,-Modules,-Packages,-and-Libraries/Readme|9. Modules & Packages]] — the standard-library counterpart