Skip to content

Latest commit

Β 

History

55 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

🦈 Surfshark Proxy Studio

Enterprise Distributed Egress Engine, Physical vs Virtual Location Discovery & Apple-Grade Telemetry Hub

Python 3.11+ Docker Multi-Node Cloudflare Tunnel Pytest 60/60 PASS Apple Dark UI License MIT


🌟 Overview

Surfshark Proxy Studio is an enterprise-grade proxy orchestration engine and distributed egress gateway. Engineered for high-throughput web scraping, API load distribution, and automated workflows, it transforms Surfshark's global infrastructure into an isolated, containerized multi-node HTTP/SOCKS5 proxy cluster with sub-second health checks, automated server intelligence, and a sleek real-time monitoring dashboard.

🌐 Live Public Gateway


πŸš€ Key Highlights

  • Physical vs Virtual Server Intelligence Engine
    Real-time auto-discovery across 142+ Surfshark clusters. Automatically categorizes physical bare-metal nodes (95 locations) versus virtual routing locations (47 locations), evaluates live latency, and ranks egress nodes.

  • Docker Multi-Node Egress Pool
    Containerized egress architecture with strict Linux network namespace isolation. Spin up multiple proxy exit nodes concurrently on dedicated ports (31001-31005 HTTP, 32001-32005 SOCKS5) without route leaking or IP collision.

  • Pure Python HTTP Automation Client
    Zero browser driver overhead. Execute direct, high-concurrency REST interactions using modern HTTP clients (requests / urllib / httpx) with transparent per-request or sticky session IP rotation.

  • Apple-Grade Dark Glassmorphic Dashboard
    Engineered with Tailwind CSS and modern DOM styling. Features live telemetry, real-time node latency visualizers, egress health scores, protocol distribution matrices, and one-click proxy credential exports.

  • Zero-Trust Cloudflare Tunnel Ingress
    Pre-configured for Cloudflare Tunnels (cloudflared). Deploy securely on any local VPS or private server without opening public firewall ports or exposing internal gateway sockets.


πŸ—οΈ System Architecture

               +-------------------------------------------------------------+
               |                  External Clients & Scraping Ops            |
               +------------------------------+------------------------------+
                                              |
                        +---------------------+---------------------+
                        | HTTPS Ingress                             | HTTP / SOCKS5 Proxy Traffic
                        v                                           v
         +-----------------------------+             +-------------------------------+
         |   Cloudflare Edge Network   |             |   Public Proxy Hub :33101-33105|
         | (proxy-studio.indrayuda...) |             +---------------+---------------+
         +--------------+--------------+                             |
                        | Secure Tunnel                              | Authenticated Proxy Bridge
                        v                                            |
         +-----------------------------+                             v
         |     cloudflared Daemon      |              +-----------------------------+
         +--------------+--------------+              |  Docker Multi-Node Pods     |
                        |                             |  +-----------------------+  |
            +-----------+-----------+                 |  | Node 01: ID (Physical)|  |
            |                       |                 |  | SOCKS5 / HTTP Proxy   |  |
            v                       v                 |  +-----------------------+  |
+-----------------------+  +-----------------------+  |  | Node 02: SG (Physical)|  |
|  Apple Dark Glass UI  |  |  FastAPI Core Engine  |  |  | SOCKS5 / HTTP Proxy   |  |
|  /studio              |  |  Port :8808           |  |  +-----------------------+  |
+-----------------------+  +-----------+-----------+  |  | Node 03: JP (Physical)|  |
                                       |              |  | SOCKS5 / HTTP Proxy   |  |
                                       | SQLite / ORM |  +-----------------------+  |
                                       v              +--------------+--------------+
                        +--------------------------+                 |
                        | Cluster Discovery Engine |                 v
                        | 142 Surfshark Endpoints  |======> [ Global Surfshark Nodes ]
                        +--------------------------+

πŸ“‚ Project Directory Structure

surfshark-proxy-studio/
β”œβ”€β”€ configs/
β”‚   β”œβ”€β”€ nodes.yaml               # Active node definitions & server_type filters
β”‚   β”œβ”€β”€ surfshark-locations.json # Discovered 142+ clusters with physical/virtual tags
β”‚   └── surfshark/               # OpenVPN profiles per node container
β”œβ”€β”€ docker/
β”‚   └── node/
β”‚       β”œβ”€β”€ Dockerfile           # Alpine + OpenVPN + Tinyproxy + Dante SOCKS5
β”‚       └── entrypoint.sh        # Self-healing VPN tunnel & proxy daemon
β”œβ”€β”€ src/proxy_studio/
β”‚   β”œβ”€β”€ api.py                   # FastAPI application & Apple Dark Studio UI
β”‚   β”œβ”€β”€ cli.py                   # Typer CLI manager
β”‚   β”œβ”€β”€ config.py                # Configuration loader & validator
β”‚   β”œβ”€β”€ db.py                    # SQLite database engine
β”‚   β”œβ”€β”€ models.py                # Node, observation & score data models
β”‚   β”œβ”€β”€ scoring.py               # IP quality & captcha readiness scoring
β”‚   β”œβ”€β”€ public_proxy.py          # Authenticated public proxy forwarder
β”‚   └── services/
β”‚       β”œβ”€β”€ nodes.py             # Node management, start/stop/restart
β”‚       β”œβ”€β”€ profile_rotation.py  # Smart Physical/Virtual rotation engine
β”‚       β”œβ”€β”€ query.py             # Filtered proxy exporter & statistics
β”‚       β”œβ”€β”€ safe_start.py        # Throttled container pool bootstrap
β”‚       └── scanner.py           # Egress IP & IPQualityScore probe runner
β”œβ”€β”€ scripts/
β”‚   β”œβ”€β”€ generate_surfshark_location_catalog.py # 142-cluster auto-discovery
β”‚   β”œβ”€β”€ luckywatch_client.py     # Pure Python HTTP automation client
β”‚   └── visual_qa_audit.py       # Multi-viewport browser visual tester
β”œβ”€β”€ tests/
β”‚   β”œβ”€β”€ test_api.py              # REST API endpoint tests
β”‚   β”œβ”€β”€ test_cli.py              # CLI commands test suite
β”‚   β”œβ”€β”€ test_profile_rotation.py # Physical vs Virtual rotation tests
β”‚   β”œβ”€β”€ test_scoring.py          # Quality calculation verification
β”‚   └── test_server_type_filter.py # Filter engine exhaustive tests
β”œβ”€β”€ docker-compose.yml           # Multi-node compose stack
β”œβ”€β”€ pyproject.toml               # Python package specification
└── README.md

⚑ Quickstart Guide

1. Prerequisites

  • Docker Engine 24.0+ & Docker Compose v2+
  • Python 3.11+ (if running bare metal)
  • Active Surfshark VPN Credentials (OpenVPN manual service credentials)

2. Configuration & Credentials

Place manual OpenVPN credentials in secrets/surfshark.auth:

your_openvpn_username
your_openvpn_password

Configure nodes in configs/nodes.yaml:

manager:
  bind_host: 127.0.0.1
  bind_port: 8808
  database_url: sqlite:///data/proxy-studio.sqlite

nodes:
  - id: surfshark-node-01
    http_proxy_port: 31001
    socks_proxy_port: 32001
    server_type: physical # Options: physical | virtual | any
    rotation_mode: reconnect_profile
    enabled: true

3. Running with Docker Compose

Start the proxy studio stack:

# Start all node containers in detached mode
docker compose up -d

# Check running status
docker compose ps

The services will be available at:

  • Web UI Dashboard: http://localhost:8808/studio
  • HTTP Proxy Node 1: http://127.0.0.1:31001
  • SOCKS5 Proxy Node 1: socks5://127.0.0.1:32001

πŸ’» CLI & Automation Usage

Command Line Interface

# Initialize database & sync configured nodes
python -m proxy_studio.cli init

# Scan all active nodes for IP quality & risk scores
python -m proxy_studio.cli scan

# Check status of all proxies in terminal table
python -m proxy_studio.cli status

# Rotate node to a physical location
python -m proxy_studio.cli rotate surfshark-node-01 --server-type physical

# Export clean proxies formatted for scrapers
python -m proxy_studio.cli export --risk-max 30

Pure Python Scraper Integration

import urllib.request
import json

# Route Python request directly through Node 01
proxy_url = "http://127.0.0.1:31001"

opener = urllib.request.build_opener(
    urllib.request.ProxyHandler({"http": proxy_url, "https": proxy_url})
)
req = urllib.request.Request("https://api.ipify.org?format=json", headers={"User-Agent": "curl/7.81.0"})

with opener.open(req, timeout=10) as res:
    data = json.loads(res.read())
    print(f"Egress IP: {data.get('ip')}")

πŸ“‘ REST API Reference

Method Endpoint Description
GET /studio Apple Dark Minimalist Web Dashboard
GET /proxies List scored proxies with ?server_type=physical|virtual filter
GET /nodes Inspect container status and active profile mapping
POST /nodes/{id}/rotate Rotate node profile with optional ?server_type= constraint
POST /nodes/{id}/restart Restart node Docker container
POST /scan Trigger full on-demand egress rescan

Example: Query Filtered Physical Proxies (curl)

curl -s "https://proxy-studio.indrayuda.my.id/proxies?server_type=physical" | jq .

πŸ§ͺ Testing & Verification Suite

The repository contains an exhaustive 60-test verification suite covering catalog discovery, rotation constraints, database persistence, REST endpoints, and scoring algorithms.

pytest -v

Test Suite Execution Summary

============================= test session starts ==============================
platform linux -- Python 3.11.15, pytest-9.1.1, pluggy-1.6.0
rootdir: /root/.openclaw/workspace/projects/surfshark-proxy-studio
configfile: pyproject.toml
collected 60 items

tests/test_api.py ...........                                            [ 18%]
tests/test_cli.py ....                                                   [ 25%]
tests/test_config.py ..                                                  [ 28%]
tests/test_db.py ....                                                    [ 35%]
tests/test_docker_node.py ......                                         [ 45%]
tests/test_nodes.py ..                                                   [ 48%]
tests/test_probes.py .......                                             [ 60%]
tests/test_profile_rotation.py ...                                       [ 65%]
tests/test_qa_verification_prism.py ...                                  [ 70%]
tests/test_query.py ...                                                  [ 75%]
tests/test_scanner.py ...                                                [ 80%]
tests/test_scoring.py ........                                           [ 93%]
tests/test_server_type_filter.py ....                                    [100%]

============================= 60 passed in 14.64s ==============================

πŸ›‘οΈ Security & Privacy Notice

This tool is designed for legitimate research, automation, and privacy enhancement. Always ensure compliance with target services terms of service and relevant regulations. All credentials and OpenVPN configs are isolated within local secrets files and never exposed in public repositories.


πŸ“„ License

This project is licensed under the MIT License. See the LICENSE file for details.


Crafted with precision by Indra Yuda (@IndraYuda13)

About

Distributed egress proxy manager with physical vs. virtual node routing and live dashboard.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages