Skip to content

Releases: Edgaras0x4E/web-loader-engine

v0.1.4

24 Apr 09:54

Choose a tag to compare

User Agent Rotation & Browser Log Control - Added configurable user agents and a way to silence warnings.

  • New USER_AGENT_ROTATION env var with strategies off (default), round_robin, random - rotates per request
  • Provide the pool via USER_AGENT_POOL (inline, |- or newline-separated) or USER_AGENT_POOL_FILE (path to a file, one UA per line, # comments supported). The file takes precedence when both are set
  • DEFAULT_USER_AGENT overrides the hardcoded default used when rotation is off and no header is set
  • x-user-agent header now accepts special values: rotate forces rotation even when USER_AGENT_ROTATION=off, and default forces the configured default
  • Precedence: explicit header → rotation (if enabled) → configured default. Empty pool safely falls back to the default with a warning at startup
  • New BROWSER_LOG_LEVEL env var (default error) silences chromiumoxide's noisy WS Invalid message warnings emitted when Chromium sends CDP events the driver doesn't yet model. Accepts off, error, warn, info, debug, trace - operates independently of RUST_LOG

v0.1.3

22 Apr 09:34

Choose a tag to compare

Chromium Egress Proxy Support - Chromium now honors HTTPS_PROXY/HTTP_PROXY/NO_PROXY from the environment so the browser's outbound traffic can be routed through an egress proxy.

  • On launch, if HTTPS_PROXY (or HTTP_PROXY as fallback) is set, Chromium is started with --proxy-server=<url>
  • If NO_PROXY is set, its value is translated to Chrome's bypass-list syntax and passed via --proxy-bypass-list=<list> (commas → semicolons, *.domain.domain)
  • When no proxy env vars are set, behavior is unchanged — dev/local runs need no configuration
  • The Rust HTTP client (reqwest) already honors these vars natively, so direct HTTP fetches and browser fetches now share the same egress path

v0.1.2

19 Apr 06:31

Choose a tag to compare

Screenshot Delivery Fix - Screenshot URLs returned by the API are now actually reachable.

  • Fixed issue where /load responses advertised a screenshot_url that returned 404 when fetched
  • Saved screenshots are now served directly from the configured SCREENSHOT_DIR
  • Safe by design: path-traversal attempts (e.g. /screenshots/../etc/passwd) return 404
  • Respects the same API key authentication as the rest of the API when one is configured

v0.1.1

03 Feb 11:04

Choose a tag to compare

Browser Pool Resilience - Fixed critical issue where dead browser connections would cause requests to hang indefinitely.

  • Added automatic browser health detection with 5-second timeout on page creation
  • Implemented connection error detection for Ws(AlreadyClosed) and related WebSocket errors
  • Auto-recovery: dead browsers are now automatically recreated on connection failure
  • Request-level retry logic (up to 3 retries) for transient connection errors
  • Health endpoint now exposes healthy status and recreation_count for monitoring

Health response now includes:

{
  "status": "ok",
  "version": "0.1.1",
  "browser_pool": {
    "available": 10,
    "total": 10,
    "healthy": true,
    "recreation_count": 1
  }
}

Monitor recreation_count increasing to track browser recovery events.