Skip to content

feat(tiers): add DuckDuckGo anomaly challenge detection and escalation - #120

Open
nklowns wants to merge 1 commit into
germondai:devfrom
nklowns:feat/duckduckgo-anomaly-detection
Open

feat(tiers): add DuckDuckGo anomaly challenge detection and escalation#120
nklowns wants to merge 1 commit into
germondai:devfrom
nklowns:feat/duckduckgo-anomaly-detection

Conversation

@nklowns

@nklowns nklowns commented Sep 7, 2026

Copy link
Copy Markdown

Summary

Adds conservative DuckDuckGo anomaly challenge detection (//duckduckgo.com/anomaly.js, anomaly-modal) to Tier 1 and the MITM proxy, escalating requests to browser tiers rather than misclassifying the challenge page as successful content.

Linked issues

Fixes #119

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing behavior to change)
  • Documentation only

Checklist

  • I opened or commented on an issue before starting this PR (non-trivial changes only)
  • I ran bun run check and it passes locally
  • I added or updated tests for the behavior change (if applicable)
  • I updated CHANGELOG.md under ## [Unreleased] for user-visible changes
  • I read CONTRIBUTING.md and CODE_OF_CONDUCT.md

AGPL notice

By submitting this PR, I confirm my contribution is my own work and I agree to license it under AGPL-3.0.

- Detect DuckDuckGo anomaly challenge markers (anomaly.js, anomaly-modal) in packages/tiers/src/utils/detect.ts
- Distinguish DuckDuckGo challenges from Cloudflare in isCloudflarePage
- Escalate Tier 1 requests to browser tiers when anomaly challenge is detected
- Detect DuckDuckGo anomaly challenges in proxy directForward to trigger scrape escalation
- Add test coverage for detection, negative cases, Cloudflare priority, and proxy handling
- Update CHANGELOG.md under Unreleased

Fixes germondai#119
@nklowns
nklowns requested a review from germondai as a code owner September 7, 2026 20:20
@nklowns

nklowns commented Sep 7, 2026

Copy link
Copy Markdown
Author

Verification & Implementation Notes

  • Offline test coverage: 100% hermetic unit tests added in packages/tiers/tests/duckduckgoDetection.test.ts and apps/api/src/proxy/__tests__/directForward.test.ts. Covers challenge detection, negative cases (standard DuckDuckGo SERP), Cloudflare priority precedence, and proxy direct forward escalation.
  • Verification Gate: Verified locally with bun run verify (lint, formatting, typecheck across all 5 workspaces, tests, and web/docs production builds pass).
  • Target & Isolation: Clean commit rebased onto upstream dev with 0 conflicts, strictly scoped to DuckDuckGo anomaly challenge detection and tier escalation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat] Detect DuckDuckGo anomaly challenge and escalate to browser tier

1 participant