Skip to content
Merged
11 changes: 11 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,17 @@ UNBROWSE_API_KEY=your-unbrowse-api-key
# BIND_HOST=0.0.0.0
# PORT=2026

# Proxies whose X-Real-IP the Gateway trusts when it counts failed logins per
# client (comma-separated IPs, CIDRs or hostnames; a hostname is re-resolved
# every 10s). The Docker stack defaults to its bundled `nginx` service; without
# it every browser login shares nginx's address, so five wrong passwords from
# anyone lock everyone out. Behind your own front door, nginx sees that proxy's
# address instead: configure nginx's real_ip module for it as well.
# Outside Docker, unset or empty is direct mode: X-Real-IP is ignored. In the
# Docker stack an empty value also falls back to `nginx`, since every browser
# request arrives through it.
# AUTH_TRUSTED_PROXIES=nginx

# Optional:
# FIRECRAWL_API_KEY=your-firecrawl-api-key
# VOLCENGINE_API_KEY=your-volcengine-api-key
Expand Down
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -804,6 +804,20 @@ This release closes that milestone with **439 merged pull requests**.

### Fixed

- **auth:** In the Docker stack, five wrong passwords from one client no longer
lock every user out of login for five minutes. Failed logins are counted per
client IP, and the Gateway honors `X-Real-IP` only from a peer listed in
`AUTH_TRUSTED_PROXIES`, which the compose files never set; every browser
request reaches the Gateway from the `nginx` container, so all logins shared
nginx's address and one lockout. `AUTH_TRUSTED_PROXIES` now also accepts
hostnames, resolved off the event loop and cached for 10 seconds (failures
included), and both compose files default it to the bundled `nginx` service.
nginx overwrites `X-Real-IP` with `$remote_addr` on every Gateway route, so
a client cannot choose its own address. An `AUTH_TRUSTED_PROXIES` value in
`.env` still takes precedence, including under `make docker-start`, which now
exports it for Compose interpolation like the proxy variables. Deployments
behind another reverse proxy also need nginx's `real_ip` module for that
proxy; see `.env.example`. ([#6519])
- **scheduler:** "Run once now" on a one-time task before its run time no longer
cancels the scheduled run. The trial launched as the task's own run: the task
was marked `running`, and the trial's outcome then finished it (`completed`,
Expand Down Expand Up @@ -9296,3 +9310,4 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
[#6501]: https://github.com/bytedance/deer-flow/pull/6501
[#6506]: https://github.com/bytedance/deer-flow/pull/6506
[#6512]: https://github.com/bytedance/deer-flow/pull/6512
[#6519]: https://github.com/bytedance/deer-flow/pull/6519
7 changes: 7 additions & 0 deletions CHANGELOG_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -694,6 +694,12 @@

### 修复

- **认证:** 在 Docker 部署中,某个客户端输错 5 次密码不会再让所有用户 5 分钟内无法登录。登录失败按客户端 IP 计数,而 Gateway
只在 TCP peer 属于 `AUTH_TRUSTED_PROXIES` 时信任 `X-Real-IP`,compose 文件却从未设置该变量;所有浏览器请求都经由 `nginx`
容器到达 Gateway,因此所有登录共用 nginx 的地址和同一个锁定。现在 `AUTH_TRUSTED_PROXIES` 也接受主机名,在事件循环外解析并
缓存 10 秒(包括解析失败),两个 compose 文件默认将其设为内置的 `nginx` 服务。nginx 在每个转发到 Gateway 的路由上都用
`$remote_addr` 覆盖 `X-Real-IP`,客户端无法自选地址。`.env` 中设置的 `AUTH_TRUSTED_PROXIES` 仍然优先,`make docker-start` 也是如此:它现在会像代理变量一样导出该值供 Compose 插值。位于其他反向代理之后的
部署还需要为该代理配置 nginx 的 `real_ip` 模块,详见 `.env.example`。([#6519])
- **调度器:** 在一次性任务的执行时间之前点击“立即运行一次”,不会再取消原定的执行。此前这次试运行被当作任务本身的
执行:任务被标记为 `running`,试运行结束后又按其结果把任务终结为 `completed`、`failed` 或 `cancelled`。轮询器只认领
`enabled` 的任务,因此到了 `run_at` 也不会执行,尽管 `next_run_at` 仍显示该时间。现在在执行时间之前启动的试运行
Expand Down Expand Up @@ -7544,3 +7550,4 @@ DeerFlow 2.0 是围绕"超级智能体"框架的彻底重写,核心包含子
[#6484]: https://github.com/bytedance/deer-flow/pull/6484
[#6506]: https://github.com/bytedance/deer-flow/pull/6506
[#6512]: https://github.com/bytedance/deer-flow/pull/6512
[#6519]: https://github.com/bytedance/deer-flow/pull/6519
126 changes: 102 additions & 24 deletions backend/app/gateway/routers/auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import logging
import os
import re
import socket
import time
import urllib.parse
from ipaddress import ip_address, ip_network
Expand Down Expand Up @@ -222,30 +223,107 @@ def _login_throttle_policy() -> tuple[int, float]:
return local.max_login_attempts, local.lockout_seconds


def _trusted_proxies() -> list:
"""Parse ``AUTH_TRUSTED_PROXIES`` env var into a list of ip_network objects.

Comma-separated CIDR or single-IP entries. Empty / unset = no proxy is
trusted (direct mode). Invalid entries are skipped with a logger warning.
Read live so env-var overrides take effect immediately and tests can
``monkeypatch.setenv`` without poking a module-level cache.
# Hostname entries (e.g. the compose service ``nginx``) are re-resolved at most
# this often: a restarted proxy container comes back on a new address, and
# nginx's own resolver uses the same validity (docker/nginx/nginx.conf).
_TRUSTED_PROXY_HOST_TTL_SECONDS = 10.0
_HOSTNAME_LABEL = re.compile(r"[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?")
# hostname -> (monotonic expiry, resolved addresses). Failed lookups are cached
# as an empty set too, so unauthenticated requests cannot drive a lookup storm.
_trusted_proxy_host_cache: dict[str, tuple[float, frozenset]] = {}
# hostname -> the lookup in progress. Concurrent cache misses (a burst of
# setup-status / login requests when the entry expires) join it, so a refresh
# costs one resolver call however many requests arrive while DNS is slow.
_trusted_proxy_host_inflight: dict[str, asyncio.Task[frozenset]] = {}


def _is_hostname(entry: str) -> bool:
"""RFC 1123 hostname whose last label is not numeric (so ``10.0.0`` is not one)."""
labels = entry.split(".")
return len(entry) <= 253 and all(_HOSTNAME_LABEL.fullmatch(label) for label in labels) and not labels[-1].isdigit()


def _trusted_proxies() -> tuple[list, list[str]]:
"""Parse ``AUTH_TRUSTED_PROXIES`` into (ip_network objects, hostnames).

Comma-separated CIDR, single-IP or hostname entries. Empty / unset = no
proxy is trusted (direct mode). A hostname stands for every address it
resolves to, so a proxy without a fixed address (the compose ``nginx``
service) can still be named. Invalid entries are skipped with a logger
warning. The variable is read live so env-var overrides take effect
immediately and tests can ``monkeypatch.setenv``; only the addresses a
hostname resolves to are cached (``_trusted_proxy_host_addresses``).
"""
raw = os.getenv("AUTH_TRUSTED_PROXIES", "").strip()
if not raw:
return []
return [], []
nets = []
hostnames = []
for entry in raw.split(","):
entry = entry.strip()
if not entry:
continue
try:
nets.append(ip_network(entry, strict=False))
except ValueError:
logger.warning("AUTH_TRUSTED_PROXIES: ignoring invalid entry %r", entry)
return nets
if _is_hostname(entry):
hostnames.append(entry)
else:
logger.warning("AUTH_TRUSTED_PROXIES: ignoring invalid entry %r", entry)
return nets, hostnames


async def _resolve_trusted_proxy_host(hostname: str) -> frozenset:
"""Resolve ``hostname`` in the loop's executor and cache the result (or the failure)."""
try:
infos = await asyncio.get_running_loop().getaddrinfo(hostname, None, type=socket.SOCK_STREAM)
addresses = frozenset(ip_address(info[4][0]) for info in infos)
except (OSError, ValueError) as exc:
logger.warning("AUTH_TRUSTED_PROXIES: could not resolve %r (%s); not trusting it for %.0fs", hostname, exc, _TRUSTED_PROXY_HOST_TTL_SECONDS)
addresses = frozenset()
_trusted_proxy_host_cache[hostname] = (time.monotonic() + _TRUSTED_PROXY_HOST_TTL_SECONDS, addresses)
return addresses


async def _trusted_proxy_host_addresses(hostname: str) -> frozenset:
"""Addresses ``hostname`` resolves to, cached for ``_TRUSTED_PROXY_HOST_TTL_SECONDS``.

The lookup runs in the loop's executor (``getaddrinfo`` blocks). A failed
lookup trusts nothing until the entry expires, which is the direct-mode
behavior, not an error for the request. One lookup per hostname runs at a
time: the cache check and the in-flight registration happen without a
yield in between, and later callers join that task. Callers wait through
``asyncio.shield``, so one cancelled request (a client disconnect) cannot
cancel the lookup the others joined; the task caches its own result, so
it is kept even if every caller went away.
"""
cached = _trusted_proxy_host_cache.get(hostname)
if cached is not None and time.monotonic() < cached[0]:
return cached[1]
task = _trusted_proxy_host_inflight.get(hostname)
if task is None:
task = asyncio.get_running_loop().create_task(_resolve_trusted_proxy_host(hostname))
_trusted_proxy_host_inflight[hostname] = task

def _release(done: asyncio.Task[frozenset]) -> None:
if _trusted_proxy_host_inflight.get(hostname) is done:
del _trusted_proxy_host_inflight[hostname]

task.add_done_callback(_release)
return await asyncio.shield(task)


async def _is_trusted_proxy(peer_ip) -> bool:
nets, hostnames = _trusted_proxies()
if any(peer_ip in net for net in nets):
return True
for hostname in hostnames:
if peer_ip in await _trusted_proxy_host_addresses(hostname):
return True
return False


def _get_client_ip(request: Request) -> str:
async def _get_client_ip(request: Request) -> str:
"""Extract the real client IP for rate limiting.

Trust model:
Expand All @@ -255,30 +333,30 @@ def _get_client_ip(request: Request) -> str:
by the client itself.
- ``X-Real-IP`` is **only** honored if the TCP peer is in the
``AUTH_TRUSTED_PROXIES`` allowlist (set via env var, comma-separated
CIDR or single IPs). When set, the gateway is assumed to be behind a
reverse proxy (nginx, Cloudflare, ALB, …) that overwrites
``X-Real-IP`` with the original client address.
CIDR, single IPs or hostnames). When set, the gateway is assumed to be
behind a reverse proxy (nginx, Cloudflare, ALB, …) that overwrites
``X-Real-IP`` with the original client address. The compose files
name their bundled ``nginx`` service here by default.
- With no ``AUTH_TRUSTED_PROXIES`` set, ``X-Real-IP`` is silently
ignored — closing the bypass where any client could rotate the
header to dodge per-IP rate limits in dev / direct-gateway mode.

``X-Forwarded-For`` is intentionally NOT used because it is naturally
client-controlled at the *first* hop and the trust chain is harder to
audit per-request.
audit per-request. A request without ``X-Real-IP`` never resolves a
hostname entry.
"""
peer_host = request.client.host if request.client else None
real_ip = request.headers.get("x-real-ip", "").strip()

trusted = _trusted_proxies()
if trusted and peer_host:
if peer_host and real_ip:
try:
peer_ip = ip_address(peer_host)
if any(peer_ip in net for net in trusted):
real_ip = request.headers.get("x-real-ip", "").strip()
if real_ip:
return real_ip
except ValueError:
# peer_host wasn't a parseable IP (e.g. "unknown") — fall through
pass
peer_ip = None
if peer_ip is not None and await _is_trusted_proxy(peer_ip):
return real_ip

return peer_host or "unknown"

Expand Down Expand Up @@ -358,7 +436,7 @@ async def login_local(
remember_me: bool = Form(default=True),
):
"""Local email/password login."""
client_ip = _get_client_ip(request)
client_ip = await _get_client_ip(request)
await _check_rate_limit(client_ip)

user = await get_local_provider().authenticate({"email": form_data.username, "password": form_data.password})
Expand Down Expand Up @@ -672,7 +750,7 @@ async def revoke_pat(request: Request, pat_id: str):
@router.get("/setup-status")
async def setup_status(request: Request):
"""Check if an admin account exists. Returns needs_setup=True when no admin exists."""
client_ip = _get_client_ip(request)
client_ip = await _get_client_ip(request)
now = time.time()

# Return cached result when within TTL — avoids 429 on multi-tab reconnection.
Expand Down
2 changes: 1 addition & 1 deletion backend/docs/AUTH_DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ enum UserScope:
- 成功后签发 JWT,放入 `access_token` HttpOnly cookie。
- 响应体只返回 `expires_in` 和 `needs_setup`,不返回 token。

登录失败会按客户端 IP 计数。IP 解析只在 TCP peer 属于 `AUTH_TRUSTED_PROXIES` 时信任 `X-Real-IP`,不使用 `X-Forwarded-For`。阈值与锁定时长可通过 `auth.local.max_login_attempts`(默认 5)和 `auth.local.lockout_seconds`(默认 300 秒)配置,按次实时读取,改配置后下一次登录即生效,无需重启 Gateway(`max_login_attempts` 最小为 2:单次失败不得锁定 IP。时长热改按方向生效:下调可提前释放进行中的锁定、收紧阈值会保留已计数的失败;上调只延长仍在锁定期内的锁定,不会复活已服满原时长的锁定)。计数器的存放位置由 `auth.local.throttle_storage` 决定(启动时解析一次,改动需重启):默认 `auto` 在 `database.backend` 为 `sqlite` / `postgres` 时使用应用数据库中的 `login_throttle` 表,所有共享该数据库的 Gateway 副本对同一 IP 执行同一份限制;`memory` 退回进程内计数器(N 个副本意味着攻击者有 N × `max_login_attempts` 次机会,且一个副本上的锁定对其他副本不可见,多实例部署会在启动时打 WARNING);`db` 强制使用共享表,数据库为 `memory` 时降级为进程内计数器并打 WARNING,已配置的数据库引擎不可用时则拒绝启动(`auto` 在该情况下降级并打 WARNING)。锁定期内的再次失败不会重置 `locked_at`:锁定时长从锁定开始计算,而不是从最后一次尝试计算。
登录失败会按客户端 IP 计数。IP 解析只在 TCP peer 属于 `AUTH_TRUSTED_PROXIES` 时信任 `X-Real-IP`,不使用 `X-Forwarded-For`。`AUTH_TRUSTED_PROXIES` 接受逗号分隔的 IP、CIDR 或主机名;主机名在事件循环外解析,结果(包括解析失败)缓存 10 秒,因此重启后换了地址的代理容器仍能被识别。两个 compose 文件默认把它设为内置的 `nginx` 服务:否则所有浏览器登录都以 nginx 的地址计数,任何人输错 5 次密码就会锁住所有用户。nginx 在每个转发到 Gateway 的 location 中都用 `$remote_addr` 覆盖 `X-Real-IP`,这是信任它的前提。阈值与锁定时长可通过 `auth.local.max_login_attempts`(默认 5)和 `auth.local.lockout_seconds`(默认 300 秒)配置,按次实时读取,改配置后下一次登录即生效,无需重启 Gateway(`max_login_attempts` 最小为 2:单次失败不得锁定 IP。时长热改按方向生效:下调可提前释放进行中的锁定、收紧阈值会保留已计数的失败;上调只延长仍在锁定期内的锁定,不会复活已服满原时长的锁定)。计数器的存放位置由 `auth.local.throttle_storage` 决定(启动时解析一次,改动需重启):默认 `auto` 在 `database.backend` 为 `sqlite` / `postgres` 时使用应用数据库中的 `login_throttle` 表,所有共享该数据库的 Gateway 副本对同一 IP 执行同一份限制;`memory` 退回进程内计数器(N 个副本意味着攻击者有 N × `max_login_attempts` 次机会,且一个副本上的锁定对其他副本不可见,多实例部署会在启动时打 WARNING);`db` 强制使用共享表,数据库为 `memory` 时降级为进程内计数器并打 WARNING,已配置的数据库引擎不可用时则拒绝启动(`auto` 在该情况下降级并打 WARNING)。锁定期内的再次失败不会重置 `locked_at`:锁定时长从锁定开始计算,而不是从最后一次尝试计算。

### 注册

Expand Down
28 changes: 28 additions & 0 deletions backend/tests/blocking_io/test_login_local_throttle.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,3 +150,31 @@ async def test_store_reset_and_probe_do_not_block_loop(store) -> None:

assert await store.get(_CLIENT_IP) is None
await store.reset(_CLIENT_IP) # idempotent on a clean IP


async def test_trusted_proxy_hostname_resolution_does_not_block_loop(store, monkeypatch) -> None:
"""A hostname in AUTH_TRUSTED_PROXIES (the compose default ``nginx``) is
resolved on the login path; ``getaddrinfo`` must run off the event loop."""
monkeypatch.setenv("AUTH_TRUSTED_PROXIES", "localhost")
auth_router._trusted_proxy_host_cache.clear()
request = Request(
{
"type": "http",
"method": "POST",
"path": "/api/v1/auth/login/local",
"headers": [(b"x-real-ip", _CLIENT_IP.encode())],
"query_string": b"",
"client": ("127.0.0.1", 44000),
"server": ("testserver", 80),
}
)
for _ in range(_POLICY["max_attempts"]):
await store.record_failure(_CLIENT_IP, **_POLICY) # the forwarded client is locked
try:
with pytest.raises(HTTPException) as exc_info:
await auth_router.login_local(request, Response(), _form(), remember_me=True)
finally:
auth_router._trusted_proxy_host_cache.clear()

# 429 proves the lockout was looked up under X-Real-IP, i.e. "localhost" resolved.
assert exc_info.value.status_code == 429
Loading
Loading