You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Usage-stats job loads connections without their password, so every run fails ClickHouse auth #3612
Every 4 hours the API logs one AUTHENTICATION_FAILED per saved connection that has a password, and ClickHouse logs a matching WRONG_PASSWORD for the connection's user from the HyperDX host:
[ERROR][@clickhouse/client][Connection] Query: HTTP request error.
...
Caused by: ClickHouseError: <user>: Authentication failed: password is incorrect, or there is no user with such name.
code: '516',
type: 'AUTHENTICATION_FAILED'
The same connections work everywhere else in HyperDX (search, dashboards, alerts), with the same stored password.
Cause: getClickhouseTableSize() in packages/api/src/tasks/usageStats.ts loads connections with
constconnections=awaitConnection.find();
but the password field in packages/api/src/models/connection.ts is declared with select: false, so connection.password is undefined when the job builds its ClickhouseClient. The query on system.parts is sent with an empty password and rejected.
Expected: the job reads the password, as other code paths do via getConnectionById(..., selectPassword = true), e.g.
(The usage stats the job reports are presumably empty or incomplete as a result, since the failure is swallowed by its catch.)
Steps to reproduce
Reproduces on play.hyperdx.io? didn't try (it needs a password-protected ClickHouse user and access to the API logs)
Run HyperDX with USAGE_STATS_ENABLED unset (it defaults to on).
Add a connection whose ClickHouse user has a non-empty password, and at least one source on it.
Restart the API (the job runs at startup, then every 4h) and watch the API logs, or ClickHouse's system.text_log, for the auth failure.
How are you running HyperDX?
HyperDX-only image docker.hyperdx.io/hyperdx/hyperdx:2.40.0 under podman compose, with its own MongoDB and a self-managed ClickHouse 26.9. The code is unchanged on main as of commit 631002374 (2026-11-10).
Where does it show up?
API (background task tasks/usageStats).
Debug info
N/A (API background task).
Logs
See above. Workaround: set USAGE_STATS_ENABLED=false.
Activity