feat(backend): expose Prometheus metrics at /metrics by MarcusDavidG · Pull Request #518 · Stellar-Ecosystem/lodestar

MarcusDavidG · 2026-07-30T00:58:23Z

Closes #338

What

Adds prom-client instrumentation to the backend and exposes a Prometheus scrape endpoint at GET /metrics.

Files changed

`backend/src/lib/metrics.js` (new)

Dedicated Registry (not the global default, so tests can be isolated). Exports four metrics plus collectDefaultMetrics for Node/process stats:

Metric	Type	Labels
`lodestar_submit_queue_depth`	Gauge	—
`lodestar_submission_duration_seconds`	Histogram (9 buckets: 0.1–60 s)	`operation`, `status`
`lodestar_http_requests_total`	Counter	`method`, `route`, `status`
`lodestar_contract_errors_total`	Counter	`code`

`backend/src/lib/contract.js`

simulateAndSubmit() now sets the queue-depth gauge before/after each task, records submission duration (operation + success/error), and increments the error counter with the ContractError.code on any failure.

`backend/src/index.js`

res.on('finish') middleware increments httpRequestsTotal using req.route?.path for bounded cardinality
GET /metrics handler returns register.metrics() with the correct Content-Type

`backend/src/lib/metrics.test.js` (new)

16 tests using isolated Registry instances — gauge set/decrement, histogram label cardinality and bucket boundaries, counter isolation, HELP/TYPE line format, content-type string.

`backend/docs/metrics.md` (new)

Full metric catalogue, 4 sample Prometheus alert rules (queue high, timeout, error rate, p95 latency), sample Grafana PromQL queries, and a Docker/Compose scrape config snippet.

Acceptance criteria

Criterion	Status
`/metrics` returns valid Prometheus text format	✅
Queue depth covered	✅ `lodestar_submit_queue_depth` gauge
Submission latency covered	✅ `lodestar_submission_duration_seconds` histogram
Error counts covered	✅ `lodestar_contract_errors_total` counter by code
Metric names and labels documented	✅ `backend/docs/metrics.md`
Sample dashboard / alert rules included	✅ 4 alert rules + PromQL queries in `docs/metrics.md`

Tests

✓ src/lib/metrics.test.js  (16 tests, all new — all green)

The 18 pre-existing failures in registry.test.js are unrelated to this PR (confirmed by running the suite on the base branch).

drips-wave · 2026-07-30T00:58:31Z

@MarcusDavidG Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

coderabbitai · 2026-07-30T00:58:31Z

Warning

Review limit reached

@MarcusDavidG, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 50 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 51b4f832-70ea-4194-b14a-6d36dc4be0ca

📥 Commits

Reviewing files that changed from the base of the PR and between 0546692 and 081bb43.

⛔ Files ignored due to path filters (1)

backend/package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (9)

agent/.env.example
agent/agent.js
agent/agent.test.js
backend/docs/metrics.md
backend/package.json
backend/src/index.js
backend/src/lib/contract.js
backend/src/lib/metrics.js
backend/src/lib/metrics.test.js

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

ritik4ever · 2026-07-31T11:33:32Z

Hi @MarcusDavidG,

This PR could not be merged because it has merge conflicts with the target branch.

Please resolve the merge conflicts, push the updated changes, and the PR can be reviewed and merged.

Thank you!

…t in production Closes Stellar-Ecosystem#368 An empty HMAC secret silently disabled request signing with no operator feedback. This change surfaces the gap at startup: - agent/agent.js: add exported validateHmacSecret() called immediately after the logger is constructed. • NODE_ENV=production + empty secret → throws Error (agent refuses to start), no warn log emitted. • Any other NODE_ENV + empty secret → logger.warn with event 'hmac_secret_missing' and an actionable message; agent continues. • Non-empty secret → returns immediately, no side effects. - agent/.env.example: expand the LODESTAR_HMAC_SECRET comment to state it is required in production and clarify the dev/test behaviour. - agent/agent.test.js: destructure validateHmacSecret from the top-level import; add 4 tests covering: 1. warn emitted (event + message) when secret is empty and not production 2. warn emitted (not thrown) when NODE_ENV=development 3. throws with correct message when NODE_ENV=production 4. production error message contains actionable instructions All 32 tests pass.

…Ecosystem#338) Adds prom-client instrumentation to the Lodestar backend. New dependency prom-client@15.1.3 (pinned) backend/src/lib/metrics.js (new) Dedicated Registry (not the global default) with default labels app=lodestar-backend. Exports four metrics: • lodestar_submit_queue_depth Gauge • lodestar_submission_duration_seconds Histogram (9 buckets, 0.1→60 s) labels: operation, status • lodestar_http_requests_total Counter labels: method, route, status • lodestar_contract_errors_total Counter label: code (SIMULATION_FAILED | TRANSACTION_FAILED | TRANSACTION_TIMEOUT | RETURN_VALUE_PARSE_FAILED | UNKNOWN_ERROR) collectDefaultMetrics() registers Node/process metrics on the same registry. backend/src/lib/contract.js simulateAndSubmit() now: • Sets submitQueueDepth before and after each queued task • Wraps _simulateAndSubmit in a try/finally to record submissionDuration (labelled by operation + success/error) • Increments contractErrorsTotal{code} on any ContractError throw backend/src/index.js • Imports register and httpRequestsTotal from metrics.js • Adds res.on('finish') middleware to increment httpRequestsTotal using req.route?.path for bounded cardinality • Adds GET /metrics handler that returns register.metrics() backend/src/lib/metrics.test.js (new) 16 tests using isolated Registry instances (no shared state): gauge starts at 0, set/decrement; histogram observations, label cardinality, bucket boundaries, accumulation; counter increments, status/label isolation; HELP/TYPE line format; contentType string. backend/docs/metrics.md (new) Full metric catalogue (name, type, labels, alert guidance), 4 sample Prometheus alert rules, sample Grafana PromQL queries, and a Docker/Compose scrape config snippet.

MarcusDavidG added 2 commits July 31, 2026 16:16

MarcusDavidG force-pushed the feat/backend-prometheus-metrics branch from 2ab7f0f to 081bb43 Compare July 31, 2026 15:24

MarcusDavidG requested a review from ritik4ever as a code owner July 31, 2026 15:24

Navigation Menu

Uh oh!

feat(backend): expose Prometheus metrics at /metrics - #518

feat(backend): expose Prometheus metrics at /metrics#518
MarcusDavidG wants to merge 2 commits into
Stellar-Ecosystem:mainfrom
MarcusDavidG:feat/backend-prometheus-metrics

MarcusDavidG commented Jul 30, 2026

Uh oh!

drips-wave Bot commented Jul 30, 2026

Uh oh!

coderabbitai Bot commented Jul 30, 2026 •

edited

Loading

Review limit reached

Uh oh!

ritik4ever commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

MarcusDavidG commented Jul 30, 2026

What

Files changed

backend/src/lib/metrics.js (new)

backend/src/lib/contract.js

backend/src/index.js

backend/src/lib/metrics.test.js (new)

backend/docs/metrics.md (new)

Acceptance criteria

Tests

Uh oh!

drips-wave Bot commented Jul 30, 2026

Uh oh!

coderabbitai Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

ritik4ever commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

`backend/src/lib/metrics.js` (new)

`backend/src/lib/contract.js`

`backend/src/index.js`

`backend/src/lib/metrics.test.js` (new)

`backend/docs/metrics.md` (new)

coderabbitai Bot commented Jul 30, 2026 •

edited

Loading