Skip to content

feat(live): show every replica's traffic in the live log view - #1341

Draft
SantiagoDePolonia wants to merge 3 commits into
mainfrom
feat/live-logs-across-replicas
Draft

SantiagoDePolonia wants to merge 3 commits into
mainfrom
feat/live-logs-across-replicas

Conversation

@SantiagoDePolonia

@SantiagoDePolonia SantiagoDePolonia commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Description

  • The dashboard's live log view now shows traffic from every replica, not only the one the browser is connected to. Before, it showed roughly 1/N of the traffic.
  • It adds no database writes of its own. Each audit and usage batch the loggers already write is stamped with the process's writer id and write time. That is two new nullable columns (writer_id, written_at) on audit_logs and usage, or two fields on MongoDB, and a partial index on written_at. PostgreSQL builds the index CONCURRENTLY in the background.
  • While at least one dashboard stream is connected, each replica's live broker reads, every 2 seconds, the rows other writers stamped in the last 10 seconds. It publishes the ones it hasn't published yet. With no subscriber it runs no query. The window is measured on the reader's own clock, and published ids are remembered for that window. That makes every remote entry appear exactly once, even across overlapping polls and batches that become visible late.
  • Delay: a remote request appears after its replica flushes it (LOGGING_FLUSH_INTERVAL / USAGE_FLUSH_INTERVAL, 5s by default, or sooner at 100 buffered entries) plus up to one 2s poll. That is about 7s at most.
  • Remote events are audit.flushed and usage.flushed, the persisted, completed entry with status, duration, model, provider, attempts, usage and cost. There are no in-flight, audit.started/updated or per-chunk stream events. Bodies are left out of the event and load from the database when the entry is expanded. At most 200 audit and 200 usage entries are streamed per poll, newest first.
  • Local events are unchanged and stay instant. Rows this process wrote are excluded by writer id, so they are never duplicated.
  • Dashboard: a live entry that is new to a list is now placed by request timestamp instead of always on top. Late remote rows land below newer local ones, and a late request that is older than its thread head joins the thread's children.
  • The single-instance hot path is unchanged. The stamp is one pointer per entry, set once per batch at flush. The perf guard numbers match main.

Tests: new store tests for RecentWrites / EntriesByID / LogEntriesByID, the schema migration and the stamping, run on SQLite, PostgreSQL and MongoDB. Feed tests cover polling only while subscribed, exactly-once publishing, late-visible batches, the start window, idle and restart, retry, and the per-poll cap. A two-replica test on one store runs on all three backends. Dashboard ordering tests were added. make test-race passed with PostgreSQL and MongoDB, and the perf guard, lint and npm test / npm run check passed.

View guided diff Turn on auto-fix

@mintlify

mintlify Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
gomodel 🟢 Ready View Preview Oct 30, 2026, 2:50 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@coderabbitai

coderabbitai Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the size:XXL 1,000+ effective changed lines (test files excluded in mixed PRs). label Oct 30, 2026
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 76.28362% with 97 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/usage/background_index_postgresql.go 46.93% 26 Missing ⚠️
internal/auditlog/recent_writes.go 77.22% 23 Missing ⚠️
internal/usage/recent_writes.go 76.05% 17 Missing ⚠️
internal/app/app.go 0.00% 14 Missing ⚠️
internal/live/remote.go 91.96% 9 Missing ⚠️
internal/storage/writer.go 0.00% 4 Missing ⚠️
internal/live/broker.go 93.75% 2 Missing ⚠️
internal/app/init_server.go 0.00% 1 Missing ⚠️
internal/usage/factory.go 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

This branch was successfully deployed

1 active deployment
staging - docs — 2964b07e Deployed Oct 30, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ effective changed lines (test files excluded in mixed PRs).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants