docs(profiler): stage 0 design contracts for backend redesign#708
Draft
vlsi wants to merge 2 commits into
Draft
Conversation
Plans the move from collector→Postgres→maintenance→parquet to a single
Go binary that demultiplexes agent streams, persists them via local
WAL + trace cache, ships parquet to S3, and serves queries via fan-out
+ merge across collector replicas plus S3.
Adds:
- profiler-plan.md (root): roadmap, stages 0–6, decisions log,
code map.
- backend/docs/design/README.md: index of contract docs and a
summary of decisions.
- backend/docs/design/stage0-progress.md: stage 0 tracker
(markdown checklist) and append-only decisions log.
- backend/docs/design/01-write-contract.md: first draft —
dictionary WAL, trace stream cache, parquet schema, local PV
layout, S3 layout, flush semantics, configuration.
Stage 1 implementation is blocked on review of items V1–V6 in
01-write-contract.md, which describe agent-side wire behavior the
author has not yet confirmed in agent/, dumper/, runtime/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…the write contract - Translates the work plan to English to match the project policy (CLAUDE.md: "Use English for newly created documents"). - Aligns Stage 1 with the write contract: the agent protocol parser reused from `backend/libs/parser/parser.go` (Go), not from the Java collector. Raw trace bytes are persisted via a trace stream cache on local PV, replacing the earlier "spill-to-SQLite" sketch (the collector never assembles partial calls — see 01-write-contract.md Section 1). - Decision #2 in the "Decisions taken" section now reflects the agreed outcome: the new collector is written in Go and the Java collector is removed after Stage 4. - Appendix A points to both the Java collector (legacy) and the Go parser (future) for the TCP-stream-receiver row. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
profiler-plan.md: six stages, with stage 0 (contracts and diagrams) as the gate. Captures decisions from the team discussion and follow-up Q&A: no Postgres in prod, single Go binary, RWX-PV not required, contracts-first development.backend/docs/design/01-write-contract.md: what the new Go collector writes — dictionary WAL, trace stream cache, parquet schema, local PV layout, S3 layout, flush semantics, configuration.backend/docs/design/stage0-progress.md: checklist plus append-only decisions log for the contract phase.backend/docs/design/README.md: pointer to the five contract docs (only Add initial source code #1 is drafted; Add issues and PR templates #2–Fix links in README.md #5 are TBD).Why draft
Stage 1 cannot start until items V1–V6 in
01-write-contract.mdare confirmed against the agent code (agent/,dumper/,runtime/). Posting as draft to collect that input. Three more design docs (read contract, lifecycle, storage layout) and the Mermaid diagrams are still pending.Open decisions called out in the docs
trace_idcolumn shape in the parquet schema: single string vs three integers (01-write-contract.mdSection 5.3 Q-A).01-write-contract.mdSection 9).Test plan
🤖 Generated with Claude Code