Skip to content

feat(batch): inspect saved reports and captured evidence - #6509

Closed
Guo-Yixin wants to merge 48 commits into
bytedance:mainfrom
Guo-Yixin:feat/batch-review-extension
Closed

Guo-Yixin wants to merge 48 commits into
bytedance:mainfrom
Guo-Yixin:feat/batch-review-extension

Conversation

@Guo-Yixin

@Guo-Yixin Guo-Yixin commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Closes #6440

Relationship to #6554

These are separate, main-based PRs, split as requested in review. #6554 fixes existing browser/layout regressions and can land independently of #6509; it does not contain the batch-report feature.

For the final combined validation, my proposed sequence is:

  1. When fix(frontend): preserve side panel layout and catalog readiness #6554 is ready for acceptance, let that standalone fix land in main.
  2. I will then merge the updated main into feat(batch): inspect saved reports and captured evidence #6509, rerun affected local checks and use the official full CI for merge validation.
  3. feat(batch): inspect saved reports and captured evidence #6509 can then be considered on its own native batch-report scope, with the browser fixes already supplied by main.

I will handle the branch update and validation. This is a proposed integration sequence, not a request to bypass review or official CI; maintainer preference determines landing order and timing.

Why

Durable batches retain reports, acceptance verdicts and captured RAGFlow evidence, but the native batch panel only exposes previews and bulk export. Returning to a research conversation should let a user inspect one saved report and its original evidence without rerunning the agent.

This delivers the browser consumer proposed in #6440, following #6413 and #6416. The RFC establishes scope; it does not imply maintainer approval.

What changed

  • Batches → View items → View report opens the selected saved result in the existing default or Custom Agent conversation, on desktop and mobile. The action appears only for terminal items or items with a stored preview; pagination and retry stay in the native batch panel.

  • Reports reuse SafeStreamdown, the native Markdown link renderer, shared chat artifact images and knowledge-source dialogs. /mnt/ output links/images resolve through the current thread's artifact route. Relative image paths do not borrow the parent conversation's artifact inventory. Report HTML stays inert.

  • Execution and acceptance are separate. Missing/empty criteria show No criteria; Unverified means existing criteria lack conclusive verification, including checker errors. Checked/holds/detail outcomes are shown per criterion. Missing reports, read failures and truncation have explicit states.

  • The selected-result GET uses the existing owner-scoped batch router and threads:read admission, including with the worker stopped. It reads one immutable position and projects a bounded response off the request event loop. The HTTP/model reader share their internal item-field allowlist; the model-facing output is unchanged. Prompts, execution specifications, leases and arbitrary artifacts are excluded.

  • Only the selected result's captured evidence can satisfy citations. Saved evidence with blank names/excerpts is rejected, while live-chat citation collection retains its existing empty-field behavior. Neither providers nor the agent are rerun, and parent sources cannot fill missing saved evidence.

  • Open nonterminal results refresh until terminal. Closing or changing user/thread/item cancels obsolete reads and removes the report/source dialogs; retries invalidate cached results, and new revisions close obsolete evidence. The one-million-character report ceiling and whole-excerpt limits remain enforced.

  • The separate extension, handwritten parser, duplicate authorization path, public extension contract and version bump are withdrawn. The unrelated ChatBox panel-width fix and sidecar/mobile E2E edits are also withdrawn for separate follow-up. No contribution-specific dependency or schema changes are included; upstream dependency upgrades are retained in the merge.

Surface area

  • Frontend UI — native batch report dialog and reused report/link/image/source components

  • Backend API — selected-result read in the existing batch router

  • Agents / LangGraph — no graph, prompt or model-facing channel changes

  • Sandbox

  • Skills

  • Dependencies

  • Default behavior change — native batch report inspection

  • Docs / tests / CI only — runtime changes are included

Screenshots / Recording

Native browser captures show the conversation entry, saved report and captured source. The batch HTTP fixture uses the real SQLite repository/native router with synthetic test authentication and a stopped worker; its example chart is a synthetic browser-route fixture. File-link checks verify the target href; the synthetic chart actually loads at 80px. These captures do not verify downloading a real report.md or the production artifact-serving chain.

Native entry Saved report Captured evidence
Native batch report entry Native saved report Captured evidence

Bug fix verification

The deleted plugin's list-contained tilde fence activated a citation inside code and suppressed the following citation. Native DOM/browser regressions cover that structure and an unclosed quoted fence: code stays literal and genuine citations remain actionable on default/Custom Agent routes and desktop/mobile.

Review regression Before fix After fix
Live citation compatibility, optional acceptance and inspectable-item controls 13 failed, 49 passed 62 passed
Saved output-file link and thread switch 1 failed, 21 passed 22 passed
Saved artifact image; encoded paths, external/relative paths and thread switch 1 failed, 22 passed 23 report tests passed; 221 related report/utility tests passed

The source validation, route authorization, retry/revision cancellation and million-character plain-text regressions are included in the complete suites below. No test assertion or skip was relaxed. These deterministic checks establish application behavior; they do not claim improved model accuracy or complex-Markdown performance.

Validation

Latest conflict resolution — 2026-10-10

Published head: c6462e7ef04993153872e91730a844c65b8c60b5, incorporating official main db9b0b4b. Remote contributor updates through d9157f18 were preserved before merging upstream. The only manual conflict resolution is backend/app/gateway/routers/AGENTS.md: retain upstream MCP committed-reconciliation/drained-cleanup and custom-skill locking guidance, then retain the saved native batch report contract.

The upstream sync also carries runtime/UI changes, including goal edit guards. Independent review verified that the contribution's 30 text-file patches against the new base are unchanged, the shared image extraction retains upstream behavior, and the goal/focus guards survive the automatic merge. No browser-fix changes from #6554 were folded into this PR.

Affected-path validation after resolution Result
Windows backend native projection/HTTP/acceptance/model-reader/router and guidance regressions 97 passed; 2 PostgreSQL-gated cases skipped because this local run has no PostgreSQL URI
Frontend report/source/shared-message/API regressions 80 passed across 7 files, no skips
pnpm check Lint and types passed
Changed backend Ruff lint/format Passed; 3 files formatted
Actual PR-range agent guidance / contribution diff whitespace 0 errors, 5 soft warnings; diff check passed
Independent merge review No blocking finding; upstream and contribution contracts retained

This update used affected-path checks rather than repeating the full local suites. Earlier full-suite evidence below retains its actual SHA and is not relabeled as testing this new head. All six official workflows at this head remain action_required, awaiting maintainer approval.

Previous completed full-suite evidence

The final upstream merge includes MCP cache reconciliation, E2B filename quoting and Windows file-URI test corrections. Current-head full hosted backend validation and replay are complete and green. node2 also passes a fresh 4-case real-backend replay at integration 3b606f19, with retries disabled. The completed browser results below retain their actual tested SHAs; frontend and native batch fixture/router/projection source are unchanged by this merge. The updated integration head is 3b606f19. Its complete hosted backend, frontend, blocking-I/O, lint/types/build and replay checks also pass. The complete node2 399/5/4 browser and 2,810-unit result remains at its actual SHA c1a75104, with byte-identical frontend and native batch fixture/router/projection/service; it is not relabeled as a newly executed full default/auth browser run.

Previously validated native head: fc150be0fa1b3f64ea201643c38c545b44a8f2eb, incorporating official main 19befe76a6a7636a47c18e34be65ca01c572793d with ordinary merges. The native report implementation is unchanged by the final upstream merges. Upstream #6552 now supplies cancellation-safe cleanup; the superseded test synchronization has been removed by restoring the lifecycle test byte-for-byte to upstream. All nine inline review threads have implementation/reply evidence and are resolved. AnnaSuSu approved the earlier native implementation at 0b0e4dc0; that approval is not presented as a review of subsequent upstream merges.

The independent browser reliability changes are submitted separately in #6554, per review. The integration branch combines the two only for validation. #6509 will receive those fixes after upstream accepts them.

The personal-fork PR uses the unmodified existing workflows and the trusted official base/head range. It is CI verification rather than official upstream approval. The initial branch-creation lint failure came from checking every inherited guidance chain; the actual PR-range check passes without changing a limit, workflow, assertion or skip.

Previous complete hosted validation at fc150be0 Result
Complete backend / PostgreSQL 17 / Redis 7 27,374 passed, 105 conditional skips; default-dependency collection 27,697
Complete default / auth browser E2E 398 / 5 passed; zero failures and zero flaky
Complete frontend unit tests 2,808 passed across 293 files
Strict blocking-I/O 235 passed
Replay / backend golden 4 / 1 passed; both jobs succeeded
Guidance / lint / format / types / build All jobs succeeded
node2 at fc150be0, PostgreSQL-gated SQL/HTTP/evidence/artifact plus affected upstream paths 733 passed, 6 platform-conditional skips; 235 strict blocking-I/O passed; guidance/lint/format/lock/diff passed
Separate integration head c1a75104, including #6554 Complete node2 default/auth/real-backend replay: 399 / 5 / 4 passed, zero failed/flaky, retries disabled; 2,810 frontend passes; check/format passed

node2 uses an isolated Ubuntu 24.04 container with Actions Python 3.12.15, locked uv dependencies including PostgreSQL extras, Node 24.21.0, pnpm 10.26.2 and pinned Playwright Chromium 1217, PostgreSQL 17.11 and Redis 7.4.11. Its latest full browser validation disables retries; only the production web-server startup allowance is extended to 900 seconds for VM resources. Browser authentication and report-image serving are synthetic as described above. Real artifact-router response-byte tests are additional HTTP coverage and do not turn the browser href/image checks into a real end-to-end file download.

Earlier runs and diagnostic failures are retained in the local evidence archive; the current result table supersedes the older validation matrix. Conditional skips are reported explicitly and are not counted as passes. Official upstream Unit Tests, Frontend Unit Tests, Lint Check, Backend Blocking IO, E2E and Replay E2E still need maintainer approval to run. Labels and CLA are separate checks. Official suite success is not claimed, and neither merge nor auto-merge is enabled.

Diff against official main Added Removed
Production 586 114
Tests / fixture 1,203 0
Documentation 64 19
CI trigger paths 6 0
Code total 1,789 114
All text 1,859 133

Three PNG screenshots are additional binary files; upstream merge changes are excluded.

AI assistance

Tool(s) used: OpenAI Codex.

How you used it: Codex reproduced the issues, implemented the fixes and regression tests, ran validation, and prepared this description. A separate AI reviewer examined the changes. Human review responsibility remains with the contributor.

  • I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

@github-actions github-actions Bot added area:backend Gateway / runtime / core backend under backend/ area:ci GitHub Actions, CI config, repo tooling area:deps Dependency manifests / lockfiles area:docs Documentation and Markdown only area:frontend Next.js frontend under frontend/ needs-validation Touches front/back contract surface; needs real-path validation risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/XL PR changes 700+ lines labels Oct 8, 2026

@willem-bd willem-bd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read-only static review of pinned current revision; verified introduced findings below. No PR code or tests executed.

@Guo-Yixin

Copy link
Copy Markdown
Contributor Author

@ggnnggez @Totoro-qaq @AnnaSuSu @willem-bd Follow-up validation is complete for the published head 0b0e4dc0 (official main 5ecc1c2a is included).

The current diff completes the native batch panel, with the existing owner-scoped router and report/link/image/source renderers. The separate plugin/public contract/parser and unrelated ChatBox/sidecar/mobile changes are withdrawn. All nine inline comments have concrete fixes and replies; I rechecked the current implementation with a separate AI reviewer and have marked those addressed threads resolved. This does not imply an approving review.

Unmodified hosted fork workflows Result
Final-head full backend, PostgreSQL 17/Redis 7 27,254 passed, 105 skipped, no failures; 27,577 collected with default dependencies
Final-head frontend unit suite 2,808 passed / 293 files
Final-head blocking-I/O 235 passed
Final-head lint/guidance/type/build All jobs passed
Final-head real-backend replay/golden Both jobs passed
Complete default/auth E2E at 331e0e3c 398 / 5 passed; no failures or flaky tests

The E2E SHA differs only in backend/packages/harness/deerflow/subagents/AGENTS.md; runtime/frontend/test/lock/workflow bytes match the current head. E2E did not retrigger on the guidance-only push. No assertions, skips or workflows were relaxed. node2 additionally passes 95 native/PostgreSQL/guidance checks, 235 strict blocking tests and lint/format/lock/diff at the exact published head. The PR description retains the earlier VM browser failures and their control results separately.

The description is updated with these completed results. Official PR CI is still action_required, including Unit Tests and E2E Tests. Those runs need maintainer approval before they can provide official checks, and an approving review is still outstanding.

@AnnaSuSu AnnaSuSu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head 0b0e4dc0bfd080ae62b7e2adf572f15114f309b5 against base 5ecc1c2ac2de624af354bc515d5cb04f80df226c.

The revised implementation fits the native batch workflow: selected-result inspection reuses the existing owner-scoped router and report, link, image and citation components. Saved evidence remains isolated from surrounding conversation sources, and report/source dialogs are removed on scope changes. I found no blocking functional regressions in the reviewed scope.

Independent local validation:

  • Backend projection, real SQLite HTTP admission, persisted evidence, model-facing reader and batch service/router regressions: 137 passed, 18 skipped.
  • Frontend report/citation tests and shared renderer compatibility checks: 94 passed.
  • Native browser coverage for default/Custom Agent chats, desktop/mobile, and denied HTTP reads: 5 passed.
  • pnpm check, production build, changed-file Ruff lint/format and diff checks passed.

The PostgreSQL-gated cases were not exercised locally, and I did not rerun the complete backend suite. Browser coverage uses synthetic authentication and mocked artifact-image serving; output-file checks verify the target URL rather than a real artifact download.

The official PR workflows still await maintainer approval to run. Please complete those checks before merging.

@Guo-Yixin

Copy link
Copy Markdown
Contributor Author

@ggnnggez The browser corrections are now submitted separately in #6554, as requested. #6509's published head remains 0b0e4dc0; its native batch scope is unchanged.

The standalone frontend fix passes the unchanged hosted default/auth E2E suites (394 / 5, zero failures or flakes), frontend unit suite (2,772), lint/types/production build and real-backend replay. node2 additionally passes all 15 focused browser cases with retries disabled, including the original exact scroll assertions and rapid reopen. Its full combined browser run is in progress after freeing completed build caches.

For integration verification, 35f31bb1 combines #6509 and #6554 in a separate validation branch. It passes complete hosted backend PostgreSQL 17/Redis 7 (27,254 passed, 105 conditional skips), default/auth browser E2E (399 / 5, no failures/flakes), frontend unit tests (2,810), strict blocking-I/O (235), lint/types/build and replay/golden. The E2E run and backend run identify that integration SHA; these are not mislabeled as new tests of #6509's unchanged published head.

#6554's main-based backend shard 4 still hits the cancellation-test hang tracked by #6552; it is disclosed separately. Production shutdown logic is unchanged here. I will synchronize #6509 after the independent upstream fixes are accepted and rerun the affected checks against the resulting published head.

@Guo-Yixin

Copy link
Copy Markdown
Contributor Author

@AnnaSuSu Thanks for the independent review and approval. Your validation boundaries are retained in the description; I have not treated mocked browser artifact serving as a real report-file download.

The upstream #6552 fix is now included. GitHub's automatic branch update misplaced the old test-only synchronization in a new lifecycle test; bce3f597 restores test_subagent_batch_service.py byte-for-byte to official main 02ce9ab2, retaining the new cleanup ownership, cancellation, concurrency, restart and Gateway deadline assertions. The focused lifecycle matrix is 31 passed, and a separate AI reviewer verified this merge repair.

At the exact published head bce3f597, node2 now passes 233 focused native SQL/HTTP/evidence/model-reader, lifecycle, real artifact-router and guidance tests with the PostgreSQL 17 URI enabled, zero skips. It also passes 235 strict blocking-I/O tests, lint/format/lock/diff and PR-range guidance. The existing artifact-router tests assert actual temporary-file response bytes; this is separate from the batch browser's href/image checks.

Complete hosted backend/PostgreSQL validation is running again for the updated revision. The independent browser corrections remain in #6554, and the complete node2 browser/auth/replay validation of the separately identified integration head is in progress. Official PR workflows still require maintainer approval; I will report their actual status separately and keep the latest full-suite results bound to their tested SHAs.

@Guo-Yixin

Guo-Yixin commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor Author

@AnnaSuSu @ggnnggez Final update after merging official main 19befe76 without conflicts. #6509 is now at fc150be0; the native batch feature and separate browser corrections retain their review boundaries.

The complete current-head backend/PostgreSQL 17/Redis 7 workflow passes 27,374 tests / 105 conditional skips. Current frontend unit tests (2,808), strict blocking-I/O (235), lint/guidance/types/format/production build and replay/golden all pass. #6509's newly executed full default/auth browser workflow passes 398 / 5 with zero failures/flakes.

node2's fresh native/affected-upstream matrix at fc150be0 passes 733 / 6 platform skips, strict blocking-I/O 235 and guidance/lint/format/lock/diff. Fresh real-backend replay at integration 3b606f19 passes 4 cases with retries disabled. Its complete hosted backend also passes 27,374 / 105 conditional skips.

The complete node2 default/auth/replay browser 399 / 5 / 4, all 2,810 frontend tests and check/format passed at c1a75104, with retries disabled and zero failures/flakes. Frontend and native batch fixture/router/projection/service sources are byte-identical at 3b606f19, independently verified; the old run remains labeled with its actual SHA. Standalone browser 394 / 5 at a5077457 likewise covers byte-identical frontend sources at 672f7d1f. Updated runtime validation is recorded separately rather than relabeling old runs.

The original scroll/transform/order assertions are retained; the missing-row synchronization still rejects the first complete reversed ordering. Synthetic report auth/image/href coverage remains disclosed separately from real artifact HTTP response-byte testing. All nine #6509 inline threads remain resolved; AnnaSuSu's approval applies to the earlier reviewed native implementation. Official upstream workflows still need maintainer approval. The independent #6554 remains separate; #6509 will receive it after upstream acceptance.

Integration note: #6554 can land independently of #6509. My proposed integration sequence is to let #6554 land when ready, then I will update #6509 from main and rerun its complete validation before the batch feature is considered for acceptance. I will handle those follow-up steps; landing order and timing remain at maintainer discretion. This relationship is now stated at the top of both PR descriptions.

@Guo-Yixin

Copy link
Copy Markdown
Contributor Author

Resolved the router-guide merge conflict at c6462e7e and synchronized official main through db9b0b4b, retaining remote branch updates. Both upstream MCP/custom-skill guidance and the native batch report contract are preserved. GitHub now reports this branch as mergeable.

Independent review confirms the contribution's code patches are unchanged against the new base and upstream goal/focus guards are preserved. Focused validation: backend 97 passed / 2 PostgreSQL-conditional skips, frontend 80 passed, pnpm check, changed-file Ruff lint/format, PR-range guidance (0 errors) and contribution diff checks passed. The PostgreSQL skips are not counted as passes. No full local suite was repeated; prior full-suite results keep their actual tested SHAs.

The six official workflows for the new head are still awaiting maintainer approval. #6554 remains a separate main-based browser fix; after it lands I will update this branch, run the affected local checks and use the official full CI for merge validation.

@Guo-Yixin

Copy link
Copy Markdown
Contributor Author

@willem-bd @ggnnggez A quick follow-up when you have time: all nine inline review threads have been addressed and resolved, and GitHub currently reports #6509 as conflict-free.

The unrelated browser/layout fixes were split into #6554 as requested. Its official backend/frontend unit tests, E2E, Replay and lint checks have now passed. The six official validation workflows for #6509's current head (237bbdac) are still awaiting approval to run; prior validation remains labeled with its actual tested SHA in the PR description.

Could you take another look and, if appropriate, approve those workflows so the current head can receive upstream validation? I will handle updating #6509 from main and the affected checks after the separate fix lands. Thanks for your time and review.

@Guo-Yixin

Copy link
Copy Markdown
Contributor Author

@WillemJiang Could you help with maintainer review and CI approval when you have time?

#6509 now completes report/evidence inspection in the native batch panel. All nine inline review threads are resolved, the branch is conflict-free, and Totoro-qaq has approved the current head (237bbdac); AnnaSuSu approved an earlier revision. The six official validation workflows for the current head are still awaiting approval to run, including Unit Tests and E2E Tests.

As requested during review, the unrelated browser/layout fixes are separate in #6554. Its official unit, E2E, Replay and lint workflows have passed, and it is awaiting review. I will handle synchronizing #6509 and the affected checks after that fix lands. Prior validation is linked by its tested SHA in the descriptions.

Thanks for your time.

@Guo-Yixin

Copy link
Copy Markdown
Contributor Author

Closing this PR as requested so the earlier batch-review/browser work is no longer left open. Its validation and review history remain available in the PR; no merge is requested.

@Guo-Yixin

Copy link
Copy Markdown
Contributor Author

Closing this PR as requested.

@Guo-Yixin Guo-Yixin closed this Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agents Agents, subagents, graph wiring, prompts, langgraph.json area:backend Gateway / runtime / core backend under backend/ area:ci GitHub Actions, CI config, repo tooling area:docs Documentation and Markdown only area:frontend Next.js frontend under frontend/ needs-validation Touches front/back contract surface; needs real-path validation risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/XL PR changes 700+ lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC] Browser review of durable batch reports and captured evidence

5 participants