chore: sync main into staging - #156
Merged
suguanYang merged 34 commits intoJun 16, 2026
Merged
Conversation
Release staging to main - 2026-05-19
Sync staging → main
Update Metrics
Release 0602
Route oversized PDF shards through S3 for MinerU
…t-s3-mineru Revert "Route oversized PDF shards through S3 for MinerU"
…flow Move production release flow to GitHub Releases
* feat(retrieval): enhance agentic navigation with scope tracking, outline collection, and fallback auto-collect (#125) * feat: enhance navigation state with error handling, custom tool parameters, and improved workflow plan caching * feat(retrieval): enhance agentic navigation with scope tracking, outline collection, and fallback auto-collect - Replace scope_stack with flat current_scope for cleaner BACK/DRILL navigation - Add structured exit_reason tracking across all navigation exit paths - Implement fallback auto-collect when navigation interrupted by budget/latency/max_steps - Fix outline collections to not exclude children from future navigation - Add BACK target validation with path-parent fallback - Fix evidence tree indentation for nested sections - Fix pyright type errors in plan_service by using explicit kwargs - Fix lint: remove extraneous f-string prefixes in asset tools * Resolve Issue #126 (#127) * feat: inject discovery signals as advisory hints into agent document selection prompts * fix(agentic): resolve evidence tree orphan nesting, outline leakage, and enforce English agent reasoning - Fix _build_outline_subtree to reparent existing children under newly created parent nodes, preventing orphan depth errors (Bug 1) - Filter outline_items in _hydrate_collected to exclude ancestor/sibling context from load_child_sections, eliminating echo structures (Bug 1) - Change render_evidence guard from has_content() to has_leaf_content(), skipping documents with outline-only trees when no chunks were collected (Bug 2) - Add English language requirement to COLLECTOR_PROMPT and _PLANNER_PROMPT for agent-generated free-text fields like reason and reasoning_summary (Bug 3) - Remove unused imports flagged by ruff lint Closes #117 * refactor: remove inspect tool, update asset search with optional VLM, and improve navigation stop/fallback logic * feat: implement H2-aware shard refinement logic to improve document splitting granularity and adherence to size constraints. * test(contract): update discovery selection contract tests to match id_to_path signature * chore(agentic): remove legacy topic_hints dead code after c0261eb refactor (#128) - Remove image_topic_hints / table_topic_hints params from build_asset_tools_block (hints were never used in the new prompt format) - Remove dead load_asset_topic_hints function from assets.py - Remove hint-loading block and wasted DB queries from navigate_step in tools.py * fix: route oversized pdf shards through s3 * feat(agentic): add discovery fallback, budget-aware prompts, and VLM model upgrade (#133) - Add Phase 2.5 discovery fallback for unnavigated documents that have bottom-discovery hits but were missed by KG document selection - Add adjust_budget_snapshot() to pre-adjust budget state shown to LLM - Update prompt_service with model-specific formatting (DeepSeek/Qwen) - Upgrade IMAGE_MODEL to qwen2.5-vl-72b-instruct for better VLM quality - Remove unused DocTreeNode import (lint fix) - Clean up navigation tools dead code Closes #132 * feat(agentic): reconcile navigation asset search flow (#136) * refactor: remove legacy retrieval routes and standardize agentic orchestration with enhanced decision tracing and token tracking * feat: implement structured DecisionTraceStep tracking and refine agentic navigation prompt constraints * feat(agentic): reconcile navigation asset search flow --------- Co-authored-by: suguanYang <wangbinqi77@gmail.com>
* feat: unify PDF document profile routing * feat: add TOC profiling support and upgrade Qwen models to 3.6-flash * fix: update Qwen model references to version 3.6-flash in AGENTS.md and README.md * test: stabilize worker contract imports
…king (#143) * refactor: streamline TOC handling and remove deprecated settings - Removed PDF_PROFILE_TOC_ENABLED from configuration and adjusted related logic in document profiling. - Simplified TOC detection logic in PDF and Atlas parsers, ensuring consistent behavior across document types. - Updated tests to reflect changes in TOC handling and ensure proper functionality without the deprecated flag. * feat: add page memory parse track * feat: add usage_task tracking to various document processing components - Introduced usage_task parameter in LLM calls across multiple services including summary_builder, react_loop, planner, and various document agent tools. - Enhanced token tracking and stage profiling in document ingestion processes to improve performance monitoring and cost estimation. - Updated relevant functions to utilize new token and stage tracking methods, ensuring accurate resource usage reporting. Cherry-picked from feat/wuchengke/2026-06-11 (d5d7d042) * fix: add parse_track column to test document insert helpers The page_memory migration added a NOT NULL parse_track column to the documents table, but the contract test helpers (contract_database.py, test_job_creation_contract.py, test_documents_contract.py) were still inserting documents without it, causing all retrieval contract tests to fail with NotNullViolationError. * fix: move imports inside test functions to survive module cache eviction The worker contract conftest clears shared.core.config and app.* modules from sys.modules between tests. Top-level imports of settings and service functions become stale references pointing at evicted module objects. Moving all imports inside each test function body ensures the monkeypatch target and the function under test share the same settings instance. Also fixes the parse_track NOT NULL test failures from the previous commit (verified: 174 passed, 0 failed locally with the exact CI test command).
* Add anonymous self-hosted telemetry client * Fix LocalStack SNS subscription confirmation * Fix telemetry shutdown task wait
fix: update page billing rate
Use PostHog SDK for self-hosted telemetry
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| if doc is not None: | ||
| try: | ||
| doc.close() | ||
| except Exception: |
| if self._profile_plan_row is not None: | ||
| self._db.expunge(self._profile_plan_row) | ||
| self._profile_plan_row = None | ||
| except Exception: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Syncs staging to the current main tree through a merge commit because staging is protected and cannot be force-updated. The previous staging tip is backed up at chore/wangbinqi/staging-backup-20260617. Supersedes PR #155.