Skip to content

feat: report durable full-pipeline source progress - #498

Open
jmagar wants to merge 6 commits into
mainfrom
codex/full-pipeline-progress
Open

feat: report durable full-pipeline source progress#498
jmagar wants to merge 6 commits into
mainfrom
codex/full-pipeline-progress

Conversation

@jmagar

@jmagar jmagar commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • make source progress runner-owned and durable for every adapter, with optional fine-grained adapter snapshots
  • report canonical counts across discovering, diffing, fetching, enriching, normalizing, preparing, batching, embedding, vectorizing, upserting, publishing, and completion
  • project canonical source kinds and phase-aware progress through CLI, JSON status, MCP task payloads/notifications, events, and provider heartbeats
  • add graceful progress-persistence degradation, structured warnings, redaction-aware vector counts, and embed=false phase correctness
  • split the touched orchestration, vectorization, reservation, and fake-store inspection code to satisfy the monolith policy without allowlist additions
  • bump the synchronized CLI release version to 7.2.7 and regenerate OpenAPI/public-API artifacts

Verification

  • cargo fmt --all -- --check
  • cargo clippy --all-targets --locked -- -D warnings
  • cargo test --no-run --workspace --lib --locked
  • adapter fallback and 22 web acquisition tests
  • coordinator, vectorization, event, service projection, job store, CLI, API, MCP, and web transport suites
  • deterministic 130-file multi-acquisition-batch and multi-embedding-batch integration test
  • embed=false integration regression
  • npm web lint and production build
  • cargo xtask check
  • cargo xtask check-fetch-divergence
  • cargo xtask check-crate-contracts
  • cargo xtask check-release-versions --base origin/main --head HEAD --mode pr
  • pre-commit and pre-push hooks

Closes #494

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement durable full-pipeline source progress across all adapters and stages

1 participant