Problem
Trace 019e1abc4a46b94ec4874f74d5ec13ac for staging job job_f2fcb52b2db7 showed the parser produced and exported 72 chunks, but final document/retrieval state only published 11 chunks because publication dedup reused the deduped chunk list for document_chunks.
Expected
Result ZIP, job_chunks, and document_chunks should all preserve the full parsed result chunk set. Deduplication should remove duplicate local media after export and record stats, but it should not shrink document-facing chunk state.
Evidence
Fix Direction
Keep publication dedup stats and local media cleanup, but publish the full normalized parsed chunk list during job finalization.
Problem
Trace
019e1abc4a46b94ec4874f74d5ec13acfor staging jobjob_f2fcb52b2db7showed the parser produced and exported 72 chunks, but final document/retrieval state only published 11 chunks because publication dedup reused the deduped chunk list fordocument_chunks.Expected
Result ZIP,
job_chunks, anddocument_chunksshould all preserve the full parsed result chunk set. Deduplication should remove duplicate local media after export and record stats, but it should not shrink document-facing chunk state.Evidence
job_chunkspreservation.61/72 duplicate chunks skippedand graph publication withchunks=11.Fix Direction
Keep publication dedup stats and local media cleanup, but publish the full normalized parsed chunk list during job finalization.