Skip to content

[codex] Add high-fidelity PDF file attachment plumbing#983

Open
mimeding wants to merge 1 commit intoosaurus-ai:mainfrom
mimeding:codex/pdf-io-production-ready
Open

[codex] Add high-fidelity PDF file attachment plumbing#983
mimeding wants to merge 1 commit intoosaurus-ai:mainfrom
mimeding:codex/pdf-io-production-ready

Conversation

@mimeding
Copy link
Copy Markdown
Contributor

@mimeding mimeding commented Apr 30, 2026

Business rationale

File fidelity is a core part of the Osaurus harness: users need PDFs, presentations, spreadsheets, audio, video, and other local documents to survive chat, plugin, and tool handoffs without being flattened into lossy text or accidentally dropped. Preserving private files as local artifacts keeps the model's prompt lightweight while giving trusted local tools access to the original bytes when the user asks for high-fidelity read, edit, render, or export workflows.

Coding rationale

Attachment.Kind.file carries filename, MIME type, size, host path, and extracted preview so rich files can be treated differently from inline text while keeping old attachment decoding compatible. AttachmentFileStore centralizes file preservation under Osaurus-controlled paths instead of scattering copy logic through chat views. _context.attachments is injected into external and plugin tool payloads as a narrow metadata surface, so tools can opt into real files without bloating every prompt. Attachment spillover preserves file attachments before falling back to blobs because the bytes already live outside chat history JSON.

What changed

  • Added high-fidelity Attachment.Kind.file support with MIME/category helpers and compatibility decoding.
  • Added AttachmentFileStore and Osaurus path support for preserved attachment files.
  • Preserved supported uploads from ChatView / FloatingInputCard while retaining compact extracted previews.
  • Added _context.attachments to ExternalTool and v2 plugin host payloads.
  • Updated plugin host structs/ABI comments for attachment metadata.
  • Kept preserved files out of blob spillover while maintaining dedup/blob behavior for other attachment kinds.
  • Added tests for file preservation, spillover behavior, plugin payload injection, host API shape, decoding compatibility, and file-type support.

Validation

  • git fetch origin && git rebase origin/main
  • swift build --package-path Packages/OsaurusCore
  • swift build --package-path Packages/OsaurusCore -c release
  • swift test --package-path Packages/OsaurusCore (1445 tests in 195 suites passed; SandboxIntegrationTests skipped by guard)
  • xcrun swift-format lint --strict on every touched Swift file
  • swiftlint lint --strict on every touched Swift file
  • git diff --check origin/main...HEAD
  • CLI gate skipped: this slice does not touch Packages/OsaurusCLI.

Non-scope

  • Does not implement the PDF or PPTX companion plugins.
  • Does not add document-format adapters or emitters.
  • Does not change remote upload/sync behavior; files remain local Osaurus-managed artifacts.

Residual risks

  • Plugin authors still need to handle host_path carefully and preserve folder/path containment in their own tools.
  • Very large real-world files may need format-specific preview limits beyond the generic preserved-file path introduced here.

@mimeding mimeding force-pushed the codex/pdf-io-production-ready branch from 497f813 to 579226f Compare May 1, 2026 02:53
@mimeding mimeding marked this pull request as ready for review May 1, 2026 03:35
@mimeding mimeding force-pushed the codex/pdf-io-production-ready branch 2 times, most recently from 177514a to e5b5e76 Compare May 1, 2026 17:21
@mimeding
Copy link
Copy Markdown
Contributor Author

mimeding commented May 1, 2026

@tpae could you please review the core high-fidelity file I/O contract in this PR?

The key design question is whether this is the right boundary for PDF/PPT/PPTX support:

  • Osaurus preserves original PDF/PPT/PPTX bytes outside chat-history JSON.
  • Tool/plugin calls receive a compact _context.attachments manifest with { id, filename, mime_type, file_size, host_path }.
  • Lightweight text extraction remains only model context/preview; high-fidelity work is expected to go through native v2 plugins using the preserved file path.

Question: are you comfortable with trusted native plugins receiving host_path for preserved attachments, or would you prefer an additional host API/file-read indirection before we merge this contract?

Follow-up blocker: the companion osaurus-pdf plugin implementation is local and validated, but I need guidance on where its private GitHub remote/repo should live before opening that plugin PR.

Business rationale: File fidelity is a core part of the Osaurus harness: users need PDFs, presentations, spreadsheets, audio, video, and other local documents to survive chat, plugin, and tool handoffs without being flattened into lossy text or accidentally dropped. Preserving private files as local artifacts strengthens trust because the app can reason over previews while keeping the original bytes available on the user's machine.

Coding rationale: Attachment.Kind.file carries filename, MIME type, size, host path, and extracted preview so downstream code can distinguish rich files from inline text. AttachmentFileStore centralizes local preservation under Osaurus-controlled paths instead of scattering copy logic across views, while ExternalTool and PluginHostAPI expose a bounded _context.attachments surface for tools. Spillover preserves file attachments before falling back to blobs, and the new tests pin security boundaries, plugin API shape, and high-fidelity handoff behavior.

Co-authored-by: Codex <codex@openai.com>
@mimeding mimeding force-pushed the codex/pdf-io-production-ready branch from e5b5e76 to 22b2568 Compare May 3, 2026 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant