Skip to content

[S0-02] Audit complete real-paper Electron journey and record every failure #26

Description

@dkritarth

Outcome

Produce current, evidence-backed truth for Vellum's real user journey. Existing merged cards and unit tests are not accepted as proof.

Dependencies

Blocked by #25. Do not start until #25 is closed and this issue receives status:ready.

Scope

Use one real born-digital paper and an isolated test data directory. Exercise input classification, ingest, library, tab opening, reader navigation, search, Details, summary, Ask, model selection, chat persistence, and restart behavior. File one focused bug issue for every reproducible failure; do not hide failures inside this audit issue.

Acceptance criteria

  • Test paper and expected metadata are recorded.
  • Every core step has pass/fail result and screenshot or console evidence.
  • Codex Ask response is checked against actual paper text for grounding.
  • Empty, loading, error, and restart states are exercised.
  • Every failure has its own linked issue with reproduction steps.
  • No defect is fixed inside this audit unless required only to make audit tooling run.

Live verification procedure

  1. Start from clean isolated data/ state.
  2. Launch npm run dev.
  3. Ingest real arXiv ID, then repeat with same input to check idempotency.
  4. Open Library; search/sort; open paper tab.
  5. Navigate pages, zoom, TOC, and in-document search.
  6. Inspect Details and summary against PDF.
  7. Ask factual and interpretive questions through Codex.
  8. Start new chat, reload app, restart app, and confirm state.
  9. Capture renderer/main console output and screenshots.

Deliverable

Attach concise test matrix to issue or PR. Create separate bug issues with severity, expected/actual behavior, reproduction, suspected area, and evidence.

Required PR evidence

PR must use Closes #26, remain focused on this issue, and include:

  • failing test or reproducible baseline before behavior changes;
  • focused and full automated results;
  • npm test, npm run typecheck, npm run build, and git diff --check;
  • exact live Electron steps and observed results;
  • screenshots for visible UI changes when tooling permits;
  • console errors/warnings and unresolved limitations;
  • real-paper or signed-in ACP evidence where applicable.

Mocks prove code paths only. They never prove Electron integration, persistence across restart, PDF rendering, or signed-in ACP behavior.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:electronElectron main, preload, IPC, or live app workarea:uxUser-facing workflow or interfacephase:0-stabilizationPhase 0: prove and repair core journeystatus:in-progressImplementation PR exists or work is actively claimed

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions