Skip to content

Trip Editor E2E tests must prove real contracts, not mocked CRUD paths #297

@stef-k

Description

@stef-k

Problem

Trip Editor manual release-candidate testing repeatedly found basic CRUD, state, image, and parity failures after agents reported green or focused E2E results. The immediate cause is not that browser testing is useless; it is that several tests currently fake or mock critical backend behavior and then get described as end-to-end proof.

Mocked Playwright tests are useful for frontend rendering states, but they cannot prove CRUD correctness, authorization rules, persistence, affected-order responses, or real endpoint behavior.

Concrete recent example from #295:

  • A browser test proved the Region selector/request path for Unassigned Places.
  • The actual backend update path still returned 403 when moving an existing place from a normal region to Unassigned Places.
  • Manual testing found the failure.
  • The fix required backend mutation tests plus a browser assertion that the sidebar visually moved the place after the real contract was corrected.

Required Outcome

Make the Trip Editor test suite honest about what it proves, and replace weak mocked/proxy assertions with real contract coverage for release-critical workflows.

Testing Policy

  • Tests that mock mutation responses must not be described as E2E proof for CRUD correctness.
  • Mocked Playwright tests may prove frontend rendering, loading, error, and difficult browser-only states.
  • Release-critical CRUD requires backend-rule coverage plus browser-visible outcome coverage.
  • If a browser test uses mocked data for a mutation, the test/report must explicitly state that it proves only frontend behavior and must be paired with backend tests for the same mutation contract.
  • Full Trip Editor E2E passing is not release proof unless the important tests prove real user contracts.

Audit Scope

Audit the Trip Editor workflows below and classify existing tests as:

  • contract: proves final user-visible outcome;
  • backend-rule: proves server mutation/authorization/validation/persistence;
  • integration: uses real endpoint plus UI interaction;
  • mocked-only: proves frontend rendering only;
  • proxy: proves only selector/button/request/presence;
  • missing: no meaningful proof.

Workflows:

  • Trip metadata save, public trip, share progress.
  • Region create/edit/delete/reorder.
  • Place create/edit/delete/reorder/move between regions/Unassigned Places.
  • Place geosearch add.
  • Place coordinate pick.
  • Area create/edit/delete/reorder/polygon map work.
  • Segment create/edit/delete/reorder/route map work.
  • Visit progress/history.
  • Rich notes text/images/alignment/sanitize/save.
  • Sidebar search/filter.
  • Map utilities: fit, recenter, focus, zoom display, measure, copy link.
  • Published production bundle loading.
  • Light/dark/narrow layout.

Fix Requirements

For every high-risk or release-critical workflow:

  • add or strengthen backend tests for mutation rules;
  • add or strengthen focused browser tests for final user-visible state;
  • remove or relabel tests that only prove mocked request/render behavior;
  • avoid adding broad slow tests when a focused contract test is enough;
  • keep runtime under control by using focused specs during development and one full-suite smoke only near final release-candidate validation.

Validation

When this issue is implemented, provide:

  • a coverage classification matrix;
  • list of tests strengthened or removed;
  • focused validation results;
  • full Trip Editor E2E runtime and result only once the focused contract coverage is corrected;
  • remaining manual-only RC checks, limited to visual feel and final confirmation rather than basic CRUD discovery.

Notes

This is not a request to add more fake E2E tests. It is a request to make the suite truthful and useful so manual testing is no longer the primary mechanism for discovering basic editor failures.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions