Skip to content

test(ingestion): add fixture-based end-to-end coverage for fetch runtime and persistence - #42

Merged
CrisSTEM merged 1 commit into
mainfrom
test/33-ingestion-e2e
Apr 16, 2026
Merged

test(ingestion): add fixture-based end-to-end coverage for fetch runtime and persistence #42
CrisSTEM merged 1 commit into
mainfrom
test/33-ingestion-e2e

Conversation

@CrisSTEM

Copy link
Copy Markdown
Owner

Summary

This PR adds fixture-based end-to-end ingestion coverage for the v0.3.0 runtime and persistence path.

The repository already had source-specific integration coverage for GitHub, news RSS, changelog, runtime dispatch, and CLI non-zero behavior. This PR adds the missing cross-source runtime-level tests that exercise real fetchers against mocked upstream responses and verify persisted rows in the same temporary SQLite database.

Linked issue(s)

Milestone fit

This PR strengthens the v0.3.0 ingestion milestone by adding deterministic end-to-end coverage for the live runtime path without introducing any new product behavior.

It stays inside milestone scope by:

  • using temp SQLite databases
  • using fixture/mocked upstream responses only
  • validating persisted fetch_runs, metric_snapshots, and content_items
  • proving rerun dedupe for content items
  • proving mixed success/failure recording at runtime level

What changed

  • added internal/fetch/runtime_e2e_test.go
  • added a cross-source success-path runtime test that:
    • runs GitHub ingestion
    • runs news RSS ingestion twice
    • runs changelog RSS ingestion twice
    • verifies persisted rows across fetch_runs, metric_snapshots, and content_items
    • verifies rerun dedupe preserves first-write content rows
  • added a mixed success/failure runtime test that:
    • runs multiple GitHub bindings with one simulated upstream failure
    • verifies successful bindings still persist rows
    • verifies failed bindings record failed fetch_runs
    • verifies the failing binding does not persist snapshots
    • verifies subsequent news and changelog runs still succeed in the same database
  • reused the existing deterministic httptest helpers and fixture pack patterns already present in internal/fetch

Why this shape

Most of the issue scope was already covered in main through:

  • source-specific fetcher integration tests
  • runtime dispatch tests
  • CLI fetch exit-code tests

Given that local repository state, this PR keeps the change minimal and auditable by adding the missing multi-source E2E runtime coverage instead of refactoring the existing test suite.

Out of scope

  • live network ingestion tests
  • scheduler coverage
  • frontend or API tests
  • product/runtime behavior changes
  • release documentation beyond test coverage

Validation

  • go test ./internal/fetch -count=1
  • go test ./... -count=1
  • go vet ./...
  • python3 ./scripts/trim-whitespace.py --check
  • make ci

Documentation impact

  • No docs update needed

Reviewer focus

  • internal/fetch/runtime_e2e_test.go
    • cross-source runtime coverage
    • same-database persistence assertions
    • rerun dedupe assertions
    • mixed success/failure recording
  • confirm the new tests complement, rather than duplicate, existing source-level integration coverage

Auditability checklist

  • Branch name follows project convention
  • PR scope matches the linked issue and milestone
  • No live-network CI dependency was introduced
  • Relevant tests were added for runtime-level persistence coverage
  • Deferred work is explicitly called out
  • No unimplemented capabilities are implied

Notes for merge

Preferred merge method:

  • Issue PRs: Rebase and merge
  • Release aggregation PRs: Merge commit
  • Use Squash and merge only when the branch became noisy and preserving intermediate commits no longer adds value

@CrisSTEM
CrisSTEM merged commit 5d396b1 into main Apr 16, 2026
1 check passed
@CrisSTEM
CrisSTEM deleted the test/33-ingestion-e2e branch April 16, 2026 03:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(ingestion): add fixture-based end-to-end coverage for fetch runtime and persistence

1 participant