Skip to content

[PROD-008] .NET idempotency rerun E2E and Docker PostgreSQL validation#629

Merged
ktalpay merged 1 commit into
developfrom
feature/prod-008-dotnet-idempotency-rerun-e2e-docker-postgresql-validation
May 22, 2026
Merged

[PROD-008] .NET idempotency rerun E2E and Docker PostgreSQL validation#629
ktalpay merged 1 commit into
developfrom
feature/prod-008-dotnet-idempotency-rerun-e2e-docker-postgresql-validation

Conversation

@ktalpay

@ktalpay ktalpay commented May 22, 2026

Copy link
Copy Markdown
Owner

Summary

Adds the .NET opt-in Docker PostgreSQL E2E/idempotency validation baseline.

This validates .NET schema bootstrap, source-cycle parsing, source-specific master/detail insert, idempotent rerun duplicate skips, year-state progression, no-available-source-year behavior, and failed persistence rollback behavior against an externally supplied PostgreSQL target.

This does not make .NET or the whole project production-ready. Full all-source .NET E2E execution and Python/.NET persisted parity validation remain incomplete.

Files changed

  • docs/postgresql-runtime-readiness-checklist.md
  • docs/production-packaging-operator-runbook.md
  • docs/production-parity-contract.md
  • src/dotnet/README.md
  • tests/dotnet/CarbonOps.Parser.Contracts.Tests/DotNetPostgreSQLIntegrationE2ETests.cs

Docker PostgreSQL opt-in behavior

Adds opt-in .NET PostgreSQL integration tests guarded by:

CARBONOPS_RUN_DOTNET_POSTGRESQL_INTEGRATION=1

The tests accept externally supplied PostgreSQL settings via either:

CARBONOPS_DOTNET_POSTGRESQL_TEST_DSN

or split CARBONOPS_PARSER_POSTGRES_* settings through the existing .NET production config boundary.

Default tests do not open PostgreSQL.

E2E behavior validated

The opt-in E2E path validates:

  • schema bootstrap idempotency
  • DEFRA/DESNZ local fixture parsing
  • source-specific master/detail insert
  • duplicate rerun skip behavior
  • source-family year-state progression
  • no_available_source_year no-op behavior
  • failed persistence does not advance year-state

Idempotency/rerun behavior

The E2E test verifies:

  • first run inserts master/detail rows
  • second run inserts zero new master/detail rows
  • second run reports duplicate skips
  • row counts remain stable after rerun

Year-state behavior

The E2E test verifies:

  • successful persistence records the 2024 source-family year
  • next target year becomes 2025
  • no-available-source-year does not advance state
  • failed persistence does not advance state

Redaction/safety behavior

The tests assert diagnostics redact PostgreSQL password and connection-string values.

The test path uses externally supplied credentials only. No DSN, password, token, or local secret is committed.

No uncontrolled live network access is introduced.

Production parity status

This satisfies only the .NET idempotency/rerun E2E and Docker PostgreSQL validation baseline item from the production parity contract.

Current verdict remains:

  • Project-level production-ready: no.
  • Python runtime production path: yes.
  • .NET runtime production path: no; Docker PostgreSQL E2E/idempotency validation baseline added.

Validation performed

  • git diff --check: passed
  • dotnet restore src/dotnet/CarbonOps.Parser.sln: passed
  • dotnet build src/dotnet/CarbonOps.Parser.sln --configuration Release --no-restore --disable-build-servers -m:1: passed, 0 warnings/errors
  • focused .NET tests including opt-in guard: 39 passed
  • opt-in PostgreSQL E2E with externally supplied DSN: 7 passed
  • python -m pytest tests/test_production_parity_contract.py: 3 passed
  • python -m pytest tests/test_release_validation_gate.py: 7 passed

Known unrelated baseline

The full .NET contract test project has previously shown deterministic fixture/hash failures outside the PROD-008 E2E surface. This PR does not broaden into those failures.

Remaining blockers

The following .NET parity items remain:

  • full three-source .NET Docker PostgreSQL E2E
  • service-level .NET run-once execution
  • Python/.NET persisted parity validation
  • final project production-ready verdict

Verdict

Project-level production-ready: no.
Python runtime production path: yes.
.NET runtime production path: no; Docker PostgreSQL E2E/idempotency validation baseline added.

Task-ID: PROD-008
Task-Issue: #628

@ktalpay
ktalpay merged commit e72d041 into develop May 22, 2026
2 checks passed
@ktalpay
ktalpay deleted the feature/prod-008-dotnet-idempotency-rerun-e2e-docker-postgresql-validation branch May 22, 2026 18:08

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be10349bf8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

await Assert.ThrowsAnyAsync<PostgresException>(
() => session.PersistSourceFamilyYearAsync(invalidBatch));

Assert.Equal(0, await CountYearStateRowsAsync(dataSource, SourceFamily.DefraDesnz, 2024));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Make rollback assertion independent of prior year-state rows

When CARBONOPS_DOTNET_POSTGRESQL_TEST_SCHEMA is set (the documented fallback when schema creation isn’t allowed), all integration tests in this class share one schema. DefraSourceCycleInsertRerunAndYearStateAreIdempotentWhenEnabled inserts defra_desnz/2024 into source_family_year_states, so this assertion can fail based solely on test execution order rather than rollback behavior. Capture the row count before the invalid persist and assert it is unchanged afterward to avoid order-dependent failures in the supported explicit-schema mode.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant