Test identifier handling against real engines and hostile schemas - #27
Merged
Merged
Conversation
A mixed-case Postgres schema failed every query and shipped that way for weeks. Nothing in CI had ever pointed at a schema we did not write ourselves, so every database test shared the same blind spots. This runs the normalise-then-guard path against Postgres, MySQL and SQLite over schemas built to break the rules: mixed case, reserved words, spaces, unicode, names the parser treats as keywords. No model is involved; it is not needed to surface this class of defect and would only make the result non-deterministic. Two rules keep it from going quiet. On a folding engine each fixture first asserts the bare form genuinely fails, so a fixture that would pass without the fix is reported as testing nothing. An engine that cannot be reached is skipped rather than passed, and CI names the engines it requires so the job cannot go green on the embedded one alone. Removing the normalisation makes three fixtures fail with the error users saw: relation "customers" does not exist.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A mixed-case Postgres schema failed every query and shipped that way for weeks. The reason is structural: nothing in CI had ever pointed at a schema we did not write ourselves, so every database test shared the same blind spots by construction.
pnpm test:schemasruns the normalise-then-guard path against Postgres, MySQL and SQLite over schemas built to break the rules: mixed case, reserved words, spaces and symbols, unicode, and names the SQL parser treats as keywords.No model is involved. This class of defect does not need one to surface, and a model would only make the result non-deterministic.
What keeps it honest
--require=postgres,mysql,sqlite, so the job cannot go green on the embedded engine alone when the service containers fail to start.Verified against itself
Removing the normalisation makes three fixtures fail with the error users actually saw:
Restored: 26 passed, 0 failed, 0 skipped across the three engines.
Adds a
schema-regressionCI job with Postgres and MySQL services, and documents the gate in CONTRIBUTING.