-
Notifications
You must be signed in to change notification settings - Fork 35
test: [expected failures] function-library fixtures pending spec decisions #165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
leongdl
wants to merge
1
commit into
OpenJobDescription:mainline
Choose a base branch
from
leongdl:conformance-func-lib-expected-failures
base: mainline
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
54 changes: 54 additions & 0 deletions
54
conformance-tests/2023-09/EXPR/jobs/proposed/README-func-lib.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| # Proposed EXPR fixtures (function library) — jobs | ||
|
|
||
| Fixtures listed here are parked instead of live, because a reference | ||
| implementation fails them or because the spec is silent on the behaviour | ||
| they pin. Placement is kind-level (`<component>/<kind>/proposed/`); | ||
| promotion is a mechanical move up one directory. The runner does not scan | ||
| `proposed/`. This README is named per fixture family | ||
| (`README-func-lib.md`) because other expected-failures PRs park fixtures | ||
| in this same directory with their own READMEs. | ||
|
|
||
| Observations are per-implementation (`rs` = openjd-rs, `py` = the Python | ||
| openjd CLI); dual results from the 2026-08-12 sweep, re-verified against | ||
| the current upstream/main rs build where noted. | ||
|
|
||
| ## `expr2.2.4--center-odd-padding.test.yaml` — spec silence, py outlier | ||
|
|
||
| §2.2.4 does not say which side of `center()` receives the extra space for | ||
| odd padding. Measured for `center("hi", 7)`: | ||
|
|
||
| | Engine | Output | Extra space | | ||
| |---|---|---| | ||
| | CPython `str.center` | `' hi '` | LEFT (verified by execution) | | ||
| | openjd-rs (current upstream/main) | `' hi '` | LEFT — matches CPython (changed by upstream #305 + RFC 0005 coercion sync; re-verified this session) | | ||
| | Python openjd CLI | `' hi '` | RIGHT — now the outlier | | ||
|
|
||
| The fixture asserts the CPython/rs behaviour — the de facto answer. | ||
| Classification: **spec decision needed** ("follow CPython" would make this | ||
| promotable and the Python CLI's split a plain bug). An earlier revision of | ||
| this README claimed both implementations were right-heavy and that the | ||
| fixture's expectation was wrong; both claims were incorrect (rs changed, | ||
| and CPython is left-heavy). | ||
|
|
||
| ## `expr2.2.4--isdigit-unicode.test.yaml` — spec ambiguity, no divergence | ||
|
|
||
| §2.2.4 never defines "digit". Both openjd implementations agree on | ||
| ASCII-only (`isdigit("٣")` is false on rs AND py); CPython's host | ||
| `str.isdigit` says True; §2.2.5 gives `\d` explicit Unicode semantics. | ||
| The fixture asserts the Unicode reading as a strawman — the OPPOSITE of | ||
| the current de facto agreement. Classification: **spec decision needed** | ||
| (define "digit"; ASCII-only is the likely ratification given both | ||
| implementations agree, in which case flip the expectation rather than | ||
| promote as-is). | ||
|
|
||
| ## `expr2.2.6--repr-py-newline-roundtrip.test.yaml` — bug in BOTH | ||
|
|
||
| §2.2.6: repr_py "follows the behavior of Python's repr", whose example | ||
| escapes `\n`. BOTH implementations emit a raw newline inside the quoted | ||
| literal, producing invalid Python (`ast.literal_eval` raises | ||
| SyntaxError). Classification: **implementation bug in both**; the spec is | ||
| explicit. Promote once fixed. End-to-end twin through WRAP_ACTIONS | ||
| forwarding: `WRAP_ACTIONS/jobs/proposed/ | ||
| wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml` (wrap-actions | ||
| expected-failures PR) — note that twin is additionally gated on the §5.2 | ||
| ArgString newline question. |
36 changes: 36 additions & 0 deletions
36
conformance-tests/2023-09/EXPR/jobs/proposed/expr2.2.4--center-odd-padding.test.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # PARKED: spec-silent behaviour with a live implementation divergence. | ||
| # | ||
| # Expression Language §2.2.4 specifies center(s, width) only as "Center, pad | ||
| # with spaces to width" — it does not say which side receives the extra space | ||
| # when the padding is odd. | ||
| # | ||
| # Observed for center("hi", 7) (5 pad spaces to split), 2026-08 measurements: | ||
| # CPython str.center: " hi " (3 left, 2 right — extra space LEFT; | ||
| # left = (width - len) // 2 + ((width - len) & width & 1)) | ||
| # openjd-rs: " hi " (matches CPython since upstream #305 + | ||
| # the RFC 0005 coercion sync; verified again on the | ||
| # current upstream/main build) | ||
| # Python openjd CLI: " hi " (2 left, 3 right — now the OUTLIER) | ||
| # | ||
| # The expected: block below asserts the CPython/openjd-rs behaviour, which | ||
| # has become the de facto answer. Do NOT move this fixture into jobs/ until | ||
| # the spec picks a side ("follow CPython" would make this promotable and the | ||
| # Python CLI's split a plain bug). | ||
| template: | ||
| specificationVersion: jobtemplate-2023-09 | ||
| extensions: | ||
| - EXPR | ||
| name: TestJob | ||
| steps: | ||
| - name: Step1 | ||
| script: | ||
| actions: | ||
| onRun: | ||
| command: python | ||
| args: | ||
| - -c | ||
| - | | ||
| print(r'CENTER:[{{ center("hi", 7) }}]') | ||
| expected: | ||
| output: | ||
| - "CENTER:[ hi ]" |
40 changes: 40 additions & 0 deletions
40
conformance-tests/2023-09/EXPR/jobs/proposed/expr2.2.4--isdigit-unicode.test.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # PARKED: spec ambiguity — NO divergence between the openjd implementations. | ||
| # | ||
| # Expression Language §2.2.4 specifies isdigit(s) as "True if all characters | ||
| # are digits and string is non-empty" without defining "digit": ASCII 0-9 or | ||
| # Unicode digit category (Nd)? §2.2.5 explicitly gives \d Unicode semantics, | ||
| # which pulls one way; both openjd implementations pull the other. | ||
| # | ||
| # Observed for isdigit("٣") (U+0663 ARABIC-INDIC DIGIT THREE): | ||
| # openjd-rs: false (ASCII-only digit test) | ||
| # Python openjd CLI: false (agrees — ASCII-only) | ||
| # CPython str.isdigit: True (Unicode Nd/No digit test — the host | ||
| # language differs from both) | ||
| # | ||
| # The expected: block below asserts the Unicode reading as a strawman, | ||
| # matching §2.2.5's Unicode posture — i.e. the OPPOSITE of the current | ||
| # de facto agreement. If the spec ratifies ASCII-only (the likely outcome | ||
| # given both implementations agree), flip the expectation instead of | ||
| # promoting as-is. Do NOT move this fixture into jobs/ until the spec | ||
| # defines "digit" (and likewise "alphabetic"/"alphanumeric"/"whitespace" | ||
| # for the sibling predicates). | ||
| template: | ||
| specificationVersion: jobtemplate-2023-09 | ||
| extensions: | ||
| - EXPR | ||
| name: TestJob | ||
| steps: | ||
| - name: Step1 | ||
| script: | ||
| actions: | ||
| onRun: | ||
| command: python | ||
| args: | ||
| - -c | ||
| - | | ||
| print(r'ISDIGIT_ARABIC:{{ isdigit("٣") }}') | ||
| print(r'ISDIGIT_DEVANAGARI:{{ isdigit("३") }}') | ||
| expected: | ||
| output: | ||
| - ISDIGIT_ARABIC:true | ||
| - ISDIGIT_DEVANAGARI:true |
45 changes: 45 additions & 0 deletions
45
conformance-tests/2023-09/EXPR/jobs/proposed/expr2.2.6--repr-py-newline-roundtrip.test.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # PARKED: implementation bug found by the repr_py round-trip battery. | ||
| # | ||
| # Expression Language §2.2.6: repr_py "follows the behavior of Python's repr". | ||
| # Python's repr("a\nb") is 'a\nb' (backslash-n ESCAPED, 7 characters, one line). | ||
| # BOTH implementations (openjd-rs and the Python CLI — 2026-08-12 sweep) emit | ||
| # a RAW newline inside the single quotes — which is not a valid | ||
| # Python string literal at all: | ||
| # | ||
| # ast.literal_eval on the emitted output raises | ||
| # "SyntaxError: unterminated string literal (detected at line 1)" | ||
| # | ||
| # So the output cannot be embedded in a generated Python script, which is the | ||
| # entire purpose of repr_py. Backslash, quote, and tab handling round-trip | ||
| # correctly (tab is emitted raw rather than as \t, which deviates from repr() | ||
| # but still parses back byte-identical); newline is the case that produces | ||
| # invalid output. Classification: implementation bug in BOTH implementations | ||
| # (spec is | ||
| # explicit via the repr() reference). This fixture is the regression test to | ||
| # move into jobs/ once fixed. | ||
| template: | ||
| specificationVersion: jobtemplate-2023-09 | ||
| extensions: | ||
| - EXPR | ||
| name: TestJob | ||
| steps: | ||
| - name: Step1 | ||
| let: | ||
| - 'q = repr_py("a\nb")' | ||
| script: | ||
| actions: | ||
| onRun: | ||
| command: python | ||
| args: | ||
| - -c | ||
| - | | ||
| import ast | ||
| import sys | ||
| got = ast.literal_eval(sys.argv[1]) | ||
| print("NEWLINE:" + ("PASS" if got == "a\nb" else "FAIL got=" + repr(got))) | ||
| - "{{ q }}" | ||
| expected: | ||
| output: | ||
| - NEWLINE:PASS | ||
| forbidden: | ||
| - FAIL |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quorum verdict: GOOD (5/5, one stale comment) — spec: EL §2.2.6 (repr_py follows Python repr; the spec's own example escapes \n) — genuinely mandated, and the README's 'bug in BOTH implementations, dual-run verified' matches the sweep. The ast.literal_eval round-trip asserts semantics rather than byte-exact escaping — well built for promotion. Fix: the fixture's inline comment still says 'bug in openjd-rs' — update to both.