-
Notifications
You must be signed in to change notification settings - Fork 35
test: Add 25 WRAP_ACTIONS conformance fixtures for coverage gaps #157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: mainline
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD (5/5) — spec: §4.3.1 / RFC 0008 scope rule ( |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # RFC 0008 Template variables: "Templates MUST NOT reference | ||
| # WrappedAction.* outside the three wrap hooks. Schedulers MUST reject | ||
| # templates that violate this scope rule." Here the out-of-scope reference | ||
| # is buried inside an EXPR function call — repr_py(WrappedAction.Command) | ||
| # in the env's own onEnter — not written as a bare {{WrappedAction.X}} | ||
| # format string. Catches a validator that only scope-checks direct | ||
| # variable interpolations and does not walk expression arguments (the | ||
| # same blind spot as the historical Task.File.Run.name bug, which behaved | ||
| # differently inside a function call than as a direct reference). Single | ||
| # defect: the template is valid except for the WrappedAction.Command | ||
| # reference inside the function call in onEnter. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: WrapActionInFuncall | ||
| script: | ||
| actions: | ||
| onEnter: | ||
| command: python | ||
| args: ["-c", "print({{repr_py(WrappedAction.Command)}})"] | ||
| onWrapEnvEnter: | ||
| command: python | ||
| args: ["-c", "print({{repr_py(WrappedAction.Command)}})"] | ||
| onWrapTaskRun: | ||
| command: python | ||
| args: ["-c", "print({{repr_py(WrappedAction.Command)}})"] | ||
| onWrapEnvExit: | ||
| command: python | ||
| args: ["-c", "print({{repr_py(WrappedAction.Command)}})"] |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD-WITH-NITS (3 reviewers flagged the same nit) — spec: §4.3.1 scope rule + §3.6.2 let-scope table. Fixture is invalid under any reading, but the header's "single defect = onEnter's use of the binding; the hooks' use is fine" framing conflicts with §3.6.2: |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # RFC 0008 Template variables: "Templates MUST NOT reference | ||
| # WrappedAction.* outside the three wrap hooks. Schedulers MUST reject | ||
| # templates that violate this scope rule." Here the out-of-scope reference | ||
| # is buried inside a script-level `let` binding. Per Template Schemas | ||
| # 3.6.2, <EnvironmentScript>.let's symbol scope does not include | ||
| # WrappedAction.* at all — the binding itself is the violation (script- | ||
| # level let is evaluated once at environment entry and shared by ALL of | ||
| # the script's actions, including onEnter/onExit where WrappedAction.* | ||
| # never exists). Catches a validator that scope-checks format strings in | ||
| # action fields but never walks let-binding expressions (the | ||
| # `let`-indirection blind spot). The template is valid except for the | ||
| # WrappedAction-derived binding and its consumption. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: WrapActionInLet | ||
| script: | ||
| let: | ||
| - wrapped_cmd = WrappedAction.Command | ||
| actions: | ||
| onEnter: | ||
| command: python | ||
| args: ["-c", "print({{repr_py(wrapped_cmd)}})"] | ||
| onWrapEnvEnter: | ||
| command: python | ||
| args: ["-c", "print({{repr_py(wrapped_cmd)}})"] | ||
| onWrapTaskRun: | ||
| command: python | ||
| args: ["-c", "print({{repr_py(wrapped_cmd)}})"] | ||
| onWrapEnvExit: | ||
| command: python | ||
| args: ["-c", "print({{repr_py(wrapped_cmd)}})"] |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD-WITH-NITS — spec: §4.3.1 ( |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # RFC 0008 Template variables: WrappedEnv.Name is available only in | ||
| # onWrapEnvEnter and onWrapEnvExit. "Templates MUST NOT reference | ||
| # WrappedEnv.* outside onWrapEnvEnter and onWrapEnvExit. Schedulers MUST | ||
| # reject templates that violate this scope rule." Referencing it in | ||
| # onWrapTaskRun — a wrap hook, but the wrong one — must be rejected at | ||
| # validation time. Catches a validator that treats all three hooks as one | ||
| # scope. Single defect: the template is valid except for the WrappedEnv.Name | ||
| # reference in onWrapTaskRun. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: WrapEnvNameInTaskRun | ||
| script: | ||
| actions: | ||
| onWrapEnvEnter: | ||
| command: python | ||
| args: ["-c", "print(r'{{WrappedEnv.Name}}')"] | ||
| onWrapTaskRun: | ||
| command: python | ||
| args: ["-c", "print(r'{{WrappedEnv.Name}}')"] | ||
| onWrapEnvExit: | ||
| command: python | ||
| args: ["-c", "print(r'{{WrappedEnv.Name}}')"] |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD-WITH-NITS — spec: §4.3.1 ( |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # RFC 0008 Template variables: WrappedStep.Name is available only in | ||
| # onWrapTaskRun. "Templates MUST NOT reference ... WrappedStep.* outside | ||
| # onWrapTaskRun. Schedulers MUST reject templates that violate this scope | ||
| # rule." Referencing it in onWrapEnvEnter must be rejected at validation | ||
| # time — an environment's lifecycle action has no associated step. Catches | ||
| # a validator that injects WrappedStep.* into all three hook scopes. | ||
| # Single defect: the template is valid except for the WrappedStep.Name | ||
| # reference in onWrapEnvEnter. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: WrapStepNameInEnter | ||
| script: | ||
| actions: | ||
| onWrapEnvEnter: | ||
| command: python | ||
| args: ["-c", "print(r'{{WrappedStep.Name}}')"] | ||
| onWrapTaskRun: | ||
| command: python | ||
| args: ["-c", "print(r'{{WrappedStep.Name}}')"] | ||
| onWrapEnvExit: | ||
| command: python | ||
| args: ["-c", "print(r'{{WrappedEnv.Name}}')"] |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD-WITH-NITS — spec: §4.3.1. Correct twin of the enter-hook fixture; coverage reviewer notes it's a near-duplicate (same validator scope-table path) — cheap enough to keep, but the pair likely catches the same bugs. Same repr_py nit. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # RFC 0008 Template variables: WrappedStep.Name is available only in | ||
| # onWrapTaskRun. Referencing it in onWrapEnvExit must be rejected at | ||
| # validation time. The exit hook is the twin of the enter hook, but a | ||
| # validator can plausibly scope the two env hooks differently (exit hooks | ||
| # fire during teardown when a "current step" may exist in the runtime's | ||
| # state), so both directions are pinned. Single defect: the template is | ||
| # valid except for the WrappedStep.Name reference in onWrapEnvExit. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: WrapStepNameInExit | ||
| script: | ||
| actions: | ||
| onWrapEnvEnter: | ||
| command: python | ||
| args: ["-c", "print(r'{{WrappedEnv.Name}}')"] | ||
| onWrapTaskRun: | ||
| command: python | ||
| args: ["-c", "print(r'{{WrappedStep.Name}}')"] | ||
| onWrapEnvExit: | ||
| command: python | ||
| args: ["-c", "print(r'{{WrappedStep.Name}}')"] |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD-WITH-NITS — spec: §5 (timeout fmtstring @fb1) + §5.3 (mode/notifyPeriod fmtstring @fb1), accept-side twin of two mainline reject fixtures — a real gap filled. Nit: it sits in WRAP_ACTIONS/env_templates but additionally requires FEATURE_BUNDLE_1, and the runner does not gate by declared extensions — an implementation with WRAP_ACTIONS+EXPR but no FB1 falsely fails this "valid" fixture. Consider noting the extra requirement or relocating. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # RFC 0008 Cancelation behavior + Template Schemas 5/5.3 (FEATURE_BUNDLE_1): | ||
| # the positive twin of 4.2--wrap-fmtstring-timeout-no-feature-bundle and | ||
| # wrap-cancelation-fmtstring-mode-no-feature-bundle. A format-string | ||
| # `timeout` and a format-string `cancelation.mode` on a wrap hook are the | ||
| # FEATURE_BUNDLE_1-gated forms of the round-trip forwarding pattern; with | ||
| # the extension listed, validation MUST ACCEPT them. Catches a validator | ||
| # that rejects the format-string forms unconditionally ("format strings in | ||
| # timeout are not allowed") instead of gating them on the extension — the | ||
| # reject-side fixtures alone cannot distinguish "gated" from "never | ||
| # permitted". | ||
| # | ||
| # NOTE: unlike its WRAP_ACTIONS siblings, this valid fixture additionally | ||
| # requires FEATURE_BUNDLE_1 support. The runner does not gate fixtures by | ||
| # declared extensions, so an implementation with WRAP_ACTIONS+EXPR but no | ||
| # FEATURE_BUNDLE_1 will (correctly, per its own capabilities) reject this | ||
| # template and fail the fixture. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| - FEATURE_BUNDLE_1 | ||
| environment: | ||
| name: WrapFmtStringGatedAccepted | ||
| script: | ||
| actions: | ||
| onWrapEnvEnter: | ||
| command: echo | ||
| args: ["{{WrappedEnv.Name}}"] | ||
| timeout: "{{WrappedAction.Timeout}}" | ||
| onWrapTaskRun: | ||
| command: echo | ||
| args: ["{{WrappedAction.Command}}"] | ||
| timeout: "{{WrappedAction.Timeout}}" | ||
| cancelation: | ||
| mode: "{{WrappedAction.Cancelation.Mode}}" | ||
| notifyPeriodInSeconds: "{{WrappedAction.Cancelation.NotifyPeriodInSeconds}}" | ||
| onWrapEnvExit: | ||
| command: echo | ||
| args: ["{{WrappedEnv.Name}}"] | ||
| timeout: "{{WrappedAction.Timeout}}" |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD (5/5) — spec: §4.3 constraint 1 (all-or-nothing). Minimal, schema-valid but for the missing onWrapTaskRun; rejection can only come from the named rule. Completes the invalid matrix with its three siblings (mainline had only 2 of 6 cells). |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # RFC 0008 all-or-nothing rule: defining onWrapEnvEnter and onWrapEnvExit | ||
| # without onWrapTaskRun is invalid. This is the "env lifecycle only" subset | ||
| # an author writes when they only want to intercept environment setup and | ||
| # teardown — plausible, but rejected: the rule guarantees execution context | ||
| # parity across every wrapped lifecycle phase, tasks included. Single | ||
| # defect: the template is valid except for the missing onWrapTaskRun. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: EnterExitNoTaskRun | ||
| script: | ||
| actions: | ||
| onWrapEnvEnter: | ||
| command: echo | ||
| args: ["{{WrappedEnv.Name}}"] | ||
| onWrapEnvExit: | ||
| command: echo | ||
| args: ["{{WrappedEnv.Name}}"] |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD (5/5) — spec: §4.3 constraint 1 (all-or-nothing), enter-only subset. Clean single defect. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # RFC 0008 all-or-nothing rule: an environment that defines any wrap hook | ||
| # must define all three (onWrapEnvEnter, onWrapTaskRun, onWrapEnvExit). | ||
| # Defining only onWrapEnvEnter is invalid. Catches a validator that only | ||
| # checks for the presence of onWrapTaskRun when deciding whether the | ||
| # all-or-nothing rule applies. Single defect: the template is valid except | ||
| # for the missing onWrapTaskRun and onWrapEnvExit hooks. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: OnlyWrapEnter | ||
| script: | ||
| actions: | ||
| onWrapEnvEnter: | ||
| command: echo | ||
| args: ["{{WrappedEnv.Name}}"] |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD (5/5) — spec: §4.3 constraint 1, exit-only subset. Coverage reviewer rates it the weakest matrix cell (siblings almost certainly catch the same validators) but completing the matrix is cheap and guards asymmetric validators. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # RFC 0008 all-or-nothing rule: an environment that defines any wrap hook | ||
| # must define all three (onWrapEnvEnter, onWrapTaskRun, onWrapEnvExit). | ||
| # Defining only onWrapEnvExit is invalid. Catches a validator that treats | ||
| # the exit hook as optional teardown rather than part of the atomic hook | ||
| # set. Single defect: the template is valid except for the missing | ||
| # onWrapEnvEnter and onWrapTaskRun hooks. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: OnlyWrapExit | ||
| script: | ||
| actions: | ||
| onWrapEnvExit: | ||
| command: echo | ||
| args: ["{{WrappedEnv.Name}}"] |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD (5/5) — spec: §4.3 constraint 1, missing-enter direction. All refs in legal hooks; only the missing onWrapEnvEnter can trigger rejection in a correct implementation. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # RFC 0008 all-or-nothing rule: defining onWrapTaskRun and onWrapEnvExit | ||
| # without onWrapEnvEnter is invalid. Catches a validator that only demands | ||
| # the hooks needed for the phases it has seen so far (tasks and teardown) | ||
| # instead of the full atomic set. Single defect: the template is valid | ||
| # except for the missing onWrapEnvEnter. | ||
| specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: RunExitNoEnter | ||
| script: | ||
| actions: | ||
| onWrapTaskRun: | ||
| command: echo | ||
| args: ["{{WrappedAction.Command}}"] | ||
| onWrapEnvExit: | ||
| command: echo | ||
| args: ["{{WrappedEnv.Name}}"] |
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quorum verdict: GOOD-WITH-NITS — spec: RFC 0008 stdout forwarding (runtime scans the wrap script's stdout only, never the grand-child's stream directly). Effect-based negative control, correctly paired with the excludes-host-vars sibling. Nit (service-compat reviewer): the forbidden literal |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # RFC 0008 "Stdout forwarding and macro propagation" (negative control): | ||
| # runtimes MUST scan the WRAP SCRIPT's stdout for macros and "MUST NOT | ||
| # scan the grand-child's stdout directly". If the wrap script discards the | ||
| # grand-child's stdout (here: redirected to DEVNULL), a macro the | ||
| # grand-child emitted never reaches the wrap script's stdout, so the | ||
| # runtime must never see it and the variable must NOT surface in | ||
| # WrappedAction.Environment. Catches a runtime that locates and scans the | ||
| # grand-child process's output stream directly — the design the RFC | ||
| # explicitly rejected ("would require locating the grand-child across | ||
| # heterogeneous runtimes"). | ||
| template: | ||
| specificationVersion: jobtemplate-2023-09 | ||
| name: WrapDiscardedGrandChildStdoutLosesMacro | ||
| jobEnvironments: | ||
| - name: InnerEnv | ||
| script: | ||
| actions: | ||
| onEnter: | ||
| command: python | ||
| args: ["-c", "print('openjd_env: LOST_VAR=should-not-surface')"] | ||
| steps: | ||
| - name: Step1 | ||
| script: | ||
| actions: | ||
| onRun: | ||
| command: echo | ||
| args: ["placeholder"] | ||
| environments: | ||
| - specificationVersion: environment-2023-09 | ||
| extensions: | ||
| - WRAP_ACTIONS | ||
| - EXPR | ||
| environment: | ||
| name: WrapEnv | ||
| script: | ||
| actions: | ||
| # Run the wrapped onEnter but discard its stdout: its macro line | ||
| # must be lost. | ||
| onWrapEnvEnter: | ||
| command: python | ||
| args: | ||
| - "-c" | ||
| - "import subprocess,sys; sys.exit(subprocess.run([{{repr_py(WrappedAction.Command)}}]+{{repr_py(WrappedAction.Args)}}, stdout=subprocess.DEVNULL).returncode)" | ||
| onWrapTaskRun: | ||
| command: python | ||
| args: | ||
| - "-c" | ||
| - "for e in {{repr_py(WrappedAction.Environment)}}:\n print('WAENV='+e)\nprint('WAENV_DUMP_DONE')" | ||
| onWrapEnvExit: | ||
| command: python | ||
| args: | ||
| - "-c" | ||
| - "import subprocess,sys; sys.exit(subprocess.run([{{repr_py(WrappedAction.Command)}}]+{{repr_py(WrappedAction.Args)}}).returncode)" | ||
| expected: | ||
| output: | ||
| - WAENV_DUMP_DONE | ||
| # Only the WAENV= form is forbidden: the raw macro string | ||
| # "LOST_VAR=should-not-surface" also occurs inside the resolved wrap-hook | ||
| # command line (via repr_py(WrappedAction.Args)), so forbidding it would | ||
| # false-fail any runner that echoes resolved commands into the log. | ||
| forbidden: | ||
| - WAENV=LOST_VAR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quorum finding: NOT-GOOD as-is — the added tree diagram documents
proposed/containingwrap-repr-py-escapes-newline-in-wrapped-args.test.yamland aproposed/README.md, but neither exists on this branch (they live on PR #162's branch). Either note the dependency on #162 or drop those lines; the 24 real fixture entries in the README are complete and correctly grouped.