Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions crates/openjd-cli/tests/cli_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -789,6 +789,45 @@ mod redacted_env {
}
}

// ============================================================
// Group 13: Self-asserting tasks (openjd-specifications conformance shape)
// ============================================================

/// The conformance suite's single-task job fixtures assert their own output: the
/// `onRun` action spawns the case's command as a child, reproduces its output, and
/// exits non-zero when that output does not match. That only holds if we capture a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The grandchild-output property this module says it pins is not actually exercised.

In self_asserting_task.yaml the action script runs subprocess.run(COMMAND, capture_output=True, ...) and then re-writes completed.stdout / completed.stderr to its own stdout/stderr. The grandchild's pipes are read and closed by the intermediate Python process — openjd only ever sees the direct child's fds. So this test would still pass if openjd dropped grandchild output entirely; it only re-verifies direct-child capture, which is already covered elsewhere in this file.

To actually pin the stated invariant, the grandchild has to inherit the action's stdout/stderr rather than have them captured, e.g. drop capture_output=True and check the returncode instead:

completed = subprocess.run(COMMAND)  # grandchild inherits our stdout/stderr

and then assert on openjd's captured stdout from the Rust side. As written, the second test (test_assertion_failure_fails_the_run) also largely duplicates the existing test_task_fails_still_exits_env exit-status coverage.

/// grandchild's output and propagate the action's exit status, so pin both
/// directions here instead of relying on the external suite to catch a regression.
mod self_asserting_task {
use super::*;

#[test]
fn test_grandchild_output_captured_and_assertion_passes() {
let template = templates_dir().join("self_asserting_task.yaml");
let (code, stdout, stderr) = run_cli(&["run", template.to_str().unwrap()]);
assert_eq!(code, 0, "should succeed. stderr: {stderr}");
// Printed by the grandchild and echoed by the action. Missing means output
// from a process we did not spawn ourselves was dropped.
assert!(stdout.contains("OUTPUT:EXPECTED_VALUE"), "stdout: {stdout}");
assert!(!stdout.contains("ASSERT_FAILED"), "stdout: {stdout}");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion can never fire. The fixture writes ASSERT_FAILED to stderr (sys.stderr.write("ASSERT_FAILED: ...")), and the failure path is only reachable when the script exits 1 — which the preceding assert_eq!(code, 0) already rules out. The companion test correctly searches format!("{stdout}{stderr}"); this one should do the same (or just drop the check as redundant).

}

#[test]
fn test_assertion_failure_fails_the_run() {
let template = templates_dir().join("self_asserting_task.yaml");
let (code, stdout, stderr) = run_cli(&[
"run",
template.to_str().unwrap(),
"-p",
"Printed=WRONG_VALUE",
]);
let output = format!("{stdout}{stderr}");
assert_ne!(code, 0, "should fail. stdout: {stdout} stderr: {stderr}");
assert!(output.contains("ASSERT_FAILED"), "output: {output}");
assert!(output.contains("OUTPUT:WRONG_VALUE"), "output: {output}");
}
}

// ============================================================
// Group 6: Run Command (test_run_command.py)
// ============================================================
Expand Down
40 changes: 40 additions & 0 deletions crates/openjd-cli/tests/templates/self_asserting_task.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
specificationVersion: "jobtemplate-2023-09"
name: Self Asserting Task
description: >
An onRun action that spawns its own child process, reproduces that child's
output, and turns a comparison of it into the task's exit status. This is the
shape the openjd-specifications conformance suite uses for its single-task job
fixtures, so losing grandchild output or dropping the action's exit status
silently weakens that whole suite.

parameterDefinitions:
- name: Printed
type: STRING
default: "EXPECTED_VALUE"

steps:
- name: Assert
script:
actions:
onRun:
command: python
args: ["{{Task.File.Assert}}"]
embeddedFiles:
- name: Assert
type: TEXT
data: |
import subprocess
import sys

COMMAND = [sys.executable, "-c", "print(r'OUTPUT:{{Param.Printed}}')"]
EXPECTED = ["OUTPUT:EXPECTED_VALUE"]

completed = subprocess.run(COMMAND, capture_output=True, text=True)
sys.stdout.write(completed.stdout)
sys.stderr.write(completed.stderr)
output = completed.stdout + completed.stderr

missing = [line for line in EXPECTED if line not in output]
for _ in missing:
sys.stderr.write("ASSERT_FAILED: expected output line not found\n")
sys.exit(1 if missing else 0)
Loading