Skip to content

tests: fix binary test passing when failed and llvm-cov deprecated flag #165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
artiepoole merged 4 commits into from
Mar 30, 2026
+23 −4
Merged

tests: fix binary test passing when failed and llvm-cov deprecated flag #165

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7111cc1
tests: coverage - fix llvm-cov deprecated --export-prefix
artiepoole Mar 27, 2026
d0ce10a
tests: workflows: fix binary test passing even if the script errored
artiepoole Mar 27, 2026
5dcbd95
tests: binary - the TF job did not catch the error from _run it seems
artiepoole Mar 27, 2026
d2f512f
tests: coverage-test default RUSTFLAGS to empty in export call
artiepoole Mar 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions .github/testflinger-assets/test_binary_device_script.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env bash

set -x
set -euxo pipefail

echo "INFO: Installing dependencies"
sudo env DEBIAN_FRONTEND=noninteractive apt update && sudo env DEBIAN_FRONTEND=noninteractive apt install build-essential -y
Expand All @@ -18,5 +18,15 @@ cd fpgad
mkdir -p artifacts
sudo journalctl -f -n0 > artifacts/journal.log 2>&1 &
JOURNAL_PID=$!

# Temporarily disable -e to capture the test exit status even on failure
set +e
./tests/coverage_test.sh 2>&1 | tee artifacts/coverage_test.log
sudo kill ${JOURNAL_PID} || true
TEST_EXIT_STATUS=${PIPESTATUS[0]}
set -e

# Cleanup regardless of test result
sudo kill ${JOURNAL_PID} || true

# Exit with the test's exit status
exit $TEST_EXIT_STATUS
9 changes: 9 additions & 0 deletions .github/testflinger-assets/testflinger_job.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ test_data:
- agent: device_script.sh
local: ./device_script.sh
test_cmds: |
set -e
# get hw-cert-team helper tools
echo "INFO: setting up scriptlets"
export TOOLS_PATH=tools
Expand All @@ -47,8 +48,16 @@ test_data:

echo "INFO: Running script on DUT"
_run 'chmod +x /home/ubuntu/device_script.sh'

# Run the test script, but always retrieve artifacts even if it fails
set +e
_run '/home/ubuntu/device_script.sh'
TEST_SCRIPT_EXIT=$?
set -e

echo "INFO: Retrieving artifacts"
mkdir -p artifacts
_get fpgad/artifacts/* artifacts

# Fail the job if the test script failed
exit $TEST_SCRIPT_EXIT
4 changes: 2 additions & 2 deletions tests/coverage_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@
set -xeu

# Enable llvm-coverage build results
eval "$(cargo llvm-cov show-env --export-prefix)"
eval "$(cargo llvm-cov show-env --sh)"

export RUSTFLAGS="$RUSTFLAGS -C llvm-args=-runtime-counter-relocation"
export RUSTFLAGS="${RUSTFLAGS:-} -C llvm-args=-runtime-counter-relocation"

# build the daemon only, to avoid getting coverage for cli (no tests written)
cargo build --bin daemon
Expand Down
Loading