feat: add not supported status code

[dabrain34]

In case of 69 exit code from an app, the test should be reported as not supported and be considered as non-error status.

[dabrain34]

KhronosGroup/Vulkan-Video-Samples@0d69850

[dabrain34]

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/10088#note_3191662

[mdimopoulos]

Waiting for https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/10088#note_3191662 to conclude before proceeding with a proper review here

[ylatuya]

@dabrain34 I have opened #323 with a new runner that can be customized to handle errors more easily.

[dabrain34]

That's sound great, let me give it a try

[rsanchez87]

Do you agree to close this PR, @dabrain34 ? Have you seen the changes in this PR?

[dabrain34]

why do you want to close this MR as the idea would be to treat not supported differently from fail and not exit on error?

[rsanchez87]

why do you want to close this MR as the idea would be to treat not supported differently from fail and not exit on error?

You're right, sorry for the confusion. These two PRs are complementary, looking more in detail, this one handles the reporting side, #323 handles the runner side. We'll just need to coordinate the exit codes between them...

[rsanchez87]

Looks good to me, the full path from exit code 69 through to JUnit/CSV/markdown reports is consistent and well wired up.

[rsanchez87]

@dabrain34 , there are merge conflicts with master, looks like _generate_json_summary and _calculate_timeout_adjustment were added in the same area. NOT_SUPPORTED addition to the result_map in _generate_csv_summary plus the new methods from master.

Can you rebase?

[ylatuya]

We should support this in an agnostic way through the NotSupportedExpeception without binding it to the 69 exit code since each decoder implementation might signal this in different ways. As a matter of fact, the GStreamer decoders will not use 69 exit code but instead the runner should raise that exception so it can be handled in a generic way by the test case.

[rsanchez87]

#319 (comment)

[dabrain34]

We should support this in an agnostic way through the NotSupportedExpeception without binding it to the 69 exit code since each decoder implementation might signal this in different ways. As a matter of fact, the GStreamer decoders will not use 69 exit code but instead the runner should raise that exception so it can be handled in a generic way by the test case.

sounds good to me

[dabrain34]

@rsanchez87 rebased and modified to support NotSupported exception in Vulkan Video Samples decoder class

[dabrain34]

@rsanchez87 is there something missing in this PR ? I would need this PR to validate Vulkan VP9 test suite upon nvidia driver as they dont support resolution lower than 128x128

[rsanchez87]

@rsanchez87 is there something missing in this PR ? I would need this PR to validate Vulkan VP9 test suite upon nvidia driver as they dont support resolution lower than 128x128

Hi @dabrain34 , sorry and thanks for the ping. I haven’t had the chance to review this PR yet, it slipped through on my side. I’ll take a look as soon as possible and get back to you.

[rsanchez87]

@dabrain34 can you check the ruff failures here, after rebase from master? https://github.com/fluendo/fluster/actions/runs/24728494934/job/72337197798

change needed into deps:

import subprocess
from typing import Any, Dict, Optional

from fluster.codec import Codec, OutputFormat
from fluster.decoder import Decoder, register_decoder
from fluster.utils import file_checksum, run_command

# BSD sysexits.h EX_UNAVAILABLE — media not supported.
EX_UNAVAILABLE = 69

[rsanchez87]

LGTM, NotSupportedError is defined in the base decoder layer