rename test_need as test_ is a reserved prefix in pytest by AlexanderLanin · Pull Request #156 · eclipse-score/docs-as-code

AlexanderLanin · 2025-07-22T21:40:09Z

We want to include test results into docs build. Step 1 is generating a test results xml file.

State:

proof of concept
concept review
refactoring, cleaning up

Current Output:

<?xml version='1.0' encoding='utf-8'?>
<testsuites>
    <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="4" time="0.028"
        timestamp="2025-07-22T22:10:49.476848+00:00" hostname="HI4-C-0004X">
        <testcase classname="" name="test_run_incremental" time="0.001" />
        <testcase classname="" name="test_build_incremental_and_exec_it" time="0.000" />
        <testcase classname="" name="test_esbonio_old" time="0.000" />
        <testcase classname="" name="test3" time="0.000" />
    </testsuite>
    <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="27" time="2.198"
        timestamp="2025-07-22T22:13:14.556392+00:00" hostname="HI4-C-0004X">
        <testcase classname="" name="test_get_cache_filename" time="0.001" />
        <testcase classname="" name="test_find_need_direct_match" time="0.000" />
        <testcase classname="" name="test_find_need_with_prefix" time="0.000" />
        <testcase classname="" name="test_find_need_multiple_prefixes" time="0.000" />
        <testcase classname="" name="test_find_need_not_found" time="0.000" />
        <testcase classname="" name="test_group_by_need" time="0.000" />
        <testcase classname="" name="test_group_by_need_empty_list" time="0.000" />
        <testcase classname="" name="test_parse_git_output_ssh_format" time="0.000" />
        <testcase classname="" name="test_parse_git_output_https_format" time="0.000" />
        <testcase classname="" name="test_parse_git_output_ssh_format_without_git_suffix"
            time="0.000" />
        <testcase classname="" name="test_parse_git_output_invalid_format" time="0.000" />
        <testcase classname="" name="test_parse_git_output_empty_string" time="0.000" />
        <testcase classname="" name="test_get_github_repo_info_ssh_remote" time="0.027" />
        <testcase classname="" name="test_get_github_repo_info_https_remote" time="0.013" />
        <testcase classname="" name="test_get_github_repo_info_multiple_remotes" time="0.015" />
        <testcase classname="" name="test_get_current_git_hash" time="0.013" />
        <testcase classname="" name="test_get_current_git_hash_invalid_repo" time="0.002" />
        <testcase classname="" name="test_get_github_link_with_real_repo" time="0.018" />
        <testcase classname="" name="test_cache_file_operations" time="0.001" />
        <testcase classname="" name="test_cache_file_with_encoded_comments" time="0.001" />
        <testcase classname="" name="test_group_by_need_and_find_need_integration" time="0.000" />
        <testcase classname="" name="test_end_to_end_with_real_files" time="0.028" />
        <testcase classname="" name="test_multiple_commits_hash_consistency" time="0.022" />
        <testcase classname="" name="test_git_operations_with_no_commits" time="0.008" />
        <testcase classname="" name="test_git_repo_with_no_remotes" time="0.012" />
        <testcase classname="" name="test_source_link_integration_ok" time="1.194" />
        <testcase classname="" name="test_source_link_integration_non_existent_id" time="0.147" />
    </testsuite>
    <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="19" time="2.801"
        timestamp="2025-07-22T22:13:04.917186+00:00" hostname="HI4-C-0004X">
        <properties>
            <property name="Verifies" value="['GD__automatic_document_header_generation']" />
            <property name="Description"
                value="It should check the generation of the header information." />
            <property name="ASIL" value="ASIL_D" />
            <property name="Priority" value="1" />
            <property name="TestType" value="Requirements-based test" />
            <property name="DerivationTechnique" value="Analysis of requirements" />
            <property name="Verifies" value="['GD__automatic_document_header_generation']" />
            <property name="Description"
                value="It should check the generation of the header information." />
            <property name="ASIL" value="ASIL_D" />
            <property name="Priority" value="1" />
            <property name="TestType" value="Requirements-based test" />
            <property name="DerivationTechnique" value="Analysis of requirements" />
        </properties>
        <testcase classname="" name="test_register" time="0.006" />
        <testcase classname="" name="test_generate_hash" time="0.000" />
        <testcase classname="" name="test_request_from_directive_github_data" time="0.001" />
        <testcase classname="" name="test_request_from_directive_commit_data" time="0.001" />
        <testcase classname="" name="test_debug" time="0.001" />
        <testcase classname="" name="test_extract_merge_commit_data" time="0.001" />
        <testcase classname="" name="test_extract_merge_commit_data_error" time="0.002" />
        <testcase classname="" name="test_extract_github_data" time="0.002" />
        <testcase classname="" name="test_extract_github_data_exception" time="0.000" />
        <testcase classname="" name="test_extract_org" time="0.000" />
        <testcase classname="" name="test_extract_repo" time="0.000" />
        <testcase classname="" name="test_extract_github_token" time="0.000" />
        <testcase classname="" name="test_extract_pull_request" time="0.000" />
        <testcase classname="" name="test_extract_team_info" time="0.001" />
        <testcase classname="" name="test_extract_approvers" time="0.001" />
        <testcase classname="" name="test_append_approver_teams" time="0.000" />
        <testcase classname="" name="test_extract_reviewers" time="0.001" />
        <testcase classname="" name="test_header_service_integration_github_data" time="1.331" />
        <testcase classname="" name="test_header_service_integration_commit_data" time="0.120" />
    </testsuite>
    <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="23" time="12.741"
        timestamp="2025-07-22T22:13:14.611133+00:00" hostname="HI4-C-0004X">
        <testcase classname=".TestCheckOptions" name="test_unknown_directive" time="0.002" />
        <testcase classname=".TestCheckOptions" name="test_unknown_directive_extra_option"
            time="0.001" />
        <testcase classname=".TestCheckOptions" name="test_missing_mandatory_options_info"
            time="0.001" />
        <testcase classname=".TestCheckOptions" name="test_invalid_option_type" time="0.001" />
        <testcase classname=".TestCheckOptions"
            name="test_unknown_option_present_in_neither_req_opt_neither_opt_opt" time="0.002" />
        <testcase classname="" name="test_rst_files[graph/test_metamodel_graph.rst]" time="2.494" />
        <testcase classname="" name="test_rst_files[options/test_options_options.rst]" time="1.285" />
        <testcase classname="" name="test_rst_files[options/test_options_extra_option.rst]"
            time="1.117" />
        <testcase classname=""
            name="test_rst_files[attributes/test_attributes_format_id_length.rst]" time="1.180" />
        <testcase classname="" name="test_rst_files[attributes/test_attributes_external_prefix.rst]"
            time="1.154" />
        <testcase classname=""
            name="test_rst_files[attributes/test_attributes_format_id_format.rst]" time="1.255" />
        <testcase classname="" name="test_rst_files[attributes/test_attributes_format_title.rst]"
            time="1.197" />
        <testcase classname=""
            name="test_rst_files[attributes/test_attributes_format_description.rst]" time="1.109" />
        <testcase classname=""
            name="test_rst_files[id_contains_feature/test_id_contains_feature.rst]" time="1.189" />
        <testcase classname=".TestStandards" name="test_my_pie_linked_standard_requirements"
            time="0.000" />
        <testcase classname=".TestStandards" name="test_my_pie_linked_standard_workproducts"
            time="0.000" />
        <testcase classname=".TestStandards"
            name="test_my_pie_workproducts_contained_in_exactly_one_workflow" time="0.000" />
        <testcase classname=".TestStandards" name="test_get_standards_needs" time="0.000" />
        <testcase classname=".TestStandards" name="test_get_standards_workproducts" time="0.000" />
        <testcase classname=".TestStandards" name="test_get_compliance_req_needs" time="0.000" />
        <testcase classname=".TestStandards" name="test_get_compliance_wp_needs" time="0.000" />
        <testcase classname=".TestStandards" name="test_get_workflows" time="0.000" />
        <testcase classname=".TestStandards" name="test_get_workproducts" time="0.000" />
    </testsuite>
</testsuites>

github-actions · 2025-07-22T21:41:19Z

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //src:license-check

Status: ⚠️ Needs Review

Click to expand output

[License Check Output]
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 97fca3d5-2b9e-42f5-9bb5-183c071c7815
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: src
Loading: 0 packages loaded
    currently loading: src
Analyzing: target //src:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //src:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //src:license-check (96 packages loaded, 10 targets configured)

Analyzing: target //src:license-check (104 packages loaded, 10 targets configured)

Analyzing: target //src:license-check (104 packages loaded, 10 targets configured)

Analyzing: target //src:license-check (148 packages loaded, 2002 targets configured)

Analyzing: target //src:license-check (153 packages loaded, 2652 targets configured)

Analyzing: target //src:license-check (158 packages loaded, 2687 targets configured)

INFO: Analyzed target //src:license-check (163 packages loaded, 4946 targets configured).
[1 / 1] no actions running
INFO: Found 1 target...
Target //src:license.check.license_check up-to-date:
  bazel-bin/src/license.check.license_check
  bazel-bin/src/license.check.license_check.jar
INFO: Elapsed time: 20.006s, Critical Path: 0.79s
INFO: 13 processes: 3 disk cache hit, 9 internal, 1 processwrapper-sandbox.
INFO: Build completed successfully, 13 total actions
INFO: Running command line: bazel-bin/src/license.check.license_check src/formatted.txt -review -project automotive.score -repo https://github.com/eclipse-score/docs-as-code -token otyhZ4eaRYK1tKLNNF-Y
[main] INFO Querying Eclipse Foundation for license data for 84 items.
[main] INFO Found 58 items.
[main] INFO Querying ClearlyDefined for license data for 26 items.
[main] ERROR Error response from ClearlyDefined 429

github-actions · 2025-07-22T21:43:26Z

The created documentation from the pull request is available at: docu-html

MaximilianSoerenPollak · 2025-07-23T07:46:33Z

I think this makes sense overall from the approach.

We should have the properties (LineNR, File, etc.) in the testcase xml line. So like:

<testcase classname="" name="test_name" time="0.001" file="src/extensions/tests/test_file.py" lineNr=70 reqs="feat_req_1, feat_req_2" ... (whatever other things we need)/>

AlexanderLanin · 2025-07-23T08:06:46Z

I think this makes sense overall from the approach.

We should have the properties (LineNR, File, etc.) in the testcase xml line. So like:
<testcase classname="" name="test_name" time="0.001" file="src/extensions/tests/test_file.py" lineNr=70 reqs="feat_req_1, feat_req_2" ... (whatever other things we need)/>

Before adding any properties I simply wanted the xml file. Assuming this approach is correct, we'd need to move this to a generic place like tooling. That will take some days on it's own. Quite some effort just to use bazel caching :(

MaximilianSoerenPollak · 2025-07-23T08:28:36Z

I think this makes sense overall from the approach.
We should have the properties (LineNR, File, etc.) in the testcase xml line. So like:
<testcase classname="" name="test_name" time="0.001" file="src/extensions/tests/test_file.py" lineNr=70 reqs="feat_req_1, feat_req_2" ... (whatever other things we need)/>
Before adding any properties I simply wanted the xml file. Assuming this approach is correct, we'd need to move this to a generic place like tooling. That will take some days on it's own. Quite some effort just to use bazel caching :(

That does make sense. Not sure where in the end the best way is for this to live, but for now let's keep it here and go step by step.

MaximilianSoerenPollak · 2025-08-18T13:14:33Z

@AlexanderLanin with #197 merged, can we close this?

AlexanderLanin added 2 commits July 22, 2025 23:10

rename test_need as test_ is a reserved prefix in pytest

20a1c53

WIP

1a6aafe

github-project-automation Bot added this to INF - Infrastructure Community Jul 22, 2025

github-project-automation Bot moved this to Draft in INF - Infrastructure Community Jul 22, 2025

overwrite bazel xml file

23c2ec1

AlexanderLanin closed this Aug 22, 2025

github-project-automation Bot moved this from Draft to Done in INF - Infrastructure Community Aug 22, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rename test_need as test_ is a reserved prefix in pytest#156

rename test_need as test_ is a reserved prefix in pytest#156
AlexanderLanin wants to merge 3 commits into
mainfrom
xml

AlexanderLanin commented Jul 22, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 22, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 22, 2025

Uh oh!

MaximilianSoerenPollak commented Jul 23, 2025

Uh oh!

AlexanderLanin commented Jul 23, 2025

Uh oh!

MaximilianSoerenPollak commented Jul 23, 2025

Uh oh!

MaximilianSoerenPollak commented Aug 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

AlexanderLanin commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

License Check Results

Uh oh!

github-actions Bot commented Jul 22, 2025

Uh oh!

MaximilianSoerenPollak commented Jul 23, 2025

Uh oh!

AlexanderLanin commented Jul 23, 2025

Uh oh!

MaximilianSoerenPollak commented Jul 23, 2025

Uh oh!

MaximilianSoerenPollak commented Aug 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AlexanderLanin commented Jul 22, 2025 •

edited

Loading

github-actions Bot commented Jul 22, 2025 •

edited

Loading