XML Test parsing & testlinks#197
Conversation
License Check Results🚀 The license check job ran with the Bazel command: bazel run //src:license-checkStatus: ✅ Passed Click to expand output |
|
ToDO:
|
|
Any ideas for better names for the |
|
So I have been at work in this. One big issue I just stumbled accross, if I have the 'grouped cache' or just the testlink cache, I do not build the TestCaseNeeds again. |
| '<<meta_all(no_links=True, exclude=["layout","style", "fully_verified_by", "status", "security", "safety"])>>', | ||
| '<<meta_links_all(exclude=["fully_verified_by", "fully_verifies_back", "partially_verified_by", "partially_verifies_back"])>>', |
There was a problem hiding this comment.
outcommented partially & fully verifies_back. in the end render
| @@ -0,0 +1,113 @@ | |||
| [ | |||
There was a problem hiding this comment.
Golden files made for golden file assertion testing in integration tests
There was a problem hiding this comment.
-> expected_grouped_json?
There was a problem hiding this comment.
better question: do we really care about the json content? Shouldn't we test end to end?
| @@ -0,0 +1,126 @@ | |||
| import json | |||
There was a problem hiding this comment.
Simple Half written Half Gen tests. Good start but probably need some work.
will check coverage later
| @@ -0,0 +1,107 @@ | |||
| import json | |||
There was a problem hiding this comment.
Half written, half Generated. Probably needs more work
| "verify_type": "fully", | ||
| "result": "passed", | ||
| "result_text": "" | ||
| } |
There was a problem hiding this comment.
Notice that the 'skipped' test is missing here.
There was a problem hiding this comment.
Whatever is noteworthy should be in the code. Comments!
There was a problem hiding this comment.
Where do I put this? As I can't put it in the json :D?
The place where we write the testlink json?
| @@ -0,0 +1,105 @@ | |||
| # ******************************************************************************* | |||
There was a problem hiding this comment.
Newly created File to house the 'grouped' dataclass.
Please leave feedback for a better name (for the file & Dataclasses)
| # It's mandatory that the test either partially or fully verifies a requirement | ||
| # if self.PartiallyVerifies is None and self.FullyVerifies is None: | ||
| # raise ValueError( | ||
| # f"TestCase: {self.id} Error. Either 'PartiallyVerifies' or 'FullyVerifies' must be provided." | ||
| # ) | ||
| # Skipped tests should always have a reason associated with them | ||
| # if "skipped" in self.result.keys() and not list(self.result.values())[0]: | ||
| # raise ValueError( | ||
| # f"TestCase: {self.id} Error. Test was skipped without provided reason, reason is mandatory for skipped tests." | ||
| # ) |
There was a problem hiding this comment.
Commented out for now, to not break all tests due to them not having properties.
There was a problem hiding this comment.
We need to support gradual integration. Noone will be able to update docs-as-code if it suddenly throws 100000 errors.
| def build_test_needs_from_files( | ||
| app: Sphinx, env: BuildEnvironment, xml_paths: list[Path] | ||
| ) -> list[TestCaseNeed]: | ||
| """ | ||
| Reading in all test.xml files, and building 'testcase' external need objects out of them. | ||
|
|
||
| Returns: | ||
| - list[TestCaseNeed] | ||
| """ | ||
| tcns: list[TestCaseNeed] = [] | ||
| for f in xml_paths: | ||
| b, z = read_test_xml_file(f) | ||
| for non_prop_test in z: | ||
| logger.warning( | ||
| f"Test: {non_prop_test} has no properties. Could not create need" | ||
| ) | ||
| # Now we build the needs from it | ||
| tcns.extend(b) | ||
| for c in b: | ||
| construct_need(app, c, env) | ||
| return tcns |
There was a problem hiding this comment.
I think I can solve this better, just didn't have a good idea yet.
|
/consumer-test |
| Tool Requirements | ||
| ================================= | ||
|
|
||
| TESTCASE EXAMPLES |
There was a problem hiding this comment.
Remove
| @@ -0,0 +1,557 @@ | |||
| # ******************************************************************************* | |||
There was a problem hiding this comment.
This was 'test_source_link` before.
Not sure why git didn't realize the renaming?
| # Cleanup | ||
| # Don't know if we need this? | ||
| # os.environ.pop("BUILD_WORKSPACE_DIRECTORY", None) | ||
|
|
There was a problem hiding this comment.
Delete
| from src.extensions.score_source_code_linker.testlink import DataOfTestCase | ||
|
|
||
|
|
||
| # Unsure if I should make these last a session or not |
There was a problem hiding this comment.
Change this to use hardcoded XML string files instead of this. (in next PR)
| # Outcomment this to enable more verbose logging | ||
| # LOGGER.setLevel("DEBUG") | ||
| # Uncomment this to enable more verbose logging | ||
| LOGGER.setLevel("DEBUG") |
There was a problem hiding this comment.
Should comment this out again
| for prefix in prefixes: | ||
| prefixed_id = f"{prefix}{id}" | ||
| if prefixed_id in all_needs: | ||
| LOGGER.warning("linking to external needs is not supported!") |
There was a problem hiding this comment.
Should we do more here?
| # LOGGER.warning( | ||
| # f"Putting links into need: {need['id']}. SCL: {source_code_links.links.CodeLinks}\nTESTLINKS: {source_code_links.links.TestLinks}" |
There was a problem hiding this comment.
Delete this
|
Followup PR's :
Not sure what else I'm missing for now. |
MaximilianSoerenPollak
merged commit 3cee6b0
into
eclipse-score:main
github-project-automation
Bot
moved this from Draft
to Done
in INF - Infrastructure Community
* Add Test.xml parsing * Add testlink's to requirements (where found) * Added unit- & integration-tests * Splitted helper library to avoid circular imports * Remove junitparser Thanks for the infra Team for help in working out the concept & finding the issues here.
2 participants
No description provided.