Upload test results to xray rest API

[DianeMornas]

Context:
Create a simple cli to upload the test results to xray plugin

• should work with parameterized tests and simple tests
• uploading the junit xml report works only with Generic xray ticket type but does not work with Manual, Automated, Automated Cucumber ticket types. Use dictionary or json format for the test results instead of junit xml

previous implementation 0.30.0
2 steps
1 - run the pykiso tests and generate the junit xml containing the test results
2 - post the test results to the xray REST endpoint /api/v2/import/execution/junit

POC proposal
1 - run the pykiso tests and generate the ALL_STEP_REPORT ordered dictionary, then upload the test results to the xray REST endpoint /api/v2/import/execution

After discussions:

• Run a pykiso test with the xray decorator and generate the junit xml. Convert the junit to json. Upload the json to xray
• Move all xray code into xray folder and write the unit tests related to xray. Restore test_execution.py, assert_step_report.py and clip.py

doc: https://docs.getxray.app/display/XRAYCLOUD/Import+Execution+Results+-+REST+v2#ImportExecutionResultsRESTv2-JUnitXMLresultsMultipart

[sebastianpfischer]

@DianeMornas , any update about the parsing after the test-execution?

[DianeMornas]

@sebastianpfischer , I re-worked on the POC.

• I moved the xray code into the tool/xray folder.
• Generate the HTM report supports the parameterized tests and the failure logs
• I added a pykiso option flag "--save-step-report" to serialize the test result reports in a pickle file
• I added methods to serialize and deserialize the test results report
• If the user would like to send the test result to xray, there are 2 steps
  1. run the test with the CLI: pykiso -c .\examples\xray.yaml --step-report xray_report.html --save-step-report
  2. run the CLI to deserialize the pickle files and upload the test results to xray: poetry run xray --user $Env:XRAY_CLIENT_ID --password $Env:XRAY_CLIENT_SECRET --url $Env:XRAY_API_BASE_URL upload --path-results-pickle "./"

In a future ticket, I plan to parse and filter the information from the junit xml to create the dictionary to send to xray REST endpoint.

[sebastianpfischer]

@DianeMornas , let's separate the PRs in two. One for adding the step functionality, the other to add the xray functionality.

I also do not think this is a good usecase to use pickle. I would expect a better encoding method toi be chosen.

So what are todays limitation for the second method with junit?

[DianeMornas]

@sebastianpfischer , some news, now we have 2 PRs:

• Feature/ html step report supports parameterized tests #542 to support parameterized tests within the HTML step report (regarding the html step report, I agree, if needed the global logic should be refactored)
• Feature/parameterized tests with xray poc #535 to be able with a CLI and junit xml file(s) to upload the test results to xray

I deleted the POC with pickle, now we are able to parse the junit xml file directly and collect the info to build a dictionary to be sent to xray. If in the test results we have parameterized test (a test key is used several time in the same test function) we create one test execution ticket per test cases in xray. If we have not parameterized tests, we create only one test execution ticket with all the test results.

[DianeMornas]

Hi @sebastianpfischer, after discussions with @Pog3k , I reworked on this PR.
The goal of this PR was to improve the HTML step report by handling the parameterized tests. Two changes have been implemented:

• fix: on master branch when the test is parameterized the test description is missing (see report_junit_param_master.png) -> add test description even if the test is parameterized (see report_junit_param.png)
• improvement: add the failure log column to display the traceback when the test fails (see report_junit_param.png)