TRCLI cannot update results to testrail website - custom_step_results is not a valid boolean

[davidbaschung]

TestRail CLI Version

Any version beyond 1.5.0

CLI Environment

Microsoft Windows 10 Professionnel, pip 24.1.1, python 3.11

TestRail Version

TestRail v8.0.4 Default (7036)

TestRail Instance Type

Professional Cloud

Current behavior

PS C:\Users\david\Documents\programmation\SwissTXT\live-results-center> trcli -v -y -c "trcli-config.yml" parse_junit -f "test-results/junit-report.xml"
>> 
TestRail CLI v1.9.5

. . . (etc. , more CLI rows)

Test run: https://swisstxt.testrail.net/index.php?/runs/view/1122
Adding results: 0/3
**** API Call
method: POST
url: https://swisstxt.testrail.net/index.php?/api/v2/add_results_for_cases/1122
payload: {'results': [{'case_id': 18536, 'status_id': 1, 'comment': '', 'elapsed': '9s', 'attachments': [], 'custom_step_results': []}, {'case_id': 18535, 'status_id': 1, 'comment': '', 'elapsed': '11s', 'attachments': [], 'custom_step_results': []}, {'case_id': 18534, 'status_id': 1, 'comment': '', 'elapsed': '9s', 'attachments': [], 'custom_step_results': []}]}
response status code: 400
response body: {'error': 'Field :custom_step_results is not a valid boolean.'}
****

. . . (etc. , more CLI rows, then an error happens and the upload to TestRail's website fails)
The complete log is available in this file below :
request on V1.9.5 - error.txt

Desired behavior

PS C:\Users\david\Documents\programmation\SwissTXT\live-results-center> trcli -v -y -c "trcli-config.yml" parse_junit -f "test-results/junit-report.xml"
>> 
TestRail CLI v1.5.0

. . . (etc. , more CLI rows)

Run created: https://swisstxt.testrail.net/index.php?/runs/view/1134
Adding results: 0/3
**** API Call
method: POST
url: https://swisstxt.testrail.net/index.php?/api/v2/add_results_for_cases/1134
payload: {'results': [{'case_id': 18538, 'status_id': 1, 'comment': '', 'elapsed': '9s', 'attachments': []}, {'case_id': 18537, 'status_id': 1, 'comment': '', 'elapsed': '11s', 'attachments': []}, {'case_id': 18539, 'status_id': 1, 'comment': '', 'elapsed': '9s', 'attachments': []}]}
response status code: 200
response body: [{'id': 9507, 'test_id': 199762, 'status_id': 1, 'created_on': 1721133096, 'assignedto_id': None, 'comment': None, 'version': None, 'elapsed': '9s', 'defects': None, 'created_by': 1, 'custom_testrail_bdd_scenario_results': None, 'custom_step_results': False, 'attachment_ids': []}, {'id': 9508, 'test_id': 199761, 'status_id': 1, 'created_on': 1721133096, 'assignedto_id': None, 'comment': None, 'version': None, 'elapsed': '11s', 'defects': None, 'created_by': 1, 'custom_testrail_bdd_scenario_results': None, 'custom_step_results': False, 'attachment_ids': []}, {'id': 9509, 'test_id': 199763, 'status_id': 1, 'created_on': 1721133096, 'assignedto_id': None, 'comment': None, 'version': None, 'elapsed': '9s', 'defects': None, 'created_by': 1, 'custom_testrail_bdd_scenario_results': None, 'custom_step_results': False, 'attachment_ids': []}]
****
Adding results: 3/3, Done.
No attachments found to upload.
Submitted 3 test results in 4.9 secs.
PS C:\Users\david\Documents\programmation\SwissTXT\live-results-center> 

(end of the CLI logs)
The complete log is available in this file below :
request on V1.5.0 - success.txt

More Details

Any TestRail version beyond 1.5.0 will fail to upload the junit-report.xml file to testrail.

The bug cause is visible in the logs above, in the payload of the POST requests to the endpoint containing api/v2/add_results_for_cases/ in its URL and ending with an ID.

Apparently, the older versions of the TRCLI were sending the value of custom_step_results as a boolean. Since the version 1.6.0, the TRCLI is sending the value as an empty array. This causes an error for the newer versions of the TRCLI, which still use the same endpoint, but without success.

(Also I noticed some variables named custom_custom_step_results but I'm unsure it has anything to do with this)

[acuanico-tr-galt]

We are evaluating/investigating the ticket.

[acuanico-tr-galt]

Hello @davidbaschung can you provide a sample file you're using when uploading tests? Thank you.