You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks @xbcsmith - I agree we should standardise where possible however, different subjects may have different semantics: in case of test cases, failure refers to an assertion that is not verified, while error refers to an unexpected error, typically something that lies within the infrastructure required for the execution of the test.
In core.md we state that outcome is a
String
enumerated assuccess
,error
orfailure
.Examples where we differ are as follows:
In testing-events.md the outcome of the testSuite execution, one of
pass
,fail
,cancel
,error
conformance/testcaserun_finished.json: "outcome": "pass",
conformance/testsuiterun_finished.json: "outcome": "fail",
conformance/pipelinerun_finished.json: "outcome": "failure",
conformance/taskrun_finished.json: "outcome": "failure",
schemas/testcaserunfinished.json: "outcome": {"type": "string", "enum": [ "pass", "fail", "cancel", "error"]},
The outcome of this issue should be standardization on
success
,error
orfailure
as a typeString
for alloutcome
.The text was updated successfully, but these errors were encountered: