Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardizing outcome type and values in doc, schema, and examples #237

Open
xbcsmith opened this issue Jul 8, 2024 · 1 comment
Open

Comments

@xbcsmith
Copy link

xbcsmith commented Jul 8, 2024

In core.md we state that outcome is a String enumerated as success, error or failure.

Examples where we differ are as follows:

In testing-events.md the outcome of the testSuite execution, one of pass, fail, cancel, error

conformance/testcaserun_finished.json: "outcome": "pass",
conformance/testsuiterun_finished.json: "outcome": "fail",
conformance/pipelinerun_finished.json: "outcome": "failure",
conformance/taskrun_finished.json: "outcome": "failure",

schemas/testcaserunfinished.json: "outcome": {"type": "string", "enum": [ "pass", "fail", "cancel", "error"]},

The outcome of this issue should be standardization on success, error or failure as a type String for all outcome.

@afrittoli
Copy link
Contributor

Thanks @xbcsmith - I agree we should standardise where possible however, different subjects may have different semantics: in case of test cases, failure refers to an assertion that is not verified, while error refers to an unexpected error, typically something that lies within the infrastructure required for the execution of the test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: No status
Development

No branches or pull requests

2 participants