Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indicate test suite in reports #72

Open
solonovamax opened this issue Sep 20, 2024 · 0 comments
Open

Indicate test suite in reports #72

solonovamax opened this issue Sep 20, 2024 · 0 comments

Comments

@solonovamax
Copy link

Currently, when reporting the test results, the report will contain something like the following:

{
    "uuid":        "08caf457-234b-44fb-ab2c-7f1dfe8a85d9",
    "historyId":   "ca.solostudios.nyx.sonatype.SonatypePublishPluginTest/the sonatype publish plugin -- the class -- applying -- apply it",
    "testCaseId":  "ca.solostudios.nyx.sonatype.SonatypePublishPluginTest/the sonatype publish plugin -- the class -- applying -- apply it",
    "fullName":    "Feature: the sonatype publish plugin / Given: the class / Upon: applying / Should: apply it",
    "labels":      [
        { "name": "framework", "value": "kotest" },
        { "name": "host", "value": "solo-main" },
        { "name": "language", "value": "kotlin" },
        { "name": "testClass", "value": "SonatypePublishPluginTest" },
        { "name": "package", "value": "ca.solostudios.nyx.sonatype" },
        { "name": "suite", "value": "ca.solostudios.nyx.sonatype.SonatypePublishPluginTest" },
        { "name": "severity", "value": "normal" },
        { "name": "thread", "value": "[email protected] @coroutine#11(22)" }
    ],
    "links":       [ ],
    "name":        "Feature: the sonatype publish plugin Given: the class Upon: applying Should: apply it",
    "status":      "passed",
    "stage":       "finished",
    "steps":       [
        { "name": "ca.solostudios.nyx.sonatype.SonatypePublishPluginTest", "status": "passed", "steps": [ ], "attachments": [ ], "parameters": [ ], "start": 0, "stop": 0 },
        { "name": "the sonatype publish plugin", "status": "passed","steps": [ ], "attachments": [ ], "parameters": [ ], "start": 0, "stop": 0 },
        { "name": "the class", "status": "passed", "steps": [ ], "attachments": [ ], "parameters": [ ], "start": 0, "stop": 0 },
        { "name": "applying", "status": "passed", "steps": [ ], "attachments": [ ], "parameters": [ ], "start": 0, "stop": 0 }
    ],
    "attachments": [ ],
    "parameters":  [ ],
    "start":       1726633366631,
    "stop":        1726633366633
}

Here, you can see that the "suite" is marked as ca.solostudios.nyx.sonatype.SonatypePublishPluginTest.

However, this is not actually the suite.

Instead, I believe that the suite should either be the name of the source set, or the name of the task.

Let's say I have the following test source sets (this example is for a gradle plugin as it's what I'm working on, and I believe it should illustrate the most complex example)

  • functionalTest: functional/integration test suite
    test tasks:
    • functionalTest8.6: tests against gradle 8.6
    • functionalTest8.7: tests against gradle 8.7
    • ...
    • functionalTest8.10: tests against gradle 8.10
  • test: non-functional/integration test suite
    test tasks:
    • test

so, I believe that the following should be output instead:

{
    "uuid":        "08caf457-234b-44fb-ab2c-7f1dfe8a85d9",
    "historyId":   "ca.solostudios.nyx.sonatype.SonatypePublishPluginTest/the sonatype publish plugin -- the class -- applying -- apply it",
    "testCaseId":  "ca.solostudios.nyx.sonatype.SonatypePublishPluginTest/the sonatype publish plugin -- the class -- applying -- apply it",
    "fullName":    "Feature: the sonatype publish plugin / Given: the class / Upon: applying / Should: apply it",
    "labels":      [
        { "name": "framework", "value": "kotest" },
        { "name": "host", "value": "solo-main" },
        { "name": "language", "value": "kotlin" },
        { "name": "testClass", "value": "SonatypePublishPluginTest" },
        { "name": "package", "value": "ca.solostudios.nyx.sonatype" },
        { "name": "suite", "value": "test" },
        { "name": "severity", "value": "normal" },
        { "name": "thread", "value": "[email protected] @coroutine#11(22)" }
    ],
    "links":       [ ],
    "name":        "Feature: the sonatype publish plugin Given: the class Upon: applying Should: apply it",
    "status":      "passed",
    "stage":       "finished",
    "steps":       [
        { "name": "ca.solostudios.nyx.sonatype.SonatypePublishPluginTest", "status": "passed", "steps": [ ], "attachments": [ ], "parameters": [ ], "start": 0, "stop": 0 },
        { "name": "the sonatype publish plugin", "status": "passed","steps": [ ], "attachments": [ ], "parameters": [ ], "start": 0, "stop": 0 },
        { "name": "the class", "status": "passed", "steps": [ ], "attachments": [ ], "parameters": [ ], "start": 0, "stop": 0 },
        { "name": "applying", "status": "passed", "steps": [ ], "attachments": [ ], "parameters": [ ], "start": 0, "stop": 0 }
    ],
    "attachments": [ ],
    "parameters":  [ ],
    "start":       1726633366631,
    "stop":        1726633366633
}

here, the suite label is replaced with test. In the case of, say, the functionalTest8.6, it would be replaced with functionalTest8.6. However, alternatively, it could be replaced with functionalTest and

{
    "name": "Gradle Version",
    "value": "8.6"
}

could be added to parameters instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant