Running tests in parallel

[mixedCase]

Is your feature request related to a problem? Please describe.

I'd like to parallelize the execution of most of my tests in order to speed up their execution.

To be more specific I've a series of tests that heavily bottleneck on IO not only during execution but also in their fixtures. While an asynchronous IO runtime on the Java-side would be best I don't think that's going to be viable for a few years at least, so having multiple threads to nanny the syscalls is the next best thing and the only alternative to projects who are actually bottlenecking on CPU instead.

Describe the solution you'd like

To be able to set metadata to specific deftests as well as ns declarations (which should be equivalent to marking all of its deftests as parallel), to indicate that these tests can become part of a grand pool of tests that can be executed in parallel by a group of workers before the synchronized tests run.

Nice to haves:

• Being able to specify a custom worker cap as a system property, with a sensible default like the system's virtual core count.
• Buffered output when running Poly CLI for extra clarity, meaning results would only be printed once all tests for a given namespace are done in case they're all parallel, or waiting until the beginning of the namespace's synchronous execution phase to print the results of its parallel tests right before executing the synchronous ones. Failures could be shown when they occur and releasing the buffer for its namespace then and there, stopping execution just like poly test currently does for synchronous tests.

Describe alternatives you've considered

• Plug-in API for custom test runners. -- Feels overkill, and likely to introduce more complexity than having a single test runner with even the superset of all popular test runner features.
• Testing-specific CLI alternative to poly. -- It would likely lag behind Polylith if maintained out of tree and cause a lot more of maintenance effort.

[svdo]

Maybe only slightly related, but I'd also love to easily be able to use kaocha for example.

[furkan3ayraktar]

@mixedCase This should be solved by #196. Once that PR ready and merged, you could write your own custom test runner to
parallelize the tests.

@svdo There will be a custom koacha test runner that @imrekoszo is developing. Once the custom test runner support is there, you can plug it in.

[svdo]

Awesome! 😎

[imrekoszo]

With parallel execution support coming to kaocha and #196 coming to Polylith, this should be a possibility soon.

[tengstrand]

Hi guys !

Now when it's possible to run tests using Kaocha in the latest 0.2.15-alpha release, can we close this issue?
@mixedCase @imrekoszo @svdo

[mixedCase]

I believe so. Can't test myself as I don't find myself working in Clojure anymore but I think that does it. Closing.