Implement parallel executor service without ForkJoinPool

[marcphilipp]

This PR introduces a new implementation of HierarchicalTestExecutorService that runs rests in parallel and has limited work stealing capabilities but is not based on ForkJoinPool. It avoids its pitfalls (such as #3945) and #3108 but may require additional threads because its work stealing is limited to direct children. Contrary to the ForkJoinPool implementation, the new executor service guarantees that no more than parallelism test nodes are executed in parallel.

My intention is to initially ship this implementation as an opt-in feature (via the new junit.jupiter.execution.parallel.executor configuration parameter) in 6.1, make it an opt-out feature in 6.2, and drop support for the ForkJoinPool-based implementation in a later to-be-determined release.

The PR is not yet finished but feedback is already welcome! If you use parallel test execution in your projects (or other test engines), it would be great if you could try out the new implementation and report your observations.

Resolves #3108.

---

I hereby agree to the terms of the JUnit Contributor License Agreement.

---

Definition of Done

• There are no TODOs left in the code
• Method preconditions are checked and documented in the method's Javadoc
• Coding conventions (e.g. for logging) have been followed
• Change is covered by automated tests including corner cases, errors, and exception handling
• Public API has Javadoc and @API annotations
• Change is documented in the User Guide and Release Notes

[marcphilipp]

I changed how to opt-in to the feature and have updated the docs.

@mpkorstanje I'd appreciate your feedback! 🙂

[marcphilipp]

Suggested change

	+ "' to hide this message.");
	+ "' to hide this message and use the original implementation.");

[mpkorstanje]

Suggested change

	* New `WorkerThreadPoolHierarchicalTestExecutorService` implementation of parallel test
	* New `WorkerThreadPoolHierarchicalTestExecutorService` implementation used for parallel test

[mpkorstanje]

This reads like a flow of consciousness.

I'd suggest rewriting it to <objective> <action> format.

		This implementation is based on a regular thread pool and a work queue shared among all worker threads.

		Each worker thread scans the shared work queue for tasks to run. Since the tasks represent hierarchically
		structured tests, container tasks will call `submit(TestTask)` or `invokeAll(List<TestTask>)` for their
		children, recursively.
		
		To maintain the desired parallelism -- regardless whether the user code performs any blocking operations --
		a fixed number of worker leases is configured. Whenever a task is submitted to the work queue to be executed
		concurrently, an attempt is made to acquire a worker lease. If a worker lease was acquired, a worker thread is
		started. Each worker thread attempts to "steal" queue entries for its children and execute them itself prior to waiting
		for its children to finish.
		 
		To optimize CPU utilization, whenever a worker thread does need to block, it temporarily gives up its worker
		lease and attempts to start another worker thread to compensate for the reduced `parallelism`. If the max pool size does 
		not permit starting another thread, the attempt is ignored in case there are still other active worker threads.
		
		The same happens in case a resource lock needs to be acquired.
		
		To minimize the number of idle workers, worker threads will prefer to steal top level tasks, while working
		through their own task hierarchy in a depth first fashion. Furthermore, child tasks with execution mode
		`CONCURRENT` are submitted to the shared queue be prior to executing those with execution mode `SAME_THREAD`
		directly.

[mpkorstanje]

I'm having some second thoughts about this. This is an exhaustible resource, especially compared to ConcurrentSkipListSet. Not that I think any one could reasonably exhaust it, but those are famous last words.

[mpkorstanje]

I've had a look at replacing the ConcurrentSkipListSet with a PriorityBlockingQueue. Because we depend on both queue and iteration order, the PriorityBlockingQueue with its "messy" iteration order can't be used because it is not guaranteed that the executing thread and work stealer will approach each other from opposite directions..

Resolved the potential exhaustion in 1f9ffde by using the UniqueId to create an arbitrary ordering. This however meant that invokeAll order wasn't used any more. 592a05e solves that.