A way to hook before/after job execution (and/or a way to see a number of tasks in the processing queue)

[nagisa]

An application I'm working on that uses rayon has gotten to a state where, if the CPU is constantly utilized to the max, the memory keeps growing in an unbounded manner. We have troubleshooted this down to the fact that we keep putting all sorts of auxiliary tasks to said queue and the computer is simply unable to keep up with the inflow of those tasks.

I would like to make sure this sort of an issue does not require another extended investigation. In order to achieve this, I would like to add a counter metric or some other exposition of the current rayon's queue depth.

At the most basic level a simple accessor function would be good enough, but I think it might be not be particularly convenient to replicate this information to, say, prometheus metrics.

A much more convenient option would be if the pools provided an ability to register some callbacks similar to the builder methods that allow tweaking how the threads are spawned.

For example in my case:

1. A callback that's fired whenever a job is pushed into the queue;
2. A callback when a job is popped and/or whenever pool finishes executing code for a job;
3. One could imagine further additional such callbacks (e.g. whenever a job execution begins...)