Monitor progress of selected tasks, without including their dependencies

[benjaminhwilliams]

When using a progress bar or widget, one usually wants to show all the tasks from a given computation. However, I would like to be able to monitor only specific tasks, such as the tasks belonging to just one layer, without incorporating all the tasks on which they depend.

In my use case, I have a task graph consisting of several layers, most of which are quick and cheap to compute, and a couple of layers performing slower operations. I can isolate some chosen tasks easily, for example

*_, top_layer = collection.dask.layers.values()
futures = top_layer.values()

but if I want to show their progress with a progress bar or widget, all their dependencies get included automatically, since distributed.diagnostics.progress.Progress always calls <ibid>.dependent_keys.

From looking at the code and documentation, I haven't been able to see a way to monitor just the progress of specified tasks. Have I missed something? Or would this be a welcome enhancement? If so, I have a branch ready to go, as a starting point, and I'd be happy to open a PR.

Example

All the layers of my task graph are quick and easy to compute, except for the much slower last layer. When the computation begins, the workers quickly consume lots of the cheap tasks until they all become preoccupied with the rate-limiting tasks. At this point, the graph progress looks like this

and the progress bar looks like this

[##########                              ] | 26% Completed |  12.4s

It then hovers at 26% for a long time, until the first few of the slow tasks finish. This progress bar is misleading to the user. Very little meaningful progress has been made, as the number of completed tasks is a poor proxy for the computation duration in this case. It would be nice just to show the progress of the rate-limiting last layer.