How can you associate metadata with a submitted task in the task_stream using the futures interface?

[dask-user]

We are submitting a task like so:

client.submit(func, *args, **kwargs, key='key1')

However, we'd like to associate metadata with this task, such as the user who submitted the task. I see that TaskState has a metadata field, and I see that client.get_task_stream() has an empty metadata dict returned for each task, so I'm assuming this is possible. I've found client.set_metadata(), so I can do this:

client.set_metadata(['key1', 'user'], getpass.getuser())

and afterwards I can run client.get_metadata('key1') to get the metadata as a dict. But this metadata seems to be independent of the task_stream. When I call client.get_task_stream() the metadata field for those tasks are still empty dicts.

[mrocklin]

You might want to try Dask annotations

with dask.annotate(user="matthew"):
    future = client.submit(func, args)

These annotations aren't easily queryable though. They end up on the TaskState but most user-visible functions don't inspect them.

[dask-user]

nice, that seems to work in conjunctions with client.run_on_scheduler. Thanks.