comprehensive solution for scheduler affinity

[ericniebler]

See the PR that reimplements scheduler affinity for unifex::task: facebookexperimental/libunifex#527.

It's possible that this work is necessary in order to write the std::task paper (see #36)

[inbal2l]

Jessica will get back to us with more information (thank you!), will be discussed then.

[lewissbaker]

We looked at the libunifex code which contains a with_scheduler_affinity algorithm/customisation-point and the task coroutine type applies this to every co_await of a sender, passing through the coroutine's current scheduler.

However, there didn't seem to be any usage/customisation of this with_scheduler_affinity algorithm in other algorithms and there wasn't any documentation on it that we could find and so it was difficult to understand the semantics.

Note that there is also something that fills a similar purpose in folly::coro called co_viaIfAsync, which takes a "semi-awaitable" and an executor and produces an awaitable. The caller is required to start this operation on the associated context and the operation is required to either complete synchronously or schedule its completion onto the specified executor.

I assume we will probable want something similar for a std::task with scheduler affinity.

[lewissbaker]

As a follow up, I found a brief description of with_scheduler_affinity from libunifex in a PR description:

This diff changes how unifex::task<> implements Scheduler affinity to work more like folly::coro::Task<>. What Folly calls co_viaIfAsync, this diff calls with_scheduler_affinity, which is a new CPO.

with_scheduler_affinity is a Sender algorithm that maps the given Sender to another Sender that must meet new postconditions if the Receiver that it's connected to meets certain preconditions. The new preconditions are:

• the eventual Receiver must provide a "current Scheduler";
• the result of with_scheduler_affinity must be started on the Receiver's current Scheduler; and
• stop requests delivered to the new Sender must be delivered on the Receiver's current Scheduler.

The Sender returned from with_scheduler_affinity must complete on its Receiver's current Scheduler (i.e. must complete where it was started) so long as all the above preconditions are met.

with_scheduler_affinity has two default implementations for a Sender type S:

• if sender_traits<S>::is_always_scheduler_affine is true, the default is the identity;
• if sender_traits<S>::is_always_scheduler_affine is false, the default is, essentially, a typed_via back to the correct scheduler.

Any Sender can customize with_scheduler_affinity, regardless of its is_always_scheduler_affine property.