Add better support for customization of runners

[basnijholt]

(original issue on GitLab)

opened by Joseph Weston (@jbweston) at 2018-06-11T08:35:40.607Z

We have several examples of adaptive usage where people need some functionality that sits between the Learner and the Runner.

Examples are

• DataSaver: The learned function returns a bunch of information, and we want to learn only a part of this information (e.g. a dict is returned, and we want only to learn the values with key result), but also keep the full output, for future analysis
• Timer: We want to time the execution of the learned function
• Cache: We want to cache some information produced by function calls, to use it in other function calls

The fact that gitlab:!71 is necessary indicates that we need to think about the way that this 'middleware' is implemented. It may be that making (semi-)transparent wrappers for Learners is the right design, but we should make it clear that this is what we are doing, in this case.

At the moment runners execute (more or less) the following in a loops: x, _ = ask(); tell(x, f(x)) The idea with a middleware is that you'd place a pair of functions, g and h, in between f: x, _ = ask(); tell(x, g(f(h(x))). The question would then be what the exact signatures of g and h need to be (e.g. in the previous example g is not passed enough information to be able to associate the value it receives with a corresponding x value), and whether they are better modelled by a class, a generator, or whatever.

[basnijholt]

originally posted by Anton Akhmerov (@anton-akhmerov) at 2018-06-11T10:02:47.420Z on GitLab

Why do you consider this runner customization though? I can well imagine a learner that takes into account the time it takes to compute a value to determine which points bring most benefit per unit of time.

[basnijholt]

originally posted by Anton Akhmerov (@anton-akhmerov) at 2018-06-11T10:05:09.451Z on GitLab

Also, just to make it clear: I fully agree with the need to support this kind of extra functionality better.

[basnijholt]

originally posted by Anton Akhmerov (@anton-akhmerov) at 2018-06-11T10:09:29.999Z on GitLab

Hm, along the lines of gitlab:!71: we could have a single LearnerWrapper class that:

• Explicitly stores an instance of another learner
• Stores g
• Provides its wrapped methods for ask and tell.

[basnijholt]

originally posted by Joseph Weston (@jbweston) at 2018-06-11T10:57:26.524Z on GitLab

Why do you consider this runner customization though?

Because the learner requires no knowledge of the middlewares, but the runner has to know to apply them (even if it does not know what each one does).

I can well imagine a learner that takes into account the time it takes to compute a value to determine which points bring most benefit per unit of time.

It is not clear to me that such a thing can be implemented independently of a given learner, so I am not sure this issue is the place to discuss this.

---

The examples I gave above are independent of the Learner, and they just need cooperation from the Runner. Ideally we should be able to compose arbitrary combinations of such things. Maybe it would be useful to collect a list of all the awkward use-cases we have com across so far and see whether this idea of composable middlewares can solve at least some of them.

[basnijholt]

originally posted by Joseph Weston (@jbweston) at 2018-06-11T11:00:08.096Z on GitLab

Hm, along the lines of gitlab:!71: we could have a single LearnerWrapper class that:

• Explicitly stores an instance of another learner
• Stores g
• Provides its wrapped methods for ask and tell.

But the best user experience would be if the wrapper was (almost) transparent. This involves forwarding all methods (except the ones we want to override) to the wrapped learner. As I said above, this may end up being the best way to compose things, but it seems like we're doing more work than we need to, somehow.

[basnijholt]

originally posted by Joseph Weston (@jbweston) at 2018-06-20T17:51:10.307Z on GitLab

Possibly related is the question of how to best abstract the "strategy" that a runner should use.

At the moment there is only a single strategy implemented. In pseudo code:

xs = ask(N)
submit(xs)
while not done:
    x, y = wait(1)
    tell(x, y)
    x = ask(1)
    submit(x)

This strategy is non-deterministic, and is not optimal for the case where function evaluations take ~ the same amount of time.

An alternative would be the batch strategy:

xs = ask(N)
submit(xs)
while not done:
    xs, ys = wait(N)
    tell(xs, ys)
    xs = ask(N)
    submit(xs)

This is deterministic and does not make excessive calls to ask.

Another alternative would be a mixed strategy:

xs = ask(N)
submit(xs)
while not done:
    xs, ys = wait_until(timeout=1)  # wait until at least 1 future is finished and 1 second has elapsed, or until all futures are done
    tell(xs, ys)
    xs = ask(len(xs))
    submit(xs)

There are probably other useful strategies.

---

The way to do this at the moment is to create a new runner and implement _run, however there are a lot of details that an implementer has to take care
of, which are irrelevant as far as the strategy is concerned. In addition we would have to implement a Synchronous and asynchronous version for each strategy,
which is really annoying.

It seems that we should be able to separate the concerns of the strategy (the core part of the runner) from the gluing code.