Implement reproducible seeding

[marcharper]

This PR

• adds an instanceable random number generator object RandomGenerator, and a bulk generating version BulkRandomGenerator
• makes Player, EvolvablePlayer, Match, Tournament, MoranProcess, and Fingerprint objects seed reproducible; updates every strategy to use a local RNG
• eliminates global seeding throughout the library, but does preserve a module-level seeding operation (mostly for a few tests, maybe can be eliminated)
• moves a few functions around (play from Player to Match, for example), see Possibly move play method away from Player #1181
• re-writes many tests and adds several others; upgraded our use of hypothesis to 4.0+ while doing so (closes Hypothesis deprecation warnings #1214)
• fixes Setting a seed is ineffective when parallelising #1277

The solution to #1277 is quite a bit more complex than discussed on the issue, so let me explain it all now so that the code changes will make sense.

The initial issue was the lack of exact reproducibility of multi-threaded tournaments. Here the crux of the issue is that once we start using threads, a global (seeded) random number generator (RNG) isn't necessarily going to generate values in the same order for all the matches across threads because threads may get pre-empted and so on. The solution discussed at #1277 is to generate seeds in the tournament class from a single initial seed and pass those into the threads so that they can be used to make the output of Matches reproducible.

There are a lot of ins and outs at this point because tournaments and matches can be noisy, metaplayers can contain subplayers, and in principle we might eventually have a Match that is itself multi-threaded, so how we make Matches seed-reproducible isn't so obvious. Since a Match can be noisy it needs access to a seeded RNG, as do the players, which currently call global RNG. So each stochastic player has to be re-worked to use a local or referenced RNG, which they could share with the Match or they could have their own.

If Players have their own RNG they also need to be seeded. That brings up the issue of how to handle resetting and cloning in the case of a seeded Player RNG, whether to initialize Players with a non-seeded RNG, and whether matches, when re-run, should reset all the seeds or not. However, given a seed, a Match can generate new seeds or RNGs for the Players and so maintain reproducibility, even if we don't reset any of the seeds in subsequent matches. That reduces the need to create a lot of RNG objects (and we need only generate them in the case that a Match is stochastic).

Finally note that whatever we do has to also work for axelrod-fortran. Those strategies require random values to be passed into the strategy function in each round, so those players also need to use a deterministic RNG.

Ultimately I decided on the following principles:

• Since a player must have an opponent, it is the Match that is the logical smallest context of reproducibility; all others derive reproducibility from it. Players are seedable, but since a Match also needs a seed for noise, the best practice is for matches to subsequently seed players in most cases.
• Players should probably not share RNGs with opponents or clones because it could cause subtle errors
• Cloning a player shouldn't reset its seed or RNG. Rather the clone should have no seed, and if reproducibility via a seed is desired, it's up to the Match or user to supply it
• Repeating an existing Match object need not necessary produce identical results to the last run. Like a player, its seed could be reset by the user if desired, otherwise the old seed can be used to generate a new seed. However creating a new match with identical parameters and the same seed should produce exactly the same sequence of match results.
• Note however that the way we run Tournaments by generating all the matches in __init__ means that rerunning a tournament will produce the same results. If we don't want that to be the case then we need to generate the matches on each invokation of play(). This could be changed...
• Note further that the order of matches in a tournament still isn't identical, so neither are the result sets because of the reordering, but the list of ranked strategies is the same.
• I decided it could be confusing for the user to be able to supply seeds to Players and Matches because then there has to be options for the Match to not overwrite seeds in that case, and I prefer one and only one obvious way to seed. In this case my preference is that a single seed gets reproducibility for a Tournament, Match, or Moran Process, and I couldn't think of a good use case where one would need to control all the seeds.
• A Player doesn't need a seed until it's time to play. There's not a compelling reason to generate a seed on initialization, for example, but the set_seed function can be used for testing purposes. The exception is the Evolvable players, which currently randomize on initialization (and can also now be seeded)
• The players' seeds don't belong in match_attributes because they are not the same for all players like the other match attributes

Then the following implementation details follow:

• Since players, matches, tests, etc need access to local-ish RNG, it was necessary to wrap the existing contents of random_.py into an object that can be seeded and instanced, removing the global RNG
• In the case where a Match is not supplied a seed, a random seed is used, mimicking the current behavior (with no guarantee of reproducibility)
• Matches then generate seeds to pass to the players who can then generate their own RNG
• It no longer makes sense for a Player to have a play function since a stochastic player or noisy Match must have a local RNG for reproducibility. The Player.play function was applying noise so I moved it to Match, and updated all uses of Player.play
• Player.__eq__ ignores the RNG / seed . This mimicks the current behavior where the global seed had to be set for a player and a clone to behave identically. Now a local seed has to be set for both (e.g. via the Match seed) for reproducibility to be valid.
• Seeds are set on players rather than in the __init__ kwargs, so the seed is not naturally reused on cloning or resetting. I toyed with the alternative -- it's possible to generate seeds on the fly with __new__ but it's hacky and leads to a lot of seeds that get overwritten or are unecessarily generated (e.g. for deterministic players), makes a bunch of tests harder to reason about, and so on.

Note that the order of players in a Match can affect the outcome (since the seeds generated by the Match would be reversed if the order was reversed). The other parameters like noise of course also must be the same.

Consequences

• There is now more locally stable testing with seeds, but new seeds have to be chosen for many tests, which are still broken. I'll fix them once we're happy with the rest of the changes
• In practice this actually simplifies of the use of RNG throughout the library
• Now we rely only on numpy seeding rather than numpy and stdlib
• We'll need to document this all clearly

Some things still to do:

• make axelrod-fortran players seedable (can reuse the new RNG objects)
• make evolvable players and evolutionary algorithms properly reproducible (also uses threading)

---

Fixes #1277
Fixes #1283
Fixes #1181
Fixes #1214

[drvinceknight]

Wow, thanks a lot for working on this @marcharper it looks like it was a chunk of work to get to the bottom of.

I've read your explanation/solution proposal and it all looks excellent to me. I don't think this only fixes #1277 but it also makes the whole seeding much nicer from a user point of view as well . Let me know if you'd like to share out some of the remaining work (or if I can help in any other way).

[marcharper]

I sat down thinking it wouldn't be that difficult and then it keep growing. I can handle the rest of the changes -- there are a few thorny cases to handle like making some of the transformers play nicely with deterministic (non-seeded players) but probably easier for me to tackle now that I'm pretty far in.

Do you have any opinions on the names of the new classes or variables? I used self._random for the Player local RNGs, but maybe just self._rng is better.

Perhaps you can take a look at the tests and look for more hypothesis opportunities or entirely new tests (maybe after this PR is finished).

The axelrod-fortran update should be easy, just need to reuse the new RNG classes. Folding in the evolutionary algorithms should happen soon after and then I think we're in good shape.

[drvinceknight]

Do you have any opinions on the names of the new classes or variables? I used self._random for the Player local RNGs, but maybe just self._rng is better.

I am happy with the names you got so far. Forgive me in advance if I make suggestions for changing one or two when I review more carefully but I don't think that'll happen and I doubt I'll feel strongly. In this specific case I think the only preference I'd have is self._random_number_generator but I don't feel strongly at all. Your call.

Perhaps you can take a look at the tests and look for more hypothesis opportunities or entirely new tests (maybe after this PR is finished).

Yeah good idea. I'll pick that up afterwards.

[gaffney2010]

Note however that the way we run Tournaments by generating all the matches in init means that rerunning a tournament will produce the same results. If we don't want that to be the case then we need to generate the matches on each invokation of play().

Is this so? The way I'm reading it, both _run_serial and _run_parallel start by building chunks using self.match_generator, which should have the seed in the state you left it at the previous run. Am I misunderstanding?

[marcharper]

I think you're right, we'd need to have match generator reuse its seed if we want the second run to match the prior run. But running two tournaments in row (e.g. calling play() twice) with different Tournament instances (and the same seed) should generate the same pair of results since the match generator is exhausted by Tournament before batching out matches.

[drvinceknight]

Making a note here to make sure we don't forget to modify the documentation if #1329 goes in before.

I'd recommend we still give hypothesis an upper bound as in the past they've developed things so quickly it's often broken things but at least the new cron CI builds would catch that.

[marcharper]

Update: Since this is taking a while I decided to go ahead and finish it rather than fold in the rest of the dojo and #1282. This branch was rebased and many tests fixed.

Files still containing failing tests:

test_moran
test_strategy_transformers
test_axelrod_second
test_calculator
test_cycler
test_gambler
test_hmm
test_memoryone
test_meta
test_mindcontrol
test_punisher
test_qlearner
test_selfsteem
test_stalker
test_titfortat
test_worse_and_worse
test_zero_determinant

[drvinceknight]

This is a big piece of work 👍 Let me know if I can assist in any way.

[peteroupc]

Let me give you another idea.

I am aware that the problem of thread schedulers and reproducible "randomness" has been tackled by using a different design from traditional PRNGs. Notably, so-called "splittable" PRNGs have been developed for use in "isolated parallel computations that may generate subtasks", such as event processing that could be done by individual threads. See also freeorion/freeorion#2856 (comment), the JAX PRNG Design Notes, and Java's SplittableRandom documentation.

[marcharper]

Hi @peteroupc thanks for the links. I'm a fan of Jax :)

IIUC what we're doing here is quite similar in that each task ends up with it's own PRNG (in fact each match and player, though they are all deterministically linked in a tree depending on the splitting path), so in this case I'm not sure that we gain much from using splittable PRNGs. Since we're going for 100% reproducibility, we'll still have the same issue in that we would need to split out all the PRNGs initially just as we're generating seeds here because we can't guarantee that the order of dispatch will be the same in the distributed case. Am I missing another advantage of using splittable PRNGs that's relevant?

[drvinceknight]

Suggested change

	rg = RandomGenerator(seed)
	r = rg.random()
	random = RandomGenerator(seed)
	r = random.random()

To be consistent with changes elsewhere?

I'd actually suggest rag = RandomerGenerator(seed) and using that elsewhere.

Not at all vital to change: whatever you prefer @marcharper.

[marcharper]

Sure. I've been using the RandomGenerator object as a drop-in replacement for the standard random module, but it's not quite the same.

[marcharper]

I changed it to rng for now. We can decide on a universal approach throughout pending your documentation effort.

[drvinceknight]

• I've gone through tests/unit: have some minor suggestions/questions.

I'm really enjoying reviewing this PR @marcharper: such a nice piece of work 👍

[drvinceknight]

That's all I'm going to have time to do today I'm afraid. I'll be getting back to this on Friday when I'll go over the implemented functionality itself (I might circle back to the tests/docs): really impressive work @marcharper :) 👍

[drvinceknight]

I've gone through everything now @marcharper. Impressive piece of work.

I wonder if writing up (I'm mainly thinking of a diagram) "how parallelisation works" in the docs somewhere (perhaps in the reference pages) could be a helpful resource. I also wonder if it could be beneficial if someone that's not you writes that and you review. I'd particularly like that person to be me 😆.

[Nikoleta-v3]

I have now gone over all the changes. I have left some minor comments. Really nice piece of work 😄 💪

[drvinceknight]

What an awesome piece of work @marcharper 👏 👏 👏