Repository navigation
Claude parallelisation optimisation - #3
Conversation
…yBee_fork into future-reduce-globals
…rsede createPrevailingVirginQueens_internal() now also takes a Colony, which is done in the main session without futures; the per-colony work is in createPrevailingVirginQueen_internal(). swarm() and supersede() use it for Colony and MultiColony, so all created virgin queens (including csd homozygous ones) are recorded as virgin queens for a Colony too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
createPrevailingVirginQueens_internal() now does the work for a Colony and, for a MultiColony, calls itself on each colony (with only the queen) on the parallel workers via future_mapply(); given ids, it returns the records instead of adding them to simParamBee. Seeded results are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
createCastePop(), pullCastePop(), and removeCastePop() on a MultiColony now send colonies to parallel workers with future_mapply(), which has much less overhead than future_lapply() with plan(multisession) (buildUp() 14.5 -> 7 s, removeWorkers() 10.5 -> 2.2 s with 1000 colonies and 4 workers). Seeded results are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I've checked and it all looks good now (I've resolved any issues in the process) |
|
This is merging in the speed-up adjustments from Claude. |
Before, we were passing all the information to worker processes in multisessions. That actually broke on HPC because the amount of stuff that should be given to each worker exceeded the memory allocated. Now, each worker only receives the bit that it needs to work on. Still yet to be tested on Eddie.