Batch randomly cancels in-progress jobs if re-submitting

[ppinchuk]

Sometimes kicking off a batch job while it is already running cancels the set that was kicked off first (but does not cancel all sets that are running).

You submit the batch, then change the configs and kick off the batch again - when you do that, only one of the two sets that were running get canceled. However, you wouldn't expect anything to be cancelled because it was "submitted"/running.

No config paths were changed.

Use case for this would be to kick batch off again for some failed jobs while others are still running.

This is all using --monitor-background.