@MaximilienLC Do you mean this?

trainer = Trainer(
      accelerator="cpu",
      devices=[1, 3],
  )

TypeError: `devices` selected with `CPUAccelerator` should be an int > 0.

This is correct. It does not make sense to select device indices on a CPU. Device indices are meant for accelerators like CUDA GPUs or TPUs.

If you select devices=1 (int) on CPU, PyTorch will already use all cores and threads when appropriate. And devices > 1 just simulates DDP on CPU but it does not give you any benefits in terms of speed, it's only meant for debugging DDP.

If there is documentation that contradicts this, please point me to it so we can update it. Thanks!

I guess this section could add that info for CPU.

So you mean devices=1 on CPU equates to using all CPUs?

So you mean devices=1 on CPU equates to using all CPUs?

Yes PyTorch will use all CPUs if it can parallelize the operation accordingly.

I agree that this should not throw an exception assuming the current documentation is correct. For example, the device could come from an environment variable

trainer = Trainer(
    accelerator=os.environ.get("DEVICE", "cpu"),
    devices=-1,  # use all devices
)

So when deploying the code to different machines, it would break for CPU. Maybe a warning would be more adequate to notify the user that something is not optimally configured.

Would devices="auto" also use all devices?

Hi! Would like to work on this one.

@awaelchli, wanted to clarify, what is more logical here: change the exception in the case accelerator="cpu", so that it is not thrown for devices equal to -1 (as pointed above), or the current behavior is correct and we only need to add more info to the devices section in the trainer.py docstring?

Humbly I think the solution should be to remove List[int] as an input type for cpu accelerator, @awaelchli please review this PR!