on_post_move_to_device is leaky

[ananthsub]

The decorator here is leaky: https://github.com/PyTorchLightning/pytorch-lightning/blob/963c26764682fa4cf64c93c5a7572ae0040e9c32/pytorch_lightning/core/decorators.py#L73-L108

It assumes it is called with the model_to_device method, and that self has access to model which implements on_post_move_to_device and has parameters defined.

The hook here: https://github.com/PyTorchLightning/pytorch-lightning/blob/963c26764682fa4cf64c93c5a7572ae0040e9c32/pytorch_lightning/core/hooks.py#L341-L355

is only called by the TPU backend. Is it intended to be called by other plugins?

Since this is under the core/decorators.py, one might assume that this is more general. Should this be an implementation detail of the TPU plugins instead? @kaushikb11

[tchaton]

This hook is intended only for TPU as TPU don't support tying parameters on cpu.

However, I believe we could actually depreciate it once #8555 is implemented.