Predicting from checkpoint with CLI #20715
Replies: 3 comments 5 replies
-
|
After giving this some more thought, I suppose the generated |
Beta Was this translation helpful? Give feedback.
-
|
My current workaround is class MyLightningCLI(LightningCLI):
def after_instantiate_classes(self) -> None:
if self.subcommand == "predict" and self.config["predict"]["ckpt_path"]:
logger.warning("Hijacking model after instantiate.")
config_from_cli = self.parser.parse_args(self.parser.args, defaults=False)
model_args = config_from_cli["predict"].get("model", {})
logger.debug(f"{model_args=}")
self.model = MyModel.load_from_checkpoint(
self.config["predict"]["ckpt_path"], **model_args
)
logger.debug(f"Model loaded from checkpoint: {self.model}")This is not great, but works for my current use-case. I suppose the |
Beta Was this translation helpful? Give feedback.
-
|
The purpose of the Regarding your Small note, if you need argument links applied on instantiation, then |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the detailed reply, @mauvilsa! Firstly, thanks for the clarification of the purpose of the On the # cli.py
from lightning.pytorch.cli import LightningCLI
from lightning.pytorch.demos.boring_classes import DemoModel, BoringDataModule
class DemoModelWithHyperparameters(DemoModel):
def __init__(self, *args, **kwargs):
self.save_hyperparameters()
super().__init__(*args, **kwargs)
def cli_main():
cli = LightningCLI(DemoModelWithHyperparameters, BoringDataModule)
if __name__ == "__main__":
cli_main()and then run $ python src/lightning_cli_load_checkpoint/cli.py fit --trainer.max_epochs 1 --model.out_dim 2
$ python src/lightning_cli_load_checkpoint/cli.py predict --ckpt_path <path_to_checkpoint>gives The problem, as far as I can tell, is that the model is first instantiated with the default (+ cli supplied) parameters, which is where the tensors are allocated. During the checkpoint loading, which is done later, the weights and hyperparameters from the checkpoint is attempted loaded into the module, however, without reinstantiating the module or reallocating tensors. In many situations, this is not an issue, however, where tensor sizes are changed, like in this example, it fails. Ps. The |
Beta Was this translation helpful? Give feedback.
-
|
@Northo you are completely right. How We can fix this directly in the |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the clarification! If relevant, I would be willing to help in making a PR for this. |
Beta Was this translation helpful? Give feedback.
-
|
PS. I raised this in #20801 |
Beta Was this translation helpful? Give feedback.
-
|
I will comment in #20801 once I have thought about this. |
Beta Was this translation helpful? Give feedback.
-
I want to predict with a model for which I have a checkpoint. The arguments used to train the model (producing the checkpoint), causes the model to have different layer sizes than the default values. When using
ckpt_path, this then causes the loading to fail, due to a size mismatch.Two obvious workarounds are
predictcall (--model.some_layer_size 4above), orLightningModule.load_from_checkpoint.Explicitly giving all the flags becomes very cumbersome when there are many. The second approach might be a good one, though I haven't yet tried implementing it.
Is there something I am overlooking here? What is the recommended way to predict from a checkpoint like this?
Any input greatly appreciated:)
Beta Was this translation helpful? Give feedback.
All reactions