Wandb logger summary max accuracy instead last

[ValeKnappich]

Weights and biases has a summary per run, that shows an aggregation of the metric throughout the run. By default this shows the last value of the respective metric. While this makes sense for some scenarios, it certainly doesn't when using checkpointing; then you would want to compare the best performance. When using plain wandb (without lightning) one can update the summary every epoch or after the training as run.summary["val_acc"] = max(val_acc_values).

I tried doing the same thing in the Module Hooks (on_epoch_end and on_train_end) but in the end the value will again be just the last value. Since it works fine without lightning, the problem is in the integration. How do I properly tell Lightning to use the best value instead of the last for the wandb summary?

Workarounds are appreciated. Thanks.

[morganmcg1]

Hey @ValeKnappich,

You can use wandb.define_metric to also get wandb to track the max value for that metric. To use with PTL, maybe try calling it after you have initialised your wandb_logger and initialised wandb run? Links below might help

https://docs.wandb.ai/guides/track/log#customize-axes-and-summaries-with-define_metric

https://docs.wandb.ai/ref/python/run#define_metric