TypeError: state_dict() got multiple values for argument 'destination'

[elazzouzi1080]

Describe the bug

When I want to save my best model after one epoch, I have this error: TypeError: state_dict() got multiple values for argument 'destination'

2023-06-14 10:23:21,212 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:21,212 Corpus: 5483 train + 5483 dev + 5483 test sentences
2023-06-14 10:23:21,213 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:21,213 Train: 5483 sentences
2023-06-14 10:23:21,214 (train_with_dev=False, train_with_test=False)
2023-06-14 10:23:21,214 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:21,214 Training Params:
2023-06-14 10:23:21,215 - learning_rate: "0.1"
2023-06-14 10:23:21,215 - mini_batch_size: "64"
2023-06-14 10:23:21,215 - max_epochs: "3"
2023-06-14 10:23:21,216 - shuffle: "True"
2023-06-14 10:23:21,216 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:21,216 Plugins:
2023-06-14 10:23:21,217 - AnnealOnPlateau | patience: '3', anneal_factor: '0.5', min_learning_rate: '0.0001'
2023-06-14 10:23:21,217 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:21,217 Final evaluation on model from best epoch (best-model.pt)
2023-06-14 10:23:21,217 - metric: "('micro avg', 'f1-score')"
2023-06-14 10:23:21,218 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:21,218 Computation:
2023-06-14 10:23:21,218 - compute on device: cuda:0
2023-06-14 10:23:21,219 - embedding storage: none
2023-06-14 10:23:21,219 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:21,219 Model training base path: "/donnees/home/elazzouzi/NER_BERT_BILSTM_CRF_FINAL/LSTM_CRF_FastText_mBERT"
2023-06-14 10:23:21,219 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:21,220 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:21,898 epoch 1 - iter 8/86 - loss 0.16275612 - time (sec): 0.68 - samples/sec: 4082.16 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:22,542 epoch 1 - iter 16/86 - loss 0.16906576 - time (sec): 1.32 - samples/sec: 4212.56 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:23,289 epoch 1 - iter 24/86 - loss 0.14910738 - time (sec): 2.07 - samples/sec: 4145.49 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:23,870 epoch 1 - iter 32/86 - loss 0.15717436 - time (sec): 2.65 - samples/sec: 4297.59 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:24,501 epoch 1 - iter 40/86 - loss 0.15527718 - time (sec): 3.28 - samples/sec: 4338.57 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:25,232 epoch 1 - iter 48/86 - loss 0.15027606 - time (sec): 4.01 - samples/sec: 4300.37 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:25,912 epoch 1 - iter 56/86 - loss 0.14482447 - time (sec): 4.69 - samples/sec: 4295.03 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:26,628 epoch 1 - iter 64/86 - loss 0.13921061 - time (sec): 5.41 - samples/sec: 4284.00 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:27,214 epoch 1 - iter 72/86 - loss 0.13296037 - time (sec): 5.99 - samples/sec: 4330.59 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:27,772 epoch 1 - iter 80/86 - loss 0.12782567 - time (sec): 6.55 - samples/sec: 4378.60 - lr: 0.100000 - momentum: 0.000000
2023-06-14 10:23:28,297 ----------------------------------------------------------------------------------------------------
2023-06-14 10:23:28,298 EPOCH 1 done: loss 0.1260 - lr: 0.100000
100%|██████████| 86/86 [00:09<00:00, 9.05it/s]
2023-06-14 10:23:37,825 DEV : loss 0.07393791526556015 - f1-score (micro avg) 0.7315
2023-06-14 10:23:37,883 - 0 epochs without improvement
2023-06-14 10:23:37,884 - 0 epochs without improvement
2023-06-14 10:23:37,885 - 0 epochs without improvement
2023-06-14 10:23:37,885 saving best model

---

TypeError Traceback (most recent call last)
Input In [27], in <cell line: 1>()
----> 1 trainer.train('/donnees/home/elazzouzi/NER_BERT_BILSTM_CRF_FINAL/LSTM_CRF_FastText_mBERT/',
2 train_with_dev = False,
3 learning_rate = 0.1,
4 mini_batch_size = 64,
5 #optimizer = torch.optim.AdamW,
6 max_epochs = 3,
7 embeddings_storage_mode = 'none')

File ~/.local/lib/python3.8/site-packages/flair/trainers/trainer.py:195, in ModelTrainer.train(self, base_path, anneal_factor, patience, min_learning_rate, initial_extra_patience, anneal_with_restarts, learning_rate, decoder_learning_rate, mini_batch_size, eval_batch_size, mini_batch_chunk_size, max_epochs, optimizer, train_with_dev, train_with_test, main_evaluation_metric, monitor_test, monitor_train_sample, use_final_model_for_eval, gold_label_dictionary_for_eval, exclude_labels, sampler, shuffle, shuffle_first_epoch, embeddings_storage_mode, epoch, save_final_model, save_optimizer_state, save_model_each_k_epochs, create_file_logs, create_loss_file, write_weights, plugins, attach_default_scheduler, **kwargs)
184 for var in [
185 "self",
186 "anneal_factor",
(...)
192 "kwargs",
193 ]:
194 local_variables.pop(var)
--> 195 return self.train_custom(**local_variables, **kwargs)

File ~/.local/lib/python3.8/site-packages/flair/trainers/trainer.py:691, in ModelTrainer.train_custom(self, base_path, learning_rate, decoder_learning_rate, mini_batch_size, eval_batch_size, mini_batch_chunk_size, max_epochs, optimizer, train_with_dev, train_with_test, main_evaluation_metric, monitor_test, monitor_train_sample, use_final_model_for_eval, gold_label_dictionary_for_eval, exclude_labels, sampler, shuffle, shuffle_first_epoch, embeddings_storage_mode, epoch, save_final_model, save_optimizer_state, save_model_each_k_epochs, create_file_logs, create_loss_file, write_weights, plugins, **kwargs)
689 if save_best_model and current_epoch_has_best_model_so_far:
690 log.info("saving best model")
--> 691 self.model.save(base_path / "best-model.pt", checkpoint=save_optimizer_state)
693 # - SWAPlugin -> restores SGD weights from SWA
694 self.dispatch("after_training_loop")

File ~/.local/lib/python3.8/site-packages/flair/nn/model.py:105, in Model.save(self, model_file, checkpoint)
100 def save(self, model_file: Union[str, Path], checkpoint: bool = False):
101 """Saves the current model to the provided file.
102
103 :param model_file: the model file
104 """
--> 105 model_state = self._get_state_dict()
107 # write out a "model card" if one is set
108 if self.model_card is not None:

File ~/.local/lib/python3.8/site-packages/flair/models/sequence_tagger_model.py:589, in SequenceTagger._get_state_dict(self)
586 def _get_state_dict(self):
587 """Returns the state dictionary for this model."""
588 model_state = {
--> 589 **super()._get_state_dict(),
590 "embeddings": self.embeddings.save_embeddings(use_state_dict=False),
591 "hidden_size": self.hidden_size,
592 "tag_dictionary": self.label_dictionary,
593 "tag_format": self.tag_format,
594 "tag_type": self.tag_type,
595 "use_crf": self.use_crf,
596 "use_rnn": self.use_rnn,
597 "rnn_layers": self.rnn_layers,
598 "use_dropout": self.use_dropout,
599 "use_word_dropout": self.use_word_dropout,
600 "use_locked_dropout": self.use_locked_dropout,
601 "rnn_type": self.rnn_type,
602 "reproject_embeddings": self.reproject_embeddings,
603 "weight_dict": self.weight_dict,
604 "train_initial_hidden_state": self.train_initial_hidden_state,
605 }
607 return model_state

File ~/.local/lib/python3.8/site-packages/flair/nn/model.py:74, in Model._get_state_dict(self)
72 def _get_state_dict(self):
73 """Returns the state dictionary for this model."""
---> 74 state_dict = {"state_dict": self.state_dict()}
76 # Always include the name of the Model class for which the state dict holds
77 state_dict["cls"] = self.class.name

File /appli/anaconda/lib/python3.8/site-packages/torch/nn/modules/module.py:1259, in Module.state_dict(self, destination, prefix, keep_vars)
1257 for name, module in self._modules.items():
1258 if module is not None:
-> 1259 module.state_dict(destination, prefix + name + '.', keep_vars=keep_vars)
1260 for hook in self._state_dict_hooks.values():
1261 hook_result = hook(self, destination, prefix, local_metadata)

File /appli/anaconda/lib/python3.8/site-packages/torch/nn/modules/module.py:1259, in Module.state_dict(self, destination, prefix, keep_vars)
1257 for name, module in self._modules.items():
1258 if module is not None:
-> 1259 module.state_dict(destination, prefix + name + '.', keep_vars=keep_vars)
1260 for hook in self._state_dict_hooks.values():
1261 hook_result = hook(self, destination, prefix, local_metadata)

File ~/.local/lib/python3.8/site-packages/flair/embeddings/token.py:438, in WordEmbeddings.state_dict(self, destination, prefix, keep_vars, *args)
436 if list(self.modules()) == [self]:
437 self.embedding = self.embedding
--> 438 return super().state_dict(*args, destination=destination, prefix=prefix, keep_vars=keep_vars)

TypeError: state_dict() got multiple values for argument 'destination'

To Reproduce

from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings
from flair.embeddings import TransformerWordEmbeddings
from flair.models import SequenceTagger
from flair.trainers import ModelTrainer

embedding_types = [

    # Fastetxt embeddings
    WordEmbeddings("fr"),
    
    TransformerWordEmbeddings('./BERT_base_multilingual_cased_Huggingface/',  fine_tune=False , allow_long_sentences=True , model_max_length=512),
 
]
embeddings : StackedEmbeddings = StackedEmbeddings(embeddings=embedding_types)


tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type= tag_type,
                        use_crf= True,
                        use_rnn= True)

trainer = ModelTrainer(tagger, corpus )
trainer.train('/donnees/home/elazzouzi/NER_BERT_BILSTM_CRF_FINAL/LSTM_CRF_FastText_mBERT/',
          train_with_dev = False,
          learning_rate = 0.1,
          mini_batch_size = 64,
          #optimizer = torch.optim.AdamW,
          max_epochs = 3,
          embeddings_storage_mode = 'none')

Expected behavior

2023-06-14 10:23:37,825 DEV : loss 0.07393791526556015 - f1-score (micro avg) 0.7315
2023-06-14 10:23:37,883 - 0 epochs without improvement
2023-06-14 10:23:37,884 - 0 epochs without improvement
2023-06-14 10:23:37,885 - 0 epochs without improvement
2023-06-14 10:23:37,885 saving best model

Logs and Stack traces

No response

Screenshots

No response

Additional Context

No response

Environment

:219: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject

Versions:

Flair

0.12.2

Pytorch

1.9.1+cu111

Transformers

4.21.0.dev0

GPU

True

[helpmefindaname]

Hi @elazzouzi1080
updating torch to a newer version should fix your problem, however if you still want to use that old version, you can checkout #3272 and see if that works.