Skip to content

Commit 13acec6

Browse files
committed
word_language_model: Fix Transformer init_weights
Model was not getting initialized property since it was using the decoder object instead of decoder weight to initialize zeros. Signed-off-by: Eli Uriegas <[email protected]>
1 parent edfa9f2 commit 13acec6

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

word_language_model/model.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ def _generate_square_subsequent_mask(self, sz):
133133
def init_weights(self):
134134
initrange = 0.1
135135
nn.init.uniform_(self.encoder.weight, -initrange, initrange)
136-
nn.init.zeros_(self.decoder)
136+
nn.init.zeros_(self.decoder.weight)
137137
nn.init.uniform_(self.decoder.weight, -initrange, initrange)
138138

139139
def forward(self, src, has_mask=True):

0 commit comments

Comments
 (0)