Skip to content

Commit

Permalink
Fix: ScaledDotProductAttention.forward throws AttributeError when exe…
Browse files Browse the repository at this point in the history
…cuted within `enable_torch_sdp`
  • Loading branch information
dallmann-uniwue committed Sep 30, 2024
1 parent e7e7e9d commit 48f1878
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion curated_transformers/layers/attention.py
Original file line number Diff line number Diff line change
Expand Up @@ -708,7 +708,7 @@ def forward(
key=key,
value=value,
attn_mask=logit_mask,
dropout_p=self.dropout_prob if self.training else 0.0,
dropout_p=self.dropout.p if self.training else 0.0,
)

# Torch SDP returns NaNs for pieces where every is piece masked out.
Expand Down

0 comments on commit 48f1878

Please sign in to comment.