Clarify self-attention layer documentation #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

glyh wants to merge 2 commits into LearningInfiniTensor:main from glyh:fix2

glyh commented Feb 2, 2025 •

edited

Loading

Attention层对score归一化应该用的是masked_softmax而不是softmax
标记清楚用户需要修改的代码位置

glyh added 2 commits

February 2, 2025 11:49


          fix typo: attention layer should use masked_softmax instead of softmax

e12eb6c


          clarify self-attention implementation

821c024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet