Skip to content

[AArch64][SME] Fix restoring callee-saves from FP with hazard padding #143371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jun 18, 2025
Merged

[AArch64][SME] Fix restoring callee-saves from FP with hazard padding #143371

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
c965294
[AArch64][SME] Fix restoring callee-saves from FP with hazard padding
MacDue Jun 9, 2025
0cedfe5
Avoid unsafe subs
MacDue Jun 10, 2025
f101ba2
Remove BP hack
MacDue Jun 12, 2025
b2144ed
Rework a little
MacDue Jun 13, 2025
903f3fd
Reword
MacDue Jun 13, 2025
7fc63d9
Reword
MacDue Jun 13, 2025
98ee22f
Update llvm/lib/Target/AArch64/AArch64FrameLowering.cpp
MacDue Jun 14, 2025
11b953a
Fix typo
MacDue Jun 16, 2025
d9c517b
Simplify
MacDue Jun 16, 2025
698e146
Remove BP support
MacDue Jun 16, 2025
edc8b5f
Add braces
MacDue Jun 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 25 additions & 12 deletions llvm/lib/Target/AArch64/AArch64FrameLowering.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2535,20 +2535,33 @@ void AArch64FrameLowering::emitEpilogue(MachineFunction &MF,
DeallocateAfter, TII, MachineInstr::FrameDestroy, false,
NeedsWinCFI, &HasWinCFI);
} else if (SVEStackSize) {
// If we have stack realignment or variable sized objects on the stack,
// restore the stack pointer from the frame pointer prior to SVE CSR
// restoration.
if (AFI->isStackRealigned() || MFI.hasVarSizedObjects()) {
if (int64_t CalleeSavedSize = AFI->getSVECalleeSavedStackSize()) {
// Set SP to start of SVE callee-save area from which they can
// be reloaded. The code below will deallocate the stack space
// space by moving FP -> SP.
emitFrameOffset(MBB, RestoreBegin, DL, AArch64::SP, AArch64::FP,
StackOffset::getScalable(-CalleeSavedSize), TII,
int64_t SVECalleeSavedSize = AFI->getSVECalleeSavedStackSize();
// If we have stack realignment or variable-sized objects we must use the
// FP to restore SVE callee saves (as there is an unknown amount of
// data/padding between the SP and SVE CS area).
Register BaseForSVEDealloc =
(AFI->isStackRealigned() || MFI.hasVarSizedObjects()) ? AArch64::FP
: AArch64::SP;
if (SVECalleeSavedSize && BaseForSVEDealloc == AArch64::FP) {
Register CalleeSaveBase = AArch64::FP;
if (int64_t CalleeSaveBaseOffset =
AFI->getCalleeSaveBaseToFrameRecordOffset()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, is the CalleeSaveBaseToFrameRecordOffset always equal to the stack hazard size in this scenario? (i.e. when the frame record is at the bottom of the GPR callee saves and there are SVE callee-saves).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, later in emitEpilogue, it's also used to restore the non-scalable CS from the FP, which means those still work in this case. I think it's preferable to using the hazard padding size directly as what we want to know is how to get to the base of the callee saves from the FP, and that's this offset.

// If we have have an non-zero offset to the non-SVE CS base we need to
// compute the base address by subtracting the offest in a temporary
// register first (to avoid briefly deallocating the SVE CS).
CalleeSaveBase = MBB.getParent()->getRegInfo().createVirtualRegister(
&AArch64::GPR64RegClass);
emitFrameOffset(MBB, RestoreBegin, DL, CalleeSaveBase, AArch64::FP,
StackOffset::getFixed(-CalleeSaveBaseOffset), TII,
MachineInstr::FrameDestroy);
}
} else {
if (AFI->getSVECalleeSavedStackSize()) {
// The code below will deallocate the stack space space by moving the
// SP to the start of the SVE callee-save area.
emitFrameOffset(MBB, RestoreBegin, DL, AArch64::SP, CalleeSaveBase,
StackOffset::getScalable(-SVECalleeSavedSize), TII,
MachineInstr::FrameDestroy);
} else if (BaseForSVEDealloc == AArch64::SP) {
if (SVECalleeSavedSize) {
// Deallocate the non-SVE locals first before we can deallocate (and
// restore callee saves) from the SVE area.
emitFrameOffset(
Expand Down
Loading