Alex for next cfi manual#668
Closed
AlexGhiti wants to merge 85 commits into
Closed
Conversation
AlexGhiti
force-pushed
the
alex-for-next-cfi_manual
branch
from
2342ab7 to
97f5795
Compare
__ASSEMBLY__ is only defined by the Makefile of the kernel, so this is not really useful for uapi headers (unless the userspace Makefile defines it, too). Let's switch to __ASSEMBLER__ which gets set automatically by the compiler when compiling assembly code. This is a completely mechanical patch (done with a simple "sed -i" statement). Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Alexandre Ghiti <alex@ghiti.fr> Cc: linux-riscv@lists.infradead.org Signed-off-by: Thomas Huth <thuth@redhat.com> Link: https://lore.kernel.org/r/20250606070952.498274-2-thuth@redhat.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
While the GCC and Clang compilers already define __ASSEMBLER__ automatically when compiling assembly code, __ASSEMBLY__ is a macro that only gets defined by the Makefiles in the kernel. This can be very confusing when switching between userspace and kernelspace coding, or when dealing with uapi headers that rather should use __ASSEMBLER__ instead. So let's standardize on the __ASSEMBLER__ macro that is provided by the compilers now. This originally was a completely mechanical patch (done with a simple "sed -i" statement), with some manual fixups during rebasing of the patch later. Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Alexandre Ghiti <alex@ghiti.fr> Cc: linux-riscv@lists.infradead.org Signed-off-by: Thomas Huth <thuth@redhat.com> Link: https://lore.kernel.org/r/20250606070952.498274-3-thuth@redhat.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
… header files" Thomas Huth <thuth@redhat.com> says: The kernel Makefiles define the __ASSEMBLY__ macro to provide a way to use headers in both, assembly and C source code. However, all the supported versions of the GCC and Clang compilers also define the macro __ASSEMBLER__ automatically already when compiling assembly code, so some kernel headers are using __ASSEMBLER__ instead. With regards to userspace code, this seems also to be constant source of confusion, see for example these links here: https://lore.kernel.org/kvm/20250222014526.2302653-1-seanjc@google.com/ https://stackoverflow.com/questions/28924355/gcc-assembler-preprocessor-not-compatible-with-standard-headers https://forums.raspberrypi.com/viewtopic.php?p=1652944#p1653834 riscv-software-src/opensbi#199 To avoid confusion in the future, it would make sense to standardize on the macro that gets defined by the compiler, so this patch series changes all occurances of __ASSEMBLY__ into __ASSEMBLER__. I split the patches per architecture to ease the review, and I also split the uapi headers from the normal ones in case we decide that uapi needs to be treated differently from the normal headers here. The x86 and parisc patches already got merged via their specific architecture tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a295e4ef1ca8 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8a141be3233af7 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=cccaea1d66e94b https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e2b6a188625a2b * patches from https://lore.kernel.org/r/20250606070952.498274-1-thuth@redhat.com: riscv: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers riscv: Replace __ASSEMBLY__ with __ASSEMBLER__ in uapi headers Link: https://lore.kernel.org/r/20250606070952.498274-1-thuth@redhat.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
%.pi.o files are built as prerequisites of other objects. There is no need to use extra-y, which is planned for deprecation. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Link: https://lore.kernel.org/r/20250602181023.528550-1-masahiroy@kernel.org Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
These extensions depends on the F one. Add a validation callback checking for the F extension to be present. Now that extensions are correctly reported using the F/D presence, we can remove the has_fpu() check in hwprobe_isa_ext0(). Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20250527100001.33284-1-cleger@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Simply fix a typo. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-By: Clément Léger <cleger@rivosinc.com> Link: https://lore.kernel.org/r/20250620-dev-alex-insn_duplicate_v5_manual-v5-1-d865dc9ad180@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
RV_X() macro is defined in two different ways which is error prone. So harmonize its first definition and add another macro RV_X_mask() for the second one. Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Clément Léger <cleger@rivosinc.com> Link: https://lore.kernel.org/r/20250620-dev-alex-insn_duplicate_v5_manual-v5-2-d865dc9ad180@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
kernel/traps_misaligned.c and kvm/vcpu_insn.c define the same macros to extract information from the instructions. Let's move the definitions into asm/insn.h to avoid this duplication. Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Clément Léger <cleger@rivosinc.com> Link: https://lore.kernel.org/r/20250620-dev-alex-insn_duplicate_v5_manual-v5-3-d865dc9ad180@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Alexandre Ghiti <alexghiti@rivosinc.com> says: The instructions parsing macros were duplicated and one of them had different implementations, which is error prone. So let's consolidate those macros in asm/insn.h. * patches from https://lore.kernel.org/r/20250620-dev-alex-insn_duplicate_v5_manual-v5-0-d865dc9ad180@rivosinc.com: riscv: Move all duplicate insn parsing macros into asm/insn.h riscv: Strengthen duplicate and inconsistent definition of RV_X() riscv: Fix typo EXRACT -> EXTRACT Link: https://lore.kernel.org/r/20250620-dev-alex-insn_duplicate_v5_manual-v5-0-d865dc9ad180@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Similar to other instruction-processing macros/functions, branch_rs2_idx should be in insn.h. Move it into insn.h as RV_EXTRACT_RS2_REG. This new name matches the style in insn.h. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/107d4a6c1818bf169be2407b273a0483e6d55bbb.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Similar to other instruction-processing macros/functions, branch_funct3 should be in insn.h. Move it into insn.h as RV_EXTRACT_FUNCT3. This new name matches the style in insn.h. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/200c29a26338f19d09963fa02562787e8cfa06f2.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Use RV_EXTRACT_JTYPE_IMM, instead of reimplementing it in simulate_jal(). Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/af502036738d381c6bdb96a236d21bab8c343f74.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Use RV_EXTRACT_RS1_REG instead of reimplementing its code. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/b441038c991da11a7a48ea7140ab00e3bb119387.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Use RV_EXTRACT_BTYPE_IMM, instead of reimplementing it in simulate_branch(). Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/07dde8313b787261f07ae25240c97c514cd02d11.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Use RVC_EXTRACT_JTYPE_IMM, instead of reimplementing it in simulate_c_j(). Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/24497deaab06d6b12cb84923606ec26f67e25424.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Use RVC_EXTRACT_C2_RS1_REG, instead of reimplementing it in simulate_c_jr_jalr(). Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/d56955cd683411c6d2f63d13c78e0572462a3269.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Use RVC_EXTRACT_BTYPE_IMM, instead of reimplementing it in simulate_c_bnez_beqz(). Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/8a8ed970f279fa5f24c90d840c2130e37bc6d16e.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Use RV_EXTRACT_RD_REG, instead of reimplementing its code. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/b31e5b41df5839a76103348e54dc034c8a43447a.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Use RV_EXTRACT_UTYPE_IMM, instead of reimplementing it in simulate_auipc(). Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/8f0defce9f1f23f1b44bb9750ed083cfc124213c.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Use RV_EXTRACT_ITYPE_IMM, instead of re-implementing it in simulate_jalr(). Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Nam Cao <namcao@linutronix.de> Link: https://lore.kernel.org/r/8ae34e966c312ae5cf6c09a35ddc290cce942208.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Nam Cao <namcao@linutronix.de> says: Hi, There is some instruction-processing code in kprobes simulate code. These code should be insn.h. In fact, most of them is duplicating insn.h. This series remove the duplicated bits and make use of macros already defined in insn.h. The non-duplicated bits are moved into insn.h. * patches from https://lore.kernel.org/r/cover.1747215274.git.namcao@linutronix.de: riscv: kprobes: Remove duplication of RV_EXTRACT_ITYPE_IMM riscv: kprobes: Remove duplication of RV_EXTRACT_UTYPE_IMM riscv: kprobes: Remove duplication of RV_EXTRACT_RD_REG riscv: kprobes: Remove duplication of RVC_EXTRACT_BTYPE_IMM riscv: kprobes: Remove duplication of RVC_EXTRACT_C2_RS1_REG riscv: kproves: Remove duplication of RVC_EXTRACT_JTYPE_IMM riscv: kprobes: Remove duplication of RV_EXTRACT_BTYPE_IMM riscv: kprobes: Remove duplication of RV_EXTRACT_RS1_REG riscv: kprobes: Remove duplication of RV_EXTRACT_JTYPE_IMM riscv: kprobes: Move branch_funct3 to insn.h riscv: kprobes: Move branch_rs2_idx to insn.h Link: https://lore.kernel.org/r/cover.1747215274.git.namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Add KUnit test for riscv kprobes, mostly for simulated instructions. The test install kprobes into multiple sample functions, and check that these functions still return the expected magic value. This test can detect some kprobe bugs reported in the past (in Link:). Link: https://lore.kernel.org/linux-riscv/20241119111056.2554419-1-namcao@linutronix.de/ Link: https://lore.kernel.org/stable/c7e463c0-8cad-4f4e-addd-195c06b7b6de@iscas.ac.cn/ Link: https://lore.kernel.org/linux-riscv/20230829182500.61875-1-namcaov@gmail.com/ Signed-off-by: Nam Cao <namcao@linutronix.de> Tested-by: Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20250513151631.3520793-1-namcao@linutronix.de Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
The variable type of offset should be consistent with the relevant interfaces of mmap which described in commit 295f100 ("syscalls: mmap(): use unsigned offset type consistently"). Otherwise, a user input with the top bit set would result in a negative page offset rather than a large one. [ alex: Add missing " in commit log ] Signed-off-by: Jessica Liu <liu.xuemei1@zte.com.cn> Reviewed-by: Nutty Liu <liujingqi@lanxincomputing.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20250707193411886Kc-TWknP0PER2_sEg-byb@zte.com.cn Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Move vendor errata definitions into errata_list_vendors.h. Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org> Tested-by: Han Gao <rabenda.cn@gmail.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20250713155321.2064856-2-guoren@kernel.org Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
The early version of XuanTie C910 core has a store merge buffer delay problem. The store merge buffer could improve the store queue performance by merging multi-store requests, but when there are not continued store requests, the prior single store request would be waiting in the store queue for a long time. That would cause significant problems for communication between multi-cores. This problem was found on sg2042 & th1520 platforms with the qspinlock lock torture test. So appending a fence w.o could immediately flush the store merge buffer and let other cores see the write result. This will apply the WRITE_ONCE errata to handle the non-standard behavior via appending a fence w.o instruction for WRITE_ONCE(). This problem is only observed on the sg2042 hardware platform by running the lock_torture test program for half an hour. The problem was not found in the user space application, because interrupt can break the livelock. Reviewed-by: Leonardo Bras <leobras@redhat.com> Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Tested-by: Han Gao <rabenda.cn@gmail.com> Link: https://lore.kernel.org/r/20250713155321.2064856-3-guoren@kernel.org Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
These two C files don't reference things defined in simd.h or types.h so remove these redundant #inclusions. Fixes: 6093faa ("raid6: Add RISC-V SIMD syndrome and recovery calculations") Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Chunyan Zhang <zhangchunyan@iscas.ac.cn> Reviewed-by: Nutty Liu <liujingqi@lanxincomputing.com> Link: https://lore.kernel.org/r/20250718072711.3865118-2-zhangchunyan@iscas.ac.cn Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Since wp$$==wq$$, it doesn't need to load the same data twice, use move instruction to replace one of the loads to let the program run faster. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Chunyan Zhang <zhangchunyan@iscas.ac.cn> Link: https://lore.kernel.org/r/20250718072711.3865118-3-zhangchunyan@iscas.ac.cn Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
…ectorized code To avoid the inline assembly code to break what the compiler could have vectorized, this code must be built without compiler support for vector. Signed-off-by: Chunyan Zhang <zhangchunyan@iscas.ac.cn> Link: https://lore.kernel.org/r/20250718072711.3865118-4-zhangchunyan@iscas.ac.cn Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
To support userspace raid6test, this patch adds __KERNEL__ ifdef for kernel header inclusions also userspace wrapper definitions to allow code to be compiled in userspace. This patch also drops the NSIZE macro, instead of using the vector length, which can work for both kernel and user space. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Chunyan Zhang <zhangchunyan@iscas.ac.cn> Link: https://lore.kernel.org/r/20250718072711.3865118-5-zhangchunyan@iscas.ac.cn Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Make an entry for cfi extensions in extensions.yaml. Signed-off-by: Deepak Gupta <debug@rivosinc.com> Acked-by: Rob Herring (Arm) <robh@kernel.org> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-2-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
This patch adds support for detecting zicfiss and zicfilp. zicfiss and zicfilp stands for unprivleged integer spec extension for shadow stack and branch tracking on indirect branches, respectively. This patch looks for zicfiss and zicfilp in device tree and accordinlgy lights up bit in cpu feature bitmap. Furthermore this patch adds detection utility functions to return whether shadow stack or landing pads are supported by cpu. Reviewed-by: Zong Li <zong.li@sifive.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-3-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
zicfiss and zicfilp extension gets enabled via b3 and b2 in *envcfg CSR. menvcfg controls enabling for S/HS mode. henvcfg control enabling for VS while senvcfg controls enabling for U/VU mode. zicfilp extension extends *status CSR to hold `expected landing pad` bit. A trap or interrupt can occur between an indirect jmp/call and target instr. `expected landing pad` bit from CPU is recorded into xstatus CSR so that when supervisor performs xret, `expected landing pad` state of CPU can be restored. zicfiss adds one new CSR - CSR_SSP: CSR_SSP contains current shadow stack pointer. Signed-off-by: Deepak Gupta <debug@rivosinc.com> Reviewed-by: Charlie Jenkins <charlie@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-4-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
…ry/exit Carves out space in arch specific thread struct for cfi status and shadow stack in usermode on riscv. This patch does following - defines a new structure cfi_status with status bit for cfi feature - defines shadow stack pointer, base and size in cfi_status structure - defines offsets to new member fields in thread in asm-offsets.c - Saves and restore shadow stack pointer on trap entry (U --> S) and exit (S --> U) Shadow stack save/restore is gated on feature availiblity and implemented using alternative. CSR can be context switched in `switch_to` as well but soon as kernel shadow stack support gets rolled in, shadow stack pointer will need to be switched at trap entry/exit point (much like `sp`). It can be argued that kernel using shadow stack deployment scenario may not be as prevalant as user mode using this feature. But even if there is some minimal deployment of kernel shadow stack, that means that it needs to be supported. And thus save/restore of shadow stack pointer in entry.S instead of in `switch_to.h`. Reviewed-by: Charlie Jenkins <charlie@rivosinc.com> Reviewed-by: Zong Li <zong.li@sifive.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-5-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
`arch_calc_vm_prot_bits` is implemented on risc-v to return VM_READ | VM_WRITE if PROT_WRITE is specified. Similarly `riscv_sys_mmap` is updated to convert all incoming PROT_WRITE to (PROT_WRITE | PROT_READ). This is to make sure that any existing apps using PROT_WRITE still work. Earlier `protection_map[VM_WRITE]` used to pick read-write PTE encodings. Now `protection_map[VM_WRITE]` will always pick PAGE_SHADOWSTACK PTE encodings for shadow stack. Above changes ensure that existing apps continue to work because underneath kernel will be picking `protection_map[VM_WRITE|VM_READ]` PTE encodings. Reviewed-by: Zong Li <zong.li@sifive.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-6-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
This patch implements creating shadow stack pte (on riscv). Creating shadow stack PTE on riscv means that clearing RWX and then setting W=1. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-7-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
pte_mkwrite creates PTEs with WRITE encodings for underlying arch. Underlying arch can have two types of writeable mappings. One that can be written using regular store instructions. Another one that can only be written using specialized store instructions (like shadow stack stores). pte_mkwrite can select write PTE encoding based on VMA range (i.e. VM_SHADOW_STACK) Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-8-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
`fork` implements copy on write (COW) by making pages readonly in child and parent both. ptep_set_wrprotect and pte_wrprotect clears _PAGE_WRITE in PTE. Assumption is that page is readable and on fault copy on write happens. To implement COW on shadow stack pages, clearing up W bit makes them XWR = 000. This will result in wrong PTE setting which says no perms but V=1 and PFN field pointing to final page. Instead desired behavior is to turn it into a readable page, take an access (load/store) fault on sspush/sspop (shadow stack) and then perform COW on such pages. This way regular reads would still be allowed and not lead to COW maintaining current behavior of COW on non-shadow stack but writeable memory. On the other hand it doesn't interfere with existing COW for read-write memory. Assumption is always that _PAGE_READ must have been set and thus setting _PAGE_READ is harmless. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-9-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
As discussed extensively in the changelog for the addition of this
syscall on x86 ("x86/shstk: Introduce map_shadow_stack syscall") the
existing mmap() and madvise() syscalls do not map entirely well onto the
security requirements for shadow stack memory since they lead to windows
where memory is allocated but not yet protected or stacks which are not
properly and safely initialised. Instead a new syscall map_shadow_stack()
has been defined which allocates and initialises a shadow stack page.
This patch implements this syscall for riscv. riscv doesn't require token
to be setup by kernel because user mode can do that by itself. However to
provide compatibility and portability with other architectues, user mode
can specify token set flag.
Reviewed-by: Zong Li <zong.li@sifive.com>
Signed-off-by: Deepak Gupta <debug@rivosinc.com>
Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-10-09b468d7beab@rivosinc.com
Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Userspace specifies CLONE_VM to share address space and spawn new thread. `clone` allow userspace to specify a new stack for new thread. However there is no way to specify new shadow stack base address without changing API. This patch allocates a new shadow stack whenever CLONE_VM is given. In case of CLONE_VFORK, parent is suspended until child finishes and thus can child use parent shadow stack. In case of !CLONE_VM, COW kicks in because entire address space is copied from parent to child. `clone3` is extensible and can provide mechanisms using which shadow stack as an input parameter can be provided. This is not settled yet and being extensively discussed on mailing list. Once that's settled, this commit will adapt to that. Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-11-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Implement architecture agnostic prctls() interface for setting and getting shadow stack status. prctls implemented are PR_GET_SHADOW_STACK_STATUS, PR_SET_SHADOW_STACK_STATUS and PR_LOCK_SHADOW_STACK_STATUS. As part of PR_SET_SHADOW_STACK_STATUS/PR_GET_SHADOW_STACK_STATUS, only PR_SHADOW_STACK_ENABLE is implemented because RISCV allows each mode to write to their own shadow stack using `sspush` or `ssamoswap`. PR_LOCK_SHADOW_STACK_STATUS locks current configuration of shadow stack enabling. Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-12-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Three architectures (x86, aarch64, riscv) have support for indirect branch
tracking feature in a very similar fashion. On a very high level, indirect
branch tracking is a CPU feature where CPU tracks branches which uses
memory operand to perform control transfer in program. As part of this
tracking on indirect branches, CPU goes in a state where it expects a
landing pad instr on target and if not found then CPU raises some fault
(architecture dependent)
x86 landing pad instr - `ENDBRANCH`
arch64 landing pad instr - `BTI`
riscv landing instr - `lpad`
Given that three major arches have support for indirect branch tracking,
This patch makes `prctl` for indirect branch tracking arch agnostic.
To allow userspace to enable this feature for itself, following prtcls are
defined:
- PR_GET_INDIR_BR_LP_STATUS: Gets current configured status for indirect
branch tracking.
- PR_SET_INDIR_BR_LP_STATUS: Sets a configuration for indirect branch
tracking.
Following status options are allowed
- PR_INDIR_BR_LP_ENABLE: Enables indirect branch tracking on user
thread.
- PR_INDIR_BR_LP_DISABLE; Disables indirect branch tracking on user
thread.
- PR_LOCK_INDIR_BR_LP_STATUS: Locks configured status for indirect branch
tracking for user thread.
Reviewed-by: Mark Brown <broonie@kernel.org>
Reviewed-by: Zong Li <zong.li@sifive.com>
Signed-off-by: Deepak Gupta <debug@rivosinc.com>
Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-13-09b468d7beab@rivosinc.com
Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
prctls implemented are: PR_SET_INDIR_BR_LP_STATUS, PR_GET_INDIR_BR_LP_STATUS and PR_LOCK_INDIR_BR_LP_STATUS Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-14-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
zicfiss / zicfilp introduces a new exception to priv isa `software check exception` with cause code = 18. This patch implements software check exception. Additionally it implements a cfi violation handler which checks for code in xtval. If xtval=2, it means that sw check exception happened because of an indirect branch not landing on 4 byte aligned PC or not landing on `lpad` instruction or label value embedded in `lpad` not matching label value setup in `x7`. If xtval=3, it means that sw check exception happened because of mismatch between link register (x1 or x5) and top of shadow stack (on execution of `sspopchk`). In case of cfi violation, SIGSEGV is raised with code=SEGV_CPERR. SEGV_CPERR was introduced by x86 shadow stack patches. To keep uprobes working, handle the uprobe event first before reporting the CFI violation in software-check exception handler. Because when the landing pad is activated, if the uprobe point is set at the lpad instruction at the beginning of a function, the system triggers a software -check exception instead of an ebreak exception due to the exception priority, then uprobe can't work successfully. Co-developed-by: Zong Li <zong.li@sifive.com> Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-15-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
The function save_v_state() served two purposes. First, it saved extension context into the signal stack. Then, it constructed the extension header if there was no fault. The second part is independent of the extension itself. As a result, we can pull that part out, so future extensions may reuse it. This patch adds arch_ext_list and makes setup_sigcontext() go through all possible extensions' save() callback. The callback returns a positive value indicating the size of the successfully saved extension. Then the kernel proceeds to construct the header for that extension. The kernel skips an extension if it does not exist, or if the saving fails for some reasons. The error code is propagated out on the later case. This patch does not introduce any functional changes. Signed-off-by: Andy Chiu <andybnac@gmail.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-16-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Save shadow stack pointer in sigcontext structure while delivering signal. Restore shadow stack pointer from sigcontext on sigreturn. As part of save operation, kernel uses `ssamoswap` to save snapshot of current shadow stack on shadow stack itself (can be called as a save token). During restore on sigreturn, kernel retrieves token from top of shadow stack and validates it. This allows that user mode can't arbitrary pivot to any shadow stack address without having a token and thus provide strong security assurance between signaly delivery and sigreturn window. Use ABI compatible way of saving/restoring shadow stack pointer into signal stack. This follows what Vector extension, where extra registers are placed in a form of extension header + extension body in the stack. The extension header indicates the size of the extra architectural states plus the size of header itself, and a magic identifier of the extension. Then, the extensions body contains the new architectural states in the form defined by uapi. Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-17-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Updating __show_regs to print captured shadow stack pointer as well. On tasks where shadow stack is disabled, it'll simply print 0. Signed-off-by: Deepak Gupta <debug@rivosinc.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-18-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Expose a new register type NT_RISCV_USER_CFI for risc-v cfi status and state. Intentionally both landing pad and shadow stack status and state are rolled into cfi state. Creating two different NT_RISCV_USER_XXX would not be useful and wastage of a note type. Enabling, disabling and locking of feature is not allowed via ptrace set interface. However setting `elp` state or setting shadow stack pointer are allowed via ptrace set interface . It is expected `gdb` might have use to fixup `elp` state or `shadow stack` pointer. Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-19-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Adding enumeration of zicfilp and zicfiss extensions in hwprobe syscall. Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-20-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
This commit adds a kernel command line option using which user cfi can be disabled. User backward cfi and forward cfi can be enabled independently. Kernel command line parameter "riscv_nousercfi" can take below values: - "all" : Disable forward and backward cfi both. - "bcfi" : Disable backward cfi. - "fcfi" : Disable forward cfi Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-21-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Kernel will have to perform shadow stack operations on user shadow stack. Like during signal delivery and sigreturn, shadow stack token must be created and validated respectively. Thus shadow stack access for kernel must be enabled. In future when kernel shadow stacks are enabled for linux kernel, it must be enabled as early as possible for better coverage and prevent imbalance between regular stack and shadow stack. After `relocate_enable_mmu` has been done, this is as early as possible it can enabled. Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-22-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
user mode tasks compiled with zicfilp may call indirectly into vdso (like hwprobe indirect calls). Add landing pad compile support in vdso. vdso with landing pad in it will be nop for tasks which have not enabled landing pad. This patch allows to run user mode tasks with cfi eanbled and do no harm. Future work can be done on this to do below - labeled landing pad on vdso functions (whenever labeling support shows up in gnu-toolchain) - emit shadow stack instructions only in vdso compiled objects as part of kernel compile. Signed-off-by: Jim Shu <jim.shu@sifive.com> Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-23-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
This patch creates a config for shadow stack support and landing pad instr support. Shadow stack support and landing instr support can be enabled by selecting `CONFIG_RISCV_USER_CFI`. Selecting `CONFIG_RISCV_USER_CFI` wires up path to enumerate CPU support and if cpu support exists, kernel will support cpu assisted user mode cfi. If CONFIG_RISCV_USER_CFI is selected, select `ARCH_USES_HIGH_VMA_FLAGS`, `ARCH_HAS_USER_SHADOW_STACK` and DYNAMIC_SIGFRAME for riscv. Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-24-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Adding documentation on landing pad aka indirect branch tracking on riscv and kernel interfaces exposed so that user tasks can enable it. Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-25-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Adding documentation on shadow stack for user mode on riscv and kernel interfaces exposed so that user tasks can enable it. Reviewed-by: Zong Li <zong.li@sifive.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-26-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Adds kselftest for RISC-V control flow integrity implementation for user mode. There is not a lot going on in kernel for enabling landing pad for user mode. cfi selftest are intended to be compiled with zicfilp and zicfiss enabled compiler. Thus kselftest simply checks if landing pad / shadow stack for the process are enabled or not and executes ptrace selftests on cfi. selftest then register a signal handler for SIGSEGV. Any control flow violation are reported as SIGSEGV with si_code = SEGV_CPERR. Test will fail on receiving any SEGV_CPERR. Shadow stack part has more changes in kernel and thus there are separate tests for that - Exercise `map_shadow_stack` syscall - `fork` test to make sure COW works for shadow stack pages - gup tests Kernel uses FOLL_FORCE when access happens to memory via /proc/<pid>/mem. Not breaking that for shadow stack. - signal test. Make sure signal delivery results in token creation on shadow stack and consumes (and verifies) token on sigreturn - shadow stack protection test. attempts to write using regular store instruction on shadow stack memory must result in access faults - ptrace test: adds landing pad violation, clears ELP and continues In case toolchain doesn't support cfi extension, cfi kselftest wont get built. Test outut ========== """ TAP version 13 1..5 This is to ensure shadow stack is indeed enabled and working This is to ensure shadow stack is indeed enabled and working ok 1 shstk fork test ok 2 map shadow stack syscall ok 3 shadow stack gup tests ok 4 shadow stack signal tests ok 5 memory protections of shadow stack memory """ Signed-off-by: Deepak Gupta <debug@rivosinc.com> Suggested-by: Charlie Jenkins <charlie@rivosinc.com> Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-27-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Deepak Gupta <debug@rivosinc.com> says: Basics and overview =================== Software with larger attack surfaces (e.g. network facing apps like databases, browsers or apps relying on browser runtimes) suffer from memory corruption issues which can be utilized by attackers to bend control flow of the program to eventually gain control (by making their payload executable). Attackers are able to perform such attacks by leveraging call-sites which rely on indirect calls or return sites which rely on obtaining return address from stack memory. To mitigate such attacks, risc-v extension zicfilp enforces that all indirect calls must land on a landing pad instruction `lpad` else cpu will raise software check exception (a new cpu exception cause code on riscv). Similarly for return flow, risc-v extension zicfiss extends architecture with - `sspush` instruction to push return address on a shadow stack - `sspopchk` instruction to pop return address from shadow stack and compare with input operand (i.e. return address on stack) - `sspopchk` to raise software check exception if comparision above was a mismatch - Protection mechanism using which shadow stack is not writeable via regular store instructions More information an details can be found at extensions github repo [1]. Equivalent to landing pad (zicfilp) on x86 is `ENDBRANCH` instruction in Intel CET [3] and branch target identification (BTI) [4] on arm. Similarly x86's Intel CET has shadow stack [5] and arm64 has guarded control stack (GCS) [6] which are very similar to risc-v's zicfiss shadow stack. x86 and arm64 support for user mode shadow stack is already in mainline. Kernel awareness for user control flow integrity ================================================ This series picks up Samuel Holland's envcfg changes [2] as well. So if those are being applied independently, they should be removed from this series. Enabling: In order to maintain compatibility and not break anything in user mode, kernel doesn't enable control flow integrity cpu extensions on binary by default. Instead exposes a prctl interface to enable, disable and lock the shadow stack or landing pad feature for a task. This allows userspace (loader) to enumerate if all objects in its address space are compiled with shadow stack and landing pad support and accordingly enable the feature. Additionally if a subsequent `dlopen` happens on a library, user mode can take a decision again to disable the feature (if incoming library is not compiled with support) OR terminate the task (if user mode policy is strict to have all objects in address space to be compiled with control flow integirty cpu feature). prctl to enable shadow stack results in allocating shadow stack from virtual memory and activating for user address space. x86 and arm64 are also following same direction due to similar reason(s). clone/fork: On clone and fork, cfi state for task is inherited by child. Shadow stack is part of virtual memory and is a writeable memory from kernel perspective (writeable via a restricted set of instructions aka shadow stack instructions) Thus kernel changes ensure that this memory is converted into read-only when fork/clone happens and COWed when fault is taken due to sspush, sspopchk or ssamoswap. In case `CLONE_VM` is specified and shadow stack is to be enabled, kernel will automatically allocate a shadow stack for that clone call. map_shadow_stack: x86 introduced `map_shadow_stack` system call to allow user space to explicitly map shadow stack memory in its address space. It is useful to allocate shadow for different contexts managed by a single thread (green threads or contexts) risc-v implements this system call as well. signal management: If shadow stack is enabled for a task, kernel performs an asynchronous control flow diversion to deliver the signal and eventually expects userspace to issue sigreturn so that original execution can be resumed. Even though resume context is prepared by kernel, it is in user space memory and is subject to memory corruption and corruption bugs can be utilized by attacker in this race window to perform arbitrary sigreturn and eventually bypass cfi mechanism. Another issue is how to ensure that cfi related state on sigcontext area is not trampled by legacy apps or apps compiled with old kernel headers. In order to mitigate control-flow hijacting, kernel prepares a token and place it on shadow stack before signal delivery and places address of token in sigcontext structure. During sigreturn, kernel obtains address of token from sigcontext struture, reads token from shadow stack and validates it and only then allow sigreturn to succeed. Compatiblity issue is solved by adopting dynamic sigcontext management introduced for vector extension. This series re-factor the code little bit to allow future sigcontext management easy (as proposed by Andy Chiu from SiFive) config and compilation: Introduce a new risc-v config option `CONFIG_RISCV_USER_CFI`. Selecting this config option picks the kernel support for user control flow integrity. This optin is presented only if toolchain has shadow stack and landing pad support. And is on purpose guarded by toolchain support. Reason being that eventually vDSO also needs to be compiled in with shadow stack and landing pad support. vDSO compile patches are not included as of now because landing pad labeling scheme is yet to settle for usermode runtime. To get more information on kernel interactions with respect to zicfilp and zicfiss, patch series adds documentation for `zicfilp` and `zicfiss` in following: Documentation/arch/riscv/zicfiss.rst Documentation/arch/riscv/zicfilp.rst References ========== [1] - https://github.com/riscv/riscv-cfi [2] - https://lore.kernel.org/all/20240814081126.956287-1-samuel.holland@sifive.com/ [3] - https://lwn.net/Articles/889475/ [4] - https://developer.arm.com/documentation/109576/0100/Branch-Target-Identification [5] - https://www.intel.com/content/dam/develop/external/us/en/documents/catc17-introduction-intel-cet-844137.pdf [6] - https://lwn.net/Articles/940403/ * patches from https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-0-09b468d7beab@rivosinc.com: (27 commits) kselftest/riscv: kselftest for user mode cfi riscv: Documentation for shadow stack on riscv riscv: Documentation for landing pad / indirect branch tracking riscv: create a config for shadow stack and landing pad instr support arch/riscv: compile vdso with landing pad and shadow stack note riscv: enable kernel access to shadow stack memory via FWFT sbi call riscv: kernel command line option to opt out of user cfi riscv/hwprobe: zicfilp / zicfiss enumeration in hwprobe riscv/ptrace: riscv cfi status and state via ptrace and in core files riscv/kernel: update __show_regs to print shadow stack register riscv/signal: save and restore of shadow stack for signal riscv: signal: abstract header saving for setup_sigcontext riscv/traps: Introduce software check exception and uprobe handling riscv: Implements arch agnostic indirect branch tracking prctls prctl: arch-agnostic prctl for indirect branch tracking riscv: Implements arch agnostic shadow stack prctls riscv/shstk: If needed allocate a new shadow stack on clone riscv/mm: Implement map_shadow_stack() syscall riscv/mm: write protect and shadow stack riscv/mm: teach pte_mkwrite to manufacture shadow stack PTEs ... Link: https://lore.kernel.org/r/20250731-v5_user_cfi_series-v19-0-09b468d7beab@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
AlexGhiti
force-pushed
the
alex-for-next-cfi_manual
branch
from
97f5795 to
e8f6f1c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.