Rust KCFI support
Alice Ryhl
reviewerv2 cover Rust KCFI support
The control flow integrity (kCFI) sanitizer is an important sanitizer that is often used in production. This patch series makes it possible to use kCFI and Rust together.
The second patch in this series depends on the next version of [1], which Miguel will send soon. It also depends on [2].
---
Changes in v2:
- Fix for FineIBT.
- Add more info to commit messages and config descrptions.
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Alice Ryhl (1):
cfi: add CONFIG_CFI_ICALL_NORMALIZE_INTEGERS
Matthew Maurer (1):
rust: cfi: add support for CFI_CLANG with Rust
Makefile | 10 ++++++++++ arch/Kconfig | 16 ++++++++++++++++ arch/x86/Makefile | 4 ++++ init/Kconfig | 4 +++- rust/Makefile | 2 +- scripts/generate_rust_target.rs | 1 + 6 files changed, 35 insertions(+), 2 deletions(-)
---
base-commit: 8718bc07faa6ddf4f7335a12e4cdd4ffc796bbd8
change-id: 20240725-kcfi-c592898e2bfb
Best regards,
Signature · 1 line
Alice Ryhl <[email protected]>
Alice Ryhl
reviewerPATCH v2 1/2 cfi: add CONFIG_CFI_ICALL_NORMALIZE_INTEGERS
Introduce a Kconfig option for enabling the experimental option to normalize integer types. This ensures that integer types of the same size and signedness are considered compatible by the Control Flow Integrity sanitizer.
The security impact of this flag is minimal. When Sami Tolvanen looked into it, he found that integer normalization reduced the number of unique type hashes in the kernel by ~1%, which is acceptable.
This option exists for compatibility with Rust, as C and Rust do not have the same set of integer types. There are cases where C has two different integer types of the same size and signedness, but Rust only has one integer type of that size and signedness. When Rust calls into C functions using such types in their signature, this results in CFI failures. One example is 'unsigned long long' and 'unsigned long' which are both 64-bit on LP64 targets, so on those targets this flag will give both types the same CFI tag.
This flag changes the ABI heavily. It is not applied automatically when CONFIG_RUST is turned on to make sure that the CONFIG_RUST option does not change the ABI of C code. For example, some build may need to make other changes atomically with toggling this flag. Having it be a separate option makes it possible to first turn on normalized integer tags, and then later turn on CONFIG_RUST.
Similarly, when turning on CONFIG_RUST in a build, you may need a few attempts where the RUST=y commit gets reverted a few times. It is inconvenient if reverting RUST=y also requires reverting the changes you made to support normalized integer tags.
To avoid having this flag impact builds that don't care about this, the next patch in this series will make CONFIG_RUST turn on this option using select rather than depends on.
--- Makefile | 3 +++ arch/Kconfig | 16 ++++++++++++++++ 2 files changed, 19 insertions(+)
Makefile +3 -0
@@ -952,6 +952,9 @@ endif ifdef CONFIG_CFI_CLANG CC_FLAGS_CFI := -fsanitize=kcfi +ifdef CONFIG_CFI_ICALL_NORMALIZE_INTEGERS + CC_FLAGS_CFI += -fsanitize-cfi-icall-experimental-normalize-integers +endif KBUILD_CFLAGS += $(CC_FLAGS_CFI) export CC_FLAGS_CFI endif
arch/Kconfig +16 -0
@@ -826,6 +826,22 @@ config CFI_CLANG https://clang.llvm.org/docs/ControlFlowIntegrity.html +config CFI_ICALL_NORMALIZE_INTEGERS + bool "Normalize CFI tags for integers" + depends on CFI_CLANG + depends on $(cc-option,-fsanitize=kcfi -fsanitize-cfi-icall-experimental-normalize-integers) + help + This option normalizes the CFI tags for integer types so that all + integer types of the same size and signedness receive the same CFI + tag. + + The option is separate from CONFIG_RUST because it affects the ABI. + When working with build systems that care about the ABI, it is + convenient to be able to turn on this flag first, before Rust is + turned on. + + This option is necessary for using CFI with Rust. If unsure, say N. + config CFI_PERMISSIVE bool "Use CFI in permissive mode" depends on CFI_CLANG
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
Alice Ryhl
reviewerPATCH v2 2/2 rust: cfi: add support for CFI_CLANG with Rust
In reply to Alice RyhlFrom: Matthew Maurer <[email protected]>
Make it possible to use the Control Flow Integrity (CFI) sanitizer when Rust is enabled. Enabling CFI with Rust requires that CFI is configured to normalize integer types so that all integer types of the same size and signedness are compatible under CFI.
Rust and C use the same LLVM backend for code generation, so Rust KCFI is compatible with the KCFI used in the kernel for C. In the case of FineIBT, CFI also depends on -Zpatchable-function-entry for rewriting the function prolouge, so we set that flag for Rust as well. The flag for FineIBT requires rustc 1.80.0 or later, so include a Kconfig requirement for that.
Enabling Rust will select CFI_ICALL_NORMALIZE_INTEGERS because the flag is required to use Rust with CFI. Using select rather than depends on avoids the case where Rust is not visible in menuconfig due to CFI_ICALL_NORMALIZE_INTEGERS not being enabled. One disadvantage of select is that RUST must depends on all of the things that CFI_ICALL_NORMALIZE_INTEGERS depends on to avoid invalid configurations.
Alice has been using KCFI on her phone for several months, so it is reasonably well tested on arm64.
--- Makefile | 7 +++++++ arch/x86/Makefile | 4 ++++ init/Kconfig | 4 +++- rust/Makefile | 2 +- scripts/generate_rust_target.rs | 1 + 5 files changed, 16 insertions(+), 2 deletions(-)
Makefile +7 -0
@@ -955,6 +955,13 @@ CC_FLAGS_CFI := -fsanitize=kcfi ifdef CONFIG_CFI_ICALL_NORMALIZE_INTEGERS CC_FLAGS_CFI += -fsanitize-cfi-icall-experimental-normalize-integers endif +ifdef CONFIG_RUST + # Always pass -Zsanitizer-cfi-normalize-integers as CONFIG_RUST selects + # CONFIG_CFI_ICALL_NORMALIZE_INTEGERS. + RUSTC_FLAGS_CFI := -Zsanitizer=kcfi -Zsanitizer-cfi-normalize-integers + KBUILD_RUSTFLAGS += $(RUSTC_FLAGS_CFI) + export RUSTC_FLAGS_CFI +endif KBUILD_CFLAGS += $(CC_FLAGS_CFI) export CC_FLAGS_CFI endif
arch/x86/Makefile +4 -0
@@ -237,6 +237,10 @@ ifdef CONFIG_CALL_PADDING PADDING_CFLAGS := -fpatchable-function-entry=$(CONFIG_FUNCTION_PADDING_BYTES),$(CONFIG_FUNCTION_PADDING_BYTES) KBUILD_CFLAGS += $(PADDING_CFLAGS) export PADDING_CFLAGS + +PADDING_RUSTFLAGS := -Zpatchable-function-entry=$(CONFIG_FUNCTION_PADDING_BYTES),$(CONFIG_FUNCTION_PADDING_BYTES) +KBUILD_RUSTFLAGS += $(PADDING_RUSTFLAGS) +export PADDING_RUSTFLAGS endif KBUILD_LDFLAGS += -m elf_$(UTS_MACHINE)
init/Kconfig +3 -1
@@ -1905,11 +1905,13 @@ config RUST bool "Rust support" depends on HAVE_RUST depends on RUST_IS_AVAILABLE - depends on !CFI_CLANG depends on !MODVERSIONS depends on !GCC_PLUGINS depends on !RANDSTRUCT depends on !DEBUG_INFO_BTF || PAHOLE_HAS_LANG_EXCLUDE + depends on !CFI_CLANG || RUSTC_VERSION >= 107900 && $(cc-option,-fsanitize=kcfi -fsanitize-cfi-icall-experimental-normalize-integers) + select CFI_ICALL_NORMALIZE_INTEGERS if CFI_CLANG + depends on !FINEIBT || RUSTC_VERSION >= 108000 help Enables Rust support in the kernel.
rust/Makefile +1 -1
@@ -305,7 +305,7 @@ $(obj)/bindings/bindings_helpers_generated.rs: $(src)/helpers.c FORCE quiet_cmd_exports = EXPORTS $@ cmd_exports = \ $(NM) -p --defined-only $< \ - | awk '/ (T|R|D) / {printf "EXPORT_SYMBOL_RUST_GPL(%s);\n",$$3}' > $@ + | awk '$$2~/(T|R|D)/ && $$3!~/__cfi/ {printf "EXPORT_SYMBOL_RUST_GPL(%s);\n",$$3}' > $@ $(obj)/exports_core_generated.h: $(obj)/core.o FORCE $(call if_changed,exports)
scripts/generate_rust_target.rs +1 -0
@@ -192,6 +192,7 @@ fn main() { } ts.push ("features" , features); ts.push ("llvm-target" ,"x86_64-linux-gnu" ); + ts.push ("supported-sanitizers" , ["kcfi" ]); ts.push ("target-pointer-width" ,"64" ); }else if cfg.has ("X86_32" ) {// This only works on UML, as i386 otherwise needs regparm support in rustc
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
Alice Ryhl
reviewerPATCH 1/1 rust: cfi: add support for CFI_CLANG with Rust
From: Matthew Maurer <[email protected]>
Make it possible to use the Control Flow Integrity (CFI) sanitizer when Rust is enabled. Enabling CFI with Rust requires that CFI is configured to normalize integer types so that all integer types of the same size and signedness are compatible under CFI.
Rust and C use the same LLVM backend for code generation, so Rust KCFI is compatible with the KCFI used in the kernel for C. In the case of FineIBT, CFI also depends on -Zpatchable-function-entry for rewriting the function prolouge, so we set that flag for Rust as well. The flag for FineIBT requires rustc 1.80.0 or later, so include a Kconfig requirement for that.
Enabling Rust will select CFI_ICALL_NORMALIZE_INTEGERS because the flag is required to use Rust with CFI. Using select rather than depends on avoids the case where Rust is not visible in menuconfig due to CFI_ICALL_NORMALIZE_INTEGERS not being enabled. One disadvantage of select is that RUST must depends on all of the things that CFI_ICALL_NORMALIZE_INTEGERS depends on to avoid invalid configurations.
Alice has been using KCFI on her phone for several months, so it is reasonably well tested on arm64.
---
This is an alternate version that shows how to resolve the conflict with the "rust: fix export of bss symbols" patch [1].
As for the conflict with the KASAN patchset [2], it should be resolved by adding both strings to the supported-sanitizers list in sorted order.
[1]: https://lore.kernel.org/r/[email protected]
[2]: https://lore.kernel.org/r/[email protected]
Makefile | 7 +++++++ arch/x86/Makefile | 4 ++++ init/Kconfig | 4 +++- rust/Makefile | 2 +- scripts/generate_rust_target.rs | 1 + 5 files changed, 16 insertions(+), 2 deletions(-)
Makefile +7 -0
@@ -955,6 +955,13 @@ CC_FLAGS_CFI := -fsanitize=kcfi ifdef CONFIG_CFI_ICALL_NORMALIZE_INTEGERS CC_FLAGS_CFI += -fsanitize-cfi-icall-experimental-normalize-integers endif +ifdef CONFIG_RUST + # Always pass -Zsanitizer-cfi-normalize-integers as CONFIG_RUST selects + # CONFIG_CFI_ICALL_NORMALIZE_INTEGERS. + RUSTC_FLAGS_CFI := -Zsanitizer=kcfi -Zsanitizer-cfi-normalize-integers + KBUILD_RUSTFLAGS += $(RUSTC_FLAGS_CFI) + export RUSTC_FLAGS_CFI +endif KBUILD_CFLAGS += $(CC_FLAGS_CFI) export CC_FLAGS_CFI endif
arch/x86/Makefile +4 -0
@@ -237,6 +237,10 @@ ifdef CONFIG_CALL_PADDING PADDING_CFLAGS := -fpatchable-function-entry=$(CONFIG_FUNCTION_PADDING_BYTES),$(CONFIG_FUNCTION_PADDING_BYTES) KBUILD_CFLAGS += $(PADDING_CFLAGS) export PADDING_CFLAGS + +PADDING_RUSTFLAGS := -Zpatchable-function-entry=$(CONFIG_FUNCTION_PADDING_BYTES),$(CONFIG_FUNCTION_PADDING_BYTES) +KBUILD_RUSTFLAGS += $(PADDING_RUSTFLAGS) +export PADDING_RUSTFLAGS endif KBUILD_LDFLAGS += -m elf_$(UTS_MACHINE)
init/Kconfig +3 -1
@@ -1905,11 +1905,13 @@ config RUST bool "Rust support" depends on HAVE_RUST depends on RUST_IS_AVAILABLE - depends on !CFI_CLANG depends on !MODVERSIONS depends on !GCC_PLUGINS depends on !RANDSTRUCT depends on !DEBUG_INFO_BTF || PAHOLE_HAS_LANG_EXCLUDE + depends on !CFI_CLANG || RUSTC_VERSION >= 107900 && $(cc-option,-fsanitize=kcfi -fsanitize-cfi-icall-experimental-normalize-integers) + select CFI_ICALL_NORMALIZE_INTEGERS if CFI_CLANG + depends on !FINEIBT || RUSTC_VERSION >= 108000 help Enables Rust support in the kernel.
rust/Makefile +1 -1
@@ -305,7 +305,7 @@ $(obj)/bindings/bindings_helpers_generated.rs: $(src)/helpers.c FORCE quiet_cmd_exports = EXPORTS $@ cmd_exports = \ $(NM) -p --defined-only $< \ - | awk '/ (T|R|D|B) / {printf "EXPORT_SYMBOL_RUST_GPL(%s);\n",$$3}' > $@ + | awk '$$2~/(T|R|D|B)/ && $$3!~/__cfi/ {printf "EXPORT_SYMBOL_RUST_GPL(%s);\n",$$3}' > $@ $(obj)/exports_core_generated.h: $(obj)/core.o FORCE $(call if_changed,exports)
scripts/generate_rust_target.rs +1 -0
@@ -192,6 +192,7 @@ fn main() { } ts.push ("features" , features); ts.push ("llvm-target" ,"x86_64-linux-gnu" ); + ts.push ("supported-sanitizers" , ["kcfi" ]); ts.push ("target-pointer-width" ,"64" ); }else if cfg.has ("X86_32" ) {// This only works on UML, as i386 otherwise needs regparm support in rustc
Signature · 1 line
2.46.0.184.g6999bdac58-goog
Re: [PATCH v2b] rust: cfi: add support for CFI_CLANG with Rust
arch/x86/Makefile ifdef CONFIG_CALL_PADDING · line 243
Quoted 4 lines · Alice Ryhl
arch/x86/Makefile +3
-0+PADDING_RUSTFLAGS := -Zpatchable-function-entry=$(CONFIG_FUNCTION_PADDING_BYTES),$(CONFIG_FUNCTION_PADDING_BYTES) +KBUILD_RUSTFLAGS += $(PADDING_RUSTFLAGS) +export PADDING_RUSTFLAGS
It is possible to have CALL_PADDING && !FINEIBT, which means one can trigger a build error if the compiler is not recent enough. Should we test for CALL_PADDING here?
Quoted 1 line
+ depends on !FINEIBT ||
RUSTC_VERSION>= 108000
Cheers,
Miguel
Re: [PATCH v2b] rust: cfi: add support for CFI_CLANG with Rust
Quoted 4 lines · Miguel Ojeda
It is possible to have
CALL_PADDING&& !FINEIBT, which means one can trigger a build error if the compiler is not recent enough. Should we test forCALL_PADDINGhere?
Alice confirmed offline that she agrees, so I just changed the requirement in rust-next.
[ Replaced !FINEIBT requirement with !CALL_PADDING to prevent
a build error on older Rust compilers. Fixed typo. - Miguel ]
Cheers,
Miguel
Re: [PATCH v2b] rust: cfi: add support for CFI_CLANG with Rust
Quoted 3 lines · Miguel Ojeda
[ Replaced
!FINEIBTrequirement with!CALL_PADDINGto prevent
a build error on older Rust compilers. Fixed typo. - Miguel ]
I also noticed we need Rust 1.81.0 for the flag rather than 1.80.0 -- I will send a patch later and/or rebase.
Cheers,
Miguel
Quoted 5 lines · Alice Ryhl
Alice Ryhl (1):
cfi: addCONFIG_CFI_ICALL_NORMALIZE_INTEGERSMatthew Maurer (1):
rust: cfi: add support forCFI_CLANGwith Rust
Hi Alice,
Quoted 17 lines · Alice Ryhl
The control flow integrity (kCFI) sanitizer is an important sanitizer that is often used in production. This patch series makes it possible to use kCFI and Rust together.
The second patch in this series depends on the next version of [1], which Miguel will send soon. It also depends on [2].
Signed-off-by Alice Ryhl <[email protected]>---
Changes in v2:
- Fix for FineIBT.
- Add more info to commit messages and config descrptions.
- Link to v1: https://lore.kernel.org/r/[email protected]
Quoted 6 lines
---
Alice Ryhl (1):
cfi: addCONFIG_CFI_ICALL_NORMALIZE_INTEGERSMatthew Maurer (1):
rust: cfi: add support forCFI_CLANGwith Rust
Thanks for sorting this out!
Sami
Quoted 23 lines · Alice Ryhl
The control flow integrity (kCFI) sanitizer is an important sanitizer that is often used in production. This patch series makes it possible to use kCFI and Rust together.
The second patch in this series depends on the next version of [1], which Miguel will send soon. It also depends on [2].
Signed-off-by Alice Ryhl <[email protected]>---
Changes in v2:
- Fix for FineIBT.
- Add more info to commit messages and config descrptions.
- Link to v1: https://lore.kernel.org/r/[email protected]---
Alice Ryhl (1):
cfi: addCONFIG_CFI_ICALL_NORMALIZE_INTEGERSMatthew Maurer (1):
rust: cfi: add support forCFI_CLANGwith Rust
Quoted 7 lines
Makefile | 10 ++++++++++ arch/Kconfig | 16 ++++++++++++++++ arch/x86/Makefile | 4 ++++ init/Kconfig | 4 +++- rust/Makefile | 2 +- scripts/generate_rust_target.rs | 1 + 6 files changed, 35 insertions(+), 2 deletions(-)
This is great to have! I assume this will go via the Rust tree, so:
Signature · 1 line
Kees Cook
Quoted 23 lines · Alice Ryhl
The control flow integrity (kCFI) sanitizer is an important sanitizer that is often used in production. This patch series makes it possible to use kCFI and Rust together.
The second patch in this series depends on the next version of [1], which Miguel will send soon. It also depends on [2].
Signed-off-by Alice Ryhl <[email protected]>---
Changes in v2:
- Fix for FineIBT.
- Add more info to commit messages and config descrptions.
- Link to v1: https://lore.kernel.org/r/[email protected]---
Alice Ryhl (1):
cfi: addCONFIG_CFI_ICALL_NORMALIZE_INTEGERSMatthew Maurer (1):
rust: cfi: add support forCFI_CLANGwith Rust
Quoted 7 lines
Makefile | 10 ++++++++++ arch/Kconfig | 16 ++++++++++++++++ arch/x86/Makefile | 4 ++++ init/Kconfig | 4 +++- rust/Makefile | 2 +- scripts/generate_rust_target.rs | 1 + 6 files changed, 35 insertions(+), 2 deletions(-)
To test this patch: I started from v6.11-rc3 tag and applied the new version of Miguel's RUSTC_VERSION_TEXT patch [1] and the Support Arrays in Target JSON patch [2], before applying this patch. I am on Rust's beta channel and Clang/LLVM 19.
I also built a v6.11-rc3 kernel without these patches to establish a baseline for LKDTM output in dmesg when testing CFI [3]. I built the v6.11-rc3 kernel by starting with an x86_64_defconfig, then enabling CFI_CLANG, CFI_PERMISSIVE, and LKDTM.
When applying [1], there was an patch does not apply error. I had to manually change the init/Kconfig RUSTC_VERSION_TEXT to Miguel's change in [1]. No issues encountered applying [2] afterwards. Similarly, was able to automerge this patch without any issues.
Then I built the kernel starting with x86_64_defconfig and enabling: RUST, CFI_CLANG, CFI_ICALL_NORMALIZE_INTEGERS, CFI_PERMISSIVE and LKDTM. Compiled the kernel, load into qemu with Busybox rootfs, test CFI within LKDTM per Kees's blog [3]. I saw the same expected behavior from LKDTM after applying these patches when compared with the behavior from LKDTM on a v6.11-rc3 build without these patches.
--
Gatlin Newhouse
Quoted 23 lines · Gatlin Newhouse
To test this patch: I started from v6.11-rc3 tag and applied the new version of Miguel's
RUSTC_VERSION_TEXTpatch [1] and the Support Arrays in Target JSON patch [2], before applying this patch. I am on Rust's beta channel and Clang/LLVM 19.I also built a v6.11-rc3 kernel without these patches to establish a baseline for LKDTM output in dmesg when testing CFI [3]. I built the v6.11-rc3 kernel by starting with an
x86_64_defconfig, then enablingCFI_CLANG,CFI_PERMISSIVE, and LKDTM.When applying [1], there was an patch does not apply error. I had to manually change the init/Kconfig
RUSTC_VERSION_TEXTto Miguel's change in [1]. No issues encountered applying [2] afterwards. Similarly, was able to automerge this patch without any issues.Then I built the kernel starting with
x86_64_defconfigand enabling: RUST,CFI_CLANG,CFI_ICALL_NORMALIZE_INTEGERS,CFI_PERMISSIVEand LKDTM. Compiled the kernel, load into qemu with Busybox rootfs, test CFI within LKDTM per Kees's blog [3]. I saw the same expected behavior from LKDTM after applying these patches when compared with the behavior from LKDTM on a v6.11-rc3 build without these patches.
Quoted 5 lines
Tested-by Gatlin Newhouse <[email protected]>
I wanted to point out that this is an excellent testing report.
Cheers,
Miguel
Miguel Ojeda
AppliedQuoted 11 lines · Alice Ryhl
The control flow integrity (kCFI) sanitizer is an important sanitizer that is often used in production. This patch series makes it possible to use kCFI and Rust together.
The second patch in this series depends on the next version of [1], which Miguel will send soon. It also depends on [2].
Signed-off-by Alice Ryhl <[email protected]>
Applied to rust-next -- thanks everyone!
Cheers,
Miguel
Quoted 3 lines · Alice Ryhl
The control flow integrity (kCFI) sanitizer is an important sanitizer that is often used in production. This patch series makes it possible to use kCFI and Rust together.
So about this -- there's a proposal for a modification to kCFI here:
https://github.com/llvm/llvm-project/pull/117121
And Sami notes that this would break this Rust thing. Assuming all the relevant crabs are present on this thread, could you please comment?
Alice Ryhl
reviewerQuoted 6 lines · Peter Zijlstra
Quoted 3 lines · Alice Ryhl
The control flow integrity (kCFI) sanitizer is an important sanitizer that is often used in production. This patch series makes it possible to use kCFI and Rust together.
Quoted 6 lines
So about this -- there's a proposal for a modification to kCFI here:
https://github.com/llvm/llvm-project/pull/117121
And Sami notes that this would break this Rust thing. Assuming all the relevant crabs are present on this thread, could you please comment?
Thanks for sharing this link. I'll leave a comment.
Alice
Quoted 7 lines · Alice Ryhl
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Alice Ryhl
The control flow integrity (kCFI) sanitizer is an important sanitizer that is often used in production. This patch series makes it possible to use kCFI and Rust together.
Quoted 8 lines
Quoted 6 lines
So about this -- there's a proposal for a modification to kCFI here:
https://github.com/llvm/llvm-project/pull/117121
And Sami notes that this would break this Rust thing. Assuming all the relevant crabs are present on this thread, could you please comment?
Thanks for sharing this link. I'll leave a comment.
Thanks!