Skip to contentSkip to conversation

x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

PATCH Applied Paweł Anikiel 7 participants 45 messages Apr 10, 2025 – May 9, 2025

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST

Calling core::fmt::write() from rust code while FineIBT is enabled results in a kernel panic:

[ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
[ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
[ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
[ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
[ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
[ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
[ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
[ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
[ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
[ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
[ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
[ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
[ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
[ 4614.336777] PKRU: 55555554
[ 4614.339786] Call Trace:
[ 4614.342524]  <TASK>
[ 4614.344867]  ? __die_body+0x69/0xb0
[ 4614.348786]  ? die+0xa9/0xd0
[ 4614.352000]  ? do_trap+0x89/0x160
[ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.360413]  ? handle_invalid_op+0x69/0x90
[ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.369654]  ? exc_invalid_op+0x36/0x60
[ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
[ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
[ 4614.388005]  exc_control_protection+0x49/0x70
[ 4614.392871]  asm_exc_control_protection+0x2b/0x60
[ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
[ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
[ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
[ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
[ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
[ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
[ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
[ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
[ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
[ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250

This happens because core::fmt::write() calls core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

library/core/src/fmt/rt.rs:

171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {

This causes a Control Protection exception, because FineIBT has sealed off the original function's endbr64.

This makes rust currently incompatible with FineIBT. Add a Kconfig dependency that prevents FineIBT from getting turned on by default if rust is enabled.

---
 arch/x86/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/Kconfig +1 -1
index 4b9f378e05f6..b31811a92286 100644
@@ -2367,7 +2367,7 @@ config STRICT_SIGALTSTACK_SIZE
 
 config CFI_AUTO_DEFAULT
 	bool "Attempt to use FineIBT by default at boot time"
-	depends on FINEIBT
+	depends on FINEIBT && !RUST
 	default y
 	help
 	  Attempt to use FineIBT by default at boot time. If enabled,
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
maintainer NAK
#1 lore raw
Quoted 55 lines · Paweł Anikiel

Calling core::fmt::write() from rust code while FineIBT is enabled results in a kernel panic:

[ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
[ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
[ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
[ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
[ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
[ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
[ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
[ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
[ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
[ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
[ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
[ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
[ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
[ 4614.336777] PKRU: 55555554
[ 4614.339786] Call Trace:
[ 4614.342524]  <TASK>
[ 4614.344867]  ? __die_body+0x69/0xb0
[ 4614.348786]  ? die+0xa9/0xd0
[ 4614.352000]  ? do_trap+0x89/0x160
[ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.360413]  ? handle_invalid_op+0x69/0x90
[ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.369654]  ? exc_invalid_op+0x36/0x60
[ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
[ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
[ 4614.388005]  exc_control_protection+0x49/0x70
[ 4614.392871]  asm_exc_control_protection+0x2b/0x60
[ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
[ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
[ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
[ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
[ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
[ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
[ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
[ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
[ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
[ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250

This happens because core::fmt::write() calls core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

library/core/src/fmt/rt.rs:

171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
Quoted 6 lines

This causes a Control Protection exception, because FineIBT has sealed off the original function's endbr64.

This makes rust currently incompatible with FineIBT. Add a Kconfig dependency that prevents FineIBT from getting turned on by default if rust is enabled.

Why ?!, what's wrong with removing that no_sanitize() instead?

How is it still compatible with kCFI and not with FineIBT?

NAK

maintainer
#2 lore raw
Quoted 59 lines · Peter Zijlstra
Quoted 58 lines · Paweł Anikiel

Calling core::fmt::write() from rust code while FineIBT is enabled results in a kernel panic:

[ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
[ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
[ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
[ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
[ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
[ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
[ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
[ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
[ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
[ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
[ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
[ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
[ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
[ 4614.336777] PKRU: 55555554
[ 4614.339786] Call Trace:
[ 4614.342524]  <TASK>
[ 4614.344867]  ? __die_body+0x69/0xb0
[ 4614.348786]  ? die+0xa9/0xd0
[ 4614.352000]  ? do_trap+0x89/0x160
[ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.360413]  ? handle_invalid_op+0x69/0x90
[ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.369654]  ? exc_invalid_op+0x36/0x60
[ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
[ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
[ 4614.388005]  exc_control_protection+0x49/0x70
[ 4614.392871]  asm_exc_control_protection+0x2b/0x60
[ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
[ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
[ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
[ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
[ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
[ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
[ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
[ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
[ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
[ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250

This happens because core::fmt::write() calls core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

library/core/src/fmt/rt.rs:

171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {

This causes a Control Protection exception, because FineIBT has sealed off the original function's endbr64.

Quoted 7 lines
Quoted 3 lines

This makes rust currently incompatible with FineIBT. Add a Kconfig dependency that prevents FineIBT from getting turned on by default if rust is enabled.

Why ?!, what's wrong with removing that no_sanitize() instead?

How is it still compatible with kCFI and not with FineIBT?

FWIW, CFI violations of any kind are a no-no. They're not accepted in C and they're not accepted in Rust. This is clearly a Rust bug that needs to be fixed ASAP. Disabling CFI is not an option.

maintainer
#3 lore raw
Quoted 8 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 6 lines · Paweł Anikiel

Calling core::fmt::write() from rust code while FineIBT is enabled results in a kernel panic:

This happens because core::fmt::write() calls core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

library/core/src/fmt/rt.rs:

171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {

Miguel, I cannot find this code in the kernel tree. Is this again because Rust is not free-standing and relies on external code?

Can you please fix that. Building against external code that is not under our control is a problem.

#4 lore raw
Quoted 19 lines · Peter Zijlstra
Quoted 8 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 6 lines · Paweł Anikiel

Calling core::fmt::write() from rust code while FineIBT is enabled results in a kernel panic:

This happens because core::fmt::write() calls core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

library/core/src/fmt/rt.rs:

171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
Quoted 2 lines

Miguel, I cannot find this code in the kernel tree. Is this again because Rust is not free-standing and relies on external code?

The code I referenced is from here:
https://github.com/rust-lang/rust/blob/master/library/core/src/fmt/rt.rs#L139

reviewer
#5 lore raw
In reply to Peter Zijlstra
Quoted 19 lines · Peter Zijlstra
Quoted 8 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 6 lines · Paweł Anikiel

Calling core::fmt::write() from rust code while FineIBT is enabled results in a kernel panic:

This happens because core::fmt::write() calls core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

library/core/src/fmt/rt.rs:

171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
Quoted 5 lines

Miguel, I cannot find this code in the kernel tree. Is this again because Rust is not free-standing and relies on external code?

Can you please fix that. Building against external code that is not under our control is a problem.

It's from the core standard library, which is shipped together with the compiler.

Alice

#6 lore raw
In reply to Peter Zijlstra
Quoted 3 lines · Peter Zijlstra

Miguel, I cannot find this code in the kernel tree. Is this again because Rust is not free-standing and relies on external code?

Yes, this comes from the core standard library. If it requires a change, then it needs to be fixed upstream.

Quoted 2 lines

Can you please fix that. Building against external code that is not under our control is a problem.

The only way to fix it would be to vendor the standard library, for each Rust compiler version -- and there is one new every 6 weeks. Thus, quite a few copies of it. Or keep patches (one per version) that we apply on the fly.

Cheers,
Miguel

maintainer
#7 lore raw
Quoted 8 lines · Miguel Ojeda
Quoted 3 lines · Peter Zijlstra

Miguel, I cannot find this code in the kernel tree. Is this again because Rust is not free-standing and relies on external code?

Yes, this comes from the core standard library. If it requires a change, then it needs to be fixed upstream.

Quoted 7 lines
Quoted 2 lines

Can you please fix that. Building against external code that is not under our control is a problem.

The only way to fix it would be to vendor the standard library, for each Rust compiler version -- and there is one new every 6 weeks. Thus, quite a few copies of it. Or keep patches (one per version) that we apply on the fly.

New compiler can't build old core?

#8 lore raw
Quoted 2 lines · Peter Zijlstra

New compiler can't build old core?

No, that is not intended to work -- core is tied to the compiler.

Cheers,
Miguel

maintainer
#9 lore raw
Quoted 5 lines · Miguel Ojeda
Quoted 2 lines · Peter Zijlstra

New compiler can't build old core?

No, that is not intended to work -- core is tied to the compiler.

OMFG, you just gave me another reason to hate on this rust-bucket of a language :-/ What language can't even build an old version of its own runtime :-(

Anyway, given this core nonsense is now CFI unsound, it is no longer suitable to build the kernel. So either refrain from using part of the core that trips this, or get it fixed asap.

#10 lore raw
Quoted 4 lines · Peter Zijlstra

Anyway, given this core nonsense is now CFI unsound, it is no longer suitable to build the kernel. So either refrain from using part of the core that trips this, or get it fixed asap.

We discussed it a bit yesterday. Still, until it gets resolved one way or another, we should be avoiding people to build the kernel like that, no?

Cheers,
Miguel

maintainer
#11 lore raw
Quoted 9 lines · Miguel Ojeda
Quoted 4 lines · Peter Zijlstra

Anyway, given this core nonsense is now CFI unsound, it is no longer suitable to build the kernel. So either refrain from using part of the core that trips this, or get it fixed asap.

We discussed it a bit yesterday. Still, until it gets resolved one way or another, we should be avoiding people to build the kernel like that, no?

Remove the offending Rust code? Afaict from this github issue, it is just some formatting nonsense. Surely code can be adjusted to not use that?

#12 lore raw
Quoted 4 lines · Peter Zijlstra

Remove the offending Rust code? Afaict from this github issue, it is just some formatting nonsense. Surely code can be adjusted to not use that?

If you mean not using the formatting machinery from our side, then that is a major change -- we should just fix it upstream, really.

If you mean fixing it upstream, definitely, but we should still prevent people from building an invalid kernel, i.e. when Alice's PR or similar lands upstream, then we can relax the depends on based on the Rust version (which is something we have done for other bits).

Cheers,
Miguel

maintainer
#13 lore raw
Quoted 8 lines · Miguel Ojeda
Quoted 4 lines · Peter Zijlstra

Remove the offending Rust code? Afaict from this github issue, it is just some formatting nonsense. Surely code can be adjusted to not use that?

If you mean not using the formatting machinery from our side, then that is a major change -- we should just fix it upstream, really.

That is what I mean, yeah. I've no idea what is or is not common in this rust code. It still looks like line noise to me :/

But given that FineIBT isn't at all new, I was under the assumption that this was some shiny new rust driver that did something that hadn't been done before. And it could now not do that.

Quoted 4 lines

If you mean fixing it upstream, definitely, but we should still prevent people from building an invalid kernel, i.e. when Alice's PR or similar lands upstream, then we can relax the depends on based on the Rust version (which is something we have done for other bits).

So why wasn't any of this a problem when Rust enabled kCFI? Surely the testing back then included FineIBT. That has been in longer than rust's kcfi support (integer type confusion etc.).

reviewer
#14 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 5 lines · Miguel Ojeda

On Thu, Apr 10, 2025 at 3:57 PM Peter Zijlstra <[email protected]> wrote:
If you mean fixing it upstream, definitely, but we should still prevent people from building an invalid kernel, i.e. when Alice's PR or similar lands upstream, then we can relax the depends on based on the Rust version (which is something we have done for other bits).

So why wasn't any of this a problem when Rust enabled kCFI? Surely the testing back then included FineIBT. That has been in longer than rust's kcfi support (integer type confusion etc.).

I assume this has been a problem for a while. The testing we did with FineIBT must not have triggered this codepath. But Rust drivers have used the codepath for a long time.

Alice

reviewer
#15 lore raw
In reply to Peter Zijlstra
Quoted 11 lines · Peter Zijlstra
Quoted 5 lines · Miguel Ojeda
Quoted 2 lines · Peter Zijlstra

New compiler can't build old core?

No, that is not intended to work -- core is tied to the compiler.

OMFG, you just gave me another reason to hate on this rust-bucket of a language :-/ What language can't even build an old version of its own runtime :-(

I don't think there's anything unreasonable about having a place to put logic that depends on compiler internals.

Quoted 3 lines

Anyway, given this core nonsense is now CFI unsound, it is no longer suitable to build the kernel. So either refrain from using part of the core that trips this, or get it fixed asap.

I submitted a PR that I believe should fix it:
https://github.com/rust-lang/rust/pull/139632

Alice

maintainer
#16 lore raw
Quoted 7 lines · Alice Ryhl
Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Miguel Ojeda
Quoted 2 lines · Peter Zijlstra

New compiler can't build old core?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

No, that is not intended to work -- core is tied to the compiler.

OMFG, you just gave me another reason to hate on this rust-bucket of a language :-/ What language can't even build an old version of its own runtime :-(

I don't think there's anything unreasonable about having a place to put logic that depends on compiler internals.

Having compiler internals exposed to such an extend that programs can make use of them _is_ weird. It creates the whole problem of where to put it.

C++ doesn't have anything like this afaik. Esp. if you disable rtti and exception muck, you get a very minimal runtime. And even if you don't, it certainly doesn't break every 6 weeks. ABIs and all that.

Maybe I've been living in C/C++ land for too long, but you either expose a full language feature and then everybody can use it, runtime, libraries and user code alike, or you don't.

Quoted 6 lines
Quoted 3 lines

Anyway, given this core nonsense is now CFI unsound, it is no longer suitable to build the kernel. So either refrain from using part of the core that trips this, or get it fixed asap.

I submitted a PR that I believe should fix it:
https://github.com/rust-lang/rust/pull/139632

Thanks!

#17 lore raw
Quoted 4 lines · Peter Zijlstra

Maybe I've been living in C/C++ land for too long, but you either expose a full language feature and then everybody can use it, runtime, libraries and user code alike, or you don't.

I find it unlikely that C++ stdlibs happen to build and work properly with compilers they are not tested for. Say, MS' STL with GCC or libc++ with MSVC.

At the very least, you need to agree on builtins or #ifdef stuff manually for magic bits, and you need to avoid to rely on any compiler detail (or bug... :) anywhere (say, the non-conforming template behavior from old MSVC), and so on. So you need an active effort to make it work, I would expect.

And with new complex features like modules, I find it even more unlikely the first implementations of a compiler's stdlib would happen to work on the first implementation of the feature of another compiler...

Now, it is true that Rust's core uses a lot of internal features, precisely because they don't expect to be built by anything else that the current (and current - 1) compiler, so they actually take advantage of that.

It would perhaps be nice to split the "really requires magic" in core from the rest somehow. In this case it wouldn't have helped though, since the formatting machinery still uses builtins last time I looked.

Relatedly, GCC Rust's goal is to build an old core at the moment, so that they have a fixed set of things to solve.

Cheers,
Miguel

maintainer
#18 lore raw
Quoted 9 lines · Miguel Ojeda
Quoted 4 lines · Peter Zijlstra

Maybe I've been living in C/C++ land for too long, but you either expose a full language feature and then everybody can use it, runtime, libraries and user code alike, or you don't.

I find it unlikely that C++ stdlibs happen to build and work properly with compilers they are not tested for. Say, MS' STL with GCC or libc++ with MSVC.

Ignoring MSVC, which is a horrible joke of a compiler (or did Sutter finally cluebat that team into building a sane compiler?), there are definitely cross compiler STL implementations around.

Quoted 5 lines

At the very least, you need to agree on builtins or #ifdef stuff manually for magic bits, and you need to avoid to rely on any compiler detail (or bug... :) anywhere (say, the non-conforming template behavior from old MSVC), and so on. So you need an active effort to make it work, I would expect.

Again, with the exception of MSVC, Boost builds on most C++ compilers and is the staging ground for many new library features.

Quoted 4 lines

And with new complex features like modules, I find it even more unlikely the first implementations of a compiler's stdlib would happen to work on the first implementation of the feature of another compiler...

Sure, shit happens, but... at least they try, it *should* work.

And the old STL (although STL really is far more a library than runtime, it being fully optional) will most certainly build on a new compiler from the same family.

And I know Linus hates on C++ something mighty, but in this parallel universe where he doesn't, I would still recommend the kernel to not use STL and instead build its own libraries (or borrow some nice pieces here and there).

Quoted 4 lines

Now, it is true that Rust's core uses a lot of internal features, precisely because they don't expect to be built by anything else that the current (and current - 1) compiler, so they actually take advantage of that.

So I don't think this is a good thing. Even builtin stuff should be more stable than this.

Quoted 4 lines

It would perhaps be nice to split the "really requires magic" in core from the rest somehow. In this case it wouldn't have helped though, since the formatting machinery still uses builtins last time I looked.

Right, to the point where we can carry a copy of the non-magic part locally that matches the minimum version requirement. And no other bits are to be used by in-tree rust code.

Quoted 2 lines

Relatedly, GCC Rust's goal is to build an old core at the moment, so that they have a fixed set of things to solve.

Very sensible. The Rust project should have an stable core subset / variant :-)

#19 lore raw
Quoted 4 lines · Peter Zijlstra

Ignoring MSVC, which is a horrible joke of a compiler (or did Sutter finally cluebat that team into building a sane compiler?), there are definitely cross compiler STL implementations around.

Definitely, there are, e.g. libc++ and libstdc++, but my point was that it doesn't happen by accident, i.e. it requires effort (perhaps even on both sides).

[ As for MSVC, last time I used it (2020?), it was way better compared to the VC++6 times -- at least they started to try to be conforming, e.g. all the work behind /permissive-, including the two-phase name lookup I mentioned. They also improved a bunch of thing for C to be able to build projects out there. ]

Quoted 2 lines

Again, with the exception of MSVC, Boost builds on most C++ compilers and is the staging ground for many new library features.

Hmm... I am not sure what you mean -- Boost works with MSVC, and Boost tries to use standard C++. Even if it may rely on hacks or compiler extensions at points, Boost does not need to implement e.g. a magic std::launder, and I hope they don't use builtins etc.

Quoted 3 lines

And the old STL (although STL really is far more a library than runtime, it being fully optional) will most certainly build on a new compiler from the same family.

[ To clarify, by "STL" above I was referring to the Microsoft's STL project (i.e. they call it STL, but it is the full C++ standard library). ]

The pure "STL" stuff, i.e. containers and algorithms and such, yeah, that may not require any magic. Some Rust core types and functions can also be reimplemented in normal Rust too.

Quoted 4 lines

And I know Linus hates on C++ something mighty, but in this parallel universe where he doesn't, I would still recommend the kernel to not use STL and instead build its own libraries (or borrow some nice pieces here and there).

Yeah, we take that approach when needed; e.g. nowadays we don't use alloc, which is another part of the Rust standard library that contains things like the equivalent to std::vector, and we also have our own equivalents of core types sometimes, e.g. our CStr.

And, yeah, I agree it would be ideal to be able to reuse/customize some parts more.

Cheers,
Miguel

reviewer
#20 lore raw
In reply to Peter Zijlstra
Quoted 22 lines · Peter Zijlstra
Quoted 7 lines · Alice Ryhl
Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Miguel Ojeda
Quoted 2 lines · Peter Zijlstra

New compiler can't build old core?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

No, that is not intended to work -- core is tied to the compiler.

OMFG, you just gave me another reason to hate on this rust-bucket of a language :-/ What language can't even build an old version of its own runtime :-(

I don't think there's anything unreasonable about having a place to put logic that depends on compiler internals.

Having compiler internals exposed to such an extend that programs can make use of them _is_ weird. It creates the whole problem of where to put it.

Quoted 7 lines

C++ doesn't have anything like this afaik. Esp. if you disable rtti and exception muck, you get a very minimal runtime. And even if you don't, it certainly doesn't break every 6 weeks. ABIs and all that.

Maybe I've been living in C/C++ land for too long, but you either expose a full language feature and then everybody can use it, runtime, libraries and user code alike, or you don't.

Well, I would say that Rust has just decided to expose such language features in a different way from how C/C++ exposes them. Core is *how* rust exposes the features. There are quite a few language features that are exposed via a type or method in core that you use like any other type or method. I think it works relatively well because you can expose new features by adding things to core, which means that you don't need any new syntax for the feature. It's just ordinary method calls into code that you could not implement yourself.

Alice

#21 lore raw
In reply to Alice Ryhl
Quoted 3 lines · Alice Ryhl

I submitted a PR that I believe should fix it:
https://github.com/rust-lang/rust/pull/139632

This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.

Peter: are you OK with landing a patch like this? Well, modified to look like this now that we know the version:

depends on FINEIBT
depends on !RUST || RUSTC_VERSION >= 108800

(assuming we confirm the nightly build works properly)

Thanks!

Cheers,
Miguel

reviewer Reviewed-by
#22 lore raw
Quoted 7 lines · Miguel Ojeda
Quoted 3 lines · Alice Ryhl

I submitted a PR that I believe should fix it:
https://github.com/rust-lang/rust/pull/139632

This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.

Quoted 7 lines

Peter: are you OK with landing a patch like this? Well, modified to look like this now that we know the version:

depends on FINEIBT
depends on !RUST || RUSTC_VERSION >= 108800

(assuming we confirm the nightly build works properly)

With that change:

I have tested locally that the upstream PR works together with CFI, although I did it by backporting the PR to the rustc version I'm using rather than by using the latest nightly. Not sure if a Tested-by on *this* patch is warranted from that test ... but if you think so feel free to add that too.

Alice

maintainer
#23 lore raw
In reply to Miguel Ojeda
Quoted 7 lines · Miguel Ojeda
Quoted 3 lines · Alice Ryhl

I submitted a PR that I believe should fix it:
https://github.com/rust-lang/rust/pull/139632

This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.

Quoted 7 lines

Peter: are you OK with landing a patch like this? Well, modified to look like this now that we know the version:

depends on FINEIBT
depends on !RUST || RUSTC_VERSION >= 108800

(assuming we confirm the nightly build works properly)

I don't much like it -- disabling FineIBT at config time like this also kills the CFI type rehash. Not to mention that FineIBT is a Spectre-BHI mitigation, esp. with that arity thing on top.

I don't suppose we can simply mandate this rust version?

Speaking of which, I should go install this latest rust thing....

#24 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 10 lines · Miguel Ojeda
Quoted 3 lines · Alice Ryhl

I submitted a PR that I believe should fix it:
https://github.com/rust-lang/rust/pull/139632

This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.

Peter: are you OK with landing a patch like this? Well, modified to look like this now that we know the version:

Quoted 7 lines
Quoted 4 lines

depends on FINEIBT
depends on !RUST || RUSTC_VERSION >= 108800

(assuming we confirm the nightly build works properly)

I don't much like it -- disabling FineIBT at config time like this also kills the CFI type rehash.

This isn't disabling CONFIG_FINEIBT (which gates cfi_rand), it's making FineIBT not enabled by default at boot time. This is actually when I created CONFIG_CFI_AUTO_DEFAULT: to be able to have kCFI _with_ type rehashing still enabled.

Quoted 2 lines

Not to mention that FineIBT is a Spectre-BHI
mitigation, esp. with that arity thing on top.

That's certainly true, but the needed overlapping corner cases seem to make this tolerable?

Quoted 1 line

I don't suppose we can simply mandate this rust version?

Perhaps capture the needed version in the Kconfig change at least, so it becomes self-documenting.

-Kees

Signature · 1 line
Kees Cook
maintainer
#25 lore raw
Quoted 17 lines · Kees Cook
Quoted 16 lines · Peter Zijlstra
Quoted 14 lines · Miguel Ojeda
Quoted 3 lines · Alice Ryhl

I submitted a PR that I believe should fix it:
https://github.com/rust-lang/rust/pull/139632

This landed, scheduled for Rust 1.88 (2025-06-26) if all goes well.

Peter: are you OK with landing a patch like this? Well, modified to look like this now that we know the version:

depends on FINEIBT
depends on !RUST || RUSTC_VERSION >= 108800

(assuming we confirm the nightly build works properly)

Quoted 7 lines
Quoted 2 lines

I don't much like it -- disabling FineIBT at config time like this also kills the CFI type rehash.

This isn't disabling CONFIG_FINEIBT (which gates cfi_rand), it's making FineIBT not enabled by default at boot time. This is actually when I created CONFIG_CFI_AUTO_DEFAULT: to be able to have kCFI _with_ type rehashing still enabled.

Urgh, yeah, reading hard :-/

OK, Ack

Reviewed-by Alice Ryhl Acked-by Peter Zijlstra
#26 lore raw
Quoted 4 lines · Peter Zijlstra

Urgh, yeah, reading hard :-/

OK, Ack

Thanks Peter (and Kees for clarifying) -- not sure how you/others prefer to route this patch or if you expect a v2, but I got the following (attached). I converted your Ack.

Cheers,
Miguel

From 2c6636c82f8c6c18e18c085fe69d0cd3dbe3b991 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Pawe=C5=82=20Anikiel?= <[email protected]>
Date: Thu, 10 Apr 2025 11:54:20 +0000
Subject: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST or Rust >=
1.88
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Calling core::fmt::write() from rust code while FineIBT is enabled results in a kernel panic:

[ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
[ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
[ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
[ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
[ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
[ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
[ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
[ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
[ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
[ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
[ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
[ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
[ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
[ 4614.336777] PKRU: 55555554
[ 4614.339786] Call Trace:
[ 4614.342524]  <TASK>
[ 4614.344867]  ? __die_body+0x69/0xb0
[ 4614.348786]  ? die+0xa9/0xd0
[ 4614.352000]  ? do_trap+0x89/0x160
[ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.360413]  ? handle_invalid_op+0x69/0x90
[ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.369654]  ? exc_invalid_op+0x36/0x60
[ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
[ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
[ 4614.388005]  exc_control_protection+0x49/0x70
[ 4614.392871]  asm_exc_control_protection+0x2b/0x60
[ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
[ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
[ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
[ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
[ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
[ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
[ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
[ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
[ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
[ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250

This happens because core::fmt::write() calls core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

library/core/src/fmt/rt.rs:

171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {

This causes a Control Protection exception, because FineIBT has sealed off the original function's endbr64.

This makes rust currently incompatible with FineIBT. Add a Kconfig dependency that prevents FineIBT from getting turned on by default if rust is enabled.

[ Rust 1.88.0 (scheduled for 2025-06-26) should have this fixed [1],
and thus we relaxed the condition with Rust >= 1.88.

When objtool lands checking for this with e.g. [2], the plan is to ideally run that in upstream Rust's CI to prevent regressions early [3], since we do not control core's source code.

Alice tested the Rust PR backported to an older compiler.

- Miguel ]

---
 arch/x86/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/x86/Kconfig +1 -0
index 4b9f378e05f6..5873c9e39919 100644
@@ -2368,6 +2368,7 @@ config STRICT_SIGALTSTACK_SIZE
 config CFI_AUTO_DEFAULT
 	bool "Attempt to use FineIBT by default at boot time"
 	depends on FINEIBT
+	depends on !RUST || RUSTC_VERSION >= 108800
 	default y
 	help
 	  Attempt to use FineIBT by default at boot time. If enabled,

base-commit: c1b4071ec3a6a594df6c49bf8f04a60a88072525

Signature · 1 line
2.49.0
maintainer
#27 lore raw
Quoted 9 lines · Miguel Ojeda

From 2c6636c82f8c6c18e18c085fe69d0cd3dbe3b991 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Pawe=C5=82=20Anikiel?= <[email protected]>
Date: Thu, 10 Apr 2025 11:54:20 +0000
Subject: [PATCH] x86/Kconfig: make CFI_AUTO_DEFAULT depend on !RUST or Rust >=
1.88
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Quoted 8 lines

Calling core::fmt::write() from rust code while FineIBT is enabled results in a kernel panic:

[ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
[ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
[ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
[ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0

Might want to 'compress' the splat a bit, lots of useless bits on etc.

Quoted 22 lines
[ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
[ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
[ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
[ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
[ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
[ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
[ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
[ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
[ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
[ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250

This happens because core::fmt::write() calls core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

library/core/src/fmt/rt.rs:

171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
Quoted 2 lines

This causes a Control Protection exception, because FineIBT has sealed off the original function's endbr64.

Right, doing an indirect call without the CFI adornment to a CFI adorned function is explicitly not supported.

Quoted 10 lines

This makes rust currently incompatible with FineIBT. Add a Kconfig dependency that prevents FineIBT from getting turned on by default if rust is enabled.

[ Rust 1.88.0 (scheduled for 2025-06-26) should have this fixed [1],
and thus we relaxed the condition with Rust >= 1.88.

When objtool lands checking for this with e.g. [2], the plan is to ideally run that in upstream Rust's CI to prevent regressions early [3], since we do not control core's source code.

I'm still firmly of the opinion Rust should grow up and have a stable core which we can pull into the kernel. Relying on that much out of tree code is 'unfortunate'.

Quoted 33 lines

Alice tested the Rust PR backported to an older compiler.

- Miguel ]

---
 arch/x86/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/x86/Kconfig +1 -0
index 4b9f378e05f6..5873c9e39919 100644
@@ -2368,6 +2368,7 @@ config STRICT_SIGALTSTACK_SIZE
 config CFI_AUTO_DEFAULT
 	bool "Attempt to use FineIBT by default at boot time"
 	depends on FINEIBT
+	depends on !RUST || RUSTC_VERSION >= 108800
 	default y
 	help
 	  Attempt to use FineIBT by default at boot time. If enabled,

base-commit: c1b4071ec3a6a594df6c49bf8f04a60a88072525

Signature · 1 line
2.49.0
Applied
#28 lore raw
In reply to Miguel Ojeda
Quoted 4 lines · Miguel Ojeda

Thanks Peter (and Kees for clarifying) -- not sure how you/others prefer to route this patch or if you expect a v2, but I got the following (attached). I converted your Ack.

Applied to rust-fixes -- thanks everyone!

(If someone else prefers to carry it, please shout)

[ Rust 1.88.0 (scheduled for 2025-06-26) should have this fixed [1],
and thus we relaxed the condition with Rust >= 1.88.

When objtool lands checking for this with e.g. [2], the plan is to ideally run that in upstream Rust's CI to prevent regressions early [3], since we do not control core's source code.

Alice tested the Rust PR backported to an older compiler.

Peter would like that Rust provides a stable core which can be pulled into the kernel: "Relying on that much out of tree code is 'unfortunate'".

- Miguel ]

[ Reduced splat. - Miguel ]

Cheers,
Miguel

reviewer
#29 lore raw
Quoted 20 lines · Miguel Ojeda
Quoted 4 lines · Miguel Ojeda

Thanks Peter (and Kees for clarifying) -- not sure how you/others prefer to route this patch or if you expect a v2, but I got the following (attached). I converted your Ack.

Applied to rust-fixes -- thanks everyone!

(If someone else prefers to carry it, please shout)

[ Rust 1.88.0 (scheduled for 2025-06-26) should have this fixed [1],
and thus we relaxed the condition with Rust >= 1.88.

When objtool lands checking for this with e.g. [2], the plan is to ideally run that in upstream Rust's CI to prevent regressions early [3], since we do not control core's source code.

Alice tested the Rust PR backported to an older compiler.

Quoted 7 lines

Peter would like that Rust provides a stable core which can be pulled into the kernel: "Relying on that much out of tree code is 'unfortunate'".

- Miguel ]

[ Reduced splat. - Miguel ]

Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right approach.

Shouldn't the depends on clause go on config FINEIBT instead? After all, the current patch just means that you can't make FineIBT the default option. But you can still pass kcfi=fineibt on boot to enble FineIBT which would result in the same crash.

Alice

#30 lore raw
Quoted 8 lines · Alice Ryhl

Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right approach.

Shouldn't the depends on clause go on config FINEIBT instead? After all, the current patch just means that you can't make FineIBT the default option. But you can still pass kcfi=fineibt on boot to enble FineIBT which would result in the same crash.

I thought the intention was to avoid disabling FineIBT at config time:
https://lore.kernel.org/all/202504161442.66CE2596@keescook/

If another approach is needed, then we can change it on top if that is OK (I am about to submit the PR).

Cheers,
Miguel

#31 lore raw
Quoted 6 lines · Miguel Ojeda
Quoted 4 lines · Alice Ryhl

Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right approach.

Quoted 7 lines
Quoted 4 lines

Shouldn't the depends on clause go on config FINEIBT instead? After all, the current patch just means that you can't make FineIBT the default option. But you can still pass kcfi=fineibt on boot to enble FineIBT which would result in the same crash.

I thought the intention was to avoid disabling FineIBT at config time:
https://lore.kernel.org/all/202504161442.66CE2596@keescook/

I think you mean enabling FineIBT - yes, this patch does exactly that, it's still possible to enable it by hand with cfi=

Quoted 2 lines

If another approach is needed, then we can change it on top if that is OK (I am about to submit the PR).

Putting it on CONFIG_FINEIBT was my initial idea, but it turns out that this creates a dependency cycle in Kconfig:

error: recursive dependency detected!
symbol RUST depends on CALL_PADDING symbol CALL_PADDING is selected by FINEIBT symbol FINEIBT depends on RUST

Not sure if it's possible to make it work.

reviewer
#32 lore raw
Quoted 25 lines · Paweł Anikiel
Quoted 6 lines · Miguel Ojeda
Quoted 4 lines · Alice Ryhl

Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right approach.

Quoted 7 lines
Quoted 4 lines

Shouldn't the depends on clause go on config FINEIBT instead? After all, the current patch just means that you can't make FineIBT the default option. But you can still pass kcfi=fineibt on boot to enble FineIBT which would result in the same crash.

I thought the intention was to avoid disabling FineIBT at config time:
https://lore.kernel.org/all/202504161442.66CE2596@keescook/

I think you mean enabling FineIBT - yes, this patch does exactly that, it's still possible to enable it by hand with cfi=

Quoted 2 lines

If another approach is needed, then we can change it on top if that is OK (I am about to submit the PR).

Putting it on CONFIG_FINEIBT was my initial idea, but it turns out that this creates a dependency cycle in Kconfig:

Quoted 6 lines

error: recursive dependency detected!
symbol RUST depends on CALL_PADDING symbol CALL_PADDING is selected by FINEIBT symbol FINEIBT depends on RUST

Not sure if it's possible to make it work.

To fix that, we probably have to do one of:

Option 1: Upgrade the rustc MSRV past 1.81 and drop the depends on
CALL_PADDING.

Option 2: Split config FINEIBT into two options:

config HAVE_FINEIBT

	def_bool y
	depends on X86_KERNEL_IBT && CFI_CLANG && MITIGATION_RETPOLINE
	depends on !RUST || RUSTC_VERSION >= 108800

config FINEIBT
bool "Accept cfi=fineibt option at boot"
depends on HAVE_FINEIBT
select CALL_PADDING

Alice

#33 lore raw
Quoted 41 lines · Alice Ryhl
Quoted 25 lines · Paweł Anikiel
Quoted 6 lines · Miguel Ojeda
Quoted 4 lines · Alice Ryhl

Actually ... I don't think putting it on CFI_AUTO_DEFAULT the right approach.

Quoted 7 lines
Quoted 4 lines

Shouldn't the depends on clause go on config FINEIBT instead? After all, the current patch just means that you can't make FineIBT the default option. But you can still pass kcfi=fineibt on boot to enble FineIBT which would result in the same crash.

I thought the intention was to avoid disabling FineIBT at config time:
https://lore.kernel.org/all/202504161442.66CE2596@keescook/

I think you mean enabling FineIBT - yes, this patch does exactly that, it's still possible to enable it by hand with cfi=

Quoted 2 lines

If another approach is needed, then we can change it on top if that is OK (I am about to submit the PR).

Putting it on CONFIG_FINEIBT was my initial idea, but it turns out that this creates a dependency cycle in Kconfig:

Quoted 6 lines

error: recursive dependency detected!
symbol RUST depends on CALL_PADDING symbol CALL_PADDING is selected by FINEIBT symbol FINEIBT depends on RUST

Not sure if it's possible to make it work.

To fix that, we probably have to do one of:

Option 1: Upgrade the rustc MSRV past 1.81 and drop the depends on
CALL_PADDING.

Option 2: Split config FINEIBT into two options:

config HAVE_FINEIBT

Quoted 8 lines
	def_bool y
	depends on X86_KERNEL_IBT && CFI_CLANG && MITIGATION_RETPOLINE
	depends on !RUST || RUSTC_VERSION >= 108800

config FINEIBT
bool "Accept cfi=fineibt option at boot"
depends on HAVE_FINEIBT
select CALL_PADDING

Right -- if you want to make FINEIBT unselectable, this will need some plumbing. Right now, it's kind of tied in with the hash shuffling, so it'll need more than just a Kconfig change to properly split it.

Signature · 1 line
Kees Cook
#34 lore raw
In reply to Paweł Anikiel
Quoted 9 lines · Paweł Anikiel
Quoted 3 lines · Miguel Ojeda

I thought the intention was to avoid disabling FineIBT at config time:
https://lore.kernel.org/all/202504161442.66CE2596@keescook/

I think you mean enabling FineIBT - yes, this patch does exactly that, it's still possible to enable it by hand with cfi=

No, I meant avoiding to disable it at config time due to the rehash discussion between Peter & Kees in the Lore link I mentioned (I see Kees has replied again -- thanks!)

Cheers,
Miguel

#35 lore raw
In reply to Peter Zijlstra
Quoted 58 lines · Peter Zijlstra
Quoted 56 lines · Peter Zijlstra
Quoted 55 lines · Paweł Anikiel

Calling core::fmt::write() from rust code while FineIBT is enabled results in a kernel panic:

[ 4614.199779] kernel BUG at arch/x86/kernel/cet.c:132!
[ 4614.205343] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 4614.211781] CPU: 2 UID: 0 PID: 6057 Comm: dmabuf_dump Tainted: G     U     O       6.12.17-android16-0-g6ab38c534a43 #1 9da040f27673ec3945e23b998a0f8bd64c846599
[ 4614.227832] Tainted: [U]=USER, [O]=OOT_MODULE
[ 4614.241247] RIP: 0010:do_kernel_cp_fault+0xea/0xf0
[ 4614.246621] Code: c6 15 8d ad ac 48 0f 44 f1 48 8d 04 80 48 8d 14 45 d0 37 42 ac 48 c7 c7 22 99 bb ac e8 9f 7a 05 00 0f 0b eb 9a 67 0f b9 40 12 <0f> 0b cc cc cc cc 66 0f 1f 00 41 81 ea 00 00 00 00 74 03 0f 0b 90
[ 4614.267606] RSP: 0018:ffffb95acfa4b978 EFLAGS: 00010097
[ 4614.273464] RAX: 0000000000000057 RBX: ffffb95acfa4b9b8 RCX: 3ff1c813cb576300
[ 4614.281426] RDX: ffff9a50b792b8d0 RSI: ffff9a50b791d548 RDI: ffff9a50b791d548
[ 4614.289408] RBP: ffffb95acfa4b980 R08: 0000000000000d7c R09: ffffffffad45d500
[ 4614.297399] R10: 0000000000002874 R11: 0000000000000004 R12: 0000000000000000
[ 4614.305369] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
[ 4614.313345] FS:  000076fa106dcfe8(0000) GS:ffff9a50b7900000(0000) knlGS:0000000000000000
[ 4614.322386] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4614.328806] CR2: 000076f8fc207bd0 CR3: 0000000227242000 CR4: 0000000000f52eb0
[ 4614.336777] PKRU: 55555554
[ 4614.339786] Call Trace:
[ 4614.342524]  <TASK>
[ 4614.344867]  ? __die_body+0x69/0xb0
[ 4614.348786]  ? die+0xa9/0xd0
[ 4614.352000]  ? do_trap+0x89/0x160
[ 4614.355721]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.360413]  ? handle_invalid_op+0x69/0x90
[ 4614.364985]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.369654]  ? exc_invalid_op+0x36/0x60
[ 4614.373959]  ? asm_exc_invalid_op+0x1f/0x30
[ 4614.378643]  ? do_kernel_cp_fault+0xea/0xf0
[ 4614.383335]  ? do_kernel_cp_fault+0x31/0xf0
[ 4614.388005]  exc_control_protection+0x49/0x70
[ 4614.392871]  asm_exc_control_protection+0x2b/0x60
[ 4614.398144] RIP: 0010:_RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x0/0x20
[ 4614.407792] Code: 48 f7 df 48 0f 48 f9 48 89 f2 89 c6 5d e9 18 fd ff ff 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 41 81 ea 14 61 af 2c 74 03 0f 0b 90 <66> 0f 1f 00 55 48 89 e5 48 89 f2 48 8b 3f be 01 00 00 00 5d e9 e7
[ 4614.428775] RSP: 0018:ffffb95acfa4ba68 EFLAGS: 00010246
[ 4614.434609] RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000000
[ 4614.442587] RDX: 0000000000000007 RSI: ffffb95acfa4ba70 RDI: ffffb95acfa4bc88
[ 4614.450557] RBP: ffffb95acfa4bae0 R08: ffff0a00ffffff05 R09: 0000000000000070
[ 4614.458527] R10: 0000000000000000 R11: ffffffffab67eaf0 R12: ffffb95acfa4bcc8
[ 4614.466493] R13: ffffffffac5d50f0 R14: 0000000000000000 R15: 0000000000000000
[ 4614.474473]  ? __cfi__RNvXs5_NtNtNtCs3o2tGsuHyou_4core3fmt3num3impyNtB9_7Display3fmt+0x10/0x10
[ 4614.484118]  ? _RNvNtCs3o2tGsuHyou_4core3fmt5write+0x1d2/0x250

This happens because core::fmt::write() calls core::fmt::rt::Argument::fmt(), which currently has CFI disabled:

library/core/src/fmt/rt.rs:

171     // FIXME: Transmuting formatter in new and indirectly branching to/calling
172     // it here is an explicit CFI violation.
173     #[allow(inline_no_sanitize)]
174     #[no_sanitize(cfi, kcfi)]
175     #[inline]
176     pub(super) unsafe fn fmt(&self, f: &mut Formatter<'_>) -> Result {
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

This causes a Control Protection exception, because FineIBT has sealed off the original function's endbr64.

This makes rust currently incompatible with FineIBT. Add a Kconfig dependency that prevents FineIBT from getting turned on by default if rust is enabled.

Why ?!, what's wrong with removing that no_sanitize() instead?

I'll let the rust maintainers answer this one.

Quoted 2 lines
Quoted 2 lines

How is it still compatible with kCFI and not with FineIBT?

kCFI keeps the original function's endbr64. If a callsite disables CFI, it simply jumps to that function (without any magic value checks), the endbr64 is there, so no #CP.

With FineIBT, the original function's endbr64 gets replaced with a nop, and the callsite is supposed to be modified to jump to the CFI preamble. With CFI disabled, that modification doesn't happen (cfi_rewrite_callers() quietly ignores that callsite), so we jump to the nop, and we get a #CP.

Quoted 4 lines

FWIW, CFI violations of any kind are a no-no. They're not accepted in C and they're not accepted in Rust. This is clearly a Rust bug that needs to be fixed ASAP. Disabling CFI is not an option.

This is a known issue, but it doesn't seem to have high priority:
https://github.com/rust-lang/rust/issues/115199

maintainer
#36 lore raw
Quoted 9 lines · Paweł Anikiel

I'll let the rust maintainers answer this one.

Quoted 2 lines
Quoted 2 lines

How is it still compatible with kCFI and not with FineIBT?

kCFI keeps the original function's endbr64. If a callsite disables CFI, it simply jumps to that function (without any magic value checks), the endbr64 is there, so no #CP.

Quoted 5 lines

With FineIBT, the original function's endbr64 gets replaced with a nop, and the callsite is supposed to be modified to jump to the CFI preamble. With CFI disabled, that modification doesn't happen (cfi_rewrite_callers() quietly ignores that callsite), so we jump to the nop, and we get a #CP.

Oh, you're attempting to do a no-cfi indirect branch? Yeah, you don't get to do that.

I should get objtool to warn about those. They undermine the point of CFI.

Quoted 6 lines
Quoted 3 lines

FWIW, CFI violations of any kind are a no-no. They're not accepted in C and they're not accepted in Rust. This is clearly a Rust bug that needs to be fixed ASAP. Disabling CFI is not an option.

This is a known issue, but it doesn't seem to have high priority:
https://github.com/rust-lang/rust/issues/115199

Now you get to fix it :) Because such code is unacceptable for the kernel.

Just because some crazy Rust person sticks this in their runtime, doesn't mean you get to use it in the kernel.

maintainer
#37 lore raw

PATCH 1/1 objtool: Detect __nocfi calls

Quoted 2 lines · Peter Zijlstra

I should get objtool to warn about those. They undermine the point of CFI.

---
Subject: objtool: Detect __nocfi calls

Detect and WARN about no_sanitize(kcfi) indirect calls.

Apparently there were a few in some Rust 'core' that got included in the kernel and things went *bang*.

This is not a supported form for kernel code. So detect and warn about it.

Adds an annotation for the two cases where we have to live with them:

- EFI stubs;
- kexec handover.

Notably, EFI calls fully disable IBT, as such using runtime EFI services is a significant security issue. If you can exploit the kexec handover, you get to keep it.

---

arch/x86/kernel/machine_kexec_64.c +1 -0
index cc73f9708464..67ba0db92272 100644
@@ -442,6 +442,7 @@ void __nocfi machine_kexec(struct kimage *image)
 
 	__ftrace_enabled_restore(save_ftrace_enabled);
 }
+ANNOTATE_NOCFI_SYM(machine_kexec);
 
 /* arch-dependent functionality related to kexec file-based syscall */
 
arch/x86/platform/efi/efi_stub_64.S +1 -0
index 2206b8bc47b8..2bee139ae1ee 100644
@@ -11,6 +11,7 @@
 #include <asm/nospec-branch.h>
 
 SYM_FUNC_START(__efi_call)
+	ANNOTATE_NOCFI
 	pushq %rbp
 	movq %rsp, %rbp
 	and $~0xf, %rsp
include/linux/objtool.h +3 -0
index 366ad004d794..518daea19699 100644
@@ -185,6 +185,8 @@
  */
 #define ANNOTATE_REACHABLE(label)	__ASM_ANNOTATE(label, ANNOTYPE_REACHABLE)
 
+#define ANNOTATE_NOCFI_SYM(sym)		asm(__ASM_ANNOTATE(sym, ANNOTYPE_NOCFI))
+
 #else
 #define ANNOTATE_NOENDBR		ANNOTATE type=ANNOTYPE_NOENDBR
 #define ANNOTATE_RETPOLINE_SAFE		ANNOTATE type=ANNOTYPE_RETPOLINE_SAFE
@@ -194,6 +196,7 @@
 #define ANNOTATE_INTRA_FUNCTION_CALL	ANNOTATE type=ANNOTYPE_INTRA_FUNCTION_CALL
 #define ANNOTATE_UNRET_BEGIN		ANNOTATE type=ANNOTYPE_UNRET_BEGIN
 #define ANNOTATE_REACHABLE		ANNOTATE type=ANNOTYPE_REACHABLE
+#define ANNOTATE_NOCFI			ANNOTATE type=ANNOTYPE_NOCFI
 #endif
 
 #if defined(CONFIG_NOINSTR_VALIDATION) && \
include/linux/objtool_types.h +1 -0
index df5d9fa84dba..aceac94632c8 100644
@@ -65,5 +65,6 @@ struct unwind_hint {
 #define ANNOTYPE_IGNORE_ALTS		6
 #define ANNOTYPE_INTRA_FUNCTION_CALL	7
 #define ANNOTYPE_REACHABLE		8
+#define ANNOTYPE_NOCFI			9
 
 #endif /* _LINUX_OBJTOOL_TYPES_H */
tools/include/linux/objtool_types.h +1 -0
index df5d9fa84dba..aceac94632c8 100644
@@ -65,5 +65,6 @@ struct unwind_hint {
 #define ANNOTYPE_IGNORE_ALTS		6
 #define ANNOTYPE_INTRA_FUNCTION_CALL	7
 #define ANNOTYPE_REACHABLE		8
+#define ANNOTYPE_NOCFI			9
 
 #endif /* _LINUX_OBJTOOL_TYPES_H */
tools/objtool/check.c +39 -0
index 4a1f6c3169b3..868601760953 100644
@@ -1382,6 +1382,26 @@ static int add_call_dest(struct objtool_file *file, struct instruction *insn,
 
 static int add_retpoline_call(struct objtool_file *file, struct instruction *insn)
 {
+	struct symbol *sym = insn->sym;
+
+	/*
+	 * kCFI call sites look like:
+	 *
+	 *     movl $(-0x12345678), %r10d
+	 *     addl -4(%r11), %r10d
+	 *     jz 1f
+	 *     ud2
+	 *  1: cs call __x86_indirect_thunk_r11
+	 *
+	 * Verify all indirect calls are kCFI adorned by checking for the UD2.
+	 * Notably, doing __nocfi calls to regular (cfi) functions is broken.
+	 */
+	if (opts.cfi && sym && sym->type == STT_FUNC && !sym->nocfi) {
+		struct instruction *prev = prev_insn_same_sym(file, insn);
+		if (!prev || prev->type != INSN_BUG)
+			WARN_INSN(insn, "no-cfi indirect call!");
+	}
+
 	/*
 	 * Retpoline calls/jumps are really dynamic calls/jumps in disguise,
 	 * so convert them accordingly.
@@ -2334,6 +2354,8 @@ static int read_annotate(struct objtool_file *file,
 
 static int __annotate_early(struct objtool_file *file, int type, struct instruction *insn)
 {
+	struct symbol *sym;
+
 	switch (type) {
 
 	/* Must be before add_special_section_alts() */
@@ -2348,6 +2370,19 @@ static int __annotate_early(struct objtool_file *file, int type, struct instruct
 		insn->noendbr = 1;
 		break;
 
+	/*
+	 * Must be before add_{jump,call}_destination(), specifically any
+	 * add_retpoline_call().
+	 */
+	case ANNOTYPE_NOCFI:
+		sym = insn->sym;
+		if (!sym) {
+			WARN_INSN(insn, "dodgy NOCFI annotation");
+			break;
+		}
+		insn->sym->nocfi = 1;
+		break;
+
 	default:
 		break;
 	}
@@ -2428,6 +2463,10 @@ static int __annotate_late(struct objtool_file *file, int type, struct instructi
 		insn->dead_end = false;
 		break;
 
+	case ANNOTYPE_NOCFI:
+		/* early */
+		break;
+
 	default:
 		ERROR_INSN(insn, "Unknown annotation type: %d", type);
 		return -1;
tools/objtool/include/objtool/elf.h +1 -0
index c7c4e87ebe88..f60604d30793 100644
@@ -70,6 +70,7 @@ struct symbol {
 	u8 local_label       : 1;
 	u8 frame_pointer     : 1;
 	u8 ignore	     : 1;
+	u8 nocfi             : 1;
 	struct list_head pv_target;
 	struct reloc *relocs;
 };
maintainer
#38 lore raw

Re: [PATCH] objtool: Detect __nocfi calls

Quoted 7 lines · Peter Zijlstra
Quoted 2 lines · Peter Zijlstra

I should get objtool to warn about those. They undermine the point of CFI.

---
Subject: objtool: Detect __nocfi calls

"Warn on indirect calls in __nocfi functions" ?

Quoted 10 lines

static int add_retpoline_call(struct objtool_file *file, struct instruction *insn) {

tools/objtool/check.c +8 -0
+	struct symbol *sym = insn->sym;
+
+	/*
+	 * kCFI call sites look like:
+	 *
+	 *     movl $(-0x12345678), %r10d
+	 *     addl -4(%r11), %r10d
+	 *     jz 1f

tools/objtool/check.c static int add_call_dest(struct objtool_file *file, struct instruction *insn, · line 1402

Quoted 10 lines
tools/objtool/check.c +10 -0
+	 *     ud2
+	 *  1: cs call __x86_indirect_thunk_r11
+	 *
+	 * Verify all indirect calls are kCFI adorned by checking for the UD2.
+	 * Notably, doing __nocfi calls to regular (cfi) functions is broken.
+	 */
+	if (opts.cfi && sym && sym->type == STT_FUNC && !sym->nocfi) {
+		struct instruction *prev = prev_insn_same_sym(file, insn);
+		if (!prev || prev->type != INSN_BUG)
+			WARN_INSN(insn, "no-cfi indirect call!");

Since this can break things pretty badly at runtime, this should actually fail the build on CONFIG_OBJTOOL_WERROR.

The warning counts aren't plumbed in this early, so can this check be done later? validate_retpoline() or validate_call()?

Signature · 1 line
Josh
maintainer
#39 lore raw

Re: [PATCH] objtool: Detect __nocfi calls

Quoted 10 lines · Josh Poimboeuf
Quoted 7 lines · Peter Zijlstra
Quoted 2 lines · Peter Zijlstra

I should get objtool to warn about those. They undermine the point of CFI.

---
Subject: objtool: Detect __nocfi calls

"Warn on indirect calls in __nocfi functions" ?

Yeah, I suppose that's more accurate.

Quoted 15 lines
Quoted 15 lines

static int add_retpoline_call(struct objtool_file *file, struct instruction *insn) {

tools/objtool/check.c +13 -0
+	struct symbol *sym = insn->sym;
+
+	/*
+	 * kCFI call sites look like:
+	 *
+	 *     movl $(-0x12345678), %r10d
+	 *     addl -4(%r11), %r10d
+	 *     jz 1f
+	 *     ud2
+	 *  1: cs call __x86_indirect_thunk_r11
+	 *
+	 * Verify all indirect calls are kCFI adorned by checking for the UD2.
+	 * Notably, doing __nocfi calls to regular (cfi) functions is broken.
Quoted 8 lines

tools/objtool/check.c static int add_call_dest(struct objtool_file *file, struct instruction *insn, · line 1402

Quoted 5 lines
tools/objtool/check.c +5 -0
+	 */
+	if (opts.cfi && sym && sym->type == STT_FUNC && !sym->nocfi) {
+		struct instruction *prev = prev_insn_same_sym(file, insn);
+		if (!prev || prev->type != INSN_BUG)
+			WARN_INSN(insn, "no-cfi indirect call!");

Since this can break things pretty badly at runtime, this should actually fail the build on CONFIG_OBJTOOL_WERROR.

Oh right, I still got to adjust to the new world order here :-)

Quoted 2 lines

The warning counts aren't plumbed in this early, so can this check be done later? validate_retpoline() or validate_call()?

Hmm, let me have a poke around, see what can be done.

#40 lore raw

Re: [PATCH] objtool: Detect __nocfi calls

In reply to Peter Zijlstra
Quoted 8 lines · Peter Zijlstra

Detect and WARN about no_sanitize(kcfi) indirect calls.

Apparently there were a few in some Rust 'core' that got included in the kernel and things went *bang*.

This is not a supported form for kernel code. So detect and warn about it.

That was quick!

It seems to work -- I got:

LD vmlinux.o
vmlinux.o: warning: objtool:
_RNvNtCs8DPF7ip8WBQ_4core3fmt5write+0x170: no-cfi indirect call!

Cheers,
Miguel

#41 lore raw

Re: [PATCH] objtool: Detect __nocfi calls

In reply to Peter Zijlstra

Hi Peter,

Quoted 8 lines · Peter Zijlstra
Quoted 2 lines · Peter Zijlstra

I should get objtool to warn about those. They undermine the point of CFI.

---
Subject: objtool: Detect __nocfi calls

Quoted 7 lines

Detect and WARN about no_sanitize(kcfi) indirect calls.

Apparently there were a few in some Rust 'core' that got included in the kernel and things went *bang*.

This is not a supported form for kernel code. So detect and warn about it.

Cool, this looks useful!

Quoted 8 lines

Adds an annotation for the two cases where we have to live with them:

- EFI stubs;
- kexec handover.

Notably, EFI calls fully disable IBT, as such using runtime EFI services is a significant security issue. If you can exploit the kexec handover, you get to keep it.

OK, with this applied I now see a warning about the __nocfi call in Rust code:

vmlinux.o: warning: objtool: _RNvNtCsjWi3sh0wSlE_4core3fmt5write+0x170: no-cfi indirect call!

But an allmodconfig build reveals a few more warnings:

arch/x86/kvm/kvm.o: warning: objtool: x86_emulate_insn+0xaf7: no-cfi indirect call! arch/x86/kvm/kvm.o: warning: objtool: em_das+0x290: no-cfi indirect call! arch/x86/kvm/kvm.o: warning: objtool: em_imul_3op+0x15f: no-cfi indirect call! arch/x86/kvm/kvm.o: warning: objtool: em_aam+0x21c: no-cfi indirect call! arch/x86/kvm/kvm.o: warning: objtool: em_aad+0x1dc: no-cfi indirect call! arch/x86/kvm/kvm.o: warning: objtool: em_loop+0x312: no-cfi indirect call! arch/x86/kvm/kvm.o: warning: objtool: em_cmpxchg+0x329: no-cfi indirect call! arch/x86/kvm/kvm.o: warning: objtool: em_bsf_c+0x1b7: no-cfi indirect call! arch/x86/kvm/kvm.o: warning: objtool: em_bsr_c+0x1b7: no-cfi indirect call! arch/x86/kvm/kvm-intel.o: warning: objtool: vmx_do_interrupt_irqoff+0xe: no-cfi indirect call! drivers/misc/lkdtm/lkdtm.o: warning: objtool: execute_location+0x5a: no-cfi indirect call! drivers/pci/controller/pci-hyperv.o: warning: objtool: hv_do_hypercall+0x150: no-cfi indirect call! drivers/hv/hv_balloon.o: warning: objtool: hv_free_page_report+0x5da: no-cfi indirect call! drivers/hv/hv_vmbus.o: warning: objtool: hv_post_message+0x457: no-cfi indirect call! drivers/hv/hv_vmbus.o: warning: objtool: vmbus_set_event+0x2a2: no-cfi indirect call! vmlinux.o: warning: objtool: hyperv_flush_tlb_multi+0xe96: no-cfi indirect call! vmlinux.o: warning: objtool: hv_do_hypercall+0x12b: no-cfi indirect call! vmlinux.o: warning: objtool: hyperv_flush_guest_mapping+0x2e3: no-cfi indirect call! vmlinux.o: warning: objtool: hyperv_flush_guest_mapping_range+0x36a: no-cfi indirect call! vmlinux.o: warning: objtool: hv_do_hypercall+0x150: no-cfi indirect call! vmlinux.o: warning: objtool: hv_snp_boot_ap+0xb08: no-cfi indirect call!
vmlinux.o: warning: objtool: hv_vtom_set_host_visibility+0x54a: no-cfi indirect call! vmlinux.o: warning: objtool: __send_ipi_one+0x362: no-cfi indirect call!
vmlinux.o: warning: objtool: __send_ipi_mask_ex+0x655: no-cfi indirect call! vmlinux.o: warning: objtool: __send_ipi_mask+0x635: no-cfi indirect call! vmlinux.o: warning: objtool: hv_do_hypercall+0x150: no-cfi indirect call! vmlinux.o: warning: objtool: hv_query_ext_cap+0x175: no-cfi indirect call! vmlinux.o: warning: objtool: get_vtl+0x38c: no-cfi indirect call!
vmlinux.o: warning: objtool: hv_get_partition_id+0x224: no-cfi indirect call!

Sami

maintainer
#42 lore raw

Re: [PATCH] objtool: Detect __nocfi calls

Quoted 1 line · Sami Tolvanen

But an allmodconfig build reveals a few more warnings:

Sigh, let me go look at all that and address Josh's feedback.

Thanks!

maintainer
#43 lore raw

Re: [PATCH] objtool: Detect __nocfi calls

Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Sami Tolvanen

But an allmodconfig build reveals a few more warnings:

Sigh, let me go look at all that and address Josh's feedback.

Latest sits here:

git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git x86/core

I'll have the robots chew on it over the weekend and then post on Monday.