Skip to contentSkip to conversation

x86: Pin cr4 FSGSBASE

PATCH In review Andi Kleen 6 participants 17 messages May 26, 2020 – May 27, 2020

Subsystems (2)
lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

PATCH 1/1 x86: Pin cr4 FSGSBASE

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

Note this patch will need to be undone when the full FSGSBASE patches are merged. But it's a reasonable solution for v5.2+ stable at least. Sadly the older kernels don't have the necessary infrastructure for this (although a simpler version of this could be added there too)

---
 arch/x86/kernel/cpu/common.c | 5 +++++
 1 file changed, 5 insertions(+)
arch/x86/kernel/cpu/common.c +5 -0
index bed0cb83fe24..1f5b7871ae9a 100644
@@ -385,6 +385,11 @@ void native_write_cr4(unsigned long val)
 		/* Warn after we've set the missing bits. */
 		WARN_ONCE(bits_missing, "CR4 bits went missing: %lx!?\n",
 			  bits_missing);
+		if (val & X86_CR4_FSGSBASE) {
+			WARN_ONCE(1, "CR4 unexpectedly set FSGSBASE!?\n");
+			val &= ~X86_CR4_FSGSBASE;
+			goto set_register;
+		}
 	}
 }
 EXPORT_SYMBOL(native_write_cr4);
Signature · 1 line
2.25.4
maintainer
#1 lore raw
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now? Why stop with this type of check, why not just forbid them entirely if we don't trust them? :)

Quoted 17 lines

Note this patch will need to be undone when the full FSGSBASE patches are merged. But it's a reasonable solution for v5.2+ stable at least. Sadly the older kernels don't have the necessary infrastructure for this (although a simpler version of this could be added there too)

---
 arch/x86/kernel/cpu/common.c | 5 +++++
 1 file changed, 5 insertions(+)
arch/x86/kernel/cpu/common.c +0 -0
index bed0cb83fe24..1f5b7871ae9a 100644

arch/x86/kernel/cpu/common.c void native_write_cr4(unsigned long val) · line 389

Quoted 5 lines
arch/x86/kernel/cpu/common.c +2 -0
@@ -385,6 +385,11 @@ void native_write_cr4(unsigned long val)
 		/* Warn after we've set the missing bits. */
 		WARN_ONCE(bits_missing, "CR4 bits went missing: %lx!?\n",
 			  bits_missing);
+		if (val & X86_CR4_FSGSBASE) {
+			WARN_ONCE(1, "CR4 unexpectedly set FSGSBASE!?\n");

Like this will actually be noticed by anyone who calls this? What is a user supposed to do about this?

What about those systems that panic-on-warn?

arch/x86/kernel/cpu/common.c void native_write_cr4(unsigned long val) · line 390

Quoted 1 line
arch/x86/kernel/cpu/common.c +1 -0
+			val &= ~X86_CR4_FSGSBASE;

So you just prevented them from setting this, thereby fixing up their broken code that will never be fixed because you did this? Why do this?

thanks,

greg k-h

#2 lore raw
Quoted 11 lines · Greg KH
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now? Why stop with this type of check, why not just forbid them entirely if we don't trust them? :)

Oh, I have a bunch of patches pending for that :-)

It will basically decode the module text and refuse to load the module for most CPL0 instruction.

maintainer
#3 lore raw
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Greg KH
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

Quoted 5 lines
Quoted 3 lines

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now? Why stop with this type of check, why not just forbid them entirely if we don't trust them? :)

Oh, I have a bunch of patches pending for that :-)

Ah, I thought I had seen something like that go by a while ago.

It's sad that we have to write a "don't do stupid things" checker for kernel modules now :(

Quoted 2 lines

It will basically decode the module text and refuse to load the module for most CPL0 instruction.

Ok, so why would Andi's patch even be needed then? Andi, why post this?

thanks,

greg k-h

#4 lore raw
Quoted 14 lines · Greg KH
Quoted 13 lines · Peter Zijlstra
Quoted 11 lines · Greg KH
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now? Why stop with this type of check, why not just forbid them entirely if we don't trust them? :)

Quoted 6 lines
Quoted 1 line

Oh, I have a bunch of patches pending for that :-)

Ah, I thought I had seen something like that go by a while ago.

It's sad that we have to write a "don't do stupid things" checker for kernel modules now :(

Because people... they get stuff from the interweb and run it :/ The days that admins actually knew what they're doing is long long gone.

Quoted 4 lines
Quoted 2 lines

It will basically decode the module text and refuse to load the module for most CPL0 instruction.

Ok, so why would Andi's patch even be needed then? Andi, why post this?

Andi's patch cures a particularly bad module that floats around that people use, probably without being aware that it's an insta-root hole.

My patches will be a while (too many things in the fire :/) and will certainly not be for stable.

maintainer
#5 lore raw
Quoted 17 lines · Peter Zijlstra
Quoted 16 lines · Greg KH
Quoted 14 lines · Peter Zijlstra
Quoted 11 lines · Greg KH
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now? Why stop with this type of check, why not just forbid them entirely if we don't trust them? :)

Oh, I have a bunch of patches pending for that :-)

Quoted 7 lines
Quoted 4 lines

Ah, I thought I had seen something like that go by a while ago.

It's sad that we have to write a "don't do stupid things" checker for kernel modules now :(

Because people... they get stuff from the interweb and run it :/ The days that admins actually knew what they're doing is long long gone.

{sigh}

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

It will basically decode the module text and refuse to load the module for most CPL0 instruction.

Ok, so why would Andi's patch even be needed then? Andi, why post this?

Andi's patch cures a particularly bad module that floats around that people use, probably without being aware that it's an insta-root hole.

Ok, fair enough, thanks for the context.

greg k-h

#6 lore raw
In reply to Greg KH
Quoted 10 lines · Greg KH
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now?

Well it's a specific case where we know they're opening a root hole unintentionally. This is just an pragmatic attempt to protect the users in the short term.

Quoted 2 lines

Why stop with this type of check, why not just forbid them entirely if we don't trust them? :)

Would be pointless -- lots of people rely on them, so such a rule wouldn't survive very long in production kernels.

Quoted 5 lines
Quoted 5 lines
arch/x86/kernel/cpu/common.c +0 -0
index bed0cb83fe24..1f5b7871ae9a 100644
@@ -385,6 +385,11 @@ void native_write_cr4(unsigned long val)
Quoted 8 lines

arch/x86/kernel/cpu/common.c void native_write_cr4(unsigned long val) · line 389

Quoted 5 lines
arch/x86/kernel/cpu/common.c +2 -0
 		/* Warn after we've set the missing bits. */
 		WARN_ONCE(bits_missing, "CR4 bits went missing: %lx!?\n",
 			  bits_missing);
+		if (val & X86_CR4_FSGSBASE) {
+			WARN_ONCE(1, "CR4 unexpectedly set FSGSBASE!?\n");

Like this will actually be noticed by anyone who calls this? What is a user supposed to do about this?

In the long term they would need to apply the proper patches for FSGSBASE.

Quoted 2 lines

What about those systems that panic-on-warn?

I assume they're ok with "panic on root hole"

Quoted 5 lines

arch/x86/kernel/cpu/common.c void native_write_cr4(unsigned long val) · line 390

Quoted 2 lines
arch/x86/kernel/cpu/common.c +1 -0
+			val &= ~X86_CR4_FSGSBASE;

So you just prevented them from setting this, thereby fixing up their broken code that will never be fixed because you did this? Why do this?

If they rely on the functionality they will apply the proper patches then. Or at least they will be aware that they have a root hole, which they are currently not.

-Andi

#7 lore raw
Quoted 8 lines · Andi Kleen
Quoted 5 lines · Greg KH

arch/x86/kernel/cpu/common.c void native_write_cr4(unsigned long val) · line 389

Quoted 2 lines · Andi Kleen
arch/x86/kernel/cpu/common.c +2 -0
+		if (val & X86_CR4_FSGSBASE) {
+			WARN_ONCE(1, "CR4 unexpectedly set FSGSBASE!?\n");

What about those systems that panic-on-warn?

I assume they're ok with "panic on root hole"

Exactly. :) The pinning infrastructure is pretty small; will that just get backported? (Also, we can probably rework the pinning to avoid the special-casing and use a mask/value pair to notice a bit getting turned _on_ as well...)

Signature · 1 line
Kees Cook
maintainer
#8 lore raw
In reply to Andi Kleen
Quoted 9 lines · Andi Kleen
Quoted 8 lines · Greg KH
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

Quoted 6 lines
Quoted 2 lines

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now?

Well it's a specific case where we know they're opening a root hole unintentionally. This is just an pragmatic attempt to protect the users in the short term.

Can't you just go and fix those out-of-tree kernel modules instead? What's keeping you all from just doing that instead of trying to force the kernel to play traffic cop?

thanks,

greg k-h

#9 lore raw
Quoted 13 lines · Greg KH
Quoted 12 lines · Andi Kleen
Quoted 10 lines · Greg KH
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now?

Quoted 7 lines
Quoted 3 lines

Well it's a specific case where we know they're opening a root hole unintentionally. This is just an pragmatic attempt to protect the users in the short term.

Can't you just go and fix those out-of-tree kernel modules instead? What's keeping you all from just doing that instead of trying to force the kernel to play traffic cop?

We'd very much welcome any help really, but we're under impression that this couldn't be done correctly in a module, so this hack occured.

This was written in 2015 as part of original (research) codebase for those reasons:
- A module is easier to deploy by scientists, who are no kernel developers and no sysadmins either, so applying patchset and recompiling kernel is a big ask.
- It has no implications on security in SGX/Graphene threat model and in expected deployment scenario.
- This had no meaning to the actual research being done, so it wasn't cared about.

Let me expand the second point, because I understand both the module and the explanation looks wrong.

Graphene is intended to be run in a cloud, where the CPU time is sold in a form of virtual machine, so the VM kernel, which would load this module, is not trusted by hardware owner, so s/he don't care. But the owner of the enclave also doesn't care, because SGX' threat model assumes adversary who is capable of arbitrary code execution in both kernel and userspace outside enclave. So the kernel immediately outside the enclave is a no-man's land, untrusted by both sides and forsaken, reduced to a compatibility layer between x86 and ELF.

I acknowledge this is unusual threat model and certainly to mainline developers, who rarely encounter userspace that is more trusted than kernel.

What we've failed at is to properly explain this, because if someone loads this module outside of this expected scenario, will certainly be exposed to a gaping root hole. Therefore we acknowledge this patch and as part of Graphene we'll probably maintain a patchset, until the support is upstream. Right now this will take us some time to change from our current kernel interfaces.

Signature · 7 lines
pozdrawiam / best regards
Wojtek Porczyk
Graphene / Invisible Things Lab
 
 I do not fear computers,
 I fear lack of them.
    -- Isaac Asimov
maintainer
#10 lore raw
Quoted 18 lines · Wojtek Porczyk
Quoted 17 lines · Greg KH
Quoted 15 lines · Andi Kleen
Quoted 10 lines · Greg KH
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now?

Well it's a specific case where we know they're opening a root hole unintentionally. This is just an pragmatic attempt to protect the users in the short term.

Quoted 6 lines
Quoted 3 lines

Can't you just go and fix those out-of-tree kernel modules instead? What's keeping you all from just doing that instead of trying to force the kernel to play traffic cop?

We'd very much welcome any help really, but we're under impression that this couldn't be done correctly in a module, so this hack occured.

Really? How is this hack anything other than a "prevent a kernel module from doing something foolish" change?

Why can't you just change the kernel module's code to not do this? What prevents that from happening right now which would prevent the need to change a core api from being abused in such a way?

Quoted 25 lines

This was written in 2015 as part of original (research) codebase for those reasons:
- A module is easier to deploy by scientists, who are no kernel developers and no sysadmins either, so applying patchset and recompiling kernel is a big ask.
- It has no implications on security in SGX/Graphene threat model and in expected deployment scenario.
- This had no meaning to the actual research being done, so it wasn't cared about.

Let me expand the second point, because I understand both the module and the explanation looks wrong.

Graphene is intended to be run in a cloud, where the CPU time is sold in a form of virtual machine, so the VM kernel, which would load this module, is not trusted by hardware owner, so s/he don't care. But the owner of the enclave also doesn't care, because SGX' threat model assumes adversary who is capable of arbitrary code execution in both kernel and userspace outside enclave. So the kernel immediately outside the enclave is a no-man's land, untrusted by both sides and forsaken, reduced to a compatibility layer between x86 and ELF.

I acknowledge this is unusual threat model and certainly to mainline developers, who rarely encounter userspace that is more trusted than kernel.

Quoted 6 lines

What we've failed at is to properly explain this, because if someone loads this module outside of this expected scenario, will certainly be exposed to a gaping root hole. Therefore we acknowledge this patch and as part of Graphene we'll probably maintain a patchset, until the support is upstream. Right now this will take us some time to change from our current kernel interfaces.

I'm sorry, but I still do not understand. Your kernel module calls the core with this bit being set, and this new kernel patch is there to prevent the bit from being set and will WARN_ON() if it happens. Why can't you just change your module code to not set the bit?

Do you have a pointer to the kernel module code that does this operation which this core kernel change will try to prevent from happening?

thanks,

greg k-h

#11 lore raw
Quoted 23 lines · Greg KH
Quoted 22 lines · Wojtek Porczyk
Quoted 20 lines · Greg KH
Quoted 9 lines · Andi Kleen
Quoted 8 lines · Greg KH
Quoted 6 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

Quoted 6 lines
Quoted 2 lines

So we are trying to "protect" ourselves from broken out-of-tree kernel modules now?

Well it's a specific case where we know they're opening a root hole unintentionally. This is just an pragmatic attempt to protect the users in the short term.

Can't you just go and fix those out-of-tree kernel modules instead? What's keeping you all from just doing that instead of trying to force the kernel to play traffic cop?

Quoted 5 lines
Quoted 2 lines

We'd very much welcome any help really, but we're under impression that this couldn't be done correctly in a module, so this hack occured.

Really? How is this hack anything other than a "prevent a kernel module from doing something foolish" change?

By "this hack" I meant our module [1], which just enables FSGSBASE bit without doing everything else that Sasha's patchset does, and that "everything else" is there to prevent a gaping root hoole.

Original author wanted module for the reason snipped below, but Sasha's patchset can't be packaged into module. I'll be happy to be corrected on this point, I personally have next to no kernel programming experience.

This kernel change I think is correct, because if kernel assumes some invariants, it's a good idea to actually enforce them, isn't it? So we don't have anything against this kernel change. We'll have to live with it, with our hand forced.

Quoted 3 lines

Why can't you just change the kernel module's code to not do this? What prevents that from happening right now which would prevent the need to change a core api from being abused in such a way?

[snip]

Quoted 4 lines

I'm sorry, but I still do not understand. Your kernel module calls the core with this bit being set, and this new kernel patch is there to prevent the bit from being set and will WARN_ON() if it happens. Why can't you just change your module code to not set the bit?

Because we need userspace access to wrfsbase, which this bit enables:

https://github.com/oscarlab/graphene/blob/58c53ad747579225bf29e3506d883586ff4b8eee/Pal/src/host/Linux-SGX/sgx_api.h#L94-L98
https://github.com/oscarlab/graphene/blob/0dd84ddf943d256e5494f07cb41b1d0ece847c1a/Pal/src/host/Linux-SGX/enclave_entry.S#L675
https://github.com/oscarlab/graphene/blob/e4e16aa10e3c2221170aee7da66370507cc52428/Pal/src/host/Linux-SGX/db_misc.c#L69

Quoted 2 lines

Do you have a pointer to the kernel module code that does this operation which this core kernel change will try to prevent from happening?

Sure: https://github.com/oscarlab/graphene-sgx-driver/blob/a73de5fed96fc330fc0417d262ba5e87fea128c2/gsgx.c#L32-L39

The whole module is 86 sloc, and the sole purpose is to set this bit on load and clear on unload.

[1] ^

Signature · 7 lines
pozdrawiam / best regards
Wojtek Porczyk
Graphene / Invisible Things Lab
 
 I do not fear computers,
 I fear lack of them.
    -- Isaac Asimov
#14 lore raw
In reply to Andi Kleen
Quoted 38 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Since there seem to be kernel modules floating around that set FSGSBASE incorrectly, prevent this in the CR4 pinning. Currently CR4 pinning just checks that bits are set, this also checks that the FSGSBASE bit is not set, and if it is clears it again.

Note this patch will need to be undone when the full FSGSBASE patches are merged. But it's a reasonable solution for v5.2+ stable at least. Sadly the older kernels don't have the necessary infrastructure for this (although a simpler version of this could be added there too)

---
 arch/x86/kernel/cpu/common.c | 5 +++++
 1 file changed, 5 insertions(+)
arch/x86/kernel/cpu/common.c +5 -0
index bed0cb83fe24..1f5b7871ae9a 100644
@@ -385,6 +385,11 @@ void native_write_cr4(unsigned long val)
 		/* Warn after we've set the missing bits. */
 		WARN_ONCE(bits_missing, "CR4 bits went missing: %lx!?\n",
 			  bits_missing);
+		if (val & X86_CR4_FSGSBASE) {
+			WARN_ONCE(1, "CR4 unexpectedly set FSGSBASE!?\n");
+			val &= ~X86_CR4_FSGSBASE;
+			goto set_register;
+		}
 	}
 }
 EXPORT_SYMBOL(native_write_cr4);
Signature · 1 line
2.25.4

And if this is going to be more permanent, we can separate the mask (untested):

arch/x86/kernel/cpu/common.c +10 -11
index bed0cb83fe24..ead64f7420a5 100644
@@ -347,6 +347,8 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c)
 	cr4_clear_bits(X86_CR4_UMIP);
 }
 
+static const unsigned long cr4_pinned_mask =
+	X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP | X86_CR4_FSGSBASE;
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 
@@ -371,20 +373,20 @@ EXPORT_SYMBOL(native_write_cr0);
 
 void native_write_cr4(unsigned long val)
 {
-	unsigned long bits_missing = 0;
+	unsigned long bits_changed = 0;
 
 set_register:
 	asm volatile("mov %0,%%cr4": "+r" (val), "+m" (cr4_pinned_bits));
 
 	if (static_branch_likely(&cr_pinning)) {
-		if (unlikely((val & cr4_pinned_bits) != cr4_pinned_bits)) {
-			bits_missing = ~val & cr4_pinned_bits;
-			val |= bits_missing;
+		if (unlikely((val & cr4_pinned_mask) != cr4_pinned_bits)) {
+			bits_changed = ~val & cr4_pinned_mask;
+			val = (val & ~cr4_pinned_mask) | cr4_pinned_bits;
 			goto set_register;
 		}
 		/* Warn after we've set the missing bits. */
-		WARN_ONCE(bits_missing, "CR4 bits went missing: %lx!?\n",
-			  bits_missing);
+		WARN_ONCE(bits_changed, "pinned CR4 bits changed: %lx!?\n",
+			  bits_changed);
 	}
 }
 EXPORT_SYMBOL(native_write_cr4);
@@ -396,7 +398,7 @@ void cr4_init(void)
 	if (boot_cpu_has(X86_FEATURE_PCID))
 		cr4 |= X86_CR4_PCIDE;
 	if (static_branch_likely(&cr_pinning))
-		cr4 |= cr4_pinned_bits;
+		cr4 = (cr4 & ~cr4_pinned_mask) | cr4_pinned_bits;
 
 	__write_cr4(cr4);
 
@@ -411,10 +413,7 @@ void cr4_init(void)
  */
 static void __init setup_cr_pinning(void)
 {
-	unsigned long mask;
-
-	mask = (X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP);
-	cr4_pinned_bits = this_cpu_read(cpu_tlbstate.cr4) & mask;
+	cr4_pinned_bits = this_cpu_read(cpu_tlbstate.cr4) & cr4_pinned_mask;
 	static_key_enable(&cr_pinning.key);
 }
 
Signature · 1 line
Kees Cook
author
#15 lore raw
Quoted 2 lines

And if this is going to be more permanent, we can separate the mask (untested):

The FSGSBASE one should not be permanent, it will be replaced with the full FSGSBASE patches that set that bit correctly.

I was a bit wary of enforcing it for all bits, there might be other CR4 bits which have benign uses. But I guess the risk of breaking something there is low.

-Andi

Quoted 67 lines
arch/x86/kernel/cpu/common.c +10 -11
index bed0cb83fe24..ead64f7420a5 100644
@@ -347,6 +347,8 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c)
 	cr4_clear_bits(X86_CR4_UMIP);
 }
 
+static const unsigned long cr4_pinned_mask =
+	X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP | X86_CR4_FSGSBASE;
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 
@@ -371,20 +373,20 @@ EXPORT_SYMBOL(native_write_cr0);
 
 void native_write_cr4(unsigned long val)
 {
-	unsigned long bits_missing = 0;
+	unsigned long bits_changed = 0;
 
 set_register:
 	asm volatile("mov %0,%%cr4": "+r" (val), "+m" (cr4_pinned_bits));
 
 	if (static_branch_likely(&cr_pinning)) {
-		if (unlikely((val & cr4_pinned_bits) != cr4_pinned_bits)) {
-			bits_missing = ~val & cr4_pinned_bits;
-			val |= bits_missing;
+		if (unlikely((val & cr4_pinned_mask) != cr4_pinned_bits)) {
+			bits_changed = ~val & cr4_pinned_mask;
+			val = (val & ~cr4_pinned_mask) | cr4_pinned_bits;
 			goto set_register;
 		}
 		/* Warn after we've set the missing bits. */
-		WARN_ONCE(bits_missing, "CR4 bits went missing: %lx!?\n",
-			  bits_missing);
+		WARN_ONCE(bits_changed, "pinned CR4 bits changed: %lx!?\n",
+			  bits_changed);
 	}
 }
 EXPORT_SYMBOL(native_write_cr4);
@@ -396,7 +398,7 @@ void cr4_init(void)
 	if (boot_cpu_has(X86_FEATURE_PCID))
 		cr4 |= X86_CR4_PCIDE;
 	if (static_branch_likely(&cr_pinning))
-		cr4 |= cr4_pinned_bits;
+		cr4 = (cr4 & ~cr4_pinned_mask) | cr4_pinned_bits;
 
 	__write_cr4(cr4);
 
@@ -411,10 +413,7 @@ void cr4_init(void)
  */
 static void __init setup_cr_pinning(void)
 {
-	unsigned long mask;
-
-	mask = (X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP);
-	cr4_pinned_bits = this_cpu_read(cpu_tlbstate.cr4) & mask;
+	cr4_pinned_bits = this_cpu_read(cpu_tlbstate.cr4) & cr4_pinned_mask;
 	static_key_enable(&cr_pinning.key);
 }
 
Signature · 2 lines
-- 
Kees Cook
#16 lore raw
Quoted 5 lines · Andi Kleen
Quoted 2 lines

And if this is going to be more permanent, we can separate the mask (untested):

The FSGSBASE one should not be permanent, it will be replaced with the full FSGSBASE patches that set that bit correctly.

Well, even with full FSGSBASE patches on, that CR4 bit should never get changed after boot.