Skip to contentSkip to conversation

Revert "KVM: x86: WARN and reject loading KVM if NX is supported but not enabled"

PATCH In review Sean Christopherson 3 participants 8 messages Jun 25, 2021 – Jul 21, 2021

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

PATCH 1/1 Revert "KVM: x86: WARN and reject loading KVM if NX is supported but not enabled"

Let KVM load if EFER.NX=0 even if NX is supported, the analysis and testing (or lack thereof) for the non-PAE host case was garbage.

If the kernel won't be using PAE paging, .Ldefault_entry in head_32.S skips over the entire EFER sequence. Hopefully that can be changed in the future to allow KVM to require EFER.NX, but the motivation behind KVM's requirement isn't yet merged. Reverting and revisiting the mess at a later date is by far the safest approach.

This reverts commit 8bbed95d2cb6e5de8a342d761a89b0a04faed7be.

Fixes 8bbed95d2cb6 ("KVM: x86: WARN and reject loading KVM if NX is supported but not enabled")

---

Hopefully it's not too late to just drop the original patch...

 arch/x86/kvm/x86.c | 3 ---
 1 file changed, 3 deletions(-)
arch/x86/kvm/x86.c +0 -3
index 4a597aafe637..1cc02a3685d0 100644
@@ -10981,9 +10981,6 @@ int kvm_arch_hardware_setup(void *opaque)
 	int r;
 
 	rdmsrl_safe(MSR_EFER, &host_efer);
-	if (WARN_ON_ONCE(boot_cpu_has(X86_FEATURE_NX) &&
-			 !(host_efer & EFER_NX)))
-		return -EIO;
 
 	if (boot_cpu_has(X86_FEATURE_XSAVES))
 		rdmsrl(MSR_IA32_XSS, host_xss);
Signature · 1 line
2.32.0.93.g670b81a890-goog
maintainer
#1 lore raw
Quoted 26 lines · Sean Christopherson

Let KVM load if EFER.NX=0 even if NX is supported, the analysis and testing (or lack thereof) for the non-PAE host case was garbage.

If the kernel won't be using PAE paging, .Ldefault_entry in head_32.S skips over the entire EFER sequence. Hopefully that can be changed in the future to allow KVM to require EFER.NX, but the motivation behind KVM's requirement isn't yet merged. Reverting and revisiting the mess at a later date is by far the safest approach.

This reverts commit 8bbed95d2cb6e5de8a342d761a89b0a04faed7be.

Fixes 8bbed95d2cb6 ("KVM: x86: WARN and reject loading KVM if NX is supported but not enabled")

---

Hopefully it's not too late to just drop the original patch...

  arch/x86/kvm/x86.c | 3 ---
  1 file changed, 3 deletions(-)
arch/x86/kvm/x86.c +0 -0
index 4a597aafe637..1cc02a3685d0 100644

arch/x86/kvm/x86.c int kvm_arch_hardware_setup(void *opaque) · line 10986

Quoted 9 lines
arch/x86/kvm/x86.c +0 -3
@@ -10981,9 +10981,6 @@ int kvm_arch_hardware_setup(void *opaque)
  	int r;
 
  	rdmsrl_safe(MSR_EFER, &host_efer);
-	if (WARN_ON_ONCE(boot_cpu_has(X86_FEATURE_NX) &&
-			 !(host_efer & EFER_NX)))
-		return -EIO;
 
  	if (boot_cpu_has(X86_FEATURE_XSAVES))
  		rdmsrl(MSR_IA32_XSS, host_xss);

So do we want this or "depends on X86_64 || X86_PAE"?

Paolo

author maintainer
#2 lore raw
Quoted 21 lines · Paolo Bonzini
Quoted 20 lines · Sean Christopherson

Let KVM load if EFER.NX=0 even if NX is supported, the analysis and testing (or lack thereof) for the non-PAE host case was garbage.

If the kernel won't be using PAE paging, .Ldefault_entry in head_32.S skips over the entire EFER sequence. Hopefully that can be changed in the future to allow KVM to require EFER.NX, but the motivation behind KVM's requirement isn't yet merged. Reverting and revisiting the mess at a later date is by far the safest approach.

This reverts commit 8bbed95d2cb6e5de8a342d761a89b0a04faed7be.

Fixes 8bbed95d2cb6 ("KVM: x86: WARN and reject loading KVM if NX is supported but not enabled")

---

Hopefully it's not too late to just drop the original patch...

  arch/x86/kvm/x86.c | 3 ---
  1 file changed, 3 deletions(-)
Quoted 15 lines

arch/x86/kvm/x86.c int kvm_arch_hardware_setup(void *opaque) · line 10985

Quoted 13 lines
arch/x86/kvm/x86.c +0 -3
index 4a597aafe637..1cc02a3685d0 100644
@@ -10981,9 +10981,6 @@ int kvm_arch_hardware_setup(void *opaque)
  	int r;
  	rdmsrl_safe(MSR_EFER, &host_efer);
-	if (WARN_ON_ONCE(boot_cpu_has(X86_FEATURE_NX) &&
-			 !(host_efer & EFER_NX)))
-		return -EIO;
  	if (boot_cpu_has(X86_FEATURE_XSAVES))
  		rdmsrl(MSR_IA32_XSS, host_xss);
 

So do we want this or "depends on X86_64 || X86_PAE"?

Hmm, I'm leaning towards keeping !PAE support purely for testing the !PAE<->PAE MMU transitions for nested virtualization. It's not much coverage, and the !PAE NPT horror is a much bigger testing gap (because KVM doesn't support it), but on the other hand setting EFER.NX for !PAE kernels appears to be trivial, e.g.

arch/x86/kernel/head_32.S +0 -6
index 67f590425d90..bfbea25a9fe8 100644
@@ -214,12 +214,6 @@ SYM_FUNC_START(startup_32_smp)
        andl $~1,%edx                   # Ignore CPUID.FPU
        jz .Lenable_paging              # No flags or only CPUID.FPU = no CR4
 
-       movl pa(mmu_cr4_features),%eax
-       movl %eax,%cr4
-
-       testb $X86_CR4_PAE, %al         # check if PAE is enabled
-       jz .Lenable_paging
-
        /* Check if extended functions are implemented */
        movl $0x80000000, %eax
        cpuid

My only hesitation is the risk of somehow breaking ancient CPUs by falling into the NX path. Maybe try forcing EFER.NX=1 for !PAE, and fall back to requiring PAE if that gets NAK'd or needs to be reverted for whatever reason?

maintainer
#3 lore raw
Quoted 61 lines · Sean Christopherson
Quoted 21 lines · Paolo Bonzini
Quoted 20 lines · Sean Christopherson

Let KVM load if EFER.NX=0 even if NX is supported, the analysis and testing (or lack thereof) for the non-PAE host case was garbage.

If the kernel won't be using PAE paging, .Ldefault_entry in head_32.S skips over the entire EFER sequence. Hopefully that can be changed in the future to allow KVM to require EFER.NX, but the motivation behind KVM's requirement isn't yet merged. Reverting and revisiting the mess at a later date is by far the safest approach.

This reverts commit 8bbed95d2cb6e5de8a342d761a89b0a04faed7be.

Fixes 8bbed95d2cb6 ("KVM: x86: WARN and reject loading KVM if NX is supported but not enabled")

---

Hopefully it's not too late to just drop the original patch...

   arch/x86/kvm/x86.c | 3 ---
   1 file changed, 3 deletions(-)
Quoted 15 lines

arch/x86/kvm/x86.c int kvm_arch_hardware_setup(void *opaque) · line 10985

Quoted 13 lines
arch/x86/kvm/x86.c +0 -3
index 4a597aafe637..1cc02a3685d0 100644
@@ -10981,9 +10981,6 @@ int kvm_arch_hardware_setup(void *opaque)
   	int r;
   	rdmsrl_safe(MSR_EFER, &host_efer);
-	if (WARN_ON_ONCE(boot_cpu_has(X86_FEATURE_NX) &&
-			 !(host_efer & EFER_NX)))
-		return -EIO;
   	if (boot_cpu_has(X86_FEATURE_XSAVES))
   		rdmsrl(MSR_IA32_XSS, host_xss);
 

So do we want this or "depends on X86_64 || X86_PAE"?

Hmm, I'm leaning towards keeping !PAE support purely for testing the !PAE<->PAE MMU transitions for nested virtualization. It's not much coverage, and the !PAE NPT horror is a much bigger testing gap (because KVM doesn't support it), but on the other hand setting EFER.NX for !PAE kernels appears to be trivial, e.g.

arch/x86/kernel/head_32.S +0 -6
index 67f590425d90..bfbea25a9fe8 100644
@@ -214,12 +214,6 @@ SYM_FUNC_START(startup_32_smp)
         andl $~1,%edx                   # Ignore CPUID.FPU
         jz .Lenable_paging              # No flags or only CPUID.FPU = no CR4
 
-       movl pa(mmu_cr4_features),%eax
-       movl %eax,%cr4
-
-       testb $X86_CR4_PAE, %al         # check if PAE is enabled
-       jz .Lenable_paging
-
         /* Check if extended functions are implemented */
         movl $0x80000000, %eax
         cpuid
Quoted 4 lines

My only hesitation is the risk of somehow breaking ancient CPUs by falling into the NX path. Maybe try forcing EFER.NX=1 for !PAE, and fall back to requiring PAE if that gets NAK'd or needs to be reverted for whatever reason?

For now let's revert anyway. Thanks!

Paolo

#4 lore raw
In reply to Sean Christopherson
Quoted 38 lines · Sean Christopherson
Quoted 21 lines · Paolo Bonzini
Quoted 20 lines · Sean Christopherson

Let KVM load if EFER.NX=0 even if NX is supported, the analysis and testing (or lack thereof) for the non-PAE host case was garbage.

If the kernel won't be using PAE paging, .Ldefault_entry in head_32.S skips over the entire EFER sequence. Hopefully that can be changed in the future to allow KVM to require EFER.NX, but the motivation behind KVM's requirement isn't yet merged. Reverting and revisiting the mess at a later date is by far the safest approach.

This reverts commit 8bbed95d2cb6e5de8a342d761a89b0a04faed7be.

Fixes 8bbed95d2cb6 ("KVM: x86: WARN and reject loading KVM if NX is supported but not enabled")

---

Hopefully it's not too late to just drop the original patch...

  arch/x86/kvm/x86.c | 3 ---
  1 file changed, 3 deletions(-)
Quoted 15 lines

arch/x86/kvm/x86.c int kvm_arch_hardware_setup(void *opaque) · line 10985

Quoted 13 lines
arch/x86/kvm/x86.c +0 -3
index 4a597aafe637..1cc02a3685d0 100644
@@ -10981,9 +10981,6 @@ int kvm_arch_hardware_setup(void *opaque)
  	int r;
  	rdmsrl_safe(MSR_EFER, &host_efer);
-	if (WARN_ON_ONCE(boot_cpu_has(X86_FEATURE_NX) &&
-			 !(host_efer & EFER_NX)))
-		return -EIO;
  	if (boot_cpu_has(X86_FEATURE_XSAVES))
  		rdmsrl(MSR_IA32_XSS, host_xss);
 

So do we want this or "depends on X86_64 || X86_PAE"?

Quoted 2 lines

Hmm, I'm leaning towards keeping !PAE support purely for testing the !PAE<->PAE MMU transitions for nested virtualization. It's not much coverage, and the !PAE

May I ask what "!PAE<->PAE MMU transition for nested virtualization" means? Running L1 KVM with !PAE and L0 in PAE? I had thought KVM can only function with PAE set(though I did not see any check of CR4 in kvm_arch_init()). Did I miss something?

Quoted 21 lines

NPT horror is a much bigger testing gap (because KVM doesn't support it), but on the other hand setting EFER.NX for !PAE kernels appears to be trivial, e.g.

arch/x86/kernel/head_32.S +0 -6
index 67f590425d90..bfbea25a9fe8 100644
@@ -214,12 +214,6 @@ SYM_FUNC_START(startup_32_smp)
        andl $~1,%edx                   # Ignore CPUID.FPU
        jz .Lenable_paging              # No flags or only CPUID.FPU = no CR4
 
-       movl pa(mmu_cr4_features),%eax
-       movl %eax,%cr4
-
-       testb $X86_CR4_PAE, %al         # check if PAE is enabled
-       jz .Lenable_paging
-
        /* Check if extended functions are implemented */
        movl $0x80000000, %eax
        cpuid
Quoted 4 lines

My only hesitation is the risk of somehow breaking ancient CPUs by falling into the NX path. Maybe try forcing EFER.NX=1 for !PAE, and fall back to requiring PAE if that gets NAK'd or needs to be reverted for whatever reason?

One more dumb question: are you planning to set NX for linux with !PAE? Why do we need EFER in that case? Thanks! :)

B.R.
Yu

author maintainer
#5 lore raw
Quoted 11 lines · Yu Zhang
Quoted 5 lines · Sean Christopherson
Quoted 1 line · Paolo Bonzini

So do we want this or "depends on X86_64 || X86_PAE"?

Hmm, I'm leaning towards keeping !PAE support purely for testing the !PAE<->PAE MMU transitions for nested virtualization. It's not much coverage, and the !PAE

May I ask what "!PAE<->PAE MMU transition for nested virtualization" means? Running L1 KVM with !PAE and L0 in PAE? I had thought KVM can only function with PAE set(though I did not see any check of CR4 in kvm_arch_init()). Did I miss something?

When L1 uses shadow paging, L0 KVM's uses a single MMU instance for both L1 and L2, and relies on the MMU role to differentiate between L1 and L2. KVM requires PAE for shadow paging, but does not require PAE in the host kernel. So when L1 KVM uses shadow paging, it can effectively use !PAE paging for L1 and PAE paging for L2. L0 KVM needs to handle that the !PAE<->PAE transitions when switching between L1 and L2, e.g. needs to correctly reinitialize the MMU context.

Quoted 21 lines
Quoted 21 lines

NPT horror is a much bigger testing gap (because KVM doesn't support it), but on the other hand setting EFER.NX for !PAE kernels appears to be trivial, e.g.

arch/x86/kernel/head_32.S +0 -6
index 67f590425d90..bfbea25a9fe8 100644
@@ -214,12 +214,6 @@ SYM_FUNC_START(startup_32_smp)
        andl $~1,%edx                   # Ignore CPUID.FPU
        jz .Lenable_paging              # No flags or only CPUID.FPU = no CR4
 
-       movl pa(mmu_cr4_features),%eax
-       movl %eax,%cr4
-
-       testb $X86_CR4_PAE, %al         # check if PAE is enabled
-       jz .Lenable_paging
-
        /* Check if extended functions are implemented */
        movl $0x80000000, %eax
        cpuid
Quoted 6 lines
Quoted 4 lines

My only hesitation is the risk of somehow breaking ancient CPUs by falling into the NX path. Maybe try forcing EFER.NX=1 for !PAE, and fall back to requiring PAE if that gets NAK'd or needs to be reverted for whatever reason?

One more dumb question: are you planning to set NX for linux with !PAE?

Yep.

Quoted 1 line

Why do we need EFER in that case? Thanks! :)

Because as you rightly remembered above, KVM always uses PAE paging for the guest, even when the host is !PAE. And KVM also requires EFER.NX=1 for the guest when using shadow paging to handle a potential SMEP and !WP case.

#6 lore raw
Quoted 13 lines · Sean Christopherson
Quoted 11 lines · Yu Zhang
Quoted 5 lines · Sean Christopherson
Quoted 1 line · Paolo Bonzini

So do we want this or "depends on X86_64 || X86_PAE"?

Hmm, I'm leaning towards keeping !PAE support purely for testing the !PAE<->PAE MMU transitions for nested virtualization. It's not much coverage, and the !PAE

May I ask what "!PAE<->PAE MMU transition for nested virtualization" means? Running L1 KVM with !PAE and L0 in PAE? I had thought KVM can only function with PAE set(though I did not see any check of CR4 in kvm_arch_init()). Did I miss something?

Quoted 6 lines

When L1 uses shadow paging, L0 KVM's uses a single MMU instance for both L1 and L2, and relies on the MMU role to differentiate between L1 and L2. KVM requires PAE for shadow paging, but does not require PAE in the host kernel. So when L1 KVM uses shadow paging, it can effectively use !PAE paging for L1 and PAE paging for L2. L0 KVM needs to handle that the !PAE<->PAE transitions when switching between L1 and L2, e.g. needs to correctly reinitialize the MMU context.

Hah... Actually, I do have a misunderstanding here. The host does not need to be PAE. Thanks for the explanation! :)

Quoted 29 lines
Quoted 22 lines
Quoted 22 lines

NPT horror is a much bigger testing gap (because KVM doesn't support it), but on the other hand setting EFER.NX for !PAE kernels appears to be trivial, e.g.

arch/x86/kernel/head_32.S +0 -6
index 67f590425d90..bfbea25a9fe8 100644
@@ -214,12 +214,6 @@ SYM_FUNC_START(startup_32_smp)
        andl $~1,%edx                   # Ignore CPUID.FPU
        jz .Lenable_paging              # No flags or only CPUID.FPU = no CR4
 
-       movl pa(mmu_cr4_features),%eax
-       movl %eax,%cr4
-
-       testb $X86_CR4_PAE, %al         # check if PAE is enabled
-       jz .Lenable_paging
-
        /* Check if extended functions are implemented */
        movl $0x80000000, %eax
        cpuid
Quoted 6 lines
Quoted 4 lines

My only hesitation is the risk of somehow breaking ancient CPUs by falling into the NX path. Maybe try forcing EFER.NX=1 for !PAE, and fall back to requiring PAE if that gets NAK'd or needs to be reverted for whatever reason?

One more dumb question: are you planning to set NX for linux with !PAE?

Quoted 8 lines

Yep.

Quoted 1 line

Why do we need EFER in that case? Thanks! :)

Because as you rightly remembered above, KVM always uses PAE paging for the guest, even when the host is !PAE. And KVM also requires EFER.NX=1 for the guest when using shadow paging to handle a potential SMEP and !WP case.

Just saw this in update_transition_efer(), which now enables efer.nx in shadow unconditionally. But I guess the host kernel still needs to set efer.nx for !PAE(e.g. in head_32.S), because the guest may not touch efer at all. Is this correct?

B.R.
Yu

author maintainer
#7 lore raw
Quoted 11 lines · Yu Zhang
Quoted 6 lines · Sean Christopherson
Quoted 1 line · Yu Zhang

Why do we need EFER in that case? Thanks! :)

Because as you rightly remembered above, KVM always uses PAE paging for the guest, even when the host is !PAE. And KVM also requires EFER.NX=1 for the guest when using shadow paging to handle a potential SMEP and !WP case.

Just saw this in update_transition_efer(), which now enables efer.nx in shadow unconditionally. But I guess the host kernel still needs to set efer.nx for !PAE(e.g. in head_32.S),

Yep, and that's what I messed up.

Quoted 1 line

because the guest may not touch efer at all. Is this correct?

KVM doesn't require EFER.NX "because the guest may not touch efer at all", it requires EFER.NX to handle scenarios where KVM needs to make a guest page !EXECUTABLE even if EFER is not exposed to the guest (thanks to SMEP && !WP).