KVM: x86: Don't attempt to load PDPTRs when 64-bit mode is enabled
PATCH 1/1 KVM: x86: Don't attempt to load PDPTRs when 64-bit mode is enabled
Don't attempt to load PDPTRs if EFER.LME=1, i.e. if 64-bit mode is enabled. A recent change to reload the PDTPRs when CR0.CD or CR0.NW is toggled botched the EFER.LME handling and sends KVM down the PDTPR path when is_paging() is true, i.e. when the guest toggles CD/NW in 64-bit mode.
Split the CR0 checks for 64-bit vs. 32-bit PAE into separate paths. The 64-bit path is specifically checking state when paging is toggled on, i.e. CR0.PG transititions from 0->1. The PDPTR path now needs to run if the new CR0 state has paging enabled, irrespective of whether paging was already enabled. Trying to shave a few cycles to make the PDPTR path an "else if" case is a mess.
---
The other way to fix this, with a much smaller diff stat, is to simply move the !is_page(vcpu) check inside (vcpu->arch.efer & EFER_LME). But that results in a ridiculous amount of nested conditionals for what is a very straightforward check e.g.
if (cr0 & X86_CR0_PG) {
if (vcpu->arch.efer & EFER_LME) }
if (!is_paging(vcpu)) {
...
}
}
}Since this doesn't need to be backported anywhere, I didn't see any value in having an intermediate step.
arch/x86/kvm/x86.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
arch/x86/kvm/x86.c +12 -12
@@ -819,22 +819,22 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)if ((cr0 &X86_CR0_PG ) && !(cr0 &X86_CR0_PE ))return 1 ; -if (cr0 &X86_CR0_PG ) {#ifdef CONFIG_X86_64 -if (!is_paging (vcpu) && (vcpu->arch.efer &EFER_LME )) { -int cs_db, cs_l; +if ((vcpu->arch.efer &EFER_LME ) && !is_paging (vcpu) && + (cr0 &X86_CR0_PG )) { +int cs_db, cs_l; -if (!is_pae (vcpu)) -return 1 ; - kvm_x86_ops.get_cs_db_l_bits (vcpu, &cs_db, &cs_l); -if (cs_l) -return 1 ; - } else -#endif -if (is_pae (vcpu) && ((cr0 ^ old_cr0) & pdptr_bits) && - !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu))) +if (!is_pae (vcpu)) +return 1 ; + kvm_x86_ops.get_cs_db_l_bits (vcpu, &cs_db, &cs_l); +if (cs_l)return 1 ; } +#endif +if (!(vcpu->arch.efer &EFER_LME ) && (cr0 &X86_CR0_PG ) && +is_pae (vcpu) && ((cr0 ^ old_cr0) & pdptr_bits) && + !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu))) +return 1 ;if (!(cr0 &X86_CR0_PG ) &&kvm_read_cr4_bits (vcpu,X86_CR4_PCIDE ))return 1 ;
Signature · 1 line
2.26.0
Quoted 44 lines · Sean Christopherson
Don't attempt to load PDPTRs if EFER.LME=1, i.e. if 64-bit mode is enabled. A recent change to reload the PDTPRs when CR0.CD or CR0.NW is toggled botched the EFER.LME handling and sends KVM down the PDTPR path when
is_paging()is true, i.e. when the guest toggles CD/NW in 64-bit mode.Split the CR0 checks for 64-bit vs. 32-bit PAE into separate paths. The 64-bit path is specifically checking state when paging is toggled on, i.e. CR0.PG transititions from 0->1. The PDPTR path now needs to run if the new CR0 state has paging enabled, irrespective of whether paging was already enabled. Trying to shave a few cycles to make the PDPTR path an "else if" case is a mess.
Fixes d42e3fae6faed ("kvm: x86: Read PDPTEs on CR0.CD and CR0.NW changes")Cc Jim Mattson <[email protected]>Cc Oliver Upton <[email protected]>Cc Peter Shier <[email protected]>Signed-off-by Sean Christopherson <[email protected]>---
The other way to fix this, with a much smaller diff stat, is to simply move the !is_page(vcpu) check inside (
vcpu->arch.efer&EFER_LME). But that results in a ridiculous amount of nested conditionals for what is a very straightforward check e.g.if (cr0 & X86_CR0_PG) { if (vcpu->arch.efer & EFER_LME) } if (!is_paging(vcpu)) { ... } } }Since this doesn't need to be backported anywhere, I didn't see any value in having an intermediate step.
arch/x86/kvm/x86.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)arch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · line 824
Quoted 9 lines
arch/x86/kvm/x86.c +2
-3@@ -819,22 +819,22 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)if ((cr0 &X86_CR0_PG ) && !(cr0 &X86_CR0_PE ))return 1 ; -if (cr0 &X86_CR0_PG ) {#ifdef CONFIG_X86_64 -if (!is_paging (vcpu) && (vcpu->arch.efer &EFER_LME )) { -int cs_db, cs_l; +if ((vcpu->arch.efer &EFER_LME ) && !is_paging (vcpu) && + (cr0 &X86_CR0_PG )) {
it seems we have more than one occurance of "if (vcpu->arch.efer & EFER_LME)" under "#ifdef CONFIG_X86_64" and we alredy have
static inline int is_long_mode(struct kvm_vcpu *vcpu) {
#ifdef CONFIG_X86_64
return vcpu->arch.efer & EFER_LMA;
#else
return 0;#endif
}
so if we use this instead, the compilers will just throw away the non-reachable blocks when !(#ifdef CONFIG_X86_64), right?
Quoted 25 lines
arch/x86/kvm/x86.c +10
-9+int cs_db, cs_l; -if (!is_pae (vcpu)) -return 1 ; - kvm_x86_ops.get_cs_db_l_bits (vcpu, &cs_db, &cs_l); -if (cs_l) -return 1 ; - } else -#endif -if (is_pae (vcpu) && ((cr0 ^ old_cr0) & pdptr_bits) && - !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu))) +if (!is_pae (vcpu)) +return 1 ; + kvm_x86_ops.get_cs_db_l_bits (vcpu, &cs_db, &cs_l); +if (cs_l) return 1; } +#endif +if (!(vcpu->arch.efer &EFER_LME ) && (cr0 &X86_CR0_PG ) && +is_pae (vcpu) && ((cr0 ^ old_cr0) & pdptr_bits) && + !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu))) +return 1 ;if (!(cr0 &X86_CR0_PG ) &&kvm_read_cr4_bits (vcpu,X86_CR4_PCIDE ))return 1 ;
Signature · 1 line
Vitaly
Quoted 25 lines · Vitaly Kuznetsov
Quoted 5 lines · Sean Christopherson
arch/x86/kvm/x86.c +0
-0arch/x86/kvm/x86.c int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · line 824
Quoted 9 lines
arch/x86/kvm/x86.c +2
-3@@ -819,22 +819,22 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)if ((cr0 &X86_CR0_PG ) && !(cr0 &X86_CR0_PE ))return 1 ; -if (cr0 &X86_CR0_PG ) {#ifdef CONFIG_X86_64 -if (!is_paging (vcpu) && (vcpu->arch.efer &EFER_LME )) { -int cs_db, cs_l; +if ((vcpu->arch.efer &EFER_LME ) && !is_paging (vcpu) && + (cr0 &X86_CR0_PG )) {it seems we have more than one occurance of "if (
vcpu->arch.efer&EFER_LME)" under "#ifdefCONFIG_X86_64" and we alredy havestatic inline int
is_long_mode(struct kvm_vcpu *vcpu){#ifdef CONFIG_X86_64 return vcpu->arch.efer & EFER_LMA; #else return 0;
Quoted 5 lines
#endif
}so if we use this instead, the compilers will just throw away the non-reachable blocks when !(#ifdef
CONFIG_X86_64), right?
EFER.LME vs. EFER.LMA. The kvm_set_cr0() check is specifically looking at the case where EFER.LME=1, EFER.LMA=0, and CR0.PG is being toggled on, i.e. long mode is being enabled. EFER_LMA won't be set until vmx_set_cr0() does enter_lmode().
Quoted 29 lines · Sean Christopherson
Quoted 28 lines · Vitaly Kuznetsov
Quoted 5 lines · Sean Christopherson
arch/x86/kvm/x86.c +0
-0arch/x86/kvm/x86.c int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · line 824
Quoted 9 lines
arch/x86/kvm/x86.c +2
-3@@ -819,22 +819,22 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)if ((cr0 &X86_CR0_PG ) && !(cr0 &X86_CR0_PE ))return 1 ; -if (cr0 &X86_CR0_PG ) {#ifdef CONFIG_X86_64 -if (!is_paging (vcpu) && (vcpu->arch.efer &EFER_LME )) { -int cs_db, cs_l; +if ((vcpu->arch.efer &EFER_LME ) && !is_paging (vcpu) && + (cr0 &X86_CR0_PG )) {it seems we have more than one occurance of "if (
vcpu->arch.efer&EFER_LME)" under "#ifdefCONFIG_X86_64" and we alredy havestatic inline int
is_long_mode(struct kvm_vcpu *vcpu){#ifdef CONFIG_X86_64 return vcpu->arch.efer & EFER_LMA; #else return 0;#endif
}
Quoted 7 lines
Quoted 2 lines
so if we use this instead, the compilers will just throw away the non-reachable blocks when !(#ifdef
CONFIG_X86_64), right?EFER.LME vs. EFER.LMA. The
kvm_set_cr0()check is specifically looking at the case where EFER.LME=1, EFER.LMA=0, and CR0.PG is being toggled on, i.e. long mode is being enabled.EFER_LMAwon't be set untilvmx_set_cr0()doesenter_lmode().
Oops, shame on me :-(
Would it make sense to introduce something like is_long_mode() (e.g. is_efer_lme()) for LME and #ifdef CONFIG_X86_64? I see the same checks in vmx_set_cr0()/svm_set_cr0())?
Signature · 1 line
Vitaly
Quoted 6 lines · Sean Christopherson
Don't attempt to load PDPTRs if EFER.LME=1, i.e. if 64-bit mode is enabled. A recent change to reload the PDTPRs when CR0.CD or CR0.NW is toggled botched the EFER.LME handling and sends KVM down the PDTPR path when
is_paging()is true, i.e. when the guest toggles CD/NW in 64-bit mode.
Oops!
I don't think "is_paging()" is relevant here, so much as "EFER.LME=1." As you note below, KVM *should* go down the PDPTR path when is_paging() is true and EFER.LME=0.
Quoted 7 lines
Split the CR0 checks for 64-bit vs. 32-bit PAE into separate paths. The 64-bit path is specifically checking state when paging is toggled on, i.e. CR0.PG transititions from 0->1. The PDPTR path now needs to run if the new CR0 state has paging enabled, irrespective of whether paging was already enabled. Trying to shave a few cycles to make the PDPTR path an "else if" case is a mess.
Quoted 5 lines
Fixes d42e3fae6faed ("kvm: x86: Read PDPTEs on CR0.CD and CR0.NW changes")Cc Jim Mattson <[email protected]>Cc Oliver Upton <[email protected]>Cc Peter Shier <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 9 lines · Jim Mattson
Quoted 6 lines · Sean Christopherson
Don't attempt to load PDPTRs if EFER.LME=1, i.e. if 64-bit mode is enabled. A recent change to reload the PDTPRs when CR0.CD or CR0.NW is toggled botched the EFER.LME handling and sends KVM down the PDTPR path when
is_paging()is true, i.e. when the guest toggles CD/NW in 64-bit mode.
Quoted 5 lines
Oops!
I don't think "
is_paging()" is relevant here, so much as "EFER.LME=1." As you note below, KVM *should* go down the PDPTR path whenis_paging()is true and EFER.LME=0.
It's relevant for the EFER.LME=1 case as it's used to detect CR0.PG 0->1.
Though maybe we're in violent agreement?
Quoted 13 lines · Sean Christopherson
Quoted 11 lines · Jim Mattson
Quoted 6 lines · Sean Christopherson
Don't attempt to load PDPTRs if EFER.LME=1, i.e. if 64-bit mode is enabled. A recent change to reload the PDTPRs when CR0.CD or CR0.NW is toggled botched the EFER.LME handling and sends KVM down the PDTPR path when
is_paging()is true, i.e. when the guest toggles CD/NW in 64-bit mode.Oops!
Quoted 7 lines
Quoted 3 lines
I don't think "
is_paging()" is relevant here, so much as "EFER.LME=1." As you note below, KVM *should* go down the PDPTR path whenis_paging()is true and EFER.LME=0.It's relevant for the EFER.LME=1 case as it's used to detect CR0.PG 0->1.
Though maybe we're in violent agreement?
We're in agreement conceptually, but I find your original text lacking in clarity. :-)
Quoted 13 lines · Sean Christopherson
Don't attempt to load PDPTRs if EFER.LME=1, i.e. if 64-bit mode is enabled. A recent change to reload the PDTPRs when CR0.CD or CR0.NW is toggled botched the EFER.LME handling and sends KVM down the PDTPR path when
is_paging()is true, i.e. when the guest toggles CD/NW in 64-bit mode.Split the CR0 checks for 64-bit vs. 32-bit PAE into separate paths. The 64-bit path is specifically checking state when paging is toggled on, i.e. CR0.PG transititions from 0->1. The PDPTR path now needs to run if the new CR0 state has paging enabled, irrespective of whether paging was already enabled. Trying to shave a few cycles to make the PDPTR path an "else if" case is a mess.
Quoted 6 lines
Fixes d42e3fae6faed ("kvm: x86: Read PDPTEs on CR0.CD and CR0.NW changes")Cc Jim Mattson <[email protected]>Cc Oliver Upton <[email protected]>Cc Peter Shier <[email protected]>Signed-off-by Sean Christopherson <[email protected]>---
Ping. This really needs to be in the initial pull for 5.9, as is kvm/queue has a 100% fatality rate for me.
Quoted 2 lines · Sean Christopherson
Ping. This really needs to be in the initial pull for 5.9, as is kvm/queue has a 100% fatality rate for me.
I agree completely, but I am curious what guest you have that toggles CD/NW in 64-bit mode.
Quoted 8 lines · Jim Mattson
Quoted 2 lines · Sean Christopherson
Ping. This really needs to be in the initial pull for 5.9, as is kvm/queue has a 100% fatality rate for me.
I agree completely, but I am curious what guest you have that toggles CD/NW in 64-bit mode.
It's my OVMF build. I assume it's MTRR programming, but I haven't bothered to debug that far. Nor do I want to, the EDKII build framework makes my head explode :-).
Quoted 68 lines · Sean Christopherson
Don't attempt to load PDPTRs if EFER.LME=1, i.e. if 64-bit mode is enabled. A recent change to reload the PDTPRs when CR0.CD or CR0.NW is toggled botched the EFER.LME handling and sends KVM down the PDTPR path when
is_paging()is true, i.e. when the guest toggles CD/NW in 64-bit mode.Split the CR0 checks for 64-bit vs. 32-bit PAE into separate paths. The 64-bit path is specifically checking state when paging is toggled on, i.e. CR0.PG transititions from 0->1. The PDPTR path now needs to run if the new CR0 state has paging enabled, irrespective of whether paging was already enabled. Trying to shave a few cycles to make the PDPTR path an "else if" case is a mess.
Fixes d42e3fae6faed ("kvm: x86: Read PDPTEs on CR0.CD and CR0.NW changes")Cc Jim Mattson <[email protected]>Cc Oliver Upton <[email protected]>Cc Peter Shier <[email protected]>Signed-off-by Sean Christopherson <[email protected]>---
The other way to fix this, with a much smaller diff stat, is to simply move the !is_page(vcpu) check inside (
vcpu->arch.efer&EFER_LME). But that results in a ridiculous amount of nested conditionals for what is a very straightforward check e.g.if (cr0 & X86_CR0_PG) { if (vcpu->arch.efer & EFER_LME) } if (!is_paging(vcpu)) { ... } } }Since this doesn't need to be backported anywhere, I didn't see any value in having an intermediate step.
arch/x86/kvm/x86.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)arch/x86/kvm/x86.c +7
-12@@ -819,22 +819,22 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)if ((cr0 &X86_CR0_PG ) && !(cr0 &X86_CR0_PE ))return 1 ; -if (cr0 &X86_CR0_PG ) {#ifdef CONFIG_X86_64 -if (!is_paging (vcpu) && (vcpu->arch.efer &EFER_LME )) { -int cs_db, cs_l; +if ((vcpu->arch.efer &EFER_LME ) && !is_paging (vcpu) && + (cr0 &X86_CR0_PG )) { +int cs_db, cs_l; -if (!is_pae (vcpu)) -return 1 ; - kvm_x86_ops.get_cs_db_l_bits (vcpu, &cs_db, &cs_l); -if (cs_l) -return 1 ; - } else -#endif -if (is_pae (vcpu) && ((cr0 ^ old_cr0) & pdptr_bits) && - !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu))) +if (!is_pae (vcpu)) +return 1 ; + kvm_x86_ops.get_cs_db_l_bits (vcpu, &cs_db, &cs_l); +if (cs_l)
arch/x86/kvm/x86.c int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · line 840
Quoted 10 lines
arch/x86/kvm/x86.c +5
-0return 1; } +#endif +if (!(vcpu->arch.efer &EFER_LME ) && (cr0 &X86_CR0_PG ) && +is_pae (vcpu) && ((cr0 ^ old_cr0) & pdptr_bits) && + !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu))) +return 1 ;if (!(cr0 &X86_CR0_PG ) &&kvm_read_cr4_bits (vcpu,X86_CR4_PCIDE ))return 1 ;
I also investigated this issue (also same thing, OVMF doesn't boot), and after looking at the intel and amd's PRM, this looks like correct solution. I also tested this and it works.
Best regards,
Maxim Levitsky
Quoted 69 lines · Maxim Levitsky
Quoted 67 lines · Sean Christopherson
Don't attempt to load PDPTRs if EFER.LME=1, i.e. if 64-bit mode is enabled. A recent change to reload the PDTPRs when CR0.CD or CR0.NW is toggled botched the EFER.LME handling and sends KVM down the PDTPR path when
is_paging()is true, i.e. when the guest toggles CD/NW in 64-bit mode.Split the CR0 checks for 64-bit vs. 32-bit PAE into separate paths. The 64-bit path is specifically checking state when paging is toggled on, i.e. CR0.PG transititions from 0->1. The PDPTR path now needs to run if the new CR0 state has paging enabled, irrespective of whether paging was already enabled. Trying to shave a few cycles to make the PDPTR path an "else if" case is a mess.
Fixes d42e3fae6faed ("kvm: x86: Read PDPTEs on CR0.CD and CR0.NW changes")Cc Jim Mattson <[email protected]>Cc Oliver Upton <[email protected]>Cc Peter Shier <[email protected]>Signed-off-by Sean Christopherson <[email protected]>---
The other way to fix this, with a much smaller diff stat, is to simply move the !is_page(vcpu) check inside (
vcpu->arch.efer&EFER_LME). But that results in a ridiculous amount of nested conditionals for what is a very straightforward check e.g.if (cr0 & X86_CR0_PG) { if (vcpu->arch.efer & EFER_LME) } if (!is_paging(vcpu)) { ... } } }Since this doesn't need to be backported anywhere, I didn't see any value in having an intermediate step.
arch/x86/kvm/x86.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)arch/x86/kvm/x86.c +6
-12@@ -819,22 +819,22 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)if ((cr0 &X86_CR0_PG ) && !(cr0 &X86_CR0_PE ))return 1 ; -if (cr0 &X86_CR0_PG ) {#ifdef CONFIG_X86_64 -if (!is_paging (vcpu) && (vcpu->arch.efer &EFER_LME )) { -int cs_db, cs_l; +if ((vcpu->arch.efer &EFER_LME ) && !is_paging (vcpu) && + (cr0 &X86_CR0_PG )) { +int cs_db, cs_l; -if (!is_pae (vcpu)) -return 1 ; - kvm_x86_ops.get_cs_db_l_bits (vcpu, &cs_db, &cs_l); -if (cs_l) -return 1 ; - } else -#endif -if (is_pae (vcpu) && ((cr0 ^ old_cr0) & pdptr_bits) && - !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu))) +if (!is_pae (vcpu)) +return 1 ; + kvm_x86_ops.get_cs_db_l_bits (vcpu, &cs_db, &cs_l);
Quoted 8 lines
arch/x86/kvm/x86.c int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · line 837
Quoted 8 lines
arch/x86/kvm/x86.c +6
-0+if (cs_l) return 1; } +#endif +if (!(vcpu->arch.efer &EFER_LME ) && (cr0 &X86_CR0_PG ) && +is_pae (vcpu) && ((cr0 ^ old_cr0) & pdptr_bits) && + !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu))) +return 1 ;
It might be worth commenting on the subtlety of the test below being skipped if the PDPTEs were loaded above. I'm assuming that the PDPTEs shouldn't be loaded if the instruction faults.
Quoted 13 lines
arch/x86/kvm/x86.c int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · line 840
Quoted 2 lines
arch/x86/kvm/x86.c +0
-0if (!(cr0 &X86_CR0_PG ) &&kvm_read_cr4_bits (vcpu,X86_CR4_PCIDE ))return 1 ;I also investigated this issue (also same thing, OVMF doesn't boot), and after looking at the intel and amd's PRM, this looks like correct solution. I also tested this and it works.
Reviewed-by Maxim Levitsky <[email protected]>Best regards,
Maxim Levitsky