KVM: x86: nSVM: Improve PAT virtualization
v2 cover KVM: x86: nSVM: Improve PAT virtualization
Currently, KVM's implementation of nested SVM treats the PAT MSR the same way whether or not nested NPT is enabled: L1 and L2 share a single PAT. However, the APM specifies that when nested NPT is enabled, the host (L1) and the guest (L2) should have independent PATs: hPAT for L1 and gPAT for L2. This patch series implements the architectural specification in KVM.
The existing PAT MSR (vcpu->arch.pat) is used for hPAT, and the vmcb02.save.g_pat field is used for gPAT. With nested NPT enabled, guest accesses to the IA32_PAT MSR are redirected to gPAT, which is stored in vmcb02->save.g_pat. All other accesses, including userspace accesses via KVM_{GET,SET}_MSRS, continue to reference hPAT.
The special handling of userspace accesses ensures save/restore forward compatibility (i.e. resuming a new checkpoint on an older kernel). When an old kernel restores a checkpoint from a new kernel, the gPAT will be lost, and L2 will simply use L1's PAT, which is the behavior of the old kernel anyway.
v1 -> v2:
Adhere to the architectural specification
Drop the preservation of vmcb01->g_pat across virtual SMM Store the gPAT rather than the hPAT in the nested state (save.g_pat) Fix forward compatibility
Handle backward compatibility when MSRs are restored after nested state (setq-default fill-column 75) [Sean]
Or the KVM_STATE_SVM_VALID_GPAT bit into flags [Sean]
Jim Mattson (8):
KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT
KVM: x86: nSVM: Cache g_pat in vmcb_save_area_cached
KVM: x86: nSVM: Add validity check for vmcb12 g_pat
KVM: x86: nSVM: Set vmcb02.g_pat correctly for nested NPT
KVM: x86: nSVM: Save gPAT to vmcb12.g_pat on VMEXIT
KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE
KVM: x86: nSVM: Handle restore of legacy nested state
KVM: selftests: nSVM: Add svm_nested_pat test
arch/x86/include/uapi/asm/kvm.h | 3 + arch/x86/kvm/svm/nested.c | 49 ++- arch/x86/kvm/svm/svm.c | 39 +- arch/x86/kvm/svm/svm.h | 7 + tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/x86/svm_nested_pat_test.c | 357 ++++++++++++++++++ 6 files changed, 442 insertions(+), 14 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86/svm_nested_pat_test.c
base-commit: f62b64b970570c92fe22503b0cdc65be7ce7fc7c
Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH v2 1/8 KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT
When the vCPU is in guest mode with nested NPT enabled, guest accesses to IA32_PAT are redirected to the gPAT register, which is stored in vmcb02->save.g_pat.
Non-guest accesses (e.g. from userspace) to IA32_PAT are always redirected to hPAT, which is stored in vcpu->arch.pat.
This is architected behavior. It also makes it possible to restore a new checkpoint on an old kernel with reasonable semantics. After the restore, gPAT will be lost, and L2 will run on L1's PAT. Note that the old kernel would have always run L2 on L1's PAT.
--- arch/x86/kvm/svm/svm.c | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-)
arch/x86/kvm/svm/svm.c +24 -7
@@ -2846,6 +2846,13 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_AMD64_DE_CFG : msr_info->data = svm->msr_decfg;break ; +case MSR_IA32_CR_PAT : +if (!msr_info->host_initiated &&is_guest_mode (vcpu) && +nested_npt_enabled (svm)) + msr_info->data = svm->vmcb->save.g_pat;/* gPAT */ +else + msr_info->data = vcpu->arch.pat;/* hPAT */ +break ;default :return kvm_get_msr_common (vcpu, msr_info); }
@@ -2929,14 +2936,24 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)break ;case MSR_IA32_CR_PAT : - ret =kvm_set_msr_common (vcpu, msr); -if (ret) -break ; +if (!kvm_pat_valid (data)) +return 1 ; - svm->vmcb01.ptr->save.g_pat = data; -if (is_guest_mode (vcpu)) -nested_vmcb02_compute_g_pat (svm); -vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); +if (!msr->host_initiated &&is_guest_mode (vcpu) && +nested_npt_enabled (svm)) { + svm->vmcb->save.g_pat = data;/* gPAT */ +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + }else { + vcpu->arch.pat = data;/* hPAT */ +if (npt_enabled) { + svm->vmcb01.ptr->save.g_pat = data; +vmcb_mark_dirty (svm->vmcb01.ptr,VMCB_NPT ); +if (is_guest_mode (vcpu)) { + svm->vmcb->save.g_pat = data; +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + } + } + }break ;case MSR_IA32_SPEC_CTRL : if (!msr->host_initiated &&
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 1/8] KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT
Quoted 50 lines · Jim Mattson
When the vCPU is in guest mode with nested NPT enabled, guest accesses to
IA32_PATare redirected to the gPAT register, which is stored invmcb02->save.g_pat.Non-guest accesses (e.g. from userspace) to
IA32_PATare always redirected to hPAT, which is stored invcpu->arch.pat.This is architected behavior. It also makes it possible to restore a new checkpoint on an old kernel with reasonable semantics. After the restore, gPAT will be lost, and L2 will run on L1's PAT. Note that the old kernel would have always run L2 on L1's PAT.
Signed-off-by Jim Mattson <[email protected]>--- arch/x86/kvm/svm/svm.c | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-)arch/x86/kvm/svm/svm.c +9
-7@@ -2846,6 +2846,13 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_AMD64_DE_CFG : msr_info->data = svm->msr_decfg;break ; +case MSR_IA32_CR_PAT : +if (!msr_info->host_initiated &&is_guest_mode (vcpu) && +nested_npt_enabled (svm)) + msr_info->data = svm->vmcb->save.g_pat;/* gPAT */ +else + msr_info->data = vcpu->arch.pat;/* hPAT */ +break ;default :return kvm_get_msr_common (vcpu, msr_info); }@@ -2929,14 +2936,24 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)break ;case MSR_IA32_CR_PAT : - ret =kvm_set_msr_common (vcpu, msr); -if (ret) -break ; +if (!kvm_pat_valid (data)) +return 1 ; - svm->vmcb01.ptr->save.g_pat = data; -if (is_guest_mode (vcpu)) -nested_vmcb02_compute_g_pat (svm); -vmcb_mark_dirty (svm->vmcb,VMCB_NPT );
arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 2951
Quoted 10 lines
arch/x86/kvm/svm/svm.c +10
-0+if (!msr->host_initiated &&is_guest_mode (vcpu) && +nested_npt_enabled (svm)) { + svm->vmcb->save.g_pat = data;/* gPAT */ +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + }else { + vcpu->arch.pat = data;/* hPAT */ +if (npt_enabled) { + svm->vmcb01.ptr->save.g_pat = data; +vmcb_mark_dirty (svm->vmcb01.ptr,VMCB_NPT ); +if (is_guest_mode (vcpu)) {
Oops. That should be "is_guest_mode(vcpu) && !nested_npt_enabled(svm)".
Quoted 11 lines
arch/x86/kvm/svm/svm.c +5
-1+ svm->vmcb->save.g_pat = data; +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + } + } + }break ;case MSR_IA32_SPEC_CTRL : if (!msr->host_initiated && --2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 1/8] KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT
In reply to Jim MattsonQuoted 11 lines · Jim Mattson
When the vCPU is in guest mode with nested NPT enabled, guest accesses to
IA32_PATare redirected to the gPAT register, which is stored invmcb02->save.g_pat.Non-guest accesses (e.g. from userspace) to
IA32_PATare always redirected to hPAT, which is stored invcpu->arch.pat.This is architected behavior. It also makes it possible to restore a new checkpoint on an old kernel with reasonable semantics. After the restore, gPAT will be lost, and L2 will run on L1's PAT. Note that the old kernel would have always run L2 on L1's PAT.
This creates a difference in MSR_IA32_CR_PAT handling with nested SVM and nested VMX, right?
AFAICT, reading MSR_IA32_CR_PAT while an L2 VMX guest is running will read L2's PAT. With this change, the same scenario on SVM will read L1's PAT. We can claim that it was always L1's PAT though, because we've always been running L2 with L1's PAT.
I am just raising this in case it's a problem to have different behavior for SVM and VMX. I understand that we need to do this to be able to save/restore L1's PAT with SVM in guest mode and maintain backward compatibility.
IIUC VMX does not have the same issue because host and guest PAT are both in vmcs12 and are both saved/restored appropriately.
Quoted 25 lines
Signed-off-by Jim Mattson <[email protected]>--- arch/x86/kvm/svm/svm.c | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-)arch/x86/kvm/svm/svm.c +7
-0@@ -2846,6 +2846,13 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_AMD64_DE_CFG : msr_info->data = svm->msr_decfg;break ; +case MSR_IA32_CR_PAT : +if (!msr_info->host_initiated &&is_guest_mode (vcpu) && +nested_npt_enabled (svm)) + msr_info->data = svm->vmcb->save.g_pat;/* gPAT */ +else + msr_info->data = vcpu->arch.pat;/* hPAT */ +break ;default :return kvm_get_msr_common (vcpu, msr_info); }
arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 2936
Quoted 10 lines
arch/x86/kvm/svm/svm.c +2
-4@@ -2929,14 +2936,24 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)break ;case MSR_IA32_CR_PAT : - ret =kvm_set_msr_common (vcpu, msr); -if (ret) -break ; +if (!kvm_pat_valid (data)) +return 1 ; - svm->vmcb01.ptr->save.g_pat = data;
This is a bug fix, L2 is now able to alter L1's PAT, right?
arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 2939
Quoted 3 lines
arch/x86/kvm/svm/svm.c +0
-3-if (is_guest_mode (vcpu)) -nested_vmcb02_compute_g_pat (svm); -vmcb_mark_dirty (svm->vmcb,VMCB_NPT );
This looks like another bug fix, it seems like we'll update vmcb01 but clear the clean bit in vmcb02 if we're in guest mode.
Probably worth calling these out (and CC:stable, Fixes:.., etc)?
We probably need a comment here explaining the gPAT vs hPAT case, I don't think it's super obvious why we only redirect L2's own accesses to its PAT but not userspace's.
arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 2947
Quoted 6 lines
arch/x86/kvm/svm/svm.c +6
-0+if (!msr->host_initiated &&is_guest_mode (vcpu) && +nested_npt_enabled (svm)) { + svm->vmcb->save.g_pat = data;/* gPAT */ +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + }else { + vcpu->arch.pat = data;/* hPAT */
Should we call kvm_set_msr_common() here instead of setting vcpu->arch.pat? The kvm_pat_valid() call would be redundant but that should be fine. My main concern is if kvm_set_msr_common() gains more logic for MSR_IA32_CR_PAT that isn't reflected here. Probably unlikely tho..
arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 2951
Quoted 4 lines
arch/x86/kvm/svm/svm.c +4
-0+if (npt_enabled) { + svm->vmcb01.ptr->save.g_pat = data; +vmcb_mark_dirty (svm->vmcb01.ptr,VMCB_NPT ); +if (is_guest_mode (vcpu)) {
IIUC (with the fix you mentioned) this is because L1 and L2 share the PAT if nested NPT is disabled, and if we're already in guest mode then we also need to update vmcb02 (as it was already created based on vmcb01 with the old PAT). Probably worth a comment.
Quoted 11 lines
arch/x86/kvm/svm/svm.c +5
-0+ svm->vmcb->save.g_pat = data; +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + } + } + }break ;case MSR_IA32_SPEC_CTRL : if (!msr->host_initiated &&Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 1/8] KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT
Quoted 9 lines · Yosry Ahmed
Quoted 7 lines · Jim Mattson
When the vCPU is in guest mode with nested NPT enabled, guest accesses to
IA32_PATare redirected to the gPAT register, which is stored invmcb02->save.g_pat.Non-guest accesses (e.g. from userspace) to
IA32_PATare always redirected to hPAT, which is stored invcpu->arch.pat.
Quoted 7 lines
Quoted 4 lines
This is architected behavior. It also makes it possible to restore a new checkpoint on an old kernel with reasonable semantics. After the restore, gPAT will be lost, and L2 will run on L1's PAT. Note that the old kernel would have always run L2 on L1's PAT.
This creates a difference in
MSR_IA32_CR_PAThandling with nested SVM and nested VMX, right?
Correct.
Quoted 5 lines
AFAICT, reading
MSR_IA32_CR_PATwhile an L2 VMX guest is running will read L2's PAT. With this change, the same scenario on SVM will read L1's PAT. We can claim that it was always L1's PAT though, because we've always been running L2 with L1's PAT.
Quoted 7 lines
I am just raising this in case it's a problem to have different behavior for SVM and VMX. I understand that we need to do this to be able to save/restore L1's PAT with SVM in guest mode and maintain backward compatibility.
IIUC VMX does not have the same issue because host and guest PAT are both in vmcs12 and are both saved/restored appropriately.
Correct.
Quoted 34 lines
Quoted 34 lines
Signed-off-by Jim Mattson <[email protected]>--- arch/x86/kvm/svm/svm.c | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-)arch/x86/kvm/svm/svm.c +9
-3@@ -2846,6 +2846,13 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_AMD64_DE_CFG : msr_info->data = svm->msr_decfg;break ; +case MSR_IA32_CR_PAT : +if (!msr_info->host_initiated &&is_guest_mode (vcpu) && +nested_npt_enabled (svm)) + msr_info->data = svm->vmcb->save.g_pat;/* gPAT */ +else + msr_info->data = vcpu->arch.pat;/* hPAT */ +break ;default :return kvm_get_msr_common (vcpu, msr_info); }@@ -2929,14 +2936,24 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)break ;case MSR_IA32_CR_PAT : - ret =kvm_set_msr_common (vcpu, msr); -if (ret) -break ; +if (!kvm_pat_valid (data)) +return 1 ;
Quoted 3 lines
arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 2936
Quoted 1 line
arch/x86/kvm/svm/svm.c +0
-1- svm->vmcb01.ptr->save.g_pat = data;This is a bug fix, L2 is now able to alter L1's PAT, right?
Yes, L1 and L2 share a PAT today. The whole series is fixing that mistake.
Quoted 6 lines
arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 2939
Quoted 3 lines
arch/x86/kvm/svm/svm.c +0
-3-if (is_guest_mode (vcpu)) -nested_vmcb02_compute_g_pat (svm); -vmcb_mark_dirty (svm->vmcb,VMCB_NPT );This looks like another bug fix, it seems like we'll update vmcb01 but clear the clean bit in vmcb02 if we're in guest mode.
I will split this out into a separate patch, since this is tangential to the rest of the series.
Quoted 5 lines
Probably worth calling these out (and CC:stable, Fixes:.., etc)?
We probably need a comment here explaining the gPAT vs hPAT case, I don't think it's super obvious why we only redirect L2's own accesses to its PAT but not userspace's.
Point taken.
Quoted 7 lines
arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 2947
Quoted 6 lines
arch/x86/kvm/svm/svm.c +6
-0+if (!msr->host_initiated &&is_guest_mode (vcpu) && +nested_npt_enabled (svm)) { + svm->vmcb->save.g_pat = data;/* gPAT */ +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + }else { + vcpu->arch.pat = data;/* hPAT */
Quoted 5 lines
Should we call
kvm_set_msr_common()here instead of settingvcpu->arch.pat? Thekvm_pat_valid()call would be redundant but that should be fine. My main concern is ifkvm_set_msr_common()gains more logic forMSR_IA32_CR_PATthat isn't reflected here. Probably unlikely tho..
There are already three open-coded assignments to vcpu->arch.pat on the VMX side. Perhaps this should be avoided, but it's tangential to this series.
Quoted 9 lines
arch/x86/kvm/svm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 2951
Quoted 4 lines
arch/x86/kvm/svm/svm.c +4
-0+if (npt_enabled) { + svm->vmcb01.ptr->save.g_pat = data; +vmcb_mark_dirty (svm->vmcb01.ptr,VMCB_NPT ); +if (is_guest_mode (vcpu)) {IIUC (with the fix you mentioned) this is because L1 and L2 share the PAT if nested NPT is disabled, and if we're already in guest mode then we also need to update vmcb02 (as it was already created based on vmcb01 with the old PAT). Probably worth a comment.
Noted.
Quoted 11 lines
Quoted 11 lines
arch/x86/kvm/svm/svm.c +5
-1+ svm->vmcb->save.g_pat = data; +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + } + } + }break ;case MSR_IA32_SPEC_CTRL : if (!msr->host_initiated && --2.52.0.457.g6b5491de43-goog
PATCH v2 2/8 KVM: x86: nSVM: Cache g_pat in vmcb_save_area_cached
In reply to Jim MattsonTo avoid TOCTTOU issues, all fields in the vmcb12 save area that are subject to validation must be copied to svm->nested.save prior to validation, since vmcb12 is writable by the guest. Add g_pat to this set in preparation for validting it.
--- arch/x86/kvm/svm/nested.c | 2 ++ arch/x86/kvm/svm/svm.h | 1 + 2 files changed, 3 insertions(+)
arch/x86/kvm/svm/nested.c +2 -0
@@ -506,6 +506,8 @@ static void __nested_copy_vmcb_save_to_cache(struct vmcb_save_area_cached *to, to->dr6 = from->dr6; to->dr7 = from->dr7; + + to->g_pat = from->g_pat; } void nested_copy_vmcb_save_to_cache(struct vcpu_svm *svm,
arch/x86/kvm/svm/svm.h +1 -0
@@ -145,6 +145,7 @@ struct vmcb_save_area_cached {u64 cr0;u64 dr7;u64 dr6; +u64 g_pat; };struct vmcb_ctrl_area_cached {
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 2/8] KVM: x86: nSVM: Cache g_pat in vmcb_save_area_cached
Quoted 6 lines · Jim Mattson
To avoid TOCTTOU issues, all fields in the vmcb12 save area that are subject to validation must be copied to
svm->nested.saveprior to validation, since vmcb12 is writable by the guest. Addg_patto this set in preparation for validting it.Fixes 3d6368ef580a ("KVM: SVM: Add VMRUN handler")
g_pat is not currently used from VMCB12, so this patch isn't technically fixing an issue, right? I suspect this only applies to patch 3.
Anyway, I think it's probably best to squash this into patch 3. Also maybe CC stable?
Quoted 1 line
Signed-off-by Jim Mattson <[email protected]>
If you decide to keep this as an individual patch, feel free to add:
Quoted 33 lines
--- arch/x86/kvm/svm/nested.c | 2 ++ arch/x86/kvm/svm/svm.h | 1 + 2 files changed, 3 insertions(+)arch/x86/kvm/svm/nested.c +2
-0@@ -506,6 +506,8 @@ static void __nested_copy_vmcb_save_to_cache(struct vmcb_save_area_cached *to, to->dr6 = from->dr6; to->dr7 = from->dr7; + + to->g_pat = from->g_pat; } void nested_copy_vmcb_save_to_cache(struct vcpu_svm *svm,arch/x86/kvm/svm/svm.h +1
-0@@ -145,6 +145,7 @@ struct vmcb_save_area_cached {u64 cr0;u64 dr7;u64 dr6; +u64 g_pat; };struct vmcb_ctrl_area_cached {Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 2/8] KVM: x86: nSVM: Cache g_pat in vmcb_save_area_cached
Quoted 11 lines · Yosry Ahmed
Quoted 6 lines · Jim Mattson
To avoid TOCTTOU issues, all fields in the vmcb12 save area that are subject to validation must be copied to
svm->nested.saveprior to validation, since vmcb12 is writable by the guest. Addg_patto this set in preparation for validting it.Fixes 3d6368ef580a ("KVM: SVM: Add VMRUN handler")
g_patis not currently used from VMCB12, so this patch isn't technically fixing an issue, right? I suspect this only applies to patch 3.
The fact that VMCB12.g_pat isn't used *is* the issue. This patch is part of the fix. To make the fix easier to review, it's broken into several pieces.
Quoted 2 lines
Anyway, I think it's probably best to squash this into patch 3. Also maybe CC stable?
I will squash it in the next version.
Quoted 39 lines
Quoted 1 line
Signed-off-by Jim Mattson <[email protected]>If you decide to keep this as an individual patch, feel free to add:
Reviewed-by Yosry Ahmed <[email protected]>Quoted 33 lines
--- arch/x86/kvm/svm/nested.c | 2 ++ arch/x86/kvm/svm/svm.h | 1 + 2 files changed, 3 insertions(+)arch/x86/kvm/svm/nested.c +2
-0@@ -506,6 +506,8 @@ static void __nested_copy_vmcb_save_to_cache(struct vmcb_save_area_cached *to, to->dr6 = from->dr6; to->dr7 = from->dr7; + + to->g_pat = from->g_pat; } void nested_copy_vmcb_save_to_cache(struct vcpu_svm *svm,arch/x86/kvm/svm/svm.h +1
-1@@ -145,6 +145,7 @@ struct vmcb_save_area_cached {u64 cr0;u64 dr7;u64 dr6; +u64 g_pat; };struct vmcb_ctrl_area_cached { --2.52.0.457.g6b5491de43-goog
PATCH v2 3/8 KVM: x86: nSVM: Add validity check for vmcb12 g_pat
In reply to Jim MattsonAdd a validity check for g_pat, so that when nested paging is enabled for vmcb12, an invalid g_pat causes an immediate VMEXIT with exit code VMEXIT_INVALID, as specified in the APM, volume 2: "Nested Paging and VMRUN/VMEXIT."
Update the signature of __nested_vmcb_check_save() to include a pointer to a struct vmcb_ctrl_area_cached, since the g_pat validity check depend on the nested paging control bit.
--- arch/x86/kvm/svm/nested.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/nested.c +9 -3
@@ -369,7 +369,8 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu,/* Common checks that apply to both L1 and L2 state. */ static bool __nested_vmcb_check_save (struct kvm_vcpu *vcpu, -struct vmcb_save_area_cached *save) +struct vmcb_save_area_cached *save, +struct vmcb_ctrl_area_cached *control) {if (CC (!(save->efer &EFER_SVME )))return false;
@@ -400,6 +401,10 @@ static bool __nested_vmcb_check_save(struct kvm_vcpu *vcpu,if (CC (!kvm_valid_efer (vcpu, save->efer)))return false; +if (CC ((control->nested_ctl &SVM_NESTED_CTL_NP_ENABLE ) && + npt_enabled && !kvm_pat_valid (save->g_pat))) +return false; +return true; }
@@ -407,8 +412,9 @@ static bool nested_vmcb_check_save(struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);struct vmcb_save_area_cached *save = &svm->nested.save; +struct vmcb_ctrl_area_cached *ctl = &svm->nested.ctl; -return __nested_vmcb_check_save (vcpu, save); +return __nested_vmcb_check_save (vcpu, save, ctl); }static bool nested_vmcb_check_controls (struct kvm_vcpu *vcpu)
@@ -1892,7 +1898,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,if (!(save->cr0 &X86_CR0_PG ) || !(save->cr0 &X86_CR0_PE ) || (save->rflags &X86_EFLAGS_VM ) || - !__nested_vmcb_check_save (vcpu, &save_cached)) + !__nested_vmcb_check_save (vcpu, &save_cached, &ctl_cached)) goto out_free;
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 3/8] KVM: x86: nSVM: Add validity check for vmcb12 g_pat
Quoted 30 lines · Jim Mattson
Add a validity check for
g_pat, so that when nested paging is enabled for vmcb12, an invalidg_patcauses an immediate VMEXIT with exit codeVMEXIT_INVALID, as specified in the APM, volume 2: "Nested Paging and VMRUN/VMEXIT."Update the signature of
__nested_vmcb_check_save()to include a pointer to a structvmcb_ctrl_area_cached, since theg_patvalidity check depend on the nested paging control bit.Fixes 3d6368ef580a ("KVM: SVM: Add VMRUN handler")Signed-off-by Jim Mattson <[email protected]>--- arch/x86/kvm/svm/nested.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)arch/x86/kvm/svm/nested.c +2
-1@@ -369,7 +369,8 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu,/* Common checks that apply to both L1 and L2 state. */ static bool __nested_vmcb_check_save (struct kvm_vcpu *vcpu, -struct vmcb_save_area_cached *save) +struct vmcb_save_area_cached *save, +struct vmcb_ctrl_area_cached *control) {if (CC (!(save->efer &EFER_SVME )))return false;
arch/x86/kvm/svm/nested.c static bool __nested_vmcb_check_save(struct kvm_vcpu *vcpu, · line 405
Quoted 5 lines
arch/x86/kvm/svm/nested.c +2
-0@@ -400,6 +401,10 @@ static bool __nested_vmcb_check_save(struct kvm_vcpu *vcpu,if (CC (!kvm_valid_efer (vcpu, save->efer)))return false; +if (CC ((control->nested_ctl &SVM_NESTED_CTL_NP_ENABLE ) && + npt_enabled && !kvm_pat_valid (save->g_pat)))
If this lands after "KVM: nSVM: Drop the non-architectural consistency check for NP_ENABLE" [1], the npt_enabled check can be dropped, as SVM_NESTED_CTL_NP_ENABLE will be cleared if the guest cannot use NPTs.
Quoted 17 lines
arch/x86/kvm/svm/nested.c +4
-1+return false; +return true; }@@ -407,8 +412,9 @@ static bool nested_vmcb_check_save(struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);struct vmcb_save_area_cached *save = &svm->nested.save; +struct vmcb_ctrl_area_cached *ctl = &svm->nested.ctl; -return __nested_vmcb_check_save (vcpu, save); +return __nested_vmcb_check_save (vcpu, save, ctl); }static bool nested_vmcb_check_controls (struct kvm_vcpu *vcpu)
arch/x86/kvm/svm/nested.c static int svm_set_nested_state(struct kvm_vcpu *vcpu, · line 1901
Quoted 5 lines
arch/x86/kvm/svm/nested.c +1
-1@@ -1892,7 +1898,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,if (!(save->cr0 &X86_CR0_PG ) || !(save->cr0 &X86_CR0_PE ) || (save->rflags &X86_EFLAGS_VM ) || - !__nested_vmcb_check_save (vcpu, &save_cached)) + !__nested_vmcb_check_save (vcpu, &save_cached, &ctl_cached))
save_cached here is from vmcb01, but ctl_cached is from vmcb12, right? So we're conditioning the PAT validity check on L1's gPAT on NPT being enabled for L2 IIUC.
Perhaps we should pass 'bool guest_npt' instead of the cached control area? Then we can just pass npt_enabled here IIUC.
Quoted 6 lines
arch/x86/kvm/svm/nested.c +0
-0goto out_free;Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 3/8] KVM: x86: nSVM: Add validity check for vmcb12 g_pat
Quoted 35 lines · Yosry Ahmed
Quoted 33 lines · Jim Mattson
Add a validity check for
g_pat, so that when nested paging is enabled for vmcb12, an invalidg_patcauses an immediate VMEXIT with exit codeVMEXIT_INVALID, as specified in the APM, volume 2: "Nested Paging and VMRUN/VMEXIT."Update the signature of
__nested_vmcb_check_save()to include a pointer to a structvmcb_ctrl_area_cached, since theg_patvalidity check depend on the nested paging control bit.Fixes 3d6368ef580a ("KVM: SVM: Add VMRUN handler")Signed-off-by Jim Mattson <[email protected]>--- arch/x86/kvm/svm/nested.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)arch/x86/kvm/svm/nested.c +2
-1@@ -369,7 +369,8 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu,/* Common checks that apply to both L1 and L2 state. */ static bool __nested_vmcb_check_save (struct kvm_vcpu *vcpu, -struct vmcb_save_area_cached *save) +struct vmcb_save_area_cached *save, +struct vmcb_ctrl_area_cached *control) {if (CC (!(save->efer &EFER_SVME )))return false;@@ -400,6 +401,10 @@ static bool __nested_vmcb_check_save(struct kvm_vcpu *vcpu,if (CC (!kvm_valid_efer (vcpu, save->efer)))return false;
Quoted 8 lines
arch/x86/kvm/svm/nested.c static bool __nested_vmcb_check_save(struct kvm_vcpu *vcpu, · line 405
Quoted 2 lines
arch/x86/kvm/svm/nested.c +2
-0+if (CC ((control->nested_ctl &SVM_NESTED_CTL_NP_ENABLE ) && + npt_enabled && !kvm_pat_valid (save->g_pat)))If this lands after "KVM: nSVM: Drop the non-architectural consistency check for
NP_ENABLE" [1], thenpt_enabledcheck can be dropped, asSVM_NESTED_CTL_NP_ENABLEwill be cleared if the guest cannot use NPTs.
I don't see it today, but I will probably forget to keep checking. :)
Quoted 27 lines
Quoted 17 lines
arch/x86/kvm/svm/nested.c +4
-1+return false; +return true; }@@ -407,8 +412,9 @@ static bool nested_vmcb_check_save(struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);struct vmcb_save_area_cached *save = &svm->nested.save; +struct vmcb_ctrl_area_cached *ctl = &svm->nested.ctl; -return __nested_vmcb_check_save (vcpu, save); +return __nested_vmcb_check_save (vcpu, save, ctl); }static bool nested_vmcb_check_controls (struct kvm_vcpu *vcpu)arch/x86/kvm/svm/nested.c static int svm_set_nested_state(struct kvm_vcpu *vcpu, · line 1901
Quoted 5 lines
arch/x86/kvm/svm/nested.c +1
-1@@ -1892,7 +1898,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,if (!(save->cr0 &X86_CR0_PG ) || !(save->cr0 &X86_CR0_PE ) || (save->rflags &X86_EFLAGS_VM ) || - !__nested_vmcb_check_save (vcpu, &save_cached)) + !__nested_vmcb_check_save (vcpu, &save_cached, &ctl_cached))
save_cachedhere is from vmcb01, butctl_cachedis from vmcb12, right? So we're conditioning the PAT validity check on L1's gPAT on NPT being enabled for L2 IIUC.
Quoted 2 lines
Perhaps we should pass 'bool
guest_npt' instead of the cached control area? Then we can just passnpt_enabledhere IIUC.
I'm going to cache it on its own to avoid confusion.
Quoted 6 lines
Quoted 6 lines
arch/x86/kvm/svm/nested.c +0
-1goto out_free; --2.52.0.457.g6b5491de43-goog
PATCH v2 4/8 KVM: x86: nSVM: Set vmcb02.g_pat correctly for nested NPT
In reply to Jim MattsonWhen nested NPT is enabled in vmcb12, copy the (cached and validated) vmcb12 g_pat field to the guest PAT register. Under KVM, the guest PAT register lives in the vmcb02 g_pat field.
When NPT is enabled, but nested NPT is disabled, copy L1's IA32_PAT MSR to the vmcb02 g_pat field, since L2 shares the IA32_PAT MSR with L1,
When NPT is disabled, the vmcb02 g_pat field is ignored by hardware.
--- arch/x86/kvm/svm/nested.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/nested.c +13 -3
@@ -656,9 +656,6 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm, struct vmcb *vmcb12struct vmcb *vmcb02 = svm->nested.vmcb02.ptr;struct kvm_vcpu *vcpu = &svm->vcpu; -nested_vmcb02_compute_g_pat (svm); -vmcb_mark_dirty (vmcb02,VMCB_NPT ); -/* Load the nested guest state */ if (svm->nested.vmcb12_gpa != svm->nested.last_vmcb12_gpa) { new_vmcb12 = true;
@@ -666,6 +663,19 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm, struct vmcb *vmcb12 svm->nested.force_msr_bitmap_recalc = true; } +if (npt_enabled) { +if (nested_npt_enabled (svm)) { +if (unlikely (new_vmcb12 || +vmcb_is_dirty (vmcb12,VMCB_NPT ))) { + vmcb02->save.g_pat = svm->nested.save.g_pat; +vmcb_mark_dirty (vmcb02,VMCB_NPT ); + } + }else { + vmcb02->save.g_pat = vcpu->arch.pat; +vmcb_mark_dirty (vmcb02,VMCB_NPT ); + } + } +if (unlikely (new_vmcb12 ||vmcb_is_dirty (vmcb12,VMCB_SEG ))) { vmcb02->save.es = vmcb12->save.es; vmcb02->save.cs = vmcb12->save.cs;
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 4/8] KVM: x86: nSVM: Set vmcb02.g_pat correctly for nested NPT
Quoted 20 lines · Jim Mattson
When nested NPT is enabled in vmcb12, copy the (cached and validated) vmcb12
g_patfield to the guest PAT register. Under KVM, the guest PAT register lives in the vmcb02g_patfield.When NPT is enabled, but nested NPT is disabled, copy L1's
IA32_PATMSR to the vmcb02g_patfield, since L2 shares theIA32_PATMSR with L1,When NPT is disabled, the vmcb02
g_patfield is ignored by hardware.Fixes 15038e147247 ("KVM: SVM: obey guest PAT")Signed-off-by Jim Mattson <[email protected]>--- arch/x86/kvm/svm/nested.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)arch/x86/kvm/svm/nested.c +0
-0
arch/x86/kvm/svm/nested.c static void nested_vmcb02_prepare_save(struct vcpu_svm *svm, struct vmcb *vmcb12 · line 659
Quoted 4 lines
arch/x86/kvm/svm/nested.c +0
-1@@ -656,9 +656,6 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm, struct vmcb *vmcb12struct vmcb *vmcb02 = svm->nested.vmcb02.ptr;struct kvm_vcpu *vcpu = &svm->vcpu; -nested_vmcb02_compute_g_pat (svm);
This is last use of the function, right? Should we drop it now?
Quoted 28 lines
arch/x86/kvm/svm/nested.c +13
-2-vmcb_mark_dirty (vmcb02,VMCB_NPT ); -/* Load the nested guest state */ if (svm->nested.vmcb12_gpa != svm->nested.last_vmcb12_gpa) { new_vmcb12 = true;@@ -666,6 +663,19 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm, struct vmcb *vmcb12 svm->nested.force_msr_bitmap_recalc = true; } +if (npt_enabled) { +if (nested_npt_enabled (svm)) { +if (unlikely (new_vmcb12 || +vmcb_is_dirty (vmcb12,VMCB_NPT ))) { + vmcb02->save.g_pat = svm->nested.save.g_pat; +vmcb_mark_dirty (vmcb02,VMCB_NPT ); + } + }else { + vmcb02->save.g_pat = vcpu->arch.pat; +vmcb_mark_dirty (vmcb02,VMCB_NPT ); + } + } +if (unlikely (new_vmcb12 ||vmcb_is_dirty (vmcb12,VMCB_SEG ))) { vmcb02->save.es = vmcb12->save.es; vmcb02->save.cs = vmcb12->save.cs;Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH v2 5/8 KVM: x86: nSVM: Save gPAT to vmcb12.g_pat on VMEXIT
In reply to Jim MattsonAccording to the APM volume 3 pseudo-code for "VMRUN," when nested paging is enabled in the vmcb, the guest PAT register (gPAT) is saved to the vmcb on emulated VMEXIT.
Under KVM, the guest PAT register lives in the vmcb02 g_pat field. Save this value to the vmcb12 g_pat field on emulated VMEXIT.
--- arch/x86/kvm/svm/nested.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kvm/svm/nested.c +3 -0
@@ -1189,6 +1189,9 @@ int nested_svm_vmexit(struct vcpu_svm *svm) vmcb12->save.dr6 = svm->vcpu.arch.dr6; vmcb12->save.cpl = vmcb02->save.cpl; +if (nested_npt_enabled (svm)) + vmcb12->save.g_pat = svm->vmcb->save.g_pat; +if (guest_cpu_cap_has (vcpu,X86_FEATURE_SHSTK )) { vmcb12->save.s_cet = vmcb02->save.s_cet; vmcb12->save.isst_addr = vmcb02->save.isst_addr;
Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH v2 6/8 KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE
In reply to Jim MattsonAdd a 'flags' field to the SVM nested state header, and use bit 0 of the flags to indicate that gPAT is stored in the nested state.
If in guest mode with NPT enabled, store the current vmcb->save.g_pat value into the vmcb save area of the nested state, and set the flag.
Note that most of the vmcb save area in the nested state is populated with dead (and potentially already clobbered) vmcb01 state. A few fields hold L1 state to be restored at VMEXIT. Previously, the g_pat field was in the former category.
Also note that struct kvm_svm_nested_state_hdr is included in a union padded to 120 bytes, so there is room to add the flags field without changing any offsets.
--- arch/x86/include/uapi/asm/kvm.h | 3 +++ arch/x86/kvm/svm/nested.c | 13 ++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)
arch/x86/include/uapi/asm/kvm.h +3 -0
@@ -495,6 +495,8 @@ struct kvm_sync_regs {#define KVM_STATE_VMX_PREEMPTION_TIMER_DEADLINE 0x00000001 +#define KVM_STATE_SVM_VALID_GPAT BIT(0) +/* vendor-independent attributes for system fd (group 0) */ #define KVM_X86_GRP_SYSTEM 0# define KVM_X86_XCOMP_GUEST_SUPP 0
@@ -530,6 +532,7 @@ struct kvm_svm_nested_state_data {struct kvm_svm_nested_state_hdr {__u64 vmcb_pa; +__u32 flags; };/* for KVM_CAP_NESTED_STATE */
arch/x86/kvm/svm/nested.c +12 -1
@@ -1789,6 +1789,8 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,/* First fill in the header and copy it out. */ if (is_guest_mode (vcpu)) { kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa; +if (nested_npt_enabled (svm)) + kvm_state.hdr.svm.flags |=KVM_STATE_SVM_VALID_GPAT ; kvm_state.size +=KVM_STATE_NESTED_SVM_VMCB_SIZE ; kvm_state.flags |=KVM_STATE_NESTED_GUEST_MODE ;
@@ -1823,6 +1825,11 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,if (r)return -EFAULT ; +/* +* vmcb01->save.g_pat is dead now, so it is safe to overwrite it with +* vmcb02->save.g_pat, whether or not nested NPT is enabled. +*/ + svm->vmcb01.ptr->save.g_pat = svm->vmcb->save.g_pat;if (copy_to_user (&user_vmcb->save, &svm->vmcb01.ptr->save,sizeof (user_vmcb->save)))return -EFAULT ;
@@ -1904,7 +1911,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, goto out_free;/* -* Validate host state saved from before VMRUN (see +* Validate host state saved from before VMRUN and gPAT (see * nested_svm_check_permissions). */ __nested_copy_vmcb_save_to_cache (&save_cached, save);
@@ -1951,6 +1958,10 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,if (ret) goto out_free; +if (is_guest_mode (vcpu) &&nested_npt_enabled (svm) && + (kvm_state.hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT )) + svm->vmcb->save.g_pat = save_cached.g_pat; + svm->nested.force_msr_bitmap_recalc = true;kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu);
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 6/8] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE
Quoted 57 lines · Jim Mattson
Add a 'flags' field to the SVM nested state header, and use bit 0 of the flags to indicate that gPAT is stored in the nested state.
If in guest mode with NPT enabled, store the current
vmcb->save.g_patvalue into the vmcb save area of the nested state, and set the flag.Note that most of the vmcb save area in the nested state is populated with dead (and potentially already clobbered) vmcb01 state. A few fields hold L1 state to be restored at VMEXIT. Previously, the
g_patfield was in the former category.Also note that struct
kvm_svm_nested_state_hdris included in a union padded to 120 bytes, so there is room to add the flags field without changing any offsets.Signed-off-by Jim Mattson <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 3 +++ arch/x86/kvm/svm/nested.c | 13 ++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)arch/x86/include/uapi/asm/kvm.h +3
-0@@ -495,6 +495,8 @@ struct kvm_sync_regs {#define KVM_STATE_VMX_PREEMPTION_TIMER_DEADLINE 0x00000001 +#define KVM_STATE_SVM_VALID_GPAT BIT(0) +/* vendor-independent attributes for system fd (group 0) */ #define KVM_X86_GRP_SYSTEM 0# define KVM_X86_XCOMP_GUEST_SUPP 0@@ -530,6 +532,7 @@ struct kvm_svm_nested_state_data {struct kvm_svm_nested_state_hdr {__u64 vmcb_pa; +__u32 flags; };/* for KVM_CAP_NESTED_STATE */ arch/x86/kvm/svm/nested.c +2
-0@@ -1789,6 +1789,8 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,/* First fill in the header and copy it out. */ if (is_guest_mode (vcpu)) { kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa; +if (nested_npt_enabled (svm)) + kvm_state.hdr.svm.flags |=KVM_STATE_SVM_VALID_GPAT ; kvm_state.size +=KVM_STATE_NESTED_SVM_VMCB_SIZE ; kvm_state.flags |=KVM_STATE_NESTED_GUEST_MODE ;
arch/x86/kvm/svm/nested.c static int svm_get_nested_state(struct kvm_vcpu *vcpu, · line 1832
Quoted 8 lines
arch/x86/kvm/svm/nested.c +5
-0@@ -1823,6 +1825,11 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,if (r)return -EFAULT ; +/* +* vmcb01->save.g_pat is dead now, so it is safe to overwrite it with +* vmcb02->save.g_pat, whether or not nested NPT is enabled. +*/ + svm->vmcb01.ptr->save.g_pat = svm->vmcb->save.g_pat;
Is this too disgusting? Should I extend the payload by 8 bytes instead? It seems like such a waste, since most of the save area is dead/unused. Maybe I could define a new sparse save state structure, with the ~200 bytes that are currently used, surrounded by padding for the other 500+ bytes. Then, I could just grab 8 bytes of the padding, and it wouldn't seem quite as hacky .
Quoted 26 lines
arch/x86/kvm/svm/nested.c +5
-2if (copy_to_user (&user_vmcb->save, &svm->vmcb01.ptr->save,sizeof (user_vmcb->save)))return -EFAULT ;@@ -1904,7 +1911,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, goto out_free;/* -* Validate host state saved from before VMRUN (see +* Validate host state saved from before VMRUN and gPAT (see * nested_svm_check_permissions). */ __nested_copy_vmcb_save_to_cache (&save_cached, save);@@ -1951,6 +1958,10 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,if (ret) goto out_free; +if (is_guest_mode (vcpu) &&nested_npt_enabled (svm) && + (kvm_state.hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT )) + svm->vmcb->save.g_pat = save_cached.g_pat; + svm->nested.force_msr_bitmap_recalc = true;kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); --2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 6/8] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE
Quoted 61 lines · Jim Mattson
Quoted 60 lines · Jim Mattson
Add a 'flags' field to the SVM nested state header, and use bit 0 of the flags to indicate that gPAT is stored in the nested state.
If in guest mode with NPT enabled, store the current
vmcb->save.g_patvalue into the vmcb save area of the nested state, and set the flag.Note that most of the vmcb save area in the nested state is populated with dead (and potentially already clobbered) vmcb01 state. A few fields hold L1 state to be restored at VMEXIT. Previously, the
g_patfield was in the former category.Also note that struct
kvm_svm_nested_state_hdris included in a union padded to 120 bytes, so there is room to add the flags field without changing any offsets.Signed-off-by Jim Mattson <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 3 +++ arch/x86/kvm/svm/nested.c | 13 ++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)arch/x86/include/uapi/asm/kvm.h +3
-0@@ -495,6 +495,8 @@ struct kvm_sync_regs {#define KVM_STATE_VMX_PREEMPTION_TIMER_DEADLINE 0x00000001 +#define KVM_STATE_SVM_VALID_GPAT BIT(0) +/* vendor-independent attributes for system fd (group 0) */ #define KVM_X86_GRP_SYSTEM 0# define KVM_X86_XCOMP_GUEST_SUPP 0@@ -530,6 +532,7 @@ struct kvm_svm_nested_state_data {struct kvm_svm_nested_state_hdr {__u64 vmcb_pa; +__u32 flags; };/* for KVM_CAP_NESTED_STATE */ arch/x86/kvm/svm/nested.c +2
-0@@ -1789,6 +1789,8 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,/* First fill in the header and copy it out. */ if (is_guest_mode (vcpu)) { kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa; +if (nested_npt_enabled (svm)) + kvm_state.hdr.svm.flags |=KVM_STATE_SVM_VALID_GPAT ; kvm_state.size +=KVM_STATE_NESTED_SVM_VMCB_SIZE ; kvm_state.flags |=KVM_STATE_NESTED_GUEST_MODE ;@@ -1823,6 +1825,11 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,if (r)return -EFAULT ;
Quoted 12 lines
arch/x86/kvm/svm/nested.c static int svm_get_nested_state(struct kvm_vcpu *vcpu, · line 1832
Quoted 5 lines
arch/x86/kvm/svm/nested.c +5
-0+/* +* vmcb01->save.g_pat is dead now, so it is safe to overwrite it with +* vmcb02->save.g_pat, whether or not nested NPT is enabled. +*/ + svm->vmcb01.ptr->save.g_pat = svm->vmcb->save.g_pat;Is this too disgusting? Should I extend the payload by 8 bytes instead? It seems like such a waste, since most of the save area is dead/unused. Maybe I could define a new sparse save state structure, with the ~200 bytes that are currently used, surrounded by padding for the other 500+ bytes. Then, I could just grab 8 bytes of the padding, and it wouldn't seem quite as hacky .
I think this would be cleaner than reusing the vmcb01 field.
One question though, if we decide to start doing save/restore for one of the save area fields in vmcb01 in the currently unused 500+ bytes (i.e. the padding), would this be a problem? IIUC the 8 bytes we'll use for gPAT will overlap with an existing unused field.
Quoted 27 lines
Quoted 27 lines
arch/x86/kvm/svm/nested.c +5
-2if (copy_to_user (&user_vmcb->save, &svm->vmcb01.ptr->save,sizeof (user_vmcb->save)))return -EFAULT ;@@ -1904,7 +1911,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, goto out_free;/* -* Validate host state saved from before VMRUN (see +* Validate host state saved from before VMRUN and gPAT (see * nested_svm_check_permissions). */ __nested_copy_vmcb_save_to_cache (&save_cached, save);@@ -1951,6 +1958,10 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,if (ret) goto out_free; +if (is_guest_mode (vcpu) &&nested_npt_enabled (svm) && + (kvm_state.hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT )) + svm->vmcb->save.g_pat = save_cached.g_pat; + svm->nested.force_msr_bitmap_recalc = true;kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); --2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 6/8] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE
In reply to Jim MattsonQuoted 77 lines · Jim Mattson
Add a 'flags' field to the SVM nested state header, and use bit 0 of the flags to indicate that gPAT is stored in the nested state.
If in guest mode with NPT enabled, store the current
vmcb->save.g_patvalue into the vmcb save area of the nested state, and set the flag.Note that most of the vmcb save area in the nested state is populated with dead (and potentially already clobbered) vmcb01 state. A few fields hold L1 state to be restored at VMEXIT. Previously, the
g_patfield was in the former category.Also note that struct
kvm_svm_nested_state_hdris included in a union padded to 120 bytes, so there is room to add the flags field without changing any offsets.Signed-off-by Jim Mattson <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 3 +++ arch/x86/kvm/svm/nested.c | 13 ++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)arch/x86/include/uapi/asm/kvm.h +3
-0@@ -495,6 +495,8 @@ struct kvm_sync_regs {#define KVM_STATE_VMX_PREEMPTION_TIMER_DEADLINE 0x00000001 +#define KVM_STATE_SVM_VALID_GPAT BIT(0) +/* vendor-independent attributes for system fd (group 0) */ #define KVM_X86_GRP_SYSTEM 0# define KVM_X86_XCOMP_GUEST_SUPP 0@@ -530,6 +532,7 @@ struct kvm_svm_nested_state_data {struct kvm_svm_nested_state_hdr {__u64 vmcb_pa; +__u32 flags; };/* for KVM_CAP_NESTED_STATE */ arch/x86/kvm/svm/nested.c +8
-1@@ -1789,6 +1789,8 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,/* First fill in the header and copy it out. */ if (is_guest_mode (vcpu)) { kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa; +if (nested_npt_enabled (svm)) + kvm_state.hdr.svm.flags |=KVM_STATE_SVM_VALID_GPAT ; kvm_state.size +=KVM_STATE_NESTED_SVM_VMCB_SIZE ; kvm_state.flags |=KVM_STATE_NESTED_GUEST_MODE ;@@ -1823,6 +1825,11 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,if (r)return -EFAULT ; +/* +* vmcb01->save.g_pat is dead now, so it is safe to overwrite it with +* vmcb02->save.g_pat, whether or not nested NPT is enabled. +*/ + svm->vmcb01.ptr->save.g_pat = svm->vmcb->save.g_pat;if (copy_to_user (&user_vmcb->save, &svm->vmcb01.ptr->save,sizeof (user_vmcb->save)))return -EFAULT ;@@ -1904,7 +1911,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, goto out_free;/* -* Validate host state saved from before VMRUN (see +* Validate host state saved from before VMRUN and gPAT (see * nested_svm_check_permissions). */ __nested_copy_vmcb_save_to_cache (&save_cached, save);
arch/x86/kvm/svm/nested.c static int svm_set_nested_state(struct kvm_vcpu *vcpu, · line 1963
Quoted 6 lines
arch/x86/kvm/svm/nested.c +3
-0@@ -1951,6 +1958,10 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,if (ret) goto out_free; +if (is_guest_mode (vcpu) &&nested_npt_enabled (svm) && + (kvm_state.hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT )) + svm->vmcb->save.g_pat = save_cached.g_pat;
is_guest_mode() should always be true here, right?
Quoted 7 lines
arch/x86/kvm/svm/nested.c +1
-0+ svm->nested.force_msr_bitmap_recalc = true;kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu);Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH v2 7/8 KVM: x86: nSVM: Handle restore of legacy nested state
In reply to Jim MattsonWhen nested NPT is enabled and KVM_SET_NESTED_STATE is used to restore an old checkpoint (without a valid gPAT), the current IA32_PAT value must be copied to vmcb02->save.g_pat.
Unfortunately, the current IA32_PAT value may be restored by KVM_SET_MSRS after KVM_SET_NESTED_STATE.
Introduce a new boolean, svm->nested.restore_gpat_from_pat. If set, svm_vcpu_pre_run() will copy vcpu->arch.pat to vmcb02->save.g_pat and clear the boolean.
--- arch/x86/kvm/svm/nested.c | 9 ++++++--- arch/x86/kvm/svm/svm.c | 8 ++++++++ arch/x86/kvm/svm/svm.h | 6 ++++++ 3 files changed, 20 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/nested.c +6 -3
@@ -1958,9 +1958,12 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,if (ret) goto out_free; -if (is_guest_mode (vcpu) &&nested_npt_enabled (svm) && - (kvm_state.hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT )) - svm->vmcb->save.g_pat = save_cached.g_pat; +if (is_guest_mode (vcpu) &&nested_npt_enabled (svm)) { + svm->nested.restore_gpat_from_pat = + !(kvm_state->hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT ); +if (kvm_state->hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT ) + svm->vmcb->save.g_pat = save_cached.g_pat; + } svm->nested.force_msr_bitmap_recalc = true;
arch/x86/kvm/svm/svm.c +8 -0
@@ -4217,9 +4217,17 @@ static void svm_cancel_injection(struct kvm_vcpu *vcpu)static int svm_vcpu_pre_run (struct kvm_vcpu *vcpu) { +struct vcpu_svm *svm =to_svm (vcpu); +if (to_kvm_sev_info (vcpu->kvm)->need_init)return -EINVAL ; +if (svm->nested.restore_gpat_from_pat) { + svm->vmcb->save.g_pat = vcpu->arch.pat; +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + svm->nested.restore_gpat_from_pat = false; + } +return 1 ; }
arch/x86/kvm/svm/svm.h +6 -0
@@ -219,6 +219,12 @@ struct svm_nested_state {* on its side. */ bool force_msr_bitmap_recalc; + +/* +* Indicates that vcpu->arch.pat should be copied to +* vmcb02->save.g_pat at the next vcpu_run. +*/ +bool restore_gpat_from_pat; };struct vcpu_sev_es_state {
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH v2 7/8] KVM: x86: nSVM: Handle restore of legacy nested state
Quoted 48 lines · Jim Mattson
When nested NPT is enabled and
KVM_SET_NESTED_STATEis used to restore an old checkpoint (without a valid gPAT), the currentIA32_PATvalue must be copied tovmcb02->save.g_pat.Unfortunately, the current
IA32_PATvalue may be restored byKVM_SET_MSRSafterKVM_SET_NESTED_STATE.Introduce a new boolean,
svm->nested.restore_gpat_from_pat. If set,svm_vcpu_pre_run()will copyvcpu->arch.pattovmcb02->save.g_patand clear the boolean.Signed-off-by Jim Mattson <[email protected]>--- arch/x86/kvm/svm/nested.c | 9 ++++++--- arch/x86/kvm/svm/svm.c | 8 ++++++++ arch/x86/kvm/svm/svm.h | 6 ++++++ 3 files changed, 20 insertions(+), 3 deletions(-)arch/x86/kvm/svm/nested.c +6
-3@@ -1958,9 +1958,12 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,if (ret) goto out_free; -if (is_guest_mode (vcpu) &&nested_npt_enabled (svm) && - (kvm_state.hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT )) - svm->vmcb->save.g_pat = save_cached.g_pat; +if (is_guest_mode (vcpu) &&nested_npt_enabled (svm)) { + svm->nested.restore_gpat_from_pat = + !(kvm_state->hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT ); +if (kvm_state->hdr.svm.flags &KVM_STATE_SVM_VALID_GPAT ) + svm->vmcb->save.g_pat = save_cached.g_pat; + } svm->nested.force_msr_bitmap_recalc = true;arch/x86/kvm/svm/svm.c +1
-0@@ -4217,9 +4217,17 @@ static void svm_cancel_injection(struct kvm_vcpu *vcpu)static int svm_vcpu_pre_run (struct kvm_vcpu *vcpu) { +struct vcpu_svm *svm =to_svm (vcpu);
arch/x86/kvm/svm/svm.c static void svm_cancel_injection(struct kvm_vcpu *vcpu) · line 4230
Quoted 10 lines
arch/x86/kvm/svm/svm.c +7
-0+if (to_kvm_sev_info (vcpu->kvm)->need_init)return -EINVAL ; +if (svm->nested.restore_gpat_from_pat) { + svm->vmcb->save.g_pat = vcpu->arch.pat; +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); + svm->nested.restore_gpat_from_pat = false; + } +
Something like this is probably needed in svm_get_nested_state() as well, but without clearing the boolean.
Quoted 23 lines
arch/x86/kvm/svm/svm.c +0
-0return 1 ; }arch/x86/kvm/svm/svm.h +6
-1@@ -219,6 +219,12 @@ struct svm_nested_state {* on its side. */ bool force_msr_bitmap_recalc; + +/* +* Indicates that vcpu->arch.pat should be copied to +* vmcb02->save.g_pat at the next vcpu_run. +*/ +bool restore_gpat_from_pat; };struct vcpu_sev_es_state { --2.52.0.457.g6b5491de43-goog
PATCH v2 8/8 KVM: selftests: nSVM: Add svm_nested_pat test
In reply to Jim MattsonVerify KVM's virtualization of the PAT MSR and--when nested NPT is enabled--the vmcb12 g_pat field and the guest PAT register (gPAT).
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/x86/svm_nested_pat_test.c | 357 ++++++++++++++++++ 2 files changed, 358 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/svm_nested_pat_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -109,6 +109,7 @@ TEST_GEN_PROGS_x86 += x86/state_test TEST_GEN_PROGS_x86 += x86/vmx_preemption_timer_test TEST_GEN_PROGS_x86 += x86/svm_vmcall_test TEST_GEN_PROGS_x86 += x86/svm_int_ctl_test +TEST_GEN_PROGS_x86 += x86/svm_nested_pat_test TEST_GEN_PROGS_x86 += x86/svm_nested_shutdown_test TEST_GEN_PROGS_x86 += x86/svm_nested_soft_inject_test TEST_GEN_PROGS_x86 += x86/tsc_scaling_sync
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +357 -0
357 lines not on this page · raw message
Signature · 1 line
2.52.0.457.g6b5491de43-goog