: Check "load IA32_PERF_GLOBAL_CTRL" on vmentry of nested guests
Patch 1 through 7 add the necessary KVM code for checking and enabling "load IA32_PERF_GLOBAL_CTRL" VM-{exit,entry} controls.
Patch# 8 adds a unit test for the "load IA32_PERF_GLOBAL_CTRL" VM-exit control. I will send a separate patch for the unit test for "load IA32_PERF_GLOBAL_CTRL" VM-entry control.
[PATCH 1/8][KVMnVMX]: Enable "load IA32_PERF_GLOBAL_CTRL" VM-exit control [PATCH 2/8][KVM nVMX]: Enable "load IA32_PERF_GLOBAL_CTRL" VM-entry [PATCH 3/8][KVM VMX]: Add a function to check reserved bits in [PATCH 4/8][KVM nVMX]: Check "load IA32_PERF_GLOBAL_CTRL" VM-exit control [PATCH 5/8][KVM nVMX]: Check "load IA32_PERF_GLOBAL_CTRL" VM-entry control [PATCH 6/8][KVM nVMX]: Load IA32_PERF_GLOBAL_CTRL MSR on vmentry of nested [PATCH 7/8][KVM nVMX]: Enable "load IA32_PERF_GLOBAL_CTRL VM-{entry,exit} [PATCH 8/8][KVM nVMX]: Test "load IA32_PERF_GLOBAL_CTRL" controls on vmentry
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/asm/msr-index.h | 7 +++++++ arch/x86/kvm/vmx/nested.c | 19 +++++++++++++++++-- arch/x86/kvm/vmx/vmx.c | 12 ++++++++++++ arch/x86/kvm/x86.c | 20 ++++++++++++++++++++ 5 files changed, 57 insertions(+), 2 deletions(-)
Krish Sadhukhan (7):
nVMX: Enable "load IA32_PERF_GLOBAL_CTRL" VM-exit control for nested guests
nVMX: Enable "load IA32_PERF_GLOBAL_CTRL" VM-entry control for nested guests
VMX: Add a function to check reserved bits in MSR_CORE_PERF_GLOBAL_CTRL
nVMX: Check "load IA32_PERF_GLOBAL_CTRL" VM-exit control on vmentry of nested guests
nVMX: Check "load IA32_PERF_GLOBAL_CTRL" VM-entry control on vmentry of nested guests
nVMX: Load IA32_PERF_GLOBAL_CTRL MSR on vmentry of nested guests
nVMX: Enable "load IA32_PERF_GLOBAL_CTRL VM-{entry,exit} controls
x86/vmx_tests.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+)
Krish Sadhukhan (1):
nVMX: Test "load IA32_PERF_GLOBAL_CTRL" controls on vmentry of nested guests
PATCH 1/8 : Enable "load IA32_PERF_GLOBAL_CTRL" VM-exit control for nested guests
--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/nested.c +2 -1
@@ -5493,7 +5493,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps, msrs->exit_ctls_high |=VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR |VM_EXIT_LOAD_IA32_EFER |VM_EXIT_SAVE_IA32_EFER | -VM_EXIT_SAVE_VMX_PREEMPTION_TIMER |VM_EXIT_ACK_INTR_ON_EXIT ; +VM_EXIT_SAVE_VMX_PREEMPTION_TIMER |VM_EXIT_ACK_INTR_ON_EXIT | +VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL ;/* We support free control of debug control saving. */ msrs->exit_ctls_low &= ~VM_EXIT_SAVE_DEBUG_CONTROLS ;
Signature · 1 line
2.17.2
Re: [PATCH 1/8][KVMnVMX]: Enable "load IA32_PERF_GLOBAL_CTRL" VM-exit control for nested guests
Quoted 11 lines · Krish Sadhukhan
Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/x86/kvm/vmx/nested.c +0
-0
arch/x86/kvm/vmx/nested.c void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps, · line 5497
Quoted 6 lines
arch/x86/kvm/vmx/nested.c +2
-1@@ -5493,7 +5493,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps, msrs->exit_ctls_high |=VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR |VM_EXIT_LOAD_IA32_EFER |VM_EXIT_SAVE_IA32_EFER | -VM_EXIT_SAVE_VMX_PREEMPTION_TIMER |VM_EXIT_ACK_INTR_ON_EXIT ; +VM_EXIT_SAVE_VMX_PREEMPTION_TIMER |VM_EXIT_ACK_INTR_ON_EXIT | +VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL ;
This exposes the control to L1 but doesn't implement the backing functionality. The backing functionality, consistency checks and exposure to L1 should be a single patch. The consistency checks could be added earlier, but I don't see much value in doing so given that the checks are (currently) a few lines.
Quoted 6 lines
arch/x86/kvm/vmx/nested.c +0
-0/* We support free control of debug control saving. */ msrs->exit_ctls_low &= ~VM_EXIT_SAVE_DEBUG_CONTROLS ;Signature · 1 line
2.17.2
Re: [PATCH 1/8][KVMnVMX]: Enable "load IA32_PERF_GLOBAL_CTRL" VM-exit control for nested guests
Quoted 7 lines · Sean Christopherson
Quoted 6 lines · Krish Sadhukhan
Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Quoted 16 lines
arch/x86/kvm/vmx/nested.c void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps, · line 5497
Quoted 11 lines
arch/x86/kvm/vmx/nested.c +2
-1@@ -5493,7 +5493,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps, msrs->exit_ctls_high |=VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR |VM_EXIT_LOAD_IA32_EFER |VM_EXIT_SAVE_IA32_EFER | -VM_EXIT_SAVE_VMX_PREEMPTION_TIMER |VM_EXIT_ACK_INTR_ON_EXIT ; +VM_EXIT_SAVE_VMX_PREEMPTION_TIMER |VM_EXIT_ACK_INTR_ON_EXIT | +VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL ;This exposes the control to L1 but doesn't implement the backing functionality. The backing functionality, consistency checks and exposure to L1 should be a single patch. The consistency checks could be added earlier, but I don't see much value in doing so given that the checks are (currently) a few lines.
I will combine the exposure of the control and its backing functionality into a single patch. But I would prefer to keep the consistency checks in separate patches just to make it a gradual progression i.e., first enabling the controls in a patch and then checking their consistency in the successive patch.
Quoted 7 lines
Quoted 7 lines
arch/x86/kvm/vmx/nested.c +0
-0/* We support free control of debug control saving. */ msrs->exit_ctls_low &= ~VM_EXIT_SAVE_DEBUG_CONTROLS ;Signature · 1 line
2.17.2
PATCH 2/8 : Enable "load IA32_PERF_GLOBAL_CTRL" VM-entry control for nested guests
In reply to Krish Sadhukhan--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/nested.c +2 -1
@@ -5511,7 +5511,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps,#endif VM_ENTRY_LOAD_IA32_PAT ; msrs->entry_ctls_high |= - (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR |VM_ENTRY_LOAD_IA32_EFER ); + (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR |VM_ENTRY_LOAD_IA32_EFER | +VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL );/* We support free control of debug control loading. */ msrs->entry_ctls_low &= ~VM_ENTRY_LOAD_DEBUG_CONTROLS ;
Signature · 1 line
2.17.2
Re: [PATCH 2/8][KVM nVMX]: Enable "load IA32_PERF_GLOBAL_CTRL" VM-entry control for nested guests
Quoted 11 lines · Krish Sadhukhan
Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/x86/kvm/vmx/nested.c +0
-0
arch/x86/kvm/vmx/nested.c void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps, · line 5515
Quoted 6 lines
arch/x86/kvm/vmx/nested.c +2
-1@@ -5511,7 +5511,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps,#endif VM_ENTRY_LOAD_IA32_PAT ; msrs->entry_ctls_high |= - (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR |VM_ENTRY_LOAD_IA32_EFER ); + (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR |VM_ENTRY_LOAD_IA32_EFER | +VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL );
Same comments as the VM-Exit patch.
Quoted 6 lines
arch/x86/kvm/vmx/nested.c +0
-0/* We support free control of debug control loading. */ msrs->entry_ctls_low &= ~VM_ENTRY_LOAD_DEBUG_CONTROLS ;Signature · 1 line
2.17.2
PATCH 3/8 : Add a function to check reserved bits in MSR_CORE_PERF_GLOBAL_CTRL
In reply to Krish Sadhukhan--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/asm/msr-index.h | 7 +++++++ arch/x86/kvm/x86.c | 20 ++++++++++++++++++++ 3 files changed, 28 insertions(+)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1311,6 +1311,7 @@ int kvm_emulate_instruction_from_buffer(struct kvm_vcpu *vcpu,void kvm_enable_efer_bits (u64 );bool kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer); +bool kvm_valid_perf_global_ctrl (u64 perf_global);int kvm_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr);int kvm_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr);
arch/x86/include/asm/msr-index.h +7 -0
@@ -775,6 +775,13 @@#define MSR_CORE_PERF_GLOBAL_CTRL 0x0000038f#define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x00000390 +/* MSR_CORE_PERF_GLOBAL_CTRL bits */ +#define PERF_GLOBAL_CTRL_PMC0_ENABLE (1ull << 0) +#define PERF_GLOBAL_CTRL_PMC1_ENABLE (1ull << 1) +#define PERF_GLOBAL_CTRL_FIXED0_ENABLE (1ull << 32) +#define PERF_GLOBAL_CTRL_FIXED1_ENABLE (1ull << 33) +#define PERF_GLOBAL_CTRL_FIXED2_ENABLE (1ull << 34) +/* Geode defined MSRs */ #define MSR_GEODE_BUSCONT_CONF0 0x00001900
arch/x86/kvm/x86.c +20 -0
@@ -89,8 +89,19 @@ EXPORT_SYMBOL_GPL(kvm_mce_cap_supported);#ifdef CONFIG_X86_64 static u64 __read_mostly efer_reserved_bits = ~((u64)(EFER_SCE |EFER_LME |EFER_LMA )); +static +u64 __read_mostly perf_global_ctrl_reserved_bits = + ~((u64)(PERF_GLOBAL_CTRL_PMC0_ENABLE | +PERF_GLOBAL_CTRL_PMC1_ENABLE | +PERF_GLOBAL_CTRL_FIXED0_ENABLE | +PERF_GLOBAL_CTRL_FIXED1_ENABLE | +PERF_GLOBAL_CTRL_FIXED2_ENABLE ));#else static u64 __read_mostly efer_reserved_bits = ~((u64 )EFER_SCE ); +static +u64 __read_mostly perf_global_ctrl_reserved_bits = + ~((u64)(PERF_GLOBAL_CTRL_PMC0_ENABLE | +PERF_GLOBAL_CTRL_PMC1_ENABLE ));#endif #define VM_STAT (x) offsetof(struct kvm, stat.x), KVM_STAT_VM
@@ -1255,6 +1266,15 @@ static int do_get_msr_feature(struct kvm_vcpu *vcpu, unsigned index, u64 *data)return 0 ; } +bool kvm_valid_perf_global_ctrl (u64 perf_global) +{ +if (perf_global & perf_global_ctrl_reserved_bits) +return false; + +return true; +} +EXPORT_SYMBOL_GPL (kvm_valid_perf_global_ctrl); +bool kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer) {if (efer & efer_reserved_bits)
Signature · 1 line
2.17.2
Re: [PATCH 3/8][KVM VMX]: Add a function to check reserved bits in MSR_CORE_PERF_GLOBAL_CTRL
Quoted 25 lines · Krish Sadhukhan
Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/asm/msr-index.h | 7 +++++++ arch/x86/kvm/x86.c | 20 ++++++++++++++++++++ 3 files changed, 28 insertions(+)arch/x86/include/asm/kvm_host.h +1
-0@@ -1311,6 +1311,7 @@ int kvm_emulate_instruction_from_buffer(struct kvm_vcpu *vcpu,void kvm_enable_efer_bits (u64 );bool kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer); +bool kvm_valid_perf_global_ctrl (u64 perf_global);int kvm_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr);int kvm_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr);arch/x86/include/asm/msr-index.h +0
-0
arch/x86/include/asm/msr-index.h line 779
Quoted 5 lines
arch/x86/include/asm/msr-index.h +2
-0@@ -775,6 +775,13 @@#define MSR_CORE_PERF_GLOBAL_CTRL 0x0000038f#define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x00000390 +/* MSR_CORE_PERF_GLOBAL_CTRL bits */ +#define PERF_GLOBAL_CTRL_PMC0_ENABLE (1ull << 0)
BIT and BIT_ULL
Quoted 16 lines
arch/x86/include/asm/msr-index.h +5
-0+#define PERF_GLOBAL_CTRL_PMC1_ENABLE (1ull << 1) +#define PERF_GLOBAL_CTRL_FIXED0_ENABLE (1ull << 32) +#define PERF_GLOBAL_CTRL_FIXED1_ENABLE (1ull << 33) +#define PERF_GLOBAL_CTRL_FIXED2_ENABLE (1ull << 34) +/* Geode defined MSRs */ #define MSR_GEODE_BUSCONT_CONF0 0x00001900arch/x86/kvm/x86.c +0
-0@@ -89,8 +89,19 @@ EXPORT_SYMBOL_GPL(kvm_mce_cap_supported);#ifdef CONFIG_X86_64 static u64 __read_mostly efer_reserved_bits = ~((u64)(EFER_SCE |EFER_LME |EFER_LMA ));
arch/x86/kvm/x86.c EXPORT_SYMBOL_GPL(kvm_mce_cap_supported); · line 101
Quoted 10 lines
arch/x86/kvm/x86.c +8
-0+static +u64 __read_mostly perf_global_ctrl_reserved_bits = + ~((u64)(PERF_GLOBAL_CTRL_PMC0_ENABLE | +PERF_GLOBAL_CTRL_PMC1_ENABLE | +PERF_GLOBAL_CTRL_FIXED0_ENABLE | +PERF_GLOBAL_CTRL_FIXED1_ENABLE | +PERF_GLOBAL_CTRL_FIXED2_ENABLE ));#else static u64 __read_mostly efer_reserved_bits = ~((u64 )EFER_SCE ); +static
Why is static on a different line?
arch/x86/kvm/x86.c EXPORT_SYMBOL_GPL(kvm_mce_cap_supported); · line 104
Quoted 3 lines
arch/x86/kvm/x86.c +3
-0+u64 __read_mostly perf_global_ctrl_reserved_bits = + ~((u64)(PERF_GLOBAL_CTRL_PMC0_ENABLE | +PERF_GLOBAL_CTRL_PMC1_ENABLE ));
Why are the fixed bits reserved on a 32-bit build?
arch/x86/kvm/x86.c static int do_get_msr_feature(struct kvm_vcpu *vcpu, unsigned index, u64 *data) · line 1271
Quoted 10 lines
arch/x86/kvm/x86.c +3
-0#endif #define VM_STAT (x) offsetof(struct kvm, stat.x), KVM_STAT_VM@@ -1255,6 +1266,15 @@ static int do_get_msr_feature(struct kvm_vcpu *vcpu, unsigned index, u64 *data)return 0 ; } +bool kvm_valid_perf_global_ctrl (u64 perf_global) +{ +if (perf_global & perf_global_ctrl_reserved_bits)
If the check were correct, this could be:
return !(perf_blobal & perf_global_ctrl_reserved_bits);
But the check isn't correct, the number of counters is variable, i.e. the helper should query the guest's CPUID 0xA (ignoring for the moment the fact that this bypasses the PMU handling of guest vs. host ownership).
Quoted 12 lines
arch/x86/kvm/x86.c +6
-0+return false; + +return true; +} +EXPORT_SYMBOL_GPL (kvm_valid_perf_global_ctrl); +bool kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer) {if (efer & efer_reserved_bits)Signature · 1 line
2.17.2
Re: [PATCH 3/8][KVM VMX]: Add a function to check reserved bits in MSR_CORE_PERF_GLOBAL_CTRL
Quoted 83 lines · Sean Christopherson
Quoted 25 lines · Krish Sadhukhan
Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/asm/msr-index.h | 7 +++++++ arch/x86/kvm/x86.c | 20 ++++++++++++++++++++ 3 files changed, 28 insertions(+)arch/x86/include/asm/kvm_host.h +1
-0@@ -1311,6 +1311,7 @@ int kvm_emulate_instruction_from_buffer(struct kvm_vcpu *vcpu,void kvm_enable_efer_bits (u64 );bool kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer); +bool kvm_valid_perf_global_ctrl (u64 perf_global);int kvm_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr);int kvm_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr);arch/x86/include/asm/msr-index.h +0
-0arch/x86/include/asm/msr-index.h line 779
Quoted 5 lines
arch/x86/include/asm/msr-index.h +2
-0@@ -775,6 +775,13 @@#define MSR_CORE_PERF_GLOBAL_CTRL 0x0000038f#define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x00000390 +/* MSR_CORE_PERF_GLOBAL_CTRL bits */ +#define PERF_GLOBAL_CTRL_PMC0_ENABLE (1ull << 0)BIT and
BIT_ULLQuoted 16 lines
arch/x86/include/asm/msr-index.h +5
-0+#define PERF_GLOBAL_CTRL_PMC1_ENABLE (1ull << 1) +#define PERF_GLOBAL_CTRL_FIXED0_ENABLE (1ull << 32) +#define PERF_GLOBAL_CTRL_FIXED1_ENABLE (1ull << 33) +#define PERF_GLOBAL_CTRL_FIXED2_ENABLE (1ull << 34) +/* Geode defined MSRs */ #define MSR_GEODE_BUSCONT_CONF0 0x00001900arch/x86/kvm/x86.c +0
-0@@ -89,8 +89,19 @@ EXPORT_SYMBOL_GPL(kvm_mce_cap_supported);#ifdef CONFIG_X86_64 static u64 __read_mostly efer_reserved_bits = ~((u64)(EFER_SCE |EFER_LME |EFER_LMA ));arch/x86/kvm/x86.c EXPORT_SYMBOL_GPL(kvm_mce_cap_supported); · line 101
Quoted 10 lines
arch/x86/kvm/x86.c +8
-0+static +u64 __read_mostly perf_global_ctrl_reserved_bits = + ~((u64)(PERF_GLOBAL_CTRL_PMC0_ENABLE | +PERF_GLOBAL_CTRL_PMC1_ENABLE | +PERF_GLOBAL_CTRL_FIXED0_ENABLE | +PERF_GLOBAL_CTRL_FIXED1_ENABLE | +PERF_GLOBAL_CTRL_FIXED2_ENABLE ));#else static u64 __read_mostly efer_reserved_bits = ~((u64 )EFER_SCE ); +staticWhy is static on a different line?
arch/x86/kvm/x86.c EXPORT_SYMBOL_GPL(kvm_mce_cap_supported); · line 104
Quoted 3 lines
arch/x86/kvm/x86.c +3
-0+u64 __read_mostly perf_global_ctrl_reserved_bits = + ~((u64)(PERF_GLOBAL_CTRL_PMC0_ENABLE | +PERF_GLOBAL_CTRL_PMC1_ENABLE ));Why are the fixed bits reserved on a 32-bit build?
arch/x86/kvm/x86.c static int do_get_msr_feature(struct kvm_vcpu *vcpu, unsigned index, u64 *data) · line 1271
Quoted 10 lines
arch/x86/kvm/x86.c +3
-0#endif #define VM_STAT (x) offsetof(struct kvm, stat.x), KVM_STAT_VM@@ -1255,6 +1266,15 @@ static int do_get_msr_feature(struct kvm_vcpu *vcpu, unsigned index, u64 *data)return 0 ; } +bool kvm_valid_perf_global_ctrl (u64 perf_global) +{ +if (perf_global & perf_global_ctrl_reserved_bits)If the check were correct, this could be:
Quoted 5 lines
return !(perf_blobal & perf_global_ctrl_reserved_bits);But the check isn't correct, the number of counters is variable, i.e. the helper should query the guest's CPUID
0xA(ignoring for the moment the fact that this bypasses the PMU handling of guest vs. host ownership).
Aren't the reserved bits already easily calculated as ~pmu->global_ctrl_mask? Well, there's also bit 48, potentially, I guess, but I don't think kvm virtualizes it.
Once we have this concept, shouldn't intel_pmu_set_msr() return non-zero if an attempt is made to set a reserved bit of this MSR?
Quoted 12 lines
Quoted 12 lines
arch/x86/kvm/x86.c +6
-1+return false; + +return true; +} +EXPORT_SYMBOL_GPL (kvm_valid_perf_global_ctrl); +bool kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer) {if (efer & efer_reserved_bits) --2.17.2
PATCH 4/8 : Check "load IA32_PERF_GLOBAL_CTRL" VM-exit control on vmentry of nested guests
In reply to Krish SadhukhanAccording to section "Checks on Host Control Registers and MSRs" in Intel SDM vol 3C, the following check is performed on vmentry of nested guests:
"If the "load IA32_PERF_GLOBAL_CTRL" VM-exit control is 1, bits reserved in the IA32_PERF_GLOBAL_CTRL MSR must be 0 in the field for that register."
--- arch/x86/kvm/vmx/nested.c | 5 +++++ 1 file changed, 5 insertions(+)
arch/x86/kvm/vmx/nested.c +5 -0
@@ -2595,6 +2595,11 @@ static int nested_check_host_control_regs(struct kvm_vcpu *vcpu, !nested_host_cr4_valid (vcpu, vmcs12->host_cr4) || !nested_cr3_valid(vcpu, vmcs12->host_cr3)) return -EINVAL; + +if (vmcs12->vm_exit_controls &VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL && + !kvm_valid_perf_global_ctrl (vmcs12->host_ia32_perf_global_ctrl)) +return -EINVAL ; +/* * If the load IA32_EFER VM-exit control is 1, bits reserved in the * IA32_EFER MSR must be 0 in the field for that register. In addition,
Signature · 1 line
2.17.2
Re: [PATCH 4/8][KVM nVMX]: Check "load IA32_PERF_GLOBAL_CTRL" VM-exit control on vmentry of nested guests
Quoted 18 lines · Krish Sadhukhan
According to section "Checks on Host Control Registers and MSRs" in Intel SDM vol 3C, the following check is performed on vmentry of nested guests:
"If the "load
IA32_PERF_GLOBAL_CTRL" VM-exit control is 1, bits reserved in theIA32_PERF_GLOBAL_CTRLMSR must be 0 in the field for that register."Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 5 +++++ 1 file changed, 5 insertions(+)arch/x86/kvm/vmx/nested.c +0
-0
arch/x86/kvm/vmx/nested.c static int nested_check_host_control_regs(struct kvm_vcpu *vcpu, · line 2600
Quoted 6 lines
arch/x86/kvm/vmx/nested.c +3
-0@@ -2595,6 +2595,11 @@ static int nested_check_host_control_regs(struct kvm_vcpu *vcpu, !nested_host_cr4_valid (vcpu, vmcs12->host_cr4) || !nested_cr3_valid(vcpu, vmcs12->host_cr3)) return -EINVAL; + +if (vmcs12->vm_exit_controls &VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL && + !kvm_valid_perf_global_ctrl (vmcs12->host_ia32_perf_global_ctrl))
If vmcs12->host_ia32_perf_global_ctrl were ever actually consumed, this needs to ensure L1 isn't able to take control of counters that are owned by the host.
Quoted 8 lines
arch/x86/kvm/vmx/nested.c +2
-0+return -EINVAL ; +/* * If the load IA32_EFER VM-exit control is 1, bits reserved in the * IA32_EFER MSR must be 0 in the field for that register. In addition, Signature · 1 line
2.17.2
Re: [PATCH 4/8][KVM nVMX]: Check "load IA32_PERF_GLOBAL_CTRL" VM-exit control on vmentry of nested guests
Quoted 14 lines · Sean Christopherson
Quoted 13 lines · Krish Sadhukhan
According to section "Checks on Host Control Registers and MSRs" in Intel SDM vol 3C, the following check is performed on vmentry of nested guests:
"If the "load
IA32_PERF_GLOBAL_CTRL" VM-exit control is 1, bits reserved in theIA32_PERF_GLOBAL_CTRLMSR must be 0 in the field for that register."Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 5 +++++ 1 file changed, 5 insertions(+)
Quoted 14 lines
arch/x86/kvm/vmx/nested.c static int nested_check_host_control_regs(struct kvm_vcpu *vcpu, · line 2600
Quoted 11 lines
arch/x86/kvm/vmx/nested.c +3
-0@@ -2595,6 +2595,11 @@ static int nested_check_host_control_regs(struct kvm_vcpu *vcpu, !nested_host_cr4_valid (vcpu, vmcs12->host_cr4) || !nested_cr3_valid(vcpu, vmcs12->host_cr3)) return -EINVAL; + +if (vmcs12->vm_exit_controls &VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL && + !kvm_valid_perf_global_ctrl (vmcs12->host_ia32_perf_global_ctrl))If
vmcs12->host_ia32_perf_global_ctrlwere ever actually consumed, this needs to ensure L1 isn't able to take control of counters that are owned by the host.
Sorry, I didn't understand your concern. Could you please explain how L1 can control L0's counters ?
Quoted 9 lines
Quoted 9 lines
arch/x86/kvm/vmx/nested.c +2
-0+return -EINVAL ; +/* * If the load IA32_EFER VM-exit control is 1, bits reserved in the * IA32_EFER MSR must be 0 in the field for that register. In addition, Signature · 1 line
2.17.2
Re: [PATCH 4/8][KVM nVMX]: Check "load IA32_PERF_GLOBAL_CTRL" VM-exit control on vmentry of nested guests
Quoted 32 lines · Krish Sadhukhan
Quoted 14 lines · Sean Christopherson
Quoted 13 lines · Krish Sadhukhan
According to section "Checks on Host Control Registers and MSRs" in Intel SDM vol 3C, the following check is performed on vmentry of nested guests:
"If the "load
IA32_PERF_GLOBAL_CTRL" VM-exit control is 1, bits reserved in theIA32_PERF_GLOBAL_CTRLMSR must be 0 in the field for that register."Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 5 +++++ 1 file changed, 5 insertions(+)Quoted 14 lines
arch/x86/kvm/vmx/nested.c static int nested_check_host_control_regs(struct kvm_vcpu *vcpu, · line 2600
Quoted 11 lines
arch/x86/kvm/vmx/nested.c +3
-0@@ -2595,6 +2595,11 @@ static int nested_check_host_control_regs(struct kvm_vcpu *vcpu, !nested_host_cr4_valid (vcpu, vmcs12->host_cr4) || !nested_cr3_valid(vcpu, vmcs12->host_cr3)) return -EINVAL; + +if (vmcs12->vm_exit_controls &VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL && + !kvm_valid_perf_global_ctrl (vmcs12->host_ia32_perf_global_ctrl))If
vmcs12->host_ia32_perf_global_ctrlwere ever actually consumed, this needs to ensure L1 isn't able to take control of counters that are owned by the host.
Quoted 2 lines
Sorry, I didn't understand your concern. Could you please explain how L1 can control L0's counters ?
MSR_CORE_PERF_GLOBAL_CTRL isn't virtualized in the sense that there is only one MSR in hardware (per logical CPU). Loading an arbitrary value into hardware on a nested VM-Exit via vmcs12->host_ia32_perf_global_ctrl means L1 could toggle counters on/off without L0's knowledge. See intel_guest_get_msrs() and intel_pmu_set_msr().
Except that your patches don't actually do anything functional with vmcs12->host_ia32_perf_global_ctrl, hence my confusion over what you're trying to accomplish. If KVM advertises VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL to L1, then L1 will think it can use the VMCS field to handle MSR_CORE_PERF_GLOBAL_CTRL when running L2, e.g. instead of adding MSR_CORE_PERF_GLOBAL_CTRL to the MSR load lists, which will break L1 (assuming nested PMU works today).
Re: [PATCH 4/8][KVM nVMX]: Check "load IA32_PERF_GLOBAL_CTRL" VM-exit control on vmentry of nested guests
Quoted 43 lines · Sean Christopherson
Quoted 32 lines · Krish Sadhukhan
Quoted 14 lines · Sean Christopherson
Quoted 13 lines · Krish Sadhukhan
According to section "Checks on Host Control Registers and MSRs" in Intel SDM vol 3C, the following check is performed on vmentry of nested guests:
"If the "load
IA32_PERF_GLOBAL_CTRL" VM-exit control is 1, bits reserved in theIA32_PERF_GLOBAL_CTRLMSR must be 0 in the field for that register."Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 5 +++++ 1 file changed, 5 insertions(+)Quoted 14 lines
arch/x86/kvm/vmx/nested.c static int nested_check_host_control_regs(struct kvm_vcpu *vcpu, · line 2600
Quoted 11 lines
arch/x86/kvm/vmx/nested.c +3
-0@@ -2595,6 +2595,11 @@ static int nested_check_host_control_regs(struct kvm_vcpu *vcpu, !nested_host_cr4_valid (vcpu, vmcs12->host_cr4) || !nested_cr3_valid(vcpu, vmcs12->host_cr3)) return -EINVAL; + +if (vmcs12->vm_exit_controls &VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL && + !kvm_valid_perf_global_ctrl (vmcs12->host_ia32_perf_global_ctrl))If
vmcs12->host_ia32_perf_global_ctrlwere ever actually consumed, this needs to ensure L1 isn't able to take control of counters that are owned by the host.Quoted 2 lines
Sorry, I didn't understand your concern. Could you please explain how L1 can control L0's counters ?
MSR_CORE_PERF_GLOBAL_CTRLisn't virtualized in the sense that there is only one MSR in hardware (per logical CPU). Loading an arbitrary value into hardware on a nested VM-Exit viavmcs12->host_ia32_perf_global_ctrlmeans L1 could toggle counters on/off without L0's knowledge. Seeintel_guest_get_msrs()andintel_pmu_set_msr().
Quoted 7 lines
Except that your patches don't actually do anything functional with
vmcs12->host_ia32_perf_global_ctrl, hence my confusion over what you're trying to accomplish. If KVM advertisesVM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRLto L1, then L1 will think it can use the VMCS field to handleMSR_CORE_PERF_GLOBAL_CTRLwhen running L2, e.g. instead of addingMSR_CORE_PERF_GLOBAL_CTRLto the MSR load lists, which will break L1 (assuming nested PMU works today).
I think nested PMU should work today if the vmcs12 VM-entry and VM-exit MSR-load lists are used to swap IA32_PERF_GLOBAL_CTRL values between L1 and L2. Note that atomic_switch_perf_msrs() is called on every vmx_vcpu_run(), for vmcs02 as well as vmcs01.
This change set should aim to provide equivalent functionality with the new(er) VM-entry and VM-exit controls for "load IA32_PERF_GLOBAL_CTRL," so that L1 doesn't have to resort to the MSR-load lists.
PATCH 5/8 : Check "load IA32_PERF_GLOBAL_CTRL" VM-entry control on vmentry of nested guests
In reply to Krish SadhukhanAccording to section "CHECKING AND LOADING GUEST STATE" in Intel SDM vol 3C, the following checks are performed on vmentry of nested guests:
"If the "load IA32_PERF_GLOBAL_CTRL" VM-entry control is 1, bits reserved in the IA32_PERF_GLOBAL_CTRL MSR must be 0 in the field for that register."
--- arch/x86/kvm/vmx/nested.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/kvm/vmx/nested.c +4 -0
@@ -2691,6 +2691,10 @@ static int nested_vmx_check_vmentry_postreqs(struct kvm_vcpu *vcpu,return 1 ; } +if (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL && + !kvm_valid_perf_global_ctrl (vmcs12->guest_ia32_perf_global_ctrl)) +return 1 ; +/* * If the load IA32_EFER VM-entry control is 1, the following checks * are performed on the field for the IA32_EFER MSR:
Signature · 1 line
2.17.2
Re: [PATCH 5/8][KVM nVMX]: Check "load IA32_PERF_GLOBAL_CTRL" VM-entry control on vmentry of nested guests
Quoted 19 lines · Krish Sadhukhan
According to section "CHECKING AND LOADING GUEST STATE" in Intel SDM vol 3C, the following checks are performed on vmentry of nested guests:
"If the "load
IA32_PERF_GLOBAL_CTRL" VM-entry control is 1, bits reserved in theIA32_PERF_GLOBAL_CTRLMSR must be 0 in the field for that register."Signed-off-by Krish Sadhukhan <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/vmx/nested.c +0
-0
arch/x86/kvm/vmx/nested.c static int nested_vmx_check_vmentry_postreqs(struct kvm_vcpu *vcpu, · line 2697
Quoted 7 lines
arch/x86/kvm/vmx/nested.c +4
-0@@ -2691,6 +2691,10 @@ static int nested_vmx_check_vmentry_postreqs(struct kvm_vcpu *vcpu,return 1 ; } +if (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL && + !kvm_valid_perf_global_ctrl (vmcs12->guest_ia32_perf_global_ctrl)) +return 1 ; +
I'd rather see this built on an interface like:
bool kvm_valid_msr_value(u32 msr_index, u64 value);
But as long as we don't end up with a plethora of kvm_valid_MSR_NAME(u64 value) functions, this looks fine.
PATCH 6/8 : Load IA32_PERF_GLOBAL_CTRL MSR on vmentry of nested guests
In reply to Krish SadhukhanAccording to section "Loading Guest State" in Intel SDM vol 3C, the IA32_PERF_GLOBAL_CTRL MSR is loaded on vmentry of nested guests:
"If the “load IA32_PERF_GLOBAL_CTRL” VM-entry control is 1, the
IA32_PERF_GLOBAL_CTRL MSR is loaded from the IA32_PERF_GLOBAL_CTRL field."
--- arch/x86/kvm/vmx/nested.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/kvm/vmx/nested.c +4 -0
@@ -2300,6 +2300,10 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, vcpu->arch.cr0_guest_owned_bits &= ~vmcs12->cr0_guest_host_mask;vmcs_writel (CR0_GUEST_HOST_MASK , ~vcpu->arch.cr0_guest_owned_bits); +if (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ) +vmcs_write64 (GUEST_IA32_PERF_GLOBAL_CTRL , + vmcs12->guest_ia32_perf_global_ctrl); +if (vmx->nested.nested_run_pending && (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PAT )) {vmcs_write64 (GUEST_IA32_PAT , vmcs12->guest_ia32_pat);
Signature · 1 line
2.17.2
Re: [PATCH 6/8][KVM nVMX]: Load IA32_PERF_GLOBAL_CTRL MSR on vmentry of nested guests
Quoted 31 lines · Krish Sadhukhan
According to section "Loading Guest State" in Intel SDM vol 3C, the
IA32_PERF_GLOBAL_CTRLMSR is loaded on vmentry of nested guests:"If the “load IA32_PERF_GLOBAL_CTRL” VM-entry control is 1, the
IA32_PERF_GLOBAL_CTRLMSR is loaded from theIA32_PERF_GLOBAL_CTRLfield."Signed-off-by Krish Sadhukhan <[email protected]>Suggested-by Jim Mattson <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/vmx/nested.c +4
-1@@ -2300,6 +2300,10 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, vcpu->arch.cr0_guest_owned_bits &= ~vmcs12->cr0_guest_host_mask;vmcs_writel (CR0_GUEST_HOST_MASK , ~vcpu->arch.cr0_guest_owned_bits); +if (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ) +vmcs_write64 (GUEST_IA32_PERF_GLOBAL_CTRL , + vmcs12->guest_ia32_perf_global_ctrl); +if (vmx->nested.nested_run_pending && (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PAT )) {vmcs_write64 (GUEST_IA32_PAT , vmcs12->guest_ia32_pat); --
Quoted 2 lines
2.17.2
This isn't quite right. The GUEST_IA32_PERF_GLOBAL_CTRL value is just going to get overwritten later by atomic_switch_perf_msrs().
Instead of writing the vmcs12 value directly into the vmcs02, you should call kvm_set_msr(), exactly as it would have been called if MSR_CORE_PERF_GLOBAL_CTRL had been in the vmcs12 VM-entry MSR-load list. Then, atomic_switch_perf_msrs() will automatically do the right thing.
Re: [PATCH 6/8][KVM nVMX]: Load IA32_PERF_GLOBAL_CTRL MSR on vmentry of nested guests
Quoted 24 lines · Jim Mattson
Quoted 22 lines · Krish Sadhukhan
According to section "Loading Guest State" in Intel SDM vol 3C, the
IA32_PERF_GLOBAL_CTRLMSR is loaded on vmentry of nested guests:"If the “load IA32_PERF_GLOBAL_CTRL” VM-entry control is 1, the
IA32_PERF_GLOBAL_CTRLMSR is loaded from theIA32_PERF_GLOBAL_CTRLfield."Signed-off-by Krish Sadhukhan <[email protected]>Suggested-by Jim Mattson <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/vmx/nested.c +0
-0@@ -2300,6 +2300,10 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, vcpu->arch.cr0_guest_owned_bits &= ~vmcs12->cr0_guest_host_mask;vmcs_writel (CR0_GUEST_HOST_MASK , ~vcpu->arch.cr0_guest_owned_bits);
Quoted 12 lines
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +4
-1+if (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ) +vmcs_write64 (GUEST_IA32_PERF_GLOBAL_CTRL , + vmcs12->guest_ia32_perf_global_ctrl); +if (vmx->nested.nested_run_pending && (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PAT )) {vmcs_write64 (GUEST_IA32_PAT , vmcs12->guest_ia32_pat); --2.17.2
This isn't quite right. The
GUEST_IA32_PERF_GLOBAL_CTRLvalue is just going to get overwritten later byatomic_switch_perf_msrs().
atomic_switch_perf_msrs() gest called from vmx_vcpu_run() which I thought was executing before handle_vmlaunch() stuff that lead to prepare_vmcs02(). Did I miss something in the call-chain ?
Quoted 5 lines
Instead of writing the vmcs12 value directly into the vmcs02, you should call
kvm_set_msr(), exactly as it would have been called ifMSR_CORE_PERF_GLOBAL_CTRLhad been in the vmcs12 VM-entry MSR-load list. Then,atomic_switch_perf_msrs()will automatically do the right thing.
Re: [PATCH 6/8][KVM nVMX]: Load IA32_PERF_GLOBAL_CTRL MSR on vmentry of nested guests
Quoted 41 lines · Krish Sadhukhan
Quoted 24 lines · Jim Mattson
Quoted 22 lines · Krish Sadhukhan
According to section "Loading Guest State" in Intel SDM vol 3C, the
IA32_PERF_GLOBAL_CTRLMSR is loaded on vmentry of nested guests:"If the “load IA32_PERF_GLOBAL_CTRL” VM-entry control is 1, the
IA32_PERF_GLOBAL_CTRLMSR is loaded from theIA32_PERF_GLOBAL_CTRLfield."Signed-off-by Krish Sadhukhan <[email protected]>Suggested-by Jim Mattson <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/vmx/nested.c +0
-0@@ -2300,6 +2300,10 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, vcpu->arch.cr0_guest_owned_bits &= ~vmcs12->cr0_guest_host_mask;vmcs_writel (CR0_GUEST_HOST_MASK , ~vcpu->arch.cr0_guest_owned_bits);Quoted 12 lines
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +4
-1+if (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ) +vmcs_write64 (GUEST_IA32_PERF_GLOBAL_CTRL , + vmcs12->guest_ia32_perf_global_ctrl); +if (vmx->nested.nested_run_pending && (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PAT )) {vmcs_write64 (GUEST_IA32_PAT , vmcs12->guest_ia32_pat); --2.17.2
This isn't quite right. The
GUEST_IA32_PERF_GLOBAL_CTRLvalue is just going to get overwritten later byatomic_switch_perf_msrs().
Quoted 3 lines
atomic_switch_perf_msrs()gest called fromvmx_vcpu_run()which I thought was executing beforehandle_vmlaunch()stuff that lead toprepare_vmcs02(). Did I miss something in the call-chain ?
Handle_vmlaunch is called on a VM-exit from vmcs01 as the result of a VMLAUNCH instruction. Atomic_switch_perf_msrs() is called on every VM-entry (to either vmcs01 or vmcs02).
Quoted 5 lines
Quoted 5 lines
Instead of writing the vmcs12 value directly into the vmcs02, you should call
kvm_set_msr(), exactly as it would have been called ifMSR_CORE_PERF_GLOBAL_CTRLhad been in the vmcs12 VM-entry MSR-load list. Then,atomic_switch_perf_msrs()will automatically do the right thing.
Re: [PATCH 6/8][KVM nVMX]: Load IA32_PERF_GLOBAL_CTRL MSR on vmentry of nested guests
In reply to Jim MattsonQuoted 37 lines · Jim Mattson
Quoted 30 lines · Krish Sadhukhan
According to section "Loading Guest State" in Intel SDM vol 3C, the
IA32_PERF_GLOBAL_CTRLMSR is loaded on vmentry of nested guests:"If the “load IA32_PERF_GLOBAL_CTRL” VM-entry control is 1, the
IA32_PERF_GLOBAL_CTRLMSR is loaded from theIA32_PERF_GLOBAL_CTRLfield."Signed-off-by Krish Sadhukhan <[email protected]>Suggested-by Jim Mattson <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/vmx/nested.c +4
-1@@ -2300,6 +2300,10 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, vcpu->arch.cr0_guest_owned_bits &= ~vmcs12->cr0_guest_host_mask;vmcs_writel (CR0_GUEST_HOST_MASK , ~vcpu->arch.cr0_guest_owned_bits); +if (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ) +vmcs_write64 (GUEST_IA32_PERF_GLOBAL_CTRL , + vmcs12->guest_ia32_perf_global_ctrl); +if (vmx->nested.nested_run_pending && (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PAT )) {vmcs_write64 (GUEST_IA32_PAT , vmcs12->guest_ia32_pat); --Quoted 2 lines
2.17.2
This isn't quite right. The
GUEST_IA32_PERF_GLOBAL_CTRLvalue is just going to get overwritten later byatomic_switch_perf_msrs().
Quoted 5 lines
Instead of writing the vmcs12 value directly into the vmcs02, you should call
kvm_set_msr(), exactly as it would have been called ifMSR_CORE_PERF_GLOBAL_CTRLhad been in the vmcs12 VM-entry MSR-load list. Then,atomic_switch_perf_msrs()will automatically do the right thing.
I notice that the existing code for VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL in load_vmcs12_host_state() doesn't use kvm_set_msr():
if (vmcs12->vm_exit_controls & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL) vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, vmcs12->host_ia32_perf_global_ctrl);
This should also be changed to use kvm_set_msr() then ?
Re: [PATCH 6/8][KVM nVMX]: Load IA32_PERF_GLOBAL_CTRL MSR on vmentry of nested guests
Quoted 50 lines · Krish Sadhukhan
Quoted 37 lines · Jim Mattson
Quoted 30 lines · Krish Sadhukhan
According to section "Loading Guest State" in Intel SDM vol 3C, the
IA32_PERF_GLOBAL_CTRLMSR is loaded on vmentry of nested guests:"If the “load IA32_PERF_GLOBAL_CTRL” VM-entry control is 1, the
IA32_PERF_GLOBAL_CTRLMSR is loaded from theIA32_PERF_GLOBAL_CTRLfield."Signed-off-by Krish Sadhukhan <[email protected]>Suggested-by Jim Mattson <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/nested.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/vmx/nested.c +4
-1@@ -2300,6 +2300,10 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, vcpu->arch.cr0_guest_owned_bits &= ~vmcs12->cr0_guest_host_mask;vmcs_writel (CR0_GUEST_HOST_MASK , ~vcpu->arch.cr0_guest_owned_bits); +if (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ) +vmcs_write64 (GUEST_IA32_PERF_GLOBAL_CTRL , + vmcs12->guest_ia32_perf_global_ctrl); +if (vmx->nested.nested_run_pending && (vmcs12->vm_entry_controls &VM_ENTRY_LOAD_IA32_PAT )) {vmcs_write64 (GUEST_IA32_PAT , vmcs12->guest_ia32_pat); --Quoted 2 lines
2.17.2
This isn't quite right. The
GUEST_IA32_PERF_GLOBAL_CTRLvalue is just going to get overwritten later byatomic_switch_perf_msrs().Quoted 5 lines
Instead of writing the vmcs12 value directly into the vmcs02, you should call
kvm_set_msr(), exactly as it would have been called ifMSR_CORE_PERF_GLOBAL_CTRLhad been in the vmcs12 VM-entry MSR-load list. Then,atomic_switch_perf_msrs()will automatically do the right thing.I notice that the existing code for
VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRLinload_vmcs12_host_state()doesn't usekvm_set_msr():
Quoted 6 lines
if (vmcs12->vm_exit_controls & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL) vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, vmcs12->host_ia32_perf_global_ctrl);This should also be changed to use
kvm_set_msr()then ?
Yes. The VMWRITE here is incorrect, but fortunately it is also unreachable.
PATCH 7/8 : Enable "load IA32_PERF_GLOBAL_CTRL VM-{entry,exit} controls
In reply to Krish Sadhukhan...based on whether the guest CPU supports PMU
--- arch/x86/kvm/vmx/vmx.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/x86/kvm/vmx/vmx.c +12 -0
@@ -6964,6 +6964,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)static void nested_vmx_entry_exit_ctls_update (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +bool pmu_enabled =guest_cpuid_has_pmu (vcpu);if (kvm_mpx_supported ()) {bool mpx_enabled =guest_cpuid_has (vcpu,X86_FEATURE_MPX );
@@ -6976,6 +6977,17 @@ static void nested_vmx_entry_exit_ctls_update(struct kvm_vcpu *vcpu) vmx->nested.msrs.exit_ctls_high &= ~VM_EXIT_CLEAR_BNDCFGS ; } } +if (pmu_enabled) { + vmx->nested.msrs.entry_ctls_high |= +VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ; + vmx->nested.msrs.exit_ctls_high |= +VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL ; + }else { + vmx->nested.msrs.entry_ctls_high &= + ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ; + vmx->nested.msrs.exit_ctls_high &= + ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL ; + } }static void update_intel_pt_cfg (struct kvm_vcpu *vcpu)
Signature · 1 line
2.17.2
Re: [PATCH 7/8][KVM nVMX]: Enable "load IA32_PERF_GLOBAL_CTRL VM-{entry,exit} controls
Quoted 14 lines · Krish Sadhukhan
...based on whether the guest CPU supports PMU
Signed-off-by Krish Sadhukhan <[email protected]>Suggested-by Jim Mattson <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)arch/x86/kvm/vmx/vmx.c +0
-0
arch/x86/kvm/vmx/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 6967
Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +1
-0@@ -6964,6 +6964,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)static void nested_vmx_entry_exit_ctls_update (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +bool pmu_enabled =guest_cpuid_has_pmu (vcpu);
A revert has been sent for the patch that added guest_cpuid_has_pmu().
Regardless, checking only the guest's CPUID 0xA is not sufficient, e.g. at the bare minimum, exposing the controls can be done if and only if cpu_has_load_perf_global_ctrl() is true.
In general, it's difficult for me to understand exactly what functionality you intend to introduce. Proper changelogs would be very helpful.
Quoted 24 lines
arch/x86/kvm/vmx/vmx.c +11
-0if (kvm_mpx_supported ()) {bool mpx_enabled =guest_cpuid_has (vcpu,X86_FEATURE_MPX );@@ -6976,6 +6977,17 @@ static void nested_vmx_entry_exit_ctls_update(struct kvm_vcpu *vcpu) vmx->nested.msrs.exit_ctls_high &= ~VM_EXIT_CLEAR_BNDCFGS ; } } +if (pmu_enabled) { + vmx->nested.msrs.entry_ctls_high |= +VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ; + vmx->nested.msrs.exit_ctls_high |= +VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL ; + }else { + vmx->nested.msrs.entry_ctls_high &= + ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ; + vmx->nested.msrs.exit_ctls_high &= + ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL ; + } }static void update_intel_pt_cfg (struct kvm_vcpu *vcpu)Signature · 1 line
2.17.2
Re: [PATCH 7/8][KVM nVMX]: Enable "load IA32_PERF_GLOBAL_CTRL VM-{entry,exit} controls
Quoted 23 lines · Sean Christopherson
Quoted 14 lines · Krish Sadhukhan
...based on whether the guest CPU supports PMU
Signed-off-by Krish Sadhukhan <[email protected]>Suggested-by Jim Mattson <[email protected]>Reviewed-by Karl Heubaum <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)arch/x86/kvm/vmx/vmx.c +0
-0arch/x86/kvm/vmx/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 6967
Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +1
-0@@ -6964,6 +6964,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)static void nested_vmx_entry_exit_ctls_update (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +bool pmu_enabled =guest_cpuid_has_pmu (vcpu);A revert has been sent for the patch that added
guest_cpuid_has_pmu().
Quoted 3 lines
Regardless, checking only the guest's CPUID
0xAis not sufficient, e.g. at the bare minimum, exposing the controls can be done if and only ifcpu_has_load_perf_global_ctrl()is true.
I don't think cpu_has_load_perf_global_ctrl() is required. Support for the VM-entry and VM-exit controls, "load IA32_PERF_GLOBAL_CTRL," can be completely emulated by kvm, since add_atomic_switch_msr() is capable of falling back on the VM-entry and VM-exit MSR-load lists if !cpu_has_load_perf_global_ctrl().
The only requirement should be kvm_pmu_is_valid_msr(MSR_CORE_PERF_GLOBAL_CTRL).
Quoted 2 lines
In general, it's difficult for me to understand exactly what functionality you intend to introduce. Proper changelogs would be very helpful.
I concur!
PATCH 8/8 : Test "load IA32_PERF_GLOBAL_CTRL" controls on vmentry of nested guests
In reply to Krish Sadhukhan ...to verify KVM performs the appropriate consistency checks for loadingIA32_PERF_GLOBAL_CTRL as part of running a nested guest.
According to section "Checks on Host Control Registers and MSRs" in Intel SDM vol 3C, the following checks are performed on vmentry of nested guests:
If the "load IA32_PERF_GLOBAL_CTRL" VM-exit control is 1, bits reserved in the IA32_PERF_GLOBAL_CTRL MSR must be 0 in the field for that register.
--- x86/vmx_tests.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+)
x86/vmx_tests.c +59 -0
@@ -4995,6 +4995,63 @@ static void test_sysenter_field(u32 field, const char *name)vmcs_write (field, addr_saved); } +#define PERF_GLOBAL_CTRL_VALID_BITS 0x0000000700000003 + +static void test_perf_global_ctl (u32 field,const char * field_name, +u32 ctrl_field,u64 ctrl_bit) +{ +u32 ctrl_saved =vmcs_read (ctrl_field); +u64 perf_global_saved =vmcs_read (field); +u64 i, val; + +vmcs_write (ctrl_field, ctrl_saved & ~ctrl_bit); +for (i =0 ; i <64 ; i++) { + val =1ull << i; +vmcs_write (field, val); +report_prefix_pushf ("\"load IA32_PERF_GLOBAL_CTRL\" " +"VM-exit control is off, HOST_PERF_GLOBAL_CTRL %lx" , val); +test_vmx_vmlaunch (0 , false); +report_prefix_pop (); + } + +vmcs_write (ctrl_field, ctrl_saved | ctrl_bit); +for (i =0 ; i <64 ; i++) { + val =1ull << i; +vmcs_write (field, val); +report_prefix_pushf ("\"load IA32_PERF_GLOBAL_CTRL\" " +"VM-exit control is on, HOST_PERF_GLOBAL_CTRL %lx" , val); +if (PERF_GLOBAL_CTRL_VALID_BITS & (1ull << i)) { +test_vmx_vmlaunch (0 , false); + }else { +test_vmx_vmlaunch (VMXERR_ENTRY_INVALID_HOST_STATE_FIELD , + false); + } +report_prefix_pop (); + } + +vmcs_write (ctrl_field, ctrl_saved); +vmcs_write (field, perf_global_saved); +} + +/* +* If the "load IA32_PERF_GLOBAL_CTRL" VM-exit control is 1, bits reserved +* in the IA32_PERF_GLOBAL_CTRL MSR must be 0 in the field for that +* register. +* +* [Intel SDM] +*/ +static void test_host_perf_global_ctl (void ) +{ +if (!(ctrl_exit_rev.clr &EXI_LOAD_PERF )) { +printf ("\"load IA32_PERF_GLOBAL_CTRL\" VM-exit control not " +"supported\n" ); +return ; + } + +test_perf_global_ctl (HOST_PERF_GLOBAL_CTRL ,"HOST_PERF_GLOBAL_CTRL" , +EXI_CONTROLS ,EXI_LOAD_PERF ); +} +/* * Check that the virtual CPU checks the VMX Host State Area as * documented in the Intel SDM.
@@ -5010,6 +5067,8 @@ static void vmx_host_state_area_test(void)test_sysenter_field (HOST_SYSENTER_ESP ,"HOST_SYSENTER_ESP" );test_sysenter_field (HOST_SYSENTER_EIP ,"HOST_SYSENTER_EIP" ); + +test_host_perf_global_ctl (); }static bool valid_vmcs_for_vmentry (void )
Signature · 1 line
2.17.2
Quoted 8 lines · Krish Sadhukhan
Patch 1 through 7 add the necessary KVM code for checking and enabling "load
IA32_PERF_GLOBAL_CTRL" VM-{exit,entry} controls.Patch# 8 adds a unit test for the "load
IA32_PERF_GLOBAL_CTRL" VM-exit control. I will send a separate patch for the unit test for "loadIA32_PERF_GLOBAL_CTRL" VM-entry control.
Quoted 8 lines
[PATCH 1/8][KVMnVMX]: Enable "load
IA32_PERF_GLOBAL_CTRL" VM-exit control [PATCH 2/8][KVM nVMX]: Enable "loadIA32_PERF_GLOBAL_CTRL" VM-entry [PATCH 3/8][KVM VMX]: Add a function to check reserved bits in [PATCH 4/8][KVM nVMX]: Check "loadIA32_PERF_GLOBAL_CTRL" VM-exit control [PATCH 5/8][KVM nVMX]: Check "loadIA32_PERF_GLOBAL_CTRL" VM-entry control [PATCH 6/8][KVM nVMX]: LoadIA32_PERF_GLOBAL_CTRLMSR on vmentry of nested [PATCH 7/8][KVM nVMX]: Enable "loadIA32_PERF_GLOBAL_CTRLVM-{entry,exit} [PATCH 8/8][KVM nVMX]: Test "loadIA32_PERF_GLOBAL_CTRL" controls on vmentry
These subjects are wrong, should be:
[PATCH n/8] KVM: nVMX: Blah blah blah
or something along those lines. Wrapping the scope in square braces causes the scope to be dropped when the patch is applied. They're also different from the diffstat, which is odd.
And nearly all of the patches are missing proper changelogs.
Quoted 23 lines
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/asm/msr-index.h | 7 +++++++ arch/x86/kvm/vmx/nested.c | 19 +++++++++++++++++-- arch/x86/kvm/vmx/vmx.c | 12 ++++++++++++ arch/x86/kvm/x86.c | 20 ++++++++++++++++++++ 5 files changed, 57 insertions(+), 2 deletions(-)Krish Sadhukhan (7):
nVMX: Enable "loadIA32_PERF_GLOBAL_CTRL" VM-exit control for nested guests
nVMX: Enable "loadIA32_PERF_GLOBAL_CTRL" VM-entry control for nested guests
VMX: Add a function to check reserved bits inMSR_CORE_PERF_GLOBAL_CTRL
nVMX: Check "loadIA32_PERF_GLOBAL_CTRL" VM-exit control on vmentry of nested guests
nVMX: Check "loadIA32_PERF_GLOBAL_CTRL" VM-entry control on vmentry of nested guests
nVMX: LoadIA32_PERF_GLOBAL_CTRLMSR on vmentry of nested guests
nVMX: Enable "loadIA32_PERF_GLOBAL_CTRLVM-{entry,exit} controlsx86/vmx_tests.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+)Krish Sadhukhan (1):
nVMX: Test "loadIA32_PERF_GLOBAL_CTRL" controls on vmentry of nested guests