KVM: selftests: AVX support + fixes
Sean Christopherson
reviewercover KVM: selftests: AVX support + fixes
Enable CR4.OSXSAVE and XCR0.AVX by default when creating selftests vCPUs in order to play nice with compilers that have been configured to enable -march=x86-64-v3 by default.
While it would be easier to force v2 (or earlier), there are enough tests that want XCR0 configured that it will (hopefully) be a net postive to enable all XCR0 features by default.
The only real hiccup is the CR4/CPUID sync test, which disables CR4.OSXSAVE to verify KVM toggles the associated CPUID bit. And if it calls memset() while OSXAVE is disabled, kablooie. Fixing that requires a bit of assembly, but overall I think it's worth carrying a few lines of assembly in order to gain test coverage for running AVX instructions in guests, and boy are compilers good at abusing AVX :-)
Fix a few bugs/warts found along the way. Notably, the CPUID test has an array out-of-bounds bug that can result in false passes (I only noticed because it was getting a false pass on gcc).
Sean Christopherson (11):
KVM: selftests: Fix out-of-bounds reads in CPUID test's array lookups
KVM: selftests: Precisely mask off dynamic fields in CPUID test
KVM: selftests: Mask off OSPKE and OSXSAVE when comparing CPUID
entries
KVM: selftests: Rework OSXSAVE CR4=>CPUID test to play nice with AVX
insns
KVM: selftests: Configure XCR0 to max supported value by default
KVM: selftests: Verify XCR0 can be "downgraded" and "upgraded"
KVM: selftests: Drop manual CR4.OSXSAVE enabling from CR4/CPUID sync
test
KVM: selftests: Drop manual XCR0 configuration from AMX test
KVM: selftests: Drop manual XCR0 configuration from state test
KVM: selftests: Drop manual XCR0 configuration from SEV smoke test
KVM: selftests: Ensure KVM supports AVX for SEV-ES VMSA FPU test
.../selftests/kvm/include/x86_64/processor.h | 5 ++ .../selftests/kvm/lib/x86_64/processor.c | 24 +++++++ tools/testing/selftests/kvm/x86_64/amx_test.c | 23 ++----- .../testing/selftests/kvm/x86_64/cpuid_test.c | 67 ++++++++++++------- .../kvm/x86_64/cr4_cpuid_sync_test.c | 53 +++++++++------ .../selftests/kvm/x86_64/sev_smoke_test.c | 19 ++---- .../testing/selftests/kvm/x86_64/state_test.c | 5 -- .../selftests/kvm/x86_64/xcr0_cpuid_test.c | 11 ++- 8 files changed, 122 insertions(+), 85 deletions(-)
base-commit: efbc6bd090f48ccf64f7a8dd5daea775821d57ec
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Sean Christopherson
reviewerPATCH 1/11 KVM: selftests: Fix out-of-bounds reads in CPUID test's array lookups
When looking for a "mangled", i.e. dynamic, CPUID entry, terminate the walk based on the number of array _entries_, not the size in bytes of the array. Iterating based on the total size of the array can result in false passes, e.g. if the random data beyond the array happens to match a CPUID entry's function and index.
--- tools/testing/selftests/kvm/x86_64/cpuid_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/x86_64/cpuid_test.c +1 -1
@@ -60,7 +60,7 @@ static bool is_cpuid_mangled(const struct kvm_cpuid_entry2 *entrie) {int i; -for (i =0 ; i <sizeof (mangled_cpuids); i++) { +for (i =0 ; i <ARRAY_SIZE (mangled_cpuids); i++) {if (mangled_cpuids[i].function == entrie->function && mangled_cpuids[i].index == entrie->index)return true;
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 01/11] KVM: selftests: Fix out-of-bounds reads in CPUID test's array lookups
Quoted 17 lines · Sean Christopherson
When looking for a "mangled", i.e. dynamic, CPUID entry, terminate the walk based on the number of array _entries_, not the size in bytes of the array. Iterating based on the total size of the array can result in false passes, e.g. if the random data beyond the array happens to match a CPUID entry's function and index.
Fixes fb18d053b7f8 ("selftest: kvm: x86: test KVM_GET_CPUID2 and guest visible CPUIDs against KVM_GET_SUPPORTED_CPUID")Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/x86_64/cpuid_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/testing/selftests/kvm/x86_64/cpuid_test.c +0
-0
tools/testing/selftests/kvm/x86_64/cpuid_test.c static bool is_cpuid_mangled(const struct kvm_cpuid_entry2 *entrie) · line 66
Quoted 8 lines
tools/testing/selftests/kvm/x86_64/cpuid_test.c +1
-1@@ -60,7 +60,7 @@ static bool is_cpuid_mangled(const struct kvm_cpuid_entry2 *entrie) {int i; -for (i =0 ; i <sizeof (mangled_cpuids); i++) { +for (i =0 ; i <ARRAY_SIZE (mangled_cpuids); i++) {if (mangled_cpuids[i].function == entrie->function && mangled_cpuids[i].index == entrie->index)return true;
Signature · 1 line
Vitaly
Sean Christopherson
reviewerPATCH 2/11 KVM: selftests: Precisely mask off dynamic fields in CPUID test
In reply to Sean ChristophersonWhen comparing vCPU CPUID entries against KVM's supported CPUID, mask off only the dynamic fields/bits instead of skipping the entire entry. Precisely masking bits isn't meaningfully more difficult than skipping entire entries, and will be necessary to maintain test coverage when a future commit enables OSXSAVE by default, i.e. makes one bit in all of CPUID.0x1 dynamic.
--- .../testing/selftests/kvm/x86_64/cpuid_test.c | 61 +++++++++++-------- 1 file changed, 36 insertions(+), 25 deletions(-)
tools/testing/selftests/kvm/x86_64/cpuid_test.c +36 -25
@@ -12,17 +12,16 @@#include "kvm_util.h" #include "processor.h" -/* CPUIDs known to differ */ -struct { -u32 function; -u32 index; -} mangled_cpuids[] = { -/* -* These entries depend on the vCPU's XCR0 register and IA32_XSS MSR, -* which are not controlled for by this test. -*/ - {.function =0xd , .index =0 }, - {.function =0xd , .index =1 }, +struct cpuid_mask { +union { +struct { +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; + }; +u32 regs[4 ]; + }; };static void test_guest_cpuids (struct kvm_cpuid2 *guest_cpuid)
@@ -56,17 +55,23 @@ static void guest_main(struct kvm_cpuid2 *guest_cpuid)GUEST_DONE (); } -static bool is_cpuid_mangled (const struct kvm_cpuid_entry2 *entrie) +static struct cpuid_mask get_const_cpuid_mask (const struct kvm_cpuid_entry2 *entry) { -int i; +struct cpuid_mask mask; -for (i =0 ; i <ARRAY_SIZE (mangled_cpuids); i++) { -if (mangled_cpuids[i].function == entrie->function && - mangled_cpuids[i].index == entrie->index) -return true; +memset (&mask,0xff ,sizeof (mask)); + +switch (entry->function) { +case 0xd : +/* +* CPUID.0xD.{0,1}.EBX enumerate XSAVE size based on the current +* XCR0 and IA32_XSS MSR values. +*/ +if (entry->index <2 ) + mask.ebx =0 ; +break ; } - -return false; +return mask; } static void compare_cpuids(const struct kvm_cpuid2 *cpuid1,
@@ -79,6 +84,8 @@ static void compare_cpuids(const struct kvm_cpuid2 *cpuid1, "CPUID nent mismatch: %d vs. %d", cpuid1->nent, cpuid2->nent);for (i =0 ; i < cpuid1->nent; i++) { +struct cpuid_mask mask; + e1 = &cpuid1->entries[i]; e2 = &cpuid2->entries[i];
@@ -88,15 +95,19 @@ static void compare_cpuids(const struct kvm_cpuid2 *cpuid1, i, e1->function, e1->index, e1->flags, e2->function, e2->index, e2->flags); -if (is_cpuid_mangled (e1)) -continue ; +/* Mask off dynamic bits, e.g. OSXSAVE, when comparing entries. */ + mask =get_const_cpuid_mask (e1); -TEST_ASSERT (e1->eax == e2->eax && e1->ebx == e2->ebx && - e1->ecx == e2->ecx && e1->edx == e2->edx, +TEST_ASSERT ((e1->eax & mask.eax) == (e2->eax & mask.eax) && + (e1->ebx & mask.ebx) == (e2->ebx & mask.ebx) && + (e1->ecx & mask.ecx) == (e2->ecx & mask.ecx) && + (e1->edx & mask.edx) == (e2->edx & mask.edx),"CPUID 0x%x.%x differ: 0x%x:0x%x:0x%x:0x%x vs 0x%x:0x%x:0x%x:0x%x" , e1->function, e1->index, - e1->eax, e1->ebx, e1->ecx, e1->edx, - e2->eax, e2->ebx, e2->ecx, e2->edx); + e1->eax & mask.eax, e1->ebx & mask.ebx, + e1->ecx & mask.ecx, e1->edx & mask.edx, + e2->eax & mask.eax, e2->ebx & mask.ebx, + e2->ecx & mask.ecx, e2->edx & mask.edx); } }
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 02/11] KVM: selftests: Precisely mask off dynamic fields in CPUID test
Quoted 100 lines · Sean Christopherson
When comparing vCPU CPUID entries against KVM's supported CPUID, mask off only the dynamic fields/bits instead of skipping the entire entry. Precisely masking bits isn't meaningfully more difficult than skipping entire entries, and will be necessary to maintain test coverage when a future commit enables OSXSAVE by default, i.e. makes one bit in all of CPUID.0x1 dynamic.
Signed-off-by Sean Christopherson <[email protected]>--- .../testing/selftests/kvm/x86_64/cpuid_test.c | 61 +++++++++++-------- 1 file changed, 36 insertions(+), 25 deletions(-)tools/testing/selftests/kvm/x86_64/cpuid_test.c +32
-23@@ -12,17 +12,16 @@#include "kvm_util.h" #include "processor.h" -/* CPUIDs known to differ */ -struct { -u32 function; -u32 index; -} mangled_cpuids[] = { -/* -* These entries depend on the vCPU's XCR0 register and IA32_XSS MSR, -* which are not controlled for by this test. -*/ - {.function =0xd , .index =0 }, - {.function =0xd , .index =1 }, +struct cpuid_mask { +union { +struct { +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; + }; +u32 regs[4 ]; + }; };static void test_guest_cpuids (struct kvm_cpuid2 *guest_cpuid)@@ -56,17 +55,23 @@ static void guest_main(struct kvm_cpuid2 *guest_cpuid)GUEST_DONE (); } -static bool is_cpuid_mangled (const struct kvm_cpuid_entry2 *entrie) +static struct cpuid_mask get_const_cpuid_mask (const struct kvm_cpuid_entry2 *entry) { -int i; +struct cpuid_mask mask; -for (i =0 ; i <ARRAY_SIZE (mangled_cpuids); i++) { -if (mangled_cpuids[i].function == entrie->function && - mangled_cpuids[i].index == entrie->index) -return true; +memset (&mask,0xff ,sizeof (mask)); + +switch (entry->function) { +case 0xd : +/* +* CPUID.0xD.{0,1}.EBX enumerate XSAVE size based on the current +* XCR0 and IA32_XSS MSR values. +*/ +if (entry->index <2 ) + mask.ebx =0 ; +break ; } - -return false; +return mask; } static void compare_cpuids(const struct kvm_cpuid2 *cpuid1,@@ -79,6 +84,8 @@ static void compare_cpuids(const struct kvm_cpuid2 *cpuid1, "CPUID nent mismatch: %d vs. %d", cpuid1->nent, cpuid2->nent);for (i =0 ; i < cpuid1->nent; i++) { +struct cpuid_mask mask; + e1 = &cpuid1->entries[i]; e2 = &cpuid2->entries[i];@@ -88,15 +95,19 @@ static void compare_cpuids(const struct kvm_cpuid2 *cpuid1, i, e1->function, e1->index, e1->flags, e2->function, e2->index, e2->flags); -if (is_cpuid_mangled (e1)) -continue ; +/* Mask off dynamic bits, e.g. OSXSAVE, when comparing entries. */ + mask =get_const_cpuid_mask (e1); - TEST_ASSERT(e1->eax == e2->eax && e1->ebx == e2->ebx && - e1->ecx == e2->ecx && e1->edx == e2->edx, + TEST_ASSERT((e1->eax & mask.eax) == (e2->eax & mask.eax) && + (e1->ebx & mask.ebx) == (e2->ebx & mask.ebx) && + (e1->ecx & mask.ecx) == (e2->ecx & mask.ecx) && + (e1->edx & mask.edx) == (e2->edx & mask.edx),
tools/testing/selftests/kvm/x86_64/cpuid_test.c static void compare_cpuids(const struct kvm_cpuid2 *cpuid1, · line 112
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/cpuid_test.c +4
-2"CPUID 0x%x.%x differ: 0x%x:0x%x:0x%x:0x%x vs 0x%x:0x%x:0x%x:0x%x" , e1->function, e1->index, - e1->eax, e1->ebx, e1->ecx, e1->edx, - e2->eax, e2->ebx, e2->ecx, e2->edx); + e1->eax & mask.eax, e1->ebx & mask.ebx, + e1->ecx & mask.ecx, e1->edx & mask.edx, + e2->eax & mask.eax, e2->ebx & mask.ebx, + e2->ecx & mask.ecx, e2->edx & mask.edx); } }
Signature · 1 line
Vitaly
Sean Christopherson
reviewerPATCH 3/11 KVM: selftests: Mask off OSPKE and OSXSAVE when comparing CPUID entries
In reply to Sean ChristophersonMask off OSPKE and OSXSAVE, which are toggled based on corresponding CR4 enabling bits, when comparing vCPU CPUID against KVM's supported CPUID. This will allow setting OSXSAVE by default when creating vCPUs, without causing test failures (KVM doesn't enumerate OSXSAVE=1).
--- tools/testing/selftests/kvm/x86_64/cpuid_test.c | 6 ++++++ 1 file changed, 6 insertions(+)
tools/testing/selftests/kvm/x86_64/cpuid_test.c +6 -0
@@ -62,6 +62,12 @@ static struct cpuid_mask get_const_cpuid_mask(const struct kvm_cpuid_entry2 *entmemset (&mask,0xff ,sizeof (mask));switch (entry->function) { +case 0x1 : + mask.regs[X86_FEATURE_OSXSAVE .reg] &= ~BIT (X86_FEATURE_OSXSAVE .bit); +break ; +case 0x7 : + mask.regs[X86_FEATURE_OSPKE .reg] &= ~BIT (X86_FEATURE_OSPKE .bit); +break ;case 0xd :/* * CPUID.0xD.{0,1}.EBX enumerate XSAVE size based on the current
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 03/11] KVM: selftests: Mask off OSPKE and OSXSAVE when comparing CPUID entries
Quoted 17 lines · Sean Christopherson
Mask off OSPKE and OSXSAVE, which are toggled based on corresponding CR4 enabling bits, when comparing vCPU CPUID against KVM's supported CPUID. This will allow setting OSXSAVE by default when creating vCPUs, without causing test failures (KVM doesn't enumerate OSXSAVE=1).
Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/x86_64/cpuid_test.c | 6 ++++++ 1 file changed, 6 insertions(+)tools/testing/selftests/kvm/x86_64/cpuid_test.c +0
-0@@ -62,6 +62,12 @@ static struct cpuid_mask get_const_cpuid_mask(const struct kvm_cpuid_entry2 *entmemset (&mask,0xff ,sizeof (mask));
tools/testing/selftests/kvm/x86_64/cpuid_test.c static struct cpuid_mask get_const_cpuid_mask(const struct kvm_cpuid_entry2 *ent · line 73
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/cpuid_test.c +6
-0switch (entry->function) { +case 0x1 : + mask.regs[X86_FEATURE_OSXSAVE .reg] &= ~BIT (X86_FEATURE_OSXSAVE .bit); +break ; +case 0x7 : + mask.regs[X86_FEATURE_OSPKE .reg] &= ~BIT (X86_FEATURE_OSPKE .bit); +break ;case 0xd :/* * CPUID.0xD.{0,1}.EBX enumerate XSAVE size based on the current
Signature · 1 line
Vitaly
Sean Christopherson
reviewerPATCH 4/11 KVM: selftests: Rework OSXSAVE CR4=>CPUID test to play nice with AVX insns
In reply to Sean ChristophersonRework the CR4/CPUID sync test to clear CR4.OSXSAVE, do CPUID, and restore CR4.OSXSAVE in assembly, so that there is zero chance of AVX instructions being executed while CR4.OSXSAVE is disabled. This will allow enabling CR4.OSXSAVE by default for selftests vCPUs as a general means of playing nice with AVX instructions.
--- .../kvm/x86_64/cr4_cpuid_sync_test.c | 46 +++++++++++++------ 1 file changed, 32 insertions(+), 14 deletions(-)
tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c +32 -14
@@ -19,15 +19,14 @@#include "kvm_util.h" #include "processor.h" -static inline bool cr4_cpuid_is_sync (void ) -{ -uint64_t cr4 =get_cr4 (); - -return (this_cpu_has (X86_FEATURE_OSXSAVE ) == !!(cr4 &X86_CR4_OSXSAVE )); -} +#define MAGIC_HYPERCALL_PORT 0x80static void guest_code (void ) { +u32 regs[4 ] = { + [KVM_CPUID_EAX ] =X86_FEATURE_OSXSAVE .function, + [KVM_CPUID_ECX ] =X86_FEATURE_OSXSAVE .index, + };uint64_t cr4;/* turn on CR4.OSXSAVE */
@@ -36,13 +35,29 @@ static void guest_code(void)set_cr4 (cr4);/* verify CR4.OSXSAVE == CPUID.OSXSAVE */ -GUEST_ASSERT (cr4_cpuid_is_sync ()); +GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE )); -/* notify hypervisor to change CR4 */ -GUEST_SYNC (0 ); +/* +* Notify hypervisor to clear CR4.0SXSAVE, do CPUID and save output, +* and then restore CR4. Do this all in assembly to ensure no AVX +* instructions are executed while OSXSAVE=0. +*/ + asmvolatile ( + "out %%al, $" __stringify(MAGIC_HYPERCALL_PORT) "\n\t" + "cpuid\n\t" + "mov %%rdi, %%cr4\n\t" + : "+a" (regs[KVM_CPUID_EAX]), + "=b" (regs[KVM_CPUID_EBX]), + "+c" (regs[KVM_CPUID_ECX]), + "=d" (regs[KVM_CPUID_EDX]) + : "D" (get_cr4()) + ); -/* check again */ -GUEST_ASSERT (cr4_cpuid_is_sync ()); +/* Verify KVM cleared OSXSAVE in CPUID when it was cleared in CR4. */ +GUEST_ASSERT (!(regs[X86_FEATURE_OSXSAVE .reg] &BIT (X86_FEATURE_OSXSAVE .bit))); + +/* Verify restoring CR4 also restored OSXSAVE in CPUID. */ +GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE ));GUEST_DONE (); }
@@ -62,13 +77,16 @@ int main(int argc, char *argv[])vcpu_run (vcpu);TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); -switch (get_ucall (vcpu, &uc)) { -case UCALL_SYNC : +if (vcpu->run->io.port ==MAGIC_HYPERCALL_PORT && + vcpu->run->io.direction ==KVM_EXIT_IO_OUT ) {/* emulate hypervisor clearing CR4.OSXSAVE */ vcpu_sregs_get (vcpu, &sregs); sregs.cr4 &= ~X86_CR4_OSXSAVE ;vcpu_sregs_set (vcpu, &sregs); -break ; +continue ; + } + +switch (get_ucall (vcpu, &uc)) {case UCALL_ABORT :REPORT_GUEST_ASSERT (uc);break ;
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 04/11] KVM: selftests: Rework OSXSAVE CR4=>CPUID test to play nice with AVX insns
Quoted 81 lines · Sean Christopherson
Rework the CR4/CPUID sync test to clear CR4.OSXSAVE, do CPUID, and restore CR4.OSXSAVE in assembly, so that there is zero chance of AVX instructions being executed while CR4.OSXSAVE is disabled. This will allow enabling CR4.OSXSAVE by default for selftests vCPUs as a general means of playing nice with AVX instructions.
Signed-off-by Sean Christopherson <[email protected]>--- .../kvm/x86_64/cr4_cpuid_sync_test.c | 46 +++++++++++++------ 1 file changed, 32 insertions(+), 14 deletions(-)tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c +28
-13@@ -19,15 +19,14 @@#include "kvm_util.h" #include "processor.h" -static inline bool cr4_cpuid_is_sync (void ) -{ -uint64_t cr4 =get_cr4 (); - -return (this_cpu_has (X86_FEATURE_OSXSAVE ) == !!(cr4 &X86_CR4_OSXSAVE )); -} +#define MAGIC_HYPERCALL_PORT 0x80static void guest_code (void ) { +u32 regs[4 ] = { + [KVM_CPUID_EAX ] =X86_FEATURE_OSXSAVE .function, + [KVM_CPUID_ECX ] =X86_FEATURE_OSXSAVE .index, + };uint64_t cr4;/* turn on CR4.OSXSAVE */ @@ -36,13 +35,29 @@ static void guest_code(void)set_cr4 (cr4);/* verify CR4.OSXSAVE == CPUID.OSXSAVE */ -GUEST_ASSERT (cr4_cpuid_is_sync ()); +GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE )); -/* notify hypervisor to change CR4 */ -GUEST_SYNC (0 ); +/* +* Notify hypervisor to clear CR4.0SXSAVE, do CPUID and save output, +* and then restore CR4. Do this all in assembly to ensure no AVX +* instructions are executed while OSXSAVE=0. +*/ + asmvolatile ( + "out %%al, $" __stringify(MAGIC_HYPERCALL_PORT) "\n\t" + "cpuid\n\t" + "mov %%rdi, %%cr4\n\t" + : "+a" (regs[KVM_CPUID_EAX]), + "=b" (regs[KVM_CPUID_EBX]), + "+c" (regs[KVM_CPUID_ECX]), + "=d" (regs[KVM_CPUID_EDX]) + : "D" (get_cr4()) + ); -/* check again */ -GUEST_ASSERT (cr4_cpuid_is_sync ()); +/* Verify KVM cleared OSXSAVE in CPUID when it was cleared in CR4. */ +GUEST_ASSERT (!(regs[X86_FEATURE_OSXSAVE .reg] &BIT (X86_FEATURE_OSXSAVE .bit))); + +/* Verify restoring CR4 also restored OSXSAVE in CPUID. */ +GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE ));GUEST_DONE (); }@@ -62,13 +77,16 @@ int main(int argc, char *argv[])vcpu_run (vcpu);TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); -switch (get_ucall (vcpu, &uc)) { -case UCALL_SYNC : +if (vcpu->run->io.port ==MAGIC_HYPERCALL_PORT && + vcpu->run->io.direction ==KVM_EXIT_IO_OUT ) {/* emulate hypervisor clearing CR4.OSXSAVE */ vcpu_sregs_get (vcpu, &sregs);
tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c int main(int argc, char *argv[]) · line 92
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c +4
-1sregs.cr4 &= ~X86_CR4_OSXSAVE ;vcpu_sregs_set (vcpu, &sregs); -break ; +continue ; + } + +switch (get_ucall (vcpu, &uc)) {case UCALL_ABORT :REPORT_GUEST_ASSERT (uc);break ;
Signature · 1 line
Vitaly
Sean Christopherson
reviewerPATCH 5/11 KVM: selftests: Configure XCR0 to max supported value by default
In reply to Sean ChristophersonTo play nice with compilers generating AVX instructions, set CR4.OSXSAVE and configure XCR0 by default when creating selftests vCPUs. Some distros have switched gcc to '-march=x86-64-v3' by default, and while it's hard to find a CPU which doesn't support AVX today, many KVM selftests fail with
==== Test Assertion Failure ====
lib/x86_64/processor.c:570: Unhandled exception in guest
pid=72747 tid=72747 errno=4 - Interrupted system call
Unhandled exception '0x6' at guest RIP '0x4104f7'due to selftests not enabling AVX by default for the guest. The failure is easy to reproduce elsewhere with:
$ make clean && CFLAGS='-march=x86-64-v3' make -j && ./x86_64/kvm_pv_test
E.g. gcc-13 with -march=x86-64-v3 compiles this chunk from selftests' kvm_fixup_exception():
regs->rip = regs->r11;
regs->r9 = regs->vector;
regs->r10 = regs->error_code;into this monstronsity (which is clever, but oof):
405313: c4 e1 f9 6e c8 vmovq %rax,%xmm1 405318: 48 89 68 08 mov %rbp,0x8(%rax) 40531c: 48 89 e8 mov %rbp,%rax 40531f: c4 c3 f1 22 c4 01 vpinsrq $0x1,%r12,%xmm1,%xmm0 405325: 49 89 6d 38 mov %rbp,0x38(%r13) 405329: c5 fa 7f 45 00 vmovdqu %xmm0,0x0(%rbp)
Alternatively, KVM selftests could explicitly restrict the compiler to -march=x86-64-v2, but odds are very good that punting on AVX enabling will simply result in tests that "need" AVX doing their own thing, e.g. there are already three or so additional cleanups that can be done on top.
--- .../selftests/kvm/include/x86_64/processor.h | 5 ++++ .../selftests/kvm/lib/x86_64/processor.c | 24 +++++++++++++++++++ .../selftests/kvm/x86_64/xcr0_cpuid_test.c | 6 ++--- 3 files changed, 32 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/include/x86_64/processor.h +5 -0
@@ -1049,6 +1049,11 @@ static inline void vcpu_set_cpuid(struct kvm_vcpu *vcpu)vcpu_ioctl (vcpu,KVM_GET_CPUID2 , vcpu->cpuid); } +static inline void vcpu_get_cpuid (struct kvm_vcpu *vcpu) +{ +vcpu_ioctl (vcpu,KVM_GET_CPUID2 , vcpu->cpuid); +} +void vcpu_set_cpuid_property (struct kvm_vcpu *vcpu,struct kvm_x86_cpu_property property,uint32_t value);
tools/testing/selftests/kvm/lib/x86_64/processor.c +24 -0
@@ -506,6 +506,8 @@ static void vcpu_init_sregs(struct kvm_vm *vm, struct kvm_vcpu *vcpu) sregs.cr0 =X86_CR0_PE |X86_CR0_NE |X86_CR0_PG ; sregs.cr4 |=X86_CR4_PAE |X86_CR4_OSFXSR ; +if (kvm_cpu_has (X86_FEATURE_XSAVE )) + sregs.cr4 |=X86_CR4_OSXSAVE ; sregs.efer |= (EFER_LME |EFER_LMA |EFER_NX );kvm_seg_set_unusable (&sregs.ldt);
@@ -519,6 +521,20 @@ static void vcpu_init_sregs(struct kvm_vm *vm, struct kvm_vcpu *vcpu)vcpu_sregs_set (vcpu, &sregs); } +static void vcpu_init_xcrs (struct kvm_vm *vm,struct kvm_vcpu *vcpu) +{ +struct kvm_xcrs xcrs = { + .nr_xcrs =1 , + .xcrs[0 ].xcr =0 , + .xcrs[0 ].value =kvm_cpu_supported_xcr0 (), + }; + +if (!kvm_cpu_has (X86_FEATURE_XSAVE )) +return ; + +vcpu_xcrs_set (vcpu, &xcrs); +} +static void set_idt_entry (struct kvm_vm *vm,int vector,unsigned long addr,int dpl,unsigned short selector) {
@@ -675,6 +691,7 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id) vcpu =__vm_vcpu_add (vm, vcpu_id);vcpu_init_cpuid (vcpu,kvm_get_supported_cpuid ());vcpu_init_sregs (vm, vcpu); +vcpu_init_xcrs (vm, vcpu);/* Setup guest general purpose registers */ vcpu_regs_get (vcpu, ®s);
@@ -686,6 +703,13 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id) mp_state.mp_state =0 ;vcpu_mp_state_set (vcpu, &mp_state); +/* +* Refresh CPUID after setting SREGS and XCR0, so that KVM's "runtime" +* updates to guest CPUID, e.g. for OSXSAVE and XSAVE state size, are +* reflected into selftests' vCPU CPUID cache, i.e. so that the cache +* is consistent with vCPU state. +*/ +vcpu_get_cpuid (vcpu);return vcpu; }
tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c +3 -3
@@ -48,16 +48,16 @@ do { \static void guest_code (void ) { -uint64_t xcr0_reset; +uint64_t initial_xcr0;uint64_t supported_xcr0;int i, vector;set_cr4 (get_cr4 () |X86_CR4_OSXSAVE ); - xcr0_reset =xgetbv (0 ); + initial_xcr0 =xgetbv (0 ); supported_xcr0 =this_cpu_supported_xcr0 (); -GUEST_ASSERT (xcr0_reset ==XFEATURE_MASK_FP ); +GUEST_ASSERT (initial_xcr0 == supported_xcr0);/* Check AVX */ ASSERT_XFEATURE_DEPENDENCIES(supported_xcr0,
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 05/11] KVM: selftests: Configure XCR0 to max supported value by default
Quoted 31 lines · Sean Christopherson
To play nice with compilers generating AVX instructions, set CR4.OSXSAVE and configure XCR0 by default when creating selftests vCPUs. Some distros have switched gcc to '-march=x86-64-v3' by default, and while it's hard to find a CPU which doesn't support AVX today, many KVM selftests fail with
==== Test Assertion Failure ====
lib/x86_64/processor.c:570: Unhandled exception in guest pid=72747 tid=72747 errno=4 - Interrupted system call Unhandled exception '0x6' at guest RIP '0x4104f7'due to selftests not enabling AVX by default for the guest. The failure is easy to reproduce elsewhere with:
$ make clean && CFLAGS='-march=x86-64-v3' make -j && ./x86_64/kvm_pv_testE.g. gcc-13 with -march=x86-64-v3 compiles this chunk from selftests'
kvm_fixup_exception():regs->rip = regs->r11; regs->r9 = regs->vector; regs->r10 = regs->error_code;into this monstronsity (which is clever, but oof):
405313: c4 e1 f9 6e c8 vmovq %rax,%xmm1 405318: 48 89 68 08 mov %rbp,0x8(%rax) 40531c: 48 89 e8 mov %rbp,%rax 40531f: c4 c3 f1 22 c4 01 vpinsrq $0x1,%r12,%xmm1,%xmm0 405325: 49 89 6d 38 mov %rbp,0x38(%r13) 405329: c5 fa 7f 45 00 vmovdqu %xmm0,0x0(%rbp)
Quoted 4 lines
Alternatively, KVM selftests could explicitly restrict the compiler to -march=x86-64-v2, but odds are very good that punting on AVX enabling will simply result in tests that "need" AVX doing their own thing, e.g. there are already three or so additional cleanups that can be done on top.
Ideally, we may still want to precisely pin the set of instructions which are used to generete guest code in selftests as the environment where this code runs is defined by us and it may not match the host. I can easily imaging future CPU features leading to similar issues in case they require explicit enablement. To achive this, we can probably separate guest code from each test into its own compilation unit.
Quoted 96 lines
Reported-by Vitaly Kuznetsov <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- .../selftests/kvm/include/x86_64/processor.h | 5 ++++ .../selftests/kvm/lib/x86_64/processor.c | 24 +++++++++++++++++++ .../selftests/kvm/x86_64/xcr0_cpuid_test.c | 6 ++--- 3 files changed, 32 insertions(+), 3 deletions(-)tools/testing/selftests/kvm/include/x86_64/processor.h +5
-0@@ -1049,6 +1049,11 @@ static inline void vcpu_set_cpuid(struct kvm_vcpu *vcpu)vcpu_ioctl (vcpu,KVM_GET_CPUID2 , vcpu->cpuid); } +static inline void vcpu_get_cpuid (struct kvm_vcpu *vcpu) +{ +vcpu_ioctl (vcpu,KVM_GET_CPUID2 , vcpu->cpuid); +} +void vcpu_set_cpuid_property (struct kvm_vcpu *vcpu,struct kvm_x86_cpu_property property,uint32_t value);tools/testing/selftests/kvm/lib/x86_64/processor.c +24
-0@@ -506,6 +506,8 @@ static void vcpu_init_sregs(struct kvm_vm *vm, struct kvm_vcpu *vcpu) sregs.cr0 =X86_CR0_PE |X86_CR0_NE |X86_CR0_PG ; sregs.cr4 |=X86_CR4_PAE |X86_CR4_OSFXSR ; +if (kvm_cpu_has (X86_FEATURE_XSAVE )) + sregs.cr4 |=X86_CR4_OSXSAVE ; sregs.efer |= (EFER_LME |EFER_LMA |EFER_NX );kvm_seg_set_unusable (&sregs.ldt);@@ -519,6 +521,20 @@ static void vcpu_init_sregs(struct kvm_vm *vm, struct kvm_vcpu *vcpu)vcpu_sregs_set (vcpu, &sregs); } +static void vcpu_init_xcrs (struct kvm_vm *vm,struct kvm_vcpu *vcpu) +{ +struct kvm_xcrs xcrs = { + .nr_xcrs =1 , + .xcrs[0 ].xcr =0 , + .xcrs[0 ].value =kvm_cpu_supported_xcr0 (), + }; + +if (!kvm_cpu_has (X86_FEATURE_XSAVE )) +return ; + +vcpu_xcrs_set (vcpu, &xcrs); +} +static void set_idt_entry (struct kvm_vm *vm,int vector,unsigned long addr,int dpl,unsigned short selector) {@@ -675,6 +691,7 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id) vcpu =__vm_vcpu_add (vm, vcpu_id);vcpu_init_cpuid (vcpu,kvm_get_supported_cpuid ());vcpu_init_sregs (vm, vcpu); +vcpu_init_xcrs (vm, vcpu);/* Setup guest general purpose registers */ vcpu_regs_get (vcpu, ®s);@@ -686,6 +703,13 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id) mp_state.mp_state =0 ;vcpu_mp_state_set (vcpu, &mp_state); +/* +* Refresh CPUID after setting SREGS and XCR0, so that KVM's "runtime" +* updates to guest CPUID, e.g. for OSXSAVE and XSAVE state size, are +* reflected into selftests' vCPU CPUID cache, i.e. so that the cache +* is consistent with vCPU state. +*/ +vcpu_get_cpuid (vcpu);return vcpu; }tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c +1
-1@@ -48,16 +48,16 @@ do { \static void guest_code (void ) { -uint64_t xcr0_reset; +uint64_t initial_xcr0;uint64_t supported_xcr0;int i, vector;set_cr4 (get_cr4 () |X86_CR4_OSXSAVE );
tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c do { \ · line 63
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c +2
-2- xcr0_reset =xgetbv (0 ); + initial_xcr0 =xgetbv (0 ); supported_xcr0 =this_cpu_supported_xcr0 (); -GUEST_ASSERT (xcr0_reset ==XFEATURE_MASK_FP ); +GUEST_ASSERT (initial_xcr0 == supported_xcr0);/* Check AVX */ ASSERT_XFEATURE_DEPENDENCIES(supported_xcr0,
Thanks!
Signature · 1 line
Vitaly
Sean Christopherson
reviewerRe: [PATCH 05/11] KVM: selftests: Configure XCR0 to max supported value by default
Quoted 38 lines · Vitaly Kuznetsov
Quoted 31 lines · Sean Christopherson
To play nice with compilers generating AVX instructions, set CR4.OSXSAVE and configure XCR0 by default when creating selftests vCPUs. Some distros have switched gcc to '-march=x86-64-v3' by default, and while it's hard to find a CPU which doesn't support AVX today, many KVM selftests fail with
==== Test Assertion Failure ====
lib/x86_64/processor.c:570: Unhandled exception in guest pid=72747 tid=72747 errno=4 - Interrupted system call Unhandled exception '0x6' at guest RIP '0x4104f7'due to selftests not enabling AVX by default for the guest. The failure is easy to reproduce elsewhere with:
$ make clean && CFLAGS='-march=x86-64-v3' make -j && ./x86_64/kvm_pv_testE.g. gcc-13 with -march=x86-64-v3 compiles this chunk from selftests'
kvm_fixup_exception():regs->rip = regs->r11; regs->r9 = regs->vector; regs->r10 = regs->error_code;into this monstronsity (which is clever, but oof):
405313: c4 e1 f9 6e c8 vmovq %rax,%xmm1 405318: 48 89 68 08 mov %rbp,0x8(%rax) 40531c: 48 89 e8 mov %rbp,%rax 40531f: c4 c3 f1 22 c4 01 vpinsrq $0x1,%r12,%xmm1,%xmm0 405325: 49 89 6d 38 mov %rbp,0x38(%r13) 405329: c5 fa 7f 45 00 vmovdqu %xmm0,0x0(%rbp)Quoted 4 lines
Alternatively, KVM selftests could explicitly restrict the compiler to -march=x86-64-v2, but odds are very good that punting on AVX enabling will simply result in tests that "need" AVX doing their own thing, e.g. there are already three or so additional cleanups that can be done on top.
Quoted 5 lines
Ideally, we may still want to precisely pin the set of instructions which are used to generete guest code in selftests as the environment where this code runs is defined by us and it may not match the host. I can easily imaging future CPU features leading to similar issues in case they require explicit enablement.
Maybe. I suspect the cross-section of features that require explicit enablement *and* will be generated by the compiler for "regular" code will be limited to AVX and the like. E.g. the only new in -v4 is AVX512.
Quoted 2 lines
To achive this, we can probably separate guest code from each test into its own compilation unit.
Hopefully we don't need to worry about that for years and years :-)
Sean Christopherson
reviewerPATCH 6/11 KVM: selftests: Verify XCR0 can be "downgraded" and "upgraded"
In reply to Sean ChristophersonNow that KVM selftests enable all supported XCR0 features by default, add a testcase to the XCR0 vs. CPUID test to verify that the guest can disable everything except the legacy FPU in XCR0, and then re-enable the full feature set, which is kinda sorta what the test did before XCR0 was setup by default.
--- tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c | 5 +++++ 1 file changed, 5 insertions(+)
tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c +5 -0
@@ -79,6 +79,11 @@ static void guest_code(void)ASSERT_ALL_OR_NONE_XFEATURE (supported_xcr0,XFEATURE_MASK_XTILE ); + vector =xsetbv_safe (0 ,XFEATURE_MASK_FP ); +__GUEST_ASSERT (!vector, +"Expected success on XSETBV(FP), got vector '0x%x'" , + vector); + vector =xsetbv_safe (0 , supported_xcr0); __GUEST_ASSERT(!vector, "Expected success on XSETBV(0x%lx), got vector '0x%x'",
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 06/11] KVM: selftests: Verify XCR0 can be "downgraded" and "upgraded"
Quoted 17 lines · Sean Christopherson
Now that KVM selftests enable all supported XCR0 features by default, add a testcase to the XCR0 vs. CPUID test to verify that the guest can disable everything except the legacy FPU in XCR0, and then re-enable the full feature set, which is kinda sorta what the test did before XCR0 was setup by default.
Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c | 5 +++++ 1 file changed, 5 insertions(+)tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c +0
-0@@ -79,6 +79,11 @@ static void guest_code(void) ASSERT_ALL_OR_NONE_XFEATURE(supported_xcr0,
tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c static void guest_code(void) · line 89
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/xcr0_cpuid_test.c +5
-0XFEATURE_MASK_XTILE); + vector =xsetbv_safe (0 ,XFEATURE_MASK_FP ); +__GUEST_ASSERT (!vector, +"Expected success on XSETBV(FP), got vector '0x%x'" , + vector); + vector =xsetbv_safe (0 , supported_xcr0); __GUEST_ASSERT(!vector, "Expected success on XSETBV(0x%lx), got vector '0x%x'",
Signature · 1 line
Vitaly
Sean Christopherson
reviewerPATCH 7/11 KVM: selftests: Drop manual CR4.OSXSAVE enabling from CR4/CPUID sync test
In reply to Sean ChristophersonNow that CR4.OSXSAVE is enabled by default, drop the manual enabling from CR4/CPUID sync test and instead assert that CR4.OSXSAVE is enabled.
--- tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c +2 -5
@@ -27,12 +27,9 @@ static void guest_code(void) [KVM_CPUID_EAX] = X86_FEATURE_OSXSAVE.function, [KVM_CPUID_ECX ] =X86_FEATURE_OSXSAVE .index, }; -uint64_t cr4; -/* turn on CR4.OSXSAVE */ - cr4 =get_cr4 (); - cr4 |=X86_CR4_OSXSAVE ; -set_cr4 (cr4); +/* CR4.OSXSAVE should be enabled by default (for selftests vCPUs). */ +GUEST_ASSERT (get_cr4 () &X86_CR4_OSXSAVE );/* verify CR4.OSXSAVE == CPUID.OSXSAVE */ GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE ));
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 07/11] KVM: selftests: Drop manual CR4.OSXSAVE enabling from CR4/CPUID sync test
Quoted 17 lines · Sean Christopherson
Now that CR4.OSXSAVE is enabled by default, drop the manual enabling from CR4/CPUID sync test and instead assert that CR4.OSXSAVE is enabled.
Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c +0
-1@@ -27,12 +27,9 @@ static void guest_code(void) [KVM_CPUID_EAX] = X86_FEATURE_OSXSAVE.function, [KVM_CPUID_ECX] = X86_FEATURE_OSXSAVE.index, }; -uint64_t cr4;
tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c static void guest_code(void) · line 35
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c +2
-4-/* turn on CR4.OSXSAVE */ - cr4 =get_cr4 (); - cr4 |=X86_CR4_OSXSAVE ; -set_cr4 (cr4); +/* CR4.OSXSAVE should be enabled by default (for selftests vCPUs). */ +GUEST_ASSERT (get_cr4 () &X86_CR4_OSXSAVE );/* verify CR4.OSXSAVE == CPUID.OSXSAVE */ GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE ));
Signature · 1 line
Vitaly
Sean Christopherson
reviewerPATCH 8/11 KVM: selftests: Drop manual XCR0 configuration from AMX test
In reply to Sean ChristophersonNow that CR4.OSXSAVE and XCR0 are setup by default, drop the manual enabling of OXSAVE and XTILE from the AMX test.
--- tools/testing/selftests/kvm/x86_64/amx_test.c | 23 ++++--------------- 1 file changed, 4 insertions(+), 19 deletions(-)
tools/testing/selftests/kvm/x86_64/amx_test.c +4 -19
@@ -86,6 +86,8 @@ static inline void __xsavec(struct xstate *xstate, uint64_t rfbm)static void check_xtile_info (void ) { +GUEST_ASSERT ((xgetbv (0 ) &XFEATURE_MASK_XTILE ) ==XFEATURE_MASK_XTILE ); +GUEST_ASSERT (this_cpu_has_p (X86_PROPERTY_XSTATE_MAX_SIZE_XCR0 ));GUEST_ASSERT (this_cpu_property (X86_PROPERTY_XSTATE_MAX_SIZE_XCR0 ) <=XSAVE_SIZE );
@@ -122,29 +124,12 @@ static void set_tilecfg(struct tile_config *cfg) } } -static void init_regs (void ) -{ -uint64_t cr4, xcr0; - -GUEST_ASSERT (this_cpu_has (X86_FEATURE_XSAVE )); - -/* turn on CR4.OSXSAVE */ - cr4 =get_cr4 (); - cr4 |=X86_CR4_OSXSAVE ; -set_cr4 (cr4); -GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE )); - - xcr0 =xgetbv (0 ); - xcr0 |=XFEATURE_MASK_XTILE ; -xsetbv (0x0 , xcr0); -GUEST_ASSERT ((xgetbv (0 ) &XFEATURE_MASK_XTILE ) ==XFEATURE_MASK_XTILE ); -} -static void __attribute__((__flatten__))guest_code (struct tile_config *amx_cfg,struct tile_data *tiledata,struct xstate *xstate) { -init_regs (); +GUEST_ASSERT (this_cpu_has (X86_FEATURE_XSAVE ) && +this_cpu_has (X86_FEATURE_OSXSAVE ));check_xtile_info ();GUEST_SYNC (1 );
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 08/11] KVM: selftests: Drop manual XCR0 configuration from AMX test
Quoted 40 lines · Sean Christopherson
Now that CR4.OSXSAVE and XCR0 are setup by default, drop the manual enabling of OXSAVE and XTILE from the AMX test.
Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/x86_64/amx_test.c | 23 ++++--------------- 1 file changed, 4 insertions(+), 19 deletions(-)tools/testing/selftests/kvm/x86_64/amx_test.c +2
-15@@ -86,6 +86,8 @@ static inline void __xsavec(struct xstate *xstate, uint64_t rfbm)static void check_xtile_info (void ) { +GUEST_ASSERT ((xgetbv (0 ) &XFEATURE_MASK_XTILE ) ==XFEATURE_MASK_XTILE ); +GUEST_ASSERT (this_cpu_has_p (X86_PROPERTY_XSTATE_MAX_SIZE_XCR0 ));GUEST_ASSERT (this_cpu_property (X86_PROPERTY_XSTATE_MAX_SIZE_XCR0 ) <=XSAVE_SIZE );@@ -122,29 +124,12 @@ static void set_tilecfg(struct tile_config *cfg) } } -static void init_regs (void ) -{ -uint64_t cr4, xcr0; - -GUEST_ASSERT (this_cpu_has (X86_FEATURE_XSAVE )); - -/* turn on CR4.OSXSAVE */ - cr4 =get_cr4 (); - cr4 |=X86_CR4_OSXSAVE ; -set_cr4 (cr4); -GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE )); - - xcr0 =xgetbv (0 ); - xcr0 |=XFEATURE_MASK_XTILE ; -xsetbv (0x0 , xcr0);
tools/testing/selftests/kvm/x86_64/amx_test.c static void set_tilecfg(struct tile_config *cfg) · line 132
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/amx_test.c +2
-4-GUEST_ASSERT ((xgetbv (0 ) &XFEATURE_MASK_XTILE ) ==XFEATURE_MASK_XTILE ); -} -static void __attribute__((__flatten__))guest_code (struct tile_config *amx_cfg,struct tile_data *tiledata,struct xstate *xstate) { -init_regs (); +GUEST_ASSERT (this_cpu_has (X86_FEATURE_XSAVE ) && +this_cpu_has (X86_FEATURE_OSXSAVE ));
Maybe
GUEST_ASSERT(get_cr4() & X86_CR4_OSXSAVE);
also (or instead), just like cr4_cpuid_sync_test?
tools/testing/selftests/kvm/x86_64/amx_test.c static void set_tilecfg(struct tile_config *cfg) · line 134
Quoted 2 lines
tools/testing/selftests/kvm/x86_64/amx_test.c +0
-0check_xtile_info ();GUEST_SYNC (1 );
Signature · 1 line
Vitaly
Sean Christopherson
reviewerPATCH 9/11 KVM: selftests: Drop manual XCR0 configuration from state test
In reply to Sean ChristophersonNow that CR4.OSXSAVE and XCR0 are setup by default, drop the manual enabling from the state test, which is fully redundant with the default behavior.
--- tools/testing/selftests/kvm/x86_64/state_test.c | 5 ----- 1 file changed, 5 deletions(-)
tools/testing/selftests/kvm/x86_64/state_test.c +0 -5
@@ -145,11 +145,6 @@ static void __attribute__((__flatten__)) guest_code(void *arg)memset (buffer,0xcc ,sizeof (buffer)); -set_cr4 (get_cr4 () |X86_CR4_OSXSAVE ); -GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE )); - -xsetbv (0 ,xgetbv (0 ) | supported_xcr0); -/* * Modify state for all supported xfeatures to take them out of * their "init" state, i.e. to make them show up in XSTATE_BV.
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 09/11] KVM: selftests: Drop manual XCR0 configuration from state test
Quoted 15 lines · Sean Christopherson
Now that CR4.OSXSAVE and XCR0 are setup by default, drop the manual enabling from the state test, which is fully redundant with the default behavior.
Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/x86_64/state_test.c | 5 ----- 1 file changed, 5 deletions(-)tools/testing/selftests/kvm/x86_64/state_test.c +0
-0@@ -145,11 +145,6 @@ static void __attribute__((__flatten__)) guest_code(void *arg)
tools/testing/selftests/kvm/x86_64/state_test.c static void __attribute__((__flatten__)) guest_code(void *arg) · line 150
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/state_test.c +0
-5memset (buffer,0xcc ,sizeof (buffer)); -set_cr4 (get_cr4 () |X86_CR4_OSXSAVE ); -GUEST_ASSERT (this_cpu_has (X86_FEATURE_OSXSAVE )); - -xsetbv (0 ,xgetbv (0 ) | supported_xcr0); -/* * Modify state for all supported xfeatures to take them out of * their "init" state, i.e. to make them show up in XSTATE_BV.
Signature · 1 line
Vitaly
Sean Christopherson
reviewerPATCH 10/11 KVM: selftests: Drop manual XCR0 configuration from SEV smoke test
In reply to Sean ChristophersonNow that CR4.OSXSAVE and XCR0 are setup by default, drop the manual enabling from the SEV smoke test that validates FPU state can be transferred into the VMSA.
In guest_code_xsave(), explicitly set the Requested-Feature Bitmask (RFBM) to exactly XFEATURE_MASK_X87_AVX instead of relying on the host side of things to enable only X87_AVX features in guest XCR0. I.e. match the RFBM for the host XSAVE.
--- .../testing/selftests/kvm/x86_64/sev_smoke_test.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)
tools/testing/selftests/kvm/x86_64/sev_smoke_test.c +2 -13
@@ -41,8 +41,8 @@ static void guest_sev_code(void)/* Stash state passed via VMSA before any compiled code runs. */ extern void guest_code_xsave (void ); asm("guest_code_xsave:\n" -"mov $-1, %eax\n" -"mov $-1, %edx\n" + "mov $" __stringify(XFEATURE_MASK_X87_AVX) ", %eax\n" +"xor %edx, %edx\n" "xsave (%rdi)\n" "jmp guest_sev_es_code");
@@ -70,12 +70,6 @@ static void test_sync_vmsa(uint32_t policy)double x87val =M_PI ;struct kvm_xsave __attribute__((aligned (64 ))) xsave = {0 }; -struct kvm_sregs sregs; -struct kvm_xcrs xcrs = { - .nr_xcrs =1 , - .xcrs[0 ].xcr =0 , - .xcrs[0 ].value =XFEATURE_MASK_X87_AVX , - }; vm =vm_sev_create_with_one_vcpu (KVM_X86_SEV_ES_VM , guest_code_xsave, &vcpu); gva = vm_vaddr_alloc_shared(vm, PAGE_SIZE, KVM_UTIL_MIN_VADDR,
@@ -84,11 +78,6 @@ static void test_sync_vmsa(uint32_t policy)vcpu_args_set (vcpu,1 , gva); -vcpu_sregs_get (vcpu, &sregs); - sregs.cr4 |=X86_CR4_OSFXSR |X86_CR4_OSXSAVE ; -vcpu_sregs_set (vcpu, &sregs); - -vcpu_xcrs_set (vcpu, &xcrs); asm("fninit\n" "vpcmpeqb %%ymm4, %%ymm4, %%ymm4\n" "fldl %3\n"
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Sean Christopherson
reviewerPATCH 11/11 KVM: selftests: Ensure KVM supports AVX for SEV-ES VMSA FPU test
In reply to Sean ChristophersonVerify that KVM's supported XCR0 includes AVX (and earlier features) when running the SEV-ES VMSA XSAVE test. In practice, the issue will likely never pop up, since KVM support for AVX predates KVM support for SEV-ES, but checking for KVM support makes the requirement more obvious.
--- tools/testing/selftests/kvm/x86_64/sev_smoke_test.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/x86_64/sev_smoke_test.c +3 -1
@@ -181,6 +181,8 @@ static void test_sev_es_shutdown(void)int main (int argc,char *argv[]) { +const u64 xf_mask =XFEATURE_MASK_X87_AVX ; +TEST_REQUIRE (kvm_cpu_has (X86_FEATURE_SEV ));test_sev (guest_sev_code,SEV_POLICY_NO_DBG );
@@ -193,7 +195,7 @@ int main(int argc, char *argv[])test_sev_es_shutdown ();if (kvm_has_cap (KVM_CAP_XCRS ) && - (xgetbv (0 ) &XFEATURE_MASK_X87_AVX ) ==XFEATURE_MASK_X87_AVX ) { + (xgetbv (0 ) &kvm_cpu_supported_xcr0 () & xf_mask) == xf_mask) {test_sync_vmsa (0 );test_sync_vmsa (SEV_POLICY_NO_DBG ); }
Signature · 1 line
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH 11/11] KVM: selftests: Ensure KVM supports AVX for SEV-ES VMSA FPU test
Quoted 24 lines · Sean Christopherson
Verify that KVM's supported XCR0 includes AVX (and earlier features) when running the SEV-ES VMSA XSAVE test. In practice, the issue will likely never pop up, since KVM support for AVX predates KVM support for SEV-ES, but checking for KVM support makes the requirement more obvious.
Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/x86_64/sev_smoke_test.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)tools/testing/selftests/kvm/x86_64/sev_smoke_test.c +2
-0@@ -181,6 +181,8 @@ static void test_sev_es_shutdown(void)int main (int argc,char *argv[]) { +const u64 xf_mask =XFEATURE_MASK_X87_AVX ; +TEST_REQUIRE (kvm_cpu_has (X86_FEATURE_SEV ));test_sev (guest_sev_code,SEV_POLICY_NO_DBG );
tools/testing/selftests/kvm/x86_64/sev_smoke_test.c int main(int argc, char *argv[]) · line 201
Quoted 8 lines
tools/testing/selftests/kvm/x86_64/sev_smoke_test.c +1
-1@@ -193,7 +195,7 @@ int main(int argc, char *argv[])test_sev_es_shutdown ();if (kvm_has_cap (KVM_CAP_XCRS ) && - (xgetbv (0 ) &XFEATURE_MASK_X87_AVX ) ==XFEATURE_MASK_X87_AVX ) { + (xgetbv (0 ) &kvm_cpu_supported_xcr0 () & xf_mask) == xf_mask) {test_sync_vmsa (0 );test_sync_vmsa (SEV_POLICY_NO_DBG ); }
Signature · 1 line
Vitaly
Paolo Bonzini
maintainerQuoted 15 lines · Sean Christopherson
Enable CR4.OSXSAVE and XCR0.AVX by default when creating selftests vCPUs in order to play nice with compilers that have been configured to enable -march=x86-64-v3 by default.
While it would be easier to force v2 (or earlier), there are enough tests that want XCR0 configured that it will (hopefully) be a net postive to enable all XCR0 features by default.
The only real hiccup is the CR4/CPUID sync test, which disables CR4.OSXSAVE to verify KVM toggles the associated CPUID bit. And if it calls
memset()while OSXAVE is disabled, kablooie. Fixing that requires a bit of assembly, but overall I think it's worth carrying a few lines of assembly in order to gain test coverage for running AVX instructions in guests, and boy are compilers good at abusing AVX :-)
Quoted 3 lines
Fix a few bugs/warts found along the way. Notably, the CPUID test has an array out-of-bounds bug that can result in false passes (I only noticed because it was getting a false pass on gcc).
I think this is not -rc/stable material, so for now I'm applying Vitaly's patch, plus patch 1 from this series.
Paolo
Quoted 28 lines
Sean Christopherson (11):
KVM: selftests: Fix out-of-bounds reads in CPUID test's array lookups
KVM: selftests: Precisely mask off dynamic fields in CPUID test
KVM: selftests: Mask off OSPKE and OSXSAVE when comparing CPUID
entries
KVM: selftests: Rework OSXSAVE CR4=>CPUID test to play nice with AVX
insns
KVM: selftests: Configure XCR0 to max supported value by default
KVM: selftests: Verify XCR0 can be "downgraded" and "upgraded"
KVM: selftests: Drop manual CR4.OSXSAVE enabling from CR4/CPUID sync
test
KVM: selftests: Drop manual XCR0 configuration from AMX test
KVM: selftests: Drop manual XCR0 configuration from state test
KVM: selftests: Drop manual XCR0 configuration from SEV smoke test
KVM: selftests: Ensure KVM supports AVX for SEV-ES VMSA FPU test.../selftests/kvm/include/x86_64/processor.h | 5 ++ .../selftests/kvm/lib/x86_64/processor.c | 24 +++++++ tools/testing/selftests/kvm/x86_64/amx_test.c | 23 ++----- .../testing/selftests/kvm/x86_64/cpuid_test.c | 67 ++++++++++++------- .../kvm/x86_64/cr4_cpuid_sync_test.c | 53 +++++++++------ .../selftests/kvm/x86_64/sev_smoke_test.c | 19 ++---- .../testing/selftests/kvm/x86_64/state_test.c | 5 -- .../selftests/kvm/x86_64/xcr0_cpuid_test.c | 11 ++- 8 files changed, 122 insertions(+), 85 deletions(-)base-commit:
efbc6bd090f48ccf64f7a8dd5daea775821d57ec
Sean Christopherson
reviewerQuoted 9 lines · Sean Christopherson
Enable CR4.OSXSAVE and XCR0.AVX by default when creating selftests vCPUs in order to play nice with compilers that have been configured to enable -march=x86-64-v3 by default.
While it would be easier to force v2 (or earlier), there are enough tests that want XCR0 configured that it will (hopefully) be a net postive to enable all XCR0 features by default.
[...]
Applied to kvm-x86 selftests, minus patch 1 which went into 6.12. At some point in the 6.13 cycle I'll send a revert for the "march" madness.
[01/11] KVM: selftests: Fix out-of-bounds reads in CPUID test's array lookups
(no commit info)
[02/11] KVM: selftests: Precisely mask off dynamic fields in CPUID test
https://github.com/kvm-x86/linux/commit/c0124e2e74a7 [03/11] KVM: selftests: Mask off OSPKE and OSXSAVE when comparing CPUID entries https://github.com/kvm-x86/linux/commit/01e2827157ef [04/11] KVM: selftests: Rework OSXSAVE CR4=>CPUID test to play nice with AVX insns https://github.com/kvm-x86/linux/commit/cf50f01336d3 [05/11] KVM: selftests: Configure XCR0 to max supported value by default https://github.com/kvm-x86/linux/commit/331b8ddaebc1 [06/11] KVM: selftests: Verify XCR0 can be "downgraded" and "upgraded" https://github.com/kvm-x86/linux/commit/d87b459428c0 [07/11] KVM: selftests: Drop manual CR4.OSXSAVE enabling from CR4/CPUID sync test https://github.com/kvm-x86/linux/commit/86502f01b8b9 [08/11] KVM: selftests: Drop manual XCR0 configuration from AMX test https://github.com/kvm-x86/linux/commit/fd7b6d77fa6d [09/11] KVM: selftests: Drop manual XCR0 configuration from state test https://github.com/kvm-x86/linux/commit/818646fea3ea [10/11] KVM: selftests: Drop manual XCR0 configuration from SEV smoke test https://github.com/kvm-x86/linux/commit/ce22d24024ea [11/11] KVM: selftests: Ensure KVM supports AVX for SEV-ES VMSA FPU test https://github.com/kvm-x86/linux/commit/08cc7ab1a6ca
Sean Christopherson
reviewerQuoted 28 lines · Sean Christopherson
Quoted 9 lines · Sean Christopherson
Enable CR4.OSXSAVE and XCR0.AVX by default when creating selftests vCPUs in order to play nice with compilers that have been configured to enable -march=x86-64-v3 by default.
While it would be easier to force v2 (or earlier), there are enough tests that want XCR0 configured that it will (hopefully) be a net postive to enable all XCR0 features by default.
[...]
Applied to kvm-x86 selftests, minus patch 1 which went into 6.12. At some point in the 6.13 cycle I'll send a revert for the "march" madness.
[01/11] KVM: selftests: Fix out-of-bounds reads in CPUID test's array lookups
(no commit info)
[02/11] KVM: selftests: Precisely mask off dynamic fields in CPUID testhttps://github.com/kvm-x86/linux/commit/c0124e2e74a7 [03/11] KVM: selftests: Mask off OSPKE and OSXSAVE when comparing CPUID entries https://github.com/kvm-x86/linux/commit/01e2827157ef [04/11] KVM: selftests: Rework OSXSAVE CR4=>CPUID test to play nice with AVX insns https://github.com/kvm-x86/linux/commit/cf50f01336d3 [05/11] KVM: selftests: Configure XCR0 to max supported value by default https://github.com/kvm-x86/linux/commit/331b8ddaebc1 [06/11] KVM: selftests: Verify XCR0 can be "downgraded" and "upgraded" https://github.com/kvm-x86/linux/commit/d87b459428c0 [07/11] KVM: selftests: Drop manual CR4.OSXSAVE enabling from CR4/CPUID sync test https://github.com/kvm-x86/linux/commit/86502f01b8b9
Quoted 8 lines
[08/11] KVM: selftests: Drop manual XCR0 configuration from AMX test
https://github.com/kvm-x86/linux/commit/fd7b6d77fa6d [09/11] KVM: selftests: Drop manual XCR0 configuration from state test https://github.com/kvm-x86/linux/commit/818646fea3ea [10/11] KVM: selftests: Drop manual XCR0 configuration from SEV smoke test https://github.com/kvm-x86/linux/commit/ce22d24024ea [11/11] KVM: selftests: Ensure KVM supports AVX for SEV-ES VMSA FPU test https://github.com/kvm-x86/linux/commit/08cc7ab1a6ca
And because I mucked up the mmu_stress_test/vcpu_get_reg() series and had to yank it out, the hashes for this series got changed:
[02/11] KVM: selftests: Precisely mask off dynamic fields in CPUID test
https://github.com/kvm-x86/linux/commit/f2c5aa31670d [03/11] KVM: selftests: Mask off OSPKE and OSXSAVE when comparing CPUID entries https://github.com/kvm-x86/linux/commit/164cea33bfed [04/11] KVM: selftests: Rework OSXSAVE CR4=>CPUID test to play nice with AVX insns https://github.com/kvm-x86/linux/commit/2b9a126a2986 [05/11] KVM: selftests: Configure XCR0 to max supported value by default https://github.com/kvm-x86/linux/commit/8b14c4d85d03 [06/11] KVM: selftests: Verify XCR0 can be "downgraded" and "upgraded" https://github.com/kvm-x86/linux/commit/8ae01bf64caa [07/11] KVM: selftests: Drop manual CR4.OSXSAVE enabling from CR4/CPUID sync test https://github.com/kvm-x86/linux/commit/3678c7f6114f [08/11] KVM: selftests: Drop manual XCR0 configuration from AMX test https://github.com/kvm-x86/linux/commit/d87331890a38 [09/11] KVM: selftests: Drop manual XCR0 configuration from state test https://github.com/kvm-x86/linux/commit/28439090ece6 [10/11] KVM: selftests: Drop manual XCR0 configuration from SEV smoke test https://github.com/kvm-x86/linux/commit/3c4c128d02ed [11/11] KVM: selftests: Ensure KVM supports AVX for SEV-ES VMSA FPU test https://github.com/kvm-x86/linux/commit/89f8869835e4