Add functional test for AMD VMSAVE/VMLOAD instructions
cover Add functional test for AMD VMSAVE/VMLOAD instructions
Thanks to a minor bug in LTP KVM library, LTP test kvm_svm02 found a kernel bug in emulation of VMSAVE and VMLOAD instructions in nested VMs. Add a thorough functional test for both instructions which can detect and pinpoint emulation bugs.
Also implement basic printf-like formatting for tst_res() and tst_brk() so that the test can print incorrect register values to simplify result analysis. Only standard integer and character conversions are supported. Floating point number conversions, field alignment and padding are not implemented.
Martin Doucha (9):
KVM: Disable EBP register use in 32bit code
KVM: Implement strchr() and basic sprintf()
KVM: Implement printf-like formatting for tst_res() and tst_brk()
kvm_svm02: Fix saved stack segment index value kvm_find_free_descriptor(): Skip descriptor 0
KVM: Add system control MSR constants
KVM: Add VMSAVE/VMLOAD functions to x86 SVM library
KVM: Add functional test for VMSAVE/VMLOAD instructions
KVM: Move kvm_pagefault01 to the end of KVM runfile
runtest/kvm | 4 +- testcases/kernel/kvm/.gitignore | 1 + testcases/kernel/kvm/Makefile | 2 +- testcases/kernel/kvm/include/kvm_guest.h | 23 +- testcases/kernel/kvm/include/kvm_x86.h | 10 + testcases/kernel/kvm/include/kvm_x86_svm.h | 6 + testcases/kernel/kvm/kvm_svm02.c | 14 +- testcases/kernel/kvm/kvm_svm04.c | 307 ++++++++++++++++++ testcases/kernel/kvm/lib_guest.c | 348 ++++++++++++++++++++- testcases/kernel/kvm/lib_x86.c | 20 +- 10 files changed, 713 insertions(+), 22 deletions(-) create mode 100644 testcases/kernel/kvm/kvm_svm04.c
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 1/9 KVM: Disable EBP register use in 32bit code
The EBP register points to the stack segment by default but GCC uses it to access data segment without the proper prefix. This works fine on most systems because the stack and data segments are usually identical. However, KVM environment intentionally enforces strict limits on the stack segment and access to the data segment using unprefixed EBP would trigger stack segment fault exception in 32bit LTP builds (stack segment limits are ignored in 64bit mode).
--- testcases/kernel/kvm/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/kvm/Makefile +1 -1
@@ -22,7 +22,7 @@ ifeq ($(HOST_CPU),x86_64) endif ifeq ($(HOST_CPU),x86) - GUEST_CFLAGS += -m32 + GUEST_CFLAGS += -m32 -ffixed-ebp ASFLAGS += --32 endif
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/9] KVM: Disable EBP register use in 32bit code
Hi Martin,
Quoted 18 lines
The EBP register points to the stack segment by default but GCC uses it to access data segment without the proper prefix. This works fine on most systems because the stack and data segments are usually identical. However, KVM environment intentionally enforces strict limits on the stack segment and access to the data segment using unprefixed EBP would trigger stack segment fault exception in 32bit LTP builds (stack segment limits are ignored in 64bit mode).
Signed-off-by Martin Doucha <[email protected]>--- testcases/kernel/kvm/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)testcases/kernel/kvm/Makefile +0
-0
testcases/kernel/kvm/Makefile ifeq ($(HOST_CPU),x86_64) · line 25
Quoted 5 lines
testcases/kernel/kvm/Makefile +1
-1@@ -22,7 +22,7 @@ ifeq ($(HOST_CPU),x86_64) endif ifeq ($(HOST_CPU),x86) - GUEST_CFLAGS += -m32 + GUEST_CFLAGS += -m32 -ffixed-ebp
FYI this will fail on 32 bit build on clang:
clang: error: unknown argument: '-ffixed-ebp'
I don't want to block this patchset which brings important test, but it'd be great to fix it.
Is there clang equivalent? Or is it even needed for clang?
Either way, we need to detect clang. I don't think simple
ifeq ($(CXX),clang)
would be enough, because cc can be alias to clang.
Maybe wrap it with version detection:
ifeq ($(shell $(CC) -v 2>&1 | grep "clang version"), 1)
GUEST_CFLAGS += -ffixed-ebp
endif
Kind regards,
Petr
Quoted 2 lines
testcases/kernel/kvm/Makefile +0
-0ASFLAGS += --32 endif
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/9] KVM: Disable EBP register use in 32bit code
Quoted 11 lines · Petr Vorel
testcases/kernel/kvm/Makefile ifeq ($(HOST_CPU),x86_64) · line 25
Quoted 3 lines
ifeq ($(HOST_CPU),x86)testcases/kernel/kvm/Makefile +1
-1- GUEST_CFLAGS += -m32 + GUEST_CFLAGS += -m32 -ffixed-ebpFYI this will fail on 32 bit build on clang:
clang: error: unknown argument: '-ffixed-ebp'
I don't want to block this patchset which brings important test, but it'd be great to fix it.
Quoted 7 lines
Is there clang equivalent? Or is it even needed for clang?
Either way, we need to detect clang. I don't think simple
ifeq ($(CXX),clang)
would be enough, because cc can be alias to clang.
Hmm, I need to fix this. I guess that configure should just check for -ffixed-ebp support. Fortunately, clang doesn't generate code that would trigger stack segment fault so the workaround is only needed for GCC.
Could you review and merge the trivial patches (4, 5, 6, 7) so that I don't need to resubmit everything?
Signature · 11 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/9] KVM: Disable EBP register use in 32bit code
Quoted 14 lines · Petr Vorel
Quoted 12 lines
testcases/kernel/kvm/Makefile ifeq ($(HOST_CPU),x86_64) · line 25
Quoted 3 lines
ifeq ($(HOST_CPU),x86)testcases/kernel/kvm/Makefile +1
-1- GUEST_CFLAGS += -m32 + GUEST_CFLAGS += -m32 -ffixed-ebpFYI this will fail on 32 bit build on clang:
clang: error: unknown argument: '-ffixed-ebp'
I don't want to block this patchset which brings important test, but it'd be great to fix it.
Is there clang equivalent? Or is it even needed for clang?
Quoted 7 lines
Quoted 5 lines
Either way, we need to detect clang. I don't think simple
ifeq ($(CXX),clang)
would be enough, because cc can be alias to clang.
Hmm, I need to fix this. I guess that configure should just check for -ffixed-ebp support.
Yes, that would be ideal solution which I definitely don't want to force you.
Quoted 2 lines
Fortunately, clang doesn't generate code that would trigger stack segment fault so the workaround is only needed for GCC.
Great!
Quoted 2 lines
Could you review and merge the trivial patches (4, 5, 6, 7) so that I don't need to resubmit everything?
Sure!
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/9] KVM: Disable EBP register use in 32bit code
In reply to Martin DouchaHi Martin,
...
Quoted 2 lines
Could you review and merge the trivial patches (4, 5, 6, 7) so that I don't need to resubmit everything?
I could even rebase 9th (missing kvm_svm04) to merge it before. Please let me know if change order was actually needed only for kvm_svm04 (it does not look like from the commit message).
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/9] KVM: Disable EBP register use in 32bit code
Quoted 9 lines · Petr Vorel
Hi Martin,
...
Quoted 2 lines
Could you review and merge the trivial patches (4, 5, 6, 7) so that I don't need to resubmit everything?
I could even rebase 9th (missing
kvm_svm04) to merge it before. Please let me know if change order was actually needed only forkvm_svm04(it does not look like from the commit message).
Feel free to rebase it. It's needed for kvm_svm01 and kvm_svm02 as well.
Signature · 11 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 2/9 KVM: Implement strchr() and basic sprintf()
In reply to Martin DouchaAdd basic implementation of sprintf() that supports string, pointer and integer arguments but without advanced formatting options like field alignment and padding.
--- testcases/kernel/kvm/include/kvm_guest.h | 11 + testcases/kernel/kvm/lib_guest.c | 312 +++++++++++++++++++++++ 2 files changed, 323 insertions(+)
testcases/kernel/kvm/include/kvm_guest.h +11 -0
@@ -39,9 +39,15 @@ typedef unsigned int uint32_t;typedef long long int64_t ;typedef unsigned long long uint64_t ;typedef unsigned long uintptr_t ; +typedef __builtin_va_list va_list ;#define NULL ((void *)0) +#define va_start __builtin_va_start +#define va_arg __builtin_va_arg +#define va_end __builtin_va_end +#define va_copy __builtin_va_copy +void *memset (void *dest,int val,size_t size);void *memzero (void *dest,size_t size);void *memcpy (void *dest,const void *src,size_t size);
@@ -49,6 +55,11 @@ void *memcpy(void *dest, const void *src, size_t size);char *strcpy (char *dest,const char *src);char *strcat (char *dest,const char *src);size_t strlen (const char *str); +char *strchr (const char *s,int c); +char *strrchr (const char *s,int c); + +int vsprintf (char *dest,const char *fmt,va_list ap); +int sprintf (char *dest,const char *fmt, ...);/* Exit the VM by looping on a HLT instruction forever */ void kvm_exit (void ) __attribute__((noreturn));
testcases/kernel/kvm/lib_guest.c +312 -0
324 lines not on this page · raw message
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 3/9 KVM: Implement printf-like formatting for tst_res() and tst_brk()
In reply to Martin Doucha--- testcases/kernel/kvm/include/kvm_guest.h | 12 +++++--- testcases/kernel/kvm/lib_guest.c | 36 +++++++++++++++++++++--- 2 files changed, 40 insertions(+), 8 deletions(-)
testcases/kernel/kvm/include/kvm_guest.h +8 -4
@@ -68,12 +68,16 @@ void kvm_exit(void) __attribute__((noreturn));void kvm_yield (void );void tst_res_ (const char *file,const int lineno,int result, -const char *message); -#define tst_res (result, msg) tst_res_(__FILE__, __LINE__, (result), (msg)) +const char *fmt, ...) + __attribute__ ((format (printf,4 ,5 ))); +#define tst_res (result, fmt, ...) \ + tst_res_(__FILE__, __LINE__, (result), (fmt), ##__VA_ARGS__)void tst_brk_ (const char *file,const int lineno,int result, -const char *message) __attribute__((noreturn)); -#define tst_brk (result, msg) tst_brk_(__FILE__, __LINE__, (result), (msg)) +const char *fmt, ...) __attribute__((noreturn)) + __attribute__ ((format (printf,4 ,5 ))); +#define tst_brk (result, fmt, ...) \ + tst_brk_(__FILE__, __LINE__, (result), (fmt), ##__VA_ARGS__)/* * Send asynchronous notification to host without stopping VM execution and
testcases/kernel/kvm/lib_guest.c +32 -4
@@ -443,6 +443,7 @@ static void tst_fatal_error(const char *file, const int lineno, test_result->result =TBROK ; test_result->lineno = lineno; test_result->file_addr = (uintptr_t )file; +/* Avoid sprintf() here in case of bugs */ strcpy (test_result->message, message);strcat (test_result->message," at address 0x" );ptr2hex (test_result->message +strlen (test_result->message), ip);
@@ -451,19 +452,46 @@ static void tst_fatal_error(const char *file, const int lineno, }void tst_res_ (const char *file,const int lineno,int result, -const char *message) +const char *fmt, ...) { +va_list args; +int ret; + +va_start (args, fmt); test_result->result = result; test_result->lineno = lineno; test_result->file_addr = (uintptr_t )file; -strcpy (test_result->message, message); + ret =vsprintf (test_result->message, fmt, args); +va_end (args); + +if (ret <0 ) { +tst_brk_ (file, lineno,TBROK ,"Invalid tst_res() format: %s" , + fmt); + } +kvm_yield (); }void tst_brk_ (const char *file,const int lineno,int result, -const char *message) +const char *fmt, ...) { -tst_res_ (file, lineno, result, message); +va_list args; +int ret; + +va_start (args, fmt); + test_result->result = result; + test_result->lineno = lineno; + test_result->file_addr = (uintptr_t )file; + ret =vsprintf (test_result->message, fmt, args); +va_end (args); + +if (ret <0 ) { + test_result->result =TBROK ; +strcpy (test_result->message,"Invalid tst_brk() format: " ); +strcat (test_result->message, fmt); + } + +kvm_yield ();kvm_exit (); }
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 4/9 kvm_svm02: Fix saved stack segment index value
In reply to Martin DouchaThe VMCB init helper function expects GDT item index while the VMCB segment selector values are byte offsets. Shift the byte offset of saved stack segment selector to get the correct GDT index.
--- testcases/kernel/kvm/kvm_svm02.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/kvm/kvm_svm02.c +1 -1
@@ -96,7 +96,7 @@ void main(void) vmsave_buf =kvm_alloc_vmcb ();/* Save allocated stack for later VM reinit */ - ss = vcpu->vmcb->ss.selector; + ss = vcpu->vmcb->ss.selector >>3 ; rsp = vcpu->vmcb->rsp;/* Load partial state from vmsave_buf and save it to vcpu->vmcb */
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 5/9 kvm_find_free_descriptor(): Skip descriptor 0
In reply to Martin DouchaThe GDT/LDT descriptor 0 should always be empty. Start search for free descriptor table entry at index 1.
--- testcases/kernel/kvm/lib_x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/kvm/lib_x86.c +1 -1
@@ -174,7 +174,7 @@ int kvm_find_free_descriptor(const struct segment_descriptor *table,const struct segment_descriptor *ptr;size_t i; -for (i =0 , ptr = table; i < size; i++, ptr++) { +for (i =1 , ptr = table +1 ; i < size; i++, ptr++) {if (!(ptr->flags_lo &SEGFLAG_PRESENT ))return i;
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 6/9 KVM: Add system control MSR constants
In reply to Martin Doucha--- testcases/kernel/kvm/include/kvm_x86.h | 10 ++++++++++ 1 file changed, 10 insertions(+)
testcases/kernel/kvm/include/kvm_x86.h +10 -0
@@ -68,7 +68,17 @@/* Model-specific CPU register constants */ +#define MSR_SYSENTER_CS 0x174 +#define MSR_SYSENTER_ESP 0x175 +#define MSR_SYSENTER_EIP 0x176#define MSR_EFER 0xc0000080 +#define MSR_STAR 0xc0000081 +#define MSR_LSTAR 0xc0000082 +#define MSR_CSTAR 0xc0000083 +#define MSR_SFMASK 0xc0000084 +#define MSR_FS_BASE 0xc0000100 +#define MSR_GS_BASE 0xc0000101 +#define MSR_KERNEL_GS_BASE 0xc0000102#define MSR_VM_CR 0xc0010114#define MSR_VM_HSAVE_PA 0xc0010117
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 7/9 KVM: Add VMSAVE/VMLOAD functions to x86 SVM library
In reply to Martin Doucha--- testcases/kernel/kvm/include/kvm_x86_svm.h | 6 ++++++ testcases/kernel/kvm/kvm_svm02.c | 12 ++---------- testcases/kernel/kvm/lib_x86.c | 18 ++++++++++++++++++ 3 files changed, 26 insertions(+), 10 deletions(-)
testcases/kernel/kvm/include/kvm_x86_svm.h +6 -0
@@ -163,4 +163,10 @@ struct kvm_svm_vcpu *kvm_create_svm_vcpu(int (*guest_main)(void),void kvm_svm_vmrun (struct kvm_svm_vcpu *cpu); +/* Load FS, GS, TR and LDTR state from vmsave_buf */ +void kvm_svm_vmload (struct kvm_vmcb *buf); + +/* Save current FS, GS, TR and LDTR state to vmsave_buf */ +void kvm_svm_vmsave (struct kvm_vmcb *buf); +#endif /* KVM_X86_SVM_H_ */
testcases/kernel/kvm/kvm_svm02.c +2 -10
@@ -33,22 +33,14 @@ static void *vmsave_buf;/* Load FS, GS, TR and LDTR state from vmsave_buf */ static int guest_vmload (void ) { - asm ( -"vmload %0\n" - : - :"a" (vmsave_buf) - ); +kvm_svm_vmload (vmsave_buf);return 0 ; }/* Save current FS, GS, TR and LDTR state to vmsave_buf */ static int guest_vmsave (void ) { - asm ( -"vmsave %0\n" - : - :"a" (vmsave_buf) - ); +kvm_svm_vmsave (vmsave_buf);return 0 ; }
testcases/kernel/kvm/lib_x86.c +18 -0
@@ -393,3 +393,21 @@ struct kvm_svm_vcpu *kvm_create_svm_vcpu(int (*guest_main)(void), ret->vmcb = vmcb;return ret; } + +void kvm_svm_vmload (struct kvm_vmcb *buf) +{ + asm ( +"vmload %0\n" + : + :"a" (buf) + ); +} + +void kvm_svm_vmsave (struct kvm_vmcb *buf) +{ + asm ( +"vmsave %0\n" + : + :"a" (buf) + ); +}
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 7/9] KVM: Add VMSAVE/VMLOAD functions to x86 SVM library
Hi Martin,
merged patches 4, 5, 6, 7 as we discussed in the first patch.
Thanks!
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 8/9 KVM: Add functional test for VMSAVE/VMLOAD instructions
In reply to Martin Doucha--- runtest/kvm | 1 + testcases/kernel/kvm/.gitignore | 1 + testcases/kernel/kvm/kvm_svm04.c | 307 +++++++++++++++++++++++++++++++ 3 files changed, 309 insertions(+) create mode 100644 testcases/kernel/kvm/kvm_svm04.c
runtest/kvm +1 -0
@@ -2,3 +2,4 @@ kvm_pagefault01 kvm_pagefault01 kvm_svm01 kvm_svm01 kvm_svm02 kvm_svm02 kvm_svm03 kvm_svm03 +kvm_svm04 kvm_svm04
testcases/kernel/kvm/.gitignore +1 -0
@@ -2,3 +2,4 @@ /kvm_svm01 /kvm_svm02 /kvm_svm03 +/kvm_svm04
testcases/kernel/kvm/kvm_svm04.c new file +307 -0
307 lines not on this page · raw message
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 9/9 KVM: Move kvm_pagefault01 to the end of KVM runfile
In reply to Martin DouchaThe kvm_pagefault01 test reloads the KVM module with special parameters which may change kernel code paths and hide open bugs from other KVM tests. Move the test to the end of KVM schedule to prevent interference.
--- runtest/kvm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
runtest/kvm +2 -1
@@ -1,5 +1,6 @@ -kvm_pagefault01 kvm_pagefault01 kvm_svm01 kvm_svm01 kvm_svm02 kvm_svm02 kvm_svm03 kvm_svm03 kvm_svm04 kvm_svm04 +# Tests below may interfere with bug reproducibility +kvm_pagefault01 kvm_pagefault01
Signature · 5 lines
2.44.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 9/9] KVM: Move kvm_pagefault01 to the end of KVM runfile
Hi Martin,
Quoted 3 lines
The
kvm_pagefault01test reloads the KVM module with special parameters which may change kernel code paths and hide open bugs from other KVM tests. Move the test to the end of KVM schedule to prevent interference.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 9/9] KVM: Move kvm_pagefault01 to the end of KVM runfile
In reply to Martin DouchaHi Martin,
Quoted 3 lines
The
kvm_pagefault01test reloads the KVM module with special parameters which may change kernel code paths and hide open bugs from other KVM tests. Move the test to the end of KVM schedule to prevent interference.
Merged rebased, as we discussed at the first patch. Thanks!
On a brief look the other patches looks very well (apart from that clang issue), but I need some time for actual review.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp