selftests: kvm: fix sync_regs_test with newer gccs
PATCH 1/1 selftests: kvm: fix sync_regs_test with newer gccs
Commit 204c91eff798a ("KVM: selftests: do not blindly clobber registers in
guest asm") was intended to make test more gcc-proof, however, the result
is exactly the opposite: on newer gccs (e.g. 8.2.1) the test breaks with
==== Test Assertion Failure ====
x86_64/sync_regs_test.c:168: run->s.regs.regs.rbx == 0xBAD1DEA + 1 pid=14170 tid=14170 - Invalid argument
1 0x00000000004015b3: main at sync_regs_test.c:166 (discriminator 6)
2 0x00007f413fb66412: ?? ??:0
3 0x000000000040191d: _start at ??:? rbx sync regs value incorrect 0x1.
Apparently, compile is still free to play games with registers even when they have variables attaches.
Re-write guest code with 'asm volatile' by embedding ucall there and making sure rbx is preserved.
--- .../selftests/kvm/x86_64/sync_regs_test.c | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-)
tools/testing/selftests/kvm/x86_64/sync_regs_test.c +11 -10
@@ -22,18 +22,19 @@#define VCPU_ID 5 +#define UCALL_PIO_PORT ((uint16_t)0x1000) + +/* +* ucall is embedded here to protect against compiler reshuffling registers +* before calling a function. In this test we only need to get KVM_EXIT_IO +* vmexit and preserve RBX, no additional information is needed. +*/ void guest_code (void ) { -/* -* use a callee-save register, otherwise the compiler -* saves it around the call to GUEST_SYNC. -*/ - registeru32 stage asm("rbx" ); -for (;;) { -GUEST_SYNC (0 ); - stage++; - asmvolatile ("" : :"r" (stage)); - } + asmvolatile ("1: in %[port], %%al\n" +"add $0x1, %%rbx\n" +"jmp 1b" + : : [port]"d" (UCALL_PIO_PORT ) :"rax" ,"rbx" ); }static void compare_regs (struct kvm_regs *left,struct kvm_regs *right)
Signature · 1 line
2.20.1
Quoted 47 lines · Vitaly Kuznetsov
Commit
204c91eff798a("KVM: selftests: do not blindly clobber registers in
guest asm") was intended to make test more gcc-proof, however, the result
is exactly the opposite: on newer gccs (e.g. 8.2.1) the test breaks with==== Test Assertion Failure ====
x86_64/sync_regs_test.c:168:run->s.regs.regs.rbx==0xBAD1DEA+ 1 pid=14170 tid=14170 - Invalid argument1 0x00000000004015b3: main at sync_regs_test.c:166 (discriminator 6) 2 0x00007f413fb66412: ?? ??:0 3 0x000000000040191d: _start at ??:?rbx sync regs value incorrect
0x1.Apparently, compile is still free to play games with registers even when they have variables attaches.
Re-write guest code with 'asm volatile' by embedding ucall there and making sure rbx is preserved.
Fixes 204c91eff798a ("KVM: selftests: do not blindly clobber registers in guest asm")Signed-off-by Vitaly Kuznetsov <[email protected]>--- .../selftests/kvm/x86_64/sync_regs_test.c | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-)tools/testing/selftests/kvm/x86_64/sync_regs_test.c +7
-5@@ -22,18 +22,19 @@#define VCPU_ID 5 +#define UCALL_PIO_PORT ((uint16_t)0x1000) + +/* +* ucall is embedded here to protect against compiler reshuffling registers +* before calling a function. In this test we only need to get KVM_EXIT_IO +* vmexit and preserve RBX, no additional information is needed. +*/ void guest_code (void ) { -/* -* use a callee-save register, otherwise the compiler -* saves it around the call to GUEST_SYNC. -*/ - registeru32 stage asm("rbx" );
tools/testing/selftests/kvm/x86_64/sync_regs_test.c line 38
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/sync_regs_test.c +4
-5-for (;;) { -GUEST_SYNC (0 ); - stage++; - asmvolatile ("" : :"r" (stage)); - } + asmvolatile ("1: in %[port], %%al\n" +"add $0x1, %%rbx\n" +"jmp 1b" + : : [port]"d" (UCALL_PIO_PORT ) :"rax" ,"rbx" ); }
A better solution might be something like:
register u32 stage = 0;
for (;;) {
asm volatile("in %[port], %%al"
:
: "b" (stage), [port] "d" (UCALL_PIO_PORT)
: "rax");
stage++;
}(Gmail no doubt has mangled the indentation. Sorry.)
Quoted 69 lines · Jim Mattson
Quoted 47 lines · Vitaly Kuznetsov
Commit
204c91eff798a("KVM: selftests: do not blindly clobber registers in
guest asm") was intended to make test more gcc-proof, however, the result
is exactly the opposite: on newer gccs (e.g. 8.2.1) the test breaks with==== Test Assertion Failure ====
x86_64/sync_regs_test.c:168:run->s.regs.regs.rbx==0xBAD1DEA+ 1 pid=14170 tid=14170 - Invalid argument1 0x00000000004015b3: main at sync_regs_test.c:166 (discriminator 6) 2 0x00007f413fb66412: ?? ??:0 3 0x000000000040191d: _start at ??:?rbx sync regs value incorrect
0x1.Apparently, compile is still free to play games with registers even when they have variables attaches.
Re-write guest code with 'asm volatile' by embedding ucall there and making sure rbx is preserved.
Fixes 204c91eff798a ("KVM: selftests: do not blindly clobber registers in guest asm")Signed-off-by Vitaly Kuznetsov <[email protected]>--- .../selftests/kvm/x86_64/sync_regs_test.c | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-)tools/testing/selftests/kvm/x86_64/sync_regs_test.c +7
-5@@ -22,18 +22,19 @@#define VCPU_ID 5 +#define UCALL_PIO_PORT ((uint16_t)0x1000) + +/* +* ucall is embedded here to protect against compiler reshuffling registers +* before calling a function. In this test we only need to get KVM_EXIT_IO +* vmexit and preserve RBX, no additional information is needed. +*/ void guest_code (void ) { -/* -* use a callee-save register, otherwise the compiler -* saves it around the call to GUEST_SYNC. -*/ - registeru32 stage asm("rbx" );tools/testing/selftests/kvm/x86_64/sync_regs_test.c line 38
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/sync_regs_test.c +4
-5-for (;;) { -GUEST_SYNC (0 ); - stage++; - asmvolatile ("" : :"r" (stage)); - } + asmvolatile ("1: in %[port], %%al\n" +"add $0x1, %%rbx\n" +"jmp 1b" + : : [port]"d" (UCALL_PIO_PORT ) :"rax" ,"rbx" ); }A better solution might be something like:
register u32 stage = 0;
for (;;) { asm volatile("in %[port], %%al" : : "b" (stage), [port] "d" (UCALL_PIO_PORT) : "rax"); stage++; }
Quoted 1 line
(Gmail no doubt has mangled the indentation. Sorry.)
The incoming value of rbx matters, I believe it's set to 0xBAD1DEA and the host then expects 0xBAD1DEA + 1.
Quoted 46 lines · Vitaly Kuznetsov
Commit
204c91eff798a("KVM: selftests: do not blindly clobber registers in
guest asm") was intended to make test more gcc-proof, however, the result
is exactly the opposite: on newer gccs (e.g. 8.2.1) the test breaks with==== Test Assertion Failure ====
x86_64/sync_regs_test.c:168:run->s.regs.regs.rbx==0xBAD1DEA+ 1 pid=14170 tid=14170 - Invalid argument1 0x00000000004015b3: main at sync_regs_test.c:166 (discriminator 6) 2 0x00007f413fb66412: ?? ??:0 3 0x000000000040191d: _start at ??:?rbx sync regs value incorrect
0x1.Apparently, compile is still free to play games with registers even when they have variables attaches.
Re-write guest code with 'asm volatile' by embedding ucall there and making sure rbx is preserved.
Fixes 204c91eff798a ("KVM: selftests: do not blindly clobber registers in guest asm")Signed-off-by Vitaly Kuznetsov <[email protected]>--- .../selftests/kvm/x86_64/sync_regs_test.c | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-)tools/testing/selftests/kvm/x86_64/sync_regs_test.c +7
-5@@ -22,18 +22,19 @@#define VCPU_ID 5 +#define UCALL_PIO_PORT ((uint16_t)0x1000) + +/* +* ucall is embedded here to protect against compiler reshuffling registers +* before calling a function. In this test we only need to get KVM_EXIT_IO +* vmexit and preserve RBX, no additional information is needed. +*/ void guest_code (void ) { -/* -* use a callee-save register, otherwise the compiler -* saves it around the call to GUEST_SYNC. -*/ - registeru32 stage asm("rbx" );
tools/testing/selftests/kvm/x86_64/sync_regs_test.c line 38
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/sync_regs_test.c +4
-5-for (;;) { -GUEST_SYNC (0 ); - stage++; - asmvolatile ("" : :"r" (stage)); - } + asmvolatile ("1: in %[port], %%al\n" +"add $0x1, %%rbx\n" +"jmp 1b" + : : [port]"d" (UCALL_PIO_PORT ) :"rax" ,"rbx" ); }
To make the code truly bulletproof, is it possible to rename guest_code() to guest_code_wrapper() and then export 1: as guest_code? VM-Enter will jump directly to the relevant code and gcc can't touch rbx. E.g.:
asm volatile("1: ..."
".global guest_code"
"guest_code: " _ASM_PTR " 1b");Not sure if that works with how the selftests are compiled. It may also be possible to simply replace '1' with 'guest_code'.
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/sync_regs_test.c +0
-0static void compare_regs (struct kvm_regs *left,struct kvm_regs *right)Signature · 1 line
2.20.1
Paolo Bonzini
maintainerQuoted 62 lines · Sean Christopherson
Quoted 46 lines · Vitaly Kuznetsov
Commit
204c91eff798a("KVM: selftests: do not blindly clobber registers in
guest asm") was intended to make test more gcc-proof, however, the result
is exactly the opposite: on newer gccs (e.g. 8.2.1) the test breaks with==== Test Assertion Failure ====
x86_64/sync_regs_test.c:168:run->s.regs.regs.rbx==0xBAD1DEA+ 1 pid=14170 tid=14170 - Invalid argument1 0x00000000004015b3: main at sync_regs_test.c:166 (discriminator 6) 2 0x00007f413fb66412: ?? ??:0 3 0x000000000040191d: _start at ??:?rbx sync regs value incorrect
0x1.Apparently, compile is still free to play games with registers even when they have variables attaches.
Re-write guest code with 'asm volatile' by embedding ucall there and making sure rbx is preserved.
Fixes 204c91eff798a ("KVM: selftests: do not blindly clobber registers in guest asm")Signed-off-by Vitaly Kuznetsov <[email protected]>--- .../selftests/kvm/x86_64/sync_regs_test.c | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-)tools/testing/selftests/kvm/x86_64/sync_regs_test.c +7
-5@@ -22,18 +22,19 @@#define VCPU_ID 5 +#define UCALL_PIO_PORT ((uint16_t)0x1000) + +/* +* ucall is embedded here to protect against compiler reshuffling registers +* before calling a function. In this test we only need to get KVM_EXIT_IO +* vmexit and preserve RBX, no additional information is needed. +*/ void guest_code (void ) { -/* -* use a callee-save register, otherwise the compiler -* saves it around the call to GUEST_SYNC. -*/ - registeru32 stage asm("rbx" );tools/testing/selftests/kvm/x86_64/sync_regs_test.c line 38
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/sync_regs_test.c +4
-5-for (;;) { -GUEST_SYNC (0 ); - stage++; - asmvolatile ("" : :"r" (stage)); - } + asmvolatile ("1: in %[port], %%al\n" +"add $0x1, %%rbx\n" +"jmp 1b" + : : [port]"d" (UCALL_PIO_PORT ) :"rax" ,"rbx" ); }To make the code truly bulletproof, is it possible to rename
guest_code()toguest_code_wrapper()and then export 1: asguest_code? VM-Enter will jump directly to the relevant code and gcc can't touch rbx. E.g.:
Quoted 6 lines
asm volatile("1: ..." ".global guest_code" "guest_code: " _ASM_PTR " 1b");Not sure if that works with how the selftests are compiled. It may also be possible to simply replace '1' with '
guest_code'.
There is no practical difference with Vitaly's patch. The first _vcpu_run has no pre-/post-conditions on the value of %rbx:
run->kvm_valid_regs = TEST_SYNC_FIELDS;
rv = _vcpu_run(vm, VCPU_ID);
TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
"Unexpected exit reason: %u (%s),\n",
run->exit_reason,
exit_reason_str(run->exit_reason));
/*
* Then it goes on comparing regs/sregs/events, but does not
* check for specific values.
*/As soon as that first _vcpu_run succeeds, you're stuck in the in/add/jmp loop and the compiler can't trick you anymore.
So, I'm queuing the patch.
Paolo
Quoted 67 lines · Paolo Bonzini
Quoted 66 lines · Sean Christopherson
Quoted 46 lines · Vitaly Kuznetsov
Commit
204c91eff798a("KVM: selftests: do not blindly clobber registers in
guest asm") was intended to make test more gcc-proof, however, the result
is exactly the opposite: on newer gccs (e.g. 8.2.1) the test breaks with==== Test Assertion Failure ====
x86_64/sync_regs_test.c:168:run->s.regs.regs.rbx==0xBAD1DEA+ 1 pid=14170 tid=14170 - Invalid argument1 0x00000000004015b3: main at sync_regs_test.c:166 (discriminator 6) 2 0x00007f413fb66412: ?? ??:0 3 0x000000000040191d: _start at ??:?rbx sync regs value incorrect
0x1.Apparently, compile is still free to play games with registers even when they have variables attaches.
Re-write guest code with 'asm volatile' by embedding ucall there and making sure rbx is preserved.
Fixes 204c91eff798a ("KVM: selftests: do not blindly clobber registers in guest asm")Signed-off-by Vitaly Kuznetsov <[email protected]>--- .../selftests/kvm/x86_64/sync_regs_test.c | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-)tools/testing/selftests/kvm/x86_64/sync_regs_test.c +7
-5@@ -22,18 +22,19 @@#define VCPU_ID 5 +#define UCALL_PIO_PORT ((uint16_t)0x1000) + +/* +* ucall is embedded here to protect against compiler reshuffling registers +* before calling a function. In this test we only need to get KVM_EXIT_IO +* vmexit and preserve RBX, no additional information is needed. +*/ void guest_code (void ) { -/* -* use a callee-save register, otherwise the compiler -* saves it around the call to GUEST_SYNC. -*/ - registeru32 stage asm("rbx" );tools/testing/selftests/kvm/x86_64/sync_regs_test.c line 38
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/sync_regs_test.c +4
-5-for (;;) { -GUEST_SYNC (0 ); - stage++; - asmvolatile ("" : :"r" (stage)); - } + asmvolatile ("1: in %[port], %%al\n" +"add $0x1, %%rbx\n" +"jmp 1b" + : : [port]"d" (UCALL_PIO_PORT ) :"rax" ,"rbx" ); }To make the code truly bulletproof, is it possible to rename
guest_code()toguest_code_wrapper()and then export 1: asguest_code? VM-Enter will jump directly to the relevant code and gcc can't touch rbx. E.g.:asm volatile("1: ..." ".global guest_code" "guest_code: " _ASM_PTR " 1b");
Quoted 6 lines
Quoted 2 lines
Not sure if that works with how the selftests are compiled. It may also be possible to simply replace '1' with '
guest_code'.There is no practical difference with Vitaly's patch. The first
_vcpu_runhas no pre-/post-conditions on the value of %rbx:
I think what Sean was suggesting is to prevent GCC from inserting anything (and thus clobbering RBX) between the call to guest_call() and the beginning of 'asm volatile' block by calling *inside* 'asm volatile' block instead.
Quoted 13 lines
run->kvm_valid_regs = TEST_SYNC_FIELDS;
rv = _vcpu_run(vm, VCPU_ID);
TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
"Unexpected exit reason: %u (%s),\n",
run->exit_reason,
exit_reason_str(run->exit_reason));
/*
* Then it goes on comparing regs/sregs/events, but does not
* check for specific values.
*/Quoted 5 lines
As soon as that first
_vcpu_runsucceeds, you're stuck in the in/add/jmp loop and the compiler can't trick you anymore.So, I'm queuing the patch.
Thanks!
Signature · 1 line
Vitaly
Paolo Bonzini
maintainerQuoted 8 lines · Vitaly Kuznetsov
Quoted 2 lines · Paolo Bonzini
There is no practical difference with Vitaly's patch. The first
_vcpu_runhas no pre-/post-conditions on the value of %rbx:I think what Sean was suggesting is to prevent GCC from inserting anything (and thus clobbering RBX) between the call to
guest_call()and the beginning of 'asm volatile' block by calling *inside* 'asm volatile' block instead.
Yes, but there is no way that clobbering RBX will break the test, because RBX is not initialized until after the first _vcpu_run succeeds.
Paolo
Quoted 5 lines · Paolo Bonzini
Quoted 4 lines · Vitaly Kuznetsov
Quoted 2 lines · Paolo Bonzini
There is no practical difference with Vitaly's patch. The first
_vcpu_runhas no pre-/post-conditions on the value of %rbx:
Quoted 8 lines
Quoted 4 lines
I think what Sean was suggesting is to prevent GCC from inserting anything (and thus clobbering RBX) between the call to
guest_call()and the beginning of 'asm volatile' block by calling *inside* 'asm volatile' block instead.Yes, but there is no way that clobbering RBX will break the test, because RBX is not initialized until after the first
_vcpu_runsucceeds.
Right, we're always resuming after so potential clobbering doesn't matter. Thanks!
Signature · 1 line
Vitaly
Quoted 5 lines · Paolo Bonzini
Quoted 4 lines · Vitaly Kuznetsov
Quoted 2 lines · Paolo Bonzini
There is no practical difference with Vitaly's patch. The first
_vcpu_runhas no pre-/post-conditions on the value of %rbx:
Quoted 7 lines
Quoted 4 lines
I think what Sean was suggesting is to prevent GCC from inserting anything (and thus clobbering RBX) between the call to
guest_call()and the beginning of 'asm volatile' block by calling *inside* 'asm volatile' block instead.Yes, but there is no way that clobbering RBX will break the test, because RBX is not initialized until after the first
_vcpu_runsucceeds.
Ah, nice, wasn't aware of that.