KVM: x86: Use asm_inline() instead of asm() in kvm_hypercall[0-4]()
Subsystems (4)
PATCH 1/2 KVM: x86: Use asm_inline() instead of asm() in kvm_hypercall[0-4]()
Use asm_inline() to instruct the compiler that the size of asm() is the minimum size of one instruction, ignoring how many instructions the compiler thinks it is. ALTERNATIVE macro that expands to several pseudo directives causes instruction length estimate to count more than 20 instructions.
bloat-o-meter reports minimal code size increase
(x86_64 defconfig, gcc-14.2.1):
add/remove: 0/0 grow/shrink: 1/0 up/down: 10/0 (10)
Function old new delta
-----------------------------------------------------
__send_ipi_mask 525 535 +10
Total: Before=23751224, After=23751234, chg +0.00%
due to different compiler decisions with more precise size estimations.
No functional change intended.
--- arch/x86/include/asm/kvm_para.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/include/asm/kvm_para.h +5 -5
@@ -38,7 +38,7 @@ static inline long kvm_hypercall0(unsigned int nr)if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr,0 ,0 ,0 ,0 ); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr) : "memory");
@@ -52,7 +52,7 @@ static inline long kvm_hypercall1(unsigned int nr, unsigned long p1)if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr, p1,0 ,0 ,0 ); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1) : "memory");
@@ -67,7 +67,7 @@ static inline long kvm_hypercall2(unsigned int nr, unsigned long p1,if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr, p1, p2,0 ,0 ); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2) : "memory");
@@ -82,7 +82,7 @@ static inline long kvm_hypercall3(unsigned int nr, unsigned long p1,if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr, p1, p2, p3,0 ); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2), "d"(p3) : "memory");
@@ -98,7 +98,7 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned long p1,if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr, p1, p2, p3, p4); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2), "d"(p3), "S"(p4) : "memory");
Signature · 1 line
2.49.0
PATCH 2/2 KVM: VMX: Use LEAVE in vmx_do_interrupt_irqoff()
Micro-optimize vmx_do_interrupt_irqoff() by substituting MOV %RBP,%RSP; POP %RBP instruction sequence with equivalent LEAVE instruction. GCC compiler does this by default for a generic tuning and for all modern processors:
DEF_TUNE (X86_TUNE_USE_LEAVE, "use_leave",
m_386 | m_CORE_ALL | m_K6_GEODE | m_AMD_MULTIPLE | m_ZHAOXIN | m_TREMONT | m_CORE_HYBRID | m_CORE_ATOM | m_GENERIC)
The new code also saves a couple of bytes, from:
27: 48 89 ec mov %rbp,%rsp 2a: 5d pop %rbp
to:
27: c9 leave
No functional change intended.
--- arch/x86/kvm/vmx/vmenter.S | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/kvm/vmx/vmenter.S +1 -2
@@ -59,8 +59,7 @@ * without the explicit restore, thinks the stack is getting walloped. * Using an unwind hint is problematic due to x86-64's dynamic alignment. */ - mov %_ASM_BP, %_ASM_SP - pop %_ASM_BP + leave RET .endm
Signature · 1 line
2.49.0
Sean Christopherson
maintainerRe: [PATCH 2/2] KVM: VMX: Use LEAVE in vmx_do_interrupt_irqoff()
Quoted 4 lines · Uros Bizjak
Micro-optimize
vmx_do_interrupt_irqoff()by substituting MOV %RBP,%RSP; POP %RBP instruction sequence with equivalent LEAVE instruction. GCC compiler does this by default for a generic tuning and for all modern processors:
Out of curisoity, is LEAVE actually a performance win, or is the benefit essentially just the few code bytes saves?
Quoted 45 lines
DEF_TUNE(X86_TUNE_USE_LEAVE, "use_leave",m_386 | m_CORE_ALL | m_K6_GEODE | m_AMD_MULTIPLE | m_ZHAOXIN | m_TREMONT | m_CORE_HYBRID | m_CORE_ATOM | m_GENERIC)The new code also saves a couple of bytes, from:
27: 48 89 ec mov %rbp,%rsp 2a: 5d pop %rbpto:
27: c9 leave
No functional change intended.
Signed-off-by Uros Bizjak <[email protected]>Cc Sean Christopherson <[email protected]>Cc Paolo Bonzini <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc "H. Peter Anvin" <[email protected]>--- arch/x86/kvm/vmx/vmenter.S | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)arch/x86/kvm/vmx/vmenter.S +1
-2@@ -59,8 +59,7 @@ * without the explicit restore, thinks the stack is getting walloped. * Using an unwind hint is problematic due to x86-64's dynamic alignment. */ - mov %_ASM_BP, %_ASM_SP - pop %_ASM_BP + leave RET .endmSignature · 1 line
2.49.0
Re: [PATCH 2/2] KVM: VMX: Use LEAVE in vmx_do_interrupt_irqoff()
Quoted 9 lines · Sean Christopherson
Quoted 4 lines · Uros Bizjak
Micro-optimize
vmx_do_interrupt_irqoff()by substituting MOV %RBP,%RSP; POP %RBP instruction sequence with equivalent LEAVE instruction. GCC compiler does this by default for a generic tuning and for all modern processors:Out of curisoity, is LEAVE actually a performance win, or is the benefit essentially just the few code bytes saves?
It is hard to say for out-of-order execution cores, especially when the stack engine is thrown to the mix (these two instructions, plus following RET, all update %rsp).
The pragmatic solution was to do what the compiler does and use the compiler's choice, based on the tuning below.
Quoted 3 lines
Quoted 3 lines
DEF_TUNE(X86_TUNE_USE_LEAVE, "use_leave",
m_386|m_CORE_ALL|m_K6_GEODE|m_AMD_MULTIPLE|m_ZHAOXIN|m_TREMONT|m_CORE_HYBRID|m_CORE_ATOM|m_GENERIC)
The tuning is updated when a new target is introduced to the compiler and is based on various measurements by the processor manufacturer. The above covers the majority of recent processors (plus generic tuning), so I guess we won't fail by following the suit. OTOH, any performance difference will be negligible.
Quoted 8 lines
Quoted 8 lines
The new code also saves a couple of bytes, from:
27: 48 89 ec mov %rbp,%rsp 2a: 5d pop %rbpto:
27: c9 leave
Thanks,
Uros.
Sean Christopherson
maintainerNit, this is guest code, i.e. should use "kvm/x86:" for the scope. No need to send a new version just for that.
Quoted 87 lines · Uros Bizjak
Use
asm_inline()to instruct the compiler that the size ofasm()is the minimum size of one instruction, ignoring how many instructions the compiler thinks it is. ALTERNATIVE macro that expands to several pseudo directives causes instruction length estimate to count more than 20 instructions.bloat-o-meter reports minimal code size increase
(x86_64defconfig, gcc-14.2.1):add/remove: 0/0 grow/shrink: 1/0 up/down: 10/0 (10)
Function old new delta
-----------------------------------------------------
__send_ipi_mask525 535 +10Total: Before=23751224, After=23751234, chg +0.00%
due to different compiler decisions with more precise size estimations.
No functional change intended.
Signed-off-by Uros Bizjak <[email protected]>Cc Sean Christopherson <[email protected]>Cc Paolo Bonzini <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc "H. Peter Anvin" <[email protected]>--- arch/x86/include/asm/kvm_para.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)arch/x86/include/asm/kvm_para.h +5
-5@@ -38,7 +38,7 @@ static inline long kvm_hypercall0(unsigned int nr)if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr,0 ,0 ,0 ,0 ); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr) : "memory");@@ -52,7 +52,7 @@ static inline long kvm_hypercall1(unsigned int nr, unsigned long p1)if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr, p1,0 ,0 ,0 ); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1) : "memory");@@ -67,7 +67,7 @@ static inline long kvm_hypercall2(unsigned int nr, unsigned long p1,if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr, p1, p2,0 ,0 ); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2) : "memory");@@ -82,7 +82,7 @@ static inline long kvm_hypercall3(unsigned int nr, unsigned long p1,if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr, p1, p2, p3,0 ); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2), "d"(p3) : "memory");@@ -98,7 +98,7 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned long p1,if (cpu_feature_enabled (X86_FEATURE_TDX_GUEST ))return tdx_kvm_hypercall (nr, p1, p2, p3, p4); - asm volatile(KVM_HYPERCALL + asm_inline volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2), "d"(p3), "S"(p4) : "memory");Signature · 1 line
2.49.0
Quoted 10 lines · Uros Bizjak
Use
asm_inline()to instruct the compiler that the size ofasm()is the minimum size of one instruction, ignoring how many instructions the compiler thinks it is. ALTERNATIVE macro that expands to several pseudo directives causes instruction length estimate to count more than 20 instructions.bloat-o-meter reports minimal code size increase
(x86_64defconfig, gcc-14.2.1):[...]
Applied patch 2 to kvm-x86 vmx (I'll let Paolo grab the guest change). Thanks!
[1/2] KVM: x86: Use asm_inline() instead of asm() in kvm_hypercall[0-4]()
(no commit info)
[2/2] KVM: VMX: Use LEAVE in vmx_do_interrupt_irqoff()
https://github.com/kvm-x86/linux/commit/798b9b1cb0e5