VMSCAPE optimization for BHI variant
Subsystems (6)
Pawan Gupta
reviewerv2 cover VMSCAPE optimization for BHI variant
v2:
- Added check for IBPB feature in vmscape_select_mitigation(). (David)
- s/vmscape=auto/vmscape=on/ (David)
- Added patch to remove LFENCE from VMSCAPE BHB-clear sequence.
- Rebased to v6.18-rc1.
v1: https://lore.kernel.org/r/[email protected]
Hi All,
These patches aim to improve the performance of a recent mitigation for VMSCAPE[1] vulnerability. This improvement is relevant for BHI variant of VMSCAPE that affect Alder Lake and newer processors.
The current mitigation approach uses IBPB on kvm-exit-to-userspace for all affected range of CPUs. This is an overkill for CPUs that are only affected by the BHI variant. On such CPUs clearing the branch history is sufficient for VMSCAPE, and also more apt as the underlying issue is due to poisoned branch history.
Roadmap:
- First patch introduces clear_bhb_long_loop() for processors with larger branch history tables.
- Second patch replaces IBPB on exit-to-userspace with branch history clearing sequence.
Below is the iPerf data for transfer between guest and host, comparing IBPB and BHB-clear mitigation. BHB-clear shows performance improvement over IBPB in most cases.
Platform: Emerald Rapids
Baseline: vmscape=off
(pN = N parallel connections)
| iPerf user-net | IBPB | BHB Clear | |----------------|---------|-----------| | UDP 1-vCPU_p1 | -12.5% | 1.3% | | TCP 1-vCPU_p1 | -10.4% | -1.5% | | TCP 1-vCPU_p1 | -7.5% | -3.0% | | UDP 4-vCPU_p16 | -3.7% | -3.7% | | TCP 4-vCPU_p4 | -2.9% | -1.4% | | UDP 4-vCPU_p4 | -0.6% | 0.0% | | TCP 4-vCPU_p4 | 3.5% | 0.0% |
| iPerf bridge-net | IBPB | BHB Clear | |------------------|---------|-----------| | UDP 1-vCPU_p1 | -9.4% | -0.4% | | TCP 1-vCPU_p1 | -3.9% | -0.5% | | UDP 4-vCPU_p16 | -2.2% | -3.8% | | TCP 4-vCPU_p4 | -1.0% | -1.0% | | TCP 4-vCPU_p4 | 0.5% | 0.5% | | UDP 4-vCPU_p4 | 0.0% | 0.9% | | TCP 1-vCPU_p1 | 0.0% | 0.9% |
| iPerf vhost-net | IBPB | BHB Clear | |-----------------|---------|-----------| | UDP 1-vCPU_p1 | -4.3% | 1.0% | | TCP 1-vCPU_p1 | -3.8% | -0.5% | | TCP 1-vCPU_p1 | -2.7% | -0.7% | | UDP 4-vCPU_p16 | -0.7% | -2.2% | | TCP 4-vCPU_p4 | -0.4% | 0.8% | | UDP 4-vCPU_p4 | 0.4% | -0.7% | | TCP 4-vCPU_p4 | 0.0% | 0.6% |
---
Pawan Gupta (3):
x86/bhi: Add BHB clearing for CPUs with larger branch history x86/vmscape: Replace IBPB with branch history clear on exit to userspace x86/vmscape: Remove LFENCE from BHB clearing long loop
Documentation/admin-guide/hw-vuln/vmscape.rst | 8 ++++ Documentation/admin-guide/kernel-parameters.txt | 4 +- arch/x86/entry/entry_64.S | 63 ++++++++++++++++++------- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/entry-common.h | 12 +++-- arch/x86/include/asm/nospec-branch.h | 5 +- arch/x86/kernel/cpu/bugs.c | 53 +++++++++++++++------ arch/x86/kvm/x86.c | 5 +- 8 files changed, 110 insertions(+), 41 deletions(-)
---
base-commit: 3a8660878839faadb4f1a6dd72c3179c1df56787
change-id: 20250916-vmscape-bhb-d7d469977f2f
Best regards,
Signature · 1 line
Pawan
Pawan Gupta
reviewerPATCH v2 1/3 x86/bhi: Add BHB clearing for CPUs with larger branch history
Add a version of clear_bhb_loop() that works on CPUs with larger branch history table such as Alder Lake and newer. This could serve as a cheaper alternative to IBPB mitigation for VMSCAPE.
clear_bhb_loop() and the new clear_bhb_long_loop() only differ in the loop counter. Convert the asm implementation of clear_bhb_loop() into a macro that is used by both the variants, passing counter as an argument.
There is no difference in the output of:
$ objdump --disassemble=clear_bhb_loop vmlinux
before and after this commit.
--- arch/x86/entry/entry_64.S | 47 ++++++++++++++++++++++++++---------- arch/x86/include/asm/nospec-branch.h | 3 +++ 2 files changed, 37 insertions(+), 13 deletions(-)
arch/x86/entry/entry_64.S +34 -13
@@ -1499,11 +1499,6 @@ SYM_CODE_END(rewind_stack_and_make_dead) * from the branch history tracker in the Branch Predictor, therefore removing * user influence on subsequent BTB lookups. * - * It should be used on parts prior to Alder Lake. Newer parts should use the - * BHI_DIS_S hardware control instead. If a pre-Alder Lake part is being - * virtualized on newer hardware the VMM should protect against BHI attacks by - * setting BHI_DIS_S for the guests. - * * CALLs/RETs are necessary to prevent Loop Stream Detector(LSD) from engaging * and not clearing the branch history. The call tree looks like: *
@@ -1529,11 +1524,12 @@ SYM_CODE_END(rewind_stack_and_make_dead) * that all RETs are in the second half of a cacheline to mitigate Indirect * Target Selection, rather than taking the slowpath via its_return_thunk. */ -SYM_FUNC_START(clear_bhb_loop) +.macro __CLEAR_BHB_LOOP outer_loop_count:req, inner_loop_count:req ANNOTATE_NOENDBR push %rbp mov %rsp, %rbp - movl $5, %ecx + + movl $\outer_loop_count, %ecx ANNOTATE_INTRA_FUNCTION_CALL call 1f jmp 5f
@@ -1542,29 +1538,54 @@ SYM_FUNC_START(clear_bhb_loop) * Shift instructions so that the RET is in the upper half of the * cacheline and don't take the slowpath to its_return_thunk. */ - .skip 32 - (.Lret1 - 1f), 0xcc + .skip 32 - (.Lret1_\@ - 1f), 0xcc ANNOTATE_INTRA_FUNCTION_CALL 1: call 2f -.Lret1: RET +.Lret1_\@: + RET .align 64, 0xcc /* - * As above shift instructions for RET at .Lret2 as well. + * As above shift instructions for RET at .Lret2_\@ as well. * - * This should be ideally be: .skip 32 - (.Lret2 - 2f), 0xcc + * This should be ideally be: .skip 32 - (.Lret2_\@ - 2f), 0xcc * but some Clang versions (e.g. 18) don't like this. */ .skip 32 - 18, 0xcc -2: movl $5, %eax +2: movl $\inner_loop_count, %eax 3: jmp 4f nop 4: sub $1, %eax jnz 3b sub $1, %ecx jnz 1b -.Lret2: RET +.Lret2_\@: + RET 5: lfence + pop %rbp RET +.endm + +/* + * This should be used on parts prior to Alder Lake. Newer parts should use the + * BHI_DIS_S hardware control instead. If a pre-Alder Lake part is being + * virtualized on newer hardware the VMM should protect against BHI attacks by + * setting BHI_DIS_S for the guests. + */ +SYM_FUNC_START(clear_bhb_loop) + __CLEAR_BHB_LOOP 5, 5 SYM_FUNC_END(clear_bhb_loop) EXPORT_SYMBOL_GPL(clear_bhb_loop) STACK_FRAME_NON_STANDARD(clear_bhb_loop) + +/* + * A longer version of clear_bhb_loop to ensure that the BHB is cleared on CPUs + * with larger branch history tables (i.e. Alder Lake and newer). BHI_DIS_S + * protects the kernel, but to mitigate the guest influence on the host + * userspace either IBPB or this sequence should be used. See VMSCAPE bug. + */ +SYM_FUNC_START(clear_bhb_long_loop) + __CLEAR_BHB_LOOP 12, 7 +SYM_FUNC_END(clear_bhb_long_loop) +EXPORT_SYMBOL_GPL(clear_bhb_long_loop) +STACK_FRAME_NON_STANDARD(clear_bhb_long_loop)
arch/x86/include/asm/nospec-branch.h +3 -0
@@ -388,6 +388,9 @@ extern void write_ibpb(void);#ifdef CONFIG_X86_64 extern void clear_bhb_loop (void ); +extern void clear_bhb_long_loop (void ); +#else +static inline void clear_bhb_long_loop (void ) {}#endif extern void (*x86_return_thunk)(void );
Signature · 1 line
2.34.1
Pawan Gupta
reviewerPATCH v2 2/3 x86/vmscape: Replace IBPB with branch history clear on exit to userspace
In reply to Pawan GuptaIBPB mitigation for VMSCAPE is an overkill for CPUs that are only affected by the BHI variant of VMSCAPE. On such CPUs, eIBRS already provides indirect branch isolation between guest and host userspace. But, a guest could still poison the branch history.
To mitigate that, use the recently added clear_bhb_long_loop() to isolate the branch history between guest and userspace. Add cmdline option 'vmscape=on' that automatically selects the appropriate mitigation based on the CPU.
--- Documentation/admin-guide/hw-vuln/vmscape.rst | 8 ++++ Documentation/admin-guide/kernel-parameters.txt | 4 +- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/entry-common.h | 12 +++--- arch/x86/include/asm/nospec-branch.h | 2 +- arch/x86/kernel/cpu/bugs.c | 53 ++++++++++++++++++------- arch/x86/kvm/x86.c | 5 ++- 7 files changed, 61 insertions(+), 24 deletions(-)
Documentation/admin-guide/hw-vuln/vmscape.rst +8 -0
@@ -86,6 +86,10 @@ The possible values in this file are: run a potentially malicious guest and issues an IBPB before the first exit to userspace after VM-exit. + * 'Mitigation: Clear BHB before exit to userspace': + + As above, conditional BHB clearing mitigation is enabled. + * 'Mitigation: IBPB on VMEXIT': IBPB is issued on every VM-exit. This occurs when other mitigations like
@@ -108,3 +112,7 @@ The mitigation can be controlled via the ``vmscape=`` command line parameter: Force vulnerability detection and mitigation even on processors that are not known to be affected. + + * ``vmscape=on``: + + Choose the mitigation based on the VMSCAPE variant the CPU is affected by.
Documentation/admin-guide/kernel-parameters.txt +3 -1
@@ -8104,9 +8104,11 @@ off - disable the mitigation ibpb - use Indirect Branch Prediction Barrier - (IBPB) mitigation (default) + (IBPB) mitigation force - force vulnerability detection even on unaffected processors + on - (default) automatically select IBPB + or BHB clear mitigation based on CPU vsyscall= [X86-64,EARLY] Controls the behavior of vsyscalls (i.e. calls to
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -499,6 +499,7 @@#define X86_FEATURE_IBPB_EXIT_TO_USER (21*32+14)/* Use IBPB on exit-to-userspace, see VMSCAPE bug */ #define X86_FEATURE_ABMC (21*32+15)/* Assignable Bandwidth Monitoring Counters */ #define X86_FEATURE_MSR_IMM (21*32+16)/* MSR immediate form instructions */ +#define X86_FEATURE_CLEAR_BHB_EXIT_TO_USER (21*32+17)/* Clear branch history on exit-to-userspace, see VMSCAPE bug */ /* * BUG word(s)
arch/x86/include/asm/entry-common.h +7 -5
@@ -94,11 +94,13 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs,*/ choose_random_kstack_offset (rdtsc ()); -/* Avoid unnecessary reads of 'x86_ibpb_exit_to_user' */ -if (cpu_feature_enabled (X86_FEATURE_IBPB_EXIT_TO_USER ) && -this_cpu_read (x86_ibpb_exit_to_user)) { -indirect_branch_prediction_barrier (); -this_cpu_write (x86_ibpb_exit_to_user, false); +if (unlikely (this_cpu_read (x86_pred_flush_pending))) { +if (cpu_feature_enabled (X86_FEATURE_IBPB_EXIT_TO_USER )) +indirect_branch_prediction_barrier (); +else if (cpu_feature_enabled (X86_FEATURE_CLEAR_BHB_EXIT_TO_USER )) +clear_bhb_long_loop (); + +this_cpu_write (x86_pred_flush_pending, false); } }#define arch_exit_to_user_mode_prepare arch_exit_to_user_mode_prepare
arch/x86/include/asm/nospec-branch.h +1 -1
@@ -534,7 +534,7 @@ void alternative_msr_write(unsigned int msr, u64 val, unsigned int feature) : "memory"); } -DECLARE_PER_CPU (bool, x86_ibpb_exit_to_user); +DECLARE_PER_CPU (bool, x86_pred_flush_pending);static inline void indirect_branch_prediction_barrier (void ) {
arch/x86/kernel/cpu/bugs.c +38 -15
99 lines not on this page · raw message
arch/x86/kvm/x86.c +3 -2
@@ -11397,8 +11397,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)* set for the CPU that actually ran the guest, and not the CPU that it * may migrate to. */ -if (cpu_feature_enabled (X86_FEATURE_IBPB_EXIT_TO_USER )) -this_cpu_write (x86_ibpb_exit_to_user, true); +if (cpu_feature_enabled (X86_FEATURE_IBPB_EXIT_TO_USER ) || +cpu_feature_enabled (X86_FEATURE_CLEAR_BHB_EXIT_TO_USER )) +this_cpu_write (x86_pred_flush_pending, true);/* * Consume any pending interrupts, including the possible source of
Signature · 1 line
2.34.1
Sean Christopherson
maintainerRe: [PATCH v2 2/3] x86/vmscape: Replace IBPB with branch history clear on exit to userspace
arch/x86/include/asm/nospec-branch.h void alternative_msr_write(unsigned int msr, u64 val, unsigned int feature) · line 537
Quoted 10 lines · Pawan Gupta
arch/x86/include/asm/nospec-branch.h +1
-1@@ -534,7 +534,7 @@ void alternative_msr_write(unsigned int msr, u64 val, unsigned int feature) : "memory"); } -DECLARE_PER_CPU (bool, x86_ibpb_exit_to_user); +DECLARE_PER_CPU (bool, x86_pred_flush_pending);
Rather than "flush pending", what about using "need" in the name to indicate that a flush is necessary? That makes it more obvious that e.g. KVM is marking the CPU as needing a flush by some other code, as opposed to implying that KVM itself has a pending flush.
And maybe spell out "prediction"? Without the context of features being checked, I don't know that I would be able to guess "prediction".
E.g. x86_need_prediction_flush?
Or x86_prediction_flush_exit_to_user if we would prefer to clarify when the flush needs to occur?
Pawan Gupta
reviewerRe: [PATCH v2 2/3] x86/vmscape: Replace IBPB with branch history clear on exit to userspace
Quoted 17 lines · Sean Christopherson
arch/x86/include/asm/nospec-branch.h void alternative_msr_write(unsigned int msr, u64 val, unsigned int feature) · line 537
Quoted 10 lines · Pawan Gupta
arch/x86/include/asm/nospec-branch.h +1
-1@@ -534,7 +534,7 @@ void alternative_msr_write(unsigned int msr, u64 val, unsigned int feature) : "memory"); } -DECLARE_PER_CPU (bool, x86_ibpb_exit_to_user); +DECLARE_PER_CPU (bool, x86_pred_flush_pending);Rather than "flush pending", what about using "need" in the name to indicate that a flush is necessary? That makes it more obvious that e.g. KVM is marking the CPU as needing a flush by some other code, as opposed to implying that KVM itself has a pending flush.
Quoted 7 lines
And maybe spell out "prediction"? Without the context of features being checked, I don't know that I would be able to guess "prediction".
E.g.
x86_need_prediction_flush?Or
x86_prediction_flush_exit_to_userif we would prefer to clarify when the flush needs to occur?
Ok, ya this is more clear. I would want to make a small change, instead of "prediction_flush", "predictor_flush" reads better to me. Changing it to:x86_predictor_flush_exit_to_user.
Sean Christopherson
maintainerRe: [PATCH v2 2/3] x86/vmscape: Replace IBPB with branch history clear on exit to userspace
Quoted 21 lines · Pawan Gupta
Quoted 20 lines · Sean Christopherson
arch/x86/include/asm/nospec-branch.h void alternative_msr_write(unsigned int msr, u64 val, unsigned int feature) · line 537
Quoted 10 lines · Pawan Gupta
arch/x86/include/asm/nospec-branch.h +1
-1@@ -534,7 +534,7 @@ void alternative_msr_write(unsigned int msr, u64 val, unsigned int feature) : "memory"); } -DECLARE_PER_CPU (bool, x86_ibpb_exit_to_user); +DECLARE_PER_CPU (bool, x86_pred_flush_pending);Rather than "flush pending", what about using "need" in the name to indicate that a flush is necessary? That makes it more obvious that e.g. KVM is marking the CPU as needing a flush by some other code, as opposed to implying that KVM itself has a pending flush.
And maybe spell out "prediction"? Without the context of features being checked, I don't know that I would be able to guess "prediction".
Quoted 8 lines
Quoted 4 lines
E.g.
x86_need_prediction_flush?Or
x86_prediction_flush_exit_to_userif we would prefer to clarify when the flush needs to occur?Ok, ya this is more clear. I would want to make a small change, instead of "
prediction_flush", "predictor_flush" reads better to me. Changing it to:x86_predictor_flush_exit_to_user.
LOL, see, told you I couldn't guest the word. :-D
"predictor" is way better, thanks!
Pawan Gupta
reviewerRe: [PATCH v2 2/3] x86/vmscape: Replace IBPB with branch history clear on exit to userspace
In reply to Pawan GuptaQuoted 9 lines · Pawan Gupta
IBPB mitigation for VMSCAPE is an overkill for CPUs that are only affected by the BHI variant of VMSCAPE. On such CPUs, eIBRS already provides indirect branch isolation between guest and host userspace. But, a guest could still poison the branch history.
To mitigate that, use the recently added
clear_bhb_long_loop()to isolate the branch history between guest and userspace. Add cmdline option 'vmscape=on' that automatically selects the appropriate mitigation based on the CPU.
[...]
Quoted 7 lines
arch/x86/include/asm/entry-common.h +0
-0@@ -94,11 +94,13 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs,*/ choose_random_kstack_offset (rdtsc ());
arch/x86/include/asm/entry-common.h static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs, · line 100
Quoted 10 lines
arch/x86/include/asm/entry-common.h +4
-5-/* Avoid unnecessary reads of 'x86_ibpb_exit_to_user' */ -if (cpu_feature_enabled (X86_FEATURE_IBPB_EXIT_TO_USER ) && -this_cpu_read (x86_ibpb_exit_to_user)) { -indirect_branch_prediction_barrier (); -this_cpu_write (x86_ibpb_exit_to_user, false); +if (unlikely (this_cpu_read (x86_pred_flush_pending))) { +if (cpu_feature_enabled (X86_FEATURE_IBPB_EXIT_TO_USER )) +indirect_branch_prediction_barrier (); +else if (cpu_feature_enabled (X86_FEATURE_CLEAR_BHB_EXIT_TO_USER ))
I realize that IBPB and BHB clear doesn't have to be mutually exclusive. IBPB does avoids the need to clear BHB because it flushes the indirect branches and BHB isn't useful anymore. But, this code doesn't need to prevent both from being executed. This should be enforced during mitigation selection. Updating the patch to allow both here.
Quoted 5 lines
arch/x86/include/asm/entry-common.h +3
-0+clear_bhb_long_loop (); + +this_cpu_write (x86_pred_flush_pending, false); } }
Pawan Gupta
reviewerPATCH v2 3/3 x86/vmscape: Remove LFENCE from BHB clearing long loop
In reply to Pawan GuptaLong loop is used to clear the branch history when switching from a guest to host userspace. The LFENCE barrier is not required in this case as ring transition itself acts as a barrier.
Move the prologue, LFENCE and epilogue out of __CLEAR_BHB_LOOP macro to allow skipping the LFENCE in the long loop variant. Rename the long loop function to clear_bhb_long_loop_no_barrier() to reflect the change.
--- arch/x86/entry/entry_64.S | 32 ++++++++++++++++++++------------ arch/x86/include/asm/entry-common.h | 2 +- arch/x86/include/asm/nospec-branch.h | 4 ++-- 3 files changed, 23 insertions(+), 15 deletions(-)
arch/x86/entry/entry_64.S +20 -12
@@ -1525,10 +1525,6 @@ SYM_CODE_END(rewind_stack_and_make_dead) * Target Selection, rather than taking the slowpath via its_return_thunk. */ .macro __CLEAR_BHB_LOOP outer_loop_count:req, inner_loop_count:req - ANNOTATE_NOENDBR - push %rbp - mov %rsp, %rbp - movl $\outer_loop_count, %ecx ANNOTATE_INTRA_FUNCTION_CALL call 1f
@@ -1560,10 +1556,7 @@ SYM_CODE_END(rewind_stack_and_make_dead) jnz 1b .Lret2_\@: RET -5: lfence - - pop %rbp - RET +5: .endm /*
@@ -1573,7 +1566,15 @@ SYM_CODE_END(rewind_stack_and_make_dead) * setting BHI_DIS_S for the guests. */ SYM_FUNC_START(clear_bhb_loop) + ANNOTATE_NOENDBR + push %rbp + mov %rsp, %rbp + __CLEAR_BHB_LOOP 5, 5 + + lfence + pop %rbp + RET SYM_FUNC_END(clear_bhb_loop) EXPORT_SYMBOL_GPL(clear_bhb_loop) STACK_FRAME_NON_STANDARD(clear_bhb_loop)
@@ -1584,8 +1585,15 @@ STACK_FRAME_NON_STANDARD(clear_bhb_loop) * protects the kernel, but to mitigate the guest influence on the host * userspace either IBPB or this sequence should be used. See VMSCAPE bug. */ -SYM_FUNC_START(clear_bhb_long_loop) +SYM_FUNC_START(clear_bhb_long_loop_no_barrier) + ANNOTATE_NOENDBR + push %rbp + mov %rsp, %rbp + __CLEAR_BHB_LOOP 12, 7 -SYM_FUNC_END(clear_bhb_long_loop) -EXPORT_SYMBOL_GPL(clear_bhb_long_loop) -STACK_FRAME_NON_STANDARD(clear_bhb_long_loop) + + pop %rbp + RET +SYM_FUNC_END(clear_bhb_long_loop_no_barrier) +EXPORT_SYMBOL_GPL(clear_bhb_long_loop_no_barrier) +STACK_FRAME_NON_STANDARD(clear_bhb_long_loop_no_barrier)
arch/x86/include/asm/entry-common.h +1 -1
@@ -98,7 +98,7 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs,if (cpu_feature_enabled (X86_FEATURE_IBPB_EXIT_TO_USER ))indirect_branch_prediction_barrier ();else if (cpu_feature_enabled (X86_FEATURE_CLEAR_BHB_EXIT_TO_USER )) -clear_bhb_long_loop (); +clear_bhb_long_loop_no_barrier ();this_cpu_write (x86_pred_flush_pending, false); }
arch/x86/include/asm/nospec-branch.h +2 -2
@@ -388,9 +388,9 @@ extern void write_ibpb(void);#ifdef CONFIG_X86_64 extern void clear_bhb_loop (void ); -extern void clear_bhb_long_loop (void ); +extern void clear_bhb_long_loop_no_barrier (void );#else -static inline void clear_bhb_long_loop (void ) {} +static inline void clear_bhb_long_loop_no_barrier (void ) {}#endif extern void (*x86_return_thunk)(void );
Signature · 1 line
2.34.1
[AMD Official Use Only - AMD Internal Distribution Only]
Quoted 99 lines
-----Original Message-----
From: Pawan Gupta <[email protected]>
Sent: Wednesday, October 15, 2025 8:52 PM
To: [email protected]; H. Peter Anvin <[email protected]>; Josh Poimboeuf <[email protected]>; Kaplan, David <[email protected]>; Sean Christopherson <[email protected]>; Paolo Bonzini <[email protected]>
Cc: [email protected]; [email protected]; Asit Mallick <[email protected]>; Tao Zhang <[email protected]>
Subject: [PATCH v2 0/3] VMSCAPE optimization for BHI variantCaution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
v2:
- Added check for IBPB feature invmscape_select_mitigation(). (David)
- s/vmscape=auto/vmscape=on/ (David)
- Added patch to remove LFENCE from VMSCAPE BHB-clear sequence.
- Rebased to v6.18-rc1.v1: https://lore.kernel.org/r/20250924-vmscape-bhb-v1-0-
[email protected]Hi All,
These patches aim to improve the performance of a recent mitigation for VMSCAPE[1] vulnerability. This improvement is relevant for BHI variant of VMSCAPE that affect Alder Lake and newer processors.
The current mitigation approach uses IBPB on kvm-exit-to-userspace for all affected range of CPUs. This is an overkill for CPUs that are only affected by the BHI variant. On such CPUs clearing the branch history is sufficient for VMSCAPE, and also more apt as the underlying issue is due to poisoned branch history.
Roadmap:
- First patch introduces
clear_bhb_long_loop()for processors with larger branch history tables.
- Second patch replaces IBPB on exit-to-userspace with branch history clearing sequence.Below is the iPerf data for transfer between guest and host, comparing IBPB and BHB-clear mitigation. BHB-clear shows performance improvement over IBPB in most cases.
Platform: Emerald Rapids
Baseline: vmscape=off(pN = N parallel connections)
| iPerf user-net | IBPB | BHB Clear | |----------------|---------|-----------| | UDP 1-vCPU_p1 | -12.5% | 1.3% | | TCP 1-vCPU_p1 | -10.4% | -1.5% | | TCP 1-vCPU_p1 | -7.5% | -3.0% | | UDP 4-vCPU_p16 | -3.7% | -3.7% | | TCP 4-vCPU_p4 | -2.9% | -1.4% | | UDP 4-vCPU_p4 | -0.6% | 0.0% | | TCP 4-vCPU_p4 | 3.5% | 0.0% || iPerf bridge-net | IBPB | BHB Clear | |------------------|---------|-----------| | UDP 1-vCPU_p1 | -9.4% | -0.4% | | TCP 1-vCPU_p1 | -3.9% | -0.5% | | UDP 4-vCPU_p16 | -2.2% | -3.8% | | TCP 4-vCPU_p4 | -1.0% | -1.0% | | TCP 4-vCPU_p4 | 0.5% | 0.5% | | UDP 4-vCPU_p4 | 0.0% | 0.9% | | TCP 1-vCPU_p1 | 0.0% | 0.9% || iPerf vhost-net | IBPB | BHB Clear | |-----------------|---------|-----------| | UDP 1-vCPU_p1 | -4.3% | 1.0% | | TCP 1-vCPU_p1 | -3.8% | -0.5% | | TCP 1-vCPU_p1 | -2.7% | -0.7% | | UDP 4-vCPU_p16 | -0.7% | -2.2% | | TCP 4-vCPU_p4 | -0.4% | 0.8% | | UDP 4-vCPU_p4 | 0.4% | -0.7% | | TCP 4-vCPU_p4 | 0.0% | 0.6% |[1] https://comsec.ethz.ch/research/microarch/vmscape-exposing-and-exploiting-
incomplete-branch-predictor-isolation-in-cloud-environments/---
Pawan Gupta (3):
x86/bhi: Add BHB clearing for CPUs with larger branch history x86/vmscape: Replace IBPB with branch history clear on exit to userspace x86/vmscape: Remove LFENCE from BHB clearing long loopDocumentation/admin-guide/hw-vuln/vmscape.rst | 8 ++++ Documentation/admin-guide/kernel-parameters.txt | 4 +- arch/x86/entry/entry_64.S | 63 ++++++++++++++++++------- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/entry-common.h | 12 +++-- arch/x86/include/asm/nospec-branch.h | 5 +- arch/x86/kernel/cpu/bugs.c | 53 +++++++++++++++------ arch/x86/kvm/x86.c | 5 +- 8 files changed, 110 insertions(+), 41 deletions(-)
Quoted 8 lines
---
base-commit:3a8660878839faadb4f1a6dd72c3179c1df56787
change-id: 20250916-vmscape-bhb-d7d469977f2fBest regards,
--
Pawan
Looks good to me.
Pawan Gupta
reviewerQuoted 105 lines · David
[AMD Official Use Only - AMD Internal Distribution Only]
Quoted 103 lines
-----Original Message-----
From: Pawan Gupta <[email protected]>
Sent: Wednesday, October 15, 2025 8:52 PM
To: [email protected]; H. Peter Anvin <[email protected]>; Josh Poimboeuf <[email protected]>; Kaplan, David <[email protected]>; Sean Christopherson <[email protected]>; Paolo Bonzini <[email protected]>
Cc: [email protected]; [email protected]; Asit Mallick <[email protected]>; Tao Zhang <[email protected]>
Subject: [PATCH v2 0/3] VMSCAPE optimization for BHI variantCaution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
v2:
- Added check for IBPB feature invmscape_select_mitigation(). (David)
- s/vmscape=auto/vmscape=on/ (David)
- Added patch to remove LFENCE from VMSCAPE BHB-clear sequence.
- Rebased to v6.18-rc1.v1: https://lore.kernel.org/r/20250924-vmscape-bhb-v1-0-
[email protected]Hi All,
These patches aim to improve the performance of a recent mitigation for VMSCAPE[1] vulnerability. This improvement is relevant for BHI variant of VMSCAPE that affect Alder Lake and newer processors.
The current mitigation approach uses IBPB on kvm-exit-to-userspace for all affected range of CPUs. This is an overkill for CPUs that are only affected by the BHI variant. On such CPUs clearing the branch history is sufficient for VMSCAPE, and also more apt as the underlying issue is due to poisoned branch history.
Roadmap:
- First patch introduces
clear_bhb_long_loop()for processors with larger branch history tables.
- Second patch replaces IBPB on exit-to-userspace with branch history clearing sequence.Below is the iPerf data for transfer between guest and host, comparing IBPB and BHB-clear mitigation. BHB-clear shows performance improvement over IBPB in most cases.
Platform: Emerald Rapids
Baseline: vmscape=off(pN = N parallel connections)
| iPerf user-net | IBPB | BHB Clear | |----------------|---------|-----------| | UDP 1-vCPU_p1 | -12.5% | 1.3% | | TCP 1-vCPU_p1 | -10.4% | -1.5% | | TCP 1-vCPU_p1 | -7.5% | -3.0% | | UDP 4-vCPU_p16 | -3.7% | -3.7% | | TCP 4-vCPU_p4 | -2.9% | -1.4% | | UDP 4-vCPU_p4 | -0.6% | 0.0% | | TCP 4-vCPU_p4 | 3.5% | 0.0% || iPerf bridge-net | IBPB | BHB Clear | |------------------|---------|-----------| | UDP 1-vCPU_p1 | -9.4% | -0.4% | | TCP 1-vCPU_p1 | -3.9% | -0.5% | | UDP 4-vCPU_p16 | -2.2% | -3.8% | | TCP 4-vCPU_p4 | -1.0% | -1.0% | | TCP 4-vCPU_p4 | 0.5% | 0.5% | | UDP 4-vCPU_p4 | 0.0% | 0.9% | | TCP 1-vCPU_p1 | 0.0% | 0.9% || iPerf vhost-net | IBPB | BHB Clear | |-----------------|---------|-----------| | UDP 1-vCPU_p1 | -4.3% | 1.0% | | TCP 1-vCPU_p1 | -3.8% | -0.5% | | TCP 1-vCPU_p1 | -2.7% | -0.7% | | UDP 4-vCPU_p16 | -0.7% | -2.2% | | TCP 4-vCPU_p4 | -0.4% | 0.8% | | UDP 4-vCPU_p4 | 0.4% | -0.7% | | TCP 4-vCPU_p4 | 0.0% | 0.6% |[1] https://comsec.ethz.ch/research/microarch/vmscape-exposing-and-exploiting-
incomplete-branch-predictor-isolation-in-cloud-environments/---
Pawan Gupta (3):
x86/bhi: Add BHB clearing for CPUs with larger branch history x86/vmscape: Replace IBPB with branch history clear on exit to userspace x86/vmscape: Remove LFENCE from BHB clearing long loopDocumentation/admin-guide/hw-vuln/vmscape.rst | 8 ++++ Documentation/admin-guide/kernel-parameters.txt | 4 +- arch/x86/entry/entry_64.S | 63 ++++++++++++++++++------- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/entry-common.h | 12 +++-- arch/x86/include/asm/nospec-branch.h | 5 +- arch/x86/kernel/cpu/bugs.c | 53 +++++++++++++++------ arch/x86/kvm/x86.c | 5 +- 8 files changed, 110 insertions(+), 41 deletions(-)---
base-commit:3a8660878839faadb4f1a6dd72c3179c1df56787
change-id: 20250916-vmscape-bhb-d7d469977f2f
Quoted 8 lines
Quoted 4 lines
Best regards,
--
PawanLooks good to me.
Acked-by David Kaplan <[email protected]>
Thanks.