Skip to contentSkip to conversation

KVM: VMX: Drop obsolete branch hint prefixes from inline asm

PATCH Applied Uros Bizjak 5 participants 14 messages Feb 11 – Mar 5

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/2 KVM: VMX: Drop obsolete branch hint prefixes from inline asm

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

No functional change intended.

Cc Sean Christopherson <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
 arch/x86/kvm/vmx/vmx_ops.h | 3 ---
 1 file changed, 3 deletions(-)
arch/x86/kvm/vmx/vmx_ops.h +0 -3
index 96677576c836..1000d37f5b0c 100644
@@ -119,7 +119,6 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field)
 #else /* !CONFIG_CC_HAS_ASM_GOTO_OUTPUT */
 
 	asm volatile("1: vmread %[field], %[output]\n\t"
-		     ".byte 0x3e\n\t" /* branch taken hint */
 		     "ja 3f\n\t"
 
 		     /*
@@ -191,7 +190,6 @@ static __always_inline unsigned long vmcs_readl(unsigned long field)
 #define vmx_asm1(insn, op1, error_args...)				\
 do {									\
 	asm goto("1: " __stringify(insn) " %0\n\t"			\
-			  ".byte 0x2e\n\t" /* branch not taken hint */	\
 			  "jna %l[error]\n\t"				\
 			  _ASM_EXTABLE(1b, %l[fault])			\
 			  : : op1 : "cc" : error, fault);		\
@@ -208,7 +206,6 @@ fault:									\
 #define vmx_asm2(insn, op1, op2, error_args...)				\
 do {									\
 	asm goto("1: "  __stringify(insn) " %1, %0\n\t"			\
-			  ".byte 0x2e\n\t" /* branch not taken hint */	\
 			  "jna %l[error]\n\t"				\
 			  _ASM_EXTABLE(1b, %l[fault])			\
 			  : : op1, op2 : "cc" : error, fault);		\
Signature · 1 line
2.53.0
#1 lore raw

PATCH 2/2 KVM: VMX: Use ASM_INPUT_RM in __vmcs_writel

Use the ASM_INPUT_RM macro for VMCS write operation in vmx_ops.h to work around clang problems with "rm" asm constraint. clang seems to always chose the memory input, while it is almost always the worst choice.

Cc Sean Christopherson <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
 arch/x86/kvm/vmx/vmx_ops.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx_ops.h +1 -1
index 1000d37f5b0c..81784befaaf4 100644
@@ -221,7 +221,7 @@ fault:									\
 
 static __always_inline void __vmcs_writel(unsigned long field, unsigned long value)
 {
-	vmx_asm2(vmwrite, "r"(field), "rm"(value), field, value);
+	vmx_asm2(vmwrite, "r" (field), ASM_INPUT_RM (value), field, value);
 }
 
 static __always_inline void vmcs_write16(unsigned long field, u16 value)
Signature · 1 line
2.53.0
#3 lore raw
In reply to Uros Bizjak
Quoted 7 lines

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

It's actually worse than this.

The branch-taken hint has new meaning in Lion Cove cores and later, along with a warning saying "performance penalty for misuse".

i.e. "only insert this prefix after profiling".

~Andrew

#4 lore raw
Quoted 8 lines · Andrew Cooper
Quoted 7 lines

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

Quoted 4 lines

It's actually worse than this.

The branch-taken hint has new meaning in Lion Cove cores and later, along with a warning saying "performance penalty for misuse".

Well that's just lovely. Sounds like maybe this should be tagged for stable@?

#5 lore raw
In reply to Andrew Cooper
Quoted 8 lines · On Wed, 11 Feb 2026 10:57:31 +0000 Andrew Cooper <[email protected]>…
Quoted 7 lines

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

Quoted 6 lines

It's actually worse than this.

The branch-taken hint has new meaning in Lion Cove cores and later, along with a warning saying "performance penalty for misuse".

i.e. "only insert this prefix after profiling".

Don't they really have much the same meaning as before? Perhaps the branch prediction logic is ignored (which would make getting then wrong very bad).
But here (and a few other places) the branch really is 'never taken', so perhaps the branch hints should have been removed 20 years ago and now is the time to add them back?

Of course, the branch-hint can only have an effect when the branch prediction logic gets to see the branch.
The initial instruction prefetch (etc) is done without regard to the contents of the memory being read - so initially all branches are effectively predicted 'not-taken'.

(I got the impression that some AMD cpu 'attached' some of the branch prediction info to the cache-line - and would use it when the cache line was re-used for an entirely different address. Hence the issues where non-branch instructions would be predicted as branches.)

David

Quoted 3 lines

~Andrew

#6 lore raw
Quoted 10 lines · David Laight
Quoted 7 lines · On Wed, 11 Feb 2026 10:57:31 +0000 Andrew Cooper <[email protected]>…
Quoted 7 lines

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

Quoted 7 lines
Quoted 6 lines

It's actually worse than this.

The branch-taken hint has new meaning in Lion Cove cores and later, along with a warning saying "performance penalty for misuse".

i.e. "only insert this prefix after profiling".

Don't they really have much the same meaning as before?

Architecturally yes, microarchitecturally very much not.

For a branch known to the predictor, there is no effect.  If a branch unknown to the predictor gets decoded, it triggers a frontend flush and resteer.

It is only useful for programs large enough to exceed the working set of the conditional predictor, and for which certain branches are known to be ~always taken.

Putting the prefix on a branch that isn't ~always taken is worse than not having the prefix in the first place, hence the warning.

~Andrew

#7 lore raw
Quoted 25 lines · Andrew Cooper
Quoted 10 lines · David Laight
Quoted 7 lines · On Wed, 11 Feb 2026 10:57:31 +0000 Andrew Cooper <[email protected]>…
Quoted 7 lines

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

Quoted 7 lines
Quoted 6 lines

It's actually worse than this.

The branch-taken hint has new meaning in Lion Cove cores and later, along with a warning saying "performance penalty for misuse".

i.e. "only insert this prefix after profiling".

Don't they really have much the same meaning as before?

Architecturally yes, microarchitecturally very much not.

For a branch known to the predictor, there is no effect.  If a branch unknown to the predictor gets decoded, it triggers a frontend flush and resteer.

Quoted 6 lines

It is only useful for programs large enough to exceed the working set of the conditional predictor, and for which certain branches are known to be ~always taken.

Putting the prefix on a branch that isn't ~always taken is worse than not having the prefix in the first place, hence the warning.

These branches indeed ~always follow the hinted path (not taken in this case).

So it sounds like this definitely isn't stable@ material, and maybe even begs the question if dropping the hints is a net positive?

#8 lore raw
Quoted 29 lines · Sean Christopherson
Quoted 28 lines · Andrew Cooper
Quoted 10 lines · David Laight
Quoted 7 lines · On Wed, 11 Feb 2026 10:57:31 +0000 Andrew Cooper <[email protected]>…
Quoted 7 lines

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

Quoted 7 lines
Quoted 6 lines

It's actually worse than this.

The branch-taken hint has new meaning in Lion Cove cores and later, along with a warning saying "performance penalty for misuse".

i.e. "only insert this prefix after profiling".

Don't they really have much the same meaning as before?

Architecturally yes, microarchitecturally very much not.

For a branch known to the predictor, there is no effect.  If a branch unknown to the predictor gets decoded, it triggers a frontend flush and resteer.

It is only useful for programs large enough to exceed the working set of the conditional predictor, and for which certain branches are known to be ~always taken.

Quoted 6 lines
Quoted 2 lines

Putting the prefix on a branch that isn't ~always taken is worse than not having the prefix in the first place, hence the warning.

These branches indeed ~always follow the hinted path (not taken in this case).

So it sounds like this definitely isn't stable@ material, and maybe even begs the question if dropping the hints is a net positive?

The new behaviour only exists for hint-taken. Because it only has any effect for a branch unknown to the predictor, the behaviour without this hint would be as if it were a larger basic block.

hint-not-takens have no behaviour since the Pentium 4 that I'm aware of.

This change is almost certainly marginal at best.  It's not as if VMREAD/VMWRITE lead to good code gen even at the best of times.

~Andrew

#9 lore raw
Quoted 26 lines · Andrew Cooper
Quoted 25 lines · Sean Christopherson
Quoted 24 lines · Andrew Cooper
Quoted 10 lines · David Laight
Quoted 7 lines · On Wed, 11 Feb 2026 10:57:31 +0000 Andrew Cooper <[email protected]>…
Quoted 7 lines

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

Quoted 7 lines
Quoted 6 lines

It's actually worse than this.

The branch-taken hint has new meaning in Lion Cove cores and later, along with a warning saying "performance penalty for misuse".

i.e. "only insert this prefix after profiling".

Don't they really have much the same meaning as before?

Architecturally yes, microarchitecturally very much not.

For a branch known to the predictor, there is no effect.  If a branch unknown to the predictor gets decoded, it triggers a frontend flush and resteer.

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

It is only useful for programs large enough to exceed the working set of the conditional predictor, and for which certain branches are known to be ~always taken.

Putting the prefix on a branch that isn't ~always taken is worse than not having the prefix in the first place, hence the warning.

These branches indeed ~always follow the hinted path (not taken in this case).

Doh, forgot the !CC_HAS_ASM_GOTO_OUTPUT case uses a branch-taken hint.

Quoted 11 lines
Quoted 2 lines

So it sounds like this definitely isn't stable@ material, and maybe even begs the question if dropping the hints is a net positive?

The new behaviour only exists for hint-taken. Because it only has any effect for a branch unknown to the predictor, the behaviour without this hint would be as if it were a larger basic block.

hint-not-takens have no behaviour since the Pentium 4 that I'm aware of.

This change is almost certainly marginal at best.  It's not as if VMREAD/VMWRITE lead to good code gen even at the best of times.

Yeah, but adding in them in the first place was even more marginal (I added the hints as much for documentation purposes as anything else). Absent proof that having the hints is a net positive, I'm inclined to trust the compiler folks on what is/isn't optimal, and drop them.

#10 lore raw
Quoted 9 lines · Sean Christopherson
Quoted 4 lines · Andrew Cooper
Quoted 1 line · Sean Christopherson

On Wed, Feb 11, 2026, Andrew Cooper wrote:

This change is almost certainly marginal at best.  It's not as if VMREAD/VMWRITE lead to good code gen even at the best of times.

Yeah, but adding in them in the first place was even more marginal (I added the hints as much for documentation purposes as anything else). Absent proof that having the hints is a net positive, I'm inclined to trust the compiler folks on what is/isn't optimal, and drop them.

Branch mispredicts in the P4 could easily eat up 150 cycles before the frontend got it's act together.

However, optimising VMREAD/VMWRITE and not the whole kernel seems somewhat futile.

~Andrew

#11 lore raw
Quoted 6 lines · Andrew Cooper
Quoted 5 lines · Sean Christopherson
Quoted 4 lines · Andrew Cooper
Quoted 1 line · Sean Christopherson

On Wed, Feb 11, 2026, Andrew Cooper wrote:

This change is almost certainly marginal at best.  It's not as if VMREAD/VMWRITE lead to good code gen even at the best of times.

Quoted 7 lines
Quoted 4 lines

Yeah, but adding in them in the first place was even more marginal (I added the hints as much for documentation purposes as anything else). Absent proof that having the hints is a net positive, I'm inclined to trust the compiler folks on what is/isn't optimal, and drop them.

Branch mispredicts in the P4 could easily eat up 150 cycles before the frontend got it's act together.

Heh, I'm willing to risk getting yelled at by the one person running KVM on P4.

Quoted 4 lines

However, optimising VMREAD/VMWRITE and not the whole kernel seems somewhat futile.

~Andrew

#12 lore raw
In reply to Andrew Cooper
Quoted 16 lines · On Wed, 11 Feb 2026 13:55:35 +0000 Andrew Cooper <[email protected]>…
Quoted 15 lines · David Laight
Quoted 12 lines · On Wed, 11 Feb 2026 10:57:31 +0000 Andrew Cooper <[email protected]>…
Quoted 7 lines

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

It's actually worse than this.

The branch-taken hint has new meaning in Lion Cove cores and later, along with a warning saying "performance penalty for misuse".

Quoted 8 lines
Quoted 2 lines
Quoted 1 line

i.e. "only insert this prefix after profiling".

Don't they really have much the same meaning as before?

Architecturally yes, microarchitecturally very much not.

For a branch known to the predictor, there is no effect.  If a branch unknown to the predictor gets decoded, it triggers a frontend flush and resteer.

That'll be 'decoded taken'.
I suspect that it is less 'painful' than a normal mispredict since it happens as lot earlier.
Of course, if you get it wrong, there will be a mispredict penalty as well.

David

Quoted 9 lines

It is only useful for programs large enough to exceed the working set of the conditional predictor, and for which certain branches are known to be ~always taken.

Putting the prefix on a branch that isn't ~always taken is worse than not having the prefix in the first place, hence the warning.

~Andrew

Acked-by on v1 2/2
#2 lore raw

Re: [PATCH 2/2] KVM: VMX: Use ASM_INPUT_RM in __vmcs_writel

In reply to Uros Bizjak
Quoted 5 lines · Uros Bizjak

Use the ASM_INPUT_RM macro for VMCS write operation in vmx_ops.h to work around clang problems with "rm" asm constraint. clang seems to always chose the memory input, while it is almost always the worst choice.

Quoted 8 lines
Cc Sean Christopherson <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>

FWIW, I hope this issue will soon be fixed in clang properly so we can add a version check to this workaround:

https://github.com/llvm/llvm-project/pull/181973

Quoted 20 lines
---
 arch/x86/kvm/vmx/vmx_ops.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx_ops.h +1 -1
index 1000d37f5b0c..81784befaaf4 100644
@@ -221,7 +221,7 @@ fault:									\
 
 static __always_inline void __vmcs_writel(unsigned long field, unsigned long value)
 {
-	vmx_asm2(vmwrite, "r"(field), "rm"(value), field, value);
+	vmx_asm2(vmwrite, "r" (field), ASM_INPUT_RM (value), field, value);
 }
 
 static __always_inline void vmcs_write16(unsigned long field, u16 value)
Signature · 1 line
2.53.0
maintainer Applied
#13 lore raw
In reply to Uros Bizjak
Quoted 9 lines · Uros Bizjak

Remove explicit branch hint prefixes (.byte 0x2e / 0x3e) from VMX inline assembly sequences.

These prefixes (CS/DS segment overrides used as branch hints on very old x86 CPUs) have been ignored by modern processors for a long time. Keeping them provides no measurable benefit and only enlarges the generated code.

[...]

Applied to kvm-x86 vmx, thanks!

[1/2] KVM: VMX: Drop obsolete branch hint prefixes from inline asm

      https://github.com/kvm-x86/linux/commit/6dad59124e15
[2/2] KVM: VMX: Use ASM_INPUT_RM in __vmcs_writel
      https://github.com/kvm-x86/linux/commit/192f777b3af0

--
https://github.com/kvm-x86/linux/tree/next