Skip to contentSkip to conversation

x86/bugs: Use fixed addressing for VERW operand

PATCH In review Pawan Gupta 3 participants 8 messages Feb 26, 2024 – Feb 29, 2024

Subsystems (2)
lore
Share & tools
lore

Pawan Gupta

author reviewer
#0 lore raw

PATCH 1/1 x86/bugs: Use fixed addressing for VERW operand

Macro used for MDS mitigation executes VERW with relative addressing for the operand. This is unnecessary and creates a problem for backports on older kernels that don't support relocations in alternatives. Relocation support was added by commit 270a69c4485d ("x86/alternative: Support relocations in alternatives"). Also asm for fixed addressing is much more cleaner than relative RIP addressing.

Simplify the asm by using fixed addressing for VERW operand.

Fixes baf8361e5455 ("x86/bugs: Add asm helpers for executing VERW")
---
 arch/x86/include/asm/nospec-branch.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/nospec-branch.h +1 -1
index 2aa52cab1e46..ab19c7f1167b 100644
@@ -323,7 +323,7 @@
  * Note: Only the memory operand variant of VERW clears the CPU buffers.
  */
 .macro CLEAR_CPU_BUFFERS
-	ALTERNATIVE "", __stringify(verw _ASM_RIP(mds_verw_sel)), X86_FEATURE_CLEAR_CPU_BUF
+	ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF
 .endm
 
 #else /* __ASSEMBLY__ */

---
base-commit: d206a76d7d2726f3b096037f2079ce0bd3ba329b
change-id: 20240226-verw-arg-fix-796a63088c4d

#1 lore raw
Quoted 22 lines · Pawan Gupta

Macro used for MDS mitigation executes VERW with relative addressing for the operand. This is unnecessary and creates a problem for backports on older kernels that don't support relocations in alternatives. Relocation support was added by commit 270a69c4485d ("x86/alternative: Support relocations in alternatives"). Also asm for fixed addressing is much more cleaner than relative RIP addressing.

Simplify the asm by using fixed addressing for VERW operand.

Fixes baf8361e5455 ("x86/bugs: Add asm helpers for executing VERW")
---
  arch/x86/include/asm/nospec-branch.h | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/nospec-branch.h +0 -0
index 2aa52cab1e46..ab19c7f1167b 100644

arch/x86/include/asm/nospec-branch.h line 326

Quoted 5 lines
arch/x86/include/asm/nospec-branch.h +1 -1
@@ -323,7 +323,7 @@
   * Note: Only the memory operand variant of VERW clears the CPU buffers.
   */
  .macro CLEAR_CPU_BUFFERS
-	ALTERNATIVE "", __stringify(verw _ASM_RIP(mds_verw_sel)), X86_FEATURE_CLEAR_CPU_BUF
+	ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF

Actually thinking about it more and discussing with Jiri (cc'ed), will this work with KASLR enabled?

Quoted 8 lines
arch/x86/include/asm/nospec-branch.h +0 -1
  .endm
 
  #else /* __ASSEMBLY__ */
 
---

base-commit: d206a76d7d2726f3b096037f2079ce0bd3ba329b
change-id: 20240226-verw-arg-fix-796a63088c4d

#2 lore raw
Quoted 20 lines · Nikolay Borisov
Quoted 17 lines · Pawan Gupta

Macro used for MDS mitigation executes VERW with relative addressing for the operand. This is unnecessary and creates a problem for backports on older kernels that don't support relocations in alternatives. Relocation support was added by commit 270a69c4485d ("x86/alternative: Support relocations in alternatives"). Also asm for fixed addressing is much more cleaner than relative RIP addressing.

Simplify the asm by using fixed addressing for VERW operand.

Fixes baf8361e5455 ("x86/bugs: Add asm helpers for executing VERW")

---
  arch/x86/include/asm/nospec-branch.h | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 14 lines
Quoted 7 lines
arch/x86/include/asm/nospec-branch.h +0 -0
index 2aa52cab1e46..ab19c7f1167b 100644
@@ -323,7 +323,7 @@
    * Note: Only the memory operand variant of VERW clears the CPU buffers.
    */
Quoted 5 lines

  .macro CLEAR_CPU_BUFFERS
-    ALTERNATIVE "", __stringify(verw _ASM_RIP(mds_verw_sel)), X86_FEATURE_CLEAR_CPU_BUF
+    ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF

Actually thinking about it more and discussing with Jiri (cc'ed), will this work with KASLR enabled?

I might of course be wrong. We appear to rely on the asm+linker here. Please see also:
https://lore.kernel.org/r/[email protected]

thanks,

Signature · 2 lines
js
suse labs
author reviewer
#3 lore raw
Quoted 39 lines · Jiri Slaby
Quoted 29 lines · Nikolay Borisov
Quoted 26 lines · Pawan Gupta

Macro used for MDS mitigation executes VERW with relative addressing for the operand. This is unnecessary and creates a problem for backports on older kernels that don't support relocations in alternatives. Relocation support was added by commit 270a69c4485d ("x86/alternative: Support relocations in alternatives"). Also asm for fixed addressing is much more cleaner than relative RIP addressing.

Simplify the asm by using fixed addressing for VERW operand.

Fixes baf8361e5455 ("x86/bugs: Add asm helpers for executing VERW")

---
  arch/x86/include/asm/nospec-branch.h | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/nospec-branch.h +0 -0
index 2aa52cab1e46..ab19c7f1167b 100644
@@ -323,7 +323,7 @@
    * Note: Only the memory operand variant of VERW clears the CPU

buffers.
   */

Quoted 8 lines
Quoted 5 lines

  .macro CLEAR_CPU_BUFFERS
-    ALTERNATIVE "", __stringify(verw _ASM_RIP(mds_verw_sel)), X86_FEATURE_CLEAR_CPU_BUF
+    ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF

Actually thinking about it more and discussing with Jiri (cc'ed), will this work with KASLR enabled?

Quoted 1 line

I might of course be wrong. We appear to rely on the asm+linker here.

You were right, with KASLR enabled, instructions with fixed addressing in alternatives don't get relocated. I guess we will have to keep rip-relative as is. Thanks for catching that.

author reviewer
#4 lore raw
Quoted 40 lines · Pawan Gupta
Quoted 39 lines · Jiri Slaby
Quoted 37 lines · Nikolay Borisov
Quoted 24 lines · Pawan Gupta

Macro used for MDS mitigation executes VERW with relative addressing for the operand. This is unnecessary and creates a problem for backports on older kernels that don't support relocations in alternatives. Relocation support was added by commit 270a69c4485d ("x86/alternative: Support relocations in alternatives"). Also asm for fixed addressing is much more cleaner than relative RIP addressing.

Simplify the asm by using fixed addressing for VERW operand.

Fixes baf8361e5455 ("x86/bugs: Add asm helpers for executing VERW")

---
  arch/x86/include/asm/nospec-branch.h | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/nospec-branch.h +0 -0
index 2aa52cab1e46..ab19c7f1167b 100644
@@ -323,7 +323,7 @@
    * Note: Only the memory operand variant of VERW clears the CPU
Quoted 7 lines

buffers.
   */
  .macro CLEAR_CPU_BUFFERS
-    ALTERNATIVE "", __stringify(verw _ASM_RIP(mds_verw_sel)), X86_FEATURE_CLEAR_CPU_BUF
+    ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF

Actually thinking about it more and discussing with Jiri (cc'ed), will this work with KASLR enabled?

Quoted 5 lines
Quoted 1 line

I might of course be wrong. We appear to rely on the asm+linker here.

You were right, with KASLR enabled, instructions with fixed addressing in alternatives don't get relocated. I guess we will have to keep rip-relative as is. Thanks for catching that.

Looks like this is not settled yet, it was naive of me to trust gdb on /proc/kcore earlier with KASLR enabled.

With the below debug patch it appears the relocation with fixed addresses is working as expected with KASLR enabled.

arch/x86/include/asm/nospec-branch.h +1 -1
index 6a1fe302c1fd..ae4168db453d 100644
@@ -323,7 +323,7 @@
  * Note: Only the memory operand variant of VERW clears the CPU buffers.
  */
 .macro CLEAR_CPU_BUFFERS
-	ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF
+	ALTERNATIVE "", __stringify(verw mds_verw_sel; ud2), X86_FEATURE_CLEAR_CPU_BUF
 .endm
 
 #else /* __ASSEMBLY__ */

---

With the patch adding #UD after verw I get below panic:

[    1.402347] traps: PANIC: double fault, error_code: 0x0
[    1.402350] double fault: 0000 [#1] PREEMPT SMP PTI
[    1.402353] CPU: 0 PID: 1 Comm: init Not tainted 6.7.6-00007-ga040ab9fe827-dirty #27
[    1.402355] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[    1.402356] RIP: 0010:entry_SYSRETQ_unsafe_stack+0xb/0x10
[    1.402365] Code: c7 eb 08 48 89 c7 48 0f ba ef 3f 48 81 cf 00 08 00 00 48 81 cf 00 10 00 00 0f 22 df 58 5f 5c 0f 01 f8 0f 00 2c 25 00 00 60 ab <0f> 0b 48 0f 07 cc 0f 1f 44 00 00 56 48 8b 74 24 08 48 89 7c 24 08
[    1.402366] RSP: 0018:00007ffe253461f8 EFLAGS: 000100c6
[    1.402369] RAX: 000055e5c94f7000 RBX: 00007f3d8d9b4e80 RCX: 00007f3d8d9b8f9b
[    1.402370] RDX: 0000000000000000 RSI: 00007ffe253461ec RDI: 0000000000000000
[    1.402371] RBP: 00007ffe25346290 R08: 0000000000000000 R09: 0000000000000022
[    1.402372] R10: 0000000000000000 R11: 0000000000000246 R12: 00007f3d8d993000
[    1.402373] R13: 0000002300000007 R14: 0000000000000007 R15: 00007ffe253462a0
[    1.402378] FS:  0000000000000000(0000) GS:ffff906d3bc00000(0000) knlGS:ffff906d3bc00000
[    1.402380] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    1.402381] CR2: 00007ffe253461e8 CR3: 00000001002a8003 CR4: 00000000007706f0
[    1.402382] PKRU: 55555554
[    1.402383] Call Trace:
[    1.402384]  <#DF>
[    1.402386]  ? die+0x37/0x90
[    1.402390]  ? exc_double_fault+0xcf/0x180
[    1.402394]  ? asm_exc_double_fault+0x23/0x30
[    1.402397]  ? entry_SYSRETQ_unsafe_stack+0xb/0x10
[    1.402400]  </#DF>
[    1.402401] Modules linked in:
[    1.402403] Dumping ftrace buffer:
[    1.402406]    (ftrace buffer empty)
[    1.402407] ---[ end trace 0000000000000000 ]---
[    1.402408] RIP: 0010:entry_SYSRETQ_unsafe_stack+0xb/0x10
[    1.402411] Code: c7 eb 08 48 89 c7 48 0f ba ef 3f 48 81 cf 00 08 00 00 48 81 cf 00 10 00 00 0f 22 df 58 5f 5c 0f 01 f8 0f 00
 2c 25 00 00 60 ab <0f> 0b 48 0f 07 cc 0f 1f 44 00 00 56 48 8b 74 24 08 48 89 7c 24 08
[    1.402412] RSP: 0018:00007ffe253461f8 EFLAGS: 000100c6
[    1.402413] RAX: 000055e5c94f7000 RBX: 00007f3d8d9b4e80 RCX: 00007f3d8d9b8f9b
[    1.402414] RDX: 0000000000000000 RSI: 00007ffe253461ec RDI: 0000000000000000
[    1.402415] RBP: 00007ffe25346290 R08: 0000000000000000 R09: 0000000000000022
[    1.402416] R10: 0000000000000000 R11: 0000000000000246 R12: 00007f3d8d993000
[    1.402417] R13: 0000002300000007 R14: 0000000000000007 R15: 00007ffe253462a0
[    1.402419] FS:  0000000000000000(0000) GS:ffff906d3bc00000(0000) knlGS:ffff906d3bc00000
[    1.402420] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    1.402421] CR2: 00007ffe253461e8 CR3: 00000001002a8003 CR4: 00000000007706f0
[    1.402422] PKRU: 55555554
[    1.402423] Kernel panic - not syncing: Fatal exception in interrupt
[    1.402551] Dumping ftrace buffer:
[    1.402552]    (ftrace buffer empty)
[    1.402552] Kernel Offset: 0x29400000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
                              ^
			      Note the KASLR offset
			      ~~~~~~~~~~~~~~~~~~~~~

Disassembling the code stream in the trace ... :

eb 08 48 89 c7 48 0f ba ef 3f 48 81 cf 00 08 00 00 48 81 cf 00 10 00 00 0f 22 df 58 5f 5c 0f 01 f8 0f 00 2c 25 00 00 60 ab 0f 0b 48 0f 07 cc 0f 1f 44 00 00 56 48 8b 74 24 08 48 89 7c 24 08

... gives:

jmp 0xa
mov rdi, rax
bts rdi, 0x3f
or rdi, 0x800
or rdi, 0x1000
mov cr3, rdi
pop rax
pop rdi
pop rsp
swapgs
verw word ptr [0xffffffffab600000] <----- mds_verw_sel
ud2
sysretq
int3
nop dword ptr [rax + rax]
push rsi
mov rsi, qword ptr [rsp + 8]
mov qword ptr [rsp + 8], rdi

$ objdump -S --disassemble=mds_verw_sel vmlinux

ffffffff82200000 <mds_verw_sel>:
ffffffff82200000: 18 00 sbb %al,(%rax)

Adding KASLR offset to mds_verw_sel:

0xffffffff82200000 + 0x29400000 = 0xffffffffab600000

which is the address of mds_verw_sel after KASLR that we saw in the code stream disassembly. To me it looks like fixed addresses for VERW is working as intended. Please let me know if I am missing something. It will be great if someone can independently verify this.

Even with fixed addressing working with KASLR it may still be a good idea to stick with RIP relative addresses as they are less likely to be broken with relocation.

And sorry for the noise.

#5 lore raw
Quoted 47 lines · Pawan Gupta
Quoted 40 lines · Pawan Gupta
Quoted 39 lines · Jiri Slaby
Quoted 37 lines · Nikolay Borisov
Quoted 24 lines · Pawan Gupta

Macro used for MDS mitigation executes VERW with relative addressing for the operand. This is unnecessary and creates a problem for backports on older kernels that don't support relocations in alternatives. Relocation support was added by commit 270a69c4485d ("x86/alternative: Support relocations in alternatives"). Also asm for fixed addressing is much more cleaner than relative RIP addressing.

Simplify the asm by using fixed addressing for VERW operand.

Fixes baf8361e5455 ("x86/bugs: Add asm helpers for executing VERW")
---
   arch/x86/include/asm/nospec-branch.h | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/nospec-branch.h +0 -0
index 2aa52cab1e46..ab19c7f1167b 100644
@@ -323,7 +323,7 @@
    * Note: Only the memory operand variant of VERW clears the CPU
Quoted 7 lines

buffers.
   */
  .macro CLEAR_CPU_BUFFERS
-    ALTERNATIVE "", __stringify(verw _ASM_RIP(mds_verw_sel)), X86_FEATURE_CLEAR_CPU_BUF
+    ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF

Actually thinking about it more and discussing with Jiri (cc'ed), will this work with KASLR enabled?

Quoted 5 lines
Quoted 1 line

I might of course be wrong. We appear to rely on the asm+linker here.

You were right, with KASLR enabled, instructions with fixed addressing in alternatives don't get relocated. I guess we will have to keep rip-relative as is. Thanks for catching that.

Quoted 5 lines

Looks like this is not settled yet, it was naive of me to trust gdb on /proc/kcore earlier with KASLR enabled.

With the below debug patch it appears the relocation with fixed addresses is working as expected with KASLR enabled.

As I wrote already, asm+linker converts the fixed address to rip-rela anyway:

https://lore.kernel.org/all/[email protected]/

I also raised questions in there:
====
The assembler generates a relocation for the fixed address anyway. And the linker resolves it as rip-relative. At least the pair from my binutils-2.42.

But if it generates a rip-relative address, is < 6.5 with no support of rip-rel in alternatives still fine?

Another question: can we rely on the assembler to generate a relocation and on the linker to resolve it as rip-relative?
====

thanks,

Signature · 2 lines
js
suse labs
author reviewer
#6 lore raw
Quoted 59 lines · Jiri Slaby
Quoted 47 lines · Pawan Gupta
Quoted 40 lines · Pawan Gupta
Quoted 39 lines · Jiri Slaby
Quoted 37 lines · Nikolay Borisov
Quoted 24 lines · Pawan Gupta

Macro used for MDS mitigation executes VERW with relative addressing for the operand. This is unnecessary and creates a problem for backports on older kernels that don't support relocations in alternatives. Relocation support was added by commit 270a69c4485d ("x86/alternative: Support relocations in alternatives"). Also asm for fixed addressing is much more cleaner than relative RIP addressing.

Simplify the asm by using fixed addressing for VERW operand.

Fixes baf8361e5455 ("x86/bugs: Add asm helpers for executing VERW")
---
   arch/x86/include/asm/nospec-branch.h | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/nospec-branch.h +0 -0
index 2aa52cab1e46..ab19c7f1167b 100644
@@ -323,7 +323,7 @@
    * Note: Only the memory operand variant of VERW clears the CPU
Quoted 7 lines

buffers.
   */
  .macro CLEAR_CPU_BUFFERS
-    ALTERNATIVE "", __stringify(verw _ASM_RIP(mds_verw_sel)), X86_FEATURE_CLEAR_CPU_BUF
+    ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF

Actually thinking about it more and discussing with Jiri (cc'ed), will this work with KASLR enabled?

Quoted 5 lines
Quoted 1 line

I might of course be wrong. We appear to rely on the asm+linker here.

You were right, with KASLR enabled, instructions with fixed addressing in alternatives don't get relocated. I guess we will have to keep rip-relative as is. Thanks for catching that.

Quoted 5 lines

Looks like this is not settled yet, it was naive of me to trust gdb on /proc/kcore earlier with KASLR enabled.

With the below debug patch it appears the relocation with fixed addresses is working as expected with KASLR enabled.

As I wrote already, asm+linker converts the fixed address to rip-rela anyway:

https://lore.kernel.org/all/[email protected]/

Quoted 8 lines

I also raised questions in there:
====
The assembler generates a relocation for the fixed address anyway. And the linker resolves it as rip-relative. At least the pair from my binutils-2.42.

But if it generates a rip-relative address, is < 6.5 with no support of rip-rel in alternatives still fine?

In that case backports < 6.5 can do:

+.macro CLEAR_CPU_BUFFERS
+ ALTERNATIVE "jmp .Lskip_verw_\@", "", X86_FEATURE_CLEAR_CPU_BUF
+ verw _ASM_RIP(mds_verw_sel)
+.Lskip_verw_\@:
+.endm

As done in Nikolay's 5.4 backport:

https://lore.kernel.org/stable/[email protected]/

Quoted 2 lines

Another question: can we rely on the assembler to generate a relocation and on the linker to resolve it as rip-relative?

This is definitely not my area of expertise, but with the above approach VERW should be inlined always, and rip-relative should be resolved as with any other instruction.

author reviewer
#7 lore raw
In reply to Pawan Gupta
Quoted 22 lines · Pawan Gupta

Macro used for MDS mitigation executes VERW with relative addressing for the operand. This is unnecessary and creates a problem for backports on older kernels that don't support relocations in alternatives. Relocation support was added by commit 270a69c4485d ("x86/alternative: Support relocations in alternatives"). Also asm for fixed addressing is much more cleaner than relative RIP addressing.

Simplify the asm by using fixed addressing for VERW operand.

Fixes baf8361e5455 ("x86/bugs: Add asm helpers for executing VERW")
---
 arch/x86/include/asm/nospec-branch.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/nospec-branch.h +0 -0
index 2aa52cab1e46..ab19c7f1167b 100644

arch/x86/include/asm/nospec-branch.h line 326

Quoted 5 lines
arch/x86/include/asm/nospec-branch.h +1 -1
@@ -323,7 +323,7 @@
  * Note: Only the memory operand variant of VERW clears the CPU buffers.
  */
 .macro CLEAR_CPU_BUFFERS
-	ALTERNATIVE "", __stringify(verw _ASM_RIP(mds_verw_sel)), X86_FEATURE_CLEAR_CPU_BUF
+	ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF

Extremely sorry this change will not work with KASLR. The patch was rushed to facilitate backports, and was only tested on qemu that had KASLR disabled. :( Lesson learnt.

Please drop this patch.