Skip to contentSkip to conversation

x86/cet: Use dedicated NOP4 for cf_clobber

PATCH In review Andrew Cooper 3 participants 8 messages Mar 8, 2022 – Mar 11, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/cet: Use dedicated NOP4 for cf_clobber

For livepatching, we need to look at a potentially clobbered function and determine whether it used to have an ENDBR64 instruction.

Use a non-default 4-byte P6 long nop, not emitted by toolchains, and introduce the was_endbr64() predicate.

---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>
CC: Bjoern Doebel <[email protected]>
CC: Michael Kurth <[email protected]>
CC: Martin Pohlack <[email protected]>

Bjoern: For the livepatching code, I think you want:

if ( is_endbr64(...) || was_endbr64(...) )

      needed += ENDBR64_LEN;
---
 xen/arch/x86/alternative.c       | 10 +++++++++-
 xen/arch/x86/include/asm/endbr.h | 12 ++++++++++++
 2 files changed, 21 insertions(+), 1 deletion(-)
xen/arch/x86/alternative.c +9 -1
index d41eeef1bcaf..ffb1b1d960c8 100644
@@ -362,7 +362,15 @@ static void init_or_livepatch _apply_alternatives(struct alt_instr *start,
             if ( !is_kernel_text(ptr) || !is_endbr64(ptr) )
                 continue;
 
-            add_nops(ptr, ENDBR64_LEN);
+            /*
+             * Can't use add_nops() here.  ENDBR64_POISON is specifically
+             * different to NOP4 so it can be spotted after the fact.
+             *
+             * All CET-capable hardware uses P6 NOPS (no need to plumb through
+             * ideal_nops), and doesn't require a branch to synchronise the
+             * instruction stream.
+             */
+            memcpy(ptr, ENDBR64_POISON, ENDBR64_LEN);
             clobbered++;
         }
 
xen/arch/x86/include/asm/endbr.h +12 -0
index 6090afeb0bd8..5e1e55cb467d 100644
@@ -52,4 +52,16 @@ static inline void place_endbr64(void *ptr)
     *(uint32_t *)ptr = gen_endbr64();
 }
 
+/*
+ * After clobbering ENDBR64, we may need to confirm that the site used to
+ * contain an ENDBR64 instruction.  Use an encoding which isn't the default
+ * P6_NOP4.
+ */
+#define ENDBR64_POISON "\x66\x0f\x1f\x00" /* osp nopl (%rax) */
+
+static inline bool was_endbr64(const void *ptr)
+{
+    return *(const uint32_t *)ptr == 0x001f0f66;
+}
+
 #endif /* XEN_ASM_ENDBR_H */
Signature · 1 line
2.11.0
#1 lore raw
Quoted 5 lines · Andrew Cooper

For livepatching, we need to look at a potentially clobbered function and determine whether it used to have an ENDBR64 instruction.

Use a non-default 4-byte P6 long nop, not emitted by toolchains, and introduce the was_endbr64() predicate.

Did you consider using ENDBR32 for this purpose? I'm worried that the pattern you picked is still too close to what might be used (down the road) by a tool chain. If ENDBR32 isn't suitable for some reason, how about "nop %sp" or "nopw (%rsp)" (and then maybe even "data16" substituted by rex, cs, ds, es, or ss)?

One neat thing about ENDBR32 would be that you wouldn't even need memcpy() - you'd merely flip the low main opcode bit.

Quoted 4 lines

Bjoern: For the livepatching code, I think you want:

  if ( is_endbr64(...) || was_endbr64(...) )
      needed += ENDBR64_LEN;

Looks like I didn't fully understand the problem then from your initial description. The adjustment here (and the one needed in Björn's patch) is to compensate for the advancing of the targets of altcalls past the ENDBR?

xen/arch/x86/include/asm/endbr.h static inline void place_endbr64(void *ptr) · line 60

Quoted 12 lines
xen/arch/x86/include/asm/endbr.h +6 -0
@@ -52,4 +52,16 @@ static inline void place_endbr64(void *ptr)
     *(uint32_t *)ptr = gen_endbr64();
 }
 
+/*
+ * After clobbering ENDBR64, we may need to confirm that the site used to
+ * contain an ENDBR64 instruction.  Use an encoding which isn't the default
+ * P6_NOP4.
+ */
+#define ENDBR64_POISON "\x66\x0f\x1f\x00" /* osp nopl (%rax) */

In case this remains as is - did you mean "opsz" instead of "osp"? But this really is "nopw (%rax)" anyway.

Jan

#2 lore raw
Quoted 7 lines · Jan Beulich
Quoted 5 lines · Andrew Cooper

For livepatching, we need to look at a potentially clobbered function and determine whether it used to have an ENDBR64 instruction.

Use a non-default 4-byte P6 long nop, not emitted by toolchains, and introduce the was_endbr64() predicate.

Did you consider using ENDBR32 for this purpose?

No, and no because that's very short sighted.  Even 4 non-nops would be better than ENDBR32, because they wouldn't create actually-usable codepaths in corner cases we care to exclude.

Quoted 3 lines

I'm worried that
the pattern you picked is still too close to what might be used (down the road) by a tool chain.

This is what Linux are doing too, but no - I'm not worried.  The encoding isn't the only protection; toolchains also have no reason to put a nop4 at the head of functions; nop5 is the common one to find.

Quoted 2 lines

One neat thing about ENDBR32 would be that you wouldn't even need memcpy() - you'd merely flip the low main opcode bit.

Not relevant.  You're taking the SMC pipeline hit for any sized write, and a single movl is far less cryptic.

Quoted 8 lines
Quoted 4 lines

Bjoern: For the livepatching code, I think you want:

  if ( is_endbr64(...) || was_endbr64(...) )
      needed += ENDBR64_LEN;

Looks like I didn't fully understand the problem then from your initial description. The adjustment here (and the one needed in Björn's patch) is to compensate for the advancing of the targets of altcalls past the ENDBR?

No.  Consider this scenario:

callee:
    endbr64
    ...

altcall:
    call *foo(%rip)

During boot, we rewrite altcall to be call callee+4 and turn endbr64 into nops, so it now looks like:

callee:
    nop4
    ...

altcall:
    call callee+4

Then we want to livepatch callee to callee_new, so we get

callee_new:
    endbr64
    ...

in the livepatch itself.

Now, to actually patch, we need to memcpy(callee+4, "jmp callee_new", 5).

The livepatch logic calling is_endbr(callee) doesn't work because it's now a nop4, which is why it needs a was_endbr64(callee) too.

Quoted 7 lines
Quoted 7 lines
xen/arch/x86/include/asm/endbr.h +0 -0
@@ -52,4 +52,16 @@ static inline void place_endbr64(void *ptr)
     *(uint32_t *)ptr = gen_endbr64();
 }
 
Quoted 8 lines

xen/arch/x86/include/asm/endbr.h static inline void place_endbr64(void *ptr) · line 60

Quoted 6 lines
xen/arch/x86/include/asm/endbr.h +6 -0
+/*
+ * After clobbering ENDBR64, we may need to confirm that the site used to
+ * contain an ENDBR64 instruction.  Use an encoding which isn't the default
+ * P6_NOP4.
+ */
+#define ENDBR64_POISON "\x66\x0f\x1f\x00" /* osp nopl (%rax) */

In case this remains as is - did you mean "opsz" instead of "osp"? But this really is "nopw (%rax)" anyway.

Oh, osp is the nasm name.  I'll switch to nopw.

~Andrew

Reviewed-by
#3 lore raw
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Jan Beulich
Quoted 3 lines · Andrew Cooper

For livepatching, we need to look at a potentially clobbered function and determine whether it used to have an ENDBR64 instruction.

Quoted 7 lines
Quoted 3 lines
Quoted 2 lines

Use a non-default 4-byte P6 long nop, not emitted by toolchains, and introduce the was_endbr64() predicate.

Did you consider using ENDBR32 for this purpose?

No, and no because that's very short sighted.  Even 4 non-nops would be better than ENDBR32, because they wouldn't create actually-usable codepaths in corner cases we care to exclude.

Well - I thought of ENDBR32 because elsewhere you said we don't need to be worried about any byte sequence resembling that insn, since for it to become "usable" an attacker would first need to have managed to manufacture a 32-bit ring0 code segment. Now you say effectively the opposite.

Quoted 7 lines
Quoted 3 lines

I'm worried that
the pattern you picked is still too close to what might be used (down the road) by a tool chain.

This is what Linux are doing too, but no - I'm not worried.  The encoding isn't the only protection; toolchains also have no reason to put a nop4 at the head of functions; nop5 is the common one to find.

Well, okay - let's hope for the best then.

Quoted 34 lines
Quoted 8 lines
Quoted 4 lines

Bjoern: For the livepatching code, I think you want:

  if ( is_endbr64(...) || was_endbr64(...) )
      needed += ENDBR64_LEN;

Looks like I didn't fully understand the problem then from your initial description. The adjustment here (and the one needed in Björn's patch) is to compensate for the advancing of the targets of altcalls past the ENDBR?

No.  Consider this scenario:

callee:
    endbr64
    ...

altcall:
    call *foo(%rip)

During boot, we rewrite altcall to be call callee+4 and turn endbr64 into nops, so it now looks like:

callee:
    nop4
    ...

altcall:
    call callee+4

Then we want to livepatch callee to callee_new, so we get

callee_new:
    endbr64
    ...

Quoted 6 lines

in the livepatch itself.

Now, to actually patch, we need to memcpy(callee+4, "jmp callee_new", 5).

The livepatch logic calling is_endbr(callee) doesn't work because it's now a nop4, which is why it needs a was_endbr64(callee) too.

Sounds like exactly what I was thinking of. Perhaps my description wasn't sufficiently clear / unambiguous then.

Quoted 15 lines
Quoted 6 lines
Quoted 6 lines
xen/arch/x86/include/asm/endbr.h +0 -0
@@ -52,4 +52,16 @@ static inline void place_endbr64(void *ptr)
     *(uint32_t *)ptr = gen_endbr64();
 }
 
Quoted 8 lines

xen/arch/x86/include/asm/endbr.h static inline void place_endbr64(void *ptr) · line 60

Quoted 6 lines
xen/arch/x86/include/asm/endbr.h +6 -0
+/*
+ * After clobbering ENDBR64, we may need to confirm that the site used to
+ * contain an ENDBR64 instruction.  Use an encoding which isn't the default
+ * P6_NOP4.
+ */
+#define ENDBR64_POISON "\x66\x0f\x1f\x00" /* osp nopl (%rax) */

In case this remains as is - did you mean "opsz" instead of "osp"? But this really is "nopw (%rax)" anyway.

Quoted 1 line

Oh, osp is the nasm name.  I'll switch to nopw.

Jan

#4 lore raw
Quoted 9 lines · Jan Beulich
Quoted 8 lines · Andrew Cooper
Quoted 7 lines · Jan Beulich
Quoted 5 lines · Andrew Cooper

For livepatching, we need to look at a potentially clobbered function and determine whether it used to have an ENDBR64 instruction.

Use a non-default 4-byte P6 long nop, not emitted by toolchains, and introduce the was_endbr64() predicate.

Did you consider using ENDBR32 for this purpose?

Quoted 8 lines
Quoted 3 lines

No, and no because that's very short sighted.  Even 4 non-nops would be better than ENDBR32, because they wouldn't create actually-usable codepaths in corner cases we care to exclude.

Well - I thought of ENDBR32 because elsewhere you said we don't need to be worried about any byte sequence resembling that insn, since for it to become "usable" an attacker would first need to have managed to manufacture a 32-bit ring0 code segment. Now you say effectively the opposite.

We've got ~0 risk of having any embedded ENDBR32's because we never refer to the opcode, and therefore adding 2x 0.7s delay to scan the binary on build is a waste.  If the check were free, it would be a different matter.

At any point, if we were to introduce references to ENDBR32, we'd want to start scanning for embedded sequences.

But at no point do we want to intentionally remove our defence in depth created by both having no CS32 code segment, and no ENDBR32 instructions.

Quoted 42 lines
Quoted 7 lines
Quoted 4 lines

I'm worried that
the pattern you picked is still too close to what might be used (down the road) by a tool chain.

This is what Linux are doing too, but no - I'm not worried.  The encoding isn't the only protection; toolchains also have no reason to put a nop4 at the head of functions; nop5 is the common one to find.

Well, okay - let's hope for the best then.

Quoted 33 lines
Quoted 8 lines
Quoted 4 lines

Bjoern: For the livepatching code, I think you want:

  if ( is_endbr64(...) || was_endbr64(...) )
      needed += ENDBR64_LEN;

Looks like I didn't fully understand the problem then from your initial description. The adjustment here (and the one needed in Björn's patch) is to compensate for the advancing of the targets of altcalls past the ENDBR?

No.  Consider this scenario:

callee:
    endbr64
    ...

altcall:
    call *foo(%rip)

During boot, we rewrite altcall to be call callee+4 and turn endbr64 into nops, so it now looks like:

callee:
    nop4
    ...

altcall:
    call callee+4

Then we want to livepatch callee to callee_new, so we get

callee_new:
    endbr64
    ...

Quoted 8 lines
Quoted 6 lines

in the livepatch itself.

Now, to actually patch, we need to memcpy(callee+4, "jmp callee_new", 5).

The livepatch logic calling is_endbr(callee) doesn't work because it's now a nop4, which is why it needs a was_endbr64(callee) too.

Sounds like exactly what I was thinking of. Perhaps my description wasn't sufficiently clear / unambiguous then.

Ah yes - I think I did misinterpret what you wrote.  I hope everything is clear now.

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

xen/arch/x86/include/asm/endbr.h static inline void place_endbr64(void *ptr) · line 60

Quoted 13 lines
xen/arch/x86/include/asm/endbr.h +6 -0
@@ -52,4 +52,16 @@ static inline void place_endbr64(void *ptr)
     *(uint32_t *)ptr = gen_endbr64();
 }
 
+/*
+ * After clobbering ENDBR64, we may need to confirm that the site used to
+ * contain an ENDBR64 instruction.  Use an encoding which isn't the default
+ * P6_NOP4.
+ */
+#define ENDBR64_POISON "\x66\x0f\x1f\x00" /* osp nopl (%rax) */

In case this remains as is - did you mean "opsz" instead of "osp"? But this really is "nopw (%rax)" anyway.

Quoted 2 lines
Quoted 1 line

Oh, osp is the nasm name.  I'll switch to nopw.

Thanks.

~Andrew

#5 lore raw
Quoted 14 lines · Andrew Cooper
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

xen/arch/x86/include/asm/endbr.h static inline void place_endbr64(void *ptr) · line 60

Quoted 12 lines
xen/arch/x86/include/asm/endbr.h +6 -0
@@ -52,4 +52,16 @@ static inline void place_endbr64(void *ptr)
     *(uint32_t *)ptr = gen_endbr64();
 }
 
+/*
+ * After clobbering ENDBR64, we may need to confirm that the site used to
+ * contain an ENDBR64 instruction.  Use an encoding which isn't the default
+ * P6_NOP4.
+ */
+#define ENDBR64_POISON "\x66\x0f\x1f\x00" /* osp nopl (%rax) */

In case this remains as is - did you mean "opsz" instead of "osp"? But this really is "nopw (%rax)" anyway.

Quoted 3 lines
Quoted 2 lines
Quoted 1 line

Oh, osp is the nasm name.  I'll switch to nopw.

Thanks.

It does occur to me that we can extend check-endbr.sh for this.

xen/arch/x86/indirect-thunk.S +0 -0
index 7cc22da0ef93..3baaf7ab4983 100644
@@ -38,6 +38,7 @@

         .section .text.__x86_indirect_thunk_\reg, "ax", @progbits
 ENTRY(__x86_indirect_thunk_\reg)
+       nopw (%rax)
         ALTERNATIVE_2 __stringify(IND_THUNK_RETPOLINE \reg),              \
         __stringify(IND_THUNK_LFENCE \reg), X86_FEATURE_IND_THUNK_LFENCE, \
         __stringify(IND_THUNK_JMP \reg),    X86_FEATURE_IND_THUNK_JMP

xen/tools/check-endbr.sh +0 -0
index 9799c451a18d..652ac8d0b983 100755
@@ -67,7 +67,7 @@ eval $(${OBJDUMP} -j .text $1 -h |

 ${OBJCOPY} -j .text $1 -O binary $TEXT_BIN
 if $perl_re
 then
-    LC_ALL=C grep -aobP '\363\17\36\372' $TEXT_BIN
+    LC_ALL=C grep -aobP '\363\17\36\372|\x66\x0f\x1f\x00' $TEXT_BIN else
     grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN
 fi | awk -F':' '{printf "%s%x\n", "'$vma_hi'", int(0x'$vma_lo') + $1}'

Quoted 1 line

$ALL

yields:

check-endbr.sh xen-syms Fail: Found 15 embedded endbr64 instructions 0xffff82d040377f00: __x86_indirect_thunk_rax at /local/xen.git/xen/arch/x86/indirect-thunk.S:55 0xffff82d040377f20: __x86_indirect_thunk_rcx at ??:? 0xffff82d040377f40: __x86_indirect_thunk_rdx at ??:? 0xffff82d040377f60: __x86_indirect_thunk_rbx at ??:? 0xffff82d040377f80: __x86_indirect_thunk_rbp at ??:? 0xffff82d040377fa0: __x86_indirect_thunk_rsi at ??:? 0xffff82d040377fc0: __x86_indirect_thunk_rdi at ??:? 0xffff82d040377fe0: __x86_indirect_thunk_r8 at ??:? 0xffff82d040378000: __x86_indirect_thunk_r9 at ??:? 0xffff82d040378020: __x86_indirect_thunk_r10 at ??:? 0xffff82d040378040: __x86_indirect_thunk_r11 at ??:? 0xffff82d040378060: __x86_indirect_thunk_r12 at ??:? 0xffff82d040378080: __x86_indirect_thunk_r13 at ??:? 0xffff82d0403780a0: __x86_indirect_thunk_r14 at ??:? 0xffff82d0403780c0: __x86_indirect_thunk_r15 at ??:?
...
check-endbr.sh xen.efi Fail: Found 15 embedded endbr64 instructions 0xffff82d040377f00: ?? at /local/xen.git/xen/arch/x86/indirect-thunk.S:55 0xffff82d040377f20: ?? at head.o:?
0xffff82d040377f40: ?? at head.o:?
0xffff82d040377f60: ?? at head.o:?
0xffff82d040377f80: ?? at head.o:?
0xffff82d040377fa0: ?? at head.o:?
0xffff82d040377fc0: ?? at head.o:?
0xffff82d040377fe0: ?? at head.o:?
0xffff82d040378000: ?? at head.o:?
0xffff82d040378020: ?? at head.o:?
0xffff82d040378040: ?? at head.o:?
0xffff82d040378060: ?? at head.o:?
0xffff82d040378080: ?? at head.o:?
0xffff82d0403780a0: ?? at head.o:?
0xffff82d0403780c0: ?? at head.o:?

Obviously the changes to check-endbr want cleaning up, but I think it's entirely within scope to check for ENDBR64_POISON too, and we can do it without adding an extra pass.  Would you be happier with this check added?

But we also have some clear errors with debug symbols.  It's perhaps not terribly surprising that irp/endr only gets file/line for the first instance, and at least ELF manage to get the function name right, but EFI is a mess and manages to get the wrong file.  Any idea how to get rather less nonsense out of the debug symbols?

~Andrew

#6 lore raw
Quoted 85 lines · Andrew Cooper
Quoted 14 lines · Andrew Cooper
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

xen/arch/x86/include/asm/endbr.h static inline void place_endbr64(void *ptr) · line 60

Quoted 12 lines
xen/arch/x86/include/asm/endbr.h +6 -0
@@ -52,4 +52,16 @@ static inline void place_endbr64(void *ptr)
     *(uint32_t *)ptr = gen_endbr64();
 }
 
+/*
+ * After clobbering ENDBR64, we may need to confirm that the site used to
+ * contain an ENDBR64 instruction.  Use an encoding which isn't the default
+ * P6_NOP4.
+ */
+#define ENDBR64_POISON "\x66\x0f\x1f\x00" /* osp nopl (%rax) */

In case this remains as is - did you mean "opsz" instead of "osp"? But this really is "nopw (%rax)" anyway.

Quoted 3 lines
Quoted 2 lines
Quoted 1 line

Oh, osp is the nasm name.  I'll switch to nopw.

Thanks.

It does occur to me that we can extend check-endbr.sh for this.

xen/arch/x86/indirect-thunk.S +0 -0
index 7cc22da0ef93..3baaf7ab4983 100644
@@ -38,6 +38,7 @@

         .section .text.__x86_indirect_thunk_\reg, "ax", @progbits
 ENTRY(__x86_indirect_thunk_\reg)
+       nopw (%rax)
         ALTERNATIVE_2 __stringify(IND_THUNK_RETPOLINE \reg),              \
         __stringify(IND_THUNK_LFENCE \reg), X86_FEATURE_IND_THUNK_LFENCE, \
         __stringify(IND_THUNK_JMP \reg),    X86_FEATURE_IND_THUNK_JMP

xen/tools/check-endbr.sh +0 -0
index 9799c451a18d..652ac8d0b983 100755
@@ -67,7 +67,7 @@ eval $(${OBJDUMP} -j .text $1 -h |

 ${OBJCOPY} -j .text $1 -O binary $TEXT_BIN
 if $perl_re
 then
-    LC_ALL=C grep -aobP '\363\17\36\372' $TEXT_BIN
+    LC_ALL=C grep -aobP '\363\17\36\372|\x66\x0f\x1f\x00' $TEXT_BIN else
     grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN
 fi | awk -F':' '{printf "%s%x\n", "'$vma_hi'", int(0x'$vma_lo') + $1}'

Quoted 1 line

$ALL

yields:

check-endbr.sh xen-syms Fail: Found 15 embedded endbr64 instructions 0xffff82d040377f00: __x86_indirect_thunk_rax at /local/xen.git/xen/arch/x86/indirect-thunk.S:55 0xffff82d040377f20: __x86_indirect_thunk_rcx at ??:? 0xffff82d040377f40: __x86_indirect_thunk_rdx at ??:? 0xffff82d040377f60: __x86_indirect_thunk_rbx at ??:? 0xffff82d040377f80: __x86_indirect_thunk_rbp at ??:? 0xffff82d040377fa0: __x86_indirect_thunk_rsi at ??:? 0xffff82d040377fc0: __x86_indirect_thunk_rdi at ??:? 0xffff82d040377fe0: __x86_indirect_thunk_r8 at ??:? 0xffff82d040378000: __x86_indirect_thunk_r9 at ??:? 0xffff82d040378020: __x86_indirect_thunk_r10 at ??:? 0xffff82d040378040: __x86_indirect_thunk_r11 at ??:? 0xffff82d040378060: __x86_indirect_thunk_r12 at ??:? 0xffff82d040378080: __x86_indirect_thunk_r13 at ??:? 0xffff82d0403780a0: __x86_indirect_thunk_r14 at ??:? 0xffff82d0403780c0: __x86_indirect_thunk_r15 at ??:?
...
check-endbr.sh xen.efi Fail: Found 15 embedded endbr64 instructions 0xffff82d040377f00: ?? at /local/xen.git/xen/arch/x86/indirect-thunk.S:55 0xffff82d040377f20: ?? at head.o:?
0xffff82d040377f40: ?? at head.o:?
0xffff82d040377f60: ?? at head.o:?
0xffff82d040377f80: ?? at head.o:?
0xffff82d040377fa0: ?? at head.o:?
0xffff82d040377fc0: ?? at head.o:?
0xffff82d040377fe0: ?? at head.o:?
0xffff82d040378000: ?? at head.o:?
0xffff82d040378020: ?? at head.o:?
0xffff82d040378040: ?? at head.o:?
0xffff82d040378060: ?? at head.o:?
0xffff82d040378080: ?? at head.o:?
0xffff82d0403780a0: ?? at head.o:?
0xffff82d0403780c0: ?? at head.o:?

Quoted 3 lines

Obviously the changes to check-endbr want cleaning up, but I think it's entirely within scope to check for ENDBR64_POISON too, and we can do it without adding an extra pass.  Would you be happier with this check added?

Yes, this would feel better. Thanks for having continued to think about it.

Quoted 3 lines

But we also have some clear errors with debug symbols.  It's perhaps not terribly surprising that irp/endr only gets file/line for the first instance,

I have to admit I would expect it to at least figure the file. But there's no .debug_line contents at all for ..._rcx .. ..._r15.

Quoted 3 lines

and at least ELF manage to get the function name right, but EFI is a mess and manages to get the wrong file.  Any idea how to get rather less nonsense out of the debug symbols?

A random example with a symbol from a C file works here, at least.

Jan

Reviewed-by on v1 1/1
#7 lore raw
In reply to Andrew Cooper
Quoted 71 lines · Andrew Cooper

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

For livepatching, we need to look at a potentially clobbered function and determine whether it used to have an ENDBR64 instruction.

Use a non-default 4-byte P6 long nop, not emitted by toolchains, and introduce the was_endbr64() predicate.

---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>
CC: Bjoern Doebel <[email protected]>
CC: Michael Kurth <[email protected]>
CC: Martin Pohlack <[email protected]>

Bjoern: For the livepatching code, I think you want:

   if ( is_endbr64(...) || was_endbr64(...) )
       needed += ENDBR64_LEN;
---
  xen/arch/x86/alternative.c       | 10 +++++++++-
  xen/arch/x86/include/asm/endbr.h | 12 ++++++++++++
  2 files changed, 21 insertions(+), 1 deletion(-)
xen/arch/x86/alternative.c +9 -1
index d41eeef1bcaf..ffb1b1d960c8 100644
@@ -362,7 +362,15 @@ static void init_or_livepatch _apply_alternatives(struct alt_instr *start,
              if ( !is_kernel_text(ptr) || !is_endbr64(ptr) )
                  continue;
 
-            add_nops(ptr, ENDBR64_LEN);
+            /*
+             * Can't use add_nops() here.  ENDBR64_POISON is specifically
+             * different to NOP4 so it can be spotted after the fact.
+             *
+             * All CET-capable hardware uses P6 NOPS (no need to plumb through
+             * ideal_nops), and doesn't require a branch to synchronise the
+             * instruction stream.
+             */
+            memcpy(ptr, ENDBR64_POISON, ENDBR64_LEN);
              clobbered++;
          }
 
xen/arch/x86/include/asm/endbr.h +12 -1
index 6090afeb0bd8..5e1e55cb467d 100644
@@ -52,4 +52,16 @@ static inline void place_endbr64(void *ptr)
      *(uint32_t *)ptr = gen_endbr64();
  }
 
+/*
+ * After clobbering ENDBR64, we may need to confirm that the site used to
+ * contain an ENDBR64 instruction.  Use an encoding which isn't the default
+ * P6_NOP4.
+ */
+#define ENDBR64_POISON "\x66\x0f\x1f\x00" /* osp nopl (%rax) */
+
+static inline bool was_endbr64(const void *ptr)
+{
+    return *(const uint32_t *)ptr == 0x001f0f66;
+}
+
  #endif /* XEN_ASM_ENDBR_H */
--
Quoted 1 line

2.11.0

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879