Skip to contentSkip to conversation

common: honor CONFIG_CC_SPLIT_SECTIONS also for assembly functions

PATCH In review Jan Beulich 3 participants 6 messages Apr 1, 2025 – Feb 13 RISC-V

lore
Share & tools
lore
#0 lore raw

PATCH v9 1/1 common: honor CONFIG_CC_SPLIT_SECTIONS also for assembly functions

Leverage the new infrastructure in xen/linkage.h to also switch to per- function sections (when configured), deriving the specific name from the "base" section in use at the time FUNC() is invoked.

---
TBD: Since we use .subsection in UNLIKELY_START(), a perhaps not really
wanted side effect of this change is that respective out-of-line code now moves much closer to its original (invoking) code.

TBD: Of course something with the same overall effect, but less
impactful might do in Config.mk. E.g. $(filter-out -D%,$(3)) instead of $(firstword (3)). In fact Roger wants the detection to be in Kconfig, for LIVEPATCH to depend on it. Yet the whole underlying discussion there imo would first need settling (and therefore reviving).

Note that we'd need to split DATA() in order to separate r/w, r/o, and BSS contributions. Further splitting might be needed to also support more advanced attributes (e.g. merge), hence why this isn't done right here. Sadly while a new section's name can be derived from the presently in use, its attributes cannot be. Perhaps the only thing we can do is give DATA() a 2nd mandatory parameter. Then again I guess most data definitions could be moved to C anyway.
---
v9: Move Arm32 SYM_PUSH_SECTION() overrides here.
v7: Override SYM_PUSH_SECTION() in arch/x86/indirect-thunk.S. Re-base,
notably to deal with fallout from fba250ae604e ("xen/arm64: head:
Add missing code symbol annotations").
v6: Deal with x86'es entry_PF() and entry_int82() falling through to the
next "function". Re-base.
v5: Re-base over changes earlier in the series.
v4: Re-base.
v2: Make detection properly fail on old gas (by adjusting
cc-option-add-closure).

Config.mk +1 -1
@@ -102,7 +102,7 @@ cc-option = $(shell if $(1) $(2:-Wno-%=-
 # Usage: $(call cc-option-add CFLAGS,CC,-march=winchip-c6)
 cc-option-add = $(eval $(call cc-option-add-closure,$(1),$(2),$(3)))
 define cc-option-add-closure
-    ifneq ($$(call cc-option,$$($(2)),$(3),n),n)
+    ifneq ($$(call cc-option,$$($(2)),$(firstword $(3)),n),n)
         $(1) += $(3)
     endif
 endef
xen/Makefile +3 -0
@@ -415,6 +415,9 @@ AFLAGS += -D__ASSEMBLY__
 
 $(call cc-option-add,AFLAGS,CC,-Wa$$(comma)--noexecstack)
 
+# Check to see whether the assembler supports the --sectname-subst option.
+$(call cc-option-add,AFLAGS,CC,-Wa$$(comma)--sectname-subst -DHAVE_AS_SECTNAME_SUBST)
+
 LDFLAGS-$(call ld-option,--warn-rwx-segments) += --no-warn-rwx-segments
 
 CFLAGS += $(CFLAGS-y)
xen/arch/arm/arm32/head.S +7 -0
@@ -48,6 +48,13 @@
 
         .section .text.header, "ax", %progbits
         .arm
+/*
+ * Code below wants to all live in the section established above.  Annotations
+ * from xen/linkage.h therefore may not switch sections (honoring
+ * CONFIG_CC_SPLIT_SECTIONS).  Override the respective macro.
+ */
+#undef SYM_PUSH_SECTION
+#define SYM_PUSH_SECTION(name, attr)
 
         /*
          * This must be the very first address in the loaded image.
xen/arch/arm/arm32/mmu/head.S +7 -0
@@ -160,6 +160,13 @@
 .endm
 
 .section .text.idmap, "ax", %progbits
+/*
+ * Code below wants to all live in the section established above.  Annotations
+ * from xen/linkage.h therefore may not switch sections (honoring
+ * CONFIG_CC_SPLIT_SECTIONS).  Override the respective macro.
+ */
+#undef SYM_PUSH_SECTION
+#define SYM_PUSH_SECTION(name, attr)
 
 /*
  * Rebuild the boot pagetable's first-level entries. The structure
xen/arch/arm/arm64/head.S +8 -0
@@ -28,6 +28,14 @@
 #include <asm/arm64/efibind.h>
 #endif
 
+/*
+ * Code here is, at least in part, ordering sensitive.  Annotations
+ * from xen/linkage.h therefore may not switch sections (honoring
+ * CONFIG_CC_SPLIT_SECTIONS).  Override the respective macro.
+ */
+#undef SYM_PUSH_SECTION
+#define SYM_PUSH_SECTION(name, attr)
+
 #define __HEAD_FLAG_PAGE_SIZE   ((PAGE_SHIFT - 10) / 2)
 
 #define __HEAD_FLAG_PHYS_BASE   1
xen/arch/arm/xen.lds.S +3 -0
@@ -140,6 +140,9 @@ SECTIONS
   .init.text : {
        _sinittext = .;
        *(.init.text)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+       *(.init.text.*)
+#endif
        _einittext = .;
        . = ALIGN(PAGE_SIZE);        /* Avoid mapping alt insns executable */
        *(.altinstr_replacement)
xen/arch/ppc/xen.lds.S +3 -0
@@ -103,6 +103,9 @@ SECTIONS
     DECL_SECTION(.init.text) {
         _sinittext = .;
         *(.init.text)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        *(.init.text.*)
+#endif
         _einittext = .;
         . = ALIGN(PAGE_SIZE);        /* Avoid mapping alt insns executable */
     } :text
xen/arch/riscv/xen.lds.S +3 -0
@@ -98,6 +98,9 @@ SECTIONS
     .init.text : {
         _sinittext = .;
         *(.init.text)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        *(.init.text.*)
+#endif
         _einittext = .;
         . = ALIGN(PAGE_SIZE);        /* Avoid mapping alt insns executable */
     } :text
xen/arch/x86/indirect-thunk.S +4 -0
@@ -11,6 +11,10 @@
 
 #include <asm/asm_defns.h>
 
+/* Section placement is done explicitly here; override the respective macro. */
+#undef SYM_PUSH_SECTION
+#define SYM_PUSH_SECTION(name, attr)
+
 .macro IND_THUNK_RETPOLINE reg:req
         call 1f
         int3
xen/arch/x86/x86_64/compat/entry.S +3 -0
@@ -31,6 +31,9 @@ FUNC(entry_int82)
 
         mov   %rsp, %rdi
         call  do_entry_int82
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        jmp   compat_test_all_events
+#endif
 END(entry_int82)
 
 /* %rbx: struct vcpu */
xen/arch/x86/x86_64/entry.S +7 -1
@@ -771,6 +771,9 @@ END(common_interrupt)
 FUNC(entry_PF)
         ENDBR64
         movb  $X86_EXC_PF, EFRAME_entry_vector(%rsp)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        jmp   handle_exception
+#endif
 END(entry_PF)
 /* No special register assumptions. */
 FUNC(handle_exception, 0)
@@ -1084,8 +1087,11 @@ FUNC(entry_NMI)
         ENDBR64
         pushq $0
         movb  $X86_EXC_NMI, EFRAME_entry_vector(%rsp)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        jmp   handle_ist_exception
+#endif
 END(entry_NMI)
-
+/* No special register assumptions. */
 FUNC(handle_ist_exception)
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         SAVE_ALL
xen/arch/x86/xen.lds.S +6 -0
@@ -85,6 +85,9 @@ SECTIONS
        . = ALIGN(PAGE_SIZE);
        _stextentry = .;
        *(.text.entry)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+       *(.text.entry.*)
+#endif
        . = ALIGN(PAGE_SIZE);
        _etextentry = .;
 
@@ -204,6 +207,9 @@ SECTIONS
 #endif
        _sinittext = .;
        *(.init.text)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+       *(.init.text.*)
+#endif
        *(.text.startup)
        _einittext = .;
        /*
xen/include/xen/linkage.h +18 -1
@@ -18,6 +18,14 @@
 
 #define SYM_ALIGN(align...) .balign align
 
+#if defined(HAVE_AS_SECTNAME_SUBST) && defined(CONFIG_CC_SPLIT_SECTIONS)
+# define SYM_PUSH_SECTION(name, attr) \
+         .pushsection %S.name, attr, %progbits; \
+         .equ .Lsplit_section, 1
+#else
+# define SYM_PUSH_SECTION(name, attr)
+#endif
+
 #define SYM_L_GLOBAL(name) .globl name; .hidden name
 #define SYM_L_WEAK(name)   .weak name
 #define SYM_L_LOCAL(name)  /* nothing */
@@ -32,7 +40,14 @@
         SYM_ALIGN(align);                         \
         name:
 
-#define END(name) .size name, . - name
+#define END(name) \
+        .size name, . - name; \
+        .ifdef .Lsplit_section; \
+            .if .Lsplit_section; \
+                .popsection; \
+                .equ .Lsplit_section, 0; \
+            .endif; \
+        .endif
 
 /*
  * CODE_FILL in particular may need to expand to nothing (e.g. for RISC-V), in
@@ -47,6 +62,7 @@
 #endif
 
 #define FUNC(name, align...) \
+        SYM_PUSH_SECTION(name, "ax"); \
         SYM(name, FUNC, GLOBAL, DO_CODE_ALIGN(align))
 #define LABEL(name, align...) \
         SYM(name, NONE, GLOBAL, DO_CODE_ALIGN(align))
@@ -54,6 +70,7 @@
         SYM(name, DATA, GLOBAL, LASTARG(DATA_ALIGN, ## align), DATA_FILL)
 
 #define FUNC_LOCAL(name, align...) \
+        SYM_PUSH_SECTION(name, "ax"); \
         SYM(name, FUNC, LOCAL, DO_CODE_ALIGN(align))
 #define LABEL_LOCAL(name, align...) \
         SYM(name, NONE, LOCAL, DO_CODE_ALIGN(align))
Reviewed-by on v9 1/1
#1 lore raw
Quoted 6 lines · Jan Beulich

Leverage the new infrastructure in xen/linkage.h to also switch to per- function sections (when configured), deriving the specific name from the "base" section in use at the time FUNC() is invoked.

Though I have two suggestions below.

Quoted 34 lines

---
TBD: Since we use .subsection in UNLIKELY_START(), a perhaps not really
wanted side effect of this change is that respective out-of-line code now moves much closer to its original (invoking) code.

TBD: Of course something with the same overall effect, but less
impactful might do in Config.mk. E.g. $(filter-out -D%,$(3)) instead of $(firstword (3)). In fact Roger wants the detection to be in Kconfig, for LIVEPATCH to depend on it. Yet the whole underlying discussion there imo would first need settling (and therefore reviving).

Note that we'd need to split DATA() in order to separate r/w, r/o, and BSS contributions. Further splitting might be needed to also support more advanced attributes (e.g. merge), hence why this isn't done right here. Sadly while a new section's name can be derived from the presently in use, its attributes cannot be. Perhaps the only thing we can do is give DATA() a 2nd mandatory parameter. Then again I guess most data definitions could be moved to C anyway.
---
v9: Move Arm32 SYM_PUSH_SECTION() overrides here.
v7: Override SYM_PUSH_SECTION() in arch/x86/indirect-thunk.S. Re-base,
notably to deal with fallout from fba250ae604e ("xen/arm64: head:
Add missing code symbol annotations").
v6: Deal with x86'es entry_PF() and entry_int82() falling through to the
next "function". Re-base.
v5: Re-base over changes earlier in the series.
v4: Re-base.
v2: Make detection properly fail on old gas (by adjusting
cc-option-add-closure).

Config.mk cc-option = $(shell if $(1) $(2:-Wno-%=- · line 102

Quoted 1 line
Config.mk +0 -0
@@ -102,7 +102,7 @@ cc-option = $(shell if $(1) $(2:-Wno-%=-
  # Usage: $(call cc-option-add CFLAGS,CC,-march=winchip-c6)

Maybe expand to illustrate extra flags will also be passed (-DHAVE_AS_SECTNAME_SUBST)?

Quoted 11 lines
Config.mk +1 -1
  cc-option-add = $(eval $(call cc-option-add-closure,$(1),$(2),$(3)))
  define cc-option-add-closure
-    ifneq ($$(call cc-option,$$($(2)),$(3),n),n)
+    ifneq ($$(call cc-option,$$($(2)),$(firstword $(3)),n),n)
          $(1) += $(3)
      endif
  endef

xen/arch/arm/arm32/head.S line 57

Quoted 10 lines
xen/arch/arm/arm32/head.S +7 -0
@@ -48,6 +48,13 @@
 
          .section .text.header, "ax", %progbits
          .arm
+/*
+ * Code below wants to all live in the section established above.  Annotations
+ * from xen/linkage.h therefore may not switch sections (honoring
+ * CONFIG_CC_SPLIT_SECTIONS).  Override the respective macro.
+ */
+#undef SYM_PUSH_SECTION
+#define SYM_PUSH_SECTION(name, attr)

I put this through CI and it passed as-is, so it doesn't need to change.
However, included in a different branch with some --gc-sections
experiments, I needed to add SYM_PUSH_SECTION re-definitions like above to:

xen/arch/ppc/ppc64/head.S

or ppc failed the linker script ASSERT(_stext_exceptions == EXCEPTION_VECTORS_START);

And these:

xen/arch/riscv/riscv64/head.S
xen/arch/arm/arm64/mmu/head.S

riscv and arm64 built, but hung when booting in CI tests.

There are also these:
xen/arch/arm/arm32/mpu/head.S
xen/arch/arm/arm64/mpu/head.S

They aren't built or tested in CI, FWICT. And they are not in alternate sections. Maybe an ARM person can chime in on those.

Thanks,
Jason

#2 lore raw
Quoted 9 lines · Jason Andryuk
Quoted 6 lines · Jan Beulich

Leverage the new infrastructure in xen/linkage.h to also switch to per- function sections (when configured), deriving the specific name from the "base" section in use at the time FUNC() is invoked.

Thanks, but ...

Quoted 1 line

Though I have two suggestions below.

... at least the latter really needs addressing.

Quoted 7 lines

Config.mk cc-option = $(shell if $(1) $(2:-Wno-%=- · line 102

Quoted 4 lines
Config.mk +0 -0
@@ -102,7 +102,7 @@ cc-option = $(shell if $(1) $(2:-Wno-%=-
  # Usage: $(call cc-option-add CFLAGS,CC,-march=winchip-c6)

Maybe expand to illustrate extra flags will also be passed (-DHAVE_AS_SECTNAME_SUBST)?

I'm not sure here, but I can certainly add another example.

Quoted 18 lines

xen/arch/arm/arm32/head.S line 57

Quoted 13 lines
xen/arch/arm/arm32/head.S +7 -0
@@ -48,6 +48,13 @@
 
          .section .text.header, "ax", %progbits
          .arm
+/*
+ * Code below wants to all live in the section established above.  Annotations
+ * from xen/linkage.h therefore may not switch sections (honoring
+ * CONFIG_CC_SPLIT_SECTIONS).  Override the respective macro.
+ */
+#undef SYM_PUSH_SECTION
+#define SYM_PUSH_SECTION(name, attr)

I put this through CI and it passed as-is, so it doesn't need to change.
However, included in a different branch with some --gc-sections
experiments, I needed to add SYM_PUSH_SECTION re-definitions like above to:

Quoted 4 lines

xen/arch/ppc/ppc64/head.S

or ppc failed the linker script ASSERT(_stext_exceptions == EXCEPTION_VECTORS_START);

Yes, and exceptions-asm.S as well, to keep the code in .text.exceptions.

Quoted 10 lines

And these:

xen/arch/riscv/riscv64/head.S
xen/arch/arm/arm64/mmu/head.S

riscv and arm64 built, but hung when booting in CI tests.

There are also these:
xen/arch/arm/arm32/mpu/head.S
xen/arch/arm/arm64/mpu/head.S

Yeah, I need to re-scan the tree for all .section directives in *.S files.

This is getting out of hand, I fear, so rather than putting such overrides in perhaps dozens of places I shall try to think of some less intrusive approach.

I further think I need another prereq change. Right now e.g. PPC64's .text.exceptions would collide with the code generated for a hypothetical function named exceptions(). I don't like Linux'es .text..* etc very much, but unless I can think of anything better we may need to follow that model.

Jan

#3 lore raw
In reply to Jason Andryuk
Quoted 7 lines · Jason Andryuk
Quoted 6 lines · Jan Beulich

Leverage the new infrastructure in xen/linkage.h to also switch to per- function sections (when configured), deriving the specific name from the "base" section in use at the time FUNC() is invoked.

I don't seem to have the original patch in my inbox, so I'll just answer here.

About the assembly modifications on the exception entry points:

With split sections the linker is free to reorder all of them as it sees fit, which probably means we want int3 after every jump to prevent straight-line speculation from allocating an XSA number for us. It's possible the linker might inject them, but it might also not. Better to err on the side of caution.

Though more generally, I'd just keep all exception entry points in the same section. They'd never get GC'ed anyway and we're paying an extra branch in the #PF path for no reason.

Cheers,
Alejandro

#4 lore raw
Quoted 9 lines · Alejandro Vallejo
Quoted 7 lines · Jason Andryuk
Quoted 6 lines · Jan Beulich

Leverage the new infrastructure in xen/linkage.h to also switch to per- function sections (when configured), deriving the specific name from the "base" section in use at the time FUNC() is invoked.

Quoted 8 lines

I don't seem to have the original patch in my inbox, so I'll just answer here.

About the assembly modifications on the exception entry points:

With split sections the linker is free to reorder all of them as it sees fit, which probably means we want int3 after every jump to prevent straight-line speculation from allocating an XSA number for us. It's possible the linker might inject them, but it might also not. Better to err on the side of caution.

We're lacking such INT3 elsewhere, hence why this is the topic of separate (existing) work. See how, for example, we're also not using -mharden-sls=all. See e.g. [1] for a very old posting. Even in my outbox I can't find newer postings covering more stuff. Intermediately some of this was posted to security@ only, but there clearly was the plan to have all of this in public.

Quoted 3 lines

Though more generally, I'd just keep all exception entry points in the same section. They'd never get GC'ed anyway and we're paying an extra branch in the #PF path for no reason.

Inserting a branch there was, iirc, asked for by someone independent of this work. But yes, suppressing too fine grained section splits is an option.

Jan

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-11/msg01542.html

#5 lore raw
Quoted 14 lines · Jan Beulich
Quoted 13 lines · Alejandro Vallejo
Quoted 7 lines · Jason Andryuk
Quoted 6 lines · Jan Beulich

Leverage the new infrastructure in xen/linkage.h to also switch to per- function sections (when configured), deriving the specific name from the "base" section in use at the time FUNC() is invoked.

I don't seem to have the original patch in my inbox, so I'll just answer here.

About the assembly modifications on the exception entry points:

Quoted 7 lines
Quoted 4 lines

With split sections the linker is free to reorder all of them as it sees fit, which probably means we want int3 after every jump to prevent straight-line speculation from allocating an XSA number for us. It's possible the linker might inject them, but it might also not. Better to err on the side of caution.

We're lacking such INT3 elsewhere, hence why this is the topic of separate (existing) work.

Maybe so, but split sections changes things qualitatively in that now you don't really know what's after the exception entry point. Previously, if the CPU was to speculate ahead in most exception they'd eventually hit the spec mitigations of entry_DF before being able to reach anywhere truly dangerous. entry_PF's straight line led to the mitigations too. Same with NMI...

Having them all in separate sections shuffled at the linker's will is way too dangerous, IMO. We clearly need individual function markers for livepatching, but section-wise it's fine to put everything that can't possibly be GCd in a single section.

Quoted 1 line

See how, for example, we're also not using -mharden-sls=all.

Hmm. I can see how -mharden-sls=all might impact perf in places we don't want, but surely -mharden-sls=return can only be good?

Quoted 3 lines

See e.g. [1] for a very old posting. Even in my outbox I can't find newer postings covering more stuff. Intermediately some of this was posted to security@ only, but there clearly was the plan to have all of this in public.

Thanks for the context.

That'd address the speculation problem, but we'd still suffer branches in avoidable places.

It'd be nice to have a general means preventing dangerous SLS, but that's largely orthogonal to the new challenges that arise with split sections, I think.

Quoted 7 lines
Quoted 4 lines

Though more generally, I'd just keep all exception entry points in the same section. They'd never get GC'ed anyway and we're paying an extra branch in the #PF path for no reason.

Inserting a branch there was, iirc, asked for by someone independent of this work. But yes, suppressing too fine grained section splits is an option.

My .02 is that splitting that which cannot be GCd serves no purpose and increases the cognitive burden of an already very complex system.

Quoted 4 lines

Jan

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-11/msg01542.html

Cheers,
Alejandro