livepatch: set -f{function,data}-sections compiler option
PATCH 1/1 livepatch: set -f{function,data}-sections compiler option
If livepatching support is enabled build the hypervisor with -f{function,data}-sections compiler options, which is required by the livepatching tools to detect changes and create livepatches.
This shouldn't result in any functional change on the hypervisor binary image, but does however require some changes in the linker script in order to handle that each function and data item will now be placed into its own section in object files. As a result add catch-all for .text, .data and .bss in order to merge each individual item section into the final image.
The main difference will be that .text.startup will end up being part of .text rather than .init, and thus won't be freed. Such section only seems to appear when using -Os, which not the default for debug or production binaries.
The benefit of having CONFIG_LIVEPATCH enable those compiler options is that the livepatch build tools no longer need to fiddle with the build system in order to enable them. Note the current livepatch tools are broken after the recent build changes due to the way they attempt to set -f{function,data}-sections.
--- xen/Makefile | 4 ++++ xen/arch/arm/xen.lds.S | 9 +++++++++ xen/arch/x86/xen.lds.S | 9 +++++++++ xen/common/Kconfig | 4 +++- 4 files changed, 25 insertions(+), 1 deletion(-)
xen/Makefile +4 -0
@@ -269,6 +269,10 @@ else CFLAGS += -fomit-frame-pointer endif +ifeq ($(CONFIG_LIVEPATCH),y) +CFLAGS += -ffunction-sections -fdata-sections +endif + CFLAGS += -nostdinc -fno-builtin -fno-common CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith $(call cc-option-add,CFLAGS,CC,-Wvla)
xen/arch/arm/xen.lds.S +9 -0
@@ -33,6 +33,9 @@ SECTIONS *(.text) *(.text.cold) *(.text.unlikely) +#ifdef CONFIG_LIVEPATCH + *(.text.*) +#endif *(.fixup) *(.gnu.warning) _etext = .; /* End of text section */
@@ -96,6 +99,9 @@ SECTIONS *(.data.rel) *(.data.rel.*) +#ifdef CONFIG_LIVEPATCH + *(.data.*) +#endif CONSTRUCTORS } :text
@@ -208,6 +214,9 @@ SECTIONS . = ALIGN(SMP_CACHE_BYTES); __per_cpu_data_end = .; *(.bss) +#ifdef CONFIG_LIVEPATCH + *(.bss.*) +#endif . = ALIGN(POINTER_ALIGN); __bss_end = .; } :text
xen/arch/x86/xen.lds.S +9 -0
@@ -88,6 +88,9 @@ SECTIONS *(.text.cold) *(.text.unlikely) +#ifdef CONFIG_LIVEPATCH + *(.text.*) +#endif *(.fixup) *(.gnu.warning) _etext = .; /* End of text section */
@@ -292,6 +295,9 @@ SECTIONS *(.data) *(.data.rel) *(.data.rel.*) +#ifdef CONFIG_LIVEPATCH + *(.data.*) +#endif CONSTRUCTORS } PHDR(text)
@@ -308,6 +314,9 @@ SECTIONS . = ALIGN(SMP_CACHE_BYTES); __per_cpu_data_end = .; *(.bss) +#ifdef CONFIG_LIVEPATCH + *(.bss.*) +#endif . = ALIGN(POINTER_ALIGN); __bss_end = .; } PHDR(text)
xen/common/Kconfig +3 -1
@@ -353,7 +353,9 @@ config CRYPTO config LIVEPATCH bool "Live patching support" default X86 - depends on "$(XEN_HAS_BUILD_ID)" = "y" + depends on "$(XEN_HAS_BUILD_ID)" = "y" && \ + $(cc-option,-ffunction-sections) && \ + $(cc-option,-fdata-sections) ---help--- Allows a running Xen hypervisor to be dynamically patched using binary patches without rebooting. This is primarily used to binarily
Signature · 1 line
2.34.1
Quoted 11 lines · Roger Pau Monne
If livepatching support is enabled build the hypervisor with -f{function,data}-sections compiler options, which is required by the livepatching tools to detect changes and create livepatches.
This shouldn't result in any functional change on the hypervisor binary image, but does however require some changes in the linker script in order to handle that each function and data item will now be placed into its own section in object files. As a result add catch-all for .text, .data and .bss in order to merge each individual item section into the final image.
Quoted 4 lines
The main difference will be that .text.startup will end up being part of .text rather than .init, and thus won't be freed. Such section only seems to appear when using -Os, which not the default for debug or production binaries.
That's too optimistic a statement imo. I've observed it appear with -Os, but looking at gcc's gcc/varasm.c:default_function_section() there's ample room for this appearing for other reasons. Also you don't mention .text.exit, which will no longer be discarded.
xen/Makefile else · line 274
Quoted 9 lines
xen/Makefile +3
-0@@ -269,6 +269,10 @@ else CFLAGS += -fomit-frame-pointer endif +ifeq ($(CONFIG_LIVEPATCH),y) +CFLAGS += -ffunction-sections -fdata-sections +endif
Perhaps
CFLAGS-$(CONFIG_LIVEPATCH) += -ffunction-sections -fdata-sections
?
xen/arch/x86/xen.lds.S SECTIONS · line 93
Quoted 9 lines
xen/arch/x86/xen.lds.S +3
-0@@ -88,6 +88,9 @@ SECTIONS *(.text.cold) *(.text.unlikely) +#ifdef CONFIG_LIVEPATCH + *(.text.*) +#endif
This coming after the "cold" and "unlikely" special sections and ahead of .fixup isn't very nice. Also from looking at the linker scripts ld supplies I'm getting the impression that there could/ would then also be e.g. .text.cold.* and .text.unlikely.* which you'd want to separate.
We may want to put the entry point in a special .text.head, put that first, and then follow ld in putting cold/unlikely stuff ahead of main .text.
For the reason given in the description I can see why a conditional is warranted here. But ...
Quoted 11 lines
xen/arch/x86/xen.lds.S +3
-0@@ -292,6 +295,9 @@ SECTIONS *(.data) *(.data.rel) *(.data.rel.*) +#ifdef CONFIG_LIVEPATCH + *(.data.*) +#endif CONSTRUCTORS } PHDR(text)
xen/arch/x86/xen.lds.S SECTIONS · line 319
Quoted 6 lines
xen/arch/x86/xen.lds.S +3
-0@@ -308,6 +314,9 @@ SECTIONS . = ALIGN(SMP_CACHE_BYTES); __per_cpu_data_end = .; *(.bss) +#ifdef CONFIG_LIVEPATCH + *(.bss.*) +#endif
... are these two really in need of being conditional?
xen/common/Kconfig config CRYPTO · line 358
Quoted 10 lines
xen/common/Kconfig +3
-1@@ -353,7 +353,9 @@ config CRYPTO config LIVEPATCH bool "Live patching support" default X86 - depends on "$(XEN_HAS_BUILD_ID)" = "y" + depends on "$(XEN_HAS_BUILD_ID)" = "y" && \ + $(cc-option,-ffunction-sections) && \ + $(cc-option,-fdata-sections)
Is this for certain Clang versions? Gcc has been supporting this in 4.1.x already (didn't check when it was introduced).
Jan
Quoted 34 lines · Jan Beulich
Quoted 11 lines · Roger Pau Monne
If livepatching support is enabled build the hypervisor with -f{function,data}-sections compiler options, which is required by the livepatching tools to detect changes and create livepatches.
This shouldn't result in any functional change on the hypervisor binary image, but does however require some changes in the linker script in order to handle that each function and data item will now be placed into its own section in object files. As a result add catch-all for .text, .data and .bss in order to merge each individual item section into the final image.
Quoted 4 lines
The main difference will be that .text.startup will end up being part of .text rather than .init, and thus won't be freed. Such section only seems to appear when using -Os, which not the default for debug or production binaries.
That's too optimistic a statement imo. I've observed it appear with -Os, but looking at gcc's gcc/varasm.c:default_function_section() there's ample room for this appearing for other reasons. Also you don't mention .text.exit, which will no longer be discarded.
xen/Makefile else · line 274
Quoted 9 lines
xen/Makefile +3
-0@@ -269,6 +269,10 @@ else CFLAGS += -fomit-frame-pointer endif +ifeq ($(CONFIG_LIVEPATCH),y) +CFLAGS += -ffunction-sections -fdata-sections +endifPerhaps
Quoted 3 lines
CFLAGS-$(CONFIG_LIVEPATCH) += -ffunction-sections -fdata-sections
?
Sure.
Quoted 16 lines
xen/arch/x86/xen.lds.S SECTIONS · line 93
Quoted 9 lines
xen/arch/x86/xen.lds.S +3
-0@@ -88,6 +88,9 @@ SECTIONS *(.text.cold) *(.text.unlikely) +#ifdef CONFIG_LIVEPATCH + *(.text.*) +#endifThis coming after the "cold" and "unlikely" special sections and ahead of .fixup isn't very nice. Also from looking at the linker scripts ld supplies I'm getting the impression that there could/ would then also be e.g. .text.cold.* and .text.unlikely.* which you'd want to separate.
Quoted 3 lines
We may want to put the entry point in a special .text.head, put that first, and then follow ld in putting cold/unlikely stuff ahead of main .text.
I can give that a try.
Quoted 13 lines
For the reason given in the description I can see why a conditional is warranted here. But ...
Quoted 10 lines
xen/arch/x86/xen.lds.S +3
-0@@ -292,6 +295,9 @@ SECTIONS *(.data) *(.data.rel) *(.data.rel.*) +#ifdef CONFIG_LIVEPATCH + *(.data.*) +#endif CONSTRUCTORS } PHDR(text)
Quoted 9 lines
xen/arch/x86/xen.lds.S SECTIONS · line 319
Quoted 7 lines
xen/arch/x86/xen.lds.S +3
-0@@ -308,6 +314,9 @@ SECTIONS . = ALIGN(SMP_CACHE_BYTES); __per_cpu_data_end = .; *(.bss) +#ifdef CONFIG_LIVEPATCH + *(.bss.*) +#endif... are these two really in need of being conditional?
Will drop if you agree. I didn't want to risk introducing unwanted changes in the !CONFIG_LIVEPATCH case.
Quoted 5 lines
Quoted 5 lines
xen/common/Kconfig +0
-0@@ -353,7 +353,9 @@ config CRYPTO config LIVEPATCH bool "Live patching support"
Quoted 8 lines
xen/common/Kconfig config CRYPTO · line 358
Quoted 5 lines
xen/common/Kconfig +3
-1default X86 - depends on "$(XEN_HAS_BUILD_ID)" = "y" + depends on "$(XEN_HAS_BUILD_ID)" = "y" && \ + $(cc-option,-ffunction-sections) && \ + $(cc-option,-fdata-sections)Is this for certain Clang versions? Gcc has been supporting this in 4.1.x already (didn't check when it was introduced).
I've checked clang and it seems to be prevent in at least Clang 5, which is likely enough?
I've added the check just to be on the safe side.
Thanks, Roger.
Quoted 22 lines · Roger Pau Monné
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Roger Pau Monne
xen/arch/x86/xen.lds.S +3
-0@@ -292,6 +295,9 @@ SECTIONS *(.data) *(.data.rel) *(.data.rel.*) +#ifdef CONFIG_LIVEPATCH + *(.data.*) +#endif CONSTRUCTORS } PHDR(text)Quoted 9 lines
xen/arch/arm/xen.lds.S SECTIONS · line 319
Quoted 7 lines
xen/arch/arm/xen.lds.S +3
-0@@ -308,6 +314,9 @@ SECTIONS . = ALIGN(SMP_CACHE_BYTES); __per_cpu_data_end = .; *(.bss) +#ifdef CONFIG_LIVEPATCH + *(.bss.*) +#endif... are these two really in need of being conditional?
Quoted 2 lines
Will drop if you agree. I didn't want to risk introducing unwanted changes in the !CONFIG_LIVEPATCH case.
The only "unwanted" change I can imagine here would be that we place a section which the linker would otherwise need to guess how to place, for being "orphan".
Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
xen/common/Kconfig +0
-0@@ -353,7 +353,9 @@ config CRYPTO config LIVEPATCH bool "Live patching support"Quoted 8 lines
xen/common/Kconfig config CRYPTO · line 358
Quoted 5 lines
xen/common/Kconfig +3
-1default X86 - depends on "$(XEN_HAS_BUILD_ID)" = "y" + depends on "$(XEN_HAS_BUILD_ID)" = "y" && \ + $(cc-option,-ffunction-sections) && \ + $(cc-option,-fdata-sections)Is this for certain Clang versions? Gcc has been supporting this in 4.1.x already (didn't check when it was introduced).
Quoted 2 lines
I've checked clang and it seems to be prevent in at least Clang 5, which is likely enough?
Clang5 accepts the options fine here. But that wouldn't be enough, ./README says "Clang/LLVM 3.5 or later".
Quoted 1 line
I've added the check just to be on the safe side.
Well, yes, if you're unsure and the old version can't be checked, then perhaps indeed better to probe.
Jan
Quoted 47 lines · Jan Beulich
Quoted 22 lines · Roger Pau Monné
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Roger Pau Monne
xen/arch/x86/xen.lds.S +3
-0@@ -292,6 +295,9 @@ SECTIONS *(.data) *(.data.rel) *(.data.rel.*) +#ifdef CONFIG_LIVEPATCH + *(.data.*) +#endif CONSTRUCTORS } PHDR(text)Quoted 9 lines
xen/arch/arm/xen.lds.S SECTIONS · line 319
Quoted 7 lines
xen/arch/arm/xen.lds.S +3
-0@@ -308,6 +314,9 @@ SECTIONS . = ALIGN(SMP_CACHE_BYTES); __per_cpu_data_end = .; *(.bss) +#ifdef CONFIG_LIVEPATCH + *(.bss.*) +#endif... are these two really in need of being conditional?
Quoted 2 lines
Will drop if you agree. I didn't want to risk introducing unwanted changes in the !CONFIG_LIVEPATCH case.
The only "unwanted" change I can imagine here would be that we place a section which the linker would otherwise need to guess how to place, for being "orphan".
Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
xen/common/Kconfig +0
-0@@ -353,7 +353,9 @@ config CRYPTO config LIVEPATCH bool "Live patching support"Quoted 8 lines
xen/common/Kconfig config CRYPTO · line 358
Quoted 5 lines
xen/common/Kconfig +3
-1default X86 - depends on "$(XEN_HAS_BUILD_ID)" = "y" + depends on "$(XEN_HAS_BUILD_ID)" = "y" && \ + $(cc-option,-ffunction-sections) && \ + $(cc-option,-fdata-sections)Is this for certain Clang versions? Gcc has been supporting this in 4.1.x already (didn't check when it was introduced).
Quoted 2 lines
I've checked clang and it seems to be prevent in at least Clang 5, which is likely enough?
Quoted 7 lines
Clang5 accepts the options fine here. But that wouldn't be enough, ./README says "Clang/LLVM 3.5 or later".
Quoted 1 line
I've added the check just to be on the safe side.
Well, yes, if you're unsure and the old version can't be checked, then perhaps indeed better to probe.
OK, so I've managed to probe clang 3.5.0 and it does support -f{function,data}-sections so we can drop the check.
Thanks, Roger.
Quoted 5 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne
xen/arch/x86/xen.lds.S +0
-0@@ -88,6 +88,9 @@ SECTIONS
Quoted 11 lines
xen/arch/x86/xen.lds.S SECTIONS · line 93
Quoted 5 lines
xen/arch/x86/xen.lds.S +3
-0*(.text.cold) *(.text.unlikely) +#ifdef CONFIG_LIVEPATCH + *(.text.*) +#endifThis coming after the "cold" and "unlikely" special sections and ahead of .fixup isn't very nice. Also from looking at the linker scripts ld supplies I'm getting the impression that there could/ would then also be e.g. .text.cold.* and .text.unlikely.* which you'd want to separate.
Thinking about it, with -ffunction-sections startup code cannot reasonably be placed in .text.startup, or else there would be confusion with code originating from a function named startup(). But of course .text.startup.* is similarly difficult to arrange for not ending up in the output's main .text, so adding .text.startup.* and .text.exit.* later in the script wouldn't gain us anything.
Jan
xen/common/Kconfig config CRYPTO · line 358
Quoted 12 lines · Roger Pau Monne
xen/common/Kconfig +3
-1@@ -353,7 +353,9 @@ config CRYPTO config LIVEPATCH bool "Live patching support" default X86 - depends on "$(XEN_HAS_BUILD_ID)" = "y" + depends on "$(XEN_HAS_BUILD_ID)" = "y" && \ + $(cc-option,-ffunction-sections) && \ + $(cc-option,-fdata-sections)
cc-option can take multiple, so just $(cc-option,-ffunction-sections -fdata-sections)
However, we in practice want these in combination with $(LD) --gc-sections anyway although that wants to be separately configurable.
Therefore, we probably want something like:
config FUNC_SECTIONS
bool
config LIVEPATCH
selects FUNC_SECTIONS
or so, so in the future we can add "config LD_GC_SECTIONS" which also selects FUNC_SECTIONS.
Thoughts?
~Andrew
Quoted 25 lines · Andrew Cooper
xen/common/Kconfig config CRYPTO · line 358
Quoted 12 lines · Roger Pau Monne
xen/common/Kconfig +3
-1@@ -353,7 +353,9 @@ config CRYPTO config LIVEPATCH bool "Live patching support" default X86 - depends on "$(XEN_HAS_BUILD_ID)" = "y" + depends on "$(XEN_HAS_BUILD_ID)" = "y" && \ + $(cc-option,-ffunction-sections) && \ + $(cc-option,-fdata-sections)cc-option can take multiple, so just $(cc-option,-ffunction-sections -fdata-sections)
However, we in practice want these in combination with $(LD) --gc-sections anyway although that wants to be separately configurable.
Therefore, we probably want something like:
config
FUNC_SECTIONS
bool
Quoted 7 lines
config LIVEPATCH
selectsFUNC_SECTIONSor so, so in the future we can add "config
LD_GC_SECTIONS" which also selectsFUNC_SECTIONS.Thoughts?
Do we want separate options for ffunction-sections and fdata-sections options, or is FUNC_SECTIONS supposed to cover them both?
I assume you are fine with Jan's suggestion to not check for the option presence, since it should be in all supported versions.
Thanks, Roger.