x86/boot: Avoid relocations in trampoline code to physical addresses
PATCH v2 1/2 x86/boot: Avoid relocations in trampoline code to physical addresses
The trampoline could have "manual" relocation entries (created by assembly macros and some code to use them) and (in case of PE) normal executable relocations.
Remove some normal executable ones. In this way we don't have to worry about applying both correctly (they need proper order which is hard to spot looking at the code).
Specifically in efi_arch_post_exit_boot trampoline is copied after fixing relocations with efi_arch_relocate_image. These time dependencies between different part of code are hard to spot making hard to change code. In this case the copy is done in a state where code should be run at higher locations so it would be better to reduce the code between calling efi_arch_relocate_image and jumping to higher location. Absolute symbols are defined by linker in order to avoid relocations. These symbols use a "_PA" suffix to avoid possible clashes. phys_addr macro is used to make more clear the address we want and making symbol search easier.sym_offs and sym_esi are undefined to avoid the usage of them which would lead to relocations.
--- xen/arch/x86/boot/head.S | 6 ++++-- xen/arch/x86/boot/trampoline.S | 9 ++++++++- xen/arch/x86/boot/wakeup.S | 4 ++-- xen/arch/x86/xen.lds.S | 9 +++++++-- 4 files changed, 21 insertions(+), 7 deletions(-)
---
Changes since v1:
- add subject prefix to commit messages;
- more explanation of the improvement;
- use upper case for suffix to avoid clashes;
- undefine some macros to avoid usage of them;
- use some macros to make code more clear;
- use hidden symbols;
- split change in 2 commits.
xen/arch/x86/boot/head.S +4 -2
@@ -870,8 +870,10 @@ cmdline_parse_early: reloc: .incbin "reloc.bin" +#include "x86_64.S" + + .section .init.text, "ax", @progbits + ENTRY(trampoline_start) #include "trampoline.S" ENTRY(trampoline_end) - -#include "x86_64.S"
xen/arch/x86/boot/trampoline.S +8 -1
@@ -16,6 +16,11 @@ * not guaranteed to persist. */ +/* We don't want relocation in the trampoline. + * Undefine some macros generating relocations. */ +#undef sym_offs +#undef sym_esi + /* NB. bootsym() is only usable in real mode, or via BOOT_PSEUDORM_DS. */ #undef bootsym #define bootsym(s) ((s)-trampoline_start)
@@ -34,6 +39,8 @@ .long 111b - (off) - .; \ .popsection +#define phys_addr(sym) sym ## _PA + /* Start of the permanent trampoline code. */ .code16
@@ -73,7 +80,7 @@ trampoline_protmode_entry: mov %ecx,%cr4 /* Load pagetable base register. */ - mov $sym_offs(idle_pg_table),%eax + mov $phys_addr(idle_pg_table), %eax add bootsym_rel(trampoline_xen_phys_start,4,%eax) mov %eax,%cr3
xen/arch/x86/boot/wakeup.S +2 -2
@@ -99,7 +99,7 @@ wakeup_32: mov $bootsym_rel(wakeup_stack, 4, %esp) # check saved magic again - mov $sym_offs(saved_magic),%eax + mov $phys_addr(saved_magic), %eax add bootsym_rel(trampoline_xen_phys_start, 4, %eax) mov (%eax), %eax cmp $0x9abcdef0, %eax
@@ -112,7 +112,7 @@ wakeup_32: mov %ecx, %cr4 /* Load pagetable base register */ - mov $sym_offs(idle_pg_table),%eax + mov $phys_addr(idle_pg_table), %eax add bootsym_rel(trampoline_xen_phys_start,4,%eax) mov %eax,%cr3
xen/arch/x86/xen.lds.S +7 -2
@@ -22,7 +22,7 @@ ENTRY(efi_start) #define FORMAT "elf64-x86-64" -ENTRY(start_pa) +ENTRY(start_PA) #endif /* EFI */
@@ -71,7 +71,12 @@ SECTIONS __2M_text_start = .; /* Start of 2M superpages, mapped RX. */ #endif - start_pa = ABSOLUTE(start - __XEN_VIRT_START); +#define DEFINE_PA_ADDRESS(sym) \ + HIDDEN(sym ## _PA = ABSOLUTE(sym - __XEN_VIRT_START)) + + DEFINE_PA_ADDRESS(start); + DEFINE_PA_ADDRESS(saved_magic); + DEFINE_PA_ADDRESS(idle_pg_table); . = __XEN_VIRT_START + XEN_IMG_OFFSET; _start = .;
Signature · 1 line
2.46.0
PATCH v2 2/2 x86/boot: Avoid relocations in trampoline code to virtual addresses
Add virt_addr macro to make more clear the address we want and making symbol search easier.
We don't generate direct absolute symbols to virtual addresses to avoid linker warnings.
--- xen/arch/x86/boot/trampoline.S | 3 ++- xen/arch/x86/boot/wakeup.S | 2 +- xen/arch/x86/xen.lds.S | 2 ++ 3 files changed, 5 insertions(+), 2 deletions(-)
xen/arch/x86/boot/trampoline.S +2 -1
@@ -40,6 +40,7 @@ .popsection #define phys_addr(sym) sym ## _PA +#define virt_addr(sym) (sym ## _PA + __XEN_VIRT_START) /* Start of the permanent trampoline code. */
@@ -120,7 +121,7 @@ trampoline_protmode_entry: .code64 start64: /* Jump to high mappings. */ - movabs $__high_start, %rdi + movabs $virt_addr(__high_start), %rdi jmpq *%rdi #include "video.h"
xen/arch/x86/boot/wakeup.S +1 -1
@@ -156,7 +156,7 @@ wakeup_32: .code64 wakeup_64: /* Jump to high mappings and the higher-level wakeup code. */ - movabs $s3_resume, %rbx + movabs $virt_addr(s3_resume), %rbx jmp *%rbx bogus_saved_magic:
xen/arch/x86/xen.lds.S +2 -0
@@ -77,6 +77,8 @@ SECTIONS DEFINE_PA_ADDRESS(start); DEFINE_PA_ADDRESS(saved_magic); DEFINE_PA_ADDRESS(idle_pg_table); + DEFINE_PA_ADDRESS(__high_start); + DEFINE_PA_ADDRESS(s3_resume); . = __XEN_VIRT_START + XEN_IMG_OFFSET; _start = .;
Signature · 1 line
2.46.0
Re: [PATCH v2 2/2] x86/boot: Avoid relocations in trampoline code to virtual addresses
Quoted 2 lines · Frediano Ziglio
Add
virt_addrmacro to make more clear the address we want and making symbol search easier.
This is not good enough a reason to make this change, especially in light of the address space randomization aspect mentioned in reply to patch 1. While there it only _may_ get in the way, here I'm sure it _will_. Which isn't to say we can't go this route, just that it needs proper weighing (and hence a better justification to start from).
Jan
Quoted 6 lines · Frediano Ziglio
The trampoline could have "manual" relocation entries (created by assembly macros and some code to use them) and (in case of PE) normal executable relocations.
Remove some normal executable ones. In this way we don't have to worry about applying both correctly (they need proper order which is hard to spot looking at the code).
I don't theink the order of applying relocations matters - the overall outcome will be the same for any order. What does matter is ...
Quoted 2 lines
Specifically in
efi_arch_post_exit_boottrampoline is copied after fixing relocations withefi_arch_relocate_image.
... whether they're applied by the time certain operations take place.
Quoted 3 lines
These time dependencies
between different part of code are hard to spot making hard to change code.
Relocation and copying sitting literally next to each other makes it not really hard to spot, imo.
Quoted 7 lines
In this case the copy is done in a state where code should be run at higher locations so it would be better to reduce the code between calling
efi_arch_relocate_imageand jumping to higher location. Absolute symbols are defined by linker in order to avoid relocations. These symbols use a "_PA" suffix to avoid possible clashes.phys_addrmacro is used to make more clear the address we want and making symbol search easier.
At the price of introducing more absolute symbols, which are often frowned upon. For example I fear this may (and the 2nd patch will) get in the way of us (finally) randomizing Xen's virtual position at load/boot time. Especially with xen.efi (where we already have the base relocs) this shouldn't be overly difficult to arrange - as long as there are no absolute symbols to take care of (ones used only very early are okay of course).
Quoted 5 lines
xen/arch/x86/boot/head.S +0
-0@@ -870,8 +870,10 @@ cmdline_parse_early: reloc: .incbin "reloc.bin"
xen/arch/x86/boot/head.S cmdline_parse_early: · line 877
Quoted 10 lines
xen/arch/x86/boot/head.S +4
-2+#include "x86_64.S" + + .section .init.text, "ax", @progbits + ENTRY(trampoline_start) #include "trampoline.S" ENTRY(trampoline_end) - -#include "x86_64.S"
I take it that this is superseded by the patch introducing .init.trampoline?
Jan
Quoted 15 lines · Jan Beulich
Quoted 6 lines · Frediano Ziglio
The trampoline could have "manual" relocation entries (created by assembly macros and some code to use them) and (in case of PE) normal executable relocations.
Remove some normal executable ones. In this way we don't have to worry about applying both correctly (they need proper order which is hard to spot looking at the code).I don't theink the order of applying relocations matters - the overall outcome will be the same for any order. What does matter is ...
Quoted 2 lines
Specifically in
efi_arch_post_exit_boottrampoline is copied after fixing relocations withefi_arch_relocate_image.
Quoted 9 lines
... whether they're applied by the time certain operations take place.
Quoted 3 lines
These time dependencies
between different part of code are hard to spot making hard to change code.Relocation and copying sitting literally next to each other makes it not really hard to spot, imo.
I was thinking that there should probably be a single function that does the relocation and also copy the trampoline in the final spot.
Quoted 8 lines
Quoted 7 lines
In this case the copy is done in a state where code should be run at higher locations so it would be better to reduce the code between calling
efi_arch_relocate_imageand jumping to higher location. Absolute symbols are defined by linker in order to avoid relocations. These symbols use a "_PA" suffix to avoid possible clashes.phys_addrmacro is used to make more clear the address we want and making symbol search easier.
Quoted 8 lines
At the price of introducing more absolute symbols, which are often frowned upon. For example I fear this may (and the 2nd patch will) get in the way of us (finally) randomizing Xen's virtual position at load/boot time. Especially with xen.efi (where we already have the base relocs) this shouldn't be overly difficult to arrange - as long as there are no absolute symbols to take care of (ones used only very early are okay of course).
Considering that bootloaders (both GRUB and EFI) uses 1-to-1 mapping or physical addressing and that we wrap our 64 bit ELF in a 32 bit ELF I would assume that we want the randomization done by our code and not by the bootloader. In this case, I would suggest designing the output in order to use position independent code/data and do the randomization/relocation needed. That involves doing something similar to mkelf32 also for EFI output.
This goes quite a lot out of the target of this series, but I agree this series clash a bit with address randomization (going in a different direction).
I suppose I can simply respect the order of calls and drop this series.
Quoted 6 lines
Quoted 6 lines
xen/arch/x86/boot/head.S +0
-0@@ -870,8 +870,10 @@ cmdline_parse_early: reloc: .incbin "reloc.bin"
Quoted 13 lines
xen/arch/x86/boot/head.S cmdline_parse_early: · line 877
Quoted 9 lines
xen/arch/x86/boot/head.S +4
-2+#include "x86_64.S" + + .section .init.text, "ax", @progbits + ENTRY(trampoline_start) #include "trampoline.S" ENTRY(trampoline_end) - -#include "x86_64.S"I take it that this is superseded by the patch introducing .init.trampoline?
No, they happen to clash/conflict and looks similar because they move the trampoline inclusion later but for completely different reasons, one for section continuation, the other for macro preservation.
Quoted 1 line
Jan
Frediano