Skip to contentSkip to conversation

Clean up x86_64 boot code

RFC PATCH Superseded David Woodhouse 3 participants 17 messages May 1, 2019 – May 2, 2019

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover Clean up x86_64 boot code

Some cleanups for the boot path, originally inspired by an attempt to avoid scribbling on arbitrarily-chosen low memory.

In the no-real-mode case we don't need to bounce through low memory at all; we can run the 32-bit trampoline in-place in the Xen image.

The variables containing information which is optionally discovered by the real-mode boot code can be put back in place in the Xen image and we can dispense with the bootsym() pointer gymnastics in C code which access them in low memory.

I haven't yet got to reloc(), which I think exists only to ensure that the various breadcrumbs left all over the place by the Multiboot bootloader aren't scribbled on when we copy the 16-bit boot trampoline into low memory. I'd quite like to kill reloc() and pass the original pointer up to 64-bit code to be handled in C.

That would require finding a *safe* location to put the 16-bit boot trampoline though, which doesn't already contain anything that the bootloader created for us.

In fact, isn't there already a chance that head.S will choose a location for the trampoline which is already part of a module or contains one of the Multiboot breadcrumbs?

David Woodhouse (7):
x86/wakeup: Stop using %fs for lidt/lgdt
x86/boot: Remove gratuitous call back into low-memory code x86/boot: Only jump into low trampoline code for real-mode boot x86/boot: Split bootsym() into four types of relocations x86/boot: Rename trampoline_{start,end} to boot_trampoline_{start,end} x86/boot: Copy 16-bit boot variables back up to Xen image x86/boot: Do not use trampoline for no-real-mode boot paths

 xen/arch/x86/acpi/power.c         |   6 +-
 xen/arch/x86/boot/edd.S           |  18 ++--
 xen/arch/x86/boot/head.S          |  89 ++++++++++++------
 xen/arch/x86/boot/mem.S           |  35 +++-----
 xen/arch/x86/boot/trampoline.S    | 145 +++++++++++++++++++++++-------
 xen/arch/x86/boot/video.S         |  36 ++++----
 xen/arch/x86/boot/wakeup.S        |  23 ++---
 xen/arch/x86/cpu/common.c         |   2 +-
 xen/arch/x86/cpu/intel.c          |   2 +-
 xen/arch/x86/efi/efi-boot.h       |  31 +------
 xen/arch/x86/mm.c                 |   2 +-
 xen/arch/x86/platform_hypercall.c |  18 ++--
 xen/arch/x86/setup.c              |  72 ++++++++++++---
 xen/arch/x86/smpboot.c            |   6 +-
 xen/arch/x86/tboot.c              |   6 +-
 xen/arch/x86/x86_64/mm.c          |   2 +-
 xen/arch/x86/xen.lds.S            |  27 ++++--
 xen/include/asm-x86/acpi.h        |   2 +-
 xen/include/asm-x86/config.h      |  10 +--
 xen/include/asm-x86/e820.h        |   5 +-
 xen/include/asm-x86/edd.h         |   1 -
 21 files changed, 339 insertions(+), 199 deletions(-)
Signature · 7 lines
2.17.2


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

RFC PATCH 1/7 x86/wakeup: Stop using %fs for lidt/lgdt

From: David Woodhouse <[email protected]>

The wakeup code is now relocated alongside the trampoline code, so %ds is just fine here.

---
 xen/arch/x86/boot/wakeup.S | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)
xen/arch/x86/boot/wakeup.S +2 -9
index f9632eef95..8c52819171 100644
@@ -40,11 +40,8 @@ ENTRY(wakeup_start)
         movw    %ax, %fs
         movw    $0x0e00 + 'L', %fs:(0x10)
 
-        # boot trampoline is under 1M, and shift its start into
-        # %fs to reference symbols in that area
-        mov     wakesym(trampoline_seg), %fs
-        lidt    %fs:bootsym(idt_48)
-        lgdt    %fs:bootsym(gdt_48)
+        lidt    bootsym(idt_48)
+        lgdt    bootsym(gdt_48)
 
         movw    $1, %ax
         lmsw    %ax             # Turn on CR0.PE 
@@ -102,10 +99,6 @@ GLOBAL(video_mode)
         .long 0
 GLOBAL(video_flags)
         .long 0
-trampoline_seg: .word 0
-        .pushsection .trampoline_seg, "a"
-        .long   trampoline_seg - .
-        .popsection
 
         .code32
 
Signature · 7 lines
2.17.2


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v1 1/7
#2 lore raw

Re: [RFC PATCH 1/7] x86/wakeup: Stop using %fs for lidt/lgdt

Quoted 4 lines · David Woodhouse

From: David Woodhouse <[email protected]>

The wakeup code is now relocated alongside the trampoline code, so %ds is just fine here.

AFACIT, this is as a consequence of c/s bc4a68f21f43c2 "x86: make run-time part of trampoline relocatable" ?

Quoted 1 line

I don't have a sensible way of testing this ATM, but the change looks correct.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw

Re: [RFC PATCH 1/7] x86/wakeup: Stop using %fs for lidt/lgdt

Quoted 5 lines · Andrew Cooper
Quoted 4 lines · David Woodhouse

From: David Woodhouse <[email protected]>

The wakeup code is now relocated alongside the trampoline code, so %ds is just fine here.

Quoted 8 lines

AFACIT, this is as a consequence of c/s bc4a68f21f43c2 "x86: make run-time part of trampoline relocatable" ?

Quoted 1 line

I don't have a sensible way of testing this ATM, but the change looks correct.

I'm afraid testing says no.  S3 works fine without this change, and resets with it.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw

Re: [RFC PATCH 1/7] x86/wakeup: Stop using %fs for lidt/lgdt

Quoted 2 lines · Andrew Cooper

I'm afraid testing says no. S3 works fine without this change, and resets with it.

Thanks for testing. That's obvious in retrospect — although the wakeup code is relocated alongside the trampoline code, it runs in real mode with its own segment, and %ip=0000. So the idt_48 and gdt_48 really do need to use "wakesym". For which they need to be shifted later in trampline.S so that they're actually within the range of that segment.

However, neither staging-4.11 nor master are working here even before the patch, so while I can fix my patch to go back to the existing breakage and not add my own, I haven't managed to test. This is what I get:

[root@localhost ~]# echo mem > /sys/power/state
(XEN) Preparing system for ACPI S3 state.
(XEN) Disabling non-boot CPUs ...
(XEN) Broke affinity for irq 14
(XEN) Removing cpu 1 from runqueue 1
(XEN) No cpus left on runqueue, disabling
(XEN) Entering ACPI S3 state.
QEMU 2.11.1 monitor - type 'help' for more information
(qemu) system_wakeup
(qemu) (XEN) mce_intel.c:780: MCA Capability: firstbank 1, extended MCE MSR 0, SER (XEN) Finishing wakeup from ACPI S3 state.
(XEN) Enabling non-boot CPUs ...
(XEN) Adding cpu 1 to runqueue 0
(XEN) Assertion 'c != old_pool && (c != NULL || old_pool != NULL)' failed at schedule.c:1848
(XEN) ----[ Xen-4.13-unstable x86_64 debug=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e008:[<ffff82d08023c1b0>] schedule_cpu_switch+0x25b/0x318
(XEN) RFLAGS: 0000000000010246 CONTEXT: hypervisor

(XEN) rax: ffff82d0805c7060   rbx: 0000000000000001   rcx: 00000031bbf05100
(XEN) rdx: ffff83023c509840   rsi: ffff83023c509840   rdi: 0000000000000001
(XEN) rbp: ffff8300bfa8fce0   rsp: ffff8300bfa8fc80   r8:  000000000008f000
(XEN) r9:  ffff82d0805c4ec0   r10: 00000000bf9f7600   r11: 0000000979798d09
(XEN) r12: ffff83023c509840   r13: ffff82d080470720   r14: 0000000000000004
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000026e0
(XEN) cr3: 00000000bfa83000   cr2: 0000000000000000
(XEN) fsb: 00007f91b5b6c740   gsb: ffff880075c00000   gss: 0000000000000000

(XEN) ds: 0018 es: 0000 fs: b800 gs: 0000 ss: e010 cs: e008
(XEN) Xen code around <ffff82d08023c1b0> (schedule_cpu_switch+0x25b/0x318):
(XEN) 85 d2 0f 85 f1 fd ff ff <0f> 0b 0f 0b 0f 0b 0f 0b 45 84 c0 75 39 48 8b 46 (XEN) Xen stack trace from rsp=ffff8300bfa8fc80:

(XEN)    0000000000000000 00007cff40570347 ffff82d0803818aa ffff82cfffffb300
(XEN)    ffff82cfffffb280 0000000000000001 0000000000000000 0000000000000001
(XEN)    ffff83023c509840 ffff82d080596128 0000000000000004 0000000000000000
(XEN)    ffff8300bfa8fd10 ffff82d080203f3e 0000000000000286 0000000000000001
(XEN)    0000000000000001 ffff82d080596128 ffff8300bfa8fd40 ffff82d080204108
(XEN)    ffff82d0804859e0 ffff82d080485368 ffff82d080485360 ffff82d080485348
(XEN)    ffff8300bfa8fd90 ffff82d0802251bf 0000000000000000 0000000000000001
(XEN)    ffff8300bfa8fd80 0000000000000000 0000000000000001 ffff82d0803e9e81
(XEN)    0000000000000001 ffff8300bfa8ffff ffff8300bfa8fdc0 ffff82d0802039d0
(XEN)    ffff8300bfa8ffff ffff82d080485340 0000000000000001 0000000000000003
(XEN)    ffff8300bfa8fdf0 ffff82d080203bc9 0000000000000000 0000000000000003
(XEN)    ffff8300bfa8ffff 00000000000026e0 ffff8300bfa8fe40 ffff82d0802dcd03
(XEN)    ffff83023c49b000 0000000000000282 0000000000000000 ffff83023c51ef90
(XEN)    ffff83023c49b000 0000000000000000 0000000000000000 ffff8300bfa8ffff
(XEN)    ffff8300bfa8fe60 ffff82d080208355 ffff83023c49b1d8 ffff82d0805c7170
(XEN)    ffff8300bfa8fe80 ffff82d08023f7b4 ffff8300bfa8fe80 ffff82d0805c7160
(XEN)    ffff8300bfa8feb0 ffff82d08023fadc 0000000000000000 ffff82d0805c7170
(XEN)    ffff82d08059b880 ffff82d0805b20a0 ffff8300bfa8fef0 ffff82d080272397
(XEN)    ffff83023c4fb000 ffff83023c4fb000 ffff83023c49b000 ffff83023c508000
(XEN)    0000000000000000 ffff83023c4b8000 ffff8300bfa8fda8 ffffc900025abd60
(XEN) Xen call trace:
(XEN)    [<ffff82d08023c1b0>] schedule_cpu_switch+0x25b/0x318
(XEN)    [<ffff82d080203f3e>] cpupool.c#cpupool_assign_cpu_locked+0x31/0x126
(XEN)    [<ffff82d080204108>] cpupool.c#cpu_callback+0xd5/0x31d
(XEN)    [<ffff82d0802251bf>] notifier_call_chain+0x64/0x8f
(XEN)    [<ffff82d0802039d0>] cpu_up+0xca/0xec
(XEN)    [<ffff82d080203bc9>] enable_nonboot_cpus+0x87/0xd9
(XEN)    [<ffff82d0802dcd03>] power.c#enter_state_helper+0xd7/0x48f
(XEN)    [<ffff82d080208355>] domain.c#continue_hypercall_tasklet_handler+0x4a/0xab
(XEN)    [<ffff82d08023f7b4>] tasklet.c#do_tasklet_work+0x7a/0xb2
(XEN)    [<ffff82d08023fadc>] do_tasklet+0x58/0x8a
(XEN)    [<ffff82d080272397>] domain.c#idle_loop+0x5e/0xb9

(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion 'c != old_pool && (c != NULL || old_pool != NULL)' failed at schedule.c:1848 (XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw

RFC PATCH 2/7 x86/boot: Remove gratuitous call back into low-memory code

In reply to David Woodhouse

From: David Woodhouse <[email protected]>

We appear to have implemented a memcpy() in the low-memory trampoline which we then call into from __start_xen(), for no adequately defined reason.

Kill it with fire.

---
 xen/arch/x86/boot/mem.S    | 27 +++++----------------------
 xen/arch/x86/setup.c       | 12 ++++++++++++
 xen/include/asm-x86/e820.h |  5 ++---
 3 files changed, 19 insertions(+), 25 deletions(-)
xen/arch/x86/boot/mem.S +5 -22
index c6a9bd4d3b..2d61d28835 100644
@@ -7,7 +7,7 @@ get_memory_map:
 
 .Lmeme820:
         xorl    %ebx, %ebx                      # continuation counter
-        movw    $bootsym(e820map), %di             # point into the whitelist
+        movw    $bootsym(bios_e820map), %di     # point into the whitelist
                                                 # so we can have the bios
                                                 # directly write into it.
 
@@ -22,8 +22,8 @@ get_memory_map:
         cmpl    $SMAP,%eax                      # check the return is `SMAP'
         jne     .Lmem88
 
-        incw    bootsym(e820nr)
-        cmpw    $E820_BIOS_MAX,bootsym(e820nr)  # up to this many entries
+        incw    bootsym(bios_e820nr)
+        cmpw    $E820_BIOS_MAX,bootsym(bios_e820nr) # up to this many entries
         jae     .Lmem88
 
         movw    %di,%ax
@@ -66,27 +66,10 @@ get_memory_map:
 
         ret
 
-/*
- * Copy E820 map obtained from BIOS to a buffer allocated by Xen.
- * Input: %rdi: target address of e820 entry array
- *        %esi: maximum number of entries to copy
- * Output: %eax: number of entries copied
- */
-        .code64
-ENTRY(e820map_copy)
-        mov     %esi, %eax
-        lea     e820map(%rip), %rsi
-        mov     e820nr(%rip), %ecx
-        cmp     %ecx, %eax
-        cmova   %ecx, %eax                      # number of entries to move
-        imul    $5, %eax, %ecx
-        rep movsl                               # do the move
-        ret
-
         .align  4
-e820map:
+GLOBAL(bios_e820map)
         .fill   E820_BIOS_MAX*20,1,0
-e820nr:
+GLOBAL(bios_e820nr)
         .long   0
 GLOBAL(lowmem_kb)
         .long   0
xen/arch/x86/setup.c +12 -0
index a353d76f9a..5fa7d3b79c 100644
@@ -664,6 +664,18 @@ static char * __init cmdline_cook(char *p, const char *loader_name)
     return p;
 }
 
+static int copy_bios_e820(struct e820entry *map, unsigned int limit)
+{
+    unsigned int n = bootsym(bios_e820nr);
+    if (n > limit)
+        n = limit;
+
+    if (n)
+        memcpy(map, bootsym(bios_e820map), sizeof(*map) * n);
+
+    return n;
+}
+
 void __init noreturn __start_xen(unsigned long mbi_p)
 {
     char *memmap_type = NULL;
xen/include/asm-x86/e820.h +2 -3
index ee317b17aa..52916fb75d 100644
@@ -37,8 +37,7 @@ extern struct e820map e820_raw;
 
 /* These symbols live in the boot trampoline. */
 extern unsigned int lowmem_kb, highmem_kb;
-unsigned int e820map_copy(struct e820entry *map, unsigned int limit);
-
-#define copy_bios_e820 bootsym(e820map_copy)
+extern struct e820map bios_e820map[];
+extern unsigned int bios_e820nr;
 
 #endif /*__E820_HEADER*/
Signature · 7 lines
2.17.2


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v1 2/7
#6 lore raw

Re: [RFC PATCH 2/7] x86/boot: Remove gratuitous call back into low-memory code

Quoted 7 lines · David Woodhouse

From: David Woodhouse <[email protected]>

We appear to have implemented a memcpy() in the low-memory trampoline which we then call into from __start_xen(), for no adequately defined reason.

Kill it with fire.

Absolutely.

Quoted 1 line

With the following fixup folded:

xen/arch/x86/setup.c +3 -5
index 213a1bd..e607a85 100644
@@ -673,13 +673,11 @@ static char * __init cmdline_cook(char *p, const char *loader_name)
     return p;
 }
 
-static int copy_bios_e820(struct e820entry *map, unsigned int limit)
+static unsigned int copy_bios_e820(struct e820entry *map, unsigned int limit)
 {
-    unsigned int n = bootsym(bios_e820nr);
-    if (n > limit)
-        n = limit;
+    unsigned int n = min(bootsym(bios_e820nr), limit);
 
-    if (n)
+    if ( n )
         memcpy(map, bootsym(bios_e820map), sizeof(*map) * n);
 
     return n;

Interestingly, the version with min() results in surprisingly better code:

add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-21 (-21)
Function old new delta
__start_xen 13807 13786 -21
Total: Before=3363289, After=3363268, chg -0.00%

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#7 lore raw

Re: [RFC PATCH 2/7] x86/boot: Remove gratuitous call back into low-memory code

In reply to David Woodhouse
Quoted 4 lines
Quoted 1 line
Quoted 1 line · On 01.05.19 at 13:17, <[email protected]> wrote:

We appear to have implemented a memcpy() in the low-memory trampoline which we then call into from __start_xen(), for no adequately defined reason.

May I suggest that in cases like this you look at the commit introducing the function? It supplies a reason:

"Add a new raw e820 table for common purpose and copy the BIOS buffer
to it. Doing the copying in assembly avoids the need to export the symbols for the BIOS E820 buffer and number of entries."

I have to admit that I see value in this, as it avoids introduction of wrong accesses to these variables. Therefore I'd like to ask for better justification of the change.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#8 lore raw

Re: [RFC PATCH 2/7] x86/boot: Remove gratuitous call back into low-memory code

Quoted 10 lines · Jan Beulich
Quoted 4 lines
Quoted 1 line
Quoted 1 line · On 01.05.19 at 13:17, <[email protected]> wrote:

We appear to have implemented a memcpy() in the low-memory trampoline which we then call into from __start_xen(), for no adequately defined reason.

May I suggest that in cases like this you look at the commit introducing the function? It supplies a reason:

"Add a new raw e820 table for common purpose and copy the BIOS buffer
to it. Doing the copying in assembly avoids the need to export the symbols for the BIOS E820 buffer and number of entries."

I completely agree with David here.  That description is completely insufficient for justifying why the logic was done that way in the end, and I would not have accepted the patch in that form at all.

To be clear.  I understand (and agree) why having our main copy of the e820 table in the trampoline is a bad thing, and moving the main copy out of the trampoline is fine.

What isn't fine is why, in the adjusted world, we call back into what appears to be the trampoline, but isn't, to get access to data which the calling code can already access.  In particular...

Quoted 2 lines

I have to admit that I see value in this, as it avoids introduction of wrong accesses to these variables.

...this reasoning is bogus.  We're either accessing the data itself, or the memcpy function, but there is no possible way to programatically avoid "wrong" access into the trampoline, because we're still accessing it.

Therefore, I don't understand what possible benefit not exporting the data has in the first place, and why complicating it with a call to a function (which isn't actually executing where it appears to live), is considered a benefit.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#9 lore raw

Re: [RFC PATCH 2/7] x86/boot: Remove gratuitous call back into low-memory code

Quoted 33 lines · Jan Beulich
Quoted 10 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line · On 01.05.19 at 13:17, <[email protected]> wrote:

We appear to have implemented a memcpy() in the low-memory trampoline which we then call into from __start_xen(), for no adequately defined reason.

May I suggest that in cases like this you look at the commit introducing the function? It supplies a reason:

"Add a new raw e820 table for common purpose and copy the BIOS buffer
to it. Doing the copying in assembly avoids the need to export the symbols for the BIOS E820 buffer and number of entries."

I completely agree with David here.  That description is completely insufficient for justifying why the logic was done that way in the end, and I would not have accepted the patch in that form at all.

To be clear.  I understand (and agree) why having our main copy of the e820 table in the trampoline is a bad thing, and moving the main copy out of the trampoline is fine.

What isn't fine is why, in the adjusted world, we call back into what appears to be the trampoline, but isn't, to get access to data which the calling code can already access.  In particular...

Quoted 2 lines

I have to admit that I see value in this, as it avoids introduction of wrong accesses to these variables.

...this reasoning is bogus.  We're either accessing the data itself, or the memcpy function, but there is no possible way to programatically avoid "wrong" access into the trampoline, because we're still accessing it.

Quoted 4 lines

Therefore, I don't understand what possible benefit not exporting the data has in the first place, and why complicating it with a call to a function (which isn't actually executing where it appears to live), is considered a benefit.

Note that after bringing these two gratuitously differently handled symbols in line with everything else in the boot trampoline, a later patch in the series puts all this stuff into its own data section which is copied back up to the Xen image at the end of the real mode phase of boot, and all the pointer gymnastics for all of them go away completely

--
dwmw2

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#10 lore raw

Re: [RFC PATCH 2/7] x86/boot: Remove gratuitous call back into low-memory code

In reply to Andrew Cooper
Quoted 28 lines
Quoted 1 line
Quoted 1 line · On 02.05.19 at 11:23, <[email protected]> wrote:
Quoted 10 lines · Jan Beulich
Quoted 4 lines
Quoted 1 line
Quoted 1 line · On 01.05.19 at 13:17, <[email protected]> wrote:

We appear to have implemented a memcpy() in the low-memory trampoline which we then call into from __start_xen(), for no adequately defined reason.

May I suggest that in cases like this you look at the commit introducing the function? It supplies a reason:

"Add a new raw e820 table for common purpose and copy the BIOS buffer
to it. Doing the copying in assembly avoids the need to export the symbols for the BIOS E820 buffer and number of entries."

I completely agree with David here. That description is completely insufficient for justifying why the logic was done that way in the end, and I would not have accepted the patch in that form at all.

To be clear. I understand (and agree) why having our main copy of the e820 table in the trampoline is a bad thing, and moving the main copy out of the trampoline is fine.

What isn't fine is why, in the adjusted world, we call back into what appears to be the trampoline, but isn't, to get access to data which the calling code can already access. In particular...

Quoted 2 lines

I have to admit that I see value in this, as it avoids introduction of wrong accesses to these variables.

Quoted 8 lines

...this reasoning is bogus. We're either accessing the data itself, or the memcpy function, but there is no possible way to programatically avoid "wrong" access into the trampoline, because we're still accessing it.

Therefore, I don't understand what possible benefit not exporting the data has in the first place, and why complicating it with a call to a function (which isn't actually executing where it appears to live), is considered a benefit.

Without having gone back to the old thread, I think part of the motivation to avoid such direct data accesses was that an initial version of the patch actually broke things by introducing such accesses.

Apart from that, despite not being a heavy C++ user, I very much appreciate the language's fundamental desire to avoid direct accesses to data elements, providing functions (methods) to do so instead.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#11 lore raw

Re: [RFC PATCH 2/7] x86/boot: Remove gratuitous call back into low-memory code

In reply to Andrew Cooper
Quoted 3 lines · Andrew Cooper

...this reasoning is bogus. We're either accessing the data itself, or the memcpy function, but there is no possible way to programatically avoid "wrong" access into the trampoline, because we're still accessing it.

Just to be clear, now I'm back at a real computer: yes there is a way to avoid "wrong" access into the trampoline from the 64-bit C code. It's the 6th patch in my series, for which this one is a necessary preliminary cleanup.

http://git.infradead.org/users/dwmw2/xen.git/commitdiff/f54044fb3ad5d

It rips out all those horrid bootsym() pointer gymnastics in the 64-bit code, except for the limited cases where we really do need to put things in place for the permanently AP/wakeup trampolines.

It puts all these boot-discovered variables into their own section, then copies them back into the Xen image when the 16-bit boot code has finished running, to be accessed from there ever after.

Which also paves the way for the no-real-mode boot not actually putting *anything* into low memory during boot at all.

This allows us to clean up the EFI code to stop having to do that, too.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#12 lore raw

RFC PATCH 3/7 x86/boot: Only jump into low trampoline code for real-mode boot

In reply to David Woodhouse

From: David Woodhouse <[email protected]>

If the no-real-mode flag is set, don't go there at all. This is a prelude to not even putting it there in the first place.

---
 xen/arch/x86/boot/head.S       | 10 ++++++++++
 xen/arch/x86/boot/trampoline.S |  4 ----
 2 files changed, 10 insertions(+), 4 deletions(-)
xen/arch/x86/boot/head.S +10 -0
index db19ac6fd8..7c30de3671 100644
@@ -734,7 +734,17 @@ trampoline_setup:
         /* Switch to low-memory stack which lives at the end of trampoline region. */
         mov     sym_fs(trampoline_phys),%edi
         lea     TRAMPOLINE_SPACE+TRAMPOLINE_STACK_SPACE(%edi),%esp
+        cmpb    $0, sym_fs(skip_realmode)
+        jz      1f
+        /* If no-real-mode, jump straight to trampoline_protmode_entry */
+        lea     trampoline_protmode_entry-trampoline_start(%edi),%eax
+        /* EBX == 0 indicates we are the BP (Boot Processor). */
+        xor     %ebx,%ebx
+        jmp     2f
+1:
+        /* Go via 16-bit code in trampoline_boot_cpu_entry */
         lea     trampoline_boot_cpu_entry-trampoline_start(%edi),%eax
+2:
         pushl   $BOOT_CS32
         push    %eax
 
xen/arch/x86/boot/trampoline.S +0 -4
index 5588c7986a..df0ffd5013 100644
@@ -187,9 +187,6 @@ start64:
 
         .code32
 trampoline_boot_cpu_entry:
-        cmpb    $0,bootsym_rel(skip_realmode,5)
-        jnz     .Lskip_realmode
-
         /* Load pseudo-real-mode segments. */
         mov     $BOOT_PSEUDORM_DS,%eax
         mov     %eax,%ds
@@ -269,7 +266,6 @@ trampoline_boot_cpu_entry:
         mov     %eax,%gs
         mov     %eax,%ss
 
-.Lskip_realmode:
         /* EBX == 0 indicates we are the BP (Boot Processor). */
         xor     %ebx,%ebx
 
Signature · 7 lines
2.17.2


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#13 lore raw

RFC PATCH 4/7 x86/boot: Split bootsym() into four types of relocations

In reply to David Woodhouse

From: David Woodhouse <[email protected]>

As a first step toward using the low-memory trampoline only when necessary for a legacy boot without no-real-mode, clean up the relocations into three separate groups.

• bootsym() is now used only at boot time when no-real-mode isn't set.

• bootdatasym() is for variables containing information discovered by the 16-bit boot code. This is currently accessed directly in place in low memory by Xen at runtime, but will be copied back to its location in high memory to avoid the pointer gymnastics (and because a subsequent patch will stop copying the 16-bit boot code into low memory at all when it isn't being used).

• trampsym() is for the permanent 16-bit trampoline used for AP startup and for wake from sleep. This is not used at boot, and can be copied into (properly allocated) low memory once the system is running.

• tramp32sym() is used both at boot and for AP startup/wakeup. During boot it can be used in-place, running from the physical address of the Xen image. For AP startup it can't, because at that point there isn't a full 1:1 mapping of all memory; only the low trampoline page is mapped.

No (intentional) functional change yet; just a "cleanup" to allow the various parts to be treated separately in subsequent patches.

---
 xen/arch/x86/boot/edd.S        | 16 +++----
 xen/arch/x86/boot/head.S       | 22 +++++++--
 xen/arch/x86/boot/mem.S        | 12 ++---
 xen/arch/x86/boot/trampoline.S | 86 ++++++++++++++++++++++++++--------
 xen/arch/x86/boot/video.S      |  6 +--
 xen/arch/x86/boot/wakeup.S     | 16 +++----
 xen/arch/x86/efi/efi-boot.h    |  8 ++--
 xen/arch/x86/xen.lds.S         | 15 ++++--
 8 files changed, 125 insertions(+), 56 deletions(-)
xen/arch/x86/boot/edd.S +8 -8
index 3df712bce1..434bbbd960 100644
@@ -41,7 +41,7 @@ get_edd:
 # This code is sensitive to the size of the structs in edd.h
 edd_start:
         /* ds:si points at fn48 results. Fn41 results go immediately before. */
-        movw    $bootsym(boot_edd_info)+EDDEXTSIZE, %si
+        movw    $bootdatasym(boot_edd_info)+EDDEXTSIZE, %si
         movb    $0x80, %dl                      # BIOS device 0x80
 
 edd_check_ext:
@@ -56,7 +56,7 @@ edd_check_ext:
         movb    %dl, %ds:-8(%si)                # store device number
         movb    %ah, %ds:-7(%si)                # store version
         movw    %cx, %ds:-6(%si)                # store extensions
-        incb    bootsym(boot_edd_info_nr)       # note that we stored something
+        incb    bootdatasym(boot_edd_info_nr)   # note that we stored something
 
 edd_get_device_params:
         movw    $EDDPARMSIZE, %ds:(%si)         # put size
@@ -97,7 +97,7 @@ edd_legacy_done:
 edd_next:
         incb    %dl                             # increment to next device
         jz      edd_done
-        cmpb    $EDD_INFO_MAX,bootsym(boot_edd_info_nr)
+        cmpb    $EDD_INFO_MAX,bootdatasym(boot_edd_info_nr)
         jb      edd_check_ext
 
 edd_done:
@@ -108,11 +108,11 @@ edd_done:
 .Ledd_mbr_sig_start:
         pushw   %es
         movb    $0x80, %dl                      # from device 80
-        movw    $bootsym(boot_mbr_signature), %bx # store buffer ptr in bx
+        movw    $bootdatasym(boot_mbr_signature), %bx # store buffer ptr in bx
 .Ledd_mbr_sig_read:
         pushw   %bx
-        movw    $bootsym(boot_edd_info), %bx
-        movzbw  bootsym(boot_edd_info_nr), %cx
+        movw    $bootdatasym(boot_edd_info), %bx
+        movzbw  bootdatasym(boot_edd_info_nr), %cx
         jcxz    .Ledd_mbr_sig_default
 .Ledd_mbr_sig_find_info:
         cmpb    %dl, (%bx)
@@ -151,12 +151,12 @@ edd_done:
         jne     .Ledd_mbr_sig_next
         movb    %dl, (%bx)                      # store BIOS drive number
         movl    %ecx, 4(%bx)                    # store signature from MBR
-        incb    bootsym(boot_mbr_signature_nr)  # note that we stored something
+        incb    bootdatasym(boot_mbr_signature_nr) # note that we stored something
         addw    $8, %bx                         # increment sig buffer ptr
 .Ledd_mbr_sig_next:
         incb    %dl                             # increment to next device
         jz      .Ledd_mbr_sig_done
-        cmpb    $EDD_MBR_SIG_MAX, bootsym(boot_mbr_signature_nr)
+        cmpb    $EDD_MBR_SIG_MAX, bootdatasym(boot_mbr_signature_nr)
         jb      .Ledd_mbr_sig_read
 .Ledd_mbr_sig_done:
         popw    %es
xen/arch/x86/boot/head.S +19 -3
index 7c30de3671..5b4f211a9b 100644
@@ -706,14 +706,30 @@ trampoline_setup:
         cmp     $sym_offs(__trampoline_rel_stop),%edi
         jb      1b
 
-        /* Patch in the trampoline segment. */
+        mov     $sym_offs(__trampoline32_rel_start),%edi
+1:
+        mov     %fs:(%edi),%eax
+        add     %edx,%fs:(%edi,%eax)
+        add     $4,%edi
+        cmp     $sym_offs(__trampoline32_rel_stop),%edi
+        jb      1b
+
+        mov     $sym_offs(__bootsym_rel_start),%edi
+1:
+        mov     %fs:(%edi),%eax
+        add     %edx,%fs:(%edi,%eax)
+        add     $4,%edi
+        cmp     $sym_offs(__bootsym_rel_stop),%edi
+        jb      1b
+
+        /* Patch in the boot trampoline segment. */
         shr     $4,%edx
-        mov     $sym_offs(__trampoline_seg_start),%edi
+        mov     $sym_offs(__bootsym_seg_start),%edi
 1:
         mov     %fs:(%edi),%eax
         mov     %dx,%fs:(%edi,%eax)
         add     $4,%edi
-        cmp     $sym_offs(__trampoline_seg_stop),%edi
+        cmp     $sym_offs(__bootsym_seg_stop),%edi
         jb      1b
 
         /* Do not parse command line on EFI platform here. */
xen/arch/x86/boot/mem.S +6 -6
index 2d61d28835..aa39608442 100644
@@ -7,7 +7,7 @@ get_memory_map:
 
 .Lmeme820:
         xorl    %ebx, %ebx                      # continuation counter
-        movw    $bootsym(bios_e820map), %di     # point into the whitelist
+        movw    $bootdatasym(bios_e820map), %di # point into the whitelist
                                                 # so we can have the bios
                                                 # directly write into it.
 
@@ -22,8 +22,8 @@ get_memory_map:
         cmpl    $SMAP,%eax                      # check the return is `SMAP'
         jne     .Lmem88
 
-        incw    bootsym(bios_e820nr)
-        cmpw    $E820_BIOS_MAX,bootsym(bios_e820nr) # up to this many entries
+        incw    bootdatasym(bios_e820nr)
+        cmpw    $E820_BIOS_MAX,bootdatasym(bios_e820nr) # up to this many entries
         jae     .Lmem88
 
         movw    %di,%ax
@@ -35,7 +35,7 @@ get_memory_map:
 .Lmem88:
         movb    $0x88, %ah
         int     $0x15
-        movw    %ax,bootsym(highmem_kb)
+        movw    %ax,bootdatasym(highmem_kb)
 
 .Lmeme801:
         stc                                     # fix to work around buggy
@@ -58,11 +58,11 @@ get_memory_map:
         shll    $6,%edx                         # and go from 64k to 1k chunks
         movzwl  %cx, %ecx
         addl    %ecx, %edx                      # add in lower memory
-        movl    %edx,bootsym(highmem_kb)        # store extended memory size
+        movl    %edx,bootdatasym(highmem_kb)    # store extended memory size
 
 .Lint12:
         int     $0x12
-        movw    %ax,bootsym(lowmem_kb)
+        movw    %ax,bootdatasym(lowmem_kb)
 
         ret
 
xen/arch/x86/boot/trampoline.S +66 -20

168 lines not on this page · raw message

xen/arch/x86/boot/video.S +3 -3
index 335a51c9b5..03907e9e9a 100644
@@ -45,7 +45,7 @@
 #define PARAM_VESAPM_SEG        0x24
 #define PARAM_VESAPM_OFF        0x26
 #define PARAM_VESA_ATTRIB       0x28
-#define _param(param) bootsym(boot_vid_info)+(param)
+#define _param(param) bootdatasym(boot_vid_info)+(param)
 
 video:  xorw    %ax, %ax
         movw    %ax, %gs        # GS is zero
@@ -917,7 +917,7 @@ store_edid:
         cmpw    $0x004f, %ax            # Call failed?
         jne     .Lno_edid
 
-        movw    %bx, bootsym(boot_edid_caps)
+        movw    %bx, bootdatasym(boot_edid_caps)
 
         cmpb    $2, bootsym(opt_edid)   # EDID forced on cmdline (edid=force)?
         je      .Lforce_edid
@@ -933,7 +933,7 @@ store_edid:
         movw    $0x01, %bx
         movw    $0x00, %cx
         movw    $0x00, %dx
-        movw    $bootsym(boot_edid_info), %di
+        movw    $bootdatasym(boot_edid_info), %di
         int     $0x10
 
 .Lno_edid:
xen/arch/x86/boot/wakeup.S +8 -8
index 8c52819171..1c58e2affb 100644
@@ -40,12 +40,12 @@ ENTRY(wakeup_start)
         movw    %ax, %fs
         movw    $0x0e00 + 'L', %fs:(0x10)
 
-        lidt    bootsym(idt_48)
-        lgdt    bootsym(gdt_48)
+        lidt    trampsym(idt_48)
+        lgdt    trampsym(gdt_48)
 
         movw    $1, %ax
         lmsw    %ax             # Turn on CR0.PE 
-        ljmpl   $BOOT_CS32, $bootsym_rel(wakeup_32, 6)
+        ljmpl   $BOOT_CS32, $trampsym_rel(wakeup_32, 6)
 
 /* This code uses an extended set of video mode numbers. These include:
  * Aliases for standard modes
@@ -110,11 +110,11 @@ wakeup_32:
         mov     $BOOT_DS, %eax
         mov     %eax, %ds
         mov     %eax, %ss
-        mov     $bootsym_rel(wakeup_stack, 4, %esp)
+        mov     $trampsym_rel(wakeup_stack, 4, %esp)
 
         # check saved magic again
         mov     $sym_offs(saved_magic),%eax
-        add     bootsym_rel(trampoline_xen_phys_start, 4, %eax)
+        add     trampsym_rel(trampoline_xen_phys_start, 4, %eax)
         mov     (%eax), %eax
         cmp     $0x9abcdef0, %eax
         jne     bogus_saved_magic
@@ -127,12 +127,12 @@ wakeup_32:
 
         /* Load pagetable base register */
         mov     $sym_offs(idle_pg_table),%eax
-        add     bootsym_rel(trampoline_xen_phys_start,4,%eax)
+        add     trampsym_rel(trampoline_xen_phys_start,4,%eax)
         mov     %eax,%cr3
 
         /* Will cpuid feature change after resume? */
         /* Set up EFER (Extended Feature Enable Register). */
-        mov     bootsym_rel(cpuid_ext_features,4,%edi)
+        mov     trampsym_rel(cpuid_ext_features,4,%edi)
         test    $0x20100800,%edi /* SYSCALL/SYSRET, No Execute, Long Mode? */
         jz      .Lskip_eferw
         movl    $MSR_EFER,%ecx
@@ -154,7 +154,7 @@ wakeup_32:
 1:
 
         /* Now in compatibility mode. Long-jump to 64-bit mode */
-        ljmp    $BOOT_CS64, $bootsym_rel(wakeup_64,6)
+        ljmp    $BOOT_CS64, $trampsym_rel(wakeup_64,6)
 
         .code64
 wakeup_64:
xen/arch/x86/efi/efi-boot.h +4 -4
index 5789d2cb70..abc7d3e3b7 100644
@@ -99,7 +99,7 @@ static void __init efi_arch_relocate_image(unsigned long delta)
 }
 
 extern const s32 __trampoline_rel_start[], __trampoline_rel_stop[];
-extern const s32 __trampoline_seg_start[], __trampoline_seg_stop[];
+extern const s32 __trampoline32_rel_start[], __trampoline32_rel_stop[];
 
 static void __init relocate_trampoline(unsigned long phys)
 {
@@ -115,10 +115,10 @@ static void __init relocate_trampoline(unsigned long phys)
           trampoline_ptr < __trampoline_rel_stop;
           ++trampoline_ptr )
         *(u32 *)(*trampoline_ptr + (long)trampoline_ptr) += phys;
-    for ( trampoline_ptr = __trampoline_seg_start;
-          trampoline_ptr < __trampoline_seg_stop;
+    for ( trampoline_ptr = __trampoline32_rel_start;
+          trampoline_ptr < __trampoline32_rel_stop;
           ++trampoline_ptr )
-        *(u16 *)(*trampoline_ptr + (long)trampoline_ptr) = phys >> 4;
+        *(u32 *)(*trampoline_ptr + (long)trampoline_ptr) += phys;
 }
 
 static void __init place_string(u32 *addr, const char *s)
xen/arch/x86/xen.lds.S +11 -4
index a76e2ec8ef..54ac0c0993 100644
@@ -221,11 +221,18 @@ SECTIONS
        *(.init.data.rel.*)
        . = ALIGN(4);
        __trampoline_rel_start = .;
-       *(.trampoline_rel)
+       *(.trampsym_rel)
        __trampoline_rel_stop = .;
-       __trampoline_seg_start = .;
-       *(.trampoline_seg)
-       __trampoline_seg_stop = .;
+       __trampoline32_rel_start = .;
+       *(.tramp32sym_rel)
+       __trampoline32_rel_stop = .;
+       __bootsym_rel_start = .;
+       *(.bootsym_rel)
+       *(.bootdatasym_rel)
+       __bootsym_rel_stop = .;
+       __bootsym_seg_start = .;
+       *(.bootsym_seg)
+       __bootsym_seg_stop = .;
        /*
         * struct alt_inst entries. From the header (alternative.h):
         * "Alternative instructions for different CPU types or capabilities"
Signature · 7 lines
2.17.2


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#14 lore raw

RFC PATCH 5/7 x86/boot: Rename trampoline_{start, end} to boot_trampoline_{start, end}

In reply to David Woodhouse

From: David Woodhouse <[email protected]>

In preparation for splitting the boot and permanent trampolines from each other. Some of these will change back, but most are boot so do the plain search/replace that way first, then a subsequent patch will extract the permanent trampoline code.

---
 xen/arch/x86/boot/head.S       | 12 ++++++------
 xen/arch/x86/boot/trampoline.S | 10 +++++-----
 xen/arch/x86/boot/video.S      |  4 ++--
 xen/arch/x86/efi/efi-boot.h    |  4 ++--
 xen/arch/x86/mm.c              |  2 +-
 xen/arch/x86/setup.c           |  4 ++--
 xen/arch/x86/tboot.c           |  6 +++---
 xen/arch/x86/x86_64/mm.c       |  2 +-
 xen/arch/x86/xen.lds.S         |  6 +++---
 xen/include/asm-x86/config.h   |  6 +++---
 10 files changed, 28 insertions(+), 28 deletions(-)
xen/arch/x86/boot/head.S +6 -6
index 5b4f211a9b..82342769c7 100644
@@ -753,20 +753,20 @@ trampoline_setup:
         cmpb    $0, sym_fs(skip_realmode)
         jz      1f
         /* If no-real-mode, jump straight to trampoline_protmode_entry */
-        lea     trampoline_protmode_entry-trampoline_start(%edi),%eax
+        lea     trampoline_protmode_entry-boot_trampoline_start(%edi),%eax
         /* EBX == 0 indicates we are the BP (Boot Processor). */
         xor     %ebx,%ebx
         jmp     2f
 1:
         /* Go via 16-bit code in trampoline_boot_cpu_entry */
-        lea     trampoline_boot_cpu_entry-trampoline_start(%edi),%eax
+        lea     trampoline_boot_cpu_entry-boot_trampoline_start(%edi),%eax
 2:
         pushl   $BOOT_CS32
         push    %eax
 
         /* Copy bootstrap trampoline to low memory, below 1MB. */
-        mov     $sym_offs(trampoline_start),%esi
-        mov     $((trampoline_end - trampoline_start) / 4),%ecx
+        mov     $sym_offs(boot_trampoline_start),%esi
+        mov     $((boot_trampoline_end - boot_trampoline_start) / 4),%ecx
         rep movsl %fs:(%esi),%es:(%edi)
 
         /* Jump into the relocated trampoline. */
@@ -778,8 +778,8 @@ cmdline_parse_early:
 reloc:
 #include "reloc.S"
 
-ENTRY(trampoline_start)
+ENTRY(boot_trampoline_start)
 #include "trampoline.S"
-ENTRY(trampoline_end)
+ENTRY(boot_trampoline_end)
 
 #include "x86_64.S"
xen/arch/x86/boot/trampoline.S +5 -5
index e77b4bea36..0f4a740fcb 100644
@@ -31,7 +31,7 @@
  *                to be used for AP startup.
  */
 #undef bootsym
-#define bootsym(s) ((s)-trampoline_start)
+#define bootsym(s) ((s)-boot_trampoline_start)
 
 #define bootsym_rel(sym, off, opnd...)     \
         bootsym(sym),##opnd;               \
@@ -47,7 +47,7 @@
         .long 111b - (off) - .;            \
         .popsection
 
-#define bootdatasym(s) ((s)-trampoline_start)
+#define bootdatasym(s) ((s)-boot_trampoline_start)
 #define bootdatasym_rel(sym, off, opnd...) \
         bootdatasym(sym),##opnd;           \
 111:;                                      \
@@ -56,7 +56,7 @@
         .popsection
 
 #undef trampsym
-#define trampsym(s) ((s)-trampoline_start)
+#define trampsym(s) ((s)-boot_trampoline_start)
 
 #define trampsym_rel(sym, off, opnd...)    \
         trampsym(sym),##opnd;              \
@@ -66,7 +66,7 @@
         .popsection
 
 #undef tramp32sym
-#define tramp32sym(s) ((s)-trampoline_start)
+#define tramp32sym(s) ((s)-boot_trampoline_start)
 
 #define tramp32sym_rel(sym, off, opnd...)  \
         tramp32sym(sym),##opnd;            \
@@ -226,7 +226,7 @@ start64:
 
 /* The first page of trampoline is permanent, the rest boot-time only. */
 /* Reuse the boot trampoline on the 1st trampoline page as stack for wakeup. */
-        .equ    wakeup_stack, trampoline_start + PAGE_SIZE
+        .equ    wakeup_stack, boot_trampoline_start + PAGE_SIZE
         .global wakeup_stack
 
 /* From here on early boot only. */
xen/arch/x86/boot/video.S +2 -2
index 03907e9e9a..5087c6a4d5 100644
@@ -15,8 +15,8 @@
 
 #include "video.h"
 
-/* Scratch space layout: trampoline_end to trampoline_end+0x1000. */
-#define modelist       bootsym(trampoline_end)   /* 2kB (256 entries) */
+/* Scratch space layout: boot_trampoline_end to boot_trampoline_end+0x1000. */
+#define modelist       bootsym(boot_trampoline_end)   /* 2kB (256 entries) */
 #define vesa_glob_info (modelist + 0x800)        /* 1kB */
 #define vesa_mode_info (vesa_glob_info + 0x400)  /* 1kB */
 
xen/arch/x86/efi/efi-boot.h +2 -2
index abc7d3e3b7..f6f435a4c5 100644
@@ -232,7 +232,7 @@ static void __init noreturn efi_arch_post_exit_boot(void)
     u64 cr4 = XEN_MINIMAL_CR4 & ~X86_CR4_PGE, efer;
 
     efi_arch_relocate_image(__XEN_VIRT_START - xen_phys_start);
-    memcpy((void *)trampoline_phys, trampoline_start, cfg.size);
+    memcpy((void *)trampoline_phys, boot_trampoline_start, cfg.size);
 
     /* Set system registers and transfer control. */
     asm volatile("pushq $0\n\tpopfq");
@@ -565,7 +565,7 @@ static void __init efi_arch_memory_setup(void)
     cfg.addr = 0x100000;
 
     if ( efi_enabled(EFI_LOADER) )
-        cfg.size = trampoline_end - trampoline_start;
+        cfg.size = boot_trampoline_end - boot_trampoline_start;
     else
         cfg.size = TRAMPOLINE_SPACE + TRAMPOLINE_STACK_SPACE;
 
xen/arch/x86/mm.c +1 -1
index e6a4cb28f8..7a2e188916 100644
@@ -314,7 +314,7 @@ void __init arch_init_memory(void)
      */
     BUG_ON(pvh_boot && trampoline_phys != 0x1000);
     for ( i = 0;
-          i < (pvh_boot ? (1 + PFN_UP(trampoline_end - trampoline_start))
+          i < (pvh_boot ? (1 + PFN_UP(boot_trampoline_end - boot_trampoline_start))
                         : 0x100);
           i++ )
         share_xen_page_with_guest(mfn_to_page(_mfn(i)), dom_io, SHARE_rw);
xen/arch/x86/setup.c +2 -2
index 5fa7d3b79c..b714e3c237 100644
@@ -1839,8 +1839,8 @@ int __hwdom_init xen_in_range(unsigned long mfn)
     if ( !xen_regions[0].s )
     {
         /* S3 resume code (and other real mode trampoline code) */
-        xen_regions[region_s3].s = bootsym_phys(trampoline_start);
-        xen_regions[region_s3].e = bootsym_phys(trampoline_end);
+        xen_regions[region_s3].s = bootsym_phys(boot_trampoline_start);
+        xen_regions[region_s3].e = bootsym_phys(boot_trampoline_end);
 
         /*
          * This needs to remain in sync with the uses of the same symbols in
xen/arch/x86/tboot.c +3 -3
index fb4616ae83..9e3d9c4c69 100644
@@ -369,9 +369,9 @@ void tboot_shutdown(uint32_t shutdown_type)
          */
         g_tboot_shared->num_mac_regions = 3;
         /* S3 resume code (and other real mode trampoline code) */
-        g_tboot_shared->mac_regions[0].start = bootsym_phys(trampoline_start);
-        g_tboot_shared->mac_regions[0].size = bootsym_phys(trampoline_end) -
-                                              bootsym_phys(trampoline_start);
+        g_tboot_shared->mac_regions[0].start = bootsym_phys(boot_trampoline_start);
+        g_tboot_shared->mac_regions[0].size = bootsym_phys(boot_trampoline_end) -
+                                              bootsym_phys(boot_trampoline_start);
         /* hypervisor .text + .rodata */
         g_tboot_shared->mac_regions[1].start = (uint64_t)__pa(&_stext);
         g_tboot_shared->mac_regions[1].size = __pa(&__2M_rodata_end) -
xen/arch/x86/x86_64/mm.c +1 -1
index cca4ae926e..ffe8ba2086 100644
@@ -697,7 +697,7 @@ void __init zap_low_mappings(void)
 
     /* Replace with mapping of the boot trampoline only. */
     map_pages_to_xen(trampoline_phys, maddr_to_mfn(trampoline_phys),
-                     PFN_UP(trampoline_end - trampoline_start),
+                     PFN_UP(boot_trampoline_end - boot_trampoline_start),
                      __PAGE_HYPERVISOR);
 }
 
xen/arch/x86/xen.lds.S +3 -3
index 54ac0c0993..5f36b88b92 100644
@@ -361,12 +361,12 @@ ASSERT(IS_ALIGNED(cpu0_stack, STACK_SIZE), "cpu0_stack misaligned")
 ASSERT(IS_ALIGNED(__init_begin, PAGE_SIZE), "__init_begin misaligned")
 ASSERT(IS_ALIGNED(__init_end,   PAGE_SIZE), "__init_end misaligned")
 
-ASSERT(IS_ALIGNED(trampoline_start, 4), "trampoline_start misaligned")
-ASSERT(IS_ALIGNED(trampoline_end,   4), "trampoline_end misaligned")
+ASSERT(IS_ALIGNED(boot_trampoline_start, 4), "boot_trampoline_start misaligned")
+ASSERT(IS_ALIGNED(boot_trampoline_end,   4), "boot_trampoline_end misaligned")
 ASSERT(IS_ALIGNED(__bss_start,      8), "__bss_start misaligned")
 ASSERT(IS_ALIGNED(__bss_end,        8), "__bss_end misaligned")
 
-ASSERT((trampoline_end - trampoline_start) < TRAMPOLINE_SPACE - MBI_SPACE_MIN,
+ASSERT((boot_trampoline_end - boot_trampoline_start) < TRAMPOLINE_SPACE - MBI_SPACE_MIN,
     "not enough room for trampoline and mbi data")
 ASSERT((wakeup_stack - wakeup_stack_start) >= WAKEUP_STACK_MIN,
     "wakeup stack too small")
xen/include/asm-x86/config.h +3 -3
index 9ef9d03ca7..9b7ed50825 100644
@@ -91,11 +91,11 @@
 #ifndef __ASSEMBLY__
 extern unsigned long trampoline_phys;
 #define bootsym_phys(sym)                                 \
-    (((unsigned long)&(sym)-(unsigned long)&trampoline_start)+trampoline_phys)
+    (((unsigned long)&(sym)-(unsigned long)&boot_trampoline_start)+trampoline_phys)
 #define bootsym(sym)                                      \
     (*RELOC_HIDE((typeof(&(sym)))__va(__pa(&(sym))),      \
-                 trampoline_phys-__pa(trampoline_start)))
-extern char trampoline_start[], trampoline_end[];
+                 trampoline_phys-__pa(boot_trampoline_start)))
+extern char boot_trampoline_start[], boot_trampoline_end[];
 extern char trampoline_realmode_entry[];
 extern unsigned int trampoline_xen_phys_start;
 extern unsigned char trampoline_cpu_started;
Signature · 7 lines
2.17.2


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#15 lore raw

RFC PATCH 6/7 x86/boot: Copy 16-bit boot variables back up to Xen image

In reply to David Woodhouse

From: David Woodhouse <[email protected]>

Ditch the bootsym() access from C code for the variables populated by 16-bit boot code. As well as being cleaner this also paves the way for not having the 16-bit boot code in low memory for no-real-mode or EFI loader boots at all.

---
 xen/arch/x86/boot/edd.S           |  2 ++
 xen/arch/x86/boot/head.S          | 16 +++++++++++++++
 xen/arch/x86/boot/mem.S           |  2 ++
 xen/arch/x86/boot/trampoline.S    | 33 ++++++++++++++++++++++++++++---
 xen/arch/x86/boot/video.S         | 30 +++++++++++++++-------------
 xen/arch/x86/platform_hypercall.c | 18 ++++++++---------
 xen/arch/x86/setup.c              | 23 +++++++++++----------
 xen/arch/x86/xen.lds.S            |  8 +++++++-
 xen/include/asm-x86/edd.h         |  1 -
 9 files changed, 93 insertions(+), 40 deletions(-)
xen/arch/x86/boot/edd.S +2 -0
index 434bbbd960..138d04c964 100644
@@ -163,6 +163,7 @@ edd_done:
 .Ledd_mbr_sig_skip:
         ret
 
+        .pushsection .data.boot16, "aw", @progbits
 GLOBAL(boot_edd_info_nr)
         .byte   0
 GLOBAL(boot_mbr_signature_nr)
@@ -171,3 +172,4 @@ GLOBAL(boot_mbr_signature)
         .fill   EDD_MBR_SIG_MAX*8,1,0
 GLOBAL(boot_edd_info)
         .fill   EDD_INFO_MAX * (EDDEXTSIZE + EDDPARMSIZE), 1, 0
+        .popsection
xen/arch/x86/boot/head.S +16 -0
index 82342769c7..7d6c8d3292 100644
@@ -732,6 +732,17 @@ trampoline_setup:
         cmp     $sym_offs(__bootsym_seg_stop),%edi
         jb      1b
 
+        /* Relocations for the boot data section. */
+        mov     sym_fs(trampoline_phys),%edx
+        add     $(boot_trampoline_end - boot_trampoline_start),%edx
+        mov     $sym_offs(__bootdatasym_rel_start),%edi
+1:
+        mov     %fs:(%edi),%eax
+        add     %edx,%fs:(%edi,%eax)
+        add     $4,%edi
+        cmp     $sym_offs(__bootdatasym_rel_stop),%edi
+        jb      1b
+
         /* Do not parse command line on EFI platform here. */
         cmpb    $0,sym_fs(efi_platform)
         jnz     1f
@@ -769,6 +780,11 @@ trampoline_setup:
         mov     $((boot_trampoline_end - boot_trampoline_start) / 4),%ecx
         rep movsl %fs:(%esi),%es:(%edi)
 
+        /* Copy boot data template to low memory. */
+        mov     $sym_offs(bootdata_start),%esi
+        mov     $((bootdata_end - bootdata_start + 3) / 4),%ecx
+        rep movsl %fs:(%esi),%es:(%edi)
+
         /* Jump into the relocated trampoline. */
         lret
 
xen/arch/x86/boot/mem.S +2 -0
index aa39608442..86f0fa9af7 100644
@@ -67,6 +67,7 @@ get_memory_map:
         ret
 
         .align  4
+        .pushsection .data.boot16, "aw", @progbits
 GLOBAL(bios_e820map)
         .fill   E820_BIOS_MAX*20,1,0
 GLOBAL(bios_e820nr)
@@ -75,3 +76,4 @@ GLOBAL(lowmem_kb)
         .long   0
 GLOBAL(highmem_kb)
         .long   0
+	.popsection
xen/arch/x86/boot/trampoline.S +30 -3
index 0f4a740fcb..fdfee2edb1 100644
@@ -47,11 +47,15 @@
         .long 111b - (off) - .;            \
         .popsection
 
-#define bootdatasym(s) ((s)-boot_trampoline_start)
+        .pushsection .data.boot16, "aw", @progbits
+GLOBAL(bootdata_start)
+        .popsection
+
+#define bootdatasym(s) ((s)-bootdata_start+(boot_trampoline_end-boot_trampoline_start))
 #define bootdatasym_rel(sym, off, opnd...) \
         bootdatasym(sym),##opnd;           \
 111:;                                      \
-        .pushsection .bootdatasym_rel, "a";\
+        .pushsection .bootsym_rel, "a";\
         .long 111b - (off) - .;            \
         .popsection
 
@@ -100,7 +104,7 @@ GLOBAL(trampoline_cpu_started)
         .word   0
 idt_48: .word   0, 0, 0 # base = limit = 0
         .word   0
-gdt_48: .word   6*8-1
+gdt_48: .word   7*8-1
         .long   tramp32sym_rel(trampoline_gdt,4)
 
 /* Start of tramp32sym section which can be used in place during boot */
@@ -312,6 +316,23 @@ trampoline_boot_cpu_entry:
         mov     %eax,%gs
         mov     %eax,%ss
 
+        /*
+         * Copy locally-gathered data back up into the Xen physical image
+         */
+        mov     $BOOT_FS,%eax
+        mov     %eax,%es
+
+        mov     $sym_offs(bootdata_end),%ecx
+        mov     $sym_offs(bootdata_start),%edi
+        sub     %edi,%ecx
+        mov     $bootdatasym_rel(bootdata_start,4,%esi)
+        rep movsb %ds:(%esi),%es:(%edi)
+
+        /*
+         * %es still points to BOOT_FS but trampoline_protmode_entry
+         * reloads it anyway.
+         */
+
         /* EBX == 0 indicates we are the BP (Boot Processor). */
         xor     %ebx,%ebx
 
@@ -339,8 +360,10 @@ vesa_size:
         .word   0,0,0                           /* width x depth x height */
 #endif
 
+        .pushsection .data.boot16, "aw", @progbits
 GLOBAL(kbd_shift_flags)
         .byte   0
+        .popsection
 
 rm_idt: .word   256*4-1, 0, 0
 
@@ -349,3 +372,7 @@ rm_idt: .word   256*4-1, 0, 0
 #ifdef CONFIG_VIDEO
 #include "video.S"
 #endif
+
+        .pushsection .data.boot16, "aw", @progbits
+GLOBAL(bootdata_end)
+        .popsection
xen/arch/x86/boot/video.S +16 -14
index 5087c6a4d5..4608464b77 100644
@@ -15,10 +15,10 @@
 
 #include "video.h"
 
-/* Scratch space layout: boot_trampoline_end to boot_trampoline_end+0x1000. */
-#define modelist       bootsym(boot_trampoline_end)   /* 2kB (256 entries) */
-#define vesa_glob_info (modelist + 0x800)        /* 1kB */
-#define vesa_mode_info (vesa_glob_info + 0x400)  /* 1kB */
+/* Scratch space layout: bootdata_end to bootdata_end+0x1000. */
+#define modelist(t)       bootdatasym_rel(bootdata_end,2,t)         /* 2KiB (256 entries) */
+#define vesa_glob_info(t) bootdatasym_rel((bootdata_end+0x800),2,t) /* 1KiB */
+#define vesa_mode_info(t) bootdatasym_rel((bootdata_end+0xc00),2,t) /* 1KiB */
 
 /* Retrieve Extended Display Identification Data. */
 #define CONFIG_FIRMWARE_EDID
@@ -113,7 +113,7 @@ mopar2: movb    %al, _param(PARAM_VIDEO_LINES)
 
 # Fetching of VESA frame buffer parameters
 mopar_gr:
-        leaw    vesa_mode_info, %di
+        leaw    vesa_mode_info(%di)
         movb    $0x23, _param(PARAM_HAVE_VGA)
         movw    16(%di), %ax
         movw    %ax, _param(PARAM_LFB_LINELENGTH)
@@ -134,7 +134,7 @@ mopar_gr:
         movw    %ax, _param(PARAM_VESA_ATTRIB)
 
 # get video mem size
-        leaw    vesa_glob_info, %di
+        leaw    vesa_glob_info(%di)
         xorl    %eax, %eax
         movw    18(%di), %ax
         movl    %eax, _param(PARAM_LFB_SIZE)
@@ -226,7 +226,7 @@ an1:    call    prtstr
         leaw    bootsym(listhdr), %si   # Table header
         call    prtstr
         movb    $0x30, %dl              # DL holds mode number
-        leaw    modelist, %si
+        leaw    modelist(%si)
 lm1:    cmpw    $ASK_VGA, (%si)         # End?
         jz      lm2
 
@@ -435,13 +435,13 @@ setmenu:
         jmp     mode_set
 
 check_vesa:
-        leaw    vesa_glob_info, %di
+        leaw    vesa_glob_info(%di)
         movw    $0x4f00, %ax
         int     $0x10
         cmpw    $0x004f, %ax
         jnz     setbad
 
-        leaw    vesa_mode_info, %di
+        leaw    vesa_mode_info(%di)
         subb    $VIDEO_FIRST_VESA>>8, %bh
         movw    %bx, %cx                # Get mode information structure
         movw    $0x4f01, %ax
@@ -509,7 +509,7 @@ inidx:  outb    %al, %dx                # Read from indexed VGA register
 
 setvesabysize:
         call    mode_table
-        leaw    modelist,%si
+        leaw    modelist(%si)
 1:      add     $8,%si
         cmpw    $ASK_VGA,-8(%si)        # End?
         je      _setbad
@@ -669,7 +669,7 @@ mode_table:
         orw     %di, %di
         jnz     mtab1
 
-        leaw    modelist, %di           # Store standard modes:
+        leaw    modelist(%di)           # Store standard modes:
         movw    $VIDEO_80x25,(%di)      # The 80x25 mode (ALL)
         movw    $0x50,2(%di)
         movw    $0x19,4(%di)
@@ -684,7 +684,7 @@ mode_table:
 
         movw    $ASK_VGA, (%di)         # End marker
         movw    %di, bootsym(mt_end)
-mtab1:  leaw    modelist, %si           # SI=mode list, DI=list end
+mtab1:  leaw    modelist(%si)           # SI=mode list, DI=list end
 ret0:   ret
 
 # Modes usable on all standard VGAs
@@ -700,7 +700,7 @@ vga_modes_end:
 # Detect VESA modes.
 vesa_modes:
         movw    %di, %bp                # BP=original mode table end
-        leaw    vesa_glob_info, %di
+        leaw    vesa_glob_info(%di)
         movw    $0x4f00, %ax            # VESA Get card info call
         int     $0x10
         movw    %di, %si
@@ -897,7 +897,7 @@ store_edid:
         cmpb    $1, bootsym(opt_edid)   # EDID disabled on cmdline (edid=no)?
         je      .Lno_edid
 
-        leaw    vesa_glob_info, %di
+        leaw    vesa_glob_info(%di)
         movw    $0x4f00, %ax
         int     $0x10
         cmpw    $0x004f, %ax
@@ -990,6 +990,7 @@ name_bann:      .asciz  "Video adapter: "
 
 force_size:     .word   0       # Use this size instead of the one in BIOS vars
 
+        .pushsection .data.boot16, "aw", @progbits
 GLOBAL(boot_vid_info)
         .byte   0, 0    /* orig_x, orig_y */
         .byte   3       /* text mode 3    */
@@ -1001,3 +1002,4 @@ GLOBAL(boot_edid_info)
         .fill   128,1,0x13
 GLOBAL(boot_edid_caps)
         .word   0x1313
+        .popsection
xen/arch/x86/platform_hypercall.c +9 -9
index ea18c3215a..5c8953ae19 100644
@@ -333,10 +333,10 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
             u16 length;
 
             ret = -ESRCH;
-            if ( op->u.firmware_info.index >= bootsym(boot_edd_info_nr) )
+            if ( op->u.firmware_info.index >= boot_edd_info_nr )
                 break;
 
-            info = bootsym(boot_edd_info) + op->u.firmware_info.index;
+            info = boot_edd_info + op->u.firmware_info.index;
 
             /* Transfer the EDD info block. */
             ret = -EFAULT;
@@ -372,10 +372,10 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
             const struct mbr_signature *sig;
 
             ret = -ESRCH;
-            if ( op->u.firmware_info.index >= bootsym(boot_mbr_signature_nr) )
+            if ( op->u.firmware_info.index >= boot_mbr_signature_nr )
                 break;
 
-            sig = bootsym(boot_mbr_signature) + op->u.firmware_info.index;
+            sig = boot_mbr_signature + op->u.firmware_info.index;
 
             op->u.firmware_info.u.disk_mbr_signature.device = sig->device;
             op->u.firmware_info.u.disk_mbr_signature.mbr_signature =
@@ -391,13 +391,13 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
 #ifdef CONFIG_VIDEO
             if ( op->u.firmware_info.index != 0 )
                 break;
-            if ( *(u32 *)bootsym(boot_edid_info) == 0x13131313 )
+            if ( *(u32 *)boot_edid_info == 0x13131313 )
                 break;
 
             op->u.firmware_info.u.vbeddc_info.capabilities =
-                bootsym(boot_edid_caps);
+                boot_edid_caps;
             op->u.firmware_info.u.vbeddc_info.edid_transfer_time =
-                bootsym(boot_edid_caps) >> 8;
+                boot_edid_caps >> 8;
 
             ret = 0;
             if ( __copy_field_to_guest(u_xenpf_op, op, u.firmware_info.
@@ -405,7 +405,7 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
                  __copy_field_to_guest(u_xenpf_op, op, u.firmware_info.
                                        u.vbeddc_info.edid_transfer_time) ||
                  copy_to_compat(op->u.firmware_info.u.vbeddc_info.edid,
-                                bootsym(boot_edid_info), 128) )
+                                boot_edid_info, 128) )
                 ret = -EFAULT;
 #endif
             break;
@@ -422,7 +422,7 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
             if ( op->u.firmware_info.index != 0 )
                 break;
 
-            op->u.firmware_info.u.kbd_shift_flags = bootsym(kbd_shift_flags);
+            op->u.firmware_info.u.kbd_shift_flags = kbd_shift_flags;
 
             ret = 0;
             if ( __copy_field_to_guest(u_xenpf_op, op,
xen/arch/x86/setup.c +11 -12
index b714e3c237..4dd4c4161c 100644
@@ -507,7 +507,7 @@ extern struct boot_video_info boot_vid_info;
 static void __init parse_video_info(void)
 {
 #ifdef CONFIG_VIDEO
-    struct boot_video_info *bvi = &bootsym(boot_vid_info);
+    struct boot_video_info *bvi = &boot_vid_info;
 
     /* vga_console_info is filled directly on EFI platform. */
     if ( efi_enabled(EFI_BOOT) )
@@ -666,12 +666,11 @@ static char * __init cmdline_cook(char *p, const char *loader_name)
 
 static int copy_bios_e820(struct e820entry *map, unsigned int limit)
 {
-    unsigned int n = bootsym(bios_e820nr);
+    unsigned int n = bios_e820nr;
     if (n > limit)
         n = limit;
-
     if (n)
-        memcpy(map, bootsym(bios_e820map), sizeof(*map) * n);
+        memcpy(map, bios_e820map, sizeof(*map) * n);
 
     return n;
 }
@@ -802,15 +801,15 @@ void __init noreturn __start_xen(unsigned long mbi_p)
     }
 
     /* Print VBE/DDC EDID information. */
-    if ( bootsym(boot_edid_caps) != 0x1313 )
+    if ( boot_edid_caps != 0x1313 )
     {
-        u16 caps = bootsym(boot_edid_caps);
+        u16 caps = boot_edid_caps;
         printk(" VBE/DDC methods:%s%s%s; ",
                (caps & 1) ? " V1" : "",
                (caps & 2) ? " V2" : "",
                !(caps & 3) ? " none" : "");
         printk("EDID transfer time: %d seconds\n", caps >> 8);
-        if ( *(u32 *)bootsym(boot_edid_info) == 0x13131313 )
+        if ( *(u32 *)boot_edid_info == 0x13131313 )
         {
             printk(" EDID info not retrieved because ");
             if ( !(caps & 3) )
@@ -825,9 +824,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 
     printk("Disc information:\n");
     printk(" Found %d MBR signatures\n",
-           bootsym(boot_mbr_signature_nr));
+           boot_mbr_signature_nr);
     printk(" Found %d EDD information structures\n",
-           bootsym(boot_edd_info_nr));
+           boot_edd_info_nr);
 
     /* Check that we have at least one Multiboot module. */
     if ( !(mbi->flags & MBI_MODULES) || (mbi->mods_count == 0) )
@@ -899,14 +898,14 @@ void __init noreturn __start_xen(unsigned long mbi_p)
             bytes += map->size + 4;
         }
     }
-    else if ( bootsym(lowmem_kb) )
+    else if ( lowmem_kb )
     {
         memmap_type = "Xen-e801";
         e820_raw.map[0].addr = 0;
-        e820_raw.map[0].size = bootsym(lowmem_kb) << 10;
+        e820_raw.map[0].size = lowmem_kb << 10;
         e820_raw.map[0].type = E820_RAM;
         e820_raw.map[1].addr = 0x100000;
-        e820_raw.map[1].size = bootsym(highmem_kb) << 10;
+        e820_raw.map[1].size = highmem_kb << 10;
         e820_raw.map[1].type = E820_RAM;
         e820_raw.nr_map = 2;
     }
xen/arch/x86/xen.lds.S +7 -1
index 5f36b88b92..436781cefa 100644
@@ -228,11 +228,13 @@ SECTIONS
        __trampoline32_rel_stop = .;
        __bootsym_rel_start = .;
        *(.bootsym_rel)
-       *(.bootdatasym_rel)
        __bootsym_rel_stop = .;
        __bootsym_seg_start = .;
        *(.bootsym_seg)
        __bootsym_seg_stop = .;
+       __bootdatasym_rel_start = .;
+       *(.bootdatasym_rel)
+       __bootdatasym_rel_stop = .;
        /*
         * struct alt_inst entries. From the header (alternative.h):
         * "Alternative instructions for different CPU types or capabilities"
@@ -275,6 +277,10 @@ SECTIONS
   DECL_SECTION(.data) {
        *(.data.page_aligned)
        *(.data)
+       . = ALIGN(16);
+       __bootdata_start = .;
+       *(.data.boot16)
+       __bootdata_end = .;
        *(.data.rel)
        *(.data.rel.*)
        CONSTRUCTORS
xen/include/asm-x86/edd.h +0 -1
index afaa23732a..a4d6b4d90e 100644
@@ -143,7 +143,6 @@ struct __packed mbr_signature {
     u32 signature;
 };
 
-/* These all reside in the boot trampoline. Access via bootsym(). */
 extern struct mbr_signature boot_mbr_signature[];
 extern u8 boot_mbr_signature_nr;
 extern struct edd_info boot_edd_info[];
Signature · 7 lines
2.17.2


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#16 lore raw

RFC PATCH 7/7 x86/boot: Do not use trampoline for no-real-mode boot paths

In reply to David Woodhouse

From: David Woodhouse <[email protected]>

Where booted from EFI or with no-real-mode, there is no need to stomp on low memory with the 16-boot code. Instead, just go straight to trampoline_protmode_entry() at its physical location within the Xen image.

For now, the boot code (including the EFI loader path) still determines what the trampoline_phys address should be. The trampoline is actually relocated for that address and copied into low memory, from a relocate_trampoline() call made from __start_xen().

For subsequent AP startup and wakeup, the 32-bit trampoline can't trivially be used in-place as that region isn't mapped. So copy it down to low memory too, having relocated it (again) to work from there.

---
 xen/arch/x86/acpi/power.c      |  6 +--
 xen/arch/x86/boot/head.S       | 67 ++++++++++++++++------------------
 xen/arch/x86/boot/trampoline.S | 30 +++++++++++----
 xen/arch/x86/cpu/common.c      |  2 +-
 xen/arch/x86/cpu/intel.c       |  2 +-
 xen/arch/x86/efi/efi-boot.h    | 31 ++--------------
 xen/arch/x86/mm.c              |  2 +-
 xen/arch/x86/setup.c           | 43 ++++++++++++++++++++--
 xen/arch/x86/smpboot.c         |  6 +--
 xen/arch/x86/tboot.c           |  6 +--
 xen/arch/x86/x86_64/mm.c       |  2 +-
 xen/include/asm-x86/acpi.h     |  2 +-
 xen/include/asm-x86/config.h   | 10 ++---
 13 files changed, 117 insertions(+), 92 deletions(-)
xen/arch/x86/acpi/power.c +3 -3
index a704c7c340..a1ce8acb21 100644
@@ -152,9 +152,9 @@ static void acpi_sleep_prepare(u32 state)
         return;
 
     if ( acpi_sinfo.vector_width == 32 )
-        *(uint32_t *)wakeup_vector_va = bootsym_phys(wakeup_start);
+        *(uint32_t *)wakeup_vector_va = trampsym_phys(wakeup_start);
     else
-        *(uint64_t *)wakeup_vector_va = bootsym_phys(wakeup_start);
+        *(uint64_t *)wakeup_vector_va = trampsym_phys(wakeup_start);
 }
 
 static void acpi_sleep_post(u32 state) {}
@@ -388,7 +388,7 @@ static void tboot_sleep(u8 sleep_state)
     g_tboot_shared->acpi_sinfo.wakeup_vector = acpi_sinfo.wakeup_vector;
     g_tboot_shared->acpi_sinfo.vector_width = acpi_sinfo.vector_width;
     g_tboot_shared->acpi_sinfo.kernel_s3_resume_vector =
-                                              bootsym_phys(wakeup_start);
+                                              trampsym_phys(wakeup_start);
 
     switch ( sleep_state )
     {
xen/arch/x86/boot/head.S +32 -35

97 lines not on this page · raw message

xen/arch/x86/boot/trampoline.S +23 -7
index fdfee2edb1..1ce815e0da 100644
@@ -60,7 +60,7 @@ GLOBAL(bootdata_start)
         .popsection
 
 #undef trampsym
-#define trampsym(s) ((s)-boot_trampoline_start)
+#define trampsym(s) ((s)-perm_trampoline_start)
 
 #define trampsym_rel(sym, off, opnd...)    \
         trampsym(sym),##opnd;              \
@@ -70,7 +70,7 @@ GLOBAL(bootdata_start)
         .popsection
 
 #undef tramp32sym
-#define tramp32sym(s) ((s)-boot_trampoline_start)
+#define tramp32sym(s) ((s)-perm_trampoline_start)
 
 #define tramp32sym_rel(sym, off, opnd...)  \
         tramp32sym(sym),##opnd;            \
@@ -83,6 +83,7 @@ GLOBAL(bootdata_start)
 
         .code16
 
+ENTRY(perm_trampoline_start)
 GLOBAL(trampoline_realmode_entry)
         mov     %cs,%ax
         mov     %ax,%ds
@@ -90,7 +91,7 @@ GLOBAL(trampoline_realmode_entry)
         cld
         cli
         lidt    trampsym(idt_48)
-        lgdt    trampsym(gdt_48)
+        lgdtl   trampsym(gdt_48)
         mov     $1,%bl                    # EBX != 0 indicates we are an AP
         xor     %ax, %ax
         inc     %ax
@@ -230,11 +231,23 @@ start64:
 
 /* The first page of trampoline is permanent, the rest boot-time only. */
 /* Reuse the boot trampoline on the 1st trampoline page as stack for wakeup. */
-        .equ    wakeup_stack, boot_trampoline_start + PAGE_SIZE
+        .equ    wakeup_stack, perm_trampoline_start + PAGE_SIZE
         .global wakeup_stack
 
+ENTRY(perm_trampoline_end)
+
 /* From here on early boot only. */
 
+ENTRY(boot_trampoline_start)
+
+        .word   0
+boot16_idt:
+        .word   0, 0, 0 # base = limit = 0
+        .word   0
+boot16_gdt:
+        .word   7*8-1
+        .long   tramp32sym_rel(trampoline_gdt,4)
+
         .code32
 trampoline_boot_cpu_entry:
         /* Load pseudo-real-mode segments. */
@@ -298,8 +311,8 @@ trampoline_boot_cpu_entry:
         cli
 
         /* Reset GDT and IDT. Some BIOSes clobber GDTR. */
-        lidt    bootsym(idt_48)
-        lgdt    bootsym(gdt_48)
+        lidt    bootsym(boot16_idt) # XXX
+        lgdtl   bootsym(boot16_gdt)
 
         /* Enter protected mode, and flush insn queue. */
         xor     %ax,%ax
@@ -337,7 +350,8 @@ trampoline_boot_cpu_entry:
         xor     %ebx,%ebx
 
         /* Jump to the common bootstrap entry point. */
-        jmp     trampoline_protmode_entry
+        mov     $tramp32sym_rel(trampoline_protmode_entry,4,%eax)
+        jmp     *%eax
 
 #include "video.h"
 
@@ -373,6 +387,8 @@ rm_idt: .word   256*4-1, 0, 0
 #include "video.S"
 #endif
 
+ENTRY(boot_trampoline_end)
+
         .pushsection .data.boot16, "aw", @progbits
 GLOBAL(bootdata_end)
         .popsection
xen/arch/x86/cpu/common.c +1 -1
index 1d21ff9467..b268e010c5 100644
@@ -408,7 +408,7 @@ static void generic_identify(struct cpuinfo_x86 *c)
 		      &c->x86_capability[cpufeat_word(X86_FEATURE_LAHF_LM)],
 		      &c->x86_capability[cpufeat_word(X86_FEATURE_SYSCALL)]);
 	if (c == &boot_cpu_data)
-		bootsym(cpuid_ext_features) =
+		trampsym(cpuid_ext_features) =
 			c->x86_capability[cpufeat_word(X86_FEATURE_NX)];
 
 	if (c->extended_cpuid_level >= 0x80000004)
xen/arch/x86/cpu/intel.c +1 -1
index 8e23ed6379..e7f794a601 100644
@@ -240,7 +240,7 @@ static void early_init_intel(struct cpuinfo_x86 *c)
 				 MSR_IA32_MISC_ENABLE_XD_DISABLE);
 	if (disable) {
 		wrmsrl(MSR_IA32_MISC_ENABLE, misc_enable & ~disable);
-		bootsym(trampoline_misc_enable_off) |= disable;
+		trampsym(trampoline_misc_enable_off) |= disable;
 	}
 
 	if (disable & MSR_IA32_MISC_ENABLE_LIMIT_CPUID)
xen/arch/x86/efi/efi-boot.h +3 -28
index f6f435a4c5..ecec334d60 100644
@@ -98,29 +98,6 @@ static void __init efi_arch_relocate_image(unsigned long delta)
     }
 }
 
-extern const s32 __trampoline_rel_start[], __trampoline_rel_stop[];
-extern const s32 __trampoline32_rel_start[], __trampoline32_rel_stop[];
-
-static void __init relocate_trampoline(unsigned long phys)
-{
-    const s32 *trampoline_ptr;
-
-    trampoline_phys = phys;
-
-    if ( !efi_enabled(EFI_LOADER) )
-        return;
-
-    /* Apply relocations to trampoline. */
-    for ( trampoline_ptr = __trampoline_rel_start;
-          trampoline_ptr < __trampoline_rel_stop;
-          ++trampoline_ptr )
-        *(u32 *)(*trampoline_ptr + (long)trampoline_ptr) += phys;
-    for ( trampoline_ptr = __trampoline32_rel_start;
-          trampoline_ptr < __trampoline32_rel_stop;
-          ++trampoline_ptr )
-        *(u32 *)(*trampoline_ptr + (long)trampoline_ptr) += phys;
-}
-
 static void __init place_string(u32 *addr, const char *s)
 {
     char *alloc = NULL;
@@ -223,7 +200,7 @@ static void __init efi_arch_pre_exit_boot(void)
     {
         if ( !cfg.addr )
             blexit(L"No memory for trampoline");
-        relocate_trampoline(cfg.addr);
+        trampoline_phys = cfg.addr;
     }
 }
 
@@ -232,7 +209,6 @@ static void __init noreturn efi_arch_post_exit_boot(void)
     u64 cr4 = XEN_MINIMAL_CR4 & ~X86_CR4_PGE, efer;
 
     efi_arch_relocate_image(__XEN_VIRT_START - xen_phys_start);
-    memcpy((void *)trampoline_phys, boot_trampoline_start, cfg.size);
 
     /* Set system registers and transfer control. */
     asm volatile("pushq $0\n\tpopfq");
@@ -565,14 +541,14 @@ static void __init efi_arch_memory_setup(void)
     cfg.addr = 0x100000;
 
     if ( efi_enabled(EFI_LOADER) )
-        cfg.size = boot_trampoline_end - boot_trampoline_start;
+        cfg.size = perm_trampoline_end - perm_trampoline_start;
     else
         cfg.size = TRAMPOLINE_SPACE + TRAMPOLINE_STACK_SPACE;
 
     status = efi_bs->AllocatePages(AllocateMaxAddress, EfiLoaderData,
                                    PFN_UP(cfg.size), &cfg.addr);
     if ( status == EFI_SUCCESS )
-        relocate_trampoline(cfg.addr);
+        trampoline_phys = cfg.addr;
     else
     {
         cfg.addr = 0;
@@ -664,7 +640,6 @@ static void __init efi_arch_load_addr_check(EFI_LOADED_IMAGE *loaded_image)
         blexit(L"Xen must be loaded below 4Gb.");
     if ( xen_phys_start & ((1 << L2_PAGETABLE_SHIFT) - 1) )
         blexit(L"Xen must be loaded at a 2Mb boundary.");
-    trampoline_xen_phys_start = xen_phys_start;
 }
 
 static bool __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
xen/arch/x86/mm.c +1 -1
index 7a2e188916..17a55b4b8a 100644
@@ -314,7 +314,7 @@ void __init arch_init_memory(void)
      */
     BUG_ON(pvh_boot && trampoline_phys != 0x1000);
     for ( i = 0;
-          i < (pvh_boot ? (1 + PFN_UP(boot_trampoline_end - boot_trampoline_start))
+          i < (pvh_boot ? (1 + PFN_UP(perm_trampoline_end - perm_trampoline_start))
                         : 0x100);
           i++ )
         share_xen_page_with_guest(mfn_to_page(_mfn(i)), dom_io, SHARE_rw);
xen/arch/x86/setup.c +40 -3

67 lines not on this page · raw message

xen/arch/x86/smpboot.c +3 -3
index 7e76cc3d68..02d97a1aa3 100644
@@ -48,7 +48,7 @@
 #include <mach_wakecpu.h>
 #include <smpboot_hooks.h>
 
-#define setup_trampoline()    (bootsym_phys(trampoline_realmode_entry))
+#define setup_trampoline()    (trampsym_phys(trampoline_realmode_entry))
 
 unsigned long __read_mostly trampoline_phys;
 
@@ -600,7 +600,7 @@ static int do_boot_cpu(int apicid, int cpu)
         {
             boot_error = 1;
             smp_mb();
-            if ( bootsym(trampoline_cpu_started) == 0xA5 )
+            if ( trampsym(trampoline_cpu_started) == 0xA5 )
                 /* trampoline started but...? */
                 printk("Stuck ??\n");
             else
@@ -616,7 +616,7 @@ static int do_boot_cpu(int apicid, int cpu)
     }
 
     /* mark "stuck" area as not stuck */
-    bootsym(trampoline_cpu_started) = 0;
+    trampsym(trampoline_cpu_started) = 0;
     smp_mb();
 
     smpboot_restore_warm_reset_vector();
xen/arch/x86/tboot.c +3 -3
index 9e3d9c4c69..006f0678a6 100644
@@ -369,9 +369,9 @@ void tboot_shutdown(uint32_t shutdown_type)
          */
         g_tboot_shared->num_mac_regions = 3;
         /* S3 resume code (and other real mode trampoline code) */
-        g_tboot_shared->mac_regions[0].start = bootsym_phys(boot_trampoline_start);
-        g_tboot_shared->mac_regions[0].size = bootsym_phys(boot_trampoline_end) -
-                                              bootsym_phys(boot_trampoline_start);
+        g_tboot_shared->mac_regions[0].start = trampsym_phys(perm_trampoline_start);
+        g_tboot_shared->mac_regions[0].size = trampsym_phys(perm_trampoline_end) -
+                                              trampsym_phys(perm_trampoline_start);
         /* hypervisor .text + .rodata */
         g_tboot_shared->mac_regions[1].start = (uint64_t)__pa(&_stext);
         g_tboot_shared->mac_regions[1].size = __pa(&__2M_rodata_end) -
xen/arch/x86/x86_64/mm.c +1 -1
index ffe8ba2086..dbbbc64beb 100644
@@ -697,7 +697,7 @@ void __init zap_low_mappings(void)
 
     /* Replace with mapping of the boot trampoline only. */
     map_pages_to_xen(trampoline_phys, maddr_to_mfn(trampoline_phys),
-                     PFN_UP(boot_trampoline_end - boot_trampoline_start),
+                     PFN_UP(perm_trampoline_end - perm_trampoline_start),
                      __PAGE_HYPERVISOR);
 }
 
xen/include/asm-x86/acpi.h +1 -1
index c138def1bd..dffe02a363 100644
@@ -110,7 +110,7 @@ extern int acpi_scan_nodes(u64 start, u64 end);
 #ifdef CONFIG_ACPI_SLEEP
 
 extern struct acpi_sleep_info acpi_sinfo;
-#define acpi_video_flags bootsym(video_flags)
+#define acpi_video_flags trampsym(video_flags)
 struct xenpf_enter_acpi_sleep;
 extern int acpi_enter_sleep(struct xenpf_enter_acpi_sleep *sleep);
 extern int acpi_enter_state(u32 state);
xen/include/asm-x86/config.h +5 -5
index 9b7ed50825..dbd7cb255a 100644
@@ -90,12 +90,12 @@
 
 #ifndef __ASSEMBLY__
 extern unsigned long trampoline_phys;
-#define bootsym_phys(sym)                                 \
-    (((unsigned long)&(sym)-(unsigned long)&boot_trampoline_start)+trampoline_phys)
-#define bootsym(sym)                                      \
+#define trampsym_phys(sym)                                 \
+    (((unsigned long)&(sym)-(unsigned long)&perm_trampoline_start)+trampoline_phys)
+#define trampsym(sym)                                      \
     (*RELOC_HIDE((typeof(&(sym)))__va(__pa(&(sym))),      \
-                 trampoline_phys-__pa(boot_trampoline_start)))
-extern char boot_trampoline_start[], boot_trampoline_end[];
+                 trampoline_phys-__pa(perm_trampoline_start)))
+extern char perm_trampoline_start[], perm_trampoline_end[];
 extern char trampoline_realmode_entry[];
 extern unsigned int trampoline_xen_phys_start;
 extern unsigned char trampoline_cpu_started;
Signature · 7 lines
2.17.2


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel