kexec: put bzImage and ramdisk above 4G for x86 64bit

So could limit range for 4g above buffers.

Quoted 1 line · Yinghai Lu

So could limit range for 4g above buffers.

What is wrong with mem-min and mem-max? At this point in the patchset it looks like you are introducing mem64-min and mem64-max as a hack to avoid fixing mem-min and mem-max properly.

Quoted 2 lines · Eric W. Biederman

Yinghai Lu <[email protected]> writes:

Quoted 5 lines

Quoted 1 line

So could limit range for 4g above buffers.

What is wrong with mem-min and mem-max? At this point in the patchset it looks like you are introducing mem64-min and mem64-max as a hack to avoid fixing mem-min and mem-max properly.

if we set mem-min high, some buffers for purgatory and real_mode can not be allocated properly.

mem64-min and mem64-max are used to limit range for buffer that could stay above 4g.
like limit them one range belong to one node only.

Quoted 4 lines · Yinghai Lu

Quoted 2 lines · Eric W. Biederman

Yinghai Lu <[email protected]> writes:

Quoted 8 lines

Quoted 5 lines

Quoted 1 line

So could limit range for 4g above buffers.

What is wrong with mem-min and mem-max? At this point in the patchset it looks like you are introducing mem64-min and mem64-max as a hack to avoid fixing mem-min and mem-max properly.

if we set mem-min high, some buffers for purgatory and real_mode can not be allocated properly.

Let's see. For a 32bit kexec that is a fundamental limit, even if we are booting a 64bit kernel.

For a 64bit kexec we have a 64bit purgatory so it should not be a problem to relocate it higher.

Hmm. I'm not certain about the real_mode bits. Splitting out the 64bit bzImage loader from the 32bit bzImage loader should allow a lot of the legacy bits to be deleted. Past that I think we simply down in the real of needing a command line pointer that is 64bit instead of the current 32bit one. That we should be able to fix by fixing the boot protocol.

Since the real mode bits when loading a 64bit kernel are just a parameter area there should be no fundamental reason for them to be below 4G.

The code needs to default to loading the kernel in the non kdump case at the address it was compiled to run at. But for the rest I really don't see why we can't load the kernel very high.

Quoted 9 lines · Eric W. Biederman

Quoted 7 lines · Yinghai Lu

Quoted 5 lines · Eric W. Biederman

Quoted 1 line · Yinghai Lu

So could limit range for 4g above buffers.

What is wrong with mem-min and mem-max? At this point in the

patchset

Quoted 1 line

Quoted 1 line

Quoted 1 line

it looks like you are introducing mem64-min and mem64-max as a hack

to

Quoted 5 lines

Quoted 4 lines

Quoted 1 line

avoid fixing mem-min and mem-max properly.

if we set mem-min high, some buffers for purgatory and real_mode can not be allocated properly.

Quoted 7 lines

Let's see. For a 32bit kexec that is a fundamental limit, even if we are booting a 64bit kernel.

For a 64bit kexec we have a 64bit purgatory so it should not be a problem to relocate it higher.

Hmm. I'm not certain about the real_mode bits. Splitting out the

64bit

Quoted 1 line

bzImage loader from the 32bit bzImage loader should allow a lot of

the

Quoted 1 line

legacy bits to be deleted. Past that I think we simply down in the

real

Quoted 1 line

of needing a command line pointer that is 64bit instead of the

current

Quoted 1 line

32bit one. That we should be able to fix by fixing the boot

protocol.

Quoted 8 lines

Since the real mode bits when loading a 64bit kernel are just a parameter area there should be no fundamental reason for them to be below 4G.

The code needs to default to loading the kernel in the non kdump case at the address it was compiled to run at. But for the rest I really don't see why we can't load the kernel very high.

then for setup data (boot param) and command line, we have to set extra ident
mapping for them in kernel arch/x86/kernel/head_64.S

with my current patchset for kernel and kexec, we only need to set ident mapping
for [_text, _end) in kernel arch/x86/kernel/head_64.S

Yinghai

Sorry, with "real mode" are we talking about struct boot_params or the actual realmode trampoline? The latter needs to be < 640K as a hw requirement.

Quoted 1 line · H. Peter Anvin

Sorry, with "real mode" are we talking about struct boot_params or the actual realmode trampoline? The latter needs to be < 640K as a hw requirement.

boot_params.

when kexec bzImage64, kexec will copy that from bzImage file, and update it....and
pass pointer to arch/x86/boot/compressed/head_64.c via %rsi

On Sat, Nov 17, 2012 at 12:25 AM, Eric W. Biederman <[email protected]> wrote:

then for setup data (boot param) and command line, we have to set extra ident mapping for them in kernel arch/x86/kernel/head_64.S

Quoted 6 lines · Yinghai Lu

On Sat, Nov 17, 2012 at 12:25 AM, Eric W. Biederman <[email protected]> wrote:

then for setup data (boot param) and command line, we have to set extra ident mapping for them in kernel arch/x86/kernel/head_64.S

find the solution for this one:
We don't need to set ident map for boot_param and command line in head_64.S

as we already get arch/x86/kernel/head64.c, and could use early_ioremap to access
and copy them out.

Quoted 8 lines · Yinghai Lu

Quoted 6 lines · Yinghai Lu

On Sat, Nov 17, 2012 at 12:25 AM, Eric W. Biederman <[email protected]> wrote:

then for setup data (boot param) and command line, we have to set extra ident mapping for them in kernel arch/x86/kernel/head_64.S

Quoted 7 lines

find the solution for this one:
We don't need to set ident map for boot_param and command line in head_64.S

as we already get arch/x86/kernel/head64.c, and could use early_ioremap to access
and copy them out.

I don't think that is feasible; we need access to this structure extremely early. I think if you're using the 64-bit entry point it has to be an entry condition that those structures are mapped. It simply becomes a precondition for using the 64-bit entry point.

Quoted 5 lines

I don't think that is feasible; we need access to this structure extremely early. I think if you're using the 64-bit entry point it has to be an entry condition that those structures are mapped. It simply becomes a precondition for using the 64-bit entry point.

64bit purgatory coming from kexec should be running with a page table that identity maps everything loaded by kexec and in practice all of memory.

We should still be using that page table when we enter the new kernel.

However then the 64bit kernel switches to it's own page tables.

So after that point we potentially need early_ioremap, because the kernel's page tables may not have covered the boot_params. Certainly we need to do something to ensure the kernels page table covers boot_params.

Quoted 10 lines · Eric W. Biederman

Quoted 5 lines

I don't think that is feasible; we need access to this structure extremely early. I think if you're using the 64-bit entry point it has to be an entry condition that those structures are mapped. It simply becomes a precondition for using the 64-bit entry point.

64bit purgatory coming from kexec should be running with a page table that identity maps everything loaded by kexec and in practice all of memory.

Quoted 8 lines

We should still be using that page table when we enter the new kernel.

However then the 64bit kernel switches to it's own page tables.

So after that point we potentially need early_ioremap, because the kernel's page tables may not have covered the boot_params. Certainly we need to do something to ensure the kernels page table covers boot_params.

The easy way to fix that is to do what we do on 32 bits: we copy it into the bss before we turn paging on (the 64-bit equivalent is before we switch to the new page tables.) That way we know where it is, at least for the bounded-size data items.

Quoted 5 lines

The easy way to fix that is to do what we do on 32 bits: we copy it into the bss before we turn paging on (the 64-bit equivalent is before we switch to the new page tables.) That way we know where it is, at least for the bounded-size data items.

Whatever works. It sounds like six of one half a dozen of the other to me. Last I looked early_ioremap works as soon as paging is enabled on x86_64.

"H. Peter Anvin" <[email protected]> writes:

64bit purgatory coming from kexec should be running with a page table that identity maps everything loaded by kexec and in practice all of memory.

We should still be using that page table when we enter the new kernel.

However then the 64bit kernel switches to it's own page tables.

So after that point we potentially need early_ioremap, because the kernel's page tables may not have covered the boot_params. Certainly we need to do something to ensure the kernels page table covers boot_params.

Quoted 6 lines · Eric W. Biederman

"H. Peter Anvin" <[email protected]> writes:

64bit purgatory coming from kexec should be running with a page table that identity maps everything loaded by kexec and in practice all of memory.

there is lots of R_X86_64_32 and R_X86_64_32S for 64bit purgatory.

those come from global variables...could kill some by converting them static...

but still have some global string or ro data....

build one big file include all .S ?

Quoted 7 lines · Yinghai Lu

there is lots of R_X86_64_32 and R_X86_64_32S for 64bit purgatory.

those come from global variables...could kill some by converting them static...

but still have some global string or ro data....

Quoted 1 line

build one big file include all .S ?

For R_x86_64_32 and R_x86_64_32S the problem is that the instructions are using absolute 32bit addresses.

It is probably overkill but we should be able to solve this with by adding "-mcmodel=large" to the build of purgatory.

Hopefully there are not 32bit or 16bit assembly routines getting linked in and using problemenatic instructions.

Quoted 8 lines · Eric W. Biederman

Quoted 7 lines · Yinghai Lu

there is lots of R_X86_64_32 and R_X86_64_32S for 64bit purgatory.

those come from global variables...could kill some by converting them static...

but still have some global string or ro data....

Quoted 7 lines

Quoted 1 line

build one big file include all .S ?

For R_x86_64_32 and R_x86_64_32S the problem is that the instructions are using absolute 32bit addresses.

It is probably overkill but we should be able to solve this with by adding "-mcmodel=large" to the build of purgatory.

it kill some...

still have left.... looks they from .S

yhlu@linux-siqj:~/xx/xx/utils/kexec-tools> readelf --relocs purgatory/purgatory.ro | grep R_X86_64_32
0000000005e9 00020000000b R_X86_64_32S 0000000000000000 .rodata + c0

Quoted 20 lines · Yinghai Lu

Quoted 8 lines · Eric W. Biederman

Quoted 7 lines · Yinghai Lu

there is lots of R_X86_64_32 and R_X86_64_32S for 64bit purgatory.

those come from global variables...could kill some by converting them static...

but still have some global string or ro data....

Quoted 7 lines

Quoted 1 line

build one big file include all .S ?

For R_x86_64_32 and R_x86_64_32S the problem is that the instructions are using absolute 32bit addresses.

It is probably overkill but we should be able to solve this with by adding "-mcmodel=large" to the build of purgatory.

it kill some...

Quoted 5 lines

still have left.... looks they from .S

yhlu@linux-siqj:~/xx/xx/utils/kexec-tools> readelf --relocs purgatory/purgatory.ro | grep R_X86_64_32
0000000005e9 00020000000b R_X86_64_32S 0000000000000000 .rodata + c0

looks like .S did not get that -mcmodel=large applied..

Quoted 23 lines · Yinghai Lu

Quoted 22 lines · Yinghai Lu

Quoted 8 lines · Eric W. Biederman

Quoted 7 lines · Yinghai Lu

there is lots of R_X86_64_32 and R_X86_64_32S for 64bit purgatory.

those come from global variables...could kill some by converting them static...

but still have some global string or ro data....

Quoted 7 lines

Quoted 1 line

build one big file include all .S ?

For R_x86_64_32 and R_x86_64_32S the problem is that the instructions are using absolute 32bit addresses.

It is probably overkill but we should be able to solve this with by adding "-mcmodel=large" to the build of purgatory.

it kill some...

still have left.... looks they from .S

Quoted 5 lines

Quoted 3 lines

yhlu@linux-siqj:~/xx/xx/utils/kexec-tools> readelf --relocs purgatory/purgatory.ro | grep R_X86_64_32
0000000005e9 00020000000b R_X86_64_32S 0000000000000000 .rodata + c0

looks like .S did not get that -mcmodel=large applied..

so -mcmodel=large only work with .c, but does not have effects on .S ?

Quoted 24 lines · Yinghai Lu

Quoted 23 lines · Yinghai Lu

Quoted 22 lines · Yinghai Lu

Quoted 8 lines · Eric W. Biederman

Quoted 7 lines · Yinghai Lu

there is lots of R_X86_64_32 and R_X86_64_32S for 64bit purgatory.

those come from global variables...could kill some by converting them static...

but still have some global string or ro data....

Quoted 7 lines

Quoted 1 line

build one big file include all .S ?

For R_x86_64_32 and R_x86_64_32S the problem is that the instructions are using absolute 32bit addresses.

It is probably overkill but we should be able to solve this with by adding "-mcmodel=large" to the build of purgatory.

it kill some...

still have left.... looks they from .S

Quoted 7 lines

Quoted 5 lines

Quoted 3 lines

yhlu@linux-siqj:~/xx/xx/utils/kexec-tools> readelf --relocs purgatory/purgatory.ro | grep R_X86_64_32
0000000005e9 00020000000b R_X86_64_32S 0000000000000000 .rodata + c0

looks like .S did not get that -mcmodel=large applied..

so -mcmodel=large only work with .c, but does not have effects on .S ?

Yes it -mcmodel=large is about which instructions you generate. The instructions used determine the relocates.

Just for playing with it the following patch modifies things so purgatory is 64bit clean.

I don't know yet what to do with the 32bit and 16bit assembly.

Eric

purgatory/Makefile +1 -0

index ee1679c..e39adec 100644
@@ -64,6 +64,7 @@ $(PURGATORY): $(PURGATORY_OBJS)
        $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^
 
 #      $(LD) $(LDFLAGS) $(EXTRA_LDFLAGS) --no-undefined -e purgatory_start -r -o $@ $(PURGATORY_OBJS) $(UTIL_LIB)
+       $(STRIP) --strip-debug $@
 
 echo::
        @echo "PURGATORY_SRCS $(PURGATORY_SRCS)"

purgatory/arch/x86_64/Makefile +5 -3

index 22b4228..2a768c6 100644
@@ -2,7 +2,7 @@
 # Purgatory x86_64
 #
 
-x86_64_PURGATORY_SRCS_native = purgatory/arch/x86_64/entry64-32.S
+#x86_64_PURGATORY_SRCS_native = purgatory/arch/x86_64/entry64-32.S
 x86_64_PURGATORY_SRCS_native += purgatory/arch/x86_64/entry64.S
 x86_64_PURGATORY_SRCS_native += purgatory/arch/x86_64/setup-x86_64.S
 x86_64_PURGATORY_SRCS_native += purgatory/arch/x86_64/stack.S

@@ -16,9 +16,11 @@ dist += purgatory/arch/x86_64/Makefile $(x86_64_PURGATORY_SRCS_native)       \
        purgatory/arch/x86_64/purgatory-x86_64.h
 
 # Don't add sources in i386/ to dist, as i386/Makefile adds them
-x86_64_PURGATORY_SRCS +=  purgatory/arch/i386/entry32-16.S
-x86_64_PURGATORY_SRCS += purgatory/arch/i386/entry32-16-debug.S
+#x86_64_PURGATORY_SRCS +=  purgatory/arch/i386/entry32-16.S
+#x86_64_PURGATORY_SRCS += purgatory/arch/i386/entry32-16-debug.S
 x86_64_PURGATORY_SRCS += purgatory/arch/i386/crashdump_backup.c
 x86_64_PURGATORY_SRCS += purgatory/arch/i386/console-x86.c
 x86_64_PURGATORY_SRCS += purgatory/arch/i386/vga.c
 x86_64_PURGATORY_SRCS += purgatory/arch/i386/pic.c
+
+x86_64_PURGATORY_EXTRA_CFLAGS = -mcmodel=large

purgatory/arch/x86_64/entry64.S +3 -2

index 666023c..e3223b7 100644
@@ -37,9 +37,10 @@ entry64:
        movl    %eax, %fs
        movl    %eax, %gs
 
-       movq    $stack_init, %rsp
+       leaq    stack_init(%rip), %rsp
        pushq   $0x10 /* CS */
-       pushq   $new_cs_exit
+       leaq    new_cs_exit(%rip), %rax
+       pushq   %rax
        lretq
 new_cs_exit:
 

purgatory/arch/x86_64/setup-x86_64.S +0 -0

index 74997fa..95572d8 100644
@@ -42,10 +42,10 @@ purgatory_start:
        /* In 64bit mode the code segment is meaningless */
 

Quoted 1 line · Yinghai Lu

so -mcmodel=large only work with .c, but does not have effects on .S ?

Yes it -mcmodel=large is about which instructions you generate. The instructions used determine the relocates.

Just for playing with it the following patch modifies things so purgatory is 64bit clean.

I don't know yet what to do with the 32bit and 16bit assembly.

Quoted 6 lines · Eric W. Biederman

Quoted 1 line · Yinghai Lu

so -mcmodel=large only work with .c, but does not have effects on .S ?

Yes it -mcmodel=large is about which instructions you generate. The instructions used determine the relocates.

Quoted 4 lines

Just for playing with it the following patch modifies things so purgatory is 64bit clean.

I don't know yet what to do with the 32bit and 16bit assembly.

so we should two purgatory for x86_64 ?
like one for 64bit and one for 32bit/16bit

Quoted 3 lines

so we should two purgatory for x86_64 ?
like one for 64bit and one for 32bit/16bit

The problem is that the current 32bit assembly code is not possition independent. If we can rewrite that assembly code to be position independent no relocs will be generated and we should be good.

The 16bit entry point code was already written as position independent code so it should not cause problems.

Just having one piece of code to deal with (if we can figure it out) looks to be the simpler and more maintainable solution.

Quoted 8 lines · Eric W. Biederman

Quoted 3 lines

so we should two purgatory for x86_64 ?
like one for 64bit and one for 32bit/16bit

The problem is that the current 32bit assembly code is not possition independent. If we can rewrite that assembly code to be position independent no relocs will be generated and we should be good.

Quoted 5 lines

The 16bit entry point code was already written as position independent code so it should not cause problems.

Just having one piece of code to deal with (if we can figure it out) looks to be the simpler and more maintainable solution.

sure.

Quoted 16 lines · Yinghai Lu

Quoted 13 lines · Eric W. Biederman

Quoted 3 lines

so we should two purgatory for x86_64 ?
like one for 64bit and one for 32bit/16bit

The problem is that the current 32bit assembly code is not possition independent. If we can rewrite that assembly code to be position independent no relocs will be generated and we should be good.

The 16bit entry point code was already written as position independent code so it should not cause problems.

Just having one piece of code to deal with (if we can figure it out) looks to be the simpler and more maintainable solution.

Quoted 1 line

sure.

just resent -v2 patches for kernel parts.

attached are changes for kexec-tools without put 64 bit purgartory above 4g...

hope you can work out 32bit position independent.

could return 0, and let the caller retry with new ranges.

Signed-off-by Yinghai Lu <[email protected]>

---
 kexec/kexec.c |    7 +++----
 1 files changed, 3 insertions(+), 4 deletions(-)

kexec/kexec.c +0 -0

index 8928be0..0f8aec8 100644
@@ -366,10 +366,9 @@ unsigned long add_buffer_phys_virt(struct kexec_info *info,
 	memsz = (memsz + (pagesize - 1)) & ~(pagesize - 1);
 
 	base = locate_hole(info, memsz, buf_align, buf_min, buf_max, buf_end);

only do that for 64bit bzImage, and will fall back to low if fail to get high.

Signed-off-by Yinghai Lu <[email protected]>

---
 kexec/arch/i386/x86-linux-setup.c |   30 ++++++++++++++++++++++--------
 1 files changed, 22 insertions(+), 8 deletions(-)

kexec/arch/i386/x86-linux-setup.c +16 -6

index 53d9df9..b0e6119 100644
@@ -69,20 +69,34 @@ void setup_linux_bootloader_parameters(
 	}
 
 	/* Load the initrd if we have one */
+	initrd_base = 0;
 	if (initrd_buf) {
-		initrd_base = add_buffer(info,
-			initrd_buf, initrd_size, initrd_size,
-			4096, INITRD_BASE, initrd_addr_max, -1);
+		if (real_mode->protocol_version >= 0x020c &&
+		    real_mode->code64_start_offset) {
+			initrd_base = add_buffer(info,
+				initrd_buf, initrd_size, initrd_size,
+				4096, 1UL<<32, ULONG_MAX, -1);
+			if (!initrd_base)
+				initrd_base = add_buffer(info,
+					initrd_buf, initrd_size, initrd_size,
+					4096, 1UL<<30, 1UL<<32, -1);
 
 
 
+		}
+		if (!initrd_base)
+			initrd_base = add_buffer(info,
+				initrd_buf, initrd_size, initrd_size,
+				4096, INITRD_BASE, initrd_addr_max, -1);
 		dbgprintf("Loaded initrd at 0x%lx size 0x%lx\n", initrd_base,
 			initrd_size);
-	} else {
-		initrd_base = 0;
+	} else
 		initrd_size = 0;
-	}

only do that for bzImage64, and it could have command line above 4g.

Signed-off-by Yinghai Lu <[email protected]>

---
 kexec/arch/i386/x86-linux-setup.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

kexec/arch/i386/x86-linux-setup.c +0 -0

index b0e6119..c929166 100644

need to make sure pass right 64bit start address to go there directly later.

(quoted diff) +0 -2

-v2: add kexec-bzImage64.c according to Eric.
-v3: don't need to purgatory under 2g after Eric's change to purgatory code.

Signed-off-by Yinghai Lu <[email protected]>

---
 kexec/arch/x86_64/Makefile          |    1 +
 kexec/arch/x86_64/kexec-bzImage64.c |  316 +++++++++++++++++++++++++++++++++++
 kexec/arch/x86_64/kexec-x86_64.c    |    1 +
 kexec/arch/x86_64/kexec-x86_64.h    |    5 +
 4 files changed, 323 insertions(+), 0 deletions(-)
 create mode 100644 kexec/arch/x86_64/kexec-bzImage64.c

kexec/arch/x86_64/Makefile +1 -0

index 405bdf5..1cf10f9 100644
@@ -13,6 +13,7 @@ x86_64_KEXEC_SRCS += kexec/arch/i386/crashdump-x86.c
 x86_64_KEXEC_SRCS_native =  kexec/arch/x86_64/kexec-x86_64.c
 x86_64_KEXEC_SRCS_native += kexec/arch/x86_64/kexec-elf-x86_64.c
 x86_64_KEXEC_SRCS_native += kexec/arch/x86_64/kexec-elf-rel-x86_64.c
+x86_64_KEXEC_SRCS_native += kexec/arch/x86_64/kexec-bzImage64.c
 
 x86_64_KEXEC_SRCS += $(x86_64_KEXEC_SRCS_native)
 

kexec/arch/x86_64/kexec-bzImage64.c new file +316 -0

new file mode 100644
index 0000000..6835ef6
@@ -0,0 +1,316 @@
+/*
+ * kexec: Linux boots Linux
+ *
+ * Copyright (C) 2003-2010  Eric Biederman ([email protected])
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation (version 2 of the License).
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#define _GNU_SOURCE
+#include <stddef.h>
+#include <stdio.h>
+#include <string.h>
+#include <limits.h>
+#include <stdlib.h>
+#include <errno.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <getopt.h>
+#include <elf.h>
+#include <boot/elf_boot.h>
+#include <ip_checksum.h>
+#include <x86/x86-linux.h>
+#include "../../kexec.h"
+#include "../../kexec-elf.h"
+#include "../../kexec-syscall.h"
+#include "kexec-x86_64.h"
+#include "../i386/x86-linux-setup.h"
+#include "../i386/crashdump-x86.h"
+#include <arch/options.h>
+
+static const int probe_debug = 0;
+
+int bzImage64_probe(const char *buf, off_t len)
+{
+	const struct x86_linux_header *header;
+	if ((uintmax_t)len < (uintmax_t)(2 * 512)) {
+		if (probe_debug) {
+			fprintf(stderr, "File is too short to be a bzImage!\n");
+		}
+		return -1;
+	}
+	header = (const struct x86_linux_header *)buf;
+	if (memcmp(header->header_magic, "HdrS", 4) != 0) {
+		if (probe_debug) {
+			fprintf(stderr, "Not a bzImage\n");
+		}
+		return -1;
+	}
+	if (header->boot_sector_magic != 0xAA55) {
+		if (probe_debug) {
+			fprintf(stderr, "No x86 boot sector present\n");
+		}
+		/* No x86 boot sector present */
+		return -1;
+	}
+	if (header->protocol_version < 0x020C) {
+		if (probe_debug) {
+			fprintf(stderr, "Must be at least protocol version 2.12\n");
+		}
+		/* Must be at least protocol version 2.12 */
+		return -1;
+	}
+	if ((header->loadflags & 1) == 0) {
+		if (probe_debug) {
+			fprintf(stderr, "zImage not a bzImage\n");
+		}
+		/* Not a bzImage */
+		return -1;
+	}
+	if (!header->code64_start_offset) {
+		if (probe_debug) {
+			fprintf(stderr, "Not a bzImage64\n");
+		}
+		/* Must have non zero offset */
+		return -1;
+	}
+	if (!header->relocatable_kernel) {
+		if (probe_debug) {
+			fprintf(stderr, "Not a relocatable bzImage64\n");
+		}
+		/* Must be relocatable */
+		return -1;
+	}
+	/* I've got a bzImage64 */
+	if (probe_debug) {
+		fprintf(stderr, "It's a relocatable bzImage64\n");
+	}
+	return 0;
+}
+
+void bzImage64_usage(void)
+{
+	printf(	"    --command-line=STRING Set the kernel command line to STRING.\n"
+		"    --append=STRING       Set the kernel command line to STRING.\n"
+		"    --reuse-cmdline       Use kernel command line from running system.\n"
+		"    --initrd=FILE         Use FILE as the kernel's initial ramdisk.\n"
+		"    --ramdisk=FILE        Use FILE as the kernel's initial ramdisk.\n"
+		);
+}
+
+static int do_bzImage64_load(struct kexec_info *info,
+	const char *kernel, off_t kernel_len,
+	const char *command_line, off_t command_line_len,
+	const char *initrd, off_t initrd_len)
+{
+	struct x86_linux_header setup_header;
+	struct x86_linux_param_header *real_mode;
+	int setup_sects;
+	size_t size;
+	int kern16_size;
+	unsigned long setup_base, setup_size;
+	struct entry64_regs regs64;
+	char *modified_cmdline;
+	unsigned long cmdline_end;
+	unsigned long code64_start_offset = 0;
+	unsigned long kernel64_load_addr = 0;
+
+	/*
+	 * Find out about the file I am about to load.
+	 */
+	if ((uintmax_t)kernel_len < (uintmax_t)(2 * 512))
+		return -1;
+
+	memcpy(&setup_header, kernel, sizeof(setup_header));
+	setup_sects = setup_header.setup_sects;
+	if (setup_sects == 0)
+		setup_sects = 4;
+
+	kern16_size = (setup_sects +1) *512;
+	if (kernel_len < kern16_size) {
+		fprintf(stderr, "BzImage truncated?\n");
+		return -1;
+	}
+
+	if ((uintmax_t)command_line_len > (uintmax_t)setup_header.cmdline_size) {
+		dbgprintf("Kernel command line too long for kernel!\n");
+		return -1;
+	}
+
+	dbgprintf("bzImage is relocatable\n");
+
+	code64_start_offset = setup_header.code64_start_offset;
+	dbgprintf("code64_start_offset: 0x%lx\n", code64_start_offset);
+
+	/* Need to append some command line parameters internally in case of
+	 * taking crash dumps.
+	 */
+	if (info->kexec_flags & (KEXEC_ON_CRASH | KEXEC_PRESERVE_CONTEXT)) {
+		modified_cmdline = xmalloc(COMMAND_LINE_SIZE);
+		memset((void *)modified_cmdline, 0, COMMAND_LINE_SIZE);
+		if (command_line) {
+			strncpy(modified_cmdline, command_line,
+					COMMAND_LINE_SIZE);
+			modified_cmdline[COMMAND_LINE_SIZE - 1] = '\0';
+		}
+
+		/* If panic kernel is being loaded, additional segments need
+		 * to be created. load_crashdump_segments will take care of
+		 * loading the segments as high in memory as possible, hence
+		 * in turn as away as possible from kernel to avoid being
+		 * stomped by the kernel.
+		 */
+		if (load_crashdump_segments(info, modified_cmdline, -1, 0) < 0)
+			return -1;
+
+		/* Use new command line buffer */
+		command_line = modified_cmdline;
+		command_line_len = strlen(command_line) +1;
+	}
+
+	/* x86_64 purgatory could be anywhere */
+	elf_rel_build_load(info, &info->rhdr, purgatory, purgatory_size,
+					0x3000, -1, -1, 0);
+	dbgprintf("Loaded purgatory at addr 0x%lx\n", info->rhdr.rel_addr);
+	/* The argument/parameter segment */
+	setup_size = kern16_size + command_line_len + PURGATORY_CMDLINE_SIZE;
+	real_mode = xmalloc(setup_size);
+	memcpy(real_mode, kernel, kern16_size);
+
+	/* No real mode code will be executing. setup segment can be loaded
+	 * anywhere as we will be just reading command line.
+	 */
+	setup_base = add_buffer(info, real_mode, setup_size, setup_size,
+				16, 0x3000, -1, -1);
+
+	dbgprintf("Loaded setup data and command line at 0x%lx\n",
+			setup_base);
+
+	/* The main kernel segment */
+	size = kernel_len - kern16_size;
+
+	/* align to 1G to avoid cross the PUD_SIZE boundary */
+	/* try above 4G at first */
+	kernel64_load_addr = add_buffer(info, kernel + kern16_size,
+					size, size, 1UL<<30,
+					1UL<<32, ULONG_MAX,
+					-1);
+
+	/* 1G to 4G */
+	if (!kernel64_load_addr)
+		kernel64_load_addr = add_buffer(info, kernel + kern16_size,
+						size, size, 1UL<<30,
+						1UL<<30, 1UL<<32,
+						-1);
+
+	/* under 1g */
+	if (!kernel64_load_addr)
+		kernel64_load_addr = add_buffer(info, kernel + kern16_size,
+						size, size,
+						real_mode->kernel_alignment,
+						0x100000, 1UL<<30,
+						1);
+	if (kernel64_load_addr)
+		dbgprintf("Loaded 64bit kernel at 0x%lx\n", kernel64_load_addr);
+	else
+		die("can not load bzImage64");
+
+	/* Tell the kernel what is going on */
+	setup_linux_bootloader_parameters(info, real_mode, setup_base,
+			kern16_size, command_line, command_line_len,
+			initrd, initrd_len);
+
+	elf_rel_get_symbol(&info->rhdr, "entry64_regs", &regs64, sizeof(regs64));
+	regs64.rbx = 0;           /* Bootstrap processor */
+	regs64.rsi = setup_base;  /* Pointer to the parameters */
+	regs64.rip = kernel64_load_addr + code64_start_offset; /* the entry point */
+	regs64.rsp = elf_rel_get_addr(&info->rhdr, "stack_end"); /* Stack, unused */
+	elf_rel_set_symbol(&info->rhdr, "entry64_regs", &regs64, sizeof(regs64));
+
+	cmdline_end = setup_base + kern16_size + command_line_len - 1;
+	elf_rel_set_symbol(&info->rhdr, "cmdline_end", &cmdline_end,
+			   sizeof(unsigned long));
+
+	/* Fill in the information BIOS calls would normally provide. */
+	setup_linux_system_parameters(real_mode, info->kexec_flags);
+
+	return 0;
+}
+
+int bzImage64_load(int argc, char **argv, const char *buf, off_t len,
+	struct kexec_info *info)
+{
+	char *command_line = NULL;
+	const char *ramdisk, *append = NULL;
+	char *ramdisk_buf;
+	off_t ramdisk_length;
+	int command_line_len;
+	int opt;
+	int result;
+
+	/* See options.h -- add any more there, too. */
+	static const struct option options[] = {
+		KEXEC_ARCH_OPTIONS
+		{ "command-line",	1, 0, OPT_APPEND },
+		{ "append",		1, 0, OPT_APPEND },
+		{ "reuse-cmdline",	0, 0, OPT_REUSE_CMDLINE },
+		{ "initrd",		1, 0, OPT_RAMDISK },
+		{ "ramdisk",		1, 0, OPT_RAMDISK },
+		{ 0,			0, 0, 0 },
+	};
+	static const char short_options[] = KEXEC_ARCH_OPT_STR "d";
+
+	ramdisk = 0;
+	ramdisk_length = 0;
+	while((opt = getopt_long(argc, argv, short_options, options, 0)) != -1) {
+		switch(opt) {
+		default:
+			/* Ignore core options */
+			if (opt < OPT_ARCH_MAX) {
+				break;
+			}
+		case '?':
+			usage();
+			return -1;
+			break;
+		case OPT_APPEND:
+			append = optarg;
+			break;
+		case OPT_REUSE_CMDLINE:
+			command_line = get_command_line();
+			break;
+		case OPT_RAMDISK:
+			ramdisk = optarg;
+			break;
+		}
+	}
+	command_line = concat_cmdline(command_line, append);
+	command_line_len = 0;
+	if (command_line) {
+		command_line_len = strlen(command_line) +1;
+	}
+	ramdisk_buf = 0;
+	if (ramdisk) {
+		ramdisk_buf = slurp_file(ramdisk, &ramdisk_length);
+	}
+	result = do_bzImage64_load(info,
+		buf, len,
+		command_line, command_line_len,
+		ramdisk_buf, ramdisk_length);
+
+	free(command_line);
+	return result;
+}

kexec/arch/x86_64/kexec-x86_64.c +1 -0

index 6c42c32..5c23e01 100644
@@ -37,6 +37,7 @@ struct file_type file_type[] = {
 	{ "multiboot-x86", multiboot_x86_probe, multiboot_x86_load,
 	  multiboot_x86_usage },
 	{ "elf-x86", elf_x86_probe, elf_x86_load, elf_x86_usage },
+	{ "bzImage64", bzImage64_probe, bzImage64_load, bzImage64_usage },
 	{ "bzImage", bzImage_probe, bzImage_load, bzImage_usage },
 	{ "beoboot-x86", beoboot_probe, beoboot_load, beoboot_usage },
 	{ "nbi-x86", nbi_probe, nbi_load, nbi_usage },

kexec/arch/x86_64/kexec-x86_64.h +0 -0

index a97cd71..b820ae8 100644

Now we have limit kdump reserved under 896M, because kexec has the limitation. and also bzImage need to stay under 4g.

Quoted 7 lines · Yinghai Lu

Quoted 5 lines · Eric W. Biederman

Quoted 4 lines · Yinghai Lu

there is lots of R_X86_64_32 and R_X86_64_32S for 64bit purgatory.

those come from global variables...could kill some by converting

them static...

Quoted 6 lines

Quoted 6 lines

Quoted 4 lines

but still have some global string or ro data....

build one big file include all .S ?

For R_x86_64_32 and R_x86_64_32S the problem is that the

instructions

Quoted 5 lines

Quoted 4 lines

are using absolute 32bit addresses.

It is probably overkill but we should be able to solve this with by adding "-mcmodel=large" to the build of purgatory.

Quoted 7 lines

it kill some...

still have left.... looks they from .S

yhlu@linux-siqj:~/xx/xx/utils/kexec-tools> readelf --relocs purgatory/purgatory.ro | grep R_X86_64_32
0000000005e9 00020000000b R_X86_64_32S 0000000000000000 .rodata

+ c0

looks like .S did not get that -mcmodel=large applied..

Quoted 6 lines · Yinghai Lu

Quoted 4 lines · Eric W. Biederman

"H. Peter Anvin" <[email protected]> writes:

64bit purgatory coming from kexec should be running with a page

table

Quoted 6 lines

Quoted 2 lines

that identity maps everything loaded by kexec and in practice all of memory.

there is lots of R_X86_64_32 and R_X86_64_32S for 64bit purgatory.

those come from global variables...could kill some by converting them

static...

Quoted 4 lines

but still have some global string or ro data....

build one big file include all .S ?

For R_x86_64_32 and R_x86_64_32S the problem is that the instructions are using absolute 32bit addresses.

It is probably overkill but we should be able to solve this with by adding "-mcmodel=large" to the build of purgatory.

Hopefully there are not 32bit or 16bit assembly routines getting linked in and using problemenatic instructions.

Eric

I think you just need one of the PIC models, or the medium model.

also need to make sure pass right 64bit start address to go there directly later.

Signed-off-by Yinghai Lu <[email protected]>

---
 kexec/arch/i386/kexec-bzImage.c |   53 ++++++++++++++++++++++++++++++++++++--
 1 files changed, 50 insertions(+), 3 deletions(-)

kexec/arch/i386/kexec-bzImage.c +48 -3

index 6998587..3e705ca 100644
@@ -18,8 +18,10 @@
  */
 
 #define _GNU_SOURCE
+#include <stddef.h>
 #include <stdio.h>
 #include <string.h>
+#include <limits.h>
 #include <stdlib.h>
 #include <errno.h>
 #include <sys/types.h>

@@ -35,6 +37,7 @@
 #include "../../kexec-elf.h"
 #include "../../kexec-syscall.h"
 #include "kexec-x86.h"
+#include "../x86_64/kexec-x86_64.h"
 #include "x86-linux-setup.h"
 #include "crashdump-x86.h"
 #include <arch/options.h>

@@ -111,12 +114,15 @@ int do_bzImage_load(struct kexec_info *info,
 	size_t size;
 	int kern16_size;
 	unsigned long setup_base, setup_size;
+	struct entry64_regs regs64;
 	struct entry32_regs regs32;
 	struct entry16_regs regs16;
 	unsigned int relocatable_kernel = 0;
 	unsigned long kernel32_load_addr;
 	char *modified_cmdline;
 	unsigned long cmdline_end;
+	unsigned long code64_start_offset = 0;
+	unsigned long kernel64_load_addr = 0;
 
 	/*
 	 * Find out about the file I am about to load.

@@ -154,6 +160,13 @@ int do_bzImage_load(struct kexec_info *info,
 		dbgprintf("bzImage is relocatable\n");
 	}
 
+	if (setup_header.protocol_version >= 0x020C) {
+		code64_start_offset = setup_header.code64_start_offset;
+		if (code64_start_offset)
+			dbgprintf("code64_start_offset: 0x%lx\n",
+					 code64_start_offset);
+	}
+
 	/* Can't use bzImage for crash dump purposes with real mode entry */
 	if((info->kexec_flags & KEXEC_ON_CRASH) && real_mode_entry) {
 		fprintf(stderr, "Can't use bzImage for crash dump purposes"

@@ -250,7 +263,26 @@ int do_bzImage_load(struct kexec_info *info,
 				kernel32_max_addr = real_mode->initrd_addr_max;
 		}
 
-		kernel32_load_addr = add_buffer(info, kernel + kern16_size,
+		if (!real_mode_entry && code64_start_offset) {
+			/* align to 1G to avoid cross the PUD_SIZE boundary */
+			kernel64_load_addr = add_buffer(
+						info, kernel + kern16_size,
+						size, size, 1UL<<30,
+						1UL<<32, ULONG_MAX,
+						-1);
+			if (!kernel64_load_addr)
+				kernel64_load_addr = add_buffer(
+						info, kernel + kern16_size,
+						size, size, 1UL<<30,
+						1UL<<30, 1UL<<32,
+						-1);
+			if (kernel64_load_addr)
+				kernel64_load_addr += code64_start_offset;
+		}
+
+		if (!kernel64_load_addr)
+			kernel32_load_addr = add_buffer(
+						info, kernel + kern16_size,
 						size, size, kern_align,
 						0x100000, kernel32_max_addr,
 						1);

@@ -260,8 +292,11 @@ int do_bzImage_load(struct kexec_info *info,
 		add_segment(info, kernel + kern16_size, size,
 				kernel32_load_addr, size);
 	}
-		
-	dbgprintf("Loaded 32bit kernel at 0x%lx\n", kernel32_load_addr);
+
+	if (kernel64_load_addr)
+		dbgprintf("Loaded 64bit kernel at 0x%lx\n", kernel64_load_addr);
+	else
+		dbgprintf("Loaded 32bit kernel at 0x%lx\n", kernel32_load_addr);
 
 	/* Tell the kernel what is going on */
 	setup_linux_bootloader_parameters(info, real_mode, setup_base,

@@ -271,6 +306,16 @@ int do_bzImage_load(struct kexec_info *info,
 	/* Get the initial register values */
 	elf_rel_get_symbol(&info->rhdr, "entry16_regs", &regs16, sizeof(regs16));
 	elf_rel_get_symbol(&info->rhdr, "entry32_regs", &regs32, sizeof(regs32));
+	if (kernel64_load_addr) {
+		elf_rel_get_symbol(&info->rhdr, "entry64_regs", &regs64, sizeof(regs64));
+		regs64.rbx = 0;           /* Bootstrap processor */
+		regs64.rsi = setup_base;  /* Pointer to the parameters */
+		regs64.rip = kernel64_load_addr; /* the entry point */
+		regs64.rsp = elf_rel_get_addr(&info->rhdr, "stack_end"); /* Stack, unused */
+		elf_rel_set_symbol(&info->rhdr, "entry64_regs", &regs64, sizeof(regs64));
+
+		goto cmd_line;
+	}
 	/*
 
 	 * Initialize the 32bit start information.

Quoted 5 lines · Eric W. Biederman

Quoted 2 lines · Yinghai Lu

also need to make sure pass right 64bit start address to go there directly later.

Quoted 6 lines

There are some silly things here but I think the clean way to handle this is to create a kexec/arch/i386/kexec-bzImage64.c that handles loading a bzImage with a 64bit entry point.

That should keep the code simpler and easier to read. Although it will probably introduce a bit more code.

please check attached one with kexec-bzImage64.c

Now we have limit kdump reseved under 896M, because kexec has the limitation. and also bzImage need to stay under 4g.

kernel parts changes could be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-x86-boot

here patches are for kexec tools to load bzImage and ramdisk high acccording to new added boot header fields.

Yinghai Lu (8):
Add min/max macro
x86: add boot header member for version 2.12
add mem64_min/max control
Move out mem_min/max checking in locate_hole
seperate checking 64bit mem range
debug print out for add_buf
x86: put ramdisk high for 64bit bzImage
x86: put 64bit bzImage high

 include/x86/x86-linux.h           |   20 ++++++++++-
 kexec/add_buffer.c                |    9 ++++-
 kexec/arch/i386/kexec-bzImage.c   |   53 ++++++++++++++++++++++++++--
 kexec/arch/i386/x86-linux-setup.c |   32 ++++++++++++-----
 kexec/arch/ppc/include/types.h    |   12 ------
 kexec/kexec.c                     |   69 +++++++++++++++++++++++++++++-------
 kexec/kexec.h                     |   21 +++++++++++-
 7 files changed, 175 insertions(+), 41 deletions(-)

Signature · 1 line

1.7.7

Quoted 2 lines

here patches are for kexec tools to load bzImage and ramdisk high acccording to new added boot header fields.

So does this allow me to load other segments also above 4G? (purgatory, setup/bootparams, command line etc). I remember that purgaotry has some 32bit relocations so had to be loaded below 2G.