Elf loader fixes
PATCH 1/1 Elf loader fixes
Hi folks,
The Xen ELF kernel loader is quite quirky wrt. physical and virtual addresses, probably for historical reasons, linux got that wrong too until very recently (kexec merge in 2.6.14 or so). The patch below fixes that.
Changes:
* Fix linux kernel ELF entry point (also submitted to lkml)
* Drop LOAD_OFFSET re-#define hack in xen headers.
* Fix both dom0 and libxc elf loaders.
* add quick mode so loading old linux kernels doesn't break.
Linux-wise everything should be OK with that, but it might break other OS'es which also use the ELF loader (in case they create bug-compatible ELF headers with broken paddr entries ...).
please apply,
Gerd
linux-2.6-xen-sparse/arch/i386/kernel/vmlinux.lds.S +1 -1
@@ -10,7 +10,7 @@ OUTPUT_FORMAT("elf32-i386", "elf32-i386", "elf32-i386") OUTPUT_ARCH(i386) -ENTRY(phys_startup_32) +ENTRY(startup_32) jiffies = jiffies_64; SECTIONS {
linux-2.6-xen-sparse/include/asm-i386/mach-xen/asm/page.h +0 -4
@@ -288,10 +288,6 @@#endif #define __KERNEL_START (__PAGE_OFFSET + __PHYSICAL_START) -#undef LOAD_OFFSET -#define LOAD_OFFSET 0 - -#define PAGE_OFFSET ((unsigned long)__PAGE_OFFSET)#define VMALLOC_RESERVE ((unsigned long)__VMALLOC_RESERVE)#define MAXMEM (HYPERVISOR_VIRT_START-__PAGE_OFFSET-__VMALLOC_RESERVE)
tools/libxc/xc_load_elf.c +22 -6
@@ -138,10 +138,10 @@ phdr = (Elf_Phdr *)(image + ehdr->e_phoff + (h*ehdr->e_phentsize));if ( !is_loadable_phdr (phdr) )continue ; -if ( phdr->p_paddr < kernstart ) - kernstart = phdr->p_paddr; -if ( (phdr->p_paddr + phdr->p_memsz) > kernend ) - kernend = phdr->p_paddr + phdr->p_memsz; +if ( phdr->p_vaddr < kernstart ) + kernstart = phdr->p_vaddr; +if ( (phdr->p_vaddr + phdr->p_memsz) > kernend ) + kernend = phdr->p_vaddr + phdr->p_memsz; } if ( (kernstart > kernend) ||
@@ -189,7 +189,18 @@for ( done =0 ; done < phdr->p_filesz; done += chunksz ) { - pa = (phdr->p_paddr + done) - dsi->v_start; +if (phdr->p_paddr == phdr->p_vaddr) { +/* +* Bug compatibility alert: In older linux kernels +* p_paddr is broken, it doesn't contain the physical +* address but instead is identical to p_vaddr. Thus +* we can't use it directly, instead we'll guess it +* using dsi->v_start. +*/ + pa = (phdr->p_vaddr + done) - dsi->v_start; + }else { + pa = (phdr->p_paddr + done); + } va =xc_map_foreign_range ( xch, dom,PAGE_SIZE ,PROT_WRITE , parray[pa>>PAGE_SHIFT ]); chunksz = phdr->p_filesz - done;
@@ -202,7 +213,12 @@for ( ; done < phdr->p_memsz; done += chunksz ) { - pa = (phdr->p_paddr + done) - dsi->v_start; +if (phdr->p_paddr == phdr->p_vaddr) { +/* bug compatibility alert, see above */ + pa = (phdr->p_vaddr + done) - dsi->v_start; + }else { + pa = (phdr->p_paddr + done); + } va =xc_map_foreign_range ( xch, dom,PAGE_SIZE ,PROT_WRITE , parray[pa>>PAGE_SHIFT ]); chunksz = phdr->p_memsz - done;
xen/arch/x86/domain.c +1 -1
@@ -346,7 +346,7 @@ struct vcpu *v, struct vcpu_guest_context *c) { struct domain *d = v->domain; -unsigned long phys_basetab; +unsigned long phys_basetab =0 ;int i, rc;/*
xen/common/elf.c +42 -15
90 lines not on this page · raw message
xen/include/xen/sched.h +2 -0
@@ -165,6 +165,8 @@unsigned long v_end;unsigned long v_kernstart;unsigned long v_kernend; +unsigned long p_kernstart; +unsigned long p_kernend;unsigned long v_kernentry;/* Initialised by loader: Private. */ unsigned int load_symtab;
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
I'm afraid this won't work for 64-bits, because of the way the vsyscall page gets set up there. If you dump the program headers you'll see that the respective segment is at an address close to the top of the address space, which would make the image appear to need nearly 2 Gb of memory (making the load fail if there's less than that available to the domain).
Also, I'm unclear why you need to make persistent the physical address information for the dom0 case, and hence why the bug compatibility hack is needed there.
Jan
Quoted 1 line
Quoted 1 line
Quoted 1 line
Gerd Hoffmann <[email protected]> 22.02.06 12:37:26 >>>
Hi folks,
The Xen ELF kernel loader is quite quirky wrt. physical and virtual addresses, probably for historical reasons, linux got that wrong too until very recently (kexec merge in 2.6.14 or so). The patch below fixes that.
Changes:
* Fix linux kernel ELF entry point (also submitted to lkml)
* Drop LOAD_OFFSET re-#define hack in xen headers.
* Fix both dom0 and libxc elf loaders.
* add quick mode so loading old linux kernels doesn't break.
Linux-wise everything should be OK with that, but it might break other OS'es which also use the ELF loader (in case they create bug-compatible ELF headers with broken paddr entries ...).
please apply,
Gerd
Quoted 4 lines · Jan Beulich
I'm afraid this won't work for 64-bits, because of the way the vsyscall page gets set up there. If you dump the program headers you'll see that the respective segment is at an address close to the top of the address space, which would make the image appear to need nearly 2 Gb of memory (making the load fail if there's less than that available to the domain).
Yep, will not work work as-is, the domain builder needs more fixes for that.
Quoted 2 lines
Also, I'm unclear why you need to make persistent the physical address information for the dom0 case, and hence why the bug compatibility hack is needed there.
Well, the libxc loader needs it to be able to load both old kernels and kernels with correct physical addresses in the phdr.
The dom0 loader doesn't really need it, at least on x32, right now it doesn't look at the physical addresses anyway. Instead it depends on the VIRT_BASE hack to get it right. It's sort of documentation only ;)
For x64 it will certainly be needed to get the memory size calculations right (use physical not virtual addresses) and fix the issue you've outlined above that way.
cheers,
Gerd
Signature · 3 lines
Gerd 'just married' Hoffmann <[email protected]> I'm the hacker formerly known as Gerd Knorr. http://www.suse.de/~kraxel/just-married.jpeg
Quoted 4 lines · Jan Beulich
I'm afraid this won't work for 64-bits, because of the way the vsyscall page gets set up there. If you dump the program headers you'll see that the respective segment is at an address close to the top of the address space, which would make the image appear to need nearly 2 Gb of memory (making the load fail if there's less than that available to the domain).
Hmm, who sets up the vsyscall mapping right now? The linux kernel I guess? The elf kernel loader certainly doesn't look like it cares much about virtual address (other than VIRT_BASE) ...
cheers,
Gerd
Signature · 3 lines
Gerd 'just married' Hoffmann <[email protected]> I'm the hacker formerly known as Gerd Knorr. http://www.suse.de/~kraxel/just-married.jpeg
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Gerd Hoffmann <[email protected]> 22.02.06 16:12:25 >>>
Quoted 1 line · Jan Beulich
I'm afraid this won't work for 64-bits, because of the way the vsyscall page gets set up there. If you dump the
program
Quoted 1 line
Quoted 1 line
headers you'll see that the respective segment is at an address close to the top of the address space, which would
make
Quoted 5 lines
Quoted 2 lines
the image appear to need nearly 2 Gb of memory (making the load fail if there's less than that available to the domain).
Hmm, who sets up the vsyscall mapping right now? The linux kernel I guess?
Yes.
Quoted 2 lines
The elf kernel loader certainly doesn't look like it cares much about virtual address (other than
VIRT_BASE) ...
Correct. And the same applies to boot loaders.
Jan
Quoted 4 lines
Quoted 2 lines
The elf kernel loader certainly doesn't look like it cares much about virtual address (other than
VIRT_BASE) ...Correct. And the same applies to boot loaders.
Well, the difference between xen and other boot loaders is that xen boots the kernel with paging already enabled ...
Right now it's completely broken: the loader works only with virtual addresses in the paddr header field. It can't stay that way, especially if we'll want to have kexec working with xenlinux kernels some day ;)
As I see things there are basically two ways to fixup this: Either create a simple linear mapping using paddr + VIRT_BASE, which would be pretty close to the current behaviour (and other boot loaders). Or do a complete virtual memory setup, which probably can't be done without major changes in the domain builder ...
cheers,
Gerd
Signature · 3 lines
Gerd 'just married' Hoffmann <[email protected]> I'm the hacker formerly known as Gerd Knorr. http://www.suse.de/~kraxel/just-married.jpeg
Quoted 8 lines · Gerd Hoffmann
Quoted 4 lines
Quoted 2 lines
The elf kernel loader certainly doesn't look like it cares much about virtual address (other than
VIRT_BASE) ...Correct. And the same applies to boot loaders.
Well, the difference between xen and other boot loaders is that xen boots the kernel with paging already enabled ...
actually, there have historically been a ton of boot loaders that boot with paging already enabled: just go back to the early Suns; part of the task of SunOS was to move the firmware's page tables to kernel space and use them until it could build its own.
I hit this when I ported sunos to a machine with firmware that did not set up page tables.
it's been very common, in the non-PC world, to have paging turned on very early.
Just a historical comment of no value to this discussion :-)
ron
Quoted 5 lines · Gerd Hoffmann
As I see things there are basically two ways to fixup this: Either create a simple linear mapping using paddr +
VIRT_BASE, which would be pretty close to the current behaviour (and other boot loaders). Or do a complete virtual memory setup, which probably can't be done without major changes in the domain builder ...
hi,
as previously mentioned here and at the XenSummit, I have a simple yet functioning domU bootloader which defers ELF-decoding to domU-space. It currently only works when the input is a ramdisk image, created with a small tool called 'pack. It used to work from a TCP-connection (provided by the fabulous UIP which is still in my tree), hence the weird state-machine look of the code.
I suppose taking this approach could solve all our domU-building woes for good. It should also provide a more elegant solution to the attestation problems the Intel guys were talking about and trying to solve with their two-stage domain building proposal.
Clone my tree at http://www.distlab.dk/hg/index.cgi/xen-gfx.hg and have a look at the extras/mstrap subdir. The 'pack' tool is in tools/migrate.
You will need Jam to build, or you have to roll your own Makefile. The tree also contains my very experimental 'Blink' display system as demoed at the Summit in tools/gfx, and my self-migration patch to XenLinux which at the moment is mostly useful for self-checkpointing to disk (see tools/minimig.c). Your mileage may vary.
Jacob