kexec for linux 2.6.6
kexec for Linux 2.6.6 is now available at:
http://developer.osdl.org/rddunlap/kexec/2.6.6/
kexec userspace tools are at:
http://developer.osdl.org/rddunlap/kexec/kexec-tools/
For 2.6.6, the kexec_load syscall number had to move due to some new syscall additions in 2.6.6. However, I didn't update the kexec userspace program for that 1-line change. The change is in kexec-syscall.c, line 46:
change
#define __NR_kexec_load 274
to
#define __NR_kexec_load 283
(for i386).
There is one outstanding patch from Albert Herranz that I will review and possibly use/merge soon. His email is here:
http://lists.osdl.org/pipermail/fastboot/2004-May/000290.html
And if anyone has suggestions for handling a variable/moving syscall number (target), I'm interested in hearing them.
--
~Randy
Quoted 2 lines · Randy.Dunlap
And if anyone has suggestions for handling a variable/moving syscall number (target), I'm interested in hearing them.
If all architectures would finally get a vdso implementation you could just add the necessary stub in the vdso, add a symbol in the symbol table of the vdso, and use in the userlevel code
sym = dlsym (RTLD_DEFAULT, "the_symbol_name")
If the returned value is not NULL the symbol exists.
I've described this many times as one of the huge advantages of vdsos, hopefully this time it clicks.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Quoted 9 lines · Ulrich Drepper
Quoted 2 lines · Randy.Dunlap
And if anyone has suggestions for handling a variable/moving syscall number (target), I'm interested in hearing them.
If all architectures would finally get a vdso implementation you could just add the necessary stub in the vdso, add a symbol in the symbol table of the vdso, and use in the userlevel code
Quoted 6 lines
sym = dlsym (
RTLD_DEFAULT, "the_symbol_name")If the returned value is not NULL the symbol exists.
I've described this many times as one of the huge advantages of vdsos, hopefully this time it clicks.
For the momen the only finished port is x86, so we should be able to do that, it would make the kernel patch a little bigger though. Last time I saw that conversation I thought you didn't like symbols in the vdso for syscalls because it slowed things down.
If no one is opposed that sounds like a fairly sane idea.
Eric
Quoted 8 lines · Eric W. Biederman
Quoted 1 line
sym = dlsym (
RTLD_DEFAULT, "the_symbol_name")[...]
For the momen the only finished port is x86, so we should be able to do that, it would make the kernel patch a little bigger though. Last time I saw that conversation I thought you didn't like symbols in the vdso for syscalls because it slowed things down.
I don't want to use this in glibc for every syscall. But for your random application in need of a syscall it's fine.
And there is one more thing: the above code is actually not what should be used. The symbol able entries should be position independent. So one will have to compute the final address (which will be fun for archs with function descriptors). I'll have to see how randomization is actually implemented. The __kernel_vsyscall symbol is probably not changed, so we need an out-of-band mechanisms to report the load address to the userlevel code.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Quoted 7 lines · Ulrich Drepper
Quoted 4 lines · Eric W. Biederman
Quoted 1 line
sym = dlsym (
RTLD_DEFAULT, "the_symbol_name")[...]
Quoted 7 lines
Quoted 4 lines
For the momen the only finished port is x86, so we should be able to do that, it would make the kernel patch a little bigger though. Last time I saw that conversation I thought you didn't like symbols in the vdso for syscalls because it slowed things down.
I don't want to use this in glibc for every syscall. But for your random application in need of a syscall it's fine.
The question that had come up earlier was fast path syscalls like gettimeofday.
Quoted 7 lines
And there is one more thing: the above code is actually not what should be used. The symbol able entries should be position independent. So one will have to compute the final address (which will be fun for archs with function descriptors). I'll have to see how randomization is actually implemented. The
__kernel_vsyscallsymbol is probably not changed, so we need an out-of-band mechanisms to report the load address to the userlevel code.
We currently have AT_SYSINFO_EHDR and AT_SYSINFO which should report the basic location information.
As a first draft we should be able to use the standard ELF mechanisms
for this. It is not like PIC shared libraries were new. Or is
there some specific problem you are thinking of with respect to randomization?
Eric
Quoted 4 lines · Eric W. Biederman
As a first draft we should be able to use the standard ELF mechanisms
for this. It is not like PIC shared libraries were new. Or is
there some specific problem you are thinking of with respect to randomization?
The official kernel does not have vdso randomization. Ingo has a patch for the Red Hat kernel which is used in the FC2 kernel. The patch effectively only changes the location at which the vdso is mapped. It does not change the vdso content. So the __kernel_vsyscall symbol in the vdso's symbol table is not changed.
AT_SYSINFO is the right way to go forward but it is not directly accessible to userlevel code. And it is no pointer which will make architectures with function descriptors unhappy.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Quoted 13 lines · Ulrich Drepper
Quoted 4 lines · Eric W. Biederman
As a first draft we should be able to use the standard ELF mechanisms
for this. It is not like PIC shared libraries were new. Or is
there some specific problem you are thinking of with respect to randomization?The official kernel does not have vdso randomization. Ingo has a patch for the Red Hat kernel which is used in the FC2 kernel. The patch effectively only changes the location at which the vdso is mapped. It does not change the vdso content. So the
__kernel_vsyscallsymbol in the vdso's symbol table is not changed.
Quoted 3 lines
AT_SYSINFOis the right way to go forward but it is not directly accessible to userlevel code. And it is no pointer which will make architectures with function descriptors unhappy.
It sounds like the vdso just needs to be treated as a prelinked vdso. You can find everything you need with AT_SYSINFO_EHDR.
In the case of function descriptors they should be in a data segment that can get copied to another page, and corrected. Leaving the code segment at it's randomized location.
Eric
On 12 May 2004 10:57:27 -0600 Eric W. Biederman wrote:
| Ulrich Drepper <[email protected]> writes: | | > Eric W. Biederman wrote: | > | > > As a first draft we should be able to use the standard ELF mechanisms | > > for this. It is not like PIC shared libraries were new. Or is | > > there some specific problem you are thinking of with respect to | > > randomization? | > | > The official kernel does not have vdso randomization. Ingo has a patch | > for the Red Hat kernel which is used in the FC2 kernel. The patch | > effectively only changes the location at which the vdso is mapped. It | > does not change the vdso content. So the __kernel_vsyscall symbol in | > the vdso's symbol table is not changed.
[1:]
| > AT_SYSINFO is the right way to go forward but it is not directly | > accessible to userlevel code. And it is no pointer which will make | > architectures with function descriptors unhappy. | | It sounds like the vdso just needs to be treated as a prelinked | vdso. You can find everything you need with AT_SYSINFO_EHDR. | | In the case of function descriptors they should be in a data segment | that can get copied to another page, and corrected. Leaving the code | segment at it's randomized location.
Andrew tells me that he is OK with reserving a syscall number for kexec, which is easy & quick. I don't know when vdso will be available (for non-x86[2]) or when the AT_SYSINFO data will work for userlevel code[1. above].
So is there any reason not to reserve the syscall number for kexec for now? (patch is below)
--
~Randy
[2] kexec is currently only available for x86, but there is interest in it for ia64 and ppc64 at least.
arch/i386/kernel/entry.S +1 -0
@@ -891,5 +891,6 @@ ENTRY(sys_call_table) .long sys_mq_timedreceive /* 280 */ .long sys_mq_notify .long sys_mq_getsetattr + .long sys_ni_syscall /* reserved for kexec */ syscall_table_size=(.-sys_call_table)
include/asm-i386/unistd.h +2 -1
@@ -288,8 +288,9 @@#define __NR_mq_timedreceive (__NR_mq_open+3)#define __NR_mq_notify (__NR_mq_open+4)#define __NR_mq_getsetattr (__NR_mq_open+5) +#define __NR_sys_kexec_load 283 -#define NR_syscalls 283 +#define NR_syscalls 284/* user-visible error numbers are in the range -1 - -124: see <asm-i386/errno.h> */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Randy.Dunlap
Randy> I don't know when vdso will be available (for non-x86[2]) or
Randy> [2] kexec is currently only available for x86, but there is interest Randy> in it for ia64 and ppc64 at least.
ia64 does have VDSO (and has had it for some time).
I quite like Uli's idea.
--david
Quoted 12 lines · David Mosberger
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Randy.Dunlap
Randy> I don't know when vdso will be available (for non-x86[2]) or
Randy> [2] kexec is currently only available for x86, but there is interest Randy> in it for ia64 and ppc64 at least.
ia64 does have VDSO (and has had it for some time).
I quite like Uli's idea.
Is anyone doing VDSO development for x86? I don't recall seeing anything.
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line · Andrew Morton
ia64 does have VDSO (and has had it for some time).
I quite like Uli's idea.
Andrew> Is anyone doing VDSO development for x86? I don't recall Andrew> seeing anything.
It's already there?
--david
$ tail -17 arch/i386/kernel/vsyscall.lds
/*
* This controls what symbols we export from the DSO.
*/
VERSION
{
LINUX_2.5 {
global:
__kernel_vsyscall;
__kernel_sigreturn;
__kernel_rt_sigreturn;local: *;
};
}
/* The ELF entry point can be used to set the AT_SYSINFO value. */ ENTRY(__kernel_vsyscall);
Quoted 11 lines · David Mosberger
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Andrew Morton
ia64 does have VDSO (and has had it for some time).
I quite like Uli's idea.
Andrew> Is anyone doing VDSO development for x86? I don't recall Andrew> seeing anything.
It's already there?
But it needs work for the kexec requirement. If that work is purely the addition of the kexec entry point then OK, that can be part of the kexec patch.
But if we need additional infrastructure to "add new syscalls via VDSO" then this should be in the base kernel, even if it's empty, yes?
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Andrew Morton
Andrew> But it needs work for the kexec requirement.
Sorry, I wasn't suggesting that kexec should be done that way _now_.
I don't think it's nearly as simple as just adding an entry point in the DSO. For example, how would strace be affected by the new entry points? The syscall entry path probably would have to be almost duplicated to do a syscall-number-free entry. What about syscall restarting? Would VDSO entry points work with statically linked binaries? etc.
Perhaps Uli has answers to all of these, but no matter how you do it, it'd be a pretty dramatic shift in how applications interact with the kernel.
--david
Quoted 2 lines · Andrew Morton
But if we need additional infrastructure to "add new syscalls via VDSO" then this should be in the base kernel, even if it's empty, yes?
Linus has vetoed dynamic syscall registration a few times. And I agree with him, dynamic syscalls are the best way to get completely crappy interfaces.
Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · Andrew Morton
But if we need additional infrastructure to "add new syscalls via VDSO" then this should be in the base kernel, even if it's empty, yes?
Linus has vetoed dynamic syscall registration a few times. And I agree with him, dynamic syscalls are the best way to get completely crappy interfaces.
And why not definitely assign sort of a "multiplexed syscall" entry, a la sys_socketcall() ? It could be shared by lots of non-mainline projects and have a greater chance of being stable along the time.
Cheers,
Willy
Quoted 3 lines · Willy Tarreau
And why not definitely assign sort of a "multiplexed syscall" entry, a la
sys_socketcall()? It could be shared by lots of non-mainline projects and have a greater chance of being stable along the time.
That already exists and is every bit as hated as it should be. It's called ioctl().
-- wli
Quoted 2 lines · Willy Tarreau
And why not definitely assign sort of a "multiplexed syscall" entry, a la
sys_socketcall()?
Because that's the worst possible API you could imagine.
Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · Andrew Morton
But if we need additional infrastructure to "add new syscalls via VDSO" then this should be in the base kernel, even if it's empty, yes?
Linus has vetoed dynamic syscall registration a few times. And I agree with him, dynamic syscalls are the best way to get completely crappy interfaces.
The only thing I was thinking of doing was to export the symbol __kernel__NR_kexec_load. With a little care we could probably export the system call numbers just as easily from /proc/kallsyms.
At this point that idea seems to add no real benefit. Except for allowing for a user space that can more easily track syscall renumber in the kernel, which seems to be the wrong problem to solve.
So if kexec could actually get a reserved system call number that would be the best solution I have seen in this thread.
Andrew how close are we to a point where we can look at kexec inclusion?
Eric
Quoted 3 lines · [email protected] (Eric W. Biederman) wrote:
So if kexec could actually get a reserved system call number that
would be the best solution I have seen in this thread.
I have no problem with that - it's a major feature, vendors will, I assume, ship it so it's worth blowing the four bytes to pin the ABI down for everyone.
Quoted 1 line
Andrew how close are we to a point where we can look at kexec inclusion?
Well it's not exactly head-of-queue. Do any vendors actually intend to ship it?
Quoted 5 lines · Andrew Morton
Quoted 3 lines · [email protected] (Eric W. Biederman) wrote:
So if kexec could actually get a reserved system call number that
would be the best solution I have seen in this thread.
Quoted 8 lines
I have no problem with that - it's a major feature, vendors will, I assume, ship it so it's worth blowing the four bytes to pin the ABI down for everyone.
Quoted 1 line
Andrew how close are we to a point where we can look at kexec inclusion?
Well it's not exactly head-of-queue. Do any vendors actually intend to ship it?
I know of a few products kexec is embedded in already. As for the general purpose distro's I don't know.
Eric
Quoted 2 lines · Eric W. Biederman
So if kexec could actually get a reserved system call number that would be the best solution I have seen in this thread.
Could you please show the syscall we should reserve, e.g. it's API. We had quite some histroy of syscalls that got reserved and the real submission was rejected neverless because the API was broken.
Quoted 8 lines · Christoph Hellwig
Quoted 2 lines · Eric W. Biederman
So if kexec could actually get a reserved system call number that would be the best solution I have seen in this thread.
Could you please show the syscall we should reserve, e.g. it's API. We had quite some histroy of syscalls that got reserved and the real submission was rejected neverless because the API was broken.
The (old) kexec patch I have here implements the API which is described at
http://lwn.net/Articles/15468/. I doubt if it changed?
Quoted 2 lines · Andrew Morton
The (old) kexec patch I have here implements the API which is described at
http://lwn.net/Articles/15468/. I doubt if it changed?
That API looks sane to me.
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Andrew Morton
The (old) kexec patch I have here implements the API which is described at
http://lwn.net/Articles/15468/. I doubt if it changed?That API looks sane to me.
yup. Here's the syscall itself, btw:
+/*
+ * Exec Kernel system call: for obvious reasons only root may call it.
+ *
+ * This call breaks up into three pieces.
+ * - A generic part which loads the new kernel from the current
+ * address space, and very carefully places the data in the
+ * allocated pages.
+ *
+ * - A generic part that interacts with the kernel and tells all of
+ * the devices to shut down. Preventing on-going dmas, and placing + * the devices in a consistent state so a later kernel can + * reinitialize them.
+ *
+ * - A machine specific part that includes the syscall number
+ * and the copies the image to it's final destination. And
+ * jumps into the image at entry.
+ *
+ * kexec does not sync, or unmount filesystems so if you need
+ * that to happen you need to do that yourself.
+ */
+struct kimage *kexec_image = 0;
+
+asmlinkage long sys_kexec_load(unsigned long entry, unsigned long nr_segments,
+ struct kexec_segment *segments, unsigned long flags)
+{
+struct kexec_segment {
+ void *buf;
+ size_t bufsz;
+ void *mem;
+ size_t memsz;
+};
I have no plans to change this API. It has been reviewed and agreed upon. At some point we might run into a problem where the flag field is needed but currently it is a must be zero field.
Eric
On Thu, 13 May 2004 08:49:31 +0100 Christoph Hellwig wrote:
| On Thu, May 13, 2004 at 12:37:27AM -0700, Andrew Morton wrote: | > The (old) kexec patch I have here implements the API which is described at | > http://lwn.net/Articles/15468/. I doubt if it changed? | | That API looks sane to me.
and current patches are kept at:
http://developer.osdl.org/rddunlap/kexec/
for review, testing, etc.
--
~Randy
Quoted 37 lines · Randy.Dunlap
On 12 May 2004 10:57:27 -0600 Eric W. Biederman wrote:
| Ulrich Drepper <[email protected]> writes: | | > Eric W. Biederman wrote: | > | > > As a first draft we should be able to use the standard ELF mechanisms | > > for this. It is not like PIC shared libraries were new. Or is | > > there some specific problem you are thinking of with respect to | > > randomization? | > | > The official kernel does not have vdso randomization. Ingo has a patch | > for the Red Hat kernel which is used in the FC2 kernel. The patch | > effectively only changes the location at which the vdso is mapped. It | > does not change the vdso content. So the __kernel_vsyscall symbol in | > the vdso's symbol table is not changed.[1:]
| > AT_SYSINFO is the right way to go forward but it is not directly | > accessible to userlevel code. And it is no pointer which will make | > architectures with function descriptors unhappy. | | It sounds like the vdso just needs to be treated as a prelinked | vdso. You can find everything you need with AT_SYSINFO_EHDR. | | In the case of function descriptors they should be in a data segment | that can get copied to another page, and corrected. Leaving the code | segment at it's randomized location.Andrew tells me that he is OK with reserving a syscall number for kexec, which is easy & quick. I don't know when vdso will be available (for non-x86[2]) or when the
AT_SYSINFOdata will work for userlevel code[1. above].So is there any reason not to reserve the syscall number for kexec for now? (patch is below)
Quoted 6 lines
--
~Randy[2] kexec is currently only available for x86, but there is interest in it for ia64 and ppc64 at least.
Also been work on x86-64 and ppc32. So if we are going to reserve
syscall numbers it would also be nice to have those reserved as well.
Eric
Quoted 2 lines · Eric W. Biederman
Also been work on x86-64 and ppc32. So if we are going to reserve
syscall numbers it would also be nice to have those reserved as well.
Don't forget about ppc64.
Signature · 2 lines
Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center
Quoted 5 lines · Adam Litke
Quoted 2 lines · Eric W. Biederman
Also been work on x86-64 and ppc32. So if we are going to reserve
syscall numbers it would also be nice to have those reserved as well.Don't forget about ppc64.
Randy already mentioned it. :)
Making the list of at least attempted ports:
x86, ppc64, ia64, x86-64, ppc32
But x86 and ppc64 seem to have the most developer interest.
Eric
Followup to: <[email protected]> By author: [email protected] (Eric W. Biederman) In newsgroup: linux.dev.kernel
Quoted 8 lines
Randy already mentioned it. :)
Making the list of at least attempted ports:x86, ppc64, ia64, x86-64, ppc32
But x86 and ppc64 seem to have the most developer interest.
It should probably be reserved on all architectures, unless the architecture maintainers explicitly don't want to support it.
-hpa
Quoted 20 lines · Randy.Dunlap
kexec for Linux 2.6.6 is now available at:
http://developer.osdl.org/rddunlap/kexec/2.6.6/kexec userspace tools are at:
http://developer.osdl.org/rddunlap/kexec/kexec-tools/For 2.6.6, the
kexec_loadsyscall number had to move due to some new syscall additions in 2.6.6. However, I didn't update the kexec userspace program for that 1-line change. The change is inkexec-syscall.c, line 46:change
#define__NR_kexec_load274
to
#define__NR_kexec_load283(for i386).
Quoted 8 lines
There is one outstanding patch from Albert Herranz that I will review and possibly use/merge soon. His email is here:
http://lists.osdl.org/pipermail/fastboot/2004-May/000290.htmlAnd if anyone has suggestions for handling a variable/moving syscall number (target), I'm interested in hearing them.
The easiest would be to update the ./configure script, and have the syscall number as a parameter. Quite possibly it could just point to an appropriate file that would grep for __NR_kexec_load. This does not prevent the recompile problem but it does help a little.
On the issue of needing $BIGNUM testers, I see a small challenge. kexec is near useless on an unstable kernel. So it really needs to get into a stable series or people will be fighting kernel bugs so much that they won't be able to see kexec specific issues.
Possibly the sanest route is 2.7.early and then 2.6 at about the same time. At any point that kexec makes it into a mainline kernel though the syscall number will be frozen and things will get easier.
Eric