Skip to contentSkip to conversation

Kexec jump: The first step to kexec base

PATCH Al Boldi 2 participants 5 messages Jul 13, 2007 – Jul 14, 2007

lore
Share & tools
lore

Al Boldi

author
#0 lore raw
Quoted 2 lines · Ying

Kexec jump - The first step to kexec base hibernation

Thanks a lot!

Quoted 11 lines

Kexec base hibernation has some potential advantages over uswsusp and suspend2. Some most obvious advantages are:

1. The hibernation image size can exceed half of memory size easily.
2. The hibernation image can be written to and read from almost anywhere, such as USB disk, NFS.

This patch implements the functionality of "jumping from kexeced kernel to original kernel". That is, the following sequence is possible:

Quoted 6 lines

1. Boot a kernel A
2. Work under kernel A
3. Kexec another kernel B in kernel A
4. Work under kernel B
5. Jump from kernel B to kernel A
6. Continue work under kernel A

Great!

Quoted 4 lines

This is the first step to implement kexec based hibernation. If the memory image of kernel A is written to or read from a permanent media in step 4, a preliminary version of kexec based hibernation can be implemented.

Sounds easy.

Quoted 5 lines

The kernel B is run as a crashdump kernel in reserved memory region. This is the biggest constrains of the patch. It is planed to be eliminated in the next version. That is, instead of reserving memory region previously, the needed memory region is backuped before kexec and restored after jumping back.

Actually, backing up from within the normal kernel may produce inter-dependency deadlocks. It's much better to just force-free some memory chunk out to swap.

Quoted 16 lines

Another constrains of the patch is that the CONFIG_ACPI must be turned off to make kexec jump work. Because ACPI will put devices into low power state, the kexeced kernel can not be booted properly under it. This constrains can be eliminated by separating the suspend method and hibernation method of the devices as proposed earlier in the LKML.

The kexec jump is implemented in the framework of software suspend. In fact, the kexec based hibernation can be seen as just implementing the image writing and reading method of software suspend with a kexeced Linux kernel.

Now, only the i386 architecture is supported. The patch is based on Linux kernel 2.6.22, and has been tested on my IBM T42.

Usage:

Quoted 7 lines

1. Compile kernel with following options selected:

CONFIG_X86_32=y
CONFIG_RELOCATABLE=y # not needed strictly, but it is more convenient with it CONFIG_KEXEC=y
CONFIG_SOFTWARE_SUSPEND=y
CONFIG_KEXEC_HIBERNATION=y

What about CONFIG_CRASH_DUMP=y ?
Also CONFIG_PHYSICAL_START=1000000 (=16M instead of default 1M).

Quoted 5 lines

2. Compile the kexec-tools with kdump and kjump patches added, the kdump patch can be found at:

http://lse.sourceforge.net/kdump/patches/kexec-tools-1.101-kdump10.patch

Quoted 8 lines

While, the kexec-tools kjump patch is appended with the mail.

3. Boot compiled kernel, the reserved crash kernel memory region must be added to kernel command line as following:

crashkernel=<XX>M@<XX>M

Where, <XX> should be replaced by the real memory size and position.

Ok, booted with crashkernel=16M@16M.

Quoted 3 lines

4. Switch hibernation image operations, through shell command as follow:

echo kexec > /sys/power/hibernation_image_ops

Ok.

Quoted 5 lines

5. Boot the kexeced kernel as a crashdump kernel, the same kernel can be used if CONFIG_RELOCATABLE=y is selected. The kernel command line option as following must be appended to kernel command line.

kexec_jump_buf_pfn=cat /sys/kernel/kexec_jump_buf_pfn

You mean like:

# kexec -p vmlinux 
        --args-linux 
        --append="root=/dev/sda1 
                  kexec_jump_buf_pfn = `cat /sys/kernel/kexec_jump_buf_pfn`"
# cat c > /proc/sysrq

This loads ok.

Quoted 2 lines

6. In the kexec booted kernel, switch hibernation image operations, as in 4.

Ok.

Quoted 7 lines

7. In the kexec booted kernel, trigger the jumping back with following shell command.

echo <a>:<b> > /sys/power/resume

Where <a> and <b> is non-negative integer, at least one of them must be non-zero.

This doesn't switch it, but rather reboots my machine.

Thanks!

--
Al

author
#1 lore raw

Re: [PATCH 0/2] Kexec jump: The first step to kexec base hibernation

Quoted 7 lines · Rafael J. Wysocki

We have quite an efficient restoration code in the kernel right now. It's able to upload big images (something like total RAM minus the size of the boot kernel, initrd and, optionally, the resume application), which is much more than we're able to save. :-)

It can work with images uploaded via /dev/snapshot from the user space (specific image format is required, but that can be changed easily).

Then all that is need is to dump /dev/oldmem under the hibernate kernel and then feed it into /dev/snapshot when booting the normal kernel.

Or is there more to it?

Thanks!

--
Al

#2 lore raw

Re: [PATCH 0/2] Kexec jump: The first step to kexec base hibernation

Quoted 6 lines · Al Boldi
Quoted 5 lines · Rafael J. Wysocki

We have quite an efficient restoration code in the kernel right now. It's able to upload big images (something like total RAM minus the size of the boot kernel, initrd and, optionally, the resume application), which is much more than we're able to save. :-)

Quoted 7 lines
Quoted 2 lines

It can work with images uploaded via /dev/snapshot from the user space (specific image format is required, but that can be changed easily).

Then all that is need is to dump /dev/oldmem under the hibernate kernel and then feed it into /dev/snapshot when booting the normal kernel.

Or is there more to it?

Not really. Please see my recent message in the other branch of this thread.

Greetings,
Rafael

Signature · 1 line
"Premature optimization is the root of all evil." - Donald Knuth
author
#3 lore raw

Re: [PATCH 0/2] Kexec jump: The first step to kexec base hibernation

Quoted 39 lines · Ying

Hibernating process:

1. Normal kernel running
2. Hibernating is triggered, sys_kexec_load is used to load hibernating kernel and initramfs into memory. Then sys_reboot(LINUX_REBOOT_CMD_KSPAWN) is invoked.
3. In sys_reboot, kexec_jump is called to save device/CPU state, then relocate_kernel is called. kexec_jump and relocate_kernel reside in individual page in 16M~512M.
4. In relocate_kernel, 0~16M is backupped firstly, then the hibernating kernel and initramfs is copied to 0~16M, after that, the hibernating kernel is booted.
5. In hibernating kernel, the memory of normal kernel (it is in 16M~512M) is saved into a hibernation image through /dev/mem and ELF header.

Resume process:

1. Resuming kernel is booted as a normal kernel, but the memory is restricted to 0~16M.
2. Checking whether there is a effective hibernation image. If there isn't, the memory of 16M~512M is hot added, and the normal boot up process continues; If there is, a resuming process is triggered.
3. sys_kexec_load is used to restore the memory state of hibernated kernel. The sys_kexec_load works in crashdump way, that is, the hibernation image is copied to destination location in 16M~512M in sys_kexec_load instead of relocate_kernel. There is no half of memory size restriction.
4. sys_reboot is called to trigger jumping back, which will jump back to kexec_jump of hibernated kernel.
5. In kexec_jump of hibernated kernel, the memory of 0~16M is copied back from the backup area in 16M~512M. The memory state of hibernated kernel is restored totally. The CPU and device state can be restored after that.

Quoted 8 lines

If there is too much difficulty to hot add memory in step 2. A more conservative method can be used as step 1 and step 2.

1. A normal kernel is booted.
2. Checking whether there is a effective hibernation image. If there isn't, continue the normal boot process; otherwise, a resuming kernel is kexeced in memory 0~16M. The resuming process will continue in kexeced resuming kernel.

During Resume no kexec'ing is necessary; the normal kernel should be able to read the hibernation image via the /dev/snapshot interface.

Thanks!

--
Al

author
#4 lore raw

Re: [PATCH 0/2] Kexec jump: The first step to kexec base hibernation

In reply to Al Boldi
Quoted 9 lines · Rafael J. Wysocki
Quoted 8 lines · Ying
Quoted 7 lines · On Thu, 2007-07-12 at 20:06 -0700, [email protected] wrote:
Quoted 6 lines
Quoted 3 lines

I agree, a stipped down hibernate kernel can be very small, not allocating this memory until it's needed is a step for the final polishing.

I'm not sure if I agree with that. In any case, having to use two different kernels for hibernation would be a big drawback.

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

I see it as a big advantage to not have to use the main kernel for the suspend. please keep it as an option at least.

Yes. It has additional bonus to make it possible to write/read image from a program other than main kernel. For example, for a specific mobile device product (Such as Intel MID), a customized ultra-small program (or kernel) can be composed to write/read image. That way, the hibernate/resume time can be reduced to minimal.

Sounds great!

Quoted 2 lines

You don't need kexec for that. This is how the userland hibernation (aka uswsusp) works.

I don't think so. uswsusp runs under the normal kernel which implies an interdependency, whereas a device driver stub'd with a small single proc kernel would run in its own space.

The question is, how small could this kernel stub be made and still leverage the device driver pool?

Thanks!

--
Al