Skip to contentSkip to conversation

why kexec insists on syncing with recent kernels?

DISCUSSION Tomasz Chmielewski 5 participants 9 messages Feb 7, 2008 – Feb 11, 2008

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

According to kernel/kexec.c:

* kexec does not sync, or unmount filesystems so if you need
* that to happen you need to do that yourself.

I saw this was true with 2.6.18 kernel (i.e., it didn't sync), but kexec syncs with recent kernels (I checked 2.6.23.14 and 2.6.24):

# kexec -e
md: stopping all md devices
sd 2:0:0:0: [sdb] Synchronizing SCSI cache

With kexec on 2.6.18 it was executing a loaded kernel immediately.

Generally, it's a good thing to sync before jumping into a new kernel, but it breaks my setup here after upgrading from 2.6.18 to 2.6.24.

Why?

I have a couple of diskless (iSCSI-boot) machines with a buggy BIOS (old Supermicro P4SBR/P4SBE) which randomly freeze after rebooting (the machine shuts down just fine, but instead of booting again, showing BIOS bootup messages etc. you can just see blank screen).

Therefore, I use kexec as a workaround for this rebooting problem.

The way kexec works now makes rebooting unreliable again:
- network interfaces are brought down,
- kernel tries to sync - it never will, as we're booted off network, which is down

Any ideas why kexec insists on syncing?

Signature · 2 lines
Tomasz Chmielewski
http://blog.wpkg.org
#1 lore raw
Quoted 6 lines · Tomasz Chmielewski

According to kernel/kexec.c:

* kexec does not sync, or unmount filesystems so if you need
* that to happen you need to do that yourself.

In latest kexec code I do see it syncing. But it does not unmount the filesystems. So this comment looks like partially wrong.

Quoted 6 lines

I saw this was true with 2.6.18 kernel (i.e., it didn't sync), but kexec syncs with recent kernels (I checked 2.6.23.14 and 2.6.24):

# kexec -e
md: stopping all md devices
sd 2:0:0:0: [sdb] Synchronizing SCSI cache

Which kexec-tools you are using?

syncing is initiated by user space so changing kernel will not have any effect (as long as user space is same). I think just that message are spitted by kernel, so probably 2.6.18 did not spit any message and 2.6.24 does.

Quoted 15 lines

With kexec on 2.6.18 it was executing a loaded kernel immediately.

Generally, it's a good thing to sync before jumping into a new kernel, but it breaks my setup here after upgrading from 2.6.18 to 2.6.24.

Why?

I have a couple of diskless (iSCSI-boot) machines with a buggy BIOS (old Supermicro P4SBR/P4SBE) which randomly freeze after rebooting (the machine shuts down just fine, but instead of booting again, showing BIOS bootup messages etc. you can just see blank screen).

Quoted 7 lines

Therefore, I use kexec as a workaround for this rebooting problem.

The way kexec works now makes rebooting unreliable again:
- network interfaces are brought down,
- kernel tries to sync - it never will, as we're booted off network, which is down

Kexec has got an option -x --no-ifdown, which will not bring the network down. Try that. "kexec- -e -x"

Quoted 1 line

Any ideas why kexec insists on syncing?

To me it makes sense. Just making sure that cache changes make to the file system before you boot into new kernel.

In latest kexec-tools, I do see sync() is done first and then network interfaces are brought down.

Try latest kexec tools from:

http://www.vergenet.net/~horms/linux/kexec/kexec-tools/testing/kexec-tools-testing-20071017-rc.tar.gz

and see if it works fine for you.

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#2 lore raw
Quoted 11 lines · Vivek Goyal
Quoted 6 lines · Tomasz Chmielewski

According to kernel/kexec.c:

* kexec does not sync, or unmount filesystems so if you need
* that to happen you need to do that yourself.

In latest kexec code I do see it syncing. But it does not unmount the filesystems. So this comment looks like partially wrong.

Quoted 8 lines
Quoted 6 lines

I saw this was true with 2.6.18 kernel (i.e., it didn't sync), but kexec syncs with recent kernels (I checked 2.6.23.14 and 2.6.24):

# kexec -e
md: stopping all md devices
sd 2:0:0:0: [sdb] Synchronizing SCSI cache

Which kexec-tools you are using?

# kexec -v
kexec 1.101 released 15 February 2005
Quoted 4 lines

syncing is initiated by user space so changing kernel will not have any effect (as long as user space is same). I think just that message are spitted by kernel, so probably 2.6.18 did not spit any message and 2.6.24 does.

Yes and no.
I just booted 2.6.24 on a diskless system (Mandriva) I normally use with 2.6.18 kernel, did kexec -e... And it executed the kernel immediately, without any syncing.
On Debian, with the same 2.6.24 kernel, it does sync.

So what user space part does the syncing (and how to prevent it)?

(...)

Quoted 8 lines
Quoted 5 lines

The way kexec works now makes rebooting unreliable again:
- network interfaces are brought down,
- kernel tries to sync - it never will, as we're booted off network, which is down

Kexec has got an option -x --no-ifdown, which will not bring the network down. Try that. "kexec- -e -x"

It does seem to help, thanks.

Why it has to be the last option specified?

I tried -f option before (don't call shutdown), but it didn't help.

Quoted 11 lines
Quoted 1 line

Any ideas why kexec insists on syncing?

To me it makes sense. Just making sure that cache changes make to the file system before you boot into new kernel.

In latest kexec-tools, I do see sync() is done first and then network interfaces are brought down.

Try latest kexec tools from:

http://www.vergenet.net/~horms/linux/kexec/kexec-tools/testing/kexec-tools-testing-20071017-rc.tar.gz

Good to have a newer version, I'll try that, too.

Signature · 8 lines
Tomasz Chmielewski
http://wpkg.org


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#8 lore raw
In reply to Tomasz Chmielewski
Quoted 27 lines · Tomasz Chmielewski

According to kernel/kexec.c:

* kexec does not sync, or unmount filesystems so if you need
* that to happen you need to do that yourself.

I saw this was true with 2.6.18 kernel (i.e., it didn't sync), but kexec syncs with recent kernels (I checked 2.6.23.14 and 2.6.24):

# kexec -e
md: stopping all md devices
sd 2:0:0:0: [sdb] Synchronizing SCSI cache

With kexec on 2.6.18 it was executing a loaded kernel immediately.

Generally, it's a good thing to sync before jumping into a new kernel, but it breaks my setup here after upgrading from 2.6.18 to 2.6.24.

Why?

I have a couple of diskless (iSCSI-boot) machines with a buggy BIOS (old Supermicro P4SBR/P4SBE) which randomly freeze after rebooting (the machine shuts down just fine, but instead of booting again, showing BIOS bootup messages etc. you can just see blank screen).

Quoted 8 lines

Therefore, I use kexec as a workaround for this rebooting problem.

The way kexec works now makes rebooting unreliable again:
- network interfaces are brought down,
- kernel tries to sync - it never will, as we're booted off network, which is down

Any ideas why kexec insists on syncing?

kexec calls device_shutdown(), so ->shutdown() will be called for all devices.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#3 lore raw
In reply to Tomasz Chmielewski
Quoted 30 lines · Tomasz Chmielewski
Quoted 11 lines · Vivek Goyal
Quoted 6 lines · Tomasz Chmielewski

According to kernel/kexec.c:

* kexec does not sync, or unmount filesystems so if you need
* that to happen you need to do that yourself.

In latest kexec code I do see it syncing. But it does not unmount the filesystems. So this comment looks like partially wrong.

Quoted 8 lines
Quoted 6 lines

I saw this was true with 2.6.18 kernel (i.e., it didn't sync), but kexec syncs with recent kernels (I checked 2.6.23.14 and 2.6.24):

# kexec -e
md: stopping all md devices
sd 2:0:0:0: [sdb] Synchronizing SCSI cache

Which kexec-tools you are using?

# kexec -v
kexec 1.101 released 15 February 2005
Quoted 4 lines

syncing is initiated by user space so changing kernel will not have any effect (as long as user space is same). I think just that message are spitted by kernel, so probably 2.6.18 did not spit any message and 2.6.24 does.

Quoted 7 lines

Yes and no.
I just booted 2.6.24 on a diskless system (Mandriva) I normally use with 2.6.18 kernel, did kexec -e... And it executed the kernel immediately, without any syncing.
On Debian, with the same 2.6.24 kernel, it does sync.

So what user space part does the syncing (and how to prevent it)?

Syncing is initiated by kexec-tools. Following is the code in kexec/kexec.c in kexec-tools-testing.tar.gz

        if ((result == 0) && do_sync) {
                sync();

I think this problem has nothing to do with syncing. There seems to be some dependency on not shutting down network here. You might want to debug, exactly where does it get stuck.

- Specify earlyprintk= parameter for second kernel and see if control is reaching to second kernel.

- Otherwise specify --console-serial parameter on "kexec -l" commandline and it should display a message "I am in purgatory" on serial console. This will just mean that control has reached at least till purgatory.

Right now there does not seem to be any option to prevent syncing and I don't know why would one like to have one.

Quoted 9 lines

(...)

Quoted 6 lines
Quoted 5 lines

The way kexec works now makes rebooting unreliable again:
- network interfaces are brought down,
- kernel tries to sync - it never will, as we're booted off network, which is down

Quoted 7 lines
Quoted 2 lines

Kexec has got an option -x --no-ifdown, which will not bring the network down. Try that. "kexec- -e -x"

It does seem to help, thanks.

Why it has to be the last option specified?

I have no idea. This might be an stale comment. Try putting -x before -e.

Quoted 2 lines

I tried -f option before (don't call shutdown), but it didn't help.

Even if you did -f, it must have shutdown the network. I think somehow in latest kernels there is some dependency on network and that's why not shutting down network in this case is helping you.

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#4 lore raw
Quoted 77 lines · On Fri, 8 Feb 2008 11:04:08 -0500 Vivek Goyal wrote:
Quoted 30 lines · Tomasz Chmielewski
Quoted 11 lines · Vivek Goyal
Quoted 6 lines · Tomasz Chmielewski

According to kernel/kexec.c:

* kexec does not sync, or unmount filesystems so if you need
* that to happen you need to do that yourself.

In latest kexec code I do see it syncing. But it does not unmount the filesystems. So this comment looks like partially wrong.

Quoted 8 lines
Quoted 6 lines

I saw this was true with 2.6.18 kernel (i.e., it didn't sync), but kexec syncs with recent kernels (I checked 2.6.23.14 and 2.6.24):

# kexec -e
md: stopping all md devices
sd 2:0:0:0: [sdb] Synchronizing SCSI cache

Which kexec-tools you are using?

# kexec -v
kexec 1.101 released 15 February 2005
Quoted 4 lines

syncing is initiated by user space so changing kernel will not have any effect (as long as user space is same). I think just that message are spitted by kernel, so probably 2.6.18 did not spit any message and 2.6.24 does.

Quoted 7 lines

Yes and no.
I just booted 2.6.24 on a diskless system (Mandriva) I normally use with 2.6.18 kernel, did kexec -e... And it executed the kernel immediately, without any syncing.
On Debian, with the same 2.6.24 kernel, it does sync.

So what user space part does the syncing (and how to prevent it)?

Syncing is initiated by kexec-tools. Following is the code in kexec/kexec.c in kexec-tools-testing.tar.gz

        if ((result == 0) && do_sync) {
                sync();

I think this problem has nothing to do with syncing. There seems to be some dependency on not shutting down network here. You might want to debug, exactly where does it get stuck.

- Specify earlyprintk= parameter for second kernel and see if control is reaching to second kernel.

- Otherwise specify --console-serial parameter on "kexec -l" commandline and it should display a message "I am in purgatory" on serial console. This will just mean that control has reached at least till purgatory.

Right now there does not seem to be any option to prevent syncing and I don't know why would one like to have one.

Quoted 9 lines

(...)

Quoted 6 lines
Quoted 5 lines

The way kexec works now makes rebooting unreliable again:
- network interfaces are brought down,
- kernel tries to sync - it never will, as we're booted off network, which is down

Quoted 7 lines
Quoted 2 lines

Kexec has got an option -x --no-ifdown, which will not bring the network down. Try that. "kexec- -e -x"

It does seem to help, thanks.

Why it has to be the last option specified?

Quoted 8 lines

I have no idea. This might be an stale comment. Try putting -x before -e.

Quoted 2 lines

I tried -f option before (don't call shutdown), but it didn't help.

Even if you did -f, it must have shutdown the network. I think somehow in latest kernels there is some dependency on network and that's why not shutting down network in this case is helping you.

I'm seeing NFS mounts take forever to unmount (at shutdown/reboot). (forever => 1 hour ... or never completes)

Is this similar to the problem that the OP is asking about?

---
~Randy

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#5 lore raw

Randy Dunlap schrieb:

(...)

Quoted 8 lines
Quoted 3 lines

Even if you did -f, it must have shutdown the network. I think somehow in latest kernels there is some dependency on network and that's why not shutting down network in this case is helping you.

I'm seeing NFS mounts take forever to unmount (at shutdown/reboot). (forever => 1 hour ... or never completes)

Is this similar to the problem that the OP is asking about?

Is it a diskless station?

Even in not, just make sure you don't shut the network down before NFS is actually unmounted...?

Signature · 7 lines
Tomasz Chmielewski
http://wpkg.org

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#6 lore raw
Quoted 11 lines · Tomasz Chmielewski

Randy Dunlap schrieb:

(...)

Quoted 7 lines
Quoted 3 lines

Even if you did -f, it must have shutdown the network. I think somehow in latest kernels there is some dependency on network and that's why not shutting down network in this case is helping you.

I'm seeing NFS mounts take forever to unmount (at shutdown/reboot). (forever => 1 hour ... or never completes)

Quoted 6 lines
Quoted 1 line

Is this similar to the problem that the OP is asking about?

Is it a diskless station?

Even in not, just make sure you don't shut the network down before NFS is actually unmounted...?

Network is shutdown just before kexec -e finally enters the kernel and starts preparing to jump to the new kernel. Any syncing operation, or File system unmounting operation will be done before that.

Can't understand why not disabling the network will help in NFS umount or syncing operation. It has to be something else.

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#7 lore raw
In reply to Tomasz Chmielewski
Quoted 11 lines · Tomasz Chmielewski

Randy Dunlap schrieb:

(...)

Quoted 7 lines
Quoted 3 lines

Even if you did -f, it must have shutdown the network. I think somehow in latest kernels there is some dependency on network and that's why not shutting down network in this case is helping you.

I'm seeing NFS mounts take forever to unmount (at shutdown/reboot). (forever => 1 hour ... or never completes)

Quoted 6 lines
Quoted 1 line

Is this similar to the problem that the OP is asking about?

Is it a diskless station?

Even in not, just make sure you don't shut the network down before NFS is actually unmounted...?

JFYI, on PS3, kexec works fine with NFS root.

With kind regards,

Geert Uytterhoeven
Software Architect

Sony Network and Software Technology Center Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453	
Fax:      +32 (0)2 700 8622	
E-mail:   [email protected]	
Internet: http://www.sony-europe.com/

Sony Network and Software Technology Center Europe
A division of Sony Service Centre (Europe) N.V. Registered office: Technologielaan 7 · B-1840 Londerzeel · Belgium VAT BE 0413.825.160 · RPR Brussels
Fortis Bank Zaventem · Swift GEBABEBB08A · IBAN BE39001382358619

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec