Skip to contentSkip to conversation

Remotely rebooting a machine with state 'D' processes, how?

DISCUSSION Axel Thimm 11 participants 15 messages Aug 10, 2001 – Aug 19, 2001

lore
Share & tools
lore

Axel Thimm

author
#0 lore raw

How can I reboot a stuck machine remotely, when there are uninterruptable processes arround? shutdown -r, reboot [-n] [-f], telinit 6 do not give the intended results. Localy I can use Alt-SysRq-S/U/B, but what if I still have a remote ssh connection and don't want to have to get to the machines location?

Of course the real problem are the processes themselves, but being able to revive a machine is also nice ;)

Regards, Axel.

Signature · 1 line
[email protected]
#1 lore raw

It has been awhile since I had this problem, but I recall
shutdown -r -n now
doing the job.

-- Brian

Quoted 10 lines · Axel Thimm

How can I reboot a stuck machine remotely, when there are uninterruptable processes arround? shutdown -r, reboot [-n] [-f], telinit 6 do not give the intended results. Localy I can use Alt-SysRq-S/U/B, but what if I still have a remote ssh connection and don't want to have to get to the machines location?

Of course the real problem are the processes themselves, but being able to revive a machine is also nice ;)

Regards, Axel.

#2 lore raw
In reply to Axel Thimm

In article <[email protected]> you write:

Quoted 7 lines

How can I reboot a stuck machine remotely, when there are uninterruptable processes arround? shutdown -r, reboot [-n] [-f], telinit 6 do not give the intended results. Localy I can use Alt-SysRq-S/U/B, but what if I still have a remote ssh connection and don't want to have to get to the machines location?

Of course the real problem are the processes themselves, but being able to revive a machine is also nice ;)

You have to use the reboot() system call directly as root, with the proper arguments to make it avoid doing even any sync. See

man 2 reboot

for details.

Linus

#3 lore raw
Quoted 6 lines · Linus Torvalds

In article <[email protected]> you write:

You have to use the reboot() system call directly as root, with the proper arguments to make it avoid doing even any sync. See

man 2 reboot

How do you do this when the process in the D state is holding the BKL?

Signature · 4 lines
Debian GNU/Linux 2.2 is out! ( http://www.debian.org/ )
Email:  Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#4 lore raw
Quoted 9 lines · Herbert Xu
Quoted 6 lines · Linus Torvalds

In article <[email protected]> you write:

You have to use the reboot() system call directly as root, with the proper arguments to make it avoid doing even any sync. See

man 2 reboot

How do you do this when the process in the D state is holding the BKL?

the kernel should not sleep when it is holding the BKL (or any semaphore).

Signature · 3 lines
Robert M. Love
rml at ufl.edu
rml at tech9.net
#5 lore raw
In reply to Herbert Xu
Quoted 10 lines · Herbert Xu
Quoted 6 lines · Linus Torvalds

In article <[email protected]> you write:

You have to use the reboot() system call directly as root, with the proper arguments to make it avoid doing even any sync. See

man 2 reboot

How do you do this when the process in the D state is holding the BKL?

If it's in D state, it will be sleeping, and will have released the BKL.

Besides, does the reboot system call actually get the BKL? I don't think it should need it..

Linus

#6 lore raw
Quoted 3 lines · Linus Torvalds

Besides, does the reboot system call actually get the BKL? I don't think it should need it..

Actually, the machine in question turned out to be UP :)

However, it does have RAID 1 and the notifier call chain stuff looks like a killer to me since it leads to do_md_stop.

Perhaps we need a RESTART3 that restarts without notifying?

Signature · 4 lines
Debian GNU/Linux 2.2 is out! ( http://www.debian.org/ )
Email:  Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#7 lore raw
Quoted 1 line · Herbert Xu

Perhaps we need a RESTART3 that restarts without notifying?

That would be the one line outb that Chris Wedgewood posted a few notes back in the thread. If my memory of x86 bios is correct he simple poked the control register that equates to pushing the reset button.

Signature · 2 lines
now the forces of openness have a powerful and
  unexpected new ally - http://ibm.com/linux
#8 lore raw
In reply to Linus Torvalds
Quoted 11 lines · Herbert Xu
Quoted 3 lines · Linus Torvalds

Besides, does the reboot system call actually get the BKL? I don't think it should need it..

Actually, the machine in question turned out to be UP :)

However, it does have RAID 1 and the notifier call chain stuff looks like a killer to me since it leads to do_md_stop.

Perhaps we need a RESTART3 that restarts without notifying?

Interesting...

I have an oldworld ppc mac with RAID1 compiled on 2.2.19, and a bad floppy made badblocks unkillable.

Is there another way to kill that will work when kill -9 won't?

#9 lore raw
In reply to Linus Torvalds
Quoted 11 lines · Mike Fedyk
Quoted 10 lines · Herbert Xu
Quoted 3 lines · Linus Torvalds

Besides, does the reboot system call actually get the BKL? I don't think it should need it..

Actually, the machine in question turned out to be UP :)

However, it does have RAID 1 and the notifier call chain stuff looks like a killer to me since it leads to do_md_stop.

Quoted 6 lines
Quoted 1 line

Perhaps we need a RESTART3 that restarts without notifying?

Interesting...

I have an oldworld ppc mac with RAID1 compiled on 2.2.19, and a bad floppy made badblocks unkillable.

I think that was probably because the pmac floppy driver has some issues..

Signature · 5 lines
Troy Benjegerdes | master of mispeeling | 'da hozer' |  [email protected]
-----"If this message isn't misspelled, I didn't write it" -- Me -----
"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's 
why I draw cartoons. It's my life." -- Charles Shulz
#10 lore raw
In reply to Linus Torvalds

In article <[email protected]>,

Quoted 9 lines · [email protected] (Herbert Xu) writes:
Quoted 6 lines · Linus Torvalds

In article <[email protected]> you write:

You have to use the reboot() system call directly as root, with the proper arguments to make it avoid doing even any sync. See

man 2 reboot

How do you do this when the process in the D state is holding the BKL?

A process in D state sleeps and the BKL is always automatically dropped when a process sleeps.

Bigger problems are other semaphores like inode sems, especially when they belong to important shared libraries or directories.

-Andi

#11 lore raw
In reply to Linus Torvalds

On Sat, Aug 11, 2001 at 08:50:35AM +1000, Herbert Xu wrote:

How do you do this when the process in the D state is holding the BKL?

On ia32:

        outb(0xfe, 0x64);

I have a command-line proglet for doing this, and also a patch to add to SysReq is people really want such a thing.

--cw

#12 lore raw
In reply to Linus Torvalds
Quoted 3 lines · Andi Kleen

A process in D state sleeps and the BKL is always automatically dropped when a process sleeps.

OK. What about console_lock since reboot(2) insists on doing a printk?

Signature · 4 lines
Debian GNU/Linux 2.2 is out! ( http://www.debian.org/ )
Email:  Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#13 lore raw
In reply to Linus Torvalds

[email protected] (Linus Torvalds) wrote on 10.08.01 in <[email protected]>:

Quoted 9 lines

In article <[email protected]> you write:

Quoted 7 lines

How can I reboot a stuck machine remotely, when there are uninterruptable processes arround? shutdown -r, reboot [-n] [-f], telinit 6 do not give the intended results. Localy I can use Alt-SysRq-S/U/B, but what if I still have a remote ssh connection and don't want to have to get to the machines location?
Of course the real problem are the processes themselves, but being able to revive a machine is also nice ;)

Quoted 6 lines

You have to use the reboot() system call directly as root, with the proper arguments to make it avoid doing even any sync. See

man 2 reboot

for details.

I thought that was exactly what reboot -n -f does: don't sync, don't call shutdown, just reboot immediately. That's certainly what I have (successfully) used that for in the past.

MfG Kai

author
#14 lore raw
In reply to Axel Thimm
Quoted 7 lines · Linus Torvalds

In article <[email protected]> you write:

Quoted 6 lines

How can I reboot a stuck machine remotely, when there are uninterruptable processes arround? shutdown -r, reboot [-n] [-f], telinit 6 do not give the intended results. Localy I can use Alt-SysRq-S/U/B, but what if I still have a remote ssh connection and don't want to have to get to the machines location?

Quoted 5 lines
Quoted 2 lines

Of course the real problem are the processes themselves, but being able to revive a machine is also nice ;)

You have to use the reboot() system call directly as root, with the proper arguments to make it avoid doing even any sync. See man 2 reboot for details.

Would there be a way to also simulate the effects of Alt-SysRq-S and Alt-SysRq-U? A simple sync falls also into D-state, Alt-SysRq-S does not, as far as I have had to use it.

Would an `emergency-reboot' binary that calls these three kernel calls be possible and also good-to-have if it's not a too bad idea?

Is there a way to have this automated in some kind of a kernel software watchdog? Given certain conditions (like processes in D-state for longer than a specified time) the kernel might first try to call a userland-reboot and this timing out the kernel might use the sync;umount;reboot procedure of Alt-SysRq-S/U/B.

But even when I write this I realize the security hole potentials. If someone could simulate the watchdog conditions for a reboot, he might be able to have this machine reboot all the time.

Signature · 1 line
[email protected]