Skip to contentSkip to conversation

hibernate/suspend-to-disk: to turn power or not?

DISCUSSION Michael Tokarev 7 participants 13 messages Jan 30, 2008 – Jul 14, 2008

lore
Share & tools
lore
#0 lore raw

I'm trying to "glue" hibernation and UPS control together, and have a question.

When the system power comes off an UPS (Uninterruptable Power Supply I mean), it's probably a good idea to turn the UPS off when shutting the system down or hibernating.

Even with shutdown (not related to hibernating) there's an interesting twist: modern systems can remember the previous on/off status when the power gets cut and restores - BIOS-controlled option that allows the system to automatically start when the input power returns. Now, when we shutting down a system, we need to turn the UPS *before* powering down the system but *after* the shutdown procedure has been completed. This is done by replacing poweroff with halt, and ordering the UPS to turn itself off after a small delay (needed for the poweroff command to complete) - this way, system will be on but in a safe-to-powercut mode, and in order to turn it back on only the UPS has to be turned on.

But the question comes as how to control the UPS when we replace shutdown with hibernation. Ideally I want to send some commands to the UPS after hibernation has been completed (since I don't know how much time it will require to hibernate), but before the machine will be turned off by the kernel. Or at the very least, to be able to stop the kernel turning the machine off after hibernation process is complete - this way, I can order the UPS to turn the power off after some delay and hope hibernation process will finish when the UPS will finally cut the power (not all UPSes are able to do timely shutdown like this).

What's my way here?

Thanks!

/mjt

P.S. Surprisingly, there's NO software that works with UPSes and implements even the basic shutdown process completely (not even mentioning hibernation). Most common is just to turn the system off without touching the UPS. Yes it will work (till the UPS will run out of battery), but how about servers which should be up-n-running, ie. which should restart automatically?.. Oh well.

#1 lore raw
Quoted 6 lines · On Wednesday 30 January 2008 20:18:40 you wrote:

I'm trying to "glue" hibernation and UPS control together, and have a question.

When the system power comes off an UPS (Uninterruptable Power Supply I mean), it's probably a good idea to turn the UPS off when shutting the system down or hibernating.

This depends on how you can turn the UPS back on... If pushing the button manually is not an issue or the UPS can still be controlled remotely (e.g. via network) when being off it doesn't matter.

Quoted 6 lines

Even with shutdown (not related to hibernating) there's an interesting twist: modern systems can remember the previous on/off status when the power gets cut and restores - BIOS-controlled option that allows the system to automatically start when the input power returns. Now, when we shutting down a system, we

Quoted 8 lines

need to turn the UPS *before* powering down the system but *after* the shutdown procedure has been completed. This is done by replacing poweroff with halt, and ordering the UPS to turn itself off after a small delay (needed for the poweroff command to complete) - this way, system will be on but in a safe-to-powercut mode, and in order to turn it back on only the UPS has to be turned on.

All systems I've seen that provide such an option in the BIOS list three possible actions on power-back:
- remain off
- go back to previous state
- boot

In your case it's possibly easier to choose the option to always boot when power comes back.

Quoted 4 lines

But the question comes as how to control the UPS when we replace shutdown with hibernation. Ideally I want to send some commands to the UPS after hibernation has been completed (since I don't know how much time it

Quoted 8 lines

will require to hibernate), but before the machine will be turned off by the kernel. Or at the very least, to be able to stop the kernel turning the machine off after hibernation process is complete - this way, I can order the UPS to turn the power off after some delay and hope hibernation process will finish when the UPS will finally cut the power (not all UPSes are able to do timely shutdown like this).

Can you configure your UPS to poweroff when load on its output drops below a given threshold? (I know that such a feature exists on master-slave power outlets where all slave outlets are powered only when the master outlet has a sufficient load)

Another path may be to dig into hibernate via kexec:
http://kerneltrap.org/node/11756

Quoted 6 lines

What's my way here?

Thanks!

/mjt

Quoted 8 lines

P.S. Surprisingly, there's NO software that works with UPSes and implements even the basic shutdown process completely (not even mentioning hibernation). Most common is just to turn the system off without touching the UPS. Yes it will work (till the UPS will run out of battery), but how about servers which should be up-n-running, ie. which should restart automatically?.. Oh well.

Can't the UPS software trigger scripts or be controlled using scripts?

Regards,
Bruno

#2 lore raw
Quoted 9 lines · Bruno Prémont
Quoted 6 lines · On Wednesday 30 January 2008 20:18:40 you wrote:

I'm trying to "glue" hibernation and UPS control together, and have a question.

When the system power comes off an UPS (Uninterruptable Power Supply I mean), it's probably a good idea to turn the UPS off when shutting the system down or hibernating.

This depends on how you can turn the UPS back on...

Most UPSes has an option to turn themselves on when the power returns (except of very dumb ones).

Quoted 3 lines

If pushing the button manually is not an issue or the UPS can still be controlled remotely (e.g. via network) when being off it doesn't matter.

It does not matter really.

[]

Quoted 10 lines
Quoted 9 lines

returns. Now, when we shutting down a system, we need to turn the UPS *before* powering down the system but *after* the shutdown procedure has been completed. This is done by replacing poweroff with halt, and ordering the UPS to turn itself off after a small delay (needed for the poweroff command to complete) - this way, system will be on but in a safe-to-powercut mode, and in order to turn it back on only the UPS has to be turned on.

Quoted 5 lines

All systems I've seen that provide such an option in the BIOS list three possible actions on power-back:
- remain off
- go back to previous state
- boot

Or sometimes a subset of the 3 - like "always off or return to previous state". But this does not matter again - I'm not asking about how to control the power-on feature, or how to control a particular UPS, -- instead, I'm asking how to hook up something into the hibernate path, to the very end of it, to do whatever is needed, and/or how to control whether after the hibernate is complete the kernel should turn the system off or not.

Quoted 2 lines

In your case it's possibly easier to choose the option to always boot when power comes back.

Whatever works and is most suitable given the particular situation. It doesn't really matter here.

[]

Quoted 4 lines

Can you configure your UPS to poweroff when load on its output drops below a given threshold? (I know that such a feature exists on master-slave power outlets where all slave outlets are powered only when the master outlet has a sufficient load)

Certain models can be configured this way. But the question stands still, because many models can't be programmed this way.

What's needed is to run something at the end of hibernate process, nothing more nothing less.

Quoted 2 lines

Another path may be to dig into hibernate via kexec:
http://kerneltrap.org/node/11756

Well.. that might work after all. It doesn't work with vanilla kernel, does it? I mean, I can get *my* system to work as I like it to work, but it can't work on another Linux system...

Quoted 11 lines
Quoted 8 lines

P.S. Surprisingly, there's NO software that works with UPSes and implements even the basic shutdown process completely (not even mentioning hibernation). Most common is just to turn the system off without touching the UPS. Yes it will work (till the UPS will run out of battery), but how about servers which should be up-n-running, ie. which should restart automatically?.. Oh well.

Can't the UPS software trigger scripts or be controlled using scripts?

In some cases it can, but usually in a very limited way. It's just like no one wrote good software about this topic... ;) Or good scripts, for that matter. That's exactly what I'm trying to do, and certain questions (like this one) pops up.

Thanks.

/mjt

#3 lore raw
In reply to Michael Tokarev
Quoted 35 lines · Michael Tokarev

I'm trying to "glue" hibernation and UPS control together, and have a question.

When the system power comes off an UPS (Uninterruptable Power Supply I mean), it's probably a good idea to turn the UPS off when shutting the system down or hibernating.

Even with shutdown (not related to hibernating) there's an interesting twist: modern systems can remember the previous on/off status when the power gets cut and restores - BIOS-controlled option that allows the system to automatically start when the input power returns. Now, when we shutting down a system, we need to turn the UPS *before* powering down the system but *after* the shutdown procedure has been completed. This is done by replacing poweroff with halt, and ordering the UPS to turn itself off after a small delay (needed for the poweroff command to complete) - this way, system will be on but in a safe-to-powercut mode, and in order to turn it back on only the UPS has to be turned on.

But the question comes as how to control the UPS when we replace shutdown with hibernation. Ideally I want to send some commands to the UPS after hibernation has been completed (since I don't know how much time it will require to hibernate), but before the machine will be turned off by the kernel. Or at the very least, to be able to stop the kernel turning the machine off after hibernation process is complete - this way, I can order the UPS to turn the power off after some delay and hope hibernation process will finish when the UPS will finally cut the power (not all UPSes are able to do timely shutdown like this).

Quoted 1 line

What's my way here?

If your box hibernates and resumes correctly in the shutdown mode (ie. basically without ACPI assistance), you can use s2disk for hibernation and modify it to switch off the UPS instead of powering off the box.

Greetings,
Rafael

#4 lore raw
Quoted 3 lines · Rafael J. Wysocki
Quoted 2 lines · Michael Tokarev

I'm trying to "glue" hibernation and UPS control
together, and have a question.

[]

Quoted 1 line

If your box hibernates and resumes correctly in the shutdown mode (ie.

Ohh-well.. :)

I was thinking about more or less general solution to this. Yes, my boxes counts as primary targets, because we've seen so many various failures due to power loss and because I want to finally stop it somehow. The thing basically works with plain shutdown (no suspend is involved), but it turned out that many esp. cheap UPSes (the ones used by "masses") don't check the status of their batteries up until they lose AC power, when it's too late... The result is that when you lose AC power, you discover that you've only a few secs to perform shutdown, and hence hibernate is the only option, as it usually is much faster than to try to shut down stuff (esp. when things like squid or a big database are on the way, with their looong time to shutdown cleanly).

Quoted 2 lines

basically without ACPI assistance), you can use s2disk for hibernation and modify it to switch off the UPS instead of powering off the box.

Heh. First, I didn't know about s2disk up until now. I've seen uswsusp utils some years before, but I thought it's about some patch for vanilla kernel (some time ago it was indeed the case) and that the utils aren't needed for current kernels (I always used "echo disk > /sys/power/state" to suspend).

Now, and I hate to say that, I found that this my test box is totally unsuitable for testing this all. Because it all just Does Not Work. Because:

o 2.6.24 kernel fails to suspend properly. It "saves pages", prints "Suspending console", when prints "Sl" and nothing more happens. At this time, I can manually poweroff the machine, and resume works. The same happens when running 32bits or 64bits kernel (it's an amd x2-64 system).

o uswsusp is unaware of 64bits kernel and 32bits userland. Looking at the source, that's probably (and for sure in certain places) more than just missing compat_ioctl wrappers for those:

ioctl32(s2disk:3574): Unknown cmd fd(4) cmd(400c330d){t:'3';sz:12} arg(ffa4578c) on /dev/snapshot ioctl32(s2disk:3574): Unknown cmd fd(4) cmd(4004330a){t:'3';sz:4} arg(00000805) on /dev/snapshot

"probably" is where it reads the swap space - it's just my guess that s2disk (and resume) will do some wrong here. "for sure" is vbe code - since 32bits vbetools package fails badly with 64bit kernel, it seems that it is looking at the wrong place in /dev/mem.

o when running 32bits 2.6.24 kernel, s2disk fails the same way as pure in-kernel solution fails, above. And it fails to resume, too, -- unlike pure in-kernel solution.

I know kernel-only way worked with 2.6.23 (both 32 and 64 bits) - compiling that now...

It's sad when while working on something you discover that "underlying" tools you use also don't work, and start fixing those, when discover that something else fails, and so on, until you finally gave up and abandom the whole initial idea... Oh well, but c'est la vie...

Back to the original question and a proposed solution.

I'm looking at the uswsusp source (while the kernel compiles), and have a question here. Is it possible to call some external application (typically a shell script) to do the final work after when the image has been written? I mean in principle - I understand there are some limitations here, but I don't know which exactly.

For the given task (send some command to the damn UPS), it typically involves writing/reading something to/from a given serial port (/dev/ttySxx), or to an USB device, or to a network -- depending on the UPS in use. There are so many different UPSes out there, with so many different and stupid so-called "protocols", that it's impossible to teach s2disk about them all.

If such a thing is possible, the next question is obvious -- where exactly it belongs in suspend.c, and again, under which conditions it can be called (for example, how about different methods - shutdown/platform/etc?).

Also, if I need the system to stay on while the UPS cuts the power (in order for it to start up automatically, provided "restore to previous state on AC power resume" BIOS option is enabled), should such a script (which is called from s2disk) just sleep letting the UPS to do the work, or should it pass information back to s2disk to NOT to poweroff the machine? I mean, just like in case of "regular" shutdown when we're running off battery, between when the UPS control script gets called and when it's really safe to cut the power off, some more things has to be done - like stopping disks, "remointing" sw raid arrays read-only (to mark them as "clean") etc - this is done by calling reboot(RB_HALT) or reboot(RB_POWEROFF), which, in turn, is done by halt(8). So when s2disk calls this script, is it Okay to assume that it's safe to cut the power, or something more has to be done by s2disk when the script finishes?

I *hope* it's easy to see where my questions comes from, and what exactly I'm trying to do... ;)

(kernel compile finished, rebooting to see if s2disk works with 2.6.23 - 32bits for now, as it definitely will not work with 64bits kernel...)

/mjt

#5 lore raw

Hi!

Quoted 5 lines

o 2.6.24 kernel fails to suspend properly. It "saves pages", prints "Suspending console", when prints "Sl" and nothing more happens. At this time, I can manually poweroff the machine, and resume works. The same happens when running 32bits or 64bits kernel (it's an amd x2-64 system).

Patch welcome, it certainly works here.

Quoted 5 lines

o uswsusp is unaware of 64bits kernel and 32bits userland. Looking at the source, that's probably (and for sure in certain places) more than just missing compat_ioctl wrappers for those:

Quoted 8 lines

ioctl32(s2disk:3574): Unknown cmd fd(4) cmd(400c330d){t:'3';sz:12} arg(ffa4578c) on /dev/snapshot ioctl32(s2disk:3574): Unknown cmd fd(4) cmd(4004330a){t:'3';sz:4} arg(00000805) on /dev/snapshot

"probably" is where it reads the swap space - it's just my guess that s2disk (and resume) will do some wrong here. "for sure" is vbe code - since 32bits vbetools package fails badly with 64bit kernel, it seems that it is looking at the wrong place in /dev/mem.

Hmm.. yes, it would be nice to fix that.

Quoted 5 lines

It's sad when while working on something you discover that "underlying" tools you use also don't work, and start fixing those, when discover that something else fails, and so on, until you finally gave up and abandom the whole initial idea... Oh well, but c'est la vie...

:-).

Quoted 8 lines

Back to the original question and a proposed solution.

I'm looking at the uswsusp source (while the kernel compiles), and have a question here. Is it possible to call some external application (typically a shell script) to do the final work after when the image has been written? I mean in principle - I understand there are some limitations here, but I don't know which exactly.

No, you can't exec() anything. That would write mtime back to disk and cause badness.

Quoted 7 lines

For the given task (send some command to the damn UPS), it typically involves writing/reading something to/from a given serial port (/dev/ttySxx), or to an USB device, or to a network -- depending on the UPS in use. There are so many different UPSes out there, with so many different and stupid so-called "protocols", that it's impossible to teach s2disk about them all.

Create libups.so, and link s2disk to it?

Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#6 lore raw

Pavel Machek wrote:
[]

Quoted 9 lines
Quoted 6 lines

I'm looking at the uswsusp source (while the kernel compiles), and have a question here. Is it possible to call some external application (typically a shell script) to do the final work after when the image has been written? I mean in principle - I understand there are some limitations here, but I don't know which exactly.

No, you can't exec() anything. That would write mtime back to disk and cause badness.

Now that's.. interesting.

s2disk writes to a swap device/file, which should update mtime of this device node/file. Isn't it something which also causes the same badness?

Also, if the only concern is mtime update, what's really wrong with it? I mean, regardless of whether such update will finally hit the disk or not, there's not much difference really - it updates just mtime field, and there should be no harm in that. Unless such update first goes to a journal (in a journalling filesystem) - which DOES modify some on-disk structures.

Quoted 4 lines
Quoted 2 lines

it typically involves writing/reading something to/from a given serial port (/dev/ttySxx), or to an USB device,

Create libups.so, and link s2disk to it?

And what's the difference here again? We'll open a serial port and write something to it - which, again, will update mtime of that device node. Unless the said node is on a tmpfs, exactly the same badness will happen. Not all the world is udev, after all.

So I don't get the reason why we can't exec something here, still. (And, for example, call splashy commands as external processes, instead of linking all this cruft into s2disk and resume.)

What I'm thinking about here is - s2ram mlock()s its memory. If it will fork/exec something, that something will obviously NOT be locked like that. Is it of some concern? Probably not, because that something will be executed after we've taken the snapshot.

/mjt

#7 lore raw
Quoted 9 lines · Michael Tokarev

Pavel Machek wrote:
[]

Quoted 7 lines
Quoted 6 lines

I'm looking at the uswsusp source (while the kernel compiles), and have a question here. Is it possible to call some external application (typically a shell script) to do the final work after when the image has been written? I mean in principle - I understand there are some limitations here, but I don't know which exactly.

Quoted 8 lines
Quoted 2 lines

No, you can't exec() anything. That would write mtime back to disk and cause badness.

Now that's.. interesting.

s2disk writes to a swap device/file, which should update mtime of this device node/file. Isn't it something which also causes the same badness?

No, please read the source carefully. We create a special tmpfs for that.

Quoted 7 lines

Also, if the only concern is mtime update, what's really wrong with it? I mean, regardless of whether such update will finally hit the disk or not, there's not much difference really - it updates just mtime field, and there should be no harm in that. Unless such update first goes to a journal (in a journalling filesystem) - which DOES modify some on-disk structures.

Yes, we're concerned about the journaling.

Quoted 10 lines
Quoted 4 lines
Quoted 2 lines

it typically involves writing/reading something to/from a given serial port (/dev/ttySxx), or to an USB device,

Create libups.so, and link s2disk to it?

And what's the difference here again? We'll open a serial port and write something to it - which, again, will update mtime of that device node. Unless the said node is on a tmpfs, exactly the same badness will happen. Not all the world is udev, after all.

We already have a tmpfs for that.

Quoted 10 lines

So I don't get the reason why we can't exec something here, still. (And, for example, call splashy commands as external processes, instead of linking all this cruft into s2disk and resume.)

What I'm thinking about here is - s2ram mlock()s its memory. If it will fork/exec something, that something will obviously NOT be locked like that. Is it of some concern? Probably not, because that something will be executed after we've taken the snapshot.

Please don't exec() things from s2disk/s2both. Just link them to appropriate libraries and modify them to do things you need.

Thanks,
Rafael

#8 lore raw
In reply to Pavel Machek
Quoted 11 lines · Pavel Machek
Quoted 8 lines

Back to the original question and a proposed solution.

I'm looking at the uswsusp source (while the kernel compiles), and have a question here. Is it possible to call some external application (typically a shell script) to do the final work after when the image has been written? I mean in principle - I understand there are some limitations here, but I don't know which exactly.

No, you can't exec() anything. That would write mtime back to disk and cause badness.

SysRq-U remounts the disks readonly. Why not make the disks readonly once the kernel state is in swap, why not let any program run and give an error to writes?

I know, long dead thread, but I too would like to tell the UPS to turn off once the system state is resting safely in swap. Currently my modified apcupsd scripts tell the UPS to turn off, then tells the kernel to hibernate, and hopes the kernel finishes in time. I would much rather do things the other way, but echo disk > /sys/power/disk doesn't return until power is back on if everything goes well.

Signature · 3 lines
CC me on replies
David Fries <[email protected]>
http://fries.net/~david/ (PGP encryption key available)
#9 lore raw
Quoted 10 lines · David Fries
Quoted 9 lines · Pavel Machek
Quoted 8 lines

Back to the original question and a proposed solution.

I'm looking at the uswsusp source (while the kernel compiles), and have a question here. Is it possible to call some external application (typically a shell script) to do the final work after when the image has been written? I mean in principle - I understand there are some limitations here, but I don't know which exactly.

Quoted 6 lines
Quoted 2 lines

No, you can't exec() anything. That would write mtime back to disk and cause badness.

SysRq-U remounts the disks readonly. Why not make the disks readonly once the kernel state is in swap, why not let any program run and give an error to writes?

IIRC sysrq-U does not contain all the locking that's neccessary to make it completely safe. OTOH, it tends to work...

Quoted 6 lines

I know, long dead thread, but I too would like to tell the UPS to turn off once the system state is resting safely in swap. Currently my modified apcupsd scripts tell the UPS to turn off, then tells the kernel to hibernate, and hopes the kernel finishes in time. I would much rather do things the other way, but echo disk > /sys/power/disk doesn't return until power is back on if everything goes well.

If you use s2disk, you can execute C code at the point you want...

Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#10 lore raw
In reply to Michael Tokarev

Hi.

Quoted 35 lines · Michael Tokarev

I'm trying to "glue" hibernation and UPS control together, and have a question.

When the system power comes off an UPS (Uninterruptable Power Supply I mean), it's probably a good idea to turn the UPS off when shutting the system down or hibernating.

Even with shutdown (not related to hibernating) there's an interesting twist: modern systems can remember the previous on/off status when the power gets cut and restores - BIOS-controlled option that allows the system to automatically start when the input power returns. Now, when we shutting down a system, we need to turn the UPS *before* powering down the system but *after* the shutdown procedure has been completed. This is done by replacing poweroff with halt, and ordering the UPS to turn itself off after a small delay (needed for the poweroff command to complete) - this way, system will be on but in a safe-to-powercut mode, and in order to turn it back on only the UPS has to be turned on.

But the question comes as how to control the UPS when we replace shutdown with hibernation. Ideally I want to send some commands to the UPS after hibernation has been completed (since I don't know how much time it will require to hibernate), but before the machine will be turned off by the kernel. Or at the very least, to be able to stop the kernel turning the machine off after hibernation process is complete - this way, I can order the UPS to turn the power off after some delay and hope hibernation process will finish when the UPS will finally cut the power (not all UPSes are able to do timely shutdown like this).

Quoted 1 line

What's my way here?

That should be doable. How is your UPS connected? Presumably, with some modifications to the appropriate driver, we could send the commands when we're ready to shutdown. It would probably be useful whether or not your hibernating (if not, sending the commands could always be made an option).

Regards,

Nigel

#11 lore raw

Nigel Cunningham wrote:
[]

Quoted 4 lines

That should be doable. How is your UPS connected? Presumably, with some modifications to the appropriate driver, we could send the commands when we're ready to shutdown. It would probably be useful whether or not your hibernating (if not, sending the commands could always be made an option).

You mean adding stuff to some KERNEL driver? Like to a serial driver if the UPS is connected to a COM-port??

I'm afraid either I don't understand what you're talking about here, or, if I got you right, that YOU don't understand what you're talking about...

Come on, teaching kernel about various idiotic UPSes out there is more than insane... ;)

/mjt

#12 lore raw

Hi Michael.

Quoted 7 lines · Michael Tokarev

Nigel Cunningham wrote:
[]

Quoted 4 lines

That should be doable. How is your UPS connected? Presumably, with some modifications to the appropriate driver, we could send the commands when we're ready to shutdown. It would probably be useful whether or not your hibernating (if not, sending the commands could always be made an option).

Quoted 8 lines

You mean adding stuff to some KERNEL driver? Like to a serial driver if the UPS is connected to a COM-port??

I'm afraid either I don't understand what you're talking about here, or, if I got you right, that YOU don't understand what you're talking about...

Come on, teaching kernel about various idiotic UPSes out there is more than insane... ;)

I wasn't meaning the kernel should have to know about various idiotic UPSses (yes, I've got experience with them too, so I understand what you're talking about there). What I was thinking was that maybe it might be possible to give the kernel some simple (configurable) string to send out the UPS port when it's time to power off. Of course you're going to point out that a simple string won't do in every case (though I think it would do for at least some of the idiotic UPS ones).

I was just wondering aloud if there was a simpler way, but on further reflection, I guess there's no way around it. Whether this is done with kexec, uswsusp or TuxOnIce, it's going to involve creating (TuxOnIce / kexec ) or extending a userspace (uswsusp) binary that's got to somehow interface with the driver and get the job done.

Nigel