PM / Hibernate: Fix s2disk regression related to unlock_system_sleep()

From: Rafael J. Wysocki <[email protected]>
Subject: PM / Hibernate: Fix s2disk regression related to unlock_system_sleep()

Commit bcda53faf5814c0c6025a0bd47108adfcbe9f199, "PM / Sleep: Replace mutex_[un]lock(&pm_mutex) with [un]lock_system_sleep()", modified snapshot_read() and snapshot_write() in kernel/power/user.c, among other things, by making them use lock_system_sleep() and unlock_system_sleep() instead of just locking and unlocking pm_mutex. Unfortunately, however, this was a mistake, because these routines are supposed to be executed after processes have been frozen (i.e. when system_freezing_cnt is nonzero), so when unlock_system_sleep() is executed by one of them, this causes the caller to execute try_to_freeze() and go into the refrigerator as a result. This, in turn, deadlocks the suspend process and locks up the system.

Fix the problem by reverting the part of commit bcda53faf5814c0c6025a that changed snapshot_read() and snapshot_write().

Signed-off-by Rafael J. Wysocki <[email protected]>

---

+       /*
+        * Don't use freezer_count() because we don't want the
+        * call to try_to_freeze() here.
+        */

Hello,

Review

Quoted 4 lines · Srivatsa S. Bhat

(quoted diff) +4 -0

+       /*
+        * Don't use freezer_count() because we don't want the
+        * call to try_to_freeze() here.
+        */

Please explain "why" here.

+       /*
+        * Don't use freezer_count() because we don't want the call to
+        * try_to_freeze() here.
+        *
+        * Reason:
+        * unlock_system_sleep() gets called in snapshot_read() and
+        * snapshot_write() when the freezing condition is still in effect.
+        * Which means, if we use try_to_freeze() here, it would make them
+        * enter the refrigerator, thus causing suspend-to-disk to lockup.
+        */

Hello,

include/linux/suspend.h static inline void lock_system_sleep(void) · line 377

Quoted 10 lines · Srivatsa S. Bhat

include/linux/suspend.h +10 -0

+       /*
+        * Don't use freezer_count() because we don't want the call to
+        * try_to_freeze() here.
+        *
+        * Reason:
+        * unlock_system_sleep() gets called in snapshot_read() and
+        * snapshot_write() when the freezing condition is still in effect.
+        * Which means, if we use try_to_freeze() here, it would make them
+        * enter the refrigerator, thus causing suspend-to-disk to lockup.
+        */

Yeap, much better but wouldn't it be better if the description was higher level? ie. Explain why try_to_freeze() doesn't make sense for this operation semantically and then, optionally, give an example of breakage. It usually is lack of proper rationale / reasoning that confuses people reading the code.

From the above depiction, it is clear that try_to_freeze() at Point 1 is

I agree, but I was trying to keep the comment from growing too long ;)

Hello, Srivatsa.

Quoted 1 line · Srivatsa S. Bhat

I agree, but I was trying to keep the comment from growing too long ;)

It doesn't have to be long. It just has to give some meaning to the decision. AFAICS, it is correct to call try_to_freeze() on unlock_system_sleep() regardless of 20sec window. There's no guarantee the unlocking task is gonna hit try_to_freeze() elsewhere and not calling it actually makes the interface buggy.

Quoted 7 lines

Quoted 1 line

I agree, but I was trying to keep the comment from growing too long ;)

It doesn't have to be long. It just has to give some meaning to the decision. AFAICS, it is correct to call try_to_freeze() on unlock_system_sleep() regardless of 20sec window. There's no guarantee the unlocking task is gonna hit try_to_freeze() elsewhere and not calling it actually makes the interface buggy.

To give an example,

/*
* XXX: Open code SKIP clearing for now to avoid invoking
* try_to_freeze(). This isn't correct but this function is
* called from deep inside hibernation path and calling
* try_to_freeze() leads to hang during hibernation. This
* will be properly fixed soon. See commit message for
* more details.
*/

Hi!

Quoted 10 lines

Quoted 7 lines

Quoted 1 line

I agree, but I was trying to keep the comment from growing too long ;)

It doesn't have to be long. It just has to give some meaning to the decision. AFAICS, it is correct to call try_to_freeze() on unlock_system_sleep() regardless of 20sec window. There's no guarantee the unlocking task is gonna hit try_to_freeze() elsewhere and not calling it actually makes the interface buggy.

To give an example,

Quoted 8 lines

/*
* XXX: Open code SKIP clearing for now to avoid invoking
* try_to_freeze(). This isn't correct but this function is
* called from deep inside hibernation path and calling
* try_to_freeze() leads to hang during hibernation. This
* will be properly fixed soon. See commit message for
* more details.
*/

Which paths are affected?

With uswsusp, we have userland in control of hibernation process; I'd say almost anything can be called...

Quoted 13 lines

Quoted 10 lines

To give an example,

/*
* XXX: Open code SKIP clearing for now to avoid invoking
* try_to_freeze(). This isn't correct but this function is
* called from deep inside hibernation path and calling
* try_to_freeze() leads to hang during hibernation. This
* will be properly fixed soon. See commit message for
* more details.
*/

Which paths are affected?

Quoted 2 lines

With uswsusp, we have userland in control of hibernation process; I'd say almost anything can be called...

Hi Pavel,
I am afraid you are looking at a stale comment. We finalized on a different comment altogether.

Hi!

Quoted 17 lines

Quoted 15 lines

Quoted 10 lines

To give an example,

/*
* XXX: Open code SKIP clearing for now to avoid invoking
* try_to_freeze(). This isn't correct but this function is
* called from deep inside hibernation path and calling
* try_to_freeze() leads to hang during hibernation. This
* will be properly fixed soon. See commit message for
* more details.
*/

Which paths are affected?

With uswsusp, we have userland in control of hibernation process; I'd say almost anything can be called...

Quoted 3 lines

Hi Pavel,
I am afraid you are looking at a stale comment. We finalized on a different comment altogether.

I know. But the underlying problem (uswsusp code is userland, and can do anything) is still there, right? I guess limitations for applications using uswsusp interface should be documented somewhere.

Hello, Srivatsa.

Quoted 1 line · Srivatsa S. Bhat

I agree, but I was trying to keep the comment from growing too long ;)

It doesn't have to be long. It just has to give some meaning to the decision. AFAICS, it is correct to call try_to_freeze() on unlock_system_sleep() regardless of 20sec window. There's no guarantee the unlocking task is gonna hit try_to_freeze() elsewhere and not calling it actually makes the interface buggy.

That said, it causes a problem because unlock_system_sleep() is called in a special context during later stage of hibernation where the usual expectation - that a freezable task which sees a freezing condition should freeze - doesn't hold.

The correct solution would be somehow marking that condition so that either try_to_freeze() doesn't get invoked or gets nullified -

e.g. making the SKIP thing a counter and ensure the hibernating task has it elevated throughout the whole process. Alternatively, if the code path is limited enough, using a different version of the unlock function, unlock_system_sleep_nofreeze() or whatever, would work too - this is a popular approach for synchronization functions which interacts with scheduler and preemption.

For now, as a quick fix, maybe not calling try_to_freeze() unconditionally is okay, I don't know, but it's a hack.

Somehow I don't think its a hack, based on my perception as described above. But feel free to prove me wrong :-)

Hello,

Quoted 2 lines · Srivatsa S. Bhat

Somehow I don't think its a hack, based on my perception as described above. But feel free to prove me wrong :-)

Thanks for the explanation. Yeah, I agree and it's much simpler this way, which is nice. So, in short, because freezing state can't change across lock_system_sleep(), there's no reason to check for freezing state on unlock and this nicely resolves the freezer problem together.

The only thing to be careful is, then, we need to set and clear SKIP inside pm_mutex.

Quoted 11 lines · Tejun Heo

Hello,

Quoted 2 lines · Srivatsa S. Bhat

Somehow I don't think its a hack, based on my perception as described above. But feel free to prove me wrong :-)

Thanks for the explanation. Yeah, I agree and it's much simpler this way, which is nice. So, in short, because freezing state can't change across lock_system_sleep(), there's no reason to check for freezing state on unlock and this nicely resolves the freezer problem together.

Absolutely!

Quoted 3 lines

The only thing to be careful is, then, we need to set and clear SKIP inside pm_mutex.

Not exactly. We need to set SKIP before grabbing pm_mutex and clear it inside pm_mutex. The reason is that we decided to set SKIP in the first place just to avoid the freezer from declaring failure when we are blocked on pm_mutex. If we move it to *after* mutex_lock(&pm_mutex), that original intention itself is not satisfied, and we will hit freezing failures - IOW making the set and clear exercise useless!

So, something like this should work perfectly:

lock_system_sleep()

{
	freezer_do_not_count();
	mutex_lock(&pm_mutex);
	current->flags &= ~PF_FREEZER_SKIP;
}

But in the interest of making the code look a bit symmetric, we can do:

lock_system_sleep()

{
	freezer_do_not_count();
	mutex_lock(&pm_mutex);
}

unlock_system_sleep()

{
	current->flags &= ~PF_FREEZER_SKIP;
	mutex_unlock(&pm_mutex);
}

From: Srivatsa S. Bhat <[email protected]>
Subject: [PATCH] PM / Hibernate: Rewrite unlock_system_sleep() to fix s2disk regression

Commit 33e638b, "PM / Sleep: Use the freezer_count() functions in [un]lock_system_sleep() APIs" introduced an undesirable change in the behaviour of unlock_system_sleep() since freezer_count() internally calls try_to_freeze() - which we don't need in unlock_system_sleep().

And commit bcda53f, "PM / Sleep: Replace mutex_[un]lock(&pm_mutex) with [un]lock_system_sleep()" made these APIs wide-spread. This caused a regression in suspend-to-disk where snapshot_read() and snapshot_write() were getting frozen due to the try_to_freeze embedded in unlock_system_sleep(), since these functions were invoked when the freezing condition was still in effect.

Fix this by rewriting unlock_system_sleep() by open-coding freezer_count() and dropping the try_to_freeze() part. Not only will this fix the regression but this will also ensure that the API only does what it is intended to do, and nothing more, under the hood.

While at it, make the code more correct and robust by ensuring that the PF_FREEZER_SKIP flag gets cleared with pm_mutex held, to avoid a race with the freezer.

Reported-by Rafael J. Wysocki <[email protected]>

Signed-off-by Srivatsa S. Bhat <[email protected]>

Quoted 46 lines · Srivatsa S. Bhat

Quoted 11 lines · Tejun Heo

Hello,

Quoted 2 lines · Srivatsa S. Bhat

Somehow I don't think its a hack, based on my perception as described above. But feel free to prove me wrong :-)

Thanks for the explanation. Yeah, I agree and it's much simpler this way, which is nice. So, in short, because freezing state can't change across lock_system_sleep(), there's no reason to check for freezing state on unlock and this nicely resolves the freezer problem together.

Absolutely!

Quoted 3 lines

The only thing to be careful is, then, we need to set and clear SKIP inside pm_mutex.

Not exactly. We need to set SKIP before grabbing pm_mutex and clear it inside pm_mutex. The reason is that we decided to set SKIP in the first place just to avoid the freezer from declaring failure when we are blocked on pm_mutex. If we move it to *after* mutex_lock(&pm_mutex), that original intention itself is not satisfied, and we will hit freezing failures - IOW making the set and clear exercise useless!

So, something like this should work perfectly:

lock_system_sleep()

{
	freezer_do_not_count();
	mutex_lock(&pm_mutex);
	current->flags &= ~PF_FREEZER_SKIP;
}

But in the interest of making the code look a bit symmetric, we can do:

lock_system_sleep()

{
	freezer_do_not_count();
	mutex_lock(&pm_mutex);
}

Quoted 6 lines

unlock_system_sleep()

{
	current->flags &= ~PF_FREEZER_SKIP;
	mutex_unlock(&pm_mutex);
}

So how about this patch, with the comment updated?
(and the changelog updated as well to reflect the movement of code)

---
From: Srivatsa S. Bhat <[email protected]>
Subject: [PATCH] PM / Hibernate: Rewrite unlock_system_sleep() to fix s2disk regression

Commit 33e638b, "PM / Sleep: Use the freezer_count() functions in [un]lock_system_sleep() APIs" introduced an undesirable change in the behaviour of unlock_system_sleep() since freezer_count() internally calls try_to_freeze() - which we don't need in unlock_system_sleep().

And commit bcda53f, "PM / Sleep: Replace mutex_[un]lock(&pm_mutex) with [un]lock_system_sleep()" made these APIs wide-spread. This caused a regression in suspend-to-disk where snapshot_read() and snapshot_write() were getting frozen due to the try_to_freeze embedded in unlock_system_sleep(), since these functions were invoked when the freezing condition was still in effect.

Fix this by rewriting unlock_system_sleep() by open-coding freezer_count() and dropping the try_to_freeze() part. Not only will this fix the regression but this will also ensure that the API only does what it is intended to do, and nothing more, under the hood.

While at it, make the code more correct and robust by ensuring that the PF_FREEZER_SKIP flag gets cleared with pm_mutex held, to avoid a race with the freezer.

Reported-by Rafael J. Wysocki <[email protected]>

Signed-off-by Srivatsa S. Bhat <[email protected]>

---

 include/linux/suspend.h |   17 ++++++++++++++++-
 1 files changed, 16 insertions(+), 1 deletions(-)

include/linux/suspend.h +16 -2

index 95040cc..cb9d3f4 100644
@@ -363,8 +363,23 @@ static inline void lock_system_sleep(void)
 
 static inline void unlock_system_sleep(void)
 {
+	/*
+	 * Don't use freezer_count() because we don't want the call to
+	 * try_to_freeze() here.
+	 *
+	 * Reason:
+	 * Fundamentally, we just don't need it, because freezing condition
+	 * doesn't come into effect until we release the pm_mutex lock,
+	 * since the freezer always works with pm_mutex held.
+	 *
+	 * More importantly, in the case of hibernation,
+	 * unlock_system_sleep() gets called in snapshot_read() and
+	 * snapshot_write() when the freezing condition is still in effect.
+	 * Which means, if we use try_to_freeze() here, it would make them
+	 * enter the refrigerator, thus causing hibernation to lockup.
+	 */
+	current->flags &= ~PF_FREEZER_SKIP;
 	mutex_unlock(&pm_mutex);
-	freezer_count();
 }
 
 #else /* !CONFIG_PM_SLEEP */
 
 
--

To unsubscribe from this list: send the line "unsubscribe linux-pm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 84 lines · Srivatsa S. Bhat

Quoted 46 lines · Srivatsa S. Bhat

Quoted 11 lines · Tejun Heo

Hello,

Quoted 2 lines · Srivatsa S. Bhat

Somehow I don't think its a hack, based on my perception as described above. But feel free to prove me wrong :-)

Thanks for the explanation. Yeah, I agree and it's much simpler this way, which is nice. So, in short, because freezing state can't change across lock_system_sleep(), there's no reason to check for freezing state on unlock and this nicely resolves the freezer problem together.

Absolutely!

Quoted 3 lines

The only thing to be careful is, then, we need to set and clear SKIP inside pm_mutex.

Not exactly. We need to set SKIP before grabbing pm_mutex and clear it inside pm_mutex. The reason is that we decided to set SKIP in the first place just to avoid the freezer from declaring failure when we are blocked on pm_mutex. If we move it to *after* mutex_lock(&pm_mutex), that original intention itself is not satisfied, and we will hit freezing failures - IOW making the set and clear exercise useless!

So, something like this should work perfectly:

lock_system_sleep()

{
	freezer_do_not_count();
	mutex_lock(&pm_mutex);
	current->flags &= ~PF_FREEZER_SKIP;
}

But in the interest of making the code look a bit symmetric, we can do:

lock_system_sleep()

{
	freezer_do_not_count();
	mutex_lock(&pm_mutex);
}

Quoted 6 lines

unlock_system_sleep()

{
	current->flags &= ~PF_FREEZER_SKIP;
	mutex_unlock(&pm_mutex);
}

So how about this patch, with the comment updated?
(and the changelog updated as well to reflect the movement of code)

---
From: Srivatsa S. Bhat <[email protected]>
Subject: [PATCH] PM / Hibernate: Rewrite unlock_system_sleep() to fix s2disk regression

Commit 33e638b, "PM / Sleep: Use the freezer_count() functions in [un]lock_system_sleep() APIs" introduced an undesirable change in the behaviour of unlock_system_sleep() since freezer_count() internally calls try_to_freeze() - which we don't need in unlock_system_sleep().

And commit bcda53f, "PM / Sleep: Replace mutex_[un]lock(&pm_mutex) with [un]lock_system_sleep()" made these APIs wide-spread. This caused a regression in suspend-to-disk where snapshot_read() and snapshot_write() were getting frozen due to the try_to_freeze embedded in unlock_system_sleep(), since these functions were invoked when the freezing condition was still in effect.

Fix this by rewriting unlock_system_sleep() by open-coding freezer_count() and dropping the try_to_freeze() part. Not only will this fix the regression but this will also ensure that the API only does what it is intended to do, and nothing more, under the hood.

While at it, make the code more correct and robust by ensuring that the PF_FREEZER_SKIP flag gets cleared with pm_mutex held, to avoid a race with the freezer.

Quoted 2 lines

Reported-by Rafael J. Wysocki <[email protected]>

Signed-off-by Srivatsa S. Bhat <[email protected]>

So, I'd prefer lock_system_sleep() to be open-coded too, to avoid situations in which someone changes freezer_do_not_count() and freezer_count() leaving unmodified unlock_system_sleep() behind.

Also, have you actually tested this?

Quoted 85 lines · Rafael J. Wysocki

Quoted 84 lines · Srivatsa S. Bhat

Quoted 46 lines · Srivatsa S. Bhat

Quoted 11 lines · Tejun Heo

Hello,

Quoted 2 lines · Srivatsa S. Bhat

Somehow I don't think its a hack, based on my perception as described above. But feel free to prove me wrong :-)

Thanks for the explanation. Yeah, I agree and it's much simpler this way, which is nice. So, in short, because freezing state can't change across lock_system_sleep(), there's no reason to check for freezing state on unlock and this nicely resolves the freezer problem together.

Absolutely!

Quoted 3 lines

The only thing to be careful is, then, we need to set and clear SKIP inside pm_mutex.

Not exactly. We need to set SKIP before grabbing pm_mutex and clear it inside pm_mutex. The reason is that we decided to set SKIP in the first place just to avoid the freezer from declaring failure when we are blocked on pm_mutex. If we move it to *after* mutex_lock(&pm_mutex), that original intention itself is not satisfied, and we will hit freezing failures - IOW making the set and clear exercise useless!

So, something like this should work perfectly:

lock_system_sleep()

{
	freezer_do_not_count();
	mutex_lock(&pm_mutex);
	current->flags &= ~PF_FREEZER_SKIP;
}

But in the interest of making the code look a bit symmetric, we can do:

lock_system_sleep()

{
	freezer_do_not_count();
	mutex_lock(&pm_mutex);
}

Quoted 6 lines

unlock_system_sleep()

{
	current->flags &= ~PF_FREEZER_SKIP;
	mutex_unlock(&pm_mutex);
}

So how about this patch, with the comment updated?
(and the changelog updated as well to reflect the movement of code)

---
From: Srivatsa S. Bhat <[email protected]>
Subject: [PATCH] PM / Hibernate: Rewrite unlock_system_sleep() to fix s2disk regression

Commit 33e638b, "PM / Sleep: Use the freezer_count() functions in [un]lock_system_sleep() APIs" introduced an undesirable change in the behaviour of unlock_system_sleep() since freezer_count() internally calls try_to_freeze() - which we don't need in unlock_system_sleep().

And commit bcda53f, "PM / Sleep: Replace mutex_[un]lock(&pm_mutex) with [un]lock_system_sleep()" made these APIs wide-spread. This caused a regression in suspend-to-disk where snapshot_read() and snapshot_write() were getting frozen due to the try_to_freeze embedded in unlock_system_sleep(), since these functions were invoked when the freezing condition was still in effect.

Fix this by rewriting unlock_system_sleep() by open-coding freezer_count() and dropping the try_to_freeze() part. Not only will this fix the regression but this will also ensure that the API only does what it is intended to do, and nothing more, under the hood.

While at it, make the code more correct and robust by ensuring that the PF_FREEZER_SKIP flag gets cleared with pm_mutex held, to avoid a race with the freezer.

Quoted 7 lines

Quoted 2 lines

Reported-by Rafael J. Wysocki <[email protected]>

Signed-off-by Srivatsa S. Bhat <[email protected]>

So, I'd prefer lock_system_sleep() to be open-coded too, to avoid situations in which someone changes freezer_do_not_count() and freezer_count() leaving unmodified unlock_system_sleep() behind.

Hehe, actually, I had left lock_system_sleep() as it is, intentionally, due to a far-fetched idea to keep the code sane: I felt that people who go through this code would naturally feel tempted to replace any open-coded stuff (if possible) with the appropriate function call. And my rationale was that when they see that lock_system_sleep() has a function call but unlock_system_sleep() has been open-coded, they would immediately know that this was not yet another case of "oh I wasn't aware of this function, hence I open-coded it", but it was rather "I was very much aware of the available functions, but I chose to open-code it intentionally - so touch it only if you really know what you are doing" :-)

And I guess, now that job of guarding the code from "obvious code cleanups" like that would be done by the comment I added as per Tejun's suggestion. And moreover, your point to open-code in lock_system_sleep() sounds more compelling than my far-fetched idea :-) So okay, I will open-code it..

Quoted 1 line

Also, have you actually tested this?

Oh I almost forgot to mention: I have not specifically tested this with the userspace s2disk utility (part of uswsusp/suspend-utils package) as such. I have only tested it in general (by invoking hibernate by writing to /sys/power/disk, and testing different levels of pm_test framework etc) to ensure that this patch doesn't break anything else.
So, any help with testing the userspace utilities would be great!

So Rafael, could you please check if the following patch solves the regression for you? Thanks a lot!

---
From: Srivatsa S. Bhat <[email protected]>
Subject: [PATCH] PM / Hibernate: Rewrite unlock_system_sleep() to fix s2disk regression

Commit 33e638b, "PM / Sleep: Use the freezer_count() functions in [un]lock_system_sleep() APIs" introduced an undesirable change in the behaviour of unlock_system_sleep() since freezer_count() internally calls try_to_freeze() - which we don't need in unlock_system_sleep().

And commit bcda53f, "PM / Sleep: Replace mutex_[un]lock(&pm_mutex) with [un]lock_system_sleep()" made these APIs wide-spread. This caused a regression in suspend-to-disk where snapshot_read() and snapshot_write() were getting frozen due to the try_to_freeze embedded in unlock_system_sleep(), since these functions were invoked when the freezing condition was still in effect.

Fix this by rewriting unlock_system_sleep() by open-coding freezer_count() and dropping the try_to_freeze() part. Not only will this fix the regression but this will also ensure that the API only does what it is intended to do, and nothing more, under the hood.

While at it, make the code more correct and robust by ensuring that the PF_FREEZER_SKIP flag gets cleared with pm_mutex held, to avoid a race with the freezer.

Also, to be on the safer side, open-code freezer_do_not_count() as well (inside lock_system_sleep()), to ensure that any unrelated modification to freezer[_do_not]_count() does not break things again!

Reported-by Rafael J. Wysocki <[email protected]>

Acked-by Tejun Heo <[email protected]>

Signed-off-by Srivatsa S. Bhat <[email protected]>

---

 include/linux/suspend.h |   19 +++++++++++++++++--
 1 files changed, 17 insertions(+), 2 deletions(-)

include/linux/suspend.h +17 -3

index 95040cc..91784a4 100644
@@ -357,14 +357,29 @@ extern bool pm_save_wakeup_count(unsigned int count);
 
 static inline void lock_system_sleep(void)
 {
-	freezer_do_not_count();
+	current->flags |= PF_FREEZER_SKIP;
 	mutex_lock(&pm_mutex);
 }
 
 static inline void unlock_system_sleep(void)
 {
+	/*
+	 * Don't use freezer_count() because we don't want the call to
+	 * try_to_freeze() here.
+	 *
+	 * Reason:
+	 * Fundamentally, we just don't need it, because freezing condition
+	 * doesn't come into effect until we release the pm_mutex lock,
+	 * since the freezer always works with pm_mutex held.
+	 *
+	 * More importantly, in the case of hibernation,
+	 * unlock_system_sleep() gets called in snapshot_read() and
+	 * snapshot_write() when the freezing condition is still in effect.
+	 * Which means, if we use try_to_freeze() here, it would make them
+	 * enter the refrigerator, thus causing hibernation to lockup.
+	 */
+	current->flags &= ~PF_FREEZER_SKIP;
 	mutex_unlock(&pm_mutex);
-	freezer_count();
 }
 
 #else /* !CONFIG_PM_SLEEP */
 
 
--

To unsubscribe from this list: send the line "unsubscribe linux-pm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html