Skip to contentSkip to conversation

PM: Prevent frozen user mode helpers from failing the freezing of tasks

PATCH Oleg Nesterov 4 participants 10 messages Jun 25, 2007 – Jun 27, 2007

lore
Share & tools
lore
#0 lore raw
Quoted 16 lines · Rafael J. Wysocki

static int usermodehelper_pm_callback(struct notifier_block *nfb,

(quoted diff) +6 -1
 					unsigned long action,
 					void *ignored)
 {
+	long retval;
+
 	switch (action) {
 	case PM_HIBERNATION_PREPARE:
 	case PM_SUSPEND_PREPARE:
 		usermodehelper_disabled = 1;
-		return NOTIFY_OK;
+		/*
+		 * From now on call_usermodehelper_exec() won't start any new
+		 * helpers, so it is sufficient if running_helpers turns out to
+		 * be zero at one point (it may be increased later, but that

Review

Quoted 10 lines
(quoted diff) +10 -0
+		 * doesn't matter).
+		 */
+		retval = wait_event_timeout(running_helpers_waitq,
+					atomic_read(&running_helpers) == 0,
+					RUNNING_HELPERS_TIMEOUT);
+		if (retval) {
+			return NOTIFY_OK;
+		} else {
+			usermodehelper_disabled = 0;
+			return NOTIFY_BAD;

I think this is racy. First, this needs smp_mb() between "usermodehelper_disabled = 1" and wait_event_timeout().

Second, call_usermodehelper's path should first increment the counter, and only then check usermodehelper_disabled, and it needs an mb() in between too. Otherwise, the helper can see usermodehelper_disabled == 0, then PM_SUSPEND_PREPARE comes and returns NOTIFY_OK, then the helper increments the counter and starts application.

Sadly, you can't use srcu/qrcu because it doesn't handle timeouts.

Oleg.

#1 lore raw
Quoted 22 lines · Oleg Nesterov
Quoted 21 lines · Rafael J. Wysocki

static int usermodehelper_pm_callback(struct notifier_block *nfb,

(quoted diff) +11 -1
 					unsigned long action,
 					void *ignored)
 {
+	long retval;
+
 	switch (action) {
 	case PM_HIBERNATION_PREPARE:
 	case PM_SUSPEND_PREPARE:
 		usermodehelper_disabled = 1;
-		return NOTIFY_OK;
+		/*
+		 * From now on call_usermodehelper_exec() won't start any new
+		 * helpers, so it is sufficient if running_helpers turns out to
+		 * be zero at one point (it may be increased later, but that
+		 * doesn't matter).
+		 */
+		retval = wait_event_timeout(running_helpers_waitq,
+					atomic_read(&running_helpers) == 0,
+					RUNNING_HELPERS_TIMEOUT);
Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+		if (retval) {
+			return NOTIFY_OK;
+		} else {
+			usermodehelper_disabled = 0;
+			return NOTIFY_BAD;

I think this is racy. First, this needs smp_mb() between "usermodehelper_disabled = 1" and wait_event_timeout().

Yes ...

Quoted 2 lines

Second, call_usermodehelper's path should first increment the counter, and only then check usermodehelper_disabled,

It does this already.

Quoted 5 lines

and it needs an mb() in between too. Otherwise, the helper can see usermodehelper_disabled == 0, then PM_SUSPEND_PREPARE comes and returns NOTIFY_OK, then the helper increments the counter and starts application.

Sadly, you can't use srcu/qrcu because it doesn't handle timeouts.

OK

If I understand that correctly, it should suffice to place smp_mb() after usermodehelper_disabled = 1; in usermodehelper_pm_callback() and another smp_mb() after atomic_inc(&running_helpers); in new_helper().

Is that correct?

Greetings,
Rafael

Signature · 1 line
"Premature optimization is the root of all evil." - Donald Knuth
#2 lore raw
Quoted 7 lines · Rafael J. Wysocki
Quoted 2 lines · Oleg Nesterov

Second, call_usermodehelper's path should first increment the counter, and only then check usermodehelper_disabled,

It does this already.

Ah, sorry, I looked at this patch without seeing the underlying code.

BTW, isn't it better to rename new_helper/helper_finished to helper_lock/helper_unlock ?

Quoted 5 lines

If I understand that correctly, it should suffice to place smp_mb() after usermodehelper_disabled = 1; in usermodehelper_pm_callback() and another smp_mb() after atomic_inc(&running_helpers); in new_helper().

Is that correct?

Afaics, yes.

Oleg.

#3 lore raw
In reply to Oleg Nesterov
Quoted 31 lines · Oleg Nesterov
Quoted 16 lines · Rafael J. Wysocki

static int usermodehelper_pm_callback(struct notifier_block *nfb,

(quoted diff) +6 -1
 					unsigned long action,
 					void *ignored)
 {
+	long retval;
+
 	switch (action) {
 	case PM_HIBERNATION_PREPARE:
 	case PM_SUSPEND_PREPARE:
 		usermodehelper_disabled = 1;
-		return NOTIFY_OK;
+		/*
+		 * From now on call_usermodehelper_exec() won't start any new
+		 * helpers, so it is sufficient if running_helpers turns out to
+		 * be zero at one point (it may be increased later, but that

Review

Quoted 10 lines
(quoted diff) +10 -0
+		 * doesn't matter).
+		 */
+		retval = wait_event_timeout(running_helpers_waitq,
+					atomic_read(&running_helpers) == 0,
+					RUNNING_HELPERS_TIMEOUT);
+		if (retval) {
+			return NOTIFY_OK;
+		} else {
+			usermodehelper_disabled = 0;
+			return NOTIFY_BAD;

I think this is racy. First, this needs smp_mb() between "usermodehelper_disabled = 1" and wait_event_timeout().

Quoted 6 lines

Second, call_usermodehelper's path should first increment the counter, and only then check usermodehelper_disabled, and it needs an mb() in between too. Otherwise, the helper can see usermodehelper_disabled == 0, then PM_SUSPEND_PREPARE comes and returns NOTIFY_OK, then the helper increments the counter and starts application.

Sadly, you can't use srcu/qrcu because it doesn't handle timeouts.

Interesting... So the thought is to have a synchronize_srcu_timeout() or something similar that waited for a grace period to elapse or for a timeout to expire, whichever comes first? It should not be too hard to arrange something, if needed.

Thanx, Paul

#4 lore raw

synchronize_qrcu_timeout()

Quoted 9 lines · Paul E. McKenney
Quoted 2 lines · Oleg Nesterov

Sadly, you can't use srcu/qrcu because it doesn't handle timeouts.

Interesting... So the thought is to have a synchronize_srcu_timeout() or something similar that waited for a grace period to elapse or for a timeout to expire, whichever comes first? It should not be too hard to arrange something, if needed.

Yes. As for qrcu (see http://marc.info/?t=116484476500001), I think it is easy. First, we add "int interrupted" into struct qrcu_struct, then something like this

	long synchronize_qrcu_timeout(struct qrcu_struct *qp, long tout)
	{
		int idx;

		smp_mb();
		mutex_lock(&qp->mutex);

	again:
		idx = qp->completed & 0x1;

		if (unlikely(qp->interrupted)) {
			idx ^= 0x1;
			goto restart;
		}


		if (atomic_read(qp->ctr + idx) == 1)
			goto out;

		atomic_inc(qp->ctr + (idx ^ 0x1));
		qp->completed++;

		atomic_dec(qp->ctr + idx);
	restart:
		__wait_event_timeout(qp->wq, !atomic_read(qp->ctr + idx), tout);
		if (unlikely(!tout)) {
			qp->interrupted = 1;
			goto out;
		}

		if (unlikely(qp->interrupted)) {
			qp->interrupted = 0;
			goto again;
		}

	out:
		mutex_unlock(&qp->mutex);
		smp_mb();

		return tout;
	}

Of course, this needs some other changes to implement synchronize_qrcu() on top of synchronize_qrcu_timeout(). I also think that this doesn't break

	[RFC PATCH] QRCU fastpath optimization
	http://marc.info/?l=linux-kernel&m=117125577710947

Looks like we can do something similar for srcu.

Oleg.

#5 lore raw

Re: synchronize_qrcu_timeout()

Quoted 7 lines · Oleg Nesterov
Quoted 5 lines · Paul E. McKenney
Quoted 2 lines · Oleg Nesterov

Sadly, you can't use srcu/qrcu because it doesn't handle timeouts.

Quoted 7 lines
Quoted 4 lines

Interesting... So the thought is to have a synchronize_srcu_timeout() or something similar that waited for a grace period to elapse or for a timeout to expire, whichever comes first? It should not be too hard to arrange something, if needed.

Yes. As for qrcu (see http://marc.info/?t=116484476500001), I think it is easy. First, we add "int interrupted" into struct qrcu_struct, then something like this

Even simpler, we don't need ->interrupted.

	long synchronize_qrcu_timeout(struct qrcu_struct *qp, long tout)
	{
		int idx, prv;

		smp_mb();
		mutex_lock(&qp->mutex);

		idx = qp->completed & 0x1;
		prv = idx ^ 0x1;

		if (unlikely(atomic_read(qp->ctr + prv))) {
			// the previous call has not succeed,
			// finish the wait
			__wait_event_timeout(qp->wq, !atomic_read(qp->ctr + prv), tout);
			if (unlikely(!tout))
				goto out;
		}

		if (atomic_read(qp->ctr + idx) == 1)
			goto out;

		atomic_inc(qp->ctr + prv);
		qp->completed++;

		atomic_dec(qp->ctr + idx);
		__wait_event_timeout(qp->wq, !atomic_read(qp->ctr + idx), tout);
	out:
		mutex_unlock(&qp->mutex);
		smp_mb();

		return tout;
	}

Oleg.

#6 lore raw

Re: synchronize_qrcu_timeout()

Quoted 13 lines · Oleg Nesterov
Quoted 12 lines · Oleg Nesterov
Quoted 9 lines · Paul E. McKenney
Quoted 2 lines · Oleg Nesterov

Sadly, you can't use srcu/qrcu because it doesn't handle timeouts.

Interesting... So the thought is to have a synchronize_srcu_timeout() or something similar that waited for a grace period to elapse or for a timeout to expire, whichever comes first? It should not be too hard to arrange something, if needed.

Quoted 8 lines
Quoted 2 lines

Yes. As for qrcu (see http://marc.info/?t=116484476500001), I think it is easy. First, we add "int interrupted" into struct qrcu_struct, then something like this

Even simpler, we don't need ->interrupted.

	long synchronize_qrcu_timeout(struct qrcu_struct *qp, long tout)
	{
		int idx, prv;

Could we get less encrypted variable names? tout? Yes, I can decipher it.
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#7 lore raw

Re: synchronize_qrcu_timeout()

In reply to Oleg Nesterov
Quoted 13 lines · Oleg Nesterov
Quoted 12 lines · Oleg Nesterov
Quoted 9 lines · Paul E. McKenney
Quoted 2 lines · Oleg Nesterov

Sadly, you can't use srcu/qrcu because it doesn't handle timeouts.

Interesting... So the thought is to have a synchronize_srcu_timeout() or something similar that waited for a grace period to elapse or for a timeout to expire, whichever comes first? It should not be too hard to arrange something, if needed.

Quoted 4 lines
Quoted 2 lines

Yes. As for qrcu (see http://marc.info/?t=116484476500001), I think it is easy. First, we add "int interrupted" into struct qrcu_struct, then something like this

Even simpler, we don't need ->interrupted.

I have to ask...

What sorts of performance characteristics are needed here? The reason that I ask is because putting a straight synchronize_qrcu() into a workqueue (or something similar) and then using a timer to provide any needed wakeup seems a lot simpler than rearranging the innards of synchronize_qrcu().

(Yes, I am feeling cowardly. Why do you ask?)

Thanx, Paul

Quoted 41 lines
	long synchronize_qrcu_timeout(struct qrcu_struct *qp, long tout)
	{
		int idx, prv;

		smp_mb();
		mutex_lock(&qp->mutex);

		idx = qp->completed & 0x1;
		prv = idx ^ 0x1;

		if (unlikely(atomic_read(qp->ctr + prv))) {
			// the previous call has not succeed,
			// finish the wait
			__wait_event_timeout(qp->wq, !atomic_read(qp->ctr + prv), tout);
			if (unlikely(!tout))
				goto out;
		}

		if (atomic_read(qp->ctr + idx) == 1)
			goto out;

		atomic_inc(qp->ctr + prv);
		qp->completed++;

		atomic_dec(qp->ctr + idx);
		__wait_event_timeout(qp->wq, !atomic_read(qp->ctr + idx), tout);
	out:
		mutex_unlock(&qp->mutex);
		smp_mb();

		return tout;
	}

Oleg.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#8 lore raw

Re: synchronize_qrcu_timeout()

Quoted 20 lines · Paul E. McKenney
Quoted 13 lines · Oleg Nesterov
Quoted 12 lines · Oleg Nesterov
Quoted 9 lines · Paul E. McKenney
Quoted 2 lines · Oleg Nesterov

Sadly, you can't use srcu/qrcu because it doesn't handle timeouts.

Interesting... So the thought is to have a synchronize_srcu_timeout() or something similar that waited for a grace period to elapse or for a timeout to expire, whichever comes first? It should not be too hard to arrange something, if needed.

Quoted 4 lines
Quoted 2 lines

Yes. As for qrcu (see http://marc.info/?t=116484476500001), I think it is easy. First, we add "int interrupted" into struct qrcu_struct, then something like this

Even simpler, we don't need ->interrupted.

Quoted 7 lines

I have to ask...

What sorts of performance characteristics are needed here? The reason that I ask is because putting a straight synchronize_qrcu() into a workqueue (or something similar) and then using a timer to provide any needed wakeup seems a lot simpler than rearranging the innards of synchronize_qrcu().

Didn't think about that... But yes, we can implement synchronize_qrcu_timeout() on top of synchronize_qrcu() as you suggested. Hovewer, this implementation will add more code to .text compared to changing synchronize_qrcu().

Note also that we can't share the "context" of synchronize_qrcu_timeout() in that case. Each caller of synchronize_qrcu_timeout() needs a separate wait_queue_head_t + work_struct. This context can't live on stack, because it should survive after the timeout.

If we change synchronize_qrcu() instead, we only add

if (unlikely(atomic_read(qp->ctr + prv))) {
__wait_event_timeout(qp->wq, !atomic_read(qp->ctr + prv), tout); if (unlikely(!tout))
goto out;
}

to the slow path. This has nearly zero perfomance penalty. Yes, we have to wait if the previous call was interrupted by timeout. But in that case we lost nothing, we spend the same time waiting for qp->mutex when the previous call succeeds.

That said, I am not sure synchronize_Xrcu_timeout() is terribly useful. Rafael could use it, but it is not better for this particular case. Except the code re-use, which is always good.

Oleg.

#9 lore raw

Re: synchronize_qrcu_timeout()

Quoted 53 lines · Oleg Nesterov
Quoted 20 lines · Paul E. McKenney
Quoted 13 lines · Oleg Nesterov
Quoted 12 lines · Oleg Nesterov
Quoted 9 lines · Paul E. McKenney
Quoted 2 lines · Oleg Nesterov

Sadly, you can't use srcu/qrcu because it doesn't handle timeouts.

Interesting... So the thought is to have a synchronize_srcu_timeout() or something similar that waited for a grace period to elapse or for a timeout to expire, whichever comes first? It should not be too hard to arrange something, if needed.

Quoted 4 lines
Quoted 2 lines

Yes. As for qrcu (see http://marc.info/?t=116484476500001), I think it is easy. First, we add "int interrupted" into struct qrcu_struct, then something like this

Even simpler, we don't need ->interrupted.

Quoted 7 lines

I have to ask...

What sorts of performance characteristics are needed here? The reason that I ask is because putting a straight synchronize_qrcu() into a workqueue (or something similar) and then using a timer to provide any needed wakeup seems a lot simpler than rearranging the innards of synchronize_qrcu().

Didn't think about that... But yes, we can implement synchronize_qrcu_timeout() on top of synchronize_qrcu() as you suggested. Hovewer, this implementation will add more code to .text compared to changing synchronize_qrcu().

Note also that we can't share the "context" of synchronize_qrcu_timeout() in that case. Each caller of synchronize_qrcu_timeout() needs a separate wait_queue_head_t + work_struct. This context can't live on stack, because it should survive after the timeout.

If we change synchronize_qrcu() instead, we only add

(quoted diff) +0 -0
 		if (unlikely(atomic_read(qp->ctr + prv))) {
 			__wait_event_timeout(qp->wq, !atomic_read(qp->ctr + prv), tout);
 			if (unlikely(!tout))
 				goto out;
 		}

to the slow path. This has nearly zero perfomance penalty. Yes, we have to wait if the previous call was interrupted by timeout. But in that case we lost nothing, we spend the same time waiting for qp->mutex when the previous call succeeds.

Quoted 3 lines

That said, I am not sure synchronize_Xrcu_timeout() is terribly useful. Rafael could use it, but it is not better for this particular case. Except the code re-use, which is always good.

Well, if there are more cases in which it's useful, we'll be able to modify this code to use it too in the future.

Greetings,
Rafael

Signature · 1 line
"Premature optimization is the root of all evil." - Donald Knuth