Skip to contentSkip to conversation

Thoughts on this RCU idle entry/exit patch?

PATCH Paul E. McKenney 2 participants 5 messages Oct 7, 2013 – Oct 9, 2013

lore
Share & tools
lore
#0 lore raw

Hello, Frederic!

The following patch seems to me to be a good idea to better handle task nesting. Any reason why it would be a bad thing?

Thanx, Paul

------------------------------------------------------------------------

rcu: Allow task-level idle entry/exit nesting

The current task-level idle entry/exit code forces an entry/exit on each call, regardless of the nesting level. This commit therefore properly accounts for nesting.

kernel/rcutree.c +8 -6
index 106f7f5cdd1d..f0be20886617 100644
@@ -411,11 +411,12 @@ static void rcu_eqs_enter(bool user)
 	rdtp = this_cpu_ptr(&rcu_dynticks);
 	oldval = rdtp->dynticks_nesting;
 	WARN_ON_ONCE((oldval & DYNTICK_TASK_NEST_MASK) == 0);
-	if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE)
+	if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE) {
 		rdtp->dynticks_nesting = 0;
-	else
+		rcu_eqs_enter_common(rdtp, oldval, user);
+	} else {
 		rdtp->dynticks_nesting -= DYNTICK_TASK_NEST_VALUE;
-	rcu_eqs_enter_common(rdtp, oldval, user);
+	}
 }
 
 /**
@@ -533,11 +534,12 @@ static void rcu_eqs_exit(bool user)
 	rdtp = this_cpu_ptr(&rcu_dynticks);
 	oldval = rdtp->dynticks_nesting;
 	WARN_ON_ONCE(oldval < 0);
-	if (oldval & DYNTICK_TASK_NEST_MASK)
+	if (oldval & DYNTICK_TASK_NEST_MASK) {
 		rdtp->dynticks_nesting += DYNTICK_TASK_NEST_VALUE;
-	else
+	} else {
 		rdtp->dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
-	rcu_eqs_exit_common(rdtp, oldval, user);
+		rcu_eqs_exit_common(rdtp, oldval, user);
+	}
 }
 
 /**
#1 lore raw
Quoted 9 lines · Paul E. McKenney

Hello, Frederic!

The following patch seems to me to be a good idea to better handle task nesting. Any reason why it would be a bad thing?

Thanx, Paul

------------------------------------------------------------------------

Quoted 7 lines

rcu: Allow task-level idle entry/exit nesting

The current task-level idle entry/exit code forces an entry/exit on each call, regardless of the nesting level. This commit therefore properly accounts for nesting.

Looks good. In fact, the current code is even buggy because two nesting rcu_user_eqs() as in:

        rcu_eqs_enter()
            rcu_eqs_enter()
            rcu_eqs_exit()
        rcu_eqs_exit()

would result in rdtp->dynticks wrong increment, right?

So that's even a bug fix. I wonder if it's a regression. That said rcu_eqs_enter_common() should warn on such miscount, so may be these functions actually don't nest in practice or you would have received such warnings.

So I wonder, do we want to continue to allow this nesting? I remember that DYNTICK_TASK_NEST_* stuff is there to protects against non finishing interrupts on some archs (I also remember that this, or at least a practical scenario for this, was hard to really define though :o) But then wouldn't it involve other kind of scenario like this?

       rcu_irq_enter()
           rcu_eqs_enter()
           rcu_eqs_exit()
           ...

Anyway, that's just random thougths on further simplifications, in any case, this patch looks good.

Thanks.

Quoted 38 lines
kernel/rcutree.c +8 -6
index 106f7f5cdd1d..f0be20886617 100644
@@ -411,11 +411,12 @@ static void rcu_eqs_enter(bool user)
 	rdtp = this_cpu_ptr(&rcu_dynticks);
 	oldval = rdtp->dynticks_nesting;
 	WARN_ON_ONCE((oldval & DYNTICK_TASK_NEST_MASK) == 0);
-	if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE)
+	if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE) {
 		rdtp->dynticks_nesting = 0;
-	else
+		rcu_eqs_enter_common(rdtp, oldval, user);
+	} else {
 		rdtp->dynticks_nesting -= DYNTICK_TASK_NEST_VALUE;
-	rcu_eqs_enter_common(rdtp, oldval, user);
+	}
 }
 
 /**
@@ -533,11 +534,12 @@ static void rcu_eqs_exit(bool user)
 	rdtp = this_cpu_ptr(&rcu_dynticks);
 	oldval = rdtp->dynticks_nesting;
 	WARN_ON_ONCE(oldval < 0);
-	if (oldval & DYNTICK_TASK_NEST_MASK)
+	if (oldval & DYNTICK_TASK_NEST_MASK) {
 		rdtp->dynticks_nesting += DYNTICK_TASK_NEST_VALUE;
-	else
+	} else {
 		rdtp->dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
-	rcu_eqs_exit_common(rdtp, oldval, user);
+		rcu_eqs_exit_common(rdtp, oldval, user);
+	}
 }
 
 /**
#2 lore raw
Quoted 21 lines · Frederic Weisbecker
Quoted 9 lines · Paul E. McKenney

Hello, Frederic!

The following patch seems to me to be a good idea to better handle task nesting. Any reason why it would be a bad thing?

Thanx, Paul

------------------------------------------------------------------------

Quoted 7 lines

rcu: Allow task-level idle entry/exit nesting

The current task-level idle entry/exit code forces an entry/exit on each call, regardless of the nesting level. This commit therefore properly accounts for nesting.

Looks good. In fact, the current code is even buggy because two nesting rcu_user_eqs() as in:

Quoted 6 lines
        rcu_eqs_enter()
            rcu_eqs_enter()
            rcu_eqs_exit()
        rcu_eqs_exit()

would result in rdtp->dynticks wrong increment, right?

That was my thought, but I figured I should run it past you in case there was some subtle tie-in to NO_HZ_FULL.

Quoted 3 lines

So that's even a bug fix. I wonder if it's a regression. That said rcu_eqs_enter_common() should warn on such miscount, so may be these functions actually don't nest in practice or you would have received such warnings.

And the lack of such warnings was another reason I felt the need to check with you.

Quoted 5 lines

So I wonder, do we want to continue to allow this nesting? I remember that DYNTICK_TASK_NEST_* stuff is there to protects against non finishing interrupts on some archs (I also remember that this, or at least a practical scenario for this, was hard to really define though :o) But then wouldn't it involve other kind of scenario like this?

Quoted 7 lines
       rcu_irq_enter()
           rcu_eqs_enter()
           rcu_eqs_exit()
           ...

Anyway, that's just random thougths on further simplifications, in any case, this patch looks good.

Yep, if no task-level nesting is ever required, things could be a bit simpler. I would be a bit slow about making such a change, though. After all, the need to deal with Hotel California interrupts means that handling nesting isn't that big of a deal comparatively. ;-)

May I add your Reviewed-by?

Thanx, Paul

Quoted 41 lines

Thanks.

Quoted 38 lines
kernel/rcutree.c +8 -6
index 106f7f5cdd1d..f0be20886617 100644
@@ -411,11 +411,12 @@ static void rcu_eqs_enter(bool user)
 	rdtp = this_cpu_ptr(&rcu_dynticks);
 	oldval = rdtp->dynticks_nesting;
 	WARN_ON_ONCE((oldval & DYNTICK_TASK_NEST_MASK) == 0);
-	if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE)
+	if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE) {
 		rdtp->dynticks_nesting = 0;
-	else
+		rcu_eqs_enter_common(rdtp, oldval, user);
+	} else {
 		rdtp->dynticks_nesting -= DYNTICK_TASK_NEST_VALUE;
-	rcu_eqs_enter_common(rdtp, oldval, user);
+	}
 }
 
 /**
@@ -533,11 +534,12 @@ static void rcu_eqs_exit(bool user)
 	rdtp = this_cpu_ptr(&rcu_dynticks);
 	oldval = rdtp->dynticks_nesting;
 	WARN_ON_ONCE(oldval < 0);
-	if (oldval & DYNTICK_TASK_NEST_MASK)
+	if (oldval & DYNTICK_TASK_NEST_MASK) {
 		rdtp->dynticks_nesting += DYNTICK_TASK_NEST_VALUE;
-	else
+	} else {
 		rdtp->dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
-	rcu_eqs_exit_common(rdtp, oldval, user);
+		rcu_eqs_exit_common(rdtp, oldval, user);
+	}
 }
 
 /**
#3 lore raw
Quoted 11 lines · Paul E. McKenney
Quoted 10 lines · Frederic Weisbecker

So I wonder, do we want to continue to allow this nesting? I remember that DYNTICK_TASK_NEST_* stuff is there to protects against non finishing interrupts on some archs (I also remember that this, or at least a practical scenario for this, was hard to really define though :o) But then wouldn't it involve other kind of scenario like this?

       rcu_irq_enter()
           rcu_eqs_enter()
           rcu_eqs_exit()
           ...
Quoted 7 lines
Quoted 2 lines

Anyway, that's just random thougths on further simplifications, in any case, this patch looks good.

Yep, if no task-level nesting is ever required, things could be a bit simpler. I would be a bit slow about making such a change, though. After all, the need to deal with Hotel California interrupts means that handling nesting isn't that big of a deal comparatively. ;-)

Right, well ideally it would be even best to fix the corner case(s) if there aren't that many of them. I mean calling rcu_irq_exit() from the end of those half interrupts I guess. It would make it much simpler than this complicated nesting handled on the core code. But I agree there is a bit of unknown out there, so yeah lets be prudent :)

Quoted 1 line

May I add your Reviewed-by?

Sure, thanks!

#4 lore raw
Quoted 20 lines · Frederic Weisbecker
Quoted 11 lines · Paul E. McKenney
Quoted 10 lines · Frederic Weisbecker

So I wonder, do we want to continue to allow this nesting? I remember that DYNTICK_TASK_NEST_* stuff is there to protects against non finishing interrupts on some archs (I also remember that this, or at least a practical scenario for this, was hard to really define though :o) But then wouldn't it involve other kind of scenario like this?

       rcu_irq_enter()
           rcu_eqs_enter()
           rcu_eqs_exit()
           ...
Quoted 7 lines
Quoted 2 lines

Anyway, that's just random thougths on further simplifications, in any case, this patch looks good.

Yep, if no task-level nesting is ever required, things could be a bit simpler. I would be a bit slow about making such a change, though. After all, the need to deal with Hotel California interrupts means that handling nesting isn't that big of a deal comparatively. ;-)

Quoted 8 lines

Right, well ideally it would be even best to fix the corner case(s) if there aren't that many of them. I mean calling rcu_irq_exit() from the end of those half interrupts I guess. It would make it much simpler than this complicated nesting handled on the core code. But I agree there is a bit of unknown out there, so yeah lets be prudent :)

Quoted 1 line

May I add your Reviewed-by?

Sure, thanks!

Done!

Thanx, Paul