Skip to contentSkip to conversation

rcuperf: Make rcuperf kernel test more robust for !expedited mode

PATCH Applied Joel Fernandes (Google) 3 participants 7 messages Jul 4, 2019 – Jul 13, 2019 RCU

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 rcuperf: Make rcuperf kernel test more robust for !expedited mode

It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:

I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because rcu_unexpedited_gp() would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. The system_state approximately aligns with when rcu_unexpedited_gp() is called and works well in practice.

I also tried late_initcall however it is still too early to be meaningful for this case.

---
 kernel/rcu/rcuperf.c | 8 ++++++++
 1 file changed, 8 insertions(+)
kernel/rcu/rcuperf.c +8 -0
index 4513807cd4c4..5a879d073c1c 100644
@@ -375,6 +375,14 @@ rcu_perf_writer(void *arg)
 	if (holdoff)
 		schedule_timeout_uninterruptible(holdoff * HZ);
 
+	/*
+	 * Wait until rcu_end_inkernel_boot() is called for normal GP tests
+	 * so that RCU is not always expedited for normal GP tests.
+	 * The system_state test is approximate, but works well in practice.
+	 */
+	while (!gp_exp && system_state != SYSTEM_RUNNING)
+		schedule_timeout_uninterruptible(1);
+
 	t = ktime_get_mono_fast_ns();
 	if (atomic_inc_return(&n_rcu_perf_writer_started) >= nrealwriters) {
 		t_rcu_perf_writer_started = t;
Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
#1 lore raw
Quoted 19 lines · Joel Fernandes (Google)

It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:

I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because rcu_unexpedited_gp() would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.

Quoted 8 lines

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. The system_state approximately aligns with when rcu_unexpedited_gp() is called and works well in practice.

I also tried late_initcall however it is still too early to be meaningful for this case.

Good catch, queued, thank you!

Thanx, Paul

Quoted 26 lines
---
 kernel/rcu/rcuperf.c | 8 ++++++++
 1 file changed, 8 insertions(+)
kernel/rcu/rcuperf.c +8 -0
index 4513807cd4c4..5a879d073c1c 100644
@@ -375,6 +375,14 @@ rcu_perf_writer(void *arg)
 	if (holdoff)
 		schedule_timeout_uninterruptible(holdoff * HZ);
 
+	/*
+	 * Wait until rcu_end_inkernel_boot() is called for normal GP tests
+	 * so that RCU is not always expedited for normal GP tests.
+	 * The system_state test is approximate, but works well in practice.
+	 */
+	while (!gp_exp && system_state != SYSTEM_RUNNING)
+		schedule_timeout_uninterruptible(1);
+
 	t = ktime_get_mono_fast_ns();
 	if (atomic_inc_return(&n_rcu_perf_writer_started) >= nrealwriters) {
 		t_rcu_perf_writer_started = t;
Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
#2 lore raw
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)

It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:

I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because rcu_unexpedited_gp() would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.

Quoted 2 lines
Quoted 2 lines

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. The system_state approximately aligns

Just minor typo..

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. ...

Should be

To fix this properly, let us check if system_state is set to SYSTEM_RUNNING before starting the test. ...

Thanks,
Byungchul

Quoted 37 lines
Quoted 6 lines

with when rcu_unexpedited_gp() is called and works well in practice.

I also tried late_initcall however it is still too early to be meaningful for this case.

Good catch, queued, thank you!

Thanx, Paul

Quoted 26 lines
---
 kernel/rcu/rcuperf.c | 8 ++++++++
 1 file changed, 8 insertions(+)
kernel/rcu/rcuperf.c +8 -0
index 4513807cd4c4..5a879d073c1c 100644
@@ -375,6 +375,14 @@ rcu_perf_writer(void *arg)
 	if (holdoff)
 		schedule_timeout_uninterruptible(holdoff * HZ);
 
+	/*
+	 * Wait until rcu_end_inkernel_boot() is called for normal GP tests
+	 * so that RCU is not always expedited for normal GP tests.
+	 * The system_state test is approximate, but works well in practice.
+	 */
+	while (!gp_exp && system_state != SYSTEM_RUNNING)
+		schedule_timeout_uninterruptible(1);
+
 	t = ktime_get_mono_fast_ns();
 	if (atomic_inc_return(&n_rcu_perf_writer_started) >= nrealwriters) {
 		t_rcu_perf_writer_started = t;
Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
#3 lore raw
Quoted 26 lines · Byungchul Park
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)

It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:

I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because rcu_unexpedited_gp() would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.

Quoted 2 lines
Quoted 2 lines

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. The system_state approximately aligns

Just minor typo..

Quoted 7 lines

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. ...

Should be

To fix this properly, let us check if system_state is set to SYSTEM_RUNNING before starting the test. ...

That's a fair point. I wonder if Paul already fixed it up in his tree, however I am happy to resend if he hasn't. Paul, how would you like to handle this commit log nit?

it is just 'if ..' to 'is SYSTEM_RUNNING'

#4 lore raw
Quoted 32 lines · Joel Fernandes
Quoted 31 lines · Byungchul Park
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)

It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:

I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because rcu_unexpedited_gp() would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.

Quoted 2 lines
Quoted 2 lines

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. The system_state approximately aligns

Just minor typo..

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. ...

Should be

Quoted 8 lines
Quoted 2 lines

To fix this properly, let us check if system_state is set to SYSTEM_RUNNING before starting the test. ...

That's a fair point. I wonder if Paul already fixed it up in his tree, however I am happy to resend if he hasn't. Paul, how would you like to handle this commit log nit?

it is just 'if ..' to 'is SYSTEM_RUNNING'

It now reads as follows:

To fix this properly, this commit waits until system_state is set to SYSTEM_RUNNING before starting the test. This change is made just before kernel_init() invokes rcu_end_inkernel_boot(), and this latter is what turns off boot-time expediting of RCU grace periods.

I dropped the last paragraph about late_initcall(). And I suspect that the last clause from rcu_gp_is_expedited() can be dropped:

bool rcu_gp_is_expedited(void)

{
	return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
	       rcu_scheduler_active == RCU_SCHEDULER_INIT;
}

This is because rcu_expedited_nesting is initialized to 1, and is decremented in rcu_end_inkernel_boot(), which is called long after rcu_scheduler_active has been set to RCU_SCHEDULER_RUNNING, which happens at core_initcall() time. So if the last clause says "true", so does the second-to-last clause.

The similar check in rcu_gp_is_normal() is still need, however, to allow the power-management subsystem to invoke synchronize_rcu() just after the scheduler has been initialized, but before RCU is aware of this.

So, how about the commit shown below?

Thanx, Paul

------------------------------------------------------------------------

commit 1f7e72efe3c761c2b34da7b59e01ad69c657db10

Author: Paul E. McKenney <[email protected]>
Date:   Fri Jul 5 08:05:10 2019 -0700

rcu: Remove redundant "if" condition from rcu_gp_is_expedited()

Because rcu_expedited_nesting is initialized to 1 and not decremented until just before init is spawned, rcu_expedited_nesting is guaranteed to be non-zero whenever rcu_scheduler_active == RCU_SCHEDULER_INIT. This commit therefore removes this redundant "if" equality test.

kernel/rcu/update.c +1 -2
index 249517058b13..64e9cc8609e7 100644
@@ -136,8 +136,7 @@ static atomic_t rcu_expedited_nesting = ATOMIC_INIT(1);
  */
 bool rcu_gp_is_expedited(void)
 {
-	return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
-	       rcu_scheduler_active == RCU_SCHEDULER_INIT;
+	return rcu_expedited || atomic_read(&rcu_expedited_nesting);
 }
 EXPORT_SYMBOL_GPL(rcu_gp_is_expedited);
 
author Reviewed-by
#5 lore raw
Quoted 42 lines · Paul E. McKenney
Quoted 32 lines · Joel Fernandes
Quoted 31 lines · Byungchul Park
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)

It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:

I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because rcu_unexpedited_gp() would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.

Quoted 2 lines
Quoted 2 lines

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. The system_state approximately aligns

Just minor typo..

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. ...

Should be

Quoted 8 lines
Quoted 2 lines

To fix this properly, let us check if system_state is set to SYSTEM_RUNNING before starting the test. ...

That's a fair point. I wonder if Paul already fixed it up in his tree, however I am happy to resend if he hasn't. Paul, how would you like to handle this commit log nit?

it is just 'if ..' to 'is SYSTEM_RUNNING'

Quoted 7 lines

It now reads as follows:

To fix this properly, this commit waits until system_state is set to SYSTEM_RUNNING before starting the test. This change is made just before kernel_init() invokes rcu_end_inkernel_boot(), and this latter is what turns off boot-time expediting of RCU grace periods.

Ok, looks good to me, thanks.

And for below patch,

Quoted 53 lines

I dropped the last paragraph about late_initcall(). And I suspect that the last clause from rcu_gp_is_expedited() can be dropped:

bool rcu_gp_is_expedited(void)

{
	return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
	       rcu_scheduler_active == RCU_SCHEDULER_INIT;
}

This is because rcu_expedited_nesting is initialized to 1, and is decremented in rcu_end_inkernel_boot(), which is called long after rcu_scheduler_active has been set to RCU_SCHEDULER_RUNNING, which happens at core_initcall() time. So if the last clause says "true", so does the second-to-last clause.

The similar check in rcu_gp_is_normal() is still need, however, to allow the power-management subsystem to invoke synchronize_rcu() just after the scheduler has been initialized, but before RCU is aware of this.

So, how about the commit shown below?

Thanx, Paul

------------------------------------------------------------------------

commit 1f7e72efe3c761c2b34da7b59e01ad69c657db10

Author: Paul E. McKenney <[email protected]>
Date:   Fri Jul 5 08:05:10 2019 -0700

rcu: Remove redundant "if" condition from rcu_gp_is_expedited()

Because rcu_expedited_nesting is initialized to 1 and not decremented until just before init is spawned, rcu_expedited_nesting is guaranteed to be non-zero whenever rcu_scheduler_active == RCU_SCHEDULER_INIT. This commit therefore removes this redundant "if" equality test.

kernel/rcu/update.c +1 -2
index 249517058b13..64e9cc8609e7 100644
@@ -136,8 +136,7 @@ static atomic_t rcu_expedited_nesting = ATOMIC_INIT(1);
  */
 bool rcu_gp_is_expedited(void)
 {
-	return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
-	       rcu_scheduler_active == RCU_SCHEDULER_INIT;
+	return rcu_expedited || atomic_read(&rcu_expedited_nesting);
 }
 EXPORT_SYMBOL_GPL(rcu_gp_is_expedited);
 
#6 lore raw
Quoted 51 lines · Joel Fernandes
Quoted 42 lines · Paul E. McKenney
Quoted 32 lines · Joel Fernandes
Quoted 31 lines · Byungchul Park
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)

It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:

I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because rcu_unexpedited_gp() would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.

Quoted 2 lines
Quoted 2 lines

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. The system_state approximately aligns

Just minor typo..

To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. ...

Should be

Quoted 8 lines
Quoted 2 lines

To fix this properly, let us check if system_state is set to SYSTEM_RUNNING before starting the test. ...

That's a fair point. I wonder if Paul already fixed it up in his tree, however I am happy to resend if he hasn't. Paul, how would you like to handle this commit log nit?

it is just 'if ..' to 'is SYSTEM_RUNNING'

Quoted 7 lines

It now reads as follows:

To fix this properly, this commit waits until system_state is set to SYSTEM_RUNNING before starting the test. This change is made just before kernel_init() invokes rcu_end_inkernel_boot(), and this latter is what turns off boot-time expediting of RCU grace periods.

Quoted 5 lines

Ok, looks good to me, thanks.

And for below patch,

Applied, thank you!

Thnax, Paul

Quoted 54 lines
Quoted 53 lines

I dropped the last paragraph about late_initcall(). And I suspect that the last clause from rcu_gp_is_expedited() can be dropped:

bool rcu_gp_is_expedited(void)

{
	return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
	       rcu_scheduler_active == RCU_SCHEDULER_INIT;
}

This is because rcu_expedited_nesting is initialized to 1, and is decremented in rcu_end_inkernel_boot(), which is called long after rcu_scheduler_active has been set to RCU_SCHEDULER_RUNNING, which happens at core_initcall() time. So if the last clause says "true", so does the second-to-last clause.

The similar check in rcu_gp_is_normal() is still need, however, to allow the power-management subsystem to invoke synchronize_rcu() just after the scheduler has been initialized, but before RCU is aware of this.

So, how about the commit shown below?

Thanx, Paul

------------------------------------------------------------------------

commit 1f7e72efe3c761c2b34da7b59e01ad69c657db10

Author: Paul E. McKenney <[email protected]>
Date:   Fri Jul 5 08:05:10 2019 -0700

rcu: Remove redundant "if" condition from rcu_gp_is_expedited()

Because rcu_expedited_nesting is initialized to 1 and not decremented until just before init is spawned, rcu_expedited_nesting is guaranteed to be non-zero whenever rcu_scheduler_active == RCU_SCHEDULER_INIT. This commit therefore removes this redundant "if" equality test.

kernel/rcu/update.c +1 -2
index 249517058b13..64e9cc8609e7 100644
@@ -136,8 +136,7 @@ static atomic_t rcu_expedited_nesting = ATOMIC_INIT(1);
  */
 bool rcu_gp_is_expedited(void)
 {
-	return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
-	       rcu_scheduler_active == RCU_SCHEDULER_INIT;
+	return rcu_expedited || atomic_read(&rcu_expedited_nesting);
 }
 EXPORT_SYMBOL_GPL(rcu_gp_is_expedited);