rcuperf: Make rcuperf kernel test more robust for !expedited mode
PATCH 1/1 rcuperf: Make rcuperf kernel test more robust for !expedited mode
It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:
I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because rcu_unexpedited_gp() would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.
To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. The system_state approximately aligns with when rcu_unexpedited_gp() is called and works well in practice.
I also tried late_initcall however it is still too early to be meaningful for this case.
--- kernel/rcu/rcuperf.c | 8 ++++++++ 1 file changed, 8 insertions(+)
kernel/rcu/rcuperf.c +8 -0
@@ -375,6 +375,14 @@ rcu_perf_writer(void *arg)if (holdoff)schedule_timeout_uninterruptible (holdoff *HZ ); +/* +* Wait until rcu_end_inkernel_boot() is called for normal GP tests +* so that RCU is not always expedited for normal GP tests. +* The system_state test is approximate, but works well in practice. +*/ +while (!gp_exp && system_state !=SYSTEM_RUNNING ) +schedule_timeout_uninterruptible (1 ); + t =ktime_get_mono_fast_ns ();if (atomic_inc_return (&n_rcu_perf_writer_started) >= nrealwriters) { t_rcu_perf_writer_started = t;
Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
Quoted 19 lines · Joel Fernandes (Google)
It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:
I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because
rcu_unexpedited_gp()would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.
Quoted 8 lines
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. Thesystem_stateapproximately aligns with whenrcu_unexpedited_gp()is called and works well in practice.I also tried
late_initcallhowever it is still too early to be meaningful for this case.Signed-off-by Joel Fernandes (Google) <[email protected]>
Good catch, queued, thank you!
Thanx, Paul
Quoted 26 lines
--- kernel/rcu/rcuperf.c | 8 ++++++++ 1 file changed, 8 insertions(+)kernel/rcu/rcuperf.c +8
-0@@ -375,6 +375,14 @@ rcu_perf_writer(void *arg)if (holdoff)schedule_timeout_uninterruptible (holdoff *HZ ); +/* +* Wait until rcu_end_inkernel_boot() is called for normal GP tests +* so that RCU is not always expedited for normal GP tests. +* The system_state test is approximate, but works well in practice. +*/ +while (!gp_exp && system_state !=SYSTEM_RUNNING ) +schedule_timeout_uninterruptible (1 ); + t =ktime_get_mono_fast_ns ();if (atomic_inc_return (&n_rcu_perf_writer_started) >= nrealwriters) { t_rcu_perf_writer_started = t;Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)
It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:
I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because
rcu_unexpedited_gp()would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.
Quoted 2 lines
Quoted 2 lines
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. Thesystem_stateapproximately aligns
Just minor typo..
To fix this properly, let us check if system_state if SYSTEM_RUNNING is set before starting the test. ...
Should be
To fix this properly, let us check if system_state is set to SYSTEM_RUNNING before starting the test. ...
Thanks,
Byungchul
Quoted 37 lines
Quoted 6 lines
with when
rcu_unexpedited_gp()is called and works well in practice.I also tried
late_initcallhowever it is still too early to be meaningful for this case.Signed-off-by Joel Fernandes (Google) <[email protected]>Good catch, queued, thank you!
Thanx, Paul
Quoted 26 lines
--- kernel/rcu/rcuperf.c | 8 ++++++++ 1 file changed, 8 insertions(+)kernel/rcu/rcuperf.c +8
-0@@ -375,6 +375,14 @@ rcu_perf_writer(void *arg)if (holdoff)schedule_timeout_uninterruptible (holdoff *HZ ); +/* +* Wait until rcu_end_inkernel_boot() is called for normal GP tests +* so that RCU is not always expedited for normal GP tests. +* The system_state test is approximate, but works well in practice. +*/ +while (!gp_exp && system_state !=SYSTEM_RUNNING ) +schedule_timeout_uninterruptible (1 ); + t =ktime_get_mono_fast_ns ();if (atomic_inc_return (&n_rcu_perf_writer_started) >= nrealwriters) { t_rcu_perf_writer_started = t;Signature · 1 line
2.22.0.410.gd8fdbe21b5-goog
Quoted 26 lines · Byungchul Park
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)
It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:
I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because
rcu_unexpedited_gp()would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.Quoted 2 lines
Quoted 2 lines
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. Thesystem_stateapproximately alignsJust minor typo..
Quoted 7 lines
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. ...Should be
To fix this properly, let us check if
system_stateis set toSYSTEM_RUNNINGbefore starting the test. ...
That's a fair point. I wonder if Paul already fixed it up in his tree, however I am happy to resend if he hasn't. Paul, how would you like to handle this commit log nit?
it is just 'if ..' to 'is SYSTEM_RUNNING'
Quoted 32 lines · Joel Fernandes
Quoted 31 lines · Byungchul Park
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)
It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:
I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because
rcu_unexpedited_gp()would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.Quoted 2 lines
Quoted 2 lines
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. Thesystem_stateapproximately alignsJust minor typo..
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. ...Should be
Quoted 8 lines
Quoted 2 lines
To fix this properly, let us check if
system_stateis set toSYSTEM_RUNNINGbefore starting the test. ...That's a fair point. I wonder if Paul already fixed it up in his tree, however I am happy to resend if he hasn't. Paul, how would you like to handle this commit log nit?
it is just 'if ..' to 'is
SYSTEM_RUNNING'
It now reads as follows:
To fix this properly, this commit waits until system_state is set to SYSTEM_RUNNING before starting the test. This change is made just before kernel_init() invokes rcu_end_inkernel_boot(), and this latter is what turns off boot-time expediting of RCU grace periods.
I dropped the last paragraph about late_initcall(). And I suspect that the last clause from rcu_gp_is_expedited() can be dropped:
bool rcu_gp_is_expedited(void)
{
return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
rcu_scheduler_active == RCU_SCHEDULER_INIT;
}This is because rcu_expedited_nesting is initialized to 1, and is decremented in rcu_end_inkernel_boot(), which is called long after rcu_scheduler_active has been set to RCU_SCHEDULER_RUNNING, which happens at core_initcall() time. So if the last clause says "true", so does the second-to-last clause.
The similar check in rcu_gp_is_normal() is still need, however, to allow the power-management subsystem to invoke synchronize_rcu() just after the scheduler has been initialized, but before RCU is aware of this.
So, how about the commit shown below?
Thanx, Paul
------------------------------------------------------------------------
commit 1f7e72efe3c761c2b34da7b59e01ad69c657db10
Author: Paul E. McKenney <[email protected]> Date: Fri Jul 5 08:05:10 2019 -0700
rcu: Remove redundant "if" condition from rcu_gp_is_expedited()
Because rcu_expedited_nesting is initialized to 1 and not decremented until just before init is spawned, rcu_expedited_nesting is guaranteed to be non-zero whenever rcu_scheduler_active == RCU_SCHEDULER_INIT. This commit therefore removes this redundant "if" equality test.
kernel/rcu/update.c +1 -2
@@ -136,8 +136,7 @@ static atomic_t rcu_expedited_nesting = ATOMIC_INIT(1);*/ bool rcu_gp_is_expedited (void ) { -return rcu_expedited ||atomic_read (&rcu_expedited_nesting) || - rcu_scheduler_active ==RCU_SCHEDULER_INIT ; +return rcu_expedited ||atomic_read (&rcu_expedited_nesting); }EXPORT_SYMBOL_GPL (rcu_gp_is_expedited);
Joel Fernandes
Reviewed-byQuoted 42 lines · Paul E. McKenney
Quoted 32 lines · Joel Fernandes
Quoted 31 lines · Byungchul Park
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)
It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:
I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because
rcu_unexpedited_gp()would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.Quoted 2 lines
Quoted 2 lines
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. Thesystem_stateapproximately alignsJust minor typo..
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. ...Should be
Quoted 8 lines
Quoted 2 lines
To fix this properly, let us check if
system_stateis set toSYSTEM_RUNNINGbefore starting the test. ...That's a fair point. I wonder if Paul already fixed it up in his tree, however I am happy to resend if he hasn't. Paul, how would you like to handle this commit log nit?
it is just 'if ..' to 'is
SYSTEM_RUNNING'
Quoted 7 lines
It now reads as follows:
To fix this properly, this commit waits until
system_stateis set toSYSTEM_RUNNINGbefore starting the test. This change is made just beforekernel_init()invokesrcu_end_inkernel_boot(), and this latter is what turns off boot-time expediting of RCU grace periods.
Ok, looks good to me, thanks.
And for below patch,
Quoted 53 lines
I dropped the last paragraph about
late_initcall(). And I suspect that the last clause fromrcu_gp_is_expedited()can be dropped:bool
rcu_gp_is_expedited(void){ return rcu_expedited || atomic_read(&rcu_expedited_nesting) || rcu_scheduler_active == RCU_SCHEDULER_INIT; }This is because
rcu_expedited_nestingis initialized to 1, and is decremented inrcu_end_inkernel_boot(), which is called long afterrcu_scheduler_activehas been set toRCU_SCHEDULER_RUNNING, which happens atcore_initcall()time. So if the last clause says "true", so does the second-to-last clause.The similar check in
rcu_gp_is_normal()is still need, however, to allow the power-management subsystem to invokesynchronize_rcu()just after the scheduler has been initialized, but before RCU is aware of this.So, how about the commit shown below?
Thanx, Paul
------------------------------------------------------------------------
commit
1f7e72efe3c761c2b34da7b59e01ad69c657db10Author: Paul E. McKenney <[email protected]> Date: Fri Jul 5 08:05:10 2019 -0700rcu: Remove redundant "if" condition from
rcu_gp_is_expedited()Because
rcu_expedited_nestingis initialized to 1 and not decremented until just before init is spawned,rcu_expedited_nestingis guaranteed to be non-zero wheneverrcu_scheduler_active==RCU_SCHEDULER_INIT. This commit therefore removes this redundant "if" equality test.Signed-off-by Paul E. McKenney <[email protected]>kernel/rcu/update.c +1
-2@@ -136,8 +136,7 @@ static atomic_t rcu_expedited_nesting = ATOMIC_INIT(1);*/ bool rcu_gp_is_expedited (void ) { -return rcu_expedited ||atomic_read (&rcu_expedited_nesting) || - rcu_scheduler_active ==RCU_SCHEDULER_INIT ; +return rcu_expedited ||atomic_read (&rcu_expedited_nesting); }EXPORT_SYMBOL_GPL (rcu_gp_is_expedited);
Paul E. McKenney
AppliedQuoted 51 lines · Joel Fernandes
Quoted 42 lines · Paul E. McKenney
Quoted 32 lines · Joel Fernandes
Quoted 31 lines · Byungchul Park
Quoted 20 lines · Paul E. McKenney
Quoted 19 lines · Joel Fernandes (Google)
It is possible that the rcuperf kernel test runs concurrently with init starting up. During this time, the system is running all grace periods as expedited. However, rcuperf can also be run for normal GP tests. Right now, it depends on a holdoff time before starting the test to ensure grace periods start later. This works fine with the default holdoff time however it is not robust in situations where init takes greater than the holdoff time to finish running. Or, as in my case:
I modified the rcuperf test locally to also run a thread that did preempt disable/enable in a loop. This had the effect of slowing down init. The end result was that the "batches:" counter in rcuperf was 0 causing a division by 0 error in the results. This counter was 0 because only expedited GPs seem to happen, not normal ones which led to the rcu_state.gp_seq counter remaining constant across grace periods which unexpectedly happen to be expedited. The system was running expedited RCU all the time because
rcu_unexpedited_gp()would not have run yet from init. In other words, the test would concurrently with init booting in expedited GP mode.Quoted 2 lines
Quoted 2 lines
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. Thesystem_stateapproximately alignsJust minor typo..
To fix this properly, let us check if
system_stateifSYSTEM_RUNNINGis set before starting the test. ...Should be
Quoted 8 lines
Quoted 2 lines
To fix this properly, let us check if
system_stateis set toSYSTEM_RUNNINGbefore starting the test. ...That's a fair point. I wonder if Paul already fixed it up in his tree, however I am happy to resend if he hasn't. Paul, how would you like to handle this commit log nit?
it is just 'if ..' to 'is
SYSTEM_RUNNING'Quoted 7 lines
It now reads as follows:
To fix this properly, this commit waits until
system_stateis set toSYSTEM_RUNNINGbefore starting the test. This change is made just beforekernel_init()invokesrcu_end_inkernel_boot(), and this latter is what turns off boot-time expediting of RCU grace periods.
Quoted 5 lines
Ok, looks good to me, thanks.
And for below patch,
Reviewed-by Joel Fernandes (Google) <[email protected]>
Applied, thank you!
Thnax, Paul
Quoted 54 lines
Quoted 53 lines
I dropped the last paragraph about
late_initcall(). And I suspect that the last clause fromrcu_gp_is_expedited()can be dropped:bool
rcu_gp_is_expedited(void){ return rcu_expedited || atomic_read(&rcu_expedited_nesting) || rcu_scheduler_active == RCU_SCHEDULER_INIT; }This is because
rcu_expedited_nestingis initialized to 1, and is decremented inrcu_end_inkernel_boot(), which is called long afterrcu_scheduler_activehas been set toRCU_SCHEDULER_RUNNING, which happens atcore_initcall()time. So if the last clause says "true", so does the second-to-last clause.The similar check in
rcu_gp_is_normal()is still need, however, to allow the power-management subsystem to invokesynchronize_rcu()just after the scheduler has been initialized, but before RCU is aware of this.So, how about the commit shown below?
Thanx, Paul
------------------------------------------------------------------------
commit
1f7e72efe3c761c2b34da7b59e01ad69c657db10Author: Paul E. McKenney <[email protected]> Date: Fri Jul 5 08:05:10 2019 -0700rcu: Remove redundant "if" condition from
rcu_gp_is_expedited()Because
rcu_expedited_nestingis initialized to 1 and not decremented until just before init is spawned,rcu_expedited_nestingis guaranteed to be non-zero wheneverrcu_scheduler_active==RCU_SCHEDULER_INIT. This commit therefore removes this redundant "if" equality test.Signed-off-by Paul E. McKenney <[email protected]>kernel/rcu/update.c +1
-2@@ -136,8 +136,7 @@ static atomic_t rcu_expedited_nesting = ATOMIC_INIT(1);*/ bool rcu_gp_is_expedited (void ) { -return rcu_expedited ||atomic_read (&rcu_expedited_nesting) || - rcu_scheduler_active ==RCU_SCHEDULER_INIT ; +return rcu_expedited ||atomic_read (&rcu_expedited_nesting); }EXPORT_SYMBOL_GPL (rcu_gp_is_expedited);