Skip to contentSkip to conversation

srcu: shrink the num of srcu_size_state

PATCH In review Pingfan Liu 4 participants 17 messages Nov 28, 2022 – Dec 16, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover srcu: shrink the num of srcu_size_state

This series fix a bug and improve the note of SRCU_SIZE_*

Cc: Lai Jiangshan <[email protected]>
Cc: "Paul E. McKenney" <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: Josh Triplett <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Mathieu Desnoyers <[email protected]>
Cc: "Zhang, Qiang1" <[email protected]>
To: [email protected]

Pingfan Liu (3):
srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL
srcu: protect against concurrent srcu_gp_end()
srcu: Decrease the srcu size state numbers

 include/linux/srcutree.h | 15 +++++++++------
 kernel/rcu/srcutree.c    | 19 ++++++++-----------
 2 files changed, 17 insertions(+), 17 deletions(-)
Signature · 1 line
2.31.1
#1 lore raw

PATCH 1/3 srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]

---
 kernel/rcu/srcutree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +1 -1
index 1c304fec89c0..fe0759d89c2d 100644
@@ -1092,7 +1092,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	check_init_srcu_struct(ssp);
 	idx = srcu_read_lock(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, 0);
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
#2 lore raw

RE: [PATCH 1/3] srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

Quoted 30 lines

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
kernel/rcu/srcutree.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +0 -0
index 1c304fec89c0..fe0759d89c2d 100644

kernel/rcu/srcutree.c static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp, · line 1096

Quoted 6 lines
kernel/rcu/srcutree.c +1 -1
@@ -1092,7 +1092,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	check_init_srcu_struct(ssp);
 	idx = srcu_read_lock(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, 0);

Hi Pingfan

should it be used get_boot_cpu_id() ?

Thanks
Zqiang

Quoted 5 lines
kernel/rcu/srcutree.c +0 -0
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
#3 lore raw

Re: [PATCH 1/3] srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

Quoted 39 lines · On Mon, Nov 28, 2022 at 08:40:22AM +0000, Zhang, Qiang1 wrote:
Quoted 30 lines

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
kernel/rcu/srcutree.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +0 -0
index 1c304fec89c0..fe0759d89c2d 100644

kernel/rcu/srcutree.c static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp, · line 1096

Quoted 6 lines
kernel/rcu/srcutree.c +1 -1
@@ -1092,7 +1092,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	check_init_srcu_struct(ssp);
 	idx = srcu_read_lock(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, 0);

Hi Pingfan

Quoted 2 lines

should it be used get_boot_cpu_id() ?

I am not sure which branch to work on.

Thanks,

Pingfan

Quoted 8 lines

Thanks
Zqiang

Quoted 5 lines
kernel/rcu/srcutree.c +0 -0
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
maintainer
#4 lore raw

Re: [PATCH 1/3] srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

Quoted 40 lines · Pingfan Liu
Quoted 39 lines · On Mon, Nov 28, 2022 at 08:40:22AM +0000, Zhang, Qiang1 wrote:
Quoted 30 lines

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
kernel/rcu/srcutree.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +0 -0
index 1c304fec89c0..fe0759d89c2d 100644

kernel/rcu/srcutree.c static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp, · line 1096

Quoted 6 lines
kernel/rcu/srcutree.c +1 -1
@@ -1092,7 +1092,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	check_init_srcu_struct(ssp);
 	idx = srcu_read_lock(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, 0);

Hi Pingfan

Quoted 3 lines
Quoted 1 line

should it be used get_boot_cpu_id() ?

I am not sure which branch to work on.

For RCU patches, the "dev" branch of the -rcu tree.

For more information, please see:

https://mirrors.edge.kernel.org/pub/linux/kernel/people/paulmck/rcutodo.html

Thanx, Paul

Quoted 12 lines

Thanks,

Pingfan

Quoted 8 lines

Thanks
Zqiang

Quoted 5 lines
kernel/rcu/srcutree.c +0 -0
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
#5 lore raw

PATCH v2 1/1 srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

In reply to Pingfan Liu

From: Pingfan Liu <[email protected]>

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
v1 -> v2:
rebase onto dev branch

 kernel/rcu/srcutree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +1 -1
index b25f77df9e5e..b0278d1e9c1f 100644
@@ -1165,7 +1165,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	 */
 	idx = __srcu_read_lock_nmisafe(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, get_boot_cpu_id());
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
maintainer
#6 lore raw

Re: [PATCHv2] srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

Quoted 8 lines · Pingfan Liu

From: Pingfan Liu <[email protected]>

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

Quoted 5 lines

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

During the SRCU_SIZE_WAIT_BARRIER state, all callbacks are queued onto the boot CPU's callback list, but the srcu_barrier() function scans all of the CPUs' callback lists.

So can't this cause srcu_gp_start_if_needed() to rcu_segcblist_advance() and rcu_segcblist_accelerate() a callback list that is guaranteed to be empty? Without this change, the might-not-be-empty boot CPU's callback list is dealt with.

What am I missing here? More specifically, what benefits does this change provide? I am not yet seeing any.

Thanx, Paul

Quoted 31 lines
Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
v1 -> v2:
rebase onto dev branch

 kernel/rcu/srcutree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +1 -1
index b25f77df9e5e..b0278d1e9c1f 100644
@@ -1165,7 +1165,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	 */
 	idx = __srcu_read_lock_nmisafe(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, get_boot_cpu_id());
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
#7 lore raw

Re: [PATCHv2] srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

Sorry to reply late. I was interrupted by something else.

Quoted 11 lines · Paul E. McKenney
Quoted 10 lines · Pingfan Liu

From: Pingfan Liu <[email protected]>

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

Quoted 8 lines
Quoted 3 lines

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

During the SRCU_SIZE_WAIT_BARRIER state, all callbacks are queued onto the boot CPU's callback list, but the srcu_barrier() function scans all of the CPUs' callback lists.

So at present, your concern is around how it affects srcu_barrier(), right?

Quoted 5 lines

So can't this cause srcu_gp_start_if_needed() to rcu_segcblist_advance() and rcu_segcblist_accelerate() a callback list that is guaranteed to be empty? Without this change, the might-not-be-empty boot CPU's callback list is dealt with.

Is it a problem that srcu_gp_start_if_needed() asks rcu_segcblist_advance() and rcu_segcblist_accelerate() to manipulate an not-be-empty callback list ? At SRCU_SIZE_WAIT_BARRIER state, srcu_barrier() will take care of all cpus' callback list.

Quoted 3 lines

What am I missing here? More specifically, what benefits does this change provide? I am not yet seeing any.

I am trying to improve the note around SRCU_SIZE_xx, and this is a trivial improve from 8 SRCU_SIZE state to 5.

Thanks,

Pingfan

Quoted 33 lines

Thanx, Paul

Quoted 31 lines
Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
v1 -> v2:
rebase onto dev branch

 kernel/rcu/srcutree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +1 -1
index b25f77df9e5e..b0278d1e9c1f 100644
@@ -1165,7 +1165,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	 */
 	idx = __srcu_read_lock_nmisafe(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, get_boot_cpu_id());
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
maintainer
#8 lore raw

Re: [PATCHv2] srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

Quoted 18 lines · Pingfan Liu

Sorry to reply late. I was interrupted by something else.

Quoted 15 lines · Paul E. McKenney
Quoted 13 lines · Pingfan Liu

From: Pingfan Liu <[email protected]>

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

Quoted 5 lines
Quoted 3 lines

During the SRCU_SIZE_WAIT_BARRIER state, all callbacks are queued onto the boot CPU's callback list, but the srcu_barrier() function scans all of the CPUs' callback lists.

So at present, your concern is around how it affects srcu_barrier(), right?

At present, as always, I am concerned about all potential added bugs and potential added complexity, both in terms of lines of code and in terms of size and irregularity of state space.

And yes, part of the concern about both correctness and complexity is the interaction between call_srcu() and srcu_barrier(). Right now, the design very straightforwardly ensures that rcu_barrier() and the rest of SRCU is looking at all CPU's callback lists before call_srcu() places any callbacks on non-boot CPUs. This is because all the srcu_barrier() calls that think that the system is still in SRCU_SIZE_SMALL or SRCU_SIZE_ALLOC mode must have completed before any call_srcu() invocation uses a non-boot CPU's callback list.

Your change causes rcu_barrier() and call_srcu() to start using non-boot CPUs' callback lists at the same time. This change therefore introduces significant state-space complexity. And maybe even bugs, if not today then some later time when someone makes a change that looks obviously correct on the surface, but which subtly weakens the ordering.

Quoted 9 lines
Quoted 4 lines

So can't this cause srcu_gp_start_if_needed() to rcu_segcblist_advance() and rcu_segcblist_accelerate() a callback list that is guaranteed to be empty? Without this change, the might-not-be-empty boot CPU's callback list is dealt with.

Is it a problem that srcu_gp_start_if_needed() asks rcu_segcblist_advance() and rcu_segcblist_accelerate() to manipulate an not-be-empty callback list ? At SRCU_SIZE_WAIT_BARRIER state, srcu_barrier() will take care of all cpus' callback list.

The other way around, please. My concern is instead that it *fails* to rcu_segcblist_advance() and rcu_segcblist_accelerate() the might-not-be-empty callback lists.

Quoted 5 lines
Quoted 2 lines

What am I missing here? More specifically, what benefits does this change provide? I am not yet seeing any.

I am trying to improve the note around SRCU_SIZE_xx, and this is a trivial improve from 8 SRCU_SIZE state to 5.

I do understand that you would like to remove SRCU_SIZE_* states. What I need from you is why it is important to remove these states. What is being gained by removing them? Why is that gain so important that we should (at the very least) make the state space more complex? And does this change really avoid introducing bugs?

Thanx, Paul

Quoted 37 lines

Thanks,

Pingfan

Quoted 33 lines

Thanx, Paul

Quoted 31 lines
Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
v1 -> v2:
rebase onto dev branch

 kernel/rcu/srcutree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +1 -1
index b25f77df9e5e..b0278d1e9c1f 100644
@@ -1165,7 +1165,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	 */
 	idx = __srcu_read_lock_nmisafe(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, get_boot_cpu_id());
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
#9 lore raw

Re: [PATCHv2] srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

Quoted 29 lines · Paul E. McKenney
Quoted 18 lines · Pingfan Liu

Sorry to reply late. I was interrupted by something else.

Quoted 15 lines · Paul E. McKenney
Quoted 13 lines · Pingfan Liu

From: Pingfan Liu <[email protected]>

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

Quoted 5 lines
Quoted 3 lines

During the SRCU_SIZE_WAIT_BARRIER state, all callbacks are queued onto the boot CPU's callback list, but the srcu_barrier() function scans all of the CPUs' callback lists.

So at present, your concern is around how it affects srcu_barrier(), right?

At present, as always, I am concerned about all potential added bugs and potential added complexity, both in terms of lines of code and in terms of size and irregularity of state space.

Quoted 9 lines

And yes, part of the concern about both correctness and complexity is the interaction between call_srcu() and srcu_barrier(). Right now, the design very straightforwardly ensures that rcu_barrier() and the rest of SRCU is looking at all CPU's callback lists before call_srcu() places any callbacks on non-boot CPUs. This is because all the srcu_barrier() calls that think that the system is still in SRCU_SIZE_SMALL or SRCU_SIZE_ALLOC mode must have completed before any call_srcu() invocation uses a non-boot CPU's callback list.

Catch your point. Thanks for patient explanation. And indeed this does not rely on anything like memory order to build a total order, and more reliable.

Quoted 6 lines

Your change causes rcu_barrier() and call_srcu() to start using non-boot CPUs' callback lists at the same time. This change therefore introduces significant state-space complexity. And maybe even bugs, if not today then some later time when someone makes a change that looks obviously correct on the surface, but which subtly weakens the ordering.

Yes, that is true.

Quoted 17 lines
Quoted 9 lines
Quoted 4 lines

So can't this cause srcu_gp_start_if_needed() to rcu_segcblist_advance() and rcu_segcblist_accelerate() a callback list that is guaranteed to be empty? Without this change, the might-not-be-empty boot CPU's callback list is dealt with.

Is it a problem that srcu_gp_start_if_needed() asks rcu_segcblist_advance() and rcu_segcblist_accelerate() to manipulate an not-be-empty callback list ? At SRCU_SIZE_WAIT_BARRIER state, srcu_barrier() will take care of all cpus' callback list.

The other way around, please. My concern is instead that it *fails* to rcu_segcblist_advance() and rcu_segcblist_accelerate() the might-not-be-empty callback lists.

Quoted 3 lines
Quoted 2 lines

What am I missing here? More specifically, what benefits does this change provide? I am not yet seeing any.

Quoted 9 lines
Quoted 2 lines

I am trying to improve the note around SRCU_SIZE_xx, and this is a trivial improve from 8 SRCU_SIZE state to 5.

I do understand that you would like to remove SRCU_SIZE_* states. What I need from you is why it is important to remove these states. What is being gained by removing them? Why is that gain so important that we should (at the very least) make the state space more complex? And does this change really avoid introducing bugs?

I am driven by the curiosity about the size state machine, and try to document it as Frederic asked [1].

And yes as you demand, it should not introduce any complexity without gains.

Again, appreciate for your detailed explanation. I have a better understainding of the design. Later I will try to document it with a patch.

[1]: https://lore.kernel.org/rcu/20221117142025.GE839309@lothringen/

Quoted 39 lines

Thanx, Paul

Quoted 37 lines

Thanks,

Pingfan

Quoted 33 lines

Thanx, Paul

Quoted 31 lines
Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
v1 -> v2:
rebase onto dev branch

 kernel/rcu/srcutree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +1 -1
index b25f77df9e5e..b0278d1e9c1f 100644
@@ -1165,7 +1165,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	 */
 	idx = __srcu_read_lock_nmisafe(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, get_boot_cpu_id());
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
maintainer
#10 lore raw

Re: [PATCHv2] srcu: Eliminate the requirement of SRCU_SIZE_WAIT_CALL

Quoted 83 lines · Pingfan Liu
Quoted 29 lines · Paul E. McKenney
Quoted 18 lines · Pingfan Liu

Sorry to reply late. I was interrupted by something else.

Quoted 15 lines · Paul E. McKenney
Quoted 13 lines · Pingfan Liu

From: Pingfan Liu <[email protected]>

The state SRCU_SIZE_WAIT_CALL is only used in srcu_gp_start_if_needed(). And it is not needed. Because counter_wrap_check has guarantee that both srcu_gp_seq_needed and srcu_gp_seq_needed_exp are not far behind srcu_gp_seq and no false alarm will be raised by the statement in srcu_gp_start_if_needed()

ULONG_CMP_LT(sdp->srcu_gp_seq_needed, s)

As a result, once if SRCU_SIZE_WAIT_BARRIER is seen, the tree snp can be used immediately, not need to wait for another srcu_gp_end() to update the srcu_gp_seq_needed and srcu_gp_seq_needed_exp to avoid false alarm.

Quoted 5 lines
Quoted 3 lines

During the SRCU_SIZE_WAIT_BARRIER state, all callbacks are queued onto the boot CPU's callback list, but the srcu_barrier() function scans all of the CPUs' callback lists.

So at present, your concern is around how it affects srcu_barrier(), right?

At present, as always, I am concerned about all potential added bugs and potential added complexity, both in terms of lines of code and in terms of size and irregularity of state space.

Quoted 9 lines

And yes, part of the concern about both correctness and complexity is the interaction between call_srcu() and srcu_barrier(). Right now, the design very straightforwardly ensures that rcu_barrier() and the rest of SRCU is looking at all CPU's callback lists before call_srcu() places any callbacks on non-boot CPUs. This is because all the srcu_barrier() calls that think that the system is still in SRCU_SIZE_SMALL or SRCU_SIZE_ALLOC mode must have completed before any call_srcu() invocation uses a non-boot CPU's callback list.

Catch your point. Thanks for patient explanation. And indeed this does not rely on anything like memory order to build a total order, and more reliable.

Quoted 6 lines

Your change causes rcu_barrier() and call_srcu() to start using non-boot CPUs' callback lists at the same time. This change therefore introduces significant state-space complexity. And maybe even bugs, if not today then some later time when someone makes a change that looks obviously correct on the surface, but which subtly weakens the ordering.

Yes, that is true.

Quoted 17 lines
Quoted 9 lines
Quoted 4 lines

So can't this cause srcu_gp_start_if_needed() to rcu_segcblist_advance() and rcu_segcblist_accelerate() a callback list that is guaranteed to be empty? Without this change, the might-not-be-empty boot CPU's callback list is dealt with.

Is it a problem that srcu_gp_start_if_needed() asks rcu_segcblist_advance() and rcu_segcblist_accelerate() to manipulate an not-be-empty callback list ? At SRCU_SIZE_WAIT_BARRIER state, srcu_barrier() will take care of all cpus' callback list.

The other way around, please. My concern is instead that it *fails* to rcu_segcblist_advance() and rcu_segcblist_accelerate() the might-not-be-empty callback lists.

Quoted 3 lines
Quoted 2 lines

What am I missing here? More specifically, what benefits does this change provide? I am not yet seeing any.

Quoted 9 lines
Quoted 2 lines

I am trying to improve the note around SRCU_SIZE_xx, and this is a trivial improve from 8 SRCU_SIZE state to 5.

I do understand that you would like to remove SRCU_SIZE_* states. What I need from you is why it is important to remove these states. What is being gained by removing them? Why is that gain so important that we should (at the very least) make the state space more complex? And does this change really avoid introducing bugs?

I am driven by the curiosity about the size state machine, and try to document it as Frederic asked [1].

Quoted 6 lines

And yes as you demand, it should not introduce any complexity without gains.

Again, appreciate for your detailed explanation. I have a better understainding of the design. Later I will try to document it with a patch.

That sounds good, and I look forward to seeing your state-explanation patch!

Thanx, Paul

Quoted 41 lines

[1]: https://lore.kernel.org/rcu/20221117142025.GE839309@lothringen/

Quoted 39 lines

Thanx, Paul

Quoted 37 lines

Thanks,

Pingfan

Quoted 33 lines

Thanx, Paul

Quoted 31 lines
Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
v1 -> v2:
rebase onto dev branch

 kernel/rcu/srcutree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +1 -1
index b25f77df9e5e..b0278d1e9c1f 100644
@@ -1165,7 +1165,7 @@ static unsigned long srcu_gp_start_if_needed(struct srcu_struct *ssp,
 	 */
 	idx = __srcu_read_lock_nmisafe(ssp);
 	ss_state = smp_load_acquire(&ssp->srcu_size_state);
-	if (ss_state < SRCU_SIZE_WAIT_CALL)
+	if (ss_state < SRCU_SIZE_WAIT_BARRIER)
 		sdp = per_cpu_ptr(ssp->sda, get_boot_cpu_id());
 	else
 		sdp = raw_cpu_ptr(ssp->sda);
Signature · 1 line
2.31.1
#11 lore raw

PATCH 2/3 srcu: protect against concurrent srcu_gp_end()

In reply to Pingfan Liu

At present, the code chunk "Transition to big if needed" is out of the lock srcu_cb_mutex, which allows the following scene:

srcu_gp_end() on cpu1 srcu_gp_end() on cpu2

init_srcu_struct_nodes()
^^ scheduled out in the middle
init_srcu_struct_nodes() ^^ overwrite the context set up by cpu1

Squashing out the race window by putting the code chunk under the protection of srcu_cb_mutex.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]

---
 kernel/rcu/srcutree.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
kernel/rcu/srcutree.c +7 -7
index fe0759d89c2d..281cd69fe804 100644
@@ -811,6 +811,13 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 			spin_unlock_irqrestore_rcu_node(sdp, flags);
 		}
 
+	/* Transition to big if needed. */
+	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
+		if (ss_state == SRCU_SIZE_ALLOC)
+			init_srcu_struct_nodes(ssp, GFP_KERNEL);
+		else
+			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
+	}
 	/* Callback initiation done, allow grace periods after next. */
 	mutex_unlock(&ssp->srcu_cb_mutex);
 
@@ -826,13 +833,6 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 		spin_unlock_irq_rcu_node(ssp);
 	}
 
-	/* Transition to big if needed. */
-	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
-		if (ss_state == SRCU_SIZE_ALLOC)
-			init_srcu_struct_nodes(ssp, GFP_KERNEL);
-		else
-			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
-	}
 }
 
 /*
Signature · 1 line
2.31.1
#12 lore raw

RE: [PATCH 2/3] srcu: protect against concurrent srcu_gp_end()

Quoted 11 lines

Subject: [PATCH 2/3] srcu: protect against concurrent srcu_gp_end()

At present, the code chunk "Transition to big if needed" is out of the lock srcu_cb_mutex, which allows the following scene:

srcu_gp_end() on cpu1 srcu_gp_end() on cpu2

init_srcu_struct_nodes()
^^ scheduled out in the middle
init_srcu_struct_nodes() ^^ overwrite the context set up by cpu1

Hi Pingfan

This scenario shouldn't happen.
The srcu_gp_end() is invoked only in process_srcu() workfn. for the same ssp->work, workqueue can guaranteed that the process_srcu() workfn is executed serially. (view __queue_work () details)

Thanks
Zqiang

Quoted 52 lines

Squashing out the race window by putting the code chunk under the protection of srcu_cb_mutex.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
kernel/rcu/srcutree.c | 14 +++++++-------

1 file changed, 7 insertions(+), 7 deletions(-)
kernel/rcu/srcutree.c +7 -7
index fe0759d89c2d..281cd69fe804 100644
@@ -811,6 +811,13 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 			spin_unlock_irqrestore_rcu_node(sdp, flags);
 		}
 
+	/* Transition to big if needed. */
+	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
+		if (ss_state == SRCU_SIZE_ALLOC)
+			init_srcu_struct_nodes(ssp, GFP_KERNEL);
+		else
+			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
+	}
 	/* Callback initiation done, allow grace periods after next. */
 	mutex_unlock(&ssp->srcu_cb_mutex);
 
@@ -826,13 +833,6 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 		spin_unlock_irq_rcu_node(ssp);
 	}
 
-	/* Transition to big if needed. */
-	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
-		if (ss_state == SRCU_SIZE_ALLOC)
-			init_srcu_struct_nodes(ssp, GFP_KERNEL);
-		else
-			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
-	}
 }
 
 /*
Signature · 1 line
2.31.1
#13 lore raw

Re: [PATCH 2/3] srcu: protect against concurrent srcu_gp_end()

Quoted 12 lines · On Mon, Nov 28, 2022 at 09:20:18AM +0000, Zhang, Qiang1 wrote:
Quoted 11 lines

Subject: [PATCH 2/3] srcu: protect against concurrent srcu_gp_end()

At present, the code chunk "Transition to big if needed" is out of the lock srcu_cb_mutex, which allows the following scene:

srcu_gp_end() on cpu1 srcu_gp_end() on cpu2

init_srcu_struct_nodes()
^^ scheduled out in the middle
init_srcu_struct_nodes() ^^ overwrite the context set up by cpu1

Quoted 7 lines

Hi Pingfan

This scenario shouldn't happen.
The srcu_gp_end() is invoked only in process_srcu() workfn. for the same ssp->work, workqueue can guaranteed that the process_srcu() workfn is executed serially. (view __queue_work () details)

How about this:

In process_one_work()

set_work_pool_and_clear_pending(work, pool->id);
-----> another ssp->work can be queued, which can raise the scenario mentioned above

  worker->current_func(work);

Thanks,

Pingfan

Quoted 57 lines

Thanks
Zqiang

Quoted 52 lines

Squashing out the race window by putting the code chunk under the protection of srcu_cb_mutex.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
kernel/rcu/srcutree.c | 14 +++++++-------

1 file changed, 7 insertions(+), 7 deletions(-)
kernel/rcu/srcutree.c +7 -7
index fe0759d89c2d..281cd69fe804 100644
@@ -811,6 +811,13 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 			spin_unlock_irqrestore_rcu_node(sdp, flags);
 		}
 
+	/* Transition to big if needed. */
+	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
+		if (ss_state == SRCU_SIZE_ALLOC)
+			init_srcu_struct_nodes(ssp, GFP_KERNEL);
+		else
+			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
+	}
 	/* Callback initiation done, allow grace periods after next. */
 	mutex_unlock(&ssp->srcu_cb_mutex);
 
@@ -826,13 +833,6 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 		spin_unlock_irq_rcu_node(ssp);
 	}
 
-	/* Transition to big if needed. */
-	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
-		if (ss_state == SRCU_SIZE_ALLOC)
-			init_srcu_struct_nodes(ssp, GFP_KERNEL);
-		else
-			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
-	}
 }
 
 /*
Signature · 1 line
2.31.1
#14 lore raw

RE: [PATCH 2/3] srcu: protect against concurrent srcu_gp_end()

Quoted 23 lines · On Mon, Nov 28, 2022 at 09:20:18AM +0000, Zhang, Qiang1 wrote:
Quoted 11 lines

Subject: [PATCH 2/3] srcu: protect against concurrent srcu_gp_end()

At present, the code chunk "Transition to big if needed" is out of the lock srcu_cb_mutex, which allows the following scene:

srcu_gp_end() on cpu1 srcu_gp_end() on cpu2

init_srcu_struct_nodes()
^^ scheduled out in the middle
init_srcu_struct_nodes() ^^ overwrite the context set up by cpu1

Hi Pingfan

This scenario shouldn't happen.
The srcu_gp_end() is invoked only in process_srcu() workfn. for the same ssp->work, workqueue can guaranteed that the process_srcu() workfn is executed serially. (view __queue_work () details)

How about this:

Quoted 9 lines

In process_one_work()

set_work_pool_and_clear_pending(work, pool->id);
-----> another ssp->work
can be queued, which can raise the scenario mentioned above

worker->current_func(work);

Hi Pingfan

The following two code snippets ensure that this scenario does not happen

__queue_work():

last_pool = get_work_pool(work);
        if (last_pool && last_pool != pwq->pool) {
                struct worker *worker;

                raw_spin_lock(&last_pool->lock);

                worker = find_worker_executing_work(last_pool, work);

                if (worker && worker->current_pwq->wq == wq) {
                        pwq = worker->current_pwq;

and
process_one_work():

collision = find_worker_executing_work(pool, work);
        if (unlikely(collision)) {
                move_linked_works(work, &collision->scheduled, NULL);
                return;
        }

Thanks
Zqiang

Quoted 3 lines

Thanks,

Pingfan

Quoted 57 lines

Thanks
Zqiang

Quoted 52 lines

Squashing out the race window by putting the code chunk under the protection of srcu_cb_mutex.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
kernel/rcu/srcutree.c | 14 +++++++-------

1 file changed, 7 insertions(+), 7 deletions(-)
kernel/rcu/srcutree.c +7 -7
index fe0759d89c2d..281cd69fe804 100644
@@ -811,6 +811,13 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 			spin_unlock_irqrestore_rcu_node(sdp, flags);
 		}
 
+	/* Transition to big if needed. */
+	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
+		if (ss_state == SRCU_SIZE_ALLOC)
+			init_srcu_struct_nodes(ssp, GFP_KERNEL);
+		else
+			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
+	}
 	/* Callback initiation done, allow grace periods after next. */
 	mutex_unlock(&ssp->srcu_cb_mutex);
 
@@ -826,13 +833,6 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 		spin_unlock_irq_rcu_node(ssp);
 	}
 
-	/* Transition to big if needed. */
-	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
-		if (ss_state == SRCU_SIZE_ALLOC)
-			init_srcu_struct_nodes(ssp, GFP_KERNEL);
-		else
-			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
-	}
 }
 
 /*
Signature · 1 line
2.31.1
#15 lore raw

Re: [PATCH 2/3] srcu: protect against concurrent srcu_gp_end()

Quoted 52 lines · On Mon, Nov 28, 2022 at 02:47:46PM +0000, Zhang, Qiang1 wrote:
Quoted 23 lines · On Mon, Nov 28, 2022 at 09:20:18AM +0000, Zhang, Qiang1 wrote:
Quoted 11 lines

Subject: [PATCH 2/3] srcu: protect against concurrent srcu_gp_end()

At present, the code chunk "Transition to big if needed" is out of the lock srcu_cb_mutex, which allows the following scene:

srcu_gp_end() on cpu1 srcu_gp_end() on cpu2

init_srcu_struct_nodes()
^^ scheduled out in the middle
init_srcu_struct_nodes() ^^ overwrite the context set up by cpu1

Hi Pingfan

This scenario shouldn't happen.
The srcu_gp_end() is invoked only in process_srcu() workfn. for the same ssp->work, workqueue can guaranteed that the process_srcu() workfn is executed serially. (view __queue_work () details)

How about this:

Quoted 9 lines

In process_one_work()

set_work_pool_and_clear_pending(work, pool->id);
-----> another ssp->work
can be queued, which can raise the scenario mentioned above

worker->current_func(work);

Hi Pingfan

The following two code snippets ensure that this scenario does not happen

__queue_work():

last_pool = get_work_pool(work);
        if (last_pool && last_pool != pwq->pool) {
                struct worker *worker;

                raw_spin_lock(&last_pool->lock);

                worker = find_worker_executing_work(last_pool, work);

                if (worker && worker->current_pwq->wq == wq) {
                        pwq = worker->current_pwq;
Quoted 9 lines

and
process_one_work():

collision = find_worker_executing_work(pool, work);
        if (unlikely(collision)) {
                move_linked_works(work, &collision->scheduled, NULL);
                return;
        }

Appreciate for pointing out this. It is helpful.

And I will send out a new patch about it.

Thanks,

Pingfan

Quoted 64 lines

Thanks
Zqiang

Quoted 3 lines

Thanks,

Pingfan

Quoted 57 lines

Thanks
Zqiang

Quoted 52 lines

Squashing out the race window by putting the code chunk under the protection of srcu_cb_mutex.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]
---
kernel/rcu/srcutree.c | 14 +++++++-------

1 file changed, 7 insertions(+), 7 deletions(-)
kernel/rcu/srcutree.c +7 -7
index fe0759d89c2d..281cd69fe804 100644
@@ -811,6 +811,13 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 			spin_unlock_irqrestore_rcu_node(sdp, flags);
 		}
 
+	/* Transition to big if needed. */
+	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
+		if (ss_state == SRCU_SIZE_ALLOC)
+			init_srcu_struct_nodes(ssp, GFP_KERNEL);
+		else
+			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
+	}
 	/* Callback initiation done, allow grace periods after next. */
 	mutex_unlock(&ssp->srcu_cb_mutex);
 
@@ -826,13 +833,6 @@ static void srcu_gp_end(struct srcu_struct *ssp)
 		spin_unlock_irq_rcu_node(ssp);
 	}
 
-	/* Transition to big if needed. */
-	if (ss_state != SRCU_SIZE_SMALL && ss_state != SRCU_SIZE_BIG) {
-		if (ss_state == SRCU_SIZE_ALLOC)
-			init_srcu_struct_nodes(ssp, GFP_KERNEL);
-		else
-			smp_store_release(&ssp->srcu_size_state, ss_state + 1);
-	}
 }
 
 /*
Signature · 1 line
2.31.1
#16 lore raw

PATCH 3/3 srcu: Decrease the srcu size state numbers

In reply to Pingfan Liu

With the previous patch, srcu_cb_mutex is held when setting SRCU_SIZE_WAIT_BARRIER. It means that only two seq snap value can exist. And these two snaps can not see the active snp tree if srcu_gp_start_if_needed() fetches the srcu_size_state before it is updated to SRCU_SIZE_WAIT_BARRIER.

After dispatching these two snap value's callbacks, the mask in snp->srcu_data_have_cbs[idx] is assured to be right. Hence the size state following SRCU_SIZE_WAIT_BARRIER can be shrink to three.

Cc Lai Jiangshan <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc "Zhang, Qiang1" <[email protected]>

To: [email protected]

---
 include/linux/srcutree.h | 15 +++++++++------
 kernel/rcu/srcutree.c    |  3 ---
 2 files changed, 9 insertions(+), 9 deletions(-)
include/linux/srcutree.h +9 -6
index e3014319d1ad..34cb9435c14d 100644
@@ -94,13 +94,16 @@ struct srcu_struct {
 /* Values for size state variable (->srcu_size_state). */
 #define SRCU_SIZE_SMALL		0
 #define SRCU_SIZE_ALLOC		1
+/* If snp tree is initialized */
 #define SRCU_SIZE_WAIT_BARRIER	2
-#define SRCU_SIZE_WAIT_CALL	3
-#define SRCU_SIZE_WAIT_CBS1	4
-#define SRCU_SIZE_WAIT_CBS2	5
-#define SRCU_SIZE_WAIT_CBS3	6
-#define SRCU_SIZE_WAIT_CBS4	7
-#define SRCU_SIZE_BIG		8
+/*
+ * The following two states for the in-flight seq snap, who may not see
+ * the snp tree is active.
+ */
+#define SRCU_SIZE_WAIT_CBS1	3
+#define SRCU_SIZE_WAIT_CBS2	4
+/* All seq snap see the active tree */
+#define SRCU_SIZE_BIG		5
 
 /* Values for state variable (bottom bits of ->srcu_gp_seq). */
 #define SRCU_STATE_IDLE		0
kernel/rcu/srcutree.c +0 -3
index 281cd69fe804..bdf392e2ef63 100644
@@ -1652,11 +1652,8 @@ static const char * const srcu_size_state_name[] = {
 	"SRCU_SIZE_SMALL",
 	"SRCU_SIZE_ALLOC",
 	"SRCU_SIZE_WAIT_BARRIER",
-	"SRCU_SIZE_WAIT_CALL",
 	"SRCU_SIZE_WAIT_CBS1",
 	"SRCU_SIZE_WAIT_CBS2",
-	"SRCU_SIZE_WAIT_CBS3",
-	"SRCU_SIZE_WAIT_CBS4",
 	"SRCU_SIZE_BIG",
 	"SRCU_SIZE_???",
 };
Signature · 1 line
2.31.1