Skip to contentSkip to conversation

srcu: Queue sdp->work when the delay timer is successfully deleted

PATCH In review Zqiang 3 participants 10 messages Jul 9 – Aug 4

Subsystems (3)
lore
Share & tools
lore

Zqiang

author reviewer
#0 lore raw

PATCH 1/1 srcu: Queue sdp->work when the delay timer is successfully deleted

In the cleanup_srcu_struct(), when iterating over per-cpu's srcu_data, the timer_delete_sync(&sdp->delay_work) is called to cancel the delay timer before flush_work(&sdp->work).

However, if the timer_delete_sync() returns 1 means that it successfully deleted an pending timer before it had a chance to fire, also means that the sdp->work cannot be queued, the subsequent flush_work(&sdp->work) will returns immediately without waiting for anything, this causes SRCU callbacks to not be processed.

Fix this by checking the return value of timer_delete_sync(), if it returns 1, explicitly queue sdp->work so that the following flush_work() can correctly wait for the work to complete.

---
 kernel/rcu/srcutree.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/rcu/srcutree.c +5 -1
index 7c2f7cc131f7..02c322b7c6f1 100644
@@ -725,7 +725,11 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 	for_each_possible_cpu(cpu) {
 		struct srcu_data *sdp = per_cpu_ptr(ssp->sda, cpu);
 
-		timer_delete_sync(&sdp->delay_work);
+		//In most scenarios, calling srcu_barrier before cleanup
+		//will not trigger WARN_ON().
+		if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+					rcu_cpu_beenfullyonline(sdp->cpu))
+			queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);
 		flush_work(&sdp->work);
 		if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
 			return; /* Forgot srcu_barrier(), so just leak it! */
Signature · 1 line
2.17.1
maintainer
#1 lore raw
Quoted 10 lines · Zqiang

In the cleanup_srcu_struct(), when iterating over per-cpu's srcu_data, the timer_delete_sync(&sdp->delay_work) is called to cancel the delay timer before flush_work(&sdp->work).

However, if the timer_delete_sync() returns 1 means that it successfully deleted an pending timer before it had a chance to fire, also means that the sdp->work cannot be queued, the subsequent flush_work(&sdp->work) will returns immediately without waiting for anything, this causes SRCU callbacks to not be processed.

Quoted 5 lines

Fix this by checking the return value of timer_delete_sync(), if it returns 1, explicitly queue sdp->work so that the following flush_work() can correctly wait for the work to complete.

Good catch! I updated the commit log and comment as shown below. Does that work for you?

Thanx, Paul

Quoted 23 lines
---
 kernel/rcu/srcutree.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/rcu/srcutree.c +5 -1
index 7c2f7cc131f7..02c322b7c6f1 100644
@@ -725,7 +725,11 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 	for_each_possible_cpu(cpu) {
 		struct srcu_data *sdp = per_cpu_ptr(ssp->sda, cpu);
 
-		timer_delete_sync(&sdp->delay_work);
+		//In most scenarios, calling srcu_barrier before cleanup
+		//will not trigger WARN_ON().
+		if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+					rcu_cpu_beenfullyonline(sdp->cpu))
+			queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);
 		flush_work(&sdp->work);
 		if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
 			return; /* Forgot srcu_barrier(), so just leak it! */
Signature · 1 line
2.17.1

------------------------------------------------------------------------

commit d0b7ba8ff706ad1e3a51ffa46ee79b60ff0203f3

Author: Zqiang <[email protected]>
Date:   Thu Jul 9 18:06:02 2026 +0800

srcu: Queue sdp->work when the delay timer is successfully deleted

In the cleanup_srcu_struct() function, when iterating over per-cpu's srcu_data, timer_delete_sync(&sdp->delay_work) is called to cancel the delayed work before doing flush_work(&sdp->work).

However, suppose that timer_delete_sync() returns 1, which means that it successfully deleted an pending timer before it had a chance to fire. But this also means that the sdp->work will not be queued, so that the subsequent flush_work(&sdp->work) will returns immediately without waiting for anything. Taken together, all of this means that any recently queued SRCU callbacks to not be invoked, which can result in memory leaks, hangs, or worse.

Fix this by checking the return value of timer_delete_sync(), if it returns 1, explicitly queue sdp->work so that the callbacks will be invoked and the following flush_work() will correctly wait for all of those callbacks to finish executing.

kernel/rcu/srcutree.c +5 -1
index 4a00e90e17fc07..8b4e7783bdc963 100644
@@ -701,7 +701,11 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 	for_each_possible_cpu(cpu) {
 		struct srcu_data *sdp = per_cpu_ptr(ssp->sda, cpu);
 
-		timer_delete_sync(&sdp->delay_work);
+		// Call srcu_barrier() before this cleanup_srcu_struct()
+		// to avoid triggering this WARN_ON().
+		if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+					rcu_cpu_beenfullyonline(sdp->cpu))
+			queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);
 		flush_work(&sdp->work);
 		if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
 			return; /* Forgot srcu_barrier(), so just leak it! */
#2 lore raw
In reply to Zqiang

Hello Zqiang,

Quoted 24 lines · Zqiang

In the cleanup_srcu_struct(), when iterating over per-cpu's srcu_data, the timer_delete_sync(&sdp->delay_work) is called to cancel the delay timer before flush_work(&sdp->work).

However, if the timer_delete_sync() returns 1 means that it successfully deleted an pending timer before it had a chance to fire, also means that the sdp->work cannot be queued, the subsequent flush_work(&sdp->work) will returns immediately without waiting for anything, this causes SRCU callbacks to not be processed.

Fix this by checking the return value of timer_delete_sync(), if it returns 1, explicitly queue sdp->work so that the following flush_work() can correctly wait for the work to complete.

---
 kernel/rcu/srcutree.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/rcu/srcutree.c +0 -0
index 7c2f7cc131f7..02c322b7c6f1 100644

kernel/rcu/srcutree.c void cleanup_srcu_struct(struct srcu_struct *ssp) · line 732

Quoted 9 lines
kernel/rcu/srcutree.c +5 -1
@@ -725,7 +725,11 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 	for_each_possible_cpu(cpu) {
 		struct srcu_data *sdp = per_cpu_ptr(ssp->sda, cpu);
 
-		timer_delete_sync(&sdp->delay_work);
+		//In most scenarios, calling srcu_barrier before cleanup
+		//will not trigger WARN_ON().
+		if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+					rcu_cpu_beenfullyonline(sdp->cpu))
+			queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);

I started seeing this on my tests, it is not trivial to decode this one, but, I can try harder if _really_ needed.

07:11:44 Invalid logical block size (65535)
Invalid logical block size (256)
07:11:45 WARNING: kernel/rcu/srcutree.c:706 at cleanup_srcu_struct+0x408/0x7e8, CPU#50: stress-ng-loop/1233713 ^[[36m[warn]^[[0m

           Modules linked in: sch_htb(E) cls_u32(E) cls_cgroup(E) cls_basic(E) act_police(E) act_mirred(E) act_bpf(E) sch_fq(E) br_netfilter(E) udp_diag(E) i
           CPU: 50 UID: 0 PID: 1233713 Comm: stress-ng-loop Kdump: loaded Tainted: G            E       7.2.0-rc5-next-20260730upstream-baseline #1 PREEMPT(f
           Tainted: [E]=UNSIGNED_MODULE
           pstate: 03401009 (nzcv daif +PAN -UAO +TCO +DIT +SSBS BTYPE=--)
           pc : cleanup_srcu_struct+0x408/0x7e8
           lr : cleanup_srcu_struct+0x3dc/0x7e8
           sp : ffff800150357b30
           x29: ffff800150357b50 x28: fffffdffbdb51c40 x27: 0000000000000000
           x26: dfff800000000000 x25: 1fffe000202daa5f x24: ffff800083bbb010
           x23: ffff800083f67210 x22: ffff0001016d52f8 x21: 00007dc98b656c40
           x20: 1ffff000107ece42 x19: 0000000000000003 x18: 1fffe006d6db3f20
           x17: 0000000000000000 x16: ffff800083f67000 x15: 0000000000000001
           x14: 1fffe006d6db6388 x13: 0000000000000000 x12: 0000000000000000
           x11: 0000000000000003 x10: fffffdffbdb51e40 x9 : dfff800000000000
           x8 : 0000000100000000 x7 : 0000000000000000 x6 : ffff80008148f060
           x5 : 0000000000000000 x4 : 0000000000000001 x3 : 0000000000000010
           x2 : ffff800150357a60 x1 : ffff80008036827c x0 : 0000000000000001
           Call trace:
            cleanup_srcu_struct+0x408/0x7e8 (P)
            blk_mq_free_tag_set+0x4b4/0x5f0
            loop_remove+0x50/0xd8 [loop]
            loop_control_ioctl+0x340/0x438 [loop]
            __arm64_sys_ioctl+0xc04/0x1310
            invoke_syscall+0x74/0x188
            do_el0_svc+0x10c/0x198
            el0_svc+0x64/0x260
            el0t_64_sync_handler+0x84/0x130
            el0t_64_sync+0x198/0x1a0
           irq event stamp: 0
           hardirqs last  enabled at (0): [<0000000000000000>] 0x0
           hardirqs last disabled at (0): [<ffff800080100734>] copy_process+0x109c/0x31e8
           softirqs last  enabled at (0): [<ffff800080100760>] copy_process+0x10c8/0x31e8
           softirqs last disabled at (0): [<0000000000000000>] 0x0
           ---[ end trace 0000000000000000 ]---
author reviewer
#3 lore raw
Quoted 40 lines

Hello Zqiang,

Quoted 28 lines · Zqiang

In the cleanup_srcu_struct(), when iterating over per-cpu's srcu_data,
the timer_delete_sync(&sdp->delay_work) is called to cancel the delay timer before flush_work(&sdp->work).

However, if the timer_delete_sync() returns 1 means that it successfully deleted an pending timer before it had a chance to fire, also means that the sdp->work cannot be queued, the subsequent flush_work(&sdp->work) will returns immediately without waiting for anything, this causes SRCU callbacks to not be processed.

Fix this by checking the return value of timer_delete_sync(), if it returns 1, explicitly queue sdp->work so that the following flush_work() can correctly wait for the work to complete.

---

 kernel/rcu/srcutree.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 7c2f7cc131f7..02c322b7c6f1 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -725,7 +725,11 @@ void cleanup_srcu_struct(struct srcu_struct *ssp) for_each_possible_cpu(cpu) {
struct srcu_data *sdp = per_cpu_ptr(ssp->sda, cpu);

Quoted 7 lines

- timer_delete_sync(&sdp->delay_work);
+ //In most scenarios, calling srcu_barrier before cleanup
+ //will not trigger WARN_ON().
+ if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+ rcu_cpu_beenfullyonline(sdp->cpu))
+ queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);

Quoted 3 lines

I started seeing this on my tests, it is not trivial to decode this one, but, I can try harder if _really_ needed.

The scenario I can think of is that we missed the call to srcu_barrier() before cleanup_srcu_struct():

loop_add()
->blk_mq_alloc_tag_set

    init_srcu_struct(&set->tags_srcu)

    blk_mq_alloc_set_map_and_rqs() {
     ->__blk_mq_alloc_rq_maps()
       ->__blk_mq_alloc_map_and_rqs() return error
         goto out_unwind: __blk_mq_free_map_and_rqs()
                          ->blk_mq_free_rq_map()
                            ->blk_mq_free_tags()
                              ->call_srcu(&set->tags_srcu, &tags->rcu_head, blk_mq_free_tags_callback);
    } return error 
     
    goto out_free_mq_map:
         ....
         cleanup_srcu_struct(&set->tags_srcu)
         -> trigger WARN_ON(timer_delete_sync(&sdp->delay_work)

Can you try the following patch?

block/blk-mq.c +1 -0
index 2c850330a32b..6eaea75d0363 100644
@@ -4925,6 +4925,7 @@ int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set)
        kfree(set->tags);
        set->tags = NULL;
 out_cleanup_tags_srcu:
+       srcu_barrier(&set->tags_srcu);
        cleanup_srcu_struct(&set->tags_srcu);
 out_cleanup_srcu:
        if (set->flags & BLK_MQ_F_BLOCKING)

Thanks
Zqiang

Quoted 39 lines

07:11:44 Invalid logical block size (65535)
Invalid logical block size (256)
07:11:45 WARNING: kernel/rcu/srcutree.c:706 at cleanup_srcu_struct+0x408/0x7e8, CPU#50: stress-ng-loop/1233713 ^[[36m[warn]^[[0m

 Modules linked in: sch_htb(E) cls_u32(E) cls_cgroup(E) cls_basic(E) act_police(E) act_mirred(E) act_bpf(E) sch_fq(E) br_netfilter(E) udp_diag(E) i
 CPU: 50 UID: 0 PID: 1233713 Comm: stress-ng-loop Kdump: loaded Tainted: G E 7.2.0-rc5-next-20260730upstream-baseline #1 PREEMPT(f
 Tainted: [E]=UNSIGNED_MODULE
 pstate: 03401009 (nzcv daif +PAN -UAO +TCO +DIT +SSBS BTYPE=--)
 pc : cleanup_srcu_struct+0x408/0x7e8
 lr : cleanup_srcu_struct+0x3dc/0x7e8
 sp : ffff800150357b30
 x29: ffff800150357b50 x28: fffffdffbdb51c40 x27: 0000000000000000
 x26: dfff800000000000 x25: 1fffe000202daa5f x24: ffff800083bbb010
 x23: ffff800083f67210 x22: ffff0001016d52f8 x21: 00007dc98b656c40
 x20: 1ffff000107ece42 x19: 0000000000000003 x18: 1fffe006d6db3f20
 x17: 0000000000000000 x16: ffff800083f67000 x15: 0000000000000001
 x14: 1fffe006d6db6388 x13: 0000000000000000 x12: 0000000000000000
 x11: 0000000000000003 x10: fffffdffbdb51e40 x9 : dfff800000000000
 x8 : 0000000100000000 x7 : 0000000000000000 x6 : ffff80008148f060
 x5 : 0000000000000000 x4 : 0000000000000001 x3 : 0000000000000010
 x2 : ffff800150357a60 x1 : ffff80008036827c x0 : 0000000000000001
 Call trace:
 cleanup_srcu_struct+0x408/0x7e8 (P)
 blk_mq_free_tag_set+0x4b4/0x5f0
 loop_remove+0x50/0xd8 [loop]
 loop_control_ioctl+0x340/0x438 [loop]
 __arm64_sys_ioctl+0xc04/0x1310
 invoke_syscall+0x74/0x188
 do_el0_svc+0x10c/0x198
 el0_svc+0x64/0x260
 el0t_64_sync_handler+0x84/0x130
 el0t_64_sync+0x198/0x1a0
 irq event stamp: 0
 hardirqs last enabled at (0): [<0000000000000000>] 0x0
 hardirqs last disabled at (0): [<ffff800080100734>] copy_process+0x109c/0x31e8
 softirqs last enabled at (0): [<ffff800080100760>] copy_process+0x10c8/0x31e8
 softirqs last disabled at (0): [<0000000000000000>] 0x0
 ---[ end trace 0000000000000000 ]---
#4 lore raw
Quoted 65 lines · Zqiang
Quoted 40 lines

Hello Zqiang,

Quoted 28 lines · Zqiang

In the cleanup_srcu_struct(), when iterating over per-cpu's srcu_data,
the timer_delete_sync(&sdp->delay_work) is called to cancel the delay timer before flush_work(&sdp->work).

However, if the timer_delete_sync() returns 1 means that it successfully deleted an pending timer before it had a chance to fire, also means that the sdp->work cannot be queued, the subsequent flush_work(&sdp->work) will returns immediately without waiting for anything, this causes SRCU callbacks to not be processed.

Fix this by checking the return value of timer_delete_sync(), if it returns 1, explicitly queue sdp->work so that the following flush_work() can correctly wait for the work to complete.

---

 kernel/rcu/srcutree.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 7c2f7cc131f7..02c322b7c6f1 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -725,7 +725,11 @@ void cleanup_srcu_struct(struct srcu_struct *ssp) for_each_possible_cpu(cpu) {
struct srcu_data *sdp = per_cpu_ptr(ssp->sda, cpu);

Quoted 7 lines

- timer_delete_sync(&sdp->delay_work);
+ //In most scenarios, calling srcu_barrier before cleanup
+ //will not trigger WARN_ON().
+ if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+ rcu_cpu_beenfullyonline(sdp->cpu))
+ queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);

Quoted 3 lines

I started seeing this on my tests, it is not trivial to decode this one, but, I can try harder if _really_ needed.

The scenario I can think of is that we missed the call to srcu_barrier() before cleanup_srcu_struct():

loop_add()
->blk_mq_alloc_tag_set

    init_srcu_struct(&set->tags_srcu)

    blk_mq_alloc_set_map_and_rqs() {
     ->__blk_mq_alloc_rq_maps()
       ->__blk_mq_alloc_map_and_rqs() return error
         goto out_unwind: __blk_mq_free_map_and_rqs()
                          ->blk_mq_free_rq_map()
                            ->blk_mq_free_tags()
                              ->call_srcu(&set->tags_srcu, &tags->rcu_head, blk_mq_free_tags_callback);
    } return error 
     
    goto out_free_mq_map:
         ....
         cleanup_srcu_struct(&set->tags_srcu)
         -> trigger WARN_ON(timer_delete_sync(&sdp->delay_work)
Quoted 1 line

Can you try the following patch?

I tried it, and it does not help -- the WARN still fires at the same rate. I think the analysis points at the wrong call site.

Setup: linux-next-20260731 (arm64), 32 vCPU VM, HZ=1000, PROVE_LOCKING and DEBUG_OBJECTS_TIMERS enabled, reproducer stress-ng --loop 32 --timeout 60s.

baseline 3 x WARN srcutree.c:706 in 60s
+ your blk-mq patch 4 x WARN srcutree.c:706 in 60s

(3 vs 4 is just jitter on a one-jiffy race, not a regression.)

The reason it cannot help is that the splat comes from blk_mq_free_tag_set(), not from the blk_mq_alloc_tag_set() error path your patch touches. All four splats in the patched run have the same trace:

    cleanup_srcu_struct+0x274/0x450 (P)
    blk_mq_free_tag_set+0x1a4/0x1e0
    loop_remove+0x2c/0x78
    loop_control_ioctl+0x248/0x2a0
    __arm64_sys_ioctl+0x9c0/0xb00

I also put a pr_warn() at out_cleanup_tags_srcu: to be sure -- it fired zero times over the whole run, so loop_add() never takes that error path in this workload.

Why do you wangt to have this
WARN_ON(timer_delete_sync(&sdp->delay_work)) ?

timer_delete_sync() != 0 means "a timer was armed", not "callbacks are pending".

author reviewer
#5 lore raw
Quoted 68 lines · Zqiang
Quoted 66 lines

Hello Zqiang,

Quoted 25 lines · Zqiang

In the cleanup_srcu_struct(), when iterating over per-cpu's srcu_data, the timer_delete_sync(&sdp->delay_work) is called to cancel the delay timer before flush_work(&sdp->work).

However, if the timer_delete_sync() returns 1 means that it successfully deleted an pending timer before it had a chance to fire, also means that the sdp->work cannot be queued, the subsequent flush_work(&sdp->work) will returns immediately without waiting for anything, this causes SRCU callbacks to not be processed.

Fix this by checking the return value of timer_delete_sync(), if it returns 1, explicitly queue sdp->work so that the following flush_work() can correctly wait for the work to complete.

---
kernel/rcu/srcutree.c | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)
kernel/rcu/srcutree.c +0 -0
index 7c2f7cc131f7..02c322b7c6f1 100644

kernel/rcu/srcutree.c void cleanup_srcu_struct(struct srcu_struct *ssp) · line 733

Quoted 10 lines
kernel/rcu/srcutree.c +5 -1
@@ -725,7 +725,11 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 for_each_possible_cpu(cpu) {
 struct srcu_data *sdp = per_cpu_ptr(ssp->sda, cpu);
 
- timer_delete_sync(&sdp->delay_work);
+ //In most scenarios, calling srcu_barrier before cleanup
+ //will not trigger WARN_ON().
+ if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+ rcu_cpu_beenfullyonline(sdp->cpu))
+ queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);
 

I started seeing this on my tests, it is not trivial to decode this one, but, I can try harder if _really_ needed.

The scenario I can think of is that we missed the call to srcu_barrier() before cleanup_srcu_struct():

loop_add()
->blk_mq_alloc_tag_set
init_srcu_struct(&set->tags_srcu)

blk_mq_alloc_set_map_and_rqs() {
->__blk_mq_alloc_rq_maps()
->__blk_mq_alloc_map_and_rqs() return error
goto out_unwind: __blk_mq_free_map_and_rqs() ->blk_mq_free_rq_map()
->blk_mq_free_tags()
->call_srcu(&set->tags_srcu, &tags->rcu_head, blk_mq_free_tags_callback); } return error

goto out_free_mq_map:
....
cleanup_srcu_struct(&set->tags_srcu)
-> trigger WARN_ON(timer_delete_sync(&sdp->delay_work)

Quoted 8 lines
Quoted 2 lines

Can you try the following patch?

I tried it, and it does not help -- the WARN still fires at the same rate. I think the analysis points at the wrong call site.

Setup: linux-next-20260731 (arm64), 32 vCPU VM, HZ=1000, PROVE_LOCKING and DEBUG_OBJECTS_TIMERS enabled, reproducer stress-ng --loop 32 --timeout 60s.

Thanks for provide testing methods, I will also testing it.

Quoted 10 lines

baseline 3 x WARN srcutree.c:706 in 60s
+ your blk-mq patch 4 x WARN srcutree.c:706 in 60s

(3 vs 4 is just jitter on a one-jiffy race, not a regression.)

The reason it cannot help is that the splat comes from blk_mq_free_tag_set(), not from the blk_mq_alloc_tag_set() error path your patch touches. All four splats in the patched run have the same trace:

Quoted 5 lines
 cleanup_srcu_struct+0x274/0x450 (P)
 blk_mq_free_tag_set+0x1a4/0x1e0
 loop_remove+0x2c/0x78
 loop_control_ioctl+0x248/0x2a0
 __arm64_sys_ioctl+0x9c0/0xb00

This may trigger a new srcu grace period again during the window period between srcu-barrier() and cleanup_srcu_struct().

Quoted 10 lines

I also put a pr_warn() at out_cleanup_tags_srcu: to be sure -- it fired zero times over the whole run, so loop_add() never takes that error path in this workload.

Why do you wangt to have this
WARN_ON(timer_delete_sync(&sdp->delay_work)) ?

timer_delete_sync() != 0 "a timer was armed", not "callbacks are pending".

There are only two types of return values for timer_delete_sync(), return 0 or 1, the timer_delete_sync() != 0 means that this timer was pending and has been deactivated, right?

Thanks
Zqiang

Quoted 1 line
author reviewer
#6 lore raw
Quoted 107 lines
Quoted 69 lines · Zqiang

Hello Zqiang,

Quoted 25 lines · Zqiang

In the cleanup_srcu_struct(), when iterating over per-cpu's srcu_data, the timer_delete_sync(&sdp->delay_work) is called to cancel the delay timer before flush_work(&sdp->work).

However, if the timer_delete_sync() returns 1 means that it successfully deleted an pending timer before it had a chance to fire, also means that the sdp->work cannot be queued, the subsequent flush_work(&sdp->work) will returns immediately without waiting for anything, this causes SRCU callbacks to not be processed.

Fix this by checking the return value of timer_delete_sync(), if it returns 1, explicitly queue sdp->work so that the following flush_work() can correctly wait for the work to complete.

---
kernel/rcu/srcutree.c | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)
kernel/rcu/srcutree.c +0 -0
index 7c2f7cc131f7..02c322b7c6f1 100644

kernel/rcu/srcutree.c void cleanup_srcu_struct(struct srcu_struct *ssp) · line 733

Quoted 10 lines
kernel/rcu/srcutree.c +5 -1
@@ -725,7 +725,11 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 for_each_possible_cpu(cpu) {
 struct srcu_data *sdp = per_cpu_ptr(ssp->sda, cpu);
 
- timer_delete_sync(&sdp->delay_work);
+ //In most scenarios, calling srcu_barrier before cleanup
+ //will not trigger WARN_ON().
+ if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+ rcu_cpu_beenfullyonline(sdp->cpu))
+ queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);
 

I started seeing this on my tests, it is not trivial to decode this one, but, I can try harder if _really_ needed.

The scenario I can think of is that we missed the call to srcu_barrier() before cleanup_srcu_struct():

loop_add()
->blk_mq_alloc_tag_set
init_srcu_struct(&set->tags_srcu)

blk_mq_alloc_set_map_and_rqs() {
->__blk_mq_alloc_rq_maps()
->__blk_mq_alloc_map_and_rqs() return error
goto out_unwind: __blk_mq_free_map_and_rqs() ->blk_mq_free_rq_map()
->blk_mq_free_tags()
->call_srcu(&set->tags_srcu, &tags->rcu_head, blk_mq_free_tags_callback); } return error

goto out_free_mq_map:
....
cleanup_srcu_struct(&set->tags_srcu)
-> trigger WARN_ON(timer_delete_sync(&sdp->delay_work)

Quoted 8 lines

Can you try the following patch?

I tried it, and it does not help -- the WARN still fires at the same rate. I think the analysis points at the wrong call site.

Setup: linux-next-20260731 (arm64), 32 vCPU VM, HZ=1000, PROVE_LOCKING and DEBUG_OBJECTS_TIMERS enabled, reproducer stress-ng --loop 32 --timeout 60s.

Thanks for provide testing methods, I will also testing it.

Quoted 11 lines

baseline 3 x WARN srcutree.c:706 in 60s
+ your blk-mq patch 4 x WARN srcutree.c:706 in 60s

(3 vs 4 is just jitter on a one-jiffy race, not a regression.)

The reason it cannot help is that the splat comes from blk_mq_free_tag_set(), not from the blk_mq_alloc_tag_set() error path your patch touches. All four splats in the patched run have the same trace:

Quoted 6 lines
 cleanup_srcu_struct+0x274/0x450 (P)
 blk_mq_free_tag_set+0x1a4/0x1e0
 loop_remove+0x2c/0x78
 loop_control_ioctl+0x248/0x2a0
 __arm64_sys_ioctl+0x9c0/0xb00

This may trigger a new srcu grace period again during the window period between srcu-barrier() and cleanup_srcu_struct().

Quoted 8 lines

I also put a pr_warn() at out_cleanup_tags_srcu: to be sure -- it fired
zero times over the whole run, so loop_add() never takes that error path in this workload.

Why do you wangt to have this
WARN_ON(timer_delete_sync(&sdp->delay_work)) ?

Quoted 6 lines
Quoted 3 lines

timer_delete_sync() != 0 "a timer was armed", not "callbacks are pending".

There are only two types of return values for timer_delete_sync(), return 0 or 1, the timer_delete_sync() != 0 means that this timer was pending and has been deactivated, right?

How about this?

kernel/rcu/srcutree.c +2 -1
index b9fe57ff9100..62da31ec2bd0 100644
@@ -751,7 +751,8 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 
                // Call srcu_barrier() before this cleanup_srcu_struct()
                // to avoid triggering this WARN_ON().
-               if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+               if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist) &&
+                               timer_delete_sync(&sdp->delay_work)) &&
                                        rcu_cpu_beenfullyonline(sdp->cpu))
                        queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);
                flush_work(&sdp->work);

Thanks
Zqiang

Quoted 8 lines

Thanks
Zqiang

Quoted 3 lines
author reviewer
#7 lore raw
Quoted 121 lines
Quoted 107 lines · Zqiang

Hello Zqiang,

Quoted 25 lines · Zqiang

In the cleanup_srcu_struct(), when iterating over per-cpu's srcu_data, the timer_delete_sync(&sdp->delay_work) is called to cancel the delay timer before flush_work(&sdp->work).

However, if the timer_delete_sync() returns 1 means that it successfully deleted an pending timer before it had a chance to fire, also means that the sdp->work cannot be queued, the subsequent flush_work(&sdp->work) will returns immediately without waiting for anything, this causes SRCU callbacks to not be processed.

Fix this by checking the return value of timer_delete_sync(), if it returns 1, explicitly queue sdp->work so that the following flush_work() can correctly wait for the work to complete.

---
kernel/rcu/srcutree.c | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)
kernel/rcu/srcutree.c +0 -0
index 7c2f7cc131f7..02c322b7c6f1 100644

kernel/rcu/srcutree.c void cleanup_srcu_struct(struct srcu_struct *ssp) · line 733

Quoted 10 lines
kernel/rcu/srcutree.c +5 -1
@@ -725,7 +725,11 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 for_each_possible_cpu(cpu) {
 struct srcu_data *sdp = per_cpu_ptr(ssp->sda, cpu);
 
- timer_delete_sync(&sdp->delay_work);
+ //In most scenarios, calling srcu_barrier before cleanup
+ //will not trigger WARN_ON().
+ if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+ rcu_cpu_beenfullyonline(sdp->cpu))
+ queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);
 

I started seeing this on my tests, it is not trivial to decode this one, but, I can try harder if _really_ needed.

The scenario I can think of is that we missed the call to srcu_barrier() before cleanup_srcu_struct():

loop_add()
->blk_mq_alloc_tag_set
init_srcu_struct(&set->tags_srcu)

blk_mq_alloc_set_map_and_rqs() {
->__blk_mq_alloc_rq_maps()
->__blk_mq_alloc_map_and_rqs() return error
goto out_unwind: __blk_mq_free_map_and_rqs() ->blk_mq_free_rq_map()
->blk_mq_free_tags()
->call_srcu(&set->tags_srcu, &tags->rcu_head, blk_mq_free_tags_callback); } return error

goto out_free_mq_map:
....
cleanup_srcu_struct(&set->tags_srcu)
-> trigger WARN_ON(timer_delete_sync(&sdp->delay_work)

Can you try the following patch?

I tried it, and it does not help -- the WARN still fires at the same rate. I think the analysis points at the wrong call site.

Setup: linux-next-20260731 (arm64), 32 vCPU VM, HZ=1000, PROVE_LOCKING and DEBUG_OBJECTS_TIMERS enabled, reproducer stress-ng --loop 32 --timeout 60s.

Thanks for provide testing methods, I will also testing it.

baseline 3 x WARN srcutree.c:706 in 60s
+ your blk-mq patch 4 x WARN srcutree.c:706 in 60s

(3 vs 4 is just jitter on a one-jiffy race, not a regression.)

The reason it cannot help is that the splat comes from blk_mq_free_tag_set(), not from the blk_mq_alloc_tag_set() error path your patch touches. All four splats in the patched run have the same trace:

 cleanup_srcu_struct+0x274/0x450 (P)
 blk_mq_free_tag_set+0x1a4/0x1e0
 loop_remove+0x2c/0x78
 loop_control_ioctl+0x248/0x2a0
 __arm64_sys_ioctl+0x9c0/0xb00

This may trigger a new srcu grace period again during the window period between srcu-barrier() and cleanup_srcu_struct().

I also put a pr_warn() at out_cleanup_tags_srcu: to be sure -- it fired zero times over the whole run, so loop_add() never takes that error path in this workload.

Why do you wangt to have this
WARN_ON(timer_delete_sync(&sdp->delay_work)) ?

Quoted 7 lines

timer_delete_sync() != 0 "a timer was armed", not "callbacks are pending".

There are only two types of return values for timer_delete_sync(), return 0 or 1, the timer_delete_sync() != 0 means that this timer was pending and has been deactivated, right?

How about this?

kernel/rcu/srcutree.c +0 -0
index b9fe57ff9100..62da31ec2bd0 100644

kernel/rcu/srcutree.c void cleanup_srcu_struct(struct srcu_struct *ssp) · line 758

Quoted 9 lines
kernel/rcu/srcutree.c +2 -1
@@ -751,7 +751,8 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 
 // Call srcu_barrier() before this cleanup_srcu_struct()
 // to avoid triggering this WARN_ON().
- if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
+ if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist) &&
+ timer_delete_sync(&sdp->delay_work)) &&
 rcu_cpu_beenfullyonline(sdp->cpu))
 queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);
 flush_work(&sdp->work);

Please ignore this change, this is my mistake.

There is a scenario where the 'rcu_segcblist_n_cbs(&sdp->srcu_cblist) == 0' srcu_barrier() cannot intercept.
however, for sup->srcu_size_state being between SRCU_SIZE_WAIT_BARRIER and SRCU_SIZE_BIG, we will queue sdp->delay_work for all CPUs belonging to the leaf SRCU node, regardless of whether there is a callback on the current CPU's sdp->srcu_cblist in srcu_gp_end().

so the timer_delete_sync(&sdp->delay_work) should be called unconditionally. to ensure that the timer callback has ended or the timer which in pending status has been successfully deleted.

kernel/rcu/srcutree.c +3 -2
index b9fe57ff9100..d13c12f61150 100644
@@ -751,8 +751,9 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 
                // Call srcu_barrier() before this cleanup_srcu_struct()
                // to avoid triggering this WARN_ON().
-               if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
-                                       rcu_cpu_beenfullyonline(sdp->cpu))
+               if (WARN_ON(timer_delete_sync(&sdp->delay_work) &&
+                       rcu_segcblist_n_cbs(&sdp->srcu_cblist)) &&
+                               rcu_cpu_beenfullyonline(sdp->cpu))
                        queue_work_on(sdp->cpu, rcu_gp_wq, &sdp->work);
                flush_work(&sdp->work);
                if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))

Thanks
Zqiang

Quoted 9 lines

Thanks
Zqiang

Quoted 4 lines

Thanks
Zqiang

#8 lore raw

kernel/rcu/srcutree.c void cleanup_srcu_struct(struct srcu_struct *ssp) · line 756

Quoted 13 lines · Zqiang
kernel/rcu/srcutree.c +3 -2
index b9fe57ff9100..d13c12f61150 100644
@@ -751,8 +751,9 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
 
                // Call srcu_barrier() before this cleanup_srcu_struct()
                // to avoid triggering this WARN_ON().
-               if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
-                                       rcu_cpu_beenfullyonline(sdp->cpu))
+               if (WARN_ON(timer_delete_sync(&sdp->delay_work) &&
+                       rcu_segcblist_n_cbs(&sdp->srcu_cblist)) &&
+                               rcu_cpu_beenfullyonline(sdp->cpu))

Isn't the WARN_ON() the same? thus it will hit again?

author reviewer
#9 lore raw
Quoted 10 lines · Zqiang
Quoted 8 lines
kernel/rcu/srcutree.c +0 -0
 index b9fe57ff9100..d13c12f61150 100644
 --- a/kernel/rcu/srcutree.c
 +++ b/kernel/rcu/srcutree.c
 @@ -751,8 +751,9 @@ void cleanup_srcu_struct(struct srcu_struct *ssp)
Quoted 9 lines
Quoted 8 lines

// Call srcu_barrier() before this cleanup_srcu_struct() // to avoid triggering this WARN_ON().
- if (WARN_ON(timer_delete_sync(&sdp->delay_work)) &&
- rcu_cpu_beenfullyonline(sdp->cpu))
+ if (WARN_ON(timer_delete_sync(&sdp->delay_work) &&
+ rcu_segcblist_n_cbs(&sdp->srcu_cblist)) &&
+ rcu_cpu_beenfullyonline(sdp->cpu))

Isn't the WARN_ON() the same? thus it will hit again?

WARN_ON(timer_delete_sync(&sdp->delay_work) && rcu_segcblist_n_cbs(&sdp->srcu_cblist))

Thanks
Zqiang

Quoted 1 line