get/put_cpu() usage in block/blk-mq.c
Hi Jens,
I was about to fix up this splat..
[ 445.022141] loop: module loaded [ 445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801 [ 445.085873] caller is debug_smp_processor_id+0x17/0x20 [ 445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G E 4.12.0-default #40 [ 445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698 , BIOS -[D6E150AUS-1.10]- 12/15/2010 [ 445.108564] Call Trace: [ 445.111016] dump_stack+0x65/0x89 [ 445.114330] check_preemption_disabled+0xde/0xf0 [ 445.118945] debug_smp_processor_id+0x17/0x20 [ 445.123300] blk_stat_add+0xb0/0x130 [ 445.126876] __blk_mq_complete_request+0xb5/0x150 [ 445.131575] blk_mq_complete_request+0x16/0x20 [ 445.136020] loop_queue_work+0x5f/0xaa0 [loop] [ 445.140461] ? _raw_spin_unlock_irq+0x21/0x40 [ 445.144816] ? finish_task_switch+0x85/0x270 [ 445.149085] ? __schedule+0x291/0x8c0 [ 445.152747] kthread_worker_fn+0xc2/0x1d0 [ 445.156754] kthread+0x114/0x150 [ 445.159983] ? __kthread_init_worker+0xb0/0xb0 [ 445.164424] ? kthread_park+0x60/0x60 [ 445.168085] ret_from_fork+0x2c/0x40
..but then noticed that __blk_mq_run_hw_queue() is also called under get_cpu(), but recently grew a might_sleep(), so maybe wants some block layer eyeballs.
-Mike
Quoted 17 lines · Mike Galbraith
Hi Jens,
I was about to fix up this splat..
[ 445.022141] loop: module loaded [ 445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801 [ 445.085873] caller is debug_smp_processor_id+0x17/0x20 [ 445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G E 4.12.0-default #40 [ 445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698 , BIOS -[D6E150AUS-1.10]- 12/15/2010 [ 445.108564] Call Trace: [ 445.111016] dump_stack+0x65/0x89 [ 445.114330] check_preemption_disabled+0xde/0xf0 [ 445.118945] debug_smp_processor_id+0x17/0x20 [ 445.123300] blk_stat_add+0xb0/0x130 [ 445.126876] __blk_mq_complete_request+0xb5/0x150 [ 445.131575] blk_mq_complete_request+0x16/0x20 [ 445.136020] loop_queue_work+0x5f/0xaa0 [loop]
Quoted 8 lines
[ 445.140461] ? _raw_spin_unlock_irq+0x21/0x40 [ 445.144816] ? finish_task_switch+0x85/0x270 [ 445.149085] ? __schedule+0x291/0x8c0 [ 445.152747] kthread_worker_fn+0xc2/0x1d0 [ 445.156754] kthread+0x114/0x150 [ 445.159983] ? __kthread_init_worker+0xb0/0xb0 [ 445.164424] ? kthread_park+0x60/0x60 [ 445.168085] ret_from_fork+0x2c/0x40
Is it from this_cpu_ptr() in blk_stat_add()?
Quoted 3 lines
..but then noticed that
__blk_mq_run_hw_queue()is also called underget_cpu(), but recently grew amight_sleep(), so maybe wants some block layer eyeballs.
That part is fine. The might sleep depends on BLOCKING being set in the hardware queue flags. And if that is set, then we don't call it under get_cpu().
Signature · 1 line
Jens Axboe
Quoted 27 lines · Jens Axboe
Quoted 17 lines · Mike Galbraith
Hi Jens,
I was about to fix up this splat..
[ 445.022141] loop: module loaded [ 445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801 [ 445.085873] caller is debug_smp_processor_id+0x17/0x20 [ 445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G E 4.12.0-default #40 [ 445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698 , BIOS -[D6E150AUS-1.10]- 12/15/2010 [ 445.108564] Call Trace: [ 445.111016] dump_stack+0x65/0x89 [ 445.114330] check_preemption_disabled+0xde/0xf0 [ 445.118945] debug_smp_processor_id+0x17/0x20 [ 445.123300] blk_stat_add+0xb0/0x130 [ 445.126876] __blk_mq_complete_request+0xb5/0x150 [ 445.131575] blk_mq_complete_request+0x16/0x20 [ 445.136020] loop_queue_work+0x5f/0xaa0 [loop]Quoted 8 lines
[ 445.140461] ? _raw_spin_unlock_irq+0x21/0x40 [ 445.144816] ? finish_task_switch+0x85/0x270 [ 445.149085] ? __schedule+0x291/0x8c0 [ 445.152747] kthread_worker_fn+0xc2/0x1d0 [ 445.156754] kthread+0x114/0x150 [ 445.159983] ? __kthread_init_worker+0xb0/0xb0 [ 445.164424] ? kthread_park+0x60/0x60 [ 445.168085] ret_from_fork+0x2c/0x40
Quoted 1 line
Is it from
this_cpu_ptr()inblk_stat_add()?
Yeah.
Quoted 7 lines
Quoted 3 lines
..but then noticed that
__blk_mq_run_hw_queue()is also called underget_cpu(), but recently grew amight_sleep(), so maybe wants some block layer eyeballs.That part is fine. The might sleep depends on BLOCKING being set in the hardware queue flags. And if that is set, then we don't call it under
get_cpu().
Ah, right.
-Mike
Quoted 28 lines · Mike Galbraith
Quoted 27 lines · Jens Axboe
Quoted 25 lines · Mike Galbraith
Hi Jens,
I was about to fix up this splat..
[ 445.022141] loop: module loaded [ 445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801 [ 445.085873] caller is debug_smp_processor_id+0x17/0x20 [ 445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G E 4.12.0-default #40 [ 445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698 , BIOS -[D6E150AUS-1.10]- 12/15/2010 [ 445.108564] Call Trace: [ 445.111016] dump_stack+0x65/0x89 [ 445.114330] check_preemption_disabled+0xde/0xf0 [ 445.118945] debug_smp_processor_id+0x17/0x20 [ 445.123300] blk_stat_add+0xb0/0x130 [ 445.126876] __blk_mq_complete_request+0xb5/0x150 [ 445.131575] blk_mq_complete_request+0x16/0x20 [ 445.136020] loop_queue_work+0x5f/0xaa0 [loop] [ 445.140461] ? _raw_spin_unlock_irq+0x21/0x40 [ 445.144816] ? finish_task_switch+0x85/0x270 [ 445.149085] ? __schedule+0x291/0x8c0 [ 445.152747] kthread_worker_fn+0xc2/0x1d0 [ 445.156754] kthread+0x114/0x150 [ 445.159983] ? __kthread_init_worker+0xb0/0xb0 [ 445.164424] ? kthread_park+0x60/0x60 [ 445.168085] ret_from_fork+0x2c/0x40
Quoted 3 lines
Quoted 1 line
Is it from
this_cpu_ptr()inblk_stat_add()?Yeah.
So why is this complaining, doesn't rcu_read_lock() disable preemption?
Signature · 1 line
Jens Axboe
Quoted 29 lines · Jens Axboe
Quoted 28 lines · Mike Galbraith
Quoted 27 lines · Jens Axboe
Quoted 25 lines · Mike Galbraith
Hi Jens,
I was about to fix up this splat..
[ 445.022141] loop: module loaded [ 445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801 [ 445.085873] caller is debug_smp_processor_id+0x17/0x20 [ 445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G E 4.12.0-default #40 [ 445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698 , BIOS -[D6E150AUS-1.10]- 12/15/2010 [ 445.108564] Call Trace: [ 445.111016] dump_stack+0x65/0x89 [ 445.114330] check_preemption_disabled+0xde/0xf0 [ 445.118945] debug_smp_processor_id+0x17/0x20 [ 445.123300] blk_stat_add+0xb0/0x130 [ 445.126876] __blk_mq_complete_request+0xb5/0x150 [ 445.131575] blk_mq_complete_request+0x16/0x20 [ 445.136020] loop_queue_work+0x5f/0xaa0 [loop] [ 445.140461] ? _raw_spin_unlock_irq+0x21/0x40 [ 445.144816] ? finish_task_switch+0x85/0x270 [ 445.149085] ? __schedule+0x291/0x8c0 [ 445.152747] kthread_worker_fn+0xc2/0x1d0 [ 445.156754] kthread+0x114/0x150 [ 445.159983] ? __kthread_init_worker+0xb0/0xb0 [ 445.164424] ? kthread_park+0x60/0x60 [ 445.168085] ret_from_fork+0x2c/0x40
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Is it from
this_cpu_ptr()inblk_stat_add()?Yeah.
So why is this complaining, doesn't
rcu_read_lock()disable preemption?
Ah, I guess it doesn't if PREEMPT_RCU is set. How about the below?
block/blk-stat.c +10 -7
@@ -96,13 +96,16 @@ void blk_stat_add(struct request *rq)rcu_read_lock ();list_for_each_entry_rcu (cb, &q->stats->callbacks, list) { -if (blk_stat_is_active (cb)) { - bucket = cb->bucket_fn (rq); -if (bucket <0 ) -continue ; - stat = &this_cpu_ptr (cb->cpu_stat)[bucket]; -__blk_stat_add (stat, value); - } +if (!blk_stat_is_active (cb)) +continue ; + + bucket = cb->bucket_fn (rq); +if (bucket <0 ) +continue ; + + stat = &get_cpu_var (cb->cpu_stat)[bucket]; +__blk_stat_add (stat, value); +put_cpu_var (cb->cpu_stat); }rcu_read_unlock (); }
Signature · 1 line
Jens Axboe
Quoted 33 lines · Jens Axboe
Quoted 32 lines · Jens Axboe
Quoted 31 lines · Mike Galbraith
Quoted 30 lines · Jens Axboe
Quoted 28 lines · Mike Galbraith
Hi Jens,
I was about to fix up this splat..
[ 445.022141] loop: module loaded [ 445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801 [ 445.085873] caller is debug_smp_processor_id+0x17/0x20 [ 445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G E 4.12.0-default #40 [ 445.098838] Hardware name: IBM System x3550 M3 -[7944K3G] -/69Y5698 , BIOS -[D6E150AUS-1.10]- 12/15/2010 [ 445.108564] Call Trace: [ 445.111016] dump_stack+0x65/0x89 [ 445.114330] check_preemption_disabled+0xde/0xf0 [ 445.118945] debug_smp_processor_id+0x17/0x20 [ 445.123300] blk_stat_add+0xb0/0x130 [ 445.126876] __blk_mq_complete_request+0xb5/0x150 [ 445.131575] blk_mq_complete_request+0x16/0x20 [ 445.136020] loop_queue_work+0x5f/0xaa0 [loop] [ 445.140461] ? _raw_spin_unlock_irq+0x21/0x40 [ 445.144816] ? finish_task_switch+0x85/0x270 [ 445.149085] ? __schedule+0x291/0x8c0 [ 445.152747] kthread_worker_fn+0xc2/0x1d0 [ 445.156754] kthread+0x114/0x150 [ 445.159983] ? __kthread_init_worker+0xb0/0xb0 [ 445.164424] ? kthread_park+0x60/0x60 [ 445.168085] ret_from_fork+0x2c/0x40
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Is it from
this_cpu_ptr()inblk_stat_add()?Yeah.
So why is this complaining, doesn't
rcu_read_lock()disable preemption?Ah, I guess it doesn't if
PREEMPT_RCUis set. How about the below?
Should do it. I was about to run LTP (where it turned up) again anyway, I'll add this. No news is good news, what you should hear.
Quoted 29 lines
block/blk-stat.c +10
-7@@ -96,13 +96,16 @@ void blk_stat_add(struct request *rq)rcu_read_lock ();list_for_each_entry_rcu (cb, &q->stats->callbacks, list) { -if (blk_stat_is_active (cb)) { - bucket = cb->bucket_fn (rq); -if (bucket <0 ) -continue ; - stat = &this_cpu_ptr (cb->cpu_stat)[bucket]; -__blk_stat_add (stat, value); - } +if (!blk_stat_is_active (cb)) +continue ; + + bucket = cb->bucket_fn (rq); +if (bucket <0 ) +continue ; + + stat = &get_cpu_var (cb->cpu_stat)[bucket]; +__blk_stat_add (stat, value); +put_cpu_var (cb->cpu_stat); }rcu_read_unlock (); }
Quoted 38 lines · Mike Galbraith
Quoted 37 lines · Jens Axboe
Quoted 36 lines · Jens Axboe
Quoted 34 lines · Mike Galbraith
Quoted 30 lines · Jens Axboe
Quoted 20 lines · Mike Galbraith
Hi Jens,
I was about to fix up this splat..
[ 445.022141] loop: module loaded [ 445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801 [ 445.085873] caller is debug_smp_processor_id+0x17/0x20 [ 445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G E 4.12.0-default #40 [ 445.098838] Hardware name: IBM System x3550 M3 -[7944K3G] -/69Y5698 , BIOS -[D6E150AUS-1.10]- 12/15/2010 [ 445.108564] Call Trace: [ 445.111016] dump_stack+0x65/0x89 [ 445.114330] check_preemption_disabled+0xde/0xf0 [ 445.118945] debug_smp_processor_id+0x17/0x20 [ 445.123300] blk_stat_add+0xb0/0x130 [ 445.126876] __blk_mq_complete_request+0xb5/0x150 [ 445.131575] blk_mq_complete_request+0x16/0x20 [ 445.136020] loop_queue_work+0x5f/0xaa0 [loop]Quoted 8 lines
[ 445.140461] ? _raw_spin_unlock_irq+0x21/0x40 [ 445.144816] ? finish_task_switch+0x85/0x270 [ 445.149085] ? __schedule+0x291/0x8c0 [ 445.152747] kthread_worker_fn+0xc2/0x1d0 [ 445.156754] kthread+0x114/0x150 [ 445.159983] ? __kthread_init_worker+0xb0/0xb0 [ 445.164424] ? kthread_park+0x60/0x60 [ 445.168085] ret_from_fork+0x2c/0x40Quoted 1 line
Is it from
this_cpu_ptr()inblk_stat_add()?Yeah.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
So why is this complaining, doesn't
rcu_read_lock()disable preemption?Ah, I guess it doesn't if
PREEMPT_RCUis set. How about the below?Should do it. I was about to run LTP (where it turned up) again anyway, I'll add this. No news is good news, what you should hear.
Thanks, let me know, so I can add your tested-by (or whatever you prefer).
Signature · 1 line
Jens Axboe
Quoted 7 lines · Jens Axboe
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Is it from
this_cpu_ptr()inblk_stat_add()?Yeah.
So why is this complaining, doesn't
rcu_read_lock()disable preemption?
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Ah, I guess it doesn't if
PREEMPT_RCUis set. How about the below?Should do it. I was about to run LTP (where it turned up) again anyway, I'll add this. No news is good news, what you should hear.
Thanks, let me know, so I can add your tested-by (or whatever you prefer).
Sure, but s/get_cpu_var/get_cpu_ptr first.
-MIke
Quoted 12 lines · Mike Galbraith
Quoted 10 lines · Jens Axboe
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Is it from
this_cpu_ptr()inblk_stat_add()?Yeah.
So why is this complaining, doesn't
rcu_read_lock()disable preemption?Ah, I guess it doesn't if
PREEMPT_RCUis set. How about the below?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Should do it. I was about to run LTP (where it turned up) again anyway, I'll add this. No news is good news, what you should hear.
Thanks, let me know, so I can add your tested-by (or whatever you prefer).
Sure, but s/get_cpu_var/get_cpu_ptr first.
Doh.
block/blk-stat.c +10 -7
@@ -96,13 +96,16 @@ void blk_stat_add(struct request *rq)rcu_read_lock ();list_for_each_entry_rcu (cb, &q->stats->callbacks, list) { -if (blk_stat_is_active (cb)) { - bucket = cb->bucket_fn (rq); -if (bucket <0 ) -continue ; - stat = &this_cpu_ptr (cb->cpu_stat)[bucket]; -__blk_stat_add (stat, value); - } +if (!blk_stat_is_active (cb)) +continue ; + + bucket = cb->bucket_fn (rq); +if (bucket <0 ) +continue ; + + stat = &get_cpu_ptr (cb->cpu_stat)[bucket]; +__blk_stat_add (stat, value); +put_cpu_ptr (cb->cpu_stat); }rcu_read_unlock (); }
Signature · 1 line
Jens Axboe
Quoted 3 lines · Jens Axboe
Quoted 1 line
Sure, but s/get_cpu_var/get_cpu_ptr first.
Doh.
Ditto, box OTOH spotted the booboo instantly :)
-Mike