Skip to contentSkip to conversation

get/put_cpu() usage in block/blk-mq.c

DISCUSSION Mike Galbraith 2 participants 10 messages May 9, 2017 Block layer

lore
Share & tools
lore
#0 lore raw

Hi Jens,

I was about to fix up this splat..

[  445.022141] loop: module loaded
[  445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801
[  445.085873] caller is debug_smp_processor_id+0x17/0x20
[  445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G            E   4.12.0-default #40
[  445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698     , BIOS -[D6E150AUS-1.10]- 12/15/2010
[  445.108564] Call Trace:
[  445.111016]  dump_stack+0x65/0x89
[  445.114330]  check_preemption_disabled+0xde/0xf0
[  445.118945]  debug_smp_processor_id+0x17/0x20
[  445.123300]  blk_stat_add+0xb0/0x130
[  445.126876]  __blk_mq_complete_request+0xb5/0x150
[  445.131575]  blk_mq_complete_request+0x16/0x20
[  445.136020]  loop_queue_work+0x5f/0xaa0 [loop]
[  445.140461]  ? _raw_spin_unlock_irq+0x21/0x40
[  445.144816]  ? finish_task_switch+0x85/0x270
[  445.149085]  ? __schedule+0x291/0x8c0
[  445.152747]  kthread_worker_fn+0xc2/0x1d0
[  445.156754]  kthread+0x114/0x150
[  445.159983]  ? __kthread_init_worker+0xb0/0xb0
[  445.164424]  ? kthread_park+0x60/0x60
[  445.168085]  ret_from_fork+0x2c/0x40

..but then noticed that __blk_mq_run_hw_queue() is also called under get_cpu(), but recently grew a might_sleep(), so maybe wants some block layer eyeballs.

-Mike

#1 lore raw
Quoted 17 lines · Mike Galbraith

Hi Jens,

I was about to fix up this splat..

[  445.022141] loop: module loaded
[  445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801
[  445.085873] caller is debug_smp_processor_id+0x17/0x20
[  445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G            E   4.12.0-default #40
[  445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698     , BIOS -[D6E150AUS-1.10]- 12/15/2010
[  445.108564] Call Trace:
[  445.111016]  dump_stack+0x65/0x89
[  445.114330]  check_preemption_disabled+0xde/0xf0
[  445.118945]  debug_smp_processor_id+0x17/0x20
[  445.123300]  blk_stat_add+0xb0/0x130
[  445.126876]  __blk_mq_complete_request+0xb5/0x150
[  445.131575]  blk_mq_complete_request+0x16/0x20
[  445.136020]  loop_queue_work+0x5f/0xaa0 [loop]
Quoted 8 lines
[  445.140461]  ? _raw_spin_unlock_irq+0x21/0x40
[  445.144816]  ? finish_task_switch+0x85/0x270
[  445.149085]  ? __schedule+0x291/0x8c0
[  445.152747]  kthread_worker_fn+0xc2/0x1d0
[  445.156754]  kthread+0x114/0x150
[  445.159983]  ? __kthread_init_worker+0xb0/0xb0
[  445.164424]  ? kthread_park+0x60/0x60
[  445.168085]  ret_from_fork+0x2c/0x40

Is it from this_cpu_ptr() in blk_stat_add()?

Quoted 3 lines

..but then noticed that __blk_mq_run_hw_queue() is also called under get_cpu(), but recently grew a might_sleep(), so maybe wants some block layer eyeballs.

That part is fine. The might sleep depends on BLOCKING being set in the hardware queue flags. And if that is set, then we don't call it under get_cpu().

Signature · 1 line
Jens Axboe
#2 lore raw
Quoted 27 lines · Jens Axboe
Quoted 17 lines · Mike Galbraith

Hi Jens,

I was about to fix up this splat..

[  445.022141] loop: module loaded
[  445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801
[  445.085873] caller is debug_smp_processor_id+0x17/0x20
[  445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G            E   4.12.0-default #40
[  445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698     , BIOS -[D6E150AUS-1.10]- 12/15/2010
[  445.108564] Call Trace:
[  445.111016]  dump_stack+0x65/0x89
[  445.114330]  check_preemption_disabled+0xde/0xf0
[  445.118945]  debug_smp_processor_id+0x17/0x20
[  445.123300]  blk_stat_add+0xb0/0x130
[  445.126876]  __blk_mq_complete_request+0xb5/0x150
[  445.131575]  blk_mq_complete_request+0x16/0x20
[  445.136020]  loop_queue_work+0x5f/0xaa0 [loop]
Quoted 8 lines
[  445.140461]  ? _raw_spin_unlock_irq+0x21/0x40
[  445.144816]  ? finish_task_switch+0x85/0x270
[  445.149085]  ? __schedule+0x291/0x8c0
[  445.152747]  kthread_worker_fn+0xc2/0x1d0
[  445.156754]  kthread+0x114/0x150
[  445.159983]  ? __kthread_init_worker+0xb0/0xb0
[  445.164424]  ? kthread_park+0x60/0x60
[  445.168085]  ret_from_fork+0x2c/0x40
Quoted 1 line

Is it from this_cpu_ptr() in blk_stat_add()?

Yeah.

Quoted 7 lines
Quoted 3 lines

..but then noticed that __blk_mq_run_hw_queue() is also called under get_cpu(), but recently grew a might_sleep(), so maybe wants some block layer eyeballs.

That part is fine. The might sleep depends on BLOCKING being set in the hardware queue flags. And if that is set, then we don't call it under get_cpu().

Ah, right.

-Mike

#3 lore raw
Quoted 28 lines · Mike Galbraith
Quoted 27 lines · Jens Axboe
Quoted 25 lines · Mike Galbraith

Hi Jens,

I was about to fix up this splat..

[  445.022141] loop: module loaded
[  445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801
[  445.085873] caller is debug_smp_processor_id+0x17/0x20
[  445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G            E   4.12.0-default #40
[  445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698     , BIOS -[D6E150AUS-1.10]- 12/15/2010
[  445.108564] Call Trace:
[  445.111016]  dump_stack+0x65/0x89
[  445.114330]  check_preemption_disabled+0xde/0xf0
[  445.118945]  debug_smp_processor_id+0x17/0x20
[  445.123300]  blk_stat_add+0xb0/0x130
[  445.126876]  __blk_mq_complete_request+0xb5/0x150
[  445.131575]  blk_mq_complete_request+0x16/0x20
[  445.136020]  loop_queue_work+0x5f/0xaa0 [loop]
[  445.140461]  ? _raw_spin_unlock_irq+0x21/0x40
[  445.144816]  ? finish_task_switch+0x85/0x270
[  445.149085]  ? __schedule+0x291/0x8c0
[  445.152747]  kthread_worker_fn+0xc2/0x1d0
[  445.156754]  kthread+0x114/0x150
[  445.159983]  ? __kthread_init_worker+0xb0/0xb0
[  445.164424]  ? kthread_park+0x60/0x60
[  445.168085]  ret_from_fork+0x2c/0x40
Quoted 3 lines
Quoted 1 line

Is it from this_cpu_ptr() in blk_stat_add()?

Yeah.

So why is this complaining, doesn't rcu_read_lock() disable preemption?

Signature · 1 line
Jens Axboe
#4 lore raw
Quoted 29 lines · Jens Axboe
Quoted 28 lines · Mike Galbraith
Quoted 27 lines · Jens Axboe
Quoted 25 lines · Mike Galbraith

Hi Jens,

I was about to fix up this splat..

[  445.022141] loop: module loaded
[  445.078116] BUG: using smp_processor_id() in preemptible [00000000] code: loop0/3801
[  445.085873] caller is debug_smp_processor_id+0x17/0x20
[  445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G            E   4.12.0-default #40
[  445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]-/69Y5698     , BIOS -[D6E150AUS-1.10]- 12/15/2010
[  445.108564] Call Trace:
[  445.111016]  dump_stack+0x65/0x89
[  445.114330]  check_preemption_disabled+0xde/0xf0
[  445.118945]  debug_smp_processor_id+0x17/0x20
[  445.123300]  blk_stat_add+0xb0/0x130
[  445.126876]  __blk_mq_complete_request+0xb5/0x150
[  445.131575]  blk_mq_complete_request+0x16/0x20
[  445.136020]  loop_queue_work+0x5f/0xaa0 [loop]
[  445.140461]  ? _raw_spin_unlock_irq+0x21/0x40
[  445.144816]  ? finish_task_switch+0x85/0x270
[  445.149085]  ? __schedule+0x291/0x8c0
[  445.152747]  kthread_worker_fn+0xc2/0x1d0
[  445.156754]  kthread+0x114/0x150
[  445.159983]  ? __kthread_init_worker+0xb0/0xb0
[  445.164424]  ? kthread_park+0x60/0x60
[  445.168085]  ret_from_fork+0x2c/0x40
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Is it from this_cpu_ptr() in blk_stat_add()?

Yeah.

So why is this complaining, doesn't rcu_read_lock() disable preemption?

Ah, I guess it doesn't if PREEMPT_RCU is set. How about the below?

block/blk-stat.c +10 -7
index 6c2f40940439..022f5925a427 100644
@@ -96,13 +96,16 @@ void blk_stat_add(struct request *rq)
 
 	rcu_read_lock();
 	list_for_each_entry_rcu(cb, &q->stats->callbacks, list) {
-		if (blk_stat_is_active(cb)) {
-			bucket = cb->bucket_fn(rq);
-			if (bucket < 0)
-				continue;
-			stat = &this_cpu_ptr(cb->cpu_stat)[bucket];
-			__blk_stat_add(stat, value);
-		}
+		if (!blk_stat_is_active(cb))
+			continue;
+
+		bucket = cb->bucket_fn(rq);
+		if (bucket < 0)
+			continue;
+
+		stat = &get_cpu_var(cb->cpu_stat)[bucket];
+		__blk_stat_add(stat, value);
+		put_cpu_var(cb->cpu_stat);
 	}
 	rcu_read_unlock();
 }
Signature · 1 line
Jens Axboe
#5 lore raw
Quoted 33 lines · Jens Axboe
Quoted 32 lines · Jens Axboe
Quoted 31 lines · Mike Galbraith
Quoted 30 lines · Jens Axboe
Quoted 28 lines · Mike Galbraith

Hi Jens,

I was about to fix up this splat..

[  445.022141] loop: module loaded
[  445.078116] BUG: using smp_processor_id() in preemptible
[00000000] code: loop0/3801
[  445.085873] caller is debug_smp_processor_id+0x17/0x20
[  445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G       
     E   4.12.0-default #40
[  445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]
-/69Y5698     , BIOS -[D6E150AUS-1.10]- 12/15/2010
[  445.108564] Call Trace:
[  445.111016]  dump_stack+0x65/0x89
[  445.114330]  check_preemption_disabled+0xde/0xf0
[  445.118945]  debug_smp_processor_id+0x17/0x20
[  445.123300]  blk_stat_add+0xb0/0x130
[  445.126876]  __blk_mq_complete_request+0xb5/0x150
[  445.131575]  blk_mq_complete_request+0x16/0x20
[  445.136020]  loop_queue_work+0x5f/0xaa0 [loop]
[  445.140461]  ? _raw_spin_unlock_irq+0x21/0x40
[  445.144816]  ? finish_task_switch+0x85/0x270
[  445.149085]  ? __schedule+0x291/0x8c0
[  445.152747]  kthread_worker_fn+0xc2/0x1d0
[  445.156754]  kthread+0x114/0x150
[  445.159983]  ? __kthread_init_worker+0xb0/0xb0
[  445.164424]  ? kthread_park+0x60/0x60
[  445.168085]  ret_from_fork+0x2c/0x40
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Is it from this_cpu_ptr() in blk_stat_add()?

Yeah.

So why is this complaining, doesn't rcu_read_lock() disable preemption?

Ah, I guess it doesn't if PREEMPT_RCU is set. How about the below?

Should do it. I was about to run LTP (where it turned up) again anyway, I'll add this. No news is good news, what you should hear.

Quoted 29 lines
block/blk-stat.c +10 -7
index 6c2f40940439..022f5925a427 100644
@@ -96,13 +96,16 @@ void blk_stat_add(struct request *rq)
 
 	rcu_read_lock();
 	list_for_each_entry_rcu(cb, &q->stats->callbacks, list) {
-		if (blk_stat_is_active(cb)) {
-			bucket = cb->bucket_fn(rq);
-			if (bucket < 0)
-				continue;
-			stat = &this_cpu_ptr(cb->cpu_stat)[bucket];
-			__blk_stat_add(stat, value);
-		}
+		if (!blk_stat_is_active(cb))
+			continue;
+
+		bucket = cb->bucket_fn(rq);
+		if (bucket < 0)
+			continue;
+
+		stat = &get_cpu_var(cb->cpu_stat)[bucket];
+		__blk_stat_add(stat, value);
+		put_cpu_var(cb->cpu_stat);
 	}
 	rcu_read_unlock();
 }
#6 lore raw
Quoted 38 lines · Mike Galbraith
Quoted 37 lines · Jens Axboe
Quoted 36 lines · Jens Axboe
Quoted 34 lines · Mike Galbraith
Quoted 30 lines · Jens Axboe
Quoted 20 lines · Mike Galbraith

Hi Jens,

I was about to fix up this splat..

[  445.022141] loop: module loaded
[  445.078116] BUG: using smp_processor_id() in preemptible
[00000000] code: loop0/3801
[  445.085873] caller is debug_smp_processor_id+0x17/0x20
[  445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G       
     E   4.12.0-default #40
[  445.098838] Hardware name: IBM System x3550 M3 -[7944K3G]
-/69Y5698     , BIOS -[D6E150AUS-1.10]- 12/15/2010
[  445.108564] Call Trace:
[  445.111016]  dump_stack+0x65/0x89
[  445.114330]  check_preemption_disabled+0xde/0xf0
[  445.118945]  debug_smp_processor_id+0x17/0x20
[  445.123300]  blk_stat_add+0xb0/0x130
[  445.126876]  __blk_mq_complete_request+0xb5/0x150
[  445.131575]  blk_mq_complete_request+0x16/0x20
[  445.136020]  loop_queue_work+0x5f/0xaa0 [loop]
Quoted 8 lines
[  445.140461]  ? _raw_spin_unlock_irq+0x21/0x40
[  445.144816]  ? finish_task_switch+0x85/0x270
[  445.149085]  ? __schedule+0x291/0x8c0
[  445.152747]  kthread_worker_fn+0xc2/0x1d0
[  445.156754]  kthread+0x114/0x150
[  445.159983]  ? __kthread_init_worker+0xb0/0xb0
[  445.164424]  ? kthread_park+0x60/0x60
[  445.168085]  ret_from_fork+0x2c/0x40
Quoted 1 line

Is it from this_cpu_ptr() in blk_stat_add()?

Yeah.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

So why is this complaining, doesn't rcu_read_lock() disable preemption?

Ah, I guess it doesn't if PREEMPT_RCU is set. How about the below?

Should do it. I was about to run LTP (where it turned up) again anyway, I'll add this. No news is good news, what you should hear.

Thanks, let me know, so I can add your tested-by (or whatever you prefer).

Signature · 1 line
Jens Axboe
#7 lore raw
Quoted 7 lines · Jens Axboe
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Is it from this_cpu_ptr() in blk_stat_add()?

Yeah.

So why is this complaining, doesn't rcu_read_lock() disable preemption?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Ah, I guess it doesn't if PREEMPT_RCU is set. How about the below?

Should do it. I was about to run LTP (where it turned up) again anyway, I'll add this. No news is good news, what you should hear.

Thanks, let me know, so I can add your tested-by (or whatever you prefer).

Sure, but s/get_cpu_var/get_cpu_ptr first.

-MIke

#8 lore raw
Quoted 12 lines · Mike Galbraith
Quoted 10 lines · Jens Axboe
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Is it from this_cpu_ptr() in blk_stat_add()?

Yeah.

So why is this complaining, doesn't rcu_read_lock() disable preemption?

Ah, I guess it doesn't if PREEMPT_RCU is set. How about the below?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Should do it. I was about to run LTP (where it turned up) again anyway, I'll add this. No news is good news, what you should hear.

Thanks, let me know, so I can add your tested-by (or whatever you prefer).

Sure, but s/get_cpu_var/get_cpu_ptr first.

Doh.

block/blk-stat.c +10 -7
index 6c2f40940439..c52356d90fe3 100644
@@ -96,13 +96,16 @@ void blk_stat_add(struct request *rq)
 
 	rcu_read_lock();
 	list_for_each_entry_rcu(cb, &q->stats->callbacks, list) {
-		if (blk_stat_is_active(cb)) {
-			bucket = cb->bucket_fn(rq);
-			if (bucket < 0)
-				continue;
-			stat = &this_cpu_ptr(cb->cpu_stat)[bucket];
-			__blk_stat_add(stat, value);
-		}
+		if (!blk_stat_is_active(cb))
+			continue;
+
+		bucket = cb->bucket_fn(rq);
+		if (bucket < 0)
+			continue;
+
+		stat = &get_cpu_ptr(cb->cpu_stat)[bucket];
+		__blk_stat_add(stat, value);
+		put_cpu_ptr(cb->cpu_stat);
 	}
 	rcu_read_unlock();
 }
Signature · 1 line
Jens Axboe
#9 lore raw
Quoted 3 lines · Jens Axboe
Quoted 1 line

Sure, but s/get_cpu_var/get_cpu_ptr first.

Doh.

Ditto, box OTOH spotted the booboo instantly :)

-Mike