Skip to contentSkip to conversation

get_online_cpus() from a preemptible() context (bug?)

DISCUSSION James Morse 2 participants 6 messages Nov 3, 2017 – Nov 8, 2017

lore
Share & tools
lore
#0 lore raw

Hi Thomas, Peter,

I'm trying to work out what stops a thread being pre-empted and migrated between calling get_online_cpus() and put_online_cpus().

According to __percpu_down_read(), its the pre-empt count:

Quoted 3 lines

* Due to having preemption disabled the decrement happens on
* the same CPU as the increment, avoiding the
* increment-on-one-CPU-and-decrement-on-another problem.

So this:

Review

Quoted 7 lines

void cpus_read_lock(void)

{
       percpu_down_read(&cpu_hotplug_lock);
(quoted diff) +3 -0
+
+       /* Can we migrated before we release this per-cpu lock? */
+       WARN_ON(preemptible());
 }

should never fire?

It does, some of the offenders:

Quoted 9 lines

kmem_cache_create
apply_workqueue_attrs
stop_machine
static_key_enable
lru_add_drain_all
__cpuhp_setup_state
kmem_cache_shrink
vmstat_shepherd
__cpuhp_state_add_instance

Trying to leave preempt disabled between the down/up leads to scheduling-while-atomic instead.

Can you point out what I've missed here?

Thanks,

James

#1 lore raw
Quoted 20 lines · James Morse

Hi Thomas, Peter,

I'm trying to work out what stops a thread being pre-empted and migrated between calling get_online_cpus() and put_online_cpus().

According to __percpu_down_read(), its the pre-empt count:

Quoted 3 lines

* Due to having preemption disabled the decrement happens on
* the same CPU as the increment, avoiding the
* increment-on-one-CPU-and-decrement-on-another problem.

So this:

Review

Quoted 7 lines

void cpus_read_lock(void)

{
       percpu_down_read(&cpu_hotplug_lock);
(quoted diff) +3 -0
+
+       /* Can we migrated before we release this per-cpu lock? */
+       WARN_ON(preemptible());
 }
Quoted 1 line

should never fire?

It should.. You're reading a comment on __percpu_down_read() and using percpu_down_read(), _not_ the same function ;-)

If you look at percpu_down_read(), you'll note it'll disable preemption before calling __percpu_down_read().

And yes, that whole percpu-rwsem code is fairly magical :-)

#2 lore raw
Quoted 5 lines · Peter Zijlstra
Quoted 4 lines · James Morse

Hi Thomas, Peter,

I'm trying to work out what stops a thread being pre-empted and migrated between calling get_online_cpus() and put_online_cpus().

Nothing; why would you think it would? All those functions guarantee is that any CPU observed as being online says online (and its converse, that a CPU observed as being offline, says offline, although less people care about that one).

That is; it serializes against CPU hotplug, nothing else.

#3 lore raw
In reply to Peter Zijlstra

Hi Peter,

(combining your replies)

Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · James Morse

I'm trying to work out what stops a thread being pre-empted and migrated between calling get_online_cpus() and put_online_cpus().

Nothing; why would you think it would?

To stop the this_cpu_*() operations in down/up being applied on different CPUs, affecting a different percpu:read_count.

Quoted 21 lines

All those functions guarantee is
that any CPU observed as being online says online (and its converse, that a CPU observed as being offline, says offline, although less people care about that one).

Quoted 15 lines

According to __percpu_down_read(), its the pre-empt count:

Quoted 3 lines

* Due to having preemption disabled the decrement happens on
* the same CPU as the increment, avoiding the
* increment-on-one-CPU-and-decrement-on-another problem.

So this:

Quoted 7 lines

void cpus_read_lock(void)

{
       percpu_down_read(&cpu_hotplug_lock);
(quoted diff) +3 -0
+
+       /* Can we migrated before we release this per-cpu lock? */
+       WARN_ON(preemptible());
 }
Quoted 4 lines
Quoted 1 line

should never fire?

It should.. You're reading a comment on __percpu_down_read() and using percpu_down_read(), _not_ the same function ;-)

Yes, sorry, I thought you did a better job of describing the case I'm trying to work-out.

Quoted 2 lines

If you look at percpu_down_read(), you'll note it'll disable preemption before calling __percpu_down_read().

Yes, this is how __percpu_down_read() protects the combination of it's fast/slow paths.

But next percpu_down_read() calls preempt_enable(), I can't see what stops us migrating before percpu_up_read() preempt_disable()s to call __this_cpu_dec(), which now affects a different variable.

Quoted 1 line

And yes, that whole percpu-rwsem code is fairly magical :-)

I think I'll file this under magical. That rcu_sync_is_idle() must know something I don't!

Thanks!

James

#4 lore raw
Quoted 10 lines · James Morse
Quoted 2 lines

If you look at percpu_down_read(), you'll note it'll disable preemption before calling __percpu_down_read().

Yes, this is how __percpu_down_read() protects the combination of it's fast/slow paths.

But next percpu_down_read() calls preempt_enable(), I can't see what stops us migrating before percpu_up_read() preempt_disable()s to call __this_cpu_dec(), which now affects a different variable.

Ah, so the two operations that comment talks about are:

    percpu_down_read_preempt_disable()
      preempt_disable();
1)    __this_cpu_inc(*sem->read_count);
      if (unlikely(!rcu_sync_is_idle(&sem->rss)))
	__percpu_down_read()
	  smp_mb()
	  if (likely(!smp_load_acquire(&sem->readers_block))) // false
	  __percpu_up_read()
	    smp_mb()
2)	   __this_cpu_dec(*sem->read_count);
	    rcuwait_wake_up(&sem->writer);
	  preempt_enable_no_resched();

If you want more detail on this, I'll actually have to go think :-)

#5 lore raw

Hi Peter,

Quoted 27 lines · Peter Zijlstra
Quoted 10 lines · James Morse
Quoted 2 lines

If you look at percpu_down_read(), you'll note it'll disable preemption before calling __percpu_down_read().

Yes, this is how __percpu_down_read() protects the combination of it's fast/slow paths.

But next percpu_down_read() calls preempt_enable(), I can't see what stops us migrating before percpu_up_read() preempt_disable()s to call __this_cpu_dec(), which now affects a different variable.

Ah, so the two operations that comment talks about are:

    percpu_down_read_preempt_disable()
      preempt_disable();
1)    __this_cpu_inc(*sem->read_count);
      if (unlikely(!rcu_sync_is_idle(&sem->rss)))
	__percpu_down_read()
	  smp_mb()
	  if (likely(!smp_load_acquire(&sem->readers_block))) // false
	  __percpu_up_read()
	    smp_mb()
2)	   __this_cpu_dec(*sem->read_count);
	    rcuwait_wake_up(&sem->writer);
	  preempt_enable_no_resched();
Quoted 1 line

If you want more detail on this, I'll actually have to go think :-)

I think this was the answer to a much smarter question than mine!

I've tried (and failed) to break it instead. To answer my own question:

I thought this was potentially-broken because the __this_cpu_{add,dec}() out in {get,put}_online_cpus() will operate on different per-cpu read_count variables if we migrate. (not the pair above)

This isn't a problem as the only thing that reads the read_count is readers_active_check(), which per_cpu_sum()s them all together before comparing against zero. As they are all unsigned-ints it uses unsigned-overflow to do the right thing. This even works if a CPU holding a vital part of the read_count is offline, as per_cpu_sum() uses for_each_possible_cpu().

Thanks!

James