Skip to contentSkip to conversation

Race between RCU and rmmod

DISCUSSION David Howells 4 participants 4 messages May 2, 2007

lore
Share & tools
lore
#0 lore raw

Hi Dipankar, Rusty,

I seem to have found a race between RCU and rmmod. What I see appears to be an RCU destructor function that has a call pending but lives in a module, gets deleted before the RCU callback is processed:

RIP: 0010:[<ffffffff880329b7>]  [<ffffffff880329b7>]
RSP: 0018:ffffffff805a4f08  EFLAGS: 00010296
RAX: 0000000000000026 RBX: ffff81002fe53a38 RCX: 0000000000000026
RDX: ffff810001f730c0 RSI: 0000000000000001 RDI: 0000000000000002
RBP: ffffffff805a4f18 R08: 0000000000000002 R09: ffffffff80228eab
R10: ffffffff805a4c98 R11: ffffffff805a4e28 R12: ffff81003979e140
R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffffffff80552000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: ffffffff880329b7 CR3: 0000000000201000 CR4: 00000000000006e0
Process ksoftirqd/0 (pid: 3, threadinfo ffff81003e626000, task ffff810001f730c0)
Stack:  ffff81002fe53a38 ffff810001dc5040 ffffffff805a4f48 ffffffff8023946a
 ffff810001dc5140 0000000000000000 000000000000000a 0000000000000000
 ffffffff805a4f58 ffffffff8023952c ffffffff805a4f78 ffffffff8022dca3
Call Trace:
 <IRQ>  [<ffffffff8023946a>] __rcu_process_callbacks+0x15d/0x1fc
 [<ffffffff8023952c>] rcu_process_callbacks+0x23/0x44
 [<ffffffff8022dca3>] tasklet_action+0x5e/0xb2
 [<ffffffff8022db99>] __do_softirq+0x5f/0xe3
 [<ffffffff8022daa9>] ksoftirqd+0x0/0x91
 [<ffffffff8020a94c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020c562>] do_softirq+0x39/0x9f
 [<ffffffff8022dafb>] ksoftirqd+0x52/0x91
 [<ffffffff8023b79e>] kthread+0xd8/0x10e
 [<ffffffff8020a5d8>] child_rip+0xa/0x12
 [<ffffffff8041b100>] _spin_unlock_irq+0x2b/0x31
 [<ffffffff80209cec>] restore_args+0x0/0x30
 [<ffffffff8023b6c6>] kthread+0x0/0x10e
 [<ffffffff8020a5ce>] child_rip+0x0/0x12

I think that rmmod needs to clear the RCU destructor queue, probably inside of __try_stop_module().

David

#1 lore raw
Quoted 11 lines · David Howells

Hi Dipankar, Rusty,

I seem to have found a race between RCU and rmmod. What I see appears to be an RCU destructor function that has a call pending but lives in a module, gets deleted before the RCU callback is processed:

RIP: 0010:[<ffffffff880329b7>]  [<ffffffff880329b7>]

I think that rmmod needs to clear the RCU destructor queue, probably inside of __try_stop_module().

This is why we have rcu_barrier() although the corresponding documentation patch seems to have got dropped. Modules that use RCU must call rcu_barrier() in their cleanup routine.

Thanks
Dipankar

#2 lore raw
Quoted 8 lines · Dipankar Sarma
Quoted 7 lines · David Howells

Hi Dipankar, Rusty,

I seem to have found a race between RCU and rmmod. What I see appears to be an RCU destructor function that has a call pending but lives in a module, gets deleted before the RCU callback is processed:

Quoted 8 lines
Quoted 4 lines
RIP: 0010:[<ffffffff880329b7>]  [<ffffffff880329b7>]

I think that rmmod needs to clear the RCU destructor queue, probably inside of __try_stop_module().

This is why we have rcu_barrier() although the corresponding documentation patch seems to have got dropped. Modules that use RCU must call rcu_barrier() in their cleanup routine.

Hi David, Dipankar,

My first thought was wondering if doing it for them isn't a bad thing,
but they need to do it if they've got other teardown which would mess up RCU callbacks anyway...

Cheers,
Rusty.

#3 lore raw
In reply to Dipankar Sarma
Quoted 8 lines · On Wed, 2 May 2007 17:30:51 +0530 Dipankar Sarma <[email protected]> wrote:
Quoted 7 lines · David Howells

Hi Dipankar, Rusty,

I seem to have found a race between RCU and rmmod. What I see appears to be an RCU destructor function that has a call pending but lives in a module, gets deleted before the RCU callback is processed:

Quoted 9 lines
Quoted 4 lines
RIP: 0010:[<ffffffff880329b7>]  [<ffffffff880329b7>]

I think that rmmod needs to clear the RCU destructor queue, probably inside of __try_stop_module().

This is why we have rcu_barrier() although the corresponding documentation patch seems to have got dropped. Modules that use RCU must call rcu_barrier() in their cleanup routine.

hm, I never knew that.

akpm:/usr/src/linux-2.6.21> grep -r rcu_ drivers | wc -l 182
akpm:/usr/src/linux-2.6.21> grep -r rcu_barrier drivers | wc -l 0

For a start we should undrop that documentation patch, please.