Skip to contentSkip to conversation

Bug in workqueues [was: Usb devices randomly aren't detected with 2.6.28-rc1-git1]

DISCUSSION Alan Stern 4 participants 9 messages Oct 26, 2008 1 message not in the archive

lore
Share & tools
lore

Alan Stern

author
#1 lore raw
In reply to a message not in the archive

This is no longer a USB issue, so I'm asking for help from the general kernel community. Background can be found in Bugzilla #11853.

Quoted 17 lines · Mikko C.

Mikko C. ha scritto:

Quoted 16 lines

Alan Stern ha scritto:

Quoted 7 lines

I don't see anything in there that looks particularly suspicious. The nature of your errors suggests that the default workqueue has crashed or hung, but it shows up okay in the dump. Are you sure this dump was made when the devices failed to appear?

Yes, I'm 100% sure.

Quoted 4 lines

What happens when you try to rmmod the ALSA modules? Does rmmod crash with an error or does it hang? If it hangs, can you get another task dump showing the hanging process?

Quoted 6 lines
Quoted 5 lines

Whenever I try rmmoding something, it hangs (no crash) and I'm not able to do anything, beside moving the mouse.
I posted here: http://marc.info/?l=linux-kernel&m=122502213503239&w=4
but that's probably not enough, so I will try getting a full dump.

Here it is: http://bugzilla.kernel.org/attachment.cgi?id=18452&action=view

The task dump shows rmmod waiting for flush_workqueue(). But the events/0 task doesn't appear to be hung, and the task dump taken before running rmmod shows events/0 doing something different.

So apparently flush_workqueue() isn't working.

Alan Stern

#2 lore raw
Quoted 27 lines · Alan Stern

This is no longer a USB issue, so I'm asking for help from the general kernel community. Background can be found in Bugzilla #11853.

Quoted 22 lines · Mikko C.

Mikko C. ha scritto:

Quoted 21 lines

Alan Stern ha scritto:

Quoted 7 lines

I don't see anything in there that looks particularly suspicious. The nature of your errors suggests that the default workqueue has crashed or hung, but it shows up okay in the dump. Are you sure this dump was made when the devices failed to appear?

Yes, I'm 100% sure.

Quoted 4 lines

What happens when you try to rmmod the ALSA modules? Does rmmod crash with an error or does it hang? If it hangs, can you get another task dump showing the hanging process?

Whenever I try rmmoding something, it hangs (no crash) and I'm not able to do anything, beside moving the mouse.
I posted here: http://marc.info/?l=linux-kernel&m=122502213503239&w=4
but that's probably not enough, so I will try getting a full dump.

Quoted 7 lines
Quoted 1 line

Here it is: http://bugzilla.kernel.org/attachment.cgi?id=18452&action=view

The task dump shows rmmod waiting for flush_workqueue(). But the events/0 task doesn't appear to be hung, and the task dump taken before running rmmod shows events/0 doing something different.

So apparently flush_workqueue() isn't working.

Let's make that more visible (adding CCs). :-)

Thanks,
Rafael

#3 lore raw
Quoted 4 lines · Rafael J. Wysocki
Quoted 2 lines

So apparently flush_workqueue() isn't working.

Let's make that more visible (adding CCs). :-)

Isn't this the same thing that was fixed by commit 4403b406d4369a275d483ece6ddee0088cc0d592: aka 'Revert "Call init_workqueues before pre smp initcalls."'

The bug was that init_workqueues was called too early, causing it to have wrong initialization of its CPU masks, which caused various random problems since it wouldn't run workqueues on anything but the boot CPU.

It is hidden by various config options (eg if you enable suspend, for example), which is why it probably took some time for people to notice.

Linus

#4 lore raw
Quoted 10 lines · Linus Torvalds
Quoted 4 lines · Rafael J. Wysocki
Quoted 2 lines

So apparently flush_workqueue() isn't working.

Let's make that more visible (adding CCs). :-)

Isn't this the same thing that was fixed by commit 4403b406d4369a275d483ece6ddee0088cc0d592: aka 'Revert "Call init_workqueues before pre smp initcalls."'

OK, I've closed bug #11853 with this assumption. Mikko, if it happens again with 2.6.28-rc1-git3 or later, please reopen.

Thanks,
Rafael

#5 lore raw
Quoted 3 lines · Rafael J. Wysocki

OK, I've closed bug #11853 with this assumption. Mikko, if it happens again with 2.6.28-rc1-git3 or later, please reopen.

Note: I tried to find Mikko's config, but zlin.dk times out. So I cannot check that he actually has CONFIG_HOTPLUG_CPU off, which is probably required to actually trigger the bug.

But it does sound likely.

However, the "Thunderbird crashes out for me" thing in that same thread may be totally unrelated.

Linus

#6 lore raw

Linus Torvalds ha scritto:

Quoted 8 lines · Rafael J. Wysocki
Quoted 4 lines

OK, I've closed bug #11853 with this assumption. Mikko, if it happens again with 2.6.28-rc1-git3 or later, please reopen.

Note: I tried to find Mikko's config, but zlin.dk times out. So I cannot check that he actually has CONFIG_HOTPLUG_CPU off, which is probably required to actually trigger the bug.

I have:
# CONFIG_HOTPLUG_CPU is not set

Here's the full config: http://pastebin.com/d1481da76
Does -rc1-git2 not have the fix yet?
If not, I'll wait for -git3 and report back.

Quoted 4 lines

But it does sound likely.

However, the "Thunderbird crashes out for me" thing in that same thread may be totally unrelated.

My thunderbird freezes when I receive a message (because it plays a sound I guess). So it's probably related to some sound issues.
Thanks

#7 lore raw
Quoted 2 lines · Mikko C.

Here's the full config: http://pastebin.com/d1481da76

Ok, you do indeed not have HOTPLUG_CPU, so I'd be willing to bet this is it.

Quoted 1 line

Does -rc1-git2 not have the fix yet?

It should indeed be in -rc1-git2, so you can test it that way.

Linus

#8 lore raw

Linus Torvalds ha scritto:

Quoted 5 lines · Mikko C.
Quoted 3 lines

Here's the full config: http://pastebin.com/d1481da76

Quoted 7 lines

Ok, you do indeed not have HOTPLUG_CPU, so I'd be willing to bet this is it.

Quoted 2 lines

Does -rc1-git2 not have the fix yet?

It should indeed be in -rc1-git2, so you can test it that way.

-git2 seems to fix all the issues I had:

- rmmod works fine
- Sound is back and apps using it don't freeze
- usb devices seem to be detected correctly
- I will test wifi, but the first boot was ok (if not, I guess it's not related to this).

I guess next time I'll be more patient and wait a little more before trying a new kernel as soon as it's out :P

Thanks!!!

Mikko

#9 lore raw
Quoted 3 lines · Mikko C.

I guess next time I'll be more patient and wait a little more before trying a new kernel as soon as it's out :P

No, no, please do test. It's how we find these things. It got missed on most machines just because if you happened to have suspend enabled (or just hotplugging cpus in general), the cpu hotplug code would hide the problem.

So having people test things with their odd configs is all good. Even if it inevitably causes some wasted effort too...

Linus