Bug in workqueues [was: Usb devices randomly aren't detected with 2.6.28-rc1-git1]
A message that is not in the archive (sent off-list or not yet received): [email protected]
This is no longer a USB issue, so I'm asking for help from the general kernel community. Background can be found in Bugzilla #11853.
Quoted 17 lines · Mikko C.
Mikko C. ha scritto:
Quoted 16 lines
Alan Stern ha scritto:
Quoted 7 lines
I don't see anything in there that looks particularly suspicious. The nature of your errors suggests that the default workqueue has crashed or hung, but it shows up okay in the dump. Are you sure this dump was made when the devices failed to appear?
Yes, I'm 100% sure.
Quoted 4 lines
What happens when you try to rmmod the ALSA modules? Does rmmod crash with an error or does it hang? If it hangs, can you get another task dump showing the hanging process?
Quoted 6 lines
Quoted 5 lines
Whenever I try rmmoding something, it hangs (no crash) and I'm not able to do anything, beside moving the mouse.
I posted here: http://marc.info/?l=linux-kernel&m=122502213503239&w=4
but that's probably not enough, so I will try getting a full dump.Here it is: http://bugzilla.kernel.org/attachment.cgi?id=18452&action=view
The task dump shows rmmod waiting for flush_workqueue(). But the events/0 task doesn't appear to be hung, and the task dump taken before running rmmod shows events/0 doing something different.
So apparently flush_workqueue() isn't working.
Alan Stern
Quoted 27 lines · Alan Stern
This is no longer a USB issue, so I'm asking for help from the general kernel community. Background can be found in Bugzilla #11853.
Quoted 22 lines · Mikko C.
Mikko C. ha scritto:
Quoted 21 lines
Alan Stern ha scritto:
Quoted 7 lines
I don't see anything in there that looks particularly suspicious. The nature of your errors suggests that the default workqueue has crashed or hung, but it shows up okay in the dump. Are you sure this dump was made when the devices failed to appear?
Yes, I'm 100% sure.
Quoted 4 lines
What happens when you try to rmmod the ALSA modules? Does rmmod crash with an error or does it hang? If it hangs, can you get another task dump showing the hanging process?
Whenever I try rmmoding something, it hangs (no crash) and I'm not able to do anything, beside moving the mouse.
I posted here: http://marc.info/?l=linux-kernel&m=122502213503239&w=4
but that's probably not enough, so I will try getting a full dump.
Quoted 7 lines
Quoted 1 line
Here it is: http://bugzilla.kernel.org/attachment.cgi?id=18452&action=view
The task dump shows rmmod waiting for
flush_workqueue(). But the events/0 task doesn't appear to be hung, and the task dump taken before running rmmod shows events/0 doing something different.So apparently
flush_workqueue()isn't working.
Let's make that more visible (adding CCs). :-)
Thanks,
Rafael
Quoted 4 lines · Rafael J. Wysocki
Quoted 2 lines
So apparently
flush_workqueue()isn't working.Let's make that more visible (adding CCs). :-)
Isn't this the same thing that was fixed by commit 4403b406d4369a275d483ece6ddee0088cc0d592: aka 'Revert "Call init_workqueues before pre smp initcalls."'
The bug was that init_workqueues was called too early, causing it to have wrong initialization of its CPU masks, which caused various random problems since it wouldn't run workqueues on anything but the boot CPU.
It is hidden by various config options (eg if you enable suspend, for example), which is why it probably took some time for people to notice.
Linus
Quoted 10 lines · Linus Torvalds
Quoted 4 lines · Rafael J. Wysocki
Quoted 2 lines
So apparently
flush_workqueue()isn't working.Let's make that more visible (adding CCs). :-)
Isn't this the same thing that was fixed by commit
4403b406d4369a275d483ece6ddee0088cc0d592: aka 'Revert "Callinit_workqueuesbefore pre smp initcalls."'
OK, I've closed bug #11853 with this assumption. Mikko, if it happens again with 2.6.28-rc1-git3 or later, please reopen.
Thanks,
Rafael
Quoted 3 lines · Rafael J. Wysocki
OK, I've closed bug #11853 with this assumption. Mikko, if it happens again with 2.6.28-rc1-git3 or later, please reopen.
Note: I tried to find Mikko's config, but zlin.dk times out. So I cannot check that he actually has CONFIG_HOTPLUG_CPU off, which is probably required to actually trigger the bug.
But it does sound likely.
However, the "Thunderbird crashes out for me" thing in that same thread may be totally unrelated.
Linus
Linus Torvalds ha scritto:
Quoted 8 lines · Rafael J. Wysocki
Quoted 4 lines
OK, I've closed bug #11853 with this assumption. Mikko, if it happens again with 2.6.28-rc1-git3 or later, please reopen.
Note: I tried to find Mikko's config, but zlin.dk times out. So I cannot check that he actually has
CONFIG_HOTPLUG_CPUoff, which is probably required to actually trigger the bug.
I have:
# CONFIG_HOTPLUG_CPU is not set
Here's the full config: http://pastebin.com/d1481da76
Does -rc1-git2 not have the fix yet?
If not, I'll wait for -git3 and report back.
Quoted 4 lines
But it does sound likely.
However, the "Thunderbird crashes out for me" thing in that same thread may be totally unrelated.
My thunderbird freezes when I receive a message (because it plays a sound I guess). So it's probably related to some sound issues.
Thanks
Quoted 2 lines · Mikko C.
Here's the full config: http://pastebin.com/d1481da76
Ok, you do indeed not have HOTPLUG_CPU, so I'd be willing to bet this is it.
Quoted 1 line
Does -rc1-git2 not have the fix yet?
It should indeed be in -rc1-git2, so you can test it that way.
Linus
Linus Torvalds ha scritto:
Quoted 5 lines · Mikko C.
Quoted 3 lines
Here's the full config: http://pastebin.com/d1481da76
Quoted 7 lines
Ok, you do indeed not have
HOTPLUG_CPU, so I'd be willing to bet this is it.Quoted 2 lines
Does -rc1-git2 not have the fix yet?
It should indeed be in -rc1-git2, so you can test it that way.
-git2 seems to fix all the issues I had:
- rmmod works fine
- Sound is back and apps using it don't freeze
- usb devices seem to be detected correctly
- I will test wifi, but the first boot was ok (if not, I guess it's not related to this).
I guess next time I'll be more patient and wait a little more before trying a new kernel as soon as it's out :P
Thanks!!!
Mikko
Quoted 3 lines · Mikko C.
I guess next time I'll be more patient and wait a little more before trying a new kernel as soon as it's out :P
No, no, please do test. It's how we find these things. It got missed on most machines just because if you happened to have suspend enabled (or just hotplugging cpus in general), the cpu hotplug code would hide the problem.
So having people test things with their odd configs is all good. Even if it inevitably causes some wasted effort too...
Linus