sym53c8xx: WARNING on module unload, crashes on further usage
Hi,
we're experiencing problems using changer connected on LSI Logic / Symbios Logic 53c875 adaptor (using sym53c8xx module). Sometimes changer gets stuck, and reloading sym53c8xx module is the only way to help. But it alwas issues following warning, and on subsequent mtx usage, the whole machine usually crashes...
the warning issued on module unload:
WARNING: at /home/src/redhat/BUILD/linux-2.6.29/arch/x86/include/asm/dma-mapping.h:297 ___free_dma_mem_cluster+0xba/0xc0 [sym53c8xx]() Hardware name: PDSM4+ Modules linked in: ipmi_si ipmi_devintf ipmi_msghandler nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs ipv6 reiserfs dm_mirror dm_region_hash dm_log dm_multipath scsi_dh dm_mod sbs sbshc fan battery ac parport_pc lp parport nvram st osst joydev sg usbhid container piix pata_acpi ide_pci_generic sym53c8xx(-) scsi_transport_spi arcmsr thermal processor thermal_sys buttone1000e rng_core i2c_i801 ata_piix i3000_edac iTCO_wdt ata_generic i2c_core shpchp edac_core pci_hotplug pcspkr rtc ide_gd_mod ide_core ahci libata sd_mod scsi_mod raid456 async_xor async_memcpy async_tx xor raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: freq_tabl
e]
Pid: 5495, comm: rmmod Not tainted 2.6.29lb.00_01_PRE09 #1
Call Trace: [<c022e7d6>] warn_slowpath+0x86/0xa0 [<c02e312f>] ? free_proc_entry+0x5f/0x80 [<c02e312f>] ? free_proc_entry+0x5f/0x80 [<f80ae38a>] ___free_dma_mem_cluster+0xba/0xc0 [sym53c8xx] [<f80ae231>] __sym_mfree+0x91/0xe0 [sym53c8xx] [<c046d82c>] ? _spin_lock_irqsave+0x4c/0x60 [<f80ae741>] __sym_mfree_dma+0x51/0x80 [sym53c8xx] [<f80a7851>] sym_hcb_free+0x51/0x190 [sym53c8xx] [<f80a5659>] sym_free_resources+0x39/0x80 [sym53c8xx] [<c03352b9>] ? __delay+0x9/0x10 [<f80a5735>] sym_detach+0x95/0xb0 [sym53c8xx] [<f80a5775>] sym2_remove+0x25/0x50 [sym53c8xx] [<c0340c99>] pci_device_remove+0x19/0x40 [<c03954e9>] __device_release_driver+0x49/0x80 [<c03955b7>] driver_detach+0x97/0xa0 [<c03948b3>] bus_remove_driver+0x73/0x90 [<c0395a5f>] driver_unregister+0x2f/0x40 [<c0340ef1>] pci_unregister_driver+0x21/0x80 [<f80af0dd>] sym2_exit+0xd/0x20 [sym53c8xx] [<c0258b06>] sys_delete_module+0x126/0x1f0 [<c021aa76>] ? do_page_fault+0x1f6/0x630 [<c0268da0>] ? audit_syscall_exit+0x310/0x380 [<c028d699>] ? sys_munmap+0x39/0x50 [<c0203415>] sysenter_do_call+0x12/0x2c
We're having those problems on 2.6.29, but it was happening also on older kernels (2.6.26, ...) Could somebody have a look on this?
Should more information be needed, I'll gladly provide it.
thanks a lot in advance.
best regards
nik
Signature · 13 lines
------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] -------------------------------------
Quoted 4 lines · Nikola Ciprich
Hi,
we're experiencing problems using changer connected on LSI Logic / Symbios Logic 53c875 adaptor (using sym53c8xx module). Sometimes changer gets stuck, and reloading sym53c8xx module is the only way to help. But it alwas issues following warning, and on subsequent mtx usage, the whole machine usually crashes...
The warning is actually harmless. What error messages do you get when the machine crashes?
Signature · 4 lines
Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step."
Hi Matthew,
well, I've got VERY crappy screen photo, my collegue made using his cellphone, You can check it on http://nik.lbox.cz/downloads/sym53c8xx-crash.jpg
But we can quite easily reproduce the problem, I'll try to do it today evening and make a better screenshot...
I'll report further then.
Regarding warning, couldn't it then be surpressed, if it's harmless? Kernel then gets tainted, which makes me a bit nervous :-)
regards
nik
Quoted 19 lines · Matthew Wilcox
Quoted 4 lines · Nikola Ciprich
Hi,
we're experiencing problems using changer connected on LSI Logic / Symbios Logic 53c875 adaptor (using sym53c8xx module). Sometimes changer gets stuck, and reloading sym53c8xx module is the only way to help. But it alwas issues following warning, and on subsequent mtx usage, the whole machine usually crashes...The warning is actually harmless. What error messages do you get when the machine crashes?
Signature · 8 lines
Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step." -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 13 lines
------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] -------------------------------------
Hmm,
maybe a bit usefull can be begin of crash message, he got on console:
databox [488754.946366] BUG: spinlock recursion on CPU#1, swapper/0
databox [488754.950343] lock: f70ddbe0, .magic: dead4ead, .owner: swapper/0, .owner_cpu: 1
n.
Quoted 51 lines · Nikola Ciprich
Hi Matthew,
well, I've got VERY crappy screen photo, my collegue made using his cellphone, You can check it on http://nik.lbox.cz/downloads/sym53c8xx-crash.jpg
But we can quite easily reproduce the problem, I'll try to do it today evening and make a better screenshot...
I'll report further then.
Regarding warning, couldn't it then be surpressed, if it's harmless? Kernel then gets tainted, which makes me a bit nervous :-)
regards
nikQuoted 19 lines · Matthew Wilcox
Quoted 4 lines · Nikola Ciprich
Hi,
we're experiencing problems using changer connected on LSI Logic / Symbios Logic 53c875 adaptor (using sym53c8xx module). Sometimes changer gets stuck, and reloading sym53c8xx module is the only way to help. But it alwas issues following warning, and on subsequent mtx usage, the whole machine usually crashes...The warning is actually harmless. What error messages do you get when the machine crashes?
Signature · 8 lines
Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step." -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.htmlSignature · 17 lines
------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] ------------------------------------- -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 13 lines
------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] -------------------------------------
crash unrelated to sym53c8xx
Quoted 5 lines · Nikola Ciprich
Hmm,
maybe a bit usefull can be begin of crash message, he got on console:databox [488754.946366] BUG: spinlock recursion on CPU#1, swapper/0
databox [488754.950343] lock:f70ddbe0, .magic:dead4ead, .owner: swapper/0, .owner_cpu: 1
Yes, that's somewhat more useful ... the tail of the stack trace in the photo indicates this isn't (at least directly) a bug in the sym53c8xx driver. Can you get the rest of the crash message? netconsole might help here if you don't have a serial console.
Quoted 10 lines · Nikola Ciprich
Quoted 8 lines
Hi Matthew,
well, I've got VERY crappy screen photo, my collegue made using his cellphone, You can check it on http://nik.lbox.cz/downloads/sym53c8xx-crash.jpg
But we can quite easily reproduce the problem, I'll try to do it today evening and make a better screenshot...
I'll report further then.
Regarding warning, couldn't it then be surpressed, if it's harmless? Kernel then gets tainted, which makes me a bit nervous :-)
There's a few reasons this warning hasn't been fixed yet:
- It only appears on module unload. Most people don't unload sym53c8xx.
- The problem that it's warning about is only a problem on ARM. There's not a huge userbase of ARM users with sym53c8xx cards.
- Fixing it requires a fairly large rewrite of how sym2 manages its memory. I've done about half the work required, but I've basically run out of interest.
Quoted 62 lines
Quoted 43 lines
regards
nikQuoted 19 lines · Matthew Wilcox
Quoted 4 lines · Nikola Ciprich
Hi,
we're experiencing problems using changer connected on LSI Logic / Symbios Logic 53c875 adaptor (using sym53c8xx module). Sometimes changer gets stuck, and reloading sym53c8xx module is the only way to help. But it alwas issues following warning, and on subsequent mtx usage, the whole machine usually crashes...The warning is actually harmless. What error messages do you get when the machine crashes?
Signature · 8 lines
Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step." -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.htmlSignature · 17 lines
------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] ------------------------------------- -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.htmlSignature · 17 lines
------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] ------------------------------------- -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 4 lines
Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step."
Re: crash unrelated to sym53c8xx
Quoted 4 lines
Yes, that's somewhat more useful ... the tail of the stack trace in the photo indicates this isn't (at least directly) a bug in the sym53c8xx driver. Can you get the rest of the crash message? netconsole might help here if you don't have a serial console.
OK, I'll try to get the whole message using IPMI console. I'll post it in ~2 hours...
Quoted 8 lines
There's a few reasons this warning hasn't been fixed yet:
- It only appears on module unload. Most people don't unload sym53c8xx.
- The problem that it's warning about is only a problem on ARM. There's not a huge userbase of ARM users with sym53c8xx cards.
- Fixing it requires a fairly large rewrite of how sym2 manages its memory. I've done about half the work required, but I've basically run out of interest.
Hmm, maybe I don't understand what You mean by ARM, but it's i686 machine, not arm (actually it's opteron, but running i686 OS).
but if it's reallu harmless, I don't have that big problem with it. only the crashes really worry me :)
Re: crash unrelated to sym53c8xx
Quoted 8 lines · Nikola Ciprich
Quoted 8 lines
There's a few reasons this warning hasn't been fixed yet:
- It only appears on module unload. Most people don't unload sym53c8xx.
- The problem that it's warning about is only a problem on ARM. There's not a huge userbase of ARM users with sym53c8xx cards.
- Fixing it requires a fairly large rewrite of how sym2 manages its memory. I've done about half the work required, but I've basically run out of interest.
Quoted 4 lines
Hmm, maybe I don't understand what You mean by ARM, but it's i686 machine, not arm (actually it's opteron, but running i686 OS).
but if it's reallu harmless, I don't have that big problem with it. only the crashes really worry me :)
The warning is there on x86 to warn us that we're doing something that will cause problems on ARM.
Signature · 4 lines
Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step."
Re: crash unrelated to sym53c8xx
OK, now I understand, thanks for explaining...
unfortunately I'm now unable to reproduce the crash including backtrace, machine always hangs completely without any error reported :(
I've setup netconsole logging and I'll wait if it crashes and report something during normal usage.
I'll report if it happens.
thanks for Your time.
have a nice day
nik
Quoted 27 lines · Matthew Wilcox
Quoted 8 lines · Nikola Ciprich
Quoted 8 lines
There's a few reasons this warning hasn't been fixed yet:
- It only appears on module unload. Most people don't unload sym53c8xx.
- The problem that it's warning about is only a problem on ARM. There's not a huge userbase of ARM users with sym53c8xx cards.
- Fixing it requires a fairly large rewrite of how sym2 manages its memory. I've done about half the work required, but I've basically run out of interest.Quoted 4 lines
Hmm, maybe I don't understand what You mean by ARM, but it's i686 machine, not arm (actually it's opteron, but running i686 OS).
but if it's reallu harmless, I don't have that big problem with it. only the crashes really worry me :)The warning is there on x86 to warn us that we're doing something that will cause problems on ARM.
Signature · 8 lines
Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step." -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 14 lines
------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] -------------------------------------
Re: crash unrelated to sym53c8xx
Hi Matthew,
finally we caught the crash even with backtrace, here it goes:
[65468.555522] BUG: spinlock lockup on CPU#1, scsi_eh_9/5912, ec5625e0 [65468.555522] Pid: 5912, comm: scsi_eh_9 Tainted: G W 2.6.28lb.10 #1 [65468.555522] Call Trace: [65468.555522] [<c0333654>] _raw_spin_lock+0x144/0x180 [65468.555522] [<c046178c>] _spin_lock_irqsave+0x4c/0x60 [65468.555522] [<c031d111>] ? blk_put_request+0x21/0x50 [65468.555522] [<c031d111>] blk_put_request+0x21/0x50 [65468.555522] [<f827163c>] sg_finish_rem_req+0x9c/0xf0 [sg] [65468.555522] [<f8272589>] sg_rq_end_io+0x1a9/0x2b0 [sg] [65468.555522] [<c0461afc>] ? _spin_unlock_irqrestore+0x2c/0x50 [65468.555522] [<c031cda0>] end_that_request_last+0x50/0x230 [65468.555522] [<c046178c>] ? _spin_lock_irqsave+0x4c/0x60 [65468.555522] [<c031d05e>] blk_end_io+0x5e/0x90 [65468.555522] [<c031d0ea>] blk_end_request+0x1a/0x20 [65468.555522] [<f81a9305>] scsi_end_request+0x25/0x90 [scsi_mod] [65468.555522] [<f81aa072>] scsi_io_completion+0x92/0x420 [scsi_mod] [65468.555522] [<f81a973b>] ? scsi_device_unbusy+0x8b/0xb0 [scsi_mod] [65468.555522] [<f81a3538>] scsi_finish_command+0x98/0x100 [scsi_mod] [65468.555522] [<f81a790c>] ? scsi_eh_ready_devs+0x54c/0x720 [scsi_mod] [65468.555522] [<f81a6eba>] scsi_eh_flush_done_q+0x4a/0x100 [scsi_mod] [65468.555522] [<f81a815d>] scsi_error_handler+0x31d/0x520 [scsi_mod] [65468.555522] [<c0461afc>] ? _spin_unlock_irqrestore+0x2c/0x50 [65468.555522] [<c021eda9>] ? complete+0x49/0x60 [65468.555522] [<f81a7e40>] ? scsi_error_handler+0x0/0x520 [scsi_mod] [65468.555522] [<c023fda2>] kthread+0x42/0x70 [65468.555522] [<c023fd60>] ? kthread+0x0/0x70 [65468.555522] [<c02041c7>] kernel_thread_helper+0x7/0x10
Will it help?
BR
nik
x
Signature · 13 lines
------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] -------------------------------------
Re: crash unrelated to sym53c8xx
Quoted 31 lines · On Wed, 8 Apr 2009 10:30:02 +0200 Nikola Ciprich <[email protected]>…
Hi Matthew,
finally we caught the crash even with backtrace, here it goes:[65468.555522] BUG: spinlock lockup on CPU#1, scsi_eh_9/5912, ec5625e0 [65468.555522] Pid: 5912, comm: scsi_eh_9 Tainted: G W 2.6.28lb.10 #1 [65468.555522] Call Trace: [65468.555522] [<c0333654>] _raw_spin_lock+0x144/0x180 [65468.555522] [<c046178c>] _spin_lock_irqsave+0x4c/0x60 [65468.555522] [<c031d111>] ? blk_put_request+0x21/0x50 [65468.555522] [<c031d111>] blk_put_request+0x21/0x50 [65468.555522] [<f827163c>] sg_finish_rem_req+0x9c/0xf0 [sg] [65468.555522] [<f8272589>] sg_rq_end_io+0x1a9/0x2b0 [sg] [65468.555522] [<c0461afc>] ? _spin_unlock_irqrestore+0x2c/0x50 [65468.555522] [<c031cda0>] end_that_request_last+0x50/0x230 [65468.555522] [<c046178c>] ? _spin_lock_irqsave+0x4c/0x60 [65468.555522] [<c031d05e>] blk_end_io+0x5e/0x90 [65468.555522] [<c031d0ea>] blk_end_request+0x1a/0x20 [65468.555522] [<f81a9305>] scsi_end_request+0x25/0x90 [scsi_mod] [65468.555522] [<f81aa072>] scsi_io_completion+0x92/0x420 [scsi_mod] [65468.555522] [<f81a973b>] ? scsi_device_unbusy+0x8b/0xb0 [scsi_mod] [65468.555522] [<f81a3538>] scsi_finish_command+0x98/0x100 [scsi_mod] [65468.555522] [<f81a790c>] ? scsi_eh_ready_devs+0x54c/0x720 [scsi_mod] [65468.555522] [<f81a6eba>] scsi_eh_flush_done_q+0x4a/0x100 [scsi_mod] [65468.555522] [<f81a815d>] scsi_error_handler+0x31d/0x520 [scsi_mod] [65468.555522] [<c0461afc>] ? _spin_unlock_irqrestore+0x2c/0x50 [65468.555522] [<c021eda9>] ? complete+0x49/0x60 [65468.555522] [<f81a7e40>] ? scsi_error_handler+0x0/0x520 [scsi_mod] [65468.555522] [<c023fda2>] kthread+0x42/0x70 [65468.555522] [<c023fd60>] ? kthread+0x0/0x70 [65468.555522] [<c02041c7>] kernel_thread_helper+0x7/0x10
Quoted 1 line
Will it help?
This looks like a sg bug.
2.6.30-rc1 includes the following commit to fix this bug:
commit 015640edb1f346e0b2eda703587c4cd1c310ec1d
Author: FUJITA Tomonori <[email protected]> Date: Fri Apr 3 19:28:06 2009 +0900
[SCSI] sg: fix q->queue_lock on scsi_error_handler path
It's tagged 'Cc: [email protected]' so it will be merged into stable trees too.