Skip to contentSkip to conversation

New: hdparm -W1 /dev/sda causes oops and hard lockup

Bug 13572 [email protected] 2 participants 7 messages Jun 18, 2009 – Jun 29, 2009 SCSI

lore
Share & tools
lore
#0 lore raw

http://bugzilla.kernel.org/show_bug.cgi?id=13572

           Summary: hdparm -W1 /dev/sda causes oops and hard lockup
           Product: IO/Storage
           Version: 2.5
    Kernel Version: 2.6.30, 2.6.30-rc6
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: SCSI
        AssignedTo: [email protected]
        ReportedBy: [email protected]
        Regression: No

Worked in 2.6.29.

[  257.940560] ata1.00: configured for UDMA/100 
[  257.940694] ata1: EH complete 
[  257.940799] BUG: unable to handle kernel NULL pointer dereference at
00000228 
[  257.940965] IP: [<c031dae3>] scsi_device_get+0x3/0x60 
[  257.941116] *pde = 00000000  
[  257.941224] Oops: 0000 [#2] SMP 
[  257.941380] last sysfs file:
/sys/devices/LNXSYSTM:00/device:00/PNP0A03:00/device:0a/PNP0C09:00/ACPI0003:00/power_supply/ACAD/online 
[  257.941470] Modules linked in: netconsole ipt_ECN iptable_mangle xt_recent

xt_tcpudp ipt_REJECT ipt_LOG iptable_filter ip_tables x_tables dm_mod usb_storage snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm_oss snd_mixer_oss snd_pcm uhci_hcd ehci_hcd snd_timer snd i2c_i801 rsrc_nonstatic e100 psmouse mousedev soundcore rtc_cmos snd_page_alloc i2c_core usbcore pcmcia_core mii rtc_core rtc_lib evdev intel_agp 8250_pnp(-) intel_rng rng_core agpgart sonypi [last unloaded: parport]

[  257.943760]  
[  257.943825] Pid: 937, comm: scsi_eh_0 Tainted: G      D   
(2.6.30+vc+ipmi+ow #2) PCG-R505TL(UC)       
[  257.943905] EIP: 0060:[<c031dae3>] EFLAGS: 00010007 CPU: 0 
[  257.943982] EIP is at scsi_device_get+0x3/0x60 
[  257.944051] EAX: fffffff8 EBX: fffffff8 ECX: 00008484 EDX: 00000246 
[  257.944123] ESI: fffffff8 EDI: cb9ac400 EBP: 00000000 ESP: cb9d4f68 
[  257.944192]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 
[  257.944262] Process scsi_eh_0 (pid: 937, ti=cb9d4000 task=cb9cd860
task.ti=cb9d4000) 
[  257.944337] Stack: 
[  257.944394]  cb9ac400 c031dcaa 00000246 00000293 cb9ac400 cb9d4fc0 00000000
c0322c93 
[  257.944415]  00000000 00000086 cb82cfb0 cb824d20 00000001 cb9cdae8 cb9d4fc0
00000000 
[  257.944415]  00000003 cb824d2c cb9ac444 cb9ac454 c01230d0
[  257.944415] Call Trace: 
[  257.944415]  [<c031dcaa>] ? __scsi_iterate_devices+0x3a/0x70 
[  257.944415]  [<c0322c93>] ? scsi_error_handler+0xe3/0x5a0 
[  257.944415]  [<c01230d0>] ? complete+0x40/0x60 
[  257.944415]  [<c0322bb0>] ? scsi_error_handler+0x0/0x5a0 
[  257.944415] Code: 24 24 c4 69 90 89 d8 e8 81 c1 eb 0d 90  90  90 c3 30 00 0c
ff c3 74 
[  257.944415] EIP: [<c031dae3>]  SS:ESP 0068:cb9d4f68 
[  257.944415] ---[ end trace 962c3e68e35a750c ]--- 

OOPS was collected via netconsole.

Signature · 3 lines
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#1 lore raw

[Bug 13572] hdparm -W1 /dev/sda causes oops and hard lockup

http://bugzilla.kernel.org/show_bug.cgi?id=13572

Vladimir Prodan <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
Regression|No |Yes

Signature · 3 lines
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
maintainer
#2 lore raw
In reply to [email protected]
Quoted 18 lines · On Thu, 2009-06-18 at 16:05 +0000, [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=13572

           Summary: hdparm -W1 /dev/sda causes oops and hard lockup
           Product: IO/Storage
           Version: 2.5
    Kernel Version: 2.6.30, 2.6.30-rc6
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: SCSI
        AssignedTo: [email protected]
        ReportedBy: [email protected]
        Regression: No
Quoted 4 lines

Worked in 2.6.29.

[  257.940560] ata1.00: configured for UDMA/100 
[  257.940694] ata1: EH complete 

This is printed just before the libata-eh:ata_scsi_error() exits

Quoted 2 lines
[  257.940799] BUG: unable to handle kernel NULL pointer dereference at

00000228

This is 0 offset by sdev_state

Quoted 1 line
[  257.940965] IP: [<c031dae3>] scsi_device_get+0x3/0x60 

So clearly the SCSI device going into scsi_device_get was a NULL pointer.

Quoted 24 lines
[  257.941116] *pde = 00000000  
[  257.941224] Oops: 0000 [#2] SMP 
[  257.941380] last sysfs file:
/sys/devices/LNXSYSTM:00/device:00/PNP0A03:00/device:0a/PNP0C09:00/ACPI0003:00/power_supply/ACAD/online 
[  257.941470] Modules linked in: netconsole ipt_ECN iptable_mangle xt_recent

xt_tcpudp ipt_REJECT ipt_LOG iptable_filter ip_tables x_tables dm_mod usb_storage snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm_oss snd_mixer_oss snd_pcm uhci_hcd ehci_hcd snd_timer snd i2c_i801 rsrc_nonstatic e100 psmouse mousedev soundcore rtc_cmos snd_page_alloc i2c_core usbcore pcmcia_core mii rtc_core rtc_lib evdev intel_agp 8250_pnp(-) intel_rng rng_core agpgart sonypi [last unloaded: parport]

[  257.943760]  
[  257.943825] Pid: 937, comm: scsi_eh_0 Tainted: G      D   
(2.6.30+vc+ipmi+ow #2) PCG-R505TL(UC)       
[  257.943905] EIP: 0060:[<c031dae3>] EFLAGS: 00010007 CPU: 0 
[  257.943982] EIP is at scsi_device_get+0x3/0x60 
[  257.944051] EAX: fffffff8 EBX: fffffff8 ECX: 00008484 EDX: 00000246 
[  257.944123] ESI: fffffff8 EDI: cb9ac400 EBP: 00000000 ESP: cb9d4f68 
[  257.944192]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 
[  257.944262] Process scsi_eh_0 (pid: 937, ti=cb9d4000 task=cb9cd860
task.ti=cb9d4000) 
[  257.944337] Stack: 
[  257.944394]  cb9ac400 c031dcaa 00000246 00000293 cb9ac400 cb9d4fc0 00000000

c0322c93

Quoted 8 lines
[  257.944415]  00000000 00000086 cb82cfb0 cb824d20 00000001 cb9cdae8 cb9d4fc0
00000000 
[  257.944415]  00000003 cb824d2c cb9ac444 cb9ac454 c01230d0
[  257.944415] Call Trace: 
[  257.944415]  [<c031dcaa>] ? __scsi_iterate_devices+0x3a/0x70 
[  257.944415]  [<c0322c93>] ? scsi_error_handler+0xe3/0x5a0 
[  257.944415]  [<c01230d0>] ? complete+0x40/0x60 
[  257.944415]  [<c0322bb0>] ? scsi_error_handler+0x0/0x5a0 

Unfortunately, as seems to be the wont on x86 these days, this trace is complete rubbish: if we theorise that the __scsi_iterate_devices is correct, then we're at shost_for_each_device().

Thus, I think we must be in scsi_restart_operations(), which is the only thing that happens after the strategy handler exits.

So, it looks a bit like corruption in the host device list ... or a scsi_device_get() is no longer pinning a device in that list. I'm at a loss to find any commit between 2.6.29 and now that would cause this, though ... could you try a bisection to find the offending commit?

Thanks,

James

#3 lore raw

[Bug 13572] hdparm -W1 /dev/sda causes oops and hard lockup

In reply to [email protected]

http://bugzilla.kernel.org/show_bug.cgi?id=13572

--- Comment #1 from Anonymous Emailer <[email protected]> 2009-06-18 23:05:54 ---
Reply-To: [email protected]

Quoted 18 lines · On Thu, 2009-06-18 at 16:05 +0000, [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=13572

           Summary: hdparm -W1 /dev/sda causes oops and hard lockup
           Product: IO/Storage
           Version: 2.5
    Kernel Version: 2.6.30, 2.6.30-rc6
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: SCSI
        AssignedTo: [email protected]
        ReportedBy: [email protected]
        Regression: No
Quoted 4 lines

Worked in 2.6.29.

[  257.940560] ata1.00: configured for UDMA/100 
[  257.940694] ata1: EH complete 

This is printed just before the libata-eh:ata_scsi_error() exits

Quoted 2 lines
[  257.940799] BUG: unable to handle kernel NULL pointer dereference at

00000228

This is 0 offset by sdev_state

Quoted 1 line
[  257.940965] IP: [<c031dae3>] scsi_device_get+0x3/0x60 

So clearly the SCSI device going into scsi_device_get was a NULL pointer.

Quoted 24 lines
[  257.941116] *pde = 00000000  
[  257.941224] Oops: 0000 [#2] SMP 
[  257.941380] last sysfs file:
/sys/devices/LNXSYSTM:00/device:00/PNP0A03:00/device:0a/PNP0C09:00/ACPI0003:00/power_supply/ACAD/online 
[  257.941470] Modules linked in: netconsole ipt_ECN iptable_mangle xt_recent

xt_tcpudp ipt_REJECT ipt_LOG iptable_filter ip_tables x_tables dm_mod usb_storage snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm_oss snd_mixer_oss snd_pcm uhci_hcd ehci_hcd snd_timer snd i2c_i801 rsrc_nonstatic e100 psmouse mousedev soundcore rtc_cmos snd_page_alloc i2c_core usbcore pcmcia_core mii rtc_core rtc_lib evdev intel_agp 8250_pnp(-) intel_rng rng_core agpgart sonypi [last unloaded: parport]

[  257.943760]  
[  257.943825] Pid: 937, comm: scsi_eh_0 Tainted: G      D   
(2.6.30+vc+ipmi+ow #2) PCG-R505TL(UC)       
[  257.943905] EIP: 0060:[<c031dae3>] EFLAGS: 00010007 CPU: 0 
[  257.943982] EIP is at scsi_device_get+0x3/0x60 
[  257.944051] EAX: fffffff8 EBX: fffffff8 ECX: 00008484 EDX: 00000246 
[  257.944123] ESI: fffffff8 EDI: cb9ac400 EBP: 00000000 ESP: cb9d4f68 
[  257.944192]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 
[  257.944262] Process scsi_eh_0 (pid: 937, ti=cb9d4000 task=cb9cd860
task.ti=cb9d4000) 
[  257.944337] Stack: 
[  257.944394]  cb9ac400 c031dcaa 00000246 00000293 cb9ac400 cb9d4fc0 00000000

c0322c93

Quoted 8 lines
[  257.944415]  00000000 00000086 cb82cfb0 cb824d20 00000001 cb9cdae8 cb9d4fc0
00000000 
[  257.944415]  00000003 cb824d2c cb9ac444 cb9ac454 c01230d0
[  257.944415] Call Trace: 
[  257.944415]  [<c031dcaa>] ? __scsi_iterate_devices+0x3a/0x70 
[  257.944415]  [<c0322c93>] ? scsi_error_handler+0xe3/0x5a0 
[  257.944415]  [<c01230d0>] ? complete+0x40/0x60 
[  257.944415]  [<c0322bb0>] ? scsi_error_handler+0x0/0x5a0 

Unfortunately, as seems to be the wont on x86 these days, this trace is complete rubbish: if we theorise that the __scsi_iterate_devices is correct, then we're at shost_for_each_device().

Thus, I think we must be in scsi_restart_operations(), which is the only thing that happens after the strategy handler exits.

So, it looks a bit like corruption in the host device list ... or a scsi_device_get() is no longer pinning a device in that list. I'm at a loss to find any commit between 2.6.29 and now that would cause this, though ... could you try a bisection to find the offending commit?

Thanks,

James

Signature · 3 lines
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#4 lore raw

[Bug 13572] hdparm -W1 /dev/sda causes oops and hard lockup

In reply to [email protected]

http://bugzilla.kernel.org/show_bug.cgi?id=13572

Vladimir Prodan <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID

--- Comment #2 from Vladimir Prodan <[email protected]> 2009-06-19 14:42:49 --- James, thank you! You're right, it seems it was reproducible corruption of device list caused by some IPMI code. Sorry, I am using IPMI patches from
http://openipmi.sf.net/ since 2004 (2.6.6) and have no problems with them until now. Unfinished git bisection shows that problem is around here:

bad cced7d0 2009-05-19 21:51:07 +0400 [Corey Minyard] serial: Add a polled interface to the 8250 driver

    9483290 2009-05-19 21:51:06 +0400 [Corey Minyard] serial: Add support for
doing a console using the poll interface
    5bdfc86 2009-05-19 21:51:06 +0400 [Corey Minyard] serial: Add a poll
interface to the serial core
ok  6acac9a 2009-05-19 21:51:05 +0400 [Corey Minyard] serial: Add circular
buffer helpers
    62ed3da 2009-05-19 21:51:05 +0400 [Corey Minyard] serial: Add a function to
abstract the tty code from the drivers
    6657022 2009-05-19 21:51:10 +0400 [Corey Minyard] ipmi_emu

ok af85f59 2009-05-19 21:51:10 +0400 [Corey Minyard] ipmi_si_max_busy-fixed

I'll try find the cause and tell Corey about it.

Signature · 3 lines
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#5 lore raw

[Bug 13572] hdparm -W1 /dev/sda causes oops and hard lockup

In reply to [email protected]

http://bugzilla.kernel.org/show_bug.cgi?id=13572

Vladimir Prodan <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |CLOSED

Signature · 3 lines
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
#6 lore raw

[Bug 13572] hdparm -W1 /dev/sda causes oops and hard lockup

In reply to [email protected]

http://bugzilla.kernel.org/show_bug.cgi?id=13572

Andrew Morton <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

                 CC|                            |[email protected]
         AssignedTo|[email protected]  |[email protected]

--- Comment #3 from Andrew Morton <[email protected]> 2009-06-29 21:37:03 --- Assigning this to Corey.

Signature · 3 lines
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.