USB/SCSI slab corruption.

DISCUSSION Dave Jones 4 participants 4 messages Jan 12, 2006 – Jan 13, 2006 SCSI

lore

Share & tools

lore

Dave Jones

author

2006-01-12 23:36 UTC #0 lore raw

A nasty 2.6.15 report from one of our Fedora users.

Dave

Quoted 70 lines

then when it got to map my USB drive:
SCSI device sdb: 321672960 512-byte hdwr sectors (164697 MB) slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or memory outside object was overwritten

 [<c0153593>] cache_free_debugcheck+0x1ab/0x22f     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<c01540e2>] kmem_cache_free+0x2a/0x69     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<f8862c8b>] scsi_io_completion+0x207/0x4d2 [scsi_mod]     [<f885dcad>] 
scsi_finish_command+0x82/0xd0 [scsi_mod] 
 [<f885db8e>] scsi_softirq+0xc0/0x137 [scsi_mod]     [<c012a152>] 
__do_softirq+0x72/0xdc 
 [<c01063d3>] do_softirq+0x4b/0x4f 
 ======================= 
 [<c01062b5>] do_IRQ+0x55/0x86     [<c0104aca>] common_interrupt+0x1a/0x20 
 [<c010201a>] default_idle+0x0/0x55     [<c0102046>] default_idle+0x2c/0x55 
 [<c01020ef>] cpu_idle+0x6c/0xa7 
f79a41d0: redzone 1: 0x170fc2a5, redzone 2: 0xc014d2d1. 
slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or 
memory outside object was overwritten 
 [<c0153593>] cache_free_debugcheck+0x1ab/0x22f     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<c01540e2>] kmem_cache_free+0x2a/0x69     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<f8862c8b>] scsi_io_completion+0x207/0x4d2 [scsi_mod]     [<f885dcad>] 
scsi_finish_command+0x82/0xd0 [scsi_mod] 
 [<f885db8e>] scsi_softirq+0xc0/0x137 [scsi_mod]     [<c012a152>] 
__do_softirq+0x72/0xdc 
 [<c01063d3>] do_softirq+0x4b/0x4f 
 ======================= 
 [<c01062b5>] do_IRQ+0x55/0x86     [<c0104aca>] common_interrupt+0x1a/0x20 
 [<c010201a>] default_idle+0x0/0x55     [<c0102046>] default_idle+0x2c/0x55 
 [<c01020ef>] cpu_idle+0x6c/0xa7 
f79a41d0: redzone 1: 0x170fc2a5, redzone 2: 0xc014d2d1.

sdb: got wrong page
sdb: assuming drive cache: write through
SCSI device sdb: 321672960 512-byte hdwr sectors (164697 MB) slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or memory outside object was overwritten

 [<c0153593>] cache_free_debugcheck+0x1ab/0x22f     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<c01540e2>] kmem_cache_free+0x2a/0x69     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<f8862c8b>] scsi_io_completion+0x207/0x4d2 [scsi_mod]     [<f885dcad>] 
scsi_finish_command+0x82/0xd0 [scsi_mod] 
 [<f885db8e>] scsi_softirq+0xc0/0x137 [scsi_mod]     [<c012a152>] 
__do_softirq+0x72/0xdc 
 [<c01063d3>] do_softirq+0x4b/0x4f 
 ======================= 
 [<c01062b5>] do_IRQ+0x55/0x86     [<c016a311>] filp_close+0x3e/0x62 
 [<c0104aca>] common_interrupt+0x1a/0x20 
f79a41d0: redzone 1: 0x170fc2a5, redzone 2: 0xc014d2d1. 
slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or 
memory outside object was overwritten 
 [<c0153593>] cache_free_debugcheck+0x1ab/0x22f     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<c01540e2>] kmem_cache_free+0x2a/0x69     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<f8862c8b>] scsi_io_completion+0x207/0x4d2 [scsi_mod]     [<f885dcad>] 
scsi_finish_command+0x82/0xd0 [scsi_mod] 
 [<f885db8e>] scsi_softirq+0xc0/0x137 [scsi_mod]     [<c012a152>] 
__do_softirq+0x72/0xdc 
 [<c01063d3>] do_softirq+0x4b/0x4f 
 ======================= 
 [<c01062b5>] do_IRQ+0x55/0x86     [<c0104aca>] common_interrupt+0x1a/0x20 
f79a41d0: redzone 1: 0x170fc2a5, redzone 2: 0xc014d2d1.

sdb: got wrong page
sdb: assuming drive cache: write through
sdb: sdb1 sdb2 sdb4 < sdb5 sdb6 sdb7 sdb8 sdb9 >
sd 3:0:0:0: Attached scsi disk sdb

Stefan Richter

2006-01-13 00:23 UTC #1 lore raw

Quoted 18 lines · Dave Jones

A nasty 2.6.15 report from one of our Fedora users.
Dave
Quoted 14 lines
then when it got to map my USB drive:
SCSI device sdb: 321672960 512-byte hdwr sectors (164697 MB) slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or memory outside object was overwritten
 [<c0153593>] cache_free_debugcheck+0x1ab/0x22f     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<c01540e2>] kmem_cache_free+0x2a/0x69     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<f8862c8b>] scsi_io_completion+0x207/0x4d2 [scsi_mod]     [<f885dcad>] 
scsi_finish_command+0x82/0xd0 [scsi_mod] 
 [<f885db8e>] scsi_softirq+0xc0/0x137 [scsi_mod]     [<c012a152>] 
__do_softirq+0x72/0xdc 
 [<c01063d3>] do_softirq+0x4b/0x4f 
 ======================= 

[...]

This looks very similar to this report about an SBP-2 HDD under 2.6.14-rc4: http://marc.theaimsgroup.com/?t=112931959700002

Signature · 14 lines

Stefan Richter
-=====-=-==- ---= -==-=
http://arcgraph.de/sr/


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
[email protected]
To unsubscribe, use the last form field at:
https://lists.sourceforge.net/lists/listinfo/linux-usb-devel

James Bottomley

2006-01-13 01:30 UTC #3 lore raw

In reply to Dave Jones

Quoted 8 lines · Dave Jones

Quoted 8 lines
SCSI device sdb: 321672960 512-byte hdwr sectors (164697 MB) slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or memory outside object was overwritten
 [<c0153593>] cache_free_debugcheck+0x1ab/0x22f     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<c01540e2>] kmem_cache_free+0x2a/0x69     [<c014d3c2>] 
mempool_free+0x3a/0x73 
 [<f8862c8b>] scsi_io_completion+0x207/0x4d2 [scsi_mod]     [<f885dcad>] 

This is a classic symptom of done() being called twice on the same command. Best guess would be a driver issue.

James

F. Poncin

2006-01-13 09:39 UTC #2 lore raw

In reply to Stefan Richter

Quoted 4 lines · Stefan Richter

Quoted 2 lines · Dave Jones
A nasty 2.6.15 report from one of our Fedora users.

Quoted 8 lines

Quoted 4 lines
Dave
Quoted 1 line
then when it got to map my USB drive: > SCSI device sdb:
321672960 512-byte hdwr sectors (164697 MB) > slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or > memory outside object was overwritten > [<c0153593>] cache_free_debugcheck+0x1ab/0x22f [<c014d3c2>] > mempool_free+0x3a/0x73 > [<c01540e2>] kmem_cache_free+0x2a/0x69 [<c014d3c2>] > mempool_free+0x3a/0x73 > [<f8862c8b>] scsi_io_completion+0x207/0x4d2 [scsi_mod] [<f885dcad>] > scsi_finish_command+0x82/0xd0 [scsi_mod] > [<f885db8e>] scsi_softirq+0xc0/0x137 [scsi_mod] [<c012a152>] > __do_softirq+0x72/0xdc > [<c01063d3>] do_softirq+0x4b/0x4f > =======================
[...]
This looks very similar to this report about an SBP-2 HDD under 2.6.14-rc4: http://marc.theaimsgroup.com/?t=112931959700002

As discussed with Andrew Morton yesterday, this seems to be fixed on my side (latest kernel)

extract from dmesg:

ieee1394: Node added: ID:BUS[0-00:1023] GUID[000bc20300011c51]
ieee1394: Host added: ID:BUS[0-01:1023] GUID[0000d10080967536] scsi2 : SBP-2 IEEE-1394
ieee1394: sbp2: Logged into SBP-2 device
ieee1394: Node 0-00:1023: Max speed [S400] - Max payload [2048]
Vendor: Initio Model: ST3400832A Rev: 4.07
Type: Direct-Access ANSI SCSI revision: 00
SCSI device sdc: 781422768 512-byte hdwr sectors (400088 MB)
sdc: Write Protect is off
sdc: Mode Sense: 00 13 00 00
sdc: got wrong page
sdc: assuming drive cache: write through
SCSI device sdc: 781422768 512-byte hdwr sectors (400088 MB)
sdc: Write Protect is off
sdc: Mode Sense: 00 13 00 00
sdc: got wrong page
sdc: assuming drive cache: write through
sdc: sdc1
sd 2:0:0:0: Attached scsi disk sdc

Quoted 77 lines · F. Poncin

Hello,
I found this in several recent boot logs.
Summary: slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or memory outside object was overwritten Kernel version: 2.6.14-rc4-g9149ccfa
Steps to reproduce: on boot
Hardware: Dell 8300 + External USB disk enclosures
I'm not subscribed to the list. Please Cc:
Additional info / test on request.
extract from dmesg:
scsi2 : SCSI emulation for IEEE-1394 SBP-2 Devices
ieee1394: sbp2: Logged into SBP-2 device
ieee1394: Node 0-00:1023: Max speed [S400] - Max payload [2048]
Vendor: Initio Model: ST3400832A Rev: 4.07
Type: Direct-Access ANSI SCSI revision: 00
SCSI device sdc: 781422768 512-byte hdwr sectors (400088 MB)
slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or memory outside object was overwritten
[<c014a43b>] cache_free_debugcheck+0x15e/0x215
[<c0144c73>] mempool_free+0x6c/0x73
[<c014ae71>] kmem_cache_free+0x25/0x59
[<c0144c73>] mempool_free+0x6c/0x73
[<f8891d92>] scsi_io_completion+0x1fd/0x4ac [scsi_mod]
[<f8826d3f>] sd_rw_intr+0x155/0x30e [sd_mod]
[<c0148e2b>] poison_obj+0x1c/0x38
[<c032ca6e>] _spin_lock+0x1c/0x75
[<f888cc33>] scsi_finish_command+0x82/0xb5 [scsi_mod]
[<c032ca6e>] _spin_lock+0x1c/0x75
[<f888cb0a>] scsi_softirq+0xc0/0x141 [scsi_mod]
[<c01252e1>] tasklet_action+0x59/0xbe
[<c0124f79>] __do_softirq+0x69/0xd5
[<c0104ceb>] do_softirq+0x57/0x5b
=======================
[<c01250c0>] irq_exit+0x42/0x44
[<c0104bbc>] do_IRQ+0x5c/0x8e
[<c032b277>] schedule+0x627/0xcd8
[<c01036a6>] common_interrupt+0x1a/0x20
[<c0100e91>] mwait_idle+0x25/0x4a
[<c023d243>] acpi_processor_idle+0x0/0x29e
[<c023d344>] acpi_processor_idle+0x101/0x29e
[<c023d243>] acpi_processor_idle+0x0/0x29e
[<c0100ceb>] cpu_idle+0x45/0x7d
[<c03f088a>] start_kernel+0x18c/0x1cb
[<c03f030b>] unknown_bootoption+0x0/0x1b0
c233b7a8: redzone 1: 0x170fc2a5, redzone 2: 0xc0144b47.
sdc: asking for cache data failed
sdc: assuming drive cache: write through
SCSI device sdc: 781422768 512-byte hdwr sectors (400088 MB)
slab error in cache_free_debugcheck(): cache `sgpool-8': double free, or memory outside object was overwritten
[<c014a43b>] cache_free_debugcheck+0x15e/0x215
[<c0144c73>] mempool_free+0x6c/0x73
[<c014ae71>] kmem_cache_free+0x25/0x59
[<c0144c73>] mempool_free+0x6c/0x73
[<f8891d92>] scsi_io_completion+0x1fd/0x4ac [scsi_mod]
[<f8826d3f>] sd_rw_intr+0x155/0x30e [sd_mod]
[<c0148e2b>] poison_obj+0x1c/0x38
[<c032ca6e>] _spin_lock+0x1c/0x75
[<f888cc33>] scsi_finish_command+0x82/0xb5 [scsi_mod]
[<c032ca6e>] _spin_lock+0x1c/0x75
[<f888cb0a>] scsi_softirq+0xc0/0x141 [scsi_mod]
[<c01252e1>] tasklet_action+0x59/0xbe
[<c0124f79>] __do_softirq+0x69/0xd5
[<c0104ceb>] do_softirq+0x57/0x5b
=======================
[<c01250c0>] irq_exit+0x42/0x44
[<c0104bbc>] do_IRQ+0x5c/0x8e
[<c01036a6>] common_interrupt+0x1a/0x20
[<c0100e91>] mwait_idle+0x25/0x4a
[<c023d243>] acpi_processor_idle+0x0/0x29e
[<c023d344>] acpi_processor_idle+0x101/0x29e
[<c023d243>] acpi_processor_idle+0x0/0x29e
[<c0100ceb>] cpu_idle+0x45/0x7d
[<c03f088a>] start_kernel+0x18c/0x1cb
[<c03f030b>] unknown_bootoption+0x0/0x1b0
c233b7a8: redzone 1: 0x170fc2a5, redzone 2: 0xc0144b47.

Quoted 7 lines

sdc: asking for cache data failed
sdc: assuming drive cache: write through
sdc:<6>FDC 0 is a post-1991 82077
Signature · 1 line
Frédéric Poncin

-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html