Skip to contentSkip to conversation

New: sym53c8xx: weird behavior with udev

Bugme-new Andrew Morton 4 participants 14 messages Apr 1, 2008 – Apr 2, 2008 1 message not in the archive SCSI

lore
Share & tools
lore

Andrew Morton

author
#1 lore raw
In reply to a message not in the archive

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 48 lines · On Tue, 1 Apr 2008 01:01:22 -0700 (PDT) [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id374

           Summary: sym53c8xx: weird behavior with udev
           Product: Drivers
           Version: 2.5
     KernelVersion: 2.6.24.4
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
        AssignedTo: [email protected]
        ReportedBy: [email protected]

Latest working kernel version: 2.6.22.9
Earliest failing kernel version: 2.6.23
Distribution: Gentoo
Hardware Environment: sparc64 (Sun Blade 100)
Software Environment:
Problem Description:

Since kernel 2.6.23, I have been having problems getting the sungem network device working on one of my two Blade 100s, see bug #10273.

After debugging this, I found that this seems to be somehow related to sym53c8xx and udev.

If I allow udev to load sym53c8xx during boot, the attached disks work fine but the network does not work at all. While the device is up and mii-diag says there is link beat, no packets can be sent or received, and attempts to use the network result in "network unreachable" errors.

If I blacklist sym53c8xx in /etc/modprobe.d/blacklist, let the machine boot normally and then manually load sym53c8xx after everything is settled, both the scsi disks and the network appear to work fine.

None of this happened in kernel versions 2.6.22.9 and earlier, 2.6.23 was the first to start showing this behavior, and it still persists in 2.6.24.

I've tried playing with the option CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE as that is what seems to have changed between kernel versions 2.6.22 and 2.6.23. The recommended value for my machine is 0, but I have tried the other possible values without result. I have also tried toggling CONFIG_SCSI_SYM53C8XX_MMIO, also without result.

Quoted 3 lines

Steps to reproduce:

Let udev load sym53c8xx in kernel 2.6.23 or newer.

urgh. Perhaps it's related to platform IRQ routing or something.

I'd suggest that the next step would be to send us the `dmesg -s 1000000' output for both good and bad kernels. A comparison might show where things went bad.

#2 lore raw
Quoted 8 lines · On Tue, 1 Apr 2008 01:15:18 -0700 Andrew Morton <[email protected]>…

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

urgh. Perhaps it's related to platform IRQ routing or something.

I'd suggest that the next step would be to send us the `dmesg -s 1000000' output for both good and bad kernels. A comparison might show where things went bad.

Alright, here goes. Attached three files.

dmesg-2.6.22.9-good.txt is the dmesg output from 2.6.22.9 where everything works as it should. dmesg-2.6.24.4-bad.txt is the dmesg output from 2.6.24.4 with sym53cxx loaded by udev and the network not responding. dmesg-2.6.24.4-good.txt is the dmesg output from 2.6.24.4 with sym53cxx blacklisted and the network working fine.

Signature · 1 line
Jos van der Ende <[email protected]>

PROMLIB: Sun IEEE Boot Prom 'OBP 4.17.1 2005/04/11 14:31'
PROMLIB: Root node compatible:
Linux version 2.6.22.9 (angel@phoenix) (gcc version 4.1.2 (Gentoo 4.1.2 p1.0.2)) #2 Mon Dec 17 22:46:30 CET 2007
ARCH: SUN4U
Ethernet address: 00:03:ba:08:61:7c
PROM: Built device tree with 60215 bytes of memory.
On node 0 totalpages: 130613
Normal zone: 1567 pages used for memmap
Normal zone: 0 pages reserved
Normal zone: 129046 pages, LIFO batch:15
Built 1 zonelists. Total pages: 129046
Kernel command line: root=/dev/hda4
PID hash table entries: 4096 (order: 12, 32768 bytes)
clocksource: mult[b40001] shift[16]
clockevent: mult[16c16bf] shift[32]
Console: colour dummy device 80x25
Dentry cache hash table entries: 131072 (order: 7, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 6, 524288 bytes)
Memory: 1028152k available (1816k kernel code, 568k data, 96k init) [fffff80000000000,000000006ff0c000]
SLUB: Genslabs=23, HWalign=32, Order=0-2, MinObjects=8, CPUs=1, Nodes=1
Calibrating delay using timer specific routine.. 11.12 BogoMIPS (lpj=55625)
Mount-cache hash table entries: 512
NET: Registered protocol family 16
PCI: Probing for controllers.
/pci@1f,0: SABRE PCI Bus Module
/pci@1f,0: PCI IO[1fe02000000] MEM[1ff00000000]
PCI: Scanning PBM /pci@1f,0
PCI quirk: region 0800-083f claimed by ali7101 ACPI
PCI quirk: region 0600-061f claimed by ali7101 SMB
isa0: [dma -> (floppy) (parallel)] [power] [serial] [serial]
ebus0: [flashprom] [eeprom] [idprom]
power: Control reg at 1fe02000800 ... powerd running.
/pci@1f,0/ebus@c/eeprom@1,0: Clock regs at 000001fff1000000
Time: hbtick clocksource has been installed.
Switched to high resolution mode on CPU 0
NET: Registered protocol family 2
IP route cache hash table entries: 8192 (order: 3, 65536 bytes)
TCP established hash table entries: 32768 (order: 6, 524288 bytes)
TCP bind hash table entries: 32768 (order: 5, 262144 bytes)
TCP: Hash tables configured (established 32768 bind 32768)
TCP reno registered
io scheduler noop registered (default)
Activating ISA DMA hang workarounds.
f007d648: ttyS0 at MMIO 0x1fe020003f8 (irq = 9) is a 16550A
Console: ttyS0 (SU)
f007f1a0: ttyS1 at MMIO 0x1fe020002e8 (irq = 9) is a 16550A
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
ALI15X3: IDE controller at PCI slot 0000:00:0d.0
PCI: Enabling device: (0000:00:0d.0), cmd 5
ALI15X3: chipset revision 195
ALI15X3: 100% native mode on irq 15

    ide0: BM-DMA at 0x1fe02000a20-0x1fe02000a27, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0x1fe02000a28-0x1fe02000a2f, BIOS settings: hdc:pio, hdd:pio

Probing IDE interface ide0...
hda: ST38410A, ATA DISK drive
hdb: HL-DT-ST RW/DVD GCC-4480B, ATAPI CD/DVD-ROM drive
hda: selected mode 0x44
hdb: drive side 80-wire cable detection failed, limiting max speed to UDMA33
hdb: selected mode 0x42
ide0 at 0x1fe02000a00-0x1fe02000a07,0x1fe02000a1a on irq 15
Probing IDE interface ide1...
hda: max request size: 128KiB
hda: 16841664 sectors (8622 MB) w/512KiB Cache, CHS=16708/16/63, UDMA(66)
hda: cache flushes not supported
hda: hda1 hda2 hda3 hda4 hda5
mice: PS/2 mouse device common for all mice
TCP cubic registered
VFS: Mounted root (ext2 filesystem) readonly.
NET: Registered protocol family 1
PCI: Enabling device: (0000:00:08.0), cmd 3
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
hdb: ATAPI 48X DVD-ROM CD-R/RW drive, 2048kB Cache, UDMA(33)
Uniform CD-ROM driver Revision: 3.20
usbcore: registered new device driver usb
ohci_hcd: 2006 August 04 USB 1.1 'Open' Host Controller (OHCI) Driver
SCSI subsystem initialized
sungem.c:v0.98 8/24/03 David S. Miller ([email protected])
PHY ID: 437421, addr: 1
eth0: Sun GEM (PCI) 10/100/1000BaseT Ethernet 00:03:ba:08:61:7c
eth0: Found Generic MII PHY
ohci1394: fw-host0: OHCI-1394 1.0 (PCI): IRQ=[12] MMIO=[1ff00420000-1ff004207ff] Max Packet=[2048] IR/IT contexts=[4/4]
PCI: Enabling device: (0000:00:0c.3), cmd 2
ohci_hcd 0000:00:0c.3: OHCI Host Controller
ohci_hcd 0000:00:0c.3: new USB bus registered, assigned bus number 1
ohci_hcd 0000:00:0c.3: irq 13, io mem 0x1ff02000000
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 4 ports detected
PCI: Enabling device: (0000:01:02.0), cmd 147
sym0: <875> rev 0x14 at pci 0000:01:02.0 irq 16
sym0: No NVRAM, ID 7, Fast-20, SE, parity checking
sym0: SCSI BUS has been reset.
scsi0 : sym-2.2.3
ieee1394: Host added: ID:BUS[0-00:1023] GUID[0003bafffe08617c]
ieee1394: Node added: ID:BUS[0-01:1023] GUID[0003bafffe0859fb]
eth1394: eth1: IPv4 over IEEE 1394 (fw-host0)
udev: renamed network interface eth1 to fw0
eth0: Link is up at 100 Mbps, full-duplex.
scsi 0:0:0:0: Direct-Access HITACHI DK32EJ72NSUN72G PQ0B PQ: 0 ANSI: 3
target0:0:0: tagged command queuing enabled, command queue depth 16.
target0:0:0: Beginning Domain Validation
target0:0:0: asynchronous
target0:0:0: wide asynchronous
target0:0:0: FAST-20 WIDE SCSI 40.0 MB/s ST (50 ns, offset 16)
target0:0:0: Domain Validation skipping write tests
target0:0:0: Ending Domain Validation
scsi 0:0:5:0: Sequential-Access EXABYTE EXB-89008E030203 V41e PQ: 0 ANSI: 2
target0:0:5: Beginning Domain Validation
target0:0:5: asynchronous
target0:0:5: wide asynchronous
target0:0:5: FAST-10 WIDE SCSI 20.0 MB/s ST (100 ns, offset 15)
target0:0:5: Domain Validation skipping write tests
target0:0:5: Ending Domain Validation
PCI: Enabling device: (0000:01:02.1), cmd 147
sym1: <875> rev 0x14 at pci 0000:01:02.1 irq 17
sym1: No NVRAM, ID 7, Fast-20, SE, parity checking
sym1: SCSI BUS has been reset.
scsi1 : sym-2.2.3
scsi 1:0:0:0: Direct-Access SEAGATE ST373307LSUN72G 0507 PQ: 0 ANSI: 3
target1:0:0: tagged command queuing enabled, command queue depth 16.
target1:0:0: Beginning Domain Validation
target1:0:0: asynchronous
target1:0:0: wide asynchronous
target1:0:0: FAST-20 WIDE SCSI 40.0 MB/s ST (50 ns, offset 16)
target1:0:0: Domain Validation skipping write tests
target1:0:0: Ending Domain Validation
sd 0:0:0:0: [sda] 143374738 512-byte hardware sectors (73408 MB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: fb 00 10 08
sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
sd 0:0:0:0: [sda] 143374738 512-byte hardware sectors (73408 MB)
st: Version 20070203, fixed bufsize 32768, s/g segs 256
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: fb 00 10 08
sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
sda: sda1 sda3
sd 0:0:0:0: [sda] Attached SCSI disk
st 0:0:5:0: Attached scsi tape st0
st 0:0:5:0: st0: try direct i/o: yes (alignment 512 B)
sd 1:0:0:0: [sdb] 143374738 512-byte hardware sectors (73408 MB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: db 00 10 08
sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
sd 1:0:0:0: [sdb] 143374738 512-byte hardware sectors (73408 MB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: db 00 10 08
sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
sdb: sdb1 sdb3
sd 1:0:0:0: [sdb] Attached SCSI disk
md: raid1 personality registered for level 1
md: md0 stopped.
md: bind<sda1>
md: bind<sdb1>
raid1: raid set md0 active with 2 out of 2 mirrors
device-mapper: ioctl: 4.11.0-ioctl (2006-10-12) initialised: [email protected]
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-0, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
ReiserFS: dm-2: found reiserfs format "3.6" with standard journal
ReiserFS: dm-2: using ordered data mode
ReiserFS: dm-2: journal params: device dm-2, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: dm-2: checking transaction log (dm-2)
ReiserFS: dm-2: Using r5 hash to sort names
ReiserFS: dm-3: found reiserfs format "3.6" with standard journal
ReiserFS: dm-3: using ordered data mode
ReiserFS: dm-3: journal params: device dm-3, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: dm-3: checking transaction log (dm-3)
ReiserFS: dm-3: Using r5 hash to sort names
Adding 1999856k swap on /dev/hda2. Priority:-1 extents:1 across:1999856k
rtc_sun_init: Registered Mostek RTC driver.
eth0: Link is up at 100 Mbps, full-duplex.
eth0: Pause is disabled
NET: Registered protocol family 17

PROMLIB: Sun IEEE Boot Prom 'OBP 4.17.1 2005/04/11 14:31'
PROMLIB: Root node compatible:
Linux version 2.6.24.4 (root@seraphim) (gcc version 4.1.2 (Gentoo 4.1.2 p1.0.1)) #5 Mon Mar 31 23:47:14 CEST 2008
ARCH: SUN4U
Ethernet address: 00:03:ba:08:61:7c
[0000000200000000-fffff80000400000] page_structs=262144 node=0 entry=0/0 [0000000200000000-fffff80000800000] page_structs=262144 node=0 entry=1/0 [0000000200000000-fffff80000c00000] page_structs=262144 node=0 entry=2/0 [0000000200000000-fffff80001000000] page_structs=262144 node=0 entry=3/0
OF stdout device is: /pci@1f,0/isa@7/serial@0,3f8
PROM: Built device tree with 60471 bytes of memory.
On node 0 totalpages: 130606
Normal zone: 1567 pages used for memmap
Normal zone: 0 pages reserved
Normal zone: 129039 pages, LIFO batch:15
Movable zone: 0 pages used for memmap
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129039
Kernel command line: root=/dev/hda4
PID hash table entries: 4096 (order: 12, 32768 bytes)
clocksource: mult[b40001] shift[16]
clockevent: mult[16c16bf] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled
Dentry cache hash table entries: 131072 (order: 7, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 6, 524288 bytes)
Memory: 1025864k available (1856k kernel code, 616k data, 104k init) [fffff80000000000,000000006ff0c000]
SLUB: Genslabs=12, HWalign=32, Order=0-2, MinObjects=8, CPUs=1, Nodes=1
Calibrating delay using timer specific routine.. 11.12 BogoMIPS (lpj=55635)
Mount-cache hash table entries: 512
net_namespace: 120 bytes
NET: Registered protocol family 16
PCI: Probing for controllers.
/pci@1f,0: SABRE PCI Bus Module
/pci@1f,0: PCI IO[1fe02000000] MEM[1ff00000000]
PCI: Scanning PBM /pci@1f,0
PCI quirk: region 0800-083f claimed by ali7101 ACPI
PCI quirk: region 0600-061f claimed by ali7101 SMB
isa0: [dma -> (floppy) (parallel)] [power] [serial] [serial]
ebus0: [flashprom] [eeprom] [idprom]
power: Control reg at 1fe02000800
/pci@1f,0/ebus@c/eeprom@1,0: Clock regs at 000001fff1000000
NET: Registered protocol family 2
Time: hbtick clocksource has been installed.
Switched to NOHz mode on CPU #0
IP route cache hash table entries: 8192 (order: 3, 65536 bytes)
TCP established hash table entries: 32768 (order: 6, 524288 bytes)
TCP bind hash table entries: 32768 (order: 5, 262144 bytes)
TCP: Hash tables configured (established 32768 bind 32768)
TCP reno registered
Mini RTC Driver
io scheduler noop registered (default)
Activating ISA DMA hang workarounds.
f007d648: ttyS0 at MMIO 0x1fe020003f8 (irq = 9) is a 16550A
Console: ttyS0 (SU)
console [ttyS0] enabled
f007f1a0: ttyS1 at MMIO 0x1fe020002e8 (irq = 9) is a 16550A
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
ALI15X3: IDE controller (0x10b9:0x5229 rev 0xc3) at PCI slot 0000:00:0d.0
PCI: Enabling device: (0000:00:0d.0), cmd 5
ALI15X3: 100% native mode on irq 15

    ide0: BM-DMA at 0x1fe02000a20-0x1fe02000a27, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0x1fe02000a28-0x1fe02000a2f, BIOS settings: hdc:pio, hdd:pio

Probing IDE interface ide0...
hdb: HL-DT-ST RW/DVD GCC-4480B, ATAPI CD/DVD-ROM drive
hda: ST38410A, ATA DISK drive
hda: host max PIO5 wanted PIO255(auto-tune) selected PIO4
hda: UDMA/66 mode selected
hdb: host max PIO5 wanted PIO255(auto-tune) selected PIO4
hdb: drive side 80-wire cable detection failed, limiting max speed to UDMA33
hdb: UDMA/33 mode selected
ide0 at 0x1fe02000a00-0x1fe02000a07,0x1fe02000a1a on irq 15
Probing IDE interface ide1...
hda: max request size: 128KiB
hda: 16841664 sectors (8622 MB) w/512KiB Cache, CHS=16708/16/63
hda: cache flushes not supported
hda: hda1 hda2 hda3 hda4 hda5
mice: PS/2 mouse device common for all mice
TCP cubic registered
VFS: Mounted root (ext2 filesystem) readonly.
NET: Registered protocol family 1
PCI: Enabling device: (0000:00:08.0), cmd 3
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
SCSI subsystem initialized
hdb: ATAPI 48X DVD-ROM CD-R/RW drive, 2048kB Cache
Uniform CD-ROM driver Revision: 3.20
usbcore: registered new device driver usb
ohci_hcd: 2006 August 04 USB 1.1 'Open' Host Controller (OHCI) Driver
sungem.c:v0.98 8/24/03 David S. Miller ([email protected])
PHY ID: 437421, addr: 1
eth0: Sun GEM (PCI) 10/100/1000BaseT Ethernet 00:03:ba:08:61:7c
eth0: Found Generic MII PHY
ohci1394: fw-host0: OHCI-1394 1.0 (PCI): IRQ=[12] MMIO=[1ff00420000-1ff004207ff] Max Packet=[2048] IR/IT contexts=[4/4]
PCI: Enabling device: (0000:00:0c.3), cmd 2
ohci_hcd 0000:00:0c.3: OHCI Host Controller
ohci_hcd 0000:00:0c.3: new USB bus registered, assigned bus number 1
ohci_hcd 0000:00:0c.3: irq 13, io mem 0x1ff02000000
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 4 ports detected
PCI: Enabling device: (0000:01:02.0), cmd 147
sym0: <875> rev 0x14 at pci 0000:01:02.0 irq 16
sym0: No NVRAM, ID 7, Fast-20, SE, parity checking
sym0: SCSI BUS has been reset.
scsi0 : sym-2.2.3
ieee1394: Host added: ID:BUS[0-00:1023] GUID[0003bafffe08617c]
ieee1394: Node added: ID:BUS[0-01:1023] GUID[0003bafffe0859fb]
eth1394: eth1: IPv4 over IEEE 1394 (fw-host0)
udev: renamed network interface eth1 to fw0
eth0: Link is up at 100 Mbps, full-duplex.
scsi 0:0:0:0: Direct-Access HITACHI DK32EJ72NSUN72G PQ0B PQ: 0 ANSI: 3
target0:0:0: tagged command queuing enabled, command queue depth 16.
target0:0:0: Beginning Domain Validation
target0:0:0: asynchronous
target0:0:0: wide asynchronous
target0:0:0: FAST-20 WIDE SCSI 40.0 MB/s ST (50 ns, offset 16)
target0:0:0: Domain Validation skipping write tests
target0:0:0: Ending Domain Validation
scsi 0:0:5:0: Sequential-Access EXABYTE EXB-89008E030203 V41e PQ: 0 ANSI: 2
target0:0:5: Beginning Domain Validation
target0:0:5: asynchronous
target0:0:5: wide asynchronous
target0:0:5: FAST-10 WIDE SCSI 20.0 MB/s ST (100 ns, offset 15)
target0:0:5: Domain Validation skipping write tests
target0:0:5: Ending Domain Validation
PCI: Enabling device: (0000:01:02.1), cmd 147
sym1: <875> rev 0x14 at pci 0000:01:02.1 irq 17
sym1: No NVRAM, ID 7, Fast-20, SE, parity checking
sym1: SCSI BUS has been reset.
scsi1 : sym-2.2.3
scsi 1:0:0:0: Direct-Access SEAGATE ST373307LSUN72G 0507 PQ: 0 ANSI: 3
target1:0:0: tagged command queuing enabled, command queue depth 16.
target1:0:0: Beginning Domain Validation
target1:0:0: asynchronous
target1:0:0: wide asynchronous
target1:0:0: FAST-20 WIDE SCSI 40.0 MB/s ST (50 ns, offset 16)
target1:0:0: Domain Validation skipping write tests
target1:0:0: Ending Domain Validation
Driver 'sd' needs updating - please use bus_type methods
sd 0:0:0:0: [sda] 143374738 512-byte hardware sectors (73408 MB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: fb 00 10 08
sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
sd 0:0:0:0: [sda] 143374738 512-byte hardware sectors (73408 MB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: fb 00 10 08
sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
sda: sda1 sda3
sd 0:0:0:0: [sda] Attached SCSI disk
sd 1:0:0:0: [sdb] 143374738 512-byte hardware sectors (73408 MB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: db 00 10 08
sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
sd 1:0:0:0: [sdb] 143374738 512-byte hardware sectors (73408 MB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: db 00 10 08
sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
sdb: sdb1 sdb3
sd 1:0:0:0: [sdb] Attached SCSI disk
st: Version 20070203, fixed bufsize 32768, s/g segs 256
Driver 'st' needs updating - please use bus_type methods
st 0:0:5:0: Attached scsi tape st0
st 0:0:5:0: st0: try direct i/o: yes (alignment 512 B)
md: raid1 personality registered for level 1
md: md0 stopped.
md: bind<sdb1>
md: bind<sda1>
md: kicking non-fresh sdb1 from array!
md: unbind<sdb1>
md: export_rdev(sdb1)
raid1: raid set md0 active with 1 out of 2 mirrors
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.12.0-ioctl (2007-10-02) initialised: [email protected]
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-0, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
ReiserFS: dm-2: found reiserfs format "3.6" with standard journal
ReiserFS: dm-2: using ordered data mode
ReiserFS: dm-2: journal params: device dm-2, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: dm-2: checking transaction log (dm-2)
ReiserFS: dm-2: Using r5 hash to sort names
ReiserFS: dm-3: found reiserfs format "3.6" with standard journal
ReiserFS: dm-3: using ordered data mode
ReiserFS: dm-3: journal params: device dm-3, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: dm-3: checking transaction log (dm-3)
ReiserFS: dm-3: Using r5 hash to sort names
Adding 1999856k swap on /dev/hda2. Priority:-1 extents:1 across:1999856k
eth0: Link is up at 100 Mbps, full-duplex.
eth0: Pause is disabled
NET: Registered protocol family 17

PROMLIB: Sun IEEE Boot Prom 'OBP 4.17.1 2005/04/11 14:31'
PROMLIB: Root node compatible:
Linux version 2.6.24.4 (root@seraphim) (gcc version 4.1.2 (Gentoo 4.1.2 p1.0.1)) #5 Mon Mar 31 23:47:14 CEST 2008
ARCH: SUN4U
Ethernet address: 00:03:ba:08:61:7c
[0000000200000000-fffff80000400000] page_structs=262144 node=0 entry=0/0 [0000000200000000-fffff80000800000] page_structs=262144 node=0 entry=1/0 [0000000200000000-fffff80000c00000] page_structs=262144 node=0 entry=2/0 [0000000200000000-fffff80001000000] page_structs=262144 node=0 entry=3/0
OF stdout device is: /pci@1f,0/isa@7/serial@0,3f8
PROM: Built device tree with 60471 bytes of memory.
On node 0 totalpages: 130606
Normal zone: 1567 pages used for memmap
Normal zone: 0 pages reserved
Normal zone: 129039 pages, LIFO batch:15
Movable zone: 0 pages used for memmap
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129039
Kernel command line: root=/dev/hda4
PID hash table entries: 4096 (order: 12, 32768 bytes)
clocksource: mult[b40001] shift[16]
clockevent: mult[16c16bf] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled
Dentry cache hash table entries: 131072 (order: 7, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 6, 524288 bytes)
Memory: 1025864k available (1856k kernel code, 616k data, 104k init) [fffff80000000000,000000006ff0c000]
SLUB: Genslabs=12, HWalign=32, Order=0-2, MinObjects=8, CPUs=1, Nodes=1
Calibrating delay using timer specific routine.. 11.12 BogoMIPS (lpj=55632)
Mount-cache hash table entries: 512
net_namespace: 120 bytes
NET: Registered protocol family 16
PCI: Probing for controllers.
/pci@1f,0: SABRE PCI Bus Module
/pci@1f,0: PCI IO[1fe02000000] MEM[1ff00000000]
PCI: Scanning PBM /pci@1f,0
PCI quirk: region 0800-083f claimed by ali7101 ACPI
PCI quirk: region 0600-061f claimed by ali7101 SMB
isa0: [dma -> (floppy) (parallel)] [power] [serial] [serial]
ebus0: [flashprom] [eeprom] [idprom]
power: Control reg at 1fe02000800
/pci@1f,0/ebus@c/eeprom@1,0: Clock regs at 000001fff1000000
NET: Registered protocol family 2
Time: hbtick clocksource has been installed.
Switched to NOHz mode on CPU #0
IP route cache hash table entries: 8192 (order: 3, 65536 bytes)
TCP established hash table entries: 32768 (order: 6, 524288 bytes)
TCP bind hash table entries: 32768 (order: 5, 262144 bytes)
TCP: Hash tables configured (established 32768 bind 32768)
TCP reno registered
Mini RTC Driver
io scheduler noop registered (default)
Activating ISA DMA hang workarounds.
f007d648: ttyS0 at MMIO 0x1fe020003f8 (irq = 9) is a 16550A
Console: ttyS0 (SU)
console [ttyS0] enabled
f007f1a0: ttyS1 at MMIO 0x1fe020002e8 (irq = 9) is a 16550A
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
ALI15X3: IDE controller (0x10b9:0x5229 rev 0xc3) at PCI slot 0000:00:0d.0
PCI: Enabling device: (0000:00:0d.0), cmd 5
ALI15X3: 100% native mode on irq 15

    ide0: BM-DMA at 0x1fe02000a20-0x1fe02000a27, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0x1fe02000a28-0x1fe02000a2f, BIOS settings: hdc:pio, hdd:pio

Probing IDE interface ide0...
hdb: HL-DT-ST RW/DVD GCC-4480B, ATAPI CD/DVD-ROM drive
hda: ST38410A, ATA DISK drive
hda: host max PIO5 wanted PIO255(auto-tune) selected PIO4
hda: UDMA/66 mode selected
hdb: host max PIO5 wanted PIO255(auto-tune) selected PIO4
hdb: drive side 80-wire cable detection failed, limiting max speed to UDMA33
hdb: UDMA/33 mode selected
ide0 at 0x1fe02000a00-0x1fe02000a07,0x1fe02000a1a on irq 15
Probing IDE interface ide1...
hda: max request size: 128KiB
hda: 16841664 sectors (8622 MB) w/512KiB Cache, CHS=16708/16/63
hda: cache flushes not supported
hda: hda1 hda2 hda3 hda4 hda5
mice: PS/2 mouse device common for all mice
TCP cubic registered
VFS: Mounted root (ext2 filesystem) readonly.
NET: Registered protocol family 1
sungem.c:v0.98 8/24/03 David S. Miller ([email protected])
PHY ID: 437421, addr: 1
eth0: Sun GEM (PCI) 10/100/1000BaseT Ethernet 00:03:ba:08:61:7c
eth0: Found Generic MII PHY
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
ohci1394: fw-host0: OHCI-1394 1.0 (PCI): IRQ=[12] MMIO=[1ff00420000-1ff004207ff] Max Packet=[2048] IR/IT contexts=[4/4]
hdb: ATAPI 48X DVD-ROM CD-R/RW drive, 2048kB Cache
Uniform CD-ROM driver Revision: 3.20
usbcore: registered new device driver usb
ohci_hcd: 2006 August 04 USB 1.1 'Open' Host Controller (OHCI) Driver
PCI: Enabling device: (0000:00:0c.3), cmd 2
ohci_hcd 0000:00:0c.3: OHCI Host Controller
ohci_hcd 0000:00:0c.3: new USB bus registered, assigned bus number 1
ohci_hcd 0000:00:0c.3: irq 13, io mem 0x1ff02000000
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 4 ports detected
PCI: Enabling device: (0000:00:08.0), cmd 3
ieee1394: Host added: ID:BUS[0-00:1023] GUID[0003bafffe08617c]
eth0: Link is up at 100 Mbps, full-duplex.
ieee1394: Node added: ID:BUS[0-01:1023] GUID[0003bafffe0859fb]
eth1394: eth1: IPv4 over IEEE 1394 (fw-host0)
udev: renamed network interface eth1 to fw0
md: raid1 personality registered for level 1
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.12.0-ioctl (2007-10-02) initialised: [email protected]
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-0, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
ReiserFS: dm-2: found reiserfs format "3.6" with standard journal
ReiserFS: dm-2: using ordered data mode
ReiserFS: dm-2: journal params: device dm-2, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: dm-2: checking transaction log (dm-2)
ReiserFS: dm-2: Using r5 hash to sort names
ReiserFS: dm-3: found reiserfs format "3.6" with standard journal
ReiserFS: dm-3: using ordered data mode
ReiserFS: dm-3: journal params: device dm-3, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: dm-3: checking transaction log (dm-3)
ReiserFS: dm-3: Using r5 hash to sort names
Adding 1999856k swap on /dev/hda2. Priority:-1 extents:1 across:1999856k
eth0: Link is up at 100 Mbps, full-duplex.
eth0: Pause is disabled
NET: Registered protocol family 17

maintainer
#3 lore raw
In reply to Andrew Morton
Quoted 11 lines · Andrew Morton

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 3 lines

Steps to reproduce:

Let udev load sym53c8xx in kernel 2.6.23 or newer.

urgh. Perhaps it's related to platform IRQ routing or something.

I'd suggest that the next step would be to send us the `dmesg -s 1000000' output for both good and bad kernels. A comparison might show where things went bad.

Yes, that would be my guess too ... although I don't see anything amiss
in the dmesg I note you have two ethernet interfaces:

eth0: Sun GEM (PCI) 10/100/1000BaseT Ethernet 00:03:ba:08:61:7c
eth1394: eth1: IPv4 over IEEE 1394 (fw-host0)

I'm assuming eth0 is the problem?

Could you also send us the output of /proc/interrupts, /proc/iomem and /proc/ioports just to see if we have a problem. Also, if eth0 is on its own interrupt line, does the interrupt count rise even while the interface is non functional?

Thanks,

James

#4 lore raw
Quoted 6 lines · On Tue, 01 Apr 2008 09:11:55 -0500 James Bottomley…
Quoted 4 lines · Andrew Morton

I'd suggest that the next step would be to send us the `dmesg -s 1000000' output for both good and bad kernels. A comparison might show where things went bad.

Quoted 7 lines

Yes, that would be my guess too ... although I don't see anything amiss
in the dmesg I note you have two ethernet interfaces:

eth0: Sun GEM (PCI) 10/100/1000BaseT Ethernet 00:03:ba:08:61:7c
eth1394: eth1: IPv4 over IEEE 1394 (fw-host0)

I'm assuming eth0 is the problem?

Yes, the Sun GEM interface is the problem. Ethernet over FireWire (a direct link between the webserver and database server) works just fine. In a minimal config with no devices enabled except the SU serial line (needed for console), the dual SCSI adapter and the Sun GEM nic, the problem still remained the same.

Quoted 4 lines

Could you also send us the output of /proc/interrupts, /proc/iomem and /proc/ioports just to see if we have a problem. Also, if eth0 is on its own interrupt line, does the interrupt count rise even while the interface is non functional?

Here goes:

seraphim ~ # cat /proc/interrupts
CPU0

  0:       7259     <NULL>  timer                                               
  8:          0      sun4u  power                                               
  9:        405      sun4u  su(serial)                                          
 11:          1      sun4u  eth0                                                
 12:         95      sun4u  ohci1394                                            
 13:          0      sun4u  ohci_hcd:usb1                                       
 14:          0      sun4u  ALI 5451                                            
 15:       2265      sun4u  ide0                                                
 16:        293      sun4u  sym53c8xx                                           
 17:         69      sun4u  sym53c8xx

seraphim ~ # cat /proc/iomem
1fe020002e8-1fe020002ef : su
1fe020003f8-1fe020003ff : su
1fe02000800-1fe02000803 : power
1ff00000000-1ffffffffff : /pci@1f,0
1ff000a0000-1ff000bffff : Video RAM area
1ff000c0000-1ff000c7fff : Video ROM
1ff000f0000-1ff000fffff : System ROM
1ff00400000-1ff0041ffff : sungem
1ff00420000-1ff004207ff : ohci1394
1ff00424000-1ff00425fff : ALI 5451
1ff02000000-1ff02ffffff : ohci_hcd
1ff03000000-1ff03001fff : sym53c8xx
1ff03002000-1ff03003fff : sym53c8xx
1ff03004000-1ff03005fff : sym53c8xx
1ff03006000-1ff03007fff : sym53c8xx
1ffc0000000-1ffdfffffff : IOMMU
1fff1000000-1fff1001fff : clock

seraphim ~ # cat /proc/ioports
00000600-0000061f : ali1535_smbus
1fe02000000-1fe02ffffff : /pci@1f,0
1fe02000600-1fe0200061f : 0000:00:03.0
1fe02000800-1fe0200083f : 0000:00:03.0
1fe02000900-1fe020009ff : ALI 5451
1fe02000a00-1fe02000a07 : ide0
1fe02000a1a-1fe02000a1a : ide0
1fe02000a20-1fe02000a27 : ide0
1fe02000a28-1fe02000a2f : ide1
1fe02001000-1fe020010ff : sym53c8xx
1fe02001100-1fe020011ff : sym53c8xx

Signature · 1 line
Jos van der Ende <[email protected]>
#5 lore raw
In reply to James Bottomley

Hello all,

I did a bit more testing, and I think this may be related to the order in which modules are loaded.

If I let udev load sungem, and load sym53c8xx manually, everything works.

If I let udev load sym53c8xx, and load sungem manually, I get the non-functional network.

If I let udev load both modules, I also get the non-functional network. While udev loads sungem first and sym53c8xx later, I don't suppose it waits for one module to 'settle' before loading the next. :-)

So to sum it up, the bug is triggered if sym53c8xx is loaded before sungem is.

Signature · 1 line
Jos van der Ende <[email protected]>
maintainer
#6 lore raw
Quoted 10 lines · Jos van der Ende

Hello all,

I did a bit more testing, and I think this may be related to the order in which modules are loaded.

If I let udev load sungem, and load sym53c8xx manually, everything works.

If I let udev load sym53c8xx, and load sungem manually, I get the non-functional network.

If I let udev load both modules, I also get the non-functional network. While udev loads sungem first and sym53c8xx later, I don't suppose it waits for one module to 'settle' before loading the next. :-)

That's odd ... it's behaving like a resource conflict. However, the ports and interrupt trace didn't betray anything. What does lspci -vv say for each of the devices? Also, if you remove the sym2 module in the problem case, does the sungem come back to life?

I'm afraid I can't see anything relevant looking over the sym2 changes, so you might need to bisect this to identify the culprit.

James

#7 lore raw
Quoted 3 lines · On Tue, 01 Apr 2008 15:19:29 -0500 James Bottomley…

That's odd ... it's behaving like a resource conflict. However, the ports and interrupt trace didn't betray anything. What does lspci -vv say for each of the devices?

Output from lspci -vv attached.

Quoted 2 lines

Also, if you remove the sym2 module in the
problem case, does the sungem come back to life?

No, once it is hosed it stays hosed until the next boot. Fiddling with the wrong ioports maybe?

Quoted 2 lines

I'm afraid I can't see anything relevant looking over the sym2 changes, so you might need to bisect this to identify the culprit.

Working on that, but it is a hassle as this bitty-box needs some time to compile a kernel. 2.6.23-rc1 didn't boot, for starters.

Signature · 1 line
Jos van der Ende <[email protected]>
maintainer
#8 lore raw
Quoted 8 lines · Jos van der Ende
Quoted 3 lines · On Tue, 01 Apr 2008 15:19:29 -0500 James Bottomley…

That's odd ... it's behaving like a resource conflict. However, the ports and interrupt trace didn't betray anything. What does lspci -vv say for each of the devices?

Output from lspci -vv attached.

Thanks ... unfortunately looks normal too. The gem has a single memory region; the sym2 has 2 mem and one IO region, all of which show up in the /proc/iomem|ports.

Quoted 4 lines
Quoted 2 lines

Also, if you remove the sym2 module in the
problem case, does the sungem come back to life?

No, once it is hosed it stays hosed until the next boot. Fiddling with the wrong ioports maybe?

Yes ... that's what I guess. Just as one last grasp at a straw, is there any difference in /proc/iomem or /proc/ioports for the working case (sungem loaded first followed by sym2)?

Quoted 4 lines
Quoted 2 lines

I'm afraid I can't see anything relevant looking over the sym2 changes, so you might need to bisect this to identify the culprit.

Working on that, but it is a hassle as this bitty-box needs some time to compile a kernel. 2.6.23-rc1 didn't boot, for starters.

Sorry ... can't think of much else that will help.

James

#9 lore raw
Quoted 3 lines · On Tue, 01 Apr 2008 16:14:14 -0500 James Bottomley…

Yes ... that's what I guess. Just as one last grasp at a straw, is there any difference in /proc/iomem or /proc/ioports for the working case (sungem loaded first followed by sym2)?

Nope, exactly the same.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I'm afraid I can't see anything relevant looking over the sym2 changes, so you might need to bisect this to identify the culprit.

First results are in: 2.6.23-rc1 could not boot, 2.6.23-rc2 already had the problem.

Signature · 1 line
Jos van der Ende <[email protected]>
#10 lore raw
In reply to James Bottomley

Maybe it's nothing, but I did notice that sym53c8xx compiles with a warning of a possibly uninitialized variable:

CC [M] drivers/scsi/sym53c8xx_2/sym_glue.o
drivers/scsi/sym53c8xx_2/sym_glue.c: In function 'sym_eh_handler':
drivers/scsi/sym53c8xx_2/sym_glue.c:612: warning: 'io_reset' may be used uninitialized in this function

Signature · 1 line
Jos van der Ende <[email protected]>
#11 lore raw
Quoted 5 lines · Jos van der Ende

Maybe it's nothing, but I did notice that sym53c8xx compiles with a warning of a possibly uninitialized variable:

CC [M] drivers/scsi/sym53c8xx_2/sym_glue.o
drivers/scsi/sym53c8xx_2/sym_glue.c: In function 'sym_eh_handler':
drivers/scsi/sym53c8xx_2/sym_glue.c:612: warning: 'io_reset' may be used uninitialized in this function

Yeah, that's nothing. It's actually a bug in GCC that produces that warning (and it's code that'll never be executed on your platform anyway).

Signature · 4 lines
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."
#12 lore raw

Finally, bisecting is done. :-)

Well, it took more reboots than a typical Windows XP installation (and thank the heavens for my Sparc64 cross compiler on my Core 2 Duo), but this seems to be the culprit:

5a606b72a4309a656cd1a19ad137dc5557c4b8ea is first bad commit commit 5a606b72a4309a656cd1a19ad137dc5557c4b8ea

Author: David S. Miller <[email protected]>
Date:   Mon Jul 9 22:40:36 2007 -0700

[SPARC64]: Do not ACK an INO if it is disabled or inprogress.

This is also a partial workaround for a bug in the LDOM firmware which double-transmits RX inos during high load. Without this, such an event causes the kernel to loop forever in the interrupt call chain ACK'ing but never actually running the IRQ handler (and thus clearing the interrupt condition in the device).

There is still a bad potential effect when double INOs occur, not covered by this changeset. Namely, if the INO is already on the per-cpu INO vector list, we still blindly re-insert it and thus we can end up losing interrupts already linked in after it.

We could deal with that by traversing the list before insertion, but that's too expensive for this edge case.

Signed-off-by David S. Miller <[email protected]>

:040000 040000 7e65c9b16e6c37f2c3f83195c5a57b4d2b8f0a7c e7a7bedcc88d33793a6525e9337a1a51982bc513 M arch

Signature · 1 line
Jos van der Ende <[email protected]>
maintainer
#13 lore raw
Quoted 1 line · Jos van der Ende

Finally, bisecting is done. :-)

Thanks for doing this ... we'd never have found it by looking at the driver code ...

Quoted 5 lines

Well, it took more reboots than a typical Windows XP installation (and thank the heavens for my Sparc64 cross compiler on my Core 2 Duo), but this seems to be the culprit:

5a606b72a4309a656cd1a19ad137dc5557c4b8ea is first bad commit

Reading the code for this, it seems that something fiddled with the IRQ_DISABLED or IRQ_PENDING flags when it came time for the ->eoi() so the gem interrupt is always held pending (because it's never ended).

Since the sym2 is on interrupts 16 and 17 and gem on 11 (and the descriptors are separate entities in the irq_desc array) I can't really see how sym2 would be doing this.

James

#14 lore raw
Quoted 3 lines · On Wed, 02 Apr 2008 10:49:19 -0500 James Bottomley…

Reading the code for this, it seems that something fiddled with the IRQ_DISABLED or IRQ_PENDING flags when it came time for the ->eoi() so the gem interrupt is always held pending (because it's never ended).

So Andrew's first hunch that interrupts were somehow involved is right.

Quoted 3 lines

Since the sym2 is on interrupts 16 and 17 and gem on 11 (and the descriptors are separate entities in the irq_desc array) I can't really see how sym2 would be doing this.

Yeah, that has me baffled too. Still, the fact is that I can only trigger the bug by loading sym53c8xx before sungem. I have yet to find any other conditions that trigger it. Loading sungem before sym53c8xx on an affected kernel gives no trouble at all.

Signature · 1 line
Jos van der Ende <[email protected]>