linux 2.6.9 still having network code problems

DISCUSSION Christopher Chan 4 participants 14 messages Nov 2, 2004 – Nov 4, 2004

Subsystems (3)

Networking Network drivers libata

lore

Share & tools

lore

Christopher Chan

author

2004-11-02 08:14 UTC #0 lore raw

Mr. Morton,

I'm ccing you due to the oom-killer stuff.

Previously with 2.6.7 I had to use this values in sysctl to be able to continue to access the box:

net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.route.gc_thresh = 65536
net.ipv4.route.max_size = 1048576

2.6.9 without this values gave me similar problems with 2.6.7 without the above values.

Sample log lines below:

There are multiple oom-killer lines (different pid for smtpd process) and the network related messages repeated until the box was rebooted about 45 mins later.

Nov  1 03:15:58 spf5-3 kernel: oom-killer: gfp_mask=0x1d2
Nov  1 03:15:58 spf5-3 kernel: DMA per-cpu:
Nov  1 03:15:58 spf5-3 kernel: cpu 0 hot: low 2, high 6, batch 1
Nov  1 03:15:58 spf5-3 kernel: cpu 0 cold: low 0, high 2, batch 1
Nov  1 03:15:58 spf5-3 kernel: cpu 1 hot: low 2, high 6, batch 1
Nov  1 03:15:58 spf5-3 kernel: cpu 1 cold: low 0, high 2, batch 1
Nov  1 03:15:58 spf5-3 kernel: Normal per-cpu:
Nov  1 03:15:58 spf5-3 kernel: cpu 0 hot: low 32, high 96, batch 16
Nov  1 03:15:58 spf5-3 kernel: cpu 0 cold: low 0, high 32, batch 16
Nov  1 03:15:58 spf5-3 kernel: cpu 1 hot: low 32, high 96, batch 16
Nov  1 03:15:58 spf5-3 kernel: cpu 1 cold: low 0, high 32, batch 16
Nov  1 03:15:58 spf5-3 kernel: HighMem per-cpu:
Nov  1 03:15:58 spf5-3 kernel: cpu 0 hot: low 16, high 48, batch 8
Nov  1 03:15:58 spf5-3 kernel: cpu 0 cold: low 0, high 16, batch 8
Nov  1 03:15:58 spf5-3 kernel: cpu 1 hot: low 16, high 48, batch 8
Nov  1 03:15:58 spf5-3 kernel: cpu 1 cold: low 0, high 16, batch 8
Nov  1 03:15:58 spf5-3 kernel:
Nov  1 03:15:58 spf5-3 kernel: Free pages:        1264kB (128kB HighMem)
Nov  1 03:15:58 spf5-3 kernel: Active:204122 inactive:68 dirty:0 writeback:0 unstable:0 free:316 slab:27673 mapped:204956 pagetables:22623
Nov  1 03:15:58 spf5-3 kernel: DMA free:16kB min:16kB low:32kB high:48kB active:10544kB inactive:0kB present:16384kB
Nov  1 03:15:59 spf5-3 kernel: protections[]: 0 0 0
Nov  1 03:15:59 spf5-3 kernel: Normal free:1120kB min:936kB low:1872kB high:2808kB active:693760kB inactive:556kB present:901120kB
Nov  1 03:15:59 spf5-3 kernel: protections[]: 0 0 0
Nov  1 03:15:59 spf5-3 kernel: HighMem free:128kB min:128kB low:256kB high:384kB active:112132kB inactive:152kB present:131072kB
Nov  1 03:15:59 spf5-3 kernel: protections[]: 0 0 0
Nov  1 03:15:59 spf5-3 kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 16kB
Nov  1 03:15:59 spf5-3 kernel: Normal: 104*4kB 4*8kB 0*16kB 1*32kB 0*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1248kB
Nov  1 03:15:59 spf5-3 kernel: HighMem: 0*4kB 0*8kB 0*16kB 2*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 128kB
Nov  1 03:15:59 spf5-3 kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Nov  1 03:16:00 spf5-3 kernel: Out of Memory: Killed process 21861 (smtpd).
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed at net/ipv4/tcp.c (1348)
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed at net/ipv4/tcp.c (1348)
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed at net/ipv4/tcp.c (1348)

Andrew Morton

2004-11-02 09:19 UTC #1 lore raw

Quoted 5 lines · Christopher Chan

Mr. Morton,
I'm ccing you due to the oom-killer stuff.

gee thanks ;)

Quoted 6 lines

Previously with 2.6.7 I had to use this values in sysctl to be able to continue to access the box:
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.route.gc_thresh = 65536
net.ipv4.route.max_size = 1048576

Why?

Quoted 2 lines

2.6.9 without this values gave me similar problems with 2.6.7 without the above values.

What problems?

Quoted 25 lines

Sample log lines below:

There are multiple oom-killer lines (different pid for smtpd process) and the network related messages repeated until the box was rebooted about 45 mins later.

Nov  1 03:15:58 spf5-3 kernel: oom-killer: gfp_mask=0x1d2
Nov  1 03:15:58 spf5-3 kernel: DMA per-cpu:
Nov  1 03:15:58 spf5-3 kernel: cpu 0 hot: low 2, high 6, batch 1
Nov  1 03:15:58 spf5-3 kernel: cpu 0 cold: low 0, high 2, batch 1
Nov  1 03:15:58 spf5-3 kernel: cpu 1 hot: low 2, high 6, batch 1
Nov  1 03:15:58 spf5-3 kernel: cpu 1 cold: low 0, high 2, batch 1
Nov  1 03:15:58 spf5-3 kernel: Normal per-cpu:
Nov  1 03:15:58 spf5-3 kernel: cpu 0 hot: low 32, high 96, batch 16
Nov  1 03:15:58 spf5-3 kernel: cpu 0 cold: low 0, high 32, batch 16
Nov  1 03:15:58 spf5-3 kernel: cpu 1 hot: low 32, high 96, batch 16
Nov  1 03:15:58 spf5-3 kernel: cpu 1 cold: low 0, high 32, batch 16
Nov  1 03:15:58 spf5-3 kernel: HighMem per-cpu:
Nov  1 03:15:58 spf5-3 kernel: cpu 0 hot: low 16, high 48, batch 8
Nov  1 03:15:58 spf5-3 kernel: cpu 0 cold: low 0, high 16, batch 8
Nov  1 03:15:58 spf5-3 kernel: cpu 1 hot: low 16, high 48, batch 8
Nov  1 03:15:58 spf5-3 kernel: cpu 1 cold: low 0, high 16, batch 8
Nov  1 03:15:58 spf5-3 kernel:
Nov  1 03:15:58 spf5-3 kernel: Free pages:        1264kB (128kB HighMem)
Nov  1 03:15:58 spf5-3 kernel: Active:204122 inactive:68 dirty:0

Quoted 1 line

writeback:0 unstable:0 free:316 slab:27673 mapped:204956 pagetables:22623

You ran out of memory. All your memory is in use by userspace processes.

Quoted 16 lines

Nov  1 03:15:58 spf5-3 kernel: DMA free:16kB min:16kB low:32kB high:48kB 
active:10544kB inactive:0kB present:16384kB
Nov  1 03:15:59 spf5-3 kernel: protections[]: 0 0 0
Nov  1 03:15:59 spf5-3 kernel: Normal free:1120kB min:936kB low:1872kB 
high:2808kB active:693760kB inactive:556kB present:901120kB
Nov  1 03:15:59 spf5-3 kernel: protections[]: 0 0 0
Nov  1 03:15:59 spf5-3 kernel: HighMem free:128kB min:128kB low:256kB 
high:384kB active:112132kB inactive:152kB present:131072kB
Nov  1 03:15:59 spf5-3 kernel: protections[]: 0 0 0
Nov  1 03:15:59 spf5-3 kernel: DMA: 0*4kB 0*8kB 1*16kB 0*32kB 0*64kB 
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 16kB
Nov  1 03:15:59 spf5-3 kernel: Normal: 104*4kB 4*8kB 0*16kB 1*32kB 
0*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1248kB
Nov  1 03:15:59 spf5-3 kernel: HighMem: 0*4kB 0*8kB 0*16kB 2*32kB 1*64kB 
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 128kB
Nov  1 03:15:59 spf5-3 kernel: Swap cache: add 0, delete 0, find 0/0,

Quoted 1 line

race 0+0

Try mounting some swapspace.

Quoted 20 lines

Nov  1 03:16:00 spf5-3 kernel: Out of Memory: Killed process 21861 (smtpd).
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) 
failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) 
failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (tp->copied_seq == 
tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed at 
net/ipv4/tcp.c (1348)
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) 
failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (tp->copied_seq == 
tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed at 
net/ipv4/tcp.c (1348)
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) 
failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (tp->copied_seq ==

Quoted 2 lines

tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed at net/ipv4/tcp.c (1348)

This is a networking bug.

Yes...it's the same one that won't allow me to use the e100 driver with NAPI enabled with or without them sysctl tweaks and the same one that forces me to use the tweaks without NAPI enabled for the e100 driver.

With NAPI enabled, I see the problem when under pressure.

FYI, the 2.6.9 kernel is running with NAPI enabled.

Andrew Morton

2004-11-02 09:41 UTC #3 lore raw

Quoted 3 lines · Christopher Chan

DOH. the kernel swapped the controller order on me...fstab entries
referred to the wrong device names...

Eh? That shouldn't happen. Could you please generate a full report of this, send it to the appropriate list and cc myself?

Although if that's the 2.6.9 behaviour it's probably a bit late to fix it up.

Christopher Chan

author

2004-11-02 09:04 UTC #4 lore raw

linux 2.6.9 controller order reordered. was: [Re: linux 2.6.9 still having network code problems]

Quoted 11 lines · Andrew Morton

Quoted 1 line · Christopher Chan
DOH. the kernel swapped the controller order on me...fstab entries referred to the wrong device names...
Eh? That shouldn't happen. Could you please generate a full report of this, send it to the appropriate list and cc myself?
Although if that's the 2.6.9 behaviour it's probably a bit late to fix it up.

Hardware: Tyan mb with Serverworks chipset and onboard Promise controller.
The two disks on promise channel since Serverworks does not support ATA100

FC2 Installation will recognise the disks as hde and hdg fstab is accordingly setup as so for swap (everything else are md)

Changing to a 2.6.9 kernel that has the promise driver compiled in will change the disks to hda and hdc.

Andrew Morton

2004-11-02 09:09 UTC #5 lore raw

Re: linux 2.6.9 controller order reordered. was: [Re: linux 2.6.9 still having network code problems]

Quoted 15 lines · Christopher Chan

Quoted 6 lines · Andrew Morton
Quoted 2 lines · Christopher Chan
DOH. the kernel swapped the controller order on me...fstab entries referred to the wrong device names...
Quoted 6 lines
Eh? That shouldn't happen. Could you please generate a full report of this, send it to the appropriate list and cc myself?
Although if that's the 2.6.9 behaviour it's probably a bit late to fix it up.

Quoted 8 lines

Hardware: Tyan mb with Serverworks chipset and onboard Promise controller.
The two disks on promise channel since Serverworks does not support ATA100
FC2 Installation will recognise the disks as hde and hdg fstab is accordingly setup as so for swap (everything else are md)
Changing to a 2.6.9 kernel that has the promise driver compiled in will change the disks to hda and hdc.

Could you send the dmesg output so we can see the probing activity?

Christopher Chan

author

2004-11-02 09:21 UTC #6 lore raw

Re: linux 2.6.9 controller order reordered. was: [Re: linux 2.6.9 still having network code problems]

Quoted 11 lines

Quoted 7 lines
Hardware: Tyan mb with Serverworks chipset and onboard Promise controller.
The two disks on promise channel since Serverworks does not support ATA100
FC2 Installation will recognise the disks as hde and hdg fstab is accordingly setup as so for swap (everything else are md)
Changing to a 2.6.9 kernel that has the promise driver compiled in will change the disks to hda and hdc.
Could you send the dmesg output so we can see the probing activity?

With FC2 2.6.8-1.521smp

~~~~~~~~~~~~~~~~~~~~~~~~

Oct 14 11:34:50 spf5-3 kernel: RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
Oct 14 11:34:50 spf5-3 kernel: Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
Oct 14 11:34:50 spf5-3 kernel: ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Oct 14 11:34:50 spf5-3 nfslock: rpc.statd startup succeeded
Oct 14 11:34:50 spf5-3 kernel: PDC20267: IDE controller at PCI slot 0000:00:03.0
Oct 14 11:34:50 spf5-3 kernel: PDC20267: chipset revision 2
Oct 14 11:34:50 spf5-3 kernel: PDC20267: ROM enabled at 0xfead0000
Oct 14 11:34:50 spf5-3 kernel: PDC20267: 100%% native mode on irq 161
Oct 14 11:34:50 spf5-3 kernel: PDC20267: (U)DMA Burst Bit ENABLED Primary MASTER Mode Secondary MASTER Mode.
Oct 14 11:34:50 spf5-3 kernel:     ide2: BM-DMA at 0xdf00-0xdf07, BIOS settings: hde:pio, hdf:pio
Oct 14 11:34:50 spf5-3 kernel:     ide3: BM-DMA at 0xdf08-0xdf0f, BIOS settings: hdg:pio, hdh:pio
Oct 14 11:34:50 spf5-3 kernel: hde: Maxtor 5T040H4, ATA DISK drive
Oct 14 11:34:50 spf5-3 kernel: Using deadline io scheduler
Oct 14 11:34:50 spf5-3 kernel: ide2 at 0xdfe0-0xdfe7,0xdfae on irq 161
Oct 14 11:34:50 spf5-3 kernel: hdg: Maxtor 5T040H4, ATA DISK drive
Oct 14 11:34:50 spf5-3 kernel: ide3 at 0xdfa0-0xdfa7,0xdfaa on irq 161
Oct 14 11:34:50 spf5-3 kernel: SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1
Oct 14 11:34:50 spf5-3 kernel: PCI: Enabling device 0000:00:0f.1 (0000 -> 0001)
Oct 14 11:34:50 spf5-3 kernel: SvrWks OSB4: chipset revision 0
Oct 14 11:34:50 spf5-3 kernel: SvrWks OSB4: not 100%% native mode: will probe irqs later
Oct 14 11:34:50 spf5-3 kernel:     ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hda:pio, hdb:pio
Oct 14 11:34:50 spf5-3 kernel:     ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdc:pio, hdd:pio
Oct 14 11:34:50 spf5-3 kernel: hde: max request size: 128KiB
Oct 14 11:34:50 spf5-3 kernel: hde: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
Oct 14 11:34:50 spf5-3 kernel:  hde: hde1 hde2 hde3 hde4 < hde5 >
Oct 14 11:34:50 spf5-3 kernel: hdg: max request size: 128KiB
Oct 14 11:34:50 spf5-3 kernel: hdg: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
Oct 14 11:34:50 spf5-3 kernel:  hdg: hdg1 hdg2 hdg3 hdg4 < hdg5 >

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

With 2.6.9

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Using deadline io scheduler
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize Compaq SMART2 Driver (v 2.6.0)
HP CISS Driver (v 2.6.2)
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PDC20267: IDE controller at PCI slot 0000:00:03.0
PDC20267: chipset revision 2
PDC20267: ROM enabled at 0xfead0000
PDC20267: 100% native mode on irq 31
PDC20267: (U)DMA Burst Bit ENABLED Primary MASTER Mode Secondary MASTER Mode.

    ide0: BM-DMA at 0xdf00-0xdf07, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0xdf08-0xdf0f, BIOS settings: hdc:pio, hdd:pio

Probing IDE interface ide0...
hda: Maxtor 5T040H4, ATA DISK drive
ide0 at 0xdfe0-0xdfe7,0xdfae on irq 31
Probing IDE interface ide1...
hdc: Maxtor 5T040H4, ATA DISK drive
ide1 at 0xdfa0-0xdfa7,0xdfaa on irq 31
SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1
PCI: Enabling device 0000:00:0f.1 (0000 -> 0001) SvrWks OSB4: chipset revision 0
SvrWks OSB4: not 100% native mode: will probe irqs later

    ide2: BM-DMA at 0xffa0-0xffa7, BIOS settings: hde:pio, hdf:pio
    ide3: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdg:pio, hdh:pio

Probing IDE interface ide2...
Probing IDE interface ide3...
Probing IDE interface ide2...
Probing IDE interface ide3...
Probing IDE interface ide4...
ide4: Wait for ready failed before probe !
Probing IDE interface ide5...
ide5: Wait for ready failed before probe !
hda: max request size: 128KiB
hda: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hda: cache flushes not supported
hda: hda1 hda2 hda3 hda4 < hda5 >
hdc: max request size: 128KiB
hdc: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hdc: cache flushes not supported
hdc: hdc1 hdc2 hdc3 hdc4 < hdc5 >

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Andrew Morton

2004-11-02 09:27 UTC #7 lore raw

Re: linux 2.6.9 controller order reordered. was: [Re: linux 2.6.9 still having network code problems]

Quoted 109 lines · Christopher Chan

Quoted 4 lines
Quoted 4 lines
Hardware: Tyan mb with Serverworks chipset and onboard Promise controller.
The two disks on promise channel since Serverworks does not support ATA100
Quoted 9 lines
Quoted 5 lines
FC2 Installation will recognise the disks as hde and hdg fstab is accordingly setup as so for swap (everything else are md)
Changing to a 2.6.9 kernel that has the promise driver compiled in will change the disks to hda and hdc.
Could you send the dmesg output so we can see the probing activity?
With FC2 2.6.8-1.521smp
~~~~~~~~~~~~~~~~~~~~~~~~
Oct 14 11:34:50 spf5-3 kernel: RAMDISK driver initialized: 16 RAM disks 
of 16384K size 1024 blocksize
Oct 14 11:34:50 spf5-3 kernel: Uniform Multi-Platform E-IDE driver 
Revision: 7.00alpha2
Oct 14 11:34:50 spf5-3 kernel: ide: Assuming 33MHz system bus speed for 
PIO modes; override with idebus=xx
Oct 14 11:34:50 spf5-3 nfslock: rpc.statd startup succeeded
Oct 14 11:34:50 spf5-3 kernel: PDC20267: IDE controller at PCI slot 
0000:00:03.0
Oct 14 11:34:50 spf5-3 kernel: PDC20267: chipset revision 2
Oct 14 11:34:50 spf5-3 kernel: PDC20267: ROM enabled at 0xfead0000
Oct 14 11:34:50 spf5-3 kernel: PDC20267: 100%% native mode on irq 161
Oct 14 11:34:50 spf5-3 kernel: PDC20267: (U)DMA Burst Bit ENABLED 
Primary MASTER Mode Secondary MASTER Mode.
Oct 14 11:34:50 spf5-3 kernel:     ide2: BM-DMA at 0xdf00-0xdf07, BIOS 
settings: hde:pio, hdf:pio
Oct 14 11:34:50 spf5-3 kernel:     ide3: BM-DMA at 0xdf08-0xdf0f, BIOS 
settings: hdg:pio, hdh:pio
Oct 14 11:34:50 spf5-3 kernel: hde: Maxtor 5T040H4, ATA DISK drive
Oct 14 11:34:50 spf5-3 kernel: Using deadline io scheduler
Oct 14 11:34:50 spf5-3 kernel: ide2 at 0xdfe0-0xdfe7,0xdfae on irq 161
Oct 14 11:34:50 spf5-3 kernel: hdg: Maxtor 5T040H4, ATA DISK drive
Oct 14 11:34:50 spf5-3 kernel: ide3 at 0xdfa0-0xdfa7,0xdfaa on irq 161
Oct 14 11:34:50 spf5-3 kernel: SvrWks OSB4: IDE controller at PCI slot 
0000:00:0f.1
Oct 14 11:34:50 spf5-3 kernel: PCI: Enabling device 0000:00:0f.1 (0000 
-> 0001)
Oct 14 11:34:50 spf5-3 kernel: SvrWks OSB4: chipset revision 0
Oct 14 11:34:50 spf5-3 kernel: SvrWks OSB4: not 100%% native mode: will 
probe irqs later
Oct 14 11:34:50 spf5-3 kernel:     ide0: BM-DMA at 0xffa0-0xffa7, BIOS 
settings: hda:pio, hdb:pio
Oct 14 11:34:50 spf5-3 kernel:     ide1: BM-DMA at 0xffa8-0xffaf, BIOS 
settings: hdc:pio, hdd:pio
Oct 14 11:34:50 spf5-3 kernel: hde: max request size: 128KiB
Oct 14 11:34:50 spf5-3 kernel: hde: 80043264 sectors (40982 MB) 
w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
Oct 14 11:34:50 spf5-3 kernel:  hde: hde1 hde2 hde3 hde4 < hde5 >
Oct 14 11:34:50 spf5-3 kernel: hdg: max request size: 128KiB
Oct 14 11:34:50 spf5-3 kernel: hdg: 80043264 sectors (40982 MB) 
w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
Oct 14 11:34:50 spf5-3 kernel:  hdg: hdg1 hdg2 hdg3 hdg4 < hdg5 >
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
With 2.6.9
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Using deadline io scheduler
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize Compaq SMART2 Driver (v 2.6.0)
HP CISS Driver (v 2.6.2)
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PDC20267: IDE controller at PCI slot 0000:00:03.0
PDC20267: chipset revision 2
PDC20267: ROM enabled at 0xfead0000
PDC20267: 100% native mode on irq 31
PDC20267: (U)DMA Burst Bit ENABLED Primary MASTER Mode Secondary MASTER Mode.
     ide0: BM-DMA at 0xdf00-0xdf07, BIOS settings: hda:pio, hdb:pio
     ide1: BM-DMA at 0xdf08-0xdf0f, BIOS settings: hdc:pio, hdd:pio
Probing IDE interface ide0...
hda: Maxtor 5T040H4, ATA DISK drive
ide0 at 0xdfe0-0xdfe7,0xdfae on irq 31
Probing IDE interface ide1...
hdc: Maxtor 5T040H4, ATA DISK drive
ide1 at 0xdfa0-0xdfa7,0xdfaa on irq 31
SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1
PCI: Enabling device 0000:00:0f.1 (0000 -> 0001) SvrWks OSB4: chipset revision 0
SvrWks OSB4: not 100% native mode: will probe irqs later
     ide2: BM-DMA at 0xffa0-0xffa7, BIOS settings: hde:pio, hdf:pio
     ide3: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdg:pio, hdh:pio
Probing IDE interface ide2...
Probing IDE interface ide3...
Probing IDE interface ide2...
Probing IDE interface ide3...
Probing IDE interface ide4...
ide4: Wait for ready failed before probe !
Probing IDE interface ide5...
ide5: Wait for ready failed before probe !
hda: max request size: 128KiB
hda: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hda: cache flushes not supported
hda: hda1 hda2 hda3 hda4 < hda5 >
hdc: max request size: 128KiB
hdc: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hdc: cache flushes not supported
hdc: hdc1 hdc2 hdc3 hdc4 < hdc5 >

Quoted 1 line

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Well the problem is that "Wait for ready" thing. Was that an alanism?

Alan Cox

2004-11-02 17:43 UTC #8 lore raw

Re: linux 2.6.9 controller order reordered. was: [Re: linux 2.6.9 still having network code problems]

Quoted 1 line · Andrew Morton

Well the problem is that "Wait for ready" thing. Was that an alanism?

What are you on about ? The problem is that in the two traces his serverworks has moved from ide0 to ide2.

Alan

Christopher Chan

author

2004-11-03 06:57 UTC #9 lore raw

Re: linux 2.6.9 controller order reordered. was: [Re: linux 2.6.9 still having network code problems]

Quoted 3 lines

What are you on about ? The problem is that in the two traces his serverworks has moved from ide0 to ide2.

Here's the other case on another box that has onboard Promise plus an add-on Promise card:

2.6.9:

ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PDC20267: IDE controller at PCI slot 0000:01:02.0
PDC20267: chipset revision 2
PDC20267: ROM enabled at 0xfebd0000
PDC20267: 100% native mode on irq 27
PDC20267: (U)DMA Burst Bit ENABLED Primary PCI Mode Secondary PCI Mode.

    ide0: BM-DMA at 0xef00-0xef07, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0xef08-0xef0f, BIOS settings: hdc:pio, hdd:pio

Probing IDE interface ide0...
Probing IDE interface ide1...
PDC20267: IDE controller at PCI slot 0000:00:03.0
PDC20267: chipset revision 2
PDC20267: ROM enabled at 0xfeae0000
PDC20267: 100% native mode on irq 31
PDC20267: (U)DMA Burst Bit ENABLED Primary MASTER Mode Secondary MASTER Mode.

    ide2: BM-DMA at 0xdf00-0xdf07, BIOS settings: hde:pio, hdf:pio
    ide3: BM-DMA at 0xdf08-0xdf0f, BIOS settings: hdg:pio, hdh:pio

Probing IDE interface ide2...
hde: Maxtor 5T040H4, ATA DISK drive
ide2 at 0xdfe0-0xdfe7,0xdfae on irq 31
Probing IDE interface ide3...
hdg: Maxtor 5T040H4, ATA DISK drive
ide3 at 0xdfa0-0xdfa7,0xdfaa on irq 31
SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1 SvrWks OSB4: chipset revision 0
SvrWks OSB4: not 100% native mode: will probe irqs later

    ide4: BM-DMA at 0xffa0-0xffa7, BIOS settings: hdi:pio, hdj:pio
    ide5: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdk:pio, hdl:pio

Probing IDE interface ide4...
Probing IDE interface ide5...
Probing IDE interface ide0...
Probing IDE interface ide1...
Probing IDE interface ide4...
Probing IDE interface ide5...
hde: max request size: 128KiB
hde: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hde: cache flushes not supported
hde: hde1 hde2 hde3 hde4 < hde5 >
hdg: max request size: 128KiB
hdg: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hdg: cache flushes not supported

2.6.8-1.521smp (FC2):

Oct 14 04:46:49 spf0 kernel: ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Oct 14 04:46:49 spf0 kernel: PDC20267: IDE controller at PCI slot 0000:01:02.0
Oct 14 04:46:49 spf0 kernel: PDC20267: chipset revision 2
Oct 14 04:46:49 spf0 kernel: PDC20267: ROM enabled at 0xfebd0000
Oct 14 04:46:49 spf0 kernel: PDC20267: 100%% native mode on irq 153
Oct 14 04:46:49 spf0 kernel: PDC20267: (U)DMA Burst Bit ENABLED Primary PCI Mode Secondary PCI Mode.
Oct 14 04:46:49 spf0 kernel:     ide2: BM-DMA at 0xef00-0xef07, BIOS settings: hde:pio, hdf:pio
Oct 14 04:46:49 spf0 kernel:     ide3: BM-DMA at 0xef08-0xef0f, BIOS settings: hdg:pio, hdh:pio
Oct 14 04:46:49 spf0 kernel: PDC20267: IDE controller at PCI slot 0000:00:03.0
Oct 14 04:46:49 spf0 kernel: PDC20267: chipset revision 2
Oct 14 04:46:49 spf0 kernel: PDC20267: ROM enabled at 0xfeae0000
Oct 14 04:46:49 spf0 kernel: PDC20267: 100%% native mode on irq 161
Oct 14 04:46:49 spf0 kernel: PDC20267: (U)DMA Burst Bit ENABLED Primary MASTER Mode Secondary MASTER Mode.
Oct 14 04:46:49 spf0 kernel:     ide4: BM-DMA at 0xdf00-0xdf07, BIOS settings: hdi:pio, hdj:pio
Oct 14 04:46:49 spf0 kernel:     ide5: BM-DMA at 0xdf08-0xdf0f, BIOS settings: hdk:pio, hdl:pio
Oct 14 04:46:49 spf0 kernel: hdi: Maxtor 5T040H4, ATA DISK drive
Oct 14 04:46:49 spf0 kernel: Using cfq io scheduler
Oct 14 04:46:49 spf0 kernel: ide4 at 0xdfe0-0xdfe7,0xdfae on irq 161
Oct 14 04:46:49 spf0 kernel: hdk: Maxtor 5T040H4, ATA DISK drive
Oct 14 04:46:49 spf0 kernel: ide5 at 0xdfa0-0xdfa7,0xdfaa on irq 161
Oct 14 04:46:49 spf0 kernel: SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1
Oct 14 04:46:49 spf0 kernel: SvrWks OSB4: chipset revision 0
Oct 14 04:46:49 spf0 kernel: SvrWks OSB4: not 100%% native mode: will probe irqs later
Oct 14 04:46:49 spf0 kernel:     ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hda:pio, hdb:pio
Oct 14 04:46:49 spf0 kernel:     ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdc:pio, hdd:pio
Oct 14 04:46:49 spf0 kernel: hdi: max request size: 128KiB
Oct 14 04:46:49 spf0 kernel: hdi: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
Oct 14 04:46:49 spf0 kernel:  hdi: hdi1 hdi2 hdi3 hdi4 < hdi5 >
Oct 14 04:46:49 spf0 kernel: hdk: max request size: 128KiB
Oct 14 04:46:49 spf0 kernel: hdk: 80043264 sectors (40982 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
Oct 14 04:46:49 spf0 kernel:  hdk: hdk1 hdk2 hdk3 hdk4 < hdk5 >

Bartlomiej Zolnierkiewicz

2004-11-03 13:00 UTC #10 lore raw

Re: linux 2.6.9 controller order reordered. was: [Re: linux 2.6.9 still having network code problems]

Do you have CONFIG_BLK_DEV_OFFBOARD=y in your config?

Quoted 3 lines · Christopher Chan

Quoted 3 lines
What are you on about ? The problem is that in the two traces his serverworks has moved from ide0 to ide2.

Christopher Chan

author

2004-11-03 15:46 UTC #11 lore raw

Re: linux 2.6.9 controller order reordered. was: [Re: linux 2.6.9 still having network code problems]

Quoted 1 line · Bartlomiej Zolnierkiewicz

Do you have CONFIG_BLK_DEV_OFFBOARD=y in your config?

Yes....

Quoted 7 lines · Christopher Chan

Quoted 3 lines
Quoted 3 lines
What are you on about ? The problem is that in the two traces his serverworks has moved from ide0 to ide2.

Bartlomiej Zolnierkiewicz

2004-11-03 17:01 UTC #12 lore raw

Re: linux 2.6.9 controller order reordered. was: [Re: linux 2.6.9 still having network code problems]

Quoted 4 lines · Christopher Chan

Quoted 1 line · Bartlomiej Zolnierkiewicz
Do you have CONFIG_BLK_DEV_OFFBOARD=y in your config?
Yes....

This explains the issue...

Quoted 8 lines

Quoted 6 lines · Christopher Chan
Quoted 3 lines
Quoted 3 lines
What are you on about ? The problem is that in the two traces his serverworks has moved from ide0 to ide2.

Christopher Chan

author

2004-11-04 01:08 UTC #13 lore raw

In reply to Christopher Chan

Quoted 22 lines · Christopher Chan

Quoted 22 lines

Quoted 19 lines

Previously with 2.6.7 I had to use this values in sysctl to be able to continue to access the box:

net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.route.gc_thresh = 65536
net.ipv4.route.max_size = 1048576

Nov  1 03:16:00 spf5-3 kernel: Out of Memory: Killed process 21861 (smtpd).
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed at net/ipv4/tcp.c (1348)
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed at net/ipv4/tcp.c (1348)
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  1 03:16:00 spf5-3 kernel: recvmsg bug: copied 1E619F78 seq 1E61A378
Nov  1 03:16:00 spf5-3 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed at net/ipv4/tcp.c (1348)

Quoted 9 lines

Quoted 2 lines
This is a networking bug.
Yes...it's the same one that won't allow me to use the e100 driver with NAPI enabled with or without them sysctl tweaks and the same one that forces me to use the tweaks without NAPI enabled for the e100 driver.
With NAPI enabled, I see the problem when under pressure.
FYI, the 2.6.9 kernel is running with NAPI enabled.

2.6.9 kernel with NAPI enabled and sysctl tweaks listed at the top of mail still triggers network code problems:

Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed a
t net/ipv4/tcp.c (1348)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed a
t net/ipv4/tcp.c (1348)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed a
t net/ipv4/tcp.c (1348)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed a
t net/ipv4/tcp.c (1348)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed a
t net/ipv4/tcp.c (1348)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed a
t net/ipv4/tcp.c (1348)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed a
t net/ipv4/tcp.c (1348)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed a
t net/ipv4/tcp.c (1348)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:46:45 spf0 kernel: KERNEL: assertion (tp->copied_seq == tp->rcv_nxt || (flags & (MSG_PEEK | MSG_TRUNC))) failed a
t net/ipv4/tcp.c (1348)
Nov  3 10:47:56 spf0 kernel: kb)->end_seq)) failed at net/ipv4/tcp.c (1061)
Nov  3 10:47:56 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:47:56 spf0 kernel: KERNEL: assertion (!skb || before(tp->copied_seq, TCP_SKB_CB(skb)->end_seq)) failed at net/ipv
4/tcp.c (1061)
......................................
Nov  3 10:47:59 spf0 kernel: printk: 156 messages suppressed.
Nov  3 10:47:59 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:47:59 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:47:59 spf0 kernel: KERNEL: assertion (!skb || before(tp->copied_seq, TCP_SKB_CB(skb)->end_seq)) failed at net/ipv
4/tcp.c (1061)
Nov  3 10:47:59 spf0 kernel: printk: 106 messages suppressed.
Nov  3 10:47:59 spf0 kernel: printk: 121 messages suppressed.
Nov  3 10:47:59 spf0 kernel: printk: 222 messages suppressed.
Nov  3 10:47:59 spf0 kernel: printk: 151 messages suppressed.
Nov  3 10:47:59 spf0 kernel: KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c (1284)
Nov  3 10:47:59 spf0 kernel: recvmsg bug: copied D51512C9 seq D51512FA
Nov  3 10:47:59 spf0 kernel: printk: 163 messages suppressed.
Nov  3 10:48:04 spf0 kernel: printk: 216 messages suppressed.

apparent recovery...

Nov  3 11:15:44 spf0 kernel: printk: 15 messages suppressed.
Nov  3 11:23:42 spf0 kernel: printk: 61 messages suppressed.

Box is gone...

Please continue to keep available the NAPI disable/enable option in the e100 driver.