Skip to contentSkip to conversation

PROBLEM: Alpha SMP Low Outbound Bandwidth

DISCUSSION Jay Thorne 5 participants 11 messages May 25, 2001 – May 28, 2001

lore
Share & tools
lore

Jay Thorne

author
#0 lore raw

[1.] One line summary of the problem:
Kernel 2.4.4 ac15
Tested with several cards and pieces of software, the outbound bandwidth on a quad cpu alpha is 2 megabytes a second or less on a 100 mbit switched ethernet network. Other machines on same switch do 10 or more megabytes per second. Switch is DLink 3624, 24 port, only 12 ports in use.

[2.] Full description of the problem/report:
Using a quad 400Mhz Dodge/Rawhide machine with Tulip or VIARhine cards, on wuFTP, the outbound bandwidth tops out at 2 megabytes per second and the inbound at 6 megabytes per second. Also noticeable are apparent slowdowns or console lockups/sluggishness during the transfer.

[3.] Keywords (i.e., modules, networking, kernel):
networking, alpha, tulip, via_rhine

[4.] Kernel version (from /proc/version):
Linux version 2.4.4-ac15 (root@lister) (gcc version 2.96 20000731 (SuSE Linux 7.1/Alpha)) #1 SMP Thu May 24 18:41:13 PDT 2001

[5.] Output of Oops.. message (if applicable) with symbolic information
resolved (see Documentation/oops-tracing.txt)

[6.] A small shell script or example program which triggers the
problem (if possible)

Problem machine:
ncftp /tmp > put foo
foo: 34.38 MB 5.16
MB/s
ncftp /tmp > get -z foo baz
baz: 34.38 MB 1.16
MB/s

other machine on same switch to same ftp server.
ncftp /home/jay > get foo
foo: 34.38 MB 10.12
MB/s
ncftp /home/jay > put -z foo baz
foo: 34.38 MB 9.93
MB/s

[7.] Environment
[7.1.] Software (add the output of the ver_linux script here)

Linux lister 2.4.4-ac15 #1 SMP Thu May 24 18:41:13 PDT 2001 alpha unknown

Gnu C                  2.96
Gnu make               3.79.1
binutils               2.10.0.33
util-linux             2.10q
mount                  2.10q
modutils               2.4.2
e2fsprogs              1.19
pcmcia-cs              3.1.22
PPP                    2.4.0
isdn4k-utils           3.1pre1a
Linux C Library        so.6.1
Dynamic linker (ldd)   2.2
Procps                 2.0.7
Net-tools              1.57
Kbd                    1.02
Sh-utils               2.0
Modules Loaded         tulip via-rhine

[7.2.] Processor information (from /proc/cpuinfo):
lister:/usr/src/linux # cat /proc/cpuinfo
cpu : Alpha
cpu model : EV56
cpu variation : 7
cpu revision : 0

cpu serial number       :
system type             : Rawhide
system variation        : Dodge
system revision         : 0
system serial number    : NI70904KB0
cycle frequency [Hz]    : 400000000
timer frequency [Hz]    : 1200.00
page size [bytes]       : 8192
phys. address bits      : 40
max. addr. space #      : 127
BogoMIPS                : 738.12
kernel unaligned acc    : 1646246

(pc=fffffc000042a3d8,va=fffffc005d9b784e)
user unaligned acc : 0 (pc=0,va=0)
platform string : AlphaServer 4100 5/400 4MB
cpus detected : 4
cpus active : 4
cpu active mask : 000000000000000f

[7.3.] Module information (from /proc/modules):
lister:/usr/src/linux # cat /proc/modules

tulip                  59296   1
via-rhine              16464   0 (autoclean)

[7.4.] Loaded driver and hardware information (/proc/ioports, /proc/iomem)
lister:/usr/src/linux # cat /proc/ioports 00000000-0000ffff : PCI IO bus 0
00000000-0000001f : dma1
00000020-0000003f : pic1
00000040-0000005f : timer
00000060-0000006f : keyboard
00000070-00000080 : rtc

    00000070-0000007f : rtc
  000000a0-000000bf : pic2
  000000c0-000000df : dma2
  000002f8-000002ff : serial(auto)
  000003f8-000003ff : serial(auto)
  00008000-000080ff : VIA Technologies, Inc. Ethernet Controller
    00008000-000080ff : via-rhine
  00008400-0000847f : Digital Equipment Corporation DECchip 21140
[FasterNet]
    00008400-0000847f : tulip
200000000-20000ffff : PCI IO bus 1
  200008000-2000080ff : Symbios Logic Inc. (formerly NCR) 53c810
    200008000-20000807f : ncr53c8xx

200009000-2000090fe : qlogicisp
lister:/usr/src/linux # cat /proc/iomem
00000000-ffffffff : PCI mem bus 0
00000000-07ffffff : HAE0

    02200000-0223ffff : Digital Equipment Corporation DECchip 21140
[FasterNet]
    02240000-0224ffff : S3 Inc. 86c764/765 [Trio32/64/64V+]
    02250000-0225ffff : VIA Technologies, Inc. Ethernet Controller
    02260000-022600ff : VIA Technologies, Inc. Ethernet Controller
      02260000-022600ff : via-rhine
    02261000-0226107f : Digital Equipment Corporation DECchip 21140
[FasterNet]
      02261000-0226107f : tulip
200000000-2ffffffff : PCI mem bus 1
  200000000-207ffffff : HAE0
    202200000-2022000ff : Symbios Logic Inc. (formerly NCR) 53c810

[7.5.] PCI information ('lspci -vvv' as root) lister:/usr/src/linux # lspci -vvv
00:01.0 Non-VGA unclassified device: Intel Corporation 82375EB (rev 05)
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 32
Interrupt: pin ? routed to IRQ 32

00:02.0 VGA compatible controller: S3 Inc. 86c764/765 [Trio32/64/64V+] (rev 54) (prog-if 00 [VGA])
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at 000000000c000000 (32-bit, non-prefetchable)
        Expansion ROM at 0000000002240000 [size=64K]

00:03.0 Ethernet controller: Digital Equipment Corporation DECchip 21140 [FasterNet] (rev 22)
Subsystem: Digital Equipment Corporation: Unknown device 500a
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

        Latency: 32 (5000ns min, 10000ns max), cache line size 10
        Interrupt: pin A routed to IRQ 20
        Region 0: I/O ports at 8400 [size=128]
        Region 1: Memory at 0000000002261000 (32-bit, non-prefetchable)
[size=128]
        Expansion ROM at 0000000002200000 [disabled] [size=256K]

00:05.0 Ethernet controller: VIA Technologies, Inc. Ethernet Controller (rev 42)
Subsystem: D-Link System Inc: Unknown device 1400
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping+ SERR+ FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

        Latency: 32 (750ns min, 2000ns max), cache line size 10
        Interrupt: pin A routed to IRQ 28
        Region 0: I/O ports at 8000 [size=256]
        Region 1: Memory at 0000000002260000 (32-bit, non-prefetchable)

[size=256]
Expansion ROM at 0000000002250000 [disabled] [size=64K]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

01:01.0 SCSI storage controller: Symbios Logic Inc. (formerly NCR) 53c810 (rev 02)
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

        Latency: 32
        Interrupt: pin A routed to IRQ 56
        Region 0: I/O ports at 200008000 [size=256]
        Region 1: Memory at 0000000202200000 (32-bit, non-prefetchable)

[size=256]

01:02.0 PCI bridge: Digital Equipment Corporation DECchip 21050 (rev 02) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 32, cache line size 10
Bus: primary=01, secondary=02, subordinate=02, sec-latency=0 I/O behind bridge: 00009000-00009fff
Memory behind bridge: 02300000-023fffff
Prefetchable memory behind bridge: 00100000-000fffff
BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-

02:00.0 SCSI storage controller: Q Logic ISP1020 (rev 02)
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 32, cache line size 10
Interrupt: pin A routed to IRQ 40
Region 0: I/O ports at 200009000 [size=256]
Region 1: Memory at 0000000202310000 (32-bit, non-prefetchable)
[size=4K]

        Expansion ROM at 0000000202300000 [disabled] [size=64K]

[7.6.] SCSI information (from /proc/scsi/scsi)
lister:/usr/src/linux # cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: DEC Model: RZ1CB-CS (C) DEC Rev: 0844
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 01 Lun: 00
Vendor: DEC Model: RZ1CB-CA (C) DEC Rev: LYJ0
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 02 Lun: 00
Vendor: DEC Model: RZ1CB-CA (C) DEC Rev: LYJ0
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 03 Lun: 00
Vendor: DEC Model: RZ1CB-CA (C) DEC Rev: LYJ0
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi1 Channel: 00 Id: 05 Lun: 00
Vendor: DEC Model: RRD45 (C) DEC Rev: 0436
Type: CD-ROM ANSI SCSI revision: 02

[7.7.] Other information that might be relevant to the problem
(please look in /proc and include all information that you think to be relevant):

There is a tcpdump available. 4.6 megabytes for anyone who wants it.

[X.] Other notes, patches, fixes, workarounds:

During one of the slowdowns, the console appeared to lock up. holding down the ALT key seemed to make it wake up.

Signature · 2 lines
--
Jay Thorne Manager, Systems & Technology, UserFriendly Media, Inc.
#1 lore raw

Hello Jay,

I see that you are using the tulip driver. Could you try the de4x5 driver??

Best Regards,

--George

Quoted 256 lines · Jay Thorne

[1.] One line summary of the problem:
Kernel 2.4.4 ac15
Tested with several cards and pieces of software, the outbound bandwidth on a quad cpu alpha is 2 megabytes a second or less on a 100 mbit switched ethernet network. Other machines on same switch do 10 or more megabytes per second. Switch is DLink 3624, 24 port, only 12 ports in use.

[2.] Full description of the problem/report:
Using a quad 400Mhz Dodge/Rawhide machine with Tulip or VIARhine cards, on wuFTP, the outbound bandwidth tops out at 2 megabytes per second and the inbound at 6 megabytes per second. Also noticeable are apparent slowdowns or console lockups/sluggishness during the transfer.

[3.] Keywords (i.e., modules, networking, kernel):
networking, alpha, tulip, via_rhine

[4.] Kernel version (from /proc/version):
Linux version 2.4.4-ac15 (root@lister) (gcc version 2.96 20000731 (SuSE Linux 7.1/Alpha)) #1 SMP Thu May 24 18:41:13 PDT 2001

[5.] Output of Oops.. message (if applicable) with symbolic information
resolved (see Documentation/oops-tracing.txt)

[6.] A small shell script or example program which triggers the
problem (if possible)

Problem machine:
ncftp /tmp > put foo
foo: 34.38 MB 5.16
MB/s
ncftp /tmp > get -z foo baz
baz: 34.38 MB 1.16
MB/s

other machine on same switch to same ftp server.
ncftp /home/jay > get foo
foo: 34.38 MB 10.12
MB/s
ncftp /home/jay > put -z foo baz
foo: 34.38 MB 9.93
MB/s

[7.] Environment
[7.1.] Software (add the output of the ver_linux script here)

Linux lister 2.4.4-ac15 #1 SMP Thu May 24 18:41:13 PDT 2001 alpha unknown

Gnu C                  2.96
Gnu make               3.79.1
binutils               2.10.0.33
util-linux             2.10q
mount                  2.10q
modutils               2.4.2
e2fsprogs              1.19
pcmcia-cs              3.1.22
PPP                    2.4.0
isdn4k-utils           3.1pre1a
Linux C Library        so.6.1
Dynamic linker (ldd)   2.2
Procps                 2.0.7
Net-tools              1.57
Kbd                    1.02
Sh-utils               2.0
Modules Loaded         tulip via-rhine

[7.2.] Processor information (from /proc/cpuinfo):
lister:/usr/src/linux # cat /proc/cpuinfo
cpu : Alpha
cpu model : EV56
cpu variation : 7
cpu revision : 0

cpu serial number       :
system type             : Rawhide
system variation        : Dodge
system revision         : 0
system serial number    : NI70904KB0
cycle frequency [Hz]    : 400000000
timer frequency [Hz]    : 1200.00
page size [bytes]       : 8192
phys. address bits      : 40
max. addr. space #      : 127
BogoMIPS                : 738.12
kernel unaligned acc    : 1646246

(pc=fffffc000042a3d8,va=fffffc005d9b784e)
user unaligned acc : 0 (pc=0,va=0)
platform string : AlphaServer 4100 5/400 4MB
cpus detected : 4
cpus active : 4
cpu active mask : 000000000000000f

[7.3.] Module information (from /proc/modules):
lister:/usr/src/linux # cat /proc/modules

tulip                  59296   1
via-rhine              16464   0 (autoclean)

[7.4.] Loaded driver and hardware information (/proc/ioports, /proc/iomem)
lister:/usr/src/linux # cat /proc/ioports 00000000-0000ffff : PCI IO bus 0
00000000-0000001f : dma1
00000020-0000003f : pic1
00000040-0000005f : timer
00000060-0000006f : keyboard
00000070-00000080 : rtc

    00000070-0000007f : rtc
  000000a0-000000bf : pic2
  000000c0-000000df : dma2
  000002f8-000002ff : serial(auto)
  000003f8-000003ff : serial(auto)
  00008000-000080ff : VIA Technologies, Inc. Ethernet Controller
    00008000-000080ff : via-rhine
  00008400-0000847f : Digital Equipment Corporation DECchip 21140
[FasterNet]
    00008400-0000847f : tulip
200000000-20000ffff : PCI IO bus 1
  200008000-2000080ff : Symbios Logic Inc. (formerly NCR) 53c810
    200008000-20000807f : ncr53c8xx

200009000-2000090fe : qlogicisp
lister:/usr/src/linux # cat /proc/iomem
00000000-ffffffff : PCI mem bus 0
00000000-07ffffff : HAE0

    02200000-0223ffff : Digital Equipment Corporation DECchip 21140
[FasterNet]
    02240000-0224ffff : S3 Inc. 86c764/765 [Trio32/64/64V+]
    02250000-0225ffff : VIA Technologies, Inc. Ethernet Controller
    02260000-022600ff : VIA Technologies, Inc. Ethernet Controller
      02260000-022600ff : via-rhine
    02261000-0226107f : Digital Equipment Corporation DECchip 21140
[FasterNet]
      02261000-0226107f : tulip
200000000-2ffffffff : PCI mem bus 1
  200000000-207ffffff : HAE0
    202200000-2022000ff : Symbios Logic Inc. (formerly NCR) 53c810

[7.5.] PCI information ('lspci -vvv' as root) lister:/usr/src/linux # lspci -vvv
00:01.0 Non-VGA unclassified device: Intel Corporation 82375EB (rev 05)
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 32
Interrupt: pin ? routed to IRQ 32

00:02.0 VGA compatible controller: S3 Inc. 86c764/765 [Trio32/64/64V+] (rev 54) (prog-if 00 [VGA])
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at 000000000c000000 (32-bit, non-prefetchable)
        Expansion ROM at 0000000002240000 [size=64K]

00:03.0 Ethernet controller: Digital Equipment Corporation DECchip 21140 [FasterNet] (rev 22)
Subsystem: Digital Equipment Corporation: Unknown device 500a
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

        Latency: 32 (5000ns min, 10000ns max), cache line size 10
        Interrupt: pin A routed to IRQ 20
        Region 0: I/O ports at 8400 [size=128]
        Region 1: Memory at 0000000002261000 (32-bit, non-prefetchable)
[size=128]
        Expansion ROM at 0000000002200000 [disabled] [size=256K]

00:05.0 Ethernet controller: VIA Technologies, Inc. Ethernet Controller (rev 42)
Subsystem: D-Link System Inc: Unknown device 1400
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping+ SERR+ FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

        Latency: 32 (750ns min, 2000ns max), cache line size 10
        Interrupt: pin A routed to IRQ 28
        Region 0: I/O ports at 8000 [size=256]
        Region 1: Memory at 0000000002260000 (32-bit, non-prefetchable)

[size=256]
Expansion ROM at 0000000002250000 [disabled] [size=64K]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

01:01.0 SCSI storage controller: Symbios Logic Inc. (formerly NCR) 53c810 (rev 02)
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

        Latency: 32
        Interrupt: pin A routed to IRQ 56
        Region 0: I/O ports at 200008000 [size=256]
        Region 1: Memory at 0000000202200000 (32-bit, non-prefetchable)

[size=256]

01:02.0 PCI bridge: Digital Equipment Corporation DECchip 21050 (rev 02) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 32, cache line size 10
Bus: primary=01, secondary=02, subordinate=02, sec-latency=0 I/O behind bridge: 00009000-00009fff
Memory behind bridge: 02300000-023fffff
Prefetchable memory behind bridge: 00100000-000fffff
BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-

02:00.0 SCSI storage controller: Q Logic ISP1020 (rev 02)
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 32, cache line size 10
Interrupt: pin A routed to IRQ 40
Region 0: I/O ports at 200009000 [size=256]
Region 1: Memory at 0000000202310000 (32-bit, non-prefetchable)
[size=4K]

        Expansion ROM at 0000000202300000 [disabled] [size=64K]

[7.6.] SCSI information (from /proc/scsi/scsi)
lister:/usr/src/linux # cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: DEC Model: RZ1CB-CS (C) DEC Rev: 0844
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 01 Lun: 00
Vendor: DEC Model: RZ1CB-CA (C) DEC Rev: LYJ0
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 02 Lun: 00
Vendor: DEC Model: RZ1CB-CA (C) DEC Rev: LYJ0
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 03 Lun: 00
Vendor: DEC Model: RZ1CB-CA (C) DEC Rev: LYJ0
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi1 Channel: 00 Id: 05 Lun: 00
Vendor: DEC Model: RRD45 (C) DEC Rev: 0436
Type: CD-ROM ANSI SCSI revision: 02

[7.7.] Other information that might be relevant to the problem
(please look in /proc and include all information that you think to be relevant):

There is a tcpdump available. 4.6 megabytes for anyone who wants it.

[X.] Other notes, patches, fixes, workarounds:

During one of the slowdowns, the console appeared to lock up. holding down the ALT key seemed to make it wake up.

author
#2 lore raw
Quoted 4 lines · George France

Hello Jay,

I see that you are using the tulip driver. Could you try the de4x5 driver??

Its worse: reports 3.1 MBs and 1.6 MBs

Signature · 2 lines
--
Jay Thorne Manager, Systems & Technology, UserFriendly Media, Inc.
#3 lore raw
In reply to Jay Thorne
Quoted 7 lines · Jay Thorne
Quoted 4 lines · George France

Hello Jay,

I see that you are using the tulip driver. Could you try the de4x5 driver??

Its worse: reports 3.1 MBs and 1.6 MBs

wuftp is not exactly a performance benchmark, have you tried 'netperf'?

--George

author
#4 lore raw
Quoted 4 lines · George France
Quoted 3 lines · Jay Thorne
Quoted 2 lines · George France

Hello Jay,

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

I see that you are using the tulip driver. Could you try the de4x5 driver??

Its worse: reports 3.1 MBs and 1.6 MBs

wuftp is not exactly a performance benchmark, have you tried 'netperf'?

--George

While I agree with you completely that wuftpd is not exactly a performance leader, this is the simplest way to recreate a problem I was having with a much more complex setup involving apache and SMP and a whole bunch of things.

I posted 2 weeks ago and got no response, I assume because everyone thought it was my software. After reducing the problem to eliminate the possibility that my code is the real problem, I'm left with a quite repeatable state. I have two nearly identical machines, one with 466 mhz cpus the other with 400mhz, and they both do the same thing. The via-rhine performs similarly to the de4x5.

Netperf is a pretty good idea. Should not be a cpu bottleneck. Thats a good thing. So pretty much the same results as wu-ftpd: Note that I used the 466 mhz quad with a via-rhine, since the 400 locked up and was still fscking when I started this test.

             Recv   Send    Send                          
             Socket Socket  Message  Elapsed              
             Size   Size    Size     Time     Throughput  
             bytes  bytes   bytes    secs.    10^6bits/sec  
To alpha     87380  16384  16384    10.02      39.25   
x86 local    87380  16384  16384    9.99      559.46
alpha local  87380  16384  16384    10.01     547.27   
alp to x86   87380  16384  16384    10.01      25.77   
another x86  87380  16384  16384    9.99      553.67   
to same x86  87380  16384  16384    10.00      82.79   
and back     87380  16384  16384    10.00      93.89   

But Wu-ftpd is an easy to set up test bench, and is ubiquitous enough that anyone with an alpha running SMP can test it. Note that this software and the server in question were tested to run at 10+ megabytes per second with x86 boxes. The server is a PIII500 running 2.4.4, so its not like I'm comparing apples to oranges. The second x86 is an athlon 600.

So even factoring out wuftp is not helping much here. I'm fairly convinced that something is strange because after the de4x5 test, the box locked up. So either a> I have two identically boned 4 cpu boxen or b> the interprocessor/locking/resource management has some kind of problem. Note that under uniprocessor I get near identical to x86 performance, clock for clock and no lock ups.

Signature · 2 lines
--
Jay Thorne Manager, Systems & Technology, UserFriendly Media, Inc.
#5 lore raw
Quoted 10 lines · Jay Thorne

Netperf is a pretty good idea. Should not be a cpu bottleneck. Thats a good thing. So pretty much the same results as wu-ftpd: Note that I used the 466 mhz quad with a via-rhine, since the 400 locked up and was still fscking when I started this test.

             Recv   Send    Send                          
             Socket Socket  Message  Elapsed              
             Size   Size    Size     Time     Throughput  
             bytes  bytes   bytes    secs.    10^6bits/sec  
Quoted 7 lines
To alpha     87380  16384  16384    10.02      39.25   
x86 local    87380  16384  16384    9.99      559.46
alpha local  87380  16384  16384    10.01     547.27   
alp to x86   87380  16384  16384    10.01      25.77   
another x86  87380  16384  16384    9.99      553.67   
to same x86  87380  16384  16384    10.00      82.79   
and back     87380  16384  16384    10.00      93.89   

What type of NIC is in the x86'es? If they are not the same, what happens if you put one of those in the alphas?
(and what happens with the via-rhine if put in one of the x86'es?)

Alphas don't like unaligned memory accesses (not sure how bad those are). I think that you can get some idea on the frequence of those with 'cat /proc/cpuinfo' where it should say "kernel unaligned acc" and then some numbers.

If that first number keeps going up then the driver(s) or something isn't being nice. Well, just a though. Oh yes, your original message showed a value of 1646246 for kernel space unaligned accesses. Is that high?

The userspace value is 0, so I assume that is high. The value for 'pc=' should give the address of where the last unaligned access took place (look it up in System.map or /proc/ksyms)

Does the value grow faster when you run your netperf tests?

That still doesn't explain the SMP vs UP difference.

/Urban

#6 lore raw
In reply to Jay Thorne
Quoted 2 lines · Jay Thorne

But Wu-ftpd is an easy to set up test bench, and is ubiquitous enough that anyone with an alpha running SMP can test it. Note that this

My smp alpha box drives a single tulip over 12MB/sec in full duplex using tcp without any problem at all. So I definitely cannot reproduce. You may want to try to reproduce with 2.4.5pre6aa1 btw. If you've not tried it yet you can consider also using egcs 1.1.2 as compiler just in case.

You may also want to keep an eye on the VM, on alpha I see very weird things happening.

Andrea

#7 lore raw
In reply to Jay Thorne
Quoted 2 lines · Jay Thorne

[1.] One line summary of the problem:
Kernel 2.4.4 ac15

....

Quoted 1 line

Using a quad 400Mhz Dodge/Rawhide machine with Tulip or VIARhine cards,

....
[ description of a slowdown skipped ].

Well, it looks that you have at least something to slow down. I could not get a single packet through my tulip on Alpha from at least 2.4.4-ac11 and up. You can consider that an ultimate slowdown. I tried also a driver from http://sourceforge.net/projects/tulip/ and results are the same. This NIC, Digital DS21143 Tulip rev 65, works just fine with various earlier kernels, including assorted 2.4.3 variants. It is on 10baseT netwok - which may, or may not, be relevant here.

Michal

#8 lore raw
In reply to Jay Thorne

Hello Andrea,

Jay, if the problem still exist in 2.4.5-pre6aa1 (please try the new kernel), then I will have tech op's check this on Tuesday (Monday is a US holiday). We should be able to duplicate this in the hardware lab and find the problem with a logic analyser.

Best Regards,

--George

Quoted 19 lines · Andrea Arcangeli
Quoted 2 lines · Jay Thorne

But Wu-ftpd is an easy to set up test bench, and is ubiquitous enough that anyone with an alpha running SMP can test it. Note that this

My smp alpha box drives a single tulip over 12MB/sec in full duplex using tcp without any problem at all. So I definitely cannot reproduce. You may want to try to reproduce with 2.4.5pre6aa1 btw. If you've not tried it yet you can consider also using egcs 1.1.2 as compiler just in case.

You may also want to keep an eye on the VM, on alpha I see very weird things happening.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

author
#9 lore raw

Problem solved, thanks to the rawhide patch from Richard Henderson ([email protected]) posted on Sunday. Performance is ~10megs/second both directions, using tulip, de4x5 or via-rhine.

Using 2.4.4-ac15 it works fine. I'm now trying 2.4.5

Andrea, 2.4.5aa1 oopses just after probing the scsi cards. I've tried the 2.4.4 series aa patches and had similar failure on boot.

Its too fast to see the error, so I'm building a serial console version to capture it. Is an easy way to tell an alpha to stop dead so I can copy the oops?

Quoted 32 lines · George France

Hello Andrea,

Jay, if the problem still exist in 2.4.5-pre6aa1 (please try the new kernel), then I will have tech op's check this on Tuesday (Monday is a US holiday). We should be able to duplicate this in the hardware lab and find the problem with a logic analyser.

Best Regards,

--George

Quoted 19 lines · Andrea Arcangeli
Quoted 2 lines · Jay Thorne

But Wu-ftpd is an easy to set up test bench, and is ubiquitous enough that anyone with an alpha running SMP can test it. Note that this

My smp alpha box drives a single tulip over 12MB/sec in full duplex using tcp without any problem at all. So I definitely cannot reproduce. You may want to try to reproduce with 2.4.5pre6aa1 btw. If you've not tried it yet you can consider also using egcs 1.1.2 as compiler just in case.

You may also want to keep an eye on the VM, on alpha I see very weird things happening.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 2 lines
--
Jay Thorne Manager, Systems & Technology, UserFriendly Media, Inc.
#10 lore raw
In reply to Jay Thorne
Quoted 3 lines · Jay Thorne

Problem solved, thanks to the rawhide patch from Richard Henderson ([email protected]) posted on Sunday. Performance is ~10megs/second both directions, using tulip, de4x5 or via-rhine.

Well Done, Richard.

Quoted 9 lines

Using 2.4.4-ac15 it works fine. I'm now trying 2.4.5

Andrea, 2.4.5aa1 oopses just after probing the scsi cards. I've tried the 2.4.4 series aa patches and had similar failure on boot.

Its too fast to see the error, so I'm building a serial console version to capture it. Is an easy way to tell an alpha to stop dead so I can copy the oops?

try adding 'console=ttyS0,9600 console=tty0' to the comand line args passed to the kernel at boot time. if you are using SRM and aboot, 'b <dev> -fl i' followed by the 'l' command, then a 'b' command.

regards,

--George