Skip to contentSkip to conversation

Slow PCI-E speed

DISCUSSION [email protected] 1 participant 1 message Apr 13, 2011

lore
Share & tools
lore
#0 lore raw

Hi

Even if this is probably,even if I really think it's not, hardware related I'm putting this question out there cause I have no idea on what can be causing my problems.

Recently I've had the following cards in a HP DL145 G2

1x Myrinet 10G PCI-E 4x
2x Intel Quad 82546EB 1GE NIC
1x OCZ Revodrive 120GB

--

None of the cards above can deliver more than 375MB/s. The tests for the Myrinet-card is done using multiple wgets. For the Intel Quads using LACP and multiple wgets. The server I'm wget:ing from has shown performance >=500Mbit/s in the past. And for the Revodrive using dd/cat. Everything to /dev/null.

Because the DL145 G2 is quite old I thought it would be the chipsets etc being the bottleneck.

I then replaced the server with a Supermicro X7DBE (2x E5405 CPUs with 13 GB Ram) which is based on a Intel 5000P Chipset. The Revodrive card now sits in a dedicated PCI-E 4x slot connected to the 5000P chipset. The other cards sits on an Intel ESB2-chipset connected with a 10.7GB/s pipe to the 5000P (where the memory,cpu etc is).

The Revodrive should be able to handle ~500MB/s and well the Myrinet ~1000MB/s (atleast way more than 375MB/s). And the Intel Quads atleast more than 375MB/s when bonded..

But everything tops out at 375MB/s. And yes..375 = 3Gbit/s :)

--

Like I said this probably isn't a Linux kernel problem but I want to know if someone has come across something like this before and maybe maybe...there's some kernel setting that fixes it cause i'm clueless.

Thanks!

// Sebastian H