Skip to contentSkip to conversation

Poor I/O Performance with MegaRaid SATA 150-4; bug or feature?

DISCUSSION Matt M. Valites 2 participants 3 messages Apr 16, 2005 – Apr 17, 2005

lore
Share & tools
lore
#0 lore raw

Hail List,

I've been banging my head against this for a few days, and I wanted to see if anyone here could lend a hand.

I have the following configuration:
P4 3.x Ghz
2GB Ram;
2 x 36GB WD Raptors; in a RAID1 (sda)
2 x 74GB WD Raptor (those 10K RPM SATA drives) in a RAID1(sdb) Two free PCI-X slots, one of which occupied by a LSI MegaRaid SATA 150-4.

The problem is I/O on either one of these RAID devices seems to
be less than half what I'm expecting. The file system used in my testing is
XFS, and I'm running kernel 2.6.11.6.

The test I'm doing is a simple:

# time dd if=/dev/zero of=./crap.file bs=1024 count=209715
Which results in a runtime of about ~53s, in the best case, with all the
scary write cache enabled.    I've tried with deadline, and
anticipatory.  I've also tried several kernels, namely a recent 2.4, so
I could test megaraid and megaraid2, similar results.

On my desktop box, with one of these drives connected via SATA, i get ~25s, also XFS. (2.6.11-gentoo-r6 x86_64).

Is this an expected result? I'm seeing much higher numbers posted around the 'Net. Most of those results are from Windows boxes.

I've uploaded my kernel config, lspci -v, and two opreports of a bonnie++ run
to: http://www.muixa.com/lkml/

Any thoughts would be appreciated.
Thanks,
--
Matt M. Valites

#1 lore raw
Quoted 23 lines · Matt M. Valites

Hail List,

I've been banging my head against this for a few days, and I wanted to see if anyone here could lend a hand.

I have the following configuration:
P4 3.x Ghz
2GB Ram;
2 x 36GB WD Raptors; in a RAID1 (sda)
2 x 74GB WD Raptor (those 10K RPM SATA drives) in a RAID1(sdb) Two free PCI-X slots, one of which occupied by a LSI MegaRaid SATA 150-4.

The problem is I/O on either one of these RAID devices seems to
be less than half what I'm expecting. The file system used in my testing is
XFS, and I'm running kernel 2.6.11.6.

The test I'm doing is a simple:

# time dd if=/dev/zero of=./crap.file bs=1024 count=209715
Which results in a runtime of about ~53s, in the best case, with all the
scary write cache enabled.    I've tried with deadline, and
anticipatory.  I've also tried several kernels, namely a recent 2.4, so
I could test megaraid and megaraid2, similar results.
Quoted 8 lines

On my desktop box, with one of these drives connected via SATA, i get ~25s, also XFS. (2.6.11-gentoo-r6 x86_64).

Is this an expected result? I'm seeing much higher numbers posted around the 'Net. Most of those results are from Windows boxes.

I've uploaded my kernel config, lspci -v, and two opreports of a bonnie++ run
to: http://www.muixa.com/lkml/

I also have one of those cards, at home. I've come to the conclusion that they're just too old. No NCQ and such other fancy features (for gods sake, the controllers on the card are sil 3112's!). It's probably not even PCI-X native.

The only thing that can bring its performance reseanably up to speed is using write-back instead of write-through on the array. Also try enabling the write-cache on the drives (all doable in the cards bios config, not sure if this is what you meant with "with all the scary write cache enabled"). Doing this is on the other hand not very good for your data integrity, not good at all.

If only NCQ/TCQ was in, it would have a chance of having decent performance using write-through. A cool experiment would be setting up the drives as invidual drives on the card, and use md software raid over it.

Next time I'll probably just use md software raid over a 3ware 9xxx (JBOD-mode) or AHCI controller. I'm feeling quite uneasy about vendor lock in nowadays. Groan.

Signature · 2 lines
Cheers,
André Tomt
#2 lore raw
Quoted 51 lines · Andre Tomt
Quoted 28 lines · Matt M. Valites

Hail List,

I've been banging my head against this for a few days, and I wanted to see if anyone here could lend a hand.

I have the following configuration:
P4 3.x Ghz
2GB Ram;
2 x 36GB WD Raptors; in a RAID1 (sda)
2 x 74GB WD Raptor (those 10K RPM SATA drives) in a RAID1(sdb) Two free PCI-X slots, one of which occupied by a LSI MegaRaid SATA 150-4.

The problem is I/O on either one of these RAID devices seems to
be less than half what I'm expecting. The file system used in my
testing is
XFS, and I'm running kernel 2.6.11.6.

The test I'm doing is a simple:

# time dd if=/dev/zero of=./crap.file bs=1024 count=209715
Which results in a runtime of about ~53s, in the best case, with all the
scary write cache enabled.    I've tried with deadline, and
anticipatory.  I've also tried several kernels, namely a recent 2.4, so
I could test megaraid and megaraid2, similar results.

On my desktop box, with one of these drives connected via SATA, i get ~25s, also XFS. (2.6.11-gentoo-r6 x86_64).

Quoted 7 lines

Is this an expected result? I'm seeing much higher numbers posted around the
'Net. Most of those results are from Windows boxes.

I've uploaded my kernel config, lspci -v, and two opreports of a bonnie++ run
to: http://www.muixa.com/lkml/

I also have one of those cards, at home. I've come to the conclusion that they're just too old. No NCQ and such other fancy features (for gods sake, the controllers on the card are sil 3112's!). It's probably not even PCI-X native.

The only thing that can bring its performance reseanably up to speed is using write-back instead of write-through on the array. Also try enabling the write-cache on the drives (all doable in the cards bios config, not sure if this is what you meant with "with all the scary write cache enabled"). Doing this is on the other hand not very good for your data integrity, not good at all.

Quoted 9 lines

If only NCQ/TCQ was in, it would have a chance of having decent performance using write-through. A cool experiment would be setting up the drives as invidual drives on the card, and use md software raid over it.

Next time I'll probably just use md software raid over a 3ware 9xxx (JBOD-mode) or AHCI controller. I'm feeling quite uneasy about vendor lock in nowadays. Groan.

Andre,

Thanks for the reply.

I wouldn't have expected NCQ/TCQ to have such an effect on a direct I/O
test like dd. Even more disturbing, changing to a raid0 gives me
close to the same results.
When I enable write-cache, and "cached I/O" in the cards BIOS, i get
results closer to a fast 7200RPM disk. Still nothing like the
performance I know these raptors are capable of. Write-cache isn't an option for this machine, since it's slated to hold important version-control data :)

I could punt on this HW RAID idea, and just go with AHCI, but i've only
got two ports free on the board. So, i need at least two more ports to
pull it off, and I don't know of any SATA add-in card that has good
driver support... Have you tested the RAID1 throughput of the 3ware
9x cards?

(this is why I still like SCSI... )
--
Matt M. Valites