Skip to contentSkip to conversation

Promise SX8 driver performance

DISCUSSION Matt 3 participants 7 messages Nov 29, 2004 – Mar 26, 2005

lore
Share & tools
lore

Matt

author
#0 lore raw

Hello all. I've got a Promise SX8 SATA controller and the performance seems a little off. I'm using a debian-built 2.6.9 kernel and the read/write I/O seems to level off around 80-90 MB/s. When I use the same hardware in Windows, sequential I/O goes up to 218 MB/s. (That's about the limit of my motherboard's buses.) If I move some disks to the on-board SATA controller, I can get the I/O to go higher but never as high as in Windows.

Is this a problem with my expectations, my testing, or the driver?

Some Bonnie numbers for a raid 0 array of Seagate 7200.7's (2 GB file, sequential, MB/s):

Disks/cntl        Write    Read
-----             -----    ----
1 (sx8)              57      63
2 (sx8)              69      75

2 (ESB) 111 121
3 (sx8) 75 79
3 (1 sx8, 2 ESB) 141 125
4 (sx8) 89 80
4 (2 sx8, 2 ESB) 135 150

5 (sx8)              88      79
6 (sx8)              91      75

Some numbers from Sandra (ugh) in W2K Pro with the latest Promise drivers and using software RAID 0 (sequential, MB/s):

Disks/cntl      Write    Read
-----           -----    ----
1 (sx8)            56      62
2 (sx8)           106     122
3 (sx8)           151     176
4 (sx8)           178     209
5 (sx8)           180     218
6 (sx8)           179     218
6 (4 sx8, 2 ESB)  169     207

My full hardware:

Supermicro P4SCi motherboard (7210 chipset, 6300ESB SATA on-board) 3.0 GHz P4 Northwood w/ HT enabled
(2) 512MB DDR400 dual channel (non-ECC)
Promise SX8 SATA controller
(6) 200 GB Seagate 7200.7
(1) 40 GB Seagate Baraccuda IV (boot)
Adaptec 2940U

#1 lore raw

Promise SX8 performance issues and CARM_MAX_Q

Hi,

Playing with a recently acquired Promise SX8 card, we've found similar performance results to Matt's post to lkml a few months back at http://marc.theaimsgroup.com/?l=linux-kernel&m=110175890323356&w=2

It appears that the driver is only submitting one command at a time per port, which is at least one cause of the slowdowns. By raising CARM_MAX_Q from 1 to 3 in drivers/block/sx8.c (it was 3 in an earlier pre-merge incarnation of carmel.c), we're getting very notable speed improvements, with no side effects just yet.

Knowing very little about what this change has actually done, I've a few questions:

- Should this be considered dangerous?
- Why was it taken from 3 to 1?
- Is CARM_MAX_Q a number defined (or limited) by the hardware?

Thanks in advance,

Bernard.

Signature · 1 line
 Bernard Blackham <bernard at blackham dot com dot au>
#2 lore raw

Re: Promise SX8 performance issues and CARM_MAX_Q

Quoted 12 lines · Bernard Blackham

Hi,

Playing with a recently acquired Promise SX8 card, we've found similar performance results to Matt's post to lkml a few months back at http://marc.theaimsgroup.com/?l=linux-kernel&m=110175890323356&w=2

It appears that the driver is only submitting one command at a time per port, which is at least one cause of the slowdowns. By raising CARM_MAX_Q from 1 to 3 in drivers/block/sx8.c (it was 3 in an earlier pre-merge incarnation of carmel.c), we're getting very notable speed improvements, with no side effects just yet.

Quoted 6 lines

Knowing very little about what this change has actually done, I've a few questions:

- Should this be considered dangerous?
- Why was it taken from 3 to 1?
- Is CARM_MAX_Q a number defined (or limited) by the hardware?

In multi-port stress tests, we couldn't get SX8 to function reliably without locking up or corrupting data, with more than one outstanding command.

Maybe a new firmware has solved this by now.

Jeff

#3 lore raw

Re: Promise SX8 performance issues and CARM_MAX_Q

Quoted 21 lines · Jeff Garzik
Quoted 12 lines · Bernard Blackham

Hi,

Playing with a recently acquired Promise SX8 card, we've found similar performance results to Matt's post to lkml a few months back at http://marc.theaimsgroup.com/?l=linux-kernel&m=110175890323356&w=2

It appears that the driver is only submitting one command at a time per port, which is at least one cause of the slowdowns. By raising CARM_MAX_Q from 1 to 3 in drivers/block/sx8.c (it was 3 in an earlier pre-merge incarnation of carmel.c), we're getting very notable speed improvements, with no side effects just yet.

Quoted 5 lines

Knowing very little about what this change has actually done, I've a few questions:
- Should this be considered dangerous?
- Why was it taken from 3 to 1?
- Is CARM_MAX_Q a number defined (or limited) by the hardware?

Quoted 8 lines

In multi-port stress tests, we couldn't get SX8 to function reliably without locking up or corrupting data, with more than one outstanding command.

Maybe a new firmware has solved this by now.

Jeff

Indeed there does seem to be new firmware out as of 2/23/05. I ran my tests with the 9/10/04 firmware but I did not adjust the CARM_MAX_Q value. Do either of you happen to know what firmware revisions you tested under?

I've put the machine with the SX8 controller into production despite the performance issues so I'm not going to be of any use for testing revisions to the driver :(

Matt

#4 lore raw

Re: Promise SX8 performance issues and CARM_MAX_Q

Quoted 28 lines · Matthew Collins
Quoted 26 lines · Jeff Garzik
Quoted 12 lines · Bernard Blackham

Hi,

Playing with a recently acquired Promise SX8 card, we've found similar performance results to Matt's post to lkml a few months back at http://marc.theaimsgroup.com/?l=linux-kernel&m=110175890323356&w=2

It appears that the driver is only submitting one command at a time per port, which is at least one cause of the slowdowns. By raising CARM_MAX_Q from 1 to 3 in drivers/block/sx8.c (it was 3 in an earlier pre-merge incarnation of carmel.c), we're getting very notable speed improvements, with no side effects just yet.

Quoted 5 lines

Knowing very little about what this change has actually done, I've a few questions:
- Should this be considered dangerous?
- Why was it taken from 3 to 1?
- Is CARM_MAX_Q a number defined (or limited) by the hardware?

In multi-port stress tests, we couldn't get SX8 to function reliably without locking up or corrupting data, with more than one outstanding command.

Maybe a new firmware has solved this by now.

Quoted 7 lines
Quoted 4 lines

Jeff

Indeed there does seem to be new firmware out as of 2/23/05. I ran my tests with the 9/10/04 firmware but I did not adjust the CARM_MAX_Q value. Do either of you happen to know what firmware revisions you tested under?

I've put the machine with the SX8 controller into production despite the performance issues so I'm not going to be of any use for testing revisions to the driver :(

The driver was developed on a pre-production board, so its entirely possible Promise fixed this issue.

The driver should be solid for _at least_ CARM_MAX_Q==31, presuming that the firmware doesn't choke.

Anybody willing to
(a) change CARM_MAX_Q
(b) use the latest firmware
(c) stress the card with fsx, badblocks, iozone, and other tools on -multiple ports- simultaneously.

?

Jeff

#5 lore raw

Re: Promise SX8 performance issues and CARM_MAX_Q

Quoted 4 lines · Jeff Garzik
Quoted 4 lines

Indeed there does seem to be new firmware out as of 2/23/05. I ran my tests with the 9/10/04 firmware but I did not adjust the CARM_MAX_Q value. Do either of you happen to know what firmware revisions you tested under?

BIOS boot messages didn't seem to indicate a firmware version, but this card was only available in stock in early March, so its fairly likely to have the latest firmware (2/23/05).

Quoted 2 lines

The driver should be solid for _at least_ CARM_MAX_Q==31, presuming that the firmware doesn't choke.

We've run it here with CARM_MAX_Q = 32 with a custom stress testing script and haven't had any incidents so far. CARM_MAX_Q = 33 goes bad instantly.

Quoted 2 lines

(c) stress the card with fsx, badblocks, iozone, and other tools on -multiple ports- simultaneously.

I'll give those a workout over the weekend and let you know the results.

Thanks,

Bernard.

Signature · 1 line
 Bernard Blackham <bernard at blackham dot com dot au>
#6 lore raw

Re: Promise SX8 performance issues and CARM_MAX_Q

In reply to Jeff Garzik
Quoted 5 lines · Jeff Garzik

The driver was developed on a pre-production board, so its entirely possible Promise fixed this issue.

The driver should be solid for _at least_ CARM_MAX_Q==31, presuming that the firmware doesn't choke.

Stressing this SX8 card broke with CARM_MAX_Q anything higher than
16. Tests included simultaneous iozones, bonnie++'s, fsx, badblocks, and mkfs'ing, on 4 ports. Setting CARM_MAX_Q to 17 or above resulted in errors such as:

Buffer I/O error on device sx80_0p1, logical block 108828 lost page write due to I/O error on sx80_0p1
end_request: I/O error, dev sx80_2, sector 389167

It's been thrashing away with various tests most of today with CARM_MAX_Q = 16, and it hasn't missed a beat.

Thanks must go to Matt Johnston who actually did all the work of giving the card and drives a hard time.

Bernard.

Signature · 1 line
 Bernard Blackham <bernard at blackham dot com dot au>