Skip to contentSkip to conversation

strange performance of raid0

DISCUSSION Keld Jørn Simonsen 4 participants 10 messages Feb 15, 2004 – Feb 17, 2004 MD RAID

lore
Share & tools
lore
#1 lore raw
In reply to Keld Jørn Simonsen

The you don't get as good performance because hdc1 is slow enough to really drag it down. Think about it, if reads are all going about sequentially, it has to spend alot of time waiting on that 16mb/s drive compared to the others.

You probably can't get 100mb/s with the two because that would be pretty efficient, and there's probably more overhead than that.

Shouldn't the 80gb drive be going faster than 16mb/s though? Have you checked hdparm to make sure dma and all the goodies are turned on for it?

Quoted 23 lines · Keld Jørn Simonsen

Hi

I have some strange performance results on a raid0
I have 4 IDE disks on two controllers, the one on the motherboard of the duron 1 GHz mackine, the other a promise TX2 plus SATA + PATA controller. I run kernel 2.4.22

The disks and hdparm -t on each of them

/dev/hdc1 seagate 80 GB                   16 MB/s
/dev/sda1 maxtor sata 200 GB              50 MB/s
/dev/sdb7 maxtor 160 GB                   54 MB/s

The partitions are al about 5 GB each.

If I make a raid0 device of all of them I get a thruput of 45 MB/s IIf I exclude the hdc1 partition, I get around 75 MB/s. The system is a little loaded - but that would be normal operating conditions. CPU is 90 % idle. I have about 100 MB free RAM.

Why is it slower with 3 disks in the raid0 than
2 disks? Why don't I get approx 100 MB/s out of just the two?

Quoted 8 lines

Best regards
Keld

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#0 lore raw

Hi

I have some strange performance results on a raid0
I have 4 IDE disks on two controllers, the one on the motherboard of the duron 1 GHz mackine, the other a promise TX2 plus SATA + PATA controller. I run kernel 2.4.22

The disks and hdparm -t on each of them

/dev/hdc1 seagate 80 GB                   16 MB/s
/dev/sda1 maxtor sata 200 GB              50 MB/s
/dev/sdb7 maxtor 160 GB                   54 MB/s

The partitions are al about 5 GB each.

If I make a raid0 device of all of them I get a thruput of 45 MB/s IIf I exclude the hdc1 partition, I get around 75 MB/s. The system is a little loaded - but that would be normal operating conditions. CPU is 90 % idle. I have about 100 MB free RAM.

Why is it slower with 3 disks in the raid0 than
2 disks? Why don't I get approx 100 MB/s out of just the two?

Best regards
Keld

#3 lore raw
In reply to Keld Jørn Simonsen
Quoted 4 lines · Keld Jørn Simonsen

I would have thought the disks would have worked towards gaining performance enhancements in some cumulative way, is that not so, in theory?

So why is 3 disks slower than 2?

Well, I'm not a complete guru so maybe I'm wrong, but in a stripe the point is that you are pressing more platters into simultaneous service. There's a certain amount of synchronization because the things your computer needs at higher levels are spread across multiple disks, so it wants them at about the same time. If the first two disks provide the data quickly and the third doesn't, the system probably will have to wait for that third one before it can move on.

Thats my suspicion anyways.

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
In reply to Matt Thrailkill
Quoted 4 lines · Matt Thrailkill

The you don't get as good performance because hdc1 is slow enough to really drag it down. Think about it, if reads are all going about sequentially, it has to spend alot of time waiting on that 16mb/s drive compared to the others.

I would have thought the disks would have worked towards gaining performance enhancements in some cumulative way, is that not so, in theory?

So why is 3 disks slower than 2?

Quoted 6 lines

You probably can't get 100mb/s with the two because that would be pretty efficient, and there's probably more overhead than that.

Shouldn't the 80gb drive be going faster than 16mb/s though? Have you checked hdparm to make sure dma and all the goodies are turned on for it?

I have a 40 GB segate disk also, on the same motherboard IDE controller, it runs about 40 MB/s. So yes, the hdc1 should go faster, and I have measured something like 40 MB/s on it in idler times. I think it is because it is running other processes. The machine is a ftp mirror and the disk has RedHat and Fedora ISO images, so it is quite popular.

Quoted 21 lines · Keld Jørn Simonsen
Quoted 20 lines

Hi

I have some strange performance results on a raid0
I have 4 IDE disks on two controllers, the one on the motherboard of the duron 1 GHz mackine, the other a promise TX2 plus SATA + PATA controller. I run kernel 2.4.22

The disks and hdparm -t on each of them

/dev/hdc1 seagate 80 GB                   16 MB/s
/dev/sda1 maxtor sata 200 GB              50 MB/s
/dev/sdb7 maxtor 160 GB                   54 MB/s

The partitions are al about 5 GB each.

If I make a raid0 device of all of them I get a thruput of 45 MB/s IIf I exclude the hdc1 partition, I get around 75 MB/s. The system is a little loaded - but that would be normal operating conditions. CPU is 90 % idle. I have about 100 MB free RAM.

Quoted 5 lines
Quoted 5 lines

Why is it slower with 3 disks in the raid0 than
2 disks? Why don't I get approx 100 MB/s out of just the two?

Best regards
Keld

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
Quoted 18 lines · Keld Jørn Simonsen
Quoted 4 lines · Matt Thrailkill

The you don't get as good performance because hdc1 is slow enough to really drag it down. Think about it, if reads are all going about sequentially, it has to spend alot of time waiting on that 16mb/s drive compared to the others.

I would have thought the disks would have worked towards gaining performance enhancements in some cumulative way, is that not so, in theory?

So why is 3 disks slower than 2?

Quoted 6 lines

You probably can't get 100mb/s with the two because that would be pretty efficient, and there's probably more overhead than that.

Shouldn't the 80gb drive be going faster than 16mb/s though? Have you checked hdparm to make sure dma and all the goodies are turned on for it?

Quoted 6 lines

I have a 40 GB segate disk also, on the same motherboard IDE controller, it runs about 40 MB/s. So yes, the hdc1 should go faster, and I have measured something like 40 MB/s on it in idler times. I think it is because it is running other processes. The machine is a ftp mirror and the disk has RedHat and Fedora ISO images, so it is quite popular.

Hmm, I have ext3 filesystems, and they are updating the atime in the inodes. Could that be it? inode flushing obstructing the striping?

Best regards
Keld
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#5 lore raw
In reply to Keld Jørn Simonsen
Quoted 2 lines
Quoted 2 lines

The you don't get as good performance because hdc1 is slow enough to really drag it down.

right.

Quoted 2 lines

I would have thought the disks would have worked towards gaining performance enhancements in some cumulative way, is that not so, in theory?

raid0 stripes IO across devices, so more devices help, in general. but suppose your config stores 1k per disk. a 3k write would touch all three disks. when would that write complete? according to your numbers, hdc1 takes much longer than the other two. that means that a large upper-level IO request (all three disks) will complete at about 3x16 MB/s (ignoring overhead, etc).

Quoted 1 line

So why is 3 disks slower than 2?

because two of the disks are sometimes idle while the slow one completes. since striping is always parallel across all disks, the system can't somehow find other IO to do on the fast disks while waiting for the slow one.

Quoted 2 lines
Quoted 2 lines

You probably can't get 100mb/s with the two because that would be pretty efficient, and there's probably more overhead than that.

100 MB/s is a pretty agressive goal for a duron: the speed of the CPU and memory bandwidth *DO* have a significant effect on how well raid0 scales, and where it tops out. hdparm -t is also quite cpu-inefficient, as benchmarks go.

Quoted 3 lines
Quoted 3 lines

Shouldn't the 80gb drive be going faster than 16mb/s though? Have you checked hdparm to make sure dma and all the goodies are turned on for it?

16 MB/s is a clear config or measurement mistake. 16 MB/s is the speed of disks from 5+ years ago, and >= 80G didn't exist back then.

regards, mark hahn.

#6 lore raw
Quoted 8 lines · Mark Hahn
Quoted 2 lines
Quoted 2 lines

The you don't get as good performance because hdc1 is slow enough to really drag it down.

right.

Quoted 2 lines

I would have thought the disks would have worked towards gaining performance enhancements in some cumulative way, is that not so, in theory?

Quoted 6 lines

raid0 stripes IO across devices, so more devices help, in general. but suppose your config stores 1k per disk. a 3k write would touch all three disks. when would that write complete? according to your numbers, hdc1 takes much longer than the other two. that means that a large upper-level IO request (all three disks) will complete at about 3x16 MB/s (ignoring overhead, etc).

So max thruput in sequential read would never be faster than the slowest disk times the total number of drives. Sounds right there, then 3 * 16 is 48, close to the observed value of 45.

Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

You probably can't get 100mb/s with the two because that would be pretty efficient, and there's probably more overhead than that.

100 MB/s is a pretty agressive goal for a duron: the speed of the CPU and memory bandwidth *DO* have a significant effect on how well raid0 scales, and where it tops out. hdparm -t is also quite cpu-inefficient, as benchmarks go.

Why should the duron CPU speed be a problem? Or the RAM?

Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

Shouldn't the 80gb drive be going faster than 16mb/s though? Have you checked hdparm to make sure dma and all the goodies are turned on for it?

16 MB/s is a clear config or measurement mistake. 16 MB/s is the speed of disks from 5+ years ago, and >= 80G didn't exist back then.

yeah, something is wrong there. I observed a going of about 40 MB/s a couple of days ago on the same disk.

keld

#7 lore raw
In reply to Keld Jørn Simonsen
Quoted 7 lines · Keld Jørn Simonsen

Hi

I have some strange performance results on a raid0
I have 4 IDE disks on two controllers, the one on the motherboard of the duron 1 GHz mackine, the other a promise TX2 plus SATA + PATA controller. I run kernel 2.4.22

Quoted 5 lines

The disks and hdparm -t on each of them

/dev/hdc1 seagate 80 GB                   16 MB/s
/dev/sda1 maxtor sata 200 GB              50 MB/s
/dev/sdb7 maxtor 160 GB                   54 MB/s

Changed my hdc1 disk with my hda1 disk, a seagate 40 GB disk running at something like 40 GB/s.

On a cat Mandrake...iso >/dev/null 681 MB, I now got 102 MB/s on the raid0. I am happy.

Keld
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#8 lore raw
In reply to Keld Jørn Simonsen
Quoted 10 lines · Keld Jørn Simonsen

Hi

I have some strange performance results on a raid0
I have 4 IDE disks on two controllers, the one on the motherboard of the duron 1 GHz mackine, the other a promise TX2 plus SATA + PATA controller. I run kernel 2.4.22

The disks and hdparm -t on each of them

/dev/hdc1 seagate 80 GB                   16 MB/s
/dev/sda1 maxtor sata 200 GB              50 MB/s
/dev/sdb7 maxtor 160 GB                   54 MB/s

Well, if you're lucky, these might be useful as comparative numbers between the different drives on the same system. Just as likely not, though, hdparm rather sucks as a benchmark.

Quoted 6 lines

The partitions are al about 5 GB each.

If I make a raid0 device of all of them I get a thruput of 45 MB/s IIf I exclude the hdc1 partition, I get around 75 MB/s. The system is a little loaded - but that would be normal operating conditions. CPU is 90 % idle. I have about 100 MB free RAM.

Let's assume that the above numbers have a basis in reality. :) If you've got disks with widely varying speeds, then the best performance can often be hand from setting up a linear RAID volume, rather than a RAID0. RAID 0 is really designed to have matching disks, as it distributes data evenly across them. With Linear, and ext2 (erm, I'm assuming 3 as well, I haven't heard anything different), you can sometimes get better performance with smaller writes, because ext2 "scatters" data around the filesystem, in order to avoid fragmentation.
HTH,
Greg
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#9 lore raw
Quoted 11 lines · Gregory Leblanc
Quoted 9 lines · Keld Jørn Simonsen

Hi

I have some strange performance results on a raid0
I have 4 IDE disks on two controllers, the one on the motherboard of the duron 1 GHz mackine, the other a promise TX2 plus SATA + PATA controller. I run kernel 2.4.22

The disks and hdparm -t on each of them

Quoted 7 lines
Quoted 3 lines
/dev/hdc1 seagate 80 GB                   16 MB/s
/dev/sda1 maxtor sata 200 GB              50 MB/s
/dev/sdb7 maxtor 160 GB                   54 MB/s

Well, if you're lucky, these might be useful as comparative numbers between the different drives on the same system. Just as likely not, though, hdparm rather sucks as a benchmark.

Yes, I only use them as crude benchmark measuers, but they seem indicative.

Quoted 7 lines
Quoted 6 lines

The partitions are al about 5 GB each.

If I make a raid0 device of all of them I get a thruput of 45 MB/s IIf I exclude the hdc1 partition, I get around 75 MB/s. The system is a little loaded - but that would be normal operating conditions. CPU is 90 % idle. I have about 100 MB free RAM.

Quoted 8 lines

Let's assume that the above numbers have a basis in reality. :) If you've got disks with widely varying speeds, then the best performance can often be hand from setting up a linear RAID volume, rather than a RAID0. RAID 0 is really designed to have matching disks, as it distributes data evenly across them. With Linear, and ext2 (erm, I'm assuming 3 as well, I haven't heard anything different), you can sometimes get better performance with smaller writes, because ext2 "scatters" data around the filesystem, in order to avoid fragmentation.

Yes, that would be an idea. Anyway I replaced the hdc1 with hda1 on a seagate 40 GB disk, which hdparm said could do about 40 MB/s, and then - when I was lucky, I could actually get about 120 MB/s thruput on a "cat file >/dev/null", 681 MB in 5.63 secs. This on a mildly loaded production system. Not bad!
Then I am beginning to hit the 1 Gbit/s limit on the PCI bus.

Best regards
Keld
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html