Abysmal RAID 0 performance on 2.4.0-test10 for IDE?
Quoted 10 lines · Felix von Leitner
Thus spake Rik van Riel ([email protected]):
Quoted 7 lines
Quoted 5 lines
One more detail: top says the CPU is 50% system when reading from either one of the disk or raid devices. That seems awfully high considering that the Promise controller claims to do UDMA.
Any comments?
Your program reads in data at 30MB/second, on a memory bus that most likely supports something like 60 to 100MB/second.
100.
So that's 30% for the UDMA controller and maybe
30% for the CPU (if your program reads in all the
data).
Quoted 4 lines
Quoted 2 lines
Part of this memory bandwidth is needed for the UDMA controller to push the data to memory, probably between 30% and 50%.
That would be 30%.
Add to that the overhead of allocating and reclaiming the memory, doing the RAID mapping, sending commands to the hard disk, ...
Quoted 5 lines
Quoted 2 lines
Every time the UDMA controller has the memory bus for itself the CPU will busy-wait on memory, which shows up as CPU busy time.
So, you are saying, when I add a gigabit ethernet card, CPU will hit 100% at about 30 MB/second? That sounds like a weak architecture ;-)
Hey, there's a reason PCs are so cheap ;)
regards,
Rik
--
Hollywood goes for world dumbination,
Trailer at 11. http://www.surriel.com/
http://www.conectiva.com/ http://distro.conectiva.com.br/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 8 lines · Felix von Leitner
Quoted 6 lines
Thus spake Rik van Riel ([email protected]):
Quoted 5 lines
Quoted 5 lines
One more detail: top says the CPU is 50% system when reading from either one of the disk or raid devices. That seems awfully high considering that the Promise controller claims to do UDMA.
Any comments?
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Your program reads in data at 30MB/second, on a memory bus that most likely supports something like 60 to 100MB/second.
100.
So that's 30% for the UDMA controller and maybe
30% for the CPU (if your program reads in all the
data).
Where are you getting 100MB/s?
The PCI bus can move around 130MB/sec, but RAM is lots faster. A single PC100 DIMM can move 800MB/sec.
This P100 laptop I'm typing on gets better than 100MB/s ram reads.
Anyway, in clarification, Rik mentioned that two reads from different disk (arrays?) on the same controller at the same time get more or less the same speed.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 2 lines · Ian Stirling
Where are you getting 100MB/s?
The PCI bus can move around 130MB/sec, but RAM is lots faster.
I'll clarify your clarification further. :) Your typical PC has 33MHz 32-bit PCI. Increasing it to 66MHz or 64-bit can double the transfer rate, and doing both can quadruple it. (Perhaps I've overlooked a detail or oversimplified something, in which case I'd appreciate being corrected.)
-Barry K. Nathan <[email protected]>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 1 line · Ian Stirling
The PCI bus can move around 130MB/sec,
in bursts yes, but sustained data bandwidth of PCI is a lot lower, maybe 30 to 50MB/s. And you won't get sustained RAID performance > sustained PCI performance.
Quoted 3 lines
Anyway, in clarification, Rik mentioned that two reads from different disk (arrays?) on the same controller at the same time get more or less the same speed.
try scsi.
--paulj
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
RAID - IDE - here we go again...
In reply to Ian StirlingQuoted 7 lines · Paul Jakma
Quoted 1 line · Ian Stirling
The PCI bus can move around 130MB/sec,
in bursts yes, but sustained data bandwidth of PCI is a lot lower, maybe 30 to 50MB/s. And you won't get sustained RAID performance > sustained PCI performance.
Much higher than 30-50 - but yes, the total bandwidth won't exceed the slowest channel.
Quoted 6 lines
Quoted 4 lines
Anyway, in clarification, Rik mentioned that two reads from different disk (arrays?) on the same controller at the same time get more or less the same speed.
try scsi.
SCSI won't get you a faster PCI bus. Guys, *PLEASE*, - everyone on this list
knows what the respective virtues and horrors of SCSI and IDE are. Configured
properly, both can perform well, configured wrongly, both will suck.
The timings below are from a dual PII-350, Asus P2B-DS, it has a six disk SCSI RAID and a five disk IDE RAID. The IDE raid is configured properly with one channel per disk - which would have solved the performance problem in the array that spawned this thread.
(By the way: the SCSI RAID is configured with three controllers for the six disks, because of the low SCSI bus bandwidth, reality rules in the SCSI world as well)
Kernel is 2.2, RAID is 0.90, IDE is Andre's
---------------------------------------
Dual PII-350, 256 MB RAM, test on 1 GB file
Filesystem: 75 GB ext2fs RAID: Linux Softare RAID-5 Disks: 5 pcs. IBM Deskstar 75 GXP (30GB) Controller: 3 pcs. Promise PDC-2067
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
1024 4998 98.0 28660 45.9 14586 50.4 5386 98.5 71468 79.1 338.6 5.2 28 MB/sec write, 71MB/sec read on RAID-5.
---------------------------------------
Same box, test on 1 GB file
Filesystem: 18 GB ext2fs RAID: Linux Softare RAID-0
This array is built on other partitions on the same disks as above.
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
1024 5381 99.7 44267 50.1 22613 62.1 5469 99.4 98075 91.4 311.0 5.0 44 MB/sec write, 98 MB/sec read on RAID-0. That's thru *one* PCI bus folks.
Please, for further comments or another IDE-hotplug-can!-cannot!-can-so!, let's
take this to linux-raid or #offtopic ;)
Enough said.
Signature · 10 lines
................................................................ : [email protected] : And I see the elder races, : :.........................: putrid forms of man : : Jakob Østergaard : See him rise and claim the earth, : : OZ9ABN : his downfall is at hand. : :.........................:............{Konkhra}...............: - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Quoted 8 lines · Paul Jakma
Quoted 1 line · Ian Stirling
The PCI bus can move around 130MB/sec,
in bursts yes, but sustained data bandwidth of PCI is a lot lower, maybe 30 to 50MB/s. And you won't get sustained RAID performance > sustained PCI performance.
No. A well-designed card and driver doing cache-line sized transfers can achieve ~100MB/s. On the IBM (Sequent) NUMA machines, we achieved in excess of 3GB/s sustained read I/O (database full table scan) on a 16-quad (32 PCI bus) system. That works out at around 100MB/s per bus.
Regards,
Tim
Signature · 7 lines
Tim Wright - [email protected] or [email protected] or [email protected] IBM Linux Technology Center, Beaverton, Oregon "Nobody ever said I was charming, they said "Rimmer, you're a git!"" RD VI - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Quoted 10 lines · [email protected] (Tim Wright) writes:
Quoted 8 lines · Paul Jakma
Quoted 1 line · Ian Stirling
The PCI bus can move around 130MB/sec,
in bursts yes, but sustained data bandwidth of PCI is a lot lower, maybe 30 to 50MB/s. And you won't get sustained RAID performance > sustained PCI performance.
Quoted 4 lines
No. A well-designed card and driver doing cache-line sized transfers can achieve ~100MB/s. On the IBM (Sequent) NUMA machines, we achieved in excess of 3GB/s sustained read I/O (database full table scan) on a 16-quad (32 PCI bus) system. That works out at around 100MB/s per bus.
Sadly, I am sure that your "well-designed" system must be costly as hell... :(
Signature · 8 lines
Mathieu CHOUQUET-STRINGER E-Mail : [email protected] Learning French is trivial: the word for horse is cheval, and everything else follows in the same way. -- Alan J. Perlis - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/