Horrible drive performance under concurrent i/o jobs (dlh problem?)
Hi list readers (and hopefully writers),
after getting crazy with our main server in the company for over a week now, this list is possibly my last help - I am no kernel programmer but suscpect it to be a kernel problem. Reading through the list did not help me (although I already thought so, see below).
We are running a 3ware Escalade 7850 Raid controller with 7 IBM Deskstar GXP 180 disks in Raid 5 mode, so it builds a 1.11TB disk. There's one partition on it, /dev/sda1, formatted with Reiserfs format 3.6. The Board is an MSI 6501 (K7D Master) with 1GB RAM but only one processor.
We were running the Raid smoothly while there was not much I/O - but when we tried to produce large amounts of data last week, read and write performance went down to inacceptable low rates. The load of the machine went high up to 8,9,10... and every disk access stopped processes from responding for a few seconds (nedit, ls). An "rm" of many small files made the machine not react to "reboot" anymore, I had to reset it.
So copied everything away to a software raid and tried all the disk tuning stuff (min-, max-readahead, bdflush, elvtune). Nothing helped. Last Sunday I then found a hint about a bug introduced in kernel 2.4.19-pre6 which could be fixed using a "dlh", disk latency hack - or going back to 2.4.18. Last is what I did ( from 2.4.20 )
It seemed to help in a way that I could copy about 350GB back to the RAID in about 3-4 hrs overnight. So I THOUGHT everything would be fine. Since Tuesday morning my collegues are trying to produce large amounts of data again - and every concurrent I/O operation blocks all the others. We cannot work with that.
When I am working all alone on the disk creating a 1 GB file by
time dd if=/dev/zero of=testfile bs=1G count=1
results in real times from 14 seconds when I am very lucky up to 4 minutes usually.
Watching vmstat 1 shows me that "bo" drops quickly down from rates in the 10 or 20 thousands to low rates of about 2 or 3 thousands when the runs take so long.
Can anyone of you please tell me how can I find out if this is a kernel problem or a hardware 3ware-device problem? Is there a way to see the difference? Or could it come from running an SMP kernel although I have only one CPU in my board?
I would be very happy about every answer, really!
Torben
I hope I'm not entering this discussion too late to be of help.
Quoted 3 lines
We are running a 3ware Escalade 7850 Raid controller with 7 IBM Deskstar GXP 180 disks in Raid 5 mode, so it builds a 1.11TB disk.
Raid 0 is better in terms of speed.
Quoted 2 lines
There's one partition on it,
/dev/sda1, formatted with ReiserFS format 3.6.
Oh no! This is bad news, both in terms of speed and security.
Lumping everything into one partition makes it impossible to protect against the SUID/GUID security vulnerability (security), and requires all reads and writes to be funneled through one partition (speed).
At an absolute MINIMUM, your partitions should be divided into:
/boot 032 MB
/tmp 512 MB
swap 1.5 - 3.0 times the amount of RAM,
not to exceed a 2 GB per swap partition limit
/root 5 GB
/var 10 GB
/usr 20% of what is leftover
/home 50% of what is leftover,
or at least 32 MB per user
/ 30% of what is leftover
The above numbers are my recommendations for your 1.1 TB Raid Array ONLY. Please don't flame me about how those numbers aren't right for everybody's systems.
Additionally, in your case, I'd add a /data partition to store this huge amount of rapidly generated data you're talking about. That way, if you should need to reformat, remount, whatever, the partition with the data, you won't have to take down or redo the whole system.
Quoted 2 lines
Or could it come from running an SMP kernel although I have only one CPU in my board?
This is a bad idea. The SMP kernel includes a whole load of extra code which is totally unnecessary in a Uniprocessor environment. All that extra code will do is slow down your system and take up extra memory.
Switch to a Uniprocessor kernel, or see my next point.
Quoted 2 lines
The Board is an MSI 6501 (K7D Master) with 1GB RAM but only one processor.
Upgrade to 4 GB of RAM (if possible) and add a second processor (if possible).
Quoted 1 line
From what you're telling me, you're going to need all the RAM you can get.
Further, SCSI really works better with two or more processors. SCSI is designed to take advantage of multiple processors. If you're not running multiple processors, you might as well be running IDE, IMHO.
Quoted 3 lines
Watching vmstat 1 shows me that "bo" drops quickly down from rates in the 10 or 20 thousands to low rates of about 2 or 3 thousands when the runs take so long.
I'm willing to bet that your system is spending all of its time flushing the buffers. When you're generating gigabytes of data, your buffers are going to need to be huge.
I have no idea how to do this.
Hope this helped.
Joseph Wagner
Quoted 6 lines · Joseph D. Wagner
Date: Fri, 20 Dec 2002 14:40:30 -0600
From: Joseph D. Wagner <[email protected]>
To: 'Torben Frey' <[email protected]>, [email protected]
Subject: RE: Horrible drive performance under concurrent i/o jobs (dlh
problem?)
Quoted 7 lines
I hope I'm not entering this discussion too late to be of help.
Quoted 3 lines
We are running a 3ware Escalade 7850 Raid controller with 7 IBM Deskstar GXP 180 disks in Raid 5 mode, so it builds a 1.11TB disk.
Raid 0 is better in terms of speed.
IIRC raid0 is striping with no redundancy, good for speed, very bad for reliability. Raid 1 or 1+0 are the fastest modes to use that give redundancy for the data, but they cost you half your disk space. In addition everything I have seen and experianced about the 3ware cards indicates that they are very fast for these modes, but much slower for Raid 4 or 5
Quoted 8 lines
Quoted 2 lines
There's one partition on it,
/dev/sda1, formatted with ReiserFS format 3.6.Oh no! This is bad news, both in terms of speed and security.
Lumping everything into one partition makes it impossible to protect against the SUID/GUID security vulnerability (security), and requires all reads and writes to be funneled through one partition (speed).
Here I have to disagree with you.
for default system installs that don't mount things noexec or readonly there is little if any security difference between one large partition and lots of small partitions. If you do have to time to use the mount flags to protect your system then it is an advantage, unfortunantly in the real world overworked sysadmins seldom have the time to do this (and if it's not done at build time it becomes almost impossible to do later becouse things get into places that would violate your mount restrictions)
As for speed, as long as you are on the same spindles there is no definante speed gain for having lots of partitions and there is a definante cost to having lots of partitions. If you think about it, if you have seperate partitions you KNOW that you will have to seek across a large portion of the disk to get from /root to /var where they may not be seperated that much if they are one filesystem.
Quoted 11 lines
At an absolute MINIMUM, your partitions should be divided into:
/boot 032 MB
/tmp 512 MB
swap 1.5 - 3.0 times the amount of RAM,
not to exceed a 2 GB per swap partition limit
/root 5 GB
/var 10 GB
/usr 20% of what is leftover
/home 50% of what is leftover,
or at least 32 MB per user
/ 30% of what is leftover
The other problem with lots of partitions is that you almost always get into a situation where one partition fills up and you have to go to a significant amount of work to repartition things.
Quoted 9 lines
The above numbers are my recommendations for your 1.1 TB Raid Array ONLY. Please don't flame me about how those numbers aren't right for everybody's systems.
Additionally, in your case, I'd add a /data partition to store this huge amount of rapidly generated data you're talking about. That way, if you should need to reformat, remount, whatever, the partition with the data, you won't have to take down or redo the whole system.
Quoted 8 lines
Quoted 2 lines
Or could it come from running an SMP kernel although I have only one CPU in my board?
This is a bad idea. The SMP kernel includes a whole load of extra code which is totally unnecessary in a Uniprocessor environment. All that extra code will do is slow down your system and take up extra memory.
Switch to a Uniprocessor kernel, or see my next point.
Definantly. Also make sure you compile your own kernel optimizing for the CPU that you have. this can be a 30% improvement with no other changes.
Quoted 7 lines
Quoted 2 lines
The Board is an MSI 6501 (K7D Master) with 1GB RAM but only one processor.
Upgrade to 4 GB of RAM (if possible) and add a second processor (if possible).
From what you're telling me, you're going to need all the RAM you can get.
I am a little more shy about going beyond 1G of ram. things are getting better, but there are still lots of things that can only exist in the lowest Gig of ram and if you have lots of memory you run the possibility of filling it up and having the machine lockup. If you need the ram add it, but on my machines that don't _NEED_ that much ram I put in 1G and use 960M of it (himem disabled). on systems doing a lot of disk IO I do put large amounts of cache on the scsi controllers.
Quoted 3 lines
Further, SCSI really works better with two or more processors. SCSI is designed to take advantage of multiple processors. If you're not running multiple processors, you might as well be running IDE, IMHO.
Here I will disagree slightly. I see very significant advantages running SCSI even on single CPU machines. it all depends on your workload.
Quoted 9 lines
Quoted 3 lines
Watching vmstat 1 shows me that "bo" drops quickly down from rates in the 10 or 20 thousands to low rates of about 2 or 3 thousands when the runs take so long.
I'm willing to bet that your system is spending all of its time flushing the buffers. When you're generating gigabytes of data, your buffers are going to need to be huge.
I have no idea how to do this.
you definantly need to experiment with all the other types of journaling filesystems. the performance tradeoffs of the different options are not well understood (a combination of a lack of benchmarks of the different types and the tendancy of benchmarks to show the best aspect of the filesystem while missing the problem areas) and performance changes so drasticly based on workloads and tuning parameters (journal size, flush delays, etc)
David Lang
Quoted 4 lines
Quoted 1 line
Raid 0 is better in terms of speed.
IIRC raid0 is striping with no redundancy, good for speed, very bad for reliability.
I left that out figuring that since he's a system administrator that he'd already know that.
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
There's one partition on it,
/dev/sda1Oh no! This is bad news, both in terms of speed and security.
Quoted 8 lines
Quoted 5 lines
Lumping everything into one partition makes it impossible to protect against the SUID/GUID security vulnerability (security), and requires all reads and writes to be funneled through one partition (speed).
Here I have to disagree with you.
for default system installs... [trimmed]
Oh yeah, I agree that default installs are woefully insufficient and inadequate when it comes to security. Red Hat should really get its act together. OpenBSD, for example, has only had two security holes in the default installation over the past three years.
However, if he doesn't use separate partitions in the first place, he's going to have to go back and repartition from scratch just to plug that security hole.
On the other hand, if he does use separate partitions from the beginning, all he as to do is change some mount options in the /etc/fstab file.
Quoted 9 lines
As for speed, as long as you are on the same spindles there is no definante speed gain for having lots of partitions and there is a definante cost to having lots of partitions. If you think about it, if you have separate partitions you KNOW that you will have to seek across a large portion of the disk to get from /root to /var where they may not be seperated that much if they are one filesystem.
Ok, now here's where you're just plain wrong.
SHORT ANSWER: Segregating partitions reduces seek time. Period.
LONG ANSWER: Reads and writes tend to be grouped within a partition. For example, if you're starting a program, you're going to be doing a lot of reads somewhere in the /usr partition. If the program uses temporary files, you're going to do a lot of reads & writes in the /tmp partition. If you're saving a file, you're going to be doing lots of writes to the /home partition. Hence, since most disk accesses occur in groups within a partition, preference should be giving to reducing seek time WITHIN a partition, rather than reducing seek time BETWEEN partitions.
Quoted 12 lines
Quoted 11 lines
At an absolute MINIMUM, your partitions should be divided into:
/boot 032 MB
/tmp 512 MB
swap 1.5 - 3.0 times the amount of RAM,
not to exceed a 2 GB per swap partition limit
/root 5 GB
/var 10 GB
/usr 20% of what is leftover
/home 50% of what is leftover,
or at least 32 MB per user
/ 30% of what is leftover
Quoted 5 lines
The other problem with lots of partitions is that you almost always get into a situation where one partition fills up and you have to go to a significant amount of work to repartition things.
That's why my recommended numbers are as large as they are, and that's why the /usr, /home, and / partitions are percentages of space leftover instead of fixed sizes. I see no situation where my recommended numbers are too low for his system.
Quoted 3 lines
Quoted 1 line
Switch to a Uniprocessor kernel, or see my next point.
Definantly.
Finally. We agree on something. 8-)
Quoted 2 lines
[trimmed]...you run the possibility of filling [RAM] up and having the machine lockup.
Oh come on! Linux is the most stable, reliable, wonderful operating system in existence. It's the be-all, end-all of operating systems. How can it "lockup"?!?
Hahahahhahahahhahahhahah!
But seriously, you are correct in saying that Linux is piss poor in utilizing memory beyond 1 GB, but hopefully someday either 1) the Linux goons will get their act together and write some better code, or 2) he'll switch to Windows 2000/XP Pro which CAN efficiently and effectively utilize memory beyond 1 GB.
Quoted 8 lines
Quoted 4 lines
Further, SCSI really works better with two or more processors. SCSI is designed to take advantage of multiple processors. If you're not running multiple processors, you might as well be running IDE, IMHO.
Here I will disagree slightly. I see very significant advantages running SCSI even on single CPU machines. it all depends on your workload.
But isn't that the point? His workload is so high that he needs the second CPU to manage other processes while this program is generating vast amounts of data?
Joseph Wagner
Quoted 14 lines · Joseph D. Wagner
Quoted 9 lines
As for speed, as long as you are on the same spindles there is no definante speed gain for having lots of partitions and there is a definante cost to having lots of partitions. If you think about it, if you have separate partitions you KNOW that you will have to seek across a large portion of the disk to get from /root to /var where they may not be seperated that much if they are one filesystem.
Ok, now here's where you're just plain wrong.
SHORT ANSWER: Segregating partitions reduces seek time. Period.
Quoted 8 lines
LONG ANSWER: Reads and writes tend to be grouped within a partition. For example, if you're starting a program, you're going to be doing a lot of reads somewhere in the /usr partition. If the program uses temporary files, you're going to do a lot of reads & writes in the /tmp partition. If you're saving a file, you're going to be doing lots of writes to the /home partition. Hence, since most disk accesses occur in groups within a partition, preference should be giving to reducing seek time WITHIN a partition, rather than reducing seek time BETWEEN partitions.
with one partition you MAY have to seek across the disk to get from one file to another (depends on the optimization of the filesystem)
with multiple partitions you WILL have to seek across the disk becouse files on one partition are forrced by your partitioning to be on a seperate part of the drive.
if all your access it to the same file it won't matter how you are partitioned, but if you read a file from one filesystem, put intermediate results in /tmp, then put the final result back on the first filesystem you will end up doing a LOT of seeking between the partitions.
I am not saying that a single partition is nessasarily faster, but I am saying that multiple partitions are not a definante win, and under some conditions can be a significant loss.
it's like filesystem type, you need to look at what you are doing and plan accordingly.
David Lang
Quoted 10 lines
SHORT ANSWER: Segregating partitions reduces seek time. Period.
LONG ANSWER: Reads and writes tend to be grouped within a partition. For example, if you're starting a program, you're going to be doing a lot of reads somewhere in the /usr partition. If the program uses temporary files, you're going to do a lot of reads & writes in the /tmp partition. If you're saving a file, you're going to be doing lots of writes to the /home partition. Hence, since most disk accesses occur in groups within a partition, preference should be giving to reducing seek time WITHIN a partition, rather than reducing seek time BETWEEN partitions.
keep in mind that only around half of the seek time is because of the partition! Taking an IBM 120GXP as an example:
Average seek: 8.5ms
Full stroke seek: 15.0ms
Time to rotate disk one round: 1/(7200/60)*1000 = 8.3ms
Then, the sector you're looking for, will, by average, be half a round away from where you are, and thus, giving the minimum average seek time 8.3/2 = 4.15ms or something like half the seek time. Concidering this, you may gain a maximum <= 50% gain in using smaller partitions.
btw. anyone that knows the zone layout on IBM drives?
roy
Quoted 15 lines · Roy Sigurd Karlsbakk
Quoted 14 lines
SHORT ANSWER: Segregating partitions reduces seek time. Period.
LONG ANSWER: Reads and writes tend to be grouped within a partition. For
example, if you're starting a program, you're going to be doing a lot of
reads somewhere in the /usr partition. If the program uses temporary files,
you're going to do a lot of reads & writes in the /tmp partition. If you're
saving a file, you're going to be doing lots of writes to the /home partition. Hence, since most disk accesses occur in groups within a partition, preference should be giving to reducing seek time WITHIN a partition, rather than reducing seek time BETWEEN partitions.
Quoted 6 lines
keep in mind that only around half of the seek time is because of the partition! Taking an IBM 120GXP as an example:
Average seek: 8.5ms
Full stroke seek: 15.0ms
Time to rotate disk one round: 1/(7200/60)*1000 = 8.3ms
I'm afraid your math is off.
The rotational frequency should be 7200*60/sec which makes for 2.31 us which would produce an average rotational latency of 1.16us if such a condition even still applies. My expectation is that the whole track is buffered starting from the first sector that syncs thereby making the time rotfreq + rotfreq/nsect or something similar. In any case the rotational latency or frequency is orders of magnitude smaller than the seek time, even between adjacent tracks/cylinders.
If the the stated average seek is 50% of full stroke and not based on reality then 76% of the cost of an average seek is attributed to distance and likewise 87% of the cost of a full. Based on that i'd say the seek distance is a much bigger player than you are assuming. If it weren't the value of elevators would be much less.
Quoted 7 lines
Then, the sector you're looking for, will, by average, be half a round away from where you are, and thus, giving the minimum average seek time 8.3/2 = 4.15ms or something like half the seek time. Concidering this, you may gain a maximum <= 50% gain in using smaller partitions.
btw. anyone that knows the zone layout on IBM drives?
Having chimed in i'll also mention that having the filesystems right-sized and small should produce better locality of reference for multiple files and large files given the tendency of our filesystems to spread their directories across the cylinders. One big filesystem is as likely to have the assorted files spread from one end of the disk to the other as you will get with several smaller ones. Witness the discussions that introduce the orlov allocator to ext[23].
As for the repartitioning when a filesystems outgrows its partition that is reason #1 for lvm. Care should be taken though because lvm can also destroy locality through discontinuous extent allocation.
Signature · 5 lines
________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [email protected] Remember Cernan and Schmitt
Quoted 5 lines · jw schultz
I'm afraid your math is off.
The rotational frequency should be 7200*60/sec which makes for 2.31 us which would produce an average rotational latency of 1.16us if such a condition even still applies.
No, I think you're just a bit off. A 7200RPM drive *does* revolve in 8.3ms. A 2.31us rotation time would be 432900 RPS or approx. 26,000,000 RPM. Which is pushing it.
The 8.3ms rotational latency is clearly visible as a wide band if you graph access time against distance.
http://www.goop.org/~jeremy/seek-buf.eps is an example of a 7200RPM Western Digital drive.
Quoted 6 lines
My expectation is that the whole track is buffered starting from the first sector that syncs thereby making the time rotfreq + rotfreq/nsect or something similar. In any case the rotational latency or frequency is orders of magnitude smaller than the seek time, even between adjacent tracks/cylinders.
Track to track seek time is typically around 1ms. Rotational latency is often the dominating factor in access time.
Quoted 4 lines
If the the stated average seek is 50% of full stroke and not based on reality then 76% of the cost of an average seek is attributed to distance and likewise 87% of the cost of a full.
Well, average seek is a vague concept. If you assume that all seeks are randomly distributed, then it might mean a half-stroke seek. But almost all seeks are short, so the weighting means that short seek time is the most important to optimise (which drive vendors do: they use techniques like dumping more current into the coils for short seeks because they know it is short; if they dumped the same current for a long seek, it would burn things out). Also, there are only two cylinders between which you can have the maximal seek, whereas every adjacent pair of cylinders can have a minimal seek; in other words the physical nature of the drive means that short seeks will dominate, even with random seeking.
Rotational latency tends to dominate these days: a fast drive can do a track to track seek in 1ms, and full-stroke seek in 9ms or so. That means that a 1ms track-to-track seek can take longer than a full stroke if you include the 8.3ms variation.
Elevators are important for keeping seeks short. This is partly to reduce the seek time, but mostly because drives are optimised so that the track-to-track skew is set up to minimise rotational latency. The further you seek, the more likely you are to miss the rotlat deadline and have to take a full rotation penalty.
J
Quoted 3 lines · jw schultz
The rotational frequency should be 7200*60/sec which makes for 2.31 us which would produce an average rotational latency of 1.16us if such a condition even still applies.
That would be 432000 rotations per second, meaning that the edge of a 3.5" disk would travel at almost 120 kilometers per second and be stressed by some pretty impressive G forces, which I'm too lazy to calculate.
Good thing a 7200 RPM disk only spins 120 times a second, that's a lot safer in consumer applications. ;)
Rik
Signature · 3 lines
Bravely reimplemented by the knights who say "NIH". http://www.surriel.com/ http://guru.conectiva.com/ Current spamtrap: <a href=mailto:"[email protected]">[email protected]</a>
Quoted 20 lines · jw schultz
Quoted 6 lines · Roy Sigurd Karlsbakk
keep in mind that only around half of the seek time is because of the partition! Taking an IBM 120GXP as an example:
Average seek: 8.5ms
Full stroke seek: 15.0ms
Time to rotate disk one round: 1/(7200/60)*1000 = 8.3msI'm afraid your math is off.
The rotational frequency should be 7200*60/sec which makes for 2.31 us which would produce an average rotational latency of 1.16us if such a condition even still applies. My expectation is that the whole track is buffered starting from the first sector that syncs thereby making the time rotfreq + rotfreq/nsect or something similar. In any case the rotational latency or frequency is orders of magnitude smaller than the seek time, even between adjacent tracks/cylinders.
Quoted 6 lines
If the the stated average seek is 50% of full stroke and not based on reality then 76% of the cost of an average seek is attributed to distance and likewise 87% of the cost of a full. Based on that i'd say the seek distance is a much bigger player than you are assuming. If it weren't the value of elevators would be much less.
No. Your math is correct. Mine is upside down. Don't know where that came from. Apologies for the bad smell.
Signature · 5 lines
________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [email protected] Remember Cernan and Schmitt
Quoted 1 line · jw schultz
If the the stated average seek is 50% of full stroke and not
no, stated average seek = one third (~33%) of full stroke
google for "average seek stroke" if you don't believe me :)
-Barry K. Nathan <[email protected]>
Looks like at least partially off-topic, but...
Quoted 2 lines · Joseph D. Wagner
Lumping everything into one partition makes it impossible to protect against the SUID/GUID security vulnerability (security),
Why not? chmod ug-s /usr/bin/file... works as expected (?) You worry about users creating suid files in /home? If they can do that, your system is already broken (into).
Quoted 2 lines
and requires all reads and
writes to be funneled through one partition (speed).
But that's higher speed, statistically. Especially if it have to run between your /tmp at the start of disk and /home at the end otherwise. Not that I recommend putting /home on / fs.
Quoted 2 lines
At an absolute MINIMUM, your partitions should be divided into:
/boot 032 MB
Only needed if your BIOS has problems booting from /.
Quoted 1 line
/tmp 512 MB
Some of my machines wouldn't be able to work with that small /tmp.
Quoted 2 lines
swap 1.5 - 3.0 times the amount of RAM,
not to exceed a 2 GB per swap partition limit
Why? I thought you need that much swap as you need - and no more. One of my machines doesn't even have any swap at all.
Quoted 1 line
/root 5 GB
For /root/.ssh etc? You must be using many authorized_keys :-)
Quoted 1 line
/var 10 GB
I always preferred to have separate /var/spool/{mqueue,postfix/etc} and /var/spool/lp (and so on). The size should just be appropriate.
Quoted 1 line
/usr 20% of what is leftover
Hmm... Why not check how much do that need?
Quoted 2 lines
/home 50% of what is leftover,
or at least 32 MB per user
Why not check how much do users need?
Quoted 1 line
/ 30% of what is leftover
And that's for?
Basically, you need as much space on particular fs as much you need. No more, no less.
If (in particular case) I have a disk bigger than I need (they no longer sell <= 20 GB IDE), the unused space might well be just unused space.
Quoted 3 lines
The above numbers are my recommendations for your 1.1 TB Raid Array ONLY. Please don't flame me about how those numbers aren't right for everybody's systems.
I'm not trying to. However, they might be right for nearly no system (except yours, possibly).
Quoted 5 lines
Quoted 2 lines
The Board is an MSI 6501 (K7D Master) with 1GB RAM but only one processor.
Upgrade to 4 GB of RAM (if possible) and add a second processor (if possible).
Why do you think adding extra RAM and CPU can increase disk performance? Cache, yes, but we don't even know if the problem is there. CPU with RAID5, yes, but not with a RAID controller (with software RAID5 only).
Distributing load over multiple disks/controllers may help better.
Writing 1 GB of data might well take 14 seconds and more, but should not take 4 minutes in a idle and correctly configured system.
I don't know the controller in question, but chances are it's just to slow and has no enough bandwidth. Or _it_ might require more RAM.
Switching from RAID5 to RAID1 might help as well, but it's quite expensive.
Quoted 3 lines
Further, SCSI really works better with two or more processors. SCSI is designed to take advantage of multiple processors. If you're not running multiple processors, you might as well be running IDE, IMHO.
Not true. SCSI works best with just a multi-request (i.e. multiprocess) system. The CPU is way faster than any disk, and it can switch processes fast while they are waiting for disk I/O.
ps xa|grep D
Signature · 2 lines
Krzysztof Halasa Network Administrator
Quoted 13 lines · Torben Frey
Hi list readers (and hopefully writers),
after getting crazy with our main server in the company for over a week now, this list is possibly my last help - I am no kernel programmer but suscpect it to be a kernel problem. Reading through the list did not help me (although I already thought so, see below).
We are running a 3ware Escalade 7850 Raid controller with 7 IBM Deskstar GXP 180 disks in Raid 5 mode, so it builds a 1.11TB disk. There's one partition on it,
/dev/sda1, formatted with Reiserfs format 3.6. The Board is an MSI 6501 (K7D Master) with 1GB RAM but only one processor.
Quoted 7 lines
We were running the Raid smoothly while there was not much I/O - but when we tried to produce large amounts of data last week, read and write performance went down to inacceptable low rates. The load of the machine went high up to 8,9,10... and every disk access stopped processes from responding for a few seconds (nedit, ls). An "rm" of many small files made the machine not react to "reboot" anymore, I had to reset it.
Can you provide solid numbers (say Mb/s) of single dd's of varying size? Of concurrent dd's? etc...
Quoted 7 lines
When I am working all alone on the disk creating a 1 GB file by time dd if=/dev/zero of=testfile bs=1G count=1
results in real times from 14 seconds when I am very lucky up to 4 minutes usually.
Watching vmstat 1 shows me that "bo" drops quickly down from rates in the 10 or 20 thousands to low rates of about 2 or 3 thousands when the runs take so long.
Yes, and provide us with vmstat, top, cat /proc/meminfo output and the like!
--
vda