Raid 1 (non) performance
Hello I might have a strange raid setup that results in bad raid 1 performance. I would be very happy if someone have any pointers about this so that I can get some decent performance out of my box...
This is an mailserver running postfix and dovecot (imap/pop). It's running Ubuntu Dapper amd64 with an 2.6.17.6 since the original dapper kernel doesn't support the sata controller. The motherboard is some kind of HT1000 based thing with a single opteron 146 processor with 2gb ram, the sata driver I use is sata_svw. The two disks forming the raid1 set are Maxtor 6V250F0 sata disks. The /var filesystem has some (+j) attributes set on the postfix directories.
I'm not sure what more information is needed, please point out if something is missing.
The raid setup:
# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sda3[0] sdb3[1]
236725696 blocks [2/2] [UU]md1 : active raid1 sda2[0] sdb2[1]
4192896 blocks [2/2] [UU]
md0 : active raid1 sda1[0] sdb1[1]
4192832 blocks [2/2] [UU]
unused devices: <none>
Partitions:
# mount|grep ext/dev/md1 on / type ext3 (rw,noatime,errors=remount-ro) /dev/md2 on /var type ext3 (rw,noatime)
The tune2fs options and journal sizes on the filesystems:
/dev/md1: Journal size: 67108864 (64mb)
ext_attr, resize_inode, dir_index, filetype, sparse_super, large_file
/dev/md2: Journal size: 402653184 (384mb)
resize_inode, dir_index, filetype, sparse_super, large_file
Swaps:
# swapon -s Filename Type Size Used Priority /dev/md0 partition 4192824 116 -1
I've tried to modify the queuing by doing this, to disable the write cache and enable CFQ. The CFQ choice is rather random.
for disk in sda sdb; do
blktool /dev/$disk wcache off
hdparm -q -W 0 /dev/$disk
echo cfq > /sys/block/$disk/queue/scheduler
done
When I search a imap account having 5 gb (approx 500000 mail) worth of mail the box seems to block alot, somehow processes monopolize the IO bandwidth, not like the many other systems I'm running. The raid1 should be an improvement over using the disks directly on the read side atleast.
Any ideas as of why the performance/fairness is out the window? Do you need any more info?
Regards,
Magnus
Quoted 10 lines · Magnus Naeslund(k)
# cat /proc/mdstat Personalities : [raid1] md2 : active raid1 sda3[0] sdb3[1] 236725696 blocks [2/2] [UU]md1 : active raid1 sda2[0] sdb2[1]
4192896 blocks [2/2] [UU]md0 : active raid1 sda1[0] sdb1[1]
4192832 blocks [2/2] [UU]
i see you have split /var and / on the same spindle... if your /home is on / then you're causing extra seek action by having two active filesystems on the same spindles. another option to consider is to make / small and mostly read-only and move /home to /var/home (and use a symlink or mount --bind to place it at /home).
or just put everything in one big / filesystem.
hopefully your swap isn't being used much anyhow.
try "iostat -kx /dev/sd* 5" and see if the split is causing you troubles -- i/o activity on more than one partition at once.
Quoted 5 lines
I've tried to modify the queuing by doing this, to disable the write cache and enable CFQ. The CFQ choice is rather random.
for disk in sda sdb; do
blktool/dev/$diskwcache off
hdparm -q -W 0/dev/$disk
turning off write caching is a recipe for disasterous performance on most ata disks... unfortunately. better to buy a UPS and set up nut or apcupsd or something to handle shutdown. or just take your chances.
-dean
Quoted 7 lines · dean gaudet
i see you have split /var and / on the same spindle... if your /home is on / then you're causing extra seek action by having two active filesystems on the same spindles. another option to consider is to make / small and mostly read-only and move /home to
/var/home(and use a symlink or mount --bind to place it at /home).
Yes I have it like this now:
/var/bind/home on /home type bind (rw,bind,noatime)
Quoted 2 lines
hopefully your swap isn't being used much anyhow.
Zero used, infact.
Quoted 3 lines
try "iostat -kx
/dev/sd*5" and see if the split is causing you troubles -- i/o activity on more than one partition at once.
It seems to be roughly the same numbers on the disks.
Quoted 4 lines
turning off write caching is a recipe for disasterous performance on most ata disks... unfortunately. better to buy a UPS and set up nut or apcupsd or something to handle shutdown. or just take your chances.
This is a mail server, I would like it to not lose mail if there is an power outage. Is there anything one can do without buying more hardware?
Thanks for the feedback,
Magnus
Quoted 26 lines · Magnus Naeslund(k)
Quoted 4 lines · dean gaudet
i see you have split /var and / on the same spindle... if your /home is on / then you're causing extra seek action by having two active filesystems on the same spindles. another option to consider is to make / small and mostly read-only and move /home to
/var/home(and use a symlink or mount --bind to place it at /home).Yes I have it like this now:
/var/bind/homeon /home type bind (rw,bind,noatime)Quoted 3 lines
hopefully your swap isn't being used much anyhow.
Zero used, infact.
Quoted 4 lines
try "iostat -kx
/dev/sd*5" and see if the split is causing you troubles -- i/o activity on more than one partition at once.It seems to be roughly the same numbers on the disks.
Quoted 6 lines
Quoted 3 lines
turning off write caching is a recipe for disasterous performance on most ata disks... unfortunately. better to buy a UPS and set up nut or apcupsd or something to handle shutdown. or just take your chances.
This is a mail server, I would like it to not lose mail if there is an power outage. Is there anything one can do without buying more hardware?
Nothing you can do will prevent loss of mail locally if the power goes out when the mail has gotten to memory and not disk, or the metadata has not been updated on the disk. Unless you want to hack your SMTP interface to not return an "accepted" status until you have flushed everything, get a UPS.
Hint, I looked at this for an ISP internal mailer, and performance trying to flush is so bad it's only justified for a very low volume of mail. As in legal or financial stuff. If your mail is that critical you buy expensive hardware when justified. Otherwise spend a few hundred bucks on a good UPS and install the software.
Signature · 3 lines
bill davidsen <[email protected]> CTO TMR Associates, Inc Doing interesting things with small computers since 1979