disk performance strange/low
I've got an Areca RAID Controller with 5x 1TB disks, and in the last time I often found it to be strange. Now I investigated, and found this.
Quoted 1 line
From another server I NFS mount a share, and make
dd if=/dev/sda of=/server/share/test.dd bs=1024k
so that should do big sequential writes. But the performance goes up and down, like you see in this output from "iostat -kx 5 555". You see write KB/s is 5824, 23207, 87283, 704, 40187. Up and down all the time. Could that be solved by the patch you discussed last time, or is there something else?
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-
sz avgqu-sz await svctm %util
sdb 0,00 401,20 1,20 77,09 4,78 5824,00
148,91 152,25 2595,77 12,76 99,92
avg-cpu: %user %nice %system %iowait %steal %idle
0,00 0,00 34,77 67,03 2,20 7,21
sdb 0,00 1094,40 2,20 245,80 8,80 23207,20
187,23 153,64 43,24 4,03 100,00
avg-cpu: %user %nice %system %iowait %steal %idle
0,00 0,00 41,08 56,83 0,30 15,65
sdb 0,00 295,20 6,40 791,20 25,60 87283,40
218,93 119,19 259,89 1,21 96,56
avg-cpu: %user %nice %system %iowait %steal %idle
0,00 0,00 61,45 55,92 0,50 4,62
sdb 0,00 3,60 0,00 4,60 0,00 704,80
306,43 140,01 2979,30 217,39 100,00
avg-cpu: %user %nice %system %iowait %steal %idle
0,00 0,00 21,12 75,50 0,10 10,06
sdb 0,00 1252,80 0,20 270,00 0,80 40187,20
297,47 151,38 1160,49 3,70 99,84
mfg zmi
Signature · 10 lines
// Michael Monnerie, Ing.BSc ----- http://it-management.at // Tel: 0660 / 415 65 31 .network.your.ideas. // PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import" // Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4 // Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · Michael Monnerie
I've got an Areca RAID Controller with 5x 1TB disks, and in the last time I often found it to be strange. Now I investigated, and found this.
Quoted 1 line
From another server I NFS mount a share, and make
dd if=/dev/sda of=/server/share/test.dd bs=1024k
so that should do big sequential writes. But the performance goes up and down, like you see in this output from "iostat -kx 5 555". You see write KB/s is 5824, 23207, 87283, 704, 40187. Up and down all the time. Could that be solved by the patch you discussed last time, or is there something else?
Yes, that sounds like the problem of the VM not feeding us enough pages to write. What kernel is this on? Latest 2.6.31-rc has a workaround for it.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 3 lines · On Montag 17 August 2009 Christoph Hellwig wrote:
Yes, that sounds like the problem of the VM not feeding us enough pages to write. What kernel is this on? Latest 2.6.31-rc has a workaround for it.
2.6.27.23-0.1-xen from openSUSE 11.1
I can't switch as I use it with the free Xen. Thanks for the info.
mfg zmi
Signature · 10 lines
// Michael Monnerie, Ing.BSc ----- http://it-management.at // Tel: 0660 / 415 65 31 .network.your.ideas. // PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import" // Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4 // Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 7 lines · Michael Monnerie
Quoted 3 lines · On Montag 17 August 2009 Christoph Hellwig wrote:
Yes, that sounds like the problem of the VM not feeding us enough pages to write. ?What kernel is this on? ?Latest 2.6.31-rc has a workaround for it.
2.6.27.23-0.1-xen from openSUSE 11.1
I can't switch as I use it with the free Xen. Thanks for the info.
Try backporting commit c8a4051c3731b6db224482218cfd535ab9393ff8 from mainline - it should apply just fine to a 2.6.27-ish kernel.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 11 lines · On Dienstag 18 August 2009 Christoph Hellwig wrote:
Quoted 7 lines · Michael Monnerie
Quoted 3 lines · On Montag 17 August 2009 Christoph Hellwig wrote:
Yes, that sounds like the problem of the VM not feeding us enough pages to write. ?What kernel is this on? ?Latest 2.6.31-rc has a workaround for it.
2.6.27.23-0.1-xen from openSUSE 11.1
I can't switch as I use it with the free Xen. Thanks for the info.Try backporting commit
c8a4051c3731b6db224482218cfd535ab9393ff8from mainline - it should apply just fine to a 2.6.27-ish kernel.
Thank you. I'm not a patcher, could you tell me how to do that?
mfg zmi
Signature · 5 lines
// Michael Monnerie, Ing.BSc ----- http://it-management.at // Tel: 0660 / 415 65 31 .network.your.ideas. // PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import" // Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4 // Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Christoph Hellwig
Reviewed-by Felix BlyakherQuoted 7 lines · Michael Monnerie
Quoted 6 lines · On Dienstag 18 August 2009 Christoph Hellwig wrote:
Quoted 5 lines · Michael Monnerie
Quoted 3 lines · On Montag 17 August 2009 Christoph Hellwig wrote:
Yes, that sounds like the problem of the VM not feeding us enough pages to write. ?What kernel is this on? ?Latest 2.6.31-rc has a workaround for it.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
2.6.27.23-0.1-xen from openSUSE 11.1
I can't switch as I use it with the free Xen. Thanks for the info.Try backporting commit
c8a4051c3731b6db224482218cfd535ab9393ff8from mainline - it should apply just fine to a 2.6.27-ish kernel.Thank you. I'm not a patcher, could you tell me how to do that?
Just take the patch I attached and apply it using
patch -p1 < xfs-nr-to-write.diff
if you want to make sure it applies cleanly do a
patch -p1 --dry-run < xfs-nr-to-write.diff
beforee to check if it doesn't cause any reject.
After that rebuild your kernel.
commit c8a4051c3731b6db224482218cfd535ab9393ff8
Author: Eric Sandeen <[email protected]> Date: Fri Jul 31 00:02:17 2009 -0500
xfs: bump up nr_to_write in xfs_vm_writepage
VM calculation for nr_to_write seems off. Bump it way up, this gets simple streaming writes zippy again. To be reviewed again after Jens' writeback changes.
fs/xfs/linux-2.6/xfs_aops.c +8 -0
@@ -1268,6 +1268,14 @@ xfs_vm_writepage(if (!page_has_buffers (page))create_empty_buffers (page,1 << inode->i_blkbits,0 ); + +/* +* VM calculation for nr_to_write seems off. Bump it way +* up, this gets simple streaming writes zippy again. +* To be reviewed again after Jens' writeback changes. +*/ + wbc->nr_to_write *=4 ; +/* * Convert delayed allocate, unwritten or unmapped space * to real space and flush out to disk.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs