Device utilization with RAID-1
hi,
I didn't want to complain in general about SW RAID-1 performance. I simply think something is wrong with my setup and I have currently no idea how to improve.
The basic questions (where I did not find an answer, neither in FAQ's nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization (around 20%) without any noticeable actions on the computer?
b) Should (as long as no resync happens) the state of mdadm active or clean?
cheers,
harald
well, I have only 2 hard drives and no space for more..
Am 16.08.2011 03:29, schrieb Roberto Spadim:
Quoted 51 lines
try raid10 far layout
2011/8/15 Harald Nikolisin <[email protected]
<mailto:[email protected]>>Since a long time I'm unhappy with the performance of my RAID-1 system. Investigation with atop and iostat unveils that the disk utilization is always on a certain level although nothing happens on the system. In the case of reading or writing files the utilization boosts always to 100% for a long time. Very ugly examples are "Firefox starting" or "zypper updates". That is snapshot of the output of iostat: Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,33 43,18 33,32 24,43 sdb 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,35 45,59 39,73 29,13 md0 0,00 0,00 0,00 0,67 0,00 5,33 8,00 0,00 0,00 0,00 0,00 md1 0,00 0,00 0,00 0,33 0,00 5,33 16,00 0,00 0,00 0,00 0,00 md2 0,00 0,00 0,00 0,33 0,00 1,00 3,00 0,00 0,00 0,00 0,00 md3 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 md4 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 md5 0,00 0,00 0,00 0,33 0,00 0,67 2,00 0,00 0,00 0,00 0,00 I checked with mdadm if a resync happens or so, but this is not the case. The state says "active" on all RAID devices - btw. what is the difference to "clean" ? thanks for any hints, harald -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] <mailto:[email protected]> More majordomo info at http://vger.kernel.org/majordomo-info.htmlSignature · 2 lines
Roberto Spadim Spadim Technology / SPAEmpresarial
Quoted 10 lines · On Thu, 18 Aug 2011 02:26:17 +0200 Harald Nikolisin <[email protected]> wrote:
hi,
I didn't want to complain in general about SW RAID-1 performance. I simply think something is wrong with my setup and I have currently no idea how to improve.
The basic questions (where I did not find an answer, neither in FAQ's nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization (around 20%) without any noticeable actions on the computer?
No. If the array is resyncing or recovering then you would expect utilization for as many hours as it takes - but that would show in /proc/mdstat.
Quoted 1 line
b) Should (as long as no resync happens) the state of mdadm active or clean?
If anything has been written to the device in the last 200msec (including e.g. access time updates) then expect it to be 'active'.
If nothing has been written for 200msecc or more, then expect it to be clean.
If you crash while it is active, a resync is needed. If you crash while it is clean, no resync is needed. If you don't crash at all .... that is best :-)
NeilBrown
Quoted 64 lines
cheers,
haraldwell, I have only 2 hard drives and no space for more..
Am 16.08.2011 03:29, schrieb Roberto Spadim:
Quoted 37 lines
try raid10 far layout
2011/8/15 Harald Nikolisin <[email protected]
<mailto:[email protected]>>Since a long time I'm unhappy with the performance of my RAID-1 system. Investigation with atop and iostat unveils that the disk utilization is always on a certain level although nothing happens on the system. In the case of reading or writing files the utilization boosts always to 100% for a long time. Very ugly examples are "Firefox starting" or "zypper updates". That is snapshot of the output of iostat: Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,33 43,18 33,32 24,43 sdb 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,35 45,59 39,73 29,13 md0 0,00 0,00 0,00 0,67 0,00 5,33 8,00 0,00 0,00 0,00 0,00 md1 0,00 0,00 0,00 0,33 0,00 5,33 16,00 0,00 0,00 0,00 0,00 md2 0,00 0,00 0,00 0,33 0,00 1,00 3,00 0,00 0,00 0,00 0,00 md3 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 md4 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 md5 0,00 0,00 0,00 0,33 0,00 0,67 2,00 0,00 0,00 0,00 0,00 I checked with mdadm if a resync happens or so, but this is not the case. The state says "active" on all RAID devices - btw. what is the difference to "clean" ?Quoted 14 lines
thanks for any hints, harald -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] <mailto:[email protected]> More majordomo info at http://vger.kernel.org/majordomo-info.htmlSignature · 2 lines
Roberto Spadim Spadim Technology / SPAEmpresarial--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Am 18.08.2011 03:42, schrieb NeilBrown:
Quoted 24 lines
Quoted 10 lines · On Thu, 18 Aug 2011 02:26:17 +0200 Harald Nikolisin <[email protected]> wrote:
hi,
I didn't want to complain in general about SW RAID-1 performance. I simply think something is wrong with my setup and I have currently no idea how to improve.
The basic questions (where I did not find an answer, neither in FAQ's nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization (around 20%) without any noticeable actions on the computer?No. If the array is resyncing or recovering then you would expect utilization for as many hours as it takes - but that would show in
/proc/mdstat.Quoted 1 line
b) Should (as long as no resync happens) the state of mdadm active or clean?
If anything has been written to the device in the last 200msec (including e.g. access time updates) then expect it to be 'active'.
If nothing has been written for 200msecc or more, then expect it to be clean.
Quoted 6 lines
If you crash while it is active, a resync is needed. If you crash while it is clean, no resync is needed. If you don't crash at all .... that is best :-)
NeilBrown
I second that ;) Have you checked the SMART-Attributes of your disks, are they still OK? But if they weren't, you wouldn't see that they're a bit more busy, you'd only feel it from bad performance.
Indeed I think you need to find out which processes create your I/O load, as it seems to be kind of a badly configured service/daemon which slows down your whole computer that way... It's probably a good idea to start with dstat and a wide screen :)
Stefan
Quoted 71 lines
Quoted 62 lines
cheers,
haraldwell, I have only 2 hard drives and no space for more..
Am 16.08.2011 03:29, schrieb Roberto Spadim:
Quoted 37 lines
try raid10 far layout
2011/8/15 Harald Nikolisin <[email protected]
<mailto:[email protected]>>Since a long time I'm unhappy with the performance of my RAID-1 system. Investigation with atop and iostat unveils that the disk utilization is always on a certain level although nothing happens on the system. In the case of reading or writing files the utilization boosts always to 100% for a long time. Very ugly examples are "Firefox starting" or "zypper updates". That is snapshot of the output of iostat: Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,33 43,18 33,32 24,43 sdb 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,35 45,59 39,73 29,13 md0 0,00 0,00 0,00 0,67 0,00 5,33 8,00 0,00 0,00 0,00 0,00 md1 0,00 0,00 0,00 0,33 0,00 5,33 16,00 0,00 0,00 0,00 0,00 md2 0,00 0,00 0,00 0,33 0,00 1,00 3,00 0,00 0,00 0,00 0,00 md3 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 md4 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 md5 0,00 0,00 0,00 0,33 0,00 0,67 2,00 0,00 0,00 0,00 0,00 I checked with mdadm if a resync happens or so, but this is not the case. The state says "active" on all RAID devices - btw. what is the difference to "clean" ?Quoted 14 lines
thanks for any hints, harald -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] <mailto:[email protected]> More majordomo info at http://vger.kernel.org/majordomo-info.htmlSignature · 2 lines
Roberto Spadim Spadim Technology / SPAEmpresarialQuoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 20 lines · NeilBrown
Quoted 10 lines · On Thu, 18 Aug 2011 02:26:17 +0200 Harald Nikolisin <[email protected]> wrote:
hi,
I didn't want to complain in general about SW RAID-1 performance. I simply think something is wrong with my setup and I have currently no idea how to improve.
The basic questions (where I did not find an answer, neither in FAQ's nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization (around 20%) without any noticeable actions on the computer?No. If the array is resyncing or recovering then you would expect utilization for as many hours as it takes - but that would show in
/proc/mdstat.Quoted 1 line
b) Should (as long as no resync happens) the state of mdadm active or clean?
Quoted 7 lines
If anything has been written to the device in the last 200msec (including e.g. access time updates) then expect it to be 'active'.
If nothing has been written for 200msecc or more, then expect it to be clean.If you crash while it is active, a resync is needed. If you crash while it is clean, no resync is needed. If you don't crash at all .... that is best :-)
I think this info should be wikified if not yet.
btw, I've experimented a bit on my /boot array (it doesn't being updated, checked with iostat ), and:
root@m2:~# for i in {1..5};do mdname="md0"; echo "iteration $i";
(mdadm --detail /dev/$mdname|grep 'State ';cat
/sys/block/$mdname/md/array_state;grep "$mdname :" /proc/mdstat);sleep
1;done
iteration 1
State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 2
State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 3
State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 4
State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 5
State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]so, mdadm --detail & array_state shows array is "clean", while /proc/mdstat shows array is "active" (no reads/writes happen).
Some value is lieing or being misunderdstanded by me...
Quoted 73 lines
Quoted 63 lines
cheers,
haraldwell, I have only 2 hard drives and no space for more..
Am 16.08.2011 03:29, schrieb Roberto Spadim:
Quoted 48 lines
try raid10 far layout
2011/8/15 Harald Nikolisin <[email protected]
<mailto:[email protected]>>Since a long time I'm unhappy with the performance of my RAID-1 system. Investigation with atop and iostat unveils that the disk utilization is always on a certain level although nothing happens on the system. In the case of reading or writing files the utilization boosts always to 100% for a long time. Very ugly examples are "Firefox starting" or "zypper updates".
That is snapshot of the output of iostat:Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,33 43,18 33,32 24,43
sdb 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,35 45,59 39,73 29,13
md0 0,00 0,00 0,00 0,67 0,00 5,33 8,00 0,00 0,00 0,00 0,00
md1 0,00 0,00 0,00 0,33 0,00 5,33 16,00 0,00 0,00 0,00 0,00
md2 0,00 0,00 0,00 0,33 0,00 1,00 3,00 0,00 0,00 0,00 0,00
md3 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00
md4 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00
md5 0,00 0,00 0,00 0,33 0,00 0,67 2,00 0,00 0,00 0,00 0,00I checked with mdadm if a resync happens or so, but this is not the case. The state says "active" on all RAID devices - btw. what is the difference to "clean" ?
thanks for any hints,
harald
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] <mailto:[email protected]>
More majordomo info at http://vger.kernel.org/majordomo-info.htmlQuoted 3 lines
--
Roberto Spadim
Spadim Technology / SPAEmpresarialQuoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 6 lines
Best regards, [COOLCOLD-RIPN] -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 58 lines · On Thu, 18 Aug 2011 17:44:19 +0400 CoolCold <[email protected]> wrote:
Quoted 20 lines · NeilBrown
Quoted 10 lines · On Thu, 18 Aug 2011 02:26:17 +0200 Harald Nikolisin <[email protected]> wrote:
hi,
I didn't want to complain in general about SW RAID-1 performance. I simply think something is wrong with my setup and I have currently no idea how to improve.
The basic questions (where I did not find an answer, neither in FAQ's nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization (around 20%) without any noticeable actions on the computer?No. If the array is resyncing or recovering then you would expect utilization for as many hours as it takes - but that would show in
/proc/mdstat.Quoted 1 line
b) Should (as long as no resync happens) the state of mdadm active or clean?
Quoted 7 lines
If anything has been written to the device in the last 200msec (including e.g. access time updates) then expect it to be 'active'.
If nothing has been written for 200msecc or more, then expect it to be clean.If you crash while it is active, a resync is needed. If you crash while it is clean, no resync is needed. If you don't crash at all .... that is best :-)
I think this info should be wikified if not yet.
btw, I've experimented a bit on my /boot array (it doesn't being updated, checked with iostat ), and:
root@m2:~# for i in {1..5};do mdname="md0"; echo "iteration $i"; (mdadm --detail /dev/$mdname|grep 'State ';cat /sys/block/$mdname/md/array_state;grep "$mdname :" /proc/mdstat);sleep 1;done iteration 1 State : clean clean md0 : active raid1 sda1[0] sdb1[1] iteration 2 State : clean clean md0 : active raid1 sda1[0] sdb1[1] iteration 3 State : clean clean md0 : active raid1 sda1[0] sdb1[1] iteration 4 State : clean clean md0 : active raid1 sda1[0] sdb1[1] iteration 5 State : clean clean md0 : active raid1 sda1[0] sdb1[1]
Quoted 4 lines
so, mdadm --detail &
array_stateshows array is "clean", while/proc/mdstatshows array is "active" (no reads/writes happen).Some value is lieing or being misunderdstanded by me...
In mdstat you have 'active' or 'inactive'. You cannot access an array at all until it is active. If you are assembling an array bit by bit with "mdadm -I", it will be inactive until all the devices appear. Then it will be active.
In mdadm "State :" you have 'active' or 'clean'. as described above. It used to be 'dirty' or 'clean' but people were confused by having 'dirty' arrays in normal operation. So I changed it to 'active' and now it confuses a different set of people. You just can't win can you :-)
NeilBrown
Quoted 69 lines
Quoted 69 lines
Quoted 68 lines
cheers,
haraldwell, I have only 2 hard drives and no space for more..
Am 16.08.2011 03:29, schrieb Roberto Spadim:
Quoted 48 lines
try raid10 far layout
2011/8/15 Harald Nikolisin <[email protected]
<mailto:[email protected]>>Since a long time I'm unhappy with the performance of my RAID-1 system. Investigation with atop and iostat unveils that the disk utilization is always on a certain level although nothing happens on the system. In the case of reading or writing files the utilization boosts always to 100% for a long time. Very ugly examples are "Firefox starting" or "zypper updates".
That is snapshot of the output of iostat:Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,33 43,18 33,32 24,43
sdb 0,00 0,00 0,00 7,33 0,00 43,33 5,91 0,35 45,59 39,73 29,13
md0 0,00 0,00 0,00 0,67 0,00 5,33 8,00 0,00 0,00 0,00 0,00
md1 0,00 0,00 0,00 0,33 0,00 5,33 16,00 0,00 0,00 0,00 0,00
md2 0,00 0,00 0,00 0,33 0,00 1,00 3,00 0,00 0,00 0,00 0,00
md3 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00
md4 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00 0,00
md5 0,00 0,00 0,00 0,33 0,00 0,67 2,00 0,00 0,00 0,00 0,00I checked with mdadm if a resync happens or so, but this is not the case. The state says "active" on all RAID devices - btw. what is the difference to "clean" ?
thanks for any hints,
harald
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] <mailto:[email protected]>
More majordomo info at http://vger.kernel.org/majordomo-info.htmlQuoted 3 lines
--
Roberto Spadim
Spadim Technology / SPAEmpresarial--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 6 lines · NeilBrown
..
In mdstat you have 'active' or 'inactive'. You cannot access an array at all until it is active. If you are assembling an array bit by bit with "mdadm -I", it will be inactive until all the devices appear. Then it will be active.
Quoted 6 lines
In mdadm "State :" you have 'active' or 'clean'. as described above. It used to be 'dirty' or 'clean' but people were confused by having 'dirty' arrays in normal operation. So I changed it to 'active' and now it confuses a different set of people. You just can't win can you :-)
NeilBrown
mdstat:
"Enabled" or "Disabled" perhaps?
That matches what most commercial hardware RAID interfaces use.
Signature · 3 lines
Cheers, Maurice Hilarius eMail: /[email protected]/
Quoted 15 lines · maurice
Quoted 14 lines · NeilBrown
..
In mdstat you have 'active' or 'inactive'. You cannot access an array at all
until it is active. If you are assembling an array bit by bit with "mdadm -I", it will be inactive until all the devices appear. Then it will be active.In mdadm "State :" you have 'active' or 'clean'. as described above. It used
to be 'dirty' or 'clean' but people were confused by having 'dirty' arrays in
normal operation. So I changed it to 'active' and now it confuses a different set of people. You just can't win can you :-)
Quoted 6 lines
Quoted 1 line
NeilBrown
mdstat:
"Enabled" or "Disabled" perhaps?That matches what most commercial hardware RAID interfaces use.
Does it? It sounds more like an administrative action than a current status. I would have thought "online" or "offline" - unless that means something else somewhere else.
And for mdadm state: how about "busy" and "idle"? Hmm maybe just "busy" instead of "active" or "dirty"; we don't want to start an array with --assume-idle...
Cheers,
John.