raidd5:Only move IO_THRESHOLD stripes from delay_list to hold_list once.
PATCH 1/1 raidd5:Only move IO_THRESHOLD stripes from delay_list to hold_list once.
To improve write perfomance by decreasing the preread stripe,only move IO_THRESHOLD stripes from delay_list to hold_list once.
Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.
At default condition: speed is 95MB/s.
At the condition of preread_bypass_threshold was equal zero:speed is 105MB/s. Using this patch:speed is 123MB/s.
If preread_bypass_threshold was zero,the performance will be better,but not better than this patch.
I think maybe two reason:
1:If bio is REQ_SYNC
2:In function __get_priority_stripe():
Quoted 4 lines
Quoted 4 lines
} else if (!list_empty(&conf->hold_list) &&
((conf->bypass_threshold && conf->bypass_count > conf->bypass_threshold) || atomic_read(&conf->pending_full_writes) == 0)) {
Preread_bypass_threshold is one condition of getting stripe from hold_list.So only control the number of hold_list can get better performance.
--- drivers/md/raid5.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
drivers/md/raid5.c +3 -0
@@ -3662,6 +3662,7 @@ finish:static void raid5_activate_delayed (struct r5conf *conf) { +int count =0 ;if (atomic_read (&conf->preread_active_stripes) <IO_THRESHOLD ) {while (!list_empty (&conf->delayed_list)) {struct list_head *l = conf->delayed_list.next;
@@ -3672,6 +3673,8 @@ static void raid5_activate_delayed(struct r5conf *conf)if (!test_and_set_bit (STRIPE_PREREAD_ACTIVE , &sh->state))atomic_inc (&conf->preread_active_stripes);list_add_tail (&sh->lru, &conf->hold_list); +if (++count >=IO_THRESHOLD ) +break ; } } }
Signature · 1 line
1.7.5.4
[ adding Shaohua ]
Quoted 15 lines · majianpeng
To improve write perfomance by decreasing the preread stripe,only move
IO_THRESHOLDstripes fromdelay_listtohold_listonce.Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.At default condition: speed is 95MB/s.
At the condition ofpreread_bypass_thresholdwas equal zero:speed is 105MB/s. Using this patch:speed is 123MB/s.If
preread_bypass_thresholdwas zero,the performance will be better,but not better than this patch.
I think maybe two reason:
1:If bio isREQ_SYNC
2:In function__get_priority_stripe():
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
} else if (!list_empty(&conf->hold_list) &&
((conf->bypass_threshold && conf->bypass_count > conf->bypass_threshold) || atomic_read(&conf->pending_full_writes) == 0)) {
Preread_bypass_thresholdis one condition of getting stripe from hold_list.So only control the number ofhold_listcan get better performance.
So this is a pretty obvious tradeoff of increased latency for improved throughput. Any idea how much this change affects latency? Especially in the fast device case?
On 2012-07-14 07:56 Dan Williams <[email protected]> Wrote:
Quoted 22 lines
[ adding Shaohua ]
Quoted 19 lines · majianpeng
To improve write perfomance by decreasing the preread stripe,only move
IO_THRESHOLDstripes fromdelay_listtohold_listonce.Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.At default condition: speed is 95MB/s.
At the condition ofpreread_bypass_thresholdwas equal zero:speed is 105MB/s. Using this patch:speed is 123MB/s.If
preread_bypass_thresholdwas zero,the performance will be better,but not better than this patch.
I think maybe two reason:
1:If bio isREQ_SYNC
2:In function__get_priority_stripe():Quoted 4 lines
Quoted 4 lines
} else if (!list_empty(&conf->hold_list) &&
((conf->bypass_threshold && conf->bypass_count > conf->bypass_threshold) || atomic_read(&conf->pending_full_writes) == 0)) {
Quoted 7 lines
Quoted 3 lines
Preread_bypass_thresholdis one condition of getting stripe from hold_list.So only control the number ofhold_listcan get better performance.So this is a pretty obvious tradeoff of increased latency for improved throughput. Any idea how much this change affects latency? Especially in the fast device case?
I did not think the latency.If it only fetch preread_bypass_threshold stripes from delay_list to host_list,the latency can be control by userspace.
The code like :
static void raid5_activate_delayed(struct r5conf *conf)
{
+ int count = 0;
if (atomic_read(&conf->preread_active_stripes) < IO_THRESHOLD) { while (!list_empty(&conf->delayed_list)) {
struct list_head *l = conf->delayed_list.next;
@@ -3672,6 +3673,8 @@ static void raid5_activate_delayed(struct r5conf *conf)
if (!test_and_set_bit(STRIPE_PREREAD_ACTIVE, &sh->state)) atomic_inc(&conf->preread_active_stripes); list_add_tail(&sh->lru, &conf->hold_list);
+ if (++count >= conf->preread_active_stripes)
+ break;
}
Quoted 41 lines · On Fri, 13 Jul 2012 18:31:11 +0800 majianpeng <[email protected]> wrote:
To improve write perfomance by decreasing the preread stripe,only move
IO_THRESHOLDstripes fromdelay_listtohold_listonce.Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.At default condition: speed is 95MB/s.
At the condition ofpreread_bypass_thresholdwas equal zero:speed is 105MB/s. Using this patch:speed is 123MB/s.If
preread_bypass_thresholdwas zero,the performance will be better,but not better than this patch.
I think maybe two reason:
1:If bio isREQ_SYNC
2:In function__get_priority_stripe():Quoted 4 lines
Quoted 4 lines
} else if (!list_empty(&conf->hold_list) &&
((conf->bypass_threshold && conf->bypass_count > conf->bypass_threshold) || atomic_read(&conf->pending_full_writes) == 0)) {
Preread_bypass_thresholdis one condition of getting stripe from hold_list.So only control the number ofhold_listcan get better performance.Signed-off-by Jianpeng Ma <[email protected]>--- drivers/md/raid5.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)drivers/md/raid5.c +1
-0@@ -3662,6 +3662,7 @@ finish:static void raid5_activate_delayed (struct r5conf *conf) { +int count =0 ;if (atomic_read (&conf->preread_active_stripes) <IO_THRESHOLD ) {while (!list_empty (&conf->delayed_list)) {struct list_head *l = conf->delayed_list.next;
drivers/md/raid5.c static void raid5_activate_delayed(struct r5conf *conf) · line 3680
Quoted 8 lines
drivers/md/raid5.c +2
-0@@ -3672,6 +3673,8 @@ static void raid5_activate_delayed(struct r5conf *conf)if (!test_and_set_bit (STRIPE_PREREAD_ACTIVE , &sh->state))atomic_inc (&conf->preread_active_stripes);list_add_tail (&sh->lru, &conf->hold_list); +if (++count >=IO_THRESHOLD ) +break ; } } }
I tried this patch - against my current for-next tree - on my own modest hardware and could not measure any difference in write throughput.
Maybe some other patch has fixed something.
However it is still reading a lot during a write-only test and that is not ideal. It would be nice if we could arrange that it didn't read at all.
NeilBRown
On 2012-07-16 15:46 NeilBrown <[email protected]> Wrote:
Quoted 56 lines · On Fri, 13 Jul 2012 18:31:11 +0800 majianpeng <[email protected]> wrote:
Quoted 42 lines
To improve write perfomance by decreasing the preread stripe,only move
IO_THRESHOLDstripes fromdelay_listtohold_listonce.Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.At default condition: speed is 95MB/s.
At the condition ofpreread_bypass_thresholdwas equal zero:speed is 105MB/s. Using this patch:speed is 123MB/s.If
preread_bypass_thresholdwas zero,the performance will be better,but not better than this patch.
I think maybe two reason:
1:If bio isREQ_SYNC
2:In function__get_priority_stripe():Quoted 4 lines
Quoted 4 lines
} else if (!list_empty(&conf->hold_list) &&
((conf->bypass_threshold && conf->bypass_count > conf->bypass_threshold) || atomic_read(&conf->pending_full_writes) == 0)) {
Preread_bypass_thresholdis one condition of getting stripe from hold_list.So only control the number ofhold_listcan get better performance.Signed-off-by Jianpeng Ma <[email protected]>--- drivers/md/raid5.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)drivers/md/raid5.c +1
-0@@ -3662,6 +3662,7 @@ finish:static void raid5_activate_delayed (struct r5conf *conf) { +int count =0 ;if (atomic_read (&conf->preread_active_stripes) <IO_THRESHOLD ) {while (!list_empty (&conf->delayed_list)) {struct list_head *l = conf->delayed_list.next;drivers/md/raid5.c static void raid5_activate_delayed(struct r5conf *conf) · line 3680
Quoted 8 lines
drivers/md/raid5.c +2
-0@@ -3672,6 +3673,8 @@ static void raid5_activate_delayed(struct r5conf *conf)if (!test_and_set_bit (STRIPE_PREREAD_ACTIVE , &sh->state))atomic_inc (&conf->preread_active_stripes);list_add_tail (&sh->lru, &conf->hold_list); +if (++count >=IO_THRESHOLD ) +break ; } } }I tried this patch - against my current for-next tree - on my own modest hardware and could not measure any difference in write throughput.
Quoted 5 lines
Maybe some other patch has fixed something.
However it is still reading a lot during a write-only test and that is not ideal. It would be nice if we could arrange that it didn't read at all.
By compare to kernel 2.6.18/2.6.32, there are not any reading. So i think it should more work to do .
Quoted 2 lines
NeilBRown