Skip to contentSkip to conversation

cfq-iosched: don't stop async queue with async requests pending

PATCH In review Shaohua Li 4 participants 17 messages Jan 13, 2010 – Jan 19, 2010 Block layer

lore
Share & tools
lore

Shaohua Li

author
#0 lore raw

PATCH 1/1 cfq-iosched: don't stop async queue with async requests pending

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

block/cfq-iosched.c +2 -1
index 918c7fd..8198079 100644
@@ -2222,7 +2222,8 @@ static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq)
 	/*
 	 * Drain async requests before we start sync IO
 	 */
-	if (cfq_should_idle(cfqd, cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])
+	if (cfq_cfqq_sync(cfqq) && cfq_should_idle(cfqd, cfqq)
+		&& cfqd->rq_in_driver[BLK_RW_ASYNC])
 		return false;
 
 	/*
#1 lore raw
Quoted 8 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

Corrado

Quoted 23 lines
block/cfq-iosched.c +2 -2
index 918c7fd..8198079 100644
@@ -2222,7 +2222,8 @@ static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq)
        /*
         * Drain async requests before we start sync IO
         */
-       if (cfq_should_idle(cfqd, cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])
+       if (cfq_cfqq_sync(cfqq) && cfq_should_idle(cfqd, cfqq)
+               && cfqd->rq_in_driver[BLK_RW_ASYNC])
                return false;
 
        /*
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Signature · 10 lines
__________________________________________________________________________

dott. Corrado Zoccolo                          mailto:[email protected]
PhD - Department of Computer Science - University of Pisa, Italy
--------------------------------------------------------------------------
The self-confidence of a warrior is not the self-confidence of the average
man. The average man seeks certainty in the eyes of the onlooker and calls
that self-confidence. The warrior seeks impeccability in his own eyes and
calls that humbleness.
                               Tales of Power - C. Castaneda
author
#2 lore raw
Quoted 10 lines · Corrado Zoccolo
Quoted 8 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

Quoted 2 lines

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

I'm considering this option too, but it appears we need make async queue idle to maintain domain time slice.

Thanks,
Shaohua

#3 lore raw
Quoted 11 lines · Shaohua Li
Quoted 10 lines · Corrado Zoccolo
Quoted 8 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

Quoted 4 lines
Quoted 2 lines

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

I'm considering this option too, but it appears we need make async queue idle to maintain domain time slice.

IMHO, we don't have to wait on async write service tree. Generally aysnc write queus contain many requests and they are not like reads where next request is expected. So idling on aysnc write service tree is waste of time and will lead to reduced throughput.

Vivek

author
#4 lore raw
Quoted 14 lines · Vivek Goyal
Quoted 13 lines · Shaohua Li
Quoted 12 lines · Corrado Zoccolo
Quoted 8 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

Quoted 7 lines
Quoted 2 lines

I'm considering this option too, but it appears we need make async queue idle to maintain domain time slice.

IMHO, we don't have to wait on async write service tree. Generally aysnc write queus contain many requests and they are not like reads where next request is expected. So idling on aysnc write service tree is waste of time and will lead to reduced throughput.

I fully agree async queue doesn't need wait. I thought the purpose we add the last queue check in cfq_should_idle is we want a service tree or a group has dedicated slice, because before the service tree/group slice is expired, new queue can jump in and if we don't idle, the new queue can only run at next slice. Not sure if I understand the code correctly.
you are the expert of iogroup. If I'm wrong, I'll be happy to change cfq_should_idle(), which is the ideal place to be changed at my first glance.

Thanks,
Shaohua

#5 lore raw
Quoted 20 lines · Shaohua Li
Quoted 11 lines · Vivek Goyal
Quoted 10 lines · Shaohua Li
Quoted 9 lines · Corrado Zoccolo
Quoted 8 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

I'm considering this option too, but it appears we need make async queue idle to maintain domain time slice.

IMHO, we don't have to wait on async write service tree. Generally aysnc write queus contain many requests and they are not like reads where next request is expected. So idling on aysnc write service tree is waste of time and will lead to reduced throughput.

Quoted 5 lines

I fully agree async queue doesn't need wait. I thought the purpose we add the last queue check in cfq_should_idle is we want a service tree or a group has dedicated slice, because before the service tree/group slice is expired, new queue can jump in and if we don't idle, the new queue can only run at next slice. Not sure if I understand the code correctly.

Hi Shaohua,

If a cfq queue is the last one in the io group, if we expire this cfqq immediately, io group will be removed from service tree. When io group gets backlogged again, it will be put at the end of service tree, so it loses its previous share. so we add the last check here from the fairness point of view.

Thanks,
Gui

Quoted 5 lines

you are the expert of iogroup. If I'm wrong, I'll be happy to change cfq_should_idle(), which is the ideal place to be changed at my first glance.

Thanks,
Shaohua

author
#6 lore raw
Quoted 27 lines · Gui Jianfeng
Quoted 20 lines · Shaohua Li
Quoted 11 lines · Vivek Goyal
Quoted 10 lines · Shaohua Li
Quoted 9 lines · Corrado Zoccolo
Quoted 8 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

I'm considering this option too, but it appears we need make async queue idle to maintain domain time slice.

IMHO, we don't have to wait on async write service tree. Generally aysnc write queus contain many requests and they are not like reads where next request is expected. So idling on aysnc write service tree is waste of time and will lead to reduced throughput.

Quoted 5 lines

I fully agree async queue doesn't need wait. I thought the purpose we add the last queue check in cfq_should_idle is we want a service tree or a group has dedicated slice, because before the service tree/group slice is expired, new queue can jump in and if we don't idle, the new queue can only run at next slice. Not sure if I understand the code correctly.

Quoted 6 lines

Hi Shaohua,

If a cfq queue is the last one in the io group, if we expire this cfqq immediately, io group will be removed from service tree. When io group gets backlogged again, it will be put at the end of service tree, so it loses its previous share. so we add the last check here from the fairness point of view.

ya, this is what I'm understanding. So we can't return false for async queue in cfq_should_idle if the queue is the last one of service tree.

Thanks,
Shaohua

#7 lore raw
Quoted 27 lines · Shaohua Li
Quoted 26 lines · Gui Jianfeng
Quoted 25 lines · Shaohua Li
Quoted 11 lines · Vivek Goyal
Quoted 10 lines · Shaohua Li
Quoted 9 lines · Corrado Zoccolo
Quoted 8 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

I'm considering this option too, but it appears we need make async queue idle to maintain domain time slice.

IMHO, we don't have to wait on async write service tree. Generally aysnc write queus contain many requests and they are not like reads where next request is expected. So idling on aysnc write service tree is waste of time and will lead to reduced throughput.

I fully agree async queue doesn't need wait. I thought the purpose we add the last queue check in cfq_should_idle is we want a service tree or a group has dedicated slice, because before the service tree/group slice is expired, new queue can jump in and if we don't idle, the new queue can only run at next slice. Not sure if I understand the code correctly.

Quoted 8 lines
Quoted 6 lines

Hi Shaohua,

If a cfq queue is the last one in the io group, if we expire this cfqq immediately, io group will be removed from service tree. When io group gets backlogged again, it will be put at the end of service tree, so it loses its previous share. so we add the last check here from the fairness point of view.

ya, this is what I'm understanding. So we can't return false for async queue in cfq_should_idle if the queue is the last one of service tree.

I see your point, whether can we add the extra sync queue check into cfq_should_idle for common use?

Thanks,
Gui

Quoted 6 lines

Thanks,
Shaohua

author
#8 lore raw
Quoted 34 lines · Gui Jianfeng
Quoted 33 lines · Shaohua Li
Quoted 32 lines · Gui Jianfeng
Quoted 20 lines · Shaohua Li
Quoted 11 lines · Vivek Goyal
Quoted 10 lines · Shaohua Li
Quoted 9 lines · Corrado Zoccolo
Quoted 8 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

I'm considering this option too, but it appears we need make async queue idle to maintain domain time slice.

IMHO, we don't have to wait on async write service tree. Generally aysnc write queus contain many requests and they are not like reads where next request is expected. So idling on aysnc write service tree is waste of time and will lead to reduced throughput.

Quoted 5 lines

I fully agree async queue doesn't need wait. I thought the purpose we add the last queue check in cfq_should_idle is we want a service tree or a group has dedicated slice, because before the service tree/group slice is expired, new queue can jump in and if we don't idle, the new queue can only run at next slice. Not sure if I understand the code correctly.

Hi Shaohua,

If a cfq queue is the last one in the io group, if we expire this cfqq immediately, io group will be removed from service tree. When io group gets backlogged again, it will be put at the end of service tree, so it loses its previous share. so we add the last check here from the fairness point of view.

Quoted 5 lines
Quoted 2 lines

ya, this is what I'm understanding. So we can't return false for async queue in cfq_should_idle if the queue is the last one of service tree.

I see your point, whether can we add the extra sync queue check into cfq_should_idle for common use?

yes.

Thanks,
Shaohua

#9 lore raw
In reply to Shaohua Li
Quoted 28 lines · Shaohua Li
Quoted 27 lines · Gui Jianfeng
Quoted 25 lines · Shaohua Li
Quoted 11 lines · Vivek Goyal
Quoted 10 lines · Shaohua Li
Quoted 9 lines · Corrado Zoccolo
Quoted 8 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

I'm considering this option too, but it appears we need make async queue idle to maintain domain time slice.

IMHO, we don't have to wait on async write service tree. Generally aysnc write queus contain many requests and they are not like reads where next request is expected. So idling on aysnc write service tree is waste of time and will lead to reduced throughput.

I fully agree async queue doesn't need wait. I thought the purpose we add the last queue check in cfq_should_idle is we want a service tree or a group has dedicated slice, because before the service tree/group slice is expired, new queue can jump in and if we don't idle, the new queue can only run at next slice. Not sure if I understand the code correctly.

Quoted 9 lines
Quoted 6 lines

Hi Shaohua,

If a cfq queue is the last one in the io group, if we expire this cfqq immediately, io group will be removed from service tree. When io group gets backlogged again, it will be put at the end of service tree, so it loses its previous share. so we add the last check here from the fairness point of view.

ya, this is what I'm understanding. So we can't return false for async queue in cfq_should_idle if the queue is the last one of service tree.

Yes cfq_should_idle() can check for async queue and return false.

Regarding group loosing fair share, currently all async queues are in root group and not in individual groups, so this particular change should not affect a lot. We will continue to idle on sync-idle and sync-noidle service tree. Only async service tree is the exception.

Once we introduce per group async queue in future, we shall have to come up with something else, if need be.

So keep this as a separate patch. I think in the presence of mixed workload, (readers and buffered writers), it might give little performance boost. We need to test it though.

Thanks
Vivek

#10 lore raw
Quoted 76 lines

-----Original Message-----
From: Vivek Goyal [mailto:[email protected]]
Sent: Thursday, January 14, 2010 7:09 PM
To: Li, Shaohua
Cc: Gui Jianfeng; Corrado Zoccolo; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH]cfq-iosched: don't stop async queue with async requests pending

Quoted 6 lines · Shaohua Li
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Shaohua Li
Quoted 3 lines · Vivek Goyal
Quoted 2 lines · Shaohua Li
Quoted 1 line · Corrado Zoccolo

On Wed, Jan 13, 2010 at 8:44 AM, Shaohua Li <[email protected]>

Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

My SSD speed of direct write is about 80m/s, while I test page

writeback,

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

the speed can only go to 68m/s. Below patch fixes this. It appears we missused cfq_should_idle in cfq_may_dispatch.

cfq_should_idle

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

means a queue should idle because it's seekless sync queue or

it's the last queue,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

which is to maintain service tree time slice. So it doesn't mean

the

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

last queue is always a sync queue. If the last queue is asyn

queue,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

we definitely shouldn't stop dispatch requests because of

pending async

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

requests.

An other option is that cfq_should_idle returns false for async queues, since cfq will never idle on them.

I'm considering this option too, but it appears we need make async

queue

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

idle to maintain domain time slice.

IMHO, we don't have to wait on async write service tree. Generally

aysnc

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

write queus contain many requests and they are not like reads where

next

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

request is expected. So idling on aysnc write service tree is waste

of

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

time and will lead to reduced throughput.

I fully agree async queue doesn't need wait. I thought the purpose

we add the last

Quoted 1 line
Quoted 1 line
Quoted 1 line

queue check in cfq_should_idle is we want a service tree or a group

has dedicated

Quoted 1 line
Quoted 1 line
Quoted 1 line

slice, because before the service tree/group slice is expired, new

queue can jump

Quoted 1 line
Quoted 1 line
Quoted 1 line

in and if we don't idle, the new queue can only run at next slice.

Not sure if I

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

understand the code correctly.

Hi Shaohua,

If a cfq queue is the last one in the io group, if we expire this cfqq

immediately,

Quoted 1 line
Quoted 1 line

io group will be removed from service tree. When io group gets

backlogged again, it

Quoted 1 line
Quoted 1 line

will be put at the end of service tree, so it loses its previous share.

so we add

Quoted 2 lines
Quoted 1 line

the last check here from the fairness point of view.

ya, this is what I'm understanding. So we can't return false for async

queue

Quoted 2 lines

in cfq_should_idle if the queue is the last one of service tree.

Yes cfq_should_idle() can check for async queue and return false.

Regarding group loosing fair share, currently all async queues are in root group and not in individual groups, so this particular change should not affect a lot. We will continue to idle on sync-idle and sync-noidle service tree. Only async service tree is the exception.

Quoted 6 lines

Once we introduce per group async queue in future, we shall have to come up with something else, if need be.

So keep this as a separate patch. I think in the presence of mixed workload, (readers and buffered writers), it might give little performance boost. We need to test it though.

Ok, if you thought this method doesn't break group, here is the updated patch. I'm sorry to send the attached patch, my mailbox has trouble.

From: Shaohua Li <[email protected]>
Title: cfq-iosched: don't stop async queue with async requests pending

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. As currently async queues are all in root group, no idle for async queue doesn't lose group slice fairness. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests, which will make cfq_may_dispatch able to dispatch more requests.

block/cfq-iosched.c +1 -1
index 918c7fd..ce608a0 100644
@@ -1803,7 +1803,7 @@ static bool cfq_should_idle(struct cfq_data *cfqd, struct cfq_queue *cfqq)
 	 * Otherwise, we do only if they are the last ones
 	 * in their service tree.
 	 */
-	return service_tree->count == 1;
+	return service_tree->count == 1 && cfqq_type(cfqq) != ASYNC_WORKLOAD;
 }
 
 static void cfq_arm_slice_timer(struct cfq_data *cfqd)
#11 lore raw

On Tue, Jan 19, 2010 at 08:52:01AM +0800, Li, Shaohua wrote:

[..]

Quoted 7 lines
Quoted 7 lines

Yes cfq_should_idle() can check for async queue and return false.

Regarding group loosing fair share, currently all async queues are in root group and not in individual groups, so this particular change should not affect a lot. We will continue to idle on sync-idle and sync-noidle service tree. Only async service tree is the exception.

Quoted 8 lines
Quoted 6 lines

Once we introduce per group async queue in future, we shall have to come up with something else, if need be.

So keep this as a separate patch. I think in the presence of mixed workload, (readers and buffered writers), it might give little performance boost. We need to test it though.

Ok, if you thought this method doesn't break group, here is the updated patch. I'm sorry to send the attached patch, my mailbox has trouble.

Hi Shaohua,

I did some testing on cfq group functionality and I did not see any significant impact of this patch.

I am yet to write some test cases for mixed workload testing and see the impact of this patch. Will get back to you soon.

Thanks
Vivek

#12 lore raw
In reply to Shaohua Li
Quoted 16 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

block/cfq-iosched.c +0 -0
index 918c7fd..8198079 100644

block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) · line 2227

Quoted 7 lines
block/cfq-iosched.c +2 -1
@@ -2222,7 +2222,8 @@ static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq)
 	/*
 	 * Drain async requests before we start sync IO
 	 */
-	if (cfq_should_idle(cfqd, cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])
+	if (cfq_cfqq_sync(cfqq) && cfq_should_idle(cfqd, cfqq)
+		&& cfqd->rq_in_driver[BLK_RW_ASYNC])
 		return false;

So are we driving queue depth as 1 when pure buffered writes are going on? Because in that case service_tree->count=1 and cfq_should_idle() will return 1 and looks like we will not dispatch next write till previous write is over?

A general question. Why do we need to drain async requests before we start sync IO? How does that help?

A related question, even if we have to do that, why do we check for cfq_should_idle()? Why can't we just do following.

	if (cfq_cfqq_sync(cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])

Thanks
Vivek

#13 lore raw
Quoted 21 lines · Vivek Goyal
Quoted 20 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

block/cfq-iosched.c +0 -1
index 918c7fd..8198079 100644
@@ -2222,7 +2222,8 @@ static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq)
       /*
        * Drain async requests before we start sync IO
        */
-     if (cfq_should_idle(cfqd, cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])
Quoted 8 lines

block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) · line 2227

Quoted 3 lines
block/cfq-iosched.c +2 -0
+     if (cfq_cfqq_sync(cfqq) && cfq_should_idle(cfqd, cfqq)
+             && cfqd->rq_in_driver[BLK_RW_ASYNC])
               return false;

So are we driving queue depth as 1 when pure buffered writes are going on? Because in that case service_tree->count=1 and cfq_should_idle() will return 1 and looks like we will not dispatch next write till previous write is over?

Yes, it seems so. It has to be fixed.

Quoted 9 lines

A general question. Why do we need to drain async requests before we start sync IO? How does that help?

A related question, even if we have to do that, why do we check for cfq_should_idle()? Why can't we just do following.

       if (cfq_cfqq_sync(cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])

This would wait also for seeky queues. I think we drain to avoid disrupting a sync
stream with far writes, but it is not needed when the queues are already seeky.

Thanks,
Corrado

Quoted 8 lines

Thanks
Vivek
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

#14 lore raw
Quoted 37 lines · Corrado Zoccolo
Quoted 21 lines · Vivek Goyal
Quoted 20 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

block/cfq-iosched.c +0 -1
index 918c7fd..8198079 100644
@@ -2222,7 +2222,8 @@ static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq)
       /*
        * Drain async requests before we start sync IO
        */
-     if (cfq_should_idle(cfqd, cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])
Quoted 8 lines

block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) · line 2227

Quoted 3 lines
block/cfq-iosched.c +2 -0
+     if (cfq_cfqq_sync(cfqq) && cfq_should_idle(cfqd, cfqq)
+             && cfqd->rq_in_driver[BLK_RW_ASYNC])
               return false;

So are we driving queue depth as 1 when pure buffered writes are going on? Because in that case service_tree->count=1 and cfq_should_idle() will return 1 and looks like we will not dispatch next write till previous write is over?

Yes, it seems so. It has to be fixed.

Quoted 4 lines

A general question. Why do we need to drain async requests before we start sync IO? How does that help?

Quoted 8 lines
Quoted 5 lines

A related question, even if we have to do that, why do we check for cfq_should_idle()? Why can't we just do following.

       if (cfq_cfqq_sync(cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])

This would wait also for seeky queues. I think we drain to avoid disrupting a sync
stream with far writes, but it is not needed when the queues are already seeky.

Ok, that makes sense. But again the conflict is single disk vs RAID. RAID array will most likely be doing some write caching or writes and reads might be travelling to different disks hence sequential reads should not be significantly impacted. Another case to optimize if we can reliably identify RAIDs.

Thanks
Vivek

#15 lore raw
In reply to Shaohua Li
Quoted 16 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

block/cfq-iosched.c +0 -0
index 918c7fd..8198079 100644

block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) · line 2227

Quoted 7 lines
block/cfq-iosched.c +2 -1
@@ -2222,7 +2222,8 @@ static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq)
 	/*
 	 * Drain async requests before we start sync IO
 	 */
-	if (cfq_should_idle(cfqd, cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])
+	if (cfq_cfqq_sync(cfqq) && cfq_should_idle(cfqd, cfqq)
+		&& cfqd->rq_in_driver[BLK_RW_ASYNC])
 		return false;

It seems only sync queue could open the idle window, so i don't think we really need this.

Thanks,
Gui

author
#16 lore raw
Quoted 12 lines · Gui Jianfeng
Quoted 11 lines · Shaohua Li

My SSD speed of direct write is about 80m/s, while I test page writeback, the speed can only go to 68m/s. Below patch fixes this.
It appears we missused cfq_should_idle in cfq_may_dispatch. cfq_should_idle means a queue should idle because it's seekless sync queue or it's the last queue, which is to maintain service tree time slice. So it doesn't mean the last queue is always a sync queue. If the last queue is asyn queue, we definitely shouldn't stop dispatch requests because of pending async requests.

Quoted 14 lines

block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) · line 2227

Quoted 12 lines
block/cfq-iosched.c +2 -1
index 918c7fd..8198079 100644
@@ -2222,7 +2222,8 @@ static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq)
 	/*
 	 * Drain async requests before we start sync IO
 	 */
-	if (cfq_should_idle(cfqd, cfqq) && cfqd->rq_in_driver[BLK_RW_ASYNC])
+	if (cfq_cfqq_sync(cfqq) && cfq_should_idle(cfqd, cfqq)
+		&& cfqd->rq_in_driver[BLK_RW_ASYNC])
 		return false;

It seems only sync queue could open the idle window, so i don't think we really need this.

the purpose here isn't to open idle window. with it, we could send more requests instead of wait previous request finish. Vivek's mail explains the behavior correctly.

Thanks,
Shaohua