Skip to contentSkip to conversation

block: Make CFQ default to IOPS mode on SSDs

PATCH In review Tahsin Erdogan 3 participants 9 messages May 19, 2015 – Jun 10, 2015 Block layer

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 block: Make CFQ default to IOPS mode on SSDs

CFQ idling causes reduced IOPS throughput on non-rotational disks. Since disk head seeking is not applicable to SSDs, it doesn't really help performance by anticipating future near-by IO requests.

By turning off idling (and switching to IOPS mode), we allow other processes to dispatch IO requests down to the driver and so increase IO throughput.

Following FIO benchmark results were taken on a cloud SSD offering with idling on and off:

Idling iops avg-lat(ms) stddev bw
------------------------------------------------------

    On     7054    90.107         38.697     28217KB/s
   Off    29255    21.836         11.730    117022KB/s
fio --name=temp --size=100G --time_based --ioengine=libaio \
    --randrepeat=0 --direct=1 --invalidate=1 --verify=0 \
    --verify_fatal=0 --rw=randread --blocksize=4k --group_reporting=1 \
    --filename=/dev/sdb --runtime=10 --iodepth=64 --numjobs=10

And the following is from a local SSD run:

Idling iops avg-lat(ms) stddev bw
------------------------------------------------------

    On    19320    33.043         14.068     77281KB/s
   Off    21626    29.465         12.662     86507KB/s
fio --name=temp --size=5G --time_based --ioengine=libaio \
    --randrepeat=0 --direct=1 --invalidate=1 --verify=0 \
    --verify_fatal=0 --rw=randread --blocksize=4k --group_reporting=1 \
    --filename=/fio_data --runtime=10 --iodepth=64 --numjobs=10
---
 block/cfq-iosched.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block/cfq-iosched.c +1 -1
index 5da8e6e..402be01 100644
@@ -4460,7 +4460,7 @@ static int cfq_init_queue(struct request_queue *q, struct elevator_type *e)
 	cfqd->cfq_slice[1] = cfq_slice_sync;
 	cfqd->cfq_target_latency = cfq_target_latency;
 	cfqd->cfq_slice_async_rq = cfq_slice_async_rq;
-	cfqd->cfq_slice_idle = cfq_slice_idle;
+	cfqd->cfq_slice_idle = blk_queue_nonrot(q) ? 0 : cfq_slice_idle;
 	cfqd->cfq_group_idle = cfq_group_idle;
 	cfqd->cfq_latency = 1;
 	cfqd->hw_tag = -1;
Signature · 1 line
2.2.0.rc0.207.ga3a616c
#1 lore raw
Quoted 53 lines · Tahsin Erdogan

CFQ idling causes reduced IOPS throughput on non-rotational disks. Since disk head seeking is not applicable to SSDs, it doesn't really help performance by anticipating future near-by IO requests.

By turning off idling (and switching to IOPS mode), we allow other processes to dispatch IO requests down to the driver and so increase IO throughput.

Following FIO benchmark results were taken on a cloud SSD offering with idling on and off:

Idling iops avg-lat(ms) stddev bw
------------------------------------------------------

    On     7054    90.107         38.697     28217KB/s
   Off    29255    21.836         11.730    117022KB/s
fio --name=temp --size=100G --time_based --ioengine=libaio \
    --randrepeat=0 --direct=1 --invalidate=1 --verify=0 \
    --verify_fatal=0 --rw=randread --blocksize=4k --group_reporting=1 \
    --filename=/dev/sdb --runtime=10 --iodepth=64 --numjobs=10

And the following is from a local SSD run:

Idling iops avg-lat(ms) stddev bw
------------------------------------------------------

    On    19320    33.043         14.068     77281KB/s
   Off    21626    29.465         12.662     86507KB/s
fio --name=temp --size=5G --time_based --ioengine=libaio \
    --randrepeat=0 --direct=1 --invalidate=1 --verify=0 \
    --verify_fatal=0 --rw=randread --blocksize=4k --group_reporting=1 \
    --filename=/fio_data --runtime=10 --iodepth=64 --numjobs=10
---
 block/cfq-iosched.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block/cfq-iosched.c +1 -2
index 5da8e6e..402be01 100644
@@ -4460,7 +4460,7 @@ static int cfq_init_queue(struct request_queue *q, struct elevator_type *e)
        cfqd->cfq_slice[1] = cfq_slice_sync;
        cfqd->cfq_target_latency = cfq_target_latency;
        cfqd->cfq_slice_async_rq = cfq_slice_async_rq;
-       cfqd->cfq_slice_idle = cfq_slice_idle;
+       cfqd->cfq_slice_idle = blk_queue_nonrot(q) ? 0 : cfq_slice_idle;
        cfqd->cfq_group_idle = cfq_group_idle;
        cfqd->cfq_latency = 1;
        cfqd->hw_tag = -1;
--
Quoted 2 lines

2.2.0.rc0.207.ga3a616c

Ping...

#2 lore raw
Quoted 50 lines · Tahsin Erdogan
Quoted 49 lines · Tahsin Erdogan

CFQ idling causes reduced IOPS throughput on non-rotational disks. Since disk head seeking is not applicable to SSDs, it doesn't really help performance by anticipating future near-by IO requests.

By turning off idling (and switching to IOPS mode), we allow other processes to dispatch IO requests down to the driver and so increase IO throughput.

Following FIO benchmark results were taken on a cloud SSD offering with idling on and off:

Idling iops avg-lat(ms) stddev bw
------------------------------------------------------

    On     7054    90.107         38.697     28217KB/s
   Off    29255    21.836         11.730    117022KB/s
fio --name=temp --size=100G --time_based --ioengine=libaio \
    --randrepeat=0 --direct=1 --invalidate=1 --verify=0 \
    --verify_fatal=0 --rw=randread --blocksize=4k --group_reporting=1 \
    --filename=/dev/sdb --runtime=10 --iodepth=64 --numjobs=10

And the following is from a local SSD run:

Idling iops avg-lat(ms) stddev bw
------------------------------------------------------

    On    19320    33.043         14.068     77281KB/s
   Off    21626    29.465         12.662     86507KB/s
fio --name=temp --size=5G --time_based --ioengine=libaio \
    --randrepeat=0 --direct=1 --invalidate=1 --verify=0 \
    --verify_fatal=0 --rw=randread --blocksize=4k --group_reporting=1 \
    --filename=/fio_data --runtime=10 --iodepth=64 --numjobs=10
---
 block/cfq-iosched.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block/cfq-iosched.c +1 -1
index 5da8e6e..402be01 100644
@@ -4460,7 +4460,7 @@ static int cfq_init_queue(struct request_queue *q, struct elevator_type *e)
        cfqd->cfq_slice[1] = cfq_slice_sync;
        cfqd->cfq_target_latency = cfq_target_latency;
        cfqd->cfq_slice_async_rq = cfq_slice_async_rq;
-       cfqd->cfq_slice_idle = cfq_slice_idle;
+       cfqd->cfq_slice_idle = blk_queue_nonrot(q) ? 0 : cfq_slice_idle;
Quoted 8 lines
Quoted 6 lines
block/cfq-iosched.c +0 -1
        cfqd->cfq_group_idle = cfq_group_idle;
        cfqd->cfq_latency = 1;
        cfqd->hw_tag = -1;
--

2.2.0.rc0.207.ga3a616c

Ping...

Trying once more..

maintainer
#3 lore raw
Quoted 60 lines · Tahsin Erdogan
Quoted 50 lines · Tahsin Erdogan
Quoted 49 lines · Tahsin Erdogan

CFQ idling causes reduced IOPS throughput on non-rotational disks. Since disk head seeking is not applicable to SSDs, it doesn't really help performance by anticipating future near-by IO requests.

By turning off idling (and switching to IOPS mode), we allow other processes to dispatch IO requests down to the driver and so increase IO throughput.

Following FIO benchmark results were taken on a cloud SSD offering with idling on and off:

Idling iops avg-lat(ms) stddev bw
------------------------------------------------------

     On     7054    90.107         38.697     28217KB/s
    Off    29255    21.836         11.730    117022KB/s
fio --name=temp --size=100G --time_based --ioengine=libaio \
     --randrepeat=0 --direct=1 --invalidate=1 --verify=0 \
     --verify_fatal=0 --rw=randread --blocksize=4k --group_reporting=1 \
     --filename=/dev/sdb --runtime=10 --iodepth=64 --numjobs=10

And the following is from a local SSD run:

Idling iops avg-lat(ms) stddev bw
------------------------------------------------------

     On    19320    33.043         14.068     77281KB/s
    Off    21626    29.465         12.662     86507KB/s
fio --name=temp --size=5G --time_based --ioengine=libaio \
     --randrepeat=0 --direct=1 --invalidate=1 --verify=0 \
     --verify_fatal=0 --rw=randread --blocksize=4k --group_reporting=1 \
     --filename=/fio_data --runtime=10 --iodepth=64 --numjobs=10
---
  block/cfq-iosched.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
block/cfq-iosched.c +1 -1
index 5da8e6e..402be01 100644
@@ -4460,7 +4460,7 @@ static int cfq_init_queue(struct request_queue *q, struct elevator_type *e)
         cfqd->cfq_slice[1] = cfq_slice_sync;
         cfqd->cfq_target_latency = cfq_target_latency;
         cfqd->cfq_slice_async_rq = cfq_slice_async_rq;
-       cfqd->cfq_slice_idle = cfq_slice_idle;
+       cfqd->cfq_slice_idle = blk_queue_nonrot(q) ? 0 : cfq_slice_idle;
Quoted 8 lines
Quoted 6 lines
block/cfq-iosched.c +0 -1
         cfqd->cfq_group_idle = cfq_group_idle;
         cfqd->cfq_latency = 1;
         cfqd->hw_tag = -1;
--

2.2.0.rc0.207.ga3a616c

Ping...

Quoted 1 line

Trying once more..

This one worked :-). I agree, it's probably the sane thing to do, I'll apply this for 4.2.

Signature · 1 line
Jens Axboe
#4 lore raw
In reply to Tahsin Erdogan

Hi,

block/cfq-iosched.c static int cfq_init_queue(struct request_queue *q, struct elevator_type *e) · line 4466

Quoted 11 lines · Tahsin Erdogan
block/cfq-iosched.c +1 -1
@@ -4460,7 +4460,7 @@ static int cfq_init_queue(struct request_queue *q, struct elevator_type *e)
 	cfqd->cfq_slice[1] = cfq_slice_sync;
 	cfqd->cfq_target_latency = cfq_target_latency;
 	cfqd->cfq_slice_async_rq = cfq_slice_async_rq;
-	cfqd->cfq_slice_idle = cfq_slice_idle;
+	cfqd->cfq_slice_idle = blk_queue_nonrot(q) ? 0 : cfq_slice_idle;
 	cfqd->cfq_group_idle = cfq_group_idle;
 	cfqd->cfq_latency = 1;
 	cfqd->hw_tag = -1;

Did you test this patch with regular AHCI SSD devices? Applying it on top of v4.1-rc7 makes no difference, slice_idle is still initialized to 8 in my setup, while rotational is 0.

Isn't the elevator initialized long before the non-rotational flag is actually set on the device (which probably happens after it's probed on the scsi bus)?

maintainer
#5 lore raw
Quoted 19 lines · Romain Francoise

Hi,

block/cfq-iosched.c static int cfq_init_queue(struct request_queue *q, struct elevator_type *e) · line 4466

Quoted 11 lines · Tahsin Erdogan
block/cfq-iosched.c +1 -1
@@ -4460,7 +4460,7 @@ static int cfq_init_queue(struct request_queue *q, struct elevator_type *e)
  	cfqd->cfq_slice[1] = cfq_slice_sync;
  	cfqd->cfq_target_latency = cfq_target_latency;
  	cfqd->cfq_slice_async_rq = cfq_slice_async_rq;
-	cfqd->cfq_slice_idle = cfq_slice_idle;
+	cfqd->cfq_slice_idle = blk_queue_nonrot(q) ? 0 : cfq_slice_idle;
  	cfqd->cfq_group_idle = cfq_group_idle;
  	cfqd->cfq_latency = 1;
  	cfqd->hw_tag = -1;

Did you test this patch with regular AHCI SSD devices? Applying it on top of v4.1-rc7 makes no difference, slice_idle is still initialized to 8 in my setup, while rotational is 0.

Quoted 3 lines

Isn't the elevator initialized long before the non-rotational flag is actually set on the device (which probably happens after it's probed on the scsi bus)?

You are absolutely correct. What happens is that the queue is allocated and initialized, and cfq checks the flag. But the flag is set later in the process, when we have finished probing the device checked if it's rotational or not.

There are a few options to handle this. The attached might work, not tested at all. Basically it adds an io sched registration hook, that is called when we are adding the disk on the queue. Non-rotational detection should be done at that point.

Does that work for you?

Signature · 1 line
Jens Axboe
block/cfq-iosched.c +11 -1
index c808ad87652d..af8918eb7cd5 100644
@@ -4508,7 +4508,7 @@ static int cfq_init_queue(struct request_queue *q, struct elevator_type *e)
 	cfqd->cfq_slice[1] = cfq_slice_sync;
 	cfqd->cfq_target_latency = cfq_target_latency;
 	cfqd->cfq_slice_async_rq = cfq_slice_async_rq;
-	cfqd->cfq_slice_idle = blk_queue_nonrot(q) ? 0 : cfq_slice_idle;
+	cfqd->cfq_slice_idle = cfq_slice_idle;
 	cfqd->cfq_group_idle = cfq_group_idle;
 	cfqd->cfq_latency = 1;
 	cfqd->hw_tag = -1;
@@ -4525,6 +4525,15 @@ out_free:
 	return ret;
 }
 
+static void cfq_registered_queue(struct request_queue *q)
+{
+	struct elevator_queue *e = q->elevator;
+	struct cfq_data *cfqd = e->elevator_data;
+
+	if (blk_queue_nonrot(q))
+		cfqd->cfq_slice_idle = 0;
+}
+
 /*
  * sysfs parts below -->
  */
@@ -4640,6 +4649,7 @@ static struct elevator_type iosched_cfq = {
 		.elevator_may_queue_fn =	cfq_may_queue,
 		.elevator_init_fn =		cfq_init_queue,
 		.elevator_exit_fn =		cfq_exit_queue,
+		.elevator_registered_fn =	cfq_registered_queue,
 	},
 	.icq_size	=	sizeof(struct cfq_io_cq),
 	.icq_align	=	__alignof__(struct cfq_io_cq),
block/elevator.c +2 -0
index 59794d0d38e3..5f0452734a40 100644
@@ -810,6 +810,8 @@ int elv_register_queue(struct request_queue *q)
 		}
 		kobject_uevent(&e->kobj, KOBJ_ADD);
 		e->registered = 1;
+		if (e->type->ops.elevator_registered_fn)
+			e->type->ops.elevator_registered_fn(q);
 	}
 	return error;
 }
include/linux/elevator.h +2 -0
index 45a91474487d..638b324f0291 100644
@@ -39,6 +39,7 @@ typedef void (elevator_deactivate_req_fn) (struct request_queue *, struct reques
 typedef int (elevator_init_fn) (struct request_queue *,
 				struct elevator_type *e);
 typedef void (elevator_exit_fn) (struct elevator_queue *);
+typedef void (elevator_registered_fn) (struct request_queue *);
 
 struct elevator_ops
 {
@@ -68,6 +69,7 @@ struct elevator_ops
 
 	elevator_init_fn *elevator_init_fn;
 	elevator_exit_fn *elevator_exit_fn;
+	elevator_registered_fn *elevator_registered_fn;
 };
 
 #define ELV_NAME_MAX	(16)
#6 lore raw

Thanks for catching this.

In my testing, I was switching to cfq through sysfs. Since disk initialization happens earlier than manual switching, I didn't hit this problem.

Quoted 44 lines · Jens Axboe
Quoted 23 lines · Romain Francoise

Hi,

Quoted 12 lines · Tahsin Erdogan
block/cfq-iosched.c +1 -1
@@ -4460,7 +4460,7 @@ static int cfq_init_queue(struct request_queue *q,
 struct elevator_type *e)
        cfqd->cfq_slice[1] = cfq_slice_sync;
        cfqd->cfq_target_latency = cfq_target_latency;
        cfqd->cfq_slice_async_rq = cfq_slice_async_rq;
-       cfqd->cfq_slice_idle = cfq_slice_idle;
+       cfqd->cfq_slice_idle = blk_queue_nonrot(q) ? 0 : cfq_slice_idle;
        cfqd->cfq_group_idle = cfq_group_idle;
        cfqd->cfq_latency = 1;
Quoted 1 line
        cfqd->hw_tag = -1;

Did you test this patch with regular AHCI SSD devices? Applying it on top of v4.1-rc7 makes no difference, slice_idle is still initialized to 8 in my setup, while rotational is 0.

Quoted 3 lines

Isn't the elevator initialized long before the non-rotational flag is actually set on the device (which probably happens after it's probed on the scsi bus)?

You are absolutely correct. What happens is that the queue is allocated and initialized, and cfq checks the flag. But the flag is set later in the process, when we have finished probing the device checked if it's rotational or not.

There are a few options to handle this. The attached might work, not tested at all. Basically it adds an io sched registration hook, that is called when we are adding the disk on the queue. Non-rotational detection should be done at that point.

Does that work for you?

--
Jens Axboe

#7 lore raw
In reply to Jens Axboe

Hi,

Quoted 6 lines · Jens Axboe

There are a few options to handle this. The attached might work, not tested at all. Basically it adds an io sched registration hook, that is called when we are adding the disk on the queue. Non-rotational detection should be done at that point.

Does that work for you?

Yep, that works perfectly in my (admittedly limited) testing; slice_idle is correctly set to 0 on non-rotational devices and keeps its default value of 8 otherwise. Feel free to add my Tested-by.

Thanks!

maintainer
#8 lore raw
Quoted 8 lines · Romain Francoise

Hi,

Quoted 5 lines · Jens Axboe

There are a few options to handle this. The attached might work, not tested at all. Basically it adds an io sched registration hook, that is called when we are adding the disk on the queue. Non-rotational detection should be done at that point.

Quoted 7 lines
Quoted 1 line

Does that work for you?

Yep, that works perfectly in my (admittedly limited) testing; slice_idle is correctly set to 0 on non-rotational devices and keeps its default value of 8 otherwise. Feel free to add my Tested-by.

Thanks!

Thanks for testing, it is now committed.

Signature · 1 line
Jens Axboe