Skip to contentSkip to conversation

Remove BIO_RW_SYNCIO flag from kcopyd

PATCH In review Mikulas Patocka 2 participants 4 messages Oct 25, 2010 – Nov 23, 2010 Device mapper

lore
Share & tools
lore

Mikulas Patocka

author maintainer
#0 lore raw

PATCH 1/1 Remove BIO_RW_SYNCIO flag from kcopyd

Remove BIO_RW_SYNCIO flag from kcopyd

This improves write throughput twice when writing to the origin with snapshot on the same device.

This helps for CFQ I/O scheduler that has separate queues for sync and async I/O, async is optimized for throughput, sync for latency. Reallocations are triggered by writes that are usually async, so mark it as async as well.

---
 drivers/md/dm-kcopyd.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/md/dm-kcopyd.c +1 -1
Index: linux-2.6.36-rc7-fast/drivers/md/dm-kcopyd.c
===================================================================
@@ -345,7 +345,7 @@ static int run_io_job(struct kcopyd_job 
 {
 	int r;
 	struct dm_io_request io_req = {
-		.bi_rw = job->rw | REQ_SYNC | REQ_UNPLUG,
+		.bi_rw = job->rw | REQ_UNPLUG,
 		.mem.type = DM_IO_PAGE_LIST,
 		.mem.ptr.pl = job->pages,
 		.mem.offset = job->offset,
#1 lore raw

Hi Mikulas,

Quoted 4 lines · Mikulas Patocka

Remove BIO_RW_SYNCIO flag from kcopyd

This improves write throughput twice when writing to the origin with snapshot on the same device.

Can you please be more specific on the performance improvement you observed? You're saying something like: baseline=10MB/s patched=20MB/s?

Did you use a larger chunk_size for the snapshot device (e.g. 512K)?

I can't reproduce such significant performance improvements but my testbed may be overly constrained (I'm passing an entire disk from host into KVM guest using virtio). I can look to test on bare metal early next week (unless you have more insight that would obviate the need to do so).

Quoted 4 lines

This helps for CFQ I/O scheduler that has separate queues for sync and async I/O, async is optimized for throughput, sync for latency. Reallocations are triggered by writes that are usually async, so mark it as async as well.

This change is at odds with a previous patch you made to reduce latency of SYNC IO, see linux-2.6 commit:
7ff14a36159d947872870 "dm: unplug queues in threads"

So we're now trading latency for throughput? Given the concerns about snapshot-origin throughput (on lvm-general) I think that is reasonable. But I just want to make sure we're consciously favoring throughput over latency (I'll likely update the patch header to explicitly say as much).

Quoted 18 lines

Index: linux-2.6.36-rc7-fast/drivers/md/dm-kcopyd.c
===================================================================

drivers/md/dm-kcopyd.c +1 -2
@@ -345,7 +345,7 @@ static int run_io_job(struct kcopyd_job 
 {
 	int r;
 	struct dm_io_request io_req = {
-		.bi_rw = job->rw | REQ_SYNC | REQ_UNPLUG,
+		.bi_rw = job->rw | REQ_UNPLUG,
 		.mem.type = DM_IO_PAGE_LIST,
 		.mem.ptr.pl = job->pages,
 		.mem.offset = job->offset,
 
--

dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel

author maintainer
#2 lore raw

Hi

Quoted 5 lines · Mike Snitzer

Hi Mikulas,

On Sun, Oct 24 2010 at 9:02pm -0400,
Mikulas Patocka <[email protected]> wrote:

Quoted 7 lines
Quoted 4 lines

Remove BIO_RW_SYNCIO flag from kcopyd

This improves write throughput twice when writing to the origin with snapshot on the same device.

Can you please be more specific on the performance improvement you observed? You're saying something like: baseline=10MB/s patched=20MB/s?

Yes. I rechecked it now, and it is approximatelly so:
(for chunk size 4k, 32k, 512k)
unpatched: 8.5, 8.6, 9.3 MB/s
this patch: 15.2, 18.5, 17.5 MB/s

Quoted 1 line

Did you use a larger chunk_size for the snapshot device (e.g. 512K)?

It has effect for all chunk sizes.

Quoted 4 lines

I can't reproduce such significant performance improvements but my testbed may be overly constrained (I'm passing an entire disk from host into KVM guest using virtio). I can look to test on bare metal early next week (unless you have more insight that would obviate the need to do so).

My scenario is: LSI MPT controller, U320 15k SCSI disk, 64GiB file mapped with dm-loop, volume group on dm-loop, with both snapshot and origin in it.

Quoted 10 lines
Quoted 4 lines

This helps for CFQ I/O scheduler that has separate queues for sync and async I/O, async is optimized for throughput, sync for latency. Reallocations are triggered by writes that are usually async, so mark it as async as well.

This change is at odds with a previous patch you made to reduce latency of SYNC IO, see linux-2.6 commit:
7ff14a36159d947872870 "dm: unplug queues in threads"

So we're now trading latency for throughput?

Yes. I think throughput matters more than latency for writes.

Quoted 4 lines

Given the concerns about
snapshot-origin throughput (on lvm-general) I think that is reasonable. But I just want to make sure we're consciously favoring throughput over latency (I'll likely update the patch header to explicitly say as much).

Yes. Write it there.

If someone need latency for writes, we could propagate the SYNC flag from the original write bio here. But if no one needs it, it would be needless complication.

Mikulas

Quoted 19 lines
Quoted 18 lines

Index: linux-2.6.36-rc7-fast/drivers/md/dm-kcopyd.c
===================================================================

drivers/md/dm-kcopyd.c +1 -2
@@ -345,7 +345,7 @@ static int run_io_job(struct kcopyd_job 
 {
 	int r;
 	struct dm_io_request io_req = {
-		.bi_rw = job->rw | REQ_SYNC | REQ_UNPLUG,
+		.bi_rw = job->rw | REQ_UNPLUG,
 		.mem.type = DM_IO_PAGE_LIST,
 		.mem.ptr.pl = job->pages,
 		.mem.offset = job->offset,
 
--

dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel

#3 lore raw

PATCH v2 1/1 dm kcopyd: remove SYNC from flags used in dm_io_request

In reply to Mikulas Patocka

From: Mikulas Patocka <[email protected]>

Removing the REQ_SYNC flag improves write throughput twice when writing to the origin with a snapshot on the same device (using the CFQ I/O scheduler).

Sequential write throughput (chunksize of 4k, 32k, 512k)

unpatched:  8.5,  8.6,  9.3 MB/s
patched:   15.2, 18.5, 17.5 MB/s

Snapshot exception reallocations are triggered by writes that are usually async, so mark the associated dm_io_request as async as well. This helps when using the CFQ I/O scheduler because it has separate queues for sync and async I/O, async is optimized for throughput, sync for latency. With this change we're consciously favoring throughput over latency.

---
 drivers/md/dm-kcopyd.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

v2: updated patch header

drivers/md/dm-kcopyd.c +1 -1
Index: linux-2.6.36-rc7-fast/drivers/md/dm-kcopyd.c
===================================================================
@@ -345,7 +345,7 @@ static int run_io_job(struct kcopyd_job 
 {
 	int r;
 	struct dm_io_request io_req = {
-		.bi_rw = job->rw | REQ_SYNC | REQ_UNPLUG,
+		.bi_rw = job->rw | REQ_UNPLUG,
 		.mem.type = DM_IO_PAGE_LIST,
 		.mem.ptr.pl = job->pages,
 		.mem.offset = job->offset,