Skip to contentSkip to conversation

block: Reserve only one queue tag for sync IO if only 3 tags are available

PATCH In review Jan Kara 2 participants 3 messages Jun 28, 2013 Block layer

lore
Share & tools
lore

Jan Kara

author
#0 lore raw

PATCH 1/1 block: Reserve only one queue tag for sync IO if only 3 tags are available

In case a device has three tags available we still reserve two of them for sync IO. That leaves only a single tag for async IO such as writeback from flusher thread which results in poor performance.

Allow async IO to consume two tags in case queue has three tag availabe to get a decent async write performance.

This patch improves streaming write performance on a machine with such disk from ~21 MB/s to ~52 MB/s. Also postmark throughput in presence of streaming writer improves from 8 to 12 transactions per second so sync IO doesn't seem to be harmed in presence of heavy async writer.

---
 block/blk-tag.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
block/blk-tag.c +9 -2
index cc345e1..3f33d86 100644
@@ -348,9 +348,16 @@ int blk_queue_start_tag(struct request_queue *q, struct request *rq)
 	 */
 	max_depth = bqt->max_depth;
 	if (!rq_is_sync(rq) && max_depth > 1) {
-		max_depth -= 2;
-		if (!max_depth)
+		switch (max_depth) {
+		case 2:
 			max_depth = 1;
+			break;
+		case 3:
+			max_depth = 2;
+			break;
+		default:
+			max_depth -= 2;
+		}
 		if (q->in_flight[BLK_RW_ASYNC] > max_depth)
 			return 1;
 	}
Signature · 1 line
1.8.1.4
maintainer
#1 lore raw
Quoted 11 lines · Jan Kara

In case a device has three tags available we still reserve two of them for sync IO. That leaves only a single tag for async IO such as writeback from flusher thread which results in poor performance.

Allow async IO to consume two tags in case queue has three tag availabe to get a decent async write performance.

This patch improves streaming write performance on a machine with such disk from ~21 MB/s to ~52 MB/s. Also postmark throughput in presence of streaming writer improves from 8 to 12 transactions per second so sync IO doesn't seem to be harmed in presence of heavy async writer.

That's pretty crazy! Never seen a device like that. But yes, it obviously exists, and the 2 reserved tags doesn't work well for that. So it's an improvement.

Thanks, will queue up for 3.11.

Signature · 1 line
Jens Axboe
author
#2 lore raw
Quoted 8 lines · Jens Axboe
Quoted 7 lines · Jan Kara

In case a device has three tags available we still reserve two of them for sync IO. That leaves only a single tag for async IO such as writeback from flusher thread which results in poor performance.

Allow async IO to consume two tags in case queue has three tag availabe to get a decent async write performance.

Quoted 8 lines
Quoted 4 lines

This patch improves streaming write performance on a machine with such disk from ~21 MB/s to ~52 MB/s. Also postmark throughput in presence of streaming writer improves from 8 to 12 transactions per second so sync IO doesn't seem to be harmed in presence of heavy async writer.

That's pretty crazy! Never seen a device like that. But yes, it obviously exists, and the 2 reserved tags doesn't work well for that. So it's an improvement.

Yeah, I agree it's a weird hardware. It's a ppc64 machine where storage
identifies as:
scsi2 : IBM 573E Storage Adapter

scsi 2:0:4:0: Direct-Access     IBM      ST373453LC       C51C PQ: 0 ANSI: 3
scsi 2:0:5:0: Direct-Access     IBM   H0 ST373207LC       C70D PQ: 0 ANSI: 4
scsi 2:0:8:0: Direct-Access     IBM   H0 ST373207LC       C70D PQ: 0 ANSI: 4

It took me a while to figure out why this machine observes performance
characteristics noone else sees.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR