Skip to contentSkip to conversation

Revert "block/mq-deadline: use correct way to throttling write requests"

PATCH Applied Bart Van Assche 4 participants 6 messages Mar 13, 2024 – Mar 14, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Revert "block/mq-deadline: use correct way to throttling write requests"

The code "max(1U, 3 * (1U << shift) / 4)" comes from the Kyber I/O scheduler. The Kyber I/O scheduler maintains one internal queue per hwq and hence derives its async_depth from the number of hwq tags. Using this approach for the mq-deadline scheduler is wrong since the mq-deadline scheduler maintains one internal queue for all hwqs combined. Hence this revert.

Cc Damien Le Moal <[email protected]>
Cc Harshit Mogalapalli <[email protected]>
Cc Zhiguo Niu <[email protected]>
Fixes d47f9717e5cf ("block/mq-deadline: use correct way to throttling write requests")
---
 block/mq-deadline.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
block/mq-deadline.c +1 -2
index f958e79277b8..02a916ba62ee 100644
@@ -646,9 +646,8 @@ static void dd_depth_updated(struct blk_mq_hw_ctx *hctx)
 	struct request_queue *q = hctx->queue;
 	struct deadline_data *dd = q->elevator->elevator_data;
 	struct blk_mq_tags *tags = hctx->sched_tags;
-	unsigned int shift = tags->bitmap_tags.sb.shift;
 
-	dd->async_depth = max(1U, 3 * (1U << shift)  / 4);
+	dd->async_depth = max(1UL, 3 * q->nr_requests / 4);
 
 	sbitmap_queue_min_shallow_depth(&tags->bitmap_tags, dd->async_depth);
 }
maintainer Applied
#1 lore raw
Quoted 8 lines · Bart Van Assche

The code "max(1U, 3 * (1U << shift) / 4)" comes from the Kyber I/O scheduler. The Kyber I/O scheduler maintains one internal queue per hwq and hence derives its async_depth from the number of hwq tags. Using this approach for the mq-deadline scheduler is wrong since the mq-deadline scheduler maintains one internal queue for all hwqs combined. Hence this revert.

[...]

Applied, thanks!

[1/1] Revert "block/mq-deadline: use correct way to throttling write requests"

      commit: 256aab46e31683d76d45ccbedc287b4d3f3e322b

Best regards,

Signature · 1 line
Jens Axboe
#2 lore raw
In reply to Bart Van Assche

Hi Bart,

Just as mentioned in original patch, "dd->async_depth = max(1UL, 3 * q->nr_requests / 4);", this limitation methods look likes won't have a limit effect, because tag allocated is based on sbitmap, not based the whole nr_requests. Right?
Thanks!

For write requests, when we assign a tags from sched_tags, data->shallow_depth will be passed to sbitmap_find_bit, see the following code:

nr = sbitmap_find_bit_in_word(&sb->map[index],

			min_t (unsigned int,
			__map_depth(sb, index),
			depth),
			alloc_hint, wrap);

The smaller of data->shallow_depth and __map_depth(sb, index) will be used as the maximum range when allocating bits.

For a mmc device (one hw queue, deadline I/O scheduler):
q->nr_requests = sched_tags = 128, so according to the previous calculation method, dd->async_depth = data->shallow_depth = 96, and the platform is 64bits with 8 cpus, sched_tags.bitmap_tags.sb.shift=5, sb.maps[]=32/32/32/32, 32 is smaller than 96, whether it is a read or a write I/O, tags can be allocated to the maximum range each time, which has not throttling effect.

-----邮件原件-----
发件人: Bart Van Assche <[email protected]>
发送时间: 2024年3月14日 5:42
收件人: Jens Axboe <[email protected]>
抄送: [email protected]; Christoph Hellwig <[email protected]>; Bart Van Assche <[email protected]>; [email protected]; Damien Le Moal <[email protected]>; Harshit Mogalapalli <[email protected]>; 牛志国 (Zhiguo Niu) <[email protected]> 主题: [PATCH] Revert "block/mq-deadline: use correct way to throttling write requests"

注意: 这封邮件来自于外部。除非你确定邮件内容安全,否则不要点击任何链接和附件。
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

The code "max(1U, 3 * (1U << shift) / 4)" comes from the Kyber I/O scheduler. The Kyber I/O scheduler maintains one internal queue per hwq and hence derives its async_depth from the number of hwq tags. Using this approach for the mq-deadline scheduler is wrong since the mq-deadline scheduler maintains one internal queue for all hwqs combined. Hence this revert.

Cc Damien Le Moal <[email protected]>
Cc Harshit Mogalapalli <[email protected]>
Cc Zhiguo Niu <[email protected]>
Fixes d47f9717e5cf ("block/mq-deadline: use correct way to throttling write requests")
---
 block/mq-deadline.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
block/mq-deadline.c +1 -2
@@ -646,9 +646,8 @@ static void dd_depth_updated(struct blk_mq_hw_ctx *hctx)
        struct request_queue *q = hctx->queue;
        struct deadline_data *dd = q->elevator->elevator_data;
        struct blk_mq_tags *tags = hctx->sched_tags;
-       unsigned int shift = tags->bitmap_tags.sb.shift;
 
-       dd->async_depth = max(1U, 3 * (1U << shift)  / 4);
+       dd->async_depth = max(1UL, 3 * q->nr_requests / 4);
 
        sbitmap_queue_min_shallow_depth(&tags->bitmap_tags, dd->async_depth);  }
#3 lore raw
Quoted 17 lines · 牛志国 (Zhiguo Niu)

Just as mentioned in original patch, "dd->async_depth = max(1UL, 3 * q->nr_requests / 4);", this limitation methods look likes won't have a limit effect, because tag allocated is based on sbitmap, not based the whole nr_requests. Right?
Thanks!

For write requests, when we assign a tags from sched_tags, data->shallow_depth will be passed to sbitmap_find_bit, see the following code:

nr = sbitmap_find_bit_in_word(&sb->map[index],

			min_t (unsigned int,
			__map_depth(sb, index),
			depth),
			alloc_hint, wrap);

The smaller of data->shallow_depth and __map_depth(sb, index) will be used as the maximum range when allocating bits.

Quoted 7 lines

For a mmc device (one hw queue, deadline I/O scheduler):
q->nr_requests = sched_tags = 128, so according to the previous calculation method, dd->async_depth = data->shallow_depth = 96, and the platform is 64bits with 8 cpus, sched_tags.bitmap_tags.sb.shift=5, sb.maps[]=32/32/32/32, 32 is smaller than 96, whether it is a read or a write I/O, tags can be allocated to the maximum range each time, which has not throttling effect.

Whether or not the code in my patch effectively performs throttling, we need this revert to be merged. The patch that is being reverted ("block/mq-deadline: use correct way to throttling write requests") ended up in Greg KH's stable branches. Hence, the first step is to revert that patch and tag it with "Cc: stable" such that the revert lands in the stable branches.

Thanks,

Bart.

maintainer
#4 lore raw
Quoted 25 lines · Bart Van Assche
Quoted 17 lines · ??? (Zhiguo Niu)

Just as mentioned in original patch, "dd->async_depth = max(1UL, 3 * q->nr_requests / 4);", this limitation methods look likes won't have a limit effect, because tag allocated is based on sbitmap, not based the whole nr_requests. Right?
Thanks!

For write requests, when we assign a tags from sched_tags, data->shallow_depth will be passed to sbitmap_find_bit, see the following code:

nr = sbitmap_find_bit_in_word(&sb->map[index],

            min_t (unsigned int,
            __map_depth(sb, index),
            depth),
            alloc_hint, wrap);

The smaller of data->shallow_depth and __map_depth(sb, index) will be used as the maximum range when allocating bits.

Quoted 7 lines

For a mmc device (one hw queue, deadline I/O scheduler):
q->nr_requests = sched_tags = 128, so according to the previous calculation method, dd->async_depth = data->shallow_depth = 96, and the platform is 64bits with 8 cpus, sched_tags.bitmap_tags.sb.shift=5, sb.maps[]=32/32/32/32, 32 is smaller than 96, whether it is a read or a write I/O, tags can be allocated to the maximum range each time, which has not throttling effect.

Quoted 6 lines

Whether or not the code in my patch effectively performs throttling, we need this revert to be merged. The patch that is being reverted ("block/mq-deadline: use correct way to throttling write requests") ended up in Greg KH's stable branches. Hence, the first step is to revert that patch and tag it with "Cc: stable" such that the revert lands in the stable branches.

Indeed, no amount of arguing is going to change that fact. Zhiguo, it caused a regression. Rather than argue on why the change is correct, it'd be much more productive to figure out a future solution.

Signature · 1 line
Jens Axboe
#5 lore raw
In reply to Bart Van Assche

Hi Bart, Jens and Zhiguo,

Quoted 7 lines · Bart Van Assche

The code "max(1U, 3 * (1U << shift) / 4)" comes from the Kyber I/O scheduler. The Kyber I/O scheduler maintains one internal queue per hwq and hence derives its async_depth from the number of hwq tags. Using this approach for the mq-deadline scheduler is wrong since the mq-deadline scheduler maintains one internal queue for all hwqs combined. Hence this revert.

Thanks a lot for helping with this performance regression[1].

Regards,
Harshit

[1] https://lore.kernel.org/all/[email protected]/

Quoted 25 lines
Cc Damien Le Moal <[email protected]>
Cc Harshit Mogalapalli <[email protected]>
Cc Zhiguo Niu <[email protected]>
Fixes d47f9717e5cf ("block/mq-deadline: use correct way to throttling write requests")
---
  block/mq-deadline.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)
block/mq-deadline.c +1 -2
index f958e79277b8..02a916ba62ee 100644
@@ -646,9 +646,8 @@ static void dd_depth_updated(struct blk_mq_hw_ctx *hctx)
  	struct request_queue *q = hctx->queue;
  	struct deadline_data *dd = q->elevator->elevator_data;
  	struct blk_mq_tags *tags = hctx->sched_tags;
-	unsigned int shift = tags->bitmap_tags.sb.shift;
 
-	dd->async_depth = max(1U, 3 * (1U << shift)  / 4);
+	dd->async_depth = max(1UL, 3 * q->nr_requests / 4);
 
  	sbitmap_queue_min_shallow_depth(&tags->bitmap_tags, dd->async_depth);
  }