blktrace: Provide event for request merging
Subsystems (2)
PATCH 1/1 blktrace: Provide event for request merging
Currently blk-mq does not report any event when two requests get merged in the elevator. This then results in difficult to understand sequence of events like:
...
8,0 34 1579 0.608765271 2718 I WS 215023504 + 40 [dbench] 8,0 34 1584 0.609184613 2719 A WS 215023544 + 56 <- (8,4) 2160568 8,0 34 1585 0.609184850 2719 Q WS 215023544 + 56 [dbench] 8,0 34 1586 0.609188524 2719 G WS 215023544 + 56 [dbench] 8,0 3 602 0.609684162 773 D WS 215023504 + 96 [kworker/3:1H] 8,0 34 1591 0.609843593 0 C WS 215023504 + 96 [0]
and you can only guess (after quite some headscratching since the above excerpt is intermixed with a lot of other IO) that request 215023544+56 got merged to request 215023504+40. Provide proper event for request merging like we used to do in the legacy block layer.
--- block/blk-merge.c | 2 ++ include/trace/events/block.h | 15 +++++++++++++++ kernel/trace/blktrace.c | 10 ++++++++++ 3 files changed, 27 insertions(+)
block/blk-merge.c +2 -0
@@ -793,6 +793,8 @@ static struct request *attempt_merge(struct request_queue *q,*/ blk_account_io_merge_request (next); +trace_block_rq_merge (q, next); +/* * ownership of bio passed from next to req, return 'next' for * the caller to free
include/trace/events/block.h +15 -0
@@ -211,6 +211,21 @@ DEFINE_EVENT(block_rq, block_rq_issue,TP_ARGS (q, rq) ); +/** +* block_rq_merge - merge request with another one in the elevator +* @q: queue holding operation +* @rq: block IO operation operation request +* +* Called when block operation request @rq from queue @q is merged to another +* request queued in the elevator. +*/ +DEFINE_EVENT (block_rq, block_rq_merge, + + TP_PROTO(struct request_queue *q, struct request *rq), + +TP_ARGS (q, rq) +); +/** * block_bio_bounce - used bounce buffer when processing block operation * @q: queue holding the block operation
kernel/trace/blktrace.c +10 -0
@@ -834,6 +834,13 @@ static void blk_add_trace_rq_issue(void *ignore, blk_trace_request_get_cgid(q, rq)); } +static void blk_add_trace_rq_merge (void *ignore, +struct request_queue *q,struct request *rq) +{ +blk_add_trace_rq (rq,0 ,blk_rq_bytes (rq),BLK_TA_BACKMERGE , +blk_trace_request_get_cgid (q, rq)); +} +static void blk_add_trace_rq_requeue (void *ignore,struct request_queue *q,struct request *rq)
@@ -1115,6 +1122,8 @@ static void blk_register_tracepoints(void)WARN_ON (ret); ret =register_trace_block_rq_issue (blk_add_trace_rq_issue,NULL );WARN_ON (ret); + ret =register_trace_block_rq_merge (blk_add_trace_rq_merge,NULL ); +WARN_ON (ret); ret =register_trace_block_rq_requeue (blk_add_trace_rq_requeue,NULL );WARN_ON (ret); ret =register_trace_block_rq_complete (blk_add_trace_rq_complete,NULL );
@@ -1161,6 +1170,7 @@ static void blk_unregister_tracepoints(void)unregister_trace_block_bio_bounce (blk_add_trace_bio_bounce,NULL );unregister_trace_block_rq_complete (blk_add_trace_rq_complete,NULL );unregister_trace_block_rq_requeue (blk_add_trace_rq_requeue,NULL ); +unregister_trace_block_rq_merge (blk_add_trace_rq_merge,NULL );unregister_trace_block_rq_issue (blk_add_trace_rq_issue,NULL );unregister_trace_block_rq_insert (blk_add_trace_rq_insert,NULL );
Signature · 1 line
2.16.4
Jan,
Quoted 12 lines · Jan Kara
Currently blk-mq does not report any event when two requests get merged in the elevator. This then results in difficult to understand sequence of events like:
...
8,0 34 1579 0.608765271 2718 I WS 215023504 + 40 [dbench] 8,0 34 1584 0.609184613 2719 A WS 215023544 + 56 <- (8,4) 2160568 8,0 34 1585 0.609184850 2719 Q WS 215023544 + 56 [dbench] 8,0 34 1586 0.609188524 2719 G WS 215023544 + 56 [dbench] 8,0 3 602 0.609684162 773 D WS 215023504 + 96 [kworker/3:1H] 8,0 34 1591 0.609843593 0 C WS 215023504 + 96 [0]
Quoted 6 lines
and you can only guess (after quite some headscratching since the above excerpt is intermixed with a lot of other IO) that request 215023544+56 got merged to request 215023504+40. Provide proper event for request merging like we used to do in the legacy block layer.
Signed-off-by Jan Kara <[email protected]>
The attempt_merge function is also responsible for the discard merging which doesn't have any direction specified in ELEVATOR_XXX identifiers. In this patch we care unconditionally generating back merge event irrespective of the req_op.
Do we need to either generate event iff ELEVATOR_BACK_MERGE true case or add another trace point for discard ? given that it may lead to confusion since elevator flags for the discard doesn't specify the direction.
Hello,
Quoted 25 lines · Chaitanya Kulkarni
Quoted 12 lines · Jan Kara
Currently blk-mq does not report any event when two requests get merged in the elevator. This then results in difficult to understand sequence of events like:
...
8,0 34 1579 0.608765271 2718 I WS 215023504 + 40 [dbench] 8,0 34 1584 0.609184613 2719 A WS 215023544 + 56 <- (8,4) 2160568 8,0 34 1585 0.609184850 2719 Q WS 215023544 + 56 [dbench] 8,0 34 1586 0.609188524 2719 G WS 215023544 + 56 [dbench] 8,0 3 602 0.609684162 773 D WS 215023504 + 96 [kworker/3:1H] 8,0 34 1591 0.609843593 0 C WS 215023504 + 96 [0]Quoted 6 lines
and you can only guess (after quite some headscratching since the above excerpt is intermixed with a lot of other IO) that request 215023544+56 got merged to request 215023504+40. Provide proper event for request merging like we used to do in the legacy block layer.
Signed-off-by Jan Kara <[email protected]>The
attempt_mergefunction is also responsible for the discard merging which doesn't have any direction specified inELEVATOR_XXXidentifiers. In this patch we care unconditionally generating back merge event irrespective of thereq_op.
Quoted 4 lines
Do we need to either generate event iff
ELEVATOR_BACK_MERGEtrue case or add another trace point for discard ? given that it may lead to confusion since elevator flags for the discard doesn't specify the direction.
attempt_merge() is always called so that 'req' is always the request with the lower sector, 'next' is the request with a higher sector, and 'next' is discarded and 'req' is updated. So attempt_merge() always performs only one direction of a merge and I don't think it makes any sence to distinguish back merges and forward merges in this case. So discards don't need any special treatment either AFAICT.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 9 lines · Jan Kara
Quoted 9 lines
The
attempt_mergefunction is also responsible for the discard merging which doesn't have any direction specified inELEVATOR_XXXidentifiers. In this patch we care unconditionally generating back merge event irrespective of thereq_op.Do we need to either generate event iff
ELEVATOR_BACK_MERGEtrue case or add another trace point for discard ? given that it may lead to confusion since elevator flags for the discard doesn't specify the direction.
Quoted 7 lines
attempt_merge()is always called so that 'req' is always the request with the lower sector, 'next' is the request with a higher sector, and 'next' is discarded and 'req' is updated. Soattempt_merge()always performs only one direction of a merge and I don't think it makes any sence to distinguish back merges and forward merges in this case. So discards don't need any special treatment either AFAICT.
Please disregard my comment regarding adding a separate tracpoint for discard.
I understand the code and that is what I'm saying it will only back-merge.
The concern is just like we have ELEVATOR_BACK_MERGE used in attempt_merge, we should also have ELEVATOR_DISCARD_BACK_MERGE and use it the attempt_merge() then this tracepoint will map to the back-merge which your patch does with the use of BLK_TC_BACKMERGE nothing needed in your patch to change regarding tracepoint.
Quoted 3 lines
Honza
-- Jan Kara <[email protected]> SUSE Labs, CR
block/blk-merge.c static struct request *attempt_merge(struct request_queue *q, · line 796
Quoted 3 lines · Jan Kara
block/blk-merge.c +1
-0blk_account_io_merge_request (next); +trace_block_rq_merge (q, next);
q can be derived from next, no need to explicitly pass it. And yes, I know a lot of existing tracepoints do so, but I plan to fix that up as well.
Quoted 8 lines · Christoph Hellwig
block/blk-merge.c static struct request *attempt_merge(struct request_queue *q, · line 796
Quoted 3 lines · Jan Kara
block/blk-merge.c +1
-0blk_account_io_merge_request (next); +trace_block_rq_merge (q, next);q can be derived from next, no need to explicitly pass it. And yes, I know a lot of existing tracepoints do so, but I plan to fix that up as well.
OK, I'll update the patch.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 8 lines · Christoph Hellwig
block/blk-merge.c static struct request *attempt_merge(struct request_queue *q, · line 796
Quoted 3 lines · Jan Kara
block/blk-merge.c +1
-0blk_account_io_merge_request (next); +trace_block_rq_merge (q, next);q can be derived from next, no need to explicitly pass it. And yes, I know a lot of existing tracepoints do so, but I plan to fix that up as well.
I had a look into it now and I could do this but that would mean that block_rq_merge trace event would now differ from all other similar events and we couldn't use block_rq event class and have to define our own and so overall it would IMO make future conversion to get rid of 'q' argument more difficult, not simpler. So I can do this but are you really sure?
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 8 lines · Jan Kara
Quoted 7 lines · Christoph Hellwig
block/blk-merge.c static struct request *attempt_merge(struct request_queue *q, · line 796
Quoted 3 lines · Jan Kara
blk_account_io_merge_request(next);block/blk-merge.c +1
-0+trace_block_rq_merge (q, next);q can be derived from next, no need to explicitly pass it. And yes, I know a lot of existing tracepoints do so, but I plan to fix that up as well.
Quoted 8 lines
I had a look into it now and I could do this but that would mean that
block_rq_mergetrace event would now differ from all other similar events and we couldn't useblock_rqevent class and have to define our own and so overall it would IMO make future conversion to get rid of 'q' argument more difficult, not simpler. So I can do this but are you really sure?Honza
-- Jan Kara <[email protected]> SUSE Labs, CR
If you guys are okay then I can look into this and send out a patch and we can keep this patch separate.
Quoted 10 lines · Jan Kara
Quoted 8 lines · Christoph Hellwig
block/blk-merge.c static struct request *attempt_merge(struct request_queue *q, · line 796
Quoted 3 lines · Jan Kara
block/blk-merge.c +1
-0blk_account_io_merge_request (next); +trace_block_rq_merge (q, next);q can be derived from next, no need to explicitly pass it. And yes, I know a lot of existing tracepoints do so, but I plan to fix that up as well.
Quoted 5 lines
I had a look into it now and I could do this but that would mean that
block_rq_mergetrace event would now differ from all other similar events and we couldn't useblock_rqevent class and have to define our own and so overall it would IMO make future conversion to get rid of 'q' argument more difficult, not simpler. So I can do this but are you really sure?
Ok, let's keep the original version for now then.
Quoted 12 lines · Jan Kara
Currently blk-mq does not report any event when two requests get merged in the elevator. This then results in difficult to understand sequence of events like:
...
8,0 34 1579 0.608765271 2718 I WS 215023504 + 40 [dbench] 8,0 34 1584 0.609184613 2719 A WS 215023544 + 56 <- (8,4) 2160568 8,0 34 1585 0.609184850 2719 Q WS 215023544 + 56 [dbench] 8,0 34 1586 0.609188524 2719 G WS 215023544 + 56 [dbench] 8,0 3 602 0.609684162 773 D WS 215023504 + 96 [kworker/3:1H] 8,0 34 1591 0.609843593 0 C WS 215023504 + 96 [0]
Quoted 6 lines
and you can only guess (after quite some headscratching since the above excerpt is intermixed with a lot of other IO) that request 215023544+56 got merged to request 215023504+40. Provide proper event for request merging like we used to do in the legacy block layer.
Signed-off-by Jan Kara <[email protected]>
Jens, it seems people are fine with this patch in the end. Can you please merge it? Thanks!
Honza
Quoted 83 lines
--- block/blk-merge.c | 2 ++ include/trace/events/block.h | 15 +++++++++++++++ kernel/trace/blktrace.c | 10 ++++++++++ 3 files changed, 27 insertions(+)block/blk-merge.c +2
-0@@ -793,6 +793,8 @@ static struct request *attempt_merge(struct request_queue *q,*/ blk_account_io_merge_request (next); +trace_block_rq_merge (q, next); +/* * ownership of bio passed from next to req, return 'next' for * the caller to free include/trace/events/block.h +15
-0@@ -211,6 +211,21 @@ DEFINE_EVENT(block_rq, block_rq_issue,TP_ARGS (q, rq) ); +/** +* block_rq_merge - merge request with another one in the elevator +* @q: queue holding operation +* @rq: block IO operation operation request +* +* Called when block operation request @rq from queue @q is merged to another +* request queued in the elevator. +*/ +DEFINE_EVENT (block_rq, block_rq_merge, + + TP_PROTO(struct request_queue *q, struct request *rq), + +TP_ARGS (q, rq) +); +/** * block_bio_bounce - used bounce buffer when processing block operation * @q: queue holding the block operation kernel/trace/blktrace.c +10
-0@@ -834,6 +834,13 @@ static void blk_add_trace_rq_issue(void *ignore, blk_trace_request_get_cgid(q, rq)); } +static void blk_add_trace_rq_merge (void *ignore, +struct request_queue *q,struct request *rq) +{ +blk_add_trace_rq (rq,0 ,blk_rq_bytes (rq),BLK_TA_BACKMERGE , +blk_trace_request_get_cgid (q, rq)); +} +static void blk_add_trace_rq_requeue (void *ignore,struct request_queue *q,struct request *rq)@@ -1115,6 +1122,8 @@ static void blk_register_tracepoints(void)WARN_ON (ret); ret =register_trace_block_rq_issue (blk_add_trace_rq_issue,NULL );WARN_ON (ret); + ret =register_trace_block_rq_merge (blk_add_trace_rq_merge,NULL ); +WARN_ON (ret); ret =register_trace_block_rq_requeue (blk_add_trace_rq_requeue,NULL );WARN_ON (ret); ret =register_trace_block_rq_complete (blk_add_trace_rq_complete,NULL );@@ -1161,6 +1170,7 @@ static void blk_unregister_tracepoints(void)unregister_trace_block_bio_bounce (blk_add_trace_bio_bounce,NULL );unregister_trace_block_rq_complete (blk_add_trace_rq_complete,NULL );unregister_trace_block_rq_requeue (blk_add_trace_rq_requeue,NULL ); +unregister_trace_block_rq_merge (blk_add_trace_rq_merge,NULL );unregister_trace_block_rq_issue (blk_add_trace_rq_issue,NULL );unregister_trace_block_rq_insert (blk_add_trace_rq_insert,NULL );Signature · 1 line
2.16.4
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Hi Jan and Jens
Quoted 13 lines · Jan Kara
Quoted 12 lines · Jan Kara
Currently blk-mq does not report any event when two requests get merged in the elevator. This then results in difficult to understand sequence of events like:
...
8,0 34 1579 0.608765271 2718 I WS 215023504 + 40 [dbench] 8,0 34 1584 0.609184613 2719 A WS 215023544 + 56 <- (8,4) 2160568 8,0 34 1585 0.609184850 2719 Q WS 215023544 + 56 [dbench] 8,0 34 1586 0.609188524 2719 G WS 215023544 + 56 [dbench] 8,0 3 602 0.609684162 773 D WS 215023504 + 96 [kworker/3:1H] 8,0 34 1591 0.609843593 0 C WS 215023504 + 96 [0]
Quoted 9 lines
Quoted 6 lines
and you can only guess (after quite some headscratching since the above excerpt is intermixed with a lot of other IO) that request 215023544+56 got merged to request 215023504+40. Provide proper event for request merging like we used to do in the legacy block layer.
Signed-off-by Jan Kara<[email protected]>Jens, it seems people are fine with this patch in the end. Can you please merge it? Thanks!
I'm sending blktrace cleanup based on the discussion [1], if you guys want I can also add this patch and keep Jan Author and have cleanup done with this patch included separtely.
Please let me know that is not accepted I'll send my series separately.
[1] https://marc.info/?l=linux-block&m=159296928424805&w=2
Quoted 1 line
Jens Axboe
maintainerQuoted 18 lines · Jan Kara
Quoted 17 lines · Jan Kara
Currently blk-mq does not report any event when two requests get merged in the elevator. This then results in difficult to understand sequence of events like:
...
8,0 34 1579 0.608765271 2718 I WS 215023504 + 40 [dbench] 8,0 34 1584 0.609184613 2719 A WS 215023544 + 56 <- (8,4) 2160568 8,0 34 1585 0.609184850 2719 Q WS 215023544 + 56 [dbench] 8,0 34 1586 0.609188524 2719 G WS 215023544 + 56 [dbench] 8,0 3 602 0.609684162 773 D WS 215023504 + 96 [kworker/3:1H] 8,0 34 1591 0.609843593 0 C WS 215023504 + 96 [0]and you can only guess (after quite some headscratching since the above excerpt is intermixed with a lot of other IO) that request 215023544+56 got merged to request 215023504+40. Provide proper event for request merging like we used to do in the legacy block layer.
Quoted 4 lines
Quoted 1 line
Signed-off-by Jan Kara <[email protected]>Jens, it seems people are fine with this patch in the end. Can you please merge it? Thanks!
Applied for 5.9, thanks.
Signature · 1 line
Jens Axboe