Skip to contentSkip to conversation

block: only return started requests from blk_mq_tag_to_rq()

PATCH In review Hannes Reinecke 4 participants 6 messages Jun 19, 2020 – Jun 22, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 2/2 block: only return started requests from blk_mq_tag_to_rq()

blk_mq_tag_to_rq() is used from within the driver to map a tag to a request. As such it should only return requests which are already started (ie passed to the driver); otherwise the driver might trip over requests which it has never seen and random crashes will occur.

---
 block/blk-mq.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
block/blk-mq.c +5 -1
index 4f57d27bfa73..f02d18113f9e 100644
@@ -815,9 +815,13 @@ EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list);
 
 struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag)
 {
+	struct request *rq;
+
 	if (tag < tags->nr_tags) {
 		prefetch(tags->rqs[tag]);
-		return tags->rqs[tag];
+		rq = tags->rqs[tag];
+		if (blk_mq_request_started(rq))
+			return rq;
 	}
 
 	return NULL;
Signature · 1 line
2.16.4
#1 lore raw
Quoted 21 lines · Hannes Reinecke

blk_mq_tag_to_rq() is used from within the driver to map a tag to a request. As such it should only return requests which are already started (ie passed to the driver); otherwise the driver might trip over requests which it has never seen and random crashes will occur.

---
  block/blk-mq.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
block/blk-mq.c +1 -0
index 4f57d27bfa73..f02d18113f9e 100644
@@ -815,9 +815,13 @@ EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list);
 
  struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag)
  {
+	struct request *rq;

block/blk-mq.c EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list); · line 827

Quoted 10 lines
block/blk-mq.c +4 -1
+
  	if (tag < tags->nr_tags) {
  		prefetch(tags->rqs[tag]);
-		return tags->rqs[tag];
+		rq = tags->rqs[tag];
+		if (blk_mq_request_started(rq))
+			return rq;
  	}
 
  	return NULL;

This becomes particularly obnoxious for SCSI drivers using scsi_host_find_tag() for cleaning up stale commands (ie drivers like qla4xxx, fnic, and snic). All other drivers use it from the completion routine, so one can expect a valid (and started) tag here. So for those it shouldn't matter.

But still, if there are objections I could look at fixing it within the SCSI stack; although that would most likely mean I'll have to implement the above patch as an additional function.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
maintainer
#2 lore raw
Quoted 38 lines · Hannes Reinecke
Quoted 21 lines · Hannes Reinecke

blk_mq_tag_to_rq() is used from within the driver to map a tag to a request. As such it should only return requests which are already started (ie passed to the driver); otherwise the driver might trip over requests which it has never seen and random crashes will occur.

---
  block/blk-mq.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
block/blk-mq.c +1 -0
index 4f57d27bfa73..f02d18113f9e 100644
@@ -815,9 +815,13 @@ EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list);
 
  struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag)
  {
+	struct request *rq;

block/blk-mq.c EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list); · line 827

Quoted 10 lines
block/blk-mq.c +4 -1
+
  	if (tag < tags->nr_tags) {
  		prefetch(tags->rqs[tag]);
-		return tags->rqs[tag];
+		rq = tags->rqs[tag];
+		if (blk_mq_request_started(rq))
+			return rq;
  	}
 
  	return NULL;

This becomes particularly obnoxious for SCSI drivers using scsi_host_find_tag() for cleaning up stale commands (ie drivers like qla4xxx, fnic, and snic).
All other drivers use it from the completion routine, so one can expect a valid (and started) tag here. So for those it shouldn't matter.

Quoted 3 lines

But still, if there are objections I could look at fixing it within the SCSI stack; although that would most likely mean I'll have to implement the above patch as an additional function.

The helper does exactly what it should, return a request associated with a tag. Either add the logic to the caller, or provide a new helper that does what you need. I'd be inclined to just add it to the caller that needs it.

Signature · 1 line
Jens Axboe
#3 lore raw
Quoted 39 lines · Jens Axboe
Quoted 38 lines · Hannes Reinecke
Quoted 21 lines · Hannes Reinecke

blk_mq_tag_to_rq() is used from within the driver to map a tag to a request. As such it should only return requests which are already started (ie passed to the driver); otherwise the driver might trip over requests which it has never seen and random crashes will occur.

---
  block/blk-mq.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
block/blk-mq.c +1 -0
index 4f57d27bfa73..f02d18113f9e 100644
@@ -815,9 +815,13 @@ EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list);
 
  struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag)
  {
+	struct request *rq;

block/blk-mq.c EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list); · line 827

Quoted 10 lines
block/blk-mq.c +4 -1
+
  	if (tag < tags->nr_tags) {
  		prefetch(tags->rqs[tag]);
-		return tags->rqs[tag];
+		rq = tags->rqs[tag];
+		if (blk_mq_request_started(rq))
+			return rq;
  	}
 
  	return NULL;

This becomes particularly obnoxious for SCSI drivers using scsi_host_find_tag() for cleaning up stale commands (ie drivers like qla4xxx, fnic, and snic).
All other drivers use it from the completion routine, so one can expect a valid (and started) tag here. So for those it shouldn't matter.

Quoted 8 lines
Quoted 3 lines

But still, if there are objections I could look at fixing it within the SCSI stack; although that would most likely mean I'll have to implement the above patch as an additional function.

The helper does exactly what it should, return a request associated with a tag. Either add the logic to the caller, or provide a new helper that does what you need. I'd be inclined to just add it to the caller that needs it.

I agree, even though the idea is good for most of driver usage since driver should only care started request.

It may cause kernel oops in some corner case, such as blk_mq_poll_hybrid(), blk_poll() may see one not-started request, one example is nvme_execute_rq_polled().

Thanks,
Ming

#4 lore raw
In reply to Hannes Reinecke
Quoted 37 lines · Hannes Reinecke
Quoted 20 lines · Hannes Reinecke

blk_mq_tag_to_rq() is used from within the driver to map a tag to a request. As such it should only return requests which are already started (ie passed to the driver); otherwise the driver might trip over requests which it has never seen and random crashes will occur.

---
  block/blk-mq.c | 6 +++++-

  1 file changed, 5 insertions(+), 1 deletion(-)
block/blk-mq.c +1 -0
index 4f57d27bfa73..f02d18113f9e 100644
@@ -815,9 +815,13 @@ EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list);
     struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags,
 unsigned int tag)
   {
+    struct request *rq;

block/blk-mq.c EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list); · line 827

Quoted 10 lines
block/blk-mq.c +4 -1
+
       if (tag < tags->nr_tags) {
           prefetch(tags->rqs[tag]);
-        return tags->rqs[tag];
+        rq = tags->rqs[tag];
+        if (blk_mq_request_started(rq))
+            return rq;
       }
         return NULL;
 

This becomes particularly obnoxious for SCSI drivers using scsi_host_find_tag() for cleaning up stale commands (ie drivers like qla4xxx, fnic, and snic).
All other drivers use it from the completion routine, so one can expect a valid (and started) tag here. So for those it shouldn't matter.

Quoted 3 lines

But still, if there are objections I could look at fixing it within the SCSI stack; although that would most likely mean I'll have to implement the above patch as an additional function.

Hi Hannes,

Will the above patch make the fast path of every block driver slightly slower?

Shouldn't SCSI drivers (and other block drivers) use blk_mq_tagset_busy_iter() to clean up stale commands instead of iterating over all tags and calling blk_mq_tag_to_rq() directly?

Thanks,

Bart.

#5 lore raw
Quoted 42 lines · Bart Van Assche
Quoted 37 lines · Hannes Reinecke
Quoted 20 lines · Hannes Reinecke

blk_mq_tag_to_rq() is used from within the driver to map a tag to a request. As such it should only return requests which are already started (ie passed to the driver); otherwise the driver might trip over requests which it has never seen and random crashes will occur.

---
   block/blk-mq.c | 6 +++++-
   1 file changed, 5 insertions(+), 1 deletion(-)
block/blk-mq.c +1 -0
index 4f57d27bfa73..f02d18113f9e 100644
@@ -815,9 +815,13 @@ EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list);
     struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags,
 unsigned int tag)
   {
+    struct request *rq;

block/blk-mq.c EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list); · line 827

Quoted 10 lines
block/blk-mq.c +4 -1
+
       if (tag < tags->nr_tags) {
           prefetch(tags->rqs[tag]);
-        return tags->rqs[tag];
+        rq = tags->rqs[tag];
+        if (blk_mq_request_started(rq))
+            return rq;
       }
         return NULL;
 

This becomes particularly obnoxious for SCSI drivers using scsi_host_find_tag() for cleaning up stale commands (ie drivers like qla4xxx, fnic, and snic).
All other drivers use it from the completion routine, so one can expect a valid (and started) tag here. So for those it shouldn't matter.

Quoted 3 lines

But still, if there are objections I could look at fixing it within the SCSI stack; although that would most likely mean I'll have to implement the above patch as an additional function.

Quoted 9 lines

Hi Hannes,

Will the above patch make the fast path of every block driver slightly slower?

Shouldn't SCSI drivers (and other block drivers) use blk_mq_tagset_busy_iter() to clean up stale commands instead of iterating over all tags and calling blk_mq_tag_to_rq() directly?

You can only iterate over all tags with blk_mq_tag_to_rq() if requests are identified with tags, ie if there is a 1:1 mapping between tags and internal commands. Quite some drivers have their internal housekeeping (like hpsa), or do not track all commands via tags (eg fnic or csiostor).
For those the block layer iterator will not work as designed.

I'm currently preparing a patchset to clean that up (cf my patchset 'reserved tags for SCSI'), but that will probably take some time until it'll be accepted.

And even then some drivers have to rely on scsi_host_find_tag() in eg TMF completions to figure out if the command for which the TMF was sent is still active.

But we can move the check into the drivers if you are worried about performance impacts; it's a bit lame if you ask me, but if that's the way it should be handled, fine by me.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer