Skip to contentSkip to conversation

scsi (uas) disks no longer using tagged command queuing

REGRESSION 3.17 In review Hans de Goede 4 participants 15 messages Sep 9, 2014 – Sep 15, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi All,

While working on making error handling in the uas driver more robust, I noticed that all the commands being send to a sata ssd hooked up over uas were untagged, where I would expect tcq to be used, as that is the big advantage of uas over usb-storage / bot.

Taking the uas.c file from 3.17, and building it for 3.16 restores the use of tcq (debugged by adding a printk blk_rq_tagged + request->tag).

So either uas is doing something wrong which happened to work in 3.16, or something has broken in 3.17.

I've already added debug printk-s of scsi_device->tagged_supported, queue_depth, ordered_tags and simple_tags and those all look good (1, 29, 1, 1).

I've also tried setting disable_blk_mq and that does not help.

Any hints to help debugging this further (other then a bisect) are appreciated. If no-one has any smart ideas I guess I'll end up doing a full bisect.

Regards,

Hans

#1 lore raw
Quoted 10 lines · Hans de Goede

Taking the uas.c file from 3.17, and building it for 3.16 restores the use of tcq (debugged by adding a printk blk_rq_tagged + request->tag).

So either uas is doing something wrong which happened to work in 3.16, or something has broken in 3.17.

I've already added debug printk-s of scsi_device->tagged_supported, queue_depth, ordered_tags and simple_tags and those all look good (1, 29, 1, 1).

Quoted 5 lines

I've also tried setting disable_blk_mq and that does not help.

Any hints to help debugging this further (other then a bisect) are appreciated. If no-one has any smart ideas I guess I'll end up doing a full bisect.

scsi-mq isn't enabled by default, so setting disable_blk_mq should indeed not make a difference.

One interesting thing with uas is that it uses scsi_init_shared_tag_map, which only few drivers do.

Can you apply a debug patch like the one below and see if that helps to poinpoint down the issue?

block/blk-tag.c +3 -1
index a185b86..584db25 100644
@@ -175,8 +175,10 @@ int blk_queue_init_tags(struct request_queue *q, int depth,
 			return rc;
 		queue_flag_set(QUEUE_FLAG_QUEUED, q);
 		return 0;
-	} else
+	} else {
+		printk("%s: grabbing reference to shared tag structure!\n", __func__);
 		atomic_inc(&tags->refcnt);
+	}
 
 	/*
 	 * assign it, all done
include/scsi/scsi_tcq.h +6 -1
index cdcc90b..7f3b5cb 100644
@@ -164,8 +164,13 @@ static inline int scsi_init_shared_tag_map(struct Scsi_Host *shost, int depth)
 	 */
 	if (!shost->bqt) {
 		shost->bqt = blk_init_tags(depth);
-		if (!shost->bqt)
+		if (!shost->bqt) {
+			printk("failed to init host-wide tag map!\n");
 			return -ENOMEM;
+		}
+		printk("initialized host-wide tag map!\n");
+	} else {
+		printk("host-wide tag map already set!\n");
 	}
 
 	return 0;
#2 lore raw

Hi,

Quoted 17 lines · Christoph Hellwig
Quoted 10 lines · Hans de Goede

Taking the uas.c file from 3.17, and building it for 3.16 restores the use of tcq (debugged by adding a printk blk_rq_tagged + request->tag).

So either uas is doing something wrong which happened to work in 3.16, or something has broken in 3.17.

I've already added debug printk-s of scsi_device->tagged_supported, queue_depth, ordered_tags and simple_tags and those all look good (1, 29, 1, 1).

Quoted 5 lines

I've also tried setting disable_blk_mq and that does not help.

Any hints to help debugging this further (other then a bisect) are appreciated. If no-one has any smart ideas I guess I'll end up doing a full bisect.

Quoted 8 lines

scsi-mq isn't enabled by default, so setting disable_blk_mq should indeed not make a difference.

One interesting thing with uas is that it uses scsi_init_shared_tag_map, which only few drivers do.

Can you apply a debug patch like the one below and see if that helps to poinpoint down the issue?

I've applied the patch, this results in the following new dmesg output when using uas:

[  120.602632] initialized host-wide tag map!

Thank you for looking into this.

Regards,

Hans

#3 lore raw
Quoted 6 lines · Hans de Goede

I've applied the patch, this results in the following new dmesg output when using uas:

[  120.602632] initialized host-wide tag map!

Thank you for looking into this.

So we're initializing the tag map, but scsi_activate_tcq doesn't pick it up. I can't really come up with a good explanation for it, but there even without that there is an elephant in the room: as part of the scsi-mq series I moved the bqt field used for this into a union with the new blk_mq_tag_set. Below is a patch to get rid of that union, can you try if that fixes it?

include/scsi/scsi_host.h +2 -4
index d0f69a3..bcffff2 100644
@@ -584,10 +584,8 @@ struct Scsi_Host {
 	 * Area to keep a shared tag map (if needed, will be
 	 * NULL if not).
 	 */
-	union {
-		struct blk_queue_tag	*bqt;
-		struct blk_mq_tag_set	tag_set;
-	};
+	struct blk_queue_tag	*bqt;
+	struct blk_mq_tag_set	tag_set;
 
 	atomic_t host_busy;		   /* commands actually active on low-level */
 	atomic_t host_blocked;

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw

Hi,

Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Hans de Goede

I've applied the patch, this results in the following new dmesg output when using uas:

[  120.602632] initialized host-wide tag map!
Quoted 8 lines
Quoted 1 line

Thank you for looking into this.

So we're initializing the tag map, but scsi_activate_tcq doesn't pick it up. I can't really come up with a good explanation for it, but there even without that there is an elephant in the room: as part of the scsi-mq series I moved the bqt field used for this into a union with the new blk_mq_tag_set. Below is a patch to get rid of that union, can you try if that fixes it?

Unfortunately that does not fix it :|

Regards,

Hans

Quoted 10 lines
include/scsi/scsi_host.h +0 -0
index d0f69a3..bcffff2 100644
@@ -584,10 +584,8 @@ struct Scsi_Host {
 	 * Area to keep a shared tag map (if needed, will be
 	 * NULL if not).

include/scsi/scsi_host.h struct Scsi_Host { · line 591

Quoted 10 lines
include/scsi/scsi_host.h +2 -4
 	 */
-	union {
-		struct blk_queue_tag	*bqt;
-		struct blk_mq_tag_set	tag_set;
-	};
+	struct blk_queue_tag	*bqt;
+	struct blk_mq_tag_set	tag_set;
 
 	atomic_t host_busy;		   /* commands actually active on low-level */
 	atomic_t host_blocked;

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#5 lore raw
Quoted 8 lines · Hans de Goede
Quoted 6 lines

So we're initializing the tag map, but scsi_activate_tcq doesn't pick it up. I can't really come up with a good explanation for it, but there even without that there is an elephant in the room: as part of the scsi-mq series I moved the bqt field used for this into a union with the new blk_mq_tag_set. Below is a patch to get rid of that union, can you try if that fixes it?

Unfortunately that does not fix it :|

Alright, I guess we need the big bisect hammer unfortunately.

We should be able to start with trying the first and last commit of the big SCSI merge:

start: fcc95a763444017288b318d48367098850c23c0d
end: c21a2c1a4973c8dde32557970fdb44eaa9489aeb

#6 lore raw

Hi,

Quoted 10 lines · Christoph Hellwig
Quoted 8 lines · Hans de Goede
Quoted 6 lines

So we're initializing the tag map, but scsi_activate_tcq doesn't pick it up. I can't really come up with a good explanation for it, but there even without that there is an elephant in the room: as part of the scsi-mq series I moved the bqt field used for this into a union with the new blk_mq_tag_set. Below is a patch to get rid of that union, can you try if that fixes it?

Unfortunately that does not fix it :|

Quoted 7 lines

Alright, I guess we need the big bisect hammer unfortunately.

We should be able to start with trying the first and last commit of the big SCSI merge:

start: fcc95a763444017288b318d48367098850c23c0d
end: c21a2c1a4973c8dde32557970fdb44eaa9489aeb

The bisect points to:

[hans@shalem linux]$ git bisect bad
Bisecting: 0 revisions left to test after this (roughly 0 steps) [d285203cf647d7c97db3a1c33794315c9008593f] scsi: add support for a blk-mq based I/O path.

I've double checked that that commit is bad, and one commit earlier in the history is good, so that really is the one.

I've tried again to fix the union thing you spotted, which is introduced by that commit, and again it does not help.

I've taken a quick look at the commit, and my guess with be this has something to do with the changes surrounding __scsi_alloc_queue .

Regards,

Hans

#7 lore raw

PATCH 1/1 scsi: fix regression that accidentally disabled block-based tcq

Please try the fix below, looks like the commit broke TCQ for all drivers using block-level tagging.

---

Quoted 1 line

From 865a19b760d2786fe37d3b5c151a4ecea4c0e95e Mon Sep 17 00:00:00 2001

From: Christoph Hellwig <[email protected]>
Date: Fri, 12 Sep 2014 16:00:19 -0700
Subject: scsi: fix regression that accidentally disabled block-based tcq

The scsi blk-mq support accidentally flipped a conditional, which lead to never enabling block based tcq when using the legacy request path.

Fixes d285203cf647d7c9 scsi: add support for a blk-mq based I/O path.
---
 include/scsi/scsi_tcq.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/scsi/scsi_tcq.h +1 -1
index cdcc90b..e645835 100644
@@ -68,7 +68,7 @@ static inline void scsi_activate_tcq(struct scsi_device *sdev, int depth)
 		return;
 
 	if (!shost_use_blk_mq(sdev->host) &&
-	    blk_queue_tagged(sdev->request_queue))
+	    !blk_queue_tagged(sdev->request_queue))
 		blk_queue_init_tags(sdev->request_queue, depth,
 				    sdev->host->bqt);
 
Signature · 1 line
1.9.1
#8 lore raw

Re: [PATCH] scsi: fix regression that accidentally disabled block-based tcq

Hi,

Quoted 2 lines · Christoph Hellwig

Please try the fix below, looks like the commit broke TCQ for all drivers using block-level tagging.

Yes this one does the trick and fixes things. Note the git tree I used for testing also had your previous fix to split up the blk_tcq union in 2 separate struct members. Let me know if you want me to re-test without that fix.

Thanks & Regards,

Hans

Quoted 31 lines

---
From 865a19b760d2786fe37d3b5c151a4ecea4c0e95e Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <[email protected]>
Date: Fri, 12 Sep 2014 16:00:19 -0700
Subject: scsi: fix regression that accidentally disabled block-based tcq

The scsi blk-mq support accidentally flipped a conditional, which lead to never enabling block based tcq when using the legacy request path.

Fixes d285203cf647d7c9 scsi: add support for a blk-mq based I/O path.
---
 include/scsi/scsi_tcq.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/scsi/scsi_tcq.h +1 -1
index cdcc90b..e645835 100644
@@ -68,7 +68,7 @@ static inline void scsi_activate_tcq(struct scsi_device *sdev, int depth)
 		return;
 
 	if (!shost_use_blk_mq(sdev->host) &&
-	    blk_queue_tagged(sdev->request_queue))
+	    !blk_queue_tagged(sdev->request_queue))
 		blk_queue_init_tags(sdev->request_queue, depth,
 				    sdev->host->bqt);
 
#9 lore raw

Re: [PATCH] scsi: fix regression that accidentally disabled block-based tcq

Quoted 4 lines · Hans de Goede

Yes this one does the trick and fixes things. Note the git tree I used for testing also had your previous fix to split up the blk_tcq union in 2 separate struct members. Let me know if you want me to re-test without that fix.

I'm pretty sure that isn't needed, so double testing it indeed doesn't would be helpful. I'd like to add your Tested-by: tag after that, too.

author Tested-by
#10 lore raw

Re: [PATCH] scsi: fix regression that accidentally disabled block-based tcq

Hi,

Quoted 8 lines · Christoph Hellwig
Quoted 4 lines · Hans de Goede

Yes this one does the trick and fixes things. Note the git tree I used for testing also had your previous fix to split up the blk_tcq union in 2 separate struct members. Let me know if you want me to re-test without that fix.

I'm pretty sure that isn't needed, so double testing it indeed doesn't would be helpful.

I can confirm that things still work fine with the union left in place.

Quoted 1 line

I'd like to add your Tested-by: tag after that, too.

Ack:

Regards,

Hans

#11 lore raw

review-ping: Re: [PATCH] scsi: fix regression that accidentally disabled block-based tcq

Can I get some reviews for this patch so that we can get into 3.17-rc?

Reviewed-by on v1 1/1
#12 lore raw

Re: [PATCH] scsi: fix regression that accidentally disabled block-based tcq

In reply to Christoph Hellwig
Quoted 24 lines · Christoph Hellwig

Please try the fix below, looks like the commit broke TCQ for all drivers using block-level tagging.

---
From 865a19b760d2786fe37d3b5c151a4ecea4c0e95e Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <[email protected]>
Date: Fri, 12 Sep 2014 16:00:19 -0700
Subject: scsi: fix regression that accidentally disabled block-based tcq

The scsi blk-mq support accidentally flipped a conditional, which lead to never enabling block based tcq when using the legacy request path.

Fixes d285203cf647d7c9 scsi: add support for a blk-mq based I/O path.
---
  include/scsi/scsi_tcq.h | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
include/scsi/scsi_tcq.h +0 -0
index cdcc90b..e645835 100644

include/scsi/scsi_tcq.h static inline void scsi_activate_tcq(struct scsi_device *sdev, int depth) · line 74

Quoted 8 lines
include/scsi/scsi_tcq.h +1 -1
@@ -68,7 +68,7 @@ static inline void scsi_activate_tcq(struct scsi_device *sdev, int depth)
  		return;
 
  	if (!shost_use_blk_mq(sdev->host) &&
-	    blk_queue_tagged(sdev->request_queue))
+	    !blk_queue_tagged(sdev->request_queue))
  		blk_queue_init_tags(sdev->request_queue, depth,
  				    sdev->host->bqt);
 
author Reviewed-by on v1 1/1
#13 lore raw

Re: [PATCH] scsi: fix regression that accidentally disabled block-based tcq

In reply to Christoph Hellwig

Hi,

Quoted 24 lines · Christoph Hellwig

Please try the fix below, looks like the commit broke TCQ for all drivers using block-level tagging.

---
From 865a19b760d2786fe37d3b5c151a4ecea4c0e95e Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <[email protected]>
Date: Fri, 12 Sep 2014 16:00:19 -0700
Subject: scsi: fix regression that accidentally disabled block-based tcq

The scsi blk-mq support accidentally flipped a conditional, which lead to never enabling block based tcq when using the legacy request path.

Fixes d285203cf647d7c9 scsi: add support for a blk-mq based I/O path.
---
  include/scsi/scsi_tcq.h | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
include/scsi/scsi_tcq.h +0 -0
index cdcc90b..e645835 100644

include/scsi/scsi_tcq.h static inline void scsi_activate_tcq(struct scsi_device *sdev, int depth) · line 74

Quoted 8 lines
include/scsi/scsi_tcq.h +1 -1
@@ -68,7 +68,7 @@ static inline void scsi_activate_tcq(struct scsi_device *sdev, int depth)
  		return;
 
  	if (!shost_use_blk_mq(sdev->host) &&
-	    blk_queue_tagged(sdev->request_queue))
+	    !blk_queue_tagged(sdev->request_queue))
  		blk_queue_init_tags(sdev->request_queue, depth,
  				    sdev->host->bqt);
 

Regards,

Hans

Reviewed-by on v1 1/1
#14 lore raw

Re: [PATCH] scsi: fix regression that accidentally disabled block-based tcq

In reply to Christoph Hellwig
Quoted 24 lines · Christoph Hellwig

Please try the fix below, looks like the commit broke TCQ for all drivers using block-level tagging.

---
From 865a19b760d2786fe37d3b5c151a4ecea4c0e95e Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <[email protected]>
Date: Fri, 12 Sep 2014 16:00:19 -0700
Subject: scsi: fix regression that accidentally disabled block-based tcq

The scsi blk-mq support accidentally flipped a conditional, which lead to never enabling block based tcq when using the legacy request path.

Fixes d285203cf647d7c9 scsi: add support for a blk-mq based I/O path.
---
 include/scsi/scsi_tcq.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/scsi/scsi_tcq.h +0 -0
index cdcc90b..e645835 100644

include/scsi/scsi_tcq.h static inline void scsi_activate_tcq(struct scsi_device *sdev, int depth) · line 73

Quoted 7 lines
include/scsi/scsi_tcq.h +1 -1
@@ -68,7 +68,7 @@ static inline void scsi_activate_tcq(struct scsi_device *sdev, int depth)
 		return;
 
 	if (!shost_use_blk_mq(sdev->host) &&
-	    blk_queue_tagged(sdev->request_queue))
+	    !blk_queue_tagged(sdev->request_queue))
 		blk_queue_init_tags(sdev->request_queue, depth,
 				    sdev->host->bqt);

Introduced by d285203:

- if (!blk_queue_tagged(sdev->request_queue))
+ if (!shost_use_blk_mq(sdev->host) &&
+ blk_queue_tagged(sdev->request_queue))

Seems like an obvious fix.